A normalized Hadamard matrix is said to be completely noncyclic if no two row vectors are shift equivalent in its punctured matrix (i.e., with the first column removed). In this paper we present an infinite recursive construction for completely noncyclic quaternary Hadamard matrices. These Hadamard matrices are useful in constructing low correlation zone sequences.