This paper examines the European colonisation of Portland Bay, Port Phillip District of New South Wales from 1828 to 1836. Its starting point is the mercantile origins of settlement which began with activities of sealers in the 1820s and whalers from 1833. In 1834 the Henty family from Launceston but originally from Sussex, England, established a permanent base there. The paper argues that these whaling establishments, hitherto unstudied in the architectural literature, with their industrial infrastructure (“grey architecture” after Bremner) were the beginning of European architecture in Victoria. Evidence is drawn from primary sources including sketches of the bay by John Helder Wedge (1835), the Henty Journals (1834–36), the 1835 watercolour of the bay by George Jackson and the record of Major Thomas Mitchell’s visit to the bay in 1836. By uncovering the Henty family’s extensive mercantile interests in Australia from their arrival on the Swan River in 1829 it reveals the relationship between whaling, trade, shipping, and settlement.