The stone structure was originally built as the stables for Milthorpe, the 14-room heritage sandstone residence in Crescent Street, built in 1872 by Richard Sim. There is much evidence of the early stables building in the house, clearly 1870s, with smooth surface sandstone rather than the later quarry-faced finish, and timbers above. The addition of the upper timber and brick structure was probably carried out in the early 1900s when Milthorpe was remodelled to give a Federation style appearance.