CFA crews from Daylesford, Glenlyon, Tylden and Musk were called to the incident at 4.54am near Daylesford-Malmsbury Road.
The fire, which had originated from a dormant burn-off from earlier in the week, had affected around 15 hectares of land before crews were able to slow it down at around 8am.
There were 15 CFA vehicles on scene and Forest Fire Management Victoria (FFMVic) tankers on scene, as a bulldozer worked to create containment lines and is expected to be brought under control by 11am.
District 15 Commander Steve Poulter said the fire was located in an “inaccessible and dense area”, however, firefighters from CFA and FFMVic have almost contained it.
“The burning was fairly narrow but it was spotting ahead of the fire into some inaccessible areas,” Mr Poulter said.
“The winds were pretty strong, which made it spot ahead of the fire, but it hasn’t yet affected the state forest area.
“We’ve managed to almost contain it right on the edge of the state forest.”
Mr Poulter said crews are confident in bringing the fire under control as temperatures in the area start to drop.
“The winds won’t get any stronger until 5.00pm today, which won’t be an issue as it should be contained by then,” Commander Poulter said.
Commander Luke Kneebone said the property owner conducting the burn-off had "done everything right" and was caught out.
"He cleared around the burn-off, had water out there and conducted it early in the week knowing the warmer weather was coming," Mr Kneebone said.
"He just got caught out.
"This is a good lesson to property owners that they must thoroughly extinguish their burn-offs, otherwise they can reignite four days later like this fire."