The colonists in this game do a LOT of walking. Allowing them to seek warmth in ANY house seems like a good idea.
Also Taverns should be a source of food and warmth. Course that means the tavern keeper would have to make food runs but that seems reasonable.