"They always have something to guess about you, don't they?"
Something does always have to be happening for them. When your neighbors are cornfields and cows, gossip/speculation are as good as gold.
"For anyone else I'd immediately start targeting the "staying inside" thing, but prior context has me knowing better here. "
I just dont leave the house unless I'm working or on the weekends. It's my preference but also reinforced by the fact that I dont have a car and live in the middle of no where.