People would accuse the Gospels of being some (secret) reverse harem, unless the disciples and apostles were also female. (It could also explain some of Jesus' more feminist-leaning behaviors, the actual often-unmentioned female disciples, and other stuff.)
I wonder if that might explain why Jesus also didn't stick with the family trade of carpentry, the missing childhood years (there being no "boyhood" history to record, actually), among some other things.