But if one child gets more attention than a subsequent one, don't you think that has the possibility to affect a lot of things? Perhaps not height, but IQ etc.
Not in normal families. I think it takes neglect. I strongly doubt my brother has 10 IQ points on me because he had more attention (which he didn't have).
I don't think it takes neglect, but a lack of attention/push in that direction. With that being said, of course the genes are going to matter as well. The social factors lie on top of that.