If I recall correctly, even brains of rich and poor people differ due to stress and other factors affecting our development in the childhood. Not drastically, but notable enough nonetheless.