There are socks for special purposes: athletic socks, crew socks etc.

If you saw somebody sitting with white sport socks and a suit, you'd kind of think maybe they couldn't find a clean pair of socks that day or something.