Despite their personal sturdiness and the immense range of their wanderings (they settled Europe and Asia from the Iberian Peninsula to the Altai Mountains of Siberia), the latest genetic studies indicate their total population never managed to reach six figures.
WSJ: Neanderthals: Why Us and Not Them?