This made him extremely unpopular with his fellow townsmen because the pilgrimages to the Kaaba were the chief source of such prosperity as Mecca enjoyed.