Recent literature has documented the economic and political consequences of colonialism, but we have little systematic evidence about the origins of colonial investments despite their long-term impact. Combining multiple data sources, I present evidence that they were very unequally distributed across the districts of 16 British and French African colonies. How did colonial states allocate their investments? Some focus on ethnic disparities in colonial treatment, following a divide and rule logic, while others emphasize the role of natural resources. I argue that observable geographic features led some places to become centers of pre-colonial coastal trade, which increased later colonial investments not only in infrastructure but also in health and education. Although the context was highly extractive, pre-colonial commerce also helps explain the limited diffusion of investments within each colony. A de facto decentralized colonial state and European settlement are two of the mechanisms behind the persistence of inequality over time.