In many developing countries, food environments are changing rapidly, with modern retailers—such as supermarkets—gaining in importance. Previous studies have suggested that the rise of modern retailers contributes to overweight and obesity. Effects of modern retailers on dietary quality have not been analyzed previously due to the unavailability of individual-level dietary data. Here, we address this research gap with data from randomly selected households in Lusaka, Zambia. Anthropometric and food-intake data from 930 adults and 499 children were analyzed to estimate effects of purchasing food in modern retailers on body weight, height, and dietary quality while controlling for income and other confounding factors. The food expenditure share spent in modern retailers was found to be positively associated with overweight in adults, but not in children. For children, a positive association between expenditures in modern retailers and height was identified. Modern retailers contribute to higher consumption of ultra-processed foods and calories. But they also increase protein and micronutrient intakes among adults and children, mainly through higher consumption of meat and dairy. The findings underline that modern retailers can influence diets and nutrition in positive and negative ways. Differentiated regulatory policies are needed to shape food environments for healthy food choices and nutrition.