We use mental models of the world – also called cognitive maps – to guide behavior. Several mental illnesses, including schizophrenia and substance use disorder, involve disruptions in this ability. The lateral orbitofrontal cortex (lOFC) is typically thought to support behavior by deploying cognitive maps to simulate outcomes during decision making, but recent evidence suggested that it may instead support behavior by underlying the update or creation of such maps. Costa et al. tested between these two alternatives using outcome-specific devaluation and a high-potency chemogenetic approach. Selectively inactivating lOFC principal neurons when male rats learned distinct cue-outcome associations, but prior to outcome devaluation, disrupted subsequent inference, confirming a role for the lOFC in creating new maps. However, lOFC inactivation surprisingly led to generalized devaluation, a result inconsistent with a complete mapping failure. Using a novel reinforcement learning framework, the authors show that this effect is best explained by a circumscribed deficit in credit assignment precision during map construction, suggesting that lOFC plays a selective role in defining the specificity of associations that comprise cognitive maps.