Observational studies have shown consistently increased likelihood of dementia or mild cognitive impairment diagnoses in people with higher air pollution exposure history, but evidence has been less consistent for associations with cognitive test performance. We estimated the association between baseline neighbourhood-level exposure to airborne pollutants (particulate matter and nitrogen oxides) and (1) cognitive test performance at baseline and (2) cognitive score change between baseline and 2.8-year follow-up, in 86,759 middle- to older-aged adults from the UK Biobank general population cohort. Unadjusted regression analyses indicated small but consistent negative associations between air pollutant exposure and baseline cognitive performance. Following adjustment for a range of key confounders, associations were inconsistent in direction and of very small magnitude. The largest of these indicated that 1 interquartile range higher air pollutant exposure was associated on average with 0.35% slower reaction time (95% CI: 0.13, 0.57), a 2.92% higher error rate on a visuospatial memory test (95% CI: 1.24, 4.62), and numeric memory scores that were 0.58 points lower (95% CI: -0.96, -0.19). Follow-up analyses of cognitive change scores did not show evidence of associations. The findings indicate that in this sample, which is five-fold larger than any previous cross-sectional study, the association between air pollution exposure and cognitive performance was weak. Ongoing follow-up of the UK Biobank cohort will allow investigation of longer-term associations into old age, including longitudinal tracking of cognitive performance and incident dementia outcomes.