Evidence suggests that lifestyle factors, e.g. physical activity, moderate the manifestation of genetic susceptibility to obesity. The present study uses UK Biobank data to investigate interaction between polygenic scores (PGS) for two obesity indicators, and lifestyle and psychosocial factors in the prediction of the two indicators, with attention to sex-specific effects. Analyses were of 112 151 participants (58 914 females; 40 to 73 years) whose genetic data passed quality control. Moderation effects were analysed in linear regression models predicting body mass index (BMI) and waist-to-hip ratio (WHR), including interaction terms for PGS and each exposure. Greater physical activity, more education, higher income, moderate vs low alcohol consumption, and low material deprivation were each associated with a relatively lower risk for manifestation of genetic susceptibility to obesity (p < 0.001); the moderating effects of physical activity and alcohol consumption were greater in women than men (three-way interaction: p = 0.009 and p = 0.008, respectively). More income and less neuroticism were related to reduced manifestation of genetic susceptibility to high WHR (p = 0.007; p = 0.003); the effect of income was greater in women (three-way interaction: p = 0.001). Lifestyle and psychosocial factors appear to offset genetic risk for adiposity in mid to late adulthood, with some sex-specific associations.