Long-term field monitoring of leaf pigment content is informative for understanding plant responses to environments distinct from regulated chambers but is impractical by conventional destructive measurements. We developed PlantServation, a method incorporating robust image-acquisition hardware and deep learning-based software that extracts leaf color by detecting plant individuals automatically. As a case study, we applied PlantServation to examine environmental and genotypic effects on the pigment anthocyanin content estimated from leaf color. We processed >4 million images of small individuals of four Arabidopsis species in the field, where the plant shape, color, and background vary over months. Past radiation, coldness, and precipitation significantly affected the anthocyanin content. The synthetic allopolyploid A. kamchatica recapitulated the fluctuations of natural polyploids by integrating diploid responses. The data support a long-standing hypothesis stating that allopolyploids can inherit and combine the traits of progenitors. PlantServation facilitates the study of plant responses to complex environments termed “in natura”.