Piper, the son of solicitor, Charles Piper, was born in Epsom, Surrey, in 1903. He was educated at Epsom College and trained at the Richmond School of Art, followed by the Royal College of Art in London. He turned from abstraction early in his career, concentrating on a more naturalistic but distinctive approach.
Piper was appointed an official war artist in World War II from 1940-1942. The morning after the air raid that destroyed Coventry Cathedral, Piper produced his first painting of bomb damage, Interior of Coventry Cathedral now exhibited at the Herbert Art Gallery. Jeffery Daniels in The Times described the painting of the ruins as "all the more poignant for the exclusion of a human element". It has been described as "Britain's Guernica".
Piper collaborated with many others, including the poet John Betjeman (on the Shell Guides, and with potter Geoffrey Eastop and artist Ben Nicholson. In later years he produced many limited-edition prints.
From 1950 Piper worked in stained glass in partnership with Patrick Reyntiens, whom he had met through John Betjeman. They designed the stained glass windows for the new Coventry Cathedral, and later for the Chapel of Robinson College, Cambridge. Washington National Cathedral prominently features his large window, "The Land Is Bright". He designed windows for many smaller churches and created tapestries for Chichester Cathedral and Hereford Cathedral. He was a set designer for the theatre, including the Kenton Theatre in Henley and Llandaff Cathedral in Cardiff. He designed many of the premiere productions of Benjamin Britten's operas at Glyndebourne Festival Opera, the Royal Opera House, La Fenice and the Aldeburgh Festival, as well as for some of the operas of Alun Hoddinott. In 2012 a major exhibition 'John Piper and the Church' examined his relationship with the Church and his contribution to the development of modern art within churches. Piper wrote extensively on modern art in books and articles. With his wife, Myfanwy Piper, he founded the contemporary art journal, Axis.
On 28 June 1992 John Piper died at his home at Fawley Bottom, Buckinghamshire where he had lived for most of his life. His children are painters Edward Piper and Sebastian Piper, and his grandchildren include painter Luke Piper and sculptor Henry Piper.
His auction record, £325,250, was set at Sotheby's on 15 July 2008 for "Forms on Dark Blue", a 3' by 4' oil painted in 1936.