He was born in London and educated at the Mercers' School and St John's College, Oxford, where he acted as an assistant at the Bodleian Library. He was first appointed to a position at the British Museum in 1800, and was chief librarian from 1827 to 1856, although Anthony Panizzi was effectively the administrative head from 1837.
Ellis was knighted in 1833. He edited various works on antiques and wrote an Introduction to Domesday Book.