Born on Priestley Plantation near New Orleans, Richardson initiated the Romanesque Revival in the United States. He studied for a year at the University of Louisiana, entering Harvard University in 1856 and graduating three years later. Deciding to become an architect, Richardson studied at the École des Beaux-Arts in Paris from 1860 to 1862, when he lost his financial support due to the Civil War. He then worked in the office of French architect Theodore Labrouste until he returned to the United States in 1865, settling in New York. The following year, Richardson was awarded his first commission, the Church of Unity in Springfield, Massachusetts (demolished). After marrying in 1867, he moved to Staten Island, living adjacent to Frederick Law Olmsted, Sr., with whom he would frequently collaborate. That year Richardson formed an eleven-year partnership, Gambrill & Richardson, with architect Charles Gambrill. Richardson’s New York projects include portions of the interior of the New York State Capitol (1870s) in Albany, and the State Asylum for the Insane (Richardson Olmsted Complex) in Buffalo (1870-1872), the first collaboration involving Frederick Law Olmsted, Sr., Calvert Vaux, and Richardson. Other collaborations with Olmsted include the South Street entrance of Arnold Arboretum (1870s), and the Boylston Street Bridge (1880-1884) and Stony Brook Gate House in the Back Bay Fens in Boston. In 1874 Richardson relocated to Brookline, Massachusetts, designing residences, educational buildings such as Sever (1878-1880) and Austin (1880-1884) halls at Harvard University, railroad stations, libraries, and commercial and civic structures, including Trinity Church in Boston (1872-1877).
Richardson died in Brookline at the age of 47 while at the top of his profession after having suffered from chronic nephritis throughout his career. He was buried in Walnut Hills Cemetery in Brookline. The Boston architectural firm of Shepley, Rutan and Coolidge completed his unfinished projects.