Born and raised in Kansas City, Missouri, Ellifrit graduated from the University of Illinois in 1932 with a B.L.A. His coursework also included specialized study in city planning. He won the Ryerson Traveling Fellowship and spent much of 1933 studying cities in England, France, Italy, and Greece. He returned to the United States, taking part-time jobs until being hired by the National Park Service at Great Smoky Mountains National Park, where he helped to plan campgrounds, trail systems, and road systems. He later contributed to the restoration of the Fort Pulaski National Monument near Savannah, Georgia.
In 1935, Ellifrit began work as a landscape architect with the firm Hare & Hare in Kansas City, working primarily on projects in Houston. These included the design of Hermann Square and Hermann Park, and other smaller parks across the city. In 1939, Ellifrit arrived in Houston to serve as the firm’s representative. One year later, he became the director of the city’s newly constituted Department of City Planning, a post he would hold for 23 years. He served his country during World War II, and then worked with S. Herbert Hare (the city’s professional planning consultant) to revise a 1929 plan for Houston’s parks and major thoroughfares. Ellifrit spearheaded the westward expansion of the Brays Bayou Parkway and the creation of the White Oak Bayou Parkway, and he was chiefly responsible for the layout for the city’s freeway system. He also oversaw the addition of several neighborhood parks throughout Houston, and his Civic Center Plan (1957) eventually led to the renewal of the Civic Center District. He died in Houston at the age of 89.