Born in Brooklyn, New York, Schermerhorn studied science at the city’s Polytechnic Institute and went on to study engineering at Rensselaer Polytechnic Institute. His first employment followed with the Department of City Works in Brooklyn, however, he pursued landscape architecture and, in 1901, Andrew Carnegie commissioned him to design a terrace and garden for his Fifth Avenue residence, now the Cooper-Hewitt, National Design Museum.
Schermerhorn displayed a particular interest in Colonial Revival design, beginning in 1908 with a submission to House and Garden for a collection of Long Island country estates, and ultimately becoming Chairman of the Landscape Architects Committee for the restoration of Colonial Williamsburg.
In 1915, Schermerhorn won the competition for Telewana, now Jacob Riis Park, in Queens. His practice grew with planning and park design projects in New York, New Jersey, Florida, and South Carolina.
Schermerhorn worked for such notable designers as Guy Lowell, Charles Wellford Leavitt, Jr., and Nathan Franklin Barrett. He lectured at Columbia University from 1935 to 1939 and, throughout his career, published articles on landscape architecture, architecture, and gardening.