Overlooking the Ohio River at a location originally known as “Bald” Hill, the park derived its historic name from its Native American associations as it was thought to have been cleared of trees by Native people. Another hilltop park in the city’s unique collection, it derived its name, Frederick H. Alms Memorial Park, as a memorial to Mr. Alms by his wife, Eleanora. The parkland was purchased by the Park Board with funds from the Alms estate, ultimately resulting in nearly 94 acres of open space.

In 1929, A. D. Taylor designed the park’s entrance ensemble framing the arrival of a Picturesque road flanked with symmetrical piers and stone walls. Taylor also designed the landscape plan for the park’s centerpiece, an Italianate pavilion, designed by architects Stanley Matthews and Charles Wilkins Short, Jr., atop the former Bald Hill.

Other park features include a comfort station designed by architect Carl Freund and built in 1936 with funds from the Works Progress Administration, and a memorial statue to Stephen Foster Collins, created by Arturo Ivone in 1937 and donated by Josiah Kirby Lilly, an Indianapolis-based admirer of the composer. 
The pavilion is one of three in the city’s park system, along with Ault and Mount Echo Parks, with landscape plans designed by A.D. Taylor.