This park was named for Albert Kelly, an early settler in Oregon. He was born in 1814, in Pulaski County, Kentucky. In 1848, Albert and three of his brothers set out for Oregon. He and brother Clinton got as far as Independence, Missouri, when Albert's livestock were scattered by a storm and lost. Clinton kept on going, but Albert turned back and delayed coming west until the following year.
Albert first scouted the area now known as Holladay Park, but was discouraged by the lack of a reliable supply of water. Later, he and Finice Caruthers explored the area west of the hills, and found it much more to his liking. Albert made his land claim and in 1850, he and his sons put up a cabin of logs and shakes, and moved into it just in time for the birth of his daughter Martha (later, Mrs. O.P.S. Plummer). Albert and his family farmed the land, and Albert served his church as a circuit-riding minister until his death in 1873.
The park came into being in 1956 when Mrs. Hildegarde Plummer Withers offered 9 acres of the family land to then Parks Superintendent Buckley - if it would be dedicated as a park and named after her grandfather, Albert Kelly. The offer was accepted, and for $25,000, the city of Portland took title to the land. Through a series of land swaps and changes in 1959, 1963 and 1965, the park was somewhat reshaped and enlarged to its present size of 12.8 acres.