About 1868, citizens of Colorado City showed an interest in building a library and gained sufficient funds to buy a small collection of books. On January 7, 1896, the Women's Christian Temperance Union of Colorado City formally opened and dedicated a library they called the Woods Free Library, named after Mrs. Lydia Woods of Manitou Springs who had donated $1,000 to purchase books. In 1901, the library was moved to the Templeton Building at 8 North 25th Street. In 1904, a new library on West Pikes Peak Avenue was opened, thanks to a gift from Andrew Carnegie.

When Colorado City was annexed to the city of Colorado Springs in 1917, the library board of directors sold its invested rights and interests to the Colorado Springs Public Library for $1, and it became the West End Branch. In May 1977, it was rededicated as the Old Colorado City Branch Library to reflect the resurgence of historical interest in the area. The 1904 Carnegie building - now part of the Old Colorado City Historic District - was renovated in 1980, and received additional parking and handicapped access in 1994. A renovation campaign was launched in early 2003 to preserve its historic architecture.