In 1804, Ohio University opened its doors, becoming the first university in the Northwest Territory. In 1906, the university established the first all-male glee club in the state of Ohio with the mission “to share profound and far-reaching joy through music.” However, the Ohio University Glee Club was disbanded during the events of WWII. Several years later, in 1989, Ohio University called upon an influential vocal professor to revive the organization. Ira T. Zook created the foundation for the Singing Men of Ohio and his legacy lives on today.

After Ira T. Zook's death in 2000, R.D. Mathey came from Bowling Green State University to direct the group for one year. He brought two songs “Please Mr. Columbus" and "Johnny Schmoker" that have become staples of the group. He also served as the guest conductor for the alumni concert in 2014. Upon Mathey's departure, former SMO President, Dr. Raymond Feener took over as director of the group. He directed the ensemble until 2005.  Dr. Stephen Caracciolo took over for Dr. Feener and directed SMO until 2008. He brought staple arrangements "Unitas per Harmonium" and the "Star Spangled Banner” to the group. After working at Ohio University for over 20 years, Dr. Peter Jarjisian took the helm of the group from 2008-2012. Dr. Jarjisian is credited for taking the group to a high level of musicality. In 2012, Dr. Daniel Hall moved to Ohio from Texas with his family to take over as Director of Choral Studies at Ohio University. After five years of serving the group since , Dr. Hall has moved to Pennsylvania to continue his conducting career. He was excellent at embodying the brotherhood in this group. In 2017, Dr. Bradley M. Naylor left the state of Texas to join the Singing Men of Ohio as the current director and conductor.

Today, the Singing Men of Ohio is made up around sixty members. They perform at choral concerts on campus throughout the year and, each spring, they tour around the country.

During the 2018-2019 academic year, the Singing Men of Ohio will celebrate their 30th anniversary by going on a weeklong tour to Canada and with their annual Alumni Weekend celebration soon after.