He began his career with John Hull and the Joymakers and performed with groups such as the Prophets, Dixie Echoes, Searchers, Masters V, and the Stamps Quartet. However, his greatest impact in gospel music came from his career with the Statesmen Quartet as he brought class and dignity to the stage, especially as he sang "Beyond the Gates". Jack is also known as a prolific Gospel songwriter having penned over 600 songs like "Some Dawning", "Jesus is Coming for Me", and "I Will Rise Up From My Grave." 

The last recording Jack made was entitled "The Grand Ole Gospel Reunion Quartet Sings Mosie Lister Favorites," and features Jack’s melodious voice. Jack Toney’s awesome singing voice and his love for God and people remain his legacy.