Walter Williams Sr. They formed the group in Canton, Ohio in 1958 while attending Canton McKinley High School. Originally known as The Triumphs, and then The Mascots, the friends began recording with "Miracles" in 1961, which was a moderate hit in the Cleveland area. In 1963, they took the name "The O'Jays", in tribute to Cleveland radio disc jockey Eddie O'Jay who was part of the powerful management team of Frankie Crocker, Herb Hamlett & Eddie O'Jay, (Toop, 1991). In 1963, The O'Jays released "Lonely Drifter," their first national chart hit on the Billboard Hot 100, peaking at #93. Their debut album, released shortly thereafter was Comin' Through.

Throughout the 1960s, they continued to chart with minor hits such as "Lipstick Traces" (which they performed nationally on the ABC Television program, Shivaree), "Stand In For Love," "Stand Tall," "Let It All Out," "I'll Be Sweeter Tomorrow," "Look Over Your Shoulder," "Deeper In Love With You," and "One Night Affair." However, while they issued dozens of singles throughout the decade, they never once hit the US top 40 (although "Lipstick Traces" made it to #19 in Canada). On the R&B charts, The O'Jays were somewhat more prominent, but their only top 10 R&B single prior to 1972 was 1968's "I'll Be Sweeter Tomorrow."

In spite of their success as a touring group and on the R&B charts, the group had been considering quitting the music industry in 1972; around that time original members Bill Isles and Bobby Massey departed, leaving the group a trio. The remaining three original members, Eddie Levert, William Powell, and Walter Williams, soldiered on and Gamble & Huff, a team of producers and songwriters with whom the O'Jays had been working for several years, signed them to their Philadelphia International label. Suddenly, The O'Jays' finally scored with their first million-seller, "Back Stabbers", from the album of the same name. This album produced several more hit singles, including "992 Arguments," "Sunshine," "Time To Get Down," and the #1 pop smash, "Love Train."

During the remainder of the 1970s the O'Jays continued releasing hit singles, including "Put Your Hands Together" (Pop #10), "For the Love of Money" (Pop #9), "Give the People What They Want, "Let Me Make Love To You", "I Love Music" (Pop #5), "Livin' for the Weekend," "Message in Our Music" and "Darlin' Darlin' Baby (Sweet Tender Love)". Original member William Powell died of cancer in 1977 at age 35.

After adding Sammy Strain (born December 9, 1939) (of Little Anthony and the Imperials), the O'Jays continued recording, though with limited success. 1978's "Use ta Be My Girl" was their final Top Five hit, though they continued placing songs on the R&B charts throughout the 1980s. The O'Jays success was not confined to the United States, as they also logged up nine hit singles in the United Kingdom between 1972 and 1983, including four tracks that reached the Top 20 in the UK Singles Chart. Their 1987 album, Let Me Touch You, was a breakthrough and included the #1 R&B hit "Lovin' You". Though they continued charting on the R&B charts in the late 1980s and early 1990s, the O'Jays never again achieved pop success. In 1992, Sammy Strain left the group, and returned to the Imperials, to be replaced by Nathaniel Best, and later by Eric Grant. Later in the 1990s, the group did little recording, though they remained a popular live draw. Their latest album was Imagination in 2004.

In 2003, they co-starred in the movie The Fighting Temptations, which starred Cuba Gooding Jr. and Beyoncé Knowles. In the film, they played three barbers who joined the local church choir to help out the film's protagonists’ Darrin (Gooding) and Lilly (Knowles) who were the choir director and lead singer, respectively.

In 2005, the O'Jays were inducted into the Rock and Roll Hall of Fame. Original members Eddie Levert, Walter Williams, Bobby Massey and, posthumously, William Powell, were inducted. In a note of controversy, Sammy Strain was also inducted with the group, while original member Bill Isles was not. (Strain is one of the few artists in popular music history who is a double RRHOF inductee: with The O'Jays in 2005, and Little Anthony and the Imperials in 2009). In 2006, the O'Jays performed at the ESPY awards, hosted by Lance Armstrong. "For the Love of Money" was the theme song to the hit reality TV show The Apprentice, starring Donald Trump, and the group once performed the song live on the show.