Toussaint was born in Aruba, Dutch Antilles, and was raised in Saint Thomas and New York City. He learned to play calypso as a child and attended Berklee College of Music in the late 1970s, studying under Bill Pierce (saxophonist). In 1979 he formed a group with Wallace Roney and from 1982 to 1986 was a member of Art Blakey and the Jazz Messengers alongside Terence Blanchard, Donald Harrison, Mulgrew Miller and Lonnie Plaxico. With Blakey he recorded three studio albums, including New York Scene, which won a Grammy for Best Jazz Instrumental Performance.
In 1987, Toussaint moved to London, England, when he was invited to be artist-in-residence at the Guildhall School of Music by Lionel Grigson, at the time the school's professor of jazz. In the late 1980s Toussaint had a regular slot at the fabled Dingwalls club in Camden Town on Sunday afternoons. Since then, Toussaint has maintained a profile as a band leader in the UK and Europe, playing with British musicians including, among others, Steve Williamson, Courtney Pine, Julian Joseph, Jason Rebello, Cleveland Watkiss. Toussaint has also performed in groups led by McCoy Tyner, Gil Evans, Kirk Lightsey, Cedar Walton, Max Roach, Horace Silver and Jeff Tain Watts. In addition he has collaborated with Lionel Loueke.
Toussaint has released 10 albums as a leader, his most recent entitled Tate Song in February 2014 with LYTE Records.
In 2015/16 Toussaint devised a project dedicated to his mentor Blakey. The project was entitled "Roots & Herbs: The Blakey Project" and featured an all-star lineup of British jazz musicians, including Julian Joseph, Byron Wallen, Dennis Rollins, Jason Rebello and Shane Forbes. The group toured the UK extensively throughout 2015 / 2016