Born in Newton, Massachusetts, Mangini first started playing the drums when he was two and a half years old, being inspired by Ringo Starr of The Beatles. He would practice two to four hours a day and by the time he was nine he was already mimicking Buddy Rich performances. Other influences include Bobby Colomby, Danny Seraphine, John Bonham, Neil Peart and Terry Bozzio.

By high school, he was performing in bands and participating in the prestigious All-State, and All-Eastern United States ensembles.

After graduating from Waltham Senior High School in 1981, Mangini put aside his music studies to pursue a computer science major at Bentley University. After graduating Mike started programming software for the Patriot Missile program. At the same time, he was working on a program that studied the links between the human brain and body.

In 1987, Mangini scored one of his first high-profile gigs, playing drums for Boston's Rick Berlin Band. This would be his first collaboration with bassist Philip Bynoe. He also taught drums privately in Boston during this time.

In 1991, Mangini joined the thrash metal band Annihilator, playing drums on several tracks for the band's album Set the World on Fire. He went on to tour with the band in support of this album until 1994. That year, he joined Boston hard rock band Extreme, replacing original drummer Paul Geary. Mangini played on several songs on Extreme's 1995 album Waiting for the Punchline, and appeared with the band on the Late Show with David Letterman.

After Extreme disbanded in 1996, Mike was informed through drummer Jonathan Mover that guitarist Steve Vai was auditioning drummers for his live band. Mike successfully auditioned for Vai's band and relocated to Los Angeles. From late 1996 to early 2000, Mangini was the highly regarded drummer of Vai's live ensemble, which also included bassist Philip Bynoe, guitarist Mike Keneally and beginning in 2000, guitarist Dave Weiner. Mangini appeared on Vai's studio albums Fire Garden and The Ultra Zone, and the live album Alive in an Ultra World.

During a hiatus on Vai's Ultra Zone tour in 2000, Mangini returned to Boston, where he served as an associate professor at Berklee College of Music. He also formed the short-lived band Tribe of Judah with former Extreme frontman Gary Cherone and bassist Pat Badger. Mangini appeared on other albums during this time, including a Rush tribute record entitled Subdivisions and a second Annihilator album titled All for You. He also recorded tracks for artists Sal DiFusco, Bill Lonero and Chris Emerson.

In 2005, Mangini accepted a full-time teaching position at the Berklee College of Music in Percussion Department. He resigned in 2010 after joining Dream Theater.

Mangini appeared on a third Annihilator album in 2007, titled Metal.

In late 2010, Mangini, along with Marco Minnemann, Peter Wildoer, Virgil Donati, Aquiles Priester, Derek Roddy and Thomas Lang, auditioned to succeed Mike Portnoy as the drummer for Dream Theater. He had previously appeared on three solo albums by Dream Theater's lead singer, James LaBrie. Mangini was announced as Dream Theater's new drummer on April 29, 2011, several months after he actually joined. He has appeared on every Dream Theater release since.

Mangini served as the drummer for Dream Theater bandmate John Petrucci on the 2012 G3 tour.