The champions wasted no time going ahead thanks to an early strike from Sami Khedira.
Star signing Cristiano Ronaldo sent a close range effort wide after being played through, denying him the prospect for a debut goal.
Ronaldo then pounced on a loft pass from Juan Cuadrado and took a first time shot from an acute angle which went narrowly wide.
Costa unleashed a stunning strike with the outside boot but it went over the top. Chievo drew level as halftime rained in with a low strike from Mariusz Stepinski.
A thunderous shot from Ronaldo forced Roberto Sorrentino to make a very smart block on 48 minutes. before Chievo were then handed a spot kick following a foul on Emanuele Giaccherini by Cancelo.
The former stood up and dusted himself off to convert the spot kick with ease, much to the delight of the home fans.
A change from Massimiliano Allegri brought in Federico Benardeschi for Cuadrado. The former Fiorentina man nearly made an instant impact, but Sorrentino met his shot well.
What followed was a succession of chances for Ronaldo. On 61 minutes, Sorrentino spared his blushes with a tip over the top after meeting a cross with the head.
Another piece of magic from Ronaldo happened, as he took a curled shot towards the far corner but Sorrentino prevented him yet again following 66 minutes.
Despite their resistance, Chievo finally crumbled as the returning Leonardo Bonucci pounced to grab the equaliser.
Substitute Mario Mandzukic thought he had secured the winner a minute to the end, but after lengthy consultation with VAR, the goal was disallowed.
It didn’t taint their belief and it only inspired them to eventually go on and grab the winner, as Bernadeschi pounced at the end of added time for an opening day win.