On Monday club president Florentino Perez announced that Mourinho and the club had mutually agreed to part ways at the end of the season after being in charge for three years.
Mourinho managed to win just one Copa del Rey, one Liga title and the Spanish Supercup, while also failing in his pursuit of La Decima - what would be the club's 10th Champions League title, and Calderon believes Mourinho was never the right coach for a team like Madrid.
"Of course he has been a failure," Calderon told Sky Sports' Revista De La Liga. "It is the worst record of any Real Madrid manager who has had three years or more there.
"I insist that it is not only that reason. I think you can be unsuccessful at a club for many reasons. What I really don't like and what many followers of Real Madrid don't like is the way he has behaved.
"I don't know what he did in England but his behaviour in Spain wasn't compatible with Real Madrid's history. We have a part of our hymn which says 'when we lose, we shake hands' - we don't poke fingers in eyes, which is what he did to Tito Vilanova.
"I have always said that he wasn't the only one to blame - that's also down to the people who brought him to Real Madrid.
"Mourinho was like that before he arrived at Real Madrid," he adds.