Keller was dismissed after supervising just two wins from 10 games in all competitions this season. He had been in charge of the Bundesliga outfit since December 2012.
"Schalke have parted company with Jens Keller," read a brief statement on the club's official Twitter feed. "Roberto Di Matteo takes over."
Di Matteo, who led Chelsea to Champions League glory as a caretaker boss in 2012, has been without a full-time job since being fired by the Blues in November 2012.