Gumbo, who also serves as an assistant coach with his country's national team, was last week suspended by the Emalahleni-based side - not long after Vusi Mkhatshwa returned to the club.
After finishing a respectable fifth in the second tier last term under Mkhatshwa, Spurs currently find themselves in 14th spot with four points from six games.
Spurs CEO Freddy Mapulane confirmed to KickOff.com that Mkhatshwa will continue to guide the team on a caretaker basis.
"It's done, we have officially parted ways with Mr Rahman Gumbo. He [Mkhatshwa] will be acting yes."
Spurs will hope to snap a five-game winless streak when they host fourth-placed Stellenbosch FC on Saturday.