With nothing much to show in the first half, and chances few and far between, Siwelele started to gain momentum in the second stanza and went ahead through Dumisani Zwane and Vuyani Ntanga, however Marc Van Heerden kick-started Usuthu’s comeback with a bizarre goal that sailed over Patrick Tignyemb.
AmaZulu’s tails were up and smelled the blood of Celtic's defence, when Khulekani Madondo equalized from the edge of the penalty area three minutes later, and Bornwell Mwape found the winner from close range after the ball eluded the entire Celtic defence from a Marc Van Heerden free-kick.
However, the German said it was shocking to lose the game after having controlled for most of 80 minutes, until Usuthu started to crack the whip following the sending off of Wandisile Letlabika for a second bookable offense.
"It is shocking to go [out] in such a game, after dominating most of the time, and you give it away like that in the final minutes," said Middendorp.
"I'm not pleased at going out in the first round, and [with regards to Letlabika’s sending off] it’s a little bit threatening to have this in this area of consistency, and I don't know how to say… There are some situations that are quite strange, but as I said, I don't have much to blame the referee in this, it happened,” he added.
"With a yellow card and second yellow card, the player must be a little bit more smart and not tackle in that nature, but it was not a foul.
“Last week it was Gabadinho [Mhango], and when you see it again, you jump and you follow the ball and in the same moment something sliding in and it's a yellow card. But I think we need to respond [appeal]."
Bloemfontein Celtic will have a full two-week break due to the FIFA window, and will next see action against Polokwane City on 19 Ocyober at Kaizer Sebothelo Stadium; and if things continue to go pear-shaped, the Owners of the club will start to ask questions as the club lie 14th on the Absa Premiership log, with six points from seven games.