Gavin Hunt handed a debut to Cameroonian Eva Nga, as Celtic gave a start to Menzi Masuku who replaced the injured Tebogo Potsane.
Celtic looked for a third successive victory, but Phathushedzo Nange handed Wits the lead on 12 minutes with a top corner finish that cannoned off the underside of the bar before nestling the net.
Ndumiso Mabena failed to pull his side level two minutes later when he fired over the top from an inviting opportunity after Ricardo Goss scuffed a clearance.
The Clever Boys pushed for more goals in the latter stages of the half, but were unsuccessful in their quest as the visitors steadied ship.
Siwelele were level five minutes after halftime when Masuku ran to the back post to tap home Siphelele Luthuli's low cross despite the best efforts of Ricardo Goss.
Hunt looked to the bench following the 64th minute and brought in Haashim Domingo, who replaced Terrence Dzvukamanja.
Celtic went ahead after Andile Fikizolo tapped home Masuku's cross from the right side in the 77th minute.
The lead was cut short five minutes later as Domingo's low shot trickled into the net after a deflection wrong-footed Jackson Mabokgwane.
Wits seized control in the last eight minutes when Gift Motupa back-heeled a rebound into an unguarded net after seeing his initial effort parried by Mabokgwane, thus ensuring victory.
Wits: Goss, Hlanti, Nonyane, Hlatshwayo, Macheke, Hotto, Alexander, Nange, Pelembe, Dzvukamanja, Nga.
Subs: Petersen, Douhadji, Monare, Domingo, Doutie, Motupa, Human.
Celtic: Mabokgwane, Mashikinya, Ngobeni, Rihkotso, Mabena, Khasipe, Masuku, Luthuli, Letsaolo, Mahashe, Phalane.
Subs: Chaine, Fikizolo, Sera, Tchilimbou, Maema, Baloyi, Letlabika.