There was a good early chance for Kadi in the sixth minute after an error by Mohammed Awal, but his shot punched wide by Shu-Aib Walters.
United's Mthokozisi Yende had to leave the field in the 36th minute after he had been roughly tackled by Wits stand-in captain Aaron Mokoena.
In the 44th the Team of Choice should have taken the lead when Delron Buckley's defence-splitting pass found Orlando Smeekes on the left, but the bulky attacker blasted his shot way over the bar.
Neither team created many chances in the first half, and perhaps the slippery surface was to blame as there were heavy showers two hours before the match.
Seven minutes after the interval Wits' Rwandan midfielder Papy Faty found some space in the middle of the park and unleashed a shot from 30 metres that went just over the bar.
The visitors took the lead in the 62nd minute after some persistence from Kadi paid off: Walters failed to clear the ball, allowing Kadi to hit home from a tight angle.
Kadi could have grabbed his second just two minutes later, after a brilliant cross by Bryce Moon from the right, but he directed his header wide.
The Clever Boys hung on for the full three points and their first ever away win over Maritzburg United.