The Chilli Boys took the lead in the 30th minute via the long ball route as Cole Alexander’s 45-yard cross was flicked on and into the back of the net by Petrus Ngebo.
Celtic were the stronger side in the closing stages of the first half, but could not create any clear-cut chances on goal.
But after the break the Chilli Boys were back on top with a couple of quick chances, with Ngebo off target from 16 yards out.
And their lead was doubled nine minutes after the restart as Diego Brown went on a scything solo run which ended in a sublime finish which left Patrick Tignyemb rooted to the spot.
Celtic kept on coming at the home side and were rewarded for their efforts when Lennox Bacela drew Maxwell Khubeka into a foul in the box.
Bacela converted from the spot himself, and it signalled an intense period of pressure from Phunya Sele Sele as they attacked relentlessly.
Somehow though, the Cape side managed to keep the ball out the net as they held on heroically to take home the three points.