The action was end-to-end with both sides having numerous chances to score, but it took a goal in stoppage time from Nigerian Prince Olomu, his fifth of the season, to seal the points for Phunya Sele Sele.
Tshepo Bulu forced a fine early save from Celtic goalkeeper Patrick Tignyemb with a 30-yard screamer, before Sandile Ndlovu fired wide from the angle.
Patrick Malokase had an incredible chance to open the scoring when he found himself one-on-one with AmaKhosi goalkeeper Arthur Bartman, but the veteran stopper kept out his low drive with his legs.
Thabang Rooi struck the Celtic crossbar with a thunderous header from Josta Dladla’s corner, as close as either side came in the first half.
Chiefs might have had a penalty when Wellington Sambu cleared the ball against the arm of his own player, Johannes Motlhaping, while at the other end Olomu fired wide after an excellent lay-off from Ndovu.
Reneilwe Letsholonyane forced a good save from Tignyemb with a cross-come-shot, but then missed a sitter as he was put clear through on goal with 20 minutes to play, but put the ball wide with just the keeper to beat.
Bulu then missed another open goal as Tignyemb could only parry Tinashe Nengomasha’s shot into his path, but the young striker fired over with the empty net gaping.
The goal finally came in stoppage time as the Chiefs defence stood watching when Darrel Smith crossed for Olomu to side-foot home from six yards.