Lennox Bacela got the winner 11 minutes from time.
In the second minute of the match Thulasizwe Mbuyane was offered a chance to shoot from outside the box on the left side, but his shot flew wide across the face of goal.
In the 10th minute the home side had their first chance, when Ruzaigh Gamildien tried a shot from a tight angle, but Moeneeb Josephs made an easy save. The Celtic striker will reflect that he should have crossed for his two teammates in the box.
Onyekachi Okonkwo curled a beautiful right footed shot from 20 metres out a few minutes later, but Patrick Tignyemb parried the ball out for a corner.
Midfield dynamo Oupa Manyisa then tried shot from a tight angle on the right but his effort was also parried away for a corner.
Just after the half-hour mark there was a double chance for Pirates as Mbuyane rattled the woodwork from eight metres out. The ball rebounded and he had a second go from just outside the small box but his low shot was gathered by Tignyemb.
In the 65th Lerato Manzini fluffed a golden opportunity one-on-one with Moeneeb Josephs, but the keeper narrowed down the shooting angle and blocked his shot.
In the 79th the stadium was in raptures when the home side found the lead. After a long throw-in, Joel Mogorosi lobbed the ball over the Pirates defence and Bacela sneaked past Lucky Lekgwathi and Rooi Mahamutsa and put the ball beyond Josephs.
Celtic should have doubled their lead in the 87th minute when Mduduzi Nyanda unleashed a shot which was deflected toward an empty net, but Lekgwathi cleared the ball off the line.