Moses Spandeel and Ntokozo Sikhakhane scored for the hosts, who dominated for most of the match.
However, it was Swallows who threatened in the very first minute of the game when Alton Meiring’s curling free-kick hit the post.
Spandeel handed Phunya Sele Sele the lead three minutes later when he scored after Swallows’ failure to clear Patrick Malokase’s cross.
The visitors went close in the 21st minute when Sandile Ndlovu combined well with Meiring, but the former Mamelodi Sundowns striker’s effort was punched clear by Celtic stopper Shuaib Walters.
Celtic threatened again in the 25th minute when Lebogang Mothibantwa’s volley was tipped over by Swallows keeper Greg Etafia, who was called into action again seconds later to tip over Petrus Mahlatsi’s header from the resultant corner.
The score remained 1-0 at the break.
Swallows came back strong in the opening stages of the second half, with Lefa Tsutsulupa instrumental. The midfielder went close in the 52nd minute when he latched onto a good cross by Japhet Zwane, but his effort hit the near post.
The match then went through a dull phase and only came to life in the final 10 minutes of the game.
With Swallows pushing more men forward in a bid to score an equaliser, they left themselves vulnerable at the back and were almost caught out in the 80th minute when Celtic substitute Carlo Scott ran clear, but Etafia produced a superb save to deny the striker.
Celtic were awarded a penalty a minute later when Etafia fouled Spandeel inside the area, but the home crowd felt the referee should have played advantage as Siphesihle Ngobese slotted home the loose ball.
However, referee Robert Smith stood by his decision, with John Maduka stepping up to take the spot-kick, which was superbly saved by Etafia.
Celtic were awarded another penalty in the 90th minute when Ngobese was fouled inside the area by Vincent Kobola and Sikhakhane sent Etafia the wrong as he found the corner of the net.