Celtic had led for 40 minutes of the match after Darryl Smith had opened the scoring toward the end of the first half, but late goals from Patrick Mabedi, Sandile Ndlovu and Maimane Phiri saw the Birds claim a remarkable win that builds on their triumph over Mamelodi Sundowns in midweek.
Celtic almost found an opening in the sixth minute when an overhead kick fell kindly for Ntokozo Sikhakhane, but the midfielder sliced his shot wide of goal.
Six minutes later Lebohang Kukame's control let him down and allowed goalkeeper Greg Etafia to sprint off his line and deny the Celtic striker a great scoring chance after a slick through-ball by Sikhakhane.
Alton Meiring spurned a great chance to give Swallows the lead midway through the first half, shooting straight at keeper Posnett Omony after a superb ball from Japhet Zwane out on the left flank.
Defender Bevan Fransman brought out the best from Omony in the 33rd minute with a rasping drive that forced the Ugandan goal minder into a fine acrobatic save.
Darryl Smith put Celtic ahead with a fine solo effort in the 39th minute, powering his way through from midfield and beating Etafia with a well-placed shot.
Meiring should have given Swallows an immediate reply but he missed an absolute sitter from close range after superb work by Zwane.
Kleinbooi Taaibos and James Chamanga both went close for their respective teams in the dying moments of the first half, but it remained 1-0 in favour of Celtic at half time.
The visitors almost found themselves 2-0 up 11 minutes into the second half when Timothy Batabaire directed a header on target, but Lefa Tsutsulupa did well to turn the ball away.
In the 65th minute Smith almost bagged a second, but his powerful drive from a tight angle was beaten away by Etafia.
Swallows brought themselves back on level terms in the 79th minute when Patrick Mabedi helped Cecil Oersen's free kick into the back of the net.
Ten minutes later Ndlovu put them into a 2-1 lead, firing home from close range after a scramble in the penalty area.
And Phiri wrapped up the scoring deep into injury time, sweeping home a goal on the counter to add insult to injury as far as Celtic were concerned.