The game saw Pirates scoring early on through Phenyo Mongala. Celtic equalised through Mark Mayambela, but Zimbabwean Zvenyika Makonese grabbed only his second ever goal in top flight South African football to seal a 94th minute win for the Sea Robbers.
The first half was action packed with the play shifting from end to end as both sides created several good opportunities on goal.
Early play suggested Celtic would score first and this was nearly the case when Prince Olomu hit the upright after just two minutes, having been threaded through on goal by Excellent Walaza.
But five minutes later Pirates went in front with virtually their first attack when Mongala got in front of keeper Patrick Tignyemb to head home Dikgang Mabalane’s excellently flighted cross.
Mabalane himself hit the bar five minutes later with a cracking strike as Bucs enjoyed a period of domination.
The Free State side, however, began to work their way back into the game and came close when Johannes Motlhaping’s cross just eluded two unmarked team-mates, while in the 28th minute Mayambela’s hammer-blow of a shot show-cased Pirates keeper and man-of-the-match Senzo Meyiwa at his very best.
More was to come from Phunya Sele Sele as Ditheko Mototo’s header just cleared the bar, while Malokase’s control was lacking when he looked set to head down on goal.
Just before the break there were more worries for the Pirates defence when Segolela blazed a trail to the by-line, but Bucs were able to escape.
Celtic continued to enjoy the upper hand after the break, making several incisions into the Pirates half through the likes of Wellington Sambu and Segolela, but they were not close to finding the end product.
However, the complexion of the game changed in the 69th minute when Mayambela squeezed a penalty under Meyiwa after Makonese was ruled to have fouled Walaza.
The game remained there for the taking with both teams having their moments in attack in the closing 20 minutes.
Celtic though looked dangerous and also looked set to score what would surely have been the winner in the 91st minute when Segolela set up Denis Kalala, but he was denied by another outstanding Meyiwa save.
Phunya Sele Sele at least deserved extra time, but they were cruelly denied this opportunity when Makonese scored from a header with only seconds to go to the final whistle.
Pirates (4-4-2): Meyiwa; Jele, Makonese, Lekgwathi, Manyisa; Mabalane, Mothinbantwa, Kamwendo (Fanteni 80’), Mongala (Nyanda 85’); Mbuyane, Mabena
Celtic (4-4-2): Tignyemb; Khaile, Sambu, Motlhaping, Goldstone; Mayambela, Mototo, Olomu, Segolela; Walaza (Spandeel 70’), Malokase (Kalala 58’)