The match ended 3-3 after extra time and Phunya Sele Sele, who scored twice at the death of regulation time, triumphed 5-3 on penalties to join SuperSport United, Mamelodi Sundowns and Free State Stars in the semi-finals.
There were few chances created in the first half, before Bucs took the lead through a superb free-kick from Daine Klate, which curled into the top corner.
On 40 minutes the visitors almost doubled their advantage, with midfielder Andile Jali exchanging passes with Collins Mbesuma before firing a left-footed shot just wide of the goalkeeper's right-hand post.
The second half got off to a flying start, with three goals in the first 12 minutes.
Pirates made it 2-0 on 51 minutes when Oupa Manyisa chested down a cross from Klate before firing a low shot into the bottom corner.
Six minutes later the home side reduced the deficit to 2-1 when Lennox Bacela converted a penalty. The striker earned the spot-kick himself, being tripped up by Siyabonga Sangweni just inside the 18-yard area.
Bucs restored their two-goal lead in the 57th minute, when Thabo Matlaba scored his first goal for the club: a swerving long-range shot that wrong-footed Patrick Tignyemb before crashing into the back of the net.
Pirates substitute Ndumiso Mabena had a chance to kill off the hosts in the 86th minute when he stole the ball off Dominic Isaacs and had only the keeper to beat, but the young striker completely fluffed the chance – a moment that came back to haunt the Soweto giants.
Celtic made it 3-2 in the 90th minute when Ruzaigh Gamildien powered home a header, and in a dramatic ending substitute Joel Mogorosi made it 3-3 five minutes into injury time with a magnificent overhead kick, after a goal-mouth scramble.
Extra time was prosaic compared to the drama before it, and the tired teams eventually settled for a penalty shoot-out.
Celtic converted all five of their kicks, while Oupa Manyisa was the unlucky player to fail with his attempt, handing the Bloemfontein side a 5-3 victory.
Penalty sequence: Bacela scored, Jali scored; Manzini scored, Manyiza saved; Motshekgwa scored, Sangeweni scored; Mogorosi scored, Phungwayo scored; Isaacs scored.