Pirates nearly broke the deadlock after four minutes. Gabuza broke through on goal and forced Harold Ndlovu into a fine save.
There was a good tempo to the opening 15 minutes of the encounter, with the sides playing free-flowing football.
Polokwane started to see more of the ball after the 20-minute mark, although they struggled to test Pirates shot-stopper Wayne Sandilands.
The visitors should have taken the lead in the 33rd minute but Mpho Makola somehow fired over the crossbar from six-yards out.
The Buccaneers ended the half as the stronger of the two sides, but they were unable to break the deadlock as the sides went into the break goalless.
Pirates broke the deadlock five minutes into the second half as Thembinkosi Lorch broke into the box and teed up Gabuza for the simple tap-in.
Ten minutes later, Rise and Shine were level when Rendani Ndou headed into an empty net after some good work by Rodney Ramagalela down the right.
Pirates coach Milutin Sredojevic responded with a change after 67 minutes as Justin Shonga was introduced for his debut.
Polokwane then took the lead in the 69th minute as Ramagalela shrugged off a challenge before firing past Sandilands.
Sandilands then came to Pirates' rescue in the 78th minute as he rushed off his line to deny Sibusiso Mbonani from close range.
Shonga nearly marked his debut with a goal in the 86th minute, but the Zambian could only fire into the side-netting from a tight angle.
However, Pirates eventually leveled matters in the first minute of stoppage time courtesy of a looping header from Gabuza to earn his side a point.