Baroka looked to sneak into the top eight after suffering a string of poor results recently, and were keen to give interim coach Doctor Khumalo a bright start.
Celtic, who have also experienced a bad run of form the better part of this year, were also trying to sneak into the top half after playing to a goalless draw with Chippa United in midweek.
After a slow start from both sides, the game got into the groove as the half progressed and chances kept coming in from both ends of the spectrum.
As neither side were able to break the deadlock despite the few half-chances propping up, the first half ended goalless.
It was the same start for both in the second half, with the amount of goal-scoring chances limited to a tee.
The breakthrough finally and it was the visitors courtesy of Victor Letsoalo to make it count with a low shot on 59 minutes.
Celtic showed why they boast the fifth worst defensive record, as they conceded two minutes later through Abdi Banda.
The comeback looked to be completed nine minutes to the end after Banda grabbed his brace with a cool finish.
However, Celtic were not to be denied as they salvaged a point right at the death, with Kabelo Dlamini getting his name on the score-sheet in second-half stoppage time.