Mzwanele Mahashe opened the scoring early for the hosts with a header before Lyle Lakay's deflected effort levelled matters.
Bakgaga made a great start to proceedings as Mahashe climbed well to meet a deep freekick and direct his header past Patrick Tignyemb.
Baroka were dominant in the opening stanza with Jose Garcia proving celtic's tormentor in chief but when afforded an opportunity in the 30th minute his effort was tame.
The visitors were let off the hook a few times as Baroka missed the target and only managed to go into the break ahead by a single goal.
After the interval it looked as though the second stanza may shape up similarly to the first but Lakay had other ideas as he let fly from outside the area and profited from a deflection to leave Oscarine Masuluke stranded as the ball went past him.
Baroka had the ball in the back of the net from a free kick in the dying seconds but it appeared the referee hadn't yet blown the whilstle to indicate they could continue and the set piece required a retake.
The Limpopo-based side remain 15th but move level on points with Free State Stars whilst Celtic go above Chippa United.