With many of the best chances having gone the way of City it took an excellent Lyle Lakay strike in the second-half to separate the two sides.
The home team should have opened the scoring as early as the sixth minute with Jabulani Maluleke's pin-point corner finding the head of Sibusiso Mbonani who could do nothing more than head over.
Minutes later they had another opportunity from a corner and although the initial delivery failed to locate anyone, once it was swung in again Thapelo Tshilo headed the ball against the crossbar.
Just before the midway point of the opening stanza City struck the woodwork again as Vusimuzi Mngomezulu charged onto Rodney Ramagalela's low cross but he was slightly offside in any case.
The frame of the goal once again came to the visitors' aid minutes later as Ramagalela steered the ball towards goal from range but his daisy-cutter effort came back off the upright.
In the 38th minute Mbonani made a great sliding tackle to deny Tshegofatso Mabasa the chance to shoot after the ball squirted through to him in a dangerous area.
Mabasa then had another opportunity when slipped through and he arrowed the ball at George Chigova, who was forced to make his first save of the game, before almost gobbling up the rebound before it was put behind.
From the resultant corner Tshepo Rikhotso shaved the upright with his headed effort to close out an exciting first half.
Upon the resumption of play, after some nasty fisticuffs in the tunnel following the barring of Luc Eymael from the changerooms, both sides took a while to look threatening again.
In the 52nd minute City scythed through the heart of Celtic's midfield and Patrick Tignyemb was forced to rush off his line to deny Walter Musona and after Tlolane's defence splitting ball minutes later the veteran stopper was forced to do the same to this time keep out Mngomezulu.
The home side continued to apply pressure on Celtic but were undone in the 74th minute when the visitors ventured forward and Lakay let fly from range and beat Chigova with his sweetly hit effort.
Now chasing the game City continued their assualt on Tignyemb's goal with one of their best chances coming via the cultured boot of Maluleke who forced the 'keeper to palm away his free-kick.
In the dying moments Ramagalela again hit the post before the rebound was put wide as Celtic picked up a valuable three points that sees them leave the drop zone.