The hosts had a big chance to open the scoring after nine minutes as Sibusiso Masina's poor clearance fell to Mzwanele Mahashe, who fired straight at Sage Stephens from point-blank range.
Five minutes later, Baroka broke the deadlock as Talent Chawapiwa cut onto his left foot and rifled the ball home from the edge of the box.
City, meanwhile, had a couple of good passing moments in the half, but they struggled to break down the home side's defence.
That proved to be the pattern for the Citizens for the rest of the first half, as Baroka shot-stopper Virgil Vries endured a quiet opening 45 minutes.
Benni McCarthy's charges upped the tempo in the second half as they pushed for an equaliser, while Teko Modise was also introduced for Masina in the 58th minute.
City had the ball in the back of the net after 68 minutes as the ball was bundled over the line. However, the strike was ruled out for a handball.
Ayanda Patosi should have levelled matters for the visitors with 15 minutes remaining but the midfielder failed to direct his header on target from close range.
The Citizens threw everything forward in the closing stages of the second half, but Baroka managed to see out the rest of the match to claim the win.