Thapelo Morena made a telling run in the fifth minute and whipped a low cross into the area but Thokozani Sekotlong arrived a second late and was unable to convert.
The Brazilians tried to find opening after 15 minutes but the Citizens were solid defensively and allowed them no space in the final third.
Sekotlong nearly opened the scoring for Sundowns in the 18th minute with a thunderous low strike from 25-yards out that sailed agonisingly wide.
City grabbed the lead in the 27th minute through Nana Akosah-Bempah, who turned home Aubrey Ngoma's cross with a composed finish following a counter-attack, 0-1.
With seven minutes left in the half, Sage Stephens pulled off a brilliant save to tip Hlompho Kekana's curling free-kick over the crossbar.
Roland Putsche could have doubled City's lead four minutes before the break but Denis Onyango produced a brilliant reflex save to deny the midfielder.
The visitors were dealt a blow just before the halftime break when Ngoma pulled up with an injury and was forced off.
Benni McCarthy was forced into making a substitution at the start of the second half as Ngoma made way for Teko Modise.
The Citizens defended deep in the opening 10 minutes of the second half as the hosts pushed for the equaliser.
Thamsanqa Mkhize should have punished the Brazilians in the 62nd minute when he was teed up in the six-yard box, but the defender ballooned his shot over the crossbar.
Mosimane made his first alteration in the 75th minute, as he brought on Yannick Zakri in place of Sekotlong while McCarthy brought in Victor Obinna for Akosah-Bempah.
Putsche almost scored a spectacular goal with five minutes left of play after hitting a thunderous dipping strike from 30-yards, but Onyango did well to smother it.
City kept their shape at the back and frustrated the hosts in the final stages of the match to claim a vital win that sees them move up to second spot.