Celtic dominated for most of game and threatened in the 23rd minute when Batabaire rose highest to meet Siphesihle Ngobese’s corner, but Usuthu keeper Lawrence Ncala produced a superb save to deny the Celtic defender.
AmaZulu were resigned to long range efforts on the visitors’ goal, rarely forcing Celtic keeper Shuaib Walters into making a save.
The match remained goalless at the break.
Phunya Sele Sele continued their dominance in the second half and deservedly went ahead in the 78th minute when Batabaire headed home a corner by Moses Spandeel.
The hosts almost equalised in the 90th minute when Belux Kasongo headed over from a long throw-in by Lovers Mohlala.