Celtic made a few changes as Sipho Chaine, Wandisile Letlabika, Justice Chabalala and Victor Letsoalo came in, whilst Usuthu rested top scorer Bonginkosi Ntuli for Lehlohonolo Majoro.
Celtic were denied the opener on 25 minutes after Chabalala's header bounced off the inside post before trickling across the goal-line with Siyabonga Mbatha on hand to smother.
Usuthu did hit the front though when Chawapihwa raced onto a through ball to slip through an on-rushing Chaine 34 minutes gone.
Inside two minutes, Celtic threatened to go level but Mbatha acrobatically tipped Letsoalo's header behind for a corner following a cross from the right.
Siwelele eventually levelled matters just before halftime when Michael Morton turned a cross from the left into his own net whilst attempting a to clear.
However, Siwelele surged ahead within 65 minutes after skipper Letlabika got on the end of a free-kick to volley home from the back post.
Usuthu introduced their top scorer Ntuli following the 71st minute as Vukusic launched a comeback with Morton replaced.
Celtic almost had a third within a minute after Ishmael Njoti beat the offside trap to round Mbatha before having shot cleared off the line by Sibusiso Mabiliso.
Those nerves were put to the bed on 86 minutes as Njoti arrived into the area to turn home a low ball into the area to extend the home side's advantage after Usuthu failed to clear their lines.
Celtic increased their advantage two minutes later as Neo Maema left Mbatha flat-footed from the penalty spot after Maduna brought down Ndumiso Mabena.
Siwelele were in the mood for more, but had already done enough to confirm their spot in the last 16.
Celtic: Chaine, Rikhotso, Nyamende, Maloisane, Letlabika, Chabalala, Phalane, De Jongh, Maema, Njoti, Letsoalo.
Subs: Mabokgwane, Masuku, Mabena, Mashikinya, Luthuli, Baloyi, Matjila.
AmaZulu: Mbatha, Sikhakhane, Mabiliso, P.Gumede, Khumalo, Vilakazi, Morton, Ncobeni, Magaqa, Chawapihwa, Majoro.
Subs: Boshoff, Masilela, M.Gumede, Maduna, Lacny, Shangase, Ntuli.