Lerato Lamola opened the scoring midway through the first half before Hellings Mhango added the second just before the break.
Goodman Dlamini was the first to test Patrick Tignyemb with a low shot in the 14th minute, but it was the visitors who broke the deadlock three minutes later as Lamola profited from Marc van Heerden’s defensive error before rounding Tapuwa Kapini and firing home.
Celtic doubled their lead ten minutes later as Mhango surged into the box from the left side before firing over Kapini and into the roof of the net.
Usuthu came close to reducing the deficit in the 37th minute, but Tignyemb produced a great stop to deny Bongani Ndulula before holding on to another effort from Philani Shange.
The first chance of the second half fell for the away side in the 66th minute as Joel Mogorosi was played through on goal on the right side, only to rush his effort wide of the target.
Two minutes later it was another substitute, Bornwell Mwape, who almost pulled one goal back for the home side after Van Heerden picked him out, but Tignyemb kept out his solid strike.
Both sides had opportunities in the closing minutes, but Mogorosi fired just over the crossbar in the 84t minute, while Mwape saw his diving header also flying over the target at the other end.
Tignyemb made another good save to deny Usuthu a late consolation goal as he dived low to stop a header from Goodman Dlamini in added time.