Abafana Bes'thende asserted themselves in the early exchanges of the encounter with the hosts still settling in.
Siyabonga Dube had a chance to put the visitors ahead in the 13th minute with shot from the edge of the box but it sailed over the crossbar.
Ea Lla Koto settled into the match only after 25 minutes and started to cause problems almost immediately through Jantjie.
Siphelele Magubane had a wonderful opportunity to get on the score-sheet in the 37th minute after a flick-on inside the box by Knox Mutizwa but his half-volley went over the bar.
Mohammed Anas should have made sure the hosts went into the break ahead but he fired wide of the target from close range.
Dlamini broke the deadlock eventually for Stars four minutes into the second half, as he converted Anas's cross with a toe-poke inside the area, 1-0.
Jantjie made it 2-0 three minutes before the hour mark with a wonderful free-kick from the edge of the box.
However, Abafana Bes'thende immediately responded with a goal from Magubane, who was set up by Zolani Nkombelo.
Arrows threw everything at Stars after 75 minutes, while the hosts were waiting to hit on the counter-attack.
Anas was a constant threat for Luc Eymael's charges with five minutes left on the clock on counter-attacks.
Stars were compact at the back and did not allow the visitors too many chances in the closing minutes. They held on in the end to run out 2-1 winners.