Celtic debutant Pelembe, a loan signing from Orlando Pirates, opened the scoring with an unstoppable 30-yard strike in the 34th minute.
Parker missed a penalty soon after but made amends with a first-time shot from range in the 63rd minute at Free State Stadium.
A talking point in the match emerged when referee Robert Smith awarded the spot kick after Bevan Fransman was adjudged to have impeded Siphelele Mthembu in the box. Fransman had already been booked in the 22nd minute but escaped a red card.