The hosts took the game to Bafana in the opening exchanges and enjoyed most of the possession without causing any problems.
Fourteen minutes into the half, Bafana broke the deadlock when Tokelo Rantie collected a pass from Bongani Zungu and accelerated past two defenders to slot home from 12-yards out.
Despite going a goal down, Cape Verde continued to dominate possession, although they were finding it tough to carve out clear-cut chances.
The hosts eventually made their dominance count when Nuno Rocha fired a deflected effort past Ronwen Williams in the 31st minute after the Bafana defence failed to clear their lines.
Seven minutes later, Cape Verde were awarded a penalty when Ryan Mendes was adjudged to have been brought down by Dean Furman. Rocha converted the spot-kick to hand his side lead.
Baxter opted for a change seven minutes into the second half and withdrew Lebogang Manyama for Bradley Grobler.
Just as Bafana started to get into the game, they were reduced to 10 men after 66 minutes when Erick Mathoho was shown a straight red card for a late challenge.
Williams then came to Bafana's rescue in the 73rd minute. The shot-stopper produced a good save at his near post to keep the score at 2-1.
With 10 minutes remaining, Baxter introduced Percy Tau for Rantie as Bafana pushed for an equalising goal.
However, Bafana were unable to find a second goal as they suffered their first defeat in Group D, leaving them with four points from three matches.
Cape Verde: Dias. Almeida, Stopira, Rodrigues, Vally, Soares, Rocha, Babanco, Fernandes, Tavares, Mendes.
South Africa: Williams, Langerman, Mathoho, Hlatshwayo, Mphahlele, Furman, Zungu, Dolly, Zwane, Manyama (Grobler 52'), Rantie (Tau 80').