Bafana came with an all-out attacking side, with a number of its first team regulars unavailable due to clubs refusal to release them for the competition.
The opening start was rigid from both sides, although the first chance fell to Bafana but Lebogang Maboe fired over the top following Luther Singh’s cutback.
It took a few minutes for the next chance to come, but Singh shot straight at the goalkeeper.
Madagascar looked troubling every time they got on the break, although their first attempt came after 31 minutes through Bourah.
It was the hosts who came up with the biggest chance but they were unable to make it count as Maboe spurned a one-on-one opportunity in the 34th minute.
After the break, Madagascar dangerman Bourah broke free on the side and got behind the defence, but Reyaad Pieterse made a stern block, with the follow-up cleared for a corner by his defence.
Bafana were unlucky not to go in front from a corner as Aubrey Modiba pounced on a loose ball but struck the upright after 62 minutes.
With five minutes to go, Bafana threw their last dice with a double change in Jamie Webber and Lyle Foster.
However, the visitors would eventually seal victory on penalties, where misses from Webber and Singh left Bafana on the brink, before Toby stepped up to beat Pieterse for a 4-3 win and a place in the semi-finals.
Bafana: Pieterse Xulu, Modiba, Fielies, Madisha, Makaringe, Maboe, Singh, Ndlovu, Moon, Motupa.