After Judas Moseamedi had a goal ruled out for an offside against Menzi Masuku after just three minutes, Botswana took the lead in the 16th minute when Okgabetse Makgantai's audacious attempt from 25 yards out caught goalkeeper Reyaad Pieterse off his line.
In the second half Botswana had a spell of pressure but could not turn their corner-kicks into goals, and substitute Thabiso Kutumela put South Africa in the lead with an easy finish after a quick free-kick found Motupa, who drew the goalkeeper before giving Kutumela the tap-in. There was a suspicion of offside against Motupa but the goal stood.
But South Africa were not to be denied and found a winner in the 88th minute when Modiba's cross hit Joel Mogorosi's hand as he charged down the ball, and the referee was kind enough to point to the spot. Motupa stepped up again and made no mistake.
The tournament win is South Africa's fourth COSAFA Cup trophy having also won it in 2002, 2007 and 2008.
Swaziland won the bronze medal after beating DR Congo 1-0 in the third-place playoff earlier on Saturday, Sabelo Ndzinisa netting their goal in the 41st minute.
Botswana: Masule, Gadibolae, Gaolaolwe, Kebue, Makgantai, Galenamothlale (Seakanyeng 60'), Tshireletso, Mafoko, Sesinyi (Mogorosi 60'), Ditsele, Ramoraka