A lakclustre second-half showing resulted in Bafana's half-time lead being overturned as their 18-game unbeaten run ended with a 2-1 defeat at the Moruleng Stadium.
Baxter named an entirely new line-up from their momentous 2019 Africa Cup of Nations qualifying win away to Nigeria over the weekend, handing debuts to two players on the night.
However, the decision had an adverse effect as the new-look outfit crumbled to allow Zambia a sniff at victory before capitulating midway through the second stanza.
Baxter conceded his side learnt a few valuable lessons on how not to play and felt, had it not been for poor finishing, a draw would have been a fair outcome.
"We showed in the second half that we can't switch off, we have to keep doing the right things and when we don't do that then, instead of being a threat to the opposition, we're a threat to ourselves," Baxter said in his post-match interview with SABC Sport.
"But I'm pleased with the first-half performance and the last 15 minutes [of the match] we sort of kicked back into life and created a few chances.
"On the run of play we should have probably scored there and a draw would have ended a fair result, but if you miss chances you're always at risk of losing the game."