Baxter earlier in the week said failing to beat Libya at Moses Mabhida Stadium in Durban today will detract from the momentous win over Nigeria in June 2017.
Bafana beat the Super Eagles 2-0 in Uyo to register their first competitive victory over the west Africans in what was Baxter's first game on his return to the national team hotseat following the sacking of Shakes Mashaba.
Baxter's men will next battle the Mediterranean Knights, who top the pool on goal difference courtesy of a 5-1 rout of Seychelles in their respective AFCON qualifying opener.
The top two finishers in the group will qualify for next year's continental showpiece, taking place in Cameroon from June 7 to 30.
"Obviously beating Nigeria, they are the powerhouses of African football, and beating them it shows that we can beat everyone," said Williams during a training session in the week at Princess Magogo Stadium in KwaMashu.
"So we need to take control of the group and win our home games. I mean if you win your home games it'll put us on 12 points, so we need to stay focused and believe in ourselves.
"We've got the quality ... it's time that we show it on the field.
"We want to qualify, we want to get the points on the board. It's our home game and we also want to put the disappointment behind us of last year," he added, referring to Bafana's dismal 2018 World Cup qualifying campaign.
"So we want to start on a positive note, to go there and get the three points."