Williams has been called up Baxter for back-to-back World Cup qualifiers with Cape Verde, along with clubmate Reyaad Pieterse and Orlando Pirates keeper Wayne Sandilands.
The 25-year-old was part of Baxter's Bafana squad that beat Nigeria 2-0 in an AFCON qualifier in June, providing cover to Itumeleng Khune.
With Khune and Darren Keet currently injured Williams has a chance to establish himself at international level, having endured a forgettable debut when a Neymar-inspired Brazil hit five unanswered goals past him at FNB Stadium four years ago.
"Things are looking up for me now with coach Stuart Baxter there and he knows me well," Williams tells KickOff.com.
"He knows what I can do, he knows me well. Obviously I haven't had the best Bafana Bafana record so far. Hopefully that can change with coach Stuart there and hopefully I can cement my place this season."
Baxter was in charge of SuperSport prior to being named as Shakes Mashaba's successor in the national team hotseat in May.
Bafana visit Cape Verde in Praia on Friday before hosting the return leg at Durban's Moses Mabhida Stadium four days later.