The Kaizer Chiefs icon starred in the nation's sole Africa Cup of Nations triumph on home soil in 1996.
Since then Bafana have largely struggled to match expectations, most recently failing to qualify for the 2018 World Cup after also missing out on this year's AFCON.
Khumalo, who earned 50 caps and scored nine goals for Bafana, believes SAFA are failing to set up the proper structures to bring success.
The Baroka technical director said: "Are we as South Africans doing the right things in terms of getting our national team in a proper way in order for us to challenge the rest of the world? I don't think we are.
"Are we empowering the national team coaches in terms of them to prepare the teams? It doesn't matter which age group or gender, whether its Banyana Banyana or males, we are not.
"But the bottom line is that we need to get structures together because clearly there's something we are not doing right that was done 20 years ago. And believe it or not, if we are not getting our acts together and our structures together, there's no way that we can challenge the rest of the world.
"I personally believe that I'm from the school that learned from the best, and with us winning the 1996 AFCON, with us qualifying for the World Cup in 1998 in France, we learned from the best through difficult situations. But do we have to go back to that? Hell no, we don't have to do that.
"We have to go forward in terms of what is it that we need to do as South Africans to get ourselves to compete with the rest of the world."
The 50-year-old also wonders about the mandate of Bafana coach Stuart Baxter, who he won four trophies with at Chiefs when working under the Brit from 2012 to 2015.
"When you appoint Stuart Baxter, what is your objectives? You have to have your objectives," said the ex-Amakhosi assistant coach.
"Because I met with Stuart and what he explained to me it was more than what I ever thought. But I think it's more than the objectives that probably our association don't have.
"And probably this is an eye opener for us to plan for the future, not plan for the moment.
"So what I'm trying to say is that, let's look at the bigger picture. Are we trying to become a country that is supposed to be world beaters or are we supposed to be a country that adds numbers?"
Bafana will end their World Cup qualifying campaign tonight against Group D winners Senegal at Stade Leopold Sedar Senghor in Dakar (21h30 SA time).