Stuart Baxter's men must beat the Senegalese in Polokwane and in the return match four days later in Dakar to qualify for the 2018 World Cup in Russia.
Khumalo, who earned 50 caps and starred in Bafana's sole African Nations Cup triumph on home soil in 1996, hopes for a packed house at Peter Mokaba Stadium.
"I just want to urge all South Africans to come and support Bafana Bafana on the 10th of November as they are going to be playing against Senegal in Limpopo, Polokwane, Peter Mokaba Stadium," Khumalo said in a short video uploaded on Twitter by SAFA.
"Come show your love for your national team," added the newly appointed Baroka technical director.
Bafana are bottom of Group D with four points from as many games - two points behind Cape Verde and Burkina Faso, and four shy of the pace-setting Teranga Lions.
Kick-off is at 19h00, with tickets selling for R30.