Siphosakhe Ntiya-Ntiya and Siphelele Ntshangase got rare starts in the Chiefs line-up while Kgotso Moleko, Happy Mashiane, Bernard Parker and Darrel Matsheke were all recalled.
Gavin Hunt's charges had a lot of spark in the opening exchanges going forward but were unable to carve out any openings.
The Natal Rich Boyz took a shock lead in the 29th minute after shot-stopper Daniel Akpeyi mistimed a corner towards the far post, which allowed Thabani Dube to head into an empty net.
Amakhosi almost pulled a goal back a minute later after Matsheke went through on the right side of the box and played a cut-back at the near post towards Samir Kurkovic but the Serbian's close range shot was saved by goalkeeper Malcolm Jacobs.
Parker also had a chance to equalise for the hosts just before the break with a close range volley that blocked before Nkosingiphile Ngcobo went close with a first-time effort moments later.
After making four changes at the start of the second half, Nurkovic came close again to breaking the deadlock again five minutes later with a header inside the six-yard box that was saved by Jacobs.
Things went from bad to worse for Amakhosi as they were reduced to ten men in the 64th minute after Ntiya-Ntiya picked up a second booking for a tackle from.
However, Leonardo Castro went on to level matters for Chiefs in the last 13 minutes with an improvised half-volley on the floor after a cut-back by Dumisani Zuma.
Castro had a glorious chance to snatch the winner for Amakhosi in the last two minutes after a cross by Zuma towards the far post but he headed his effort over the bar from close range.
Chiefs were made to rue that miss as Siyabonga Vilane netted the winner for the Natal Rich Boyz in stoppage time with a strike into an empty net after Beyzel Goldstone's strike had struck the upright, sending the second tier side into the next round.
Chiefs: Akpeyi, Moleko, Cardoso, Akumu, Ntiya-Ntiya, Matsheke, Mashiane, Ngcobo, Ntshangase, Parker, Nurkovic.
Subs: Bvuma, Katsande, Castro, Manyama, Zuma, Frosler, Sasman, Blom, Radebe
Richards Bay: Jacobs, Maphathe, Mabua, Zulu, Dube, Mtolo, Mayo, Bakah, Ntsundwana, Mthembu, Ogbonna.
Subs: Mpakumpaku, Vilane, Goldstone, Mabuya, Zwane, Radebe, Mcineka, Ndlovu, Maphakisa.