The strugglers shocked the title-contenders 2-0 at the Absa Stadium with goals from Cyril Nzama and Jimmy Zakazaka.
While the windy conditions in East London made life difficult on the field, a full house of spectators at least ensured a great atmosphere.
Bay United belied their lowly log position with a storming start to the game, enjoying the majority of ball possession, dictating the pace of the play and forcing Downs keeper Calvin Marlin into a couple of early saves.
Marlin was first called into action in the fifth minute, having to be alert to make sure that Rodrigue Ebongue's cross-like shot did not dip under the bar.
United strung some attractive passing movements together, and in the 17th minute a free-flowing move down the right saw Bulelani Matross getting space inside the box to shoot, but his effort was directed straight at Marlin.
Umlilo's attacking endeavours would have been worthless without a goal to show for their efforts, but this was not to be the case as they took a deserved 24th-minute lead.
Former Kaizer Chiefs right back Nzama was the goalscorer; the 34-year old powering home his third goal of the season with a header following a free-kick from the right.
The goal stung Sundowns into action, and in the 29th minute a move involving several players should have seen Collins Mbesuma scoring, but the Zambian found himself off-side.
Five minutes later the Brazilians did have the ball in the back of the net through Vuyo Mere, but Esrom Nyandoro's preceding free-kick was deemed to have been taken too quickly.
Having soaked up the pressure, Bay United came close to adding a second on the counter-attack when Khayalethu Ngangqu went inches wide with a first-time volley.
The home side were quickly reminded of Sundown's potential in attack when Surprise Moriri crashed a shot into the bar in the 44th minute.
Umlilo were soon back on attack and a fantastic move opened up space for Zakazaka to go for goal from inside the right-hand-side of the box, but Marlin pulled off a very good save to deny Bay United the cushion of a second goal.
Sundowns began to raise their game in the second half, but on the whole they found the home defence to be determined and well organised, with skipper Duran Francis leading by example.
The best early chance of the second half fell to Sibusiso Zuma when he burst into the box, but Umlilo goal keeper Arthur Bartman made an excellent save.
There was a further example of Khabo Zondo's side's fighting spirit when they survived a goalmouth scramble in which Sundowns looked set to score.
Having been a virtual spectator for most of the second half, Marlin did well in the 76th minute to ensure his side remained in with a chance when he made a double block following an outstanding counter-attack involving Nhlanhla Vilakazi, Matross and Zakazaka.
With Sundowns searching for an equaliser, the threat of the counter-attack was always there for United, in the 77th minute that is exactly what happened when Zuma held onto the ball for too long and was dispossessed.
Umlilo quickly swept up field and, after Marlin had blocked the initial shot, Zakazaka was on hand to put the rebound into the top corner.
Downs nearly pulled a goal back seconds after the restart, but Bartman did well to block Zuma's near-post effort.
There were chances for both sides to score in the closing minutes, but the score-line remained unchanged as United went on to bag a crucial three points in their relegation battle.
The result does not lift Umlilo off the foot of the table, but will certainly be a confidence booster as they look to pull of a late escape act.
The Eastern Cape side have showed on numerous occasions this season that they can beat the big guns of South African soccer, but their big test comes in their next two games, when they clash with fellow relegation candidates Thanda Royal Zulu and Maritzburg United.