A former United academy graduate, Pogba returned to Old Trafford before the season for a world record transfer fee of £89 million after achieving superstar status in four years at Italian club Juventus.
The 24-year-old though has only showed flashes of his brilliance for the Red Devils with his performances being labeled as inconsistent while he has also been criticised for all his activity on social media.
Brown, who won five Premier League titles and two Champions League medals with United, feels it is only a matter of time before Pogba finds his best form.
The former England defender said: "I know a lot of people say all the other stuff is getting to him a little bit, but I don't necessarily think it is.
"I'd just be advising him to get on with his football. Everyone has their own lives out of football and that's just the way it is. Obviously people are highlighting it a bit more because of who he is.
"He joined for a lot of money and we haven't seen his best form yet, but I'm sure it's only a matter of time before he starts to deliver.
"People are talking about his lifestyle off the pitch too much. But what counts is on the pitch and I'm sure he will produce good performances soon."