He has got to move more to the mainstream He has to be where the moderates and independents are that love the president.
He added that the President needed "more loyalists" to implement his plans.
A full day after the violence erupted, and after an initial statement in which Trump made no mention of white extremism, a White House spokesperson issued a statement saying, "The president said very strongly in his statement yesterday that he condemns all forms of violence, bigotry, and hatred".
A former Wall Street financier, Mr Scaramucci was sacked from his job at the end of a chaotic 10 days which started with White House press secretary Steve Spicer resigning.
"I've never sat down with Steve Bannon and said, 'Hey, are you a white nationalist or a white supremacist?' But I think the toleration of it by Steve Bannon is inexcusable", Scaramucci said.
"You also got this sort of Bannon-bart influence in there, which I think is a snag on the president", Scamucci said Sunday on ABC's This Week, referencing Bannon's past job as the head of the nationalist website Breitbart News.
Lawmakers on both sides of the political aisle have called out Trump's apparent reluctance to specifically mention the white supremacist groups who gathered in Charlottesville for a "Unite the Right" rally.
He said he believed the call was "off the record" and that it was "a very deceitful thing" for Lizza to record the call.
Scaramucci was sacked only 10 days into his job as communications director after the New Yorker published an obscenity-laced interview he had with a reporter. But I think it's important for the people around him, George, to give him direct advice, to be blunt with him.
Scaramucci's short tenure was accompanied by a series of scandals in the media, with the then-communications director scolding previous and acting White House staff members. "I think the president knows what he's going to do".