Spurs had a goal controversially ruled out by VAR during the game, and the manager has questioned the decision.
He said: "The referee is in the office. I never complain at the guy with the whistle because it's not the referee any more.
"I think the referee should always be the man on the pitch and the people in the office should just support and assist.
"That's not the way it is. The referee who is not very good on the pitch, we cannot expect that he's good in the office.
"We're going in a direction that's really bad for the beautiful game, the game that everybody fell in love with."
Despite the decision, Mourinho insisted his team need to stronger mentally in order to achieve their ambitions.
The Champions League-winning boss acknowledged that his side failed to respond to the setback of the VAR decision.
Mourinho - who has previously managed the likes of Chelsea and Manchester United - said: "We have to do better, we have to be mentally stronger to cope with what happened during the game.
"We cannot mentally die after Michael Oliver's decision."