United recorded their first league away win since February despite Canaries goalkeeper Tim Krul saving penalties from Martial and Marucs Rashford.
Scott McTominay gave the visitors the lead with United's record 2,000th Premier League goal, while VAR intervened to award United a spot-kick only for Rashford's effort to be saved - but he made up for it moments later with his 50th club goal.
Martial - who made his first start since August and scored the winner during the midweek Europa League win over Partizan Belgrade - atoned for his own penalty miss with a goal in the 73rd minute before Onel Hernandez netted a late consolation for Norwich.
On the importance of Martial, United boss Solskjaer said: "It's vital for us. He's a top striker and to have him back fit, smiling, enjoying his football - that's a big thing for us.
"He's clever in his movement. We feel Anthony is an important player for us and we've felt the affect of him not being there so of course, I'm pleased to have him back and scoring goals."
When asked about his side's missed penalties, the United boss said lightheartedly: "We've won a few penalty shoot-outs this season, in the Carabao Cup pre-season tours, and they all score penalties, apart from when it comes to the ones which matter."
Reflecting on the game, Norwich boss Daniel Farke admitted it was always going to be difficult for his side to surprise on a weekly basis despite their impressive win over Manchester City on September 14.
He said: "We were not physical enough. My players always invest everything and we showed character in the second half but we were not good enough in the first half.
"If you are not there and everything is not top class, you have no chance as a newly-promoted side.
"There were a few things I was not happy with in the first half."