Giroud's clever header in the 85th minute secured the points after Lacazette had taken just 94 seconds to make his mark with a header from Mohamed Elneny's curling cross, equalling the record for the fastest opening goal in a Premier League season.
The Premier League has got off to a lively start, with the players trooping off after a massively entertaining 45 minutes of football. "We kept the faith and kept believing in our game to push hard", said Giroud who has vowed to fight for his place.
"I opened the door for him at some stage as it's important for him to play but I don't want him to go", Wenger said at the post-match news conference.
Will the speed of Lacazette's first goal be a sign of things to come?
Arsene Wenger's side scored twice in the last 10 minutes as Aaron Ramsey equalised before Olivier Giroud bagged the victor at the Emirates Stadium. Last season when Arsenal faced Liverpool in the season opener they had ended up on the losing side another 4-3 thriller.
Is he going to do the job of Shinji Okazaki?
Marc Albrighton served in a ideal ball from the left, curling a cross for Vardy, who capped the sequence against the run of play with his first goal of the season.
Arsenal staged a dramatic fightback to kick off the new Premier League season with a thrilling 4-3 victory against Leicester on Friday.
Danny Welbeck leveled proceedings deep in added time, scrambling the ball home from close range after it had been squared to him by Sead Kolasinac.
The impressive Vardy scored his second in the 53rd minute to put Leicester in front once more, curling Riyad Mahrez's corner into the bottom far corner amid poor Arsenal marking.
Xhaka sent the ball in and it fell to Giroud, who powered in a header off the underside of the bar, sparking wild celebrations at the Emirates.