Liverpool arrived at the London Stadium knowing that if they avoided defeat they would extend their unbeaten run to 26 matches, beating the club’s all-time record set by Bob Paisley’s team in 1982.
They looked on course to do just that following a wonderful free-kick from Trent Alexander-Arnold.
But West Ham have lofty ambitions themselves this season and proved it in the second half when Pablo Fornals, whose corner had been palmed in by Reds keeper Alisson for the opening goal, broke away to score a second.
Kurt Zouma then headed a third and although Divock Origi pulled one back with seven minutes remaining, the Hammers held on for a notable victory.
However, Klopp was adamant West Ham’s first goal should not have stood as Angelo Ogbonna had impeded Alisson, and he felt Aaron Cresswell should have been red-carded for a first-half challenge over the top of the ball on Jordan Henderson.
He said: “In the first half we had complete control of game, scored a wonderful equaliser and were in charge of the game.
“In the second half they scored from a counter and a set-piece. But also that happened because we should have controlled the game still in the second half. It looked like we lost patience.
“So we lost that game. But I think it’s a clear foul on Alisson, how can it not be? When you see the dynamic of the situation how can that not be a foul?
“Then I saw only the situation with Aaron and Hendo now, and the way I see it is it’s a clear red card. There is no discussion. You cannot go like this into a challenge.”
A memorable win, David Moyes’ first in eight attempts against Klopp, saw West Ham leapfrog Liverpool into third place heading into the international break.
“I think we’re there on merit. You can see the quality and attitude of the players,” said Hammers boss Moyes.
“I want us to challenge the teams in the top four, whoever that may be. I’ve laid down that challenge to the players. Hopefully one of them could be us.
“Today was a challenge, Liverpool have been in such good form, But we’ve got some resilience, and counter-attacking and corner kicks are not a bad recipe.
“The biggest thing is this is a busy workload for any club. Look at the run of games we’ve got, while we are still growing the seeds of how we are going to become this better team.
“I want us to be competitive. I hope before now people were thinking West Ham’s a tough game. Today’s opposition have very little weaknesses. I want to give the players the credit they deserve.”