An attempt at overplaying from Alisson led to Rachid Ghezzal scoring just after the hour mark, with Liverpool required to hold on as the Foxes looked the better of the two sides.
But as the final whistle blew, Klopp could relish his side maintaining their position at the top of the Premier League with four wins from their first four games.
“Result-wise it could not have been better, of course; performance-wise, we know that we can improve,” he told reporters after the game.
“We really know that we could have already done better, because again the start today showed how good we can be.
“But it’s still early in the season, so we’re not 100 percent stable in these situations, to do the right things constantly.”
The manager took exception to the condition of the pitch at the King Power, suggesting his side could have adapted their approach to counter the dry surface.
Liverpool are next in action on September 15, when they head to Wembley to take on Tottenham after the September international break.
With Spurs similarly impressive in the opening weeks of the season, the Reds will be required to retain their winning mentality if they are to fend off one of their title rivals.