The German boss changed his entire starting line-up from the one which faced Stoke on Tuesday and was almost made to pay by an impressive Grecians outfit who twice took the lead.
Tom Nichols’ opener was swiftly equalised by 19-year-old Jerome Sinclair but Adam Bogdan conceded direct from a Lee Holmes corner to gift the hosts a 2-1 half-time advantage.
Brad Smith rescued a replay for the Barclays Premier League outfit with 17 minutes to go but Klopp was still left wanting more from his side.
“They (Liverpool team) tried everything but it is obvious what we have to learn in future, we have to be more robust,” he told BBC 1.
“We had a few good moments but, in a few one-on-one situations, they were more robust than us so we have to work on that for sure.
“Both goals were really unlucky for us, the first goal was their first offensive situation and the second one, yeah, ok…”
Just three Reds players boasted more than three first-team appearances while Kevin Stewart, Tiago Ilori and Ryan Kent made their debuts for the club.
Yet Klopp did concede he threw them in at the deep end given the tough conditions at St James Park.
“It was really difficult on the pitch but Exeter did really well,” he added. “They (Exeter) had a few parts where they played football, maybe they know on which part of the pitch it is possible to play football and which part you have to play long balls.
“Now we have to play again, I can’t believe another game, but you want it like this and we’ll be there.”
Liverpool have never lost to fourth-tier opposition in the FA Cup before but Nichols felt his side deserved a second chance at Anfield.
“I thought the boys were brilliant,” he said. “I thought we thoroughly deserved a draw. We didn’t park the bus, we gave it a right go and we’ve got a trip to Anfield.”
On his goal, he added: “I’d been doing some work with our coaches the day before, they were saying ‘you’ve got to run quicker into the box’, it was a great ball in and I got on the end of it.
“It was a great start, it was a shame we couldn’t hang on but we’ve got a trip to Anfield.
“The fans have been different class for the past four or five games, the atmosphere was unbelievable.”
EXETER 2-2 LIVERPOOL
RATINGS: Smith MOTM, but Bogdan flops
TALKING POINTS: Youngsters outshine senior trio
OPINION: Youngsters with spirited showing
FAN REACTION: “Well done kids”
TACTICAL: Klopp tweaks in-match to get draw