Anderson was sticking to his guns, though, and after saving match point the eighth seed secured another break of the Federer serve and snatched a set back.
Federer was appearing at Wimbledon for the 20th time and 15 years after he first won the title, there were inevitably questions about whether the father of four had played his last match at his favourite tournament.
It was also Federer's earliest departure from Wimbledon since his shock second-round defeat against Sergiy Stakhovsky in 2013.
John Isner is also through to the last eight for the first time, seven years after reaching his only previous grand slam quarter-final at the US Open. You have to have that mindset.
The three-time champion spoke to Serbian media after his straight sets victory over Khachanov and revealed he has put in a request to play on Centre Court, having played there only once in this year's tournament.
Yet the 22-year-old, playing in the last 16 for the second successive grand slam, saw his hopes severely hindered by mistakes and a sliced forehand wide gave Djokovic a pair of break points - only one of which he needed.
South Africa's Kevin Anderson rallied from two sets down and match point to beat Roger Federer at Wimbledon on Wednesday.
There's Roger Federer and Rafael Nadal, of course, and Novak Djokovic, a third member of what was once the Big Four in men's tennis.
He was leading by two sets and 5-4 in the third when, with Anderson serving, he got to Ad-Out.
It was Federer who faltered at 11-11, double-faulting to hand Anderson a break point which he converted when the defending champion's tired forehand smacked the net.
Not only has he yet to drop a set through four matches at the All-England Club this year, but he hasn't had his serve broken, and it wasn't until his last match against Adrian Mannarino that he faced his first break point. The eight-time champion has won 32 consecutive sets at Wimbledon and two away from emulating the record he had achieved between 2005 and 2006.
"I think Rafa and Del Potro had to be on Centre Court", Murray told BBC.
"If you go out there with doubts or unsure what's going to happen, like I maybe did a little bit in that first set, it's not going to go your way".