Federer, meanwhile, was in total control before wasting four set points in the second set on his way to a 7-5, 6-7 (7), 6-2, 6-4 win against Dmitry Tursunov. The Swiss star next plays No 18 Ernests Gulbis of Latvia, who reached the fourth round here for the first time since 2008 - the year he reached the last eight – after beating Radek Stepanek 6-3, 6-2, 7-5. (Agencies)
"He played really well and went for big shots," Federer said. "It was difficult to finish the match, so obviously I'm relieved."
Djokovic dropped serve early to trail 3-1, made sloppy unforced errors in the tiebreaker, and was broken back in the fourth set after leading 4-2. He sealed the victory on his first match point when the 25th-seeded Croat double-faulted.
The six-time Grand Slam champion next plays either 13th-seeded Jo-Wilfried Tsonga of France or No. 22 Jerzy Janowicz of Poland, who played later on Thursday.
Djokovic is looking to win the French Open for the first time and lost to eight-time champion Rafael Nadal in the semifinals last year.
Federer, who had won his opening two matches in straight sets, converted only four of his 21 break-point opportunities against the 31st-seeded Russian.
The 17-time Grand Slam champion even joked when asked moments after the match if his poor conversion rate worried him.
"I pretend there isn't a problem," he said, breaking into a huge grin. "I'll go for the next one, and the next one, and the next one."
Shortly after winning the second-set tiebreaker, Tursunov appeared to have a minor meltdown while trailing 2-1 in the third. He argued with the chair umpire and took a medical timeout. He left the court for several minutes.
Federer is trying to win the title at Roland Garros for the second time. He is the 2009 champion, but lost in the quarterfinals last year.
"I don't want to tell all my cards. But I have a game plan," Gulbis said. "You don't need to be scared to do certain things against him, because most of the people they go on the court and they lose before it has started."
In the women's third round, third-seeded Agnieszka Radwanska became the latest favorite to be eliminated, losing 6-4, 6-4 to unseeded Ajla Tomljanovic of Croatia.
The loss means the top three seeded women have all been eliminated, following the exits of No 1 Serena Williams and No 2 Li Na.
"After seeing the two first seeds go out, you kind of feel like 'I can do this, too,'" Tomljanovic said.
It was the earliest Grand Slam exit for Radwanska, the 2012 Wimbledon finalist, since losing in the third round here two years ago. She reached the quarterfinals at the Australian Open and Roland Garros last year, the semifinals at Wimbledon and the fourth round at the US Open.
Her loss leaves No 4 Simona Halep of Romania as the highest-seeded woman remaining, although No 7 Maria Sharapova of Russia is now arguably the clear favorite. Sharapova was set to play Paula Ormaechea later Thursday.
The 21-year Tomljanovic, who had never previously played at Roland Garros, is into the fourth round for the first time at a Grand Slam. She is ranked 72nd and has never won a WTA title.
She next plays No 14 Carla Suarez Navarro of Spain, who beat American teen Taylor Townsend 6-2, 6-2.
"I definitely think the tournament is not over for me," Tomljanovic said.
Former US Open champion Samantha Stosur, seeded 19th, also advanced to the fourth round after beating No 9 Dominika Cibulkova of Slovakia 6-4, 6-4.
Federer, meanwhile, was in total control before wasting four set points in the second set on his way to a 7-5, 6-7 (7), 6-2, 6-4 win against Dmitry Tursunov.
The Swiss star next plays No 18 Ernests Gulbis of Latvia, who reached the fourth round here for the first time since 2008 - the year he reached the last eight – after beating Radek Stepanek 6-3, 6-2, 7-5.