Whyte has had an agonising wait for a title shot and took aim at one of the holders by suggesting the Gypsy King was overpraised for his 2015 victory against Wladimir Klitschko in 2015.
He told Sky Sports: "Everyone lets him get away with talking s***. It's frustrating.
"Klitschko was a great fighter, a great champion, but when he fought Klitschko, I don't know what happened to Klitschko that day.
"Wladimir, sometimes he's good and other times - it didn't seem like he was there to fight.
"With Wladimir, it's like some fights he's good, then other fights it's like he's just psychologically not there. He wasn't there that night for sure and we never got to see the rematch."
Whyte strengthed his title credentials with a win over Alexander Povetkin last month and says he is beating better fighters than both Fury and former world title holder Deontay Wilder.
He said: "Fury and Wilder get too much credit for their padded, cherry-picked records and have only beaten one current Ring Magazine top 10 fighter each.
"I've beaten three. Fury calls himself the greatest of all time, yet has turned down fights against me, even when ordered by the WBC. The guy's a joker."
The Brit could be in line to fight Fury after the heavyweight takes on Anthony Joshua but doubts that it will happen.
Whyte said: "He's avoided me two times before, so let's see, but who knows."