ThO2 is a promising fuel for next-generation nuclear reactors. As a critical quantity measuring its radiation tolerance, the dependence of the threshold displacement energy on temperature and crystal orientation in ThO2 is unclear and established using comprehensive molecular dynamics simulations in this work. For both Th and O primary knock-on atoms (PKAs), the thresholds, denoted as E_d^Th and E_d^O, respectively, are calculated using two different interatomic potentials. Similar temperature and orientation dependence are observed, albeit with some quantitative differences. While on average over all orientations, higher energy is required for Th PKAs than O PKAs to displace atoms, the polar-averaged E_d^Th is significantly lower than that for E_d^O. Further, E_d^Th and E_d^O show different crystal orientation dependence and temperature dependence. Notably, the cubic symmetry in the fluorite structure is followed by E_d^Th but does not hold for E_d^O because of the existence of two sublattices. The much higher average E_d^O than E_d^Th and their different temperature dependence are interpreted by the distinct recombination rates of Th and O Frenkel pairs in thermal spikes, resulting from the substantially lower migration barriers of O vacancies and interstitials. The recombination of O vacancies and interstitials, both of which are charged, is further enhanced by the Coulomb interaction at small Frenkel pair separations. The new findings are discussed for their generality in fluorite-structured oxides by comparing the results in ThO2 and UO2.