Deuteronomy 2:12 (ESV)

12 The Horites also lived in Seir formerly, but the people of Esau dispossessed them and destroyed them from before them and settled in their place, as Israel did to the land of their possession, which the LORD gave to them.)

The Horites were distinct from the Rephaim and Emim. They may well have been the Hurrians, who were a non-Semitic people that appeared in various places in Syria and Palestine. The Horites in the region of Seir had been dispossessed by the people of Esau.