Deuteronomy 4:27 (ESV)

27 And the LORD will scatter you among the peoples, and you will be left few in number among the nations where the LORD will drive you.

Here lies a threat of exile from the Promised Land. Israel would be scattered among the nations and reduced to a tiny number if God's people gave themselves to idolatry.