All the people that threatened him are dead (Exodus 2:15). He does not have to be afraid. This sign to leave exile and go home can also be found in Matthew 2:20, where Joseph has to take Jesus and Mary and go home because it is safer. Jesus is saved from the killing of Herod, just like Moses from Pharaoh.
19 And the LORD said to Moses in Midian, “Go back to Egypt, for all the men who were seeking your life are dead.”