While we might have expected Ehud to pull out this weapon and take it with him, it is likely he left it in because he had no time to lose, to clean it and try to hide it under his garment, and could not risk staining his garments or hands with blood should he remove the weapon.
1
22 And the hilt also went in after the blade, and the fat closed over the blade, for he did not pull the sword out of his belly; and the dung came out.