What Gedaliah told them is not only politically prudent, it is also accepting what the Lord had decided: Judah was to be disciplined due to their sins. In this sense, Nebuchadnezzar is a servant of the Lord (see Jeremiah 43:10).
Note that from Jeremiah 40:6 we learn that Jeremiah was going to live with Gedaliah in Mizpah, and in the following verse we are told of the visit by the army commanders. Thus, Jeremiah experienced all of this, and it is not unlikely that Jeremiah was present with the conversations that took place (see the report in Jeremiah 40:1 – 43:13).
24 And Gedaliah swore to them and their men, saying, “Do not be afraid because of the Chaldean officials. Live in the land and serve the king of Babylon, and it shall be well with you.”