We are not explicitly told why these people wept. Some think they wept because the new temple was not going to be as glorious as the first temple. Even though the second temple was larger than the first temple (compare 2 Chronicles 4:1 with Ezra 6:3–4),it was never going to be as intricately designed and beautiful as the first temple.1 When Solomon built his temple, he had 3,750 tonnes of gold, 37,500 tonnes of silver, numerous skilled workers (more than 150,000 workers in total), lots of precious stones, 38,000 Levites to supervise the building, best foreign workers available to work the stone, gold, iron and bronze, and the people could build in peaceful circumstances without threat and interruption from their enemies (see 1 Chronicles 22:1 – 29:9 and 2 Chronicles 2:1–18.) Compare that with the exiles who have around 500 kilograms of gold and 2.9 tons of silver (Ezra 2:69). They were also only about 50,000 people if you include the servants (Ezra 2:64–65). There is no way the people would be able to build a temple that is as magnificent as the previous temple was.
Most probably, however, the people wept because they realized that the new temple would never contain the ark of the covenant. The ark of the covenant was a wooden chest covered with gold with two cherubims on top (Exodus 25:10–22). It represented God’s presence in the Old Testament and God promised that he would meet with Moses above the cover between the two cherubim (Exodus 25:22). In the tabernacle, the ark was placed in the Holy of Holies (Exodus 26:34) and it was shielded by a curtain (Exodus 40:3). Solomon placed the ark in the holy of holies when he finished building the temple (1 Kings 8:1–11) and the glory of the Lord filled the temple. The last time we read about the ark is in 2 Chronicles 35:1–6 (2 Kings 23:21–23) where the ark is returned to the Temple. Presumably, when king Nebuchadnezzar invaded Judah, he removed the ark from the temple along with the other vessels (2 Kings 24:13; 2 Kings 25:13–17). Since the ark did not come back to Jerusalem (Ezra 1:7–11) it must have been destroyed. Thus the presence/glory of the Lord had left Jerusalem, as Ezekiel had seen in his vision (Ezekiel 10:1–22), but with the return from exile, the presence/glory of the Lord had not returned (Ezekiel 43:1–12).
The second temple would be rebuilt, but God’s presence would no longer be found in this temple. The glory of the Lord had not returned from the exile. Thus, when the temple was completed in Ezra 6:13–22, the glory of the LORD did not fill the temple in a cloud as happened in the past (Exodus 40:34; Leviticus 9:23; 1 Kings 8:10–11). The glory days of the old covenant were past. But the glory days of the new covenant had not yet come. The older priests and Levites realized that the restoration and the return from exile was not yet the full restoration that God had promised in Jeremiah (Jeremiah 30:1 – 31:40, Jeremiah 33:1–26). The people weep because the final joyous consummation of God’s restoration still lies in the future.2
13 so that the people could not distinguish the sound of the joyful shout from the sound of the people’s weeping, for the people shouted with a great shout, and the sound was heard far away.