The house of the Lord
is just another way of speaking about the tabernacle/temple (Exodus 23:19; Exodus 34:26). In the Old Testament, the temple was the place where the Lord dwelled among Israel. His presence was manifested in the holy of holies within the temple. This was where old covenant believers would go to meet with the Lord, enjoy fellowship with him, and offer their sacrifices (see Leviticus).
During the Old Testament period, Jerusalem was the place where God made his special presence known to his people. In this sense, it was the centre of the world.1
1 I was glad when they said to me, “Let us go to the house of the LORD!”