To illustrate his point, Paul describes a hypothetical situation in which the whole church is together for public worship and everyone speaks in tongues (supposedly without interpretation).
The phrase outsiders or unbelievers
broadly refers to all those who are not part of the church. The Christian worship service has always been a public service, open also for non-Christians to attend.
It is not difficult to imagine what unbelievers would think if they would find everyone in the church speaking in different languages. At Pentecost the apostles themselves were accused of drunkenness (Acts 2:13).
23 If, therefore, the whole church comes together and all speak in tongues, and outsiders or unbelievers enter, will they not say that you are out of your minds?