In the goroutine execution model, any I/O call appears blocking, halting the code execution for the goroutine. However, underneath, network calls in the OS are actually non-blocking. When a network call occurs, the goroutine transitions to a "waiting" state, and the call itself is registered with a Go Scheduler component called NetPoller.
NetPoller itself is a wrapper, an interface over various implementations of non-blocking I/O in the host operating systems Go runs on: Windows, *BSD, Linux. In the case of Linux, NetPoller is a wrapper around epoll. When a response arrives for a non-blocking network call, NetPoller moves the goroutine to a "runnable" state and adds it to one of the queues of runnable goroutines.
However, non-blocking calls are not the only type; there are also blocking system calls. In this case, the goroutine transitions to a waiting state along with the OS thread, which is known as "M" in the Go Scheduler's terminology. The OS thread remains blocked with the waiting goroutine. Meanwhile, the logical processor (P) and its local queue of runnable goroutines look for another OS thread, another M. When the response from the system call arrives, the M thread looks for a logical processor for its goroutine (or, like NetPoller, adds the goroutine in a "runnable" state to one of the queues), while the M thread itself goes to sleep in the OS Thread Cache.
Finally, there is the logical processor, referred to as "P", which was mentioned earlier. It polls the queues of runnable goroutines, drains the local queues of other logical processors (P)—a mechanism known as Work Stealing—and checks the global queue of runnable goroutines. It also leaves OS threads (M) blocked by system calls, seeking out unblocked OS threads (M).
Thus, between a goroutine and a physical core, there are two levels—P and M—which assist each other during different types of goroutine blocking. If a goroutine is paused by a non-blocking call, there is no need to free the OS thread (M); it can simply take another goroutine from the queue. If a goroutine blocks the thread (M), execution can switch to another thread to continue running other goroutines. As a result, the physical processor never idles during I/O-bound tasks.
Based on articles about the Go Scheduler by Bill Kennedy and Daniel Morsing.
P.S. An LLM corrected me, pointing out that when M is blocked by a system call, P detaches, and another M looks for it. It doesn't exactly "leave" on its own. But that's beside the point. In principle, this was already implied, and it doesn't change the overall mechanics.
P.P.S. Here is what I realized: the Go runtime scheduler is not needed to "avoid physical cores going idle". It is needed to avoid a process written in Go falling into a waiting state, or avoid expensive context switching due to another system call or network request.
Goroutines and G/P/M defragment the OS process. As a result, the OS process falls into the "wait" state less often, and context switches at the OS level happen less often. Because the M/P pair allows constantly feeding RUNNABLE units of execution (G) onto a running M. Context switching of goroutines in Userspace is cheap.