You must design a single server that sustains 100k concurrent client connections, most of them idle but long-lived. The naive thread-per-connection model collapses at this scale, so explain the I/O and concurrency model you would build instead, why it scales where the naive one does not, how many worker threads you would run relative to CPU cores, and which operating-system limits you would have to tune.

Why not thread-per-connection

100k threads means 100k stacks (~1–8 MB each by default) plus relentless context switching. The kernel scheduler thrashes, and most threads sit idle inside a blocking read() anyway. This is the C10k problem (in its modern form, C10M).

The event-driven model

One thread serves thousands of sockets: the kernel reports which descriptors are ready, and the thread processes only those.

• select()/poll() — O(n) per call: the kernel rescans the entire set every time. Does not scale.
• epoll (Linux), kqueue (BSD/macOS), IOCP (Windows) — O(1): the interest set is registered once, the kernel returns only ready descriptors.

int ep = epoll_create1(0);

epoll_event ev{};
ev.events = EPOLLIN | EPOLLET;        // edge-triggered
ev.data.fd = listen_fd;
epoll_ctl(ep, EPOLL_CTL_ADD, listen_fd, &ev);

std::vector<epoll_event> events(1024);
for (;;) {
    int n = epoll_wait(ep, events.data(), events.size(), -1);
    for (int i = 0; i < n; ++i) {
        int fd = events[i].data.fd;
        if (fd == listen_fd) accept_new_connections(ep, listen_fd);
        else                 handle_ready_socket(fd);   // non-blocking read/write
    }
}

The full architecture

• Non-blocking sockets (O_NONBLOCK) — otherwise one slow client stalls the whole event loop.
• Thread pool ≈ one per CPU core, each with its own epoll; SO_REUSEPORT spreads incoming accepts.
• Kernel tuning: raise ulimit -n (fd limit), widen the ephemeral port range, grow socket buffers and the accept queue backlog.
• No blocking operations inside the event loop: offload heavy work (disk, DB) to a separate pool.