The question this answers
When does a workload need its own kernel, and when is an isolated process on a shared kernel enough?
The platform team runs forty small internal services plus one workload that executes code submitted by customers. The forty need to start fast and pack densely. The one needs a boundary that holds even if the code inside it is actively hostile.
An explicit answer to "how strong does the boundary need to be", so isolation is chosen from the threat model rather than from whichever technology the team already uses.
Two stacks, and the layer where they diverge
A virtual machine gets virtual hardware from a hypervisor and boots a guest kernel on it. Everything above that guest kernel — init, drivers, filesystem, your process — belongs to the VM and is invisible to its neighbours. A container gets namespaces and cgroups applied to a process running on the host kernel. Everything above the kernel is packaged in the image; the kernel itself is borrowed.
That is the only structural difference, and every practical consequence falls out of it. A VM boots an operating system, so it starts in tens of seconds and carries a hundreds-of-megabytes memory floor before your application allocates anything. A container executes a process, so it starts in milliseconds and its floor is the process itself. Ten VMs on a host mean ten kernels, ten page caches, ten sets of system daemons; forty containers mean one of each.
The hypervisor internals — trap-and-emulate, paravirtualized devices, nested paging — are taught in the Operating Systems domain and in Hypervisors and Shared Hosts. What matters here is the boundary they produce, and what it is worth.
The comparison, including the axis people leave out
Startup, density and portability are the axes everyone quotes, and containers win all three by a wide margin. The axis that decides real architectures is the fourth one: how much has to go wrong for one workload to reach another. On a hypervisor, an escape means defeating virtual hardware emulation. On a shared kernel, an escape means one exploitable syscall path — and the kernel exposes a very large number of them.
This is not an argument that containers are insecure. It is an argument that the boundary has a different *shape*. For your own trusted code, a kernel exploit is a remote possibility that patching addresses. For code a stranger uploaded, it is the threat model, and the honest answer is a VM boundary — which is exactly what the providers offering "serverless containers" quietly run underneath: a container interface on top of a per-tenant lightweight VM.
| Dimension | Virtual machine | Container | What decides it |
|---|---|---|---|
| Startup to serving traffic | Tens of seconds — firmware, kernel boot, init, then your app | Milliseconds to seconds — a process start, plus the image pull if cold | Whether a kernel has to boot |
| Memory floor per instance | Hundreds of MB before your application allocates | Roughly your process | One kernel each versus one kernel shared |
| Density per host | Tens | Hundreds | The per-instance floor |
| Isolation boundary | Hypervisor + separate kernel | Shared host kernel, narrowed by namespaces and cgroups | How many layers an escape must defeat |
| Blast radius of a kernel CVE | One guest | Every container on the host | Whose kernel it is |
| Portability of the artifact | Provider-shaped machine image, per-platform | One OCI image, any compatible host | Whether hardware is part of the artifact |
| Kernel version choice | Per VM — old and new side by side | Whatever the host runs, for everyone | Ownership of the kernel |
| Patching model | Patch the guest, or rebuild the image and replace | Rebuild the image for userland; the host team patches the kernel | Which half you own |
When the shared kernel is disqualifying
Three situations make the shared kernel the deciding factor rather than a caveat. Hostile tenants: you execute code you did not write, from parties who do not trust each other — a CI runner for public pull requests, a notebook service, an agent sandbox. Regulatory separation: an auditor requires that two workloads share no operating system, and a namespace is not an answer they will accept. Kernel-level requirements: the workload needs a specific kernel version, a custom module, or privileged operations that would erase the container boundary anyway.
The mitigation is not "harden the container". It is to put a real boundary underneath it: a VM per tenant, or a sandboxed runtime that gives each container its own minimal kernel. You keep the image, the pipeline and the registry; you replace only what the container sits on. That is usually a small change to the platform and a large change to the threat model.
Key points
- A VM boots its own kernel on virtual hardware; a container is an isolated process on the host kernel. Every other difference follows from that.
- Containers win startup, density and artifact portability by an order of magnitude — this is not close.
- The shared kernel is a genuine isolation caveat: a kernel vulnerability is a host-wide event, not a per-workload one.
- For hostile or mutually distrusting code, a VM boundary per tenant is the correct answer, and it does not cost you your images or pipeline.
- VMs let each workload pin its own kernel version; containers inherit whatever the host runs, for everyone on it.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • The hypervisor presents virtual CPUs, memory and devices; the guest kernel drives them as if they were hardware and schedules the workload itself.
- • A container runtime instead asks the host kernel for new namespaces (mount, PID, network, UTS, IPC, user) and a cgroup, then execs the process inside them.
- • The container's syscalls go straight to the host kernel — no guest kernel, no device emulation, which is why the overhead is close to a bare process.
- • Isolation is therefore *subtractive*: the process is a normal one, minus the visibility and capabilities that were taken away. Anything not taken away is still there.
- • A sandboxed runtime restores the missing layer by booting a stripped kernel per container, or by intercepting syscalls in userspace, trading a little latency for a hypervisor-grade boundary.
- • With VMs you own guest OS patching per machine; with containers you own userland patching per image and the host team owns the kernel for everyone.
- • You own the host kernel upgrade schedule for a container fleet — and a kernel upgrade drains every workload on the node, so it is a rollout, not a maintenance window.
- • You own the decision of what runs on shared hosts. "Everything on the shared pool" is a decision even when nobody made it.
- • You own capability hygiene:
--privileged, host path mounts and a mounted container socket each erase most of the boundary in one flag.
- • A container needs a kernel feature the host does not have; it works in the developer's environment and fails on the fleet with a permission or unsupported-operation error.
- • One workload exhausts a host-wide kernel resource — PIDs, inotify watches, conntrack entries — and unrelated containers on the same node start failing in ways their own metrics cannot explain.
- • A kernel CVE lands and every container host must be drained and rebooted; with VMs the same CVE is patched per guest without moving anything.
- • A team runs a privileged container to make a monitoring agent work, and the isolation boundary silently drops to nothing.
- • A VM fleet is chosen for isolation and then all forty services are packed into one VM anyway — the boundary was never actually applied.
- • Container scale-out is bounded by image pull and process start; VM scale-out is bounded by boot time, which is why VM autoscaling reacts in minutes and container autoscaling in seconds.
- • Density: what runs out on a container host first is memory, then host-wide kernel tables, long before CPU.
- • A VM fleet scales in coarse units — you buy a whole machine's worth of floor for each workload, which is why small services are wasteful on VMs.
- • Mixed fleets scale worst: two capacity pools, two autoscalers, two failure models, one team.
- • Both are real boundaries. The hypervisor boundary is smaller and older; the kernel syscall surface is enormous and changes every release.
- • A container escape is a host compromise, and on a shared host that means every co-tenant workload and every credential mounted into them.
- • Non-root user, dropped capabilities, read-only root filesystem and seccomp are the difference between a thin boundary and almost none. See Container Security and Its Limits in the Security domain.
- • Untrusted tenant code belongs behind a VM or sandboxed runtime. This is the one place in this module where the simpler option is the *stronger* one.
- • Containers reduce cost mainly through density: the per-instance memory floor disappears, so the same host serves far more workloads.
- • VMs cost more per workload at small sizes and roughly the same at large ones — a single large service saturating a machine gains little from containerization on the bill.
- • Sandboxed runtimes reclaim the boundary and give back some density and a little latency; that is the price of isolating untrusted code.
- • The hidden container cost is the platform around it: registry, build pipeline, node pool management, and often an orchestrator.
- • Container restarts and exit codes per node, so a bad node is distinguishable from a bad workload.
- • Node-level kernel resource counters — PID count, conntrack table, open file descriptors — which is where shared-kernel contention actually shows up.
- • Per-workload CPU throttling, since cgroup throttling is invisible in host CPU utilization.
- • The signal that lies: host CPU and memory look fine while one container is throttled to a crawl against its own limit.
- • Plain VMs, one workload each, when you have a handful of services and no packaging problem. Fewer moving parts, and the isolation question answers itself.
- • A managed container platform that hides the host fleet, when you want container packaging without owning nodes or kernels.
- • A sandboxed container runtime when you want container ergonomics with a VM-grade boundary — the right answer for untrusted code.
- • For a single large service that fills a machine, containerizing changes very little. The density argument only pays when workloads are small relative to the host.
- • Density and startup speed are bought with a shared kernel — a boundary you do not control and cannot patch per workload.
- • Artifact portability is bought with a build pipeline and a registry that a VM-based deployment does not need.
- • A VM per tenant buys a strong boundary and costs the memory floor, the boot time and the reduced density on every single instance.
- • Running both models buys flexibility and costs two operational systems — often the worst outcome for a small team.
What people believe, and what is true
Containers are less secure than VMs.
They have a thinner boundary, which is a different statement. A patched, non-root, capability-dropped container running first-party code is a reasonable boundary; the same container running a stranger's code is not.
Containers replaced VMs.
Almost every container in production runs inside a VM. The question is who owns that VM and how many tenants share it.
A VM is always the safer default.
A VM you never patch is worse than a container you rebuild weekly. Boundary strength and patch cadence are separate axes.