SMT, Spatial Multithreading, and the battle for a CPU core

What SMT is trying to recover

Four kinds of parallelism are easy to blur together:

Instruction-level parallelism

One thread supplies independent instructions that a wide core can execute together. Bigger windows, better branch prediction, and more execution ports help.

Thread-level parallelism

Several software threads run at once, either on separate physical cores or as hardware threads sharing one core.

Data-level parallelism

SIMD and vector instructions apply one operation to many values. This is separate from SMT, although both use some of the same execution resources.

Core-level parallelism

More physical cores replicate pipelines, schedulers, and private caches. That costs much more silicon than another hardware-thread context.

A superscalar core already searches for parallel work inside one instruction stream. SMT supplies a second stream, giving the scheduler more candidates when the first cannot fill the machine. Its value can be expressed as a deliberately rough balance:

This is why a second hardware thread never means a second core, and why an operating system should not treat two logical CPUs as two equal physical CPUs.

How classic SMT2 shares a core

Intel markets its long-running implementation as Hyper-Threading. AMD uses the generic SMT name. A conventional SMT2 core exposes two logical processors, each with the architectural state needed to look independent to the operating system: registers, control state, and a separate retirement identity.

Most of the expensive machinery is still shared. That can include fetch and decode bandwidth, out-of-order scheduling structures, execution units, load and store resources, translation structures, caches, memory bandwidth, and the core's power allowance. The exact policy differs by microarchitecture. Some queues are partitioned, some are competitively shared, and some reserve capacity for each thread to prevent starvation.1

“The rest of the data structures are either partitioned in half when HT is enabled or are ‘competitively’ shared.”8

Why sharing works

A cache miss may stop one thread for tens or hundreds of cycles. Another thread can issue useful work while the first waits. Even without a long stall, two instruction streams may complement one another: one happens to need integer units while the other uses vector or load/store capacity.

The best case is not that two threads each run at full single-thread speed. It is that together they recover enough otherwise-idle capacity to finish more work per second. On the right server workloads, that can be a large gain for a relatively small increase in core area.

Why sharing sometimes hurts

The sibling can evict cache lines, consume predictor capacity, occupy scheduler entries, saturate memory bandwidth, or make the core hit a power limit sooner. One important thread may slow down even when aggregate throughput rises. Tail latency can become less predictable because the interference depends on what the sibling happens to be doing.

What NVIDIA changes with Spatial MT

NVIDIA introduced Spatial Multithreading with the Olympus cores in its Vera data-center CPU. Vera has 88 cores and exposes 176 threads. NVIDIA says each core can run two tasks with partitioned core resources, and presents the feature as a way to provide stable performance, stronger isolation, and predictable tail latency under sustained load.23

That gives Spatial MT a different default priority from classic SMT. Conventional SMT lets either thread opportunistically consume much of a shared machine when capacity is available. Spatial MT deliberately constrains resource use so one task has fewer ways to disrupt the other. The apparent cost is less opportunistic borrowing; the intended benefit is more repeatable service.

NVIDIA's target helps explain the choice. Agentic and reinforcement-learning systems may run thousands of compilation, code-analysis, Python, tool, and sandbox tasks while GPUs wait for results. Aggregate capacity matters, but a long tail of slow CPU tasks can also hold up the next stage of the loop.

Spatial MT can be pictured as dividing a wide core into two execution territories, but that is only a mental model. It is not equivalent to two complete physical cores. The shared L3 cache, coherency fabric, memory controllers, power delivery, and other system resources remain possible points of interaction.

Why servers use four or eight threads per core

IBM POWER systems can offer selectable SMT1, SMT2, SMT4, and SMT8 modes. Oracle's SPARC M8 core supports as many as eight hardware strands and dynamically allocates core resources among the active strands.45

The rationale is specific to server work. Databases, virtual machines, Java middleware, and transaction-processing systems often have many independent requests. They also encounter memory latency, cache misses, and synchronization delays. When one strand waits, several others may be ready.

Eight-way SMT does not imply an eightfold speed-up. Decode width, execution bandwidth, cache capacity, memory bandwidth, and power are still finite. As the number of active strands rises, per-thread performance usually falls while aggregate throughput may rise.

Selectable modes matter because no one setting is best. A latency-sensitive database thread may prefer SMT1 or SMT2. A highly concurrent workload with frequent stalls may complete more work under SMT4 or SMT8. The useful setting depends on response-time targets as much as total work completed.

“The flexibility is enormous: the POWER8 can work with two heavy threads but can also transform itself into a lightweight thread machine gun.”9

The label "eight threads per core" also hides very different machines. Some early SPARC T-series cores used many strands to keep relatively narrow pipelines busy. SPARC M8 is a four-wide, out-of-order design with up to 192 instructions in flight. POWER cores are also large out-of-order server cores. The common feature is a large pool of ready contexts, not one common microarchitecture.

Why some wide cores omit SMT

Apple's A-series and M-series performance cores do not expose SMT. Apple combines wide performance cores with physically separate efficiency cores. An important foreground thread can occupy the P-core, while background work can run on an E-core without competing for the same private queues and execution units.

Interactive latency

User-visible work is often governed by a few critical threads. Exclusive access to a P-core makes their performance more predictable.

Energy proportionality

A small physical core can run background work with less active circuitry than a wide performance core.

Strong single-thread engine

Large instruction windows, broad execution, strong prediction, and large caches extract substantial work from one instruction stream.

System-level allocation

Silicon and power can instead go to E-cores, caches, graphics, media engines, and machine-learning acceleration.

“What really defines Apple’s Firestorm CPU core from other designs in the industry is just the sheer width of the microarchitecture.”10

This explanation is an architectural inference, not a published Apple statement that SMT was rejected for one specific reason. The defensible observation is narrower: Apple obtains additional throughput from separate small cores while preserving exclusive access to each performance core.

Lunar Lake makes a similar trade in x86

Intel removed Hyper-Threading from the Lion Cove P-cores used in Lunar Lake. The product combines four P-cores with four Skymont E-cores, for eight physical cores and eight threads.6

Intel presented the decision as an area and mobile-efficiency trade. Its projected architecture comparison claimed comparable single-thread IPC without Hyper-Threading, 15% lower dynamic capacitance, 10% less core area, 15% better performance per power, and 30% better performance per power per area. These are Intel's architecture projections, not independent end-product measurements.

“By eliminating HT, Intel reduces power consumption and simplifies thermal management, which should extend battery life in ultra-thin notebooks.”11

The hybrid topology changes the scheduling calculation. A free P-core is preferable to its SMT sibling, and a free E-core may also be preferable for secondary work. If the logical sibling is consistently the least attractive destination, removing it and spending the budget elsewhere can make sense.

Lunar Lake did not prove that SMT is obsolete. It showed that SMT can be optional when physical E-cores already supply secondary capacity and battery life is central to the product.

Sharing is also an isolation decision

Two mutually untrusted threads sharing fine-grained structures can observe timing effects or contend for resources in ways that reveal information. Cache side channels and transient-execution research have repeatedly shown that microarchitectural sharing complicates isolation.7

That does not make every SMT deployment insecure. The risk depends on the processor, firmware, operating system, mitigations, placement policy, and threat model. A cloud provider may avoid placing different trust domains on sibling threads. A high-assurance operator may disable SMT when the isolation requirement is worth more than the throughput.

Spatial partitioning is relevant because reducing fine-grained sharing may reduce interference and some potential channels. It is not a blanket security guarantee. Shared caches, fabrics, memory controllers, and software layers can still expose interactions, and NVIDIA does not claim that Spatial MT eliminates every cross-thread side channel.

The scheduler completes the design

A topology-aware scheduler knows which logical CPUs share a core, cache, die, or package. Its placement policy often follows a rough hierarchy:

Server software can refine this with CPU affinity, workload pinning, SMT-mode selection, and trust-domain placement. An application that mistakes every logical processor for a full core can instead oversubscribe the machine and lose performance.

The architectures side by side

Choosing for a real workload

Desktop compilation, rendering, and multitasking

SMT2 remains a sensible compromise. It can use pipeline gaps while several independent tasks are ready, although more physical cores are more valuable when area and power permit.

Gaming

Fast physical cores, cache capacity, and memory latency usually matter most. SMT can help background tasks and engines with enough parallel work, but the result varies by game and system.

Thin laptops

Non-SMT P-cores plus smaller physical E-cores can simplify placement and improve performance per watt. Apple Silicon and Lunar Lake are examples of that system-level choice, not proof that every laptop should follow it.

Databases and virtual-machine hosts

Many-way SMT can work well when there are numerous independent requests and frequent stalls. Benchmark the real workload, including tail latency, rather than relying on logical CPU count.

Dense AI infrastructure

Spatial MT is designed for the case where predictable concurrent service matters more than peak opportunistic sharing. Its real advantage will need validation on shipping systems and production workloads.

High-assurance systems

Restrict or disable SMT when sibling sharing conflicts with the threat model. Partitioning may help, but it does not replace a complete analysis of the whole memory and software stack.

There is no universal winner. Classic SMT maximizes shared utilization. Many-way SMT maximizes the pool of ready server work. Spatial MT prioritizes predictable concurrency. Heterogeneous client CPUs move secondary work onto separate small cores.

Glossary

Hardware thread

An architectural execution context visible to the operating system as a logical processor.

Physical core

A core with its own main execution engine. It may expose one or several hardware threads.

SMT

Simultaneous multithreading. Instructions from more than one hardware thread can be considered for execution inside one core.

Hardware strand

Oracle's term for a hardware thread in its SPARC architecture documentation.

IPC

Instructions per cycle, a workload-dependent measurement rather than a fixed property of a processor.

Tail latency

The slow end of a response-time distribution, often expressed as the 99th or 99.9th percentile.

P-core and E-core

Performance and efficiency cores optimized for different power, latency, and throughput points.

Noisy neighbor

A colocated workload that degrades another workload through contention for shared resources.

Related CPU microarchitecture reads

The AnandTech archive is especially useful here because it connects vendor block diagrams to measured behavior. These articles provide a path from the basic mechanics of out-of-order execution to several different answers to the resource-sharing problem.

• Understanding the Cell Microprocessor: out-of-order architectures explains instruction and scheduling windows, dependency tracking, and why a core searches beyond a stalled instruction.
• Nehalem: Hyper-Threading shows which structures Intel duplicated, partitioned, or shared when SMT returned to the Core lineage.
• Haswell's frontend follows the evolution of Intel's four-wide x86 frontend, micro-op cache, prediction, and two-way SMT.
• The Bulldozer aftermath is a useful counterexample: clustered multithreading shared a frontend and floating-point resources without behaving like conventional SMT or two complete cores.
• POWER8's heavy SMT examines how the core changes between SMT2, SMT4, and SMT8, including where eight active threads become cramped.
• The Zen 3 deep dive traces frontend, execution, cache, and latency improvements inside a mature AMD SMT design.
• Apple's Firestorm microarchitecture uses microbenchmarks to estimate an eight-wide decoder, a large out-of-order window, and broad execution resources.
• Golden Cove and Gracemont explains Intel's hybrid P-core and E-core organization before Lunar Lake removed SMT from its P-cores.
• Lunar Lake and Lion Cove covers the wider frontend, split out-of-order domains, cache redesign, and removal of Hyper-Threading.