Revision as of 23:33, 13 May 2021

Sunny Cove (SNC), the successor to Palm Cove, is a high-performance 10 nm x86-64 core microarchitecture designed by Intel for an array of server and client products, including Ice Lake (Client), Ice Lake (Server), Lakefield, and the Nervana NNP-I. The microarchitecture was developed by Intel's R&D Center (IDC) in Haifa, Israel.

History

Intel Core roadmap

Sunny Cove was originally unveiled by Intel at their 2018 architecture day. Intel originally intended for Sunny Cove to succeed Palm Cove in late 2017 which was intended to be the first 10 nm-based core and the proper successor to Skylake. Prolonged delays and problems with their 10 nm process resulted in a number of improvised derivatives of Skylake including Kaby Lake, Coffee Lake, and Comet Lake. For all practical purposes, Palm Cove has been skipped and Intel has gone directly to Sunny Cove. Sunny Cove debuted in mid-2019.

Process Technology

Sunny Cove is designed to take advantage of Intel's 10 nm process.

Implementations

The Sunny Cove core is integrated into a number of Intel designs.

Architecture

Key changes from Palm Cove/Skylake

Skylake to Sunny Cove changes

Sunny Cove enhancements

Sunny Cove buffers

• Performance
  • IPC uplift (Intel self-reported average 18% IPC across proxy benchmarks such as SPEC CPU2006/SPEC CPU2017)
• Front-end
  • 1.5x larger µOP cache (2.25k entries, up from 1536)
  • Smarter prefetchers
  • Improved branch predictor
  • ITLB
    • Double 2M page entries (16 entries, up from 8)
  • Larger IDQ (70 µOPs, up from 64)
  • LSD can detect up to 70 µOP loops (up from 64)
• Back-end
  • Wider allocation (5-way, up from 4-way)
  • 1.6x larger ROB (352, up from 224 entries)
  • Scheduler
    • Larger scheduler (160, up from 97 entries)
    • Larger dispatch (10-way, up from 8-way)
• Execution Engine
  • Execution ports rebalanced
  • 2x store data ports (up from 1)
  • 2x store address AGU (up from 1)
  • New paired store capabilities
  • Replaced 2 generic AGUs with two load AGUs
• Memory subsystem
  • LSU
    • 1.8x more inflight loads (128, up from 72 entries)
    • 1.3x more inflight stores (72, up from 56 entries)
  • 1.5x larger L1 data cache (48 KiB, up from 32 KiB)
  • 2x larger L2 cache (512 KiB, up from 256 KiB)
    • Larger STLBs
      • Larger 1G table (1024-entry, up from 16)
      • Larger 4k table (2048 entries, up from 1536)
      • New 1,024-entry 2M/4M table
  • 5-Level Paging
    • Large virtual address (57 bits, up from 48 bits)
    • Significantly large virtual address space (128 PiB, up from 256 TiB)

This list is incomplete; you can help by expanding it.

New instructions

Sunny Cove introduced a number of new instructions:

• SHA - Hardware acceleration for SHA hashing operations
• CLWB - Force cache line write-back without flush
• RDPID - Read Processor ID
• Additional AVX-512 extensions:
  • AVX512VPOPCNTDQ - AVX-512 Vector Population Count Doubleword and Quadword
  • AVX512VNNI - AVX-512 Vector Neural Network Instructions
  • AVX512GFNI - AVX-512 Galois Field New Instructions
  • AVX512VAES - AVX-512 Vector AES
  • AVX512VBMI2 - AVX-512 Vector Bit Manipulation, Version 2
  • AVX512BITALG - AVX-512 Bit Algorithms
  • AVX512VPCLMULQDQ - AVX-512 Vector Vector Carry-less Multiply
• SSE_GFNI - SSE-based Galois Field New Instructions
• AVX_GFNI - AVX-based Galois Field New Instructions
• Split Lock Detection - detection and cause an exception for split locks
• Fast Short REP MOV

Only on server parts (Ice Lake (Server)):

• TME - Total Memory Encryption
• PCONFIG Platform Configuration
• WBNOINVD Write-back and do not invalidate cache
• ENCLV - SGX oversubscription instructions

Block diagram

Overview

Sunny Cove is Intel's microarchitecture for the CPU core which is incorporated into a number of client and server chips that succeed Palm Cove (and effectively the Skylake series of derivatives). Sunny Cove is just the core which is implemented in a numerous chips made by Intel including Lakefield, Ice Lake (Client), Ice Lake (Server), and the Nervana NNP accelerator. Sunny Cove introduces a large set of enhancements that significantly improves the performance of legacy code and new code through the extraction of parallelism as well as new features. Those include a significantly deep out-of-window pipeline, a wider execution back-end, higher load-store bandwidth, lower effective access latencies, and bigger caches.

Pipeline

Like its predecessors, Sunny Cove focuses on extracting performance and reducing power through a number of key ways. Intel builds Sunny Cove on previous microarchitectures, descendants of Sandy Bridge. For the core to increase the overall performance, Intel focused on extracting additional parallelism.

Broad Overview

At a 5,000 foot view, Sunny Cove represents the logical evolution from Skylake and Haswell. Therefore, despite some significant differences from the previous microarchitecture, the overall designs is fundamentally the same and can be seen as enhancements over Skylake rather than a complete change.

The pipeline can be broken down into three areas: the front-end, back-end or execution engine, and the memory subsystem. The goal of the front-end is to feed the back-end with a sufficient stream of operations which it gets by decoding instructions coming from memory. The front-end has two major pathways: the µOPs cache path and the legacy path. The legacy path is the traditional path whereby variable-length x86 instructions are fetched from the level 1 instruction cache, queued, and consequently get decoded into simpler, fixed-length µOPs. The alternative and much more desired path is the µOPs cache path whereby a cache containing already decoded µOPs receives a hit allowing the µOPs to be sent directly to the decode queue.

Regardless of which path an instruction ends up taking it will eventually arrive at the decode queue. The IDQ represents the end of the front-end and the in-order part of the machine and the start of the execution engine which operates out-of-order.

In the back-end, the micro-operations visit the reorder buffer. It's there where register allocation, renaming, and retiring takes place. At this stage a number of other optimizations are also done. From the reorder buffer, µOPs are sent to the unified scheduler. The scheduler has a number of exit ports, each wired to a set of different execution units. Some units can perform basic ALU operations, others can do multiplication and division, with some units capable of more complex operations such as various vector operations. The scheduler is effectively in charge of queuing the µOPs on the appropriate port so they can be executed by the appropriate unit.

Some µOPs deal with memory access (e.g. load & store). Those will be sent on dedicated scheduler ports that can perform those memory operations. Store operations go to the store buffer which is also capable of performing forwarding when needed. Likewise, Load operations come from the load buffer. Sunny Cove features a dedicated 48 KiB level 1 data cache and a dedicated 32 KiB level 1 instruction cache. It also features a core-private 512 KiB L2 cache that is shared by both of the L1 caches.

Each core enjoys a slice of a third level of cache that is shared by all the core. For Ice Lake (Client) which incorporates Sunny Cove cores, there are either two cores or four cores connected together on a single chip.

Front-end

The front-end is tasked with the challenge of fetching the complex x86 instructions from memory, decoding them, and delivering them to the execution units. In other words, the front end needs to be able to consistently deliver enough µOPs from the instruction code stream to keep the back-end busy. When the back-end is not being fully utilized, the core is not reaching its full performance. A poorly or under-performing front-end will translate directly to a poorly performing core. This challenge is further complicated by various redirection such as branches and the complex nature of the x86 instructions themselves.

Fetch & pre-decoding

On their first pass, instructions should have already been prefetched from the L2 cache and into the L1 cache. The L1 is a 32 KiB, 8-way set associative cache, identical in size and organization to previous generations. Sunny Cove fetching is done on a 16-byte fetch window. A window size that has not changed in a number of generations. Up to 16 bytes of code can be fetched each cycle. Note that the fetcher is shared evenly between the two threads so that each thread gets every other cycle. At this point they are still macro-ops (i.e. variable-length x86 architectural instruction). Instructions are brought into the pre-decode buffer for initial preparation.

x86 instructions are complex, variable length, have inconsistent encoding, and may contain multiple operations. At the pre-decode buffer, the instructions boundaries get detected and marked. This is a fairly difficult task because each instruction can vary from a single byte all the way up to fifteen. Moreover, determining the length requires inspecting a couple of bytes of the instruction. In addition to boundary marking, prefixes are also decoded and checked for various properties such as branches. As with previous microarchitectures, the pre-decoder has a throughput of 6 macro-ops per cycle or until all 16 bytes are consumed, whichever happens first. Note that the predecoder will not load a new 16-byte block until the previous block has been fully exhausted. For example, suppose a new chunk was loaded, resulting in 7 instructions. In the first cycle, 6 instructions will be processed and a whole second cycle will be wasted for that last instruction. This will produce the much lower throughput of 3.5 instructions per cycle which is considerably less than optimal. Likewise, if the 16-byte block resulted in just 4 instructions with 1 byte of the 5th instruction received, the first 4 instructions will be processed in the first cycle and a second cycle will be required for the last instruction. This will produce an average throughput of 2.5 instructions per cycle. Note that there is a special case for length-changing prefix (LCPs) which will incur additional pre-decoding costs. Real code is often less than 4 bytes which usually results in a good rate.

All of this works along with the branch prediction unit which attempts to guess the flow of instructions. In Sunny Cove, the branch predictor has also been improved. The intimate improvements done in the branch predictor were not further disclosed by Intel.

Instruction Queue & MOP-Fusion

cmp eax, [mem]
jne loop

cmpjne eax, [mem], loop

See also: Macro-Operation Fusion

The pre-decoded instructions are delivered to the Instruction Queue (IQ). In Broadwell, the Instruction Queue has been increased to 25 entries duplicated over for each thread (i.e. 50 total entries). It's unclear if that has changed with Sunny Cove. One key optimization the instruction queue does is macro-op fusion. Sunny Cove can fuse two macro-ops into a single complex one in a number of cases. In cases where a test or compare instruction with a subsequent conditional jump is detected, it will be converted into a single compare-and-branch instruction. Those fused instructions remain fused throughout the entire pipeline and get executed as a single operation by the branch unit thereby saving bandwidth everywhere. Only one such fusion can be performed during each cycle.

Decoding

Up to five (3 + 2 fused or up to 5 unfused) pre-decoded instructions are sent to the decoders each cycle. Like the fetchers, the Decoders alternate between the two thread each cycle. Decoders read in macro-operations and emit regular, fixed length µOPs. The five decoders are asymmetric; the first one, Decoder 0, is a complex decoder while the other four are simple decoders. A simple decoder is capable of translating instructions that emit a single fused-µOP. By contrast, a complex decoder can decode anywhere from one to four fused-µOPs. Overall up to 5 simple instructions can be decoded each cycle with lesser amounts if the complex decoder needs to emit addition µOPs; i.e., for each additional µOP the complex decoder needs to emit, 1 less simple decoder can operate. In other words, for each additional µOP the complex decoder emits, one less decoder is active.

MSROM & Stack Engine

There are more complex instructions that are not trivial to be decoded even by complex decoder. For instructions that transform into more than four µOPs, the instruction detours through the microcode sequencer (MS) ROM. When that happens, up to 4 µOPs/cycle are emitted until the microcode sequencer is done. During that time, the decoders are disabled.

x86 has dedicated stack machine operations. Instructions such as PUSH, POP, as well as CALL, and RET all operate on the stack pointer (ESP). Without any specialized hardware, such operations would need to be sent to the back-end for execution using the general purpose ALUs, using up some of the bandwidth and utilizing scheduler and execution units resources. Since Pentium M, Intel has been making use of a Stack Engine. The Stack Engine has a set of three dedicated adders it uses to perform and eliminate the stack-updating µOPs (i.e. capable of handling three additions per cycle). Instruction such as PUSH are translated into a store and a subtraction of 4 from ESP. The subtraction in this case will be done by the Stack Engine. The Stack Engine sits after the decoders and monitors the µOPs stream as it passes by. Incoming stack-modifying operations are caught by the Stack Engine. This operation alleviate the burden of the pipeline from stack pointer-modifying µOPs. In other words, it's cheaper and faster to calculate stack pointer targets at the Stack Engine than it is to send those operations down the pipeline to be done by the execution units (i.e., general purpose ALUs).

µOP cache

See also: Sandy Bridge § New µOP cache

Decoding the variable-length, inconsistent, and complex x86 instructions is a nontrivial task. It's also expensive in terms of performance and power. Therefore, the best way for the pipeline to avoid those things is to simply not decode the instructions. This is the job of the µOP cache or the Decoded Stream Buffer (DSB). Sunny Cove's µOP cache is organized similarly to all previous generations since its introduction in Sandy Bridge, however, its size has increased. The cache is organized into 32 sets of 8 cache lines with each line holding up to 8 µOP for a total of 2,048 µOPs. As with Skylake, the µOP cache operates on 64-byte fetch windows. The micro-operation cache is competitively shared between the two threads and can also hold pointers to the microcode. The µOP cache has an average hit rate of 80% or greater.

A hit in the µOP allows for up to 6 µOPs (i.e., entire line) per cycle to be sent directly to the Instruction Decode Queue (IDQ), bypassing all the pre-decoding and decoding that would otherwise have to be done. Whereas the legacy decode path works in 16-byte instruction fetch windows, the µOP cache has no such restriction and can deliver 6 µOPs/cycle corresponding to the much bigger 64-byte window. Previously (e.g., Broadwell), the bandwidth was lower at 4 µOP per cycle. The 1.5x bandwidth increase greatly improves the numbers of µOP that the back-end can take advantage of in the out-of-order part of the machine. This change attempts to improve instruction rate by alleviating bubbles, however everything is still hard-limited by the rename and retire which puts an absolute ceiling rate of four fused µOPs per cycle.

Allocation Queue

The emitted µOPs from the decoders are sent directly to the Allocation Queue (AQ) or Instruction Decode Queue (IDQ). The Allocation Queue acts as the interface between the front-end (in-order) and the back-end (out-of-order). Like Skylake, the IDQ is no longer competitively shared; it's partitioned between two active threads. Sunny Cove's Allocation Queue increased from 64-µOPs/thread to 70 for a total of 140 entries or roughly 10% more than Skylake. The queue's purpose is effectively to help absorb bubbles which may be introduced in the front-end, ensuring that a steady stream of 6 µOPs are delivered each cycle.

µOP-Fusion & LSD

The IDQ does a number of additional optimizations as it queues instructions. The Loop Stream Detector (LSD) is a mechanism inside the IDQ capable of detecting loops that fit in the IDQ and lock them down. That is, the LSD can stream the same sequence of µOPs directly from the IDQ continuously without any additional fetching, decoding, or utilizing additional caches or resources. Streaming continues indefinitely until reaching a branch mis-prediction. Note that while the LSD is active, the rest of the front-end is effectively disabled.

The LSD in Sunny Cove can take advantage of the larger IDQ; capable of detecting loops up to 70 µOPs per thread. The LSD is particularly excellent in for many common algorithms that are found in many programs (e.g., tight loops, intensive calc loops, searches, etc..).

Back-end

This section is empty; you can help add the missing info by editing this page.

This article is a work in progress!

Die

Core

• 10nm+ process
• Core from an Ice Lake SoC
• ~6.91 mm² die size
  • ~3.5 mm x ~1.97 mm

Core group

• 10nm+ process
• Quad-core from an Ice Lake SoC
• ~30.73 mm² die size
  • ~7.86 mm x ~3.91 mm

Bibliography

• Intel Architecture Day 2018, December 11, 2018