An accelerator' or an offload engine is a piece of specialized hardware circuitry that implements various functions in order to perform a set of operations with higher performance or greater energy efficiency than a general-purpose microprocessor. The use of accelerators by software to speed up various operations is called hardware acceleration. Accelerators are used to significantly improve the performance of certain workloads. They are often a separate component attached to a system, but may also be integrated directly into the processor.

Note that while an accelerator can also be a co-processor, this isn't always the case. The distinction lies in how an accelerator interact with the host processor. Whereas a co-processor is typically connected to the internals of the host processor which then passes it instructions to execute, an accelerator is typically viewed as an independent I/O device which is programmed though that interface.

Motivation & Overview

The use of accelerators is not a new idea. Already in the 1980s, floating point co-processors were an early example of accelerator adaptation. However, higher integration enabled by Moore's Law allowed companies to incorporate those functionalities onto the host processor, eliminating their purpose. For the next few decades afterward, general-purpose microprocessors continued to improve in single-core performance through Dennard scaling and Moore's Law. Modern SoCs incorporate an array of integrated accelerators for performing tasks such as cryptography and compression acceleration.

However, by the late 2000s Dennard scaling collapsed and single-core performance improvements started diminishing. Those deficiencies sparked renewed interest in accelerators. Accelerators can be implemented in silicon, as in the case of ASICs, or as soft hardware. Because this is hardware that is performing that equivalent of software code executing computationally heavy algorithms, the result is almost always either higher performance or more power efficient than what optimized code on a general-purpose microprocessor would be capable of.

Types of accelerators

Generally, accelerators fall into two categories:

• Data-driven accelerators - accelerators that are operate on a set of data independent of the CPU.

• Algorithm-driven accelerators - accelerators that perform a specific algorithm on dedicated hardware.