An accelerator or an offload engine is a piece of specialized hardware circuitry that implements various functions in order to perform a set of operations with higher performance or greater energy efficiency than a general-purpose microprocessor. The use of accelerators by software to speed up various operations is called hardware acceleration. Accelerators are used to significantly improve the performance of certain workloads. They are often a separate component attached to a system, but may also be integrated directly into the processor.

Note that while an accelerator can also be a co-processor, this isn't always the case. The distinction lies in how an accelerator interact with the host processor. Whereas a co-processor is typically connected to the internals of the host processor which then passes it instructions to execute, an accelerator is typically viewed as an independent I/O device which is programmed though an interface.

Motivation & Overview

The use of accelerators is not a new idea. Already in the 1980s, floating point co-processors were an early example of accelerator adaptation. However, higher integration enabled by Moore's Law allowed companies to incorporate those functionalities onto the host processor, eliminating their purpose. For the next few decades afterward, general-purpose microprocessors continued to improve in single-core performance through Dennard scaling and Moore's Law. Modern SoCs incorporate an array of integrated accelerators for performing tasks such as cryptography and compression acceleration.

However, by the late 2000s Dennard scaling collapsed and single-core performance improvements started diminishing. Those deficiencies sparked renewed interest in accelerators. Accelerators can be implemented in silicon, as in the case of ASICs, or as soft hardware. Because this is hardware that is performing the equivalent of software code executing computationally heavy algorithms, the result is almost always either higher performance or higher power efficiency than what optimized code on a general-purpose microprocessor would be capable of.

Types of accelerators

Generally, accelerators fall into three categories, albeit some fall into more than one.

• Data-driven accelerators - accelerators that are operate on a set of data independent of the CPU
  • Vector accelerator - accelerators used for vector performing large vector operations
  • AI accelerator - accelerators that operate on predictive models such as artificial neural networks
• Algorithm-driven accelerators - accelerators that perform a specific algorithm on dedicated hardware
  • Compression accelerator - accelerators that perform compression using various algorithms
  • Cryptographic accelerator - accelerators that perform various cryptography operations
  • Graphics accelerator - accelerators used for the manipulation and creation of images
  • DSP accelerator - accelerators that performs algorithms used for the processing of digial signals
• Programmable accelerator - accelerators that come in the form of an FPGA and can implement any function necessary

Performance considerations

A critical parameter when performing performance analysis is speedup which consts of the accelerator's execution time, data transfer time, and sync time with host processor.

Where input and output time are the bus transactions including the data transfer and the setup time.