CAPI stands for "Coherent Accelerator Processor Interface" which enables FPGA to access Host memory by virtual address. You can find more introduction about this interface on https://developer.ibm.com/linuxonpower/capi/. It is an important feature to develop hardware accelerators in heterogeneous computing. In this document, the "hardware accelerators" are built on FPGA.

A complete accelerator has software part (APP, or Applications) running on CPU Processor and the hardware part (AFU, Acceleration Function Unit) running on FPGA chip. APP and AFU are sharing host memory, that means, they both can read and write the 2^64 range of virtual memory address. To make it happen, CAPI technology has a CAPP (Coherent Acceleration Processor Proxy) logic unit in Processor chip, and also needs a PSL (Processor Service Layer) logic unit in FPGA chip. For CAPI1.0 and CAPI2.0, the interconnection between processor and FPGA is using PCIe physical links and PCIe form factor. CAPI1.0 uses PCIe Gen3x8 and CAPI2.0 uses PCIe Gen4x8 or Gen3x16. (For OpenCAPI, the physical links and the connected datalink layer and transportation layer all change to OpenCAPI. Please check https://opencapi.org for more information. It is not covered in this document.)

Let's focus on the FPGA side. A customer FPGA card needs to have a PSL module (Processor Service Interface) to become a "CAPI-enabled" card. This PSL module is provided by IBM. For CAPI1.0, it is in the form of a post-implemented dcp file (Xilinx Vivado design checkpoint). For CAPI2.0, it is encrypted source code. They can be downloaded at https://www.ibm.com/systems/power/openpower. From the menu, select "CAPI","Coherent Accelerator Processor Interface (CAPI)" or directly click the "CAPI" icon to go to the CAPI section. Then download the appropriate files depending on your target system being POWER8 (CAPI 1.0) or POWER9 (CAPI 2.0). You need to register an IBM ID to download them. A project from FPGA Vendors (i.e, a Xilinx Vivado project), including PSL module and sample user logic (AFU), is delivered to acceleration developers. This project is called HDK (Hardware Development Kit).

Working on HDK, developers need to know the details about PSL interface specifications. You can find CAPI1.0 PSL Spec and CAPI2.0 PSL Spec or search "PSL/AFU interface" in your web browser. But they have differences. To hide the differences on the interface and provide an industry standard interface protocol (AXI), we also created SNAP framework. SNAP is the abbreviation of Storage, Networking and Analytics Programming. It is an open-source framework https://github.com/open-power/snap. On the FPGA side, SNAP framework adds a PSL/AXI bridge, a DDR SDRAM controller and an optional NVMe controller. Thus, the developer can focus on their acceleration kernel logic (here we call it hardware action) and interface the framework via several AXI ports.

For both CAPI1.0 and CAPI2.0, people can choose to work on HDK or on SNAP. The preferred way is to work on SNAP. In the following chapters, we will introduce: Enable a FPGA card in SNAP Generate a PSL Checkpoint (CAPI1.0 only) Work with HDK (CAPI1.0) Work with HDK (CAPI2.0) For most of the new cards on Power9, just reading the chapter of "Enable a FPGA card in SNAP" is enough. You can find abundant materials on how to develop an accelerator with SNAP (Training videos, "docs" folder on github, or other webpages) so they are not discussed in this document.