CAPI stands for "Coherent Accelerator Processor Interface" which enables FPGA to access Host memory by virtual address. You can find more introduction about this interface on https://developer.ibm.com/linuxonpower/capi/. It is an important feature to develop hardware accelerators in heterogeneous computing. In this document, the "hardware accelerators" are built on FPGA.

A complete accelerator has software part (APP, or Applications) running on CPU Processor and the hardware part (AFU, Acceleration Function Unit) running on FPGA chip. APP and AFU are sharing host memory, that means, they both can read and write the 2^64 range of virtual memory address. To make it happen, CAPI technology has a CAPP (Coherent Acceleration Processor Proxy) logic unit in Processor chip, and also needs a PSL (Processor Service Layer) logic unit in FPGA chip. For CAPI1.0 and CAPI2.0, the interconnection between processor and FPGA is using PCIe physical links and PCIe form factor. CAPI1.0 uses PCIe Gen3x8. CAPI2.0 uses PCIe Gen4x8 or Gen3x16. OpenCAPI is not covered in this document. Please check https://opencapi.org for more information.

Let's focus on the FPGA side. A customer FPGA card needs to have a PSL module (Processor Service Interface) to become a "CAPI-enabled" card. This PSL module is provided by OpenPower Foundation and is an IBM IP. For CAPI1.0, PSL module and the surrounding board specific modules are provided in the form of a routed dcp file (Xilinx Vivado design checkpoint). It's usually called b_route_design.dcp. For CAPI2.0, PSL is an IP package with encrypted source code. It's named like ibm.com_CAPI_PSL9_WRAP_2.00.zip. They can be downloaded at https://www.ibm.com/systems/power/openpower. From the menu, select "CAPI","Coherent Accelerator Processor Interface (CAPI)" or directly click the "CAPI" icon to go to the CAPI section. Then download the appropriate files depending on your target system being POWER8 (CAPI 1.0) or POWER9 (CAPI 2.0). You need to register an IBM ID to download them. For a new FPGA card, if you want to enable CAPI on it, it simply means to create a board supporting package which includes the PSL module onto the FPGA and let it work. There are two levels: HDK and SNAP.

For HDK, a project from FPGA Vendors (i.e, a Xilinx Vivado project) which is composed of BSP (Board Supporting Package, containing PSL module) and sample user logic (AFU), is delivered to acceleration developers. This project is called HDK (Hardware Development Kit).

The developers working on HDK level need to know the details about PSL interface specifications and write Verilog/VHDL logic to interact to it. Please refer to CAPI1.0 PSL Spec and CAPI2.0 PSL Spec or search "PSL/AFU interface" in your web browser. As a full development environment, you also need SDK (Software Development Kit) which contains the example application software code and PSLSE (PSL Simulation Engine) for a software-hardware together simulation to guarantee the correctness of accelerator design. HDK provides the maximum available FPGA resource area and the shortest latency. However, we recommend developers to work on SNAP because SNAP simplifies the developing work significantly.

SNAP is the abbreviation of Storage, Networking and Analytics Programming. It is an open-source acceleration development framework https://github.com/open-power/snap. On the FPGA side, SNAP framework adds a PSL/AXI bridge, a DDR SDRAM controller and an optional NVMe controller. Thus, the developer can focus on their acceleration kernel logic (here we call it hardware action) and interface the framework via several AXI ports.

This document focus on CAPI2.0. For CAPI1.0 enablement, the BSP part is a little different, please contact an IBM representative for more information. The SNAP part is the same. In following chapters, we introduce how to: Enable BSP Enable SNAP We assume the reader knows how to work on Vivado Project and SNAP already. You can find many materials on how to develop an accelerator with SNAP (Training videos, "docs" folder on snap github, or other webpages) so they are not discussed in this document.