QDMA Subsystem for PCI Express

After selecting a device and outlining your design, follow these steps to get started.

1. Choose your IP Solution

AMD offers multiple solutions:

Integrated Block

This solution provides the highest level of design flexibility to customer architecture, with the drawback of requiring the most back-end work. If you are looking to streamline a high-performance solution and want a high level of control of operations, choose this IP. Once at the stream interface, use the descriptors (packetized formats for these interfaces) to interact with the data and form responses or process the responses. This solution is available in Endpoint or Root Port mode.

Bridge

This solution is a combination of the Integrated Block for PCI Express® and the Bridge. It is the most complex option and provides the most prebuilt functionality. It includes a full DMA engine on the backside of the Integrated Block. This can be quickly and easily set up and controlled from the host system, but initiates the transactions from the Endpoint. There are two directional channels – Host2Card (Endpoint initiates MemRd requests to host to pull data to the card) and Card2Host (Endpoint initiates MemWr requests to push data to the host). The solution also includes the bypass port, which is a version of the Bridge that allows the host to directly transact with an AXI4 device behind the DMA/Bridge.  This solution is available in Endpoint or Root Port mode.

DMA

The DMA Subsystem for PCI Express includes the Integrated Block for PCI Express and has an AXI4-Stream to AXI4 memory mapping bridge. This easy-to-use solution has an AXI subsystem and interconnection scheme. The IP takes in the descriptors from the four functional interfaces in the Integrated Block, and translates these descriptors into AXI4-compliant interactions – a Master bridge for the CQ-CC, and the Slave bridge for the RQ-RC.  This solution is only available in Endpoint mode.

QDMA

The QDMA Subsystem for PCI Express implements a high performance, configurable Scatter Gather DMA for use with the Integrated Block for PCI Express. The IP provides an optional AXI4-MM or AXI4-Stream user interface. The QDMA solution provides support for multiple Physical/ Virtual Functions with scalable queues, and is ideal for applications that require small packet performance at low latency. 

2. Configure the IP

For full details on configuring the IP in the Vivado™ Integrated Design Environment, review the Customizing and Generating the Core section in the Design Flow Steps chapter of the IP product guide. See References below.

At a minimum, set the following parameters in the Basic tab in the Customize IP dialog box:

• Lane Width
• Link Width
• GT Location
• Integrated Block Location
• Input Clock Frequency

An example of configuration selections is shown in the UltraScale FPGA Integrated Block Video: How to Create a PCI Express Design https://www.xilinx.com/video/hardware/create-pci-express-design-ultrascale-fpga.html

After the IP has been configured, generate the IP solution.

3. Generate the Example Design for a Demonstration Board

All PCIe® solutions provide an example design. For details, review the Example Design chapter of the IP product guide. See References below.

AMD provides tutorials on bringing up the IP example design on various platforms:

• VCU118 - https://www.xilinx.com/member/forms/download/design-license.html?filename=xtp444-vcu118-pcie-c-2017-4.pdf&cid=bd8ee708-5f26-4a9a-a1c7-f21ec8115cc8
• KCU116 - https://www.xilinx.com/member/forms/download/design-license.html?filename=xtp462-kcu116-pcie-c-2018-1.pdf&cid=552a9d3e-0bd3-4283-a356-cf0b1bff9a2e
• ZCU106 - https://www.xilinx.com/member/forms/download/design-license.html?filename=xtp493-zcu106-ipi-c-2018-1.pdf&cid=2549a7d4-9193-4b54-80be-4e2702e48a6b

For information on how to simulate the example design, see the Simulating the Example Design section in the Example Design chapter in the IP product guide. See References below.

For information on testing or validating the example design on the demonstration board, see the following:

• Check that the PCI Express link comes up. Use LSPCI or a Teledyne LeCroy tool to see which PCI™, PCI Express, chipsets, and others are enumerated in your BIOS. For more details, see:
  • Answer Record about LSPCI
  • Windows LeCroy Tool
     
• Verify that you see the sample traffic on the PCIe interface. If not, see the Debugging chapter of the IP product guide for help.

For DMA applications, AMD provides software drivers.

• For the DMA drivers, see the DMA driver answer record.
• For the QDMA software driver:
  • Vivado 2018.3 and later - https://github.com/Xilinx/dma_ip_drivers
  • Vivado 2018.2 and previous - https://www.xilinx.com/support/answers/70928.html.

4. Port the Example Design to Your Board

Now that you have built the example design and simulated it, you are ready to generate a bitstream and test it out on your board. For information on how to program your FPGA, see the Vivado Design Suite User Guide: Programming and Debugging (UG908).

5. Integration

Now you are ready to integrate the IP into your application. The first step is to integrate your application code.

• Review the example design to see how the user interfaces can be integrated into your design.  See the user interfaces described in the Port Description section of the Product Specification chapter in the IP product guide.
• The goal is to replace the example traffic generator with your application code.

For PCI Express, locate the provided support wrapper (xdma_0_support_i) in the file hierarchy above the XCI file (xdma_0_i) and instantiate the wrapper in your application. This does not include the sample application (xdma_app_i). You will need to provide your own application.  

6. References

The IP product guides associated with the AMD IP solutions for PCIe are as follows:

7 Series FPGAs Integrated Block for PCI Express Product Guide (PG054)

Virtex 7 FPGA Gen3 Integrated Block for PCI Express Product Guide (PG023)

AXI Memory Mapped to PCI Express (PCIe) Gen2 Product Guide (PG055)

UltraScale Devices Gen3 Integrated Block for PCI Express Product Guide (PG156)

UltraScale+ Devices Integrated Block for PCI Express Product Guide (PG213)

AXI Bridge for PCI Express Gen3 Subsystem Product Guide (PG194)

DMA/Bridge Subsystem for PCI Express Product Guide (PG195)

QDMA Subsystem for PCI Express Product Guide (PG302)