AR# 71105

DMA Subsystem for PCI Express (Vivado 2018.1) - MSI Interrupt FIFO can overflow in Root Port configuration in Bridge Mode

Description

Version Found: v4.1

Version Resolved and other Known Issues: (Xilinx Answer 65443)

The MSI Interrupt FIFO in the Bridge Mode of the DMA/Bridge Subsystem is limited to 16 outstanding interrupts at a time. 

If more than this are received from downstream devices, the Interrupt FIFO will overflow.

This can result in lost interrupts from devices and downstream timeouts.

For example:

Downstream device - NVMe Drives - Linux output

[   63.813514] nvme nvme0: I/O 5 QID 0 timeout, completion polled
[  123.877515] nvme nvme0: I/O 5 QID 0 timeout, completion polled
[  184.037513] nvme nvme0: I/O 5 QID 0 timeout, completion polled
[  244.805512] nvme nvme0: I/O 5 QID 0 timeout, completion polled
[  304.869514] nvme nvme0: I/O 5 QID 0 timeout, completion polled

Note: This article is related to (Xilinx Answer 71106)


This article is part of the PCI Express Solution Centre

(Xilinx Answer 34536)Xilinx Solution Center for PCI Express

Solution

In the Vivado 2018.1 release of the IP (version 4.1), a new MSI Interrupt Decode mode has been added to the Root Port Bridge Registers. 

Using a decode register ensures that each MSI Vector assigned to a downstream device has a separate interface output from the core.

This new mode is documented in (PG194) and available in the IP in the 2018.1 release.

To enable the new MSI interrupt Decode, run the following commands:

For IP Catalog Usage:

set_property -dict [list CONFIG.msi_rx_pin_en {true}] [get_ips <ip_name>]

For IPI Board Usage (after configuring the IP to be in AXI Bridge Mode, and Root Port Mode):

set_property -dict [list CONFIG.msi_rx_pin_en {true}] [get_bd_cells <ip_name>]

Two new output ports will be added to the IP:

  1. interrupt_out_msi_vec0to31 (Corresponding to Bridge Register Offset 0x160)
  2. interrupt_out_msi_vec32to63 (Corresponding to Bridge Register Offset 0x164)



Note:

  • The PCI Express Specification sets a limit on the number of available MSI vectors for multi-vector usage. This solution dictates that a total of 64 MSI vectors are available to downstream devices on enumeration.

Revision History:

06/03/2018 - Initial Release

Linked Answer Records

Master Answer Records

Associated Answer Records

AR# 71105
Date 06/05/2018
Status Active
Type Known Issues
IP