News:

Attention: For security reasons,please choose a user name *different* from your login name.
Also make sure to choose a secure password and change it regularly.

Main Menu

PCIe Link down

Started by johnabel, April 26, 2020, 04:28:57 PM

Previous topic - Next topic

johnabel


I configured the PS on TE0803 + TE0808 to use PCIe as root to interface an NVMe SSD. When it boots it says 'Link down':
[    4.044563] console [ttyPS0] enabled
[    4.048163] bootconsole [cdns0] disabled
[    4.048163] bootconsole [cdns0] disabled
[    4.056432] of-fpga-region fpga-full: FPGA Region probed
[    4.065364] nwl-pcie fd0e0000.pcie: Link is DOWN
[    4.070015] nwl-pcie fd0e0000.pcie: host bridge /amba/pcie@fd0e0000 ranges:
[    4.076987] nwl-pcie fd0e0000.pcie:   MEM 0xe0000000..0xefffffff -> 0xe0000000
[    4.084206] nwl-pcie fd0e0000.pcie:   MEM 0x600000000..0x7ffffffff -> 0x600000000
[    4.091799] nwl-pcie fd0e0000.pcie: PCI host bridge to bus 0000:00
[    4.097980] pci_bus 0000:00: root bus resource [bus 00-ff]
[    4.103462] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xefffffff]
[    4.110326] pci_bus 0000:00: root bus resource [mem 0x600000000-0x7ffffffff pref]
[    4.120556] pci 0000:00:00.0: BAR 0: assigned [mem 0xe0000000-0xe00fffff]
[    4.127348] pci 0000:00:00.0: PCI bridge to [bus 01-0c]
[    4.132660] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[    4.139073] xilinx-zynqmp-dma fd500000.dma: ZynqMP DMA driver Probe success
[    4.146196] xilinx-zynqmp-dma fd510000.dma: ZynqMP DMA driver Probe success
[    4.153322] xilinx-zynqmp-dma fd520000.dma: ZynqMP DMA driver Probe success

What is strange is that the same configuration (apparently, there aren't many things to configure) works in another board with a similar device (a 3EG).

JH

Hi,
can you try out prebuilt boot.bin  and image.ub of our newest  19.2 reference design:
https://wiki.trenz-electronic.de/display/PD/TE0803+StarterKit
br
John

johnabel

I think I forgot to enable output 0 in the clock chip, as that goes to PCI

johnabel

I created the header file for the Si5338 to output 100 MHz on channel 0 for the PCI, then created an FSBL, copied it to the Petalinux project folder and packaged the boot image with that FSBL, everything else the same, but still says 'Link down'

JH

Does our prebuilt Boot.bin and image.ub from 19.2 works?

br
John

johnabel

I copied UBOOT.bin from   prebuilt\boot_images\4ev_4gb\u-boot
and image.ub from  prebuilt\os\petalinux\4GB
It sees the link up:

[    4.244339] nwl-pcie fd0e0000.pcie: Link is UP
[    4.248823] nwl-pcie fd0e0000.pcie: host bridge /amba/pcie@fd0e0000 ranges:
[    4.255794] nwl-pcie fd0e0000.pcie:   MEM 0xe0000000..0xefffffff -> 0xe0000000
[    4.263014] nwl-pcie fd0e0000.pcie:   MEM 0x600000000..0x7ffffffff -> 0x600000000
[    4.270610] nwl-pcie fd0e0000.pcie: PCI host bridge to bus 0000:00
[    4.276787] pci_bus 0000:00: root bus resource [bus 00-ff]
[    4.282269] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xefffffff]
[    4.289142] pci_bus 0000:00: root bus resource [mem 0x600000000-0x7ffffffff pref]
[    4.298566] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.504 Gb/s with 8 GT/s x4 link)
[    4.313780] pci 0000:00:00.0: BAR 8: assigned [mem 0xe0000000-0xe00fffff]
[    4.320571] pci 0000:01:00.0: BAR 0: assigned [mem 0xe0000000-0xe0003fff 64bit]
[    4.327888] pci 0000:00:00.0: PCI bridge to [bus 01-0c]
[    4.333110] pci 0000:00:00.0:   bridge window [mem 0xe0000000-0xe00fffff]
[    4.340545] xilinx-dpdma fd4c0000.dma: Xilinx DPDMA engine is probed
[    4.347147] xilinx-zynqmp-dma fd500000.dma: ZynqMP DMA driver Probe success
[    4.354271] xilinx-zynqmp-dma fd510000.dma: ZynqMP DMA driver Probe success
[    4.361397] xilinx-zynqmp-dma fd520000.dma: ZynqMP DMA driver Probe success


and lspci sees the device:
root@petalinux:~# lspci -v
00:00.0 PCI bridge: Xilinx Corporation Device d021 (prog-if 00 [Normal decode])
        Flags: fast devsel, IRQ 255
        Bus: primary=00, secondary=01, subordinate=0c, sec-latency=0
        I/O behind bridge: 00000000-00000fff [size=4K]
        Memory behind bridge: e0000000-e00fffff [size=1M]
        Prefetchable memory behind bridge: None
        Capabilities: [40] Power Management version 3
        Capabilities: [60] Express Root Port (Slot-), MSI 00
        Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [10c] Virtual Channel
        Capabilities: [128] Vendor Specific Information: ID=1234 Rev=1 Len=018 <?>

01:00.0 Non-Volatile memory controller: Device 1db2:2302 (rev 03) (prog-if 02 [NVM Express])
        Subsystem: Device 1db2:2302
        Flags: fast devsel, IRQ 255
        Memory at e0000000 (64-bit, non-prefetchable) [disabled] [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [b0] MSI-X: Enable- Count=16 Masked-
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [158] Secondary PCI Express <?>
        Capabilities: [178] Latency Tolerance Reporting
        Capabilities: [180] L1 PM Substates



but it doesn't appear in /dev/  or listed with mount:

root@petalinux:~# ls /dev/n* -l
crw-------    1 root     root       10,  60 Jan  1  1970 /dev/network_latency
crw-------    1 root     root       10,  59 Jan  1  1970 /dev/network_throughput
crw-rw-rw-    1 root     root        1,   3 Jan  1  1970 /dev/null


root@petalinux:~# mount
rootfs on / type rootfs (rw,size=1884476k,nr_inodes=471119)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
devtmpfs on /dev type devtmpfs (rw,relatime,size=1884476k,nr_inodes=471119,mode=755)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /var/volatile type tmpfs (rw,relatime)
/dev/mmcblk1p1 on /run/media/mmcblk1p1 type vfat (rw,relatime,gid=6,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
root@petalinux:~#



JH

you must check linux  drivers...not everything is installed on the reference designs.
br
John


johnabel

I have been trying different options without success... one of the thing that puzzles me is that one of the drives (Samsung) works but not the one I really need to test (ATP).
On the other hand, the ATP works on another FPGA board (MYIR, ultrascale+, similar to TE08003) but I need to test it on the TE0803...
About drivers, I'm not aware ATP needs anything special, not included with Petalinux

JH

Hi,
how did you generate linux for myir board?
Compare  setup (kernel config, rootfs config...) from both boards to find the different.
br
John

johnabel

The Myir linux was built from scratch from the hardware specification. No need to fiddle with the device tree or other exotic stuff, just tweak the kernel and rootfs config. It's worth comparing, I'll do that when back to it.

JH

Hi,
you use the same vivado and petalinux on the TE0803 project like with MYIR Project?
--> when you compare, you should also use the same version. This make it maybe easier to find the different.

br
John





johnabel

I'm back full time to this problem. I used different releases (18.3 and 19.2) for each board. I also compared the device tree files but there wasn't any significant difference (only the gem3 ethernet was in one of them). Now the link is up, but it usually crashes with a kernel panic at boot.

johnabel

All my problems are gone by including both the fsbl and pmufw from the prebuilt folder along with my kernel an image.