TEBF0808 Detailed USB 3 Operation Analysis.

[Kalicutt]

I initially sent these messages to Trenz support, but figure they belong here in order to help out the most people:

I am the primary engineer pursuing the USB 2.0/3.0 function on one of our designs that use your SoM (TE0808-04-9EG-2IE) and am currently working on a design on the evaluation kit (TEBF0808-04).  I am working with the latest release files you guys provide for Vivado 2018.2.  I work on an Ubuntu 16.04 work station (supported by Xilinx for Vivado 2018.2).

I am currently trying to reproduce the behavior of the pre-built Petalinux image (BOOT.BIN and image.ub) provided with the 2018.2 release.  With the release image, I am able to get USB 2.0 and USB 3.0 working perfectly fine, but not when I build the image from source.  These are the steps I followed to create my own image from source:

 1. Ran _create_linux_setup.sh and chose maximum setup
 2. Modified design_basic_settings.sh with my path to Vivado
    installation as well as short ID for the TE0808-04-9EG-2IE (9eg_2ib_sk)
 3. Made sure all tools and files used have proper permissions for
    execution and are owned by the current user
 4. Sourced vivado_create_project_guimode.sh
 5. Ran synthesis, implementation, and bitstream generation.
 6. Exported hardware to SDK with bitstream included
 7. Created FSBL for design (all defaults)
 8. Extracted the config files from the os files on the release (config,
    rootfs_config, and the config.gz containing the kernel config from
    the image, as well as the system-user.dtsi for updated device tree
    settings)
 9. Created a petalinux project under the project.sdk/petabuild folder
    (petalinux-create -t project -n USB3_Test --template zynqMP)
10. Imported hardware I built from the script provided (petalinux-config
    --get-hw-description $path_to_hdf)
11. Loaded project configuration used in release (petalinux-config, then
    use load command)
12. Loaded rootfs configuration used in release (petalinux-config -c
    rootfs, then use load command)
13. Loaded kernel configuration used in release (petalinux-config -c
    kernel, then use load command)
14. Replaced the default
project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi with
    the one provided in the release files
15. Built the project: petalinux-build
16. Packaged the project: petalinux-package --boot --fsbl $path_to_fsbl
    --fpga $path_to_exported_bitstream --uboot --force
17. Copied images/linux/image.ub and images/linux/BOOT.BIN to boot
    partition of properly formatted MicroSD card
18. Booted the hardware using this image built from source

I noticed that instead of proper USB operation, I get the error message: "[   14.187998] usb usb1-port1: connect-debounce failed" and neither USB 2.0 or 3.0 are working, there is no interface to a hard drive that I have inserted into a USB port under /dev/sdX.

Is there something I am missing?  A step I am not doing properly or some customization of the build I am not doing properly that explains this failure?

-------------------------

I have found a few places that there were differences between my build and the pre-built one, for example I added the recipes-kernel folder from the os folder.  The major difference, however, was in the TE modified FSBL, when I added the pre-built FSBL to my image, suddenly USB 3 was working.  I am trying to build the FSBL from source, using the template you guys provide under sw_libs, and while it compiles it does not behave properly when I try to boot using it...  Any idea why?  I also flashed the OTP memory of a SI5345 chip with the same frequency plan as what was discussed in the TE modified FSBL and tried to boot using a default FSBL image, but USB 3 did not work with a message related to dwc3 not being able to find the PHY.  Can you guys shed any light on that?

What I am trying to figure out now, specifically, is if there is some requirement for the SRST_B to be asserted after the silabs chip is programmed due to some timing issue in the Zynq US+ PS boot sequence.  I am also trying to determine what impact, if any, changing the clock plan (not frequency output to USB3 MGT bank) has on performance when reset is still asserted.

[JH]

Hi,

you must use modified FSBL sources to program SI5345 (need for GTP CLKs --> USB3, SATA, PCIe...)

Can you tell me exactly what is not working, if you try to generate the modified FSBL template with SDK?

You can include repository with on of this instructions:

• https://wiki.trenz-electronic.de/display/PD/SDK+Projects#SDKProjects-CreateSDKProjectfromVivado

If you has problems, with the templete on ubuntu, let me know. Alternatively generate default Xilinx FSBL and copy the sources from the template folder into your SDK project folder.


PS: there is also a petalinux template included (./os/petalinux), how to use it it described here:

• https://wiki.trenz-electronic.de/display/PD/PetaLinux+KICKstart#PetaLinuxKICKstart-CreatingaProjectfromVivadoProject

br

John

[Kalicutt]

Hello John,

I used the modified FSBL sources to make the FSBL from a template (TE modified Zynq MPSoC FSBL) on both Ubuntu 16.04 (going through entire design flow) as well as making this component and bitstream on Windows 10 PC.  Nothing works, I am getting stuck right after FSBL during boot.  If I have JTAG connected I am getting a tamper attempt CSU error, otherwise I do not see anything but a 0x4B error for no valid SD image found.

I also tried building the FSBL by just using the default Xilinx template for MPSoC, then modifying the necessary files: Si5345-Registers.h, si534x.c, si534x.h, xfsbl_board.c, xfsbl_board.h, and xfsbl_main.c.  This did not behave any differently.  This is the UART output I see when booting from modified FSBL:

Xilinx Zynq MP First Stage Boot Loader
Release 2018.2   Aug  1 2018  -  09:56:36
--------------------------------------------------------------------------------
TE0808 Board Initialisation
Si534x Init Function
Si534x Init Complete
PCIe Reset Complete
--------------------------------------------------------------------------------

Then it hangs here.

This is different than when I use the pre-built fsbl provided as a binary, that one boots fine.  I also notice there is about a 30 KB difference in size between the file I generate (130 KB) and the pre-built (160 KB) so I have to assume there is something different.

I am very familiar with the Petalinux flow, I have used it with other boards without a problem, and with this one it works fine as long as when I package the BOOT.BIN I use the pre-built FSBL, so I don't think that is the problem.

A question I have is, is it possible to just pre-program the SI5345 with the Silabs field programmer with a correct frequency plan and then skip the modified FSBL?  If so, I have tried that too without any luck, I can boot but then I get a debounce USB error or a PHY not found error.

After some debugging I found that if I commented out this portion:
Status = si534x_i2c_write(0x77, 0x00,  0x10);    // Configure I2C Switch
     if (Status != XFSBL_SUCCESS) {
        xil_printf("Error: Configure I2C Switch\r\n");
        goto END;
     }
     Status = si534x_init(0x69);                      // Configure clocks
     if (Status != XFSBL_SUCCESS) {
        xil_printf("Error: Configure si534x CLK\r\n");
        goto END;
     }
It would at least not freeze up, but this is the part that configures the switch and clocks.  But if I program the SI5345 using OTP memory, why would this not work?

[JH]

Hi,

did you remove compiler flags "-Os -flto -ffat-lto-objects" on 2018.2 SDK to generate FSBL by yourself?

• https://wiki.trenz-electronic.de/display/PD/TE0808+StarterKit#TE0808StarterKit-zynqmp_fsbl

I get problems to generate correct FSBL on WinOS with 2018.2 tools and default compiler flags. Without this compiler Flags it works fine. I think this is the different of the file size. But I got an error on PMU Firmware load process, without this modification.

So I understand this correctly:
1. with prebuilt FSBL it works -> also USB
2. with your FSBL with our modification it freeze after the output you has print.
3. with your FSBL and without our SI5345 I2C configuration it will boot, but without working

Can you add FSBL debug flags and recreate again:

• http://www.wiki.xilinx.com/FSBL

Is there a additional output on the console, before it freeze?

Program OTP of the SI5345 with correct frequency should also works than FSBL SI5345 configuration isn't necessary. One possible reason, that this is not working: Wrong PS GTP Refclk configuration (--> wrong HDF to generate all files). So PS is not configured correctly. Did you use our reference design with the board part configuration, or your own?

br
John

[Kalicutt]

Everything I am using is Trenz source code and projects for configuration, except for the Silabs OTP programming.  I do not believe that source is included in the design files, is it possible to get access to that?  I got the general settings from comments inside Si5345-Registers.c, but I am not sure what the exact design is.

I will try the compiler flag changes ASAP and reply.

[Kalicutt]

Removing these compiler flags (also removed the debug levels) has fixed the issue.  I can now boot the device with the default source-built FSBL TE modified.  USB 3.0/2.0 also works fine.
Next I would like to be able to boot the eval kit with USB functional but without having to flash the Silabs on startup.  So far this has not been possible with any clock projects I have tried.  Is the source project available?  I am attaching the one I created using the comments Si5345-Registers.h for reference.  Our board does not support flashing the SI5345 during boot without modifications to hardware, so this is important for us.  It was assumed to be programmed with a field programmer first.

I did find the information about removing those compiler flags here:
https://wiki.trenz-electronic.de/display/PD/TE0808+StarterKit
but you may want to make more explicit mention of the website in the README of the project.

EDIT: Also would be useful to have the CPLD source code used on the TEBF0808, but barring that, is there a way to know exactly what is happening with the various reset signals at power on?  In xfsbl_board.c there is this comment "/* Set MIO30 to HIGH --> this force cpld to reboot PS after CLK Initialisation on first power up */" what does this mean exactly?  Is SRST_B being asserted after boot?  If so I have never seen it asserted again after boot, and I never see FSBL execute twice, which I would expect to happen if it was reset.

[Kalicutt]

Update: I was able to modify Si5345_Registers.h to follow the clockbuilder plan I uploaded with my last reply, and I am still able to use USB 3.  Seems that removes the possibility it is the clock plan itself.  So I am left with some sort of strange timing problem with the resets and clocks coming up, or with Silabs chip not working properly if not programmed over I2C first...

I programmed the SoM over the J17 on TEBF0808 with the Silabs Field Programmer without any error, and have verified that I get the proper frequency output on the TE0808 in the past, so I am still quite confused.

[JH]

Very strange.
Both should work, with I2C or NVM configuration.

To your last post:
1. SI Clock builder Project is included: https://wiki.trenz-electronic.de/display/PD/TE0808+StarterKit#TE0808StarterKit-AdditionalSoftware
2. MIO30 reboot is for PCIe, First time system will be reboot 2 time to get PCIe --> SI5345 clk is not fast enough available over I2C. I will implement a better solution in the future
3. CPLD Source Code of the carrier is available:

• https://wiki.trenz-electronic.de/display/PD/TEBF0808+CPLD+Firmware
• https://shop.trenz-electronic.de/en/Download/?path=Trenz_Electronic/Modules_and_Module_Carriers/5.2x7.6/5.2x7.6_Carriers/TEBF0808/REV04/Firmware

br
John

[Kalicutt]

I will take a detailed look at the CPLD to see if there is anything there that I may be misunderstanding.  I will also try flashing the OTP on a fresh SoM with this exact source build you provided and see if I can skip programming.
I do not believe it will work, but I will just make 100% sure.
Is it possible for you guys to reproduce this issue?  Use a TEBF0808-04 (in PC mount) with TE0808-04-9EG-2IE, program Silabs, and skip I2C flash during FSBL, then check USB functionality?

EDIT: Tried the above step programming the default image found under misc folder, still get the same USB problem ("dwc3-of-simple ff9d0000.usb0: dwc3_simple_set_phydata: Can't find usb3-phy")
Can you describe more about the item 2 you mention?  What is the problem with PCIe with the SI5345 clock?  It may be related to this USB 3 problem.

[Kalicutt]

I updated both CPLD, master and slave, to rev 6 firmware to make sure everything is up to date, but I do not notice any difference in behavior.
I also looked through the CPLD firmware and saw that it seems like MIO30 going low should somehow trigger the master reset (include POR_B assertion) but I am not seeing that happen in my system.  If that was occurring I should see two FSBL executions on power on, correct?
I also probed these signals to be sure, I attached those.

I tried using switch S2 in order to reset the board manually after first boot sequence, after Linux was done booting.  This did not change the behavior or USB, it still reports the same error.
I attached the entire boot log for reference.

[Kalicutt]

I did some probing on the TE0808 while installed in TEBF0808, I am noticing that when I skip the I2C programming of the Silabs chip from FSBL, I do not see any clock output on clk 5 ( b505 clk 2 which is used for USB 3).  This is with the default image I grabbed from the misc folder in the 2018.2 design files downloaded to the TE0808 via Silabs field programmer.
There is no difference in the signals for configuration (PLL_SEL0, PLL_SEL1, RST), and I confirmed both the 25 MHz and 50 MHz inputs to the SI5345 are operational.  The only difference I notice besides no output clock, is that LOL is asserted (low) when I skip the I2C program.  I am not sure yet why this is, but it seems for some reason the Silabs is not able to properly initialize when not programmed via I2C.

I have some idea on how to diagnose this by checking some registers on the device (without programming it).  I will look into this, but any tips on why this is happening would be appreciated.

[JH]

Hi,

  This is with the default image I grabbed from the misc folder in the 2018.2 design files downloaded to the TE0808 via Silabs field programmer.

Are you sure, that you didn't modified the project?
I've checked

• StarterKit\misc\Si5345\Si5345-RevB-0808-02A-Registers.h
• StarterKit\sw_lib\sw_apps\zynqmp_fsbl\srcSi5345-Registers.h

It's the same. Also 27MHz for output 5 is enabled in the StarterKit\misc\Si5345\Si5345-RevB-0808-02A-Project.slabtimeproj project, which is used to generate the header file.

I've the header files works, also. You can also temporary program SI with SILab Programmer instead of NVM. Restart System with FSBL without SI configuration. Can you try out this please?
Can you tell me also you SI Clock Builder Pro Version.

I've programmed SI5345 NVM on other module series without problems. But I will try out programming also on a TE0808.

br
John
 

[Kalicutt]

Hey John,

My clockbuilder pro software version is 2.26.0.1.
I do not see an option to program NVM vs programming volatile memory, when I use the tool it just mentions NVM programming.
When I start the starter kit with TE0808 without programming SI5345 from FSBL, I do not get any clock outputs whatsoever.  I programmed the default project under misc folder to this SoM that I am testing before, however.

I wish to confirm that the NVM was written to by reading the available memory count from registers, and also query some initialization parameters, but I need to write some software for that.  You guys don't happen to have something to help out there?

[JH]

Hi,

I do not see an option to program NVM vs programming volatile memory, when I use the tool it just mentions NVM programming.

see:

• https://www.silabs.com/documents/public/user-guides/ug286-clockbuilderpro-dongle.pdf

page 20.  You see it when you open the clock builder project file and the field programmer is connected.

br
John

[Kalicutt]

Hey John,

Yes this is the way that I flashed the OTP on the TE0808.  What I mean to say is it does not let you choose between flashing non volatile vs volatile memory, it makes no mention of this being a temporary flash.

I'll try using the NVM burn tool instead to see if that is the issue.

[JH]

Hi,

i do not understand. Page 20 (Figure 4.22) is configuration of volatile memory(nearly the same like over I2C from FSBL), page 8 figure 4.5 is configuration of non volatile memory. You can not switch, this is a little bit strange. An on NVM programming (I think it's only 2 times possible on SI5345), you must select again to correct project.

So you programmed volatile memory until now?

br
John

[Kalicutt]

Yes it seems I have been programming volatile memory only, and so there is a chance this entire problem has been due to this misunderstanding...
I will get back to you on the USB 3 problem if I see it with this fix.

[Kalicutt]

I programmed NVM on SI5345 with default Trenz clock project, then used the Xilinx default FSBL, and I am able to get USB 3 functionality.  Thank you for helping me along.