TE0726 - SDK debugging not working 90% of times

[eustaquio]

Hi.

I bought a ZynqBerry TE0726, and after some days of playing with the default installed image, I started to experiment myself using my usual workflow in other development boards.

I generate a simple PS System, and export it to SDK. Then I generate the BSP and write a simple xil_printf("Hello World") program to run on baremetal ARM. The FPGA  configuration from both vivado hw_server and SDK seems to work properly, and I can see the die temperature from a Vivado Hardware Session.  SDK works properly and states that XSDB server starts succesfully.

The problem is that when I try to run or debug the system, I get no response from the board and a not working SDK debug system on 90% of times, and I am randomly successful the other 10%. Here is a list of things that I have desperately tried to do on SDK.

- I started with the original flash program, both leds switched off, UART properly configured.
- I pressed program FPGA button, the leds blinks for a wile and the programming seems to be successful. I have sometimes a trash output on the UART, but other times not. After this, the leds remain switched off.
- I right click on my application, Run As [Debug As sometimes]-> Launch on Hardware (GDB). I have previously configured the runs to "Reset the Entire System" and "Program FPGA".
- SDK do not complain, but 90% of the times, the system does nothing. I can send data through the UART and some leds blink but the expected response not arrives from the board. Breakpoints are present, but never reached in SDK debug view. Debugger says that both cores are running, although nothing happens(as I said, breakpoints are never reached). At this point I can press the terminate button in SDK and try again, although sometimes it complains and says it needs to make a power cycle reset.
- Around 10% of times, one retry is completely succesfull and the program is running correctly: it answers through UART with the expected printf and breakpoints are reached properly on debug view. Once I switch off, I have the same problem.

I have no clue of what is happening, so I tried different things:

- I changed in the hw_server the USB cable speed to very slow options. The FPGA takes more to configure, but the result is the same.
- I thought the flash program might be interfering, so I erased the Flash. The erase was successful and both leds are kept on. When I try now, I have a 100% times freezed debug in SDK. Although when I send data through the UART one led blinks.
- My board was supposed to be a TE0726-02M (this says the box label), although I found that the PCB has a different version printed: TE0726-02. So I regenerated all the project and .bit in Vivado with both target boards to see if I can find a difference, but the result is still the same.
- I downloaded the reference design to try the Hello World example, but the cmd files do not work on my Linux system.

Has anyone an idea of what I am doing wrong?

Best regards.

[Antti Lukats]

a few comments:

1) Xilinx has never managed to get JTAG fully working, they try and try, but do not succeed fully.
2) changing the cable speed would make no difference
3) zynq JTAG is DISABLED after power up or por reset due to security concerns, so if BOOTROM does not timely enable the JTAG, JTAG remains dead and debugging is not possible
4) system debugger is preferred over GDB for debugging
5) TE0726-02 in the article number only PCB revision is shown, the 02M refers to PCB REV 02, full memory assembly option
6) programming the FPGA (that PL portion of the zynq) makes no difference to the PS debugging at all, that is SDK debugger must work no matter if fpga is programmed or not
7) trying to start FSBL in debugger when FSBL was already executed will yield to debugger freeze, this is not a bug this is xilinx security feature
jtag access on windows and linux may have different issues, it can be it works on windows better (or worse)

WARNING: writing a BAD FSBL into flash may prevent further JTAG access completely, if the FSBL freezes then JTAG access remains disabled and further communication is not possible. on Zynqberry recovery is possible if that is the case.

if you just ERASED the flash, then you should be able to write it also.

OK, question: do you see 2 TAPS in vivado hw manager?
Can you program the PL from vivadod hw manager?

in SDK, try:
open XMD window
connect arm hw
rst
rst
xdisconnect -cable

after that try SDK flash programming, use known good FSBL only.

[eustaquio]

Many thanks Antti for your comments and supports. Thanks to you, I was finally able to find a debug way that works every time I try.

The FSBL I was using (automatically generated by SDK) seems to be the problem, since every time I use it it, the debug freezes. Using the prebuilt FSBL provided in the reference design instead of mine solved the problem.

The only workflow I found to make the thing works is the following:

- Develop my application in SDK.
- Generate a Boot Image using the reference design FSBL and my application ELF file.
- Program the Flash with this boot image.
- Right Click on the application project -> Debug As -> Launch on Hardware (System Debugger)

This way I can properly debug the application and use the Breakpoints every time I try.

I have some more questions, however, if someone have the time to answer:

- Why just doing the last step don't work? For example, here http://forum.trenz-electronic.de/index.php/topic,497.0.html a user publish an introduction of OpenCL on ZynqBerry, and never mentions that he needs to program the flash, he says that he just do the last step. If I do that, my new application is never loaded, and while the SDK debug window displays the new code, the Zynq is running the old flashed application (I can see it in the UART outputs, and in the fact that the breakpoints seems to not work).

- I have this output in my console when I flash the FPGA with Vivado:

Enabling extended memory access checks for Zynq.
Writes to reserved memory are not permitted and reads return 0.
To disable this feature, run "debugconfig -memory_access_check disable".

Could this be the problem? Perhaps I am placing the FSLB or the App on addresses where I shouldn't?

- Since the FSBL generated by default by SDK doesn't work, how I create a well "good FSBL" in case I need it?

Once gain, thank you for your help.

Best Regards.

[Antti Lukats]

Many thanks Antti for your comments and supports. Thanks to you, I was finally able to find a debug way that works every time I try.

The FSBL I was using (automatically generated by SDK) seems to be the problem, since every time I use it it, the debug freezes. Using the prebuilt FSBL provided in the reference design instead of mine solved the problem.

The only workflow I found to make the thing works is the following:

- Develop my application in SDK.
- Generate a Boot Image using the reference design FSBL and my application ELF file.
- Program the Flash with this boot image.
- Right Click on the application project -> Debug As -> Launch on Hardware (System Debugger)

This way I can properly debug the application and use the Breakpoints every time I try.

I have some more questions, however, if someone have the time to answer:

- Why just doing the last step don't work? For example, here http://forum.trenz-electronic.de/index.php/topic,497.0.html a user publish an introduction of OpenCL on ZynqBerry, and never mentions that he needs to program the flash, he says that he just do the last step. If I do that, my new application is never loaded, and while the SDK debug window displays the new code, the Zynq is running the old flashed application (I can see it in the UART outputs, and in the fact that the breakpoints seems to not work).

- I have this output in my console when I flash the FPGA with Vivado:

Could this be the problem? Perhaps I am placing the FSLB or the App on addresses where I shouldn't?

- Since the FSBL generated by default by SDK doesn't work, how I create a well "good FSBL" in case I need it?

Once gain, thank you for your help.

Best Regards.

this message comes ALWAYS no matter what zynq board you have. This is not an issue.

What you mean default FSBL does not work? This is the way we do it, from known good Vivado project export HDF and then create FSBL in SDK, and it works. There are nothing special to get this default SDK generated FSBL working. WE may add there some user code, but that code has no related for the basic FSBL boot loading operations.