Problems running linux kernel 3.13 for TE0720

[amb_kosh]

Hello,

I'm trying to build a working version of github.com/Xilinx/linux-xlnx (master-branch) for the TE0720 (+ TE0701-03). I've managed to rebuild kernel linux-te-3.9, even using changed options. But for the Xilinx-tree I've build several versions of the kernel using different configurations without success - e.g. te_zynq_defconfig from 3.9, xilinx_zynq_defconfig or the kernel patches from Trenz and others. I also tried several devicetrees, in fact I did not test all the combinations of dtb+kernel. They were all building correctly but did not start on the Trenz module:

U-Boot 2013.01-00011-gc260602-dirty (May 16 2014 - 15:39:43)

I2C:   ready
DRAM:  1 GiB
WARNING: Caches not enabled
MMC:   zynq_sdhci: 0
Using default environment

In:    serial
Out:   serial
Err:   serial
Net:   Gem.e000b000
Hit any key to stop autoboot:  0
zynq-uboot> run tftpboot
TFTPing Linux to RAM...
[...]
## Booting kernel from Legacy Image at 03000000 ...
   Image Name:   Linux-3.13.0-xilinx
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3558520 Bytes = 3.4 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 02000000 ...
   Image Name:   Zynq Embedded Linux
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    5848260 Bytes = 5.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02a00000
   Booting using the fdt blob at 0x02a00000
   Loading Kernel Image ... OK
OK
   Loading Ramdisk to 1fa6c000, end 1ffffcc4 ... OK
   Loading Device Tree to 1fa67000, end 1fa6bc54 ... OK

Starting kernel ...

The kernel with the xilinx_zynq_defconfig did indeed load and start on a ZedBoard (with its own dtb) but not on the Trenz board. As you might see, I also rebuild U-Boot, which is working correctly.
Does someone has a clue what I am missing?

regards

Jörg

[Oleksandr Kiyenko]

Hello,
There is some bug in xilinx clock manager driver which cause boot problems with some system configurations. In this example preblem occur when UART1 is not exist in DTB. If you add uart1 (which not really used as TE0720 use UART0 by default) system should boot. For example you can take my DTB from https://github.com/Trenz-Electronic/TE0720-GigaZee-Reference-Designs/tree/master/TE0720-01_TE0701-03_Base_Vivado-2014.1/ready_for_download
This repository use latest kernel from xilinx and boot ok on TE0720-01 + TE0701-03

Best regards
Oleksandr Kiyenko

[amb_kosh]

Hello Oleksandr,

there seems to be more to this issue, than the devicetree. I checked the zynq_gigazee.dts that I was using and found two uarts in there.
And I did an investigation on the startup behaviour using your files and mine.

file	yours	mine
boot.bin	1	4
uImage (kernel 3.13)	2	5
devicetree	3	6

Now I tried all possible cominations regarding the startup behaviour, the results are:

123	OK
126	NOK
153	OK
156	NOK
423	NOK
426	NOK
453	NOK
456	NOK

So at least I could exchange the kernel but in future I will also have to use my own bootloader. On the other hand I could use my bootloader for starting kernel linux-te-3.9. btw: my goal is to build a kernel 3.14-rt

regards

Jörg

[Oleksandr Kiyenko]

Hello Jörg,
Which vivado version are you use? There is some changes in latest vivado which can affect to boot. For example UART base frequency and interrupt numering is differ than in previous versions.
If you use latest 2014.1 vivado, better take my reference project https://github.com/Trenz-Electronic/TE0720-GigaZee-Reference-Designs/tree/master/TE0720-01_TE0701-03_Base_Vivado-2014.1 so this way we will have the same settings and it will be easily to work with your issues. I also have interest to build RT kernel so it will be good if you can share your work details.
Which repository for kernel are you use ?

Best regards
Oleksandr

[amb_kosh]

Up till now I am using SDK 2013.4 on Ubuntu 12.04 (but this could be changed quickly). I'm not using Vivado as I don't build any FPGA code, this is done by a colleague. To build my own boot.bin I have used the FSBL and bitstream file by Trenz and just replaced U-Boot (with the one from the Xilinx repo)
For the rt-kernel... I cloned the Xilinx kernel repo (branch master-next), applied the RT_PREEMPT_PATCH and build the kernel, so far without complications. Then I tried to start the kernel on the Trenz board...

Is it advisable to install the new SDK and regenerate all involved code except the bitstream (FSBL, u-boot, boot.bin, kernel, devicetree)? If possible I would like to avoid Vivado.

[amb_kosh]

Hello Oleksandr,

I now have installed Vivado 2014.1 (on Linux). I took your project and generated the bitstream, build the FSBL and a boot.bin.
Unfortunately with this boot.bin - I think it's related to the FSBL - the board isn't booting at all, so I'm not seeing any output. Which seems to me that even U-Boot isn't coming up.

regards
Jörg

[Oleksandr Kiyenko]

Hello Jörg,

Unfortunatelly I did't work with linux version of vivado yet, so it's hard to say what was wrong, but most probably FSBL was incorrectly generated.
Actually, to work only with linux you can use precompiled files to generate boot.bin. I can  send you my FSBL.elf and bit file if you want.

Best regards
Oleksandr

[amb_kosh]

Hello Oleksandr,

now I've managed to build an FSBL by myself (Windows + Linux) - The problem with Linux was presumably the project position on a vbox shared drive.
The problem I have now is with U-Boot, because with my new FSBL my own U-Boot, which worked flawless up till now, just prints out invalid characters. This rather looks like a wrong baudrate setting. It seems to be interactive, but I can't really tell obviously.

regards
Jörg

[Oleksandr Kiyenko]

Hi Jörg

Vivado 2014.1 generate FSBL with UART base frequency 100 MHz, instead of 50 MHz in previous versions.
You should change u-boot UART settings and rebuild u-boot.

Best regards
Oleksandr

[amb_kosh]

So now I'm kind of back at the starting point. FSBL, bitstream and U-Boot are build by myself but now it seems I'm unable to start any kernel. Neither my own nor yours and also the devicetree seems to make no difference.
I've used zynq-gigazee.dts from linux-te-3.9 for building a dtb for linux-xlinx-3.13. And I've also changed uart-clock to the new value (dont know if this was neccessary) but this didn't help either. I'm still (or again) seeing no output from the linux kernel.

[amb_kosh]

Hello Oleksandr,

for a sophisticated failure search I think it would be helpful to have your FSBL and bitstream. As it seems that I'm unable to build a working FSBL this perhaps gets me closer to a solution.

Thank you

Jörg

[Oleksandr Kiyenko]

Hi Jörg,
Files is to big to attach. Give me your e-mail or contact me a.kienko(at)trenz-electronic.de

Best regards
Oleksandr

[amb_kosh]

Hello Oleksandr,

for your interest, the problem was caused by vivado. Although I regenerated the bitstream, the second UART wasn't included. In vivado I just had to open the block design and open the settings for the second uart and close it right away. After regenerating the bitstream the second uart was included in the system.xml and then at last the SDK built a working FSBL.

Thank you

Jörg

[mica]

Hello,

I have pretty much the same problem as Jörg. My hardware is nearly the same TE0720-02 (+ TE0701-03). I was able to build the linux kernel from the Trenz git repository linux-te-3.9. I used TE0720-01-2IF.dts from that repository. With that the board bootet successfully from sdcard.

Now I'm trying to upgrade to Kernel 3.14, using the Xilinx git repository (https://github.com/Xilinx/linux-xlnx), master branch. Problem is, there is no TE0720-01-2IF.dts available in that repository. So I am using TE0720-01-2IF.dts from the Trenz git repository linux-te-3.9 instead. The kernel and the devicetree.dtb are built successfully. But they don't boot. Last message I see in the terminal is from uboot saying "Starting kernel ...". No more messages appear.

In both cases I use uboot 2013.01 built from the Trenz git repository, and the FSBL from example "TE0720-01-Base" (http://www.trenz-electronic.de/de/download/d0/Trenz_Electronic/d1/TE0720-GigaZee/d2/reference_designs.html). I don't use custom FPGA logic right now.

What is going wrong? I don't really understand that from the previous posts here in the forum.
Why is TE0720-01-2IF.dts not working with Kernel 3.14?
How does that relate to the UART base frequency and interrupt numbering?

Right now I am not able to generate my own dts file, because I don't have Vivado to create a BSP. I only have Xilinx SDK 2014.2 for Windows.
Is there a reference TE0720-01-2IF.dts available for Kernel 3.14? I would like to try that.

My goal is same as Jörg's: Build a kernel 3.14-rt using RT_PREEMPT_PATCH.

Thank you,
Michael

[Oleksandr Kiyenko]

Hello Michael,

DTS for kernel 3.13, 3.14 have a lot of differences from 3.9. You can't use this DTS for new kernels. You can use dtb from
https://github.com/Trenz-Electronic/TE0720-GigaZee-Reference-Designs/tree/master/TE0720-01_Base_Vivado-2014.1/ready_for_download
Also I attach DTS to this post.


Best regards
Oleksandr Kiyenko

[mica]

Hello Oleksandr,

your example project "TE0720-01_Base_Vivado-2014.1/ready_for_download" is working on my board.

Then I tried the dtb from that example with my kernel 3.14 and root-fs. That doesn't work.
Next I tried the dts (attached to your previous post). I rebuilt dtb from that and tested with my kernel 3.14 and root-fs. That also doesn't work.

Looks like my kernel 3.14 + root-fs is not running. But I have no idea why. There is no output to the terminal from the kernel. No ethernet, too. So I am kind of stuck now.

Any idea?

When building kernel 3.9 from the Trenz repository, I use kernel defconfig "te_zynq_defconfig".
When building kernel 3.14 from the Xilinx repository, I use kernel defconfig "xilinx_zynq_defconfig".
Might this be the reason? What defconfig do you use for your kernel 3.13/3.14?

Kind Regards,
Michael

[Oleksandr Kiyenko]

Hello Michael,

Rootfs is not so important on begining (during initial kernel load), it important from moment when kernel mount it.
As for me to debug your situation with kernel, I see steps:
1) Build new kernel
start from
make ARCH=arm xilinx_zynq_defconfig
and then via
make ARCH=arm menuconfig
edit kernel paramethers to add to kernel blocks used in your devicetree
RTC, I2C GPIO switch, MEMs (if some TE0720/TE0703 peripherials are not used it should be removed from device tree)
2) Build DTB from my DTS (atteched to my previous post).
3) Try to run linux with this set of files
4) If it working you can try to patch kernel

Best regards
Oleksandr

[danwwright]

I too have been struggling with this same issue but just now discovered this thread.

Would it be possible for you to post the kernel config file and dts that have been shown to boot successfully when using the xilinx linux kernel git?

[Oleksandr Kiyenko]

Hello,

I already post dts in this thread, so I attach only my current kernel config.

Best regards
Oleksandr Kiyenko

[daveo]

Another 'me too' post with the system hanging at "Starting kernel ...".

I followed the instructions at https://wiki.trenz-electronic.de/display/TE0720/Xilinx+repository and compiled a kernel based on the 'xlnx_3.14' branch of linux-xlnx.

Then I applied the u-boot patch against master-next of u-boot-xlnx, and modified accordingly.  [But without knowing how to generate a boot.bin outside of Vivado, I have not yet tried running this u-boot image -- any tips here?]

I've tried various combinations of boot.bin, devicetree.dtb, uImage, and uramdisk.tar.gz with no conclusive results yet, other than that the Trenz-provided images work and if I use my own uImage it hangs.

Part of me wonders whether this is a failure of getting the kernel command line arguments.  Since both the console output and the default IP address is set via the cmdline, if that is messed up I'm not sure what would happen.

Best regards,
Dave Oostdyk

[danwwright]

Thanks- but it looks like the dts posted is for the te0720-02 and I have the te0720-01... should there be any dts differences?

[Oleksandr Kiyenko]

Hello daveo,
I am not sure that my patch for old version of kernel and u-boot can be used for 3.14. To test your project you can take your kernel with some minimal dts (just change console settings to uart0 in some of example zynq dts and remove not used peripherals).
Best way to debug is take precompiled kernel devicetree and ramdisk and replace it one by one by your own.

Best regards
Oleksandr Kiyenko.

[daveo]

I seem to be missing just enough kernel sources and/or history to accomplish my goal.

1.  The linux-te-3.9 repository builds a kernel that I am able to boot successfully.  This could serve as a useful basis for moving forward, except this repository was created as a 'dump' of a kernel checkout, and it isn't clear to me _which_ checkout formed the basis of the linux-te-3.9 repository.

2.  The link, https://github.com/Trenz-Electronic/TE0720-GigaZee-Reference-Designs/tree/master/TE0720-01_TE0701-03_Base_Vivado-2014.1/ready_for_download contains a 3.12.0 kernel image which I can boot, but I haven't been able to find either sources or a repository with this kernel version.

3.  The linux-xlnx kernel compiles, yet I haven't been able to successfully boot it; the system hangs at "Starting Linux ...".  Is there a particular branch of this kernel that is recommended for the TE0720?

4.  Out of curiosity I gave the 'linux-stable' kernel a try, using some of the .dts files found in linux-xlnx.  Same situation with hanging at boot as for the linux-xlnx image.

My primary goal for the time being is to fix the clock precision of clock_gettime() and associated functions.  There are a number of threads on the topic, such as http://forums.xilinx.com/t5/Embedded-Linux/Zynq-kernel-debugging-using-FTRACE-timestamp-resolution/m-p/414533#M7888%29...  It seems that Kernel 3.12 _should_ have the CONFIG_ARM_GLOBAL_TIMER option available, but it doesn't seem to be present in the (binary) 3.12.0 image posted earlier in this forum.  The linux-stable and linux-xlnx have this option but of course cannot seem to boot.

Therefore my questions are:

* How to get a booting kernel image that has CONFIG_ARM_GLOBAL_TIMER enabled?

* Which devicetree image should I be using for the TE0720-01-2IF?
  - TE0720-01-2IF.dts  (boots)
  - TE0720-01-xxF.dts  (does not boot; is this newer than TE0720-01-2IF.dts?)
  - TE0720-02.dts      (does this apply to the -01-2IF?)
  - One of the various zynq-*.dts files from linux-xlnx?

* Do any of the .dts files that work with the TE0720 actually define ps7_globaltimer_0 as discussed in the Xilinx forum linked above?

Thank you for any help.

Dave Oostdyk

[Oleksandr Kiyenko]

Hello daveo,

My last 3.13 kernel from xilinx repository
commit f27f400f43062b28d2b6f0977e50492b851d7464
Date:   Mon Apr 7 10:30:15 2014 -0700
Think that it should have options you need.
I attach kernel config and devicetree.
It's not the latest revision as now we working on petalinux release for this board.

Best regards
Oleksandr Kiyenko

[daveo]

Hi Oleksandr,

Thank you for sharing your exact kernel version, config, and devicetree.  That helps eliminate some variables with testing!

Unfortunately I still get the problem of hanging at "Starting Linux ..." even with this kernel.  I've tried three different arm coss-compile toolchains to rule out any issues there (Vivado 2013.4, buildroot, and gentoo-crossdev).  I'm currently testing with boot.bin and uramdisk.tar.gz from taken from your TE0720-01_Base_Vivado-2014.1/ready_for_download reference design.

Any suggestions?

Thanks for the help thus far!

Dave O.

[Oleksandr Kiyenko]

Hi Dave,

Ususally problem with kernel start is related to memory. There is some check list
1) Check that your FSBL have correct DDR settongs (Run memory test from SDK)
2) Check memory settings in u-boot (u-boot should be configured to use same amount of memory than linux kernel or less)
3) Check that right amount of memory is configured in your device tree
4) Check that your device tree is match your hardware (device drivers configured in device tree but not existed in hardware can also result such problem)
5) Check that your system FSBL/u-boot/device-tree is configured to use UART0 (otherwise kernel boot messages will be sent to non-existent UART)
6) In your hardware project should be enabled UART0 and timer.

Better to start from working project like https://github.com/Trenz-Electronic/TE0720-GigaZee-Reference-Designs/tree/master/TE0720-01_Base_Vivado-2014.1
and sytart modify/replace system files to your own.

Best regards
Oleksandr Kiyenko

[danwwright]

I too have been stuck at the same point for a while now and decided to wait for a code update.

Now that Vivado 2014.3 is out will there soon be updated code from Trenz to work with this new Xilinx release (such as a new 2014.3 bsp)?

[Antti Lukats]

hi

unfortunately maybe problems-delay: Vivado/SDK 2014.3 are released, but not petalinux or device tree generation for new vivado, so currently it is NOT possible to use Vivado 2014.3 for linux development. We are waiting for Xilinx response on this issue.

[danwwright]

Now that Vivado 2014.4 and Petalinux 2014.4 are both out will there soon be a matched set of bsp, Xilinx repository Linux config file and device tree dts from Trenz to permit the latest Xilinx Linux kernel to boot correctly on the te0720-1?

[Thorsten Trenz]

Hi,
yes. we are working on it right now. We will announce it here, as soon as it is online.

Best Regards
Thorsten Trenz

[danwwright]

Great!  And as an unrelated question are there any plans for future Trenz GigaZee modules with other Zynq chip variants (like the Zynq 7z015, 7z030, 7z045 etc.)?  We're looking for a future module with a Zynq chip variant that supports multi-gigabit serial transceivers (if the current module connectors have sufficient signal integrity to supports these).

[danwwright]

From the Zynq-7000 All Programmable SoC Technical Reference Manual:
"Low-power gigabit transceivers (7z015, 7z030, 7z045, and 7z100 devices)..."

[Antti Lukats]

7015, 7030 module, compatible to TE0720 is coming, yes.

[danwwright]

Great!  When?  Any very rough timeline you could give would be helpful.

[Thorsten Trenz]

He,
we expect protos end of January.

Best Regards
Thorsten Trenz

[danwwright]

Any progress or release timeline for the matched set of bsp, Xilinx repository Linux config file and device tree dts from Trenz to permit the latest Xilinx Linux kernel to boot correctly on the te0720-1 (all of this for the 2014.4 releases of Vivado and PetaLinux)?

[Antti Lukats]

YES. Most everything seems to be OK.

1) flashcp has problems whem image to write crosses 16Mbyte boundary, not sure if this is new issue or old.

Release very soon.

[danwwright]

Great!  Is the problem just with the flashcp utility or is general reading of files from flash by the OS affected too?

[Antti Lukats]

no issues reading, also when read crosses the 16MB boundary. seems a flaw with flashcp or some driver-setting. We had to workaround in one project to place the image.ub above 16MB to prevent the partition to span both halfs of the flash.

[danwwright]

I had earlier asked about the matched set of bsp, Xilinx repository Linux config file and device tree dts from Trenz to permit the latest Xilinx Linux kernel to boot correctly on the te0720-1 (all of this for the 2014.4 releases of Vivado and PetaLinux).  Has this now been released or has it been delayed?

[Antti Lukats]

http://www.trenz-electronic.de/de/download/d0/Trenz_Electronic/d1/TE0720-GigaZee/d2/Test_Designs/d3/TE0720-02-GPIO.html

GPIO BSP, it has UART0 on MIO14, MIO15 all other B2B pins connected to GPIO devices

[danwwright]

Excellent!  Does this code support the te0720-1 too or just the te0720-2?

[Antti Lukats]

It is ok for -01 too

[lcn]

After read all th post seems that I'm facing same problem than the author and daveo comments.

I followed the instructions at https://wiki.trenz-electronic.de/display/TE0720/Xilinx+repository , compiled a kernel based on git://github.com/Xilinx/u-boot-xlnx.git and  git://github.com/Xilinx/linux-xlnx.git xilinx repositories

Then I applied the u-boot patch against master-next of u-boot-xlnx and  linux-xlnx (there is no support for TE0720 from original sources) and get the same result:

U-Boot 2013.01-00011-gc260602-dirty (Feb 10 2015 - 18:53:45)

I2C:   ready
DRAM:  1 GiB
WARNING: Caches not enabled
MMC:   zynq_sdhci: 0
Using default environment

In:    serial
Out:   serial
Err:   serial
Net:   Gem.e000b000
Hit any key to stop autoboot:  0
Trenz-Electronic TE0720-02-2EF 0.03
Copying Linux from SD to RAM...
reading uImage
3356936 bytes read in 561 ms (5.7 MiB/s)
reading devicetree.dtb
9028 bytes read in 18 ms (489.3 KiB/s)
## Booting kernel from Legacy Image at 03000000 ...
   Image Name:   Linux-3.18.0-xilinx-gde831db
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3356872 Bytes = 3.2 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02a00000
   Booting using the fdt blob at 0x02a00000
   Loading Kernel Image ... OK
OK
   Loading Device Tree to 1fffa000, end 1ffff343 ... OK

Starting kernel ...

Petalinux aside (not an option for me), is not yet released  from Trenz  support to permit to use the latest Xilinx Linux kernel to boot correctly on the te0720??

Thanks

[Antti Lukats]

petalinux is the LATEST kernerl, it all works with it

petalinux = yocto 1.7 and kernel 3.18

if you can not work witjh petalinux, you can install petalinux ONCE and copy the settings from there

[lcn]

Thank you for your quick reply, I'll try the solution that you propose me .

Surely, I won't work with Petalinux and I suppose there are other people at same situation. It would be so kind if Trenz could release a patch for git://github.com/Xilinx/u-boot-xlnx.git and  git://github.com/Xilinx/linux-xlnx.git xilinx repositories or a short tutorial for guide in  the copy of TE0720 settings that are included only in Petalinux.

Thanks

[Antti Lukats]

with petalinux there are NO PATCHES needed.

petalinux is Xilinx OFFICIAL linux so we do all we can do support Xilinx Linux in first place.

We will have plain yocto based linux version also soon, but for the moment we focus on supporting the main linux that is choosen by Xilinx for Zynq and Microblaze: petalinux

[lcn]

I have been working  on several projects with Zedboard, Zybo and ZC706 with this workflow dowloading sources from U-boot and Kernel from repositories, all these years without Petalinux and Linaro at my rootfs, always very happy and everything  OK.

We have a project that currently works fine at Zedboard with these workflow, interfacing a custom FMC board.  All drivers for IC integrated in our board are fully supported with xilinx kernel, and packages and libraries supported at Linaro.

We are porting it to TE0720 because need an industrial platform for our product, but it is a worth of time to port ALL our libraries that have packages at Linaro to Petalinux (same with drivers). I really don't have a need to use Petalinux until now.

[Antti Lukats]

please We do understand you do not want use petalinux. It is clear.
There is no need to either.

But petalinux is basically nothing more than:

yocto + linux

there is no special linux with petalinux, it is xilinx-linux
so as TE0720 works with petalinux (that uses xilinx-linux) we assume it works equally well with xilinx-linux without petalinux (so not yocto, etc)

asaik there should be no issues using TE0720 with recent kernel without petalinux, maybe I am mistaken.

[lcn]

Yes, it is clear that if TE0720 works with Petalinux it should work with current xilinx-linux founded at xilinx repositories, the only thing I need is a proper config for TE0720. It would be so kind and easy for user if Trenz provides these files (or patches) without the need of install whole Petalinux Tools!

I think Trenz provides a fantastic support and there is available a lot of material (HW, SW, reference designs, etc) and also patches availables for older version of xilinx-linux and u-boot-xlnx, that's why I don't understand the lack of support for this workflow (as you said in previous post it should be an easy thing)