Author Topic: SFP+ using TEBF08008 and TE0808  (Read 314 times)

hossam84

  • Active Member
  • *
  • Posts: 20
SFP+ using TEBF08008 and TE0808
« on: December 11, 2019, 07:14:17 AM »
Hello,

I am trying to use the SFP+ port of the TEBF08008 base board using the TE0808 board. I checked the datasheets and schematic of both board and I think I configured the pins accordingly.
I used the GEM0 for the purpose of SFP+ connection. IO used EMIO for the GEM0 (shown in the attachment) and then connected the GMII_ENET0 to gmii_gem_pcs_pma of the 1G/2.5G Ethernet PCS/PMA or SGMII IP. Also, I connected the MDIO_ENET0 to the mdio_pcs_pma of the 1G/2.5G Ethernet PCS/PMA or SGMII IP (shown in the attachment).
For the differential clock of the GT, I used the differential clock of the board as show in the schematic. I used SFP+ of MGT Lane 2 (B230_RX2_P, B230_RX2_N, B230_TX2_P, B230_TX2_N).

I configured the 1G/2.5G Ethernet PCS/PMA or SGMII IP same as the example of Xilinx application note xapp1306 - ps_emio_eth_1g.

After generating bit stream I export the hardware to run the SDK with lwip echo server example. I tried to make any connection with host which has the next-562sfp-10g module but I didn't get anything.

Is there anyone who has tested the SFP(+) who can provide me with some hints or guidlines.

Thanks in advance

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #1 on: December 11, 2019, 09:25:13 AM »
Hello,
did you use passive copper cable or optical cable? --> if optical check supported frequencies.
Did you check that the MGTs REF clock is configured an valid? Which Reference CLK did you use? SI on the carrier is not programmed by default this is done temporary only with our modified FSBL in the reference design. --> it can also happens that you need other reference frequency like we default provide with the reference design
Maybe it's only a problem of Xilinx barmetal app, they run seldom out of the box.

What you can do to at first is to test MGT over SFP: Create Xilinx IBERT core for both SFP with your target frequency and loop back  to the second SFP port on the TEBF0808.
br
John



hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #2 on: December 11, 2019, 10:32:19 AM »
Thanks for your reply.

did you use passive copper cable or optical cable? --> if optical check supported frequencies.
I am using Passive copper Cable Assembly from Molex (https://www.digikey.com/products/en?keywords=74752-2301)

Did you check that the MGTs REF clock is configured an valid? Which Reference CLK did you use? SI on the carrier is not programmed by default this is done temporary only with our modified FSBL in the reference design. --> it can also happens that you need other reference frequency like we default provide with the reference design
Maybe it's only a problem of Xilinx barmetal app, they run seldom out of the box.



I used the reference clock from the board --> B230_CLK0_P, B230_CLK0_N (I checked it from the schematic of TE0808 - page 11) --> It is driven from Si5345A chip.
If the Si5345A chip is not programmed by default --> Is there a procedure for programming it without FSBL?


What you can do to at first is to test MGT over SFP: Create Xilinx IBERT core for both SFP with your target frequency and loop back  to the second SFP port on the TEBF0808.
I will do your suggestion and get back to you if there is any issue.

Thanks,
H. H.
« Last Edit: December 11, 2019, 10:34:20 AM by hossam84 »

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #3 on: December 11, 2019, 10:47:39 AM »
Hi,

Quote
I used the reference clock from the board --> B230_CLK0_P, B230_CLK0_N (I checked it from the schematic of TE0808 - page 11) --> It is driven from Si5345A chip.
If the Si5345A chip is not programmed by default --> Is there a procedure for programming it without FSBL?


SI Clock Builder Pro Software and Clock Builder Pro Field Programmer , see also:
https://wiki.trenz-electronic.de/display/PD/Si5345

--> I2C bus access for external programmer is connector marked as "D" in:
https://wiki.trenz-electronic.de/display/PD/TEBF0808+Getting+Started#TEBF0808GettingStarted-Overview

br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #4 on: December 12, 2019, 01:51:09 AM »
Hi,

As I don't have the Clock Builder Pro Field Programmer, I decided to use the provided SDK-FSBL from TRENZ (TE modified) for programming SI53545.

I checked which pins I used to provide clocks to the GT blocks which as follow (Correct me if I am wrong):
Starting from SI53545 --> (OUT1-PIN 28 (CLK1_P)/OUT1B -PIN27 (CLK1_N)) generates 125MHz (based on the SDK TE modified FSBL) --> goes to (B230_CLK0_P and B230_CLK0_N) --> goes to G8 (MGTREFCLK0P_230) and G7 (MGTREFCLK0N_230)

I depend on that the OUT clocks are programmed as following based on the SDK TE modified FSBL:
* Outputs:
 *    OUT0: 100 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT1: 125 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT2: 100 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT3: Unused
 *    OUT4: 100 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT5: 27 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT6: 78.8 MHz [ 78 + 4/5 MHz ]
 *          Enabled, LVDS 1.8 V
 *    OUT7: 156.25 MHz [ 156 + 1/4 MHz ]
 *          Enabled, LVDS 1.8 V
 *    OUT8: 25 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT9: Unused


Is it OK?

Thanks,
H. H.

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #5 on: December 12, 2019, 07:09:03 AM »
Hi,
yes, see: StarterKit\sw_lib\sw_apps\zynqmp_fsbl\src\te_Si5345-Registers.h

PS: there is also a CLK builder pro project available: StarterKit\misc\Si5345
You can generate your own header for FSBL: replace: StarterKit\sw_lib\sw_apps\zynqmp_fsbl\src\te_Si5345-Registers.h with the new generated

br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #6 on: December 12, 2019, 09:09:01 AM »
Hi John,

Is there any example design uses SFP(+) using TRENZ boards?

Does anyone tested the SFP(+) of the boards (e.g. TEBF08008 and TE0808)?

Thanks,

H.H.

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #7 on: December 12, 2019, 09:28:47 AM »
Hi,
at the moment we haven't any example online. It's planned to do this but i've can't tell you any timeline.

Which TE0808 assembly variant did you bough?
I can send you a internal project which based on Xilinx ibert, but I need your assembly variant.
Please write me to support@trenz-electronic.de
Or you create your own IBERT.
  • boot at first our Hello TE0808  starterkit example from SD --> you should see Hello TE0808 as endless loop. --> o PLL is initialised with this project as long as you do not power off the module
  • export Xilinx IBERT example from Vivado (Instantiate Xilinx ibert and export example project after you has configured IBERT correctly)
  • load generated bitfile from the Xilinx IBERT example over Vivado HW manager
br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #8 on: December 12, 2019, 10:04:35 AM »
Hello John,

I really appreciate your help. I sent you an email with the following:

We have the following TRENZ Boards:

- Baseboard --> TEBF0808-04A

- Module --> TE0808-04-15EG-1EE

Thanks in advance

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #9 on: December 16, 2019, 12:36:51 AM »
Hello,

I used the provided IBERT and tested the SFP in loop-back and it looks working.

I surveyed the xilinx forum for some information about the SFP with the 1G/2.5G PCS/PMA/SGMII IP block and also the Xilinx application note (XAPP1306) and many of them mentioned that the TX_Disable pin should be tied to 0 (ground) to enable start of the communication. I checked the schematic and I found this pin is connected to the CPLD (SFP1_TX_DIS) but I don't know the value of it.

So, what is the value of this pin in the CPLD configurations?? -- I found it -->  Low active (here https://wiki.trenz-electronic.de/display/PD/TEBF0808+TRM#TEBF0808TRM-MGTInterfacesSFP+andFireFly)
I think this is the correct configuration but why it is not working with the 1G/2.5G PCS/PMA/SGMII IP block???

Thanks,
H. H.
« Last Edit: December 16, 2019, 12:42:08 AM by hossam84 »

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #10 on: December 16, 2019, 08:14:03 AM »
Hi,
yo can do following:

1. check if the MGT PLLs are locked in the Xilinx IP and if there is a link (maybe you must active some debug interfaces -->or add system ibert (second ibert, which can connect to some MGT IPs from Xilinx) if possible).

2. Start SW debugger and check where's exact the problem.

PS: SFP1_TX_DIS is only need for active SFP cable. Passive sfp copper cable did not check the disable pin.


br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #11 on: December 24, 2019, 02:48:32 AM »
Hello,

Currently, I am testing the SFP+ port using Xilinx Aurora 8b/10 IP. What confusing me is the CLOCK pin for the MGT.
In the TRM it is mentioned that:
B230_RX3_P, B230_RX3_N, pins J1-3, J1-5      | 1 reference clock signal (B230_CLK1) from B2B connector
B230_TX3_P, B230_TX3_N,  pins J1-2, J1-4      | J3 (pins J3-59, J3-61) to bank's pins G8/G7


B230_RX2_P, B230_RX2_N, pins J1-9, J1-11    | 1 reference clock signal (B230_CLK0) from programmable
B230_TX2_P, B230_TX2_N, pins J1-8, J1-10    | PLL clock generator U5 to bank's pins E8/E7


However, in the schematic it is opposite:
B230_CLK0_P --> G8
B230_CLK0_n --> G7


and
B230_CLK1_P --> E8
B230_CLK1_n --> E7


Could yo please let me know which one is correct. I got lost trying to use SFP with Aurora or Ethernet.

Thanks,
H. H

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #12 on: December 27, 2019, 06:49:54 AM »
Hi,

I have done testing the SFP+ port using Xilinx Aurora 8b/10 IP. The below link contains my project to whom it may concern.

https://github.com/hossamfadeel/SFP-cable-in-loopback-using-Aurora-8b-10b

Currently, I am trying with 10g ethernet using SFP. Any recommendations are appreciated.

Thanks,
H. H

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #13 on: January 06, 2020, 09:25:02 AM »
Hi,

schematics are correct. On TRM it's an copy paste error, I will fix it.

PS: With U+ Device you can add a System IBERT onto your Aurora Design to check what's happens.
 
br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #14 on: January 09, 2020, 03:57:48 AM »
Hello,

I did most of the things that can be done to test the SFP with Ethernet.
Let me go step by step of what I have done and my observation so if some one can tell me what is the wrong thing I may did:

1. I configured the clock using FSBL and I test the SFP in loopback with the IBERT IP and Aurora and it is working fine.
Note: when I program the FSBL I get the following message: "Error while launching program: Timeout Reached. Mask poll failed at ADDRESS: 0XFD4023E4 MASK: 0x00000010"
and "Error while launching program: Timeout Reached. Mask poll failed at ADDRESS: 0XFD4063E4 MASK: 0x00000010"
I checked Xilinx forum and they suggest to comment the dedicate register which made it run without complaining.
The output of the FSBL in the terminal is attached as well as other snapshots.

2. I followed the Xilinx XAPP1306 application not and I used same configuration of the "Using PS GEM through EMIO" and made the same configuration for the 1G/2.5G Ethernet PCS/PMA or
SGMII LogiCORE IP. I configured it as 1000BASE-T because I use an SFP to RJ-45 converter (I am using 10Gteck SFP to RJ45 Copper Module - 1000BASE-T).
3. In SDK I used the LWIP echo server example and what I got is the following:
   -------lwIP TCP echo server------
TCP packet sent to port 6001 will be echoed back
start PHY autonegotiation
waiting for PHY to complete autonegatiation

---> Stuck here

I uploaded my design on github for your check.
https://github.com/hossamfadeel/StarterKit

Please help me I tested with two trenz TE0808 but no success. We have a tight schedule to deliver the project and go to mass product using trenz TE0808 module in our design.


Regards,
H. H

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #15 on: January 09, 2020, 07:06:19 AM »
Hi,
Quote
0XFD4023E4
https://www.xilinx.com/html_docs/registers/ug1087/ug1087-zynq-ultrascale-registers.html

Xilinx init script did not use our modification to initilise SI on the module, so REF CLKS for GTR Interfaces are missing in startup fails.So I think that can be the same problem for your design
What you can do:
  • Create Boot.bin with the modified FSBL  and maybe simple hello world
  • Boot your system to initilise SI
  • set Boot Mode to JTAG and press reset Button (longer than 3 second --> reset button has double functionality, at first soft reset, after the hard reset). Important do not press power button!
  • Connect JTAG and try again.
br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #16 on: January 09, 2020, 07:35:00 AM »
Hi JH,

Currently, I succeeded on making SFP working wih ethernet BUT the steps I do is like a puzzle.
If I go straightforward with programming the FPGA then SDK lwip program --> doesn't work.

What I have to do every time I power on the board is as following:
1. Program the FPGA then FSBL using SDK.
2. Program the FPGA then lwip echo server example using SDK.

If I try with programming the FSBL, the  lwip echo server example doesn't work.


Thanks,
H.H




JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #17 on: January 09, 2020, 09:21:55 AM »
Quote
If I try with programming the FSBL, the  lwip echo server example doesn't work.
What did you mean?
Put boot system  with boot.bin which includes FSBL, bitstream, and echo server application?
br
John

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #18 on: January 10, 2020, 12:01:45 AM »
Hi,

Actually, I am using standalone application and what I mean is ---> when I program the FSBL then lwip example without programming FPGA bitstream again the lwip echo server example doesn't work.

BUT, ---> when I program the FSBL then lwip example with programming FPGA bitstream again the lwip echo server example does work.
See attached file.

Thanks
H. H

JH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1091
Re: SFP+ using TEBF08008 and TE0808
« Reply #19 on: January 10, 2020, 08:24:56 AM »
What's happens when you create boot.bin with FSBL, bitstream and your lwip barmatel example?

For me it sound like timing or reset issue or buf of the sdk.

Did you add reset block in your BD design --> delay the reset a little bit more, maybe this helps.

br
Jhn

hossam84

  • Active Member
  • *
  • Posts: 20
Re: SFP+ using TEBF08008 and TE0808
« Reply #20 on: January 13, 2020, 12:03:04 AM »
Ok.

I will try it. Thanks for your help and suggestions.

Regards,
Hossam