SFP+ using TEBF08008 and TE0808

[hossam84]

Hello,

I am trying to use the SFP+ port of the TEBF08008 base board using the TE0808 board. I checked the datasheets and schematic of both board and I think I configured the pins accordingly.
I used the GEM0 for the purpose of SFP+ connection. IO used EMIO for the GEM0 (shown in the attachment) and then connected the GMII_ENET0 to gmii_gem_pcs_pma of the 1G/2.5G Ethernet PCS/PMA or SGMII IP. Also, I connected the MDIO_ENET0 to the mdio_pcs_pma of the 1G/2.5G Ethernet PCS/PMA or SGMII IP (shown in the attachment).
For the differential clock of the GT, I used the differential clock of the board as show in the schematic. I used SFP+ of MGT Lane 2 (B230_RX2_P, B230_RX2_N, B230_TX2_P, B230_TX2_N).

I configured the 1G/2.5G Ethernet PCS/PMA or SGMII IP same as the example of Xilinx application note xapp1306 - ps_emio_eth_1g.

After generating bit stream I export the hardware to run the SDK with lwip echo server example. I tried to make any connection with host which has the next-562sfp-10g module but I didn't get anything.

Is there anyone who has tested the SFP(+) who can provide me with some hints or guidlines.

Thanks in advance

[JH]

Hello,
did you use passive copper cable or optical cable? --> if optical check supported frequencies.
Did you check that the MGTs REF clock is configured an valid? Which Reference CLK did you use? SI on the carrier is not programmed by default this is done temporary only with our modified FSBL in the reference design. --> it can also happens that you need other reference frequency like we default provide with the reference design
Maybe it's only a problem of Xilinx barmetal app, they run seldom out of the box.

What you can do to at first is to test MGT over SFP: Create Xilinx IBERT core for both SFP with your target frequency and loop back  to the second SFP port on the TEBF0808.
br
John

[hossam84]

Thanks for your reply.

did you use passive copper cable or optical cable? --> if optical check supported frequencies.
I am using Passive copper Cable Assembly from Molex (https://www.digikey.com/products/en?keywords=74752-2301)

Did you check that the MGTs REF clock is configured an valid? Which Reference CLK did you use? SI on the carrier is not programmed by default this is done temporary only with our modified FSBL in the reference design. --> it can also happens that you need other reference frequency like we default provide with the reference design
Maybe it's only a problem of Xilinx barmetal app, they run seldom out of the box.


I used the reference clock from the board --> B230_CLK0_P, B230_CLK0_N (I checked it from the schematic of TE0808 - page 11) --> It is driven from Si5345A chip.
If the Si5345A chip is not programmed by default --> Is there a procedure for programming it without FSBL?


What you can do to at first is to test MGT over SFP: Create Xilinx IBERT core for both SFP with your target frequency and loop back  to the second SFP port on the TEBF0808.
I will do your suggestion and get back to you if there is any issue.

Thanks,
H. H.

[JH]

Hi,

I used the reference clock from the board --> B230_CLK0_P, B230_CLK0_N (I checked it from the schematic of TE0808 - page 11) --> It is driven from Si5345A chip.
If the Si5345A chip is not programmed by default --> Is there a procedure for programming it without FSBL?

SI Clock Builder Pro Software and Clock Builder Pro Field Programmer , see also:
https://wiki.trenz-electronic.de/display/PD/Si5345

--> I2C bus access for external programmer is connector marked as "D" in:
https://wiki.trenz-electronic.de/display/PD/TEBF0808+Getting+Started#TEBF0808GettingStarted-Overview

br
John

[hossam84]

Hi,

As I don't have the Clock Builder Pro Field Programmer, I decided to use the provided SDK-FSBL from TRENZ (TE modified) for programming SI53545.

I checked which pins I used to provide clocks to the GT blocks which as follow (Correct me if I am wrong):
Starting from SI53545 --> (OUT1-PIN 28 (CLK1_P)/OUT1B -PIN27 (CLK1_N)) generates 125MHz (based on the SDK TE modified FSBL) --> goes to (B230_CLK0_P and B230_CLK0_N) --> goes to G8 (MGTREFCLK0P_230) and G7 (MGTREFCLK0N_230)

I depend on that the OUT clocks are programmed as following based on the SDK TE modified FSBL:
* Outputs:
 *    OUT0: 100 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT1: 125 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT2: 100 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT3: Unused
 *    OUT4: 100 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT5: 27 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT6: 78.8 MHz [ 78 + 4/5 MHz ]
 *          Enabled, LVDS 1.8 V
 *    OUT7: 156.25 MHz [ 156 + 1/4 MHz ]
 *          Enabled, LVDS 1.8 V
 *    OUT8: 25 MHz
 *          Enabled, LVDS 1.8 V
 *    OUT9: Unused

Is it OK?

Thanks,
H. H.

[JH]

Hi,
yes, see: StarterKit\sw_lib\sw_apps\zynqmp_fsbl\src\te_Si5345-Registers.h

PS: there is also a CLK builder pro project available: StarterKit\misc\Si5345
You can generate your own header for FSBL: replace: StarterKit\sw_lib\sw_apps\zynqmp_fsbl\src\te_Si5345-Registers.h with the new generated

br
John

[hossam84]

Hi John,

Is there any example design uses SFP(+) using TRENZ boards?

Does anyone tested the SFP(+) of the boards (e.g. TEBF08008 and TE0808)?

Thanks,

H.H.

[JH]

Hi,
at the moment we haven't any example online. It's planned to do this but i've can't tell you any timeline.

Which TE0808 assembly variant did you bough?
I can send you a internal project which based on Xilinx ibert, but I need your assembly variant.
Please write me to support@trenz-electronic.de
Or you create your own IBERT.

• boot at first our Hello TE0808  starterkit example from SD --> you should see Hello TE0808 as endless loop. --> o PLL is initialised with this project as long as you do not power off the module
• export Xilinx IBERT example from Vivado (Instantiate Xilinx ibert and export example project after you has configured IBERT correctly)
• load generated bitfile from the Xilinx IBERT example over Vivado HW manager

br
John

[hossam84]

Hello John,

I really appreciate your help. I sent you an email with the following:

We have the following TRENZ Boards:

- Baseboard --> TEBF0808-04A

- Module --> TE0808-04-15EG-1EE

Thanks in advance

[hossam84]

Hello,

I used the provided IBERT and tested the SFP in loop-back and it looks working.

I surveyed the xilinx forum for some information about the SFP with the 1G/2.5G PCS/PMA/SGMII IP block and also the Xilinx application note (XAPP1306) and many of them mentioned that the TX_Disable pin should be tied to 0 (ground) to enable start of the communication. I checked the schematic and I found this pin is connected to the CPLD (SFP1_TX_DIS) but I don't know the value of it.

So, what is the value of this pin in the CPLD configurations?? -- I found it -->  Low active (here https://wiki.trenz-electronic.de/display/PD/TEBF0808+TRM#TEBF0808TRM-MGTInterfacesSFP+andFireFly)
I think this is the correct configuration but why it is not working with the 1G/2.5G PCS/PMA/SGMII IP block???

Thanks,
H. H.

[JH]

Hi,
yo can do following:

1. check if the MGT PLLs are locked in the Xilinx IP and if there is a link (maybe you must active some debug interfaces -->or add system ibert (second ibert, which can connect to some MGT IPs from Xilinx) if possible).

2. Start SW debugger and check where's exact the problem.

PS: SFP1_TX_DIS is only need for active SFP cable. Passive sfp copper cable did not check the disable pin.


br
John

[hossam84]

Hello,

Currently, I am testing the SFP+ port using Xilinx Aurora 8b/10 IP. What confusing me is the CLOCK pin for the MGT.
In the TRM it is mentioned that:
B230_RX3_P, B230_RX3_N, pins J1-3, J1-5      | 1 reference clock signal (B230_CLK1) from B2B connector
B230_TX3_P, B230_TX3_N,  pins J1-2, J1-4      | J3 (pins J3-59, J3-61) to bank's pins G8/G7


B230_RX2_P, B230_RX2_N, pins J1-9, J1-11    | 1 reference clock signal (B230_CLK0) from programmable
B230_TX2_P, B230_TX2_N, pins J1-8, J1-10    | PLL clock generator U5 to bank's pins E8/E7


However, in the schematic it is opposite:
B230_CLK0_P --> G8
B230_CLK0_n --> G7

and
B230_CLK1_P --> E8
B230_CLK1_n --> E7

Could yo please let me know which one is correct. I got lost trying to use SFP with Aurora or Ethernet.

Thanks,
H. H

[hossam84]

Hi,

I have done testing the SFP+ port using Xilinx Aurora 8b/10 IP. The below link contains my project to whom it may concern.

https://github.com/hossamfadeel/SFP-cable-in-loopback-using-Aurora-8b-10b

Currently, I am trying with 10g ethernet using SFP. Any recommendations are appreciated.

Thanks,
H. H

[JH]

Hi,

schematics are correct. On TRM it's an copy paste error, I will fix it.

PS: With U+ Device you can add a System IBERT onto your Aurora Design to check what's happens.
 
br
John

[hossam84]

Hello,

I did most of the things that can be done to test the SFP with Ethernet.
Let me go step by step of what I have done and my observation so if some one can tell me what is the wrong thing I may did:

1. I configured the clock using FSBL and I test the SFP in loopback with the IBERT IP and Aurora and it is working fine.
Note: when I program the FSBL I get the following message: "Error while launching program: Timeout Reached. Mask poll failed at ADDRESS: 0XFD4023E4 MASK: 0x00000010"
and "Error while launching program: Timeout Reached. Mask poll failed at ADDRESS: 0XFD4063E4 MASK: 0x00000010"
I checked Xilinx forum and they suggest to comment the dedicate register which made it run without complaining.
The output of the FSBL in the terminal is attached as well as other snapshots.

2. I followed the Xilinx XAPP1306 application not and I used same configuration of the "Using PS GEM through EMIO" and made the same configuration for the 1G/2.5G Ethernet PCS/PMA or
SGMII LogiCORE IP. I configured it as 1000BASE-T because I use an SFP to RJ-45 converter (I am using 10Gteck SFP to RJ45 Copper Module - 1000BASE-T).
3. In SDK I used the LWIP echo server example and what I got is the following:
   -------lwIP TCP echo server------
TCP packet sent to port 6001 will be echoed back
start PHY autonegotiation
waiting for PHY to complete autonegatiation

---> Stuck here

I uploaded my design on github for your check.
https://github.com/hossamfadeel/StarterKit

Please help me I tested with two trenz TE0808 but no success. We have a tight schedule to deliver the project and go to mass product using trenz TE0808 module in our design.


Regards,
H. H

[JH]

Hi,

0XFD4023E4

https://www.xilinx.com/html_docs/registers/ug1087/ug1087-zynq-ultrascale-registers.html

Xilinx init script did not use our modification to initilise SI on the module, so REF CLKS for GTR Interfaces are missing in startup fails.So I think that can be the same problem for your design
What you can do:

• Create Boot.bin with the modified FSBL  and maybe simple hello world
• Boot your system to initilise SI
• set Boot Mode to JTAG and press reset Button (longer than 3 second --> reset button has double functionality, at first soft reset, after the hard reset). Important do not press power button!
• Connect JTAG and try again.

br
John

[hossam84]

Hi JH,

Currently, I succeeded on making SFP working wih ethernet BUT the steps I do is like a puzzle.
If I go straightforward with programming the FPGA then SDK lwip program --> doesn't work.

What I have to do every time I power on the board is as following:
1. Program the FPGA then FSBL using SDK.
2. Program the FPGA then lwip echo server example using SDK.

If I try with programming the FSBL, the  lwip echo server example doesn't work.


Thanks,
H.H

[JH]

If I try with programming the FSBL, the  lwip echo server example doesn't work.

What did you mean?
Put boot system  with boot.bin which includes FSBL, bitstream, and echo server application?
br
John

[hossam84]

Hi,

Actually, I am using standalone application and what I mean is ---> when I program the FSBL then lwip example without programming FPGA bitstream again the lwip echo server example doesn't work.

BUT, ---> when I program the FSBL then lwip example with programming FPGA bitstream again the lwip echo server example does work.
See attached file.

Thanks
H. H

[JH]

What's happens when you create boot.bin with FSBL, bitstream and your lwip barmatel example?

For me it sound like timing or reset issue or buf of the sdk.

Did you add reset block in your BD design --> delay the reset a little bit more, maybe this helps.

br
Jhn

[hossam84]

Ok.

I will try it. Thanks for your help and suggestions.

Regards,
Hossam

[abr_ac]

I am trying to use the SFP port of the TEBF0808-04A base board using the TE0807-02-4BE21-AK.
I'm having the same problem to test the SFP in loopback with Lwip echo server example (I looped the fiber cable back to the same SFP) with the RPU (Cortex-R5_0). did you manage to have a working example ? in case if you don't have it right, here are my configurations (vitis2019.2, vivado2019.2) :
I use the reference design SK_DEMO1 and in vivado i've added the 1G/2.5G Ethernet PCS/PLA or SGMI IP with the following parameters :

• Ethernet MAC: Zynq PS Gigabit Ethernet Controller
• Standard: 1000BASEX,
• Core Functionality - Physical interface: Device Specific Transceiver, Transceiver options: Ref clk: 125 MHz, Transceiver Location: X0Y3, DRP clock Frequency: 50 MHz (the other options are kept to default including enabling the auto-negotiation
• gtrefclk_in is connected to the MGTRFCLK1N/P226 --> B10 --> B227CLK1P/N to have the 125 MHz
• independant_clock_bufg to the plclk_1 with the value of 50MHz
• phyaddr to a constant value of 9 (as in the previous posts)
• config_vector to a constant value of 0b10000
• config_valid, an_adv_config_val, an_restart_config and reset to a constant value of 0
• an_adv_config_vector to a constant value of 0b0000000110100000
• signal_detect to a constant value of 1
• sfp is connected to  B230-RX2_N/P --> B226_RX2_N/P --> F1/F2 and B230-TX2_N/P --> B226_TX2_N/P --> F5/F6 

I've exported the hardware design using the command: TE::hw_build_design -export_prebuilt and i've generated the programming files with TE::sw_run_vitis -all (i don't use for now the SD, card and petalinux)
In vivado i fist run the FSBL which I hope configure the OUT1 to 125MHz of the Si5345 and then create a new plateform for the baremetal cortex-R5 with lwip.
When i start the debug session i have the following result (normally i should get for the phy address a value of 9 and if the link is up or down):
--------------------------------------------------------------------------------
TE0807 TE_XFsbl_HookPsuInit_Custom
Configure Carrier I2C Switch 0x77
Configure PLL: SI5345-B
Si534x Init Registers Write.
Si534x Init Complete.
Status   0xC:0x0, 0xE:0x0, 0xD:0x0, 0x11:0, 0xF:0 (...waiting for calibration...256PLL Status Register   0xC:0x0, 0xE:0x0, 0xD:0x0, 0x11:0, 0xF:0.
USB Reset Complete
PCIe Reset Complete

--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Xilinx Zynq MP First Stage Boot Loader (TE modified)
Release 2019.2   Oct 29 2020  -  15:43:26
Device Name: XCZU4EG

--------------------------------------------------------------------------------
TE0807 TE_XFsbl_BoardInit_Custom
Configure Carrier I2C Switch 0x73 for EEPROM access

--------------------------------------------------------------------------------
PMU-FW is not running, certain applications may not be supported.


-----lwIP TCP echo server ------
TCP packets sent to port 6001 will be echoed back
Start PHY autonegotiation
Waiting for PHY to complete autonegotiation.
autonegotiation complete
link speed for phy address 0: 1000
Board IP: 192.168.1.10
Netmask : 255.255.255.0
Gateway : 192.168.1.1
TCP echo server started @ port 7

[GuinnessTrinker]

I found some nice example designs on

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/478937213/MPSoC+PS+and+PL+Ethernet+Example+Projects

They use Xilinx board ZCU102

- Using PS GEM through MIO
- Using PS GEM through EMIO
- Using PL 1G Ethernet
- Using PL 10G Ethernet

What are major challenges when porting these designs to TEBF08008 and TE0808 ??

[JH]

Hi,
I would recommend to start with Starterkit design.
Create this design and regenerate all files without any changes. If this works, start to modify this project (at first PS+PL), than FSBL if need (maybe you need other SI5345 CLK, than Linux part) with this parts from the Xilinx example which are need for ETH over SFP.

br
John

[abr_ac]

Hi JH,

Thanks for the reply, i managed to solve the problem (it was a clock problem: gtrefclk_in)

Best regards,