TE0820 SFP+ cage connected to GT lanes not working

[jan.blaesi]

Hi!

Now that our secondary ethernet is working properly, we are trying to make our SFP+ cages work.
Both are directly connected to the PS GTR lanes (GEM1, GEM2) using SGMII, there is no PHY.
We are unsure if this could work at all or if we are just missing something in our configuration.

The kernel seems to do some communication (but probably only over I2C):

Code Select Expand


# dmesg | grep eth1
[    3.679039] macb ff0c0000.ethernet eth1: Cadence GEM rev 0x50070106 at 0xff0c0000 irq 38 (xx:xx:xx:xx:xx:xx)
[    4.439943] macb ff0c0000.ethernet eth1: validation of inband/1000base-x with support 0000000,00000200,00004440 failed: -22
[   12.458391] macb ff0c0000.ethernet eth1: configuring for fixed/sgmii link mode

# dmesg | grep eth2
[    3.705274] macb ff0d0000.ethernet eth2: Cadence GEM rev 0x50070106 at 0xff0d0000 irq 39 (xx:xx:xx:xx:xx:xx)
[   12.402258] macb ff0d0000.ethernet eth2: configuring for fixed/sgmii link mode

# dmesg | grep sfp
[    4.110090] sfp sfp-gem1: Host maximum power 1.0W
[    4.142013] sfp sfp-gem2: Host maximum power 1.0W
[    4.462082] sfp sfp-gem1: module AVAGO            AFBR-5715APZ     rev      sn xxxxxxxxxxx      dc 170812


Here is the device tree from our PetaLinux:

Code Select Expand


&gem1 {
    sfp = <&sfp0>;
    local-mac-address = [xx xx xx xx xx xx];
    phy-mode = "sgmii";
    is-internal-pcspma;
    fixed-link {
        speed = <1000>;
        full-duplex;
    };
};

&gem2 {
    sfp = <&sfp1>;
    local-mac-address = [xx xx xx xx xx xx];
    phy-mode = "sgmii";
    is-internal-pcspma;
    fixed-link {
        speed = <1000>;
        full-duplex;
    };
};

/ {
    sfp0: sfp-gem1 {
        compatible = "sff,sfp";
        i2c-bus = <&i2c_sfp0>;
        sfp,ethernet = <&gem1>;

        tx-fault-gpios = <&gpio_sfp0 0 GPIO_ACTIVE_HIGH>;
        tx-disable-gpios = <&gpio_sfp0 1 GPIO_ACTIVE_HIGH>;
        rate-select0-gpios = <&gpio_sfp0 2 GPIO_ACTIVE_HIGH>;
        los-gpios = <&gpio_sfp0 3 GPIO_ACTIVE_HIGH>;
        mod-def0-gpios = <&gpio_sfp0 7 GPIO_ACTIVE_LOW>;
    };

    sfp1: sfp-gem2 {
        compatible = "sff,sfp";
        i2c-bus = <&i2c_sfp1>;
        sfp,ethernet = <&gem2>;

        tx-fault-gpios = <&gpio_sfp1 0 GPIO_ACTIVE_HIGH>;
        tx-disable-gpios = <&gpio_sfp1 1 GPIO_ACTIVE_HIGH>;
        rate-select0-gpios = <&gpio_sfp1 2 GPIO_ACTIVE_HIGH>;
        los-gpios = <&gpio_sfp1 3 GPIO_ACTIVE_HIGH>;
        mod-def0-gpios = <&gpio_sfp1 7 GPIO_ACTIVE_LOW>;
    };
};


We also tried switching phy-mode to "moca", but that resulted in a "fixed link not detected" message in the dmesg.

Best regards

[M Kirberg]

I do not see an obvious error.

Is this a custom Carrier?

[jan.blaesi]

Yes, it is on a custom carrier board.

[M Kirberg]

Did you check Reference Clocks/Locking PLLs via IBERT?

[jan.blaesi]

We did not check this, however the 88E1512 PHY connected to GEM0 using the same reference clock works without problems.
As the IBERT process seems rather involved we will have to look into this, if we have all hardware required.

Best regards

[M Kirberg]

Same Reference Clock?

Am I missing something?
88E1512 needs 25MHz(XTAL_in) but GT Ref Clocks for GEM need 125MHz?

You do not need hardware for IBERT for checking that the PLL Locks only. If you have a Loopback Adapter for SFP then you can do additional BER tests

[jan.blaesi]

I meant the reference clock of the SGMII used to connect the 88E1512 (attached to PS GTR GEM0).
Good to know, then we will try to get IBERT running and see if this gets us any more indicators on what is wrong.

Best regards

[jan.blaesi]

As the same reference clock works without a problem for GEM0, we assume the problem is somewhere else.

We are wondering if our setup could work at all, directly attaching a SFP cage to the PS GTR lane without a PHY inbetween.

Best regards

[rkbluecubed]

We are wondering if our setup could work at all, directly attaching a SFP cage to the PS GTR lane without a PHY inbetween.

That should work just fine.  The SFP itself is serving as your PHY.