Hi,
yes it should be possible. But you didn't have longer access to SD, so QSPI boot only.
See schematics TEB0745 page 4 and TE0745 page 5
From schematics, Bank33, MGT0..3 (and external CLK inputs), some MIO (GPIO, LED Ctrl, Usr Button, I2C Reset for SFP I2C mux, SD) are not available in case you remove this connector. I would recommend to de-active this unused MIO on your Vivado PS IP.
br
John