News:

Attention: For security reasons,please choose a user name *different* from your login name.
Also make sure to choose a secure password and change it regularly.

Main Menu

TE0726 ZynqBerry - 720p / 60 fps support

Started by wige, June 30, 2016, 09:59:59 AM

Previous topic - Next topic

wige

Hi,

as the RPi camera supports 720p at 60 fps, I have tried to get this working on the ZynqBerry by simply increasing the PLL multiplier setting of the camera. For the camera as well as the CSI interface I didn't detect any issues up to a multiplier setting of 0x85 (~59.9 fps).

But it turned out that the Demosaicing IP (from Trenz Demo Design, build 34) is the major bottleneck. I have observed a throughput of something around 4 clock cycles per pixel - can't remember the exact value. This limits the max. framerate to something around 40 fps.

I have written a more efficient version (as this IP is just a quick hack it still creates some artefacts at the left and top borders :( ) and with that one I am getting closer to 60 fps. Unfortunately, I do not get full 60 fps (just ~50 fps) because the Unpack IP seems to be the 2nd bottleneck of the VideoIn pipe.
Okay, so far so good...

My question is, if anybody else is working on the throughput improvement - or even has a working update of the Demosaicing IP available ;D

Regards
Winfried

Antti Lukats

Hi

this is very hearth warming to hear, and if you have improvements you can share we are interested in merging them.

What we have made and made available for free is initial proof of concept processing pipeline, so the IP what we have now are good for the demonstrator purpose. We of course would like to work more on them, as time permits. There are possible many places for major improvements.

At the moment we have no better stuff to give.

wige

Hi,

yes, indeed! The most important good news is that the CSI PHY is able to handle 720p60. Initially I suspected issues in that area and was quite positively surprised that even with the ZynqBerry we can push the camera to its limits.

I fully understand that you (= Trenz Team) have other stuff on your todo list and are not able to deal with this issue.
But perhaps there is somebody else out there who is interested in higher frame rates.
As my time is limited as well, I would be happy if we could join forces...

wige

#3
I have spend some time on the redesign of the Unpacking an Demosaicing IP to get higher frame rates than 30 fps. Finally it ended up with a complete redesign of these two IPs. Current Status: https://netcase.hs-osnabrueck.de/index.php/s/cbYyssOdh6bMzJm

A few remarks:
1. 60 fps is possible (tested at 720p60) -- Do not forget: You need to increase the PLL multiplier setting of the camera (Reg 0x3036) in order to get higher frame rates
2. I have moved the gamma correction to a separate IP
3. As I use ARGB (instead of ABGR) the demosaicing is set to ARGB by default. If you like to get ABGR, just uncheck the box in the configuration GUI
4. The core demosaicing functionality could/should be improved. Currently it's still rather simple. But it's at least good enough for a proof of concept.
5. The IP is fast enough to cope with the 720p60 pixel rate even when lowering the IP clk frequency. I have used 115 MHz. Other frequencies may (or may not) work as well.

Regards
Winfried


Antti Lukats

Thank You!

amazing, Awesome!

I am really pleased. We do what we can but we can not do it all alone.

We will check the IP cores.

br
Antti Lukats

wige

Hi Antti,

just in case you find some time to play with the camera, here is something you could try as well:
1. With settings from v4l2 driver and again PLL multiplier tweaking, I was able to get even more fps at VGA resolution (I measured ~110 fps).
2. It seems that AWB is switched off by default. After setting the two registers 0x5001/0x5002 to 0x01/0x41 the camera delivers more realistic colors.

Regards

Sensei

Quote from: wige on June 30, 2016, 12:28:58 PM
Hi,
yes, indeed! The most important good news is that the CSI PHY is able to handle 720p60. Initially I suspected issues in that area and was quite positively surprised that even with the ZynqBerry we can push the camera to its limits.
I fully understand that you (= Trenz Team) have other stuff on your todo list and are not able to deal with this issue.

I am very happy to see than there people working to get the best of a board like this, it got so much potential.
I love than Trenz team is on-board too, i was trying to work with HDMI video stream (CSI camera) from a raspberry pi   to be process in a Pynq board.
I end up buying a ZynqBerry, all in one right .
I got no to much experience with PL(FPGA) but many years as a programmer so I will see how can i help too.
My goal is to use the ZynqBerry as a Hardware base Color recognition sensor (a pretty powerful one, more that 60fps)   :)

Quote from: wige on June 30, 2016, 12:28:58 PM
But perhaps there is somebody else out there who is interested in higher frame rates.
As my time is limited as well, I would be happy if we could join forces...

I will also be happy to join forces with you guys