Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Framerate limit #195

Merged
merged 11 commits into from
Mar 9, 2022
Merged

Framerate limit #195

merged 11 commits into from
Mar 9, 2022

Conversation

15498th
Copy link
Contributor

@15498th 15498th commented Dec 16, 2021

Allow to limit stream framerate in order to make it stable and lower load on the board for usecases where high fps is not required.
In order to keep code simple limit is set not in frames per second but in milliseconds between frames to make it integer.
Web-interface uses drop-down list with set of pre-defined fps values.

15498th added 3 commits December 17, 2021 00:14
Add global variable and #define in config  to set default.
Add new parameter in /control handler and preferences.
@easytarget
Copy link
Owner

Thankyou, especially for doing the ov3660 too!
However, while this is a good idea, it needs a slightly different implementation:

  • Using delay() to implement this is the wrong approach, a better approach is:
    1. Note the end time of the each frame request
    2. When the next frame is requested check to see if the frame rate time has expired, if not, loop until it does before beginning the frame request.
    3. eg.. treat this as a limit, not a delay.
  • Also; I find the UI changes confusing, values like '3333' fps make no sense, you can never approach such framerates even at super-low resolutions.

@15498th
Copy link
Contributor Author

15498th commented Dec 18, 2021

I changed UI values to correspond with values used in commands, please see if it makes more sense now.
It seems commit with API.md update got missed, so I added it now.
Using time limit instead of fps might be confusing by itself, but /control doesn't handle float and I believe this specific command doesn't seems to be worth complicating parse logic.

I'm not sure how to understand "loop until it does". Do you want something like this instead of delay() call?

while((esp_timer_get_time() - last_frame)/1000 < minFrameTime) {
    delay(1);
}

It seems redundant, but I can add it if you consider this right approach. Is 1ms good enough?

@easytarget
Copy link
Owner

easytarget commented Dec 18, 2021

Cool, I'll have a look at this asap. I think we're in lockdown again from tomorrow (:netherlands:) so I should have some spare time at my desk.. sigh.

I want to upload and test this for myself, then I'll be better able to understand how the UI works ;-)

And yes, I was thinking of an idle loop until the timeout is reached, but I've been thinking overnight and I'm wondering how best to do this without locking the whole cam process up while we wait for the next frame to be served.. especially for values like '1 frame per second'..)

I'm seriously thinking:

  1. Create an interrupt for processing the request.. eg. on the first frame the client gets an immediate frame response, and will make an immediate request for the next frame. but if that request arrives within the minimum frame time we should set an interrupt for the appropriate response time and relinquish control back to the main loop until it is time to reset the grab the next frame and serve the response.
  2. This is, frankly, a halfway-house step towards supporting proper multicast stream operation (see support for multiclient streaming #51 ).
  3. Something nasty has happened to the camera code between esp-core 2.0.0 and esp-core 2.0.1; it looks like a whole bunch of possibly breaking changes have occurred and I'm seeing some really odd behaviour on my test rig with 2.0.1. Watch for a blocker Issue regarding this soon. I want to test and understand something before I write it up.
  4. How much easier this would be to do in Python (sorry, but I've been doing a load of Python programming lately, and I'm missing the ease with which it handles things like interrupts and exception handling/processing.)

15498th added 2 commits December 19, 2021 01:28
Putting delay after finishing serving the frame causes image to appear
in stream with a lag. The reason for this appears to be that browser
relies on next frame content boundary in order to determine the end of
image.

That means that in order for browser to show frame right after receiving
it, server needs to send next frame content boundary right after image
itself, and delay to limit framerate should be applied after content
boundary, not before it.

According to https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
sending first frame without content boundary will likely cause its
content to be ignored, which seems to be acceptable price for not
treating first iteration of the loop as special case.
@15498th
Copy link
Contributor Author

15498th commented Dec 18, 2021

With this change it should look better with low fps. Compare streaming something like seconds (on clock) on 0.3fps with and without these commits.

@easytarget
Copy link
Owner

Note; Not forgotten about this but I'm doing some hotfixes first since expressif have broken stuff with the 2.0.1 core, etc. it means I'll be pushing to master branch again before I can come back to this, sorry ;-)

@15498th
Copy link
Contributor Author

15498th commented Dec 21, 2021

Sure thing.
Though as my board doesn't have working PSRAM, I likely won't be able to test anything esp-idf 4 based.

@easytarget easytarget mentioned this pull request Mar 9, 2022
@easytarget easytarget changed the base branch from master to 4-0-rc1 March 9, 2022 15:30
@easytarget easytarget merged commit 9426f3f into easytarget:4-0-rc1 Mar 9, 2022
@easytarget
Copy link
Owner

Thankyou for this, sorry it took me ages to get round to merging it.

@easytarget
Copy link
Owner

Just a quick note; I've got this fully merged into the 4.0RC now, and had a quick test/play with it..
It works very well, so thankyou once again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants