Add multiclient streaming support

[daschr]

Hi,

first, I would like to thank you for this great project.
I've added multiclient streaming support by using a client queue and a FreeRTOS task for broadcasting the current frames to connected clients (open sockets).

Maybe you can check this out.

Cheers,

David

[easytarget]

Ooh wow, probably my number 1 desired feature. Aka #51
Thank you, I'll check it out later

Initial thoughts are that it needs to support both the streamCount and streamKill mechanisms, but this may be best done by a function that returns how many streams tasks are running, and another that halts all running streams.
Both will be useful for debugging people with connection and browser issues..etc.

[easytarget]

Created a feature branch for this

[daschr]

Hi,

I've added the required methods for getting the number of active streams and killing all of them (see ba3c09f#diff-6514cf5faf6a9458e0775cf7f86a55bd5e65d9683cdba647de66d59478144072R33-R34) and adapted the the code in ba3c09f#diff-d1d9020991f0a5fde49e6b3a31812bffb258a12eaa271e106f4252e23d57fdf0. Additionally, I think one should add a configuration option in the myconfig.h for the desired FPS (see ba3c09f#diff-d1d9020991f0a5fde49e6b3a31812bffb258a12eaa271e106f4252e23d57fdf0R796-R798).

[easytarget]

Very cool, I wish I was as fluid with C coding ;-)
I've got to repair my test module (broken connector on FTDI board, some delicate soldering required, might not happen immediately) I'll merge to the local branch and test this out.

[easytarget]

That works really well in terms of serving multiple streams.
Thoughts:

1. autolamp; doesn't correctly start/stop with the streams.
2. Put CAM_STREAMER_MAX_CLIENTS in myconfig.h; to stop the inevitable 'I've got 47 streams running and...' type questions, Check current streams vs this in 'app_http.cpp' when a stream is requested and fail + log properly when limit reached.
3. Restore some debug info when stream running, frame size and capture time should be enough; the original stream code gives, maybe, too much info.
4. FPS; currently fixed at 2 and not modifiable by the UI framerate limit setting, a feature added late to the 4.0 release.
   2 fps is a good default but this needs setting from the defaults or preferences file on startup, and then allow it to be modified on the fly by the user via the existing UI dropdown.

I need to think about these, the stream limit and autolamp should be easy'ish, as should getting the debug sorted (there is a debug toggle for the serial output, which can be used in place of the current #defines).
I'm less sure about the fps; though I guess it can be done by passing MIN_FRAME_TIME when the streamer is initialised instead of fps, and a function to set frame_delay on the fly in cam_streamer; invoked as appropriate when the preferences file is read or the user changes the settings.

[daschr]

You've got very good points and each should be doable. I'll take a look on it tomorrow and make a pull request.

[abratchik]

Hi All. May be an old subject but just waned to understand the idea of the multi-client behind the proposed solution a bit better.

I have refactored this whole project in PR #280 and where the frame capture is timer-based. This works pretty well and the frames could be broadcasted to all the connected clients theoretically, this is pretty easy. However, there are several questions to be understood. For example, assume we have 2 clients connected simultaneously, for the sake of simplicity. Let's consider the following scenarios:

1. One has started the video stream while the other decides to take a still image. We could take one frame and send to the 2nd guy, not a problem. However, flash lamp cannot be triggered in this case, because this will affect the streaming for the 1st one.
2. Should we allow 2nd guy to change any settings (camera/frame rate etc) or not? I'd say no, since the 1st client is already running the stream and any changes will affect his experience. This means that all the settings for the 2nd client should not be possible while the stream is running.
3. if the 1st client has started the stream, the second should only be able to connect and see the same image/stream, which is already served for the 1st one. No settings changes should be allowed and no flash photography either.
4. If the 2nd client connects to the stream started by the 1st one, then the 1st one should also be blocked from doing any changes, which may impact the experience for the 2nd guy. This means the 1st client also should not be able to change any settings until and unless he is the only one served.
5. If one of the clients disconnects, and the disconnection message delivery fails for some reason then there have to be some sort of client ping/auto-cleanup on timeout, otherwise the settings will remain locked for the 1st guy even after the 2nd one is dropped.

Overall, I believe implementing this feature in the ESP32CAM sketch doesn't worth the effort, for the following reasons:

1. The solution will be complex and the overall user experience will not be as nice, since parallel clients will be competing for resources of the board, which cannot be shared effectively (i.e. flash lamp, camera sensor etc).
2. Feeding the camera video stream for multiple clients can be implemented on a proxy server much more efficiently and serve as many parallel clients as required, subject to proxy server capacity. In this case, the ESP CAM will have to push the frames to one client only (the proxy server). Moreover, one could also implement a video compression on the proxy so that clients would receive an MPEG stream instead of frame series - this should help on the server bandwidth requirements quite significantly.

Would be great to hear your thoughts on this.