Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: first cut at object detection on ios #14

Merged
merged 1 commit into from
Mar 27, 2024

Conversation

cdiddy77
Copy link
Owner

Object detection inference working (a little) on ios

Similar to PR #11 this is exactly enough code to prove that we are getting inferences from mediapipe on iOS

What is working

  • If you run the demo app, you may see a very slow frame rate camera. More significantly, in the logging terminal, you will see logging spew indicating that the javascript layer is receiving inference results from the ios layer. These inferences may or may not be correct. But they are inferences that are the result of sending video frames to mediapipe's ObjectDetection task.

What needs doing

  • Need to verify that the camera frame rate is still 30fps, even though the inferences aren't flowing quite that fast.

The details

  • On iOS, similar to Android, there is an ObjectDetectionModule and an ObjectDetectionFrameProcessorPlugin. ObjectDetectionModule has methods for creating and releasing an ObjectDetector, and fires events when object detectors produce results or errors. The frame processor plugin forwards the frame to the underlying ObjectDetectorHelper which forwards to MediaPipe. When MediaPipe generates a result or an error, those are forwarded through the event mechanism.

@cdiddy77 cdiddy77 requested a review from dwgray March 25, 2024 22:33
@cdiddy77 cdiddy77 merged commit 2172df5 into main Mar 27, 2024
5 checks passed
@cdiddy77 cdiddy77 deleted the object-detection-ios-start branch March 27, 2024 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant