Sunday, October 6, 2013

Raspbot helps find my cofee

Ever lost your coffee and needed some help finding it? I have and it can be a stressful situation, as you don't have the coffee to help calm your nerves. That is where the raspbot jumps in to save the day. Just make sure you have a distinct enough mug, so that it does not get lost in the background. This makes use of OpenCV and object detection. It streams the video from a raspberry pi to a server, which is running OpenCV and has a GPU. Then GPU is then used to decode the H264 stream. Then either the GPU or the CPU is used to analyze a frame for interesting information (in this case, where is the mug). OpenCV has GPU and CPU support, so you can choose.
System overview

Here is the code for the server https://github.com/arcanon/raspbot/blob/master/video_reader.cpp. The other core code is also in that repository. You obviously need to OpenCV. I was running on windows, you have to fight a bit to get it compiled on windows, I will give some tips at some point on that, maybe its also easier with newer versions.

Capture Thread:
1.) Winsock buffer connects to netcat client on PI
2.) Buffers data and passes to H264StreamSource
3.) Data then Passed to GPU for decoding

Detection Thread: 
1.) Frame from capture thread recieved
2.) Converts to grey scale
3.) uses ORB/SURF for feature extraction. On CPU or GPU.
4.) Compares against detection object.
5.) Sets up commands to send to PI

Command Thread: 
1.) Connects to Python client on PI
2.) When Detection thread has commands, these are sent to the PI.
Server Threads

Raspberry PI threads/processes

No comments: