For the past three days I have been working with Egeniq and SURFNet on Lecture Recorder. The purpose of the project is to provide a low-cost solution for automatic lecture recording for universities. The initial prototype was built by Egeniq and now we are collaborating to take the solution to next level.
The software architecture composes of multiple small daemons, which have a limited set of responsibilities. These daemons communicate using ZeroMQ, so they can be distributed to different machines in the network if needed.
Currently the software takes input from Kinect and that input is processes using OpenNI/NITE to detect skeleton. The head position of the skeleton is projected to a 2D plane, which gives us X and Y coordinates. The coordinates are published in a ZeroMQ PUB socket.
In the initial demo version we have another daemon that subscribes to the coordinate stream and moves Sony EVI-D70 camera using libvisca. The translation from Kinect coordinates to the camera is very primitive at the moment and lacks a lot of features such as smooth tracking. The plan is to add a director component in the middle, which subscribes to the sensor input feed, adds higher level logic and publishes the results onwards on another ZeroMQ socket. This has two advantages: being able to 'unify' the sensor input and to be able to reuse business logic between different controllers.
The software architecture is designed to be pluggable so that in the future more sensors and controllers can be added. It is also possible to chain the components, which enables for example multiple directors in the middle. The architecture makes it very easy to test each component separate as the previous node in the chain can be mocked using simple PHP scripts.
Here are some videos of the initial work using OpenNI/NITE and libvisca:
Skeleton / joint tracking from Mikko Koppanen on Vimeo.
Controlling Sony EVI-D70 using input from Kinect from Mikko Koppanen on Vimeo.
