New Media Laboratory
Our customer is a consulting and development laboratory that provides auto active, reactive and interactive objects and installations in media–based environments and architectures. The interaction is organized in museums, exhibitions, and special proof–of–concept technology rooms.
T–Gallery is a special room located in Bonn, Germany to demonstrate coordinated interaction of a wide set of technology prototypes in telecommunication and new media areas.
The project is intended to integrate different mobile, visual, telecommunication, and speech and hand motion recognition facilities in a joint scenario–based environment.
One of the tasks was to arrange a video conference when design, management, and usability personnel are engaged in interactive communication on the design of a new car. They needed to see each other while any background (i.e. furniture or other people) was to be cut–off from video streams. Manipulations with a 3D car model using rotation, pointing out or scaling were to be enabled. Some of the conference participants could communicate using mobile devices (to see video and manipulate the object), while others were supposed to participate in discussions and operate the model by hand motions in front of a special wall with integrated screen, video, and audio equipment.
This problem was subdivided into the following parts:
- Provide reliable and low–latency multi–person video conferencing using the latest HTML5 (WebRTC) technologies
- Manage start / stop of the online conference and monitor the list of participants using an internal scenario–based server (written on Ruby on Rails with EventMachine support)
- Analyze how to integrate KINNECT with HTML5 video streams to cut–off the background image that may include other people present in the room, equipment, and furniture
- Analyze possibilities of implementing video conferencing on mobile devices such as Android and Chrome on Android
- Analyze how a 3D modeling system can provide a video stream to the conference
- Perform an automatic test of the application on mobile devices. The logic includes selection of a car and rotation of a model.
The project was implemented in several stages. The 1st stage was to analyze the existing infrastructure and use cases of the laboratory, and to provide new ideas. After the 1st stage was completed, IBA received the task to investigate the current capabilities of HTML5 with regard to video conferencing, and to improve the existing environment for testing on mobile devices. The 3d stage of the project was to develop an appropriate library, demonstrate video conferencing and test the application on mobile devices, as well as provide research documents and a roadmap for future development.
- JS (google closure, js)
- Node JS
- Java (netty server, special protocol and video libraries)
- Android SDK
Operation Systems and Tools:
- Ubuntu Linux
- Oracle Virtual Box
IBA Group worked on the customer site in cooperation with the existing design, management, and development teams.
The code repository server (git) and the virtualization server that hosts service bundles are located at the customer site.
Daily meetings and weekly status reports allowed for the efficient management and implementation of the project.
IBA Group successfully analyzed complex media and technological areas, provided a proof of the concept library, and demonstrated prototypes for the task of a multi–person video communication and automated testing on a mobile environment.
The KINNECT, mobile, and 3D modeling integration problems were analyzed, as well and appropriate roadmaps were created. All the tasks were executed in a comparatively short time period and with good quality, as reported by the customer.