(An Ongoing Project)


Image and Video Processing Lab, The Chinese University of Hong Kong


1. System description


We aim at developing a system that can deliver both 3D and free-view experiences to the end user. The system consists of a server and a client. At the server side, two color-depth cameras (Kinect) are used to capture the scene. Each camera records one view with both color and depth information. The two views of the scene together with the camera calibration results are transmitted over the internet to the client. At the client side, the system detects the head position of the viewer, and displays the corresponding view to create the free-view experience, that is, when the viewer moves his/her head, the synthesized view will change accordingly as if he/she is watching the scene on site. To enhance the immersive feeling, the system will display the scene in 3D mode.


2. System illustration



3. Technical components

4. Current status


Table 1. A prototype system has been developed with the following processing components



5. Future works


Research works are being carried out focusing on several technical topics: multiple cameras calibration, depth enhancement, rendering, and head pose estimation.


6. Group members


-   PostDoc: Songnan Li (snli@ee.cuhk.edu.hk)

-   PhD student: Lu Sheng (lsheng@ee.cuhk.edu.hk)

-   Supervisor: King Ngi Ngan (knngan@ee.cuhk.edu.hk)


7. References


[1] Z. Zhang, "A flexible new technique for camera calibration," TPAMI, 22(11), pp. 1330-1334, 2000.

[2] A fast JPEG coding library - libjpeg turbo, available at: http://libjpeg-turbo.virtualgl.org 

[3] A fast JPEG-LS coding library - CharLS, available at: http://charls.codeplex.com

[4] Boost.Asio library for network transmission, available at: http://www.boost.org

[5] OpenNI, available at: http://openni.org

[6] N. Fukushima's code, available at: http://nma.web.nitech.ac.jp/fukushima/openni/NICVSample4.cpp

[7] NVIDIA 3D vision, http://www.nvidia.com/object/3d-vision-main.html