See all ideas

Real-time 3D Interactive Telepresence

Submitted on August 24, 2012

The problem

Fast gigabit connections would enable real-time 3D interactive telepresence applications

The solution

Two common 3D graphics representations include triangle meshes and point clouds. Triangle meshes are a set of triangles that roughly approximates objects. They are prevalent in video games, as relatively few triangles need be used to represent objects in-game. This allows fast frame rates, which gamers demand.

Another method to represent 3D objects involves using a set of coloured points known as a point cloud. Point clouds can be thought of as reciprocals to triangle meshes. Whereas triangle meshes contain relatively few triangles and render quickly, point clouds are dense, more accurate, and take longer to render.

Unlike triangle meshes, which may be animated, point clouds do not deform. Users may need to wait several seconds before a point cloud is downloaded, processed and ready to be viewed. Once ready, the user can interact with the cloud. However, the cloud will remain as a rigid body.

The introduction of fast gigabit connections opens up an entirely new medium for using point clouds. With such bandwidth, several point clouds could be transferred one after another. This would allow the creation and use of real-time 3D interactive telepresence applications. This has tremendous implications for the fields of education and health. For example, physicians working remotely could get real-time 3D views of their patients. Educators could record instruction in 3D and allow students to play this data back interactively.

In order to achieve this, an array of MicroSoft Kinect sensors could be used for the recording of point cloud data. Plug-ins would be necessary for the browser to communicate with the sensors and capture the data. Once the data arrives at the destination, the client application can render the visual data using WebGL and audio data using WebRTC.

How will your idea make people's live's better?

Educators could record their instruction in 3D and allow students to play back the data using different viewpoints. Physicians working remotely could more easily interact with- and get a 3D view of their patients. Video conferencing can be enhanced to 3D in the browser.

How does your idea take advantage of next-generation networks?

The current state of bandwidth limitations prevent such an application from being practically usable.

Andor Salga

I graduated from the Bachelor's of Software Development degree program at Seneca College. I also hold a diploma in Computer Programming and Analysis, specializing in 3D game development using C/C++. I worked as a research assistant and technical advisor at Seneca's Centre for Development of Open Technology developing open source WebGL libraries such as Processing.js, XB PointStream and C3DL.

comments powered by Disqus