Michael Frank Deering: VR: HoloCam

Overview

HoloCam was a research prototype 32-channel live video camera I designed and built at Sun to experiment with large array of camera data. It has always been amusing that because my commercial products used polygon rendering, I was considered "anti-image based rendering", while at the same time my HoloCam prototype is often referenced as one of the first experiments in image based rendering. Many other people did commercialize large arrays of cameras for commercial products: a film camera for "frozen in time" effects; several electronic multiple cameras (especially in The Matrix^TM) but nearly all of these came after my first HoloCam work prior to 1992. More recently Marc Lavoy and his students at Stanford has been doing many interesting things with 1 and 2 dimensional arrays of cameras that go well beyond the early HoloCam work.

Concept

The concept is to have a large linear array of a of video cameras. (The more general case described in the patent includes curved arrays, 2D arrays, and 3D arrays of cameras.) With such a linear array, a verity of effects are possible:

• Take a still frame at the same moment in time from all the cameras. When the resulting stack of images is played back sequentially in time, you get a "rotate around a frozen moment in time" effect.
• Take a still frame at the same moment in time from all the cameras. Display the images as a multi-channel stereogram. The results is a stereo hardcopy of the 3D scene.
• Track the movements of a user's head in real-time, and use the predicted future location of the user's left and right eye to mux two of the video channels to a stereo display in front of the user. This was a common demo mode for HoloCam, and allowed a user to look at live scenes in front of the camera in look-around stereo.
• Record all the channels of video, and play them back later from arbitrary selections of cameras. This allowed a number of combination effects to be done. Due to the problems recording the analog video outputs of these early cameras (1 SPARCStation per channel), we couldn't experiment with this mode as much as we wanted to.
• Reshuffle the pixels from multiple different cameras to create images from a virtual point of view (e.g., not limited to where the cameras actually were). Again because of technical limitations (camera calibration) we didn't do much of this.

Image

The link below is to a higher resolution of one of the HoloCam prototypes.

tif ( 5 MB)