## Stereoscopic Camera for OpenGL · 2010-02-10 02:15 by Black in Graphics Programming

Even to create stereoscopic content digitally, cameras are used. But more than real cameras, a lot of freedom and control lies in the hands of the user. The position and projection can be freely decided without respecting things like the size of the camera, inexactness in the manufacturing of their optics or their weight.

For further reading, see Paul Bourke’s Page on stereo pair creation.

### Camera Theory

Cameras in OpenGL are defined by filling the modelview matrix and the projection matrix with values. The modelview matrix defines the position of the camera relative to the origin or the object space, the projection matrix defines how coordinates in space are mapped to screen.

The projection matrix can be chosen freely, but normally two basic types of cameras are used: Orthographic and Perspective. Perspective cameras create projections very similar to how the human eye sees the world, objects appear smaller the further they are from the camera. Orthographic cameras project objects preserving parallel lines and their proportions. It is mostly used in technical drawings.

### Stereo Pairs

A simple method to use perspective cameras to create stereoscopic footage is to converged their viewing axis. With hardware cameras, this is often used for macro recordings or recordings in closed rooms. The advantage is that the parallax plane is determined when recording, so post-processing needs are low. In addition, the cameras do not have to be as close together as in the next method. The biggest drawback is that the left and right sides of the image do not overlap and have to be cut away or ignored, and that the divergence behind the parallax plane is very strong and can easily lead to unfuseable content. This method should be avoided when ever possible.

A better method is to use perspective cameras with parallel axis. It requires the cameras to be relatively close together and well aligned, both of which is no problem to do in software. Unlike converged cameras, the maximal divergence at infinity is fixed, so even recordings containing far objects can work. The zero parallax plane lies at infinity. It can be moved by creating asymmetric view frustums, effectively horizontally moving both images.

For special visualizations, parallel cameras with converged axis can be used. And similar as with perspective converged cameras, extreme caution has to be taken to not create strongly diverging images. This method should only be used to show objects that are very close to the parallax plane.

### Implementation with OpenGL

As part of ExaminationRoom, I implemented a flexible camera class. The source and header can be downloaded and used relatively freely. As all of my code on this page, they are licensed under the GPL and MIT licenses. This class is not meant to be used directly in an other project since a lot of code is specific to ER, but I am sure the core can be of use as example.

In my implementation, camera positions are defined by their position, their viewing direction, their up-vector and their separation (distance between the cameras). The projection is influenced by the field-of-view, the distance to the zero-parallax plane (the plane where separation of corresponding points is zero) and of course the type of the projection.

camera.h [6.01 kB]

- private:
- Tool::Point pos_;
- Tool::Vector dir_;
- Tool::Vector up_;
- float sep_;
- float fov_;
- float ppd_;
- Tool::ScreenProject * spL_;
- Tool::ScreenProject * spR_;
- Camera::Type type_;

The core of the class is the creation of the matrixes. The call to `glFrustum`

sets the projection matrix, the modelview matrix is created with the utility method `gluLookAt`

. The separation between the cameras has to be considered for both. The camera uses vertical field-of-view, so that the height of the image does not change between standard and widescreen viewport aspect ratios.

camera.cpp [9.43 kB]

- void Camera::loadMatrix(float offsetCamera)
- {
- GlErrorTool::getErrors("Camera::loadMatrix:1");
- GLint viewport[4];
- glGetIntegerv(GL_VIEWPORT, viewport);
- float aspect = (float)viewport[2]/viewport[3];
- float fovTan = tanf((fov_/2)/180*M_PI);
- if (type() == Camera::Perspective)
- {
- // http://local.wasp.uwa.edu.au/~pbourke/projection/stereorender/
- float fTop, fBottom, fLeft, fRight, fNear, fFar;
- // Calculate fNear and fFar based on paralax plane distance hardcoded factors
- fNear = ppd_*nearFactor;
- fFar = ppd_*farFactor;
- // Calculate fTop and fBottom based on vertical field-of-view and distance
- fTop = fovTan*fNear;
- fBottom = -fTop;
- // Calculate fLeft and fRight basaed on aspect ratio
- fLeft = fBottom*aspect;
- fRight = fTop*aspect;
- glMatrixMode(GL_PROJECTION);
- // Projection matrix is a frustum, of which fLeft and fRight are not symetric
- // to set the zero paralax plane. The cameras are parallel.
- glPushMatrix();
- glLoadIdentity();
- glFrustum(fLeft+offsetCamera, fRight+offsetCamera, fBottom, fTop, fNear, fFar);
- glMatrixMode(GL_MODELVIEW);
- glPushMatrix();
- glLoadIdentity();
- // Rotation of camera and adjusting eye position
- Vector sepVec = cross(dir_, up_); // sepVec is normalized because dir and up are normalized
- sepVec *= offsetCamera/nearFactor;
- // Set camera position, direction and orientation
- gluLookAt(pos_.x - sepVec.x, pos_.y - sepVec.y, pos_.z - sepVec.z,
- pos_.x - sepVec.x + dir_.x, pos_.y - sepVec.y + dir_.y, pos_.z - sepVec.z + dir_.z,
- up_.x, up_.y, up_.z);
- GlErrorTool::getErrors("Camera::loadMatrix:2");
- }

The perspective projection is used in most places. For ExaminationRoom, one of the feature requests was the ability to disable selected depth cues. A very strong cue is size relative to the environment. To disable this cue, parallel projection with converged cameras as described above is used instead. The values for the projection matrix were chosen so that the objects at the zero-parallax plane would not change their size when switching between the projection types. The projection matrix is derived from the normal orthographic projection created by OpenGL’s `glOrtho`

by shearing it.

camera.cpp [9.43 kB]

- else if (type() == Camera::Parallel)
- {
- float fTop, fBottom, fLeft, fRight, fNear, fFar;
- // Calculate fNear and fFar based on paralax plane distance and a hardcoded factor
- // Note: the zero paralax plane is exactly in between near and far
- fFar = ppd_*farFactor;
- fNear = 2*ppd_ - fFar; // = ppd_ - (fFar-ppd_);
- // Set fTop and fBottom based on field-of-view and paralax plane distance
- // This is done to make the scaling of the image at the paralax plane the same
- // as in perspective mode
- fTop = fovTan*ppd_;
- fBottom = -fTop;
- // Set left and right baased on aspect ratio
- fLeft = fBottom*aspect;
- fRight = fTop*aspect;
- glMatrixMode(GL_PROJECTION);
- glPushMatrix();
- glLoadIdentity();
- // http://wireframe.doublemv.com/2006/08/11/projections-and-opengl/
- // Note: The code there is wrong, see below for correct code
- // Create oblique projection matrix by shearing an orthographic
- // Projection matrix. Those cameras are converged.
- const float shearMatrix[] = {
- 1, 0, 0, 0,
- 0, 1, 0, 0,
- -offsetCamera/nearFactor, 0, 1, 0,
- 0, 0, 0, 1
- };
- glMultMatrixf(shearMatrix);
- glOrtho(fLeft, fRight, fBottom, fTop, fNear, fFar);
- glMatrixMode(GL_MODELVIEW);
- glPushMatrix();
- glLoadIdentity();
- // Rotation of camera
- // Note: The position of both left and right camera is at the same place
- // because the offset is already calculated by the shearing, which also sets
- // the zero paralax plane.
- gluLookAt(pos_.x, pos_.y, pos_.z,
- pos_.x + dir_.x, pos_.y + dir_.y, pos_.z + dir_.z,
- up_.x, up_.y, up_.z);
- GlErrorTool::getErrors("Camera::loadMatrix:3");
- }

Hopefully this is useful to someone :)