[ Up ]

ALE 3D Scene Reconstruction

In general, 3D modeling of scenes may allow handling of cases that per-frame projective transformations do not allow, such as registering frames having different viewpoints over a non-flat or specular scene, or recovery of depth from frames having different viewpoints. ALE's implementation of 3D scene reconstruction is considered experimental in versions prior to 0.8.3.

Models

The method of modeling used in 0.8.2 and later is spatial subdivision based on regions of interest, followed by iterative update of the estimated occupancy and color of each region of interest. (For a general discussion of occupancy, see Hans Moravec's work on depth recovery.) In 0.8.1, the method of modeling is decomposition and deformation of a surface. In earlier versions, a per-frame, per-pixel model of depth is used.

Recovered Color and Depth Output

The color and depth output options provide a reconstruction of the scene as viewed from a particular frame's vantage point, or from an arbitrary camera view [0.8.2 and later]. In 0.8.2 and later, normalization of output values by accumulated occupancy can optionally be disabled. Version 0.8.3 adds support for options --et and --3dpx, which can be useful for 3D cropping applications. --et specifies the minimum accumulated weight required for a pixel to be defined, allowing refinements of crops based on clip planes (--rc and --fc) established prior to scene reconstruction, whereas --3dpx specifies a set of depths for which pixels are considered undefined, where depth calculation is performed after scene reconstruction.

--3dv <n> <o>     Render, to file <o>, colors as viewed from frame <n>.
--3dd <n> <o>     Render, to file <o>, depths as viewed from frame <n>.
--3dvp <args> <o> Render, to file <o>, colors viewed with projective <args>.  [0.8.2 and later]
--3ddp <args> <o> Render, to file <o>, depths viewed with projective <args>.  [0.8.2 and later]
                    <args> are W H V x y z P Y R:
                         W    image width.
                         H    image height.
                         V    camera view angle.
                         x    translation x component.
                         y    translation y component.
                         z    translation z component.
                         P    rotation around x-axis.
                         Y    rotation around y-axis.
                         R    rotation around z-axis.
--occ-norm        Normalize output with accumulated occupancy.          [default]     [0.8.2 and later]
--occ-nonorm      Don't normalize output with accumulated occupancy.                  [0.8.2 and later]
--et <x>          Set encounter threshold <x> for defined pixels.  [default is 0]     [0.8.3 and later]
--3dpx <args>     Exclude a specified spatial volume following full-scene             [0.8.3 and later]
                  reconstruction, assuming a single depth at each point
                  in the output.  <args> are:
                         <xmin> <xmax> <ymin> <ymax> <zmin> <zmax>

Camera parameter adjustment

In versions 0.8.1 and later, camera parameters can be adjusted according to control points to decrease the error between points in different frames. Perturbation bounds for adjustment can be specified, including an additional upper bound on view angle perturbation, and a stereo threshold can be set for discarding control points after adjustment is complete. An initial camera view angle can also be specified. Versions 0.8.2 and later allow calculation of error by median rather than mean, and also allow disabling of adjustment of orientation and/or position.

--view-angle <x>  Set the initial diagonal view angle to <x> degrees.
                     (Default is 43.7 degrees or device-specific.)
--cpp-upper=<x>   Set upper bound <x> for camera parameter perturbation,           [0.8.1 and later]
                  in pixels or degrees [default is 32]                            
--cpp-lower=<x>   Set lower bound <x> for camera parameter perturbation,           [0.8.1 and later]
                  in pixels or degrees [default is 0.125]                    
--va-upper=<x>    View-angle perturbation upper bound in degrees  [default is 32]  [0.8.1 and later]
--st <x>          Control point stereo threshold is <x> pixels.    [default is 4]  [0.8.1 and later]
--cpp-err-mean    Use RMS error to determine camera parameters.         [default]        [0.8.2 and later]
--cpp-err-median  Use median error to determine camera parameters.                       [0.8.2 and later]
--vp-adjust       Adjust the view point                                 [default]        [0.8.2 and later]
--vp-noadjust     Do not adjust the view point                                           [0.8.2 and later]
--vo-adjust       Adjust the view orientation                           [default]        [0.8.2 and later]
--vo-noadjust     Do not adjust the view orientation                                     [0.8.2 and later]

Transformation data files [0.8.2 and later]

Transformation data file options allow reading and writing of camera position, orientation, and view angle information.
--3d-trans-load=x Load initial transformation settings from file x                       [0.8.2 and later]
--3d-trans-save=x Save final transformation data in file x                               [0.8.2 and later]

Transformation data file format [0.8.2 and later]

The transformation data file uses the following format, starting with a version command, and then describing view angle and euclidean transformation of each frame in turn:

W 0
V <view angle>
E <width> <height> <X> <Y> <Z> <P> <Y> <R>
V <view angle>
E <width> <height> <X> <Y> <Z> <P> <Y> <R>
.
.
.
V <view angle>
E <width> <height> <X> <Y> <Z> <P> <Y> <R>

Model costs [0.8.1 only]

After the cameras are adjusted, a 2D surface is decomposed and deformed to reduce local cost measures, calculated from angles between neighboring triangles, edge lengths, and differences in color between different views. Multipliers for edge length and angle costs can be specified on the command line.

--ecm <x>         Set edge length cost multiplier <x>.         [default is 0.001]  [0.8.1 only]
--acm <x>         Set angle cost multiplier <x>.               [default is 0.001]  [0.8.1 only]

Model rules [0.8.2 and later]

In versions 0.8.2 and later, clip planes can be specified, and a color falloff exponent for pairwise occupancy estimation can be specified. Larger exponents are more forgiving of color differences between pixels in different frames. In versions 0.8.3 and later, error contributions from cameras outside of the generating pair can be assigned non-zero multipliers (--tcem), the number of occupancy and color update iterations can be specified (--oui), the number of points in space generated for each pixel pair can be specified (--pa), and the type of frame pairings can be specified (--pc).

--fc <x>          Set front-clip to <x> (0 < x < 1)                [default is 0]  [0.8.2 and later]
--rc <x>          Set rear-clip to <x> (1 < x < inf)             [default is inf]  [0.8.2 and later]
--fx <x>          Set falloff exponent to <x>                      [default is 0]  [0.8.2 and later]
--tcem <x>        Set third-camera error multiplier to <x>         [default is 0]  [0.8.3 and later]
--oui <x>         Set occupancy update iterations to <x>          [default is 10]  [0.8.3 and later]
--pa <x>          Set pairwise ambiguity to <x>                    [default is 3]  [0.8.3 and later]
--pc <type>       Set the type of pairwise comparisons:                            [0.8.3 and later]
                     auto     Determine comparisons automatically.      [default]
                     all      Perform all comparisons.



Copyright 2004, 2005 David Hilvert

Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.