RGB-Depth Processing

Classes
struct	cv::linemod::QuantizedPyramid::Candidate
	Candidate feature with a score. More...
class	cv::linemod::ColorGradient
	Modality that computes quantized gradient orientations from a color image. More...
class	cv::rgbd::DepthCleaner
class	cv::linemod::DepthNormal
	Modality that computes quantized surface normals from a dense depth map. More...
class	cv::linemod::Detector
	Object detector using the LINE template matching algorithm with any set of modalities. More...
struct	cv::linemod::Feature
	Discriminant feature described by its location and label. More...
class	cv::rgbd::ICPOdometry
struct	cv::linemod::Match
	Represents a successful template match. More...
class	cv::linemod::Modality
	Interface for modalities that plug into the LINE template matching representation. More...
class	cv::rgbd::Odometry
struct	cv::rgbd::OdometryFrame
class	cv::linemod::QuantizedPyramid
	Represents a modality operating over an image pyramid. More...
struct	cv::rgbd::RgbdFrame
class	cv::rgbd::RgbdICPOdometry
class	cv::rgbd::RgbdNormals
class	cv::rgbd::RgbdOdometry
class	cv::rgbd::RgbdPlane
struct	cv::linemod::Template

Enumerations
enum	{   cv::rgbd::OdometryFrame::CACHE_SRC = 1,   cv::rgbd::OdometryFrame::CACHE_DST = 2,   cv::rgbd::OdometryFrame::CACHE_ALL = CACHE_SRC + CACHE_DST }
enum	{   cv::rgbd::Odometry::ROTATION = 1,   cv::rgbd::Odometry::TRANSLATION = 2,   cv::rgbd::Odometry::RIGID_BODY_MOTION = 4 }
enum	cv::rgbd::DepthCleaner::DEPTH_CLEANER_METHOD { cv::rgbd::DepthCleaner::DEPTH_CLEANER_NIL }
enum	cv::rgbd::RgbdNormals::RGBD_NORMALS_METHOD {   cv::rgbd::RgbdNormals::RGBD_NORMALS_METHOD_FALS,   cv::rgbd::RgbdNormals::RGBD_NORMALS_METHOD_LINEMOD,   cv::rgbd::RgbdNormals::RGBD_NORMALS_METHOD_SRI }
enum	cv::rgbd::RgbdPlane::RGBD_PLANE_METHOD { cv::rgbd::RgbdPlane::RGBD_PLANE_METHOD_DEFAULT }

Functions
	cv::linemod::QuantizedPyramid::Candidate::Candidate (int x, int y, int label, float score)
	cv::linemod::Feature::Feature (int x, int y, int label)
	cv::linemod::Match::Match (int x, int y, float similarity, const String &class_id, int template_id)
void	cv::linemod::colormap (const Mat &quantized, Mat &dst)
	Debug function to colormap a quantized image for viewing. More...
void	cv::rgbd::depthTo3d (InputArray depth, InputArray K, OutputArray points3d, InputArray mask=noArray())
void	cv::rgbd::depthTo3dSparse (InputArray depth, InputArray in_K, InputArray in_points, OutputArray points3d)
Ptr< Detector >	cv::linemod::getDefaultLINE ()
	Factory function for detector using LINE algorithm with color gradients. More...
Ptr< Detector >	cv::linemod::getDefaultLINEMOD ()
	Factory function for detector using LINE-MOD algorithm with color gradients and depth normals. More...
bool	cv::rgbd::isValidDepth (const float &depth)
bool	cv::rgbd::isValidDepth (const double &depth)
bool	cv::rgbd::isValidDepth (const short int &depth)
bool	cv::rgbd::isValidDepth (const unsigned short int &depth)
bool	cv::rgbd::isValidDepth (const int &depth)
bool	cv::rgbd::isValidDepth (const unsigned int &depth)
void	cv::rgbd::registerDepth (InputArray unregisteredCameraMatrix, InputArray registeredCameraMatrix, InputArray registeredDistCoeffs, InputArray Rt, InputArray unregisteredDepth, const Size &outputImagePlaneSize, OutputArray registeredDepth, bool depthDilation=false)
void	cv::rgbd::rescaleDepth (InputArray in, int depth, OutputArray out)
void	cv::rgbd::warpFrame (const Mat &image, const Mat &depth, const Mat &mask, const Mat &Rt, const Mat &cameraMatrix, const Mat &distCoeff, Mat &warpedImage, Mat *warpedDepth=0, Mat *warpedMask=0)

Detailed Description

Enumeration Type Documentation

anonymous enum

These constants are used to set a type of cache which has to be prepared depending on the frame role: srcFrame or dstFrame (see compute method of the Odometry class). For the srcFrame and dstFrame different cache data may be required, some part of a cache may be common for both frame roles.

Parameters

CACHE_SRC	The cache data for the srcFrame will be prepared.
CACHE_DST	The cache data for the dstFrame will be prepared.
CACHE_ALL	The cache data for both srcFrame and dstFrame roles will be computed.

Enumerator
CACHE_SRC
CACHE_DST
CACHE_ALL

anonymous enum

A class of transformation

Enumerator
ROTATION
TRANSLATION
RIGID_BODY_MOTION

enum cv::rgbd::DepthCleaner::DEPTH_CLEANER_METHOD

NIL method is from Modeling Kinect Sensor Noise for Improved 3d Reconstruction and Tracking by C. Nguyen, S. Izadi, D. Lovel

Enumerator
DEPTH_CLEANER_NIL

enum cv::rgbd::RgbdNormals::RGBD_NORMALS_METHOD

Enumerator
RGBD_NORMALS_METHOD_FALS
RGBD_NORMALS_METHOD_LINEMOD
RGBD_NORMALS_METHOD_SRI

enum cv::rgbd::RgbdPlane::RGBD_PLANE_METHOD

Enumerator
RGBD_PLANE_METHOD_DEFAULT

Function Documentation

cv::linemod::QuantizedPyramid::Candidate::Candidate ( int  x, int  y, int  label, float  score  )

inline

cv::linemod::Feature::Feature ( int  x, int  y, int  label  )

inline

cv::linemod::Match::Match ( int  x, int  y, float  similarity, const String &  class_id, int  template_id  )

inline

void cv::linemod::colormap	(	const Mat &	quantized,
		Mat &	dst
	)

Debug function to colormap a quantized image for viewing.

void cv::rgbd::depthTo3d	(	InputArray	depth,
		InputArray	K,
		OutputArray	points3d,
		InputArray	mask = noArray()
	)

Converts a depth image to an organized set of 3d points. The coordinate system is x pointing left, y down and z away from the camera

Parameters

depth	the depth image (if given as short int CV_U, it is assumed to be the depth in millimeters (as done with the Microsoft Kinect), otherwise, if given as CV_32F or CV_64F, it is assumed in meters)
K	The calibration matrix
points3d	the resulting 3d points. They are of depth the same as depth if it is CV_32F or CV_64F, and the depth of K if depth is of depth CV_U
mask	the mask of the points to consider (can be empty)

void cv::rgbd::depthTo3dSparse	(	InputArray	depth,
		InputArray	in_K,
		InputArray	in_points,
		OutputArray	points3d
	)

Parameters

depth	the depth image
in_K
in_points	the list of xy coordinates
points3d	the resulting 3d points

Ptr<Detector> cv::linemod::getDefaultLINE

(

)

Factory function for detector using LINE algorithm with color gradients.

Default parameter settings suitable for VGA images.

Ptr<Detector> cv::linemod::getDefaultLINEMOD

(

)

Factory function for detector using LINE-MOD algorithm with color gradients and depth normals.

Default parameter settings suitable for VGA images.

bool cv::rgbd::isValidDepth ( const float &  depth )

inline

Checks if the value is a valid depth. For CV_16U or CV_16S, the convention is to be invalid if it is a limit. For a float/double, we just check if it is a NaN

Parameters

depth

the depth to check for validity

bool cv::rgbd::isValidDepth ( const double &  depth )

inline

bool cv::rgbd::isValidDepth ( const short int &  depth )

inline

bool cv::rgbd::isValidDepth ( const unsigned short int &  depth )

inline

bool cv::rgbd::isValidDepth ( const int &  depth )

inline

bool cv::rgbd::isValidDepth ( const unsigned int &  depth )

inline

void cv::rgbd::registerDepth	(	InputArray	unregisteredCameraMatrix,
		InputArray	registeredCameraMatrix,
		InputArray	registeredDistCoeffs,
		InputArray	Rt,
		InputArray	unregisteredDepth,
		const Size &	outputImagePlaneSize,
		OutputArray	registeredDepth,
		bool	depthDilation = false
	)

Registers depth data to an external camera Registration is performed by creating a depth cloud, transforming the cloud by the rigid body transformation between the cameras, and then projecting the transformed points into the RGB camera.

uv_rgb = K_rgb * [R | t] * z * inv(K_ir) * uv_ir

Currently does not check for negative depth values.

Parameters

unregisteredCameraMatrix	the camera matrix of the depth camera
registeredCameraMatrix	the camera matrix of the external camera
registeredDistCoeffs	the distortion coefficients of the external camera
Rt	the rigid body transform between the cameras. Transforms points from depth camera frame to external camera frame.
unregisteredDepth	the input depth data
outputImagePlaneSize	the image plane dimensions of the external camera (width, height)
registeredDepth	the result of transforming the depth into the external camera
depthDilation	whether or not the depth is dilated to avoid holes and occlusion errors (optional)

void cv::rgbd::rescaleDepth	(	InputArray	in,
		int	depth,
		OutputArray	out
	)

If the input image is of type CV_16UC1 (like the Kinect one), the image is converted to floats, divided by 1000 to get a depth in meters, and the values 0 are converted to std::numeric_limits<float>::quiet_NaN() Otherwise, the image is simply converted to floats

Parameters

in	the depth image (if given as short int CV_U, it is assumed to be the depth in millimeters (as done with the Microsoft Kinect), it is assumed in meters)
depth	the desired output depth (floats or double)
out	The rescaled float depth image

void cv::rgbd::warpFrame	(	const Mat &	image,
		const Mat &	depth,
		const Mat &	mask,
		const Mat &	Rt,
		const Mat &	cameraMatrix,
		const Mat &	distCoeff,
		Mat &	warpedImage,
		Mat *	warpedDepth = 0,
		Mat *	warpedMask = 0
	)

Warp the image: compute 3d points from the depth, transform them using given transformation, then project color point cloud to an image plane. This function can be used to visualize results of the Odometry algorithm.

Parameters

image	The image (of CV_8UC1 or CV_8UC3 type)
depth	The depth (of type used in depthTo3d fuction)
mask	The mask of used pixels (of CV_8UC1), it can be empty
Rt	The transformation that will be applied to the 3d points computed from the depth
cameraMatrix	Camera matrix
distCoeff	Distortion coefficients
warpedImage	The warped image.
warpedDepth	The warped depth.
warpedMask	The warped mask.