PointCloudRegistration.jl

PointCloudRegistration.jl is a package for performing rigid and nonrigid registration of point clouds, also known as superimposing or aligning.

Point clouds

By point cloud, we refer to a set of points, i.e. a subset of $\mathbb{R}^N$ for some integer $N$. We then also say that it is $N$-dimensional. A point cloud can represent the atom coordinates of a molecule, the surface of a 3D object, or many other things.

Computationally, we can store them as lists of vectors of equal length. In fact, this package represents point clouds as a subtype of AbstractVector{SVector{N, T}} via its PointCloud type that can optionally also have weights for the points. It is often convenient to construct a PointCloud from a matrix where every column is one point.

Let us create two point clouds source and target where target is a rotated and shifted version of source:

julia> using PointCloudRegistration
julia> source = [1.0:5.0 zeros(5)]'2×5 adjoint(::Matrix{Float64}) with eltype Float64: 1.0 2.0 3.0 4.0 5.0 0.0 0.0 0.0 0.0 0.0
julia> angle = deg2rad(20)0.3490658503988659
julia> rotation = [cos(angle) -sin(angle); sin(angle) cos(angle)]2×2 Matrix{Float64}: 0.939693 -0.34202 0.34202 0.939693
julia> translation = [13., 42.]2-element Vector{Float64}: 13.0 42.0
julia> target = rotation * coords .+ translationERROR: UndefVarError: `coords` not defined in `Main` Suggestion: check for spelling errors or missing imports.

How can we recover rotation and translation from source and target?

API

PointCloudRegistration.PointCloudType

Representation of a weighted point cloud. An instance pc of PointCloud{N, T} stores N-dimensional points with coordinate type T. Also, it acts as an AbstractMatrix{T} with N rows and length(pc.points) columns.

The purpose of this type is to bring the input data into a form that enables efficient execution of the registration algorithms. As for the public interface, you can consider PointCloud to be defined as

struct PointCloud{N, T} <: AbstractMatrix{T}
    points::AbstractVector{SVector{N, T}}
    weights::AbstractVector{<: Real}
    # ... private fields ...
end

with the SVector type from StaticArrays.jl.

Simple usage

You can construct a PointCloud from a matrix or a vector of vectors:

julia> PointCloud([1. 2. 3.; 4. 5. 6.])
2-dimensional point cloud with 3 points of eltype Float64
 1.0  2.0  3.0
 4.0  5.0  6.0
and unit weights

julia> PointCloud([[1., 4.], [2., 5.], [3., 6.]])
2-dimensional point cloud with 3 points of eltype Float64
 1.0  2.0  3.0
 4.0  5.0  6.0
and unit weights

Type stability

Since the dimensionality of the point cloud is part of the type, the two calls above are not type stable (unless the dimensionality can be inferred, e. g. when the number of rows of the matrix is statically known). This is not a big deal because every function operating on PointClouds is then type stable (function barrier). If you really want the construction of a PointCloud to be type stable, you can use the PointCloud{N} variant:

julia> PointCloud{2}([1. 2. 3.; 4. 5. 6.])
2-dimensional point cloud with 3 points of eltype Float64
 1.0  2.0  3.0
 4.0  5.0  6.0
and unit weights

julia> PointCloud{3}([1. 2. 3.; 4. 5. 6.])
ERROR: ArgumentError: matrix must have given number of rows 3
[...]

julia> isconcretetype(Core.Compiler.return_type(PointCloud, Tuple{Matrix{Float64}}))
false

julia> isconcretetype(Core.Compiler.return_type(PointCloud{2}, Tuple{Matrix{Float64}}))
true

Weights

Optionally, each point can have an individual non-negative weight.

julia> PointCloud([1. 2. 3.; 4. 5. 6.], [.5, 1., .5])
2-dimensional point cloud with 3 points of eltype Float64
 1.0  2.0  3.0
 4.0  5.0  6.0
and weights
 3-element Vector{Float64}
 0.5  1.0  0.5

If no weights are specified, implicit unit weights are used. (They are not explicitly stored and have minimal runtime cost for computations.)

PointCloudRegistration.PointCloudMethod
PointCloud(points)
PointCloud{N}(points)

Create a PointCloud from the given points (matrix or vector of vectors) and use implicit unit weights. Optionally provide the dimensionality N for better type inference.

PointCloudRegistration.PointCloudMethod
PointCloud(points, weights)
PointCloud{N}(points, weights)

Create a PointCloud from the given points (matrix or vector of vectors) and weights. Optionally provide the dimensionality N for better type inference.

PointCloudRegistration.register_rmsdFunction
register_rmsd(source, target)

Find a CoordinateTransformations.AffineMap that rotates and translates source in a way that minimizes the Root Mean Square Distance to target, i.e.

\[% \operatorname{arg\;min}\limits_{ % \text{rotation } R \text{ and translation } t % } \sqrt{ \frac{1}{n} \sum_{i = 1}^n \Vert R y_i + t - x_i \Vert^2 }\]

for source points $y_i$, target points $x_i$, rotation $R$, and translation $t$. This assumes that the $i$-th point in source corresponds to the $i$-th point in target, so source and target must have the same size.

source and target can each either be matrices with one point per column or PointClouds.

This is a konvex optimization problem with a closed form solution (Kabsch algorithm) but is susceptible to outliers or wrong correspondences.

Use this function if you do not expect outliers or wrong correspondences and you need maximum speed.

PointCloudRegistration.register_gmcFunction
register_gmc(source, target[; scale, restarts, iterations, rng, accumulator])

Find a CoordinateTransformations.AffineMap that rotates and translates source in a way that minimizes the Geman-McClure loss to target, i.e.

\[\sum_{i = 1}^n \operatorname{GMC}_\rho(\Vert R y_i + t - x_i \Vert)\]

where

\[\operatorname{GMC}_\rho(r) = \frac{r^2}{\rho^2 + r^2}\]

for source points $y_i$, target points $x_i$, rotation $R$, and translation $t$. This assumes that the $i$-th point in source corresponds to the $i$-th point in target, so source and target must have the same size.

source and target can each either be matrices with one point per column or PointClouds.

The Geman-McClure loss has a scale parameter $\rho$ that defines the range of distances that affect the loss (making it robust against outliers). $\rho$ corresponds to the scale keyword argument and you can learn about how to use the keyword arguments in this section.

Use this function if you expect outliers or wrong correspondences.

PointCloudRegistration.register_kcFunction
register_kc(source, target[; scale, axisalign, restarts, iterations, rng, accumulator])

Find a CoordinateTransformations.AffineMap that rotates and translates source in a way that maximizes the Kernel Correlation to target, i.e.

\[\sum_{i = 1}^n \sum_{j = 1}^m \exp\left(- \frac{\Vert R y_i + t - x_i \Vert}{2 \sigma^2}\right)\]

for source points $y_i$, target points $x_i$, rotation $R$, and translation $t$.

source and target can each either be matrices with one point per column or PointClouds.

The bandwidth parameter $\sigma$ corresponds to the scale keyword argument and you can learn about how to use the keyword arguments in this section.

Use this function if you do not know correspondences.

register_kc(source, prepared_target[; restarts, iterations, rng, accumulator])

Find a CoordinateTransformations.AffineMap that rotates and translates source in a way that maximizes the Kernel Correlation to prepared_target which was computed by prepare_target_kc.

Use this function if you plan to register multiple sources to the same target.

PointCloudRegistration.prepare_target_kcFunction
prepare_target_kc(target[; scale, axisalign])

Perform all the necessary precomputation for register_kc. This function is especially useful if you plan to register multiple sources to the same target.

X = PointCloud(rand(3, 100))
Y1 = PointCloud(rand(3, 150))
Y2 = PointCloud(rand(3, 130))

prepd_X = prepare_target_kc(X) # this takes some time

T1 = register_kc(prepd_X, Y1) # this is fast
T2 = register_kc(prepd_X, Y2) # this is fast

Common keyword arguments

This packages implements registration with respect to two non-convex losses/scores. While they are inherently different, the respective register_gmc and register_kc functions share a common interface to deal with this non-convexity, namely certain keyword arguments:

scale

Default: TargetScales(5)

Both the Geman-McClure loss and the Kernel Correlation have a scale parameter that determines to what distances they are sensitive to. Thus, the scale should eventually take a value that is relevant for the application at hand. In the simplest case, you can just set to a scalar value.

However, optimizing the rotation and translation can easily get stuck in a non-global optimum when starting with a scale too small. Too avoid that, you can specify how the scale should be successively decreased. For maximum control, you can set scale to any AbstractVector{<: Real}.

If you are unsure what values are sensible to use, two heuristics are implemented.

PointCloudRegistration.DownToType
DownTo(scale, [steps = 5])

Annealing plan starting from the largest standard deviation of the target point cloud in any direction, going down to scale in steps steps with logarithmic progression.

PointCloudRegistration.TargetScalesType
TargetScales([steps = 5])

Annealing plan starting from the largest standard deviation of the target point cloud in any direction, going down to average nearest neighbor distance in the target, in steps steps with logarithmic progression.

restarts

Default: 5

The non-convex nature of the optimization makes it prone to run into local optima. The parameter restarts specifies how often to perform a restart from a random initial transformation.

iterations

Default: 10

How many optimizaton iterations to perform per restart. For register_kc, fewer iterations may happen if convergence occurs.

rng

Default: Random.default_rng()

The random number generator to use to generate random initializations for each restart. You can specify this keyword argument to ensure reproducible initializations.