Package com.irurueta.ar.sfm

Class BasePairedViewsSparseReconstructor<C extends BasePairedViewsSparseReconstructorConfiguration<C>,R extends BasePairedViewsSparseReconstructor<C,R,L>,L extends BasePairedViewsSparseReconstructorListener<R>>

java.lang.Object
com.irurueta.ar.sfm.BasePairedViewsSparseReconstructor<C,R,L>
Type Parameters:
C - type of configuration.
R - type of re-constructor.
L - type of listener.
Direct Known Subclasses:
BaseSlamPairedViewsSparseReconstructor, PairedViewsSparseReconstructor
public abstract class BasePairedViewsSparseReconstructor<C extends BasePairedViewsSparseReconstructorConfiguration<C>,R extends BasePairedViewsSparseReconstructor<C,R,L>,L extends BasePairedViewsSparseReconstructorListener<R>> extends Object
Base class in charge of estimating cameras and 3D reconstructed points from sparse image point correspondences in pairs of views. Views are processed in pairs so that fundamental matrix is estimated and pairs of cameras and reconstructed points are computed. Because view pairs are processed separately, the scale of each view pair is estimated individually, hence the scale will need to be

  • Field Details

    • MIN_NUMBER_OF_VIEWS

      public static final int MIN_NUMBER_OF_VIEWS
      Minimum required number of views.
      See Also:

    • DEFAULT_SCALE

      protected static final double DEFAULT_SCALE
      Default scale.
      See Also:

    • currentMetricEstimatedCamera

      protected EstimatedCamera currentMetricEstimatedCamera
      Current estimated camera in a metric stratum (i.e. up to scale).

    • previousMetricEstimatedCamera

      protected EstimatedCamera previousMetricEstimatedCamera
      Previous estimated camera in a metric stratum (i.e. up to scale).

    • metricReconstructedPoints

      protected List<ReconstructedPoint3D> metricReconstructedPoints
      Reconstructed 3D points for current pair of views in a metric stratum (i.e. up to scale).

    • referenceEuclideanTransformation

      protected com.irurueta.geometry.MetricTransformation3D referenceEuclideanTransformation
      Transformation to set reference frame on estimated pair of Euclidean cameras. This is used when estimating a new pair of Euclidean cameras to transform such pair to the location and rotation of the last estimated Euclidean camera so that the first camera of the pair is not referred to the world origin.

    • currentScale

      protected double currentScale
      Current estimated scale. This will typically converge to a constant value as more views are processed. The smaller the variance of estimated scale, the more accurate the scale will be.

    • currentEuclideanEstimatedCamera

      protected EstimatedCamera currentEuclideanEstimatedCamera
      Current estimated camera in euclidean stratum (i.e. with actual scale).

    • previousEuclideanEstimatedCamera

      protected EstimatedCamera previousEuclideanEstimatedCamera
      Previous estimated camera in Euclidean stratum (i.e. with actual scale).

    • euclideanReconstructedPoints

      protected List<ReconstructedPoint3D> euclideanReconstructedPoints
      Reconstructed 3D points for current pair of views in Euclidean stratum (i.e. with actual scale).

    • configuration

      protected C extends BasePairedViewsSparseReconstructorConfiguration<C> configuration
      Configuration for this re-constructor.

    • listener

      protected L extends BasePairedViewsSparseReconstructorListener<R> listener
      Listener in charge of handling events such as when reconstruction starts, ends, when certain data is needed or when estimation of data has been computed.

    • failed

      protected volatile boolean failed
      Indicates whether reconstruction has failed or not.

    • running

      protected volatile boolean running
      Indicates whether reconstruction is running or not.

    • previousViewId

      protected int previousViewId
      ID of previous view.

    • currentViewId

      protected int currentViewId
      ID of current view.

    • lastEuclideanCameraCenter

      protected com.irurueta.geometry.Point3D lastEuclideanCameraCenter
      Center of current Euclidean camera on last view pair.

    • lastEuclideanCameraRotation

      protected com.irurueta.geometry.Rotation3D lastEuclideanCameraRotation
      Rotation of current Euclidean camera on last view pair.

    • lastMetricCameraCenter

      private com.irurueta.geometry.Point3D lastMetricCameraCenter
      Center of current metric camera on last view pair.

    • mLastMetricCameraRotation

      private com.irurueta.geometry.Rotation3D mLastMetricCameraRotation
      Rotation of current metric camera on last view pair.

    • invMetricCameraRotation

      private com.irurueta.geometry.Rotation3D invMetricCameraRotation
      Inverse metric camera rotation. This is reused for memory efficiency.

    • currentEstimatedFundamentalMatrix

      private EstimatedFundamentalMatrix currentEstimatedFundamentalMatrix
      Current estimated fundamental matrix.

    • cancelled

      private volatile boolean cancelled
      Indicates whether reconstruction has been cancelled or not.

    • viewCount

      private int viewCount
      Counter of number of processed views.

    • finished

      private boolean finished
      Indicates whether reconstruction has finished or not.

    • previousViewSamples

      private List<Sample2D> previousViewSamples
      Samples on previous view.

    • currentViewSamples

      private List<Sample2D> currentViewSamples
      Samples on last processed view (i.e. current view).

    • matches

      private final List<MatchedSamples> matches
      Matches between first and current view. Views are always processed in pairs.

    • referenceMetricTransformation

      private com.irurueta.geometry.EuclideanTransformation3D referenceMetricTransformation
      Transformation to set reference frame on estimated pair of metric cameras. This is used when estimating a new pair of metric cameras to transform such pair to the location and rotation of last estimated metric camera so that the first camera of the pair is not referred to the world origin.

  • Constructor Details

    • BasePairedViewsSparseReconstructor

      protected BasePairedViewsSparseReconstructor(C configuration, L listener)
      Constructor.
      Parameters:
      configuration - configuration for this re-constructor.
      listener - listener in charge of handling events.
      Throws:
      NullPointerException - if listener or configuration is not provided.

  • Method Details

    • getConfiguration

      public C getConfiguration()
      Gets configuration for this re-constructor.
      Returns:
      configuration for this reconstructor.

    • getListener

      public L getListener()
      Gets listener in charge of handling events such as when reconstruction starts, ends, when certain data is needed or when estimation of data has been computed.
      Returns:
      listener in charge of handling events.

    • isRunning

      public boolean isRunning()
      Indicates whether reconstruction is running or not.
      Returns:
      true if reconstruction is running, false if reconstruction has stopped for any reason.

    • isCancelled

      public boolean isCancelled()
      Indicates whether reconstruction has been cancelled or not.
      Returns:
      true if reconstruction has been cancelled, false otherwise.

    • hasFailed

      public boolean hasFailed()
      Indicates whether reconstruction has failed or not.
      Returns:
      true if reconstruction has failed, false otherwise.

    • isFinished

      public boolean isFinished()
      Indicates whether the reconstruction has finished.
      Returns:
      true if reconstruction has finished, false otherwise.

    • getViewCount

      public int getViewCount()
      Gets counter of number of processed views.
      Returns:
      counter of number of processed views.

    • getCurrentEstimatedFundamentalMatrix

      public EstimatedFundamentalMatrix getCurrentEstimatedFundamentalMatrix()
      Gets estimated fundamental matrix for current view. This fundamental matrix relates current view with the previously processed one.
      Returns:
      current estimated fundamental matrix.

    • getCurrentEuclideanEstimatedCamera

      public EstimatedCamera getCurrentEuclideanEstimatedCamera()
      Gets estimated euclidean camera for current view (i.e. with actual scale).
      Returns:
      current estimated euclidean camera.

    • getPreviousEuclideanEstimatedCamera

      public EstimatedCamera getPreviousEuclideanEstimatedCamera()
      Gets estimated Euclidean camera for previous view (i.e. with actual scale).
      Returns:
      previous estimated euclidean camera.

    • getEuclideanReconstructedPoints

      public List<ReconstructedPoint3D> getEuclideanReconstructedPoints()
      Gets Euclidean reconstructed 3D points (i.e. with actual scale) for current pair of views.
      Returns:
      active euclidean reconstructed 3D points.

    • getCurrentScale

      public double getCurrentScale()
      Gets current estimated scale. This will typically converge to a constant value as more views are processed. The smaller the variance of estimated scale, the more accurate the scale will be.
      Returns:
      current estimated scale.

    • getPreviousViewSamples

      public List<Sample2D> getPreviousViewSamples()
      Gets samples on previous view.
      Returns:
      samples on previous view.

    • getCurrentViewSamples

      public List<Sample2D> getCurrentViewSamples()
      Gets samples on current view.
      Returns:
      samples on current view.

    • processOneViewPair

      public boolean processOneViewPair()
      Process one view-pair of all the available data during the reconstruction. This method can be called multiple times instead of start() to build the reconstruction step by step, one view pair at a time. This method is useful when data is gathered on real time from a camera and the number of views is unknown.
      Returns:
      true if more views can be processed, false when reconstruction has finished.

    • isFirstViewPair

      public boolean isFirstViewPair()
      Indicates whether current view pair is the first one.
      Returns:
      true if current view pair is the first one, false otherwise.

    • isAdditionalViewPair

      public boolean isAdditionalViewPair()
      Indicates whether current view pair is an additional one.
      Returns:
      true if current view pair is an additional one, false otherwise.

    • start

      public void start()
      Starts reconstruction of all available data to reconstruct the whole scene. If reconstruction has already started and is running, calling this method has no effect. This method is useful when all data is available before starting the reconstruction.

    • cancel

      public void cancel()
      Cancels reconstruction. If reconstruction has already been cancelled, calling this method has no effect.

    • reset

      public void reset()
      Resets this instance so that a reconstruction can be started from the beginning without cancelling current one.

    • getCurrentMetricEstimatedCamera

      protected EstimatedCamera getCurrentMetricEstimatedCamera()
      Gets estimated metric camera for current view (i.e. up to scale).
      Returns:
      current estimated metric camera.

    • getPreviousMetricEstimatedCamera

      protected EstimatedCamera getPreviousMetricEstimatedCamera()
      Gets estimated camera for previous view (i.e. up to scale).
      Returns:
      previous estimated metric camera.

    • getMetricReconstructedPoints

      protected List<ReconstructedPoint3D> getMetricReconstructedPoints()
      Gets metric reconstructed 3D points (i.e. up to scale) for current pair of views.
      Returns:
      active metric reconstructed 3D points.

    • transformPairOfCamerasAndPoints

      protected boolean transformPairOfCamerasAndPoints(boolean isInitialPairOfViews, boolean hasAbsoluteOrientation)
      Transforms cameras on current pair of views so that they are referred to last kept location and rotation and upgrades cameras from metric stratum to Euclidean stratum.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      hasAbsoluteOrientation - true if absolute orientation is required, false otherwise.
      Returns:
      true if cameras were successfully transformed.

    • processFirstViewPair

      private boolean processFirstViewPair()
      Processes data for the first view pair.
      Returns:
      true if view pair was successfully processed, false otherwise.

    • processAdditionalViewPair

      private boolean processAdditionalViewPair()
      Processes data for an additional view pair.
      Returns:
      true if view pair was successfully processed, false otherwise.

    • processViewPair

      private boolean processViewPair(boolean isInitialPairOfViews)
      Processed data for a view pair.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if view pair was successfully processed, false otherwise.

    • hasAbsoluteOrientation

      protected abstract boolean hasAbsoluteOrientation()
      Indicates whether implementations of a re-constructor uses absolute orientation or not.
      Returns:
      true if absolute orientation is used, false, otherwise.

    • hasEnoughSamples

      private boolean hasEnoughSamples(List<Sample2D> samples)
      Indicates whether there are enough samples to estimate a fundamental matrix.
      Parameters:
      samples - samples to check.
      Returns:
      true if there are enough samples, false otherwise.

    • hasEnoughMatches

      private boolean hasEnoughMatches(List<MatchedSamples> matches)
      Indicates whether there are enough matches to estimate a fundamental matrix.
      Parameters:
      matches - matches to check.
      Returns:
      true if there are enough matches, false otherwise.

    • hasEnoughSamplesOrMatches

      private boolean hasEnoughSamplesOrMatches(int count)
      Indicates whether there are enough matches or samples to estimate a fundamental matrix.
      Parameters:
      count - number of matches or samples.
      Returns:
      true if there are enough matches or samples, false otherwise.

    • estimateFundamentalMatrix

      private boolean estimateFundamentalMatrix(List<MatchedSamples> matches, int viewId1, int viewId2)
      Estimates fundamental matrix for provided matches, when 3D points lay in a general non-degenerate 3D configuration.
      Parameters:
      matches - pairs of matches to find fundamental matrix.
      viewId1 - id of first view.
      viewId2 - id of second view.
      Returns:
      true if estimation succeeded, false otherwise.

    • estimatePlanarFundamentalMatrix

      private boolean estimatePlanarFundamentalMatrix(List<MatchedSamples> matches, int viewId1, int viewId2)
      Estimates fundamental matrix for provided matches, when 3D points lay in a planar 3D scene.
      Parameters:
      matches - pairs of matches to find fundamental matrix.
      viewId1 - id of first view.
      viewId2 - id of second view.
      Returns:
      true if estimation succeeded, false otherwise.

    • estimatePairOfCamerasAndPoints

      private boolean estimatePairOfCamerasAndPoints(boolean isInitialPairOfViews)
      Estimates a pair of cameras and reconstructed points.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsDAQAndEssential

      private boolean estimateInitialCamerasAndPointsDAQAndEssential(boolean isInitialPairOfViews)
      Estimates initial cameras and reconstructed points using the Dual Absolute Quadric to estimate intrinsic parameters and then use those intrinsic parameters with the essential matrix.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsDAQ

      private boolean estimateInitialCamerasAndPointsDAQ(boolean isInitialPairOfViews)
      Estimates initial cameras and reconstructed points using the Dual Absolute Quadric.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsDIAC

      private boolean estimateInitialCamerasAndPointsDIAC(boolean isInitialPairOfViews)
      Estimates initial cameras and reconstructed points using Dual Image of Absolute Conic.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsEssential

      private boolean estimateInitialCamerasAndPointsEssential(boolean isInitialPairOfViews)
      Estimates initial cameras and reconstructed points using the essential matrix and provided intrinsic parameters that must have been set during offline calibration.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsEssential

      private boolean estimateInitialCamerasAndPointsEssential(com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsic1, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsic2)
      Estimates initial cameras and reconstructed points using the essential matrix and provided intrinsic parameters that must have been set during offline calibration.
      Parameters:
      intrinsic1 - intrinsic parameters of 1st camera.
      intrinsic2 - intrinsic parameters of 2nd camera.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • keepLastCenterAndRotation

      private boolean keepLastCenterAndRotation()
      Keeps center and rotation of last camera (current camera on previous view pair).
      Returns:
      false if camera and rotation were successfully kept, true otherwise.

    • fixFundamentalMatrix

      private void fixFundamentalMatrix(FundamentalMatrix fundamentalMatrix, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicZeroPrincipalPoint1, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicZeroPrincipalPoint2, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicPrincipalPoint1, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicPrincipalPoint2) throws EpipolarException, com.irurueta.geometry.estimators.NotReadyException
      Fixes fundamental matrix to account for principal point different from zero when using DAQ estimation.
      Parameters:
      fundamentalMatrix - fundamental matrix to be fixed.
      intrinsicZeroPrincipalPoint1 - intrinsic parameters of camera 1 assuming zero principal point.
      intrinsicZeroPrincipalPoint2 - intrinsic parameters of camera 2 assuming zero principal point.
      intrinsicPrincipalPoint1 - intrinsic parameters of camera 1 using proper principal point.
      intrinsicPrincipalPoint2 - intrinsic parameters of camera 2 using proper principal point.
      Throws:
      EpipolarException - if something fails.
      com.irurueta.geometry.estimators.NotReadyException - never happens.