Package com.irurueta.ar.sfm

Class BaseSparseReconstructor<C extends BaseSparseReconstructorConfiguration<C>,R extends BaseSparseReconstructor<C,R,L>,L extends BaseSparseReconstructorListener<R>>

java.lang.Object
com.irurueta.ar.sfm.BaseSparseReconstructor<C,R,L>
Type Parameters:
C - type of configuration.
R - type of re-constructor.
L - type of listener.
Direct Known Subclasses:
BaseSlamSparseReconstructor, KnownBaselineSparseReconstructor, SparseReconstructor
public abstract class BaseSparseReconstructor<C extends BaseSparseReconstructorConfiguration<C>,R extends BaseSparseReconstructor<C,R,L>,L extends BaseSparseReconstructorListener<R>> extends Object
Base class in charge of estimating cameras and 3D reconstructed points from sparse image point correspondences for multiple views.

  • Field Details

    • MIN_NUMBER_OF_VIEWS

      public static final int MIN_NUMBER_OF_VIEWS
      Minimum required number of views.
      See Also:

    • DEFAULT_SCALE

      protected static final double DEFAULT_SCALE
      Default scale.
      See Also:

    • currentMetricEstimatedCamera

      protected EstimatedCamera currentMetricEstimatedCamera
      Current estimated camera in a metric stratum (i.e. up to scale).

    • previousMetricEstimatedCamera

      protected EstimatedCamera previousMetricEstimatedCamera
      Previous estimated camera in a metric stratum (i.e. up to scale).

    • activeMetricReconstructedPoints

      protected List<ReconstructedPoint3D> activeMetricReconstructedPoints
      Reconstructed 3D points which still remain active to match next view in a metric stratum (i.e. up to scale).

    • currentScale

      protected double currentScale
      Current estimated scale. This will typically converge to a constant value as more views are processed. The smaller the variance of estimated scale, the more accurate the scale will be.

    • currentEuclideanEstimatedCamera

      protected EstimatedCamera currentEuclideanEstimatedCamera
      Current estimated camera in euclidean stratum (i.e. with actual scale).

    • previousEuclideanEstimatedCamera

      protected EstimatedCamera previousEuclideanEstimatedCamera
      Previous estimated camera in Euclidean stratum (i.e. with actual scale).

    • activeEuclideanReconstructedPoints

      protected List<ReconstructedPoint3D> activeEuclideanReconstructedPoints
      Reconstructed 3D points which still remain active to match next view in Euclidean stratum (i.e. with actual scale).

    • configuration

      protected C extends BaseSparseReconstructorConfiguration<C> configuration
      Configuration for this re-constructor.

    • listener

      protected L extends BaseSparseReconstructorListener<R> listener
      Listener in charge of handling events such as when reconstruction starts, ends, when certain data is needed or when estimation of data has been computed.

    • failed

      protected volatile boolean failed
      Indicates whether reconstruction has failed or not.

    • running

      protected volatile boolean running
      Indicates whether reconstruction is running or not.

    • currentEstimatedFundamentalMatrix

      private EstimatedFundamentalMatrix currentEstimatedFundamentalMatrix
      Current estimated fundamental matrix.

    • cancelled

      private volatile boolean cancelled
      Indicates whether reconstruction has been cancelled or not.

    • viewCount

      private int viewCount
      Counter of number of processed views.

    • finished

      private boolean finished
      Indicates whether reconstruction has finished or not.

    • allPreviousViewSamples

      private List<Sample2D> allPreviousViewSamples
      All samples (tracked and non-tracked) on previous view.

    • previousViewTrackedSamples

      private List<Sample2D> previousViewTrackedSamples
      Tracked samples on previous view.

    • currentViewTrackedSamples

      private List<Sample2D> currentViewTrackedSamples
      Tracked samples on last processed view (i.e. current view).

    • currentViewNewlySpawnedSamples

      private List<Sample2D> currentViewNewlySpawnedSamples
      New samples on las processed view (i.e. current view).

    • matches

      private final List<MatchedSamples> matches
      Active matches between current and previous views.

    • previousViewId

      private int previousViewId
      ID of previous view.

    • currentViewId

      private int currentViewId
      ID of current view.

  • Constructor Details

    • BaseSparseReconstructor

      protected BaseSparseReconstructor(C configuration, L listener)
      Constructor.
      Parameters:
      configuration - configuration for this re-constructor.
      listener - listener in charge of handling events.
      Throws:
      NullPointerException - if listener or configuration is not provided.

  • Method Details

    • getConfiguration

      public C getConfiguration()
      Gets configuration for this re-constructor.
      Returns:
      configuration for this re-constructor.

    • getListener

      public L getListener()
      Gets listener in charge of handling events such as when reconstruction starts, ends, when certain data is needed or when estimation of data has been computed.
      Returns:
      listener in charge of handling events.

    • isRunning

      public boolean isRunning()
      Indicates whether reconstruction is running or not.
      Returns:
      true if reconstruction is running, false if reconstruction has stopped for any reason.

    • isCancelled

      public boolean isCancelled()
      Indicates whether reconstruction has been cancelled or not.
      Returns:
      true if reconstruction has been cancelled, false otherwise.

    • hasFailed

      public boolean hasFailed()
      Indicates whether reconstruction has failed or not.
      Returns:
      true if reconstruction has failed, false otherwise.

    • isFinished

      public boolean isFinished()
      Indicates whether the reconstruction has finished.
      Returns:
      true if reconstruction has finished, false otherwise.

    • getViewCount

      public int getViewCount()
      Gets counter of number of processed views.
      Returns:
      counter of number of processed views.

    • getCurrentEstimatedFundamentalMatrix

      public EstimatedFundamentalMatrix getCurrentEstimatedFundamentalMatrix()
      Gets estimated fundamental matrix for current view. This fundamental matrix relates current view with the previously processed one.
      Returns:
      current estimated fundamental matrix.

    • getCurrentMetricEstimatedCamera

      public EstimatedCamera getCurrentMetricEstimatedCamera()
      Gets estimated metric camera for current view (i.e. up to scale).
      Returns:
      current estimated metric camera.

    • getPreviousMetricEstimatedCamera

      public EstimatedCamera getPreviousMetricEstimatedCamera()
      Gets estimated camera for previous view (i.e. up to scale).
      Returns:
      previous estimated metric camera.

    • getCurrentEuclideanEstimatedCamera

      public EstimatedCamera getCurrentEuclideanEstimatedCamera()
      Gets estimated euclidean camera for current view (i.e. with actual scale).
      Returns:
      current estimated euclidean camera.

    • getPreviousEuclideanEstimatedCamera

      public EstimatedCamera getPreviousEuclideanEstimatedCamera()
      Gets estimated Euclidean camera for previous view (i.e. with actual scale).
      Returns:
      previous estimated euclidean camera.

    • getActiveMetricReconstructedPoints

      public List<ReconstructedPoint3D> getActiveMetricReconstructedPoints()
      Gets metric reconstructed 3D points (i.e. up to scale) which still remain active to match next view.
      Returns:
      active metric reconstructed 3D points.

    • getActiveEuclideanReconstructedPoints

      public List<ReconstructedPoint3D> getActiveEuclideanReconstructedPoints()
      Gets Euclidean reconstructed 3D points (i.e. with actual scale) which still remain active to match next view.
      Returns:
      active euclidean reconstructed 3D points.

    • getCurrentScale

      public double getCurrentScale()
      Gets current estimated scale. This will typically converge to a constant value as more views are processed. The smaller the variance of estimated scale, the more accurate the scale will be.
      Returns:
      current estimated scale.

    • getPreviousViewTrackedSamples

      public List<Sample2D> getPreviousViewTrackedSamples()
      Gets tracked samples on previous view.
      Returns:
      tracked samples on previous view.

    • getCurrentViewTrackedSamples

      public List<Sample2D> getCurrentViewTrackedSamples()
      Gets tracked samples (from previous view) on current view.
      Returns:
      tracked samples on current view

    • getCurrentViewNewlySpawnedSamples

      public List<Sample2D> getCurrentViewNewlySpawnedSamples()
      Gets new samples (not tracked) on current view.
      Returns:
      new samples on current view.

    • processOneView

      public boolean processOneView()
      Process one view of all the available data during the reconstruction. This method can be called multiple times instead of start() to build the reconstruction step by step, one view at a time. This method is useful when data is gathered on real time from a camera and the number of views is unknown.
      Returns:
      true if more views can be processed, false when reconstruction has finished.

    • isFirstView

      public boolean isFirstView()
      Indicates whether current view is the first view.
      Returns:
      true if current view is the first view, false otherwise.

    • isSecondView

      public boolean isSecondView()
      Indicates whether current view is the second view.
      Returns:
      true if current view is the second view, false otherwise.

    • isAdditionalView

      public boolean isAdditionalView()
      Indicates whether current view is an additional view.
      Returns:
      true if current view is an additional view, false otherwise.

    • start

      public void start()
      Starts reconstruction of all available data to reconstruct the whole scene. If reconstruction has already started and is running, calling this method has no effect. This method is useful when all data is available before starting the reconstruction.

    • cancel

      public void cancel()
      Cancels reconstruction. If reconstruction has already been cancelled, calling this method has no effect.

    • reset

      public void reset()
      Resets this instance so that a reconstruction can be started from the beginning without cancelling current one.

    • postProcessOne

      protected abstract boolean postProcessOne(boolean isInitialPairOfViews)
      Called when processing one frame is successfully finished. This can be done to estimate scale on those implementations where scale can be measured or is already known.
      Parameters:
      isInitialPairOfViews - true if initial pair of views is being processed, false otherwise.
      Returns:
      true if post-processing succeeded, false otherwise.

    • processFirstView

      private boolean processFirstView()
      Processes data for first view.
      Returns:
      true if view was successfully processed, false otherwise.

    • processSecondView

      private boolean processSecondView()
      Processes data for second view.
      Returns:
      true if view was successfully processed, false otherwise.

    • processAdditionalView

      private boolean processAdditionalView()
      Processes data for one additional view.
      Returns:
      true if view was successfully processed, false otherwise.

    • reconstructAndRefineMatches

      private void reconstructAndRefineMatches()
      Reconstructs new 3D points or refines existing ones taking into account existing matches and estimated cameras

    • setUpCameraEstimatorMatches

      private double[] setUpCameraEstimatorMatches(List<com.irurueta.geometry.Point3D> points3D, List<com.irurueta.geometry.Point2D> points2D)
      Setups current matched 3D/2D points to estimate a pinhole camera.
      Parameters:
      points3D - 3D matched points.
      points2D - 2D matched points.
      Returns:
      quality scores for matched points.

    • estimateIntrinsicsDIAC

      private com.irurueta.geometry.PinholeCameraIntrinsicParameters estimateIntrinsicsDIAC()
      Estimates additional camera intrinsics using DIAC (Dual Image of Absolute Conic) method.
      Returns:
      additional camera intrinsics or null if something fails.

    • estimateIntrinsicsDAQ

      private com.irurueta.geometry.PinholeCameraIntrinsicParameters estimateIntrinsicsDAQ()
      Estimates additional cameras intrinsics using DAQ (Dual Absolute Quadric) method.
      Returns:
      additional camera intrinsics or null if something fails.

    • hasEnoughSamplesForCameraEstimation

      private boolean hasEnoughSamplesForCameraEstimation(List<com.irurueta.geometry.Point3D> points3D, List<com.irurueta.geometry.Point2D> points2D)
      Indicates whether there are enough matched points to estimate an additional camera.
      Parameters:
      points3D - 3D matched points to check.
      points2D - 2D matched points to check.
      Returns:
      true if there are enough matched points, false otherwise.

    • hasEnoughMatchesForCameraEstimation

      private boolean hasEnoughMatchesForCameraEstimation(List<MatchedSamples> matches)
      Indicates whether there are enough matches to estimate an additional camera.
      Parameters:
      matches - matches to check.
      Returns:
      true if there are enough matches, false otherwise.

    • hasEnoughSamplesOrMatchesForCameraEstimation

      private boolean hasEnoughSamplesOrMatchesForCameraEstimation(int count)
      Indicates whether there are enough matches or samples to estimate an additional camera.
      Parameters:
      count - number of matches or samples.
      Returns:
      true if there are enough matches or samples, false otherwise.

    • hasEnoughSamplesForFundamentalMatrixEstimation

      private boolean hasEnoughSamplesForFundamentalMatrixEstimation(List<Sample2D> samples)
      Indicates whether there are enough samples to estimate a fundamental matrix.
      Parameters:
      samples - samples to check.
      Returns:
      true if there are enough samples, false otherwise.

    • hasEnoughMatchesForFundamentalMatrixEstimation

      private boolean hasEnoughMatchesForFundamentalMatrixEstimation(List<MatchedSamples> matches)
      Indicates whether there are enough matches to estimate a fundamental matrix.
      Parameters:
      matches - matches to check.
      Returns:
      true if there are enough matches, false otherwise.

    • hasEnoughSamplesOrMatchesForFundamentalMatrixEstimation

      private boolean hasEnoughSamplesOrMatchesForFundamentalMatrixEstimation(int count)
      Indicates whether there are enough matches or samples to estimate a fundamental matrix.
      Parameters:
      count - number of matches or samples.
      Returns:
      true if there are enough matches or samples, false otherwise.

    • estimateFundamentalMatrix

      private boolean estimateFundamentalMatrix(List<MatchedSamples> matches, int viewId1, int viewId2, boolean isInitialPairOfViews)
      Estimates fundamental matrix for provided matches, when 3D points lay in a general non-degenerate 3D configuration.
      Parameters:
      matches - pairs of matches to find fundamental matrix.
      viewId1 - id of first view being related by estimated fundamental matrix.
      viewId2 - id of second view being related by estimated fundamental matrix.
      isInitialPairOfViews - true if fundamental matrix needs to be estimated for the initial pair of views, false otherwise.
      Returns:
      true if estimation succeeded, false otherwise.

    • estimatePlanarFundamentalMatrix

      private boolean estimatePlanarFundamentalMatrix(List<MatchedSamples> matches, int viewId1, int viewId2, boolean isInitialPairOfViews)
      Estimates fundamental matrix for provided matches, when 3D points lay in a planar 3D scene.
      Parameters:
      matches - pairs of matches to find fundamental matrix.
      viewId1 - id of first view being related by estimated fundamental matrix.
      viewId2 - id of second view being related by estimated fundamental matrix.
      isInitialPairOfViews - true if fundamental matrix needs to be estimated for the initial pair of views, false otherwise.
      Returns:
      true if estimation succeeded, false otherwise.

    • getPositionForViewId

      private int getPositionForViewId(int[] viewIds, int viewId)
      Gets position of a view id within provided array of view id's.
      Parameters:
      viewIds - array of view IDs where search is done.
      viewId - view id to be searched.
      Returns:
      position where view id is found or -1 if not found.

    • estimateInitialCamerasAndPoints

      private boolean estimateInitialCamerasAndPoints()
      Estimates initial cameras and reconstructed points.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsDAQAndEssential

      private boolean estimateInitialCamerasAndPointsDAQAndEssential()
      Estimates initial cameras and reconstructed points using the Dual Absolute Quadric to estimate intrinsic parameters and then use those intrinsic parameters with the essential matrix.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsDAQ

      private boolean estimateInitialCamerasAndPointsDAQ()
      Estimates initial cameras and reconstructed points using the Dual Absolute Quadric.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsDIAC

      private boolean estimateInitialCamerasAndPointsDIAC()
      Estimates initial cameras and reconstructed points using Dual Image of Absolute Conic.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsEssential

      private boolean estimateInitialCamerasAndPointsEssential()
      Estimates initial cameras and reconstructed points using the essential matrix and provided intrinsic parameters that must have been set during offline calibration.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • estimateInitialCamerasAndPointsEssential

      private boolean estimateInitialCamerasAndPointsEssential(com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsic1, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsic2)
      Estimates initial cameras and reconstructed points using the essential matrix and provided intrinsic parameters that must have been set during offline calibration.
      Parameters:
      intrinsic1 - intrinsic parameters of 1st camera.
      intrinsic2 - intrinsic parameters of 2nd camera.
      Returns:
      true if cameras and points could be estimated, false if something failed.

    • fixFundamentalMatrix

      private void fixFundamentalMatrix(FundamentalMatrix fundamentalMatrix, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicZeroPrincipalPoint1, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicZeroPrincipalPoint2, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicPrincipalPoint1, com.irurueta.geometry.PinholeCameraIntrinsicParameters intrinsicPrincipalPoint2) throws EpipolarException, com.irurueta.geometry.estimators.NotReadyException
      Fixes fundamental matrix to account for principal point different from zero when using DAQ estimation.
      Parameters:
      fundamentalMatrix - fundamental matrix to be fixed.
      intrinsicZeroPrincipalPoint1 - intrinsic parameters of camera 1 assuming zero principal point.
      intrinsicZeroPrincipalPoint2 - intrinsic parameters of camera 2 assuming zero principal point.
      intrinsicPrincipalPoint1 - intrinsic parameters of camera 1 using proper principal point.
      intrinsicPrincipalPoint2 - intrinsic parameters of camera 2 using proper principal point.
      Throws:
      EpipolarException - if something fails.
      com.irurueta.geometry.estimators.NotReadyException - never happens.