Class PinholeCameraRefiner<S1,S2>

Type Parameters:
S1 - type of matched samples in 1st set.
S2 - type of matched samples in 2nd set.
Direct Known Subclasses:
LinePlaneCorrespondencePinholeCameraRefiner, PointCorrespondencePinholeCameraRefiner

public abstract class PinholeCameraRefiner<S1,S2> extends PairMatchesAndInliersDataRefiner<PinholeCamera,S1,S2>
Base class for pinhole camera refiners. Implementations of this class refine a pinhole camera by taking into account an initial estimation, inlier point or line matches and their residuals. This class can be used to find a solution that minimizes error of inliers in LMSE terms. Typically, a refiner is used by a robust estimator, however it can also be useful in some other situations.
  • Field Details

    • DEFAULT_SUGGEST_SKEWNESS_VALUE_ENABLED

      public static final boolean DEFAULT_SUGGEST_SKEWNESS_VALUE_ENABLED
      Default value indicating whether skewness value is suggested or not. By default, this is disabled.
      See Also:
    • DEFAULT_SUGGESTED_SKEWNESS_VALUE

      public static final double DEFAULT_SUGGESTED_SKEWNESS_VALUE
      Default value of skewness to be suggested when suggestion is enabled. By default suggested skewness is zero.
      See Also:
    • DEFAULT_SUGGEST_HORIZONTAL_FOCAL_LENGTH_ENABLED

      public static final boolean DEFAULT_SUGGEST_HORIZONTAL_FOCAL_LENGTH_ENABLED
      Default value indicating whether horizontal focal length value is suggested or not. By default, this is disabled.
      See Also:
    • DEFAULT_SUGGEST_VERTICAL_FOCAL_LENGTH_ENABLED

      public static final boolean DEFAULT_SUGGEST_VERTICAL_FOCAL_LENGTH_ENABLED
      Default value indicating whether vertical focal length value is suggested or not. By default, this is disabled.
      See Also:
    • DEFAULT_SUGGEST_ASPECT_RATIO_ENABLED

      public static final boolean DEFAULT_SUGGEST_ASPECT_RATIO_ENABLED
      Default value indicating whether aspect ratio is suggested or not. By default, this is disabled.
      See Also:
    • DEFAULT_SUGGESTED_ASPECT_RATIO_VALUE

      public static final double DEFAULT_SUGGESTED_ASPECT_RATIO_VALUE
      Default value of aspect ratio to be suggested when suggestion is enabled. By default, suggested aspect ratio is 1.0, although also -1.0 is a typical value when vertical coordinates increase downwards.
      See Also:
    • DEFAULT_SUGGEST_PRINCIPAL_POINT_ENABLED

      public static final boolean DEFAULT_SUGGEST_PRINCIPAL_POINT_ENABLED
      Default value indicating whether principal point is suggested or not. By default, this is disabled.
      See Also:
    • DEFAULT_SUGGEST_ROTATION_ENABLED

      public static final boolean DEFAULT_SUGGEST_ROTATION_ENABLED
      Default value indicating whether rotation is suggested or not. By default, this is disabled.
      See Also:
    • DEFAULT_SUGGEST_CENTER_ENABLED

      public static final boolean DEFAULT_SUGGEST_CENTER_ENABLED
      Default value indicating whether center is suggested or not. By default, this is disabled.
      See Also:
    • refinementStandardDeviation

      protected double refinementStandardDeviation
      Standard deviation used for Levenberg-Marquardt fitting during refinement. Returned value gives an indication of how much variance each residual has. Typically, this value is related to the threshold used on each robust estimation, since residuals of found inliers are within the range of such threshold.
    • suggestSkewnessValueEnabled

      private boolean suggestSkewnessValueEnabled
      Indicates whether skewness value is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedSkewnessValue

      private double suggestedSkewnessValue
      Suggested skewness value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • suggestHorizontalFocalLengthEnabled

      private boolean suggestHorizontalFocalLengthEnabled
      Indicates whether horizontal focal length is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedHorizontalFocalLengthValue

      private double suggestedHorizontalFocalLengthValue
      Suggested horizontal focal length value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • suggestVerticalFocalLengthEnabled

      private boolean suggestVerticalFocalLengthEnabled
      Indicates whether vertical focal length is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedVerticalFocalLengthValue

      private double suggestedVerticalFocalLengthValue
      Suggested vertical focal length value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • suggestAspectRatioEnabled

      private boolean suggestAspectRatioEnabled
      Indicates whether aspect ratio is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedAspectRatioValue

      private double suggestedAspectRatioValue
      Suggested aspect ratio value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • suggestPrincipalPointEnabled

      private boolean suggestPrincipalPointEnabled
      Indicates whether principal point is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedPrincipalPointValue

      private InhomogeneousPoint2D suggestedPrincipalPointValue
      Suggested principal point value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • suggestRotationEnabled

      private boolean suggestRotationEnabled
      Indicates whether camera rotation is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedRotationValue

      private Quaternion suggestedRotationValue
      Suggested rotation to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • suggestCenterEnabled

      private boolean suggestCenterEnabled
      Indicates whether camera center is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
    • suggestedCenterValue

      private InhomogeneousPoint3D suggestedCenterValue
      Suggested center to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
    • residualIntrinsic

      private PinholeCameraIntrinsicParameters residualIntrinsic
      Instance to be reused to compute residual for intrinsic parameters.
    • residualPrincipalPoint

      private InhomogeneousPoint2D residualPrincipalPoint
      Instance to be reused to compute residual on principal point.
    • residualRotation

      private Quaternion residualRotation
      Instance to be reused to compute residual on rotation.
    • residualCenter

      private InhomogeneousPoint3D residualCenter
      Instance to be reused to compute center.
  • Constructor Details

    • PinholeCameraRefiner

      protected PinholeCameraRefiner()
      Constructor.
    • PinholeCameraRefiner

      protected PinholeCameraRefiner(PinholeCamera initialEstimation, boolean keepCovariance, BitSet inliers, double[] residuals, int numInliers, List<S1> samples1, List<S2> samples2, double refinementStandardDeviation)
      Constructor.
      Parameters:
      initialEstimation - initial estimation to be set.
      keepCovariance - true if covariance of estimation must be kept after refinement, false otherwise.
      inliers - set indicating which of the provided matches are inliers.
      residuals - residuals for matched samples.
      numInliers - number of inliers on initial estimation.
      samples1 - 1st set of paired samples.
      samples2 - 2nd set of paired samples.
      refinementStandardDeviation - standard deviation used for Levenberg-Marquardt fitting.
    • PinholeCameraRefiner

      protected PinholeCameraRefiner(PinholeCamera initialEstimation, boolean keepCovariance, com.irurueta.numerical.robust.InliersData inliersData, List<S1> samples1, List<S2> samples2, double refinementStandardDeviation)
      Constructor.
      Parameters:
      initialEstimation - initial estimation to be set.
      keepCovariance - true if covariance of estimation must be kept after refinement, false otherwise.
      inliersData - inlier data, typically obtained from a robust estimator.
      samples1 - 1st set of paired samples.
      samples2 - 2nd set of paired samples.
      refinementStandardDeviation - standard deviation used for Levenberg-Marquardt fitting.
  • Method Details

    • getRefinementStandardDeviation

      public double getRefinementStandardDeviation()
      Gets standard deviation used for Levenberg-Marquardt fitting during refinement. Returned value gives an indication of how much variance each residual has. Typically, this value is related to the threshold used on each robust estimation, since residuals of found inliers are within the range of such threshold.
      Returns:
      standard deviation used for refinement.
    • setRefinementStandardDeviation

      public void setRefinementStandardDeviation(double refinementStandardDeviation) throws LockedException
      Sets standard deviation used for Levenberg-Marquardt fitting during refinement. Returned value gives an indication of how much variance each residual has. Typically, this value is related to the threshold used on each robust estimation, since residuals of found inliers are within the range of such threshold.
      Parameters:
      refinementStandardDeviation - standard deviation used for refinement.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestSkewnessValueEnabled

      public boolean isSuggestSkewnessValueEnabled()
      Indicates whether skewness value is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if skewness value is suggested, false otherwise.
    • setSuggestSkewnessValueEnabled

      public void setSuggestSkewnessValueEnabled(boolean suggestSkewnessValueEnabled) throws LockedException
      Specifies whether skewness value is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestSkewnessValueEnabled - true if skewness value is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedSkewnessValue

      public double getSuggestedSkewnessValue()
      Gets suggested skewness value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested skewness value.
    • setSuggestedSkewnessValue

      public void setSuggestedSkewnessValue(double suggestedSkewnessValue) throws LockedException
      Sets suggested skewness value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedSkewnessValue - suggested skewness value.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestHorizontalFocalLengthEnabled

      public boolean isSuggestHorizontalFocalLengthEnabled()
      Indicates whether horizontal focal length is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if horizontal focal length is suggested, false otherwise.
    • setSuggestHorizontalFocalLengthEnabled

      public void setSuggestHorizontalFocalLengthEnabled(boolean suggestHorizontalFocalLengthEnabled) throws LockedException
      Specifies whether horizontal focal length is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestHorizontalFocalLengthEnabled - true if horizontal focal length is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedHorizontalFocalLengthValue

      public double getSuggestedHorizontalFocalLengthValue()
      Gets suggested horizontal focal length value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested horizontal focal length value.
    • setSuggestedHorizontalFocalLengthValue

      public void setSuggestedHorizontalFocalLengthValue(double suggestedHorizontalFocalLengthValue) throws LockedException
      Sets suggested horizontal focal length value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedHorizontalFocalLengthValue - suggested horizontal focal length value.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestVerticalFocalLengthEnabled

      public boolean isSuggestVerticalFocalLengthEnabled()
      Indicates whether vertical focal length is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if vertical focal length is suggested, false otherwise.
    • setSuggestVerticalFocalLengthEnabled

      public void setSuggestVerticalFocalLengthEnabled(boolean suggestVerticalFocalLengthEnabled) throws LockedException
      Specifies whether vertical focal length is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestVerticalFocalLengthEnabled - true if vertical focal length is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedVerticalFocalLengthValue

      public double getSuggestedVerticalFocalLengthValue()
      Gets suggested vertical focal length value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested vertical focal length.
    • setSuggestedVerticalFocalLengthValue

      public void setSuggestedVerticalFocalLengthValue(double suggestedVerticalFocalLengthValue) throws LockedException
      Sets suggested vertical focal length value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedVerticalFocalLengthValue - suggested vertical focal length.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestAspectRatioEnabled

      public boolean isSuggestAspectRatioEnabled()
      Indicates whether aspect ratio is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if aspect ratio is suggested, false otherwise.
    • setSuggestAspectRatioEnabled

      public void setSuggestAspectRatioEnabled(boolean suggestAspectRatioEnabled) throws LockedException
      Specifies whether aspect ratio is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestAspectRatioEnabled - true if aspect ratio is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedAspectRatioValue

      public double getSuggestedAspectRatioValue()
      Gets suggested aspect ratio value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested aspect ratio value.
    • setSuggestedAspectRatioValue

      public void setSuggestedAspectRatioValue(double suggestedAspectRatioValue) throws LockedException
      Sets suggested aspect ratio value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedAspectRatioValue - suggested aspect ratio value.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestPrincipalPointEnabled

      public boolean isSuggestPrincipalPointEnabled()
      Indicates whether principal point is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if principal point is suggested, false otherwise.
    • setSuggestPrincipalPointEnabled

      public void setSuggestPrincipalPointEnabled(boolean suggestPrincipalPointEnabled) throws LockedException
      Specifies whether principal point is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestPrincipalPointEnabled - true if principal point is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedPrincipalPointValue

      public InhomogeneousPoint2D getSuggestedPrincipalPointValue()
      Gets suggested principal point value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested principal point value to be reached when suggestion is enabled.
    • setSuggestedPrincipalPointValue

      public void setSuggestedPrincipalPointValue(InhomogeneousPoint2D suggestedPrincipalPointValue) throws LockedException
      Sets suggested principal point value to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedPrincipalPointValue - suggested principal point value to be reached when suggestion is enabled.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestRotationEnabled

      public boolean isSuggestRotationEnabled()
      Indicates whether camera rotation is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if camera rotation is suggested, false otherwise.
    • setSuggestRotationEnabled

      public void setSuggestRotationEnabled(boolean suggestRotationEnabled) throws LockedException
      Specifies whether camera rotation is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestRotationEnabled - true if camera rotation is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedRotationValue

      public Quaternion getSuggestedRotationValue()
      Gets suggested rotation to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested rotation to be reached when suggestion is enabled.
    • setSuggestedRotationValue

      public void setSuggestedRotationValue(Quaternion suggestedRotationValue) throws LockedException
      Sets suggested rotation to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedRotationValue - suggested rotation to be reached when suggestion is enabled.
      Throws:
      LockedException - if estimator is locked.
    • isSuggestCenterEnabled

      public boolean isSuggestCenterEnabled()
      Indicates whether camera center is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Returns:
      true if camera center is suggested, false otherwise.
    • setSuggestCenterEnabled

      public void setSuggestCenterEnabled(boolean suggestCenterEnabled) throws LockedException
      Specifies whether camera center is suggested or not. When enabled, the estimator will attempt to enforce suggested value in an iterative manner starting from an initially estimated camera. Even when suggestion is enabled, the iterative algorithm might not reach suggested value if the initial value largely differs from the suggested value.
      Parameters:
      suggestCenterEnabled - true if camera is suggested, false otherwise.
      Throws:
      LockedException - if estimator is locked.
    • getSuggestedCenterValue

      public InhomogeneousPoint3D getSuggestedCenterValue()
      Gets suggested center to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Returns:
      suggested center to be reached when suggestion is enabled.
    • setSuggestedCenterValue

      public void setSuggestedCenterValue(InhomogeneousPoint3D suggestedCenterValue) throws LockedException
      Sets suggested center to be reached when suggestion is enabled. Suggested value should be close to the initially estimated value otherwise the iterative refinement might not converge to provided value.
      Parameters:
      suggestedCenterValue - suggested center to be reached when suggestion is enabled.
      Throws:
      LockedException - if estimator is locked.
    • refine

      Refines provided initial estimation.
      Specified by:
      refine in class Refiner<PinholeCamera>
      Returns:
      refines estimation.
      Throws:
      NotReadyException - if not enough input data has been provided.
      LockedException - if estimator is locked because refinement is already in progress.
      RefinerException - if refinement fails for some reason (e.g. unable to converge to a result).
    • suggestionResidual

      protected double suggestionResidual(double[] params, double weight)
      Residual term for any required suggestions.
      Parameters:
      params - parameters being optimized. In the following order: skewness, horizontal focal length, vertical focal length, horizontal principal point, vertical principal point, quaternion A, quaternion B, quaternion C, quaternion D, center x, center y, center z.
      weight - weight to apply to obtained residual. This weight increases on each iteration to help into achieving required suggested values.
      Returns:
      term for any required suggestion.
    • parametersToCamera

      protected void parametersToCamera(double[] params, PinholeCamera result)
      Sets array of parameters into a pinhole camera. This method is used internally during refinement.
      Parameters:
      params - parameters to be set. In the following order: skewness, horizontal focal length, vertical focal length, horizontal principal point, vertical principal point, quaternion A, quaternion B, quaternion C, quaternion D, center x, center y, center z.
      result - instance where parameters will be set.
    • cameraToParameters

      protected void cameraToParameters(PinholeCamera camera, double[] result) throws CameraException, NotAvailableException
      Sets camera parameters into array of parameters. This method is used internally during refinement.
      Parameters:
      camera - camera to obtain parameters to be set into array.
      result - array where extracted parameters are stored. In the following order: skewness, horizontal focal length, vertical focal length, horizontal principal point, vertical principal point, quaternion A, quaternion B, quaternion C, quaternion D, center x, center y, center z.
      Throws:
      CameraException - if camera cannot be decomposed.
      NotAvailableException - if any camera component cannot be retrieved.
    • hasSuggestions

      protected boolean hasSuggestions()
      Indicates whether obtained solution requires refinement to apply provided suggestions.
      Returns:
      true if solution requires refinement to apply provided suggestions, false otherwise.
    • hasIntrinsicSuggestions

      private boolean hasIntrinsicSuggestions()
      Indicates whether suggestions for any intrinsic parameter are required or not.
      Returns:
      true if suggestions for any intrinsic parameters are required, false otherwise.
    • hasExtrinsicSuggestions

      private boolean hasExtrinsicSuggestions()
      Indicates whether suggestions for any extrinsic parameter are required or not.
      Returns:
      true if suggestions for any extrinsic parameter are required, false otherwise.