Training a pose detection algorithm, and deriving an object pose using a trained pose detection algorithm

{
  "publication_number": "US2021241476A1",
  "country": "US",
  "kind": "A1",
  "title": "Training a pose detection algorithm, and deriving an object pose using a trained pose detection algorithm",
  "abstract": "Training a pose detection algorithm and deriving an object pose using a trained pose detection algorithm. Specifying a first view of a 3D model with respect to a virtual camera, generating a depth map by projecting the 3D model based on the first view, generating a second view by perturbing the first view, selecting reference 2D locations in an overlapping area of the 3D model between the first view and the second view. Transforming the reference 2D locations into a 3D point set corresponding to the second view. For each reference 2D location, calculating a depth difference between the first view and the second view at the reference 2D location. Associating the perturbation between the second view and the first view with the calculated depth differences, and training an algorithm model to identify a pose of an object corresponding to the 3D model.",
  "claims": [
    "1. A non-transitory computer readable medium that embodies instructions that cause one or more processors to perform a method for training an object detection algorithm, the method comprising: (a) specifying a first view of a 3D model with respect to a virtual camera; (b) generating a depth map by projecting the 3D model based on the first view; (c) generating a second view around the first view by perturbing the first view; (d) selecting reference 2D locations from the first depth map, the reference 2D locations being in an overlapping area of the 3D model between the first view and the second view; (e) transforming the reference 2D locations into a 3D-point set corresponding to the second view; (f) for each reference 2D location, calculating a depth difference between the first view and the second view at the reference 2D location using (i) the 3D-point set and (ii) the depth map; (g) associating the perturbation between the second view and the first view with the calculated depth differences; and (h) training an algorithm model using, as an input to the algorithm, the perturbation and the depth difference associated with each other, to identify a pose of an object corresponding to the 3D model. 2. The non-transitory computer readable medium according to claim 1, wherein the second view includes a plurality of training views and a plurality of validation views. 3. The non-transitory computer readable medium according to claim 2, wherein the method further includes measuring a performance of the algorithm model using the plurality of validation views. 4. The non-transitory computer readable medium according to claim 3, wherein the perturbing of step (c) includes perturbing the first view by a first amount, and the method further includes determining a validation error for each of the plurality of validation views, wherein when the validation error is greater than a predetermined threshold, repeating step (c) using a second perturbation amount. 5. The non-transitory computer readable medium according to claim 3, wherein the perturbing of step (c) includes perturbing the first view by a first amount, and the method further includes determining a validation error for each of the plurality of validation views, wherein when the validation error is less than or equal to a predetermined threshold, the method further includes (c1) marking all of the second views as visited poses and labeling the second views as neighboring poses of the first view. 6. The non-transitory computer readable medium according to claim 2, wherein the training views are split into six degrees of freedom (DOF), and six random numbers are generated, a different one of the six random numbers corresponding to each of the degrees of freedom. 7. The non-transitory computer readable medium according to claim 1, wherein the training the algorithm model of step (h) includes (h1) learning a mapping function by using use an ensemble of weak mapping regressors. 8. The non-transitory computer readable medium according to claim 7, wherein the ensemble of weak mapping regressors are a decision tree. 9. The non-transitory computer readable medium according to claim 7, wherein the ensemble of weak mapping regressors are determined using a bootstrap aggregation principle. 10. A non-transitory computer readable medium that embodies instructions that cause one or more processors to perform a method for deriving a pose of an object, the method comprising: (a) generating a depth map based on sensor data acquired by a depth sensor sensing an object; (b) deriving a first pose of the object corresponding to the depth map; (c) acquiring 3D feature points based on a 3D model corresponding to the object; (d) acquiring 2D feature points by projecting the 3D feature points onto a virtual plane using the first pose; (e) acquiring depth values from the depth math, the depth values associated with 2D locations of the depth map, the locations being at the same locations as the 2D feature points on the virtual plane; and (f) deriving a second pose of the object by applying a trained algorithm model using, as an input to the trained algorithm, a difference between (i) Z coordinate values of the 3D feature points and (ii) the respective depth values. 11. The non-transitory computer readable medium according to claim 10, wherein the trained algorithm model is a model is generated by performing: (f1) specifying a first view of a 3D model with respect to a virtual camera; (f2) generating a depth map by projecting the 3D model based on the first view; (f3) generating a second view around the first view by perturbing the first view; (f4) selecting reference 2D locations from the first depth map, the reference 2D locations being in an overlapping area of the 3D model between the first view and the second view; (f5) transforming the reference 2D locations into a 3D-point set corresponding to the second view; (f6) for each reference 2D location, calculating a depth difference between the first view and the second view at the reference 2D location using (i) the 3D-point set and (ii) the depth map; (f7) associating the perturbation between the second view and the first view with the calculated depth differences; and (f8) training an algorithm model to identify a pose of an object corresponding to the 3D model based on the association of the perturbation; and (f9) identifying the second pose of an object corresponding to the 3D model based on the association of the perturbation. 12. The non-transitory computer readable medium according to claim 11, wherein the applying the trained algorithm model of step (f) includes learning a mapping function by using use an ensemble of weak mapping regressors. 13. The non-transitory computer readable medium according to claim 12, wherein the ensemble of weak mapping regressors are a decision tree. 14. The non-transitory computer readable medium according to claim 13, wherein the decision tree is a pyramid of ensemble of trees comprising a plurality of layers. 15. The non-transitory computer readable medium according to claim 14, wherein the plurality of layers include a topmost layer performing a most coarse refinement of the first pose, and a lowermost layer performing a most fine refinement of the first pose, and the second pose is determined upon performance of the most fine refinement. 16. The non-transitory computer readable medium according to claim 14, wherein the pyramid of ensemble of trees utilizes a hyper parameter learner to determine a number of layers of the pyramid. 17. The non-transitory computer readable medium according to claim 15, wherein the plurality of layers provides for sequential, progressively finer refinement during each of the plurality of layers until the refined first pose with a lowest residual error is reached. 18. The non-transitory computer readable medium according to claim 17, wherein the pose with the lowest residual error is added to a container of best refinements. 19. The non-transitory computer readable medium according to claim 13, wherein each layer is a refinement of candidate poses based on the depth difference for one of the overlapping points."
  ],
  "cpc": [
    "G06T 7/55",
    "G06N 20/20",
    "G06N 5/01",
    "G06N 5/022",
    "G06T 17/10",
    "G06T 2207/10024",
    "G06T 2207/10028",
    "G06T 2207/20081",
    "G06T 7/344",
    "G06T 7/75"
  ],
  "assignees": [
    "SEIKO EPSON CORP"
  ],
  "filing_date": "2020-01-30",
  "publication_date": "2021-08-05",
  "priority_date": "2020-01-30",
  "application_number": "US-202016776675-A",
  "family_id": "77062951"
}