Method for determining distance information from images of a spatial region

{
  "publication_number": "US11039114B2",
  "country": "US",
  "kind": "B2",
  "title": "Method for determining distance information from images of a spatial region",
  "abstract": "A method includes defining a disparity range having discrete disparities and taking first, second, and third images of a spatial region using first, second, and third imaging units. The imaging units are arranged in an isosceles triangle geometry. The method includes determining first similarity values for a pixel of the first image for all the discrete disparities along a first epipolar line associated with the pixel in the second image. The method includes determining second similarity values for the pixel for all discrete disparities along a second epipolar line associated with the pixel in the third image. The method includes combining the first and second similarity values and determining a common disparity based on the combined similarity values. The method includes determining a distance to a point within the spatial region for the pixel from the common disparity and the isosceles triangle geometry.",
  "claims": [
    "1. A method for determining distance information, the method comprising: defining a disparity range having a number of discrete disparities; taking a first image of a spatial region with a first imaging unit, a second image of the spatial region with a second imaging unit, and a third image of the spatial region with a third imaging unit, wherein the first imaging unit, the second imaging unit, and the third imaging unit are arranged in a defined imaging geometry in which the imaging units form an isosceles triangle; determining first similarity values for at least one pixel of the first image for all discrete disparities in the defined disparity range along a first epipolar line associated with the pixel in the second image; determining second similarity values for the at least one pixel of the first image for all discrete disparities in the defined disparity range along a second epipolar line associated with the pixel in the third image; combining the first similarity values with the second similarity values; determining a common disparity between the first image, the second image, and the third image for the at least one pixel based on the combined similarity values; and determining a distance to a point within the spatial region for the at least one pixel from the common disparity and the defined imaging geometry.",
    "2. The method of claim 1, further comprising: carrying out a scene analysis to detect foreign objects in a hazardous area of a technical installation based on the distance to the point within the spatial region; and transferring the technical installation into a safe state in response to detection of a foreign object.",
    "3. The method of claim 1 further comprising: based on the distance to the point within the spatial region, selectively detecting a foreign object within the spatial region; and in response to detection of the foreign object, switching off a robot operating within the spatial region.",
    "4. The method of claim 1, wherein the first imaging unit, the second imaging unit, and the third imaging unit form an isosceles, right-angled triangle in the defined imaging geometry.",
    "5. The method of claim 1, wherein: the first similarity values are determined by comparing the at least one pixel of the first image and its surroundings with each pixel and its surroundings within the defined disparity range along the first epipolar line in the second image, and the second similarity values are determined by comparing the at least one pixel of the first image and its surrounding with each pixel and its surroundings within the defined disparity range along the second epipolar line in the third image.",
    "6. The method of claim 5, further comprising determining, for comparison of the at least one pixel and its surroundings with each pixel and its surroundings along the first epipolar line and the second epipolar line, a sum of at least one of absolute differences and quadratic differences.",
    "7. The method of claim 1, wherein the first similarity values and the second similarity values are added together.",
    "8. The method of claim 1, wherein determining the common disparity for the at least one pixel includes an extreme value search in the combined similarity values.",
    "9. The method of claim 8, wherein the extreme value search is a search for a minimum.",
    "10. The method of claim 1, wherein the first image, the second image, and the third image are transformed relative to each other such that the first epipolar line extends along a first axis and the second epipolar line extends along a second axis perpendicular to the first epipolar line.",
    "11. The method of claim 10, wherein: the first image, the second image, and the third image each comprise an equal number of pixel lines and an equal number of pixel columns, the first epipolar line extends in the second image along a pixel line that corresponds to the pixel line of the first image in which the at least one pixel is located, and the second epipolar line extends in the third image along a pixel column that corresponds to the pixel column of the first image in which the at least one pixel is located.",
    "12. The method of claim 1, wherein a common disparity is determined for all pixels of the first image.",
    "13. The method of claim 1, wherein a common disparity is determined for a defined number of pixels of the first image only.",
    "14. The method of claim 1, further comprising: determining third similarity values for at least one further pixel of the second image for all discrete disparities in the defined disparity range along a first epipolar line in the first image associated with the further pixel; determining fourth similarity values for the at least one further pixel of the second image for all discrete disparities in the defined disparity range along a second epipolar line associated with the further pixel in the third image; and determining further distance information from the third and fourth similarity values.",
    "15. An apparatus for determining distance information from images of a spatial region, the apparatus comprising: a first imaging unit configured to take a first image of the spatial region; a second imaging unit configured to take a second image of the spatial region; a third imaging unit configured to take a third image of the spatial region; and an image processing unit configured to determine first similarity values and second similarity values for at least one pixel of the first image within a defined disparity range having a number of discrete disparities, wherein the first imaging unit, the second imaging unit, and the third imaging unit are arranged in a defined imaging geometry in which the imaging units form an isosceles triangle, wherein the image processing unit is configured to: determine the first similarity values for the at least one pixel for all discrete disparities in the defined disparity range along a first epipolar line associated with the pixel in the second image, and determine the second similarity values for the at least one pixel for all discrete disparities in the defined disparity range along a second epipolar line associated with the pixel in the third image, and wherein the image processing unit is further configured to: combine the first similarity values with the second similarity values, determine a common disparity for the at least one pixel between the first image, the second image, and the third image based on the combined similarity values, and determine a distance to a point within the spatial region for the at least one pixel from the common disparity and the defined imaging geometry.",
    "16. The apparatus of claim 15, wherein the image processing unit is an FPGA.",
    "17. The apparatus of claim 15, further comprising: an evaluation unit configured to carry out a scene analysis to detect foreign objects in a hazardous area of a technical installation based on the distance to the point within the spatial region; and safety equipment configured to transfer the technical installation into a safe state in response to the evaluation unit detecting a foreign object.",
    "18. The apparatus of claim 15, wherein the first imaging unit, the second imaging unit, and the third imaging unit are arranged in a common housing.",
    "19. A non-transitory computer-readable medium comprising instructions including: arranging a first imaging unit, a second imaging unit, and a third imaging unit in a defined imaging geometry, in which the imaging units form an isosceles triangle; defining a disparity range having a number of discrete disparities; taking a first image of a spatial region with the first imaging unit, a second image of the spatial region with the second imaging unit, and a third image of the spatial region with the third imaging unit; determining first similarity values for at least one pixel of the first image for all discrete disparities in the defined disparity range along a first epipolar line associated with the pixel in the second image; determining second similarity values for the at least one pixel of the first image for all discrete disparities in the defined disparity range along a second epipolar line associated with the pixel in the third image; combining the first similarity values with the second similarity values; determining a common disparity between the first image, the second image, and the third image for the at least one pixel based on the combined similarity values; and determining a distance to a point within the spatial region for the at least one pixel from the common disparity and the defined imaging geometry.",
    "20. The computer-readable medium of claim 19 further comprising instructions including: based on the distance to the point within the spatial region, selectively detecting a foreign object within the spatial region; and in response to detection of the foreign object, switching off a robot operating within the spatial region."
  ],
  "description_excerpt": "The present disclosure relates to a method for determining distance information from images of a spatial region and a corresponding apparatus. Furthermore, the disclosure relates to the use of such apparatus for safeguarding a hazardous area of a technical installation.\n\nDistance information of objects in a spatial region can be determined by offset images of the spatial region. The offset (disparity) between the projections of an object in the offset images depends on the distance of the object to the imaging units, so that with a known offset the distance to an object can be determined by triangulation using the imaging geometry. To determine the offset, corresponding elements must be found in the offset images. This process is called correspondence analysis. While finding corresponding elements in different images is easy for humans, it is a great challenge for a computer. Therefore, assignment errors can occur, which ultimately lead to incorrectly measured distances. In standard stereo systems, i.e. systems in which a stereo image taken with two sensors is used, the assignment problems between the left image and right images are countered with complex algorithms for correspondence analysis. Despite all efforts, however, assignment problems cannot be completely excluded or reliably reduced to a tolerable level. It is therefore necessary, especially for safety-critical applications where a distance must be reliably detected at least in a defined range, to take additional measures to verify the measured distance.",
  "cpc": [
    "H04N 13/128",
    "B25J 9/1674",
    "F16P 3/142",
    "G01C 11/30",
    "G01C 3/14",
    "G01V 8/20",
    "G06T 7/596",
    "H04N 13/243",
    "H04N 2013/0081"
  ],
  "ipc": [
    "B25J 9/16",
    "F16P 3/14",
    "G01C 11/30",
    "G01C 3/14",
    "G01V 8/20",
    "H04N 13/00",
    "H04N 13/128",
    "H04N 13/243"
  ],
  "assignees": [
    "PILZ GMBH & CO KG"
  ],
  "inventors": [
    "HAUSSMANN JOERG"
  ],
  "filing_date": "2020-06-22",
  "publication_date": "2021-06-15",
  "grant_date": "2021-06-15",
  "priority_date": "2017-12-21",
  "application_number": "US-202016908122-A",
  "family_id": "64902050",
  "citations": [
    "DE102008020326B3",
    "US2009129666A1",
    "US2010150455A1",
    "US2016165216A1",
    "US2016309134A1",
    "US2017359565A1",
    "WO2007107214A2"
  ]
}