Method for detecting objects in a warehouse and/or for spatial orientation in a warehouse

{
  "publication_number": "US10198805B2",
  "country": "US",
  "kind": "B2",
  "title": "Method for detecting objects in a warehouse and/or for spatial orientation in a warehouse",
  "abstract": "A method for detecting objects in a warehouse and/or for spatial orientation in a warehouse includes: acquiring image data with a 3-D camera which is fastened to an industrial truck so that a viewing direction of the 3-D camera has a defined horizontal angle, wherein the 3-D camera has an image sensor with sensor elements arranged matrix-like and the image data comprises a plurality of pixels, wherein distance information is assigned to each pixel, calculating angle information for a plurality of image elements, which each specify an angle between a surface represented by the image element and a vertical reference plane, determining a predominant direction based on the frequency of the calculated angle information, calculating the positions of the of the acquired pixels along the predominant direction, detecting at least one main plane of the warehouse based on a frequency distribution of the calculated positions.",
  "claims": [
    "1. A method for detecting objects in a warehouse and/or for spatial orientation in a warehouse having the following steps: acquiring image data with a 3-D camera which is fastened to an industrial truck so that a viewing direction of the 3-D camera has a defined angle to the horizontal, wherein the 3-D camera has an image sensor with sensor elements arranged matrix-like and the image data comprises a plurality of pixels, wherein a color value and distance information is assigned to each pixel, segmenting the image data into contiguous segments based on the color values, calculating segment attributes for each segment, classifying the segments based on the segment attributes, detecting a warehouse object based on the classification.",
    "2. The method according to claim 1, wherein one or more of the following properties are used as segment attributes: dimensions (b, h, t) average color value, number of pixels, relative depth information (Zrel).",
    "3. The method according to claim 1, further comprising a support vector machine is used for the classification.",
    "4. The method according to claim 1, wherein with the classification of segments, one or more of the following classes are considered: vertical shelf supports, horizontal shelf supports, signs, transport boxes, walls, pallets.",
    "5. The method according to claim 4, wherein segments lying above one another, which were classified as vertical shelf supports, and/or segments lying next to each other, which were classified as horizontal shelf supports, are respectively combined into a segment group.",
    "6. The method according to claim 4, further comprising a segment is detected as a pallet if the following criteria are satisfied: the segment was classified with a minimum probability as a pallet and/or a center point of the segment is located within a predefined height range above a horizontal shelf-support and within a predefined distance range behind the front thereof and/or a comparison of the front side of the segment with a predefined template of a pallet yields a minimum degree of similarity.",
    "7. An industrial truck having an apparatus for detecting objects in a warehouse and/or for spatial orientation in a warehouse, wherein the apparatus comprises the following: a 3-D camera for acquiring image data, wherein the 3-D camera is fastened to the industrial truck so that a viewing direction of the 3-D camera (18) has a defined angle to the horizontal, has an image sensor with sensor elements arranged matrix-like and the image data comprises a plurality of pixels, to each of which a color value and distance information is assigned, an evaluation apparatus for evaluating the image data that is designed for the purpose of segmenting the image data into contiguous segments based on the color values, calculating segment attributes for each of the segments, classifying the segments based on the segment attribute, and detecting a warehouse object based on the classification."
  ],
  "description_excerpt": "The invention relates to a method for detecting objects in a warehouse and/or for spatial orientation in a warehouse, in which image data is acquired and evaluated. For this purpose it is known in the field of warehouse logistics to equip industrial trucks with laser scanners. These, in particular in combination with reflective markers positioned in the warehouse, can deliver information about the position of the industrial truck in the warehouse, and in this manner supplement or completely replace other navigation systems. The use of laser scanners is known also for supporting storage and retrieval procedures.\n\nIt is known from the document EP 2 468 678 A2 to arrange a time-of-flight camera system on an industrial truck, and in this manner to acquire a three-dimensional image of the surroundings of the industrial truck. This is used in particular for supporting an operator in that storage and retrieval procedures are automated to some extent.\n\nBased on this background, the object of the invention is to provide a method for detecting objects in the warehouse and/or for spatial orientation in a warehouse, with which the acquisition and evaluation of three-dimensional image data is better adapted to the specific use conditions of an industrial truck so that it can be implemented easily and reliably, and an industrial truck suited for this purpose.\n\nThis object is solved by the method with the features of claim 1. Advantageous designs are specified with the subsequent sub-claims.",
  "cpc": [
    "G06V 20/64",
    "B60R 1/00",
    "B60R 2300/10",
    "B66F 9/0755",
    "B66F 9/24",
    "G06F 18/24",
    "G06K 9/00201",
    "G06K 9/00691",
    "G06K 9/4652",
    "G06K 9/52",
    "G06K 9/6267",
    "G06T 2200/04",
    "G06T 2207/20076",
    "G06T 7/0004",
    "G06T 7/10",
    "G06T 7/50",
    "G06T 7/60",
    "G06T 7/70",
    "G06T 7/73",
    "G06T 7/90",
    "G06V 10/56",
    "G06V 20/36",
    "H04N 13/0203",
    "H04N 13/204"
  ],
  "ipc": [
    "B60R 1/00",
    "G06T 7/00",
    "G06T 7/10",
    "G06T 7/50",
    "G06T 7/60",
    "G06T 7/70",
    "G06T 7/73",
    "G06T 7/90",
    "G06V 10/56",
    "H04N 13/204"
  ],
  "assignees": [
    "Jungheinrich AG"
  ],
  "inventors": [
    "Philipp Halata"
  ],
  "filing_date": "2016-08-11",
  "publication_date": "2019-02-05",
  "grant_date": "2019-02-05",
  "priority_date": "2013-02-15",
  "application_number": "US-201615233991-A",
  "family_id": "50070372",
  "cited_by_count": 7,
  "citations": [
    "US6014461A",
    "US5938710A",
    "US6487303B1",
    "US6736587B2",
    "US7219769B2",
    "US20080180707A1",
    "EP2136319A2",
    "US8160366B2",
    "US20100231926A1",
    "US20110026835A1",
    "US20110190008A1",
    "EP2468678A1",
    "US9709394B2",
    "US20130322740A1"
  ]
}