Image processing apparatus, image processing method, and robot system

{
  "publication_number": "US11138684B2",
  "country": "US",
  "kind": "B2",
  "title": "Image processing apparatus, image processing method, and robot system",
  "abstract": "An image processing apparatus includes a two-dimensional image storage unit configured to store a plurality of two-dimensional image data captured by photographing an identical imaging target object under different exposure conditions; a distance image storage unit configured to store distance image data including a pixel array of a known relationship to a pixel array of the two-dimensional image data; a pixel extraction unit configured to extract, among pixels in each of the two-dimensional image data, a first pixel at which a difference in brightness between identical pixels is less than a predetermined value; and a distance image adjusting unit configured to specify a second pixel of the distance image data at a position corresponding to the first pixel in the pixel array, and to set the second pixel as a non-imaging pixel in the distance image data.",
  "claims": [
    "1. An image processing apparatus comprising: a two-dimensional image storage unit configured to store a plurality of two-dimensional image data captured by photographing an identical imaging target object under different exposure conditions; a distance image storage unit configured to store distance image data representative of distance information depending on a spatial position of the imaging target object, the distance image data including a pixel array of a known relationship to a pixel array of the two-dimensional image data; a pixel extraction unit configured to extract, among a plurality of pixels in each of the two-dimensional image data, a first pixel at which a difference in brightness between identical pixels is less than a predetermined value; and a distance image adjusting unit configured to specify a second pixel of the distance image data at a position corresponding to the first pixel in the pixel array, and to set the second pixel as a non-imaging pixel in the distance image data.",
    "2. The image processing apparatus according to claim 1, wherein the first pixel is a pixel which holds imaging information of a space in the imaging target object.",
    "3. A robot system comprising: a robot; a robot controller configured to control the robot; and the image processing apparatus according to claim 1, wherein the robot controller is configured to cause the robot to handle the imaging target object, based on the distance image data acquired as a result of the distance image adjusting unit setting the second pixel as the non-imaging pixel.",
    "4. The robot system according to claim 3, further comprising: an image acquisition apparatus configured to acquire the plurality of two-dimensional image data and the distance image data.",
    "5. The robot system according to claim 3, wherein the imaging target object includes surfaces of a plurality of objects juxtaposed with a gap interposed, the first pixel holds imaging information of the gap, and the robot individually grasps each of the plurality of objects.",
    "6. An image processing method comprising: storing a plurality of two-dimensional image data captured by photographing an identical imaging target object under different exposure conditions; storing distance image data representative of distance information depending on a spatial position of the imaging target object, the distance image data including a pixel array of a known relationship to a pixel array of the two-dimensional image data; extracting, among a plurality of pixels in each of the two-dimensional image data, a first pixel at which a difference in brightness between identical pixels is less than a predetermined value; and specifying a second pixel of the distance image data at a position corresponding to the first pixel in the pixel array, and setting the second pixel as a non-imaging pixel in the distance image data."
  ],
  "description_excerpt": "1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and a robot system. 2. Description of the Related Art There is known a robot system configured to recognize an object by a camera and to handle the recognized object by a robot. For example, Japanese Unexamined Patent Publication (Kokai) No. 2016-185573A discloses a robot system including a target object selection unit configured to select a target object; a proximity state determination unit configured to determine whether another object is disposed in proximity to the target object; an avoidance vector determination unit configured to determine an avoidance vector such that no interference with the object occurs; and a picking path correction unit configured to generate a corrected path which is obtained by correcting a picking path, based on the avoidance vector. A system disclosed in Japanese Unexamined Patent Publication (Kokai) No. 2013-186088A performs three-dimensional position/attitude measurement of a target object by using a first sensor configured to acquire two-dimensional (2D) information or three-dimensional (3D) information of the target object and a second sensor configured to acquire 2D information or 3D information of the target object.",
  "cpc": [
    "G06T 7/12",
    "B25J 13/08",
    "G01B 11/00",
    "G06K 2209/21",
    "G06K 9/2054",
    "G06T 1/0014",
    "G06T 2207/10016",
    "G06T 2207/10028",
    "G06T 2207/10144",
    "G06T 2207/30108",
    "G06T 7/174",
    "G06T 7/70",
    "G06V 10/22",
    "G06V 2201/07"
  ],
  "ipc": [
    "G06K 9/20",
    "G06T 1/00"
  ],
  "assignees": [
    "FANUC CORP"
  ],
  "inventors": [
    "Yoshida Junichirou",
    "TAKIZAWA SHOUTA"
  ],
  "filing_date": "2020-04-22",
  "publication_date": "2021-10-05",
  "grant_date": "2021-10-05",
  "priority_date": "2019-04-25",
  "application_number": "US-202016854919-A",
  "family_id": "72921541",
  "citations": [
    "JP2013186088A",
    "JP2016185573A",
    "US10059002B2",
    "US10127622B2",
    "US10157495B2",
    "US10198792B2",
    "US10212408B1",
    "US10292321B2",
    "US10313569B2",
    "US10339668B2",
    "US10380911B2",
    "US10434654B2",
    "US10477175B2",
    "US10498963B1",
    "US10502556B2",
    "US10582180B2",
    "US10863105B1",
    "US10869010B2",
    "US10917543B2",
    "US10986267B2",
    "US2004125206A1",
    "US2005162644A1",
    "US2009027509A1",
    "US2010033619A1",
    "US2014176761A1",
    "US2015371398A1",
    "US2016155235A1",
    "US2016175964A1",
    "US2016279809A1",
    "US2017257540A1",
    "US2018023947A1",
    "US2018084708A1",
    "US2019033067A1",
    "US2020021743A1",
    "US2020175352A1",
    "US2020226729A1",
    "US5386228A",
    "US6453069B1",
    "US6988610B2",
    "US7599533B2",
    "US7672517B2",
    "US7701455B2",
    "US7756299B2",
    "US8174611B2",
    "US8340464B2",
    "US8459073B2",
    "US8654219B2",
    "US8699821B2",
    "US8730318B2",
    "US8780113B1",
    "US9113142B2",
    "US9247153B2",
    "US9270902B2",
    "US9275464B2",
    "US9294695B2",
    "US9412172B2",
    "US9582888B2",
    "US9832404B2"
  ]
}