Image processing system and image processing method

{
  "publication_number": "US12418718B2",
  "country": "US",
  "kind": "B2",
  "title": "Image processing system and image processing method",
  "abstract": "An image processing system includes: an industrial machine; a visual sensor; a visual pattern arranged on a part to be detected, for identifying a relative positional relationship between the visual sensor and the visual pattern; an image processing unit that determines the relative positional relationship between the visual sensor and the visual pattern on the basis of an image of the visual pattern captured by the visual sensor; and a focusing operation control unit that, with the visual pattern within the field of view of the visual sensor, causes the visual sensor to capture the visual pattern while changing the relative positions of the visual sensor and the visual pattern in a predetermined direction with reference to the visual pattern, on the basis of the determined relative positional relationship to obtain a focus degree of the visual pattern, whereby the visual sensor is focused on the visual pattern.",
  "claims": [
    "1. An image processing system, comprising: an industrial machine; a visual sensor equipped on a predetermined movable part of the industrial machine; a visual pattern for determining a relative positional relation between the visual sensor and the visual pattern placed at a detection target spot of a workpiece; an image processing unit configured to determine the relative positional relation between the visual sensor and the visual pattern from an image of the visual pattern captured by the visual sensor; and a focusing operation control unit configured to cause the visual sensor to focus on the visual pattern by causing the visual sensor to capture the image of the visual pattern while moving, based on the determined relative positional relation, the visual sensor by the industrial machine to an in-focus position where the visual sensor focuses on the visual pattern to change a relative position between the visual sensor and the visual pattern in a predetermined direction with the visual pattern as a basis, and finding a degree of focusing of the visual pattern, in a state of the visual pattern being captured in a visual field of the visual sensor, wherein the visual sensor is calibrated in advance by using the visual pattern to obtain calibration data, and the focusing operation control unit is configured to use the calibration data representing a position and an attitude of the visual sensor with a predetermined reference position in the industrial machine as a basis to direct the visual sensor to the predetermined direction with the visual pattern as a basis.",
    "2. The image processing system according to claim 1, wherein the focusing operation control unit is configured to change the relative position between the visual sensor and the visual pattern in the predetermined direction with respect to a plane on which the visual pattern is formed.",
    "3. The image processing system according to claim 1, wherein the focusing operation control unit is configured to move the visual sensor in a predetermined operating range with respect to the visual pattern, and move the visual sensor to the in-focus position by comparing a degree of focusing reference value when a degree of focusing of the visual pattern is highest with the degree of focusing acquired while moving the visual sensor.",
    "4. The image processing system according to claim 1, wherein the industrial machine is an industrial robot.",
    "5. An image processing method, comprising: determining a relative positional relation between a visual sensor and a visual pattern placed at a detection target spot of a workpiece from an image of the visual pattern captured by the visual sensor, wherein the visual sensor is equipped on a predetermined movable part of an industrial machine; and causing the visual sensor to focus on the visual pattern by causing the visual sensor to capture the image of the visual pattern while moving, based on the determined relative positional relation, the visual sensor by the industrial machine to an in-focus position where the visual sensor focuses on the visual pattern to change a relative position between the visual sensor and the visual pattern in a predetermined direction with the visual pattern as a basis, and finding a degree of focusing of the visual pattern, in a state of the visual pattern being captured in a visual field of the visual sensor, wherein the visual sensor is calibrated in advance by using the visual pattern to obtain calibration data, and the method further comprises using the calibration data representing a position and an attitude of the visual sensor with a predetermined reference position in the industrial machine as a basis to direct the visual sensor to the predetermined direction with the visual pattern."
  ],
  "description_excerpt": "The present invention relates to an image processing system and an image processing method.\n\nSystems configured to install a visual sensor on a movable part of an industrial robot or another industrial machine and perform visual inspection, position detection, and the like of a workpiece by image processing are known (for example, see PTL 1). In order to install a visual sensor on a movable part of an industrial robot or the like and perform position detection and the like of a workpiece, finding a relative positional relation of a coordinate system of the visual sensor with a standard coordinate system set to the industrial robot or the like, i.e., calibration of the visual sensor is required. Various techniques are known in the relevant field as technologies for calibration of a visual sensor (such as PTLs 2 to 5). In a robot system as described above, adjustment needs to be performed in order to focus a camera on an inspection target position. In such a case, the distance between a target object and the camera is generally adjusted by a user performing adjustment by turning a focusing ring of the camera or the user moving the robot by operating a teach pendant. On the other hand, PTL 5 describes that “an image processing system SYS determines a normal line V with respect to a set measurement point Wp on a determined workpiece W according to placement status of the workpiece W and varies the position and the attitude of a two-dimensional camera 310 in such a way that the determined normal line V matches the optical axis of the two-dimensional camera 310 (S 1).",
  "cpc": [
    "H04N 23/67",
    "B25J 9/1692",
    "G05B 2219/39045",
    "G06T 7/80",
    "H04N 23/61"
  ],
  "ipc": [
    "G06T 7/80",
    "H04N 23/61",
    "H04N 23/67"
  ],
  "assignees": [
    "FANUC CORP"
  ],
  "inventors": [
    "Fu wanfeng",
    "NAMIKI YUTA"
  ],
  "filing_date": "2021-12-03",
  "publication_date": "2025-09-16",
  "grant_date": "2025-09-16",
  "priority_date": "2020-12-10",
  "application_number": "US-202118043039-A",
  "family_id": "81973253",
  "citations": [
    "CN110136208A",
    "JP2010079845A",
    "JP2011221612A",
    "JP2014128845A",
    "JP2014180720A",
    "JP2016097474A",
    "JP2016221645A",
    "JP2017212812A",
    "JP2018091774A",
    "JP2018192569A",
    "JP2018194542A",
    "JP2020047049A",
    "JPH08210816A",
    "US2006017720A1",
    "US2014039666A1",
    "US2016354929A1",
    "US2018333859A1",
    "US2018338090A1",
    "US2022114703A1"
  ]
}