Technologies for pan tilt unit calibration

{
  "publication_number": "US9807292B2",
  "country": "US",
  "kind": "B2",
  "title": "Technologies for pan tilt unit calibration",
  "abstract": "Technologies for calibrating a pan tilt unit with a robot include a robot controller to move a camera of the pan tilt unit about a first rotational axis of the pan tilt unit to at least three different first axis positions. The robot controller records a first set of positions of a monitored component of the robot in a frame of reference of the robot and a position of the camera in a frame of reference of the pan tilt unit during a period in which the monitored component is within a field of view of the camera for each of the at least three different first axis positions. Further, the robot controller moves the camera about a second rotational axis of the pan tilt unit to at least three different second axis positions and records a second set of positions of the monitored component in the frame of reference of the robot and a position of the camera in the frame of reference of the pan tilt unit during a period in which the monitored component is within a field of view of the camera for each of the at least three different second axis positions. Further, the robot controller determines a transformation from the frame of reference of the robot to the frame of reference of the pan tilt unit based on the first set of recorded positions and the second set of recorded positions.",
  "claims": [
    "1. A robot system for calibrating a pan tilt unit, the robot system comprising: a robot; arm control circuitry configured to operate an articulating arm and a robot tool of the robot; pan tilt unit control circuitry to (i) move a camera of the pan tilt unit about a first rotational axis of the pan tilt unit to at least three different first axis positions and (ii) move the camera about a second rotational axis of the pan tilt unit to at least three different second axis positions; position recording circuitry to (i) record a first set of positions of the robot tool in a frame of reference of the robot and a position of the camera in a frame of reference of the pan tilt unit during a period in which the robot tool is within a field of view of the camera for each of the at least three different first axis positions and (ii) record a second set of positions of the robot tool in the frame of reference of the robot and a position of the camera in the frame of reference of the pan tilt unit during a period in which the robot tool is within a field of view of the camera for each of the at least three different second axis positions; and transformation circuitry to determine a transformation from the frame of reference of the robot to the frame of reference of the pan tilt unit based on the first set of recorded positions, the second set of recorded positions, a first origin of the frame of reference of the robot, a second origin of the frame of reference of the pan tilt unit, a first intersection line between one or more planes generated using the first set of recorded positions and the second set of recorded positions, and a second intersection line between one or more planes generated using the at least three different first axis positions and the at least three different second axis positions.",
    "2. The robot system of claim 1, wherein the frame of reference of the robot is a coordinate system defined by a base of the robot.",
    "3. The robot system of claim 1, wherein to determine the transformation comprises to: determine a first plane defined by the first set of recorded positions; determine a second plane defined by the second set of recorded positions; determine a third plane defined by the at least three different first axis positions; and determine a fourth plane defined by the at least three different second axis positions.",
    "4. The robot system of claim 3, wherein to determine the transformation comprises to determine (i) the first intersection line between the first plane and the second plane and (ii) the second intersection line between the third plane and the fourth plane.",
    "5. The robot system of claim 4, wherein to determine the transformation comprises to determine (i) the first origin of the frame of reference of the robot based on the first intersection line and (ii) the second origin of the frame of reference of the pan tilt unit based on the second intersection line.",
    "6. The robot system of claim 1, wherein the arm control circuitry is to move the robot tool to a position such that the robot tool is centered within a field of view of the camera for each of the at least three different first axis positions; and wherein to record the first set of positions comprises to record the first set of positions during a period in which the robot tool is centered within the field of view of the camera.",
    "7. The robot system of claim 1, wherein to move the camera about the first rotational axis comprises to pan the camera; and wherein to move the camera about the second rotational axis comprises to tilt the camera.",
    "8. The robot system of claim 1, wherein the transformation circuitry is further to: determine a position of the robot tool in the frame of reference of the robot; and determine a corresponding position of the robot tool in the frame of reference of the pan tilt unit based on the transformation.",
    "9. The robot system of claim 8, wherein the pan tilt unit control circuitry is further to move the camera to a position in which the corresponding position of the robot tool is within a field of view of the camera.",
    "10. A method for calibrating a pan tilt unit with a robot, the method comprising: moving a camera of the pan tilt unit about a first rotational axis of the pan tilt unit to at least three different first axis positions; recording a first set of positions of a monitored component of the robot in a frame of reference of the robot and a position of the camera in a frame of reference of the pan tilt unit during a period in which the monitored component is within a field of view of the camera for each of the at least three different first axis positions; moving the camera about a second rotational axis of the pan tilt unit to at least three different second axis positions; recording a second set of positions of the monitored component in the frame of reference of the robot and a position of the camera in the frame of reference of the pan tilt unit during a period in which the monitored component is within a field of view of the camera for each of the at least three different second axis positions; and determining a transformation from the frame of reference of the robot to the frame of reference of the pan tilt unit based on the first set of recorded positions, the second set of recorded positions, a first origin of the frame of reference of the robot, a second origin of the frame of reference of the pan tilt unit, a first intersection line between one or more planes generated using the first set of recorded positions and the second set of recorded positions, and a second intersection line between one or more planes generated using the at least three different first axis positions and the at least three different second axis positions.",
    "11. The method of claim 10, wherein the frame of reference of the robot is a coordinate system defined by a base of the robot.",
    "12. The method of claim 10, wherein the monitored component is a robot tool secured to an articulated arm of the robot.",
    "13. The method of claim 10, wherein determining the transformation comprises: determining a first plane defined by the first set of recorded positions; determining a second plane defined by the second set of recorded positions; determine a third plane defined by the at least three different first axis positions; and determine a fourth plane defined by the at least three different second axis positions.",
    "14. The method of claim 13, wherein determining the transformation comprises determining (i) the first intersection line between the first plane and the second plane and (ii) the second intersection line between the third plane and the fourth plane.",
    "15. The method of claim 14, wherein determining the transformation comprises determining (i) the first origin of the frame of reference of the robot based on the first intersection line and (ii) the second origin of the frame of reference of the pan tilt unit based on the second intersection line.",
    "16. The method of claim 10, further comprising improving an accuracy of the transformation by applying a Levenberg-Marquardt algorithm based on the first set of recorded positions and the second set of recorded positions.",
    "17. The method of claim 10, wherein recording the first set of positions comprises recording a first set of positions during a period in which the monitored component is centered within a field of view of the camera.",
    "18. The method of claim 10, wherein moving the camera about the first rotational axis comprises panning the camera; and wherein moving the camera about the second rotational axis comprises tilting the camera.",
    "19. One or more non-transitory machine-readable storage media comprising a plurality of instructions stored thereon that, in response to execution by a robot controller, cause the robot controller to: operate a camera of a pan tilt unit to move the camera about a first rotational axis of the pan tilt unit to at least three different first axis positions; record a first set of positions of a robot tool of a robot in a frame of reference of the robot and a position of the camera in a frame of reference of the pan tilt unit during a period in which the robot tool is within a field of view of the camera for each of the at least three different first axis positions; operate the camera to move the camera about a second rotational axis of the pan tilt unit to at least three different second axis positions; record a second set of positions of the robot tool in the frame of reference of the robot and a position of the camera in the frame of reference of the pan tilt unit during a period in which the robot tool is within a field of view of the camera for each of the at least three different second axis positions; and determine a transformation from the frame of reference of the robot to the frame of reference of the pan tilt unit based on the first set of recorded positions, the second set of recorded positions, a first origin of the frame of reference of the robot, a second origin of the frame of reference of the pan tilt unit, a first intersection line between one or more planes generated using the first set of recorded positions and the second set of recorded positions, and a second intersection line between one or more planes generated using the at least three different first axis positions and the at least three different second axis positions.",
    "20. The one or more non-transitory machine-readable storage media of claim 19, wherein to determine the transformation comprises to: determine a first plane defined by the first set of recorded positions; determine a second plane defined by the second set of recorded positions; determine a third plane defined by the at least three different first axis positions; determine a fourth plane defined by the at least three different second axis positions determine the first intersection line of the first plane and the second plane; determine the second intersection line between the third plane and the fourth plane; determine the first origin of the frame of reference of the robot based on the first intersection line; and determine the second origin of the frame of reference of the pan tilt unit based on the second intersection line."
  ],
  "description_excerpt": "Robot tools are often monitored (e.g., tracked and/or visualized) for remote control applications, telepresence robotic applications, robotic program debugging, and various other robotic applications. For example, in many embodiments, a tool positioned at the end of an articulated arm of a robot is monitored by a camera capable of moving to change its field of view (e.g., a pan tilt unit). In order to monitor the robot tool and/or ensure that the robot tool is maintained within the field of view of the camera, computer vision and image processing algorithms are generally employed. It should be appreciated that computer vision and image processing algorithms and techniques may be significantly computationally intensive and/or result in delay associated with a large number of real-time computations.\n\nAccording to one aspect, a robot controller for calibrating a pan tilt unit may include an arm control module configured to operate an articulating arm and a robot tool of the robot, a pan tilt unit control module to move a camera of the pan tilt unit about a first rotational axis of the pan tilt unit to at least three different first axis positions and move the camera about a second rotational axis of the pan tilt unit to at least three different second axis positions, a position recording module to record a first set of positions of the robot tool in a frame of reference of the robot and a position of the camera in a frame of reference of the pan tilt unit during a period in ...",
  "cpc": [
    "B25J 9/1692",
    "B25J 9/161",
    "B25J 9/1697",
    "G05B 2219/40617",
    "G06K 9/52",
    "G06T 7/20",
    "G06T 7/60",
    "G06T 7/70",
    "H04N 17/002",
    "H04N 23/66",
    "H04N 23/695",
    "H04N 5/232",
    "H04N 5/23203",
    "H04N 7/18",
    "Y10S 901/09"
  ],
  "ipc": [
    "B25J 9/16",
    "G06K 9/52",
    "G06T 7/00",
    "G06T 7/20",
    "G06T 7/60",
    "G06T 7/70",
    "H04N 17/00",
    "H04N 5/232",
    "H04N 7/18"
  ],
  "assignees": [
    "ABB Schweiz AG"
  ],
  "inventors": [
    "Remus Boca",
    "Jianjun Wang",
    "Thomas Fuhlbrigge",
    "Biao Zhang"
  ],
  "filing_date": "2015-06-30",
  "publication_date": "2017-10-31",
  "grant_date": "2017-10-31",
  "priority_date": "2015-06-30",
  "application_number": "US-201514755476-A",
  "family_id": "56551547",
  "cited_by_count": 1,
  "citations": [
    "EP1607194A2"
  ]
}