Data processing apparatus, data processing method, and data processing system

{
  "publication_number": "US12367553B2",
  "country": "US",
  "kind": "B2",
  "title": "Data processing apparatus, data processing method, and data processing system",
  "abstract": "A data processing apparatus is disclosed which processes data for operating a movable segment of a machine in correspondence with a position of a controller. The data processing apparatus includes a control unit or processor configured to obtain controller information including the position, when an operator operates the movable segment using the controller, obtain capture image data generated by image capturing of the movable segment while the movable segment is operated in correspondence with the position, generate composite image data by superimposing a controller image indicating the position included in the controller information, on an area corresponding to the movable segment on the capture image data, and display the composite image data on a display device viewable by the operator.",
  "claims": [
    "1. A data processing apparatus for processing data for operating a movable segment of a machine in correspondence with a position of a controller, the data processing apparatus comprising a processor configured to: obtain controller information including the position, when an operator operates the movable segment using the controller; obtain capture image data generated by image capturing of the movable segment while the movable segment is operated depending on the position; generate composite image data by superimposing a controller image indicating the position included in the controller information, on an area corresponding to the movable segment on the capture image data; display the composite image data on a display device viewable by the operator; and identify a difference between an actual position of the movable segment and a computational position of the movable segment calculated based on the position included in the controller information.",
    "2. The data processing apparatus according to claim 1, wherein the processor is configured to obtain the controller information including orientation of the controller in addition to the position; and generate the composite image data by superimposing the controller image indicating the position and orientation included in the controller information.",
    "3. The data processing apparatus according to claim 2, wherein the controller image is an image indicating a fingertip or a wrist of the operator holding the controller, corresponding to the orientation included in the controller information.",
    "4. The data processing apparatus according to claim 1, wherein, in the case where the movable segment moved, the control unit is configured to superimpose the controller image on an area corresponding to the movable segment after movement, on the capture image data.",
    "5. The data processing apparatus according to claim 1, wherein the processor is configured to associate an image indicating the state where the controller is operated with the controller image, and superimpose the image indicating a state where the controller is operated with the controller image on the capture image data.",
    "6. The data processing apparatus according to claim 1, wherein the processor is configured to superimpose an image indicating operation to be performed next, for the controller, according to task content scheduled to be performed next by the operator, on the capture image data.",
    "7. The data processing apparatus according to claim 1, wherein the processor is configured to: obtain the controller information including the orientation of the controller in addition to the position; and identify a difference between actual position and orientation of the movable segment and computational position and orientation of the movable segment based on the position and the orientation included in the controller information.",
    "8. The data processing apparatus according to claim 1, wherein the processor is configured to associate the difference with the controller image, and display the difference on the display device.",
    "9. The data processing apparatus according to claim 1, wherein the processor is configured to further perform operation of the movable segment in correspondence with the position included in the controller information, after making correction corresponding to the difference for the position included in the controller information.",
    "10. The data processing apparatus according to claim 9, wherein, on condition that the difference is not more than a predetermine value, the processor is configured to operate the movable segment in correspondence with the position included in the controller information.",
    "11. A data processing method for processing data for operating a movable segment of a machine in correspondence with a position of a controller, the method comprising: obtaining controller information including the position, when an operator operates the movable segment using the controller; obtaining capture image data generated by image capturing of the movable segment while the movable segment is operated depending on the position; generating composite image data by superimposing a controller image indicating the position included in the controller information, on an area corresponding to the movable segment on the capture image data; displaying the composite image data on a display device viewable by the operator; and identifying a difference between an actual position of the movable segment and a computational position of the movable segment calculated based on the position included in the controller information.",
    "12. A non-transitory computer program product having computing program instructions thereon wherein the computing program instructions when executed by a computer, configure the computer to: process data for operating a movable segment of a machine in correspondence with a position of a controller; obtain controller information including the position, when an operator operates the movable segment using the controller; obtain capture image data generated by image capturing of the movable position while the movable position is operated in correspondence with the position; generate composite image data by superimposing a controller image indicating the position included in the controller information, on an area corresponding to the movable segment on the capture image data; display the composite image data on a display device viewable by the operator; and identify a difference between an actual position of the movable segment and a computational position of the movable segment calculated based on the position included in the controller information.",
    "13. A data processing system comprising: an operation device associated with a controller for operating a machine; and a data processing apparatus for processing data for controlling a movable segment of the apparatus in correspondence with the position of the controller, wherein the data processing apparatus has a processor configured to: obtain controller information including the position, when an operator operates the movable segment using the controller; obtain capture image data generated by image capturing of the movable segment while the movable segment is operated in correspondence with the position; generate composite image data by superimposing a controller image indicating the position included in the controller information, on an area corresponding to the movable segment on the capture image data; display the composite image data on a display device viewable by the operator; and identify a difference between an actual position of the movable segment and a computational position of the movable segment calculated based on the position included in the controller information; and wherein the operation controller comprises: a communication unit configured to transmit the controller information based on contents of operation by the operator, to the data processing apparatus; and a display unit viewable to the operator, and configured to display the composite image data transmitted by the data processing apparatus."
  ],
  "description_excerpt": "The present disclosure relates to a data processing apparatus, a data processing method, and a data processing system.\n\nJP2020-040155 discloses a technique of operating a robot using a controller attached to an operator by remote control, and displaying a capture image generated by an image capture device provided for the robot, on a terminal used by the operator who operates the robot.\n\nIn the case where an operator operates a robot while viewing a capture image around a machine such as a robot, the operator cannot view the controller at hand. Therefore, there have been problems that, since it is difficult for the operator to recognize the position and orientation of the controller and the positions and orientations of movable segments of the machine at the same time, the operator cannot operate the machine easily.\n\nThe present disclosure has been made taking this point into account, and an object of the present disclosure is to provide a data processing apparatus, etc. which make it possible for an operator of a machine such as a robot to operate the machine using a controller easily.\n\nA data processing apparatus according to an aspect of the present disclosure processes data for operating a movable segment of a machine in correspondence with a position of a controller.",
  "cpc": [
    "G06T 5/50",
    "B25J 13/06",
    "B25J 3/00",
    "B25J 9/1697",
    "G06F 3/0304",
    "G06F 3/0346",
    "G06F 3/04815",
    "G06F 3/04847",
    "G06T 11/60",
    "G06T 2207/20221",
    "G06T 7/70",
    "G09G 5/377",
    "H04N 21/4131",
    "H04N 21/4223",
    "H04N 21/431",
    "H04N 21/4316",
    "H04Q 2209/40",
    "H04Q 9/00"
  ],
  "ipc": [
    "G06T 5/50",
    "G06T 7/70",
    "G09G 5/377"
  ],
  "assignees": [
    "Telexistence Inc"
  ],
  "inventors": [
    "Jade FOUNTAIN",
    "Yusuke Mizushina"
  ],
  "filing_date": "2022-04-26",
  "publication_date": "2025-07-22",
  "grant_date": "2025-07-22",
  "priority_date": "2021-04-28",
  "application_number": "US-202218286912-A",
  "family_id": "83848406",
  "cited_by_count": 0,
  "citations": [
    "JP2011189431A",
    "JP2014065100A",
    "US20160288318A1",
    "US20200163731A1"
  ]
}