Remote terminal and method for displaying image of designated area received from mobile robot

{
  "publication_number": "US11375162B2",
  "country": "US",
  "kind": "B2",
  "title": "Remote terminal and method for displaying image of designated area received from mobile robot",
  "abstract": "A mobile robot including an image pickup unit, further includes an accumulation unit configured to accumulate imaging data taken in the past, a reception unit configured to receive designation of a spatial area from a remote terminal, and a processing unit configured, when it is possible to shoot the spatial area received by the reception unit by the image pickup unit, to perform shooting and transmit obtained imaging data to the remote terminal, and when the shooting is impossible, to transmit imaging data including an image of the spatial area accumulated in the accumulation unit to the remote terminal and start a moving process in order to shoot the spatial area by the image pickup unit.",
  "claims": [
    "1. A remote terminal configured to receive imaging data from a mobile robot including an image pickup unit and display the received imaging data, comprising: a communication unit configured to: transmit a designation signal to the mobile robot, the designation signal being generated by designating a spatial area that a user wants to observe; and receive the imaging data for the spatial area from the mobile robot; and a CPU programmed to: determine whether or not the imaging data for the spatial area received by the communication unit is one that is obtained by performing shooting of the spatial area by using the image pickup unit after the communication unit transmits the designation signal; control a display unit to display an image of the imaging data that was obtained before the communication unit transmits the designation signal in the display unit with an indication that the imaging data for the spatial area was obtained before the communication unit transmits the designation signal; and control the display unit to display an image of the imaging data that was obtained after the communication unit transmits the designation signal in the display unit with an indication that the imaging data for the spatial area was obtained after the communication unit transmits the designation signal.",
    "2. The remote terminal according to claim 1, wherein when the display unit displays an image of imaging data that is determined, by the CPU, not to be the one that is obtained by performing shooting by using the image pickup unit after the communication unit transmits the designation signal, and then the CPU determines that imaging data newly received by the communication unit is the one that is obtained by performing shooting by using the image pickup unit after the communication unit transmits the designation signal, the display unit displays an image of the newly-received imaging data in the display unit.",
    "3. The remote terminal according to claim 1, wherein when imaging data that is received by the communication unit while the display unit is displaying an image of imaging data that is determined, by the CPU, not to be the one that is obtained by performing shooting by using the image pickup unit after the communication unit transmits the designation signal contains additional information indicating that the received imaging data is one that is obtained while the mobile robot is moving, the display unit displays an image of the imaging data containing the additional information together with the currently-displayed image.",
    "4. The remote terminal according to claim 1, wherein the display unit displays, as an initial displayed image, an image of imaging data that the communication unit has received from the mobile robot before the communication unit transmits the designation signal in the display unit, and the communication unit transmits, to the mobile robot, the designation signal generated for the spatial area designated by the user by using the image displayed in the display unit as a reference.",
    "5. A non-transitory computer readable medium storing a control program for a remote terminal configured to receive imaging data from a mobile robot including an image pickup unit and display the received imaging data, for causing a computer to execute: a transmitting step of transmitting a designation signal to the mobile robot, the designation signal being generated by designating a spatial area that a user wants to observe; a receiving step of receiving the imaging data for the spatial area from the mobile robot; a determining step of determining whether or not the imaging data for the spatial area received in the receiving step is one that is obtained by performing shooting of the spatial area by using the image pickup unit after the designation signal is transmitted in the transmitting step; a display control step of displaying an image of the imaging data that was obtained before transmitting the designation signal in a display unit with an indication that the imaging data for the spatial area was obtained before transmitting the designation signal; and a display control step of displaying an image of the imaging data that was obtained after transmitting the designation signal in the display unit with an indication that the imaging data for the spatial area was obtained after transmitting the designation signal.",
    "6. A control method for a remote terminal configured to receive imaging data from a mobile robot including an image pickup unit and display the received imaging data, comprising: a transmitting step of transmitting a designation signal to the mobile robot, the designation signal being generated by designating a spatial area that a user wants to observe; a receiving step of receiving the imaging data for the spatial area from the mobile robot; a determining step of determining whether or not the imaging data for the spatial area received in the receiving step is one that is obtained by performing shooting of the spatial area by using the image pickup unit after the designation signal is transmitted in the transmitting step; a display control step of displaying an image of the imaging data that was obtained before transmitting the designation signal in a display unit with an indication that the imaging data for the spatial area was obtained before transmitting the designation signal; and a display control step of displaying an image of the imaging data that was obtained after transmitting the designation signal in the display unit with an indication that the imaging data for the spatial area was obtained after transmitting the designation signal."
  ],
  "description_excerpt": "The present disclosure relates to a mobile robot, a remote terminal, a control program for a mobile robot, a control program for a remote terminal, a control system, a control method for a mobile robot, and a control method for a remote terminal.\n\nA technique for performing work by operating a mobile robot located in a remote place is known. Further, a technique for observing a remote place through images sent from a mobile robot that autonomously moves is known (see, for example, International Patent Publication No. WO 2009/011380).\n\nThe present inventors have found the following problem. There are cases where an operator in a remote place wants to check a spatial area that cannot be observed by a mobile robot at that moment (e.g., from the current place) in addition to checking a spatial area that can be observed at that moment. However, it takes some time for the mobile robot to move to the designated spatial area that cannot be observed and hence, during the period in which the mobile robot is moving, a user who remotely operates the mobile robot is just kept waiting.\n\nThe present disclosure provides a mobile robot and the like that enables a user to check a situation of a spatial area that the user wants to check without delay.",
  "cpc": [
    "H04N 7/185",
    "B25J 19/023",
    "B25J 5/007",
    "B25J 9/1689",
    "B25J 9/1697",
    "G05D 1/0038",
    "G05D 1/0044",
    "G05D 1/0251",
    "G05D 1/0263",
    "G05D 1/0276",
    "H04N 13/239",
    "H04N 23/63",
    "H04N 23/64",
    "H04N 23/66",
    "H04N 23/661",
    "H04N 23/695",
    "H04N 5/23203",
    "H04N 5/23299",
    "H04N 5/445",
    "H04N 5/77",
    "H04N 5/772",
    "H04N 5/9305"
  ],
  "ipc": [
    "B25J 9/16",
    "H04N 5/232",
    "H04N 5/445",
    "H04N 5/93",
    "H04N 7/18"
  ],
  "assignees": [
    "Toyota Motor Corp"
  ],
  "inventors": [
    "Takashi Yamamoto"
  ],
  "filing_date": "2020-07-22",
  "publication_date": "2022-06-28",
  "grant_date": "2022-06-28",
  "priority_date": "2018-09-06",
  "application_number": "US-202016935623-A",
  "family_id": "69719767",
  "cited_by_count": 0,
  "citations": [
    "JP2001315080A",
    "WO2004032086A1",
    "US20050159841A1",
    "US20060076917A1",
    "US20100188188A1",
    "WO2009011380A1",
    "US20090135264A1",
    "US20120215380A1",
    "US20150100461A1",
    "US20120277914A1",
    "US20140303775A1",
    "US20170171513A1",
    "JP2016068161A",
    "US10000284B1"
  ]
}