Remote communication method, remote communication system, and autonomous movement device

{
  "publication_number": "US10321065B2",
  "country": "US",
  "kind": "B2",
  "title": "Remote communication method, remote communication system, and autonomous movement device",
  "abstract": "A remote communication method of a remote communication system including a first robot deployed in a first place and a first presentation device deployed in a second place, includes capturing an image of a first user in the first place with a camera, transmitting the captured image of the first user from the first robot to the first presentation device, receiving the captured image from the first robot, displaying the captured image, detecting a first amount of change indicating a change in a position of a second user in the second place with reference to a second robot that virtually represents the first user, transmitting the first amount of change to the first robot, receiving the first amount of change from the first presentation device, and moving the first robot to change a point of view of the camera in response to the first amount of change.",
  "claims": [
    "1. A remote communication method of a remote communication system including a first robot deployed in a first place and a first presentation device deployed in a second place different from the first place, the remote communication method comprising: capturing, using a camera of the first robot, an image of a first user in the first place; transmitting, using the first robot, the captured image of the first user to the first presentation device; receiving, using the first presentation device, the captured image from the first robot; displaying, using the first presentation device, the captured image; detecting, using the first presentation device, a first amount of change indicating a change in a position of a second user in the second place with reference to a second robot that virtually represents the first user; transmitting, using the first presentation device, the first amount of change to the first robot; receiving, using the first robot, the first amount of change from the first presentation device, the first robot moving to change a point of view of the camera in response to the first amount of change.",
    "2. The remote communication method according to claim 1, wherein the first presentation device is worn by the second user, the communication method further comprises: measuring, using the first presentation device, an initial distance between the second robot and the first presentation device when the second user wearing the first presentation device faces the second robot; setting, using the first presentation device, a first polar coordinate system having an origin thereof at a position of the second robot, and setting a straight line connecting the position of the second robot to a position of the first presentation device to be a polar axis of the first polar coordinate system; and transmitting, using the first presentation device, the measured initial distance to the first robot, receiving, using the first robot, the initial distance from the first presentation device, the first robot moving to a position that is apart from the first user by the initial distance, wherein the position is a position at which the first robot faces to the first user; and setting, using the first robot, a second polar coordinate system having an origin thereof at a position of the first user, and setting a straight line connecting the position of the first robot to the position of the first user to be a polar axis of the second polar coordinate system.",
    "3. The remote communication method according to claim 2, further comprising: with the first presentation device moved in position in response to a movement of the second user, measuring, using the first presentation device, a distance between a position of the first presentation device after being moved and the position of the second robot in the first polar coordinate system; measuring, using the first presentation device, an angle made by a straight line connecting the position of the first presentation device after being moved to the position of the second robot in the first polar coordinate system and the polar axis of the first polar coordinate system, wherein the first amount of change includes the measured distance and the measured angle; and identifying, using the first robot, in the first place a position to which the first robot is to be moved in response to the distance and the angle indicated by the first amount of change, wherein the first robot moves to the identified position in the moving in response to the first amount of change, and wherein a distance between the identified position and the position of the first user is equal to the distance indicated by the first amount of change, and an angle made between a straight line connecting the identified position to the position of the first user and the polar axis of the second polar coordinate system is equal to the angle indicated by the first amount of change.",
    "4. The remote communication method according to claim 3, further comprising, determining, using the first presentation device, whether the measured angle is equal to or above a predetermined angle, wherein the first amount of change is detected when it is determined that the measured angle is equal to or above the predetermined angle.",
    "5. The remote communication method according to claim 2, further comprising, emitting, using the second robot, a visible light beam in a downward direction from a front of the second robot.",
    "6. The remote communication method according to claim 2, further comprising, emitting, using the second robot, an infrared light beam from a front of the second robot, wherein the initial distance is measured when the infrared light beam, emitted from the second robot, is detected on a front of the first presentation device.",
    "7. The remote communication method according to claim 2, further comprising: detecting, using the first presentation device, a second amount of change indicating a change in an orientation of the second user's face; receiving, using the first robot, the second amount of change from the first presentation device; and changing, using the first robot, a shooting direction of the camera in response to the second amount of change by rotating the camera.",
    "8. The remote communication method according to claim 7, wherein the detecting of the second amount of change comprises measuring an angle of rotation in the orientation of the second user's face with reference to the polar axis of the first polar coordinate system, wherein the measured angle of rotation is detected as the second amount of change, wherein in the changing, the camera rotates until the angle of the shooting direction of the camera with reference to the polar axis of the second polar coordinate system matches the angle indicated by the second amount of change.",
    "9. The remote communication method according to claim 7, wherein the remote communication system further includes a second presentation device deployed in the first place and worn by the first user, and wherein the remote communication method further comprises: with the first amount of change detected, transmitting, using the first presentation device, to the second presentation device, first notification information prompting the first user to stay still before the first amount of change is transmitted to the first robot; with the second amount of change detected, transmitting, using the first presentation device, the first notification information to the second presentation device before the second amount of change is transmitted to the first robot; receiving, using the second presentation device, the first notification information from the first presentation device; and displaying, using the second presentation device, the first notification information on the second presentation device.",
    "10. The remote communication method according to claim 9, wherein the first notification information is displayed when the first notification information is received before the second presentation device detects a third amount of change indicating a change in the position of the first user with reference to the first robot; or wherein the first notification information is displayed when the first notification information is received before the second presentation device detects a fourth amount of change indicating a change in the orientation of the first user's face with reference to the first robot.",
    "11. The remote communication method according to claim 7, wherein the remote communication system includes a second presentation device deployed in the first place, wherein the second presentation device is worn by the first user, and wherein the remote communication method further comprises: transmitting, using the second presentation device, to the first presentation device, second notification information prompting the second user to stay still, when the second presentation device detects a third amount of change indicating a change in the position of the first user or a fourth amount of change indicating a change in the orientation of the first user's face with reference to the first robot; receiving, using the first presentation device, the second notification information from the second presentation device; and displaying, using the first presentation device, the second notification information on the first presentation device.",
    "12. The remote communication method according to claim 11, wherein the second notification information is displayed when the second notification information is received before the first presentation device detects the first amount of change, or wherein the second notification information is displayed when the second notification information is received before the first presentation device detects the second amount of change.",
    "13. A remote communication system comprising: a first robot deployed in a first place; and a presentation device deployed in a second place different from the first place, wherein the first robot includes: a camera that captures an image of a first user in the first place; and a transmitter that transmits the image of the first user captured by the camera to the presentation device, wherein the presentation device includes: a receiver that receives the captured image from the first robot; a display that displays the captured image; a detector that detects a first amount of change in a position of a second user with reference to a second robot that virtually represents the first user; and a transmitter that transmits the first amount of change to the first robot, and wherein the first robot further includes: a receiver that receives the first amount of change from the presentation device, a mover that moves the first robot, and a movement controller that controls the mover to change a point of view of the camera in response to the first amount of change.",
    "14. A first robot comprising: a camera that captures an image of a first user in a first place; a transmitter that transmits the image of the first user captured by the camera to a presentation device deployed in a second place different from the first place; and a receiver that receives from the presentation device a first amount of change indicating a change in a position of a second user with reference to a second robot that virtually represents the first user, wherein the first robot is deployed in the first place, and wherein the first robot further comprises: a mover that moves the first robot deployed in the first place; and a movement controller that controls the mover to change a point of view of the camera of the first robot in response to the first amount of change.",
    "15. The first robot according to claim 14, wherein the mover includes a motor that changes a position of the first robot, and the movement controller controls the motor of the mover to change the point of view of the camera of the first robot in response to the first amount of change.",
    "16. The first robot according to claim 14, wherein the movement controller controls the mover to change a position of the first robot such that the position of the first robot with reference to a position of the first user corresponds to the position of the second user with reference to a position of the second robot."
  ],
  "description_excerpt": "The present disclosure relates to a remote communication system that includes a first robot that autonomously moves and captures an image of a first user in a first place, and a first presentation device that presents the image captured by the first robot to a second user, different from the first user, in a second place different from the first place. The present disclosure also relates to a remote communication method of the remote communication system and an autonomous movement device that autonomously moves.\n\nCommunication performance of robots with humans advance day by day, and telepresence robots that are autonomous robots having teleconference and remote control functionalities have been developed. A telepresence robot that is an avatar of a user as a sender walks around in a remote area, and communicates with a user serving as a communication destination as described in Japanese Unexamined Patent Application Publication No. 2007-216363. The telepresence robot thus gives a sense of presence more than ever to the user as the communication destination while maintaining good communication. As a result, telepresence robots are beginning to be used in work-from-home business, a variety of events, offices, and educational fields.\n\nHowever, in the related art technique described above, each of persons present in the remote place is unable to have the feeling of presence as if his or her partner were present there, and there is still room for improvement.",
  "cpc": [
    "H04N 7/15",
    "B25J 13/006",
    "B25J 5/00",
    "G06T 7/20",
    "H04L 67/14",
    "H04N 23/661",
    "H04N 23/69",
    "H04N 23/695",
    "H04N 5/23296",
    "H04N 7/142",
    "H04N 7/185"
  ],
  "ipc": [
    "G06T 7/20",
    "H04N 5/232"
  ],
  "assignees": [
    "Panasonic Intellectual Property Management Co Ltd"
  ],
  "inventors": [
    "Hongjie Shi"
  ],
  "filing_date": "2018-06-08",
  "publication_date": "2019-06-11",
  "grant_date": "2019-06-11",
  "priority_date": "2017-06-23",
  "application_number": "US-201816003564-A",
  "family_id": "64692889",
  "cited_by_count": 7,
  "citations": [
    "JP2004209562A",
    "JP2004230479A",
    "JP2006146435A",
    "JP2007216363A",
    "US20170142349A1",
    "JP2016208416A"
  ]
}