Method for displaying image and electronic device thereof

{
  "publication_number": "US10217443B2",
  "country": "US",
  "kind": "B2",
  "title": "Method for displaying image and electronic device thereof",
  "abstract": "An electronic device and a method for displaying an image transmitted by a robot and remotely controlling a movement of the robot are provided. The electronic device includes a communicator, an inputter, a display, and a controller. The communicator receives, from the robot, an image photographed by the robot and time information. The inputter receives a command to control the robot. The display displays a screen corresponding to the image. The controller calculates a time delay between the electronic device and the robot based on the time information, control the display to display a graphical object to be overlaid on the screen, and control the communicator to transmit the command to the robot. The graphical object represents a field of view of the robot at a current time. The field of view of the robot at the current time may be estimated based on the time delay information.",
  "claims": [
    "1. An electronic device for remotely controlling a movement of a robot, the electronic device comprising: a communicator; an inputter; a display; and a processor configured to: control the communicator to receive, from the robot, a first time and an image photographed by the robot at the first time, calculate a time delay between the electronic device and the robot based on a difference between the first time and a second time, which is a current time of the electronic device after the first time, obtain a position of the robot at the second time based on the time delay, control the display to display the image and a graphical object, which indicates information corresponding to a viewpoint of the robot from the obtained position at the second time, to be overlaid on the image, and control the communicator to transmit a command to control the robot, received through the inputter, to the robot.",
    "2. The electronic device as claimed in claim 1, wherein the communicator is further configured to receive, from the robot, a moving speed of the robot, and the processor is further configured to determine a position of the graphical object overlaid on the image based on the moving speed of the robot.",
    "3. The electronic device as claimed in claim 1, wherein the graphical object is a quadrilateral having a same aspect ratio as the display.",
    "4. The electronic device as claimed in claim 1, wherein the image is photographed by a camera included in the robot, and the processor is further configured to determine a position and a size of the graphical object to correspond to a field of view of the camera based on a distance that the robot travels during a time corresponding to the time delay.",
    "5. The electronic device as claimed in claim 1, wherein a size of the graphical object changes as time elapses.",
    "6. The electronic device as claimed in claim 1, wherein the processor is further configured to control the display to display the image and the graphical object in a first area of the display, and display a time bar and a delay bar based on the time delay in a second area of the display.",
    "7. The electronic device as claimed in claim 6, wherein a length of the delay bar changes as time elapses.",
    "8. The electronic device as claimed in claim 7, wherein the processor is further configured to control the display to display a second graphical object corresponding to the command in the second area.",
    "9. The electronic device as claimed in claim 6, wherein the communicator is further configured to receive, from the robot, information regarding a change in direction of the robot, and the processor is further configured to determine a display position of the graphical object based on the information regarding the change in direction, and control the display to display the graphical object to be overlaid at the display position on the display.",
    "10. The electronic device as claimed in claim 9, wherein the processor is further configured to control the display to display a rotation axis around which the robot is rotating.",
    "11. The electronic device as claimed in claim 1, wherein the processor is further configured to control the display to display the graphical object to be overlaid on the image further based on a moving speed of the robot.",
    "12. A method for displaying a screen of an electronic device remotely controlling a movement of a robot, the method comprising: receiving, from the robot, a first time and an image photographed by the robot at the first time; calculating a time delay between the electronic device and the robot based on a difference between the first time and a second time, which is a current time of the electronic device after the first time; obtaining a position of the robot at the second time based on the time delay; displaying the image and a graphical object, which indicates information corresponding to a viewpoint of the robot from the obtained position at the second time, to be overlaid on the image.",
    "13. The method as claimed in claim 12, wherein the displaying the graphical object comprises determining a position of the graphical object based on a moving speed of the robot.",
    "14. The method as claimed in claim 12, wherein the graphical object is a quadrilateral having a same aspect ratio as the screen.",
    "15. The method as claimed in claim 14, wherein a size of the graphical object changes as time elapses.",
    "16. The method as claimed in claim 15, wherein the method further comprises displaying the image and the graphical object in a first area of the screen, and displaying a time bar and a delay bar based on the time delay in a second area of the screen.",
    "17. The method as claimed in claim 16, wherein a length of the delay bar changes as time elapses.",
    "18. The method as claimed in claim 17, wherein the method further comprises displaying a second graphical object corresponding to a command in the second area.",
    "19. The method as claimed in claim 16, wherein the method further comprises receiving, from the robot, information regarding a change in direction of the robot, determining a display position of the graphical object based on the information regarding the change in direction, and displaying the graphical object to be overlaid at the display position on the screen.",
    "20. The method as claimed in claim 19, wherein the method further comprises displaying on the screen a rotation axis around which the robot rotating.",
    "21. The method as claimed in claim 12, wherein the displaying the graphical object comprises determining a position and a size of the graphical object to correspond to a field of view of a camera of the robot based on a distance that the robot travels during a time corresponding to the time delay.",
    "22. The method as claimed in claim 12, wherein the graphical object is overlaid on the screen further based on a moving speed of the robot."
  ],
  "description_excerpt": "1. Field\n\nApparatuses and methods consistent with exemplary embodiments relate to an electronic device capable of remotely controlling a mobile robot and a method of displaying on a screen of the electronic device an image photographed by the mobile robot and information relevant thereto.\n\n2. Description of the Related Art\n\nWith advancements in electronic and robotic technology, systems have been developed where a camera is installed at a movable robot, an image is photographed by the robot, the photographed image is remotely monitored through communication, and the robot is remotely controlled.\n\nIn systems which perform remote monitoring by using such a robot, a remote user can control a mobile robot by watching on a mobile terminal the images photographed by the mobile robot. That is, a user may move the mobile robot in various directions (e.g., left, right, up, down) while watching on the mobile terminal a series of images transmitted by the mobile robot. However, while processing the images photographed by the mobile robot, time delay may occur. Specifically, due to the delay in transmission in the communication network which connects the mobile robot with the remote user's mobile terminal, time delay may occur between an image displayed on the mobile terminal of the user and the remote control signal of the user, and thus, it may be difficult for the user to control the mobile robot.\n\nExemplary embodiments may overcome the above disadvantages and other disadvantages not described above.",
  "cpc": [
    "G09G 5/377",
    "B25J 9/161",
    "B25J 9/1697",
    "G01C 11/00",
    "G01C 23/00",
    "G05D 1/2247",
    "G05D 1/226",
    "G05D 2109/254",
    "G06F 3/1415",
    "G06F 3/1454",
    "G09G 2320/106",
    "G09G 2340/12",
    "G09G 5/006",
    "G09G 5/38",
    "H04N 23/661"
  ],
  "ipc": [
    "B25J 9/16",
    "G06F 3/14",
    "G09G 5/00",
    "G09G 5/377",
    "G09G 5/38"
  ],
  "assignees": [
    "Samsung Electronics Co Ltd"
  ],
  "inventors": [
    "Yoon-sang Kim",
    "Jae-keun Na",
    "Ho-Chul Shin"
  ],
  "filing_date": "2017-01-10",
  "publication_date": "2019-02-26",
  "grant_date": "2019-02-26",
  "priority_date": "2016-01-28",
  "application_number": "US-201715402709-A",
  "family_id": "59385632",
  "cited_by_count": 0,
  "citations": [
    "US20080121097A1",
    "US7641613B2",
    "JP2005161498A",
    "WO2008097252A2",
    "US20080046940A1",
    "US8718822B1",
    "US20140032013A1",
    "US8954853B2",
    "US20150121222A1",
    "US20150057801A1",
    "US20140180478A1",
    "KR20140103494A",
    "WO2014126303A1",
    "WO2014170439A1",
    "JP2015047666A",
    "EP3051808A1",
    "US20160274743A1"
  ]
}