Output control apparatus, display terminal, remote control system, control method, and non-transitory computer-readable medium

{
  "publication_number": "US11991477B2",
  "country": "US",
  "kind": "B2",
  "title": "Output control apparatus, display terminal, remote control system, control method, and non-transitory computer-readable medium",
  "abstract": "An output control apparatus is communicable with a communication apparatus through a communication network. The communication apparatus includes a first image capturing device configured to capture a subject at a remote site to acquire a first image and a second image capturing device configure to capture a part of the subject to acquire a second image. The output control apparatus includes circuitry to: receive the first image transmitted from the communication apparatus; output the received first image so as to be displayed on a display; receive, from the communication apparatus, the second image acquired by capturing a part of the subject corresponding to a display position of the first image displayed on the display; output the received second image so as to be displayed on the display; and control the display to display the first image and the second image that are output.",
  "claims": [
    "1. A display terminal communicable with a communication apparatus including an image capturing device configured to capture an image of a subject, the communication apparatus being provided in a mobile object located at a remote site, the display terminal comprising: circuitry configured to receive, from the mobile object, state information indicating a movement state of the mobile object; receive a wide-angle image captured by the image capturing device of the communication apparatus; transmit, to the mobile object, a request for changing an image quality of the wide-angle image, based on the received state information of the mobile object; control a display to display an area of the wide-angle image, the wide-angle image being represented as a solid sphere in a three-dimensional virtual space; change a position of a virtual camera, provided at a position of a viewpoint from which the wide-angle image is viewed, in accordance with movement of an external input device coupled to the display terminal to change the area displayed on the display; generate a display image in which an object image, indicating a position of the mobile object, is superimposed on the wide-angle image; locate the object image at a center position of the three-dimensional virtual space; and control the display to display the display image, wherein the circuitry generates the display image as being viewed from the virtual camera which is at a position away from the center position.",
    "2. The display terminal of claim 1, wherein the circuitry is further configured to detect the movement of the external input device, control the display to display a first image indicating a first area of the wide-angle image, and control the display to display a second image, indicating a second area of the wide-angle image and the second area being different from the first area, instead of the first image, in response to detecting the movement of the external input device.",
    "3. The display terminal of claim 2, wherein the circuitry is further configured to calculate a movement amount of the external input device, and control the display to display the second image indicating the second area, and the second area is away from a center of the first area displayed on the display by a distance corresponding to the calculated movement amount.",
    "4. The display terminal of claim 1, wherein the circuitry generates, as the display image, an image viewed from the virtual camera which is provided behind the center position so that the object image is viewable.",
    "5. The display terminal of claim 1, wherein in response to a change of the area displayed on the display, the circuitry generates, as the display image, an image viewed from the virtual camera whose position is changed so that the object image is viewable.",
    "6. The display terminal of claim 1, wherein the circuitry generates the display image in which a drive direction image indicating a drive direction of the mobile object is superimposed on the wide-angle image.",
    "7. The display terminal of claim 6, wherein the circuitry is further configured to locate the drive direction image in the three-dimensional virtual space and generates an image viewed from the virtual camera as the display image.",
    "8. A remote control system, comprising: the display terminal of claim 1; and the mobile object, including: the communication apparatus; and a first display configured to indicate a direction in which a user of the display terminal is looking at the wide-angle image.",
    "9. The remote control system of claim 8, wherein the mobile object further includes second circuitry configured to receive, from the display terminal, a captured image obtained by capturing a user of the display terminal, and the first display displays the received captured image.",
    "10. The remote control system according to claim 8, wherein the first display is one of a convex display and a flat display provided at a front side and at a rear side of the mobile object.",
    "11. The remote control system of claim 8, wherein the first display includes an indicator lamp configured to turned on indicating a direction in which the user of the display terminal is looking at the wide-angle image.",
    "12. The remote control system of claim 8, wherein the first display is provided on an exterior of the mobile object.",
    "13. A display control method performed by a display terminal communicable with a communication apparatus including an image capturing device configured to capture an image of a subject, the communication apparatus being provided in a mobile object located at a remote site, the display control method comprising: receiving, from the mobile object, state information indicating a movement state of the mobile object; receiving a wide-mule image captured by the image capturing device of the communication apparatus; transmitting, to the mobile object, a request for changing an image quality of the wide-angle image, based on the received state information of the mobile object; controlling a display to display an area of the wide-angle image, the wide-angle image being represented as a solid sphere in a three-dimensional virtual space; changing a position of a virtual camera, provided at a position of a viewpoint from which the wide-angle image is viewed, in accordance with movement of an external input device that is coupled to the display terminal to change the area displayed on the display; generating a display image in which an object image, indicating a position of the mobile object, is superimposed on the wide-angle image; locating the object image at a center position of the three-dimensional virtual space; and controlling the display to display the display image, wherein the generating includes generating the display image as being viewed from the virtual camera which is at a position away from the center position.",
    "14. The display control method of claim 13, further comprising: detecting the movement of the external input device; controlling the display to display a first image indicating a first area of the wide-angle image; and controlling the display to display a second image, indicating a second area of the wide-angle image and the second area being different from the first area, instead of the first image, in response to detecting the movement of the external input device.",
    "15. The display control method of claim 14, further comprising: calculating a movement amount of the external input device; and controlling the display to display the second image indicating the second area, wherein the second area is away from a center of the first area displayed on the display by a distance corresponding to the calculated movement amount.",
    "16. A non-transitory computer-readable medium storing computer executable instructions which, when executed by a display terminal communicable with a communication apparatus including an image capturing device configured to capture an image of a subject, the communication apparatus being provided in a mobile object located at a remote site, cause the display terminal to: receive, from the mobile object, state information indicating a movement state of the mobile object; receive a wide-angle image captured by the image capturing device of the communication apparatus; transmit, to the mobile object, a request for changing an image quality of the wide-angle image, based on the received state information of the mobile object; control a display to display an area of the wide-angle image, the wide-angle image being represented as a solid sphere in a three-dimensional virtual space; change a position of a virtual camera, provided at a position of a viewpoint from which the wide-angle image is viewed, in accordance with movement of an external input device that is coupled to the display terminal to change the area displayed on the display; generate a display image in which an object image, indicating a position of the mobile object, is superimposed on the wide-angle image; locate the object image at a center position of the three-dimensional virtual space; and control the display to display the display image, wherein the display image is generated as being viewed from the virtual camera which is at a position away from the center position."
  ],
  "description_excerpt": "Technical Field The present disclosure relates to an output control apparatus, a display terminal, a remote control system, a control method, and a non-transitory computer readable medium. Description of the Related Art A remote control system is known in which a telepresence robot (hereinafter, referred to as a “robot” in order to simplify the description) provided at a remote site is remotely controlled using a display terminal located at a different site through a communication network. This remote control system controls the display terminal to display an image captured by an image capturing device provided at the robot, whereby allowing a user to check information of the site where the robot is provided from a remote location. Another technique is known that uses a robot provided with different types of image capturing devices, to allow a user to check surroundings of the robot in more detail. For example, a radio-controlled mobile object is known that includes a front camera and a rear camera that images the front and the rear of a traveling apparatus and an overhead camera that can image the entire surroundings around the mobile object.\n\nAccording to an embodiment, an output control apparatus is communicable with a communication apparatus through a communication network. The communication apparatus includes a first image capturing device configured to capture a subject at a remote site to acquire a first image and a second image capturing device configure to capture a part of the subject to acquire a second image.",
  "cpc": [
    "H04N 7/181",
    "B25J 19/023",
    "B25J 5/005",
    "B25J 9/1689",
    "B25J 9/1697",
    "G05B 2219/40298",
    "G05B 2219/40617",
    "G05D 1/0038",
    "H04L 67/12",
    "H04N 13/117",
    "H04N 13/183",
    "H04N 23/66",
    "H04N 23/695",
    "H04N 23/698",
    "H04N 23/90",
    "H04N 5/272",
    "H04N 7/142",
    "H04N 7/147",
    "H04N 7/185"
  ],
  "ipc": [
    "B25J 9/16",
    "H04L 67/12",
    "H04N 13/117",
    "H04N 13/183",
    "H04N 23/66",
    "H04N 23/695",
    "H04N 23/698",
    "H04N 23/90",
    "H04N 5/272",
    "H04N 7/18"
  ],
  "assignees": [
    "RICOH CO LTD"
  ],
  "inventors": [
    "ITOH ATSUSHI"
  ],
  "filing_date": "2021-12-28",
  "publication_date": "2024-05-21",
  "grant_date": "2024-05-21",
  "priority_date": "2019-07-31",
  "application_number": "US-202117563079-A",
  "family_id": "71786853",
  "citations": [
    "JP2006270175A",
    "JP2007288527A",
    "JP2013112030A",
    "JP2016031576A",
    "JP2017041780A",
    "JP2017218105A",
    "JP2019087974A",
    "JP2019114194A",
    "JP2020005146A",
    "JP2020017829A",
    "JP2020022152A",
    "KR20160094168A",
    "US2003132948A1",
    "US2004100563A1",
    "US2006092181A1",
    "US2007078566A1",
    "US2007165033A1",
    "US2008123903A1",
    "US2008239102A1",
    "US2009013262A1",
    "US2009147090A1",
    "US2011228985A1",
    "US2011298885A1",
    "US2012094754A1",
    "US2012229507A1",
    "US2013215146A1",
    "US2014085661A1",
    "US2014316611A1",
    "US2015208070A1",
    "US2016046024A1",
    "US2016088287A1",
    "US2016196030A1",
    "US2016277772A1",
    "US2017220879A1",
    "US2017257576A1",
    "US2018009108A1",
    "US2018013957A1",
    "US2018104591A1",
    "US2018107863A1",
    "US2018177561A1",
    "US2018210442A1",
    "US2018247457A1",
    "US2018279847A1",
    "US2018329609A1",
    "US2018332219A1",
    "US2019011981A1",
    "US2019089643A1",
    "US2019101405A1",
    "US2019149744A1",
    "US2019205003A1",
    "US2019289201A1",
    "US2019289203A1",
    "US2020007751A1",
    "US2020036892A1",
    "US2020084432A1",
    "US6809760B1"
  ]
}