Monitoring method and device for mobile target, monitoring system and mobile robot

{
  "publication_number": "US2020184658A1",
  "country": "US",
  "kind": "A1",
  "title": "Monitoring method and device for mobile target, monitoring system and mobile robot",
  "abstract": "The present application provides a monitoring method and device for a mobile target, a monitoring system and mobile robot. The present application through the technical solution that acquiring multiple-frame images captured by an image acquisition device under an moving state of a robot in a monitored region, selecting at least two-frame images with an overlapped region from the multiple-frame images, performing comparison between the selected images by image compensation method or feature matching method, and outputting monitoring information containing a mobile target which moves relative to a static target based on the result of comparison, wherein the position of the mobile target has an attribute of indefinite change, the mobile target in the monitored region can be recognized precisely during movement of the mobile robot, and monitoring information about the mobile target can be generated to prompt correspondingly, thereby safety of the monitored region can be effectively ensured.",
  "claims": [
    "1. A monitoring device for a mobile target, used in a mobile robot, the mobile robot comprises a movement device and an image acquisition device, wherein, the monitoring device for a mobile target comprises: at least one processing device; at least one storage device, configured to store images captured by the image acquisition device under an operating state of the movement device; at least one program, wherein the at least one program is stored in the at least one storage device, and is invoked by the at least one processing device such that the monitoring device performs a monitoring method for a mobile target; the monitoring method for a mobile target comprises the following steps: acquiring multiple-frame images captured by the image acquisition device under the operating state of the movement device; and outputting monitoring information containing a mobile target which moves relative to a static target according to a result of performing comparison between at least two-frame images selected from the multiple-frame images; wherein the at least two-frame images are images captured by the image acquisition device within partially overlapped field of view, and a position of the mobile target in each of the at least two-frame images has an attribute of indefinite change. 2. The monitoring device for a mobile target of claim 1, wherein the step of performing comparison between at least two-frame images selected from the multiple-frame images comprises the following steps: detecting a suspected target based on the comparison between the at least two-frame images; and tracking the suspected target to determine the mobile target. 3. The monitoring device for a mobile target of claim 2, wherein the step of detecting a suspected target according to the comparison between the at least two-frame images comprises the following steps: performing image compensation on the at least two-frame images based on movement information of the movement device within a time period between the at least two-frame images; and performing subtraction processing on the compensated at least two-frame images to form a difference image, and detecting the suspected target from the difference image. 4. The monitoring device for a mobile target of claim 1, wherein the step of performing comparison between at least two-frame images selected from the multiple-frame images comprises the following steps: detecting a suspected target based on a matching operation on corresponding feature information in the at least two-frame images; and tracking the suspected target to determine the mobile target. 5. The monitoring device for a mobile target of claim 4, wherein the step of detecting a suspected target based on a matching operation on corresponding feature information in the at least two-frame images comprises the following steps: extracting each feature point in the at least two-frame images respectively, and matching extracted each feature point in the at least two-frame images with a reference three-dimensional coordinate system; wherein the reference three-dimensional coordinate system is formed through performing three-dimensional modeling on a mobile space, and the reference three-dimensional coordinate system is marked with coordinate of each feature point on all static targets in the mobile space; and detecting a feature point set as the suspected target, the feature point set is composed of feature points in the at least two-frame images that are not matched with the reference three-dimensional coordinate system. 6. The monitoring device for a mobile target of claim 2, wherein the step of tracking the suspected target to determine the mobile target comprises the following steps: obtaining a moving track of a suspected target through tracking the suspected target; and determining the suspected target as the mobile target when the moving track of the suspected target is continuous. 7. The monitoring device for a mobile target of claim 4, wherein the step of tracking the suspected target to determine the mobile target comprises the following steps: obtaining a moving track of a suspected target through tracking the suspected target; and determining the suspected target as the mobile target when the moving track of the suspected target is continuous. 8. The monitoring device for a mobile target of claim 1, further comprising the following steps: performing object recognition on the mobile target in the captured images, wherein the object recognition is performed by an object recognizer, the object recognizer includes a trained neural network; and outputting the monitoring information according to result of the object recognition. 9. The monitoring device for a mobile target of claim 1, further comprising the following steps: uploading captured images or videos containing images to a cloud server to perform object recognition on the mobile target in the image; wherein the cloud server includes an object recognizer which includes trained neural networks; and receiving a result of object recognition from the cloud server and outputting the monitoring information. 10. The monitoring device for a mobile target of claim 1, wherein the monitoring information comprises one or more of image information, video information, audio information and text information. 11. A mobile robot, comprising: a movement device, configured to control movement of the mobile robot according to received control instruction; an image acquisition device, configured to capture multiple-frame images under an operating state of the movement device; and the monitoring device of claim 1. 12. A monitoring system, comprising: a cloud server; and a mobile robot, connected with the cloud server; wherein the mobile robot performs the following steps: acquiring multiple-frame images during movement of the mobile robot; outputting detection information containing a mobile target which moves relative to a static target according to a result of performing comparison between at least two-frame images selected from the multiple-frame images; uploading the captured images or videos containing images to the cloud server based on the detection information; and outputting monitoring information based on a result of object recognition received from the cloud server; or, wherein the mobile robot performs the following steps: acquiring multiple-frame images during movement of the mobile robot, and uploading the multiple-frame images to the cloud server; and the cloud server performs the following steps: outputting detection information containing a mobile target which moves relative to a static target according to a result of performing comparison between at least two-frame images selected from the multiple-frame images; and outputting an object recognition result of the mobile target to the mobile robot according to a result of performing object recognition on the mobile target in multiple-frame images, such that the mobile robot outputs monitoring information."
  ],
  "description_excerpt": "The present application relates to the field of intelligent mobile robots, in particular to a monitoring method and device for a mobile target, a monitoring system and a mobile robot.\n\nNowadays, the world has entered a critical period of reform and development as well as social transformations. Unprecedented social changes, such as profound changes of economic systems, rapid transformations of social structures, accelerated adjustment of benefit patterns, rapid changes of views and values, and great increase in floating populations, have brought tremendous vitality to social development of the world, meanwhile, various security risks have also emerged, and a dramatic change in public security has also put enormous pressure on social security.\n\nIn order to ensure security of home environments, many people now consider to install an anti-theft system in their home, and the existing manners for anti-theft mainly include protection by means of persons and protection by means of objects (for example, anti-theft doors, iron barriers, etc.). Under some conditions, the following security ways are also used at home: installing an infrared ray anti-theft alarming device, installing electromagnetic password lock or installing monitoring camera at home. The above anti-theft manners are fixed and obvious, these monitoring devices cannot accurately detect moving targets which illegally invade, and those who break in illegally may easily evade monitoring of these anti-theft devices, therefore, reliable, effective and flexible security cannot be provided.",
  "cpc": [
    "G06T 7/254",
    "B25J 9/1697",
    "G06K 9/4604",
    "G06N 3/008",
    "G06N 3/045",
    "G06N 3/0464",
    "G06N 3/08",
    "G06T 2207/10016",
    "G06T 2207/30232",
    "G06T 7/246",
    "G06T 7/285",
    "G06T 7/292",
    "G06T 7/33",
    "G06V 10/757",
    "G06V 20/10",
    "G06V 20/41",
    "G06V 20/46",
    "G06V 20/48",
    "G06V 2201/07",
    "H04L 67/06",
    "H04N 5/144",
    "H04N 7/18"
  ],
  "ipc": [
    "B25J 9/16",
    "G06K 9/46",
    "G06N 3/08",
    "G06T 7/285",
    "G06T 7/292"
  ],
  "assignees": [
    "Ankobot Shanghai Smart Technologies Co Ltd",
    "Ankobot Shenzhen Smart Technologies Co Ltd"
  ],
  "inventors": [
    "Yuwei Cui"
  ],
  "filing_date": "2019-07-26",
  "publication_date": "2020-06-11",
  "priority_date": "2018-12-05",
  "application_number": "US-201916522717-A",
  "family_id": "66191860",
  "cited_by_count": 28
}