Fall detection and assistance

{
  "publication_number": "US11288509B2",
  "country": "US",
  "kind": "B2",
  "title": "Fall detection and assistance",
  "abstract": "A method for controlling a robotic device based on observed object locations is presented. The method includes observing objects in an environment. The method also includes generating a probability distribution for locations of the observed objects. The method further includes controlling the robotic device to perform an action when an object is at a location in the environment with a location probability that is less than a threshold.",
  "claims": [
    "1. A method for controlling a robotic device based on observed object locations, comprising: observing a set of objects in an environment over a period of time prior to a current time; generating a probability distribution for locations of each object of the set of objects in the environment based on observing the set of objects over the period of time; identifying, at the current time, an object of the set of objects at a location in the environment; determining a probability of the object being at the location in the environment based on identifying the object at the location, the probability being based on the probability distribution associated with the object for the location; and controlling the robotic device to perform an action based on the probability being less than a threshold.",
    "2. The method of claim 1, further comprising estimating a continuous distribution using the observations of the objects over the period of time.",
    "3. The method of claim 2, in which the probability distribution is based on the continuous distribution.",
    "4. The method of claim 1, further comprising: generating a cost map from the probability distribution; overlaying the cost map on the environment; and controlling the robotic device based on the cost map.",
    "5. The method of claim 4, in which the action comprising at least one of providing assistance to the object, contacting emergency services, or a combination thereof.",
    "6. The method of claim 1, in which the object is a human.",
    "7. The method of claim 1, in which the location is an unlikely location for the object based on the probability being less than the threshold.",
    "8. An apparatus for controlling a robotic device based on observed object locations, the apparatus comprising: a memory; and at least one processor coupled to the memory, the at least one processor configured: to observe a set of objects in an environment over a period of time prior to a current time; to generate a probability distribution for locations of each object of the set of objects in the environment based on observing the set of objects over the period of time; to identify, at the current time, an object of the set of objects at a location in the environment; to determine a probability of the object being at the location in the environment based on identifying the object at the location, the probability being based on the probability distribution associated with the object for the location; and to control the robotic device to perform an action based on the probability being less than a threshold.",
    "9. The apparatus of claim 8, in which the at least one processor is further configured to estimate a continuous distribution using the observations of the objects over the period of time.",
    "10. The apparatus of claim 9, in which the probability distribution is based on the continuous distribution.",
    "11. The apparatus of claim 8, in which the at least one processor is further configured: to generate a cost map from the probability distribution; to overlay the cost map on the environment; and to control the robotic device based on the cost map.",
    "12. The apparatus of claim 11, in which the action comprising at least one of providing assistance to the object, contacting emergency services, or a combination thereof.",
    "13. The apparatus of claim 8, in which the object is a human.",
    "14. The apparatus of claim 8, in which the location is an unlikely location for the object based on the probability being less than the threshold.",
    "15. A non-transitory computer-readable medium having program code recorded thereon for controlling a robotic device based on observed object locations, the program code executed by a processor and comprising: program code to observe a set of objects in an environment over a period of time prior to a current time; program code to generate a probability distribution for locations of each object of the set of objects in the environment based on observing the set of objects over the period of time; program code to identify, at the current time, an object of the set of objects at a location in the environment; program code to determine a probability of the object being at the location in the environment based on identifying the object at the location, the probability being based on the probability distribution associated with the object for the location; and program code to control the robotic device to perform an action based on the probability being less than a threshold.",
    "16. The non-transitory computer-readable medium of claim 15, in which the program code further comprises program code to estimate a continuous distribution using the observations of the objects over the period of time.",
    "17. The non-transitory computer-readable medium of claim 16, in which the probability distribution is based on the continuous distribution.",
    "18. The non-transitory computer-readable medium of claim 15, in which the program code further comprises: program code to generate a cost map from the probability distribution; program code to overlay the cost map on the environment; and program code to control the robotic device based on the cost map.",
    "19. The non-transitory computer-readable medium of claim 18, in which the action comprising at least one of providing assistance to the object, contacting emergency services, or a combination thereof.",
    "20. The non-transitory computer-readable medium of claim 15, in which the location is an unlikely location for the object based on the probability being less than the threshold."
  ],
  "description_excerpt": "Certain aspects of the present disclosure generally relate to object detection and, more particularly, to a system and method for detecting an out of place object based on a spatial and temporal knowledge of objects in an environment.\n\nA robotic device may use one or more sensors, such as a camera, to identify objects in an environment. A location of each identified object may be estimated. Additionally, a label may be assigned to each localized object. In conventional object localization systems, each estimated object location may be stored in a database along with an estimation time stamp.\n\nConventional object detection systems may be limited to detecting objects in an environment. It is desirable to improve an object detection system's ability to detect an out of place object. It is also desirable for robotic systems to perform an action upon detecting an out of place object.\n\nIn one aspect of the present disclosure, a method for controlling a robotic device based on observed object locations is disclosed. The method includes observing objects in an environment. The method also includes generating a probability distribution for locations of the observed objects. The method further includes controlling the robotic device to perform an action when an object is at a location in the environment with a location probability that is less than a threshold.\n\nIn another aspect of the present disclosure, a non-transitory computer-readable medium with non-transitory program code recorded thereon is disclosed. The program code is for controlling a robotic device based on observed object locations.",
  "cpc": [
    "B25J 9/16",
    "B25J 11/00",
    "B25J 11/0005",
    "B25J 9/0003",
    "G05B 2219/40264",
    "G05D 1/0217",
    "G05D 1/0246",
    "G05D 1/0274",
    "G06F 18/23",
    "G06K 9/00214",
    "G06K 9/00664",
    "G06V 10/762",
    "G06V 10/764",
    "G06V 10/82",
    "G06V 20/10",
    "G06V 20/653",
    "G08B 21/043",
    "G08B 21/0438",
    "G08B 21/0469",
    "G08B 21/0476"
  ],
  "ipc": [
    "B25J 9/00",
    "G06V 10/762",
    "G06V 10/764"
  ],
  "assignees": [
    "Toyota Research Institute Inc"
  ],
  "inventors": [
    "Astrid Jackson",
    "Brandon Northcutt"
  ],
  "filing_date": "2019-11-12",
  "publication_date": "2022-03-29",
  "grant_date": "2022-03-29",
  "priority_date": "2019-11-12",
  "application_number": "US-201916681366-A",
  "family_id": "75846636",
  "cited_by_count": 1,
  "citations": [
    "US20070192910A1",
    "US20070269077A1",
    "US20110112839A1",
    "US20120173018A1",
    "US20140050455A1",
    "US20140214255A1",
    "US20140288705A1",
    "US20170157769A1",
    "US20180012370A1",
    "US20180107226A1",
    "US20190197861A1",
    "US20200333142A1",
    "US20190286921A1",
    "US20190340775A1",
    "US20200017317A1",
    "US20200053325A1",
    "US20200086482A1",
    "US20200174481A1",
    "US20200207375A1",
    "US20200207356A1",
    "US10937178B1",
    "US20200410259A1",
    "US20200410063A1",
    "US20210096571A1"
  ]
}