Imitation learning system

{
  "publication_number": "US11893468B2",
  "country": "US",
  "kind": "B2",
  "title": "Imitation learning system",
  "abstract": "Apparatuses, systems, and techniques to identify a goal of a demonstration. In at least one embodiment, video data of a demonstration is analyzed to identify a goal. Object trajectories identified in the video data are analyzed with respect to a task predicate satisfied by a respective object trajectory, and with respect to motion predicate. Analysis of the trajectory with respect to the motion predicate is used to assess intentionality of a trajectory with respect to the goal.",
  "claims": [
    "1. A computer-implemented method, comprising: segmenting video data into at least a first segment and a second segment the first segment comprising video data representative of a first trajectory of a first object manipulated in a demonstration performed in a first set of circumstances, the second segment comprising video data representative of a second trajectory of a second object manipulated in the demonstration; identifying a motion predicate satisfied by the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that movement of the first object on the first trajectory enabled movement of the second object on the second trajectory; identifying a task predicate satisfied by the second trajectory, based at least in part on the second trajectory satisfying a logical condition defined in a domain definition; identifying a goal of the demonstration based at least in part on the task predicate; and causing one or more robotic manipulation devices to move from a first pose to a second pose based, at least in part, on performing the goal in a different second set of circumstances.",
    "2. The computer-implemented method of claim 1, further comprising: using the task predicate to identify the goal based, at least in part, on determining that the second trajectory does not enable another trajectory of an object manipulated in the demonstration.",
    "3. The computer-implemented method of claim 1, wherein causing the one or more robotic manipulation devices to move from the first pose to the second pose comprises causing the one or more robotic manipulation devices to perform one or more manipulations to satisfy one or more predicates of the goal.",
    "4. The computer-implemented method of claim 1, wherein the first set of circumstances and the different second set of circumstances comprise first and second environments, respectively, and the second environment is other than a first environment.",
    "5. The computer-implemented method of claim 3, wherein at least one manipulation of an object in the demonstration is to be excluded from the one or more manipulations performed by the one or more robotic manipulation devices.",
    "6. The computer-implemented method of claim 1, wherein the goal is identified from a set of task predicates that excludes an additional task predicate satisfied by the first trajectory.",
    "7. The computer-implemented method of claim 6, wherein the additional task predicate is excluded based, at least in part, on the determination that movement of the first object on the first trajectory enabled movement of the second object on the second trajectory.",
    "8. The computer-implemented method of claim 1, wherein the domain definition comprises a list of objects and a list of task predicates satisfiable by manipulation of objects in the list of objects.",
    "9. A system, comprising: at least one processor; and a memory comprising instructions that, as a result of execution by the at least one processor, cause the system to at least: segment video data into a first segment and a second segment, the first segment comprising video data representative of a first trajectory of a first object manipulated in a demonstration performed in a first set of circumstances, the second segment comprising video data representative of a second trajectory of a second object manipulated in the demonstration; identify a motion predicate satisfied by manipulation of the first object in accordance with the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that manipulation of the first object in accordance with the first trajectory enabled motion of the second object in accordance with the second trajectory; identify a task predicate satisfied by manipulation of the second object in accordance with the second trajectory; identify a goal of the demonstration based at least in part on the task predicate; and cause one or more robotic manipulation devices to move from a first pose to a second pose based, at least in part, on performing the goal in a different second set of circumstances.",
    "10. The system of claim 9, wherein the task predicate is identified based at least in part on determining that the second trajectory does not enable another trajectory.",
    "11. The system of claim 9, wherein the memory comprises further instructions that, as a result of execution by the at least one processor, cause the system to at least cause the one or more robotic manipulation devices to determine one or more predicates of the goal, wherein causing the one or more robotic manipulation devices to move from the first pose to the second pose comprises causing the one or more robotic manipulation devices to perform one or more manipulations to satisfy the one or more predicates of the goal.",
    "12. The system of claim 9, wherein the first set of circumstances comprises a first environment that differs from a second environment of the different second set of circumstances.",
    "13. The system of claim 9, wherein the goal of the demonstration is identified based, at least in part, on an estimate of intentionality of the task predicate.",
    "14. The system of claim 13, wherein the estimate of intentionality is based, at least in part, on a relationship between the first trajectory and the second trajectory.",
    "15. The system of claim 9, wherein a first task predicate satisfied by the first trajectory is assessed low intentionality based, at least in part, on the second trajectory being enabled by the first trajectory.",
    "16. A non-transitory computer-readable medium comprising instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to: divide video data into a first segment and a second segment, the first segment comprising video data representative of a first trajectory of a first object manipulated in a demonstration performed in a first set of circumstances, the second segment comprising video data representative of a second trajectory of a second object; identify a motion predicate satisfied by the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that manipulation of the first object enabled the second trajectory; identify a task predicate satisfied by the second trajectory; identify a goal of the demonstration based at least in part on the task predicate; and cause one or more robotic manipulation devices to move from a first pose to a second pose based, at least in part, on performing the goal in a different second set of circumstances.",
    "17. The non-transitory computer-readable medium of claim 16, wherein the task predicate is used to identify the goal based, at least in part, on determining that the second trajectory does not enable another trajectory.",
    "18. The non-transitory computer-readable medium of claim 16, comprising further instructions that, as a result of execution by the one or more processors of the computer system, cause the computer system to cause the one or more robotic manipulation devices to move from the first pose to the second pose by instructing the one or more robotic manipulation devices to manipulate objects to achieve task predicates of the goal.",
    "19. The non-transitory computer-readable medium of claim 16, wherein the goal of the demonstration is identified based, at least in part, on an estimated intentionality of the task predicate.",
    "20. The non-transitory computer-readable medium of claim 19, wherein the estimated intentionality is based, at least in part, on analysis of the first trajectory with respect to the second trajectory.",
    "21. A robotic device, comprising: one or more robotic manipulation devices; at least one processor; and a memory comprising instructions that, as a result of execution by the at least one processor, cause the robotic device to at least: move the one or more robotic manipulation devices from a first pose to a second pose based, at least in part, on performing a goal in a subsequent set of circumstances based at least in part on a demonstration performed in a previous set of circumstances, wherein the goal is determined by: segmenting video data of the demonstration into a first segment and a second segment, the first segment comprising video data representative of a first trajectory of a first object, the second segment comprising video data representative of a second trajectory of a second object; identifying a motion predicate satisfied by the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that the first trajectory enabled the second trajectory; identifying a task predicate satisfied by manipulation of the second object in accordance with the second trajectory; and identifying the goal based at least in part on the task predicate.",
    "22. The robotic device of claim 21, wherein the task predicate is identified based, at least in part, on determining that the second trajectory causes conformance with one or more conditions of the task predicate and does not enable another trajectory.",
    "23. The robotic device of claim 21, wherein the memory comprises instructions that, as a result of execution by the at least one processor, cause the robotic device to at least: determine a first set of predicates based at least in part on the goal; and identify manipulations of the one or more robotic manipulation devices that satisfy one or more conditions of the first set of predicates, the robotic device to use the identified manipulations to move the one or more robotic manipulation devices from the first pose to the second pose.",
    "24. The robotic device of claim 23, wherein a second set of predicates comprises the motion predicate and the task predicate, one or more of the first set of predicates are different than one or more of the second set of predicates.",
    "25. The robotic device of claim 21, wherein the task predicate used to determine the goal is associated with an estimate of relevance that is above a threshold amount.",
    "26. The robotic device of claim 25, wherein the estimate of relevance is based, at least in part, on a relationship between two trajectories.",
    "27. The computer-implemented method of claim 3, further comprising: determining the one or more predicates of the goal.",
    "28. The computer-implemented method of claim 27, further comprising: determining at least one third trajectory that satisfies the one or more predicates of the goal, wherein causing the one or more robotic manipulation devices to move from the first pose to the second pose comprises causing the one or more robotic manipulation devices to move from the first pose to the second pose in accordance with the at least one third trajectory."
  ],
  "description_excerpt": "This application claims the benefit of U.S. Provisional Application No. 62/900,226, entitled “MOTION REASONING FOR GOAL-BASED IMITATION LEARNING,” filed Sep. 13, 2019, the entire contents of which is incorporated herein by reference.\n\nAt least one embodiment pertains to training robots to achieve a goal based on observing a demonstration. For example, at least one embodiment pertains to a robotic device that observes a demonstration and, using various novel techniques described herein, identifies the goal of the demonstration.\n\nTraining robots to provide assistance presents a variety of challenges. For example, a user might wish to teach a robot to perform a task based on a demonstration of that task by a user. However, techniques for training a robot based on observation may require significant amounts of computing resources, or may be prone to error.\n\nFIG. 1 illustrates an example of demonstration learning, according to at least one embodiment;\n\nFIG. 2 illustrates an example of video data segmentation, according to at least one embodiment;\n\nFIG. 3 illustrates an example of predicate identification, according to at least one embodiment;\n\nFIG. 4 illustrates an example of trajectory analysis, according to at least one embodiment;\n\nFIG. 5 illustrates example aspects of demonstration learning, according to at least one embodiment;\n\nFIG. 6 illustrates an example of goal identification in demonstration learning, according to at least one embodiment;\n\nFIG. 7 illustrates an example of goal reproduction by a robotic manipulation device, according to at least one embodiment;",
  "cpc": [
    "G06N 3/008",
    "G06N 20/00",
    "G06N 3/045",
    "G06N 3/0464",
    "G06N 3/08",
    "G06N 3/09",
    "G06N 3/0985",
    "G06N 5/04"
  ],
  "ipc": [
    "G06N 3/00",
    "G06N 20/00",
    "G06N 3/008"
  ],
  "assignees": [
    "Nvidia Corp"
  ],
  "inventors": [
    "Yu-Wei Chao",
    "De-An Huang",
    "Christopher Jason Paxton",
    "Animesh GARG",
    "Dieter Fox"
  ],
  "filing_date": "2020-07-16",
  "publication_date": "2024-02-06",
  "grant_date": "2024-02-06",
  "priority_date": "2019-09-13",
  "application_number": "US-202016931211-A",
  "family_id": "74868569",
  "cited_by_count": 1,
  "citations": [
    "US20190051061A1",
    "WO2019171060A1",
    "US20210370980A1"
  ]
}