Interpreting human-robot instructions

{
  "publication_number": "US10606898B2",
  "country": "US",
  "kind": "B2",
  "title": "Interpreting human-robot instructions",
  "abstract": "A system includes a robot having a module that includes a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function including using a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework.",
  "claims": [
    "1. A system comprising: a robot comprising a module to interpret natural language commands, the module comprising a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function comprising: a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework; an Abstract Markov Decision Process to represent a decision-making problem involving a hierarchy of the robot's states and actions, the Abstract Markov Decision Process comprising: a set of states that define an environment specified in an object-oriented fashion with object classes and attributes; a set of actions an agent can execute to transition between states or to invoke a lower-level AMDP subtask; a transition probability distribution over all possible next states given a current state and executed action; a numerical reward earned for a particular transition; a discount factor or effective time horizon under consideration; and a state projection function that maps lower-level states to higher-level (AMDP) states.",
    "2. A system comprising: a robot comprising a module comprising a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function comprising: a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework; an Abstract Markov Decision Process to represent a decision-making problem involving a hierarchy of the robot's states and actions, the Abstract Markov Decision Process comprising: a set of states that define an environment specified in an object-oriented fashion with object classes and attributes; a set of actions an agent can execute to transition between states or to invoke a lower-level AMDP subtask; a transition probability distribution over all possible next states given a current state and executed action; a numerical reward earned for a particular transition; a discount factor or effective time horizon under consideration; and a state projection function that maps lower-level states to higher-level (AMDP) states.",
    "3. A method comprising: providing a mobile-manipulator robot; and providing a module that interprets and grounds natural language commands to a mobile-manipulator robot at multiple levels of abstraction, the module comprising a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of a hierarchical planning framework, the functions comprising an Abstract Markov Decision Process to represent a decision-making problem involving a hierarchy of the robot's states and actions, the Abstract Markov Decision Process comprising: a set of states that define an environment specified in an object-oriented fashion with object classes and attributes; a set of actions an agent can execute to transition between states or to invoke a lower-level AMDP subtask; a transition probability distribution over all possible next states given a current state and executed action; a numerical reward earned for a particular transition; a discount factor or effective time horizon under consideration; and a state projection function that maps lower-level states to higher-level (AMDP) states."
  ],
  "description_excerpt": "The invention generally relates to robots, and more specifically to interpreting human-robot instructions.\n\nIn general, humans can ground natural language commands to tasks at both abstract and fine-grained levels of specificity. For instance, a human forklift operator can be instructed to perform a high-level action, like “grab a pallet” or a low-level action like “tilt back a little bit.” While robots are also capable of grounding language commands to tasks, previous methods implicitly assume that all commands and tasks reside at a single, fixed level of abstraction. Additionally, those approaches that do not use abstraction experience inefficient planning and execution times due to the large, intractable state-action spaces, which closely resemble real world complexity.\n\nExisting approaches generally map between natural language commands and a formal representation at some fixed level of abstraction. While effective at directing robots to complete predefined tasks, mapping to fixed sequences of robot actions is unreliable when faced with a changing or stochastic environment.\n\nThe following presents a simplified summary of the innovation in order to provide a basic understanding of some aspects of the invention. This summary is not an extensive overview of the invention. It is intended to neither identify key or critical elements of the invention nor delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.",
  "cpc": [
    "G06F 16/90332",
    "G06F 15/76",
    "G06F 17/2809",
    "G06F 40/42",
    "G06N 20/00",
    "G06N 3/008",
    "G06N 3/044",
    "G06N 3/0442",
    "G06N 3/0445",
    "G06N 3/045",
    "G06N 3/0454",
    "G06N 3/0499",
    "G06N 3/08",
    "G06N 3/09",
    "G06N 3/092",
    "G06N 7/005",
    "G06N 7/01",
    "G10L 2015/223",
    "G10L 2015/225"
  ],
  "ipc": [
    "G06F 17/28",
    "G06F 17/30",
    "G06F 15/76",
    "G06F 16/9032",
    "G06N 3/00",
    "G06N 3/04",
    "G06N 3/08",
    "G06N 7/00",
    "G10L 15/10",
    "G10L 15/20"
  ],
  "assignees": [
    "Brown University"
  ],
  "inventors": [
    "Stefanie Tellex",
    "Dilip Arumugam",
    "Siddharth Karamcheti",
    "Nakul Gopalan",
    "Lawson L. S. Wong"
  ],
  "filing_date": "2018-04-19",
  "publication_date": "2020-03-31",
  "grant_date": "2020-03-31",
  "priority_date": "2017-04-19",
  "application_number": "US-201815957651-A",
  "family_id": "63854624",
  "cited_by_count": 11,
  "citations": [
    "US9015093B1",
    "US20160161946A1"
  ]
}