System and methods for a virtual assistant to manage and use context in a natural language dialog

{
  "publication_number": "US10418032B1",
  "country": "US",
  "kind": "B1",
  "title": "System and methods for a virtual assistant to manage and use context in a natural language dialog",
  "abstract": "A dialog with a conversational virtual assistant includes a sequence of user queries and systems responses. Queries are received and interpreted by a natural language understanding system. Dialog context information gathered from user queries and system responses is stored in a layered context data structure. Incomplete queries, which do not have sufficient information to result in an actionable interpretation, become actionable with use of context data. The system recognizes the need to access context data, and retrieves from context layers information required to transform the query into an executable one. The system may then act on the query and provide an appropriate response to the user. Context data buffers forget information, perhaps selectively, with the passage of time, and after a sufficient number and type of intervening queries.",
  "claims": [
    "1. A method of disambiguating interpretations, the method comprising: receiving from a client a natural language query and an associated dialog history, the natural language query having an incomplete interpretation, the incomplete interpretation having at least one slot name whose value is ambiguous, the associated dialog history including one or more entries for history items, each history item including a previous interpretation and being associated with a named slot from the set comprising WHO, WHAT, WHEN, WHERE and HOW MANY, and each history item entry being associated with a slot value, saliency weight and sequence number; selecting, from the associated dialog history, a history item including a previous interpretation, the history item being selected, from the associated dialog history, based on the slot name of the history item, the number sequence of the history item and the saliency weight of the history item, such that the selected history item matches the natural language query having the incomplete interpretation for purposes of completing the at least one slot name whose value is ambiguous; merging the previous interpretation of the selected history item with the incomplete interpretation to form a merged interpretation; creating an updated dialog history by adding, to the associated dialog history, a new dialog history layer comprising the merged interpretation and at least one additional history item having a slot name, a slot value, a saliency weight, and a new sequence number; and sending the updated dialog history to the client.",
    "2. The method of claim 1 wherein at least one of the history items of the associated dialog history stores a variable-length array of values.",
    "3. The method of claim 1 wherein the set is open-ended, the method further comprises creating a new history item within the new dialog history layer, and the new history item has a new slot name.",
    "4. The method of claim 1 wherein: the natural language query received from the client is a stream of data and the associated dialog history is received at a beginning of the natural language query; and the updated dialog history sent to the client is part of structured output.",
    "5. The method of claim 1, wherein the matching history item is selected by (i) comparing the slot name of each history item with the at least one slot name whose value is ambiguous, (ii) considering the saliency weight for each history item, where the saliency weight indicates an importance of each respective history item in the associated dialog history and (iii) considering the sequence number of each history item, where the sequence number indicates a time sequence in which each respective history item has been placed in the associated dialog history.",
    "6. The method of claim 5, wherein the matching history item is selected by (i) identifying one or more history items, from the associated dialog history, having a slot name that matches the at least one slot name whose value is ambiguous and (ii) selecting, as the selected matching history item, one of the identified one or more history items based on the saliency weights and the sequence numbers of the identified one or more history items.",
    "7. A non-transitory computer-readable recording medium having computer program instructions recorded thereon, the computer program instructions, when executed on a computer processor, causing the computer processor to perform a method of disambiguating interpretations, the method comprising: receiving from a client a natural language query and an associated dialog history, the natural language query having an incomplete interpretation, the incomplete interpretation having at least one slot name whose value is ambiguous, the associated dialog history including one or more entries for history items, each history item including a previous interpretation and being associated with a named slot from the set comprising WHO, WHAT, WHEN, WHERE and HOW MANY, and each history item entry being associated with a slot value, saliency weight and sequence number; selecting, from the associated dialog history, a history item including a previous interpretation, the history item being selected, from the associated dialog history, based on the slot name of the history item, the number sequence of the history item and the saliency weight of the history item, such that the selected history item matches the natural language query having the incomplete interpretation for purposes of completing the at least one slot name whose value is ambiguous; merging the previous interpretation of the selected history item with the incomplete interpretation to form a merged interpretation; creating an updated dialog history by adding, to the associated dialog history, a new dialog history layer comprising the merged interpretation and at least one additional history item having a slot name, a slot value, a saliency weight, and a new sequence number; and sending the updated dialog history to the client.",
    "8. The non-transitory computer-readable recording medium of claim 7, wherein the matching history item is selected by (i) comparing the slot name of each history item with the at least one slot name whose value is ambiguous, (ii) considering the saliency weight for each history item, where the saliency weight indicates an importance of each respective history item in the associated dialog history and (iii) considering the sequence number of each history item, where the sequence number indicates a time sequence in which each respective history item has been placed in the associated dialog history.",
    "9. The non-transitory computer-readable recording medium of claim 8, wherein the matching history item is selected by (i) identifying one or more history items, from the associated dialog history, having a slot name that matches the at least one slot name whose value is ambiguous and (ii) selecting, as the selected matching history item, one of the identified one or more history items based on the saliency weights and the sequence numbers of the identified one or more history items.",
    "10. A system including one or more processors coupled to memory, the memory loaded with computer program instructions for disambiguating interpretations, the computer program instructions, when executed on the processors, implement actions comprising receiving from a client a natural language query and an associated dialog history, the natural language query having an incomplete interpretation, the incomplete interpretation having at least one slot name whose value is ambiguous, the associated dialog history including one or more entries for history items, each history item including a previous interpretation and being associated with a named slot from the set comprising WHO, WHAT, WHEN, WHERE and HOW MANY, and each history item entry being associated with a slot value, saliency weight and sequence number; selecting, from the associated dialog history, a history item including a previous interpretation, the history item being selected, from the associated dialog history, based on the slot name of the history item, the number sequence of the history item and the saliency weight of the history item, such that the selected history item matches the natural language query having the incomplete interpretation for purposes of completing the at least one slot name whose value is ambiguous; merging the previous interpretation of the selected history item with the incomplete interpretation to form a merged interpretation; creating an updated dialog history by adding, to the associated dialog history, a new dialog history layer comprising the merged interpretation and at least one additional history item having a slot name, a slot value, a saliency weight, and a new sequence number; and sending the updated dialog history to the client.",
    "11. The system of claim 10, wherein the matching history item is selected by (i) comparing the slot name of each history item with the at least one slot name whose value is ambiguous, (ii) considering the saliency weight for each history item, where the saliency weight indicates an importance of each respective history item in the associated dialog history and (iii) considering the sequence number of each history item, where the sequence number indicates a time sequence in which each respective history item has been placed in the associated dialog history.",
    "12. The system of claim 11, wherein the matching history item is selected by (i) identifying one or more history items, from the associated dialog history, having a slot name that matches the at least one slot name whose value is ambiguous and (ii) selecting, as the selected matching history item, one of the identified one or more history items based on the saliency weights and the sequence numbers of the identified one or more history items."
  ],
  "description_excerpt": "The present invention relates to natural language understanding systems, and in particular, to supporting the conversational capabilities of a virtual assistant.\n\nSpeech recognition and natural language understanding capabilities of mobile devices have grown rapidly in recent years. Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) allow users of electronic devices to interact with computer systems using a subset of natural language, in spoken or written form. Users interact with a virtual assistant and present queries that typically ask for information or request an action. The queries are processed by an automated agent that attempts to recognize the structure and meaning of the user's query, and when successful, to create a response and to present it to the user. The term assistant is anthropomorphic: it refers to a human-like interface that receives user queries and responds in terms that users understand; the term agent refers instead to the computer-based implementation of the functionality that the assistant presents to users. These two terms are closely related, and they are often used interchangeably.\n\nVarious approaches to the understanding of natural language input are known in the art. One of them is called syntax-based semantics. This approach starts with the use of a context-free grammar (CfG) to recognize syntactically well-formed natural language sentences while excluding ill-formed ones. Context-free grammars are well known in the art. A CfG comprises an alphabet, that consists of terminal and non-terminal symbols, and a set of production rules.",
  "cpc": [
    "G10L 15/22",
    "G06F 16/24522",
    "G06F 16/258",
    "G06F 16/3329",
    "G10L 15/1815",
    "G10L 15/19",
    "G10L 15/26"
  ],
  "ipc": [
    "G06F 16/2452",
    "G06F 16/25",
    "G10L 15/19",
    "G10L 15/22"
  ],
  "assignees": [
    "SoundHound Inc"
  ],
  "inventors": [
    "Keyvan Mohajer",
    "Christopher Wilson",
    "Bernard Mont-Reynaud",
    "Regina Collecchia"
  ],
  "filing_date": "2016-05-24",
  "publication_date": "2019-09-17",
  "grant_date": "2019-09-17",
  "priority_date": "2015-04-10",
  "application_number": "US-201615163485-A",
  "family_id": "67909135",
  "cited_by_count": 165,
  "citations": [
    "US5685000A",
    "US6377913B1",
    "US20020038213A1",
    "US20020133355A1",
    "US20020135618A1",
    "US20100267345A1",
    "US20080201135A1",
    "US20090055165A1",
    "US20090150156A1",
    "US20110119047A1",
    "US20120016678A1",
    "US8706503B2",
    "US20120290298A1",
    "US20140040748A1",
    "US8577671B1",
    "US8606568B1",
    "US20140309990A1",
    "US9772994B2",
    "US20150142704A1",
    "US20150340033A1",
    "US20170110127A1",
    "US20160188565A1"
  ]
}