System and method for relationship identification

{
  "publication_number": "US10902350B2",
  "country": "US",
  "kind": "B2",
  "title": "System and method for relationship identification",
  "abstract": "The present teaching relates to method, system, and medium for generating training data for generating a relationship identification model. Sentences are received as input. Each of the sentences is aligned with a fact previously stored to create an alignment. Confidence scores for the alignments are computed and then used, together with the alignments to train a relationship identification model.",
  "claims": [
    "1. A method implemented on a machine having at least one processor, storage, and a communication platform for generating training data, comprising: receiving, via the communication platform, sentences as input; for each of the sentences, aligning the sentence with a representation of a fact previously stored to generate an alignment of the sentence with the fact, and computing a confidence score associated with the alignment; and generating, via a plurality of long-short term memory (LSTM)-based encoders, a relationship identification model based on alignments of the sentences and their associated confidence scores.",
    "2. The method of claim 1, wherein the step of the aligning comprises: extracting a plurality of entities from the sentence; identifying the representation of a fact stored previously that has the plurality of entities; and creating the alignment between the sentence and the representation of the fact.",
    "3. The method of claim 1, wherein the step of the computing comprises: determining a co-occurrence frequency associated with each of the alignments with respect to the alignments of the sentences; and determining the confidence score of the alignment based on the co-occurrence frequency.",
    "4. The method of claim 1, further comprising: analyzing each of the sentences received; and obtaining a plurality types of features associated with each of the sentences, wherein the plurality types of features include at least one of dependency features, word features, and POS features associated with each of the sentences and are used in training the training data generation model.",
    "5. The method of claim 4, wherein the step of the generating the relationship identification model comprises: receiving the plurality types of features associated with each of the sentences and the confidence score associated with each of the sentences; combining outputs from the plurality of LSTM encoders to generate a combined output for each of the sentences; and adjusting a plurality of parameters associated with the relationship identification model based on the combined outputs associated with the sentences.",
    "6. The method of claim 1, further comprising: receiving an additional sentence; computing a plurality types of features associated with the additional sentence; and estimating a relationship expressed by the additional sentence based on the plurality of types of features and the relationship identification model.",
    "7. The method of claim 6, wherein the step of estimating comprises: providing each of the plurality types of features to each of LSTM-based multi-encoders associated with the relationship identification model; obtaining a combined output based on outputs from the LSTM based multi-encoders generated based on the plurality of types of features; and determining the relationship of the additional sentence based on the combined output.",
    "8. A non-transitory computer readable medium including computer executable instructions wherein the instructions, when executed by a computer, cause the computer to perform: receiving sentences as input; for each of the sentences, aligning the sentence with a representation of a fact previously stored to generate an alignment of the sentence with the fact, and computing a confidence score associated with the alignment; and generating, via a plurality of long-short term memory (LSTM)-based encoders, a relationship identification model based on alignments of the sentences and their associated confidence scores.",
    "9. The medium of claim 8, wherein the step of the aligning comprises: extracting a plurality of entities from the sentence; identifying the representation of a fact stored previously that has the plurality of entities; and creating the alignment between the sentence and the representation of the fact.",
    "10. The medium of claim 8, wherein the step of the computing comprises: determining a co-occurrence frequency associated with each of the alignments with respect to the alignments of the sentences; and determining the confidence score of the alignment based on the co-occurrence frequency.",
    "11. The medium of claim 8, wherein the information, when read by the machine, further causes the machine to perform the following: analyzing each of the sentences received; and obtaining a plurality types of features associated with each of the sentences, wherein the plurality types of features include at least one of dependency features, word features, and POS features associated with each of the sentences and are used in training the training data generation model.",
    "12. The medium of claim 11, wherein the step of the generating the relationship identification model comprises: receiving the plurality types of features associated with each of the sentences and the confidence score associated with each of the sentences; combining outputs from the plurality of LSTM encoders to generate a combined output for each of the sentences; and adjusting a plurality of parameters associated with the relationship identification model based on the combined outputs associated with the sentences.",
    "13. The medium of claim 8, wherein the information, when read by the machine, further causes the machine to perform the following: receiving an additional sentence; computing a plurality types of features associated with the additional sentence; and estimating a relationship expressed by the additional sentence based on the plurality of types of features and the relationship identification model.",
    "14. The medium of claim 13, wherein the step of estimating comprises: providing each of the plurality types of features to each of LSTM-based multi-encoders associated with the relationship identification model; obtaining a combined output based on outputs from the LSTM-based multi-encoders generated based on the plurality of types of features; and determining the relationship of the additional sentence based on the combined output.",
    "15. A system for generating training data, comprising: an alignment unit configured for receiving sentences as input, and for each of the sentences, aligning the sentence with a representation of a fact previously stored to generate an alignment of the sentence with the fact; an alignment confidence determiner configured for computing a confidence score associated with each alignment; and a long-short term memory (LSTM)-based multi-encoder training unit configured for generating a relationship identification model based on alignments of the sentences and their associated confidence scores.",
    "16. The system of claim 15, wherein the aligning unit is further configured for: extracting a plurality of entities from the sentence; identifying the representation of a fact stored previously that has the plurality of entities; and creating the alignment between the sentence and the representation of the fact.",
    "17. The system of claim 15, wherein the alignment confidence determiner comprises: a co-occurrence frequency determiner configured for determining a co-occurrence frequency associated with each of the alignments with respect to the alignments of the sentences; and an alignment confidence score determiner configured for determining the confidence score of the alignment based on the co-occurrence frequency.",
    "18. The system of claim 15, further comprising a feature extractor configured for: analyzing each of the sentences received; and obtaining a plurality types of features associated with each of the sentences, wherein the plurality types of features include at least one of dependency features, word features, and POS features associated with each of the sentences and are used in training the training data generation model.",
    "19. The system of claim 18, wherein the LSTM-based multi-encoder training unit comprises: a plurality of LSTM encoders configured for receiving the plurality types of features associated with each of the sentences and the confidence score associated with each of the sentences; a state combiner configured for combining outputs from the plurality of LSTM encoders to generate a combined output for each of the sentences; and a feedback learning based model parameter modifier configured for adjusting a plurality of parameters associated with the relationship identification model based on the combined outputs associated with the sentences.",
    "20. The system of claim 15, wherein the information, when read by the machine, further causes the machine to perform the following: receiving an additional sentence; computing a plurality types of features associated with the additional sentence; and estimating a relationship expressed by the additional sentence based on the plurality of types of features and the relationship identification model."
  ],
  "description_excerpt": "The present teaching generally relates to data processing. More specifically, the present teaching relates to identifying relationships from text data.\n\nIn the age of the Internet, amount of data available becomes explosive. Great effort has been made to analyze the vast amount of data to make some sense out of it in order to improve the efficiency associated with data access. For example, some attempts have been made to identify relationships expressed or implied by textual information. Such identified relationships may help users to quickly access relevant information they are interested in. For instance, there are many sentences in online content indicating who was born in London in 1945. In this case, “was born in 1945” is a relationship between a person who was born in London and city London. With a correct identification of all sentences that has this relationship, it is efficient to get a list of people who were born in 1945 in London or further compute the statistics on how many people were born in 1945 in London.\n\nTo automatically detect various relationships from text data, conventionally a model is trained based on data that have been labeled with such relationships. This is illustrated in FIG. 1A (PRIOR ART), in which a training data collection mechanism 110 collects textual data and sends to a ground truth labeling unit 120 for labeling the data with different relationships. Such labeled training data are then stored in a labeled training data storage 130, which are then used by a training mechanism 140 as training data to generate a model (not shown) for recognizing relationships from data.",
  "cpc": [
    "G06N 20/00",
    "G06F 40/20",
    "G06F 40/253",
    "G06F 40/295",
    "G06F 40/30",
    "G06N 3/044",
    "G06N 3/0442",
    "G06N 3/08",
    "G06N 3/0895",
    "G06N 3/09",
    "G06N 5/025",
    "G06Q 30/0241"
  ],
  "ipc": [
    "G06N 99/00",
    "G06F 40/253",
    "G06F 40/295",
    "G06F 40/30",
    "G06N 20/00"
  ],
  "assignees": [
    "Verizon Media Inc"
  ],
  "inventors": [
    "Siddhartha BANERJEE",
    "Kostas Tsioutsiouliklis"
  ],
  "filing_date": "2018-07-20",
  "publication_date": "2021-01-26",
  "grant_date": "2021-01-26",
  "priority_date": "2018-07-20",
  "application_number": "US-201816041073-A",
  "family_id": "69161939",
  "cited_by_count": 1,
  "citations": [
    "US20080221878A1",
    "US20110270604A1",
    "US20170358295A1",
    "US20180121787A1",
    "US20180121799A1",
    "US20180121788A1",
    "US10049103B2",
    "US20180329883A1",
    "US20180336183A1"
  ]
}