Multi-agent trajectory prediction

{
  "publication_number": "US11816901B2",
  "country": "US",
  "kind": "B2",
  "title": "Multi-agent trajectory prediction",
  "abstract": "Methods and systems for training a trajectory prediction model and performing a vehicle maneuver include encoding a set of training data to generate encoded training vectors, where the training data includes trajectory information for agents over time. Trajectory scenarios are simulated based on the encoded training vectors, with each simulated trajectory scenario representing one or more agents with respective agent trajectories, to generate simulated training data. A predictive neural network model is trained using the simulated training data to generate predicted trajectory scenarios based on a detected scene.",
  "claims": [
    "1. A method for training a trajectory prediction model, comprising: encoding a set of training data to generate encoded training vectors, where the training data includes trajectory information for road agents over time; simulating a plurality of simulated trajectory scenarios based on the encoded training vectors, each simulated trajectory scenario representing one or more road agents with respective road agent trajectories, including simulating accelerating and lane change decisions for the one or more road agents, to generate simulated training data, wherein lane change simulations include evaluating the following inequalities: a ~ c - a c + p ⁢ { (a ~ n - a n) + (a ~ o - a o) } > Δ ⁢ a th - Δ ⁢ a bias (a ~ n - a n) > - b safe n, (a ~ c - a c) > - b safe c where a is a current acceleration, ã represents a new acceleration after lane change, p is a politeness factor, Δa th is a lane changing threshold, b safe is a deceleration, and Δa bias is a bias for a particular lane, and where subscripts/superscripts c, n, and o denote a current agent, new agents, and old agents, respectively; and training a predictive neural network model using the simulated training data to generate predicted trajectory scenarios based on a detected scene.",
    "2. The method of claim 1, wherein the predictive neural network model includes an autoencoder section and a convolutional long-short term memory section with state pooling.",
    "3. The method of claim 2, wherein state pooling pools previous state information from a final layer of the convolutional long-short term memory section for all of the one or more road agents and initializes a next state with the previous state information.",
    "4. The method of claim 1, wherein at least one of the plurality of simulated trajectory scenarios includes multiple road agents.",
    "5. The method of claim 4, wherein the predictive model generates the predicted trajectory scenarios for all of the multiple road agents in each simulated trajectory scenario at once.",
    "6. The method of claim 1, wherein acceleration simulation may include generating an acceleration value as: (1 - (v v 0) δ - (s * (ν, Δ ⁢ v) s) 2) where s is a lead road agent of the one or more road agents, v is a velocity of a decision-making road agent, Δv is a velocity difference between the lead road agent and the decision-making road agent, \\delta is an exponent that influences how acceleration decreases with velocity, s is an actual distance between the lead road agent and the decision-making road agent, and s* is a minimum gap between the lead road agent and the decision-making road agent, and v, is a reference velocity."
  ],
  "cpc": [
    "G06V 20/56",
    "B60W 2420/403",
    "B60W 2420/42",
    "B60W 2554/4045",
    "B60W 30/0953",
    "B60W 30/0956",
    "B60W 60/0027",
    "G06F 18/214",
    "G06F 18/24137",
    "G06N 3/044",
    "G06N 3/0442",
    "G06N 3/045",
    "G06N 3/0455",
    "G06N 3/0464",
    "G06N 3/0475",
    "G06N 3/08",
    "G06N 3/084",
    "G06N 3/09",
    "G06V 10/82"
  ],
  "assignees": [
    "NEC LAB AMERICA INC",
    "NEC CORP"
  ],
  "filing_date": "2021-02-26",
  "publication_date": "2023-11-14",
  "grant_date": "2023-11-14",
  "priority_date": "2020-03-04",
  "application_number": "US-202117187157-A",
  "family_id": "77556414",
  "citations": [
    "US2004088392A1",
    "US2004176936A1",
    "US2014278052A1",
    "US2018124423A1",
    "US2018141544A1",
    "US2018194354A1",
    "US2019095731A1",
    "US2019111933A1",
    "US2019129831A1",
    "US2019163182A1",
    "US2019212749A1",
    "US2019287404A1",
    "US2019384304A1",
    "US5474453A"
  ]
}