Recurrent neural network-based artificial intelligence system for time series predictions

{
  "publication_number": "US10936947B1",
  "country": "US",
  "kind": "B1",
  "title": "Recurrent neural network-based artificial intelligence system for time series predictions",
  "abstract": "At a network-accessible artificial intelligence service for time series predictions, a recurrent neural network model is trained using a plurality of time series of demand observations to generate demand forecasts for various items. A probabilistic demand forecast is generated for a target item using multiple executions of the trained model. Within the training set used for the model, the count of demand observations of the target item may differ from the count of demand observations of other items. A representation of the probabilistic demand forecast may be provided via a programmatic interface.",
  "claims": [
    "1. A system, comprising: one or more computing devices of a recurrent neural-network based artificial intelligence service for time series predictions; wherein the one or more computing devices are configured to: obtain an indication of a first data set comprising a respective time series of demand observations for a plurality of items; train a recurrent neural network model using a plurality of time series of the first data set, wherein the recurrent neural network is trained to make probabilistic demand forecasts for a plurality of different items, and wherein training of the recurrent neural network model comprises performing a compensation operation to the plurality of times series of the first data set to compensate for differences between respective distributions of the demand observations for different items within the first data set; receive indications of different target items for which respective probabilistic demand forecasts are to be generated, wherein the first data set does not include a time series comprising demand observations of at least one of the target items; obtain respective probabilistic demand forecasts for the different target items using a plurality of executions of the same recurrent neural network model, wherein the respective probabilistic demand forecasts comprise respective aggregations of individual demand forecasts indicated by respective executions of the plurality of executions; and provide, via a programmatic interface, respective representations of the probabilistic demand forecasts.",
    "2. The system as recited in claim 1, wherein to obtain the probabilistic demand forecasts, the one or more computing devices are configured to: predict, using the recurrent neural network model, one or more parameters of a first demand distribution corresponding to a first time step; and utilize a sample from the first demand distribution as input of the model, wherein a second set of nodes of the model is used to predict parameters of a second demand distribution corresponding to a second time step.",
    "3. The system as recited in claim 1, wherein the compensation operation comprises: generating one or more sub-sequences of a time series of the first data set; and utilizing the one or more sub-sequences as respective training examples.",
    "4. The system as recited in claim 1, wherein the compensation operation comprises: assigning a first weight to a first time series of the first data set during training of the recurrent neural network model; and assigning a second weight to a second time series of the first data set during training of the recurrent neural network model.",
    "5. The system as recited in claim 1, wherein the compensation operation comprises applying a scaling function to at least one of: (a) an input of the recurrent neural network model for a particular time series, or (b) an output of the recurrent neural network model for the particular time series.",
    "6. A method, comprising: performing, by one or more computing devices: obtaining an indication of a first data set comprising a respective time series of demand observations for a plurality of items; training a recurrent neural network model to generate demand forecasts for a plurality of different items, wherein training of the recurrent neural network model is based at least in part on (a) analysis of a plurality of time series of the first data set and (b) performing one or more compensation operations to the plurality of times series of the first data set to compensate for statistical differences between respective distributions of the demand observations for different items within the first data set, wherein, within the training data set of the recurrent neural network model, a count of demand observations of at least one other item exceeds a count of demand observations of a target item; generating respective probabilistic demand forecasts for different target items using one or more executions of the same recurrent neural network model; and providing, via a programmatic interface, respective representations of the probabilistic demand forecasts.",
    "7. The method as recited in claim 6, wherein generating the probabilistic demand forecasts comprises: predicting, using the recurrent neural network model, one or more parameters of a first demand distribution corresponding to a first time step; and utilizing a sample from the first demand distribution as input to predict parameters of a second demand distribution corresponding to a second time step.",
    "8. The method as recited in claim 6, wherein a particular compensation operation comprises excluding a sample of the first data set from the training data set.",
    "9. The method as recited in claim 6, wherein a particular compensation operation of the one or more compensation operations comprises: assigning a first weight to a first time series of the first data set during training of the recurrent neural network model; and assigning a second weight to a second time series of the first data set during training of the recurrent neural network model.",
    "10. The method as recited in claim 6, wherein a particular compensation operation of the one or more compensation operations comprises: applying a normalization function to at least one of: (a) an input of the recurrent neural network model, or (b) an output of the recurrent neural network model.",
    "11. The method as recited in claim 6, further comprising performing, by the one or more computing devices: utilizing, as input for at least one execution of the one or more executions of the recurrent neural network model, an indication of a similarity between at least one of the target items and a second item, wherein, within the training data set, a count of demand observations of the second item exceeds the count of demand observations of the at least one target item.",
    "12. The method as recited in claim 11, wherein the indication of a similarity comprises one or more of: (a) a price, (b) a product category, (c) item introduction timing information, or (d) marketing information.",
    "13. The method as recited in claim 6, further comprising performing, by the one or more computing devices: utilizing, as input for at least one execution of the one or more executions of the recurrent neural network model, a time series of demand of at least one of the target items.",
    "14. The method as recited in claim 6, further comprising performing, by the one or more computing devices: selecting a first sub-sequence of elements of a first time series as a first training example for the recurrent neural network model; and selecting a second sub-sequence of elements of the first time series as a second training example for the recurrent neural network model, wherein the first sub-sequence differs from the second sub-sequence in one or more of: (a) a starting offset within the first time series or (b) a number of elements.",
    "15. The method as recited in claim 6, wherein training the neural network model comprises utilizing a probabilistic sampling technique to determine whether input for a particular time step is to include (a) a sample from a demand distribution predicted by the recurrent neural network model for a previous time step or (b) a demand observation corresponding to the previous time step, wherein according to the probabilistic sampling technique, the probability of utilizing the sample increases as the total number of training iterations increases.",
    "16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors cause the one or more processors to: obtain an indication of a first data set comprising a respective time series of observations for a plurality of items; train a recurrent neural network model to generate forecasts for at least some different items of the plurality of items, wherein training of the recurrent neural network model is based at least in part on (a) analysis of a plurality of time series of the first data set and (b) performing one or more compensation operations to the plurality of times series of the first data set to compensate for differences between respective distributions of the demand observations for different items within the first data set, wherein, within the training data set of the recurrent neural network model, a count of observations of at least one other item exceeds a count of observations of a target item; generate respective probabilistic demand forecasts for different target items using one or more executions of the same recurrent neural network model; and provide, via a programmatic interface, respective representations of the probabilistic demand forecasts.",
    "17. The non-transitory computer-accessible storage medium as recited in claim 16, wherein to generate the probabilistic forecasts, the program instructions when executed on one or more processors cause the one or more processors to: predict, using the recurrent neural network model, one or more parameters of a first output distribution corresponding to a first time step; and utilize a sample from the first output distribution as input to predict parameters of a second output distribution corresponding to a second time step.",
    "18. The non-transitory computer-accessible storage medium as recited in claim 16, wherein to train the recurrent neural network model, the program instructions when executed on one or more processors cause the one or more processors to: identify statistical differences among a plurality of time series of the first data set; and perform one or more compensation operations based on the statistical differences.",
    "19. The non-transitory computer-accessible storage medium as recited in claim 16, wherein to train the recurrent neural network model, the program instructions when executed on one or more processors cause the one or more processors to: identify a first product category to which a first item of the plurality of items belongs, and a second product category to which a second item of the plurality of items belongs; and perform one or more compensation operations based on a difference between the number of items belonging to the first category and the number of items belonging to the second category.",
    "20. The non-transitory computer-accessible storage medium as recited in claim 16, wherein a first time series of the plurality of time series includes a first group of demand observations for a first item corresponding to a first year, and a second group of demand observations for the first item corresponding to a second year, wherein a particular holiday correlated with changes in demand for the first item occurs on a particular date in the first year, wherein the particular holiday occurs on a different date in the second year, wherein the program instructions when executed on one or more processors cause the one or more processors to: include, within input for at least one execution of the one or more executions of the recurrent neural network model, an indication of a future date on which the particular holiday is to occur, wherein a demand predicted for at least one of the target items by the recurrent neural network model is based at least in part on the future date.",
    "21. The non-transitory computer-accessible storage medium as recited in claim 16, wherein individual ones of the time series of the first data set represent demand observations for a respective item of the plurality of items, wherein the probabilistic demand forecast for at least one of the target items comprises a particular probabilistic demand forecast, wherein the program instructions when executed on one or more processors cause the one or more processors to: transmit a representation of the particular probabilistic demand forecast to one or more of: (a) an automated ordering system, wherein the automated ordering system is configured to generate one or more orders for one or more items based at least in part on the particular probabilistic demand forecast, (b) a discount planning system, (c) a facilities planning system, (d) a promotions planning system, or (e) a product placement planning system for a physical store.",
    "22. The non-transitory computer-accessible storage medium as recited in claim 16, wherein to generate the probabilistic demand forecasts, the program instructions when executed on one or more processors cause the one or more processors to: predict one or more parameters of at least one of: (a) a negative binomial distribution corresponding to an integer-valued time series or (b) a Gaussian distribution corresponding to a real-valued time series.",
    "23. The non-transitory computer-accessible storage medium as recited in claim 16, wherein to generate the probabilistic demand forecasts, the program instructions when executed on one or more processors cause the one or more processors to: utilize an approximate method to estimate at least a probability distribution of demand.",
    "24. The non-transitory computer-accessible storage medium as recited in claim 23, wherein the approximate method comprises a use of a generative adversarial network."
  ],
  "description_excerpt": "For many kinds of business and scientific applications, the ability to generate accurate forecasts of future values of various measures (e.g., retail sales, or demands for various types of goods and products) based on previously collected data is a critical requirement. The previously collected data often consists of a sequence of observations called a “time series” or a “time series data set” obtained at respective points in time, with values of the same collection of one or more variables obtained for each point in time (such as the per-day sales for a particular inventory item over a number of months, which may be recorded at an Internet-based retailer). Time series data sets are used in a variety of application domains, including for example weather forecasting, finance, econometrics, medicine, control engineering, astronomy and the like.\n\nThe statistical properties of some time series data, such as the demand data for products or items that may not necessarily be sold very frequently, can make it harder to generate forecasts. For example, an Internet-based footwear retailer may sell hundreds of different shoes, and for most days in a given time interval, there may be zero (or very few) sales of a particular type of shoe. Relatively few winter shoes may be sold for much of the summer months of a given year in this example scenario. On the other hand, when sales of such infrequently-sold items do pick up, they may be bursty - e.g., a lot of winter shoes may be sold in advance of, or during, a winter storm. The demand for some items may also be correlated with price reductions, holiday periods and other factors.",
  "cpc": [
    "G06Q 30/0202",
    "G06F 17/18",
    "G06F 30/20",
    "G06N 3/044",
    "G06N 3/0442",
    "G06N 3/0445",
    "G06N 3/045",
    "G06N 3/0455",
    "G06N 3/0475",
    "G06N 3/08",
    "G06N 3/084",
    "G06N 3/088",
    "G06N 3/09",
    "G06N 3/094",
    "G06N 3/0985"
  ],
  "ipc": [
    "G06F 17/18",
    "G06F 30/20",
    "G06N 3/04",
    "G06N 3/08"
  ],
  "assignees": [
    "Amazon Technologies Inc"
  ],
  "inventors": [
    "Valentin Flunkert",
    "David Jean Bernard Alfred Salinas"
  ],
  "filing_date": "2017-01-26",
  "publication_date": "2021-03-02",
  "grant_date": "2021-03-02",
  "priority_date": "2017-01-26",
  "application_number": "US-201715417070-A",
  "family_id": "74683051",
  "cited_by_count": 72,
  "citations": [
    "US20140108094A1",
    "WO2015060866A1"
  ]
}