Patent · US10360214B2 · B2 · US

Ensuring reproducibility in an artificial intelligence infrastructure

(11) Publication number

US10360214B2

(21) Application number

16/045,814

(22) Filing date

2018-07-26

(30) Priority date

2017-10-19

(43) Publication date

2019-07-23

(45) Date of grant

2019-07-23

(51) IPC

G06N 3/08; G06T 1/20; G06T 1/60; G06F 16/00; G06F 16/22; G06F 16/2453; G06F 3/06; G06N 20/00

(52) CPC

G06F Electric digital data processing: 16/24534, 16/152, 16/2255, 16/254, 18/213, 3/06, 3/061, 3/0629, 3/064, 3/0647, 3/0652, 3/0679, 9/50, 9/5005, 9/5011, 9/5016, 9/5027, 9/505
G06N Computing arrangements based on specific computational models: 20/00, 3/08
G06T Image data processing or generation, in general: 1/20, 1/60, 2200/28

(73) Assignee

Pure Storage Inc

(72) Inventors

Brian Gold; Emily Watkins; Ivan Jibaja; Igor Ostrovsky; Roy Kim

(54) Title

Ensuring reproducibility in an artificial intelligence infrastructure

(57) Abstract

Ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; and storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input.

Full text

View on Google Patents

Claims (17)

A method of ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (Gal) servers, the method comprising: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input; determining, by the artificial intelligence infrastructure, whether data related to a previously executed machine learning model should be tiered off of the one or more storage systems; and responsive to determining that the data related to the previously executed machine learning model should be tiered off of the one or more storage systems: storing the data related to the previously executed machine learning model in lower-tier storage; and removing, from the one or more storage systems, the data related to the previously executed machine learning model.
The method of claim 1 wherein storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the dataset, the one or more transformations applied to the dataset, and the transformed dataset, a hash value; and storing, within the one or more storage systems, the hash value.
The method of claim 1 wherein storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more machine learning models and the transformed dataset, a hash value; and storing, within the one or more storage systems, the hash value.
The method of claim 1 further comprising: identifying, by the unified management plane, differences between a machine learning model and a machine learning model previously executed by the artificial intelligence infrastructure; and storing, within the one or more storage systems, only the portion of the machine learning model that differs from the machine learning models previously executed by the artificial intelligence infrastructure.
The method of claim 1 further comprising identifying, from amongst a plurality of machine learning models, a preferred machine learning model.
The method of claim 1 further comprising tracking the improvement of a particular machine learning model over time.
An artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (GMT) servers, the artificial intelligence infrastructure configured to carry out the steps of: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input; determining, by the artificial intelligence infrastructure, whether data related to a previously executed machine learning model should be tiered off of the one or more storage systems; and responsive to determining that the data related to the previously executed machine learning model should be tiered off of the one or more storage systems: storing the data related to the previously executed machine learning model in lower-tier storage; and removing, from the one or more storage systems, the data related to the previously executed machine learning model.
The artificial intelligence infrastructure of claim 7 wherein storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more transformations applied to the dataset and the transformed dataset, a hash value; and storing, within the one or more storage systems, the hash value.
The artificial intelligence infrastructure of claim 7 wherein storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more machine learning models, a hash value; and storing, within the one or more storage systems, the hash value.
The artificial intelligence infrastructure of claim 7 wherein the artificial intelligence infrastructure is further configured to carry out the steps of: identifying, by the unified management plane, differences between a machine learning model and a machine learning model previously executed by the artificial intelligence infrastructure; and storing, within the one or more storage systems, only the portion of the machine learning model that differs from the machine learning models previously executed by the artificial intelligence infrastructure.
The artificial intelligence infrastructure of claim 7 wherein the artificial intelligence infrastructure is further configured to carry out the step of identifying, from amongst a plurality of machine learning models, a preferred machine learning model.
The artificial intelligence infrastructure of claim 7 wherein the artificial intelligence infrastructure is further configured to carry out the step of tracking the improvement of a particular machine learning model over time.
An apparatus for ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (Gal) servers, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input; determining, by the artificial intelligence infrastructure, whether data related to a previously executed machine learning model should be tiered off of the one or more storage systems; and responsive to determining that the data related to the previously executed machine learning model should be tiered off of the one or more storage systems: storing the data related to the previously executed machine learning model in lower-tier storage; and removing, from the one or more storage systems, the data related to the previously executed machine learning model.
The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: identifying, by the unified management plane, differences between a machine learning model and a machine learning model previously executed by the artificial intelligence infrastructure; and storing, within the one or more storage systems, only the portion of the machine learning model that differs from the machine learning models previously executed by the artificial intelligence infrastructure.
The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of identifying, from amongst a plurality of machine learning models, a preferred machine learning model.
The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of tracking the improvement of a particular machine learning model over time.
The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more machine learning models, a hash value; and storing, within the one or more storage systems, the hash value.

Description

FIG. 1A illustrates a first example system for data storage in accordance with some implementations.

FIG. 1B illustrates a second example system for data storage in accordance with some implementations.

FIG. 1C illustrates a third example system for data storage in accordance with some implementations.

FIG. 1D illustrates a fourth example system for data storage in accordance with some implementations.

FIG. 2A is a perspective view of a storage cluster with multiple storage nodes and internal storage coupled to each storage node to provide network attached storage, in accordance with some embodiments.

FIG. 2B is a block diagram showing an interconnect switch coupling multiple storage nodes in accordance with some embodiments.

FIG. 2C is a multiple level block diagram, showing contents of a storage node and contents of one of the non-volatile solid state storage units in accordance with some embodiments.

FIG. 2D shows a storage server environment, which uses embodiments of the storage nodes and storage units of some previous figures in accordance with some embodiments.

FIG. 2E is a blade hardware block diagram, showing a control plane, compute and storage planes, and authorities interacting with underlying physical resources, in accordance with some embodiments.

FIG. 2F depicts elasticity software layers in blades of a storage cluster, in accordance with some embodiments.

FIG. 2G depicts authorities and storage resources in blades of a storage cluster, in accordance with some embodiments.

Citations (173)

EP0725324A2
US5706210A
US5799200A
US6012032A
US5933598A
US6085333A
US6647514B1
US6643641B1
US20020013802A1
US6789162B1
US20030145172A1
US20030191783A1
US20030225961A1
US7334124B2
US7146521B1
US7107389B2
US20040080985A1
US20040153844A1
US20040111573A1
US20040193814A1
US20100332754A1
US7437530B1
US20040260967A1
US7089272B1
US20050160416A1
US20050188246A1
US20050216800A1
US7493424B1
US20070168321A1
US8042163B1
US20060015771A1
US20060129817A1
US20060161726A1
US20060230245A1
US7689609B2
US20060239075A1
US20070022227A1
US20070028068A1
US20070055702A1
US20070109856A1
US20070150689A1
US20070220227A1
US7899780B1
US20070294564A1
US20070294563A1
US20140020083A1
US20080005587A1
US20080077825A1
US8620970B2
US7669029B1
US8437189B1
US20080162674A1
US20080195833A1
US8200887B2
US9081713B1
US9311182B2
US9189334B2
US9632870B2
US20080270678A1
US20080282045A1
US7743003B1
US20090077340A1
US20090100115A1
US8271700B1
US7743191B1
US20090198889A1
US8949863B1
US20100052625A1
US8086585B1
US20140215590A1
US8762642B2
US20100211723A1
US9134922B2
US20100246266A1
US20100257142A1
US20100262764A1
US20100325345A1
US20150026387A1
US20120303919A1
US20110072290A1
US20110125955A1
US20110131231A1
US8387136B2
US20110167221A1
US20120023144A1
US20120054264A1
US8566546B1
US20120079318A1
US20150121137A1
US20120131253A1
US8812860B1
US20130227201A1
WO2012087648A2
US8465332B2
US8578442B1
US20120311000A1
US8751463B1
US8769622B2
US20130007845A1
US20130031414A1
US8931041B1
US20130036272A1
US8527544B1
US20130071087A1
US8909854B2
US20150089569A1
US20140258526A1
WO2013071087A1
US20130311434A1
US20130145447A1
US20130219164A1
US8613066B1
US8800009B1
US20130191555A1
US20130198459A1
US20130205173A1
US20130290607A1
US20130318297A1
US20130332614A1
US20140282983A1
US20140074850A1
US20140082715A1
US20140086146A1
US20140090009A1
US8990905B1
US20140096220A1
US8850546B1
US20140101434A1
US9209973B2
US20140164774A1
US20140173232A1
US20140195636A1
WO2014110137A1
US20140201541A1
US20140201512A1
US20140208155A1
US20140229654A1
US20140230017A1
WO2014145149A1
US20140285917A1
US20140325262A1
US20140351627A1
US20140373104A1
US9124569B2
US20140373126A1
US8898346B1
US8984602B1
US20150074463A1
US20150095515A1
US20150113203A1
US20150134920A1
US20150149822A1
US20150193169A1
US9250823B1
US20150378888A1
WO2016015008A1
US20160098323A1
US20160071017A1
US20160352834A1
WO2016190938A1
US20160352830A1
US20160352720A1
US20160350009A1
US9444822B1
WO2016195961A1
WO2016195958A1
WO2016195759A1
US9300660B1
US20170091670A1
US20170220949A1
US9507532B1
US20170344910A1
US20180189674A1

Record as JSON

{
  "publication_number": "US10360214B2",
  "country": "US",
  "kind": "B2",
  "title": "Ensuring reproducibility in an artificial intelligence infrastructure",
  "abstract": "Ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; and storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input.",
  "claims": [
    "1. A method of ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (Gal) servers, the method comprising: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input; determining, by the artificial intelligence infrastructure, whether data related to a previously executed machine learning model should be tiered off of the one or more storage systems; and responsive to determining that the data related to the previously executed machine learning model should be tiered off of the one or more storage systems: storing the data related to the previously executed machine learning model in lower-tier storage; and removing, from the one or more storage systems, the data related to the previously executed machine learning model.",
    "2. The method of claim 1 wherein storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the dataset, the one or more transformations applied to the dataset, and the transformed dataset, a hash value; and storing, within the one or more storage systems, the hash value.",
    "3. The method of claim 1 wherein storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more machine learning models and the transformed dataset, a hash value; and storing, within the one or more storage systems, the hash value.",
    "4. The method of claim 1 further comprising: identifying, by the unified management plane, differences between a machine learning model and a machine learning model previously executed by the artificial intelligence infrastructure; and storing, within the one or more storage systems, only the portion of the machine learning model that differs from the machine learning models previously executed by the artificial intelligence infrastructure.",
    "5. The method of claim 1 further comprising identifying, from amongst a plurality of machine learning models, a preferred machine learning model.",
    "6. The method of claim 1 further comprising tracking the improvement of a particular machine learning model over time.",
    "7. An artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (GMT) servers, the artificial intelligence infrastructure configured to carry out the steps of: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input; determining, by the artificial intelligence infrastructure, whether data related to a previously executed machine learning model should be tiered off of the one or more storage systems; and responsive to determining that the data related to the previously executed machine learning model should be tiered off of the one or more storage systems: storing the data related to the previously executed machine learning model in lower-tier storage; and removing, from the one or more storage systems, the data related to the previously executed machine learning model.",
    "8. The artificial intelligence infrastructure of claim 7 wherein storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more transformations applied to the dataset and the transformed dataset, a hash value; and storing, within the one or more storage systems, the hash value.",
    "9. The artificial intelligence infrastructure of claim 7 wherein storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input further comprises: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more machine learning models, a hash value; and storing, within the one or more storage systems, the hash value.",
    "10. The artificial intelligence infrastructure of claim 7 wherein the artificial intelligence infrastructure is further configured to carry out the steps of: identifying, by the unified management plane, differences between a machine learning model and a machine learning model previously executed by the artificial intelligence infrastructure; and storing, within the one or more storage systems, only the portion of the machine learning model that differs from the machine learning models previously executed by the artificial intelligence infrastructure.",
    "11. The artificial intelligence infrastructure of claim 7 wherein the artificial intelligence infrastructure is further configured to carry out the step of identifying, from amongst a plurality of machine learning models, a preferred machine learning model.",
    "12. The artificial intelligence infrastructure of claim 7 wherein the artificial intelligence infrastructure is further configured to carry out the step of tracking the improvement of a particular machine learning model over time.",
    "13. An apparatus for ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (Gal) servers, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input; determining, by the artificial intelligence infrastructure, whether data related to a previously executed machine learning model should be tiered off of the one or more storage systems; and responsive to determining that the data related to the previously executed machine learning model should be tiered off of the one or more storage systems: storing the data related to the previously executed machine learning model in lower-tier storage; and removing, from the one or more storage systems, the data related to the previously executed machine learning model.",
    "14. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: identifying, by the unified management plane, differences between a machine learning model and a machine learning model previously executed by the artificial intelligence infrastructure; and storing, within the one or more storage systems, only the portion of the machine learning model that differs from the machine learning models previously executed by the artificial intelligence infrastructure.",
    "15. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of identifying, from amongst a plurality of machine learning models, a preferred machine learning model.",
    "16. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of tracking the improvement of a particular machine learning model over time.",
    "17. The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: generating, by the artificial intelligence infrastructure applying a predetermined hash function to the one or more machine learning models, a hash value; and storing, within the one or more storage systems, the hash value."
  ],
  "description_excerpt": "FIG. 1A illustrates a first example system for data storage in accordance with some implementations.\n\nFIG. 1B illustrates a second example system for data storage in accordance with some implementations.\n\nFIG. 1C illustrates a third example system for data storage in accordance with some implementations.\n\nFIG. 1D illustrates a fourth example system for data storage in accordance with some implementations.\n\nFIG. 2A is a perspective view of a storage cluster with multiple storage nodes and internal storage coupled to each storage node to provide network attached storage, in accordance with some embodiments.\n\nFIG. 2B is a block diagram showing an interconnect switch coupling multiple storage nodes in accordance with some embodiments.\n\nFIG. 2C is a multiple level block diagram, showing contents of a storage node and contents of one of the non-volatile solid state storage units in accordance with some embodiments.\n\nFIG. 2D shows a storage server environment, which uses embodiments of the storage nodes and storage units of some previous figures in accordance with some embodiments.\n\nFIG. 2E is a blade hardware block diagram, showing a control plane, compute and storage planes, and authorities interacting with underlying physical resources, in accordance with some embodiments.\n\nFIG. 2F depicts elasticity software layers in blades of a storage cluster, in accordance with some embodiments.\n\nFIG. 2G depicts authorities and storage resources in blades of a storage cluster, in accordance with some embodiments.",
  "cpc": [
    "G06F 16/24534",
    "G06F 16/152",
    "G06F 16/2255",
    "G06F 16/254",
    "G06F 18/213",
    "G06F 3/06",
    "G06F 3/061",
    "G06F 3/0629",
    "G06F 3/064",
    "G06F 3/0647",
    "G06F 3/0652",
    "G06F 3/0679",
    "G06F 9/50",
    "G06F 9/5005",
    "G06F 9/5011",
    "G06F 9/5016",
    "G06F 9/5027",
    "G06F 9/505",
    "G06N 20/00",
    "G06N 3/08",
    "G06T 1/20",
    "G06T 1/60",
    "G06T 2200/28"
  ],
  "ipc": [
    "G06N 3/08",
    "G06T 1/20",
    "G06T 1/60",
    "G06F 16/00",
    "G06F 16/22",
    "G06F 16/2453",
    "G06F 3/06",
    "G06N 20/00"
  ],
  "assignees": [
    "Pure Storage Inc"
  ],
  "inventors": [
    "Brian Gold",
    "Emily Watkins",
    "Ivan Jibaja",
    "Igor Ostrovsky",
    "Roy Kim"
  ],
  "filing_date": "2018-07-26",
  "publication_date": "2019-07-23",
  "grant_date": "2019-07-23",
  "priority_date": "2017-10-19",
  "application_number": "US-201816045814-A",
  "family_id": "63452703",
  "cited_by_count": 31,
  "citations": [
    "EP0725324A2",
    "US5706210A",
    "US5799200A",
    "US6012032A",
    "US5933598A",
    "US6085333A",
    "US6647514B1",
    "US6643641B1",
    "US20020013802A1",
    "US6789162B1",
    "US20030145172A1",
    "US20030191783A1",
    "US20030225961A1",
    "US7334124B2",
    "US7146521B1",
    "US7107389B2",
    "US20040080985A1",
    "US20040153844A1",
    "US20040111573A1",
    "US20040193814A1",
    "US20100332754A1",
    "US7437530B1",
    "US20040260967A1",
    "US7089272B1",
    "US20050160416A1",
    "US20050188246A1",
    "US20050216800A1",
    "US7493424B1",
    "US20070168321A1",
    "US8042163B1",
    "US20060015771A1",
    "US20060129817A1",
    "US20060161726A1",
    "US20060230245A1",
    "US7689609B2",
    "US20060239075A1",
    "US20070022227A1",
    "US20070028068A1",
    "US20070055702A1",
    "US20070109856A1",
    "US20070150689A1",
    "US20070220227A1",
    "US7899780B1",
    "US20070294564A1",
    "US20070294563A1",
    "US20140020083A1",
    "US20080005587A1",
    "US20080077825A1",
    "US8620970B2",
    "US7669029B1",
    "US8437189B1",
    "US20080162674A1",
    "US20080195833A1",
    "US8200887B2",
    "US9081713B1",
    "US9311182B2",
    "US9189334B2",
    "US9632870B2",
    "US20080270678A1",
    "US20080282045A1",
    "US7743003B1",
    "US20090077340A1",
    "US20090100115A1",
    "US8271700B1",
    "US7743191B1",
    "US20090198889A1",
    "US8949863B1",
    "US20100052625A1",
    "US8086585B1",
    "US20140215590A1",
    "US8762642B2",
    "US20100211723A1",
    "US9134922B2",
    "US20100246266A1",
    "US20100257142A1",
    "US20100262764A1",
    "US20100325345A1",
    "US20150026387A1",
    "US20120303919A1",
    "US20110072290A1",
    "US20110125955A1",
    "US20110131231A1",
    "US8387136B2",
    "US20110167221A1",
    "US20120023144A1",
    "US20120054264A1",
    "US8566546B1",
    "US20120079318A1",
    "US20150121137A1",
    "US20120131253A1",
    "US8812860B1",
    "US20130227201A1",
    "WO2012087648A2",
    "US8465332B2",
    "US8578442B1",
    "US20120311000A1",
    "US8751463B1",
    "US8769622B2",
    "US20130007845A1",
    "US20130031414A1",
    "US8931041B1",
    "US20130036272A1",
    "US8527544B1",
    "US20130071087A1",
    "US8909854B2",
    "US20150089569A1",
    "US20140258526A1",
    "WO2013071087A1",
    "US20130311434A1",
    "US20130145447A1",
    "US20130219164A1",
    "US8613066B1",
    "US8800009B1",
    "US20130191555A1",
    "US20130198459A1",
    "US20130205173A1",
    "US20130290607A1",
    "US20130318297A1",
    "US20130332614A1",
    "US20140282983A1",
    "US20140074850A1",
    "US20140082715A1",
    "US20140086146A1",
    "US20140090009A1",
    "US8990905B1",
    "US20140096220A1",
    "US8850546B1",
    "US20140101434A1",
    "US9209973B2",
    "US20140164774A1",
    "US20140173232A1",
    "US20140195636A1",
    "WO2014110137A1",
    "US20140201541A1",
    "US20140201512A1",
    "US20140208155A1",
    "US20140229654A1",
    "US20140230017A1",
    "WO2014145149A1",
    "US20140285917A1",
    "US20140325262A1",
    "US20140351627A1",
    "US20140373104A1",
    "US9124569B2",
    "US20140373126A1",
    "US8898346B1",
    "US8984602B1",
    "US20150074463A1",
    "US20150095515A1",
    "US20150113203A1",
    "US20150134920A1",
    "US20150149822A1",
    "US20150193169A1",
    "US9250823B1",
    "US20150378888A1",
    "WO2016015008A1",
    "US20160098323A1",
    "US20160071017A1",
    "US20160352834A1",
    "WO2016190938A1",
    "US20160352830A1",
    "US20160352720A1",
    "US20160350009A1",
    "US9444822B1",
    "WO2016195961A1",
    "WO2016195958A1",
    "WO2016195759A1",
    "US9300660B1",
    "US20170091670A1",
    "US20170220949A1",
    "US9507532B1",
    "US20170344910A1",
    "US20180189674A1"
  ]
}

Record 2,699 of 8,000 in Patents full text (MLC-0201). Request the full dataset.