Online utility-driven spatially-referenced data collector for classification

{
  "publication_number": "US11267128B2",
  "country": "US",
  "kind": "B2",
  "title": "Online utility-driven spatially-referenced data collector for classification",
  "abstract": "Data associated with a region, acquired by a robot may be passed to a previously trained classifier. The classifier outputs a classification label L, and a confidence score C. Responsive to determining that the confidence score C is below a threshold T, the acquired data can be added to a training data set associated with the classifier, and the classifier retrained using the training data set which include at least information from the acquired data. Responsive to determining that the confidence score C is below the threshold T, at least one candidate region having characteristic similarity to the region can be identified. Responsive to determining that the confidence score C is not below the threshold T, at least one candidate region having a different characteristic from the region can be identified. The robot may be caused to acquire data associated with the candidate region.",
  "claims": [
    "1. A system comprising: a hardware processor; a memory coupled to the hardware processor; the hardware processor operable to at least: pass data acquired by a robot to a classifier, the classifier outputting a classification label L, and a confidence score C, the data associated with a region; responsive to determining that the confidence score C is below a threshold T, add the acquired data to a training data set associated with the classifier, and retrain the classifier using the training data set, wherein the training data set includes at least information from the acquired data; responsive to determining that the confidence score C is below the threshold T, identify at least one candidate region having characteristic similarity to the region, the characteristic similarity determined based on meeting a criterion; responsive to determining that the confidence score C is not below the threshold T, identify at least one candidate region having a different characteristic from the region, wherein additional data acquired from said at least one candidate region can be used to retrain the classifier.",
    "2. The system of claim 1, wherein the hardware processor is further operable to cause the robot to acquire data associated with the candidate region.",
    "3. The system of claim 1, wherein said at least one candidate region comprises a plurality of candidate regions and the hardware processor is further operable to compute a navigation route including at least some of the plurality of candidate regions.",
    "4. The system of claim 1, wherein said at least one candidate region comprises a plurality of candidate regions and the hardware processor is further operable to rank the plurality of candidate regions by similarity to the region.",
    "5. The system of claim 1, wherein said at least one candidate region comprises a plurality of candidate regions and the hardware processor is further operable to rank the plurality of candidate regions by differences to the region.",
    "6. The system of claim 1, wherein the data associated with the candidate region is further added to the training data set for further retraining the classifier.",
    "7. The system of claim 1, wherein the data acquired by the robot includes at least image data.",
    "8. The system of claim 3, wherein the navigation route is computed to optimize at least a cost of travelling to the candidate regions.",
    "9. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to: pass data acquired by a robot to a classifier, the classifier outputting a classification label L, and a confidence score C, the data associated with a region; responsive to determining that the confidence score C is below a threshold T, add the acquired data to a training data set associated with the classifier, and retrain the classifier using the training data set, wherein the training data set includes at least information from the acquired data; responsive to determining that the confidence score C is below the threshold T, identify at least one candidate region having characteristic similarity to the region, the characteristic similarity determined based on meeting a criterion; responsive to determining that the confidence score C is not below the threshold T, identify at least one candidate region having a different characteristic from the region, wherein additional data acquired from said at least one candidate region can be used to retrain the classifier.",
    "10. The computer program product of claim 8, wherein the device is further caused to cause the robot to acquire data associated with the candidate region.",
    "11. The computer program product of claim 8, wherein said at least one candidate region comprises a plurality of candidate regions and the device is further caused to compute a navigation route including at least some of the plurality of candidate regions.",
    "12. The computer program product of claim 8, wherein said at least one candidate region comprises a plurality of candidate regions and the device is further caused to rank the plurality of candidate regions by similarity to the region, responsive to determining that the confidence score C is below a threshold T.",
    "13. The computer program product of claim 8, wherein said at least one candidate region comprises a plurality of candidate regions and the device is further caused to rank the plurality of candidate regions by differences to the region, responsive to determining that the confidence score C is not below the threshold T.",
    "14. The computer program product of claim 8, wherein the data associated with the candidate region is further added to the training data set for further retraining the classifier.",
    "15. The computer program product of claim 8, wherein the data acquired by the robot includes at least image data.",
    "16. The computer program product of claim 11, wherein the navigation route is computed to optimize at least a cost of travelling to the candidate regions.",
    "17. A computer-implemented method comprising: passing data acquired by a robot to a classifier, the classifier outputting a classification label L, and a confidence score C, the data associated with a region; responsive to determining that the confidence score C is below a threshold T, adding the acquired data to a training data set associated with the classifier, and retraining the classifier using the training data set, wherein the training data set includes at least information from the acquired data; responsive to determining that the confidence score C is below the threshold T, identifying at least one candidate region having characteristic similarity to the region, the characteristic similarity determined based on meeting a criterion; responsive to determining that the confidence score C is not below the threshold T, identifying at least one candidate region having a different characteristic from the region, wherein additional data acquired from said at least one candidate region can be used to retrain the classifier.",
    "18. The computer-implemented method of claim 17, further comprising causing the robot to acquire data associated with the candidate region.",
    "19. The computer-implemented method of claim 17, wherein said at least one candidate region comprises a plurality of candidate regions, and the method further comprises computing a navigation route including at least some of the plurality of candidate regions.",
    "20. The computer-implemented method of claim 17, wherein said at least one candidate region comprises a plurality of candidate regions and the method further comprises ranking the plurality of candidate regions by similarity to the region."
  ],
  "description_excerpt": "The present application relates generally to computers and computer applications, and more particularly to machine learning and training datasets.\n\nMachine learning systems rely on availability and diversity of data in order to train accurate models. Some of the challenges associated with the construction of training sets include unbalanced databases and small amount of available data where the amount of data available may be insufficient for all categories. In unbalanced databases, some categories may be under-represented in the dataset, thus impacting the accuracy of their classification. New or open problem can also pose a challenge. For instance, if a problem is new or if a new category is discovered, lack of balance and insufficiency of data set can emerge. Approaches such as brute force approaches for navigating in a physical environment with the goal of collecting comprehensive datasets may be expensive and ineffective, and the marginal gains such approaches may bring to a relatively incomplete dataset may be minimal.\n\nA system, in one aspect, can include a hardware processor. A memory can be coupled to the hardware processor. The hardware processor may be operable to pass data acquired by a robot to a previously trained classifier. The classifier can output a classification label L, and a confidence score C, the data associated with a region.",
  "cpc": [
    "G06V 10/764",
    "B25J 9/1664",
    "B25J 9/1697",
    "G01C 21/20",
    "G06F 18/214",
    "G06F 18/22",
    "G06F 18/24",
    "G06K 9/6215",
    "G06K 9/6267",
    "G06V 10/774",
    "G06V 10/82",
    "G06V 20/10"
  ],
  "ipc": [
    "B25J 9/00",
    "B25J 9/16",
    "G06K 9/62"
  ],
  "assignees": [
    "International Business Machines Corp"
  ],
  "inventors": [
    "Dario Augusto Borges Oliveira",
    "Andrea Britto Mattos Lima",
    "Priscilla Barreira Avegliano",
    "Carlos Henrique Cardonha"
  ],
  "filing_date": "2019-05-08",
  "publication_date": "2022-03-08",
  "grant_date": "2022-03-08",
  "priority_date": "2019-05-08",
  "application_number": "US-201916406626-A",
  "family_id": "73046882",
  "cited_by_count": 0,
  "citations": [
    "US5548516A",
    "US6917893B2",
    "US7155336B2",
    "US20060104494A1",
    "US8260485B1",
    "US20100226582A1",
    "US20130011083A1",
    "US8341223B1",
    "US20140316614A1",
    "US20160018224A1",
    "US20150363717A1",
    "US20190126484A1",
    "US10272566B2",
    "US9454907B2",
    "US9454157B1",
    "US20170162060A1",
    "US10081426B2",
    "US10123674B2",
    "US9988781B2"
  ]
}