Method for detecting and re-identifying objects using a neural network

{
  "publication_number": "US11978218B2",
  "country": "US",
  "kind": "B2",
  "title": "Method for detecting and re-identifying objects using a neural network",
  "abstract": "A method for detecting and re-identifying objects using a neural network. The method includes the steps: extracting features from an image, the features comprising information about at least one object in the image; detecting the at least one object in the image using an anchor-based object detection based on the extracted features, classification data being determined by a classification for detecting the object with the aid of at least one anchor and regression data being determined by a regression; and re-identifying the at least one object by determining embedding data based on the extracted features, the embedding data representing an object description for the at least one feature of the image.",
  "claims": [
    "1. A method using a neural network, the method comprising the following steps: extracting features from each of a plurality of images, the features including information about objects in the images; detecting the objects in the images using an anchor-based object detection based on the extracted features, the detecting including: determining classification data by a classification of the objects using at least one anchor; determining regression data by a regression; and determining, respectively for each of the images, a respective embedding data vector that describes one of the objects based on the extracted features; and for each respective one of at least one of the objects, tracking the respective object between different ones of the images based on a degree of similarity between the embedding data vectors respectively determined for the different ones of the images, the tracking including identifying that the object identified as being in the different ones of the images is the same.",
    "2. The method as recited in claim 1, wherein the images between which the respective object is tracked are temporally successive images, and the tracking is further based on the determined classification data and the determined regression data.",
    "3. The method as recited in claim 1, wherein the detecting of the at least one object and the tracking occur using the same neural network.",
    "4. The method as recited in claim 1, wherein the embedding data vector is determined by an embedding layer learned with a loss function.",
    "5. The method as recited in claim 4, wherein the loss function includes a metrics, the metrics including an L2norm or a cosine distance.",
    "6. The method as recited in claim 4, wherein the embedding layer is learned with object re-identification, distances being used between the embedding data vectors of detected objects.",
    "7. The method as recited in claim 4, wherein the embedding layer is learned with temporal detection, the embedding data of detected objects being used as input for a tracking algorithm.",
    "8. A neural network, comprising: a feature extractor configured to extract features from each of a plurality of images, the features including information about objects in the images; a classification layer configured to determine classification data for detecting the objects in the images using an anchor-based object detection; a regression layer configured to determine regression data for detecting the objects in the images using the anchor-based object detection; and an embedding layer configured to determine, respectively for each of the images, a respective embedding data vector that describes one of the objects based on the extracted features; wherein the neural network is configured to, for each respective one of at least one of the objects, track the respective object between different ones of the images based on a degree of similarity between the embedding data vectors respectively determined for the different ones of the images, the tracking including identifying that the object identified as being in the different ones of the images is the same.",
    "9. A control method for an at least partially autonomous robot, the method comprising the following steps: receiving image data of the at least partially autonomous robot, the image data including a plurality of images representing a surroundings of the robot; applying a neural network method using a neural network, the neural network method including: extracting features from each of the plurality of images, the features including information about objects in the images; detecting the objects in the images using an anchor-based object detection based on the extracted features, the detecting including: determining classification data by a classification of the objects using at least one anchor; determining regression data by a regression; and determining, respectively for each of the images, a respective embedding data vector that describes one of the objects based on the extracted features; and for each respective one of at least one of the objects, tracking the respective object between different ones of the images based on a degree of similarity between the embedding data vectors respectively determined for the different ones of the images, the tracking including identifying that the object identified as being in the different ones of the images is the same; and controlling the at least partially autonomous robot based on the tracking.",
    "10. A non-transitory machine-readable storage medium on which is stored a computer program that is executable by a computer, and that, when executed, causes the computer to perform a method using a neural network, the method comprising the following steps: extracting features from each of a plurality of images, the features including information about objects in the images; detecting the objects in the images using an anchor-based object detection based on the extracted features, the detecting including: determining classification data by a classification of the objects using at least one anchor; and determining regression data by a regression; and determining, respectively for each of the images, a respective embedding data vector that describes one of the objects based on the extracted features; and for each respective one of at least one of the objects, tracking the respective object between different ones of the images based on a degree of similarity between the embedding data vectors respectively determined for the different ones of the images, the tracking including identifying that the object identified as being in the different ones of the images is the same."
  ],
  "description_excerpt": "The present invention relates to a method for detecting and re-identifying objects using a neural network, to a neural network, to a control method, to a computer program and to a machine-readable storage medium.\n\nWhen using an external tracking algorithm for detecting and re-identifying objects, the external tracking algorithm must find again a box with the same object in successive images, the so-called frames. There are various approaches which for example compare the content of the boxes to one another or take the position, size and the aspect ratio into consideration. These methods are often error-prone if objects are near one another or objects are for a time span partially or entirely concealed.\n\nThere may therefore exist the desire for an improved method for detecting and re-identifying objects using a neural network.\n\nSpecific embodiments of the present invention and expedient developments of the present invention derive from the description herein and the figures.\n\nAccording to one aspect of the present invention, a method for detecting and re-identifying objects using a neural network is provided. In an example embodiment of the present invention, the method includes the following steps: Extracting features from an image, the features comprising information about at least one object in the image. Detecting the at least one object in the image using an anchor-based object detection based on the extracted features, classification data being determined by a classification for detecting the object with the aid of at least one anchor and regression data being determined by a regression.",
  "cpc": [
    "G06T 7/246",
    "B25J 9/1697",
    "G05B 2219/40543",
    "G05B 2219/40563",
    "G06F 18/22",
    "G06F 18/24",
    "G06N 3/045",
    "G06N 3/0464",
    "G06N 3/08",
    "G06N 3/09",
    "G06T 2207/20081",
    "G06T 2207/20084",
    "G06T 7/20",
    "G06V 10/255",
    "G06V 10/82",
    "G06V 20/52"
  ],
  "ipc": [
    "B25J 9/16",
    "G06F 18/22",
    "G06F 18/24",
    "G06N 3/08",
    "G06T 7/20",
    "G06T 7/246",
    "G06V 10/20",
    "G06V 10/82",
    "G06V 20/52"
  ],
  "assignees": [
    "Robert Bosch GmbH"
  ],
  "inventors": [
    "Denis Stalz-John",
    "Tamas Kapelner"
  ],
  "filing_date": "2020-10-19",
  "publication_date": "2024-05-07",
  "grant_date": "2024-05-07",
  "priority_date": "2019-10-25",
  "application_number": "US-202017073780-A",
  "family_id": "75379092",
  "cited_by_count": 0,
  "citations": [
    "US20190205643A1",
    "US20200097742A1",
    "US20200294310A1",
    "US20200294257A1"
  ]
}