Vision Based Target Tracking Using Tracklets

{
  "publication_number": "US2018114072A1",
  "country": "US",
  "kind": "A1",
  "title": "Vision Based Target Tracking Using Tracklets",
  "abstract": "A non-hierarchical and iteratively updated tracking system includes a first module for creating an initial trajectory model for multiple targets from a set of received image detections. A second module is connected to the first module to provide identification of multiple targets using a target model, and a third module is connected to the second module to solve a joint object function and maximal condition probability for the target module. A tracklet module can update the first module trajectory module, and after convergence, output a trajectory model for multiple targets.",
  "claims": [
    "1. A non-hierarchical and iteratively updated tracking system, comprising: a first module for creating an initial trajectory model for multiple targets from a set of received image detections; a second module connected to the first module to provide identification of multiple targets using a target model; a third module connected to the second module to solve a joint object function and maximal condition probability for the target module; and a tracklet module that updates the first module trajectory module, and after convergence, outputs a trajectory model for multiple targets. 2. The iteratively updated tracking system of claim 1, wherein the first module includes sliding windows initializable from at least one of a first frame and a previous sliding window. 3. The iteratively updated tracking system of claim 1, wherein the second module provides identification of multiple targets using a Markov random field model. 4. The iteratively updated tracking system of claim 1, wherein the third module finds an optimal target using a loopy belief propagation algorithm. 5. An iteratively updated tracking system, comprising: a first module for creating an initial trajectory model for multiple targets from a set of received image detections; a second module connected to the first module and including a Markov random field model to provide identification of multiple targets; a third module connected to the second module and including a loopy belief propagation algorithm to solve a joint object function and maximal condition probability of the Markov random field model; and a tracklet module that updates the first module trajectory module, and after convergence, outputs a trajectory model for multiple targets. 6. The iteratively updated tracking system of claim 5, wherein the first module includes sliding windows initializable from at least one of a first frame and a previous sliding window. 7. The iteratively updated tracking system of claim 5, wherein the second module finds an optimal target assignment that maximizes the conditional probability of Markov random field modelled targets based on the image detections. 8. The iteratively updated tracking system of claim 5, wherein the third module finds an optimal target assignment that maximizes the conditional probability of Markov random field modelled targets defined as: P  (L | Y; Γ) = 1 Z p  ∏ i  Φ  (l i, Y i; Γ)  ∏  Ψ  (l i, l j, Y i, Y j; Γ) 9. A non-hierarchical and iteratively updated tracking method, comprising the steps of: creating an initial trajectory model for multiple targets from a set of received image detections; using information from the initial trajectory model to identify multiple targets using a target model; solving a joint object function and maximal condition probability for the target model; and updating the trajectory model, and after convergence, outputting a trajectory model for multiple targets. 10. A non-hierarchical and iteratively updated tracking system, comprising: a sample collection module that takes tracklets and generates samples; an online metric learning module that uses the generated samples to form an initial appearance module metric with a regularized pairwise constrained component analysis (PCCA) algorithm; and a tracklet association module that receives appearance model metric data and can update the sample collection module and link tracklets into a final trajectory. 11. The non-hierarchical and iteratively updated tracking system of claim 10, wherein the sample collection module includes sliding windows to generate initial samples. 12. The iteratively updated tracking system of claim 10, wherein the sample collection module receives tracklets formed by a pairwise Markov random field model. 13. The iteratively updated tracking system of claim 10, wherein the sample collection module constrains tracklets by requiring each target object have one tracklet. 14. The iteratively updated tracking system of claim 10, wherein the sample collection module constrains tracklets by requiring each target object to have velocity changes less than a predetermined average velocity. 15. The iteratively updated tracking system of claim 10, wherein the online metric module learns a projection matrix based on logistic loss function. 16. The iteratively updated tracking system of claim 10, wherein the online metric module learns a projection matrix based on an objective function: min P   E  (P) = ∑ k = 1 N ′   β  (y k  (D P 2  (x m k, x n k) - 1)) + λ   P  2 17. The iteratively updated tracking system of claim 10, wherein the tracklet association module uses appearance and velocity descriptors to link tracklets. 18. The iteratively updated tracking system of claim 10, wherein the tracklet association module calculates an affinity score using an objective function: S ij (F i,F j)= C a (F i,F j) C v (F i,F j) 19. A non-hierarchical and iteratively updated tracking method, comprising the steps of: receiving tracklets and generating samples using a sample collection module; generating samples with an online metric learning module that uses the generated samples to form an initial appearance module metric with a regularized pairwise constrained component analysis (PCCA) algorithm; and receiving appearance model metric data and updating the sample collection module; and linking tracklets into a final trajectory. 20. The non-hierarchical and iteratively updated tracking method of claim 17, wherein the sample collection module includes sliding windows to generate initial samples. 21. The iteratively updated tracking method of claim 17, wherein the sample collection module receives tracklets formed by a pairwise Markov random field model. 22. The iteratively updated tracking method of claim 17, wherein the sample collection module constrains tracklets by requiring each target object have one tracklet. 23. The iteratively updated tracking system of claim 17, wherein the sample collection module constrains tracklets by requiring each target object to have velocity changes less than a predetermined average velocity."
  ],
  "description_excerpt": "The present disclosure relates generally to a deep learning system capable of tracking objects seen in video surveillance systems. Single and multiple targets can be tracked by linking tracklets into object trajectories.\n\nTracking multiple visual targets from security cameras is a challenging problem. Targets must be identified, trajectories determined, and target identity maintained over time. This can be difficult in complex scene due to the existence of occlusions, clutter, changes in illumination and appearance, and target interactions. As compared to tracking single objects, tracking multiple objects is much more complex. If only a single object is to be tracked, the state of only one target needs modelling, with detections from other targets eliminated as false alarms.\n\nPreviously object identification and association between frames was accomplished locally, i.e. using local information from a few neighboring frames or frame by frame. Image information cues such as appearance, motion, size and location can be used to measure the similarity between detections from two consecutive frames. However, given only the image information in a small-time window, local association methods do not deal gracefully with long-term occlusion due to the ambiguous and noisy observations, resulting in tracking failures (e.g., trajectory fragmentation and identity switches).\n\nIn contrast to the local tracking methods, global inference techniques that evaluate over all trajectories simultaneously for a longer period can be used.",
  "cpc": [
    "H04N 7/181",
    "B25J 9/1697",
    "G06F 18/295",
    "G06K 9/00771",
    "G06K 9/3241",
    "G06T 2207/20084",
    "G06T 2207/30232",
    "G06T 2207/30241",
    "G06T 7/215",
    "G06T 7/277",
    "G06V 10/85",
    "G06V 20/52",
    "H04N 7/183"
  ],
  "ipc": [
    "G06K 9/00",
    "G06K 9/32",
    "G06T 7/215",
    "H04N 7/18"
  ],
  "assignees": [
    "VMAXX Inc"
  ],
  "inventors": [
    "Jinjun Wang",
    "Rui Shi",
    "Shun Zhang"
  ],
  "filing_date": "2017-10-24",
  "publication_date": "2018-04-26",
  "priority_date": "2016-10-25",
  "application_number": "US-201715792557-A",
  "family_id": "61970983",
  "cited_by_count": 31,
  "citations": [
    "US20160161606A1",
    "US20170154212A1",
    "US20180025500A1"
  ]
}