Large-Scale Multi-Target Tracking Problem for Interacting Targets

The unique physical properties of nanoparticles depend on their sizes and shapes. Therefore, an ability to precisely control the size of nanoparticles and tune their morphology will allow scientists and engineers to modify their physical properties, which will lead to many potential applications. To precisely control nanoparticles' sizes and shapes requires a deep understanding of their growth mechanism. To understand their growth mechanism, a direct observation and its quantitative analysis are both necessary. In the direct observation study, the electron microscopy method has shown promises, because the in situ method enables researchers to see the growth process using video recordings. In these video recordings, each frame displays an image from an electron microscope. However, this method yields a vast number of electron images; therefore, analyzing these images to monitor the nanoparticles' growth is a challenging task. The objective of this dissertation is to develop an automation process to capture the complex growth event of nanoparticles in a sequence of electron microscope images. The automation process consists of two tasks: detect nanoparticles in an electron microscope image that has a non-uniform background and significant noise; and then track these detected nanoparticles in a large number of video frames obtained from a single camera. In each frame, complex interaction among these nanoparticles exists; therefore, the tracking algorithm will capture the complex interaction among these nanoparticles. Two solutions are proposed in this dissertation. To detect nanoparticles, an electron microscope image is converted to a binary image through a process called image binarization. To perform the image binarization step, the background of the electron microscope image is first estimated with a robust regression technique; then, it is subtracted from the input image. Afterwards, a global thresholding algorithm is applied to the subtracted outcome in order to achieve the binary image. To track these detected nanoparticles in a large number of video frames, an online algorithm has been created. This algorithm leverages the multi-way data association, which is capable of tracking complex interaction among nanoparticles but suffers from computational inefficiency for a large number of video frames. The online algorithm forms fragmented trajectories between two consecutive frames (i.e. frame-by-frame data association). When missed-association between nanoparticles occur, the algorithm augments these missed-associated nanopartiles to nanoparticles in the second frame in the frame-by-frame data association step. Then, the algorithm continues forming trajectories with the multi-way data association for the incoming video frame. When these augmented nanoparticles are associated within the sliding window, the algorithm initiates the creation of tracks, which connect missed-associated nanoparticles at their respective time frames to their correspondents at the incoming video frame. While working on the second solution, we also created a computer simulation model to generate multi-target datasets with their respective ground-truth associations.The generated datasets and their respective ground-truth associations will serve as a benchmark data to test and evaluate multi-target tracking algorithms. The simulation model serves two purposes: cover all complexity of multi-target tracking scenarios, which public datasets lack; and provide the ground-truth target tracking and association so that the evaluation of multi-target tracking algorithms can be performed without any manual video annotation process.