Intrinsic Properties and Biases of Deep Learning Models

Statistical data models based on the deep learning paradigm have shown remarkable performance in many domains, surpassing human performance in a set of tasks under restricted settings. However, the fundamental reasons enabling these achievements have not been well understood. The understanding of the underlying mechanisms of modern deep learning models is paramount for the following reasons. First, we would like to properly use the models for downstream tasks; for this purpose we need to identify and understand the failure modes of the models. Second, we would like to identify and quantify the biases the models exhibit to facilitate the fair use of the models when protected groups of population are involved. However, illuminating the underlying mechanisms is very challenging due to their size and complexity. In this dissertation, we study intrinsic properties of two publicly available deep learning models, BERT in the domain of natural language processing, and Glow in the domain of computer vision. Both models have achieved the state of the art for their respective tasks at the time of their publication, and such remarkable performance makes them attractive study subjects. While the natural language processing and computer vision domains are drastically different, common analysis techniques can be applied to both models; the observation also implies that similar techniques could be used in other domains not covered by the current dissertation. In both cases, we analyze the geometric properties of the embeddings and quantify the layer-by-layer transformations through correlations and other analyses. The results lead to novel insights on how the models generate intermediate representations and give rise to their performance, and enable better understanding of the underlying mechanisms.