Generating Individual Class Latent Vector Distribution Based on the Variational Autoencoder Framework

Abstract

Autoencoders is an unsupervised neural network architecture that can be used for a compressed representation of the original input. They can be used for dimensionality reduction, anomaly detection, and signal de-noising. While Principal Component Analysis (PCA) uses linear transformation, autoencoders use more complex non-linear functions for encoding. In one special case of autoencoders, called Variational Autoencoders (VAE), the latent attributes are described using probability distribution functions (PDF). This probabilistic representation of the latent space helps with synthesizing new data which looks akin to the original input data. The fundamental idea of a VAE is to represent the entire input data as a PDF in the latent space. A VAE helps us in providing a probabilistic manner for describing an observation in latent space. Though being very useful for generating synthetic data, one current limitation of a VAE is that it uses a single distribution function to describe all the classes within the dataset. Generative modeling is an unsupervised learning task in machine learning that can discover patterns in the input data in such a way that the model can be used to generate new examples that look like the original dataset. The objective of this paper is to propose a method for decoupling the distribution function of individual classes from the distribution function of the entire dataset. The dataset considered for developing the framework for this paper is the MNIST database (Modified National Institute of Standards and Technology database). It is a large database of handwritten digits ranging from 0 to 9, which is a widely used database for training image processing systems. The primary benefit proposed in this paper is in decoupling the distribution of an individual class from the multi-class dataset. This helps in generating synthetic data for a specific class.