Image-Synthesis-With-Generative-Adversarial-Networks-GANs-Using-CIFAR-10

CS-GY 6953 / ECE-GY 7123 Deep Learning || Project || Spring 2024
New York University (NYU) Tandon School of Engineering

Abstract

This project thoroughly studies image synthesis with Generative Adversarial Networks (GANs) using CIFAR-10 dataset as the benchmark. GAN architecture trained with a discriminator and generator together, augmented with Wasserstein loss and gradient penalty (WGAN-GP) for improved stability and convergence. Fréchet Inception Distance (FID) and Inception Score (IS) as evaluation metrics, indicating the quality and diversity of generated images. With an FID score of 12.06 and an IS score of 1.0, the generated images closely resemble real CIFAR-10 samples. This project underscores the proficiency of GANs in image synthesis, mainly in conjunction with WGAN-GP. Our findings contribute to advancing and providing valuable insights for future research in this domain.

Please go through the documents folder for more info!

Model Architecture

GANs consist of two neural networks, a discriminator and a generator which are trained simultaneously through a mini-max game network:

• Generator: Generate photorealistic images indistinguishable from real images. It has convolutional layers, batch normalization, ReLU activation functions, and tanh activation function in the output layer.

• Discriminator: Distinguish between real and fake images generated by the generator. It is similar to generator architecture with leaky ReLU activation, and sigmoid activation in the output layer.

Please go through the project report in documents folder for more info!

Dataset

The CIFAR-10 dataset contains 60000 32x32 color images. The dataset has 10 classes, each having 6000 images and these classes are completely mutually exclusive. Training a GAN on CIFAR-10 requires the generator to learn patterns and textures to generate realistic images without demanding a lot of compute resources, making it a suitable benchmark for evaluating the capabilities of GAN architectures for the project.

Performance Evaluation

Fréchet Inception Distance [FID]

Measures the similarity between the distributions of features extracted from real images and generated images.
Lower score indicates that the generated images are more similar to the real images in terms of their visual features.

Inception Score [IS]

Measures the quality and diversity of generated images based on the output probabilities of an image classification model (Pre-trained ResNet18 Model in our case).
Higher score indicates that the generated images are both diverse and of high quality.

Results

Generative Adversarial Network (GAN) model have performed well. The negative values of the discriminator loss indicate that the discriminator is able to distinguish between real and fake images effectively. The generator loss values indicate the ability of the generator to produce images that are realistic enough to deceive the discriminator. Loss values stabilize towards the later epochs, indicating that the model has reached a stable state.

Discriminator and Generator Training Loss Plot

Please check DLProjectNotebook.ipynb file to view how we trained this model.

Fréchet Inception Distance

FID score: 12.056572421708601

Low distance between the feature distributions of real and generated images, suggests that the generated images are relatively close to the real images in terms of their visual features.

Inception Score

IS score: 1.0

Generated images are of good quality and exhibit a high degree of diversity in terms of their content and appearance.

Please check ModelEvaluation.ipynb file to view how we calculated these scores.

Conclusion

Based on these evaluation metrics and inspection of generated images, GAN model has performed well, generating good-quality and diverse images that relatively closely resemble the real ones from CIFAR-10 dataset.

GAN Generated Image

References

  1. Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. click here
  2. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio. (2014). Generative Adversarial Networks. click here
  3. T.Karras,T.Aila,S.Laine,andJ.Lehtinen,Progressive Growing of GANs for Improved Quality, Stability, and Variation. 2018. click here
  4. Bhatia, Snehal and Rozenn Dahyot. “Using WGAN for Improving Imbalanced Classification Performance.” Irish Conference on Artificial Intelligence and Cognitive Science (2019). click here
  5. Y. Chong, J. Lee, D. E. Carlson, and A. W. Rivenson, ”Effectively Unbiased FID and Inception Score and Where to Find Them,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. click here
  6. A. Radford, L. Metz, and S. Chintala, ”Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv:1512.03385, 2015. click here
  7. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, ”Generative adversarial nets,” arXiv:1406.2661, 2014. click here

Team Members

  1. Durga Avinash Kodavalla dk4852
  2. Priyangshu Pal pp2833
  3. Ark Pandey ap8652