CycleGAN is a representative work of image conversion. Unlike pix2pix, its sample data needn’t be matched strictly and only two types of domain are enough to realize the transformation. The feature is to first convert the image from one domain to another through a cycle and then, turn back. If both conversion is accurate, the converted image should be basically the same as the input image. Through such a cycle, CycleGAN will transform the image matching before and after the conversion, similar to supervised learning. And it proves to greatly improve the conversion effect.
This project realizes face conversion between men and women through celebA data set, and realizes face detection and real-time conversion in a video through OpenCV.
Figure 1: CycleGAN consists of four parts, two generation models, G (x - > y) from X domain to y domain and G (Y - > x) from y domain to X domain; The two discrimination models, x-domain and y-domain, are responsible for judging whether the images generated by the generated models are realistic enough.
Figure 2: The basic algorithm flow structure.
Figure 3: Network structure diagram. It’s mainly composed of generator module and discriminator module. The least square loss is used to replace the standard Gan loss; Take G as an example, maintain a set of 50 historical false y images. Every time G generates a false y, it adds it to the set, and then randomly takes out a false y from the set, and inputs it to the discriminator together with a true y for discrimination. In this way, the false y set represents the average ability of G to generate y according to x, which makes the training more stable.
The code is built with the following libraries:
- Python 3.7
- Tensorflow 1.15.0
- OpenCV
- Dlib
- OS
- Imageio
- pickle
- Numpy
- Keras
We use the dataset of CelebA as an example for description. We need to first dowload the CelebA dataset and the corresponding label’list_attr_celeba.txt'.
- Extract the dataset to ./data/ntu/nturgb+d_skeletons/
- Process the data set and label under the gender conversion folder
#The images in celeba dataset are processed into 256 * 256 size, and saved to the male and female folders in the data folder according to gender, which includes more than 80000 male images and 120000 female images respectively.
python 性别转换/generate_gender_images.py --data_dir = './data' --male_dir = './data/male' female_dir = 'data/female'
#Using build_ Data. Py converts the image to tfrecords format, where x_ input_ Dir and Y_ input_ Dir is the previously saved male and female file. X_ output_ File and Y_ output_ File are the location where the tfrecords file is placed. It’s placed in the male.tfrecords and female.tfrecords files of the data folder.
python 性别转换/CycleGAN-TensorFlow/bulid_data.py --X_input_dir ‘../data/male/’ --Y_input_dir ‘../data/female/’ --X_output_file ‘../data/male.tfrecords/’ --Y_output_file ‘../data/female.tfrecords/’
Training CycleGAN model with train.py
python 性别转换/CycleGAN-TensorFlow/train.py --X ‘../data/male.tfrecords’ --Y ‘../data/female.tfrecords’
#Using export_ Graph.py packages the model into a. Pb file, and the generated file is in the preconditioned folder.
python 性别转换/CycleGAN-TensorFlow/export_graph.py --checkpoint_dir ../checkpoints/20210624-0231/ --output_graph_def ../pretrained
#By using the model of information.py to process pictures, we can convert single picture to gender.
python 性别转换/CycleGAN-TensorFlow/inference.py--model ../pretrained/male2female.pb --input .jpg --output .jpg
#Through main_ 1. Py to realize the gender conversion of multiple pictures.
python 性别转换/CycleGAN-TensorFlow/main_1.py--model ../pretrained/male2female.pb or../pretrained/female2male.pb
--images ../faces/male/*.jpg or ../faces/female/*.jpg
#Through main_ 4. Realize face detection and gender conversion.
python 性别转换/CycleGAN-TensorFlow/main_4.py--model ../pretrained/male2female.pb and ../pretrained/female2male.pb
This repository holds the code for the following paper:
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks.ICCV, 2017.
@inproceedings{
title={Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks},
author={Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A, Efros, UC Berkeley},
year={2017},
}
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.