Research at NVIDIA Generating and Editing High-Resolution Synthetic Images with GANs

Video Transcript:

Rendering photorealistic images using a standard graphics pipeline is complex. What if we could render images with a generative network instead? Starting with high-level labels that are much easier to create and edit. We present a generative adversarial framework for synthesizing 2K photorealistic images. To show the advantage of this approach, we provide an interactive interface for users to quickly edit objects in the scene. Let's try out our tools. We can select different colors for the car or choose diverse textures for the road. We can also remove objects by replacing the existing labels with different ones, for example, replacing the trees with buildings or cars with roads. Moreover, we can add new objects to the scene, like inserting a new car at a particular location with a desired object size. Finally, we can just paint on the label map to create novel scenes, like adding more trees into the image. Note that these shapes, colors, and textures are completely learned from the models without having any human intervention or handcrafted filters. Maybe the street view is a little boring. Let's do some portrait editing at 1K resolution. Here we can modify the appearance of each facial part, for example, choosing different skin colors, or selecting different shapes or eye colors. Again, the shapes and textures are completely learned with no human intervention. We can also modify the original label map to create different effects, what happens if we make the eyes bigger. In this example, we can choose different skin colors or lighting conditions for the photo. We can modify the face by changing the eyebrows' styles, adding more or less hair. How about adding a mustache to the face? Or a beard along with a mustache? Thank you. Please find our code and models on our website.