A Guide to AI-Powered Video Synthesis: Transforming Videos with EbSynth

Have you ever wanted to create a video with a unique, stylized look? If so, you might be interested in EbSynth, an open-source tool that uses machine learning to synthesize video from a single source image. In this guide, we’ll explore the background and functionality of EbSynth, and provide a step-by-step tutorial to help you get started with this powerful tool.

EbSynth was developed by a team of researchers at Disney Research in Zurich, Switzerland. The project was led by Oliver Wang, a computer graphics researcher who had previously worked on Disney’s animated feature films. The goal of the project was to create a tool that could be used to stylize video in new and interesting ways, using the power of machine learning.

How EbSynth Works

EbSynth uses a machine learning technique called neural style transfer to synthesize new frames of video from a single source image. Neural style transfer involves training a neural network to learn the visual style of a particular image or set of images, and then using that network to transfer that style onto other images or video frames.

To use EbSynth, you first need to provide the tool with a source image that represents the style you want to apply to your video. This can be any image you like, such as a painting, a photograph, or a still frame from a movie. Once you’ve provided the source image, EbSynth uses a pre-trained neural network to analyze the image and extract its visual style.

Once EbSynth has analyzed the source image, you can use the tool to synthesize new frames of video that match the style of the source image. To do this, you simply provide EbSynth with a video file, and it will automatically generate new frames that match the style of the source image. You can adjust various settings to control the output of the tool, such as the number of frames to generate, the speed of the output video, and the resolution of the output frames.

Creating A Short Video Mood for a Fashion Film

1. Choose a source video as a base. The most important things are movement and general shapes, the visual style can be changed.
2. Use video-editing software like AfterEffects or PremierePro to change the aspect ratio and composition of the source video. To fill in empty space, I like to use a colored checker pattern. This makes it easier for EbSynth to »grab on« to the source video so there’s fewer moving artifacts when synthesizing.
3. Then export the video file into its single .png frames.
4. Choose the first frame as the input and generate an image that represents the visual style you want to apply to your video. This could be any style you like, such as a painting, a photograph, or a still frame from a movie. In this case, the outfits and the settings are transformed via img2img generation and inpainting. The output will we your »Keyframe«.
5. Open up EbSynth and select the frames of your original video in the »Video« section. Select your AI augmented keyframe(s) in the »Keyframe« section. The filenames have to match! EbSynth will automatically analyze the sources and suggest from which start frame to which end frame you would like to synth. You can adjust these settings.
6. Click on the »Synth« button to start the synthesis process. EbSynth will analyze your source image and generate new frames of video that match its visual style.
7. Use video-editing software once again to transform all the single frames easily back into a video file!

EbSynth: Video to Video Synthesis

EbSynth is a powerful tool for creating stylized video using the power of machine learning. By providing the tool with a single source image, you can generate new frames of video that match its visual style, allowing you to create videos with a unique and eye-catching look. While there is a learning curve to using EbSynth effectively, the results can be stunning and well worth the effort.

Whether you’re a filmmaker looking to add a unique visual flair to your work, or just a creative person looking to experiment with AI-powered video synthesis, EbSynth is a tool that is well worth exploring. With its powerful machine learning algorithms and intuitive interface, this tool is a great way to unlock new possibilities for video creation and exploration.

So, what are you waiting for? Download EbSynth today and start exploring the exciting world of AI-powered video synthesis!