Adobe Firefly is about to make its biggest leap in AI video yet with a new model and Veo 3 integration

1 2 minutes read

Adobe release new Firefly video generation model that’s better than ever
The Firefly Web App now supports non-Adobe video generation models like Veo 3
A new Generate Sound Effects beta app makes it easy to add sound effects to your videos

Adobe Firefly has given its AI video generation capabilities a timely upgrade. It has upgraded its video model to version 1.9, which brings more realism and precision in storytelling. It’s available in the Firefly Web App right now.

Adobe cites one of the strengths of its new Firefly Video Model as “generating dynamic landscapes from natural vistas to urban environments. The model also demonstrates remarkable capability with animal motion and behavior, atmospheric elements like weather patterns and particle effects, and mastering both 2D and 3D animation.”

You can see this demonstrated in this example video of a cinematic drone shot going between the trees of a snowy forest at sunset golden hour:

Adobe has also partnered with other generative video models, so you can now select Veo 3, Luma, Runway, and Topaz all from within the Firefly Web app.

Generate Sound Effects (beta)

As part of the new Firefly, there’s also a new beta version of Generate Sound Effects, for creating custom, high-quality audio from text prompts or voice cues.

It involves you making voice sounds that are close to the sort of sound effect you want in your video, and the AI then works out what sort of sound you really want to add. So, in a beach scene, if you start making “Kaaw! Kaaw!” noises, the AI works out that you want seagull sounds.

I’ve had a go at using the voice cues method, and it certainly made everybody in the office think I had gone mad! Here’s an example of how it works:

More video enhancements

Using Firefly, you can create AI-generated video from either a text prompt or a reference image, but you can now also upload a reference video, and Firefly will generate a new video that transfers the original composition to your generation. The new video model has a new level of precision control when you’re directing video content.

There are also several style presets available, allowing you to apply a distinct visual style with a single click. Presets available include claymation, anime, line art, and 2D.

Finally, there’s keyframe cropping. You can upload your first and last frames, select how your image will be cropped, and describe the scene, and Firefly will generate a video that fits the format.

Source link