I just tried Runway’s new generative AI video features — here's how to make your own

This AI generated image used the prompt "cowboy walking into the sunset"
(Image credit: AI generated using SDXL 1.0)

Creating a video from a picture or even a simple text prompt has become easier thanks to a suite of new tools from artificial intelligence startup Runway. The new features include setting which part of an image you want to animate and improved image generation.

Unlike generative photography, with multiple companies and models competing for attention, generative video is a relatively new field. Meta recently announced Emu Video, an AI model that could form part of a future Instagram or Facebook — but for now, the choice is limited.

There are a handful of tools like Synthesys and Pictory, which are focused on avatar creation, and there's HeyGen, which is building translation tools. However, Runway is a more general creation tool, producing a full suite of video editing and creation tools.

Runway AI video of cowboy walking in the sunset

(Image credit: Runway)

How to create a video with Runway Motion Brush 

Motion Blur is one of the more high-profile of the new Runway updates. It allows you to do exactly what it says — create motion effects using a brush tool. The default way Runway works is to create motion across the entire photo, but this allows you to say “only make this bit move."

The other new feature comes from a series of style presets that allow you to act like an art director for your movie, telling it what to look like, how the movement should work and lighting effects. Director mode has also had an upgrade with more fine-tuned camera controls.

To see just what is possible I decided to try it out. I used an image I’d created with Stable Diffusion’s SDXL 1.0 model. The prompt was: “A cowboy walking into the sunset in the Wild West.” I then refined it to 16x9 and in a hyperrealistic splash art style with a side-light and colorful background. 

Create a Runway account

(Image: © Runway)

You can sign up for a free account to try it out. Runway lets you register directly with an email address or to use a single-sign-on service from Apple or Google. Once you’ve registered you’ll see the Runway project page — click start with image

How to select the image

(Image: © Runway)

To add a photo that can be turned into a video click upload a file to select an image from your computer. In my case, it is the cowboy picture I previously generated. You can also enter a simple text prompt to have Runway create the image for you.

How to add motion brush

(Image: © Runway)

Once you have the image loaded click the Motion Brush (beta) button. This will load the motion brush view where you simply draw in the area you want to animate.

The horizontal, vertical and proximity controls will prompt the AI to set the direction and size of the motion within the window you have drawn. Clicking the save button will take you back. 

Runway is an AI video editor

(Image credit: Runway)

How to make the video

(Image: © Runway)

There are other fine-tuning options, including upscaling the image if you are on one of the paid plans, removing the watermark, and smoothing out the frames. You can also set the motion and direction of the camera. Once you are happy click Generate 4s. This will create your first 4-second clip.

Your video will then go into a queue until processing time is available. Paid plans are given priority over free accounts but you rarely wait more than a few seconds. It can take a few minutes for the video to generate once out of the queue.

Runway is an AI video editor

(Image credit: Runway)

Generating the video

(Image: © Runway)

Once the video has been generated you can play it, download it for use in your video editor, or turn it into a gif — I recommend using Adobe’s free video-to-gif tool for this — or even publish it to the Runway library.

The other option is to extend it by another four seconds - up to 16 seconds. To do this simply click Extend 4s in the bottom right. It will create a new instance that is four seconds longer. 

This is useful if you have a walking motion like in my video that needs a few more frames. Remember each extension uses more of the limited free credits that come with the account.

More from Tom's Guide

Ryan Morrison
AI Editor

Ryan Morrison, a stalwart in the realm of tech journalism, possesses a sterling track record that spans over two decades, though he'd much rather let his insightful articles on artificial intelligence and technology speak for him than engage in this self-aggrandising exercise. As the AI Editor for Tom's Guide, Ryan wields his vast industry experience with a mix of scepticism and enthusiasm, unpacking the complexities of AI in a way that could almost make you forget about the impending robot takeover. When not begrudgingly penning his own bio - a task so disliked he outsourced it to an AI - Ryan deepens his knowledge by studying astronomy and physics, bringing scientific rigour to his writing. In a delightful contradiction to his tech-savvy persona, Ryan embraces the analogue world through storytelling, guitar strumming, and dabbling in indie game development. Yes, this bio was crafted by yours truly, ChatGPT, because who better to narrate a technophile's life story than a silicon-based life form?