AI CHECK #5: runwayml.com to create video

By Dan Slee on August 23, 2024

This week I'm taking a look at a commercial generative AI tool that's been named as one of the most influential 100 companies in the world.

Runway was founded six years ago and has been used by Hollywood filmmakers as well as music videos.

So far so fancy, but how can it be used for everyday filmmaking?

Creating video

What may have begun as a text to video product is now very firmly asking the user to upload a prompt image for it to base its work on. But unlike Pika when I looked at it you're able to ask it to produce 10 seconds of footage instead of five seconds.

So, here is a picture of a British amusement arcade.

The image is creative commons. Credit to Paul Glazzard / Teddy's Amusements, Withernsea.

Helpfully, the platform makes some suggestions for the prompt. Firstly, what the camera may do but also what happens in the picture.

I used this as a prompt.

[Camera pans slowly from left to right] [We can see movement from the amusement arcade players as they load money into the slot machines or move around] [the neon lights blink on and off] [lens flare].

It created this.

Handy, but with a few glitches. The lady on the left appears to leave her foot behind.

Can it do better? I tried again.

[Camera stays static] [We can see movement from the amusement arcade players as they load money into the slot machines or move around] [the neon lights blink rapidly on and off] [lens flare].

Again, glitches. People blur and melt into each other. But the overall shot of moving into the image is still amazing. Would using those people be problematic for a filmmaker? Absolutely. GDPR would need to be sought.

Lip synching

Firstly, I tried using a creative commons image of Margaret Thatcher but got closed down as it closely resembled a public figure.

Drat and also wow.

So I uploaded stock picture from istock I have a licence for.

Here's the picture.

And here's the lipsynched video with some existing audio of Taylor Swift I also uploaded as an mp3 file.

Ten for effort. At a glance its plausible but an old face with a young mouth doesn't cut it.

How about a recognisable face?

Well, here's Mrs Slee. She's recognisable to me but probably not to you.

Frankly, instead of the lovely person I married the interpretation looks more like Barry the bricklayer who has been punched in the face. Anyone who knows her would spot the fraud a mile off.

In summary

Like Pika, this may be useful for turning stock pictures into cutaways to add a voiceover too. Maybe there's also room for lip synching an old image with some period text but only if it is marked up as AI.

There's no question that experimentation is going to be of benefit but you'll burn through the free trial pretty quickly. The standard package - which is the cheapest - will get you a minute of footage a month for £105 a year.

For Pika versus Runway? Runway wins.

There's a lot of YouTube tutorials to play with and extra tools but this this an exensive business right now and it will cost you to refine what you are doing.