Wharton's Ethan Mollick demonstrates Gemini Omni's native video editing by modifying the 1896 film Arrival of a Train

Not bad reply to a pretty open-ended prompt: "this is the original train movie that supposedly terrified audiences. edit it so that it is as scary to modern audiences as the original was to its audience at its time"

I love this.

I think people don't realize why Gemini Omni is different than other video AIs. It is fully multimodal, so it can edit video natively, too

I took the famous "train " movie from 1896 & made it a bullet train, LEGO, added a time traveler, a centipede, muppets... (see reflections?)

@emollick I've been generating hours of videos with various models, local and proprietary and despite all the amazing things out there, the video models simply lack consistency and instruction following. They look amazing but it's like trying to train a cat to follow exact instructions.

native video editing through conversation is the feature that separates it from every other video AI.

generating is one thing. editing existing footage natively without re-rendering from scratch is a completely different technical problem.

the reflections detail in a LEGO conversion of an 1896 film is the kind of consistency that usually breaks everything.

if that's running reliably Gemini Omni just became a post-production tool, not just a generation tool.

@emollick This is really cool!

@emollick 10sec makes it pretty limited

@emollick Very, very cool.

@emollick Yeah, people are comparing it to Seeddance fight choreography, but real mulitmodal what's special, for editing, and giving Flow video to video abilities.

I placed my wife's favorite stuffed animal in Apollo footage, and did style swap tricks.

https://youtu.be/PTr8qJmjfig?si=rtdSH6k6qjKHXQ2X&t=140

@emollick And did you check with the Lumiere Brothers first to see if they were OK with you doing this? (The film was called L’Arrivée d’un train en gare de La Ciotat — "Arrival of a Train at La Ciotat Station"

@emollick Pretty cool! I wonder if it could also say restore it to “iMax quality”?

@emollick Where do you use?

@emollick the minute you push this model it falls apart. its only good at following simple commands..

@emollick “The film is the first art form capable of demonstrating how matter plays tricks on man.” - Walter Benjamin

Gotta have these guys do the VO

@emollick I asked it to generate a video of a cricket batsmen (change it from cover drive to bowled) and it keeps saying it violates their policies - Its heavily censored - usually they start out less censored so I have zero hope.

@rileybrown @emollick I was told it's available in Google Flow.

@emollick Seedance 2 can edit videos too, though :)

This is interesting from a research point of view and underwhelming as a consumer app. People are comparing the apples to apples comparison ( fight scenes) where seed dance clowns on it. The apples to oranges comparison doesn’t exist because we still don’t know what the orange is.

@emollick Nice!

Maybe this will finally get my AI-loathing (but train- and Lego-loving) 16yo to accept how I pay our bills. 😌 Here he is in brick form with my 1985 Lego locomotive 🚂:

@emollick Gemini Omni's native video editing capability fundamentally shifts the workflow from frame-based generation to continuous temporal manipulation.