Pika Labs has upgraded the way its image-to-video model works. The AI video platform launched a year ago and is facing increasing competition from upcoming services like Sora from OpenAI, the Chinese Kling AI model and the newly released Luma Labs Dream Machine.
So far Pika Labs has focused on building out a broad platform, including the ability to add sound effects and even lip-synched voices to your creation.
While its video generation capabilities were state-of-the-art when it first launched, other models have made a dent in how it's perceived. In an attempt to combat some of this, ahead of the release of a new generation model, Pika Labs has improved its image understanding.
The company says you can get impressive results by starting with an image and giving it filmmaking-type camera motion instructions. It demonstrated this in its own examples which are also available to remix or reprompt in the Pika Labs web app.
Putting Pika Labs image-to-video to the test
This image-to-video model update is wild. pic.twitter.com/L7117cMnNOJune 11, 2024
My first experience trying the updated image-to-video came through some landscape/drone-type shots I generated. They made me sit up and pay attention as the motion was significantly better than previous videos I’d created. So, I decided to push things further.
I created five images using Leonardo's new Phoenix model, then fed those with a description of the required motion into Pika Labs. All settings are left on default, including motion amount.
1. Cyberpunk Carnival
First up we enter the cyberpunk world with a carnival happening at night. Lots of dazzling lights and a request to create a walkthrough of this bright world.
The Leonardo Phoenix prompt: "A vibrant cyberpunk carnival at night with neon-lit rides, futuristic food stalls, augmented reality games, and a diverse crowd of people in colourful, eccentric outfits. Holographic performers entertain the crowd amidst the dazzling lights."
Pika Labs motion prompt: “Utilize a dynamic walkthrough motion that weaves through the crowd. Start from the entrance of the carnival, moving past the food stalls and games, and ending at the center stage with the holographic performers.”
Three seconds wasn’t enough to capture the walkthrough properly but it did start the process and yes, the motion is better when you’re descriptive and use an image as a prompt.
2. Bioluminescent Mushroom Forest
Next, we’re going to get acquainted with a fungi — his name is Dave and he’s the life of the party. OK, not that kind of fun guy, but rather a mushroom forest. Here we want a slow, dolly-in motion.
The Leonardo Phoenix prompt: "A dense forest filled with towering bioluminescent mushrooms casting a surreal glow. Exotic plants with luminescent patterns and small, glowing creatures flitting between the foliage create an otherworldly atmosphere."
Pika Labs motion prompt: “Employ a slow dolly-in motion starting from the forest edge, moving deeper into the forest to reveal the larger mushrooms and glowing creatures, highlighting the transition from the forest's edge to its glowing heart.”
The motion was limited but it did what I asked. The problem is three seconds is once again not enough time to test it properly, so for this one only I decided to give it four more seconds using the extend feature. It just gave me more of the same.
3. Sky Gardens of an Ancient Civilization
For the third test, we’re assuming an ancient civilization exists and they’ve created a network of floating islands, each connected by hanging bridges. The motion is a flythrough starting below the islands and moving up.
The Leonardo Phoenix prompt: "Floating islands connected by hanging bridges, each island lush with vibrant, exotic plants and ancient stone structures adorned with intricate carvings. Waterfalls cascade from the edges, disappearing into the clouds below."
Pika Labs motion prompt: “Use a smooth flythrough motion that starts from below the floating islands, ascending towards them and navigating through the hanging bridges and waterfalls, ending at the central, most intricate structure.”
Again the problem was too slow motion, not long enough to get a true experience so I made some tweaks and changed it from smooth to rapid and broke my "default settings only" rule to increase the strength of motion to 3.
What you see here is that version. Increasing the motions strength made a big difference and changing from smooth to rapid also helped.
4. Deserted Time Capsule City
Going back to nature was the other name for this prompt. Basically, it's a world reclaimed by nature after being abandoned. The motion was a fast dolly-in. I'd learned my lesson from the previous prompts to avoid the word slow.
The Leonardo Phoenix prompt: "A futuristic city frozen in time, abandoned and overgrown with nature reclaiming the structures. Vines and flowers cover sleek skyscrapers, and wildlife roams the empty streets, with remnants of advanced technology scattered around."
Pika Labs motion prompt: “fast dolly-in motion. Start from the outskirts of the city, gradually moving into the heart of the city, showcasing the blend of nature and abandoned technology, ending at a central square overtaken by a large tree.”
For this I also kept the motion strength at three as I didn't want it to be too subtle. It didn't work as I got zero motion. So I removed everything after fast dolly-in motion from the motion prompt. I still got no motion beyond a small dog so I decided it was time to dig into the settings.
This time I increased the adherence to prompt and ticked the zoom-in motion button in the settings. That solved the problem and gave impressive motion.
5. Mystical Ice Caverns
Finally we're going into ice caverns where walls are shimmering with crystals and surrounded by frozen waterfalls. Wasn't blown away by the image but we've started on this route so we'll continue to the end. The motion was a walkthrough.
The Leonardo Phoenix prompt: "A labyrinth of ice caverns with walls that shimmer with embedded crystals, casting colourful reflections. Frozen waterfalls, glistening stalactites, and an ethereal glow from hidden sources of light create a magical atmosphere."
I went really simple with the Pika Labs motion prompt: “A smooth walkthrough motion.” I also ticked the zoom and tilt buttons and kept motion at 3.
This works. So my main takeaway is that yes, adding an image does help improve the video output and make liberal use of the settings that come with Pika Labs.
More from Tom's Guide
- Apple is bringing iPhone Mirroring to macOS Sequoia — here’s what we know
- iOS 18 supported devices: Here are all the compatible iPhones
- Apple Intelligence unveiled — all the new AI features coming to iOS 18, iPadOS 18 and macOS Sequoia
