Stay updated with our daily and weekly newsletters for the latest news and exclusive content on leading AI developments. Learn More
If you follow AI influencers or creators on social media, you’ve probably noticed they’re buzzing about a new AI video generation model called “Kling.”
This model can create incredibly realistic videos from text prompts, using in-app buttons and settings. These videos are comparable to those made by OpenAI’s exclusive, invitation-only beta model, Sora, which is currently being tested by a select group of artists and filmmakers for potentially risky or objectionable uses.
Recently, Kling shared a video on its YouTube channel mimicking one of the first third-party videos created with Sora, titled “air head” by the creative agency shy kids. Kling’s video, “a day with the Balloon Man,” is embedded below, followed by “air head” by shy kids with OpenAI’s Sora.
But what is Kling, where did it come from, and how can you use it? Let’s dive in.
What is Kling and where did it originate?
According to the South China Morning Post (SCMP), Kling was developed by Kuaishou Technology, the creator of Kuaishou – China’s second most popular short video app, known as Kwai outside of China, with 400 million daily active users. This puts Kuaishou behind only Douyin, the Chinese version of TikTok, which has 600 million daily active users.
This new model will likely attract a large number of users in China, encouraging them to try out Kuaishou, enhancing its competition with Douyin.
SCMP reports:
“The Kling AI Model, currently in trial, can turn text into video clips up to 2 minutes long at 1080p resolution, supporting various aspect ratios. It generates videos that replicate real-world scenarios and creates imaginative scenes from text instructions.”
Several sources cited by Perplexity detail that Kling:
“Uses a unique 3D Variational Autoencoder (VAE) for face and body reconstruction, allowing for detailed expression and limb movement from a single full-body image. This technology is boosted by a 3D spatiotemporal joint attention mechanism, enabling the model to handle complex scenes and movements, ensuring content adheres to the laws of physics.”
How can you use Kling and what does it cost?
You can access Kling for free through Kuaishou, Kwai, and KwaiCut (a video editing app similar to TikTok’s CapCut). However, you need a Chinese phone number to download and use the model if you’re outside of China.
Venture capital firm a16z partner Justine Moore suggested a workaround using a burner phone number through the KwaiCut app, though she wasn’t sure how advisable this is. U.S. filmmaker Dustin Hollywood also suggested using ChatGPT to translate the app menus and screens.
What is Kling capable of?
Early user videos show Kling can create a wide array of immersive, realistic, and detailed high-resolution videos, from realistic action scenes to first-person shooter video games, and high fantasy series like House of the Dragon or Game of Thrones.
Dustin Hollywood noted it takes about 2 minutes to generate a video from a moderately complex text prompt. However, the model struggles with accurately depicting race/skin color, similar to issues with Google’s Gemini AI image generation, which faced criticism.
Despite these issues, Kling is a powerful new AI tool that has filmmakers like Dustin Hollywood reconsidering their views on Sora and questioning OpenAI’s selective release strategy.
Will the emergence of Kling push U.S.-based AI video model providers like OpenAI, Runway, and Pika to enhance the quality and resolution of their video generations? It’s highly likely. Whether they can quickly match Kling’s capabilities remains to be seen.
If you’re interested in AI filmmaking or filmmaking in general, Kling’s arrival is definitely something exciting. Hopefully, a full U.S. release without the need for Chinese phone numbers is on the horizon.