Alibaba has introduced the Wan2.6 series, showcasing the latest advancements in its visual generation models. This development allows creators to feature in AI-generated videos while maintaining their own appearance and voice.
Enhanced storytelling capabilities facilitate the creation of professional-grade content with possibilities for multi-person dialogue and lengthier narratives.
Reference-to-video model
Integral to the Wan2.6 series is a new reference-to-video generation model, alongside upgrades to its pre-existing models. The Wan2.6-R2V version enables the use of character reference videos to generate scenes featuring the original character’s look and voice, thanks to text prompts.
This technology can incorporate a range of subjects—people, animals, or objects—into videos while preserving their unique characteristics throughout different scenarios. Users can create vibrant new scenes starring the referenced subjects.
Multimodal capabilities in China
Wan2.6-R2V is notable as China’s first reference-to-video model powered by multimodal capabilities
The Wan2.6-R2V is notable as China’s first reference-to-video model powered by multimodal capabilities. It allows for consistent visual and audio integration, offering an innovative tool for creators of short-form dramas by simplifying the production workflow.
In addition, the series extends improvements to its text-to-video (Wan2.6-T2V), image-to-video (Wan2.6-I2V), and various image generation models, enhancing their functionality.
Storytelling and visual consistency
The models' introduction of intelligent multi-shot storytelling offers richer narratives with visual consistency. Enhancements in audio-visual synchronisation enable authentic scene production, with audio and video seamlessly aligned for realistic outputs. Content developers benefit from video outputs of up to 15 seconds, allowing more narrative depth.
These models also support creators in producing cinematic-style content due to improved instructional precision and visual quality.
Advancements in image generation
Enhanced control over artistic styles allows for the creation of highly accurate and realistic portraits
The Wan2.6 series facilitates interleaved text-image outputs for coherent storytelling, powered by advanced logical reasoning. Enhanced control over artistic styles allows for the creation of highly accurate and realistic portraits.
The models further enhance the interpretation of extensive Chinese and English text prompts, enabling creators to craft high-quality visual narratives that accurately convey nuanced artistic intentions.
Access and deployment
Creators can access the Wan2.6 models via Model Studio on Alibaba Cloud's AI development platform and the Wan official website. Additionally, the models will become part of Qwen App, Alibaba's leading AI application.
Initially launched earlier this year, the ongoing refinements of the Wan series demonstrate Alibaba's continuing advances in AI-driven multimedia technology innovation.
Discover how AI, biometrics, and analytics are transforming casino security
Alibaba now unveiled the latest evolution of its visual generation models, the Wan2.6 series.
It enables creators to appear in AI-generated videos as themselves and in their own voices with flexible multi-shot storytelling – new features designed to unlock creative possibilities for professional-grade content production with enhanced multi-person dialogue and extended duration for richer narratives.
new reference-to-video generation model
The Wan2.6 series features a new reference-to-video generation model as well as comprehensive upgrades to its four existing models. Wan2.6-R2V enables users to upload a character reference video with both appearance and voice, utilising text prompts to generate vivid new scenes starring that same character.
Users can create videos featuring a person, animal or object, or even multiple subjects together, while preserving the distinctive look and sound of the original reference.
Multimodal reference generation capabilities
Powered by multimodal reference generation capabilities, Wan2.6-R2V is China’s first reference-to-video generation model that makes it possible for users to insert themselves or other subjects into AI-generated scenes with consistent visuals and audio. This changes the way short-form drama creators tell stories and streamlines their production process.
The Wan2.6 series also includes enhancements to its text-to-video model (Wan2.6-T2V), its image-to-video model (Wan2.6-I2V), and to its two image generation models (Wan2.6-image and Wan2.6-T2I).
Intelligent multi-shot storytelling capabilities
The new models introduce intelligent multi-shot storytelling capabilities that allow for richer, more expressive narratives with visual consistency throughout. Its improved capabilities in audio-visual synchronisation and audio-to-video generation deliver more realistic scenes with richer sound effects.
Supporting video outputs of up to 15 seconds, the models give creators more room to develop their stories. Combined with enhanced instruction-following precision and improved visual quality, they enable creators to produce cinematic-style content with professional-grade results.
Advanced logical reasoning capabilities
For image generation, the Wan2.6 series enables users to create interleaved text-image output with advanced logical reasoning capabilities, to further support coherent visual storytelling. It also demonstrates outstanding capabilities in precise artistic style control, generating realistic portraits with remarkable fidelity and image editing.
Advanced understanding of lengthy Chinese and English text prompts enables creators to produce high-quality, expressive visual content that captures nuance and artistic intent.
Alibaba Cloud's AI development platform
Users can access and deploy the models through Model Studio - Alibaba Cloud's AI development platform - and Wan’s official website. The models will also be integrated into Qwen App, Alibaba's flagship AI application.
First unveiled earlier this year, the Wan series has undergone continuous upgrades, reflecting Alibaba’s leadership and innovation in AI-driven multimedia technologies.
