Photo by Uriel Mont on Pexels

Alibaba has unveiled Wan2.1-VACE, a new open-source AI model poised to revolutionize video editing. This comprehensive tool provides a unified solution for a range of video generation and editing tasks, lowering the barrier to entry for creators of all levels.

Wan2.1-VACE allows users to generate videos from text prompts, images, and existing video content. Its robust editing toolkit boasts image referencing capabilities, advanced video repainting, selective adjustments, and video stretching functionality. Moreover, the AI model facilitates pose transfer, motion control, depth adjustment, and color manipulation within videos.

A standout feature is VACE’s precision in selectively modifying or removing specific areas within a video without impacting the surrounding content. Alibaba highlights the model’s ability to seamlessly expand images into widescreen video formats.

At the heart of VACE is a Video Condition Unit (VCU) which processes diverse multimodal inputs and a Context Adapter structure designed to deeply understand the temporal and spatial elements within videos. Alibaba anticipates VACE will find applications in diverse areas, including social media content creation, advertising, post-production workflows, and educational video development.

By making Wan2.1-VACE open-source, Alibaba aims to democratize access to cutting-edge AI technology, particularly benefiting smaller businesses and individual creators. Two versions are available: a 14-billion parameter model and a smaller, more accessible 1.3-billion parameter version, both hosted on Hugging Face, GitHub, and Alibaba Cloud’s ModelScope.