Stability AI's Stable Video Diffusion XT (SVT-XT) 1.1 foundation model, available on Hugging Face, is a diffusion model that takes in a still image as a conditioning frame and generates a short 4 second video. The notebook walks through configuring, creating, and invoking an Asynchronous Inference Endpoint backed by the SVT-XT foundation model on Amazon SageMaker.
For more information on using this repository, read the associated blog post, Generative AI Videos with Stability AI’s Stable Video Diffusion XT using Asynchronous Inference on Amazon SageMaker.
-
Public URL for Conditioning Image: Version 1 of the Notebook, svd_xt_demo_v1.ipynb, passes a publicly accessible URL of the image in the request payload used to invoke the model. Use the corresponding custom inference script, inference_v1/inference.py.
-
Conditioning Image Passed in Request Payload: Version 2 of the Notebook, svd_xt_demo_v2.ipynb, passes an encoded image in the request payload used to invoke the model. Use the corresponding custom inference script, inference_v2/inference.py.
The project now includes an interactive Streamlit application to generate the videos. Make sure you have first installed the required Python packages, as outlined in the Optional: Local Development Environment section, below. The application can be started with the following command:
streamlit run app.py --server.runOnSave true
All videos created using Notebook included in this project.
Source image |
Source image |
Source image |
Source image |
Source image |
Source image |
Source image generated with Stable Diffusion XL (SDXL) 1.0 |
All videos created using Notebook included in this project.
Source image |
Source image |
Source image |
Source image |
Setup local environment to modify project.
python3 -m pip install virtualenv -Uq
virtualenv svd-venv
python3 -m venv svd-venv
source svd-venv/bin/activatepython3 -m pip install -r local_requirements.txt -Uq- https://github.com/aws-samples/amazon-sagemaker-asynchronous-inference-computer-vision/blob/main/mask-rcnn-async-inference.ipynb
- huggingface/diffusers#6956
- https://github.com/huggingface/notebooks/blob/main/sagemaker/23_stable_diffusion_inference/sagemaker-notebook.ipynb
- https://github.com/philschmid/huggingface-inferentia2-samples/blob/main/stable-diffusion-xl/sagemaker-notebook.ipynb
- https://github.com/Stability-AI/generative-models/blob/main/scripts/sampling/simple_video_sample.py
- https://huggingface.co/docs/diffusers/en/using-diffusers/svd
- https://huggingface.co/docs/sagemaker/inference#create-a-model-artifact-for-deployment
- https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/tree/main
- https://sagemaker.readthedocs.io/en/stable/api/inference/async_inference.html
- https://www.philschmid.de/sagemaker-stable-diffusion
The contents of this repository represent my viewpoints and not of my past or current employers, including Amazon Web Services (AWS). All third-party libraries, modules, plugins, and SDKs are the property of their respective owners.