Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a Notebook, requirements.txt, and other. #19

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,4 +162,7 @@ cython_debug/
.idea/

# From inference.py
video_output_*.mp4
video_output_*.mp4

# The Folder of the Model Checkpoint.
/model_ckpt
142 changes: 142 additions & 0 deletions experiments.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"https://colab.research.google.com/github/LuthandoMaqondo/LTX-Video/blob/luthando-contribution/experiments.ipynb\" target=\"_blank\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Install The Requirements"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!git clone https://github.com/LuthandoMaqondo/LTX-Video.git\n",
"%cd /content/LTX-Video\n",
"!git checkout luthando-contribution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -r requirements.txt\n",
"# # or\n",
"# !pip install -r requirements.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Download the model Checkpoint: from Huggingface Hub"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/luthando/miniconda3/envs/ltx-video/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n",
"/home/luthando/miniconda3/envs/ltx-video/lib/python3.13/site-packages/huggingface_hub/file_download.py:834: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.\n",
"For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.\n",
" warnings.warn(\n",
"Fetching 26 files: 100%|██████████| 26/26 [26:52<00:00, 62.03s/it] \n"
]
},
{
"data": {
"text/plain": [
"'/home/luthando/Desktop/LTX-Video/model_ckpt'"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from huggingface_hub import snapshot_download\n",
"\n",
"model_path = 'model_ckpt' # The local directory to save downloaded checkpoint\n",
"snapshot_download(\"Lightricks/LTX-Video\", local_dir=model_path, local_dir_use_symlinks=False, repo_type='model')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Running Inference"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running generation with arguments: Namespace(ckpt_dir='model_ckpt', input_video_path=None, input_image_path=None, output_path=None, seed=0, num_inference_steps=2, num_images_per_prompt=1, guidance_scale=3, height=720, width=1280, num_frames=2, frame_rate=25, bfloat16=False, prompt='Hello', negative_prompt='worst quality, inconsistent motion, blurry, jittery, distorted')\n",
"Padded dimensions: 736x1280x9\n",
"Downloading shards: 100%|██████████████████████| 2/2 [00:00<00:00, 25343.23it/s]\n",
"Loading checkpoint shards: 100%|██████████████████| 2/2 [00:00<00:00, 16.82it/s]\n",
" 0%| | 0/2 [00:00<?, ?it/s]/home/luthando/miniconda3/envs/ltx-video/lib/python3.13/site-packages/torch/functional.py:534: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /home/conda/feedstock_root/build_artifacts/libtorch_1730824440083/work/aten/src/ATen/native/TensorShape.cpp:3595.)\n",
" return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]\n",
"100%|███████████████████████████████████████████| 2/2 [46:52<00:00, 1406.33s/it]\n",
"/home/luthando/Desktop/LTX-Video/ltx_video/pipelines/pipeline_ltx_video.py:1073: FutureWarning: Accessing config attribute `in_channels` directly via 'Transformer3DModel' object attribute is deprecated. Please access 'in_channels' over 'Transformer3DModel's config object instead, e.g. 'unet.config.in_channels'.\n",
" out_channels=self.transformer.in_channels\n"
]
}
],
"source": [
"!python inference.py --ckpt_dir 'model_ckpt' \\\n",
" --prompt \"A black woman with long black hair smiles at another woman with long blonde hair. The woman with black hair wears a black jacket and has a small, barely noticeable mole on her right cheek. The camera angle is a close-up, focused on the woman with brown hair's face. The lighting is warm and natural, likely from the setting sun, casting a soft glow on the scene. The scene appears to be real-life footage.\" \\\n",
" --height 320 \\\n",
" --width 640 \\\n",
" --num_frames 124 \\\n",
" --seed 100 \\\n",
" --frame_rate 16 \\\n",
" --num_inference_steps 40"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "ltx-video",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
18 changes: 18 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
torch
torchvision
torchaudio
huggingface_hub
transformers
diffusers
imageio
scikit-learn
einops
sentencepiece
beautifulsoup4
ftfy

tqdm
wandb
onnx
av
shutils