VirtualOasis commited on
Commit
37caa62
·
1 Parent(s): 46c3ea1
README.md CHANGED
@@ -8,6 +8,58 @@ sdk_version: 6.0.1
8
  app_file: app.py
9
  pinned: false
10
  short_description: automate the process of short movie creation
 
 
11
  ---
 
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  app_file: app.py
9
  pinned: false
10
  short_description: automate the process of short movie creation
11
+ tags:
12
+ - mcp-in-action-track-creative
13
  ---
14
+ **CineGen AI Director** is an AI agent designed to automate the process of short movie creation. It transforms a simple text or image idea into a fully realized video production by handling scriptwriting, storyboard generation, character design, and video synthesis using a multi-model approach.
15
 
16
+ - **Sponsor Platforms**: Uses Google Gemini (story + character prompts) and Hugging Face Inference Client with fal.ai hosting for Wan 2.2 TI2V video renders;
17
+ - **Autonomous Agent Flow**: StoryGenerator → CharacterDesigner → VideoDirector pipeline runs sequentially inside a single Gradio Blocks app, with MCP-friendly abstractions (`StoryGenerator`, `CharacterDesigner`, `VideoDirector`) designed for tool-call orchestration.
18
+ - **Evaluation Notes**: Covers reasoning (Gemini JSON storyboard spec), planning (scene/character tables that feed downstream steps), and execution (queued video renders with serialized HF jobs).
19
+
20
+ ## Artifacts for Reviewers
21
+
22
+ - **Social Media Proof**: Replace `<SOCIAL_LINK_HERE>` with your live tweet/thread/LinkedIn post so judges can verify community sharing.
23
+ - **Video Recording**: Upload a walkthrough of the Gradio agent (screen + narration) and swap `<DEMO_VIDEO_LINK>` with the shareable link.
24
+
25
+
26
+ ## 🚀 Key Features
27
+
28
+ * **End-to-End Automation**: Converts a single sentence idea into a complete short film (approx. 30s-60s runtime).
29
+ * **Intelligent Storyboarding**: Breaks down concepts into scene-by-scene visual prompts and narrative descriptions.
30
+ * **Character Consistency System**:
31
+ * Automatically identifies main characters.
32
+ * Generates visual reference sheets (Character Anchors).
33
+ * Allows users to "tag" specific characters in specific scenes to ensure visual consistency in the video generation prompt.
34
+ * **Multi-Model Video Generation**: Supports multiple state-of-the-art open-source video models via Hugging Face.
35
+ * **Robust Fallback System**: If the selected video model fails (e.g., server overload), the system automatically tries alternative models until generation succeeds.
36
+ * **Interactive Editing**:
37
+ * Edit visual prompts manually.
38
+ * Add, Insert, or Delete scenes during production.
39
+ * Regenerate specific clips or character looks.
40
+ * **Client-Side Video Merging**: Combines individual generated clips into a single continuous movie file directly in the browser without requiring a backend video processing server.
41
+
42
+
43
+ ## 🤖 AI Models & API Usage
44
+
45
+ The application orchestrates two primary AI services:
46
+
47
+ ### 1. Google Gemini API (`@google/genai`)
48
+ Used for the "Brain" and "Art Department" of the application.
49
+
50
+ * **Logic & Scripting**: `gemini-2.5-flash`
51
+ * **Role**: Analyzes the user's idea, generates the title, creates character profiles, and writes the JSON-structured storyboard with visual prompts.
52
+ * **Technique**: Uses Structured Output (JSON Schema) to ensure the app can parse the story data reliably.
53
+ * **Character Design**: `gemini-2.5-flash-image`
54
+ * **Role**: Generates static reference images for characters based on the script's descriptions.
55
+ * **Role**: Acts as the visual anchor for the user to verify character appearance before video generation.
56
+
57
+ ### 2. Hugging Face Inference API (`@huggingface/inference`)
58
+ Used for the "Production/Camera" department.
59
+
60
+ * **Video Generation Models**:
61
+ * **Wan 2.1 (Wan-AI)**: `Wan-AI/Wan2.1-T2V-14B` (Primary/Default)
62
+ * **LTX Video (Lightricks)**: `Lightricks/LTX-Video-0.9.7-distilled`
63
+ * **Hunyuan Video 1.5**: `tencent/HunyuanVideo-1.5`
64
+ * **CogVideoX**: `THUDM/CogVideoX-5b`
65
+ * **Provider**: Defaults to `fal-ai` via Hugging Face Inference for high-performance GPU access.
cinegen/__pycache__/__init__.cpython-310.pyc CHANGED
Binary files a/cinegen/__pycache__/__init__.cpython-310.pyc and b/cinegen/__pycache__/__init__.cpython-310.pyc differ
 
cinegen/__pycache__/character_engine.cpython-310.pyc CHANGED
Binary files a/cinegen/__pycache__/character_engine.cpython-310.pyc and b/cinegen/__pycache__/character_engine.cpython-310.pyc differ
 
cinegen/__pycache__/models.cpython-310.pyc CHANGED
Binary files a/cinegen/__pycache__/models.cpython-310.pyc and b/cinegen/__pycache__/models.cpython-310.pyc differ
 
cinegen/__pycache__/placeholders.cpython-310.pyc CHANGED
Binary files a/cinegen/__pycache__/placeholders.cpython-310.pyc and b/cinegen/__pycache__/placeholders.cpython-310.pyc differ
 
cinegen/__pycache__/story_engine.cpython-310.pyc CHANGED
Binary files a/cinegen/__pycache__/story_engine.cpython-310.pyc and b/cinegen/__pycache__/story_engine.cpython-310.pyc differ
 
cinegen/__pycache__/video_engine.cpython-310.pyc CHANGED
Binary files a/cinegen/__pycache__/video_engine.cpython-310.pyc and b/cinegen/__pycache__/video_engine.cpython-310.pyc differ