Spaces:
Sleeping
Sleeping
Evgueni Poloukarov
commited on
Commit
·
51429a1
1
Parent(s):
12f45c0
docs: add Session 8 summary - architecture refactoring to Gradio API
Browse files- doc/activity.md +140 -0
doc/activity.md
CHANGED
|
@@ -5300,3 +5300,143 @@ git push hf-space master:main # HuggingFace Space
|
|
| 5300 |
**Next Session**: Configure Space secrets, test notebooks on GPU, evaluate MAE
|
| 5301 |
|
| 5302 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5300 |
**Next Session**: Configure Space secrets, test notebooks on GPU, evaluate MAE
|
| 5301 |
|
| 5302 |
---
|
| 5303 |
+
|
| 5304 |
+
## Session 8: Architecture Refactoring - JupyterLab to Gradio API (Nov 14, 2025 14:00-16:00 UTC)
|
| 5305 |
+
|
| 5306 |
+
### Critical Architecture Realization
|
| 5307 |
+
|
| 5308 |
+
**Problem Identified**:
|
| 5309 |
+
- Attempted to use HF Spaces as interactive development environment (JupyterLab)
|
| 5310 |
+
- SSH sessions repeatedly killed with exit code 137 (OOM/resource limits)
|
| 5311 |
+
- Manual notebook debugging in browser too slow and error-prone
|
| 5312 |
+
- Multiple parameter errors in notebooks (df= vs dataset=, missing imports, etc.)
|
| 5313 |
+
|
| 5314 |
+
**Root Cause**:
|
| 5315 |
+
- HF Spaces are **deployment targets** for inference APIs, NOT interactive compute environments
|
| 5316 |
+
- SSH sessions have strict resource limits (different from web app containers)
|
| 5317 |
+
- JupyterLab workflow requires constant iteration → wrong paradigm for cloud GPU
|
| 5318 |
+
|
| 5319 |
+
**Architectural Insight** (from Claude Desktop suggestion):
|
| 5320 |
+
```
|
| 5321 |
+
WRONG: Local edit → Push to HF → SSH debug → Fix errors → Repeat
|
| 5322 |
+
RIGHT: Local edit → Push to HF → API call → Download results → Local analysis
|
| 5323 |
+
```
|
| 5324 |
+
|
| 5325 |
+
### Architecture Refactoring
|
| 5326 |
+
|
| 5327 |
+
**Removed**:
|
| 5328 |
+
- JupyterLab configuration (Dockerfile, start_server.sh, on_startup.sh)
|
| 5329 |
+
- Interactive notebook execution on HF Space
|
| 5330 |
+
- SSH-based debugging workflow
|
| 5331 |
+
- Docker SDK (complex custom containers)
|
| 5332 |
+
|
| 5333 |
+
**Added**:
|
| 5334 |
+
- Gradio SDK (simple, purpose-built for inference APIs)
|
| 5335 |
+
- Production inference pipeline (chronos_inference.py - 312 lines)
|
| 5336 |
+
- API-first interface (app.py - 112 lines)
|
| 5337 |
+
- Clean separation: Development (local) vs Deployment (HF Space)
|
| 5338 |
+
|
| 5339 |
+
### New Workflow Pattern
|
| 5340 |
+
|
| 5341 |
+
**Local Development**:
|
| 5342 |
+
1. Edit code in Claude Code
|
| 5343 |
+
2. Push to HF Space via git
|
| 5344 |
+
3. Call API from local machine
|
| 5345 |
+
4. Download results as parquet
|
| 5346 |
+
5. Analyze locally (Marimo notebooks)
|
| 5347 |
+
|
| 5348 |
+
**HF Space** (Deployment Only):
|
| 5349 |
+
- Purpose: GPU inference endpoint
|
| 5350 |
+
- Input: Forecast parameters via Gradio UI or API
|
| 5351 |
+
- Output: Parquet file with forecasts
|
| 5352 |
+
- No SSH debugging required
|
| 5353 |
+
|
| 5354 |
+
### Benefits
|
| 5355 |
+
|
| 5356 |
+
1. **Eliminates SSH issues**: No more exit code 137
|
| 5357 |
+
2. **Model caching**: Loaded once, stays in memory
|
| 5358 |
+
3. **Clean API**: Call from anywhere (Python, curl, web)
|
| 5359 |
+
4. **Proper separation**: Development local, inference remote
|
| 5360 |
+
5. **Cost efficient**: Only pay GPU during inference
|
| 5361 |
+
|
| 5362 |
+
### Testing Status
|
| 5363 |
+
|
| 5364 |
+
**Completed**:
|
| 5365 |
+
- ✅ Code refactoring (notebooks → API)
|
| 5366 |
+
- ✅ Gradio app creation
|
| 5367 |
+
- ✅ Git deployment (GitHub + HF Space)
|
| 5368 |
+
- ✅ Space rebuild triggered
|
| 5369 |
+
|
| 5370 |
+
**Pending** (Next Session):
|
| 5371 |
+
- ⏳ Space rebuild completion
|
| 5372 |
+
- ⏳ Web UI validation
|
| 5373 |
+
- ⏳ Smoke test API call (1 border × 7 days)
|
| 5374 |
+
- ⏳ Full forecast (38 borders × 14 days)
|
| 5375 |
+
- ⏳ MAE evaluation vs Oct 1-14 actuals
|
| 5376 |
+
|
| 5377 |
+
### Next Session Plan
|
| 5378 |
+
|
| 5379 |
+
**Priority 1**: Validate API deployment (15 min)
|
| 5380 |
+
- Check Gradio UI loads
|
| 5381 |
+
- Verify interface functional
|
| 5382 |
+
|
| 5383 |
+
**Priority 2**: Test smoke test (10 min)
|
| 5384 |
+
```python
|
| 5385 |
+
from gradio_client import Client
|
| 5386 |
+
client = Client("evgueni-p/fbmc-chronos2")
|
| 5387 |
+
result = client.predict(run_date="2025-09-30", forecast_type="smoke_test")
|
| 5388 |
+
```
|
| 5389 |
+
|
| 5390 |
+
**Priority 3**: Test full forecast (15 min)
|
| 5391 |
+
- Run all 38 borders × 14 days
|
| 5392 |
+
- Verify output schema
|
| 5393 |
+
- Check inference time <5 min
|
| 5394 |
+
|
| 5395 |
+
**Priority 4**: Evaluate MAE (20 min)
|
| 5396 |
+
- Load Oct 1-14 actuals
|
| 5397 |
+
- Calculate MAE per border
|
| 5398 |
+
- Document results
|
| 5399 |
+
|
| 5400 |
+
**Priority 5**: Update documentation (10 min)
|
| 5401 |
+
- Final results in activity.md
|
| 5402 |
+
- API usage guide
|
| 5403 |
+
- Handover checklist
|
| 5404 |
+
|
| 5405 |
+
### Files Created
|
| 5406 |
+
|
| 5407 |
+
- `src/forecasting/chronos_inference.py` (312 lines)
|
| 5408 |
+
- `app.py` (112 lines)
|
| 5409 |
+
|
| 5410 |
+
### Files Modified
|
| 5411 |
+
|
| 5412 |
+
- `README.md` - Changed to Gradio SDK
|
| 5413 |
+
- `requirements.txt` - Removed JupyterLab, added Gradio
|
| 5414 |
+
|
| 5415 |
+
---
|
| 5416 |
+
|
| 5417 |
+
**Status**: [IN PROGRESS] HF Space rebuilding with Gradio API
|
| 5418 |
+
**Timestamp**: 2025-11-14 16:00 UTC
|
| 5419 |
+
**Next Session**: Validate API, run forecasts, evaluate MAE
|
| 5420 |
+
|
| 5421 |
+
---
|
| 5422 |
+
|
| 5423 |
+
## NEXT SESSION BOOKMARK
|
| 5424 |
+
|
| 5425 |
+
**Resume with**:
|
| 5426 |
+
1. Check HF Space ready: https://huggingface.co/spaces/evgueni-p/fbmc-chronos2
|
| 5427 |
+
2. Install gradio_client: `uv pip install gradio-client`
|
| 5428 |
+
3. Run smoke test via API
|
| 5429 |
+
4. Download and verify results
|
| 5430 |
+
5. Run full forecast
|
| 5431 |
+
6. Evaluate MAE vs actuals
|
| 5432 |
+
|
| 5433 |
+
**Success criteria**:
|
| 5434 |
+
- Smoke test returns 168 rows (7 days × 24 hours)
|
| 5435 |
+
- Full forecast completes in <5 minutes
|
| 5436 |
+
- MAE ≤ 150 MW on D+1 forecasts
|
| 5437 |
+
- All 38 borders in output
|
| 5438 |
+
|
| 5439 |
+
**Key files**:
|
| 5440 |
+
- `src/forecasting/chronos_inference.py` - Main logic
|
| 5441 |
+
- `src/forecasting/dynamic_forecast.py` - Data extraction
|
| 5442 |
+
- `app.py` - Gradio interface
|