Evgueni Poloukarov Claude commited on
Commit
12f45c0
·
1 Parent(s): c0dc80e

refactor: convert HF Space from JupyterLab to Gradio API

Browse files

Architecture Change:
- Replace interactive notebooks with API endpoint
- HF Space now serves as inference API (not development environment)

New Components:
- app.py: Gradio web interface for triggering forecasts
- chronos_inference.py: Production inference pipeline
- Supports smoke test (7 days) and full forecast (14 days)

Benefits:
- No SSH resource limit issues (exit code 137)
- API-first design (can call from local machine)
- Model caching (loaded once, stays in memory)
- Results downloadable as parquet files
- Local development → Remote GPU inference workflow

Usage:
1. Web UI: https://huggingface.co/spaces/evgueni-p/fbmc-chronos2
2. Python API: gradio_client.Client("evgueni-p/fbmc-chronos2").predict()

Co-Authored-By: Claude <[email protected]>

Files changed (4) hide show
  1. README.md +33 -13
  2. app.py +138 -0
  3. requirements.txt +3 -6
  4. src/forecasting/chronos_inference.py +296 -0
README.md CHANGED
@@ -3,32 +3,52 @@ title: FBMC Chronos-2 Forecasting
3
  emoji: ⚡
4
  colorFrom: blue
5
  colorTo: green
6
- sdk: docker
 
 
7
  pinned: false
8
  tags:
9
- - jupyterlab
 
 
 
 
10
  suggested_storage: small
11
  ---
12
 
13
- # FBMC Flow-Based Market Coupling Forecasting
14
 
15
- Zero-shot electricity cross-border flow forecasting for 38 European FBMC borders using Amazon Chronos 2.
16
 
17
  ## 🚀 Quick Start
18
 
19
- This HuggingFace Space provides interactive Jupyter notebooks for running zero-shot forecasts on GPU.
20
 
21
- ### Available Notebooks
22
 
23
- 1. **`inference_smoke_test.ipynb`** - Quick validation (1 border × 7 days, ~1 min)
24
- 2. **`inference_full_14day.ipynb`** - Production forecast (38 borders × 14 days, ~5 min)
25
- 3. **`evaluation.ipynb`** - Performance analysis vs actuals
 
 
 
26
 
27
- ### How to Use
28
 
29
- 1. Open any notebook in JupyterLab
30
- 2. Run all cells (Cell → Run All)
31
- 3. View results and visualizations inline
 
 
 
 
 
 
 
 
 
 
 
32
 
33
  ## 📊 Dataset
34
 
 
3
  emoji: ⚡
4
  colorFrom: blue
5
  colorTo: green
6
+ sdk: gradio
7
+ sdk_version: 4.44.0
8
+ app_file: app.py
9
  pinned: false
10
  tags:
11
+ - forecasting
12
+ - time-series
13
+ - electricity
14
+ - zero-shot
15
+ suggested_hardware: t4-small
16
  suggested_storage: small
17
  ---
18
 
19
+ # FBMC Flow-Based Market Coupling Forecasting API
20
 
21
+ Zero-shot electricity cross-border flow forecasting for 38 European FBMC borders using Amazon Chronos-2.
22
 
23
  ## 🚀 Quick Start
24
 
25
+ This HuggingFace Space provides a **Gradio API** for GPU-accelerated zero-shot forecasting.
26
 
27
+ ### How to Use (Web Interface)
28
 
29
+ 1. **Select run date**: Choose the forecast date (YYYY-MM-DD format)
30
+ 2. **Choose forecast type**:
31
+ - **Smoke Test**: 1 border × 7 days (~30 seconds)
32
+ - **Full Forecast**: All 38 borders × 14 days (~5 minutes)
33
+ 3. **Click "Run Forecast"**
34
+ 4. **Download results**: Parquet file with probabilistic forecasts
35
 
36
+ ### How to Use (Python API)
37
 
38
+ ```python
39
+ from gradio_client import Client
40
+
41
+ client = Client("evgueni-p/fbmc-chronos2")
42
+ result_file = client.predict(
43
+ run_date="2025-09-30",
44
+ forecast_type="smoke_test"
45
+ )
46
+
47
+ # Download and analyze locally
48
+ import polars as pl
49
+ df = pl.read_parquet(result_file)
50
+ print(df.head())
51
+ ```
52
 
53
  ## 📊 Dataset
54
 
app.py ADDED
@@ -0,0 +1,138 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ FBMC Chronos-2 Forecasting API
4
+ HuggingFace Space Gradio Interface
5
+ """
6
+
7
+ import gradio as gr
8
+ import os
9
+ from datetime import datetime
10
+ from src.forecasting.chronos_inference import run_inference
11
+
12
+
13
+ # Global configuration
14
+ FORECAST_TYPES = {
15
+ "smoke_test": "Smoke Test (1 border × 7 days)",
16
+ "full_14day": "Full Forecast (All borders × 14 days)"
17
+ }
18
+
19
+
20
+ def forecast_api(run_date_str, forecast_type):
21
+ """
22
+ API endpoint for triggering forecasts.
23
+
24
+ Args:
25
+ run_date_str: Date in YYYY-MM-DD format
26
+ forecast_type: 'smoke_test' or 'full_14day'
27
+
28
+ Returns:
29
+ Path to downloadable forecast results file
30
+ """
31
+ try:
32
+ # Validate run date
33
+ run_date = datetime.strptime(run_date_str, "%Y-%m-%d")
34
+
35
+ # Run inference
36
+ result_path = run_inference(
37
+ run_date=run_date_str,
38
+ forecast_type=forecast_type,
39
+ output_dir="/tmp"
40
+ )
41
+
42
+ return result_path
43
+
44
+ except Exception as e:
45
+ error_msg = f"Error: {str(e)}"
46
+ print(error_msg)
47
+ # Return error message as text file
48
+ error_path = "/tmp/error.txt"
49
+ with open(error_path, 'w') as f:
50
+ f.write(error_msg)
51
+ return error_path
52
+
53
+
54
+ # Build Gradio interface
55
+ with gr.Blocks(title="FBMC Chronos-2 Forecasting") as demo:
56
+ gr.Markdown("""
57
+ # FBMC Chronos-2 Zero-Shot Forecasting API
58
+
59
+ **Flow-Based Market Coupling** electricity flow forecasting using Amazon Chronos-2.
60
+
61
+ This Space provides GPU-accelerated zero-shot inference for cross-border electricity flows.
62
+ """)
63
+
64
+ with gr.Row():
65
+ with gr.Column():
66
+ gr.Markdown("### Configuration")
67
+
68
+ run_date_input = gr.Textbox(
69
+ label="Run Date (YYYY-MM-DD)",
70
+ value="2025-09-30",
71
+ placeholder="2025-09-30",
72
+ info="Date when forecast is made (data up to this date is historical)"
73
+ )
74
+
75
+ forecast_type_input = gr.Radio(
76
+ choices=list(FORECAST_TYPES.keys()),
77
+ value="smoke_test",
78
+ label="Forecast Type",
79
+ info="Smoke test: Quick validation (1 border, 7 days). Full: Production forecast (all borders, 14 days)"
80
+ )
81
+
82
+ submit_btn = gr.Button("Run Forecast", variant="primary")
83
+
84
+ with gr.Column():
85
+ gr.Markdown("### Results")
86
+
87
+ output_file = gr.File(
88
+ label="Download Forecast Results",
89
+ type="filepath"
90
+ )
91
+
92
+ gr.Markdown("""
93
+ **Output format**: Parquet file with columns:
94
+ - `timestamp`: Hourly timestamps (D+1 to D+7 or D+14)
95
+ - `{border}_median`: Median forecast (MW)
96
+ - `{border}_q10`: 10th percentile (MW)
97
+ - `{border}_q90`: 90th percentile (MW)
98
+
99
+ **Inference environment**:
100
+ - GPU: NVIDIA T4 (16GB VRAM)
101
+ - Model: Chronos-T5-Large (710M parameters)
102
+ - Precision: bfloat16
103
+ """)
104
+
105
+ # Wire up the interface
106
+ submit_btn.click(
107
+ fn=forecast_api,
108
+ inputs=[run_date_input, forecast_type_input],
109
+ outputs=output_file
110
+ )
111
+
112
+ gr.Markdown("""
113
+ ---
114
+ ### About
115
+
116
+ **Zero-shot forecasting**: No model training required. The pre-trained Chronos-2 model
117
+ generalizes directly to FBMC cross-border flows using historical patterns and future covariates.
118
+
119
+ **Features**:
120
+ - 2,553 engineered features (weather, CNEC constraints, load forecasts, LTA)
121
+ - 24-month historical context (Oct 2023 - Oct 2025)
122
+ - Time-aware extraction (prevents data leakage)
123
+ - Probabilistic forecasts (10th/50th/90th percentiles)
124
+
125
+ **Performance**:
126
+ - Smoke test: ~30 seconds (1 border × 168 hours)
127
+ - Full forecast: ~5 minutes (38 borders × 336 hours)
128
+
129
+ **Project**: FBMC Flow Forecasting MVP | **Author**: Evgueni Poloukarov
130
+ """)
131
+
132
+ # Launch the app
133
+ if __name__ == "__main__":
134
+ demo.launch(
135
+ server_name="0.0.0.0",
136
+ server_port=7860,
137
+ share=False
138
+ )
requirements.txt CHANGED
@@ -1,7 +1,5 @@
1
- # JupyterLab (from HF template)
2
- jupyterlab==4.2.5
3
- tornado==6.2
4
- ipywidgets
5
 
6
  # Core ML/Data
7
  torch>=2.0.0
@@ -14,9 +12,8 @@ pyarrow>=13.0.0
14
  # HuggingFace
15
  huggingface-hub>=0.19.0
16
 
17
- # Visualization
18
  altair>=5.0.0
19
- vega-datasets
20
 
21
  # Utilities
22
  python-dotenv
 
1
+ # Gradio
2
+ gradio==4.44.0
 
 
3
 
4
  # Core ML/Data
5
  torch>=2.0.0
 
12
  # HuggingFace
13
  huggingface-hub>=0.19.0
14
 
15
+ # Visualization (for local analysis)
16
  altair>=5.0.0
 
17
 
18
  # Utilities
19
  python-dotenv
src/forecasting/chronos_inference.py ADDED
@@ -0,0 +1,296 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Chronos-2 Inference Pipeline
4
+ Standalone inference script for HuggingFace Space deployment.
5
+ """
6
+
7
+ import os
8
+ import time
9
+ from typing import List, Dict, Optional
10
+ from datetime import datetime, timedelta
11
+ import polars as pl
12
+ import pandas as pd
13
+ import torch
14
+ from datasets import load_dataset
15
+ from chronos import ChronosPipeline
16
+
17
+ from .dynamic_forecast import DynamicForecast
18
+ from .feature_availability import FeatureAvailability
19
+
20
+
21
+ class ChronosInferencePipeline:
22
+ """
23
+ Production inference pipeline for Chronos-2 zero-shot forecasting.
24
+ Designed for deployment as API endpoint on HuggingFace Spaces.
25
+ """
26
+
27
+ def __init__(
28
+ self,
29
+ model_name: str = "amazon/chronos-t5-large",
30
+ device: str = "cuda",
31
+ dtype: str = "bfloat16"
32
+ ):
33
+ """
34
+ Initialize inference pipeline.
35
+
36
+ Args:
37
+ model_name: HuggingFace model identifier
38
+ device: Device for inference ('cuda' or 'cpu')
39
+ dtype: Data type for model weights
40
+ """
41
+ self.model_name = model_name
42
+ self.device = device
43
+ self.dtype = dtype
44
+
45
+ # Model loaded on first inference (lazy loading)
46
+ self._pipeline = None
47
+ self._dataset = None
48
+ self._borders = None
49
+
50
+ def _load_model(self):
51
+ """Load Chronos model (cached after first call)"""
52
+ if self._pipeline is None:
53
+ print(f"Loading {self.model_name}...")
54
+ start_time = time.time()
55
+
56
+ dtype_map = {
57
+ "bfloat16": torch.bfloat16,
58
+ "float16": torch.float16,
59
+ "float32": torch.float32
60
+ }
61
+
62
+ self._pipeline = ChronosPipeline.from_pretrained(
63
+ self.model_name,
64
+ device_map=self.device,
65
+ torch_dtype=dtype_map.get(self.dtype, torch.bfloat16)
66
+ )
67
+
68
+ print(f"Model loaded in {time.time() - start_time:.1f}s")
69
+ print(f" Device: {next(self._pipeline.model.parameters()).device}")
70
+
71
+ return self._pipeline
72
+
73
+ def _load_dataset(self):
74
+ """Load dataset from HuggingFace (cached after first call)"""
75
+ if self._dataset is None:
76
+ print("Loading dataset from HuggingFace...")
77
+ start_time = time.time()
78
+
79
+ hf_token = os.getenv("HF_TOKEN")
80
+ dataset = load_dataset(
81
+ "evgueni-p/fbmc-features-24month",
82
+ split="train",
83
+ token=hf_token
84
+ )
85
+
86
+ # Convert to Polars
87
+ self._dataset = pl.from_arrow(dataset.data.table)
88
+
89
+ # Extract available borders
90
+ target_cols = [col for col in self._dataset.columns if col.startswith('target_border_')]
91
+ self._borders = [col.replace('target_border_', '') for col in target_cols]
92
+
93
+ print(f"Dataset loaded in {time.time() - start_time:.1f}s")
94
+ print(f" Shape: {self._dataset.shape}")
95
+ print(f" Borders: {len(self._borders)}")
96
+
97
+ return self._dataset, self._borders
98
+
99
+ def run_forecast(
100
+ self,
101
+ run_date: str,
102
+ borders: Optional[List[str]] = None,
103
+ forecast_days: int = 7,
104
+ context_hours: int = 512,
105
+ num_samples: int = 20
106
+ ) -> Dict:
107
+ """
108
+ Run zero-shot forecast for specified borders.
109
+
110
+ Args:
111
+ run_date: Forecast run date (YYYY-MM-DD format)
112
+ borders: List of borders to forecast (None = all borders)
113
+ forecast_days: Forecast horizon in days (7 or 14)
114
+ context_hours: Historical context window
115
+ num_samples: Number of probabilistic samples
116
+
117
+ Returns:
118
+ Dictionary with forecast results and metadata
119
+ """
120
+ # Load model and dataset (cached)
121
+ pipeline = self._load_model()
122
+ df, all_borders = self._load_dataset()
123
+
124
+ # Parse run date
125
+ run_datetime = datetime.strptime(run_date, "%Y-%m-%d")
126
+ run_datetime = run_datetime.replace(hour=23, minute=0)
127
+
128
+ # Determine borders to forecast
129
+ forecast_borders = borders if borders else all_borders
130
+ prediction_hours = forecast_days * 24
131
+
132
+ print(f"\nForecast configuration:")
133
+ print(f" Run date: {run_datetime}")
134
+ print(f" Borders: {len(forecast_borders)}")
135
+ print(f" Forecast horizon: {forecast_days} days ({prediction_hours} hours)")
136
+ print(f" Context window: {context_hours} hours")
137
+
138
+ # Initialize dynamic forecast system
139
+ forecaster = DynamicForecast(
140
+ dataset=df,
141
+ context_hours=context_hours,
142
+ forecast_hours=prediction_hours
143
+ )
144
+
145
+ # Run forecasts for each border
146
+ results = {
147
+ 'run_date': run_date,
148
+ 'forecast_days': forecast_days,
149
+ 'borders': {},
150
+ 'metadata': {
151
+ 'model': self.model_name,
152
+ 'device': self.device,
153
+ 'num_samples': num_samples,
154
+ 'context_hours': context_hours
155
+ }
156
+ }
157
+
158
+ total_start = time.time()
159
+
160
+ for i, border in enumerate(forecast_borders, 1):
161
+ print(f"\n[{i}/{len(forecast_borders)}] Forecasting {border}...")
162
+ border_start = time.time()
163
+
164
+ try:
165
+ # Extract data
166
+ context_data, future_data = forecaster.prepare_forecast_data(
167
+ run_date=run_datetime,
168
+ border=border
169
+ )
170
+
171
+ # Get target column name
172
+ target_col = f"target_border_{border}"
173
+
174
+ # Extract context values
175
+ context = context_data[target_col].values
176
+
177
+ # Run inference
178
+ forecast = pipeline.predict(
179
+ context=context,
180
+ prediction_length=prediction_hours,
181
+ num_samples=num_samples
182
+ )
183
+
184
+ # Calculate quantiles
185
+ forecast_numpy = forecast.numpy()
186
+
187
+ # Store results
188
+ results['borders'][border] = {
189
+ 'median': forecast_numpy.median(axis=0).tolist(),
190
+ 'q10': forecast_numpy.quantile(0.1, axis=0).tolist(),
191
+ 'q90': forecast_numpy.quantile(0.9, axis=0).tolist(),
192
+ 'inference_time_s': time.time() - border_start
193
+ }
194
+
195
+ print(f" ✓ Complete in {time.time() - border_start:.1f}s")
196
+
197
+ except Exception as e:
198
+ print(f" ✗ Error: {str(e)}")
199
+ results['borders'][border] = {'error': str(e)}
200
+
201
+ # Add summary metadata
202
+ results['metadata']['total_time_s'] = time.time() - total_start
203
+ results['metadata']['successful_borders'] = sum(
204
+ 1 for b in results['borders'].values() if 'error' not in b
205
+ )
206
+
207
+ print(f"\n{'='*60}")
208
+ print(f"FORECAST COMPLETE")
209
+ print(f"{'='*60}")
210
+ print(f"Total time: {results['metadata']['total_time_s']:.1f}s")
211
+ print(f"Successful: {results['metadata']['successful_borders']}/{len(forecast_borders)} borders")
212
+
213
+ return results
214
+
215
+ def export_to_parquet(self, results: Dict, output_path: str):
216
+ """
217
+ Export forecast results to parquet format.
218
+
219
+ Args:
220
+ results: Forecast results from run_forecast()
221
+ output_path: Path to save parquet file
222
+ """
223
+ # Create forecast timestamps
224
+ run_datetime = datetime.strptime(results['run_date'], "%Y-%m-%d")
225
+ forecast_start = run_datetime + timedelta(hours=1)
226
+ forecast_hours = results['forecast_days'] * 24
227
+
228
+ timestamps = [
229
+ forecast_start + timedelta(hours=h)
230
+ for h in range(forecast_hours)
231
+ ]
232
+
233
+ # Build DataFrame
234
+ data = {'timestamp': timestamps}
235
+
236
+ for border, forecast_data in results['borders'].items():
237
+ if 'error' not in forecast_data:
238
+ data[f'{border}_median'] = forecast_data['median']
239
+ data[f'{border}_q10'] = forecast_data['q10']
240
+ data[f'{border}_q90'] = forecast_data['q90']
241
+
242
+ df = pl.DataFrame(data)
243
+ df.write_parquet(output_path)
244
+
245
+ print(f"✓ Exported to: {output_path}")
246
+ print(f" Shape: {df.shape}")
247
+
248
+ return output_path
249
+
250
+
251
+ # Convenience function for API usage
252
+ def run_inference(
253
+ run_date: str,
254
+ forecast_type: str = "smoke_test",
255
+ borders: Optional[List[str]] = None,
256
+ output_dir: str = "/tmp"
257
+ ) -> str:
258
+ """
259
+ Run forecast and return path to results file.
260
+
261
+ Args:
262
+ run_date: Forecast run date (YYYY-MM-DD)
263
+ forecast_type: 'smoke_test' (7 days, 1 border) or 'full_14day' (14 days, all borders)
264
+ borders: Specific borders to forecast (None = use forecast_type defaults)
265
+ output_dir: Directory to save results
266
+
267
+ Returns:
268
+ Path to forecast results parquet file
269
+ """
270
+ # Initialize pipeline
271
+ pipeline = ChronosInferencePipeline()
272
+
273
+ # Configure based on forecast type
274
+ if forecast_type == "smoke_test":
275
+ forecast_days = 7
276
+ if borders is None:
277
+ # Load just to get first border
278
+ _, all_borders = pipeline._load_dataset()
279
+ borders = [all_borders[0]]
280
+ else: # full_14day
281
+ forecast_days = 14
282
+ # borders = None means all borders
283
+
284
+ # Run forecast
285
+ results = pipeline.run_forecast(
286
+ run_date=run_date,
287
+ borders=borders,
288
+ forecast_days=forecast_days
289
+ )
290
+
291
+ # Export to parquet
292
+ output_filename = f"forecast_{run_date}_{forecast_type}.parquet"
293
+ output_path = os.path.join(output_dir, output_filename)
294
+ pipeline.export_to_parquet(results, output_path)
295
+
296
+ return output_path