Execution Modes¶
Choose the right adapter for your use case.
Overview¶
graph LR
A[Data] --> B{Size?}
B -->|Fits in memory| C{Real-time?}
B -->|Too large| D[Streaming]
C -->|No| E[Batch]
C -->|Yes| F[Online]
| Mode | Use Case | Memory | Features |
|---|---|---|---|
| Batch | Complete datasets | Full | All features |
| Streaming | Large files (>100K) | Chunked | Residuals, robustness |
| Online | Real-time sensors | Fixed window | Incremental updates |
Batch Adapter¶
Standard mode for complete datasets. Supports all features.
When to Use¶
- Dataset fits in memory
- Need intervals, cross-validation, or diagnostics
- Processing complete files
Example¶
Streaming Adapter¶
Process large datasets in chunks with configurable overlap.
When to Use¶
- Dataset >100,000 points
- Memory-constrained environments
- Batch processing pipelines
Parameters¶
| Parameter | Default | Description |
|---|---|---|
chunk_size |
5000 | Points per chunk |
overlap |
500 | Overlap between chunks |
merge_strategy |
"weighted_average" |
How to merge overlaps |
Merge Strategies¶
| Strategy | Behavior |
|---|---|
"average" |
Average overlapping values |
"weighted_average" |
Distance-weighted blend (default) |
"take_first" |
Keep left chunk values |
"take_last" |
Keep right chunk values |
Example¶
let mut processor = StreamingLoess::new()
.fraction(0.3)
.iterations(2)
.chunk_size(50)
.overlap(10)
.merge_strategy("average")
.build()?;
let result = processor.process_chunk(&x_chunk, &y_chunk)?;
println!("Chunk processed: {} points", result.y.len());
let final_result = processor.finalize()?;
println!("Final: {} points", final_result.y.len());
const { StreamingLoess } = require('fastloess');
const processor = new StreamingLoess(
{ fraction: 0.3, iterations: 2 },
{ chunk_size: 5000, overlap: 500 }
);
// Process chunks
for (const {x, y} of dataChunks) {
const result = processor.processChunk(x, y);
// ...
}
const finalResult = processor.finalize();
import { StreamingLoess } from 'fastloess-wasm';
const processor = new StreamingLoess(
{ fraction: 0.3, iterations: 2 },
{ chunk_size: 5000, overlap: 500 }
);
// Process chunks
for (const {x, y} of dataChunks) {
const result = processor.processChunk(x, y);
// ...
}
const finalResult = processor.finalize();
Always call finalize()
In Rust, always call processor.finalize() after processing all chunks to retrieve buffered overlap data.
Online Adapter¶
Incremental updates with a sliding window for real-time data.
When to Use¶
- Data arrives incrementally (sensors, streams)
- Need real-time smoothed values
- Fixed memory budget
Parameters¶
| Parameter | Default | Description |
|---|---|---|
window_capacity |
1000 | Max points in window |
min_points |
2 | Points before output starts |
update_mode |
"incremental" |
Update strategy |
Update Modes¶
| Mode | Behavior | Speed |
|---|---|---|
"incremental" |
Update only affected fits | Faster |
"full" |
Recompute entire window | More accurate |
Example¶
const { OnlineLoess } = require('fastloess');
const processor = new OnlineLoess(
{ fraction: 0.2, iterations: 1 },
{ window_capacity: 100, min_points: 5, update_mode: "incremental" }
);
// Add points
for (const [x, y] of sensorStream) {
const res = processor.add_point(x, y);
if (res !== null) {
console.log(`Smoothed: ${res.smoothed.toFixed(2)}`);
}
}
import { OnlineLoess } from 'fastloess-wasm';
const processor = new OnlineLoess(
{ fraction: 0.2, iterations: 1 },
{ window_capacity: 100, min_points: 5, update_mode: "incremental" }
);
// Add points
for (const [x, y] of sensorStream) {
const res = processor.add_point(x, y);
if (res !== undefined) {
console.log(`Smoothed: ${res.smoothed.toFixed(2)}`);
}
}
#include "fastloess.hpp"
fastloess::OnlineOptions opts;
opts.fraction = 0.2;
opts.iterations = 1;
opts.window_capacity = 100;
opts.min_points = 5;
opts.update_mode = "incremental";
fastloess::OnlineLoess model(opts);
for (size_t i = 0; i < x.size(); ++i) {
auto out = model.add_point(x[i], y[i]).value();
if (out.has_value())
std::cout << out.smoothed() << std::endl;
}
Feature Comparison¶
| Feature | Batch | Streaming | Online |
|---|---|---|---|
| Confidence intervals | ✓ | ✗ | ✗ |
| Prediction intervals | ✓ | ✗ | ✗ |
| Cross-validation | ✓ | ✗ | ✗ |
| Diagnostics | ✓ | ✓ | ✗ |
| Residuals | ✓ | ✓ | ✓ |
| Robustness weights | ✓ | ✓ | ✓ |
| Parallel execution | ✓ | ✓ | ✗ |
Next Steps¶
- Parameters — All configuration options
- Tutorials — Real-time processing guide