Cross-Validation¶

Automated parameter selection via cross-validation.

Overview¶

Cross-validation helps select optimal parameters (especially fraction) by evaluating performance on held-out data.

Cross-Validation

K-Fold Cross-Validation¶

Split data into K folds, train on K-1, validate on 1.

RPythonRustJuliaNode.jsWebAssemblyC++

model <- Loess(
    cv_method = "kfold",
    cv_k = 5,
    cv_fractions = c(0.2, 0.3, 0.5, 0.7)
)
result <- model$fit(x, y)

cat("Selected fraction:", result$fraction_used, "\n")
cat("CV scores:", result$cv_scores, "\n")

model = fl.Loess(
    cv_method="kfold",
    cv_k=5,
    cv_fractions=[0.2, 0.3, 0.5, 0.7]
)
result = model.fit(x, y)

print(f"Selected fraction: {result.fraction_used}")
print(f"CV scores: {result.cv_scores}")

use fastLoess::prelude::*;

let model = Loess::new()
    .cv_method("kfold")
    .cv_k(5)
    .cv_fractions(vec![0.2, 0.3, 0.5, 0.7])
    .build()?;

let result = model.fit(&x, &y)?;

// The best fraction was automatically selected
println!("Selected fraction: {}", result.fraction_used);

if let Some(cv_scores) = &result.cv_scores {
    println!("CV scores: {:?}", cv_scores);
}

using FastLOESS

model = Loess(;
    cv_method="kfold",
    cv_k=5,
    cv_fractions=[0.2, 0.3, 0.5, 0.7]
)
result = fit(model, x, y)

println("Selected fraction: ", result.fraction_used)

const model = new Loess({
    cv_method: "kfold",
    cv_k: 5,
    cv_fractions: [0.2, 0.3, 0.5, 0.7]
});
const result = model.fit(x, y);

console.log("Selected fraction:", result.fraction_used);
console.log("CV scores:", result.cv_scores);

const model = new Loess({
    cv_method: "kfold",
    cv_k: 5,
    cv_fractions: [0.2, 0.3, 0.5, 0.7]
});
const result = model.fit(x, y);

console.log("Selected fraction:", result.fraction_used);
console.log("CV scores:", result.cv_scores);

#include "fastloess.hpp"

fastloess::LoessOptions opts;
opts.cv_fractions = {0.2, 0.3, 0.5, 0.7};
opts.cv_method = "kfold";
opts.cv_k = 5;

fastloess::Loess model(opts);
auto result = model.fit(x, y).value();

std::cout << "Selected fraction: " << result.fraction_used() << std::endl;

Leave-One-Out (LOOCV)¶

Each point is held out once. Most thorough but slowest.

RPythonRustJuliaNode.jsWebAssemblyC++

model <- Loess(
    cv_method = "loocv",
    cv_fractions = c(0.2, 0.3, 0.5, 0.7)
)
result <- model$fit(x, y)

model = fl.Loess(
    cv_method="loocv",
    cv_fractions=[0.2, 0.3, 0.5, 0.7]
)
result = model.fit(x, y)

let model = Loess::new()
    .cv_method("loocv")
    .cv_fractions(vec![0.2, 0.3, 0.5, 0.7])
    .build()?;
let result = model.fit(&x, &y)?;

model = Loess(;
    cv_method="loocv",
    cv_fractions=[0.2, 0.3, 0.5, 0.7]
)
result = fit(model, x, y)

const model = new Loess({
    cv_method: "loocv",
    cv_fractions: [0.2, 0.3, 0.5, 0.7]
});
const result = model.fit(x, y);

const model = new Loess({
    cv_method: "loocv",
    cv_fractions: [0.2, 0.3, 0.5, 0.7]
});
const result = model.fit(x, y);

#include "fastloess.hpp"

fastloess::LoessOptions opts;
opts.cv_method = "loocv";
opts.cv_fractions = {0.2, 0.3, 0.5, 0.7};

fastloess::Loess model(opts);
auto result = model.fit(x, y).value();

Seeded Randomization¶

Set a seed for reproducible fold assignments:

RPythonRustJuliaNode.jsWebAssemblyC++

model <- Loess(
    cv_method = "kfold",
    cv_k = 5,
    cv_fractions = c(0.3, 0.5, 0.7),
    cv_seed = 42
)
result <- model$fit(x, y)

model = fl.Loess(
    cv_method="kfold",
    cv_k=5,
    cv_fractions=[0.3, 0.5, 0.7],
    cv_seed=42
)
result = model.fit(x, y)

let model = Loess::new()
    .cv_method("kfold")
    .cv_k(5)
    .cv_fractions(vec![0.3, 0.5, 0.7])
    .cv_seed(42)
    .build()?;
let result = model.fit(&x, &y)?;

model = Loess(;
    cv_method="kfold",
    cv_k=5,
    cv_fractions=[0.3, 0.5, 0.7],
    cv_seed=42
)
result = fit(model, x, y)

const model = new Loess({
    cv_method: "kfold",
    cv_k: 5,
    cv_fractions: [0.3, 0.5, 0.7],
    cv_seed: 42
});
const result = model.fit(x, y);

const model = new Loess({
    cv_method: "kfold",
    cv_k: 5,
    cv_fractions: [0.3, 0.5, 0.7],
    cv_seed: 42
});
const result = model.fit(x, y);

#include "fastloess.hpp"

fastloess::LoessOptions opts;
opts.cv_fractions = {0.3, 0.5, 0.7};
opts.cv_method = "kfold";
opts.cv_k = 5;
opts.cv_seed = 42;

fastloess::Loess model(opts);
auto result = model.fit(x, y).value();

Comparison¶

Method	Folds	Speed	Variance	Bias
KFold(5)	5	Fast	Moderate	Low
KFold(10)	10	Medium	Lower	Lower
LOOCV	N	Slow	Lowest	Lowest

Recommendation

Use 5-fold or 10-fold CV for most applications. LOOCV is only worth it for small datasets (N < 100).

CV Metrics¶

Cross-validation uses MSE (Mean Squared Error) by default:

MSE = mean((y_true - y_pred)²)

Lower MSE indicates better fit on held-out data.

Interpreting Results¶

RPythonRustJuliaNode.jsWebAssemblyC++

# Example output
model <- Loess(cv_method = "kfold", cv_k = 5,
                cv_fractions = c(0.1, 0.3, 0.5, 0.7))
result <- model$fit(x, y)

# Fraction  | CV Score (MSE)
# 0.1       | 0.0542  ← Undersmoothed
# 0.3       | 0.0231  ← Best
# 0.5       | 0.0298
# 0.7       | 0.0412  ← Oversmoothed

# Example output
model = fl.Loess(cv_method="kfold", cv_k=5,
                   cv_fractions=[0.1, 0.3, 0.5, 0.7])
result = model.fit(x, y)

# Fraction  | CV Score (MSE)
# 0.1       | 0.0542  ← Undersmoothed
# 0.3       | 0.0231  ← Best
# 0.5       | 0.0298
# 0.7       | 0.0412  ← Oversmoothed

// Example output
let model = Loess::new()
    .cv_method("kfold")
    .cv_k(5)
    .cv_fractions(vec![0.1, 0.3, 0.5, 0.7])
    .build()?;

let result = model.fit(&x, &y)?;

// Fraction  | CV Score (MSE)
// 0.1       | 0.0542  ← Undersmoothed
// 0.3       | 0.0231  ← Best
// 0.5       | 0.0298
// 0.7       | 0.0412  ← Oversmoothed

# Example output
model = Loess(; cv_method="kfold", cv_k=5,
                cv_fractions=[0.1, 0.3, 0.5, 0.7])
result = fit(model, x, y)

# Fraction  | CV Score (MSE)
# 0.1       | 0.0542  ← Undersmoothed
# 0.3       | 0.0231  ← Best
# 0.5       | 0.0298
# 0.7       | 0.0412  ← Oversmoothed

// Example output
const model = new Loess({
    cv_method: "kfold",
    cv_k: 5,
    cv_fractions: [0.1, 0.3, 0.5, 0.7]
});
const result = model.fit(x, y);

// Fraction  | CV Score (MSE)
// 0.1       | 0.0542  ← Undersmoothed
// 0.3       | 0.0231  ← Best
// 0.5       | 0.0298
// 0.7       | 0.0412  ← Oversmoothed

// Example output
const model = new Loess({
    cv_method: "kfold",
    cv_k: 5,
    cv_fractions: [0.1, 0.3, 0.5, 0.7]
});
const result = model.fit(x, y);

// Fraction  | CV Score (MSE)
// 0.1       | 0.0542  ← Undersmoothed
// 0.3       | 0.0231  ← Best
// 0.5       | 0.0298
// 0.7       | 0.0412  ← Oversmoothed

fastloess::LoessOptions cv_opts;
cv_opts.cv_fractions = {0.1, 0.3, 0.5, 0.7};
cv_opts.cv_method = "kfold";
cv_opts.cv_k = 5;
fastloess::Loess model(cv_opts);
auto result = model.fit(x, y).value();

// Fraction with lowest CV score is automatically selected.
// 0.1 → 0.0542  ← Undersmoothed
// 0.3 → 0.0231  ← Best
// 0.5 → 0.0298
// 0.7 → 0.0412  ← Oversmoothed

The fraction with lowest CV score is automatically selected.

Availability¶

Batch Mode Only

Cross-validation is only available in Batch mode.

Feature	Batch	Streaming	Online
K-Fold CV	✓	✗	✗
LOOCV	✓	✗	✗

Best Practices¶

Test a range: Include fractions from 0.1 to 0.9
Use enough folds: 5-10 folds balance speed and accuracy
Set a seed: For reproducible results
Check the curve: CV optimizes MSE, but visual inspection matters