Experiments#

The experiments verify that the end-to-end pipeline works and that the neural network can learn the Black-Scholes pricing function.

The main neural network metrics are:

  • Mean Absolute Error (MAE);

  • Root Mean Squared Error (RMSE);

  • coefficient of determination \(R^2\);

  • MAPE on prices greater than 1.

The MAPE restriction avoids unstable percentage errors for options whose theoretical price is close to zero.

Diagnostic Plots#

The project produces the following plots:

  • training and validation loss;

  • Black-Scholes price versus neural network prediction;

  • distribution of prediction errors;

  • absolute error versus moneyness;

  • absolute error versus maturity;

  • absolute error versus volatility.

Final Experiment#

The final experiment artifacts selected for the report are stored in results/final/. This directory tracks the configuration snapshot, metrics, diagnostic figures, and verification notes, while generated datasets and model checkpoints remain reproducible but untracked in data/final_improved/ and outputs/final_improved/.

The final run uses:

  • 100,000 synthetic option contracts;

  • the engineered feature set with_moneyness;

  • the SiLU hidden-layer activation function;

  • up to 200 training epochs with early stopping;

  • 50,000 Monte Carlo paths;

  • a deterministic Monte Carlo evaluation subset of 512 test options.

The final neural network metrics against analytical Black-Scholes prices are:

Metric

Value

MAE

0.0428955592

RMSE

0.0620807954

\(R^2\)

0.9999927878

MAPE, price > 1

0.4802504182%

Additional Experiments#

The codebase already supports two controlled experimental switches:

  • --feature-set with_moneyness to include s0 / k as an engineered model input;

  • --activation to compare hidden-layer activation functions such as ReLU, Tanh, LeakyReLU, SiLU, and GELU.

The repository also includes a reduced-scale Support Vector Regression benchmark through scripts/run_svr_benchmark.py. SVR is evaluated on smaller synthetic datasets and over multiple random seeds because kernel methods do not scale as naturally to the full 100,000-sample setting used by the neural network experiment.

The noisy-target robustness experiment is available through scripts/run_noisy_targets_experiment.py. It perturbs only the training labels with controlled Gaussian noise and evaluates predictions against the clean analytical Black-Scholes prices. This keeps the experiment focused on robustness to imperfect labels rather than on market-price modeling.

The same controlled noisy-target setting is also repeated for the reduced-scale SVR baseline through scripts/run_noisy_svr_benchmark.py. This provides a classical ML robustness reference while keeping SVR experiments computationally bounded.

The runtime comparison is available through scripts/benchmark_runtime.py. It separates neural-network training time from pricing time and compares analytical Black-Scholes evaluation, neural-network inference, and Monte Carlo simulation.

The figures used in the report are generated with scripts/generate_report_figures.py from tracked experiment summaries. This keeps the report figures reproducible without committing intermediate plotting work.

Future Extensions#

Possible future extensions include:

  • increasing the synthetic dataset size;

  • comparing different hidden layer widths and depths;

  • measuring how the error changes with moneyness, maturity, and volatility;

  • estimating Greeks through automatic differentiation;

  • extending the pricing setup to stochastic-volatility, path-dependent, or American-style derivatives;

  • using real exchange-traded option data, which would change the research question from approximating Black-Scholes to modeling market prices.