2025 / UC Berkeley (ME249)
First-Principles vs Keras Neural Networks
Benchmarking custom optimization against RMSprop
MLRegressionOptimizationScientific ML
Keras predictions vs data
Overview
Compared a hand-coded Newton-style solver against a Keras implementation on the same physical system dataset to understand convergence behavior and error profiles.
Problem
How do custom first-principles optimization routines compare to modern deep learning optimizers for the same regression task?
Role
Course project (ME249)
Timeline
Oct 2025
Tools
Python / NumPy / TensorFlow/Keras
Data
- Physical system dataset provided in course notebooks
- Identical initialization and activation functions across pipelines
Approach
- Implemented a Newton-style update loop with manual step scheduling
- Trained a Keras model with RMSprop using matched architecture
- Compared parameter drift and prediction error across runs
Evaluation
- Normalized MAE reported ~0.012 for RMSprop vs higher baseline
- Prediction error stayed within ~1.1°C across test cases
Results
- Keras converged faster with lower sensitivity to initialization
- First-principles solver offered transparent gradient auditing
Deployment
- Reusable scripts for reproducible training runs
- Logs document sensitivity to starting weights
Limitations
- Small dataset with controlled test cases
- Single architecture comparison
Evidence
Keras predictions vs data
First-principles predictions vs data
Repro Steps
- Run keras_training.py or first_principles.py in project2
- Outputs saved to project2/out
Next Steps
- Expand to larger datasets and additional optimizers
- Add regularization to stabilize first-principles updates