Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Major] increase default batch size (deleted) #1460

Closed
wants to merge 4 commits into from
Closed

[Major] increase default batch size (deleted) #1460

wants to merge 4 commits into from

Conversation

ourownstory
Copy link
Owner

@ourownstory ourownstory commented Oct 19, 2023 •
edited

To reduce the tendency to overfit, we are inreasing the default batch size.

  • increase batch size 2-fold w.r.t log10(data size)
  • while increasing min and max 2-fold

@ourownstory ourownstory changed the title [Major] increase batch size 4-fold, while increasing min and max 2-fold [Major] increase default batch size Oct 19, 2023
@codecov
Copy link

codecov bot commented Oct 19, 2023 •
edited

Codecov Report

Merging #1460 (7e4cc7b) into main (f08dcf8) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1460   +/-   ##
=======================================
  Coverage   88.32%   88.32%           
=======================================
  Files          38       38           
  Lines        5097     5097           
=======================================
  Hits         4502     4502           
  Misses        595      595
Files Coverage Δ
neuralprophet/configure.py 90.18% <100.00%> (ø)

@github-actions
Copy link

github-actions bot commented Oct 19, 2023 •
edited

Model Benchmark

Benchmark Metric main current diff
PeytonManning_test30 MAE_val - 0.9336 -
PeytonManning_test30 RMSE_val - 1.0896 -
PeytonManning_test30 Loss_val - 0.0296 -
PeytonManning_test30 MAE - 0.3319 -
PeytonManning_test30 RMSE - 0.4694 -
PeytonManning_test30 Loss - 0.0044 -
PeytonManning_test30 time - 35.35 -
YosemiteTemps MAE_val 1.34899 0.48759 -63.86% 🎉
YosemiteTemps RMSE_val 2.00817 0.70387 -64.95% 🎉
YosemiteTemps Loss_val 0.00078 8e-05 -89.13% 🎉
YosemiteTemps MAE 1.32133 0.89641 -32.16% 🎉
YosemiteTemps RMSE 2.13713 1.63549 -23.47% 🎉
YosemiteTemps Loss 0.00064 0.00033 -47.81% 🎉
YosemiteTemps time 67.05385 121.6 81.35% ❌
YosemiteTemps_test20 MAE_val - 0.5776 -
YosemiteTemps_test20 RMSE_val - 1.024 -
YosemiteTemps_test20 Loss_val - 0.0002 -
YosemiteTemps_test20 MAE - 0.8838 -
YosemiteTemps_test20 RMSE - 1.56 -
YosemiteTemps_test20 Loss - 0.0003 -
YosemiteTemps_test20 time - 102.75 -
AirPassengers_test30 MAE_val - 26.9238 -
AirPassengers_test30 RMSE_val - 32.3881 -
AirPassengers_test30 Loss_val - 0.0083 -
AirPassengers_test30 MAE - 5.5114 -
AirPassengers_test30 RMSE - 7.1496 -
AirPassengers_test30 Loss - 0.0003 -
AirPassengers_test30 time - 19.13 -
AirPassengers MAE_val 13.0627 30.0386 129.96% ❌
AirPassengers RMSE_val 15.94532 30.8862 93.7% ❌
AirPassengers Loss_val 0.00131 0.00368 180.31% ❌
AirPassengers MAE 9.88153 5.88831 -40.41% 🎉
AirPassengers RMSE 11.73543 7.65149 -34.8% 🎉
AirPassengers Loss 0.00052 0.00018 -65.57% 🎉
AirPassengers time 6.15807 19.23 212.27% ❌
PeytonManning MAE_val 0.58162 0.34903 -39.99% 🎉
PeytonManning RMSE_val 0.72218 0.50014 -30.75% 🎉
PeytonManning Loss_val 0.01239 0.00597 -51.79% 🎉
PeytonManning MAE 0.41671 0.35143 -15.66% 🎉
PeytonManning RMSE 0.55961 0.48785 -12.82% 🎉
PeytonManning Loss 0.00612 0.00461 -24.6% 🎉
PeytonManning time 15.41275 39.97 159.33% ❌
Model training plots

Model Training

PeytonManning

PeytonManning_test30

YosemiteTemps

YosemiteTemps_test20

AirPassengers

AirPassengers_test30

@ourownstory ourownstory linked an issue Oct 19, 2023 that may be closed by this pull request
Improve auto-epoch and auto-batch size #1132
Open
@ourownstory ourownstory changed the title [Major] increase default batch size [Major] increase default batch size (deleted) Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Improve auto-epoch and auto-batch size
1 participant
@ourownstory