Caching base estimators during HPO for meta estimator in StackingCVRegressor

[Ryednap]

Could you kindly advise whether it's feasible to cache the predictions of the base estimator from the initial fit to explore subsequent Hyperparameter Optimization (HPO) trials? This would involve focusing solely on tuning the parameters of the meta-estimator, assuming the base estimators are already optimized.

Currently, the HPO pipeline takes around 5 minutes per iteration for 4-fold cross-validation, which is slower than desired. I've ensured that the seed for the HPO is set, resulting in consistent splits and predictions by the base estimators across estimations. Only the parameters of the meta-estimator vary.

My HPO pipeline is based on Optuna. Any insights or suggestions would be greatly appreciated. Thank you.