Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unittest errors with new version of sklearn #413

Open
isaacmg opened this issue Sep 27, 2021 · 2 comments
Open

Unittest errors with new version of sklearn #413

isaacmg opened this issue Sep 27, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@isaacmg
Copy link
Collaborator

isaacmg commented Sep 27, 2021

As many of you know sklearn recently upgraded to 1.0 from 0.24. The newer version generate two errors related to failing unit tests. These should be investigated and fixed.

raceback (most recent call last):
  File "/home/circleci/repo/tests/test_deployment.py", line 39, in test_infer_multi
    save_name="tes1/t2.csv", wandb_plot_id="prod_plot")
  File "/home/circleci/repo/flood_forecast/deployment/inference.py", line 106, in make_plots
    df, tensor, history, forecast_start, test, samples = self.infer_now(date, csv_path, csv_bucket, save_name)
  File "/home/circleci/repo/flood_forecast/deployment/inference.py", line 71, in infer_now
    unscaled = test.inverse_scale(tensor.numpy())
  File "/home/circleci/repo/flood_forecast/preprocessing/pytorch_loaders.py", line 150, in inverse_scale
    self.targ_scaler.inverse_transform(result_data_np)
  File "/usr/local/lib/python3.7/site-packages/sklearn/preprocessing/_data.py", line 1022, in inverse_transform
    force_all_finite="allow-nan",
  File "/usr/local/lib/python3.7/site-packages/sklearn/utils/validation.py", line 788, in check_array
    % (array.ndim, estimator_name)
ValueError: Found array with dim 3. StandardScaler expected <= 2.

Traceback (most recent call last):
  File "/home/circleci/repo/tests/pytorc_train_tests.py", line 304, in test_scaling_data
    inverse_scale = self.model.test_data.inverse_scale(scaled_src[:, 0])
  File "/home/circleci/repo/flood_forecast/preprocessing/pytorch_loaders.py", line 150, in inverse_scale
    self.targ_scaler.inverse_transform(result_data_np)
  File "/usr/local/lib/python3.7/site-packages/sklearn/preprocessing/_data.py", line 1022, in inverse_transform
    force_all_finite="allow-nan",
  File "/usr/local/lib/python3.7/site-packages/sklearn/utils/validation.py", line 765, in check_array
    "if it contains a single sample.".format(array)
ValueError: Expected 2D array, got 1D array instead:
array=[0.6595671  0.6595671  0.64500946 0.6595671  0.68504316 0.6959614
 0.7359951  0.7760289  0.81606257 0.870654   0.9288848  0.9834762
 1.0671831  1.1545293  1.2564334  1.3619767  1.482078   1.6240156
 1.7659533  1.9115304 ].
Reshape your data either using ar
@isaacmg isaacmg added the bug Something isn't working label Sep 27, 2021
@isaacmg
Copy link
Collaborator Author

isaacmg commented Sep 27, 2021

I think they likely added additional checks on data dimensions which is why this is failing.

@isaacmg
Copy link
Collaborator Author

isaacmg commented Sep 30, 2021

See #414

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant