Problem with sktime data when using Google Colab, but not when running the same code in a local Jupyter notebook.

[emammendes]

Hello,

I want to share the results of processing some data using sktime with a colleague of mine using Google Colab. The Jupyter notebook on my local machine runs smoothly without any glitches. However, when I try to run it on Google Colab, I encounter an error.

Here is the piece of code where things go wrong:

# Creating nested DataFrame for sktime X_nested = pd.DataFrame(index=np.arange(n_samples), columns=[which_var]) for i in range(n_samples): X_nested.iloc[i, 0] = pd.Series(df[which_var].iloc[i:i+window_size].values)

When I issue sktime.datatypes.check_raise(X_train, 'nested_univ') on the local jupyter notebook, python returns True. Unfortunately that is not the answer when issuing the same command on the google colab notebook.

---

TypeError Traceback (most recent call last)
in <cell line: 2>()
1 # ['nested_univ', 'numpy3D', 'numpyflat', 'pd-multiindex', 'pd-wide', 'pd-long', 'df-list', 'dask_panel']
----> 2 sktime.datatypes.check_raise(X_train, 'nested_univ')

/usr/local/lib/python3.10/dist-packages/sktime/datatypes/_check.py in check_raise(obj, mtype, scitype, var_name)
269 return True
270 else:
--> 271 raise TypeError(msg)
272
273

TypeError: input entries must be pd.Series

What am I missing?

Many thanks.

[fkiraly]

That's really hard to tell without the actual code.

A general comment I can add on this, the nested_univ datatype - the nested data container - is becoming increasingly glitchy and unstable across environment configurations and operating systems since pandas 2, due to changes in how nested structures are handled, and lack of official support.

We therefore recommend to avoid the nested_univ data container specification where possible, and instead work with numpy3D or pd-multiindex specifications.

For instance, the line X_nested.iloc[i, 0] = pd.Series(df[which_var].iloc[i:i+window_size].values) works on some pandas versions, but I think it does not work the same on newer ones, as the behaviour changed from "putting into cell" to "attempt broadcast overwrite".

Would different pandas versions explain the difference?

You can show pandas and other relevant versions by running show_versions(), imported via from sktime import show_versions.

[emammendes]

Many thanks for the quick reply.

It could be a version issue. On the local machine, the version of pandas is 2.2.2, whereas on Google Colab it is 2.0.3. So, I believe it will be the other way around: the line works with the newer version of pandas but not with the older ones.

Could you kindly point out where I can find an example that uses either numpy3D or pd-multiindex to build a data container from a dataframe originating from a CSV file? (That is, to do the same as the line mentioned in our message).

[fkiraly]

You are probably looking for the data container intro notebook, see here in binder:
https://mybinder.org/v2/gh/sktime/sktime/main?filepath=examples

The notebook AA_datatypes_and_datasets, section 4 - loading data from csv files, 4.2 and 4.3 has examples for pd-multiindex.

[emammendes]

Many thanks!