Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/token function #277

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

Feat/token function #277

wants to merge 27 commits into from

Conversation

MMenchero
Copy link
Contributor

Description:

Implemented new method for token information retrieval.

Changes:

  • Added new request_df method to TimeGPT class.
  • Added supporting methods and minor changes.
  • Supports main methods from TimeGPT class: forecast, detect_anomalies, and cross_validation.

Testing:

See tests at the end of nb.

Notes:

  • Need to remove temporary fix to a minor issue when calling the cross_validation method.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@MMenchero MMenchero requested a review from AzulGarza April 2, 2024 06:23
Copy link
Contributor

github-actions bot commented Apr 3, 2024

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 4.167 3.3738 0.0097 0.0053

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.9678 8.9966 0.0065 0.0056

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 4.7311 4.3503 0.0084 0.0073

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 7.2099 3.992 0.0084 0.0077

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805038 441118 1.61572e+06 6.04619e+06
total_time 12.5244 4.7556 0.0084 0.0078

Plot:

Copy link
Contributor

github-actions bot commented Apr 3, 2024

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.5608 3.5339 0.0091 0.0074

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 7.4293 15.3817 0.0056 0.005

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.8 123119 213677 4.68961e+06
total_time 9.3882 7.3983 0.0078 0.0069

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966295 422332 656723 3.17316e+06
total_time 11.1683 8.745 0.0077 0.0073

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805038 441118 1.61572e+06 6.04619e+06
total_time 9.6867 6.4091 0.0077 0.007

Plot:

Copy link
Contributor

github-actions bot commented Apr 3, 2024

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 4.2414 4.5077 0.0094 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 5.7826 7.8683 0.0064 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.8 123119 213677 4.68961e+06
total_time 9.4784 7.2296 0.0089 0.0079

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 8.2987 10.2321 0.0084 0.0079

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 10.1406 5.535 0.0086 0.0082

Plot:

Copy link
Member

@AzulGarza AzulGarza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @MMenchero! awesome progress! 🙌

i left a couple of comments. i think solving them will prevent the ci/cd from crashing. :)

res = response["data"]
res["requestID"] = response["requestID"]
local_time = time.localtime(time.time())
created_at = time.strftime("%Y-%m-%d %H:%M:%S", local_time)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we can use created_at = pd.Timestamp.now().strftime("%Y-%m-%d %H:%M:%S")), instead of importing time. probably this creates a compatibility issue with tenacity that explains the ci/cdm but i'm not completely sure.

@@ -889,7 +1066,7 @@ def _forecast(
):
if validate_api_key and not self.validate_api_key(log=False):
raise Exception("API Key not valid, please email ops@nixtla.io")
nixtla_client_model = _NixtlaClientModel(
self.nixtla_client_model = _NixtlaClientModel(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably it might be better not to store _NixtlaClientModel to prevent the object from being heavy. We could use logic similar to self.weights_x. here, _NixtlaClientModel stores weights_x and then NixtlaClient recovers that object. in the cases of the requests_df, _NixtlaClientModel might be storing that dataframe and then NixtlaClient appends new dataframes to the previously created one, something like:

class NixtlaClient(_NixtlaClient):
     requests_df: pd.DataFrame = pd.DataFrame(columns=[...]) # add columns of request_df

     def _forecast(...):
          nixtla_client_model = _NixtlaClientModel(...)
          self.request_df = pd.concat([
              self.request_df, 
             nixtla_client_model.request_df], ignore_index=True,
          )

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.1328 4.4132 0.009 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 6.7054 9.2113 0.0062 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 4.3342 8.3783 0.0086 0.0078

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.0356 6.2586 0.0084 0.0077

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.0142 5.6327 0.0085 0.0079

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.9354 6.4594 0.0103 0.0058

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 12.3057 3.2406 0.0065 0.0057

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 6.2116 4.7569 0.0084 0.0074

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.7584 6.674 0.0088 0.0088

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 5.9458 4.9508 0.0084 0.008

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.6912 5.9895 0.009 0.0054

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 7.7088 3.5901 0.0061 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 5.3668 5.7567 0.0084 0.0076

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 4.0966 8.4653 0.0081 0.0076

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.556 4.7745 0.0087 0.0079

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.0095 2.7405 0.0099 0.0058

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 2.7184 2.9373 0.0063 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 3.2151 3.7486 0.0087 0.009

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 4.2893 3.635 0.0085 0.0108

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.7626 11.3515 0.0086 0.0083

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 2.6035 2.3018 0.0092 0.0056

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.0631 4.3259 0.0063 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 3.0722 4.5437 0.0085 0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966295 422332 656723 3.17316e+06
total_time 5.2125 4.2181 0.0081 0.0075

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.1176 4.7093 0.0082 0.0077

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 2.7341 2.533 0.0095 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 4.3224 7.006 0.0066 0.0057

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 9.0968 6.7385 0.0087 0.0077

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.2187 6.3749 0.0086 0.0088

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.4553 4.9063 0.0085 0.0082

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 4.5938 3.2094 0.0103 0.0062

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.7404 6.9138 0.0062 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 8.2648 5.2583 0.0086 0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 5.9645 4.9266 0.0082 0.0076

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 7.4739 6.5832 0.0083 0.0076

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 4.6874 7.1259 0.0087 0.0053

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 9.8397 5.6781 0.0061 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 8.2602 8.3371 0.0087 0.0078

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.9183 4.671 0.0083 0.0077

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 7.1381 7.4159 0.0085 0.0077

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 5.637 13.1157 0.0095 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.1434 3.6444 0.0062 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 5.0692 8.2879 0.0086 0.0105

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.3219 3.6343 0.0085 0.0078

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 5.6794 5.7169 0.0085 0.0079

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.1336 4.0482 0.0095 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 6.9877 6.7907 0.0062 0.0055

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 5.1373 4.8255 0.0088 0.0076

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 7.2528 5.6403 0.0084 0.0079

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.1073 5.9913 0.0086 0.0079

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 5.7987 5.4488 0.0092 0.0053

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 6.1744 5.1086 0.0063 0.0055

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 3.8501 6.7565 0.0087 0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 8.4011 6.6929 0.0082 0.0077

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 7.5764 6.4494 0.0084 0.0078

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 8.3732 9.9708 0.009 0.0054

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.5803 5.5793 0.0058 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 4.248 8.1314 0.0082 0.0076

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 7.0817 4.359 0.0076 0.0073

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.6856 7.3261 0.0078 0.0075

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 5.2629 7.2848 0.0092 0.0048

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 8.4738 7.7945 0.0057 0.0047

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 10.3384 10.8062 0.0078 0.0069

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 9.7669 10.0022 0.0076 0.0071

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 12.081 10.8004 0.0078 0.0075

Plot:

@MMenchero
Copy link
Contributor Author

This PR is ready for review @AzulGarza

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants