-
Notifications
You must be signed in to change notification settings - Fork 285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add some notes to these models, especially the corresponding paper links #443
Comments
Yeah this should probably be updated on the ReadMe as well but Vanilla LSTM->LSTM: A basic LSTM that is suitable for multivariate time series forecasting and transfer learning. So it is called DecoderTransformer in the model_dict |
Thank you for your reply. I have another question for you.
In paper Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting, there is such a saying "We propose convolutional self-attention by employing causal convolutions to produce queries and keys in the self-attention layer." Is this the code "self.query_key = nn.Conv1d(n_embd, n_embd * n_head * 2, self.q_len).cuda()self.value = Conv1D(n_embd * n_head, 1, n_embd)"corresponding to "causal revolutions"?
Is this the code corresponding to "LogSparse Transformer"?
…------------------ 原始邮件 ------------------
发件人: "AIStream-Peelout/flow-forecast" ***@***.***>;
发送时间: 2021年11月13日(星期六) 凌晨2:29
***@***.***>;
***@***.******@***.***>;
主题: Re: [AIStream-Peelout/flow-forecast] Add some notes to these models, especially the corresponding paper links (Issue #443)
Yeah this should probably be updated on the ReadMe as well but
Vanilla LSTM->LSTM: A basic LSTM that is suitable for multivariate time series forecasting and transfer learning. Full transformer: The full original transformer with all 8 encoder and decoder blocks. Requires passing the target in at inference. Simple Multi-Head Attention->MultiAttnHeadSimple: A simple multi-head attention block and linear embedding layers. Suitable for transfer learning. Transformer with a linear decoder->CustomTransformerDecoder: A transformer with n-encoder blocks (this is tunable) and a linear decoder. DA-RNN:->ARNN A well rounded model with which utilizes a LSTM + attention. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting -> DecoderTransformer: Transformer XL (not fully supported yet): Informer -> "Informer": Beyond Efficient Transformer for Long Sequence Time-Series Forecasting DeepAR -> DeepAR (only available on certain branches"
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
|
Could you add some notes to these models, especially the corresponding paper links? For example, in this paper, Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, I don't know which model to use.
Thank you.
pytorch_model_dict = {
"MultiAttnHeadSimple": MultiAttnHeadSimple,
"SimpleTransformer": SimpleTransformer,
"TransformerXL": TransformerXL,
"DummyTorchModel": DummyTorchModel,
"LSTM": LSTMForecast,
"SimpleLinearModel": SimpleLinearModel,
"CustomTransformerDecoder": CustomTransformerDecoder,
"DARNN": DARNN,
"DecoderTransformer": DecoderTransformer,
"BasicAE": AE,
"Informer": Informer
}
The text was updated successfully, but these errors were encountered: