Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does ggml_cont really work? #735

Open
BayRanger opened this issue Feb 17, 2024 · 0 comments
Open

Does ggml_cont really work? #735

BayRanger opened this issue Feb 17, 2024 · 0 comments

Comments

@BayRanger
Copy link

BayRanger commented Feb 17, 2024

Hi, I try to perform softmax at the last dimension of a tensor. So I need to permute it first, and perform softmax at every n continguous elements. To permute the tensor, I think the expected code in ggml, should be like
auto _tensor = ggml_cont(ctx, ggml_permute(ctx,input, 1,2,0,3));
But I print the content of content of _tensor via float* data = ggml_get_data_f32(input); , the content of _tensor is all zero, later I use ggml_cont_inplace instead of ggml_cont, the data looks the same as in "input", which is not what I expected.

In pytorch, when using contiguous() after permutation, the raw data layout would be changed, that is what I expected. In ggml, while the raw data would not be changed using ggml_cont... So further explaination is wanted about the design intention of ggml_cont,... Furthermore, could someone show how to perform softmax. at a specific dimension of a tensor in ggml ? like in following code dense = np.exp(input) # Softmax. dense = dense / (np.sum(dense, axis=0)+.00001) # Should sum to 1.

@BayRanger BayRanger changed the title Does ggml_cont really works? Does ggml_cont really work? Feb 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant