Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

random test failures on older CPUs without SSE/AVX/AVX2/AVX512 #316

Open
pabs3 opened this issue Jul 10, 2022 · 1 comment
Open

random test failures on older CPUs without SSE/AVX/AVX2/AVX512 #316

pabs3 opened this issue Jul 10, 2022 · 1 comment

Comments

@pabs3
Copy link

pabs3 commented Jul 10, 2022

Describe the bug
There are strange random test failures on older CPUs that don't have SSE/AVX/AVX2/AVX512.

To Reproduce
Steps to reproduce the behavior:

  1. Build SPTAG on a machine without AVX/AVX2/AVX512.
  2. Run SPTAGTests
  3. See error

Expected behavior
The tests should work.

Analysis
I think that because of the -mavx2 -mavx -msse -msse2 -mavx512f -mavx512bw -mavx512dq options in the DistanceUtils target_compile_options, the compiler is generating newer instructions in the DistanceUtils library and these are not run on older CPUs. Removing the options not supported by the CPU and deleting the functions using instructions that those options enable fixes this issue. So the cause is definitely the options being enabled.

Suggestions
On Linux you can use GCC function multi-versioning to get the compiler to automatically check the CPU at runtime and dispatch to the right functions.

Screenshots
Some examples of the random failures:

[1] Start invoking BuildTrees.
[1] BKTKmeansK: 3, BKTLeafSize: 6, Samples: 100, BKTLambdaFactor:-1.000000 TreeNumber: 1, ThreadNum: 2.
unknown location(0): fatal error: in "SSDServingTest/TestHeadUInt8L2DEFAULT": memory access violation at address: 0x00000000: no mapping at fault address
./Test/src/SSDServingTest.cpp(444): last checkpoint: "TestHeadUInt8L2DEFAULT" test entry

*** 1 failure is detected in the test module "Main"
[1] Parallel TpTree Partition done
[1] Build TPTree time (s): 4
[1] Processing Tree 0 0%
unknown location(0): fatal error: in "AlgoTest/KDTTest": signal: illegal operand; address of failing instruction: 0x559a5eccc130
./Test/src/AlgoTest.cpp(22): last checkpoint

*** 1 failure is detected in the test module "Main"

Desktop:

  • OS: Debian
  • Browser None
  • Version 12 (bookworm)
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 10, 2022
Prevents crashes on older CPUs without wider SIMD instructions.

Fixes: microsoft#316
@pabs3
Copy link
Author

pabs3 commented Jul 12, 2022

If someone were to fix up and merge #136 that fixes #134 requesting support for 64-bit ARM, then this issue would be fixed too.

This was referenced Jul 12, 2022
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 12, 2022
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 12, 2022
Prevents crashes on older CPUs without wider SIMD instructions.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 12, 2022
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 18, 2022
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 18, 2022
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 18, 2022
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 26, 2022
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 26, 2022
pabs3 added a commit to pabs3/SPTAG that referenced this issue Nov 13, 2022
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Nov 13, 2022
pabs3 added a commit to pabs3/SPTAG that referenced this issue Feb 5, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Feb 5, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Feb 6, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Mar 1, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Mar 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Mar 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Mar 25, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Mar 25, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Mar 25, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Apr 1, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 10, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 10, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 10, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 11, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 11, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 11, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue May 11, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 16, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 16, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 16, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 16, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 16, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 16, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 25, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 25, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 25, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 25, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 27, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 27, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 27, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jun 27, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 20, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 20, 2023
Prevents crashes on older CPUs without wider SIMD instructions.

Workaround lack of target-specific template specialisation in GCC.

Fixes: microsoft#316
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 20, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 20, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 20, 2023
pabs3 added a commit to pabs3/SPTAG that referenced this issue Jul 20, 2023
dbussink pushed a commit to planetscale/SPTAG that referenced this issue Mar 13, 2024
dbussink pushed a commit to planetscale/SPTAG that referenced this issue Mar 14, 2024
dbussink pushed a commit to planetscale/SPTAG that referenced this issue Mar 14, 2024
Obsoletes: microsoft#136
Fixes: microsoft#134
Fixes: microsoft#316
Enabled-by: simd-everywhere/simde#961
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant