gctest hang on msys64 if mprotected-based incremental collection is on

[ivmai]

Source: master (2b30780)
Build: https://ci.appveyor.com/project/ivmai/bdwgc/builds/49679393/job/c6p5a9x5oxe7vrci
Host: Windows x64
AppVeyor image: Visual Studio 2022
Config: CFLAGS_EXTRA=-O3 -march=native -D GC_THREADS -D GC_GCJ_SUPPORT -D GC_TIME_LIMIT=10 -D WINXP_USE_PERF_COUNTER -D NO_MSGBOX_ON_ERROR -D NO_TEST_ENDTHREADEX

[ivmai]

Might be related: #573

[ivmai]

Not reproducible with -D NO_MPROTECT_VDB

[ivmai]

Output:

./gctest
Got 65536 bytes from OS
Got 65536 bytes from OS
Got 65536 bytes from OS
In-use heap: 0% (0 KiB pointers + 0 KiB other)
Got 65536 bytes from OS
Grow heap to 64 KiB after 0 bytes allocated
Grew heap sections array to 32 elements
Initiating full world-stop collection!
0 bytes in heap blacklisted for interior pointers

--> Marking for collection #1 after 0 allocated bytes
Pushed 1 thread stacks
World-stopped marking took 6 ms 434801 ns (6 ms in average)
Bytes recovered before sweep - f.l. count = 0
GC #1 freed 0 bytes, heap 64 KiB (+ 0 KiB unmapped + 362 KiB internal)
In-use heap: 0% (0 KiB pointers + 0 KiB other)
0 finalization entries; 0/0 short/long disappearing links alive
0 finalization-ready objects; 0/0 short/long links cleared
Finalize and initiate sweep took 0 ms 29000 ns + 0 ms 42900 ns
Complete collection took 6 ms 624600 ns
Initiating full world-stop collection!
0 bytes in heap blacklisted for interior pointers

--> Marking for collection #2 after 0 allocated bytes
Pushed 1 thread stacks
World-stopped marking took 0 ms 824600 ns (3 ms in average)
Bytes recovered before sweep - f.l. count = 0
GC #2 freed 0 bytes, heap 64 KiB (+ 0 KiB unmapped + 362 KiB internal)
In-use heap: 0% (0 KiB pointers + 0 KiB other)
0 finalization entries; 0/0 short/long disappearing links alive
0 finalization-ready objects; 0/0 short/long links cleared
Finalize and initiate sweep took 0 ms 400 ns + 0 ms 22401 ns
Complete collection took 0 ms 951900 ns
Supported VDBs: manual mprotect
Initializing mprotect virtual dirty bit implementation
Switched to incremental mode
Emulating dirty bits with mprotect/signals (covering also ptr-free pages)
Adding block map for size of 3 granules (48 bytes)
...
--> Marking for collection #1867 after 464608 allocated bytes
Marked from 53 dirty pages
Pushed 5 thread stacks
World-stopped marking took 2 ms 437600 ns (1 ms in average)
Bytes recovered before sweep - f.l. count = 0
GC #1867 freed 0 bytes, heap 4032 KiB (+ 6016 KiB unmapped + 874 KiB internal)
In-use heap: 43% (1123 KiB pointers + 616 KiB other)
0 finalization entries; 0/0 short/long disappearing links alive
0 finalization-ready objects; 0/0 short/long links cleared
Finalize and initiate sweep took 0 ms 35000 ns + 0 ms 227400 ns
Finished second reverse_test at time 5032 (0x7ff7fc988)
Finished 0x7ff7fc988

[ivmai]

Reproduced locally with
make -j -f Makefile.direct CC=gcc CFLAGS_EXTRA='-Werror -Wall -Wextra -Wpedantic -O3 -march=native -D GC_THREADS -D GC_GCJ_SUPPORT -D GC_TIME_LIMIT=10 -D WINXP_USE_PERF_COUNTER -D NO_MSGBOX_ON_ERROR -D NO_TEST_ENDTHREADEX' check