How to make systems really multithreaded?

[robo-todd]

First, I want to say that flecs is amazing. I'm using it for non-game applications and I think it is a revolution in how I can build code. I am trying (learning) how to make my system highly biased towards being entity parallel, and I'm starting to use multi threaded systems.

I have turned on multiple threads (4 in my test case) at the top of the loop, and I have some multi-threaded systems that have very heavy processing in their eval() functions. I had hoped, that each such multi-threaded system would run on a worker thread, but even though they are marked multi_threaded() they just run sequentially on the same thread (printing out the thread id in the eval call...).
There are only a small number of entities (approx the number of threads) that match these systems, and the system query does use/follow relationships to pull in inputs (read), but the output data are updating separate entities in eval.

Any pointers on how to really make the systems run on multiple threads? I've read through the docs quite a few times and I think I'm doing it right, but no luck. They don't seem to get scheduled to run on separate threads.

This would be tremendously powerful if I can get it working or understand what would make a multi_threaded() system not run multi threaded. Perhaps another way to ask the question is what would cause a multi-threaded system to be run single threaded?

I'm using C++ and I'm building from flecs v3.2.10 release code.

[copygirl]

Just to make sure, you also called world.set_threads(), right? (As mentioned in the docs here.) I believe systems will only be run simultaneously if they don't access the same components with write access, that means marking them as [in] in queries.

[robo-todd]

One more bit of information: Essentially the same behavior on v3.2.11.

[robo-todd]

This is just a ping. @SanderMertens to see if you have any thoughts?

[SanderMertens]

Hey! Sorry for the late reply, I've been busy with an upcoming release. What I think is happening is that the threads just don't have enough to do to fully utilize parallelism- there is quite a bit of context switching overhead which means that for a handful of systems/entities it may not be worth it.

There are known improvements that can reduce the scheduler overhead, but those might take a while to land.

[robo-todd]

Thanks for getting back to me.

So that is not all I'm worried about. In my actual setups I have been assuming all the entities in a given pipeline phase are processed before entities are processed in the next pipeline phase. It is very hard to make a minimal repro example for a full system. The above traces have entities overlapping in phases and I don't quite know how to reason about that from a design perspective. I can't match up the documentation with the actual behavior so I feel like I don't understand how this works or how to use it.

In my actual application, systems access multiple input entities (pull in data from a query and some traversal of relationships) and update output components assuming that entities are updated in pipeline order. If they are not updated in pipeline order then I don't know how to use the scheduler to design something effective.

[SanderMertens]

Phases are not synchronization boundaries. The scheduler checks for system component access + read/write annotations for components that aren't matched by the system query, and uses that to insert sync points. (see this part of the docs for more information: https://www.flecs.dev/flecs/md_docs_2Systems.html#sync-points and this example: https://github.com/SanderMertens/flecs/blob/master/examples/cpp/systems/sync_point/src/main.cpp)

If the scheduler detects no conditions that require a mid-frame sync, it will only happen at the end of the frame. You can see when syncs happen if you connect the explorer to your application, and open the pipeline view.