Improve code generation performance

[overbit]

• Improve build-manager by leveraging zip copy to prepare dsg-jobs output to artefact folder
• Add cache on dsg generation steps
  • cache based on on hash of dsg-spec
  • cache based on hash of entity + pluginInstallation spec
  • ..

[abrl91]

• separate between Admin and Server and add the ability to choose whether to build both or one of them
• choose the service/services to build, in case there is more than one service in the project

[mulygottlieb]

• parallel the generation of different entities
• parallel the generation of files within each entity

In terms of paralleling things

• Use promise.all() in order not to wait promise after promise
• Use Node.js worker threads (thread pool) e.g. divide the entities to X groups and each thread generates the files for its group
• Same as above but with subprocess (create small executables that generate per entity, etc. and run them from an orchestrator "main" DSG)

[mulygottlieb]

In order to test which areas in the DSG are good candidates for improvement, it is advisable to create a very large build and observe the timestamps on the DSG logs in order to identify steps that take a long time and are, therefore, good candidates for improvement attempts.
The easiest way to get to a long build is to import a very large number of entities. We have some ready examples for such files; please ask me if you need such a schema file you can import to generate 200+ entities and observe a long DSG build.

[mulygottlieb]

It is advisable to assist in the following "event tree" to remind us of areas we can consider parallelizing:

[mulygottlieb]

• Split AdminUI and Server creation - Use Redis distributed cache for the Build Manager to sync the responses of both DSG jobs
• Once we add Redis to our Architecture, we can use it in the DSG as well - either to cache some of the results (generated modules) and/or to cache the DSG Context instead of an in-memory Singelton