Ray-2.24.0

Ray Libraries

Ray Data

🎉 New Features:

• Allow user to configure timeout for actor pool (#45508)
• Add override_num_blocks to from_pandas and perform auto-partition (#44937)
• Upgrade Arrow version to 16 in CI (#45565)

💫 Enhancements:

• Clarify that num_rows_per_file isn't strict (#45529)
• Record more telemetry for newly added datasources (#45647)
• Avoid pickling LanceFragment when creating read tasks for Lance (#45392)

Ray Train

📖 Documentation:

• [HPU] Add example of Stable Diffusion fine-tuning and serving on Intel Gaudi (#45217)
• [HPU] Add example of Llama-2 fine-tuning on Intel Gaudi (#44667)

Ray Tune

🏗 Architecture refactoring:

• Improve excessive syncing warning and deprecate TUNE_RESULT_DIR, RAY_AIR_LOCAL_CACHE_DIR, local_dir (#45210)

Ray Serve

💫 Enhancements:

• Clean up Serve proxy files (#45486)

📖 Documentation:

• vllm example to serve llm models (#45430)

RLLib

💫 Enhancements:

• DreamerV3 on tf: Bug fix, so it can run again with tf==2.11.1 (2.11.0 is not available anymore) (#45419); Added weekly release test for DreamerV3.
• Added support for multi-agent off-policy algorithms (DQN and SAC) in the new (#45182)
• Config option for APPO/IMPALA to change number of GPU-loader threads (#45467)

🔨 Fixes:

• Various MetricsLogger bug fixes (#45543, #45585, #45575)
• Other fixes: #45588, #45617, #45517, #45465

📖 Documentation:

• Example script for new API stack: How-to restore 1 of n agents from a checkpoint. (#45462)
• Example script for new API stack: Autoregressive action module. #45525

Ray Core

💫 Enhancements:

• Improve node death observability (#45320, #45357, #45533, #45644, #45497)
• Ray c++ backend structured logging (#44468)

🔨 Fixes:

• Fix worker crash when getting actor name from runtime context (#45194)
• log dedup should not dedup number only lines (#45385)

📖 Documentation:

• Improve doc for --object-store-memory to describe how the default value is set (#45301)

Dashboard

🔨 Fixes:

• Move Job package uploading to another thread to unblock the event loop. (#45282)

Many thanks to all those who contributed to this release: @maxliuofficial, @simonsays1980, @GeneDer, @dudeperf3ct, @khluu, @justinvyu, @andrewsykim, @Catch-Bull, @zcin, @bveeramani, @rynewang, @angelinalg, @matthewdeng, @jjyao, @kira-lin, @harborn, @hongchaodeng, @peytondmurray, @aslonnie, @timkpaine, @982945902, @maxpumperla, @stephanie-wang, @ruisearch42, @alanwguo, @can-anyscale, @c21, @Atry, @KamenShah, @sven1977, @raulchen