Skip to content
This repository has been archived by the owner on May 8, 2024. It is now read-only.

Release 0.5.0

Latest
Compare
Choose a tag to compare
@youngsofun youngsofun released this 27 Jul 04:05
· 128 commits to master since this release

API change

  • Remove module-level api like dpark.textFile.
  • Support Streaming shuffle and Disk shuffle (Experimental, compatible).

Fixes

  • Bug when parsing mfs chunk info.

Improvement

  • Better broadcast impl using shared memory for tasks on the same slave to reduce memory cost.
  • Better offer-matching logic for MesosScheduler which remember bad slaves.
  • Refactor: style and layout.

New Feature

  • Multi segment dump to save memory.
  • Gather statics for stage.
  • Support run tests/test_rdd on mesos.
  • Add colorful progress bar for dpark.
  • Support mesos role.
  • Support multi named mesos master in conf.
  • Loghub for admin.