Skip to content

Commit

Permalink
Use deterministic flushing when building web demo .rrd files (#3064)
Browse files Browse the repository at this point in the history
### What

I saw this in #3063
<img width="482" alt="Screenshot 2023-08-22 at 13 40 24"
src="https://github.com/rerun-io/rerun/assets/1148717/dd741d41-641d-41a2-89d1-77311b719879">

and spotted several problems:

A) there was nothing in that PR changing the .rrd files
B) the comparison format uses way too many decimals
C) the comparison uses "KB" which isn't a unit

The first problem should be fixed by turning off time-based batch
flushing (first commit).

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3064) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/3064)
- [Docs
preview](https://rerun.io/preview/pr%3Aemilk%2Fdeterministic-web-demo/docs)
- [Examples
preview](https://rerun.io/preview/pr%3Aemilk%2Fdeterministic-web-demo/examples)
- [Recent benchmark results](https://ref.rerun.io/dev/bench/)
- [Wasm size tracking](https://ref.rerun.io/dev/sizes/)

---------

Co-authored-by: Jan Procházka <1665677+jprochazk@users.noreply.github.com>
  • Loading branch information
emilk and jprochazk committed Aug 22, 2023
1 parent 2f2aaca commit 168c8f5
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 14 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/reusable_track_size.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,12 @@ jobs:
run: |
entries=()
entries+=("Wasm:web_viewer/re_viewer_bg.wasm:MB")
entries+=("JS:web_viewer/re_viewer.js:KB")
entries+=("Wasm:web_viewer/re_viewer_bg.wasm:MiB")
entries+=("JS:web_viewer/re_viewer.js:kiB")
for file in web_demo/examples/**/*.rrd; do
name=$(basename $(dirname "$file"))
entries+=("$name.rrd:$file:KB")
entries+=("$name.rrd:$file:MiB")
done
data=$(python3 scripts/ci/sizes.py measure "${entries[@]}")
Expand Down
8 changes: 7 additions & 1 deletion scripts/ci/build_demo_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,18 @@ def save(self) -> None:
f"--save={rrd_path}",
]

# Configure flushing so that:
# * the resulting file size is deterministic
# * the file is chunked into small batches for better streaming
env = {**os.environ, "RERUN_FLUSH_TICK_SECS": "1000000000", "RERUN_FLUSH_NUM_BYTES": str(128 * 1024)}

subprocess.run(
args + self.build_args,
env=env,
check=True,
)

print(f"{rrd_path}: {os.path.getsize(rrd_path) / 1e6:.1f} MB")
print(f"{rrd_path}: {os.path.getsize(rrd_path) / (1024 * 1024):.1f} MiB")

def supports_save(self) -> bool:
with open(self.path) as f:
Expand Down
20 changes: 10 additions & 10 deletions scripts/ci/sizes.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"Wasm (release)":web_viewer/re_viewer_bg.wasm \
"Wasm (debug)":web_viewer/re_viewer_debug_bg.wasm
python3 scripts/ci/sizes.py measure --format=table \
python3 scripts/ci/sizes.py measure --format=github \
"Wasm (release)":web_viewer/re_viewer_bg.wasm \
"Wasm (debug)":web_viewer/re_viewer_debug_bg.wasm
Expand All @@ -30,7 +30,7 @@


def get_unit(size: int | float) -> str:
UNITS = ["B", "KB", "MB", "GB", "TB"]
UNITS = ["B", "kiB", "MiB", "GiB", "TiB"]

unit_index = 0
while size > 1024:
Expand All @@ -42,19 +42,19 @@ def get_unit(size: int | float) -> str:

DIVISORS = {
"B": 1,
"KB": 1024,
"MB": 1024 * 1024,
"GB": 1024 * 1024 * 1024,
"TB": 1024 * 1024 * 1024 * 1024,
"kiB": 1024,
"MiB": 1024 * 1024,
"GiB": 1024 * 1024 * 1024,
"TiB": 1024 * 1024 * 1024 * 1024,
}


def get_divisor(unit: str) -> int:
return DIVISORS[unit.upper()] or 1
return DIVISORS[unit]


def cell(value: float, div: float) -> str:
return str(round(value / div, 3))
return f"{value / div:.2f}"


def render_table_dict(data: list[dict[str, str]]) -> str:
Expand Down Expand Up @@ -130,7 +130,7 @@ def compare(previous_path: str, current_path: str, threshold: float) -> None:
name,
f"{cell(previous, div)} {unit}",
f"{cell(current, div)} {unit}",
sign + str(change) + "%",
f"{sign}{change:.2f}%",
)
)
elif "current" in entry:
Expand Down Expand Up @@ -162,7 +162,7 @@ def measure(files: list[str], format: Format) -> None:
output.append(
{
"name": name,
"value": str(round(size / div, 3)),
"value": str(round(size / div, 2)),
"unit": unit,
}
)
Expand Down

0 comments on commit 168c8f5

Please sign in to comment.