[CLI]: Large numbers in parameter section of sweep config breaks sweep by repeating runs

[HannesStagge]

Describe the bug

I encountered some repeated runs when running grid hyperparameter studies. See
here for another example.
@luisbergua and @QianqianF this might possibly be interesting for you too?

I tracked it to large exponential numbers (1-1e15) in my sweep configs parameter section.

The config that reproduces the error:

sweep_config = {
        "method": "grid",
        "name": this_file.stem,
        "metric": {"goal": "minimize", "name": "test/loss"},
        "parameters": {
            "param_A": {"values": [1, 3, 4, 5]},
            "param_B": {
                "values": [
                    1,
                    10,
                    100,
                    1e3,
                    1e4,
                    1e5,
                    1e6,
                    1e7,
                    1e8,
                    1e9,
                    1e10,
                    1e11,
                    1e12,
                    1e13,
                    1e14,
                    1e15,
                ]
            },
        },
    }

as opposed to

sweep_config = {
        "method": "grid",
        "name": this_file.stem,
        "metric": {"goal": "minimize", "name": "test/loss"},
        "parameters": {
            "param_A": {"values": [1, 3, 4, 5]},
            "param_B": {
                "values": [
                    0,
                    1,
                    2,
                    3,
                    4,
                    5,
                    6,
                    7,
                    8,
                    9,
                    10,
                    11,
                    12,
                    13,
                    14,
                    15,
                ]
            },
        },
    }

working fine.

Parallel coordinates for the faulty case:

Note the missing runs for param_A=[3,4,5]
And for the correct case:

Heres the complete code for the faulty and correct case

Additional Files

No response

Environment

WandB version: 0.16.2

OS: Windows-10-10.0.22631-SP0

Python version: 3.10.12

Versions of relevant libraries:
torch: 2.0.0
lightning: 2.0.9

Additional Context

I am running the agents on multiple cores with multiprocessing, but that does not seem to influence the workings from my experience.

[thanos-wandb]

Hi @HannesStagge thank you so much for the detailed report. Investigating this further and will get back to you with any updates.

[thanos-wandb]

@HannesStagge we've reproduced this issue on our end, and reported to our engineering teams. We will keep you posted with any updates. Thanks for reporting!