Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelizable hangs in certain cases #740

Open
elijahbenizzy opened this issue Mar 5, 2024 · 0 comments
Open

Parallelizable hangs in certain cases #740

elijahbenizzy opened this issue Mar 5, 2024 · 0 comments
Labels
parallelism triage label for issues that need to be triaged.

Comments

@elijahbenizzy
Copy link
Collaborator

Short description explaining the high-level reason for the new issue.

Current behavior

Hangs on this:

import logging

from hamilton import driver
from hamilton.execution.executors import SynchronousLocalTaskExecutor
from hamilton.htypes import Parallelizable, Collect

def initial() -> int:
    return 0


def parallelize_over(initial: int) -> Parallelizable[int]:
    for i in range(2):
        yield i


def internal(parallelize_over: int, initial: int) -> int:
    return parallelize_over + 1 + initial


def collect(internal: Collect[int], initial: int) -> int:
    return sum(internal) + 1


def final(collect: int) -> int:
    return collect + 1


if __name__ == "__main__":
    import __main__

    from hamilton.log_setup import setup_logging
    setup_logging(log_level=logging.DEBUG)

    dr = (
        driver
        .Builder()
        .with_modules(__main__)
        .enable_dynamic_execution(allow_experimental_mode=True)
        .with_local_executor(SynchronousLocalTaskExecutor())
        .build()
    )
    dr.visualize_execution(final_vars=["final"], inputs={}, output_file_path="parallelism.png")
    dr.execute(["final"])
@elijahbenizzy elijahbenizzy added triage label for issues that need to be triaged. parallelism labels Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parallelism triage label for issues that need to be triaged.
Projects
None yet
Development

No branches or pull requests

1 participant