Good to get to hang with you at Coalesce in San Diego @jelstongreen 😎

Thanks for reporting this. We've had a few places where either the current working directory or relative paths throw a wrench into things, so I'll give this a look.

I see what you are saying @jelstongreen 👍

When I tried it out a python script using programmatic invocations like you described, deps changed the current working directory whenever the PROJECT_DIR global config was set via the --project-dir flag or the DBT_PROJECT_DIR environment variable.

In contrast, all of the following leave the current working directory as-is (as expected):

• debug
• parse
• compile
• list
• run
• build

Also in contrast, running the following without a python script within a CLI does not change the current working directory:

pwd
dbt deps --project-dir project_a
pwd

mkdir project_a

name: "package_a"
version: "1.0.0"
config-version: 2
profile: "sandcastle"

from dbt.cli.main import dbtRunner, dbtRunnerResult
import os

# initialize
dbt = dbtRunner()

# Get and print the current working directory
cwd_before = os.getcwd()
print("Current working directory before:", cwd_before)

# "deps" modifies the current working directory when the PROJECT_DIR config is set
# Either via DBT_PROJECT_DIR enviornment variable or --project-dir CLI flag
cli_args = ["deps", "--project-dir", "project_a"]

# Run the command
res: dbtRunnerResult = dbt.invoke(cli_args)

cwd_after = os.getcwd()
print("Current working directory after:", cwd_after)

# Check if the current working directory has changed or not
assert cwd_after == cwd_before, "Current working directory has changed!"

python runner.py

Hey there @dbeatty10 thank you for taking a look and good news that you were able to reproduce. Is this likely to be fixed in an imminent release?

I encountered the same issue and thoroughly investigated the code. I found directroy changes when initializing deps task.
I'll write down what I found.

1. When deps command is executed via dbtRunner().invoke command, DepsTask class is initialized in dbt-core.core.dbt.cli.main.py. This class stores the command context and functions for deps command.
   https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/cli/main.py#L492

2. When initializing the class, directory changes via move_to_nearest_project_dir.
   https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/task/deps.py#L100

I confirmed it works same as the behaviour of dbt deps by modifying code of deps function as follows.

def deps(ctx, **kwargs):
    ...
    cur_dir = os.getcwd()
    task = DepsTask(flags, ctx.obj["project"])
    results = task.run()
    success = task.interpret_results(results)
    os.chdir(cur_dir)
    return results, success

I also confirmed that the same problem happens when executing dbt clean by python because the directory also changes when initializing CleanTask.
https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/task/clean.py#L24

If this is okay for you, I'll create a PR but what do you think about? I'm concerning about this will become breaking changes.

@rariyama Thanks for taking a look!

We're calling this a bug, because this directory switching after-effect of deps/clean is really the intended behavior. So I'd be supportive of fixing this, even if it means changing the behavior (and potentially breaking someone's workflow).

It feels like move_to_nearest_project_dir could become a context manager, that returns to the original working directory once the deps'ing / clean'ing / init'ing is complete. But very happy to hear if you have other ideas for how you'd go about fixing this :)

Hi @jtcohen6
Thank you for your feedback. I feel your idea is SGTM.
What do you think about to handle task class as a context manager? My code image is as follows.

# deps.py
class DepsTask(BaseTask):
    def __init__(self, args: Any, project: Project) -> None:
        ...
        project.project_root = str(Path(project.project_root).resolve())

        self.orig_dir = os.getcwd()
        move_to_nearest_project_dir(project.project_root)
        super().__init__(args=args, config=None, project=project)
        self.cli_vars = args.vars

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        os.chdir(self.orig_dir)
    ...

# main.py
def deps(ctx, **kwargs):
    ...
    with DepsTask(flags, ctx.obj["project"]) as task:
        results = task.run()
        success = task.interpret_results(results)
    return results, success
....

I confirmed the directory change occurs only within with statement by making the above change.