Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/changes current working dir when using a dbt project dir #9596

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

rariyama
Copy link

resolves #8997
I closed the previous PR by mistake, then I submit this one.

Problem

There is a difference in behaviour between dbt cli and dbt.invoke.
For example, When executing dbtRunner().invoke(["deps", "--project-dir", "project_dir"]) in python, directory
changes to project_dir, even though it doesn't affect when executing dbt deps.

Solution

The reason why directory changes is dbt.task.base.move_to_nearest_project_dir() is called in CleanTask , DepsTask and InitTask . Directory changes to the argument of the function.
It'll be possible to resolve this problem by using those classes as a context manager. The change will happen only in with statement.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
  • This PR includes type annotations for new and modified functions

@rariyama rariyama requested a review from a team as a code owner February 19, 2024 05:17
@cla-bot cla-bot bot added the cla:yes label Feb 19, 2024
Copy link
Contributor

@ChenyuLInx ChenyuLInx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this @rariyama
I wonder what if you can add the __enter__ and __exit__ to just the base task so we only do it once?

Also tagging @MichelleArk to see if there are any thoughts on starting this with pattern. It looks good to me.

@rariyama
Copy link
Author

Hi @ChenyuLInx . Thanks for your comments.
I agree with your idea, then I updated my code as follows on this commit 31ba979.

  • Added __enter__ and __exit__ to BaseTask
  • each child TaskClass inherits that class and behaves as a context manager.

I would like to add test code for this update. I feel like it's good enough to add the following code to integration test but what do you think about?

    def test_dbtRunner_invoke(self):
        pd = "<test-project-path>"
        dr = dbtRunner()
        commands = ["deps", "clean", "init"]

        for command in commands:
            req = [command, "--project-dir", pd, "--profiles-dir", pd]
            before_dir = os.getcwd()
            res = dr.invoke(req)
            self.assertEqual(res.success, True)
            # MEMO: Test whether directory is not changed after command is invoked.

            self.assertEqual(before_dir, os.getcwd())

@dbeatty10 dbeatty10 added the community This PR is from a community member label Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes community This PR is from a community member
Projects
None yet
3 participants