Deployment Task on Fail

[johncadengo]

Regarding this file: https://github.com/dokku/dokku/blob/master/docs/advanced-usage/deployment-tasks.md

I'm working on using the postdeploy task to notify a slack channel of a successful deploy. Is there a way to know that dokku failed to deploy, and therefore notify slack of a failed deploy?

[josegonzalez]

We don't currently have a way of doing this - I think maybe your CI pipeline would be a better place to put such a failure message, given that any point in the Dokku deploy may exit non-zero.

The alternative might be to have a wrapper function at /usr/local/bin/dokku that calls /usr/bin/dokku and then a script on failure, but that would catch more than just deploys (we could do this with other scripts as well so as to avoid introducing another path to check for the dokku bin).

Can you tell me a bit more about your particular use case? That will help me figure out the best way to handle this sort of request :)

[johncadengo]

@josegonzalez thanks for letting me know.

The use case is that my deploys to dokku take quite a bit of time. I'm using Rails 5/6 with webpacker, and precompiling assets for my apps take awhile (sometimes 3-4 minutes) even on relatively fast server with multiple cores. It seems that a lot of the webpacker dependencies don't take advantage of multiple cores and rely on single-threaded performance.

One of the issues I've run into in the past is my deploy might fail somewhere along the way—like during a migration, when a config variable is not set properly, installing dependencies, etc.—and not having an automated way to know that was the case. So I'll run a deploy, step away from my computer for a few minutes, and come back to see that it failed 30 seconds into the deploy.

From the docs on deployment tasks, I've been able to set up a slack notification upon successful deployment of an app, using the different phases in app.json, but I would like to be able to notify slack as well if the deployment fails, so I can look at the error message, attempt to fix it, and try again.

I can see that perhaps with a more sophisticated CI pipeline, I could find a solution with a different method. But it seems intuitive that the phases in app.json might include a hook for a failed deploy, and a hook for the start of the deploy as well. The best use case for me is that I would be able to notify slack at the start of a deploy, and if it succeeded, or if it failed.

[josegonzalez]

That makes sense. I think the best way around this would be like...

• Move /usr/bin/dokku to /usr/bin/dokku-core
• Create a /usr/bin/dokku that can parse arguments, execute /usr/bin/dokku with the same args, and can figure out when a command for the git plugin comes in
  • If one comes in and it fails, try to execute the failure task

Such a binary would also be a great place to add global logging (so ship all log output to a file), but might take a bit of work to figure out. Seems reasonable to me.

[josegonzalez]

If a deploy fails, what image should we use for that failure task? Which app.json should we use (the one from the previous deploy, or the code being pushed) to extract the command?

[johncadengo]

I'd recommend using the app.json from the code being pushed, as it's likely to be the user's expectation for which failure task to run. For example, if you're updating the failure task to account for a new slack endpoint, which has an updated token because the old token has been revoked, you wouldn't be able to see that the task is failed unless the new one is used. Also, if you're just creating a failure task at all, you wouldn't be able to see that the task is failed unless you're using the latest app.json.

Is there a use case for using the previous deploy's version?

[josegonzalez]

Which image would be the one we use for the deployment task? All deployment tasks must be run from an image. We could use the herokuish image in this case, but I fear that might be weird for developers that expect certain environments for their tasks.

[johncadengo]

I see. If we're using the herokuish image, would we still have access to the files from our repo in the deploy?

Would the keys be something like scripts.deployfail or scripts.dokku.deployfail?

It seems if we're using the herokuish image, we wouldn't be able to run custom scripts from our image during the dokku phase of things.

Would something like this be able to work?

{
  "scripts": {
    "dokku": {
      "deployfail": "./custom-script.sh",
    },
  }
}

[josegonzalez]

No we wouldn't have access to the repository files. It would be hard to say they would be useful. For instance, what would happen if the files depend on app dependencies but those dependencies failed to install?

[johncadengo]

I see. Would we have access to env vars?

[josegonzalez]

Yeah we could do that for sure. Basically it would be the same as any other deploy task except with a specific image. We could even have a special arg for that image, so that users could ensure a specific environment for the failure task to run in.

[johncadengo]

Ok, in that case herokuish image sounds fine to me. Not sure what expectations others might have. My original intent for accessing repository files would be to run a custom script, but that can be handled with a curl command and access to some env vars.