Add macOS pkg installer to deployment (#7554) #7555

paulober · 2023-06-08T15:23:54Z

Fixes Add macOS pkg installer #7554

paulober · 2023-06-08T15:30:10Z

See the connected issue for more details on the implementation

paulober · 2023-09-04T10:58:55Z

@vilmibm @williammartin Any updates on this?

williammartin · 2023-09-11T15:54:40Z

@paulober Sorry that your PR is languishing in review purgatory, when I went on parental leave I asked @vilmibm to take ownership but he has since left GitHub. I have not forgotten about it and we are currently figuring out how to deal with our PR backlog with team churn.

paulober · 2024-05-12T15:37:43Z

Will my work now finally after nearly one year be accepted and merged, or should i just delete it and distribute a gh pkg for macOS on my own?

(You are currently maintaining 12 artifacts for linux with 3 different packages per one of the 4 supported architectures.
And macOS people still have to install it manually from a zip folder or rely on a heavy third-party tool.)

paulober · 2024-05-20T15:08:30Z

@andyfeller Hi, if you find some time may you review this PR for merging please?

andyfeller · 2024-05-20T20:35:53Z

@andyfeller Hi, if you find some time may you review this PR for merging please?

@paulober : 🫡 Firstly, let me say thank you for putting the time and effort into this as well as your patience given the impact attrition has taken on the team over the past year.

I'm blocking off time first thing tomorrow morning to review this. Not having a background with .pkg packaging, I also want to compare this to efforts in GitHub Desktop and Mobile areas to compare as a colleague of mine has been working at refactoring how we build and sign within the Mac ecosystem.

andyfeller

@paulober : again thanks for your patience especially as my background in packaging is predominantly OSS with RPMs and DEBs.

few minor nits and some honest questions I'd appreciate your thoughts on while running down hubbers on GitHub Desktop and Mobile to understand these changes in context to those product lines. additionally, I still want to ensure the CLI team has an opportunity to review these.

build/macOS/distribution.xml

andyfeller · 2024-05-21T12:58:05Z

build/macOS/distribution.xml

@@ -0,0 +1,19 @@
+<?xml version="1.0" encoding="utf-8" standalone="yes"?>
+<installer-gui-script minSpecVersion="1">


For documentation on this file, see Apple Distribution Definition reference

Right, i think we can change the minSpecVersion to 2. But i'll check that first.

Just to be clear, I'm mostly adding reference comments because completely unfamiliar with the spec and I use comments like sticky notes for myself 😅

oh, sorry. didn't know that.

andyfeller · 2024-05-21T13:24:05Z

build/macOS/distribution.xml

+    <title>Github Cli</title>
+    <license file="LICENSE" mime-type="text/plain"/>
+    <options hostArchitectures="arm64,x86_64" customize="never" require-scripts="false" allow-external-scripts="false"/>
+    <domains enable_localSystem="true"/>


Should GitHub CLI be system-wide, per-user, or can be anywhere?

GitHub Desktop zips up the application and on start up will ask the user if they want to move it from Downloads (in my case) to Applications.

Reference

@williammartin : would appreciate your 2 cents here as brew installs packages such that any user on the workstation can add /opt/homebrew to their path where the configuration is per user.

I think I'd be inclined to say /usr/local/bin. When I look at the contents of my /usr/local/bin it feels like a familiar idea. Homebrew seems to have a variety of reasons relating to Apple Silicon to use /opt/homebrew.

I would anticipate the vast majority of users:

Don't feel the need to use amd64 and arm versions of gh

Have a single user account on their personal machine

So I think I would suggest we sidestep that mess and install into /usr/local/bin, knowing that it's on the path already.

What do you think?

build/macOS/distribution.xml

andyfeller · 2024-05-21T13:32:53Z

build/macOS/distribution.xml

+    <pkg-ref id="com.github.cli.pkg" auth="Root" packageIdentifier="com.github.cli">
+        <bundle-version/>
+    </pkg-ref>


What is the impact of this being empty versus being managed from package data?

Reference

Defines the version of the bundles delivered by the parent element. You do not typically specify this element; productbuild inserts it from the actual package data when the product archive is created.

I just checked the code i wrote one year ago and removed some of this xml as it's not needed.
The pkg-ref bundle-version part is generated by productbuild and it also uses this empty bundle-version as the real version can be looked up by installer in the included pkg or the generated named version property on pkg-ref.

script/pkgmacos

andyfeller · 2024-05-21T14:31:30Z

script/pkgmacos

+    cp "LICENSE" "build/macOS/resources"
+    touch .gi
+
+    # build distribution


For information on productbuild, see Apple "Packaging Mac software for distribution"

script/pkgmacos

andyfeller · 2024-05-21T14:36:40Z

script/pkgmacos

+while [ $# -gt 0 ]; do
+  case "$1" in
+  -h | --help )
+    print_help
+    exit 0
+    ;;
+  -* )
+    printf "unrecognized flag: %s\n" "$1" >&2
+    exit 1
+    ;;
+  * )
+    tag_name="$1"
+    shift 1
+    ;;
+  esac
+done


Are there any conditions we should short circuit running this script?

Running it on unsupported OS

If pkgbuild or productbuild aren't installed

...

Not really required but it's better.
I adopted your suggested requirements and as unsupported OS i set all macOS versions prior to macOS 12 (Monterey).
For Linux it's not accounted for as it would exit with code 127 at the start anyways as sw_vers is not available on Linux.

Do you think that's sufficient for this build script or should i also write a safe fail for Linux?

.goreleaser.yml

paulober · 2024-05-21T16:17:02Z

@andyfeller Hi, if you find some time may you review this PR for merging please?

@paulober : 🫡 Firstly, let me say thank you for putting the time and effort into this as well as your patience given the impact attrition has taken on the team over the past year.

I'm blocking off time first thing tomorrow morning to review this. Not having a background with DMG packaging, I also want to compare this to efforts in GitHub Desktop and Mobile areas to compare as a colleague of mine has been working at refactoring how we build and sign within the Mac ecosystem.

Thanks for taking the time to review. I'll address all comments.
Refactoring the macOS target build and signing process sounds like a good project, looking forward to seeing the result.

Co-authored-by: Andy Feller <andyfeller@github.com>

andyfeller · 2024-05-21T16:26:01Z

The only product tangentially relevant to GitHub that built .pkg was Mac OS Git installer.

Co-authored-by: Andy Feller <andyfeller@github.com>

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

paulober · 2024-05-21T23:47:20Z

Does gh cli has a minimum supported macOS version? If so, a script could be added to the installer to check for minimum macOS version.

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

andyfeller · 2024-05-23T19:41:47Z

Does gh cli has a minimum supported macOS version? If so, a script could be added to the installer to check for minimum macOS version.

If we look at the current gh homebrew formula, Monterey is the oldest of current versions supported now. If we go back to 1.0.0 it was High Sierra.

As we're not backporting older versions, I assume to error on the side of latest supported Mac OS versions

andyfeller · 2024-05-23T20:22:51Z

.github/workflows/deployment.yml

+      - name: Build universal macOS pkg installer
+        if: inputs.environment != 'production'
+        env:
+          TAG_NAME: ${{ inputs.tag_name }}
+        run: script/pkgmacos "$TAG_NAME"


How have you been testing script/pkgmacos locally?

I was wondering how large the new .pkg files were, so tried building it locally. I thought about trying to build this locally only to realize we probably want to add a new target to Makefile as this involves multiple steps.

Here's how far I've got so far before realizing I should simply ask 😅

andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ ./script/pkgmacos To build a universal pkg for macOS: script/pkgmacos <tag-name> To build and sign set APPLE_DEVELOPER_INSTALLER_ID environment variable before. For example, if you have a signing identity with the identifier "Developer ID Installer: Your Name (ABC123DEF)" set it in the variable. andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ ./script/pkgmacos v0.0.0 build_pkg:7: no matches found: ./share/man/man1/gh*.1 andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ vim Makefile andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ make go build -trimpath -ldflags "-X github.com/cli/cli/v2/internal/build.Date=2024-05-23 -X github.com/cli/cli/v2/internal/build.Version=v2.49.2-50-g9454d5e7 " -o bin/gh ./cmd/gh andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ ./script/pkgmacos v0.0.0 build_pkg:7: no matches found: ./share/man/man1/gh*.1 andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ vim Makefile andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ make manpages go run ./cmd/gen-docs --man-page --doc-path ./share/man/man1/ andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ ./script/pkgmacos v0.0.0

Concerning the pkg size, its about 24MB:
-rw-r--r--@ 1 paulober staff 24M May 24 14:10 gh_v2.99.8_macOS_universal.pkg

To test it locally:

make manpages make completions ./script/release --local "v2.99.8" --platform macos ./script/pkgmacos v2.99.8

And for the Makefile target. Maybe i can read into it. But as of now i know nothing about the syntax within a Makefile (only cmake).

I take that back, it wasn't that difficult. You can now build a pkg using make macospkg VERSION=v2.99.8. (goreleaser required)

this is really going to help the team, thank you for saving us future time 🫶

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

paulober · 2024-05-24T17:25:50Z

I just wanted to let you know that I have now finished all the suggested changes as well as my own changes that I wanted to incorporate.

andyfeller · 2024-05-24T18:36:09Z

I just wanted to let you know that I have now finished all the suggested changes as well as my own changes that I wanted to incorporate.

thanks for the high sign ✋ just to set expectations for you, I'm going to devote time this afternoon to review these before extended Memorial Day weekend. Being back on Wednesday, I want you to know this is something I've committed internally to wrap this up by next week at the latest.

andyfeller · 2024-05-24T18:43:54Z

Makefile

+
+.PHONY: macospkg
+macospkg: manpages completions
+ifndef VERSION
+	$(error VERSION is not set. Use `make macospkg VERSION=vX.Y.Z`)
+endif
+	./script/release --local "$(VERSION)" --platform macos
+	./script/pkgmacos $(VERSION)


Checking with and without VERSION ❤️

andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ make macospkg go run ./cmd/gen-docs --man-page --doc-path ./share/man/man1/ go build -trimpath -ldflags "-X github.com/cli/cli/v2/internal/build.Date=2024-05-24 -X github.com/cli/cli/v2/internal/build.Version=v2.49.2-64-g4db87793 " -o bin/gh ./cmd/gh mkdir -p ./share/bash-completion/completions ./share/fish/vendor_completions.d ./share/zsh/site-functions bin/gh completion -s bash > ./share/bash-completion/completions/gh bin/gh completion -s fish > ./share/fish/vendor_completions.d/gh.fish bin/gh completion -s zsh > ./share/zsh/site-functions/_gh Makefile:100: *** VERSION is not set. Use `make macospkg VERSION=vX.Y.Z`. Stop. andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer›$ make macospkg VERSION=v0.0.0 go run ./cmd/gen-docs --man-page --doc-path ./share/man/man1/ build.go: `bin/gh` is up to date. mkdir -p ./share/bash-completion/completions ./share/fish/vendor_completions.d ./share/zsh/site-functions bin/gh completion -s bash > ./share/bash-completion/completions/gh bin/gh completion -s fish > ./share/fish/vendor_completions.d/gh.fish bin/gh completion -s zsh > ./share/zsh/site-functions/_gh ./script/release --local "v0.0.0" --platform macos goreleaser release -f .goreleaser.generated.yml --clean --skip-validate --skip-publish --release-notes=$TMPDIR/tmp.6ZAch4nuYr • starting release... • loading path=.goreleaser.generated.yml • DEPRECATED: --skip-publish was deprecated in favor of --skip=publish, check https://goreleaser.com/deprecations#-skip for more details • DEPRECATED: --skip-validate was deprecated in favor of --skip=validate, check https://goreleaser.com/deprecations#-skip for more details • skipping announce, publish and validate... • loading environment variables • getting and validating git state • couldn't find any tags before "v0.0.0" • git state commit=4db87793cd2315a18a33a9b56e6234b594b8b16a branch=feature-macos-pkg-installer current_tag=v0.0.0 previous_tag=<unknown> dirty=false • pipe skipped reason=validation is disabled • parsing tag • setting defaults • DEPRECATED: archives.rlcp should not be used anymore, check https://goreleaser.com/deprecations#archivesrlcp for more info • DEPRECATED: archives.rlcp should not be used anymore, check https://goreleaser.com/deprecations#archivesrlcp for more info • DEPRECATED: archives.rlcp should not be used anymore, check https://goreleaser.com/deprecations#archivesrlcp for more info • running before hooks • running hook= make manpages GH_VERSION=0.0.0 • running hook=echo make completions • took: 2s • checking distribution directory • cleaning dist • loading go mod information • build prerequisites • writing effective config file • writing config=dist/config.yaml • building binaries • building binary=dist/macos_darwin_arm64/bin/gh • building binary=dist/macos_darwin_amd64_v1/bin/gh • running hook hook=./script/sign '/Users/andyfeller/Documents/workspace/cli/cli/dist/macos_darwin_arm64/bin/gh' skipping macOS code-signing; APPLE_DEVELOPER_ID not set • running hook hook=./script/sign '/Users/andyfeller/Documents/workspace/cli/cli/dist/macos_darwin_amd64_v1/bin/gh' skipping macOS code-signing; APPLE_DEVELOPER_ID not set • took: 9s • generating changelog • loaded "/var/folders/y5/q89b76cj7ws55x3trcyllf_80000gn/T/tmp.6ZAch4nuYr", but it is empty • archives • creating archive=dist/gh_0.0.0_macOS_amd64.zip • creating archive=dist/gh_0.0.0_macOS_arm64.zip • took: 3s • calculating checksums • storing release metadata • writing file=dist/artifacts.json • writing file=dist/metadata.json • you are using deprecated options, check the output above for details • release succeeded after 14s • thanks for using goreleaser! ./script/pkgmacos v0.0.0 pkgbuild: Inferring bundle components from contents of pkg_payload pkgbuild: Wrote package to ./dist/com.github.cli.pkg skipping macOS pkg code-signing; APPLE_DEVELOPER_INSTALLER_ID not set productbuild: Wrote product to ./dist/gh_v0.0.0_macOS_universal.pkg

Existing Mac OS release artifacts use the tag name / version in the file name but drop the `v` prefix. This does the same for the Mac OS installer.

andyfeller · 2024-05-24T19:12:29Z

script/pkgmacos

+    exit 1
+    ;;
+  * )
+    tag_name="${1#v}"


@paulober : just a heads up, I pushed up a small change to strip the leading v so the resulting artifact matched the others 👍

andyfeller@Andys-MBP:cli/cli ‹feature-macos-pkg-installer*›$ ll dist total 103056 -rw-r--r-- 1 andyfeller staff 1.2K May 24 15:09 artifacts.json -rw-r--r-- 1 andyfeller staff 4.0K May 24 15:09 config.yaml -rw-r--r-- 1 andyfeller staff 182B May 24 15:09 gh_0.0.0_checksums.txt -rw-r--r-- 1 andyfeller staff 13M May 24 15:09 gh_0.0.0_macOS_amd64.zip -rw-r--r-- 1 andyfeller staff 12M May 24 15:09 gh_0.0.0_macOS_arm64.zip -rw-r--r-- 1 andyfeller staff 24M May 24 15:09 gh_0.0.0_macOS_universal.pkg drwxr-xr-x 3 andyfeller staff 96B May 24 15:09 macos_darwin_amd64_v1 drwxr-xr-x 3 andyfeller staff 96B May 24 15:09 macos_darwin_arm64 -rw-r--r-- 1 andyfeller staff 211B May 24 15:09 metadata.json

andyfeller · 2024-05-24T19:22:49Z

Demo of using local build from make macospkg

Screenshare.-.2024-05-24.3_20_36.PM.mp4

andyfeller

I think this is good to go with our next release being the ultimate test of any problems.

paulober · 2024-05-24T19:51:35Z

Thanks for taking the time to bring the PR of the finish line. Lets see how it works with the next release. I also though about maybe adding a second install script that removes old gh man pages in case some get removed with a new release they wouldn't remain on the users disk if they upgrade with a new pkg...

paulober requested a review from a team as a code owner June 8, 2023 15:23

paulober requested review from williammartin and removed request for a team June 8, 2023 15:23

cliAutomation added the external pull request originating outside of the CLI core team label Jun 8, 2023

cliAutomation added this to Needs review 🤔 in The GitHub CLI Jun 8, 2023

paulober force-pushed the feature-macos-pkg-installer branch from 3abfe1a to a2e6927 Compare June 8, 2023 15:27

williammartin added blocked discuss Feature changes that require discussion primarily among the GitHub CLI team labels Jun 8, 2023

williammartin requested review from vilmibm and removed request for williammartin June 29, 2023 19:23

Add macOS pkg installer to deployment

f1c3534

paulober force-pushed the feature-macos-pkg-installer branch from a2e6927 to f1c3534 Compare July 7, 2023 10:01

paulober added 2 commits September 4, 2023 12:59

Merge branch 'trunk' into feature-macos-pkg-installer

53c83fd

Fix typo in macOS pkg installer build script

775476d

williammartin requested review from williammartin and removed request for vilmibm September 11, 2023 15:57

samcoe removed the discuss Feature changes that require discussion primarily among the GitHub CLI team label Oct 30, 2023

cli deleted a comment Nov 1, 2023

Merge branch 'trunk' into feature-macos-pkg-installer

630ab13

andyfeller reviewed May 21, 2024

View reviewed changes

Update pkg title

b1a6ce3

Co-authored-by: Andy Feller <andyfeller@github.com>

Update choice title

aeb0ac7

Co-authored-by: Andy Feller <andyfeller@github.com>

paulober and others added 7 commits May 21, 2024 19:39

Indentation fix

eb23e07

Co-authored-by: Andy Feller <andyfeller@github.com>

Fix indentation

f9d6b1d

Co-authored-by: Andy Feller <andyfeller@github.com>

Fix indentation

80830d7

Co-authored-by: Andy Feller <andyfeller@github.com>

Undo goreleaser change

d2d599d

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Fix indentation

6e58a2a

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Removed redundant specifications

18f41db

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Cleanup pkgmacos build script

3830c33

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Change minimum build script macOS version

9454d5e

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

andyfeller reviewed May 23, 2024

View reviewed changes

paulober added 5 commits May 24, 2024 15:19

Fix directory already exists

85f424b

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Added make macospkg target

74392ff

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Fix distribution.xml + min macos version requirements

1990952

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Added native min os version blocking

27262ff

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Integrate argument array to remove duplicate code

f66367d

Signed-off-by: paulober <44974737+paulober@users.noreply.github.com>

Merge branch 'trunk' into feature-macos-pkg-installer

4db8779

andyfeller reviewed May 24, 2024

View reviewed changes

Remove v prefix when pkgmacos is called

279d53a

Existing Mac OS release artifacts use the tag name / version in the file name but drop the `v` prefix. This does the same for the Mac OS installer.

andyfeller reviewed May 24, 2024

View reviewed changes

Update readme about MacOS pkg

bdc40a0

andyfeller approved these changes May 24, 2024

View reviewed changes

andyfeller enabled auto-merge May 24, 2024 19:35

andyfeller merged commit 1bc3cfa into cli:trunk May 24, 2024
8 checks passed

paulober deleted the feature-macos-pkg-installer branch May 24, 2024 21:24

		@@ -0,0 +1,19 @@
		<?xml version="1.0" encoding="utf-8" standalone="yes"?>
		<installer-gui-script minSpecVersion="1">

Add macOS pkg installer to deployment (#7554) #7555

Add macOS pkg installer to deployment (#7554) #7555

Conversation

paulober commented Jun 8, 2023

paulober commented Jun 8, 2023

paulober commented Sep 4, 2023

williammartin commented Sep 11, 2023

paulober commented May 12, 2024

paulober commented May 20, 2024

andyfeller commented May 20, 2024 • edited

andyfeller left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulober May 21, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulober May 21, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulober commented May 21, 2024

andyfeller commented May 21, 2024

paulober commented May 21, 2024

andyfeller commented May 23, 2024

Choose a reason for hiding this comment

paulober May 24, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulober commented May 24, 2024

andyfeller commented May 24, 2024

andyfeller May 24, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andyfeller commented May 24, 2024 • edited

andyfeller left a comment

Choose a reason for hiding this comment

paulober commented May 24, 2024

andyfeller commented May 20, 2024 •

edited

paulober May 21, 2024 •

edited

paulober May 21, 2024 •

edited

paulober May 24, 2024 •

edited

andyfeller May 24, 2024 •

edited

andyfeller commented May 24, 2024 •

edited