backport: bitcoin#24603, #26694, #24669, #22546, #22199, #25817 (mac build) #5718
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
knst
force-pushed
the
mac-improvements
branch
2 times, most recently
from
e1754af to
828f32e
Compare
knst
force-pushed
the
mac-improvements
branch
from
65ffbe2 to
44e9cc9
Compare
|
LGTM; running Guix-build now |
|
|
Canceled "Guix Build" action, waiting for infra team to figure out what's wrong with runners. |
|
same hashes (except for |
|
Guix CI failure is unrelated, should be fixed by #5727 |
hashes matched with mine (except |
UdjinM6
approved these changes
Nov 24, 2023
PastaPastaPasta
approved these changes
Nov 24, 2023
…lity 08bd338 build: optimise arm64 darwin qt build using -O1 (fanquake) Pull request description: Building the macOS M1 bitcoin-qt binary at a optimisation level higher than `-O1` causes reproducibility issues when building on different architectures. Proposing somewhat of a hammer. This would fix 1 of the 2 remaining HOSTS in bitcoin#21194. Guix Build (x86_64): ```bash 1b58b5109b32dca2509499c93347148e6bab5dca835081f8cbd3123bed72cce1 guix-build-08bd3382777b/output/arm64-apple-darwin/SHA256SUMS.part 0e0d063d3832fad7c5116dabb2ac33c919f40bda04759aad4523c6247295bc9e guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.dmg 1feb301245f2c664edcfd9ac528fe1543fc7b183b3b42637db77d57658bc2b5e guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.tar.gz 641eb100d0a281203f9d6e36e45dc0ffc772c680d6aec462434f106b4c44e295 guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin.tar.gz 9d89920626e35939aa6cf506fc85861179f3c0e18d4ef1954750cf81336a851a guix-build-08bd3382777b/output/dist-archive/bitcoin-08bd3382777b.tar.gz e7697d30084270d0b5843b3baf0d752e240c2f708f728bc2f6896f153276ca6b guix-build-08bd3382777b/output/x86_64-apple-darwin/SHA256SUMS.part dd77acee082dbfd3cdad2c564bbd3bdace8df9bf32f92cf4a2debd5a996ace49 guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.dmg 85e63fccb7af12468a04a678034c42dcd775d243b2d194a52e1086a6ffbdbe84 guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.tar.gz ff2629957608898d76a42025985e3ec4bf5dc8572794e32b4182ba6f8babb828 guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin.tar.gz ``` Guix Build (arm64): ```bash 1b58b5109b32dca2509499c93347148e6bab5dca835081f8cbd3123bed72cce1 guix-build-08bd3382777b/output/arm64-apple-darwin/SHA256SUMS.part 0e0d063d3832fad7c5116dabb2ac33c919f40bda04759aad4523c6247295bc9e guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.dmg 1feb301245f2c664edcfd9ac528fe1543fc7b183b3b42637db77d57658bc2b5e guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.tar.gz 641eb100d0a281203f9d6e36e45dc0ffc772c680d6aec462434f106b4c44e295 guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin.tar.gz 9d89920626e35939aa6cf506fc85861179f3c0e18d4ef1954750cf81336a851a guix-build-08bd3382777b/output/dist-archive/bitcoin-08bd3382777b.tar.gz e7697d30084270d0b5843b3baf0d752e240c2f708f728bc2f6896f153276ca6b guix-build-08bd3382777b/output/x86_64-apple-darwin/SHA256SUMS.part dd77acee082dbfd3cdad2c564bbd3bdace8df9bf32f92cf4a2debd5a996ace49 guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.dmg 85e63fccb7af12468a04a678034c42dcd775d243b2d194a52e1086a6ffbdbe84 guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.tar.gz ff2629957608898d76a42025985e3ec4bf5dc8572794e32b4182ba6f8babb828 guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin.tar.gz ``` ACKs for top commit: hebasto: ACK 08bd338 jarolrod: ACK 08bd338 Tree-SHA512: 48da4acb1799c3153cdaf674f287c81c3da230a3476183616b74f318baa595af45b313136eb228ba13c63e0b8206a78064734f9fd0488e1e839c9e4e1d92ba25
0a5723b macdeploy: cleanup .temp.dmg if present (fanquake) ecffe86 macdeploy: remove qt4 related code (fanquake) 639f064 macdeploy: select the plugins we need, rather than excluding those we don't (fanquake) 3d26b6b macdeploy: fix framework printing when passing -verbose (fanquake) dca6c90 macdeploy: remove unused plistlib import (fanquake) Pull request description: This includes [one followup](bitcoin#20422 (comment)) and [one bug fix](bitcoin@3d26b6b) from bitcoin#20422, as well as some simplifications to the `macdeployqtplus` code. ACKs for top commit: hebasto: ACK 0a5723b, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2. Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029
…ith system frameworks 1513727 build, qt: (Re-)sign package (Hennadii Stepanov) c26a0a5 build, qt: Align frameworks with macOS codesign tool requirements (Hennadii Stepanov) Pull request description: Fixes bitcoin#22403 This PR follows Apple [docs](https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes): > - New in macOS 11 on Macs with Apple silicon, and starting in macOS Big Sur 11 beta 6, the operating system enforces that any executable must be signed before it’s allowed to run. There isn’t a specific identity requirement for this signature: a simple ad-hoc signature is sufficient... > - ... If you use a custom workflow involving tools that modify a binary after linking (e.g. `strip` or `install_name_tool`) you might need to manually call `codesign` as an additional build phase to properly ad-hoc sign your binary. These new signatures are not bound to the specific machine that was used to build the executable, they can be verified on any other system and will be sufficient to comply with the new default code signing requirement on Macs with Apple silicon... When building with system Qt frameworks (i.e., without depends), a new string has been added to the `make deploy` log on M1-based macOS: ``` % make deploy ... + Generating .DS_Store + dist/Bitcoin-Qt.app: replacing existing signature + Preparing .dmg disk image + ... ``` This PR does not change build system behavior: - when building with depends - on Intel-based macOS ACKs for top commit: jarolrod: ACK 1513727 fanquake: ACK 1513727 - although didn't test on M1 hardware. Given the forced signing is scoped to only occur when running the deploy script on macOS, this doesn't interfere with our release signing. Tree-SHA512: 3aa778fdd6ddb54f029f632f2fe52c2ae3bb197ba564cb776493aa5c3a655bd51d10ccbe6c007372d717e9b01fc4193dd5c29ea0bc7e069dcae7e991ae259f0c
3d41521 build: perform /Applications symlink generation in macdeployqtplus (fanquake) dac6936 build: perform all .tiff copying in macdeployqtplus (fanquake) Pull request description: Rather than maintaining 2 different versions of the same code (`.tiff` copying and symlink generation), consolidate to just the Python code, and use it on macOS and Linux. Previously Linux would perform the 2 actions in the makefile, and then would still be running the `macdeployqtplus` script, so it makes sense to further consolidate deployment operations into the script. Guix Build (on x86_64): ```bash 23343f04c426c7ff078afae4e600a7028970d4d86eed8b7834696d9e4d684151 guix-build-3d415215699e/output/arm64-apple-darwin/SHA256SUMS.part c28b2a2e4888bf84369aa25804e2576347d5ab09416354ec8b95c76a9d38ff96 guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.dmg 9a57077b2bd722a7d85d26b66cbce5abdb791985fe9d9d37e884c79ba8751e24 guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.tar.gz d2b06dc5b86541798ace41dab569849f7403e7ff9ec329bda671ec84e6fad549 guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin.tar.gz 608e7d51a44ab9c5b28eb3703a0f4fe98b4adff22c77a5502786b84bd96cc188 guix-build-3d415215699e/output/dist-archive/bitcoin-3d415215699e.tar.gz 3e483705b1f9f1fb8f6afedc8ad0214a6cb00e77f766c0b03c42d56f410d4362 guix-build-3d415215699e/output/x86_64-apple-darwin/SHA256SUMS.part 9370e3e3b7d47b5a44e64554cf3b6d7e0671b072c08cd251eacc7ec72ce2b53f guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.dmg ad0f68682d78c311497669fc3d627138be37510215d259b5f0b686d93e7d83b7 guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.tar.gz e09dce4ff692ef66d1f4818083c1880bcf3a79c53112561d9e929bb6e5ffc011 guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin.tar.gz ``` ACKs for top commit: laanwj: Re-ACK 3d41521 Tree-SHA512: 80dd66a6e94c5b3e8823ccb57dcb08a8851a1e70a154b62385443f8d2d5ed5af900a0ac5003143959863586f1c7b90002fe6bff3ca5e37697253e051f69d7629
dba1231 test: previous releases: add v23.0 (Sjors Provoost) Pull request description: Follows the same pattern as d8b705f (v22.0) and 8a57a06 (v0.21.0). Starting from v23.0 there is a separate macOS release for x86_64 and aarch64. ACKs for top commit: prusnak: Approach ACK dba1231 Tree-SHA512: 249aeddd5e80e163578581e5c8e9b6579f3694abc3d1fb68dddb7b42d75021ad85266688ec4a365a6631d82a65a19873aff7ba61c0ea59d21f8adbe4b772dc16
…'t run unsigned arm64 binaries; self-sign when needed dc12f2e test: improve error msg on previous release tarball extraction failure (kdmukai) 7121fd8 test: self-sign previous release binaries for arm64 macOS (kdmukai) Pull request description: ## The Problem If you run `test/get_previous_releases.py -b` on an M1 or M2 mac, you'll get an unsigned v23.0 binary in the arm64 tarball. macOS [sets stricter requirements on ARM binaries](https://news.ycombinator.com/item?id=26996578) so the unsigned arm64 binary is apparently completely unusable without being signed/notarized(?). This means that any test that depends on a previous release (e.g. `wallet_backwards_compatibility.py`) will fail because the v23.0 node cannot launch: ``` TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/Users/kdmukai/dev/bitcoin-core/test/functional/test_framework/test_framework.py", line 563, in start_nodes node.wait_for_rpc_connection() File "/Users/kdmukai/dev/bitcoin-core/test/functional/test_framework/test_node.py", line 231, in wait_for_rpc_connection raise FailedToStartError(self._node_msg( test_framework.test_node.FailedToStartError: [node 2] bitcoind exited with status -9 during initialization ``` This can also be confirmed by downloading bitcoin-23.0-arm64-apple-darwin.tar.gz (https://bitcoincore.org/bin/bitcoin-core-23.0/) and trying to run any of the binaries manually on an M1 or M2 mac. ## Solution in this PR (UPDATED) Per @ hebasto, we can self-sign the arm64 binaries. This PR checks each binary in the previous release's "bin/" and verifies if the arm64 binary is signed. If not, attempt to self-sign and confirm success. (note: an earlier version of this PR downloaded the x86_64 binary as a workaround but this approach has been discarded) ## Longer term solution If possible, produce signed arm64 binaries in a future v23.x tarball? Note that this same problem affects the new v24.0.1 arm64 tarball so perhaps a signed v24.x.x tarball would also be ideal? That being said, this PR will check all current and future arm64 binaries and self-sign as needed, so perhaps we need not worry about pre-signing the tarball binaries. And I did test a version of `get_previous_releases.py` that includes the new v24.0.1 binaries and it successfully self-signed both v23.0 and v24.0.1, as expected. ## Further info: Somewhat related to: bitcoin#15774 (comment) And @ fanquake noted on IRC that you can confirm which binaries are or are not signed via: ``` $ codesign -v -d bitcoin-qt bitcoin-qt: code object is not signed at all ``` ACKs for top commit: hebasto: ACK dc12f2e Tree-SHA512: 644895f8e97f5ffb3c4754c1db2c48abd77fa100c2058e3c896af04806596fc2b9c807a3f3a2add5be53301ad40ca2b8171585bd254e691f6eb38714d938396b
979271a macdeploy: remove unused detached-sig-apply (fanquake) Pull request description: Signature application is now done with signapple. https://github.com/bitcoin/bitcoin/blob/8435d7f11a89bb3f93306646f62cc2179693e072/contrib/guix/libexec/codesign.sh#L84-L85 ACKs for top commit: laanwj: ACK 979271a gruve-p: ACK bitcoin@979271a achow101: ACK 979271a hebasto: ACK 979271a, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: ab51a609d00cead4f33bcfc5b5ff1008ee02363ab1f4c4bf9544631069c237bfa92eac4dfa231bff8a1d702bda6cc92b4151361f74f58e77b595e0cb82a8391a
…6_64-apple-darwin platform
PastaPastaPasta
force-pushed
the
mac-improvements
branch
from
44e9cc9 to
fd2e985
Compare
ogabrielides
pushed a commit
to ogabrielides/dash
that referenced
this pull request
Dec 4, 2023
backport: bitcoin#24603, bitcoin#26694, bitcoin#24669, bitcoin#22546, bitcoin#22199, bitcoin#25817 (mac build)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue being fixed or feature implemented
Os X doesn't let to run guix's binaries on arm64 because these binaries are not signed.
What was done?
Backported bitcoin#22546 and couple extra relevant backports or backports just better to be here to avoid conflicts now and in future
get_previous_releases.py: M1/M2 macs can't run unsigned arm64 binaries; self-sign when needed bitcoin/bitcoin#26694make deployon M1-based macOS with system frameworks bitcoin/bitcoin#22546How Has This Been Tested?
Run unit/functional tests.
@PastaPastaPasta , please, confirm that you can run guix's binaries with these changes on Mac.
Breaking Changes
N/A
Checklist: