Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix-prefetch: fix prefetching when using the hash key #190735

Merged
merged 1 commit into from
Sep 11, 2022

Conversation

winterqt
Copy link
Member

Description of changes

Backporting msteen/nix-prefetch#43 since it seems that the author/maintainer is MIA, and this is somewhat important.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@mweinelt
Copy link
Member

Thanks, works for me as well.

@mweinelt mweinelt merged commit dc53ae8 into NixOS:master Sep 11, 2022
@winterqt winterqt deleted the nix-prefetch-hash-key branch September 11, 2022 01:31
@zowoq
Copy link
Contributor

zowoq commented Sep 11, 2022

❯ nix-update gdu
$ nix eval --json --impure --extra-experimental-features nix-command --expr (
    let
      inputs = (if (builtins.hasAttr "overlays" (builtins.functionArgs (import ./.))) then { overlays = []; } else { });
    in
    with import ./. inputs;
    let
      pkg = gdu;
      raw_version_position = builtins.unsafeGetAttrPos "version" pkg;

      position = if pkg ? isRubyGem then
        raw_version_position
      else
        builtins.unsafeGetAttrPos "src" pkg;
    in {
      name = pkg.name;
      old_version = (builtins.parseDrvName pkg.name).version;
      inherit raw_version_position;
      filename = position.file;
      line = position.line;
      urls = pkg.src.urls or null;
      url = pkg.src.url or null;
      rev = pkg.src.url.rev or null;
      hash = pkg.src.outputHash or null;
      mod_sha256 = pkg.modSha256 or null;
      vendor_sha256 = pkg.vendorSha256 or null;
      cargo_sha256 = pkg.cargoHash or pkg.cargoSha256 or null;
      tests = builtins.attrNames (pkg.passthru.tests or {});
    })
fetch https://github.com/dundee/gdu/releases.atom
Update 5.16.0 -> 5.17.0 in /Users/zowoq/src/nixpkgs/pkgs/tools/system/gdu/default.nix
$ nix-prefetch (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).gdu
The package gdu-5.17.0 will be fetched as follows:
> fetchFromGitHub {
>   hash = "sha256-e2fZJ1HRoi2uMyCWgjPIXs6Q6neg8XEqtC8YcUev+60=";
>   owner = "dundee";
>   repo = "gdu";
>   rev = "v5.17.0";
> }

$ nix-prefetch { sha256 }: (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).gdu.go-modules.overrideAttrs (_: { vendorSha256 = sha256; })
error: anonymous function at (string):5:140 called without required argument 'sha256'

       at /nix/store/awg2xzkv7j8brs1r6ibvgzj7wlj75b5b-nix-prefetch-0.4.1/lib/nix-prefetch/prefetcher.nix:77:21:

           76|       else throw "The --fetchurl option does not work with builtin fetchers.";
           77|     in f // { drv = f f.args; };
             |                     ^
           78|
(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/bin/.nix-update-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/__init__.py", line 224, in main
    package = update(options)
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 154, in update
    update_go_vendor_hash(opts, package.filename, package.vendor_sha256)
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 102, in update_go_vendor_hash
    target_hash = nix_prefetch([expr])
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 77, in nix_prefetch
    res = run(["nix-prefetch"] + cmd, extra_env=extra_env)
  File "/nix/store/kyl3xafs680n3s590lcbg8d04sp1b8g6-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/utils.py", line 35, in run
    return subprocess.run(
  File "/nix/store/c4i6xn02gfljkh7n84f0qm2nbk3a5bd7-python3-3.10.6/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-prefetch', '{ sha256 }: (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).gdu.go-modules.overrideAttrs (_: { vendorSha256 = sha256; })']' returned non-zero exit status 1.

@winterqt
Copy link
Member Author

winterqt commented Sep 11, 2022

@zowoq Can you try with msteen/nix-prefetch@fbe879e applied instead? This fixes prefetching that expression, but I want to make sure I didn't break anything else in the process. (I believe) this fixes the original issue properly, as opposed to the original patch that just happened to work in the cases hexa and I tested. Thanks!

@zowoq
Copy link
Contributor

zowoq commented Sep 11, 2022

Previous one is fixed but found another:

❯ nix-update less --version 600
$ nix eval --json --impure --extra-experimental-features nix-command --expr (
    let
      inputs = (if (builtins.hasAttr "overlays" (builtins.functionArgs (import ./.))) then { overlays = []; } else { });
    in
    with import ./. inputs;
    let
      pkg = less;
      raw_version_position = builtins.unsafeGetAttrPos "version" pkg;

      position = if pkg ? isRubyGem then
        raw_version_position
      else
        builtins.unsafeGetAttrPos "src" pkg;
    in {
      name = pkg.name;
      old_version = (builtins.parseDrvName pkg.name).version;
      inherit raw_version_position;
      filename = position.file;
      line = position.line;
      urls = pkg.src.urls or null;
      url = pkg.src.url or null;
      rev = pkg.src.url.rev or null;
      hash = pkg.src.outputHash or null;
      mod_sha256 = pkg.modSha256 or null;
      vendor_sha256 = pkg.vendorSha256 or null;
      cargo_sha256 = pkg.cargoHash or pkg.cargoSha256 or null;
      tests = builtins.attrNames (pkg.passthru.tests or {});
    })
Update 608 -> 600 in /Users/zowoq/src/nixpkgs/pkgs/tools/misc/less/default.nix
$ nix-prefetch (import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).less
error: hash '02f2d9d6hyf03va28ip620gjc6rf4aikmdyk47h7frqj18pbx6m6' does not include a type, nor is the type otherwise known from context
(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/bin/.nix-update-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/__init__.py", line 224, in main
    package = update(options)
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 149, in update
    update_src_hash(opts, package.filename, package.hash)
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 90, in update_src_hash
    target_hash = nix_prefetch([expr])
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/update.py", line 77, in nix_prefetch
    res = run(["nix-prefetch"] + cmd, extra_env=extra_env)
  File "/nix/store/7lbpx9pf4hxvxbib88g99dqc9yhm7khw-nix-update-0.6.0/lib/python3.10/site-packages/nix_update/utils.py", line 35, in run
    return subprocess.run(
  File "/nix/store/c4i6xn02gfljkh7n84f0qm2nbk3a5bd7-python3-3.10.6/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-prefetch', '(import ./. (if (builtins.hasAttr "config" (builtins.functionArgs (import ./.))) then { config.checkMeta = false; overlays = []; } else { })).less']' returned non-zero exit status 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants