Skip to content

Support advanced package assembly in multi-flavour repository #3604

Description

@AyanSinhaMahapatra

When there are package manifests from multiple package ecosystems and we perform a package assembly for the package manifest that we find first, a top level package is created out of it, and while assigning resources, we ignore any package manifest of different type from package assembly as they do not match the data-file patterns of this package handler.

Suggested fix: While doing Package assembly even if a package data is in the seen_resource_paths here: https://github.com/nexB/scancode-toolkit/blob/develop/src/packagedcode/plugin_package.py#L351 but it has a package data in it which has different type from the top-level package that made this resource seen, do not ignore this resource. To track this we have to make the seen_resource_paths a dictionary with {"path": "package_type"} instead of a plain list.

See example at https://github.com/nexB/scancode-toolkit/blob/support-cargo-workspaces/tests/packagedcode/data/cargo/cargo-with-workspace.expected.json#L1527 where the package from this resource is not assembled because of a package.json at the root.

See debug logs for the issue:

DEBUG    packagedcode.plugin_package:plugin_package.py:56 get_package_and_deps: location: /home/ayansinha/nexB/write_access/scancode-extra/tests/packagedcode/data/cargo/cargo-with-workspace/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: package_data.purl: None
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: handler: <class 'packagedcode.cargo.CargoTomlHandler'>
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/tauri@2.0.0-alpha.17
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/tauri-runtime@1.0.0-alpha.4
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/tauri-build@2.0.0-alpha.11
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/restart@0.1.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56 get_package_and_deps: location: /home/ayansinha/nexB/write_access/scancode-extra/tests/packagedcode/data/cargo/cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: package_data.purl: pkg:npm/tauri-workspace@0.0.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: handler: <class 'packagedcode.npm.NpmPackageJsonHandler'>
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:npm/tauri-workspace@0.0.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/LICENSE.spdx
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/LICENSE_APACHE-2.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/LICENSE_MIT
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/README.md
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/src-tauri
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/src-tauri/Cargo.lock
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/src-tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56 get_package_and_deps: location: /home/ayansinha/nexB/write_access/scancode-extra/tests/packagedcode/data/cargo/cargo-with-workspace

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions