Skip to content

MeetingMinutes

Jump to bottom
Ayan Sinha Mahapatra edited this page Oct 16, 2023 · 121 revisions

We meet online on Mondays at 16:00 UTC as a reference. See https://www.timeanddate.com/worldclock/meeting.html to get the time in your timezone.

Join us at https://meet.jit.si/AboutCode

Old meeting notes been moved to:

Here are the running meeting notes:

AboutCode meeting - 2023-10-02 - Online on Jitsi

Participants and Agenda

  • Ziad: GSoC presentation
  • Jay: GSoC presentation
  • Ayan, Tushar, Keshav, Omkar also attending

Presentation Links:

Recorded videos will also be published shortly.

AboutCode meeting - 2023-09-25 - Online on Jitsi

Participants and Agenda

  • Keshav, Phillipe, Hritik and Jono also present
  • Omkar - workbench
  • Ziad - purl-sync
  • Ayan - SCTK planning

Discussion:

  • workbench:

https://github.com/nexB/scancode-workbench/issues/605 https://github.com/nexB/scancode-workbench/releases/tag/v4.0.0rc4 https://scancode-workbench.readthedocs.io/en/update-docs/ https://github.com/nexB/scancode-workbench/issues/607 - Run django toolbar,

Also running in shell querysets and filters, whether we are able to model everything in a query

AboutCode meeting - 2023-09-25 - Online on Jitsi

Participants and Agenda

  • Ziad: GSoC presentation
  • Hritik: Transitive support
  • Ayan, Tushar, Jono, Hritik also attending
  • Mathcing legal file to licensedb - keshav
  • fetchcode doc tests - keshav
  • rpm PR - Omkar
  • purldb:metapackage - phillipe

Discussion:

  • When we try to match a resource which is a legal file, we get a lot of matches to purldb, for example if it's an apache license file. So we should not send these files to purldb for matching, and run the usual license detection on these (and not some special hash matching for licenses instead). And we can later also add a step to validate whether it is consistent with the package/directory licenses. For now this can just be a dangling license file which needs review.
  • How to extract .rpm file and edit files inside, and then re-compress this in a .rpm file again such that it is still a valid .rpm file which loads the same way, so we can only have the essential parts for the tests. Here in the tests we had .rpm files which had the metadata and the payload was truncated, which is why the rpm tool will not work.

AboutCode meeting - 2023-09-18 - Online on Jitsi

Participants and Agenda

  • Omkar: Content for "How-to Guide" in SCWB
  • Ziad: Issue while mocking in multiple functions in pytest
  • Jay, Ayan, Tushar, Jono, Hritik, and Keshav also attending

Discussion:

AboutCode meeting - 2023-08-21 - Online on Jitsi

Participants and Agenda

  • Jay: GSoC updates
  • Ziad: GSoC updates + performance for exporter
  • Phillipe: Origin of Code
  • Ayan: SCTK wheels
  • Omkar: ToDo feature in workbench
  • Tushar, Keshav, Jono also attending

Discussion:

  • Jay: Adding some new features in bitcode to implement proper set operations to fix some bugs in the SCTK implementation. Ayan: We need to set up releases for these 3 repos from aboutcode-publisher, and temporarily I have added the package releases from my fork which can be used for testing/CI. See https://pypi.org/project/ahocode-test/ and https://pypi.org/project/bitcode-test
  • Ziad: Implemented pagination, but still the export is slow, what can be done. Will be discussed more in the weekly vulnerablecode call.
  • ToDo feature in SCWB: https://github.com/nexB/scancode-workbench/issues/593 This is creating the UI in existing views to show license/package detections which needs to be reviewed, and the initial step would be to do this for licenses, as it's more simple. In the license view we have unique license detections which are references in the todo plugin, and we need to show a exclamation mark there on the left pane. For packages we need to support cases which are undisplayed package data from the file level, and this would be similar to the license clues display.
  • SCTK wheels: This would be subject of further discussion as we need to examine in more details what is needed, and we need to support all the install options of SCTK without much hassle. We should also simplify the install-app experience instead of simplifying the developer workflow, as for developement, we need to provide the option to modify the licenses/license index and be able to have this as changes together. Also related: https://github.com/nexB/scancode-toolkit/issues/3497
  • We need to have models for code origin and package origin in purldb which would help us in matching to packages, selecting the correct origin.

AboutCode meeting - 2023-08-21 - Online on Jitsi

Participants and Agenda

  • Jay: GSoC updates
  • Ziad: GSoC updates + test exported vuln data format
  • hritik: cdxgen
  • Tushar: pyDelhi presentation
  • Phillipe, Omkar, Tushar, Keshav, Jono, Swastik, Ayan also attending
  • Undiscussed: Origin of Code, SCTK wheels (will be on agenda next week)

Discussion:

  • Couple of thoughts on Vulnerablecode data: let's not sort VCID keys, we might want to use saneyaml instead to output YAML. See https://github.com/nexB/saneyaml/. We should also serialize and output separately.
  • On a separate note, we have issues related to qualifiers in Vulnerablecode, we want to discuss this on the next Vulnerablecode call. Maybe we should keep a qualifier only if the specifier is specific to the vulnerability.
  • Hritik: wanted to add transitive dependencies support in SCIO and cdxgen Added support importing dependencies from cyclonedx boms. We are using hopper models to import/export, which has a lot of updates since. So we should look into if we can safely update to the latest there. In case of creating package/dependency UUIDs we are trusting the tools to create safe UUIDs. We need to also track how a package was created, or from which pipeline/process.
  • There are also issues in creating dependencies from Cyclonedx. Here packages is something concrete you find in the codebase, whereas a dependency is something that is referenced and may not be in the codebase, could be optional/unresolved. Since we have a dependency tree in cyclonedx which has the relationships between all the listed packages, we only had packages created, but could not capture the relationships.
  • Jay's project has been extended from the standard 12 week to a 14 week. We need to make sure the fallback libraries are actually released so the tests can run in SCTK CI and the issues can be debugged properly. We should also discuss the issues in detail preferably late this week.
  • Tushar gave a presentation on python-inspector at pyDelhi last saturday. https://www.youtube.com/watch?v=HclGLQVLBhM https://conference.pydelhi.org/#section-schedule

AboutCode meeting - 2023-08-14 - Online on Jitsi

Participants and Agenda

  • Jay: GSoC updates
  • Ziad: GSoC updates
  • Omkar: testing in workbench
  • Tushar, keshav, Ayan: Nothing to discuss

Discussion:

  • Jay: tests are failing in SCTK integration. Had to use gitpod and for some reason the integration tests for the libraries were passing, but on testing locally recently there were failures. Need to extend the GSoC project. Also need to make sure the libraries are actually released so the tests can run in SCTK CI and so there won't be issues like this.
  • Ziad: Want help to look into authentication and sending auth data. will open an issue as this was not resolved on the call.
  • Omkar: In case of .jar files which return an empty package data without a purl, and this needs to be handled differently. In the dependency table will remove the 'No Value Detected' row, but in the pie charts it's still ok to show these cases.

AboutCode meeting - 2023-08-07 - Online on Jitsi

Participants and Agenda

  • Jay: GSoC updates
  • Ziad: GSoC updates
  • Omkar: testing in workbench
  • Tushar: quine zip file in extractcode
  • Jono, keshav, Ayan: Nothing to discuss

Discussions:

AboutCode meeting - 2023-07-31 - Online on Jitsi

Participants and Agenda

  • Jay: GSoC updates
  • Ziad: GSoC updates
  • keshav: vers support in purldb
  • Hritik: purl.fyi
  • Omkar: testing in workbench
  • Tushar, Ayan, Jono, Swastik: Nothing to discuss

Discussions:

  • Jay:

some tests failing on sanexml, will push a PR for the same, need some help there. Also added PR in SCTK for fallback libraries integration: https://github.com/nexB/scancode-toolkit/pull/3476

  • Ziad:

Do we have regex for identifying purl? We can probably use the same thing in vulnerablecode where we lookup by purl and check for valid purl, we can do the same thing here too. Just importing with the PackageURL library should also work.

  • Keshav:

Support for univers in purldb package index: In cases where dependencies are not pinned, we only submit lowest package version here, and maybe we should send packageURL + vers to be indexed? https://github.com/nexB/univers/blob/main/src/univers/version_range.py Maybe we can start a basic implementation there, maybe list of tuples/mappings.

  • Hritik:

For https://purl.fyi/ it would be nice to consolidate all code we have scattered in different places, this could be a new option in purl_to_url, but there would be new dependencies, so we could also do this in purldb where we have existing code for source_urls etc. We had a GSoC project idea also on this: https://github.com/nexB/aboutcode/wiki/GSOC-2023#purldb-on-demand-retrieval-of-package-metadataarchives

  • Omkar:

Discussion on test files for workbench. We also need to support SCIO outputs once it has licenses support. It would be just adding more tests. Also UI review on deps dashboard and package/deps explorer, looks great, just one point about splitting the package-type and number of packages column into two.

AboutCode meeting - 2023-07-24 - Online on Jitsi

Participants and Agenda

  • Jay: update on GSoC
  • Jono: updating skeleton merge skeleton
  • Ziad: detecting PURL, NLP
  • Tushar, Keshav, Ayan, Omkar: no topic
  • Philippe: github using clearlydefined data (i.e. scancode)
  • Hritik: Vulntotal updates

Discussions:

AboutCode meeting - 2023-07-17 - Online on Jitsi

Participants and Agenda

  • Jay: update on your project
  • Jono: feedback on how display history on packages in purlDB
  • Ziad: safe HTML in Django
  • Tushar, Keshav: no topic
  • Ayan: absent, excused
  • Philippe: Skeleton
  • Omkar: queries on testing and depenencies

Discussions

  • Jay: update on project - Some issues in sanexml wrt. lxml to fix - Next up will be integration in ScanCode and run the tests, making then pass
  • Jono: feedback on how display history on packages in purlDB - History is simple text field. Each line is a timestamp and message - Should we return the history all times with a purl or have a different a different end-point? - A different a different end-point makes most sense
  • Ziad: safe HTML in Django - Need review of how to get the content of a file in git - Need to discuss purl-sync vocabulary - We discussed the data for following PackageURL
  • Philippe: Skeleton - The https://github.com/nexB/skeleton needs to be updated to remove Ubuntu 18 - We need a script to automate the base skeleton in many repos. Jono will give it a shot
  • Omkar: queries on testing and displaying dependencies - We discussed the display of the dependencies summary and provided feedback - We need an issue in SCTK so that it returns a name and icon for each package type or data_source - We discussed testing including tests that are data-driven

AboutCode meeting - 2023-02-06 - Online on Jitsi

Participants:

  • Tushar @tg1999
  • Keshav @keshavspace
  • Ayan @AyanSinhaMahapatra
  • swastik sharma @swastkk
  • Jay @35C4n0r
  • Akhil @lf32
  • Omkar @OmkarPh

Agenda:

  • GSoC
  • Misc

Discussion:

  • purldb still has to be updated with latest scancode and in a stable state for us to start adding good first issues there, so maybe this is better to do a month later. Meanwhile we can mark good first issues in other repositories, in vulnerablecode, scancode-toolkit, scancode.io etc for first time contributors.
  • Conclusions pipeline: Conclusions/alerts/review/to-do items in scancode.io is a workflow where we can review detections which are incorrect or needs careful manual review and where the data can be updated in place. This was asked as a tentative GSoC project but we are still finalizing the project ideas and it is advised to start looking at the project ideas list after aboutcode is selected at GSoC.
  • https://github.com/nexB/python-inspector/pull/119 was opened by swastik which was failing tests, as live packages are used for python-inspector tests and we need to regen these, we will also add this to the documentation.
  • Akhil has updated https://github.com/nexB/scancode.io/pull/450 with binary file support and replaced the Scan Text button with a Utilities drop down with the Detect License option which goes to /scantext/.
  • Phillipe needs to review https://github.com/nexB/scancode-workbench/pull/532, and please use scancode v31 with this as v32 is not supported yet here, see https://scancode-toolkit.readthedocs.io/en/latest/explanations/license-detection-reference.html for more updates there.

AboutCode meeting - 2023-01-23 - Online on Jitsi

Participants:

  • Tushar @tg1999
  • phillipe @pombredanne
  • Keshav @keshavspace
  • Ayan @AyanSinhaMahapatra
  • swastik sharma @swastkk
  • Jay @35C4n0r
  • Shrey Parekh
  • Shrijal Acharya

Agenda:

  • GSoC
  • corrupted advisories
  • yaml output
  • scancode toolkit reference scans
  • packaging and operating system support
  • cylconedx input in scancode.io

Discussion:

  • Need to review swastik's PR: https://github.com/nexB/python-inspector/pull/119
  • should we use both cyclonedx libraries from the cyclonedx-python and the hoppr library? - Keshav links: https://gitlab.com/hoppr/hoppr-cyclonedx-models/ and https://github.com/CycloneDX/cyclonedx-python short term: working with these projects to merge features We don't use XML and don't care about old versions. The hoppr library does for the last 2 cyclonedx versions, and it uses the JSON schema to create the models. We can start using hoppr/hoppr-cyclonedx-models in scancode.io and then maybe later we can use it in scancode-toolkit too.
  • JSON to XML conversion for cyclonedx -> library exists which works as a single executable in linux/windows/mac.
  • advisories which were imported by previous importers, which aren't compatible to current models. We can delete everything from a importer, when we are reimporting from the same. There's a problem of stale and outdated data, and there's a problem of not discarding data that is used elsewhere also. We can consider archiving for this, or consider adding a deprecated flag.
  • more people running non-intel architechture, which doesn't work The key thing would be a single executable: like Jono's work on a scancode.io appimage. We should also have app archives for all python versions which is python 3.7-3.11 and in linux/mac/windows. No arm for now, but would be nice. Another thing would be https://github.com/nexB/scancode-toolkit/issues/3205 If we are using other libraries, we have to write wrappers on them to match the same API. Serializing is another problem. Pyahocorasick is going to be the hardest, as this is a trie structure and saving/loading from disk is not simple.
  • https://github.com/nexB/aboutcode/wiki/GSOC-2023 GSoC project ideas were discussed, and we need to further edit this and make all the projects have a clear goal and some detailed instructions to explain them better, Ideas related to vulnerablecode will be discussed in the vulnerabelcode call tomorrow see https://github.com/nexB/vulnerablecode/wiki/WeeklyMeetings.
  • We uncovered that the scancode yaml output does not produce valid yaml in certain cases where there are license references and/or matched text in the yaml output and the license text has whitespaces/blank lines. for example, happens in the case of apache-2.0 license text. The solution can't be just to remove whitespaces as they are important, but the check has to be done at saneyaml and we have to produce valid yaml there.
  • scancode-toolkit-reference-scan scripts are not working because of the dependency issues present while pip installing older versions, and maybe we should be using git checkout instead of pip install here.

AboutCode meeting - 2023-01-16 - Online on Jitsi

Participants:

  • Tushar @tg1999
  • Jay @35C4n0r
  • phillipe @pombredanne
  • swastik sharma @swastkk
  • Keshav @keshavspace
  • Ayan @AyanSinhaMahapatra
  • Jono @jyang
  • Akhil @lf32
  • Heet Dhorajiya

Agenda:

  • scancode.io appimage
  • dependency issues
  • scancode-toolkit release
  • GSoC project ideas
  • skeleton

Discussion:

AboutCode meeting - 2023-01-02 - Online on Jitsi

Participants:

  • Tushar @tg1999
  • Hritik @Hritik14
  • Jay @35C4n0r
  • phillipe @pombredanne
  • swastik sharma @swastkk
  • Keshav @keshavspace

Agenda:

http://aboutcode.org/

Clone this wiki locally