Skip to content

Openjdk importer branch - #1589

Closed
harmonicfunc wants to merge 14 commits into
aboutcode-org:mainfrom
harmonicfunc:openjdk-importer-branch
Closed

Openjdk importer branch#1589
harmonicfunc wants to merge 14 commits into
aboutcode-org:mainfrom
harmonicfunc:openjdk-importer-branch

Conversation

@harmonicfunc

Copy link
Copy Markdown

for issue: #1496
Added a OpenJDK advisory importer that scrapes the vulnerability data from the link: https://openjdk.org/groups/vulnerability/advisories/ and then added the importer to importer registery

harmonicfunc and others added 4 commits September 12, 2024 14:07
Signed-off-by: Alok Kumar Singh <singhalokkumar960@gmail.com>
Add gsd test

Signed-off-by: ziadhany <ziadhany2016@gmail.com>
Signed-off-by: Alok Kumar Singh <singhalokkumar960@gmail.com>
Signed-off-by: Alok Kumar Singh <singhalokkumar960@gmail.com>
@harmonicfunc

Copy link
Copy Markdown
Author

@TG1999 @ambuj-1211 @ziadhany please review this pr

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @harmonicfunc, use the VulnerableCodeBaseImporterPipeline for new importers. See the detailed instructions below.

Comment thread vulnerabilities/importers/openjdk.py Outdated
from vulnerabilities.importer import VulnerabilitySeverity


class OpenJDKImporter(Importer):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harmonicfunc We now have a new Importer design. We use aboutcode.pipeline for importers/Improver,

Move this file to vulnerabilities/pipelines directory and rename this file to openjdk_importer.py :

And start with something like this:

class OpenJDKImporterPipeline(VulnerableCodeBaseImporterPipeline):
    """Collect advisories from OpenJDK."""

    root_url = "https://openjdk.org/groups/vulnerability/advisories/"
    license_url = "https://openjdk.org/legal/"
    spdx_license_expression = "CC-BY-4.0"
    importer_name = "OpenJDK Importer"

    @classmethod
    def steps(cls):
        return (
            cls.fetch_advisory,
            cls.collect_and_store_advisories,
            cls.import_new_advisories,
        )

    def fetch_advisory(self):
        self.log(f"Fetching {self.root_url}")
        self.advisory_data = requests.get(self.root_url).text

    def advisories_count(self) -> int:
       # Use self.advisory_data to return the estimated AdvisoryData to be yielded by ``collect_advisories()``. 
        pass
    
    def collect_advisories(self) -> Iterable[AdvisoryData]:
        # Yield AdvisoryData by processing the self.advisory_data (this is similar to `advisory_data()` in old importer)
        # Use self.log() to log info/errors

See this pypa_importer.py pipeline for example https://github.com/aboutcode-org/vulnerablecode/blob/main/vulnerabilities/pipelines/pypa_importer.py.

Also we're in process of migrating our existing importers/imporvers to the new pipeline architecture see #1509

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@keshav-space this would be a nice addition/update to the doc ;)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harmonicfunc

Copy link
Copy Markdown
Author

ok, on it

Signed-off-by: Alok Kumar Singh <singhalokkumar960@gmail.com>
@harmonicfunc

Copy link
Copy Markdown
Author

@keshav-space sorry but do i need to revamp or change logic parts of my code too??

@pombredanne

Copy link
Copy Markdown
Member

@harmonicfunc the core logic does not need change, just the integration in a pipeline!
I see you may have some issues with merging ... this is easy at first! You likely want to rebase or restart on the latest main branch and push a new PR, which I would guess is why you closed this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants