Add lazy loading for GitHub API calls #33
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Although the types are different, the intent is the same, so I think the name is fitting.
Now that we are lazy-loading the API pages, OctokitWrapper is handling the conversion to NextflowRelease in one function call. This means we can't test the API schema. That should be fine as we pinned the API version. We can test that the resulting objects for validity, so do that.
The pagination changes require Node.js 18 or greater, so update the test apparatus to use the latest LTS version of Node.
edmundmiller
approved these changes
Jan 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sorry if it seems like I'm polishing the silverware on the Titanic right now...but I was discovered some additional features while playing around in the code base that I thought would be helpful as a holdover for the time while I figure out how to completely switch the API.
There are some configuration and optimization changes that help make API pagination more efficient in this PR, but those are just prerequisites for the main feature: lazy loading. On the master branch right now, the action makes$N / 30$ API calls, where $N$ is the number of total Nextflow releases, no matter which release you are requesting. By implementing this PR, the action only calls the API until it finds a matching release (or can confirm that the release meets all requirements, in the case of
latest-everything). This cuts the number of API calls down to a third of their previous value (and that's for the old Nextflow versions that aren't returned on the first call).Number of API calls performed by
act -j test