Improve actions time - #459
Merged
Merged
Conversation
Initializing docker containers used to take 23-25s alone. α Now Initializing postgresql (Setup database) takes 4-6s. β α: https://github.com/nexB/vulnerablecode/runs/2623306124?check_suite_focus=true https://github.com/nexB/vulnerablecode/runs/2623290485?check_suite_focus=true https://github.com/nexB/vulnerablecode/runs/2594778880?check_suite_focus=true β: https://github.com/Hritik14/vulnerablecode/runs/2631513782?check_suite_focus=true https://github.com/Hritik14/vulnerablecode/runs/2631540610?check_suite_focus=true https://github.com/Hritik14/vulnerablecode/runs/2631552958?check_suite_focus=true Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Earlier Install Dependencies used to take 50-60s α Now it takes ~10s including cache retrival time β The total workflow time has reduced from ~1:40 to ~0:55 α: https://github.com/nexB/vulnerablecode/runs/2623306124?check_suite_focus=true https://github.com/nexB/vulnerablecode/runs/2623290485?check_suite_focus=true https://github.com/nexB/vulnerablecode/runs/2594778880?check_suite_focus=true β: https://github.com/Hritik14/vulnerablecode/runs/2632140380?check_suite_focus=true https://github.com/Hritik14/vulnerablecode/runs/2632128817?check_suite_focus=true https://github.com/Hritik14/vulnerablecode/runs/2632109016?check_suite_focus=true Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Collaborator
|
I don't think cache for venv is a good idea, it would need some manual work when deps change right ? Dropping containers -> good idea |
Collaborator
Author
|
Key of the cache depends on requirements.txt and would invalidate whenever dependencies change. So there wouldn't be any manual involvement. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Actions are majorly meant to perform tests on our codebase, yet they spend majority of their time on setting up the environment.
Currently this only speeds up the pytest actions. We may also improve other ones after a discussion.
This PR comprises of two commits. The profiling of both commits were performed in Hritik14#2 and Hritik14#3
commit 135c95f (HEAD -> actions)
Author: Hritik Vijay hritikxx8@gmail.com
Date: Thu May 20 22:25:42 2021 +0530
Use cache for pip
Earlier Install Dependencies used to take 50-60s α
Now it takes ~10s including cache retrival time β
The total workflow time has reduced from ~1:40 to ~0:55
α: https://github.com/nexB/vulnerablecode/runs/2623306124?check_suite_focus=true
https://github.com/nexB/vulnerablecode/runs/2623290485?check_suite_focus=true
https://github.com/nexB/vulnerablecode/runs/2594778880?check_suite_focus=true
β: https://github.com/Hritik14/vulnerablecode/runs/2632140380?check_suite_focus=true
https://github.com/Hritik14/vulnerablecode/runs/2632128817?check_suite_focus=true
https://github.com/Hritik14/vulnerablecode/runs/2632109016?check_suite_focus=true
Signed-off-by: Hritik Vijay hritikxx8@gmail.com
commit 9114cb4
Author: Hritik Vijay hritikxx8@gmail.com
Date: Thu May 20 01:06:20 2021 +0530
Eliminate postgresql docker container
Initializing docker containers used to take 23-25s alone. α
Now Initializing postgresql (Setup database) takes 4-6s. β
α: https://github.com/nexB/vulnerablecode/runs/2623306124?check_suite_focus=true https://github.com/nexB/vulnerablecode/runs/2623290485?check_suite_focus=true https://github.com/nexB/vulnerablecode/runs/2594778880?check_suite_focus=true
β: https://github.com/Hritik14/vulnerablecode/runs/2631513782?check_suite_focus=true https://github.com/Hritik14/vulnerablecode/runs/2631540610?check_suite_focus=true https://github.com/Hritik14/vulnerablecode/runs/2631552958?check_suite_focus=true
Signed-off-by: Hritik Vijay hritikxx8@gmail.com