feat: new page overviewing CI & code coverage #165

lwasser · 2024-01-16T19:03:01Z

This pr is the final piece of content from the now closed pr #104 that was opened in september. it contains 2 short pages

describing CI
overviewing code coverage.

because it's been so long i plan to review / edit prior to making this reviewable.

To see the specific tasks where the Asana app for GitHub is being used, see below:
- https://app.asana.com/0/0/1206992329524474

lwasser · 2024-01-16T19:03:34Z

conf.py

@@ -97,7 +97,7 @@
        "image_light": "logo-light-mode.png",
        "alt_text": "pyOpenSci Python Package Guide. The pyOpenSci logo is a purple flower with pyOpenSci under it. The o in open sci is the center of the flower",
    },
-    "header_links_before_dropdown": 4,


This makes space for a CI&data section in our navigation.

willingc · 2024-03-19T18:00:05Z

@lwasser Should we close this draft? Or do you want to leave open.

lwasser · 2024-03-21T16:18:29Z

@willingc i think we need to keep this one open. its content that i have yet to finish. so maybe a help-wanted tag is warranted. this page isn't high priority now but i can try to put it in my task list as a chip away at it task.

lwasser · 2024-03-21T16:18:59Z

tasks

rebase against main
revisit content and figure out what section it belongs in
edit
send through community review

ci-data/ci.md

blink1073

LGTM!

tkoyama010 · 2024-12-13T02:39:19Z

pre-commit.ci autofix

sneakers-the-rat · 2024-12-13T02:49:49Z

i'm a little confused about why CI and data are combined? it seems worth it to me to have a dedicated ci section and move all ci stuff there (e.g. i would probably also move the code coverage part to ci as well, because one typically wants coverage in a CI context vs. in a general testing context) and data feels more like something that would go in the tests section: "how do i handle bulky data i need for my tests." or if we were to include it in ci it should just be a section within ci rather than the section being called ci-data

webknjaz · 2024-12-18T01:01:31Z

tests/code-cov.md

+:height: 450px
+:alt: Screenshot of the code cov service - showing test coverage for the stravalib package. in this image you can see a list of package modules and the associated number of lines and % lines covered by tests. at the top of the image you can see what branch is being evaluated and the path to the repository being shown.
+
+The CodeCov platform is a useful tool if you wish to visually track code coverage. Using it you can not only get the same summary information that you can get with **pytest-cov** extension. You can also get a visual representation of what lines are covered by your tests and what lines are not covered. Code coverage is mostly useful for evaluating unit tests and/or how much of your package code is "covered". It however will not evaluate things like integration tests and end-to-end workflows.


It may be useful to mention that MyPy is also able to produce coverage reports for typing that can be uploaded to Codecov and overlaid on top of runtime testing coverage.

Here's one place where I implemented this:

https://github.com/aio-libs/yarl/blob/3967268/.github/workflows/reusable-linters.yml#L66C11-L69C51

https://github.com/aio-libs/yarl/blob/3967268/.pre-commit-config.yaml#L127C7-L127C57

It comes with a few challenges, though: https://discuss.python.org/t/is-there-any-tool-that-can-report-type-coverage-of-a-project/34962/4.

Also, there's GitHub Actions like irongut/CodeCoverageSummary that can be helpful in presenting this in job summaries: https://github.com/cherrypy/cheroot/blob/a9c1fe4/.github/workflows/ci-cd.yml#L1100C5-L1112C65.

Finally, Coveragepy itself is able to output markdown for use in job summaries (https://github.com/aio-libs/yarl/blob/3967268/.github/workflows/ci-cd.yml#L331C9-L332C36) and it can produce HTML output locally.
The latter is quite useful since it will display context details that are not represented on Codecov. Plus, it's nice not to have to push to remote and trigger an expensive CI run to work on something locally and maybe even offline.

thank you!! this is super helpful. I've updated the file to add info about Mypy and exporting results in a local or ci build to avoid needing to use the website interfaces! !!

lwasser · 2024-12-20T21:46:09Z

thank you, everyone, for the reviews!! Once Ci builds and is happy i'll merge this!!

lwasser · 2024-12-20T21:46:20Z

@all-contributors please add @webknjaz for review

allcontributors · 2024-12-20T21:46:29Z

@lwasser

I've put up a pull request to add @webknjaz! 🎉

lwasser · 2024-12-20T22:02:11Z

@all-contributors add @namurphy for code, review

allcontributors · 2024-12-20T22:02:14Z

@lwasser

@namurphy already contributed before to code, review

ENH: fixes from Jonny's review Fix: review edits from Jonny p2 Fix: typos and cleanup Fix: add example to tests ci page

@namurphy

Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/code-cov.md Co-authored-by: Nick Murphy <[email protected]> Fix: edits from @namurphy to run tests page Co-authored-by: Nick Murphy <[email protected]> Update ci-tests-data/run-tests.md Co-authored-by: Nick Murphy <[email protected]>

Co-authored-by: Trevor James Smith <[email protected]> Fix: edit from trevor Co-authored-by: Trevor James Smith <[email protected]> Fix: edit from trevor Co-authored-by: Trevor James Smith <[email protected]> Fix: edits from Trevor Co-authored-by: Trevor James Smith <[email protected]> Fix: edits from Nick Co-authored-by: Nick Murphy <[email protected]> Packaging image Fix: remove unused block Fix Update ci-tests-data/tests-ci.md Co-authored-by: Trevor James Smith <[email protected]>

Co-authored-by: Carol Willing <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Carol Willing <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Carol Willing <[email protected]> Fix: other edits from review'

Fix: force to main Fix: main Fix: update for final page review Fix: remove dated image

Co-authored-by: Jeremiah Paige <[email protected]>

lwasser · 2024-12-20T22:14:56Z

Ok, I've now renamed the CI section continuous-integration.
I've removed data from the section title. We can consider putting David's PR #110 in the tests section when we review it in 2025 as it fits there.

The ci section is quite thin, but the one page we have, I think, is a great start 🚀 thank you, everyone. This is getting merged now!! 🎉

lwasser commented Jan 16, 2024

View reviewed changes

lwasser changed the title Ci Add: new page overviewing CI Mar 21, 2024

lwasser force-pushed the ci branch 2 times, most recently from 578ad59 to 33a61e8 Compare April 3, 2024 16:37

lwasser marked this pull request as ready for review April 3, 2024 16:37

willingc added enhancement-feature something new to add to our guide draft Draft PRs - do not merge labels Jun 5, 2024

lwasser changed the title ~~Add: new page overviewing CI~~ feat: new page overviewing CI & code coverage Dec 11, 2024

lwasser changed the title ~~feat: new page overviewing CI & code coverage~~ [dec 20 merge] feat: new page overviewing CI & code coverage Dec 11, 2024

lwasser force-pushed the ci branch from 33a61e8 to a1286aa Compare December 11, 2024 21:08

lwasser added 🚀 ready-for-review and removed draft Draft PRs - do not merge labels Dec 11, 2024

ucodery approved these changes Dec 11, 2024

View reviewed changes

ci-data/ci.md Outdated Show resolved Hide resolved

blink1073 approved these changes Dec 12, 2024

View reviewed changes

webknjaz reviewed Dec 18, 2024

View reviewed changes

lwasser changed the title ~~[dec 20 merge] feat: new page overviewing CI & code coverage~~ feat: new page overviewing CI & code coverage Dec 20, 2024

allcontributors bot mentioned this pull request Dec 20, 2024

docs: add webknjaz as a contributor for review #436

Merged

lwasser and others added 3 commits December 20, 2024 15:03

Fix: add initial tests content to guide

47d43f7

ENH: fixes from Jonny's review Fix: review edits from Jonny p2 Fix: typos and cleanup Fix: add example to tests ci page

lwasser and others added 5 commits December 20, 2024 15:03

Fix: edits from @willingc to ci page

900960d

Co-authored-by: Carol Willing <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Carol Willing <[email protected]> Update ci-tests-data/ci.md Co-authored-by: Carol Willing <[email protected]> Fix: other edits from review'

Fix: force to main

848a483

Fix: force to main Fix: main Fix: update for final page review Fix: remove dated image

Fix: Edits from @ucodery

e9b849c

Co-authored-by: Jeremiah Paige <[email protected]>

'[pre-commit.ci 🤖] Apply code format tools to PR'

f16a18e

fix: cleanup

8ca6bd2

lwasser force-pushed the ci branch from 07c81e4 to 8ca6bd2 Compare December 20, 2024 22:03

lwasser added 2 commits December 20, 2024 15:11

fix: pre commit and url fixes

708b23c

fix: add toctree to ci landing so it's less empty

044b24e

lwasser merged commit 635144d into pyOpenSci:main Dec 20, 2024
4 checks passed

lwasser deleted the ci branch December 20, 2024 22:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: new page overviewing CI & code coverage #165

feat: new page overviewing CI & code coverage #165

lwasser commented Jan 16, 2024 •

edited

Loading

lwasser Jan 16, 2024

willingc commented Mar 19, 2024

lwasser commented Mar 21, 2024

lwasser commented Mar 21, 2024 •

edited

Loading

blink1073 left a comment

tkoyama010 commented Dec 13, 2024

sneakers-the-rat commented Dec 13, 2024

webknjaz Dec 18, 2024

lwasser Dec 20, 2024

lwasser commented Dec 20, 2024

lwasser commented Dec 20, 2024

allcontributors bot commented Dec 20, 2024

lwasser commented Dec 20, 2024

allcontributors bot commented Dec 20, 2024

lwasser commented Dec 20, 2024

feat: new page overviewing CI & code coverage #165

feat: new page overviewing CI & code coverage #165

Conversation

lwasser commented Jan 16, 2024 • edited Loading

lwasser Jan 16, 2024

Choose a reason for hiding this comment

willingc commented Mar 19, 2024

lwasser commented Mar 21, 2024

lwasser commented Mar 21, 2024 • edited Loading

blink1073 left a comment

Choose a reason for hiding this comment

tkoyama010 commented Dec 13, 2024

sneakers-the-rat commented Dec 13, 2024

webknjaz Dec 18, 2024

Choose a reason for hiding this comment

lwasser Dec 20, 2024

Choose a reason for hiding this comment

lwasser commented Dec 20, 2024

lwasser commented Dec 20, 2024

allcontributors bot commented Dec 20, 2024

lwasser commented Dec 20, 2024

allcontributors bot commented Dec 20, 2024

lwasser commented Dec 20, 2024

lwasser commented Jan 16, 2024 •

edited

Loading

lwasser commented Mar 21, 2024 •

edited

Loading