Update testing guide structure and add coverage testing (#1182)

stellar · Jan 17, 2025 · f381246 · f381246
1 parent 077d942
commit f381246
Show file tree

Hide file tree

Showing 10 changed files with 63 additions and 16 deletions.
diff --git a/docs/build/guides/testing/code-coverage.mdx b/docs/build/guides/testing/code-coverage.mdx
@@ -0,0 +1,53 @@
+---
+title: Code Coverage
+hide_table_of_contents: true
+description: Code coverage tools find code not tested.
+sidebar_position: 10
+---
+
+Measuring code coverage uses tools to identify lines of code that are and aren't executed by tests. Code coverage stats can give us an idea of how much of a contract is actually tested by its tests.
+
+:::tip
+
+Mutation testing is another form of coverage testing. See [Mutation Testing].
+
+:::
+
+In rust projects the `cargo-llvm-cov` tool can be used to generate coverage stats, HTML reports, and lcov files that IDEs will load to display the coverage in the code editor.
+
+Install `cargo-llvm-cov` before proceeding with the other commands.
+
+```
+cargo install cargo-llvm-cov
+```
+
+## How to Get Coverage Stats
+
+Run the test subcommand that will run the tests and output the stats per file.
+
+```
+cargo llvm-cov test
+```
+
+## How to Generate a Coverage Report with Code
+
+Run the test subcommand that will run the tests and output a set of HTML files showing which lines of code are covered.
+
+```
+cargo llvm-cov test --html --open
+```
+
+The output of the command will indicate where the HTML file has been written. Open the file in a browser.
+
+## How to Generate an LCOV File for IDEs
+
+Run the test subcommand that will run the tests and output a single `lcov.info` file.
+
+```
+cargo llvm-cov test --lcov --output-path=lcov.info
+```
+
+Load the `lcov.info` file into your IDE using it's coverage feature. In VSCode this can be done by installing the [Coverage Gutters] extension and executing the `Coverage Gutters: Watch` command.
+
+[Coverage Gutters]: https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters
+[Mutation Testing]: mutation-testing.mdx
diff --git a/docs/build/guides/testing/coverage-testing.mdx b/docs/build/guides/testing/coverage-testing.mdx
diff --git a/docs/build/guides/testing/differential-tests-with-test-snapshots.mdx b/docs/build/guides/testing/differential-tests-with-test-snapshots.mdx
@@ -2,7 +2,7 @@
 title: Differential Tests with Test Snapshots
 hide_table_of_contents: true
 description: Differential testing using automatic test snapshots.
-sidebar_position: 8
+sidebar_position: 9
 ---
 
 Tests are written to ensure that contracts behave today as expected, and in the future as well. Over time a contract may change and in all software development there remains the possibility of changes causing side-effects that are unexpected. Testing is one of the ways that we identify unexpected changes.

diff --git a/docs/build/guides/testing/differential-tests.mdx b/docs/build/guides/testing/differential-tests.mdx
@@ -2,7 +2,7 @@
 title: Differential Tests
 hide_table_of_contents: true
 description: Differential testing detects unintended changes.
-sidebar_position: 7
+sidebar_position: 8
 ---
 
 Differential testing is the testing of two things to discover differences in their behavior.

diff --git a/docs/build/guides/testing/fuzzing.mdx b/docs/build/guides/testing/fuzzing.mdx
@@ -2,7 +2,7 @@
 title: Fuzzing
 hide_table_of_contents: true
 description: Fuzzing and property testing to find unexpected behavior.
-sidebar_position: 5
+sidebar_position: 7
 ---
 
 Fuzzing is the process of providing random data to programs to identify unexpected behavior, such as crashes and panics.

diff --git a/docs/build/guides/testing/integration-tests-with-mainnet-data.mdx b/docs/build/guides/testing/integration-tests-with-mainnet-data.mdx
@@ -2,8 +2,7 @@
 title: Integration Tests with Mainnet Data
 hide_table_of_contents: true
 description: Integration testing uses dependency contracts instead of mocks.
-sidebar_position: 4
-draft: true
+sidebar_position: 6
 ---
 
 Testing with mainnet data is another form of [integration test], where not only mainnet contracts can be used, but also their data on mainnet.

diff --git a/docs/build/guides/testing/integration-tests.mdx b/docs/build/guides/testing/integration-tests.mdx
@@ -2,7 +2,7 @@
 title: Integration Tests
 hide_table_of_contents: true
 description: Integration testing uses dependency contracts instead of mocks.
-sidebar_position: 3
+sidebar_position: 5
 ---
 
 Integration tests are tests that include the integration between components, and so test a larger scope such as other contracts.

diff --git a/docs/build/guides/testing/mutation-testing.mdx b/docs/build/guides/testing/mutation-testing.mdx
@@ -2,7 +2,7 @@
 title: Mutation Testing
 hide_table_of_contents: true
 description: Mutation testing finds code not tested.
-sidebar_position: 9
+sidebar_position: 11
 ---
 
 Mutation testing is making changes to a program, either manually or automatically, to identify changes that can be made that don't get caught by tests.
@@ -33,3 +33,5 @@ The `cargo-mutants` tool can be used to automatically and iteratively modify the
    Code that is identified as not covered by a test will be outputted as a `MISSED` line in the output.
 
    Diffs of each change that was attempted can be found in the `mutants.out/diff` directory.
+
+[code coverage]: code-coverage.mdx
diff --git a/docs/build/guides/testing/test-contract-auth.mdx b/docs/build/guides/testing/test-contract-auth.mdx
@@ -2,6 +2,7 @@
 title: Test Authorization
 hide_table_of_contents: true
 description: Write tests that test contract authorization.
+sidebar_position: 3
 ---
 
 Tests can assert on the auths that are expected to occur.

diff --git a/docs/build/guides/testing/test-contract-events.mdx b/docs/build/guides/testing/test-contract-events.mdx
@@ -2,6 +2,7 @@
 title: Test Events
 hide_table_of_contents: true
 description: Write tests that test contract events.
+sidebar_position: 4
 ---
 
 Tests can assert on events that are expected to be published.