Skip to content

Commit

Permalink
Merge pull request #1279 from tpmccallum/update-spin-test-docs-post
Browse files Browse the repository at this point in the history
  • Loading branch information
Timothy McCallum authored May 9, 2024
2 parents d7924f4 + 215ce6e commit c0f1e4c
Show file tree
Hide file tree
Showing 5 changed files with 213 additions and 16,341 deletions.
225 changes: 179 additions & 46 deletions content/spin/v2/testing-apps.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,19 @@ url = "https://github.com/fermyon/developer/blob/main/content/spin/v2/testing-ap

---

The `spin-test` plugin allows you to run tests, written in WebAssembly, against a Spin application (where all Spin and WASI APIs are configurable mocks).
The [spin test plugin](https://github.com/fermyon/spin-test) allows you to run tests, written in WebAssembly, against a Spin application (where all Spin and WASI APIs are configurable mocks).

To use `spin-test` you write test scenarios for your app in any language, with WebAssembly component support, and mock out all interactions your app has with the outside world without requiring any code changes to the app itself. That means the code you test in development is the same code that runs in production.
To use `spin test` you write test scenarios for your app in any language with WebAssembly component support, and mock out all interactions your app has with the outside world without requiring any code changes to the app itself. That means the code you test in development is the same code that runs in production.

> Note: `spin-test` is still under active development and so the details here may have changed since this post was first published. Check [the spin-test repo](https://github.com/fermyon/spin-test) for the latest information on installation and usage.
> Note: `spin test` is still under active development and so the details here may have changed since this post was first published. Check [the spin-test repo](https://github.com/fermyon/spin-test) for the latest information on installation and usage.
## Prerequisites

The example below uses the [Rust programming language](https://www.rust-lang.org/) and [cargo components](https://github.com/bytecodealliance/cargo-component)(a cargo subcommand for building WebAssembly components).

## Installing the Plugin

To run `spin-test` , you’ll first need to install the canary release of the plugin. As `spin-test` matures, we’ll be making stable releases:
To run `spin test` , you’ll first need to install the canary release of the plugin. As `spin test` matures, we’ll be making stable releases:

```bash
spin plugin install -u https://github.com/fermyon/spin-test/releases/download/canary/spin-test.json
Expand All @@ -33,14 +33,29 @@ First, create an empty Spin app, change into that app folder and then add a comp
<!-- @selectiveCpy -->

```bash
$ spin new -t http-empty my-component --accept-defaults
$ cd my-component/
$ spin new -t http-empty my-app --accept-defaults
$ cd my-app/
$ spin add -t http-rust my-component --accept-defaults
```

The above commands will scaffold out the application in the following format:

<!-- @nocpy -->

```bash
my-app/
├── my-component
│   ├── Cargo.toml
│   └── src
│   └── lib.rs
└── spin.toml
```

## Creating a Test Suite

We use `cargo new` to create a test suite, and then change into that `tests` directory:
There is currently first-class support for writing tests in Rust, but any language with support for writing WebAssembly components can be used as long as the `fermyon:spin-test/test` world is targeted. You can find the definition of this world [here](https://github.com/fermyon/spin-test/blob/4dcaf79c10fc29a8da2750bdaa383b5869db1715/host-wit/world.wit#L13-L16). For this example, we'll use Rust.

We use `cargo` to create a test suite, and then change into our newly created `tests` directory:

<!-- @selectiveCpy -->

Expand All @@ -49,7 +64,24 @@ $ cargo new tests --lib
$ cd tests
```

From within that test suite, we then add the spin-test SDK reference:
After running the cargo new command we will have the following application structure:

<!-- @nocpy -->

```bash
my-app/
├── my-component
│   ├── Cargo.toml
│   └── src
│   └── lib.rs
├── spin.toml
└── tests
├── Cargo.toml
└── src
└── lib.rs
```

From within that test suite (from inside the `tests` directory), we then add the spin-test SDK reference:

<!-- @selectiveCpy -->

Expand All @@ -59,71 +91,107 @@ $ cargo add spin-test-sdk --git https://github.com/fermyon/spin-test

Then, we open the `Cargo.toml` file from within in the `tests` directory and edit to add the `crate-type` of `cdylib`:

<!-- @selectiveCpy -->

```toml
[lib]
crate-type = ["cdylib"]
```

The `my-component/tests/Cargo.toml` file will look like this after editing:
The `my-app/tests/Cargo.toml` file will look like this after editing:

```toml
[package]
name = "tests"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
spin-test-sdk = { git = "https://github.com/fermyon/spin-test", version = "0.1.0" }

[lib]
crate-type = ["cdylib"]
```

## Writing a Test

Next, create a test that `spin-test` can run as a compiled WebAssembly component.

There is currently first-class support for Rust, but any language with support for writing WebAssembly components can be used as long as the `fermyon:spin-test/test` world is targeted. You can find the definition of this world [here](https://github.com/fermyon/spin-test/blob/4dcaf79c10fc29a8da2750bdaa383b5869db1715/host-wit/world.wit#L13-L16).
Next, create a test that `spin test` can run as a compiled WebAssembly component.

Here’s an example of a test written in Rust using the [Spin Test Rust SDK](https://github.com/fermyon/spin-test/tree/main/crates/spin-test-sdk) that tests to ensure that the Spin app responds properly when the key-value store has a certain key already set.
In this example, we will write some tests appropriate to a JSON API service for information about service users. Here are two such tests written in Rust using the [Spin Test Rust SDK](https://github.com/fermyon/spin-test/tree/main/crates/spin-test-sdk). The first test ensures that the Spin app responds properly to an HTTP request. The second test ensures that the Spin app responds properly when the user data is present in the key-value store - for testing purposes, simulated by inserting it into a "virtual" store.

Open the `my-component/tests/src/lib.rs` file and fill it with the following content:
Open the `my-app/tests/src/lib.rs` file and fill it with the following content:

```rust
use spin_test_sdk::{
bindings::{
// fermyon::spin_test_virt::{http_handler, key_value},
wasi::http,
},
bindings::{fermyon::spin_test_virt, wasi, wasi::http},
spin_test,
};

#[spin_test]
fn it_works() {
make_request();
fn send_get_request_without_key() {
// Perform the request
let request = http::types::OutgoingRequest::new(http::types::Headers::new());
request.set_path_with_query(Some("/")).unwrap();
let response = spin_test_sdk::perform_request(request);

// Assert response status and body is 404
assert_eq!(response.status(), 404);
}

#[spin_test]
fn send_get_request_with_invalid_key() {
// Perform the request
let request = http::types::OutgoingRequest::new(http::types::Headers::new());
request.set_path_with_query(Some("/x?123")).unwrap();
let response = spin_test_sdk::perform_request(request);

// Assert response status and body is 404
assert_eq!(response.status(), 404);
}

fn make_request() {
#[spin_test]
fn send_get_request_with_invalid_key_id() {
// Perform the request
let request = http::types::OutgoingRequest::new(http::types::Headers::new());
request.set_path_with_query(Some("/?user_id=123")).unwrap();
request.set_path_with_query(Some("/user?0")).unwrap();
let response = spin_test_sdk::perform_request(request);

// Assert response status and body
// Assert response status and body is 404
assert_eq!(response.status(), 404);
}

#[spin_test]
fn cache_hit() {
let user_json = r#"{"id":123,"name":"Ryan"}"#;

// Configure the app's virtualised 'default' key-value store ready for the test
let key_value = spin_test_virt::key_value::Store::open("default");
// Set a specific key with a specific value
key_value.set("123", user_json.as_bytes());

// Make the request against the Spin app
let request = wasi::http::types::OutgoingRequest::new(wasi::http::types::Headers::new());
request.set_path_with_query(Some("/user?123")).unwrap();
let response = spin_test_sdk::perform_request(request);

// Assert the response status and body
assert_eq!(response.status(), 200);
let body = response.body_as_string().unwrap();
assert_eq!(body, user_json);

// Assert the key-value store was queried
assert_eq!(
key_value.calls(),
vec![spin_test_virt::key_value::Call::Get("123".to_owned())]
);
}
```

The following points are intended to unpack the above example for your understanding:

- Each function marked with `#[spin_test]` will be run as a separate test.
- Each test can perform any setup to their environment by using the APIs available in `spin_test_sdk::bindings::fermyon::spin_test_virt`.
- Requests are made to the Spin application using the `spin_test_sdk::perform_request` function.
- After requests are made, you can use the APIs in `spin_test_sdk::bindings::fermyon::spin_test_virt` to check how the request has changed the state of various resources (such as the key/value store) and make assertions that things changed in the way you expected.
- Each test can perform any setup to their environment by using the APIs available in `spin_test_sdk::bindings::fermyon::spin_test_virt`
- After requests are made, you can use the APIs in `spin_test_sdk::bindings::fermyon::spin_test_virt` to make assertions that confirm that the request has been responded to (e.g. response status equals 200) or that expected Spin API calls (e.g. `get` to the key/value API) have been made.

The test above will run inside of WebAssembly. Calls, such as Key Value storage and retrieval, never actually leave the WebAssembly sandbox. This means your tests are quick and reproducible as you don’t need to rely on running an actual web server, and you don’t need to ensure any of your app’s dependencies are running. Everything your app interacts with is mocked for you.
The tests above run inside of WebAssembly. Calls, such as Key Value storage and retrieval, never actually leave the WebAssembly sandbox. This means your tests are quick and reproducible as you don’t need to rely on running an actual web server, and you don’t need to ensure any of your app’s dependencies are running. Everything your app interacts with is mocked for you. The isolation benefits from this mocking mean that your application's actual data is never touched. There is also the added benefit of reproducibility whereby you never have to worry about leftover data from previous tests.

<!-- markdownlint-disable-next-line titlecase-rule -->
## Configure `spin-test`
Expand All @@ -136,9 +204,7 @@ Before you can run the test, you'll need to tell `spin-test` where your test liv
$ cd ..
```

Then we edit the application's manifest (the `spin.toml` file) as shown below:

<!-- @selectiveCpy -->
Then we edit the `my-app` application's manifest (the `spin.toml` file) by adding the following block:

```toml
[component.my-component.tool.spin-test]
Expand All @@ -147,13 +213,32 @@ build = "cargo component build --release"
dir = "tests"
```

The whole file will look like the following:
## Updating the App to Pass the Tests

This section provides configuration and business logic at the application level.

### Configure App Storage

We are using Key Value storage in the application and therefore need to configure the list of allowed `key_value_stores` in our `spin.toml` file (in this case we are just using the `default`). Go ahead and paste the `key_value_stores` configuration directly inside the `[component.my-component]` section, as shown below:

<!-- @nocpy -->

```toml
[component.my-component]
...
key_value_stores = ["default"]
...
```

> If you would like to learn more about Key Value storage, see [this tutorial](./key-value-store-tutorial.md).
After editing, the whole `my-app/spin.toml` file will look like the following:

```toml
spin_manifest_version = 2

[application]
name = "my-component"
name = "my-app"
version = "0.1.0"
authors = ["Fermyon Engineering <[email protected]>"]
description = ""
Expand All @@ -165,6 +250,7 @@ component = "my-component"
[component.my-component]
source = "my-component/target/wasm32-wasi/release/my_component.wasm"
allowed_outbound_hosts = []
key_value_stores = ["default"]
[component.my-component.build]
command = "cargo build --target wasm32-wasi --release"
workdir = "my-component"
Expand All @@ -176,21 +262,68 @@ build = "cargo component build --release"
dir = "tests"
```

### Adding Business Logic

The goal of tests is to ensure that the business logic in our application works as intended. We haven't yet updated our "business logic" frrom the "Hello, Fermyon" app. To make our new tests pass, copy and paste the following code into the application's source file (located at `my-app/my-component/src/lib.rs`):

```rust
use spin_sdk::{
http::{IntoResponse, Request, Response, Method},
http_component,
key_value::Store,
};

#[http_component]
fn handle_request(req: Request) -> anyhow::Result<impl IntoResponse> {
// Open the default key-value store
let store = Store::open_default()?;

let (status, body) = match *req.method() {
Method::Get => {
// Get the value associated with the request URI, or return a 404 if it's not present
match store.get(req.query())? {
Some(value) => {
println!("Found value for the key {:?}", req.query());
(200, Some(value))
}
None => {
println!("No value found for the key {:?}", req.query());
(404, None)
}
}
}
// No other methods are currently supported
_ => (405, None),
};
Ok(Response::new(status, body))
}
```

## Building and Running the Test

Finally, we're ready for our test to be run. We can do this by invoking `spin-test` from the directory where our Spin application lives:
With the application's configuration and business logic done, we're ready for our test to be run. We can do this by invoking `spin-test` from the application's root directory (`my-app`):

<!-- @selectiveCpy -->

```bash
$ spin build
$ spin-test

running 1 test
Handling request to Some(HeaderValue { inner: String("http://localhost/?user_id=123") })
test it-works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.68s
$ spin test

running 4 tests
No value found for the key "123"
test request-with-invalid-key ... ok
Found value for the key "123"
test cache-hit ... ok
No value found for the key "0"
test request-with-invalid-key-id ... ok
No value found for the key ""
test request-without-key ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.08s
```

## Next Steps

`spin-test` is still in the early days of development, so you’re likely to run into things that don’t quite work yet. We’d love to hear about your experience so we can prioritize which features and bugs to fix first. We’re excited about the future potential of using WebAssembly components for testing, and we look forward to hearing about your experiences as we continue the development of `spin-test`.
`spin-test` is still in the early days of development, so you’re likely to run into things that don’t quite work yet. We’d love to hear about your experience so we can prioritize which features and bugs to fix first. We’re excited about the future potential of using WebAssembly components for testing, and we look forward to hearing about your experiences as we continue the development of `spin-test`.

You might also like to learn about `spin doctor` which is a command-line tool that detects and helps fix issues preventing applications from building and running. For more information see the [troubleshooting applications page](./troubleshooting-application-dev.md).
6 changes: 6 additions & 0 deletions content/spin/v2/troubleshooting-application-dev.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ url = "https://github.com/fermyon/developer/blob/main/content/spin/v2/troublesho

---

## Spin Doctor

The `spin doctor` command detects problems that could stop your application building and running, and can help to fix them. These include problems like invalid manifests, missing Wasm files, and missing tools.

To troubleshoot using `spin doctor`, run the command:
Expand Down Expand Up @@ -38,3 +40,7 @@ $ spin doctor
If `spin doctor` detects a problem it can fix, you can choose to accept the fix, skip it to fix manually later, or see more details before choosing. If `spin doctor` can't fix the problem, it displays the problem so you can make your own decision about how to fix it.

> `spin doctor` is in an early stage of development, and there are many potential problems it doesn't yet check for. Please [raise an issue](https://github.com/fermyon/spin/issues/new?template=suggestion.md) if you have a problem you think `spin doctor` should check for.
## Spin Test

The [spin test plugin](https://github.com/fermyon/spin-test) allows you to write test scenarios for your application's business logic. For more information, see [testing applications](./testing-apps.md).
Loading

0 comments on commit c0f1e4c

Please sign in to comment.