Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document skill tests and update ovos-utils dependency spec #19

Merged
merged 1 commit into from
Feb 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,39 @@ To test that skill resources are defined for all supported languages,
the skill's root directory
> - <test-file\> is a relative or absolute path to the resource test file, usually `test_resources.yaml`

example `test_resources.yaml`:
```yaml
# Specify resources to test here.

# Specify languages to be tested
languages:
- "en-us"
- "uk-ua"

# vocab is lowercase .voc file basenames
vocab:
- ip
- public
- query

# dialog is .dialog file basenames (case-sensitive)
dialog:
- dot
- my address is
- my address on X is Y
- no network connection
- word_public
- word_local
# regex entities, not necessarily filenames
regex: []
intents:
# Padatious intents are the `.intent` file names
padatious: []
# Adapt intents are the name passed to the constructor
adapt:
- IPIntent
```

### Intent Tests
To test that skill intents match as expected for all supported languages,
`minerva test-intents <skill-entrypoint> <test-file>`
Expand All @@ -42,6 +75,38 @@ To test that skill intents match as expected for all supported languages,
> - <test-file\> is a relative or absolute path to the resource test file, usually `test_intents.yaml`
> - The `--padacioso` flag can be added to test with Padacioso instead of Padatious for relevant intents

example `test_intents.yaml`:
```yaml
en-us:
IPIntent:
- what is your ip address
- what is my ip address:
- IP
- what is my i.p. address
- What is your I.P. address?
- what is my public IP address?:
- public: public

uk-ua:
IPIntent:
- шо в мене за ай пі:
- IP
- покажи яка в мене за мережа:
- IP
- покажи яка в мене публічний ай пі адреса:
- public: публічний
```

#### Test Configuration
The following top-level sections can be added to intent test configuration:

- `unmatched intents`: dict of `lang` to list of `utterances` that should match
no intents. Note that this does not test for CommonQuery or CommonPlay matches.
- `common query`: dict of `lang` to list of `utterances` OR dict of `utterances`
to expected: `callback_data` (list keys or dict data), `min_confidence`, and
`max_confidence`
- `common play`: TBD

## Advanced Usage
In addition to convenient CLI methods, this package also provides test cases that
may be extended.
Expand Down
2 changes: 1 addition & 1 deletion requirements/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
click~=8.0
click-default-group~=1.2
ovos-utils~=0.0.35
ovos-utils~=0.0, >=0.0.35
ovos-workshop~=0.0.12
padacioso~=0.1
pyyaml>=5.4,<7.0
Expand Down
Loading