Merge branch 'devel' into natural-sort-upgrade

ansible_collections/arista/avd/docs/porting-guides/5.x.x.md

@@ -94,3 +94,60 @@ The following data model keys have been removed from `eos_cli_config_gen` in v5.
 | vlan_interfaces[].ipv6_address_virtual | vlan_interfaces[].ipv6_address_virtuals |
 | vlan_interfaces[].ipv6_virtual_router_address | vlan_interfaces[].ipv6_virtual_router_addresses |
 | vlan_interfaces[].vrrp | vlan_interfaces[].vrrp_ids |
+
+## Changes to role `arista.avd.eos_validate_state`
+
+### Input for hardware tests have been updated
+
+The inputs for the hardware test have changed.
+
+```yaml
+validation_role:
+  xcvr_own_manufacturers:
+    - Manufacturer 1
+    - Manufacturer 2
+  pwr_supply_states:
+     - ok
+     - notInserted
+  fan_states:
+     - ok
+     - notInserted
+```
+
+should be replaced with:
+
+```yaml
+accepted_xcvr_manufacturers:
+  - Manufacturer 1
+  - Manufacturer 2
+accepted_pwr_supply_states:
+  - ok
+  - notInserted
+accepted_fan_states:
+  - ok
+  - notInserted
+```
+
+### Filtering tests using Ansible tags support was removed
+
+The filtering of tests using Ansible tags has been removed, The `skip_tests` variable should be used instead. The `skip_tests` mechanism is much more granular and allow to filter at the test level.
+
+```bash
+ansible-playbook playbooks/fabric-validate.yaml --skip-tags ntp,bgp_checks
+```
+
+should be replaced with the following input variables:
+
+```yaml
+skip_tests:
+ - category: AvdTestNTP
+ - category: AvdTestBGP
+```
+
+and the following command line:
+
+```bash
+ansible-playbook playbooks/fabric-validate.yaml
+```
+
+The test categories are listed in the [role documentation](../../roles/eos_validate_state/README.md#test-categories)

ansible_collections/arista/avd/docs/release-notes/5.x.x.md

@@ -76,3 +76,55 @@ Breaking changes may require modifications to the inventory or playbook. See the
 for details.
 
 TODO: add level4 sections for breaking changes outlining the change and referring to porting guide for migration.
+
+### Breaking or behavioral changes in eos_validate_state
+
+Starting AVD 5.0.0, `eos_validate_state` uses ANTA as backend, which will change the default behavior (same as setting `use_anta: true` in the latest AVD 4.x.x versions).
+If you have not been using `use_anta: true` in latest versions of AVD 4.x.x, please go through the following sections for details on the changes.
+
+Please report any issues on the GitHub [discussions board](https://github.com/aristanetworks/avd/discussions).
+
+Breaking changes may require modifications to the inventory, input variables or playbooks. See the [Porting guide for AVD 5.x.x](../porting-guides/5.x.x.md)
+for details.
+
+#### Loose mode is the only supported mode
+
+In the previous implementation, loose mode was used by default to avoid stopping the playbook and hence the tests when they were run task by task in Ansible. It was possible to turn loose mode off with a variable that would make the playbook stop at the first failing tasks for whatever reason on each host.
+
+Loose mode is now the default and all the tests are always run and the errors are all present in the final report.
+
+- No porting guide entry.
+
+#### Input for hardware tests have been updated
+
+The inputs for the hardware test have changed. See the [porting guide](../porting-guides/5.x.x.md#input-for-hardware-tests-have-been-updated) for details.
+
+#### Filtering tests using Ansible tags support was removed
+
+Instead use the `skip_tests` mechanism. See the [porting guide](../porting-guides/5.x.x.md#filtering-tests-using-ansible-tags-support-was-removed) for details.
+
+#### Changes to CSV and Markdown reports
+
+- Hardware tests are now collapsed.
+- Sorting of test results is now done per device as opposed to per category.
+- Test categories, descriptions and inputs have been improved to follow ANTA.
+- CSV report headers updated from `test_id,node,test_category,test_description,test,result,failure_reason` to `id,dut,categories,test,description,inputs,result,messages`
+- Tests skipped by ANTA will be marked as `SKIPPED` in the final reports.
+
+#### Undeployed devices
+
+- All tests will be removed from the catalog for a device flagged as undeployed using the host level variable [`is_deployed: false`](https://avd.sh/en/stable/roles/eos_designs/docs/input-variables.html#flagging-a-device-as-not-deployed). Additionally, all tests take into account the `is_deployed` variable value and remove tests accordingly.
+
+!!! warning
+    Tests can also be automatically removed from the catalogs depending on the structured configuration of the devices. ANTA is therefore not aware of these tests and they will not appear in the final report. For example, the `AvdTestMLAG` tests will not be present in the test catalog of a device that does not have an MLAG configuration in its structured configuration.
+
+#### BGP test change
+
+- BGP tests will only run if `service_routing_protocols_model` is set to `multi-agent` in the structured configuration file.
+
+!!! note
+    Starting from version 4.30.1F, `service_routing_protocols_model` is set to `multi-agent` by default on EOS devices.
+
+#### Inband management reachability test change
+
+- Inband management reachability test has been refactored to support AVD inband management [data model](https://avd.sh/en/stable/roles/eos_designs/docs/input-variables.html#node-type-inband-management).

ansible_collections/arista/avd/molecule/eos_validate_state/converge.yml

@@ -16,5 +16,4 @@
       ansible.builtin.import_role:
         name: arista.avd.eos_validate_state
       vars:
-        use_anta: true
         save_catalog: true

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-leaf1a-catalog.yml

@@ -815,6 +815,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-leaf1b-catalog.yml

@@ -799,6 +799,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-leaf1c-catalog.yml

@@ -430,6 +430,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-leaf2a-catalog.yml

@@ -760,6 +760,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-leaf2c-catalog.yml

@@ -423,6 +423,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-spine1-catalog.yml

@@ -640,6 +640,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-spine2-catalog.yml

@@ -631,6 +631,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-svc-leaf1a-catalog.yml

@@ -653,6 +653,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-svc-leaf1b-catalog.yml

@@ -653,6 +653,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-wan1-catalog.yml

@@ -424,6 +424,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc1-wan2-catalog.yml

@@ -424,6 +424,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf1a-catalog.yml

@@ -783,6 +783,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf1b-catalog.yml

@@ -783,6 +783,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf1c-catalog.yml

@@ -271,6 +271,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf2a-catalog.yml

@@ -820,6 +820,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf2b-catalog.yml

@@ -789,6 +789,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf2c-catalog.yml

@@ -271,6 +271,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf3a.arista.com-catalog.yml

@@ -766,6 +766,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-leaf3b.arista.com-catalog.yml

@@ -766,6 +766,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-spine1-catalog.yml

@@ -469,6 +469,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null

ansible_collections/arista/avd/molecule/eos_validate_state/intended/test_catalogs/dc2-spine2-catalog.yml

@@ -469,6 +469,7 @@ anta.tests.stun:
       source_port: 4500
 anta.tests.system:
 - VerifyNTP: null
+- VerifyReloadCause: null
 - VerifyUptime:
     minimum: 86400
 - VerifyReloadCause: null