Merge remote-tracking branch 'origin/main' into acapy-cheqd

.github/workflows/format.yml

@@ -17,5 +17,5 @@ jobs:
       - name: Ruff Format and Lint Check
         uses: chartboost/ruff-action@v1
         with:
-          version: 0.5.7
+          version: 0.8.0
           args: "format --check"

.pre-commit-config.yaml

@@ -1,14 +1,14 @@
 repos:
   - repo: https://github.com/alessandrojcm/commitlint-pre-commit-hook
-    rev: v9.16.0
+    rev: v9.18.0
     hooks:
       - id: commitlint
         stages: [commit-msg]
         args: ["--config", ".commitlint.config.js"]
         additional_dependencies: ['@commitlint/config-conventional']
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ensure this is synced with pyproject.toml
-    rev: v0.5.7
+    rev: v0.8.1
     hooks:
       # Run the linter
       - id: ruff

CHANGELOG.md

@@ -1,5 +1,79 @@
 # Aries Cloud Agent Python Changelog
 
+## 1.1.1rc0
+
+### December 3, 2024
+
+Release 1.1.1 is a patch update to ACA-Py that contains a lengthy list of adjustments, improvements and fixes, with a focus on removing Technical Debt. The most visible change is the removal of the "in-memory wallet" implementation in favour of using the SQLite in-memory wallet (`sqlite://:memory:`), including removing the logic for handling that extra wallet type. While arguably a breaking change (and we mention it below), we're confident no one is using the in-memory wallet (right?!?) any where other than in tests. In removing the in-memory wallet, all of the unit and integration tests that used the in-memory wallet were updated to use SQLite's in-memory wallet.
+
+The first step to full support of [did:tdw](https://identity.foundation/trustdidweb/) (soon to be renamed to `did:webvh` for "`did:web` + Verifiable History") has been added to ACA-Py -- a resolver. We're working on adding new DID Registration for it and other DID Methods, enabling ACA-Py to be used easily with a variety of DID Methods.
+
+The move to the [OpenWallet Foundation](https://openwallet.foundation/) is now complete. For up to date details on what the repo move means for ACA-Py users, including steps for updating deployments, please see latest in [GitHub Issue #3250].
+
+[GitHub Issue #3250]: https://github.com/hyperledger/aries-cloudagent-python/issues/3250
+
+### 1.1.1 Deprecation Notices
+
+The same **[deprecation notices](#101-deprecation-notices)** from the [1.1.0](#110) release about AIP 1.0 protocols still apply. The protocols remain in the 1.1.1 release, but will be moved out of the core and into plugins soon. Please review these notifications carefully!
+
+### 1.1.1 Breaking Changes
+
+While there are no breaking changes in this release that might impact production deployments, the removal of the "in-memory" wallet implementation might be break some test scripts. Rather than using the in-memory wallet, tests should be updated to use SQLite's special `sqlite://:memory:` database instead. This results in a better alignment between tests and a production environment.
+
+
+#### 1.1.1 Categorized List of Pull Requests
+
+- AnonCreds VC Issuance and Presentation Enhancement / Fixes
+  - Repair release bdd tests [\#3376](https://github.com/openwallet-foundation/acapy/pull/3376) [jamshale](https://github.com/jamshale)
+  - Fix tails upload for anoncreds multitenancy [\#3346](https://github.com/openwallet-foundation/acapy/pull/3346) [jamshale](https://github.com/jamshale)
+  - Fix subwallet anoncreds upgrade check [\#3345](https://github.com/openwallet-foundation/acapy/pull/3345) [jamshale](https://github.com/jamshale)
+  - Add anoncreds issuance and presentation format [\#3331](https://github.com/openwallet-foundation/acapy/pull/3331) [jamshale](https://github.com/jamshale)
+  - Fix endorsement setup with existing connection [\#3309](https://github.com/openwallet-foundation/acapy/pull/3309) [jamshale](https://github.com/jamshale)
+  - Update accumulator value in wallet on repair [\#3299](https://github.com/openwallet-foundation/acapy/pull/3299) [jamshale](https://github.com/jamshale)
+
+- Middleware Handling and Multi-tenancy
+  - Restore `--base-wallet-routes` flag functionality [\#3344](https://github.com/openwallet-foundation/acapy/pull/3344) [esune](https://github.com/esune)
+  - :white_check_mark: Re-add ready_middleware unit tests [\#3330](https://github.com/openwallet-foundation/acapy/pull/3330) [ff137](https://github.com/ff137)
+  - :sparkles: Handle NotFound and UnprocessableEntity errors in middleware [\#3327](https://github.com/openwallet-foundation/acapy/pull/3327) [ff137](https://github.com/ff137)
+  - :art: Refactor Multitenant Manager errors and exception handling [\#3323](https://github.com/openwallet-foundation/acapy/pull/3323) [ff137](https://github.com/ff137)
+  - Don't pass rekey to sub_wallet_profile [\#3312](https://github.com/openwallet-foundation/acapy/pull/3312) [jamshale](https://github.com/jamshale)
+
+- DID Registration and Resolution
+  - fix: check routing keys on indy_vdr endpoint refresh [\#3371](https://github.com/openwallet-foundation/acapy/pull/3371) [dbluhm](https://github.com/dbluhm)
+  - Fix/universal resolver [\#3354](https://github.com/openwallet-foundation/acapy/pull/3354) [jamshale](https://github.com/jamshale)
+  - More robust verification method selection by did [\#3279](https://github.com/openwallet-foundation/acapy/pull/3279) [dbluhm](https://github.com/dbluhm)
+  - did:tdw resolver [\#3237](https://github.com/openwallet-foundation/acapy/pull/3237) [jamshale](https://github.com/jamshale)
+
+- DIDComm Updates and Enhancements
+  - :bug: Rearrange connection record deletion after hangup [\#3310](https://github.com/openwallet-foundation/acapy/pull/3310) [ff137](https://github.com/ff137)
+  - :bug: Handle failure to resolve DIDComm services in DIDXManager [\#3298](https://github.com/openwallet-foundation/acapy/pull/3298) [ff137](https://github.com/ff137)
+
+- Test Suite Updates and Artifact Publishing
+  - Add test wallet config option [\#3355](https://github.com/openwallet-foundation/acapy/pull/3355) [jamshale](https://github.com/jamshale)
+  - :art: Fix current test warnings [\#3338](https://github.com/openwallet-foundation/acapy/pull/3338) [ff137](https://github.com/ff137)
+  - :construction_worker: Fix Nightly Publish to not run on forks [\#3333](https://github.com/openwallet-foundation/acapy/pull/3333) [ff137](https://github.com/ff137)
+
+- Internal Improvements / Cleanups / Tech Debt Updates
+  - :zap: Add class caching to DeferLoad [\#3361](https://github.com/openwallet-foundation/acapy/pull/3361) [ff137](https://github.com/ff137
+  - :art: Sync Ruff version in configs and apply formatting [\#3358](https://github.com/openwallet-foundation/acapy/pull/3358) [ff137](https://github.com/ff137)
+  - :art: Replace deprecated ABC decorators [\#3357](https://github.com/openwallet-foundation/acapy/pull/3357) [ff137](https://github.com/ff137)
+  - :art: Refactor the logging module monolith [\#3319](https://github.com/openwallet-foundation/acapy/pull/3319) [ff137](https://github.com/ff137)
+  - :wrench: set default fixture scope for pytest-asyncio [\#3318](https://github.com/openwallet-foundation/acapy/pull/3318) [ff137](https://github.com/ff137)
+  - Docs (devcontainer) Change folder names [\#3317](https://github.com/openwallet-foundation/acapy/pull/3317) [loneil](https://github.com/loneil)
+  - :art: Refactor string concatenation in model descriptions [\#3313](https://github.com/openwallet-foundation/acapy/pull/3313) [ff137](https://github.com/ff137)
+  - Remove in memory wallet [\#3311](https://github.com/openwallet-foundation/acapy/pull/3311) [jamshale](https://github.com/jamshale)
+
+- Consolidate Dependabot updates and other library/dependency updates
+  - Week 49 Library upgrades [\#3368](https://github.com/openwallet-foundation/acapy/pull/3368) [jamshale](https://github.com/jamshale) 
+  - :arrow_up: Update lock file [\#3296](https://github.com/openwallet-foundation/acapy/pull/3296) [ff137](https://github.com/ff137)
+
+- Release management pull requests:
+  - 1.1.1rc0 [\#3372](https://github.com/openwallet-foundation/acapy/pull/3372) [swcurran](https://github.com/swcurran)
+
+- Dependabot PRs
+  - [Link to list of Dependabot PRs in this release](https://github.com/openwallet-foundation/acapy/pulls?q=is%3Apr+is%3Amerged+merged%3A2024-10-15..2024-12-03+author%3Aapp%2Fdependabot+)
+
+
 ## 1.1.0
 
 ### October 15, 2024

Managing-ACA-Py-Doc-Site.md

@@ -20,7 +20,7 @@ and mkdocs configuration.
 
 When the GitHub Action fires, it runs a container that carries out the following steps:
 
-- Checks out the triggering branch, either `main` or `docs-v<version>` (e.g `docs-v1.1.0`).
+- Checks out the triggering branch, either `main` or `docs-v<version>` (e.g `docs-v1.1.1`).
 - Runs the script [scripts/prepmkdocs.sh], which moves and updates some of the
   markdown files so that they fit into the generated site. See the comments in
   the scripts for details about the copying and editing done via the script. In
@@ -97,7 +97,7 @@ To delete the documentation version, do the following:
 - Check your `git status` and make sure there are no changes in the branch --
   e.g., new files that shouldn't be added to the `gh-pages` branch. If there are
   any -- delete the files so they are not added.
-- Remove the folder for the RC.  For example `rm -rf 1.1.0`
+- Remove the folder for the RC.  For example `rm -rf 1.1.1rc0`
 - Edit the `versions.json` file and remove the reference to the RC release in
   the file.
 - Push the changes via a PR to the ACA-Py `gh-pages` branch (don't PR them into

PUBLISHING.md

@@ -6,7 +6,7 @@ a major, minor or patch release, per [semver](https://semver.org/) rules.
 
 Once ready to do a release, create a local branch that includes the following updates:
 
-1. Create a local PR branch from an updated `main` branch, e.g. "1.1.0".
+1. Create a local PR branch from an updated `main` branch, e.g. "1.1.1".
 
 2. See if there are any Document Site `mkdocs` changes needed. Run the script
    `./scripts/prepmkdocs.sh; mkdocs`. Watch the log, noting particularly if
@@ -27,7 +27,7 @@ Once ready to do a release, create a local branch that includes the following up
    github account. Do not include `dependabot` PRs. For those, we put a live
    link for the date range of the release (guidance below).
 
-   To generate the list, run the script `genChangeLog.sh` command (requires you
+   To generate the list, run the `./scripts/genChangeLog.sh` scripts (requires you
    have [gh] and [jq] installed), with the date of the day before the last
    release. The day before is picked to make sure you get all of the changes.
    The script generates the list of all PRs, minus the dependabot ones, merged since
@@ -78,17 +78,17 @@ Once you have the list of PRs:
 - Check the dates in the `dependabot` URL to make sure the full period between the previous non-RC release to the date of the non-RC release you are preparing.
 - Include a PR in the list for this soon-to-be PR, initially with the "next to be issued" number for PRs/Issues. At the end output of the script is the highest numbered PR and issue. Your PR will be one higher than the highest of those two numbers. Note that you still might have to correct the number after you create the PR if someone sneaks an issue or PR in before you submit your PR.
 
-5. Check to see if there are any other PRs that should be included in the release.
+1. Check to see if there are any other PRs that should be included in the release.
 
-6. Update the ReadTheDocs in the `/docs` folder by following the instructions in
+2. Update the ReadTheDocs in the `/docs` folder by following the instructions in
    the `docs/UpdateRTD.md` file. That will likely add a number of new and modified
    files to the PR. Eliminate all of the errors in the generation process,
    either by mocking external dependencies or by fixing ACA-Py code. If
    necessary, create an issue with the errors and assign it to the appropriate
    developer. Experience has demonstrated to use that documentation generation
    errors should be fixed in the code.
 
-7. Search across the repository for the previous version number and update it
+3. Search across the repository for the previous version number and update it
    everywhere that makes sense. The CHANGELOG.md entry for the previous release
    is a likely exception, and the `pyproject.toml` in the root **MUST** be
    updated. You can skip (although it won't hurt) to update the files in the
@@ -101,28 +101,28 @@ Once you have the list of PRs:
    have dropped the previously used `-` in the release candidate version string
    to better follow the semver rules.
 
-8. Regenerate openapi.json and swagger.json by running
+4. Regenerate openapi.json and swagger.json by running
    `scripts/generate-open-api-spec` from within the `acapy_agent` folder.
 
    Command: `cd acapy_agent;../scripts/generate-open-api-spec;cd ..`
 
    Folders may not be cleaned up by the script, so the following can be run, likely with `sudo` -- `rm -rf open-api/.build`. The folder is `.gitignore`d, so there is not a danger they will be pushed, even if they are not deleted.
 
-9.  Double check all of these steps above, and then submit a PR from the branch.
+5.  Double check all of these steps above, and then submit a PR from the branch.
    Add this new PR to CHANGELOG.md so that all the PRs are included.
    If there are still further changes to be merged, mark the PR as "Draft",
    repeat **ALL** of the steps again, and then mark this PR as ready and then
    wait until it is merged. It's embarrassing when you have to do a whole new
    release just because you missed something silly...I know!
 
-10.   Immediately after it is merged, create a new GitHub tag representing the
+6.    Immediately after it is merged, create a new GitHub tag representing the
    version. The tag name and title of the release should be the same as the
    version in [pyproject.toml](https://github.com/openwallet-foundation/acapy/tree/main/pyproject.toml). Use
    the "Generate Release Notes" capability to get a sequential listing of the
    PRs in the release, to complement the manually curated Changelog. Verify on
    PyPi that the version is published.
 
-11.    New images for the release are automatically published by the GitHubAction
+7.     New images for the release are automatically published by the GitHubAction
    Workflows: [publish.yml] and [publish-indy.yml]. The actions are triggered
    when a release is tagged, so no manual action is needed. The images are
    published in the [OpenWallet Foundation Package Repository under
@@ -140,7 +140,7 @@ Once you have the list of PRs:
 [publish-indy.yml]: https://github.com/openwallet-foundation/acapy/blob/main/.github/workflows/publish-indy.yml
 
 1.   When a new release is tagged, create a new branch at the same commit with
-    the branch name in the format `docs-v<version>`, for example, `docs-v1.1.0`.
+    the branch name in the format `docs-v<version>`, for example, `docs-v1.1.1`.
     The creation of the branch triggers the execution of the [publish-docs]
     GitHub Action which generates the documentation for the new release,
     publishing it at [https://aca-py.org]. The GitHub Action also executes when

acapy_agent/admin/decorators/auth.py

@@ -1,6 +1,8 @@
 """Authentication decorators for the admin API."""
 
 import functools
+import re
+from typing import Optional, Pattern
 
 from aiohttp import web
 
@@ -48,6 +50,8 @@ def tenant_authentication(handler):
     - check for a valid bearer token in the Autorization header if running
     in multi-tenant mode
     - check for a valid x-api-key header if running in single-tenant mode
+    - check if the base wallet has access to the requested path if running
+    in multi-tenant mode
     """
 
     @functools.wraps(handler)
@@ -61,11 +65,15 @@ async def tenant_auth(request):
         )
         insecure_mode = bool(profile.settings.get("admin.admin_insecure_mode"))
         multitenant_enabled = profile.settings.get("multitenant.enabled")
+        base_wallet_allowed_route = _base_wallet_route_access(
+            profile.settings.get("multitenant.base_wallet_routes"), request.path
+        )
 
         # CORS fix: allow OPTIONS method access to paths without a token
         if (
             (multitenant_enabled and authorization_header)
             or (not multitenant_enabled and valid_key)
+            or (multitenant_enabled and valid_key and base_wallet_allowed_route)
             or insecure_mode
             or request.method == "OPTIONS"
         ):
@@ -78,3 +86,25 @@ async def tenant_auth(request):
             )
 
     return tenant_auth
+
+
+def _base_wallet_route_access(additional_routes: str, request_path: str) -> bool:
+    """Check if request path matches additional routes."""
+    additional_routes_pattern = _build_additional_routes_pattern(additional_routes)
+    return _matches_additional_routes(additional_routes_pattern, request_path)
+
+
+def _build_additional_routes_pattern(pattern_string: str) -> Optional[Pattern]:
+    """Build pattern from space delimited list of paths."""
+    # create array and add word boundary to avoid false positives
+    if pattern_string:
+        paths = pattern_string.split(" ")
+        return re.compile("^((?:)" + "|".join(paths) + ")$")
+    return None
+
+
+def _matches_additional_routes(pattern: Pattern, path: str) -> bool:
+    """Matches request path to provided pattern."""
+    if pattern and path:
+        return bool(pattern.match(path))
+    return False

acapy_agent/admin/server.py

@@ -5,7 +5,7 @@
 import re
 import warnings
 import weakref
-from typing import Callable, Coroutine, Optional, Pattern, Sequence, cast
+from typing import Callable, Coroutine, Optional
 
 import aiohttp_cors
 import jwt
@@ -280,29 +280,6 @@ def __init__(
         self.websocket_queues = {}
         self.site = None
         self.multitenant_manager = context.inject_or(BaseMultitenantManager)
-        self._additional_route_pattern: Optional[Pattern] = None
-
-    @property
-    def additional_routes_pattern(self) -> Optional[Pattern]:
-        """Pattern for configured additional routes to permit base wallet to access."""
-        if self._additional_route_pattern:
-            return self._additional_route_pattern
-
-        base_wallet_routes = self.context.settings.get("multitenant.base_wallet_routes")
-        base_wallet_routes = cast(Sequence[str], base_wallet_routes)
-        if base_wallet_routes:
-            self._additional_route_pattern = re.compile(
-                "^(?:" + "|".join(base_wallet_routes) + ")"
-            )
-        return None
-
-    def _matches_additional_routes(self, path: str) -> bool:
-        """Path matches additional_routes_pattern."""
-        pattern = self.additional_routes_pattern
-        if pattern:
-            return bool(pattern.match(path))
-
-        return False
 
     async def make_application(self) -> web.Application:
         """Get the aiohttp application instance."""

acapy_agent/admin/tests/test_auth.py

@@ -133,3 +133,27 @@ async def test_multi_tenant_valid_auth_header(self):
         decor_func = tenant_authentication(self.decorated_handler)
         await decor_func(self.request)
         self.decorated_handler.assert_called_once_with(self.request)
+
+    async def test_base_wallet_additional_route_allowed(self):
+        self.profile.settings["multitenant.base_wallet_routes"] = "/extra-route"
+        self.request = mock.MagicMock(
+            __getitem__=lambda _, k: self.request_dict[k],
+            headers={"x-api-key": "admin_api_key"},
+            method="POST",
+            path="/extra-route",
+        )
+        decor_func = tenant_authentication(self.decorated_handler)
+        await decor_func(self.request)
+        self.decorated_handler.assert_called_once_with(self.request)
+
+    async def test_base_wallet_additional_route_denied(self):
+        self.profile.settings["multitenant.base_wallet_routes"] = "/extra-route"
+        self.request = mock.MagicMock(
+            __getitem__=lambda _, k: self.request_dict[k],
+            headers={"x-api-key": "admin_api_key"},
+            method="POST",
+            path="/extra-route-wrong",
+        )
+        decor_func = tenant_authentication(self.decorated_handler)
+        with self.assertRaises(web.HTTPUnauthorized):
+            await decor_func(self.request)