Skip to content

Container FQNs containing slashes fail permissions API routing (404) and search hydration #34235

Description

@aldwyn

Affected module

Backend, Ingestion Framework

Describe the bug

When S3/storage containers are ingested with subdirectories (for example via openmetadata.json configured with dataPath and depth > 0), the container's relative path contains slashes (e.g., ba/entec/woveycur/actor). This results in an entity FQN containing slashes (e.g., S3.my-bucket.ba/entec/woveycur/actor).

This causes two related issues:

  1. Permissions API route failure (404 Not Found):
    Calling GET /api/v1/permissions/container/name/{fqn} fails with 404 because PermissionsResource.java declares @Path("/{resource}/name/{name}") with the default single-segment regex [^/]+. When reverse proxies decode %2F into /, Jersey parses the remaining segments as extra unmatched path parts and returns 404.
  2. Search hydration failure in es_search_container_by_path:
    In ESMixin (_search_es_entity), search hits matched by Elasticsearch are hydrated by calling self.get_by_name(entity=entity_type, fqn=hit["_source"]["fullyQualifiedName"], fields=fields). Because the container FQN contains slashes, the backend call fails with 404. As a result, get_by_name returns None, and _search_es_entity skips the valid entity (if entity is None: continue), causing es_search_container_by_path to return None despite the container existing in both the database and Elasticsearch.

To Reproduce

  1. Ingest an S3 bucket with an openmetadata.json containing:
    {"entries":[{"dataPath":"ba","depth":3,"structureFormat":"json"}]}
  2. Ingestion creates a container entity with name ba/entec/woveycur/actor and FQN S3.<bucket>.ba/entec/woveycur/actor.
  3. Call GET /api/v1/permissions/container/name/S3.<bucket>.ba%2Fentec%2Fwoveycur%2Factor -> returns 404.
  4. Call metadata.es_search_container_by_path(full_path="s3://<bucket>/ba/entec/woveycur/actor", fields="dataModel") -> returns None even though metadata.list_all_entities(entity=Container) shows the record exists.

Expected behavior

  • The permissions endpoint should match slash-containing FQNs and return user permissions for the container.
  • es_search_container_by_path should successfully hydrate and return the container entity using its ID from the Elasticsearch search hit.

Additional context

Addressed in PR: #34234

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    • Status
      No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions