Affected module
Ingestion

Describe the bug
Superset data models referencing Snowflake tables with ARRAY columns fail to ingest, raising an error

metadata.Ingestion [metadata_rest.py:147] Failed to ingest CreateDashboardDataModelRequest [9] due to api request failure: For column data type array, arrayDataType must not be null

To Reproduce

1. create a snowflake table with a column with ARRAY datatype
2. create a superset data model for this table
3. create a chart that uses this data model
4. create a dashboard that uses this chart
5. run a superset ingest

Expected behavior

Ingestion should proceed successfully. Arrays should either not require column data types, or the correct type should be inferred from the data source.

In Snowflake, ARRAY columns always contain VARIANT items https://docs.snowflake.com/en/sql-reference/data-types-semistructured#characteristics-of-an-array

Version:

• Python version: 3.9.17
• OpenMetadata version: 1.1.5 (also in 1.1.7)
• OpenMetadata Ingestion package version: openmetadata-ingestion==1.1.5

Additional context

Each value in an ARRAY is of type VARIANT

https://docs.snowflake.com/en/sql-reference/data-types-semistructured#characteristics-of-an-array