Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When detecting metadata from dataframes, allow me the option to turn on/off sdtype and relationship detection #2341

Open
npatki opened this issue Jan 9, 2025 · 0 comments
Labels
feature request Request for a new feature

Comments

@npatki
Copy link
Contributor

npatki commented Jan 9, 2025

Problem Description

Currently, our AI Connectors offer some customizability when detecting metadata from a database. All of them allow you to toggle on/off the detection for individual sdtypes and for keys (primary/foreign key relationships). For eg. See this API (copied below)

infer_sdtypes: A boolean describing whether to infer the sdtypes of each column

  • (default) True: Infer the sdtypes based on the data
  • False: Do not infer the sdtypes. All columns will be marked as unknown, ready for you to manually update

infer_keys: A string describing whether to infer the primary and/or foreign keys. Options are:

  • (default) 'primary_and_foreign': Infer the primary keys in each table, and the foreign keys in other tables that refer to them
  • 'primary_only': Infer only the primary keys of each table
  • None: Do not infer any keys

By contrast, when detecting metadata from DataFrames, we do not allow these options of infer_sdtypes and infer_keys. This functionality would come in useful when a user just wants to auto-detect part of the metadata (eg. maybe because they have other, explicit info they'd like to input themselves). For example, see issue #2338.

Expected behavior

It would be nice if the detect_from_dataframes function also had the same functionality as AI Connectors.

Essentially, we'd add the following parameters that would behave the exact same way as AI Connectors:

  • infer_sdtypes
  • infer_keys
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Request for a new feature
Projects
None yet
Development

No branches or pull requests

1 participant