From 06fc1a59263f458af696a255f37da664240e321a Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Jan 2025 14:48:45 +0000 Subject: [PATCH 1/6] add context about updated_at support this pr adds what type the updated_at field of the timestamp strategy should have in relation to iso date strings and unix epoch integers. Resolves # #1948 --- website/docs/reference/resource-configs/updated_at.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/updated_at.md b/website/docs/reference/resource-configs/updated_at.md index 09122859e43..39ef7ae82d7 100644 --- a/website/docs/reference/resource-configs/updated_at.md +++ b/website/docs/reference/resource-configs/updated_at.md @@ -64,7 +64,7 @@ You will get a warning if the data type of the `updated_at` column does not matc ## Description A column within the results of your snapshot query that represents when the record row was last updated. -This parameter is **required if using the `timestamp` [strategy](/reference/resource-configs/strategy)**. +This parameter is **required if using the `timestamp` [strategy](/reference/resource-configs/strategy)**. The `updated_at` field may support ISO date strings and unix epoch integers, depending on the data platform you use. ## Default From a76dafe395e91165e564349bc995acdea0769da0 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Jan 2025 14:50:28 +0000 Subject: [PATCH 2/6] Update snapshots.md --- website/docs/docs/build/snapshots.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/build/snapshots.md b/website/docs/docs/build/snapshots.md index f72f1eb75de..7af036629c1 100644 --- a/website/docs/docs/build/snapshots.md +++ b/website/docs/docs/build/snapshots.md @@ -237,7 +237,7 @@ The `timestamp` strategy requires the following configurations: | Config | Description | Example | | ------ | ----------- | ------- | -| updated_at | A column which represents when the source row was last updated | `updated_at` | +| updated_at | A column which represents when the source row was last updated. May support ISO date strings and unix epoch integers, depending on the data platform you use. | `updated_at` | **Example usage:** @@ -586,7 +586,7 @@ The following table outlines the configurations available for snapshots in versi | [strategy](/reference/resource-configs/strategy) | The snapshot strategy to use. One of `timestamp` or `check` | Yes | timestamp | | [unique_key](/reference/resource-configs/unique_key) | A column or expression for the record | Yes | id | | [check_cols](/reference/resource-configs/check_cols) | If using the `check` strategy, then the columns to check | Only if using the `check` strategy | ["status"] | -| [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare | Only if using the `timestamp` strategy | updated_at | +| [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare. May support ISO date strings and unix epoch integers, depending on the data platform you use. | Only if using the `timestamp` strategy | updated_at | | [invalidate_hard_deletes](/reference/resource-configs/invalidate_hard_deletes) | Find hard deleted records in source, and set `dbt_valid_to` current time if no longer exists | No | True | - A number of other configurations are also supported (e.g. `tags` and `post-hook`), check out the full list [here](/reference/snapshot-configs). From 275eff46aaab42dde9b0814447493694ccad2514 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Jan 2025 15:17:13 +0000 Subject: [PATCH 3/6] Update snapshots.md --- website/docs/docs/build/snapshots.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/snapshots.md b/website/docs/docs/build/snapshots.md index 7af036629c1..609d6f04ee8 100644 --- a/website/docs/docs/build/snapshots.md +++ b/website/docs/docs/build/snapshots.md @@ -80,7 +80,7 @@ The following table outlines the configurations available for snapshots: | [strategy](/reference/resource-configs/strategy) | The snapshot strategy to use. Valid values: `timestamp` or `check` | Yes | timestamp | | [unique_key](/reference/resource-configs/unique_key) | A column(s) (string or array) or expression for the record | Yes | `id` or `[order_id, product_id]` | | [check_cols](/reference/resource-configs/check_cols) | If using the `check` strategy, then the columns to check | Only if using the `check` strategy | ["status"] | -| [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare | Only if using the `timestamp` strategy | updated_at | +| [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare. May support ISO date strings and unix epoch integers, depending on the data platform you use. | Only if using the `timestamp` strategy | updated_at | | [dbt_valid_to_current](/reference/resource-configs/dbt_valid_to_current) | Set a custom indicator for the value of `dbt_valid_to` in current snapshot records (like a future date). By default, this value is `NULL`. When configured, dbt will use the specified value instead of `NULL` for `dbt_valid_to` for current records in the snapshot table.| No | string | | [snapshot_meta_column_names](/reference/resource-configs/snapshot_meta_column_names) | Customize the names of the snapshot meta fields | No | dictionary | | [hard_deletes](/reference/resource-configs/hard-deletes) | Specify how to handle deleted rows from the source. Supported options are `ignore` (default), `invalidate` (replaces the legacy `invalidate_hard_deletes=true`), and `new_record`.| No | string | From 4d7448a78b01ae97cd51dd4911c7b4451da6d449 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Jan 2025 16:34:23 +0000 Subject: [PATCH 4/6] Update website/docs/docs/build/snapshots.md --- website/docs/docs/build/snapshots.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/snapshots.md b/website/docs/docs/build/snapshots.md index 609d6f04ee8..efb3539939d 100644 --- a/website/docs/docs/build/snapshots.md +++ b/website/docs/docs/build/snapshots.md @@ -80,7 +80,7 @@ The following table outlines the configurations available for snapshots: | [strategy](/reference/resource-configs/strategy) | The snapshot strategy to use. Valid values: `timestamp` or `check` | Yes | timestamp | | [unique_key](/reference/resource-configs/unique_key) | A column(s) (string or array) or expression for the record | Yes | `id` or `[order_id, product_id]` | | [check_cols](/reference/resource-configs/check_cols) | If using the `check` strategy, then the columns to check | Only if using the `check` strategy | ["status"] | -| [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare. May support ISO date strings and unix epoch integers, depending on the data platform you use. | Only if using the `timestamp` strategy | updated_at | +| [updated_at](/reference/resource-configs/updated_at) | A column in your snapshot query results that indicates when each record was last updated, used in the `timestamp` strategy. May support ISO date strings and unix epoch integers, depending on the data platform you use. | Only if using the `timestamp` strategy | updated_at | | [dbt_valid_to_current](/reference/resource-configs/dbt_valid_to_current) | Set a custom indicator for the value of `dbt_valid_to` in current snapshot records (like a future date). By default, this value is `NULL`. When configured, dbt will use the specified value instead of `NULL` for `dbt_valid_to` for current records in the snapshot table.| No | string | | [snapshot_meta_column_names](/reference/resource-configs/snapshot_meta_column_names) | Customize the names of the snapshot meta fields | No | dictionary | | [hard_deletes](/reference/resource-configs/hard-deletes) | Specify how to handle deleted rows from the source. Supported options are `ignore` (default), `invalidate` (replaces the legacy `invalidate_hard_deletes=true`), and `new_record`.| No | string | From e233c530d0463790457231adbfdd477619a293a1 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Jan 2025 16:34:49 +0000 Subject: [PATCH 5/6] Update website/docs/docs/build/snapshots.md --- website/docs/docs/build/snapshots.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/snapshots.md b/website/docs/docs/build/snapshots.md index efb3539939d..a592056ae42 100644 --- a/website/docs/docs/build/snapshots.md +++ b/website/docs/docs/build/snapshots.md @@ -586,7 +586,7 @@ The following table outlines the configurations available for snapshots in versi | [strategy](/reference/resource-configs/strategy) | The snapshot strategy to use. One of `timestamp` or `check` | Yes | timestamp | | [unique_key](/reference/resource-configs/unique_key) | A column or expression for the record | Yes | id | | [check_cols](/reference/resource-configs/check_cols) | If using the `check` strategy, then the columns to check | Only if using the `check` strategy | ["status"] | -| [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare. May support ISO date strings and unix epoch integers, depending on the data platform you use. | Only if using the `timestamp` strategy | updated_at | +| [updated_at](/reference/resource-configs/updated_at) | A column in your snapshot query results that indicates when each record was last updated, used in the `timestamp` strategy. May support ISO date strings and unix epoch integers, depending on the data platform you use. | Only if using the `timestamp` strategy | updated_at | | [invalidate_hard_deletes](/reference/resource-configs/invalidate_hard_deletes) | Find hard deleted records in source, and set `dbt_valid_to` current time if no longer exists | No | True | - A number of other configurations are also supported (e.g. `tags` and `post-hook`), check out the full list [here](/reference/snapshot-configs). From bb8bc58d1d46076a0099dc373083246e55f30b52 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Jan 2025 17:07:08 +0000 Subject: [PATCH 6/6] update description property page (#6733) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit this pr adds add'l examples to the description property page: - data tests: generic and singular - unit tests raised in [internal slack](https://dbt-labs.slack.com/archives/C07NBMC7XPT/p1736271689456609) Resolves #6193 --- 🚀 Deployment available! Here are the direct links to the updated files: - https://docs-getdbt-com-git-udpate-description-dbt-labs.vercel.app/reference/resource-properties/description --------- Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- .../resource-properties/description.md | 157 +++++++++++++++++- 1 file changed, 154 insertions(+), 3 deletions(-) diff --git a/website/docs/reference/resource-properties/description.md b/website/docs/reference/resource-properties/description.md index cf7b2b29a5a..7c182f1794d 100644 --- a/website/docs/reference/resource-properties/description.md +++ b/website/docs/reference/resource-properties/description.md @@ -14,6 +14,7 @@ description: "This guide explains how to use the description key to add YAML des { label: 'Analyses', value: 'analyses', }, { label: 'Macros', value: 'macros', }, { label: 'Data tests', value: 'data_tests', }, + { label: 'Unit tests', value: 'unit_tests', }, ] }> @@ -150,24 +151,81 @@ macros: +You can add a description to a [singular data test](/docs/build/data-tests#singular-data-tests) or a [generic data test](/docs/build/data-tests#generic-data-tests). + ```yml +# Singular data test example + version: 2 data_tests: - name: data_test_name description: markdown_string - ``` + + + + +```yml +# Generic data test example + +version: 2 +models: + - name: model_name + columns: + - name: column_name + tests: + - unique: + description: markdown_string +``` -The `description` property is available for generic and singular data tests beginning in dbt v1.9. +The `description` property is available for [singular data tests](/docs/build/data-tests#singular-data-tests) or [generic data tests](/docs/build/data-tests#generic-data-tests) beginning in dbt v1.9. + + + + + + + + + + + +```yml +unit_tests: + - name: unit_test_name + description: "markdown_string" + model: model_name + given: ts + - input: ref_or_source_call + rows: + - {column_name: column_value} + - {column_name: column_value} + - {column_name: column_value} + - {column_name: column_value} + - input: ref_or_source_call + format: csv + rows: dictionary | string + expect: + format: dict | csv | sql + fixture: fixture_name +``` + + + + + + + +The `description` property is available for [unit tests](/docs/build/unit-tests) beginning in dbt v1.8. @@ -176,13 +234,17 @@ The `description` property is available for generic and singular data tests begi ## Definition -A user-defined description. Can be used to document: + +A user-defined description used to document: + - a model, and model columns - sources, source tables, and source columns - seeds, and seed columns - snapshots, and snapshot columns - analyses, and analysis columns - macros, and macro arguments +- data tests, and data test columns +- unit tests for models These descriptions are used in the documentation website rendered by dbt (refer to [the documentation guide](/docs/build/documentation) or [dbt Explorer](/docs/collaborate/explore-projects)). @@ -196,6 +258,18 @@ Be mindful of YAML semantics when providing a description. If your description c ## Examples +This section contains examples of how to add descriptions to various resources: + +- [Add a simple description to a model and column](#add-a-simple-description-to-a-model-and-column)
+- [Add a multiline description to a model](#add-a-multiline-description-to-a-model)
+- [Use some markdown in a description](#use-some-markdown-in-a-description)
+- [Use a docs block in a description](#use-a-docs-block-in-a-description)
+- [Link to another model in a description](#link-to-another-model-in-a-description) +- [Include an image from your repo in your descriptions](#include-an-image-from-your-repo-in-your-descriptions)
+- [Include an image from the web in your descriptions](#include-an-image-from-the-web-in-your-descriptions)
+- [Add a description to a data test](#add-a-description-to-a-data-test)
+- [Add a description to a unit test](#add-a-description-to-a-unit-test)
+ ### Add a simple description to a model and column @@ -400,3 +474,80 @@ models: If mixing images and text, also consider using a docs block. +### Add a description to a data test + + + + + + + +You can add a `description` property to a generic or singular data test. + +#### Generic data test + +This example shows a generic data test that checks for unique values in a column for the `orders` model. + + + +```yaml +version: 2 + +models: + - name: orders + columns: + - name: order_id + tests: + - unique: + description: "The order_id is unique for every row in the orders model" +``` + + +#### Singular data test + +This example shows a singular data test that checks to ensure all values in the `payments` model are not negative (≥ 0). + + + +```yaml +version: 2 +data_tests: + - name: assert_total_payment_amount_is_positive + description: > + Refunds have a negative amount, so the total amount should always be >= 0. + Therefore return records where total amount < 0 to make the test fail. + +``` + + +Note that in order for the test to run, the `tests/assert_total_payment_amount_is_positive.sql` SQL file has to exist in the `tests` directory. + +### Add a description to a unit test + + + + + + + +This example shows a unit test that checks to ensure the `opened_at` timestamp is properly truncated to a date for the `stg_locations` model. + + + +```yaml +unit_tests: + - name: test_does_location_opened_at_trunc_to_date + description: "Check that opened_at timestamp is properly truncated to a date." + model: stg_locations + given: + - input: source('ecom', 'raw_stores') + rows: + - {id: 1, name: "Rego Park", tax_rate: 0.2, opened_at: "2016-09-01T00:00:00"} + - {id: 2, name: "Jamaica", tax_rate: 0.1, opened_at: "2079-10-27T23:59:59.9999"} + expect: + rows: + - {location_id: 1, location_name: "Rego Park", tax_rate: 0.2, opened_date: "2016-09-01"} + - {location_id: 2, location_name: "Jamaica", tax_rate: 0.1, opened_date: "2079-10-27"} +``` + +