Skip to content

Commit

Permalink
Only use alphanumeric characters within batch ids
Browse files Browse the repository at this point in the history
  • Loading branch information
dbeatty10 committed Jan 17, 2025
1 parent a669c1d commit 85e0e32
Showing 1 changed file with 12 additions and 4 deletions.
16 changes: 12 additions & 4 deletions core/dbt/materializations/incremental/microbatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,13 +194,21 @@ def truncate_timestamp(timestamp: datetime, batch_size: BatchSize) -> datetime:

@staticmethod
def batch_id(start_time: datetime, batch_size: BatchSize) -> str:
return MicrobatchBuilder.format_batch_start(start_time, batch_size).replace("-", "")
return MicrobatchBuilder.format_batch_start(start_time, batch_size)

Check warning on line 197 in core/dbt/materializations/incremental/microbatch.py

View check run for this annotation

Codecov / codecov/patch

core/dbt/materializations/incremental/microbatch.py#L197

Added line #L197 was not covered by tests

@staticmethod
def format_batch_start(batch_start: datetime, batch_size: BatchSize) -> str:
return str(
batch_start.date() if (batch_start and batch_size != BatchSize.hour) else batch_start
)
"""Format the passed in datetime based on the batch_size.
2024-09-17 16:06:00 + Batchsize.day -> 20240917
2024-09-17 16:06:00 + Batchsize.hour -> 20240917T16
"""
# If we want a date only
if batch_size != BatchSize.hour:
return batch_start.strftime("%Y%m%d")

Check warning on line 208 in core/dbt/materializations/incremental/microbatch.py

View check run for this annotation

Codecov / codecov/patch

core/dbt/materializations/incremental/microbatch.py#L207-L208

Added lines #L207 - L208 were not covered by tests

# If we want date + time
return batch_start.strftime("%Y%m%dT%H")

Check warning on line 211 in core/dbt/materializations/incremental/microbatch.py

View check run for this annotation

Codecov / codecov/patch

core/dbt/materializations/incremental/microbatch.py#L211

Added line #L211 was not covered by tests

@staticmethod
def ceiling_timestamp(timestamp: datetime, batch_size: BatchSize) -> datetime:
Expand Down

0 comments on commit 85e0e32

Please sign in to comment.