Skip to content

Commit

Permalink
Improved Glue job script spreadsheet_library_tracking_metadata
Browse files Browse the repository at this point in the history
* Added step to strip whitespaces and carriage return values
  • Loading branch information
victorskl committed Jan 11, 2025
1 parent 2d0de16 commit 60a9bdd
Showing 1 changed file with 3 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,9 @@ def transform():
.name.keep()
)

# strip whitespaces, carriage return
df = df.with_columns(pl.col(pl.String).str.strip_chars())

# drop row iff all values are null
# https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.drop_nulls.html
df = df.filter(~pl.all_horizontal(pl.all().is_null()))
Expand Down

0 comments on commit 60a9bdd

Please sign in to comment.