Skip to content

Commit

Permalink
ENH: handle unicode enc error in file_write()
Browse files Browse the repository at this point in the history
One some (HPC) systems, we occasionally see random errors of the form

        UnicodeEncodeError: 'ascii' codec can't encode characters
        in position 4475-4476

when calling file_write() caused by run(..., capture_logs="file"). This may be
caused by oldish file systems that can't handle unicode files.
  • Loading branch information
elcorto committed Feb 29, 2024
1 parent 77afced commit 2971791
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion src/psweep/psweep.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,10 @@ def flatten(seq):
def file_write(fn: str, txt: str, mode="w"):
makedirs(os.path.dirname(fn))
with open(fn, mode=mode) as fd:
fd.write(txt)
try:
fd.write(txt)
except UnicodeEncodeError:
fd.write(txt.encode("ascii", errors="xmlcharrefreplace").decode())


def file_read(fn: str):
Expand Down

0 comments on commit 2971791

Please sign in to comment.