Handle invalid utf-8 bytes in engine class instead of model class

[JC1DA]

This PR removes unnecessary invalid utf-8 bytes handling code in Model class as we can better handle it in Engine class.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 84.61538% with 2 lines in your changes missing coverage. Please review.

Project coverage is 60.27%. Comparing base (aaa5f00) to head (190b3aa).

Files with missing lines	Patch %	Lines
guidance/models/_model.py	84.61%	2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1094       +/-   ##
===========================================
+ Coverage   49.79%   60.27%   +10.47%     
===========================================
  Files          71       71               
  Lines        5880     5875        -5     
===========================================
+ Hits         2928     3541      +613     
+ Misses       2952     2334      -618     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[JC1DA]

@hudson-ai can you help check this PR?

[hudson-ai]

Why is it that we only delay bytes if not echo?

[JC1DA]

there is another logic to process delayed_bytes if echo is True above.

[hudson-ai]

Seems reasonable to me, but I'm not sure I have a strong opinion on whether this should be done in the engine vs in the model. Can you help me understand why you like this approach more?

[JC1DA]

Seems reasonable to me, but I'm not sure I have a strong opinion on whether this should be done in the engine vs in the model. Can you help me understand why you like this approach more?

First, I think it does not make sense for the engine to generate bytes that the model could not even use, and have to do post-processing on the result.
Second, we need to process delayed_bytes in the engine right now anyway, so why not keeping it there :)