[ADAP-949] [Bug] dbt docs generate timeout

[dongchris]

Is this a new bug in dbt-bigquery?

• I believe this is a new bug in dbt-bigquery
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Getting a timeout error when running dbt docs generate after 600 seconds (10 mins)

Profiles.yml - job_execution_timeout_seconds: 21000

on dbt 1.6.6

slack thread - https://getdbt.slack.com/archives/C99SNSRTK/p1695856526889609

23:22:20 Encountered an error while generating catalog: Runtime Error

  | Deadline of 600.0s exceeded while calling target function, last exception: 500 Error encountered during execution. Retrying may solve the problem.

Expected Behavior

I expect dbt docs generate to complete successfully without no time out

Steps To Reproduce

dbt docs generate

Relevant log output

No response

Environment

- OS:
- Python: 3.8
- dbt-core: 1.6.6
- dbt-bigquery: 1.6.6

Additional Context

No response

[dbeatty10]

Thanks for opening this @dongchris !

I just read your most recent comment in Slack here:

hmm just retried it and it worked. seems like its instability with Google BQ API library perhaps

So I'm going to close this as resolved. But just let us know if you are still having problems, and we can re-open this.

[dataders]

re-opening per @dongchris's request in Slack. looking at #902, perhaps the BQ client has a baked in 600 second timeout if a timeout larger than 600 is provided?

[dongchris]

re-opening per @dongchris's request in Slack. looking at #902, perhaps the BQ client has a baked in 600 second timeout if a timeout larger than 600 is provided?

Can you point to where the bq client code is?
Relevant stack overflow thread https://stackoverflow.com/questions/75475875/timeout-error-when-writing-large-amounts-of-data-to-big-query

[dbeatty10]

@dongchris some of the frequently-used client code is here:

• dbt-bigquery/dbt/adapters/bigquery/connections.py

  Line 459 in f0f6d14

  def raw_execute(

• dbt-bigquery/dbt/adapters/bigquery/connections.py

  Line 732 in f0f6d14

  def _query_and_results(

I didn't research if dbt docs generate is hitting those particular locations or not though.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[dataders]

This is related to #263 and #577. There actually may be a path to solve this with the recent update to the BQ python client library

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[amychen1776]

@dongchris are you still experiencing this?
@dataders do you see any more indications of this?

[amychen1776]

Closing this for now due to lack of response.