[vdusek]

PlaywrightCrawler and StagehandCrawler now show the extra headers a browser context sends (from the fingerprint or header generator) in context.request.headers.

• BrowserController.get_context_headers(page) returns these headers. new_page() is unchanged, and the default implementation returns {}, so custom controllers keep working. CrawleePage gets a context_headers field.
• Only headers the browser sends unchanged are reported. Request-dependent ones like Accept and Sec-Fetch-* are filtered out with browserforge's only_injectable_headers, because Chromium sets them per request.
• The crawler adds only the missing keys, and only for the current attempt; they are removed when the attempt's pipeline ends. Nothing is persisted in the RQ, a retry shows the headers of its new context, and the static sub-crawl of AdaptivePlaywrightCrawler does not send them.
• An added header skips the navigation interceptor unless the user changes its value. A changed value is sent and stays on the request.
• The HTTP headers guide describes the new behavior.

Supersedes #2284.

Closes: #1055

✍️ Drafted by Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.55172% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.03%. Comparing base (06d331b) to head (75f13d5).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/crawlee/browsers/_browser_controller.py	50.00%	1 Missing ⚠️
...rawlee/crawlers/_playwright/_playwright_crawler.py	96.15%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2383      +/-   ##
==========================================
- Coverage   94.05%   94.03%   -0.02%     
==========================================
  Files         184      184              
  Lines       13662    13707      +45     
==========================================
+ Hits        12850    12890      +40     
- Misses        812      817       +5     

Flag	Coverage Δ
unit	94.03% <96.55%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[apify-factory]

Looks good, nothing blocking.

🏭 Factory review — ℹ️ not auto-approved

This change was reviewed but cannot be auto-approved: a gate failed, so the review cannot end in an approval.

check	result
Static gates	❌ 13/16 — max-files, max-lines, file-extensions
Review findings	✅ none
Reviewer escalation	❌ Extends public interfaces (BrowserController.get_context_headers, CrawleePage.context_headers) and changes what context.request.headers shows in hooks and handlers, along with which headers go through the navigation interceptor.
Juror 1 — claude-sonnet-5-5, adversarial	⏭️ skipped — a gate failed

Why it was not auto-approved

Failed gates:

• max-files: 11 changed files, limit 10
• max-lines: 261 changed lines (+238/-23), limit 200
• file-extensions: disallowed extension: docs/03_concepts/16_http_headers.mdx

The comments above are the review itself; this section only explains the approval decision. Push a change and request @apify-factory again to re-run it, or ask a human to review.

_{Reviewed 75f13d56f · workflow run}

[Mantisus]

We need to double-check how TS behaves here. But at the moment I'm not sure there's a bug here.

Request contains the headers explicitly set by the user. If we want it to contain the headers that were actually sent, it should be all of them, not only the generated ones. In that case it should also cover the HTTP clients, since CurlImpersonateHttpClient and ImpitHttpClient send their own browser headers, and HttpxHttpClient uses the header generator.

For the browser, the headers that were actually sent are already available via await context.response.request.all_headers().