Skip to content

feat(Signup): Track signup experiment conversion on activation - #8674

Open
Zaimwa9 wants to merge 3 commits into
mainfrom
feat/signup-experiment-conversion
Open

Zaimwa9 wants to merge 3 commits into
mainfrom
feat/signup-experiment-conversion

Conversation

@Zaimwa9

@Zaimwa9 Zaimwa9 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Contributes to #8595

Adds a conversion event for the signup_corporate_only experiment, reconciling the frontend exposure (transient anonymous identity) with the backend activation flow. No migration.

  • Frontend sends signup_anonymous_id (the id used for the exposure) in the register payload when the visitor entered the experiment.
  • Signup stores it in FFAdminUser.onboarding_data when it is a valid UUID; it is not exposed by /me.
  • On user_activated, the backend tracks signup_activated via OpenFeature with the anonymous id as targeting key. Tracking failures are logged (signup.conversion.tracking_failed) and never break activation.

Storing the id server-side means every activation path (resend from any device, expired link) converts without touching the activation email.

Deploy the backend before or with the frontend merge.

How did you test this code?

  • Unit tests for signup storage (valid / invalid / missing id), conversion tracked / not tracked, tracking failure and events disabled.
  • Manual on staging: sign up with a free email domain, click the activation link, check signup_activated is recorded for the anonymous identity.

@vercel

vercel Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
docs Ignored Ignored Preview Oct 5, 2026 2:27pm UTC
flagsmith-frontend-preview Ignored Ignored Preview Oct 5, 2026 2:27pm UTC
flagsmith-frontend-staging Ignored Ignored Preview Oct 5, 2026 2:27pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

📝 Walkthrough

Walkthrough

The home page includes a truthy stored signup anonymous ID in registration requests. The API stores string values with lengths from 1 to 64 characters in onboarding data. On user activation, a new signal receiver tracks signup_activated using the stored ID as the targeting key. Tracking and parsing exceptions are logged. Unit tests cover storage and activation tracking, and the event catalogue documents the exception log.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🔵 Low · up to c45aa

Signup conversion attribution can be inaccurate when a whitespace-only identifier is already stored. This is a narrow, fixable case; the change is mergeable with owner awareness.

Security Architecture Review

Security architecture risk: 🔵 Low · up to c45aa

The change does not visibly alter account permissions, and tracking exceptions are contained. The identifier remains client-controlled and is now linked to the account. Downstream telemetry isolation and activation recovery behavior are not fully verified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The repository-visible exposure is the registrant's user record and conversion attribution in the configured telemetry environment. The targeting key does not visibly select another tenant or grant credentials. Provider-side isolation, retention, and deployed environment scope are unverified.

Security Findings and Attack Paths

  • inferred — A registrant can choose a bounded targeting key and have a conversion attributed to that key when activation invokes the receiver. However, arbitrary exposure identity selection already existed in the base frontend. The new attribution linkage is not evidence of account impersonation, and no security-authoritative use of the conversion event was identified.

Trust Boundaries and Controls

  • observed — Client-controlled attribution crosses into account storage and then into server-configured telemetry. Type and length checks bound storage but do not authenticate exposure ownership. Explicit response serialization prevents the identifier from appearing in the current-user onboarding response.

Resilience and Maintainability Implications

  • observed — The conversion receiver catches parsing, client initialization, and tracking exceptions without mutating activation or permissions. Unit tests exercise parsing and tracking failures and assert logging without propagation. They do not establish recovery or exactly-once behavior.

Hardening Proposals

  • proposed — Define retention for the account-linked experiment identifier and keep conversion data explicitly non-authoritative. If future decisions require verified exposure provenance, introduce a verifiable binding rather than treating UUID format or a client-supplied key as proof of identity.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added front-end Issue related to the React Front End Dashboard api Issue related to the REST API docs Documentation updates labels Oct 5, 2026
@github-actions github-actions Bot added feature New feature or request and removed docs Documentation updates labels Oct 5, 2026
@Zaimwa9

Zaimwa9 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

@themis-blindfold review

@codecov

codecov Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.84%. Comparing base (df5e8a2) to head (c45aa49).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8674      +/-   ##
==========================================
- Coverage   98.84%   98.84%   -0.01%     
==========================================
  Files        1659     1664       +5     
  Lines       68541    68414     -127     
==========================================
- Hits        67751    67624     -127     
  Misses        790      790              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 30731320-807e-4df6-ba6a-d200b25da4e5
📥 Commits

Reviewing files that changed from the base of the PR and between edfb91d and 418d373.

📒 Files selected for processing (7)
  • api/custom_auth/signals.py
  • api/custom_auth/views.py
  • api/tests/unit/custom_auth/test_unit_custom_auth_signals.py
  • api/tests/unit/custom_auth/test_unit_custom_auth_views.py
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md
  • frontend/common/types/requests.ts
  • frontend/web/components/pages/home-page/HomePage.tsx

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

Comment thread api/custom_auth/signals.py Outdated
Comment thread api/custom_auth/views.py Outdated
@themis-blindfold

Copy link
Copy Markdown
Contributor

⚖️ Themis review: 🟠 Fix before merge

Observed: the new conversion path rejects an identity format that the existing exposure hook deliberately generates when crypto.randomUUID is unavailable. Predicted: activations using that fallback identity would not be counted. General Unit Tests and changed-file lint passed; the API unit-test matrices were still running.

Area Score
🎯 Correctness 3/5
🧪 Test coverage 3/5
📐 Code quality 3/5
🚀 Product impact 3/5

🟠 Majors

  • api/custom_auth/views.py: accept the exposure hook's fallback identity format so activated signups are tracked.
📝 Walkthrough
  • Signup form - forwards the anonymous experiment identity with the registration request after exposure.
  • Registration endpoint - validates and persists that identity in private onboarding data.
  • Activation signal - emits signup_activated against the saved anonymous identity and keeps activation non-blocking if tracking fails.
  • Observability - documents the structured tracking-failure event.
🧪 How to verify
  1. Simulate a browser without crypto.randomUUID, enter the signup experiment, register, and activate the account.
  2. Confirm the timestamp/random fallback identity is persisted and receives exactly one signup_activated event on activation.
  3. Repeat with a UUID identity and with tracking disabled or failing; activation should still complete.
  4. Run cd api && uv run pytest tests/unit/custom_auth/test_unit_custom_auth_views.py tests/unit/custom_auth/test_unit_custom_auth_signals.py.
    Automate: add a client-to-activation regression test for the crypto.randomUUID fallback identity.

Product take: This is a solid experiment-measurement improvement, but the identifier contract needs alignment before conversion reporting can be complete.

🧭 Assumptions & unverified claims

No unverified assumptions or claims.

A conversion only counts when both ends agree on its identity · reviewed at 418d373

@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Oct 5, 2026
@Zaimwa9

Zaimwa9 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

@themis-blindfold review

@themis-blindfold

Copy link
Copy Markdown
Contributor

⚖️ Themis review: ✅ Ship it

TL;DR: This connects the experiment exposure identity to the email-activation conversion without exposing it through the current-user response. The completed CI suite passed, including API unit tests, linting, documentation checks, and static analysis.

Area Score
🎯 Correctness 5/5
🧪 Test coverage 4/5
📐 Code quality 4/5
🚀 Product impact 3/5
📝 Walkthrough
  • Signup UI - includes the transient experiment identity with the registration request only after an experiment variant is available.
  • Registration and activation - preserves a bounded anonymous identifier privately in onboarding data, then reports signup_activated against that identity when the user activates.
  • Reliability and observability - malformed data and tracking failures do not interrupt activation, while failures are recorded as a structured event.
🧪 How to verify
  1. Run uv run pytest tests/unit/custom_auth/test_unit_custom_auth_signals.py tests/unit/custom_auth/test_unit_custom_auth_views.py -q from api.
  2. Enable email activation and the signup experiment, register with a free-email address, then activate from the email link; confirm one signup_activated event for the exposed anonymous identity.
  3. Repeat registration with a missing, malformed, or overlong signup_anonymous_id; confirm activation still succeeds and no conversion is emitted.
  4. Configure the event client to fail and activate an eligible user; confirm the response succeeds and signup.conversion.tracking_failed is logged without an email address.
    Automate: add an activation-endpoint integration test that asserts the conversion client receives the identifier persisted during registration.

Product take: A solid measurement improvement for the signup experiment: it attributes completed email verification to the same anonymous identity that saw the variant. Its product scope is intentionally narrow and low-risk.

🧭 Assumptions & unverified claims

No unverified assumptions or claims.

The conversion now survives the long walk from form submit to inbox click · reviewed at c45aa49

@Zaimwa9
Zaimwa9 marked this pull request as ready for review October 5, 2026 14:59
@Zaimwa9
Zaimwa9 requested review from a team as code owners October 5, 2026 14:59
@Zaimwa9
Zaimwa9 requested review from talissoncosta and removed request for a team October 5, 2026 14:59
@Zaimwa9
Zaimwa9 requested review from emyller and matthewelwell and removed request for emyller and talissoncosta October 5, 2026 14:59
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-8674 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api-test:pr-8674 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api:pr-8674 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith:pr-8674 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-private-cloud:pr-8674 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-frontend:pr-8674 Finished ✅ Results ✅

@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request labels Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #21215 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  45.5 seconds
commit  c45aa49
info  🔄 Run: #21215 (attempt 1)

🗂️ Previous results
✅ private-cloud · depot-ubuntu-latest-16 — run #21215 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  31.6 seconds
commit  c45aa49
info  🔄 Run: #21215 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #21215 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  36.1 seconds
commit  c45aa49
info  🔄 Run: #21215 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #21215 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  32 seconds
commit  c45aa49
info  🔄 Run: #21215 (attempt 1)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: e4ab75be-d648-400e-b261-599325277c76
📥 Commits

Reviewing files that changed from the base of the PR and between 418d373 and c45aa49.

📒 Files selected for processing (4)
  • api/custom_auth/signals.py
  • api/custom_auth/views.py
  • api/tests/unit/custom_auth/test_unit_custom_auth_signals.py
  • api/tests/unit/custom_auth/test_unit_custom_auth_views.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

Comment thread api/custom_auth/views.py
Comment on lines +200 to +203
if (
not isinstance(signup_anonymous_id, str)
or not 0 < len(signup_anonymous_id) <= 64
):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject whitespace-only signup identifiers.

If signup_anonymous_id is " ", this check stores it. Activation then uses that effectively blank value as the conversion targeting key. Reject values whose stripped form is empty, and add a matching registration case. Preserve the original non-blank identifier so the targeting key still matches the frontend exposure.

Based on learnings, whitespace-only strings must not pass identifier validation as non-empty values.

Source: Learnings

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Visual Regression

19 screenshots compared. See report for details.
View full report

This branch was successfully deployed

2 active (outdated) deployments
Preview – flagsmith-frontend-staging — 418d373a Deployed Oct 5, 2026 by vercel[bot]
Preview – flagsmith-frontend-preview — 418d373a Deployed Oct 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API feature New feature or request front-end Issue related to the React Front End Dashboard

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant