Skip to content

Add deterministic workflow value portfolio - #78744

Draft
mnkiefer wants to merge 1 commit into
mainfrom
mnkiefer-miniature-sniffle
Draft

mnkiefer wants to merge 1 commit into
mainfrom
mnkiefer-miniature-sniffle

Conversation

@mnkiefer

Copy link
Copy Markdown
Collaborator

Why

GitHub Agentic Workflows needs one evidence-first portfolio for every declared workflow without inventing value from run success, output volume, labels, grader scores, or legacy outcome proxies. Existing metrics history and the retained lifecycle ledger already contain the authoritative inputs, but there was no complete deterministic report or time-series surface.

What changed

  • Derive the inventory from paired top-level workflow source and lock files and fail on omissions, extras, or duplicate identities.
  • Extend the Metrics Collector with prospective lifecycle snapshots and a bounded deterministic generator that produces current/history JSON, Markdown, and SVG plots.
  • Calculate the current delivery cohort from canonical ledger delivery clocks, keep retained-population snapshots separate, and preserve coverage, ledger identity, and lifecycle clocks.
  • Report attributed value as needs_definition or unavailable unless reviewed longitudinal definitions and comparable evidence actually exist. Restricted-JavaScript operational-value declarations remain explicitly run-scoped.
  • Add a local read-only entry point: bash scripts/workflow-value-report.sh.
  • Teach Agent Performance Analyzer to consume the generated artifact rather than reconstruct value from proxies.

The current retained history resolves to 368 workflows across 57 periods: 11 complete, 40 partial, and 6 legacy unavailable. No workflow currently has a repository-local reviewed longitudinal value definition, so the report correctly records 0 measured and 368 needs_definition.

Validation

  • make agent-report-progress (passed; 73 impacted JavaScript tests, lint/typecheck, workflow drift)
  • All 368 workflow lock files compiled successfully and are in sync
  • Full retained-history preview verified at 482,742 bytes
  • One-workflow execution verified all seven output files at 25,700 bytes
  • Local command exercised successfully against temporary metrics and lifecycle branches

Security review

  • No new secrets, redirects, write permissions, or credential flows.
  • The report upload uses actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 (v7.0.2) and uploads only generated files under /tmp/gh-aw/workflow-value-report.
  • The modified repo-memory validation script only reads and validates bounded metrics/lifecycle JSON, ledger digests, timestamps, and retention limits.

Review notes

  • memory/output-lifecycle is created by the first updated Metrics Collector run; lifecycle history is prospective unless an exact ledger is explicitly seeded.
  • Historical snapshots without supported lifecycle/value fields remain unavailable gaps, never zeros.
  • This PR does not dispatch a live workflow or add a universal value score.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
];
for (const workflow of portfolio.workflows) {
lines.push(
`| \`${markdown(workflow.id)}\` | ${deliveryLabel(workflow.current.selected_window.delivery)} | ${adoptionLabel(workflow.current.selected_window.adoption)} | ${deliveryLabel(workflow.current.retained_population)} | ${adoptionLabel(workflow.current.retained_population)} | ${survivalLabel(workflow.current.retained_population)} | ${markdown(workflow.current.attributed_value.state.replaceAll("_", " "))} | ${markdown(workflow.operational_value.state.replaceAll("_", " "))} |`

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants