Skip to content

feat(scrapers): read the client-rendered drama school faculty roster #2788

Description

@quntao-z

What

https://www.drama.yale.edu/faculty/ is the David Geffen School of Drama faculty page linked from Yale's A-Z catalog. No dept-faculty-roster lane reads it, and none can as a config row: the served HTML contains only navigation and modal markup, with a single link matching a person-profile shape, so every existing extractor returns zero rows against it. The faculty list is client-rendered.

Found while auditing all 142 A-Z catalog entries for #2787. It is the only academic unit on that page with a real faculty roster that nothing reads and that a config row cannot reach.

Why it needs its own change

The repo already has the two pieces this needs, so the work is wiring rather than invention: renderWaitSelector plus renderedExtractor on DeptConfig, and createScraplingRenderedFetcher. What it does not have is an extractor for this page's markup, which is WordPress rather than the Drupal shapes the other lanes share.

Suggested shape

  1. Render the page with the existing rendered-fetch path and capture the hydrated markup.
  2. Write a pure extractor beside the others in departmentRosterScraper.ts, with a fixture test over a redacted snapshot.
  3. Add the lane with renderWaitSelector set to whatever container holds the cards, and decide the department-claim flag using the table in skills/scrapers/SKILL.md. Drama faculty are appointed in the school, so no crossListedProgramme flag is needed.
  4. Run it against Development and verify by reading the department filter, per the stored-data definition of done.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions