High Risk Legal Help Queries

A 192-item evaluation dataset focused on jurisdiction-variance, procedural detail, eligibility nuance, and collateral consequences. Items target sixteen high-risk legal scenarios identified through subject-matter expert review, including responding to an eviction lawsuit, rent withholding and repair-and-deduct, responding to a divorce petition, expungement of criminal records, protective orders, criminal plea consequences for noncitizens, and interstate custody under the UCCJEA. Each scenario is rendered across the six covered states with two sophistication levels: a moderate-length query (around 25 words) and a longer query (around 90 words) carrying overlapping facts, time pressure, embedded user context, and occasional half-cited rules.

Each item carries a Risk-Tier (Yellow, Yellow-High, Red), a Jurisdiction-Variation-Level (Mostly the Same, Somewhat Different, Very Different), a Stability-Level (Stable, Perishable), and a set of Escalation-Triggers (Deadline Imminent, DV-Safety-Risk, Irreversible Action, Multi-Jurisdiction). Rubrics anchor each item to specific Justice Knowledge Base records per state where coverage exists, and flag content gaps where it does not.

Size: 192 items (16 scenarios x 6 states x 2 sophistication levels) Jurisdictions: Michigan, Ohio, Oregon, Texas, Illinois, West Virginia

Format: Airtable

Provenance: Seed scenarios from a Stanford Legal Design Lab Content Safety Wrapper review by practitioners in MI, IL, TX; extrapolated to six states using the Justice Knowledge Base as the source-of-truth for state-specific rubrics; all LIST terms verified against the LIST taxonomy

Status: Version 3, draft