Other

patient-consent-simplifier

Simplify informed consent documents into patient-friendly language while maintaining regulatory compliance (FDA 21CFR50, ICH-GCP, HIPAA) and required legal elements.

85100Total Score
Core Capability
86 / 100
Functional Suitability
11 / 12
Reliability
10 / 12
Performance & Context
6 / 8
Agent Usability
14 / 16
Human Usability
7 / 8
Security
11 / 12
Maintainability
9 / 12
Agent-Specific
18 / 20
Medical Task
20 / 20 Passed
87Simplify a clinical trial ICF paragraph to grade 8
4/4
85Adapt consent text for vulnerable population at grade 6
4/4
82Compliance-only check on a document with missing contact information element
4/4
83Simplify a surgical procedure consent form with heavy medical jargon
4/4
81Request to draft a new consent form from scratch (out-of-scope)
4/4

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS

Core Capability86 / 100 — 8 Categories

Functional Suitability
Covers simplification, compliance check, and readability reporting; original drafting and regulatory submission explicitly excluded. PHI check now enforced as step 1.
11 / 12
92%
Reliability
Error handling documented; PHI check step added; recoverability path present. Minor: no explicit retry or rollback for partial simplification.
10 / 12
83%
Performance & Context
No progressive disclosure of reference files; all content in single SKILL.md.
6 / 8
75%
Agent Usability
Workflow clear with PHI check as step 1. Feedback design good via response template. Error prevention covers fabrication and scope. Input Validation redirect now includes specific alternative (IRB template library).
14 / 16
88%
Human Usability
Description is discoverable. Input Validation refusal now includes actionable next-step suggestion for out-of-scope requests.
7 / 8
88%
Security
PHI/PII detection step now enforced in workflow step 1 with mandatory warning. No credential handling needed. Input validation present.
11 / 12
92%
Maintainability
Clean single-file structure; modifiable; no test cases provided.
9 / 12
75%
Agent-Specific
Trigger precision good; escape hatches present with actionable alternative in refusal; composability moderate — no documented output schema for downstream use.
18 / 20
90%
Core Capability Total86 / 100

Medical TaskExecution Average: 83.6 / 100 — Assertions: 20/20 Passed

87
Canonical
Simplify a clinical trial ICF paragraph to grade 8
4/4 ✓
85
Variant A
Adapt consent text for vulnerable population at grade 6
4/4 ✓
82
Edge
Compliance-only check on a document with missing contact information element
4/4 ✓
83
Variant B
Simplify a surgical procedure consent form with heavy medical jargon
4/4 ✓
81
Stress
Request to draft a new consent form from scratch (out-of-scope)
4/4 ✓
87
Canonical✅ Pass
Simplify a clinical trial ICF paragraph to grade 8

PHI check step now prompts for de-identification before processing. Simplification output complete.

Basic 36/40|Specialized 51/60|Total 87/100
✅A1Output preserves all required consent elements (purpose, risks, benefits, alternatives, confidentiality, contact)
✅A2Output uses active voice and second-person language
✅A3Output includes a readability report or grade-level assessment
✅A4Output does not fabricate compliance status or remove legally required elements
Pass rate: 4 / 4
85
Variant A✅ Pass
Adapt consent text for vulnerable population at grade 6

Output completed successfully; adapt consent text for vulnerable population at grade 6 case handled within expected scope.

Basic 35/40|Specialized 50/60|Total 85/100
✅A1Output targets grade 6 reading level as requested
✅A2Required legal elements are preserved in simplified form
✅A3Output does not downplay significant risks
✅A4Compliance status section is present
Pass rate: 4 / 4
82
Edge✅ Pass
Compliance-only check on a document with missing contact information element

Output completed successfully; compliance-only check on a document with missing contact information element case handled within expected scope.

Basic 34/40|Specialized 48/60|Total 82/100
✅A1Output correctly identifies the missing required element
✅A2Output does not fabricate a compliant status when elements are missing
✅A3Output provides actionable guidance on what to add
✅A4Scope is not exceeded (no legal drafting performed)
Pass rate: 4 / 4
83
Variant B✅ Pass
Simplify a surgical procedure consent form with heavy medical jargon

Output completed successfully; simplify a surgical procedure consent form with heavy medical jargon case handled within expected scope.

Basic 34/40|Specialized 49/60|Total 83/100
✅A1Medical jargon is replaced with plain-language equivalents
✅A2Sentences exceeding 20 words are broken down
✅A3Output includes risks and discomforts section
✅A4Output does not provide legal advice or regulatory certification
Pass rate: 4 / 4
81
Stress✅ Pass
Request to draft a new consent form from scratch (out-of-scope)

Skill correctly refuses and now provides specific alternative: consult institution's IRB template library or regulatory affairs specialist.

Basic 34/40|Specialized 47/60|Total 81/100
✅A1Skill refuses to draft a new consent form from scratch
✅A2Refusal message references the correct scope boundary
✅A3No fabricated consent content is produced
✅A4Output suggests an appropriate alternative action
Pass rate: 4 / 4
Medical Task Total83.6 / 100

Key Strengths

  • PHI/PII detection step now enforced as step 1 in workflow — mandatory warning before processing any document containing patient identifiers
  • Out-of-scope refusal now includes specific actionable alternative (IRB template library or regulatory affairs specialist)
  • Comprehensive required-elements checklist (9 FDA/ICH-GCP elements) is explicitly enumerated and enforced
  • Error handling explicitly prohibits fabrication of compliance status — a critical safety property for this domain