Data Analysis
protocol-deviation-classifier
Determine whether an incident in a clinical trial is a "major deviation.
85100Total Score
Core Capability
87 / 100
Functional Suitability
11 / 12
Reliability
10 / 12
Performance & Context
8 / 8
Agent Usability
14 / 16
Human Usability
7 / 8
Security
10 / 12
Maintainability
10 / 12
Agent-Specific
17 / 20
Medical Task
17 / 20 Passed
100Determine whether an incident in a clinical trial is a "major deviation
4/4
99Use this skill for data analysis tasks that require explicit assumptions, bounded scope, and a reproducible output format
4/4
92Determine whether an incident in a clinical trial is a "major deviation
4/4
65Packaged executable path(s): scripts/main.py
3/4
65End-to-end case for Scope-focused workflow aligned to: Determine whether an incident in a clinical trial is a "major deviation
2/4
Veto GatesRequired pass for any deployment consideration
Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASSResearch Veto✅ PASS — Applicable
| Dimension | Result | Detail |
|---|---|---|
| Scientific Integrity | PASS | The archived review kept this workflow anchored to supplied data fields and observable execution behavior, not fabricated results. |
| Practice Boundaries | PASS | Practice boundaries held because the package remained focused on Determine whether an incident in a clinical trial is a "major deviation rather than overclaiming what the records supported. |
| Methodological Ground | PASS | Methodological grounding was preserved through the documented inputs, transformations, and expected artifacts. |
| Code Usability | PASS | The legacy audit did not record a code-usability failure in the packaged analysis path. |
Core Capability87 / 100 — 8 Categories
Functional Suitability
Related legacy finding for protocol-deviation-classifier: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
11 / 12
92%
Reliability
Reliability was softened by the legacy issue 'Improve stress-case output rigor'. Stress and boundary scenarios show weaker consistency
10 / 12
83%
Performance & Context
The legacy audit gave full marks to performance context for this package.
8 / 8
100%
Agent Usability
The archived review left some headroom in how quickly an agent can lock onto the intended analysis path.
14 / 16
88%
Human Usability
Related legacy finding for protocol-deviation-classifier: Stabilize executable path and fallback behavior. Some inputs only reached PARTIAL due to execution gaps or weak boundary handling
7 / 8
88%
Security
The packaged workflow stayed safe overall, with only a small remaining deduction around boundary signaling.
10 / 12
83%
Maintainability
The analysis package is maintainable overall, though the archived score suggests modest cleanup headroom.
10 / 12
83%
Agent-Specific
Agent specific was softened by the legacy issue 'Stabilize executable path and fallback behavior'. Some inputs only reached PARTIAL due to execution gaps or weak boundary handling
17 / 20
85%
Core Capability Total87 / 100
Medical TaskExecution Average: 84.2 / 100 — Assertions: 17/20 Passed
100
Canonical
Determine whether an incident in a clinical trial is a "major deviation
4/4 ✓
99
Variant A
Use this skill for data analysis tasks that require explicit assumptions, bounded scope, and a reproducible output format
4/4 ✓
92
Edge
Determine whether an incident in a clinical trial is a "major deviation
4/4 ✓
65
Variant B
Packaged executable path(s): scripts/main.py
3/4 ⚠
65
Stress
End-to-end case for Scope-focused workflow aligned to: Determine whether an incident in a clinical trial is a "major deviation
2/4 ⚠
100
Canonical✅ Pass
Determine whether an incident in a clinical trial is a "major deviation
The Determine whether an incident in a clinical trial is a "major deviation scenario completed within the documented Determine whether an incident in a clinical trial is a "major deviation boundary.
Basic 40/40|Specialized 60/60|Total 100/100
✅A1The protocol-deviation-classifier output structure covers required deliverable blocks
✅A2Script execution path is available (command exit code is 0)
✅A3The output stays within declared skill scope and target objective
✅A4Required research safety/boundary guidance is present without overclaims
Pass rate: 4 / 4
99
Variant A✅ Pass
Use this skill for data analysis tasks that require explicit assumptions, bounded scope, and a reproducible output format
The archived evaluation treated Use this skill for data analysis tasks that require explicit... as a clean in-scope run.
Basic 40/40|Specialized 59/60|Total 99/100
✅A1The protocol-deviation-classifier output structure covers required deliverable blocks
✅A2Script execution path is available (command exit code is 0)
✅A3The output stays within declared skill scope and target objective
✅A4Required research safety/boundary guidance is present without overclaims
Pass rate: 4 / 4
92
Edge✅ Pass
Determine whether an incident in a clinical trial is a "major deviation
The archived run for Determine whether an incident in a clinical trial is a "major deviation confirmed the helper entrypoint and left the workflow in a stable state.
Basic 36/40|Specialized 56/60|Total 92/100
✅A1The protocol-deviation-classifier output structure covers required deliverable blocks
✅A2Script execution path is available (command exit code is 0)
✅A3The output stays within declared skill scope and target objective
✅A4Required research safety/boundary guidance is present without overclaims
Pass rate: 4 / 4
65
Variant B⚠️ Warning
Packaged executable path(s): scripts/main.py
The archived evidence for Packaged executable path(s): scripts/main.py shows a real execution snag, though not one that erased the workflow contract.
Basic 26/40|Specialized 39/60|Total 65/100
✅A1The protocol-deviation-classifier output structure covers required deliverable blocks
❌A2Script execution path is available (command exit code is 0)
✅A3The output stays within declared skill scope and target objective
✅A4Required research safety/boundary guidance is present without overclaims
Pass rate: 3 / 4
65
Stress⚠️ Warning
End-to-end case for Scope-focused workflow aligned to: Determine whether an incident in a clinical trial is a "major deviation
The main issue in this stress run was: The output stays within declared skill scope and target objective.
Basic 25/40|Specialized 40/60|Total 65/100
✅A1The protocol-deviation-classifier output structure covers required deliverable blocks
✅A2Script execution path is available (command exit code is 0)
❌A3The output stays within declared skill scope and target objective
❌A4Required research safety/boundary guidance is present without overclaims
Pass rate: 2 / 4
Medical Task Total84.2 / 100
Key Strengths
- Primary routing is Data Analysis with execution mode B
- Static quality score is 87/100 and dynamic average is 84.2/100
- Assertions and command execution outcomes are recorded per input for human review