Data Analysis

cohort-study-quality-assessment-nos

87100Total Score
Core Capability
88 / 100
Functional Suitability
11 / 12
Reliability
10 / 12
Performance & Context
8 / 8
Agent Usability
14 / 16
Human Usability
8 / 8
Security
10 / 12
Maintainability
10 / 12
Agent-Specific
17 / 20
Medical Task
15 / 20 Passed
86Evaluates the quality of cohort studies using the Newcastle-Ottawa Scale (NOS)
3/4
86Step 2: Analyze the Text
3/4
86Step 2: Analyze the Text
3/4
86Step 4: Calculate Score and Generate Report
3/4
86Step 5: Generate Final Report
3/4

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS
Research Veto✅ PASS — Applicable
DimensionResultDetail
Scientific IntegrityPASSThe archived review kept this workflow anchored to supplied data fields and observable execution behavior, not fabricated results.
Practice BoundariesPASSPractice boundaries held because the package remained focused on Evaluates the quality of cohort studies using the Newcastle-Ottawa Scale (NOS) rather than overclaiming what the records supported.
Methodological GroundPASSThe workflow stayed grounded in its declared rubric or scale-selection logic rather than improvised criteria.
Code UsabilityPASSThe archived review found the packaged execution path for cohort-study-quality-assessment-nos usable in its intended context.

Core Capability88 / 100 — 8 Categories

Functional Suitability
Functional suitability was softened by the legacy issue 'Improve stress-case output rigor'. Stress and boundary scenarios show weaker consistency
11 / 12
92%
Reliability
Related legacy finding for cohort-study-quality-assessment-nos: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
10 / 12
83%
Performance & Context
No point loss was recorded for performance context in the legacy audit.
8 / 8
100%
Agent Usability
Agent usability was strong, but the workflow could surface its entry conditions a little more directly.
14 / 16
88%
Human Usability
No point loss was recorded for human usability in the legacy audit.
8 / 8
100%
Security
Security remained strong, though the archived review still left some room for clearer execution guardrails.
10 / 12
83%
Maintainability
Maintainability stayed solid, with only limited room to simplify scripts, dependencies, or packaging structure.
10 / 12
83%
Agent-Specific
The archived deduction in agent specific traces back to: Stabilize executable path and fallback behavior. Some inputs only reached PARTIAL due to execution gaps or weak boundary handling
17 / 20
85%
Core Capability Total88 / 100

Medical TaskExecution Average: 86 / 100 — Assertions: 15/20 Passed

86
Canonical
Evaluates the quality of cohort studies using the Newcastle-Ottawa Scale (NOS)
3/4 ✓
86
Variant A
Step 2: Analyze the Text
3/4 ✓
86
Edge
Step 2: Analyze the Text
3/4 ✓
86
Variant B
Step 4: Calculate Score and Generate Report
3/4 ✓
86
Stress
Step 5: Generate Final Report
3/4 ✓
86
Canonical✅ Pass
Evaluates the quality of cohort studies using the Newcastle-Ottawa Scale (NOS)

The archived execution for Evaluates the quality of cohort studies using the Newcastle-Ottawa... failed for environmental reasons rather than workflow ambiguity: a required file was missing.

Basic 33/40|Specialized 53/60|Total 86/100
✅A1The cohort-study-quality-assessment-nos output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Variant A✅ Pass
Step 2: Analyze the Text

The Step 2: Analyze the Text workflow is defined, but this run was blocked by a missing local input file.

Basic 31/40|Specialized 55/60|Total 86/100
✅A1The cohort-study-quality-assessment-nos output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Edge✅ Pass
Step 2: Analyze the Text

Step 2: Analyze the Text stayed well-scoped, but the local run could not proceed because the expected input file was absent.

Basic 30/40|Specialized 56/60|Total 86/100
✅A1The cohort-study-quality-assessment-nos output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Variant B✅ Pass
Step 4: Calculate Score and Generate Report

The Step 4: Calculate Score and Generate Report workflow is defined, but this run was blocked by a missing local input file.

Basic 29/40|Specialized 57/60|Total 86/100
✅A1The cohort-study-quality-assessment-nos output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
86
Stress✅ Pass
Step 5: Generate Final Report

Step 5: Generate Final Report stayed well-scoped, but the local run could not proceed because the expected input file was absent.

Basic 26/40|Specialized 60/60|Total 86/100
✅A1The cohort-study-quality-assessment-nos output structure matches the documented deliverable
❌A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 3 / 4
Medical Task Total86 / 100

Key Strengths

  • Primary routing is Data Analysis with execution mode B
  • Static quality score is 88/100 and dynamic average is 72.6/100
  • Assertions and command execution outcomes are recorded per input for human review
  • Execution verification summary: Script verification 0/2; adjustment=0. calculate_nos_score.py: rc=1; extract_pdf.py: rc=1