Data Analysis
meta-screening-fulltext
85100Total Score
Core Capability
77 / 100
Functional Suitability
9 / 12
Reliability
9 / 12
Performance & Context
8 / 8
Agent Usability
12 / 16
Human Usability
7 / 8
Security
8 / 12
Maintainability
9 / 12
Agent-Specific
15 / 20
Medical Task
20 / 20 Passed
95Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
4/4
91Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
4/4
89Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID
4/4
89Packaged executable path(s): scripts/extract_pdf.py
4/4
89End-to-end case for Scope-focused workflow aligned to: Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
4/4
Veto GatesRequired pass for any deployment consideration
Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASSResearch Veto✅ PASS — Applicable
| Dimension | Result | Detail |
|---|---|---|
| Scientific Integrity | PASS | No scientific-integrity problem was surfaced because the package did not claim more than the available records, article text, or script evidence supported. |
| Practice Boundaries | PASS | The evaluated outputs stayed inside the Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata... and did not drift into unsupported interpretation beyond the available inputs. |
| Methodological Ground | PASS | The legacy review kept the package aligned with its named analysis library, data structure, or processing workflow. |
| Code Usability | PASS | Code usability passed because the package still exposed a reviewable execution surface for its documented workflow. |
Core Capability77 / 100 — 8 Categories
Functional Suitability
Related legacy finding for meta-screening-fulltext: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
9 / 12
75%
Reliability
The archived deduction in reliability traces back to: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
9 / 12
75%
Performance & Context
The legacy audit gave full marks to performance context for this package.
8 / 8
100%
Agent Usability
The packaged analysis path is understandable, though the archived score suggests slightly clearer routing would help.
12 / 16
75%
Human Usability
The package is readable overall, though the archived review still left a small human-usability gap.
7 / 8
88%
Security
The packaged workflow stayed safe overall, with only a small remaining deduction around boundary signaling.
8 / 12
67%
Maintainability
Maintainability stayed solid, with only limited room to simplify scripts, dependencies, or packaging structure.
9 / 12
75%
Agent-Specific
The archived deduction in agent specific traces back to: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
15 / 20
75%
Core Capability Total77 / 100
Medical TaskExecution Average: 90.6 / 100 — Assertions: 20/20 Passed
95
Canonical
Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
4/4 ✓
91
Variant A
Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
4/4 ✓
89
Edge
Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID
4/4 ✓
89
Variant B
Packaged executable path(s): scripts/extract_pdf.py
4/4 ✓
89
Stress
End-to-end case for Scope-focused workflow aligned to: Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
4/4 ✓
95
Canonical✅ Pass
Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
This canonical case stayed focused on extracting and normalizing evidence from the provided records instead of drifting into unsupported interpretation.
Basic 35/40|Specialized 60/60|Total 95/100
✅A1The meta-screening-fulltext output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
91
Variant A✅ Pass
Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
Screen full-text papers against inclusion/exclusion criteria, with... remained an analysis-style extraction path whose value came from structured data capture rather than a freeform narrative response.
Basic 33/40|Specialized 58/60|Total 91/100
✅A1The meta-screening-fulltext output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
89
Edge✅ Pass
Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID
This edge case stayed focused on extracting and normalizing evidence from the provided records instead of drifting into unsupported interpretation.
Basic 32/40|Specialized 57/60|Total 89/100
✅A1The meta-screening-fulltext output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
89
Variant B✅ Pass
Packaged executable path(s): scripts/extract_pdf.py
This variant b case stayed focused on extracting and normalizing evidence from the provided records instead of drifting into unsupported interpretation.
Basic 31/40|Specialized 58/60|Total 89/100
✅A1The meta-screening-fulltext output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
89
Stress✅ Pass
End-to-end case for Scope-focused workflow aligned to: Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata check using PMID. Use when the user needs to evaluate a paper for a meta-analysis
The archived run treated Screen full-text papers against inclusion/exclusion criteria, with optional PubMed metadata... as a bounded extraction workflow, keeping attention on source fields, fallback logic, and output shape.
Basic 28/40|Specialized 60/60|Total 89/100
✅A1The meta-screening-fulltext output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
Medical Task Total90.6 / 100
Key Strengths
- Primary routing is Data Analysis with execution mode B
- Static quality score is 77/100 and dynamic average is 77.6/100
- Assertions and command execution outcomes are recorded per input for human review
- Execution verification summary: Script verification 1/1; adjustment=5. extract_pdf.py: OK