Evidence Insight

arxiv-database

89100Total Score
Core Capability
78 / 100
Functional Suitability
9 / 12
Reliability
9 / 12
Performance & Context
8 / 8
Agent Usability
12 / 16
Human Usability
7 / 8
Security
9 / 12
Maintainability
9 / 12
Agent-Specific
15 / 20
Medical Task
20 / 20 Passed
100Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4
96Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4
94Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4
94Packaged executable path(s): scripts/arxiv_search.py
4/4
94End-to-end case for Scope-focused workflow aligned to: Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS
Research Veto✅ PASS — Applicable
DimensionResultDetail
Scientific IntegrityPASSThe archived evaluation kept the skill tied to retrieved records or indexed source material rather than invented scientific claims.
Practice BoundariesPASSPractice boundaries held because the package remained focused on source handling, lookup, or structured evidence use.
Methodological GroundPASSThe older review treated the package logic as methodologically aligned with its stated workflow.
Code UsabilityPASSCode usability passed because the search or lookup workflow still exposed a usable entrypoint and output expectation.

Core Capability78 / 100 — 8 Categories

Functional Suitability
A modest deduction remained in functional suitability for arxiv-database in the archived review.
9 / 12
75%
Reliability
The legacy audit deducted points for arxiv-database in reliability.
9 / 12
75%
Performance & Context
The legacy audit gave full marks to performance context for this package.
8 / 8
100%
Agent Usability
The archived evaluation left some headroom for arxiv-database under agent usability.
12 / 16
75%
Human Usability
The archived deduction in human usability traces back to: Minor polish before wide rollout. No major defects found
7 / 8
88%
Security
The legacy audit deducted points for arxiv-database in security.
9 / 12
75%
Maintainability
A modest deduction remained in maintainability for arxiv-database in the archived review.
9 / 12
75%
Agent-Specific
The archived evaluation left some headroom for arxiv-database under agent specific.
15 / 20
75%
Core Capability Total78 / 100

Medical TaskExecution Average: 95.6 / 100 — Assertions: 20/20 Passed

100
Canonical
Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4 ✓
96
Variant A
Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4 ✓
94
Edge
Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4 ✓
94
Variant B
Packaged executable path(s): scripts/arxiv_search.py
4/4 ✓
94
Stress
End-to-end case for Scope-focused workflow aligned to: Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading
4/4 ✓
100
Canonical✅ Pass
Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading

The Search and retrieve scientific preprints from arXiv; use it when you need to find papers by... scenario completed within the documented Search and retrieve scientific preprints from arXiv; use it when you need to find papers by... boundary.

Basic 37/40|Specialized 60/60|Total 100/100
✅A1The arxiv-database output structure matches the documented deliverable
✅A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
96
Variant A✅ Pass
Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading

The archived evaluation treated Search and retrieve scientific preprints from arXiv; use it when you need to find papers by... as a clean in-scope run.

Basic 35/40|Specialized 60/60|Total 96/100
✅A1The arxiv-database output structure matches the documented deliverable
✅A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
94
Edge✅ Pass
Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading

The Search and retrieve scientific preprints from arXiv; use it when you need to find papers by... scenario completed within the documented Search and retrieve scientific preprints from arXiv; use it when you need to find papers by... boundary.

Basic 34/40|Specialized 60/60|Total 94/100
✅A1The arxiv-database output structure matches the documented deliverable
✅A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
94
Variant B✅ Pass
Packaged executable path(s): scripts/arxiv_search.py

The archived evaluation treated Packaged executable path(s): scripts/arxiv_search.py as a clean in-scope run.

Basic 33/40|Specialized 60/60|Total 94/100
✅A1The arxiv-database output structure matches the documented deliverable
✅A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
94
Stress✅ Pass
End-to-end case for Scope-focused workflow aligned to: Search and retrieve scientific preprints from arXiv; use it when you need to find papers by keyword/author/category, fetch metadata (abstract, DOI, PDF URL), or download PDFs for offline reading

The archived evaluation treated Search and retrieve scientific preprints from arXiv; use it when you need to find papers by... as a clean in-scope run.

Basic 30/40|Specialized 60/60|Total 94/100
✅A1The arxiv-database output structure matches the documented deliverable
✅A2The script execution path completed successfully for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
Medical Task Total95.6 / 100

Key Strengths

  • Primary routing is Evidence Insight with execution mode B
  • Static quality score is 78/100 and dynamic average is 82.6/100
  • Assertions and command execution outcomes are recorded per input for human review
  • Execution verification summary: Script verification 1/1; adjustment=5. arxiv_search.py: OK