Evidence Insight

pytdc

87100Total Score
Core Capability
87 / 100
Functional Suitability
11 / 12
Reliability
10 / 12
Performance & Context
8 / 8
Agent Usability
14 / 16
Human Usability
8 / 8
Security
9 / 12
Maintainability
10 / 12
Agent-Specific
17 / 20
Medical Task
20 / 20 Passed
92You need curated, AI-ready datasets for drug discovery tasks (e.g., ADME, toxicity, bioactivity, DTI/DDI)
4/4
88You want standardized benchmarks with consistent evaluation protocols (including multi-seed benchmark groups)
4/4
86Dataset access by task type
4/4
86Single-instance prediction: ADME, Toxicity (Tox), HTS, QM, and more
4/4
86End-to-end case for Dataset access by task type
4/4

Veto GatesRequired pass for any deployment consideration

Skill Veto✓ All 4 gates passed
✓
Operational Stability
System remains stable across varied inputs and edge cases
PASS
✓
Structural Consistency
Output structure conforms to expected skill contract format
PASS
✓
Result Determinism
Equivalent inputs produce semantically equivalent outputs
PASS
✓
System Security
No prompt injection, data leakage, or unsafe tool use detected
PASS
Research Veto✅ PASS — Applicable
DimensionResultDetail
Scientific IntegrityPASSThe archived evaluation kept the skill tied to retrieved records or indexed source material rather than invented scientific claims.
Practice BoundariesPASSThe legacy review kept this workflow on the evidence-access side of the boundary, not the advice-giving side.
Methodological GroundPASSThe legacy audit preserved a method-grounded interpretation of the Therapeutics Data Commons (PyTDC) for AI-ready therapeutic ML datasets and benchmarks; use it when you need standardized dataset loading, meaningful splits (e.g., scaffold/cold-start), and consistent evaluation for ADME/Toxicity/DTI/DDI or molecular optimization workflow.
Code UsabilityPASSCode usability passed because the search or lookup workflow still exposed a usable entrypoint and output expectation.

Core Capability87 / 100 — 8 Categories

Functional Suitability
Related legacy finding for pytdc: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
11 / 12
92%
Reliability
Reliability was softened by the legacy issue 'Improve stress-case output rigor'. Stress and boundary scenarios show weaker consistency
10 / 12
83%
Performance & Context
Performance context reached full score in the archived evaluation.
8 / 8
100%
Agent Usability
The legacy audit deducted points for pytdc in agent usability.
14 / 16
88%
Human Usability
Human usability reached full score in the archived evaluation.
8 / 8
100%
Security
The legacy audit deducted points for pytdc in security.
9 / 12
75%
Maintainability
The legacy audit deducted points for pytdc in maintainability.
10 / 12
83%
Agent-Specific
Related legacy finding for pytdc: Improve stress-case output rigor. Stress and boundary scenarios show weaker consistency
17 / 20
85%
Core Capability Total87 / 100

Medical TaskExecution Average: 87.6 / 100 — Assertions: 20/20 Passed

92
Canonical
You need curated, AI-ready datasets for drug discovery tasks (e.g., ADME, toxicity, bioactivity, DTI/DDI)
4/4 ✓
88
Variant A
You want standardized benchmarks with consistent evaluation protocols (including multi-seed benchmark groups)
4/4 ✓
86
Edge
Dataset access by task type
4/4 ✓
86
Variant B
Single-instance prediction: ADME, Toxicity (Tox), HTS, QM, and more
4/4 ✓
86
Stress
End-to-end case for Dataset access by task type
4/4 ✓
92
Canonical✅ Pass
You need curated, AI-ready datasets for drug discovery tasks (e.g., ADME, toxicity, bioactivity, DTI/DDI)

You need curated, AI-ready datasets for drug discovery tasks (e.g.,... was evaluated as a bounded documentation path, not as a runnable script workflow.

Basic 36/40|Specialized 56/60|Total 92/100
✅A1The pytdc output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
88
Variant A✅ Pass
You want standardized benchmarks with consistent evaluation protocols (including multi-seed benchmark groups)

The archived run for You want standardized benchmarks with consistent evaluation... remained guidance-driven rather than command-driven.

Basic 34/40|Specialized 54/60|Total 88/100
✅A1The pytdc output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
86
Edge✅ Pass
Dataset access by task type

This edge case stayed inside the documented workflow and remained instruction-led.

Basic 33/40|Specialized 53/60|Total 86/100
✅A1The pytdc output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
86
Variant B✅ Pass
Single-instance prediction: ADME, Toxicity (Tox), HTS, QM, and more

Single-instance prediction: ADME, Toxicity (Tox), HTS, QM, and more was evaluated as a bounded documentation path, not as a runnable script workflow.

Basic 32/40|Specialized 54/60|Total 86/100
✅A1The pytdc output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
86
Stress✅ Pass
End-to-end case for Dataset access by task type

The archived run for End-to-end case for Dataset access by task type remained guidance-driven rather than command-driven.

Basic 29/40|Specialized 57/60|Total 86/100
✅A1The pytdc output structure matches the documented deliverable
✅A2The instruction path remains actionable for the documented case
✅A3The output stays fully within the documented skill boundary
✅A4The response quality is acceptable for the documented path
Pass rate: 4 / 4
Medical Task Total87.6 / 100

Key Strengths

  • Primary routing is Evidence Insight with execution mode B
  • Static quality score is 87/100 and dynamic average is 79.6/100
  • Assertions and command execution outcomes are recorded per input for human review
  • Execution verification summary: Script verification 0/3; adjustment=0. benchmark_evaluation.py: rc=1; load_and_split_data.py: rc=1; molecular_generation.py: rc=1