io.github.hidai25/evalview-mcp

What we checked

Six static checks, weighted by risk. Every result reflects only what could be observed in the published package and repository — never intent.

• Injection surface

  unscannable25% of grade

  Tool descriptions/manifest scanned for instruction-injection patterns (imperatives at the model, hidden text, 'ignore previous', data-exfil URLs).

  No tool descriptions, server instructions, prompts, or resources were fetched; nothing to scan for injection.

• Supply chain

  fail25% of grade

  Package provenance: namespace verification, repo linkage, maintainer count, account age, postinstall scripts, typosquat distance.

  Python build script executes at build time: network call in build script; dependency "yaml" is edit-distance 2 from top-20 package "pyyaml" (typosquat candidate)

• Credential hygiene

  pass15% of grade

  How the server takes secrets (env vs plaintext config vs hardcoded); secrets appearing in tool schemas.

  Reads secrets from the environment (e.g. evalview/adapters/huggingface_adapter.py) — the recommended intake shape. PASS = no static red flag; static analysis cannot prove the code honors env-based secret handling at runtime.

• Permission scope

  fail15% of grade

  Declared tools vs. breadth (filesystem, network, exec); flags shell-exec and unbounded filesystem access.

  HIGH: shell/exec capability AND network egress together — download-and-execute risk (evalview/adapters/goose_adapter.py: subprocess.* call; evalview/adapters/crewai_adapter.py: outbound HTTP/socket call). HIGH: shell/exec capability AND destructive filesystem writes together (evalview/adapters/goose_adapter.py: subprocess.* call; evalview/adapters/openai_assistants_adapter.py: open(…, 'w'/'a'/'x') write-mode file access). discloses shell/exec capability: evalview/adapters/goose_adapter.py: subprocess.* call discloses shell/exec capability: evalview/chat.py: subprocess.* call with shell=True (command-injection prone) discloses shell/exec capability: evalview/ci/comment.py: subprocess.* call discloses shell/exec capability: evalview/cli.py: subprocess.* call discloses shell/exec capability: evalview/mcp_server.py: subprocess.* call discloses shell/exec capability: evalview/skills/adapters/base.py: subprocess.* call discloses shell/exec capability: evalview/skills/adapters/custom_adapter.py: subprocess.* call discloses shell/exec capability: evalview/skills/evaluators/deterministic.py: subprocess.* call with shell=True (command-injection prone) discloses shell/exec capability: evalview/trace_cmd/runner.py: subprocess.* call discloses shell/exec capability: evalview/tracking/regression.py: subprocess.* call discloses shell/exec capability: tests/test_dogfood_e2e.py: subprocess.* call discloses broad/destructive filesystem access: evalview/adapters/openai_assistants_adapter.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/chat.py: path traversal ('../') combined with file access discloses broad/destructive filesystem access: evalview/ci/comment.py: path traversal ('../') combined with file access discloses broad/destructive filesystem access: evalview/cli.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/cli.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: evalview/cli.py: path traversal ('../') combined with file access discloses broad/destructive filesystem access: evalview/core/golden.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/core/llm_provider.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/core/mcp_contract.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/core/project_state.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/expander.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/mcp_server.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/mcp_server.py: path traversal ('../') combined with file access discloses broad/destructive filesystem access: evalview/recorder.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/reporters/json_reporter.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/skills/agent_runner.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: evalview/skills/agent_runner.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: evalview/skills/test_generator.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: evalview/trace_cmd/runner.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: tests/test_diff_engine.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: tests/test_e2e_snapshot_check.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: tests/test_golden_store.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: tests/test_judge_cache.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) discloses broad/destructive filesystem access: tests/test_project_state.py: destructive filesystem call (shutil.rmtree/os.remove/etc.) Capability DISCLOSURE, not a verdict: static analysis sees the primitive is present and reachable, not whether its use is attacker-controlled.

• Version behavior

  unscannable10% of grade

  Diff of tool definitions between versions; new permissions or changed descriptions in a patch release (the postmark-mcp class).

  no prior version to diff (first sight of io.github.hidai25/[email protected]); version-drift / rug-pull cannot be evaluated. Cold-start risk is covered by point-in-time checks (injection_surface, supply_chain, credential_hygiene), not here.

• Transport config

  warn10% of grade

  Remote servers: TLS and auth mode (none/token/OAuth). Local servers: whether the manifest indicates it phones home.

  sends telemetry to third-party SDK (posthog) — disclosure, verify opt-in; manifest/source references Telemetry behavior — disclosure