io.github.IgorGanapolsky/rlhf-feedback-loop

What we checked

Six static checks, weighted by risk. Every result reflects only what could be observed in the published package and repository — never intent.

• Injection surface

  unscannable25% of grade

  Tool descriptions/manifest scanned for instruction-injection patterns (imperatives at the model, hidden text, 'ignore previous', data-exfil URLs).

  No tool descriptions, server instructions, prompts, or resources were fetched; nothing to scan for injection.

• Supply chain

  warn25% of grade

  Package provenance: namespace verification, repo linkage, maintainer count, account age, postinstall scripts, typosquat distance.

  dependency "@google/genai" is edit-distance 2 from popular package "openai" (typosquat candidate)

• Credential hygiene

  warn15% of grade

  How the server takes secrets (env vs plaintext config vs hardcoded); secrets appearing in tool schemas.

  .env.example: contains a hardcoded GitHub fine-grained PAT (gith…nt (len 47)) — in an example/sample file; verify it is not a real key. Reads secrets from the environment (e.g. scripts/adk-consolidator.js) — the recommended intake shape.

• Permission scope

  fail15% of grade

  Declared tools vs. breadth (filesystem, network, exec); flags shell-exec and unbounded filesystem access.

  HIGH: shell/exec capability AND network egress together — download-and-execute risk (bin/cli.js: imports child_process and calls exec/spawn; scripts/prove-adapters.js: outbound HTTP/socket call). HIGH: shell/exec capability AND destructive filesystem writes together (bin/cli.js: imports child_process and calls exec/spawn; scripts/adk-consolidator.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename)). discloses shell/exec capability: bin/cli.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/pr-manager.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-adapters.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-attribution.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-data-quality.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-intelligence.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-lancedb.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-local-intelligence.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-loop-closure.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-rlaif.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-subway-upgrades.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-training-export.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-v2-milestone.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/prove-v3-milestone.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/self-heal.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/self-healing-check.js: imports child_process and calls exec/spawn discloses shell/exec capability: scripts/test-coverage.js: imports child_process and calls exec/spawn discloses broad/destructive filesystem access: scripts/adk-consolidator.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/billing.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/billing.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/budget-guard.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: bin/cli.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/context-engine.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/contextfs.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/contract-audit.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/dpo-optimizer.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/export-dpo-pairs.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/export-kto-pairs.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/export-training.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/feedback-inbox-read.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/feedback-ingest-watcher.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/feedback-loop.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/hybrid-feedback-context.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/install-mcp.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/jsonl-watcher.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/local-model-profile.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/meta-policy.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-adapters.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-adapters.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/prove-attribution.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-automation.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-automation.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/prove-data-quality.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-intelligence.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-intelligence.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/prove-lancedb.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-local-intelligence.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-loop-closure.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-rlaif.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-subway-upgrades.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-training-export.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-training-export.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/prove-v2-milestone.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-v3-milestone.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/prove-workflow-contract.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/reminder-engine.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/risk-scorer.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: adapters/mcp/server-stdio.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: src/api/server.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: src/api/server.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/sync-version.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/thompson-sampling.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/validate-feedback.js: destructive/writing fs.* call (write/unlink/rm/chmod/rename) discloses broad/destructive filesystem access: scripts/validate-feedback.js: path traversal ('../') combined with file access discloses broad/destructive filesystem access: scripts/train_from_feedback.py: open(…, 'w'/'a'/'x') write-mode file access discloses broad/destructive filesystem access: scripts/hook-auto-capture.sh: path traversal ('../') combined with file access Capability DISCLOSURE, not a verdict: static analysis sees the primitive is present and reachable, not whether its use is attacker-controlled.

• Version behavior

  unscannable10% of grade

  Diff of tool definitions between versions; new permissions or changed descriptions in a patch release (the postmark-mcp class).

  no prior version to diff (first sight of io.github.IgorGanapolsky/[email protected]); version-drift / rug-pull cannot be evaluated. Cold-start risk is covered by point-in-time checks (injection_surface, supply_chain, credential_hygiene), not here.

• Transport config

  fail10% of grade

  Remote servers: TLS and auth mode (none/token/OAuth). Local servers: whether the manifest indicates it phones home.

  source performs runtime self-update / download-and-execute (bypasses version_behavior); manifest/source references analytics behavior — disclosure