io.github.JcJamet/ia-qa-toolbox

What we checked

Six static checks, weighted by risk. Every result reflects only what could be observed in the published package and repository — never intent.

• Injection surface

  fail25% of grade

  Tool descriptions/manifest scanned for instruction-injection patterns (imperatives at the model, hidden text, 'ignore previous', data-exfil URLs).

  tool "analyze_responses" description: text addressed to the model rather than describing a capability. tool "prompt_injection_scan" description: names an outbound copy/exfil destination (BCC/exfil verb). tool "generate_ci_workflow" param "app_base_url" description: raw-IP URL in a description (http://127.0.0.1:3000) — strong exfil tell. tool "mcp_server_evaluate" param "url" description: off-vendor URL in a description (ia-qa.com); capability text should not name where data goes. tool "mcp_server_evaluate" param "url" description: off-vendor URL in a description (localhost); capability text should not name where data goes. tool "security_headers_check" param "url" description: off-vendor URL in a description (example.com); capability text should not name where data goes. tool "cors_test" param "url" description: off-vendor URL in a description (api.example.com); capability text should not name where data goes. tool "cors_checker" param "url" description: off-vendor URL in a description (api.example.com); capability text should not name where data goes. tool "cors_checker" param "origin" description: off-vendor URL in a description (yourdomain.com); capability text should not name where data goes. tool "webhook_endpoint_create" param "base_url" description: off-vendor URL in a description (www.ia-qa.com); capability text should not name where data goes. tool "cookie_security_audit" param "url" description: off-vendor URL in a description (example.com); capability text should not name where data goes. tool "web_security_audit" param "url" description: off-vendor URL in a description (example.com); capability text should not name where data goes. tool "sandbox_scenario" description: off-vendor URL in a description (www.ia-qa.com); capability text should not name where data goes. tool "fetch_jira_issue" param "jira_base_url" description: off-vendor URL in a description (mycompany.atlassian.net); capability text should not name where data goes. tool "search_jira_issues" param "jira_base_url" description: off-vendor URL in a description (mycompany.atlassian.net); capability text should not name where data goes. tool "fetch_confluence_page" param "page_url" description: off-vendor URL in a description (mycompany.atlassian.net); capability text should not name where data goes. tool "fetch_confluence_page" param "confluence_base_url" description: off-vendor URL in a description (mycompany.atlassian.net); capability text should not name where data goes.

• Supply chain

  pass25% of grade

  Package provenance: namespace verification, repo linkage, maintainer count, account age, postinstall scripts, typosquat distance.

  repository linkage verified

• Credential hygiene

  fail15% of grade

  How the server takes secrets (env vs plaintext config vs hardcoded); secrets appearing in tool schemas.

  tool "compare_responses" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "generate_hmac" has a secret-shaped input parameter "secret" — secrets must never be tool parameters (the model would supply/handle the credential). tool "rerank_evaluate" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "web_security_audit" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "run_eval_contract" has a secret-shaped input parameter "api_keys" — secrets must never be tool parameters (the model would supply/handle the credential). tool "run_vlm_test_suite" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "multimodal_eval_guide" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "run_vlm_test_suite_batch" has a secret-shaped input parameter "api_keys" — secrets must never be tool parameters (the model would supply/handle the credential). tool "run_semantic_tests" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "metamorphic_check" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "test_skill" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "post_jira_comment" has a secret-shaped input parameter "jira_token" — secrets must never be tool parameters (the model would supply/handle the credential). tool "create_confluence_page" has a secret-shaped input parameter "confluence_token" — secrets must never be tool parameters (the model would supply/handle the credential). tool "fetch_jira_issue" has a secret-shaped input parameter "jira_token" — secrets must never be tool parameters (the model would supply/handle the credential). tool "search_jira_issues" has a secret-shaped input parameter "jira_token" — secrets must never be tool parameters (the model would supply/handle the credential). tool "jira_to_test_suite" has a secret-shaped input parameter "jira_token" — secrets must never be tool parameters (the model would supply/handle the credential). tool "jira_to_test_suite" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "fix_gherkin" has a secret-shaped input parameter "api_key" — secrets must never be tool parameters (the model would supply/handle the credential). tool "fetch_confluence_page" has a secret-shaped input parameter "confluence_token" — secrets must never be tool parameters (the model would supply/handle the credential).

• Permission scope

  unscannable15% of grade

  Declared tools vs. breadth (filesystem, network, exec); flags shell-exec and unbounded filesystem access.

  No source files, package.json, or pyproject were fetched; cannot infer capability scope.

• Version behavior

  unscannable10% of grade

  Diff of tool definitions between versions; new permissions or changed descriptions in a patch release (the postmark-mcp class).

  no prior version to diff (first sight of io.github.JcJamet/[email protected]); version-drift / rug-pull cannot be evaluated. Cold-start risk is covered by point-in-time checks (injection_surface, supply_chain, credential_hygiene), not here.

• Transport config

  fail10% of grade

  Remote servers: TLS and auth mode (none/token/OAuth). Local servers: whether the manifest indicates it phones home.

  remote MCP endpoint declares NO authentication (unauthenticated remote — nginx-ui/CVE-2026-33032 class)