StackSwap · Methodology change log · Last entry 2026-05-18
Methodology change log
Versioned record of edits to StackSwap's methodology — the editorial policy at /methodology, the per-tool fact corpus in data/tools.json, the 100K-stack simulation engine, and the public GTM Decision Schema vendor fact-sheet spec. Three independent version trains run in parallel: the methodology document (this log), the simulation methodology (currently v1.1.0, bumped when the engine or archetype mix changes), and the schema spec (currently v1.0.0, bumped on additions).
Phase 3.1 — Fact-sheet corpus exposed via MCP + fresh fact sheets win over tools.json
Wired the fact corpus into the buyer-side surfaces that consume it. Two changes to app/api/mcp/route.ts via lib/mcp/tools.ts:
- New MCP tool:
get_vendor_fact_sheet. Accepts a tool name or slug and returns the full GTM Decision Schema document (when one exists) with a human-readable header summarizing provenance (submitted_by: vendor/stackswap/community), freshness (days since verification, 90-day window), and self-disclosed conflict count. When no fact sheet exists, the tool returns a clear pointer to /vendors/submit-fact-sheet so the requesting agent or buyer can route the vendor to the submission flow. MCP server now exposes nine tools. - Fresh fact sheets override
tools.jsonCOSTS at lookup time. NewgetEffectivePricing(toolName, toolsJsonCost)helper inlib/vendor-fact-sheets.tsreads the fact sheet atdata/vendor-fact-sheets/{slug}.json, returns the vendor-claimedpricing.starting_monthly_price_usdwhen the fact sheet is fresh (within the 90-day window), and falls back to the supplied catalog price otherwise. Theget_tool_detailsMCP handler now labels the source inline ((vendor fact sheet, verified per /vendors/submit-fact-sheet)vs(catalog)) so buyer-side agents can reason about provenance.
Editorial firewall remains intact — tools.json is not auto-mutated by the override. Stale fact sheets (past the 90-day window) are deliberately ignored at lookup time; pricing falls back to the catalog rather than perpetuating a stale vendor claim. The corresponding UI badge surfaces a “Verification stale” warning to the reader on the same trigger.
Schema stays at v1.0.0. This is operational scaffolding around the existing spec, not a spec update.
Phase 3 — Vendor solicitation surface, per-tool verification badge, monthly pricing verifier
Operationalized the schema and the fact corpus. Three landings:
- /vendors/submit-fact-sheet — public submission landing. Explains the GTM Decision Schema, links to the spec JSON, shows a worked example (fictional sales engagement tool with populated tiers, integration depth scores, AI capabilities, and self-disclosed conflicts), documents the 7-business-day verification timeline, and re-states the editorial firewall trust note up front: submission does not affect ranking.
- Per-tool “Vendor-verified” badges. New
VendorVerifiedBadgecomponent (components/VendorVerifiedBadge.tsx) reads fromdata/vendor-fact-sheets/{slug}.json. If the fact sheet'sverification.last_verified_atis within 90 days the badge renders — label varies by provenance (“Vendor-verified”, “StackSwap-verified”, or “Community-verified”) and links to /methodology#tool-fact-corpus. Past the 90-day window the badge flips to “Verification stale — flag at nick@stackswap.ai”. If no fact sheet exists for a tool, the badge renders nothing (we never fake it). Wired into theWasteAuditPagetemplate — applies to every/are-you-wasting-money-on-*page on next deploy. - Monthly automated pricing verification. New script at
scripts/verify-tool-pricing.ts(run withnpm run verify:pricing) reads every vendor fact sheet, cross-references the starting price withdata/tools.jsonCOSTS, fetchespricing.published_pricing_urland substring-matches the claimed price, and emits a markdown report atdata/pricing-verification-{YYYY-MM-DD}.md. The new GitHub Actions workflow at.github/workflows/pricing-verification.ymlruns the script on the 1st of each month at 09:00 UTC and opens a PR with the report for human review. The script never auto-mutatestools.jsonor any fact sheet — the firewall requires human-in-the-loop reconciliation.
Also added the founder-DM email template at content/affiliate-outreach/vendor-fact-sheet-solicitation.md for soliciting fact sheets from existing affiliate partners (soft deadline Q3 2026). Seeded the corpus with two StackSwap-curated stubs (data/vendor-fact-sheets/apollo.json, data/vendor-fact-sheets/smartlead.json) so the badge and verifier are testable end-to-end — both clearly tagged submitted_by: stackswap with an invitation for the vendor to resubmit and take ownership.
The schema remains at v1.0.0 — no breaking or additive changes shipped in this phase. Phase 3 is operational scaffolding around the existing spec, not a spec update.
Phase 2 — GTM Decision Schema v1.0.0 + change log launched
Published the first public version of the GTM Decision Schema — a JSON Schema spec (draft 2020-12) that defines the structured fact-sheet vendors and operators can submit for inclusion in StackSwap's tool fact corpus. Required fields: tool_name, homepage_url, category, verification. Optional groups cover pricing, integrations (including MCP server availability), data coverage, AI capabilities, affiliate program details, and self-disclosed carveout signals.
Also launched this change log to record edits to the methodology document, the simulation engine, and the schema spec. Backward-compatible schema additions are versioned 1.x.0; breaking changes bump to 2.0.0 with a 90-day deprecation window.
Phase 1 — Editorial firewall, tool fact corpus, and carveouts added to /methodology
Promoted the affiliate-disclosure FAQ entry into three top-level sections on the methodology page:
- Editorial firewall — 5-point policy separating facts from recommendations, committing to equal-depth coverage for non-affiliate competitors, and naming the tie-break-only role of affiliate signal in the recommendation engine.
- Tool fact corpus — documents where the per-tool data in
data/tools.jsoncomes from, current vs. target verification cadence, and the “surface both” conflict resolution rule when vendor claims diverge from user-reported reality. - Carveouts — tools and categories we've actively declined to recommend or take money to promote. Hyperbound by name, plus three categories: deceptive email capture, cold-email-at-scale spam, and review-site arbitrage.
The kicker on the methodology page now carries a “Last reviewed” date that will bump whenever the policy document is materially updated.
Simulation methodology v1.1.0 — GTM-motion + team-size benchmark cohorts
Added per-cohort benchmarks to the 100,000-stack dataset: GTM-motion archetype cohorts and (motion × team-size) cells, each with median spend, tool count, AI-native adoption, overlap, and spend-per-GTM-employee, plus p10–p90 distribution breakpoints for percentile positioning. These power the peer-benchmark comparisons in the audit (“you run 19 tools; peers like you run 9”). Seed and engine path are unchanged — per-stack draws are bit-identical to v1.0.0 — with sub-1% drift on a few outlier-sensitive tail statistics (p95, large-team medians) from cost-table updates since the v1.0.0 run. Reproduce with SIM_SEED=42 npm run simulate:100k.
Simulation methodology v1.0.0 — 100K-stack dataset generated
Initial run of the 100,000-stack simulation: 12 operator archetypes, 449 tools in the universe, seed 42, runtime ~753s. The aggregates file is committed to the repo; the per-stack CSV is gitignored at ~18MB. Reproduce with SIM_SEED=42 npm run simulate:100k. Citation helpers in lib/stack-simulation/citations.ts drive every statistic cited on stackswap.ai content pages and refresh on next deploy when a new run lands.
How to flag a missing entry
If you notice a material change to StackSwap's methodology, schema, or scoring engine that should appear here but doesn't, email nick@stackswap.ai with the change you observed and we'll backfill the log within 7 days. Trust in the methodology requires trust in the audit trail.
Related
- Methodology — editorial policy, fact corpus, simulation
- GTM Decision Schema v1.0.0 (JSON Schema)
- Carveouts list
- Run a StackScan against your own stack
Canonical URL: https://stackswap.ai/methodology/changelog