Capability 02

Test vendors against your workloads, not their demo.

AI platforms, SaaS tools and cloud security vendors, run side by side on what you actually run. Scored on what it costs over three years, and on whether the vendor is still there at renewal.

Best fit: choosing between providers

Downloads

take it to the meeting

Service sheets follow one standard across all six capabilities. Engagement status on each sheet is taken from the live catalogue.

The problem

what goes wrong

A demo is built to succeed. Clean data, a happy path, and a sales engineer who knows every workaround. Your environment has none of those. That is why software so often underdelivers the week it meets real conditions.

There is also a survival problem. The 2026 Marketing Technology Landscape added 1,488 products over the year and retired 1,367 — about 124 in and 114 out every month. Nearly half the products removed were earning $1M–$10M. Picking a vendor is partly a bet that it still exists at renewal. Source: chiefmartec / MartechTribe ↗

What we do

four pieces

Requirements that bind

Criteria written from your workloads and weighted before any vendor is contacted. A demo doesn't get to set its own test.

Hands-on bake-off

The shortlist runs the same tasks, on the same data, through the same failure cases.

Three-year TCO

Licensing, egress, integration and the staffing cost of running it. Handed over as a model you can re-run.

Viability read

Funding, customer concentration and acquisition risk. We also check how revenue is reported: some AI vendors book cloud-channel revenue gross, which flatters the headline (a worked example).

Tools & resources

how this service uses them

Free tool AI Roadmap Builder Every use case marked "we would buy a product for it" becomes a shortlist brief. Each one gets a bake-off on your own data. Build your roadmap → Free · Mondays The Weekly Brief Funding rounds, acquisitions and pricing changes, ranked every Monday. Viability starts here. Open → Research The enterprise AI stack The eight criteria we score vendors on, layer by layer. Open →

Downloads for this service are above. All free tools run in your browser and need no email.

In the news4 stories

5 Dec 2025The Register
Microsoft 365 commercial prices rise from July 2026 to cover added AI and security
Why it matters here. Prices moved mid-contract. A three-year cost model needs room for the vendor’s next increase.
14 Jul 2025TechCrunch
Cognition buys Windsurf after OpenAI’s deal falls through and Google hires its CEO
Why it matters here. A vendor with hundreds of enterprise customers changed hands in a weekend. Ownership risk is a scored line.
21 May 2025Salesforce Ben
How customers reacted to Salesforce’s new Agentforce pricing based on Flex Credits
Why it matters here. Pricing moved from conversations to actions. Usage-based AI costs are harder to forecast.
20 May 2025Bloomberg
Microsoft-backed Builder.ai heads into insolvency after a lender seizes its cash
Why it matters here. Big-name investors did not prevent a collapse. Check the balance sheet, not the logo wall.

Picked for this service and checked against the source. More each Monday in The Weekly Brief.

What good looks like

6 of 6 covered

What buyers are increasingly measured against here, taken from the frameworks doing the measuring. Each line names the engagement that covers it. Where nothing does yet, the row says so.

Capability	What done properly looks like	Covered by
Criteria agreed before vendors are contactedProcurement practice	Scoring is written from your workloads first, so a demo cannot set the terms of the comparison.	2.2 Bake-Off Report
Testing against real data and real failure casesProcurement practice	The shortlist runs your workloads, including the edge cases, not a vendor-prepared happy path.	2.2 Bake-Off Report
Three-year total cost, not licence priceTCO practice	Licensing, egress, integration and the staffing cost of operating it are modelled together.	2.3 Three-Year TCO Model
Vendor viability treated as a scored criterionSupply-chain risk practice	Funding stage, customer concentration and acquisition risk are scored, because a product can be retired out from under you.	2.2 Bake-Off Report
A documented exit pathEU AI Act · Provider obligations	What it takes to leave — data export, integration rework, notice periods — is known before you sign.	2.3 Three-Year TCO Model
Contract monitoring between renewalsVendor management practice	Pricing changes, funding events and end-of-life notices are tracked, not discovered at renewal.	2.4 Renewal Watch

Sample report: 808 Fit & Viability Matrix

808 demo dataset

Lakeshore Example Co. · Fictional demo company · sample data, not a client

Fictional demo company · sample data, not a client

Lakeshore Example Co.

A fictional 1,200-person distributor we use to show what our reports look like. Every number below is invented for the demo.

4 vendors on the shortlist for customer-service AI
Vendor A picked: highest fit, mid-range three-year cost
Vendor B kept as runner-up: strongest viability
Vendor C dropped: viability 2 of 5

Engagements & products

status shown

The deliverables behind this capability. We publish what is live and what is still being built rather than implying a bench we don't have.

2.1 Entry Planned

Vendor Shortlist Teardown

Free published comparison

A public head-to-head of two or three vendors in one category, scored against stated criteria with the reasoning shown — including where the runner-up wins. Demonstrates the method rather than describing it.
2.2 Anchor engagement Planned

Bake-Off Report

Fixed-fee engagement · 4–6 weeks

Your shortlist run side by side against your real workloads, your data and your failure cases, scored against criteria agreed before any vendor is contacted. You get a comparison you can defend in a procurement review, and a recommendation that states its trade-offs.

Shadow Scanner The scan shows what is already in the estate, so the evaluation starts with the incumbent stack instead of a blank page.
2.3 Anchor engagement Planned

Three-Year TCO Model

Deliverable artifact · spreadsheet

A working cost model covering licensing, egress, integration and the staffing cost of actually running the thing. Handed over, not presented — so you can re-run it yourself when pricing changes.

Shadow Scanner Duplicate-spend findings feed the baseline, so the model starts from what you already pay.
2.4 Continuing Planned

Renewal Watch

Annual subscription

Ongoing monitoring of your vendor set: pricing changes, funding events, acquisitions, end-of-life notices, and a flag when a contract is coming up for renewal. Built for a market where products are retired almost as fast as they launch.

Shadow Scanner The scan defines the watch list, and re-scans keep it current as the estate shifts.

Status is kept in one place and shown as it stands. See the full catalogue across all six capabilities.

How Shadow Scanner helps

platform intelligence & risk telemetry

Before you compare new vendors, know what you already pay for. Shadow Scanner maps detected tools against sanctioned equivalents and surfaces the overlap. The evaluation then starts from the incumbent stack, not a blank page.

More about Shadow Scanner

What you walk away with

the outcome

A scored comparison you can defend in a procurement review. A recommendation that states its trade-offs, including where the runner-up would have been the better call.

The trade-off. A real bake-off takes four to six weeks and needs your data and your people's time. A shortlist from a demo takes an afternoon. The difference shows up at renewal.

The five stages

Tell us what you're running.

A person replies within one business day.

Talk to us