Before we test any tool, we map the category. The lead editor for a shortlist talks to five to eight practitioners — operators currently buying or running the kind of tool we're covering. We ask three questions: what's the workflow this category should handle, what are the tools you've used and switched away from, and what would make a tool a one-line 'yes' for you. Those answers shape our test criteria, not the vendor's marketing claims.

From a long list of every vendor in the category, we narrow to a tested set of 8–12 platforms. Selection criteria: customer base of at least 50 verifiable customers, active development (shipped a real feature in the last six months), and either public pricing or a trial we can access without a sales call. Tools that fail any of these are noted but not shortlisted.

Every shortlisted tool gets a real test — three weeks minimum, six weeks ideal. We use the tool on actual workflows: a real sales pipeline, a real support backlog, a real engineering sprint. We do not write reviews from demo recordings or trial click-throughs. The lead editor logs daily notes, captures specific failures and wins, and tags each tool against our category rubric.

We score each tool on five criteria, weighted by category. The criteria are always:

• Workflow fit — how cleanly the tool handles the real work it's bought to do
• Implementation cost — time-to-first-value, integration friction, internal change management
• Total cost of ownership — sticker price plus admin time plus hidden tier-up costs
• Reliability — uptime, data quality, escape hatches when things break
• Honesty — pricing transparency, no dark patterns, how the vendor treats trial users

Scores are 0–10 on each criterion and we publish the rubric for every shortlist so readers can see how we got to the rating.

Before publication, every shortlist is reviewed by a second editor who did not run the test. That editor's job is to push back on weak claims, missing trade-offs and language that sounds like marketing. Vendors do not see drafts before publication. After publication, vendors can submit corrections through our corrections process — factual corrections get processed within five business days; opinion pushback does not.

Every shortlist gets a full re-test once a year. We update rankings when material change happens — new pricing model, leadership change, acquisition, sunsetting of features. Every page shows the 'Last updated' date for that shortlist, and material updates produce a new test run, not a touch-up.