Most ITSM evaluation templates were written for a system of record. A modern service desk is expected to do work, not just track it, and that changes which criteria decide whether the investment pays off. Five questions that belong on every 2026 evaluation.
‍
Most ITSM evaluation templates in circulation today were written for a system of record.

They ask about ITIL process coverage, CMDB depth, licensing tiers, reporting. All still worth asking. But they were designed to answer one question — can this system reliably track our work? — and that is no longer the question that decides whether the investment pays off.

The job has changed. A modern service desk product is expected to do work, not just record it. And the moment execution enters the picture, the criteria that separate a good decision from an expensive one change completely.

Here are five that belong on every 2026 evaluation, each with the question that actually tests it.

1. Are the objectives clear — and who set them?

This one comes before any vendor conversation, and it is the most commonly skipped.

"Improve first response time" and "eliminate L0, L1 and L2 volume" are not two versions of the same goal. They are different projects. They shortlist different products, justify different budgets, and produce different org charts eighteen months later. Yet evaluations routinely begin with both objectives loosely in play and neither one chosen.

It is worth noticing where the objective came from. Incremental targets tend to emerge from the teams closest to current operations, which is entirely rational — those teams understand the constraints better than anyone. But an evaluation scoped to improve the existing model will not surface products designed to replace it.

Ask, before criteria are agreed: which outcome are we actually chasing, and who decided? A first-response-time target and an L1-elimination target will produce different shortlists.

2. Is it actually agentic, or is it agent-washed?

Gartner estimates that of the thousands of vendors now claiming agentic AI, only around 130 are real. The rest is agent washing — assistants, RPA and chatbots rebranded for the moment.

Brand recognition is not a filter for this. Some of the most confident agentic claims in the market come from the most established names in it, because retrofitting an "AI" badge onto an existing product is faster than rebuilding it.

The distinction that matters is simple. Does the product deflect a ticket by answering, or by doing? Answering requires knowledge. Doing requires authenticated, permissioned, audited execution into identity, collaboration, endpoint and HR systems — plus a record of what the agent did and a way to reverse it.

Ask: show an agent complete a multi-step task across two systems, live, with no human approving each step. Not a recorded demo. Not a slide.

3. How fast to live — and how fast to the ROI you were promised?

Long implementations are where business cases quietly expire. Sponsors move on, budgets get re-cut, and the value that was modelled in month one arrives after nobody is still measuring.

Integration capability is not the variable to test here; every vendor has APIs. The variable is what a new integration costs you — in weeks, and in whose weeks. Pre-built connector coverage, native iPaaS and MCP support determine whether your automation roadmap is a configuration exercise or a development project.

Ask: can you be live in 45 days? Which of our systems are pre-built versus custom work, and who does that work — you or us?

4. Is there a real experience layer?

Adoption is the whole ballgame. An agent nobody opens deflects nothing, however good the underlying model is, and portal adoption is where deflection programmes most often die quietly.

The practical test is location. Employees do not go somewhere new to get help; they ask where they already are. If the product lives in Microsoft Teams, Slack, email and voice rather than behind a separate login, adoption is a default rather than a change management programme.

Ask: where does the employee actually interact with this — and what is the adoption rate at a comparable enterprise? Adoption rate, not deflection rate. Vendors quote the second and avoid the first.

5. What is the security and data posture in the age of AI?

The standard vendor security review was built for software that stores data. It does not cover software that reasons over data and then acts on it.

Three things it typically misses: whether your data is used to train models, how information is handled at inference time, and what an agent is permitted to do without a human in the loop. That last one is a governance question as much as a security one, and it will land on your desk regardless of who owns the evaluation.

Ask: dedicated tenant or shared? Is our data used to train models? What can an agent execute without human approval, and where is the audit trail? Which certifications — SOC 2, ISO 27001, HIPAA, GDPR — and which data residency options?

Before you score anyone: get your own numbers

Every question above compares vendors against each other. None of them tells you what your own service desk actually needs — and that is the input the whole evaluation depends on.

The 2026 Agentic AI Benchmark for IT Service Management, built from anonymised tickets across hundreds of production service desks over eight years, found something worth knowing before you shortlist anybody: password reset, account unlock and onboarding access provisioning automate at 85% or better in every sector measured, while the largest queue in a given organisation is frequently the wrong first target. Across all ticket types, a quarter to a third of total volume is confidently automatable.

That is the shape of demand in general. Your own split will differ — and knowing it turns an evaluation from a feature comparison into a business case.

None of these five questions are hard. They are simply not the ones the old template asks.