What would count as a useful AI result?

“It answered the question” can describe a convincing demonstration while leaving the business problem unresolved. A person may still need to find a source, check a date, repair a format, or ask someone else whether the answer applies. The useful result is the completed task, including those steps.

Before choosing a measure, write down the beginning and end of that task. Identify who starts it, what information they have, and what the next person needs. This makes success specific enough to examine. It also prevents a project from optimizing the most visible part of a process while ignoring the effort around it.

Describe the present workflow using a small set of real examples. Record what people do, how long different parts take, and where questions move to someone else. Include the ordinary cases and the awkward ones. A baseline made only from the easiest examples gives a weak basis for comparison.

The starting process may have strengths worth keeping. A colleague may recognize an outdated document or know when an answer needs approval. Record those judgments alongside any delay. Otherwise a faster proposed workflow can appear to improve the process while quietly removing a safeguard that made the original result useful.

Speed and quality answer different questions. For an internal information tool, quality might concern whether the answer refers to the current policy, shows the source, respects access, and avoids filling gaps with an unsupported guess. Decide which of these properties matter before reviewing a demonstration.

Keep the measures interpretable. A single overall score can hide a pattern of mistakes concentrated in important cases. Separate categories such as correct answer, incomplete answer, wrong source, and appropriate referral to a person. The categories should help a reviewer decide what needs to change, rather than produce a number with no clear operational meaning.

Use paired examples when comparing the old and proposed processes. Give reviewers the same underlying task and record both the result and the effort needed to reach it. If one route fails, keep that case in the comparison. Removing difficult cases would conceal information the business needs for its decision.

A result that takes seconds to generate may take several minutes to verify. If every answer requires a difficult check, the review burden belongs in the comparison. Record the time and knowledge needed to accept, correct, or reject the proposed result.

Consider the whole route to completion: waiting, searching, reviewing, correcting, and handing the case onward. In an illustrative policy-search workflow, the question is whether staff reach a reliable answer with less total effort. Counting generated answers alone would reward activity even when the same people still have to do the original investigation afterward.

Measures become useful when they support a decision. State what would justify continuing, changing the design, narrowing the task, or stopping a proposed use. A team might accept an assistant for locating sources while keeping interpretation with a policy owner. That is a clearer choice than treating adoption as an all-or-nothing outcome.

Set expectations around the actual consequences of an error. A draft used for private brainstorming and a message sent to a customer have different review needs. Explain those differences in the brief. The evaluation should reflect how the result will be used, including the action someone might take because of it.

Name the person responsible for deciding whether the result remains useful after introduction. Information, working practices, and the people involved can change. A measure that described the original problem may eventually describe a process the business no longer uses.

Keep a small record of the baseline, examples, review criteria, and decision reached. That record makes later discussion more concrete. Halsted, Haines & Co. connects research, strategy, and applied AI around business needs; a clear account of success helps keep the original need visible as an idea becomes a system.