Compare two answers using criteria you write down first

The same request can produce very different answers in two services or with two versions of a prompt. If you compare them by feel, it is easy to choose the one that sounds most convincing. When you write down your criteria before reading, you choose the answer that works, not the one that sounds best.

Compare in four steps

1. 1

   Use the same task and the same information

   Send the same request to two services, or send two versions of a request to the same service. If the information differs, you will not know what caused the answers to differ.

2. 2

   Write down your criteria before reading

   Write down two or three criteria that determine whether an answer is useful, such as “sticks to the information provided,” “is no more than four sentences,” or “suits the reader.” Also decide which criterion matters most if the answers meet different criteria.

3. 3

   Assess each answer against each criterion

   Write a brief observation about each answer next to every criterion.

4. 4

   Choose and explain

   Choose the answer that meets the criteria better, and write one sentence explaining the basis for your choice.

Example: an invitation to a yard sale

You asked two assistants for a short invitation to a neighbourhood yard sale on Saturday from 10 a.m. to 2 p.m. You wrote down three criteria in order of importance beforehand: the answer contains no invented information, includes all the information provided, and has no more than four sentences.

• No invented information: Answer A added a café; Answer B added nothing.
• All information included: Answer A, yes; Answer B is missing the time.
• No more than four sentences: both meet this criterion.
• Choice: I will use Answer B and add the time myself, because I wrote down the absence of invented information as the most important criterion.

When choosing a service

The same method works when you are testing which assistant suits your task. Compare services using your own task and the same criteria, not their general reputation.