Pilot Tests¶

During the development of your interview guide, you can test your setup and questions at any time before going live. Pilot tests let you run synthetic interviews, conversations between the AI interviewer and AI respondents, so you can see how your guide behaves in practice, catch awkward questions or probing, and refine it before any real participants take part.

Simulations¶

A simulation is a reusable test setup. You create one, choose how the synthetic respondents should answer, and then run as many synthetic interviews from it as you need. Each simulation has two parts:

Setup — define how the synthetic respondents behave.
Runs — run the interviews and watch their progress.

Creating a simulation¶

On the Simulations page, the test setups are grouped by type. Click New Test under the type you want, give it a title, and create it. You can come back to any test later through its Setup and Runs links, or delete it from the menu in the corner of its card.

There are three types of simulation, differing only in how the synthetic respondents produce their answers.

Fixed Answers¶

Write a predefined answer for every main question in your guide. During the simulation the respondent gives exactly those answers, and the interviewer still generates follow-up questions where relevant. Use this when you want full control over the answers, for example to test how the interview reacts to a specific scenario. Questions that can't be answered (such as introductions) are skipped automatically, so you only fill in the ones that take a response.

Fixed AI Respondents¶

Define one or more personas, one per text box. Each persona is used as-is to generate a synthetic respondent, so the AI answers in character based on the description you provide. Use this when you have specific kinds of respondents in mind and want to see how the interview unfolds for each of them.

Shuffled AI Respondents¶

Specify a pool of background characteristics, and the system randomly mixes them to generate a varied synthetic respondent for each interview. This is the best way to quickly produce diverse respondents without writing each one out. You can provide:

Names and Gender — name/gender pairs to draw from.
Age Range — the minimum and maximum age.
Education, Occupations, Locations, Personalities — lists of options to mix from.
Communication Traits — how respondents express themselves, split into Length, Style, and Tone.
Extra traits — any additional characteristics you want in the mix.
Refusal Rate — a minimum and maximum percentage chance that a respondent will be explicitly told to refuse a given question, useful for testing how your guide handles non-responses.

Leave any field empty to leave it out of the mix. Remember to save your setup before running.

Running a simulation¶

Open a simulation's Runs tab to start synthetic interviews and review previous runs. Before running, you can adjust a few settings:

Number of synthetic interviews — how many respondents to run in this batch (up to 50).
Language model — the model the synthetic respondents use to generate their answers.
Interview language — the language the interview is conducted in.

Click Run test to start the batch. Each run appears in the Test Runs table below with its creation time, language, number of interviews, model, and status (running, completed, or failed). Running tests refresh automatically, so you can watch their progress without reloading the page.

Test Results¶

The Test Results page collects the interviews from all your pilot tests and simulations in one table. You can sort by any column (created date, number of messages, type, test name, language, status) to find what you're looking for. Results are available at any point, both while an interview is still running and after it has completed.

From here you can:

View an individual result to read the full conversation.
Download results: select the interviews you want with their checkboxes and click the download button to export them, or download a single result from its row menu.
Delete results you no longer need.