AI Assistant

Introduction

What is the AI Assistant?

The AI Assistant is Testver’s built-in chat agent. You open it from the sidebar (AI Assistant, the sparkles icon) or at the URL /ai, type a request in plain English, and the assistant works on your test-automation project the way a teammate would: it reads your files, writes and edits test scripts, runs your test suite, sends API requests, and can even open a real browser or a cloud mobile device and click through your app while you watch.

Unlike a generic chatbot, the assistant is grounded in your actual project. It can see your folder structure, your existing tests, page objects, configs, and the results of your latest runs. When it suggests a fix, it can apply that fix directly to the file on disk (after you approve), then re-run the test to prove it worked.

Note

Analogy: think of the AI Assistant as a pair-programming teammate who sits next to you. You describe what you want (“write a login test”, “fix the flaky checkout test”, “summarise yesterday’s failures”); they go off, read the code, make changes, run things, and report back. The difference is that this teammate always asks “may I?” before touching a file or running a command, and you can stop them at any moment.

Note

In one sentence: the AI Assistant is a project-aware AI agent you talk to in chat, and it does real work, reading code, editing files, running tests, calling APIs, and driving a browser or phone, with your permission.

Caution

The AI Assistant needs a configured AI provider (an LLM API key, or a local model such as Ollama/LM Studio) before it will respond. If no provider is set up, the model picker stays empty and chat requests will fail. Set this up under Settings, then come back to /ai.

Caution

This screen runs real actions on your machine and your project. The agent can create and overwrite files, run shell commands, execute your test suite, and send live API/browser/mobile traffic. It always asks permission first for these actions, but you are responsible for what you approve. Read each permission prompt before clicking Allow.

Who this guide is for

This guide is for anyone who uses the AI Assistant screen, whatever your level:

• QA engineers and testers who want to write, fix, or refactor test scripts faster.
• Developers who need to understand an unfamiliar test suite or debug failing tests.
• API testers who want to generate, validate, and chain API requests by describing them.
• Test leads and managers who want quick summaries, flaky-test reports, and dashboards from existing run data.
• Newcomers to Testver who want a guided, plain-English way to get things done without learning every menu first. No coding is strictly required to use the assistant, you talk to it in plain English, but it helps to understand what it is doing on your behalf, which is what this guide explains.

Key terms

A few words appear throughout this guide and on screen. Here is what each one means in the context of the AI Assistant.

Term	What it means here
Agent	The AI itself. Not just a text generator, it takes actions (reads files, runs tests, clicks in a browser) to complete your request, step by step.
Tool call	A single concrete action the agent takes, such as reading a file, editing a file, running a command, sending an API request, or clicking in the browser. Each tool call appears in the chat as a small labelled row (for example read_file, api_request, browser_launch).
Permission	A consent prompt. Before the agent does anything that changes your machine or sends live traffic, it shows a Permission required card and waits for you to choose Allow, Always allow, or Deny.
Mode	Which kind of work this chat is focused on: Web (test scripts), API (endpoints and chains), Mobile (iOS/Android devices), or Reporting (analytics and dashboards). The mode decides which tools and instructions the agent is given.
Model / Provider	The underlying AI brain. A provider is the company or service (e.g. OpenAI, Anthropic, a local Ollama). A model is the specific brain from that provider (e.g. a GPT or Claude model). You pick these from the model selector near the Send button.
Live preview	The split panel that appears on the right when the agent opens a browser (web mode) or a cloud phone (mobile mode), so you can watch what it is doing in real time, and optionally take control yourself.
Session	One saved conversation, kept in the history sidebar. A session remembers its messages, the agent’s tool calls and file edits, and the mode it was in. You can rename, pin, search, reopen, or delete sessions.

How it works

Every request follows the same simple loop:

1. You type a message and press Send (or Enter).
2. The assistant streams back its thinking as text, in real time.
3. When it needs to act, it makes a tool call, shown as a labelled row in the chat (a spinner while running, a green tick when done).
4. If that action changes files or sends live traffic, it first shows a Permission required card; nothing happens until you choose.
5. File edits appear inline as a colour-coded diff (lines added in green, removed in red) so you can see exactly what changed.
6. The agent keeps reading, editing, and running as needed, then finishes with a written summary of what it did.
7. The whole conversation, text, tool calls, diffs, and errors, is saved as a session you can reopen later.

Getting Started

Opening the AI Assistant

1. In the left sidebar, click AI Assistant (the sparkles icon). The browser navigates to /ai.
2. Alternatively, open the command palette and choose AI Assistant, or click the AI Assistant card on the dashboard.
3. The home screen greets you by name with a time-of-day greeting (“Good morning / afternoon / evening”) and asks “What would you like to work on today?”.
4. A rolling spotlight cycles through the three example modes (Web, API, Reporting) as a hint of what the assistant can do.
5. Choose your Mode (Web / API / Mobile / Reporting) from the dropdown in the composer toolbar, type your request, and press Enter to begin.

Tip

If you arrived here from another page, for example the Reports → Templates tab or Test Cases → “Automate with AI”, the message box may already be pre-filled with a starter prompt. It is not sent automatically: review or edit it first, then press Enter.

The screen layout

The AI Assistant is a focused, full-height chat screen. Its parts are:

Area	Where	What it does
Welcome / Timeline	Centre	Before you send anything you see the welcome screen (greeting + mode spotlight). After your first message it becomes the conversation timeline: your messages, the AI’s replies, tool calls, and file diffs.
Composer	Bottom centre	The message box plus its toolbar: attach files, pick mode, see where browser actions run, pick the model, and send (or stop).
Floating controls	Top-right	Two small buttons: + (New Session) and the History (clock) icon to show or hide the session sidebar.
History sidebar	Right edge	Slides in when you click the History icon. Search box, mode filter chips, and your saved sessions grouped by Pinned / Today / Yesterday / This Week / Older.
Live preview panel	Right split	Appears only when the agent opens a browser (web) or a cloud device (mobile). Shows a live mirror you can watch and control. Resizable, and (for web) detachable as a floating Picture-in-Picture.
Permission card	Above composer	A floating Permission required card appears just above the message box whenever the agent asks to run a tool that changes things.

At a glance

• Send with the paper-plane button or Enter; use Shift+Enter for a new line. The box grows as you type (up to a limit) and resets after sending.
• While the AI is working, the Send button becomes a Stop (square) button, press it to halt generation immediately.
• Attach files with the paperclip: documents, spreadsheets, images, and code are all supported.
• The mode you pick is remembered for next time and is saved per session.
• A small badge near the mode picker tells you whether browser actions will run on the Server or on your Local machine.
• Everything you do is saved as a session, reopen it any time from the History sidebar.

The Composer (Talking to the Agent)

The message box

The composer at the bottom is where you talk to the agent. The text box shows the placeholder “Ask anything about your project…”. Type naturally, full sentences work better than keywords. The box auto-grows as you add lines (up to a maximum height, after which it scrolls), and snaps back to a single line once you send.

• Enter sends the message.
• Shift+Enter inserts a new line without sending, useful for multi-line or multi-step requests.
• While the agent is streaming a response, the box is disabled, you cannot send a second message until the current one finishes or you stop it.

Toolbar controls

To the right of the toolbar:

Button	When shown	What it does
Send (paper plane)	When idle	Sends your message. Disabled (greyed) until you type something.
Stop (square)	While the AI is working	Immediately halts generation and any in-progress tool execution. A “Generation stopped by user” marker appears in the timeline, and any pending permission prompts are dismissed.

Tip

Use Stop the moment the agent heads in the wrong direction. You don’t have to wait for it to finish, stop it, refine your instructions, and send again.

Attaching files

Click the paperclip in the toolbar, or drag files straight onto the message area (a “Drop files here to upload” overlay confirms the drop zone). Uploaded files appear as small chips between the text box and the toolbar, each showing a type icon, the file name, and its size, with an x to remove it before sending.

Supported file types include:

Category	Examples
Documents	PDF, Word (.doc/.docx), text (.txt), Markdown (.md), HTML, XML
Spreadsheets / data	Excel (.xls/.xlsx), CSV
Presentations	PowerPoint (.ppt/.pptx)
Code	JavaScript/TypeScript (.js/.ts/.jsx/.tsx), Python (.py), Java, Ruby (.rb), Go, Gherkin (.feature), YAML (.yml/.yaml), JSON
Images	PNG, JPG/JPEG, GIF, WebP, SVG, BMP

Tip

Attach a screenshot of a failing screen, a requirements document, or a sample API response, and ask the agent to use it. For example: drop a CSV and ask “convert my login test into a data-driven test using this file”.

Reading the conversation timeline

Once you send, the centre switches to the timeline. Each kind of entry looks distinct:

Entry	Appearance
Your message	A coloured bubble aligned to the right.
AI reply	A bordered bubble aligned to the left, with rich formatting (headings, lists, code blocks).
Tool call	A small indented row with the tool name and a status icon: spinner (running), green tick (done), red cross (error), amber shield (denied). Click to expand and see arguments and results.
File diff	An inline, expandable diff card showing the file path, a New/Modified badge, and the added/removed lines.
Loading	Three bouncing dots while the agent gets started.
System note	A small centred pill, for example “Generation stopped by user”.
Error	A red bubble describing what went wrong.

Note

Some tool results render as rich cards rather than plain text. API requests show a full request/response card (method, status, timing, headers, formatted JSON body, copy/download). API chains show an expandable step-by-step card. In Reporting mode, a generated dashboard renders inline as an interactive report (KPIs, charts, tables) and a built PDF report shows an inline preview with a one-click download.

Sessions and history

Each conversation is a session. Manage them with the floating controls (top-right) and the History sidebar.

1. Click + (top-right) to start a New Session. The new chat inherits your currently selected mode.
2. Click the History (clock) icon to open the right sidebar of saved sessions.
3. Use the search box to filter sessions by title, and the mode chips (All / Web / Mobile / API / Reporting) to filter by mode.
4. Sessions are grouped by Pinned, Today, Yesterday, This Week, and Older. Each row shows a small mode icon and the title.
5. Click any session to reopen it, its full timeline (messages, tool calls, diffs) is restored, and its saved mode is reapplied.
6. Hover a session and click the ⋮ menu for Pin to Top, Rename, or Delete.

Caution

Delete is permanent. A confirmation dialog warns that deleting removes the conversation, all messages, and the full timeline history, and that this cannot be undone.

Tip

Pin the sessions you return to often (a recurring debugging thread, a working API chain) so they stay at the top under the Pinned group regardless of date.

Choosing a Mode

What a mode does

The Mode decides what the assistant is built to do for this chat, which set of tools it can use and which instructions it follows. Pick the mode that matches your task before sending. You can change it at any time; the choice is saved with the session and remembered for your next new chat.

Mode	What it does
Web	Test-automation work: writing and fixing end-to-end test scripts, page objects, locators, and frameworks. This is the mode that can open and drive a live browser.
API	API testing: generating and validating endpoint requests, JSON schemas, collections, and multi-step request chains, with rich request/response and chain result cards in the timeline.
Mobile	iOS / Android testing via Appium and cloud (or local) devices. Adds the mobile session bar above the composer so you can pick a provider, device, and app, and drives a live cloud device.
Reporting	Analytics and documents: summarising tests and run results, and generating reports, dashboards, and exports. Renders interactive dashboards and inline PDF previews in the chat.

Note

Which modes you see depends on your project. A web-only project hides Mobile; a mobile-only project hides Web; API and Reporting are always available. The home-screen spotlight highlights Web, API, and Reporting.

The mode picker

The mode picker is the chip in the composer toolbar (left side, after the paperclip). It shows the current mode’s icon and label. Click it to open a list where each mode has an icon, a name, and a one-line description; the active mode is ticked. The menu flips upward automatically if there isn’t room below.

Mode	Dropdown description
Web	Tests, page objects, locators, fixes
API	Endpoints, schemas, collections, chains
Mobile	iOS / Android via Appium and cloud devices
Reporting	Analytics, charts, summaries, exports

Tip

If your message looks like it belongs to a different mode, a soft amber banner may appear suggesting a switch (“This looks like an API request. Switch modes?”). It never changes the mode on its own, click Switch to … to accept or Keep … to dismiss.

The execution target badge

Next to the mode picker is a small badge showing where the agent’s browser actions will run:

Badge	Meaning
Server (cloud icon)	Browser actions run on the Testver server. This is the default when no local connector is registered.
Local (monitor icon)	Browser actions run on your own machine via a registered Local Browser connector; the badge may show your machine name.

Click the badge to open the Connectors page, where you can set up or manage a local connection. The badge polls status every few seconds, so it stays in sync with the Connectors page.

The model selector

On the right of the toolbar, just before Send, is the model selector. It shows the current provider’s icon and the model name. Click it to open a list grouped by provider; under each provider heading are its available models. Pick one to switch; the active model is ticked.

• Only providers that have an API key configured, or local providers (such as Ollama / LM Studio), appear in the list.
• If no providers are configured, the selector does not appear at all, set one up in Settings first.
• The assistant starts on your default provider’s active model; switching here applies to the messages you send next.

Caution

Different models have different strengths, speeds, and costs. Larger models reason better about complex code but may be slower and cost more per request. Pick the model that fits the task.

Tool Calls, Permissions & Changes

What a tool call is

When the agent acts, it makes a tool call, a single concrete operation, and each one appears as a row in the timeline with a status icon. Typical tools include reading a file, editing or creating a file, listing the project, running a command or test, sending an API request, running an API chain, launching or driving a browser, and (in Reporting mode) generating a dashboard or building a report.

Click any completed tool row to expand it and inspect the exact arguments the agent used and the result it got back. Read-only tools (like reading a file or listing folders) generally run without asking; tools that change your machine or send live traffic ask permission first.

Approving permissions

Before the agent runs a tool that changes files, runs commands, or sends live browser/API/mobile traffic, a Permission required card floats just above the message box. It shows a shield icon, the tool name, and, for file or command tools, the full path or arguments involved, so you can see exactly what is about to happen. If more requests are waiting, it shows “+N more queued”.

Button	What it does
Allow	Approves this single tool call, once. The next time the agent wants the same tool, it will ask again.
Always allow {tool}	Approves this call and auto-approves all future calls to that same tool for the rest of the session, so you aren’t re-prompted for it. Tracked on the server for this conversation.
Deny	Rejects this call. The agent is told it was denied (the tool row shows an amber shield) and continues without performing that action.

Caution

Always allow is convenient but powerful, it means that tool runs unattended for the rest of the session. Be especially careful before “always allowing” tools that write files or run commands. When in doubt, use plain Allow so you stay in the loop on each action.

Note

Permission is requested per action, every time, unless you chose “Always allow” for that tool. Starting a new session, or resetting the chat, clears any “always allow” approvals.

Reviewing file changes (diffs)

When the agent changes a file, the change is shown inline as an expandable diff card with the file path and a badge: New for a freshly created file or Modified for an edited one. Added lines are highlighted green, removed lines red, and the card summarises how many lines were added and removed. This lets you review exactly what the agent did to your code, directly in the chat.

Tip

Diffs are part of the saved session. Reopen a past conversation later and you’ll still see every file change the agent made, not just the text it wrote.

Running tests and commands

The agent can run your test suite and other commands as part of completing a request, for example, “run my tests, fix the failures, and re-run until they pass”. Each run is a tool call you approve, and its output flows back into the conversation so the agent (and you) can see what passed or failed and decide what to do next.

Caution

Running tests and commands executes real code on the machine where Testver runs (server or your local machine, per the routing badge). Only approve commands you understand and trust.

Confirming destructive actions

Destructive UI actions are confirmed separately from tool permissions. Deleting a session, for instance, always shows a confirmation dialog spelling out that the conversation and its full history will be permanently removed. Read these dialogs before confirming.

The Live Preview Panel

When the preview appears

When the agent opens a browser (in Web mode) or a cloud device (in Mobile mode), a live preview panel splits in from the right so you can watch the agent work in real time, and step in to drive it yourself. The panel is resizable: drag the divider between the chat and the preview. When no browser or device is active, the chat fills the full width and there is no preview.

Web preview controls

In Web mode, the preview is a live mirror of the browser the agent is driving (a screencast). Its toolbar gives you full control:

Control	What it does
Tabs	Across the top, click to switch between open browser tabs.
Back / Forward / Reload	Navigate the page history and refresh.
URL bar	Type an address and press Enter to navigate the live browser.
AI / User toggle	AI = the agent controls the browser (default). User = you take over, your clicks, typing, scrolling, and right-clicks are forwarded into the live page.
Pause / Resume	Appears while the AI is acting. Pause halts the agent before its next browser action; a “AI Paused” overlay shows until you Resume.
Watch mode (eye)	Toggles an overlay that visually flags each AI action on the page as it happens. On by default.
Element highlight (crosshair)	Hover the page to outline elements and show their tag/id/class, handy for understanding selectors.
Viewport presets (monitor)	Resize the browser to Mobile, Tablet, Desktop, or Full HD dimensions.
DevTools panel	Toggles a bottom panel with network and console activity from the live page.
PiP	Pops the live view out into a floating, draggable, resizable Picture-in-Picture window.
Close	Ends the browser session.

A status bar along the bottom shows the live connection state, current frame rate (fps), the viewport size, and whether the AI is working, paused, or you’re in interactive mode. You can Disconnect the screencast to save resources without closing the browser, then Connect again.

Tip

Switch to User control to do something the agent can’t, dismiss a cookie banner, log in with a one-time code, scroll to a specific spot, then switch back to AI and ask it to continue from there.

Mobile sessions

In Mobile mode, a Mobile session bar appears above the message box. Use it to pick a Provider (cloud service such as BrowserStack, or local devices), a Device (searchable list of real and virtual devices), and an App (an uploaded .apk or .ipa), then click Start session.

1. Pick a provider that shows ready (has credentials). If it shows “no creds”, add them under Connectors first.
2. Pick a device from the searchable list, real devices are listed first.
3. Pick an app, or click Upload .apk / .ipa to add one. A progress bar shows the upload (apps can be large).
4. Click Start session. The first launch can take 30 to 60 seconds; a status line keeps you informed.
5. Once live, a green Live · {device} pill and a View link (opens the provider’s live view) appear, and the right panel mirrors the device. The right-hand mirror polls device screenshots so you see the screen update; you can also record taps and assertions there to build a test. While a session is live, the provider/device/app pickers are hidden, because a cloud session is fixed to the values it started with; click End session to change them.

Caution

Cloud mobile sessions consume real device minutes from your provider. Start a session when you’re ready to work, and End session when you’re done to avoid wasting time.

Note

If a test run “claims” the live mobile device, a “Test driving this session” banner appears and your taps are paused so they don’t conflict with the running test. Control returns to you automatically when the run ends.

Taking manual control

• You can watch every action live, take manual control of a web browser, navigate and resize it, inspect elements and network/console activity, and pop the web view out as Picture-in-Picture.
• You can pause and resume the AI mid-task in the web view, and end the browser or mobile session at any time.
• You can’t type into the live web page when control is set to AI, switch to User first.
• You can’t change the mobile provider, device, or app while a session is live, end the session first.
• You can’t tap the mobile device while a test run has claimed it, your taps resume when the run finishes.

Common Tasks (How Do I…?)

I want to…	Do this
Write a new test	Switch to Web mode, then ask, e.g. “Write a Playwright login test for /login with valid, invalid, and empty-field cases.”
Fix failing tests	Ask “Analyse my recent test failures and fix them,” then approve the file edits and re-runs as the agent works.
Stop the agent	Click the Stop (square) button that replaces Send while it’s working.
Watch the browser	Stay in Web mode; the live preview splits in automatically when the agent launches a browser.
Take over the browser	In the live preview toolbar, click User. Switch back to AI when done.
Test on a real phone	Switch to Mobile mode, pick a provider/device/app in the session bar, and click Start session.
Generate API tests	Switch to API mode and describe the endpoints or chain, e.g. “Build an auth + CRUD test chain for /users.”
Get a failure summary	Switch to Reporting mode and ask “Summarise yesterday’s failures” or “Show the top 5 flaky tests this week.”
Attach a file	Click the paperclip or drag the file onto the message area, then describe what to do with it.
Switch AI model	Click the model selector (left of Send) and choose a model under any configured provider.
Reopen an old chat	Click the History icon (top-right), then click the session you want.
Keep a chat handy	Open the session’s ⋮ menu and choose Pin to Top.
Start fresh	Click + (top-right) for a New Session; it keeps your current mode.

Tips & Best Practices

• Pick the right mode first. Web for test code, API for endpoints, Mobile for devices, Reporting for analytics. The mode shapes everything the agent can do.
• Be specific. “Fix the flaky selector in checkout.spec by switching to a data-testid locator” works better than “fix my tests.”
• Review diffs before moving on. Expand each file-diff card to confirm the agent changed what you expected.
• Use Allow, not Always allow, for write actions until you trust the pattern, you stay in control of each file change.
• Stop early. If the agent drifts, press Stop, clarify, and resend rather than letting it continue.
• Take manual control when needed. Switch the live browser to User mode for logins, captchas, or cookie banners the agent can’t handle.
• Attach context. Drop requirement docs, sample responses, or failure screenshots so the agent works from real information.
• Keep sessions tidy. Pin the ones you reuse, rename them descriptively, and delete the noise.
• End mobile sessions as soon as you’re done to avoid burning cloud device minutes.
• Mind the routing badge. Know whether actions run on the Server or your Local machine before approving them.

Troubleshooting & FAQ

Symptom / Question	Answer
The model selector is empty / chat fails	No AI provider is configured. Add an API key (or a local model like Ollama/LM Studio) in Settings, then return to /ai.
The agent didn’t change a file	Check whether you clicked Deny on a permission card (the tool row shows an amber shield), or whether the request was read-only. Re-ask and Allow the edit.
I keep getting permission prompts	That’s by design for safety. For a tool you trust this session, click Always allow {tool} to stop being re-prompted for it.
No live preview appeared	The preview only shows once the agent actually launches a browser (Web mode) or you start a Mobile session. In Web mode, ask it to open the browser or navigate to a page.
I can’t type in the live browser	Control is set to AI. Click User in the live-view toolbar to take over, then switch back to AI.
The mobile provider shows “no creds”	Credentials for that cloud provider aren’t set. Click into Connectors (via Settings) and add them, then refresh the device list.
Start session fails or is slow	First launches can take 30 to 60 seconds. Ensure a device AND an app are selected (both are required), and that the provider has credentials.
My taps on the phone do nothing	A test run has claimed the device (“Test driving this session” banner). Your taps resume automatically when the run ends.
The browser closed unexpectedly	A “Browser session closed” notification appears. The agent may have closed it, or it timed out. Ask the agent to launch a browser again.
Did I lose my conversation?	No, conversations are saved as sessions. Open the History sidebar and reopen it; tool calls and diffs are restored too.
How do I start completely over?	Click + (New Session) for a clean chat. Deleting a session, by contrast, is permanent.
A suggested-mode banner keeps appearing	It’s only a soft hint based on your wording. Click Switch to … to accept or Keep … to dismiss; it never changes the mode by itself.

Caution

Remember: the AI Assistant executes real code and real actions, editing files, running commands and tests, and sending live browser, API, and mobile traffic, on the machine where Testver runs. It asks permission before each changing action, but you decide what to approve. Read every Permission required card and every confirmation dialog before clicking, and prefer Allow over Always allow for anything that writes files or runs commands.

Related

• AI Record — record clicks → script generation.
• Reports → Templates — pre-canned reporting prompts.
• Settings → LLM Providers — pick the model.