Analytics
Introduction
Section titled “Introduction”What is the Analytics & Reports screen?
Section titled “What is the Analytics & Reports screen?”The Analytics & Reports screen (heading shown in the app as Analytics & Reports, with the subtitle Test health dashboard, execution analytics, and AI-generated reports) is Testver’s read-only reporting hub. It does not let you create or edit anything; instead it reads the data you have already captured elsewhere in Testver - test cases, test cycle executions and defects - and renders it as scorecards, distribution bars, trend charts and tables across five tabs: Health Dashboard, Execution Analytics, Automation Usage, Defect Analytics and Test Suite Overview.
Think of it as the dashboard cluster in a car. The engine (your test cases and runs) does the work; the dashboard simply gathers the readings - speed, fuel, temperature - and shows them on dials so you can tell at a glance whether things are healthy or need attention. The Analytics screen never drives the car; it only reports on it.
Who this guide is for
Section titled “Who this guide is for”This guide is for anyone who wants to understand the state of testing rather than run individual tests:
- QA leads and test managers who report pass rates, health and progress to stakeholders.
- Automation engineers who want to see how much of the suite is automated and where flaky tests live.
- Developers and triagers who care about defect severity, status and resolution time.
- Project managers and product owners who want a single-screen snapshot and a downloadable PDF for status meetings.
- New team members who want to learn the shape and quality of the existing test suite quickly.
Key terms
Section titled “Key terms”Every term below appears somewhere on this screen. Keep this glossary handy as you read the rest of the guide.
| Term | What it means here |
|---|---|
| KPI (Key Performance Indicator) | A single headline number shown in a stat card at the top of a tab, e.g. Total Executions, Pass Rate, Total Defects. |
| Test Health Score | An overall 0-100 score shown in the big ring on the Health Dashboard. Labelled Excellent (>=90), Good (>=75), Fair (>=50), Poor (>=25) or Critical below that. |
| Pass Rate | Percentage of executions that passed, calculated as passed / total. Coloured green at >=75%, amber at >=50%, red below. |
| Fail Rate | Percentage of executions that failed. |
| Flaky Score | A 0-100% measure of how unreliable a test is (passes sometimes, fails other times). Red at >=60%, amber at >=30%, grey below. |
| Avg Resolution Time | Average hours between a defect’s createdAt and resolvedAt timestamps - the practical equivalent of MTTR (Mean Time To Resolve). |
| Automation Coverage / Automation Rate | Percentage of test cases or executions that are automated rather than run by hand. |
| Requirement Coverage | Percentage of requirements that have at least one linked test case. |
| Execution Freshness | How recently your tests were last executed, expressed as a 0-100 health score. |
| Sync Health | Health of script-to-test-case mappings and automation sync, 0-100. |
| Stale Test Case | A test case that has gone more than 30 days without an update. |
| Cycle | A single run of a test plan. Each cycle has its own total / passed / failed counts. |
| Trend | A time series of daily counts drawn as a small vertical bar chart (e.g. execution trend, defect creation trend, test case growth). |
How it works
Section titled “How it works”Nothing on this screen is typed in by hand. Each tab fires its own data request when you open it and then computes the visuals in your browser:
- Health Dashboard reads the health-score and coverage-stats services (
useTMHealthScore,useTMCoverageStats). - Execution Analytics reads execution analytics (
useTMExecutionAnalytics): totals, pass/fail rates, per-status and per-priority breakdowns, cycle history, daily trend and flaky tests. - Automation Usage reads cycle-level automation usage (
tmGetAutomationUsage) and aggregates manual vs automated counts itself. - Defect Analytics reads the defects list (
useTMDefects) and computes severity, status, priority, open/closed and resolution-time figures in the browser. - Test Suite Overview reads all test cases (
useTMTestCases) plus coverage stats and derives status, priority, type, automation and growth distributions.
Getting Started
Section titled “Getting Started”Opening Analytics & Reports
Section titled “Opening Analytics & Reports”- Sign in to Testver and locate the main navigation.
- Click Analytics (or browse directly to
/analytics). - The page opens on the Health Dashboard tab by default.
- To open a specific tab directly, append
?tab=to the URL with one ofhealth,analytics,automation,defectsorsuite- for example/analytics?tab=defects. The active tab is remembered in the URL, so the page is shareable and bookmarkable.
The screen layout
Section titled “The screen layout”| Area | Where | What it does |
|---|---|---|
| Page header | Top of screen | Shows the chart icon, the title Analytics & Reports and the subtitle Test health dashboard, execution analytics, and AI-generated reports. |
| Export Report button | Top-right of the header | Generates and downloads a PDF report of the analytics (see Section 8). |
| Tab bar | Directly below the header | Five tabs: Health Dashboard, Execution Analytics, Automation Usage, Defect Analytics, Test Suite Overview. The active tab has a coloured underline. |
| Content area | Below the tab bar (max width ~1400px) | Renders the dashboard for the active tab: stat cards, distribution bars, charts and tables. |
| Refresh link | Top-right inside the Health and Execution Analytics tabs | Re-fetches that tab’s data without reloading the page. |
At a glance
Section titled “At a glance”- Health Dashboard: Test Health Score ring; a summary line of total / automated test cases; a 5-card Health Breakdown grid; and two Activity Indicator cards (Stale, Recently Updated).
- Execution Analytics: four KPI cards (Total Executions, Pass Rate, Fail Rate, Avg Duration); a Status Distribution stacked bar; a Priority Breakdown grid; an expandable Cycle History table; an Execution Trend bar chart; and a Flaky Tests table.
- Automation Usage: four KPI cards (Total Executions, Automated, Manual, Automation Rate); an Overall Manual vs Automated split bar; and a Per-Cycle Breakdown table.
- Defect Analytics: four KPI cards (Total, Open, Closed/Resolved, Avg Resolution Time); Severity and Status distribution bars; a Defect Creation Trend chart; and a Priority Breakdown grid.
- Test Suite Overview: five KPI cards (Total Test Cases, Automated, With Steps, Without Steps, Types); Status and Priority distributions; a Test Case Growth chart; and an Automation Status bar.
Health Dashboard
Section titled “Health Dashboard”The default tab. It answers one question: how healthy is my test suite overall? A small Refresh link sits at the top-right to re-pull the latest figures.
Test Health Score
Section titled “Test Health Score”A large circular gauge fills clockwise from 0 to 100. The number in the centre is the overall score and the word beneath it is its band. The ring and number are colour-coded so you can read health at a glance.
| Score range | Label | Colour |
|---|---|---|
| >= 90 | Excellent | Green |
| 75 - 89 | Good | Green |
| 50 - 74 | Fair | Amber |
| 25 - 49 | Poor | Red |
| < 25 | Critical | Red |
Below the ring, a summary line reads N Test Cases - M Automated, giving immediate context for the score.
Health Breakdown
Section titled “Health Breakdown”Five cards explain why the overall score is what it is. Each card shows the metric name, its 0-100 score (green/amber/red), a progress bar and a one-line description. Together they roll up into the overall ring.
| Metric | Meaning |
|---|---|
| Automation Coverage | Percentage of test cases that have automation scripts. |
| Requirement Coverage | Requirements that have linked test cases. |
| Execution Freshness | How recently tests have been executed. |
| Quality Score | Average quality across test executions. |
| Sync Health | Script mappings and automation sync status. |
Activity Indicators
Section titled “Activity Indicators”Two cards flag movement in your suite. Each shows a count, and when the count is above zero a progress bar shows what share of total test cases it represents.
| Indicator | Meaning |
|---|---|
| Stale Test Cases (amber) | Test cases with more than 30 days since their last update. A high number suggests neglected coverage. |
| Recently Updated (green) | Test cases updated in the last 7 days - a sign of active maintenance. |
Execution Analytics
Section titled “Execution Analytics”This tab is about test runs: how many, how often they pass, how long they take, and which tests are unreliable. It also has a Refresh link.
Summary metrics
Section titled “Summary metrics”| Metric | Meaning |
|---|---|
| Total Executions | Count of all test executions recorded (thousands separators applied). |
| Pass Rate | Percentage of executions that passed, to one decimal place. |
| Fail Rate | Percentage of executions that failed, to one decimal place. |
| Avg Duration | Mean execution time, formatted as ms, seconds (e.g. 4.2s) or minutes+seconds (e.g. 1m 12s). |
Status Distribution
Section titled “Status Distribution”A single horizontal stacked bar split by execution status. Each segment is colour-coded and a legend beneath lists each status with its count and percentage. Hovering a segment shows the same detail as a tooltip.
| Status | Colour |
|---|---|
| Passed | Green |
| Failed | Red |
| Blocked | Amber |
| Skipped | Grey |
Priority Breakdown
Section titled “Priority Breakdown”Up to four cards - Critical, High, Medium, Low (priorities with zero executions are hidden). Each card shows the total for that priority plus side-by-side Passed (green) and Failed (red) bars with counts and percentages.
Cycle History
Section titled “Cycle History”A table of recent test cycles. Columns: Cycle, Plan, Total, Passed, Failed and Pass Rate (green/amber/red by the same 75/50 thresholds). Click any row to expand it: the expanded panel shows a mini distribution bar (passed / failed / blocked / skipped / in-progress) and a stats line with counts and percentages. Click again to collapse.
Execution Trend
Section titled “Execution Trend”A vertical bar chart of the last 14 days of executions. Each bar is a day, stacked by Passed (green), Failed (red) and Other (grey). Bar height is proportional to that day’s total relative to the busiest day. Hover a bar for a tooltip reading date: N total (P passed / F failed). Every other day is labelled along the x-axis, and the caption notes Last N days.
Flaky Tests
Section titled “Flaky Tests”Lists tests that pass sometimes and fail other times, sorted by Flaky Score descending. Columns: Test Title, Flaky Score (a small bar plus the percentage), Passed (green) and Failed (red). The score bar is red at >=60%, amber at >=30% and grey below - the redder the row, the more attention that test needs.
Automation Usage
Section titled “Automation Usage”This tab answers how much of our testing is automated vs done by hand? It aggregates across every executed cycle.
Summary metrics
Section titled “Summary metrics”| Metric | Meaning |
|---|---|
| Total Executions | Combined manual + automated executions, with a sub-line Across N cycles. |
| Automated | Automated execution count, with percentage of total and the automated pass rate. |
| Manual | Manual execution count, with percentage of total and the manual pass rate. |
| Automation Rate | Automated share of total executions, shown as a percentage with a green progress bar. |
Manual vs Automated split
Section titled “Manual vs Automated split”A single wide bar split into a green Automated portion and an amber Manual portion. Each portion shows its percentage inline when it is wider than 10%. A legend beneath gives the raw automated and manual counts.
Per-cycle breakdown
Section titled “Per-cycle breakdown”One row per executed cycle. The header groups columns under Manual and Automated, each with the same five sub-columns, plus a final Split column:
| Column | Meaning |
|---|---|
| Plan / Cycle | Plan name (bold) with the cycle name beneath. |
| Manual: Total / Pass / Fail / Rate / Time | Manual execution count, passes, failures, pass-rate % (green >=80, amber >=50, red below) and total manual run time. |
| Automated: Total / Pass / Fail / Rate / Time | The same five figures for automated executions. |
| Split | A mini bar (green automated vs amber manual) plus the automated percentage for that cycle. |
Defect Analytics
Section titled “Defect Analytics”This tab summarises the defects logged in Testver. All figures (severity, status, resolution time, trend) are computed from the raw defect list in the browser.
Summary metrics
Section titled “Summary metrics”| Metric | Meaning |
|---|---|
| Total Defects | Count of all defects. |
| Open Defects | Defects in status open, in_progress or reopened, with their share of the total. |
| Closed/Resolved | Defects in status closed, resolved or verified, with their share of the total. |
| Avg Resolution Time | Average hours from creation to resolution, with a sub-line noting how many resolved defects it is based on (the practical MTTR). |
Severity & Status Distribution
Section titled “Severity & Status Distribution”Two side-by-side panels of horizontal bars. Severity Distribution lists critical, blocker, high, medium and low (each with its own colour, count and percentage). Status Distribution lists every status present, sorted by count, with colour-coded bars (e.g. open=blue, in_progress=amber, resolved=green, closed=grey, reopened=red, deferred=violet).
Defect Creation Trend
Section titled “Defect Creation Trend”A vertical bar chart of the last 14 days by defect creation date. Each blue bar is one day; hover for a tooltip reading date: total (X open, Y closed). Every other day is labelled on the x-axis.
Priority Breakdown
Section titled “Priority Breakdown”Four compact cards - Critical (red), High (orange), Medium (amber), Low (green) - each showing the defect count and its percentage of the total.
Test Suite Overview
Section titled “Test Suite Overview”This tab describes the composition of your test-case library: how many there are, their statuses, priorities, types, completeness and growth over time.
Summary metrics
Section titled “Summary metrics”| Metric | Meaning |
|---|---|
| Total Test Cases | Count of all test cases. |
| Automated | Number of automated test cases, with the overall automation percentage. |
| With Steps | Test cases that have at least one defined step, with a completeness percentage. |
| Without Steps | Test cases lacking step definitions (flagged as needing work). |
| Types | How many distinct test-case types exist, with a breakdown listed in the sub-line. |
Status & Priority Distribution
Section titled “Status & Priority Distribution”Two side-by-side bar panels. Status Distribution sorts statuses by count (brand-coloured bars). Priority Distribution lists critical (red), high (orange), medium (amber) and low (green) with counts and percentages.
Test Case Growth
Section titled “Test Case Growth”A cumulative area-style bar chart over the last 30 days, where each bar’s height is the running total of test cases up to that day. Hover any bar for date: +N (Total: running total). The footer summarises Started, Current and total new test cases added in the window.
Automation Status
Section titled “Automation Status”A single stacked bar split by automation state - Automated (green), Manual (grey), In Progress (amber) and Planned (blue) - with a legend showing each state’s count and percentage.
Filters, Date Range & Grouping
Section titled “Filters, Date Range & Grouping”Be aware of an important design choice: this screen has no interactive global filters, date pickers or grouping controls. There is no dropdown to choose a project, status or owner, and no calendar to pick a custom range. Instead, scope and time windows are fixed by the design:
| Control / behaviour | How it works |
|---|---|
| Tab selection | The five tabs are the primary way to switch view. The choice is stored in the URL (?tab=…) so it survives refresh and can be shared. |
| Refresh (Health & Execution Analytics) | Re-fetches the current tab’s data on demand. Other tabs refresh automatically when reopened. |
| Date range (fixed) | Trend charts use built-in windows: Execution Trend = last 14 days, Defect Creation Trend = last 14 days, Test Case Growth = last 30 days. These cannot be changed in the UI. |
| Grouping (fixed) | Grouping is predefined per chart - by status, by priority, by severity, by cycle, by automation state or by creation day. There is no user-selectable group-by. |
| Row expansion | On the Execution Analytics Cycle History table you can click a row to expand its distribution detail - the only interactive drill-down on the screen. |
Exporting a report
Section titled “Exporting a report”The Export Report button at the top-right of the header generates a downloadable PDF of the analytics.
- Click Export Report in the page header.
- A spinner replaces the download icon while the report is generated on the server.
- When ready, the PDF downloads automatically to your browser’s downloads folder, and the icon briefly turns into a green check to confirm success.
- If generation fails, the download simply does not start (the error is logged to the browser console); try again.
Common Tasks (How Do I…?)
Section titled “Common Tasks (How Do I…?)”| I want to… | Do this |
|---|---|
| See my overall test health | Open the Health Dashboard tab and read the central score ring. |
| Understand why my health score is low | On Health Dashboard, scan the five Health Breakdown cards for the lowest (reddest) scores. |
| Check the current pass rate | Open Execution Analytics; read the Pass Rate KPI card. |
| Find unreliable tests | Open Execution Analytics and review the Flaky Tests table (sorted worst-first). |
| Compare results across cycles | Open Execution Analytics and use the Cycle History table; click a row to expand its breakdown. |
| See how automated my testing is | Open Automation Usage; read the Automation Rate card and the split bar. |
| Triage defects by severity | Open Defect Analytics and read the Severity Distribution panel. |
| Measure how fast defects get fixed | Open Defect Analytics; read the Avg Resolution Time KPI card. |
| See how the suite is growing | Open Test Suite Overview and read the Test Case Growth chart. |
| Find test cases missing steps | Open Test Suite Overview; read the Without Steps KPI card. |
| Get the latest numbers | Click Refresh (Health / Execution Analytics) or simply reopen the tab. |
| Share a specific view | Copy the URL with its ?tab= parameter, e.g. /analytics?tab=defects. |
| Produce a report for a meeting | Click Export Report to download the PDF. |
Tips & Best Practices
Section titled “Tips & Best Practices”- Watch the Stale Test Cases count. A rising stale count alongside a falling Execution Freshness score is an early warning that coverage is drifting out of date.
- Act on flaky tests early. Anything red (>=60%) in the Flaky Tests table erodes trust in your whole suite - fix or quarantine it before it spreads doubt.
- Use the colour thresholds as guardrails. Green/amber/red follow the 75/50 (and 80/50 in Automation) rule consistently, so amber is your ‘investigate’ signal and red your ‘act now’ signal.
- Read trend charts for direction, not exact values. They are deliberately compact (14 or 30 days); hover for precise counts when you need them.
- Bookmark the tab you live in. The
?tab=URL parameter means you can jump straight to, say, Defect Analytics every morning. - Refresh before exporting. Re-pull the data on the live tabs so the PDF you hand to stakeholders is current.
Troubleshooting & FAQ
Section titled “Troubleshooting & FAQ”| Symptom | Likely cause & fix |
|---|---|
| A tab shows an empty-state message | There is no underlying data yet. Run test cycles (Automation/Execution), log defects (Defect Analytics) or create test cases (Suite Overview) and return. |
| Execution Analytics says No execution data available yet | No executions have been recorded. Run some test cycles, then click Refresh. |
| Defect Analytics is empty | No defects exist. The message points you to create defects in Integrations first. |
| Failed to load health data / Failed to load execution analytics | The data request errored (often a transient backend/network issue). Click Refresh, or reload the page. |
| Numbers look out of date | Tabs cache their last fetch. Click Refresh on Health or Execution Analytics, or reopen the tab to force a re-fetch. |
| A priority or severity I expected is missing | Categories with a count of zero are hidden by design - they reappear once data exists. |
| The Export Report download never starts | Report generation failed server-side (check the browser console for the logged error) - wait a moment and click Export Report again. |
| I can’t find a date-range or filter control | By design there isn’t one. Trend windows are fixed (14 or 30 days) and grouping is predefined - see Section 8. |
| Avg Resolution Time shows a dash | No defects have both a creation and a resolution timestamp yet, so the average can’t be computed. |
| The trend chart only shows part of my history | Trend charts intentionally show the most recent 14 days (executions/defects) or 30 days (growth); older data is rolled into earlier figures. |