The Boilerplate Problem
You know the drill.
Open your IDE. Create a new test file. Import the framework. Set up the browser initialization. Write the setup method. Write the teardown. Structure the test method. Add locators. Write assertions. Add comments for your team.
For a basic login test, that’s 30-45 minutes of scaffolding before you even get to the actual test logic. Multiply that by dozens of test cases, and it’s hours of writing the same boilerplate patterns over and over.
What if you could skip straight to the refinement part?
Introducing AI Automated Test Script Generation (Now Available in Open Beta)
Today, we’re launching AI Automated Test Script Generation in TestRail Cloud—a new way to accelerate automation development for engineers.
What it does:
AI Test Script Generation produces production-quality automation scaffolding from your test cases in approximately 30 seconds. You get well-commented code with proper structure, placeholders for configuration values, and helpful implementation guidance—all based on test cases you’ve already documented in TestRail.
This is a beta feature and a first step toward deeper automation assistance. It’s free for all Cloud customers while we gather feedback and build toward a fuller vision that’s engineered to give you automation assistance where you need it most.
AI Test Script Generation is part of the TestRail 10.2 update, and will be rolling out to all TestRail instances by mid-April 2026.
How It Works
1. Select a test case
Open any test case in TestRail. The test steps and expected results you’ve documented become the foundation for the generated code.
2. Choose your framework
Select your language (Java or Python) and framework (Selenium, Playwright, Cucumber, Behave). BDD templates are available for both Cucumber and Behave. Support for more languages and frameworks will be coming soon!
3. Add context (optional)
Upload page objects, utility classes, or configuration files to help the AI generate code that fits your project’s patterns.
4. Generate
Click “Generate Script,” and in about 30 seconds, you’ll see structured code with detailed comments.
5. Refine via chat
Don’t like something? Use the chat interface to iterate. “Use Page Object Model pattern” → code updates. “Add explicit waits” → done. Refine until it matches your standards.
6. Download and integrate
Download the generated code as a ZIP file with folder structure and config files. You will need to manually integrate it into your existing automation project—this is scaffolding, not a plug-and-play solution.
What You Actually Get
As an example for what to expect, here’s what the AI generates for a login test (Java + Playwright):
java
import com.microsoft.playwright.*;
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;
/**
* Test case: Verify invalid password login behavior
* Generated from TestRail test case TC-1234
*/
public class LoginTest {
static Playwright playwright;
static Browser browser;
Page page;
@BeforeAll
static void setupAll() {
playwright = Playwright.create();
browser = playwright.chromium().launch();
}
@BeforeEach
void setup() {
page = browser.newPage();
}
@Test
void testInvalidPasswordLogin() {
// Step 1: Navigate to login page
page.navigate("${LOGIN_URL}");
// Step 2: Enter valid username
page.fill("#email", "${VALID_USERNAME}");
// Step 3: Enter invalid password
page.fill("#password", "WrongPassword");
// Step 4: Click login button
page.click("button[type='submit']");
// Step 5: Verify error message displays
assertTrue(page.isVisible(".error-alert"),
"Error message should be visible");
assertEquals("Invalid credentials",
page.textContent(".error-alert"),
"Error message text should match expected value");
}
@AfterEach
void teardown() {
page.close();
}
@AfterAll
static void teardownAll() {
browser.close();
playwright.close();
}
}You’ll notice that the result contains:
- Proper imports and setup – Framework-specific initialization done correctly
- Detailed comments – Each code section maps to the original test case steps
- Placeholders for config – ${LOGIN_URL} and ${VALID_USERNAME} instead of hardcoded values
- Assertions with messages – Not just assertions, but helpful failure messages
- Complete lifecycle – Setup, test, and teardown properly structured
In this scenario, the chat interface will then explain: “I’ve generated a Playwright test with proper setup/teardown methods. You’ll need to replace `LOGINURL‘withyouractualloginpageURLand‘{LOGIN_URL}` with your actual login page URL and ` LOGINURL‘withyouractualloginpageURLand‘{VALID_USERNAME}` with a valid test account username. The password field intentionally uses a hardcoded wrong password for this negative test case.”
That’s the kind of guidance you get—not just code, but a personalized explanation of implementation decisions.
Who This Is For
Automation engineers building or scaling test automation
You know what good automation looks like. This gives you the scaffolding you need so you can focus on sophisticated test logic, framework improvements, and edge cases instead of writing import statements for the hundredth time.
QA engineers with coding skills
You’re comfortable reading and modifying code. This accelerates your script development, especially when working with frameworks you use less frequently.
Who this is NOT for:
This feature requires automation engineering expertise. If you’re not comfortable reviewing code, integrating it into existing projects, and customizing for your environment, this tool won’t be useful yet.
What This Is (and What It Isn’t)
This IS:
- ✅ An acceleration tool that generates high-quality scaffolding
- ✅ A first step toward deeper automation assistance
- ✅ A beta feature we’re actively improving based on feedback
- ✅ Free during the beta period for all Cloud plan tiers
This ISN’T:
- ❌ A replacement for automation engineering expertise
- ❌ Production-ready code ready to execute without human review
- ❌ Integrated with your repository or IDE (you download and integrate manually)
- ❌ Aware of your existing automation framework context
- ❌ Available on TestRail Server
Why We’re Building This
At TestRail, test cases are already structured documentation of what needs to be tested. The steps, expected results, and test data are all there. But when automation engineers go to write scripts, they start from scratch in their IDE.
That handoff has always felt inefficient.
With AI, we can translate that structured test knowledge into structured code scaffolding. It’s not perfect. It’s not production-ready without review. But it’s a legitimate head start.
This is a first step. The vision includes repository integration, project-aware code generation, and multi-test-case processing. We’re not all the way there yet—but we’re starting with high-quality code generation and gathering feedback to inform what we build next.
Our goal is to build AI assistance that is ethical, sustainable, and truly useful. Your input on this beta directly shapes our roadmap and helps define AI features to come!
Supported Frameworks
8 framework combinations currently supported:
Java:
- Selenium + Maven
- Playwright + Maven
- Cucumber + Selenium + Maven (BDD)
- Cucumber + Playwright + Maven (BDD)
Python:
- Selenium + Poetry
- Playwright + Poetry
- Behave + Selenium + Poetry (BDD)
- Behave + Playwright + Poetry (BDD)
Not yet supported: C#, JavaScript/TypeScript, Ruby, other dependency managers, Cypress, WebDriverIO
If you use a currently unsupported framework, let us know through your beta feedback —that helps us prioritize what comes next.
Technical Details
Availability: TestRail Cloud only
Release status: Open beta, actively gathering feedback to improve code quality and inform roadmap
Access: All Cloud plan tiers (Free Trial, Professional, Enterprise)
Data handling: Your input, along with any optional context you provide (e.g., project-specific data, domain terms), is securely transmitted to a large language model (LLM) via encrypted APIs. Your data is not used to train or improve the underlying LLMs. Read our full AI Data Policy here.
Generated output:
- ZIP file with folder structure
- Framework-specific config files (pom.xml, pyproject.toml, etc.)
- Test script(s) with detailed comments
- Placeholders for environment-specific values
Chat refinement:
- Request pattern changes, refactoring, and improvements
- Not conversational—focused on code iteration only
- Changes persist in the current session; chat does not retain memory of past sessions
The Bottom Line
AI Automated Test Script Generation won’t write perfect production code for you. It’s in beta, it requires manual integration, and it needs your engineering expertise.
But it will save you 30-45 minutes of boilerplate work per test. It generates well-commented, properly structured scaffolding with helpful implementation guidance. And, most importantly, it’s a foundation we’re building on toward deeper automation assistance.
If you’re an automation engineer who’s tired of writing the same setup/teardown patterns over and over, give AI Test Script Generation a try!
Available now in TestRail Cloud. Free during beta.
Beta Disclaimer
AI Automated Test Script Generation is in beta and available to all TestRail Cloud customers at no additional cost. Generated code requires human review and manual integration into existing automation projects. We welcome your feedback as we continue to improve code quality and expand capabilities.
