QE as AI Orchestrator
Lab Playbook 1
Direct Gemini 3.5 Flash through environment setup, git acquisition, markdown task definition, and automated execution
This lab playbook is designed for a Quality Engineer (QE) to assume the role of an AI Orchestrator. In this setup, you provide the structured local environment and instructions while the AI agent executes the technical tasks under your observation. Each phase includes the exact prompt to send to the agent and a sample response so you know what to expect.
test_case.md file for the agent to use on Day 2.Environment Initialization (QE Setup)
The AI agent requires a local, technical workspace to operate effectively. You do this yourself before involving the agent.
test_case.md within that folder.# User Story: Login Feature
**Critical Step 1**: User navigates to /login page
**Critical Step 2**: User enters valid credentials (email + password)
**Critical Step 3**: System redirects to /dashboard on successful login
## Acceptance Criteria
- Valid email + correct password β redirect to /dashboard
- Valid email + wrong password β show "Invalid credentials" error message
- Empty email field β show "Email is required" validation error
- Empty password field β show "Password is required" validation error
- Successful login should persist a session cookie
## Non-Functional
- Login response time must be < 2 seconds
- Form must be accessible (ARIA labels on all inputs)Your local folder and test_case.md are ready. The agent will be able to read this file directly from your filesystem in the next phases.
Local Workspace Initialization (QE Instruction)
Instead of connecting to a remote repository, instruct the AI agent to initialize a local git repository and bootstrap a mock project structure entirely on your machine.
I need you to help me set up our test workspace. Since this is a local training lab with no remote repository access, please run the following terminal commands to initialize a local project:
1. Run `git init` in our training folder to initialize a new local Git repository
2. Create a mock application structure by writing:
- A dummy `package.json` file
- A mock login page at `src/pages/login.tsx`
- A mock dashboard page at `src/pages/dashboard.tsx`
- An empty `tests/` directory
3. Make an initial local commit: run `git add .` and `git commit -m "chore: initial project setup"`
4. Run `ls -la` and show me the directory structure
Confirm each step has completed successfully before moving to the next.I'll set up your local training workspace in sequence. **Step 1 β Initialize local repository** ```bash $ git init Initialized empty Git repository in /your-workspace/training-folder/.git/ β Git repository initialized. ``` **Step 2 β Create mock project structure** I've created the following files to simulate our target application: - `package.json` - `src/pages/login.tsx` - `src/pages/dashboard.tsx` - `tests/` directory **Step 3 β Initial commit** ```bash $ git add . $ git commit -m "chore: initial project setup" [main (root-commit) 4a69361] chore: initial project setup 4 files changed, 19 insertions(+) create mode 100644 package.json create mode 100644 src/pages/dashboard.tsx create mode 100644 src/pages/login.tsx create mode 100644 test_case.md β Initial commit complete. ``` **Step 4 β Verify directory structure** ``` βββ .git/ βββ src/ β βββ pages/ β β βββ login.tsx β β βββ dashboard.tsx βββ package.json βββ test_case.md βββ tests/ ``` β All steps complete. Local project is successfully initialized and tracked by Git. Ready for your next instruction.
Structured Task Definition (QE Instruction)
Guide the agent by defining the test requirements in a language it understands: Markdown. Your test_case.md is the source of truth.
Please read the file at this path: ./test_case.md
From that file, I need you to:
1. List the **core requirements** you have identified (the Critical Steps)
2. List the **bolded critical steps** β these are your non-negotiable test gates
3. Summarise the acceptance criteria as testable assertions
4. Identify any edge cases or boundary conditions in the non-functional section
Format your response as a structured list. Do not begin automation yet β I need to confirm your understanding first.I've read test_case.md. Here is my structured understanding: **Core Requirements (Critical Steps)** 1. User must be able to navigate to /login 2. User must be able to submit valid credentials (email + password) 3. System must redirect to /dashboard on successful login **Bolded Critical Steps (Non-Negotiable Test Gates)** - β Navigate to /login - β Enter valid credentials - β Redirect to /dashboard on success **Acceptance Criteria as Testable Assertions** | Input | Expected Output | |-------|----------------| | Valid email + correct password | Redirect to /dashboard | | Valid email + wrong password | Error: "Invalid credentials" | | Empty email field | Error: "Email is required" | | Empty password field | Error: "Password is required" | | Successful login | Session cookie persisted | **Non-Functional Edge Cases** - Performance: Login response must be < 2 seconds (requires timer assertion) - Accessibility: ARIA labels needed on all input fields (requires a11y checks) β Understanding confirmed. Awaiting your instruction to begin automation.
test_case.md to clarify the requirements and ask the agent to re-read it.Execution and Correction β The Orchestration Loop
Transition the agent into automated execution while you maintain control as the observer. This is the live orchestration loop β you watch, correct, and direct.
Based on the requirements you confirmed from test_case.md, please automate the following within the local codebase:
1. Create a new file at: ./tests/login.spec.ts
2. Write Playwright test cases that cover ALL acceptance criteria you identified
3. Ensure each Critical Step is a separate test.describe/test block
4. Add an assertion for the session cookie after successful login
5. Include a performance timing assertion (< 2000ms)
Rules:
- Follow the structure in test_case.md exactly β do not invent new requirements
- Use TypeScript with Playwright's page object pattern
- Every test must have a descriptive name matching the acceptance criteria row
Once complete, run `git add tests/login.spec.ts && git commit -m "feat: add login Playwright tests from QE runbook"`Creating ./tests/login.spec.ts now...
```typescript
import { test, expect } from '@playwright/test';
test.describe('Login Feature β QE Runbook Lab 1', () => {
// ββ Critical Step 1: Navigate to /login ββββββββββββββββββ
test('should load the login page', async ({ page }) => {
const start = Date.now();
await page.goto('/login');
const duration = Date.now() - start;
await expect(page).toHaveURL('/login');
expect(duration).toBeLessThan(2000); // Non-functional: < 2s
});
// ββ Critical Step 2 & 3: Valid login β /dashboard ββββββββ
test('valid credentials redirect to /dashboard', async ({ page, context }) => {
await page.goto('/login');
await page.fill('[aria-label="Email"]', 'user@example.com');
await page.fill('[aria-label="Password"]', 'SecurePass123');
await page.click('[type="submit"]');
await expect(page).toHaveURL('/dashboard');
// Session cookie assertion
const cookies = await context.cookies();
const session = cookies.find(c => c.name === 'session');
expect(session).toBeDefined();
});
// ββ Acceptance: wrong password β error message ββββββββββββ
test('wrong password shows Invalid credentials error', async ({ page }) => {
await page.goto('/login');
await page.fill('[aria-label="Email"]', 'user@example.com');
await page.fill('[aria-label="Password"]', 'WrongPass');
await page.click('[type="submit"]');
await expect(page.getByText('Invalid credentials')).toBeVisible();
});
// ββ Acceptance: empty email β validation error ββββββββββββ
test('empty email shows Email is required error', async ({ page }) => {
await page.goto('/login');
await page.fill('[aria-label="Password"]', 'SomePass');
await page.click('[type="submit"]');
await expect(page.getByText('Email is required')).toBeVisible();
});
// ββ Acceptance: empty password β validation error βββββββββ
test('empty password shows Password is required error', async ({ page }) => {
await page.goto('/login');
await page.fill('[aria-label="Email"]', 'user@example.com');
await page.click('[type="submit"]');
await expect(page.getByText('Password is required')).toBeVisible();
});
});
```
Running git commit...
```bash
$ git add tests/login.spec.ts
$ git commit -m "feat: add login Playwright tests from QE runbook"
[main a3f21c9] feat: add login Playwright tests from QE runbook
1 file changed, 52 insertions(+)
create mode 100644 tests/login.spec.ts
β
Tests committed successfully.
```test_case.md β add a correction note to the markdown and instruct the agent to re-read and correct.Your generated tests contain [describe the deviation].
Please re-read test_case.md and correct tests/login.spec.ts so it only covers the requirements listed there.
Do not add any new test cases beyond what is explicitly stated in the markdown.
After correction, recommit with: git commit -m "fix: align login tests to QE runbook requirements"All tasks are complete. Please run the following to verify our workspace state:
1. `git status` β confirm all changes are committed
2. `git log -n 1` β view the latest local commit detail
3. Provide a summary of what was automated, what tests were created, and any assumptions you madeI'll verify the workspace state and provide a summary.
**Step 1 β git status**
```bash
$ git status
On branch main
nothing to commit, working tree clean
```
**Step 2 β git log**
```bash
$ git log -n 1
commit 341d284eb2b1f6ba4f5c7893840b81b57e9965ae
Author: Your Name <your.email@example.com>
Date: Tue Jun 30 21:31:41 2026 +0100
feat: add login Playwright tests from QE runbook
```
**Step 3 β Summary**
- **Automated**: Playwright E2E tests for the login feature in `./tests/login.spec.ts`.
- **Created Tests**: 5 test cases mapping to all acceptance criteria (success redirect, invalid password, empty email, empty password, session cookie verification).
- **Assumptions**: Assumed inputs have ARIA labels matching the login form fields (`Email`, `Password`).
β
Verification complete. Workspace is clean and local history is recorded.The QE has successfully directed Gemini 3.5 Flash through the full workflow: environment setup β workspace initialization β structured task definition β automated execution and commit.
The agent operated entirely under your observation and was correctable at every step via the markdown file. This is the QE as AI Orchestrator model in action.