# Praman — AI-First SAP UI5 Test Automation for Playwright
> Praman extends Playwright with deep SAP UI5 awareness — typed control proxies, UI5 stability synchronization, FLP navigation, OData operations, Fiori Elements testing, and AI-powered test generation. Single npm package: playwright-praman.
> This file contains the complete Praman documentation (excluding auto-generated API reference).
> For topic-specific context, see llms-quickstart.txt, llms-sap-testing.txt, llms-migration.txt, and llms-architecture.txt.
## Praman
**AI-First SAP UI5 Test Automation Plugin for Playwright.**
Praman extends Playwright with deep SAP UI5 awareness — typed control proxies, UI5 stability
synchronization, FLP navigation, and AI-powered test generation.
## Get Started in 5 Steps
**1. Install**
```bash
npm install playwright-praman
```
First install resolves Playwright and other dependencies — allow 1-2 minutes.
**2. Initialize**
```bash
npx playwright init-agents --loop=vscode
npx playwright-praman init
```
Validates your environment, installs Chromium, detects your IDE, and scaffolds config files, auth setup, and a gold-standard verification test.
**3. Configure SAP Credentials**
```bash
cp .env.example .env
# Edit .env with your SAP_CLOUD_BASE_URL, SAP_CLOUD_USERNAME, SAP_CLOUD_PASSWORD
```
**4. Verify**
```bash
npx playwright test tests/bom-e2e-praman-gold-standard.spec.ts --reporter=line --headed --project=chromium
```
A passing test confirms your setup is complete.
**5. Generate Tests from Your Business Process**
Describe your business process or test case in plain language — Praman's AI agents autonomously generate the test plan and production-ready Playwright test script:
```bash
/praman-sap-coverage
# Then enter: "Test creating a purchase order with vendor 1000, material MAT-001, quantity 10"
```
The **plan → generate → heal** pipeline discovers live UI5 controls, generates typed Playwright tests, and self-heals failures — no manual test scripting required.
See the full [Getting Started](./guides/getting-started.md) guide for a detailed walkthrough.
```typescript
test('discover a UI5 control', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
const button = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create' } });
const text = await button.getText();
expect(text).toBe('Create');
});
```
## Documentation
**Setup & Onboarding**
- [Getting Started](./guides/getting-started.md) — install, configure, write your first test
- [Authentication Guide](./guides/authentication.md) — 6 SAP auth strategies
- [Agent & IDE Setup](./guides/agent-setup.md) — AI agents, seed file, and IDE configs installed by `init`
**Your Background** — start from what you already know:
[From Playwright](./guides/migration-from-playwright.md) | [From Selenium](./guides/migration-from-selenium.md) | [From wdi5](./guides/migration-from-wdi5.md) | [From Tosca](./guides/migration-from-tosca.md) | [For Business Analysts](./guides/sap-business-analyst-guide.md)
**Core References**
- [Configuration Reference](./guides/configuration.md) — all config options with defaults
- [Fixture Reference](./guides/fixtures.md) — all 12 fixture modules
- [Selector Reference](./guides/selectors.md) — `UI5Selector` fields and examples
- [Error Reference](./guides/errors.md) — 60 error codes with recovery suggestions
- [Feature Inventory](./guides/capabilities.md) — complete feature overview
> **Not sure where to start?** Visit the [Personas](/personas) page to find your role-specific entry point.
API documentation is auto-generated from source code TSDoc comments — see the **API Reference** in the navbar.
## LLM-Friendly Docs
Praman publishes documentation in the [llmstxt.org](https://llmstxt.org) standard for AI agents, RAG pipelines, and LLM tools:
| File | Content |
| ------------------------------------------------------------------- | ------------------------------------------ |
| [`llms.txt`](https://praman.dev/llms.txt) | Link index — all docs with descriptions |
| [`llms-full.txt`](https://praman.dev/llms-full.txt) | Complete documentation in a single file |
| [`llms-quickstart.txt`](https://praman.dev/llms-quickstart.txt) | Setup, fixtures, selectors, matchers |
| [`llms-sap-testing.txt`](https://praman.dev/llms-sap-testing.txt) | Auth, FLP, OData, Fiori Elements, cookbook |
| [`llms-migration.txt`](https://praman.dev/llms-migration.txt) | Migration from Playwright, wdi5, Tosca |
| [`llms-architecture.txt`](https://praman.dev/llms-architecture.txt) | Architecture, bridge, proxy, ADRs |
These files are regenerated on every build and deployed alongside the site.
---
## Getting Started
Get up and running with Praman in 5 steps.
## Prerequisites
- **Node.js** >= 20
- Access to an SAP UI5 / Fiori application
- SAP credentials (username, password, base URL)
## Step 1: Install Praman
```bash
npm install playwright-praman
# or
yarn add playwright-praman
# or
pnpm add playwright-praman
```
### What gets installed
Praman has 3 direct dependencies and 5 peer dependencies. npm resolves the full dependency tree during install.
| Package | Type | Purpose |
| ------------------------- | --------------- | -------------------------------------------------- |
| `@playwright/test` | Peer (required) | Playwright test runner (>=1.57.0) |
| `commander` | Dependency | CLI framework for `npx playwright-praman` commands |
| `pino` | Dependency | Structured JSON logging |
| `zod` | Dependency | Configuration validation and type-safe schemas |
| `@anthropic-ai/sdk` | Peer (optional) | AI test generation via Claude |
| `openai` | Peer (optional) | AI test generation via OpenAI / Azure OpenAI |
| `@opentelemetry/api` | Peer (optional) | Observability and distributed tracing |
| `@opentelemetry/sdk-node` | Peer (optional) | OpenTelemetry Node.js SDK |
:::info Install time
First install may take 1-2 minutes while npm resolves the Playwright peer dependency and its browser engine binaries. This is a one-time cost — subsequent installs use the npm cache.
:::
## Step 2: Initialize Your Project
First, initialize Playwright's agent loop for your IDE:
```bash
npx playwright init-agents --loop=vscode
```
Then, run the Praman initializer:
```bash
npx playwright-praman init
```
`init` performs 5 automated sub-steps:
### 2a. Validate Environment
Checks that Node.js >= 20, npm is available, and `@playwright/test` >= 1.57 is present. Exits with clear error messages if any check fails.
### 2b. Install Missing Packages
Installs any missing packages (`@playwright/test`, `dotenv`) and runs `npx playwright install chromium` to download the browser binary.
### 2c. Detect Your IDE
Scans for IDE marker files in your project directory and installs the appropriate agent definitions, prompts, and configuration:
| IDE / Agent | Detection Signal | What gets installed |
| ------------------ | ------------------------------------------------------ | ------------------------------------------------------- |
| **VS Code** | `.vscode/` or `TERM_PROGRAM=vscode` | Settings, extensions, code snippets |
| **Claude Code** | `CLAUDE.md` | Agents (planner, generator, healer), prompts, seed file |
| **Cursor** | `.cursor/` or `.cursorrc` | `.cursor/rules/praman.mdc` |
| **Jules** | `.jules/` | `.jules/praman-setup.md` |
| **GitHub Copilot** | `.github/copilot-instructions.md` or `.github/agents/` | Copilot agent definitions |
| **OpenCode** | `.opencode/` | OpenCode configuration |
### 2d. Scaffold Project Files
Creates the full project structure — no manual file creation needed:
| File | Purpose |
| -------------------------------------------- | -------------------------------------------------------------- |
| `playwright.config.ts` | Playwright config with `auth-setup` + `chromium` projects |
| `praman.config.ts` | Praman settings (timeouts, log level, interaction strategy) |
| `tsconfig.json` | TypeScript config with `module: "Node16"` |
| `.gitignore` | Excludes `.env`, `.auth/`, `test-results/`, `node_modules/` |
| `tests/auth.setup.ts` | SAP login — runs once before tests, saves session to `.auth/` |
| `tests/bom-e2e-praman-gold-standard.spec.ts` | Gold-standard verification test |
| `specs/bom-create-complete.plan.md` | AI-generated test plan (reference) |
| `.env.example` | Environment variable template |
| IDE-specific files | Agent definitions, prompts, rules, snippets (per IDE detected) |
| `skills/` | Praman SAP testing skill files for AI agents |
### 2e. Print Next Steps
Displays IDE-specific instructions for appending Praman agent configurations:
```bash
# Claude Code
cat node_modules/playwright-praman/docs/user-integration/claude-md-appendable.md >> CLAUDE.md
# GitHub Copilot
cat node_modules/playwright-praman/docs/user-integration/copilot-instructions-appendable.md >> .github/copilot-instructions.md
# Cursor
cat node_modules/playwright-praman/docs/user-integration/cursor-rules-appendable.mdc >> .cursorrules
```
:::tip Re-running init
`init` is safe to re-run — it skips files that already exist. Use `--force` to overwrite everything:
```bash
npx playwright-praman init --force
```
For detailed breakdown of every installed file, see the [Agent & IDE Setup](./agent-setup) guide.
:::
## Step 3: Configure SAP Credentials
`init` created a `.env.example` file. Copy it and fill in your SAP credentials:
```bash
cp .env.example .env
```
Edit `.env`:
```bash
# Required — SAP system connection
SAP_CLOUD_BASE_URL=https://your-sap-system.example.com
SAP_CLOUD_USERNAME=your-username
SAP_CLOUD_PASSWORD=your-password
# Authentication strategy
SAP_AUTH_STRATEGY=btp-saml # 'btp-saml' | 'basic' | 'office365'
# Optional
SAP_CLIENT=100 # SAP client number (OnPrem only)
SAP_LANGUAGE=EN # Display language (default: EN)
```
:::warning
**Never commit `.env`** — it contains credentials. The `.gitignore` created by `init` already excludes it.
:::
### Environment Variables Reference
| Variable | Required | Description |
| --------------------------- | -------- | ----------------------------------------------- |
| `SAP_CLOUD_BASE_URL` | Yes | SAP BTP or on-premise base URL |
| `SAP_CLOUD_USERNAME` | Yes | SAP login username |
| `SAP_CLOUD_PASSWORD` | Yes | SAP login password |
| `SAP_AUTH_STRATEGY` | Yes | Auth strategy: `btp-saml`, `basic`, `office365` |
| `SAP_CLIENT` | No | SAP client number (default: from system) |
| `SAP_LANGUAGE` | No | Display language (default: EN) |
| `PRAMAN_LOG_LEVEL` | No | Log level: `debug`, `info`, `warn`, `error` |
| `PRAMAN_SKIP_VERSION_CHECK` | No | Set `true` to skip Playwright version check |
For auth strategy details, see the [Authentication](./authentication) guide.
## Step 4: Verify Your Setup
Run the gold-standard BOM test to confirm everything is working:
```bash
npx playwright test tests/bom-e2e-praman-gold-standard.spec.ts --reporter=line --headed --project=chromium
```
### What this command does
| Flag | Purpose |
| -------------------- | ------------------------------------------------------------------------------------------ |
| `--project=chromium` | Runs the `chromium` project, which depends on `auth-setup` (auth runs first automatically) |
| `--headed` | Opens a visible browser so you can watch the test interact with SAP |
| `--reporter=line` | Compact one-line-per-step output |
The test will:
1. Run `auth-setup` first — logs into SAP and saves session to `.auth/sap-session.json`
2. Open the BOM (Bill of Material) Maintenance app via FLP tile navigation
3. Open the Create BOM dialog and verify SmartField controls
4. Exercise Material and Plant value help dialogs (OData-driven)
5. Test BOM Usage dropdown (ComboBox with `setSelectedKey`)
6. Fill the form, click Create, and handle the response gracefully
### Expected output
```text
Running 1 test using 1 worker
✓ tests/bom-e2e-praman-gold-standard.spec.ts:X:Y › BOM End-to-End Flow › Complete BOM Flow (60-120s)
1 passed
```
### What a passing test confirms
A passing gold-standard test verifies your entire setup end-to-end:
- Playwright is installed and Chromium browser is available
- SAP credentials in `.env` are correct and authentication succeeds
- Session storage (`storageState`) saves and restores auth state
- Praman fixtures (`ui5.control()`, `ui5.press()`, `ui5.fill()`, `ui5.waitForUI5()`) interact with live UI5 controls
- Dialog handling with `searchOpenDialogs: true` works
- SmartField, ComboBox, and Value Help table controls are accessible
- OData data binding (`getContextByIndex().getObject()`) returns entity data
### Troubleshooting verification failures
| Symptom | Likely Cause | Fix |
| ---------------------------------------------- | ------------------------------------------ | --------------------------------------------------------- |
| `ERR_BRIDGE_TIMEOUT` | URL is not a UI5 app, or UI5 hasn't loaded | Verify `SAP_CLOUD_BASE_URL` points to the Fiori Launchpad |
| Auth setup fails | Wrong credentials or auth strategy | Check `.env` — try `SAP_AUTH_STRATEGY=basic` for OnPrem |
| `browserType.launch: Executable doesn't exist` | Chromium not installed | Run `npx playwright install chromium` |
| Test times out at tile click | FLP space/tile name differs on your system | Edit `TEST_DATA` constants in the test file |
| `ERR_CONTROL_NOT_FOUND` | Control ID differs on your system | Use `controlType` + `properties` instead of `id` |
:::info Adapt the gold-standard test to your app
The gold-standard test targets the "Maintain Bill Of Material" app. If your SAP system does not have this app, use it as a template: copy the file, change the tile name, control IDs, and test data to match your app. The patterns (auth, navigation, dialog handling, value help) are universal across SAP Fiori apps.
:::
## Step 5: Generate Tests from Your Business Process
Now that your setup is verified, you can describe any business process or test case in plain language — Praman's AI agents will autonomously generate the test plan and production-ready Playwright test script for you.
### How it works
Run the coverage prompt in your AI agent (Claude Code, Copilot, Cursor):
**Claude Code:**
```bash
/praman-sap-coverage
```
Then describe your business process or test case. For example:
> "Test creating a purchase order: navigate to ME21N, enter vendor 1000, add material MAT-001 with quantity 10 in plant 1000, and verify the PO is posted successfully."
Or describe it at a higher level:
> "Test the complete Procure-to-Pay flow: create a purchase requisition, convert it to a purchase order, post goods receipt, and verify the invoice."
### What happens next
Praman's **plan → generate → heal** pipeline runs autonomously:
| Phase | Agent | What it does |
| ------------ | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Plan** | `praman-sap-planner` | Explores your live SAP system, discovers UI5 controls, and produces a structured test plan with steps and expected results |
| **Generate** | `praman-sap-generator` | Converts the test plan into executable Playwright + Praman test code using typed control proxies — not brittle DOM selectors |
| **Heal** | `praman-sap-healer` | Validates the generated test against the live system, fixes any failures, and ensures compliance with Praman's 7 mandatory rules |
The result is a production-ready `.spec.ts` file in your `tests/` directory — no manual test scripting required.
### Example: from description to test
**You enter:**
> "Verify the BOM creation dialog: open Create BOM, select a material and plant via value help, choose Production usage, and submit."
**Praman generates:**
```typescript
test.describe('BOM Creation', () => {
test('Create BOM with material and plant selection', async ({ ui5 }) => {
await test.step('Open Create BOM dialog', async () => {
await ui5.press({ controlType: 'sap.m.Button', properties: { text: 'Create BOM' } });
await ui5.waitForUI5();
});
await test.step('Select material via value help', async () => {
await ui5.press({ id: 'createBOMFragment--material-input-vhi', searchOpenDialogs: true });
// ... discovers and interacts with live controls
});
await test.step('Submit and verify', async () => {
await ui5.press({ id: 'createBOMFragment--OkBtn', searchOpenDialogs: true });
await ui5.waitForUI5();
});
});
});
```
:::tip From business process to Playwright test — autonomously
This is the core value of Praman: you describe **what** to test in business language, and the AI agents handle the **how** — discovering controls, generating code, and healing failures. No manual selector hunting, no brittle DOM queries, no test scripting.
:::
---
## Your First Custom Test
Create `tests/purchase-order.spec.ts`:
```typescript
test('navigate to Purchase Order app and verify table', async ({ ui5, ui5Navigation }) => {
// Step 1: Navigate to the Fiori app
await test.step('Open Purchase Order app', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
// Step 2: Wait for UI5 to stabilize
await test.step('Wait for page load', async () => {
await ui5.waitForUI5();
});
// Step 3: Discover a control by type
await test.step('Find the Create button', async () => {
const createBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Create' },
});
// The control proxy exposes all UI5 methods
const text = await createBtn.getText();
expect(text).toBe('Create');
});
// Step 4: Read table data
await test.step('Verify table has rows', async () => {
const rowCount = await ui5.table.getRowCount('myTable');
expect(rowCount).toBeGreaterThan(0);
});
});
```
## Real-World Example: BOM End-to-End Test
Praman's AI agents follow a **plan → generate → heal** pipeline. The planner explores a live SAP system, discovers controls, and produces a structured test plan. The generator converts the plan into executable Playwright + Praman test code. Below is a real example from an SAP S/4HANA Cloud BOM application.
### AI-Generated Test Plan
The AI planner agent explored the Maintain Bill of Material app and produced this test plan:
bom-create.plan.md (click to expand)
#### Application Overview
SAP S/4HANA Cloud — Maintain Bill of Material (Version 2). Fiori Elements V4 List Report with Create BOM dialog. UI5 Version 1.142.4. System: Partner Demo Customizing LXG/100.
#### 1. BOM Navigation and App Loading
##### 1.1. Navigate to Maintain BOM app from FLP
| # | Step | Expected Result |
| --- | ----------------------------------------------------- | ---------------------------------------------------------------------------- |
| 1 | Verify FLP Home page loaded after login | Page title contains "Home"; Shell Bar visible with "S/4HANA Cloud" |
| 2 | Click "Bills Of Material" tab in FLP space navigation | Tab becomes active; BOM tiles visible |
| 3 | Click "Maintain Bill Of Material (Version 2)" tile | URL changes to `#MaterialBOM-maintainMaterialBOM` |
| 4 | Wait for List Report to load | Filter bar visible (Material, Plant, BOM Usage); "Create BOM" button enabled |
#### 2. Create BOM Dialog Interactions
##### 2.1. Open Create BOM dialog and verify fields
| # | Step | Expected Result |
| --- | ------------------------- | --------------------------------------------------------------------------------------------------------- |
| 1 | Click "Create BOM" button | Dialog opens with heading "Create BOM" |
| 2 | Verify all dialog fields | Material (required), Plant, BOM Usage (required), Alternative BOM, Change Number, Valid From (pre-filled) |
| 3 | Verify footer buttons | "Create BOM" submit and "Cancel" buttons visible and enabled |
| 4 | Click "Cancel" | Dialog closes; List Report visible |
##### 2.2. Test Material Value Help
| # | Step | Expected Result |
| --- | -------------------------------------- | --------------------------------------------------------------------- |
| 1 | Click "Create BOM" to open dialog | Dialog opens |
| 2 | Click Value Help icon next to Material | "Select: Material" dialog opens with Material and Description columns |
| 3 | Verify material data loaded | At least one row with Material number and description |
| 4 | Click a material row | Value help closes; Material field populated |
| 5 | Click "Cancel" to close dialog | Dialog closes cleanly |
##### 2.3. Test Plant Value Help
| # | Step | Expected Result |
| --- | ----------------------------------- | ----------------------------------------------------------- |
| 1 | Click "Create BOM" to open dialog | Dialog opens |
| 2 | Click Value Help icon next to Plant | "Select: Plant" dialog opens with Plant, Plant Name columns |
| 3 | Verify plant data loaded | Plants listed (e.g. 1010 DE Plant, 1110 GB Plant) |
| 4 | Click a plant row | Value help closes; Plant field populated |
| 5 | Click "Cancel" to close dialog | Dialog closes cleanly |
##### 2.4. Test BOM Usage dropdown
| # | Step | Expected Result |
| --- | --------------------------------- | --------------------------------------------------------------------------------------- |
| 1 | Click "Create BOM" to open dialog | Dialog opens |
| 2 | Click Value Help on BOM Usage | Dropdown opens with usage types |
| 3 | Verify options | Production (1), Engineering/Design (2), Universal (3), Plant Maintenance (4), Sales (5) |
| 4 | Select "Production (1)" | BOM Usage field shows "Production (1)" |
| 5 | Click "Cancel" to close dialog | Dialog closes cleanly |
#### 3. Complete BOM Creation Flow
##### 3.1. Fill all fields and submit
| # | Step | Expected Result |
| --- | --------------------------------------- | ---------------------------------------------------------------------------- |
| 1 | Navigate to app, click "Create BOM" | Dialog opens |
| 2 | Select Material via Value Help | Material field populated |
| 3 | Select Plant via Value Help | Plant field populated |
| 4 | Select BOM Usage "Production (1)" | BOM Usage set |
| 5 | Verify all required fields filled | Material, BOM Usage, Valid From all have values; Create BOM enabled |
| 6 | Click "Create BOM" submit | Success: dialog closes, returns to list. Error: error message dialog appears |
| 7 | If error, close error dialog and cancel | User returns to list report |
##### 3.2. Verify validation errors
| # | Step | Expected Result |
| --- | ------------------------------------------- | -------------------------------------------------------------------- |
| 1 | Open Create BOM dialog | Dialog opens |
| 2 | Select invalid Material + Plant combination | Both fields populated |
| 3 | Select BOM Usage "Production (1)" | BOM Usage set |
| 4 | Click "Create BOM" submit | Error dialog: "Material not maintained in plant" with message M3351 |
| 5 | Click "Close" on error dialog | Error dialog closes; Create BOM dialog remains with values preserved |
| 6 | Click "Cancel" | Dialog closes; returns to list report |
### Generated Gold Standard Test
The AI generator converted scenario **3.1** (Complete BOM Creation Flow) from the plan above into the following production-grade test. Copy it into your project:
```bash
cp node_modules/playwright-praman/examples/bom-e2e-praman-gold-standard.spec.ts tests/
```
Here is the full test — 8 steps covering navigation, dialog handling, value help tables, OData-driven data selection, form fill, and graceful error recovery:
bom-e2e-praman-gold-standard.spec.ts (click to expand)
```typescript
// ── V2 SmartField Control ID Constants ──────────────────────────────
const IDS = {
// ── Dialog fields (sap.ui.comp.smartfield.SmartField) ──
materialField: 'createBOMFragment--material',
materialInput: 'createBOMFragment--material-input',
materialVHIcon: 'createBOMFragment--material-input-vhi',
materialVHDialog: 'createBOMFragment--material-input-valueHelpDialog',
materialVHTable: 'createBOMFragment--material-input-valueHelpDialog-table',
plantField: 'createBOMFragment--plant',
plantInput: 'createBOMFragment--plant-input',
plantVHIcon: 'createBOMFragment--plant-input-vhi',
plantVHDialog: 'createBOMFragment--plant-input-valueHelpDialog',
plantVHTable: 'createBOMFragment--plant-input-valueHelpDialog-table',
bomUsageField: 'createBOMFragment--variantUsage',
bomUsageCombo: 'createBOMFragment--variantUsage-comboBoxEdit',
// ── Dialog buttons ──
okBtn: 'createBOMFragment--OkBtn',
cancelBtn: 'createBOMFragment--CancelBtn',
} as const;
// ── Test Data ───────────────────────────────────────────────────────
const TEST_DATA = {
flpSpaceTab: 'Bills Of Material',
tileHeader: 'Maintain Bill Of Material',
bomUsageKey: '1', // Production
} as const;
test.describe('BOM End-to-End Flow', () => {
test('Complete BOM Flow - V2 SmartField Single Session', async ({ page, ui5 }) => {
// Step 1: Navigate to BOM Maintenance App
await test.step('Step 1: Navigate to BOM Maintenance App', async () => {
await page.goto(process.env.SAP_CLOUD_BASE_URL!);
await page.waitForLoadState('domcontentloaded');
await expect(page).toHaveTitle(/Home/, { timeout: 60000 });
// FLP space tabs — DOM click is the only reliable method
await page.getByText(TEST_DATA.flpSpaceTab, { exact: true }).click();
// Click tile with toPass() retry for slow SAP systems
await expect(async () => {
await ui5.press({
controlType: 'sap.m.GenericTile',
properties: { header: TEST_DATA.tileHeader },
});
}).toPass({ timeout: 60000, intervals: [5000, 10000] });
// Wait for Create BOM button — proves V1 app loaded
let createBtn: Awaited>;
await expect(async () => {
createBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Create BOM' },
});
const text = await createBtn.getProperty('text');
expect(text).toBe('Create BOM');
}).toPass({ timeout: 120000, intervals: [5000, 10000] });
});
// Step 2: Open Create BOM Dialog and Verify Structure
await test.step('Step 2: Open Create BOM Dialog', async () => {
await ui5.press({
controlType: 'sap.m.Button',
properties: { text: 'Create BOM' },
});
await ui5.waitForUI5();
// Verify dialog opened — Material SmartField exists inside dialog
const materialField = await ui5.control({
id: IDS.materialField,
searchOpenDialogs: true,
});
const materialType = await materialField.getControlType();
expect(materialType).toBe('sap.ui.comp.smartfield.SmartField');
expect(await materialField.getRequired()).toBe(true);
// Verify BOM Usage SmartField
const bomUsageField = await ui5.control({
id: IDS.bomUsageField,
searchOpenDialogs: true,
});
expect(await bomUsageField.getControlType()).toBe('sap.ui.comp.smartfield.SmartField');
expect(await bomUsageField.getRequired()).toBe(true);
// Verify dialog footer buttons
const createDialogBtn = await ui5.control({
id: IDS.okBtn,
searchOpenDialogs: true,
});
expect(await createDialogBtn.getProperty('text')).toBe('Create');
const cancelDialogBtn = await ui5.control({
id: IDS.cancelBtn,
searchOpenDialogs: true,
});
expect(await cancelDialogBtn.getProperty('text')).toBe('Cancel');
});
// Step 3: Test Material Value Help
await test.step('Step 3: Test Material Value Help', async () => {
await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true });
const materialDialog = await ui5.control({
id: IDS.materialVHDialog,
searchOpenDialogs: true,
});
expect(await materialDialog.isOpen()).toBe(true);
await ui5.waitForUI5();
// Get inner table via SmartTable.getTable()
const smartTable = await ui5.control({
id: IDS.materialVHTable,
searchOpenDialogs: true,
});
const innerTable = await smartTable.getTable();
const rows = (await innerTable.getRows()) as unknown[];
expect(rows.length).toBeGreaterThan(0);
// Wait for OData data to load
await expect(async () => {
let count = 0;
for (const row of rows) {
const ctx = await (
row as { getBindingContext: () => Promise }
).getBindingContext();
if (ctx) count++;
}
expect(count).toBeGreaterThan(0);
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
await materialDialog.close();
await ui5.waitForUI5();
});
// Step 4: Test Plant Value Help
await test.step('Step 4: Test Plant Value Help', async () => {
await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true });
const plantDialog = await ui5.control({
id: IDS.plantVHDialog,
searchOpenDialogs: true,
});
expect(await plantDialog.isOpen()).toBe(true);
await ui5.waitForUI5();
const smartTable = await ui5.control({
id: IDS.plantVHTable,
searchOpenDialogs: true,
});
const innerTable = await smartTable.getTable();
const rows = (await innerTable.getRows()) as unknown[];
await expect(async () => {
let count = 0;
for (const row of rows) {
const ctx = await (
row as { getBindingContext: () => Promise }
).getBindingContext();
if (ctx) count++;
}
expect(count).toBeGreaterThan(0);
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
await plantDialog.close();
await ui5.waitForUI5();
});
// Step 5: Test BOM Usage Dropdown (Inner ComboBox)
await test.step('Step 5: Test BOM Usage Dropdown', async () => {
const bomUsageCombo = await ui5.control({
id: IDS.bomUsageCombo,
searchOpenDialogs: true,
});
// Get items via proxy
const rawItems = await bomUsageCombo.getItems();
const items: Array<{ key: string; text: string }> = [];
if (Array.isArray(rawItems)) {
for (const itemProxy of rawItems) {
const key = await itemProxy.getKey();
const text = await itemProxy.getText();
items.push({ key: String(key), text: String(text) });
}
}
expect(items.length).toBeGreaterThan(0);
// Verify open/close cycle
await bomUsageCombo.open();
await ui5.waitForUI5();
expect(await bomUsageCombo.isOpen()).toBe(true);
await bomUsageCombo.close();
await ui5.waitForUI5();
expect(await bomUsageCombo.isOpen()).toBe(false);
});
// Step 6: Fill Form with Valid Data
await test.step('Step 6: Fill Form with Valid Data', async () => {
// === FILL MATERIAL ===
await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true });
const materialDialogControl = await ui5.control({
id: IDS.materialVHDialog,
searchOpenDialogs: true,
});
await expect(async () => {
expect(await materialDialogControl.isOpen()).toBe(true);
}).toPass({ timeout: 30000, intervals: [1000, 2000] });
const smartTableMat = await ui5.control({
id: IDS.materialVHTable,
searchOpenDialogs: true,
});
const innerTableMat = await smartTableMat.getTable();
let materialValue = '';
await expect(async () => {
const ctx = await innerTableMat.getContextByIndex(0);
expect(ctx).toBeTruthy();
const data = (await ctx.getObject()) as { Material?: string };
expect(data?.Material).toBeTruthy();
materialValue = data!.Material!;
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
await materialDialogControl.close();
await ui5.waitForUI5();
await ui5.fill({ id: IDS.materialInput, searchOpenDialogs: true }, materialValue);
await ui5.waitForUI5();
// === FILL PLANT ===
await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true });
const plantDialogControl = await ui5.control({
id: IDS.plantVHDialog,
searchOpenDialogs: true,
});
await expect(async () => {
expect(await plantDialogControl.isOpen()).toBe(true);
}).toPass({ timeout: 30000, intervals: [1000, 2000] });
const smartTablePlant = await ui5.control({
id: IDS.plantVHTable,
searchOpenDialogs: true,
});
const innerTablePlant = await smartTablePlant.getTable();
let plantValue = '';
await expect(async () => {
const ctx = await innerTablePlant.getContextByIndex(0);
expect(ctx).toBeTruthy();
const data = (await ctx.getObject()) as { Plant?: string };
expect(data?.Plant).toBeTruthy();
plantValue = data!.Plant!;
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
await plantDialogControl.close();
await ui5.waitForUI5();
await ui5.fill({ id: IDS.plantInput, searchOpenDialogs: true }, plantValue);
await ui5.waitForUI5();
// === FILL BOM USAGE ===
const bomUsageControl = await ui5.control({
id: IDS.bomUsageCombo,
searchOpenDialogs: true,
});
await bomUsageControl.open();
await ui5.waitForUI5();
await bomUsageControl.setSelectedKey(TEST_DATA.bomUsageKey);
await bomUsageControl.fireChange({ value: TEST_DATA.bomUsageKey });
await bomUsageControl.close();
await ui5.waitForUI5();
// === VERIFY ALL VALUES ===
const finalMaterial =
(await ui5.getValue({
id: IDS.materialInput,
searchOpenDialogs: true,
})) ?? '';
const finalPlant =
(await ui5.getValue({
id: IDS.plantInput,
searchOpenDialogs: true,
})) ?? '';
const finalBomUsageCtrl = await ui5.control({
id: IDS.bomUsageCombo,
searchOpenDialogs: true,
});
const finalBomUsageKey = (await finalBomUsageCtrl.getSelectedKey()) ?? '';
expect(finalMaterial).toBe(materialValue);
expect(finalPlant).toBe(plantValue);
expect(finalBomUsageKey).toBe(TEST_DATA.bomUsageKey);
});
// Step 7: Click Create Button and Handle Result
await test.step('Step 7: Click Create and handle result', async () => {
const createBtn = await ui5.control({
id: IDS.okBtn,
searchOpenDialogs: true,
});
expect(await createBtn.getProperty('text')).toBe('Create');
expect(await createBtn.getEnabled()).toBe(true);
await ui5.press({ id: IDS.okBtn, searchOpenDialogs: true });
await ui5.waitForUI5();
// Graceful error recovery — close error dialogs, cancel if needed
let dialogStillOpen = false;
try {
const okBtnCheck = await ui5.control({
id: IDS.okBtn,
searchOpenDialogs: true,
});
dialogStillOpen = (await okBtnCheck.getEnabled()) !== undefined;
} catch {
dialogStillOpen = false;
}
if (dialogStillOpen) {
try {
await ui5.press({ id: IDS.cancelBtn, searchOpenDialogs: true });
await ui5.waitForUI5();
} catch {
// Dialog already closed
}
}
});
// Step 8: Verify Return to BOM List
await test.step('Step 8: Verify return to BOM List Report', async () => {
const createBtn = await ui5.control(
{
controlType: 'sap.m.Button',
properties: { text: 'Create BOM' },
},
{ timeout: 30000 },
);
expect(await createBtn.getProperty('text')).toBe('Create BOM');
expect(await createBtn.getProperty('enabled')).toBe(true);
});
});
});
```
**Key patterns demonstrated:**
| Pattern | Example |
| ----------------------------------- | --------------------------------------------------- |
| `searchOpenDialogs: true` | All controls inside the Create BOM dialog |
| `ui5.fill()` | Atomic `setValue` + `fireChange` + `waitForUI5` |
| `SmartTable.getTable()` | Access inner `sap.ui.table.Table` from SmartTable |
| `getContextByIndex().getObject()` | Read OData entity data from table rows |
| `setSelectedKey()` + `fireChange()` | ComboBox selection (localization-safe) |
| `toPass()` retry | Handle slow OData loads and SAP system latency |
| Graceful error recovery | Close error dialogs and cancel without hard failure |
:::info From plan to code
Both the test plan and the gold standard test above were produced by Praman's AI agents against a live SAP S/4HANA Cloud system. Run the full **plan → generate → heal** pipeline on your own SAP app with the `/praman-sap-coverage` prompt.
:::
## Running Tests
```bash
# Run all tests
npx playwright test
# Run a specific test file
npx playwright test tests/purchase-order.spec.ts
# Run with visible browser
npx playwright test --headed
# Run with Playwright UI mode
npx playwright test --ui
```
:::tip Why `ui5.control()` instead of `page.locator()`?
SAP UI5 renders controls dynamically -- DOM IDs change between versions, themes restructure
elements, and controls nest inside generated wrappers. `ui5.control()` discovers controls through
the **UI5 runtime's own control registry**, which is the stable contract. Your tests survive DOM
restructuring, theme changes, and UI5 version upgrades.
:::
## Import Pattern
Praman uses Playwright's `mergeTests()` to combine all fixtures into a single `test` object:
```typescript
// All fixtures available via destructuring:
test('full access', async ({
ui5, // Control discovery + interaction
ui5Navigation, // FLP navigation
sapAuth, // Authentication
fe, // Fiori Elements helpers
pramanAI, // AI page discovery
intent, // Business domain intents
ui5Shell, // FLP shell header
ui5Footer, // Page footer bar
flpLocks, // SM12 lock management
flpSettings, // User settings
testData, // Test data generation
}) => {
// ...
});
```
## Common Patterns
### Control Discovery
```typescript
// By ID
const btn = await ui5.control({ id: 'submitBtn' });
// By control type + properties
const input = await ui5.control({
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor' },
});
// Multiple controls
const buttons = await ui5.controls({ controlType: 'sap.m.Button' });
```
### Input Fields (Gold Pattern)
Always use `setValue()` + `fireChange()` + `waitForUI5()` together:
```typescript
const input = await ui5.control({ id: 'vendorInput' });
await input.setValue('SUP-001');
await input.fireChange({ value: 'SUP-001' });
await ui5.waitForUI5();
```
Or use the shorthand:
```typescript
await ui5.fill({ id: 'vendorInput' }, 'SUP-001');
```
:::warning
Never use `page.fill()` or `page.type()` for UI5 Input controls. These bypass the UI5 event model, which means OData bindings, value state updates, and field validation will not trigger.
:::
### Table Operations
```typescript
// Read table data
const rows = await ui5.table.getRows('purchaseOrderTable');
const columns = await ui5.table.getColumnNames('purchaseOrderTable');
// Find a row by column values
const rowIndex = await ui5.table.findRowByValues('purchaseOrderTable', {
'Purchase Order': '4500001234',
});
// Click a row
await ui5.table.clickRow('purchaseOrderTable', rowIndex);
// Get a specific cell value
const vendor = await ui5.table.getCellByColumnName('purchaseOrderTable', 0, 'Vendor');
```
### Dialog Handling
```typescript
// Wait for a dialog to appear
const dialog = await ui5.dialog.waitFor();
// Confirm a dialog
await ui5.dialog.confirm();
// Dismiss a dialog
await ui5.dialog.dismiss();
// Find controls inside a dialog
const dialogBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'OK' },
searchOpenDialogs: true,
});
```
:::warning
Always use `searchOpenDialogs: true` when finding controls inside an `sap.m.Dialog`. Without it, `ui5.control()` only searches the main view.
:::
### Navigation
```typescript
// Navigate to a Fiori app by semantic object
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
// Navigate to a tile by title
await ui5Navigation.navigateToTile('Manage Purchase Orders');
// Navigate to a specific URL hash
await ui5Navigation.navigateToHash('#PurchaseOrder-manage&/PurchaseOrders');
// Go back
await ui5Navigation.navigateBack();
// Go to FLP home
await ui5Navigation.navigateToHome();
```
### Fiori Elements Shortcuts
```typescript
// List Report
await fe.listReport.setFilter('Vendor', 'SUP-001');
await fe.listReport.search();
await fe.listReport.navigateToItem(0);
// Object Page
const title = await fe.objectPage.getHeaderTitle();
await fe.objectPage.clickEdit();
await fe.objectPage.navigateToSection('Items');
await fe.objectPage.clickSave();
```
### Business Intent APIs
```typescript
// Create a purchase order using business terms (vocabulary-resolved)
await intent.procurement.createPurchaseOrder({
vendor: 'SUP-001',
material: 'MAT001',
quantity: 10,
plant: '1000',
companyCode: '1000',
purchasingOrg: '1000',
});
// Fill a field using business vocabulary
await intent.core.fillField('Vendor', 'SUP-001');
```
## Fixture Quick Reference
| Fixture | Purpose |
| --------------- | -------------------------------------------------- |
| `ui5` | Core control discovery and interaction |
| `ui5.table` | Table read, filter, sort, select |
| `ui5.dialog` | Dialog lifecycle (wait, confirm, dismiss) |
| `ui5.date` | DatePicker and TimePicker operations |
| `ui5.odata` | OData model reads and HTTP operations |
| `ui5Navigation` | FLP and in-app navigation |
| `ui5Footer` | Footer toolbar buttons (Save, Edit, Cancel) |
| `ui5Shell` | Shell header (Home, Notifications, User menu) |
| `fe` | Fiori Elements List Report, Object Page, Table |
| `intent` | Business intent APIs (procurement, sales, finance) |
Always wrap multi-step flows in `test.step()` for clear reporting:
```typescript
await test.step('Create purchase order', async () => {
// ... multiple interactions
});
```
## Persona Quick-Start Guides
### Test Automation Engineer
Focus on **fixtures, assertions, and `test.step()`** for structured test flows.
All fixtures are available via destructuring from the `test` callback. Use `test.step()` to organize multi-step flows for clear HTML reports.
### AI Agent (Copilot / Claude Code)
Focus on **imports, capabilities, and error codes** for automated test generation.
- **Single import**: `import { test, expect } from 'playwright-praman'`
- **Capability query**: Use `pramanAI.capabilities.forAI()` to discover available operations at runtime
- **Error codes**: All errors extend `PramanError` with `code`, `retryable`, and `suggestions[]`
- **Forbidden patterns**: Never use `page.click('#__...')`, `page.fill('#__...')`, or `page.locator('.sapM...')` for UI5 elements
### SAP Business Analyst
Focus on **intents and vocabulary** for writing tests in business terms.
```typescript
// Instead of finding controls by ID:
await intent.core.fillField('Vendor', 'SUP-001');
await intent.core.fillField('Material', 'MAT001');
await intent.core.fillField('Quantity', '10');
// Or use domain-specific intents:
await intent.procurement.createPurchaseOrder({
vendor: 'SUP-001',
material: 'MAT001',
quantity: 10,
plant: '1000',
});
```
Supported vocabulary domains: procurement (MM), sales (SD), finance (FI), manufacturing (PP), warehouse (WM/EWM), quality (QM).
## Project Structure
```text
my-sap-tests/
tests/
auth-setup.ts # Authentication (runs once)
purchase-order.spec.ts # Your test files
sales-order.spec.ts
.auth/
sap-session.json # Saved session (gitignored)
praman.config.ts
playwright.config.ts
package.json
```
## Next Steps
| Topic | Documentation |
| ------------------------- | --------------------------------------- |
| Configuration reference | [Configuration](./configuration) |
| Authentication strategies | [Authentication](./authentication) |
| Selector reference | [Selectors](./selectors) |
| Fixture reference | [Fixtures](./fixtures) |
| Error reference | [Errors](./errors) |
| Agent & IDE setup | [Agent Setup](./agent-setup) |
| Vocabulary system | [Vocabulary](./vocabulary-system) |
| Intent API | [Intent API](./intent-api) |
| Examples | [Examples](../examples/) |
| Architecture overview | [Architecture](./architecture-overview) |
---
## Configuration Reference
Praman uses a Zod-validated configuration system. All fields have defaults — an empty `{}` is valid.
## Config File
Create `praman.config.ts` in your project root:
```typescript
export default defineConfig({
logLevel: 'info',
ui5WaitTimeout: 30_000,
controlDiscoveryTimeout: 10_000,
interactionStrategy: 'ui5-native',
discoveryStrategies: ['direct-id', 'recordreplay'],
auth: {
strategy: 'basic',
baseUrl: process.env.SAP_BASE_URL!,
username: process.env.SAP_USER!,
password: process.env.SAP_PASS!,
client: '100',
language: 'EN',
},
ai: {
provider: 'azure-openai',
apiKey: process.env.AZURE_OPENAI_KEY,
endpoint: process.env.AZURE_OPENAI_ENDPOINT,
deployment: 'gpt-4o',
},
telemetry: {
openTelemetry: false,
exporter: 'otlp',
endpoint: 'http://localhost:4318',
serviceName: 'sap-e2e-tests',
},
});
```
## Top-Level Options
| Field | Type | Default | Description |
| ------------------------- | ----------------------------------------------------- | ------------------------------- | ---------------------------------------------------- |
| `logLevel` | `'error' \| 'warn' \| 'info' \| 'debug' \| 'verbose'` | `'info'` | Pino log level |
| `ui5WaitTimeout` | `number` | `30_000` | Max ms to wait for UI5 stability |
| `controlDiscoveryTimeout` | `number` | `10_000` | Max ms for control discovery |
| `interactionStrategy` | `'ui5-native' \| 'dom-first' \| 'opa5'` | `'ui5-native'` | How to interact with controls |
| `discoveryStrategies` | `string[]` | `['direct-id', 'recordreplay']` | Ordered strategy chain |
| `skipStabilityWait` | `boolean` | `false` | Skip `waitForUI5Stable()`, use brief DOM settle |
| `preferVisibleControls` | `boolean` | `true` | Prefer visible controls over hidden ones |
| `ignoreAutoWaitUrls` | `string[]` | `[]` | Additional URL patterns to block (WalkMe, analytics) |
## Auth Sub-Schema
| Field | Type | Default | Description |
| ---------- | -------------------------------------------------- | ---------- | ----------------------- |
| `strategy` | `'btp-saml' \| 'basic' \| 'office365' \| 'custom'` | `'basic'` | Authentication strategy |
| `baseUrl` | `string` (URL) | _required_ | SAP system base URL |
| `username` | `string` | — | Login username |
| `password` | `string` | — | Login password |
| `client` | `string` | `'100'` | SAP client number |
| `language` | `string` | `'EN'` | SAP logon language |
## AI Sub-Schema
| Field | Type | Default | Description |
| ----------------- | ------------------------------------------- | ---------------- | ---------------------------------------------------- |
| `provider` | `'azure-openai' \| 'openai' \| 'anthropic'` | `'azure-openai'` | LLM provider |
| `apiKey` | `string` | — | OpenAI / Azure OpenAI API key |
| `anthropicApiKey` | `string` | — | Anthropic API key (separate for security) |
| `model` | `string` | — | Model name (e.g., `'gpt-4o'`, `'claude-sonnet-4-6'`) |
| `temperature` | `number` | `0.3` | LLM temperature (0-2) |
| `maxTokens` | `number` | — | Max tokens per response |
| `endpoint` | `string` (URL) | — | Azure OpenAI resource endpoint |
| `deployment` | `string` | — | Azure OpenAI deployment name |
| `apiVersion` | `string` | — | Azure API version |
## Telemetry Sub-Schema
| Field | Type | Default | Description |
| --------------- | --------------------------------------- | --------------------- | ---------------------------- |
| `openTelemetry` | `boolean` | `false` | Enable OpenTelemetry tracing |
| `exporter` | `'otlp' \| 'azure-monitor' \| 'jaeger'` | `'otlp'` | Trace exporter |
| `endpoint` | `string` (URL) | — | Exporter endpoint URL |
| `serviceName` | `string` | `'playwright-praman'` | Service name in traces |
## Selectors Sub-Schema
| Field | Type | Default | Description |
| ----------------------- | --------- | -------- | --------------------------------- |
| `defaultTimeout` | `number` | `10_000` | Default selector timeout (ms) |
| `preferVisibleControls` | `boolean` | `true` | Prefer visible controls |
| `skipStabilityWait` | `boolean` | `false` | Skip stability wait for selectors |
## OPA5 Sub-Schema
| Field | Type | Default | Description |
| -------------------- | --------- | ------- | ----------------------------- |
| `interactionTimeout` | `number` | `5_000` | OPA5 interaction timeout (ms) |
| `autoWait` | `boolean` | `true` | Auto-wait for OPA5 readiness |
| `debug` | `boolean` | `false` | Enable OPA5 debug mode |
## Environment Variable Overrides
Top-level config fields can be overridden via environment variables:
| Environment Variable | Config Field |
| ---------------------------------- | ------------------------- |
| `PRAMAN_LOG_LEVEL` | `logLevel` |
| `PRAMAN_UI5_WAIT_TIMEOUT` | `ui5WaitTimeout` |
| `PRAMAN_CONTROL_DISCOVERY_TIMEOUT` | `controlDiscoveryTimeout` |
| `PRAMAN_INTERACTION_STRATEGY` | `interactionStrategy` |
| `PRAMAN_SKIP_STABILITY_WAIT` | `skipStabilityWait` |
**Precedence** (highest to lowest):
1. Per-call options (e.g., `ui5.control({ ... }, { timeout: 5000 })`)
2. Selectors sub-schema config
3. Top-level config
4. Environment variable overrides
5. Schema defaults
## Complete `playwright.config.ts` Example
```typescript
export default defineConfig({
testDir: './tests/e2e',
timeout: 120_000,
expect: { timeout: 10_000 },
fullyParallel: false,
retries: 1,
workers: 1,
reporter: [
['html', { open: 'never' }],
['playwright-praman/reporters', { type: 'compliance', outputDir: 'reports' }],
],
use: {
baseURL: process.env.SAP_BASE_URL,
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
},
projects: [
{
name: 'setup',
testMatch: /auth-setup\.ts/,
teardown: 'teardown',
},
{
name: 'teardown',
testMatch: /auth-teardown\.ts/,
},
{
name: 'chromium',
dependencies: ['setup'],
use: {
...devices['Desktop Chrome'],
storageState: '.auth/sap-session.json',
},
},
],
});
```
## Validation
Config is validated with Zod at load time. Invalid config fails fast:
```typescript
const config = await loadConfig();
// Throws ConfigError with ERR_CONFIG_INVALID if schema fails
// Returns Readonly — deeply frozen, mutation throws
```
---
## Fixture Reference
Praman provides 12 fixture modules merged into a single `test` object via Playwright's `mergeTests()`.
## Import
```typescript
```
## Fixture Summary
| Fixture | Scope | Type | Description |
| --------------- | ------ | -------------- | ------------------------------------------------------------------------------------- |
| `ui5` | test | Core | Control discovery, interaction, `.table`, `.dialog`, `.date`, `.odata` sub-namespaces |
| `ui5Navigation` | test | Navigation | 9 FLP navigation methods |
| `btpWorkZone` | test | Navigation | Dual-frame BTP WorkZone manager |
| `sapAuth` | test | Authentication | SAP authentication (6 strategies) |
| `fe` | test | Fiori Elements | `.listReport`, `.objectPage`, `.table`, `.list` helpers |
| `pramanAI` | test | AI | Page discovery, agentic handler, LLM, vocabulary |
| `intent` | test | Business | `.procurement`, `.sales`, `.finance`, `.manufacturing`, `.masterData` |
| `ui5Shell` | test | FLP | Shell header (home, user menu) |
| `ui5Footer` | test | FLP | Page footer bar (Save, Edit, Delete, etc.) |
| `flpLocks` | test | FLP | SM12 lock management with auto-cleanup |
| `flpSettings` | test | FLP | User settings reader (language, date format) |
| `testData` | test | Data | Template-based data generation with auto-cleanup |
| `pramanConfig` | worker | Infrastructure | Frozen config (loaded once per worker) |
| `pramanLogger` | test | Infrastructure | Test-scoped pino logger |
| `rootLogger` | worker | Infrastructure | Worker-scoped root logger |
| `tracer` | worker | Infrastructure | OpenTelemetry tracer (NoOp when disabled) |
## Auto-Fixtures
These fixtures fire automatically without being requested in the test signature:
| Auto-Fixture | Scope | Purpose |
| ---------------------- | ------ | --------------------------------------------- |
| `playwrightCompat` | worker | Playwright version compatibility checks |
| `selectorRegistration` | worker | Registers `ui5=` custom selector engine |
| `matcherRegistration` | worker | Registers 10 custom UI5 matchers |
| `requestInterceptor` | test | Blocks WalkMe, analytics, overlay scripts |
| `ui5Stability` | test | Auto-waits for UI5 stability after navigation |
## Core Fixture: `ui5`
The main fixture for control discovery and interaction.
### Control Discovery
```typescript
test('discover controls', async ({ ui5 }) => {
// Single control
const btn = await ui5.control({ id: 'saveBtn' });
// Multiple controls
const buttons = await ui5.controls({ controlType: 'sap.m.Button' });
// Interaction shortcuts
await ui5.click({ id: 'submitBtn' });
await ui5.fill({ id: 'nameInput' }, 'John Doe');
await ui5.select({ id: 'countrySelect' }, 'US');
await ui5.check({ id: 'agreeCheckbox' });
await ui5.clear({ id: 'searchField' });
});
```
### Sub-Namespaces
```typescript
test('sub-namespaces', async ({ ui5 }) => {
// Table operations
const rows = await ui5.table.getRows('poTable');
const count = await ui5.table.getRowCount('poTable');
// Dialog management
await ui5.dialog.waitFor();
await ui5.dialog.confirm();
// Date/time operations
await ui5.date.setDatePicker('startDate', new Date('2026-01-15'));
// OData model access
const data = await ui5.odata.getModelData('/PurchaseOrders');
});
```
## Navigation Fixture: `ui5Navigation`
```typescript
test('navigation', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await ui5Navigation.navigateToTile('Create Purchase Order');
await ui5Navigation.navigateToIntent('PurchaseOrder', 'create', { plant: '1000' });
await ui5Navigation.navigateToHome();
await ui5Navigation.navigateBack();
const hash = await ui5Navigation.getCurrentHash();
});
```
## Auth Fixture: `sapAuth`
```typescript
test('auth control', async ({ sapAuth, page }) => {
await sapAuth.login(page, { url, username, password, strategy: 'basic' });
expect(sapAuth.isAuthenticated()).toBe(true);
// Auto-logout on teardown
});
```
## Fiori Elements Fixture: `fe`
```typescript
test('FE patterns', async ({ fe }) => {
await fe.listReport.setFilter('Status', 'Active');
await fe.listReport.search();
await fe.listReport.navigateToItem(0);
const title = await fe.objectPage.getHeaderTitle();
await fe.objectPage.clickEdit();
await fe.objectPage.clickSave();
});
```
## AI Fixture: `pramanAI`
```typescript
test('AI discovery', async ({ pramanAI }) => {
const context = await pramanAI.discoverPage({ interactiveOnly: true });
const result = await pramanAI.agentic.generateTest(
'Create a purchase order for vendor 100001',
page,
);
});
```
## Intent Fixture: `intent`
```typescript
test('business intents', async ({ intent }) => {
await intent.procurement.createPurchaseOrder({
vendor: '100001',
material: 'MAT-001',
quantity: 10,
plant: '1000',
});
await intent.finance.postVendorInvoice({
vendor: '100001',
amount: 5000,
currency: 'EUR',
});
});
```
## Shell & Footer Fixtures
```typescript
test('shell and footer', async ({ ui5Shell, ui5Footer }) => {
await ui5Shell.expectShellHeader();
await ui5Footer.clickEdit();
// ... edit form fields ...
await ui5Footer.clickSave();
await ui5Shell.clickHome();
});
```
## Lock Management: `flpLocks`
```typescript
test('lock management', async ({ flpLocks }) => {
const count = await flpLocks.getNumberOfLockEntries('TESTUSER');
// Auto-cleanup on teardown
await flpLocks.deleteAllLockEntries('TESTUSER');
});
```
## Test Data: `testData`
```typescript
test('test data', async ({ testData }) => {
const po = testData.generate({
documentNumber: '{{uuid}}',
createdAt: '{{timestamp}}',
vendor: '100001',
});
await testData.save('po-input.json', po);
// Auto-cleanup on teardown
});
```
## Standalone Usage
Individual fixture modules can be imported for lighter setups:
```typescript
// Compose only what you need
const test = mergeTests(coreTest, authTest);
```
---
## Selector Reference
Selectors are the primary way to find UI5 controls on a page. Praman uses `UI5Selector` objects
that query the UI5 runtime's control registry — not the DOM.
## UI5Selector Fields
| Field | Type | Description |
| ------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `controlType` | `string` | Fully qualified UI5 type (e.g., `'sap.m.Button'`). When provided as a string literal, [narrows the return type](/docs/guides/typed-controls) to a control-specific interface. |
| `id` | `string \| RegExp` | Control ID or pattern |
| `viewName` | `string` | Owning view name for scoped discovery |
| `viewId` | `string` | Owning view ID for scoped discovery |
| `properties` | `Record` | Property matchers (key-value pairs) |
| `bindingPath` | `Record` | OData binding path matchers |
| `i18NText` | `Record` | i18n text matchers (translated values) |
| `ancestor` | `UI5Selector` | Parent control must match this selector |
| `descendant` | `UI5Selector` | Child control must match this selector |
| `interaction` | `UI5Interaction` | Sub-control targeting (idSuffix, domChildWith) |
| `searchOpenDialogs` | `boolean` | Also search controls inside open dialogs |
All fields are optional. At least one must be provided.
## Examples
### By ID
```typescript
const button = await ui5.control({ id: 'saveBtn' });
```
### By ID with RegExp
```typescript
const button = await ui5.control({ id: /submit/i });
```
### By Control Type
```typescript
const buttons = await ui5.controls({ controlType: 'sap.m.Button' });
```
:::tip Typed Returns
When you use `controlType` with `ui5.control()` (singular), the return type narrows to the specific
typed interface (e.g., `UI5Button`). See [Typed Control Returns](/docs/guides/typed-controls) for
full details and examples.
:::
### By Type + Properties
```typescript
const input = await ui5.control({
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor' },
});
```
### Scoped to View
```typescript
const field = await ui5.control({
controlType: 'sap.m.Input',
viewName: 'myApp.view.Detail',
properties: { value: '' },
});
```
### By Binding Path
```typescript
const field = await ui5.control({
controlType: 'sap.m.Input',
bindingPath: { value: '/PurchaseOrder/Vendor' },
});
```
### With Ancestor
```typescript
const cellInput = await ui5.control({
controlType: 'sap.m.Input',
ancestor: {
controlType: 'sap.m.Table',
id: 'poTable',
},
});
```
### With Descendant
```typescript
const form = await ui5.control({
controlType: 'sap.ui.layout.form.SimpleForm',
descendant: {
controlType: 'sap.m.Input',
properties: { placeholder: 'Vendor' },
},
});
```
### Interaction Sub-Targeting
```typescript
// Target the arrow button inside a ComboBox
const combo = await ui5.control({
controlType: 'sap.m.ComboBox',
id: 'countrySelect',
interaction: { idSuffix: 'arrow' },
});
```
### Search Inside Dialogs
```typescript
const dialogButton = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'OK' },
searchOpenDialogs: true,
});
```
## Selector String Format
Praman registers a `ui5=` custom selector engine with Playwright, enabling usage in `page.locator()`:
```typescript
const locator = page.locator('ui5={"controlType":"sap.m.Button","properties":{"text":"Save"}}');
await locator.click();
```
## Control Type Cheat Sheet
| Control | `controlType` | Common Properties |
| -------------- | ------------------------------------------- | --------------------------------------------- |
| Button | `sap.m.Button` | `text`, `icon`, `type`, `enabled` |
| Input | `sap.m.Input` | `value`, `placeholder`, `enabled`, `editable` |
| Text | `sap.m.Text` | `text` |
| Label | `sap.m.Label` | `text`, `required` |
| Select | `sap.m.Select` | `selectedKey` |
| ComboBox | `sap.m.ComboBox` | `value`, `selectedKey` |
| CheckBox | `sap.m.CheckBox` | `selected`, `text` |
| DatePicker | `sap.m.DatePicker` | `value`, `dateValue` |
| TextArea | `sap.m.TextArea` | `value`, `rows` |
| Link | `sap.m.Link` | `text`, `href` |
| GenericTile | `sap.m.GenericTile` | `header`, `subheader` |
| Table | `sap.m.Table` | `headerText` |
| SmartField | `sap.ui.comp.smartfield.SmartField` | `value`, `editable` |
| SmartTable | `sap.ui.comp.smarttable.SmartTable` | `entitySet`, `useTablePersonalisation` |
| SmartFilterBar | `sap.ui.comp.smartfilterbar.SmartFilterBar` | `entitySet` |
:::note SmartField Inner Controls
`SmartField` wraps inner controls (Input, ComboBox, etc.). `getControlType()` returns
`sap.ui.comp.smartfield.SmartField`, not the inner control type. Use `properties` matching
to target by value rather than relying on the inner control type.
:::
## Discovery Strategies
Praman uses a multi-strategy chain for control discovery:
1. **LRU Cache** (200 entries, 5s TTL) — instant for repeat lookups
2. **Direct ID** — `sap.ui.getCore().byId(id)` for exact ID matches
3. **RecordReplay** — SAP's RecordReplay API (requires UI5 >= 1.94)
4. **Registry Scan** — full `ElementRegistry` scan as fallback
Configure the strategy chain:
```typescript
export default defineConfig({
discoveryStrategies: ['direct-id', 'recordreplay'],
});
```
---
## UI5 Control Interactions
The `ui5` fixture provides high-level methods for interacting with UI5 controls. Each method
discovers the control, waits for UI5 stability, and performs the action through the configured
interaction strategy.
:::tip Typed Control Returns
When you use `controlType` in your selector, the returned control is typed to the specific interface
(e.g., `UI5Button`, `UI5Input`). This gives you autocomplete for control-specific methods. See
[Typed Control Returns](/docs/guides/typed-controls) for details.
:::
## Interaction Methods
### click
Clicks a UI5 control. Uses the configured interaction strategy (UI5-native `firePress()` by
default, with DOM click fallback).
```typescript
await ui5.click({ id: 'submitBtn' });
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Save' } });
```
### fill
Sets the value of an input control. Fires both `liveChange` and `change` events to ensure
OData model bindings update.
```typescript
await ui5.fill({ id: 'vendorInput' }, '100001');
await ui5.fill({ controlType: 'sap.m.Input', viewName: 'myApp.view.Detail' }, 'New Value');
```
### press
Triggers a press action on a control. Equivalent to `click` but semantically clearer for
buttons, links, and tiles.
```typescript
await ui5.press({ id: 'confirmBtn' });
```
### select
Selects an item in a dropdown (Select, ComboBox, MultiComboBox). Accepts selection by key or
display text.
```typescript
await ui5.select({ id: 'countrySelect' }, 'US');
await ui5.select({ id: 'statusComboBox' }, 'Active');
```
### check / uncheck
Toggles checkbox and switch controls.
```typescript
await ui5.check({ id: 'agreeCheckbox' });
await ui5.uncheck({ id: 'optionalCheckbox' });
```
### clear
Clears the value of an input control and fires change events.
```typescript
await ui5.clear({ id: 'searchField' });
```
### getText
Reads the text property of a control.
```typescript
const label = await ui5.getText({ id: 'headerTitle' });
expect(label).toBe('Purchase Order Details');
```
### getProperty
Reads any property from a UI5 control by name.
```typescript
const enabled = await ui5.getProperty({ id: 'saveBtn' }, 'enabled');
const visible = await ui5.getProperty({ id: 'statusText' }, 'visible');
```
### waitForUI5
Manually triggers a UI5 stability wait. Normally called automatically, but useful after custom
browser-side operations.
```typescript
await ui5.click({ id: 'triggerLongLoad' });
await ui5.waitForUI5(30_000); // Custom timeout in milliseconds
```
## Auto-Waiting Behavior
Every `ui5.*` method calls `waitForUI5Stable()` before executing. This differs from Playwright's
native auto-waiting in a critical way:
| Aspect | Playwright Auto-Wait | Praman Auto-Wait |
| --------------------- | -------------------------------------------- | -------------------------------------------------------- |
| **What it waits for** | DOM actionability (visible, enabled, stable) | UI5 runtime pending operations (XHR, promises, bindings) |
| **Scope** | Single element | Entire UI5 application |
| **Source of truth** | DOM state | `sap.ui.getCore().getUIDirty()` + pending request count |
| **When it fires** | Before each locator action | Before each `ui5.*` call |
Playwright's auto-wait checks whether a specific DOM element is visible and enabled. Praman's
auto-wait checks whether the entire UI5 framework has finished all asynchronous operations —
this includes OData requests triggered by other controls, model binding updates, and view
rendering.
### The Three-Tier Wait
When a page navigation occurs, Praman executes a three-tier wait:
1. **waitForUI5Bootstrap** (60s) — waits for `window.sap.ui.getCore` to exist
2. **waitForUI5Stable** (15s) — waits for zero pending async operations
3. **briefDOMSettle** (500ms) — brief pause for final DOM rendering
For subsequent interactions (not navigation), only `waitForUI5Stable` is called.
### Skipping the Stability Wait
For performance-critical paths where you know UI5 is already stable:
```typescript
export default defineConfig({
skipStabilityWait: true, // Uses briefDOMSettle (500ms) instead
});
```
## Complete Example
```typescript
test('edit and save a purchase order', async ({ ui5, ui5Navigation }) => {
// Navigate to the app
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
// Click the first row to open details
await ui5.click({
controlType: 'sap.m.ColumnListItem',
ancestor: { id: 'poTable' },
});
// Edit mode
await ui5.click({ id: 'editBtn' });
// Fill fields
await ui5.fill({ id: 'vendorInput' }, '100002');
await ui5.select({ id: 'purchOrgSelect' }, '1000');
await ui5.check({ id: 'urgentCheckbox' });
// Clear and refill
await ui5.clear({ id: 'noteField' });
await ui5.fill({ id: 'noteField' }, 'Updated via automated test');
// Read values back
const vendor = await ui5.getText({ id: 'vendorInput' });
expect(vendor).toBe('100002');
// Save
await ui5.click({ id: 'saveBtn' });
});
```
## Interaction Under the Hood
Each interaction method follows this sequence:
1. **Stability guard** — `waitForUI5Stable()` ensures no pending async operations
2. **Control discovery** — finds the control via the multi-strategy chain (cache, direct-ID,
RecordReplay, registry scan)
3. **Strategy execution** — delegates to the configured interaction strategy (ui5-native,
dom-first, or opa5)
4. **Step decoration** — wraps the entire operation in a Playwright `test.step()` for HTML
report visibility
If the primary strategy method fails (e.g., `firePress()` not available), the strategy
automatically falls back to its secondary method (e.g., DOM click).
---
## Typed Control Returns
When you call `ui5.control()` with a `controlType` string, Praman narrows the return type to a
control-specific TypeScript interface. Instead of a generic `UI5ControlBase` with 10 base methods,
you get the exact API surface for that control — autocomplete for every getter, setter, and event
method.
This works for **all 199 SAP UI5 control types** that Praman ships typed interfaces for.
## How It Works
```typescript
test('typed control example', async ({ ui5 }) => {
// Typed: returns UI5Button with getText(), press(), getEnabled(), getIcon(), ...
const btn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
const text = await btn.getText(); // string — full autocomplete
const enabled = await btn.getEnabled(); // boolean — not `any`
const icon = await btn.getIcon(); // string
// Untyped: returns UI5ControlBase (existing behavior, still works)
const ctrl = await ui5.control({ id: 'myControl' });
const value = await ctrl.getProperty('value'); // unknown
});
```
When `controlType` is provided as a **string literal** (e.g., `'sap.m.Button'`), TypeScript infers
the specific return type. When only `id` or other selectors are used, the return type falls back to
`UI5ControlBase` — exactly the same behavior as before.
:::tip Zero Runtime Change
This is purely a TypeScript type-level feature. The runtime behavior is identical — the same proxy
object is returned. The difference is that your IDE and AI agents now know the exact methods available.
:::
## Why This Matters
### For Test Automation Engineers
Without typed returns, `ui5.control()` returns `UI5ControlBase` with only 10 base methods visible
in autocomplete. Every control-specific method like `getItems()`, `getSelectedKey()`, or `getRows()`
is hidden behind a `[method: string]: (...args) => Promise` index signature.
With typed returns, you see the **full API surface** — 15 to 40+ methods depending on the control:
| Control Type | Methods Available | Examples |
| ----------------------------------- | ----------------- | ----------------------------------------------------------------------- |
| `sap.m.Button` | 20+ | `getText()`, `getIcon()`, `getEnabled()`, `getType()`, `press()` |
| `sap.m.Input` | 30+ | `getValue()`, `getPlaceholder()`, `getValueState()`, `getDescription()` |
| `sap.m.Table` | 40+ | `getItems()`, `getColumns()`, `getMode()`, `getHeaderText()` |
| `sap.m.Select` | 25+ | `getSelectedKey()`, `getSelectedItem()`, `getItems()`, `getEnabled()` |
| `sap.m.Dialog` | 25+ | `getTitle()`, `getButtons()`, `getContent()`, `isOpen()` |
| `sap.ui.comp.smarttable.SmartTable` | 35+ | `getEntitySet()`, `getTable()`, `rebindTable()` |
| `sap.ui.comp.smartfield.SmartField` | 25+ | `getValue()`, `getEditable()`, `getEntitySet()` |
| `sap.ui.mdc.FilterField` | 20+ | `getConditions()`, `getOperators()`, `getValueHelp()` |
### For AI Agents
AI agents (Claude, Copilot, Cursor, Codex) use TypeScript type information as **grounding data**.
When the type system says `UI5Button` has a `getText()` method that returns `Promise`, the
agent uses that — not hallucinated method names.
Without typed returns, agents see:
```
control: UI5ControlBase
.getId() → Promise
.getVisible() → Promise
.getProperty(name) → Promise
... 7 more base methods
[method: string]: (...args: any[]) → Promise ← agents guess from here
```
With typed returns, agents see:
```
btn: UI5Button
.getText() → Promise
.getIcon() → Promise
.getEnabled() → Promise
.getType() → Promise
.press() → Promise
.getAriaDescribedBy() → Promise
... 15+ more typed methods
```
This eliminates an entire class of agent errors — calling methods that don't exist, passing wrong
argument types, or misinterpreting return types.
## Supported Controls
Praman ships typed interfaces for **199 SAP UI5 controls** from the following namespaces:
| Namespace | Controls | Examples |
| ----------------- | -------- | ---------------------------------------------------------------------- |
| `sap.m.*` | 80+ | Button, Input, Table, Dialog, Select, ComboBox, List, Page, Panel, ... |
| `sap.ui.comp.*` | 15+ | SmartField, SmartTable, SmartFilterBar, SmartChart, SmartForm, ... |
| `sap.ui.mdc.*` | 10+ | FilterField, FilterBar, Table, ValueHelp, ... |
| `sap.f.*` | 10+ | DynamicPage, SemanticPage, FlexibleColumnLayout, Avatar, Card, ... |
| `sap.ui.layout.*` | 5+ | Grid, VerticalLayout, HorizontalLayout, Splitter, ... |
| `sap.tnt.*` | 5+ | NavigationList, SideNavigation, ToolPage, InfoLabel, ... |
| `sap.ui.table.*` | 5+ | Table, TreeTable, Column, Row, ... |
| `sap.uxap.*` | 5+ | ObjectPageLayout, ObjectPageSection, ObjectPageSubSection, ... |
The full list is available in the [API Reference](/docs/api).
## Usage Patterns
### Button Interactions
```typescript
test('button operations', async ({ ui5 }) => {
const saveBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
// All methods are typed — IDE shows autocomplete
expect(await saveBtn.getEnabled()).toBe(true);
expect(await saveBtn.getType()).toBe('Emphasized');
expect(await saveBtn.getIcon()).toBe('sap-icon://save');
await saveBtn.press();
});
```
### Input Fields
```typescript
test('input field operations', async ({ ui5 }) => {
const vendorInput = await ui5.control({
controlType: 'sap.m.Input',
viewName: 'app.view.Detail',
properties: { placeholder: 'Enter vendor' },
});
// Typed: getValue() returns Promise, not Promise
const currentValue = await vendorInput.getValue();
expect(currentValue).toBe('');
// Typed: getValueState() returns Promise
const state = await vendorInput.getValueState();
expect(state).toBe('None');
await vendorInput.enterText('100001');
});
```
### Table with Items
```typescript
test('table operations', async ({ ui5 }) => {
const table = await ui5.control({
controlType: 'sap.m.Table',
id: /poTable/,
});
// Typed: getItems() returns Promise
const items = await table.getItems();
expect(items.length).toBeGreaterThan(0);
// Typed: getMode() returns Promise
const mode = await table.getMode();
expect(mode).toBe('SingleSelectMaster');
// Typed: getHeaderText() returns Promise
const header = await table.getHeaderText();
expect(header).toContain('Purchase Orders');
});
```
### Select / ComboBox
```typescript
test('dropdown selection', async ({ ui5 }) => {
const select = await ui5.control({
controlType: 'sap.m.Select',
id: 'purchOrgSelect',
});
// Typed: getSelectedKey() returns Promise
const currentKey = await select.getSelectedKey();
expect(currentKey).toBe('1000');
// Typed: getItems() returns Promise
const options = await select.getItems();
expect(options.length).toBeGreaterThanOrEqual(3);
});
```
### SmartField (Fiori Elements)
```typescript
test('smart field in object page', async ({ ui5 }) => {
const smartField = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/CompanyCode' },
});
// Typed: getEditable() returns Promise
const editable = await smartField.getEditable();
// Typed: getValue() returns Promise
const value = await smartField.getValue();
expect(value).toBe('1000');
});
```
### SmartTable with Filters
```typescript
test('smart table entity set', async ({ ui5 }) => {
const smartTable = await ui5.control({
controlType: 'sap.ui.comp.smarttable.SmartTable',
id: /listReport/,
});
// Typed: getEntitySet() returns Promise
const entitySet = await smartTable.getEntitySet();
expect(entitySet).toBe('C_PurchaseOrderItem');
// Typed: getTable() returns proxied inner table
const innerTable = await smartTable.getTable();
});
```
### Dialog Handling
```typescript
test('dialog with typed access', async ({ ui5 }) => {
// Trigger a dialog
await ui5.click({ id: 'deleteBtn' });
const dialog = await ui5.control({
controlType: 'sap.m.Dialog',
searchOpenDialogs: true,
});
// Typed: getTitle() returns Promise
const title = await dialog.getTitle();
expect(title).toContain('Confirm');
// Typed: getButtons() returns Promise
const buttons = await dialog.getButtons();
expect(buttons.length).toBe(2);
});
```
### MDC FilterField (S/4HANA Cloud)
```typescript
test('mdc filter field', async ({ ui5 }) => {
const filterField = await ui5.control({
controlType: 'sap.ui.mdc.FilterField',
properties: { label: 'Company Code' },
});
// Typed: getConditions() returns Promise
const conditions = await filterField.getConditions();
expect(conditions).toHaveLength(0);
// Typed: getOperators() returns Promise
const operators = await filterField.getOperators();
expect(operators).toContain('EQ');
});
```
## Combining with Selectors
Typed returns work with all [selector fields](/docs/guides/selectors) — `id`, `viewName`,
`properties`, `bindingPath`, `ancestor`, `descendant`, and `searchOpenDialogs`. The only
requirement is that `controlType` is present as a string literal:
```typescript
// All of these return typed UI5Input:
await ui5.control({ controlType: 'sap.m.Input', id: 'vendorInput' });
await ui5.control({ controlType: 'sap.m.Input', viewName: 'app.view.Detail' });
await ui5.control({ controlType: 'sap.m.Input', bindingPath: { value: '/Vendor' } });
await ui5.control({ controlType: 'sap.m.Input', ancestor: { id: 'form1' } });
await ui5.control({ controlType: 'sap.m.Input', properties: { placeholder: 'Enter vendor' } });
```
## Importing the Type Map
For advanced use cases (utility functions, generic test helpers), you can import `UI5ControlMap`
directly:
```typescript
// Extract a specific control type
type MyButton = UI5ControlMap['sap.m.Button'];
type MyInput = UI5ControlMap['sap.m.Input'];
// Use in generic helpers
async function getButtonText(
ui5: { control: (s: any) => Promise },
text: string,
): Promise {
const btn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text },
});
return btn.getText() as Promise;
}
```
## Fallback Behavior
When `controlType` is **not** provided, or is a **dynamic string variable** (not a literal), the
return type is `UI5ControlBase` — the same 10 base methods plus the `[method: string]` index
signature:
```typescript
// Falls back to UI5ControlBase (no controlType)
const ctrl = await ui5.control({ id: 'someControl' });
// Falls back to UI5ControlBase (dynamic string, not literal)
const type = getControlType(); // returns string
const ctrl2 = await ui5.control({ controlType: type });
// To force typing with a dynamic string, use `as const`:
const ctrl3 = await ui5.control({ controlType: 'sap.m.Button' as const });
```
:::info Backwards Compatible
All existing tests continue to work without any changes. The fallback to `UI5ControlBase` ensures
that ID-based selectors, RegExp selectors, and dynamic controlType strings compile and behave
exactly as before.
:::
## Custom / Third-Party Controls
For SAP controls not in the built-in type map (custom controls, partner extensions), the return
type falls back to `UI5ControlBase`. You can still call any method — they just won't have
autocomplete:
```typescript
// Custom control: falls back to UI5ControlBase
const custom = await ui5.control({ controlType: 'com.mycompany.CustomInput' });
// Methods still work at runtime (dynamic proxy), just no autocomplete
const value = await custom.getValue(); // type is Promise
```
## Full Real-World Example
A complete test using typed controls in a SAP Manage Purchase Orders scenario:
```typescript
test('verify purchase order details', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await test.step('Open first purchase order', async () => {
const table = await ui5.control({
controlType: 'sap.m.Table',
id: /poTable/,
});
const items = await table.getItems();
expect(items.length).toBeGreaterThan(0);
await ui5.click({
controlType: 'sap.m.ColumnListItem',
ancestor: { id: /poTable/ },
});
});
await test.step('Verify header fields', async () => {
const vendorField = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/Supplier' },
});
const vendor = await vendorField.getValue();
expect(vendor).toBeTruthy();
const statusField = await ui5.control({
controlType: 'sap.m.ObjectStatus',
id: /objectPageHeaderStatus/,
});
const status = await statusField.getText();
expect(['Draft', 'Active', 'Approved']).toContain(status);
});
await test.step('Check item table', async () => {
const itemTable = await ui5.control({
controlType: 'sap.ui.table.Table',
id: /itemTable/,
});
const rowCount = await itemTable.getVisibleRowCount();
expect(rowCount).toBeGreaterThan(0);
});
await test.step('Edit and save', async () => {
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Edit' },
});
await ui5.fill(
{
controlType: 'sap.m.Input',
id: /noteInput/,
},
'Updated by automated test',
);
const saveBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
expect(await saveBtn.getEnabled()).toBe(true);
await saveBtn.press();
});
});
```
---
## Custom Matchers
Praman extends Playwright's `expect()` with 10 UI5-specific matchers that query control state
directly from the UI5 runtime. These matchers check the **UI5 model/control state**, not the
DOM — which is the source of truth in SAP applications.
## Key Difference From Playwright Matchers
Standard Playwright matchers inspect DOM attributes:
```typescript
// Playwright: checks DOM attribute
await expect(page.locator('#saveBtn')).toBeEnabled();
```
Praman matchers query the UI5 control API:
```typescript
// Praman: checks UI5 control state via sap.ui.getCore().byId()
await expect(page).toBeUI5Enabled('saveBtn');
```
Why does this matter? A control can appear enabled in the DOM but be disabled in the UI5 model
(e.g., during a pending OData operation). The UI5 state is the source of truth.
## Important: Matchers Receive controlId, Not Locator
Unlike Playwright's built-in matchers that operate on `Locator` objects, Praman matchers receive
a **control ID string** as their first argument and are called on `page`:
```typescript
// Correct — pass string controlId to page expectation
await expect(page).toHaveUI5Text('headerTitle', 'Purchase Order');
// Wrong — these are NOT locator-based matchers
// await expect(locator).toHaveUI5Text('Purchase Order'); // Does not work
```
## All 10 Matchers
### toHaveUI5Text
Asserts the `text` property of a control matches the expected value. Supports both exact string
and RegExp matching.
```typescript
await expect(page).toHaveUI5Text('headerTitle', 'Purchase Order Details');
await expect(page).toHaveUI5Text('headerTitle', /Purchase Order/);
```
**Error message preview:**
```
Expected: "Purchase Order Details"
Received: "Sales Order Details"
Control: headerTitle (sap.m.Title)
```
### toBeUI5Visible
Asserts the control's `visible` property is `true`.
```typescript
await expect(page).toBeUI5Visible('saveBtn');
```
**Negation:**
```typescript
await expect(page).not.toBeUI5Visible('hiddenPanel');
```
### toBeUI5Enabled
Asserts the control's `enabled` property is `true`.
```typescript
await expect(page).toBeUI5Enabled('submitBtn');
```
**Negation:**
```typescript
await expect(page).not.toBeUI5Enabled('disabledInput');
```
### toHaveUI5Property
Asserts any named property of a control matches the expected value.
```typescript
await expect(page).toHaveUI5Property('vendorInput', 'value', '100001');
await expect(page).toHaveUI5Property('saveBtn', 'type', 'Emphasized');
await expect(page).toHaveUI5Property('statusIcon', 'src', /accept/);
```
### toHaveUI5ValueState
Asserts the control's `valueState` property matches one of the UI5 value states.
```typescript
await expect(page).toHaveUI5ValueState('vendorInput', 'Error');
await expect(page).toHaveUI5ValueState('amountField', 'Success');
await expect(page).toHaveUI5ValueState('noteInput', 'None');
```
Valid value states: `'None'`, `'Success'`, `'Warning'`, `'Error'`, `'Information'`.
### toHaveUI5Binding
Asserts the control has an OData binding at the specified path.
```typescript
await expect(page).toHaveUI5Binding('vendorField', '/PurchaseOrder/Vendor');
await expect(page).toHaveUI5Binding('priceField', '/PO/NetAmount');
```
### toBeUI5ControlType
Asserts the control's fully qualified type name.
```typescript
await expect(page).toBeUI5ControlType('saveBtn', 'sap.m.Button');
await expect(page).toBeUI5ControlType('mainTable', 'sap.ui.table.Table');
```
### toHaveUI5RowCount
Asserts the number of rows in a table control.
```typescript
await expect(page).toHaveUI5RowCount('poTable', 5);
await expect(page).toHaveUI5RowCount('emptyTable', 0);
```
### toHaveUI5CellText
Asserts the text content of a specific table cell by row and column index (both zero-based).
```typescript
await expect(page).toHaveUI5CellText('poTable', 0, 0, '4500000001');
await expect(page).toHaveUI5CellText('poTable', 0, 2, 'Active');
await expect(page).toHaveUI5CellText('poTable', 1, 3, /pending/i);
```
### toHaveUI5SelectedRows
Asserts which row indices are currently selected in a table.
```typescript
await expect(page).toHaveUI5SelectedRows('poTable', [0, 2]);
await expect(page).toHaveUI5SelectedRows('poTable', []); // No selection
```
## Auto-Retry Mechanism
All Praman matchers are registered as async custom matchers. They query the UI5 runtime via
`page.evaluate()` on each poll iteration. Playwright's `expect()` auto-retries failed assertions
until the configured timeout (default 5 seconds).
This means you do not need manual retry loops:
```typescript
// This auto-retries for up to 5 seconds:
await expect(page).toHaveUI5Text('statusField', 'Approved');
```
For longer waits, increase the timeout:
```typescript
await expect(page).toHaveUI5Text('statusField', 'Approved', { timeout: 15_000 });
```
## Using expect.poll() With UI5 Methods
For custom assertions beyond the 10 built-in matchers, use `expect.poll()` with `ui5.*` methods:
```typescript
// Poll until the control value matches
await expect
.poll(async () => ui5.getProperty({ id: 'vendorInput' }, 'value'), { timeout: 10_000 })
.toBe('100001');
// Poll for table row count
await expect
.poll(async () => ui5.table.getRowCount('poTable'), { timeout: 15_000 })
.toBeGreaterThan(0);
```
## Comparison With Playwright Built-In Matchers
| Matcher Type | Source of Truth | Auto-Retry | Needs Locator |
| -------------------------- | ------------------------ | ---------- | ----------------------- |
| Playwright `toBeEnabled()` | DOM `disabled` attribute | Yes | Yes (`Locator`) |
| Praman `toBeUI5Enabled()` | UI5 `getEnabled()` API | Yes | No (string `controlId`) |
| Playwright `toHaveText()` | DOM `textContent` | Yes | Yes (`Locator`) |
| Praman `toHaveUI5Text()` | UI5 `getText()` API | Yes | No (string `controlId`) |
| Playwright `toBeVisible()` | DOM intersection/display | Yes | Yes (`Locator`) |
| Praman `toBeUI5Visible()` | UI5 `getVisible()` API | Yes | No (string `controlId`) |
Use Praman matchers when you need to check the UI5 model state. Use Playwright matchers when you
need to verify DOM-level rendering (e.g., CSS visibility, element position).
## Complete Example
```typescript
test('validate purchase order form', async ({ ui5, page }) => {
// Check form state
await expect(page).toBeUI5Visible('vendorInput');
await expect(page).toBeUI5Enabled('vendorInput');
// Fill and validate
await ui5.fill({ id: 'vendorInput' }, 'INVALID');
await expect(page).toHaveUI5ValueState('vendorInput', 'Error');
await ui5.fill({ id: 'vendorInput' }, '100001');
await expect(page).toHaveUI5ValueState('vendorInput', 'None');
await expect(page).toHaveUI5Text('vendorName', 'Acme Corp');
// Check binding
await expect(page).toHaveUI5Binding('vendorInput', '/PurchaseOrder/Vendor');
// Verify table
await expect(page).toHaveUI5RowCount('itemTable', 3);
await expect(page).toHaveUI5CellText('itemTable', 0, 0, 'MAT-001');
await expect(page).toHaveUI5CellText('itemTable', 0, 1, '10');
// Check button states
await expect(page).toBeUI5Enabled('saveBtn');
await expect(page).not.toBeUI5Enabled('deleteBtn');
await expect(page).toBeUI5ControlType('saveBtn', 'sap.m.Button');
});
```
## Registration
Matchers are auto-registered via the `matcherRegistration` worker-scoped auto-fixture. No setup
code is needed — just import `{ expect }` from `playwright-praman` and the matchers are available.
```typescript
// All 10 matchers are ready to use
```
---
## Authentication Guide
Praman supports 6 pluggable authentication strategies for SAP systems.
## Strategies
| Strategy | Use Case | Config Value |
| ---------------- | --------------------------------------------- | ---------------- |
| **On-Premise** | On-premise SAP NetWeaver / S/4HANA | `'onprem'` |
| **Cloud SAML** | SAP BTP with SAML IdP (IAS, Azure AD) | `'cloud-saml'` |
| **Office 365** | Microsoft SSO for SAP connected to Azure | `'office365'` |
| **Multi-Tenant** | SAP BTP multi-tenant apps with subdomain auth | `'multi-tenant'` |
| **API** | API key or OAuth bearer token (headless) | `'api'` |
| **Certificate** | Client certificate authentication | `'certificate'` |
## Setup Project Pattern
The recommended pattern uses Playwright's **setup projects** to authenticate once and share the session across all tests.
### 1. Auth Setup File
Create `tests/auth-setup.ts` (or use Praman's built-in `src/auth/auth-setup.ts`):
```typescript
const authFile = join(process.cwd(), '.auth', 'sap-session.json');
setup('SAP authentication', async ({ page, context }) => {
const { SAPAuthHandler } = await import('playwright-praman/auth');
const { createAuthStrategy } = await import('playwright-praman/auth');
const strategy = createAuthStrategy({
url: process.env.SAP_BASE_URL ?? '',
username: process.env.SAP_ONPREM_USERNAME ?? '',
password: process.env.SAP_ONPREM_PASSWORD ?? '',
strategy: 'onprem',
});
const handler = new SAPAuthHandler({ strategy });
await handler.login(page, {
url: process.env.SAP_BASE_URL ?? '',
username: process.env.SAP_ONPREM_USERNAME ?? '',
password: process.env.SAP_ONPREM_PASSWORD ?? '',
client: process.env.SAP_CLIENT,
language: process.env.SAP_LANGUAGE,
});
await context.storageState({ path: authFile });
});
```
### 2. Playwright Config
```typescript
export default defineConfig({
projects: [
{
name: 'setup',
testMatch: /auth-setup\.ts/,
teardown: 'teardown',
},
{
name: 'teardown',
testMatch: /auth-teardown\.ts/,
},
{
name: 'chromium',
dependencies: ['setup'],
use: {
...devices['Desktop Chrome'],
storageState: '.auth/sap-session.json',
},
},
],
});
```
### 3. Tests Use Saved Session
```typescript
// No login code needed — storageState handles it
test('navigate after auth', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
```
## Strategy Examples
### On-Premise (`onprem`)
```bash
# .env
SAP_ACTIVE_SYSTEM=onprem
SAP_ONPREM_BASE_URL=https://sapserver.example.com:8443/sap/bc/ui5_ui5/
SAP_ONPREM_USERNAME=TESTUSER
SAP_ONPREM_PASSWORD=secret
SAP_CLIENT=100
SAP_LANGUAGE=EN
```
### Cloud SAML (`cloud-saml`)
```bash
# .env
SAP_ACTIVE_SYSTEM=cloud
SAP_CLOUD_BASE_URL=https://tenant.launchpad.cfapps.eu10.hana.ondemand.com
SAP_CLOUD_USERNAME=user@company.com
SAP_CLOUD_PASSWORD=secret
```
### Office 365 SSO
```bash
# .env
SAP_ACTIVE_SYSTEM=cloud
SAP_AUTH_STRATEGY=office365
SAP_CLOUD_BASE_URL=https://tenant.launchpad.cfapps.eu10.hana.ondemand.com
SAP_CLOUD_USERNAME=user@company.onmicrosoft.com
SAP_CLOUD_PASSWORD=secret
```
## Using the `sapAuth` Fixture
For tests that need explicit auth control:
```typescript
test('explicit auth', async ({ sapAuth, page }) => {
await sapAuth.login(page, {
url: 'https://sapserver.example.com',
username: 'TESTUSER',
password: 'secret',
strategy: 'onprem',
});
expect(sapAuth.isAuthenticated()).toBe(true);
const session = sapAuth.getSessionInfo();
expect(session.user).toBe('TESTUSER');
// Auto-logout on fixture teardown
});
```
## Session Expiry
The default session timeout is 1800 seconds (30 minutes), matching SAP ICM defaults.
The 30-minute default matches SAP ICM session timeout defaults.
## BTP WorkZone Authentication
BTP WorkZone renders apps inside nested iframes. The `btpWorkZone` fixture handles frame context automatically:
```typescript
test('WorkZone app', async ({ btpWorkZone, ui5 }) => {
const workspace = btpWorkZone.getWorkspaceFrame();
// All operations route through the correct frame
});
```
---
## Navigation
Praman provides 9 navigation functions via the `ui5Navigation` fixture, plus BTP WorkZone
iframe management via `btpWorkZone`. All navigation uses the FLP's own `hasher` API for
reliable hash-based routing without full page reloads.
## Why Not page.goto()?
SAP Fiori Launchpad uses hash-based routing. Calling `page.goto()` with a new URL triggers a
full page reload, which means:
- UI5 core must re-bootstrap (30-60 seconds on BTP)
- All OData models are re-initialized
- Authentication may need to re-negotiate
Praman's navigation functions use `window.hasher.setHash()` via `page.evaluate()`, which
changes the hash fragment without a page reload. The FLP router handles the transition
in-place.
## Navigation Methods
### navigateToApp
Navigates to an app by its semantic object-action string. This is the most common navigation
method.
```typescript
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await ui5Navigation.navigateToApp('SalesOrder-create');
await ui5Navigation.navigateToApp('Material-display');
```
### navigateToTile
Clicks an FLP tile by its visible title text. Useful for homepage-based navigation.
```typescript
await ui5Navigation.navigateToTile('Create Purchase Order');
await ui5Navigation.navigateToTile('Manage Sales Orders');
```
### navigateToIntent
Navigates using a semantic object + action + optional parameters. This is the most flexible
method for parameterized navigation.
```typescript
// Basic intent
await ui5Navigation.navigateToIntent('PurchaseOrder', 'create');
// With parameters
await ui5Navigation.navigateToIntent('PurchaseOrder', 'create', {
plant: '1000',
purchOrg: '1000',
});
// Produces hash: #PurchaseOrder-create?plant=1000&purchOrg=1000
```
### navigateToHash
Sets the URL hash fragment directly. Useful for deep-linking to specific views or states.
```typescript
await ui5Navigation.navigateToHash("PurchaseOrder-manage&/PurchaseOrders('4500000001')");
await ui5Navigation.navigateToHash('Shell-home');
```
### navigateToHome
Returns to the FLP homepage.
```typescript
await ui5Navigation.navigateToHome();
```
### navigateBack / navigateForward
Browser history navigation.
```typescript
await ui5Navigation.navigateBack();
await ui5Navigation.navigateForward();
```
### searchAndOpenApp
Types a query into the FLP shell search bar and opens the first result.
```typescript
await ui5Navigation.searchAndOpenApp('Purchase Order');
await ui5Navigation.searchAndOpenApp('Vendor Master');
```
### getCurrentHash
Reads the current URL hash fragment. Useful for assertions.
```typescript
const hash = await ui5Navigation.getCurrentHash();
expect(hash).toContain('PurchaseOrder');
expect(hash).toContain('manage');
```
## Auto-Stability After Navigation
Every navigation function calls `waitForUI5Stable()` after changing the hash. This ensures the
target app has fully loaded before the next test step executes.
To skip this wait (e.g., if you have a custom wait):
```typescript
// The waitForStable option is available on navigation methods
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
// Stability wait fires automatically after this call returns
```
## Intent Navigation Patterns
### Simple Navigation
```typescript
test('open purchase order', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
```
### Parameterized Navigation
```typescript
test('open specific purchase order', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToIntent('PurchaseOrder', 'display', {
PurchaseOrder: '4500000001',
});
});
```
### Multi-Step Navigation
```typescript
test('list report to object page', async ({ ui5Navigation, ui5 }) => {
// Step 1: Navigate to list report
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
// Step 2: Click a row to navigate to object page
await ui5.click({
controlType: 'sap.m.ColumnListItem',
ancestor: { id: 'poTable' },
});
// Step 3: Verify we're on the object page
const hash = await ui5Navigation.getCurrentHash();
expect(hash).toContain('PurchaseOrders');
// Step 4: Go back
await ui5Navigation.navigateBack();
});
```
## BTP WorkZone Iframe Management
SAP BTP WorkZone renders applications inside nested iframes. The `btpWorkZone` fixture handles
frame switching transparently.
### The Frame Structure
```
Outer Frame (WorkZone shell)
└── Inner Frame (workspace / application)
└── UI5 Application
```
Without frame management, Playwright targets the outer frame and cannot find UI5 controls
inside the inner workspace frame.
### Using btpWorkZone
```typescript
test('WorkZone navigation', async ({ btpWorkZone, ui5, page }) => {
// The manager detects the frame structure automatically
const workspace = btpWorkZone.getWorkspaceFrame();
// UI5 operations automatically target the correct frame
await ui5.click({ id: 'saveBtn' });
});
```
### Cross-Frame Operations
When you need to interact with controls in both the outer shell and inner workspace:
```typescript
test('shell + workspace', async ({ btpWorkZone, ui5Shell, ui5 }) => {
// Shell header is in the outer frame
await ui5Shell.expectShellHeader();
// Application controls are in the inner frame
await ui5.click({ id: 'appButton' });
});
```
## Step Decoration
Every navigation call is wrapped in a Playwright `test.step()` for HTML report visibility:
```
test 'create purchase order'
├── ui5Navigation.navigateToApp: PurchaseOrder-create
├── ui5.fill: vendorInput
├── ui5.click: saveBtn
└── ui5Navigation.navigateToHome
```
## Complete Example
```typescript
test('end-to-end purchase order workflow', async ({ ui5Navigation, ui5, ui5Shell, ui5Footer }) => {
// Navigate to create app
await ui5Navigation.navigateToApp('PurchaseOrder-create');
// Fill form
await ui5.fill({ id: 'vendorInput' }, '100001');
await ui5.select({ id: 'purchOrgSelect' }, '1000');
// Save via footer
await ui5Footer.clickSave();
// Verify navigation to display view
const hash = await ui5Navigation.getCurrentHash();
expect(hash).toContain('PurchaseOrder');
// Navigate home
await ui5Shell.clickHome();
// Verify we're on the homepage
const homeHash = await ui5Navigation.getCurrentHash();
expect(homeHash).toContain('Shell-home');
});
```
---
## Fiori Elements Testing
The `playwright-praman/fe` sub-path provides specialized helpers for testing SAP Fiori Elements applications, including List Report, Object Page, table operations, and the FE Test Library integration.
## Getting Started
```typescript
getListReportTable,
getFilterBar,
setFilterBarField,
executeSearch,
navigateToItem,
getObjectPageLayout,
getHeaderTitle,
navigateToSection,
} from 'playwright-praman/fe';
```
## List Report Operations
### Finding the Table and Filter Bar
```typescript
// Discover the main List Report table
const tableId = await getListReportTable(page);
// Discover the filter bar
const filterBarId = await getFilterBar(page);
```
Both functions wait for UI5 stability before querying and throw a `ControlError` if the control is not found.
### Filter Bar Operations
```typescript
// Set a filter field value
await setFilterBarField(page, filterBarId, 'CompanyCode', '1000');
// Read a filter field value
const value = await getFilterBarFieldValue(page, filterBarId, 'CompanyCode');
// Clear all filter bar fields
await clearFilterBar(page, filterBarId);
// Execute search (triggers the Go button)
await executeSearch(page, filterBarId);
```
### Variant Management
```typescript
// List available variants
const variants = await getAvailableVariants(page);
// ['Standard', 'My Open Orders', 'Last 30 Days']
// Select a variant by name
await selectVariant(page, 'My Open Orders');
```
### Row Navigation
```typescript
// Navigate to a specific row (opens Object Page)
await navigateToItem(page, tableId, 0); // Click first row
```
### Options
All List Report functions accept an optional `ListReportOptions` parameter:
```typescript
interface ListReportOptions {
readonly timeout?: number; // Default: CONTROL_DISCOVERY timeout
readonly skipStabilityWait?: boolean; // Default: false
}
// Example with options
await executeSearch(page, filterBarId, { timeout: 15_000 });
```
## Object Page Operations
### Layout and Header
```typescript
// Discover the Object Page layout
const layoutId = await getObjectPageLayout(page);
// Read the header title
const title = await getHeaderTitle(page);
// 'Purchase Order 4500012345'
```
### Sections
```typescript
// Get all sections with their sub-sections
const sections = await getObjectPageSections(page);
// [
// { id: 'sec1', title: 'General Information', visible: true, index: 0, subSections: [...] },
// { id: 'sec2', title: 'Items', visible: true, index: 1, subSections: [...] },
// ]
// Navigate to a specific section
await navigateToSection(page, 'Items');
// Read form data from a section
const data = await getSectionData(page, 'General Information');
// { 'Vendor': 'Acme GmbH', 'Company Code': '1000', ... }
```
### Edit Mode
```typescript
// Check if Object Page is in edit mode
const editing = await isInEditMode(page);
// Toggle edit mode
await clickEditButton(page);
// Click custom buttons
await clickObjectPageButton(page, 'Delete');
// Save changes
await clickSaveButton(page);
```
## FE Table Helpers
For direct table manipulation (works with both List Report and Object Page tables):
```typescript
feGetTableRowCount,
feGetColumnNames,
feGetCellValue,
feClickRow,
feFindRowByValues,
} from 'playwright-praman/fe';
// Get row count
const count = await feGetTableRowCount(page, tableId);
// Get column headers
const columns = await feGetColumnNames(page, tableId);
// ['Material', 'Description', 'Quantity', 'Unit Price']
// Read a cell value by row index and column name
const material = await feGetCellValue(page, tableId, 0, 'Material');
// Click a row
await feClickRow(page, tableId, 2);
// Find a row by column values
const rowIndex = await feFindRowByValues(page, tableId, {
Material: 'RAW-0001',
Quantity: '100',
});
```
## FE List Helpers
For list-based Fiori Elements views (e.g., Master-Detail):
```typescript
feGetListItemCount,
feGetListItemTitle,
feGetListItemDescription,
feClickListItem,
feSelectListItem,
feFindListItemByTitle,
} from 'playwright-praman/fe';
// Get number of list items
const count = await feGetListItemCount(page, listId);
// Read item titles and descriptions
const title = await feGetListItemTitle(page, listId, 0);
const desc = await feGetListItemDescription(page, listId, 0);
// Interact with list items
await feClickListItem(page, listId, 0);
await feSelectListItem(page, listId, 2);
// Find item by title text
const index = await feFindListItemByTitle(page, listId, 'Purchase Order 4500012345');
```
## FE Test Library Integration
Praman integrates with SAP's own Fiori Elements Test Library (`sap.fe.test`):
```typescript
// Initialize the FE Test Library on the page
await initializeFETestLibrary(page, {
timeout: 30_000,
});
// The test library provides SAP's official FE testing API
// through the bridge, enabling assertions that match SAP's own test patterns
```
## Complete Example
A typical Fiori Elements List Report to Object Page test flow:
```typescript
getListReportTable,
getFilterBar,
setFilterBarField,
executeSearch,
navigateToItem,
getHeaderTitle,
getObjectPageSections,
getSectionData,
} from 'playwright-praman/fe';
test('browse purchase orders and view details', async ({ page, ui5 }) => {
await test.step('Navigate to List Report', async () => {
await page.goto(
'/sap/bc/ui5_ui5/ui2/ushell/shells/abap/FioriLaunchpad.html#PurchaseOrder-manage',
);
await ui5.waitForUI5();
});
await test.step('Filter by company code', async () => {
const filterBar = await getFilterBar(page);
await setFilterBarField(page, filterBar, 'CompanyCode', '1000');
await executeSearch(page, filterBar);
});
await test.step('Navigate to first purchase order', async () => {
const table = await getListReportTable(page);
await navigateToItem(page, table, 0);
await ui5.waitForUI5();
});
await test.step('Verify Object Page header', async () => {
const title = await getHeaderTitle(page);
expect(title).toContain('Purchase Order');
});
await test.step('Read General Information section', async () => {
const data = await getSectionData(page, 'General Information');
expect(data).toHaveProperty('Vendor');
expect(data).toHaveProperty('Company Code');
});
});
```
## Error Handling
All FE helpers throw structured `ControlError` or `NavigationError` instances with suggestions:
```typescript
try {
await getListReportTable(page);
} catch (error) {
// ControlError {
// code: 'ERR_CONTROL_NOT_FOUND',
// message: 'List Report table not found on current page',
// suggestions: [
// 'Verify the current page is a Fiori Elements List Report',
// 'Wait for the page to fully load before accessing the table',
// ]
// }
}
```
---
## OData Operations
Praman provides two approaches to OData data access: **model operations** (browser-side, via
`ui5.odata`) and **HTTP operations** (server-side, via `ui5.odata`). This page covers both,
including V2 vs V4 differences and the ODataTraceReporter.
## Two Approaches
| Approach | Access Path | Use Case |
| -------------------- | ---------------------------- | ----------------------------------------- |
| **Model operations** | Browser-side UI5 model | Reading data the UI already loaded |
| **HTTP operations** | Direct HTTP to OData service | CRUD operations, test data setup/teardown |
## Model Operations (Browser-Side)
Model operations query the UI5 OData model that is already loaded in the browser. No additional
HTTP requests are made — you are reading the same data the UI5 application is displaying.
### getModelData
Reads data at any model path. Returns the full object tree.
```typescript
const orders = await ui5.odata.getModelData('/PurchaseOrders');
console.log(orders.length); // Number of loaded POs
const singlePO = await ui5.odata.getModelData("/PurchaseOrders('4500000001')");
console.log(singlePO.Vendor); // '100001'
```
### getModelProperty
Reads a single property value from the model.
```typescript
const vendor = await ui5.odata.getModelProperty("/PurchaseOrders('4500000001')/Vendor");
const amount = await ui5.odata.getModelProperty("/PurchaseOrders('4500000001')/NetAmount");
```
### getEntityCount
Counts entities in a set. Reads from the model, not from $count.
```typescript
const count = await ui5.odata.getEntityCount('/PurchaseOrders');
expect(count).toBeGreaterThan(0);
```
### waitForODataLoad
Polls the model until data is available at the given path. Useful after navigation when
OData requests are still in flight.
```typescript
await ui5.odata.waitForODataLoad('/PurchaseOrders');
// Data is now available
const orders = await ui5.odata.getModelData('/PurchaseOrders');
```
Default timeout: 15 seconds. Polling interval: 100ms.
### hasPendingChanges
Checks if the model has unsaved changes (dirty state).
```typescript
await ui5.fill({ id: 'vendorInput' }, '100002');
const dirty = await ui5.odata.hasPendingChanges();
expect(dirty).toBe(true);
await ui5.click({ id: 'saveBtn' });
const clean = await ui5.odata.hasPendingChanges();
expect(clean).toBe(false);
```
### fetchCSRFToken
Fetches a CSRF token from an OData service URL. Needed for write operations via HTTP.
```typescript
const token = await ui5.odata.fetchCSRFToken('/sap/opu/odata/sap/API_PO_SRV');
// Use token in custom HTTP requests
```
### Named Model Support
By default, model operations query the component's default model. To query a named model:
```typescript
const data = await ui5.odata.getModelData('/Products', { modelName: 'productModel' });
```
## HTTP Operations (Server-Side)
HTTP operations perform direct OData CRUD operations against the service endpoint using
Playwright's request context. These are independent of the browser — useful for test data
setup, teardown, and backend verification.
### queryEntities
Performs an HTTP GET with OData query parameters.
```typescript
const orders = await ui5.odata.queryEntities('/sap/opu/odata/sap/API_PO_SRV', 'PurchaseOrders', {
filter: "Status eq 'A'",
select: 'PONumber,Vendor,Amount',
expand: 'Items',
orderby: 'PONumber desc',
top: 10,
skip: 0,
});
```
### createEntity
Performs an HTTP POST with automatic CSRF token management.
```typescript
const newPO = await ui5.odata.createEntity('/sap/opu/odata/sap/API_PO_SRV', 'PurchaseOrders', {
Vendor: '100001',
PurchOrg: '1000',
CompanyCode: '1000',
Items: [{ Material: 'MAT-001', Quantity: 10, Unit: 'EA' }],
});
console.log(newPO.PONumber); // Server-generated PO number
```
### updateEntity
Performs an HTTP PATCH with CSRF token and optional ETag for optimistic concurrency.
```typescript
await ui5.odata.updateEntity('/sap/opu/odata/sap/API_PO_SRV', 'PurchaseOrders', "'4500000001'", {
Status: 'B',
Note: 'Updated by test',
});
```
### deleteEntity
Performs an HTTP DELETE with CSRF token.
```typescript
await ui5.odata.deleteEntity('/sap/opu/odata/sap/API_PO_SRV', 'PurchaseOrders', "'4500000001'");
```
### callFunctionImport
Calls an OData function import (action).
```typescript
const result = await ui5.odata.callFunctionImport('/sap/opu/odata/sap/API_PO_SRV', 'ApprovePO', {
PONumber: '4500000001',
});
```
## OData V2 vs V4 Differences
| Aspect | OData V2 | OData V4 |
| ------------------ | --------------------------------------------------- | ------------------------------------------ |
| **Model class** | `sap.ui.model.odata.v2.ODataModel` | `sap.ui.model.odata.v4.ODataModel` |
| **URL convention** | `/sap/opu/odata/sap/SERVICE_SRV` | `/sap/opu/odata4/sap/SERVICE/srvd_a2x/...` |
| **Batch** | `$batch` with changesets | `$batch` with JSON:API format |
| **CSRF token** | Fetched via HEAD request with `X-CSRF-Token: Fetch` | Same mechanism |
| **Deep insert** | Supported via navigation properties | Supported natively |
| **Filter syntax** | `$filter=Status eq 'A'` | `$filter=Status eq 'A'` (same) |
Praman's model operations work with both V2 and V4 models transparently — the model class
handles the protocol differences. HTTP operations use the URL conventions of your service.
## OData Trace Reporter
The `ODataTraceReporter` captures all OData HTTP requests during test runs and generates a
performance trace. See the [Reporters](./reporters.md) guide for configuration.
The trace output includes per-entity-set statistics:
```json
{
"entitySets": {
"PurchaseOrders": {
"GET": { "count": 12, "avgDuration": 340, "maxDuration": 1200, "errors": 0 },
"POST": { "count": 2, "avgDuration": 890, "maxDuration": 1100, "errors": 0 },
"PATCH": { "count": 1, "avgDuration": 450, "maxDuration": 450, "errors": 0 }
},
"Vendors": {
"GET": { "count": 3, "avgDuration": 120, "maxDuration": 200, "errors": 0 }
}
},
"totalRequests": 18,
"totalErrors": 0,
"totalDuration": 8400
}
```
## Complete Example
```typescript
test('verify OData model state after form edit', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
// Wait for initial data load
await ui5.odata.waitForODataLoad('/PurchaseOrders');
// Read model data
const orders = await ui5.odata.getModelData('/PurchaseOrders');
expect(orders.length).toBeGreaterThan(0);
// Navigate to first PO
await ui5.click({
controlType: 'sap.m.ColumnListItem',
ancestor: { id: 'poTable' },
});
// Edit a field
await ui5.click({ id: 'editBtn' });
await ui5.fill({ id: 'noteField' }, 'Test note');
// Verify model has pending changes
const dirty = await ui5.odata.hasPendingChanges();
expect(dirty).toBe(true);
// Save
await ui5.click({ id: 'saveBtn' });
// Verify model is clean
const clean = await ui5.odata.hasPendingChanges();
expect(clean).toBe(false);
});
test('seed and cleanup test data via HTTP', async ({ ui5 }) => {
// Setup: create test PO via direct HTTP
const po = await ui5.odata.createEntity('/sap/opu/odata/sap/API_PO_SRV', 'PurchaseOrders', {
Vendor: '100001',
PurchOrg: '1000',
CompanyCode: '1000',
});
// ... run test steps ...
// Cleanup: delete the test PO
await ui5.odata.deleteEntity(
'/sap/opu/odata/sap/API_PO_SRV',
'PurchaseOrders',
`'${po.PONumber}'`,
);
});
```
---
## OData Mocking
Strategies for mocking OData services in Praman tests. Covers Playwright `page.route()`
interception, SAP CAP mock server integration, and offline test patterns.
## When to Mock OData
| Scenario | Approach | When to Use |
| -------------------- | --------------------------- | --------------------------------------------- |
| Unit/component tests | `page.route()` interception | No backend needed, fast, deterministic |
| Integration tests | CAP mock server | Realistic service behavior, schema validation |
| E2E tests | Real SAP backend | Full system validation, production-like |
| CI/CD pipeline | `page.route()` or CAP mock | Reliable, no SAP system dependency |
## Playwright page.route() Interception
### Basic OData Response Mock
```typescript
test.describe('Purchase Order List (Mocked)', () => {
test.beforeEach(async ({ page }) => {
// Intercept OData requests and return mock data
await page.route('**/sap/opu/odata/sap/API_PURCHASEORDER_PROCESS_SRV/**', async (route) => {
const url = route.request().url();
if (url.includes('PurchaseOrderSet')) {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
d: {
results: [
{
PurchaseOrder: '4500000001',
Supplier: '100001',
CompanyCode: '1000',
PurchasingOrganization: '1000',
NetAmount: '5000.00',
Currency: 'USD',
Status: 'Approved',
},
{
PurchaseOrder: '4500000002',
Supplier: '100002',
CompanyCode: '2000',
PurchasingOrganization: '2000',
NetAmount: '12000.00',
Currency: 'EUR',
Status: 'Pending',
},
],
},
}),
});
} else {
await route.continue();
}
});
});
test('display purchase orders from mock', async ({ ui5, ui5Navigation, ui5Matchers }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
const table = await ui5.control({
controlType: 'sap.ui.comp.smarttable.SmartTable',
});
await ui5Matchers.toHaveRowCount(table, 2);
});
});
```
### OData V4 Mock
```typescript
test.beforeEach(async ({ page }) => {
await page.route('**/odata/v4/PurchaseOrderService/**', async (route) => {
const url = route.request().url();
if (url.includes('PurchaseOrders') && route.request().method() === 'GET') {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
'@odata.context': '$metadata#PurchaseOrders',
value: [
{
PurchaseOrder: '4500000001',
Supplier: '100001',
NetAmount: 5000,
Currency: 'USD',
},
],
}),
});
} else {
await route.continue();
}
});
});
```
### Mock OData $metadata
Some UI5 apps require `$metadata` to render controls correctly:
```typescript
const metadata = `
`;
await page.route('**/$metadata', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/xml',
body: metadata,
});
});
```
### Mock Error Responses
Test how the app handles OData errors:
```typescript
test('handles OData error gracefully', async ({ ui5, ui5Navigation, page }) => {
await page.route('**/PurchaseOrderSet**', async (route) => {
await route.fulfill({
status: 500,
contentType: 'application/json',
body: JSON.stringify({
error: {
code: 'SY/530',
message: {
lang: 'en',
value: 'Internal server error during data retrieval',
},
innererror: {
errordetails: [
{
code: 'SY/530',
message: 'Database connection timeout',
severity: 'error',
},
],
},
},
}),
});
});
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
// Verify the app shows an error message
const errorDialog = await ui5.control({
controlType: 'sap.m.Dialog',
properties: { type: 'Message' },
searchOpenDialogs: true,
});
expect(errorDialog).toBeTruthy();
});
```
### Conditional Mocking (First Call vs Subsequent)
```typescript
let callCount = 0;
await page.route('**/PurchaseOrderSet**', async (route) => {
callCount++;
if (callCount === 1) {
// First call: return empty list
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ d: { results: [] } }),
});
} else {
// Subsequent calls: return data (simulates data creation)
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
d: {
results: [{ PurchaseOrder: '4500000001', Supplier: '100001' }],
},
}),
});
}
});
```
## Mock Factory Pattern
Create reusable mock factories for consistent test data:
```typescript
// tests/mocks/odata-mocks.ts
interface PurchaseOrderMock {
PurchaseOrder: string;
Supplier: string;
CompanyCode: string;
NetAmount: string;
Currency: string;
Status: string;
}
export function createPurchaseOrderMock(
overrides: Partial = {},
): PurchaseOrderMock {
return {
PurchaseOrder: '4500000001',
Supplier: '100001',
CompanyCode: '1000',
NetAmount: '5000.00',
Currency: 'USD',
Status: 'Approved',
...overrides,
};
}
export function createODataV2Response(results: T[]) {
return { d: { results } };
}
export function createODataV4Response(value: T[], context: string) {
return { '@odata.context': context, value };
}
```
Usage:
```typescript
await page.route('**/PurchaseOrderSet**', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify(
createODataV2Response([
createPurchaseOrderMock(),
createPurchaseOrderMock({ PurchaseOrder: '4500000002', Status: 'Pending' }),
]),
),
});
});
```
## SAP CAP Mock Server
For more realistic OData mocking, use SAP CAP (Cloud Application Programming Model) to run
a local OData service with schema validation.
### Setup
```bash
# Install CAP CLI
npm install --save-dev @sap/cds-dk
# Initialize mock project
mkdir tests/mock-server && cd tests/mock-server
cds init --add hana
```
### Define Service Schema
```cds
// tests/mock-server/srv/purchase-order-service.cds
using { cuid, managed } from '@sap/cds/common';
entity PurchaseOrders : cuid, managed {
PurchaseOrder : String(10);
Supplier : String(10);
CompanyCode : String(4);
PurchasingOrganization : String(4);
NetAmount : Decimal(16,3);
Currency : String(5);
Status : String(20);
Items : Composition of many PurchaseOrderItems on Items.parent = $self;
}
entity PurchaseOrderItems : cuid {
parent : Association to PurchaseOrders;
ItemNo : String(5);
Material : String(40);
Quantity : Decimal(13,3);
Plant : String(4);
NetPrice : Decimal(16,3);
}
service PurchaseOrderService {
entity PurchaseOrderSet as projection on PurchaseOrders;
}
```
### Seed Test Data
```json
// tests/mock-server/db/data/PurchaseOrders.csv
PurchaseOrder;Supplier;CompanyCode;PurchasingOrganization;NetAmount;Currency;Status
4500000001;100001;1000;1000;5000.000;USD;Approved
4500000002;100002;2000;2000;12000.000;EUR;Pending
4500000003;100003;1000;1000;800.000;USD;Draft
```
### Playwright Config with CAP Server
```typescript
// playwright.config.ts
export default defineConfig({
webServer: [
{
command: 'npx cds serve --project tests/mock-server --port 4004',
port: 4004,
reuseExistingServer: !process.env.CI,
timeout: 30_000,
},
],
use: {
baseURL: 'http://localhost:8080', // UI5 app server
},
projects: [
{
name: 'with-mock-server',
testDir: './tests/integration',
use: { ...devices['Desktop Chrome'] },
},
],
});
```
### Proxy OData Requests to CAP
```typescript
// In your UI5 app's xs-app.json or proxy config, route OData to CAP:
// /sap/opu/odata/sap/API_PURCHASEORDER_PROCESS_SRV → http://localhost:4004/odata/v4/PurchaseOrderService
// Or use page.route() to redirect:
await page.route('**/sap/opu/odata/**', async (route) => {
const url = new URL(route.request().url());
url.hostname = 'localhost';
url.port = '4004';
url.pathname = url.pathname.replace(
'/sap/opu/odata/sap/API_PURCHASEORDER_PROCESS_SRV',
'/odata/v4/PurchaseOrderService',
);
await route.continue({ url: url.toString() });
});
```
## HAR File Replay
Record and replay real OData traffic for deterministic tests:
```typescript
// Record: run test once with HAR recording
test.use({
// @ts-expect-error -- Playwright context option
recordHar: {
path: 'tests/fixtures/odata-traffic.har',
urlFilter: '**/odata/**',
},
});
// Replay: use recorded HAR in subsequent test runs
test('replay recorded OData traffic', async ({ page }) => {
await page.routeFromHAR('tests/fixtures/odata-traffic.har', {
url: '**/odata/**',
update: false,
});
// Test runs against recorded responses — fast, deterministic
});
```
## Best Practices
| Practice | Why |
| -------------------------------- | ----------------------------------------------------------------------- |
| Mock at the OData level, not DOM | UI5 models parse OData responses — mock the data, not the rendered HTML |
| Include `$metadata` mocks | SmartFields/SmartTables need metadata to render correctly |
| Use mock factories | Consistent, type-safe test data across tests |
| Test error responses too | Validate your app handles 400, 403, 500 gracefully |
| Reset mocks between tests | Use `beforeEach` to set up fresh routes |
| Match `$filter` and `$expand` | Mock responses should respect OData query options when possible |
---
## SAP Control Cookbook
A practical reference for interacting with the most common SAP UI5 controls in Praman tests.
Each control includes its fully qualified type, recommended selectors, interaction examples,
and known pitfalls.
## SmartField
`sap.ui.comp.smartfield.SmartField`
SmartField is a metadata-driven wrapper that renders different inner controls (Input, ComboBox,
DatePicker, etc.) based on OData annotations. It appears throughout SAP Fiori Elements apps.
### Selector
```typescript
// By ID
const field = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
id: /CompanyCode/,
});
// By binding path (most reliable for SmartFields)
const field = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/CompanyCode' },
});
```
### Interaction
```typescript
await ui5.fill({ controlType: 'sap.ui.comp.smartfield.SmartField', id: /CompanyCode/ }, '1000');
// Read current value
const value = await field.getProperty('value');
```
### Pitfalls
- `getControlType()` returns `sap.ui.comp.smartfield.SmartField`, **not** the inner control type
(e.g., not `sap.m.Input`). Do not selector-match by the inner type.
- SmartField in **display mode** renders as `sap.m.Text` — calling `fill()` will fail. Check
the `editable` property first.
- Value help dialogs opened by SmartField have a different control tree. Use `searchOpenDialogs: true`
to find controls inside them.
## SmartTable
`sap.ui.comp.smarttable.SmartTable`
Metadata-driven table that auto-generates columns from OData annotations. Used in most Fiori
Elements List Report pages.
### Selector
```typescript
const table = await ui5.control({
controlType: 'sap.ui.comp.smarttable.SmartTable',
id: /LineItemsSmartTable/,
});
```
### Interaction
```typescript
// Get row count
const rowCount = await table.getProperty('count');
// Click a specific row (inner table is sap.ui.table.Table or sap.m.Table)
const rows = await ui5.controls({
controlType: 'sap.m.ColumnListItem',
ancestor: { controlType: 'sap.ui.comp.smarttable.SmartTable', id: /LineItemsSmartTable/ },
});
await rows[0].click();
// Trigger table personalization
await ui5.click({
controlType: 'sap.m.OverflowToolbarButton',
properties: { icon: 'sap-icon://action-settings' },
ancestor: { controlType: 'sap.ui.comp.smarttable.SmartTable', id: /LineItemsSmartTable/ },
});
```
### Pitfalls
- SmartTable wraps either `sap.m.Table` (responsive) or `sap.ui.table.Table` (grid). Row
selectors differ by inner table type.
- Row counts may reflect only loaded rows if server-side paging is active. Use `growing`
property to check.
- Column visibility depends on table personalization variant — test data may not be visible
in default columns.
## SmartFilterBar
`sap.ui.comp.smartfilterbar.SmartFilterBar`
Renders filter fields from OData annotations. The main filter bar in Fiori Elements List Reports.
### Selector
```typescript
const filterBar = await ui5.control({
controlType: 'sap.ui.comp.smartfilterbar.SmartFilterBar',
id: /listReportFilter/,
});
```
### Interaction
```typescript
// Set a filter value
await ui5.fill(
{
controlType: 'sap.ui.comp.smartfield.SmartField',
ancestor: { controlType: 'sap.ui.comp.smartfilterbar.SmartFilterBar' },
id: /CompanyCode/,
},
'1000',
);
// Click Go / Search
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Go' },
ancestor: { controlType: 'sap.ui.comp.smartfilterbar.SmartFilterBar' },
});
// Expand filters (Adapt Filters dialog)
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Adapt Filters' },
ancestor: { controlType: 'sap.ui.comp.smartfilterbar.SmartFilterBar' },
});
```
### Pitfalls
- Filter fields may not be visible by default. Use "Adapt Filters" to add fields before setting values.
- SmartFilterBar auto-triggers search on initial load with default variant — your filter values
may be overwritten.
- Date range filters require `sap.m.DateRangeSelection` interaction, not simple `fill()`.
## OverflowToolbar
`sap.m.OverflowToolbar`
Toolbar that collapses items into an overflow popover when space is limited.
### Selector
```typescript
const toolbar = await ui5.control({
controlType: 'sap.m.OverflowToolbar',
id: /headerToolbar/,
});
```
### Interaction
```typescript
// Click a visible toolbar button
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Edit' },
ancestor: { controlType: 'sap.m.OverflowToolbar', id: /headerToolbar/ },
});
// Click button that may be in overflow
await ui5.click({
controlType: 'sap.m.OverflowToolbarButton',
properties: { text: 'Delete' },
ancestor: { controlType: 'sap.m.OverflowToolbar', id: /headerToolbar/ },
});
```
### Pitfalls
- Overflowed buttons are in a separate popover — use `searchOpenDialogs: true` if the button
is in the overflow area.
- `sap.m.OverflowToolbarButton` vs `sap.m.Button` — buttons in overflow toolbars may use either
type depending on configuration.
## IconTabBar
`sap.m.IconTabBar`
Tab strip used for section navigation in Object Pages and detail views.
### Selector
```typescript
const tabBar = await ui5.control({
controlType: 'sap.m.IconTabBar',
id: /detailTabBar/,
});
```
### Interaction
```typescript
// Switch to a specific tab by key
await ui5.click({
controlType: 'sap.m.IconTabFilter',
properties: { key: 'items' },
ancestor: { controlType: 'sap.m.IconTabBar', id: /detailTabBar/ },
});
// Switch by text
await ui5.click({
controlType: 'sap.m.IconTabFilter',
properties: { text: 'Line Items' },
});
```
### Pitfalls
- Tab content is lazy-loaded by default. After switching tabs, wait for the content to render
before interacting with controls inside it.
- Icon-only tabs have no `text` property — use `key` or `icon` to select them.
## ObjectPageLayout
`sap.uxap.ObjectPageLayout`
The layout container for Fiori Elements Object Pages. Contains header, sections, and subsections.
### Selector
```typescript
const objectPage = await ui5.control({
controlType: 'sap.uxap.ObjectPageLayout',
});
```
### Interaction
```typescript
// Scroll to a specific section
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'General Information' },
ancestor: { controlType: 'sap.uxap.AnchorBar' },
});
// Toggle header (collapse/expand)
await ui5.click({
controlType: 'sap.m.Button',
id: /expandHeaderBtn/,
});
// Access a subsection
const section = await ui5.control({
controlType: 'sap.uxap.ObjectPageSection',
properties: { title: 'Pricing' },
});
```
### Pitfalls
- Sections use lazy loading — controls in non-visible sections may not exist in the control
tree until scrolled to.
- Header facets are in `sap.uxap.ObjectPageHeaderContent`, not in the main sections.
- `sap.uxap.AnchorBar` is the internal navigation bar — section names there may differ from
actual section titles.
## ComboBox
`sap.m.ComboBox`
Single-selection dropdown with type-ahead filtering.
### Selector
```typescript
const combo = await ui5.control({
controlType: 'sap.m.ComboBox',
id: /countryCombo/,
});
```
### Interaction
```typescript
// Type and select
await ui5.fill({ controlType: 'sap.m.ComboBox', id: /countryCombo/ }, 'Germany');
// Open dropdown and select by item key
await ui5.click({ controlType: 'sap.m.ComboBox', id: /countryCombo/ });
await ui5.click({
controlType: 'sap.ui.core.ListItem',
properties: { key: 'DE' },
searchOpenDialogs: true,
});
```
### Pitfalls
- `fill()` triggers type-ahead but may not select the item. Verify with `getProperty('selectedKey')`.
- Dropdown items are rendered in a popup — use `searchOpenDialogs: true` to find them.
## MultiComboBox
`sap.m.MultiComboBox`
Multi-selection dropdown with token display.
### Selector
```typescript
const multi = await ui5.control({
controlType: 'sap.m.MultiComboBox',
id: /plantsMultiCombo/,
});
```
### Interaction
```typescript
// Select multiple items
await ui5.click({ controlType: 'sap.m.MultiComboBox', id: /plantsMultiCombo/ });
await ui5.click({
controlType: 'sap.ui.core.ListItem',
properties: { text: 'Plant 1000' },
searchOpenDialogs: true,
});
await ui5.click({
controlType: 'sap.ui.core.ListItem',
properties: { text: 'Plant 2000' },
searchOpenDialogs: true,
});
// Close dropdown
await ui5.click({ controlType: 'sap.m.MultiComboBox', id: /plantsMultiCombo/ });
// Read selected keys
const selectedKeys = await multi.getProperty('selectedKeys');
```
### Pitfalls
- Each click toggles an item. Clicking an already-selected item deselects it.
- Tokens may overflow and show "N More" — use `getProperty('selectedKeys')` to verify
selections instead of counting visible tokens.
## MultiInput
`sap.m.MultiInput`
Text input with tokenizer for multiple values and optional value help.
### Selector
```typescript
const multiInput = await ui5.control({
controlType: 'sap.m.MultiInput',
id: /materialMultiInput/,
});
```
### Interaction
```typescript
// Add tokens by typing
await ui5.fill({ controlType: 'sap.m.MultiInput', id: /materialMultiInput/ }, 'MAT-001');
// Press Enter to confirm token (use keyboard interaction)
await multiInput.pressKey('Enter');
// Open value help dialog
await ui5.click({
controlType: 'sap.ui.core.Icon',
properties: { src: 'sap-icon://value-help' },
ancestor: { controlType: 'sap.m.MultiInput', id: /materialMultiInput/ },
});
```
### Pitfalls
- Tokens require Enter key press to be committed — `fill()` alone is not sufficient.
- Value help button is a separate `sap.ui.core.Icon` control, not part of the MultiInput control.
- Removing tokens requires clicking the token's delete icon, not clearing the input.
## Select
`sap.m.Select`
Simple dropdown with no type-ahead (unlike ComboBox).
### Selector
```typescript
const select = await ui5.control({
controlType: 'sap.m.Select',
id: /currencySelect/,
});
```
### Interaction
```typescript
// Open and select an item
await ui5.click({ controlType: 'sap.m.Select', id: /currencySelect/ });
await ui5.click({
controlType: 'sap.ui.core.ListItem',
properties: { key: 'USD' },
searchOpenDialogs: true,
});
// Direct selection via key (no UI interaction)
const currentKey = await select.getProperty('selectedKey');
```
### Pitfalls
- `sap.m.Select` does not support type-ahead. Use `click()` to open, then click the item.
- List items in the dropdown popup are `sap.ui.core.ListItem`, not `sap.m.StandardListItem`.
## DatePicker
`sap.m.DatePicker`
Date input with calendar popup.
### Selector
```typescript
const datePicker = await ui5.control({
controlType: 'sap.m.DatePicker',
id: /deliveryDatePicker/,
});
```
### Interaction
```typescript
// Fill directly with a date string (preferred approach)
await ui5.fill({ controlType: 'sap.m.DatePicker', id: /deliveryDatePicker/ }, '2025-03-15');
// Open calendar and select a date
await ui5.click({
controlType: 'sap.ui.core.Icon',
properties: { src: 'sap-icon://appointment-2' },
ancestor: { controlType: 'sap.m.DatePicker', id: /deliveryDatePicker/ },
});
// Select day from calendar popup
await ui5.click({
controlType: 'sap.ui.unified.calendar.DatesRow',
searchOpenDialogs: true,
});
```
### Pitfalls
- Date format depends on user locale settings. Always use ISO format (`YYYY-MM-DD`) with
`fill()` — Praman normalizes it to the configured locale.
- `sap.m.DateRangeSelection` is a separate control type for date ranges. Do not confuse it
with `sap.m.DatePicker`.
- Calendar popup controls are in `sap.ui.unified.calendar` namespace — different from `sap.m`.
## Quick Reference Table
| Control | Type | Primary Selector Strategy | Key Method |
| ---------------- | ------------------------------------------- | ------------------------- | ---------------------------- |
| SmartField | `sap.ui.comp.smartfield.SmartField` | `bindingPath` | `fill()` |
| SmartTable | `sap.ui.comp.smarttable.SmartTable` | `id` | `getProperty('count')` |
| SmartFilterBar | `sap.ui.comp.smartfilterbar.SmartFilterBar` | `id` | `fill()` on child fields |
| OverflowToolbar | `sap.m.OverflowToolbar` | `id` | `click()` on child buttons |
| IconTabBar | `sap.m.IconTabBar` | `id` | `click()` on `IconTabFilter` |
| ObjectPageLayout | `sap.uxap.ObjectPageLayout` | singleton (one per page) | scroll to section |
| ComboBox | `sap.m.ComboBox` | `id` | `fill()` or click + select |
| MultiComboBox | `sap.m.MultiComboBox` | `id` | click to toggle items |
| MultiInput | `sap.m.MultiInput` | `id` | `fill()` + Enter key |
| Select | `sap.m.Select` | `id` | click + select item |
| DatePicker | `sap.m.DatePicker` | `id` | `fill()` with ISO date |
---
## Error Reference
Praman provides 14 typed error classes with 58 machine-readable error codes. Every error includes
the attempted operation, a `retryable` flag, and actionable recovery `suggestions`.
## Error Hierarchy
All errors extend `PramanError`, which extends `Error`:
```text
Error
└── PramanError
├── AIError (11 codes)
├── AuthError (4 codes)
├── BridgeError (5 codes)
├── ConfigError (3 codes)
├── ControlError (8 codes)
├── FLPError (5 codes)
├── IntentError (4 codes)
├── NavigationError (3 codes)
├── ODataError (3 codes)
├── PluginError (3 codes)
├── SelectorError (3 codes)
├── TimeoutError (3 codes)
└── VocabularyError (4 codes)
```
## Error Codes
### Config Errors
| Code | Description | Retryable |
| ---------------------- | ---------------------------------------------- | --------- |
| `ERR_CONFIG_INVALID` | Zod schema validation failed | No |
| `ERR_CONFIG_NOT_FOUND` | Config file not found at expected path | No |
| `ERR_CONFIG_PARSE` | Config file could not be parsed (syntax error) | No |
### Bridge Errors
| Code | Description | Retryable |
| ---------------------- | ---------------------------------------- | --------- |
| `ERR_BRIDGE_TIMEOUT` | Bridge injection exceeded timeout | Yes |
| `ERR_BRIDGE_INJECTION` | Bridge script failed to inject into page | Yes |
| `ERR_BRIDGE_NOT_READY` | Bridge not ready (UI5 not loaded) | Yes |
| `ERR_BRIDGE_VERSION` | UI5 version mismatch or unsupported | No |
| `ERR_BRIDGE_EXECUTION` | Bridge script execution failed | Yes |
### Control Errors
| Code | Description | Retryable |
| -------------------------------- | ----------------------------------------------- | --------- |
| `ERR_CONTROL_NOT_FOUND` | Control not found with given selector | Yes |
| `ERR_CONTROL_NOT_VISIBLE` | Control exists but is not visible | Yes |
| `ERR_CONTROL_NOT_ENABLED` | Control is visible but disabled | No |
| `ERR_CONTROL_NOT_INTERACTABLE` | Control cannot receive interactions | No |
| `ERR_CONTROL_PROPERTY` | Property read/write failed | No |
| `ERR_CONTROL_AGGREGATION` | Aggregation access failed | No |
| `ERR_CONTROL_METHOD` | Method invocation failed (possibly blacklisted) | No |
| `ERR_CONTROL_INTERACTION_FAILED` | Click/type/select operation failed | Yes |
### Auth Errors
| Code | Description | Retryable |
| --------------------------- | ----------------------------------------- | --------- |
| `ERR_AUTH_FAILED` | Authentication failed (wrong credentials) | No |
| `ERR_AUTH_TIMEOUT` | Login page did not respond in time | Yes |
| `ERR_AUTH_SESSION_EXPIRED` | Session expired (default: 30 min) | Yes |
| `ERR_AUTH_STRATEGY_INVALID` | Unknown or misconfigured auth strategy | No |
### Navigation Errors
| Code | Description | Retryable |
| ------------------------ | ----------------------------------- | --------- |
| `ERR_NAV_TILE_NOT_FOUND` | FLP tile not found by title | No |
| `ERR_NAV_ROUTE_FAILED` | Hash navigation did not resolve | Yes |
| `ERR_NAV_TIMEOUT` | Navigation did not complete in time | Yes |
### OData Errors
| Code | Description | Retryable |
| -------------------------- | ------------------------------------- | --------- |
| `ERR_ODATA_REQUEST_FAILED` | HTTP request to OData service failed | Yes |
| `ERR_ODATA_PARSE` | OData response could not be parsed | No |
| `ERR_ODATA_CSRF` | CSRF token fetch or validation failed | Yes |
### Selector Errors
| Code | Description | Retryable |
| ------------------------ | ------------------------------------ | --------- |
| `ERR_SELECTOR_INVALID` | Selector object is malformed | No |
| `ERR_SELECTOR_AMBIGUOUS` | Multiple controls match the selector | No |
| `ERR_SELECTOR_PARSE` | Selector string could not be parsed | No |
### Timeout Errors
| Code | Description | Retryable |
| ------------------------------- | ---------------------------------- | --------- |
| `ERR_TIMEOUT_UI5_STABLE` | UI5 did not reach stable state | Yes |
| `ERR_TIMEOUT_CONTROL_DISCOVERY` | Control discovery exceeded timeout | Yes |
| `ERR_TIMEOUT_OPERATION` | Generic operation timeout | Yes |
### AI Errors
| Code | Description | Retryable |
| ------------------------------ | ------------------------------------ | --------- |
| `ERR_AI_PROVIDER_UNAVAILABLE` | LLM provider not reachable | Yes |
| `ERR_AI_RESPONSE_INVALID` | LLM returned invalid response format | Yes |
| `ERR_AI_TOKEN_LIMIT` | Prompt exceeded token limit | No |
| `ERR_AI_RATE_LIMITED` | Provider rate limit hit | Yes |
| `ERR_AI_NOT_CONFIGURED` | AI provider not configured | No |
| `ERR_AI_LLM_CALL_FAILED` | LLM API call failed | Yes |
| `ERR_AI_RESPONSE_PARSE_FAILED` | Could not parse LLM response | Yes |
| `ERR_AI_CONTEXT_BUILD_FAILED` | Page context build failed | Yes |
| `ERR_AI_STEP_INTERPRET_FAILED` | Step interpretation failed | No |
| `ERR_AI_INVALID_REQUEST` | Invalid or malformed AI request | No |
| `ERR_AI_CAPABILITY_NOT_FOUND` | Requested capability not in registry | No |
### Plugin Errors
| Code | Description | Retryable |
| ------------------------- | --------------------------------- | --------- |
| `ERR_PLUGIN_LOAD` | Plugin module could not be loaded | No |
| `ERR_PLUGIN_INIT` | Plugin initialization failed | Yes |
| `ERR_PLUGIN_INCOMPATIBLE` | Plugin version incompatible | No |
### Vocabulary Errors
| Code | Description | Retryable |
| ------------------------------ | --------------------------------------- | --------- |
| `ERR_VOCAB_TERM_NOT_FOUND` | Business term not in vocabulary | No |
| `ERR_VOCAB_DOMAIN_LOAD_FAILED` | Domain JSON failed to load | Yes |
| `ERR_VOCAB_JSON_INVALID` | Domain vocabulary JSON malformed | No |
| `ERR_VOCAB_AMBIGUOUS_MATCH` | Multiple terms match with similar score | No |
### Intent Errors
| Code | Description | Retryable |
| ------------------------------ | ------------------------------------ | --------- |
| `ERR_INTENT_FIELD_NOT_FOUND` | Form field not found for intent | No |
| `ERR_INTENT_ACTION_FAILED` | Intent action could not be completed | Yes |
| `ERR_INTENT_NAVIGATION_FAILED` | Intent navigation to app failed | Yes |
| `ERR_INTENT_VALIDATION_FAILED` | Intent input validation failed | No |
### FLP Errors
| Code | Description | Retryable |
| --------------------------- | ----------------------------- | --------- |
| `ERR_FLP_SHELL_NOT_FOUND` | FLP shell container not found | Yes |
| `ERR_FLP_PERMISSION_DENIED` | Insufficient FLP permissions | No |
| `ERR_FLP_API_UNAVAILABLE` | FLP UShell API not available | Yes |
| `ERR_FLP_INVALID_USER` | FLP user context invalid | No |
| `ERR_FLP_OPERATION_TIMEOUT` | FLP operation timed out | Yes |
## Using Errors in Tests
### Catch and Inspect
```typescript
test('handle control not found', async ({ ui5 }) => {
try {
await ui5.control({ id: 'nonExistent' });
} catch (error) {
if (error instanceof ControlError) {
console.log(error.code); // 'ERR_CONTROL_NOT_FOUND'
console.log(error.retryable); // true
console.log(error.suggestions); // ['Verify the control ID...', ...]
console.log(error.toUserMessage()); // Human-readable
console.log(error.toAIContext()); // Machine-readable for AI agents
}
}
});
```
### Error Properties
Every `PramanError` includes:
| Property | Type | Description |
| ------------- | ----------- | ------------------------------------------------------- |
| `code` | `ErrorCode` | Machine-readable code (e.g., `'ERR_CONTROL_NOT_FOUND'`) |
| `message` | `string` | Human-readable description |
| `attempted` | `string` | What operation was attempted |
| `retryable` | `boolean` | Whether the operation can be retried |
| `suggestions` | `string[]` | 2-4 actionable recovery steps |
| `details` | `Record` | Additional context (selector, timeout, etc.) |
| `timestamp` | `string` | ISO-8601 timestamp |
### Serialization
```typescript
// For humans
error.toUserMessage();
// "Control not found: #saveBtn. Try: Verify the control ID exists..."
// For logging (JSON)
error.toJSON();
// { code, message, attempted, retryable, suggestions, details, timestamp }
// For AI agents
error.toAIContext();
// Structured context optimized for LLM consumption
```
---
## Debugging & Troubleshooting
This guide covers Praman's debugging tools, log configuration, trace viewer integration, and solutions to common issues.
## Pino Log Levels
Praman uses [pino](https://github.com/pinojs/pino) for structured JSON logging with child loggers per module and correlation IDs.
### Configuring Log Levels
```typescript
// praman.config.ts
export default {
logLevel: 'debug', // 'fatal' | 'error' | 'warn' | 'info' | 'debug' | 'trace'
};
```
Or via environment variable:
```bash
PRAMAN_LOG_LEVEL=debug npx playwright test
```
### Log Level Guide
| Level | Use When |
| ------- | ---------------------------------------------------- |
| `fatal` | Production -- only unrecoverable errors |
| `error` | Default -- errors and failures only |
| `warn` | Investigating intermittent issues |
| `info` | Monitoring test execution flow |
| `debug` | Developing or debugging bridge/proxy issues |
| `trace` | Full diagnostic output including serialized payloads |
### Creating Module Loggers
:::info[Contributor Only]
The `createLogger` and `REDACTION_PATHS` APIs use internal path aliases (`#core/*`) and are only available when developing Praman itself. End users control logging via the `PRAMAN_LOG_LEVEL` environment variable.
:::
```typescript
const logger = createLogger('my-module');
logger.info({ selector }, 'Finding control');
logger.debug({ result }, 'Control found');
logger.error({ err }, 'Control not found');
```
### Secret Redaction
Pino is configured with redaction paths to prevent secrets from appearing in logs:
```typescript
// Redacted paths include:
// - password, apiKey, token, secret, authorization
// - Nested paths: *.password, config.ai.apiKey, etc.
```
## Playwright Trace Viewer
Enable trace capture in `playwright.config.ts`:
```typescript
export default defineConfig({
use: {
trace: 'retain-on-failure', // or 'on' for all tests
},
});
```
### Viewing Traces
```bash
# After a test failure
npx playwright show-trace test-results/my-test/trace.zip
# Or open the HTML report which includes traces
npx playwright show-report
```
### Trace Viewer Limitations with Bridge page.evaluate()
The Playwright trace viewer has limitations when debugging bridge operations:
1. **page.evaluate() calls appear as opaque steps.** The trace shows that `page.evaluate()` was called but does not show the JavaScript executed inside the browser context.
2. **Bridge injection is not visible.** The bridge setup script runs via `page.evaluate()` or `addInitScript()`, which appear as single steps without internal detail.
3. **UI5 control state is not captured.** The trace viewer captures DOM snapshots but not UI5's in-memory control tree, bindings, or model data.
**Workaround: Use test.step() for visibility**
Praman wraps operations in `test.step()` calls that appear in the trace timeline:
```typescript
await test.step('Click submit button', async () => {
await ui5.click({ id: 'submitBtn' });
});
```
**Workaround: Enable debug logging**
Set `PRAMAN_LOG_LEVEL=debug` to get detailed bridge communication logs alongside the trace.
## OpenTelemetry Integration
Praman includes an OpenTelemetry (OTel) tracing layer for distributed observability.
:::info[Contributor Only]
The telemetry APIs below use internal path aliases (`#core/*`) and are only available when developing Praman itself. End users enable OTel via configuration (see below).
:::
```typescript
```
### 4-Layer Observability Stack
| Layer | Tool | Purpose |
| ----- | ----------------------- | ---------------------------------------------- |
| L1 | Playwright Reporter API | Test events via `test.step()` |
| L2 | pino structured logging | JSON logs with correlation IDs |
| L3 | OpenTelemetry (opt-in) | Spans for bridge/proxy/auth operations |
| L4 | AI Agent Telemetry | Capability introspection, deterministic replay |
### OTel Configuration
OTel is zero-overhead when disabled (NoOp tracer). To enable:
```typescript
// praman.config.ts
export default {
telemetry: {
enabled: true,
serviceName: 'praman-tests',
exporter: 'otlp', // or 'jaeger', 'azure-monitor'
endpoint: 'http://localhost:4317',
},
};
```
Spans are created for key operations:
```
praman.bridge.inject (bridge injection)
praman.bridge.evaluate (page.evaluate call)
praman.proxy.findControl (control discovery)
praman.proxy.executeMethod (method invocation)
praman.auth.login (authentication)
```
## Error Introspection: toUserMessage() and toAIContext()
Every `PramanError` provides two introspection methods:
### toUserMessage()
Human-readable error summary for terminal output:
```typescript
try {
await ui5.click({ id: 'nonExistentBtn' });
} catch (error) {
if (error instanceof PramanError) {
console.error(error.toUserMessage());
// Output:
// [ERR_CONTROL_NOT_FOUND] Control not found: nonExistentBtn
// Attempted: Find control with selector: {"id":"nonExistentBtn"}
// Suggestions:
// - Verify the control ID exists in the UI5 view
// - Check if the page has fully loaded (waitForUI5Stable)
// - Try using controlType + properties instead of ID
}
}
```
### toAIContext()
Machine-readable structured context for AI agents:
```typescript
const context = error.toAIContext();
// {
// code: 'ERR_CONTROL_NOT_FOUND',
// message: 'Control not found: nonExistentBtn',
// attempted: 'Find control with selector: {"id":"nonExistentBtn"}',
// retryable: true,
// severity: 'error',
// details: { selector: { id: 'nonExistentBtn' }, timeout: 30000 },
// suggestions: ['Verify the control ID exists...', ...],
// timestamp: '2026-02-23T10:30:00.000Z',
// }
```
### toJSON()
Full serialization for logging and persistence:
```typescript
const serialized = error.toJSON();
// Includes all fields from toAIContext() plus stack trace
```
## Common Issues and Resolutions
### Bridge Injection Timeout
**Symptom:** `BridgeError: ERR_BRIDGE_TIMEOUT` during test startup.
**Causes and fixes:**
- The page is not a UI5 application. Verify the URL loads a page with `sap.ui.require`.
- UI5 is loading slowly. Increase `controlDiscoveryTimeout` in config.
- A CSP (Content Security Policy) blocks `page.evaluate()`. Check browser console for CSP violations.
```typescript
// Increase timeout
export default {
controlDiscoveryTimeout: 60_000, // 60 seconds
};
```
### Control Not Found
**Symptom:** `ControlError: ERR_CONTROL_NOT_FOUND`
**Causes and fixes:**
- The control has not rendered yet. Add `await ui5.waitForUI5()` before the operation.
- The control ID is wrong. Use the UI5 Diagnostics tool (`Ctrl+Alt+Shift+S` in the browser) to inspect control IDs.
- The control is inside a different frame (WorkZone). Ensure the page reference points to the correct frame.
- The control is hidden. Check `preferVisibleControls` config setting.
### Stale Object Reference
**Symptom:** `Error: Object not found for UUID: xxx-xxx`
**Causes and fixes:**
- The browser-side object was evicted by TTL cleanup. Reduce test step duration or increase TTL.
- A page navigation invalidated the object map. Re-discover the object after navigation.
```typescript
// After navigation, objects from the previous page are gone
await page.goto('/newPage');
await ui5.waitForUI5();
// Re-discover objects on the new page
const model = await ui5.control({ id: 'myControl' });
```
### Stability Wait Hanging
**Symptom:** Test hangs on `waitForUI5Stable()`.
**Causes and fixes:**
- A third-party overlay (WalkMe, analytics) keeps sending HTTP requests. Use `skipStabilityWait`:
```typescript
// Global config
export default {
skipStabilityWait: true,
};
// Per-selector override
await ui5.click({ id: 'submitBtn' }, { skipStabilityWait: true });
```
### Auth Failures
**Symptom:** `AuthError: ERR_AUTH_FAILED`
**Causes and fixes:**
- Credentials are expired or wrong. Check `SAP_CLOUD_USERNAME` and `SAP_CLOUD_PASSWORD` env vars.
- The auth strategy does not match the system. SAP BTP uses SAML; on-premise may use Basic auth.
- Storage state is stale. Delete `.auth/` directory and re-run:
```bash
rm -rf .auth/
npx playwright test --project=setup
```
### page.evaluate() ReferenceError
**Symptom:** `ReferenceError: functionName is not defined` in browser console.
**Cause:** A module-level helper function was referenced inside `page.evaluate()`. Only inner functions are serialized.
**Fix:** Move the helper function inside the `page.evaluate()` callback. See the [Bridge Internals](./bridge-internals.md) guide for the detailed explanation of this serialization constraint.
## Debug Checklist
When a test fails, follow this diagnostic sequence:
1. **Read the error message.** Praman errors include `code`, `attempted`, and `suggestions[]`.
2. **Check the Playwright trace.** Run `npx playwright show-report` and open the trace for the failed test.
3. **Enable debug logging.** Set `PRAMAN_LOG_LEVEL=debug` and re-run the test.
4. **Inspect UI5 diagnostics.** Open the app in a browser and press `Ctrl+Alt+Shift+S` to see the UI5 control tree.
5. **Verify bridge injection.** In the browser console, check `window.__praman_bridge.ready` and `window.__praman_bridge.ui5Version`.
6. **Check the frame.** For WorkZone apps, ensure you are operating on the app frame, not the shell frame.
7. **Review OData traces.** If using the OData trace reporter, check `odata-trace.json` for failed HTTP calls.
---
## Feature Inventory
Complete feature inventory for Praman v1.0. Each entry documents what the feature does, why it matters,
its API, implicit behaviors, and source files.
:::tip
For the auto-generated API reference from TSDoc comments, see the **API Reference** in the navbar.
This page provides a higher-level feature overview with usage examples and context.
:::
## Overview
| Metric | Count |
| ------------------------ | ----- |
| **Public Functions** | 120+ |
| **Public Types** | 100+ |
| **Error Codes** | 58 |
| **Fixture Modules** | 12 |
| **Custom Matchers** | 10 |
| **UI5 Control Types** | 199 |
| **Auth Strategies** | 6 |
| **SAP Business Domains** | 5 |
| **Sub-Path Exports** | 6 |
For the full detailed inventory with API examples, source files, and implicit behaviors,
see the [capabilities document](https://github.com/mrkanitkar/playwright-praman/blob/main/plans/capabilities.md).
## Core Playwright Extensions
### UI5 Control Discovery
Discovers SAP UI5 controls using a multi-strategy chain (cache, direct-ID, RecordReplay, registry scan) and returns a typed proxy for interaction. Eliminates brittle CSS/XPath selectors.
```typescript
const btn = await ui5.control({ id: 'saveBtn' });
const input = await ui5.control({
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor' },
});
```
### Control Proxy (Typed Method Forwarding)
Wraps discovered controls in a JavaScript Proxy that routes method calls through `page.evaluate()` with automatic type detection and method blacklisting.
```typescript
const control = await ui5.control({ id: 'myInput' });
await control.press();
await control.enterText('Hello');
const value = await control.getValue();
```
### Three Interaction Strategies
- **UI5-native** (default): `firePress()` / `fireTap()` / DOM click fallback
- **DOM-first**: DOM click first, UI5 event fallback
- **OPA5**: SAP RecordReplay API
### 10 Custom UI5 Matchers
Extends `expect()` with UI5-specific assertions: `toHaveUI5Text`, `toBeUI5Visible`, `toBeUI5Enabled`, `toHaveUI5Property`, `toHaveUI5ValueState`, `toHaveUI5Binding`, `toBeUI5ControlType`, `toHaveUI5RowCount`, `toHaveUI5CellText`, `toHaveUI5SelectedRows`.
## SAP UI5 Control Support
- **199 typed control definitions** covering all major categories
- **Table operations** for 6 table variants (sap.m.Table, sap.ui.table.Table, TreeTable, AnalyticalTable, SmartTable, mdc.Table)
- **Dialog management** for 10+ dialog types
- **Date/time picker** operations with locale awareness
## UI5 Lifecycle & Synchronization
- **Automatic stability wait** — three-tier: bootstrap (60s) / stable (15s) / DOM settle (500ms)
- **Request interception** — blocks WalkMe, Google Analytics, Qualtrics
- **Retry with exponential backoff** — infrastructure-level retry with jitter
## Fixtures & Test Abstractions
- **12 fixture modules** merged via `mergeTests()`
- **6 authentication strategies**: basic, BTP SAML, Office 365, API, certificate, multi-tenant
- **9 FLP navigation functions**: app, tile, intent, hash, home, back, forward, search, getCurrentHash
- **FLP shell & footer** interaction
- **SM12 lock management** with auto-cleanup
- **User settings** reader (language, date format, timezone)
- **Test data generation** with UUID/timestamp substitution
## OData / API Automation
- **Model-level access** — read data from UI5 OData models in the browser
- **HTTP-level CRUD** — direct OData operations with CSRF token management
## AI & Agent Enablement
- **Page discovery** — structured `PageContext` for AI agents
- **Capability registry** — machine-readable API catalog for prompt engineering
- **Recipe registry** — reusable test pattern library
- **Agentic handler** — autonomous test generation from natural language
- **LLM abstraction** — provider-agnostic (Claude, OpenAI, Azure OpenAI)
- **Vocabulary service** — business term to UI5 selector resolution
## Configuration & Extensibility
- **Zod-validated config** with environment variable overrides
- **14 error classes, 60 error codes** with `retryable` flag and `suggestions[]`
- **Structured logging** with secret redaction
- **OpenTelemetry integration** for distributed tracing
- **Playwright step decoration** for rich HTML reports
## Domain-Specific Features
- **Fiori Elements helpers** — List Report + Object Page APIs
- **5 SAP domain intents** — Procurement, Sales, Finance, Manufacturing, Master Data
- **Compliance reporter** — tracks Praman vs raw Playwright usage
- **OData trace reporter** — per-entity-set performance statistics
- **CLI tools** — `init` (scaffold + IDE agent installation), `doctor`, `uninstall`
- **AI agents** — 3 Claude Code SAP agents (planner, generator, healer) installed by `init`
- **Seed file** — authenticated SAP browser session for AI discovery (`tests/seeds/sap-seed.spec.ts`)
---
## Capabilities & Recipes Guide
Praman ships with a capability registry and a recipe registry that describe what the framework
can do and how to do it. These registries serve two audiences: human testers browsing the API
surface, and AI agents that need structured context for test generation.
## Capabilities
A **capability** is a single API method or fixture function that Praman exposes. Each capability
has a name, description, category, usage example, and priority tier.
### Listing All Capabilities
```typescript
// List every registered capability
const all = capabilities.list();
console.log(`Total capabilities: ${all.length}`);
for (const cap of all) {
console.log(`${cap.name} (${cap.category}): ${cap.description}`);
}
```
### Filtering by Priority
Capabilities are organized into three priority tiers:
| Priority | Meaning | Example |
| ---------------- | ---------------------------------------------------------- | ------------------------------------------------ |
| `fixture` | Primary Playwright fixture API (recommended for consumers) | `ui5.control()`, `ui5Navigation.navigateToApp()` |
| `namespace` | Secondary utility or handler export | `SAPAuthHandler`, `buildPageContext()` |
| `implementation` | Internal implementation detail | Bridge adapters, serializers |
```typescript
// Only the fixture-level APIs (what most testers need)
const fixtures = capabilities.listFixtures();
console.log(`Fixture-level capabilities: ${fixtures.length}`);
// Or by any priority
const namespaceLevel = capabilities.listByPriority('namespace');
```
### Searching Capabilities
```typescript
// Search by substring in name or description
const tableOps = capabilities.find('table');
for (const cap of tableOps) {
console.log(`${cap.name}: ${cap.usageExample}`);
}
// Look up a specific capability by name
const clickCap = capabilities.findByName('clickButton');
if (clickCap) {
console.log(clickCap.usageExample);
// => await ui5.click({ id: 'submitBtn' })
}
```
### Browsing by Category
```typescript
// List all categories
const categories = capabilities.getCategories();
console.log(categories);
// => ['ui5', 'auth', 'navigate', 'table', 'dialog', 'date', ...]
// Get capabilities in a specific category (CapabilityCategory enum value)
const navCaps = capabilities.byCategory('navigate');
for (const cap of navCaps) {
console.log(`${cap.name}: ${cap.description}`);
}
```
### Looking Up by ID
Each capability has a unique kebab-case ID.
```typescript
const cap = capabilities.get('click-button');
if (cap) {
console.log(cap.name); // 'clickButton'
console.log(cap.description); // 'Clicks a UI5 button by selector'
console.log(cap.usageExample); // "await ui5.click({ id: 'submitBtn' })"
console.log(cap.category); // 'ui5' (CapabilityCategory enum value)
console.log(cap.priority); // 'fixture'
console.log(cap.qualifiedName); // 'ui5.clickButton'
}
```
### Statistics
```typescript
const stats = capabilities.getStatistics();
console.log(`Total methods: ${stats.totalMethods}`);
console.log(`Categories: ${stats.categories.join(', ')}`);
console.log(`Fixtures: ${stats.byPriority.fixture}`);
console.log(`Namespace: ${stats.byPriority.namespace}`);
console.log(`Implementation: ${stats.byPriority.implementation}`);
```
## Recipes
A **recipe** is a reusable test pattern -- a curated code snippet that demonstrates how to
accomplish a specific testing task. Recipes have a name, description, domain, priority,
capabilities, and a ready-to-use pattern.
### Recipe Metadata
| Field | Type | Description |
| -------------- | ---------------- | -------------------------------------------------------------------------- |
| `id` | `string` | Unique kebab-case identifier (e.g. `'recipe-ui5-button-click'`) |
| `name` | `string` | Short descriptive title |
| `description` | `string` | What this recipe demonstrates |
| `domain` | `string` | Domain grouping (`'ui5'`, `'auth'`, `'navigation'`, etc.) |
| `priority` | `RecipePriority` | `'essential'`, `'recommended'`, `'optional'`, `'advanced'`, `'deprecated'` |
| `capabilities` | `string[]` | Capability IDs this recipe uses (e.g. `['UI5-UI5-003']`) |
| `pattern` | `string` | Ready-to-use TypeScript code pattern |
### Selecting by Priority
```typescript
// Must-know patterns for any SAP Fiori test suite
const essential = recipes.selectByPriority('essential');
// Best-practice patterns for common scenarios
const recommended = recipes.selectByPriority('recommended');
// Specialized patterns for complex edge cases
const advanced = recipes.selectByPriority('advanced');
```
### Selecting by Domain
```typescript
// All authentication recipes
const authRecipes = recipes.selectByDomain('auth');
// All navigation recipes
const navRecipes = recipes.selectByDomain('navigation');
// List all available domains
const domains = recipes.getDomains();
console.log(domains);
```
### Combined Filtering
Use `recipes.select()` with multiple criteria.
```typescript
// Essential recipes in the auth domain
const filtered = recipes.select({
domain: 'auth',
priority: 'essential',
});
```
### Searching Recipes
```typescript
// Free-text search across name, description, and capabilities
const poRecipes = recipes.search('purchase order');
for (const r of poRecipes) {
console.log(`${r.name}: ${r.description}`);
console.log(r.pattern);
}
// Check if a recipe exists
if (recipes.has('Login to SAP BTP via SAML')) {
const steps = recipes.getSteps('Login to SAP BTP via SAML');
console.log(steps);
}
```
### Finding Recipes for a Capability
```typescript
// Find recipes that use a specific capability
const clickRecipes = recipes.forCapability('click');
for (const r of clickRecipes) {
console.log(`${r.name} [${r.priority}]`);
}
```
### Finding Recipes for a Business Process
```typescript
// Recipes for procurement workflows
const procRecipes = recipes.forProcess('procurement');
// Recipes for a specific SAP domain
const finRecipes = recipes.forDomain('finance');
```
### Top Recipes
```typescript
// Get the top 5 recipes (by insertion order)
const topFive = recipes.getTopRecipes(5);
for (const r of topFive) {
console.log(`[${r.priority}] ${r.name}`);
}
```
## AI Agent Usage
Both registries are designed to be consumed by AI agents (Claude, GPT, Gemini) as structured
context for test generation.
### capabilities.forAI()
Returns the full registry as structured JSON optimized for AI consumption.
```typescript
const aiContext = capabilities.forAI();
// Returns CapabilitiesJSON:
// {
// name: 'playwright-praman',
// version: '1.0.0',
// totalMethods: 120,
// byPriority: { fixture: 40, namespace: 50, implementation: 30 },
// fixtures: [...], // Fixture-level entries listed first
// methods: [...], // All entries
// }
```
### Provider-Specific Formatting
Each AI provider has an output format optimized for its consumption model.
```typescript
// XML-structured for Claude
const claudeContext = capabilities.forProvider('claude');
// JSON registry snapshot for OpenAI
const openaiContext = capabilities.forProvider('openai');
// JSON registry snapshot for Gemini
const geminiContext = capabilities.forProvider('gemini');
```
### recipes.forAI()
Returns all recipes in a flat array suitable for injection into an AI prompt.
```typescript
const allRecipes = recipes.forAI();
// Returns RecipeEntry[] with full pattern, capabilities, and metadata
```
### Building an AI Prompt with Capabilities and Recipes
```typescript
// Build a context string for an AI agent
const capabilityContext = JSON.stringify(capabilities.forAI(), null, 2);
const relevantRecipes = recipes.select({ domain: 'navigation' });
const prompt = `
You are a test automation agent for SAP Fiori applications.
Available capabilities:
${capabilityContext}
Relevant recipes:
${JSON.stringify(relevantRecipes, null, 2)}
Generate a Playwright test that navigates to the Purchase Order app and verifies
the table has at least one row.
`;
```
### Using with the Agentic Handler
The `pramanAI` fixture integrates capabilities and recipes automatically.
```typescript
test('AI-generated test', async ({ page, pramanAI }) => {
// Discover the current page (builds PageContext)
const context = await pramanAI.discoverPage({ interactiveOnly: true });
// Generate a test from a natural language description
// The agentic handler automatically uses capabilities + recipes as context
const result = await pramanAI.agentic.generateTest(
'Create a purchase order for vendor 100001 with material MAT-001',
page,
);
console.log(result.steps); // ['Navigate to app', 'Fill vendor', ...]
console.log(result.code); // Generated TypeScript test code
console.log(result.metadata); // Model, tokens, duration, capabilities used
});
```
## Advanced: Registering Custom Capabilities
If your project extends Praman with custom helpers, register them so AI agents can discover them.
```typescript
capabilities.registry.register({
id: 'UI5-CUSTOM-001',
qualifiedName: 'custom.approveWorkflow',
name: 'approveWorkflow',
description: 'Approves a workflow item in the SAP Fiori inbox',
category: 'ui5',
intent: 'approve',
sapModule: 'cross.fnd.fiori.inbox',
usageExample: "await customHelpers.approveWorkflow({ workitemId: '000123' })",
registryVersion: 1,
priority: 'fixture',
});
// Now AI agents can discover it
const workflowCaps = capabilities.find('workflow');
```
## Validating Recipes
Check that a recipe exists and inspect its metadata.
```typescript
const result = recipes.validate('Login to SAP BTP via SAML');
if (result.valid) {
console.log('Recipe found');
} else {
console.log('Recipe not found');
}
```
## JSON Export
Export the full registries for external tools, dashboards, or documentation generators.
```typescript
// Capabilities as JSON
const capJson = capabilities.toJSON();
// Recipes as JSON
const recipeJson = recipes.toJSON();
// Write to file for external consumption
await writeFile('capabilities.json', JSON.stringify(capJson, null, 2));
await writeFile('recipes.json', JSON.stringify(recipeJson, null, 2));
```
---
## Intent API
The `playwright-praman/intents` sub-path provides high-level, business-oriented test operations for SAP S/4HANA modules. Instead of interacting with individual UI5 controls, intent APIs express test steps in business terms.
## Overview
```typescript
fillField,
clickButton,
selectOption,
assertField,
navigateAndSearch,
} from 'playwright-praman/intents';
```
The intent layer is built on two foundations:
1. **Core wrappers** -- low-level building blocks that accept a UI5Handler slice and optional vocabulary lookup.
2. **Domain namespaces** -- 5 SAP module-specific APIs that compose the core wrappers into business flows.
## Core Wrappers
Core wrappers are the shared building blocks used by all domain intent functions. They accept a minimal `UI5HandlerSlice` interface, making them easy to unit test with mocks:
```typescript
interface UI5HandlerSlice {
control(selector: UI5Selector, options?: { timeout?: number }): Promise;
click(selector: UI5Selector): Promise;
fill(selector: UI5Selector, value: string): Promise;
select(selector: UI5Selector, key: string): Promise;
getText(selector: UI5Selector): Promise;
waitForUI5(timeout?: number): Promise;
}
```
### Available Core Wrappers
| Wrapper | Purpose |
| ------------------- | ------------------------------------------------------------------ |
| `fillField` | Fill a UI5 input control, with optional vocabulary term resolution |
| `clickButton` | Click a button by text label or selector |
| `selectOption` | Select an option in a dropdown/ComboBox |
| `assertField` | Assert the value of a field matches expected text |
| `navigateAndSearch` | Navigate to an app via FLP and execute a search |
| `confirmAndWait` | Click a confirmation button and wait for UI5 stability |
| `waitForSave` | Wait for a save operation to complete (message toast) |
### Vocabulary Integration
Core wrappers accept an optional `VocabLookup` to resolve business terms to UI5 selectors:
```typescript
interface VocabLookup {
getFieldSelector(term: string, domain?: string): Promise;
}
// With vocabulary: "Vendor" resolves to { id: 'vendorInput' }
await fillField(ui5, vocab, 'Vendor', '100001', { domain: 'procurement' });
// Without vocabulary: pass a selector directly
await fillField(ui5, undefined, { id: 'vendorInput' }, '100001');
```
When a vocabulary term cannot be resolved, the wrapper returns an `IntentResult` with `status: 'error'` and helpful suggestions.
## IntentResult Envelope
All intent functions return a typed `IntentResult`:
```typescript
interface IntentResult {
readonly status: 'success' | 'error' | 'partial';
readonly data?: T;
readonly error?: { readonly code: string; readonly message: string };
readonly metadata: {
readonly duration: number;
readonly retryable: boolean;
readonly suggestions: string[];
readonly intentName: string;
readonly sapModule: string;
readonly stepsExecuted: string[];
};
}
```
Example usage:
```typescript
const result = await procurement.createPurchaseOrder(ui5, vocab, {
vendor: '100001',
material: 'RAW-0001',
quantity: 100,
plant: '1000',
});
if (result.status === 'success') {
console.log(`PO created: ${result.data}`);
console.log(`Steps: ${result.metadata.stepsExecuted.join(' -> ')}`);
// Steps: navigate -> fillVendor -> fillMaterial -> fillQuantity -> save
}
```
## IntentOptions
All domain functions accept optional `IntentOptions`:
```typescript
interface IntentOptions {
skipNavigation?: boolean; // Skip FLP navigation (app already open)
timeout?: number; // Override default timeout (ms)
validateViaOData?: boolean; // Validate result via OData GET after save
}
// Skip navigation when already on the correct page
const result = await procurement.createPurchaseOrder(ui5, vocab, poData, {
skipNavigation: true,
timeout: 60_000,
});
```
## Domain Namespaces
### Procurement (MM -- Materials Management)
```typescript
// Create a purchase order
await procurement.createPurchaseOrder(ui5, vocab, {
vendor: '100001',
material: 'RAW-0001',
quantity: 100,
plant: '1000',
});
// Search purchase orders
await procurement.searchPurchaseOrders(ui5, vocab, { vendor: '100001' });
// Display purchase order details
await procurement.displayPurchaseOrder(ui5, vocab, '4500012345');
```
### Sales (SD -- Sales and Distribution)
```typescript
// Create a sales order
await sales.createSalesOrder(ui5, vocab, {
customer: '200001',
material: 'FG-1000',
quantity: 50,
salesOrganization: '1000',
});
// Search sales orders
await sales.searchSalesOrders(ui5, vocab, { customer: '200001' });
```
### Finance (FI -- Financial Accounting)
```typescript
// Post a journal entry
await finance.postJournalEntry(ui5, vocab, {
documentDate: '2026-02-20',
postingDate: '2026-02-20',
lineItems: [
{ glAccount: '400000', debitCredit: 'S', amount: 1000 },
{ glAccount: '110000', debitCredit: 'H', amount: 1000 },
],
});
// Post a vendor invoice
await finance.postVendorInvoice(ui5, vocab, {
vendor: '100001',
invoiceDate: '2026-02-20',
amount: 5000,
currency: 'EUR',
});
// Process a vendor payment
await finance.processPayment(ui5, vocab, {
vendor: '100001',
amount: 5000,
paymentDate: '2026-02-28',
});
```
### Manufacturing (PP -- Production Planning)
```typescript
// Create a production order
await manufacturing.createProductionOrder(ui5, vocab, {
material: 'FG-1000',
plant: '1000',
quantity: 50,
});
// Confirm a production order
await manufacturing.confirmProductionOrder(ui5, vocab, {
orderNumber: '1000012',
quantity: 50,
});
```
### Master Data (MD -- Cross-Module)
```typescript
// Create a vendor master
await masterData.createVendor(ui5, vocab, {
name: 'Acme GmbH',
country: 'DE',
taxId: 'DE123456789',
});
// Create a customer master
await masterData.createCustomer(ui5, vocab, {
name: 'Globex Corp',
country: 'US',
salesOrganization: '1000',
});
// Create a material master
await masterData.createMaterial(ui5, vocab, {
materialNumber: 'RAW-0001',
description: 'Raw material A',
materialType: 'ROH',
baseUnit: 'KG',
});
```
## Data Shapes
Each domain defines typed data interfaces:
| Type | Domain | Purpose |
| ---------------------------- | ------ | ------------------------------------ |
| `JournalEntryData` | FI | General-ledger journal entry posting |
| `VendorInvoiceData` | FI | Accounts-payable vendor invoice |
| `PaymentData` | FI | Outgoing payment processing |
| `ProductionOrderData` | PP | Shop-floor order creation |
| `ProductionConfirmationData` | PP | Production order confirmation |
| `VendorMasterData` | MD | Vendor master data creation |
| `CustomerMasterData` | MD | Customer master data creation |
| `MaterialMasterData` | MD | Material master data creation |
## Architecture
The intent layer follows strict dependency rules:
```
Domain functions (procurement.ts, sales.ts, etc.)
|
v
Core wrappers (core-wrappers.ts)
|
v
UI5HandlerSlice (structural interface) + VocabLookup (optional)
```
- Core wrappers do NOT import from any domain file (no circular dependencies).
- Domain functions compose core wrappers into SAP business flows.
- The `UI5HandlerSlice` is a structural sub-type -- it does not import the full `UI5Handler` class.
- The `VocabLookup` interface is declared inline to avoid hard dependencies on the vocabulary module.
---
## Vocabulary System
The `playwright-praman/vocabulary` sub-path provides a controlled vocabulary system for SAP S/4HANA business term resolution. It maps natural-language terms to UI5 selectors with fuzzy matching, synonym resolution, and cross-domain search.
## Overview
```typescript
const svc = createVocabularyService();
await svc.loadDomain('procurement');
const results = await svc.search('vendor');
```
## SAP Domain JSON Files
The vocabulary system ships with 6 SAP domain JSON files in `src/vocabulary/domains/`:
| File | Domain | SAP Module | Description |
| -------------------- | ------------- | ---------- | -------------------------------------------------- |
| `procurement.json` | procurement | MM | Materials Management -- vendors, POs, GR |
| `sales.json` | sales | SD | Sales & Distribution -- customers, SOs, deliveries |
| `finance.json` | finance | FI | Financial Accounting -- GL, AP, AR |
| `manufacturing.json` | manufacturing | PP | Production Planning -- orders, BOMs, routings |
| `warehouse.json` | warehouse | WM/EWM | Warehouse Management -- storage, picking |
| `quality.json` | quality | QM | Quality Management -- inspections, lots |
Each JSON file contains vocabulary terms with synonyms and optional UI5 selectors:
```json
{
"supplier": {
"name": "supplier",
"synonyms": ["vendor", "vendorId", "supplierId", "LIFNR"],
"domain": "procurement",
"sapField": "LIFNR",
"selector": {
"controlType": "sap.m.Input",
"properties": { "labelFor": "Supplier" }
}
}
}
```
## VocabularyService API
### Creating the Service
```typescript
const svc: VocabularyService = createVocabularyService();
```
### Loading Domains
Domains are loaded lazily on demand:
```typescript
await svc.loadDomain('procurement');
await svc.loadDomain('finance');
// Only procurement and finance terms are in memory
```
### Searching Terms
```typescript
// Search across all loaded domains
const results = await svc.search('vendor');
// Search within a specific domain
const results = await svc.search('vendor', 'procurement');
```
Returns `VocabularySearchResult[]` sorted by confidence descending:
```typescript
interface VocabularySearchResult {
readonly term: string; // 'supplier'
readonly synonyms: string[]; // ['vendor', 'vendorId', 'supplierId']
readonly confidence: number; // 0.0 to 1.0
readonly domain: SAPDomain; // 'procurement'
readonly sapField?: string; // 'LIFNR'
readonly selector?: UI5Selector;
}
```
### Resolving Field Selectors
When a term matches with high enough confidence (>= 0.85), you can get its UI5 selector directly:
```typescript
const selector = await svc.getFieldSelector('vendor', 'procurement');
if (selector) {
// { controlType: 'sap.m.Input', properties: { labelFor: 'Supplier' } }
await ui5.fill(selector, '100001');
}
```
Returns `undefined` when:
- No match reaches the 0.85 confidence threshold.
- Multiple matches score above 0.7 but below 0.85 (ambiguous).
### Getting Suggestions
For autocomplete and disambiguation:
```typescript
const suggestions = await svc.getSuggestions('sup');
// ['supplier', 'supplierName', 'supplyPlant', ...]
// Limit results
const top3 = await svc.getSuggestions('sup', 3);
```
### Service Statistics
```typescript
const stats = svc.getStats();
// {
// loadedDomains: ['procurement', 'sales'],
// totalTerms: 142,
// cacheHits: 37,
// cacheMisses: 5,
// }
```
## Normalization and Matching Algorithm
The vocabulary matcher uses a tiered scoring system:
### Match Tiers
| Tier | Confidence | Criteria |
| ------- | ---------- | ------------------------------------------------------------- |
| Exact | 1.0 | Query exactly matches term name or synonym (case-insensitive) |
| Prefix | 0.9 | Term name starts with the query string |
| Partial | 0.7 | Term name contains the query string |
| Fuzzy | 0.5 | Levenshtein distance <= 3 characters |
### Normalization
Before matching, both the query and term names are normalized:
- Converted to lowercase
- Leading/trailing whitespace trimmed
### Levenshtein Distance
The fuzzy matching tier uses Levenshtein edit distance (standard DP matrix approach). Terms within 3 edits of the query are considered fuzzy matches:
```
levenshtein('vendor', 'vendro') = 2 --> Fuzzy match (0.5)
levenshtein('vendor', 'vend') = 2 --> Fuzzy match (0.5)
levenshtein('vendor', 'xyz') = 6 --> No match
```
### Scoring Priority
When multiple terms match, results are sorted by:
1. Confidence score (descending)
2. Within the same confidence tier, exact matches on synonyms score the same as exact matches on the canonical name
### Disambiguation
The `getFieldSelector()` method includes disambiguation logic. If there are multiple high-confidence matches (0.7-0.85 range) without a clear winner, the method returns `undefined` rather than guessing, allowing the caller to present options to the user or AI agent.
## Integration with Intents
The vocabulary system integrates with the intent API through the `VocabLookup` interface:
```typescript
const vocab = createVocabularyService();
await vocab.loadDomain('procurement');
// fillField uses vocab to resolve 'Vendor' to a UI5 selector
await fillField(ui5, vocab, 'Vendor', '100001', { domain: 'procurement' });
```
## Supported Domain Types
```typescript
type SAPDomain =
| 'procurement' // MM - Materials Management
| 'sales' // SD - Sales & Distribution
| 'finance' // FI - Financial Accounting
| 'manufacturing' // PP - Production Planning
| 'warehouse' // WM/EWM - Warehouse Management
| 'quality'; // QM - Quality Management
```
## Extending the Vocabulary
To add terms to an existing domain, edit the corresponding JSON file in `src/vocabulary/domains/`. Each term must have:
- `name` -- canonical term name (used as the Map key)
- `synonyms` -- array of alternative names
- `domain` -- must match one of the 6 `SAPDomain` values
- `sapField` (optional) -- the underlying ABAP field name
- `selector` (optional) -- UI5 selector for direct control discovery
---
## Discovery and Interaction Strategies
Praman uses two configurable strategy systems to find and interact with UI5 controls.
**Discovery strategies** determine _how controls are located_ in the browser.
**Interaction strategies** determine _how actions are performed_ on those controls.
Both are independently configurable via environment variables.
## Part 1: Control Discovery Strategies
Discovery is the process of finding a UI5 control on the page given a `UI5Selector`. Praman runs
a **priority chain** — it tries each configured strategy in order and stops at the first match.
### The Discovery Flow
```
┌──────────────────────┐
│ ui5.control(selector)│
└──────────┬───────────┘
│
v
┌──────────────────────┐
│ Tier 0: CACHE │ In-memory proxy cache
│ (always first) │ Zero cost on repeat lookups
└──────────┬───────────┘
│ miss
v
┌──────────────────────┐
│ ensureBridgeInjected │ Inject Praman bridge into
│ (idempotent) │ the browser if not present
└──────────┬───────────┘
│
v
┌──────────────────────────────────────┐
│ getDiscoveryPriorities(selector, │
│ configuredStrategies) │
│ │
│ If ID-only selector AND direct-id │
│ is configured: promote direct-id │
│ to first position after cache │
└──────────────────┬───────────────────┘
│
v
┌────────────────────────────────────────────┐
│ For each strategy in priority order: │
│ │
│ ┌─ direct-id ──────────────────────────┐ │
│ │ Strips selector to { id } only │ │
│ │ Calls bridge.getById(id) │ │
│ │ Fastest: single lookup, no scanning │ │
│ └──────────────────────────────────────┘ │
│ │ null │
│ v │
│ ┌─ recordreplay ───────────────────────┐ │
│ │ Passes full selector │ │
│ │ Tries Tier 1 (getById) first │ │
│ │ Then Tier 2 (registry scan) │ │
│ │ Then Tier 3 (RecordReplay API) │ │
│ └──────────────────────────────────────┘ │
│ │ null │
│ v │
│ ┌─ registry ───────────────────────────┐ │
│ │ Forces full registry scan (Tier 2) │ │
│ │ Skips Tier 1 entirely │ │
│ │ Matches by type, properties, bindings│ │
│ └──────────────────────────────────────┘ │
│ │ null │
│ v │
│ throw ControlError │
│ ERR_CONTROL_NOT_FOUND │
└────────────────────────────────────────────┘
```
_Source: [discovery.ts](https://github.com/nicheui/praman/blob/main/src/proxy/discovery.ts) lines 145–201,
[discovery-factory.ts](https://github.com/nicheui/praman/blob/main/src/proxy/discovery-factory.ts) lines 74–101_
### Strategy Details
#### `direct-id` — Direct ID Lookup (Tier 1)
The fastest discovery path. Strips the selector to `{ id }` only and calls the bridge's
`getById()` accessor, which resolves the control in a single call.
**SAP UI5 APIs used:**
| API | What It Does | Reference |
| --------------------------------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `sap.ui.core.Element.registry` | Access the global element registry (UI5 >= 1.84) | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Element%23methods/sap.ui.core.Element.registry) |
| `sap.ui.core.ElementRegistry` | Legacy registry accessor (UI5 < 1.84) | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.ElementRegistry) |
| `control.getId()` | Returns the control's stable ID | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Element%23methods/getId) |
| `control.getMetadata().getName()` | Returns the fully qualified type (e.g., `sap.m.Button`) | [API Reference](https://ui5.sap.com/#/api/sap.ui.base.ManagedObject%23methods/getMetadata) |
**When it works:** The selector has an `id` field and that ID exists in the UI5 registry.
**When it skips:** The selector has no `id` field, or the ID is not found.
**Behavior in code** — `tryStrategy('direct-id', ...)` at
[discovery.ts:79–89](https://github.com/nicheui/praman/blob/main/src/proxy/discovery.ts#L79-L89):
```typescript
// Passes id-only selector for fastest getById() path
const result = await page.evaluate(browserFindControl, {
selector: { id: selector.id },
bridgeNs: BRIDGE_GLOBALS.NAMESPACE,
preferVisibleControls,
});
```
#### `recordreplay` — SAP RecordReplay API (Tier 3)
Uses SAP's `RecordReplay.findDOMElementByControlSelector()` to resolve the full selector
(including `controlType`, `properties`, `bindingPath`, `ancestor`, etc.) to a DOM element,
then bridges the DOM element back to a UI5 control via `Element.closestTo()`.
**SAP UI5 APIs used:**
| API | What It Does | Reference |
| ------------------------------------------------ | ---------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `RecordReplay.findDOMElementByControlSelector()` | Resolves a UI5 selector to a DOM element | [API Reference](https://ui5.sap.com/#/api/sap.ui.test.RecordReplay%23methods/sap.ui.test.RecordReplay.findDOMElementByControlSelector) |
| `sap.ui.core.Element.closestTo(domElement)` | Finds the UI5 control that owns a DOM element (UI5 >= 1.106) | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Element%23methods/sap.ui.core.Element.closestTo) |
| `jQuery(domElement).control(0)` | Legacy fallback: jQuery UI5 plugin for DOM-to-control resolution | — |
**When it works:** UI5 >= 1.94 with the `sap.ui.test.RecordReplay` module loaded.
**When it skips:** RecordReplay is not available (older UI5 versions or module not loaded).
**Internal sub-chain:** When `recordreplay` is configured, the browser script actually tries
Tier 1 (getById) first, then Tier 2 (registry scan), then Tier 3 (RecordReplay API) — this is
an internal optimization, not visible to the user.
**Behavior in code** — `tryStrategy('recordreplay', ...)` at
[discovery.ts:92–101](https://github.com/nicheui/praman/blob/main/src/proxy/discovery.ts#L92-L101):
```typescript
// Passes full selector for RecordReplay.findDOMElementByControlSelector()
const result = await page.evaluate(browserFindControl, {
selector: { ...selector },
bridgeNs: BRIDGE_GLOBALS.NAMESPACE,
preferVisibleControls,
});
```
#### `registry` — Full Registry Scan (Tier 2)
Scans all instantiated UI5 controls via `registry.all()` and matches by `controlType`,
`properties`, `bindingPath`, `viewName`, and visibility. The most thorough but slowest
strategy.
**SAP UI5 APIs used:**
| API | What It Does | Reference |
| ------------------------------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| `registry.all()` | Returns all instantiated UI5 controls as a map | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.ElementRegistry%23methods/all) |
| `control.getVisible()` | Check visibility (used when `preferVisibleControls` is on) | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Control%23methods/getVisible) |
| `control.getProperty(name)` | Read any property by name for matching | [API Reference](https://ui5.sap.com/#/api/sap.ui.base.ManagedObject%23methods/getProperty) |
| `control.getBinding(name)` | Get data binding for a property | [API Reference](https://ui5.sap.com/#/api/sap.ui.base.ManagedObject%23methods/getBinding) |
| `binding.getPath()` | Read the OData binding path (e.g., `/Products/0/Name`) | [API Reference](https://ui5.sap.com/#/api/sap.ui.model.Binding%23methods/getPath) |
| `control.getParent()` | Walk the control tree for ancestor/view matching | [API Reference](https://ui5.sap.com/#/api/sap.ui.base.ManagedObject%23methods/getParent) |
| `metadata.getAllProperties()` | Get all property definitions for a control type | [API Reference](https://ui5.sap.com/#/api/sap.ui.base.ManagedObjectMetadata%23methods/getAllProperties) |
| `metadata.getAllAggregations()` | Get all aggregation definitions | [API Reference](https://ui5.sap.com/#/api/sap.ui.base.ManagedObjectMetadata%23methods/getAllAggregations) |
**When it works:** Always — the element registry is available in all UI5 versions.
**When it's slow:** Pages with thousands of controls (large List Reports, complex Object Pages).
**Behavior in code** — `tryStrategy('registry', ...)` at
[discovery.ts:103–114](https://github.com/nicheui/praman/blob/main/src/proxy/discovery.ts#L103-L114):
```typescript
// Forces Tier 2 registry scan, skipping Tier 1 direct-id
const result = await page.evaluate(browserFindControl, {
selector: { ...selector },
bridgeNs: BRIDGE_GLOBALS.NAMESPACE,
preferVisibleControls,
forceRegistryScan: true,
});
```
### Smart Priority Promotion
The discovery factory at
[discovery-factory.ts:84–93](https://github.com/nicheui/praman/blob/main/src/proxy/discovery-factory.ts#L84-L93)
automatically promotes `direct-id` to first position for ID-only selectors, regardless of the
configured order:
```
Config: discoveryStrategies: ['recordreplay', 'direct-id']
Selector: { id: 'btn1' } ← ID-only
Actual: ['cache', 'direct-id', 'recordreplay'] ← direct-id promoted
Selector: { controlType: 'sap.m.Button', properties: { text: 'Save' } } ← complex
Actual: ['cache', 'recordreplay', 'direct-id'] ← config order preserved
```
### Comparison Table
| | `direct-id` | `recordreplay` | `registry` |
| -------------------- | -------------------------------- | ---------------------------------------------------- | -------------------------- |
| **Speed** | Fastest (single lookup) | Medium (API call + DOM bridge) | Slowest (full scan) |
| **Selector support** | `id` only | Full selector (type, properties, bindings, ancestor) | Full selector |
| **UI5 version** | All versions | >= 1.94 | All versions |
| **Best for** | Known stable IDs | Complex selectors, SAP standard apps | Dynamic controls, fallback |
| **Risk** | Fails if ID changes across views | Requires RecordReplay module loaded | Slow on large pages |
### UI5 Version Compatibility for Discovery
```
┌─────────────────────────────────────────────────────────────────────┐
│ UI5 Version │ direct-id │ registry │ recordreplay │
│────────────────┼────────────┼────────────┼──────────────────────────│
│ < 1.84 │ ✓ (legacy │ ✓ (Element │ ✗ │
│ │ registry) │ Registry) │ │
│ 1.84 – 1.93 │ ✓ │ ✓ (Element.│ ✗ │
│ │ │ registry) │ │
│ >= 1.94 │ ✓ │ ✓ │ ✓ │
│ >= 1.106 │ ✓ │ ✓ │ ✓ (uses closestTo) │
└─────────────────────────────────────────────────────────────────────┘
```
_Source: Registry fallback at
[find-control-fn.ts:189–198](https://github.com/nicheui/praman/blob/main/src/bridge/browser-scripts/find-control-fn.ts#L189-L198),
closestTo vs jQuery at
[find-control-fn.ts:315–326](https://github.com/nicheui/praman/blob/main/src/bridge/browser-scripts/find-control-fn.ts#L315-L326)_
---
## Part 2: Interaction Strategies
Interaction strategies define how Praman performs `press()`, `enterText()`, and `select()`
operations on discovered controls. Each strategy uses different SAP UI5 and DOM APIs with its
own internal fallback chain.
### The Interaction Flow
```
┌─────────────────────────────┐
│ ui5.click({ id: 'btn1' }) │
└──────────────┬──────────────┘
│
v
┌─────────────────────────────┐
│ Discovery (Part 1 above) │
│ Returns: control proxy │
│ with controlId │
└──────────────┬──────────────┘
│
v
┌─────────────────────────────┐
│ strategy.press(page, id) │
│ │
│ Configured at worker start │
│ via pramanConfig fixture │
└──────────────┬──────────────┘
│
┌────────────────┼────────────────┐
│ │ │
v v v
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ ui5-native │ │ dom-first │ │ opa5 │
│ (default) │ │ │ │ │
│ firePress() │ │ dom.click() │ │ RecordReplay │
│ fireSelect() │ │ firePress() │ │ .interact..()│
│ fireTap() │ │ fireSelect() │ │ firePress() │
│ dom.click() │ │ fireTap() │ │ fireSelect() │
└──────────────┘ └──────────────┘ └──────────────┘
```
_Source: [strategy-factory.ts](https://github.com/nicheui/praman/blob/main/src/bridge/interaction-strategies/strategy-factory.ts) lines 38–51,
[module-fixtures.ts](https://github.com/nicheui/praman/blob/main/src/fixtures/module-fixtures.ts) line 324_
### Strategy Interface
All three strategies implement the same contract defined in
[strategy.ts](https://github.com/nicheui/praman/blob/main/src/bridge/interaction-strategies/strategy.ts):
```typescript
interface InteractionStrategy {
readonly name: string;
press(page: Page, controlId: string): Promise;
enterText(page: Page, controlId: string, text: string): Promise;
select(page: Page, controlId: string, itemId: string): Promise;
}
```
### `ui5-native` — Default Strategy
Uses direct UI5 control event firing. The most reliable strategy for standard SAP Fiori
applications because it triggers the same code paths that real user interactions would.
**SAP UI5 APIs used:**
| API | Operation | What It Does | Reference |
| ----------------------------------- | --------- | --------------------------------------------- | ---------------------------------------------------------------------------------- |
| `control.firePress()` | press | Fires the press event on buttons, links | [API Reference](https://ui5.sap.com/#/api/sap.m.Button%23events/press) |
| `control.fireSelect()` | press | Fires the select event on selectable controls | [API Reference](https://ui5.sap.com/#/api/sap.m.Select%23events/change) |
| `control.fireTap()` | press | Fires tap event (legacy, pre-1.50 controls) | — |
| `control.getDomRef()` | press | Gets the DOM element for DOM click fallback | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Element%23methods/getDomRef) |
| `control.setValue(text)` | enterText | Sets text value on input controls | [API Reference](https://ui5.sap.com/#/api/sap.m.InputBase%23methods/setValue) |
| `control.fireLiveChange({ value })` | enterText | Fires live change as user types | [API Reference](https://ui5.sap.com/#/api/sap.m.Input%23events/liveChange) |
| `control.fireChange({ value })` | enterText | Fires change when input loses focus | [API Reference](https://ui5.sap.com/#/api/sap.m.InputBase%23events/change) |
| `control.setSelectedKey(key)` | select | Sets the selected key on dropdowns | [API Reference](https://ui5.sap.com/#/api/sap.m.Select%23methods/setSelectedKey) |
| `control.fireSelectionChange()` | select | Fires selection change event | [API Reference](https://ui5.sap.com/#/api/sap.m.Select%23events/change) |
**Fallback chains in code:**
```
press() — ui5-native-strategy.ts lines 65–97
├── firePress() ─┐
├── fireSelect() ─┤ both fire if available
│ └── success ──────→ done
├── fireTap() → done
└── getDomRef().click() → done
└── fail → throw ControlError
enterText() — ui5-native-strategy.ts lines 100–128
Sequential (all called, not fallback):
├── setValue(text)
├── fireLiveChange({ value: text })
└── fireChange({ value: text })
select() — ui5-native-strategy.ts lines 131–161
├── setSelectedKey(itemId)
├── fireSelectionChange({ selectedItem }) → done
└── fireChange({ selectedItem }) → done (fallback)
```
:::info Why firePress and fireSelect both fire
The ui5-native strategy fires both `firePress()` and `fireSelect()` on the same control
(lines 74–76). This is intentional — some controls like `SegmentedButton` items respond to
`fireSelect()` but not `firePress()`, while standard buttons respond to `firePress()` only.
Firing both covers both cases.
:::
### `dom-first` — DOM Events First
Performs DOM-level interactions first, then falls back to UI5 events. This strategy uses
Playwright's `page.evaluate(fn, args)` with typed argument objects — no string interpolation —
which eliminates script injection risks.
Best for form controls and custom composite controls that respond to DOM events but don't
expose `firePress()`.
**SAP UI5 and DOM APIs used:**
| API | Operation | Type | Reference |
| ---------------------------------------- | --------- | ---- | ------------------------------------------------------------------------------------------- |
| `control.getDomRef()` | press | UI5 | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Element%23methods/getDomRef) |
| `dom.click()` | press | DOM | [MDN Reference](https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/click) |
| `control.getFocusDomRef()` | enterText | UI5 | [API Reference](https://ui5.sap.com/#/api/sap.ui.core.Element%23methods/getFocusDomRef) |
| `HTMLInputElement.value = text` | enterText | DOM | [MDN Reference](https://developer.mozilla.org/en-US/docs/Web/API/HTMLInputElement/value) |
| `dom.dispatchEvent(new Event('input'))` | enterText | DOM | [MDN Reference](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/dispatchEvent) |
| `dom.dispatchEvent(new Event('change'))` | enterText | DOM | [MDN Reference](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/dispatchEvent) |
| `control.setValue(text)` | enterText | UI5 | [API Reference](https://ui5.sap.com/#/api/sap.m.InputBase%23methods/setValue) |
| `control.fireChange({ value })` | enterText | UI5 | [API Reference](https://ui5.sap.com/#/api/sap.m.InputBase%23events/change) |
| `control.firePress()` | press | UI5 | [API Reference](https://ui5.sap.com/#/api/sap.m.Button%23events/press) |
**Fallback chains in code:**
```
press() — dom-first-strategy.ts lines 79–135
├── getDomRef() → dom.click() → done (DOM first)
├── firePress() ─┐
├── fireSelect() ─┤ both fire if available
│ └── success ──────────────────→ done
├── fireTap() → done
└── fail → throw ControlError
enterText() — dom-first-strategy.ts lines 138–193
├── getFocusDomRef() or getDomRef()
│ └── if tagName === 'INPUT':
│ ├── input.value = text (DOM property set)
│ ├── dispatchEvent('input') (synthetic DOM event)
│ └── dispatchEvent('change') → done (early return)
└── UI5 fallback:
├── setValue(text)
└── fireChange({ value: text }) → done
select() — dom-first-strategy.ts lines 196–243
Same as ui5-native (no DOM shortcut for select):
├── setSelectedKey(itemId)
├── fireSelectionChange({ selectedItem }) → done
└── fireChange({ selectedItem }) → done
```
### `opa5` — SAP RecordReplay API
Uses SAP's official `RecordReplay.interactWithControl()` API from the OPA5 testing framework.
Most compatible with SAP testing standards. Includes an optional **auto-waiter** that polls
for UI5 stability before each interaction.
Falls back to native UI5 fire\* methods if RecordReplay is not available.
:::warning UI5 Version Requirement
The opa5 strategy requires UI5 >= 1.94 for the `sap.ui.test.RecordReplay` API. On older
versions, it falls back to `firePress()` / `setValue()` — effectively behaving like
ui5-native with reduced coverage.
:::
**SAP UI5 APIs used:**
| API | Operation | What It Does | Reference |
| ------------------------------------ | --------- | ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `RecordReplay.interactWithControl()` | all | SAP's official control interaction API | [API Reference](https://ui5.sap.com/#/api/sap.ui.test.RecordReplay%23methods/sap.ui.test.RecordReplay.interactWithControl) |
| `RecordReplay.getAutoWaiter()` | all | Gets the OPA5 auto-waiter instance | [API Reference](https://ui5.sap.com/#/api/sap.ui.test.RecordReplay%23methods/sap.ui.test.RecordReplay.getAutoWaiter) |
| `autoWaiter.hasToWait()` | all | Returns `true` while UI5 has pending async work | [API Reference](https://ui5.sap.com/#/api/sap.ui.test.autowaiter.autoWaiter%23methods/hasToWait) |
| `control.firePress()` | press | Fallback if RecordReplay unavailable | [API Reference](https://ui5.sap.com/#/api/sap.m.Button%23events/press) |
| `control.fireSelect()` | press | Fallback if RecordReplay unavailable | — |
| `control.setValue(text)` | enterText | Fallback if RecordReplay unavailable | [API Reference](https://ui5.sap.com/#/api/sap.m.InputBase%23methods/setValue) |
| `control.setSelectedKey(key)` | select | Fallback if RecordReplay unavailable | [API Reference](https://ui5.sap.com/#/api/sap.m.Select%23methods/setSelectedKey) |
**OPA5-specific configuration** — defined in
[schema.ts:119–123](https://github.com/nicheui/praman/blob/main/src/core/config/schema.ts#L119-L123):
| Field | Default | Description |
| -------------------- | ------- | ------------------------------------------------------ |
| `interactionTimeout` | `5000` | Timeout in ms for `RecordReplay.interactWithControl()` |
| `autoWait` | `true` | Poll `hasToWait()` before each interaction |
| `debug` | `false` | Log `[praman:opa5]` messages to browser console |
**Fallback chains in code:**
```
press() — opa5-strategy.ts lines 82–140
if RecordReplay available:
│ ├── autoWaiter.hasToWait() polling loop (100ms interval)
│ └── RecordReplay.interactWithControl({
│ selector: { id },
│ interactionType: 'PRESS',
│ interactionTimeout
│ }) → done
│
if RecordReplay NOT available (fallback):
├── firePress() ─┐
└── fireSelect() ─┤ → done
└── fail → throw ControlError
enterText() — opa5-strategy.ts lines 143–194
if RecordReplay available:
│ ├── autoWaiter polling
│ └── RecordReplay.interactWithControl({
│ interactionType: 'ENTER_TEXT',
│ enterText: text
│ }) → done
│
if RecordReplay NOT available:
└── setValue(text) → done
select() — opa5-strategy.ts lines 197–246
if RecordReplay available:
│ ├── autoWaiter polling
│ └── RecordReplay.interactWithControl({
│ interactionType: 'PRESS' ← uses PRESS, not SELECT
│ }) → done
│
if RecordReplay NOT available:
└── setSelectedKey(itemId) → done
```
### Understanding SAP Pages: DOM, UI5, and Web Components Together
Real SAP Fiori applications often render a mix of control types on the same page:
```
┌─ SAP Fiori Launchpad (FLP) ────────────────────────────────┐
│ │
│ ┌─ Shell Bar ──────────────────────────────────────────┐ │
│ │ Pure DOM elements (custom CSS, no UI5 control) │ │
│ │ Example: logo image, custom toolbar buttons │ │
│ │ → Only dom-first can interact with these │ │
│ └──────────────────────────────────────────────────────┘ │
│ │
│ ┌─ App Container ──────────────────────────────────────┐ │
│ │ │ │
│ │ ┌─ Standard UI5 Controls ──────────────────────┐ │ │
│ │ │ sap.m.Button, sap.m.Input, sap.m.Table │ │ │
│ │ │ → All strategies work (ui5-native best) │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─ Smart Controls (sap.ui.comp) ───────────────┐ │ │
│ │ │ SmartField wraps inner Input/ComboBox/etc. │ │ │
│ │ │ → ui5-native works on outer SmartField │ │ │
│ │ │ → dom-first can reach inner element │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─ UI5 Web Components ─────────────────────────┐ │ │
│ │ │ ui5-button, ui5-input (Shadow DOM) │ │ │
│ │ │ → dom-first required (shadow DOM piercing) │ │ │
│ │ │ → ui5-native cannot fire events on WC │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─ Third-Party / Custom Controls ──────────────┐ │ │
│ │ │ Custom composite controls, chart libraries │ │ │
│ │ │ → dom-first often the only option │ │ │
│ │ │ → ui5-native works if they extend ManagedObj │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
This is why choosing the right strategy matters. No single strategy handles every control type
on every page. The built-in fallback chains in each strategy are designed to cover the most
common combinations.
### Comparison Table
| | `ui5-native` | `dom-first` | `opa5` |
| ------------------------- | ------------------------- | ---------------------------- | ---------------------------- |
| **Speed** | Fast | Fast | Medium (auto-waiter polling) |
| **Standard UI5 controls** | Best | Good (via fallback) | Best (SAP official) |
| **Custom composites** | May fail (no `firePress`) | Best | May fail |
| **UI5 Web Components** | Does not work | Works (DOM events) | Does not work |
| **Plain DOM elements** | Falls back to DOM click | Works natively | Does not work |
| **OData model updates** | `setValue` + events | DOM events may miss bindings | RecordReplay handles |
| **UI5 version** | All | All | >= 1.94 |
| **SAP compliance** | No official API | No official API | Yes (`RecordReplay`) |
| **Fallback chain depth** | 4 levels | 4 levels | 2 levels + RecordReplay |
---
## Part 3: Configuring Strategies — A Practical Guide
### How Configuration Flows Through the Code
```
Environment Variable Zod Schema Defaults
PRAMAN_INTERACTION_STRATEGY=dom-first interactionStrategy: 'ui5-native'
PRAMAN_DISCOVERY_STRATEGIES=... discoveryStrategies: ['direct-id','recordreplay']
│ │
v v
┌──────────────────────────────────────────────────────────┐
│ loadConfig() — src/core/config/loader.ts:142 │
│ Merges: schema defaults ← env vars (env wins) │
│ Validates with Zod, returns Readonly │
└────────────────────────┬─────────────────────────────────┘
│
v
┌──────────────────────────────────────────────────────────┐
│ pramanConfig fixture — src/fixtures/core-fixtures.ts:164│
│ Worker-scoped: loaded ONCE per Playwright worker │
│ Frozen: Object.freeze(config) │
└────────────────────────┬─────────────────────────────────┘
│
┌────────────┴────────────┐
v v
┌───────────────────────┐ ┌────────────────────────────────┐
│ createInteraction │ │ new UI5Handler({ │
│ Strategy( │ │ discoveryStrategies: │
│ config.interaction │ │ config.discoveryStrategies │
│ Strategy, │ │ }) │
│ config.opa5 │ │ │
│ ) │ │ src/fixtures/module-fixtures.ts│
│ │ │ line 337 │
│ src/fixtures/ │ └────────────────────────────────┘
│ module-fixtures.ts │
│ line 324 │
└───────────────────────┘
```
_Source: [loader.ts:62–78](https://github.com/nicheui/praman/blob/main/src/core/config/loader.ts#L62-L78)
defines the env var mappings,
[module-fixtures.ts:322–347](https://github.com/nicheui/praman/blob/main/src/fixtures/module-fixtures.ts#L322-L347)
wires config to strategy and handler_
### Setting Strategies via Environment Variables
The most direct way to configure strategies without changing code.
#### Interaction Strategy
```bash
# Use DOM-first for a test run
PRAMAN_INTERACTION_STRATEGY=dom-first npx playwright test
# Use OPA5 RecordReplay
PRAMAN_INTERACTION_STRATEGY=opa5 npx playwright test
# Use the default (ui5-native) — no env var needed
npx playwright test
```
Valid values: `ui5-native`, `dom-first`, `opa5`
_Source: env var mapping at [loader.ts:70](https://github.com/nicheui/praman/blob/main/src/core/config/loader.ts#L70)_
#### Discovery Strategies
```bash
# ID lookup first, then RecordReplay, then full registry scan
PRAMAN_DISCOVERY_STRATEGIES=direct-id,recordreplay,registry npx playwright test
# Only use RecordReplay (skip ID lookup)
PRAMAN_DISCOVERY_STRATEGIES=recordreplay npx playwright test
# Only registry scan (thorough but slow)
PRAMAN_DISCOVERY_STRATEGIES=registry npx playwright test
```
Valid values (comma-separated): `direct-id`, `recordreplay`, `registry`
_Source: env var mapping at [loader.ts:72–75](https://github.com/nicheui/praman/blob/main/src/core/config/loader.ts#L72-L75),
parsed as comma-separated array at [loader.ts:106–113](https://github.com/nicheui/praman/blob/main/src/core/config/loader.ts#L106-L113)_
#### OPA5 Tuning (when using `opa5` interaction strategy)
```bash
PRAMAN_INTERACTION_STRATEGY=opa5 npx playwright test
```
The OPA5 sub-config (`interactionTimeout`, `autoWait`, `debug`) does not have env var mappings.
These are set via inline overrides or config file only.
#### Combining Both
```bash
# DOM-first interaction + registry-based discovery with all 3 tiers
PRAMAN_INTERACTION_STRATEGY=dom-first \
PRAMAN_DISCOVERY_STRATEGIES=direct-id,recordreplay,registry \
npx playwright test
```
#### In CI/CD Pipelines
```yaml
# GitHub Actions
- name: Run SAP E2E Tests
run: npx playwright test
env:
PRAMAN_INTERACTION_STRATEGY: opa5
PRAMAN_DISCOVERY_STRATEGIES: direct-id,recordreplay
SAP_CLOUD_BASE_URL: ${{ secrets.SAP_URL }}
```
```yaml
# Azure Pipelines
- script: npx playwright test
env:
PRAMAN_INTERACTION_STRATEGY: dom-first
PRAMAN_DISCOVERY_STRATEGIES: direct-id,recordreplay,registry
```
:::danger Typo in env var values discards ALL env vars
If any single env var fails Zod validation, the loader discards **all** env var overrides
and falls back to defaults. A typo like `PRAMAN_DISCOVERY_STRATEGIES=direct-id,recordreply`
(missing `a` in replay) causes both the discovery and interaction env vars to be silently
ignored. Only a `warn`-level pino log is emitted.
This is implemented at
[loader.ts:159–171](https://github.com/nicheui/praman/blob/main/src/core/config/loader.ts#L159-L171).
Verify your values match exactly: `direct-id`, `recordreplay`, `registry`, `ui5-native`,
`dom-first`, `opa5`.
:::
### Decision Matrix: Choosing Your Strategy
#### Step 1: Choose Your Interaction Strategy
```
Is your app standard SAP Fiori (SAPUI5 controls only)?
├── YES → Does your team require SAP compliance audits?
│ ├── YES → Use opa5 (SAP's official API)
│ └── NO → Use ui5-native (default, broadest fallback)
│
└── NO → Does your app have any of these?
├── UI5 Web Components (Shadow DOM) → Use dom-first
├── Custom composite controls → Use dom-first
├── Plain DOM elements mixed with UI5 → Use dom-first
└── Standard UI5 only in older version → Use ui5-native
```
#### Step 2: Choose Your Discovery Priorities
```
Do your selectors use stable IDs (e.g., { id: 'saveBtn' })?
├── YES → Include direct-id first: ['direct-id', 'recordreplay']
│ (direct-id is auto-promoted for ID-only selectors anyway)
│
└── NO → Do you use complex selectors (controlType + properties)?
├── YES → Is UI5 >= 1.94?
│ ├── YES → ['recordreplay', 'direct-id']
│ └── NO → ['registry', 'direct-id']
│
└── Mostly binding paths or ancestor matching?
└── ['registry', 'recordreplay']
```
#### Step 3: Apply the Configuration
```bash
# Scenario A: Standard Fiori, stable IDs (most common)
# Just use defaults — no env vars needed
npx playwright test
# Scenario B: Mixed controls, need DOM access
PRAMAN_INTERACTION_STRATEGY=dom-first npx playwright test
# Scenario C: SAP compliance, complex selectors
PRAMAN_INTERACTION_STRATEGY=opa5 \
PRAMAN_DISCOVERY_STRATEGIES=recordreplay,direct-id,registry \
npx playwright test
# Scenario D: Legacy app (UI5 < 1.94), dynamic controls
PRAMAN_DISCOVERY_STRATEGIES=direct-id,registry npx playwright test
```
### Recommended Configurations by App Type
| Application Type | Interaction | Discovery | Why |
| ----------------------------- | ------------ | ------------------------------------------- | -------------------------------- |
| Standard Fiori (S/4HANA) | `ui5-native` | `['direct-id', 'recordreplay']` | Stable IDs, standard controls |
| Fiori Elements List Report | `ui5-native` | `['recordreplay', 'direct-id']` | Generated IDs, complex selectors |
| Custom Fiori App | `dom-first` | `['direct-id', 'recordreplay', 'registry']` | Custom controls may lack fire\* |
| Hybrid (UI5 + Web Components) | `dom-first` | `['direct-id', 'registry']` | Shadow DOM needs DOM events |
| Migration from OPA5 | `opa5` | `['recordreplay', 'direct-id']` | Behavioral parity with OPA5 |
| Legacy (UI5 < 1.94) | `ui5-native` | `['direct-id', 'registry']` | No RecordReplay available |
### Verifying Your Configuration
To confirm which strategies are active, enable debug logging:
```bash
PRAMAN_LOG_LEVEL=debug \
PRAMAN_INTERACTION_STRATEGY=dom-first \
PRAMAN_DISCOVERY_STRATEGIES=direct-id,recordreplay \
npx playwright test --headed
```
The pino logger will output which strategy was created and which discovery chain is active.
### Pros and Cons of Environment Variables
| Pros | Cons |
| --------------------------------------------- | ---------------------------------------------- |
| Works today, no code changes | Not version-controlled (ephemeral) |
| Easy CI/CD override per pipeline | No IDE autocomplete or type checking |
| Switch strategies per test run | One typo silently discards ALL env vars |
| No file to maintain | Team must document in CI config or README |
| Highest priority (overrides all other config) | Cannot set OPA5 sub-config (timeout, autoWait) |
## See Also
- [Selector Reference](./selectors.md) — How to write `UI5Selector` objects
- [Configuration Reference](./configuration.md) — Full config schema
- [Control Interactions](./control-interactions.md) — `ui5.click()`, `ui5.fill()`, etc.
- [Bridge Internals](./bridge-internals.md) — How the bridge injects into the browser
- [Control Proxy](./control-proxy.md) — How discovered controls become proxy objects
---
## Fixture Composition
Praman uses Playwright's `mergeTests()` to compose fixture modules into a single `test` object.
This page explains the composition pattern, how to customize it, and why it exists.
## How mergeTests() Works
Playwright's `mergeTests()` combines multiple `test.extend()` definitions into one `test` object.
Each fixture module defines its own fixtures independently, and `mergeTests()` produces a unified
test function that includes all of them.
```typescript
export const test = mergeTests(coreTest, authTest, navTest);
```
When you import from `playwright-praman`, this merge has already been done for you:
```typescript
test('all fixtures available', async ({ ui5, sapAuth, ui5Navigation }) => {
// Everything is available in a single destructure
});
```
## The Fixture Module Chain
Praman merges 12 fixture modules in a specific order:
```
coreTest → ui5, pramanConfig, pramanLogger, rootLogger, tracer
stabilityTest → ui5Stability, requestInterceptor (auto)
selectorTest → selectorRegistration (auto)
matcherTest → matcherRegistration (auto)
compatTest → playwrightCompat (auto)
navTest → ui5Navigation, btpWorkZone
authTest → sapAuth
moduleTest → ui5.table, ui5.dialog, ui5.date, ui5.odata
feTest → fe (listReport, objectPage, table, list)
aiTest → pramanAI
intentTest → intent (procurement, sales, finance, manufacturing, masterData)
shellFooterTest → ui5Shell, ui5Footer
flpLocksTest → flpLocks
flpSettingsTest → flpSettings
testDataTest → testData
```
The order matters: later modules can depend on fixtures defined by earlier modules. For example,
`navTest` depends on `ui5` from `coreTest`, and `authTest` depends on `pramanConfig`.
## Why mergeTests() Over a Monolithic File
A single giant `test.extend()` call with all fixtures creates several problems:
1. **Circular dependencies** — fixtures that reference each other in one block cause TypeScript
compilation errors
2. **Bloated imports** — every test file imports every dependency, even if unused
3. **Testing isolation** — unit-testing fixture logic requires loading the entire fixture tree
4. **Readability** — a 500-line fixture definition is unmaintainable
With `mergeTests()`, each module is independently testable, tree-shakeable, and can be composed
in any combination.
## Tree-Shaking: Use Only What You Need
If your tests only need core UI5 operations and authentication, skip the full import:
```typescript
const test = mergeTests(coreTest, authTest);
test('lightweight test', async ({ ui5, sapAuth }) => {
// Only core + auth fixtures are loaded
// AI, intents, FE helpers are not initialized
});
```
This reduces worker initialization time and avoids loading optional dependencies (e.g., LLM SDKs)
that are not installed.
## Adding Custom Fixtures
Extend the merged test object with your own fixtures using `test.extend()`:
```typescript
interface MyFixtures {
adminUser: { username: string; password: string };
appUrl: string;
}
const test = base.extend({
adminUser: async ({}, use) => {
await use({
username: process.env.ADMIN_USER ?? 'admin',
password: process.env.ADMIN_PASS ?? 'secret',
});
},
appUrl: async ({ pramanConfig }, use) => {
const baseUrl = pramanConfig.auth?.baseUrl ?? 'http://localhost:8080';
await use(`${baseUrl}/sap/bc/ui5_ui5/ui2/ushell/shells/abap/FioriLaunchpad.html`);
},
});
export { test, expect };
```
## Composing Across Test Files
For large test suites, create domain-specific test objects:
```typescript
// fixtures/procurement-test.ts
export const test = base.extend({
poDefaults: async ({}, use) => {
await use({ plant: '1000', purchOrg: '1000', companyCode: '1000' });
},
});
// fixtures/finance-test.ts
export const test = base.extend({
fiscalYear: async ({}, use) => {
await use(new Date().getFullYear().toString());
},
});
```
```typescript
// tests/procurement/create-po.spec.ts
test('create PO with defaults', async ({ ui5, intent, poDefaults }) => {
await intent.procurement.createPurchaseOrder({
vendor: '100001',
material: 'MAT-001',
quantity: 10,
plant: poDefaults.plant,
});
});
```
## Fixture Scopes
Praman fixtures use two scopes:
- **Worker-scoped** — created once per Playwright worker process. Shared across all tests in
that worker. Used for expensive initialization: `pramanConfig`, `rootLogger`, `tracer`.
- **Test-scoped** — created fresh for each test. Used for stateful objects: `ui5`, `sapAuth`,
`flpLocks`, `testData`.
Worker-scoped fixtures cannot depend on test-scoped fixtures. Test-scoped fixtures can depend
on both.
## Auto-Fixtures
Five fixtures are marked with `{ auto: 'on' }` or `{ auto: true }` and fire without being
requested in the test signature:
```typescript
// These run automatically for every test:
// - playwrightCompat (worker) — version compatibility checks
// - selectorRegistration (worker) — registers ui5= selector engine
// - matcherRegistration (worker) — registers 10 custom matchers
// - requestInterceptor (test) — blocks WalkMe/analytics scripts
// - ui5Stability (test) — auto-waits for UI5 stability after navigation
```
You never destructure these — they just work.
---
## Control Proxy Pattern
Every UI5 control discovered by Praman is wrapped in a JavaScript `Proxy` that routes method
calls through Playwright's `page.evaluate()` to the real browser-side control. This page
explains how the proxy works, what it can do, and how it compares to wdi5.
## How createControlProxy() Works
When you call `ui5.control()`, Praman:
1. Discovers the control in the browser (via the multi-strategy chain)
2. Stores the control's ID as a reference
3. Returns a JavaScript `Proxy` object that intercepts all property access
```typescript
const button = await ui5.control({ id: 'saveBtn' });
// `button` is a Proxy, not the actual UI5 control
```
:::tip Typed Returns
When you provide `controlType` as a string literal, the proxy is typed to the specific control
interface — e.g., `UI5Button` instead of `UI5ControlBase`. See [Typed Control Returns](/docs/guides/typed-controls).
:::
When you call a method on the proxy (e.g., `button.getText()`), the proxy's `get` trap:
1. Checks if the method is a **built-in safe method** (press, enterText, select, etc.)
2. Checks if the method is **blacklisted** (lifecycle/internal methods)
3. If neither, forwards the call to the browser via `page.evaluate()`
```
Node.js Browser
─────── ───────
button.getText()
→ Proxy get trap
→ method forwarder
→ page.evaluate(fn, controlId)
→ sap.ui.getCore().byId(controlId)
→ control.getText()
→ return 'Save'
← result: 'Save'
```
## The 7-Type Return System
When a method executes in the browser, the result is classified into one of 7 types:
| Return Type | Example | Proxy Behavior |
| ----------------------- | ------------------------------------- | --------------------------- |
| **Primitive** | `string`, `number`, `boolean`, `null` | Returned directly |
| **Plain object** | `{ key: 'val' }` | Returned as-is (serialized) |
| **Array of primitives** | `['a', 'b']` | Returned as-is |
| **UI5 element** | Single control reference | Wrapped in a new proxy |
| **Array of elements** | Aggregation result | Array of new proxies |
| **undefined** | Setter return / void | Returns `undefined` |
| **Non-serializable** | Circular refs, DOM nodes | Throws `ControlError` |
This means you can chain proxy calls to navigate the control tree:
```typescript
const table = await ui5.control({ id: 'poTable' });
const items = await table.getItems(); // Array of proxied controls
const firstItem = items[0];
const cells = await firstItem.getCells(); // Array of proxied controls
const text = await cells[2].getText(); // 'Active'
```
## Built-In Safe Methods
These methods have dedicated implementations with proper event firing and fallback logic:
| Method | Description |
| ---------------------- | ----------------------------------------------------------------- |
| `press()` | Fires press/tap event via interaction strategy |
| `enterText(value)` | Sets value + fires liveChange + change events |
| `select(key)` | Selects item by key or text |
| `getAggregation(name)` | Returns aggregation as proxied control array |
| `exec(fn, ...args)` | Executes an arbitrary function on the browser-side control |
| `getControlMetadata()` | Returns control metadata (type, properties, aggregations, events) |
| `getControlInfoFull()` | Returns full control introspection data |
| `retrieveMembers()` | Returns all public members of the control |
## The Method Blacklist
Praman blocks 71 static methods and 2 dynamic rules to prevent dangerous operations:
```typescript
// These throw ControlError with code 'ERR_METHOD_BLACKLISTED':
await button.destroy(); // Lifecycle method
await button.rerender(); // Rendering internal
await button.setParent(other); // Tree manipulation
await button.placeAt('container'); // DOM manipulation
```
The blacklist includes:
- **Lifecycle methods** — `destroy`, `init`, `exit`, `onBeforeRendering`, `onAfterRendering`
- **Internal methods** — `_*` (any method starting with underscore)
- **Tree manipulation** — `setParent`, `insertAggregation`, `removeAllAggregation`
- **Rendering** — `rerender`, `invalidate`, `placeAt`
- **DOM access** — `getDomRef`, `$` (returns non-serializable DOM nodes)
When a blacklisted method is called, the error includes suggestions for safe alternatives.
## Custom Function Execution
For edge cases where you need browser-side logic that no proxy method covers, use `exec()`:
```typescript
const control = await ui5.control({ id: 'mySmartField' });
// Execute arbitrary function on the browser-side control
const customData = await control.exec((ctrl) => {
const innerControl = ctrl.getInnerControls()[0];
return {
type: innerControl.getMetadata().getName(),
value: innerControl.getValue(),
};
});
```
The function passed to `exec()` runs inside `page.evaluate()` — it has access to the browser
environment and the UI5 control as its first argument. Additional arguments are serialized
and passed through.
## Anti-Thenable Guard
JavaScript's `await` operator checks for a `then` property on any object. If a UI5 control
happens to have a `then` method or property, `await proxy` would accidentally consume it.
Praman's proxy includes an anti-thenable guard:
```typescript
// The proxy intercepts 'then' access and returns undefined
// This prevents accidental Promise consumption:
const control = await ui5.control({ id: 'myControl' });
// Without the guard, this second await would break:
const text = await control.getText();
```
## Control Introspection
Use introspection methods to understand a control's capabilities at runtime:
```typescript
const control = await ui5.control({ id: 'unknownControl' });
// Full metadata
const meta = await control.getControlMetadata();
console.log(meta.controlType); // 'sap.m.Input'
console.log(meta.properties); // ['value', 'placeholder', 'enabled', ...]
console.log(meta.aggregations); // ['tooltip', 'customData', ...]
console.log(meta.events); // ['change', 'liveChange', ...]
// All public members
const members = await control.retrieveMembers();
console.log(members.methods); // ['getValue', 'setValue', 'getEnabled', ...]
```
## Comparison: wdi5 WDI5Control vs Praman UI5ControlBase
| Aspect | wdi5 WDI5Control | Praman UI5ControlBase |
| -------------------- | ------------------------------ | -------------------------------------------------------------------------------- |
| **Proxy mechanism** | Class with `getProperty(name)` | ES Proxy with `get` trap |
| **Method calls** | `control.getProperty('value')` | `control.getValue()` (direct) |
| **Return handling** | Manual unwrapping | Auto-detect 7 return types |
| **Blacklist** | Partial (small list) | 71 static + 2 dynamic rules |
| **Chaining** | Limited, must re-wrap | Automatic proxy wrapping |
| **Type safety** | Partial TypeScript | Full TypeScript with [199 typed control interfaces](/docs/guides/typed-controls) |
| **Introspection** | Basic | `getControlMetadata()`, `retrieveMembers()`, `getControlInfoFull()` |
| **Custom execution** | `executeMethod()` | `exec(fn, ...args)` |
| **Step decoration** | None | Every call wrapped in `test.step()` |
| **Caching** | None | LRU cache (200 entries, 5s TTL) |
The key difference: wdi5 uses a class-based approach where you call generic methods like
`getProperty('value')`. Praman uses an ES Proxy where you call `getValue()` directly — matching
the UI5 API surface exactly.
## Performance Considerations
- **Proxy creation is cheap** — the proxy is a thin wrapper around a control ID string
- **Method calls cross the process boundary** — each call to the proxy invokes `page.evaluate()`,
which is a round-trip to the browser. Batch operations when possible.
- **Caching** — repeated `ui5.control()` calls with the same selector hit the LRU cache (200
entries, 5s TTL), returning the existing proxy without re-discovery
- **Aggregation results** — `getItems()` on a table with 1000 rows returns 1000 proxies. Use
`ui5.table.getRows()` for bulk data access instead.
---
## Bridge Internals
:::info[Contributor Only]
This page documents Praman's internal bridge architecture. The APIs shown here use internal path aliases (`#bridge/*`) and are not accessible to end users of the `playwright-praman` package. This content is intended for contributors and plugin developers.
:::
The bridge is the communication layer between Praman's Node.js test code and the SAP UI5 framework running in the browser. Understanding its internals is essential for debugging and extending Praman.
## Bridge Injection via page.evaluate()
Praman injects a bridge namespace (`window.__praman_bridge`) into the browser context using Playwright's `page.evaluate()`. The bridge provides:
- UI5 module references (RecordReplay, Element, Log)
- Version detection (`sap.ui.version`)
- Object map for non-control storage (UUID-keyed)
- `getById()` with 3-tier API resolution (Decision D19)
- Helper functions: `isPrimitive`, `saveObject`, `getObject`, `deleteObject`
### Injection Modes
Praman supports two injection strategies:
**Lazy injection** (default): Triggered on first UI5 operation via `ensureBridgeInjected()`.
```typescript
// Every proxy and handler method calls this before browser operations
await ensureBridgeInjected(page);
```
**Eager injection**: Registered via `addInitScript()` before any page loads. Useful for auth flows where the bridge must be ready before the first navigation.
```typescript
// Inject before any navigation (recommended for auth flows)
await injectBridgeEager(page);
await page.goto('https://sap-system.example.com/app');
// Bridge is already available when UI5 loads
```
### Injection Lifecycle
1. Wait for UI5 framework availability (`sap.ui.require` exists)
2. Execute bridge injection script (creates `window.__praman_bridge`)
3. Wait for bridge readiness (`window.__praman_ready === true`)
Injection tracking uses `WeakSet` to avoid memory leaks. After page navigation invalidates the bridge, call `resetPageInjection(page)` so the next operation re-injects.
```typescript
// After navigation invalidates the bridge
resetPageInjection(page);
// Next ensureBridgeInjected() call will re-inject
```
## Serialization Constraints
**This is the single most critical concept for anyone working on bridge code.**
`page.evaluate()` serializes ONLY the function body via `fn.toString()`. This has profound implications:
- Module-level functions are NOT included in the serialized output.
- Imports and closures are NOT available inside `page.evaluate()`.
- TypeScript types (imports, `as` casts, type aliases) are fine -- they are erased at compile time.
### The Inner Function Rule
ALL helper functions MUST be declared as inner function declarations inside the evaluated function:
```typescript
// CORRECT: helper is an inner function
await page.evaluate(() => {
function getControlById(id: string) {
// This function IS available in the browser context
return sap.ui.getCore().byId(id);
}
return getControlById('myButton');
});
// WRONG: helper is a module-level function
function getControlById(id: string) {
return sap.ui.getCore().byId(id);
}
await page.evaluate(() => {
// ReferenceError: getControlById is not defined
return getControlById('myButton');
});
```
### Why Unit Tests Give False Positives
Unit tests run in Node.js where module-level functions ARE accessible via closure. This means a unit test will pass even when the code would fail in the browser:
```typescript
// This module-level function is accessible in Node.js tests...
function helperFn() {
return 'works';
}
// ...so this test PASSES in Vitest but the code FAILS in the browser
it('should work', () => {
const fn = () => helperFn();
expect(fn()).toBe('works'); // PASSES in Node.js
});
// In the browser, page.evaluate(() => helperFn()) throws ReferenceError
```
If you see `sonarjs/no-identical-functions` warnings about duplicate inner functions, suppress them with an ESLint disable comment. The duplication is intentional and required.
### String-Based Scripts
For complex browser operations, Praman uses string-based scripts instead of arrow functions. This avoids serialization issues entirely:
```typescript
// Bridge injection uses string evaluation
const script = createBridgeInjectionScript(); // returns a string
await page.evaluate(script);
// Find-control and execute-method also use strings
const findScript = createFindControlScript(selector);
const result = await page.evaluate(findScript);
```
## 3-Tier API Resolution (D19)
The bridge provides a centralized `getById()` function that resolves UI5 control IDs using three fallback tiers:
```
Tier 1: Element.getElementById() (UI5 1.108+)
Tier 2: ElementRegistry.get() (UI5 1.84+)
Tier 3: sap.ui.getCore().byId() (All versions)
```
This is registered once during injection. All browser scripts call `bridge.getById()` instead of duplicating the lookup logic (which was a problem in dhikraft v2.5.0 where 6 files had independent copies).
## Frame Navigation (WorkZone Dual-Frame)
SAP BTP WorkZone (formerly Fiori Launchpad as a Service) renders applications inside nested iframes. Praman handles this transparently:
```
+----------------------------------+
| Shell Frame (WorkZone chrome) |
| +----------------------------+ |
| | App Frame (Fiori app) | |
| | UI5 controls live here | |
| +----------------------------+ |
+----------------------------------+
```
The bridge must be injected into the **app frame**, not the shell frame. Praman's fixture layer automatically detects the WorkZone dual-frame layout and targets the correct frame for bridge injection and control operations.
After frame navigation (e.g., navigating between Fiori Launchpad tiles), the bridge state in the previous frame is invalidated. The injection tracking resets and re-injects on the next operation.
## Object Map Lifecycle
Non-control UI5 objects (Models, BindingContexts, Routers, etc.) cannot be serialized across the `page.evaluate()` boundary. Praman stores them in a browser-side Map keyed by UUID:
```typescript
// Browser side: save an object reference
const uuid = bridge.saveObject(myModel, 'model');
// Returns: 'a1b2c3d4-...'
// Node side: reference the object by UUID in subsequent calls
await page.evaluate((uuid) => bridge.getObject(uuid).getProperty('/Name'), uuid);
```
### TTL-Based Cleanup (D20)
To prevent memory leaks in long test runs, objects are stored with timestamps:
```javascript
bridge.objectMap.set(uuid, {
value: obj,
type: type || 'unknown',
storedAt: Date.now(),
});
```
A cleanup function evicts entries older than the configured TTL. This pairs with the Node-side `UI5ObjectCache` (TTL + LRU eviction) to keep both sides in sync.
## Stability Checks
Before performing control operations, Praman waits for UI5 to reach a stable (idle) state:
```typescript
// waitForUI5Stable checks these conditions:
// 1. No pending HTTP requests
// 2. No pending timeouts
// 3. No pending UI5 rendering updates
// 4. RecordReplay is available and idle
await page.waitForFunction(waitForUI5StableScript, { timeout });
```
The `skipStabilityWait` option (D23) can be configured globally or per-selector for pages with third-party overlays (like WalkMe) that interfere with stability detection.
## Browser Scripts
The `src/bridge/browser-scripts/` directory contains the scripts executed in the browser context:
| Script | Purpose |
| -------------------------- | ---------------------------------------------------------------- |
| `inject-ui5.ts` | Core bridge initialization, module references, version detection |
| `find-control.ts` | UI5 control discovery by selector |
| `execute-method.ts` | Remote method invocation on UI5 controls |
| `get-selector.ts` | Reverse lookup: DOM element to UI5 selector |
| `get-version.ts` | UI5 version detection |
| `inspect-control.ts` | Control metadata introspection for AI |
| `object-map.ts` | UUID-keyed non-control object storage with TTL |
| `find-control-matchers.ts` | Property matcher generation for selectors |
## Interaction Strategies
Praman provides three strategies for interacting with UI5 controls:
| Strategy | Approach | Best For |
| ------------------- | ---------------------------------------- | -------------------------------------------------- |
| `UI5NativeStrategy` | `fire*` events -> `fireTap` -> DOM click | Standard UI5 controls, most reliable |
| `DomFirstStrategy` | DOM click + auto-detect input type | Controls with custom renderers |
| `Opa5Strategy` | `RecordReplay.interactWithControl` | Complex controls needing SAP's own interaction API |
The strategy factory selects the appropriate strategy based on configuration and control type.
---
## Architecture Overview
Praman v1.0 ships as a single npm package (`playwright-praman`) with sub-path exports, organized into a strict 5-layer architecture where lower layers never import from higher layers.
## 5-Layer Architecture
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Test Author / AI Agent │
│ import { test, expect } from 'playwright-praman' │
│ import { procurement } from 'playwright-praman/intents' │
└──────────────────────────────────┬──────────────────────────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ Layer 5 — Public API (6 sub-path exports) │
│ │
│ playwright-praman /ai /intents /vocabulary /fe /reporters │
└──────────────────────────────────┬──────────────────────────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ Layer 4 — Fixtures + Modules │
│ │
│ Fixtures core · auth · nav · stability · ai · fe · intent · flp │
│ Merged via mergeTests() into a single test object │
│ Modules table · dialog · date · navigation · odata · workzone │
│ Auth cloud-saml · office365 · onprem · api-key · cert · tenant │
└──────────────────────────────────┬──────────────────────────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ Layer 3 — Control Proxy │
│ │
│ control-proxy.ts · discovery.ts · ui5-object.ts · cache.ts │
│ 3-tier discovery: object cache → stable ID → type + property search │
└──────────────────────────────────┬──────────────────────────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ Layer 2 — Bridge │
│ │
│ injection.ts (lazy bridge bootstrap into browser context) │
│ browser-scripts/ inject-ui5 · find-control · execute-method · ... │
│ Interaction strategies: UI5Native · DomFirst · OPA5 │
└──────────────────────────────────┬──────────────────────────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ Layer 1 — Core Infrastructure │
│ │
│ config (Zod) · errors (13 subclasses) · logging (pino) │
│ telemetry (OTel) · types (199 interfaces) · utils · compat │
└──────────────────────────────────┬──────────────────────────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────────┐
│ @playwright/test (external dependency) │
│ page · browser · context · expect │
└─────────────────────────────────────────────────────────────────────────┘
```
## Layer Dependency Rules
The architecture enforces a strict **downward-only dependency rule**:
- Layer 1 (Core) has zero internal dependencies beyond Node.js builtins.
- Layer 2 (Bridge) imports only from Layer 1.
- Layer 3 (Proxy) imports from Layers 1 and 2.
- Layer 4 (Fixtures + Modules) imports from Layers 1, 2, and 3.
- Layer 5 (Public API) re-exports from Layers 1–4; no higher layer imports from it.
This is enforced at compile time via TypeScript path aliases (`#core/*`, `#bridge/*`, `#proxy/*`, `#fixtures/*`) and ESLint import rules.
:::info Internal path aliases
The aliases below are internal to Praman's source code and are not available to package consumers. They enforce layer boundaries during development.
:::
```typescript
// Allowed: Layer 3 importing from Layer 1
// Allowed: Layer 3 importing from Layer 2
// FORBIDDEN: Layer 1 importing from Layer 3 — compile error
// import { ControlProxy } from '#proxy/control-proxy.js';
```
## Sub-Path Exports
Praman ships 6 sub-path exports, each targeting a specific concern:
| Sub-Path Export | Purpose | Key Modules |
| ------------------------------ | --------------------------- | -------------------------------------------------- |
| `playwright-praman` | Core fixtures, expect, test | Fixtures, selectors, matchers |
| `playwright-praman/ai` | AI agent integration | CapabilityRegistry, RecipeRegistry, AgenticHandler |
| `playwright-praman/intents` | SAP domain intent APIs | Core wrappers, 5 domain namespaces |
| `playwright-praman/vocabulary` | Business term resolution | VocabularyService, fuzzy matcher |
| `playwright-praman/fe` | Fiori Elements helpers | ListReport, ObjectPage, FE tables |
| `playwright-praman/reporters` | Custom Playwright reporters | ComplianceReporter, ODataTraceReporter |
Each export provides both ESM and CJS outputs:
```json
{
"./ai": {
"types": {
"import": "./dist/ai/index.d.ts",
"require": "./dist/ai/index.d.cts"
},
"import": "./dist/ai/index.js",
"require": "./dist/ai/index.cjs"
}
}
```
All exports are validated by `@arethetypeswrong/cli` (attw) in CI.
## Design Decisions Summary (D1–D29)
The architecture is guided by 29 documented design decisions:
| # | Decision | Category |
| --- | ------------------------------------------------------------------------ | ------------- |
| D1 | Single package with sub-path exports (not monorepo) | Packaging |
| D2 | Internal fixture composition via `extend()` chain | Fixtures |
| D3 | ~~Version-negotiated bridge adapters~~ (removed — superseded by D4) | Bridge |
| D4 | Hybrid typed proxy: 199 typed interfaces + dynamic fallback | Proxy |
| D5 | 4-layer observability: Reporter + pino + OTel + AI telemetry | Observability |
| D6 | Zod validation at external boundaries only | Validation |
| D7 | Zod-validated `praman.config.ts` with env overrides | Config |
| D8 | Unified `PramanError` hierarchy with 13 subclasses | Errors |
| D9 | AI Mode A (SKILL.md) + Mode C (agentic fixture) | AI |
| D10 | Vitest unit tests + Playwright integration tests | Testing |
| D11 | No plugin API in v1.0 | Extension |
| D12 | Auto-generated SKILL.md + Docusaurus + TypeDoc | Docs |
| D13 | Apache 2.0 license | Legal |
| D14 | Playwright `>=1.57.0 <2.0.0` with CI matrix | Compat |
| D15 | Security: dep scanning, secret redaction, SBOM, provenance | Security |
| D16 | Single unified proxy (no double-proxy) | Proxy |
| D17 | Bidirectional proxy conversion (`UI5Object` <-> `UI5ControlProxy`) | Proxy |
| D18 | Integrated discovery factory (removed dead code) | Discovery |
| D19 | Centralized `__praman_getById()` (3-tier API resolution) | Bridge |
| D20 | Browser objectMap with TTL + cleanup | Memory |
| D21 | Shared interaction logic across strategies | Interactions |
| D22 | Auto-generated method signatures (199 interfaces, 4,092 methods) | Types |
| D23 | `skipStabilityWait` as global config + per-selector override | Stability |
| D24 | `exec()` with `new Function()` + security documentation | Security |
| D25 | Visibility preference default (prefer visible controls) | Discovery |
| D26 | UI5Object AI introspection as first-class capability | AI |
| D27 | Module size <= 300 LOC guideline (with documented exceptions) | Quality |
| D28 | Auth via Playwright project dependencies (not globalSetup) | Auth |
| D29 | Enhanced error model + AI response envelope | AI/Errors |
## Module Decomposition Rationale
The predecessor plugin (v2.5.0) suffered from monolithic files:
| File | LOC | Problem |
| ---------------------- | ----- | ---------------------------------------- |
| `ui5-handler.ts` | 2,318 | God object handling all UI5 operations |
| `plugin-fixtures.ts` | 2,263 | All 32 fixtures in one file |
| `ui5-control-proxy.ts` | 1,829 | Double-proxy with redundant interception |
Praman v1.0 decomposes these into focused modules following SRP (Single Responsibility Principle):
**Fixtures** are split by domain:
- `core-fixtures.ts` — UI5 control operations
- `auth-fixtures.ts` — Authentication strategies
- `nav-fixtures.ts` — Navigation and routing
- `stability-fixtures.ts` — Wait conditions and stability checks
- `ai-fixtures.ts` — AI agent fixtures
- `fe-fixtures.ts` — Fiori Elements fixtures
- `intent-fixtures.ts` — Intent API fixtures
- `shell-footer-fixtures.ts`, `flp-locks-fixtures.ts`, `flp-settings-fixtures.ts` — FLP-specific fixtures
**Modules** handle domain operations independently of fixtures:
- `table.ts` + `table-operations.ts` + `table-filter-sort.ts` — Grid and SmartTable
- `dialog.ts` — Dialog detection and interaction
- `date.ts` — DatePicker and range fields
- `navigation.ts` — UI5 routing and hash navigation
- `odata.ts` + `odata-http.ts` — OData model access and CRUD
- `workzone.ts` — BTP WorkZone helpers
**UI5Handler** was decomposed from a 2,318 LOC god object into:
- `ui5-handler.ts` (588 LOC) — 16 core methods
- `shell-handler.ts` (102 LOC) — FLP shell operations
- `footer-handler.ts` (119 LOC) — Footer bar operations
- `navigation.ts` (201 LOC) — Route and hash navigation
**Control proxy** was simplified from a double-proxy pattern (1,829 LOC) into:
- `control-proxy.ts` (653 LOC) — Unified single proxy handler
- `ui5-object.ts` (383 LOC) — Non-control object proxy
- `discovery.ts` (111 LOC) — 3-tier control discovery
- `cache.ts` (104 LOC) — Proxy cache with RegExp keys
Every module targets <= 300 LOC. Exceptions are documented with justification comments.
## Codebase Metrics
| Metric | Value |
| ------------------------ | --------------------------- |
| Source files | ~150 |
| Source LOC | ~29,000 |
| Public functions | 208 |
| Test files | 109 |
| Unit tests | ~2,000 passing |
| Typed control interfaces | 199 (4,092 methods) |
| Error subclasses | 13 |
| Coverage | >98% (per-file enforcement) |
| Lint errors | 0 |
| Type errors | 0 |
| ESM + CJS dual build | Validated by attw |
---
## AI Integration
Praman is an AI-first test automation plugin. The `playwright-praman/ai` sub-path provides structured interfaces for LLM-driven test generation, page analysis, and autonomous agent workflows.
## AiResponse Envelope
All AI operations return an `AiResponse` discriminated union, providing type-safe handling without casting:
```typescript
type AiResponse =
| { status: 'success'; data: T; metadata: AiResponseMetadata }
| { status: 'error'; data: undefined; error: AiResponseError; metadata: AiResponseMetadata }
| { status: 'partial'; data: Partial; error?: AiResponseError; metadata: AiResponseMetadata };
```
Narrow on `status` to get type-safe access:
```typescript
function handle(response: AiResponse): T | undefined {
if (response.status === 'success') {
return response.data; // TypeScript knows this is T
}
if (response.status === 'error') {
console.error(response.error.code, response.error.message);
if (response.metadata.retryable) {
// Agent can retry this operation
}
}
return undefined;
}
```
### AiResponseMetadata
Every response carries metadata for performance tracking and self-healing:
```typescript
interface AiResponseMetadata {
readonly duration: number; // Elapsed time in ms
readonly retryable: boolean; // Can the caller retry?
readonly suggestions: string[]; // Recovery hints for agents/testers
readonly model?: string; // Model identifier (e.g., 'gpt-4o')
readonly tokens?: number; // Total tokens consumed
}
```
## Provider Abstraction
Praman supports three LLM providers through a unified `LlmService` interface:
| Provider | SDK | Config Key |
| ---------------- | ------------------- | -------------------------- |
| Azure OpenAI | `openai` | `provider: 'azure-openai'` |
| OpenAI | `openai` | `provider: 'openai'` |
| Anthropic Claude | `@anthropic-ai/sdk` | `provider: 'anthropic'` |
Provider SDKs are loaded via dynamic `import()` and remain optional dependencies. If the SDK is not installed, a clear `AIError` with installation instructions is thrown.
### Configuration
```typescript
// praman.config.ts
export default {
ai: {
provider: 'azure-openai',
model: 'gpt-4o',
endpoint: 'https://my-resource.openai.azure.com/',
apiKey: process.env.AZURE_OPENAI_API_KEY,
temperature: 0.2,
maxTokens: 4096,
apiVersion: '2024-02-15-preview',
deployment: 'my-gpt4o-deployment',
},
};
```
### Creating an LLM Service
```typescript
const llm = createLlmService(config);
const response = await llm.complete([
{ role: 'system', content: 'You are a test generator...' },
{ role: 'user', content: 'Generate a test for the login page' },
]);
```
## AgenticCheckpoint
For long-running multi-step AI workflows, Praman provides checkpoint serialization. The `AgenticCheckpoint` allows AI agents to resume from the last successful step on failure:
```typescript
const checkpoint: AgenticCheckpoint = {
sessionId: 'sess-001',
currentStep: 2,
completedSteps: ['discover', 'plan'],
remainingSteps: ['generate', 'validate'],
state: { pageUrl: 'https://my.app/launchpad' },
timestamp: new Date().toISOString(),
};
```
The `AgenticHandler` serializes progress automatically:
```typescript
const handler = new AgenticHandler(llmService, capabilityRegistry, recipeRegistry, page);
// generateTest returns AiResponse with checkpoint data
const result = await handler.generateTest('Test the purchase order creation flow');
```
## Page Context and toAIContext()
### buildPageContext()
Discovers all UI5 controls on the page and builds a structured context for LLM prompts:
```typescript
const context = await buildPageContext(page, config);
if (context.status === 'success') {
const { controls, formFields, buttons, tables } = context.data;
// Feed to LLM as structured context
}
```
The returned `PageContext` partitions controls by semantic category:
```typescript
interface PageContext {
readonly url: string;
readonly ui5Version?: string;
readonly controls: DiscoveredControl[]; // All controls
readonly formFields: DiscoveredControl[]; // Inputs, selects, etc.
readonly buttons: DiscoveredControl[]; // Buttons, clickable triggers
readonly tables: DiscoveredControl[]; // Data tables and lists
readonly navigationElements: DiscoveredControl[];
readonly timestamp: string;
}
```
### toAIContext() on Errors
Every `PramanError` provides `toAIContext()` for machine-readable error introspection:
```typescript
try {
await ui5.click({ id: 'submitBtn' });
} catch (error) {
if (error instanceof PramanError) {
const context = error.toAIContext();
// {
// code: 'ERR_CONTROL_NOT_FOUND',
// message: 'Control not found: submitBtn',
// attempted: 'Find control with selector: {"id":"submitBtn"}',
// retryable: true,
// suggestions: ['Verify the control ID exists in the UI5 view', ...]
// }
}
}
```
## Token-Aware Metadata
All AI responses include token usage information when the provider reports it:
```typescript
const result = await handler.generateTest('Create a PO test');
if (result.status === 'success') {
console.log(result.data.metadata.tokens);
// { input: 1200, output: 340 }
console.log(result.data.metadata.model);
// 'gpt-4o'
console.log(result.data.metadata.duration);
// 1850 (ms)
}
```
This enables cost tracking and prompt optimization in CI pipelines.
## Capability Registry
The `CapabilityRegistry` exposes Praman's API surface to AI agents for test generation:
```typescript
const registry = new CapabilityRegistry();
// Query by category (CapabilityCategory enum value)
const tableCaps = registry.byCategory('table');
// Get all capabilities for AI prompt context
const aiContext = registry.forAI();
// Provider-specific formatting
const claudeFormat = registry.forProvider('claude'); // XML-structured
const openaiFormat = registry.forProvider('openai'); // JSON registry snapshot
```
Each capability entry includes:
```typescript
interface CapabilityEntry {
readonly id: string; // 'UI5-UI5-001' (UI5-PREFIX-NNN format)
readonly qualifiedName: string; // 'ui5.clickButton'
readonly name: string; // 'clickButton'
readonly description: string; // 'Clicks a UI5 button by selector'
readonly category: CapabilityCategory; // 'ui5' (enum value)
readonly priority: 'fixture' | 'namespace' | 'implementation'; // REQUIRED
readonly usageExample: string; // "await ui5.click({ id: 'submitBtn' })"
readonly registryVersion: 1; // Literal 1 for schema versioning
readonly intent?: string; // Optional intent tag
readonly sapModule?: string; // Optional SAP module tag
readonly controlTypes?: string[]; // UI5 control types
readonly async?: boolean; // Whether this is async
}
```
Capabilities are auto-generated from TSDoc annotations via `npm run generate:capabilities`. The `registryVersion` literal enables AI agents to detect stale cached data.
## Recipe Registry
The `RecipeRegistry` provides curated test patterns that AI agents can emit verbatim or adapt:
```typescript
const registry = new RecipeRegistry();
// Filter by domain and priority
const authRecipes = registry.select({ domain: 'auth', priority: 'essential' });
// Get the top N essential recipes for prompt context
const topFive = registry.getTopRecipes(5);
// Get all recipes formatted for AI consumption
const aiRecipes = registry.forAI();
```
Each recipe entry:
```typescript
interface RecipeEntry {
readonly id: string; // 'recipe-ui5-button-click' (recipe-kebab-case)
readonly name: string; // 'Button Click'
readonly description: string; // What this recipe demonstrates
readonly domain: string; // 'ui5' (domain grouping)
readonly priority: 'essential' | 'recommended' | 'optional' | 'advanced' | 'deprecated';
readonly capabilities: string[]; // ['UI5-UI5-003'] (capability IDs used)
readonly pattern: string; // Ready-to-use TypeScript code pattern
}
```
## AI Surface Modes
Praman supports two AI modes (Decision D9):
**Mode A -- SKILL.md for code-gen agents**: A strengthened skill file with typed API surface, capability/recipe registries, and usage examples. Agents like GitHub Copilot, Claude Code, and Cursor consume this to generate test code.
**Mode C -- Agentic fixture for in-test agents**: The `AgenticHandler` fixture with Zod-validated LLM output for autonomous test generation during test execution. The handler builds page context, generates test code, and validates the output against schemas.
There is no MCP server -- Praman remains a Playwright plugin and does not duplicate browser management.
---
## Agent & IDE Setup
`npx playwright-praman init` scaffolds your project and, based on the IDEs it detects,
automatically installs AI agent definitions, seed files, and IDE configuration.
## Prerequisites
| Requirement | Version |
| --------------------- | ---------------------------------------------------------------- |
| Node.js | `>=20` |
| `@playwright/test` | `>=1.57.0 <2.0.0` (peer dependency) |
| SAP UI5 / Fiori app | Any cloud or on-premise instance |
| Environment variables | `SAP_CLOUD_BASE_URL`, `SAP_CLOUD_USERNAME`, `SAP_CLOUD_PASSWORD` |
Install the package and browser binaries:
```bash
npm install --save-dev playwright-praman @playwright/test
npx playwright install
```
Praman does **not** auto-install Playwright browsers. Run `npx playwright install` once after installation.
## What `init` Installs
Run in your project root:
```bash
npx playwright init-agents --loop=vscode
npx playwright-praman init
```
### Base scaffold (always)
| Path | Description |
| ---------------------- | ------------------------ |
| `playwright.config.ts` | Playwright configuration |
| `praman.config.ts` | Praman configuration |
| `tsconfig.json` | TypeScript configuration |
| `tests/` | Test directory |
| `tests/e2e/` | E2E test directory |
| `.auth/` | Auth state storage |
### Claude Code (detected via `CLAUDE.md` or `.claude/`)
| Path | Description |
| ---------------------------------------- | ----------------------------------- |
| `.claude/agents/praman-sap-planner.md` | SAP UI5 test planner agent |
| `.claude/agents/praman-sap-generator.md` | Praman-compliant test generator |
| `.claude/agents/praman-sap-healer.md` | Failing test fixer |
| `.claude/prompts/praman-sap-plan.md` | Plan slash command |
| `.claude/prompts/praman-sap-generate.md` | Generate slash command |
| `.claude/prompts/praman-sap-heal.md` | Heal slash command |
| `.claude/prompts/praman-sap-coverage.md` | Full coverage pipeline |
| `tests/seeds/sap-seed.spec.ts` | Authenticated seed for AI discovery |
After `init`, append the Praman section to your `CLAUDE.md`:
```bash
cat node_modules/playwright-praman/docs/user-integration/claude-md-appendable.md >> CLAUDE.md
```
### VS Code (detected via `.vscode/` directory or `TERM_PROGRAM=vscode`)
| Path | Description |
| ------------------------------ | ------------------------------------- |
| `.vscode/settings.json` | Playwright + TypeScript settings |
| `.vscode/extensions.json` | Recommended extensions |
| `.vscode/praman.code-snippets` | `praman-test`, `praman-step` snippets |
### Cursor (detected via `.cursor/` or `.cursorrc`)
| Path | Description |
| -------------------------- | -------------------------- |
| `.cursor/rules/praman.mdc` | Praman rules for Cursor AI |
Append the full rules to your Cursor config:
```bash
cat node_modules/playwright-praman/docs/user-integration/cursor-rules-appendable.mdc >> .cursorrules
```
### Jules (detected via `.jules/`)
| Path | Description |
| ------------------------ | ----------------------------------- |
| `.jules/praman-setup.md` | Praman setup instructions for Jules |
### GitHub Copilot (detected via `.github/copilot-instructions.md` or `.github/agents/`)
| Path | Description |
| ---------------------------------------------- | ---------------------------------- |
| `.github/agents/praman-sap-planner.agent.md` | SAP UI5 test planner Copilot agent |
| `.github/agents/praman-sap-generator.agent.md` | Praman-compliant test generator |
| `.github/agents/praman-sap-healer.agent.md` | Failing test fixer |
After `init`, append the Praman section to your `.github/copilot-instructions.md`:
```bash
cat node_modules/playwright-praman/docs/user-integration/copilot-instructions-appendable.md >> .github/copilot-instructions.md
```
## Re-running on an Existing Project
`init` can be run in an existing project. When the project directory already exists,
`init` skips the base scaffold and only installs IDE files that are missing:
```bash
# Existing project — only installs missing agent/IDE files
npx playwright-praman init
# Overwrite all files (including existing agents and configs)
npx playwright-praman init --force
```
## IDE Detection Logic
`init` detects IDEs in this order. Multiple IDEs can be active simultaneously:
| IDE | Detection Method |
| ------------------ | ----------------------------------------------------------------------- |
| **VS Code** | `.vscode/` directory exists, OR `TERM_PROGRAM=vscode` env var |
| **Claude Code** | `CLAUDE.md` file exists in project root |
| **Cursor** | `.cursor/` directory or `.cursorrc` file exists |
| **Jules** | `.jules/` directory exists |
| **OpenCode** | `.opencode/` directory exists |
| **GitHub Copilot** | `.github/copilot-instructions.md` or `.github/agents/` directory exists |
## Manual Installation
If `init` was run before IDE tools were set up, install files manually:
### Claude Code
```bash
# Install agents
mkdir -p .claude/agents .claude/prompts tests/seeds
cp node_modules/playwright-praman/agents/claude/praman-sap-planner.md .claude/agents/
cp node_modules/playwright-praman/agents/claude/praman-sap-generator.md .claude/agents/
cp node_modules/playwright-praman/agents/claude/praman-sap-healer.md .claude/agents/
cp node_modules/playwright-praman/agents/claude/prompts/praman-sap-plan.md .claude/prompts/
cp node_modules/playwright-praman/agents/claude/prompts/praman-sap-generate.md .claude/prompts/
cp node_modules/playwright-praman/agents/claude/prompts/praman-sap-heal.md .claude/prompts/
cp node_modules/playwright-praman/agents/claude/prompts/praman-sap-coverage.md .claude/prompts/
cp node_modules/playwright-praman/seeds/sap-seed.spec.ts tests/seeds/
# Append to CLAUDE.md
cat node_modules/playwright-praman/docs/user-integration/claude-md-appendable.md >> CLAUDE.md
```
### VS Code
```bash
mkdir -p .vscode
# Install recommended settings and extensions
# (or run: npx playwright-praman init --force)
```
### Cursor
```bash
mkdir -p .cursor/rules
cp node_modules/playwright-praman/docs/user-integration/cursor-rules-appendable.mdc .cursor/rules/praman.mdc
cat node_modules/playwright-praman/docs/user-integration/cursor-rules-appendable.mdc >> .cursorrules
```
### GitHub Copilot
```bash
mkdir -p .github/agents
cp node_modules/playwright-praman/agents/copilot/praman-sap-planner.agent.md .github/agents/
cp node_modules/playwright-praman/agents/copilot/praman-sap-generator.agent.md .github/agents/
cp node_modules/playwright-praman/agents/copilot/praman-sap-healer.agent.md .github/agents/
# Append to copilot-instructions.md
cat node_modules/playwright-praman/docs/user-integration/copilot-instructions-appendable.md >> .github/copilot-instructions.md
```
## Skill Files
The skill files (SKILL.md and supporting references) are **not copied** into your project.
All Praman agents read them directly from the installed package:
```text
node_modules/playwright-praman/skills/playwright-praman-sap-testing/SKILL.md
node_modules/playwright-praman/skills/playwright-praman-sap-testing/ai-quick-reference.md
```
This ensures agents always use the current installed version without any stale copies.
## The Seed File
`tests/seeds/sap-seed.spec.ts` is the entry point for AI agent SAP discovery.
It uses **raw Playwright only** (no Praman fixtures) to authenticate against a live SAP system
and keep the browser open for the MCP server.
Required environment variables:
```bash
SAP_CLOUD_BASE_URL=https://your-system.s4hana.cloud.sap/
SAP_CLOUD_USERNAME=your-user
SAP_CLOUD_PASSWORD=your-password
```
Run the seed to open an authenticated browser session:
```bash
npx playwright test tests/seeds/sap-seed.spec.ts --project=agent-seed-test
```
The seed waits up to 20 minutes, polls for UI5 readiness, then keeps the browser open
via `pauseAtEnd: true` for MCP-connected agents to use.
## Available Agents (Claude Code)
After setup, `.claude/agents/` contains 3 Praman SAP agents:
| Agent | Slash Command | Purpose |
| ---------------------- | ---------------------- | --------------------------------------------------- |
| `praman-sap-planner` | `/praman-sap-plan` | Explore live SAP app, produce test plan + seed spec |
| `praman-sap-generator` | `/praman-sap-generate` | Generate 100% Praman-compliant tests from plan |
| `praman-sap-healer` | `/praman-sap-heal` | Fix failing tests, enforce compliance |
Use the coverage prompt to run the full pipeline:
```text
/praman-sap-coverage
"Run full test coverage for the Purchase Order Fiori app"
```
## LLM-Friendly Documentation (llms.txt)
Praman publishes its documentation in the [llmstxt.org](https://llmstxt.org) standard. AI agents can fetch these files directly for context:
| URL | Content | Use Case |
| ------------------------ | ---------------------------------------- | -------------------------------- |
| `/llms.txt` | Link index with descriptions | Discovery — find the right doc |
| `/llms-full.txt` | All 63 docs in one file | Full context for general agents |
| `/llms-quickstart.txt` | Setup, fixtures, selectors, matchers | Onboarding and first test |
| `/llms-sap-testing.txt` | Auth, FLP, OData, FE, cookbook, examples | SAP test planning and generation |
| `/llms-migration.txt` | Playwright, wdi5, Tosca migration | Migration assistants |
| `/llms-architecture.txt` | Architecture, bridge, proxy, ADRs | Architecture decisions |
### Pointing agents to llms.txt
Add to your agent instructions (CLAUDE.md, `.cursorrules`, copilot-instructions.md):
```markdown
## Praman Documentation
For Praman API and usage, fetch the appropriate llms.txt file:
- General: https://praman.dev/llms-full.txt
- SAP testing: https://praman.dev/llms-sap-testing.txt
- Quick start: https://praman.dev/llms-quickstart.txt
```
Agents with web access (Claude Code `WebFetch`, Copilot `@fetch`) can retrieve these at runtime. Agents without web access can use the locally built files from `docs/build/`.
## IDE Configuration
The sections below cover manual IDE configuration. If `init` already scaffolded your
IDE files, use these as a reference for customization.
### VS Code Extensions
| Extension | ID | Purpose |
| ------------------------ | -------------------------- | ---------------------------------------------- |
| ESLint | `dbaeumer.vscode-eslint` | Lint TypeScript with Praman's 11-plugin config |
| Playwright Test | `ms-playwright.playwright` | Run/debug tests, view traces |
| TypeScript | Built-in | TypeScript language support |
| Vitest | `vitest.explorer` | Run unit tests from sidebar |
| Pretty TypeScript Errors | `yoavbls.pretty-ts-errors` | Readable type error messages |
| Error Lens | `usernamehw.errorlens` | Inline error/warning display |
| GitLens | `eamodio.gitlens` | Git blame, history, compare |
Install via CLI:
```bash
code --install-extension dbaeumer.vscode-eslint
code --install-extension ms-playwright.playwright
code --install-extension vitest.explorer
code --install-extension yoavbls.pretty-ts-errors
code --install-extension usernamehw.errorlens
```
### VS Code Settings
Create or update `.vscode/settings.json` in your test project:
```json
{
"typescript.tsdk": "node_modules/typescript/lib",
"editor.defaultFormatter": "dbaeumer.vscode-eslint",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.fixAll.eslint": "explicit"
},
"eslint.validate": ["typescript"],
"testing.defaultGutterClickAction": "debug",
"playwright.reuseBrowser": true,
"playwright.showTrace": true
}
```
### Debugging Praman Tests
Create `.vscode/launch.json`:
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Debug Current Test File",
"type": "node",
"request": "launch",
"program": "${workspaceFolder}/node_modules/.bin/playwright",
"args": ["test", "${relativeFile}", "--headed", "--workers=1"],
"console": "integratedTerminal",
"env": {
"PWDEBUG": "1",
"SAP_BASE_URL": "https://your-sap-system.example.com",
"SAP_USERNAME": "your-user",
"SAP_PASSWORD": "your-password"
}
},
{
"name": "Debug Specific Test",
"type": "node",
"request": "launch",
"program": "${workspaceFolder}/node_modules/.bin/playwright",
"args": ["test", "${relativeFile}", "--headed", "--workers=1", "-g", "${selectedText}"],
"console": "integratedTerminal",
"env": {
"PWDEBUG": "1"
}
},
{
"name": "Debug Unit Tests (Vitest)",
"type": "node",
"request": "launch",
"program": "${workspaceFolder}/node_modules/.bin/vitest",
"args": ["run", "${relativeFile}"],
"console": "integratedTerminal"
}
]
}
```
### Playwright Inspector
Set `PWDEBUG=1` to launch the Playwright Inspector:
```bash
PWDEBUG=1 npx playwright test tests/my-test.spec.ts --headed --workers=1
```
The Inspector lets you:
- Step through test actions one at a time
- Inspect the UI5 control tree in the browser
- View selectors and their matches
- Record new actions
### Playwright Trace Viewer
After a test run, open the trace:
```bash
npx playwright show-trace test-results/my-test/trace.zip
```
The Trace Viewer shows:
- Action timeline with screenshots
- Network requests (OData calls)
- Console logs
- DOM snapshots at each step
### Environment Variables
Create a `.env` file in your project root (add to `.gitignore`):
```bash
# .env
SAP_BASE_URL=https://your-sap-system.example.com
SAP_USERNAME=test-user
SAP_PASSWORD=test-password
SAP_CLIENT=100
SAP_LANGUAGE=EN
```
Load `.env` in your Playwright config:
```typescript
// playwright.config.ts
config(); // Load .env file
export default defineConfig({
use: {
baseURL: process.env.SAP_BASE_URL,
},
});
```
### Code Snippets
Add Praman-specific code snippets to `.vscode/praman.code-snippets`:
```json
{
"Praman Test": {
"prefix": "ptest",
"scope": "typescript",
"body": [
"import { test, expect } from 'playwright-praman';",
"",
"test.describe('$1', () => {",
" test('$2', async ({ ui5, ui5Navigation, ui5Matchers }) => {",
" await test.step('$3', async () => {",
" $0",
" });",
" });",
"});"
],
"description": "Praman test with fixtures"
},
"Praman Step": {
"prefix": "pstep",
"scope": "typescript",
"body": ["await test.step('$1', async () => {", " $0", "});"],
"description": "Praman test step"
},
"UI5 Control Selector": {
"prefix": "pcontrol",
"scope": "typescript",
"body": [
"const ${1:control} = await ui5.control({",
" controlType: '${2:sap.m.Button}',",
" ${3:properties: { text: '$4' \\}},",
"});"
],
"description": "UI5 control selector"
},
"UI5 Click": {
"prefix": "pclick",
"scope": "typescript",
"body": [
"await ui5.click({",
" controlType: '${1:sap.m.Button}',",
" properties: { text: '$2' },",
"});"
],
"description": "UI5 click action"
},
"UI5 Fill": {
"prefix": "pfill",
"scope": "typescript",
"body": [
"await ui5.fill(",
" { controlType: '${1:sap.m.Input}', id: /${2:fieldId}/ },",
" '${3:value}'",
");"
],
"description": "UI5 fill input"
}
}
```
### JetBrains IDEs (WebStorm/IntelliJ)
1. Go to **Settings > Plugins > Marketplace**, search for "Playwright" and install
2. Go to **Run > Edit Configurations**, add a **Node.js** configuration:
- **JavaScript file**: `node_modules/.bin/playwright`
- **Application parameters**: `test tests/my-test.spec.ts --headed --workers=1`
- **Environment variables**: `PWDEBUG=1;SAP_BASE_URL=...`
3. JetBrains IDEs resolve `tsconfig.json` paths automatically — no additional configuration needed
### IDE Troubleshooting
| Problem | Solution |
| -------------------------------------- | ----------------------------------------------------------------- |
| ESLint not finding config | Set `eslint.workingDirectories` in VS Code settings |
| Path aliases not resolving | Verify `tsconfig.json` is in the project root |
| Playwright extension not finding tests | Set `playwright.testMatch` pattern in settings |
| Debugger not hitting breakpoints | Use `--workers=1` to disable parallelism |
| TypeScript errors in node_modules | Add `"skipLibCheck": true` to tsconfig |
| Slow IntelliSense | Exclude `node_modules`, `dist`, `test-results` in `tsconfig.json` |
## Cross-Platform Notes
- All file operations use `node:path` — no hardcoded `/` or `\` separators
- Works on Windows 10/11, macOS, and Linux
- `.auth/` uses `.gitignore` patterns — add `.auth/` to your `.gitignore`
- The seed file uses `SAP_CLOUD_BASE_URL` (not `SAP_BASE_URL`) — check your `.env`
---
## Integrating with AI Agent Frameworks
:::info Roadmap
Native integrations with the frameworks below are **on the Praman roadmap**. The architecture and adapter contracts described on this page reflect the planned design. Community contributions and early feedback are welcome — open a discussion on [GitHub](https://github.com/mrkanitkar/playwright-praman/discussions).
:::
Praman is designed from the ground up as an AI-first plugin. Its structured outputs — `AiResponse` envelopes, `IntentDescriptor` payloads, capability manifests, and vocabulary graphs — map naturally onto the tool-call and agent-step patterns used by modern AI orchestration frameworks.
This page describes the planned integration model for five major frameworks, explains what each integration will look like in practice, and lists the capabilities each adapter will expose.
---
## Integration Architecture
All framework adapters will follow the same two-layer model:
```text
┌──────────────────────────────────────────────────────────────┐
│ AI Agent Framework (LangGraph / AutoGen / OpenAI Agents…) │
│ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ Praman Adapter (playwright-praman/ai) │ │
│ │ │ │
│ │ Tool: navigateTo(intent) → ui5.press / page.goto │ │
│ │ Tool: fillField(id, val) → ui5.fill + waitForUI5 │ │
│ │ Tool: readControl(id) → ui5.getValue / getProperty│ │
│ │ Tool: waitForStable() → ui5.waitForUI5 │ │
│ │ Tool: getCapabilities() → CapabilityRegistry.list │ │
│ └────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ Playwright Browser ← Praman Fixtures + Bridge │ │
│ └────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
```
Each adapter will expose Praman's SAP operations as **typed tool definitions** that the framework's orchestrator can call, chain, and retry. The browser state (Playwright `Page`) is kept alive across agent steps using the same fixture injection model Praman already uses.
---
## LangGraph
[LangGraph](https://www.langchain.com/langgraph) (LangChain) models multi-step agent workflows as typed state machines where each node is a callable tool or sub-graph. Praman maps directly onto this model — each UI5 operation is a node with clearly typed inputs and outputs.
### Planned Capabilities
| LangGraph Concept | Praman Mapping |
| ------------------ | --------------------------------------------------------------- |
| State node | One `test.step()` → one SAP UI interaction |
| Tool node | `navigateTo`, `fillField`, `selectOption`, `assertControl` |
| Conditional edge | Branch on `AiResponse.status` (`success` / `error` / `partial`) |
| Interrupt + resume | Checkpoint serialization via Praman's `CheckpointState` type |
| Human-in-the-loop | Pause on SAP validation error, surface to operator, resume |
### What the Adapter Will Provide
```typescript
// Planned API — not yet available
const tools = createLangGraphAdapter(page, ui5);
// tools exposes: navigateTo, fillField, readControl,
// waitForStable, openValueHelp, selectFromTable,
// assertFieldValue, getCapabilities
```
### Example Agent Flow (Planned)
```typescript
const graph = new StateGraph({ channels: { messages: { value: [] } } })
.addNode('navigate', tools.navigateTo)
.addNode('openDialog', tools.press)
.addNode('fillMaterial', tools.fillField)
.addNode('fillPlant', tools.fillField)
.addNode('submit', tools.press)
.addConditionalEdges('submit', (state) =>
state.lastResult.status === 'success' ? 'verify' : 'handleError',
)
.addNode('verify', tools.assertControl)
.addNode('handleError', tools.gracefulCancel)
.compile();
```
:::tip Why LangGraph fits SAP workflows
SAP processes are inherently stateful and branching — a BOM creation may succeed, fail with a validation error, or require a different plant/material combination. LangGraph's conditional edges and interrupt/resume model handles this naturally without hard-coded `if/else` chains in test scripts.
:::
---
## AutoGen (Microsoft)
[AutoGen](https://microsoft.github.io/autogen/) is Microsoft's open-source framework for building multi-agent systems where specialized agents collaborate to complete complex tasks. Its conversational agent model aligns with Praman's planner → generator → healer pipeline.
### Planned Capabilities
| AutoGen Concept | Praman Mapping |
| ----------------- | ---------------------------------------------------------- |
| `AssistantAgent` | Praman planner — discovers SAP UI, produces test plan |
| `UserProxyAgent` | Praman executor — runs Playwright actions, reports results |
| `GroupChat` | Planner + Generator + Healer collaborate on a test suite |
| Tool registration | SAP UI tools registered on the executor agent |
| Code execution | Generated `.spec.ts` runs via `npx playwright test` |
### What the Adapter Will Provide
```typescript
// Planned API — not yet available
const sapTools = createAutoGenTools(page, ui5);
// Register on your AutoGen UserProxyAgent's function map
```
### Planned Agent Topology
```text
┌─────────────────┐ instructions ┌──────────────────┐
│ PlannerAgent │ ──────────────────► │ ExecutorAgent │
│ (AssistantAgent│ test steps │ (UserProxyAgent) │
│ + SAP skills) │ │ + Praman tools │
└─────────────────┘ └────────┬─────────┘
▲ │
│ result + errors │
└──────────────────────────────────────────┘
HealerAgent joins on failure
```
---
## Microsoft Agent Framework (Semantic Kernel)
[Semantic Kernel](https://learn.microsoft.com/en-us/semantic-kernel/overview/) is Microsoft's SDK for building AI-powered applications with plugins, planners, and memory. Praman's capability registry and vocabulary system are designed to expose as Semantic Kernel plugins.
### Planned Capabilities
| Semantic Kernel Concept | Praman Mapping |
| ----------------------- | ----------------------------------------------------------------------- |
| Native plugin | `SapUi5Plugin` — exposes all `ui5.*` fixtures as SK functions |
| Planner | Auto-plans multi-step SAP workflows from a natural language goal |
| Memory | Stores discovered control maps between agent sessions |
| Function calling | Each Praman tool becomes a describable SK kernel function |
| Filters | Pre/post hooks map to Praman's `BeforeAction` / `AfterAction` lifecycle |
### What the Adapter Will Provide
```typescript
// Planned API — not yet available
const kernel = new Kernel();
kernel.addPlugin(new SapUi5Plugin(page, ui5), 'SapUI5');
// Kernel can now plan: "Fill the BOM creation form and submit"
// and call SapUI5.fillField, SapUI5.press, SapUI5.waitForStable
```
### Plugin Functions (Planned)
| Function | Description |
| ----------------------------- | ----------------------------------- |
| `navigateToApp(appName)` | FLP navigation to a named Fiori app |
| `fillField(controlId, value)` | `ui5.fill()` with SAP field binding |
| `pressButton(controlId)` | `ui5.press()` with stability wait |
| `readFieldValue(controlId)` | `ui5.getValue()` |
| `openValueHelp(controlId)` | Open and enumerate value help table |
| `selectFromValueHelp(index)` | Pick row by OData binding index |
| `waitForStableUI()` | `ui5.waitForUI5()` |
| `getAvailableApps()` | Queries FLP tile catalog |
---
## OpenAI Agents SDK
The [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) (previously Swarm) provides a lightweight framework for building agents that hand off between each other and call registered tools. Praman tools map directly onto the SDK's function-tool model.
### Planned Capabilities
| OpenAI Agents SDK Concept | Praman Mapping |
| ------------------------- | --------------------------------------------------------------------- |
| `Agent` | One agent per Praman role (planner, executor, healer) |
| `function_tool` | Every `ui5.*` fixture becomes a callable tool definition |
| Handoffs | Planner → Generator → Healer pipeline with structured handoff context |
| Structured output | `AiResponse` maps to Pydantic / Zod response schemas |
| Tracing | Playwright `test.info().annotations` surface in OpenAI trace UI |
### What the Adapter Will Provide
```typescript
// Planned API — not yet available
const tools = createOpenAITools(page, ui5);
// tools: array of OpenAI function_tool definitions
// ready to pass to new Agent({ tools })
```
### Planned Tool Schema Example
```json
{
"name": "fill_sap_field",
"description": "Fill a SAP UI5 input field and fire the change event. Use for SmartField, MDC Field, and sap.m.Input controls.",
"parameters": {
"type": "object",
"properties": {
"controlId": { "type": "string", "description": "Stable SAP UI5 control ID from discovery" },
"value": { "type": "string", "description": "Value to set" },
"searchOpenDialogs": { "type": "boolean", "default": true }
},
"required": ["controlId", "value"]
}
}
```
---
## Google Agent Development Kit (ADK)
[Google's Agent Development Kit](https://google.github.io/adk-docs/) (ADK) is Google's open-source framework for building and deploying multi-agent systems, with first-class support for tool use, multi-agent pipelines, and Vertex AI integration. The ADK's `BaseTool` interface is a natural fit for Praman's typed SAP tools.
### Planned Capabilities
| Google ADK Concept | Praman Mapping |
| ------------------ | ---------------------------------------------------------- |
| `BaseTool` | Each `ui5.*` fixture becomes an ADK tool with typed schema |
| `LlmAgent` | SAP planner, generator, and healer as ADK agents |
| `SequentialAgent` | Plan → Generate → Heal pipeline |
| `ParallelAgent` | Concurrent discovery of multiple Fiori apps |
| Vertex AI backend | Cloud-hosted planning with Gemini models |
| Session state | Shared control map and vocabulary across agent steps |
### What the Adapter Will Provide
```typescript
// Planned API — not yet available
const sapTools = createADKTools(page, ui5);
const plannerAgent = new LlmAgent({
name: 'SapTestPlanner',
model: 'gemini-2.0-flash',
tools: sapTools,
instruction: 'Discover the SAP UI5 app and produce a structured test plan.',
});
```
### ADK Pipeline (Planned)
```text
SequentialAgent
├── SapPlannerAgent → discovers UI5 controls, writes test plan
├── SapGeneratorAgent → converts plan to playwright-praman spec
└── SapHealerAgent → runs test, reads failure, iterates
```
:::tip Vertex AI + SAP
The ADK integration will support Vertex AI backends, enabling teams to run the planning and generation pipeline entirely within Google Cloud — useful for enterprises with data residency requirements.
:::
---
## Comparison
| Framework | Best For | Orchestration Model | Praman Fit |
| --------------------- | -------------------------------------------------- | -------------------------- | -------------------------------------- |
| **LangGraph** | Complex branching SAP workflows, human-in-the-loop | State machine graph | Excellent — maps to test.step() |
| **AutoGen** | Multi-agent collaboration (planner + healer) | Conversational agents | Excellent — mirrors Praman pipeline |
| **Semantic Kernel** | Microsoft ecosystem, enterprise .NET/Python | Plugin + planner | Good — capability registry as plugin |
| **OpenAI Agents SDK** | Lightweight, fast iteration, OpenAI models | Function tools + handoffs | Good — clean tool schema mapping |
| **Google ADK** | Vertex AI, Gemini models, GCP-native pipelines | Sequential/parallel agents | Good — structured ADK tool definitions |
---
## Current Workaround: Use Praman Agents Directly
While framework-native adapters are in development, you can orchestrate Praman's three agents today using **Claude Code MCP** or **GitHub Copilot Agent Mode** with the included agent definitions:
```text
.github/agents/praman-sap-planner.agent.md
.github/agents/praman-sap-generator.agent.md
.github/agents/praman-sap-healer.agent.md
```
See [Running Your Agent for the First Time](./running-your-agent.md) for the complete pipeline.
---
## Contribute or Follow Progress
- **GitHub Discussions**: Share your framework preference and use case
- **Issues**: Track individual adapter issues under the `ai-adapters` label
- **Roadmap**: The order of delivery will reflect community demand — upvote your framework
```text
Planned delivery order (subject to change):
1. OpenAI Agents SDK adapter ← lightest surface area
2. LangGraph adapter ← most requested for SAP workflows
3. Google ADK adapter ← Vertex AI enterprise demand
4. AutoGen adapter ← multi-agent collaboration
5. Semantic Kernel plugin ← Microsoft ecosystem
```
---
## Gold Standard Test Pattern
A complete reference test that demonstrates every Praman best practice in a single runnable
example. Use this as a template for your most critical business process tests.
## What Makes a "Gold Standard" Test
| Practice | Why It Matters |
| ------------------------------------ | -------------------------------------- |
| `test.step()` for every logical step | Clear trace output, pinpoints failures |
| Fixtures for all SAP interactions | Type-safe, auto-waiting, reusable |
| Semantic selectors (not DOM) | Resilient to UI5 version upgrades |
| Custom matchers for assertions | Readable, consistent error messages |
| Structured error handling | Actionable failure diagnostics |
| Auth via setup project | One login, shared across tests |
| Data cleanup in `afterAll` | Repeatable, no test pollution |
| No `page.waitForTimeout()` | Deterministic, fast execution |
## Complete Example
```typescript
/**
* Gold Standard: Purchase Order Approval Flow
*
* Tests the complete PO creation and approval workflow:
* 1. Create PO with required approvals
* 2. Verify PO status and workflow
* 3. Approve the PO
* 4. Verify final status
* 5. Clean up test data
*/
test.describe('Purchase Order Approval Flow', () => {
// Track created entities for cleanup
let createdPONumber: string;
// ── Auth: handled by setup project dependency ──
// See playwright.config.ts: { dependencies: ['setup'] }
test.beforeEach(async ({ ui5Navigation }) => {
await test.step('navigate to Fiori Launchpad home', async () => {
await ui5Navigation.navigateToIntent('#Shell-home');
});
});
test('create and approve purchase order', async ({
ui5,
ui5Navigation,
ui5Matchers,
feListReport,
feObjectPage,
page,
}) => {
// ── Step 1: Create Purchase Order ──
await test.step('create purchase order', async () => {
await test.step('navigate to PO creation', async () => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
});
await test.step('fill header data', async () => {
await feObjectPage.fillField('Supplier', '100001');
await feObjectPage.fillField('PurchasingOrganization', '1000');
await feObjectPage.fillField('CompanyCode', '1000');
await feObjectPage.fillField('PurchasingGroup', '001');
});
await test.step('add line item', async () => {
await feObjectPage.clickSectionCreate('Items');
await feObjectPage.fillField('Material', 'MAT-APPROVE-001');
await feObjectPage.fillField('OrderQuantity', '500');
await feObjectPage.fillField('Plant', '1000');
await feObjectPage.fillField('NetPrice', '10000.00');
});
await test.step('save and capture PO number', async () => {
await feObjectPage.clickSave();
// Assert success message
const messageStrip = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
await ui5Matchers.toHaveText(messageStrip, /Purchase Order (\d+) created/);
// Capture PO number for subsequent steps and cleanup
const text = await messageStrip.getText();
const match = text.match(/Purchase Order (\d+)/);
createdPONumber = match?.[1] ?? '';
expect(createdPONumber).toBeTruthy();
});
});
// ── Step 2: Verify PO Requires Approval ──
await test.step('verify PO status requires approval', async () => {
await test.step('navigate to PO display', async () => {
await ui5Navigation.navigateToIntent(
`#PurchaseOrder-display?PurchaseOrder=${createdPONumber}`,
);
});
await test.step('check status field', async () => {
const statusField = await ui5.control({
controlType: 'sap.m.ObjectStatus',
id: /overallStatusText/,
});
await ui5Matchers.toHaveText(statusField, 'Awaiting Approval');
});
await test.step('verify workflow section exists', async () => {
await ui5.click({
controlType: 'sap.m.IconTabFilter',
properties: { key: 'workflow' },
});
const workflowTable = await ui5.control({
controlType: 'sap.m.Table',
id: /workflowItemsTable/,
});
await ui5Matchers.toHaveRowCount(workflowTable, { min: 1 });
});
});
// ── Step 3: Approve the Purchase Order ──
await test.step('approve the purchase order', async () => {
await test.step('navigate to approval inbox', async () => {
await ui5Navigation.navigateToIntent('#WorkflowTask-displayInbox');
});
await test.step('find and open PO approval task', async () => {
// Filter for our specific PO
await feListReport.setFilterValues({
TaskTitle: `Purchase Order ${createdPONumber}`,
});
await feListReport.clickGo();
// Click the task row
await ui5.click({
controlType: 'sap.m.ColumnListItem',
ancestor: { controlType: 'sap.m.Table', id: /taskListTable/ },
});
});
await test.step('click approve button', async () => {
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Approve' },
});
// Confirm approval dialog if present
const confirmButton = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Confirm' },
searchOpenDialogs: true,
});
await confirmButton.click();
});
await test.step('verify approval success', async () => {
await ui5Matchers.toHaveMessageStrip('Success', /approved/i);
});
});
// ── Step 4: Verify Final Status ──
await test.step('verify PO is approved', async () => {
await ui5Navigation.navigateToIntent(
`#PurchaseOrder-display?PurchaseOrder=${createdPONumber}`,
);
const statusField = await ui5.control({
controlType: 'sap.m.ObjectStatus',
id: /overallStatusText/,
});
await ui5Matchers.toHaveText(statusField, 'Approved');
});
});
// ── Step 5: Cleanup ──
test.afterAll(async ({ ui5 }) => {
if (createdPONumber) {
await test.step('delete test purchase order', async () => {
try {
await ui5.odata.deleteEntity('/PurchaseOrders', createdPONumber);
} catch {
// Log but do not fail the test suite on cleanup errors
console.warn(`Cleanup: failed to delete PO ${createdPONumber}`);
}
});
}
});
});
```
## Anatomy of the Gold Standard
### 1. Test Organization
```
describe('Business Process Name')
beforeEach → navigate to known starting point
test → the actual business flow
step 1 → first business action (may have sub-steps)
step 2 → verification
step 3 → next action
step N → final verification
afterAll → cleanup created data
```
### 2. Selector Strategy (Priority Order)
Use the most resilient selector available. In priority order:
```typescript
// 1. Binding path (most resilient — survives ID changes)
await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/CompanyCode' },
});
// 2. Properties (semantic — survives refactoring)
await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
// 3. ID with RegExp (flexible — survives view prefix changes)
await ui5.control({
controlType: 'sap.m.Input',
id: /materialInput/,
});
// 4. Exact ID (brittle — use only when nothing else works)
await ui5.control({
controlType: 'sap.m.Input',
id: 'myApp--detailView--materialInput',
});
```
### 3. Assertion Patterns
```typescript
// Use custom matchers for SAP-specific assertions
await ui5Matchers.toHaveText(control, 'Expected Text');
await ui5Matchers.toHaveText(control, /partial match/);
await ui5Matchers.toHaveProperty(control, 'enabled', true);
await ui5Matchers.toHaveRowCount(table, 5);
await ui5Matchers.toHaveRowCount(table, { min: 1 });
await ui5Matchers.toHaveMessageStrip('Success', /created/);
// For standard Playwright assertions on the page
await expect(page).toHaveURL(/PurchaseOrder/);
await expect(page).toHaveTitle(/SAP/);
```
### 4. Error Handling Pattern
```typescript
await test.step('handle potential error dialogs', async () => {
// Check for and dismiss error popups before proceeding
const errorDialog = await ui5.controlOrNull({
controlType: 'sap.m.Dialog',
properties: { type: 'Message' },
searchOpenDialogs: true,
});
if (errorDialog) {
const errorText = await errorDialog.getProperty('content');
// Attach error details to the test report
await test.info().attach('sap-error-dialog', {
body: JSON.stringify(errorText, null, 2),
contentType: 'application/json',
});
// Dismiss the dialog
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Close' },
searchOpenDialogs: true,
});
// Fail with actionable message
expect.fail(`SAP error dialog appeared: ${JSON.stringify(errorText)}`);
}
});
```
### 5. Data-Driven Variant
For running the same test with multiple data sets:
```typescript
const testData = [
{ supplier: '100001', material: 'MAT-001', quantity: '10', plant: '1000' },
{ supplier: '100002', material: 'MAT-002', quantity: '50', plant: '2000' },
{ supplier: '100003', material: 'MAT-003', quantity: '100', plant: '3000' },
];
for (const data of testData) {
test(`create PO for supplier ${data.supplier}`, async ({
ui5Navigation,
feObjectPage,
ui5Matchers,
}) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
await feObjectPage.fillField('Supplier', data.supplier);
await feObjectPage.fillField('Material', data.material);
await feObjectPage.fillField('OrderQuantity', data.quantity);
await feObjectPage.fillField('Plant', data.plant);
await feObjectPage.clickSave();
await ui5Matchers.toHaveMessageStrip('Success', /created/);
});
}
```
## Playwright Config for Gold Standard Tests
```typescript
// playwright.config.ts
export default defineConfig({
testDir: './tests',
timeout: 120_000, // 2 minutes for SAP transactions
retries: 1,
workers: 1, // Sequential for SAP state-dependent tests
use: {
baseURL: process.env.SAP_BASE_URL,
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'on-first-retry',
},
projects: [
{
name: 'setup',
testMatch: /auth-setup\.ts/,
use: { ...devices['Desktop Chrome'] },
},
{
name: 'gold-standard',
dependencies: ['setup'],
testMatch: /gold-standard\.spec\.ts/,
use: {
...devices['Desktop Chrome'],
storageState: '.auth/sap-session.json',
},
},
],
reporter: [
['html', { open: 'never' }],
['playwright-praman/reporters', { outputDir: 'reports' }],
],
});
```
## Checklist
Use this checklist when writing new tests to verify they meet gold standard quality:
- [ ] Every logical step wrapped in `test.step()`
- [ ] Selectors use UI5 control types (not CSS selectors)
- [ ] Assertions use Praman custom matchers where applicable
- [ ] No `page.waitForTimeout()` calls
- [ ] No hardcoded DOM selectors
- [ ] Document numbers captured and used for cross-referencing
- [ ] Data cleanup in `afterAll` or `afterEach`
- [ ] Error dialog handling for SAP message popups
- [ ] Auth handled by setup project (not inline login)
- [ ] Test can run independently (no dependency on prior test state)
- [ ] Trace and screenshot configured for failure analysis
---
## Playwright Primer for SAP Testers
A ground-up introduction to Playwright and Praman for testers coming from SAP testing backgrounds
(ECATT, Tosca, QTP/UFT, Solution Manager, CBTA). No prior Playwright or programming experience
is assumed.
## What Is Playwright?
Playwright is an open-source test automation framework from Microsoft. It drives real browsers
(Chrome, Firefox, Safari) to simulate user interactions -- clicking buttons, filling forms,
reading text -- and verifies that applications behave correctly.
Unlike SAP GUI-based tools, Playwright works with **web applications in the browser**. Since
SAP Fiori, SAP UI5, and BTP apps all run in the browser, Playwright is a natural fit.
### Why Not ECATT or Tosca?
| Concern | ECATT / Tosca | Playwright + Praman |
| -------------------- | -------------------------- | --------------------------------------------------- |
| SAP GUI support | Native | Not applicable (browser only) |
| Fiori / UI5 web apps | Limited or plugin-required | Native + UI5-aware |
| Parallel execution | Difficult | Built-in |
| CI/CD integration | Complex setup | First-class (GitHub Actions, Azure DevOps, Jenkins) |
| Cost | Licensed | Open source |
| Version control | Limited | Git-native (tests are code files) |
| AI integration | None | Built-in LLM + agentic support |
:::tip When to Keep SAP GUI Tools
If your tests target **SAP GUI transactions** (not Fiori web apps), tools like ECATT or Tosca
remain the right choice. Playwright and Praman target **browser-based** SAP applications only.
:::
## Prerequisite Learning Path
If you are starting from zero programming experience, follow this learning path before writing
Praman tests. Each step builds on the previous one.
### Step 1: Command Line Basics (1 hour)
Learn to navigate directories, run commands, and use a terminal.
- **Windows**: Open PowerShell or Windows Terminal
- **macOS/Linux**: Open Terminal
Key commands to learn:
```bash
# Navigate to a folder
cd Documents/my-project
# List files
ls
# Run a Node.js script
node my-script.js
# Run tests
npx playwright test
```
### Step 2: Node.js and npm (1 hour)
Install Node.js (version 20 or later) from [nodejs.org](https://nodejs.org).
```bash
# Verify installation
node --version # Should print v20.x.x or later
npm --version # Should print 10.x.x or later
# Create a new project
mkdir my-sap-tests
cd my-sap-tests
npm init -y
# Install dependencies
npm install playwright-praman @playwright/test
npx playwright install chromium
```
### Step 3: TypeScript Basics (2-3 hours)
Tests are written in TypeScript, a typed version of JavaScript. You need to understand:
- **Variables**: `const name = 'John';` (a value that does not change)
- **Strings**: `'Hello'` or `"Hello"` or `` `Hello ${name}` ``
- **Objects**: `{ vendor: '100001', amount: 500 }` (a group of named values)
- **Arrays**: `['item1', 'item2', 'item3']` (a list of values)
- **Functions**: `function greet(name: string) { return 'Hello ' + name; }`
- **Arrow functions**: `const greet = (name: string) => 'Hello ' + name;`
Free resource: [TypeScript in 5 minutes](https://www.typescriptlang.org/docs/handbook/typescript-in-5-minutes.html)
### Step 4: async/await (30 minutes)
Browser operations take time (loading pages, finding elements, clicking buttons). TypeScript
uses `async/await` to handle this waiting gracefully.
```typescript
// BAD: This does NOT wait for the page to load
function badExample() {
page.goto('https://my-app.com'); // Returns immediately, page not loaded yet
}
// GOOD: This waits for the page to finish loading
async function goodExample() {
await page.goto('https://my-app.com'); // Pauses here until page loads
}
```
**Rule of thumb**: Every function that interacts with the browser needs `async` in its
declaration and `await` before each browser call.
## Your First Playwright Test
Create a file called `tests/hello.test.ts`:
```typescript
test('my first test', async ({ page }) => {
// Go to a web page
await page.goto('https://example.com');
// Check the page title
await expect(page).toHaveTitle('Example Domain');
});
```
Run it:
```bash
npx playwright test tests/hello.test.ts
```
### Anatomy of a Test
```typescript
// ^^^^ ^^^^^^
// | Tool for checking results ("assertions")
// Framework for defining tests
test('description of what this test verifies', async ({ page }) => {
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^
// Human-readable test name "page" = a browser tab
await page.goto('https://my-app.com');
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// Navigate the browser to this URL
const heading = page.locator('h1');
// ^^^^^^^
// Find an element on the page
await expect(heading).toHaveText('Welcome');
// ^^^^^^ ^^^^^^^^^^
// Assert/verify Expected value
});
```
## Fixtures: Automatic Setup
In ECATT, you might use "variants" to inject test data. In Playwright, **fixtures** provide
pre-configured objects to your tests automatically.
```typescript
test('using fixtures', async ({ ui5, ui5Navigation, sapAuth }) => {
// ^^^ ^^^^^^^^^^^^^^ ^^^^^^^
// | | Authentication helper
// | Navigation helper (9 methods)
// UI5 control finder and interaction API
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
const btn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create' } });
await expect(btn).toBeUI5Enabled();
});
```
You request fixtures by name in the test function signature. Praman provides 12 fixture modules:
| Fixture | What It Does |
| --------------- | ----------------------------------------------------- |
| `ui5` | Find and interact with UI5 controls |
| `ui5Navigation` | Navigate FLP apps, tiles, intents |
| `sapAuth` | Login/logout with 6 strategies |
| `fe` | Fiori Elements List Report + Object Page helpers |
| `pramanAI` | AI page discovery and test generation |
| `intent` | Business domain intents (procurement, sales, finance) |
| `ui5Shell` | FLP shell header (home, user menu) |
| `ui5Footer` | Page footer bar (Save, Edit, Delete) |
| `flpLocks` | SM12 lock management |
| `flpSettings` | User settings reader |
| `testData` | Test data generation with auto-cleanup |
## test.step(): Structured Test Steps
In ECATT, test scripts have numbered steps. In Playwright, use `test.step()` to create the
same structure.
```typescript
test('create a purchase order', async ({ ui5, ui5Navigation, fe }) => {
await test.step('Step 1: Navigate to the PO app', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
await test.step('Step 2: Click Create', async () => {
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Create' },
});
});
await test.step('Step 3: Fill the vendor field', async () => {
await ui5.fill({ id: 'vendorInput' }, '100001');
});
await test.step('Step 4: Save', async () => {
await fe.objectPage.clickSave();
});
await test.step('Step 5: Verify success message', async () => {
const messageStrip = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
await expect(messageStrip).toBeUI5Visible();
});
});
```
Steps appear as collapsible sections in the Playwright HTML report.
## Assertions: Checking Results
Assertions verify that the application behaves correctly. If an assertion fails, the test fails.
### Standard Playwright Assertions
```typescript
// Check page title
await expect(page).toHaveTitle('Purchase Orders');
// Check URL
await expect(page).toHaveURL(/PurchaseOrder/);
// Check text content
await expect(page.locator('#message')).toHaveText('Saved');
```
### Praman UI5 Assertions
```typescript
const button = await ui5.control({ id: 'saveBtn' });
// Is the button enabled?
await expect(button).toBeUI5Enabled();
// Does it have the right text?
await expect(button).toHaveUI5Text('Save');
// Is it visible?
await expect(button).toBeUI5Visible();
// Check a specific property
await expect(button).toHaveUI5Property('type', 'Emphasized');
// Check value state (for inputs)
const input = await ui5.control({ id: 'amountInput' });
await expect(input).toHaveUI5ValueState('Error');
```
## Running Tests
### Basic Commands
```bash
# Run all tests
npx playwright test
# Run a specific test file
npx playwright test tests/purchase-order.test.ts
# Run tests matching a name pattern
npx playwright test --grep "purchase order"
# Run with visible browser (headed mode)
npx playwright test --headed
# Run in debug mode (step through interactively)
npx playwright test --debug
```
### Viewing Reports
```bash
# Open the HTML report after a test run
npx playwright show-report
```
The HTML report shows:
- Pass/fail status for each test
- `test.step()` sections collapsed per step
- Screenshots on failure (automatic)
- Traces on retry (automatic)
## Project Structure
A typical Praman test project looks like this:
```text
my-sap-tests/
tests/
auth-setup.ts # Login once, save session
auth-teardown.ts # Logout
purchase-order.test.ts # Test file
vendor.test.ts # Another test file
.auth/
sap-session.json # Saved browser session (auto-generated, git-ignored)
.env # Environment variables (git-ignored)
praman.config.ts # Praman configuration
playwright.config.ts # Playwright configuration
package.json # Dependencies
tsconfig.json # TypeScript configuration
```
## Common Patterns
### Finding a UI5 Control
```typescript
// By ID
const btn = await ui5.control({ id: 'saveBtn' });
// By type + property
const input = await ui5.control({
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor' },
});
// By binding path
const field = await ui5.control({
controlType: 'sap.m.Input',
bindingPath: { value: '/PurchaseOrder/Vendor' },
});
```
### Interacting with Controls
```typescript
// Click
await ui5.click({ id: 'submitBtn' });
// Type text
await ui5.fill({ id: 'nameInput' }, 'John Doe');
// Select from dropdown
await ui5.select({ id: 'countrySelect' }, 'US');
// Toggle checkbox
await ui5.check({ id: 'agreeCheckbox' });
// Clear a field
await ui5.clear({ id: 'searchField' });
```
### Navigating SAP Apps
```typescript
// Navigate to an app by semantic object and action
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
// Navigate to a tile by title
await ui5Navigation.navigateToTile('Create Purchase Order');
// Navigate with parameters
await ui5Navigation.navigateToIntent('PurchaseOrder', 'create', { plant: '1000' });
// Go home
await ui5Navigation.navigateToHome();
```
## Glossary for SAP Testers
| SAP Term | Playwright/Praman Equivalent |
| --------------------------- | -------------------------------------------------- |
| Test script | Test file (`.test.ts`) |
| Test step / script line | `test.step()` |
| Variant / test data set | `testData.generate()` fixture |
| Check / verification | `expect()` assertion |
| Test configuration | `playwright.config.ts` + `praman.config.ts` |
| Transaction code | `ui5Navigation.navigateToIntent()` |
| Control ID | `UI5Selector.id` |
| Element repository | UI5 control registry (queried via `ui5.control()`) |
| Test run | `npx playwright test` |
| Test report | `npx playwright show-report` (HTML) |
| Reusable component / module | Playwright fixture |
| Test suite | `describe()` block or test file |
---
## Reporters
Praman ships two custom Playwright reporters: **ComplianceReporter** for measuring Praman
adoption across your test suite, and **ODataTraceReporter** for capturing OData HTTP request
performance data.
## ComplianceReporter
Tracks whether test steps use Praman abstractions (`ui5.click()`, `ui5Navigation.*`) versus
raw Playwright calls (`page.click()`, `page.fill()`). Generates a compliance report with
per-test and aggregate statistics.
### Configuration
```typescript
// playwright.config.ts
export default {
reporter: [['html'], [ComplianceReporter, { outputDir: 'reports' }]],
};
```
### Output
The reporter writes `compliance-report.json` to the configured `outputDir`:
```json
{
"summary": {
"totalTests": 45,
"compliantTests": 38,
"rawPlaywrightTests": 3,
"mixedTests": 4,
"compliancePercentage": 84.4
},
"tests": [
{
"title": "create purchase order",
"file": "tests/procurement/create-po.spec.ts",
"status": "compliant",
"pramanSteps": 12,
"rawSteps": 0
},
{
"title": "legacy login flow",
"file": "tests/auth/login.spec.ts",
"status": "raw-playwright",
"pramanSteps": 0,
"rawSteps": 5
},
{
"title": "mixed operations",
"file": "tests/mixed/hybrid.spec.ts",
"status": "mixed",
"pramanSteps": 8,
"rawSteps": 2
}
]
}
```
### Test Status Categories
| Status | Meaning |
| ---------------- | -------------------------------------------- |
| `compliant` | All steps use Praman abstractions |
| `raw-playwright` | All steps use raw Playwright calls |
| `mixed` | Some Praman steps, some raw Playwright steps |
### Step Detection
The reporter recognizes 50+ Praman step prefixes to classify steps:
- `UI5Handler >` — core control operations
- `ui5.table.` — table operations
- `ui5.dialog.` — dialog operations
- `ui5.date.` — date/time operations
- `ui5.odata.` — OData operations
- `ui5Navigation.` — navigation operations
- `sapAuth.` — authentication operations
- `fe.listReport.` / `fe.objectPage.` — Fiori Elements helpers
- `pramanAI.` — AI operations
- `intent.` — domain intent operations
Any step not matching these prefixes is classified as a raw Playwright step.
### Use Case: Migration Tracking
If your team is migrating from raw Playwright to Praman, run the ComplianceReporter in CI to
track progress:
```typescript
// CI assertion (optional)
if (report.summary.compliancePercentage < 80) {
console.warn(`Compliance at ${report.summary.compliancePercentage}% — target is 80%`);
}
```
## ODataTraceReporter
Captures OData HTTP request traces from test runs, providing per-entity-set statistics including
call counts, durations, and error rates.
### Configuration
```typescript
// playwright.config.ts
export default {
reporter: [['html'], [ODataTraceReporter, { outputDir: 'reports' }]],
};
```
### Output
The reporter writes `odata-trace.json` to the configured `outputDir`:
```json
{
"traces": [
{
"method": "GET",
"url": "/sap/opu/odata/sap/API_PO_SRV/PurchaseOrders?$top=20&$filter=Status eq 'A'",
"entitySet": "PurchaseOrders",
"queryParams": {
"$top": "20",
"$filter": "Status eq 'A'"
},
"status": 200,
"duration": 340,
"size": 12480,
"isBatch": false
}
],
"aggregates": {
"PurchaseOrders": {
"GET": { "count": 12, "avgDuration": 340, "maxDuration": 1200, "errors": 0 },
"POST": { "count": 2, "avgDuration": 890, "maxDuration": 1100, "errors": 0 }
},
"Vendors": {
"GET": { "count": 3, "avgDuration": 120, "maxDuration": 200, "errors": 0 }
}
},
"totalRequests": 18,
"totalErrors": 0,
"totalDuration": 8400
}
```
### URL Parsing
The reporter automatically extracts entity set names and query parameters from OData URLs:
| URL Pattern | Extracted Entity Set |
| ---------------------------------------------------- | -------------------- |
| `/sap/opu/odata/sap/SRV/PurchaseOrders` | `PurchaseOrders` |
| `/sap/opu/odata/sap/SRV/PurchaseOrders('123')` | `PurchaseOrders` |
| `/sap/opu/odata/sap/SRV/PurchaseOrders('123')/Items` | `Items` |
| `/sap/opu/odata/sap/SRV/$batch` | `$batch` |
### Use Case: Performance Analysis
Identify slow or frequently called entity sets:
```typescript
// Find entity sets with high error rates
for (const [entitySet, methods] of Object.entries(trace.aggregates)) {
for (const [method, stats] of Object.entries(methods)) {
if (stats.errors > 0) {
console.warn(`${method} ${entitySet}: ${stats.errors} errors out of ${stats.count} calls`);
}
if (stats.avgDuration > 1000) {
console.warn(`${method} ${entitySet}: avg ${stats.avgDuration}ms — consider optimization`);
}
}
}
```
## Using Both Reporters Together
Both reporters can run alongside Playwright's built-in reporters:
```typescript
// playwright.config.ts
export default {
reporter: [
['html'],
['json', { outputFile: 'reports/results.json' }],
[ComplianceReporter, { outputDir: 'reports' }],
[ODataTraceReporter, { outputDir: 'reports' }],
],
};
```
After a test run, the `reports/` directory contains:
```
reports/
├── compliance-report.json ← Praman adoption metrics
├── odata-trace.json ← OData performance data
└── results.json ← Playwright test results
```
## Reporter Lifecycle
Both reporters implement Playwright's `Reporter` interface:
- `onBegin()` — initialization, create output directory
- `onTestEnd()` — process test steps and attachments
- `onEnd()` — aggregate data, write JSON output
They run passively alongside tests — no test code changes are needed. Add them to
`playwright.config.ts` and they start collecting data on the next test run.
---
## Docker & CI/CD
Praman's CI/CD pipeline runs on GitHub Actions with a 3-OS matrix, Docker support for containerized test execution, and comprehensive quality gates.
## GitHub Actions CI Pipeline
The CI workflow (`.github/workflows/ci.yml`) runs on every push to `main` and every pull request. It consists of 6 parallel jobs:
### Job Overview
```
+--------------------+ +-------------------------+ +-----------------+
| quality | | unit-tests | | build |
| (lint, typecheck, | | (3 OS x 3 Node) | | (3 OS) |
| spell, deadcode, | | ubuntu/windows/macos | | ESM + CJS + DTS |
| markdown lint) | | Node 20/22/24 | | attw, size-limit|
+--------------------+ +-------------------------+ +-----------------+
| |
+-------------------------+-------------------------------+
|
+--------------v--------------+
| integration-tests |
| (PW 1.57.0, PW 1.58.2) |
| SAP cloud auth via secrets |
+-----------------------------+
+--------------------+ +-------------------------+
| security | | docs-check |
| (npm audit) | | (TypeDoc + Docusaurus) |
+--------------------+ +-------------------------+
```
### 3-OS Matrix (Unit Tests)
Unit tests run across a full OS and Node.js matrix:
```yaml
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
node-version: [20, 22, 24]
```
This produces 9 test runs (3 OS x 3 Node versions) to catch platform-specific issues early.
### Quality Job
```yaml
- run: npm run lint # ESLint with 11 plugins, 0 errors/warnings
- run: npm run typecheck # tsc --noEmit (strict mode)
- run: npx cspell "src/**/*.ts" "docs/**/*.md" --no-progress # Spell check
- run: npx knip # Dead code detection
- run: npx markdownlint-cli2 "**/*.md" # Markdown lint
```
### Build Job
```yaml
- run: npm run build # tsup: ESM + CJS + DTS
- run: node -e "const m = require('./dist/index.cjs'); ..." # CJS smoke test
- run: npx size-limit # Bundle size check
- run: npm run check:exports # attw: validate all 6 export maps
```
### Integration Tests
Integration tests run against SAP cloud systems with Playwright version matrix:
```yaml
strategy:
matrix:
playwright-version: ['1.57.0', '1.58.2']
env:
SAP_CLOUD_BASE_URL: ${{ secrets.SAP_CLOUD_BASE_URL }}
SAP_CLOUD_USERNAME: ${{ secrets.SAP_CLOUD_USERNAME }}
SAP_CLOUD_PASSWORD: ${{ secrets.SAP_CLOUD_PASSWORD }}
```
### Security Scan
```yaml
- run: npm audit --audit-level=high
```
### Documentation Validation
```yaml
- run: npx typedoc --validation # TypeDoc API docs
- run: npm run build -- --no-minify # Docusaurus site build (in docs/)
```
### Azure Playwright Workspaces (Optional)
Triggered manually or by adding the `azure-test` label to a PR:
```yaml
- run: npx playwright test --config=playwright.service.config.ts --workers=20
env:
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
```
## npm run ci Breakdown
The local `npm run ci` command chains all quality gates:
```bash
npm run ci
# Equivalent to:
# npm run lint && npm run typecheck && npm run test:unit && npm run build
```
| Step | Command | Purpose |
| ---------- | ------------------------------------- | -------------------------------------------- |
| Lint | `eslint src/ tests/ --max-warnings=0` | 11 ESLint plugins, zero tolerance |
| Typecheck | `tsc --noEmit` | TypeScript strict mode validation |
| Unit tests | `vitest run` | Hermetic unit tests with Vitest |
| Build | `tsup` | Dual ESM + CJS output with type declarations |
Additional quality commands available:
| Command | Purpose |
| --------------------------------- | ----------------------------------------- |
| `npm run test:unit -- --coverage` | Unit tests with V8 coverage |
| `npm run check:exports` | Validate all 6 sub-path exports with attw |
| `npm run format:check` | Prettier formatting check |
| `npm run spellcheck` | CSpell spell checking |
| `npm run deadcode` | Knip dead code detection |
| `npm run mdlint` | Markdown lint |
## Docker for Running Tests
### Dockerfile for Test Execution
Create a `Dockerfile` for running Praman tests in a container:
```dockerfile
FROM mcr.microsoft.com/playwright:v1.57.0-noble
WORKDIR /app
# Copy package files and install
COPY package.json package-lock.json ./
RUN npm ci
# Copy source and test files
COPY . .
# Build the project
RUN npm run build
# Install Playwright browsers
RUN npx playwright install --with-deps chromium
# Default: run all tests
CMD ["npm", "run", "test:integration"]
```
### Running Tests in Docker
```bash
# Build the test image
docker build -t praman-tests .
# Run integration tests
docker run --rm \
-e SAP_CLOUD_BASE_URL="https://your-sap-system.example.com" \
-e SAP_CLOUD_USERNAME="testuser" \
-e SAP_CLOUD_PASSWORD="secret" \
praman-tests
# Run only unit tests
docker run --rm praman-tests npm run test:unit
# Run with coverage
docker run --rm -v $(pwd)/coverage:/app/coverage \
praman-tests npm run test:unit -- --coverage
```
### Docker Compose for Local SAP Mock
For local development with a mock SAP backend:
```yaml
# docker-compose.yml
services:
sap-mock:
image: node:20-slim
working_dir: /app
volumes:
- ./tests/mocks/odata-server:/app
command: node server.js
ports:
- '4004:4004'
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:4004/health']
interval: 5s
timeout: 3s
retries: 5
tests:
build: .
depends_on:
sap-mock:
condition: service_healthy
environment:
- SAP_CLOUD_BASE_URL=http://sap-mock:4004
- SAP_CLOUD_USERNAME=mock-user
- SAP_CLOUD_PASSWORD=mock-pass
volumes:
- ./playwright-report:/app/playwright-report
- ./test-results:/app/test-results
```
Run the full stack:
```bash
# Start mock server and run tests
docker compose up --build --abort-on-container-exit
# View test results
open playwright-report/index.html
```
## Security Practices in CI
All GitHub Actions use SHA-pinned action references:
```yaml
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
- uses: actions/setup-node@cdca7365b2dadb8aad0a33bc7601856ffabcc48e # v4
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
```
Additional security measures:
- `permissions: contents: read` (minimal permissions)
- `concurrency` groups prevent duplicate runs
- `cancel-in-progress: true` for PRs
- SAP credentials stored as GitHub Secrets, never in code
- `npm audit --audit-level=high` in every CI run
## Coverage Reporting
Coverage artifacts are uploaded per-matrix entry for comparison:
```yaml
- uses: actions/upload-artifact
with:
name: coverage-${{ matrix.os }}-node-${{ matrix.node-version }}
path: coverage/
```
Coverage thresholds are enforced per-file (not just project-wide):
| Tier | Scope | Statements | Branches |
| ------ | ------------------- | ---------- | -------- |
| Tier 1 | Error classes | 100% | 100% |
| Tier 2 | Core infrastructure | 95% | 90% |
| Tier 3 | All other modules | 90% | 85% |
## Praman-Specific Azure Playwright Configuration
When running Praman tests with [Azure Playwright Testing](https://learn.microsoft.com/en-us/azure/playwright-testing/), the bridge must communicate with cloud-hosted browsers. This works out of the box because the bridge uses standard `page.evaluate()` calls that serialize across remote browser connections.
### Service Configuration
Create a `playwright.service.config.ts` that extends your base config:
```typescript
export default defineConfig(baseConfig, getServiceConfig(baseConfig), {
workers: 20,
use: {
// Increase timeouts for cloud browser latency
actionTimeout: 15_000,
navigationTimeout: 60_000,
},
expect: {
timeout: 10_000,
},
});
```
### Environment Variables
Set these in your CI pipeline or `.env` file:
```bash
PLAYWRIGHT_SERVICE_URL=wss://eastus.api.playwright.microsoft.com/...
PLAYWRIGHT_SERVICE_ACCESS_TOKEN=
# Praman bridge timeout (higher for cloud latency)
PRAMAN_CONTROL_DISCOVERY_TIMEOUT=45000
```
### Worker and Timeout Considerations
Azure Playwright Testing supports up to 50 parallel workers. Keep these points in mind:
- **SAP session limits**: Your SAP system may restrict concurrent sessions per user. Use separate test accounts per worker shard, or reduce `workers` to match your license.
- **Bridge injection latency**: Cloud browsers add 50-200ms round-trip overhead per `page.evaluate()` call. Increase `controlDiscoveryTimeout` to 45 seconds or higher.
- **Auth state reuse**: Use Playwright's `storageState` to persist authentication and avoid re-login on every worker. This is especially important at scale because SAP IdP rate-limits login attempts.
### Running in CI
```yaml
- name: Run Praman tests on Azure Playwright
run: npx playwright test --config=playwright.service.config.ts
env:
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
PLAYWRIGHT_SERVICE_ACCESS_TOKEN: ${{ secrets.PLAYWRIGHT_SERVICE_ACCESS_TOKEN }}
SAP_CLOUD_BASE_URL: ${{ secrets.SAP_CLOUD_BASE_URL }}
PRAMAN_CONTROL_DISCOVERY_TIMEOUT: '45000'
```
---
## Cross-Browser Testing
How to configure Playwright projects for multi-browser SAP UI5 testing. Covers browser-specific
differences in UI5 rendering, project configuration, and CI matrix strategies.
## Playwright Browser Projects
Playwright supports Chromium, Firefox, and WebKit. SAP UI5 officially supports Chrome, Firefox,
Safari, and Edge (Chromium-based).
### Basic Multi-Browser Config
```typescript
// playwright.config.ts
export default defineConfig({
testDir: './tests',
timeout: 120_000,
projects: [
// Auth setup per browser
{
name: 'chromium-setup',
testMatch: /auth-setup\.ts/,
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox-setup',
testMatch: /auth-setup\.ts/,
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit-setup',
testMatch: /auth-setup\.ts/,
use: { ...devices['Desktop Safari'] },
},
// Tests per browser
{
name: 'chromium',
dependencies: ['chromium-setup'],
use: {
...devices['Desktop Chrome'],
storageState: '.auth/chromium-session.json',
},
},
{
name: 'firefox',
dependencies: ['firefox-setup'],
use: {
...devices['Desktop Firefox'],
storageState: '.auth/firefox-session.json',
},
},
{
name: 'webkit',
dependencies: ['webkit-setup'],
use: {
...devices['Desktop Safari'],
storageState: '.auth/webkit-session.json',
},
},
],
});
```
### Browser-Specific Auth Setup
Each browser needs its own storage state file because session cookies are browser-specific:
```typescript
// tests/auth-setup.ts
setup('SAP authentication', async ({ page, context, browserName }) => {
const authFile = join(process.cwd(), '.auth', `${browserName}-session.json`);
// ... perform login ...
await context.storageState({ path: authFile });
});
```
## SAP UI5 Cross-Browser Differences
UI5 controls render slightly differently across browsers. These are the most common differences
that affect test stability.
### Rendering Differences
| Control | Chrome | Firefox | Safari/WebKit |
| -------------------- | --------------------------- | --------------------------------- | ----------------------------------- |
| `sap.m.Table` | Standard rendering | Minor scrollbar width differences | Touch-optimized rendering |
| `sap.m.DatePicker` | Native date popup available | Custom UI5 popup only | Native date popup available |
| `sap.ui.table.Table` | Smooth scrolling | Row height may vary by 1-2px | Momentum scrolling |
| `sap.m.Dialog` | Standard animation | Animation timing differs | Backdrop rendering differs |
| `sap.m.Popover` | Standard positioning | Same | May position differently near edges |
### Event Handling Differences
```typescript
// Some interactions behave differently across browsers.
// Use Praman's ui5.click() and ui5.fill() — they normalize behavior.
// Avoid browser-specific workarounds like:
// await page.mouse.click(x, y); // Coordinates vary by browser
// await page.keyboard.press('Tab'); // Focus order may differ
// Instead, use semantic selectors:
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
```
### Scroll Behavior
```typescript
// Scrolling can differ across browsers. Use UI5 control methods:
await ui5.scrollToControl({
controlType: 'sap.m.Input',
id: /lastFieldOnPage/,
});
// For tables with virtual scrolling:
const table = await ui5.control({
controlType: 'sap.ui.table.Table',
id: /mainTable/,
});
await table.scrollToRow(50);
```
## Selective Browser Testing
### Tag Tests by Browser Compatibility
```typescript
// Run specific tests only on certain browsers
test('drag and drop in table (Chromium only)', async ({ ui5, browserName }) => {
test.skip(browserName !== 'chromium', 'Drag and drop is Chromium-only');
// ... drag and drop test ...
});
test('touch gestures (WebKit only)', async ({ ui5, browserName }) => {
test.skip(browserName !== 'webkit', 'Touch-specific test');
// ... touch gesture test ...
});
```
### Browser-Specific Assertions
```typescript
test('table rendering across browsers', async ({ ui5, ui5Matchers, browserName }) => {
const table = await ui5.control({ controlType: 'sap.m.Table' });
// Common assertion — works on all browsers
await ui5Matchers.toHaveRowCount(table, 10);
// Browser-specific screenshot comparison
const locator = await table.getLocator();
await expect(locator).toHaveScreenshot(`table-${browserName}.png`, {
maxDiffPixelRatio: browserName === 'webkit' ? 0.02 : 0.01,
});
});
```
## Mobile Browser Testing
SAP Fiori apps are responsive. Test mobile viewports:
```typescript
// playwright.config.ts
export default defineConfig({
projects: [
// Desktop
{
name: 'desktop-chrome',
use: { ...devices['Desktop Chrome'] },
},
// Tablet
{
name: 'ipad',
use: { ...devices['iPad Pro 11'] },
},
// Mobile
{
name: 'iphone',
use: { ...devices['iPhone 14'] },
},
// Android
{
name: 'android',
use: { ...devices['Pixel 7'] },
},
],
});
```
### Responsive Layout Tests
```typescript
test('list report adapts to mobile viewport', async ({ ui5, page, isMobile }) => {
await test.step('verify responsive table mode', async () => {
if (isMobile) {
// On mobile, SmartTable switches to responsive mode
const mTable = await ui5.controlOrNull({
controlType: 'sap.m.Table',
});
expect(mTable).not.toBeNull();
} else {
// On desktop, SmartTable may use grid table
const gridTable = await ui5.controlOrNull({
controlType: 'sap.ui.table.Table',
});
// Grid table OR responsive table depending on config
expect(gridTable).not.toBeNull();
}
});
});
```
## CI Matrix Strategy
### GitHub Actions Multi-Browser Matrix
```yaml
# .github/workflows/cross-browser.yml
name: Cross-Browser Tests
on:
push:
branches: [main]
pull_request:
jobs:
test:
strategy:
fail-fast: false
matrix:
browser: [chromium, firefox, webkit]
os: [ubuntu-latest, windows-latest, macos-latest]
exclude:
# WebKit on Windows is not officially supported by Playwright
- browser: webkit
os: windows-latest
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install ${{ matrix.browser }}
- run: npx playwright test --project=${{ matrix.browser }}
env:
SAP_BASE_URL: ${{ secrets.SAP_QA_URL }}
SAP_USERNAME: ${{ secrets.SAP_QA_USER }}
SAP_PASSWORD: ${{ secrets.SAP_QA_PASS }}
- uses: actions/upload-artifact@v4
if: failure()
with:
name: report-${{ matrix.browser }}-${{ matrix.os }}
path: playwright-report/
```
### Tiered Browser Strategy
Not all tests need to run on all browsers. Use a tiered approach:
```typescript
// playwright.config.ts
export default defineConfig({
projects: [
// Tier 1: Full suite on Chrome (primary browser)
{
name: 'chromium-full',
testDir: './tests',
use: { ...devices['Desktop Chrome'] },
},
// Tier 2: Smoke tests on Firefox
{
name: 'firefox-smoke',
testDir: './tests/smoke',
use: { ...devices['Desktop Firefox'] },
},
// Tier 3: Critical path on WebKit
{
name: 'webkit-critical',
testDir: './tests/critical',
use: { ...devices['Desktop Safari'] },
},
],
});
```
## Best Practices
| Practice | Why |
| --------------------------------------- | ------------------------------------------------------------------- |
| Use Praman fixtures, not raw Playwright | Praman normalizes cross-browser differences in UI5 interactions |
| Separate auth state per browser | Session cookies are browser-specific |
| Use `maxDiffPixelRatio` for screenshots | Rendering differs slightly per browser |
| Run Chrome as primary, others as smoke | Chrome covers most users; other browsers catch critical regressions |
| Skip known browser limitations | Use `test.skip()` with clear reason, not silent failures |
| Use `fail-fast: false` in CI matrix | One browser failure should not block others from completing |
---
## Visual Regression Testing
How to use Playwright's `toHaveScreenshot()` with SAP UI5 apps for visual regression testing.
Covers theme variations, locale-dependent rendering, masking dynamic content, and CI
integration.
## Basic Visual Regression
Playwright's built-in `toHaveScreenshot()` captures and compares page or element screenshots
across test runs.
```typescript
test.describe('Visual Regression - Purchase Order List', () => {
test('list report matches baseline', async ({ ui5Navigation, page }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
await expect(page).toHaveScreenshot('po-list-report.png', {
maxDiffPixelRatio: 0.01,
});
});
});
```
### Element-Level Screenshots
Capture just a specific control instead of the full page:
```typescript
test('smart table matches baseline', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
const table = await ui5.control({
controlType: 'sap.ui.comp.smarttable.SmartTable',
});
const locator = await table.getLocator();
await expect(locator).toHaveScreenshot('po-smart-table.png', {
maxDiffPixelRatio: 0.01,
});
});
```
## Masking Dynamic Content
SAP apps contain dynamic data (timestamps, document numbers, user names) that changes between
test runs. Mask these areas to prevent false positives.
### CSS-Based Masking
```typescript
test('object page with masked dynamic content', async ({ ui5Navigation, page }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-display?PurchaseOrder=4500000001');
await expect(page).toHaveScreenshot('po-object-page.png', {
mask: [
page.locator('[id*="createdAt"]'), // Creation timestamp
page.locator('[id*="changedAt"]'), // Last changed timestamp
page.locator('[id*="createdBy"]'), // User name
page.locator('.sapMMessageStrip'), // Dynamic messages
],
maxDiffPixelRatio: 0.01,
});
});
```
### Masking with Praman Selectors
```typescript
test('form with masked values', async ({ ui5, ui5Navigation, page }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-display?PurchaseOrder=4500000001');
// Get locators for dynamic fields
const dateField = await ui5.control({
controlType: 'sap.m.DatePicker',
id: /creationDate/,
});
const userField = await ui5.control({
controlType: 'sap.m.Text',
id: /createdByText/,
});
await expect(page).toHaveScreenshot('po-form.png', {
mask: [await dateField.getLocator(), await userField.getLocator()],
});
});
```
## Theme Variations
SAP UI5 supports multiple themes. Test visual consistency across them.
### Multi-Theme Project Config
```typescript
// playwright.config.ts
const themes = ['sap_horizon', 'sap_horizon_dark', 'sap_horizon_hcb', 'sap_horizon_hcw'];
export default defineConfig({
projects: themes.map((theme) => ({
name: `visual-${theme}`,
testDir: './tests/visual',
use: {
...devices['Desktop Chrome'],
// Pass theme as a custom parameter
launchOptions: {
args: [`--theme=${theme}`],
},
},
metadata: { theme },
})),
});
```
### Applying Themes in Tests
```typescript
test.beforeEach(async ({ page }) => {
// Apply the UI5 theme via URL parameter
const theme = test.info().project.metadata.theme ?? 'sap_horizon';
const baseUrl = process.env.SAP_BASE_URL ?? '';
await page.goto(`${baseUrl}?sap-ui-theme=${theme}`);
});
test('list report in configured theme', async ({ ui5Navigation, page }) => {
const theme = test.info().project.metadata.theme ?? 'sap_horizon';
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
await expect(page).toHaveScreenshot(`po-list-${theme}.png`, {
maxDiffPixelRatio: 0.01,
});
});
```
### High Contrast Theme Testing
High contrast themes (`sap_horizon_hcb`, `sap_horizon_hcw`) are required for accessibility
compliance:
```typescript
test('high contrast black theme renders correctly', async ({ ui5Navigation, page }) => {
// Apply high contrast theme
await page.goto(`${process.env.SAP_BASE_URL}?sap-ui-theme=sap_horizon_hcb`);
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
await expect(page).toHaveScreenshot('po-list-hcb.png', {
maxDiffPixelRatio: 0.02, // Slightly higher tolerance for HC themes
});
});
```
## Locale-Dependent Rendering
SAP apps render differently based on locale settings: date formats, number formats,
right-to-left (RTL) text direction, and translated labels.
### Multi-Locale Testing
```typescript
const locales = [
{ code: 'en', name: 'English', rtl: false },
{ code: 'de', name: 'German', rtl: false },
{ code: 'ar', name: 'Arabic', rtl: true },
{ code: 'ja', name: 'Japanese', rtl: false },
];
for (const locale of locales) {
test(`list report renders correctly in ${locale.name}`, async ({ ui5Navigation, page }) => {
await page.goto(`${process.env.SAP_BASE_URL}?sap-language=${locale.code}`);
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
await expect(page).toHaveScreenshot(`po-list-${locale.code}.png`, {
maxDiffPixelRatio: 0.02,
});
});
}
```
### RTL Layout Validation
```typescript
test('RTL layout renders correctly for Arabic', async ({ ui5Navigation, page }) => {
await page.goto(`${process.env.SAP_BASE_URL}?sap-language=ar`);
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
// Verify RTL direction is applied
const dir = await page.getAttribute('html', 'dir');
expect(dir).toBe('rtl');
await expect(page).toHaveScreenshot('po-list-rtl.png', {
maxDiffPixelRatio: 0.02,
});
});
```
## Updating Baselines
When intentional UI changes are made, update the screenshot baselines:
```bash
# Update all visual snapshots
npx playwright test tests/visual/ --update-snapshots
# Update snapshots for a specific theme
npx playwright test tests/visual/ --project=visual-sap_horizon --update-snapshots
# Update a specific test's snapshot
npx playwright test tests/visual/ -g "list report" --update-snapshots
```
## CI Integration
### Storing Snapshots in Git
Screenshot baselines should be committed to version control:
```
tests/
visual/
po-list-report.spec.ts
po-list-report.spec.ts-snapshots/
po-list-report-chromium-linux.png
po-list-report-chromium-darwin.png
po-list-report-chromium-win32.png
```
### Platform-Specific Snapshots
Font rendering differs across operating systems. Playwright automatically generates
platform-specific snapshot names:
```typescript
// Playwright appends -{browserName}-{platform}.png automatically
await expect(page).toHaveScreenshot('my-page.png');
// Generates: my-page-chromium-linux.png, my-page-chromium-darwin.png, etc.
```
### GitHub Actions Workflow
```yaml
# .github/workflows/visual-regression.yml
name: Visual Regression
on:
pull_request:
jobs:
visual:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install chromium
- run: npx playwright test tests/visual/ --project=visual-sap_horizon
env:
SAP_BASE_URL: ${{ secrets.SAP_QA_URL }}
SAP_USERNAME: ${{ secrets.SAP_QA_USER }}
SAP_PASSWORD: ${{ secrets.SAP_QA_PASS }}
- uses: actions/upload-artifact@v4
if: failure()
with:
name: visual-regression-report
path: |
playwright-report/
test-results/
```
## Configuration Options
```typescript
// Global defaults in playwright.config.ts
export default defineConfig({
expect: {
toHaveScreenshot: {
maxDiffPixelRatio: 0.01, // 1% pixel difference allowed
threshold: 0.2, // Per-pixel color threshold (0-1)
animations: 'disabled', // Disable CSS animations for stability
},
},
});
```
| Option | Default | Recommended for SAP |
| ------------------- | --------- | ---------------------------------------------- |
| `maxDiffPixelRatio` | 0 | 0.01 (1%) — handles anti-aliasing differences |
| `threshold` | 0.2 | 0.2 — default works well for UI5 |
| `animations` | `'allow'` | `'disabled'` — prevents animation-caused diffs |
| `scale` | `'css'` | `'css'` — consistent across DPI settings |
## Best Practices
| Practice | Why |
| -------------------------------------------- | --------------------------------------------------------- |
| Mask dynamic content (dates, IDs, usernames) | Prevents false positives from changing data |
| Use element screenshots for components | Full-page screenshots are brittle and slow to review |
| Set `animations: 'disabled'` | CSS animations cause non-deterministic diffs |
| Test high contrast themes | Required for accessibility compliance (WCAG) |
| Keep tolerance low (1%) | Catches real regressions while tolerating rendering noise |
| Store baselines in Git | Track visual changes alongside code changes |
| Update baselines intentionally | Run `--update-snapshots` only after verifying changes |
---
## Accessibility Testing
How to use `@axe-core/playwright` alongside Praman to validate accessibility (a11y) in SAP
UI5 and Fiori applications. Covers ARIA roles in UI5 controls, common violations, and
integration patterns.
## Setup
Install the axe-core Playwright integration:
```bash
npm install --save-dev @axe-core/playwright
```
## Basic Accessibility Scan
```typescript
test.describe('Accessibility - Purchase Order List', () => {
test('list report page has no critical a11y violations', async ({ ui5Navigation, page }) => {
await test.step('navigate to list report', async () => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
});
await test.step('run axe accessibility scan', async () => {
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.analyze();
// Attach full results to the test report for review
await test.info().attach('a11y-results', {
body: JSON.stringify(results, null, 2),
contentType: 'application/json',
});
expect(results.violations).toEqual([]);
});
});
});
```
## Targeted Scans
### Scan a Specific Region
```typescript
await test.step('scan only the table region', async () => {
const results = await new AxeBuilder({ page })
.include('.sapMTable') // CSS selector for the UI5 responsive table
.withTags(['wcag2a', 'wcag2aa'])
.analyze();
expect(results.violations).toEqual([]);
});
```
### Exclude Known False Positives
SAP UI5 renders some elements that axe may flag incorrectly (e.g., ARIA attributes on custom
controls). Use `disableRules` to suppress known false positives:
```typescript
const results = await new AxeBuilder({ page })
.disableRules([
'color-contrast', // UI5 theme handles contrast; may flag themed controls
'landmark-one-main', // FLP shell provides landmarks outside app scope
])
.analyze();
```
## ARIA Roles in SAP UI5 Controls
UI5 controls emit standard ARIA roles. Understanding these helps you write targeted scans
and verify accessibility attributes.
| UI5 Control | ARIA Role | Notes |
| -------------------- | ----------------- | -------------------------------------------------------- |
| `sap.m.Button` | `button` | Includes `aria-pressed` for toggle buttons |
| `sap.m.Input` | `textbox` | `aria-required`, `aria-invalid` from control state |
| `sap.m.Select` | `combobox` | `aria-expanded` toggled on open/close |
| `sap.m.ComboBox` | `combobox` | `aria-autocomplete="list"` |
| `sap.m.Table` | `grid` or `table` | Depends on responsive table mode |
| `sap.m.List` | `listbox` | Items have `role="option"` |
| `sap.m.Dialog` | `dialog` | `aria-modal="true"`, `aria-labelledby` |
| `sap.m.MessageStrip` | `alert` | Auto-announced by screen readers |
| `sap.m.Panel` | `region` | `aria-labelledby` from header |
| `sap.m.IconTabBar` | `tablist` | Filters have `role="tab"`, content has `role="tabpanel"` |
| `sap.m.SearchField` | `searchbox` | `aria-label` from placeholder |
| `sap.m.CheckBox` | `checkbox` | `aria-checked` state |
| `sap.m.RadioButton` | `radio` | `aria-checked`, grouped via `aria-labelledby` |
## Testing ARIA Attributes with Praman
```typescript
test('verify input field accessibility attributes', async ({ ui5, page }) => {
await test.step('check required field ARIA', async () => {
const companyCode = await ui5.control({
controlType: 'sap.m.Input',
id: /companyCodeInput/,
});
// Get the DOM element to check ARIA attributes
const locator = await companyCode.getLocator();
await expect(locator).toHaveAttribute('aria-required', 'true');
await expect(locator).toHaveAttribute('role', 'textbox');
});
await test.step('check error state ARIA', async () => {
// Trigger validation by submitting empty required field
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Save' } });
const errorInput = await ui5.control({
controlType: 'sap.m.Input',
properties: { valueState: 'Error' },
});
const locator = await errorInput.getLocator();
await expect(locator).toHaveAttribute('aria-invalid', 'true');
});
});
```
## Keyboard Navigation Testing
SAP UI5 supports keyboard navigation. Test common patterns:
```typescript
test('keyboard navigation through form fields', async ({ ui5, page }) => {
await test.step('tab through form fields', async () => {
const firstInput = await ui5.control({
controlType: 'sap.m.Input',
id: /supplierInput/,
});
const locator = await firstInput.getLocator();
// Focus and tab
await locator.focus();
await page.keyboard.press('Tab');
// Verify focus moved to next field
const activeId = await page.evaluate(() => document.activeElement?.id);
expect(activeId).toContain('purchOrgInput');
});
await test.step('escape closes dialog', async () => {
// Open a value help dialog
await ui5.click({
controlType: 'sap.ui.core.Icon',
properties: { src: 'sap-icon://value-help' },
});
// Verify dialog is open
const dialog = await ui5.control({
controlType: 'sap.m.Dialog',
searchOpenDialogs: true,
});
expect(dialog).toBeTruthy();
// Press Escape to close
await page.keyboard.press('Escape');
});
await test.step('enter activates buttons', async () => {
const saveButton = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
const locator = await saveButton.getLocator();
await locator.focus();
await page.keyboard.press('Enter');
});
});
```
## Continuous Accessibility Monitoring
### Per-Page Scan in CI
Run axe on every page transition during E2E tests:
```typescript
// tests/helpers/a11y-helper.ts
export async function assertAccessible(page: Page, context?: string): Promise {
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa'])
.disableRules(['color-contrast']) // Managed by UI5 theme
.analyze();
if (results.violations.length > 0) {
const summary = results.violations
.map((v) => `${v.id}: ${v.description} (${v.nodes.length} instances)`)
.join('\n');
throw new Error(`Accessibility violations${context ? ` on ${context}` : ''}:\n${summary}`);
}
}
```
Usage in tests:
```typescript
test('PO creation form is accessible', async ({ ui5Navigation, page }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
await assertAccessible(page, 'PO creation form');
});
```
### Reporting
Attach axe results to Playwright HTML report for post-test review:
```typescript
await test.info().attach('axe-violations', {
body: JSON.stringify(results.violations, null, 2),
contentType: 'application/json',
});
```
## Common UI5 Accessibility Issues
| Issue | Affected Controls | Fix |
| ----------------------------------------- | ------------------------------------ | ------------------------------------------- |
| Missing `aria-label` on icon-only buttons | `sap.m.Button` with `icon` only | Set `tooltip` property in app code |
| Duplicate IDs | Auto-generated fragment IDs | Use `viewId` scoping in selectors |
| Missing form labels | `sap.m.Input` without `sap.m.Label` | Ensure labels are associated via `labelFor` |
| Low contrast in custom themes | Custom CSS overrides | Test with default `sap_horizon` theme |
| Missing heading hierarchy | `sap.m.Title` without semantic level | Set `level` property (H1-H6) |
---
## Component Testing
How to test individual UI5 components in isolation using Playwright's `webServer` option with
`ui5 serve`. Covers setup, component bootstrapping, and isolated testing patterns.
## What Is Component Testing?
Component testing runs a single UI5 component (a view, fragment, or reusable control) outside
the full Fiori Launchpad. This approach provides:
- **Speed**: No FLP shell, no authentication, no full app bootstrap
- **Isolation**: Test one component without side effects from others
- **Reliability**: No backend dependency when combined with OData mocks
- **Parallelism**: Multiple workers can test different components simultaneously
## Prerequisites
- UI5 Tooling CLI: `npm install --save-dev @ui5/cli`
- A UI5 app with `ui5.yaml` configuration
- Playwright with Praman
## Project Setup
### UI5 App Structure
```
my-ui5-app/
webapp/
Component.js
manifest.json
view/
Main.view.xml
controller/
Main.controller.js
test/
component/
index.html # Component test harness
ui5.yaml
playwright.config.ts
```
### Component Test Harness
Create a minimal HTML page that bootstraps just the component under test:
```html
Component Test
```
### Playwright Config with ui5 serve
```typescript
// playwright.config.ts
export default defineConfig({
testDir: './tests/component',
timeout: 30_000,
webServer: {
command: 'npx ui5 serve --port 8080',
port: 8080,
reuseExistingServer: !process.env.CI,
timeout: 60_000,
},
use: {
baseURL: 'http://localhost:8080',
},
projects: [
{
name: 'component-tests',
use: { ...devices['Desktop Chrome'] },
},
],
});
```
## Writing Component Tests
### Basic Component Render Test
```typescript
test.describe('Main View Component', () => {
test.beforeEach(async ({ page }) => {
// Navigate to the component test harness
await page.goto('/test/component/index.html');
// Wait for UI5 to fully bootstrap
await page.waitForFunction(() => {
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
return window.sap?.ui?.getCore()?.isInitialized() === true;
});
});
test('main view renders correctly', async ({ ui5 }) => {
await test.step('page title is displayed', async () => {
const title = await ui5.control({
controlType: 'sap.m.Title',
properties: { text: 'Purchase Orders' },
});
expect(title).toBeTruthy();
});
await test.step('table is present', async () => {
const table = await ui5.control({
controlType: 'sap.m.Table',
});
expect(table).toBeTruthy();
});
await test.step('create button exists', async () => {
const btn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Create' },
});
expect(btn).toBeTruthy();
});
});
});
```
### Component with Mocked OData
Combine component testing with OData mocking for fully offline tests:
```typescript
test.describe('Purchase Order List Component (Offline)', () => {
test.beforeEach(async ({ page }) => {
// Mock OData before loading the component
await page.route('**/odata/**', async (route) => {
const url = route.request().url();
if (url.includes('$metadata')) {
await route.fulfill({
status: 200,
contentType: 'application/xml',
body: '',
});
} else if (url.includes('PurchaseOrders')) {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
d: {
results: [
{ PurchaseOrder: '4500000001', Supplier: '100001', Status: 'Approved' },
{ PurchaseOrder: '4500000002', Supplier: '100002', Status: 'Pending' },
],
},
}),
});
} else {
await route.continue();
}
});
await page.goto('/test/component/index.html');
});
test('displays purchase orders from mock data', async ({ ui5, ui5Matchers }) => {
const table = await ui5.control({ controlType: 'sap.m.Table' });
await ui5Matchers.toHaveRowCount(table, 2);
});
});
```
### Testing Individual Views
Test a single view without the full Component bootstrap:
```html
```
```typescript
test.describe('Detail View (Isolated)', () => {
test('detail view renders form fields', async ({ page, ui5 }) => {
await page.goto('/test/component/view-test.html');
const inputs = await ui5.controls({
controlType: 'sap.m.Input',
});
expect(inputs.length).toBeGreaterThan(0);
});
});
```
### Testing Fragments
```html
```
## Testing Controller Logic
Use component tests to validate controller behavior in a real browser:
```typescript
test('controller handles form validation', async ({ page, ui5 }) => {
await page.goto('/test/component/index.html');
await test.step('submit empty form triggers validation', async () => {
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
// Check for validation error state on required fields
const errorInputs = await ui5.controls({
controlType: 'sap.m.Input',
properties: { valueState: 'Error' },
});
expect(errorInputs.length).toBeGreaterThan(0);
});
await test.step('filling required fields clears errors', async () => {
await ui5.fill({ controlType: 'sap.m.Input', id: /nameInput/ }, 'Test Value');
const nameInput = await ui5.control({
controlType: 'sap.m.Input',
id: /nameInput/,
});
const valueState = await nameInput.getProperty('valueState');
expect(valueState).toBe('None');
});
});
```
## CI Integration
### Component Test Workflow
```yaml
# .github/workflows/component-tests.yml
name: Component Tests
on: [push, pull_request]
jobs:
component-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install chromium
- run: npx playwright test --project=component-tests
- uses: actions/upload-artifact@v4
if: failure()
with:
name: component-test-report
path: playwright-report/
```
## Component Testing vs E2E Testing
| Aspect | Component Test | E2E Test |
| ----------- | --------------------- | --------------------------- |
| Scope | Single view/component | Full app + backend |
| Speed | Fast (seconds) | Slow (minutes) |
| Auth | None needed | Setup project required |
| Data | Mocked (page.route) | Real SAP backend |
| CI | Every push | Nightly or pre-release |
| Parallelism | High (4+ workers) | Low (1 worker for stateful) |
| Failures | Logic/rendering bugs | Integration issues |
## Best Practices
- **One harness per component**: Keep test HTML files focused on a single component.
- **Mock at the OData level**: Let the UI5 model parse mock data for realistic behavior.
- **Reuse mock factories**: Share mock data creation across component and E2E tests.
- **Test interactions, not DOM**: Use Praman selectors and matchers, not CSS selectors.
- **Run in CI on every push**: Component tests are fast enough for PR gates.
---
## Performance Benchmarks
# Performance Benchmarks (PERF-BENCH)
Performance targets, measurement methodology, and regression detection for Praman's core
operations. Every bridge call, discovery operation, and proxy interaction has a baseline
budget. CI enforces these budgets to prevent performance regressions.
## Baseline Targets
| Operation | Target | Measured At |
| --------------------- | --------- | -------------------------------------- |
| Bridge injection | < 500 ms | `page.evaluate()` for UI5 bridge setup |
| Control discovery | < 200 ms | Single control lookup by ID |
| Method call (proxy) | < 100 ms | `control.getValue()` round-trip |
| Proxy creation | < 50 ms | `new Proxy()` wrapper instantiation |
| Stability convergence | < 2000 ms | `waitForUI5Stable()` full cycle |
These targets represent p95 values on a standard CI runner (GitHub Actions `ubuntu-latest`,
4 vCPU, 16 GB RAM). Local development machines may see faster times. SAP system latency
(network round-trip to the actual backend) is excluded -- these measure only the Praman
framework overhead.
### Why These Numbers
- **Bridge injection (500 ms)**: The bridge injects ~2 KB of JavaScript via `page.evaluate()`.
500 ms accounts for CDP round-trip, V8 compilation, and UI5 runtime handshake. First
injection is slower than subsequent calls because V8 must compile the script.
- **Control discovery (200 ms)**: Discovery uses a multi-strategy chain (cache hit, direct ID
lookup, RecordReplay, registry scan). Cache hits resolve in < 10 ms. The 200 ms budget
covers a cold registry scan -- the worst case.
- **Method call (100 ms)**: Each proxy method call serializes arguments, executes
`page.evaluate()`, and deserializes the result. 100 ms covers the CDP round-trip plus UI5
control method execution.
- **Proxy creation (50 ms)**: Creating a JavaScript `Proxy` object is a pure Node.js operation
with no browser round-trip. The 50 ms budget covers type resolution, method blacklist
validation, and handler setup.
- **Stability convergence (2000 ms)**: `waitForUI5Stable()` polls the UI5 runtime for pending
requests, timeouts, and animations. 2000 ms is the maximum wait before the three-tier
timeout ladder escalates. Most stable pages converge in < 500 ms.
## Unit Benchmarks with Vitest
Vitest includes a built-in `bench()` function for microbenchmarks. Use it for operations that
do not require a browser (proxy creation, serialization, config parsing).
:::info[Contributor Only]
The benchmarks below use internal path aliases (`#proxy/*`, `#bridge/*`) that are only available when developing Praman itself. They are not accessible to end users of the `playwright-praman` package.
:::
```typescript
// tests/benchmarks/proxy-creation.bench.ts
describe('proxy creation', () => {
bench(
'create proxy from discovery result',
() => {
createControlProxy({
controlId: '__xmlview0--saveBtn',
controlType: 'sap.m.Button',
properties: { text: 'Save', enabled: true },
domRef: '#__xmlview0--saveBtn',
});
},
{
time: 2000, // Run for 2 seconds
iterations: 100, // Minimum 100 iterations
warmupTime: 500, // 500ms warmup
},
);
bench(
'create proxy with method blacklist lookup',
() => {
createControlProxy({
controlId: '__xmlview0--mainTable',
controlType: 'sap.m.Table',
properties: { mode: 'SingleSelectLeft' },
domRef: '#__xmlview0--mainTable',
});
},
{
time: 2000,
iterations: 100,
warmupTime: 500,
},
);
});
```
Run benchmarks:
```bash
npx vitest bench tests/benchmarks/
```
Output:
```
✓ proxy creation
name hz min max mean p75 p99
create proxy from discovery result 45,230 0.018ms 0.052ms 0.022ms 0.023ms 0.045ms
create proxy with method blacklist 38,100 0.021ms 0.061ms 0.026ms 0.028ms 0.055ms
```
### Benchmark Patterns for JSON Operations
```typescript
// tests/benchmarks/json-operations.bench.ts
describe('JSON serialization', () => {
const complexSelector = {
controlType: 'sap.m.Input',
viewName: 'myApp.view.Detail',
properties: { placeholder: 'Enter vendor name' },
ancestor: { controlType: 'sap.m.VBox' },
};
bench('serialize complex selector', () => {
JSON.stringify(complexSelector);
});
bench('parse bridge result with nested objects', () => {
JSON.parse(
JSON.stringify({
controlId: '__xmlview0--input1',
controlType: 'sap.m.Input',
properties: {
value: 'Vendor 1000',
placeholder: 'Enter vendor name',
valueState: 'None',
enabled: true,
editable: true,
},
}),
);
});
});
```
## E2E Performance Measurement with Playwright Traces
For operations that require a browser (bridge injection, discovery, stability), use
Playwright's `performance.now()` markers inside `page.evaluate()`.
```typescript
// tests/benchmarks/bridge-injection.spec.ts
test.describe('performance: bridge injection', () => {
test('bridge injection completes within 500ms', async ({ page, ui5 }) => {
const timing = await test.step('measure bridge injection', async () => {
const start = performance.now();
// Force bridge re-injection by navigating to a new page
await page.goto(process.env['SAP_URL']!);
// The ui5 fixture auto-injects the bridge on first access
await ui5.waitForUI5Stable();
return performance.now() - start;
});
expect(timing).toBeLessThan(500);
});
test('control discovery completes within 200ms', async ({ ui5 }) => {
const timing = await test.step('measure discovery', async () => {
const start = performance.now();
await ui5.control({ id: /saveBtn/ });
return performance.now() - start;
});
expect(timing).toBeLessThan(200);
});
test('proxy method call completes within 100ms', async ({ ui5 }) => {
const control = await ui5.control({ id: /saveBtn/ });
const timing = await test.step('measure method call', async () => {
const start = performance.now();
await control.getText();
return performance.now() - start;
});
expect(timing).toBeLessThan(100);
});
});
```
### Measuring Stability Convergence
```typescript
test('stability convergence within 2000ms on idle page', async ({ ui5 }) => {
const timing = await test.step('measure stability wait', async () => {
const start = performance.now();
await ui5.waitForUI5Stable();
return performance.now() - start;
});
// On an idle page, stability should converge much faster than the budget
expect(timing).toBeLessThan(2000);
// Track the actual timing for trend analysis
test.info().annotations.push({
type: 'perfMetric',
description: `stability_convergence_ms:${Math.round(timing)}`,
});
});
```
## Regression Detection
### Threshold Rule: 2x Baseline Fails CI
Any metric exceeding 2x its baseline target causes a CI failure. This catches regressions
while allowing for normal variance.
| Operation | Baseline | Regression Threshold (2x) |
| --------------------- | -------- | ------------------------- |
| Bridge injection | 500 ms | 1000 ms |
| Control discovery | 200 ms | 400 ms |
| Method call | 100 ms | 200 ms |
| Proxy creation | 50 ms | 100 ms |
| Stability convergence | 2000 ms | 4000 ms |
### CI Configuration
```yaml
# .github/workflows/perf.yml
name: Performance Regression
on:
pull_request:
paths:
- 'src/bridge/**'
- 'src/proxy/**'
- 'src/core/stability/**'
jobs:
benchmarks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install dependencies
run: npm ci
- name: Run unit benchmarks
run: npx vitest bench tests/benchmarks/ --reporter=json --outputFile=bench-results.json
- name: Check regression thresholds
run: node scripts/check-bench-regression.js bench-results.json
```
### Regression Check Script
```typescript
// scripts/check-bench-regression.ts
interface BenchResult {
name: string;
mean: number; // milliseconds
}
const THRESHOLDS: Record = {
'bridge injection': 1000,
'control discovery': 400,
'method call': 200,
'proxy creation': 100,
'stability convergence': 4000,
};
const resultsFile = process.argv[2];
if (!resultsFile) {
console.error('Usage: node check-bench-regression.js ');
process.exit(1);
}
const results: BenchResult[] = JSON.parse(readFileSync(resultsFile, 'utf-8'));
let failed = false;
for (const result of results) {
const threshold = THRESHOLDS[result.name];
if (threshold && result.mean > threshold) {
console.error(
`REGRESSION: "${result.name}" mean=${result.mean.toFixed(1)}ms exceeds 2x threshold=${threshold}ms`,
);
failed = true;
}
}
if (failed) {
process.exit(1);
} else {
console.log('All benchmarks within 2x regression threshold.');
}
```
## Tracking Performance Over Time
Use Playwright's annotation system to emit performance metrics that can be extracted from
JUnit XML or JSON reporter output for trend dashboards.
```typescript
test.afterEach(async ({}, testInfo) => {
// Emit performance annotations for CI extraction
for (const annotation of testInfo.annotations) {
if (annotation.type === 'perfMetric' && annotation.description) {
const [name, value] = annotation.description.split(':');
// These appear in JUnit XML elements and JSON report annotations
console.log(`::notice title=perf::${name}=${value}`);
}
}
});
```
## Summary
| Aspect | Tool | When |
| --------------- | ------------------------ | ---------------------------------------------- |
| Microbenchmarks | `vitest bench` | Pure Node.js operations (proxy, serialization) |
| E2E timing | Playwright + `expect()` | Browser operations (injection, discovery) |
| Regression gate | CI script + 2x threshold | Every PR touching bridge/proxy/stability |
| Trend tracking | Annotations + dashboard | Ongoing, extracted from JUnit/JSON reports |
---
## Lifecycle Hook Extensibility
## Overview
Praman leverages Playwright's built-in extensibility mechanisms rather than a custom event emitter or plugin system.
This guide covers how to extend praman's test lifecycle using Playwright-native patterns.
## 1. Request Interception with `page.route()`
Intercept, modify, or mock HTTP requests at the network level.
```typescript
test('mock OData response', async ({ page }) => {
// Intercept OData entity set requests
await page.route('**/sap/opu/odata/sap/API_BUSINESS_PARTNER/**', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
d: { results: [{ BusinessPartner: 'BP001', BusinessPartnerName: 'Test Inc.' }] },
}),
});
});
// Test runs with mocked data
await page.goto('/app#BusinessPartner-manage');
});
```
Praman's built-in `requestInterceptor` fixture uses this pattern to block WalkMe and analytics requests automatically.
## 2. Fixture Setup/Teardown with `use()`
Playwright fixtures provide deterministic setup and teardown via the `use()` callback. Code before `use()` is setup; code after is teardown (guaranteed to run, even on failure).
```typescript
const test = base.extend({
// Custom fixture with setup and teardown
testOrder: async ({ page, ui5 }, use) => {
// SETUP: create test data
const orderId = await createTestOrder(page);
// Pass to test
await use(orderId);
// TEARDOWN: always runs, even if test fails
await deleteTestOrder(page, orderId);
},
});
test('verify order', async ({ testOrder }) => {
// testOrder contains the created order ID
});
```
## 3. Browser Event Listeners with `page.on()`
React to browser events like navigation, console messages, or dialog prompts.
```typescript
test('track navigation events', async ({ page }) => {
const navigations: string[] = [];
// Listen for main frame navigations
page.on('framenavigated', (frame) => {
if (frame === page.mainFrame()) {
navigations.push(frame.url());
}
});
// Listen for console errors from UI5
page.on('console', (msg) => {
if (msg.type() === 'error') {
console.warn(`UI5 console error: ${msg.text()}`);
}
});
await page.goto('/app#PurchaseOrder-manage');
});
```
Praman's `ui5Stability` fixture uses this pattern to call `waitForUI5Stable()` on main frame navigations.
## 4. Test Hooks with `beforeEach` / `afterEach`
Standard Playwright hooks for cross-cutting concerns.
```typescript
test.beforeEach(async ({ page }) => {
// Runs before every test — e.g., ensure clean state
await page.evaluate(() => {
sessionStorage.clear();
});
});
test.afterEach(async ({ page }, testInfo) => {
// Runs after every test — e.g., capture screenshot on failure
if (testInfo.status !== testInfo.expectedStatus) {
await page.screenshot({ path: `screenshots/${testInfo.title}.png` });
}
});
```
## 5. Step Instrumentation with `test.step()`
Group operations into named steps for Playwright trace viewer and HTML reporter.
```typescript
test('create purchase order', async ({ page, ui5 }) => {
await test.step('Navigate to PO creation', async () => {
await page.goto('/app#PurchaseOrder-manage');
});
await test.step('Fill header data', async () => {
await ui5.fill({ id: 'supplierInput' }, 'Supplier A');
await ui5.fill({ id: 'companyCodeInput' }, '1000');
});
await test.step('Verify creation', async () => {
await expect(page.locator('ui5=sap.m.MessageToast')).toBeVisible();
});
});
```
Praman's `@ui5Step` decorator uses `test.step({ box: true })` to automatically wrap handler methods in trace-visible steps with internal stack frames filtered.
## Combining Patterns
These patterns compose naturally. A typical SAP test fixture combines all four:
```typescript
const test = base.extend({
sapApp: async ({ page, ui5 }, use) => {
// SETUP: Mock slow service, navigate, wait
await page.route('**/slow-analytics/**', (route) => route.abort());
page.on('framenavigated', (frame) => {
if (frame === page.mainFrame()) {
// Log navigation for debugging
}
});
await page.goto('/app#MyApp-display');
await use({ page, ui5 });
// TEARDOWN: Clean up test data
},
});
```
## Why No Custom Event System?
Playwright's fixture model, `page.route()`, `page.on()`, and `test.step()` together cover every lifecycle hook scenario. A custom event emitter would:
- Duplicate Playwright's built-in capabilities
- Add maintenance burden and API surface
- Require documentation for a non-standard pattern
- Confuse users familiar with Playwright's patterns
Praman follows Playwright's philosophy: use the platform's native extensibility rather than building framework-specific abstractions.
---
## Multi-Tool Integration
Praman generates test results as standard Playwright output (HTML, JUnit XML, JSON). For teams
using external test management tools -- Tricentis qTest, TestRail, Xray for Jira -- this guide
covers adapter patterns for forwarding results.
## Architecture: Custom Playwright Reporters
Each integration follows the same pattern: a custom Playwright reporter that transforms test
results into the target tool's API format.
```
Playwright Test Run
│
▼
┌──────────────┐
│ Reporter │ onBegin / onTestEnd / onEnd
│ (adapter) │
└──────┬───────┘
│
▼
┌──────────────┐
│ Tool API │ qTest / TestRail / Xray
│ (HTTP) │
└──────────────┘
```
All adapters extend Playwright's `Reporter` interface and run passively alongside tests --
no test code changes required.
## Base Reporter Class
A shared base class handles common concerns: batching, error handling, and retry logic.
```typescript
// reporters/base-tool-reporter.ts
interface ToolReporterConfig {
/** API base URL for the test management tool */
apiUrl: string;
/** Authentication token or API key */
apiToken: string;
/** Project or container ID in the target tool */
projectId: string;
/** Batch size for API calls (default: 50) */
batchSize?: number;
/** Retry failed API calls (default: 3) */
maxRetries?: number;
}
interface PendingResult {
testTitle: string;
suiteName: string;
status: 'passed' | 'failed' | 'skipped';
duration: number;
errorMessage?: string;
annotations: Record;
}
abstract class BaseToolReporter implements Reporter {
protected readonly config: ToolReporterConfig;
protected readonly results: PendingResult[] = [];
constructor(config: ToolReporterConfig) {
this.config = config;
}
onTestEnd(test: TestCase, result: TestResult): void {
const annotations: Record = {};
for (const ann of test.annotations) {
annotations[ann.type] = ann.description ?? '';
}
this.results.push({
testTitle: test.title,
suiteName: test.parent.title,
status:
result.status === 'passed' ? 'passed' : result.status === 'failed' ? 'failed' : 'skipped',
duration: result.duration,
errorMessage: result.errors.map((e) => e.message).join('\n'),
annotations,
});
}
async onEnd(result: FullResult): Promise {
const batchSize = this.config.batchSize ?? 50;
for (let i = 0; i < this.results.length; i += batchSize) {
const batch = this.results.slice(i, i + batchSize);
await this.uploadBatch(batch);
}
}
/** Implement this method to upload a batch of results to the target tool */
protected abstract uploadBatch(batch: PendingResult[]): Promise;
/** HTTP POST with retry logic */
protected async postWithRetry(url: string, body: unknown): Promise {
const maxRetries = this.config.maxRetries ?? 3;
let lastError: Error | undefined;
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${this.config.apiToken}`,
},
body: JSON.stringify(body),
});
if (response.ok) return response;
if (response.status >= 400 && response.status < 500) {
// Client errors are not retryable
throw new Error(`API error ${response.status}: ${await response.text()}`);
}
} catch (error) {
lastError = error instanceof Error ? error : new Error(String(error));
}
// Exponential backoff with jitter
const delay = Math.min(1000 * 2 ** attempt + Math.random() * 500, 10_000);
await new Promise((resolve) => setTimeout(resolve, delay));
}
throw lastError ?? new Error('Upload failed after retries');
}
}
```
## Tricentis qTest Adapter
qTest uses a REST API for test execution results. The adapter maps Playwright tests to qTest
test runs.
```typescript
// reporters/qtest-reporter.ts
interface QTestConfig extends ToolReporterConfig {
/** qTest test cycle ID */
testCycleId: string;
}
class QTestReporter extends BaseToolReporter {
private readonly testCycleId: string;
constructor(config: QTestConfig) {
super(config);
this.testCycleId = config.testCycleId;
}
protected async uploadBatch(batch: PendingResult[]): Promise {
const testLogs = batch.map((result) => ({
name: result.testTitle,
status: this.mapStatus(result.status),
exe_start_date: new Date().toISOString(),
exe_end_date: new Date(Date.now() + result.duration).toISOString(),
note: result.errorMessage ?? '',
test_case_version_id: result.annotations['qtestCaseId'] ?? undefined,
// Map test steps from Playwright step names
test_step_logs: [],
}));
await this.postWithRetry(
`${this.config.apiUrl}/api/v3/projects/${this.config.projectId}/test-runs`,
{ test_logs: testLogs, parent_id: this.testCycleId, parent_type: 'test-cycle' },
);
}
private mapStatus(status: string): string {
switch (status) {
case 'passed':
return 'PASSED';
case 'failed':
return 'FAILED';
default:
return 'SKIPPED';
}
}
}
// Usage in playwright.config.ts:
// reporter: [[QTestReporter, {
// apiUrl: 'https://mycompany.qtestnet.com',
// apiToken: process.env.QTEST_TOKEN,
// projectId: '12345',
// testCycleId: '67890',
// }]]
```
Link tests to qTest test cases using annotations:
```typescript
test('create purchase order', async ({ ui5 }) => {
test.info().annotations.push({ type: 'qtestCaseId', description: 'TC-45678' });
// ... test steps
});
```
## TestRail Adapter
TestRail's API accepts results per test run. The adapter creates or updates a test run with
results.
```typescript
// reporters/testrail-reporter.ts
interface TestRailConfig extends ToolReporterConfig {
/** TestRail test run ID (or 'auto' to create new runs) */
runId: string | 'auto';
/** Suite ID for auto-created runs */
suiteId?: string;
}
class TestRailReporter extends BaseToolReporter {
private readonly runId: string;
private readonly suiteId?: string;
constructor(config: TestRailConfig) {
super(config);
this.runId = config.runId;
this.suiteId = config.suiteId;
}
protected async uploadBatch(batch: PendingResult[]): Promise {
let runId = this.runId;
// Auto-create a test run if configured
if (runId === 'auto') {
const response = await this.postWithRetry(
`${this.config.apiUrl}/index.php?/api/v2/add_run/${this.config.projectId}`,
{
suite_id: this.suiteId,
name: `Praman E2E - ${new Date().toISOString().slice(0, 10)}`,
include_all: false,
case_ids: batch
.map((r) => r.annotations['testrailCaseId'])
.filter(Boolean)
.map(Number),
},
);
const data = (await response.json()) as { id: number };
runId = String(data.id);
}
// Upload results
const results = batch
.filter((r) => r.annotations['testrailCaseId'])
.map((r) => ({
case_id: Number(r.annotations['testrailCaseId']),
status_id: r.status === 'passed' ? 1 : r.status === 'failed' ? 5 : 2,
elapsed: `${Math.round(r.duration / 1000)}s`,
comment: r.errorMessage ?? 'Automated test passed',
}));
await this.postWithRetry(
`${this.config.apiUrl}/index.php?/api/v2/add_results_for_cases/${runId}`,
{ results },
);
}
}
// Usage in playwright.config.ts:
// reporter: [[TestRailReporter, {
// apiUrl: 'https://mycompany.testrail.io',
// apiToken: process.env.TESTRAIL_TOKEN,
// projectId: '1',
// runId: 'auto',
// suiteId: '5',
// }]]
```
Link tests to TestRail cases:
```typescript
test('verify three-way match', async ({ ui5 }) => {
test.info().annotations.push({ type: 'testrailCaseId', description: '7891' });
// ... test steps
});
```
## Xray for Jira Adapter
Xray supports both Cloud and Server/Data Center deployments. The adapter uses the Xray REST
API to import execution results.
```typescript
// reporters/xray-reporter.ts
interface XrayConfig extends ToolReporterConfig {
/** Xray deployment type */
deployment: 'cloud' | 'server';
/** Jira project key */
jiraProjectKey: string;
/** Test plan issue key (e.g., 'PROJ-123') */
testPlanKey?: string;
}
class XrayReporter extends BaseToolReporter {
private readonly deployment: string;
private readonly jiraProjectKey: string;
private readonly testPlanKey?: string;
constructor(config: XrayConfig) {
super(config);
this.deployment = config.deployment;
this.jiraProjectKey = config.jiraProjectKey;
this.testPlanKey = config.testPlanKey;
}
protected async uploadBatch(batch: PendingResult[]): Promise {
const execution = {
testExecutionKey: undefined as string | undefined,
info: {
project: this.jiraProjectKey,
summary: `Praman E2E - ${new Date().toISOString().slice(0, 10)}`,
description: 'Automated test execution via Praman + Playwright',
testPlanKey: this.testPlanKey,
},
tests: batch
.filter((r) => r.annotations['xrayTestKey'])
.map((r) => ({
testKey: r.annotations['xrayTestKey'],
status: r.status === 'passed' ? 'PASSED' : r.status === 'failed' ? 'FAILED' : 'TODO',
comment: r.errorMessage ?? '',
executedBy: 'praman-automation',
start: new Date().toISOString(),
finish: new Date(Date.now() + r.duration).toISOString(),
})),
};
const apiBase =
this.deployment === 'cloud'
? 'https://xray.cloud.getxray.app/api/v2'
: `${this.config.apiUrl}/rest/raven/2.0/api`;
await this.postWithRetry(`${apiBase}/import/execution`, execution);
}
}
// Usage in playwright.config.ts:
// reporter: [[XrayReporter, {
// apiUrl: 'https://jira.mycompany.com',
// apiToken: process.env.XRAY_TOKEN,
// projectId: 'PROJ',
// deployment: 'cloud',
// jiraProjectKey: 'PROJ',
// testPlanKey: 'PROJ-456',
// }]]
```
Link tests to Xray test issues:
```typescript
test('validate goods receipt', async ({ ui5 }) => {
test.info().annotations.push({ type: 'xrayTestKey', description: 'PROJ-789' });
// ... test steps
});
```
## Using Multiple Reporters Simultaneously
Playwright supports multiple reporters in parallel. Combine Praman reporters with tool-specific
adapters:
```typescript
// playwright.config.ts
export default defineConfig({
reporter: [
['html'],
['junit', { outputFile: 'reports/junit-results.xml' }],
[ComplianceReporter, { outputDir: 'reports' }],
[ODataTraceReporter, { outputDir: 'reports' }],
// Add your tool-specific reporter
[
TestRailReporter,
{
apiUrl: process.env['TESTRAIL_URL']!,
apiToken: process.env['TESTRAIL_TOKEN']!,
projectId: '1',
runId: 'auto',
suiteId: '5',
},
],
],
});
```
## Annotation Convention Summary
| Annotation Type | Description | Used By |
| ---------------- | ------------------------------- | ---------------- |
| `qtestCaseId` | qTest test case ID | qTest adapter |
| `testrailCaseId` | TestRail case ID (numeric) | TestRail adapter |
| `xrayTestKey` | Xray/Jira issue key | Xray adapter |
| `calmTestCaseId` | Cloud ALM test case ID | Cloud ALM |
| `calmTestPlanId` | Cloud ALM test plan ID | Cloud ALM |
| `requirementId` | Requirement ID for traceability | All tools |
| `executionType` | `automated` / `hybrid` | Cloud ALM |
These annotations are additive -- a single test can carry annotations for multiple tools,
enabling parallel reporting to different systems from the same test run.
---
## Cloud ALM Integration
:::warning Correction notice
Earlier versions of this page incorrectly described a push model where test results are sent to Cloud ALM via HTTP POST. **That is not how Cloud ALM works.** The correct model is described below.
:::
## How the Integration Actually Works
SAP Cloud ALM Test Automation uses a **pull model**, not a push model:
| Direction | What happens |
| --------------------- | ---------------------------------------------------------------------------------- |
| Cloud ALM → Your tool | Cloud ALM calls your registered endpoint to trigger test runs and poll for results |
| Your tool → Cloud ALM | You can GET test cases from Cloud ALM using the Test Automation API |
You **cannot** push test results to Cloud ALM. Cloud ALM **pulls** them from your tool on its own schedule.
## Integration Model
To integrate a custom test automation tool with Cloud ALM, you must expose an API that Cloud ALM calls. Cloud ALM acts as the client; your tool acts as the server.
```text
┌─────────────────────────────────────────────────────────┐
│ SAP Cloud ALM │
│ │
│ 1. Sends test run trigger ──────────► Your tool API │
│ 2. Polls for results ──────────► Your tool API │
│ 3. Displays pulled results │
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ Your tool (Playwright + Praman) │
│ │
│ GET test cases ───────────────────► Cloud ALM API │
│ (optional — to know what to run) │
└─────────────────────────────────────────────────────────┘
```
## What You Can Do Today
### Fetch Test Cases from Cloud ALM
You can read test cases out of Cloud ALM using the Test Automation API. This lets you query which tests are scheduled for a given test plan or run configuration.
The API is available at SAP API Business Hub:
- [CALM_TEST_AUTOMATION API reference and tryout](https://api.sap.com/api/CALM_TEST_AUTOMATION/tryout)
Authentication uses OAuth 2.0. You can use the [SAP Cloud SDK for JavaScript](https://sap.github.io/cloud-sdk/docs/js/getting-started) to handle the credential and token management if you are building a Node.js integration layer.
### Generate JUnit XML from Playwright
Playwright's built-in JUnit reporter produces XML that Cloud ALM can consume once your tool exposes it via an endpoint Cloud ALM can poll:
```typescript
// playwright.config.ts
export default defineConfig({
reporter: [['html'], ['junit', { outputFile: 'reports/results.xml' }]],
});
```
This XML file is what Cloud ALM will pull from your tool's result endpoint — you do not POST it to Cloud ALM.
## Registering as a Test Automation Provider
To connect Cloud ALM to your Playwright + Praman setup, you register your tool as a test automation provider in Cloud ALM's configuration. Cloud ALM then uses HTTP destination configuration to call your tool's endpoints.
SAP documentation for this setup:
- [Integrating Test Automation Providers](https://help.sap.com/docs/cloud-alm/setup-administration/integrating-test-automation-providers?locale=en-US)
- [Other Test Automation Providers](https://help.sap.com/docs/cloud-alm/setup-administration/other-test-automation-providers?locale=en-US)
- [Test Automation Tool for S/4HANA Cloud](https://help.sap.com/docs/cloud-alm/setup-administration/test-automation-tool-for-s4hana-cloud?locale=en-US)
Your tool must expose at minimum:
- An endpoint to **receive a test run trigger** from Cloud ALM
- An endpoint for Cloud ALM to **poll test results**
The exact API contract your tool must implement is defined in the provider integration guide linked above.
## What Praman Provides
Praman does not include a Cloud ALM provider server — that is an application layer your team builds on top. What Praman contributes:
| Praman output | Use in Cloud ALM integration |
| --------------------------------- | -------------------------------------------------------- |
| JUnit XML (`reports/results.xml`) | Serve from your result endpoint so Cloud ALM can pull it |
| Playwright HTML report | Internal visibility; not consumed by Cloud ALM |
| `test.step()` names | Appear as step details in JUnit XML |
| Exit code 0/1 | Signal pass/fail to your trigger endpoint handler |
## API Reference
| Resource | Link |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| Cloud ALM Test Automation API | [help.sap.com — Test Automation API](https://help.sap.com/docs/cloud-alm/apis/test-automation-api?locale=en-US) |
| API tryout (SAP API Hub) | [api.sap.com — CALM_TEST_AUTOMATION](https://api.sap.com/api/CALM_TEST_AUTOMATION/tryout) |
| Provider setup guide | [Integrating Test Automation Providers](https://help.sap.com/docs/cloud-alm/setup-administration/integrating-test-automation-providers?locale=en-US) |
| Other providers guide | [Other Test Automation Providers](https://help.sap.com/docs/cloud-alm/setup-administration/other-test-automation-providers?locale=en-US) |
| S/4HANA Cloud specific | [Test Automation Tool for S/4HANA Cloud](https://help.sap.com/docs/cloud-alm/setup-administration/test-automation-tool-for-s4hana-cloud?locale=en-US) |
| SAP Cloud SDK for JS | [sap.github.io/cloud-sdk](https://sap.github.io/cloud-sdk/docs/js/getting-started) |
---
## SAP Activate Alignment
Maps Praman test patterns to SAP Activate methodology phases. This guide helps project teams
align their automated testing strategy with SAP's implementation framework.
## SAP Activate Overview
SAP Activate is SAP's methodology for implementing SAP S/4HANA and cloud solutions. It defines
six phases, each with testing activities where Praman provides automation support.
| Phase | Name | Testing Focus |
| ----- | -------- | ----------------------------------------- |
| 1 | Discover | Proof of concept, feasibility tests |
| 2 | Prepare | Test strategy, environment setup |
| 3 | Explore | Fit-to-standard, configuration validation |
| 4 | Realize | Unit, integration, string tests |
| 5 | Deploy | UAT, performance, regression |
| 6 | Run | Production monitoring, regression packs |
## Phase 1: Discover
**Goal**: Validate that Praman works with your SAP landscape.
### Praman Activities
```typescript
// Proof of concept: verify basic connectivity and control discovery
test('SAP system connectivity check', async ({ ui5, ui5Navigation, page }) => {
await test.step('access Fiori Launchpad', async () => {
await ui5Navigation.navigateToIntent('#Shell-home');
await expect(page).toHaveURL(/fiorilaunchpad/);
});
await test.step('verify UI5 runtime is available', async () => {
const isUI5 = await ui5.isUI5Available();
expect(isUI5).toBe(true);
});
await test.step('discover controls on launchpad', async () => {
const tiles = await ui5.controls({
controlType: 'sap.m.GenericTile',
});
expect(tiles.length).toBeGreaterThan(0);
});
});
```
### Deliverables
- Connectivity test script (above)
- Browser compatibility matrix (see [Cross-Browser Testing](./cross-browser.md))
- Tool evaluation report comparing Praman to Tosca, wdi5, UIVeri5
## Phase 2: Prepare
**Goal**: Establish test infrastructure and strategy.
### Praman Activities
| Activity | Praman Artifact |
| ---------------------- | ----------------------------------------------------------------- |
| Test strategy document | `docs/test-strategy.md` |
| Environment setup | `playwright.config.ts`, `praman.config.ts` |
| Auth configuration | `tests/auth-setup.ts` (see [Authentication](./authentication.md)) |
| CI/CD pipeline | `.github/workflows/e2e-tests.yml` |
| Test data strategy | OData mock setup (see [OData Mocking](./odata-mocking.md)) |
### Project Structure
```
tests/
auth-setup.ts # Authentication setup project
smoke/ # Phase 3: Fit-to-standard
fit-to-standard.spec.ts
integration/ # Phase 4: String tests
p2p-flow.spec.ts
o2c-flow.spec.ts
regression/ # Phase 5+: Regression pack
purchase-order.spec.ts
sales-order.spec.ts
performance/ # Phase 5: Performance baselines
page-load.spec.ts
playwright.config.ts
praman.config.ts
```
## Phase 3: Explore (Fit-to-Standard)
**Goal**: Validate SAP standard configuration against business requirements.
### Fit-to-Standard Tests
These tests verify that standard SAP processes work with your specific configuration
(company codes, org units, master data).
```typescript
test.describe('Fit-to-Standard: Procurement', () => {
test('standard PO creation workflow matches requirements', async ({
ui5,
ui5Navigation,
feObjectPage,
ui5Matchers,
}) => {
await test.step('verify PO creation app is accessible', async () => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
});
await test.step('verify required fields match spec', async () => {
// Check that configured mandatory fields are enforced
const supplierField = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/Supplier' },
});
const isRequired = await supplierField.getProperty('mandatory');
expect(isRequired).toBe(true);
});
await test.step('verify org structure defaults', async () => {
// Company code should default from user parameters
const companyCode = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/CompanyCode' },
});
const value = await companyCode.getProperty('value');
expect(value).toBe('1000'); // Expected default from user profile
});
await test.step('verify approval workflow triggers', async () => {
// Create PO above threshold to trigger approval
await feObjectPage.fillField('Supplier', '100001');
await feObjectPage.fillField('NetPrice', '50000');
await feObjectPage.clickSave();
await ui5Matchers.toHaveMessageStrip('Success', /created/);
// Verify status indicates approval required
});
});
});
```
### Configuration Validation Matrix
```typescript
const orgUnits = [
{ companyCode: '1000', purchOrg: '1000', plant: '1000', name: 'US Operations' },
{ companyCode: '2000', purchOrg: '2000', plant: '2000', name: 'EU Operations' },
{ companyCode: '3000', purchOrg: '3000', plant: '3000', name: 'APAC Operations' },
];
for (const org of orgUnits) {
test(`fit-to-standard: PO creation for ${org.name}`, async ({
ui5Navigation,
feObjectPage,
ui5Matchers,
}) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
await feObjectPage.fillField('CompanyCode', org.companyCode);
await feObjectPage.fillField('PurchasingOrganization', org.purchOrg);
await feObjectPage.fillField('Plant', org.plant);
await feObjectPage.fillField('Supplier', '100001');
await feObjectPage.clickSave();
await ui5Matchers.toHaveMessageStrip('Success', /created/);
});
}
```
## Phase 4: Realize (Integration and String Tests)
**Goal**: Validate end-to-end business processes and integrations.
### String Tests
String tests connect multiple processes in sequence, validating the data flow between them.
```typescript
test.describe('String Test: Procure-to-Pay', () => {
test('P2P: Purchase Requisition to Payment', async ({
ui5Navigation,
feObjectPage,
feListReport,
ui5Matchers,
}) => {
await test.step('1. Create Purchase Requisition', async () => {
await ui5Navigation.navigateToIntent('#PurchaseRequisition-create');
// Fill and save PR
});
await test.step('2. Convert PR to Purchase Order', async () => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
// Reference the PR
});
await test.step('3. Post Goods Receipt', async () => {
await ui5Navigation.navigateToIntent('#GoodsReceipt-create');
// Post GR against PO
});
await test.step('4. Enter Invoice', async () => {
await ui5Navigation.navigateToIntent('#SupplierInvoice-create');
// Enter and post invoice
});
await test.step('5. Verify Document Flow', async () => {
// Navigate back to PO and check all documents are linked
});
});
});
```
See [Business Process Examples](./business-process-examples.md) for complete runnable examples.
### Test Execution Mapping
| SAP Activate Activity | Praman Test Type | Config |
| --------------------- | -------------------------------- | ------------------------ |
| Unit Test (UT) | Component test with mocked OData | `workers: 4` |
| Integration Test (IT) | Cross-app flow with real backend | `workers: 1` |
| String Test (ST) | Multi-step P2P, O2C flows | `workers: 1, retries: 1` |
| Security Test | Auth strategy validation | Setup project |
## Phase 5: Deploy (UAT and Regression)
**Goal**: User acceptance, performance baselines, regression packs.
### Regression Pack Structure
```typescript
// playwright.config.ts — regression project
{
name: 'regression',
testDir: './tests/regression',
dependencies: ['setup'],
use: {
storageState: '.auth/sap-session.json',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
retries: 2,
}
```
### Performance Baselines
```typescript
test('page load performance baseline', async ({ ui5Navigation, page }) => {
const startTime = Date.now();
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
const loadTime = Date.now() - startTime;
// Attach timing to report
await test.info().attach('performance', {
body: JSON.stringify({ loadTime, threshold: 5000 }),
contentType: 'application/json',
});
expect(loadTime).toBeLessThan(5000); // 5-second threshold
});
```
## Phase 6: Run (Production Monitoring)
**Goal**: Continuous regression testing against production-like environments.
### Smoke Test Suite
Run a minimal set of critical path tests on a schedule:
```yaml
# .github/workflows/smoke-tests.yml
name: SAP Smoke Tests
on:
schedule:
- cron: '0 6 * * *' # Daily at 6 AM
workflow_dispatch:
jobs:
smoke:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install chromium
- run: npx playwright test tests/smoke/ --project=regression
env:
SAP_BASE_URL: ${{ secrets.SAP_QA_URL }}
SAP_USERNAME: ${{ secrets.SAP_QA_USER }}
SAP_PASSWORD: ${{ secrets.SAP_QA_PASS }}
```
## Phase-to-Praman Mapping Summary
| Phase | Praman Feature | Key Config |
| -------- | ----------------------------------------- | ------------------------------- |
| Discover | `ui5.isUI5Available()`, control discovery | Basic `playwright.config.ts` |
| Prepare | Auth setup, config, project structure | `praman.config.ts`, CI pipeline |
| Explore | Fit-to-standard tests, org validation | Data-driven test loops |
| Realize | String tests, integration flows | `workers: 1`, `test.step()` |
| Deploy | Regression packs, performance baselines | `retries: 2`, trace on retry |
| Run | Scheduled smoke tests, monitoring | Cron-triggered CI, alerts |
---
## SAP Transaction Mapping
Maps 50+ SAP transaction codes to their Fiori equivalents and the corresponding Praman
fixtures, intents, and semantic objects used for test navigation.
## How to Use This Reference
In Praman, navigation uses **semantic objects and actions** (the Fiori Launchpad intent model)
rather than transaction codes. This table helps you translate your existing SAP GUI knowledge
into Fiori-native test navigation.
```typescript
// Navigate by semantic object + action
await ui5Navigation.navigateToApp('MM-PUR-PO', {
semanticObject: 'PurchaseOrder',
action: 'create',
});
// Navigate by intent hash
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
```
## Materials Management (MM)
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | ---------------------------- | ------------ | --------------------- | --------- | ------------------------------ |
| ME21N | Create Purchase Order | F0842 | `PurchaseOrder` | `create` | `#PurchaseOrder-create` |
| ME22N | Change Purchase Order | F0842 | `PurchaseOrder` | `change` | `#PurchaseOrder-change` |
| ME23N | Display Purchase Order | F0842 | `PurchaseOrder` | `display` | `#PurchaseOrder-display` |
| ME51N | Create Purchase Requisition | F1130 | `PurchaseRequisition` | `create` | `#PurchaseRequisition-create` |
| ME52N | Change Purchase Requisition | F1130 | `PurchaseRequisition` | `change` | `#PurchaseRequisition-change` |
| ME53N | Display Purchase Requisition | F1130 | `PurchaseRequisition` | `display` | `#PurchaseRequisition-display` |
| ME2M | POs by Material | F0701 | `PurchaseOrder` | `manage` | `#PurchaseOrder-manage` |
| ME2N | POs by PO Number | F0701 | `PurchaseOrder` | `manage` | `#PurchaseOrder-manage` |
| MIGO | Goods Movement | F0843 | `GoodsReceipt` | `create` | `#GoodsReceipt-create` |
| MIRO | Enter Incoming Invoice | F0859 | `SupplierInvoice` | `create` | `#SupplierInvoice-create` |
| MM01 | Create Material | F1602 | `Material` | `create` | `#Material-create` |
| MM02 | Change Material | F1602 | `Material` | `change` | `#Material-change` |
| MM03 | Display Material | F1602 | `Material` | `display` | `#Material-display` |
| MB52 | Warehouse Stocks | F0842A | `WarehouseStock` | `display` | `#WarehouseStock-display` |
| MMBE | Stock Overview | F0842A | `MaterialStock` | `display` | `#MaterialStock-display` |
## Sales & Distribution (SD)
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | ------------------------- | ------------ | ------------------ | --------- | --------------------------- |
| VA01 | Create Sales Order | F0217 | `SalesOrder` | `create` | `#SalesOrder-create` |
| VA02 | Change Sales Order | F0217 | `SalesOrder` | `change` | `#SalesOrder-change` |
| VA03 | Display Sales Order | F0217 | `SalesOrder` | `display` | `#SalesOrder-display` |
| VA05 | List of Sales Orders | F1814 | `SalesOrder` | `manage` | `#SalesOrder-manage` |
| VL01N | Create Outbound Delivery | F0341 | `OutboundDelivery` | `create` | `#OutboundDelivery-create` |
| VL02N | Change Outbound Delivery | F0341 | `OutboundDelivery` | `change` | `#OutboundDelivery-change` |
| VL03N | Display Outbound Delivery | F0341 | `OutboundDelivery` | `display` | `#OutboundDelivery-display` |
| VF01 | Create Billing Document | F0638 | `BillingDocument` | `create` | `#BillingDocument-create` |
| VF02 | Change Billing Document | F0638 | `BillingDocument` | `change` | `#BillingDocument-change` |
| VF03 | Display Billing Document | F0638 | `BillingDocument` | `display` | `#BillingDocument-display` |
## Finance (FI)
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | ------------------------- | ------------ | ------------------- | --------- | ---------------------------- |
| FB50 | Post G/L Account Document | F0718 | `JournalEntry` | `create` | `#JournalEntry-create` |
| FB60 | Enter Incoming Invoices | F0859 | `SupplierInvoice` | `create` | `#SupplierInvoice-create` |
| FB70 | Enter Outgoing Invoices | F2555 | `CustomerInvoice` | `create` | `#CustomerInvoice-create` |
| FBL1N | Vendor Line Items | F0996 | `SupplierLineItem` | `display` | `#SupplierLineItem-display` |
| FBL3N | G/L Account Line Items | F0997 | `GLAccountLineItem` | `display` | `#GLAccountLineItem-display` |
| FBL5N | Customer Line Items | F0998 | `CustomerLineItem` | `display` | `#CustomerLineItem-display` |
| F110 | Payment Run | F1645 | `PaymentRun` | `manage` | `#PaymentRun-manage` |
| FK01 | Create Vendor | F0794 | `Supplier` | `create` | `#Supplier-create` |
| FK02 | Change Vendor | F0794 | `Supplier` | `change` | `#Supplier-change` |
| FK03 | Display Vendor | F0794 | `Supplier` | `display` | `#Supplier-display` |
| FS00 | G/L Account Master | F2217 | `GLAccount` | `manage` | `#GLAccount-manage` |
## Controlling (CO)
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | ---------------------------------- | ------------ | ---------------- | --------- | ------------------------ |
| KS01 | Create Cost Center | F1605 | `CostCenter` | `create` | `#CostCenter-create` |
| KS02 | Change Cost Center | F1605 | `CostCenter` | `change` | `#CostCenter-change` |
| KS03 | Display Cost Center | F1605 | `CostCenter` | `display` | `#CostCenter-display` |
| KP06 | Plan Cost Elements/Activity Inputs | F2741 | `CostCenterPlan` | `manage` | `#CostCenterPlan-manage` |
| CJ20N | Project Builder | F2107 | `Project` | `manage` | `#Project-manage` |
## Plant Maintenance (PM) / Asset Management
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | -------------------------- | ------------ | ------------------------- | -------- | --------------------------------- |
| IW21 | Create Notification | F1704 | `MaintenanceNotification` | `create` | `#MaintenanceNotification-create` |
| IW22 | Change Notification | F1704 | `MaintenanceNotification` | `change` | `#MaintenanceNotification-change` |
| IW31 | Create Maintenance Order | F1705 | `MaintenanceOrder` | `create` | `#MaintenanceOrder-create` |
| IW32 | Change Maintenance Order | F1705 | `MaintenanceOrder` | `change` | `#MaintenanceOrder-change` |
| IE01 | Create Equipment | F3381 | `Equipment` | `create` | `#Equipment-create` |
| IL01 | Create Functional Location | F3382 | `FunctionalLocation` | `create` | `#FunctionalLocation-create` |
## Human Capital Management (HCM)
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | ----------------------- | ------------ | --------------- | --------- | ---------------------- |
| PA20 | Display HR Master Data | F0711 | `Employee` | `display` | `#Employee-display` |
| PA30 | Maintain HR Master Data | F0711 | `Employee` | `manage` | `#Employee-manage` |
| PT01 | Create Work Schedule | F1560 | `WorkSchedule` | `manage` | `#WorkSchedule-manage` |
| CATS | Time Sheet Entry | F1286 | `TimeEntry` | `create` | `#TimeEntry-create` |
## Cross-Application
| TCode | Description | Fiori App ID | Semantic Object | Action | Intent Hash |
| ----- | ------------------- | ------------ | ----------------- | -------- | ------------------------- |
| BP | Business Partner | F1757 | `BusinessPartner` | `manage` | `#BusinessPartner-manage` |
| NWBC | SAP Fiori Launchpad | - | `Shell` | `home` | `#Shell-home` |
| SM37 | Job Overview | F1239 | `ApplicationJob` | `manage` | `#ApplicationJob-manage` |
| SU01 | User Maintenance | F1302 | `User` | `manage` | `#User-manage` |
## Praman Navigation Patterns
### By Transaction Code (Convenience Wrapper)
```typescript
// Praman resolves the TCode to its Fiori intent internally
await ui5Navigation.navigateToTransaction('ME21N');
```
### By Semantic Object and Action
```typescript
await ui5Navigation.navigateToApp('MM-PUR-PO', {
semanticObject: 'PurchaseOrder',
action: 'create',
});
```
### By Intent Hash with Parameters
```typescript
await ui5Navigation.navigateToIntent('#PurchaseOrder-display?PurchaseOrder=4500000001');
```
### By Fiori App ID
```typescript
await ui5Navigation.navigateToFioriApp('F0842', {
params: { PurchaseOrder: '4500000001' },
});
```
## Tips
- **App IDs may differ** between S/4HANA versions. Always verify against your system's FLP
content catalog.
- **Custom Fiori apps** will have their own semantic objects not listed here. Check your FLP
admin console for custom intent mappings.
- **Transaction wrappers** (SAP GUI in Fiori) use the TCode directly but are rendered inside
the FLP shell — same navigation pattern applies.
- Use `ui5Navigation.getCurrentIntent()` to verify you landed on the correct app after navigation.
---
## Business Process Examples
Complete end-to-end test examples for core SAP business processes. Each example is runnable
with Praman fixtures and demonstrates real-world patterns for Purchase Orders, Sales Orders,
Journal Entries, and cross-process P2P flows.
## Purchase Order — ME21N / Fiori
### Classic GUI (ME21N via Fiori Launchpad)
```typescript
test.describe('Purchase Order Creation (ME21N)', () => {
test('create standard purchase order', async ({ ui5, ui5Navigation, ui5Matchers }) => {
await test.step('navigate to Create Purchase Order app', async () => {
await ui5Navigation.navigateToApp('MM-PUR-PO', {
semanticObject: 'PurchaseOrder',
action: 'create',
});
});
await test.step('fill header data', async () => {
await ui5.fill({ controlType: 'sap.m.Input', id: /supplierInput/ }, '100001');
await ui5.fill({ controlType: 'sap.m.Input', id: /purchOrgInput/ }, '1000');
await ui5.fill({ controlType: 'sap.m.Input', id: /companyCodeInput/ }, '1000');
});
await test.step('add line item', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Add Item' } });
await ui5.fill({ controlType: 'sap.m.Input', id: /materialInput/ }, 'MAT-001');
await ui5.fill({ controlType: 'sap.m.Input', id: /quantityInput/ }, '100');
await ui5.fill({ controlType: 'sap.m.Input', id: /plantInput/ }, '1000');
});
await test.step('save and verify', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Save' } });
const messageStrip = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
await ui5Matchers.toHaveText(messageStrip, /Purchase Order \d+ created/);
});
});
});
```
### Fiori Elements — Manage Purchase Orders
```typescript
test.describe('Manage Purchase Orders (Fiori Elements)', () => {
test('create PO via Fiori Elements object page', async ({
ui5,
ui5Navigation,
ui5Matchers,
feListReport,
feObjectPage,
}) => {
await test.step('navigate to list report', async () => {
await ui5Navigation.navigateToApp('MM-PUR-PO', {
semanticObject: 'PurchaseOrder',
action: 'manage',
});
});
await test.step('create new entry', async () => {
await feListReport.clickCreate();
});
await test.step('fill header fields', async () => {
await feObjectPage.fillField('Supplier', '100001');
await feObjectPage.fillField('PurchasingOrganization', '1000');
await feObjectPage.fillField('CompanyCode', '1000');
await feObjectPage.fillField('PurchasingGroup', '001');
});
await test.step('add line item via table', async () => {
await feObjectPage.clickSectionCreate('Items');
await feObjectPage.fillField('Material', 'MAT-001');
await feObjectPage.fillField('OrderQuantity', '100');
await feObjectPage.fillField('Plant', '1000');
});
await test.step('save and verify', async () => {
await feObjectPage.clickSave();
await ui5Matchers.toHaveMessageStrip('Success', /Purchase Order \d+ created/);
});
});
});
```
## Sales Order — VA01 / Fiori
### Classic GUI (VA01 via Fiori Launchpad)
```typescript
test.describe('Sales Order Creation (VA01)', () => {
test('create standard sales order', async ({ ui5, ui5Navigation, ui5Matchers }) => {
await test.step('navigate to Create Sales Order app', async () => {
await ui5Navigation.navigateToApp('SD-SLS-SO', {
semanticObject: 'SalesOrder',
action: 'create',
});
});
await test.step('fill order type and org data', async () => {
await ui5.fill(
{ controlType: 'sap.m.Input', id: /orderTypeInput/ },
'OR', // Standard Order
);
await ui5.fill({ controlType: 'sap.m.Input', id: /salesOrgInput/ }, '1000');
await ui5.fill({ controlType: 'sap.m.Input', id: /distChannelInput/ }, '10');
await ui5.fill({ controlType: 'sap.m.Input', id: /divisionInput/ }, '00');
});
await test.step('fill customer and item data', async () => {
await ui5.fill({ controlType: 'sap.m.Input', id: /soldToPartyInput/ }, 'CUST-001');
await ui5.fill({ controlType: 'sap.m.Input', id: /materialInput/ }, 'MAT-100');
await ui5.fill({ controlType: 'sap.m.Input', id: /quantityInput/ }, '50');
});
await test.step('save and verify', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Save' } });
const message = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
await ui5Matchers.toHaveText(message, /Sales Order \d+ created/);
});
});
});
```
### Fiori Elements — Manage Sales Orders
```typescript
test.describe('Manage Sales Orders (Fiori Elements)', () => {
test('create sales order via object page', async ({
ui5Navigation,
ui5Matchers,
feListReport,
feObjectPage,
}) => {
await test.step('open app and create', async () => {
await ui5Navigation.navigateToApp('SD-SLS-SO', {
semanticObject: 'SalesOrder',
action: 'manage',
});
await feListReport.clickCreate();
});
await test.step('fill header', async () => {
await feObjectPage.fillField('SalesOrderType', 'OR');
await feObjectPage.fillField('SoldToParty', 'CUST-001');
await feObjectPage.fillField('SalesOrganization', '1000');
await feObjectPage.fillField('DistributionChannel', '10');
});
await test.step('add line items', async () => {
await feObjectPage.clickSectionCreate('Items');
await feObjectPage.fillField('Material', 'MAT-100');
await feObjectPage.fillField('RequestedQuantity', '50');
});
await test.step('save and capture order number', async () => {
await feObjectPage.clickSave();
await ui5Matchers.toHaveMessageStrip('Success', /Sales Order \d+ created/);
});
});
});
```
## Journal Entry — FB50 / Fiori
### Classic GUI (FB50 via Fiori Launchpad)
```typescript
test.describe('Journal Entry (FB50)', () => {
test('post G/L account journal entry', async ({ ui5, ui5Navigation, ui5Matchers }) => {
await test.step('navigate to Post Journal Entry', async () => {
await ui5Navigation.navigateToApp('FI-GL', {
semanticObject: 'JournalEntry',
action: 'create',
});
});
await test.step('fill document header', async () => {
await ui5.fill({ controlType: 'sap.m.DatePicker', id: /documentDatePicker/ }, '2025-01-15');
await ui5.fill({ controlType: 'sap.m.Input', id: /companyCodeInput/ }, '1000');
await ui5.fill({ controlType: 'sap.m.Input', id: /currencyInput/ }, 'USD');
});
await test.step('add debit line', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Add Row' } });
await ui5.fill(
{ controlType: 'sap.m.Input', id: /glAccountInput--0/ },
'400000', // Revenue account
);
await ui5.fill({ controlType: 'sap.m.Input', id: /debitAmountInput--0/ }, '1000.00');
});
await test.step('add credit line', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Add Row' } });
await ui5.fill(
{ controlType: 'sap.m.Input', id: /glAccountInput--1/ },
'113100', // Bank account
);
await ui5.fill({ controlType: 'sap.m.Input', id: /creditAmountInput--1/ }, '1000.00');
});
await test.step('post and verify', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Post' } });
const message = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
await ui5Matchers.toHaveText(message, /Document \d+ posted/);
});
});
});
```
### Fiori — Post Journal Entries (F0718)
```typescript
test.describe('Post Journal Entries Fiori (F0718)', () => {
test('create and post journal entry', async ({ ui5, ui5Navigation, ui5Matchers }) => {
await test.step('navigate to Post Journal Entries', async () => {
await ui5Navigation.navigateToIntent('#JournalEntry-create');
});
await test.step('fill header and line items', async () => {
await ui5.fill(
{ controlType: 'sap.ui.comp.smartfield.SmartField', id: /CompanyCode/ },
'1000',
);
await ui5.fill(
{ controlType: 'sap.ui.comp.smartfield.SmartField', id: /DocumentDate/ },
'2025-01-15',
);
// Debit line
await ui5.fill(
{ controlType: 'sap.ui.comp.smartfield.SmartField', id: /GLAccount.*0/ },
'400000',
);
await ui5.fill(
{ controlType: 'sap.ui.comp.smartfield.SmartField', id: /AmountInTransCrcy.*0/ },
'1000.00',
);
// Credit line
await ui5.fill(
{ controlType: 'sap.ui.comp.smartfield.SmartField', id: /GLAccount.*1/ },
'113100',
);
await ui5.fill(
{ controlType: 'sap.ui.comp.smartfield.SmartField', id: /AmountInTransCrcy.*1/ },
'-1000.00',
);
});
await test.step('post document', async () => {
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Post' } });
await ui5Matchers.toHaveMessageStrip('Success', /Document \d+ posted/);
});
});
});
```
## Purchase-to-Pay (P2P) Cross-Process E2E
The P2P flow spans multiple SAP modules: procurement, goods receipt, invoice verification,
and payment. This tutorial demonstrates a complete end-to-end test that exercises the full chain.
```typescript
test.describe('Purchase-to-Pay E2E Flow', () => {
let poNumber: string;
let grDocNumber: string;
let invoiceNumber: string;
test('complete P2P cycle', async ({
ui5,
ui5Navigation,
ui5Matchers,
feListReport,
feObjectPage,
}) => {
await test.step('Step 1: Create Purchase Order', async () => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
await feObjectPage.fillField('Supplier', '100001');
await feObjectPage.fillField('PurchasingOrganization', '1000');
await feObjectPage.fillField('CompanyCode', '1000');
await feObjectPage.clickSectionCreate('Items');
await feObjectPage.fillField('Material', 'MAT-001');
await feObjectPage.fillField('OrderQuantity', '10');
await feObjectPage.fillField('Plant', '1000');
await feObjectPage.clickSave();
const successMsg = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
const text = await successMsg.getText();
const match = text.match(/Purchase Order (\d+)/);
poNumber = match?.[1] ?? '';
expect(poNumber).toBeTruthy();
});
await test.step('Step 2: Post Goods Receipt (MIGO)', async () => {
await ui5Navigation.navigateToIntent('#GoodsReceipt-create');
await ui5.fill({ controlType: 'sap.m.Input', id: /poNumberInput/ }, poNumber);
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Load' } });
// Verify items auto-populated
const table = await ui5.control({ controlType: 'sap.m.Table', id: /itemsTable/ });
await ui5Matchers.toHaveRowCount(table, 1);
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Post' } });
const grMsg = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
const grText = await grMsg.getText();
const grMatch = grText.match(/Document (\d+)/);
grDocNumber = grMatch?.[1] ?? '';
expect(grDocNumber).toBeTruthy();
});
await test.step('Step 3: Create Invoice (MIRO)', async () => {
await ui5Navigation.navigateToIntent('#SupplierInvoice-create');
await ui5.fill({ controlType: 'sap.m.Input', id: /poReferenceInput/ }, poNumber);
await ui5.fill({ controlType: 'sap.m.Input', id: /invoiceAmountInput/ }, '5000.00');
await ui5.fill({ controlType: 'sap.m.DatePicker', id: /invoiceDatePicker/ }, '2025-02-01');
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Post' } });
const invMsg = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
});
const invText = await invMsg.getText();
const invMatch = invText.match(/Invoice (\d+)/);
invoiceNumber = invMatch?.[1] ?? '';
expect(invoiceNumber).toBeTruthy();
});
await test.step('Step 4: Verify Payment Run (F110)', async () => {
await ui5Navigation.navigateToApp('FI-AP', {
semanticObject: 'PaymentRun',
action: 'display',
});
await feListReport.setFilterValues({ Supplier: '100001', Status: 'Paid' });
await feListReport.clickGo();
await feListReport.expectRowCount({ min: 1 });
});
await test.step('Step 5: Verify complete document flow', async () => {
await ui5Navigation.navigateToIntent(`#PurchaseOrder-display?PurchaseOrder=${poNumber}`);
await ui5.click({
controlType: 'sap.m.IconTabFilter',
properties: { key: 'documentFlow' },
});
const flowTable = await ui5.control({ controlType: 'sap.m.Table', id: /docFlowTable/ });
await ui5Matchers.toHaveRowCount(flowTable, { min: 3 }); // PO + GR + Invoice
});
});
});
```
### Key Patterns in the P2P Example
| Pattern | Why |
| --------------------------------------------- | --------------------------------------------- |
| `test.step()` for each business step | Clear trace output, easy to pinpoint failures |
| Capture document numbers between steps | Enables cross-referencing across documents |
| `expect(poNumber).toBeTruthy()` after capture | Fail fast if a prior step silently failed |
| Navigate by semantic object + action | Resilient to FLP configuration changes |
| Verify document flow at the end | Confirms SAP linked all documents correctly |
## Data Cleanup Strategies
For repeatable tests, clean up created data after the test run:
```typescript
test.afterAll(async ({ ui5 }) => {
// Option 1: Use OData to delete test data
await ui5.odata.deleteEntity('/PurchaseOrders', poNumber);
// Option 2: Use an SAP cleanup BAPI via RFC
// (Requires backend RFC connection setup)
// Option 3: Flag for batch cleanup
// Store document numbers for nightly cleanup jobs
});
```
See the [Gold Standard Test Pattern](./gold-standard-test.md) for a complete template that
includes data cleanup, error handling, and all best practices.
---
## POST Upgrade Testing
How to use Praman for validating SAP system upgrades, support pack stacks, and Fiori
library updates. Covers regression strategies, comparison testing, and upgrade-specific
failure patterns.
## Why Upgrade Testing Matters
SAP upgrades (S/4HANA feature packs, support pack stacks, UI5 library updates) can introduce:
- **UI changes**: New controls, removed controls, changed layouts
- **Behavior changes**: Different default values, changed validation rules
- **API changes**: New OData properties, deprecated fields, changed entity types
- **Performance changes**: Slower/faster rendering, different loading patterns
Praman tests catch these regressions before they reach end users.
## Upgrade Testing Strategy
### Three-Phase Approach
```
Phase 1: Pre-Upgrade Baseline → Run full regression, save results
Phase 2: Post-Upgrade Validation → Run same suite on upgraded system
Phase 3: Comparison → Diff results, investigate failures
```
### Phase 1: Pre-Upgrade Baseline
Run the full regression suite and save artifacts:
```bash
# Run tests and save baseline results
npx playwright test \
--project=regression \
--reporter=json \
--output=baseline-results/ \
2>&1 | tee baseline-output.log
# Save screenshots for visual comparison
npx playwright test \
--project=regression \
--update-snapshots
```
### Phase 2: Post-Upgrade Validation
Run the identical test suite against the upgraded system:
```bash
# Run same tests against upgraded system
SAP_BASE_URL=https://upgraded-system.example.com \
npx playwright test \
--project=regression \
--reporter=json \
--output=upgrade-results/
```
### Phase 3: Comparison
Compare test results between pre and post:
```typescript
// scripts/compare-results.ts
interface TestResult {
title: string;
status: 'passed' | 'failed' | 'skipped';
duration: number;
}
const baseline: TestResult[] = JSON.parse(readFileSync('baseline-results/results.json', 'utf-8'));
const upgrade: TestResult[] = JSON.parse(readFileSync('upgrade-results/results.json', 'utf-8'));
// Find regressions: tests that passed before but fail now
const regressions = upgrade.filter((u) => {
const base = baseline.find((b) => b.title === u.title);
return base?.status === 'passed' && u.status === 'failed';
});
console.log(`Regressions: ${regressions.length}`);
regressions.forEach((r) => console.log(` REGRESSION: ${r.title}`));
```
## Upgrade-Specific Test Patterns
### UI5 Version Check
Verify the UI5 version after upgrade:
```typescript
test('verify UI5 version after upgrade', async ({ ui5, page }) => {
await test.step('check UI5 version', async () => {
const version = await ui5.getUI5Version();
expect(version).toMatch(/^1\.\d+\.\d+$/);
// Verify minimum expected version after upgrade
const [major, minor] = version.split('.').map(Number);
expect(major).toBe(1);
expect(minor).toBeGreaterThanOrEqual(120); // Expected post-upgrade version
});
});
```
### Control Existence Validation
After major upgrades, controls may be replaced or removed:
```typescript
test.describe('Post-Upgrade Control Validation', () => {
test('verify critical controls still exist', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
await test.step('SmartFilterBar exists', async () => {
const filterBar = await ui5.controlOrNull({
controlType: 'sap.ui.comp.smartfilterbar.SmartFilterBar',
});
expect(filterBar).not.toBeNull();
});
await test.step('SmartTable exists', async () => {
const table = await ui5.controlOrNull({
controlType: 'sap.ui.comp.smarttable.SmartTable',
});
expect(table).not.toBeNull();
});
await test.step('toolbar actions exist', async () => {
const createBtn = await ui5.controlOrNull({
controlType: 'sap.m.Button',
properties: { text: 'Create' },
});
expect(createBtn).not.toBeNull();
});
});
});
```
### Field Value Comparison
Compare default values and field behavior pre/post upgrade:
```typescript
test('verify default values unchanged after upgrade', async ({
ui5,
ui5Navigation,
feObjectPage,
}) => {
await ui5Navigation.navigateToIntent('#PurchaseOrder-create');
await test.step('company code defaults correctly', async () => {
const field = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/CompanyCode' },
});
const value = await field.getProperty('value');
expect(value).toBe('1000');
});
await test.step('currency defaults correctly', async () => {
const field = await ui5.control({
controlType: 'sap.ui.comp.smartfield.SmartField',
bindingPath: { path: '/DocumentCurrency' },
});
const value = await field.getProperty('value');
expect(value).toBe('USD');
});
});
```
### Performance Regression Detection
```typescript
test('page load time within acceptable range post-upgrade', async ({ ui5Navigation, page }) => {
const measurements: number[] = [];
for (let i = 0; i < 3; i++) {
const start = Date.now();
await ui5Navigation.navigateToIntent('#PurchaseOrder-manage');
measurements.push(Date.now() - start);
// Navigate away before next measurement
await ui5Navigation.navigateToIntent('#Shell-home');
}
const avgLoadTime = measurements.reduce((a, b) => a + b, 0) / measurements.length;
await test.info().attach('performance-post-upgrade', {
body: JSON.stringify({
measurements,
average: avgLoadTime,
threshold: 8000,
}),
contentType: 'application/json',
});
// Allow 20% degradation from 5s baseline
expect(avgLoadTime).toBeLessThan(8000);
});
```
## Playwright Config for Upgrade Testing
```typescript
// playwright.config.ts
export default defineConfig({
projects: [
// Auth setup
{
name: 'setup',
testMatch: /auth-setup\.ts/,
},
// Smoke tests — run first, fail fast
{
name: 'upgrade-smoke',
testDir: './tests/smoke',
dependencies: ['setup'],
use: {
...devices['Desktop Chrome'],
storageState: '.auth/sap-session.json',
},
},
// Full regression — only if smoke passes
{
name: 'upgrade-regression',
testDir: './tests/regression',
dependencies: ['upgrade-smoke'],
use: {
...devices['Desktop Chrome'],
storageState: '.auth/sap-session.json',
trace: 'on', // Full trace for upgrade analysis
screenshot: 'on', // Screenshot every test for comparison
},
retries: 1,
},
],
});
```
## Common Upgrade Failure Patterns
| Failure Pattern | Symptom | Root Cause | Fix |
| ------------------- | ----------------------------------- | -------------------------------- | -------------------------------- |
| Control not found | `ERR_CONTROL_NOT_FOUND` | Control type renamed or replaced | Update `controlType` in selector |
| Property removed | `getProperty()` returns `undefined` | OData annotation change | Update property name |
| Layout change | Screenshot diff | New Fiori design guidelines | Update visual baselines |
| New mandatory field | Save fails with validation error | New field added by upgrade | Add field fill to test |
| Deprecation | Console warnings | API deprecated in new version | Use replacement API |
| Auth flow change | Login fails | New SSO redirect | Update auth strategy |
## Upgrade Testing Checklist
- [ ] Save pre-upgrade baseline (test results + screenshots)
- [ ] Document current UI5 version: `ui5.getUI5Version()`
- [ ] Run smoke tests first (auth, navigation, basic CRUD)
- [ ] Run full regression suite
- [ ] Compare visual snapshots (see [Visual Regression](./visual-regression.md))
- [ ] Check OData responses for schema changes
- [ ] Verify performance within acceptable thresholds
- [ ] Check browser console for new deprecation warnings
- [ ] Update selectors for any renamed/replaced controls
- [ ] Update visual baselines after intentional UI changes
- [ ] Document all upgrade-related test changes for future reference
---
## Behavioral Equivalence
# Behavioral Equivalence (BEHAV-EQ)
Praman v1.0 is a ground-up rewrite. Behavioral equivalence testing ensures it produces the
same observable results as the reference implementation (wdi5) for core SAP UI5 operations,
while documenting intentional divergences.
## Golden Master Methodology
Golden master testing (also called characterization testing or approval testing) captures the
output of the reference implementation and compares it against the new implementation.
```
┌──────────────────────┐ ┌──────────────────────┐
│ wdi5 (reference) │ │ Praman (new) │
│ │ │ │
│ Run scenario S │ │ Run scenario S │
│ Capture output O_r │ │ Capture output O_n │
│ │ │ │
└──────────┬───────────┘ └──────────┬────────────┘
│ │
▼ ▼
┌─────────────────────────────────────┐
│ Compare O_r vs O_n │
│ ● Match → behavioral equivalence │
│ ● Diff → classify as bug or │
│ intentional divergence │
└─────────────────────────────────────┘
```
### What Is Compared
For each scenario, the comparison covers:
| Aspect | How Compared |
| -------------------- | ------------------------------------------------------ |
| Control discovery | Same control found (by ID, type, properties) |
| Property values | `getValue()`, `getText()`, `getProperty()` match |
| Event side effects | Same UI5 events fired (liveChange, change, press) |
| OData model state | Same model data after interaction |
| Error behavior | Same error for invalid selectors / missing controls |
| Timing (qualitative) | Both complete within reasonable time (not exact match) |
### What Is NOT Compared
- Exact timing (Praman may be faster or slower)
- Internal implementation details (how the bridge communicates)
- Log output format (Praman uses pino structured logging, wdi5 uses console)
- Error message text (Praman uses structured errors with codes, wdi5 uses plain strings)
## The 8 Parity Scenarios
These scenarios cover the core operations that most SAP UI5 test suites rely on.
### Scenario 1: Control Discovery by ID
Find a control using its stable ID.
```typescript
// wdi5 reference
const control = await browser.asControl({ selector: { id: 'container-app---main--saveBtn' } });
// Praman equivalent
const control = await ui5.control({ id: 'saveBtn' });
```
**Comparison**: Both return a proxy wrapping the same DOM element. Verified by comparing
`control.getId()` output.
**Known divergence**: Praman accepts short IDs (`saveBtn`) and resolves the full ID
internally. wdi5 requires the full generated ID including view prefix. This is an intentional
improvement.
### Scenario 2: Control Discovery by Type + Properties
Find a control using its UI5 type and property values.
```typescript
// wdi5 reference
const input = await browser.asControl({
selector: {
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor' },
viewName: 'myApp.view.Main',
},
});
// Praman equivalent
const input = await ui5.control({
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor' },
viewName: 'myApp.view.Main',
});
```
**Comparison**: Both discover the same control. Verified by comparing `getId()` and
`getMetadata().getName()`.
**Known divergence**: None -- selectors are semantically identical.
### Scenario 3: Property Read (getValue, getText)
Read a property from a discovered control.
```typescript
// wdi5 reference
const value = await input.getValue();
const text = await button.getText();
// Praman equivalent
const value = await input.getValue();
const text = await button.getText();
```
**Comparison**: Return values are strictly equal (`===`). Tested with string, number, boolean,
and null properties.
**Known divergence**: None -- both call the same UI5 control API methods.
### Scenario 4: Input Fill (enterText + Events)
Fill a value into an input control with proper event firing.
```typescript
// wdi5 reference
await input.enterText('100001');
// Praman equivalent
await ui5.fill({ id: 'vendorInput' }, '100001');
```
**Comparison**: After filling, both produce the same:
- `getValue()` returns `'100001'`
- `liveChange` event was fired (verified by model binding update)
- `change` event was fired (verified by validation trigger)
**Known divergence**: Praman fires events in the order `liveChange` then `change`, matching
SAP's documented event sequence. wdi5's event ordering depends on the WebDriverIO interaction
mode. This is an intentional alignment with SAP documentation.
### Scenario 5: Button Press (firePress / click)
Trigger a press action on a button control.
```typescript
// wdi5 reference
await button.press();
// Praman equivalent
await ui5.press({ id: 'submitBtn' });
```
**Comparison**: Both trigger the button's `press` event, which fires any attached handlers.
Verified by checking the UI state change caused by the handler (e.g., dialog opens, navigation
occurs).
**Known divergence**: Praman uses a three-tier strategy (UI5 firePress -> fireTap -> DOM
click). wdi5 uses WebDriverIO's click action. Both produce the same observable outcome, but
through different mechanisms. If `firePress` is not available on a control, Praman falls back
to DOM click; wdi5 always uses DOM click.
### Scenario 6: Table Row Count and Cell Text
Read table dimensions and cell contents.
```typescript
// wdi5 reference
const rows = await table.getAggregation('items');
const cellText = await rows[0].getCells()[2].getText();
// Praman equivalent
await expect(table).toHaveUI5RowCount(5);
await expect(table).toHaveUI5CellText(0, 2, 'Expected Value');
```
**Comparison**: Same row count and cell text values.
**Known divergence**: Praman provides dedicated matchers (`toHaveUI5RowCount`,
`toHaveUI5CellText`) that are more ergonomic than manual aggregation traversal. The underlying
data is identical, but the API surface differs intentionally for readability.
### Scenario 7: Select Dropdown Item
Select an item in a `sap.m.Select` or `sap.m.ComboBox`.
```typescript
// wdi5 reference
await select.open();
const items = await select.getAggregation('items');
await items[2].press();
// Praman equivalent
await ui5.select({ id: 'countrySelect' }, 'US');
```
**Comparison**: After selection, `getSelectedKey()` returns the same value in both.
**Known divergence**: Praman accepts the key or display text directly, without requiring
manual open/press steps. This is an intentional API simplification. The underlying
`setSelectedKey()` / `fireChange()` calls are equivalent.
### Scenario 8: Error on Missing Control
Attempt to find a control that does not exist.
```typescript
// wdi5 reference
const control = await browser.asControl({
selector: { id: 'nonExistentControl' },
});
// wdi5: returns a falsy value or throws after timeout
// Praman equivalent
const control = await ui5.control({ id: 'nonExistentControl' });
// Praman: throws ControlError with code ERR_CONTROL_NOT_FOUND
```
**Comparison**: Both fail to find the control within the configured timeout.
**Known divergence (intentional)**: Praman throws a structured `ControlError` with error code
`ERR_CONTROL_NOT_FOUND`, `retryable: true`, and `suggestions[]` array. wdi5 returns a falsy
proxy or throws a generic error. Praman's structured errors are an intentional improvement
for debuggability.
## Running Parity Tests
Parity tests live in a dedicated directory and run both implementations against the same SAP
system.
```
tests/
parity/
scenarios/
01-discovery-by-id.ts
02-discovery-by-type.ts
03-property-read.ts
04-input-fill.ts
05-button-press.ts
06-table-operations.ts
07-select-dropdown.ts
08-missing-control.ts
golden-masters/
01-discovery-by-id.json
02-discovery-by-type.json
...
compare.ts
```
### Generating Golden Masters (wdi5)
```typescript
// tests/parity/generate-golden-master.ts
interface GoldenMaster {
scenario: string;
timestamp: string;
results: Record;
}
function saveGoldenMaster(scenario: string, results: Record): void {
const master: GoldenMaster = {
scenario,
timestamp: new Date().toISOString(),
results,
};
writeFileSync(
join(__dirname, 'golden-masters', `${scenario}.json`),
JSON.stringify(master, null, 2),
);
}
// Example: capture discovery result
const control = await browser.asControl({
selector: { id: 'container-app---main--saveBtn' },
});
saveGoldenMaster('01-discovery-by-id', {
controlId: await control.getId(),
controlType: await control.getMetadata().getName(),
text: await control.getText(),
enabled: await control.getEnabled(),
});
```
### Comparing Against Golden Masters (Praman)
```typescript
// tests/parity/scenarios/01-discovery-by-id.ts
interface GoldenMaster {
scenario: string;
results: {
controlId: string;
controlType: string;
text: string;
enabled: boolean;
};
}
test('parity: discovery by ID matches golden master', async ({ ui5 }) => {
const golden: GoldenMaster = JSON.parse(
readFileSync(join(__dirname, '../golden-masters/01-discovery-by-id.json'), 'utf-8'),
);
const control = await ui5.control({ id: 'saveBtn' });
await test.step('control type matches', async () => {
const controlType = await control.getMetadata().getName();
expect(controlType).toBe(golden.results.controlType);
});
await test.step('text matches', async () => {
const text = await control.getText();
expect(text).toBe(golden.results.text);
});
await test.step('enabled state matches', async () => {
const enabled = await control.getEnabled();
expect(enabled).toBe(golden.results.enabled);
});
// controlId may differ in prefix -- check suffix only (intentional divergence)
await test.step('control ID resolves to same element', async () => {
const id = await control.getId();
expect(id).toContain('saveBtn');
});
});
```
## Divergence Classification
Every difference between Praman and wdi5 output must be classified:
| Classification | Action Required |
| --------------------------- | ----------------------------------------------- |
| **Bug** | Fix in Praman to match wdi5 behavior |
| **Intentional improvement** | Document in this page with rationale |
| **Behavioral equivalent** | Same observable outcome via different mechanism |
### Documented Intentional Divergences
| Scenario | Praman Behavior | wdi5 Behavior | Rationale |
| ------------------- | ---------------------------------------- | --------------------------------- | ----------------------------------- |
| Short ID resolution | Accepts `saveBtn`, resolves full ID | Requires full generated ID | Better DX, less brittle |
| Event ordering | `liveChange` then `change` (SAP spec) | Varies by interaction mode | Aligns with SAP documentation |
| Press mechanism | Three-tier: firePress -> fireTap -> DOM | DOM click via WebDriverIO | More reliable for UI5 controls |
| Error structure | `ControlError` with code + suggestions | Generic error or falsy return | Better debuggability |
| Table assertions | Dedicated matchers (`toHaveUI5RowCount`) | Manual aggregation traversal | Ergonomic API for common operations |
| Select API | `ui5.select(selector, value)` one-liner | Manual open -> find item -> press | Simplified API, same result |
### How to Add a New Divergence
When a parity test reveals a difference:
1. Determine if it is a **bug** or **intentional divergence**
2. If intentional, add a row to the divergence table above with rationale
3. If a bug, file an issue and fix Praman to match wdi5's observable behavior
4. Update the golden master if wdi5's behavior was also wrong (rare -- wdi5 is the reference)
## Running the Full Parity Suite
```bash
# Generate golden masters (requires wdi5 + WebDriverIO setup)
npx wdio run wdio.conf.ts --spec tests/parity/generate-golden-master.ts
# Run Praman parity tests against golden masters
npx playwright test tests/parity/scenarios/
# Compare results
npx tsx tests/parity/compare.ts
```
The compare script exits with a non-zero code if any undocumented divergence is found.
Documented divergences (listed in `divergences.json`) are excluded from the comparison.
---
## Glossary
Definitions of key terms used throughout Praman documentation. Each term includes a definition,
the equivalent concept in SAP Tosca, and the equivalent in wdi5.
## A
### Assertion
A verification that checks whether the application state matches expectations. Praman uses
Playwright's `expect()` and custom SAP-specific matchers.
| Framework | Term / API |
| ---------- | -------------------------------------- |
| **Praman** | `expect()`, `ui5Matchers.toHaveText()` |
| **Tosca** | Verify module, TBox verification |
| **wdi5** | `expect()` (Jasmine/Mocha) |
### Auto-Waiting
Playwright automatically waits for elements to be actionable (visible, enabled, stable) before
performing actions. Praman extends this with UI5-specific waiting (pending requests, framework
busy).
| Framework | Term / API |
| ---------- | ------------------------------ |
| **Praman** | Built-in, `waitForUI5Stable()` |
| **Tosca** | Implicit synchronization |
| **wdi5** | `wdi5.waitForUI5()` |
## B
### Bridge
The communication layer between Node.js (Playwright) and the browser's UI5 runtime. Praman
sends commands via `page.evaluate()` to interact with the UI5 control tree.
| Framework | Term / API |
| ---------- | ------------------------------ |
| **Praman** | `BridgeAdapter`, `#bridge/*` |
| **Tosca** | Engine / Scan |
| **wdi5** | `wdi5` bridge (WebSocket/HTTP) |
### Browser Context
A Playwright concept representing an isolated browser session with its own cookies, storage,
and cache. Used for parallel test isolation.
| Framework | Term / API |
| ---------- | ----------------------------- |
| **Praman** | `BrowserContext` (Playwright) |
| **Tosca** | Browser instance |
| **wdi5** | Browser session (WebDriverIO) |
## C
### Capability
A named feature that Praman exposes, combining multiple low-level operations into a high-level
business action. Capabilities are discoverable by AI agents.
| Framework | Term / API |
| ---------- | -------------------------------------------- |
| **Praman** | `@capability` TSDoc tag, capability registry |
| **Tosca** | Module |
| **wdi5** | Not available |
### Control
A UI5 widget (button, input, table, etc.) managed by the UI5 framework. Controls have
properties, events, and aggregations. Praman interacts with controls via their UI5 identity,
not their DOM representation.
| Framework | Term / API |
| ---------- | ------------------------------- |
| **Praman** | `ui5.control()`, `ControlProxy` |
| **Tosca** | Control / TBox |
| **wdi5** | `browser.asControl()` |
## F
### Fixture
A Playwright concept for dependency injection in tests. Fixtures provide reusable setup/teardown
logic and are declared in the test function signature.
| Framework | Term / API |
| ---------- | -------------------------------------------- |
| **Praman** | `ui5`, `ui5Navigation`, `feListReport`, etc. |
| **Tosca** | Test Configuration Parameters |
| **wdi5** | Not available (uses `before`/`after` hooks) |
## H
### Hook
A function that runs before or after tests/suites. Used for setup (auth, navigation) and
teardown (data cleanup).
| Framework | Term / API |
| ---------- | ------------------------------------------------------------------------------ |
| **Praman** | `test.beforeAll()`, `test.beforeEach()`, `test.afterAll()`, `test.afterEach()` |
| **Tosca** | SetUp / TearDown test steps |
| **wdi5** | `before()`, `beforeEach()`, `after()`, `afterEach()` |
## I
### Intent
A semantic navigation target in SAP Fiori Launchpad, composed of a semantic object and action
(e.g., `#PurchaseOrder-create`). In Praman's AI layer, intents also describe what the user
wants to accomplish.
| Framework | Term / API |
| ---------- | ------------------------------------------------------- |
| **Praman** | `ui5Navigation.navigateToIntent()`, `@intent` TSDoc tag |
| **Tosca** | Navigation module |
| **wdi5** | `wdi5.goTo()` (URL-based) |
## L
### Locator
A Playwright object that represents a way to find element(s) on the page. Locators are lazy
and auto-waiting. Praman converts UI5 selectors to locators internally.
| Framework | Term / API |
| ---------- | ------------------------------------ |
| **Praman** | `control.getLocator()`, internal use |
| **Tosca** | Scan result reference |
| **wdi5** | WebDriverIO element |
## M
### mergeTests()
A Playwright utility that combines multiple test fixture sets into one. Used to compose
Praman fixtures with custom project fixtures.
| Framework | Term / API |
| ---------- | ------------------------------------ |
| **Praman** | `mergeTests(pramanTest, customTest)` |
| **Tosca** | Module composition |
| **wdi5** | Not available |
## P
### Page Object
A design pattern that encapsulates page-specific selectors and interactions behind a clean API.
Praman's fixtures serve a similar role without requiring explicit page object classes.
| Framework | Term / API |
| ---------- | ----------------------------------------------------- |
| **Praman** | Fixtures (implicit page objects), or explicit classes |
| **Tosca** | Module |
| **wdi5** | Page object classes |
### Proxy
A typed wrapper around a remote UI5 control. The proxy provides a type-safe API for reading
properties and invoking methods, while internally communicating via the bridge.
| Framework | Term / API |
| ---------- | -------------------------- |
| **Praman** | `ControlProxy`, `#proxy/*` |
| **Tosca** | Control wrapper |
| **wdi5** | `WDI5Control` |
## R
### Recipe
A pre-built sequence of Praman operations for common SAP tasks. Recipes combine navigation,
interaction, and verification into a single reusable function.
| Framework | Term / API |
| ---------- | ------------------------------------ |
| **Praman** | `@recipe` TSDoc tag, recipe registry |
| **Tosca** | Reusable test step block |
| **wdi5** | Not available |
### Reporter
A Playwright plugin that processes test results and generates reports. Praman ships
`ComplianceReporter` and `ODataTraceReporter`.
| Framework | Term / API |
| ---------- | ------------------------------------------ |
| **Praman** | `ComplianceReporter`, `ODataTraceReporter` |
| **Tosca** | Execution log |
| **wdi5** | Allure / spec reporter |
## S
### Selector
An object describing how to find a UI5 control. Unlike CSS selectors, Praman selectors query
the UI5 control tree using control type, properties, binding paths, and ancestry.
| Framework | Term / API |
| ---------- | ---------------------- |
| **Praman** | `UI5Selector` object |
| **Tosca** | TBox properties / Scan |
| **wdi5** | `wdi5Selector` object |
### storageState
A Playwright mechanism for persisting browser session data (cookies, localStorage) to a JSON
file. Used for sharing authentication across tests.
| Framework | Term / API |
| ---------- | -------------------------------- |
| **Praman** | `context.storageState({ path })` |
| **Tosca** | Session management |
| **wdi5** | Cookie injection |
## T
### test.step()
A Playwright method that groups related actions into a named step. Steps appear in traces
and reports, making failures easy to locate.
| Framework | Term / API |
| ---------- | ----------------------------------------------------- |
| **Praman** | `await test.step('description', async () => { ... })` |
| **Tosca** | Test step |
| **wdi5** | `it()` / custom logging |
### Trace
A Playwright recording that captures screenshots, network activity, console logs, and action
timelines for post-mortem debugging.
| Framework | Term / API |
| ---------- | ----------------------------------- |
| **Praman** | `trace: 'on-first-retry'` in config |
| **Tosca** | Execution log with screenshots |
| **wdi5** | Not built-in (manual screenshots) |
## W
### Worker
A Playwright process that runs tests in parallel. Each worker gets its own browser instance
and isolated state. SAP tests typically use `workers: 1` for stateful processes.
| Framework | Term / API |
| ---------- | ----------------------------------- |
| **Praman** | `workers` in `playwright.config.ts` |
| **Tosca** | Distributed execution agent |
| **wdi5** | WebDriverIO `maxInstances` |
## Quick Reference Table
| Term | Praman | Tosca | wdi5 |
| ------------ | ---------------------------------- | ------------------ | ----------------------- |
| Find control | `ui5.control()` | TBox Scan | `browser.asControl()` |
| Click | `ui5.click()` | Click module | `element.click()` |
| Fill input | `ui5.fill()` | Set module | `element.setValue()` |
| Navigate | `ui5Navigation.navigateToIntent()` | Navigation module | `wdi5.goTo()` |
| Wait for UI5 | Automatic | Implicit sync | `wdi5.waitForUI5()` |
| Verify text | `ui5Matchers.toHaveText()` | Verify module | `expect().toHaveText()` |
| Auth setup | Setup project + `storageState` | Session management | Cookie injection |
| Run tests | `npx playwright test` | Tosca Commander | `npx wdio run` |
| Debug | `PWDEBUG=1`, trace viewer | Tosca debug mode | `--inspect` flag |
| Report | HTML + ComplianceReporter | Execution log | Allure / spec |
---
## Migration from Vanilla Playwright
Already using Playwright for web testing? This guide shows when and how to layer Praman on top
of your existing Playwright tests for SAP UI5 applications.
## When to Use `page.locator()` vs `ui5.control()`
Praman does not replace Playwright -- it extends it. Use each where it makes sense.
### Use `page.locator()` When
- Targeting **non-UI5 elements** (plain HTML, third-party widgets, SAP UI5 Web Components)
- Working with **static DOM** that does not change between UI5 versions
- Testing **login pages** before the UI5 runtime has loaded
- Interacting with **iframes**, **file uploads**, or **browser dialogs**
- Asserting on **CSS properties** or **visual layout**
### Use `ui5.control()` When
- Targeting **SAP UI5 controls** (sap.m.Button, sap.m.Input, sap.ui.table.Table, etc.)
- Control **DOM IDs are generated** and change across versions or deployments
- You need **OData binding path** or **i18n text** matching
- You need **UI5-level assertions** (value state, enabled, editable, binding)
- Working with **SmartFields** that wrap inner controls dynamically
- Navigating the **Fiori Launchpad** shell
### Decision Tree
```text
Is the element a SAP UI5 control?
YES -> Does its DOM structure change across UI5 versions/themes?
YES -> Use ui5.control() (stable contract via UI5 registry)
NO -> Either works, but ui5.control() is safer long-term
NO -> Use page.locator() (standard Playwright)
```
## Auto-Waiting Differences
Both Playwright and Praman auto-wait, but they wait for different things.
| Behavior | Playwright `page.locator()` | Praman `ui5.control()` |
| ------------- | ----------------------------------------- | ------------------------------------------------------------------------------ |
| DOM presence | Waits for element in DOM | Waits for control in UI5 registry |
| Visibility | Waits for element visible (actionability) | Prefers visible controls (`preferVisibleControls`) |
| UI5 stability | No awareness | Auto-waits for `waitForUI5Stable()` (pending requests, timeouts, promises) |
| Retry | Built-in auto-retry on locators | Multi-strategy discovery chain (cache, direct-ID, RecordReplay, registry scan) |
| Timeout | `expect.timeout` / `actionTimeout` | `controlDiscoveryTimeout` (default 10s) + `ui5WaitTimeout` (default 30s) |
| Network idle | No built-in concept | Blocks WalkMe, analytics, overlay scripts automatically |
### What `waitForUI5Stable()` Does
Before every `ui5.control()` call, Praman ensures:
1. The UI5 bootstrap has completed (core libraries loaded)
2. All pending `XMLHttpRequest` / `fetch` calls have settled
3. All JavaScript `setTimeout` / `setInterval` callbacks have fired
4. The OData model has no pending requests
5. A 500ms DOM settle period has elapsed
This eliminates the need for `page.waitForTimeout()` (which is banned in Praman) and
`page.waitForLoadState('networkidle')` (which is unreliable for SPAs).
## Hybrid Playwright + Praman Test
The most practical migration approach is a **hybrid test** that uses both APIs in the same file.
Praman's `test` and `expect` are extended versions of Playwright's -- your existing locators
continue to work.
```typescript
test('hybrid PW + Praman test', async ({ page, ui5, ui5Navigation, sapAuth }) => {
// Step 1: Playwright handles login (before UI5 loads)
await test.step('Login via SAP login page', async () => {
await page.goto(process.env.SAP_BASE_URL!);
// Plain Playwright locators for the login form
await page.locator('#USERNAME_FIELD input').fill('TESTUSER');
await page.locator('#PASSWORD_FIELD input').fill('secret');
await page.locator('#LOGIN_LINK').click();
// Wait for redirect to FLP
await page.waitForURL('**/FioriLaunchpad*');
});
// Step 2: Praman takes over once UI5 is loaded
await test.step('Navigate to PO app', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
// Step 3: Use ui5.control() for UI5 controls
await test.step('Verify table loaded', async () => {
const table = await ui5.control({
controlType: 'sap.m.Table',
id: 'poTable',
});
await expect(table).toHaveUI5RowCount({ min: 1 });
});
// Step 4: Mix both in the same step when needed
await test.step('Check page title and create button', async () => {
// Playwright for the browser title
await expect(page).toHaveTitle(/Purchase Orders/);
// Praman for the UI5 button
const createBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Create' },
});
await expect(createBtn).toBeUI5Enabled();
});
// Step 5: Playwright for screenshots and traces
await test.step('Capture evidence', async () => {
await page.screenshot({ path: 'evidence/po-list.png', fullPage: true });
});
});
```
## Converting Existing Tests Incrementally
You do not need to rewrite everything at once. The recommended approach:
### Phase 1: Drop-In Replacement (5 minutes)
Change your import and keep everything else the same.
```typescript
// Before
// After -- Praman re-exports everything from @playwright/test
// All your existing locator-based tests continue to work unchanged
test('existing test', async ({ page }) => {
await page.goto('/my-app');
await page.locator('#myButton').click();
await expect(page.locator('#result')).toHaveText('Done');
});
```
### Phase 2: Add UI5 Fixtures Gradually
Start using `ui5` for new assertions alongside existing locators.
```typescript
test('gradual adoption', async ({ page, ui5 }) => {
await page.goto('/my-app');
// Old way -- still works
await page.locator('#myButton').click();
// New way -- more resilient for UI5 controls
const result = await ui5.control({
controlType: 'sap.m.Text',
properties: { text: 'Done' },
});
await expect(result).toHaveUI5Text('Done');
});
```
### Phase 3: Replace Brittle Selectors
Find tests that break on UI5 upgrades and convert their selectors.
```typescript
// Before: Brittle CSS selector that breaks on theme/version changes
await page.locator('.sapMBtnInner.sapMBtnEmphasized').click();
// After: Stable UI5 selector via control registry
await ui5.click({
controlType: 'sap.m.Button',
properties: { type: 'Emphasized' },
});
```
### Phase 4: Adopt Navigation and Auth Fixtures
Replace manual navigation and login scripts with built-in fixtures.
```typescript
// Before: Manual hash navigation
await page.goto(
'https://my-system.com/sap/bc/ui5_ui5/ui2/ushell/shells/abap/FioriLaunchpad.html#PurchaseOrder-manage',
);
// After: Typed navigation
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
```
## Parallel Execution Guidance
Playwright parallelism works the same with Praman. Each worker gets its own browser context
with isolated cookies, storage, and UI5 state.
### Workers and SAP Sessions
```typescript
// playwright.config.ts
export default defineConfig({
// Each worker authenticates independently via storageState
workers: process.env.CI ? 2 : 1,
// SAP systems often struggle with high parallelism
// Start with 1-2 workers and increase carefully
fullyParallel: false, // Run tests within a file sequentially
retries: 1,
projects: [
{
name: 'setup',
testMatch: /auth-setup\.ts/,
teardown: 'teardown',
},
{
name: 'teardown',
testMatch: /auth-teardown\.ts/,
},
{
name: 'chromium',
dependencies: ['setup'],
use: { storageState: '.auth/sap-session.json' },
},
],
});
```
### Parallel Safety Tips
1. **Use unique test data**: The `testData` fixture generates UUIDs and timestamps to avoid conflicts
2. **Avoid shared state**: Do not rely on a specific PO number across parallel workers
3. **Lock management**: Use `flpLocks` to handle SM12 locks -- auto-cleanup prevents stale locks
4. **Session limits**: SAP systems may limit concurrent sessions per user -- use separate test users per worker if needed
```typescript
test('parallel-safe test', async ({ ui5, testData }) => {
// Generate unique test data per run
const po = testData.generate({
documentNumber: '{{uuid}}',
createdAt: '{{timestamp}}',
vendor: '100001',
});
await ui5.fill({ id: 'vendorInput' }, po.vendor);
// Each worker gets its own unique data -- no conflicts
});
```
## What You Gain Over Vanilla Playwright
| Feature | Vanilla Playwright | With Praman |
| --------------------------- | ------------------------- | ---------------------------------- |
| UI5 control registry access | Manual `page.evaluate()` | Built-in `ui5.control()` |
| UI5 stability waiting | Manual polling loops | Automatic `waitForUI5Stable()` |
| OData model access | Manual `page.evaluate()` | `ui5.odata.getModelData()` |
| SmartField handling | Fragile inner-control CSS | `controlType` + `properties` |
| FLP navigation | Manual URL construction | 9 typed methods |
| Authentication | Custom login scripts | 6 built-in strategies |
| UI5 assertions | Generic `expect` only | 10 custom matchers |
| Error recovery | Unstructured errors | Structured codes + suggestions |
| Analytics blocking | Manual route interception | Automatic request interceptor |
| Test data | Manual setup/teardown | Template generation + auto-cleanup |
---
## Migration from wdi5
Migrate your wdi5 test suite to Playwright-Praman with minimal friction. This guide maps every
wdi5 API to its Praman equivalent and highlights features that are new in Praman.
## Quick Comparison
| Aspect | wdi5 | Praman |
| ------------------ | ------------------------------------- | ----------------------------------------------- |
| Test runner | WebdriverIO | Playwright |
| UI5 bridge | `browser.asControl()` | `ui5.control()` |
| Selector engine | wdi5 selector object | `UI5Selector` object |
| Async model | WDIO auto-sync (deprecated) / `async` | Always `async/await` |
| Parallel execution | Limited (WDIO workers) | Native Playwright workers |
| Auth management | Manual login scripts | 6 built-in strategies + setup projects |
| AI integration | None | Built-in capabilities, recipes, agentic handler |
| OData helpers | None | Model-level + HTTP-level CRUD |
| FLP navigation | Manual hash navigation | 9 typed navigation methods |
## Core API Mapping
### Control Discovery
```typescript
// wdi5
const button = await browser.asControl({
selector: {
controlType: 'sap.m.Button',
properties: { text: 'Save' },
},
});
// Praman
test('find a button', async ({ ui5 }) => {
const button = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Save' },
});
});
```
### Multiple Controls
```typescript
// wdi5
const buttons = await browser.allControls({
selector: { controlType: 'sap.m.Button' },
});
// Praman
const buttons = await ui5.controls({ controlType: 'sap.m.Button' });
```
### Control Interaction
```typescript
// wdi5
const input = await browser.asControl({
selector: { id: 'nameInput' },
});
await input.enterText('John Doe');
const value = await input.getValue();
// Praman
const input = await ui5.control({ id: 'nameInput' });
await input.enterText('John Doe');
const value = await input.getValue();
// Or use the shorthand:
await ui5.fill({ id: 'nameInput' }, 'John Doe');
```
## Selector Field Mapping
The selector object structure is similar, with a few naming adjustments and additions.
| wdi5 Selector Field | Praman `UI5Selector` Field | Type | Notes |
| ------------------- | -------------------------- | ------------------------- | -------------------------------------------- |
| `controlType` | `controlType` | `string` | Identical. Fully qualified UI5 type. |
| `id` | `id` | `string \| RegExp` | Identical. Supports RegExp in Praman. |
| `viewName` | `viewName` | `string` | Identical. |
| `viewId` | `viewId` | `string` | Identical. |
| `properties` | `properties` | `Record` | Identical. Key-value property matchers. |
| `bindingPath` | `bindingPath` | `Record` | Identical. OData binding path matchers. |
| `i18NText` | `i18NText` | `Record` | Identical. i18n translated value matchers. |
| `ancestor` | `ancestor` | `UI5Selector` | Identical. Recursive parent matching. |
| `descendant` | `descendant` | `UI5Selector` | Identical. Recursive child matching. |
| `interaction` | `interaction` | `UI5Interaction` | Identical. `idSuffix` and `domChildWith`. |
| `searchOpenDialogs` | `searchOpenDialogs` | `boolean` | Identical. Search inside open dialogs. |
| `labelFor` | -- | -- | **Not available in Praman.** See note below. |
:::caution labelFor Selector
wdi5 supports a `labelFor` selector field that matches controls by their associated `sap.m.Label`.
Praman does not currently support `labelFor`. Use one of these alternatives:
```typescript
// Option 1: Match by properties directly
const input = await ui5.control({
controlType: 'sap.m.Input',
properties: { placeholder: 'Enter vendor name' },
});
// Option 2: Match by binding path
const input = await ui5.control({
controlType: 'sap.m.Input',
bindingPath: { value: '/Vendor/Name' },
});
// Option 3: Use ancestor to scope to a form group
const input = await ui5.control({
controlType: 'sap.m.Input',
ancestor: {
controlType: 'sap.ui.layout.form.FormElement',
descendant: {
controlType: 'sap.m.Label',
properties: { text: 'Vendor Name' },
},
},
});
```
:::
## Config Mapping
### wdi5 Config (wdio.conf.ts)
```typescript
// wdio.conf.ts
export const config = {
wdi5: {
waitForUI5Timeout: 30000,
logLevel: 'verbose',
url: 'https://sap-system.example.com',
skipInjectUI5OnStart: false,
},
specs: ['./test/specs/**/*.ts'],
baseUrl: 'https://sap-system.example.com',
};
```
### Praman Config (praman.config.ts + playwright.config.ts)
```typescript
// praman.config.ts
export default defineConfig({
logLevel: 'info', // wdi5 logLevel -> praman logLevel
ui5WaitTimeout: 30_000, // wdi5 waitForUI5Timeout -> praman ui5WaitTimeout
controlDiscoveryTimeout: 10_000,
interactionStrategy: 'ui5-native',
auth: {
strategy: 'basic',
baseUrl: process.env.SAP_BASE_URL!,
username: process.env.SAP_USER!,
password: process.env.SAP_PASS!,
},
});
```
```typescript
// playwright.config.ts
export default defineConfig({
testDir: './tests',
timeout: 120_000,
use: {
baseURL: process.env.SAP_BASE_URL,
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
projects: [
{ name: 'setup', testMatch: /auth-setup\.ts/, teardown: 'teardown' },
{ name: 'teardown', testMatch: /auth-teardown\.ts/ },
{
name: 'chromium',
dependencies: ['setup'],
use: { ...devices['Desktop Chrome'], storageState: '.auth/sap-session.json' },
},
],
});
```
## OPA5 Journey to test.step()
wdi5 test files often follow the OPA5 journey pattern. In Praman, use `test.step()` for
structured multi-step tests.
### wdi5 + OPA5 Journey
```typescript
// wdi5 test
describe('Purchase Order', () => {
it('should navigate to the app', async () => {
await browser.url('#/PurchaseOrder-manage');
await browser.asControl({ selector: { id: 'poTable' } });
});
it('should create a new PO', async () => {
const createBtn = await browser.asControl({
selector: { controlType: 'sap.m.Button', properties: { text: 'Create' } },
});
await createBtn.press();
});
it('should fill vendor field', async () => {
const vendorInput = await browser.asControl({
selector: { id: 'vendorInput' },
});
await vendorInput.enterText('100001');
});
});
```
### Praman + test.step()
```typescript
test('create a purchase order', async ({ ui5, ui5Navigation }) => {
await test.step('Navigate to PO app', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
await test.step('Click Create', async () => {
await ui5.click({
controlType: 'sap.m.Button',
properties: { text: 'Create' },
});
});
await test.step('Fill vendor field', async () => {
await ui5.fill({ id: 'vendorInput' }, '100001');
});
});
```
## New in Praman vs wdi5
Praman includes features that wdi5 does not offer. If you are migrating, these are worth
adopting early.
### 6 Built-In Auth Strategies
No more custom login scripts. Praman supports Basic, BTP SAML, Office 365, Custom, API, and
Certificate authentication out of the box.
```typescript
// Auth is handled by setup projects — no login code in tests
test('after auth', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
```
### 9 FLP Navigation Methods
```typescript
test('navigation', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await ui5Navigation.navigateToTile('Create Purchase Order');
await ui5Navigation.navigateToIntent('PurchaseOrder', 'create', { plant: '1000' });
await ui5Navigation.navigateToHome();
await ui5Navigation.navigateBack();
});
```
### Fiori Elements Helpers
Dedicated APIs for List Report and Object Page patterns.
```typescript
test('FE list report', async ({ fe }) => {
await fe.listReport.setFilter('Status', 'Active');
await fe.listReport.search();
await fe.listReport.navigateToItem(0);
await fe.objectPage.clickEdit();
await fe.objectPage.clickSave();
});
```
### AI-Powered Test Generation
```typescript
test('AI discovery', async ({ pramanAI }) => {
const context = await pramanAI.discoverPage({ interactiveOnly: true });
const result = await pramanAI.agentic.generateTest(
'Create a purchase order for vendor 100001',
page,
);
});
```
### Custom UI5 Matchers
10 UI5-specific Playwright matchers for expressive assertions.
```typescript
const button = await ui5.control({ id: 'saveBtn' });
await expect(button).toBeUI5Enabled();
await expect(button).toHaveUI5Text('Save');
await expect(button).toBeUI5Visible();
```
### OData Model + HTTP Operations
```typescript
test('OData', async ({ ui5 }) => {
// Model-level (reads from the in-browser UI5 OData model)
const data = await ui5.odata.getModelData('/PurchaseOrders');
// HTTP-level CRUD is also available with automatic CSRF handling
});
```
### SM12 Lock Management
```typescript
test('locks', async ({ flpLocks }) => {
const count = await flpLocks.getNumberOfLockEntries('TESTUSER');
await flpLocks.deleteAllLockEntries('TESTUSER');
// Auto-cleanup on teardown
});
```
### Structured Error Codes
Every Praman error includes a machine-readable `code`, `retryable` flag, and `suggestions[]`
array for self-healing tests. wdi5 errors are unstructured strings.
## Step-by-Step Migration Checklist
1. **Install dependencies**: `npm install playwright-praman @playwright/test`
2. **Install browsers**: `npx playwright install`
3. **Create config files**: `praman.config.ts` and `playwright.config.ts`
4. **Set up auth**: Create `tests/auth-setup.ts` using one of the 6 strategies
5. **Convert selectors**: Replace `browser.asControl({ selector: {...} })` with `ui5.control({...})`
6. **Convert assertions**: Replace WDIO `expect` with Playwright `expect` + UI5 matchers
7. **Structure tests**: Convert OPA5 journey `describe/it` blocks to `test` + `test.step()`
8. **Adopt navigation**: Replace `browser.url('#/hash')` with `ui5Navigation` methods
9. **Run and verify**: `npx playwright test`
---
## From Tosca to Playwright-Praman
A comprehensive guide for teams migrating from Tricentis Tosca to Playwright + Praman for
SAP Fiori and UI5 web application testing. Covers concept mapping, workflow changes, and
practical code examples.
## Quick Comparison
| Aspect | Tosca | Playwright + Praman |
| ------------------ | ------------------------------ | -------------------------------------------- |
| Licensing | Per-user commercial license | Open source (Apache 2.0) |
| SAP Fiori support | Via SAP UI5 scan / engine | Native UI5 control registry |
| CI/CD integration | Tosca CI adapter | First-class CLI (`npx playwright test`) |
| Version control | Tosca workspace | Git (standard branching, PRs, diffs) |
| Parallel execution | Tosca Distributed Execution | Native Playwright workers |
| AI/ML | Tosca Vision AI (image-based) | LLM-powered test generation + recipe library |
| Browser support | Chrome, Firefox, Edge | Chrome, Firefox, Safari, Edge |
| Scripting language | Tosca TestStepValues / Modules | TypeScript (full programming language) |
| Community | Tricentis community | Playwright 80k+ GitHub stars, npm ecosystem |
:::note SAP GUI Testing
Tosca excels at SAP GUI testing via its SAP engine. If your test scope includes SAP GUI
transactions (not browser-based Fiori apps), keep Tosca for those tests and use Praman for
Fiori/UI5 web apps. Both can coexist in a CI pipeline.
:::
## Concept Mapping Table
The following table maps 25 core Tosca concepts to their Playwright/Praman equivalents.
| # | Tosca Concept | Playwright/Praman Equivalent | Notes |
| --- | ------------------------- | ------------------------------------------- | --------------------------------------------------------------- |
| 1 | **Workspace** | Git repository | Tests are TypeScript files in a Git repo |
| 2 | **TestCase** | `test('name', async () => {...})` | A single test function in a `.test.ts` file |
| 3 | **TestStep** | `test.step('name', async () => {...})` | Collapsible step in HTML report |
| 4 | **TestStepValue** | Variable / parameter | `const vendor = '100001';` |
| 5 | **Module** | Playwright fixture | `async ({ ui5, fe }) => {...}` |
| 6 | **ModuleAttribute** | Fixture property / method | `ui5.control()`, `fe.listReport.search()` |
| 7 | **TestCaseDesign (TCD)** | Test data fixture | `testData.generate({ vendor: '{{uuid}}' })` |
| 8 | **TestConfiguration** | `playwright.config.ts` + `praman.config.ts` | Two config files: runner + SAP-specific |
| 9 | **ExecutionList** | Playwright project | `projects: [{ name: 'chromium', ... }]` |
| 10 | **TestDataContainer** | `.env` file + `testData` fixture | Environment vars + template generation |
| 11 | **ActionMode Verify** | `expect()` assertion | `await expect(btn).toBeUI5Enabled()` |
| 12 | **ActionMode Input** | `ui5.fill()` / `ui5.select()` | `await ui5.fill({ id: 'vendorInput' }, '100001')` |
| 13 | **ActionMode Buffer** | Variable assignment | `const value = await control.getValue()` |
| 14 | **ActionMode WaitOn** | `ui5.control()` auto-wait | Built-in `waitForUI5Stable()` |
| 15 | **Steering Parameter** | Config option / env var | `PRAMAN_LOG_LEVEL=debug npx playwright test` |
| 16 | **Recovery Scenario** | `test.afterEach()` / fixture teardown | Auto-cleanup in fixtures (locks, test data) |
| 17 | **Scan (UI5 Engine)** | `UI5Selector` object | `{ controlType: 'sap.m.Button', properties: { text: 'Save' } }` |
| 18 | **XScan / TBox** | `page.locator()` (CSS/XPath) | For non-UI5 elements; `ui5.control()` for UI5 |
| 19 | **Library** | npm package / utility module | Shared helpers via `import` |
| 20 | **Requirements** | Test annotations / tags | `test('PO @smoke', ...)` + `--grep @smoke` |
| 21 | **Execution log** | Playwright HTML report | `npx playwright show-report` |
| 22 | **Screenshots** | Automatic screenshots | `screenshot: 'only-on-failure'` in config |
| 23 | **Distributed Execution** | `workers` config | `workers: 4` in `playwright.config.ts` |
| 24 | **Tosca Commander** | VS Code / terminal | IDE + `npx playwright test` |
| 25 | **DEX Agent** | CI runner (GitHub Actions, Jenkins) | `npx playwright test` in pipeline YAML |
## Tosca TestCase to Playwright Test
### Tosca TestCase (Conceptual)
```text
TestCase: Create Purchase Order
TestStep: Login
Module: SAP_Login
Username = {TC_User}
Password = {TC_Password}
TestStep: Navigate to Create PO
Module: FLP_Navigation
Tile = "Create Purchase Order"
TestStep: Fill Header Data
Module: PO_Header
Vendor = {TC_Vendor}
PurchaseOrg = {TC_PurchOrg}
CompanyCode = {TC_CompCode}
TestStep: Add Item
Module: PO_Item
Material = {TC_Material}
Quantity = {TC_Quantity}
Plant = {TC_Plant}
TestStep: Save
Module: PO_Footer
Action = Save
TestStep: Verify Success
Module: PO_Message
MessageType = Success [Verify]
```
### Praman Equivalent
```typescript
// Test data replaces Tosca TestCaseDesign / TestDataContainer
const testInput = {
vendor: '100001',
purchaseOrg: '1000',
companyCode: '1000',
material: 'MAT-001',
quantity: '10',
plant: '1000',
};
test('create purchase order', async ({ ui5, ui5Navigation, fe, ui5Footer }) => {
// TestStep: Navigate (login handled by setup project -- no code needed)
await test.step('Navigate to Create PO', async () => {
await ui5Navigation.navigateToTile('Create Purchase Order');
});
// TestStep: Fill Header Data
await test.step('Fill header data', async () => {
await ui5.fill({ id: 'vendorInput' }, testInput.vendor);
await ui5.fill({ id: 'purchOrgInput' }, testInput.purchaseOrg);
await ui5.fill({ id: 'compCodeInput' }, testInput.companyCode);
});
// TestStep: Add Item
await test.step('Add line item', async () => {
await ui5.fill({ id: 'materialInput' }, testInput.material);
await ui5.fill({ id: 'quantityInput' }, testInput.quantity);
await ui5.fill({ id: 'plantInput' }, testInput.plant);
});
// TestStep: Save (replaces Tosca PO_Footer module)
await test.step('Save', async () => {
await ui5Footer.clickSave();
});
// TestStep: Verify (replaces Tosca ActionMode Verify)
await test.step('Verify success message', async () => {
const message = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
searchOpenDialogs: true,
});
await expect(message).toBeUI5Visible();
});
});
```
## Intent API and SAP Transaction Codes
Praman's Intent API provides high-level business operations that map to SAP transaction codes
and Fiori apps.
| SAP TCode | Fiori App | Praman Intent API |
| --------- | ----------------------- | ---------------------------------------------- |
| ME21N | Create Purchase Order | `intent.procurement.createPurchaseOrder()` |
| ME23N | Display Purchase Order | `intent.procurement.displayPurchaseOrder()` |
| VA01 | Create Sales Order | `intent.sales.createSalesOrder()` |
| VA03 | Display Sales Order | `intent.sales.displaySalesOrder()` |
| FB60 | Enter Vendor Invoice | `intent.finance.postVendorInvoice()` |
| FBL1N | Vendor Line Items | `intent.finance.displayVendorLineItems()` |
| CO01 | Create Production Order | `intent.manufacturing.createProductionOrder()` |
| MM01 | Create Material Master | `intent.masterData.createMaterial()` |
### Using the Intent API
```typescript
test('create PO via intent API', async ({ intent }) => {
await test.step('Create purchase order', async () => {
await intent.procurement.createPurchaseOrder({
vendor: '100001',
material: 'MAT-001',
quantity: 10,
plant: '1000',
});
});
await test.step('Post vendor invoice', async () => {
await intent.finance.postVendorInvoice({
vendor: '100001',
amount: 5000,
currency: 'EUR',
});
});
});
```
## Team Workflow Transition
### From Tosca Commander to Git + IDE
| Workflow | Tosca | Praman |
| ----------------- | ---------------------------- | --------------------------------------------------- |
| Create test | Tosca Commander GUI | Create `.test.ts` file in VS Code |
| Edit test | Tosca Module editor | Edit TypeScript in any IDE |
| Version control | Tosca workspace versioning | Git branches + pull requests |
| Code review | Not built-in | GitHub/GitLab PR reviews |
| Merge conflicts | Workspace conflicts (binary) | Text-based Git merge (readable diffs) |
| Share test assets | Export/import workspace | `npm install` shared packages |
| Collaborate | Workspace locks | Git branching (multiple people edit simultaneously) |
### Suggested Team Transition Plan
**Week 1-2: Foundation**
- Install Node.js, VS Code, and Playwright on developer machines
- Complete the [Playwright Primer](./playwright-primer) (2-3 hours per person)
- Set up a Git repository for the test project
**Week 3-4: First Tests**
- Convert 3-5 Tosca TestCases from a single Fiori app to Praman tests
- Set up authentication via setup projects
- Run tests locally and review the HTML report
**Week 5-6: CI Integration**
- Add `npx playwright test` to your CI/CD pipeline
- Configure `screenshot: 'only-on-failure'` and `trace: 'on-first-retry'`
- Set up environment variables for SAP credentials in CI secrets
**Week 7-8: Scale**
- Convert remaining Tosca TestCases in priority order
- Adopt Fiori Elements helpers (`fe.listReport`, `fe.objectPage`) for standard Fiori apps
- Enable parallel execution with 2-4 workers
### Role Mapping
| Tosca Role | Praman Equivalent |
| ------------------- | ------------------------------------------------------- |
| Test Designer | Test Engineer (writes `.test.ts` files) |
| Test Data Manager | Uses `testData` fixture + `.env` files |
| Automation Engineer | Same role, now using TypeScript + Playwright |
| Test Manager | Reviews PRs, reads Playwright HTML reports, monitors CI |
| Tosca Administrator | Not needed (config is in Git, CI handles execution) |
## Handling Tosca-Specific Patterns
### Recovery Scenarios
Tosca Recovery Scenarios handle unexpected dialogs or errors. In Praman, use fixture teardown
and `test.afterEach()`.
```typescript
// Automatic recovery: fixtures clean up on teardown
// - flpLocks: deletes stale SM12 locks
// - testData: removes generated test data
// - sapAuth: logs out
// Manual recovery for edge cases
test.afterEach(async ({ ui5 }) => {
// Dismiss any open dialogs
try {
const okButton = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'OK' },
searchOpenDialogs: true,
});
await okButton.press();
} catch {
// No dialog open -- nothing to dismiss
}
});
```
### Test Data Parameterization
Tosca TestCaseDesign provides data-driven testing. In Playwright, use arrays or CSV files.
```typescript
// Data-driven test (replaces Tosca TCD)
const vendors = [
{ id: '100001', name: 'Vendor A', country: 'US' },
{ id: '100002', name: 'Vendor B', country: 'DE' },
{ id: '100003', name: 'Vendor C', country: 'JP' },
];
for (const vendor of vendors) {
test(`create PO for ${vendor.name}`, async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-create');
await ui5.fill({ id: 'vendorInput' }, vendor.id);
// ... rest of test
});
}
```
### Reusable Modules
Tosca Modules are reusable test components. In Praman, extract shared logic into helper
functions or custom fixtures.
```typescript
// helpers/po-helpers.ts
export async function fillPOHeader(
ui5: ExtendedUI5Handler,
data: { vendor: string; purchaseOrg: string; companyCode: string },
): Promise {
await ui5.fill({ id: 'vendorInput' }, data.vendor);
await ui5.fill({ id: 'purchOrgInput' }, data.purchaseOrg);
await ui5.fill({ id: 'compCodeInput' }, data.companyCode);
}
```
```typescript
// tests/purchase-order.test.ts
test('create PO', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-create');
await fillPOHeader(ui5, { vendor: '100001', purchaseOrg: '1000', companyCode: '1000' });
// ...
});
```
## Quick Reference Card
| I want to... | Tosca | Praman |
| ------------------- | --------------------------------- | -------------------------------------------------------- |
| Find a UI5 control | Scan / Module with UI5 engine | `ui5.control({ controlType: '...', properties: {...} })` |
| Click a button | ActionMode Input on Button module | `await ui5.click({ id: 'myBtn' })` |
| Fill a text field | ActionMode Input on Input module | `await ui5.fill({ id: 'myInput' }, 'value')` |
| Verify a value | ActionMode Verify | `await expect(control).toHaveUI5Text('value')` |
| Buffer/read a value | ActionMode Buffer | `const val = await control.getValue()` |
| Wait for page ready | WaitOn / Recovery | Automatic `waitForUI5Stable()` |
| Navigate to app | Browser module + URL | `await ui5Navigation.navigateToApp('App-action')` |
| Take screenshot | Screenshot step | `await page.screenshot({ path: 'file.png' })` |
| Run in CI | Tosca CI adapter + DEX | `npx playwright test` in any CI |
| Parameterize data | TestCaseDesign (TCD) | `for (const row of data) { test(...) }` |
| Handle errors | Recovery Scenario | Fixture teardown + `test.afterEach()` |
---
## From Selenium WebDriver
Migrating from Selenium WebDriver (Java, Python, C#, or JavaScript) to Playwright + Praman for
SAP Fiori and UI5 web application testing. This guide maps every core Selenium concept to its
Praman equivalent, compares configuration, and provides a step-by-step migration path.
## Quick Comparison
| Aspect | Selenium WebDriver | Praman (Playwright) |
| ------------------ | -------------------------------------------------- | ------------------------------------------------------- |
| Protocol | WebDriver (W3C) over HTTP | Chrome DevTools Protocol (CDP) / BiDi |
| Selector model | `By.id`, `By.xpath`, `By.css`, `By.className` | `UI5Selector` object (controlType, properties, binding) |
| UI5 awareness | None (raw DOM only) | Native UI5 control registry access |
| Auto-waiting | None (manual `WebDriverWait` + ExpectedConditions) | Built-in `waitForUI5Stable()` + auto-retry |
| Parallel execution | Selenium Grid / TestNG / pytest-xdist | Native Playwright workers |
| CI/CD integration | Requires Grid or cloud service | Zero infrastructure (`npx playwright test`) |
| Browser install | Manual driver management (ChromeDriver) | `npx playwright install` (auto-managed) |
| Language | Java, Python, C#, JavaScript, Ruby, Kotlin | TypeScript (first-class) |
| Test runner | JUnit, TestNG, pytest, NUnit, Mocha | Playwright Test (built-in) |
| Auth management | Manual login scripts | 6 built-in strategies + setup projects |
| Reporting | Allure, ExtentReports (third-party) | Built-in HTML, JSON, JUnit reporters |
| AI integration | None | Built-in capabilities, recipes, agentic handler |
| OData helpers | None | Model-level + HTTP-level CRUD |
| FLP navigation | Manual URL construction | 9 typed navigation methods |
| Trace / debug | Screenshots only | Full trace (DOM snapshots, network, console, video) |
| Page Object Model | Class-based POMs | Playwright fixtures (dependency injection) |
## Core API Mapping
### Navigation, Discovery, Interaction, and Assertions
```java
// Selenium (Java)
driver.get("https://sap-system.example.com/.../FioriLaunchpad.html#PurchaseOrder-manage");
WebElement button = driver.findElement(By.id("__xmlview0--saveBtn"));
WebElement input = driver.findElement(By.xpath("//input[@aria-label='Vendor']"));
button.click();
input.sendKeys("100001");
String text = driver.findElement(By.id("__xmlview0--statusText")).getText();
assertEquals("Active", text);
assertTrue(driver.findElement(By.id("__xmlview0--saveBtn")).isDisplayed());
```
```typescript
// Praman
test('PO overview', async ({ ui5, ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Save' } });
await ui5.fill({ id: 'vendorInput' }, '100001');
const status = await ui5.control({ id: 'statusText' });
await expect(status).toHaveUI5Text('Active');
await expect(await ui5.control({ id: 'saveBtn' })).toBeUI5Visible();
});
```
### Waiting for Elements
```java
// Selenium (Java) -- manual explicit waits
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30));
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("__xmlview0--poTable")));
Thread.sleep(5000); // common anti-pattern
```
```typescript
// Praman -- no explicit waits needed
// ui5.control() automatically waits for UI5 bootstrap, pending OData requests,
// JS timeouts/intervals, and DOM stabilization before returning
const table = await ui5.control({ controlType: 'sap.m.Table', id: 'poTable' });
await expect(table).toHaveUI5RowCount({ min: 1 });
```
## Selector Mapping
Selenium selectors target raw DOM elements. Praman selectors target the UI5 control registry,
which survives UI5 version upgrades, theme changes, and DOM restructuring.
| Selenium Selector | Praman `UI5Selector` Equivalent | Notes |
| ------------------------------------ | ------------------------------------------------------------------------ | ---------------------------------------------- |
| `By.id("__xmlview0--myBtn")` | `{ id: 'myBtn' }` | Praman strips view prefixes automatically |
| `By.xpath("//button[@text='Save']")` | `{ controlType: 'sap.m.Button', properties: { text: 'Save' } }` | Property matching replaces XPath |
| `By.cssSelector(".sapMBtn")` | `{ controlType: 'sap.m.Button' }` | Control type replaces CSS class matching |
| `By.className("sapMInputBaseInner")` | `{ controlType: 'sap.m.Input', id: 'myInput' }` | Target the control, not its inner DOM |
| `By.linkText("Purchase Orders")` | `{ controlType: 'sap.m.Link', properties: { text: 'Purchase Orders' } }` | Text property replaces link text |
| `By.xpath("//div[@data-path]")` | `{ bindingPath: { value: '/Vendor/Name' } }` | OData binding path matching |
| `By.xpath` with `contains()` | `{ id: /partialMatch/ }` | RegExp ID matching |
| `By.xpath` with ancestor axis | `{ ancestor: { controlType: 'sap.m.Panel', id: 'headerPanel' } }` | Ancestor selector replaces XPath ancestor axis |
| N/A | `{ searchOpenDialogs: true }` | Dialog-aware search (no Selenium equivalent) |
:::tip Generated IDs
Selenium tests for SAP UI5 apps commonly break because UI5 generates DOM IDs like
`__xmlview0--__button3`. These IDs change across page reloads, UI5 versions, and theme switches.
Praman's `UI5Selector` bypasses DOM IDs entirely by querying the UI5 control registry.
:::
## Config Mapping
### Selenium Capabilities (Java)
```java
ChromeOptions options = new ChromeOptions();
options.addArguments("--headless", "--window-size=1920,1080");
WebDriver driver = new ChromeDriver(options);
driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
driver.manage().timeouts().pageLoadTimeout(Duration.ofSeconds(60));
```
### Praman Config (playwright.config.ts + praman.config.ts)
```typescript
// playwright.config.ts
export default defineConfig({
testDir: './tests',
timeout: 120_000,
retries: 1,
workers: process.env.CI ? 2 : 1,
use: {
baseURL: process.env.SAP_BASE_URL,
headless: true, // --headless
viewport: { width: 1920, height: 1080 }, // --window-size
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
projects: [
{ name: 'setup', testMatch: /auth-setup\.ts/, teardown: 'teardown' },
{ name: 'teardown', testMatch: /auth-teardown\.ts/ },
{
name: 'chromium',
dependencies: ['setup'],
use: { ...devices['Desktop Chrome'], storageState: '.auth/sap-session.json' },
},
],
});
```
```typescript
// praman.config.ts
export default defineConfig({
logLevel: 'info',
ui5WaitTimeout: 30_000, // Replaces implicitlyWait + WebDriverWait
controlDiscoveryTimeout: 10_000,
interactionStrategy: 'ui5-native',
auth: {
strategy: 'basic',
baseUrl: process.env.SAP_BASE_URL!,
username: process.env.SAP_USER!,
password: process.env.SAP_PASS!,
},
});
```
## Page Object Model to Fixtures
Selenium teams almost universally use the Page Object Model. Praman replaces class-based POMs
with Playwright's fixture system -- dependency injection with automatic teardown.
### Selenium Page Object (Java)
```java
public class PurchaseOrderPage {
@FindBy(id = "__xmlview0--vendorInput-inner") private WebElement vendorInput;
@FindBy(id = "__xmlview0--saveBtn") private WebElement saveButton;
@FindBy(css = ".sapMMsgStrip") private WebElement messageStrip;
public PurchaseOrderPage(WebDriver d) { PageFactory.initElements(d, this); }
public void fillVendor(String v) { vendorInput.clear(); vendorInput.sendKeys(v); }
public void save() { saveButton.click(); }
public String getMessage() { return messageStrip.getText(); }
}
// Test
PurchaseOrderPage po = new PurchaseOrderPage(driver);
po.fillVendor("100001");
po.save();
assertEquals("PO created", po.getMessage());
```
### Praman Fixture Equivalent
```typescript
test('create purchase order', async ({ ui5, ui5Navigation, ui5Footer }) => {
await test.step('Navigate', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-create');
});
await test.step('Fill vendor', async () => {
await ui5.fill({ id: 'vendorInput' }, '100001');
});
await test.step('Save', async () => {
await ui5Footer.clickSave();
});
await test.step('Verify success', async () => {
const message = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
searchOpenDialogs: true,
});
await expect(message).toBeUI5Visible();
});
});
```
For reusable logic across tests, extract helper functions instead of POM classes.
```typescript
// helpers/po-helpers.ts
export async function fillPOHeader(
ui5: ExtendedUI5Handler,
data: { vendor: string; purchaseOrg: string; companyCode: string },
): Promise {
await ui5.fill({ id: 'vendorInput' }, data.vendor);
await ui5.fill({ id: 'purchOrgInput' }, data.purchaseOrg);
await ui5.fill({ id: 'compCodeInput' }, data.companyCode);
}
```
## Hybrid Approach During Migration
You do not need to rewrite everything at once. Praman's `test` and `expect` are extended versions
of Playwright's -- `page.locator()` works alongside `ui5.control()` in the same test.
```typescript
test('hybrid test', async ({ page, ui5, ui5Navigation }) => {
// Playwright handles login (before UI5 loads)
await test.step('Login', async () => {
await page.goto(process.env.SAP_BASE_URL!);
await page.locator('#USERNAME_FIELD input').fill('TESTUSER');
await page.locator('#PASSWORD_FIELD input').fill('secret');
await page.locator('#LOGIN_LINK').click();
await page.waitForURL('**/FioriLaunchpad*');
});
// Praman takes over once UI5 is loaded
await test.step('Navigate and verify', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await expect(page).toHaveTitle(/Purchase Orders/);
const table = await ui5.control({ controlType: 'sap.m.Table', id: 'poTable' });
await expect(table).toHaveUI5RowCount({ min: 1 });
});
});
```
## New Features Only in Praman
Praman includes capabilities that have no Selenium equivalent. These are worth adopting early.
### 6 Built-In Auth Strategies
No more custom login scripts or cookie injection. Praman supports Basic, BTP SAML, Office 365,
Custom, API, and Certificate authentication out of the box.
```typescript
test('after auth', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
});
```
### 9 FLP Navigation Methods
```typescript
test('navigation', async ({ ui5Navigation }) => {
await ui5Navigation.navigateToApp('PurchaseOrder-manage');
await ui5Navigation.navigateToTile('Create Purchase Order');
await ui5Navigation.navigateToIntent('PurchaseOrder', 'create', { plant: '1000' });
await ui5Navigation.navigateToHome();
await ui5Navigation.navigateBack();
});
```
### Fiori Elements Helpers
Dedicated APIs for List Report and Object Page patterns -- no more fragile XPath chains.
```typescript
test('FE list report', async ({ fe }) => {
await fe.listReport.setFilter('Status', 'Active');
await fe.listReport.search();
await fe.listReport.navigateToItem(0);
await fe.objectPage.clickEdit();
await fe.objectPage.clickSave();
});
```
### AI-Powered Test Generation
```typescript
test('AI discovery', async ({ pramanAI }) => {
const context = await pramanAI.discoverPage({ interactiveOnly: true });
const result = await pramanAI.agentic.generateTest(
'Create a purchase order for vendor 100001',
page,
);
});
```
### Custom UI5 Matchers
10 UI5-specific Playwright matchers -- replacing verbose Selenium assert chains.
```typescript
const button = await ui5.control({ id: 'saveBtn' });
await expect(button).toBeUI5Enabled();
await expect(button).toHaveUI5Text('Save');
await expect(button).toBeUI5Visible();
```
### OData Model + HTTP Operations
```typescript
test('OData', async ({ ui5 }) => {
const data = await ui5.odata.getModelData('/PurchaseOrders');
// HTTP-level CRUD with automatic CSRF handling
});
```
### SM12 Lock Management
```typescript
test('locks', async ({ flpLocks }) => {
await flpLocks.deleteAllLockEntries('TESTUSER');
// Auto-cleanup on teardown
});
```
### Structured Error Codes
Every Praman error includes a machine-readable `code`, `retryable` flag, and `suggestions[]`
array for self-healing tests. Selenium errors are raw `WebDriverException` messages with stack
traces but no recovery guidance.
## Step-by-Step Migration Checklist
1. **Install dependencies**: `npm install playwright-praman @playwright/test`
2. **Install browsers**: `npx playwright install`
3. **Create config files**: `praman.config.ts` and `playwright.config.ts` (see Config Mapping)
4. **Set up auth**: Create `tests/auth-setup.ts` using one of the 6 strategies
5. **Convert selectors**: Replace `By.id()` / `By.xpath()` / `By.css()` with `UI5Selector` objects
6. **Remove explicit waits**: Delete all `WebDriverWait`, `Thread.sleep()`, `implicitlyWait` -- Praman auto-waits
7. **Convert Page Objects**: Replace POM classes with fixture destructuring and helper functions
8. **Convert assertions**: Replace JUnit/TestNG `assertEquals` with Playwright `expect` + UI5 matchers
9. **Structure tests**: Convert `@Test` methods to `test()` + `test.step()` blocks
10. **Adopt navigation**: Replace `driver.get()` URL strings with `ui5Navigation` methods
11. **Remove driver management**: Delete ChromeDriver setup, Grid config, and teardown code
12. **Run and verify**: `npx playwright test`
## Team Workflow Transition
### From Java/Maven/Grid to TypeScript/npm/Playwright
| Workflow | Selenium Stack | Praman Stack |
| ------------------ | ------------------------------------- | --------------------------------------------------- |
| Language | Java (+ Kotlin, Python, C#) | TypeScript |
| Build tool | Maven / Gradle | npm |
| Test runner | JUnit / TestNG | Playwright Test (built-in) |
| Test structure | `@Test` annotated methods in classes | `test()` functions in `.test.ts` files |
| Page Objects | Java classes with `@FindBy` | Playwright fixtures + helper functions |
| Driver management | WebDriverManager / ChromeDriver | `npx playwright install` (fully automatic) |
| Parallel execution | Selenium Grid + TestNG parallel suite | `workers: 4` in `playwright.config.ts` |
| CI/CD | Grid server + nodes + Docker | `npx playwright test` (zero infrastructure) |
| Reporting | Allure / ExtentReports plugin | Built-in HTML report (`npx playwright show-report`) |
| Environment config | Maven profiles / `testng.xml` | `.env` files + `playwright.config.ts` projects |
| Version control | Git (test code) + driver binaries | Git (everything, no binaries) |
| IDE | IntelliJ / Eclipse | VS Code (recommended) / any IDE |
### Suggested Team Transition Plan
**Week 1-2: Foundation**
- Install Node.js, VS Code, and Playwright on developer machines
- Complete the [Playwright Primer](./playwright-primer) (2-3 hours per person)
- Set up a Git repository for the test project
**Week 3-4: First Tests**
- Convert 3-5 Selenium test classes from a single Fiori app to Praman tests
- Set up authentication via setup projects (replaces Selenium login helpers)
- Run tests locally and review the HTML report
**Week 5-6: CI Integration**
- Add `npx playwright test` to your CI/CD pipeline (replaces Grid + Maven Surefire)
- Configure `screenshot: 'only-on-failure'` and `trace: 'on-first-retry'`
- Set up environment variables for SAP credentials in CI secrets
**Week 7-8: Scale**
- Convert remaining Selenium test suites in priority order
- Adopt Fiori Elements helpers (`fe.listReport`, `fe.objectPage`) for standard Fiori apps
- Enable parallel execution with 2-4 workers (replaces Selenium Grid)
- Decommission Selenium Grid infrastructure
### Role Mapping
| Selenium Role | Praman Equivalent |
| -------------------------- | ------------------------------------------------------- |
| SDET / Automation Engineer | Test Engineer (writes `.test.ts` files) |
| Grid Administrator | Not needed (no infrastructure to manage) |
| Driver / Browser Manager | Not needed (`npx playwright install`) |
| Test Architect (POMs) | Same role, designs fixtures and helper modules |
| QA Lead / Manager | Reviews PRs, reads Playwright HTML reports, monitors CI |
---
## For SAP Business Analysts
You know SAP business processes inside out. You can walk through a Purchase-to-Pay cycle
in your sleep, spot a missing three-way match before anyone else, and explain why a particular
approval workflow exists. This guide shows how that knowledge translates directly into
automated test coverage -- without requiring you to become a programmer.
## You Don't Need to Code
Praman includes an AI-powered workflow that turns plain-language business process descriptions
into executable Playwright tests. Here is how it works:
1. **You describe the process.** Write a business scenario the way you would explain it to a
colleague: "Create a purchase order for vendor 100001 with material MAT-001, quantity 10,
plant 1000. Save it and confirm the success message shows a PO number."
2. **The AI agent generates the test.** Praman's agent reads your description, opens the SAP
Fiori app in a real browser, discovers the UI controls, and writes a complete Playwright test.
3. **The test engineer reviews and commits.** Your team's test engineer validates the generated
test, adjusts selectors if needed, and adds it to version control.
4. **The test runs automatically.** Every time someone deploys a change, CI/CD runs the test
and reports pass or fail -- with screenshots on failure.
You stay in your comfort zone (business process knowledge), and the tooling handles the
technical translation.
### The AI Command
When working with the Praman agent, a single command kicks off the full pipeline:
```text
/praman-sap-coverage
```
This command tells the agent to:
- **Plan**: Analyze the SAP app and identify testable scenarios
- **Generate**: Write Playwright tests for each scenario
- **Heal**: Fix any tests that fail on first run
You provide the business context. The agent handles the code.
## Your SAP Knowledge Is the Test Plan
Every piece of business knowledge you carry maps to a specific test automation artifact.
| What You Know | How It Becomes Test Automation |
| ------------------------------------------ | -------------------------------------------- |
| Business processes (P2P, O2C, R2R) | End-to-end test scenarios |
| SAP transactions (ME21N, VA01, FB60) | Fiori app navigation via transaction mapping |
| Expected outcomes and golden rules | Assertion criteria (pass/fail checks) |
| Edge cases and business exceptions | Negative test coverage |
| Compliance and audit requirements | Automated evidence collection |
| Approval workflows and authorization roles | Role-based test configurations |
| Master data dependencies | Test data setup templates |
| Tolerance limits and thresholds | Boundary value test cases |
When a BA says "the PO should not save without a valid vendor," that is already a test
specification. Praman turns it into an executable check.
## How a Test Gets Created
There are three paths from business requirement to running test. Your team will likely use
all three depending on the situation.
### Path A: AI Agent Generates from Your Description
You write a natural-language scenario. The agent does the rest.
**Your description:**
> Create a purchase order in the Manage Purchase Orders app. Use vendor 100001, purchasing
> organization 1000, company code 1000. Add one item: material MAT-001, quantity 10, plant 1000. Save and verify the success message contains a PO number.
**What the agent produces:**
A complete, runnable Playwright test file with Praman fixtures, proper navigation, field
interactions, and assertions. The test engineer on your team reviews and approves it.
### Path B: Test Engineer Writes from Your Functional Spec
You provide a structured specification. A test engineer translates it into code.
**Your spec:**
| Step | Action | Expected Result |
| ---- | -------------------------- | ----------------------------------- |
| 1 | Open Create Purchase Order | App loads, header fields are empty |
| 2 | Enter Vendor: 100001 | Vendor name auto-populates |
| 3 | Enter PurchOrg: 1000 | Field accepted |
| 4 | Enter Material: MAT-001 | Material description auto-populates |
| 5 | Enter Quantity: 10 | Field accepted |
| 6 | Click Save | Success message with PO number |
**The test engineer maps each row to a `test.step()`.** The resulting test mirrors your
spec line by line.
### Path C: Record and Replay
A tester walks through the scenario in a browser while Praman records the interactions.
The recording is converted into a test file that can be edited and maintained. This approach
works well for complex workflows where describing every click would be tedious.
## Reading Test Results
After tests run, you get results in several formats. Here is what to look for.
### Playwright HTML Report
Open it with `npx playwright show-report`. The report shows:
- **Pass/Fail/Skip counts** at the top -- a quick health check
- **Each test** as a collapsible row. Green means pass, red means fail
- **Test steps** inside each test (the `test.step()` blocks). These match your spec steps
- **Screenshots on failure** -- an automatic browser screenshot captured at the moment of failure
- **Traces on retry** -- a full recording of every action, network call, and DOM change
When a test fails, the screenshot and trace tell you _exactly_ what the user would have seen.
You do not need to reproduce the issue manually.
### Business Process Heatmaps
When multiple tests cover a single business process (for example, P2P), Praman's reporter
can generate a heatmap showing which process steps are covered and which are not. This helps
you identify gaps in test coverage and prioritize new scenarios.
### Compliance Evidence Artifacts
Every test run produces timestamped artifacts: screenshots, trace files, and structured
JSON results. These serve as audit evidence that a process was validated at a specific point
in time. For SOX, GxP, or internal audit requirements, these artifacts can be archived
alongside your change documentation.
## SAP Transaction to Fiori App to Test Mapping
This table maps common SAP transactions to their Fiori equivalents and shows how Praman
tests interact with each.
| SAP TCode | Description | Fiori App / Intent | Praman Test Approach |
| --------- | ------------------------ | -------------------------------------- | --------------------------------------------- |
| ME21N | Create Purchase Order | `PurchaseOrder-create` | `ui5Navigation` + `feObjectPage.fillField()` |
| ME23N | Display Purchase Order | `PurchaseOrder-display` | `feListReport` search + detail verification |
| VA01 | Create Sales Order | `SalesOrder-create` | `ui5Navigation` + `feObjectPage.fillField()` |
| VA03 | Display Sales Order | `SalesOrder-display` | `feListReport` search + field assertions |
| FB60 | Enter Vendor Invoice | `SupplierInvoice-create` | `ui5.fill()` for SmartFields + post action |
| FBL1N | Vendor Line Items | `SupplierLineItem-analyzeJournalEntry` | `feListReport` filters + row count assertions |
| CO01 | Create Production Order | `ProductionOrder-create` | `ui5Navigation` + header/operation fill |
| MM01 | Create Material Master | `Material-create` | Multi-tab navigation + field fill per view |
| MIGO | Goods Movement | `GoodsReceipt-create` | PO reference + item verification + post |
| VL01N | Create Outbound Delivery | `OutboundDelivery-create` | SO reference + picking quantity + post GI |
| MIRO | Invoice Verification | `SupplierInvoice-verify` | PO reference + amount match + post |
| F-28 | Incoming Payment | `IncomingPayment-post` | Customer + invoice reference + clearing |
For the full mapping reference, see the [Transaction Mapping](./transaction-mapping) guide.
## What a Test Looks Like
Below is a complete test for creating a Purchase Order. Every line is commented to explain
what it does in business terms.
```typescript
// This line loads the Praman test framework.
// Think of it as "opening the test toolkit."
// Test data — the values we will enter into the PO form.
// Keeping them here makes it easy to change without touching the test logic.
const poData = {
vendor: '100001', // SAP vendor number
purchaseOrg: '1000', // Purchasing organization
companyCode: '1000', // Company code
material: 'MAT-001', // Material number for the line item
quantity: '10', // Order quantity
plant: '1000', // Receiving plant
};
// This defines a single test. The text in quotes is the test name
// that appears in reports.
test('create a standard purchase order', async ({
ui5, // Tool for finding and interacting with SAP UI5 controls
ui5Navigation, // Tool for navigating between SAP Fiori apps
ui5Footer, // Tool for clicking footer buttons (Save, Edit, Cancel)
}) => {
// Step 1: Navigate to the Create Purchase Order app.
// This is equivalent to entering transaction ME21N in SAP GUI,
// but for the Fiori version of the app.
await test.step('Navigate to Create Purchase Order', async () => {
await ui5Navigation.navigateToApp('PurchaseOrder-create');
});
// Step 2: Fill in the PO header fields.
// Each ui5.fill() call finds a field on screen and types a value into it.
await test.step('Fill header data', async () => {
await ui5.fill({ id: 'vendorInput' }, poData.vendor);
await ui5.fill({ id: 'purchOrgInput' }, poData.purchaseOrg);
await ui5.fill({ id: 'compCodeInput' }, poData.companyCode);
});
// Step 3: Add a line item to the PO.
await test.step('Add line item', async () => {
await ui5.fill({ id: 'materialInput' }, poData.material);
await ui5.fill({ id: 'quantityInput' }, poData.quantity);
await ui5.fill({ id: 'plantInput' }, poData.plant);
});
// Step 4: Click the Save button in the footer bar.
await test.step('Save the purchase order', async () => {
await ui5Footer.clickSave();
});
// Step 5: Verify that SAP shows a success message.
// This is the automated equivalent of visually checking
// "Purchase Order 4500000123 created" on screen.
await test.step('Verify success message', async () => {
const message = await ui5.control({
controlType: 'sap.m.MessageStrip',
properties: { type: 'Success' },
searchOpenDialogs: true,
});
await expect(message).toBeUI5Visible();
});
});
```
When this test runs, the HTML report shows five collapsible steps -- one per `test.step()`
block. If step 4 fails, you see a screenshot of the exact screen state at the moment of
failure, so you can immediately tell whether it was a data issue, a missing authorization,
or an application bug.
## Your Role in the Team
You do not need to write code to add value to test automation. Here is how your existing
activities map to the testing workflow.
| Your Current Activity | Test Automation Contribution | Artifact You Produce |
| ----------------------------------- | ------------------------------------------ | ------------------------------------- |
| Write functional specifications | Define test scenarios and expected results | Scenario descriptions or spec tables |
| Validate UAT results | Review Playwright HTML reports | Pass/fail sign-off |
| Document business rules | Define assertion criteria | Expected value lists, tolerance rules |
| Identify edge cases and exceptions | Define negative test cases | Edge case catalog |
| Manage test data in spreadsheets | Configure test data templates | Data sets for parameterized tests |
| Map process flows (Visio, Signavio) | Define end-to-end test chains | Process flow to test step mapping |
| Verify compliance requirements | Review audit evidence artifacts | Compliance checklist sign-off |
| Triage production defects | Prioritize regression test additions | Defect-to-test-case traceability |
The most effective SAP test automation teams pair a Business Analyst with a Test Engineer.
The BA defines _what_ to test and _what the expected outcome is_. The engineer handles _how_
it gets automated.
## Glossary of Technical Terms
These terms come up frequently in test automation conversations. Each is explained using
SAP-familiar analogies.
| Term | Definition |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **TypeScript** | The programming language tests are written in. Think of it as ABAP for the browser -- a typed, structured language that catches errors before runtime. |
| **npm** | The package manager for JavaScript/TypeScript projects. Similar to how you install transports in SAP -- `npm install` brings in the libraries your project needs. |
| **Fixture** | A pre-configured helper object injected into your test automatically. Similar to an ECATT variant that provides test data and tools -- you request `ui5` or `fe` in your test signature and they are ready to use. |
| **Assertion** | A check that verifies an expected outcome. The `expect()` function is the automated equivalent of a UAT tester visually confirming "the PO number appears on screen." |
| **Selector** | The address of a UI element on screen. Like a field technical name in SAP (e.g., `EKKO-EBELN`), a selector tells Praman which control to interact with. |
| **CI/CD** | Continuous Integration / Continuous Delivery. An automated pipeline that runs tests every time code changes are deployed -- like a transport release process that includes automatic quality checks. |
| **Git** | Version control for code files. Similar to SAP transport management -- it tracks who changed what, when, and why. Developers work in "branches" (like transport requests) and merge them into the main codebase. |
| **HTML Report** | The test results page generated by Playwright. Open it in any browser to see pass/fail status, screenshots, and step-by-step traces for every test. |
| **Test Step** | A named block within a test (`test.step()`). Corresponds to a row in your functional spec -- "Step 3: Enter vendor number." Steps appear as collapsible sections in the report. |
| **Headed Mode** | Running the browser visibly on screen so you can watch the test execute. Useful for demos and debugging. In CI/CD, tests run "headless" (no visible browser) for speed. |
## Next Steps
Ready to go deeper? Here is a suggested reading path.
1. **[Getting Started](./getting-started)** -- Install Praman and run your first test
2. **[Playwright Primer](./playwright-primer)** -- Ground-up introduction for non-programmers
3. **[Business Process Examples](./business-process-examples)** -- Complete P2P, O2C, and R2R test examples
4. **[Running Your Agent](./running-your-agent)** -- Set up and run the AI-powered test generation agent
5. **[Transaction Mapping](./transaction-mapping)** -- Full SAP transaction to Fiori app mapping reference
---
## Running Your Agent for the First Time
From business process to Playwright test — autonomously. This guide walks you through the complete flow of using Praman's AI agents to discover an SAP application, generate a test plan, produce a gold-standard test script, and iterate until it passes.
## Prerequisites Checklist
Before launching your agent, verify the following files exist in your project. All paths are relative to your project root.
### Agent Definitions
```text
.github/agents/praman-sap-planner.agent.md
.github/agents/praman-sap-generator.agent.md
.github/agents/praman-sap-healer.agent.md
```
:::warning Missing agents?
If these files don't exist, run `npx playwright init-agents --loop=vscode && npx playwright-praman init` (see [Agent & IDE Setup](./agent-setup)).
:::
### Skill Files
```text
.github/skills/sap-test-automation/SKILL.md
.github/skills/sap-test-automation/ai-quick-reference.md
```
These are read by all three agents. They ship inside the `playwright-praman` package and are copied by `init`.
### Copilot Instructions
Verify that `.github/copilot-instructions.md` contains the Praman section. If not, append it:
```bash
cat node_modules/playwright-praman/docs/user-integration/copilot-instructions-appendable.md >> .github/copilot-instructions.md
```
### Seed File
```text
tests/seeds/sap-seed.spec.ts
```
The seed authenticates against your SAP system and keeps the browser open for the MCP server. It uses raw Playwright (no Praman fixtures).
### Environment Variables
Create a `.env` file with your SAP credentials:
```bash
SAP_CLOUD_BASE_URL=https://your-system.s4hana.cloud.sap/
SAP_CLOUD_USERNAME=your-sap-user
SAP_CLOUD_PASSWORD=your-sap-password
```
:::danger
Never commit `.env` to version control. Add it to `.gitignore`.
:::
## The Complete Flow
```text
┌─────────────────────────────────────────────────────────┐
│ 1. PLAN → 2. GENERATE → 3. HEAL → 4. GOLD │
│ │
│ Planner Generator Healer Production │
│ explores converts fixes test ready │
│ live SAP plan to failing for CI/CD │
│ app test code tests │
└─────────────────────────────────────────────────────────┘
```
## Step 1: Launch the Planner Agent
Open **GitHub Copilot MCP Chat** (or Claude Code) and paste the following prompt. Replace the test case steps with your own business scenario.
### Prompt Template
```text
Goal: Create SAP test case and test script
1. Use praman SAP planner agent:
.github/agents/praman-sap-planner.agent.md
2. Login using credentials in .env file and use Chrome in headed mode.
Do not use sub-agents.
3. Ensure you use UI5 query and capture UI5 methods at each step.
Use UI5 methods for all control interactions.
4. Use seed file: tests/seeds/sap-seed.spec.ts
5. Here is the test case:
Login to SAP and ensure you are on the landing page.
Step 1: Navigate to Maintain Bill of Material app and click Create BOM
- expect: Create BOM dialog opens with all fields visible
Step 2: Select Material via Value Help — pick a valid material
- expect: Material field is populated with selected material
Step 3: Select Plant via Value Help — pick plant matching the material
- expect: Plant field is populated with selected plant
Step 4: Select BOM Usage "Production (1)" from dropdown
- expect: BOM Usage field shows "Production (1)"
Step 5: Verify all required fields are filled before submission
- expect: Material field has a value
- expect: BOM Usage field has value "Production (1)"
- expect: Valid From date is set
- expect: Create BOM button is enabled
Step 6: Click "Create BOM" submit button in dialog footer
- expect: If valid combination — dialog closes, BOM created,
user returns to list report
- expect: If invalid combination — error message dialog appears
Step 7: If error occurs, close error dialog and cancel
- expect: Error dialog closes
- expect: Create BOM dialog closes
- expect: User returns to list report
Output:
- Test plan: specs/
- Test script: tests/e2e/sap-cloud/
```
### What Happens Next
The agent will:
1. **Launch the MCP server** and open Chrome in headed mode
2. **Authenticate** using credentials from `.env` via the seed file
3. **Navigate** to the SAP Fiori Launchpad and open your target app
4. **Discover UI5 controls** — runs `browser_run_code` to query the UI5 control tree, capturing:
- Control IDs, types, and properties
- Value Help structures (inner tables, columns, sample data)
- MDC Field vs SmartField framework detection
- Required fields, button states, dialog structure
5. **Produce a test plan** in `specs/` with:
- Application overview (UI5 version, OData version, control framework)
- Full UI5 control map (every discovered ID, type, and value help)
- Test scenarios with step-by-step UI5 method references
6. **Generate the test script** in `tests/e2e/sap-cloud/`
## Step 2: Review the Test Plan Output
The planner produces a structured test plan. Here is what a typical plan looks like:
Example: bom-create-flow.plan.md (click to expand)
```markdown
# BOM Create Complete Flow — Test Plan
## Application Overview
- **App**: SAP S/4HANA Cloud - Maintain Bill of Material (Version 2)
- **Type**: Fiori Elements V4 List Report with Create BOM Action Parameter Dialog
- **UI5 Version**: 1.142.x
- **OData Version**: V4
- **Control Framework**: MDC (sap.ui.mdc) — NOT SmartField (sap.ui.comp)
## Discovery Date
[Auto-filled] — Live discovery via Playwright MCP + browser_run_code
## UI5 Control Map (Discovered)
### List Report — Toolbar
| Control | Type | Text |
| ---------- | ------------ | ---------- |
| Create BOM | sap.m.Button | Create BOM |
| Go | sap.m.Button | Go |
### Create BOM Dialog (sap.m.Dialog)
| Control | Type | Required |
| ----------------- | --------------------------- | -------- |
| Material (outer) | sap.ui.mdc.Field | true |
| Material (inner) | sap.ui.mdc.field.FieldInput | true |
| Plant (outer) | sap.ui.mdc.Field | false |
| Plant (inner) | sap.ui.mdc.field.FieldInput | false |
| BOM Usage (outer) | sap.ui.mdc.Field | true |
| BOM Usage (inner) | sap.ui.mdc.field.FieldInput | true |
| Valid From | sap.m.DatePicker | false |
| Create BOM (btn) | sap.m.Button | — |
| Cancel (btn) | sap.m.Button | — |
### Value Help: Material
| Property | Value |
| ----------- | ------------------------------ |
| VH Type | sap.ui.mdc.ValueHelp |
| Inner Table | sap.ui.table.Table |
| Columns | Material, Material Description |
### Value Help: Plant
| Property | Value |
| ----------- | --------------------------------- |
| VH Type | sap.ui.mdc.ValueHelp |
| Inner Table | sap.ui.table.Table |
| Columns | Plant, Plant Name, Valuation Area |
### Value Help: BOM Usage
| Property | Value |
| -------- | --------------------------------------------------------------------------------------- |
| Type | sap.ui.mdc.ValueHelp — Suggest popover (dropdown) |
| Options | Production (1), Engineering/Design (2), Universal (3), Plant Maintenance (4), Sales (5) |
## UI5 Methods Used (Praman Proxy)
| Method | Used For |
| ---------------------------- | -------------------------------------------- |
| ui5.control({ id }) | Find MDC Field, Button, Dialog by stable ID |
| ui5.press({ id }) | Click buttons (Create BOM, VH icons, Cancel) |
| ui5.fill({ id }, value) | Set field value (setValue + fireChange) |
| ui5.waitForUI5() | Wait for UI5 busyIndicator / OData calls |
| ui5.getValue({ id }) | Read MDC FieldInput display value |
| control.setValue(key) | Set MDC Field key programmatically |
| control.getProperty(prop) | Read text, enabled, title |
| control.getRequired() | Check required fields |
| control.isOpen() | Check ValueHelp/Dialog open state |
| control.close() | Close ValueHelp dialog |
| innerTable.getContextByIndex | Get OData binding from VH table row |
| ctx.getObject() | Extract entity data from binding context |
## Test Scenarios
### Scenario 1: Complete BOM Creation Flow
**Steps:**
1. Navigate to BOM App (FLP → tab → tile)
2. Open Create BOM Dialog — verify all fields
3. Select Material via Value Help
4. Select Plant via Value Help
5. Set BOM Usage to Production (1)
6. Verify all required fields
7. Submit Create BOM (handle success or error)
8. Verify return to List Report
### Scenario 2: Validation Error Handling
**Steps:**
1. Navigate & Open Dialog
2. Select an invalid Material + Plant combination
3. Set BOM Usage
4. Submit — verify error dialog appears
5. Close error dialog — verify Create BOM dialog preserved
6. Cancel — verify return to List Report
```
:::tip What to look for
Review the **UI5 Control Map** — it should list every control the planner discovered with correct IDs and types. If a control is missing, re-run the planner with a more specific test case description.
:::
## Step 3: Review the Generated Test Script
The generator produces a test script using `playwright-praman` fixtures. The output follows the gold standard format:
Example: bom-create-flow-gold.spec.ts structure (click to expand)
```typescript
// ── Control ID Constants (from discovery) ───────────────────────
const SRVD = 'com.sap.gateway.srvd.your_service.v0001';
const APP = 'your.app.namespace::YourListReport';
const IDS = {
createBOMToolbarBtn: `${APP}--fe::table::...`,
dialog: `fe::APD_::${SRVD}.CreateBOM`,
dialogOkBtn: `fe::APD_::${SRVD}.CreateBOM::Action::Ok`,
dialogCancelBtn: `fe::APD_::${SRVD}.CreateBOM::Action::Cancel`,
materialField: 'APD_::Material',
materialInner: 'APD_::Material-inner',
materialVHIcon: 'APD_::Material-inner-vhi',
// ... all discovered IDs
} as const;
test.describe('BOM Create Complete Flow', () => {
test('Complete BOM Create — single session', async ({ page, ui5 }) => {
// Step 1: Navigate to BOM App
await test.step('Step 1: Navigate to BOM Maintenance App', async () => {
await page.goto(process.env.SAP_CLOUD_BASE_URL!);
// ... FLP navigation using ui5.press() for tiles
});
// Step 2: Open Dialog and Verify Fields
await test.step('Step 2: Open Create BOM Dialog', async () => {
await ui5.press({ id: IDS.createBOMToolbarBtn });
const materialField = await ui5.control({ id: IDS.materialField });
expect(await materialField.getRequired()).toBe(true);
// ... verify all fields
});
// Step 3-5: Fill form via Value Helps and proxy methods
// Step 6: Verify all required fields
// Step 7: Submit and handle error recovery
// Step 8: Verify return to List Report
});
});
```
**Key characteristics of generated test scripts:**
| Aspect | Pattern |
| ----------------- | ---------------------------------------------------------------------- |
| Import | `import { test, expect } from 'playwright-praman'` only |
| Control IDs | Constants extracted from live discovery |
| UI5 interactions | 100% Praman fixtures (`ui5.control()`, `ui5.press()`, `ui5.fill()`) |
| Playwright native | Only for FLP tab navigation (`page.getByText()`) and `page.goto()` |
| Value Help | `innerTable.getContextByIndex(n).getObject()` for OData binding |
| Error recovery | Graceful `try/catch` — close error dialogs, cancel, verify List Report |
| Annotations | `test.info().annotations.push()` for rich HTML report output |
## Step 4: Convert to Gold Standard Format
The initial generated script may not be in full Praman gold standard format. To upgrade it, attach the script in your MCP chat and prompt:
```text
Convert the attached test script to Praman gold standard format.
```
The agent will:
- Add the Apache-2.0 license header
- Add the TSDoc compliance report block
- Ensure 100% Praman fixture usage for UI5 elements
- Add `test.step()` wrappers with descriptive names
- Add `test.info().annotations` for rich reporting
- Ensure `searchOpenDialogs: true` on all dialog controls
- Apply `toPass()` retry patterns for slow SAP systems
## Step 5: Debug and Heal
The generated test may not pass on the first run. SAP systems have varying response times, async OData loads, and environment-specific quirks.
### Iterative Healing Process
```text
┌──────────────────────────────────────────────┐
│ Run test → Fails → Agent fixes → Re-run │
│ ↑ │ │
│ └──────────────────────────────┘ │
│ │
│ Typically 3-6 iterations depending on: │
│ - SAP system response time │
│ - OData data availability │
│ - Control rendering timing │
└──────────────────────────────────────────────┘
```
Run the test in debug mode:
```bash
npx playwright test tests/e2e/sap-cloud/your-test.spec.ts --headed --debug
```
Or use the healer agent directly:
```text
Goal: Fix this failing test
Use praman SAP healer agent:
.github/agents/praman-sap-healer.agent.md
Test file: tests/e2e/sap-cloud/your-test.spec.ts
Error: [paste the error output]
```
### Common Fixes the Healer Applies
| Issue | Fix |
| --------------------------- | --------------------------------------------------------- |
| Timeout waiting for control | Add `toPass()` retry with progressive intervals |
| Value Help not open yet | Poll `control.isOpen()` before interacting |
| OData data not loaded | `getContextByIndex()` with retry loop |
| FLP tab not switching | Use `page.getByText()` DOM click instead of `ui5.press()` |
| Dialog control not found | Add `searchOpenDialogs: true` |
| Button not enabled | Wait for `ui5.waitForUI5()` after previous action |
## Step 6: Production-Ready Test
After the heal cycle completes, you have a production-ready gold standard test with:
- 100% Praman fixture usage for all UI5 elements
- Graceful error recovery (no hard failures on validation errors)
- Rich `test.info().annotations` for HTML report output
- `toPass()` retry patterns tuned to your SAP system timing
- Apache-2.0 license header and TSDoc compliance block
### Full Gold Standard Example
The script below is a complete, ready-to-run gold standard test for the **Maintain Bill of Material** app. Copy it, update the control IDs to match your system's live discovery output, and set `SAP_CLOUD_BASE_URL` in your `.env`.
bom-e2e-gold-standard.spec.ts — full source (click to expand)
```typescript
/**
* @license Apache-2.0
*
* ═══════════════════════════════════════════════════════════════
* GOLD STANDARD — BOM Complete End-to-End Flow (V1 SmartField)
* ═══════════════════════════════════════════════════════════════
*
* App: SAP S/4HANA Cloud — Maintain Bill of Material
* Type: Fiori Elements V2 List Report with Action Parameter Dialog
* UI5: 1.142.x
* OData: V2 — SmartField controls (sap.ui.comp.smartfield.SmartField)
*
* Steps:
* 1. Navigate to BOM app (FLP → space tab → tile)
* 2. Open Create BOM dialog and verify structure
* 3. Test Material value help (SmartTable → rows → binding context)
* 4. Test Plant value help
* 5. Test BOM Usage inner ComboBox (open/close/items)
* 6. Fill form with valid data from live value helps
* 7. Click Create button — handle success or validation error gracefully
* 8. Verify return to BOM List Report
*
* ═══════════════════════════════════════════════════════════════
* PRAMAN COMPLIANCE REPORT
* ═══════════════════════════════════════════════════════════════
*
* Controls Discovered: 15+
* UI5 Elements via Praman fixtures: 100%
* Playwright native (non-UI5 only): page.goto, waitForLoadState,
* getByText (FLP space tab DOM click), toHaveTitle
*
* Fixtures used:
* ui5.control (10), ui5.press (6), ui5.fill (2),
* ui5.getValue (3), ui5.waitForUI5 (15)
*
* Control proxy methods:
* getControlType, getProperty, getRequired, getEnabled, getVisible,
* isOpen, close, getTable, getRows, getBindingContext,
* getContextByIndex, getObject, getItems, getKey, getText,
* open, setSelectedKey, getSelectedKey, fireChange
*
* Auth method: storageState — credentials from .env via seed file
* Forbidden pattern scan: PASSED
* COMPLIANCE: PASSED — 100% Praman/UI5 methods for all UI5 elements
*
* SAP Best Practices:
* - Data-driven: getContextByIndex().getObject() for OData binding
* - Control-based: setSelectedKey() for ComboBox (no text matching)
* - Localization-safe: avoids text selectors for controls
* - setValue() + fireChange() via ui5.fill() for proper event propagation
* ═══════════════════════════════════════════════════════════════
*/
// ── Control ID Constants (update after live discovery on your system) ──
const IDS = {
// Dialog fields — sap.ui.comp.smartfield.SmartField
materialField: 'createBOMFragment--material',
materialInput: 'createBOMFragment--material-input',
materialVHIcon: 'createBOMFragment--material-input-vhi',
materialVHDialog: 'createBOMFragment--material-input-valueHelpDialog',
materialVHTable: 'createBOMFragment--material-input-valueHelpDialog-table',
plantField: 'createBOMFragment--plant',
plantInput: 'createBOMFragment--plant-input',
plantVHIcon: 'createBOMFragment--plant-input-vhi',
plantVHDialog: 'createBOMFragment--plant-input-valueHelpDialog',
plantVHTable: 'createBOMFragment--plant-input-valueHelpDialog-table',
bomUsageField: 'createBOMFragment--variantUsage',
bomUsageCombo: 'createBOMFragment--variantUsage-comboBoxEdit',
// Dialog buttons
okBtn: 'createBOMFragment--OkBtn',
cancelBtn: 'createBOMFragment--CancelBtn',
} as const;
// ── Test Data ─────────────────────────────────────────────────────────
const TEST_DATA = {
flpSpaceTab: 'Bills Of Material', // FLP space tab label
tileHeader: 'Maintain Bill Of Material',
bomUsageKey: '1', // '1' = Production
} as const;
// ─────────────────────────────────────────────────────────────────────
test.describe('BOM End-to-End Flow', () => {
test('Complete BOM Flow — V2 SmartField single session', async ({ page, ui5 }) => {
// ═══════════════════════════════════════════════════════════════
// STEP 1: Navigate to BOM Maintenance App
// ═══════════════════════════════════════════════════════════════
await test.step('Step 1: Navigate to BOM Maintenance App', async () => {
// FLP home page has async widgets that keep UI5 busy and never reach
// networkidle. Use domcontentloaded + title assertion only.
await page.goto(process.env.SAP_CLOUD_BASE_URL!);
await page.waitForLoadState('domcontentloaded');
// Verify FLP home loaded (Playwright web-first assertion — auto-retries)
await expect(page).toHaveTitle(/Home/, { timeout: 60000 });
// Navigate to Bills Of Material space tab.
// FLP tabs use sap.m.IconTabFilter — firePress() does not trigger tab
// switching. A DOM click is the only reliable method here.
await page.getByText(TEST_DATA.flpSpaceTab, { exact: true }).click();
// Click Maintain Bill Of Material tile.
// Wrapped in toPass() because ui5.press() calls waitForUI5() internally,
// which can time out on slow systems where FLP background OData requests
// are still active.
await expect(async () => {
await ui5.press({
controlType: 'sap.m.GenericTile',
properties: { header: TEST_DATA.tileHeader },
});
}).toPass({ timeout: 60000, intervals: [5000, 10000] });
// Wait for Create BOM button — proves the V2 app has fully loaded.
let createBtn: Awaited>;
await expect(async () => {
createBtn = await ui5.control({
controlType: 'sap.m.Button',
properties: { text: 'Create BOM' },
});
const text = await createBtn.getProperty('text');
expect(text).toBe('Create BOM');
}).toPass({ timeout: 120000, intervals: [5000, 10000] });
test.info().annotations.push({
type: 'info',
description: 'V2 List Report loaded — Create BOM button visible',
});
});
// ═══════════════════════════════════════════════════════════════
// STEP 2: Open Create BOM Dialog and Verify Structure
// ═══════════════════════════════════════════════════════════════
await test.step('Step 2: Open Create BOM Dialog', async () => {
await ui5.press({
controlType: 'sap.m.Button',
properties: { text: 'Create BOM' },
});
await ui5.waitForUI5();
// Verify Material SmartField — type and required flag
const materialField = await ui5.control({
id: IDS.materialField,
searchOpenDialogs: true,
});
expect(await materialField.getControlType()).toBe('sap.ui.comp.smartfield.SmartField');
expect(await materialField.getRequired()).toBe(true);
// Verify BOM Usage SmartField — type and required flag
const bomUsageField = await ui5.control({
id: IDS.bomUsageField,
searchOpenDialogs: true,
});
expect(await bomUsageField.getControlType()).toBe('sap.ui.comp.smartfield.SmartField');
expect(await bomUsageField.getRequired()).toBe(true);
// Verify footer buttons exist and are interactive
const createDialogBtn = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true });
const cancelDialogBtn = await ui5.control({ id: IDS.cancelBtn, searchOpenDialogs: true });
expect(await createDialogBtn.getProperty('text')).toBe('Create');
expect(await cancelDialogBtn.getProperty('text')).toBe('Cancel');
expect(await cancelDialogBtn.getEnabled()).toBe(true);
test.info().annotations.push({
type: 'info',
description: 'Dialog structure verified: required fields and footer buttons present',
});
});
// ═══════════════════════════════════════════════════════════════
// STEP 3: Test Material Value Help
// ═══════════════════════════════════════════════════════════════
await test.step('Step 3: Test Material Value Help', async () => {
await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true });
const materialDialog = await ui5.control({
id: IDS.materialVHDialog,
searchOpenDialogs: true,
});
expect(await materialDialog.isOpen()).toBe(true);
await ui5.waitForUI5();
// SmartTable.getTable() → innerTable — then getRows() for binding check
const smartTable = await ui5.control({ id: IDS.materialVHTable, searchOpenDialogs: true });
const innerTable = await smartTable.getTable();
const rows = (await innerTable.getRows()) as unknown[];
expect(rows.length).toBeGreaterThan(0);
// Wait for OData data to load using Playwright auto-retry
let rowCount = 0;
await expect(async () => {
rowCount = 0;
for (const row of rows) {
const ctx = await (
row as { getBindingContext: () => Promise }
).getBindingContext();
if (ctx) rowCount++;
}
expect(rowCount).toBeGreaterThan(0);
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
test
.info()
.annotations.push({ type: 'info', description: `Material VH: ${rowCount} rows loaded` });
await materialDialog.close();
await ui5.waitForUI5();
});
// ═══════════════════════════════════════════════════════════════
// STEP 4: Test Plant Value Help
// ═══════════════════════════════════════════════════════════════
await test.step('Step 4: Test Plant Value Help', async () => {
await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true });
const plantDialog = await ui5.control({
id: IDS.plantVHDialog,
searchOpenDialogs: true,
});
expect(await plantDialog.isOpen()).toBe(true);
await ui5.waitForUI5();
const smartTablePlant = await ui5.control({ id: IDS.plantVHTable, searchOpenDialogs: true });
const innerTablePlant = await smartTablePlant.getTable();
const plantRows = (await innerTablePlant.getRows()) as unknown[];
let plantRowCount = 0;
await expect(async () => {
plantRowCount = 0;
for (const row of plantRows) {
const ctx = await (
row as { getBindingContext: () => Promise }
).getBindingContext();
if (ctx) plantRowCount++;
}
expect(plantRowCount).toBeGreaterThan(0);
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
test
.info()
.annotations.push({ type: 'info', description: `Plant VH: ${plantRowCount} rows loaded` });
await plantDialog.close();
await ui5.waitForUI5();
});
// ═══════════════════════════════════════════════════════════════
// STEP 5: Test BOM Usage Dropdown (Inner ComboBox)
// ═══════════════════════════════════════════════════════════════
await test.step('Step 5: Test BOM Usage Dropdown', async () => {
const bomUsageCombo = await ui5.control({
id: IDS.bomUsageCombo,
searchOpenDialogs: true,
});
// Enumerate items via proxy
const rawItems = await bomUsageCombo.getItems();
const items: Array<{ key: string; text: string }> = [];
if (Array.isArray(rawItems)) {
for (const itemProxy of rawItems) {
items.push({
key: String(await itemProxy.getKey()),
text: String(await itemProxy.getText()),
});
}
}
expect(items.length).toBeGreaterThan(0);
test.info().annotations.push({
type: 'info',
description: `BOM Usage items: ${items.map((i) => `${i.key}: ${i.text}`).join(', ')}`,
});
// Verify open/close state management
await bomUsageCombo.open();
await ui5.waitForUI5();
expect(await bomUsageCombo.isOpen()).toBe(true);
await bomUsageCombo.close();
await ui5.waitForUI5();
expect(await bomUsageCombo.isOpen()).toBe(false);
});
// ═══════════════════════════════════════════════════════════════
// STEP 6: Fill Form with Valid Data
// ═══════════════════════════════════════════════════════════════
await test.step('Step 6: Fill Form with Valid Data', async () => {
// ── Fill Material ──────────────────────────────────────────
await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true });
const materialDialogCtrl = await ui5.control({
id: IDS.materialVHDialog,
searchOpenDialogs: true,
});
await expect(async () => {
expect(await materialDialogCtrl.isOpen()).toBe(true);
}).toPass({ timeout: 30000, intervals: [1000, 2000] });
const smartTableMat = await ui5.control({ id: IDS.materialVHTable, searchOpenDialogs: true });
const innerTableMat = await smartTableMat.getTable();
// Read first available material from OData binding
let materialValue = '';
await expect(async () => {
const ctx = await innerTableMat.getContextByIndex(0);
expect(ctx).toBeTruthy();
const obj = (await ctx.getObject()) as { Material?: string };
expect(obj?.Material).toBeTruthy();
materialValue = obj!.Material!;
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
await materialDialogCtrl.close();
await ui5.waitForUI5();
// ui5.fill() = atomic setValue + fireChange + waitForUI5
await ui5.fill({ id: IDS.materialInput, searchOpenDialogs: true }, materialValue);
await ui5.waitForUI5();
test.info().annotations.push({ type: 'info', description: `Material: ${materialValue}` });
// ── Fill Plant ─────────────────────────────────────────────
await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true });
const plantDialogCtrl = await ui5.control({ id: IDS.plantVHDialog, searchOpenDialogs: true });
await expect(async () => {
expect(await plantDialogCtrl.isOpen()).toBe(true);
}).toPass({ timeout: 30000, intervals: [1000, 2000] });
const smartTablePlant = await ui5.control({ id: IDS.plantVHTable, searchOpenDialogs: true });
const innerTablePlant = await smartTablePlant.getTable();
let plantValue = '';
await expect(async () => {
const ctx = await innerTablePlant.getContextByIndex(0);
expect(ctx).toBeTruthy();
const obj = (await ctx.getObject()) as { Plant?: string };
expect(obj?.Plant).toBeTruthy();
plantValue = obj!.Plant!;
}).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] });
await plantDialogCtrl.close();
await ui5.waitForUI5();
await ui5.fill({ id: IDS.plantInput, searchOpenDialogs: true }, plantValue);
await ui5.waitForUI5();
test.info().annotations.push({ type: 'info', description: `Plant: ${plantValue}` });
// ── Fill BOM Usage ─────────────────────────────────────────
const bomUsageCtrl = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true });
await bomUsageCtrl.open();
await ui5.waitForUI5();
await bomUsageCtrl.setSelectedKey(TEST_DATA.bomUsageKey);
await bomUsageCtrl.fireChange({ value: TEST_DATA.bomUsageKey });
await bomUsageCtrl.close();
await ui5.waitForUI5();
test.info().annotations.push({
type: 'info',
description: `BOM Usage: ${TEST_DATA.bomUsageKey} (Production)`,
});
// ── Pre-submission verification ────────────────────────────
const finalMaterial =
(await ui5.getValue({ id: IDS.materialInput, searchOpenDialogs: true })) ?? '';
const finalPlant =
(await ui5.getValue({ id: IDS.plantInput, searchOpenDialogs: true })) ?? '';
const finalBomUsage =
(await (
await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true })
).getSelectedKey()) ?? '';
expect(finalMaterial).toBe(materialValue);
expect(finalPlant).toBe(plantValue);
expect(finalBomUsage).toBe(TEST_DATA.bomUsageKey);
test.info().annotations.push({
type: 'info',
description: [
'Pre-submission verification:',
` Material: ${finalMaterial}`,
` Plant: ${finalPlant}`,
` BOM Usage: ${finalBomUsage}`,
].join('\n'),
});
});
// ═══════════════════════════════════════════════════════════════
// STEP 7: Click Create Button and Handle Result Gracefully
// ═══════════════════════════════════════════════════════════════
await test.step('Step 7: Click Create Button and handle result', async () => {
const createBtn = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true });
expect(await createBtn.getProperty('text')).toBe('Create');
expect(await createBtn.getEnabled()).toBe(true);
await ui5.press({ id: IDS.okBtn, searchOpenDialogs: true });
await ui5.waitForUI5();
// ── Outcome A: dialog closes → BOM created → success ───────
// ── Outcome B: validation error → dialog stays open ─────────
let dialogStillOpen = false;
try {
const okBtnCheck = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true });
dialogStillOpen = (await okBtnCheck.getEnabled()) !== undefined;
} catch {
dialogStillOpen = false;
}
// Detect error popover or MessageBox
let hasErrorDialog = false;
try {
const popover = await ui5
.control({ controlType: 'sap.m.MessagePopover', searchOpenDialogs: true })
.catch(() => null);
if (popover) hasErrorDialog = !!(await popover.isOpen().catch(() => false));
if (!hasErrorDialog) {
const msgBox = await ui5
.control({
controlType: 'sap.m.Dialog',
properties: { type: 'Message' },
searchOpenDialogs: true,
})
.catch(() => null);
if (msgBox) {
const title = await msgBox.getProperty('title').catch(() => '');
hasErrorDialog = typeof title === 'string' && title.length > 0;
}
}
} catch {
/* no error dialogs — success path */
}
// Graceful cleanup: close error dialogs, then cancel Create BOM dialog
if (hasErrorDialog) {
try {
await ui5.press({
controlType: 'sap.m.Button',
properties: { text: 'Close' },
searchOpenDialogs: true,
});
await ui5.waitForUI5();
} catch {
/* Close button not present */
}
}
if (dialogStillOpen) {
try {
await ui5.press({ id: IDS.cancelBtn, searchOpenDialogs: true });
await ui5.waitForUI5();
} catch {
/* dialog already closed */
}
}
test.info().annotations.push({
type: 'info',
description: `Create result: dialogOpen=${dialogStillOpen}, errorDialog=${hasErrorDialog}`,
});
});
// ═══════════════════════════════════════════════════════════════
// STEP 8: Verify Return to BOM List Report
// ═══════════════════════════════════════════════════════════════
await test.step('Step 8: Verify return to BOM List Report', async () => {
// Create BOM button visible + enabled proves we are back on the List Report
const listBtn = await ui5.control(
{ controlType: 'sap.m.Button', properties: { text: 'Create BOM' } },
{ timeout: 30000 },
);
expect(await listBtn.getProperty('text')).toBe('Create BOM');
expect(await listBtn.getProperty('enabled')).toBe(true);
test.info().annotations.push({
type: 'info',
description: 'Returned to BOM List Report — gold standard test complete',
});
});
});
});
```
### Key Patterns to Copy
| Pattern | Code |
| ---------------------- | ------------------------------------------------------------------ |
| Env-based URL | `await page.goto(process.env.SAP_CLOUD_BASE_URL!)` |
| FLP tab (DOM click) | `await page.getByText('Tab Name', { exact: true }).click()` |
| Retry tile press | `await expect(async () => { await ui5.press({...}) }).toPass(...)` |
| Dialog control | `await ui5.control({ id: '...', searchOpenDialogs: true })` |
| OData binding read | `(await innerTable.getContextByIndex(0).getObject())` |
| Atomic fill | `await ui5.fill({ id: '...', searchOpenDialogs: true }, value)` |
| ComboBox select | `await ctrl.setSelectedKey('1'); await ctrl.fireChange(...)` |
| Graceful error cleanup | `try { await ui5.press({...Close...}) } catch { }` |
### Run in CI
```bash
npx playwright test tests/e2e/sap-cloud/ --project=chromium
```
### Expected Output
```text
Running 1 test using 1 worker
✓ tests/e2e/sap-cloud/bom-e2e-gold-standard.spec.ts
BOM End-to-End Flow
Complete BOM Flow — V2 SmartField single session (45s)
Step 1: Navigate to BOM Maintenance App
Step 2: Open Create BOM Dialog
Step 3: Test Material Value Help
Step 4: Test Plant Value Help
Step 5: Test BOM Usage Dropdown
Step 6: Fill Form with Valid Data
Step 7: Click Create Button and handle result
Step 8: Verify return to BOM List Report
1 passed (48s)
```
## Quick Reference: Agent Prompts
| Task | Prompt |
| ------------------ | ------------------------------------------------------------------ |
| Plan a new test | `Goal: Create SAP test case. Use praman SAP planner agent...` |
| Generate from plan | `Goal: Generate test from plan. Use praman SAP generator agent...` |
| Fix a failing test | `Goal: Fix this failing test. Use praman SAP healer agent...` |
| Full pipeline | `/praman-sap-coverage` (Claude Code) |
| Convert to gold | `Convert the attached test script to Praman gold standard format` |
## Troubleshooting
### Agent can't find the SAP app
- Verify `SAP_CLOUD_BASE_URL` in `.env` points to the FLP home page
- Verify the seed file authenticates successfully: `npx playwright test tests/seeds/sap-seed.spec.ts --project=agent-seed-test --headed`
- Check that Chrome is launching in headed mode (add `--headed` if needed)
### Agent discovers wrong control types
- SAP apps use different control frameworks depending on the Fiori Elements version:
- **V2 apps**: `sap.ui.comp.smartfield.SmartField` (SmartField)
- **V4 apps**: `sap.ui.mdc.Field` with `sap.ui.mdc.field.FieldInput` (MDC)
- The planner auto-detects the framework. If it gets it wrong, specify in your prompt: "This is a Fiori Elements V4 app using MDC controls"
### Test passes locally but fails in CI
- Add longer timeouts for `toPass()` intervals in CI environments
- Ensure CI has network access to the SAP system
- Use `storageState` for auth to avoid login flow in every test
- Set `SAP_CLOUD_BASE_URL` as a CI secret, not hardcoded
### MCP server connection issues
- Verify `.mcp.json` exists with the `playwright-test` server configured
- Restart the MCP server: close and reopen your IDE
- Check that `@playwright/test` is installed: `npx playwright --version`
---
## Azure Playwright Workspaces
Run Praman SAP UI5 tests at scale using cloud-hosted browsers with **Azure Playwright Workspaces** — no local machine required. GitHub Actions runs your test code, Azure runs the browsers, and your SAP system is tested end-to-end.
## Overview
Azure Playwright Workspaces is a Microsoft-managed service that provides cloud-hosted browsers for Playwright tests. Combined with GitHub Actions, it creates a **fully automated, zero-local-machine** testing pipeline for SAP applications.
```
┌───────────┐ push/PR/cron ┌──────────────────────┐ WebSocket ┌───────────────┐
│ GitHub │ ───────────────► │ GitHub Actions │ ◄════════════► │ Azure Cloud │
│ Repo │ │ Runner (ubuntu) │ Playwright │ Browsers │
│ │ │ │ Protocol │ │
│ • tests/ │ │ Runs: │ │ Runs: │
│ • src/ │ │ • Node.js workers │ │ • Chromium │
│ • config │ │ • playwright-praman │ │ • Firefox │
│ │ │ • fixtures/proxies │ page.eval() │ • WebKit │
│ │ │ • assertions │ ──────────────►│ │
│ YOU DO │ │ │ │ Reaches SAP │
│ NOTHING │ │ YOU DO NOTHING │ ◄──results── │ Cloud direct │
└───────────┘ └──────────────────────┘ └───────────────┘
│
▼
┌──────────────────┐
│ GitHub Artifacts │
│ • HTML report │
│ • Traces │
│ • Screenshots │
└──────────────────┘
```
### What runs where
| Component | Runs On | Details |
| -------------------------- | --------------------- | ------------------------------------------------------------- |
| **Test code (`.spec.ts`)** | GitHub Actions runner | Node.js worker processes execute your test files |
| **playwright-praman** | GitHub Actions runner | Fixtures, proxy, bridge setup, matchers — all Node.js code |
| **Browser instances** | Azure cloud | Chromium, Firefox, or WebKit — managed by Microsoft |
| **`page.evaluate()`** | Azure cloud browser | UI5 bridge scripts execute in the remote browser's JS context |
| **SAP application** | SAP Cloud (BTP/S4) | Cloud browsers access your SAP system over the internet |
| **Your local machine** | Not involved | Push to GitHub and review results — that's it |
:::info Key insight
Playwright Workspaces uses `browserType.connect()` (full Playwright protocol over WebSocket). This means **all Playwright APIs work identically** — `page.evaluate()`, `addInitScript()`, `selectors.register()`, custom fixtures. Only the browser runs remotely; your test logic stays on the runner.
:::
## Architecture
### Split Execution Model
Playwright Workspaces does **not** run your tests entirely in Azure. It's a split model:
1. **Client side** (GitHub Actions runner): Playwright Test runner, worker processes, your test code, npm packages, fixtures, assertions
2. **Cloud side** (Azure): Browser instances only — DOM rendering, JavaScript execution in browser context, network requests from the browser
This split is transparent to your tests. When your test calls `page.goto()` or `page.evaluate()`, the Playwright protocol serializes the command over WebSocket to the remote browser.
### Data Flow: SAP E2E Test Execution
```
Step 1: GitHub Actions trigger (push / PR / cron / manual)
│
▼
Step 2: Runner installs deps (npm ci, playwright install)
│
▼
Step 3: @azure/playwright sets connectOptions → WebSocket to Azure
│
▼
Step 4: playwright-praman fixtures initialize
│ ├── Config loaded (Zod validation)
│ ├── UI5 bridge scripts prepared
│ └── Selector engine registered
│
▼
Step 5: Auth setup project runs
│ ├── page.goto(SAP_CLOUD_BASE_URL) → remote browser navigates to SAP
│ ├── page.evaluate() detects login form → fills credentials
│ ├── storageState saved → .auth/sap-session.json on runner
│ └── Session cookies captured for reuse
│
▼
Step 6: Test projects run (depend on auth setup)
│ ├── storageState loaded → authenticated session
│ ├── page.addInitScript() injects __praman_bridge into remote browser
│ ├── ui5.control() → page.evaluate() → bridge resolves UI5 control
│ ├── proxy.setValue() → page.evaluate() → bridge calls control method
│ ├── waitForUI5Stable() → page.evaluate() → polls getUIPending()
│ └── expect(proxy).toHaveUI5Text() → web-first retry assertion
│
▼
Step 7: Results collected
├── HTML report uploaded as GitHub artifact
├── Trace files (on failure) uploaded
├── Screenshots (on failure) uploaded
└── Test results visible in GitHub Actions UI
```
### Praman Compatibility
Every Praman feature works with remote browsers because the split is at the browser boundary:
| Praman Feature | How It Works Remotely |
| ------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| **`test.extend()` fixtures** | Fixture setup/teardown runs on the runner. `page` object points to remote browser transparently. |
| **`page.evaluate()` bridge scripts** | Function body serialized over WebSocket, executed in remote browser's JS context. Results returned. |
| **`page.addInitScript()` bridge injection** | Script sent via Playwright protocol. Executes in remote browser on every navigation. |
| **`selectors.register()` (`ui5=` engine)** | Selector engine script sent to remote browser via protocol. Works like local. |
| **Custom matchers (`toHaveUI5Text`)** | Use `page.evaluate()` internally — same serialization path. |
| **`waitForUI5Stable()`** | Polls via `page.evaluate()` in remote browser. Works identically. |
| **`storageState` auth** | JSON file on runner filesystem. Playwright sends cookies to remote browser context. |
| **Pino logging** | Runs Node.js side on the runner. No browser dependency. |
| **Traces and screenshots** | Captured by Playwright protocol from remote browser. Transferred to runner. |
## Prerequisites
- **Azure subscription** with Owner or Contributor role
- **SAP Cloud system** (BTP, S/4HANA Cloud) accessible over the internet
- **GitHub repository** with your Praman test code
- **Azure CLI** installed locally (for initial setup only)
## Setup Guide
### Step 1: Create Azure Playwright Workspace
1. Sign in to the [Azure Portal](https://portal.azure.com)
2. Select **Create a resource** → search for **Playwright Workspaces**
3. Configure:
| Field | Value |
| ------------------ | ---------------------------------------------------------------- |
| **Subscription** | Your Azure subscription |
| **Resource group** | Create new or use existing |
| **Name** | e.g., `praman-sap-testing` |
| **Location** | Choose closest to your SAP system (e.g., West Europe for EU BTP) |
4. Select **Review + Create** → **Create**
5. Once deployed, go to the resource → **Get Started** → copy the **endpoint URL**
### Step 2: Install Azure Packages
```bash
npm install --save-dev @azure/playwright @azure/identity dotenv
```
### Step 3: Create Service Config
Create `playwright.service.config.ts` in your project root:
```typescript
export default defineConfig(
config,
createAzurePlaywrightConfig(config, {
os: ServiceOS.LINUX,
credential: new DefaultAzureCredential(),
// Increase connect timeout for initial browser allocation
connectTimeout: 30_000,
}),
);
```
:::tip SAP on-premise systems
If your SAP system is behind a VPN or corporate firewall (not publicly accessible), add `exposeNetwork` to tunnel traffic through the runner:
```typescript
createAzurePlaywrightConfig(config, {
os: ServiceOS.LINUX,
credential: new DefaultAzureCredential(),
exposeNetwork: '*.your-sap-domain.internal',
});
```
For SAP BTP / S/4HANA Cloud (publicly accessible), no `exposeNetwork` is needed.
:::
### Step 4: Configure Environment
Add to your `.env` file:
```bash
PLAYWRIGHT_SERVICE_URL=wss://eastus.api.playwright.microsoft.com/accounts/YOUR_WORKSPACE_ID/browsers
```
### Step 5: Configure GitHub Secrets
In your GitHub repository → **Settings** → **Secrets and variables** → **Actions**, add:
| Secret | Value |
| ------------------------ | --------------------------------------------------------------------------------------- |
| `PLAYWRIGHT_SERVICE_URL` | The endpoint URL from Step 1 |
| `SAP_CLOUD_BASE_URL` | Your SAP system URL (e.g., `https://my-system.launchpad.cfapps.eu10.hana.ondemand.com`) |
| `SAP_CLOUD_USERNAME` | SAP test user username |
| `SAP_CLOUD_PASSWORD` | SAP test user password |
| `SAP_CLIENT` | SAP client number (e.g., `100`) |
| `AZURE_TENANT_ID` | Azure AD tenant ID (for service principal auth) |
| `AZURE_CLIENT_ID` | Service principal client ID |
| `AZURE_CLIENT_SECRET` | Service principal secret |
:::warning Test user best practices
- Use a **dedicated technical test user** — never personal credentials
- Disable MFA/2FA for the test user (or use certificate-based auth)
- Grant minimum required SAP roles for your test scenarios
- Rotate credentials regularly via GitHub secret rotation
:::
### Step 6: GitHub Actions Workflow
The CI workflow already includes an Azure Playwright Workspaces job. It runs on `workflow_dispatch` (manual) or when a PR is labeled `azure-test`:
```yaml
azure-playwright:
name: Azure Playwright Workspaces
runs-on: ubuntu-latest
timeout-minutes: 30
if: github.event_name == 'workflow_dispatch' || contains(github.event.pull_request.labels.*.name, 'azure-test')
needs: [quality, build]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
cache: 'npm'
- run: npm ci
- uses: actions/download-artifact@v4
with:
name: dist
path: dist/
- run: npx playwright install --with-deps chromium
- name: Run tests on Azure Playwright Workspaces
run: npx playwright test --config=playwright.service.config.ts --workers=20
env:
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
SAP_CLOUD_BASE_URL: ${{ secrets.SAP_CLOUD_BASE_URL }}
SAP_CLOUD_USERNAME: ${{ secrets.SAP_CLOUD_USERNAME }}
SAP_CLOUD_PASSWORD: ${{ secrets.SAP_CLOUD_PASSWORD }}
SAP_CLIENT: ${{ secrets.SAP_CLIENT }}
AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: azure-playwright-report
path: playwright-report/
retention-days: 14
```
To enable additional triggers (nightly, every push to main), add:
```yaml
on:
schedule:
- cron: '0 2 * * *' # Nightly at 2 AM UTC
push:
branches: [main]
```
## Running Tests
### Local Development (with Cloud Browsers)
```bash
# Authenticate with Azure CLI
az login
# Run a single test against cloud browsers
npx playwright test tests/e2e/my-test.spec.ts --config=playwright.service.config.ts
# Run full suite with 20 parallel workers
npx playwright test --config=playwright.service.config.ts --workers=20
```
### CI (Fully Automated)
1. Push your code to GitHub (or create a PR)
2. Apply the `azure-test` label, or trigger manually via **Actions** → **Run workflow**
3. Review results in the GitHub Actions UI
4. Download HTML report and traces from the **Artifacts** section
### VS Code Integration
1. Install the [Playwright Test extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright)
2. Open **Test Explorer** → **Select Default Profile**
3. Choose projects from `playwright.service.config.ts`
4. Run or debug tests — browsers run in Azure, debugging works locally
## Configuration Tuning
### Timeouts
Remote browsers add network latency. Adjust timeouts in your service config:
```typescript
export default defineConfig(
config,
createAzurePlaywrightConfig(config, {
os: ServiceOS.LINUX,
credential: new DefaultAzureCredential(),
connectTimeout: 30_000,
}),
{
// Override timeouts for cloud execution
timeout: 120_000, // Test timeout (SAP tests are slow)
expect: { timeout: 15_000 }, // Assertion retry timeout
use: {
actionTimeout: 30_000,
navigationTimeout: 120_000,
// Set timezone explicitly — cloud browser may differ
timezoneId: 'Europe/Berlin',
},
},
);
```
### Trace Configuration
Traces are transferred from Azure to the runner over the network. For large test suites, tune tracing:
```typescript
use: {
trace: 'retain-on-failure', // Only capture when tests fail
screenshot: 'only-on-failure', // Minimize data transfer
video: 'off', // Video is expensive over network
}
```
### Parallel Workers
SAP tests can run in parallel if they don't share state:
| Scenario | Workers | Notes |
| ---------------------------------- | ------- | --------------------------------------------- |
| Sequential SAP tests (shared data) | `1` | Current default — safe |
| Independent SAP apps | `5-10` | Each test targets a different app |
| Read-only SAP tests | `10-20` | No write conflicts |
| Cross-browser matrix | `20-50` | Same tests across Chromium + Firefox + WebKit |
```bash
npx playwright test --config=playwright.service.config.ts --workers=20
```
## Network Access Patterns
### SAP BTP / S/4HANA Cloud (Public)
```
GitHub Runner ──WebSocket──► Azure Browsers ──HTTPS──► SAP BTP (public URL)
```
No special configuration needed. Cloud browsers have standard internet access.
### SAP On-Premise (Behind Firewall)
```
GitHub Runner ──WebSocket──► Azure Browsers ──tunnel──► GitHub Runner ──VPN──► SAP On-Prem
```
Use `exposeNetwork` to route SAP traffic through the runner:
```typescript
createAzurePlaywrightConfig(config, {
exposeNetwork: '*.sap-corp.internal,10.0.0.0/8',
});
```
The runner must have network access to the SAP system (e.g., via a self-hosted GitHub runner on your corporate network).
### SAP on localhost (Development)
```
GitHub Runner ──WebSocket──► Azure Browsers ──tunnel──► GitHub Runner ──localhost──► SAP proxy
```
```typescript
createAzurePlaywrightConfig(config, {
exposeNetwork: '',
});
```
## Limitations
| Limitation | Impact | Mitigation |
| --------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------- |
| **Headed mode not available** | Cannot use `headless: false` on cloud browsers | Use local browsers for interactive debugging |
| **Network latency** | `page.evaluate()` round-trips are ~10-50ms vs ~1ms local | Increase timeouts; reduce unnecessary evaluate calls |
| **`download.path()` unavailable** | Cannot read download file path directly | Use `download.saveAs()` instead |
| **Timezone mismatch** | Cloud browser timezone may differ from SAP system | Set `timezoneId` explicitly |
| **Max 100 workers** | Hard limit per workspace | Sufficient for SAP testing (SAP itself is the bottleneck) |
| **Playwright version matching** | Client major.minor must match cloud service | Praman supports `>=1.57.0` — Azure supports recent versions |
| **No macOS cloud browsers** | Only Linux and Windows available | macOS rarely needed for SAP Fiori (web-based) |
## Cost
Azure Playwright Workspaces bills per **test minute** (browser time):
| Scenario | Est. Monthly Minutes | Cost |
| ------------------------ | ----------------------------- | ------------------------- |
| 10 SAP tests, 3 runs/day | ~450 min | Free tier or minimal |
| 50 SAP tests, nightly | ~7,500 min | Moderate |
| 100 tests, 20 workers | Same total, faster wall-clock | Same cost, faster results |
Free trial includes **100 test minutes/month**.
:::tip Cost optimization
- Use `trace: 'retain-on-failure'` to avoid capturing traces for passing tests
- Run only changed test files on PRs; full suite on nightly schedule
- Use `workers: 1` for sequential SAP tests to avoid billing for idle browser wait time
:::
## Troubleshooting
### Connection Timeout
```
Error: browserType.connect: Timeout 30000ms exceeded.
```
Increase `connectTimeout` in the service config. Check that `PLAYWRIGHT_SERVICE_URL` is correct.
### SAP System Unreachable
```
Error: page.goto: net::ERR_CONNECTION_REFUSED
```
For public SAP systems, verify the URL is accessible from Azure's network. For private systems, configure `exposeNetwork`.
### Auth Failures in CI
Ensure all SAP secrets are configured in GitHub. Check that `DefaultAzureCredential` can authenticate — in CI, this requires `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, and `AZURE_CLIENT_SECRET`.
### Flaky Tests (Timing)
Cloud browsers add latency. If tests pass locally but fail in Azure:
- Increase `actionTimeout` and `navigationTimeout`
- Add explicit `timezoneId` if date assertions fail
- Use `waitForUI5Stable()` (Praman handles this automatically)
---
## Example Reports & Data Architecture
Praman generates 12 dashboard reports that provide a comprehensive view of SAP S/4HANA quality across functional testing, performance, accessibility, integration, and geographic readiness. This guide explains how each report gets its data and what Praman is responsible for producing.
:::important
These reports are examples of what you can design and develop. Please read the documentation help for more details.
:::
## The 5 Data Source Categories
Every field in the quality dashboard traces back to one of 5 source categories:
| # | Source | What It Is |
| --- | ----------------------------------- | ------------------------------------------------------------------------------------ |
| 1 | **Playwright Test Execution** | Direct output from Playwright + Praman running tests against SAP Fiori / UI5 |
| 2 | **SAP Backend Telemetry** | Data extracted from SAP monitoring transactions (ST03N, STAD, SM37) via OData or RFC |
| 3 | **SAP Focused Run / Web Analytics** | Real User Monitoring (RUM) data from SAP observability tools |
| 4 | **Third-Party System APIs** | Health and transaction data from connected external systems |
| 5 | **Configuration / Manual Input** | Business metadata maintained by the project team (process owners, SLAs, revenue) |
## Source Category Deep Dive
Source 1: Playwright Test Execution (Praman)
This is what Praman directly produces. Every Playwright test run generates:
**Per-test-step data:**
- Transaction response time (page.evaluate + UI5 rendering + network roundtrip)
- Pass / fail / blocked status with error message and screenshot
- Memory consumption (via `performance.measureUserAgentSpecificMemory()`)
- CPU time (via `PerformanceObserver` long task monitoring)
- Network timing (via `PerformanceResourceTiming` API)
- DOM snapshot and UI5 control tree state at point of failure
**Per-test-suite data:**
- Total scenarios executed, passed, failed, blocked
- Execution duration (wall clock)
- Browser/viewport/locale configuration used
- Playwright trace file (.zip) for debugging
**Accessibility scan data (via @axe-core/playwright):**
- WCAG violations per page with rule ID, severity, DOM selector, HTML snippet
- Principle classification (perceivable / operable / understandable / robust)
- Screen reader compatibility assertions
- Keyboard navigation assertions (focus trap detection, tab order)
- Color contrast ratios (computed from rendered styles)
**Integration test data:**
- HTTP response codes from OData/REST calls during test execution
- CPI iFlow message IDs triggered during test
- Response latency of each external API call intercepted by Playwright's route handler
- Data integrity assertions (field values match expected after cross-system flow)
**How Praman captures SAP-specific metrics:**
```javascript
// Inside Playwright test, Praman uses page.evaluate() to call UI5 native APIs
const metrics = await page.evaluate(() => {
// UI5 performance measurement
const perfMon = sap.ui.require('sap/ui/performance/Measurement');
const measurements = perfMon.getAllMeasurements();
// OData request timing
const oDataModel = sap.ui.getCore().getModel();
const pendingRequests = oDataModel.getPendingChanges();
// Browser performance API
const navTiming = performance.getEntriesByType('navigation')[0];
const resourceTiming = performance.getEntriesByType('resource');
return { measurements, navTiming, resourceTiming };
});
```
Source 2: SAP Backend Telemetry
Extracted from SAP via OData services, RFC calls, or scheduled ABAP reports:
| SAP Transaction | What It Provides | How to Extract |
| ------------------------------ | ----------------------------------------------------------------- | ------------------------------------------ |
| **ST03N** (Workload Monitor) | Per-TCode response time breakdown, peak load periods, user counts | OData or RFC: `SWNC_GET_WORKLOAD_SNAPSHOT` |
| **STAD** (Statistical Records) | Dialog step detail: frontend roundtrip, network, memory per step | RFC: `SAPWL_STATREC_READ` |
| **SM37** (Job Monitor) | Batch job execution times, status, start/end timestamps | OData or RFC: `BAPI_XBP_JOB_STATUS_GET` |
| **SM21** (System Log) | Runtime errors, dumps, resource exhaustion events | RFC: `BAPI_XMI_LOGOFF` pattern |
| **SLG1** (Application Log) | Business-level errors (posting failures, workflow errors) | OData: Application Log API |
| **SM59** (RFC Destinations) | Connection status to external systems, latency | RFC: `RFC_SYSTEM_INFO` |
| **SE16/CDS Views** | Master data record counts, migration verification | OData CDS View exposure |
Source 3: SAP Focused Run / Web Analytics
Real User Monitoring data — tells you what actual users experience vs what tests simulate:
| Tool | Data Provided |
| --------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Focused Run RUM** | Browser type, OS, screen resolution, client-measured page load, network latency by geography, connection type |
| **SAP Web Analytics** | Geographic user distribution, locale preferences, Fiori app usage patterns, session duration |
Praman reads this data to set up browser contexts that match actual user conditions:
```javascript
// Praman configures Playwright context from Focused Run data
const context = await browser.newContext({
locale: 'de-DE', // from Web Analytics
viewport: { width: 1920, height: 1080 }, // from Focused Run RUM
geolocation: { latitude: 50.1, longitude: 8.7 },
});
```
Source 4: Third-Party System APIs
Health and transaction data from connected external systems:
| System | What to Query | API/Method |
| ---------------------- | --------------------------------------------------------- | ---------------------------------------------- |
| **SAP CPI** | iFlow execution logs, message status, error details | CPI OData API: `/api/v1/MessageProcessingLogs` |
| **ERP Integrations** | Connection status, OAuth token validity, sync queue depth | REST API health endpoints |
| **Banking Partners** | File delivery status, payment status codes | SFTP + EBICS/SWIFT status |
| **Logistics Carriers** | Shipment API response time, label generation success rate | Carrier tracking APIs |
Source 5: Configuration / Manual Input
Business metadata that doesn't come from any system — maintained by the project team:
- Process names, owners, business impact classification (critical/high/medium/low)
- Revenue-at-risk estimates per process
- SLA targets (response time thresholds, success rate targets)
- Rollout wave assignments (which countries in which wave)
- Company codes, currency, locale mappings
- Regulatory requirements (EAA, Section 508, etc.)
- Interface catalog (which interfaces exist, their criticality, message types)
- Role-to-process mappings, user counts per role
---
## The 12 Dashboard Reports
Click any report below to view its data model and field-by-field source mapping.
| # | Report | Primary Sources |
| --- | ------------------------------------------------------------------------------------------ | ---------------------------------- |
| 1 | [Executive Steering Committee Dashboard](#report-1-executive-steering-committee-dashboard) | Playwright, Config |
| 2 | [Business Process Deep Dive](#report-2-business-process-deep-dive) | Playwright, Config |
| 3 | [Role-Based Readiness](#report-3-role-based-readiness) | Playwright, Config, Web Analytics |
| 4 | [Data Migration Quality](#report-4-data-migration-quality) | Playwright, SAP Backend |
| 5 | [Risk Register (CFO Report)](#report-5-risk-register-cfo-report) | Playwright, Config |
| 6 | [How It Works](#report-6-how-it-works) | Static |
| 7 | [Performance Heatmap](#report-7-performance-heatmap) | Playwright, SAP Backend, Config |
| 8 | [Geographic & Customer Impact](#report-8-geographic--customer-impact) | Playwright, Focused Run, Config |
| 9 | [Real User Experience Simulation](#report-9-real-user-experience-simulation) | Playwright, Config |
| 10 | [SAP FI — Financial Accounting Quality](#report-10-sap-fi--financial-accounting-quality) | Playwright, SAP Backend, Config |
| 11 | [WCAG 2.1 AA Accessibility Compliance](#report-11-wcag-21-aa-accessibility-compliance) | Playwright (axe-core), Config |
| 12 | [Interface & Integration Quality](#report-12-interface--integration-quality) | 3rd Party, SAP Backend, Playwright |
| 13 | [End-to-End Cross-System Quality](#report-13-end-to-end-cross-system-quality) | Playwright, 3rd Party, Config |
---
Report 1: Executive Steering Committee Dashboard
**Data models:** `PROCESS_DATA`, `WEEKLY_TREND`
| Field | Source | Detail |
| ----------------------------- | ------------------- | --------------------------------------------- |
| `id`, `name`, `owner` | Config | Business process catalog |
| `tcodes` | Config | Test scenario definitions |
| `totalScenarios` | Playwright | Count of test specs in suite |
| `passed`, `failed`, `blocked` | Playwright | Test execution results |
| `businessImpact` | Config | Business classification |
| `revenueAtRisk` | Config | Finance team estimate |
| `usersAffected` | Web Analytics | User count per process |
| `failureDetails[].scenario` | Playwright | Test spec name that failed |
| `failureDetails[].severity` | Config | Pre-classified per test scenario |
| `WEEKLY_TREND[].readiness` | Playwright | Computed: (passed / total) x 100 |
| `WEEKLY_TREND[].defectsOpen` | Playwright + Config | Open defect count from failures + bug tracker |
Report 2: Business Process Deep Dive
**Data models:** `PROCESS_DATA` (same as Report 1, different visualization)
All fields same as Report 1 — this report is a different view of the same data, showing per-process drill-down with TCode-level test results.
Report 3: Role-Based Readiness
**Data models:** `ROLE_READINESS`
| Field | Source | Detail |
| --------------- | ------------- | ------------------------------------------------------ |
| `role` | Config | SAP role catalog (PFCG roles mapped to business roles) |
| `process` | Config | Role-to-process mapping |
| `readiness` | Playwright | % of test scenarios passing that this role executes |
| `users` | Web Analytics | Active user count for this role |
| `criticalPaths` | Config | Number of critical test paths assigned to this role |
| `blocked` | Playwright | Count of critical paths blocked/failing for this role |
**How readiness is computed:** Each test scenario is tagged with the SAP roles that execute it. A role's readiness = (passing scenarios for that role) / (total scenarios for that role) x 100.
Report 4: Data Migration Quality
**Data models:** `MIGRATION_DATA`
| Field | Source | Detail |
| ----------- | ---------------- | -------------------------------------------------- |
| `entity` | Config | Migration object catalog |
| `records` | SAP Backend | Record count from CDS view |
| `validated` | Playwright + SAP | Praman runs validation queries via OData CDS views |
| `errors` | Playwright + SAP | Records failing validation rules |
| `critical` | Playwright + SAP | Subset classified as blocking |
| `status` | Playwright | Computed from error rate thresholds |
```javascript
// Praman validates migration data via OData CDS views
const response = await request.get(
'/sap/opu/odata/sap/Z_MIGRATION_VALIDATION_CDS/CustomerValidation?$filter=HasErrors eq true',
);
```
Report 5: Risk Register (CFO Report)
**Data models:** Derived from `PROCESS_DATA`, `MIGRATION_DATA`, `PERF_DATA`
| Field | Source | Detail |
| ----------------- | ------------------- | ------------------------------------------------------------------ |
| Risk items | Playwright + Config | Auto-generated from failing tests, blocked scenarios, SLA breaches |
| Revenue at risk | Config | Finance team estimates linked to process failures |
| Mitigation status | Config | Manual tracking of remediation progress |
This report is **fully derived** — no unique data model. It aggregates failure data from other reports and overlays business context.
Report 6: How It Works
**Data models:** `SAP_DATA_SOURCES`
Static informational page describing the testing methodology. No dynamic data.
Report 7: Performance Heatmap
**Data models:** `PERF_DATA` (processes, transactions, batchJobs, targets)
| Field | Source | Detail |
| --------------------- | ------------------- | ---------------------------------------------- |
| **Process-level** | | |
| `status` | Playwright | Computed from transaction performance vs SLA |
| `e2eTime` | Playwright | Sum of all transaction avg times |
| `avgMemory` | Playwright | `performance.measureUserAgentSpecificMemory()` |
| `avgCpu` | Playwright | `PerformanceObserver` long-task monitoring |
| `avgNetwork` | Playwright | `PerformanceResourceTiming` network component |
| **Transaction-level** | | |
| `avgTime` | Playwright | Mean response time across all test executions |
| `p95Time` | Playwright | 95th percentile response time |
| `sla` | Config | SLA target per transaction |
| `eccBaseline` | SAP Backend (ST03N) | Historical ECC response time |
| `trend` | Playwright | Compare current avg to previous runs |
| `samples` | Playwright | Total test executions for this TCode |
| **Batch Jobs** | | |
| `s4Time` | SAP Backend (SM37) | Job runtime from SM37 job log |
| `eccTime` | SAP Backend (ST03N) | Historical ECC batch job runtime |
| `memory`, `cpu` | SAP Backend (ST06) | Server-side resource consumption |
:::info
Transaction response times come from Playwright (client-side), while batch job times come from SAP SM37 (server-side). Praman measures what the _user_ experiences; SM37 measures what the _server_ does.
:::
Report 8: Geographic & Customer Impact
**Data models:** `GLOBAL_REGIONS`, `SAP_DATA_SOURCES`
| Field | Source | Detail |
| ------------------------------------- | ------------------- | ------------------------------------------------- |
| **Country metadata** | | |
| `users` | Web Analytics | Active user count per country |
| `companyCode`, `locale`, `currency` | Config | SAP org structure |
| `wave` | Config | Rollout wave assignment |
| `peakHour` | SAP Backend (ST03N) | Peak usage hour from workload monitor |
| **Network profile** | | |
| `network.latency` | Focused Run RUM | Client-to-server latency per geography |
| `network.bandwidth` | Focused Run RUM | Measured throughput |
| **Browser/Device profile** | | |
| `browser.Chrome` | Focused Run RUM | Browser market share per country |
| `device.desktop` | Focused Run RUM | Device type distribution |
| **Simulation results** | | |
| `simulation.transactions[].optimal` | SAP Backend (ST03N) | Baseline from low-latency location |
| `simulation.transactions[].simulated` | Playwright | Measured time with country network/locale profile |
| `simulation.overallScore` | Playwright | Weighted score from all simulated transactions |
| `simulation.avgDegradation` | Playwright | ((simulated - optimal) / optimal) x 100 |
```javascript
// For each country, Praman creates a context matching real conditions
const deContext = await browser.newContext({
locale: 'de-DE',
timezoneId: 'Europe/Berlin',
viewport: { width: 1920, height: 1080 },
geolocation: { latitude: 50.1, longitude: 8.7 },
});
// Run same test scenarios — difference = geographic degradation
```
Report 9: Real User Experience Simulation
**Data models:** `GLOBAL_REGIONS`, `LOCALE_VALIDATION`, `BROWSER_COMPAT`
| Field | Source | Detail |
| ---------------------------------- | ---------- | ----------------------------------------------------- |
| `LOCALE_VALIDATION[].dateFormat` | Playwright | Validated by entering dates and checking rendering |
| `LOCALE_VALIDATION[].numberFormat` | Playwright | Validated by entering amounts and checking formatting |
| `BROWSER_COMPAT[].browser` | Config | Browser matrix for testing |
| `BROWSER_COMPAT[].status` | Playwright | Full suite execution on this browser |
| `BROWSER_COMPAT[].issues` | Playwright | Count of browser-specific failures |
Same test suite run across Chrome, Firefox, WebKit (Safari), Edge with different locale settings and responsive viewport testing.
Report 10: SAP FI — Financial Accounting Quality
**Data models:** `FI_QUALITY_DATA`
| Field | Source | Detail |
| ----------------------------- | ------------------ | ------------------------------------------------------------------ |
| **Document posting tests** | | |
| `postingScenarios` | Config | FI posting test catalog (FB01, F-02, VF01, MIRO) |
| `passed`, `failed`, `blocked` | Playwright | Test execution results per posting type |
| `responseTime` | Playwright | Time to complete posting including SAP roundtrip |
| **Balance verification** | | |
| `trialBalance.match` | Playwright + SAP | Automated comparison after posting |
| `intercompanyBalance` | Playwright + SAP | IC reconciliation verification |
| **Period-end close** | | |
| `closingTasks[].status` | Playwright | Automated tests for period-end transactions (F.01, S_ALR_87012284) |
| `closingTasks[].runtime` | SAP Backend (SM37) | Batch job execution time |
| **Tax & compliance** | | |
| `taxCalculation.accuracy` | Playwright | Tax determination results vs expected |
| `withholdingTax.status` | Playwright | WHT posting and reporting tests |
| `regulatoryReports` | Config | Country-specific reporting requirements |
Report 11: WCAG 2.1 AA Accessibility Compliance
**Data models:** `ACCESSIBILITY_DATA`
| Field | Source | Detail |
| ---------------------------------------- | --------------------- | ------------------------------------------------ |
| `apps[].score` | Playwright (axe-core) | Weighted score from violation count and severity |
| `apps[].violations` | Playwright (axe-core) | Total axe-core violations detected |
| `apps[].critical`, `serious`, `moderate` | Playwright (axe-core) | Violations by severity level |
| `apps[].screenReader` | Playwright | ARIA roles, live regions, focus management |
| `apps[].keyboard` | Playwright | Tab reachability, focus traps, Esc behavior |
| `violationsByType[].rule` | Playwright (axe-core) | axe rule ID (e.g. `color-contrast`) |
| `violationsByType[].principle` | Playwright (axe-core) | WCAG principle mapping |
| `regulatoryRequirements[]` | Config | Legal/compliance team input (EAA, Section 508) |
```javascript
test('Fiori app accessibility', async ({ page }) => {
await page.goto('/sap/bc/ui5_ui5/.../FioriLaunchpad.html#PurchaseOrder-manage');
await page.waitForSelector('sap-ui-content');
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
.analyze();
// Produces: violations[], passes[], incomplete[]
// Each violation: id, impact, description, nodes[]{html, target, failureSummary}
});
```
Report 12: Interface & Integration Quality
**Data models:** `INTERFACE_DATA`
| Field | Source | Detail |
| ----------------------------------------- | ---------------------- | -------------------------------------------- |
| **Interface catalog** | | |
| `id`, `name`, `type` | Config | Integration team documentation |
| `source`, `target`, `direction` | Config | Integration architecture |
| `frequency`, `criticality` | Config | Business classification |
| `dailyVolume` | 3rd Party + SAP | CPI message log count |
| `avgLatency` | 3rd Party + Playwright | API timing or CPI processing duration |
| `errorRate` | 3rd Party + SAP | Error messages / total messages |
| `status` | Playwright + 3rd Party | Computed from errorRate + latency vs SLA |
| **Third-party system health** | | |
| `thirdPartySystems[].status` | 3rd Party + Playwright | Aggregated from interface statuses |
| `thirdPartySystems[].uptime` | 3rd Party + SAP | CPI message success rate over rolling window |
| **Stability history** | | |
| `stabilityHistory[].totalUp`, `totalDown` | Playwright + 3rd Party | Weekly pass/fail aggregation |
| `interfaceStability[].weeksDown` | Playwright | Consecutive weeks failing |
Report 13: End-to-End Cross-System Quality
**Data models:** `E2E_SCENARIOS`
| Field | Source | Detail |
| ----------------------- | ---------------------- | ------------------------------------ |
| `id`, `name`, `process` | Config | Scenario catalog |
| `status` | Playwright | Overall: fail if any step fails |
| `criticality` | Config | Business classification |
| `sla` | Config | End-to-end SLA target |
| `revenueAtRisk` | Config | Finance team estimate |
| `steps[].system` | Config | System in the chain |
| `steps[].status` | Playwright + 3rd Party | Result of testing this specific step |
| `steps[].time` | Playwright | Measured execution time |
| `steps[].interface` | Config | Interface ID (links to Report 12) |
| `e2eTime` | Playwright | Sum of all step times |
| `dataIntegrity` | Playwright | Values match end-to-end |
---
## Data Flow Architecture
```text
+-----------------------------------------------------------------+
| DATA SOURCE LAYER |
+---------------+--------------+-------------+--------+-----------+
| Playwright | SAP Backend | Focused | 3rd | Config |
| + Praman | ST03N/STAD | Run / Web | Party | YAML/JSON |
| + axe-core | SM37/SM59 | Analytics | APIs | files |
+---------------+--------------+-------------+--------+-----------+
|
PRAMAN AGGREGATION ENGINE
Collects > Normalizes > Computes Scores > Tracks History
|
OUTPUT: dashboard-data.json
Fed into React dashboard (Docusaurus / standalone)
+-----------------------------------------------------------------+
```
## What Percentage Comes From Where?
Estimated across all ~180 unique data fields in the dashboard:
| Source | Field Count | % | What |
| ------------------------------- | ----------- | ------- | ----------------------------------------------------------------------------------------------------- |
| **Playwright + Praman** | ~75 fields | **42%** | Test results, response times, pass/fail, accessibility scans, memory/CPU/network, E2E chain status |
| **Configuration** | ~55 fields | **30%** | Process catalog, SLA targets, business impact, interface catalog, role mappings, remediation guidance |
| **SAP Backend** | ~25 fields | **14%** | ECC baselines (ST03N), batch job times (SM37), record counts (CDS), peak hours |
| **Focused Run / Web Analytics** | ~15 fields | **8%** | Geographic distribution, network latency, browser/device stats |
| **Third-Party APIs** | ~10 fields | **6%** | External system health, CPI message logs, file delivery status |
## Implementation Priority
### Phase 1 — Core (Reports 1-3, 5-7)
1. **Playwright test execution results** — pass/fail/blocked, response times, error details
2. **Config files** — process catalog, SLA targets, test scenario definitions
3. **Basic SAP telemetry** — ST03N baselines for ECC comparison
### Phase 2 — Depth (Reports 4, 8-9)
4. **Migration validation queries** — CDS views for record counts and error detection
5. **Focused Run / Web Analytics** — geographic profiles for realistic simulation
6. **Locale/browser matrix testing** — multi-context Playwright execution
### Phase 3 — Enterprise (Reports 10-13)
7. **axe-core integration** — accessibility scanning per Fiori app
8. **CPI monitoring API** — interface health, message volumes, error rates
9. **Third-party API health checks** — external system connectivity validation
10. **E2E chain orchestration** — multi-system flow testing with Playwright as orchestrator
## What Praman Needs to Ship
| Praman Feature | Reports It Feeds |
| --------------------------------------------------------- | ---------------- |
| Test execution reporter (pass/fail/time/error) | 1, 2, 3, 5, 7 |
| `page.evaluate()` SAP UI5 performance extraction | 7 |
| Browser Performance API collection (memory/CPU/network) | 7, 8 |
| Multi-context execution (locale/browser/viewport/network) | 8, 9 |
| @axe-core/playwright integration | 11 |
| HTTP request interception + timing (route handler) | 12, 13 |
| CPI OData log fetcher | 12, 13 |
| Config file parser (YAML/JSON process catalog) | All |
| Historical result aggregator (weekly trends, stability) | 1, 7, 12 |
| Dashboard data JSON generator | All |
---
## Test Data Management
Reliable SAP UI5 tests require predictable, isolated test data. Praman provides a `TestDataHandler` fixture for generating templated data, persisting it for cross-step reuse, and cleaning it up automatically on teardown.
## TestDataHandler Fixture
The `testData` fixture is available in every Praman test. It generates data from templates using placeholder tokens that are replaced at runtime:
- `{{uuid}}` -- a unique identifier (via `node:crypto.randomUUID()`)
- `{{timestamp}}` -- an ISO-8601 timestamp at generation time
```typescript
test('create a purchase order with unique data', async ({ ui5, testData }) => {
const order = testData.generate({
orderId: 'PO-{{uuid}}',
description: 'Test order created at {{timestamp}}',
amount: 1500,
});
await test.step('Fill order form', async () => {
await ui5.fill({ id: 'orderIdInput' }, order.orderId);
await ui5.fill({ id: 'descriptionInput' }, order.description);
});
await test.step('Verify order was created', async () => {
await expect(ui5.control({ id: 'orderIdDisplay' })).toHaveText(order.orderId);
});
});
```
## Persisting and Loading Data
For multi-step tests that span navigation or page reloads, persist generated data to disk and reload it later:
```typescript
test('create and verify order across pages', async ({ ui5, testData }) => {
const order = testData.generate({ id: '{{uuid}}', vendor: 'ACME-{{uuid}}' });
await test.step('Save order data for later steps', async () => {
await testData.save('order.json', order);
});
// ... navigate to a different page ...
await test.step('Load order data on verification page', async () => {
const saved = await testData.load<{ id: string; vendor: string }>('order.json');
await expect(ui5.control({ id: 'vendorDisplay' })).toHaveText(saved.vendor);
});
});
```
## Automatic Cleanup
The `testData` fixture tracks all files created via `save()`. On test teardown, call `cleanup()` to remove them:
```typescript
test('data files are cleaned up after test', async ({ testData }) => {
const item = testData.generate({ sku: 'SKU-{{uuid}}' });
await testData.save('item.json', item);
// cleanup() is called automatically during fixture teardown
});
```
If you manage `TestDataHandler` manually (outside fixtures), call `cleanup()` explicitly in your teardown logic.
## Data Isolation Across Parallel Workers
When running tests in parallel with multiple Playwright workers, each worker gets its own `testData` instance with an isolated `baseDir`. This prevents file collisions:
```typescript
// praman.config.ts
export default {
testData: {
baseDir: './test-data', // Each worker appends its workerIndex
},
};
```
Use `{{uuid}}` tokens in entity identifiers to avoid collisions at the SAP backend level as well. Two parallel workers creating a purchase order with the same ID will cause OData conflicts.
## OData Entity Creation for Test Setup
For integration tests against a live SAP system, create test entities via OData before exercising the UI:
```typescript
test('edit an existing order', async ({ ui5, page, testData }) => {
const order = testData.generate({ OrderID: '{{uuid}}', Status: 'Draft' });
await test.step('Create test entity via OData', async () => {
const response = await page.request.post('/sap/opu/odata/sap/ORDER_SRV/Orders', {
data: order,
headers: { 'Content-Type': 'application/json' },
});
expect(response.ok()).toBe(true);
});
await test.step('Navigate to order and edit', async () => {
await page.goto(`/app#/Orders/${order.OrderID}`);
await ui5.waitForUI5();
await ui5.fill({ id: 'statusField' }, 'Approved');
await ui5.click({ id: 'saveBtn' });
});
});
```
## Factory Patterns
For complex entity structures, create reusable factory functions:
```typescript
function createOrderTemplate(overrides: Record = {}) {
return {
OrderID: '{{uuid}}',
CreatedAt: '{{timestamp}}',
Status: 'Draft',
Currency: 'USD',
Items: [{ ItemID: '{{uuid}}', Quantity: 1 }],
...overrides,
};
}
test('order with custom status', async ({ testData }) => {
const order = testData.generate(createOrderTemplate({ Status: 'Approved' }));
// order.Status === 'Approved', order.OrderID === 'a1b2c3d4-...'
});
```
## Common Pitfalls
- **Hardcoded IDs in parallel runs**: Always use `{{uuid}}` for entity identifiers. Static IDs like `"TEST-001"` cause conflicts when workers hit the same SAP backend.
- **Stale persisted files**: If a previous test run crashed before cleanup, leftover files in `baseDir` can confuse subsequent runs. Use a CI step to clear the test-data directory before each run.
- **OData entity cleanup**: The `testData.cleanup()` method removes local files only. For OData entities created on the server, implement a `test.afterAll()` hook that sends DELETE requests.
- **Template nesting**: Placeholders work recursively in nested objects and arrays, but only on string values. A numeric field like `amount: 100` passes through unchanged.
---
## Parallel Execution & Sharding
Running SAP UI5 tests in parallel dramatically reduces feedback time. This guide covers Playwright's parallelization model, SAP-specific considerations for auth and locking, and CI sharding strategies.
## Enabling Parallel Execution
Playwright supports parallelism at two levels: across test files (workers) and within a file (fullyParallel). Enable both in your config:
```typescript
export default defineConfig({
fullyParallel: true,
workers: process.env.CI ? 4 : undefined, // undefined = half CPU cores
});
```
With `fullyParallel: true`, individual `test()` blocks within the same file can run concurrently. Without it, tests in a single file run sequentially while different files run in parallel across workers.
## Worker Isolation with Auth State
Each Playwright worker is an independent Node.js process. For SAP tests, share authentication state across workers using Playwright's `storageState` mechanism:
```typescript
export default defineConfig({
projects: [
{
name: 'auth-setup',
testMatch: /auth\.setup\.ts/,
},
{
name: 'sap-tests',
dependencies: ['auth-setup'],
use: {
storageState: '.auth/user.json',
},
},
],
});
```
The `auth-setup` project runs once before all workers. Each worker then reuses the saved cookies and session tokens without re-authenticating.
## SAP Fiori Launchpad Lock Considerations
SAP systems enforce object-level locking. When two parallel workers attempt to edit the same business object (e.g., a sales order), one will encounter an SAP lock error (`ENQUEUE_CONFLICT`).
Strategies to avoid lock conflicts:
- **Unique test data per worker**: Generate unique entity IDs with `{{uuid}}` so workers never edit the same object.
- **Read-only vs. write tests**: Separate display-only tests from edit tests into different projects. Run edit tests with `workers: 1`.
- **Dedicated test data pools**: Assign each worker its own pre-created entities using `workerIndex`:
```typescript
test('edit order assigned to this worker', async ({ ui5 }, testInfo) => {
const orderPool = ['ORD-001', 'ORD-002', 'ORD-003', 'ORD-004'];
const myOrder = orderPool[testInfo.workerIndex % orderPool.length];
await ui5.navigate(`/app#/Orders/${myOrder}`);
await ui5.waitForUI5();
});
```
## Praman Bridge in Parallel Workers
The Praman bridge is injected per-page, not per-worker. Each worker creates its own browser context and page, so bridge injection is fully independent. No special configuration is needed for the bridge in parallel mode.
One consideration: if `controlDiscoveryTimeout` is too low and the SAP system is under load from multiple workers, increase it:
```typescript
export default {
controlDiscoveryTimeout: 45_000, // 45 seconds under parallel load
};
```
## CI Sharding with GitHub Actions
Playwright's built-in sharding distributes test files across multiple CI jobs:
```yaml
# .github/workflows/test.yml
jobs:
test:
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npx playwright test --shard=${{ matrix.shard }}/4
- uses: actions/upload-artifact@v4
if: always()
with:
name: results-shard-${{ matrix.shard }}
path: test-results/
```
## Auth Setup for Sharded Runs
When using sharding, every shard must have access to auth state. Use `globalSetup` instead of project dependencies so each shard runs its own login:
```typescript
export default defineConfig({
globalSetup: './global-setup.ts', // runs once per shard
});
```
## Common Pitfalls
- **Too many workers against SAP**: SAP systems have session limits. If you see `ERR_AUTH_FAILED` errors in parallel runs, reduce `workers` or use more test accounts.
- **Shared mutable state**: Never store test state in module-level variables. Each worker is a separate process, so module state is not shared. Use `testData.save()` and `testData.load()` for cross-step persistence.
- **FLP tile cache**: The Fiori Launchpad caches tile data per session. Parallel workers reusing the same `storageState` share the same FLP cache, which is generally safe for read operations.
- **Shard imbalance**: Playwright distributes by file, not by test count. If one file has 50 tests and others have 5, sharding will be uneven. Keep test files roughly equal in size.
---
## Custom Control Extension
SAP projects frequently use custom UI5 controls that extend standard controls or create entirely new ones. This guide explains how to interact with custom controls through Praman when they are not part of the built-in type system.
## Finding Custom Controls by Type
Use the `controlType` selector to target custom controls by their fully qualified UI5 class name:
```typescript
test('interact with custom rating control', async ({ ui5 }) => {
const rating = await ui5.control({
controlType: 'com.myapp.controls.StarRating',
properties: { value: 4 },
});
await expect(rating).toBeDefined();
});
```
The `controlType` selector searches the UI5 control tree by class name. It works for any control registered with `sap.ui.define`, including third-party and project-specific controls.
## Combining controlType with Properties
When multiple instances of a custom control exist on the page, narrow the match with `properties`:
```typescript
await ui5.control({
controlType: 'com.myapp.controls.StatusBadge',
properties: {
status: 'Critical',
category: 'Inventory',
},
});
```
Properties are matched against the control's current property values using the UI5 `getProperty()` API. Property names must match the control's metadata definition exactly.
## Accessing Inner Controls of Composites
Many custom controls are composites that wrap standard UI5 controls internally. For example, a `SmartField` wraps an `Input`, `ComboBox`, or `DatePicker` depending on the OData metadata.
### The SmartField Pattern
`SmartField` is a common source of confusion. Its `getControlType()` returns `sap.ui.comp.smartfield.SmartField`, not the inner control type:
```typescript
test('interact with SmartField inner control', async ({ ui5 }) => {
await test.step('Set value on SmartField', async () => {
// Target the SmartField directly -- Praman handles the inner control
await ui5.fill({ id: 'vendorSmartField' }, 'ACME Corp');
await ui5.waitForUI5();
});
await test.step('Read value from SmartField', async () => {
const value = await ui5.getValue({ id: 'vendorSmartField' });
expect(value).toBe('ACME Corp');
});
});
```
### Accessing the Inner Control Directly
If you need to interact with the inner control specifically (e.g., to check its type or invoke a method not exposed by the wrapper):
```typescript
test('access SmartField inner control type', async ({ ui5, page }) => {
const innerType = await page.evaluate(() => {
const sf = sap.ui.getCore().byId('vendorSmartField');
const inner = sf?.getInnerControls?.()[0];
return inner?.getMetadata().getName();
});
expect(innerType).toBe('sap.m.Input');
});
```
## Extending ControlProxy for Custom Methods
When a custom control has domain-specific methods you call frequently, wrap the interactions in test helper functions:
```typescript
// test-helpers/custom-controls.ts
export async function setStarRating(page: Page, controlId: string, stars: number) {
await page.evaluate(
({ id, value }) => {
const control = sap.ui.getCore().byId(id);
control?.setProperty('value', value);
control?.fireEvent('change', { value });
},
{ id: controlId, value: stars },
);
}
```
Use it in tests:
```typescript
test('rate a product', async ({ ui5, page }) => {
await setStarRating(page, 'productRating', 5);
await ui5.waitForUI5();
await expect(ui5.control({ id: 'ratingDisplay' })).toHaveText('5 / 5');
});
```
## Handling Controls with Custom Renderers
Some custom controls override the standard renderer and produce non-standard DOM. When Praman's standard selectors cannot find a visible element, fall back to properties-based selectors:
```typescript
test('interact with custom-rendered control', async ({ ui5 }) => {
// Instead of relying on DOM structure, use UI5 control tree
await ui5.click({
controlType: 'com.myapp.controls.CustomButton',
properties: { text: 'Submit Order' },
});
});
```
This approach works because Praman queries the UI5 control tree via the bridge, not the DOM. The control's rendered output does not affect selector resolution.
## Common Pitfalls
- **Assuming inner control type**: A `SmartField` on an Edm.String property wraps `sap.m.Input`, but on an Edm.DateTime it wraps `sap.m.DatePicker`. Always verify against the live system.
- **Using DOM selectors for custom controls**: Custom renderers produce unpredictable DOM. Always prefer `controlType` + `properties` selectors over CSS selectors.
- **Missing fireEvent after setValue**: If your custom control listens for `change` or `liveChange` events, calling `setProperty()` alone may not trigger downstream logic. Use `fireEvent()` explicitly.
- **Namespace typos**: Custom control class names are case-sensitive. `com.myapp.Controls.StatusBadge` (capital C) is different from `com.myapp.controls.StatusBadge`.
---
## Localization & i18n Testing
SAP applications serve a global user base. Testing across languages, locales, and text directions ensures that UI elements render correctly regardless of the user's language settings. This guide covers language-independent selectors, locale switching, RTL testing, and format validation.
## Language-Independent Selectors with i18NText
Hardcoded text selectors like `{ text: 'Save' }` break when the app runs in German (`Sichern`) or Japanese. Praman's `i18NText` selector matches controls by their i18n resource bundle key instead of the displayed text:
```typescript
test('click save button regardless of language', async ({ ui5 }) => {
await ui5.click({
controlType: 'sap.m.Button',
i18NText: { propertyName: 'text', key: 'SAVE_BUTTON' },
});
});
```
The `i18NText` selector resolves the resource bundle key at runtime, matching the control whose bound text property points to that key. This makes tests language-agnostic.
### How i18NText Resolution Works
1. Praman reads the control's binding info for the specified property (`text`).
2. It checks whether the binding references a resource bundle model (typically `i18n`).
3. It compares the bound key against the `key` parameter you provided.
4. If they match, the control is selected -- regardless of what the displayed text says.
```typescript
// Match by specific resource bundle model name
await ui5.click({
controlType: 'sap.m.Button',
i18NText: {
propertyName: 'text',
key: 'SUBMIT_ORDER',
modelName: 'i18n', // optional, defaults to 'i18n'
},
});
```
## Switching Languages via SAP User Settings
To test a specific locale, set the `sap-language` URL parameter before navigating:
```typescript
test('verify German labels', async ({ ui5, page }) => {
await page.goto('/app?sap-language=DE');
await ui5.waitForUI5();
await test.step('Verify German text on save button', async () => {
const buttonText = await ui5.getText({
controlType: 'sap.m.Button',
i18NText: { propertyName: 'text', key: 'SAVE_BUTTON' },
});
expect(buttonText).toBe('Sichern');
});
});
```
### Testing Multiple Languages
Use parameterized tests to verify translations across locales:
```typescript
const locales = [
{ lang: 'EN', expected: 'Save' },
{ lang: 'DE', expected: 'Sichern' },
{ lang: 'FR', expected: 'Enregistrer' },
];
for (const { lang, expected } of locales) {
test(`save button label in ${lang}`, async ({ ui5, page }) => {
await page.goto(`/app?sap-language=${lang}`);
await ui5.waitForUI5();
const text = await ui5.getText({
controlType: 'sap.m.Button',
i18NText: { propertyName: 'text', key: 'SAVE_BUTTON' },
});
expect(text).toBe(expected);
});
}
```
## FLP Language Parameter
When testing inside the SAP Fiori Launchpad, set the language via the FLP URL hash parameter:
```typescript
test('open FLP in French', async ({ page, ui5 }) => {
await page.goto('/sap/bc/ui2/flp?sap-language=FR#Shell-home');
await ui5.waitForUI5();
});
```
The `sap-language` parameter must appear before the `#` hash. It affects all apps launched from the FLP during that session.
## RTL Layout Testing
Arabic, Hebrew, and other right-to-left (RTL) languages require layout verification. SAP UI5 automatically mirrors the layout when an RTL locale is active:
```typescript
test('verify RTL layout for Arabic', async ({ ui5, page }) => {
await page.goto('/app?sap-language=AR');
await ui5.waitForUI5();
await test.step('Check document direction', async () => {
const dir = await page.evaluate(() => document.documentElement.dir);
expect(dir).toBe('rtl');
});
await test.step('Verify toolbar alignment', async () => {
const toolbar = page.locator('.sapMTB');
await expect(toolbar).toHaveCSS('direction', 'rtl');
});
});
```
## Date and Number Format Validation
SAP formats dates and numbers according to the user's locale. Validate that formatted values match locale expectations:
```typescript
test('date format matches German locale', async ({ ui5, page }) => {
await page.goto('/app?sap-language=DE');
await ui5.waitForUI5();
const dateValue = await ui5.getText({ id: 'createdAtField' });
// German date format: DD.MM.YYYY
expect(dateValue).toMatch(/^\d{2}\.\d{2}\.\d{4}$/);
});
test('number format matches French locale', async ({ ui5, page }) => {
await page.goto('/app?sap-language=FR');
await ui5.waitForUI5();
const amount = await ui5.getText({ id: 'totalAmount' });
// French uses comma as decimal separator and space as thousands separator
expect(amount).toMatch(/[\d\s]+,\d{2}/);
});
```
## Common Pitfalls
- **Hardcoded text assertions**: Never assert against hardcoded display text in multi-locale tests. Use `i18NText` for selectors and verify the text only when explicitly testing a specific locale.
- **Missing translations**: If an i18n key is missing from a locale's `.properties` file, UI5 falls back to the default (usually English). Your test might pass even though the translation is absent.
- **RTL pseudo-mirrors**: Some CSS properties (like `margin-left`) are automatically mirrored by UI5's RTL support, but custom CSS may not be. Test RTL with actual RTL locales.
- **sap-language vs. browser locale**: The `sap-language` URL parameter overrides the browser's `Accept-Language` header. If you need browser-locale-based behavior, set the locale in the Playwright browser context instead.
- **Session stickiness**: Once the FLP establishes a language, navigating to a different app preserves it. Close and reopen the browser context to reset the language cleanly.
---
## Security Testing Patterns
SAP applications handle sensitive business data and must enforce strict security boundaries. This guide covers automated security testing patterns with Praman, including CSRF validation, XSS input testing, authorization boundaries, and session handling.
## CSRF Token Validation
SAP UI5 applications use CSRF (Cross-Site Request Forgery) tokens for write operations. Verify that the application correctly fetches and sends CSRF tokens:
```typescript
test('CSRF token is sent with POST requests', async ({ page, ui5 }) => {
await ui5.waitForUI5();
const csrfRequests: { url: string; token: string | null }[] = [];
page.on('request', (request) => {
if (request.method() === 'POST') {
csrfRequests.push({
url: request.url(),
token: request.headers()['x-csrf-token'] ?? null,
});
}
});
await test.step('Perform a write operation', async () => {
await ui5.fill({ id: 'nameInput' }, 'Test Value');
await ui5.click({ id: 'saveBtn' });
await ui5.waitForUI5();
});
await test.step('Verify CSRF token was included', async () => {
const postRequests = csrfRequests.filter((r) => r.url.includes('/odata/'));
expect(postRequests.length).toBeGreaterThan(0);
for (const req of postRequests) {
expect(req.token).not.toBeNull();
}
});
});
```
## XSS Input Testing
Test that the application properly sanitizes user input to prevent cross-site scripting:
```typescript
test('XSS payloads are sanitized in input fields', async ({ ui5, page }) => {
const xssPayloads = [
'',
'',
"javascript:alert('xss')",
];
for (const payload of xssPayloads) {
await test.step(`Test payload: ${payload.slice(0, 30)}...`, async () => {
await ui5.fill({ id: 'descriptionInput' }, payload);
await ui5.waitForUI5();
const innerHTML = await page.locator('#descriptionInput').innerHTML();
expect(innerHTML).not.toContain('