# Praman Quick Start Guide > Essential setup and core concepts: installation, configuration, fixtures, selectors, matchers. This file contains all documentation content in a single document following the llmstxt.org standard. ## Basic Test # Basic UI5 Control Discovery The simplest possible Praman test. Demonstrates control discovery by type and property, property verification, and `ui5.press()`. ## Source ```typescript test.describe('Basic UI5 Control Discovery', () => { test('find a GenericTile by header text', async ({ page, ui5 }) => { await test.step('Navigate to Fiori Launchpad', async () => { await page.goto(process.env['SAP_CLOUD_BASE_URL']!); await page.waitForLoadState('domcontentloaded'); await expect(page).toHaveTitle(/Home/, { timeout: 60_000 }); }); await test.step('Discover tile control', async () => { // Use ui5.control() to find a GenericTile by its header property. // Praman injects the UI5 bridge automatically on first call. const tile = await ui5.control({ controlType: 'sap.m.GenericTile', properties: { header: 'My App' }, }); // Verify the control type returned by the bridge const controlType = await tile.getControlType(); expect(controlType).toBe('sap.m.GenericTile'); // Read a property via the typed proxy const header = await tile.getProperty('header'); expect(header).toBe('My App'); }); await test.step('Press the tile to navigate', async () => { // ui5.press() calls firePress() on the control and waits for UI5 stability await ui5.press({ controlType: 'sap.m.GenericTile', properties: { header: 'My App' }, }); // Wait for the target app to finish loading await ui5.waitForUI5(); }); }); }); ``` ## Key Concepts - **`ui5.control()`** discovers controls through the UI5 runtime's control registry -- not the DOM - **`controlType` + `properties`** is the most common selector pattern - **`ui5.press()`** fires the UI5 press event and automatically calls `waitForUI5()` - **`test.step()`** groups actions for clear Playwright HTML reports --- ## Configuration Reference Praman uses a Zod-validated configuration system. All fields have defaults — an empty `{}` is valid. ## Config File Create `praman.config.ts` in your project root: ```typescript export default defineConfig({ logLevel: 'info', ui5WaitTimeout: 30_000, controlDiscoveryTimeout: 10_000, interactionStrategy: 'ui5-native', discoveryStrategies: ['direct-id', 'recordreplay'], auth: { strategy: 'basic', baseUrl: process.env.SAP_CLOUD_BASE_URL!, username: process.env.SAP_CLOUD_USERNAME!, password: process.env.SAP_CLOUD_PASSWORD!, client: '100', language: 'EN', }, ai: { provider: 'azure-openai', apiKey: process.env.AZURE_OPENAI_KEY, endpoint: process.env.AZURE_OPENAI_ENDPOINT, deployment: 'gpt-4o', }, telemetry: { openTelemetry: true, exporter: 'otlp', endpoint: 'http://localhost:4318', serviceName: 'sap-e2e-tests', metrics: true, }, }); ``` ## Top-Level Options | Field | Type | Default | Description | | ------------------------- | ----------------------------------------------------- | ------------------------------- | ---------------------------------------------------- | | `logLevel` | `'error' \| 'warn' \| 'info' \| 'debug' \| 'verbose'` | `'info'` | Pino log level | | `ui5WaitTimeout` | `number` | `30_000` | Max ms to wait for UI5 stability | | `controlDiscoveryTimeout` | `number` | `10_000` | Max ms for control discovery | | `interactionStrategy` | `'ui5-native' \| 'dom-first' \| 'opa5'` | `'ui5-native'` | How to interact with controls | | `discoveryStrategies` | `string[]` | `['direct-id', 'recordreplay']` | Ordered strategy chain | | `skipStabilityWait` | `boolean` | `false` | Skip `waitForUI5Stable()`, use brief DOM settle | | `preferVisibleControls` | `boolean` | `true` | Prefer visible controls over hidden ones | | `ignoreAutoWaitUrls` | `string[]` | `[]` | Additional URL patterns to block (WalkMe, analytics) | ## Auth Sub-Schema | Field | Type | Default | Description | | ---------- | -------------------------------------------------- | ---------- | ----------------------- | | `strategy` | `'btp-saml' \| 'basic' \| 'office365' \| 'custom'` | `'basic'` | Authentication strategy | | `baseUrl` | `string` (URL) | _required_ | SAP system base URL | | `username` | `string` | — | Login username | | `password` | `string` | — | Login password | | `client` | `string` | `'100'` | SAP client number | | `language` | `string` | `'EN'` | SAP logon language | ## AI Sub-Schema | Field | Type | Default | Description | | ----------------- | ------------------------------------------- | ---------------- | ---------------------------------------------------- | | `provider` | `'azure-openai' \| 'openai' \| 'anthropic'` | `'azure-openai'` | LLM provider | | `apiKey` | `string` | — | OpenAI / Azure OpenAI API key | | `anthropicApiKey` | `string` | — | Anthropic API key (separate for security) | | `model` | `string` | — | Model name (e.g., `'gpt-4o'`, `'claude-sonnet-4-6'`) | | `temperature` | `number` | `0.3` | LLM temperature (0-2) | | `maxTokens` | `number` | — | Max tokens per response | | `endpoint` | `string` (URL) | — | Azure OpenAI resource endpoint | | `deployment` | `string` | — | Azure OpenAI deployment name | | `apiVersion` | `string` | — | Azure API version | ## Telemetry Sub-Schema Telemetry is disabled by default with zero overhead. See [Telemetry Setup](./telemetry) for the full guide. | Field | Type | Default | Description | | -------------------- | --------------------------------------- | --------------------- | ---------------------------------------------------- | | `openTelemetry` | `boolean` | `false` | Enable OpenTelemetry tracing | | `exporter` | `'otlp' \| 'azure-monitor' \| 'jaeger'` | `'otlp'` | Trace exporter | | `endpoint` | `string` (URL) | — | OTLP/Jaeger endpoint URL (required when enabled) | | `serviceName` | `string` | `'playwright-praman'` | Service name in traces | | `protocol` | `'http' \| 'grpc'` | `'http'` | OTel transport protocol | | `metrics` | `boolean` | `false` | Enable metric counters and histograms | | `batchTimeout` | `number` | `5000` | Span batch export timeout (ms) | | `maxQueueSize` | `number` | `2048` | Max queued spans before dropping | | `resourceAttributes` | `Record` | `{}` | Additional OTel resource attributes | | `connectionString` | `string` | — | Azure Monitor connection string (required for Azure) | ## Selectors Sub-Schema | Field | Type | Default | Description | | ----------------------- | --------- | -------- | --------------------------------- | | `defaultTimeout` | `number` | `10_000` | Default selector timeout (ms) | | `preferVisibleControls` | `boolean` | `true` | Prefer visible controls | | `skipStabilityWait` | `boolean` | `false` | Skip stability wait for selectors | ## OPA5 Sub-Schema | Field | Type | Default | Description | | -------------------- | --------- | ------- | ----------------------------- | | `interactionTimeout` | `number` | `5_000` | OPA5 interaction timeout (ms) | | `autoWait` | `boolean` | `true` | Auto-wait for OPA5 readiness | | `debug` | `boolean` | `false` | Enable OPA5 debug mode | ## Timeout Precedence When multiple timeout values could apply, Praman uses this precedence (highest first): 1. **Per-call options** — `ui5.control({ id: 'x' }, { timeout: 5000 })` 2. **Top-level config** — `controlDiscoveryTimeout: 10000` 3. **Selectors section** — `selectors.defaultTimeout: 10000` (fallback if top-level not set) 4. **Built-in default** — 10,000ms We recommend using only `controlDiscoveryTimeout` at the top level. `selectors.defaultTimeout` exists for backward compatibility and will be removed in a future major version. For details on the 3-tier stability wait system, see [Stability & Timing](./stability-timing.md). ## Environment Variable Overrides Top-level config fields can be overridden via environment variables: | Environment Variable | Config Field | | ---------------------------------- | ------------------------- | | `PRAMAN_LOG_LEVEL` | `logLevel` | | `PRAMAN_UI5_WAIT_TIMEOUT` | `ui5WaitTimeout` | | `PRAMAN_CONTROL_DISCOVERY_TIMEOUT` | `controlDiscoveryTimeout` | | `PRAMAN_INTERACTION_STRATEGY` | `interactionStrategy` | | `PRAMAN_SKIP_STABILITY_WAIT` | `skipStabilityWait` | **General config precedence** (highest to lowest): 1. Per-call options (e.g., `ui5.control({ ... }, { timeout: 5000 })`) 2. Environment variable overrides 3. Top-level config 4. Schema defaults ## Complete `playwright.config.ts` Example ```typescript export default defineConfig({ testDir: './tests/e2e', timeout: 120_000, expect: { timeout: 10_000 }, fullyParallel: false, retries: 1, workers: 1, reporter: [ ['html', { open: 'never' }], ['playwright-praman/reporters', { type: 'compliance', outputDir: 'reports' }], ], use: { baseURL: process.env.SAP_CLOUD_BASE_URL, trace: 'on-first-retry', screenshot: 'only-on-failure', video: 'retain-on-failure', }, projects: [ { name: 'setup', testMatch: /auth-setup\.ts/, teardown: 'teardown', }, { name: 'teardown', testMatch: /auth-teardown\.ts/, }, { name: 'chromium', dependencies: ['setup'], use: { ...devices['Desktop Chrome'], storageState: '.auth/sap-session.json', }, }, ], }); ``` ## Validation Config is validated with Zod at load time. Invalid config fails fast: ```typescript const config = await loadConfig(); // Throws ConfigError with ERR_CONFIG_INVALID if schema fails // Returns Readonly — deeply frozen, mutation throws ``` --- ## UI5 Control Interactions :::info[In this guide] - Click buttons, fill inputs, and select dropdown values using the `ui5` fixture - Understand auto-waiting behavior and how it differs from Playwright's native auto-wait - Read control properties and text for assertions - Learn the three-tier wait sequence that runs during navigation - Skip stability waits for performance-critical paths ::: The `ui5` fixture provides high-level methods for interacting with UI5 controls. Each method discovers the control, waits for UI5 stability, and performs the action through the configured interaction strategy. :::warning[Common mistake] Do not use Playwright's native `page.click()` with DOM selectors to interact with UI5 controls. UI5 controls manage their own event lifecycle, and DOM-level clicks can bypass OData model updates, validation handlers, and change events. ```typescript // Wrong — bypasses UI5 event system await page.click('#__xmlview0--saveBtn'); // Correct — fires UI5 events, waits for stability await ui5.click({ id: 'saveBtn' }); ``` ::: :::tip Typed Control Returns When you use `controlType` in your selector, the returned control is typed to the specific interface (e.g., `UI5Button`, `UI5Input`). This gives you autocomplete for control-specific methods. See [Typed Control Returns](/docs/guides/typed-controls) for details. ::: ## Interaction Methods ### click Clicks a UI5 control. Uses the configured interaction strategy (UI5-native `firePress()` by default, with DOM click fallback). ```typescript await ui5.click({ id: 'submitBtn' }); await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Save' } }); ``` ### fill Sets the value of an input control. Fires both `liveChange` and `change` events to ensure OData model bindings update. ```typescript await ui5.fill({ id: 'vendorInput' }, '100001'); await ui5.fill({ controlType: 'sap.m.Input', viewName: 'myApp.view.Detail' }, 'New Value'); ``` ### press Triggers a press action on a control. Equivalent to `click` but semantically clearer for buttons, links, and tiles. ```typescript await ui5.press({ id: 'confirmBtn' }); ``` ### select Selects an item in a dropdown (Select, ComboBox, MultiComboBox). Accepts selection by key or display text. ```typescript await ui5.select({ id: 'countrySelect' }, 'US'); await ui5.select({ id: 'statusComboBox' }, 'Active'); ``` ### check / uncheck Toggles checkbox and switch controls. ```typescript await ui5.check({ id: 'agreeCheckbox' }); await ui5.uncheck({ id: 'optionalCheckbox' }); ``` ### clear Clears the value of an input control and fires change events. ```typescript await ui5.clear({ id: 'searchField' }); ``` ### getText Reads the text property of a control. ```typescript const label = await ui5.getText({ id: 'headerTitle' }); expect(label).toBe('Purchase Order Details'); ``` ### getProperty Reads any property from a UI5 control by name. ```typescript const btn = await ui5.control({ id: 'saveBtn' }); const enabled = await btn.getProperty('enabled'); const status = await ui5.control({ id: 'statusText' }); const visible = await status.getProperty('visible'); ``` ### waitForUI5 Manually triggers a UI5 stability wait. Normally called automatically, but useful after custom browser-side operations. ```typescript await ui5.click({ id: 'triggerLongLoad' }); await ui5.waitForUI5(30_000); // Custom timeout in milliseconds ``` ## Auto-Waiting Behavior Every `ui5.*` method calls `waitForUI5Stable()` before executing. This differs from Playwright's native auto-waiting in a critical way: | Aspect | Playwright Auto-Wait | Praman Auto-Wait | | --------------------- | -------------------------------------------- | -------------------------------------------------------- | | **What it waits for** | DOM actionability (visible, enabled, stable) | UI5 runtime pending operations (XHR, promises, bindings) | | **Scope** | Single element | Entire UI5 application | | **Source of truth** | DOM state | `sap.ui.getCore().getUIDirty()` + pending request count | | **When it fires** | Before each locator action | Before each `ui5.*` call | Playwright's auto-wait checks whether a specific DOM element is visible and enabled. Praman's auto-wait checks whether the entire UI5 framework has finished all asynchronous operations — this includes OData requests triggered by other controls, model binding updates, and view rendering. ### The Three-Tier Wait When a page navigation occurs, Praman executes a three-tier wait: 1. **waitForUI5Bootstrap** (60s) — waits for `window.sap.ui.getCore` to exist 2. **waitForUI5Stable** (15s) — waits for zero pending async operations 3. **briefDOMSettle** (500ms) — brief pause for final DOM rendering For subsequent interactions (not navigation), only `waitForUI5Stable` is called. ### Skipping the Stability Wait For performance-critical paths where you know UI5 is already stable: ```typescript export default defineConfig({ skipStabilityWait: true, // Uses briefDOMSettle (500ms) instead }); ``` ## Complete Example ```typescript test('edit and save a purchase order', async ({ ui5, ui5Navigation }) => { // Navigate to the app await ui5Navigation.navigateToApp('PurchaseOrder-manage'); // Click the first row to open details await ui5.click({ controlType: 'sap.m.ColumnListItem', ancestor: { id: 'poTable' }, }); // Edit mode await ui5.click({ id: 'editBtn' }); // Fill fields await ui5.fill({ id: 'vendorInput' }, '100002'); await ui5.select({ id: 'purchOrgSelect' }, '1000'); await ui5.check({ id: 'urgentCheckbox' }); // Clear and refill await ui5.clear({ id: 'noteField' }); await ui5.fill({ id: 'noteField' }, 'Updated via automated test'); // Read values back const vendor = await ui5.getText({ id: 'vendorInput' }); expect(vendor).toBe('100002'); // Save await ui5.click({ id: 'saveBtn' }); }); ``` ## Interaction Under the Hood Each interaction method follows this sequence: 1. **Stability guard** — `waitForUI5Stable()` ensures no pending async operations 2. **Control discovery** — finds the control via the multi-strategy chain (cache, direct-ID, RecordReplay, registry scan) 3. **Strategy execution** — delegates to the configured interaction strategy (ui5-native, dom-first, or opa5) 4. **Step decoration** — wraps the entire operation in a Playwright `test.step()` for HTML report visibility If the primary strategy method fails (e.g., `firePress()` not available), the strategy automatically falls back to its secondary method (e.g., DOM click). ## FAQ
Can I use page.click() instead of ui5.click()? You can, but you should not for UI5 controls. `page.click()` performs a DOM-level click that bypasses the UI5 event system. This means OData model bindings may not update, `liveChange` and `change` events may not fire, and validation logic may be skipped. Use `ui5.click()` for all UI5 controls. Reserve `page.click()` for verified non-UI5 DOM elements (e.g., custom shell header buttons).
How do I interact with controls inside a dialog? Pass `searchOpenDialogs: true` in your selector: ```typescript await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'OK' }, searchOpenDialogs: true, }); ``` Without this flag, Praman only searches the main view's control tree and will not find controls rendered inside dialog overlays.
Why does ui5.fill() fire both liveChange and change events? SAP UI5 input controls use `liveChange` for character-by-character updates (e.g., search suggestions) and `change` for final value commits (e.g., OData model updates). Praman fires both to match real user behavior. Skipping either event can cause missing suggestions, incomplete validation, or stale OData bindings.
What happens if waitForUI5Stable times out? A `TimeoutError` is thrown with details about which async operations were still pending (e.g., open XHR requests, unresolved promises). Check for long-running OData calls, infinite polling, or background model refresh timers. You can increase the timeout via `controlDiscoveryTimeout` in your Praman config.
:::tip[Next steps] - **[Selector Reference →](./selectors.md)** — Learn all UI5Selector fields for finding controls - **[Typed Control Returns →](./typed-controls.md)** — Get full autocomplete for control-specific methods - **[Custom Matchers →](./custom-matchers.md)** — Assert UI5 control state with 10 built-in matchers - **[Discovery and Interaction Strategies →](./discovery-and-interaction.md)** — Configure how controls are found and acted on ::: --- ## Custom Control Extension SAP projects frequently use custom UI5 controls that extend standard controls or create entirely new ones. This guide explains how to interact with custom controls through Praman when they are not part of the built-in type system. ## Finding Custom Controls by Type Use the `controlType` selector to target custom controls by their fully qualified UI5 class name: ```typescript test('interact with custom rating control', async ({ ui5 }) => { const rating = await ui5.control({ controlType: 'com.myapp.controls.StarRating', properties: { value: 4 }, }); await expect(rating).toBeDefined(); }); ``` The `controlType` selector searches the UI5 control tree by class name. It works for any control registered with `sap.ui.define`, including third-party and project-specific controls. ## Combining controlType with Properties When multiple instances of a custom control exist on the page, narrow the match with `properties`: ```typescript await ui5.control({ controlType: 'com.myapp.controls.StatusBadge', properties: { status: 'Critical', category: 'Inventory', }, }); ``` Properties are matched against the control's current property values using the UI5 `getProperty()` API. Property names must match the control's metadata definition exactly. ## Accessing Inner Controls of Composites Many custom controls are composites that wrap standard UI5 controls internally. For example, a `SmartField` wraps an `Input`, `ComboBox`, or `DatePicker` depending on the OData metadata. ### The SmartField Pattern `SmartField` is a common source of confusion. Its `getControlType()` returns `sap.ui.comp.smartfield.SmartField`, not the inner control type: ```typescript test('interact with SmartField inner control', async ({ ui5 }) => { await test.step('Set value on SmartField', async () => { // Target the SmartField directly -- Praman handles the inner control await ui5.fill({ id: 'vendorSmartField' }, 'ACME Corp'); await ui5.waitForUI5(); }); await test.step('Read value from SmartField', async () => { const value = await ui5.getValue({ id: 'vendorSmartField' }); expect(value).toBe('ACME Corp'); }); }); ``` ### Accessing the Inner Control Directly If you need to interact with the inner control specifically (e.g., to check its type or invoke a method not exposed by the wrapper): ```typescript test('access SmartField inner control type', async ({ ui5, page }) => { const innerType = await page.evaluate(() => { const sf = sap.ui.getCore().byId('vendorSmartField'); const inner = sf?.getInnerControls?.()[0]; return inner?.getMetadata().getName(); }); expect(innerType).toBe('sap.m.Input'); }); ``` ## Extending ControlProxy for Custom Methods When a custom control has domain-specific methods you call frequently, wrap the interactions in test helper functions: ```typescript // test-helpers/custom-controls.ts export async function setStarRating(page: Page, controlId: string, stars: number) { await page.evaluate( ({ id, value }) => { const control = sap.ui.getCore().byId(id); control?.setProperty('value', value); control?.fireEvent('change', { value }); }, { id: controlId, value: stars }, ); } ``` Use it in tests: ```typescript test('rate a product', async ({ ui5, page }) => { await setStarRating(page, 'productRating', 5); await ui5.waitForUI5(); await expect(ui5.control({ id: 'ratingDisplay' })).toHaveText('5 / 5'); }); ``` ## Handling Controls with Custom Renderers Some custom controls override the standard renderer and produce non-standard DOM. When Praman's standard selectors cannot find a visible element, fall back to properties-based selectors: ```typescript test('interact with custom-rendered control', async ({ ui5 }) => { // Instead of relying on DOM structure, use UI5 control tree await ui5.click({ controlType: 'com.myapp.controls.CustomButton', properties: { text: 'Submit Order' }, }); }); ``` This approach works because Praman queries the UI5 control tree via the bridge, not the DOM. The control's rendered output does not affect selector resolution. ## Common Pitfalls - **Assuming inner control type**: A `SmartField` on an Edm.String property wraps `sap.m.Input`, but on an Edm.DateTime it wraps `sap.m.DatePicker`. Always verify against the live system. - **Using DOM selectors for custom controls**: Custom renderers produce unpredictable DOM. Always prefer `controlType` + `properties` selectors over CSS selectors. - **Missing fireEvent after setValue**: If your custom control listens for `change` or `liveChange` events, calling `setProperty()` alone may not trigger downstream logic. Use `fireEvent()` explicitly. - **Namespace typos**: Custom control class names are case-sensitive. `com.myapp.Controls.StatusBadge` (capital C) is different from `com.myapp.controls.StatusBadge`. --- ## Custom Matchers :::info[In this guide] - Assert UI5 control state with 10 purpose-built matchers - Understand why DOM-based assertions break on i18n changes and async updates - Use auto-retrying matchers that poll the UI5 runtime until timeout - Extend assertions with `expect.poll()` for custom UI5 checks - Compare Praman matchers with Playwright built-in matchers ::: Standard Playwright assertions like `expect(text).toBe('Active')` break when your SAP app runs in a different locale, because the displayed text changes with i18n translations. DOM-based checks like `expect(locator).toBeEnabled()` can also produce false positives when a control appears enabled in the DOM but is disabled in the UI5 model during a pending OData operation. Praman extends Playwright's `expect()` with 10 UI5-specific matchers that query control state directly from the UI5 runtime. These matchers check the **UI5 model/control state**, not the DOM — which is the source of truth in SAP applications. ## Key Difference From Playwright Matchers Standard Playwright matchers inspect DOM attributes: ```typescript // Playwright: checks DOM attribute await expect(page.locator('#saveBtn')).toBeEnabled(); ``` Praman matchers query the UI5 control API: ```typescript // Praman: checks UI5 control state via sap.ui.getCore().byId() await expect(page).toBeUI5Enabled('saveBtn'); ``` Why does this matter? A control can appear enabled in the DOM but be disabled in the UI5 model (e.g., during a pending OData operation). The UI5 state is the source of truth. ## Important: Matchers Receive controlId, Not Locator Unlike Playwright's built-in matchers that operate on `Locator` objects, Praman matchers receive a **control ID string** as their first argument and are called on `page`: ```typescript // Correct — pass string controlId to page expectation await expect(page).toHaveUI5Text('headerTitle', 'Purchase Order'); // Wrong — these are NOT locator-based matchers // await expect(locator).toHaveUI5Text('Purchase Order'); // Does not work ``` ## All 10 Matchers ### toHaveUI5Text Asserts the `text` property of a control matches the expected value. Supports both exact string and RegExp matching. ```typescript await expect(page).toHaveUI5Text('headerTitle', 'Purchase Order Details'); await expect(page).toHaveUI5Text('headerTitle', /Purchase Order/); ``` **Error message preview:** ```text Expected: "Purchase Order Details" Received: "Sales Order Details" Control: headerTitle (sap.m.Title) ``` ### toBeUI5Visible Asserts the control's `visible` property is `true`. ```typescript await expect(page).toBeUI5Visible('saveBtn'); ``` **Negation:** ```typescript await expect(page).not.toBeUI5Visible('hiddenPanel'); ``` ### toBeUI5Enabled Asserts the control's `enabled` property is `true`. ```typescript await expect(page).toBeUI5Enabled('submitBtn'); ``` **Negation:** ```typescript await expect(page).not.toBeUI5Enabled('disabledInput'); ``` ### toHaveUI5Property Asserts any named property of a control matches the expected value. ```typescript await expect(page).toHaveUI5Property('vendorInput', 'value', '100001'); await expect(page).toHaveUI5Property('saveBtn', 'type', 'Emphasized'); await expect(page).toHaveUI5Property('statusIcon', 'src', /accept/); ``` ### toHaveUI5ValueState Asserts the control's `valueState` property matches one of the UI5 value states. ```typescript await expect(page).toHaveUI5ValueState('vendorInput', 'Error'); await expect(page).toHaveUI5ValueState('amountField', 'Success'); await expect(page).toHaveUI5ValueState('noteInput', 'None'); ``` Valid value states: `'None'`, `'Success'`, `'Warning'`, `'Error'`, `'Information'`. ### toHaveUI5Binding Asserts the control has an OData binding at the specified path. ```typescript await expect(page).toHaveUI5Binding('vendorField', '/PurchaseOrder/Vendor'); await expect(page).toHaveUI5Binding('priceField', '/PO/NetAmount'); ``` ### toBeUI5ControlType Asserts the control's fully qualified type name. ```typescript await expect(page).toBeUI5ControlType('saveBtn', 'sap.m.Button'); await expect(page).toBeUI5ControlType('mainTable', 'sap.ui.table.Table'); ``` ### toHaveUI5RowCount Asserts the number of rows in a table control. ```typescript await expect(page).toHaveUI5RowCount('poTable', 5); await expect(page).toHaveUI5RowCount('emptyTable', 0); ``` ### toHaveUI5CellText Asserts the text content of a specific table cell by row and column index (both zero-based). ```typescript await expect(page).toHaveUI5CellText('poTable', 0, 0, '4500000001'); await expect(page).toHaveUI5CellText('poTable', 0, 2, 'Active'); await expect(page).toHaveUI5CellText('poTable', 1, 3, /pending/i); ``` ### toHaveUI5SelectedRows Asserts which row indices are currently selected in a table. ```typescript await expect(page).toHaveUI5SelectedRows('poTable', [0, 2]); await expect(page).toHaveUI5SelectedRows('poTable', []); // No selection ``` ## Auto-Retry Mechanism All Praman matchers are registered as async custom matchers. They query the UI5 runtime via `page.evaluate()` on each poll iteration. Playwright's `expect()` auto-retries failed assertions until the configured timeout (default 5 seconds). This means you do not need manual retry loops: ```typescript // This auto-retries for up to 5 seconds: await expect(page).toHaveUI5Text('statusField', 'Approved'); ``` For longer waits, increase the timeout: ```typescript await expect(page).toHaveUI5Text('statusField', 'Approved', { timeout: 15_000 }); ``` ## Using expect.poll() With UI5 Methods For custom assertions beyond the 10 built-in matchers, use `expect.poll()` with `ui5.*` methods: ```typescript // Poll until the control value matches await expect .poll( async () => { const input = await ui5.control({ id: 'vendorInput' }); return input.getProperty('value'); }, { timeout: 10_000 }, ) .toBe('100001'); // Poll for table row count await expect .poll(async () => ui5.table.getRowCount('poTable'), { timeout: 15_000 }) .toBeGreaterThan(0); ``` ## Comparison With Playwright Built-In Matchers | Matcher Type | Source of Truth | Auto-Retry | Needs Locator | | -------------------------- | ------------------------ | ---------- | ----------------------- | | Playwright `toBeEnabled()` | DOM `disabled` attribute | Yes | Yes (`Locator`) | | Praman `toBeUI5Enabled()` | UI5 `getEnabled()` API | Yes | No (string `controlId`) | | Playwright `toHaveText()` | DOM `textContent` | Yes | Yes (`Locator`) | | Praman `toHaveUI5Text()` | UI5 `getText()` API | Yes | No (string `controlId`) | | Playwright `toBeVisible()` | DOM intersection/display | Yes | Yes (`Locator`) | | Praman `toBeUI5Visible()` | UI5 `getVisible()` API | Yes | No (string `controlId`) | Use Praman matchers when you need to check the UI5 model state. Use Playwright matchers when you need to verify DOM-level rendering (e.g., CSS visibility, element position). ## Complete Example ```typescript test('validate purchase order form', async ({ ui5, page }) => { // Check form state await expect(page).toBeUI5Visible('vendorInput'); await expect(page).toBeUI5Enabled('vendorInput'); // Fill and validate await ui5.fill({ id: 'vendorInput' }, 'INVALID'); await expect(page).toHaveUI5ValueState('vendorInput', 'Error'); await ui5.fill({ id: 'vendorInput' }, '100001'); await expect(page).toHaveUI5ValueState('vendorInput', 'None'); await expect(page).toHaveUI5Text('vendorName', 'Acme Corp'); // Check binding await expect(page).toHaveUI5Binding('vendorInput', '/PurchaseOrder/Vendor'); // Verify table await expect(page).toHaveUI5RowCount('itemTable', 3); await expect(page).toHaveUI5CellText('itemTable', 0, 0, 'MAT-001'); await expect(page).toHaveUI5CellText('itemTable', 0, 1, '10'); // Check button states await expect(page).toBeUI5Enabled('saveBtn'); await expect(page).not.toBeUI5Enabled('deleteBtn'); await expect(page).toBeUI5ControlType('saveBtn', 'sap.m.Button'); }); ``` :::warning[Common mistake] Do not pass a Locator to Praman matchers. Unlike Playwright's built-in matchers, Praman matchers receive a **control ID string** and are called on `page`: ```typescript // Wrong — Praman matchers do not accept Locator objects const locator = page.locator('ui5=sap.m.Button#saveBtn'); await expect(locator).toBeUI5Enabled(); // Does not work // Correct — pass the control ID string to page await expect(page).toBeUI5Enabled('saveBtn'); ``` ::: ## Registration Matchers are auto-registered via the `matcherRegistration` worker-scoped auto-fixture. No setup code is needed — just import `{ expect }` from `playwright-praman` and the matchers are available. ```typescript // All 10 matchers are ready to use ``` ## FAQ
Do Praman matchers auto-retry like Playwright matchers? Yes. All 10 matchers are registered as async custom matchers. They query the UI5 runtime via `page.evaluate()` on each poll iteration, and Playwright's `expect()` auto-retries until the configured timeout (default 5 seconds). You can increase the timeout per assertion: `await expect(page).toHaveUI5Text('field', 'value', { timeout: 15_000 })`.
Can I use Playwright built-in matchers alongside Praman matchers? Yes. Use Praman matchers when checking UI5 model/control state (e.g., `toBeUI5Enabled`, `toHaveUI5Text`). Use Playwright matchers when checking DOM-level rendering (e.g., `toBeVisible` for CSS visibility, `toHaveURL` for page URL, `toHaveTitle` for page title). Both work in the same test.
How do I assert a property that has no built-in matcher? Use `toHaveUI5Property` for any named property, or `expect.poll()` for fully custom checks: ```typescript // Named property await expect(page).toHaveUI5Property('myInput', 'placeholder', 'Enter value'); // Custom poll await expect.poll(async () => { const ctrl = await ui5.control({ id: 'myInput' }); return ctrl.getProperty('valueStateText'); }).toBe('Required field'); ```
:::tip[Next steps] - **[Control Interactions →](./control-interactions.md)** — Click, fill, and read controls before asserting state - **[Gold Standard Test Pattern →](./gold-standard-test.md)** — See matchers used in a complete reference test - **[Selector Reference →](./selectors.md)** — Find the right control to assert against ::: --- ## Debugging & Troubleshooting This guide covers Praman's debugging tools, log configuration, trace viewer integration, and solutions to common issues. ## Pino Log Levels Praman uses [pino](https://github.com/pinojs/pino) for structured JSON logging with child loggers per module and correlation IDs. ### Configuring Log Levels ```typescript // praman.config.ts export default { logLevel: 'debug', // 'fatal' | 'error' | 'warn' | 'info' | 'debug' | 'trace' }; ``` Or via environment variable: ```bash PRAMAN_LOG_LEVEL=debug npx playwright test ``` ### Log Level Guide | Level | Use When | | ------- | ---------------------------------------------------- | | `fatal` | Production -- only unrecoverable errors | | `error` | Default -- errors and failures only | | `warn` | Investigating intermittent issues | | `info` | Monitoring test execution flow | | `debug` | Developing or debugging bridge/proxy issues | | `trace` | Full diagnostic output including serialized payloads | ### Creating Module Loggers :::info[Contributor Only] The `createLogger` and `REDACTION_PATHS` APIs use internal path aliases (`#core/*`) and are only available when developing Praman itself. End users control logging via the `PRAMAN_LOG_LEVEL` environment variable. ::: ```typescript const logger = createLogger('my-module'); logger.info({ selector }, 'Finding control'); logger.debug({ result }, 'Control found'); logger.error({ err }, 'Control not found'); ``` ### Secret Redaction Pino is configured with redaction paths to prevent secrets from appearing in logs: ```typescript // Redacted paths include: // - password, apiKey, token, secret, authorization // - Nested paths: *.password, config.ai.apiKey, etc. ``` ## Playwright Trace Viewer Enable trace capture in `playwright.config.ts`: ```typescript export default defineConfig({ use: { trace: 'retain-on-failure', // or 'on' for all tests }, }); ``` ### Viewing Traces ```bash # After a test failure npx playwright show-trace test-results/my-test/trace.zip # Or open the HTML report which includes traces npx playwright show-report ``` ### Trace Viewer Limitations with Bridge page.evaluate() The Playwright trace viewer has limitations when debugging bridge operations: 1. **page.evaluate() calls appear as opaque steps.** The trace shows that `page.evaluate()` was called but does not show the JavaScript executed inside the browser context. 2. **Bridge injection is not visible.** The bridge setup script runs via `page.evaluate()` or `addInitScript()`, which appear as single steps without internal detail. 3. **UI5 control state is not captured.** The trace viewer captures DOM snapshots but not UI5's in-memory control tree, bindings, or model data. #### Workaround: Use test.step() for visibility Praman wraps operations in `test.step()` calls that appear in the trace timeline: ```typescript await test.step('Click submit button', async () => { await ui5.click({ id: 'submitBtn' }); }); ``` #### Workaround: Enable debug logging Set `PRAMAN_LOG_LEVEL=debug` to get detailed bridge communication logs alongside the trace. ## OpenTelemetry Integration Praman includes full OpenTelemetry (OTel) tracing and metrics for distributed observability. Telemetry is **disabled by default** with zero overhead — all operations use NoOp wrappers until you opt in. For the complete setup guide, see [Telemetry Setup](./telemetry). ### 4-Layer Observability Stack | Layer | Tool | Purpose | | ----- | ----------------------- | ---------------------------------------------- | | L1 | Playwright Reporter API | Test events via `test.step()` | | L2 | pino structured logging | JSON logs with correlation IDs | | L3 | OpenTelemetry (opt-in) | Spans + metrics for bridge/proxy/discovery | | L4 | AI Agent Telemetry | Capability introspection, deterministic replay | ### Quick Start ```bash # 1. Install OTel dependencies npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http # 2. Start Jaeger (local collector) docker compose -f docs/docker-compose.otel.yml up -d # 3. Enable telemetry export PRAMAN_TELEMETRY_ENABLED=true export PRAMAN_TELEMETRY_ENDPOINT=http://localhost:4318 # 4. Run tests — spans appear in Jaeger UI at http://localhost:16686 npx playwright test ``` Or configure via `praman.config.ts`: ```typescript export default { telemetry: { openTelemetry: true, endpoint: 'http://localhost:4318', exporter: 'otlp', // or 'jaeger', 'azure-monitor' serviceName: 'my-sap-tests', metrics: true, // enable counters and histograms }, }; ``` ### OTel Reporter The `OTelReporter` runs in the Playwright reporter process (separate from test workers) and emits spans for the test lifecycle: ```text test.run ("should create purchase order") hook: "auth.setup" fixture: "ui5" test.step: "Fill header fields" pw:api: "locator.click" expect: "expect.toBeVisible" ``` Configure in `playwright.config.ts`: ```typescript reporter: [ ['playwright-praman/reporters', { otel: true, endpoint: 'http://localhost:4318' }], ], ``` ### Metrics When metrics are enabled, Praman records: | Metric | Type | Description | | ----------------------------------- | --------- | -------------------------------------- | | `praman.test.pass` | Counter | Tests passed | | `praman.test.fail` | Counter | Tests failed | | `praman.test.skip` | Counter | Tests skipped | | `praman.test.duration` | Histogram | Test duration (ms) | | `praman.control.discovery` | Counter | Control discovery attempts | | `praman.control.discovery.duration` | Histogram | Discovery duration (ms) | | `praman.bridge.injection` | Counter | Bridge injection attempts | | `praman.bridge.evaluation.duration` | Histogram | Bridge `page.evaluate()` duration (ms) | ## OData Request Tracing Praman includes automatic OData network request capture that feeds the `ODataTraceReporter`. When enabled, every browser-level OData request (XHR/fetch) is recorded with method, URL, status code, duration, and response size — no manual instrumentation needed. ### Enable automatic tracing ```typescript // praman.config.ts export default { odataTracing: { enabled: true, // Optional: add custom URL patterns beyond the defaults urlPatterns: ['/my-custom-service/'], }, }; ``` Default URL patterns: `/sap/opu/odata/`, `/sap/opu/odata4/`, `/odata/v2/`, `/odata/v4/`. ### Add the reporter ```typescript // playwright.config.ts export default defineConfig({ reporter: [['list'], ['playwright-praman/reporters', { outputDir: 'test-results' }]], }); ``` After the test run, find `test-results/odata-trace.json` containing: - **Per-entity-set stats**: total calls, avg/max duration, error count, method breakdown - **Individual traces**: every captured request with full metadata ### Reading the report ```json { "totalRequests": 42, "totalDuration": 6300, "entityStats": [ { "entitySet": "A_Product", "totalCalls": 30, "avgDuration": 120, "maxDuration": 450, "errorCount": 0, "byMethod": { "GET": 28, "POST": 2 } } ] } ``` Use this to identify slow entity sets, excessive request counts, or high error rates. ### Manual tracing (advanced) If you need to trace requests in specific tests without enabling the auto-fixture globally, attach trace data manually: ```typescript test('trace OData calls', async ({ page }, testInfo) => { const traces: { method: string; url: string; statusCode: number; duration: number }[] = []; const pending = new Map(); page.on('request', (req) => { if (req.url().includes('/sap/opu/odata/')) { pending.set(req, Date.now()); } }); page.on('response', (resp) => { const start = pending.get(resp.request()); if (start !== undefined) { pending.delete(resp.request()); traces.push({ method: resp.request().method(), url: resp.request().url(), statusCode: resp.status(), duration: Date.now() - start, }); } }); // ... your test steps ... await testInfo.attach('odata-trace', { contentType: 'application/json', body: Buffer.from(JSON.stringify(traces)), }); }); ``` :::tip The auto-fixture does exactly this for you. Prefer `odataTracing: { enabled: true }` over manual tracing. ::: ### Scope limitation Auto-tracing captures **browser-level traffic** (XHR/fetch from the SAP Fiori app running in the browser). Node-level `page.request.*` API calls (used by `ui5.odata.createEntity()`, `ui5.odata.queryEntities()`, etc.) operate outside the browser and are not captured by the auto-fixture. ## Error Introspection: toUserMessage() and toAIContext() Every `PramanError` provides two introspection methods: ### toUserMessage() Human-readable error summary for terminal output: ```typescript try { await ui5.click({ id: 'nonExistentBtn' }); } catch (error) { if (error instanceof PramanError) { console.error(error.toUserMessage()); // Output: // [ERR_CONTROL_NOT_FOUND] Control not found: nonExistentBtn // Attempted: Find control with selector: {"id":"nonExistentBtn"} // Suggestions: // - Verify the control ID exists in the UI5 view // - Check if the page has fully loaded (waitForUI5Stable) // - Try using controlType + properties instead of ID } } ``` ### toAIContext() Machine-readable structured context for AI agents: ```typescript const context = error.toAIContext(); // { // code: 'ERR_CONTROL_NOT_FOUND', // message: 'Control not found: nonExistentBtn', // attempted: 'Find control with selector: {"id":"nonExistentBtn"}', // retryable: true, // severity: 'error', // details: { selector: { id: 'nonExistentBtn' }, timeout: 30000 }, // suggestions: ['Verify the control ID exists...', ...], // timestamp: '2026-02-23T10:30:00.000Z', // } ``` ### toJSON() Full serialization for logging and persistence: ```typescript const serialized = error.toJSON(); // Includes all fields from toAIContext() plus stack trace ``` ## Environment Variables Praman and its examples rely on environment variables for SAP system credentials. Set them before running tests: ```bash # .env (gitignored — never commit credentials) SAP_ACTIVE_SYSTEM=cloud # 'cloud' or 'onprem' SAP_CLOUD_BASE_URL=https://your-sap-system.example.com SAP_CLOUD_USERNAME=your-username SAP_CLOUD_PASSWORD=your-password SAP_AUTH_STRATEGY=btp-saml # 'basic' | 'btp-saml' | 'office365' SAP_CLIENT=100 # optional, OnPrem only SAP_LANGUAGE=EN # optional, default: EN PRAMAN_LOG_LEVEL=info # optional: 'error' | 'warn' | 'info' | 'debug' | 'trace' # ── Praman AI config overrides ────────────────────────────────────── # PRAMAN_AI_PROVIDER=openai # 'openai' | 'azure' | 'anthropic' # PRAMAN_AI_API_KEY=sk-... # PRAMAN_AI_MODEL=gpt-4o # PRAMAN_AI_TEMPERATURE=0.3 # PRAMAN_AI_ENDPOINT=https://your-azure.openai.azure.com # PRAMAN_AI_DEPLOYMENT=your-deployment # PRAMAN_AI_API_VERSION=2024-02-01 # PRAMAN_AI_ANTHROPIC_API_KEY=sk-ant-... # ── Praman telemetry overrides ────────────────────────────────────── # PRAMAN_TELEMETRY_ENABLED=true # PRAMAN_TELEMETRY_ENDPOINT=http://localhost:4318 # PRAMAN_TELEMETRY_SERVICE_NAME=praman-tests # PRAMAN_TELEMETRY_EXPORTER=otlp # PRAMAN_TELEMETRY_PROTOCOL=http # PRAMAN_TELEMETRY_METRICS_ENABLED=true # PRAMAN_TELEMETRY_BATCH_TIMEOUT=5000 # PRAMAN_TELEMETRY_MAX_QUEUE_SIZE=2048 # PRAMAN_TELEMETRY_CONNECTION_STRING=InstrumentationKey=... # ── Praman OData tracing ─────────────────────────────────────────── # PRAMAN_ODATA_TRACING_ENABLED=true ``` ### Loading .env files Playwright does not load `.env` files automatically. Use one of these approaches: ```bash # Option 1: dotenv-cli (recommended) npm install -D dotenv-cli npx dotenv -e .env -- npx playwright test # Option 2: export in shell export SAP_CLOUD_BASE_URL=https://your-sap-system.example.com export SAP_CLOUD_USERNAME=testuser export SAP_CLOUD_PASSWORD= npx playwright test # Option 3: inline (CI / one-off) SAP_CLOUD_BASE_URL=https://host SAP_CLOUD_USERNAME=user SAP_CLOUD_PASSWORD=pw npx playwright test ``` ### CI/CD secrets In GitHub Actions, store credentials as repository secrets and pass them as environment variables: ```yaml env: SAP_CLOUD_BASE_URL: ${{ secrets.SAP_CLOUD_BASE_URL }} SAP_CLOUD_USERNAME: ${{ secrets.SAP_CLOUD_USERNAME }} SAP_CLOUD_PASSWORD: ${{ secrets.SAP_CLOUD_PASSWORD }} ``` See the [Docker & CI/CD guide](./docker-cicd.md) for full pipeline examples. ## Troubleshooting Quick Reference | Symptom | Error Code | Likely Cause | Jump To | | -------------------------- | -------------------------- | -------------------------------------------------- | -------------------------------------------------------------- | | Bridge timeout on startup | `ERR_BRIDGE_TIMEOUT` | Page is not a UI5 app, or UI5 loading slowly | [Bridge Injection Timeout](#bridge-injection-timeout) | | Control not found | `ERR_CONTROL_NOT_FOUND` | Wrong ID, control not rendered, wrong frame | [Control Not Found](#control-not-found) | | Stale object reference | `ERR_BRIDGE_EXECUTION` | Navigation invalidated the object map | [Stale Object Reference](#stale-object-reference) | | Test hangs indefinitely | `ERR_TIMEOUT_UI5_STABLE` | Third-party scripts (WalkMe) block stability | [Stability Wait Hanging](#stability-wait-hanging) | | Login fails | `ERR_AUTH_FAILED` | Wrong credentials, expired session, wrong strategy | [Auth Failures](#auth-failures) | | ReferenceError in evaluate | — | Helper function not serialized into browser | [page.evaluate() ReferenceError](#pageevaluate-referenceerror) | | OData 403/404 | `ERR_ODATA_REQUEST_FAILED` | Missing CSRF token, wrong service URL | [OData Request Tracing](#odata-request-tracing) | | Empty env var errors | — | `.env` not loaded or variable not set | [Environment Variables](#environment-variables) | ## Common Issues and Resolutions ### Bridge Injection Timeout **Symptom:** `BridgeError: ERR_BRIDGE_TIMEOUT` during test startup. **Causes and fixes:** - The page is not a UI5 application. Verify the URL loads a page with `sap.ui.require`. - UI5 is loading slowly. Increase `controlDiscoveryTimeout` in config. - A CSP (Content Security Policy) blocks `page.evaluate()`. Check browser console for CSP violations. ```typescript // Increase timeout export default { controlDiscoveryTimeout: 60_000, // 60 seconds }; ``` ### Control Not Found **Symptom:** `ControlError: ERR_CONTROL_NOT_FOUND` **Causes and fixes:** - The control has not rendered yet. Add `await ui5.waitForUI5()` before the operation. - The control ID is wrong. Use the UI5 Diagnostics tool (`Ctrl+Alt+Shift+S` in the browser) to inspect control IDs. - The control is inside a different frame (WorkZone). Ensure the page reference points to the correct frame. - The control is hidden. Check `preferVisibleControls` config setting. ### Stale Object Reference **Symptom:** `Error: Object not found for UUID: xxx-xxx` **Causes and fixes:** - The browser-side object was evicted by TTL cleanup. Reduce test step duration or increase TTL. - A page navigation invalidated the object map. Re-discover the object after navigation. ```typescript // After navigation, objects from the previous page are gone await page.goto('/newPage'); await ui5.waitForUI5(); // Re-discover objects on the new page const model = await ui5.control({ id: 'myControl' }); ``` ### Stability Wait Hanging **Symptom:** Test hangs on `waitForUI5Stable()`. **Causes and fixes:** - A third-party overlay (WalkMe, analytics) keeps sending HTTP requests. Use `skipStabilityWait`: ```typescript // Global config export default { skipStabilityWait: true, }; // Per-selector override await ui5.click({ id: 'submitBtn' }, { skipStabilityWait: true }); ``` ### Auth Failures **Symptom:** `AuthError: ERR_AUTH_FAILED` **Causes and fixes:** - Credentials are expired or wrong. Check `SAP_CLOUD_USERNAME` and `SAP_CLOUD_PASSWORD` env vars. - The auth strategy does not match the system. SAP BTP uses SAML; on-premise may use Basic auth. - Storage state is stale. Delete `.auth/` directory and re-run: ```bash rm -rf .auth/ npx playwright test --project=setup ``` ### page.evaluate() ReferenceError **Symptom:** `ReferenceError: functionName is not defined` in browser console. **Cause:** A module-level helper function was referenced inside `page.evaluate()`. Only inner functions are serialized. **Fix:** Move the helper function inside the `page.evaluate()` callback. See the [Bridge Internals](./bridge-internals.md) guide for the detailed explanation of this serialization constraint. ## Debug Checklist When a test fails, follow this diagnostic sequence: 1. **Read the error message.** Praman errors include `code`, `attempted`, and `suggestions[]`. 2. **Check the Playwright trace.** Run `npx playwright show-report` and open the trace for the failed test. 3. **Enable debug logging.** Set `PRAMAN_LOG_LEVEL=debug` and re-run the test. 4. **Inspect UI5 diagnostics.** Open the app in a browser and press `Ctrl+Alt+Shift+S` to see the UI5 control tree. 5. **Verify bridge injection.** In the browser console, check `window.__praman_bridge.ready` and `window.__praman_bridge.ui5Version`. 6. **Check the frame.** For WorkZone apps, ensure you are operating on the app frame, not the shell frame. 7. **Review OData traces.** If using the OData trace reporter, check `odata-trace.json` for failed HTTP calls. --- ## Error Reference For step-by-step diagnostic help, see the [Troubleshooting Guide](./troubleshooting.md). Praman provides 14 typed error classes with 58 machine-readable error codes. Every error includes the attempted operation, a `retryable` flag, and actionable recovery `suggestions`. ## Error Hierarchy All errors extend `PramanError`, which extends `Error`: ```text Error └── PramanError ├── AIError (11 codes) ├── AuthError (4 codes) ├── BridgeError (5 codes) ├── ConfigError (3 codes) ├── ControlError (8 codes) ├── FLPError (5 codes) ├── IntentError (4 codes) ├── NavigationError (3 codes) ├── ODataError (3 codes) ├── PluginError (3 codes) ├── SelectorError (3 codes) ├── TimeoutError (3 codes) └── VocabularyError (4 codes) ``` ## Error Codes ### Config Errors | Code | Description | Retryable | | ---------------------- | ---------------------------------------------- | --------- | | `ERR_CONFIG_INVALID` | Zod schema validation failed | No | | `ERR_CONFIG_NOT_FOUND` | Config file not found at expected path | No | | `ERR_CONFIG_PARSE` | Config file could not be parsed (syntax error) | No | ### Bridge Errors | Code | Description | Retryable | | ---------------------- | ---------------------------------------- | --------- | | `ERR_BRIDGE_TIMEOUT` | Bridge injection exceeded timeout | Yes | | `ERR_BRIDGE_INJECTION` | Bridge script failed to inject into page | Yes | | `ERR_BRIDGE_NOT_READY` | Bridge not ready (UI5 not loaded) | Yes | | `ERR_BRIDGE_VERSION` | UI5 version mismatch or unsupported | No | | `ERR_BRIDGE_EXECUTION` | Bridge script execution failed | Yes | ### Control Errors | Code | Description | Retryable | | -------------------------------- | ----------------------------------------------- | --------- | | `ERR_CONTROL_NOT_FOUND` | Control not found with given selector | Yes | | `ERR_CONTROL_NOT_VISIBLE` | Control exists but is not visible | Yes | | `ERR_CONTROL_NOT_ENABLED` | Control is visible but disabled | No | | `ERR_CONTROL_NOT_INTERACTABLE` | Control cannot receive interactions | No | | `ERR_CONTROL_PROPERTY` | Property read/write failed | No | | `ERR_CONTROL_AGGREGATION` | Aggregation access failed | No | | `ERR_CONTROL_METHOD` | Method invocation failed (possibly blacklisted) | No | | `ERR_CONTROL_INTERACTION_FAILED` | Click/type/select operation failed | Yes | ### Auth Errors | Code | Description | Retryable | | --------------------------- | ----------------------------------------- | --------- | | `ERR_AUTH_FAILED` | Authentication failed (wrong credentials) | No | | `ERR_AUTH_TIMEOUT` | Login page did not respond in time | Yes | | `ERR_AUTH_SESSION_EXPIRED` | Session expired (default: 30 min) | Yes | | `ERR_AUTH_STRATEGY_INVALID` | Unknown or misconfigured auth strategy | No | ### Navigation Errors | Code | Description | Retryable | | ------------------------ | ----------------------------------- | --------- | | `ERR_NAV_TILE_NOT_FOUND` | FLP tile not found by title | No | | `ERR_NAV_ROUTE_FAILED` | Hash navigation did not resolve | Yes | | `ERR_NAV_TIMEOUT` | Navigation did not complete in time | Yes | ### OData Errors | Code | Description | Retryable | | -------------------------- | ------------------------------------- | --------- | | `ERR_ODATA_REQUEST_FAILED` | HTTP request to OData service failed | Yes | | `ERR_ODATA_PARSE` | OData response could not be parsed | No | | `ERR_ODATA_CSRF` | CSRF token fetch or validation failed | Yes | ### Selector Errors | Code | Description | Retryable | | ------------------------ | ------------------------------------ | --------- | | `ERR_SELECTOR_INVALID` | Selector object is malformed | No | | `ERR_SELECTOR_AMBIGUOUS` | Multiple controls match the selector | No | | `ERR_SELECTOR_PARSE` | Selector string could not be parsed | No | ### Timeout Errors | Code | Description | Retryable | | ------------------------------- | ---------------------------------- | --------- | | `ERR_TIMEOUT_UI5_STABLE` | UI5 did not reach stable state | Yes | | `ERR_TIMEOUT_CONTROL_DISCOVERY` | Control discovery exceeded timeout | Yes | | `ERR_TIMEOUT_OPERATION` | Generic operation timeout | Yes | ### AI Errors | Code | Description | Retryable | | ------------------------------ | ------------------------------------ | --------- | | `ERR_AI_PROVIDER_UNAVAILABLE` | LLM provider not reachable | Yes | | `ERR_AI_RESPONSE_INVALID` | LLM returned invalid response format | Yes | | `ERR_AI_TOKEN_LIMIT` | Prompt exceeded token limit | No | | `ERR_AI_RATE_LIMITED` | Provider rate limit hit | Yes | | `ERR_AI_NOT_CONFIGURED` | AI provider not configured | No | | `ERR_AI_LLM_CALL_FAILED` | LLM API call failed | Yes | | `ERR_AI_RESPONSE_PARSE_FAILED` | Could not parse LLM response | Yes | | `ERR_AI_CONTEXT_BUILD_FAILED` | Page context build failed | Yes | | `ERR_AI_STEP_INTERPRET_FAILED` | Step interpretation failed | No | | `ERR_AI_INVALID_REQUEST` | Invalid or malformed AI request | No | | `ERR_AI_CAPABILITY_NOT_FOUND` | Requested capability not in registry | No | ### Plugin Errors | Code | Description | Retryable | | ------------------------- | --------------------------------- | --------- | | `ERR_PLUGIN_LOAD` | Plugin module could not be loaded | No | | `ERR_PLUGIN_INIT` | Plugin initialization failed | Yes | | `ERR_PLUGIN_INCOMPATIBLE` | Plugin version incompatible | No | ### Vocabulary Errors | Code | Description | Retryable | | ------------------------------ | --------------------------------------- | --------- | | `ERR_VOCAB_TERM_NOT_FOUND` | Business term not in vocabulary | No | | `ERR_VOCAB_DOMAIN_LOAD_FAILED` | Domain JSON failed to load | Yes | | `ERR_VOCAB_JSON_INVALID` | Domain vocabulary JSON malformed | No | | `ERR_VOCAB_AMBIGUOUS_MATCH` | Multiple terms match with similar score | No | ### Intent Errors | Code | Description | Retryable | | ------------------------------ | ------------------------------------ | --------- | | `ERR_INTENT_FIELD_NOT_FOUND` | Form field not found for intent | No | | `ERR_INTENT_ACTION_FAILED` | Intent action could not be completed | Yes | | `ERR_INTENT_NAVIGATION_FAILED` | Intent navigation to app failed | Yes | | `ERR_INTENT_VALIDATION_FAILED` | Intent input validation failed | No | ### FLP Errors | Code | Description | Retryable | | --------------------------- | ----------------------------- | --------- | | `ERR_FLP_SHELL_NOT_FOUND` | FLP shell container not found | Yes | | `ERR_FLP_PERMISSION_DENIED` | Insufficient FLP permissions | No | | `ERR_FLP_API_UNAVAILABLE` | FLP UShell API not available | Yes | | `ERR_FLP_INVALID_USER` | FLP user context invalid | No | | `ERR_FLP_OPERATION_TIMEOUT` | FLP operation timed out | Yes | ## Using Errors in Tests ### Catch and Inspect ```typescript test('handle control not found', async ({ ui5 }) => { try { await ui5.control({ id: 'nonExistent' }); } catch (error) { if (error instanceof ControlError) { console.log(error.code); // 'ERR_CONTROL_NOT_FOUND' console.log(error.retryable); // true console.log(error.suggestions); // ['Verify the control ID...', ...] console.log(error.toUserMessage()); // Human-readable console.log(error.toAIContext()); // Machine-readable for AI agents } } }); ``` ### Error Properties Every `PramanError` includes: | Property | Type | Description | | ------------- | ----------- | ------------------------------------------------------- | | `code` | `ErrorCode` | Machine-readable code (e.g., `'ERR_CONTROL_NOT_FOUND'`) | | `message` | `string` | Human-readable description | | `attempted` | `string` | What operation was attempted | | `retryable` | `boolean` | Whether the operation can be retried | | `suggestions` | `string[]` | 2-4 actionable recovery steps | | `details` | `Record` | Additional context (selector, timeout, etc.) | | `timestamp` | `string` | ISO-8601 timestamp | ### Serialization ```typescript // For humans error.toUserMessage(); // "Control not found: #saveBtn. Try: Verify the control ID exists..." // For logging (JSON) error.toJSON(); // { code, message, attempted, retryable, suggestions, details, timestamp } // For AI agents error.toAIContext(); // Structured context optimized for LLM consumption ``` --- ## Fixture Composition Praman uses Playwright's `mergeTests()` to compose fixture modules into a single `test` object. This page explains the composition pattern, how to customize it, and why it exists. ## How mergeTests() Works Playwright's `mergeTests()` combines multiple `test.extend()` definitions into one `test` object. Each fixture module defines its own fixtures independently, and `mergeTests()` produces a unified test function that includes all of them. ```typescript export const test = mergeTests(coreTest, authTest, navTest); ``` When you import from `playwright-praman`, this merge has already been done for you: ```typescript test('all fixtures available', async ({ ui5, sapAuth, ui5Navigation }) => { // Everything is available in a single destructure }); ``` ## The Fixture Module Chain Praman merges 13 fixture modules in a specific order: ```text moduleTest → ui5.table, ui5.dialog, ui5.date, ui5.odata authTest → sapAuth navTest → ui5Navigation, btpWorkZone stabilityTest → ui5Stability, requestInterceptor (auto) controlTreeTest → controlTreeCapture (auto) feTest → fe (listReport, objectPage, table, list) aiTest → pramanAI intentTest → intent (procurement, sales, finance, manufacturing, masterData) shellFooterTest → ui5Shell, ui5Footer flpLocksTest → flpLocks flpSettingsTest → flpSettings testDataTest → testData odataTraceTest → odataTraceCapture (auto) ``` :::info[Auto-fixtures inside coreTest] `selectorRegistration`, `matcherRegistration`, and `playwrightCompat` are **auto-fixtures inside `coreTest`**, not separate fixture modules. They are registered automatically when `coreTest` is loaded. ::: The order matters: later modules can depend on fixtures defined by earlier modules. For example, `navTest` depends on `ui5` from `coreTest`, and `authTest` depends on `pramanConfig`. ## Why mergeTests() Over a Monolithic File A single giant `test.extend()` call with all fixtures creates several problems: 1. **Circular dependencies** — fixtures that reference each other in one block cause TypeScript compilation errors 2. **Bloated imports** — every test file imports every dependency, even if unused 3. **Testing isolation** — unit-testing fixture logic requires loading the entire fixture tree 4. **Readability** — a 500-line fixture definition is unmaintainable With `mergeTests()`, each module is independently testable, tree-shakeable, and can be composed in any combination. ## Tree-Shaking: Use Only What You Need If your tests only need core UI5 operations and authentication, skip the full import: ```typescript const test = mergeTests(coreTest, authTest); test('lightweight test', async ({ ui5, sapAuth }) => { // Only core + auth fixtures are loaded // AI, intents, FE helpers are not initialized }); ``` This reduces worker initialization time and avoids loading optional dependencies (e.g., LLM SDKs) that are not installed. ## Adding Custom Fixtures Extend the merged test object with your own fixtures using `test.extend()`: ```typescript interface MyFixtures { adminUser: { username: string; password: string }; appUrl: string; } const test = base.extend({ adminUser: async ({}, use) => { await use({ username: process.env.ADMIN_USER ?? 'admin', password: process.env.ADMIN_PASS ?? 'secret', }); }, appUrl: async ({ pramanConfig }, use) => { const baseUrl = pramanConfig.auth?.baseUrl ?? 'http://localhost:8080'; await use(`${baseUrl}/sap/bc/ui5_ui5/ui2/ushell/shells/abap/FioriLaunchpad.html`); }, }); export { test, expect }; ``` ## Composing Across Test Files For large test suites, create domain-specific test objects: ```typescript // fixtures/procurement-test.ts export const test = base.extend({ poDefaults: async ({}, use) => { await use({ plant: '1000', purchOrg: '1000', companyCode: '1000' }); }, }); // fixtures/finance-test.ts export const test = base.extend({ fiscalYear: async ({}, use) => { await use(new Date().getFullYear().toString()); }, }); ``` ```typescript // tests/procurement/create-po.spec.ts test('create PO with defaults', async ({ ui5, intent, poDefaults }) => { await intent.procurement.createPurchaseOrder({ vendor: '100001', material: 'MAT-001', quantity: 10, plant: poDefaults.plant, }); }); ``` ## Fixture Scopes Praman fixtures use two scopes: - **Worker-scoped** — created once per Playwright worker process. Shared across all tests in that worker. Used for expensive initialization: `pramanConfig`, `rootLogger`, `tracer`. - **Test-scoped** — created fresh for each test. Used for stateful objects: `ui5`, `sapAuth`, `flpLocks`, `testData`. Worker-scoped fixtures cannot depend on test-scoped fixtures. Test-scoped fixtures can depend on both. ## Auto-Fixtures Seven fixtures are marked with `{ auto: 'on' }` or `{ auto: true }` and fire without being requested in the test signature: ```typescript // These run automatically for every test: // - playwrightCompat (worker) — version compatibility checks // - selectorRegistration (worker) — registers ui5= selector engine // - matcherRegistration (worker) — registers 10 custom matchers // - requestInterceptor (test) — blocks WalkMe/analytics scripts // - ui5Stability (test) — auto-waits for UI5 stability after navigation // - odataTraceCapture (test) — captures OData requests for tracing // - controlTreeCapture (test) — captures control tree snapshots ``` You never destructure these — they just work. ## Cross-Fixture Dependencies (PW-MERGE-1 Pattern) When a fixture module like `navTest` needs a fixture from another module (e.g., `pramanConfig` from `coreTest`), it cannot import that fixture directly — Playwright's `test.extend()` only sees fixtures within the same call. The solution is the **option placeholder pattern**: ```typescript // In nav-fixtures.ts export const navTest = base.extend({ // Declare as option placeholder — will be provided by mergeTests() // eslint-disable-next-line @typescript-eslint/no-non-null-assertion pramanConfig: [undefined!, { option: true, scope: 'worker' }], // eslint-disable-next-line @typescript-eslint/no-non-null-assertion rootLogger: [undefined!, { option: true, scope: 'worker' }], ui5Navigation: async ({ page, pramanConfig, rootLogger }, use) => { // pramanConfig and rootLogger are available here at runtime // because mergeTests(coreTest, navTest) wires them together }, }); ``` **How it works:** 1. **`undefined!`** — TypeScript requires an initial value, but this placeholder is never used at runtime. The `!` (non-null assertion) satisfies the type checker. 2. **`{ option: true }`** — tells Playwright this fixture can be overridden. When `mergeTests(coreTest, navTest)` runs, `coreTest`'s real `pramanConfig` fixture replaces the placeholder. 3. **`scope: 'worker'`** — matches the scope of the providing fixture in `coreTest`. This pattern is used across all fixture modules that depend on `pramanConfig` or `rootLogger` (10 files, ~37 instances). The `eslint-disable` comments suppress the `no-non-null-assertion` rule since `undefined!` is intentional here, not a code smell. :::tip If you're adding a new fixture module that needs `pramanConfig` or `rootLogger`, copy this pattern from any existing fixture file (e.g., `nav-fixtures.ts`) and add your module to the `mergeTests()` call in `src/fixtures/index.ts`. ::: --- ## Fixture Reference Praman provides 13 fixture modules merged into a single `test` object via Playwright's `mergeTests()`. ## Import ```typescript ``` ## Fixture Summary | Fixture | Scope | Type | Description | | --------------- | ------ | --------------- | ------------------------------------------------------------------------------------------ | | `ui5` | test | Core | Control discovery, interaction, `.table`, `.dialog`, `.date`, `.odata` sub-namespaces | | `ui5Navigation` | test | Navigation | 11 FLP navigation methods | | `btpWorkZone` | test | Navigation | Dual-frame BTP WorkZone manager | | `sapAuth` | test | Authentication | SAP authentication (6 strategies) | | `fe` | test | Fiori Elements | `.listReport`, `.objectPage`, `.table`, `.list` helpers | | `pramanAI` | test | AI | Page discovery, agentic handler, LLM, vocabulary | | `intent` | test | Business | `.procurement`, `.sales`, `.finance`, `.manufacturing`, `.masterData` | | `ui5Shell` | test | FLP | Shell header (home, user menu) | | `ui5Footer` | test | FLP | Page footer bar (Save, Edit, Delete, etc.) | | `flpLocks` | test | FLP | SM12 lock management with auto-cleanup | | `flpSettings` | test | FLP | User settings reader (language, date format) | | `testData` | test | Data | Template-based data generation with auto-cleanup | | `pramanConfig` | worker | Infrastructure | Frozen config (loaded once per worker) | | `pramanLogger` | test | Infrastructure | Test-scoped pino logger | | `rootLogger` | worker | Infrastructure | Worker-scoped root logger | | `tracer` | worker | Infrastructure | OpenTelemetry tracer (NoOp when disabled, real when `openTelemetry: true`) | | `meter` | worker | Infrastructure | OpenTelemetry meter for metrics (NoOp when disabled, real when `metrics: true`) | | `browserBind` | test | CLI Integration | `PRAMAN_BIND=1` exposes browser to CLI agents via `browser.bind()` (Playwright 1.59+) | | `screencast` | test | CLI Integration | Chapter markers, action overlays, frame streaming via `page.screencast` (Playwright 1.59+) | ## Auto-Fixtures These fixtures fire automatically without being requested in the test signature: | Auto-Fixture | Scope | Purpose | | ---------------------- | ------ | --------------------------------------------- | | `playwrightCompat` | worker | Playwright version compatibility checks | | `selectorRegistration` | worker | Registers `ui5=` custom selector engine | | `matcherRegistration` | worker | Registers 10 custom UI5 matchers | | `requestInterceptor` | test | Blocks WalkMe, analytics, overlay scripts | | `ui5Stability` | test | Auto-waits for UI5 stability after navigation | ## Core Fixture: `ui5` The main fixture for control discovery and interaction. ### Control Discovery ```typescript test('discover controls', async ({ ui5 }) => { // Single control const btn = await ui5.control({ id: 'saveBtn' }); // Multiple controls const buttons = await ui5.controls({ controlType: 'sap.m.Button' }); // Interaction shortcuts await ui5.click({ id: 'submitBtn' }); await ui5.fill({ id: 'nameInput' }, 'John Doe'); await ui5.select({ id: 'countrySelect' }, 'US'); await ui5.check({ id: 'agreeCheckbox' }); await ui5.clear({ id: 'searchField' }); }); ``` ### Sub-Namespaces ```typescript test('sub-namespaces', async ({ ui5 }) => { // Table operations const rows = await ui5.table.getRows('poTable'); const count = await ui5.table.getRowCount('poTable'); // Dialog management await ui5.dialog.waitFor(); await ui5.dialog.confirm(); // Date/time operations await ui5.date.setDatePicker('startDate', new Date('2026-01-15')); // OData model access const data = await ui5.odata.getModelData('/PurchaseOrders'); }); ``` ## Navigation Fixture: `ui5Navigation` ```typescript test('navigation', async ({ ui5Navigation }) => { await ui5Navigation.navigateToApp('PurchaseOrder-manage'); await ui5Navigation.navigateToTile('Create Purchase Order'); await ui5Navigation.navigateToIntent( { semanticObject: 'PurchaseOrder', action: 'create' }, { plant: '1000' }, ); await ui5Navigation.navigateToHome(); await ui5Navigation.navigateBack(); const hash = await ui5Navigation.getCurrentHash(); }); ``` ## Auth Fixture: `sapAuth` ```typescript test('auth control', async ({ sapAuth, page }) => { await sapAuth.login(page, { url, username, password, strategy: 'basic' }); expect(await sapAuth.isAuthenticated(page)).toBe(true); // Auto-logout on teardown }); ``` ## Fiori Elements Fixture: `fe` ```typescript test('FE patterns', async ({ fe }) => { await fe.listReport.setFilter('Status', 'Active'); await fe.listReport.search(); await fe.listReport.navigateToItem(0); const title = await fe.objectPage.getHeaderTitle(); await fe.objectPage.clickEdit(); await fe.objectPage.clickSave(); }); ``` ## AI Fixture: `pramanAI` ```typescript test('AI discovery', async ({ pramanAI }) => { const context = await pramanAI.discoverPage({ interactiveOnly: true }); const result = await pramanAI.agentic.generateTest( 'Create a purchase order for vendor 100001', page, ); }); ``` ## Intent Fixture: `intent` ```typescript test('business intents', async ({ intent }) => { await intent.procurement.createPurchaseOrder({ vendor: '100001', material: 'MAT-001', quantity: 10, plant: '1000', }); await intent.finance.postVendorInvoice({ vendor: '100001', amount: 5000, currency: 'EUR', }); }); ``` ## Shell & Footer Fixtures ```typescript test('shell and footer', async ({ ui5Shell, ui5Footer }) => { await ui5Shell.expectShellHeader(); await ui5Footer.clickEdit(); // ... edit form fields ... await ui5Footer.clickSave(); await ui5Shell.clickHome(); }); ``` ## CLI Integration Fixtures ### Browser Bind: `browserBind` Opt-in via `PRAMAN_BIND=1`. Exposes the test browser to Playwright CLI agents. ```typescript // PRAMAN_BIND=1 npx playwright test test('agent-assisted test', async ({ browserBind }) => { if (browserBind.bound) { console.info('Agent can connect at:', browserBind.endpointUrl); // CLI agent inspects live UI while test pauses... } }); ``` See [Browser Bind & Screencast](./browser-bind.md) for full details. ### Screencast: `screencast` Chapter annotations and frame streaming for Playwright 1.59+ screencasts. ```typescript test('recorded test', async ({ screencast, ui5 }) => { await screencast.showChapter('Setup'); await ui5.click({ id: 'editBtn' }); await screencast.showChapter('Fill Form'); await ui5.fill({ id: 'nameInput' }, 'ACME Corp'); await screencast.showActions({ enabled: true }); }); ``` See [Browser Bind & Screencast](./browser-bind.md) for full details. ## Lock Management: `flpLocks` ```typescript test('lock management', async ({ flpLocks }) => { const count = await flpLocks.getNumberOfLockEntries('TESTUSER'); // Auto-cleanup on teardown await flpLocks.deleteAllLockEntries('TESTUSER'); }); ``` ## Test Data: `testData` ```typescript test('test data', async ({ testData }) => { const po = testData.generate({ documentNumber: '{{uuid}}', createdAt: '{{timestamp}}', vendor: '100001', }); await testData.save('po-input.json', po); // Auto-cleanup on teardown }); ``` ## Standalone Usage Individual fixture modules can be imported for lighter setups: ```typescript // Compose only what you need const test = mergeTests(coreTest, authTest); ``` --- ## Getting Started :::info[In this guide] - Install Praman and scaffold a complete SAP test project with one command - Configure SAP credentials and authenticate against your system - Generate production-ready Playwright tests using AI agents (no manual scripting) - Run and verify tests against a live SAP Fiori application - Understand the plan-generate-heal pipeline for continuous test creation ::: Two commands to get started. Then let AI agents generate your tests. ## Prerequisites - **Node.js** >= 22 - Access to an SAP UI5 / Fiori application - SAP credentials (username, password, base URL) ## Step 1: Install and Initialize ```bash npm install playwright-praman npx playwright-praman init ``` `init` handles everything: - Validates Node.js and npm - Installs Chromium browser - Prompts for SAP credentials and creates `.env` - Detects your IDE and installs AI agent definitions (planner, generator, healer) - Scaffolds `playwright.config.ts`, `praman.config.ts`, auth setup, and a gold-standard test
What init installs (full breakdown) Praman has 3 direct dependencies and 5 peer dependencies: | Package | Type | Purpose | | ------------------------------------------- | --------------- | --------------------------------------------------------- | | `@playwright/test` | Peer (required) | Playwright test runner (>=1.57.0) — auto-installed | | `@playwright/cli` | Peer (required) | Playwright CLI for agent browser control — auto-installed | | `commander` | Dependency | CLI framework for `npx playwright-praman` commands | | `pino` | Dependency | Structured JSON logging | | `zod` | Dependency | Configuration validation and type-safe schemas | | `@anthropic-ai/sdk` | Peer (optional) | AI test generation via Claude | | `openai` | Peer (optional) | AI test generation via OpenAI / Azure OpenAI | | `@opentelemetry/api` | Peer (optional) | Observability and distributed tracing | | `@opentelemetry/sdk-node` | Peer (optional) | OpenTelemetry Node.js SDK | | `@opentelemetry/exporter-trace-otlp-http` | Peer (optional) | OTLP trace exporter (HTTP) | | `@opentelemetry/exporter-metrics-otlp-http` | Peer (optional) | OTLP metrics exporter (HTTP) | | `@opentelemetry/sdk-metrics` | Peer (optional) | OTel metrics SDK | | `@azure/monitor-opentelemetry-exporter` | Peer (optional) | Azure Monitor exporter (beta) | ### IDE detection and agent installation | IDE / Agent | Detection Signal | What gets installed | | ------------------ | ------------------------------------------------------ | ------------------------------------------------------- | | **VS Code** | `.vscode/` or `TERM_PROGRAM=vscode` | Settings, extensions, code snippets | | **Claude Code** | `CLAUDE.md` | Agents (planner, generator, healer), prompts, seed file | | **Cursor** | `.cursor/` or `.cursorrc` | `.cursor/rules/praman.mdc` | | **Jules** | `.jules/` | `.jules/praman-setup.md` | | **GitHub Copilot** | `.github/copilot-instructions.md` or `.github/agents/` | Copilot agent definitions | | **OpenCode** | `.opencode/` | OpenCode configuration | #### Scaffolded project files | File | Purpose | | -------------------------------------------- | -------------------------------------------------------------- | | `playwright.config.ts` | Playwright config with `auth-setup` + `chromium` projects | | `praman.config.ts` | Praman settings (timeouts, log level, interaction strategy) | | `tsconfig.json` | TypeScript config with `module: "Node16"` | | `.gitignore` | Excludes `.env`, `.auth/`, `test-results/`, `node_modules/` | | `tests/auth.setup.ts` | SAP login — runs once before tests, saves session to `.auth/` | | `tests/bom-e2e-praman-gold-standard.spec.ts` | Gold-standard verification test | | `specs/bom-create-complete.plan.md` | AI-generated test plan (reference) | | `.env.example` | Environment variable template | | IDE-specific files | Agent definitions, prompts, rules, snippets (per IDE detected) | | `skills/` | Praman SAP testing skill files for AI agents | #### IDE-specific post-init commands ```bash # Claude Code cat node_modules/playwright-praman/docs/user-integration/claude-md-appendable.md >> CLAUDE.md # GitHub Copilot cat node_modules/playwright-praman/docs/user-integration/copilot-instructions-appendable.md >> .github/copilot-instructions.md # Cursor cat node_modules/playwright-praman/docs/user-integration/cursor-rules-appendable.mdc >> .cursorrules ``` `init` is safe to re-run — it skips files that already exist. Use `--force` to overwrite everything. For more details, see the [Agent & IDE Setup](./agent-setup) guide.
Environment variables reference | Variable | Required | Description | | ------------------------------------ | -------- | ----------------------------------------------- | | `SAP_CLOUD_BASE_URL` | Yes | SAP BTP or on-premise base URL | | `SAP_CLOUD_USERNAME` | Yes | SAP login username | | `SAP_CLOUD_PASSWORD` | Yes | SAP login password | | `SAP_AUTH_STRATEGY` | Yes | Auth strategy: `btp-saml`, `basic`, `office365` | | `SAP_CLIENT` | No | SAP client number (default: from system) | | `SAP_LANGUAGE` | No | Display language (default: EN) | | `PRAMAN_LOG_LEVEL` | No | Log level: `debug`, `info`, `warn`, `error` | | `PRAMAN_SKIP_VERSION_CHECK` | No | Set `true` to skip Playwright version check | | `PRAMAN_AI_PROVIDER` | No | AI provider: `openai`, `azure`, `anthropic` | | `PRAMAN_AI_API_KEY` | No | AI provider API key | | `PRAMAN_AI_MODEL` | No | AI model name | | `PRAMAN_AI_TEMPERATURE` | No | AI temperature (number, e.g. `0.3`) | | `PRAMAN_AI_ENDPOINT` | No | AI endpoint URL (Azure OpenAI) | | `PRAMAN_AI_DEPLOYMENT` | No | Azure OpenAI deployment name | | `PRAMAN_AI_API_VERSION` | No | Azure OpenAI API version | | `PRAMAN_AI_ANTHROPIC_API_KEY` | No | Anthropic API key (Claude models) | | `PRAMAN_TELEMETRY_ENABLED` | No | Enable OpenTelemetry (`true`/`false`) | | `PRAMAN_TELEMETRY_ENDPOINT` | No | OTel collector endpoint URL | | `PRAMAN_TELEMETRY_SERVICE_NAME` | No | OTel service name | | `PRAMAN_TELEMETRY_EXPORTER` | No | Exporter: `otlp`, `jaeger`, `azure-monitor` | | `PRAMAN_TELEMETRY_PROTOCOL` | No | Transport: `http` or `grpc` | | `PRAMAN_TELEMETRY_METRICS_ENABLED` | No | Enable metrics (`true`/`false`) | | `PRAMAN_TELEMETRY_BATCH_TIMEOUT` | No | Batch export timeout (ms) | | `PRAMAN_TELEMETRY_MAX_QUEUE_SIZE` | No | Max queued spans before dropping | | `PRAMAN_TELEMETRY_CONNECTION_STRING` | No | Azure Monitor connection string | | `PRAMAN_ODATA_TRACING_ENABLED` | No | Enable OData request tracing (`true`/`false`) | For auth strategy details, see the [Authentication](./authentication) guide.
## Step 2: Generate Tests with AI Agents Submit a Signavio flow, a test case description, or a business process in plain language. Praman's AI agents connect to your live SAP system, discover every UI5 control, generate a structured test plan, and deliver production-ready Playwright scripts — covering SAP end-to-end quality from requirement to deployment evidence. No scripting. No selectors. No coding. ### Prompt template Copy and paste this into your AI agent (Claude Code, Copilot, Cursor, Jules), replacing the test case with your business process: ```text Goal: Create SAP test case and test script 1. Use praman SAP planner agent: .github/agents/praman-sap-planner.agent.md 2. Login using credentials in .env file and use Chrome in headed mode. Do not use sub-agents. 3. Ensure you use UI5 query and capture UI5 methods at each step. Use UI5 methods for all control interactions. 4. Use seed file: tests/seeds/sap-seed.spec.ts 5. Here is the test case: Login to SAP and ensure you are on the landing page. Step 1: Navigate to Maintain Bill of Material app and click Create BOM - expect: Create BOM dialog opens with all fields visible Step 2: Select Material via Value Help — pick a valid material - expect: Material field is populated with selected material Step 3: Select Plant via Value Help — pick plant matching the material - expect: Plant field is populated with selected plant Step 4: Select BOM Usage "Production (1)" from dropdown - expect: BOM Usage field shows "Production (1)" Step 5: Verify all required fields are filled before submission - expect: Material field has a value - expect: BOM Usage field has value "Production (1)" - expect: Valid From date is set - expect: Create BOM button is enabled Step 6: Click "Create BOM" submit button in dialog footer - expect: If valid combination — dialog closes, BOM created, user returns to list report - expect: If invalid combination — error message dialog appears Step 7: If error occurs, close error dialog and cancel - expect: Error dialog closes - expect: Create BOM dialog closes - expect: User returns to list report Output: - Test plan: specs/ - Test script: tests/e2e/sap-cloud/ ``` **Claude Code shortcut** — run the full pipeline with a single slash command: ```bash /praman-sap-coverage ``` **GitHub Copilot / Cursor / Jules:** Use the agent definitions installed by `init` (see [Agent & IDE Setup](./agent-setup)). :::info Playwright CLI — token-efficient alternative Praman agents can also connect to the browser via the **Playwright CLI** (`@playwright/cli`) instead of MCP. The CLI uses shell commands instead of JSON-RPC tool calls, which reduces token usage in agent conversations. Both MCP and CLI are first-class options — see the [Playwright CLI Setup](./playwright-cli-setup) guide for installation and configuration. ::: ### The plan → generate → heal pipeline ```text ┌─────────────────────────────────────────────────────────────────────┐ │ Your prompt (business process) │ └──────────────────────────────┬──────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────┐ │ 1. PLANNER │ │ Connects to live SAP system → discovers UI5 controls via │ │ sap.ui.getCore() → maps SmartFields, Value Helps, OData bindings │ │ → produces structured test plan (specs/*.plan.md) │ └──────────────────────────────┬───────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────┐ │ 2. GENERATOR │ │ Reads test plan → generates Playwright + Praman test code │ │ → typed control proxies (not DOM selectors) → test.step() pattern │ │ → outputs tests/e2e/{app}/*.spec.ts │ └──────────────────────────────┬───────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────┐ │ 3. HEALER │ │ Runs generated test → captures failures → fixes selectors, timing, │ │ dialog handling → re-runs → repeats until green │ └──────────────────────────────┬───────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────┐ │ ✔ Production-ready .spec.ts — no manual scripting required │ └──────────────────────────────────────────────────────────────────────┘ ``` ### Output: test plan The planner agent explores your live SAP system and produces a structured test plan:
specs/bom-create.plan.md (click to expand) ```markdown # Maintain Bill of Material — Test Plan ## Application Overview SAP S/4HANA Cloud — Maintain Bill of Material (Version 2) Fiori Elements V4 List Report with Create BOM dialog UI5 Version: 1.142.4 | OData: V2 | Smart Controls ## Test Scenario: Complete BOM Creation Flow ### Steps 1. Navigate to Maintain BOM app from FLP - expect: List Report loads with filter bar and "Create BOM" button 2. Click "Create BOM" button - expect: Dialog opens with Material (required), Plant, BOM Usage (required), Alternative BOM, Change Number, Valid From (pre-filled) 3. Select Material via Value Help - expect: "Select: Material" dialog opens with Material and Description columns - expect: Material field populated after row selection 4. Select Plant via Value Help - expect: "Select: Plant" dialog opens with Plant, Plant Name columns - expect: Plant field populated after row selection 5. Select BOM Usage "Production (1)" from dropdown - expect: BOM Usage field shows "Production (1)" 6. Verify all required fields and submit - expect: Material, BOM Usage, Valid From all have values - expect: Create BOM button is enabled 7. Handle result - expect: Success → dialog closes, user returns to list report - expect: Error → error dialog appears, close and cancel gracefully ```
### Output: generated test script The generator converts the plan into a production-ready `.spec.ts` using Praman fixtures: ```typescript const IDS = { materialField: 'createBOMFragment--material', materialInput: 'createBOMFragment--material-input', materialVHIcon: 'createBOMFragment--material-input-vhi', bomUsageCombo: 'createBOMFragment--variantUsage-comboBoxEdit', okBtn: 'createBOMFragment--OkBtn', cancelBtn: 'createBOMFragment--CancelBtn', } as const; test.describe('BOM Creation — Complete Flow', () => { test('Create BOM end-to-end', async ({ page, ui5, ui5Navigation }) => { await test.step('Step 1: Navigate to app', async () => { await ui5Navigation.navigateToTile('Maintain Bill Of Material'); await ui5.waitForUI5(); }); await test.step('Step 2: Open Create BOM dialog', async () => { await ui5.press({ controlType: 'sap.m.Button', properties: { text: 'Create BOM' }, }); await ui5.waitForUI5(); const materialField = await ui5.control({ id: IDS.materialField, searchOpenDialogs: true, }); expect(await materialField.getRequired()).toBe(true); }); await test.step('Step 3: Select Material via Value Help', async () => { await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true }); // Agent discovers table structure, selects first valid row await ui5.waitForUI5(); }); await test.step('Step 4: Select BOM Usage', async () => { const combo = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true, }); await combo.setSelectedKey('1'); // Production await combo.fireChange({ value: '1' }); await ui5.waitForUI5(); }); await test.step('Step 5: Submit and verify', async () => { await ui5.press({ id: IDS.okBtn, searchOpenDialogs: true }); await ui5.waitForUI5(); }); }); }); ``` **Key patterns** — every generated test uses `ui5.control()` with typed proxies (not DOM selectors), `searchOpenDialogs: true` for dialog controls, `setValue()` + `fireChange()` + `waitForUI5()` for inputs, and `test.step()` for structured reporting. :::tip From business process to Playwright test — autonomously This is the core value of Praman: you describe **what** to test in business language, and the AI agents handle the **how** — discovering controls, generating code, and healing failures. No manual selector hunting, no brittle DOM queries, no test scripting. ::: For detailed agent prompt templates, example output, and the full heal cycle, see [Running Your Agent for the First Time](./running-your-agent). --- ## Optional: Verify Setup Manually
Run the gold-standard verification test :::note SAP System Requirement The gold-standard BOM test requires access to an **SAP S/4HANA Public Cloud** system or an **SAP Fiori Launchpad** where the **Bill of Material (BOM) Maintenance** tile is available. If your system does not have this app, you can still verify your setup by writing a test against any SAP Fiori app available on your launchpad — the authentication and fixture wiring will work the same way. ::: ```bash npx playwright test tests/bom-e2e-praman-gold-standard.spec.ts --reporter=line --headed --project=chromium ``` | Flag | Purpose | | -------------------- | ------------------------------------------------------------------------------------------ | | `--project=chromium` | Runs the `chromium` project, which depends on `auth-setup` (auth runs first automatically) | | `--headed` | Opens a visible browser so you can watch the test interact with SAP | | `--reporter=line` | Compact one-line-per-step output | A passing test confirms: Playwright + Chromium installed, SAP credentials valid, auth session saved, and Praman fixtures interacting with live UI5 controls.
Troubleshooting | Symptom | Likely Cause | Fix | | ---------------------------------------------- | ------------------------------------------ | --------------------------------------------------------- | | `ERR_BRIDGE_TIMEOUT` | URL is not a UI5 app, or UI5 hasn't loaded | Verify `SAP_CLOUD_BASE_URL` points to the Fiori Launchpad | | Auth setup fails | Wrong credentials or auth strategy | Check `.env` — try `SAP_AUTH_STRATEGY=basic` for OnPrem | | `browserType.launch: Executable doesn't exist` | Chromium not installed | Run `npx playwright install chromium` | | Test times out at tile click | FLP space/tile name differs on your system | Edit `TEST_DATA` constants in the test file | | `ERR_CONTROL_NOT_FOUND` | Control ID differs on your system | Use `controlType` + `properties` instead of `id` | :::info Adapt the gold-standard test to your app The gold-standard test targets the "Maintain Bill Of Material" app. If your SAP system does not have this app, use it as a template: copy the file, change the tile name, control IDs, and test data to match your app. The patterns (auth, navigation, dialog handling, value help) are universal across SAP Fiori apps. :::
---
Optional: Your First Custom Test (manual approach) ## Your First Custom Test Create `tests/purchase-order.spec.ts`: ```typescript test('navigate to Purchase Order app and verify table', async ({ ui5, ui5Navigation }) => { // Step 1: Navigate to the Fiori app await test.step('Open Purchase Order app', async () => { await ui5Navigation.navigateToApp('PurchaseOrder-manage'); }); // Step 2: Wait for UI5 to stabilize await test.step('Wait for page load', async () => { await ui5.waitForUI5(); }); // Step 3: Discover a control by type await test.step('Find the Create button', async () => { const createBtn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create' }, }); // The control proxy exposes all UI5 methods const text = await createBtn.getText(); expect(text).toBe('Create'); }); // Step 4: Read table data await test.step('Verify table has rows', async () => { const rowCount = await ui5.table.getRowCount('myTable'); expect(rowCount).toBeGreaterThan(0); }); }); ```
## Real-World Example: BOM End-to-End Test Praman's AI agents follow a **plan → generate → heal** pipeline. The planner explores a live SAP system, discovers controls, and produces a structured test plan. The generator converts the plan into executable Playwright + Praman test code. Below is a real example from an SAP S/4HANA Cloud BOM application. ### AI-Generated Test Plan The AI planner agent explored the Maintain Bill of Material app and produced this test plan:
bom-create.plan.md (click to expand) #### Application Overview SAP S/4HANA Cloud — Maintain Bill of Material (Version 2). Fiori Elements V4 List Report with Create BOM dialog. UI5 Version 1.142.4. System: Partner Demo Customizing LXG/100. #### 1. BOM Navigation and App Loading ##### 1.1. Navigate to Maintain BOM app from FLP | # | Step | Expected Result | | --- | ----------------------------------------------------- | ---------------------------------------------------------------------------- | | 1 | Verify FLP Home page loaded after login | Page title contains "Home"; Shell Bar visible with "S/4HANA Cloud" | | 2 | Click "Bills Of Material" tab in FLP space navigation | Tab becomes active; BOM tiles visible | | 3 | Click "Maintain Bill Of Material (Version 2)" tile | URL changes to `#MaterialBOM-maintainMaterialBOM` | | 4 | Wait for List Report to load | Filter bar visible (Material, Plant, BOM Usage); "Create BOM" button enabled | #### 2. Create BOM Dialog Interactions ##### 2.1. Open Create BOM dialog and verify fields | # | Step | Expected Result | | --- | ------------------------- | --------------------------------------------------------------------------------------------------------- | | 1 | Click "Create BOM" button | Dialog opens with heading "Create BOM" | | 2 | Verify all dialog fields | Material (required), Plant, BOM Usage (required), Alternative BOM, Change Number, Valid From (pre-filled) | | 3 | Verify footer buttons | "Create BOM" submit and "Cancel" buttons visible and enabled | | 4 | Click "Cancel" | Dialog closes; List Report visible | ##### 2.2. Test Material Value Help | # | Step | Expected Result | | --- | -------------------------------------- | --------------------------------------------------------------------- | | 1 | Click "Create BOM" to open dialog | Dialog opens | | 2 | Click Value Help icon next to Material | "Select: Material" dialog opens with Material and Description columns | | 3 | Verify material data loaded | At least one row with Material number and description | | 4 | Click a material row | Value help closes; Material field populated | | 5 | Click "Cancel" to close dialog | Dialog closes cleanly | ##### 2.3. Test Plant Value Help | # | Step | Expected Result | | --- | ----------------------------------- | ----------------------------------------------------------- | | 1 | Click "Create BOM" to open dialog | Dialog opens | | 2 | Click Value Help icon next to Plant | "Select: Plant" dialog opens with Plant, Plant Name columns | | 3 | Verify plant data loaded | Plants listed (e.g. 1010 DE Plant, 1110 GB Plant) | | 4 | Click a plant row | Value help closes; Plant field populated | | 5 | Click "Cancel" to close dialog | Dialog closes cleanly | ##### 2.4. Test BOM Usage dropdown | # | Step | Expected Result | | --- | --------------------------------- | --------------------------------------------------------------------------------------- | | 1 | Click "Create BOM" to open dialog | Dialog opens | | 2 | Click Value Help on BOM Usage | Dropdown opens with usage types | | 3 | Verify options | Production (1), Engineering/Design (2), Universal (3), Plant Maintenance (4), Sales (5) | | 4 | Select "Production (1)" | BOM Usage field shows "Production (1)" | | 5 | Click "Cancel" to close dialog | Dialog closes cleanly | #### 3. Complete BOM Creation Flow ##### 3.1. Fill all fields and submit | # | Step | Expected Result | | --- | --------------------------------------- | ---------------------------------------------------------------------------- | | 1 | Navigate to app, click "Create BOM" | Dialog opens | | 2 | Select Material via Value Help | Material field populated | | 3 | Select Plant via Value Help | Plant field populated | | 4 | Select BOM Usage "Production (1)" | BOM Usage set | | 5 | Verify all required fields filled | Material, BOM Usage, Valid From all have values; Create BOM enabled | | 6 | Click "Create BOM" submit | Success: dialog closes, returns to list. Error: error message dialog appears | | 7 | If error, close error dialog and cancel | User returns to list report | ##### 3.2. Verify validation errors | # | Step | Expected Result | | --- | ------------------------------------------- | -------------------------------------------------------------------- | | 1 | Open Create BOM dialog | Dialog opens | | 2 | Select invalid Material + Plant combination | Both fields populated | | 3 | Select BOM Usage "Production (1)" | BOM Usage set | | 4 | Click "Create BOM" submit | Error dialog: "Material not maintained in plant" with message M3351 | | 5 | Click "Close" on error dialog | Error dialog closes; Create BOM dialog remains with values preserved | | 6 | Click "Cancel" | Dialog closes; returns to list report |
### Generated Gold Standard Test The AI generator converted scenario **3.1** (Complete BOM Creation Flow) from the plan above into the following production-grade test. Copy it into your project: ```bash cp node_modules/playwright-praman/examples/bom-e2e-praman-gold-standard.spec.ts tests/ ``` Here is the full test — 8 steps covering navigation, dialog handling, value help tables, OData-driven data selection, form fill, and graceful error recovery:
bom-e2e-praman-gold-standard.spec.ts (click to expand) ```typescript // ── V2 SmartField Control ID Constants ────────────────────────────── const IDS = { // ── Dialog fields (sap.ui.comp.smartfield.SmartField) ── materialField: 'createBOMFragment--material', materialInput: 'createBOMFragment--material-input', materialVHIcon: 'createBOMFragment--material-input-vhi', materialVHDialog: 'createBOMFragment--material-input-valueHelpDialog', materialVHTable: 'createBOMFragment--material-input-valueHelpDialog-table', plantField: 'createBOMFragment--plant', plantInput: 'createBOMFragment--plant-input', plantVHIcon: 'createBOMFragment--plant-input-vhi', plantVHDialog: 'createBOMFragment--plant-input-valueHelpDialog', plantVHTable: 'createBOMFragment--plant-input-valueHelpDialog-table', bomUsageField: 'createBOMFragment--variantUsage', bomUsageCombo: 'createBOMFragment--variantUsage-comboBoxEdit', // ── Dialog buttons ── okBtn: 'createBOMFragment--OkBtn', cancelBtn: 'createBOMFragment--CancelBtn', } as const; // ── Test Data ─────────────────────────────────────────────────────── const TEST_DATA = { flpSpaceTab: 'Bills Of Material', tileHeader: 'Maintain Bill Of Material', bomUsageKey: '1', // Production } as const; test.describe('BOM End-to-End Flow', () => { test('Complete BOM Flow - V2 SmartField Single Session', async ({ page, ui5 }) => { // Step 1: Navigate to BOM Maintenance App await test.step('Step 1: Navigate to BOM Maintenance App', async () => { await page.goto(process.env.SAP_CLOUD_BASE_URL!); await page.waitForLoadState('domcontentloaded'); await expect(page).toHaveTitle(/Home/, { timeout: 60000 }); // FLP space tabs — DOM click is the only reliable method await page.getByText(TEST_DATA.flpSpaceTab, { exact: true }).click(); // Click tile with toPass() retry for slow SAP systems await expect(async () => { await ui5.press({ controlType: 'sap.m.GenericTile', properties: { header: TEST_DATA.tileHeader }, }); }).toPass({ timeout: 60000, intervals: [5000, 10000] }); // Wait for Create BOM button — proves V1 app loaded let createBtn: Awaited>; await expect(async () => { createBtn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create BOM' }, }); const text = await createBtn.getProperty('text'); expect(text).toBe('Create BOM'); }).toPass({ timeout: 120000, intervals: [5000, 10000] }); }); // Step 2: Open Create BOM Dialog and Verify Structure await test.step('Step 2: Open Create BOM Dialog', async () => { await ui5.press({ controlType: 'sap.m.Button', properties: { text: 'Create BOM' }, }); await ui5.waitForUI5(); // Verify dialog opened — Material SmartField exists inside dialog const materialField = await ui5.control({ id: IDS.materialField, searchOpenDialogs: true, }); const materialType = await materialField.getControlType(); expect(materialType).toBe('sap.ui.comp.smartfield.SmartField'); expect(await materialField.getRequired()).toBe(true); // Verify BOM Usage SmartField const bomUsageField = await ui5.control({ id: IDS.bomUsageField, searchOpenDialogs: true, }); expect(await bomUsageField.getControlType()).toBe('sap.ui.comp.smartfield.SmartField'); expect(await bomUsageField.getRequired()).toBe(true); // Verify dialog footer buttons const createDialogBtn = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true, }); expect(await createDialogBtn.getProperty('text')).toBe('Create'); const cancelDialogBtn = await ui5.control({ id: IDS.cancelBtn, searchOpenDialogs: true, }); expect(await cancelDialogBtn.getProperty('text')).toBe('Cancel'); }); // Step 3: Test Material Value Help await test.step('Step 3: Test Material Value Help', async () => { await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true }); const materialDialog = await ui5.control({ id: IDS.materialVHDialog, searchOpenDialogs: true, }); expect(await materialDialog.isOpen()).toBe(true); await ui5.waitForUI5(); // Get inner table via SmartTable.getTable() const smartTable = await ui5.control({ id: IDS.materialVHTable, searchOpenDialogs: true, }); const innerTable = await smartTable.getTable(); const rows = (await innerTable.getRows()) as unknown[]; expect(rows.length).toBeGreaterThan(0); // Wait for OData data to load await expect(async () => { let count = 0; for (const row of rows) { const ctx = await ( row as { getBindingContext: () => Promise } ).getBindingContext(); if (ctx) count++; } expect(count).toBeGreaterThan(0); }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); await materialDialog.close(); await ui5.waitForUI5(); }); // Step 4: Test Plant Value Help await test.step('Step 4: Test Plant Value Help', async () => { await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true }); const plantDialog = await ui5.control({ id: IDS.plantVHDialog, searchOpenDialogs: true, }); expect(await plantDialog.isOpen()).toBe(true); await ui5.waitForUI5(); const smartTable = await ui5.control({ id: IDS.plantVHTable, searchOpenDialogs: true, }); const innerTable = await smartTable.getTable(); const rows = (await innerTable.getRows()) as unknown[]; await expect(async () => { let count = 0; for (const row of rows) { const ctx = await ( row as { getBindingContext: () => Promise } ).getBindingContext(); if (ctx) count++; } expect(count).toBeGreaterThan(0); }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); await plantDialog.close(); await ui5.waitForUI5(); }); // Step 5: Test BOM Usage Dropdown (Inner ComboBox) await test.step('Step 5: Test BOM Usage Dropdown', async () => { const bomUsageCombo = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true, }); // Get items via proxy const rawItems = await bomUsageCombo.getItems(); const items: Array<{ key: string; text: string }> = []; if (Array.isArray(rawItems)) { for (const itemProxy of rawItems) { const key = await itemProxy.getKey(); const text = await itemProxy.getText(); items.push({ key: String(key), text: String(text) }); } } expect(items.length).toBeGreaterThan(0); // Verify open/close cycle await bomUsageCombo.open(); await ui5.waitForUI5(); expect(await bomUsageCombo.isOpen()).toBe(true); await bomUsageCombo.close(); await ui5.waitForUI5(); expect(await bomUsageCombo.isOpen()).toBe(false); }); // Step 6: Fill Form with Valid Data await test.step('Step 6: Fill Form with Valid Data', async () => { // === FILL MATERIAL === await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true }); const materialDialogControl = await ui5.control({ id: IDS.materialVHDialog, searchOpenDialogs: true, }); await expect(async () => { expect(await materialDialogControl.isOpen()).toBe(true); }).toPass({ timeout: 30000, intervals: [1000, 2000] }); const smartTableMat = await ui5.control({ id: IDS.materialVHTable, searchOpenDialogs: true, }); const innerTableMat = await smartTableMat.getTable(); let materialValue = ''; await expect(async () => { const ctx = await innerTableMat.getContextByIndex(0); expect(ctx).toBeTruthy(); const data = (await ctx.getObject()) as { Material?: string }; expect(data?.Material).toBeTruthy(); materialValue = data!.Material!; }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); await materialDialogControl.close(); await ui5.waitForUI5(); await ui5.fill({ id: IDS.materialInput, searchOpenDialogs: true }, materialValue); await ui5.waitForUI5(); // === FILL PLANT === await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true }); const plantDialogControl = await ui5.control({ id: IDS.plantVHDialog, searchOpenDialogs: true, }); await expect(async () => { expect(await plantDialogControl.isOpen()).toBe(true); }).toPass({ timeout: 30000, intervals: [1000, 2000] }); const smartTablePlant = await ui5.control({ id: IDS.plantVHTable, searchOpenDialogs: true, }); const innerTablePlant = await smartTablePlant.getTable(); let plantValue = ''; await expect(async () => { const ctx = await innerTablePlant.getContextByIndex(0); expect(ctx).toBeTruthy(); const data = (await ctx.getObject()) as { Plant?: string }; expect(data?.Plant).toBeTruthy(); plantValue = data!.Plant!; }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); await plantDialogControl.close(); await ui5.waitForUI5(); await ui5.fill({ id: IDS.plantInput, searchOpenDialogs: true }, plantValue); await ui5.waitForUI5(); // === FILL BOM USAGE === const bomUsageControl = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true, }); await bomUsageControl.open(); await ui5.waitForUI5(); await bomUsageControl.setSelectedKey(TEST_DATA.bomUsageKey); await bomUsageControl.fireChange({ value: TEST_DATA.bomUsageKey }); await bomUsageControl.close(); await ui5.waitForUI5(); // === VERIFY ALL VALUES === const finalMaterial = (await ui5.getValue({ id: IDS.materialInput, searchOpenDialogs: true, })) ?? ''; const finalPlant = (await ui5.getValue({ id: IDS.plantInput, searchOpenDialogs: true, })) ?? ''; const finalBomUsageCtrl = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true, }); const finalBomUsageKey = (await finalBomUsageCtrl.getSelectedKey()) ?? ''; expect(finalMaterial).toBe(materialValue); expect(finalPlant).toBe(plantValue); expect(finalBomUsageKey).toBe(TEST_DATA.bomUsageKey); }); // Step 7: Click Create Button and Handle Result await test.step('Step 7: Click Create and handle result', async () => { const createBtn = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true, }); expect(await createBtn.getProperty('text')).toBe('Create'); expect(await createBtn.getEnabled()).toBe(true); await ui5.press({ id: IDS.okBtn, searchOpenDialogs: true }); await ui5.waitForUI5(); // Graceful error recovery — close error dialogs, cancel if needed let dialogStillOpen = false; try { const okBtnCheck = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true, }); dialogStillOpen = (await okBtnCheck.getEnabled()) !== undefined; } catch { dialogStillOpen = false; } if (dialogStillOpen) { try { await ui5.press({ id: IDS.cancelBtn, searchOpenDialogs: true }); await ui5.waitForUI5(); } catch { // Dialog already closed } } }); // Step 8: Verify Return to BOM List await test.step('Step 8: Verify return to BOM List Report', async () => { const createBtn = await ui5.control( { controlType: 'sap.m.Button', properties: { text: 'Create BOM' }, }, { timeout: 30000 }, ); expect(await createBtn.getProperty('text')).toBe('Create BOM'); expect(await createBtn.getProperty('enabled')).toBe(true); }); }); }); ```
**Key patterns demonstrated:** | Pattern | Example | | ----------------------------------- | --------------------------------------------------- | | `searchOpenDialogs: true` | All controls inside the Create BOM dialog | | `ui5.fill()` | Atomic `setValue` + `fireChange` + `waitForUI5` | | `SmartTable.getTable()` | Access inner `sap.ui.table.Table` from SmartTable | | `getContextByIndex().getObject()` | Read OData entity data from table rows | | `setSelectedKey()` + `fireChange()` | ComboBox selection (localization-safe) | | `toPass()` retry | Handle slow OData loads and SAP system latency | | Graceful error recovery | Close error dialogs and cancel without hard failure | :::info From plan to code Both the test plan and the gold standard test above were produced by Praman's AI agents against a live SAP S/4HANA Cloud system. Run the full **plan → generate → heal** pipeline on your own SAP app with the `/praman-sap-coverage` prompt. ::: ## Running Tests ```bash # Run all tests npx playwright test # Run a specific test file npx playwright test tests/purchase-order.spec.ts # Run with visible browser npx playwright test --headed # Run with Playwright UI mode npx playwright test --ui ``` :::tip Why `ui5.control()` instead of `page.locator()`? SAP UI5 renders controls dynamically -- DOM IDs change between versions, themes restructure elements, and controls nest inside generated wrappers. `ui5.control()` discovers controls through the **UI5 runtime's own control registry**, which is the stable contract. Your tests survive DOM restructuring, theme changes, and UI5 version upgrades. ::: ## Import Pattern Praman uses Playwright's `mergeTests()` to combine all fixtures into a single `test` object: ```typescript // All fixtures available via destructuring: test('full access', async ({ ui5, // Control discovery + interaction ui5Navigation, // FLP navigation sapAuth, // Authentication fe, // Fiori Elements helpers pramanAI, // AI page discovery intent, // Business domain intents ui5Shell, // FLP shell header ui5Footer, // Page footer bar flpLocks, // SM12 lock management flpSettings, // User settings testData, // Test data generation }) => { // ... }); ``` ## Common Patterns ### Control Discovery ```typescript // By ID const btn = await ui5.control({ id: 'submitBtn' }); // By control type + properties const input = await ui5.control({ controlType: 'sap.m.Input', properties: { placeholder: 'Enter vendor' }, }); // Multiple controls const buttons = await ui5.controls({ controlType: 'sap.m.Button' }); ``` ### Input Fields (Gold Pattern) Always use `setValue()` + `fireChange()` + `waitForUI5()` together: ```typescript const input = await ui5.control({ id: 'vendorInput' }); await input.setValue('SUP-001'); await input.fireChange({ value: 'SUP-001' }); await ui5.waitForUI5(); ``` Or use the shorthand: ```typescript await ui5.fill({ id: 'vendorInput' }, 'SUP-001'); ``` :::warning Never use `page.fill()` or `page.type()` for UI5 Input controls. These bypass the UI5 event model, which means OData bindings, value state updates, and field validation will not trigger. ::: ### Table Operations ```typescript // Read table data const rows = await ui5.table.getRows('purchaseOrderTable'); const columns = await ui5.table.getColumnNames('purchaseOrderTable'); // Find a row by column values const rowIndex = await ui5.table.findRowByValues('purchaseOrderTable', { 'Purchase Order': '4500001234', }); // Click a row await ui5.table.clickRow('purchaseOrderTable', rowIndex); // Get a specific cell value const vendor = await ui5.table.getCellByColumnName('purchaseOrderTable', 0, 'Vendor'); ``` ### Dialog Handling ```typescript // Wait for a dialog to appear const dialog = await ui5.dialog.waitFor(); // Confirm a dialog await ui5.dialog.confirm(); // Dismiss a dialog await ui5.dialog.dismiss(); // Find controls inside a dialog const dialogBtn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'OK' }, searchOpenDialogs: true, }); ``` :::warning Always use `searchOpenDialogs: true` when finding controls inside an `sap.m.Dialog`. Without it, `ui5.control()` only searches the main view. ::: ### Navigation ```typescript // Navigate to a Fiori app by semantic object await ui5Navigation.navigateToApp('PurchaseOrder-manage'); // Navigate to a tile by title await ui5Navigation.navigateToTile('Manage Purchase Orders'); // Navigate to a specific URL hash await ui5Navigation.navigateToHash('#PurchaseOrder-manage&/PurchaseOrders'); // Go back await ui5Navigation.navigateBack(); // Go to FLP home await ui5Navigation.navigateToHome(); ``` ### Fiori Elements Shortcuts ```typescript // List Report await fe.listReport.setFilter('Vendor', 'SUP-001'); await fe.listReport.search(); await fe.listReport.navigateToItem(0); // Object Page const title = await fe.objectPage.getHeaderTitle(); await fe.objectPage.clickEdit(); await fe.objectPage.navigateToSection('Items'); await fe.objectPage.clickSave(); ``` ### Business Intent APIs ```typescript // Create a purchase order using business terms (vocabulary-resolved) await intent.procurement.createPurchaseOrder({ vendor: 'SUP-001', material: 'MAT001', quantity: 10, plant: '1000', companyCode: '1000', purchasingOrg: '1000', }); // Fill a field using business vocabulary await intent.core.fillField('Vendor', 'SUP-001'); ``` ## Fixture Quick Reference | Fixture | Purpose | | --------------- | -------------------------------------------------- | | `ui5` | Core control discovery and interaction | | `ui5.table` | Table read, filter, sort, select | | `ui5.dialog` | Dialog lifecycle (wait, confirm, dismiss) | | `ui5.date` | DatePicker and TimePicker operations | | `ui5.odata` | OData model reads and HTTP operations | | `ui5Navigation` | FLP and in-app navigation | | `ui5Footer` | Footer toolbar buttons (Save, Edit, Cancel) | | `ui5Shell` | Shell header (Home, Notifications, User menu) | | `fe` | Fiori Elements List Report, Object Page, Table | | `intent` | Business intent APIs (procurement, sales, finance) | Always wrap multi-step flows in `test.step()` for clear reporting: ```typescript await test.step('Create purchase order', async () => { // ... multiple interactions }); ``` ## Persona Quick-Start Guides ### Test Automation Engineer Focus on **fixtures, assertions, and `test.step()`** for structured test flows. All fixtures are available via destructuring from the `test` callback. Use `test.step()` to organize multi-step flows for clear HTML reports. ### AI Agent (Copilot / Claude Code) Focus on **imports, capabilities, and error codes** for automated test generation. - **Single import**: `import { test, expect } from 'playwright-praman'` - **Capability query**: Use `pramanAI.capabilities.forAI()` to discover available operations at runtime - **Error codes**: All errors extend `PramanError` with `code`, `retryable`, and `suggestions[]` - **Forbidden patterns**: Never use `page.click('#__...')`, `page.fill('#__...')`, or `page.locator('.sapM...')` for UI5 elements ### SAP Business Analyst Focus on **intents and vocabulary** for writing tests in business terms. ```typescript // Instead of finding controls by ID: await intent.core.fillField('Vendor', 'SUP-001'); await intent.core.fillField('Material', 'MAT001'); await intent.core.fillField('Quantity', '10'); // Or use domain-specific intents: await intent.procurement.createPurchaseOrder({ vendor: 'SUP-001', material: 'MAT001', quantity: 10, plant: '1000', }); ``` Supported vocabulary domains: procurement (MM), sales (SD), finance (FI), manufacturing (PP), warehouse (WM/EWM), quality (QM). ## Project Structure ```text my-sap-tests/ tests/ auth-setup.ts # Authentication (runs once) purchase-order.spec.ts # Your test files sales-order.spec.ts .auth/ sap-session.json # Saved session (gitignored) praman.config.ts playwright.config.ts package.json ``` ## FAQ
npm install fails with permission errors or EACCES Run with the correct permissions. On macOS/Linux, avoid `sudo npm install` — instead, fix npm permissions or use a Node version manager like `nvm`: ```bash nvm install 22 nvm use 22 npm install playwright-praman ``` On Windows, run your terminal as Administrator if needed.
How do I fix "browserType.launch: Executable doesn't exist"? Chromium is not installed. Run: ```bash npx playwright install chromium ``` If you ran `npx playwright-praman init`, this should have been done automatically. Re-run `init` if the browser binary is missing.
Auth setup fails with "ERR_AUTH_TIMEOUT" or wrong credentials Check your `.env` file: ```bash # .env SAP_CLOUD_BASE_URL=https://your-system.s4hana.cloud.sap/ SAP_CLOUD_USERNAME=your-user SAP_CLOUD_PASSWORD=your-password SAP_AUTH_STRATEGY=btp-saml ``` Common issues: - `SAP_CLOUD_BASE_URL` must point to the Fiori Launchpad URL, not the API endpoint - `SAP_AUTH_STRATEGY` must match your system type (`btp-saml`, `basic`, or `office365`) - Verify credentials work by logging in manually in a browser first See the [Authentication](./authentication) guide for strategy-specific troubleshooting.
Can I use Praman without AI agents (manual test writing)? Yes. AI agents are optional. You can write tests manually using Praman fixtures: ```typescript // tests/my-test.spec.ts test('manual test', async ({ ui5, ui5Navigation }) => { await ui5Navigation.navigateToApp('PurchaseOrder-manage'); const btn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create' } }); await expect(btn).toBeUI5Enabled(); }); ``` Skip the agent setup steps and go directly to writing tests with fixtures. See the [Playwright Primer](./playwright-primer) for a ground-up introduction.
:::warning[Common mistake] Do not use `page.click()` or `page.fill()` for UI5 controls. These bypass the UI5 event model, causing OData bindings, validations, and value state updates to not trigger. Always use `ui5.press()`, `ui5.fill()`, and `ui5.control()` for UI5 elements. ::: ## Next Steps | Topic | Documentation | | ------------------------- | ---------------------------------------------- | | Configuration reference | [Configuration](./configuration) | | Authentication strategies | [Authentication](./authentication) | | Selector reference | [Selectors](./selectors) | | Fixture reference | [Fixtures](./fixtures) | | Error reference | [Errors](./errors) | | Agent & IDE setup | [Agent Setup](./agent-setup) | | Playwright CLI setup | [Playwright CLI Setup](./playwright-cli-setup) | | Vocabulary system | [Vocabulary](./vocabulary-system) | | Intent API | [Intent API](./intent-api) | | Examples | [Examples](../examples/) | | Architecture overview | [Architecture](./architecture-overview) | :::tip[Next steps] - **[Authentication Guide →](./authentication)** — Configure SAP login strategies (BTP SAML, Basic Auth, Office 365) - **[Selectors Guide →](./selectors)** — Learn how to find UI5 controls by ID, type, properties, and binding path - **[Agent & IDE Setup →](./agent-setup)** — Set up AI agents for Claude Code, Copilot, Cursor, and VS Code ::: --- ## Localization & i18n Testing SAP applications serve a global user base. Testing across languages, locales, and text directions ensures that UI elements render correctly regardless of the user's language settings. This guide covers language-independent selectors, locale switching, RTL testing, and format validation. ## Language-Independent Selectors with i18NText Hardcoded text selectors like `{ text: 'Save' }` break when the app runs in German (`Sichern`) or Japanese. Praman's `i18NText` selector matches controls by their i18n resource bundle key instead of the displayed text: ```typescript test('click save button regardless of language', async ({ ui5 }) => { await ui5.click({ controlType: 'sap.m.Button', i18NText: { propertyName: 'text', key: 'SAVE_BUTTON' }, }); }); ``` The `i18NText` selector resolves the resource bundle key at runtime, matching the control whose bound text property points to that key. This makes tests language-agnostic. ### How i18NText Resolution Works 1. Praman reads the control's binding info for the specified property (`text`). 2. It checks whether the binding references a resource bundle model (typically `i18n`). 3. It compares the bound key against the `key` parameter you provided. 4. If they match, the control is selected -- regardless of what the displayed text says. ```typescript // Match by specific resource bundle model name await ui5.click({ controlType: 'sap.m.Button', i18NText: { propertyName: 'text', key: 'SUBMIT_ORDER', modelName: 'i18n', // optional, defaults to 'i18n' }, }); ``` ## Switching Languages via SAP User Settings To test a specific locale, set the `sap-language` URL parameter before navigating: ```typescript test('verify German labels', async ({ ui5, page }) => { await page.goto('/app?sap-language=DE'); await ui5.waitForUI5(); await test.step('Verify German text on save button', async () => { const buttonText = await ui5.getText({ controlType: 'sap.m.Button', i18NText: { propertyName: 'text', key: 'SAVE_BUTTON' }, }); expect(buttonText).toBe('Sichern'); }); }); ``` ### Testing Multiple Languages Use parameterized tests to verify translations across locales: ```typescript const locales = [ { lang: 'EN', expected: 'Save' }, { lang: 'DE', expected: 'Sichern' }, { lang: 'FR', expected: 'Enregistrer' }, ]; for (const { lang, expected } of locales) { test(`save button label in ${lang}`, async ({ ui5, page }) => { await page.goto(`/app?sap-language=${lang}`); await ui5.waitForUI5(); const text = await ui5.getText({ controlType: 'sap.m.Button', i18NText: { propertyName: 'text', key: 'SAVE_BUTTON' }, }); expect(text).toBe(expected); }); } ``` ## FLP Language Parameter When testing inside the SAP Fiori Launchpad, set the language via the FLP URL hash parameter: ```typescript test('open FLP in French', async ({ page, ui5 }) => { await page.goto('/sap/bc/ui2/flp?sap-language=FR#Shell-home'); await ui5.waitForUI5(); }); ``` The `sap-language` parameter must appear before the `#` hash. It affects all apps launched from the FLP during that session. ## RTL Layout Testing Arabic, Hebrew, and other right-to-left (RTL) languages require layout verification. SAP UI5 automatically mirrors the layout when an RTL locale is active: ```typescript test('verify RTL layout for Arabic', async ({ ui5, page }) => { await page.goto('/app?sap-language=AR'); await ui5.waitForUI5(); await test.step('Check document direction', async () => { const dir = await page.evaluate(() => document.documentElement.dir); expect(dir).toBe('rtl'); }); await test.step('Verify toolbar alignment', async () => { const toolbar = page.locator('.sapMTB'); await expect(toolbar).toHaveCSS('direction', 'rtl'); }); }); ``` ## Date and Number Format Validation SAP formats dates and numbers according to the user's locale. Validate that formatted values match locale expectations: ```typescript test('date format matches German locale', async ({ ui5, page }) => { await page.goto('/app?sap-language=DE'); await ui5.waitForUI5(); const dateValue = await ui5.getText({ id: 'createdAtField' }); // German date format: DD.MM.YYYY expect(dateValue).toMatch(/^\d{2}\.\d{2}\.\d{4}$/); }); test('number format matches French locale', async ({ ui5, page }) => { await page.goto('/app?sap-language=FR'); await ui5.waitForUI5(); const amount = await ui5.getText({ id: 'totalAmount' }); // French uses comma as decimal separator and space as thousands separator expect(amount).toMatch(/[\d\s]+,\d{2}/); }); ``` ## Common Pitfalls - **Hardcoded text assertions**: Never assert against hardcoded display text in multi-locale tests. Use `i18NText` for selectors and verify the text only when explicitly testing a specific locale. - **Missing translations**: If an i18n key is missing from a locale's `.properties` file, UI5 falls back to the default (usually English). Your test might pass even though the translation is absent. - **RTL pseudo-mirrors**: Some CSS properties (like `margin-left`) are automatically mirrored by UI5's RTL support, but custom CSS may not be. Test RTL with actual RTL locales. - **sap-language vs. browser locale**: The `sap-language` URL parameter overrides the browser's `Accept-Language` header. If you need browser-locale-based behavior, set the locale in the Playwright browser context instead. - **Session stickiness**: Once the FLP establishes a language, navigating to a different app preserves it. Close and reopen the browser context to reset the language cleanly. --- ## Parallel Execution & Sharding Running SAP UI5 tests in parallel dramatically reduces feedback time. This guide covers Playwright's parallelization model, SAP-specific considerations for auth and locking, and CI sharding strategies. ## Enabling Parallel Execution Playwright supports parallelism at two levels: across test files (workers) and within a file (fullyParallel). Enable both in your config: ```typescript export default defineConfig({ fullyParallel: true, workers: process.env.CI ? 4 : undefined, // undefined = half CPU cores }); ``` With `fullyParallel: true`, individual `test()` blocks within the same file can run concurrently. Without it, tests in a single file run sequentially while different files run in parallel across workers. ## Worker Isolation with Auth State Each Playwright worker is an independent Node.js process. For SAP tests, share authentication state across workers using Playwright's `storageState` mechanism: ```typescript export default defineConfig({ projects: [ { name: 'auth-setup', testMatch: /auth\.setup\.ts/, }, { name: 'sap-tests', dependencies: ['auth-setup'], use: { storageState: '.auth/user.json', }, }, ], }); ``` The `auth-setup` project runs once before all workers. Each worker then reuses the saved cookies and session tokens without re-authenticating. ## SAP Fiori Launchpad Lock Considerations SAP systems enforce object-level locking. When two parallel workers attempt to edit the same business object (e.g., a sales order), one will encounter an SAP lock error (`ENQUEUE_CONFLICT`). Strategies to avoid lock conflicts: - **Unique test data per worker**: Generate unique entity IDs with `{{uuid}}` so workers never edit the same object. - **Read-only vs. write tests**: Separate display-only tests from edit tests into different projects. Run edit tests with `workers: 1`. - **Dedicated test data pools**: Assign each worker its own pre-created entities using `workerIndex`: ```typescript test('edit order assigned to this worker', async ({ ui5, page }, testInfo) => { const orderPool = ['ORD-001', 'ORD-002', 'ORD-003', 'ORD-004']; const myOrder = orderPool[testInfo.workerIndex % orderPool.length]; await page.goto(`/app#/Orders/${myOrder}`); await ui5.waitForUI5(); }); ``` ## Praman Bridge in Parallel Workers The Praman bridge is injected per-page, not per-worker. Each worker creates its own browser context and page, so bridge injection is fully independent. No special configuration is needed for the bridge in parallel mode. One consideration: if `controlDiscoveryTimeout` is too low and the SAP system is under load from multiple workers, increase it: ```typescript export default { controlDiscoveryTimeout: 45_000, // 45 seconds under parallel load }; ``` ## CI Sharding with GitHub Actions Playwright's built-in sharding distributes test files across multiple CI jobs: ```yaml # .github/workflows/test.yml jobs: test: strategy: matrix: shard: [1, 2, 3, 4] steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22 - run: npm ci - run: npx playwright install --with-deps chromium - run: npx playwright test --shard=${{ matrix.shard }}/4 - uses: actions/upload-artifact@v4 if: always() with: name: results-shard-${{ matrix.shard }} path: test-results/ ``` ## Auth Setup for Sharded Runs When using sharding, every shard must have access to auth state. Use `globalSetup` instead of project dependencies so each shard runs its own login: ```typescript export default defineConfig({ globalSetup: './global-setup.ts', // runs once per shard }); ``` ## Common Pitfalls - **Too many workers against SAP**: SAP systems have session limits. If you see `ERR_AUTH_FAILED` errors in parallel runs, reduce `workers` or use more test accounts. - **Shared mutable state**: Never store test state in module-level variables. Each worker is a separate process, so module state is not shared. Use `testData.save()` and `testData.load()` for cross-step persistence. - **FLP tile cache**: The Fiori Launchpad caches tile data per session. Parallel workers reusing the same `storageState` share the same FLP cache, which is generally safe for read operations. - **Shard imbalance**: Playwright distributes by file, not by test count. If one file has 50 tests and others have 5, sharding will be uneven. Keep test files roughly equal in size. --- ## Playwright Primer for SAP Testers :::info[In this guide] - Understand why Playwright is the right tool for SAP Fiori and UI5 web apps - Learn the TypeScript and async/await fundamentals needed for writing tests - Write and run your first Playwright test with Praman fixtures - Use `test.step()`, assertions, and fixtures to structure SAP test flows - Map familiar SAP testing concepts (ECATT, Tosca) to Playwright equivalents ::: A ground-up introduction to Playwright and Praman for testers coming from SAP testing backgrounds (ECATT, Tosca, QTP/UFT, Solution Manager, CBTA). No prior Playwright or programming experience is assumed. ## What Is Playwright? Playwright is an open-source test automation framework from Microsoft. It drives real browsers (Chrome, Firefox, Safari) to simulate user interactions -- clicking buttons, filling forms, reading text -- and verifies that applications behave correctly. Unlike SAP GUI-based tools, Playwright works with **web applications in the browser**. Since SAP Fiori, SAP UI5, and BTP apps all run in the browser, Playwright is a natural fit. ### Why Not ECATT or Tosca? | Concern | ECATT / Tosca | Playwright + Praman | | -------------------- | -------------------------- | --------------------------------------------------- | | SAP GUI support | Native | Not applicable (browser only) | | Fiori / UI5 web apps | Limited or plugin-required | Native + UI5-aware | | Parallel execution | Difficult | Built-in | | CI/CD integration | Complex setup | First-class (GitHub Actions, Azure DevOps, Jenkins) | | Cost | Licensed | Open source | | Version control | Limited | Git-native (tests are code files) | | AI integration | None | Built-in LLM + agentic support | :::tip When to Keep SAP GUI Tools If your tests target **SAP GUI transactions** (not Fiori web apps), tools like ECATT or Tosca remain the right choice. Playwright and Praman target **browser-based** SAP applications only. ::: ## Prerequisite Learning Path If you are starting from zero programming experience, follow this learning path before writing Praman tests. Each step builds on the previous one. ### Step 1: Command Line Basics (1 hour) Learn to navigate directories, run commands, and use a terminal. - **Windows**: Open PowerShell or Windows Terminal - **macOS/Linux**: Open Terminal Key commands to learn: ```bash # Navigate to a folder cd Documents/my-project # List files ls # Run a Node.js script node my-script.js # Run tests npx playwright test ``` ### Step 2: Node.js and npm (1 hour) Install Node.js (version 22 or later) from [nodejs.org](https://nodejs.org). ```bash # Verify installation node --version # Should print v22.x.x or later npm --version # Should print 10.x.x or later # Create a new project mkdir my-sap-tests cd my-sap-tests npm init -y # Install dependencies npm install playwright-praman @playwright/test npx playwright install chromium ``` ### Step 3: TypeScript Basics (2-3 hours) Tests are written in TypeScript, a typed version of JavaScript. You need to understand: - **Variables**: `const name = 'John';` (a value that does not change) - **Strings**: `'Hello'` or `"Hello"` or `` `Hello ${name}` `` - **Objects**: `{ vendor: '100001', amount: 500 }` (a group of named values) - **Arrays**: `['item1', 'item2', 'item3']` (a list of values) - **Functions**: `function greet(name: string) { return 'Hello ' + name; }` - **Arrow functions**: `const greet = (name: string) => 'Hello ' + name;` Free resource: [TypeScript in 5 minutes](https://www.typescriptlang.org/docs/handbook/typescript-in-5-minutes.html) ### Step 4: async/await (30 minutes) Browser operations take time (loading pages, finding elements, clicking buttons). TypeScript uses `async/await` to handle this waiting gracefully. ```typescript // BAD: This does NOT wait for the page to load function badExample() { page.goto('https://my-app.com'); // Returns immediately, page not loaded yet } // GOOD: This waits for the page to finish loading async function goodExample() { await page.goto('https://my-app.com'); // Pauses here until page loads } ``` **Rule of thumb**: Every function that interacts with the browser needs `async` in its declaration and `await` before each browser call. ## Your First Playwright Test Create a file called `tests/hello.test.ts`: ```typescript // tests/hello.test.ts test('my first test', async ({ page }) => { // Go to a web page await page.goto('https://example.com'); // Check the page title await expect(page).toHaveTitle('Example Domain'); }); ``` Run it: ```bash npx playwright test tests/hello.test.ts ``` ### Anatomy of a Test ```typescript // ^^^^ ^^^^^^ // | Tool for checking results ("assertions") // Framework for defining tests test('description of what this test verifies', async ({ page }) => { // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^ // Human-readable test name "page" = a browser tab await page.goto('https://my-app.com'); // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ // Navigate the browser to this URL const heading = page.locator('h1'); // ^^^^^^^ // Find an element on the page await expect(heading).toHaveText('Welcome'); // ^^^^^^ ^^^^^^^^^^ // Assert/verify Expected value }); ``` ## Fixtures: Automatic Setup In ECATT, you might use "variants" to inject test data. In Playwright, **fixtures** provide pre-configured objects to your tests automatically. ```typescript // tests/fixtures-demo.spec.ts test('using fixtures', async ({ ui5, ui5Navigation, sapAuth }) => { // ^^^ ^^^^^^^^^^^^^^ ^^^^^^^ // | | Authentication helper // | Navigation helper (9 methods) // UI5 control finder and interaction API await ui5Navigation.navigateToApp('PurchaseOrder-manage'); const btn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create' } }); await expect(btn).toBeUI5Enabled(); }); ``` You request fixtures by name in the test function signature. Praman provides 13 fixture modules: | Fixture | What It Does | | --------------- | ----------------------------------------------------- | | `ui5` | Find and interact with UI5 controls | | `ui5Navigation` | Navigate FLP apps, tiles, intents | | `sapAuth` | Login/logout with 6 strategies | | `fe` | Fiori Elements List Report + Object Page helpers | | `pramanAI` | AI page discovery and test generation | | `intent` | Business domain intents (procurement, sales, finance) | | `ui5Shell` | FLP shell header (home, user menu) | | `ui5Footer` | Page footer bar (Save, Edit, Delete) | | `flpLocks` | SM12 lock management | | `flpSettings` | User settings reader | | `testData` | Test data generation with auto-cleanup | ## test.step(): Structured Test Steps In ECATT, test scripts have numbered steps. In Playwright, use `test.step()` to create the same structure. ```typescript // tests/purchase-order.spec.ts test('create a purchase order', async ({ ui5, ui5Navigation, fe }) => { await test.step('Step 1: Navigate to the PO app', async () => { await ui5Navigation.navigateToApp('PurchaseOrder-manage'); }); await test.step('Step 2: Click Create', async () => { await ui5.click({ controlType: 'sap.m.Button', properties: { text: 'Create' }, }); }); await test.step('Step 3: Fill the vendor field', async () => { await ui5.fill({ id: 'vendorInput' }, '100001'); }); await test.step('Step 4: Save', async () => { await fe.objectPage.clickSave(); }); await test.step('Step 5: Verify success message', async () => { const messageStrip = await ui5.control({ controlType: 'sap.m.MessageStrip', properties: { type: 'Success' }, }); await expect(messageStrip).toBeUI5Visible(); }); }); ``` Steps appear as collapsible sections in the Playwright HTML report. ## Assertions: Checking Results Assertions verify that the application behaves correctly. If an assertion fails, the test fails. ### Standard Playwright Assertions ```typescript // Check page title await expect(page).toHaveTitle('Purchase Orders'); // Check URL await expect(page).toHaveURL(/PurchaseOrder/); // Check text content await expect(page.locator('#message')).toHaveText('Saved'); ``` ### Praman UI5 Assertions ```typescript const button = await ui5.control({ id: 'saveBtn' }); // Is the button enabled? await expect(button).toBeUI5Enabled(); // Does it have the right text? await expect(button).toHaveUI5Text('Save'); // Is it visible? await expect(button).toBeUI5Visible(); // Check a specific property await expect(button).toHaveUI5Property('type', 'Emphasized'); // Check value state (for inputs) const input = await ui5.control({ id: 'amountInput' }); await expect(input).toHaveUI5ValueState('Error'); ``` ## Running Tests ### Basic Commands ```bash # Run all tests npx playwright test # Run a specific test file npx playwright test tests/purchase-order.test.ts # Run tests matching a name pattern npx playwright test --grep "purchase order" # Run with visible browser (headed mode) npx playwright test --headed # Run in debug mode (step through interactively) npx playwright test --debug ``` ### Viewing Reports ```bash # Open the HTML report after a test run npx playwright show-report ``` The HTML report shows: - Pass/fail status for each test - `test.step()` sections collapsed per step - Screenshots on failure (automatic) - Traces on retry (automatic) ## Project Structure A typical Praman test project looks like this: ```text my-sap-tests/ tests/ auth-setup.ts # Login once, save session auth-teardown.ts # Logout purchase-order.test.ts # Test file vendor.test.ts # Another test file .auth/ sap-session.json # Saved browser session (auto-generated, git-ignored) .env # Environment variables (git-ignored) praman.config.ts # Praman configuration playwright.config.ts # Playwright configuration package.json # Dependencies tsconfig.json # TypeScript configuration ``` ## Common Patterns ### Finding a UI5 Control ```typescript // By ID const btn = await ui5.control({ id: 'saveBtn' }); // By type + property const input = await ui5.control({ controlType: 'sap.m.Input', properties: { placeholder: 'Enter vendor' }, }); // By binding path const field = await ui5.control({ controlType: 'sap.m.Input', bindingPath: { value: '/PurchaseOrder/Vendor' }, }); ``` ### Interacting with Controls ```typescript // Click await ui5.click({ id: 'submitBtn' }); // Type text await ui5.fill({ id: 'nameInput' }, 'John Doe'); // Select from dropdown await ui5.select({ id: 'countrySelect' }, 'US'); // Toggle checkbox await ui5.check({ id: 'agreeCheckbox' }); // Clear a field await ui5.clear({ id: 'searchField' }); ``` ### Navigating SAP Apps ```typescript // Navigate to an app by semantic object and action await ui5Navigation.navigateToApp('PurchaseOrder-manage'); // Navigate to a tile by title await ui5Navigation.navigateToTile('Create Purchase Order'); // Navigate with parameters await ui5Navigation.navigateToIntent( { semanticObject: 'PurchaseOrder', action: 'create' }, { plant: '1000' }, ); // Go home await ui5Navigation.navigateToHome(); ``` ## Glossary for SAP Testers | SAP Term | Playwright/Praman Equivalent | | --------------------------- | -------------------------------------------------- | | Test script | Test file (`.test.ts`) | | Test step / script line | `test.step()` | | Variant / test data set | `testData.generate()` fixture | | Check / verification | `expect()` assertion | | Test configuration | `playwright.config.ts` + `praman.config.ts` | | Transaction code | `ui5Navigation.navigateToIntent()` | | Control ID | `UI5Selector.id` | | Element repository | UI5 control registry (queried via `ui5.control()`) | | Test run | `npx playwright test` | | Test report | `npx playwright show-report` (HTML) | | Reusable component / module | Playwright fixture | | Test suite | `describe()` block or test file | :::warning[Common mistake] Do not forget `await` before browser calls. Every function that interacts with the browser (clicking, filling, navigating) returns a Promise. Missing `await` causes the operation to fire without waiting for completion, leading to flaky tests and confusing errors. ```typescript // Wrong — missing await ui5.press({ id: 'saveBtn' }); // Returns immediately, button may not be clicked // Correct — await pauses until the action completes await ui5.press({ id: 'saveBtn' }); ``` ::: ## FAQ
Do I need to learn TypeScript before using Praman? You need basic TypeScript knowledge — variables, strings, objects, and `async/await`. The [Step 3: TypeScript Basics](#step-3-typescript-basics-2-3-hours) section above covers what you need in 2-3 hours. You do not need advanced TypeScript features like generics, decorators, or type guards. If you use AI agents to generate tests, you need even less TypeScript knowledge — the agents write the code for you. You only need to understand enough to review and modify generated tests.
What is the difference between page.locator() and ui5.control()? `page.locator()` is Playwright's built-in method that finds elements by CSS selectors, text, or test IDs in the DOM. It works for any web application. `ui5.control()` is Praman's method that finds controls through the **UI5 runtime control registry**. It uses control types (`sap.m.Button`), properties (`{ text: 'Save' }`), and binding paths — not DOM selectors. This is more stable because UI5 control IDs and DOM structure change between versions, but the control registry API is the stable contract. **Rule**: Use `ui5.control()` for all UI5 controls. Use `page.locator()` only for non-UI5 elements (plain HTML, FLP shell tabs, iframes).
How do I run just one test instead of all tests? Use the `--grep` flag or specify the test file directly: ```bash # Run a specific file npx playwright test tests/purchase-order.test.ts # Run tests matching a name pattern npx playwright test --grep "create purchase order" # Run in debug mode with visible browser npx playwright test tests/purchase-order.test.ts --headed --debug ```
What happens if my test fails? Playwright automatically captures screenshots on failure and generates an HTML report. View it: ```bash npx playwright show-report ``` The report shows the exact step that failed, a screenshot of the browser state, and the error message. If you use `test.step()`, the report shows which step failed within the test. For SAP-specific failures (`ERR_CONTROL_NOT_FOUND`, `ERR_BRIDGE_TIMEOUT`), Praman errors include `suggestions[]` with actionable fixes.
:::tip[Next steps] - **[Getting Started →](./getting-started)** — Install Praman and generate your first SAP test with AI agents - **[Selectors Guide →](./selectors)** — Deep dive into `ui5.control()` selector syntax for SAP controls - **[Authentication Guide →](./authentication)** — Configure SAP login for your test environment ::: --- ## Running Your Agent for the First Time :::info[In this guide] - Launch the planner agent to discover UI5 controls on a live SAP system - Review the structured test plan output with control maps and test scenarios - Generate a production-ready Playwright test script from the plan - Use the healer agent to fix failing tests iteratively - Produce a gold-standard `.spec.ts` ready for CI/CD ::: From business process to Playwright test — autonomously. This guide walks you through the complete flow of using Praman's AI agents to discover an SAP application, generate a test plan, produce a gold-standard test script, and iterate until it passes. ## Prerequisites Checklist :::warning Playwright 1.59+ MCP Server If using MCP-based agents, install `@playwright/mcp` separately — it is no longer bundled with Playwright 1.59+. ```bash npm install @playwright/mcp ``` CLI agents (files with `-cli` suffix) do not need this package. ::: Before launching your agent, verify the following files exist in your project. All paths are relative to your project root. ### Agent Definitions ```text .github/agents/praman-sap-planner.agent.md .github/agents/praman-sap-generator.agent.md .github/agents/praman-sap-healer.agent.md ``` :::warning Missing agents? If these files don't exist, run `npx playwright-praman init` (see [Agent & IDE Setup](./agent-setup)). ::: ### Skill Files ```text .github/skills/sap-test-automation/SKILL.md .github/skills/sap-test-automation/ai-quick-reference.md ``` These are read by all three agents. They ship inside the `playwright-praman` package and are copied by `init`. ### Copilot Instructions Verify that `.github/copilot-instructions.md` contains the Praman section. If not, append it: ```bash cat node_modules/playwright-praman/docs/user-integration/copilot-instructions-appendable.md >> .github/copilot-instructions.md ``` ### Seed File ```text tests/seeds/sap-seed.spec.ts ``` The seed authenticates against your SAP system and keeps the browser open for the MCP server. It uses raw Playwright (no Praman fixtures). ### Environment Variables Create a `.env` file with your SAP credentials: ```bash SAP_CLOUD_BASE_URL=https://your-system.s4hana.cloud.sap/ SAP_CLOUD_USERNAME=your-sap-user SAP_CLOUD_PASSWORD=your-sap-password ``` :::danger Never commit `.env` to version control. Add it to `.gitignore`. ::: ## The Complete Flow ```text ┌─────────────────────────────────────────────────────────┐ │ 1. PLAN → 2. GENERATE → 3. HEAL → 4. GOLD │ │ │ │ Planner Generator Healer Production │ │ explores converts fixes test ready │ │ live SAP plan to failing for CI/CD │ │ app test code tests │ └─────────────────────────────────────────────────────────┘ ``` ## Step 1: Launch the Planner Agent Open **GitHub Copilot MCP Chat** (or Claude Code) and paste the following prompt. Replace the test case steps with your own business scenario. ### Prompt Template ```text Goal: Create SAP test case and test script 1. Use praman SAP planner agent: .github/agents/praman-sap-planner.agent.md 2. Login using credentials in .env file and use Chrome in headed mode. Do not use sub-agents. 3. Ensure you use UI5 query and capture UI5 methods at each step. Use UI5 methods for all control interactions. 4. Use seed file: tests/seeds/sap-seed.spec.ts 5. Here is the test case: Login to SAP and ensure you are on the landing page. Step 1: Navigate to Maintain Bill of Material app and click Create BOM - expect: Create BOM dialog opens with all fields visible Step 2: Select Material via Value Help — pick a valid material - expect: Material field is populated with selected material Step 3: Select Plant via Value Help — pick plant matching the material - expect: Plant field is populated with selected plant Step 4: Select BOM Usage "Production (1)" from dropdown - expect: BOM Usage field shows "Production (1)" Step 5: Verify all required fields are filled before submission - expect: Material field has a value - expect: BOM Usage field has value "Production (1)" - expect: Valid From date is set - expect: Create BOM button is enabled Step 6: Click "Create BOM" submit button in dialog footer - expect: If valid combination — dialog closes, BOM created, user returns to list report - expect: If invalid combination — error message dialog appears Step 7: If error occurs, close error dialog and cancel - expect: Error dialog closes - expect: Create BOM dialog closes - expect: User returns to list report Output: - Test plan: specs/ - Test script: tests/e2e/sap-cloud/ ``` ### What Happens Next The agent will: 1. **Launch the MCP server** and open Chrome in headed mode 2. **Authenticate** using credentials from `.env` via the seed file 3. **Navigate** to the SAP Fiori Launchpad and open your target app 4. **Discover UI5 controls** — runs `browser_run_code` to query the UI5 control tree, capturing: - Control IDs, types, and properties - Value Help structures (inner tables, columns, sample data) - MDC Field vs SmartField framework detection - Required fields, button states, dialog structure 5. **Produce a test plan** in `specs/` with: - Application overview (UI5 version, OData version, control framework) - Full UI5 control map (every discovered ID, type, and value help) - Test scenarios with step-by-step UI5 method references 6. **Generate the test script** in `tests/e2e/sap-cloud/` ## Step 2: Review the Test Plan Output The planner produces a structured test plan. Here is what a typical plan looks like:
Example: bom-create-flow.plan.md (click to expand) ```markdown # BOM Create Complete Flow — Test Plan ## Application Overview - **App**: SAP S/4HANA Cloud - Maintain Bill of Material (Version 2) - **Type**: Fiori Elements V4 List Report with Create BOM Action Parameter Dialog - **UI5 Version**: 1.142.x - **OData Version**: V4 - **Control Framework**: MDC (sap.ui.mdc) — NOT SmartField (sap.ui.comp) ## Discovery Date [Auto-filled] — Live discovery via Playwright MCP + browser_run_code ## UI5 Control Map (Discovered) ### List Report — Toolbar | Control | Type | Text | | ---------- | ------------ | ---------- | | Create BOM | sap.m.Button | Create BOM | | Go | sap.m.Button | Go | ### Create BOM Dialog (sap.m.Dialog) | Control | Type | Required | | ----------------- | --------------------------- | -------- | | Material (outer) | sap.ui.mdc.Field | true | | Material (inner) | sap.ui.mdc.field.FieldInput | true | | Plant (outer) | sap.ui.mdc.Field | false | | Plant (inner) | sap.ui.mdc.field.FieldInput | false | | BOM Usage (outer) | sap.ui.mdc.Field | true | | BOM Usage (inner) | sap.ui.mdc.field.FieldInput | true | | Valid From | sap.m.DatePicker | false | | Create BOM (btn) | sap.m.Button | — | | Cancel (btn) | sap.m.Button | — | ### Value Help: Material | Property | Value | | ----------- | ------------------------------ | | VH Type | sap.ui.mdc.ValueHelp | | Inner Table | sap.ui.table.Table | | Columns | Material, Material Description | ### Value Help: Plant | Property | Value | | ----------- | --------------------------------- | | VH Type | sap.ui.mdc.ValueHelp | | Inner Table | sap.ui.table.Table | | Columns | Plant, Plant Name, Valuation Area | ### Value Help: BOM Usage | Property | Value | | -------- | --------------------------------------------------------------------------------------- | | Type | sap.ui.mdc.ValueHelp — Suggest popover (dropdown) | | Options | Production (1), Engineering/Design (2), Universal (3), Plant Maintenance (4), Sales (5) | ## UI5 Methods Used (Praman Proxy) | Method | Used For | | ---------------------------- | -------------------------------------------- | | ui5.control({ id }) | Find MDC Field, Button, Dialog by stable ID | | ui5.press({ id }) | Click buttons (Create BOM, VH icons, Cancel) | | ui5.fill({ id }, value) | Set field value (setValue + fireChange) | | ui5.waitForUI5() | Wait for UI5 busyIndicator / OData calls | | ui5.getValue({ id }) | Read MDC FieldInput display value | | control.setValue(key) | Set MDC Field key programmatically | | control.getProperty(prop) | Read text, enabled, title | | control.getRequired() | Check required fields | | control.isOpen() | Check ValueHelp/Dialog open state | | control.close() | Close ValueHelp dialog | | innerTable.getContextByIndex | Get OData binding from VH table row | | ctx.getObject() | Extract entity data from binding context | ## Test Scenarios ### Scenario 1: Complete BOM Creation Flow **Steps:** 1. Navigate to BOM App (FLP → tab → tile) 2. Open Create BOM Dialog — verify all fields 3. Select Material via Value Help 4. Select Plant via Value Help 5. Set BOM Usage to Production (1) 6. Verify all required fields 7. Submit Create BOM (handle success or error) 8. Verify return to List Report ### Scenario 2: Validation Error Handling **Steps:** 1. Navigate & Open Dialog 2. Select an invalid Material + Plant combination 3. Set BOM Usage 4. Submit — verify error dialog appears 5. Close error dialog — verify Create BOM dialog preserved 6. Cancel — verify return to List Report ```
:::tip What to look for Review the **UI5 Control Map** — it should list every control the planner discovered with correct IDs and types. If a control is missing, re-run the planner with a more specific test case description. ::: ## Step 3: Review the Generated Test Script The generator produces a test script using `playwright-praman` fixtures. The output follows the gold standard format:
Example: bom-create-flow-gold.spec.ts structure (click to expand) ```typescript // tests/e2e/sap-cloud/bom-create-flow-gold.spec.ts // ── Control ID Constants (from discovery) ─────────────────────── const SRVD = 'com.sap.gateway.srvd.your_service.v0001'; const APP = 'your.app.namespace::YourListReport'; const IDS = { createBOMToolbarBtn: `${APP}--fe::table::...`, dialog: `fe::APD_::${SRVD}.CreateBOM`, dialogOkBtn: `fe::APD_::${SRVD}.CreateBOM::Action::Ok`, dialogCancelBtn: `fe::APD_::${SRVD}.CreateBOM::Action::Cancel`, materialField: 'APD_::Material', materialInner: 'APD_::Material-inner', materialVHIcon: 'APD_::Material-inner-vhi', // ... all discovered IDs } as const; test.describe('BOM Create Complete Flow', () => { test('Complete BOM Create — single session', async ({ page, ui5 }) => { // Step 1: Navigate to BOM App await test.step('Step 1: Navigate to BOM Maintenance App', async () => { await page.goto(process.env.SAP_CLOUD_BASE_URL!); // ... FLP navigation using ui5.press() for tiles }); // Step 2: Open Dialog and Verify Fields await test.step('Step 2: Open Create BOM Dialog', async () => { await ui5.press({ id: IDS.createBOMToolbarBtn }); const materialField = await ui5.control({ id: IDS.materialField }); expect(await materialField.getRequired()).toBe(true); // ... verify all fields }); // Step 3-5: Fill form via Value Helps and proxy methods // Step 6: Verify all required fields // Step 7: Submit and handle error recovery // Step 8: Verify return to List Report }); }); ```
**Key characteristics of generated test scripts:** | Aspect | Pattern | | ----------------- | ---------------------------------------------------------------------- | | Import | `import { test, expect } from 'playwright-praman'` only | | Control IDs | Constants extracted from live discovery | | UI5 interactions | 100% Praman fixtures (`ui5.control()`, `ui5.press()`, `ui5.fill()`) | | Playwright native | Only for FLP tab navigation (`page.getByText()`) and `page.goto()` | | Value Help | `innerTable.getContextByIndex(n).getObject()` for OData binding | | Error recovery | Graceful `try/catch` — close error dialogs, cancel, verify List Report | | Annotations | `test.info().annotations.push()` for rich HTML report output | ## Step 4: Convert to Gold Standard Format The initial generated script may not be in full Praman gold standard format. To upgrade it, attach the script in your MCP chat and prompt: ```text Convert the attached test script to Praman gold standard format. ``` The agent will: - Add the Apache-2.0 license header - Add the TSDoc compliance report block - Ensure 100% Praman fixture usage for UI5 elements - Add `test.step()` wrappers with descriptive names - Add `test.info().annotations` for rich reporting - Ensure `searchOpenDialogs: true` on all dialog controls - Apply `toPass()` retry patterns for slow SAP systems ## Step 5: Debug and Heal The generated test may not pass on the first run. SAP systems have varying response times, async OData loads, and environment-specific quirks. ### Iterative Healing Process ```text ┌──────────────────────────────────────────────┐ │ Run test → Fails → Agent fixes → Re-run │ │ ↑ │ │ │ └──────────────────────────────┘ │ │ │ │ Typically 3-6 iterations depending on: │ │ - SAP system response time │ │ - OData data availability │ │ - Control rendering timing │ └──────────────────────────────────────────────┘ ``` Run the test in debug mode: ```bash npx playwright test tests/e2e/sap-cloud/your-test.spec.ts --headed --debug ``` Or use the healer agent directly: ```text Goal: Fix this failing test Use praman SAP healer agent: .github/agents/praman-sap-healer.agent.md Test file: tests/e2e/sap-cloud/your-test.spec.ts Error: [paste the error output] ``` ### Common Fixes the Healer Applies | Issue | Fix | | --------------------------- | --------------------------------------------------------- | | Timeout waiting for control | Add `toPass()` retry with progressive intervals | | Value Help not open yet | Poll `control.isOpen()` before interacting | | OData data not loaded | `getContextByIndex()` with retry loop | | FLP tab not switching | Use `page.getByText()` DOM click instead of `ui5.press()` | | Dialog control not found | Add `searchOpenDialogs: true` | | Button not enabled | Wait for `ui5.waitForUI5()` after previous action | ## Step 6: Production-Ready Test After the heal cycle completes, you have a production-ready gold standard test with: - 100% Praman fixture usage for all UI5 elements - Graceful error recovery (no hard failures on validation errors) - Rich `test.info().annotations` for HTML report output - `toPass()` retry patterns tuned to your SAP system timing - Apache-2.0 license header and TSDoc compliance block ### Full Gold Standard Example The script below is a complete, ready-to-run gold standard test for the **Maintain Bill of Material** app. Copy it, update the control IDs to match your system's live discovery output, and set `SAP_CLOUD_BASE_URL` in your `.env`.
bom-e2e-gold-standard.spec.ts — full source (click to expand) ```typescript /** * @license Apache-2.0 * * ═══════════════════════════════════════════════════════════════ * GOLD STANDARD — BOM Complete End-to-End Flow (V1 SmartField) * ═══════════════════════════════════════════════════════════════ * * App: SAP S/4HANA Cloud — Maintain Bill of Material * Type: Fiori Elements V2 List Report with Action Parameter Dialog * UI5: 1.142.x * OData: V2 — SmartField controls (sap.ui.comp.smartfield.SmartField) * * Steps: * 1. Navigate to BOM app (FLP → space tab → tile) * 2. Open Create BOM dialog and verify structure * 3. Test Material value help (SmartTable → rows → binding context) * 4. Test Plant value help * 5. Test BOM Usage inner ComboBox (open/close/items) * 6. Fill form with valid data from live value helps * 7. Click Create button — handle success or validation error gracefully * 8. Verify return to BOM List Report * * ═══════════════════════════════════════════════════════════════ * PRAMAN COMPLIANCE REPORT * ═══════════════════════════════════════════════════════════════ * * Controls Discovered: 15+ * UI5 Elements via Praman fixtures: 100% * Playwright native (non-UI5 only): page.goto, waitForLoadState, * getByText (FLP space tab DOM click), toHaveTitle * * Fixtures used: * ui5.control (10), ui5.press (6), ui5.fill (2), * ui5.getValue (3), ui5.waitForUI5 (15) * * Control proxy methods: * getControlType, getProperty, getRequired, getEnabled, getVisible, * isOpen, close, getTable, getRows, getBindingContext, * getContextByIndex, getObject, getItems, getKey, getText, * open, setSelectedKey, getSelectedKey, fireChange * * Auth method: storageState — credentials from .env via seed file * Forbidden pattern scan: PASSED * COMPLIANCE: PASSED — 100% Praman/UI5 methods for all UI5 elements * * SAP Best Practices: * - Data-driven: getContextByIndex().getObject() for OData binding * - Control-based: setSelectedKey() for ComboBox (no text matching) * - Localization-safe: avoids text selectors for controls * - setValue() + fireChange() via ui5.fill() for proper event propagation * ═══════════════════════════════════════════════════════════════ */ // ── Control ID Constants (update after live discovery on your system) ── const IDS = { // Dialog fields — sap.ui.comp.smartfield.SmartField materialField: 'createBOMFragment--material', materialInput: 'createBOMFragment--material-input', materialVHIcon: 'createBOMFragment--material-input-vhi', materialVHDialog: 'createBOMFragment--material-input-valueHelpDialog', materialVHTable: 'createBOMFragment--material-input-valueHelpDialog-table', plantField: 'createBOMFragment--plant', plantInput: 'createBOMFragment--plant-input', plantVHIcon: 'createBOMFragment--plant-input-vhi', plantVHDialog: 'createBOMFragment--plant-input-valueHelpDialog', plantVHTable: 'createBOMFragment--plant-input-valueHelpDialog-table', bomUsageField: 'createBOMFragment--variantUsage', bomUsageCombo: 'createBOMFragment--variantUsage-comboBoxEdit', // Dialog buttons okBtn: 'createBOMFragment--OkBtn', cancelBtn: 'createBOMFragment--CancelBtn', } as const; // ── Test Data ───────────────────────────────────────────────────────── const TEST_DATA = { flpSpaceTab: 'Bills Of Material', // FLP space tab label tileHeader: 'Maintain Bill Of Material', bomUsageKey: '1', // '1' = Production } as const; // ───────────────────────────────────────────────────────────────────── test.describe('BOM End-to-End Flow', () => { test('Complete BOM Flow — V2 SmartField single session', async ({ page, ui5 }) => { // ═══════════════════════════════════════════════════════════════ // STEP 1: Navigate to BOM Maintenance App // ═══════════════════════════════════════════════════════════════ await test.step('Step 1: Navigate to BOM Maintenance App', async () => { // FLP home page has async widgets that keep UI5 busy and never reach // networkidle. Use domcontentloaded + title assertion only. await page.goto(process.env.SAP_CLOUD_BASE_URL!); await page.waitForLoadState('domcontentloaded'); // Verify FLP home loaded (Playwright web-first assertion — auto-retries) await expect(page).toHaveTitle(/Home/, { timeout: 60000 }); // Navigate to Bills Of Material space tab. // FLP tabs use sap.m.IconTabFilter — firePress() does not trigger tab // switching. A DOM click is the only reliable method here. await page.getByText(TEST_DATA.flpSpaceTab, { exact: true }).click(); // Click Maintain Bill Of Material tile. // Wrapped in toPass() because ui5.press() calls waitForUI5() internally, // which can time out on slow systems where FLP background OData requests // are still active. await expect(async () => { await ui5.press({ controlType: 'sap.m.GenericTile', properties: { header: TEST_DATA.tileHeader }, }); }).toPass({ timeout: 60000, intervals: [5000, 10000] }); // Wait for Create BOM button — proves the V2 app has fully loaded. let createBtn: Awaited>; await expect(async () => { createBtn = await ui5.control({ controlType: 'sap.m.Button', properties: { text: 'Create BOM' }, }); const text = await createBtn.getProperty('text'); expect(text).toBe('Create BOM'); }).toPass({ timeout: 120000, intervals: [5000, 10000] }); test.info().annotations.push({ type: 'info', description: 'V2 List Report loaded — Create BOM button visible', }); }); // ═══════════════════════════════════════════════════════════════ // STEP 2: Open Create BOM Dialog and Verify Structure // ═══════════════════════════════════════════════════════════════ await test.step('Step 2: Open Create BOM Dialog', async () => { await ui5.press({ controlType: 'sap.m.Button', properties: { text: 'Create BOM' }, }); await ui5.waitForUI5(); // Verify Material SmartField — type and required flag const materialField = await ui5.control({ id: IDS.materialField, searchOpenDialogs: true, }); expect(await materialField.getControlType()).toBe('sap.ui.comp.smartfield.SmartField'); expect(await materialField.getRequired()).toBe(true); // Verify BOM Usage SmartField — type and required flag const bomUsageField = await ui5.control({ id: IDS.bomUsageField, searchOpenDialogs: true, }); expect(await bomUsageField.getControlType()).toBe('sap.ui.comp.smartfield.SmartField'); expect(await bomUsageField.getRequired()).toBe(true); // Verify footer buttons exist and are interactive const createDialogBtn = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true }); const cancelDialogBtn = await ui5.control({ id: IDS.cancelBtn, searchOpenDialogs: true }); expect(await createDialogBtn.getProperty('text')).toBe('Create'); expect(await cancelDialogBtn.getProperty('text')).toBe('Cancel'); expect(await cancelDialogBtn.getEnabled()).toBe(true); test.info().annotations.push({ type: 'info', description: 'Dialog structure verified: required fields and footer buttons present', }); }); // ═══════════════════════════════════════════════════════════════ // STEP 3: Test Material Value Help // ═══════════════════════════════════════════════════════════════ await test.step('Step 3: Test Material Value Help', async () => { await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true }); const materialDialog = await ui5.control({ id: IDS.materialVHDialog, searchOpenDialogs: true, }); expect(await materialDialog.isOpen()).toBe(true); await ui5.waitForUI5(); // SmartTable.getTable() → innerTable — then getRows() for binding check const smartTable = await ui5.control({ id: IDS.materialVHTable, searchOpenDialogs: true }); const innerTable = await smartTable.getTable(); const rows = (await innerTable.getRows()) as unknown[]; expect(rows.length).toBeGreaterThan(0); // Wait for OData data to load using Playwright auto-retry let rowCount = 0; await expect(async () => { rowCount = 0; for (const row of rows) { const ctx = await ( row as { getBindingContext: () => Promise } ).getBindingContext(); if (ctx) rowCount++; } expect(rowCount).toBeGreaterThan(0); }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); test .info() .annotations.push({ type: 'info', description: `Material VH: ${rowCount} rows loaded` }); await materialDialog.close(); await ui5.waitForUI5(); }); // ═══════════════════════════════════════════════════════════════ // STEP 4: Test Plant Value Help // ═══════════════════════════════════════════════════════════════ await test.step('Step 4: Test Plant Value Help', async () => { await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true }); const plantDialog = await ui5.control({ id: IDS.plantVHDialog, searchOpenDialogs: true, }); expect(await plantDialog.isOpen()).toBe(true); await ui5.waitForUI5(); const smartTablePlant = await ui5.control({ id: IDS.plantVHTable, searchOpenDialogs: true }); const innerTablePlant = await smartTablePlant.getTable(); const plantRows = (await innerTablePlant.getRows()) as unknown[]; let plantRowCount = 0; await expect(async () => { plantRowCount = 0; for (const row of plantRows) { const ctx = await ( row as { getBindingContext: () => Promise } ).getBindingContext(); if (ctx) plantRowCount++; } expect(plantRowCount).toBeGreaterThan(0); }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); test .info() .annotations.push({ type: 'info', description: `Plant VH: ${plantRowCount} rows loaded` }); await plantDialog.close(); await ui5.waitForUI5(); }); // ═══════════════════════════════════════════════════════════════ // STEP 5: Test BOM Usage Dropdown (Inner ComboBox) // ═══════════════════════════════════════════════════════════════ await test.step('Step 5: Test BOM Usage Dropdown', async () => { const bomUsageCombo = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true, }); // Enumerate items via proxy const rawItems = await bomUsageCombo.getItems(); const items: Array<{ key: string; text: string }> = []; if (Array.isArray(rawItems)) { for (const itemProxy of rawItems) { items.push({ key: String(await itemProxy.getKey()), text: String(await itemProxy.getText()), }); } } expect(items.length).toBeGreaterThan(0); test.info().annotations.push({ type: 'info', description: `BOM Usage items: ${items.map((i) => `${i.key}: ${i.text}`).join(', ')}`, }); // Verify open/close state management await bomUsageCombo.open(); await ui5.waitForUI5(); expect(await bomUsageCombo.isOpen()).toBe(true); await bomUsageCombo.close(); await ui5.waitForUI5(); expect(await bomUsageCombo.isOpen()).toBe(false); }); // ═══════════════════════════════════════════════════════════════ // STEP 6: Fill Form with Valid Data // ═══════════════════════════════════════════════════════════════ await test.step('Step 6: Fill Form with Valid Data', async () => { // ── Fill Material ────────────────────────────────────────── await ui5.press({ id: IDS.materialVHIcon, searchOpenDialogs: true }); const materialDialogCtrl = await ui5.control({ id: IDS.materialVHDialog, searchOpenDialogs: true, }); await expect(async () => { expect(await materialDialogCtrl.isOpen()).toBe(true); }).toPass({ timeout: 30000, intervals: [1000, 2000] }); const smartTableMat = await ui5.control({ id: IDS.materialVHTable, searchOpenDialogs: true }); const innerTableMat = await smartTableMat.getTable(); // Read first available material from OData binding let materialValue = ''; await expect(async () => { const ctx = await innerTableMat.getContextByIndex(0); expect(ctx).toBeTruthy(); const obj = (await ctx.getObject()) as { Material?: string }; expect(obj?.Material).toBeTruthy(); materialValue = obj!.Material!; }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); await materialDialogCtrl.close(); await ui5.waitForUI5(); // ui5.fill() = atomic setValue + fireChange + waitForUI5 await ui5.fill({ id: IDS.materialInput, searchOpenDialogs: true }, materialValue); await ui5.waitForUI5(); test.info().annotations.push({ type: 'info', description: `Material: ${materialValue}` }); // ── Fill Plant ───────────────────────────────────────────── await ui5.press({ id: IDS.plantVHIcon, searchOpenDialogs: true }); const plantDialogCtrl = await ui5.control({ id: IDS.plantVHDialog, searchOpenDialogs: true }); await expect(async () => { expect(await plantDialogCtrl.isOpen()).toBe(true); }).toPass({ timeout: 30000, intervals: [1000, 2000] }); const smartTablePlant = await ui5.control({ id: IDS.plantVHTable, searchOpenDialogs: true }); const innerTablePlant = await smartTablePlant.getTable(); let plantValue = ''; await expect(async () => { const ctx = await innerTablePlant.getContextByIndex(0); expect(ctx).toBeTruthy(); const obj = (await ctx.getObject()) as { Plant?: string }; expect(obj?.Plant).toBeTruthy(); plantValue = obj!.Plant!; }).toPass({ timeout: 60000, intervals: [1000, 2000, 5000] }); await plantDialogCtrl.close(); await ui5.waitForUI5(); await ui5.fill({ id: IDS.plantInput, searchOpenDialogs: true }, plantValue); await ui5.waitForUI5(); test.info().annotations.push({ type: 'info', description: `Plant: ${plantValue}` }); // ── Fill BOM Usage ───────────────────────────────────────── const bomUsageCtrl = await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true }); await bomUsageCtrl.open(); await ui5.waitForUI5(); await bomUsageCtrl.setSelectedKey(TEST_DATA.bomUsageKey); await bomUsageCtrl.fireChange({ value: TEST_DATA.bomUsageKey }); await bomUsageCtrl.close(); await ui5.waitForUI5(); test.info().annotations.push({ type: 'info', description: `BOM Usage: ${TEST_DATA.bomUsageKey} (Production)`, }); // ── Pre-submission verification ──────────────────────────── const finalMaterial = (await ui5.getValue({ id: IDS.materialInput, searchOpenDialogs: true })) ?? ''; const finalPlant = (await ui5.getValue({ id: IDS.plantInput, searchOpenDialogs: true })) ?? ''; const finalBomUsage = (await ( await ui5.control({ id: IDS.bomUsageCombo, searchOpenDialogs: true }) ).getSelectedKey()) ?? ''; expect(finalMaterial).toBe(materialValue); expect(finalPlant).toBe(plantValue); expect(finalBomUsage).toBe(TEST_DATA.bomUsageKey); test.info().annotations.push({ type: 'info', description: [ 'Pre-submission verification:', ` Material: ${finalMaterial}`, ` Plant: ${finalPlant}`, ` BOM Usage: ${finalBomUsage}`, ].join('\n'), }); }); // ═══════════════════════════════════════════════════════════════ // STEP 7: Click Create Button and Handle Result Gracefully // ═══════════════════════════════════════════════════════════════ await test.step('Step 7: Click Create Button and handle result', async () => { const createBtn = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true }); expect(await createBtn.getProperty('text')).toBe('Create'); expect(await createBtn.getEnabled()).toBe(true); await ui5.press({ id: IDS.okBtn, searchOpenDialogs: true }); await ui5.waitForUI5(); // ── Outcome A: dialog closes → BOM created → success ─────── // ── Outcome B: validation error → dialog stays open ───────── let dialogStillOpen = false; try { const okBtnCheck = await ui5.control({ id: IDS.okBtn, searchOpenDialogs: true }); dialogStillOpen = (await okBtnCheck.getEnabled()) !== undefined; } catch { dialogStillOpen = false; } // Detect error popover or MessageBox let hasErrorDialog = false; try { const popover = await ui5 .control({ controlType: 'sap.m.MessagePopover', searchOpenDialogs: true }) .catch(() => null); if (popover) hasErrorDialog = !!(await popover.isOpen().catch(() => false)); if (!hasErrorDialog) { const msgBox = await ui5 .control({ controlType: 'sap.m.Dialog', properties: { type: 'Message' }, searchOpenDialogs: true, }) .catch(() => null); if (msgBox) { const title = await msgBox.getProperty('title').catch(() => ''); hasErrorDialog = typeof title === 'string' && title.length > 0; } } } catch { /* no error dialogs — success path */ } // Graceful cleanup: close error dialogs, then cancel Create BOM dialog if (hasErrorDialog) { try { await ui5.press({ controlType: 'sap.m.Button', properties: { text: 'Close' }, searchOpenDialogs: true, }); await ui5.waitForUI5(); } catch { /* Close button not present */ } } if (dialogStillOpen) { try { await ui5.press({ id: IDS.cancelBtn, searchOpenDialogs: true }); await ui5.waitForUI5(); } catch { /* dialog already closed */ } } test.info().annotations.push({ type: 'info', description: `Create result: dialogOpen=${dialogStillOpen}, errorDialog=${hasErrorDialog}`, }); }); // ═══════════════════════════════════════════════════════════════ // STEP 8: Verify Return to BOM List Report // ═══════════════════════════════════════════════════════════════ await test.step('Step 8: Verify return to BOM List Report', async () => { // Create BOM button visible + enabled proves we are back on the List Report const listBtn = await ui5.control( { controlType: 'sap.m.Button', properties: { text: 'Create BOM' } }, { timeout: 30000 }, ); expect(await listBtn.getProperty('text')).toBe('Create BOM'); expect(await listBtn.getProperty('enabled')).toBe(true); test.info().annotations.push({ type: 'info', description: 'Returned to BOM List Report — gold standard test complete', }); }); }); }); ```
### Key Patterns to Copy | Pattern | Code | | ---------------------- | ------------------------------------------------------------------ | | Env-based URL | `await page.goto(process.env.SAP_CLOUD_BASE_URL!)` | | FLP tab (DOM click) | `await page.getByText('Tab Name', { exact: true }).click()` | | Retry tile press | `await expect(async () => { await ui5.press({...}) }).toPass(...)` | | Dialog control | `await ui5.control({ id: '...', searchOpenDialogs: true })` | | OData binding read | `(await innerTable.getContextByIndex(0).getObject())` | | Atomic fill | `await ui5.fill({ id: '...', searchOpenDialogs: true }, value)` | | ComboBox select | `await ctrl.setSelectedKey('1'); await ctrl.fireChange(...)` | | Graceful error cleanup | `try { await ui5.press({...Close...}) } catch { }` | ### Run in CI ```bash npx playwright test tests/e2e/sap-cloud/ --project=chromium ``` ### Expected Output ```text Running 1 test using 1 worker ✓ tests/e2e/sap-cloud/bom-e2e-gold-standard.spec.ts BOM End-to-End Flow Complete BOM Flow — V2 SmartField single session (45s) Step 1: Navigate to BOM Maintenance App Step 2: Open Create BOM Dialog Step 3: Test Material Value Help Step 4: Test Plant Value Help Step 5: Test BOM Usage Dropdown Step 6: Fill Form with Valid Data Step 7: Click Create Button and handle result Step 8: Verify return to BOM List Report 1 passed (48s) ``` ## Quick Reference: Agent Prompts | Task | Prompt | | ------------------ | ------------------------------------------------------------------ | | Plan a new test | `Goal: Create SAP test case. Use praman SAP planner agent...` | | Generate from plan | `Goal: Generate test from plan. Use praman SAP generator agent...` | | Fix a failing test | `Goal: Fix this failing test. Use praman SAP healer agent...` | | Full pipeline | `/praman-sap-coverage` (Claude Code) | | Convert to gold | `Convert the attached test script to Praman gold standard format` | ## Troubleshooting ### Agent can't find the SAP app - Verify `SAP_CLOUD_BASE_URL` in `.env` points to the FLP home page - Verify the seed file authenticates successfully: `npx playwright test tests/seeds/sap-seed.spec.ts --project=agent-seed-test --headed` - Check that Chrome is launching in headed mode (add `--headed` if needed) ### Agent discovers wrong control types - SAP apps use different control frameworks depending on the Fiori Elements version: - **V2 apps**: `sap.ui.comp.smartfield.SmartField` (SmartField) - **V4 apps**: `sap.ui.mdc.Field` with `sap.ui.mdc.field.FieldInput` (MDC) - The planner auto-detects the framework. If it gets it wrong, specify in your prompt: "This is a Fiori Elements V4 app using MDC controls" ### Test passes locally but fails in CI - Add longer timeouts for `toPass()` intervals in CI environments - Ensure CI has network access to the SAP system - Use `storageState` for auth to avoid login flow in every test - Set `SAP_CLOUD_BASE_URL` as a CI secret, not hardcoded ### MCP server connection issues - Verify `.mcp.json` exists with the `playwright-test` server configured - Restart the MCP server: close and reopen your IDE - Check that `@playwright/test` is installed: `npx playwright --version` :::warning[Common mistake] Do not include `sapAuth.login()` inside your test body when using the setup project pattern. Authentication runs once in `auth-setup.ts` and the session is shared via `storageState`. Adding login calls in tests causes duplicate logins and session conflicts. ::: ## FAQ
How many heal iterations does it typically take? Typically 3-6 iterations depending on: - **SAP system response time** — slow systems need longer `toPass()` intervals - **OData data availability** — value help tables may load asynchronously - **Control rendering timing** — dialogs and popovers may not be immediately available - **Environment-specific quirks** — different SAP systems have different control IDs and behaviors The healer agent captures the exact error, identifies the root cause, applies a fix, and re-runs the test automatically. Each iteration gets progressively closer to a passing test.
Can I run the planner without a live SAP system? No. The planner agent connects to a live SAP system to discover UI5 controls, OData bindings, and value help structures. This live discovery is what makes generated tests accurate — the planner reads actual control IDs and types from the running application. If you do not have a live system available, you can write tests manually using the [Playwright Primer](./playwright-primer) and [Selectors](./selectors) guides.
The agent generates tests with page.click() instead of ui5.press() — how do I fix this? This means the agent is not using Praman fixtures correctly. Re-run the generator or healer with this instruction in your prompt: ```text Use 100% Praman fixtures for all UI5 elements. Never use page.click(), page.fill(), or page.locator() for UI5 controls. Use ui5.press(), ui5.fill(), and ui5.control() instead. ``` Or run the gold standard conversion prompt: ```text Convert the attached test script to Praman gold standard format. ```
How do I run the full pipeline with a single command? In Claude Code, use the slash command: ```text /praman-sap-coverage ``` This runs the planner, generator, and healer in sequence. For other IDEs (Copilot, Cursor), use the prompt template from Step 1 — the agent will run all three phases automatically.
:::tip[Next steps] - **[Selectors Guide →](./selectors)** — Understand how `ui5.control()` discovers controls by ID, type, and properties - **[Control Interactions →](./control-interactions)** — Learn `ui5.press()`, `ui5.fill()`, and proxy methods for all UI5 control types - **[Authentication Guide →](./authentication)** — Configure auth strategies for different SAP system types ::: --- ## For SAP Business Analysts You know SAP business processes inside out. You can walk through a Purchase-to-Pay cycle in your sleep, spot a missing three-way match before anyone else, and explain why a particular approval workflow exists. This guide shows how that knowledge translates directly into automated test coverage -- without requiring you to become a programmer. :::info[In this guide] - Define test scenarios using business process knowledge -- no programming required - Use the AI agent to generate executable tests from plain-language descriptions - Read and interpret test results in the Playwright HTML report - Map your existing activities (functional specs, UAT, compliance) to test automation artifacts - Collaborate effectively with Test Engineers on your team ::: ## You Don't Need to Code Praman includes an AI-powered workflow that turns plain-language business process descriptions into executable Playwright tests. Here is how it works: 1. **You describe the process.** Write a business scenario the way you would explain it to a colleague: "Create a purchase order for vendor 100001 with material MAT-001, quantity 10, plant 1000. Save it and confirm the success message shows a PO number." 2. **The AI agent generates the test.** Praman's agent reads your description, opens the SAP Fiori app in a real browser, discovers the UI controls, and writes a complete [Playwright](#glossary-of-technical-terms) test. 3. **The test engineer reviews and commits.** Your team's test engineer validates the generated test, adjusts [selectors](#glossary-of-technical-terms) if needed, and adds it to [version control](#glossary-of-technical-terms). 4. **The test runs automatically.** Every time someone deploys a change, [CI/CD](#glossary-of-technical-terms) runs the test and reports pass or fail -- with screenshots on failure. You stay in your comfort zone (business process knowledge), and the tooling handles the technical translation. ### The AI Command When working with the Praman agent, a single command kicks off the full pipeline: ```text /praman-sap-coverage ``` This command tells the agent to: - **Plan**: Analyze the SAP app and identify testable scenarios - **Generate**: Write Playwright tests for each scenario - **Heal**: Fix any tests that fail on first run You provide the business context. The agent handles the code. ## Your SAP Knowledge Is the Test Plan Every piece of business knowledge you carry maps to a specific test automation artifact. | What You Know | How It Becomes Test Automation | | ------------------------------------------ | -------------------------------------------- | | Business processes (P2P, O2C, R2R) | End-to-end test scenarios | | SAP transactions (ME21N, VA01, FB60) | Fiori app navigation via transaction mapping | | Expected outcomes and golden rules | Assertion criteria (pass/fail checks) | | Edge cases and business exceptions | Negative test coverage | | Compliance and audit requirements | Automated evidence collection | | Approval workflows and authorization roles | Role-based test configurations | | Master data dependencies | Test data setup templates | | Tolerance limits and thresholds | Boundary value test cases | When a BA says "the PO should not save without a valid vendor," that is already a test specification. Praman turns it into an executable check. ## How a Test Gets Created There are three paths from business requirement to running test. Your team will likely use all three depending on the situation. ### Path A: AI Agent Generates from Your Description You write a natural-language scenario. The agent does the rest. **Your description:** > Create a purchase order in the Manage Purchase Orders app. Use vendor 100001, purchasing > organization 1000, company code 1000. Add one item: material MAT-001, quantity 10, plant 1000. Save and verify the success message contains a PO number. **What the agent produces:** A complete, runnable Playwright test file with Praman fixtures, proper navigation, field interactions, and assertions. The test engineer on your team reviews and approves it. ### Path B: Test Engineer Writes from Your Functional Spec You provide a structured specification. A test engineer translates it into code. **Your spec:** | Step | Action | Expected Result | | ---- | -------------------------- | ----------------------------------- | | 1 | Open Create Purchase Order | App loads, header fields are empty | | 2 | Enter Vendor: 100001 | Vendor name auto-populates | | 3 | Enter PurchOrg: 1000 | Field accepted | | 4 | Enter Material: MAT-001 | Material description auto-populates | | 5 | Enter Quantity: 10 | Field accepted | | 6 | Click Save | Success message with PO number | **The test engineer maps each row to a `test.step()`.** The resulting test mirrors your spec line by line. ### Path C: Record and Replay A tester walks through the scenario in a browser while Praman records the interactions. The recording is converted into a test file that can be edited and maintained. This approach works well for complex workflows where describing every click would be tedious. ## Reading Test Results After tests run, you get results in several formats. Here is what to look for. ### Playwright HTML Report Open it with `npx playwright show-report`. The report shows: - **Pass/Fail/Skip counts** at the top -- a quick health check - **Each test** as a collapsible row. Green means pass, red means fail - **Test steps** inside each test (the `test.step()` blocks). These match your spec steps - **Screenshots on failure** -- an automatic browser screenshot captured at the moment of failure - **Traces on retry** -- a full recording of every action, network call, and DOM change When a test fails, the screenshot and trace tell you _exactly_ what the user would have seen. You do not need to reproduce the issue manually. ### Business Process Heatmaps When multiple tests cover a single business process (for example, P2P), Praman's reporter can generate a heatmap showing which process steps are covered and which are not. This helps you identify gaps in test coverage and prioritize new scenarios. ### Compliance Evidence Artifacts Every test run produces timestamped artifacts: screenshots, trace files, and structured JSON results. These serve as audit evidence that a process was validated at a specific point in time. For SOX, GxP, or internal audit requirements, these artifacts can be archived alongside your change documentation. ## SAP Transaction to Fiori App to Test Mapping This table maps common SAP transactions to their Fiori equivalents and shows how Praman tests interact with each. | SAP TCode | Description | Fiori App / Intent | Praman Test Approach | | --------- | ------------------------ | -------------------------------------- | ---------------------------------------------- | | ME21N | Create Purchase Order | `PurchaseOrder-create` | `ui5Navigation` + `ui5.fill()` on object page | | ME23N | Display Purchase Order | `PurchaseOrder-display` | `fe.listReport.search()` + detail verification | | VA01 | Create Sales Order | `SalesOrder-create` | `ui5Navigation` + `ui5.fill()` on object page | | VA03 | Display Sales Order | `SalesOrder-display` | `fe.listReport.search()` + field assertions | | FB60 | Enter Vendor Invoice | `SupplierInvoice-create` | `ui5.fill()` for SmartFields + post action | | FBL1N | Vendor Line Items | `SupplierLineItem-analyzeJournalEntry` | `fe.listReport` filters + row count assertions | | CO01 | Create Production Order | `ProductionOrder-create` | `ui5Navigation` + header/operation fill | | MM01 | Create Material Master | `Material-create` | Multi-tab navigation + field fill per view | | MIGO | Goods Movement | `GoodsReceipt-create` | PO reference + item verification + post | | VL01N | Create Outbound Delivery | `OutboundDelivery-create` | SO reference + picking quantity + post GI | | MIRO | Invoice Verification | `SupplierInvoice-verify` | PO reference + amount match + post | | F-28 | Incoming Payment | `IncomingPayment-post` | Customer + invoice reference + clearing | For the full mapping reference, see the [Transaction Mapping](./transaction-mapping) guide. ## What a Test Looks Like Below is a complete test for creating a Purchase Order. Every line is commented to explain what it does in business terms. ```typescript // This line loads the Praman test framework. // Think of it as "opening the test toolkit." // Test data — the values we will enter into the PO form. // Keeping them here makes it easy to change without touching the test logic. const poData = { vendor: '100001', // SAP vendor number purchaseOrg: '1000', // Purchasing organization companyCode: '1000', // Company code material: 'MAT-001', // Material number for the line item quantity: '10', // Order quantity plant: '1000', // Receiving plant }; // This defines a single test. The text in quotes is the test name // that appears in reports. test('create a standard purchase order', async ({ ui5, // Tool for finding and interacting with SAP UI5 controls ui5Navigation, // Tool for navigating between SAP Fiori apps ui5Footer, // Tool for clicking footer buttons (Save, Edit, Cancel) }) => { // Step 1: Navigate to the Create Purchase Order app. // This is equivalent to entering transaction ME21N in SAP GUI, // but for the Fiori version of the app. await test.step('Navigate to Create Purchase Order', async () => { await ui5Navigation.navigateToApp('PurchaseOrder-create'); }); // Step 2: Fill in the PO header fields. // Each ui5.fill() call finds a field on screen and types a value into it. await test.step('Fill header data', async () => { await ui5.fill({ id: 'vendorInput' }, poData.vendor); await ui5.fill({ id: 'purchOrgInput' }, poData.purchaseOrg); await ui5.fill({ id: 'compCodeInput' }, poData.companyCode); }); // Step 3: Add a line item to the PO. await test.step('Add line item', async () => { await ui5.fill({ id: 'materialInput' }, poData.material); await ui5.fill({ id: 'quantityInput' }, poData.quantity); await ui5.fill({ id: 'plantInput' }, poData.plant); }); // Step 4: Click the Save button in the footer bar. await test.step('Save the purchase order', async () => { await ui5Footer.clickSave(); }); // Step 5: Verify that SAP shows a success message. // This is the automated equivalent of visually checking // "Purchase Order 4500000123 created" on screen. await test.step('Verify success message', async () => { const message = await ui5.control({ controlType: 'sap.m.MessageStrip', properties: { type: 'Success' }, searchOpenDialogs: true, }); await expect(message).toBeUI5Visible(); }); }); ``` When this test runs, the HTML report shows five collapsible steps -- one per `test.step()` block. If step 4 fails, you see a screenshot of the exact screen state at the moment of failure, so you can immediately tell whether it was a data issue, a missing authorization, or an application bug. ## Your Role in the Team You do not need to write code to add value to test automation. Here is how your existing activities map to the testing workflow. | Your Current Activity | Test Automation Contribution | Artifact You Produce | | ----------------------------------- | ------------------------------------------ | ------------------------------------- | | Write functional specifications | Define test scenarios and expected results | Scenario descriptions or spec tables | | Validate UAT results | Review Playwright HTML reports | Pass/fail sign-off | | Document business rules | Define assertion criteria | Expected value lists, tolerance rules | | Identify edge cases and exceptions | Define negative test cases | Edge case catalog | | Manage test data in spreadsheets | Configure test data templates | Data sets for parameterized tests | | Map process flows (Visio, Signavio) | Define end-to-end test chains | Process flow to test step mapping | | Verify compliance requirements | Review audit evidence artifacts | Compliance checklist sign-off | | Triage production defects | Prioritize regression test additions | Defect-to-test-case traceability | The most effective SAP test automation teams pair a Business Analyst with a Test Engineer. The BA defines _what_ to test and _what the expected outcome is_. The engineer handles _how_ it gets automated. :::warning[Common mistake] Do not skip the Test Engineer review step. AI-generated tests need a developer to verify that selectors match the actual app, test data is valid for your system, and assertions check the right business outcomes. Treat generated tests as a strong first draft, not a finished product. ::: ## Glossary of Technical Terms These terms come up frequently in test automation conversations. Each is explained using SAP-familiar analogies. | Term | Definition | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **TypeScript** | The programming language tests are written in. Think of it as ABAP for the browser -- a typed, structured language that catches errors before runtime. | | **npm** | The package manager for JavaScript/TypeScript projects. Similar to how you install transports in SAP -- `npm install` brings in the libraries your project needs. | | **Fixture** | A pre-configured helper object injected into your test automatically. Similar to an ECATT variant that provides test data and tools -- you request `ui5` or `fe` in your test signature and they are ready to use. | | **Assertion** | A check that verifies an expected outcome. The `expect()` function is the automated equivalent of a UAT tester visually confirming "the PO number appears on screen." | | **Selector** | The address of a UI element on screen. Like a field technical name in SAP (e.g., `EKKO-EBELN`), a selector tells Praman which control to interact with. | | **CI/CD** | Continuous Integration / Continuous Delivery. An automated pipeline that runs tests every time code changes are deployed -- like a transport release process that includes automatic quality checks. | | **Git** | Version control for code files. Similar to SAP transport management -- it tracks who changed what, when, and why. Developers work in "branches" (like transport requests) and merge them into the main codebase. | | **HTML Report** | The test results page generated by Playwright. Open it in any browser to see pass/fail status, screenshots, and step-by-step traces for every test. | | **Test Step** | A named block within a test (`test.step()`). Corresponds to a row in your functional spec -- "Step 3: Enter vendor number." Steps appear as collapsible sections in the report. | | **Headed Mode** | Running the browser visibly on screen so you can watch the test execute. Useful for demos and debugging. In CI/CD, tests run "headless" (no visible browser) for speed. | ## FAQ
How do I work with the Test Engineer on my team? You provide the business knowledge: process descriptions, expected outcomes, edge cases, and test data. The Test Engineer handles the technical implementation: writing selectors, configuring auth, and setting up CI/CD. The most effective workflow is to write scenario descriptions (plain language or spec tables), have the AI agent generate a first draft, then have the engineer review and commit the final test.
Do I need to learn TypeScript? No. You can contribute test scenarios in plain language or structured spec tables. The AI agent or a Test Engineer translates these into TypeScript test code. However, being able to read the commented test examples in this guide helps you validate that the generated test matches your intent.
How do I know if the tests cover my business process? The Playwright HTML report shows every test with pass/fail status. Each `test.step()` inside a test corresponds to a step in your process (for example, "Fill header data" or "Verify success message"). If a step is missing, you can request it be added. Praman's heatmap reporter can also visualize which process steps have test coverage and which do not.
What happens when a test fails? A failed test produces a screenshot of the exact browser state at the moment of failure, plus a trace file that replays every action, network call, and screen change. You can open these in a browser to see exactly what the automated user saw. This is often enough to identify whether the failure is a data issue, a missing authorization, or an application bug -- without reproducing the problem manually.
:::tip[Next steps] - **[Getting Started](./getting-started.md)** -- Install Praman and run your first test - **[Playwright Primer](./playwright-primer.md)** -- Ground-up introduction for non-programmers - **[Control Interactions](./control-interactions.md)** -- How tests interact with SAP UI5 controls ::: --- ## Security Testing Patterns SAP applications handle sensitive business data and must enforce strict security boundaries. This guide covers automated security testing patterns with Praman, including CSRF validation, XSS input testing, authorization boundaries, and session handling. ## CSRF Token Validation SAP UI5 applications use CSRF (Cross-Site Request Forgery) tokens for write operations. Verify that the application correctly fetches and sends CSRF tokens: ```typescript test('CSRF token is sent with POST requests', async ({ page, ui5 }) => { await ui5.waitForUI5(); const csrfRequests: { url: string; token: string | null }[] = []; page.on('request', (request) => { if (request.method() === 'POST') { csrfRequests.push({ url: request.url(), token: request.headers()['x-csrf-token'] ?? null, }); } }); await test.step('Perform a write operation', async () => { await ui5.fill({ id: 'nameInput' }, 'Test Value'); await ui5.click({ id: 'saveBtn' }); await ui5.waitForUI5(); }); await test.step('Verify CSRF token was included', async () => { const postRequests = csrfRequests.filter((r) => r.url.includes('/odata/')); expect(postRequests.length).toBeGreaterThan(0); for (const req of postRequests) { expect(req.token).not.toBeNull(); } }); }); ``` ## XSS Input Testing Test that the application properly sanitizes user input to prevent cross-site scripting: ```typescript test('XSS payloads are sanitized in input fields', async ({ ui5, page }) => { const xssPayloads = [ '', '', "javascript:alert('xss')", ]; for (const payload of xssPayloads) { await test.step(`Test payload: ${payload.slice(0, 30)}...`, async () => { await ui5.fill({ id: 'descriptionInput' }, payload); await ui5.waitForUI5(); const innerHTML = await page.locator('#descriptionInput').innerHTML(); expect(innerHTML).not.toContain('