A team’s CI/CD dashboard shows green on every build, and their last three production defects still happened in Teamcenter’s Java Rich Client, a layer their pipeline was never actually exercising

TL;DR

• Most CI/CD test runners are built around web-layer test execution: headless browsers, DOM-based assertions, standard HTTP checks.
• PLM regression that includes Java Rich Client interaction or canvas-rendered UI cannot run through a standard headless-browser CI/CD stage, since neither layer exposes a DOM for that stage to exercise.
• This means many teams running “automated regression on every build” are actually running web-layer regression on every build, with Java-layer and canvas-layer PLM coverage still happening manually or on a separate, less frequent schedule.
• Wiring cross-layer PLM regression into CI/CD requires a runner that can execute the same relational identification approach used in manual test execution, inside the pipeline’s automated trigger.
• The practical payoff is catching Java-layer and canvas-layer regressions at the same cadence as web-layer ones, rather than discovering them only during a less frequent manual pass.

What “automated on every build” actually means in most PLM environments today

Ask a QA lead whether their PLM regression suite runs automatically on every build, and most will say yes without hesitation. Look closer at what’s actually triggering on every build, and the picture usually splits in two. Web-layer regression, checks against Teamcenter’s Active Workspace or Windchill’s PDMLink web portal, genuinely does run automatically, triggered by the same CI/CD pipeline stage that handles the rest of the application’s test suite. Java Rich Client coverage and canvas-rendered attribute grid checks, by contrast, often still run manually, or on a separate weekly or monthly schedule maintained outside the main pipeline, a split most teams haven’t examined explicitly because both halves get described the same way in a status meeting: “regression is automated.”

That split matters because it’s not evenly distributed across risk. A meaningful share of PLM defects concentrate at exactly the layers that aren’t running on every build, which means the pipeline’s green status is measuring the lower-risk half of the regression scope more frequently than the higher-risk half.

Why standard CI/CD test runners stop at the web layer

A typical CI/CD test stage spins up a headless browser, a browser instance running without a visible UI, controlled programmatically to load pages and interact with DOM elements, or hits an API endpoint directly to verify a backend response. Both approaches work well for web-layer testing because both operate against interfaces that expose a queryable structure: a DOM tree for the browser, a defined request/response contract for the API.

Neither approach has a path into a Java Rich Client window, since that window is a separate native process with no DOM at all, or into a canvas-rendered grid, since canvas content is painted as pixels with no addressable structure regardless of whether it’s running inside a browser or not. This is the same underlying limitation covered elsewhere on this blog for manual test execution, applied here specifically to the CI/CD pipeline context: a headless browser stage simply has nothing to query once a workflow crosses into either of these layers.

What a cross-layer CI/CD stage actually requires

Closing this gap requires a test runner capable of relational identification across web, Java Rich Client, and canvas layers, the same underlying approach this blog has covered for manual and scheduled execution, but with one additional requirement specific to the pipeline context: the runner needs to execute in a CI-compatible mode, without requiring a visible, interactive desktop session for the Java Rich Client portion of a test.

This matters because a CI/CD build agent typically runs in a minimal, often headless environment by design, to keep build infrastructure lightweight and consistent. A test approach that assumes a full interactive desktop session, with a visible screen a human could look at, doesn’t fit that environment without additional infrastructure work. A cross-layer runner built to operate correctly in this kind of constrained environment removes that additional infrastructure burden.

A concrete pipeline configuration, walked through

Consider a nightly build pipeline for a Teamcenter implementation. The existing setup triggers a web-layer regression stage automatically against Active Workspace, reporting results into the team’s CI dashboard alongside the application’s other automated tests. Adding cross-layer PLM coverage means adding a second regression stage to the same pipeline, one that runs a Sahi Pro suite covering a BOM approval workflow spanning both Active Workspace and the Java Rich Client, using the same relational identification logic throughout the flow.

The result reports back into the same CI dashboard as the web-layer tests, not into a separate manual log a QA engineer checks independently. A build that fails the Java-layer stage shows up the same way a build that fails the web-layer stage does, in the same pipeline view, at the same point in the build process, rather than surfacing days or weeks later during whatever manual or scheduled process previously covered that layer.

What changes once this is running on every build instead of on a schedule

The practical difference shows up in how quickly a regression surfaces relative to the change that caused it. A Java-layer defect that previously surfaced during a monthly manual regression pass might not be discovered until a week or more after the change that introduced it, by which point several other changes have layered on top, making root-cause analysis slower and more expensive. The same defect, caught on the build where it was introduced, surfaces the same day, against a much smaller set of candidate changes, which is usually the difference between a quick fix and a multi-day investigation.

This also changes the conversation a QA lead can have with engineering leadership about release confidence. “Regression passed” starts meaning the same thing across every layer of the application, rather than meaning “the layer we automated first passed, and we’ll find out about the rest later.”

What to measure before and after making this change

A useful way to make this change’s value concrete to engineering leadership is measuring two specific numbers before and after: the average time between a Java-layer or canvas-layer defect being introduced and it being caught, and the number of production incidents traced back to either layer over a comparable period. Most teams have never measured the first number directly, since it requires cross-referencing a defect’s root cause against the commit history, but it’s usually the more persuasive of the two once calculated, since it makes the cost of the current manual-schedule gap concrete rather than theoretical.

It’s also worth tracking build time impact honestly rather than assuming it. A cross-layer stage added as a parallel job typically adds little to the overall pipeline duration a developer experiences, since it runs alongside the existing web-layer stage rather than after it, but this should be measured on your own pipeline rather than assumed from a general claim, since actual impact depends on your specific suite size and infrastructure.

Sources

• Standard CI/CD test runner architecture and headless browser execution models (general industry documentation)
• Sahi Pro CI/CD integration documentation (sahipro.com)
• Siemens Teamcenter Java Rich Client architecture documentation
• Sahi Pro vs Selenium