Home / Work / AISHA
CASE STUDY · AUTOMATION · SYSTEMS

Selectors break.
Sight doesn’t.

Conventional automation breaks the moment a website changes its layout. AISHA takes a different approach: it identifies elements by visual appearance and spatial relationship — the way a person would — heals itself when a pattern shifts, and keeps running through redesigns that would take a selector-based script offline.

Domain
Web automation
Approach
Computer vision + self-healing
Runtime
Rust / WebAssembly
Locator strategy
Visual, not DOM selectors
Control
Chromium DevTools Protocol
Resilience
Survives full site redesigns
CV
Vision-based element recognition
0
Selector maintenance after redesign
WASM
Browser or server-side execution
Layout changes tolerated
/ 01
The Problem

Enterprise data gathering from web sources fails constantly. CSS selectors break on redesigns. XPath fragments on DOM structure that changes with A/B tests. Automation that worked yesterday returns empty results today, silently.

Tools like Selenium and Playwright work well in controlled environments but require continuous maintenance when pointed at production sites that someone else controls. The selector maintenance burden frequently exceeds the value of the data being collected.

The underlying issue is that DOM structure is an implementation detail. Automating against it means depending on something the site owner is free to change without notice — and regularly does.

/ 02
The Approach

AISHA uses computer vision as the primary locator, not DOM selectors. Elements are identified by visual appearance and spatial relationship — a person finds the submit button because it looks like a submit button in the bottom right, not because of an attribute chain.

When a visual pattern fails to match, the system falls back to semantic understanding of page context to find the nearest functional equivalent, and updates its own reference. That is what "self-healing" means here: the automation adapts rather than raising an exception.

The engine is written in Rust and compiled to WebAssembly, so the same automation logic runs either browser-side or server-side for high-throughput orchestration, driving the browser through the Chromium DevTools Protocol.

/ 03
The Hard Parts
  1. Visual matching that tolerates redesign but not ambiguity Matching too loosely clicks the wrong control; too strictly and every minor style change breaks it. Calibrating that threshold — and knowing when to fall back to semantic context — is the core difficulty.
  2. Deciding when healing is wrong A self-healing system that confidently picks a substitute element can take a destructive action on the wrong control. Knowing when to adapt and when to stop and report is a safety property, not a convenience.
  3. Interaction timing that reflects real use Instant, perfectly regular interaction is both unrealistic and fragile on pages with asynchronous rendering. Timing is modelled statistically so interactions land after the interface is genuinely ready.
  4. Rust to WASM with browser control Compiling an automation engine to WebAssembly while retaining full DevTools Protocol control is a non-trivial architectural constraint that shaped the whole design.
/ 04
The Outcome

AISHA maintained operation through multiple complete redesigns of target web applications with no code changes — the vision layer adapted where a selector-based script would have needed rewriting each time.

The WebAssembly target enables browser-side execution where that is appropriate, or server-side orchestration for high-throughput scenarios, from enterprise data collection to automated testing of applications that resist conventional tooling.

The same resilience philosophy — assume the third party will change without warning, and degrade rather than fail — underpins the client automation systems we deploy.

Stack Rust WebAssembly Computer vision Self-healing locators Chromium DevTools Protocol Semantic page understanding Statistical interaction timing

Automation that needs weekly repair
isn’t automation.

DATA BEHIND A HOSTILE INTERFACE

The data you need is often behind
a system that doesn’t want to give it up.

Contact Us →