Cardboard Sikuli

IDE and scripting platform to automate and control desktop applications, for GUI application testing or robotic process automation (RPA).

This is a clean-room implementation of the Sikuli application based on RobotGo, Goja, and MIQT.

• OpenCV image matching
• Desktop IDE with screen grabber based on Qt 6
• It uses Javascript as the scripting language instead of Python 2

Compiling

Tested on Debian 12

apt install libopencv-dev qt6-base-dev libx11-dev xorg-dev libxtst-dev build-essential golang-go
go build -ldflags '-s -w'

Container and CI usage

Running the IDE in a container helps isolate your scripts and images from any OS changes.

# Build cardboard-sikuli and the runtime environment
# The resulting Docker image is about 1.1 GiB.
docker build -t cardboard-sikuili:latest -f cardboard-sikuli.Dockerfile .

# Run interactively (IDE)
docker run --rm cardboard-sikuili:latest
# ""Cardboard-sikuli is now running - connect to VNC on port :5901""
# Either use -p to map the IDE VNC port, or, find the container's IP from `docker inspect`

# Run a test (one shot)
docker run -v /path/to/my/script:/src cardboard-sikuili:latest --run /src/my_file.Sikuli

See also

• Original MIT Sikuli
• RaiMan's SikuliX
• Application icon from Material Design icons (eye-tracking)

Scripting reference

• SIKULI_WAIT - Global variable to set the default refresh rate between operations (milliseconds). Default = 200
• SIKULI_TIMEOUT - Global variable to set the default time to wait for an image to appear (milliseconds). Default = 3000
• SIKULI_THRESHOLD - Global variable to set the image match accuracy threshold (floating point from 0 to 1). Default = 0.9
• ClickAt(filepath, offset_x, offset_y) - Wait for image to appear, then move the mouse and click it at the specified offset. Throws a Javascript error if no match is found.