xte

Control X from the command line, and find things on the screen

Install

All systems
curl cmd.cat/xte.sh
Debian Debian
apt-get install xautomation
Ubuntu
apt-get install xautomation
Arch Arch Linux
pacman -S xautomation
image/svg+xml Kali Linux
apt-get install xautomation
Fedora
dnf install xautomation
Windows (WSL2)
sudo apt-get update sudo apt-get install xautomation
Raspbian
apt-get install xautomation

xautomation

Control X from the command line, and find things on the screen

Control X from the command line for scripts, and do "visual scraping" to find things on the screen. The control interface allows mouse movement, clicking, button up/down, key up/down, etc, using the XTest extension. The visgrep program find images inside of images and reports the coordinates, allowing programs to find buttons, etc, on the screen to click on. If you just want to control X, you may find xdotool easier to use.