Guix

Xautomation can control X from the command line for scripts, and do visual scraping to find things on the screen. The control interface allows mouse movement, clicking, button up/down, key up/down, etc, and uses the XTest extension so you don't have the annoying problems that xse has when apps ignore sent events. The visgrep program can find images inside of images and reports the coordinates, allowing programs to find buttons, etc, on the screen to click on.