Ui.Vision

Task and UI test automation with Computer Vision/OCR. Ui.Vision combines browser automation and desktop automation.

Welcome to the UI.Vision BETA Edition and thank you for helping to beta test. For bug reports, questions and suggestions, please post them in the Ui.Vision community forum at https://forum.ui.vision. ****BETA release**** Ui.Vision is an open-source RPA and web automation tool that combines classic browser automation (Selenium IDE compatible) with modern computer vision, OCR, and an AI assistant. Record macros, scrape data, fill forms, and automate any repetitive task in your browser - no coding required. **NEW in this version:** ★ AI Assistant: Describe what you want in plain English ("Create a macro that fills out this form") and the built-in AI builds, tests, and fixes the macro for you. A free AI tier is included - no API key or account needed. Power users can bring their own key (Anthropic, OpenRouter) or connect a local LLM. ★ Browser Vision: The new BClick, BType and BMove commands click and type like a real user - with visual targeting by image or text (OCR) - directly inside Chrome and Edge. No extra software needed, and unlike desktop automation tools, the browser window can stay in the background while the macro runs. Great for canvas apps, drag & drop, and websites that ignore normal automated clicks. ★ New streamlined sidebar: Record, edit, and run your macros right in the browser side panel, next to the page you are automating. The classic full IDE window is still one click away. (1) Browser Automation & Web Macro Recorder Ui.Vision's visual macro recorder and computer vision commands make automating tasks inside the web browser easy: form filling, web scraping, data extraction, UI testing, and screenshots. Existing Selenium IDE scripts can be imported. Conversion help for iMacros scripts is provided, too. (2) Visual Automation with Image Recognition and OCR Automate what you can SEE, not just what is in the HTML: click buttons by screenshot, find text on the screen with OCR, and verify page layouts visually. Image recognition and OCR now run directly inside the extension - no additional installs required. (3) Desktop Automation for Windows, Mac, and Linux Beyond web browser automation, Ui.Vision can interpret images and text on the desktop, executing actions like clicking, moving, dragging and dropping the mouse, and simulating keyboard inputs. This desktop automation requires installing the free Ui.Vision XModules, available for Windows, Mac, and Linux. These modules provide Ui.Vision with the necessary capabilities for desktop interaction. (4) AI Commands & Anthropic Claude Integration The AI commands allow you to automate complex tasks with a single line of code that would traditionally require hundreds of lines of classic commands. For example, you can teach Ui.Vision to play TicTacToe with just one short "Play this game..." prompt. And the new AI chat assistant in the sidebar creates and repairs complete macros from a plain-language description. (5) Command Line API Ui.Vision can be controlled from the command line. Thus it can be used from, and combined with, any programming or scripting language, such as Windows batch files, Linux/Mac shell scripts, Python or PowerShell. (6) Open-Source Ui.Vision is Open-Source. The source code is available on Github. (7) 100% Local Software - Your Data Stays With You The software does not send any data back to us or any other place. Everything, including image recognition and OCR processing, is done locally on your machine. The only exception to the "all data is processed locally" rule is if you use the optional AI features. These cloud features are disabled by default - and even for AI you can stay fully local by connecting Ui.Vision to a local LLM. The default computer vision (image recognition and OCR) always runs 100% locally on the machine. Whether you call it RPA, macro recorder, task automation, web scraping, screen scraping, form filler, or UI testing - if it is repetitive, Ui.Vision can automate it. **Happy Automating!** For questions and suggestions, please visit the Ui.Vision community forum at https://forum.ui.vision.