EyeAuras is computer-vision and machine-learning powered clicker that can analyze anything that happens on the screen and perform actions such as mouse movement and keyboard presses on your behalf. It can also make sounds, send messages to Telegram and across the Internet and even execute C# scripts!
¶ Main concept
- Aura - combination of Triggers, Actions, Overlays and Enabling conditions. This is what you'll share with others, and this does the actual work.
- Behavior Tree - In short, behavior trees allow specifying to a robot what it should do. It's like its brain. Actions and Triggers serve as its hands and eyes.
- Triggers - any kind of state which could be described in Active/NotActive terms. This could be Hotkey (pressed/unpressed or toggle), foreground Window check (active/not active), Image/Text/Color search result(matched/not matched). Aura Active/NotActive state is a combination of states of her child Triggers. Trigger has three states:
- Active - this means that condition is met, e.g. for WindowIsActive trigger this means that Window with matching title is Active
- Not Active - this means that conditions is not met
- Unknown - this means that for some reason it is not possible to calculate trigger state. For example, if Window specified in ImageSearch is not found that means that corresponding Trigger will have Unknown state as it is not possible to do comparison without any actual image.
- Actions - showing notifications, key/mouse presses, sending Telegram messages or e-mails - these all is considered an Action and could be used inside Aura. Actions could be assigned to one of 3 groups:
- On Enter - these are executed when Aura becomes Active
- While Active - these will be executed repeatedly while Auras is Active
- On Exit - these are executed when Aura becomes Not Active
- Overlays - always-on-top overlays which could show text, image, custom UI or anything else. Overlays are a part of Aura and are shown only while Aura is Active
- Enabling conditions - Triggers, which could be used to disable/enable Auras and Behavior Trees, e.g. disable the bot if game window does not exist
¶ Subsystems
- Export/Import - transfer, share, or backup EyeAuras packs across different users or devices
- Aura Library - a shared collection of EyeAuras packs, available for users to discover, import, and use
- Window Match Expressions - enabling selection of specific windows using a custom expression
- Text Match Expressions - Powerful tool enabling text condition validation through Regex, Text, and Lambda evaluators.
- Bindings - interconnect properties between triggers, actions, and overlays to allow for easier change of group properties and to open up new capabilities
- Default Properties
¶ Triggers
- Fixed Value - the most primitive trigger, you can control it's state by selecting it either manually or by using C# scripts
- Color Search - active when Average color of a selected region matches with Target color. Similarity threshold could be specified.
- Image Search - active when some Image is found with a specified similarity
- AI/ML Search - machine-learning powered object detection/segmentation or classification, currently supports only Yolo8 in ONNX format, may be extended later
- Text Search - active when recognized text matches with specified expression. It could be comparison by Contains, regexp or C# Lambda
- Aura Is Active - active when linked Auras have specified state (Active/NotActive)
- Hotkey Is Active - active when specified combination of keys is either held down or toggled
- Window Is Active - active when window matching specified expression is active (in foreground)
- Window Exists - active when window matching specified expression exists in the system
- Timer - periodically activates itself for specified duration
- Message Subscription - activates/deactivates when specified message is received from EyeAuras webserver. Messages are separated by Channels and could be sent using SendMessage action
- File Contains - activates when specified text is found in specified file
- Telegram Subscription - activates/deactivates when specified message is received in Telegram channel
- Volume Control - activates/deactivates when volume level of specified audio device or process reaches specified threshold
- C# Script - custom scripts that use the latest version of C# language with full access to internal EyeAuras API. As soon as API will be stabilized there will be examples/docs.
¶ Actions
- Send - emulates user input - mouse movement, clicks, keyboard presses, etc. Supports multiple input methods starting from most basic ones that are used WinAPI and all the way to hardware-level emulation that uses Usb2Kbd physical device.
- Play Sound - plays specified sound
- Win Activate - activates specified window
- Delay - waits for some time before proceeding to the next action
- Send To Telegram - sends message to Telegram channel
- Send Message - sends network message through EyeAuras infrastructure to a specified Channel. All other instances of EyeAuras on other computers can subscribe and process these messages via Message Subscription trigger
- C# Script - custom scripts that use the latest version of C# language with full access to internal EyeAuras API. As soon as API will be stabilized there will be examples/docs.
¶ Overlays
- Text - shows text, content could be modified either manually or using C# scripts
- Image - shows an image, animated/transparent GIFs are supported
- Replica - created a real-time visual clone of any specified window or its subregion. This allows you to clone parts of UI (e.g. bring cooldowns closer to the middle of you screen) or show downsized version of YouTube player while farming
- Dependencies Viewer - debugging tool which could be used to check what states linked Auras have. Very useful for debugging complex models.
- Custom UI - allows you to develop a full-blown custom user interface using combination of Microsoft Razor Pages (html+css+js) and C# to wire it all together inside EyeAuras