AI Windows: Input, Hooks, Hotkeys
AI-first map of input sending, hooks, interception, and hotkey services.
Input Hooks / Hotkeys / Send Input Discovery Map
Reference map for keyboard/mouse hooks, hotkeys, script keybinds, send-input APIs, input simulators, smoothers, suppression, and native detours.
User Intents
- Bind script methods to hotkeys.
- Detect hotkey state as aura trigger.
- Send keyboard/mouse/text input.
- Select input simulator or smoother backend.
- Track raw keyboard/mouse events.
- Suppress keybind input when needed.
Concept Model
[Keybind]is script method metadata.HotkeyIsActiveTriggeris the aura hotkey condition.ISendInputScriptingApiis the script-facing input output surface.IKeyboardEventsSourceexposes host keyboard/mouse event streams.IInputSimulatorProviderselects send-input backend.IUserInputSmootherProviderselects mouse movement smoother.- Low-level hook/detour APIs are infrastructure, not the first script API.
API Details
KeybindAttribute- script hotkey binding metadata.KeybindAttribute.Hotkey- string gesture such asF5orCtrl+Shift+P.KeybindAttribute.SuppressKey- when true, marks the original key event as handled/suppressed for both key-down and key-up handling.KeybindAttribute.IgnoreModifiers- when true, required modifiers must be present but extra modifiers are allowed.KeybindAttribute.ActivationType- selectsKeyDownorKeyUpinvocation.KeybindActivationType- key-down/key-up enum used by[Keybind].HotkeyGesture- code-friendly hotkey value. Prefer constructors/enums when the hotkey is known in code.IHotkeyConverter- string-to-HotkeyGestureconversion. Mostly useful for persisted user settings, editor input, or config text.IHotkeyTracker- runtime hotkey tracking.IsEnableddefaults totrue; set it only when intentionally disabling/re-enabling a tracker.HotkeyIsActiveTrigger- aura trigger for hotkey state.ISendInputScriptingApi- script send-input API.IInputSimulatorProvider,IInputSimulatorEx- simulator discovery/backend.IUserInputSmootherProvider,IUserInputSmoother- mouse smoothing.- Built-in smoother ids include
BenLandLinearFast,BenLandRandomizedFast,BenLandRandomizedNormal,Kalon, andMlHumanMouseV1,MlHumanMouseFastV1. MlHumanMouseV1- ML Human Move smoother. Uses a local ML.NET FastTree model pack to predict compact movement parameters, generates a bounded Bezier/easing path, and applies the default 2x ML timing multiplier.MlHumanMouseFastV1- ML Human Move Fast smoother. Uses the same model-backed route planner with a 4x ML timing multiplier.IKeyboardEventsSource- keyboard/mouse event streams.HookEngine- advanced native function detour engine.
Assembly / Package Hints
IInputSimulatorProviderlives in the Roxy shared compile surface.IInputSimulatorEx,ISendInputScriptingApi, simulator ids, and send-input contracts are shared/metadata APIs; Roxy runtime modules provide the actual simulator implementations.- Standalone embedded apps should request the Roxy capability with
UseRoxy()and wait for readiness before resolving simulator-backed services.
Prefer
- Prefer
[Keybind]for simple script keyboard shortcuts. - Prefer
HotkeyIsActiveTriggerfor user-authored aura hotkey conditions. - Prefer
new HotkeyGesture(...)in code when the key/modifiers are known. - Prefer
IHotkeyConverterwhen reading a hotkey from persisted text or user input. - Prefer
ISendInputScriptingApifor script input output. - Prefer provider-discovered simulator/smoother ids.
- Prefer osd/selection.md for selecting a target window first.
Avoid
- Avoid raw global hook subscriptions for simple hotkeys.
- Avoid parsing hard-coded hotkey strings through
IHotkeyConverterwhen aHotkeyGestureconstructor would be clearer. - Avoid setting
IHotkeyTracker.IsEnabled = truein examples unless the example first disabled it;trueis already the default. - Avoid assuming simulator ids are installed on every machine.
- Avoid using native detours for normal keyboard/mouse hooks.
Common Flows
-
Runtime hotkey with Rx:
- create
IHotkeyTrackerthroughIFactory<IHotkeyTracker>. - set
Hotkeywithnew HotkeyGesture(...)for code-defined hotkeys, orIHotkeyConverterfor persisted/user-entered strings. - set
HotkeyMode,SuppressKey,IgnoreModifiers, orHandleApplicationKeysonly when the defaults are not right. - subscribe to
WhenAnyValue(x => x.IsActive)and filter fortrue. - dispose the tracker and subscription with the owner lifetime; in top-level
scripts, use
ExecutionAnchorsfor per-run resources.
- create
-
Script method
[Keybind]:- decorate instance methods on the script object.
- methods can be public or non-public.
- multiple
[Keybind]attributes can be placed on one method. - the runtime parses the string with
IHotkeyConverter. - handler invocation is scheduled on the background scheduler.
- method parameters are resolved from the script DI container before invoke.
- if the method returns
Task, runtime waits for task completion. - exceptions are written to aura event log as failed keybind events.
- registration is active only while the script execution tracker is in the running state.
Send input without creating an aura component
In a C# Action, inject the service and send one key to the active application:
ISendInputScriptingApi SendInput { get; init; }
SendInput.KeyPress(Key.F1);
Log.Info("F1 request sent");
Roxy registers this API; the default simulator is Windows Input. The log confirms call completion, not that the application processed the key. TargetWindow does not guarantee background input. See SendInput operations for timing, coordinates and cancellation.
For key reactions use the complete Keybind example. A dynamic tracker and its subscription belong to the caller. Native event callbacks and background keybinds do not authorize direct UI mutation; use the UI dispatcher when needed. Ordinary classes receive these dependencies explicitly.
Research Anchors
KeybindAttribute,KeybindActivationType,HotkeyTrackerRuntimeVisitor,HotkeyGesture,IHotkeyConverter,IHotkeyTracker,HotkeyIsActiveTrigger,ISendInputScriptingApi,IInputSimulatorProvider,IUserInputSmootherProvider,IUserInputSmoother,MlHumanMouseV1,MlHumanMouseFastV1,IKeyboardEventsSource,HookEngine.
Search Synonyms
- hotkey
- keybind
- keyboard hook
- mouse hook
- send input
- input simulator
- input smoother
- ML Human Move smoother
- MlHumanMouseV1
- MlHumanMouseFastV1
- target window input
- suppress key
- ignore modifiers
- key down
- key up
- keybind activation type
- method parameter injection
- native detour
Related Maps
- scripting/runtime.md
- auras/triggers.md
- auras/actions.md
- osd/selection.md
- windows-subsystems/window-handles.md
Tracker, provider and detour boundaries
IHotkeyTracker Hold follows key-down/up; Click toggles on key-up, so filtering IsActive == true in Click mode does not invoke on every press. With HandleApplicationKeys false, Click events are excluded while EyeAuras itself is active. This tracker behavior is separate from KeybindAttribute.ActivationType.
InputSimulatorProvider.ResolveById returns a registered object; Resolve(idOrName) returns an observable and may create a proxy before a real backend is registered. A known name does not establish IsAvailable. The smoother provider likewise resolves registered IDs; a catalogue entry does not install a driver.
HookEngine is a separate current-process native detour mechanism. It retains the detour delegate, returns an original-call delegate and requires explicit hook enabling. Dispose disables hooks and releases its allocator. This is not keyboard hotkey registration or a guarantee of safe patching while other threads execute the affected function.