ML Shortcuts

ML Shortcuts is a macOS application that enables users to control their Mac by mapping the outputs of custom CoreML or CreateML models to keyboard shortcuts and mouse inputs. Designed for macOS 14 and later, the tool is compatible with both Apple Silicon and Intel-based machines. It allows users to import their own machine learning models—specifically supporting classifier types such as hand pose classification, hand activity classification, sound classification, and image classification—and use the recognized classes to trigger system commands or automate workflows.

The primary feature of ML Shortcuts is its ability to connect the results of machine learning model classifications directly to macOS controls without requiring any coding. Users can prototype gesture-based or sound-based interactions, test new interface ideas in real time, and automate repetitive tasks by mapping model outputs to specific shortcuts. The tool is suitable for rapid prototyping and can be integrated into design workflows, including testing interactions in design tools like Figma. It follows a "Bring Your Own Model" approach, providing a sample hand activity model but encouraging users to train and use their own models for optimal results.

ML Shortcuts automatically detects all classes in an imported model and supports both vision and sound-based models. The setup process involves importing a CoreML model, mapping classifications to desired shortcuts, and granting necessary accessibility, camera, and audio permissions. The tool is designed to be accessible to users without coding experience, making it suitable for designers, prototypers, and anyone interested in creating custom controls or experimenting with new interaction paradigms on macOS.

9 MB) and is offered under a free, pay-what-you-want pricing model. This makes it accessible for a wide range of users interested in enhancing their macOS workflow with machine learning-driven automation.