Dissection of a Bot Platform
For almost a decade, I've been quietly developing a modular computer vision based automation platform. Recently I realized that the platform–Chronos–had slowly taken over as my largest personal project.
The core idea behind Chronos was to create a generic bot, one that was not designed around any specific game. It aimed to provide all the things a bot would need in a package that could be easily snapped on to pretty much any game. I like to think of Chronos as similar to a pared-down self-driving car; it essentially solves the same problems, but in a much narrower scope that is tractable as a single developer.
Initially, the platform targeted Android games running inside an emulator. Today, it still supports Android, but is also capable of ingesting video streams and producing output in the form of keystrokes and mouse events delivered by a Raspberry Pi. It can also be hooked up to remote desktop connections, browsers, and pretty much any other source of pixels.
Sidebar: Believe it or not, I don't actually use the Chronos platform to bot in games (anymore), but I've continued maintaining the vision engine for years simply because it is fun.
I don't sell the platform, either. I've considered selling it as a CI/CD integrated GUI testing tool for development teams. If you'd be interested in that, reach out to me.
I learned a lot developing Chronos, and I'd like to share some of that learning with you. This is the start of a blog series about Chronos which will discuss the following parts:
A modular, generic bot with an automation framework based around LuaJIT and a suite of runtime optimizations to enable high framerate computer vision.
A cross platform user interface framework which allows Chronos and Mobius to run locally as a Windows application or on a remote server as a web application.
A data processing toolchain for slicing, labelling, generating, and preparing training data and script resources.
There is also a lot of supporting code for infrastructure, cloud, users, payments, and so on. I don't intend to discuss these pieces specifically, but they may come up from time to time.