The Journey of Training an AI Language Model on Miniscript

Two paradigms stand in contrast. The conventional method where programmers meticulously code rules versus the AI approach where examples are provided, and the AI learns the rules. AI is known as a universal approximator, which means it can interpolate between examples, even if the examples are two images of cats or different texts.

The internet thus have created a fertile ground for AI to bloom. However, training an AI Language Model (LLM) on any niche language like Miniscript poses unique challenges due to the scarcity of examples. Within a single day, we produced 200 examples in Satshack São Paulo, laying the groundwork for the AI model's training.

For Legends of Lightning. we took a month curating the examples, deleting, rewriting them and writing others. The goal was to ensure that the dataset was of the utmost quality, enabling the AI model to generalize effectively. We could get to about 300 high quality examples.

However, just as we were gearing up for the training in the final week, the code that had been working seamlessly throughout the project inexplicably stopped functioning. This week was marked by exhaustive testing, experimentation with different libraries, and countless variations in an attempt to resurrect the functionality of the code. Despite our best efforts, we found ourselves in a frustrating loop of trial and error, with no apparent resolution in sight. The coding gods seemed to have abandoned us, leaving the team in the throes of uncertainty.

Just when it seemed that all hope was lost, the same code that had inexplicably failed before inexplicably started working again. Despite being exhausted and drained, we thanked the coding gods for their mercy. With the code back in working order, we now face the task of training and deploying the AI LLM model in just one day while in a 16-hour bus trip back home.

Giving up is not an option. We'll do our best.