I have always been fascinated by unconventional programming tools and methods. Partially because learning them helps me become a better programmer in my job, and partially because the genius behind some of these systems tickles my brain in just the right way. Whatever the reason might be, I aim to tryout new things every few projects, and more than often it pays off.
Although a data scientist by trade, I'd like to think of myself as someone who creates tools and automations, and tools to make those automations easier, and automations to make those tool creations more powerful. But there are only so many things you can automate in your own day to day life, after a reading list system, a project manager, a wiki and a diary and quiet a few different odd gadgets to do your bidding around the house, you start inventing imaginary things to solve in your free time. You might even start to automate things in your online browser based games to make you focus more on enjoying the experience rather than on trying to mini-max your way through the game.
This blog post is a report of my time playing a fascinating online game called Torn and the many tools I started creating in it, and how LISP, of all things, helped me enjoy programming even more than I normally do.
A very brief detail about the game, there are numbers involved, and randomness, and datasets and most importantly, APIs. Doesn't take an expert to realize you can try to conquer the randomness and predict the future, thereby shaping your path slightly better. But as with most text based games, Torn is an incredibly complex game, and it would be foolish to try to replicate the entirety of its logic in your own toy project, right?
First thing I realized when I tried to take my first stab at this game was that, unlike some other games, a stupidly complex excel file with many a lines of macros is not enough, and I need something more powerful.
So I took a look at my wiki and decided to give common lisp another go, and see how far I can get by using it. At first I made a small script to notify me when certain things where running low in our groups inventory, an easy enough task that didn't take long to implement. But then someone asked if I can modify this alert to appear if an item is running lower than 250 instead of 50, and I updated the code without restarting the server executable.
Seems quiet insignificant, but let me go into details of what exactly happened.
In any and all software programs, there are two sources of values, they are either hard coded in the executable somewhere, namely they are instructions of the code, or they are loaded from somewhere else, in other words they are the input of the code. In most programming languages the distinction is pretty clear, and although you can normally call the same code with different inputs, in order to change the instructions you have to restart the system. But not in lisp.
You see, common lisp has a running image, it contains everything the lisp process has access to, including itself, all the packages and all the function definitions. And the funny thing is, the instructions themselves are modifiable, and so you can change an instruction while an image is running, and this isn't some hack you do on the language that requires your code to bend over backwards, no, this is a built-in feature of the language. You just redefine a function, add a case for a method, or change the value of a variable and at most you'd get a warning back.
Although quiet insignificant, this gave me an idea. I don't have to replicate the entire logic of the game in order to do the calculations that I want, I can just implement enough to get answers now, and leave holes for the future. So I started creating classes and methods for calculating stuff, while the image was running, and I would make frequent requests to the image to calculate things for me. Every time I would get an error because something was not defined, I would just define it with as little info as I could, and tell the debugger to resume the calculations, and slowly and overtime, I had enough of the game logic emulated that I could make certain calculations without any issues.
But this magic wasn't limited to the game logic, it was also available to the world I was interfacing with as well. Seeing as how Torn was a social game, I connected the service to a discord bot using the lispord library initially, and a small web page using CLOG, both of which are amazing tools and reading their source code helped me gain a better understanding of lisp. But there were problems here and there...
lispcord was fairly outdated, and it was missing many features available in discord. So in my project I created a file called lispcord-fixes.lisp and I gradually included things that was missing in the lispcord library, sometimes creating classes and method instances for new types of messages and channels that weren't in lispcord, and sometimes I'd overwrite the existing ones to provide a feature that was missing. It felt incredibly empowering being able to do that, while a discord bot was running. I would see an error pop up because someone called the bot in a thread and lispcord wasn't supporting threads, so I would look at how it is handling channels and replicate the code to work for threads too, and the bot replied to the original request, only about 30 minutes later.
As for the CLOG side, clog itself is meant to be a tool to help you create websites, but I also wanted to serve APIs, so I changed one of the built-in functions of CLOG to allow me to route requests with certain paths to my own function, which would decide what calculation to run.
All of these, without restarting the server.
So how does an average programming session looks like?
I often start by connecting slime to the lisp image running on the server and check for any debugger prompts that might have been captured. Because remember, I have the bare minimum implemented and there could be many missing features. If there was no errors, I'll start by calling a function that would cause an error.
The debugger often pop up with a stack trace, at each step showing named local entities. I first try to find what caused the issue, is it a missing method? An undefined input? After identifying the problem, I use one of the editor shortcuts to jump to that definition. Initially I would go to where the function was defined by hand, but after a while I added the following code to my emacs config, allowing me to map filenames as provided by lisp image to file names as stored on my machine, because remember, I'm connected to the "production" server.
(defun connect-lisp-server ()
(interactive)
(setq mslime-dirlist '(("/home/sajjad/src/2023/3.torn/" . "/home/ubuntu/torn/")
("/home/sajjad/common-lisp/clog/" . "/home/ubuntu/.quicklisp/dists/quicklisp/software/clog-20230214-git/")
("/home/sajjad/quicklisp/" . "/home/ubuntu/.quicklisp/")
))
(setq slime-from-lisp-filename-function
(lambda (f)
(let ((dirs (car (seq-filter (lambda (x) (string-prefix-p (cdr x) f)) mslime-dirlist))))
(if dirs
(concat (car dirs) (substring f (length (cdr dirs))))
f))))
(setq slime-to-lisp-filename-function
(lambda (f)
(let ((dirs (car (seq-filter (lambda (x) (string-prefix-p (car x) f)) mslime-dirlist))))
(if dirs
(concat (cdr dirs) (substring f (length (car dirs))))
f)))))
Then I start adding whatever is missing, and compile them right into the running image. At this point the compiler might throw an error or two because of unused variable names or in rare cases, because of a derived type mismatch.
Here's where the magic happens, I go back to the debugging window and "retry" the request, and more often than not, it would just work, replying to requests my friends made the night before, as if nothing happened.