20 years of programming

I didn’t know it as programming at first, but that’s exactly what I was doing when I decided to learn some scripting ² for my own Ragnarok Online private server to play with my friends. I wanted to make my own NPCs and events, and those were scripted.

I don’t think I ever got my friends to play on the server with me, but I remember having so much fun figuring out how to make the NPCs do exactly the things I wanted.

When my dad discovered that I was messing with scripting, he gave me a copy of the 6th edition of “ Java: How to Program ” (in portuguese) and told me I should learn that, since it would be more useful to me ³ . I’ll never forget that cover. A few weeks ago I had the good fortune to visit my parents, and my dad still had the book, so here’s a pic of the actual book:

This book is huge both in size and weight, especially for an average 11-year-old — it has over 1500 pages! —, but somehow I don’t ever recall being intimidated by it. I just dove straight in.

I don’t remember if the book mentions that it will explain those words in later chapters, but I remember I wanted to know right then. I couldn’t wait. I had already written public static void main a few times, and I needed to know why I had to keep writing those words.

My dad couldn’t explain things to me in a way that I understood, and I wasn’t having any of it anymore. We had an argument. I kept going through the book, but eventually the frustration was too big of having to do shit I didn’t understand because that’s what the Java gods wanted, and I stopped learning Java soon after that (lucky me).

Reflecting on it now, I’ve always had this tendency to question things that we/I have to do without knowing why we/I do them. I call it my anti-tradition and anti-dogma trait. It’s one of the key traits that contributed to the amount of technical knowledge I have now. It’s also got me in trouble before, and I’m sure it will still get me in trouble in the future, but I just can’t help it when I see things being done without knowing why they have to be done.

Even though I swore off Java, I didn’t stop programming. I don’t think I’d ever be able to stop programming, even if my life had taken a different direction. I had already been tainted by the fun of programming, even though “big people” programming had layers of bullshit on it.

A couple years after dropping Java (and Ragnarok Online), I started playing Ultima Online on a private server ⁴ after having some fun with RuneScape .

I learned some C# to mess with NPCs and custom items for an UO private server for my friends, but C# had Java vibes and I quickly lost interest. But the UO ecosystem had some tools to automate tasks in-game using scripts, and those caught my interest primarily because the development cycle was so fast: change a script and run, and you instantly see the results in-game. I wrote and sold a bunch of my scripts for in-game money, with some people paying a premium for versions of my scripts that only they got access to.

After dropping UO, I started playing Flash games and, as usual, got curious and learned how to make them. Flash games were usually written in ActionScript , and once again the quick development cycle hooked me. I made a few games, but this time I sold them for real money. A couple years later I was studying Computer Science in university.

Throughout all this time, my focus and the things I love about programming, even its meaning to me, have shifted. From making in-game experiences for my friends and I, to making game experiences to people all over the world, to exploring new things that could be done with programming; from solving programming problems competitively to being paid to build robust and scalable systems, to making little scripts only for myself. There’s no part of programming I haven’t loved. Even the things that annoy me so much now, I once loved those too.

But there are things that haven’t changed. We’ve been doing some things without thinking about why we’re doing them, and I want to talk about a particular one in this post.

On naming

These are the things that make naming hard to me: it is hard to come up with a great name and it is hard to change a name. They’re part of a positive feedback loop: because it’s hard to come up with a great name, we skip this effort most of the time ⁵ and use average names, but the more we use average names, the harder it is to change them (think of all the places we’d have to change!), which in turn contributes to making it harder to come up with better names because we’re already surrounded by average names that we need to stick with and navigate around (you’ll see an example of this soon). The things we name can even change meaning and functionality throughout development, which only adds to the problem.

The only way of completely avoiding this loop is to put the effort to come up with a great name beforehand. If that’s impractical, the most popular way I’ve seen of breaking this loop is to adopt placeholder or code ⁶ names and then go through the pain of renaming everything just once, after putting the effort (at a later time) to find a better name. The pain of renaming everything still exists ⁷ , but you’re making an effort of going through it only once.

But is there one overall great name for every situation?

Because we’re constantly naming things using words with other meanings, a name can be a reference to multiple entities, but an entity can also have multiple names. For example, we often overload names like selector, database, user, request, client and so on, and they will mean different things and point to different entities depending on the context. But we also often use multiple names to point to a single entity: user, player, or enemy could all refer to the same entity, with each name used depending on the context.

It can be irritating that in a world of so much determinism, it still makes sense to use all of these ambiguous names throughout our code, especially because the meaning of many things in the code is changing throughout development. So I can only conclude that there’s no overall great name for every situation. It will always be hard to come up with a great name, just like it is also hard to do the same outside the software world, so if one wants to prevent that positive feedback loop from occurring, we’re left with making it easy to change a name. Why is it hard to change names in the first place?

An IDE aids the programmer in renaming variables, methods, classes, and other code entities, but it’ll only do this if it understands the text it’s working with. In any modern codebase, it’ll never understand enough to cover everything you’d want renamed. Anything with multiple programming languages is already out (the tooling to deal with each language is different), not to mention renaming things across design documents, readme files, file names, and pretty much everything else people throw into code repositories nowadays. Because all this information is kept in memory, even bringing multiple tools doesn’t help, because they don’t talk to each other and they don’t share the same data.

Hell, I’d love to see IDEs even pretend that they can at least rename text inside code comments in the same language they’re already working with. To do so you usually have to resort to the generic find-and-replace, which is far from ideal with its multiple false positives.

Changing names is hard because we’ve siloed everything. The data and the software.

Example: how naming matters inside comments and outside the code

As I explored a bit in another post , when writing code we’re doing 3 things that are often in conflict with each other: talking to the machine, talking to other humans, and storing knowledge.

When talking to the machine, we’re often concerned with correctness and efficiency, and this pulls the code in a direction that many times doesn’t help us talk to other humans, which is when we’re concerned with reading comprehension and maintainability. We fall back to writing comments and other text outside the code to bridge the gap between both directions, and to make things easier, we refer to variables that are in the code.

At the same time, in most programming languages we describe the “what” and “how” with code. Take the following example (in Python):

elems = [1, 2, 3, 4, 5, 6, 7]

# Imagine that this comment is describing some complicated algorithm, or explaining a business process that is being implemented by the code below, and we want to refer to val1 in this comment as part of the explanation.
val1 = 0
for elem in elems:
  val1 += elem

# This code is doing the same thing, but the “how” is not hidden behind a function, and we can focus on describing “what” val2 is instead.
val2 = sum(elems)

val1 and val2 will have the same value when the code finishes running. However, the code around val1 is way heavier on describing how to calculate it, while the code for val2 is almost purely describing what val2 is. To get to the same understanding of what val1 is as in val2, we need to read the code for val1 and solve a small puzzle in our head, figuring out what the intention behind the code is, so we can then infer that val1 contains the sum of all elements in elems.

One could argue that all we need to do is neatly organise code in small functions so we can start working at a higher level, chaining a bunch of “what”-descriptions and keeping the “how” inside the functions, just like we did with val2. There are a bunch of older books in the literature with advice similar to this.

Anyone who has actually tried doing this with any sort of large codebase or complicated operation knows this rarely helps. Most functions aren’t like sum — they hide non-trivial code which can’t be easily summarised with a name. The complexity has to live somewhere, and that bleeds into naming as well. Chaining a bunch of these forces the reader to go into each function definition to understand what’s happening to the data.

Even at the higher-level scale we might start to see some control flow appearing throughout development, and eventually this too will be moved into a function to hide implementation details, which needs its own non-trivial name.

You bet that as the code evolves, some of these names will lose their original intention or meaning, and it’d be nice to be able to rename them and every other piece of non-code text that explains or refers to them.

Example: names gaining ambiguity, forcing workarounds

Let’s imagine that we’re building an online game. Players can perform actions, which get sent to the game server, are processed, and the resulting event is sent back to the players. Here’s some JavaScript function to process one such event.

function processGameEvent(event) {
  // Processes the consequence of a player action.
}

As we work on the game further, we discover that it’s important to transmit a player’s actions to all other players so each player knows what action led to the subsequent event, since it may not always be obvious from the consequences themselves.

What do we call the thing we receive from the server to process now? It could still be called an “event”, but that conflicts with the result of player actions, which was initially also called an event. The code below uses “event” to refer to two different entities and it’s very confusing what each one means without knowing the history of how the code evolved.

function processGameEvent(event) {
  switch (event.kind) {
    case 'playerAction':
      // Only displays the action a player took. The consequence will come afterwards.
      break;
    case 'event':
      // Processes the consequence of a player action.
      break;
  }
}

Peek into a possible future

Imagine if instead of pure plain text we had structured text that allowed us to attach data to parts of it. A very interesting thing we could do with this is to establish links between parts of text, which could even be in different documents. Attached to these links we could have constraints to help us keep the links up to date and/or serve as validation.

I did this for the examples above. I defined some links, but you’ll only see them once you click the button below.

Go ahead and click it (if you want the links to go away, just refresh the page). Some words in this page will get a dashed outline. This means they are linked to other words, and if you hover your mouse on any word you’ll see which ones are linked to it.

Purple/light purple links (the colour changes if you’re reading on light or dark mode) are purely informative links with no constraints or extra behaviour. Yellow/Brown links have an additional constraint that each part of the text in the link must have the same content. I also made it possible for you to click any word with an yellow/brown link to rename it.

I’m fairly certain there is something interesting with this solution, not only because it aims to remove barriers to changing names, but because it has broader applicability than just that, once you consider that links can have any sort of data attached to them.

But the purpose of this little demo isn’t to propose any sort of solution, but to encourage some thinking. Would it be possible to implement this in our current software world? What would it take to do that?

I don’t really want this to be implemented in our current world, but I find the thought exercise interesting, because it leads to other inquiries. Folks who have been chatting with me lately know I mostly submitted to throwing away decades of software “progress” and “innovation” because everything we create on top of our current abstractions instantly inherits copious amount of accidental complexity. I’m now exploring new things almost from scratch, in large part inspired by ideas and conclusions I’ve reached when going through these exercises.

But the important thing to take away is that changing names doesn’t need to be hard or impossible. It’s the inherited complexity, the silos that make it hard.

To be continued

Imagine if every tool had access to this structured data/linking, and if it was also part of not only code, but text documents, structured documents, spreadsheets, and other types of data that would benefit from this. Obviously in the current world this seems practically impossible.

If the mention of “link”, “structured text”, and establishing links between different documents didn’t make you instantly think of the web, well I hope you made that connection now. There are similarities between the web and what I wrote in this post, but there are also important differences that don’t make sense to go into in this post.

And when you think about code from a version control perspective and the structured text/linking stuff I wrote about in this post, one can also see how Git (and let’s throw the web back here for a second too) is far from an appropriate system for this kind of thing, although it is an extremely dominant tool in the current world.

These are both things I want to explore more, but will leave for future posts, but I still wanted to mention them here for readers who want to think more about this on their own.

Bonus: questions and answers

When I started writing this post, I asked friends what questions they’d ask someone reflecting on 20 years of programming experience. I said I’d try to fit answers to those questions in the post, but I couldn’t fit answers to all questions before going off on my rant, so this is a bonus section with the remaining questions and answers to them.

What do you wish you’d started doing at the start of your journey that is actually reasonable to expect someone with a few years of experience to do?

Read more code. Like, seriously, read more code. More than you think you should be reading. We focus too much on writing code when we start programming (it’s fun!), but we don’t spend enough time reading other people’s code. I’ve met an unreasonable amount of people into their programming careers who obviously didn’t read that much code either, so here’s my advice for people at the start of their programming journey or well into it: read more code.

What was your first “oh shit oh fuck I’m a wizard” moment with code?

I’m pretty sure there were moments before this that I just can’t remember anymore, but the earliest one I remember was writing scripts to automate things in Ultima Online. Having people pay me (in-game currency, but still money) for something that I was having fun doing and tell me that my scripts were great and kept them alive during PvP? I can’t picture a kid who wouldn’t think they were a wizard after that.

How do you decide when to dive deep into something?

I don’t decide when to dive into something, I always decide how deep I go. It depends on how much free time I have and how interested I am in the thing, but I’ll always try to learn something new by peeking under the curtains.

What’s your favourite way to restore inner peace / avoid burnout?

I spend time with my hobbies and do mechanical tasks that don’t require thinking, so I can focus my thinking internally. This is a variant of Hammock Driven Development , but for all things in life.

What does your work process look like with different types of projects?

Mechanically, first I set up a Nix flake, then I bring in development tools, use them to initialise things, and then start messing around. On an organisational level, I’ll almost always have a TODO file on the repository and figure out things to be done that way. Break big things into smaller things. On a directional level, unless there’s something that I already know needs my focus, I go with my gut feeling on what to work on at any point in time.

It’s very simple and sometimes too plain, but I like simple for this.