roncli.com blog: A Low Bang to Buck Ratio

The story behind Six Gaming is long, complex, and probably can't be covered in a single post. If I were to sum up what Six Gaming is today, the answer would be "probably should be dead, but stubbornly isn't". I knew this going into redesigning the Six Gaming website, yet I did it anyway. Why?

There's a number of personal reasons involved. My outside hope that some day everyone stops being super busy with life and commits to a podcast again, which is what the original community was built around after our WoW guild died. The only reason I decided to stop streaming it is because no one could commit to the schedule except for me. The website also has a Discord and Twitch bot that promotes streamers, hosting them on our Twitch page, which would be a great tool if the community was still active. However, it's not active, so the main reason I used to justify upgrading Six Gaming? Knowledge building.

Six Gaming is the first website that I've built that runs with a MongoDB back end. It's the second site I've built that uses Docker containers. It's also the second site I've built on my current generation of node.js website architecture, but the first time I've done it within Docker. The website uses Discord.js, Express, FullCalendar, the node.js wrapper for IGDB, the node.js wrapper for MongoDB, and the Twitch.js library that is being rebranded as the ridiculously-named Twurple. However, when I talk about my website architecture, I don't mean the libraries I'm using, but rather the way the website is put together.

I drew some inspiration from the now-defunct Rendr library. It was a node.js library that worked with Backbone.js to let you code a web site just once to render web pages on both the client and server side, making it easy to create single page applications. I was turned on to this library while working for the startup Sift back in 2013, and ultimately used it for my personal site roncli.com. Of course, as soon as I released that site, Rendr stopped getting updated, so I stopped using Rendr and started rolling my own.

The first website I used this new architecture with was the Overload Teams League. I didn't go too far with it, only making it so that there are views that can be rendered either client or server side. I didn't go as far as making it a single page application, that wouldn't come until version 2.0 of roncli.com. The back end architecture is your run of the mill MVC application, nothing too exciting going on. What made everything tick, however, was a custom-built router that I would eventually release to NPM called Hot Router. It's called that, because it has an option that lets you hot-swap controller files while the site is live while the application is running. That was super useful for debugging the first few sites created with it, but the hot swapping has become less useful now that I've gotten better at using Docker.

All of the above is setup for one of the more amusing issues I encountered while working on Six Gaming's website, and there were plenty. Being only my second Docker project, I won't talk about what I can only describe as "newbie mistakes". However, the biggest thing I found was what I term the memory leak from hell.

I discovered it when I was working on the Hot Router project. The gist of the problem is that for weeks after the launch of six.gg, I had a very slow memory leak that would break down the server after about a week. It drove me insane that I couldn't find it. The lengths I went to in order to find the leak were insane. First, I wrote my own calls to the docker.sock API, logging the metrics to Application Insights. That alone instantly doubled the cost of the server while I had the metrics active. The price you pay.

This led me to learn the memory leak was happening in the node.js Docker container. In order to find the memory leak, I had to connect the node.js instance inside Docker to Google Chrome's dev tools. Once I did that, I spent hours pouring over memory logs, slowly narrowing the problem to my shiny new router. Did making the router a module cause the memory leak? Did I screw something up porting the code over from being inline in the project to its own module?

No, the memory leak was there all along. I tried comparing Date objects to see if they were different. While you can compare to see if they are greater than or less than each other, trying to compare that they are equal or not equal actually doesn't work. This was causing Hot Router to always treat the controllers as if they were just hot swapped. It would delete the cache of the controller and re-require it. As it turns out, the act of deleting the cache and re-requiring it caused the memory leak. That, combined with the date comparison bug, resulted in a slow memory leak.

As I was fixing that bug, it dawned on me. I run the Overload Game Tracker, and that site had been suffering from a memory leak for over a year. It runs an early version of the routing code that Hot Router uses. Turns out I solved a very old memory leak by finding the leak in an entirely different application.

Anyway, Six Gaming's website has been humming along for a while. It was a lot of effort for not a lot of reward. However, this project taught me a ton about Docker, MongoDB, and more. And everything I learned from this is going into my most ambitious project yet... my own website! More on that in a future post.

Labels: Coding, Docker, Node, Rendr, Six Gaming

0 Comments

Post a Comment