roncli.com blog

I give to you the complete. The informative. The functional. roncli.com version 2.0.

While it's been out here in beta for the last couple weeks, I formally released version 2.0 on GitHub earlier this evening, shipped it out to its new home in Azure, finished up all the remaining content changes I needed to make, and popped off.

OK, so the celebration was more subdued than that, but I tell you, this has been a journey. I've done so many things that are new, I don't even know where to begin.

roncli.com is the product of nine Docker containers. All of these containers run simultaneously on the same server to get the website to function.

Certbot - For keeping the SSL certs up to date.
Logger - A node.js application that logs Docker output to Azure Application Insights.
Nginx - This web server acts as a reverse proxy for the applications on the server.
Redis - Caching for the main website.
MongoDB - The main data store for the website.
MariaDB - The data store for the photo albums.
PhotoPrism - The photo album service that stores over 3,000 pictures used on the website.
ronc.li - The redirection service for roncli.com.
roncli.com - The main website.

roncli.com uses a whole bunch of tools to make it tick, including some mainstays like express. I also have written my own, including something called Hot Router, an express router that lets me create classes that are picked up by the router to determine what page to load when a certain URL is hit. It connects to a whole bunch of third party services for things such as my blog, my music releases, my coding projects, my gaming stats, and more. And, it has a super cool résumé page that I'd been wanting to do for a long time.

Being finished with this site doesn't mean it's time to stop, though. Next, I'll be doing a release of olmod for the Overload folks, doing some long overdue updates to the Overload Tracker and the OTL, and then it's on to moving all the Overload projects off of the Windows server and retiring it, finally.

There's still a long way to go, but I feel roncli.com was the biggest project that hadn't been moved yet. It was a complete rewrite from the ground up, which the tracker and the OTL will not be. Hopefully I can complete everything I want to do within the next few months and free myself up to do some stuff other than websites for once.

We'll see how that goes in the coming months, but until then, enjoy the new roncli.com!

Labels: Azure, Coding, Docker, Node, roncli.com

Sunday, August 29, 2021

A Low Bang to Buck Ratio
Posted: 2:08:00 AM 0 comments

The story behind Six Gaming is long, complex, and probably can't be covered in a single post. If I were to sum up what Six Gaming is today, the answer would be "probably should be dead, but stubbornly isn't". I knew this going into redesigning the Six Gaming website, yet I did it anyway. Why?

There's a number of personal reasons involved. My outside hope that some day everyone stops being super busy with life and commits to a podcast again, which is what the original community was built around after our WoW guild died. The only reason I decided to stop streaming it is because no one could commit to the schedule except for me. The website also has a Discord and Twitch bot that promotes streamers, hosting them on our Twitch page, which would be a great tool if the community was still active. However, it's not active, so the main reason I used to justify upgrading Six Gaming? Knowledge building.

Six Gaming is the first website that I've built that runs with a MongoDB back end. It's the second site I've built that uses Docker containers. It's also the second site I've built on my current generation of node.js website architecture, but the first time I've done it within Docker. The website uses Discord.js, Express, FullCalendar, the node.js wrapper for IGDB, the node.js wrapper for MongoDB, and the Twitch.js library that is being rebranded as the ridiculously-named Twurple. However, when I talk about my website architecture, I don't mean the libraries I'm using, but rather the way the website is put together.

I drew some inspiration from the now-defunct Rendr library. It was a node.js library that worked with Backbone.js to let you code a web site just once to render web pages on both the client and server side, making it easy to create single page applications. I was turned on to this library while working for the startup Sift back in 2013, and ultimately used it for my personal site roncli.com. Of course, as soon as I released that site, Rendr stopped getting updated, so I stopped using Rendr and started rolling my own.

The first website I used this new architecture with was the Overload Teams League. I didn't go too far with it, only making it so that there are views that can be rendered either client or server side. I didn't go as far as making it a single page application, that wouldn't come until version 2.0 of roncli.com. The back end architecture is your run of the mill MVC application, nothing too exciting going on. What made everything tick, however, was a custom-built router that I would eventually release to NPM called Hot Router. It's called that, because it has an option that lets you hot-swap controller files while the site is live while the application is running. That was super useful for debugging the first few sites created with it, but the hot swapping has become less useful now that I've gotten better at using Docker.

All of the above is setup for one of the more amusing issues I encountered while working on Six Gaming's website, and there were plenty. Being only my second Docker project, I won't talk about what I can only describe as "newbie mistakes". However, the biggest thing I found was what I term the memory leak from hell.

I discovered it when I was working on the Hot Router project. The gist of the problem is that for weeks after the launch of six.gg, I had a very slow memory leak that would break down the server after about a week. It drove me insane that I couldn't find it. The lengths I went to in order to find the leak were insane. First, I wrote my own calls to the docker.sock API, logging the metrics to Application Insights. That alone instantly doubled the cost of the server while I had the metrics active. The price you pay.

This led me to learn the memory leak was happening in the node.js Docker container. In order to find the memory leak, I had to connect the node.js instance inside Docker to Google Chrome's dev tools. Once I did that, I spent hours pouring over memory logs, slowly narrowing the problem to my shiny new router. Did making the router a module cause the memory leak? Did I screw something up porting the code over from being inline in the project to its own module?

No, the memory leak was there all along. I tried comparing Date objects to see if they were different. While you can compare to see if they are greater than or less than each other, trying to compare that they are equal or not equal actually doesn't work. This was causing Hot Router to always treat the controllers as if they were just hot swapped. It would delete the cache of the controller and re-require it. As it turns out, the act of deleting the cache and re-requiring it caused the memory leak. That, combined with the date comparison bug, resulted in a slow memory leak.

As I was fixing that bug, it dawned on me. I run the Overload Game Tracker, and that site had been suffering from a memory leak for over a year. It runs an early version of the routing code that Hot Router uses. Turns out I solved a very old memory leak by finding the leak in an entirely different application.

Anyway, Six Gaming's website has been humming along for a while. It was a lot of effort for not a lot of reward. However, this project taught me a ton about Docker, MongoDB, and more. And everything I learned from this is going into my most ambitious project yet... my own website! More on that in a future post.

Labels: Coding, Docker, Node, Rendr, Six Gaming