Eight months ago, I wrote on The Big Picture, a summary of my initiative to get off of my Azure Windows VM and on to Linux VMs running Docker containers with Node.js websites backed by MongoDB databases. It's a meta-project of sorts, and it has taken a certain mental discipline to keep at it.

The Trax in Space 1 archive was easy to port, and since porting it I've learned that I didn't need to restrict downloads so much on it, so I increased the download limit. Six Gaming was full of challenges, and even today I find things I have to change with it because I didn't do it right the first time. I'm currently working on rebuilding this site and have been for several months now, and I'm itching to get back to the Overload-related projects, and to move on from this whole thing.

I said this back in February:

This is, of course, a multi-part project that has taken on a life of its own in recent months, and it's one I am enjoying greatly so far. It's really expanded the boundaries by which I am able to operate websites and related online services.

Boy, that's an awful lot of enthusiasm. Sigh. We were so young back then.

Since then I've worked through major bugs, tackled big architectural design questions, learned a hell of a lot about MongoDb... and have bored myself to death with this project.

Look, it's not like I knew this was going to be a huge project. Just the opposite, I knew what I was getting into. I remember how long it took me to complete roncli.com the first time, and I'm still looking at coming in at well under a fifth of the time it took me to complete it the first time around. I've learned a lot in 7 years, and it's showing with how fast I'm getting stuff done. But when I take a look at the big picture, it just seems so... slow.

I am to the point I want to work on something else. I want to update the OTL for season 7 and beyond. I want to add some amazingly cool things to olmod. I want to make games of my own. And, I want to get back to regular streaming again.

But every time I sit down to code, I am reminded of the plot: get off that monolithic Windows VM, and you'll have the freedom to do what you want and when you want when this is all done. So I go heads down, jam my EDM playlist, and spit out another couple of files on whatever I'm working on, making a bit of meaningful progress until I get bored and go for a walk, play some games, or write a blog post.

I'm sure a lot of this stems from the fact that I've been essentially stuck at home for 19 months. I'm bored to death with that, too. No train rides to The City, and very few outings outside of the usual Friday routine of dinner and groceries. While 2021 has been infinitely better than 2020 when I didn't get out at all, I can't shake the feeling this year that I'm accomplishing very little.

Of course, my GitHub contribution graph will tell you otherwise, and when I look at things logically, I know I'm getting stuff done. I just wish I could fast forward to the end of the big picture so I can move on to the next big thing... perhaps something a bit more exciting.

Labels: Azure, Coding, Overload Teams League, roncli.com, Six Gaming, Trax in Space

Sunday, August 29, 2021

A Low Bang to Buck Ratio
Posted: 2:08:00 AM 0 comments

The story behind Six Gaming is long, complex, and probably can't be covered in a single post. If I were to sum up what Six Gaming is today, the answer would be "probably should be dead, but stubbornly isn't". I knew this going into redesigning the Six Gaming website, yet I did it anyway. Why?

There's a number of personal reasons involved. My outside hope that some day everyone stops being super busy with life and commits to a podcast again, which is what the original community was built around after our WoW guild died. The only reason I decided to stop streaming it is because no one could commit to the schedule except for me. The website also has a Discord and Twitch bot that promotes streamers, hosting them on our Twitch page, which would be a great tool if the community was still active. However, it's not active, so the main reason I used to justify upgrading Six Gaming? Knowledge building.

Six Gaming is the first website that I've built that runs with a MongoDB back end. It's the second site I've built that uses Docker containers. It's also the second site I've built on my current generation of node.js website architecture, but the first time I've done it within Docker. The website uses Discord.js, Express, FullCalendar, the node.js wrapper for IGDB, the node.js wrapper for MongoDB, and the Twitch.js library that is being rebranded as the ridiculously-named Twurple. However, when I talk about my website architecture, I don't mean the libraries I'm using, but rather the way the website is put together.

I drew some inspiration from the now-defunct Rendr library. It was a node.js library that worked with Backbone.js to let you code a web site just once to render web pages on both the client and server side, making it easy to create single page applications. I was turned on to this library while working for the startup Sift back in 2013, and ultimately used it for my personal site roncli.com. Of course, as soon as I released that site, Rendr stopped getting updated, so I stopped using Rendr and started rolling my own.

The first website I used this new architecture with was the Overload Teams League. I didn't go too far with it, only making it so that there are views that can be rendered either client or server side. I didn't go as far as making it a single page application, that wouldn't come until version 2.0 of roncli.com. The back end architecture is your run of the mill MVC application, nothing too exciting going on. What made everything tick, however, was a custom-built router that I would eventually release to NPM called Hot Router. It's called that, because it has an option that lets you hot-swap controller files while the site is live while the application is running. That was super useful for debugging the first few sites created with it, but the hot swapping has become less useful now that I've gotten better at using Docker.

All of the above is setup for one of the more amusing issues I encountered while working on Six Gaming's website, and there were plenty. Being only my second Docker project, I won't talk about what I can only describe as "newbie mistakes". However, the biggest thing I found was what I term the memory leak from hell.

I discovered it when I was working on the Hot Router project. The gist of the problem is that for weeks after the launch of six.gg, I had a very slow memory leak that would break down the server after about a week. It drove me insane that I couldn't find it. The lengths I went to in order to find the leak were insane. First, I wrote my own calls to the docker.sock API, logging the metrics to Application Insights. That alone instantly doubled the cost of the server while I had the metrics active. The price you pay.

This led me to learn the memory leak was happening in the node.js Docker container. In order to find the memory leak, I had to connect the node.js instance inside Docker to Google Chrome's dev tools. Once I did that, I spent hours pouring over memory logs, slowly narrowing the problem to my shiny new router. Did making the router a module cause the memory leak? Did I screw something up porting the code over from being inline in the project to its own module?

No, the memory leak was there all along. I tried comparing Date objects to see if they were different. While you can compare to see if they are greater than or less than each other, trying to compare that they are equal or not equal actually doesn't work. This was causing Hot Router to always treat the controllers as if they were just hot swapped. It would delete the cache of the controller and re-require it. As it turns out, the act of deleting the cache and re-requiring it caused the memory leak. That, combined with the date comparison bug, resulted in a slow memory leak.

As I was fixing that bug, it dawned on me. I run the Overload Game Tracker, and that site had been suffering from a memory leak for over a year. It runs an early version of the routing code that Hot Router uses. Turns out I solved a very old memory leak by finding the leak in an entirely different application.

Anyway, Six Gaming's website has been humming along for a while. It was a lot of effort for not a lot of reward. However, this project taught me a ton about Docker, MongoDB, and more. And everything I learned from this is going into my most ambitious project yet... my own website! More on that in a future post.

Labels: Coding, Docker, Node, Rendr, Six Gaming

Monday, February 08, 2021

The Big Picture
Posted: 4:16:00 PM 0 comments

Sometimes, it's a good idea to take a step back and look at what you're doing and what you've done, and ask, "Can this be better?"

I recently did this with my Azure Windows VM. For a while, I've been just loading up any new website or venture I've created onto my VM. This, however, has proven to be problematic when it comes to some of my more recent projects, including the OTL and the corresponding Overload Game Browser. These two projects are by far my most used websites on the server, and they continually push the limits of what this VM can do. Timeouts have become more and more frequent, and as more and more data piles into the database, the problem is just going to keep getting worse.

So I asked the question, "Do I need to be on a Windows VM?"

Some years ago, the answer would have been "yes". I was running .NET Framework 4-point-something, and had a lot of Microsoft-specific things on the system, including a Microsoft SQL Server. Now, however, every site I host is written in Node.js. The only thing remaining that requires anything Microsoft is the SQL Server.

So I asked the question, "Do I even need SQL Server?" I don't think I do. MongoDB exists, and I have been figuring out how to work with that for some time.

As such, I've begun a massive project to try to move away from this Windows VM and retire it permanently. For a while, I wasn't sure how I was going to do it, but as part of a learning course pilot at work I picked up Docker. My goal is to move every project that I have on that server into a group of Docker containers and run them on their own Linux VMs. Linux VMs are much cheaper than Windows VMs, and if something starts running out of resources I can just up the VM size accordingly.

So what is going to move?

The Trax in Space 1 archive, with the near 14 GB of files in an Azure Storage account.
Six Gaming and its Discord and Twitch bots, except completed and not just a placeholder website.
roncli.com and the ronc.li redirect service, rewritten to not use RendrJS. Thanks a ton for dying on me.
The Overload Teams League, The Overload Game Browser, and the Discord bot that controls my Overload game server farm.

This is, of course, a multi-part project that has taken on a life of its own in recent months, and it's one I am enjoying greatly so far. It's really expanded the boundaries by which I am able to operate websites and related online services. In coming posts, I will talk about each project separately, and what each site's status and future is.

Labels: Azure, Coding, Overload Teams League, roncli.com, Six Gaming, Trax in Space

Tuesday, January 19, 2016

Report Card: 2015
Posted: 12:38:00 AM 0 comments

I often do a "Year in Review", but this year I thought I'd change it up a bit. Last year, in addition to reviewing what happened the previous year, I added some goals for 2015. This year, instead of just reviewing what happened last year and adding the goals, I figured I would also add in a kind of "report card", where I grade myself on how I did towards those goals from the previous year.

I'll be honest. These goals stayed with me all year long. When I thought about what I want to do, I compared that to my goals. Having goals I actually thought about has made me learn a bit more about myself than I knew before.

So, without further ado, here's the 2015 report card!

Music

2015 Goal: Complete and release full versions of "All In My Head" and "Paper"

There's no beating around the bush here, neither of these goals happened. In fact, I was musically silent for the first half of the year. I did, however, come out with three different pieces of music, but none of them were original works.

In 2015, I started a podcast with Dr. Zing and Morfod called the Self-Destruct Sequence Podcast, which is a podcast all about Descent. Dr. Zing kind of dropped off the face of the Internet a few months ago, but Morfod and I picked it back up late last year, and have been having fun with the episodes this year. I mention this in the music section because when we were putting the first episode together, we wanted to have some Descenty music, and I delivered in a big way.

Our title music, Self Destruct Sequence, is a remake of the level 1 music, while our outro, Mars Processing Sequence, is a remake of the level 8 music. Both had electronic elements, and two guitar tracks each that I performed myself. This is the first time I've actually been happy with guitar tracks that I've created, especially the lead guitar parts. Hopefully I can take my experience from this and translate that into better guitar tracks in my music in the future.

The third piece was something I couldn't get out of my head, a piano cover of the song "The Wight to Remain" by Danny Baranowsky, which is best known as the music to Zone 4 Level 3 in the game Crypt of the NecroDancer. This has always been a beautiful track to me, and could be the launching point of a piano-only soundtrack to the game, if I get so inspired to complete some day. I even put the song's score up on flat.io if you so desire to try to play it yourself.

So while I kept the dust off of my music, I didn't do what I set out to do. At all. I tried to work on "Paper" once, but couldn't get any inspiration for the one part I still don't have a working idea for. Maybe that will change in 2016, but unfortunately, this gets poor marks this year.

2015 Grade: D

2016 Goal: Complete and release full versions of "All In My Head" and "Paper". For real this time.

Coding

2015 Goal: Finish the updated roncli.com site

Well, you're looking at it, baby. I'm still quite proud of this accomplishment, and it really took a load off my shoulders last year when I completed it. It's even gotten a few updates since that fixed some bugs and added (and, sadly, removed) some features.

So with the main 2015 goal accomplished, what else have I done in 2015? A quick glance at my GitHub profile shows I've contributed to Descent's DXX-Retro source port (this was the mouse fix), and started a series of Descent 3 tools written in node.js, including the rebirth of Descent DSN. However, in recent months, coding's not exactly been a priority to me.

The main project I have going right now is getting Six Gaming's official website off the ground. It is going to be started pretty much directly from roncli.com with a Six Gaming theme. At first it's just going to be a static page with links pointing to most of our presence on social media. Later, it will include episode-specific links for the Six Gaming Podcast, and the final version will be a full fledged social site where people can come to find other people with whom to play games they enjoy.

So 2015 was good, and 2016 has a clear goal. Looks good to me!

2015 Grade: A

2016 Goal: Finish the Six Gaming site

Gaming

2015 Goal: Grow Six Gaming into a bigger and more general gaming community

So at the beginning of 2015, I had this great idea to make Six Gaming a big community, not just for World of Warcraft, but for all the games we play. Over the years, we found that many of the gamers in Six Minutes To Release enjoyed other games as well, so it made sense to want to cater to that while keeping the close-knit community we've created from the core members of the guild over the previous 7 years.

Two major events accelerated this goal to fruition. First, Showsan got the jump on me. When he returned from his service in the military, he came in and lit a fire under everyone. He basically rebooted the Six Gaming Podcast, gave it a new format and new life. Since then, we've added a ton of cohosts that bring their own unique perspective on the games that we are playing. The Podcast has stayed fresh and has been enjoyable because of it, and we recently completed our 100th episode, which we call... Episode 48. Don't ask.

Second, Warlords of Draenor happened. What a bad expansion! I would rehash what I've already written, but suffice to say that not much has changed, and on top of it we're looking at another 6-8 months until Legion, which will make a 12-14 month content drought, similar to the one at the end of Mists of Pandaria, and exactly what the devs previously said they didn't want to do. Add in their stance on flying, and I honestly couldn't tell you if I trust Blizzard to get WoW right anymore. Legion is their last chance to get it right for me, and until then I am probably going to be doing some different things.

Like Descent. Descent: Underground made it known that Descent still exists for a lot of people, and despite the issues they are having (a topic for a future blog post), I am genuinely excited for a release of a new Descent game, even if it's not by the same people who gave us the first three. Naturally, a new Descent game has raised interest in the original games, and Descent 1, 2, and 3 have all seen an uptick in activity.

Case in point is the Descent Champions Ladder. It went from 1,408 games played in the fall 2013 season, down to 938 games in spring 2014, and down further to only 504 games in fall 2014. At the end of that season, Descent: Underground was announced, and the spring 2015 season saw a massive ressurgance with 2,218 games played. The late 2015 season, which was only 3 months, had 1,416 games played, and not even 3 weeks into early 2016 we have 510 games played.

Playing Descent again has been awesome, and has inspired me to learn a new control scheme. I used to be all keyboard, and now I do mouse and keyboard. I play at the high bronze/low silver level on the DCL currently, but I feel I have a lot to learn about the game both mechanically and philosophically.

I've also been involved with the Crypt of the NecroDancer community, racing most recently in CoNDUIT 7, and will be playing again on CoNDUIT 8 on the 29th. I'm not that great, which is why I'm in the "junior" league, but it's still a whole lot of fun getting some competition in. I still need to complete story mode and an all characters run to unlock Coda (I have no false impressions that I will BEAT Coda, mind you), so there's still plenty of room for improvement. I'm mainly known for having the Bard Deathless world record at 112 wins, and have performed fairly well at deathless mode with a few other characters as well.

I've also done a ton of streaming on Twitch. While I've stopped playing League of Legends, I still stream things such as my DCL games, Crypt of the NecroDancer runs, and even some Sublevel Zero speedruns. Basically, if I play it, it ends up on Twitch at some point.

Gaming has become a central part of what I do. I play a wide variety of games, involve myself in the Six Gaming, Descent, and Crypt of the NecroDancer communities, do two weekly podcasts involving games, and stream many times a week. I don't see this slowing down in the near future, so the only thing to do in 2016 is embrace it.

I have plans to start a regular Descent-themed competition show on Twitch, and am most interested in growing my stream large enough to get partnered. I don't think partnering will happen this year, but I at least want to get started. I've played with some ideas for growing my YouTube channel as well, but nothing's really stuck, but I'll keep trying.

2015 Grade: A+

2016 Goal: Substantially increase my streaming presence on Twitch

Life

2015 Goal: Get back to the Bay Area at least once

I think I nailed this one, at least in text. However, my experience thus far has been... less than ideal.

So instead of taking a trip to the Bay Area, I actually MOVED out here. In October, I packed my things, drove West a few days, and settled into the Belmont area. Central to everything, I've done... nearly nothing since I've gotten here. Sure, I've ventured to Pier 39 in the city, finally ate some Iguanas down in San Jose, and have occasionally taken in some of the area's scenic routes, I've yet to really do something... defining when it comes to the Bay Area. Seriously, I've seen more of the Bay Area the last time Kathy and I came out here in February 2013.

This is compounded by the fact that many of the people I knew and hung out with here are gone. One of the reasons I gave for moving out here was to be close to people I knew and give myself opportunities to "get out" every so often. I have done that a grand total of zero times since I got here.

There have been several reasons why I haven't really capitalized on my new living situation. The first is that I work remotely, yet am wanting to work locally. My job in Houston allowed me to work remotely when I told them I was moving and giving what I thought was going to be my two weeks. While the remote working part of the job has been going okay, I know through my experience with Sift that this isn't something I want to do forever. I have been really slow on advancing this agenda, however.

Second is that I've found myself in somewhat of a funk recently. I can't really put my finger on why, just that I find that I don't have the energy to do a whole lot since just before Christmas. My focus has been poor in coding-related projects, and I don't really feel that I've "settled in" very well. I don't at all regret making this move, in fact I think it's the best thing I've done for myself in a long time. However, I need to really break out of whatever it is that's causing me to, for lack of a better description, lose time doing nothing.

So, yeah, I made it out here, but... what next? Well, the first thing is going to be transitioning to a local job. Even before then, though, I need to find a way to get more active again. I have a plan, it's just acting on it - or more accurately getting over what's preventing me from acting on it - is proving challenging.

I hate these lethargic, stagnant days. I've had them before and have gotten past them, and have no doubt I will again. But the time until that happens can be downright depressing some days.

Oh, and there's this house thing. Yeah, we want a house. Soon.

2015 Grade: C

2016 Goal: Get more active, find local employment, and begin planning moving into a house

Overall

I've had an up and down year for sure. I recognize that my strongest point currently is with gaming, and recently that has been the outlet I go to in order to unwind and have a good time. I also find enjoyment in coding, but need to find a way to motivate myself to finish projects like I did with roncli.com. While my music still remains on the back burner, I am doing my best to keep sharp and avoid going flat... okay, I'll stop with the music puns! I recognize that I need to make some changes in order to improve my quality of life in my new surroundings, and am hoping that I can get those taken care of quickly.

2015 Overall Grade: B

Labels: Coding, Crypt of the NecroDancer, Descent, Descent Champions Ladder, Descent: Underground, Gaming, Life, Music, roncli.com, Self-Destruct Sequence Podcast, Six Gaming, Six Gaming Podcast, Twitch, Year in Review