I have an iPad Air from 2020, and it only has 64 gigabytes of storage. The overwhelming majority of the storage is taken up by comics. In response, I turned to OPDS. A basic XML feed that allows a reader to fetch books from a server. Officially it doesn't support streaming, but there is an unofficial extension called PSE. This is vital for comic readers to keep space usage down.

There are very few OPDS servers that support PSE, and even fewer that are lightweight. So I wrote my own. Inkwell is written in Go, and serves an OPDS feed based on a folder of CBZ files that contain a ComicInfo.xml file. It's fairly simple but effective.

However I've grown to be frustrated with OPDS and it's limitations.

• XML is incredibly verbose. A simple feed of a few hundred books can be multiple megabytes. Yes, I am aware that OPDS 2.0 uses JSON instead, but that doesn't solve the other problems.
• Bad searching. The search function of OPDS is complicated, and is based on OpenSearch. This is dumb is complicates things where it could be a simple REST API.
• Lacking metadata. OPDS only supports a handful of metadata tags so readers are fairly limited on what they can show to their viewers. No support for right to left reading, no support for whether this comic is black and white.

OPDS is clearly a standard built for an older era of computing.

Enter the Open Library GraphQL Feed. OLGF. This is a work in progress schema for serving a similar role to OPDS, but designed to be more informative, flexible, and efficient.

GraphQL was the choice because while it's more complicated to implement, it offers so many benefits.

• No over/underfetching. Since clients control what data is requested, they can limit their requests to just what they need and not the whole API.
• Strong typing is great. It reduced the potential for datatype mix ups even in weakly typed languages like Javascript and Python.
• Subscriptions! No longer does the client need to constantly refresh data from the server, instead it can subscribe and the client will be notified when there is new data to fetch.

This is how APIs should have always been. OLGF will essentially merge ComicInfo.xml and OPDS 2.0 into a new standard that should easily handle even the largest library.

As powerful as GraphQL is for serving an API, it cannot serve data from a file, so a separate file server will need to exist to handle that. In order to support streaming, it'll also need to be able to open, read, and serve files from CBZ archives.

Right now I'm working on the schema. Following that is the first rough rewrite of Inkwell to support the new format. This rewrite is not going to be the most efficient, but should act more as a proof of concept. Once that has been verified to work, then the real work begins. Writing Inkwell from scratch to support OLGF, no legacy code, all written in clean and simple Go.

The server should be simple enough to implement. Maybe a few weeks to months to iron everything out.

Now onto Inkling. This is going to be the front-end. I'm planning to write it in Dart and Flutter to make a cross platform reader. Ideally this means one code-base for web, desktop, and mobile.

This is probably the part that will take the longest as I am not very familiar with Dart or Flutter. Front-end development hasn't been my focus for a while.

So yeah. That's the plan. All the source code will be publicly available as always, and hopefully this turns into enough of a success that other readers adopt OLGF.

Solder just isn't as big of a deal when modpacks are distributed via metadata files these days. Not to mention the insane level of sketchy ads on Technic's website.

So over the next few weeks, I'm gonna be migrating my modpacks to Modrinth. This will take some time though because Modrinth has a moderation system, and I'll need to deal with that, but it's coming.

Perhaps in the future I'll take another swing at a white-labelled modpack launcher like I was planning for Canvas and Palette, but that is not going to be for a while. There is much to consider with something like that, and the Minecraft authentication and launch process has gotten more complicated over the years. So, not right now.

Anyways, go play those modpacks if you want. The Magic Pack is quite old by today's standards, but I consider 1.7.10 the pinnacle of magic mods. Thaumcraft 4 was truly a great experience and I look forward to what CoFH does with TC7.

A bit of backstory: I run my own servers as a homelab. I prefer that to cloud providers as it's more affordable long term, and offers a ton of flexibility. My homelab consisted of a HPE ML350 Gen9, a HPE DL380 Gen9, and a DL360 Gen9.

Before anyone says anything, I am aware of the issues that plague HP rackmount servers, but these were bought second hand for incredibly cheap, and with the power of a modded iLO4, the fans can be quieted down significantly.

As for the OS, I use Unraid. The flexibility it offers when it comes to disk arrays is really nice, and the stability of Slackware is excellent. Unfortunately, the docker management system is a pain. It heavily encourages the more "user friendly" approach of what is essentially an app store.

Finally, for networking, everything was running through Cloudflare Tunnels. This is a very useful service that allows you to expose a web app through a reverse proxy without needing to expose any ports on your network. Unfortunately, it has one limitation that is crippling for me. The 100 MB upload limit.

So we come to a few months ago when I decided that enough is enough. I need to redesign this whole system. First thing was to decommission the ML350. It is a great machine that supports tons of PCIe devices, but it is handicapped by only supporting 8 SFF drives and the cages to extend capacity are outrageously expensive on the second hand market.

Second was to upgrade the DL380. The one I have supports 12 LFF drives which offers a ton of room for upgrades. So, I ordered 2 refurbished 12 TB drives. For the time being, this server runs all the services that require big storage such as media servers, backup systems, etc.

Now we get to the DL360. This server has become the gateway, and runs much of the smaller services that don't require terabytes of storage. This server runs a copy of Caddy and handles all proxying to the various other services, but this introduced a hiccup: communication. To address this, both servers were set up as a Docker Swarm. This allows access to overlay networks, which solve the communication issues. Instead of having to bind ports, or use a macvlan to allow for inter-container communication, I just use an overlay network.

With that, the final issue was Docker management. Unraid's Dockerman doesn't support compose at all, and I considered Portainer but I never liked the UI. Enter Arcane. Arcane is a Docker management system written in Go, and it met my needs perfectly. No abstraction, just a compose file and an environment file. Arcane doesn't support Docker Swarm yet, but that isn't an issue as I'm not running any containers via Swarm. Swarm is just being used for the network.

It took a while, but all of this was set up. I have no doubt that there are better ways to do it, but this works for my use case. It's easy to work with. Deploying new services is as simple as writing a compose file. Updating containers is as easy as clicking a button.

In the future I'd like to build a large SAN and just have a cluster of energy efficient servers doing the heavy lifting, but with the current cost of storage and memory being what they are it's very much out of the budget. Once prices do come down, who knows.

Lately I’ve been on a BG3 kick with a friend, but once that is done I’m making my way back to Minecraft. I want to set up another modded public server, but the idea of having to deal with the shitshow that is Technic is anything but appealing.

That being said, we are finally getting good weather in Canada. So I’m going to be spending a fair amount of time outside, especially getting more exercise to counteract the concerning amount of weight I put on over the winter.

So updates might be a bit sparse.

I needed something else to fill my free time. Elite Dangerous.

Now, this game is not a new one to me. I started playing back in 2019-2020, but I stopped. Five years later, and with a new HOTAS, I am enjoying it a lot. Unfortunately like all games, it has given me a two new projects.

EDAS and EDIS. The Elite Dangerous API Service, and the Elite Dangerous Informational System.

More info about them will be given on their own page, but one of them is finished. At least a first version is one.

Anyways, I’ve been enjoying the game. Recently I picked up the Odyssey DLC, and while I do think that there is definite some polish that it could use, it’s not bad. If the game is on sale, grab it.

Be warned. It’s a lot slower of a game.

To try to alleviate that, I'm switching projects for a bit. Cue, OPSE-PSE Viewer. A (hopefully) simple viewer for comics and manga, cross-platform, and written in Dart, using Flutter.

I'm also struggling to stay on task, because I've been playing some Minecraft with my brother, and I've gotten sucked into the Create rabbit hole. One major idea I've had is making a mod that is similar to Tinkers' Construct, but integrated with Create.

Tinkers' Construct is one of the OG classic mods, but it's slowly dying. The code base is ancient and frankly the entire mod needs a total rewrite to take advantage of modern Minecraft, as well as seeing a multi-loader release.

That isn't gonna happen any time soon.

However, the idea of constructing tools and weapons from individual parts is still possible. Modern Minecraft versions offer the code and tools that would allow a mod like this to be much more efficient.

There is a Fabric port of TC, called Hephaestus, but it is a literal port. Very little code has been rewritten to be more efficient.

This is where my mod would come in. It would be designed as a spiritual successor to TC, and will be integrated in the Create ecosystem. Mixers, fluid tanks, deployers, and spouts already provide the majority of what is required to get a mod like this functional.

All I would need to implement is the tool parts, the tools, and the casts, including a casting table. The casting table will be a modified depot, where the center is inset to allow a cast to be inserted. A spout would fill the cast, and a funnel could extract the part.

The main component I want to focus on is the casting and metalworking portion of the mod. Of course I still intend to add additional crafting that would allow wooden tools, bone tools, etc.

All of this is just kind of an idea though. No idea if I'll ever do it, especially since I lack any pixel art talent. That turns out to be a major hurdle.

After figuring all that out, I ran into roadblock after roadblock because of my lack of experience with Electron. Specifically, what the hell any of the standard ways of storing data is, running shell commands, etc.

So, today I went back to Wails and Go. The productivity has been so much better. Turns out, it's just better for me to stick to the languages that I know.

The biggest problem I faced was the mess of typing. Due to a bunch of stupid stuff regarding Vue, Electron, Vue Router, Tailwind, and such, Typescript was off the table. This means weakly typed JS is all I have, and that was not conducive to consistency.

I've rewritten the entire authentication system in Go, and holy shit is it so much better.

So, yeah. Shout out to GoLand.

The choice to go with NodeJS is also reinforced by the existence of another project: Helios Launcher. I'm planning on ripping that launcher apart, learning exactly how it works and rebuilding it with the features I want.

The biggest issue with Helios is that you must compile the launcher for your specific modpack. The distribution files are part of the code. This is not ideal.

Canvas will have the ability to fetch JSON files, which will have all the information about that modpack, including styling configuration, mods with their CRC32 checksums, and more.

Currently this project is on hold, as while I was trying to unravel the nightmare that is Microsoft OAuth, and Minecraft's authentication system, I ran into an unexpected problem. Minecraft now has a whitelist for their API to reduce phishing. So, now I wait.

Hopefully in the next week they approve my project, as without access to their authentication system, I will not be able to go any further with this project.

Canvas will allow modpack developers to include a JSON file for simple skinning and perhaps in the future support for custom HTML and CSS.

There is a lot of features I'd like to include, such as RSS feeds for modpack announcements, allow versioning based on a remote JSON files, and support for Modrinth modpacks. Another major feature is... URIs for simple one-click installation.

The major hurdle is deciding which language to write this in. To allow for custom pages, it kinda restricts me to a web based UI. The major candidate is Wails written in Go. Of course the fallback is Electron in NodeJS.

Wails is looking more appealing as it utilizes your operating system's WebView, which will result in a much smaller binary with no Chromium to deal with. The big issue is that Go doesn't have a library for authenticating with Microsoft, so I'll need to write the whole system myself...

As for what Palette is, think of it like my own version of Technic's Solder. A self-hosted system for delivering updates using hashes to validate which files require updating.

This will definitely be written in Go. Gin is incredibly performant, and Go compiles into machine code for high speed and concurrency. Thankfully, Palette is just a simple REST API, doing CRUD operations on a database, and some basic file management.

I've chosen to not integrate a web interface with Palette, I want it to be as lightweight as possible. I'll probably write a basic program like Solder Packer to interact with the API.

Crucially, I want this entire ecosystem to be self-sustaining and decentralized. I want to be able to put this into the world, and not have to maintain any infrastructure.

I guess we'll see how this goes. It'll definitely be a long road ahead.