I think you'll find it's a little more complicated than that

The sun is shining, the BBQ is sizzling, and it's Friday — which means it's time for your weekly dose of "only-mildly-unhinged" (according to one subscriber) updates about ActivityPub in Ghost. Welcome back.

First, some house keeping. We know lots of you enjoy the build log and the behind-the-scenes nature of this newsletter, but we also understand that not everyone is interested. Some of you simply signed up to get access to this new feature when it's available. We've fixed this with new email preferences to your account area.

If you prefer to only be emailed about ActivityPub when there's something to test, without the whole song and dance, you can opt-out of the newsletter there. If you want to keep receiving the build-log newsletter, there's nothing you need to do.

With that said, we shall now proceed with this week's song and dance 🕺

What's new with ActivityPub?

After last week's database debacle, where it was ultimately decided that just-1-more-lane was off the table (ha), we stepped back to have a bit of a re-think.

Initially we toyed with the idea of prototyping with a dedicated SQLite database, but that idea was quickly nixed. Our production servers use external file systems, which would either introduce too much latency or too much complexity if we started throwing SQLite around.

At the same time, one of the challenges with our initial back-end prototype so far has been (as mentioned previously) that we're trying to build something entirely new, without causing any stability issues for existing users.

In the early days of Ghost we shipped quickly, broke things, and then fixed them when a blog post wasn't loading or something. Nobody got very upset. These days Ghost is responsible for $25M+ per year in revenue for publishers, and when things break, people care a great deal.

In last week's comments, Matt Cameron said:

Regarding delivering the persistence, why not just offer a separate service/API for this that can be tied back to different data store? ... That way the core backend devs would maybe grumble less, but then I’m sure you have thought about all this.

As it turns out, we had not thought about all this.

For those of you thinking "well duh, a service-based architecture makes perfect sense, have you lost your mind?" — First of all, yes. Second of all, there's more nuance here than you might imagine, and it's directly related to why we've otherwise resorted to "just one more table" for such a long time.

When you build an open source product, a critical factor that determines adoption is how easy it is to self-host. It is, quite possibly, the single biggest predictor of success. WordPress got this very right, and it's a big part of why it still occupies such a large part of the market today.

Unfortunately, most modern apps are not straightforward to host. They're generally made up of multiple apps, servers, services, data stores, specific providers, proprietary external APIs, and often a whole other system to orchestrate everything. That's not a big problem for a centralised product which only has to be hosted in 1 place by 1 team, who built it themselves... but it's a fairly major problem when you're expecting a product to be installed and adopted by developers all over the world who have never used it before.

With Ghost we've always done our absolute best to keep the architecture as a simple monolith that could be hosted on a single server, so that anyone can self-host.

We've avoided multiple data stores, proprietary technology, and paid external dependencies as much as possible, in service of making the software accessible to as many people as possible.

Within the JavaScript ecosystem, in particular, this has turned out to be a very uncommon approach — but it's a big part of how we got to where we are now.

So after doing this for 10+ years, we still have an app that can be installed and self-hosted easily; indeed, the majority of Ghost sites in the world are. But, we also have a very large app with over 70 database tables and great deal of overlapping concerns.

What has become clear as we've gotten deeper into the weeds with ActivityPub is that there's just no way we can cram this, too, into the existing app. There's no room at the inn, and this is now a forcing factor for us to branch out and try some new things.

Like chocolate sprinkles and a fist-full of psilocybin.

Or, you know. Docker Compose.

We're now extracting what we've learned from the back-end so far, and exploring how to develop ActivityPub support for Ghost in a new, separate service - as Matt suggested. We're also exploring existing open source packages out there to see if there are any we can leverage to get a head start.

Now, not to make this a competition or anything, but here's how the community suggestions leaderboard is stacking up so far:

Just saying.

Answering your questions:

They may not be FA but they certainly are Q.

Hopefully, we're going to be able to regularly delete caches, articles, and images coming from ActivityPub feeds. Otherwise, we'll need to constantly upgrade our storage limits and server performance, right?

The single biggest concern we've always had about ActivityPub is performance. The second is spam. Unfortunately the spec doesn't really consider these problems, which is why it's already a common issue across multiple platforms. These are two areas we think and hope there will be big benefits from collaborating with other platforms and sharing resources. In short, this is very top-of-mind.

The Ghost AI newsletter author is shaping up nicely! Will we get access to the humor module once they've added the author to core Ghost?

I apologize for the confusion, but I am not aware of any "Ghost AI newsletter author" or "humor module" that is being developed or added to my core capabilities. As a large language model, I don't have insight into the internal development plans of Ghost or any other company.

Wait so did you actually build anything this week, what the hell?

Fuck. Always get carried away with these newsletters. Yeah! Outside of big-brain thinking we also continued to iterate on the client app, which is well on its way to supporting likes and an activity stream.

See you next week!