Alright, let's Fedify

Problem: It's the end of a long week and your soul is tired from the endless monotony of slowly but surely conforming your behaviour to accommodate social acceptability as defined by our algorithmic overlords.

Solution: Pull up a chair and engage in escapist optimism by reading about an equally disheveled group of souls working at Ghost on a quest to federate and achieve technological independence via ActivityPub.

This week's episode begins with our small group of rambunctious rabble-rousers quietly huddled around a small table, dimly lit by a flickering bulb, discussing last week's newsletter.

"What do you mean, only 6 comments?"

"We're losing them sir. They are not entertained."

"What about the pugs?"

"Pugs were always going to be a risky strategy sir, the internet was colonised by cats very early on and they still hold a majority of the web's power. There is only so much we can–"

"Deploy more ActivityPugs."

"Sir we cannot possi–"

"DEPLOY THEM ALL."

"They shall be entertained."

What's new with ActivityPub?

Last week we discussed our research into different approaches for building ActivityPub support as a separate service, and answered more of your questions. This week, as promised, we've made a decision on direction — and that's what we're going to talk about in some detail.

We've talked about how ActivityPub is too big of a thing to build directly into Ghost core, but what does it mean to make it a "separate service" ? The easiest way to think about this (as with most ActivityPub things) is by making a comparison to email, which Ghost also supports.

When we first built Ghost we made it so you could publish to the web and run a blog. A few years later, we added the ability to deliver those same posts as email newsletters. Same content, new format.

Delivering one email is not an especially challenging task, in strictly classical computer science terms. It's an open protocol, and the basics aren't hard to understand. Where things begin to get more nuanced, though, is when you want to send 100,000 emails. And know their open rate. And know how many people clicked a link. And handle unsubscribes, temporary delivery failures, permanent bounces, spam detection, DMARC alignment, return paths, and whatever fucking error Comcast Inboxes decided to throw this week.

Rare things become common at scale, and handling them properly quickly turns into significant task.

So, Ghost itself doesn't send email newsletters. Instead, it relies on an external service called Mailgun. Ghost offloads the newsletter content to Mailgun along with a list of intended recipients and then says "Seeya later sucker, I'm off for lunch." Mailgun then goes through the process of carefully delivering each message, recording analytics, errors, and any replies.

When it comes to our ActivityPub service, we're essentially looking to build the same sort of thing. An external piece of software which Ghost can offload content and a list of recipients to, and then say "Seeya later sucker, it's time for my massage." The ActivityPub service will then carefully handle sending, receiving and storing everything needed to communicate with the wider fediverse.

Great, so we need to build an entirely new app. That's cool, but also daunting, because it's a huge amount of work. Where do we even begin?

Enter: Fedify

As we touched on very briefly last week: The very first newsletter subscriber to suggest building ActivityPub as a service was Hong Minhee, the maintainer of an open source project called Fedify.

Fedify is a framework specifically designed for building ActivityPub services. You could think of it as a bit like Rails or Laravel, but specifically for fediverse apps. It's what we've decided to use to build Ghost's ActivityPub service.

Rather than tell you all about it ourselves, we sat down with Fedify's creator to let him explain the project in his own words:

Could you give a bit of background about the project - what was your inspiration and motivation to create Fedify as a library rather than a platform or product?

At first, I was trying to build a single-user microblogging software called Hollo, and of course, I implemented ActivityPub, but implementing ActivityPub from the ground up was quite painful for me.

For example, the "actor" property can be a URL, it can be an array of URLs, it can be an object, it can be an array of objects, or it can be missing altogether. If "actor" is a URL, you'll need to look up the URL if you want to know what the actor is about. This is true for the "to" property, it's true for the "cc" property, and it's true for the "object" property. If we simply care about interoperability with Mastodon, we don't need to handle all of these cases. But if the main reason we're implementing ActivityPub is for interoperability with the various implementations in the fediverse, then we need to handle all of these cases.

When I started implementing these things from scratch, I felt like I was back in the days of web programming with Perl and CGI. I tried to be smart about it, and I realized that I was already building a sloppy quasi-framework for ActivityPub. I had a sloppy ActivityPub framework and microblogging software coexisting in the same project.

I thought, “No way, I'm going to build a proper ActivityPub framework,” and Fedify is the result.

What are your goals and ambitions for Fedify? Where do you hope to see the project go in the next few years?

Fedify's goal is to help more developers implement ActivityPub into their web apps, and to maximize interoperability between implementations in the fediverse. To do this, we're implementing the Fediverse Enhancement Proposals (FEPs) specs one by one.

We also want Fedify to be a comprehensive toolbox for ActivityPub implementations, rather than just a library/framework today. (Fedify already includes CLI tools for debugging and testing.)

What have been the best and worst parts of the ActivityPub spec to deal with when working on Fedify so far?

The great thing about ActivityPub is that there's an ecosystem already in place: there's a lot of documentation, questions and answers, troubleshooting experiences, and tools for debugging and testing.

On the other hand, ActivityPub sits on top of a lot more standards than you might think, so it's sometimes necessary to juggle a lot of complex standards to get the full picture: JSON-LD, Linked Data, ActivityStreams, HTTP Signatures, Verifiable Credential Data Integrity, many FEPs… Sometimes, depending on the language or platform, you'll need to implement a subset of these standards yourself. This is the main reason I created Fedify.

We can definitely attest to the problems that Fedify is working hard to solve, because even in just a few weeks of early prototyping we were running into the issues described above right away.

This, of course, is the advantage of decentralised, open source, community-driven technology. Rather than reinventing the wheel and burning energy solving the same problems over and over again, we can work together to create shared infrastructure. Fedify will help us build faster, and in return we'll contribute back features and bugfixes that we need to Fedify.

Open source, as a philosophy, is a potluck dinner that benefits everyone.

Additionally, we intend to open source the entire app that we build with Fedify. So it's MIT licensed mixed metaphors and turtles paying it forward all the way down.

A couple of people have asked in the last few weeks if they can get involved with contributing to ActivityPub in Ghost — and the answer now, for the first time, is yes! Even as we're getting the first building blocks in place for our ActivityPub service, you can already contribute to Fedify.

Fedify is a relatively new project, so there's a lot of room for contributions: writing documentation, adding web framework integrations, adding vocabulary, refactoring, etc. However, there is a lack of guidance for contributors (there is a CONTRIBUTING docs though) and the issues are not well organized. That said, we're always happy to help in the GitHub Discussions or in the chat room on Matrix! 

Alright. That's what's new! Our next move from this point on will be starting to build out this new back-end application using Fedify.

If you have any questions or topics you'd like to see us cover in next week's newsletter, leave us a comment.

Yes, this team has only one KPI, and it is measured in comments.