Search, profiles, and beta plans

Also, Ghost has joined the Social Web Foundation

Welcome back friends from around the Fediverse. It was pretty amazing to chat with you all in response to last week's newsletter. In so many ways, it felt like the future — and we set a new record for number of comments. Thank you!

If you're a recent subscriber to this newsletter and you weren't already aware, as well as sending out each week's edition by email, we also publish it directly to the Fediverse. You can follow along on with your platform of choice, here:

@index@activitypub.ghost.org

Also, the team working on this refers to themselves as the ActivityPugs. So. Well. That's about as much of an explanation as you're going to get on that one.

This whimsical little guy is looking for some friends.

What's new with ActivityPub?

We've got so many new things to share this week that we're going to break this down into individual sections.

What's new with Ghost

This week, we started rounding out Ghost's ActivityPub client so you can search for, find, view, and follow ActivityPub profiles in the Fediverse. Now that we have most of the big chunks of functionality working, we're moving into fleshing out many of the smaller connecting pieces.

You know, the parts you might need to beta test this whole thing:

0:00

/0:11

And it works!

What's new with Fedify

Minhee has been working hard on Fedify (the ActivityPub library that Ghost uses) and this week released version 1.0. If you're a JavaScript developer interested in the Fediverse, you should definitely check it out.

This week also saw the launch of The Social Web Foundation, an independent non-profit with a mission to steward and support the Fediverse.

The SWF was created by Evan Prodromou (author of the ActivityPub standard), Mallory Knodel, and Tom Coates — all of whom have extensive experience with decentralized technology, media, and web standards.

Ghost is a launch partner and a sponsor of the Social Web Foundation, alongside Flipboard, Meta, Mastodon, Fastly, Write.as, and several others.

Scaling ActivityPub

Look. It's not all unipugs and rainbows over here. We've been making great progress for the past few weeks, but we're also confronting challenges along the way.

Remember last week when we sent you that newsletter about replies, and then replied to all your comments? Well, we have auto-scaling for our ActivityPub infrastructure turned on, and with around 5,000 followers and a decent chunk of replies, it quickly scaled itself up to 10(!) servers to handle the load.

One of the things we're learning as we go is that ActivityPub is resource-intensive, and can be pretty expensive/difficult to run robustly. Needless to say, if we're going to support tens of thousands of sites and millions of followers with Ghost, we'll have to do quite a lot more work here.

A few people have asked over the past few months: Will members who use ActivityPub (rather than email) count toward Ghost(Pro) billing? And so far the answer is yes, because ActivityPub is (quite significantly) more expensive to support than email. But, this is all very early days, and hopefully we'll figure out ways to get some of the costs and complexity down.

So what's the deal with the beta?

We're getting very close to being able to have some of you(!) help us test out ActivityPub in Ghost, so we're going to start sharing some info about how all that's going to work, and what you should expect.

This is going to take time

Firstly, and most importantly: Please understand that this will not be a quick process. We're literally going to add individual, small sites, 1-at-a-time to begin with. How quickly we add new sites depends 100% on how well (or poorly) that goes. If we immediately run into bugs and scaling issues (see section above), we won't add any new sites until those issues are resolved. If things are going smoothly, we'll expand more quickly.

What we want to be really clear about is that we don't know what the timeline for the beta rollout to [everyone] will be. It could be weeks, or it could easily be months. It just depends on how well it goes.

We made a little tongue-in-cheek joke last week about how Ghost now has more Fediverse features than Threads, and the reason for that is our respective teams are approaching the development of ActivityPub from opposite angles. The Threads team is building 1 feature at a time, then rolling it out to hundreds of millions of users. This appears slow from the outside, but they're dealing with their scale issues immediately as they go. The Ghost team has built a bunch of features all in one go, but we've rolled them out to zero users and just shared screenshots of our progress. This appears fast from the outside, but we have not yet confronted the challenges of production workloads and scale.

Either way, we promise we'll continue to keep you in the loop about what's going on in this newsletter, and we're going as quickly as we can.

As a part of the beta, we are allowing for ActivityPub data loss

For the very first stage of the beta, we'll be making data loss for ActivityPub data an acceptable parameter. Your Ghost data — that's posts, pages, members, etc — is all safe and sound as normal. No changes there. But ActivityPub data — followers, likes, replies, all that good stuff — that's not safe. Listen, I'm just going to give it to you unvarnished. There's a good chance we're going to fuck up our ActivityPub database somehow based on bugs you find, and at that point we want to be able to reset things back to zero, address the bugs, and move forward.

This is a real beta, which means: "Only suitable for testing" - not suitable for actually growing and maintaining a public Fediverse profile, yet.

Who gets access to the first?

In short: small sites, using Ghost(Pro), who publish regularly (at least weekly).

Scale issues are what we're most concerned about, and those generally map to the size of your audience and the amount of engagement you get. We'll build confidence with small publishers first, then slowly add people with larger audiences after that.

It's limited to Ghost(Pro) customers because to be able to beta test this we need access to server logs to see what's going on, and Ghost(Pro) is the only place this is running in production right now.

Of course it will be possible to self-host ActivityPub in future, too. All of this is already open source, but we aren't able to test with self-hosted sites as part of the beta.

How to apply to be one of the first in the beta

In order to select people to join the beta, we need to know some things about you - like the URL of your site and how big your audience is - so we've put together a quick survey form to collect this data.

If you'd like to be one of the first people to start testing things, hit this button:

Click here to apply for the beta