Pull missing responses into Mastodon
Ever since I've set up my own mastodon instance, there is one thing that has annoyed me:
Mastodon does not always pull responses to posts from other instances through to your own instance. This means that if you read a post from another server, you may not see all replies. This has been brought up in the Mastodon GitHub discussion before, but the Mastodon team doesn't seem to want to change this.
Personally, I consider this a problem because it means that I may not be able to see the full context of a conversation, which can make it difficult to understand what’s going on. It also means that if I post a reply, I might just repeat what others have already said, thus looking like an idiot (even more than usual...)
Over the weekend, I've come across Abhinav Sarkar's post on pulling missing responses into Mastodon, and I loved the idea:
His script simply gets remote replies to posts that users on my instance have already replied to during the last x hours, and adds them to my own server.
There is one thing I was missing from Abhinav's script though: By the time that I've replied to a remote post, it might be too late to get context. I wanted to get replies to (ideally) all posts on my home timeline.
So I've adjusted his script, and made a GitHub Action out of it, which you can find at nanos/FediFetcher.
What does FediFetcher do?
FediFetcher is a Python script that pulls missing replies to posts on your timeline from other fediverse instances into your own Mastodon instance. Additionally it can backfill profiles of accounts you have recently followed, or that have recently followed you.
For more details view the FediFetcher for Mastodon page.
How do I use FediFetcher?
FediFetcher can be run as a GitHub action. This means you don't need any infrastructure to run it on. This is great, because it can run for several minutes at a time, and so this won't slow down your home server too much while it's doing its thing. It also means you can run it against a hosted mastodon instance, as it simply uses the Mastodon API to do its work.
Alternatively you can run FediFetcher as a python script on your own server, and a pre-packaged container is also available.
How do I set up and configure FediFetcher?
Have a look at the README. It contains all the details, but the basic process is:
- Fork the repository
- Get an API key from your own instance
- Configure and enable the GitHub Action
Can I run FediFetcher on a multi-user instance?
Theoretically yes. However, the more users you have on your instance, the less you should need to, as your server will naturally pull in more replies.
The part that gets replies to posts on your home timeline will also only get the home timeline of the API key owner.
As such, this script is probably best suited for single-user, and very small instances.
Is FediFetcher production ready?
Absolutely not! I know very little python, so use at your own risk. I have, however, run it against my own server for 24 hours now, and my server hasn't blown up yet.
There are things that can be improved!
Fantastic! did I mention that I know very little python? By all means: Add a comment here, create an issue on GitHub, or, if you can, file a Pull Request.
I'm also aware, that this script needs still more exception handling, and I am working on that, so there'll likely be some improvements over the next few days and weeks.