FediFetcher for Mastodon
FediFetcher is a tool for Mastodon that automatically fetches missing replies and posts from other fediverse instances, and adds them to your own Mastodon instance.
The problem
When running a very small (or single user) Mastodon instance, one of Mastodon's short comings becomes particularly painful:
A lot of posts in your home timeline, will be from other instances. Very often, replies to those posts will not make it to your instance, and that's a shame. It means you might miss out on interesting parts of a conversation.
Additionally, if someone new from another instance follows you, their profile will often be completely empty. This is also a problem, because you have no idea who the person that's just followed you is. This is possibly even more acute if you require approval for new followers, as you have no data to base your approval or rejection on.
A solution
FediFetcher is a simple Python script that can pull missing posts into Mastodon using the Mastodon API. It can be run as either a GitHub Action, as a scheduled cron job, or a pre-packaged container.
Here is what FediFetcher can do:
It can pull missing remote replies to posts that are already on your server into your server. FediFetcher can be configured to
- fetch missing replies to posts in your home timeline,
- fetch missing replies to your bookmarks,
- fetch missing replies to your favourites.
It can also backfill profiles on your instance. FediFetcher can be configured to
- fetch missing posts from users that appear in your notifications,
- fetch missing posts from users that you have recently followed,
- fetch missing posts form users that have recently followed you,
- fetch missing posts form users that have recently sent you a follow request.
Each part of this script is fully configurable, and you can completely disable parts that you are not interested in.
Who is FediFetcher for
FediFetcher is almost certainly most suitable for users of single user and other very small mastodon instances. Whilst it could be used elsewhere, the need on larger instances will be far less, as these naturally pull in far more posts from remote instances.
How do I get an Access Token
If you are an admin on your instance, you can get an Access Token from your Mastodon Preferences.
If you are a normal user, you can use GetAuth for Mastodon to generate an Access Token.
How to configure and use FediFetcher
You can run FediFetcher as a GitHub Action, as a scheduled cron job, or a pre-packaged container. Full instructions are available on the FediFetcher GitHub repository:
Supported servers
FediFetcher makes use of the Mastodon API. It'll run against any instance implementing this API, and whilst it was built for Mastodon, it's been confirmed working against Pleroma as well.
FediFetcher will pull in posts from any server that implements the Mastodon API, including Mastodon, Pleroma, Akkoma, Pixelfed, and probably others.
Is FediFetcher respectful of existing norms on the fediverse?
There is a lot of reluctance on the fediverse against scraping. I believe FediFetcher is not at odds with this at all: It simply offers very targetted help with the federation of smaller (especially single user) instances.
Can FediFetcher be used to circumvent bans?
No. The way FediFetcher fetches content is by using the API key owner's instance's search functionality. It's functionally identical to them pasting the URL of the posts to be added into their own instance's search bar. Therefore, if an instance or user is blocked, FediFetcher will not be able to fetch content.
Any questions?
Feel free to reach out on Mastodon at @michael@thms.uk, or open an issue at nanos/FediFetcher.