blog.thms.uk

FediFetcher Step by Step guide

FediFetcher is a tool for Mastodon that automatically fetches missing replies and posts from other fediverse instances, and adds them to your own Mastodon instance.

In this post I’ll run step-by-step through a method of installing FediFetcher on your own server. The only assumption that this guide makes, is that you know how to connect to your server using SSH.

This guide is aimed at complete novices. If you think you know what you are doing, there is a much shorter guide at the FediFetcher GitHub Repository.

Requirements

  1. You need a server to run FediFetcher on. This can (but doesn’t need to) be the server that is also running Mastodon.
  2. You need SSH access to that server.
  3. You must have pip installed on that server.
    • To test if you have pip installed: Log into your server through SSH and run
      which pip
      If you get any output, you’ll be fine. If you get nothing, you don’t have pip installed.
    • To install pip if you don’t have it installed: Log into your server through SSH and run
      sudo apt install python3-pip

If you don’t have a server with SSH access available, check out other options to run FediFetcher.

1) Get an Access Token

In order to get started, you need to get a Mastodon Access Token. You can get one from GetAuth for Mastodon:

  1. Open GetAuth for Mastodon
  2. Type in your Mastodon instance’s domain, and press Go.
  3. Click Authorize.
  4. Copy the token for later user.

2) Install and configure FediFetcher

  1. Connect to your server using SSH.
  2. Download FediFetcher by running:
    git clone https://github.com/nanos/FediFetcher.git
  3. Navigate to the FediFetcher directory:
    cd FediFetcher
  4. Install Requirements:
    pip install -r requirements.txt
  5. Create a configuration file:
    1. nano artifacts/config.json
    2. Paste your config file, e.g.
      {
    "access-token": "The token you generated above",
    "server": "your.mastodon.server",
    "home-timeline-length": 200,
    "max-followings": 80,
    "from-notifications": 1
}
    3. Press CTRL + X on your keyboard, to close the editor, followed by Y, to save your changes.
  6. Make a directory to hold log files:
    mkdir artifacts/logs
  7. Schedule FediFetcher:
    1. Open your crontab editor:
      crontab -e
      Note! If you have never edited your crontab before, you may be asked to choose an editor. Choose nano (which should be the default).
    2. Move your cursor to the end of the file using the arrow keys on your keyboard, and paste the following four lines:
      # Run FediFetcher every minute, with date-stamped log files stored in artifacts/logs/
* * * * * python ~/FediFetcher/find_posts.py -c ~/FediFetcher/artifacts/config.json > ~/FediFetcher/artifacts/logs/output-`/bin/date +\%Y\%m\%d\%H\%M\%S`.log 2>&1
# Delete FediFetcher log files that are older than 2 days daily at midnight
0 0 * * * find ~/FediFetcher/artifacts/logs/ -type f -mtime +2 -name 'output*.log' -execdir rm -- '{}' \;
    3. Press CTRL + X on your keyboard, to close the editor, followed by Y, to save your changes.

That’s it. FediFetcher should now be scheduled to run every minute. You can confirm this, by checking the log files:

Checking log files

With this configuration, FediFetcher logs all output into the ~/FediFetcher/artifacts/logs/ directory, using time-stamped log files. (You have additionally configured a script that will delete log files older than 48 hours from that directory, to prevent it from just growing indefinitely.)

As FediFetcher runs every minute, a lot of these log files will simply contain the following content:

Starting FediFetcher
Lock file exists at ~/FediFetcher/artifacts/lock.lock
Lock file age is 0:19:00.597216 - below --lock-hours=24 provided.

This is totally normal! To find the latest relevant log file, run ls -l on that directory, e.g.

ls -l ~/FediFetcher/artifacts/logs/
total 192
-rw-rw-r-- 1 user user   250 Aug 14 08:09 output-20230814080901.log
-rw-rw-r-- 1 user user   250 Aug 14 08:10 output-20230814081001.log
-rw-rw-r-- 1 user user 97291 Aug 14 08:31 output-20230814081101.log
-rw-rw-r-- 1 user user   250 Aug 14 08:12 output-20230814081201.log
-rw-rw-r-- 1 user user   250 Aug 14 08:13 output-20230814081301.log

The last file in that list that is larger than the others (in this instance output-20230814081101.log) is the one that’s most likely to be of interest.

You can print the last few lines in your terminal using tail:

tail -40 ~/FediFetcher/artifacts/logs/output-20230814081101.log

If you see an error in your logfile that you don’t understand, you can open an issue on GitHub or contact me on Mastodon.

Adjusting the FediFetcher configuration

If you want to adjust your configuration at any time, simply open the config.json file, and make any changes you wish:

nano ~/FediFetcher/artifacts/config.json

See the GitHub Repository for all available options.

Updating FediFetcher

Subscribe to the #FediFetcher tag on mastodon, as that’s where I’ll announce new versions.

To update your copy of FediFetcher:

  1. Connect via SSH.
  2. Navigate to the FediFetcher directory:
    cd ~/FediFetcher
  3. Update FediFetcher:
    git pull
pip install -r requirements.txt
  4. Check the release notes to see if there are any other steps listed.