Exempting an IP from Mastodon’s rate limits

I have always wanted to exempt my FediFetcher install from Mastodon’s rate limits: rate limits are of course really crucial to the security of a (web) application, but when I run my own FediFetcher against my own instance, the frequent resolving of remote posts triggers the rate limits a lot and really slows it down.

Unfortunately, Mastodon has no configuration option to do this, but it uses the Rack::Attack gem, and that can be configured by simply adding another file to the file system. So let’s get to work.

The commands below assume a traditional systemd install. The initialisers themselves are identical on Docker Compose, but you’ll need to get the files into the container and restart it differently: I’ve implemented the same thing (plus pinning an IP address for my FediFetcher container) in my mastodon-compose project.

Confirming the IP address Rack::Attack actually sees

Since this is security critical, I really wanted to make sure I’d got the IP addresses configured correctly, rather than relying on assumptions. So the first thing to do is log the IP addresses Rack::Attack sees. Create a file at config/initializers/zz_rack_attack_safelist.rb. (The zz_ prefix ensures it loads after Mastodon’s own rack_attack.rb, since Rails loads initializers in alphabetical order.) Give it the following content:

# frozen_string_literal: true
#
# Logs whenever a throttle or blocklist matches, so you can see which rule fired
# and for which IP.
ActiveSupport::Notifications.subscribe(/rack_attack/) do |_name, _start, _finish, _id, payload|
  req = payload[:request]
  next if req.nil?

  Rails.logger.warn(
    "[rack_attack] type=#{req.env['rack.attack.match_type']} " \
    "rule=#{req.env['rack.attack.matched']} " \
    "remote_ip=#{req.remote_ip} raw_ip=#{req.ip} " \
    "xff=#{req.get_header('HTTP_X_FORWARDED_FOR').inspect} path=#{req.path}"
  )
end

# Logs the IP for every API request without ever safelisting anything
# (the block always returns false). Run FediFetcher and read remote_ip out of 
# the logs
Rack::Attack.safelist('probe (never matches)') do |req|
if req.path.start_with?('/api/')
  Rails.logger.warn(
    "[rack_attack probe] remote_ip=#{req.remote_ip} raw_ip=#{req.ip} " \
    "xff=#{req.get_header('HTTP_X_FORWARDED_FOR').inspect} " \
    "ua=#{req.get_header('HTTP_USER_AGENT').inspect} path=#{req.path}"
  )
end

false
end

Now, restart Mastodon:

sudo systemctl restart mastodon-web

Read the logs while FediFetcher is running:

journalctl -u mastodon-web -f | grep 'rack_attack probe'

You should find lines that look a bit like this:

[rack_attack probe] remote_ip=172.20.0.2 raw_ip=172.19.0.1 xff="172.20.0.2" ua="FediFetcher/8.0.0; +mstdn.thms.uk (https://go.thms.uk/ff)" path=/api/v2/search

The value you want is remote_ip, not raw_ip: that’s the address Rails derives after stripping trusted proxies, and it’s what the safelist below compares against.

It’s important to sanity check that this isn’t your reverse proxy’s IP. Stop here and fix your reverse proxy if it is!

Safelist the IP

Edit config/initializers/zz_rack_attack_safelist.rb and replace the whole contents with:

# frozen_string_literal: true
#
# safelist by IP as specified in env
RACK_ATTACK_SAFELIST_IPS = ENV.fetch('RACK_ATTACK_SAFELIST_IPS', '')
                              .split(',')
                              .map(&:strip)
                              .reject(&:empty?)
                              .filter_map do |cidr|
                                begin
                                  IPAddr.new(cidr)
                                rescue IPAddr::Error => e
                                  Rails.logger.error("[rack_attack] ignoring invalid RACK_ATTACK_SAFELIST_IPS entry #{cidr.inspect}: #{e.message}")
                                  nil
                                end
                              end

unless RACK_ATTACK_SAFELIST_IPS.empty?
  Rails.logger.info("[rack_attack] safelisting #{RACK_ATTACK_SAFELIST_IPS.map(&:to_s).join(', ')}")

  Rack::Attack.safelist('allow from safelisted clients') do |req|
    begin
      ip = IPAddr.new(req.remote_ip)
      RACK_ATTACK_SAFELIST_IPS.any? { |net| net.include?(ip) }
    rescue IPAddr::Error
      false
    end
  end
end

A safelist match short-circuits everything else, so a matching request skips all of Mastodon’s throttles and blocklists.

Then add the following to your .env.production file:

# Replace with whatever IP address you found above. Separate multiple IPs with
# commas. CIDR ranges are also supported.
RACK_ATTACK_SAFELIST_IPS=172.20.0.2

Restart Mastodon again and read the logs during boot:

sudo systemctl restart mastodon-web && journalctl -u mastodon-web -f | grep 'rack_attack'

You should find a line saying

[rack_attack] safelisting 172.20.0.2

Security

You are now giving one IP (or several) totally unfettered access to your Mastodon instance, so you really want to make sure your reverse proxy has its trusted proxy addresses locked down properly. If it doesn’t, remote_ip is derived from an X-Forwarded-For header anyone can set, and your safelist becomes trivially spoofable. But that’s beyond the scope of this post.