Known guides / performance considerations for high performance Exit nodes

Support Relay Operator
1

Hi,

Was just wondering if anyone has any tips, guides or otherwise information regarding running high performance exit nodes.

I’m experimenting with an unlimited bandwidth 10gbps server for which I currently have 20 IPs. It’s ryzen powered, 64GB of ram.

As is, I’m running all relays on this server within the same OS. I’m expecting this to potentially cause issues due to inherent OS limitations (Debian 12) regarding connection management and such.

Perhaps I’m jumping the gun here and everything will be just fine, but was wondering if someone that has done this already - has any experience to share to prevent me from making the same mistakes.

I could also go the proxmox route if need be, but thought that was too much complexity for now without enough reason. If required, I don’t want to start changing kernel variables to fix niche issues.

1 Like
2

Summary:
Roughly 1 relay per core (some prefer higher density with 1 relay per thread)
Some suggest locking CPU clock on boost
Roughly between 2-4 GB RAM per relay (I’m budgeting higher for the rare DoS on guard nodes)
If you’re inside the EU, you won’t need as many relays to saturate your 10 Gbps. 20 IPs might be enough and can start with 20 relays and add more relays per IP as needed.
If you’re outside the EU, where I am, you will need more relays to saturate 10 Gbps. I haven’t got close yet. In US East, which is best performing in US (US West was worst, 4x slower, and US South was 2x slower), I’m around 90 relays on the best server that’s doing almost 2 Gbps with guard relays for ~100 days. I’m calling this the "bandwidth under utilization tax’ Tor imposes outside of the EU.

Details:
Lengthy thread on tor-relay mailing list on this topic over the last few months as I’ve been doing the same thing.
I tried to summarize one of the great emails to be more concise: Re: Guidance on optimal Tor relay server configurations - tor-relays - lists.torproject.org

I’m experimenting with multiple 10 Gbps dedicated servers, all in the US. Each server is between 88-192 threads and 384-1000GB RAM.
Few conclusions so far:
Outside of the EU, be prepared to pay the “bandwidth under utilization tax” Tor imposes. I’m throwing away 70-80% of the bandwidth I pay for each month right now. Expensive.
Tor limits families to 360 relays. I’m using 1 x IPv4 /24 per dedicated server, so 2048 possible relays (256 IPs x 8 relays per IP). Max is around 90 relays on one server right now.