Thank you for your reply.
DNS is a complex topic with many considerations, but I think you're on
the right track. Latency wise ideally you would find some location that
covers a lot of middle ground between the relays, as to prevent some
exit relays having a <20 ms DNS latency while others have >200 ms
latency. But your relays are spread very far and wide so this may prove
challenging.
Unfortunately, some of them would indeed have >200 ms rtt, as measured
by a quick ping between them just now. Perhaps in the future, if I find
an affordable dedi host in a good AS, I may focus on them and put out a
few Gbps, which would make that more feasible. Until then, I'm drifting
around looking for smaller hosts that allow Tor.
It's been frustrating at times, with one host (vsys.host) saying that
they allow Tor exits and happily accepting my payment, then changing
their mind and asking me not to run it. Then responding again saying
they elevated it to a senior manager and that it was indeed allowed,
and then replying again apologizing and saying that no, it's not. At
least they recognized the confusion and were willing to issue a refund.
Frustratingly, they are still listed as a good ISP on the community
wiki, and there are 3 exits hosted successfully on their AS...
* Only allow your Tor exit relay servers to send DNS requests to your
DNS load balancer/recursor. This makes it harder for adversaries to
check the in-cache TTL values. Or when providing the DNS response,
randomize the TTL value then (ideally per RRSet). You could also add
latency to DNS responses so they all are similarly slow (as if the
record wasn't cached), but this has downsides of its own.
* Decouple TTL values from the original DNS record's TTL and the value
in your cache. You could even randomize this (again per RRset ideally)
to make it even less predictable.
I thought about that after reading your blog post, but I couldn't find
out how to do that without patching the Unbound source code. That is
something I could do, but at the moment I don't have the time on my
hands to maintain such a fork, at least not to the quality standards
that would be wanted for Tor.
* Preload the top 1k/10k/100k/1M/10M (depending on your networking and
hardware capacity) domain's records and keep them 'hot' by fetching
their records with a random offset before they would expire. A previous
experiment with a small preload list (1k iirc) increased DNS cache hits
from ~74% (without preloading/automatic refreshes) to ~93% (with
preloading/automatic refreshes), decreasing DNS cache misses
considerably. My assumption is that the DNS cache hit rate will increase
significantly with 1) a larger preload list and 2) more queries per
second, but this is something I can finally verify when our new
infrastructure is in place next year.
I'm unsure how to preload the top domains. I couldn't find anywhere to
import an up-to-date cache, and having a script attempt to resolve a
large number of domains and keep them hot would probably not go over
too well with most hosting providers, I imagine.
I suspect that might not be helpful with smaller relays. One of my
relays is currently doing ~10 Mbps (it's capable of much more; I'm just
assuming it'll take more time before the BW authorities raise its
consensus weight), and it has a very low cache hit rate. I suspect this
is because TTL expiration starts to be the dominant cause of loss of
cache entries, even with prefetch enabled. The hit ratio is not great.
Since last reset a few days ago:
thread0.num.cachehits=74xxx
thread0.num.cachemiss=1637xxx
thread0.num.prefetch=28xxx
For reference, this is my current Unbound config (cache size varies
from system to system depending on available resources):
server:
outgoing-interface: <outgoing ipv4 goes here>
do-ip6: no
rrset-cache-size: 64m
msg-cache-size: 32m
neg-cache-size: 8m
prefetch: yes
prefetch-key: yes
auth-zone:
name: "."
master: f.root-servers.net
master: k.root-servers.net
master: l.root-servers.net
master: j.root-servers.net
fallback-enabled: yes
for-downstream: no
for-upstream: yes
zonefile: "/var/lib/unbound/root.zone"
So "Would it be reasonable to dedicated a single, cheap VPS for DNS
queries, and have all my other exits use it as their resolver over
DoT?"? I think this is a reasonable approach, even when it adds some DNS
latency (within reason) to your exit relays. But I would also try (where
possible/feasible) to take some countermeasures against some of the more
common/easy attacks. I think in many cases (even without extensive
countermeasures) a centralized DNS recursor for your own exit relays, at
the very least isn't a significant downgrade from running them locally
on your exit servers.
The worst rtt is ~250ms from my test. I know that systemd-resolved will
keep its own local cache even if it's using an upstream resolver, but
the size is small and fixed. Would a centralized resolver be feasible
even in my case? I suppose I could try to find a very well-peered AS
and find a cheap VPS on it for only the purposes of DNS resolution (and
perhaps as a bridge relay, so bandwidth isn't going to waste).
When our DNS infra is finally upgraded you're free to use it as well (we
can whitelist all your IP addresses), but in terms of latency it might
be a bad fit since your relays are situated in other parts of the world.
Perhaps there are Tor operator that run DNS recursors in South Africa,
Moldova, US and Canada for you to use :). You could also ask in the IRC
operators channel to see whether other operators can provide you with
access to their recursors.
Maybe I can switch to it for the nearest exits. If I had more networking
experience, I'd set up my own set of DNS resolvers for the Tor network
with anycast, and allow any IP that is a Tor exit to resolve from them.
That would at least be better than using 8.8.8.8. But of course, I have
neither the time, networking experience, or trust in the community as
a new exit operator to do that.
Regards,
forest
···
_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org
