[tor-relays] Proxy fairness improvements in Snowflake

TL;DR: We're deploying a proxy rate-limiting feature to defend against blocking attacks. No changes to proxy configurations are necessary. Make sure your Snowflake proxy is up to date. If you run multiple proxies behind the same IP address, you may see some rejected polls or a small decrease in the number of connected clients. This is expected and your proxy will continue to poll the broker and collect clients.

We've been working on some features lately to improve the resilience of Snowflake to proxy enumeration attacks in which a censor can attempt to block Snowflake by blocking enough proxy IP addresses to make the system unusable. Such an attack is possible if a censor runs a modified Snowflake client and logs the IP addresses of each of the proxies it is matched with. A censor can then add these IP addresses to a block list, preventing a user in the same region from successfully connecting to those proxies.

Snowflake naturally has some resistance to this type of attack because of its large proxy pool of around 150K unique IP addresses, thanks to this community <3, and high churn from many of these proxies joining and leaving the proxy pool each day[0,1]. However, high capacity standalone Snowflake proxies present a unique challenge: they are extremely useful because of their stability and generally unrestrictive or nonexistent firewalls and NATs, but they are also the most frequently matched with users and therefore are the most likely to be blocked if an enumeration attack were to occur. A few years ago, Roger posted the results of a snowflake blocking experiment[2] that found this blocking attack to be particularly effective against users behind more restrictive NATs, because these users are proportionally more likely to be matched with high capacity standalone Snowflake proxies.

To help balance the distribution rates of proxies, we've designed and implemented a proxy fairness proposal[3]. In this proposal, we equalize proxy polling rates by interactively letting each proxy know when to poll next[4], and rate-limit proxies that poll too quickly[5]. You can read more about our justification for this in the linked issues, but to summarize: by slowing the rate of our higher capacity proxies to match lower capacity proxies, users are more likely to receive a proxy that is lower capacity and therefore less likely to have been enumerated. This change will also defend against a potential denial of service attack by malicious proxies flooding the network.

What this means for Snowflake proxy operators:

- If you are running a standalone Snowflake proxy, make sure you have the most recent version.

- If you run multiple Snowflake proxies behind the same NAT, you may see some rejected polls because of the rate limiting feature. The rate limit is per-IP, intentionally, because IP addresses are what censors will block in this kind of attack. This is normal, your proxy will continue to try again.

- You may or may not see a change in your usage stats. If you do, I'd love to hear from you on this thread about what you are seeing. This shouldn't affect overall capacity, but may mean that proxies reach their capacity more slowly.

- There has been a change to the poll-interval command-line option. This now specifies a default or initial poll rate. The proxy's poll rate will change interactively, based on information from the Snowflake broker. We've updated the README to reflect this.

We are planning a short 2 hour trial deployment of this new broker feature on Tuesday, October 27th at 15 UTC[6]. During this time you may see the above described changes take effect for your proxies. I will post an update to this thread with metrics and the outcome of this trial. If the trial goes smoothly, we will schedule a more permanent deployment of this feature.

As usual, please reach out if you experience bugs or see drastic drops in connected users. Thank you for running a Snowflake proxy. The growing number of volunteers has made a huge impact on Snowflake's blocking resistance!

[0] Snowflake, a censorship circumvention system using temporary WebRTC proxies
[1] Measure snowflake proxy churn of each pool separately (#40494) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab
[2] [anti-censorship-team] snowflake blocking experiment - anti-censorship-team - lists.torproject.org
[3] Proposal to improve the fairness and responsiveness of proxy distribution (#40507) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab
[4] Let the broker inform proxies how often to poll (#25598) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab
[5] Add back rate limiting for /proxy requests at broker (#40506) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab
[6] Broker deployment of proxy rate-limiting feature (scheduled: TBD) (#40573) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab

ยทยทยท

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

2 Likes

I ran a proxy by building it from the main branch, since there was no apt package in the beginning, that got a daily average of 50KB/s or so.
This worked fine (in a Proxmox container with -ephemeral-ports-range) until tag v2.13.1.
Anything after (on the main branch and building daily) caused all traffic to cease, so I stayed on the v2.13.1 tag.

I switched to the standard apt package snowflake-proxy (without tor) since v2.15.1, which allows me to configure ports in /etc/defaults/snowflake-proxy.
I see almost no traffic anymore: it's down to few KB total in a day. It reports NAT type open (instead of unrestricted like before).
Am I doing something wrong or is this to be expected and is there a (bigger) ramp up effect when switching versions or restarting?

I might not have time during the October 27th test but I can have a look later that day. Let me know how I can help.

ยทยทยท

On Thursday, 22 October 2026 at 22:38, Cecylia Bocovich via tor-relays <tor-relays@lists.torproject.org> wrote:

As usual, please reach out if you experience bugs or see drastic drops
in connected users. Thank you for running a Snowflake proxy. The growing
number of volunteers has made a huge impact on Snowflake's blocking
resistance!

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

TL;DR: We're deploying a proxy rate-limiting feature to defend against blocking attacks. No changes to proxy configurations are necessary. Make sure your Snowflake proxy is up to date. If you run multiple proxies behind the same IP address, you may see some rejected polls or a small decrease in the number of connected clients. This is expected and your proxy will continue to poll the broker and collect clients.

[snip]

We are planning a short 2 hour trial deployment of this new broker feature on Tuesday, October 27th at 15 UTC[6]. During this time you may see the above described changes take effect for your proxies. I will post an update to this thread with metrics and the outcome of this trial. If the trial goes smoothly, we will schedule a more permanent deployment of this feature.

The trial just ended and I've redeployed the old broker version. I've posted a writeup of our broker metrics in the corresponding issue[0] that I'll repost here. Because this trial deployment was a success, I am planning a persistent deployment of this new feature tomorrow at 17 UTC.

The trial broker deployment went really well. While the number of available proxies dropped, as expected, we had a sufficient number of available proxies in all three proxy pools. I was able to run Snowflake clients and proxies, as expected, and experienced not trouble matching or being matched during this time.

A graphana graph of available snowflake proxies during the trial period. The number of proxies drops, but remains stable from 15 to 17 UTC.

We saw the largest decrease in standalone Snowflake proxy polls. That matches with our awareness that many standalone snowflake operators run multiple snowflakes on the same IP address. There was less of an impact on web and mobile proxies, but still a slight one. This isn't suprising as it's likely for multiple people in the same household or workplace to run a Snowflake proxy, but not at the same scale as standalone proxy operators.

That we saw a decrease at all is a sign that the rate limiting feature is working.

A grafana graph of restricted client polls during the trail period. It shows that the number of idle proxies drops, but remains higher than the number of matched proxies, and no clients were denied.

Our smallest proxy pool, snowflakes for clients with restricted NATs, still had plenty of idle proxies during the trial period. This is a good sign. It means that this proxy pool has plenty of diversity in proxy IP addresses and this rate limiting feature won't place a strain on our system or impact proxy availability for clients.

A grafana graph of the status of all client requests during the trial. It shows that the number of matched proxies remains stable, the number of denied clients remains at zero, and a drop in the number of timed out requests.

No clients were turned away for lack of proxies during this trial. We did however, see a drop in the number of client timeouts. I didn't expect this feature to have an impact on the client timeouts that we're tracking client-proxy timeouts in Find and address the root cause of client rendezvous errors (#40447) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab.

[0] Broker deployment of proxy rate-limiting feature (scheduled: TBD) (#40573) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab

ยทยทยท

On 10/22/26 4:32 PM, Cecylia Bocovich via tor-relays wrote:

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

2 Likes

Thank you for the report, this is really helpful. The v2.13.1 issue you experienced was due to two issues:
- a delay in rolling out some necessary broker changes for the new NAT behaviour detection feature, and
- a change in the upstream pion/ice candidate library for collecting connection candidates
these have both been fixed in v2.15.0[0]

As to the current drop in connected clients, I have received some similar reports and am looking into them now[1]. If anyone has data or experiences they think might be useful please feel free to share here or in the issue.

[0] Current (git main, ebeacb14) snowflake proxy receives 400 response from broker (#40561) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab
[1] 2.14.1 hangs silently after some time when a -capacity is set. (#40557) ยท Issues ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab

ยทยทยท

On 10/25/26 7:51 AM, dmz21 via tor-relays wrote:

On Thursday, 22 October 2026 at 22:38, Cecylia Bocovich via tor-relays <tor-relays@lists.torproject.org> wrote:

As usual, please reach out if you experience bugs or see drastic drops
in connected users. Thank you for running a Snowflake proxy. The growing
number of volunteers has made a huge impact on Snowflake's blocking
resistance!

I ran a proxy by building it from the main branch, since there was no apt package in the beginning, that got a daily average of 50KB/s or so.
This worked fine (in a Proxmox container with -ephemeral-ports-range) until tag v2.13.1.
Anything after (on the main branch and building daily) caused all traffic to cease, so I stayed on the v2.13.1 tag.

I switched to the standard apt package snowflake-proxy (without tor) since v2.15.1, which allows me to configure ports in /etc/defaults/snowflake-proxy.
I see almost no traffic anymore: it's down to few KB total in a day. It reports NAT type open (instead of unrestricted like before).
Am I doing something wrong or is this to be expected and is there a (bigger) ramp up effect when switching versions or restarting?

I might not have time during the October 27th test but I can have a look later that day. Let me know how I can help.
_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

We've just released a new version of the snowflake proxy[0] that fixes a deadlock in the Snowflake proxy if the --capacity flag is set. Let us know if you continue to experience a low volume of traffic.

[0] Release v2.15.3 ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab

ยทยทยท

On 10/27/26 1:50 PM, Cecylia Bocovich via tor-relays wrote:

On 10/25/26 7:51 AM, dmz21 via tor-relays wrote:

On Thursday, 22 October 2026 at 22:38, Cecylia Bocovich via >> tor-relays <tor-relays@lists.torproject.org> wrote:

As usual, please reach out if you experience bugs or see drastic drops
in connected users. Thank you for running a Snowflake proxy. The growing
number of volunteers has made a huge impact on Snowflake's blocking
resistance!

I ran a proxy by building it from the main branch, since there was no apt package in the beginning, that got a daily average of 50KB/s or so.
This worked fine (in a Proxmox container with -ephemeral-ports-range) until tag v2.13.1.
Anything after (on the main branch and building daily) caused all traffic to cease, so I stayed on the v2.13.1 tag.

I switched to the standard apt package snowflake-proxy (without tor) since v2.15.1, which allows me to configure ports in /etc/defaults/snowflake-proxy.
I see almost no traffic anymore: it's down to few KB total in a day. It reports NAT type open (instead of unrestricted like before).
Am I doing something wrong or is this to be expected and is there a (bigger) ramp up effect when switching versions or restarting?

I might not have time during the October 27th test but I can have a look later that day. Let me know how I can help.

Thank you for the report, this is really helpful. The v2.13.1 issue you experienced was due to two issues:
- a delay in rolling out some necessary broker changes for the new NAT behaviour detection feature, and
- a change in the upstream pion/ice candidate library for collecting connection candidates
these have both been fixed in v2.15.0[0]

As to the current drop in connected clients, I have received some similar reports and am looking into them now[1]. If anyone has data or experiences they think might be useful please feel free to share here or in the issue.

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

This feature is now deployed! We'll be monitoring Snowflake metrics closely over the next week, and please continue to reach out if you notice any sudden changes in your Snowflake proxy traffic.

ยทยทยท

On 10/27/26 1:39 PM, Cecylia Bocovich via tor-relays wrote:

Because this trial deployment was a success, I am planning a persistent deployment of this new feature tomorrow at 17 UTC.

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

I just noticed no traffic with -capacity 20 set. I removed all flags except the -ephemeral-ports-range and installed tor-geoip (which somehow requires tor, which I disabled from starting). That did result in the proxy relaying traffic as normal. I did not know what changed it and assumed the tor-geoip (which is not a hard requirement?) to have made the difference. I should have changed only one thing at a time...

I did not receive the update from the repository yet but removing the -capacity flag made all the difference. Can I uninstall tor and tor-geoip?

Thanks for looking into this and fixing it. Once I get the update, I will increase capacity (from the default 10) gradually until reaching the limits of the VPS.

ยทยทยท

On Wednesday, 28 October 2026 at 21:30, Cecylia Bocovich via tor-relays <tor-relays@lists.torproject.org> wrote:

On 10/27/26 1:50 PM, Cecylia Bocovich via tor-relays wrote:
>
> As to the current drop in connected clients, I have received some
> similar reports and am looking into them now[1]. If anyone has data or
> experiences they think might be useful please feel free to share here
> or in the issue.
>
We've just released a new version of the snowflake proxy[0] that fixes a
deadlock in the Snowflake proxy if the --capacity flag is set. Let us
know if you continue to experience a low volume of traffic.

[0]
Release v2.15.3 ยท The Tor Project / Anti-censorship / Pluggable Transports / Snowflake ยท GitLab

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

Yes you can remove those. If for some reason that causes problems let us know.

ยทยทยท

On 10/28/26 4:41 PM, dmz21 via tor-relays wrote:

I did not receive the update from the repository yet but removing the -capacity flag made all the difference. Can I uninstall tor and tor-geoip?

_______________________________________________
tor-relays mailing list -- tor-relays@lists.torproject.org
To unsubscribe send an email to tor-relays-leave@lists.torproject.org

Thanks for the fix. A data point from a standalone proxy (Docker image, host networking, -capacity 10):

- With v2.15.2, the proxy received no offers at all for 8 hours. With v2.15.3 offers arrive again, so the deadlock fix works.

- However, v2.15.3 classifies the NAT as "moderate", whereas v2.11.0 reported "restricted" on the same network. I

In the first hour on v2.15.3, 9 out of 10 offers ended with "Timed out waiting for client to open data channel", and 0 bytes were relayed (tor_snowflake_proxy_connection_timeouts_total 9, traffic counters 0).

- The firewall does endpoint-independent mapping (static source port) with stateful filtering and no inbound port forwarding.

My guess is that "moderate" proxies are now being matched with clients they cannot actually reach. Iโ€™m keeping 2.15.3 running for more data. Happy to share more logs or test a build.

lowtide

ยทยทยท

On 10/27/26 1:50 PM, Cecylia Bocovich via tor-relays wrote:
We've just released a new version of the snowflake proxy that fixes a
deadlock in the Snowflake proxy if the --capacity flag is set. Let us
know if you continue to experience a low volume of traffic.