[tor-project] PieroV's Monthly Status Report, June 2024

Hi everyone!
Here is my status report for June 2024.

This month, I worked almost exclusively on the ESR transition.
Once we got the tags for Firefox 128.0b1, I rebased my RR branch once again and opened the MR [0] to merge it to our first tor-browser-128 branch.
After that, I finalized the document with comments on the range-diff converted to HTML with the procedure described in my previous report.
It forced me to be more meticulous about every change, and I found some errors I had initially missed.
You can find it in the linked MR.

After that, I switched to the tor-browser-build part. First for desktop, then for Android. Linux and macOS were smooth, and the builds were reproducible at the first attempt.

Windows was more involved for a couple of reasons. First, Firefox now requires the windows-rs crate. Mozilla decided not to vendor it for some reason I haven't investigated.
Then, I had problems with llvm-windres. I solved this by restoring the wrapper the upstream project deleted some years ago [1]. Indeed, Mozilla is still using it, but their version would break OpenSSL, so I used a more recent one.
After this, I could get a build, but it wasn't reproducible. I noticed that only some binaries were different, and my hypothesis was that it was binaries that included some Rust code.
We verify reproducibility by building in different machines (I used my workstation and one of our build servers). We don't share any artifacts: every machine has to compile everything from scratch (we bootstrap several compilers, including Rustc).
So, I verified my conjecture using the Rustc artifact I created on my computer on the build server, and it solved the reproducibility problem.
The differences were only in std, so I think they were due to the differences in the GCC-based mingw-w64 toolchain.
We've had a ticket about building Rustc with the LLVM-based mingw toolchain we use for everything else [2] for a long time, but it isn't trivial due to Rust's hard dependency on libgcc. A series of new target triples [3] was created to avoid hacking around to resolve this problem, but Firefox doesn't recognize them (and Mozilla doesn't officially support this toolchain).
I wrote a small patch to solve the problem, and the build became reproducible. We will have to discuss more about the disadvantages of this approach.

The Android part was much more involved. Our builds run offline, whereas Mozilla's don't. So, I had to write several workarounds, including a patch for a custom Gradle plugin, and I wrote some Groovy for the first time in my life :smile:.
The monorepo migration was not troublesome at all: the commands we used to build in firefox-android.git still work without changes. Eventually, our build scripts were simplified a little bit.
The Android builds aren't reproducible yet. One reason is that only GeckoView was patched; our Android Components and Fenix patches haven't been rebased yet, and they include some changes to make the builds reproducible. Another reason is that the developments on nimbus-fml between 115 and 128 introduced a regression, and its output became non-deterministic. I wrote a patch and opened a PR [4]. I expect all the reproducibility problems to disappear once our patches are rebased.
The merge request with all the build changes [5] has been reviewed and approved, but I am waiting for the rebase MR to be merged first.

Apart from the ESR transition work, I rebased 13.0 and 13.5 onto Firefox 115.12.0, did some QA on the 13.5 RCs, and investigated the reproducibility problem we had with the 13.5 builds.


[0] Draft: Bug 42616: Rebased alpha onto Firefox 128.0b1 (!1043) · Merge requests · The Tor Project / Applications / Tor Browser · GitLab
[1] wrappers: Remove windres-wrapper; llvm-windres is available since LLV… · mstorsjo/llvm-mingw@8915db8 · GitHub
[2] Use mingw-w64/clang toolchain to build Rust (#29320) · Issues · The Tor Project / Applications / tor-browser-build · GitLab
[3] *-pc-windows-gnullvm - The rustc book
[4] nimbus-fml: Make the output deterministic. by PieroV · Pull Request #6283 · mozilla/application-services · GitHub
[5] Draft: Bug 41155: Update toolchains for 128 (!989) · Merge requests · The Tor Project / Applications / tor-browser-build · GitLab


tor-project mailing list