[tor-project] OONI Monthly Report: May 2023

Mailing Lists tor-project
1

Hello,

This email shares OONI’s monthly report for May 2023.

# OONI Monthly Report: May 2023

Throughout May 2023, the OONI team worked on the following sprints:

  • Sprint 90 (24th April - 7th May 2023)
  • Sprint 91 (8th - 21st May 2023)
  • Sprint 92 (22nd May - 4th June 2023)

Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni

Highlights are shared in this report below.

## New partnership

We are excited to have formed a new partnership with Annir Initiative!

The goal of the partnership is to collaborate on the study of internet censorship in Libya. We published a dedicated partner page for Annir Initiative highlighting their important work: https://ooni.org/partners/annir/

## Job Opening: OONI Project Manager

On 2nd May 2023, we published a job opening for hiring an OONI Project Manager!

The job posting is available here: https://ooni.org/post/2023-job-opening-ooni-project-manager/

In an attempt to help ensure that our job opening reached interested candidates around the world, we worked on relevant outreach efforts. In particular, we posted the job opening on multiple platforms, including:

We also reached out to our partners and many other community members, requesting that they help share this opportunity with their networks.

Following the publication of this job opening, we worked on tracking and reviewing incoming applications on a rolling basis, and following-up with applicants. In May 2023, we started the first round of interviews with shortlisted candidates.

## Blog post about new OONI Explorer features

We published a blog post to share information about new OONI Explorer features for investigating internet censorship through open data.

This blog post is available here: https://ooni.org/post/2023-new-explorer-features/

Specifically, OONI Explorer includes the following new features:

We hope these new features help with monitoring and responding to censorship events worldwide.

## OONI Probe Mobile

We released OONI Probe Android 3.8.2 (https://github.com/ooni/probe-android/releases/tag/v3.8.2), which includes a new version of the engine and ships some of the previously developed bug fixes.

We also worked towards addressing the following bugs:

## OONI Probe Desktop

We released OONI Probe Desktop 3.9.0 (https://github.com/ooni/probe-desktop/releases/tag/v3.9.0) which contains the Signal test fix from OONI Probe CLI v3.17.2.

## OONI Run

We created a first iteration of the OONI Run v2 link web editor interface. Specifically, we built the frontend component for the UI, which will enable users to create and edit existing links (https://github.com/ooni/run/pull/131).

We added support in the backend API for creating/updating the OONI Run v2 descriptor and serving them to clients (https://github.com/ooni/backend/pull/678). We also discussed and reviewed internal design documents for the OONI Run v2 check-in API.

## OONI Probe CLI

We released OONI Probe CLI v3.17.3 (https://github.com/ooni/probe-cli/releases/tag/v3.17.3). This release updates to go1.19.9, fixes issues with the Signal test, and updates the bundled geoip database to 2023-05. We did not release this version because we noticed that we needed to fix more bugs, such as upgrading OpenSSL (https://github.com/ooni/probe-cli/commit/325a8412999b001f4913ab5c522ed6402c9d78fe) and fixing a test helper issue (https://github.com/ooni/probe-cli/commit/a3af5542986e314a4e766d36078bedbae5c977ef). We therefore chose to wait until 2023-06-05 to bundle the 2023-06 geoip databases.

Following a community request (https://github.com/ooni/probe/issues/2478), we started preparing for releasing version 3.18, which will include support for go1.20. In this reporting period, we upgraded the code to use go1.20 (https://github.com/ooni/probe-cli/pull/1144) and we started following the usual pre-release check-list (https://github.com/ooni/probe/issues/2417).

## Expanding OONI’s testing model to support richer testing input

We continued to work on the richer input design document (https://github.com/ooni/ooni.org/issues/1295). We open sourced a repository that contains an initial design document (https://github.com/ooni/2023-05-richer-input/blob/main/DESIGN.md) and refactors OONI Probe around the concept of richer input. The code inside of the repository shows how we can use the concept of richer input to simplify the probe implementation. We explored, in particular, how we could use richer input to streamline the implementation of the OONI Probe IM tests.

## Creating a Social Media Censorship Alert System

In May 2023, we continued to make progress towards building a Social Media Censorship Alert System. Specifically, we improved our social media blocking event detector by adding RSS/Atom feed generation. Additionally, the event detector was updated to run hourly.

This work is documented through the following pull request: https://github.com/ooni/backend/pull/651

## OONI backend

Throughout May 2023, we worked on the following backend activities:

  • Annotated faulty Signal measurements as failed (https://github.com/ooni/backend/pull/680);
  • Monitored incoming measurements following an OONI Probe Android 3.8.0 bug impacting unattended runs;
  • Increased the testing priority for URLs in Timor Leste (amid the country’s 2023 elections) and created internal dashboards to monitor the testing coverage;
  • Created an MVP for an internal dashboard for long-term measurement traffic prediction and alarming;
  • Fixed a broken dependency in the Altair library used on our Jupyter Notebook instance;
  • Fixed the Jupyter Notebook automatic runner (jupycron) to generate HTML reports on hourly, daily and weekly intervals;
  • Implemented tooling to measure the performance of the probe engine using valgring / cachegrind (https://github.com/ooni/probe-cli/pull/1137);
  • Continued working towards configuring Vector to centralize log management (setting up SSL certificates);
  • Configured the main backend host (backend-FSN) to send logs to the monitoring host;
  • Created initial internal dashboards for log monitoring.

## Test lists updates

In preparation for elections which could potentially trigger new censorship events, we collaborated with our partners and other community members on updating relevant test lists.

This involved updates to the test lists for the following countries:

We thank community members for contributing to these test lists!

## OONI profile for Mozilla 2023 Data Futures Lab Cohort

OONI is thrilled to be part of Mozilla’s 2023 Data Futures Lab Cohort (https://foundation.mozilla.org/en/blog/mozilla-welcomes-2023-data-futures-lab-cohort/)! As part of this, we’re working towards releasing an improved version of OONI Run that addresses key community feedback, supporting decentralized censorship measurement campaigns around the world.

In May 2023, Mozilla published a blog post which provides a profile of OONI (as a Mozilla Data Futures Lab awardee). This blog post includes quotes from an interview with OONI’s Arturo, and can be accessed here: https://foundation.mozilla.org/en/blog/crowdsourcing-data-as-a-tool-to-fight-internet-censorship/

## ClickHouse blog post

ClickHouse published a blog post describing how we use their database management system for OONI data: https://clickhouse.com/blog/ooni-analyzes-internet-censorship-data-with-clickhouse

We migrated to ClickHouse a few years ago, and that has enabled us to analyze and publish OONI measurements in real-time (seconds)!

## Blocking of social media in Pakistan

In response to the blocking of social media platforms in Pakistan, we shared relevant OONI data with the #KeepItOn advocacy mailing list and on social media platforms (https://twitter.com/OpenObservatory/status/1656417835365588994, https://twitter.com/OpenObservatory/status/1658489171902885897).

Specifically, starting from around 15:00 UTC on 9th May 2023, OONI data collected from Pakistan showed that several ISPs started to restrict access to Facebook, YouTube, and Twitter. You can find relevant OONI data (along with a chart) here: https://explorer.ooni.org/chart/mat?test_name=web_connectivity&axis_x=measurement_start_day&since=2023-05-09&until=2023-05-11&time_grain=hour&probe_cc=PK&axis_y=domain&domain=www.youtube.com%2Cwww.facebook.com%2Ctwitter.com%2Cwww.instagram.com

The blocking of YouTube is further suggested by Google’s Transparency Report, which shows a drop in YouTube traffic from Pakistan: https://transparencyreport.google.com/traffic/overview?hl=en&fraction_traffic=start:1682812800000;end:1683763199999;product:21;region:PK&lu=fraction_traffic

Starting from around 05:00 UTC on 10th May 2023, OONI data collected from Pakistan showed that several ISPs started to restrict access to Instagram as well: https://explorer.ooni.org/chart/mat?probe_cc=PK&since=2023-04-12&until=2023-05-12&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.instagram.com

In most measurements (for the restricted social media platforms), OONI data shows signs of TLS-level interference (as TLS handshakes time out), while some measurements show DNS-based blocking as well (returning an NXDOMAIN error). Notably, some providers implement both censorship techniques.

## Blocking of social media in Guinea

In response to the blocking of social media platforms in Guinea, we shared relevant OONI data (https://twitter.com/OpenObservatory/status/1660591036316499968).

Specifically, OONI data from Guinea suggests that some ISPs started blocking access to the following social media platforms on 17th May 2023:

For all of the above services, OONI data provides signals suggesting that access to these platforms started to be blocked on 17th May 2023. However, the block does not appear to be implemented on all networks in the country.

OONI data suggests that ISPs in Guinea implemented the blocks by means of TLS interference. Specifically, OONI data shows that the TLS handshakes result in a timeout error, which is why these social media services appear to be inaccessible on tested networks.

## Community use of OONI data

### Access Now statement in response to social media blocks in Guinea

Access Now published a statement (citing OONI data) urging authorities in Guinea to unblock social media: https://www.accessnow.org/press-release/stop-shutting-down-the-internet-guinea/

### ISOC Pulse post on social media blocks in Guinea

As part of the ISOC Pulse shutdowns project, ISOC included OONI data and analysis in their post about the social media blocks in Guinea: https://pulse.internetsociety.org/shutdowns/social-media-blockage-observed-in-guinea

## Community activities

### OONI workshop for civil society groups in Liberia

On 4th May 2023, OONI’s Maria facilitated an online OONI workshop for civil society groups in Liberia. This workshop was organized by Access Now and Liberian civil society groups to prepare for Liberia’s 2023 elections.

### OONI workshop in Tbilisi

On 9th May 2023, OONI’s Elizaveta facilitated an in-person OONI workshop as part of a training in Tbilisi (organized by Internews) for content makers from Turkmenistan.

### OONI workshop for journalists in Kazakhstan

On 17th May 2023, OONI’s Elizaveta facilitated an online OONI workshop for journalists in Kazakhstan as part of the Greater Internet Freedom (GIF) Convening of Objective Partners in Almaty.

### OONI Community Meeting

On 30th May 2023, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics:

  1. Updates from the OONI team.

  2. Addressing false positives in Signal measurements displayed on the MAT (https://github.com/ooni/backend/issues/679).

  3. Using machine learning as part of OONI data analysis (https://docs.google.com/presentation/d/1rw7a02lpTj4CcguAz_nbqzNzkACKdFvMzTLrairAQ70/edit?usp=sharing).

## Measurement coverage

In May 2023, 59,563,089 OONI Probe measurements were collected from 2,990 networks in 159 countries around the world.

This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/

~ OONI team.