2. torflow
in detail¶
by Mike Perry and others
2010?
use TorCtl [pytorctlCode], Python 2
not easy to install
not easy to maintain
[TorflowCode], [TorflowPaper], [TORFLOWSPEC], [BwAuthWiki]
2.1. How torflow
performs measurements¶
split relays into buckets by bandwidth percentile
build two hop circuits with a relay and exit from relays in the bucket
download a file from a Web server. The file size is based on the bucket
measure how long it takes
store the results
- What
torflow
is doing, in a broad sense: - * launch 2 tor clients* repeat as often as possible, running 9 different scanners:* split relays into buckets by bandwidth percentile* build two hop paths with a relay and exit from relays in the bucket* download a file from a bandwidth server, choose the size based on the bucket* measure how long it takes* store the results in a database* aggregate the results hourly:* produce a consensus weight to advertised bandwidth ratio* using a decaying weighted average* and some form of feedback (PID) control* and dump it to a fileThen authorities read this file and include it in their votes.
2.1.1. Algorithm to build two hops path¶
take all relays, order by bandwith
divide in consecutive slices of 50 relays
for each relay in slice: - take a (random?) exit in the slice (from the set of exits in that slice minus the relay itself) - build a path with the relay and the exit - download a file for 30secs - calculate bandwith
2.2. How torflow
aggregates measurements¶
hourly
calculate a bandwidth weight based on the descriptor
observed bandwidth
(advertised bandwidth ratio?)generate a
measurements
file
The measurements
file is read by the dirauths and includes the measured
bandwidth in their votes.
2.3. How the bandwidth weight is calculated¶
bw_observed = min(bandwidth-avg, bandwidth-burst, bandwidth-observed)
2.4. Example torflow
’s Bandwidth measurements file¶
aka Bandwidth list v1.0.0
1523911758
node_id=$68A483E05A2ABDCA6DA5A3EF8DB5177638A27F80 bw=760 nick=Test measured_at=1523911725 updated_at=1523911725 pid_error=4.11374090719 pid_error_sum=4.11374090719 pid_bw=57136645 pid_delta=2.12168374577 circ_fail=0.2 scanner=/filepath
node_id=$96C15995F30895689291F455587BD94CA427B6FC bw=189 nick=Test2 measured_at=1523911623 updated_at=1523911623 pid_error=3.96703337994 pid_error_sum=3.96703337994 pid_bw=47422125 pid_delta=2.65469736988 circ_fail=0.0 scanner=/filepath
2.5. Tickets¶
Bandwith Authority Renovation: [Ticket13630]
getting bandwidth servers more consistent and more geographically distributed: [Ticket24674]
torflow
component query (63) : [TorflowComp]tor-bwauth keyword query (7): [BwAuthKey]
2.6. Class diagram¶
2.7. Docs¶
To run Bandwith authority scanner [BwAuthWiki]
2.7.1. Install & run¶
./add_torctl.sh
vim ./NetworkScanners/BwAuthority/data/scanner.1/bwauthority.cfg
cd NetworkScanners/BwAuthority/
./setup.sh
./run_scan.sh
2.7.2. Example output¶
1515946341
node_id=$0E5D730466E5BB675BC9704C374DBE7E19F06165 bw=57100 nick=artlogic measured_at=1515314555 updated_at=1515314555 pid_error=4.11374090719 pid_error_sum=4.11374090719 pid_bw=57136645 pid_delta=2.12168374577 circ_fail=0.2 scanner=/scanner.1/scan-data/bws-6.5:7.4-done-2018-01-07-02:42:35
node_id=$DCC6C70E8F4340CBD633F506BC9E7D99818601D9 bw=47400 nick=eludemailrelay measured_at=1515666291 updated_at=1515666291 pid_error=3.96703337994 pid_error_sum=3.96703337994 pid_bw=47422125 pid_delta=2.65469736988 circ_fail=0.0 scanner=/scanner.1/scan-data/bws-0.8:1.6-done-2018-01-11-04:24:51
2.8. File format¶
2.4. Result format
The final output file for use by the directory authorities is comprised of
lines of the following format:
"node_id=" fingerprint SP
"bw=" new_bandwidth SP
"nick=" nickname SP
"measured_at=" slice timestamp NL
If PID control is enabled, additional values are stored. See Section 3.4
for those.
3.4. Value storage
In order to maintain the PID information, we store the following additional
fields in the output file:
"pid_error=" (PID error term as defined in Section 3.1) SP
"pid_error_sum=" (Weighted sum of PID error) SP
"pid_delta=" (Change in error) SP
"pid_bw=" (Last bandwidth value used in feedback) NL
pid_delta is purely informational, and is not used in feedback.
2.9. Scaling¶
2.10. Bandwidth measurements files¶
2.11. Measurements analysis¶
map of bandwidth bias: [MetricsWeight]
bandwidth authority variance: [BwAuthTools]
CDF graphs of bw authority votes for all of the flag combinations: [Ticket2394], [MetricsTask2394]