Tor Hackweek Project: Prototype Rust+Arti-based HTTP frontend cache for directory authorities¶
Summary: Directory authorities are under a lot of unnecessary load from excessive download requests. We have other projects in mind to reduce those requests, but one workaround is to add a frontend cache in front of one or more of the authorities' HTTP ports. With this project we'll write a Rust server use Arti's download and validation code to fetch directory information from an authority, and expose that information via a set of HTTP requests. With luck, our code will be reuseable in the future when relays or authorities are rewritten in Rust.
Goal: A proof-of-concept that can handle some HTTP requests, and relays the ones it can't understand to an authority. Ideally, more!
Skills Needed: Rust hacking experience helpful.
Team¶
- nickm (UTC-5)
Breakdown for minimal version¶
- tor-dirmgr crate: add support for retrieving text of particular documents
- new tor-dirservice crate:
- Add a set of responders that can handle http requests. These should be made to plug in to Hyper (a rust webserver).
- Teach these responders to understand tor's http requests.
- Teach them to answer requests for what we have by getting the answer from the tor-dirmgr crate.
- Have a fallback responder that forwards anything else to a configured HTTP address:port.
- Have an entry point (new crate) that launches the above as a webserver.
Additional work to be more useful as a download cache¶
- Teach tor-netdoc to handle votes
- Teach tor-netdoc to handle networkstatus consensus docs
- Teach tor-dirclient to handle more kinds of directory document (votes, ns consensuses, routerdescs, extrainfos)
- Teach tor-dirclient to be able to make unencrypted HTTP requests
- Teach tor-dirmgr how to download and store more kinds of directory documents
- Implement code to generate consensus diffs
- Teach tor-dirmgr (or similar) how to keep a good set of consensus diffs
- Teach tor-dirmgr (or similar) how to pre-compress large documents
Additional work to be more useful as an upload frontend¶
- New crate: support for storing and cross-validating incoming descriptors and extrainfos. Don't do this with the tor-dirmgr crate: use a separate implementation.
- Support for basic relay reachability testing on incoming descriptors
- Support for exclude lists and keypinning
- Support for uploading a validated descriptor to an authority