diff options
364 files changed, 33103 insertions, 39798 deletions
diff --git a/.gitignore b/.gitignore index 46917c311..673bb7871 100644 --- a/.gitignore +++ b/.gitignore @@ -63,16 +63,6 @@ /contrib/tor.logrotate /contrib/tor.wxs -# /contrib/osx/ -/contrib/osx/Makefile -/contrib/osx/Makefile.in -/contrib/osx/TorBundleDesc.plist -/contrib/osx/TorBundleInfo.plist -/contrib/osx/TorDesc.plist -/contrib/osx/TorInfo.plist -/contrib/osx/TorStartupDesc.plist -/contrib/osx/net.freehaven.tor.plist - # /contrib/suse/ /contrib/suse/tor.sh /contrib/suse/Makefile.in @@ -80,6 +70,7 @@ # /debian/ /debian/files +/debian/micro-revision.i /debian/patched /debian/tor /debian/tor.postinst.debhelper @@ -90,12 +81,27 @@ # /doc/ /doc/Makefile /doc/Makefile.in -/doc/tor.1 /doc/doxygen - -# /doc/design-paper/ -/doc/design-paper/Makefile -/doc/design-paper/Makefile.in +/doc/tor.1 +/doc/tor.1.in +/doc/tor.html +/doc/tor.html.in +/doc/tor.1.xml +/doc/tor-gencert.1 +/doc/tor-gencert.1.in +/doc/tor-gencert.html +/doc/tor-gencert.html.in +/doc/tor-gencert.1.xml +/doc/tor-resolve.1 +/doc/tor-resolve.1.in +/doc/tor-resolve.html +/doc/tor-resolve.html.in +/doc/tor-resolve.1.xml +/doc/torify.1 +/doc/torify.1.in +/doc/torify.html +/doc/torify.html.in +/doc/torify.1.xml # /doc/spec/ /doc/spec/Makefile @@ -108,8 +114,10 @@ # /src/common/ /src/common/Makefile /src/common/Makefile.in +/src/common/common_sha1.i /src/common/libor.a /src/common/libor-crypto.a +/src/common/libor-event.a # /src/config/ /src/config/Makefile @@ -121,14 +129,25 @@ # /src/or/ /src/or/Makefile /src/or/Makefile.in +/src/or/or_sha1.i /src/or/micro-revision.* /src/or/tor -/src/or/test +/src/or/tor.exe +/src/or/libtor.a + +# /src/test +/src/test/Makefile +/src/test/Makefile.in +/src/test/test + # /src/tools/ /src/tools/tor-checkkey /src/tools/tor-resolve /src/tools/tor-gencert +/src/tools/tor-checkkey.exe +/src/tools/tor-resolve.exe +/src/tools/tor-gencert.exe /src/tools/Makefile /src/tools/Makefile.in diff --git a/AUTHORS b/AUTHORS deleted file mode 100644 index e4e9bf65e..000000000 --- a/AUTHORS +++ /dev/null @@ -1,41 +0,0 @@ - This file lists the authors for Tor, - a free software project to provide anonymity on the Internet. - - For more information about Tor, see https://www.torproject.org/. - - If you got this file as a part of a larger bundle, - there are probably other authors that you should be aware of. - -Main authors: -------------- - -Roger Dingledine <arma@freehaven.net> overhauled all of the code, did -a bunch of new design work, etc. - -Nick Mathewson <nickm@freehaven.net> wrote lots of stuff too, in -particular the router and descriptor parsing, and the crypto and tls -wrappers. - -Matej Pfajfar <badbytes@freehaven.net> wrote the first version of the code -(called OR) in 2001-2002. - -Contributors: -------------- - -John Bashinski <jbash@velvet.com> contributed the initial rpm spec file. - -Christian Grothoff <grothoff@cs.purdue.edu> contributed better daemonizing -behavior. - -Steven Hazel <sah@thalassocracy.org> made 'make install' do the right -thing. - -Jason Holt <jason@lunkwill.org> contributed patches to the instructions -and the man page. - -Peter Palfrader <peter@palfrader.org> maintains everything that's -debian-specific, and has written other useful features. - -Aaron Turner <aturner@netscreen.com> contributed the first version of -the tor.sh initscripts shell script. - @@ -1,3 +1,899 @@ +Changes in version 0.2.2.19-alpha - 2010-11-22 + Yet another OpenSSL security patch broke its compatibility with Tor: + Tor 0.2.2.19-alpha makes relays work with OpenSSL 0.9.8p and 1.0.0.b. + + o Major bugfixes: + - Resolve an incompatibility with OpenSSL 0.9.8p and OpenSSL 1.0.0b: + No longer set the tlsext_host_name extension on server SSL objects; + but continue to set it on client SSL objects. Our goal in setting + it was to imitate a browser, not a vhosting server. Fixes bug 2204; + bugfix on 0.2.1.1-alpha. + + o Minor bugfixes: + - Try harder not to exceed the maximum length of 50 KB when writing + statistics to extra-info descriptors. This bug was triggered by very + fast relays reporting exit-port, entry, and dirreq statistics. + Reported by Olaf Selke. Bugfix on 0.2.2.1-alpha. Fixes bug 2183. + - Publish a router descriptor even if generating an extra-info + descriptor fails. Previously we would not publish a router + descriptor without an extra-info descriptor; this can cause fast + exit relays collecting exit-port statistics to drop from the + consensus. Bugfix on 0.1.2.9-rc; fixes bug 2195. + + +Changes in version 0.2.2.18-alpha - 2010-11-16 + Tor 0.2.2.18-alpha fixes several crash bugs that have been nagging + us lately, makes unpublished bridge relays able to detect their IP + address, and fixes a wide variety of other bugs to get us much closer + to a stable release. + + o Major bugfixes: + - Do even more to reject (and not just ignore) annotations on + router descriptors received anywhere but from the cache. Previously + we would ignore such annotations at first, but cache them to disk + anyway. Bugfix on 0.2.0.8-alpha. Found by piebeer. + - Do not log messages to the controller while shrinking buffer + freelists. Doing so would sometimes make the controller connection + try to allocate a buffer chunk, which would mess up the internals + of the freelist and cause an assertion failure. Fixes bug 1125; + fixed by Robert Ransom. Bugfix on 0.2.0.16-alpha. + - Learn our external IP address when we're a relay or bridge, even if + we set PublishServerDescriptor to 0. Bugfix on 0.2.0.3-alpha, + where we introduced bridge relays that don't need to publish to + be useful. Fixes bug 2050. + - Maintain separate TLS contexts and certificates for incoming and + outgoing connections in bridge relays. Previously we would use the + same TLS contexts and certs for incoming and outgoing connections. + Bugfix on 0.2.0.3-alpha; addresses bug 988. + - Maintain separate identity keys for incoming and outgoing TLS + contexts in bridge relays. Previously we would use the same + identity keys for incoming and outgoing TLS contexts. Bugfix on + 0.2.0.3-alpha; addresses the other half of bug 988. + - Avoid an assertion failure when we as an authority receive a + duplicate upload of a router descriptor that we already have, + but which we previously considered an obsolete descriptor. + Fixes another case of bug 1776. Bugfix on 0.2.2.16-alpha. + - Avoid a crash bug triggered by looking at a dangling pointer while + setting the network status consensus. Found by Robert Ransom. + Bugfix on 0.2.2.17-alpha. Fixes bug 2097. + - Fix a logic error where servers that _didn't_ act as exits would + try to keep their server lists more aggressively up to date than + exits, when it was supposed to be the other way around. Bugfix + on 0.2.2.17-alpha. + + o Minor bugfixes (on Tor 0.2.1.x and earlier): + - When we're trying to guess whether we know our IP address as + a relay, we would log various ways that we failed to guess + our address, but never log that we ended up guessing it + successfully. Now add a log line to help confused and anxious + relay operators. Bugfix on 0.1.2.1-alpha; fixes bug 1534. + - Bring the logic that gathers routerinfos and assesses the + acceptability of circuits into line. This prevents a Tor OP from + getting locked in a cycle of choosing its local OR as an exit for a + path (due to a .exit request) and then rejecting the circuit because + its OR is not listed yet. It also prevents Tor clients from using an + OR running in the same instance as an exit (due to a .exit request) + if the OR does not meet the same requirements expected of an OR + running elsewhere. Fixes bug 1859; bugfix on 0.1.0.1-rc. + - Correctly describe errors that occur when generating a TLS object. + Previously we would attribute them to a failure while generating a + TLS context. Patch by Robert Ransom. Bugfix on 0.1.0.4-rc; fixes + bug 1994. + - Enforce multiplicity rules when parsing annotations. Bugfix on + 0.2.0.8-alpha. Found by piebeer. + - Fix warnings that newer versions of autoconf produced during + ./autogen.sh. These warnings appear to be harmless in our case, + but they were extremely verbose. Fixes bug 2020. + + o Minor bugfixes (on Tor 0.2.2.x): + - Enable protection of small arrays whenever we build with gcc + hardening features, not only when also building with warnings + enabled. Fixes bug 2031; bugfix on 0.2.2.14-alpha. Reported by keb. + + o Minor features: + - Make hidden services work better in private Tor networks by not + requiring any uptime to join the hidden service descriptor + DHT. Implements ticket 2088. + - Rate-limit the "your application is giving Tor only an IP address" + warning. Addresses bug 2000; bugfix on 0.0.8pre2. + - When AllowSingleHopExits is set, print a warning to explain to the + relay operator why most clients are avoiding her relay. + - Update to the November 1 2010 Maxmind GeoLite Country database. + + o Code simplifications and refactoring: + - When we fixed bug 1038 we had to put in a restriction not to send + RELAY_EARLY cells on rend circuits. This was necessary as long + as relays using Tor 0.2.1.3-alpha through 0.2.1.18-alpha were + active. Now remove this obsolete check. Resolves bug 2081. + - Some options used different conventions for uppercasing of acronyms + when comparing manpage and source. Fix those in favor of the + manpage, as it makes sense to capitalize acronyms. + - Remove the torrc.complete file. It hasn't been kept up to date + and users will have better luck checking out the manpage. + - Remove the obsolete "NoPublish" option; it has been flagged + as obsolete and has produced a warning since 0.1.1.18-rc. + - Remove everything related to building the expert bundle for OS X. + It has confused many users, doesn't work right on OS X 10.6, + and is hard to get rid of once installed. Resolves bug 1274. + + +Changes in version 0.2.2.17-alpha - 2010-09-30 + Tor 0.2.2.17-alpha introduces a feature to make it harder for clients + to use one-hop circuits (which can put the exit relays at higher risk, + plus unbalance the network); fixes a big bug in bandwidth accounting + for relays that want to limit their monthly bandwidth use; fixes a + big pile of bugs in how clients tolerate temporary network failure; + and makes our adaptive circuit build timeout feature (which improves + client performance if your network is fast while not breaking things + if your network is slow) better handle bad networks. + + o Major features: + - Exit relays now try harder to block exit attempts from unknown + relays, to make it harder for people to use them as one-hop proxies + a la tortunnel. Controlled by the refuseunknownexits consensus + parameter (currently enabled), or you can override it on your + relay with the RefuseUnknownExits torrc option. Resolves bug 1751. + + o Major bugfixes (0.2.1.x and earlier): + - Fix a bug in bandwidth accounting that could make us use twice + the intended bandwidth when our interval start changes due to + daylight saving time. Now we tolerate skew in stored vs computed + interval starts: if the start of the period changes by no more than + 50% of the period's duration, we remember bytes that we transferred + in the old period. Fixes bug 1511; bugfix on 0.0.9pre5. + - Always search the Windows system directory for system DLLs, and + nowhere else. Bugfix on 0.1.1.23; fixes bug 1954. + - When you're using bridges and your network goes away and your + bridges get marked as down, recover when you attempt a new socks + connection (if the network is back), rather than waiting up to an + hour to try fetching new descriptors for your bridges. Bugfix on + 0.2.0.3-alpha; fixes bug 1981. + + o Major bugfixes (on 0.2.2.x): + - Fix compilation on Windows. Bugfix on 0.2.2.16-alpha; related to + bug 1797. + - Fix a segfault that could happen when operating a bridge relay with + no GeoIP database set. Fixes bug 1964; bugfix on 0.2.2.15-alpha. + - The consensus bandwidth-weights (used by clients to choose fast + relays) entered an unexpected edge case in September where + Exits were much scarcer than Guards, resulting in bad weight + recommendations. Now we compute them using new constraints that + should succeed in all cases. Also alter directory authorities to + not include the bandwidth-weights line if they fail to produce + valid values. Fixes bug 1952; bugfix on 0.2.2.10-alpha. + - When weighting bridges during path selection, we used to trust + the bandwidths they provided in their descriptor, only capping them + at 10MB/s. This turned out to be problematic for two reasons: + Bridges could claim to handle a lot more traffic then they + actually would, thus making more clients pick them and have a + pretty effective DoS attack. The other issue is that new bridges + that might not have a good estimate for their bw capacity yet + would not get used at all unless no other bridges are available + to a client. Fixes bug 1912; bugfix on 0.2.2.7-alpha. + + o Major bugfixes (on the circuit build timeout feature, 0.2.2.x): + - Ignore cannibalized circuits when recording circuit build times. + This should provide for a minor performance improvement for hidden + service users using 0.2.2.14-alpha, and should remove two spurious + notice log messages. Bugfix on 0.2.2.14-alpha; fixes bug 1740. + - Simplify the logic that causes us to decide if the network is + unavailable for purposes of recording circuit build times. If we + receive no cells whatsoever for the entire duration of a circuit's + full measured lifetime, the network is probably down. Also ignore + one-hop directory fetching circuit timeouts when calculating our + circuit build times. These changes should hopefully reduce the + cases where we see ridiculous circuit build timeouts for people + with spotty wireless connections. Fixes part of bug 1772; bugfix + on 0.2.2.2-alpha. + - Prevent the circuit build timeout from becoming larger than + the maximum build time we have ever seen. Also, prevent the time + period for measurement circuits from becoming larger than twice that + value. Fixes the other part of bug 1772; bugfix on 0.2.2.2-alpha. + + o Minor features: + - When we run out of directory information such that we can't build + circuits, but then get enough that we can build circuits, log when + we actually construct a circuit, so the user has a better chance of + knowing what's going on. Fixes bug 1362. + - Be more generous with how much bandwidth we'd use up (with + accounting enabled) before entering "soft hibernation". Previously, + we'd refuse new connections and circuits once we'd used up 95% of + our allotment. Now, we use up 95% of our allotment, AND make sure + that we have no more than 500MB (or 3 hours of expected traffic, + whichever is lower) remaining before we enter soft hibernation. + - If we've configured EntryNodes and our network goes away and/or all + our entrynodes get marked down, optimistically retry them all when + a new socks application request appears. Fixes bug 1882. + - Add some more defensive programming for architectures that can't + handle unaligned integer accesses. We don't know of any actual bugs + right now, but that's the best time to fix them. Fixes bug 1943. + - Support line continuations in the torrc config file. If a line + ends with a single backslash character, the newline is ignored, and + the configuration value is treated as continuing on the next line. + Resolves bug 1929. + + o Minor bugfixes (on 0.2.1.x and earlier): + - For bandwidth accounting, calculate our expected bandwidth rate + based on the time during which we were active and not in + soft-hibernation during the last interval. Previously, we were + also considering the time spent in soft-hibernation. If this + was a long time, we would wind up underestimating our bandwidth + by a lot, and skewing our wakeup time towards the start of the + accounting interval. Fixes bug 1789. Bugfix on 0.0.9pre5. + + o Minor bugfixes (on 0.2.2.x): + - Resume generating CIRC FAILED REASON=TIMEOUT control port messages, + which were disabled by the circuit build timeout changes in + 0.2.2.14-alpha. Bugfix on 0.2.2.14-alpha; fixes bug 1739. + - Make sure we don't warn about missing bandwidth weights when + choosing bridges or other relays not in the consensus. Bugfix on + 0.2.2.10-alpha; fixes bug 1805. + - In our logs, do not double-report signatures from unrecognized + authorities both as "from unknown authority" and "not + present". Fixes bug 1956, bugfix on 0.2.2.16-alpha. + + +Changes in version 0.2.2.16-alpha - 2010-09-17 + Tor 0.2.2.16-alpha fixes a variety of old stream fairness bugs (most + evident at exit relays), and also continues to resolve all the little + bugs that have been filling up trac lately. + + o Major bugfixes (stream-level fairness): + - When receiving a circuit-level SENDME for a blocked circuit, try + to package cells fairly from all the streams that had previously + been blocked on that circuit. Previously, we had started with the + oldest stream, and allowed each stream to potentially exhaust + the circuit's package window. This gave older streams on any + given circuit priority over newer ones. Fixes bug 1937. Detected + originally by Camilo Viecco. This bug was introduced before the + first Tor release, in svn commit r152: it is the new winner of + the longest-lived bug prize. + - When the exit relay got a circuit-level sendme cell, it started + reading on the exit streams, even if had 500 cells queued in the + circuit queue already, so the circuit queue just grew and grew in + some cases. We fix this by not re-enabling reading on receipt of a + sendme cell when the cell queue is blocked. Fixes bug 1653. Bugfix + on 0.2.0.1-alpha. Detected by Mashael AlSabah. Original patch by + "yetonetime". + - Newly created streams were allowed to read cells onto circuits, + even if the circuit's cell queue was blocked and waiting to drain. + This created potential unfairness, as older streams would be + blocked, but newer streams would gladly fill the queue completely. + We add code to detect this situation and prevent any stream from + getting more than one free cell. Bugfix on 0.2.0.1-alpha. Partially + fixes bug 1298. + + o Minor features: + - Update to the September 1 2010 Maxmind GeoLite Country database. + - Warn when CookieAuthFileGroupReadable is set but CookieAuthFile is + not. This would lead to a cookie that is still not group readable. + Closes bug 1843. Suggested by katmagic. + - When logging a rate-limited warning, we now mention how many messages + got suppressed since the last warning. + - Add new "perconnbwrate" and "perconnbwburst" consensus params to + do individual connection-level rate limiting of clients. The torrc + config options with the same names trump the consensus params, if + both are present. Replaces the old "bwconnrate" and "bwconnburst" + consensus params which were broken from 0.2.2.7-alpha through + 0.2.2.14-alpha. Closes bug 1947. + - When a router changes IP address or port, authorities now launch + a new reachability test for it. Implements ticket 1899. + - Make the formerly ugly "2 unknown, 7 missing key, 0 good, 0 bad, + 2 no signature, 4 required" messages about consensus signatures + easier to read, and make sure they get logged at the same severity + as the messages explaining which keys are which. Fixes bug 1290. + - Don't warn when we have a consensus that we can't verify because + of missing certificates, unless those certificates are ones + that we have been trying and failing to download. Fixes bug 1145. + - If you configure your bridge with a known identity fingerprint, + and the bridge authority is unreachable (as it is in at least + one country now), fall back to directly requesting the descriptor + from the bridge. Finishes the feature started in 0.2.0.10-alpha; + closes bug 1138. + - When building with --enable-gcc-warnings on OpenBSD, disable + warnings in system headers. This makes --enable-gcc-warnings + pass on OpenBSD 4.8. + + o Minor bugfixes (on 0.2.1.x and earlier): + - Authorities will now attempt to download consensuses if their + own efforts to make a live consensus have failed. This change + means authorities that restart will fetch a valid consensus, and + it means authorities that didn't agree with the current consensus + will still fetch and serve it if it has enough signatures. Bugfix + on 0.2.0.9-alpha; fixes bug 1300. + - Ensure DNS requests launched by "RESOLVE" commands from the + controller respect the __LeaveStreamsUnattached setconf options. The + same goes for requests launched via DNSPort or transparent + proxying. Bugfix on 0.2.0.1-alpha; fixes bug 1525. + - Allow handshaking OR connections to take a full KeepalivePeriod + seconds to handshake. Previously, we would close them after + IDLE_OR_CONN_TIMEOUT (180) seconds, the same timeout as if they + were open. Bugfix on 0.2.1.26; fixes bug 1840. Thanks to mingw-san + for analysis help. + - Rate-limit "Failed to hand off onionskin" warnings. + - Never relay a cell for a circuit we have already destroyed. + Between marking a circuit as closeable and finally closing it, + it may have been possible for a few queued cells to get relayed, + even though they would have been immediately dropped by the next + OR in the circuit. Fixes bug 1184; bugfix on 0.2.0.1-alpha. + - Never queue a cell for a circuit that's already been marked + for close. + - Never vote for a server as "Running" if we have a descriptor for + it claiming to be hibernating, and that descriptor was published + more recently than our last contact with the server. Bugfix on + 0.2.0.3-alpha; fixes bug 911. + - Squash a compile warning on OpenBSD. Reported by Tas; fixes + bug 1848. + + o Minor bugfixes (on 0.2.2.x): + - Fix a regression introduced in 0.2.2.7-alpha that marked relays + down if a directory fetch fails and you've configured either + bridges or EntryNodes. The intent was to mark the relay as down + _unless_ you're using bridges or EntryNodes, since if you are + then you could quickly run out of entry points. + - Fix the Windows directory-listing code. A bug introduced in + 0.2.2.14-alpha could make Windows directory servers forget to load + some of their cached v2 networkstatus files. + - Really allow clients to use relays as bridges. Fixes bug 1776; + bugfix on 0.2.2.15-alpha. + - Demote a warn to info that happens when the CellStatistics option + was just enabled. Bugfix on 0.2.2.15-alpha; fixes bug 1921. + Reported by Moritz Bartl. + - On Windows, build correctly either with or without Unicode support. + This is necessary so that Tor can support fringe platforms like + Windows 98 (which has no Unicode), or Windows CE (which has no + non-Unicode). Bugfix on 0.2.2.14-alpha; fixes bug 1797. + + o Testing + - Add a unit test for cross-platform directory-listing code. + + +Changes in version 0.2.2.15-alpha - 2010-08-18 + Tor 0.2.2.15-alpha fixes a big bug in hidden service availability, + fixes a variety of other bugs that were preventing performance + experiments from moving forward, fixes several bothersome memory leaks, + and generally closes a lot of smaller bugs that have been filling up + trac lately. + + o Major bugfixes: + - Stop assigning the HSDir flag to relays that disable their + DirPort (and thus will refuse to answer directory requests). This + fix should dramatically improve the reachability of hidden services: + hidden services and hidden service clients pick six HSDir relays + to store and retrieve the hidden service descriptor, and currently + about half of the HSDir relays will refuse to work. Bugfix on + 0.2.0.10-alpha; fixes part of bug 1693. + - The PerConnBWRate and Burst config options, along with the + bwconnrate and bwconnburst consensus params, initialized each conn's + token bucket values only when the connection is established. Now we + update them if the config options change, and update them every time + we get a new consensus. Otherwise we can encounter an ugly edge + case where we initialize an OR conn to client-level bandwidth, + but then later the relay joins the consensus and we leave it + throttled. Bugfix on 0.2.2.7-alpha; fixes bug 1830. + - Fix a regression that caused Tor to rebind its ports if it receives + SIGHUP while hibernating. Bugfix in 0.1.1.6-alpha; closes bug 919. + + o Major features: + - Lower the maximum weighted-fractional-uptime cutoff to 98%. This + should give us approximately 40-50% more Guard-flagged nodes, + improving the anonymity the Tor network can provide and also + decreasing the dropoff in throughput that relays experience when + they first get the Guard flag. + - Allow enabling or disabling the *Statistics config options while + Tor is running. + + o Minor features: + - Update to the August 1 2010 Maxmind GeoLite Country database. + - Have the controller interface give a more useful message than + "Internal Error" in response to failed GETINFO requests. + - Warn when the same option is provided more than once in a torrc + file, on the command line, or in a single SETCONF statement, and + the option is one that only accepts a single line. Closes bug 1384. + - Build correctly on mingw with more recent versions of OpenSSL 0.9.8. + Patch from mingw-san. + - Add support for the country code "{??}" in torrc options like + ExcludeNodes, to indicate all routers of unknown country. Closes + bug 1094. + - Relays report the number of bytes spent on answering directory + requests in extra-info descriptors similar to {read,write}-history. + Implements enhancement 1790. + + o Minor bugfixes (on 0.2.1.x and earlier): + - Complain if PublishServerDescriptor is given multiple arguments that + include 0 or 1. This configuration will be rejected in the future. + Bugfix on 0.2.0.1-alpha; closes bug 1107. + - Disallow BridgeRelay 1 and ORPort 0 at once in the configuration. + Bugfix on 0.2.0.13-alpha; closes bug 928. + - Change "Application request when we're believed to be offline." + notice to "Application request when we haven't used client + functionality lately.", to clarify that it's not an error. Bugfix + on 0.0.9.3; fixes bug 1222. + - Fix a bug in the controller interface where "GETINFO ns/asdaskljkl" + would return "551 Internal error" rather than "552 Unrecognized key + ns/asdaskljkl". Bugfix on 0.1.2.3-alpha. + - Users can't configure a regular relay to be their bridge. It didn't + work because when Tor fetched the bridge descriptor, it found + that it already had it, and didn't realize that the purpose of the + descriptor had changed. Now we replace routers with a purpose other + than bridge with bridge descriptors when fetching them. Bugfix on + 0.1.1.9-alpha. Bug 1776 not yet fixed because now we immediately + refetch the descriptor with router purpose 'general', disabling + it as a bridge. + - Fix a rare bug in rend_fn unit tests: we would fail a test when + a randomly generated port is 0. Diagnosed by Matt Edman. Bugfix + on 0.2.0.10-alpha; fixes bug 1808. + - Exit nodes didn't recognize EHOSTUNREACH as a plausible error code, + and so sent back END_STREAM_REASON_MISC. Clients now recognize a new + stream ending reason for this case: END_STREAM_REASON_NOROUTE. + Servers can start sending this code when enough clients recognize + it. Also update the spec to reflect this new reason. Bugfix on + 0.1.0.1-rc; fixes part of bug 1793. + - Delay geoip stats collection by bridges for 6 hours, not 2 hours, + when we switch from being a public relay to a bridge. Otherwise + there will still be clients that see the relay in their consensus, + and the stats will end up wrong. Bugfix on 0.2.1.15-rc; fixes bug + 932 even more. + - Instead of giving an assertion failure on an internal mismatch + on estimated freelist size, just log a BUG warning and try later. + Mitigates but does not fix bug 1125. + - Fix an assertion failure that could occur in caches or bridge users + when using a very short voting interval on a testing network. + Diagnosed by Robert Hogan. Fixes bug 1141; bugfix on 0.2.0.8-alpha. + + o Minor bugfixes (on 0.2.2.x): + - Alter directory authorities to always consider Exit-flagged nodes + as potential Guard nodes in their votes. The actual decision to + use Exits as Guards is done in the consensus bandwidth weights. + Fixes bug 1294; bugfix on 0.2.2.10-alpha. + - When the controller is reporting the purpose of circuits that + didn't finish building before the circuit build timeout, it was + printing UNKNOWN_13. Now print EXPIRED. Bugfix on 0.2.2.14-alpha. + - Our libevent version parsing code couldn't handle versions like + 1.4.14b-stable and incorrectly warned the user about using an + old and broken version of libevent. Treat 1.4.14b-stable like + 1.4.14-stable when parsing the version. Fixes bug 1731; bugfix + on 0.2.2.1-alpha. + - Don't use substitution references like $(VAR:MOD) when + $(asciidoc_files) is empty -- make(1) on NetBSD transforms + '$(:x)' to 'x' rather than the empty string. This bites us in + doc/ when configured with --disable-asciidoc. Bugfix on + 0.2.2.9-alpha; fixes bug 1773. + - Remove a spurious hidden service server-side log notice about + "Ancient non-dirty circuits". Bugfix on 0.2.2.14-alpha; fixes + bug 1741. + - Fix compilation with --with-dmalloc set. Bugfix on 0.2.2.6-alpha; + fixes bug 1832. + - Correctly report written bytes on linked connections. Found while + implementing 1790. Bugfix on 0.2.2.4-alpha. + - Fix three memory leaks: one in circuit_build_times_parse_state(), + one in dirvote_add_signatures_to_pending_consensus(), and one every + time we parse a v3 network consensus. Bugfixes on 0.2.2.14-alpha, + 0.2.2.6-alpha, and 0.2.2.10-alpha respectively; fixes bug 1831. + + o Code simplifications and refactoring: + - Take a first step towards making or.h smaller by splitting out + function definitions for all source files in src/or/. Leave + structures and defines in or.h for now. + - Remove a bunch of unused function declarations as well as a block of + #if 0'd code from the unit tests. Closes bug 1824. + - New unit tests for exit-port history statistics; refactored exit + statistics code to be more easily tested. + - Remove the old debian/ directory from the main Tor distribution. + The official Tor-for-debian git repository lives at the URL + https://git.torproject.org/debian/tor.git + + +Changes in version 0.2.2.14-alpha - 2010-07-12 + Tor 0.2.2.14-alpha greatly improves client-side handling of + circuit build timeouts, which are used to estimate speed and improve + performance. We also move to a much better GeoIP database, port Tor to + Windows CE, introduce new compile flags that improve code security, + add an eighth v3 directory authority, and address a lot of more + minor issues. + + o Major bugfixes: + - Tor directory authorities no longer crash when started with a + cached-microdesc-consensus file in their data directory. Bugfix + on 0.2.2.6-alpha; fixes bug 1532. + - Treat an unset $HOME like an empty $HOME rather than triggering an + assert. Bugfix on 0.0.8pre1; fixes bug 1522. + - Ignore negative and large circuit build timeout values that can + happen during a suspend or hibernate. These values caused various + asserts to fire. Bugfix on 0.2.2.2-alpha; fixes bug 1245. + - Alter calculation of Pareto distribution parameter 'Xm' for + Circuit Build Timeout learning to use the weighted average of the + top N=3 modes (because we have three entry guards). Considering + multiple modes should improve the timeout calculation in some cases, + and prevent extremely high timeout values. Bugfix on 0.2.2.2-alpha; + fixes bug 1335. + - Alter calculation of Pareto distribution parameter 'Alpha' to use a + right censored distribution model. This approach improves over the + synthetic timeout generation approach that was producing insanely + high timeout values. Now we calculate build timeouts using truncated + times. Bugfix on 0.2.2.2-alpha; fixes bugs 1245 and 1335. + - Do not close circuits that are under construction when they reach + the circuit build timeout. Instead, leave them building (but do not + use them) for up until the time corresponding to the 95th percentile + on the Pareto CDF or 60 seconds, whichever is greater. This is done + to provide better data for the new Pareto model. This percentile + can be controlled by the consensus. + + o Major features: + - Move to the June 2010 Maxmind GeoLite country db (rather than the + June 2009 ip-to-country GeoIP db) for our statistics that count + how many users relays are seeing from each country. Now we have + more accurate data for many African countries. + - Port Tor to build and run correctly on Windows CE systems, using + the wcecompat library. Contributed by Valerio Lupi. + - New "--enable-gcc-hardening" ./configure flag (off by default) + to turn on gcc compile time hardening options. It ensures + that signed ints have defined behavior (-fwrapv), enables + -D_FORTIFY_SOURCE=2 (requiring -O2), adds stack smashing protection + with canaries (-fstack-protector-all), turns on ASLR protection if + supported by the kernel (-fPIE, -pie), and adds additional security + related warnings. Verified to work on Mac OS X and Debian Lenny. + - New "--enable-linker-hardening" ./configure flag (off by default) + to turn on ELF specific hardening features (relro, now). This does + not work with Mac OS X or any other non-ELF binary format. + + o New directory authorities: + - Set up maatuska (run by Linus Nordberg) as the eighth v3 directory + authority. + + o Minor features: + - New config option "WarnUnsafeSocks 0" disables the warning that + occurs whenever Tor receives only an IP address instead of a + hostname. Setups that do DNS locally over Tor are fine, and we + shouldn't spam the logs in that case. + - Convert the HACKING file to asciidoc, and add a few new sections + to it, explaining how we use Git, how we make changelogs, and + what should go in a patch. + - Add a TIMEOUT_RATE keyword to the BUILDTIMEOUT_SET control port + event, to give information on the current rate of circuit timeouts + over our stored history. + - Add ability to disable circuit build time learning via consensus + parameter and via a LearnCircuitBuildTimeout config option. Also + automatically disable circuit build time calculation if we are + either a AuthoritativeDirectory, or if we fail to write our state + file. Fixes bug 1296. + - More gracefully handle corrupt state files, removing asserts + in favor of saving a backup and resetting state. + - Rename the "log.h" header to "torlog.h" so as to conflict with fewer + system headers. + + o Minor bugfixes: + - Build correctly on OSX with zlib 1.2.4 and higher with all warnings + enabled. + - When a2x fails, mention that the user could disable manpages instead + of trying to fix their asciidoc installation. + - Where available, use Libevent 2.0's periodic timers so that our + once-per-second cleanup code gets called even more closely to + once per second than it would otherwise. Fixes bug 943. + - If you run a bridge that listens on multiple IP addresses, and + some user configures a bridge address that uses a different IP + address than your bridge writes in its router descriptor, and the + user doesn't specify an identity key, their Tor would discard the + descriptor because "it isn't one of our configured bridges", and + fail to bootstrap. Now believe the descriptor and bootstrap anyway. + Bugfix on 0.2.0.3-alpha. + - If OpenSSL fails to make a duplicate of a private or public key, log + an error message and try to exit cleanly. May help with debugging + if bug 1209 ever remanifests. + - Save a couple bytes in memory allocation every time we escape + certain characters in a string. Patch from Florian Zumbiehl. + - Make it explicit that we don't cannibalize one-hop circuits. This + happens in the wild, but doesn't turn out to be a problem because + we fortunately don't use those circuits. Many thanks to outofwords + for the initial analysis and to swissknife who confirmed that + two-hop circuits are actually created. + - Make directory mirrors report non-zero dirreq-v[23]-shares again. + Fixes bug 1564; bugfix on 0.2.2.9-alpha. + - Eliminate a case where a circuit build time warning was displayed + after network connectivity resumed. Bugfix on 0.2.2.2-alpha. + + +Changes in version 0.2.1.26 - 2010-05-02 + Tor 0.2.1.26 addresses the recent connection and memory overload + problems we've been seeing on relays, especially relays with their + DirPort open. If your relay has been crashing, or you turned it off + because it used too many resources, give this release a try. + + This release also fixes yet another instance of broken OpenSSL libraries + that was causing some relays to drop out of the consensus. + + o Major bugfixes: + - Teach relays to defend themselves from connection overload. Relays + now close idle circuits early if it looks like they were intended + for directory fetches. Relays are also more aggressive about closing + TLS connections that have no circuits on them. Such circuits are + unlikely to be re-used, and tens of thousands of them were piling + up at the fast relays, causing the relays to run out of sockets + and memory. Bugfix on 0.2.0.22-rc (where clients started tunneling + their directory fetches over TLS). + - Fix SSL renegotiation behavior on OpenSSL versions like on Centos + that claim to be earlier than 0.9.8m, but which have in reality + backported huge swaths of 0.9.8m or 0.9.8n renegotiation + behavior. Possible fix for some cases of bug 1346. + - Directory mirrors were fetching relay descriptors only from v2 + directory authorities, rather than v3 authorities like they should. + Only 2 v2 authorities remain (compared to 7 v3 authorities), leading + to a serious bottleneck. Bugfix on 0.2.0.9-alpha. Fixes bug 1324. + + o Minor bugfixes: + - Finally get rid of the deprecated and now harmful notion of "clique + mode", where directory authorities maintain TLS connections to + every other relay. + + o Testsuite fixes: + - In the util/threads test, no longer free the test_mutex before all + worker threads have finished. Bugfix on 0.2.1.6-alpha. + - The master thread could starve the worker threads quite badly on + certain systems, causing them to run only partially in the allowed + window. This resulted in test failures. Now the master thread sleeps + occasionally for a few microseconds while the two worker-threads + compete for the mutex. Bugfix on 0.2.0.1-alpha. + + +Changes in version 0.2.2.13-alpha - 2010-04-24 + Tor 0.2.2.13-alpha addresses the recent connection and memory overload + problems we've been seeing on relays, especially relays with their + DirPort open. If your relay has been crashing, or you turned it off + because it used too many resources, give this release a try. + + o Major bugfixes: + - Teach relays to defend themselves from connection overload. Relays + now close idle circuits early if it looks like they were intended + for directory fetches. Relays are also more aggressive about closing + TLS connections that have no circuits on them. Such circuits are + unlikely to be re-used, and tens of thousands of them were piling + up at the fast relays, causing the relays to run out of sockets + and memory. Bugfix on 0.2.0.22-rc (where clients started tunneling + their directory fetches over TLS). + + o Minor features: + - Finally get rid of the deprecated and now harmful notion of "clique + mode", where directory authorities maintain TLS connections to + every other relay. + - Directory authorities now do an immediate reachability check as soon + as they hear about a new relay. This change should slightly reduce + the time between setting up a relay and getting listed as running + in the consensus. It should also improve the time between setting + up a bridge and seeing use by bridge users. + - Directory authorities no longer launch a TLS connection to every + relay as they startup. Now that we have 2k+ descriptors cached, + the resulting network hiccup is becoming a burden. Besides, + authorities already avoid voting about Running for the first half + hour of their uptime. + + +Changes in version 0.2.2.12-alpha - 2010-04-20 + Tor 0.2.2.12-alpha fixes a critical bug in how directory authorities + handle and vote on descriptors. It was causing relays to drop out of + the consensus. + + o Major bugfixes: + - Many relays have been falling out of the consensus lately because + not enough authorities know about their descriptor for them to get + a majority of votes. When we deprecated the v2 directory protocol, + we got rid of the only way that v3 authorities can hear from each + other about other descriptors. Now authorities examine every v3 + vote for new descriptors, and fetch them from that authority. Bugfix + on 0.2.1.23. + - Fix two typos in tor_vasprintf() that broke the compile on Windows, + and a warning in or.h related to bandwidth_weight_rule_t that + prevented clean compile on OS X. Fixes bug 1363; bugfix on + 0.2.2.11-alpha. + - Fix a segfault on relays when DirReqStatistics is enabled + and 24 hours pass. Bug found by keb. Fixes bug 1365; bugfix on + 0.2.2.11-alpha. + + o Minor bugfixes: + - Demote a confusing TLS warning that relay operators might get when + someone tries to talk to their OrPort. It is neither the operator's + fault nor can they do anything about it. Fixes bug 1364; bugfix + on 0.2.0.14-alpha. + + +Changes in version 0.2.2.11-alpha - 2010-04-15 + Tor 0.2.2.11-alpha fixes yet another instance of broken OpenSSL + libraries that was causing some relays to drop out of the consensus. + + o Major bugfixes: + - Directory mirrors were fetching relay descriptors only from v2 + directory authorities, rather than v3 authorities like they should. + Only 2 v2 authorities remain (compared to 7 v3 authorities), leading + to a serious bottleneck. Bugfix on 0.2.0.9-alpha. Fixes bug 1324. + - Fix a parsing error that made every possible value of + CircPriorityHalflifeMsec get treated as "1 msec". Bugfix + on 0.2.2.7-alpha. Rename CircPriorityHalflifeMsec to + CircuitPriorityHalflifeMsec, so authorities can tell newer relays + about the option without breaking older ones. + - Fix SSL renegotiation behavior on OpenSSL versions like on Centos + that claim to be earlier than 0.9.8m, but which have in reality + backported huge swaths of 0.9.8m or 0.9.8n renegotiation + behavior. Possible fix for some cases of bug 1346. + + o Minor features: + - Experiment with a more aggressive approach to preventing clients + from making one-hop exit streams. Exit relays who want to try it + out can set "RefuseUnknownExits 1" in their torrc, and then look + for "Attempt by %s to open a stream" log messages. Let us know + how it goes! + - Add support for statically linking zlib by specifying + --enable-static-zlib, to go with our support for statically linking + openssl and libevent. Resolves bug 1358. + + o Minor bugfixes: + - Fix a segfault that happens whenever a Tor client that is using + libevent2's bufferevents gets a hup signal. Bugfix on 0.2.2.5-alpha; + fixes bug 1341. + - When we cleaned up the contrib/tor-exit-notice.html file, we left + out the first line. Fixes bug 1295. + - When building the manpage from a tarball, we required asciidoc, but + the asciidoc -> roff/html conversion was already done for the + tarball. Make 'make' complain only when we need asciidoc (either + because we're compiling directly from git, or because we altered + the asciidoc manpage in the tarball). Bugfix on 0.2.2.9-alpha. + - When none of the directory authorities vote on any params, Tor + segfaulted when trying to make the consensus from the votes. We + didn't trigger the bug in practice, because authorities do include + params in their votes. Bugfix on 0.2.2.10-alpha; fixes bug 1322. + + o Testsuite fixes: + - In the util/threads test, no longer free the test_mutex before all + worker threads have finished. Bugfix on 0.2.1.6-alpha. + - The master thread could starve the worker threads quite badly on + certain systems, causing them to run only partially in the allowed + window. This resulted in test failures. Now the master thread sleeps + occasionally for a few microseconds while the two worker-threads + compete for the mutex. Bugfix on 0.2.0.1-alpha. + + +Changes in version 0.2.2.10-alpha - 2010-03-07 + Tor 0.2.2.10-alpha fixes a regression introduced in 0.2.2.9-alpha that + could prevent relays from guessing their IP address correctly. It also + starts the groundwork for another client-side performance boost, since + currently we're not making efficient use of relays that have both the + Guard flag and the Exit flag. + + o Major bugfixes: + - Fix a regression from our patch for bug 1244 that caused relays + to guess their IP address incorrectly if they didn't set Address + in their torrc and/or their address fails to resolve. Bugfix on + 0.2.2.9-alpha; fixes bug 1269. + + o Major features (performance): + - Directory authorities now compute consensus weightings that instruct + clients how to weight relays flagged as Guard, Exit, Guard+Exit, + and no flag. Clients that use these weightings will distribute + network load more evenly across these different relay types. The + weightings are in the consensus so we can change them globally in + the future. Extra thanks to "outofwords" for finding some nasty + security bugs in the first implementation of this feature. + + o Minor features (performance): + - Always perform router selections using weighted relay bandwidth, + even if we don't need a high capacity circuit at the time. Non-fast + circuits now only differ from fast ones in that they can use relays + not marked with the Fast flag. This "feature" could turn out to + be a horrible bug; we should investigate more before it goes into + a stable release. + + o Minor features: + - Allow disabling building of the manpages. Skipping the manpage + speeds up the build considerably. + + o Minor bugfixes (on 0.2.2.x): + - Fix a memleak in the EXTENDCIRCUIT logic. Spotted by coverity. + Bugfix on 0.2.2.9-alpha. + - Disallow values larger than INT32_MAX for PerConnBWRate|Burst + config option. Bugfix on 0.2.2.7-alpha. + - Ship the asciidoc-helper file in the tarball, so that people can + build from source if they want to, and touching the .1.txt files + doesn't break the build. Bugfix on 0.2.2.9-alpha. + + o Minor bugfixes (on 0.2.1.x or earlier): + - Fix a dereference-then-NULL-check sequence when publishing + descriptors. Bugfix on 0.2.1.5-alpha. Discovered by ekir; fixes + bug 1255. + - Fix another dereference-then-NULL-check sequence. Bugfix on + 0.2.1.14-rc. Discovered by ekir; fixes bug 1256. + - Make sure we treat potentially not NUL-terminated strings correctly. + Bugfix on 0.1.1.13-alpha. Discovered by rieo; fixes bug 1257. + + o Code simplifications and refactoring: + - Fix some urls in the exit notice file and make it XHTML1.1 strict + compliant. Based on a patch from Christian Kujau. + - Don't use sed in asciidoc-helper anymore. + - Make the build process fail if asciidoc cannot be found and + building with asciidoc isn't disabled. + + +Changes in version 0.2.2.9-alpha - 2010-02-22 + Tor 0.2.2.9-alpha makes Tor work again on the latest OS X, updates the + location of a directory authority, and cleans up a bunch of small bugs. + + o Directory authority changes: + - Change IP address for dannenberg (v3 directory authority), and + remove moria2 (obsolete v1, v2 directory authority and v0 hidden + service directory authority) from the list. + + o Major bugfixes: + - Make Tor work again on the latest OS X: when deciding whether to + use strange flags to turn TLS renegotiation on, detect the OpenSSL + version at run-time, not compile time. We need to do this because + Apple doesn't update its dev-tools headers when it updates its + libraries in a security patch. + - Fix a potential buffer overflow in lookup_last_hid_serv_request() + that could happen on 32-bit platforms with 64-bit time_t. Also fix + a memory leak when requesting a hidden service descriptor we've + requested before. Fixes bug 1242, bugfix on 0.2.0.18-alpha. Found + by aakova. + - Authorities could be tricked into giving out the Exit flag to relays + that didn't allow exiting to any ports. This bug could screw + with load balancing and stats. Bugfix on 0.1.1.6-alpha; fixes bug + 1238. Bug discovered by Martin Kowalczyk. + - When freeing a session key, zero it out completely. We only zeroed + the first ptrsize bytes. Bugfix on 0.0.2pre8. Discovered and + patched by ekir. Fixes bug 1254. + + o Minor bugfixes: + - Fix static compilation by listing the openssl libraries in the right + order. Bugfix on Tor 0.2.2.8-alpha; fixes bug 1237. + - Resume handling .exit hostnames in a special way: originally we + stripped the .exit part and used the requested exit relay. In + 0.2.2.1-alpha we stopped treating them in any special way, meaning + if you use a .exit address then Tor will pass it on to the exit + relay. Now we reject the .exit stream outright, since that behavior + might be more expected by the user. Found and diagnosed by Scott + Bennett and Downie on or-talk. + - Don't spam the controller with events when we have no file + descriptors available. Bugfix on 0.2.1.5-alpha. (Rate-limiting + for log messages was already solved from bug 748.) + - Avoid a bogus overlapped memcpy in tor_addr_copy(). Reported by + "memcpyfail". + - Make the DNSPort option work with libevent 2.x. Don't alter the + behaviour for libevent 1.x. Fixes bug 1143. Found by SwissTorExit. + - Emit a GUARD DROPPED controller event for a case we missed. + - Make more fields in the controller protocol case-insensitive, since + control-spec.txt said they were. + - Refactor resolve_my_address() to not use gethostbyname() anymore. + Fixes bug 1244; bugfix on 0.0.2pre25. Reported by Mike Mestnik. + - Fix a spec conformance issue: the network-status-version token + must be the first token in a v3 consensus or vote. Discovered by + parakeep. Bugfix on 0.2.0.3-alpha. + + o Code simplifications and refactoring: + - Generate our manpage and HTML documentation using Asciidoc. This + change should make it easier to maintain the documentation, and + produce nicer HTML. + - Remove the --enable-iphone option. According to reports from Marco + Bonetti, Tor builds fine without any special tweaking on recent + iPhone SDK versions. + - Removed some unnecessary files from the source distribution. The + AUTHORS file has now been merged into the people page on the + website. The roadmaps and design doc can now be found in the + projects directory in svn. + - Enabled various circuit build timeout constants to be controlled + by consensus parameters. Also set better defaults for these + parameters based on experimentation on broadband and simulated + high latency links. + + o Minor features: + - The 'EXTENDCIRCUIT' control port command can now be used with + a circ id of 0 and no path. This feature will cause Tor to build + a new 'fast' general purpose circuit using its own path selection + algorithms. + - Added a BUILDTIMEOUT_SET controller event to describe changes + to the circuit build timeout. + - Future-proof the controller protocol a bit by ignoring keyword + arguments we do not recognize. + - Expand homedirs passed to tor-checkkey. This should silence a + coverity complaint about passing a user-supplied string into + open() without checking it. + + Changes in version 0.2.1.25 - 2010-03-16 Tor 0.2.1.25 fixes a regression introduced in 0.2.1.23 that could prevent relays from guessing their IP address correctly. It also fixes @@ -22,6 +918,7 @@ Changes in version 0.2.1.25 - 2010-03-16 Bugfix on 0.1.1.13-alpha. Discovered by rieo; fixes bug 1257. + Changes in version 0.2.1.24 - 2010-02-21 Tor 0.2.1.24 makes Tor work again on the latest OS X -- this time for sure! @@ -75,6 +972,154 @@ Changes in version 0.2.1.23 - 2010-02-13 over the behavior introduced in 0.1.2.17. +Changes in version 0.2.2.8-alpha - 2010-01-26 + Tor 0.2.2.8-alpha fixes a crash bug in 0.2.2.7-alpha that has been + causing bridge relays to disappear. If you're running a bridge, + please upgrade. + + o Major bugfixes: + - Fix a memory corruption bug on bridges that occured during the + inclusion of stats data in extra-info descriptors. Also fix the + interface for geoip_get_bridge_stats* to prevent similar bugs in + the future. Diagnosis by Tas, patch by Karsten and Sebastian. + Fixes bug 1208; bugfix on 0.2.2.7-alpha. + + o Minor bugfixes: + - Ignore OutboundBindAddress when connecting to localhost. + Connections to localhost need to come _from_ localhost, or else + local servers (like DNS and outgoing HTTP/SOCKS proxies) will often + refuse to listen. + + +Changes in version 0.2.2.7-alpha - 2010-01-19 + Tor 0.2.2.7-alpha fixes a huge client-side performance bug, as well + as laying the groundwork for further relay-side performance fixes. It + also starts cleaning up client behavior with respect to the EntryNodes, + ExitNodes, and StrictNodes config options. + + This release also rotates two directory authority keys, due to a + security breach of some of the Torproject servers. + + o Directory authority changes: + - Rotate keys (both v3 identity and relay identity) for moria1 + and gabelmoo. + + o Major features (performance): + - We were selecting our guards uniformly at random, and then weighting + which of our guards we'd use uniformly at random. This imbalance + meant that Tor clients were severely limited on throughput (and + probably latency too) by the first hop in their circuit. Now we + select guards weighted by currently advertised bandwidth. We also + automatically discard guards picked using the old algorithm. Fixes + bug 1217; bugfix on 0.2.1.3-alpha. Found by Mike Perry. + - When choosing which cells to relay first, relays can now favor + circuits that have been quiet recently, to provide lower latency + for low-volume circuits. By default, relays enable or disable this + feature based on a setting in the consensus. You can override + this default by using the new "CircuitPriorityHalflife" config + option. Design and code by Ian Goldberg, Can Tang, and Chris + Alexander. + - Add separate per-conn write limiting to go with the per-conn read + limiting. We added a global write limit in Tor 0.1.2.5-alpha, + but never per-conn write limits. + - New consensus params "bwconnrate" and "bwconnburst" to let us + rate-limit client connections as they enter the network. It's + controlled in the consensus so we can turn it on and off for + experiments. It's starting out off. Based on proposal 163. + + o Major features (relay selection options): + - Switch to a StrictNodes config option, rather than the previous + "StrictEntryNodes" / "StrictExitNodes" separation that was missing a + "StrictExcludeNodes" option. + - If EntryNodes, ExitNodes, ExcludeNodes, or ExcludeExitNodes + change during a config reload, mark and discard all our origin + circuits. This fix should address edge cases where we change the + config options and but then choose a circuit that we created before + the change. + - If EntryNodes or ExitNodes are set, be more willing to use an + unsuitable (e.g. slow or unstable) circuit. The user asked for it, + they get it. + - Make EntryNodes config option much more aggressive even when + StrictNodes is not set. Before it would prepend your requested + entrynodes to your list of guard nodes, but feel free to use others + after that. Now it chooses only from your EntryNodes if any of + those are available, and only falls back to others if a) they're + all down and b) StrictNodes is not set. + - Now we refresh your entry guards from EntryNodes at each consensus + fetch -- rather than just at startup and then they slowly rot as + the network changes. + + o Major bugfixes: + - Stop bridge directory authorities from answering dbg-stability.txt + directory queries, which would let people fetch a list of all + bridge identities they track. Bugfix on 0.2.1.6-alpha. + + o Minor features: + - Log a notice when we get a new control connection. Now it's easier + for security-conscious users to recognize when a local application + is knocking on their controller door. Suggested by bug 1196. + - New config option "CircuitStreamTimeout" to override our internal + timeout schedule for how many seconds until we detach a stream from + a circuit and try a new circuit. If your network is particularly + slow, you might want to set this to a number like 60. + - New controller command "getinfo config-text". It returns the + contents that Tor would write if you send it a SAVECONF command, + so the controller can write the file to disk itself. + - New options for SafeLogging to allow scrubbing only log messages + generated while acting as a relay. + - Ship the bridges spec file in the tarball too. + - Avoid a mad rush at the beginning of each month when each client + rotates half of its guards. Instead we spread the rotation out + throughout the month, but we still avoid leaving a precise timestamp + in the state file about when we first picked the guard. Improves + over the behavior introduced in 0.1.2.17. + + o Minor bugfixes (compiling): + - Fix compilation on OS X 10.3, which has a stub mlockall() but + hides it. Bugfix on 0.2.2.6-alpha. + - Fix compilation on Solaris by removing support for the + DisableAllSwap config option. Solaris doesn't have an rlimit for + mlockall, so we cannot use it safely. Fixes bug 1198; bugfix on + 0.2.2.6-alpha. + + o Minor bugfixes (crashes): + - Do not segfault when writing buffer stats when we haven't observed + a single circuit to report about. Found by Fabian Lanze. Bugfix on + 0.2.2.1-alpha. + - If we're in the pathological case where there's no exit bandwidth + but there is non-exit bandwidth, or no guard bandwidth but there + is non-guard bandwidth, don't crash during path selection. Bugfix + on 0.2.0.3-alpha. + - Fix an impossible-to-actually-trigger buffer overflow in relay + descriptor generation. Bugfix on 0.1.0.15. + + o Minor bugfixes (privacy): + - Fix an instance where a Tor directory mirror might accidentally + log the IP address of a misbehaving Tor client. Bugfix on + 0.1.0.1-rc. + - Don't list Windows capabilities in relay descriptors. We never made + use of them, and maybe it's a bad idea to publish them. Bugfix + on 0.1.1.8-alpha. + + o Minor bugfixes (other): + - Resolve an edge case in path weighting that could make us misweight + our relay selection. Fixes bug 1203; bugfix on 0.0.8rc1. + - Fix statistics on client numbers by country as seen by bridges that + were broken in 0.2.2.1-alpha. Also switch to reporting full 24-hour + intervals instead of variable 12-to-48-hour intervals. + - After we free an internal connection structure, overwrite it + with a different memory value than we use for overwriting a freed + internal circuit structure. Should help with debugging. Suggested + by bug 1055. + - Update our OpenSSL 0.9.8l fix so that it works with OpenSSL 0.9.8m + too. + + o Removed features: + - Remove the HSAuthorityRecordStats option that version 0 hidden + service authorities could have used to track statistics of overall + hidden service usage. + + Changes in version 0.2.1.22 - 2010-01-19 Tor 0.2.1.22 fixes a critical privacy problem in bridge directory authorities -- it would tell you its whole history of bridge descriptors @@ -118,6 +1163,69 @@ Changes in version 0.2.1.21 - 2009-12-21 trigger assert. Fixes bug 1173. +Changes in version 0.2.2.6-alpha - 2009-11-19 + Tor 0.2.2.6-alpha lays the groundwork for many upcoming features: + support for the new lower-footprint "microdescriptor" directory design, + future-proofing our consensus format against new hash functions or + other changes, and an Android port. It also makes Tor compatible with + the upcoming OpenSSL 0.9.8l release, and fixes a variety of bugs. + + o Major features: + - Directory authorities can now create, vote on, and serve multiple + parallel formats of directory data as part of their voting process. + Partially implements Proposal 162: "Publish the consensus in + multiple flavors". + - Directory authorities can now agree on and publish small summaries + of router information that clients can use in place of regular + server descriptors. This transition will eventually allow clients + to use far less bandwidth for downloading information about the + network. Begins the implementation of Proposal 158: "Clients + download consensus + microdescriptors". + - The directory voting system is now extensible to use multiple hash + algorithms for signatures and resource selection. Newer formats + are signed with SHA256, with a possibility for moving to a better + hash algorithm in the future. + - New DisableAllSwap option. If set to 1, Tor will attempt to lock all + current and future memory pages via mlockall(). On supported + platforms (modern Linux and probably BSD but not Windows or OS X), + this should effectively disable any and all attempts to page out + memory. This option requires that you start your Tor as root -- + if you use DisableAllSwap, please consider using the User option + to properly reduce the privileges of your Tor. + - Numerous changes, bugfixes, and workarounds from Nathan Freitas + to help Tor build correctly for Android phones. + + o Major bugfixes: + - Work around a security feature in OpenSSL 0.9.8l that prevents our + handshake from working unless we explicitly tell OpenSSL that we + are using SSL renegotiation safely. We are, but OpenSSL 0.9.8l + won't work unless we say we are. + + o Minor bugfixes: + - Fix a crash bug when trying to initialize the evdns module in + Libevent 2. Bugfix on 0.2.1.16-rc. + - Stop logging at severity 'warn' when some other Tor client tries + to establish a circuit with us using weak DH keys. It's a protocol + violation, but that doesn't mean ordinary users need to hear about + it. Fixes the bug part of bug 1114. Bugfix on 0.1.0.13. + - Do not refuse to learn about authority certs and v2 networkstatus + documents that are older than the latest consensus. This bug might + have degraded client bootstrapping. Bugfix on 0.2.0.10-alpha. + Spotted and fixed by xmux. + - Fix numerous small code-flaws found by Coverity Scan Rung 3. + - If all authorities restart at once right before a consensus vote, + nobody will vote about "Running", and clients will get a consensus + with no usable relays. Instead, authorities refuse to build a + consensus if this happens. Bugfix on 0.2.0.10-alpha; fixes bug 1066. + - If your relay can't keep up with the number of incoming create + cells, it would log one warning per failure into your logs. Limit + warnings to 1 per minute. Bugfix on 0.0.2pre10; fixes bug 1042. + - Bridges now use "reject *:*" as their default exit policy. Bugfix + on 0.2.0.3-alpha; fixes bug 1113. + - Fix a memory leak on directory authorities during voting that was + introduced in 0.2.2.1-alpha. Found via valgrind. + + Changes in version 0.2.1.20 - 2009-10-15 Tor 0.2.1.20 fixes a crash bug when you're accessing many hidden services at once, prepares for more performance improvements, and @@ -193,6 +1301,262 @@ Changes in version 0.2.1.20 - 2009-10-15 getinfo and status events until we have a better design for them. +Changes in version 0.2.2.5-alpha - 2009-10-11 + Tor 0.2.2.5-alpha fixes a few compile problems in 0.2.2.4-alpha. + + o Major bugfixes: + - Make the tarball compile again. Oops. Bugfix on 0.2.2.4-alpha. + + o New directory authorities: + - Move dizum to an alternate IP address. + + +Changes in version 0.2.2.4-alpha - 2009-10-10 + Tor 0.2.2.4-alpha fixes more crash bugs in 0.2.2.2-alpha. It also + introduces a new unit test framework, shifts directry authority + addresses around to reduce the impact from recent blocking events, + and fixes a few smaller bugs. + + o Major bugfixes: + - Fix several more asserts in the circuit_build_times code, for + example one that causes Tor to fail to start once we have + accumulated 5000 build times in the state file. Bugfixes on + 0.2.2.2-alpha; fixes bug 1108. + + o New directory authorities: + - Move moria1 and Tonga to alternate IP addresses. + + o Minor features: + - Log SSL state transitions at debug level during handshake, and + include SSL states in error messages. This may help debug future + SSL handshake issues. + - Add a new "Handshake" log domain for activities that happen + during the TLS handshake. + - Revert to the "June 3 2009" ip-to-country file. The September one + seems to have removed most US IP addresses. + - Directory authorities now reject Tor relays with versions less than + 0.1.2.14. This step cuts out four relays from the current network, + none of which are very big. + + o Minor bugfixes: + - Fix a couple of smaller issues with gathering statistics. Bugfixes + on 0.2.2.1-alpha. + - Fix two memory leaks in the error case of + circuit_build_times_parse_state(). Bugfix on 0.2.2.2-alpha. + - Don't count one-hop circuits when we're estimating how long it + takes circuits to build on average. Otherwise we'll set our circuit + build timeout lower than we should. Bugfix on 0.2.2.2-alpha. + - Directory authorities no longer change their opinion of, or vote on, + whether a router is Running, unless they have themselves been + online long enough to have some idea. Bugfix on 0.2.0.6-alpha. + Fixes bug 1023. + + o Code simplifications and refactoring: + - Revise our unit tests to use the "tinytest" framework, so we + can run tests in their own processes, have smarter setup/teardown + code, and so on. The unit test code has moved to its own + subdirectory, and has been split into multiple modules. + + +Changes in version 0.2.2.3-alpha - 2009-09-23 + Tor 0.2.2.3-alpha fixes a few crash bugs in 0.2.2.2-alpha. + + o Major bugfixes: + - Fix an overzealous assert in our new circuit build timeout code. + Bugfix on 0.2.2.2-alpha; fixes bug 1103. + + o Minor bugfixes: + - If the networkstatus consensus tells us that we should use a + negative circuit package window, ignore it. Otherwise we'll + believe it and then trigger an assert. Bugfix on 0.2.2.2-alpha. + + +Changes in version 0.2.2.2-alpha - 2009-09-21 + Tor 0.2.2.2-alpha introduces our latest performance improvement for + clients: Tor tracks the average time it takes to build a circuit, and + avoids using circuits that take too long to build. For fast connections, + this feature can cut your expected latency in half. For slow or flaky + connections, it could ruin your Tor experience. Let us know if it does! + + o Major features: + - Tor now tracks how long it takes to build client-side circuits + over time, and adapts its timeout to local network performance. + Since a circuit that takes a long time to build will also provide + bad performance, we get significant latency improvements by + discarding the slowest 20% of circuits. Specifically, Tor creates + circuits more aggressively than usual until it has enough data + points for a good timeout estimate. Implements proposal 151. + We are especially looking for reports (good and bad) from users with + both EDGE and broadband connections that can move from broadband + to EDGE and find out if the build-time data in the .tor/state gets + reset without loss of Tor usability. You should also see a notice + log message telling you that Tor has reset its timeout. + - Directory authorities can now vote on arbitary integer values as + part of the consensus process. This is designed to help set + network-wide parameters. Implements proposal 167. + - Tor now reads the "circwindow" parameter out of the consensus, + and uses that value for its circuit package window rather than the + default of 1000 cells. Begins the implementation of proposal 168. + + o Major bugfixes: + - Fix a remotely triggerable memory leak when a consensus document + contains more than one signature from the same voter. Bugfix on + 0.2.0.3-alpha. + + o Minor bugfixes: + - Fix an extremely rare infinite recursion bug that could occur if + we tried to log a message after shutting down the log subsystem. + Found by Matt Edman. Bugfix on 0.2.0.16-alpha. + - Fix parsing for memory or time units given without a space between + the number and the unit. Bugfix on 0.2.2.1-alpha; fixes bug 1076. + - A networkstatus vote must contain exactly one signature. Spec + conformance issue. Bugfix on 0.2.0.3-alpha. + - Fix an obscure bug where hidden services on 64-bit big-endian + systems might mis-read the timestamp in v3 introduce cells, and + refuse to connect back to the client. Discovered by "rotor". + Bugfix on 0.2.1.6-alpha. + - We were triggering a CLOCK_SKEW controller status event whenever + we connect via the v2 connection protocol to any relay that has + a wrong clock. Instead, we should only inform the controller when + it's a trusted authority that claims our clock is wrong. Bugfix + on 0.2.0.20-rc; starts to fix bug 1074. Reported by SwissTorExit. + - We were telling the controller about CHECKING_REACHABILITY and + REACHABILITY_FAILED status events whenever we launch a testing + circuit or notice that one has failed. Instead, only tell the + controller when we want to inform the user of overall success or + overall failure. Bugfix on 0.1.2.6-alpha. Fixes bug 1075. Reported + by SwissTorExit. + - Don't warn when we're using a circuit that ends with a node + excluded in ExcludeExitNodes, but the circuit is not used to access + the outside world. This should help fix bug 1090, but more problems + remain. Bugfix on 0.2.1.6-alpha. + - Work around a small memory leak in some versions of OpenSSL that + stopped the memory used by the hostname TLS extension from being + freed. + - Make our 'torify' script more portable; if we have only one of + 'torsocks' or 'tsocks' installed, don't complain to the user; + and explain our warning about tsocks better. + + o Minor features: + - Add a "getinfo status/accepted-server-descriptor" controller + command, which is the recommended way for controllers to learn + whether our server descriptor has been successfully received by at + least on directory authority. Un-recommend good-server-descriptor + getinfo and status events until we have a better design for them. + - Update to the "September 4 2009" ip-to-country file. + + +Changes in version 0.2.2.1-alpha - 2009-08-26 + Tor 0.2.2.1-alpha disables ".exit" address notation by default, allows + Tor clients to bootstrap on networks where only port 80 is reachable, + makes it more straightforward to support hardware crypto accelerators, + and starts the groundwork for gathering stats safely at relays. + + o Security fixes: + - Start the process of disabling ".exit" address notation, since it + can be used for a variety of esoteric application-level attacks + on users. To reenable it, set "AllowDotExit 1" in your torrc. Fix + on 0.0.9rc5. + + o New directory authorities: + - Set up urras (run by Jacob Appelbaum) as the seventh v3 directory + authority. + + o Major features: + - New AccelName and AccelDir options add support for dynamic OpenSSL + hardware crypto acceleration engines. + - Tor now supports tunneling all of its outgoing connections over + a SOCKS proxy, using the SOCKS4Proxy and/or SOCKS5Proxy + configuration options. Code by Christopher Davis. + + o Major bugfixes: + - Send circuit or stream sendme cells when our window has decreased + by 100 cells, not when it has decreased by 101 cells. Bug uncovered + by Karsten when testing the "reduce circuit window" performance + patch. Bugfix on the 54th commit on Tor -- from July 2002, + before the release of Tor 0.0.0. This is the new winner of the + oldest-bug prize. + + o New options for gathering stats safely: + - Directories that set "DirReqStatistics 1" write statistics on + directory request to disk every 24 hours. As compared to the + --enable-geoip-stats flag in 0.2.1.x, there are a few improvements: + 1) stats are written to disk exactly every 24 hours; 2) estimated + shares of v2 and v3 requests are determined as mean values, not at + the end of a measurement period; 3) unresolved requests are listed + with country code '??'; 4) directories also measure download times. + - Exit nodes that set "ExitPortStatistics 1" write statistics on the + number of exit streams and transferred bytes per port to disk every + 24 hours. + - Relays that set "CellStatistics 1" write statistics on how long + cells spend in their circuit queues to disk every 24 hours. + - Entry nodes that set "EntryStatistics 1" write statistics on the + rough number and origins of connecting clients to disk every 24 + hours. + - Relays that write any of the above statistics to disk and set + "ExtraInfoStatistics 1" include the past 24 hours of statistics in + their extra-info documents. + + o Minor features: + - New --digests command-line switch to output the digests of the + source files Tor was built with. + - The "torify" script now uses torsocks where available. + - The memarea code now uses a sentinel value at the end of each area + to make sure nothing writes beyond the end of an area. This might + help debug some conceivable causes of bug 930. + - Time and memory units in the configuration file can now be set to + fractional units. For example, "2.5 GB" is now a valid value for + AccountingMax. + - Certain Tor clients (such as those behind check.torproject.org) may + want to fetch the consensus in an extra early manner. To enable this + a user may now set FetchDirInfoExtraEarly to 1. This also depends on + setting FetchDirInfoEarly to 1. Previous behavior will stay the same + as only certain clients who must have this information sooner should + set this option. + - Instead of adding the svn revision to the Tor version string, report + the git commit (when we're building from a git checkout). + + o Minor bugfixes: + - If any the v3 certs we download are unparseable, we should actually + notice the failure so we don't retry indefinitely. Bugfix on + 0.2.0.x; reported by "rotator". + - If the cached cert file is unparseable, warn but don't exit. + - Fix possible segmentation fault on directory authorities. Bugfix on + 0.2.1.14-rc. + - When Tor fails to parse a descriptor of any kind, dump it to disk. + Might help diagnosing bug 1051. + + o Deprecated and removed features: + - The controller no longer accepts the old obsolete "addr-mappings/" + or "unregistered-servers-" GETINFO values. + - Hidden services no longer publish version 0 descriptors, and clients + do not request or use version 0 descriptors. However, the old hidden + service authorities still accept and serve version 0 descriptors + when contacted by older hidden services/clients. + - The EXTENDED_EVENTS and VERBOSE_NAMES controller features are now + always on; using them is necessary for correct forward-compatible + controllers. + - Remove support for .noconnect style addresses. Nobody was using + them, and they provided another avenue for detecting Tor users + via application-level web tricks. + + o Packaging changes: + - Upgrade Vidalia from 0.1.15 to 0.2.3 in the Windows and OS X + installer bundles. See + https://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.2.3/CHANGELOG + for details of what's new in Vidalia 0.2.3. + - Windows Vidalia Bundle: update Privoxy from 3.0.6 to 3.0.14-beta. + - OS X Vidalia Bundle: move to Polipo 1.0.4 with Tor specific + configuration file, rather than the old Privoxy. + - OS X Vidalia Bundle: Vidalia, Tor, and Polipo are compiled as + x86-only for better compatibility with OS X 10.6, aka Snow Leopard. + - OS X Tor Expert Bundle: Tor is compiled as x86-only for + better compatibility with OS X 10.6, aka Snow Leopard. + - OS X Vidalia Bundle: The multi-package installer is now replaced + by a simple drag and drop to the /Applications folder. This change + occurred with the upgrade to Vidalia 0.2.3. + + Changes in version 0.2.1.19 - 2009-07-28 Tor 0.2.1.19 fixes a major bug with accessing and providing hidden services on Tor 0.2.1.3-alpha through 0.2.1.18. diff --git a/Doxyfile.in b/Doxyfile.in index b4d21c334..344ee2714 100644 --- a/Doxyfile.in +++ b/Doxyfile.in @@ -1,5 +1,4 @@ -# $Id$ -# Doxyfile 1.5.1 +# Doxyfile 1.5.6 # This file describes the settings to be used by the documentation system # doxygen (www.doxygen.org) for a project @@ -15,6 +14,14 @@ # Project related configuration options #--------------------------------------------------------------------------- +# This tag specifies the encoding used for all characters in the config file +# that follow. The default is UTF-8 which is also the encoding used for all +# text before the first occurrence of this tag. Doxygen uses libiconv (or the +# iconv built into libc) for the transcoding. See +# http://www.gnu.org/software/libiconv for the list of possible encodings. + +DOXYFILE_ENCODING = UTF-8 + # The PROJECT_NAME tag is a single word (or a sequence of words surrounded # by quotes) that should identify the project. @@ -47,23 +54,14 @@ CREATE_SUBDIRS = NO # information to generate all constant output in the proper language. # The default language is English, other supported languages are: # Afrikaans, Arabic, Brazilian, Catalan, Chinese, Chinese-Traditional, -# Croatian, Czech, Danish, Dutch, Finnish, French, German, Greek, Hungarian, -# Italian, Japanese, Japanese-en (Japanese with English messages), Korean, -# Korean-en, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Russian, -# Serbian, Slovak, Slovene, Spanish, Swedish, and Ukrainian. +# Croatian, Czech, Danish, Dutch, Farsi, Finnish, French, German, Greek, +# Hungarian, Italian, Japanese, Japanese-en (Japanese with English messages), +# Korean, Korean-en, Lithuanian, Norwegian, Macedonian, Persian, Polish, +# Portuguese, Romanian, Russian, Serbian, Slovak, Slovene, Spanish, Swedish, +# and Ukrainian. OUTPUT_LANGUAGE = English -# This tag can be used to specify the encoding used in the generated output. -# The encoding is not always determined by the language that is chosen, -# but also whether or not the output is meant for Windows or non-Windows users. -# In case there is a difference, setting the USE_WINDOWS_ENCODING tag to YES -# forces the Windows encoding (this is the default for the Windows binary), -# whereas setting the tag to NO uses a Unix-style encoding (the default for -# all platforms other than Windows). - -USE_WINDOWS_ENCODING = NO - # If the BRIEF_MEMBER_DESC tag is set to YES (the default) Doxygen will # include brief member descriptions after the members that are listed in # the file and class documentation (similar to JavaDoc). @@ -136,11 +134,19 @@ SHORT_NAMES = NO # If the JAVADOC_AUTOBRIEF tag is set to YES then Doxygen # will interpret the first line (until the first dot) of a JavaDoc-style # comment as the brief description. If set to NO, the JavaDoc -# comments will behave just like the Qt-style comments (thus requiring an -# explicit @brief command for a brief description. +# comments will behave just like regular Qt-style comments +# (thus requiring an explicit @brief command for a brief description.) JAVADOC_AUTOBRIEF = NO +# If the QT_AUTOBRIEF tag is set to YES then Doxygen will +# interpret the first line (until the first dot) of a Qt-style +# comment as the brief description. If set to NO, the comments +# will behave just like regular Qt-style comments (thus requiring +# an explicit \brief command for a brief description.) + +QT_AUTOBRIEF = NO + # The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make Doxygen # treat a multi-line C++ special comment block (i.e. a block of //! or /// # comments) as a brief description. This used to be the default behaviour. @@ -154,7 +160,7 @@ MULTILINE_CPP_IS_BRIEF = NO # If set to NO, the detailed description appears after the member # documentation. -DETAILS_AT_TOP = NO +# DETAILS_AT_TOP = NO # If the INHERIT_DOCS tag is set to YES (the default) then an undocumented # member inherits the documentation from any documented member that it @@ -190,14 +196,26 @@ ALIASES = OPTIMIZE_OUTPUT_FOR_C = YES # Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java -# sources only. Doxygen will then generate output that is more tailored for Java. -# For instance, namespaces will be presented as packages, qualified scopes -# will look different, etc. +# sources only. Doxygen will then generate output that is more tailored for +# Java. For instance, namespaces will be presented as packages, qualified +# scopes will look different, etc. OPTIMIZE_OUTPUT_JAVA = NO -# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want to -# include (a tag file for) the STL sources as input, then you should +# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran +# sources only. Doxygen will then generate output that is more tailored for +# Fortran. + +OPTIMIZE_FOR_FORTRAN = NO + +# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL +# sources. Doxygen will then generate output that is tailored for +# VHDL. + +OPTIMIZE_OUTPUT_VHDL = NO + +# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want +# to include (a tag file for) the STL sources as input, then you should # set this tag to YES in order to let doxygen match functions declarations and # definitions whose arguments contain STL classes (e.g. func(std::string); v.s. # func(std::string) {}). This also make the inheritance and collaboration @@ -205,6 +223,26 @@ OPTIMIZE_OUTPUT_JAVA = NO BUILTIN_STL_SUPPORT = NO +# If you use Microsoft's C++/CLI language, you should set this option to YES to +# enable parsing support. + +CPP_CLI_SUPPORT = NO + +# Set the SIP_SUPPORT tag to YES if your project consists of sip sources only. +# Doxygen will parse them like normal C++ but will assume all classes use public +# instead of private inheritance when no explicit protection keyword is present. + +SIP_SUPPORT = NO + +# For Microsoft's IDL there are propget and propput attributes to indicate getter +# and setter methods for a property. Setting this option to YES (the default) +# will make doxygen to replace the get and set methods by a property in the +# documentation. This will only work if the methods are indeed getting or +# setting a simple type. If this is not the case, or you want to show the +# methods anyway, you should set this option to NO. + +IDL_PROPERTY_SUPPORT = NO + # If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC # tag is set to YES, then doxygen will reuse the documentation of the first # member in the group (if any) for the other members of the group. By default @@ -220,6 +258,16 @@ DISTRIBUTE_GROUP_DOC = NO SUBGROUPING = YES +# When TYPEDEF_HIDES_STRUCT is enabled, a typedef of a struct, union, or enum +# is documented as struct, union, or enum with the name of the typedef. So +# typedef struct TypeS {} TypeT, will appear in the documentation as a struct +# with name TypeT. When disabled the typedef will appear as a member of a file, +# namespace, or class. And the struct will be named TypeS. This can typically +# be useful for C code in case the coding convention dictates that all compound +# types are typedef'ed and only the typedef is referenced, never the tag name. + +TYPEDEF_HIDES_STRUCT = NO + #--------------------------------------------------------------------------- # Build related configuration options #--------------------------------------------------------------------------- @@ -254,6 +302,14 @@ EXTRACT_LOCAL_CLASSES = YES EXTRACT_LOCAL_METHODS = NO +# If this flag is set to YES, the members of anonymous namespaces will be +# extracted and appear in the documentation as a namespace called +# 'anonymous_namespace{file}', where file will be replaced with the base +# name of the file that contains the anonymous namespace. By default +# anonymous namespace are hidden. + +EXTRACT_ANON_NSPACES = NO + # If the HIDE_UNDOC_MEMBERS tag is set to YES, Doxygen will hide all # undocumented members of documented classes, files or namespaces. # If set to NO (the default) these members will be included in the @@ -329,6 +385,12 @@ SORT_MEMBER_DOCS = YES SORT_BRIEF_DOCS = NO +# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the +# hierarchy of group names into alphabetical order. If set to NO (the default) +# the group names will appear in their defined order. + +SORT_GROUP_NAMES = NO + # If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be # sorted by fully-qualified names, including namespaces. If set to # NO (the default), the class list will be sorted only by class name, @@ -390,9 +452,21 @@ SHOW_USED_FILES = YES SHOW_DIRECTORIES = NO +# Set the SHOW_FILES tag to NO to disable the generation of the Files page. +# This will remove the Files entry from the Quick Index and from the +# Folder Tree View (if specified). The default is YES. + +SHOW_FILES = YES + +# Set the SHOW_NAMESPACES tag to NO to disable the generation of the +# Namespaces page. This will remove the Namespaces entry from the Quick Index +# and from the Folder Tree View (if specified). The default is YES. + +SHOW_NAMESPACES = YES + # The FILE_VERSION_FILTER tag can be used to specify a program or script that -# doxygen should invoke to get the current version for each file (typically from the -# version control system). Doxygen will invoke the program by executing (via +# doxygen should invoke to get the current version for each file (typically from +# the version control system). Doxygen will invoke the program by executing (via # popen()) the command <command> <input-file>, where <command> is the value of # the FILE_VERSION_FILTER tag, and <input-file> is the name of an input file # provided by doxygen. Whatever the program writes to standard output @@ -463,12 +537,20 @@ WARN_LOGFILE = INPUT = src/common \ src/or +# This tag can be used to specify the character encoding of the source files +# that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is +# also the default input encoding. Doxygen uses libiconv (or the iconv built +# into libc) for the transcoding. See http://www.gnu.org/software/libiconv for +# the list of possible encodings. + +INPUT_ENCODING = UTF-8 + # If the value of the INPUT tag contains directories, you can use the # FILE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp # and *.h) to filter out the source-files in the directories. If left # blank the following patterns are tested: # *.c *.cc *.cxx *.cpp *.c++ *.java *.ii *.ixx *.ipp *.i++ *.inl *.h *.hh *.hxx -# *.hpp *.h++ *.idl *.odl *.cs *.php *.php3 *.inc *.m *.mm *.py +# *.hpp *.h++ *.idl *.odl *.cs *.php *.php3 *.inc *.m *.mm *.py *.f90 FILE_PATTERNS = *.c \ *.h @@ -499,6 +581,14 @@ EXCLUDE_SYMLINKS = NO EXCLUDE_PATTERNS = +# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names +# (namespaces, classes, functions, etc.) that should be excluded from the +# output. The symbol name can be a fully qualified name, a word, or if the +# wildcard * is used, a substring. Examples: ANamespace, AClass, +# AClass::ANamespace, ANamespace::*Test + +EXCLUDE_SYMBOLS = + # The EXAMPLE_PATH tag can be used to specify one or more files or # directories that contain example code fragments that are included (see # the \include command). @@ -559,7 +649,7 @@ FILTER_SOURCE_FILES = NO # Note: To get rid of all source code in the generated output, make sure also # VERBATIM_HEADERS is set to NO. -SOURCE_BROWSER = NO +SOURCE_BROWSER = YES # Setting the INLINE_SOURCES tag to YES will include the body # of functions and classes directly in the documentation. @@ -572,13 +662,13 @@ INLINE_SOURCES = NO STRIP_CODE_COMMENTS = YES -# If the REFERENCED_BY_RELATION tag is set to YES (the default) +# If the REFERENCED_BY_RELATION tag is set to YES # then for each documented function all documented # functions referencing it will be listed. REFERENCED_BY_RELATION = YES -# If the REFERENCES_RELATION tag is set to YES (the default) +# If the REFERENCES_RELATION tag is set to YES # then for each documented function all documented entities # called/used by that function will be listed. @@ -678,11 +768,44 @@ HTML_ALIGN_MEMBERS = YES # If the GENERATE_HTMLHELP tag is set to YES, additional index files # will be generated that can be used as input for tools like the -# Microsoft HTML help workshop to generate a compressed HTML help file (.chm) +# Microsoft HTML help workshop to generate a compiled HTML help file (.chm) # of the generated HTML documentation. GENERATE_HTMLHELP = NO +# If the GENERATE_DOCSET tag is set to YES, additional index files +# will be generated that can be used as input for Apple's Xcode 3 +# integrated development environment, introduced with OSX 10.5 (Leopard). +# To create a documentation set, doxygen will generate a Makefile in the +# HTML output directory. Running make will produce the docset in that +# directory and running "make install" will install the docset in +# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find +# it at startup. + +GENERATE_DOCSET = NO + +# When GENERATE_DOCSET tag is set to YES, this tag determines the name of the +# feed. A documentation feed provides an umbrella under which multiple +# documentation sets from a single provider (such as a company or product suite) +# can be grouped. + +DOCSET_FEEDNAME = "Doxygen generated docs for Tor" + +# When GENERATE_DOCSET tag is set to YES, this tag specifies a string that +# should uniquely identify the documentation set bundle. This should be a +# reverse domain-name style string, e.g. com.mycompany.MyDocSet. Doxygen +# will append .docset to the name. + +DOCSET_BUNDLE_ID = org.torproject.Tor + +# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML +# documentation will contain sections that can be hidden and shown after the +# page has loaded. For this to work a browser that supports +# JavaScript and DHTML is required (for instance Mozilla 1.0+, Firefox +# Netscape 6.0+, Internet explorer 5.0+, Konqueror, or Safari). + +HTML_DYNAMIC_SECTIONS = NO + # If the GENERATE_HTMLHELP tag is set to YES, the CHM_FILE tag can # be used to specify the file name of the resulting .chm file. You # can add a path in front of the file if the result should not be @@ -703,6 +826,12 @@ HHC_LOCATION = GENERATE_CHI = NO +# If the GENERATE_HTMLHELP tag is set to YES, the CHM_INDEX_ENCODING +# is used to encode HtmlHelp index (hhk), content (hhc) and project file +# content. + +CHM_INDEX_ENCODING = + # If the GENERATE_HTMLHELP tag is set to YES, the BINARY_TOC flag # controls whether a binary table of contents is generated (YES) or a # normal table of contents (NO) in the .chm file. @@ -725,12 +854,20 @@ DISABLE_INDEX = NO ENUM_VALUES_PER_LINE = 4 -# If the GENERATE_TREEVIEW tag is set to YES, a side panel will be -# generated containing a tree-like index structure (just like the one that +# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index +# structure should be generated to display hierarchical information. +# If the tag value is set to FRAME, a side panel will be generated +# containing a tree-like index structure (just like the one that # is generated for HTML Help). For this to work a browser that supports # JavaScript, DHTML, CSS and frames is required (for instance Mozilla 1.0+, # Netscape 6.0+, Internet explorer 5.0+, or Konqueror). Windows users are -# probably better off using the HTML help feature. +# probably better off using the HTML help feature. Other possible values +# for this tag are: HIERARCHIES, which will generate the Groups, Directories, +# and Class Hiererachy pages using a tree view instead of an ordered list; +# ALL, which combines the behavior of FRAME and HIERARCHIES; and NONE, which +# disables this behavior completely. For backwards compatibility with previous +# releases of Doxygen, the values YES and NO are equivalent to FRAME and NONE +# respectively. GENERATE_TREEVIEW = NO @@ -740,6 +877,14 @@ GENERATE_TREEVIEW = NO TREEVIEW_WIDTH = 250 +# Use this tag to change the font size of Latex formulas included +# as images in the HTML documentation. The default is 10. Note that +# when you change the font size after a successful doxygen run you need +# to manually remove any form_*.png images from the HTML output directory +# to force them to be regenerated. + +FORMULA_FONTSIZE = 10 + #--------------------------------------------------------------------------- # configuration options related to the LaTeX output #--------------------------------------------------------------------------- @@ -1088,6 +1233,15 @@ PERL_PATH = /usr/bin/perl CLASS_DIAGRAMS = YES +# You can define message sequence charts within doxygen comments using the \msc +# command. Doxygen will then run the mscgen tool (see +# http://www.mcternan.me.uk/mscgen/) to produce the chart and insert it in the +# documentation. The MSCGEN_PATH tag allows you to specify the directory where +# the mscgen tool resides. If left empty the tool is assumed to be found in the +# default search path. + +MSCGEN_PATH = + # If set to YES, the inheritance and collaboration graphs will hide # inheritance and usage relations if the target is undocumented # or is not a class. @@ -1101,6 +1255,24 @@ HIDE_UNDOC_RELATIONS = YES HAVE_DOT = NO +# By default doxygen will write a font called FreeSans.ttf to the output +# directory and reference it in all dot files that doxygen generates. This +# font does not include all possible unicode characters however, so when you need +# these (or just want a differently looking font) you can specify the font name +# using DOT_FONTNAME. You need need to make sure dot is able to find the font, +# which can be done by putting it in a standard location or by setting the +# DOTFONTPATH environment variable or by setting DOT_FONTPATH to the directory +# containing the font. + +DOT_FONTNAME = FreeSans + +# By default doxygen will tell dot to use the output directory to look for the +# FreeSans.ttf font (which doxygen will put there itself). If you specify a +# different font using DOT_FONTNAME you can set the path where dot +# can find it using this tag. + +DOT_FONTPATH = + # If the CLASS_GRAPH and HAVE_DOT tags are set to YES then doxygen # will generate a graph for each documented class showing the direct and # indirect inheritance relations. Setting this tag to YES will force the @@ -1145,19 +1317,19 @@ INCLUDE_GRAPH = YES INCLUDED_BY_GRAPH = YES -# If the CALL_GRAPH and HAVE_DOT tags are set to YES then doxygen will -# generate a call dependency graph for every global function or class method. -# Note that enabling this option will significantly increase the time of a run. -# So in most cases it will be better to enable call graphs for selected -# functions only using the \callgraph command. +# If the CALL_GRAPH and HAVE_DOT options are set to YES then +# doxygen will generate a call dependency graph for every global function +# or class method. Note that enabling this option will significantly increase +# the time of a run. So in most cases it will be better to enable call graphs +# for selected functions only using the \callgraph command. CALL_GRAPH = NO -# If the CALLER_GRAPH and HAVE_DOT tags are set to YES then doxygen will -# generate a caller dependency graph for every global function or class method. -# Note that enabling this option will significantly increase the time of a run. -# So in most cases it will be better to enable caller graphs for selected -# functions only using the \callergraph command. +# If the CALLER_GRAPH and HAVE_DOT tags are set to YES then +# doxygen will generate a caller dependency graph for every global function +# or class method. Note that enabling this option will significantly increase +# the time of a run. So in most cases it will be better to enable caller +# graphs for selected functions only using the \callergraph command. CALLER_GRAPH = NO @@ -1190,39 +1362,31 @@ DOT_PATH = DOTFILE_DIRS = -# The MAX_DOT_GRAPH_WIDTH tag can be used to set the maximum allowed width -# (in pixels) of the graphs generated by dot. If a graph becomes larger than -# this value, doxygen will try to truncate the graph, so that it fits within -# the specified constraint. Beware that most browsers cannot cope with very -# large images. - -MAX_DOT_GRAPH_WIDTH = 1024 - -# The MAX_DOT_GRAPH_HEIGHT tag can be used to set the maximum allows height -# (in pixels) of the graphs generated by dot. If a graph becomes larger than -# this value, doxygen will try to truncate the graph, so that it fits within -# the specified constraint. Beware that most browsers cannot cope with very -# large images. +# The DOT_GRAPH_MAX_NODES tag can be used to set the maximum number of +# nodes that will be shown in the graph. If the number of nodes in a graph +# becomes larger than this value, doxygen will truncate the graph, which is +# visualized by representing a node as a red box. Note that doxygen if the +# number of direct children of the root node in a graph is already larger than +# DOT_GRAPH_MAX_NODES then the graph will not be shown at all. Also note +# that the size of a graph can be further restricted by MAX_DOT_GRAPH_DEPTH. -MAX_DOT_GRAPH_HEIGHT = 1024 +DOT_GRAPH_MAX_NODES = 50 # The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the # graphs generated by dot. A depth value of 3 means that only nodes reachable # from the root by following a path via at most 3 edges will be shown. Nodes # that lay further from the root node will be omitted. Note that setting this # option to 1 or 2 may greatly reduce the computation time needed for large -# code bases. Also note that a graph may be further truncated if the graph's -# image dimensions are not sufficient to fit the graph (see MAX_DOT_GRAPH_WIDTH -# and MAX_DOT_GRAPH_HEIGHT). If 0 is used for the depth value (the default), -# the graph is not depth-constrained. +# code bases. Also note that the size of a graph can be further restricted by +# DOT_GRAPH_MAX_NODES. Using a depth of 0 means no depth restriction. MAX_DOT_GRAPH_DEPTH = 0 # Set the DOT_TRANSPARENT tag to YES to generate images with a transparent -# background. This is disabled by default, which results in a white background. -# Warning: Depending on the platform used, enabling this option may lead to -# badly anti-aliased labels on the edges of a graph (i.e. they become hard to -# read). +# background. This is enabled by default, which results in a transparent +# background. Warning: Depending on the platform used, enabling this option +# may lead to badly anti-aliased labels on the edges of a graph (i.e. they +# become hard to read). DOT_TRANSPARENT = NO diff --git a/Makefile.am b/Makefile.am index bcb81e64f..5f1592618 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,4 +1,3 @@ -# $Id$ # Copyright (c) 2001-2004, Roger Dingledine # Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson # Copyright (c) 2007-2011, The Tor Project, Inc. @@ -11,7 +10,7 @@ SUBDIRS = src doc contrib DIST_SUBDIRS = src doc contrib -EXTRA_DIST = INSTALL README AUTHORS LICENSE ChangeLog \ +EXTRA_DIST = INSTALL README LICENSE ChangeLog \ ReleaseNotes tor.spec tor.spec.in #install-data-local: @@ -35,39 +34,22 @@ dist-rpm: mv $$RPM_BUILD_DIR/RPMS/* .; \ rm -rf $$RPM_BUILD_DIR -dist-osx: - @if [ "x$(prefix)" != 'x/Library/Tor' ]; then \ - echo "Configure with --prefix=/Library/Tor, please"; \ - exit 1; \ - fi; \ - if [ "x$(bindir)" != 'x/Library/Tor' ]; then \ - echo "Configure with --bindir=/Library/Tor, please"; \ - exit 1; \ - fi; \ - if [ "x$(sysconfdir)" != 'x/Library' ]; then \ - echo "Configure with --sysconfdir=/Library, please"; \ - exit 1; \ - fi; \ - if [ "x$(CONFDIR)" != 'x/Library/Tor' ]; then \ - echo "Configure with CONFDIR=/Library/Tor, please"; \ - fi - $(MAKE) all - VERSION=$(VERSION) sh ./contrib/osx/package.sh - dist: check doxygen: doxygen && cd doc/doxygen/latex && make -test: - ./src/or/test +test: all + ./src/test/test -# Avoid strlcpy.c, strlcat.c, tree.h +# Avoid strlcpy.c, strlcat.c, aes.c, OpenBSD_malloc_Linux.c, sha256.c, +# eventdns.[hc], tinytest*.[ch] check-spaces: ./contrib/checkSpace.pl -C \ src/common/*.h \ src/common/[^asO]*.c src/common/address.c \ - src/or/[^et]*.[ch] src/or/t*.c src/or/eventdns_tor.h + src/or/[^e]*.[ch] src/or/eventdns_tor.h \ + src/test/test*.[ch] src/tools/*.[ch] check-docs: ./contrib/checkOptionDocs.pl @@ -4,7 +4,7 @@ is reasonably secure, but please ensure you read the instructions and configure it properly. To build Tor from source: - ./configure; make; make install + ./configure && make && make install Home page: https://www.torproject.org/ @@ -19,5 +19,6 @@ Making applications work with Tor: https://wiki.torproject.org/noreply/TheOnionRouter/TorifyHOWTO Frequently Asked Questions: + https://www.torproject.org/faq.html https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ diff --git a/Win32Build/mingw/CHANGES-libevent b/Win32Build/mingw/CHANGES-libevent deleted file mode 100644 index e723d9755..000000000 --- a/Win32Build/mingw/CHANGES-libevent +++ /dev/null @@ -1,80 +0,0 @@ -Changes related to compilation under MinGW/any sane win32 gcc -============================================================= - -* event.c -- If gcc include "WIN32-Code/misc.h" instead of "misc.h" - -* WIN32-Code/misc.h -- Add struct prototypes for timeval and timezone - -* buffer.c -- changed type of "i" from "u_int" to "unsigned int". My MinGW wasn't - recognizing it. (u_int is normally typedef'ed to unsigned int, right?) - -* evbuffer.c -- removed incorrect win32 error checking, see bufferevent_writecb(). - (this needs to be fixed by anyone planning to use evbuffer on win32) - -* log.c -- If gcc include "WIN32-Code/misc.h" instead of "misc.h" - -* WIN32-Code/misc.c -- if gcc, include "misc.h" -- added newline at end of file to shut up gcc - -* WIN32-Code/win32.c -- Altered the prototypes of win32_*() so their argument types didn't conflict - with the function definitions. -- Casted types of win32_* to void inside win32ops so that it didn't conflict - with the definition of eventops (gcc doesn't like this) -- Altered prototype of signal_handler to be static since definition is static - (why wasn't it like this before) -- Casted the second argument of signal() to be void*, some reason my MinGW - doesn't have sighandler_t typedef'ed. - -* configure.in -- some code to check if we are compiling for WIN32. - -* Makefile.am -- if BUILD_WIN32 is defined, include WIN32-Code/misc.c and - WIN32-Code/win32.c as source files. -- if WIN32, do not build test stuff. (not windows friendly) -- if WIN32, explicitly link to ws2_32.dll - -Notes ------ -- We assume that if __GNUC__ is undefined we are building with MSVC -- If the user wishes to build a dll, they are on their own, the syntax is - compiler specific. -- Getting this warning from libtool, no idea why - "libtool: link: warning: undefined symbols not allowed in i686-pc-mingw32 - shared libraries" - - -Changes related to "custom eventops" -==================================== - -* configure.in -- add argument --enable-custom-eventops, sets USE_CUSTOM_EVENTOPS in config.h -- add argument --enable-custom-code, sets USE_CUSTOM_CODE in Makefile - -* Makefile.am -- if USE_CUSTOM_CODE, include custom/custom.c as a source file. - (I can't think of a way to pass a string to Makefile.am, so I'm stuck naming - the new source file custom.c. It just seems simpler this way, but I'm open - to suggestions) - -* event.c -- if USE_CUSTOM_EVENTOPS, use eventops as defined in custom-eventops.h - -Notes ------ -Just in case it isn't completely obvious, the goal of "custom eventops" is to -allow the user to include their own event processing system without requiring a -fork. This is accomplished through two parts. Firstly, by allowing the user to -redefine eventops. (for example, the user may wish to use epoll() exclusively). -Secondly, by allowing the user to include their own code to support a private -eventop (note, this may not be necessary, as the user may choose to include -already defined eventop's. - - diff --git a/Win32Build/mingw/README b/Win32Build/mingw/README deleted file mode 100644 index 67e75cd27..000000000 --- a/Win32Build/mingw/README +++ /dev/null @@ -1,8 +0,0 @@ -The current SVN version of Tor should compile with MinGW. - -OpenSSL and libz both compile on MinGW out of the box. - -libevent 1.1b will not build unless you apply the diff in this directory. - - - diff --git a/Win32Build/mingw/libevent-1.1b-mingw-custom.diff b/Win32Build/mingw/libevent-1.1b-mingw-custom.diff deleted file mode 100644 index 350586e35..000000000 --- a/Win32Build/mingw/libevent-1.1b-mingw-custom.diff +++ /dev/null @@ -1,338 +0,0 @@ -Only in libevent-1.1b: CHANGES -Only in libevent-1.1b: Makefile -diff -uwr libevent-1.1b-old/Makefile.am libevent-1.1b/Makefile.am ---- libevent-1.1b-old/Makefile.am Wed Aug 9 22:16:35 2006 -+++ libevent-1.1b/Makefile.am Sat Sep 2 03:49:26 2006 -@@ -1,6 +1,5 @@ - AUTOMAKE_OPTIONS = foreign no-dependencies - --SUBDIRS = . sample test - - EXTRA_DIST = acconfig.h event.h event-internal.h log.h evsignal.h event.3 \ - kqueue.c epoll_sub.c epoll.c select.c rtsig.c poll.c signal.c \ -@@ -20,8 +19,29 @@ - - lib_LTLIBRARIES = libevent.la - --libevent_la_SOURCES = event.c buffer.c evbuffer.c log.c --libevent_la_LIBADD = @LTLIBOBJS@ -+ -+if BUILD_WIN32 -+ -+SUBDIRS = . sample -+SYS_LIBS = -lws2_32 -+SYS_SRC = WIN32-Code/misc.c WIN32-Code/win32.c -+ -+else -+ -+SUBDIRS = . sample test -+SYS_LIBS = -+SYS_SRC = -+ -+endif -+ -+if USE_CUSTOM_CODE -+CUST_SRC = custom/custom.c -+else -+CUST_SRC = -+endif -+ -+libevent_la_SOURCES = event.c buffer.c evbuffer.c log.c $(CUST_SRC) $(SYS_SRC) -+libevent_la_LIBADD = @LTLIBOBJS@ $(SYS_LIBS) - libevent_la_LDFLAGS = -release @VERSION@ -version-info 1:2:0 - - include_HEADERS = event.h -Only in libevent-1.1b: Makefile.in -diff -uwr libevent-1.1b-old/WIN32-Code/misc.c libevent-1.1b/WIN32-Code/misc.c ---- libevent-1.1b-old/WIN32-Code/misc.c Wed Aug 9 21:01:14 2006 -+++ libevent-1.1b/WIN32-Code/misc.c Fri Sep 1 22:21:31 2006 -@@ -4,6 +4,12 @@ - #include <sys/timeb.h> - #include <time.h> - -+#ifdef __GNUC__ -+/*our prototypes for timeval and timezone are in here, just in case the above -+ headers don't have them*/ -+#include "misc.h" -+#endif -+ - /**************************************************************************** - * - * Function: gettimeofday(struct timeval *, struct timezone *) -diff -uwr libevent-1.1b-old/WIN32-Code/misc.h libevent-1.1b/WIN32-Code/misc.h ---- libevent-1.1b-old/WIN32-Code/misc.h Wed Aug 9 21:01:14 2006 -+++ libevent-1.1b/WIN32-Code/misc.h Fri Sep 1 18:47:09 2006 -@@ -1,6 +1,9 @@ - #ifndef MISC_H - #define MISC_H - -+struct timezone; -+struct timeval; -+ - int gettimeofday(struct timeval *,struct timezone *); - - #endif -diff -uwr libevent-1.1b-old/WIN32-Code/win32.c libevent-1.1b/WIN32-Code/win32.c ---- libevent-1.1b-old/WIN32-Code/win32.c Wed Aug 9 21:25:48 2006 -+++ libevent-1.1b/WIN32-Code/win32.c Sat Sep 2 00:45:55 2006 -@@ -60,7 +60,8 @@ - /* MSDN says this is required to handle SIGFPE */ - volatile double SIGFPE_REQ = 0.0f; - --int signal_handler(int sig); -+static int signal_handler(int sig); -+ - void signal_process(void); - int signal_recalc(void); - -@@ -77,20 +78,21 @@ - }; - - void *win32_init (void); --int win32_insert (void *, struct event *); --int win32_del (void *, struct event *); -+int win32_insert (struct win32op *, struct event *); -+int win32_del (struct win32op *, struct event *); - int win32_recalc (struct event_base *base, void *, int); --int win32_dispatch (struct event_base *base, void *, struct timeval *); -+int win32_dispatch (struct event_base *base, struct win32op *, struct timeval *); - - struct eventop win32ops = { - "win32", - win32_init, -- win32_insert, -- win32_del, -+ (int (*) (void*, struct event*)) win32_insert, -+ (int (*) (void*, struct event*)) win32_del, - win32_recalc, -- win32_dispatch -+ (int (*) (struct event_base*, void*, struct timeval*)) win32_dispatch - }; - -+ - #define FD_SET_ALLOC_SIZE(n) ((sizeof(struct win_fd_set) + ((n)-1)*sizeof(SOCKET))) - - static int -@@ -213,7 +215,13 @@ - if (ev->ev_events & (EV_READ|EV_WRITE)) - event_errx(1, "%s: EV_SIGNAL incompatible use", - __func__); -+ -+#ifndef __GNUC__ - if((int)signal(EVENT_SIGNAL(ev), signal_handler) == -1) -+#else -+ if((int)signal(EVENT_SIGNAL(ev), (void*) signal_handler) == -1) -+#endif -+ - return (-1); - - return (0); -@@ -382,8 +390,13 @@ - - /* Reinstall our signal handler. */ - TAILQ_FOREACH(ev, &signalqueue, ev_signal_next) { -+#ifndef __GNUC__ - if((int)signal(EVENT_SIGNAL(ev), signal_handler) == -1) -+#else -+ if((int)signal(EVENT_SIGNAL(ev), (void*) signal_handler) == -1) -+#endif - return (-1); -+ - } - return (0); - } -Only in libevent-1.1b-old/: aclocal.m4 -Only in libevent-1.1b: autom4te.cache -diff -uwr libevent-1.1b-old/buffer.c libevent-1.1b/buffer.c ---- libevent-1.1b-old/buffer.c Wed Aug 9 22:01:40 2006 -+++ libevent-1.1b/buffer.c Fri Sep 1 18:52:56 2006 -@@ -197,7 +197,7 @@ - u_char *data = EVBUFFER_DATA(buffer); - size_t len = EVBUFFER_LENGTH(buffer); - char *line; -- u_int i; -+ unsigned int i; - - for (i = 0; i < len; i++) { - if (data[i] == '\r' || data[i] == '\n') -Only in libevent-1.1b: config.guess -Only in libevent-1.1b: config.h -diff -uwr libevent-1.1b-old/config.h.in libevent-1.1b/config.h.in ---- libevent-1.1b-old/config.h.in Wed Aug 9 21:27:37 2006 -+++ libevent-1.1b/config.h.in Sat Sep 2 02:23:17 2006 -@@ -223,6 +223,9 @@ - /* Define to 1 if you can safely include both <sys/time.h> and <time.h>. */ - #undef TIME_WITH_SYS_TIME - -+/* Define to 1 if you want to use a custom eventops variable */ -+#undef USE_CUSTOM_EVENTOPS -+ - /* Version number of package */ - #undef VERSION - -@@ -232,11 +235,9 @@ - /* Define to empty if `const' does not conform to ANSI C. */ - #undef const - --/* Define to `__inline__' or `__inline' if that's what the C compiler -- calls it, or to nothing if 'inline' is not supported under any name. */ --#ifndef __cplusplus -+/* Define as `__inline' if that's what the C compiler calls it, or to nothing -+ if it is not supported. */ - #undef inline --#endif - - /* Define to `int' if <sys/types.h> does not define. */ - #undef pid_t -Only in libevent-1.1b: config.h.in~ -Only in libevent-1.1b: config.log -Only in libevent-1.1b: config.status -Only in libevent-1.1b: configure -diff -uwr libevent-1.1b-old/configure.in libevent-1.1b/configure.in ---- libevent-1.1b-old/configure.in Wed Aug 9 22:05:17 2006 -+++ libevent-1.1b/configure.in Sat Sep 2 03:40:15 2006 -@@ -21,6 +21,18 @@ - CFLAGS="$CFLAGS -Wall" - fi - -+AC_ARG_ENABLE(custom-eventops, -+ [ --enable-custom-eventops Use custom eventops variable], -+ AC_DEFINE([USE_CUSTOM_EVENTOPS],[1], -+ [Define to 1 to use a custom eventops variable]) -+ ,) -+AC_ARG_ENABLE(custom-code, -+ [ --enable-custom-code Use custom code from custom/], -+ customcodev=true, -+ customcodev=false) -+ -+AM_CONDITIONAL(USE_CUSTOM_CODE, test x$customcodev = xtrue) -+ - AC_PROG_LIBTOOL - - dnl Uncomment "AC_DISABLE_SHARED" to make shared librraries not get -@@ -110,6 +122,22 @@ - AC_MSG_RESULT(yes)] ,AC_MSG_RESULT(no) - ) - fi -+ -+dnl - check if the macro WIN32 is defined on this compiler. -+dnl - (this is how we check for a windows version of GCC) -+AC_MSG_CHECKING(for WIN32) -+AC_TRY_COMPILE(, -+ [ -+ #ifndef WIN32 -+ #error -+ #endif -+ ], -+ bwin32=true; AC_MSG_RESULT(yes), -+ bwin32=false; AC_MSG_RESULT(no), -+) -+ -+AM_CONDITIONAL(BUILD_WIN32, test x$bwin32 = xtrue) -+ - - dnl Checks for typedefs, structures, and compiler characteristics. - AC_C_CONST -diff -uwr libevent-1.1b-old/evbuffer.c libevent-1.1b/evbuffer.c ---- libevent-1.1b-old/evbuffer.c Wed Aug 9 21:01:14 2006 -+++ libevent-1.1b/evbuffer.c Fri Sep 1 19:18:13 2006 -@@ -154,12 +154,20 @@ - if (EVBUFFER_LENGTH(bufev->output)) { - res = evbuffer_write(bufev->output, fd); - if (res == -1) { -+#ifndef WIN32 -+/*todo. evbuffer uses WriteFile when WIN32 is set. WIN32 system calls do not -+ *set errno. thus this error checking is not portable*/ - if (errno == EAGAIN || - errno == EINTR || - errno == EINPROGRESS) - goto reschedule; - /* error case */ - what |= EVBUFFER_ERROR; -+ -+#else -+ goto reschedule; -+#endif -+ - } else if (res == 0) { - /* eof case */ - what |= EVBUFFER_EOF; -@@ -181,6 +189,7 @@ - return; - - reschedule: -+ - if (EVBUFFER_LENGTH(bufev->output) != 0) - bufferevent_add(&bufev->ev_write, bufev->timeout_write); - return; -diff -uwr libevent-1.1b-old/event.c libevent-1.1b/event.c ---- libevent-1.1b-old/event.c Wed Aug 9 21:25:48 2006 -+++ libevent-1.1b/event.c Sat Sep 2 04:22:05 2006 -@@ -30,8 +30,14 @@ - #define WIN32_LEAN_AND_MEAN - #include <windows.h> - #undef WIN32_LEAN_AND_MEAN -+ -+#ifdef __GNUC__ -+#include "WIN32-Code/misc.h" -+#else - #include "misc.h" - #endif -+ -+#endif - #include <sys/types.h> - #include <sys/tree.h> - #ifdef HAVE_SYS_TIME_H -@@ -53,6 +59,7 @@ - #include "event-internal.h" - #include "log.h" - -+ - #ifdef HAVE_SELECT - extern const struct eventop selectops; - #endif -@@ -75,6 +82,8 @@ - extern const struct eventop win32ops; - #endif - -+#ifndef USE_CUSTOM_EVENTOPS -+ - /* In order of preference */ - const struct eventop *eventops[] = { - #ifdef HAVE_WORKING_KQUEUE -@@ -101,6 +110,11 @@ - NULL - }; - -+#else -+#include "custom-eventops.h" -+#endif //USE_CUSTOM_EVENTOPS -+ -+ - /* Global state */ - struct event_list signalqueue; - -Only in libevent-1.1b: libtool -diff -uwr libevent-1.1b-old/log.c libevent-1.1b/log.c ---- libevent-1.1b-old/log.c Wed Aug 9 21:01:14 2006 -+++ libevent-1.1b/log.c Fri Sep 1 19:09:45 2006 -@@ -45,8 +45,14 @@ - #define WIN32_LEAN_AND_MEAN - #include <windows.h> - #undef WIN32_LEAN_AND_MEAN -+ -+#ifdef __GNUC__ -+#include "WIN32-Code/misc.h" -+#else - #include "misc.h" - #endif -+ -+#endif - #include <sys/types.h> - #include <sys/tree.h> - #ifdef HAVE_SYS_TIME_H -Only in libevent-1.1b/sample: Makefile -Only in libevent-1.1b/sample: Makefile.in -Only in libevent-1.1b: stamp-h1 -Only in libevent-1.1b/test: Makefile -Only in libevent-1.1b/test: Makefile.in diff --git a/Win32Build/mingw/libevent-1.1b-mingw.diff b/Win32Build/mingw/libevent-1.1b-mingw.diff deleted file mode 100644 index c7cea5b32..000000000 --- a/Win32Build/mingw/libevent-1.1b-mingw.diff +++ /dev/null @@ -1,221 +0,0 @@ -=== Makefile.am -================================================================== ---- Makefile.am (revision 8794) -+++ Makefile.am (local) -@@ -1,6 +1,5 @@ - AUTOMAKE_OPTIONS = foreign no-dependencies - --SUBDIRS = . sample test - - EXTRA_DIST = acconfig.h event.h event-internal.h log.h evsignal.h event.3 \ - kqueue.c epoll_sub.c epoll.c select.c rtsig.c poll.c signal.c \ -@@ -20,13 +19,29 @@ - - lib_LTLIBRARIES = libevent.la - --libevent_la_SOURCES = event.c buffer.c evbuffer.c log.c --libevent_la_LIBADD = @LTLIBOBJS@ -+if BUILD_WIN32 -+ -+SUBDIRS = . sample -+SYS_LIBS = -lws2_32 -+SYS_SRC = WIN32-Code/misc.c WIN32-Code/win32.c -+SYS_INCLUDES = -IWIN32-Code -+ -+else -+ -+SUBDIRS = . sample test -+SYS_LIBS = -+SYS_SRC = -+SYS_INCLUDES = -+ -+endif -+ -+libevent_la_SOURCES = event.c buffer.c evbuffer.c log.c $(SYS_SRC) -+libevent_la_LIBADD = @LTLIBOBJS@ $(SYS_LIBS) - libevent_la_LDFLAGS = -release @VERSION@ -version-info 1:2:0 - - include_HEADERS = event.h - --INCLUDES = -Icompat -+INCLUDES = -Icompat $(SYS_INCLUDES) - - man_MANS = event.3 - -=== WIN32-Code/misc.c -================================================================== ---- WIN32-Code/misc.c (revision 8794) -+++ WIN32-Code/misc.c (local) -@@ -4,6 +4,12 @@ - #include <sys/timeb.h> - #include <time.h> - -+#ifdef __GNUC__ -+/*our prototypes for timeval and timezone are in here, just in case the above -+ headers don't have them*/ -+#include "misc.h" -+#endif -+ - /**************************************************************************** - * - * Function: gettimeofday(struct timeval *, struct timezone *) -@@ -17,6 +23,7 @@ - * - ****************************************************************************/ - -+#ifndef HAVE_GETTIMEOFDAY - int gettimeofday(struct timeval *tv, struct timezone *tz) { - struct _timeb tb; - -@@ -28,6 +35,7 @@ - tv->tv_usec = ((int) tb.millitm) * 1000; - return 0; - } -+#endif - - int - win_read(int fd, void *buf, unsigned int length) -=== WIN32-Code/misc.h -================================================================== ---- WIN32-Code/misc.h (revision 8794) -+++ WIN32-Code/misc.h (local) -@@ -1,6 +1,11 @@ - #ifndef MISC_H - #define MISC_H - -+struct timezone; -+struct timeval; -+ -+#ifndef HAVE_GETTIMEOFDAY - int gettimeofday(struct timeval *,struct timezone *); -+#endif - - #endif -=== WIN32-Code/win32.c -================================================================== ---- WIN32-Code/win32.c (revision 8794) -+++ WIN32-Code/win32.c (local) -@@ -60,7 +60,8 @@ - /* MSDN says this is required to handle SIGFPE */ - volatile double SIGFPE_REQ = 0.0f; - --int signal_handler(int sig); -+static void signal_handler(int sig); -+ - void signal_process(void); - int signal_recalc(void); - -@@ -205,8 +206,9 @@ - } - - int --win32_insert(struct win32op *win32op, struct event *ev) -+win32_insert(void *op, struct event *ev) - { -+ struct win32op *win32op = op; - int i; - - if (ev->ev_events & EV_SIGNAL) { -@@ -251,8 +253,9 @@ - } - - int --win32_del(struct win32op *win32op, struct event *ev) -+win32_del(void *op, struct event *ev) - { -+ struct win32op *win32op = op; - int i, found; - - if (ev->ev_events & EV_SIGNAL) -@@ -302,9 +305,10 @@ - */ - - int --win32_dispatch(struct event_base *base, struct win32op *win32op, -+win32_dispatch(struct event_base *base, void *op, - struct timeval *tv) - { -+ struct win32op *win32op = op; - int res = 0; - int i; - int fd_count; -@@ -366,13 +370,11 @@ - } - - --static int -+static void - signal_handler(int sig) - { - evsigcaught[sig]++; - signal_caught = 1; -- -- return 0; - } - - int -=== buffer.c -================================================================== ---- buffer.c (revision 8794) -+++ buffer.c (local) -@@ -197,7 +197,7 @@ - u_char *data = EVBUFFER_DATA(buffer); - size_t len = EVBUFFER_LENGTH(buffer); - char *line; -- u_int i; -+ unsigned int i; - - for (i = 0; i < len; i++) { - if (data[i] == '\r' || data[i] == '\n') -=== configure.in -================================================================== ---- configure.in (revision 8794) -+++ configure.in (local) -@@ -111,6 +111,21 @@ - ) - fi - -+dnl - check if the macro WIN32 is defined on this compiler. -+dnl - (this is how we check for a windows version of GCC) -+AC_MSG_CHECKING(for WIN32) -+AC_TRY_COMPILE(, -+ [ -+ #ifndef WIN32 -+ #error -+ #endif -+ ], -+ bwin32=true; AC_MSG_RESULT(yes), -+ bwin32=false; AC_MSG_RESULT(no), -+) -+ -+AM_CONDITIONAL(BUILD_WIN32, test x$bwin32 = xtrue) -+ - dnl Checks for typedefs, structures, and compiler characteristics. - AC_C_CONST - AC_C_INLINE -=== evbuffer.c -================================================================== ---- evbuffer.c (revision 8794) -+++ evbuffer.c (local) -@@ -154,12 +154,20 @@ - if (EVBUFFER_LENGTH(bufev->output)) { - res = evbuffer_write(bufev->output, fd); - if (res == -1) { -+#ifndef WIN32 -+/*todo. evbuffer uses WriteFile when WIN32 is set. WIN32 system calls do not -+ *set errno. thus this error checking is not portable*/ - if (errno == EAGAIN || - errno == EINTR || - errno == EINPROGRESS) - goto reschedule; - /* error case */ - what |= EVBUFFER_ERROR; -+ -+#else -+ goto reschedule; -+#endif -+ - } else if (res == 0) { - /* eof case */ - what |= EVBUFFER_EOF; - diff --git a/Win32Build/mingw/libevent-svn-mingw.diff b/Win32Build/mingw/libevent-svn-mingw.diff deleted file mode 100644 index af4ffbbbb..000000000 --- a/Win32Build/mingw/libevent-svn-mingw.diff +++ /dev/null @@ -1,210 +0,0 @@ -=== Makefile.am -================================================================== ---- Makefile.am (revision 8794) -+++ Makefile.am (local) -@@ -1,6 +1,5 @@ - AUTOMAKE_OPTIONS = foreign no-dependencies - --SUBDIRS = . sample test - - bin_SCRIPTS = event_rpcgen.py - -@@ -22,18 +21,34 @@ - - lib_LTLIBRARIES = libevent.la - -+if BUILD_WIN32 -+ -+SUBDIRS = . sample -+SYS_LIBS = -lws2_32 -+SYS_SRC = WIN32-Code/misc.c WIN32-Code/win32.c -+SYS_INCLUDES = -IWIN32-Code -+ -+else -+ -+SUBDIRS = . sample test -+SYS_LIBS = -+SYS_SRC = -+SYS_INCLUDES = -+ -+endif -+ - libevent_la_SOURCES = event.c buffer.c evbuffer.c log.c event_tagging.c \ -- http.c evhttp.h http-internal.h evdns.c evdns.h --libevent_la_LIBADD = @LTLIBOBJS@ -+ http.c evhttp.h http-internal.h evdns.c evdns.h $(SYS_SRC) -+libevent_la_LIBADD = @LTLIBOBJS@ $(SYS_LIBS) - libevent_la_LDFLAGS = -release @VERSION@ -version-info 1:3:0 - - include_HEADERS = event.h evhttp.h evdns.h - --INCLUDES = -Icompat -+INCLUDES = -Icompat $(SYS_INCLUDES) - - man_MANS = event.3 - - verify: libevent.la -- cd $(srcdir)/test && make verify -+ cd $(srcdir)/test && make verify - - DISTCLEANFILES = *~ -=== WIN32-Code/misc.c -================================================================== ---- WIN32-Code/misc.c (revision 8794) -+++ WIN32-Code/misc.c (local) -@@ -4,6 +4,12 @@ - #include <sys/timeb.h> - #include <time.h> - -+#ifdef __GNUC__ -+/*our prototypes for timeval and timezone are in here, just in case the above -+ headers don't have them*/ -+#include "misc.h" -+#endif -+ - /**************************************************************************** - * - * Function: gettimeofday(struct timeval *, struct timezone *) -=== WIN32-Code/misc.h -================================================================== ---- WIN32-Code/misc.h (revision 8794) -+++ WIN32-Code/misc.h (local) -@@ -1,6 +1,9 @@ - #ifndef MISC_H - #define MISC_H - -+struct timezone; -+struct timeval; -+ - int gettimeofday(struct timeval *,struct timezone *); - - #endif -=== WIN32-Code/win32.c -================================================================== ---- WIN32-Code/win32.c (revision 8794) -+++ WIN32-Code/win32.c (local) -@@ -60,7 +60,8 @@ - /* MSDN says this is required to handle SIGFPE */ - volatile double SIGFPE_REQ = 0.0f; - --int signal_handler(int sig); -+static void signal_handler(int sig); -+ - void signal_process(void); - int signal_recalc(void); - -@@ -207,8 +208,9 @@ - } - - int --win32_insert(struct win32op *win32op, struct event *ev) -+win32_insert(void *op, struct event *ev) - { -+ struct win32op *win32op = op; - int i; - - if (ev->ev_events & EV_SIGNAL) { -@@ -253,8 +255,9 @@ - } - - int --win32_del(struct win32op *win32op, struct event *ev) -+win32_del(void *op, struct event *ev) - { -+ struct win32op *win32op = op; - int i, found; - - if (ev->ev_events & EV_SIGNAL) -@@ -304,9 +307,10 @@ - */ - - int --win32_dispatch(struct event_base *base, struct win32op *win32op, -+win32_dispatch(struct event_base *base, void *op, - struct timeval *tv) - { -+ struct win32op *win32op = op; - int res = 0; - int i; - int fd_count; -@@ -389,13 +393,11 @@ - free(win32op); - } - --static int -+static void - signal_handler(int sig) - { - evsigcaught[sig]++; - signal_caught = 1; -- -- return 0; - } - - int -=== buffer.c -================================================================== ---- buffer.c (revision 8794) -+++ buffer.c (local) -@@ -197,7 +197,7 @@ - u_char *data = EVBUFFER_DATA(buffer); - size_t len = EVBUFFER_LENGTH(buffer); - char *line; -- u_int i; -+ unsigned int i; - - for (i = 0; i < len; i++) { - if (data[i] == '\r' || data[i] == '\n') -=== configure.in -================================================================== ---- configure.in (revision 8794) -+++ configure.in (local) -@@ -111,6 +111,22 @@ - ) - fi - -+dnl - check if the macro WIN32 is defined on this compiler. -+dnl - (this is how we check for a windows version of GCC) -+AC_MSG_CHECKING(for WIN32) -+AC_TRY_COMPILE(, -+ [ -+ #ifndef WIN32 -+ #error -+ #endif -+ ], -+ bwin32=true; AC_MSG_RESULT(yes), -+ bwin32=false; AC_MSG_RESULT(no), -+) -+ -+AM_CONDITIONAL(BUILD_WIN32, test x$bwin32 = xtrue) -+ -+ - dnl Checks for typedefs, structures, and compiler characteristics. - AC_C_CONST - AC_C_INLINE -=== evbuffer.c -================================================================== ---- evbuffer.c (revision 8794) -+++ evbuffer.c (local) -@@ -163,12 +162,20 @@ - if (EVBUFFER_LENGTH(bufev->output)) { - res = evbuffer_write(bufev->output, fd); - if (res == -1) { -+#ifndef WIN32 -+/*todo. evbuffer uses WriteFile when WIN32 is set. WIN32 system calls do not -+ *set errno. thus this error checking is not portable*/ - if (errno == EAGAIN || - errno == EINTR || - errno == EINPROGRESS) - goto reschedule; - /* error case */ - what |= EVBUFFER_ERROR; -+ -+#else -+ goto reschedule; -+#endif -+ - } else if (res == 0) { - /* eof case */ - what |= EVBUFFER_EOF; - diff --git a/Win32Build/vc6/Tor.dsw b/Win32Build/vc6/Tor.dsw deleted file mode 100644 index 3999da18c..000000000 --- a/Win32Build/vc6/Tor.dsw +++ /dev/null @@ -1,41 +0,0 @@ -Microsoft Developer Studio Workspace File, Format Version 6.00 -# WARNING: DO NOT EDIT OR DELETE THIS WORKSPACE FILE! - -############################################################################### - -Project: "tor"=".\tor\tor.dsp" - Package Owner=<4> - -Package=<5> -{{{ -}}} - -Package=<4> -{{{ -}}} - -############################################################################### - -Project: "tor_resolve"=".\tor_resolve\tor_resolve.dsp" - Package Owner=<4> - -Package=<5> -{{{ -}}} - -Package=<4> -{{{ -}}} - -############################################################################### - -Global: - -Package=<5> -{{{ -}}} - -Package=<3> -{{{ -}}} - -############################################################################### - diff --git a/Win32Build/vc6/tor/Tor.dsp b/Win32Build/vc6/tor/Tor.dsp deleted file mode 100644 index f05e04931..000000000 --- a/Win32Build/vc6/tor/Tor.dsp +++ /dev/null @@ -1,398 +0,0 @@ -# Microsoft Developer Studio Project File - Name="tor" - Package Owner=<4> -# Microsoft Developer Studio Generated Build File, Format Version 6.00 -# ** DO NOT EDIT ** - -# TARGTYPE "Win32 (x86) Console Application" 0x0103 - -CFG=tor - Win32 Debug -!MESSAGE This is not a valid makefile. To build this project using NMAKE, -!MESSAGE use the Export Makefile command and run -!MESSAGE -!MESSAGE NMAKE /f "tor.mak". -!MESSAGE -!MESSAGE You can specify a configuration when running NMAKE -!MESSAGE by defining the macro CFG on the command line. For example: -!MESSAGE -!MESSAGE NMAKE /f "tor.mak" CFG="tor - Win32 Debug" -!MESSAGE -!MESSAGE Possible choices for configuration are: -!MESSAGE -!MESSAGE "tor - Win32 Release" (based on "Win32 (x86) Console Application") -!MESSAGE "tor - Win32 Debug" (based on "Win32 (x86) Console Application") -!MESSAGE - -# Begin Project -# PROP AllowPerConfigDependencies 0 -# PROP Scc_ProjName "" -# PROP Scc_LocalPath "" -CPP=cl.exe -RSC=rc.exe - -!IF "$(CFG)" == "tor - Win32 Release" - -# PROP BASE Use_MFC 0 -# PROP BASE Use_Debug_Libraries 0 -# PROP BASE Output_Dir "Release" -# PROP BASE Intermediate_Dir "Release" -# PROP BASE Target_Dir "" -# PROP Use_MFC 0 -# PROP Use_Debug_Libraries 0 -# PROP Output_Dir "Release" -# PROP Intermediate_Dir "Release" -# PROP Ignore_Export_Lib 0 -# PROP Target_Dir "" -# ADD BASE CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c -# ADD CPP /nologo /MT /W3 /GX /O2 /I "..\..\..\src\win32" /I "c:\openssl\include" /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c -# ADD BASE RSC /l 0x409 /d "NDEBUG" -# ADD RSC /l 0x409 /d "NDEBUG" -BSC32=bscmake.exe -# ADD BASE BSC32 /nologo -# ADD BSC32 /nologo -LINK32=link.exe -# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386 -# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib wsock32.lib ssleay32.lib libeay32.lib /nologo /subsystem:console /machine:I386 /libpath:"c:\openssl\lib\vc" - -!ELSEIF "$(CFG)" == "tor - Win32 Debug" - -# PROP BASE Use_MFC 0 -# PROP BASE Use_Debug_Libraries 1 -# PROP BASE Output_Dir "Debug" -# PROP BASE Intermediate_Dir "Debug" -# PROP BASE Target_Dir "" -# PROP Use_MFC 0 -# PROP Use_Debug_Libraries 1 -# PROP Output_Dir "Debug" -# PROP Intermediate_Dir "Debug" -# PROP Ignore_Export_Lib 0 -# PROP Target_Dir "" -# ADD BASE CPP /nologo /W3 /Gm /GX /ZI /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /GZ /c -# ADD CPP /nologo /MTd /W3 /Gm /GX /ZI /Od /I "..\..\..\src\win32" /I "c:\openssl\include" /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /GZ /c -# ADD BASE RSC /l 0x409 /d "_DEBUG" -# ADD RSC /l 0x409 /d "_DEBUG" -BSC32=bscmake.exe -# ADD BASE BSC32 /nologo -# ADD BSC32 /nologo -LINK32=link.exe -# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept -# ADD LINK32 wsock32.lib ssleay32.lib libeay32.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept /libpath:"c:\openssl\lib\vc" - -!ENDIF - -# Begin Target - -# Name "tor - Win32 Release" -# Name "tor - Win32 Debug" -# Begin Group "Source Files" - -# PROP Default_Filter "cpp;c;cxx;rc;def;r;odl;idl;hpj;bat" -# Begin Group "common" - -# PROP Default_Filter "" -# Begin Source File - -SOURCE=..\..\..\src\common\aes.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\compat.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\container.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\crypto.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\log.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\log.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\torgzip.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\tortls.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\util.c -# End Source File -# End Group -# Begin Group "zlib" - -# PROP Default_Filter "" -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\adler32.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\compress.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\crc32.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\crc32.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\deflate.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\deflate.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\gzio.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\infback.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inffast.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inffast.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inffixed.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inflate.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inflate.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inftrees.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\inftrees.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\trees.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\trees.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\uncompr.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\zconf.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\zlib.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\zutil.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\contrib\zlib\zutil.h -# End Source File -# End Group -# Begin Group "or" - -# PROP Default_Filter "" -# Begin Source File - -SOURCE=..\..\..\src\or\buffers.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\circuitbuild.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\circuitlist.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\circuituse.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\command.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\config.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\connection.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\connection_edge.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\connection_or.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\control.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\cpuworker.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\directory.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\dirserv.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\dns.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\hibernate.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\main.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\onion.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\policies.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\relay.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\rendclient.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\rendcommon.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\rendmid.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\rendservice.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\rephist.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\router.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\routerlist.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\routerparse.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\tor_main.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\tree.h -# End Source File -# End Group -# End Group -# Begin Group "Header Files" - -# PROP Default_Filter "h;hpp;hxx;hm;inl" -# Begin Source File - -SOURCE=..\..\..\src\common\aes.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\compat.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\container.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\crypto.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\fakepoll.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\or\or.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\win32\orconfig.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\test.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\torgzip.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\torint.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\tortls.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\util.h -# End Source File -# End Group -# Begin Group "Resource Files" - -# PROP Default_Filter "ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe" -# End Group -# End Target -# End Project diff --git a/Win32Build/vc6/tor_resolve/tor_resolve.dsp b/Win32Build/vc6/tor_resolve/tor_resolve.dsp deleted file mode 100644 index 63f406b9e..000000000 --- a/Win32Build/vc6/tor_resolve/tor_resolve.dsp +++ /dev/null @@ -1,134 +0,0 @@ -# Microsoft Developer Studio Project File - Name="tor_resolve" - Package Owner=<4> -# Microsoft Developer Studio Generated Build File, Format Version 6.00 -# ** DO NOT EDIT ** - -# TARGTYPE "Win32 (x86) Console Application" 0x0103 - -CFG=tor_resolve - Win32 Debug -!MESSAGE This is not a valid makefile. To build this project using NMAKE, -!MESSAGE use the Export Makefile command and run -!MESSAGE -!MESSAGE NMAKE /f "tor_resolve.mak". -!MESSAGE -!MESSAGE You can specify a configuration when running NMAKE -!MESSAGE by defining the macro CFG on the command line. For example: -!MESSAGE -!MESSAGE NMAKE /f "tor_resolve.mak" CFG="tor_resolve - Win32 Debug" -!MESSAGE -!MESSAGE Possible choices for configuration are: -!MESSAGE -!MESSAGE "tor_resolve - Win32 Release" (based on "Win32 (x86) Console Application") -!MESSAGE "tor_resolve - Win32 Debug" (based on "Win32 (x86) Console Application") -!MESSAGE - -# Begin Project -# PROP AllowPerConfigDependencies 0 -# PROP Scc_ProjName "" -# PROP Scc_LocalPath "" -CPP=cl.exe -RSC=rc.exe - -!IF "$(CFG)" == "tor_resolve - Win32 Release" - -# PROP BASE Use_MFC 0 -# PROP BASE Use_Debug_Libraries 0 -# PROP BASE Output_Dir "Release" -# PROP BASE Intermediate_Dir "Release" -# PROP BASE Target_Dir "" -# PROP Use_MFC 0 -# PROP Use_Debug_Libraries 0 -# PROP Output_Dir "Release" -# PROP Intermediate_Dir "Release" -# PROP Ignore_Export_Lib 0 -# PROP Target_Dir "" -# ADD BASE CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c -# ADD CPP /nologo /MT /W3 /GX /O2 /I "..\..\..\src\win32" /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c -# ADD BASE RSC /l 0x409 /d "NDEBUG" -# ADD RSC /l 0x409 /d "NDEBUG" -BSC32=bscmake.exe -# ADD BASE BSC32 /nologo -# ADD BSC32 /nologo -LINK32=link.exe -# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386 -# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib wsock32.lib /nologo /subsystem:console /machine:I386 - -!ELSEIF "$(CFG)" == "tor_resolve - Win32 Debug" - -# PROP BASE Use_MFC 0 -# PROP BASE Use_Debug_Libraries 1 -# PROP BASE Output_Dir "Debug" -# PROP BASE Intermediate_Dir "Debug" -# PROP BASE Target_Dir "" -# PROP Use_MFC 0 -# PROP Use_Debug_Libraries 1 -# PROP Output_Dir "Debug" -# PROP Intermediate_Dir "Debug" -# PROP Ignore_Export_Lib 0 -# PROP Target_Dir "" -# ADD BASE CPP /nologo /W3 /Gm /GX /ZI /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /GZ /c -# ADD CPP /nologo /MTd /W3 /Gm /GX /ZI /Od /I "..\..\..\src\win32" /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /GZ /c -# ADD BASE RSC /l 0x409 /d "_DEBUG" -# ADD RSC /l 0x409 /d "_DEBUG" -BSC32=bscmake.exe -# ADD BASE BSC32 /nologo -# ADD BSC32 /nologo -LINK32=link.exe -# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept -# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib wsock32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept - -!ENDIF - -# Begin Target - -# Name "tor_resolve - Win32 Release" -# Name "tor_resolve - Win32 Debug" -# Begin Group "Source Files" - -# PROP Default_Filter "cpp;c;cxx;rc;def;r;odl;idl;hpj;bat" -# Begin Source File - -SOURCE=..\..\..\src\common\compat.c -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\log.c -# End Source File -# Begin Source File - -SOURCE="..\..\..\src\tools\tor-resolve.c" -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\util.c -# End Source File -# End Group -# Begin Group "Header Files" - -# PROP Default_Filter "h;hpp;hxx;hm;inl" -# Begin Source File - -SOURCE=..\..\..\src\common\compat.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\log.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\win32\orconfig.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\torint.h -# End Source File -# Begin Source File - -SOURCE=..\..\..\src\common\util.h -# End Source File -# End Group -# Begin Group "Resource Files" - -# PROP Default_Filter "ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe" -# End Group -# End Target -# End Project diff --git a/Win32Build/vc7/Tor/Tor.sln b/Win32Build/vc7/Tor/Tor.sln deleted file mode 100644 index 3087e26cc..000000000 --- a/Win32Build/vc7/Tor/Tor.sln +++ /dev/null @@ -1,45 +0,0 @@ -Microsoft Visual Studio Solution File, Format Version 8.00 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "Tor", "Tor.vcproj", "{63A6B170-E742-400C-B3A0-9CCED3699043}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tor_resolve", "..\tor_resolve\tor_resolve.vcproj", "{E2D2762A-26BD-4A28-BD72-DDAB181324B4}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "unittests", "..\unittests\unittests.vcproj", "{F1F64693-11A9-4992-8B4B-2A67C07BD8C8}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libevent", "..\..\..\contrib\libevent\WIN32-Prj\libevent.vcproj", "{52BBFCA6-6F82-4596-BBAD-0BCFBC637B80}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Global - GlobalSection(SolutionConfiguration) = preSolution - Debug = Debug - Release = Release - EndGlobalSection - GlobalSection(ProjectConfiguration) = postSolution - {63A6B170-E742-400C-B3A0-9CCED3699043}.Debug.ActiveCfg = Debug|Win32 - {63A6B170-E742-400C-B3A0-9CCED3699043}.Debug.Build.0 = Debug|Win32 - {63A6B170-E742-400C-B3A0-9CCED3699043}.Release.ActiveCfg = Release|Win32 - {63A6B170-E742-400C-B3A0-9CCED3699043}.Release.Build.0 = Release|Win32 - {E2D2762A-26BD-4A28-BD72-DDAB181324B4}.Debug.ActiveCfg = Debug|Win32 - {E2D2762A-26BD-4A28-BD72-DDAB181324B4}.Debug.Build.0 = Debug|Win32 - {E2D2762A-26BD-4A28-BD72-DDAB181324B4}.Release.ActiveCfg = Release|Win32 - {E2D2762A-26BD-4A28-BD72-DDAB181324B4}.Release.Build.0 = Release|Win32 - {F1F64693-11A9-4992-8B4B-2A67C07BD8C8}.Debug.ActiveCfg = Debug|Win32 - {F1F64693-11A9-4992-8B4B-2A67C07BD8C8}.Debug.Build.0 = Debug|Win32 - {F1F64693-11A9-4992-8B4B-2A67C07BD8C8}.Release.ActiveCfg = Release|Win32 - {F1F64693-11A9-4992-8B4B-2A67C07BD8C8}.Release.Build.0 = Release|Win32 - {52BBFCA6-6F82-4596-BBAD-0BCFBC637B80}.Debug.ActiveCfg = Debug|Win32 - {52BBFCA6-6F82-4596-BBAD-0BCFBC637B80}.Debug.Build.0 = Debug|Win32 - {52BBFCA6-6F82-4596-BBAD-0BCFBC637B80}.Release.ActiveCfg = Release|Win32 - {52BBFCA6-6F82-4596-BBAD-0BCFBC637B80}.Release.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(ExtensibilityGlobals) = postSolution - EndGlobalSection - GlobalSection(ExtensibilityAddIns) = postSolution - EndGlobalSection -EndGlobal diff --git a/Win32Build/vc7/Tor/Tor.vcproj b/Win32Build/vc7/Tor/Tor.vcproj deleted file mode 100644 index 307521e89..000000000 --- a/Win32Build/vc7/Tor/Tor.vcproj +++ /dev/null @@ -1,357 +0,0 @@ -<?xml version="1.0" encoding="Windows-1252"?> -<VisualStudioProject - ProjectType="Visual C++" - Version="7.10" - Name="Tor" - ProjectGUID="{63A6B170-E742-400C-B3A0-9CCED3699043}" - Keyword="Win32Proj"> - <Platforms> - <Platform - Name="Win32"/> - </Platforms> - <Configurations> - <Configuration - Name="Debug|Win32" - OutputDirectory="Debug" - IntermediateDirectory="Debug" - ConfigurationType="1" - CharacterSet="2"> - <Tool - Name="VCCLCompilerTool" - Optimization="0" - AdditionalIncludeDirectories="c:\openssl\include;..\..\..\src\win32;..\..\..\contrib\libevent" - PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE" - IgnoreStandardIncludePath="FALSE" - MinimalRebuild="TRUE" - BasicRuntimeChecks="3" - RuntimeLibrary="1" - UsePrecompiledHeader="0" - WarningLevel="3" - Detect64BitPortabilityProblems="FALSE" - DebugInformationFormat="4" - CompileAs="1"/> - <Tool - Name="VCCustomBuildTool"/> - <Tool - Name="VCLinkerTool" - AdditionalDependencies="wsock32.lib t:\openssl\install\lib\vc\ssleay32.lib t:\openssl\install\lib\vc\libeay32.lib ..\..\..\contrib\libevent\win32-prj\Debug\libevent.lib ws2_32.lib" - OutputFile="$(OutDir)/Tor.exe" - LinkIncremental="2" - IgnoreDefaultLibraryNames="LIBCD" - DelayLoadDLLs="advapi32.dll" - GenerateDebugInformation="TRUE" - ProgramDatabaseFile="$(OutDir)/Tor.pdb" - SubSystem="1" - TargetMachine="1"/> - <Tool - Name="VCMIDLTool"/> - <Tool - Name="VCPostBuildEventTool"/> - <Tool - Name="VCPreBuildEventTool"/> - <Tool - Name="VCPreLinkEventTool"/> - <Tool - Name="VCResourceCompilerTool"/> - <Tool - Name="VCWebServiceProxyGeneratorTool"/> - <Tool - Name="VCXMLDataGeneratorTool"/> - <Tool - Name="VCWebDeploymentTool"/> - <Tool - Name="VCManagedWrapperGeneratorTool"/> - <Tool - Name="VCAuxiliaryManagedWrapperGeneratorTool"/> - </Configuration> - <Configuration - Name="Release|Win32" - OutputDirectory="Release" - IntermediateDirectory="Release" - ConfigurationType="1" - CharacterSet="2"> - <Tool - Name="VCCLCompilerTool" - AdditionalIncludeDirectories="c:\openssl\include;..\..\..\src\win32;..\..\..\contrib\libevent" - PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE" - IgnoreStandardIncludePath="FALSE" - RuntimeLibrary="0" - UsePrecompiledHeader="0" - WarningLevel="3" - Detect64BitPortabilityProblems="TRUE" - DebugInformationFormat="3" - CompileAs="1"/> - <Tool - Name="VCCustomBuildTool"/> - <Tool - Name="VCLinkerTool" - AdditionalDependencies="wsock32.lib c:\openssl\lib\vc\ssleay32.lib c:\openssl\lib\vc\libeay32.lib" - OutputFile="$(OutDir)/Tor.exe" - LinkIncremental="1" - DelayLoadDLLs="advapi32.dll" - GenerateDebugInformation="TRUE" - SubSystem="1" - OptimizeReferences="2" - EnableCOMDATFolding="2" - TargetMachine="1"/> - <Tool - Name="VCMIDLTool"/> - <Tool - Name="VCPostBuildEventTool"/> - <Tool - Name="VCPreBuildEventTool"/> - <Tool - Name="VCPreLinkEventTool"/> - <Tool - Name="VCResourceCompilerTool"/> - <Tool - Name="VCWebServiceProxyGeneratorTool"/> - <Tool - Name="VCXMLDataGeneratorTool"/> - <Tool - Name="VCWebDeploymentTool"/> - <Tool - Name="VCManagedWrapperGeneratorTool"/> - <Tool - Name="VCAuxiliaryManagedWrapperGeneratorTool"/> - </Configuration> - </Configurations> - <References> - </References> - <Files> - <Filter - Name="Source Files" - Filter="cpp;c;cxx;def;odl;idl;hpj;bat;asm;asmx" - UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"> - <File - RelativePath="..\..\..\src\common\aes.c"> - </File> - <File - RelativePath="..\..\..\src\or\buffers.c"> - </File> - <File - RelativePath="..\..\..\src\or\circuitbuild.c"> - </File> - <File - RelativePath="..\..\..\src\or\circuitlist.c"> - </File> - <File - RelativePath="..\..\..\src\or\circuituse.c"> - </File> - <File - RelativePath="..\..\..\src\or\command.c"> - </File> - <File - RelativePath="..\..\..\src\common\compat.c"> - </File> - <File - RelativePath="..\..\..\src\or\config.c"> - </File> - <File - RelativePath="..\..\..\src\or\connection.c"> - </File> - <File - RelativePath="..\..\..\src\or\connection_edge.c"> - </File> - <File - RelativePath="..\..\..\src\or\connection_or.c"> - </File> - <File - RelativePath="..\..\..\src\common\container.c"> - </File> - <File - RelativePath="..\..\..\src\or\control.c"> - </File> - <File - RelativePath="..\..\..\src\or\cpuworker.c"> - </File> - <File - RelativePath="..\..\..\src\common\crypto.c"> - </File> - <File - RelativePath="..\..\..\src\or\directory.c"> - </File> - <File - RelativePath="..\..\..\src\or\dirserv.c"> - </File> - <File - RelativePath="..\..\..\src\or\dns.c"> - </File> - <File - RelativePath="..\..\..\src\or\hibernate.c"> - </File> - <File - RelativePath="..\..\..\src\common\log.c"> - </File> - <File - RelativePath="..\..\..\src\or\main.c"> - </File> - <File - RelativePath="..\..\..\src\or\onion.c"> - </File> - <File - RelativePath="..\..\..\src\or\or.h"> - </File> - <File - RelativePath="..\..\..\src\win32\orconfig.h"> - </File> - <File - RelativePath="..\..\..\src\or\policies.c"> - </File> - <File - RelativePath="..\..\..\src\or\relay.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendclient.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendcommon.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendmid.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendservice.c"> - </File> - <File - RelativePath="..\..\..\src\or\rephist.c"> - </File> - <File - RelativePath="..\..\..\src\or\router.c"> - </File> - <File - RelativePath="..\..\..\src\or\routerlist.c"> - </File> - <File - RelativePath="..\..\..\src\or\routerparse.c"> - </File> - <File - RelativePath="..\..\..\src\or\tor_main.c"> - </File> - <File - RelativePath="..\..\..\src\common\torgzip.c"> - </File> - <File - RelativePath="..\..\..\src\common\torint.h"> - </File> - <File - RelativePath="..\..\..\src\common\tortls.c"> - </File> - <File - RelativePath="..\..\..\src\common\util.c"> - </File> - <Filter - Name="zlib" - Filter=""> - <File - RelativePath="..\..\..\contrib\zlib\adler32.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\compress.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\crc32.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\crc32.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\deflate.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\deflate.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\gzio.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\infback.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inffast.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inffast.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inffixed.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inflate.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inflate.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inftrees.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inftrees.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\trees.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\trees.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\uncompr.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zconf.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zlib.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zutil.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zutil.h"> - </File> - </Filter> - </Filter> - <Filter - Name="Header Files" - Filter="h;hpp;hxx;hm;inl;inc;xsd" - UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"> - <File - RelativePath="..\..\..\src\common\aes.h"> - </File> - <File - RelativePath="..\..\..\src\common\compat.h"> - </File> - <File - RelativePath="..\..\..\src\common\container.h"> - </File> - <File - RelativePath="..\..\..\src\common\crypto.h"> - </File> - <File - RelativePath="..\..\..\src\common\log.h"> - </File> - <File - RelativePath="..\..\..\src\common\torgzip.h"> - </File> - <File - RelativePath="..\..\..\src\common\tortls.h"> - </File> - <File - RelativePath="..\..\..\src\or\tree.h"> - </File> - <File - RelativePath="..\..\..\src\common\util.h"> - </File> - </Filter> - <Filter - Name="Resource Files" - Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx" - UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}"> - </Filter> - <File - RelativePath=".\ReadMe.txt"> - </File> - </Files> - <Globals> - </Globals> -</VisualStudioProject> diff --git a/Win32Build/vc7/tor_resolve/tor_resolve.vcproj b/Win32Build/vc7/tor_resolve/tor_resolve.vcproj deleted file mode 100644 index a44f1ac66..000000000 --- a/Win32Build/vc7/tor_resolve/tor_resolve.vcproj +++ /dev/null @@ -1,169 +0,0 @@ -<?xml version="1.0" encoding="Windows-1252"?> -<VisualStudioProject - ProjectType="Visual C++" - Version="7.10" - Name="tor_resolve" - ProjectGUID="{E2D2762A-26BD-4A28-BD72-DDAB181324B4}" - Keyword="Win32Proj"> - <Platforms> - <Platform - Name="Win32"/> - </Platforms> - <Configurations> - <Configuration - Name="Debug|Win32" - OutputDirectory="Debug" - IntermediateDirectory="Debug" - ConfigurationType="1" - CharacterSet="2"> - <Tool - Name="VCCLCompilerTool" - Optimization="0" - AdditionalIncludeDirectories="..\..\..\contrib\libevent;..\..\..\src\win32" - PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE" - MinimalRebuild="TRUE" - BasicRuntimeChecks="3" - RuntimeLibrary="1" - UsePrecompiledHeader="0" - WarningLevel="3" - Detect64BitPortabilityProblems="FALSE" - DebugInformationFormat="4"/> - <Tool - Name="VCCustomBuildTool"/> - <Tool - Name="VCLinkerTool" - AdditionalDependencies="wsock32.lib ..\..\..\contrib\libevent\win32-prj\Debug\libevent.lib" - OutputFile="$(OutDir)/tor_resolve.exe" - LinkIncremental="2" - GenerateDebugInformation="TRUE" - ProgramDatabaseFile="$(OutDir)/tor_resolve.pdb" - SubSystem="1" - TargetMachine="1"/> - <Tool - Name="VCMIDLTool"/> - <Tool - Name="VCPostBuildEventTool"/> - <Tool - Name="VCPreBuildEventTool"/> - <Tool - Name="VCPreLinkEventTool"/> - <Tool - Name="VCResourceCompilerTool"/> - <Tool - Name="VCWebServiceProxyGeneratorTool"/> - <Tool - Name="VCXMLDataGeneratorTool"/> - <Tool - Name="VCWebDeploymentTool"/> - <Tool - Name="VCManagedWrapperGeneratorTool"/> - <Tool - Name="VCAuxiliaryManagedWrapperGeneratorTool"/> - </Configuration> - <Configuration - Name="Release|Win32" - OutputDirectory="Release" - IntermediateDirectory="Release" - ConfigurationType="1" - CharacterSet="2"> - <Tool - Name="VCCLCompilerTool" - AdditionalIncludeDirectories="..\..\..\contrib\libevent;..\..\..\src\win32" - PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE" - RuntimeLibrary="0" - UsePrecompiledHeader="0" - WarningLevel="3" - Detect64BitPortabilityProblems="FALSE" - DebugInformationFormat="3"/> - <Tool - Name="VCCustomBuildTool"/> - <Tool - Name="VCLinkerTool" - AdditionalDependencies="wsock32.lib ..\..\..\contrib\libevent\win32-prj\Debug\libevent.lib" - OutputFile="$(OutDir)/tor_resolve.exe" - LinkIncremental="1" - GenerateDebugInformation="TRUE" - SubSystem="1" - OptimizeReferences="2" - EnableCOMDATFolding="2" - TargetMachine="1"/> - <Tool - Name="VCMIDLTool"/> - <Tool - Name="VCPostBuildEventTool"/> - <Tool - Name="VCPreBuildEventTool"/> - <Tool - Name="VCPreLinkEventTool"/> - <Tool - Name="VCResourceCompilerTool"/> - <Tool - Name="VCWebServiceProxyGeneratorTool"/> - <Tool - Name="VCXMLDataGeneratorTool"/> - <Tool - Name="VCWebDeploymentTool"/> - <Tool - Name="VCManagedWrapperGeneratorTool"/> - <Tool - Name="VCAuxiliaryManagedWrapperGeneratorTool"/> - </Configuration> - </Configurations> - <References> - </References> - <Files> - <Filter - Name="Source Files" - Filter="cpp;c;cxx;def;odl;idl;hpj;bat;asm;asmx" - UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"> - <File - RelativePath="..\..\..\src\common\compat.c"> - </File> - <File - RelativePath="..\..\..\src\common\compat.h"> - </File> - <File - RelativePath="..\..\..\src\common\container.c"> - </File> - <File - RelativePath="..\..\..\src\common\container.h"> - </File> - <File - RelativePath="..\..\..\src\common\log.c"> - </File> - <File - RelativePath="..\..\..\src\common\log.h"> - </File> - <File - RelativePath="..\..\..\src\win32\orconfig.h"> - </File> - <File - RelativePath="..\..\..\src\tools\tor-resolve.c"> - </File> - <File - RelativePath="..\..\..\src\common\torint.h"> - </File> - <File - RelativePath="..\..\..\src\common\util.c"> - </File> - <File - RelativePath="..\..\..\src\common\util.h"> - </File> - </Filter> - <Filter - Name="Header Files" - Filter="h;hpp;hxx;hm;inl;inc;xsd" - UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"> - </Filter> - <Filter - Name="Resource Files" - Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx" - UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}"> - </Filter> - <File - RelativePath=".\ReadMe.txt"> - </File> - </Files> - <Globals> - </Globals> -</VisualStudioProject> diff --git a/Win32Build/vc7/unittests/unittests.vcproj b/Win32Build/vc7/unittests/unittests.vcproj deleted file mode 100644 index 3dffa786c..000000000 --- a/Win32Build/vc7/unittests/unittests.vcproj +++ /dev/null @@ -1,342 +0,0 @@ -<?xml version="1.0" encoding="Windows-1252"?> -<VisualStudioProject - ProjectType="Visual C++" - Version="7.10" - Name="unittests" - ProjectGUID="{F1F64693-11A9-4992-8B4B-2A67C07BD8C8}" - Keyword="Win32Proj"> - <Platforms> - <Platform - Name="Win32"/> - </Platforms> - <Configurations> - <Configuration - Name="Debug|Win32" - OutputDirectory="Debug" - IntermediateDirectory="Debug" - ConfigurationType="1" - CharacterSet="2"> - <Tool - Name="VCCLCompilerTool" - Optimization="0" - AdditionalIncludeDirectories="..\..\..\src\win32;c:\openssl\include;..\..\..\contrib\libevent" - PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE" - MinimalRebuild="TRUE" - BasicRuntimeChecks="3" - RuntimeLibrary="1" - UsePrecompiledHeader="0" - WarningLevel="3" - Detect64BitPortabilityProblems="FALSE" - DebugInformationFormat="4"/> - <Tool - Name="VCCustomBuildTool"/> - <Tool - Name="VCLinkerTool" - AdditionalDependencies="wsock32.lib t:\openssl\install\lib\vc\libeay32.lib t:\openssl\install\lib\vc\ssleay32.lib ws2_32.lib ..\..\..\contrib\libevent\win32-prj\Debug\libevent.lib" - OutputFile="$(OutDir)/unittests.exe" - LinkIncremental="2" - IgnoreDefaultLibraryNames="LIBCD" - GenerateDebugInformation="TRUE" - ProgramDatabaseFile="$(OutDir)/unittests.pdb" - SubSystem="1" - TargetMachine="1"/> - <Tool - Name="VCMIDLTool"/> - <Tool - Name="VCPostBuildEventTool"/> - <Tool - Name="VCPreBuildEventTool"/> - <Tool - Name="VCPreLinkEventTool"/> - <Tool - Name="VCResourceCompilerTool"/> - <Tool - Name="VCWebServiceProxyGeneratorTool"/> - <Tool - Name="VCXMLDataGeneratorTool"/> - <Tool - Name="VCWebDeploymentTool"/> - <Tool - Name="VCManagedWrapperGeneratorTool"/> - <Tool - Name="VCAuxiliaryManagedWrapperGeneratorTool"/> - </Configuration> - <Configuration - Name="Release|Win32" - OutputDirectory="Release" - IntermediateDirectory="Release" - ConfigurationType="1" - CharacterSet="2"> - <Tool - Name="VCCLCompilerTool" - AdditionalIncludeDirectories="..\..\..\src\win32;c:\openssl\include;..\..\..\contrib\libevent" - PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE" - RuntimeLibrary="0" - UsePrecompiledHeader="0" - WarningLevel="3" - Detect64BitPortabilityProblems="FALSE" - DebugInformationFormat="3"/> - <Tool - Name="VCCustomBuildTool"/> - <Tool - Name="VCLinkerTool" - AdditionalDependencies="wsock32.lib c:\openssl\lib\vc\libeay32.lib c:\openssl\lib\vc\ssleay32.lib" - OutputFile="$(OutDir)/unittests.exe" - LinkIncremental="1" - GenerateDebugInformation="TRUE" - SubSystem="1" - OptimizeReferences="2" - EnableCOMDATFolding="2" - TargetMachine="1"/> - <Tool - Name="VCMIDLTool"/> - <Tool - Name="VCPostBuildEventTool"/> - <Tool - Name="VCPreBuildEventTool"/> - <Tool - Name="VCPreLinkEventTool"/> - <Tool - Name="VCResourceCompilerTool"/> - <Tool - Name="VCWebServiceProxyGeneratorTool"/> - <Tool - Name="VCXMLDataGeneratorTool"/> - <Tool - Name="VCWebDeploymentTool"/> - <Tool - Name="VCManagedWrapperGeneratorTool"/> - <Tool - Name="VCAuxiliaryManagedWrapperGeneratorTool"/> - </Configuration> - </Configurations> - <References> - </References> - <Files> - <Filter - Name="Source Files" - Filter="cpp;c;cxx;def;odl;idl;hpj;bat;asm;asmx" - UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"> - <File - RelativePath="..\..\..\src\common\aes.c"> - </File> - <File - RelativePath="..\..\..\src\or\buffers.c"> - </File> - <File - RelativePath="..\..\..\src\or\circuitbuild.c"> - </File> - <File - RelativePath="..\..\..\src\or\circuitlist.c"> - </File> - <File - RelativePath="..\..\..\src\or\circuituse.c"> - </File> - <File - RelativePath="..\..\..\src\or\command.c"> - </File> - <File - RelativePath="..\..\..\src\common\compat.c"> - </File> - <File - RelativePath="..\..\..\src\or\config.c"> - </File> - <File - RelativePath="..\..\..\src\or\connection.c"> - </File> - <File - RelativePath="..\..\..\src\or\connection_edge.c"> - </File> - <File - RelativePath="..\..\..\src\or\connection_or.c"> - </File> - <File - RelativePath="..\..\..\src\common\container.c"> - </File> - <File - RelativePath="..\..\..\src\or\control.c"> - </File> - <File - RelativePath="..\..\..\src\or\cpuworker.c"> - </File> - <File - RelativePath="..\..\..\src\common\crypto.c"> - </File> - <File - RelativePath="..\..\..\src\or\directory.c"> - </File> - <File - RelativePath="..\..\..\src\or\dirserv.c"> - </File> - <File - RelativePath="..\..\..\src\or\dns.c"> - </File> - <File - RelativePath="..\..\..\src\or\hibernate.c"> - </File> - <File - RelativePath="..\..\..\src\common\log.c"> - </File> - <File - RelativePath="..\..\..\src\or\main.c"> - </File> - <File - RelativePath="..\..\..\src\or\onion.c"> - </File> - <File - RelativePath="..\..\..\src\or\or.h"> - </File> - <File - RelativePath="..\..\..\src\win32\orconfig.h"> - </File> - <File - RelativePath="..\..\..\src\or\policies.c"> - </File> - <File - RelativePath="..\..\..\src\or\relay.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendclient.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendcommon.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendmid.c"> - </File> - <File - RelativePath="..\..\..\src\or\rendservice.c"> - </File> - <File - RelativePath="..\..\..\src\or\rephist.c"> - </File> - <File - RelativePath="..\..\..\src\or\router.c"> - </File> - <File - RelativePath="..\..\..\src\or\routerlist.c"> - </File> - <File - RelativePath="..\..\..\src\or\routerparse.c"> - </File> - <File - RelativePath="..\..\..\src\or\test.c"> - </File> - <File - RelativePath="..\..\..\src\common\torgzip.c"> - </File> - <File - RelativePath="..\..\..\src\common\torint.h"> - </File> - <File - RelativePath="..\..\..\src\common\tortls.c"> - </File> - <File - RelativePath="..\..\..\src\common\util.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zutil.h"> - </File> - <Filter - Name="zlib"> - <File - RelativePath="..\..\..\contrib\zlib\adler32.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\compress.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\crc32.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\crc32.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\deflate.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\deflate.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\gzio.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\infback.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inffast.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inffast.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inffixed.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inflate.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inflate.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inftrees.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\inftrees.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\trees.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\trees.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\uncompr.c"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zconf.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zlib.h"> - </File> - <File - RelativePath="..\..\..\contrib\zlib\zutil.c"> - </File> - </Filter> - </Filter> - <Filter - Name="Header Files" - Filter="h;hpp;hxx;hm;inl;inc;xsd" - UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"> - <File - RelativePath="..\..\..\src\common\aes.h"> - </File> - <File - RelativePath="..\..\..\src\common\compat.h"> - </File> - <File - RelativePath="..\..\..\src\common\container.h"> - </File> - <File - RelativePath="..\..\..\src\common\crypto.h"> - </File> - <File - RelativePath="..\..\..\src\common\log.h"> - </File> - <File - RelativePath="..\..\..\src\common\torgzip.h"> - </File> - <File - RelativePath="..\..\..\src\common\tortls.h"> - </File> - <File - RelativePath="..\..\..\src\or\tree.h"> - </File> - <File - RelativePath="..\..\..\src\common\util.h"> - </File> - </Filter> - </Files> - <Globals> - </Globals> -</VisualStudioProject> diff --git a/acinclude.m4 b/acinclude.m4 index 76e992572..ccfecff28 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -1,4 +1,3 @@ -dnl $Id$ dnl Helper macros for Tor configure.in dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson @@ -122,13 +121,13 @@ AC_CACHE_CHECK([for $1 directory], tor_cv_library_$1_dir, [ # Can we link against (but not necessarily run, or find the headers for) # the binary? - AC_LINK_IFELSE(AC_LANG_PROGRAM([$5], [$6]), + AC_LINK_IFELSE([AC_LANG_PROGRAM([$5], [$6])], [linkable=yes], [linkable=no]) if test "$linkable" = yes; then tor_$1_any_linkable=yes # Okay, we can link against it. Can we find the headers? - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([$4], [$6]), + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([$4], [$6])], [buildable=yes], [buildable=no]) if test "$buildable" = yes; then tor_cv_library_$1_dir=$tor_trydir @@ -180,7 +179,7 @@ if test "$cross_compiling" != yes; then else LDFLAGS="$tor_tryextra $orig_LDFLAGS" fi - AC_RUN_IFELSE(AC_LANG_PROGRAM([$5], [$6]), + AC_RUN_IFELSE([AC_LANG_PROGRAM([$5], [$6])], [runnable=yes], [runnable=no]) if test "$runnable" = yes; then tor_cv_library_$1_linker_option=$tor_tryextra @@ -212,7 +211,7 @@ dnl dnl TOR_CHECK_PROTYPE(1:functionname, 2:macroname, 2: includes) AC_DEFUN([TOR_CHECK_PROTOTYPE], [ AC_CACHE_CHECK([for declaration of $1], tor_cv_$1_declared, [ - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([$3],[void *ptr= $1 ;]), + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([$3],[void *ptr= $1 ;])], tor_cv_$1_declared=yes,tor_cv_$1_declared=no)]) if test x$tor_cv_$1_declared != xno ; then AC_DEFINE($2, 1, diff --git a/changes/1863_bwhist b/changes/1863_bwhist new file mode 100644 index 000000000..b94250906 --- /dev/null +++ b/changes/1863_bwhist @@ -0,0 +1,16 @@ + o Minor features + - Servers now save observed maximum bandwidth throughput rates + to their state file (along with total usage, which was already + saved) so that they can determine their correct estimated + bandwidth on restart. Resolves bug 1863, where Tor servers + would reset their estimated bandwidth to 0 after restarting. + + o Minor bugfixes + - Fix a bug in banwidth history state parsing that could have been + triggered if a future version of Tor ever changed the timing + granularity at which bandwidth history is measured. Bugfix on + Tor 0.1.1.11-alpha. + - Correctly clear out dir_read/dir_write history when there is an + error parsing any bw history value from the state file. Bugfix on + Tor 0.2.2.15-alpha. + diff --git a/changes/annotations_fix b/changes/annotations_fix deleted file mode 100644 index b259a306d..000000000 --- a/changes/annotations_fix +++ /dev/null @@ -1,10 +0,0 @@ - o Major bugfixes - - Do even more to reject (and not just ignore) annotations on - router descriptors received anywhere but from the cache. - Previously we would ignore such annotations at first, but cache - them to disk anyway. Bugfix on 0.2.0.8-alpha. Found by piebeer. - - o Minor bugfixes - - Enforce multiplicity rules when parsing annotations. Bugfix on - 0.2.0.8-alpha. Found by piebeer. - diff --git a/changes/bug1035 b/changes/bug1035 new file mode 100644 index 000000000..3d86330e6 --- /dev/null +++ b/changes/bug1035 @@ -0,0 +1,13 @@ + o Minor features (authorities) + - Take altered router IP addresses and ORPorts into account when + determining router stability. Previously, if a router changed + its IP or ORPort, the authorities would not treat it as having + any downtime for the purposes of stability calculation, whereas + clients would experience downtime since the change could take a + while to propagate to them. Resolves issue 1035. + o Minor bugfixes (authorities) + - Try to be more robust to hops back in time when calculating + router stability. Previously, if a run of uptime or downtime + appeared to be negative, the calculation could give incorrect + results. Bugfix on 0.2.0.6-alpha. + diff --git a/changes/bug1090-general b/changes/bug1090-general new file mode 100644 index 000000000..465631592 --- /dev/null +++ b/changes/bug1090-general @@ -0,0 +1,73 @@ + o Major features and bugfixes (node selection) + + - Revise and unify the meaning of the ExitNodes, EntryNodes, + ExcludeEntryNodes, ExcludeExitNodes, ExcludeNodes, and + StrictNodes options. Previously, we had been ambiguous in + describing what counted as an "exit" node, and what operations + exactly "StrictNodes 0" would permit. This created confusion + when people saw nodes built through unexpected circuits, and + made it hard to tell real bugs from surprises. We now stipulate + that the intended behavior is: + + . "Exit", in the context of ExitNodes and ExcludeExitNodes, + means a node that delivers user traffic outside the Tor + network. + . "Entry", in the context of EntryNodes and ExcludeEntryNodes, + means a node used as the first hop of a multihop circuit: + it doesn't include direct connections to directory servers. + . "ExcludeNodes" applies to all nodes. + . "StrictNodes" changes the behavior of ExcludeNodes only. + When StrictNodes is set, Tor should avoid all nodes listed + in ExcludeNodes, even when it will make user requests + fail. When StrictNodes is *not* set, then Tor should + follow ExcludeNodes whenever it can, except when it must + use an excluded node to perform self-tests, connect to a + hidden service, provide a hidden service, fulfill a .exit + request, upload directory information, or fetch directory + information. + + Collectively, the changes to implement the behavior are a fix for + bug 1090. + + - ExcludeNodes now takes precedence over EntryNodes and ExitNodes: + if a node is listed in both, it's treated as excluded. + + - ExcludeNodes now applies to directory nodes: as a preference if + StrictNodes is 0, or an absolute requirement if StrictNodes is 1. + (Don't exclude all the directory authorities and set StrictNodes + to 1 unless you really want your Tor to break.) + + - ExcludeNodes and ExcludeExitNodes now override exit enclaving. + + - ExcludeExitNodes now overrides .exit requests. + + - We don't use bridges from ExcludeNodes. + + - When StrictNodes is 1: + . We now apply ExcludeNodes to hidden service introduction points + and to rendezvous points selected by hidden service users. + This can make your hidden service less reliable: use it with + caution! + . If we have used ExcludeNodes on ourself, do not try self-tests. + . If we have excluded all the directory authorities, we will + not even try to upload our descriptor if we're a server. + . Do not honor .exit requests to an excluded node. + + - Remove a misfeature that caused us to ignore the Fast/Stable flags + if ExitNodes was set. Bugfix on 0.2.2.7-alpha. + + - When the set of permitted nodes changes, we now remove any + mappings introduced via TrackExitHosts to now-excluded nodes. + Bugfix on 0.1.0.1-rc. + + - We never cannibalize a circuit that had excluded nodes on it, + even if StrictNodes is 0. Bugfix on 0.1.0.1-rc. + + - Improve log messages related to excluded nodes. + + - Revert a change where we would be laxer about attaching streams to + circuits than when building the circuits. This was meant to + prevent a set of bugs where streams were never attachable, but our + improved code here should make this unnecessary. Bugfix on + 0.2.2.7-alpha. + diff --git a/changes/bug1090-launch-warning b/changes/bug1090-launch-warning new file mode 100644 index 000000000..3f3fbcb4d --- /dev/null +++ b/changes/bug1090-launch-warning @@ -0,0 +1,5 @@ + o Minor features: + - Keep track of how many times we launch a new circuit to handle + a given stream. Too many launches could indicate an inconsistency + between our "launch a circuit to handle this stream" logic and our + "attach our stream to one of the available circuits" logic. diff --git a/changes/bug1125 b/changes/bug1125 deleted file mode 100644 index 1331246a1..000000000 --- a/changes/bug1125 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes - - Do not log messages to the controller while shrinking buffer - freelists. Doing so would sometimes make the controller - connection try to allocate a buffer chunk, which would mess - up the internals of the freelist and cause an assertion - failure. Fixes bug 1125; fixed by Robert Ransom. Bugfix on - Tor 0.2.0.16-alpha. - diff --git a/changes/bug1141 b/changes/bug1141 deleted file mode 100644 index 9975e418d..000000000 --- a/changes/bug1141 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Fix an assertion failure that could occur in caches or bridge users - when using a very short voting interval on a testing network. - Diagnosed by Robert Hogan. Fixes bug 1141; bugfix on 0.2.0.8-alpha. - diff --git a/changes/bug1840 b/changes/bug1840 deleted file mode 100644 index 0ef2b98a3..000000000 --- a/changes/bug1840 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes: - - Allow handshaking OR connections to take a full KeepalivePeriod - seconds to handshake. Previously, we would close them after - IDLE_OR_CONN_TIMEOUT seconds, as if they were open. This is a - bugfix on 0.2.1.26. Thanks to mingw-san for analysis help. Fixes - bug 1840. - diff --git a/changes/bug1859 b/changes/bug1859 deleted file mode 100644 index 5b139f357..000000000 --- a/changes/bug1859 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes: - - Bring the logic that gathers routerinfos and assesses the - acceptability of circuits into line. This prevents a Tor OP from getting - locked in a cycle of choosing its local OR as an exit for a path (due to - a .exit request) and then rejecting the circuit because its OR is not - listed yet. Also prevent Tor clients from using an OR running in the same - instance as an exit (due to a .exit request) if the OR does not meet the - same requirements expected of an OR running elsewhere. - Fixes bug 1859; bugfix on 0.2.0-alpha. diff --git a/changes/bug1981 b/changes/bug1981 deleted file mode 100644 index 3e5e1d36f..000000000 --- a/changes/bug1981 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - When you use bridges and your network goes away and your bridges - get marked as down, recover when you attempt a new socks connection - (if the network is back) rather than waiting up to an hour to try - fetching new descriptors for your bridges. Bugfix on 0.2.0.3-alpha; - fixes bug 1981. diff --git a/changes/bug2004 b/changes/bug2004 new file mode 100644 index 000000000..4fd6c91a0 --- /dev/null +++ b/changes/bug2004 @@ -0,0 +1,4 @@ + o Minor features + - Log less aggressively about circuit timeout changes, and improve some + other circuit timeout messages. Resolves bug 2004. + diff --git a/changes/bug2050 b/changes/bug2050 deleted file mode 100644 index 3e45d3463..000000000 --- a/changes/bug2050 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Learn our external IP address when we're a relay or bridge, even if - we set PublishServerDescriptor to 0. Bugfix on 0.2.0.3-alpha, - where we introduced bridge relays that don't need to publish to - be useful. Fixes bug 2050. diff --git a/changes/bug2060 b/changes/bug2060 new file mode 100644 index 000000000..eb95aedb2 --- /dev/null +++ b/changes/bug2060 @@ -0,0 +1,4 @@ + o Minor features + - Make sure to disable DirPort if running as a bridge. DirPorts aren't + used on bridges, and it makes bridge scanning way too easy. + diff --git a/changes/bug2081_followup b/changes/bug2081_followup new file mode 100644 index 000000000..a53227c38 --- /dev/null +++ b/changes/bug2081_followup @@ -0,0 +1,9 @@ + o Minor features (authorities) + - Directory authorities now reject relays running any versions of + Tor between 0.2.1.3-alpha and 0.2.1.18 inclusive; they have + known bugs that keep RELAY_EARLY cells from working on rendezvous + circuits. Followup to fix for bug 2081. + - Directory authorities now reject relays running any version of Tor + older than 0.2.0.26-rc. That is the earliest version that fetches + current directory information correctly. Fix for bug 2156. + diff --git a/changes/bug2097-more b/changes/bug2097-more new file mode 100644 index 000000000..52351cc01 --- /dev/null +++ b/changes/bug2097-more @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Fix a logic error in directory_fetches_from_authorities that + would cause all _non_-exits refusing single-hop-like circuits to + fetch from authorities, when we wanted to have _exits_ fetch + from authorities. Fix by boboper; fixes more of 2097; bugfix on + 0.2.2.16-alpha. diff --git a/changes/bug2146.1 b/changes/bug2146.1 new file mode 100644 index 000000000..5b91c59a6 --- /dev/null +++ b/changes/bug2146.1 @@ -0,0 +1,4 @@ + - Major bugfixes: + o Fix a bug that could break accounting on 64-bit systems with large + time_t values, making them hibernate for impossibly long intervals. + Bugfix on 0.0.9pre6; fix for bug 2146; fix by boboper. diff --git a/changes/bug2181 b/changes/bug2181 new file mode 100644 index 000000000..0a095488e --- /dev/null +++ b/changes/bug2181 @@ -0,0 +1,4 @@ + o Minor features + - Log a little more clearly about the times at which we're no longer + accepting new connections. Resolves bug 2181. + diff --git a/changes/bug2190 b/changes/bug2190 index 92ecba7eb..0591acb6f 100644 --- a/changes/bug2190 +++ b/changes/bug2190 @@ -4,3 +4,8 @@ running Libevent with debug logging enabled, and running Tor with a controller watching for low-severity log messages. Bugfix on 0.1.0.2-rc. Fixes bug 2190. + - Make Libevent log messages get delievered to controllers later, + and not from inside the Libevent log handler. This prevents + unsafe reentrant Libevent calls while still letting the log + messages get through. + diff --git a/changes/bug2203 b/changes/bug2203 new file mode 100644 index 000000000..9cfbedf14 --- /dev/null +++ b/changes/bug2203 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Clients should not weight BadExit nodes as Exits in their node + selection. Similarly, directory authorities should not count + BadExit bandwidth as Exit bandwidth when computing bandwidth-weights. + Bugfix on 0.2.2.10-alpha; fixes bug 2203. + diff --git a/changes/bug2210 b/changes/bug2210 new file mode 100644 index 000000000..fe1c049fc --- /dev/null +++ b/changes/bug2210 @@ -0,0 +1,5 @@ + o ?? bugfixes: + - Fix a bug that would cause newer streams on a given circuit to + get preference when reading bytes from the network. Fixes bug + 2210. Fix by Mashael AlSabah. This bug was introduced before + the first Tor release, in svn revision r152. diff --git a/changes/bug2230_clean_1 b/changes/bug2230_clean_1 new file mode 100644 index 000000000..a4edf9439 --- /dev/null +++ b/changes/bug2230_clean_1 @@ -0,0 +1,4 @@ + o Minor features + - Backport code from 0.2.3.x to allow directory authorities to clean + their microdescriptor caches. + diff --git a/changes/bug2230_part1 b/changes/bug2230_part1 new file mode 100644 index 000000000..79f725410 --- /dev/null +++ b/changes/bug2230_part1 @@ -0,0 +1,7 @@ + o Minor bugfixes + - When loading the microdesc journal, remember its current size. + In 0.2.2, this helps prevent the microdesc journal from growing + without limit on authorities (who are the only ones to use it in + 0.2.2). Fixes a part of bug 2230; bugfix on 0.2.2.6-alpha. + Fix posted by "cypherpunks." + diff --git a/changes/bug2230_part2 b/changes/bug2230_part2 new file mode 100644 index 000000000..2664ecc1a --- /dev/null +++ b/changes/bug2230_part2 @@ -0,0 +1,5 @@ + o Minor bugfixes + - The microdesc journal is supposed to get rebuilt only if it is + at least _half_ the length of the store, not _twice_ the length + of the store. Bugfix on 0.2.2.6-alpha; fixes part of bug 2230. + diff --git a/changes/bug2230_part4 b/changes/bug2230_part4 new file mode 100644 index 000000000..f7721fad3 --- /dev/null +++ b/changes/bug2230_part4 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Authorities now clean their microdesc cache periodically and when + reading from disk initially, not only when adding new descriptors. + This prevents a bug where we could lose microdescriptors. Bugfix + on 0.2.2.6-alpha. + diff --git a/changes/bug2235 b/changes/bug2235 new file mode 100644 index 000000000..0c3bafa44 --- /dev/null +++ b/changes/bug2235 @@ -0,0 +1,3 @@ + o Minor bugfixes + - Avoid crashes when AccountingMax is set on clients. Fixes bug 2235; + Bugfix on 0.2.2.18-alpha. Diagnosed by boboper. diff --git a/changes/bug2250 b/changes/bug2250 new file mode 100644 index 000000000..95eb55d7f --- /dev/null +++ b/changes/bug2250 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix an assert that got triggered when using the TestingTorNetwork + configuration option and then issuing a GETINFO config-text control + command. Fixes bug 2250; bugfix on 0.2.1.2-alpha. + diff --git a/changes/bug2279 b/changes/bug2279 new file mode 100644 index 000000000..d31300978 --- /dev/null +++ b/changes/bug2279 @@ -0,0 +1,15 @@ + o Minor bugfixes + - Avoid a double mark-for-free warning when failing to attach a + transparent proxy connection. Fixes bug 2279. Bugfix on + Tor 0.1.2.1 alpha. + + o Minor features + - Detect attempts at the client side to open connections to private + IP addresses (like 127.0.0.1, 10.0.0.1, and so on) with a randomly + chosen exit node. Attempts to do so are always ill-defined, generally + prevented by exit policies, and usually in error. This will also + help to detect loops in transparent proxy configurations. You can + disable this feature by setting "ClientRejectInternalAddresses 0" + in your torrc. + + diff --git a/changes/bug2314 b/changes/bug2314 new file mode 100644 index 000000000..41a232812 --- /dev/null +++ b/changes/bug2314 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Fix a bunch of compile warnings revealed by mingw with gcc 4.5. Fixes + bug 2314. + diff --git a/changes/bug2317 b/changes/bug2317 new file mode 100644 index 000000000..0b9366c36 --- /dev/null +++ b/changes/bug2317 @@ -0,0 +1,9 @@ + o Major features: + - Introduce minimum/maximum values that a client is going to believe + in a consensus. This helps to avoid crashes or worse when a param + has a weird value. + + o Major bugfixes: + - Prevent crash/heap corruption when cbtnumnodes consensus parameter is + set to 0 or large values. Fixes bug 2317. + diff --git a/changes/bug2330 b/changes/bug2330 new file mode 100644 index 000000000..fc0c4d8c3 --- /dev/null +++ b/changes/bug2330 @@ -0,0 +1,7 @@ + o Minor bugfixes + - Handle SOCKS messages longer than 128 bytes long correctly, rather + than waiting forever for them to finish. Fixes bug 2330. Bugfix on + 0.2.0.16-alpha. Found by doorss. + + + diff --git a/changes/bug2331 b/changes/bug2331 new file mode 100644 index 000000000..9940b591c --- /dev/null +++ b/changes/bug2331 @@ -0,0 +1,7 @@ + o Minor bugfixes: + - Add assertions to check for overflow in arguments to + base32_encode and base32_decode; fix a signed-unsigned + comparison there too. These bugs are not actually reachable in + Tor, but it's good to prevent future errors too. Found by + doorss. + diff --git a/changes/bug2337 b/changes/bug2337 new file mode 100644 index 000000000..a4f052dc3 --- /dev/null +++ b/changes/bug2337 @@ -0,0 +1,3 @@ + o Minor bugfixes + - Detect broken platforms with a signed size_t, and refuse to + build there. Found and analyzed by doorss and rransom. diff --git a/changes/bug2346 b/changes/bug2346 new file mode 100644 index 000000000..0f78b8461 --- /dev/null +++ b/changes/bug2346 @@ -0,0 +1,6 @@ + o Minor features + - If writing the state file to disk fails, wait up to an hour + before retrying again. (Our old code would retry the write + immediately.) Fixes bug 2346. Bugfix on Tor 0.1.1.3-alpha. + + diff --git a/changes/bug2358 b/changes/bug2358 new file mode 100644 index 000000000..5e44bb9f8 --- /dev/null +++ b/changes/bug2358 @@ -0,0 +1,5 @@ + o Minor features + - Enable Address Space Layout Randomization (ASLR) and Data Execution + Prevention (DEP) by default on Windows to make it harder for + attackers to exploit vulnerabilities. Patch from John Brooks. + diff --git a/changes/bug2363 b/changes/bug2363 new file mode 100644 index 000000000..179925f65 --- /dev/null +++ b/changes/bug2363 @@ -0,0 +1,6 @@ + o Minor bugfixes + - Correctly detect failures to create DNS requests when using Libevent + versions before v2. (Before Libevent 2, we used our own evdns + implementation. Its return values for Libevent's evdns_resolve_*() + functions are not consistent with those from Libevent.) Found by + Lodger; fixes bug 2363; bugfix on 0.2.2.6-alpha. diff --git a/changes/bug2364 b/changes/bug2364 new file mode 100644 index 000000000..37de6ef52 --- /dev/null +++ b/changes/bug2364 @@ -0,0 +1,4 @@ + o Documentation + - Document the default socks host and port (127.0.0.1:9050) for + tor-resolve. + diff --git a/changes/bug2366 b/changes/bug2366 new file mode 100644 index 000000000..d171be453 --- /dev/null +++ b/changes/bug2366 @@ -0,0 +1,8 @@ + o Minor bugfixes + - When a relay decides that its DNS is too broken for it to serve + as an exit server, it advertised itself as a non-exit, but + continued to act as an exit. This could create accidental + partitioning opportunities for users. Instead, if a relay is + going to advertise reject *:* as its exit policy, it should + really act with exit policy "reject *:*". Fixes bug 2366. + Bugfix on Tor 0.1.2.5-alpha. Bugfix by user "postman" on trac. diff --git a/changes/bug2378 b/changes/bug2378 new file mode 100644 index 000000000..227968869 --- /dev/null +++ b/changes/bug2378 @@ -0,0 +1,8 @@ + o Minor bugfixes + - Correctly detect failure to allocate an OpenSSL BIO. Fixes bug 2378; + found by "cypherpunks". This bug was introduced before the + first Tor release, in svn commit r110. + + o Minor code simplifications and refactorings + - Always treat failure to allocate an RSA key as an unrecoverable + allocation error. diff --git a/changes/bug2379 b/changes/bug2379 new file mode 100644 index 000000000..0d378b7c1 --- /dev/null +++ b/changes/bug2379 @@ -0,0 +1,5 @@ + o Documentation: + - Add missing documentation for the authority-related torrc options + RephistTrackTime, BridgePassword, and V3AuthUseLegacyKey. Resolves + issue 2379. + diff --git a/changes/bug2402_redux b/changes/bug2402_redux deleted file mode 100644 index 84be04a02..000000000 --- a/changes/bug2402_redux +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes - - Use micro-revision numbers in 0.2.1.x instead: apparently, they - were more used than we had known. (Bugfix on 0.2.1.30). - - Instead of generating our micro-version numbers using SVN revisions, - use git revisions instead. Bugfix on 0.2.1.15-rc; fixes bug 2402. - diff --git a/changes/bug2403 b/changes/bug2403 new file mode 100644 index 000000000..3b29b37fb --- /dev/null +++ b/changes/bug2403 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - In the special case where you configure a public exit relay as your + bridge, Tor would be willing to use that exit relay as the last + hop in your circuit as well. Now we fail that circuit instead. + Bugfix on 0.2.0.12-alpha. Fixes bug 2403. Reported by "piebeer". + diff --git a/changes/bug2409 b/changes/bug2409 new file mode 100644 index 000000000..5523458b6 --- /dev/null +++ b/changes/bug2409 @@ -0,0 +1,4 @@ + o Minor bugfixes + - Resolve a bug in verifying signatures of directory objects + with digests longer than SHA1. Bugfix on 0.2.2.20-alpha; + fixes bug 2409; found by "piebeer". diff --git a/changes/bug2432 b/changes/bug2432 new file mode 100644 index 000000000..407c56ba9 --- /dev/null +++ b/changes/bug2432 @@ -0,0 +1,5 @@ + o Minor features: + - Provide a log message stating which geoip file we're parsing + instead of just stating that we're parsing the geoip file. + Implements ticket 2432. + diff --git a/changes/bug2433 b/changes/bug2433 new file mode 100644 index 000000000..8e47c4f02 --- /dev/null +++ b/changes/bug2433 @@ -0,0 +1,5 @@ + o Major bugfixes: + - Don't assert when changing from bridge to relay or vice versa with a controller. + The assert happened because we didn't properly initialize our keys in this case. + Bugfix on 0.2.2.18, fixes bug 2433. Issue first discovered by bastik. + diff --git a/changes/bug2450 b/changes/bug2450 new file mode 100644 index 000000000..b3b50ddb0 --- /dev/null +++ b/changes/bug2450 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Country codes aren't supported in EntryNodes until 0.2.3.x. + Don't mention them in the manpage. Fixes bug 2450, issue + spotted by keb and G-Lo. + diff --git a/changes/bug2504 b/changes/bug2504 new file mode 100644 index 000000000..791600e59 --- /dev/null +++ b/changes/bug2504 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix a bug with our locking implementation on windows that couldn't + correctly detect when a file was already locked. Fixes bug 2504, + bugfix on 0.2.1.6-alpha. + diff --git a/changes/bug2510 b/changes/bug2510 new file mode 100644 index 000000000..2c3f61355 --- /dev/null +++ b/changes/bug2510 @@ -0,0 +1,8 @@ + o Major bugfixes: + - Fix a bug where bridge users who configure the non-canonical + address of a bridge automatically switch to its canonical + address. If a bridge listens at more than one address, it should be + able to advertise those addresses independently and any non-blocked + addresses should continue to work. Bugfix on Tor 0.2.0.x. Fixes + bug 2510. + diff --git a/changes/bug2511 b/changes/bug2511 new file mode 100644 index 000000000..a27696a5f --- /dev/null +++ b/changes/bug2511 @@ -0,0 +1,6 @@ + o Major bugfixes: + - If you configured Tor to use bridge A, and then quit and + configured Tor to use bridge B instead, it would happily continue + to use bridge A if it's still reachable. While this behavior is + a feature if your goal is connectivity, in some scenarios it's a + dangerous bug. Bugfix on Tor 0.2.0.1-alpha; fixes bug 2511. diff --git a/changes/bug2572 b/changes/bug2572 new file mode 100644 index 000000000..a5cca284a --- /dev/null +++ b/changes/bug2572 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Don't crash a bridge authority on SIGHUP if it can't force itself + into its routerlist. Fixes bug 2572. + + diff --git a/changes/bug2573 b/changes/bug2573 new file mode 100644 index 000000000..7a2a80212 --- /dev/null +++ b/changes/bug2573 @@ -0,0 +1,3 @@ + o Minor packaging issues + - Create the /var/run/tor directory on startup on OpenSUSE if it is + not already created. Patch from Andreas Stieger. Fixes bug 2573. diff --git a/changes/bug2660 b/changes/bug2660 new file mode 100644 index 000000000..2aa06d36f --- /dev/null +++ b/changes/bug2660 @@ -0,0 +1,7 @@ + o Minor bugfixes: + - Fix connect() failures on some platforms (BSD, OS X). Bugfix on + 0.2.0.3-alpha; fixes first part of bug 2660. Patch by piebeer. + - Set target port in get_interface_address6() correctly. Bugfix + on 0.1.1.4-alpha and 0.2.0.3-alpha; fixes second part of bug + 2660. + diff --git a/changes/bug2683a b/changes/bug2683a new file mode 100644 index 000000000..2fe308b03 --- /dev/null +++ b/changes/bug2683a @@ -0,0 +1,3 @@ + o Minor features + - Log the source of a rejected POSTed v3 networkstatus vote. + diff --git a/changes/bug2696 b/changes/bug2696 new file mode 100644 index 000000000..6ea41d4a6 --- /dev/null +++ b/changes/bug2696 @@ -0,0 +1,5 @@ + o Minor features: + - Make compilation with clang possible when using + --enable-gcc-warnings by removing two warnings that clang hasn't + implemented yet and by fixing a few warnings. Implements ticket + 2696. diff --git a/changes/bug2698 b/changes/bug2698 new file mode 100644 index 000000000..d995788bf --- /dev/null +++ b/changes/bug2698 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Fix an issue that prevented static linking of libevent on + some platforms (notably Linux). Fixes bug 2698, bugfix on + versions 0.2.1.23/0.2.2.8-alpha (the versions introducing + the --with-static-libevent configure option). + diff --git a/changes/bug2704 b/changes/bug2704 new file mode 100644 index 000000000..821b38bc0 --- /dev/null +++ b/changes/bug2704 @@ -0,0 +1,5 @@ + o Major bugfixes: + - When writing our maximum bw for the current interval to the state + file, don't wrongly inflate that value by a factor of 10 anymore. + Fixes more of bug 2704. + diff --git a/changes/bug2704_part1 b/changes/bug2704_part1 new file mode 100644 index 000000000..eaf22812c --- /dev/null +++ b/changes/bug2704_part1 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix an issue causing calculation of Tor's average bandwidth as saved + in the state file to be 10 times smaller than it should be. Fixes the + first part of bug 2704, bugfix on tor-0.2.2.23-alpha. + diff --git a/changes/bug2704_part2 b/changes/bug2704_part2 new file mode 100644 index 000000000..962c8b709 --- /dev/null +++ b/changes/bug2704_part2 @@ -0,0 +1,5 @@ + o Major bugfixes: + - Prevent relays that read their bandwidth history from their state file + from arbitrarily inflating that value. Fixes the second half of bug + 2704, bugfix on tor-0.2.2.23-alpha. + diff --git a/changes/bug2716 b/changes/bug2716 new file mode 100644 index 000000000..4663ed318 --- /dev/null +++ b/changes/bug2716 @@ -0,0 +1,5 @@ + o Minor features: + - When a relay has failed several reachability tests, be more accurate + at recording when it became unreachable, so we can in turn provide + more accuracy at assigning Stable, Guard, HSDir, etc flags. Bugfix + on 0.2.0.6-alpha. Resolves bug 2716. diff --git a/changes/bug2722 b/changes/bug2722 new file mode 100644 index 000000000..ed132fc89 --- /dev/null +++ b/changes/bug2722 @@ -0,0 +1,11 @@ + o Minor bugfixes + - Ignore the TunnelDirConns option when determining which HSDir + relays are responsible for a hidden service descriptor ID. + Currently, clients and hidden services with TunnelDirConns off + will skip over HSDir relays which do not advertise a DirPort + when making a list of HSDirs responsible for a descriptor ID, + even though they would never try to use a HSDir's DirPort to + upload or fetch a hidden service descriptor. Fixes bug 2722; + bugfix on 0.2.1.6-alpha. + + diff --git a/changes/bug2756 b/changes/bug2756 new file mode 100644 index 000000000..0cad515a1 --- /dev/null +++ b/changes/bug2756 @@ -0,0 +1,11 @@ + o Minor bugfixes (spec conformance, performance): + - We now ask the other side of a stream (the client or the exit) + for more data on that stream when the amount of queued data on + that stream dips low enough. Previously, we wouldn't ask the + other side for more data until either it sent us more data + (which it wasn't supposed to do if it had exhausted its + window!) or until we had completely flushed all our queued + data. Fixing this should improve throughput. Fixes bug 2756; + bugfix on the earliest released versions of Tor (svn commit + r152). + diff --git a/changes/bug2757 b/changes/bug2757 new file mode 100644 index 000000000..f947afaeb --- /dev/null +++ b/changes/bug2757 @@ -0,0 +1,6 @@ + - Minor bugfixes + o Avoid a double-mark-for-free warning when failing to attach a + transparent proxy connection. (We thought we had fixed this in + 0.2.2.23-alpha, but it turns out our fix was checking the wrong + connection.) Fixes bug 2757; bugfix on 0.1.2.1-alpha (the original + bug) and 0.2.2.23-alpha (the incorrect fix). diff --git a/changes/bug2899 b/changes/bug2899 new file mode 100644 index 000000000..6af86d067 --- /dev/null +++ b/changes/bug2899 @@ -0,0 +1,4 @@ + - Minor bugfixes: + o Downgrade "no current certificates known for authority" message from + Notice to Info. Bugfix on 0.2.0.10-alpha; fixes bug 2899. + diff --git a/changes/bug2917 b/changes/bug2917 new file mode 100644 index 000000000..6b1e64334 --- /dev/null +++ b/changes/bug2917 @@ -0,0 +1,4 @@ + o Minor bugfixes + - Make the SIGNAL DUMP control-port command work on FreeBSD. Fixes + bug 2917. Bugfix on 0.1.1.1-alpha. + diff --git a/changes/bug2948 b/changes/bug2948 new file mode 100644 index 000000000..640ef625d --- /dev/null +++ b/changes/bug2948 @@ -0,0 +1,7 @@ + o Minor bugfixes + - Only limit the lengths of single HS descriptors, even when + multiple HS descriptors are published to an HSDir relay in a + single POST operation. Fixes bug 2948; bugfix on 0.2.1.5-alpha. + Found by hsdir. + + diff --git a/changes/bug2971 b/changes/bug2971 new file mode 100644 index 000000000..8b71ce040 --- /dev/null +++ b/changes/bug2971 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Be more consistent in our treatment of file system paths. ~ should + get expanded to the user's home directory in the Log config option. + Bugfix on 0.2.0.1-alpha, which introduced the feature for the -f and + --DataDirectory options. + diff --git a/changes/bug2979 b/changes/bug2979 new file mode 100644 index 000000000..fe1f45fe0 --- /dev/null +++ b/changes/bug2979 @@ -0,0 +1,9 @@ + o Minor bugfixes: + - If the Nickname configuration option wasn't given, Tor used to pick + a nickname based on the local hostname as the nickname for a relay. + Because nicknames are not very important in today's Tor and the + "Unnamed" nickname has been implemented, this is now problematic + behaviour: It leaks information about the hostname without being + useful at all. Bugfix on tor-0.1.2.2-alpha, which introduced the + Unnamed nickname. Fixes bug 2979, reported by tagnaq. + diff --git a/changes/bug3012 b/changes/bug3012 new file mode 100644 index 000000000..dfde5fa90 --- /dev/null +++ b/changes/bug3012 @@ -0,0 +1,5 @@ + o Minor features: + - Relays can go for weeks without writing out their state file. A + relay that crashes would lose its bandwidth history (including + capacity estimate), client country statistics, and so on. Now relays + checkpoint the file at least every 12 hours. Addresses bug 3012. diff --git a/changes/bug3020 b/changes/bug3020 new file mode 100644 index 000000000..b98716122 --- /dev/null +++ b/changes/bug3020 @@ -0,0 +1,7 @@ + o Minor bugfixes: + - When checking whether a hibernation period has fully elapsed, use + the amount of seconds we expect for that period instead of using + the new period that just started. This would cause an issue because + February is a really short month. Bugfix on 0.2.2.17-alpha; + fixes bug 3020. + diff --git a/changes/bug3022 b/changes/bug3022 new file mode 100644 index 000000000..9472e6d19 --- /dev/null +++ b/changes/bug3022 @@ -0,0 +1,6 @@ + o Removed features + - Caches no longer download and serve v2 networkstatus documents + unless FetchV2Networkstatus flag is set: these documents haven't + haven't been used by clients or relays since 0.2.0.x. Resolves + bug 3022. + diff --git a/changes/bug3039 b/changes/bug3039 new file mode 100644 index 000000000..7347ee38e --- /dev/null +++ b/changes/bug3039 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Write the current time into the LastWritten line in our state file, + rather than the time from the previous write attempt. Also, stop + trying to use a time of -1 in our log statements. Fixes bug 3039; + bugfix on 0.2.2.14-alpha. diff --git a/changes/bug539_removal b/changes/bug539_removal new file mode 100644 index 000000000..dbff43de1 --- /dev/null +++ b/changes/bug539_removal @@ -0,0 +1,6 @@ + o Removed code + - Removed workaround code to handle directory responses from + servers that had bug 539 (they would send HTTP status 503 + responses _and_ send a body too). Since only server versions + before 0.2.0.16-alpha/0.1.2.19 were affected, there is no longer + reason to keep the workaround in place. diff --git a/changes/bytecount b/changes/bytecount new file mode 100644 index 000000000..50c4d6b35 --- /dev/null +++ b/changes/bytecount @@ -0,0 +1,5 @@ + o Minor bugfixes + - Fix a off-by-one error in calculating some controller command argument + lengths. Fortunately, this is harmless, the controller code does + redundant NUL termination too. Found by boboper. Bugfix on + 0.1.1.1-alpha. diff --git a/changes/cbt_hi_res b/changes/cbt_hi_res new file mode 100644 index 000000000..c0df1183c --- /dev/null +++ b/changes/cbt_hi_res @@ -0,0 +1,7 @@ + o Minor features + - When expiring circuits, use microsecond timers rather than one-second + timers. This can avoid an unpleasant situation where a circuit is + launched near the end of one second and expired right near the + beginning of the next, and prevent fluctuations in circuit timeout + values. + diff --git a/changes/cbt_parallel_intro b/changes/cbt_parallel_intro new file mode 100644 index 000000000..44e377fb3 --- /dev/null +++ b/changes/cbt_parallel_intro @@ -0,0 +1,4 @@ + o Minor features + - Use computed circuit-build timeouts to decide when to launch + parallel introdution circuits. (Previously, we would retry + after 15 seconds.) diff --git a/changes/clear_trackexithost b/changes/clear_trackexithost new file mode 100644 index 000000000..b9ac6fec4 --- /dev/null +++ b/changes/clear_trackexithost @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix a bug in the code where we could keep trying to use a + TrackHostExits-based mapping after we failed to reach the intended + destination node. Fixes bug 2999. Bugfix on 0.2.0.20-rc. + diff --git a/changes/connect_err_reporting b/changes/connect_err_reporting new file mode 100644 index 000000000..61a46b658 --- /dev/null +++ b/changes/connect_err_reporting @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Be more careful about reporting the correct error from a failed + connect() operation. Under some circumstances, it was possible to + look at an incorrect value for errno when sending the end reason. + Bugfix on Tor-0.1.0.1-rc. + diff --git a/changes/count_overflow b/changes/count_overflow new file mode 100644 index 000000000..f302ff2d7 --- /dev/null +++ b/changes/count_overflow @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Correctly handle an "impossible" overflow cases in connection + byte counting, where we write or read more than 4GB on an edge + connection in single second. Bugfix on 0.1.2.8-beta. + diff --git a/changes/dirvote_null_deref b/changes/dirvote_null_deref new file mode 100644 index 000000000..65dc519f5 --- /dev/null +++ b/changes/dirvote_null_deref @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Fix a potential null-pointer dereference while computing a consensus. + Bugfix on tor-0.2.0.3-alpha, found with the help of clang's analyzer. + diff --git a/changes/doxygen b/changes/doxygen new file mode 100644 index 000000000..5e5fcd12d --- /dev/null +++ b/changes/doxygen @@ -0,0 +1,6 @@ + o Documentation changes + - Modernize the doxygen configuration file slightly. Fixes bug 2707. + - Resolve all doxygen warnings except those for missing documentation. + Fixes bug 2705. + - Add doxygen documentation for more functions, fields, and types. + diff --git a/changes/exitnodes_reliable b/changes/exitnodes_reliable new file mode 100644 index 000000000..62ef03a0c --- /dev/null +++ b/changes/exitnodes_reliable @@ -0,0 +1,7 @@ + o Minor features: + - If ExitNodes is set, still pay attention to the Fast/Stable + status of exits when picking exit nodes. (We used to ignore + these flags when ExitNodes was set, on the grounds that people + who set exitnodes wanted all of those nodes to get used, but + with the ability to pick exits by country and IP range, this + doesn't necessarily make sense any more.) diff --git a/changes/feature2711 b/changes/feature2711 new file mode 100644 index 000000000..7cdcfbfe1 --- /dev/null +++ b/changes/feature2711 @@ -0,0 +1,4 @@ + o Minor features + - Export GeoIP information on usage to bridge controller even if we have + not yet been running for 24 hours. + diff --git a/changes/fix2195-fix b/changes/fix2195-fix new file mode 100644 index 000000000..9f03c2465 --- /dev/null +++ b/changes/fix2195-fix @@ -0,0 +1,6 @@ + o Minor bugfixes: + - Do not dereference NULL if a bridge fails to build its + extra-info descriptor. Previously, we would have dereferenced + NULL. Found by an anonymous commenter on Trac. Bugfix on + 0.2.2.19-alpha. + diff --git a/changes/fix2204 b/changes/fix2204 deleted file mode 100644 index fb2771a7f..000000000 --- a/changes/fix2204 +++ /dev/null @@ -1,7 +0,0 @@ - o Major bugfixes - - Do not set the tlsext_host_name extension on server SSL objects; - only on client SSL objects. We set it to immitate a browser, not a - vhosting server. This resolves an incompatibility with openssl 0.9.8p - and openssl 1.0.0b. Fixes bug 2204; bugfix on 0.2.1.1-alpha. - - diff --git a/changes/forget-rend-descs-on-newnym b/changes/forget-rend-descs-on-newnym index ab2fd61f3..da7afbe20 100644 --- a/changes/forget-rend-descs-on-newnym +++ b/changes/forget-rend-descs-on-newnym @@ -2,6 +2,18 @@ - Forget all hidden service descriptors cached as a client when processing a SIGNAL NEWNYM command. Fixes bug 3000. Bugfix on 0.0.6. + o Major bugfixes: + - When we find that we have extended a hidden service's introduction + circuit to a relay which isn't listed as an introduction point in + the HS descriptor we currently have for the service, we now retry + one of the introduction points in the current HS descriptor. + Previously we would just give up. Bugfix on 0.2.0.10-alpha; fixes + bugs 1024 and 1930. + o Minor bugfixes: + - Don't allow v0 hidden service authorities to act as clients. + Required by fix for bug 3000. + - Ignore SIGNAL NEWNYM commands on relay-only Tor instances. + Required by fix for bug 3000. o Code simplifications and refactoring: - Allow rend_client_send_introduction to fail without closing the AP connection permanently. diff --git a/changes/full_ap_circuits b/changes/full_ap_circuits new file mode 100644 index 000000000..379a1a1b7 --- /dev/null +++ b/changes/full_ap_circuits @@ -0,0 +1,6 @@ + o Minor bugfixes + - When a client finds that an origin circuit has run out of 16-bit + stream IDs, we now mark it as unusable for new streams. + Previously, we would try to close the entire circuit. Bugfix on + Tor version 0.0.6. + diff --git a/changes/geoip-oct2010 b/changes/geoip-oct2010 deleted file mode 100644 index a7235faaa..000000000 --- a/changes/geoip-oct2010 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the October 1 2010 Maxmind GeoLite Country database. - diff --git a/changes/geoip-sep2010 b/changes/geoip-sep2010 deleted file mode 100644 index 47eb00699..000000000 --- a/changes/geoip-sep2010 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Update to the September 1 2010 Maxmind GeoLite Country database. - diff --git a/changes/geoip-update-august2010 b/changes/geoip-update-august2010 deleted file mode 100644 index 8f127852f..000000000 --- a/changes/geoip-update-august2010 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features - - Update to the August 1 2010 Maxmind GeoLite Country database. - diff --git a/changes/geoip-update-june2010 b/changes/geoip-update-june2010 deleted file mode 100644 index 7a15c0066..000000000 --- a/changes/geoip-update-june2010 +++ /dev/null @@ -1,2 +0,0 @@ - o Minor features - - Update to the June 1 2010 Maxmind GeoLite Country database. diff --git a/changes/gmtime_null b/changes/gmtime_null new file mode 100644 index 000000000..16a25408b --- /dev/null +++ b/changes/gmtime_null @@ -0,0 +1,6 @@ + o Minor bugfixes + - On some platforms, gmtime and localtime can return NULL under + certain circumstances even for well-defined values of time_t. + Try to detect and make up for this deficiency. Possible fix for + bug 2077. Bugfix on all versions of Tor. Found by boboper. + diff --git a/changes/hsdir_assignment b/changes/hsdir_assignment new file mode 100644 index 000000000..5c04b9b9b --- /dev/null +++ b/changes/hsdir_assignment @@ -0,0 +1,8 @@ + o Security fixes: + - Directory authorities now use data collected from rephist when + choosing whether to assign the HSDir flag to relays, instead of + trusting the uptime value the relay reports in its descriptor. + This helps prevent an attack where a small set of nodes with + frequently-changing identity keys can blackhole a hidden service. + (Only authorities need upgrade; others will be fine once they do.) + Bugfix on 0.2.0.10-alpha; fixes bug 2709. diff --git a/changes/kill_ftime b/changes/kill_ftime new file mode 100644 index 000000000..47f476973 --- /dev/null +++ b/changes/kill_ftime @@ -0,0 +1,7 @@ + o Code simplification and refactoring + - Remove the old 'fuzzy time' logic. It was supposed to be used + for handling calculations where we have a known amount of clock + skew and an allowed amount of unknown skew. But we only used it + in three places, and we never adjusted the known/unknown skew + values. This is still something we might want to do someday, + but if we do, we'll want to do it differently. diff --git a/changes/log-typo-2011-03-15-01 b/changes/log-typo-2011-03-15-01 new file mode 100644 index 000000000..3830df388 --- /dev/null +++ b/changes/log-typo-2011-03-15-01 @@ -0,0 +1,3 @@ + o Minor bugfixes + - Fix a minor typo in a log message. Bugfix on 0.2.2.6-alpha. + diff --git a/changes/log_domains b/changes/log_domains new file mode 100644 index 000000000..7fc0506cd --- /dev/null +++ b/changes/log_domains @@ -0,0 +1,12 @@ + o Minor features + - Make it simpler to specify "All log domains except for A and B". + Previously you needed to say "[*,~A,~B]". Now you can just say + "[~A,~B]". + - Add a LogMessageDomains option to include the domains of log messages + along with the messages. Without this, there's no way to use + log domains without reading the source or doing a lot of guessing + + o Documentation + - Add documentation for configuring logging at different severities in + different log domains. We've had this feature since 0.2.1.1-alpha, but + for some reason it never made it into the manpage. Fixes bug 2215. diff --git a/changes/maatuska-new-v3auth b/changes/maatuska-new-v3auth deleted file mode 100644 index 9508b30a2..000000000 --- a/changes/maatuska-new-v3auth +++ /dev/null @@ -1,3 +0,0 @@ - o New directory authorities: - - Set up maatuska (run by Linus Nordberg) as the eighth v3 directory - authority. diff --git a/changes/mdesc_null_deref b/changes/mdesc_null_deref new file mode 100644 index 000000000..30f028053 --- /dev/null +++ b/changes/mdesc_null_deref @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Avoid a possible null-pointer dereference when rebuilding the mdesc + cache without actually having any descriptors to cache. Bugfix on + 0.2.2.6-alpha. Issue discovered using clang's static analyzer. + diff --git a/changes/microdesc-double-free b/changes/microdesc-double-free new file mode 100644 index 000000000..932cc754b --- /dev/null +++ b/changes/microdesc-double-free @@ -0,0 +1,7 @@ + o Security fixes: + - Don't double-free a parsable, but invalid, microdescriptor, even + if it is followed in the blob we're parsing by an unparsable + microdescriptor. Fixes an issue reported in a comment on bug 2954. + Bugfix on 0.2.2.6-alpha; fix by "cypherpunks". + + diff --git a/changes/misc-reason b/changes/misc-reason deleted file mode 100644 index 80db2d257..000000000 --- a/changes/misc-reason +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Have clients begin understanding the new END_STREAM_REASON_NOROUTE - error code. diff --git a/changes/new-geoip-db b/changes/new-geoip-db deleted file mode 100644 index 06d69ea47..000000000 --- a/changes/new-geoip-db +++ /dev/null @@ -1,5 +0,0 @@ - o Major features: - - Move to the Maxmind GeoIP db (rather than the June 2009 - ip-to-country GeoIP db) for our statistics that count how many - users relays are seeing from each country. Now we have more accurate - data for many African countries. diff --git a/changes/noroute b/changes/noroute new file mode 100644 index 000000000..644deec45 --- /dev/null +++ b/changes/noroute @@ -0,0 +1,5 @@ + - Minor features + - Send END_STREAM_REASON_NOROUTE in response to EHOSTUNREACH errors. + Clients before 0.2.1.27 didn't handle NOROUTE correctly, but + such clients are already deprecated because of security bugs. + diff --git a/changes/openbsd-sysheaders b/changes/openbsd-sysheaders deleted file mode 100644 index 2babde2d7..000000000 --- a/changes/openbsd-sysheaders +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - When building with --enable-gcc-warnings on OpenBSD, disable - warnings in system headers. This makes --enable-gcc-warnings - pass on OpenBSD 4.8.
\ No newline at end of file diff --git a/changes/osx_forgotten_compilefix b/changes/osx_forgotten_compilefix new file mode 100644 index 000000000..754e09cfe --- /dev/null +++ b/changes/osx_forgotten_compilefix @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Added a forgotten cast that caused a compile warning on OS X 10.6. Bugfix + on 0.2.2.24-alpha. + diff --git a/changes/remove-debian b/changes/remove-debian deleted file mode 100644 index 6eb1896ba..000000000 --- a/changes/remove-debian +++ /dev/null @@ -1,5 +0,0 @@ - o Removed files: - - Remove the old debian/ directory from the main Tor distribution. - The official Tor-for-debian git repository lives at the URL - https://git.torproject.org/debian/tor.git . - diff --git a/changes/task2196 b/changes/task2196 new file mode 100644 index 000000000..e629fccac --- /dev/null +++ b/changes/task2196 @@ -0,0 +1,5 @@ + o Minor features: + - Report only the top 10 ports in exit-port stats in order not to + exceed the maximum extra-info descriptor length of 50 KB. Implements + task 2196. + diff --git a/changes/ticket2497 b/changes/ticket2497 new file mode 100644 index 000000000..51171412b --- /dev/null +++ b/changes/ticket2497 @@ -0,0 +1,4 @@ + o Minor features: + - Ensure that no empty [dirreq-](read|write)-history lines are added + to an extrainfo document. Implements ticket 2497. + diff --git a/changes/warn-if-get_digest-fails b/changes/warn-if-get_digest-fails new file mode 100644 index 000000000..6cfc1082a --- /dev/null +++ b/changes/warn-if-get_digest-fails @@ -0,0 +1,6 @@ + o Minor bugfixes: + - If we fail to compute the identity digest of a v3 legacy + keypair, warn, and don't use a buffer-full of junk instead. + Bugfix on 0.2.1.1-alpha; fixes bug 3106. + + diff --git a/changes/win_tmp_dir b/changes/win_tmp_dir new file mode 100644 index 000000000..13f6e7f1c --- /dev/null +++ b/changes/win_tmp_dir @@ -0,0 +1,4 @@ + o Unit tests: + - Use GetTempDir to find the proper temporary directory location on + Windows when generating temporary files for the unit tests. Patch + by Gisle Vanem. diff --git a/configure.in b/configure.in index 1fddacaad..2e449ec48 100644 --- a/configure.in +++ b/configure.in @@ -1,11 +1,10 @@ -dnl $Id$ dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson dnl Copyright (c) 2007-2008, The Tor Project, Inc. dnl See LICENSE for licensing information AC_INIT -AM_INIT_AUTOMAKE(tor, 0.2.1.25) +AM_INIT_AUTOMAKE(tor, 0.2.2.19-alpha) AM_CONFIG_HEADER(orconfig.h) AC_CANONICAL_HOST @@ -20,21 +19,6 @@ fi # the += operator on it in src/or/Makefile.am CPPFLAGS="$CPPFLAGS -I\${top_srcdir}/src/common" -AC_ARG_ENABLE(debug, - AS_HELP_STRING(--enable-debug, compile with debugging info), -[if test x$enableval = xyes; then - CFLAGS="$CFLAGS -g" -fi]) - -#XXXX ideally, we should make this into a no-op, and detect whether we're -#compiling for the iphone by using $target. -AC_ARG_ENABLE(iphone, - AS_HELP_STRING(--enable-iphone, compile with iPhone support), - [if test x$enableval = xyes ; then - tor_cv_iphone=true - CFLAGS="$CFLAGS -D__DARWIN_UNIX03 -DIPHONE" - fi]) - #XXXX020 We should make these enabled or not, before 0.2.0.x-final AC_ARG_ENABLE(buf-freelists, AS_HELP_STRING(--disable-buf-freelists, disable freelists for buffer RAM)) @@ -46,6 +30,8 @@ AC_ARG_ENABLE(static-openssl, AS_HELP_STRING(--enable-static-openssl, Link against a static openssl library. Requires --with-openssl-dir)) AC_ARG_ENABLE(static-libevent, AS_HELP_STRING(--enable-static-libevent, Link against a static libevent library. Requires --with-libevent-dir)) +AC_ARG_ENABLE(static-zlib, + AS_HELP_STRING(--enable-static-zlib, Link against a static zlib library. Requires --with-zlib-dir)) if test x$enable_buf_freelists != xno; then AC_DEFINE(ENABLE_BUF_FREELISTS, 1, @@ -65,6 +51,15 @@ AC_ARG_ENABLE(transparent, *) AC_MSG_ERROR(bad value for --enable-transparent) ;; esac], [transparent=true]) +AC_ARG_ENABLE(asciidoc, + AS_HELP_STRING(--disable-asciidoc, don't use asciidoc (disables building of manpages)), + [case "${enableval}" in + yes) asciidoc=true ;; + no) asciidoc=false ;; + *) AC_MSG_ERROR(bad value for --disable-asciidoc) ;; + esac], [asciidoc=true]) + + AC_ARG_ENABLE(threads, AS_HELP_STRING(--disable-threads, disable multi-threading support)) @@ -90,18 +85,32 @@ case $host in ;; esac -AC_ARG_ENABLE(geoip-stats, - AS_HELP_STRING(--enable-geoip-stats, enable code for directories to collect per-country statistics)) - -if test "$enable_geoip_stats" = "yes"; then - AC_DEFINE(ENABLE_GEOIP_STATS, 1, [Defined if we try to collect per-country statistics]) -fi - AC_ARG_ENABLE(gcc-warnings, AS_HELP_STRING(--enable-gcc-warnings, enable verbose warnings)) AC_ARG_ENABLE(gcc-warnings-advisory, AS_HELP_STRING(--enable-gcc-warnings-advisory, [enable verbose warnings, excluding -Werror])) +dnl Adam shostack suggests the following for Windows: +dnl -D_FORTIFY_SOURCE=2 -fstack-protector-all +dnl Others suggest '/gs /safeseh /nxcompat /dynamicbase' for non-gcc on Windows +dnl This requires that we use gcc and that we add -O2 to the CFLAGS. +AC_ARG_ENABLE(gcc-hardening, + AS_HELP_STRING(--enable-gcc-hardening, enable compiler security checks), +[if test x$enableval = xyes; then + CFLAGS="$CFLAGS -D_FORTIFY_SOURCE=2 -fstack-protector-all" + CFLAGS="$CFLAGS -fwrapv -fPIE -Wstack-protector" + CFLAGS="$CFLAGS --param ssp-buffer-size=1" + LDFLAGS="$LDFLAGS -pie" +fi]) + +dnl Linker hardening options +dnl Currently these options are ELF specific - you can't use this with MacOSX +AC_ARG_ENABLE(linker-hardening, + AS_HELP_STRING(--enable-linker-hardening, enable linker security fixups), +[if test x$enableval = xyes; then + LDFLAGS="$LDFLAGS -z relro -z now" +fi]) + AC_ARG_ENABLE(local-appdata, AS_HELP_STRING(--enable-local-appdata, default to host local application data paths on Windows)) if test "$enable_local_appdata" = "yes"; then @@ -114,6 +123,18 @@ AC_PROG_CPP AC_PROG_MAKE_SET AC_PROG_RANLIB +dnl autoconf 2.59 appears not to support AC_PROG_SED +AC_CHECK_PROG([SED],[sed],[sed],[/bin/false]) + +dnl check for asciidoc and a2x +AC_PATH_PROG([ASCIIDOC], [asciidoc], none) +AC_PATH_PROG([A2X], [a2x], none) + +AM_CONDITIONAL(USE_ASCIIDOC, test x$asciidoc = xtrue) + +AC_PATH_PROG([SHA1SUM], [sha1sum], none) +AC_PATH_PROG([OPENSSL], [openssl], none) + TORUSER=_tor AC_ARG_WITH(tor-user, [ --with-tor-user=NAME Specify username for tor daemon ], @@ -135,7 +156,7 @@ AC_SUBST(TORGROUP) dnl If WIN32 is defined and non-zero, we are building for win32 AC_MSG_CHECKING([for win32]) -AC_RUN_IFELSE([ +AC_RUN_IFELSE([AC_LANG_SOURCE([ int main(int c, char **v) { #ifdef WIN32 #if WIN32 @@ -146,7 +167,7 @@ int main(int c, char **v) { #else return 2; #endif -}], +}])], bwin32=true; AC_MSG_RESULT([yes]), bwin32=false; AC_MSG_RESULT([no]), bwin32=cross; AC_MSG_RESULT([cross]) @@ -154,14 +175,14 @@ bwin32=cross; AC_MSG_RESULT([cross]) if test "$bwin32" = cross; then AC_MSG_CHECKING([for win32 (cross)]) -AC_COMPILE_IFELSE([ +AC_COMPILE_IFELSE([AC_LANG_SOURCE([ #ifdef WIN32 int main(int c, char **v) {return 0;} #else #error int main(int c, char **v) {return x(y);} #endif -], +])], bwin32=true; AC_MSG_RESULT([yes]), bwin32=false; AC_MSG_RESULT([no])) fi @@ -173,12 +194,12 @@ AM_CONDITIONAL(BUILD_NT_SERVICES, test x$bwin32 = xtrue) dnl Enable C99 when compiling with MIPSpro AC_MSG_CHECKING([for MIPSpro compiler]) -AC_COMPILE_IFELSE(AC_LANG_PROGRAM(, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM(, [ #if (defined(__sgi) && defined(_COMPILER_VERSION)) #error return x(y); #endif -]), +])], bmipspro=false; AC_MSG_RESULT(no), bmipspro=true; AC_MSG_RESULT(yes)) @@ -202,7 +223,7 @@ dnl ------------------------------------------------------------------- dnl Check for functions before libevent, since libevent-1.2 apparently dnl exports strlcpy without defining it in a header. -AC_CHECK_FUNCS(gettimeofday ftime socketpair uname inet_aton strptime getrlimit strlcat strlcpy strtoull getaddrinfo localtime_r gmtime_r memmem strtok_r writev readv flock prctl) +AC_CHECK_FUNCS(gettimeofday ftime socketpair uname inet_aton strptime getrlimit strlcat strlcpy strtoull getaddrinfo localtime_r gmtime_r memmem strtok_r writev readv flock prctl vasprintf) using_custom_malloc=no if test x$enable_openbsd_malloc = xyes ; then @@ -250,7 +271,19 @@ tor_libevent_pkg_debian="libevent-dev" tor_libevent_devpkg_redhat="libevent-devel" tor_libevent_devpkg_debian="libevent-dev" -TOR_SEARCH_LIBRARY(libevent, $trylibeventdir, [-levent $TOR_LIB_WS32], [ +dnl On Gnu/Linux or any place we require it, we'll add librt to the Libevent +dnl linking for static builds. +STATIC_LIBEVENT_FLAGS="" +if test "$enable_static_libevent" = "yes"; then + dnl Determine if we have clock_gettime in librt + AC_SEARCH_LIBS([clock_gettime], [rt], + [have_rt=yes]) + if test "$have_rt" = yes; then + STATIC_LIBEVENT_FLAGS=" -lrt " + fi +fi + +TOR_SEARCH_LIBRARY(libevent, $trylibeventdir, [-levent $STATIC_LIBEVENT_FLAGS $TOR_LIB_WS32], [ #ifdef WIN32 #include <winsock2.h> #endif @@ -273,29 +306,35 @@ dnl Now check for particular libevent functions. save_LIBS="$LIBS" save_LDFLAGS="$LDFLAGS" save_CPPFLAGS="$CPPFLAGS" -LIBS="-levent $TOR_LIB_WS32 $LIBS" +LIBS="-levent $STATIC_LIBEVENT_FLAGS $TOR_LIB_WS32 $LIBS" LDFLAGS="$TOR_LDFLAGS_libevent $LDFLAGS" CPPFLAGS="$TOR_CPPFLAGS_libevent $CPPFLAGS" -AC_CHECK_FUNCS(event_get_version event_get_method event_set_log_callback) +AC_CHECK_FUNCS(event_get_version event_get_version_number event_get_method event_set_log_callback evdns_set_outgoing_bind_address event_base_loopexit) AC_CHECK_MEMBERS([struct event.min_heap_idx], , , [#include <event.h> ]) +AC_CHECK_HEADERS(event2/event.h event2/dns.h) + LIBS="$save_LIBS" LDFLAGS="$save_LDFLAGS" CPPFLAGS="$save_CPPFLAGS" + +AM_CONDITIONAL(USE_EXTERNAL_EVDNS, test x$ac_cv_header_event2_dns_h = xyes) + if test "$enable_static_libevent" = "yes"; then if test "$tor_cv_library_libevent_dir" = "(system)"; then AC_MSG_ERROR("You must specify an explicit --with-libevent-dir=x option when using --enable-static-libevent") else - TOR_LIBEVENT_LIBS="$TOR_LIBDIR_libevent/libevent.a" + TOR_LIBEVENT_LIBS="$TOR_LIBDIR_libevent/libevent.a $STATIC_LIBEVENT_FLAGS" fi else TOR_LIBEVENT_LIBS="-levent" fi AC_SUBST(TOR_LIBEVENT_LIBS) + dnl ------------------------------------------------------ dnl Where do you live, openssl? And how do we call you? @@ -347,6 +386,19 @@ TOR_SEARCH_LIBRARY(zlib, $tryzlibdir, [-lz], [zlibVersion(); exit(0);], [--with-zlib-dir], [/opt/zlib]) +if test "$enable_static_zlib" = "yes"; then + if test "$tor_cv_library_zlib_dir" = "(system)"; then + AC_MSG_ERROR("You must specify an explicit --with-zlib-dir=x option when + using --enable-static-zlib") + else + TOR_ZLIB_LIBS="$TOR_LIBDIR_zlib/libz.a" + echo "$TOR_LIBDIR_zlib/libz.a" + fi +else + TOR_ZLIB_LIBS="-lz" +fi +AC_SUBST(TOR_ZLIB_LIBS) + dnl Make sure to enable support for large off_t if available. AC_SYS_LARGEFILE @@ -525,7 +577,7 @@ AC_CHECK_TYPES([rlim_t], , , ]) AC_CACHE_CHECK([whether time_t is signed], tor_cv_time_t_signed, [ -AC_RUN_IFELSE(AC_LANG_SOURCE([ +AC_RUN_IFELSE([AC_LANG_SOURCE([ #ifdef HAVE_SYS_TYPES_H #include <sys/types.h> #endif @@ -535,7 +587,7 @@ AC_RUN_IFELSE(AC_LANG_SOURCE([ #ifdef HAVE_TIME_H #include <time.h> #endif -int main(int c, char**v) { if (((time_t)-1)<0) return 1; else return 0; }]), +int main(int c, char**v) { if (((time_t)-1)<0) return 1; else return 0; }])], tor_cv_time_t_signed=no, tor_cv_time_t_signed=yes, tor_cv_time_t_signed=cross) ]) @@ -548,6 +600,23 @@ if test "$tor_cv_time_t_signed" != no; then [Define to 1 iff time_t is signed]) fi +AC_CACHE_CHECK([whether size_t is signed], tor_cv_size_t_signed, [ +AC_RUN_IFELSE([AC_LANG_SOURCE([ +#ifdef HAVE_SYS_TYPES_H +#include <sys/types.h> +#endif +int main(int c, char**v) { if (((size_t)-1)<0) return 1; else return 0; }])], + tor_cv_size_t_signed=no, tor_cv_size_t_signed=yes, tor_cv_size_t_signed=cross) +]) + +if test "$tor_cv_size_t_signed" = cross; then + AC_MSG_NOTICE([Cross compiling: assuming that size_t is not signed.]) +fi + +if test "$tor_cv_size_t_signed" = yes; then + AC_MSG_ERROR([You have a signed size_t; that's grossly nonconformant.]) +fi + AC_CHECK_SIZEOF(socklen_t, , [AC_INCLUDES_DEFAULT() #ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> @@ -655,6 +724,16 @@ if test x$tcmalloc = xyes ; then LDFLAGS="-ltcmalloc $LDFLAGS" fi +# By default, we're going to assume we don't have mlockall() +# bionic and other platforms have various broken mlockall subsystems. +# Some systems don't have a working mlockall, some aren't linkable, +# and some have it but don't declare it. +AC_CHECK_FUNCS(mlockall) +AC_CHECK_DECLS([mlockall], , , [ +#ifdef HAVE_SYS_MMAN_H +#include <sys/mman.h> +#endif]) + # Allow user to specify an alternate syslog facility AC_ARG_WITH(syslog-facility, [ --with-syslog-facility=LOG syslog facility to use (default=LOG_DAEMON)], @@ -674,14 +753,14 @@ AC_CHECK_FUNC(gethostbyname_r, [ AC_MSG_CHECKING([how many arguments gethostbyname_r() wants]) OLD_CFLAGS=$CFLAGS CFLAGS="$CFLAGS $MY_CPPFLAGS $MY_THREAD_CPPFLAGS $MY_CFLAGS" - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([ #include <netdb.h> ], [[ char *cp1, *cp2; struct hostent *h1, *h2; int i1, i2; (void)gethostbyname_r(cp1,h1,cp2,i1,&h2,&i2); - ]]),[ + ]])],[ AC_DEFINE(HAVE_GETHOSTBYNAME_R) AC_DEFINE(HAVE_GETHOSTBYNAME_R_6_ARG, 1, [Define this if gethostbyname_r takes 6 arguments]) @@ -722,25 +801,25 @@ AC_CHECK_FUNC(gethostbyname_r, [ AC_CACHE_CHECK([whether the C compiler supports __func__], tor_cv_have_func_macro, - AC_COMPILE_IFELSE([ + AC_COMPILE_IFELSE([AC_LANG_SOURCE([ #include <stdio.h> -int main(int c, char **v) { puts(__func__); }], +int main(int c, char **v) { puts(__func__); }])], tor_cv_have_func_macro=yes, tor_cv_have_func_macro=no)) AC_CACHE_CHECK([whether the C compiler supports __FUNC__], tor_cv_have_FUNC_macro, - AC_COMPILE_IFELSE([ + AC_COMPILE_IFELSE([AC_LANG_SOURCE([ #include <stdio.h> -int main(int c, char **v) { puts(__FUNC__); }], +int main(int c, char **v) { puts(__FUNC__); }])], tor_cv_have_FUNC_macro=yes, tor_cv_have_FUNC_macro=no)) AC_CACHE_CHECK([whether the C compiler supports __FUNCTION__], tor_cv_have_FUNCTION_macro, - AC_COMPILE_IFELSE([ + AC_COMPILE_IFELSE([AC_LANG_SOURCE([ #include <stdio.h> -int main(int c, char **v) { puts(__FUNCTION__); }], +int main(int c, char **v) { puts(__FUNCTION__); }])], tor_cv_have_FUNCTION_macro=yes, tor_cv_have_FUNCTION_macro=no)) @@ -781,15 +860,30 @@ AC_SUBST(BINDIR) LOCALSTATEDIR=`eval echo $localstatedir` AC_SUBST(LOCALSTATEDIR) +if test "$bwin32" = true; then + # Test if the linker supports the --nxcompat and --dynamicbase options + # for Windows + save_LDFLAGS="$LDFLAGS" + LDFLAGS="-Wl,--nxcompat -Wl,--dynamicbase" + AC_MSG_CHECKING([whether the linker supports DllCharacteristics]) + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [save_LDFLAGS="$save_LDFLAGS $LDFLAGS"], + [AC_MSG_RESULT([no])] + ) + LDFLAGS="$save_LDFLAGS" +fi + # Set CFLAGS _after_ all the above checks, since our warnings are stricter # than autoconf's macros like. if test "$GCC" = yes; then - CFLAGS="$CFLAGS -Wall -g -O2" # Disable GCC's strict aliasing checks. They are an hours-to-debug # accident waiting to happen. - CFLAGS="$CFLAGS -fno-strict-aliasing" + CFLAGS="$CFLAGS -Wall -fno-strict-aliasing" else - CFLAGS="$CFLAGS -g -O" + # Autoconf sets -g -O2 by default. Override optimization level + # for non-gcc compilers + CFLAGS="$CFLAGS -O" enable_gcc_warnings=no enable_gcc_warnings_advisory=no fi @@ -798,24 +892,29 @@ fi # released versions. (Some relevant gcc versions can't handle these.) if test x$enable_gcc_warnings = xyes || test x$enable_gcc_warnings_advisory = xyes; then - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([], [ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ #if !defined(__GNUC__) || (__GNUC__ < 4) #error -#endif]), have_gcc4=yes, have_gcc4=no) +#endif])], have_gcc4=yes, have_gcc4=no) - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([], [ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ #if !defined(__GNUC__) || (__GNUC__ < 4) || (__GNUC__ == 4 && __GNUC_MINOR__ < 2) #error -#endif]), have_gcc42=yes, have_gcc42=no) +#endif])], have_gcc42=yes, have_gcc42=no) - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([], [ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ #if !defined(__GNUC__) || (__GNUC__ < 4) || (__GNUC__ == 4 && __GNUC_MINOR__ < 3) #error -#endif]), have_gcc43=yes, have_gcc43=no) +#endif])], have_gcc43=yes, have_gcc43=no) + + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [ +#if !defined(__clang__) || (__clang_major__ > 2) || (__clang_major__ == 2 && __clang_minor__ > 9) +#error +#endif])], have_clang29orlower=yes, have_clang29orlower=no) save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Wshorten-64-to-32" - AC_COMPILE_IFELSE(AC_LANG_PROGRAM([], []), have_shorten64_flag=yes, + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([], [])], have_shorten64_flag=yes, have_shorten64_flag=no) CFLAGS="$save_CFLAGS" @@ -842,13 +941,20 @@ if test x$enable_gcc_warnings = xyes || test x$enable_gcc_warnings_advisory = xy if test x$have_gcc42 = xyes ; then # These warnings break gcc 4.0.2 and work on gcc 4.2 - # XXXX020 Use -fstack-protector. # XXXX020 See if any of these work with earlier versions. - CFLAGS="$CFLAGS -Waddress -Wmissing-noreturn -Wnormalized=id -Woverride-init -Wstrict-overflow=1" + CFLAGS="$CFLAGS -Waddress -Wmissing-noreturn -Wstrict-overflow=1" + # We used to use -Wstrict-overflow=5, but that breaks us heavily under 4.3. fi - if test x$have_gcc43 = xyes ; then + if test x$have_gcc42 = xyes && test x$have_clang29orlower = xno; then + # These warnings break gcc 4.0.2 and clang, but work on gcc 4.2 + # We only disable these for clang 2.9 and lower, in case they are + # supported in later versions. + CFLAGS="$CFLAGS -Wnormalized=id -Woverride-init" + fi + + if test x$have_gcc43 = xyes ; then # These warnings break gcc 4.2 and work on gcc 4.3 # XXXX020 See if any of these work with earlier versions. CFLAGS="$CFLAGS -Wextra -Warray-bounds" @@ -866,7 +972,7 @@ fi CPPFLAGS="$CPPFLAGS $TOR_CPPFLAGS_libevent $TOR_CPPFLAGS_openssl $TOR_CPPFLAGS_zlib" -AC_CONFIG_FILES([Makefile tor.spec Doxyfile contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile contrib/osx/Makefile contrib/osx/TorBundleDesc.plist contrib/osx/TorBundleInfo.plist contrib/osx/TorDesc.plist contrib/osx/TorInfo.plist contrib/osx/TorStartupDesc.plist src/config/torrc.sample doc/tor.1 src/Makefile doc/Makefile doc/design-paper/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh]) +AC_CONFIG_FILES([Makefile tor.spec Doxyfile contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile src/config/torrc.sample src/Makefile doc/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/test/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh]) AC_OUTPUT if test -x /usr/bin/perl && test -x ./contrib/updateVersions.pl ; then diff --git a/contrib/Makefile.am b/contrib/Makefile.am index c42892a85..5aae2c819 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -1,12 +1,10 @@ -SUBDIRS = osx suse -DIST_SUBDIRS = osx suse +SUBDIRS = suse +DIST_SUBDIRS = suse confdir = $(sysconfdir)/tor -EXTRA_DIST = exitlist tor-tsocks.conf torify.1 tor.nsi.in tor.sh torctl rc.subr cross.sh tor-mingw.nsi.in package_nsis-mingw.sh tor.ico tor-ctrl.sh linux-tor-prio.sh tor-exit-notice.html +EXTRA_DIST = exitlist tor-tsocks.conf tor.nsi.in tor.sh torctl rc.subr cross.sh tor-mingw.nsi.in package_nsis-mingw.sh tor.ico tor-ctrl.sh linux-tor-prio.sh tor-exit-notice.html conf_DATA = tor-tsocks.conf bin_SCRIPTS = torify - -man_MANS = torify.1 diff --git a/contrib/auto-naming/README b/contrib/auto-naming/README index 77e6af648..e2f9ff8c2 100644 --- a/contrib/auto-naming/README +++ b/contrib/auto-naming/README @@ -1,65 +1,6 @@ -=== AUTONAMING FOR TOR === - Tor directory authorities may maintain a binding of server identities -(their long term identity key) and nicknames. In their status documents -they may for each router they know tell if this is indeed the owner of -that nickname or not. - -This toolset allows automatic maintaining of a binding list of nicknames -to identity keys, implementing Tor proposal 123[1]. - -The rules are simple: - - A router claiming to be Bob is named (i.e. added to the binding list) - if there currently does not exist a different binding for that - nickname, the router has been around for a bit (2 weeks), no other - router has used that nickname in a while (1 month). - - A binding is removed if the server that owns it has not been seen - in a long time (6 months). - - -=== REQUIREMENTS === - - * ruby, and its postgres DBI interface (Debian packages: ruby, ruby1.8, libdbi-ruby1.8, libdbd-pg-ruby1.8) - * postgres (tested with >= 8.1) - * cron - -=== SETUP === - - * copy this tree some place, like into a 'auto-naming' directory in your Tor's - data directory - * create a database and a user, modifying db-config.rb accordingly - * initialize the database by executing the sql statements in create-db.sql - * setup a cronjob that feeds the current consensus to the process-consensus - script regularly. - * once the database is sufficiently populated, maybe a month or so after the - previous step, setup a cronjob to regularly build the binding list using - the build-approved-routers script. You probably want to append a manually - managed list of rejections to that file and give it to tor as its - "approved-routers" file. - The Sample-Makefile and Sample-crontab demonstrate the method used at tor26. - - -1. https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/proposals/123-autonaming.txt - - - - -Copyright (c) 2007 Peter Palfrader - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: +(their long term identity key) and nicknames. -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. +The auto-naming scripts have been moved to svn in +projects/tor-naming/auto-naming/trunk/ -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/contrib/auto-naming/Sample-Makefile b/contrib/auto-naming/Sample-Makefile deleted file mode 100644 index e3e0351df..000000000 --- a/contrib/auto-naming/Sample-Makefile +++ /dev/null @@ -1,20 +0,0 @@ - -all: ../approved-routers - -update: - wget -q -O - http://tor.noreply.org/tor/status-vote/current/consensus | \ - ./process-consensus - -.PHONY: approved-routers-auto -approved-routers-auto: - ./build-approved-routers > "$@" - -.INTERMEDIATE: approved-routers -approved-routers: approved-routers-auto /etc/tor/approved-routers - cat $^ > "$@" - -../approved-routers: approved-routers - if ! diff -q "$<" "$@"; then \ - mv "$<" "$@" &&\ - (! [ -e /var/run/tor/tor.pid ] || kill -HUP `cat /var/run/tor/tor.pid`) ; \ - fi diff --git a/contrib/auto-naming/Sample-crontab b/contrib/auto-naming/Sample-crontab deleted file mode 100644 index b50c07bb8..000000000 --- a/contrib/auto-naming/Sample-crontab +++ /dev/null @@ -1,3 +0,0 @@ -MAILTO=admin -# cronjob for tor naming -23 * * * * make -s -C auto-naming update && make -s -C auto-naming diff --git a/contrib/auto-naming/build-approved-routers b/contrib/auto-naming/build-approved-routers deleted file mode 100755 index 805321f20..000000000 --- a/contrib/auto-naming/build-approved-routers +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/ruby - -# build-approved-routers - create a name-binding list for use at a Tor -# directory authority -# -# Copyright (c) 2007 Peter Palfrader -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -require "yaml" - -require 'db' -require 'db-config' - -verbose = ARGV.first == "-v" - -db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password']) - -db.transaction_begin -named = db.query2(" - SELECT fingerprint, router_id, nickname_id, nick, first_seen, last_seen - FROM router NATURAL JOIN router_claims_nickname NATURAL JOIN nickname - WHERE named") -while (n=named.next) do - puts "# (r##{n['router_id']},n##{n['nickname_id']}); first_seen: #{n['first_seen']}, last_seen: #{n['last_seen']}" - fpr = n['fingerprint'].split(/(....)/).delete_if{|x| x=="" }.join(' ') - puts "#{n['nick']} #{fpr}" -end -db.transaction_commit diff --git a/contrib/auto-naming/create-db.sql b/contrib/auto-naming/create-db.sql deleted file mode 100644 index 86e3e6391..000000000 --- a/contrib/auto-naming/create-db.sql +++ /dev/null @@ -1,50 +0,0 @@ - -CREATE TABLE router ( - router_id SERIAL PRIMARY KEY, - fingerprint CHAR(40) NOT NULL, - UNIQUE(fingerprint) -); --- already created implicitly due to unique contraint --- CREATE INDEX router_fingerprint ON router(fingerprint); - -CREATE TABLE nickname ( - nickname_id SERIAL PRIMARY KEY, - nick VARCHAR(30) NOT NULL, - UNIQUE(nick) -); --- already created implicitly due to unique contraint --- CREATE INDEX nickname_nick ON nickname(nick); - -CREATE TABLE router_claims_nickname ( - router_id INTEGER NOT NULL REFERENCES router(router_id) ON DELETE CASCADE, - nickname_id INTEGER NOT NULL REFERENCES nickname(nickname_id) ON DELETE CASCADE, - first_seen TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP, - last_seen TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP, - named BOOLEAN NOT NULL DEFAULT 'false', - UNIQUE(router_id, nickname_id) -); -CREATE INDEX router_claims_nickname_router_id ON router_claims_nickname(router_id); -CREATE INDEX router_claims_nickname_nickname_id ON router_claims_nickname(nickname_id); -CREATE INDEX router_claims_nickname_first_seen ON router_claims_nickname(first_seen); -CREATE INDEX router_claims_nickname_last_seen ON router_claims_nickname(last_seen); - - --- Copyright (c) 2007 Peter Palfrader --- --- Permission is hereby granted, free of charge, to any person obtaining a copy --- of this software and associated documentation files (the "Software"), to deal --- in the Software without restriction, including without limitation the rights --- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell --- copies of the Software, and to permit persons to whom the Software is --- furnished to do so, subject to the following conditions: --- --- The above copyright notice and this permission notice shall be included in --- all copies or substantial portions of the Software. --- --- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR --- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, --- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE --- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER --- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, --- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE --- SOFTWARE. diff --git a/contrib/auto-naming/db-config.rb b/contrib/auto-naming/db-config.rb deleted file mode 100644 index b1508c1b7..000000000 --- a/contrib/auto-naming/db-config.rb +++ /dev/null @@ -1,8 +0,0 @@ -$CONFIG = {} unless $CONFIG -$CONFIG['database'] = {} unless $CONFIG['database'] - -# if you use postgres' "ident sameuser" auth set dbhost to '' -$CONFIG['database']['dbhost'] = 'localhost'; -$CONFIG['database']['dbname'] = 'tornaming'; -$CONFIG['database']['user'] = 'tornaming'; -$CONFIG['database']['password'] = 'x'; diff --git a/contrib/auto-naming/db.rb b/contrib/auto-naming/db.rb deleted file mode 100644 index 822a26bad..000000000 --- a/contrib/auto-naming/db.rb +++ /dev/null @@ -1,165 +0,0 @@ -#!/usr/bin/ruby - -# Copyright (c) 2006, 2007 Peter Palfrader -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -require "dbi" - -class WeaselDbQueryHandle - def initialize(sth) - @sth = sth - end - - def next() - row = @sth.fetch_hash - if row - return row - else - @sth.finish - return nil - end - end -end - -class Db - def initialize(host, database, user, password) - @dbh = DBI.connect("dbi:Pg:#{database}:#{host}", user, password); - @dbh['AutoCommit'] = false - @transaction = false - @pre_initial_transaction=true - end - - def do(query,*args) - @dbh.do(query,*args) - end - def transaction_begin() - @dbh.do("BEGIN") unless @pre_initial_transaction - @transaction = true - @pre_initial_transaction=false - end - def transaction_commit() - @dbh.do("COMMIT") - @transaction = false - end - def transaction_rollback() - @dbh.do("ROLLBACK") - end - def get_primarykey_name(table); - #return 'ref'; - return table+'_id'; - end - - def update(table, values, keys) - cols = [] - vals = [] - values.each_pair{ |k,v| - cols << "#{k}=?" - vals << v - } - - wheres = [] - keys.each_pair{ |k,v| - wheres << "#{k}=?" - vals << v - } - - throw "update value set empty" unless cols.size > 0 - throw "where clause empty" unless wheres.size > 0 - - query = "UPDATE #{table} SET #{cols.join(',')} WHERE #{wheres.join(' AND ')}" - transaction_begin unless transaction_before=@transaction - r = @dbh.do(query, *vals) - transaction_commit unless transaction_before - return r - end - - def update_row(table, values) - pk_name = get_primarykey_name(table); - throw "Ref not defined" unless values[pk_name] - return update(table, values.clone.delete_if{|k,v| k == pk_name}, { pk_name => values[pk_name] }); - end - def insert(table, values) - cols = values.keys - vals = values.values - qmarks = values.values.collect{ '?' } - - query = "INSERT INTO #{table} (#{cols.join(',')}) VALUES (#{qmarks.join(',')})" - transaction_begin unless transaction_before=@transaction - @dbh.do(query, *vals) - transaction_commit unless transaction_before - end - - def insert_row(table, values) - pk_name = get_primarykey_name(table); - if values[pk_name] - insert(table, values) - else - transaction_begin unless transaction_before=@transaction - row = query_row("SELECT nextval(pg_get_serial_sequence('#{table}', '#{pk_name}')) AS newref"); - throw "No newref?" unless row['newref'] - values[pk_name] = row['newref'] - insert(table, values); - transaction_commit unless transaction_before - end - end - def delete_row(table, ref) - pk_name = get_primarykey_name(table); - query = "DELETE FROM #{table} WHERE #{pk_name}=?" - transaction_begin unless transaction_before=@transaction - @dbh.do(query, ref) - transaction_commit unless transaction_before - end - def query(query, *params) - sth = @dbh.execute(query, *params) - while row = sth.fetch_hash - yield row - end - sth.finish - end - # nil if no results - # hash if one match - # throw otherwise - def query_row(query, *params) - sth = @dbh.execute(query, *params) - - row = sth.fetch_hash - if row == nil - sth.finish - return nil - elsif sth.fetch_hash != nil - sth.finish - throw "More than one result when querying for #{query}" - else - sth.finish - return row - end - end - def query_all(query, *params) - sth = @dbh.execute(query, *params) - - rows = sth.fetch_all - return nil if rows.size == 0 - return rows - end - def query2(query, *params) - sth = @dbh.execute(query, *params) - return WeaselDbQueryHandle.new(sth) - end -end diff --git a/contrib/auto-naming/process-consensus b/contrib/auto-naming/process-consensus deleted file mode 100755 index dc9d207e4..000000000 --- a/contrib/auto-naming/process-consensus +++ /dev/null @@ -1,119 +0,0 @@ -#!/usr/bin/ruby - -# process-consensus - read a current consensus document, inserting the -# information into a database then calling -# update-named-status.rb to update the name-binding -# flags -# -# Copyright (c) 2007 Peter Palfrader -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -require "yaml" - -require 'db' -require 'db-config' -require 'update-named-status' - -$db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password']) - -$router_cache = {} -$nickname_cache = {} - -def parse_consensus consensus - ts = nil - routers = [] - consensus.each do |line| - (key, value) = line.split(' ',2) - case key - when "valid-after", "published": ts = DateTime.parse(value) - when "r": - (nick, fpr, _) = value.split(' ', 3) - nick.downcase! - next if nick == 'unnamed' - routers << { - 'nick' => nick, - 'fingerprint' => (fpr+'=').unpack('m').first.unpack('H*').first - } - end - end - throw "Did not find a timestamp" unless ts - throw "Did not find any routers" unless routers.size > 0 - return ts, routers -end - -def insert_routers_into_db(router, table, field, value) - pk = table+'_id' - row = $db.query_row("SELECT #{pk} FROM #{table} WHERE #{field}=?", value) - if row - return row[pk] - else - r = { field => value } - $db.insert_row( table, r ) - return r[pk] - end -end - -def handle_one_consensus(c) - puts "parsing..." if $verbose - timestamp, routers = parse_consensus c - puts "storing..." if $verbose - - routers.each do |router| - fpr = router['fingerprint'] - nick = router['nick'] - $router_cache[fpr] = router_id = ($router_cache[fpr] or insert_routers_into_db(router, 'router', 'fingerprint', router['fingerprint'])) - $nickname_cache[nick] = nickname_id = ($nickname_cache[nick] or insert_routers_into_db(router, 'nickname', 'nick', router['nick'])) - - row = $db.update( - 'router_claims_nickname', - { 'last_seen' => timestamp.to_s }, - { 'router_id' => router_id, 'nickname_id' => nickname_id} ) - case row - when 0: - $db.insert('router_claims_nickname', - { - 'first_seen' => timestamp.to_s, - 'last_seen' => timestamp.to_s, - 'router_id' => router_id, 'nickname_id' => nickname_id} ) - when 1: - else - throw "Update of router_claims_nickname returned unexpected number of affected rows(#{row})" - end - end -end - -$db.transaction_begin -if ARGV.first == '-v' - $verbose = true - ARGV.shift -end - -if ARGV.size == 0 - handle_one_consensus STDIN.readlines - do_update $verbose -else - ARGV.each do |filename| - puts filename if $verbose - handle_one_consensus File.new(filename).readlines - puts "updating..." if $verbose - do_update $verbose - end -end -$db.transaction_commit diff --git a/contrib/auto-naming/update-named-status.rb b/contrib/auto-naming/update-named-status.rb deleted file mode 100755 index b58b24d58..000000000 --- a/contrib/auto-naming/update-named-status.rb +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/ruby - -# update-named-status.rb - update the named status of routers in our database -# -# Copyright (c) 2007 Peter Palfrader -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -require "yaml" - -require 'db' -require 'db-config' - -def do_update(verbose) - now = $db.query_row("SELECT max(last_seen) AS max FROM router_claims_nickname")['max'] - unless now - STDERR.puts "Could not find the latest last_seen timestamp. Is the database empty still?" - return - end - now = "TIMESTAMP '" + now.to_s + "'" - - denamed = $db.do(" - UPDATE router_claims_nickname - SET named=false - WHERE named - AND last_seen < #{now} - INTERVAL '6 months'") - puts "de-named: #{denamed}" if verbose - - named = $db.do(" - UPDATE router_claims_nickname - SET named=true - WHERE NOT named - AND first_seen < #{now} - INTERVAL '2 weeks' - AND last_seen > #{now} - INTERVAL '2 days' - AND NOT EXISTS (SELECT * - FROM router_claims_nickname AS innertable - WHERE named - AND router_claims_nickname.nickname_id=innertable.nickname_id) "+ # if that nickname is already named, we lose. - " AND NOT EXISTS (SELECT * - FROM router_claims_nickname AS innertable - WHERE router_claims_nickname.nickname_id=innertable.nickname_id - AND router_claims_nickname.router_id <> innertable.router_id - AND last_seen > #{now} - INTERVAL '1 month') ") # if nobody else wanted that nickname in the last month we are set - puts "named: #{named}" if verbose -end - -if __FILE__ == $0 - $db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password']) - verbose = ARGV.first == "-v" - - $db.transaction_begin - do_update verbose - $db.transaction_commit -end diff --git a/contrib/checkOptionDocs.pl b/contrib/checkOptionDocs.pl index ca3fba55e..23e57b489 100755 --- a/contrib/checkOptionDocs.pl +++ b/contrib/checkOptionDocs.pl @@ -1,27 +1,17 @@ #!/usr/bin/perl -w -# $Id use strict; my %options = (); my %descOptions = (); my %torrcSampleOptions = (); -my %torrcCompleteOptions = (); my %manPageOptions = (); # Load the canonical list as actually accepted by Tor. -my $mostRecentOption; open(F, "./src/or/tor --list-torrc-options |") or die; while (<F>) { next if m!\[notice\] Tor v0\.!; if (m!^([A-Za-z0-9_]+)!) { - $mostRecentOption = lc $1; - $options{$mostRecentOption} = 1; - } elsif (m!^ !) { - $descOptions{$mostRecentOption} = 1; - if (m!\{DEPRECATED\}!) { - delete $descOptions{$mostRecentOption}; - delete $options{$mostRecentOption}; - } + $options{$1} = 1; } else { print "Unrecognized output> "; print; @@ -29,7 +19,7 @@ while (<F>) { } close F; -# Load the contents of torrc.sample and torrc.complete +# Load the contents of torrc.sample sub loadTorrc { my ($fname, $options) = @_; local *F; @@ -37,7 +27,7 @@ sub loadTorrc { while (<F>) { next if (m!##+!); if (m!#([A-Za-z0-9_]+)!) { - $options->{lc $1} = 1; + $options->{$1} = 1; } } close F; @@ -45,23 +35,14 @@ sub loadTorrc { } loadTorrc("./src/config/torrc.sample.in", \%torrcSampleOptions); -loadTorrc("./src/config/torrc.complete.in", \%torrcCompleteOptions); # Try to figure out what's in the man page. my $considerNextLine = 0; -open(F, "./doc/tor.1.in") or die; +open(F, "./doc/tor.1.txt") or die; while (<F>) { - if ($considerNextLine and - m!^\\fB([A-Za-z0-9_]+)!) { - $manPageOptions{lc $1} = 1; - next; - } - - if (m!^\.(?:SH|TP|PP)!) { - $considerNextLine = 1; next; - } else { - $considerNextLine = 0; + if (m!^\*\*([A-Za-z0-9_]+)\*\*!) { + $manPageOptions{$1} = 1; } } close F; @@ -78,11 +59,9 @@ sub subtractHashes { 0; } -subtractHashes("No online docs", \%options, \%descOptions); +# subtractHashes("No online docs", \%options, \%descOptions); # subtractHashes("Orphaned online docs", \%descOptions, \%options); -subtractHashes("Not in torrc.complete.in", \%options, \%torrcCompleteOptions); -subtractHashes("Orphaned in torrc.complete.in", \%torrcCompleteOptions, \%options); subtractHashes("Orphaned in torrc.sample.in", \%torrcSampleOptions, \%options); subtractHashes("Not in man page", \%options, \%manPageOptions); diff --git a/contrib/checkSpace.pl b/contrib/checkSpace.pl index 37f079c52..6eb32e562 100755 --- a/contrib/checkSpace.pl +++ b/contrib/checkSpace.pl @@ -20,6 +20,10 @@ for $fn (@ARGV) { if (/\t/) { print " TAB:$fn:$.\n"; } + ## Warn about markers that don't have a space in front of them + if (/^[a-zA-Z_][a-zA-Z_0-9]*:/) { + print "nosplabel:$fn:$.\n"; + } ## Warn about trailing whitespace. if (/ +$/) { print "Space\@EOL:$fn:$.\n"; @@ -28,11 +32,15 @@ for $fn (@ARGV) { if ($C && /\s(?:if|while|for|switch)\(/) { print " KW(:$fn:$.\n"; } - ## Warn about #else #if instead of #elif. - if (($lastline =~ /^\# *else/) and ($_ =~ /^\# *if/)) { + ## Warn about #else #if instead of #elif. + if (($lastline =~ /^\# *else/) and ($_ =~ /^\# *if/)) { print " #else#if:$fn:$.\n"; - } - $lastline = $_; + } + $lastline = $_; + ## Warn about unnecessary empty lines. + if ($lastnil && /^\s*}\n/) { + print " UnnecNL:$fn:$.\n"; + } ## Warn about multiple empty lines. if ($lastnil && /^$/) { print " DoubleNL:$fn:$.\n"; @@ -42,9 +50,8 @@ for $fn (@ARGV) { $lastnil = 0; } ## Terminals are still 80 columns wide in my world. I refuse to - ## accept double-line lines. Except, of course, svn Id tags - ## can make us go long. - if (/^.{80}/ && !/\$Id: /) { + ## accept double-line lines. + if (/^.{80}/) { print " Wide:$fn:$.\n"; } ### Juju to skip over comments and strings, since the tests diff --git a/contrib/cross.sh b/contrib/cross.sh index e660be780..a6085a400 100755 --- a/contrib/cross.sh +++ b/contrib/cross.sh @@ -1,5 +1,4 @@ #!/bin/bash -# $Id$ # Copyright 2006 Michael Mohr with modifications by Roger Dingledine # See LICENSE for licensing information. @@ -186,7 +185,7 @@ if [ ! -z $STRIP ] then ${HOST_TRIPLET}strip \ src/or/tor \ - src/or/test \ + src/test/test \ src/tools/tor-resolve fi diff --git a/contrib/id_to_fp.c b/contrib/id_to_fp.c index 73395e16c..55b025dfa 100644 --- a/contrib/id_to_fp.c +++ b/contrib/id_to_fp.c @@ -1,5 +1,4 @@ /* Copyright 2006 Nick Mathewson; see LICENSE for licensing information */ -/* $Id$ */ /* id_to_fp.c : Helper for directory authority ops. When somebody sends us * a private key, this utility converts the private key into a fingerprint diff --git a/contrib/nagios-check-tor-authority-cert b/contrib/nagios-check-tor-authority-cert index 0e2c1d06c..46dc7284b 100755 --- a/contrib/nagios-check-tor-authority-cert +++ b/contrib/nagios-check-tor-authority-cert @@ -8,8 +8,6 @@ # Usage: nagios-check-tor-authority-cert <authority identity fingerprint> # e.g.: nagios-check-tor-authority-cert A9AC67E64B200BBF2FA26DF194AC0469E2A948C6 -# $Id$ - # Copyright (c) 2008 Peter Palfrader <peter@palfrader.org> # # Permission is hereby granted, free of charge, to any person obtaining diff --git a/contrib/osx/Makefile.am b/contrib/osx/Makefile.am deleted file mode 100644 index 3231801c1..000000000 --- a/contrib/osx/Makefile.am +++ /dev/null @@ -1,9 +0,0 @@ -# XXX Is this define necessary, or is it redundant with the -# one from the top-level configure? -RD -confdir = $(sysconfdir)/Tor - -EXTRA_DIST = ReadMe.rtf StartupParameters.plist Tor TorBundleDesc.plist.in \ - TorBundleInfo.plist.in TorBundleWelcome.rtf TorDesc.plist.in \ - TorInfo.plist.in TorStartupDesc.plist.in TorStartupInfo.plist \ - package.sh TorPostflight addsysuser Tor_Uninstaller.applescript \ - uninstall_tor_bundle.sh package_list.txt tor_logo.gif TorPreFlight diff --git a/contrib/osx/ReadMe.rtf b/contrib/osx/ReadMe.rtf deleted file mode 100644 index cf335c388..000000000 --- a/contrib/osx/ReadMe.rtf +++ /dev/null @@ -1,7 +0,0 @@ -{\rtf1\mac\ansicpg10000\cocoartf102 -{\fonttbl\f0\fswiss\fcharset77 Helvetica;} -{\colortbl;\red255\green255\blue255;} -\margl1440\margr1440\vieww9000\viewh9000\viewkind0 -\pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\ql\qnatural - -\f0\fs24 \cf0 Tor is a toolset for a wide range of organizations and people who want to improve their safety and security on the Internet. Using Tor can help you anonymize web browsing and publishing, instant messaging, IRC, SSH, and more. Tor also provides a platform on which software developers can build new applications with built-in anonymity, safety, and privacy features.} diff --git a/contrib/osx/StartupParameters.plist b/contrib/osx/StartupParameters.plist deleted file mode 100644 index 0537b435d..000000000 --- a/contrib/osx/StartupParameters.plist +++ /dev/null @@ -1,13 +0,0 @@ -{ - Description = "Tor"; - Provides = ("tor"); - Requires = ("Network"); - Uses = ("Network"); - OrderPreference = "Last"; - Messages = - { - start = "Starting Tor"; - stop = "Stopping Tor"; - }; -} - diff --git a/contrib/osx/Tor b/contrib/osx/Tor deleted file mode 100755 index 0660fd7c8..000000000 --- a/contrib/osx/Tor +++ /dev/null @@ -1,87 +0,0 @@ -#!/bin/sh - -TORLOC=/Library/StartupItems/Tor/Tor.loc - -if [ -f $TORLOC ]; then - TORDIR=`cat /Library/StartupItems/Tor/Tor.loc` - if [ "x$TORDIR" = "x" -o ! -d $TORDIR -o ! -x $TORDIR/tor ]; then - TORDIR=/Library/Tor - fi -else - TORDIR=/Library/Tor -fi -TORCONF=$TORDIR/torrc -TORDATA=$TORDIR/var/lib/tor -TORPID=/var/run/Tor.pid -TORUSER=_tor -TORGROUP=daemon -TORCMD=$TORDIR/tor -TORLOG=/var/log/tor.log - -## Determine OSX Version -# map version to name -if [ -x /usr/bin/sw_vers ]; then -# This is poor, yet functional. We don't care about the 3rd number in -# the OS version - OSVER=`/usr/bin/sw_vers | grep ProductVersion | cut -f2 | cut -d"." -f1,2` - case "$OSVER" in - "10.6") ARCH="universal";; - "10.5") ARCH="universal";; - "10.4") ARCH="universal";; - "10.3") ARCH="ppc";; - "10.2") ARCH="ppc";; - "10.1") ARCH="ppc";; - "10.0") ARCH="ppc";; - esac -else - ARCH="unknown" -fi - -if [ $ARCH != "universal" ]; then - export EVENT_NOKQUEUE=1 -fi - -## -# Tor Service -## - -. /etc/rc.common - -StartService () -{ - - if [ -f $TORCMD ]; then - if pid=$(GetPID Tor); then - return 0 - else - ConsoleMessage "Starting Tor Service" -# Tentative -# Making sure it is not running (I know it is not a best approarch) - killall tor 2>/dev/null - $TORCMD -f "$TORCONF" --runasdaemon 1 --pidfile "$TORPID" --datadirectory "$TORDATA" --user "$TORUSER" --group "$TORGROUP" --log "notice file $TORLOG" & - fi - fi -} - -StopService () -{ - if pid=$(GetPID Tor); then - ConsoleMessage "Stopping Tor Service" - kill -TERM "${pid}" -# Just for sanity (sometimes necessary.) - killall tor 2>/dev/null - else - ConsoleMessage "Tor Service not responding." -# Just for sanity (sometimes necessary.) - killall tor 2>/dev/null - fi -} - -RestartService () { StopService; StartService; } - -if [ "$#" = 0 ]; then - echo "Syntax: tor {start|stop}" - exit 1 -fi - -RunService "$1" diff --git a/contrib/osx/TorBundleDesc.plist.in b/contrib/osx/TorBundleDesc.plist.in deleted file mode 100644 index 3ea24d012..000000000 --- a/contrib/osx/TorBundleDesc.plist.in +++ /dev/null @@ -1,14 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>IFPkgDescriptionDeleteWarning</key> - <string></string> - <key>IFPkgDescriptionDescription</key> - <string>Bundled package of Tor @VERSION@.</string> - <key>IFPkgDescriptionTitle</key> - <string>Tor Expert Bundle</string> - <key>IFPkgDescriptionVersion</key> - <string>@VERSION@</string> -</dict> -</plist> diff --git a/contrib/osx/TorBundleInfo.plist.in b/contrib/osx/TorBundleInfo.plist.in deleted file mode 100644 index c7f497f39..000000000 --- a/contrib/osx/TorBundleInfo.plist.in +++ /dev/null @@ -1,33 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>CFBundleName</key> - <string>Tor Bundle</string> - <key>CFBundleGetInfoString</key> - <string>Tor Bundle @VERSION@</string> - <key>CFBundleIdentifier</key> - <string>org.torproject.torbundle</string> - <key>CFBundleShortVersionString</key> - <string>@VERSION@</string> - <key>IFPkgFlagComponentDirectory</key> - <string>../.contained_packages</string> - <key>IFPkgFlagPackageList</key> - <array> - <dict> - <key>IFPkgFlagPackageLocation</key> - <string>Tor.pkg</string> - <key>IFPkgFlagPackageSelection</key> - <string>required</string> - </dict> - <dict> - <key>IFPkgFlagPackageLocation</key> - <string>torstartup.pkg</string> - <key>IFPkgFlagPackageSelection</key> - <string>selected</string> - </dict> - </array> - <key>IFPkgFormatVersion</key> - <real>0.10000000149011612</real> -</dict> -</plist> diff --git a/contrib/osx/TorBundleWelcome.rtf b/contrib/osx/TorBundleWelcome.rtf deleted file mode 100755 index 0afa90db5..000000000 --- a/contrib/osx/TorBundleWelcome.rtf +++ /dev/null @@ -1,20 +0,0 @@ -{\rtf1\mac\ansicpg10000\cocoartf824\cocoasubrtf420 -{\fonttbl\f0\fswiss\fcharset77 Helvetica;\f1\fswiss\fcharset77 Helvetica-Oblique;\f2\fswiss\fcharset77 Helvetica-Bold; -} -{\colortbl;\red255\green255\blue255;} -\paperw11900\paperh16840\margl1440\margr1440\vieww9000\viewh9000\viewkind0 -\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\ql\qnatural - -\f0\fs24 \cf0 Welcome to the Tor Expert installer.\ -This will install Tor in your computer.\ -\ - -\f0\b0 \ -\pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\ql\qnatural - -\f2\b \cf0 Tor -\f0\b0 is a system for using the Internet anonymously, and allowing\ -others to do so.\ -\ - For more information, please visit https://www.torproject.org/\ -\ diff --git a/contrib/osx/TorDesc.plist.in b/contrib/osx/TorDesc.plist.in deleted file mode 100644 index 6d7b154dd..000000000 --- a/contrib/osx/TorDesc.plist.in +++ /dev/null @@ -1,10 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>IFPkgDescriptionTitle</key> - <string>Tor</string> - <key>IFPkgDescriptionVersion</key> - <string>@VERSION@</string> -</dict> -</plist> diff --git a/contrib/osx/TorInfo.plist.in b/contrib/osx/TorInfo.plist.in deleted file mode 100644 index a5bc13f88..000000000 --- a/contrib/osx/TorInfo.plist.in +++ /dev/null @@ -1,28 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>CFBundleIdentifier</key> - <string>Tor @VERSION@</string> - <key>CFBundleGetInfoString</key> - <string>Tor @VERSION@</string> - <key>CFBundleName</key> - <string>Tor</string> - <key>CFBundleShortVersionString</key> - <string>@VERSION@</string> - <key>IFPkgFlagAllowBackRev</key> - <true/> - <key>IFPkgFlagAuthorizationAction</key> - <string>RootAuthorization</string> - <key>IFPkgFlagFollowLinks</key> - <true/> - <key>IFPkgFlagIsRequired</key> - <true/> - <key>IFPkgFlagRootVolumeOnly</key> - <true/> - <key>IfPkgFlagBackgroundScaling</key> - <string>proportional</string> - <key>IFPkgFlagBackgroundAlignment</key> - <string>bottomleft</string> -</dict> -</plist> diff --git a/contrib/osx/TorPostflight b/contrib/osx/TorPostflight deleted file mode 100644 index 3f52b9850..000000000 --- a/contrib/osx/TorPostflight +++ /dev/null @@ -1,123 +0,0 @@ -#!/bin/sh -# ==================================================================== -# TorPostFlight is distributed under this license -# -# Copyright (c) 2006 Andrew Lewman -# Copyright (c) 2008 The Tor Project -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are -# met: -# -# * Redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer. -# -# * Redistributions in binary form must reproduce the above -# copyright notice, this list of conditions and the following disclaimer -# in the documentation and/or other materials provided with the -# distribution. -# -# * Neither the names of the copyright owners nor the names of its -# contributors may be used to endorse or promote products derived from -# this software without specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -# ====================================================================== - -# TorPostflight gets invoked after any install or upgrade. - -ADDSYSUSER=$RECEIPT_PATH/addsysuser -if [ ! -x "$ADDSYSUSER" ]; then - echo "Could not find addsysuser script." - exit 1 -fi - -TORUSER=_tor -TORGROUP=daemon -TARGET=$2/Library/Tor -TORDIR=$TARGET/var/lib/tor -LOGFILE=/var/log/tor.log - -# Check defaults for TARGET -if [ "$TARGET" == "//Library/Tor" ]; then - TARGET=/Library/Tor -fi - -# Create user $TORUSER in group daemon. If it's already there, great. -$ADDSYSUSER $TORUSER "Tor System user" $TORDIR - -# Create the tor directory, if it doesn't exist. -if [ ! -d $TORDIR ]; then - mkdir -p $TORDIR -fi -# Check its permissions. -chown $TORUSER $TORDIR -chgrp daemon $TORDIR -chmod 700 $TORDIR - -if [ ! -f $LOGFILE ]; then - touch $LOGFILE - chown $TORUSER $LOGFILE - chgrp daemon $LOGFILE - chmod 660 $LOGFILE -fi - -# Create the configuration file only if there wasn't one already. -if [ ! -f $TARGET/torrc ]; then - cp $TARGET/torrc.sample $TARGET/torrc -fi - -# Put the geoip database into the datadir -if [ ! -f $TORDIR/geoip ]; then - cp $PACKAGE_PATH/Contents/Resources/geoip $TORDIR/geoip -fi - -# Ensure symbolic links -cd /usr/bin -if [ -e /usr/bin/tor -a ! -L /usr/bin/tor ]; then - mv tor tor_old -fi -if [ -e /usr/bin/tor-resolve -a ! -L /usr/bin/tor-resolve ]; then - mv tor-resolve tor-resolve_old -fi -ln -sf $TARGET/tor . -ln -sf $TARGET/tor-resolve . - -cd /usr/share/man/man1 -MAN1=$TARGET/share/man/man1 -#ln -sf $MAN1/*.1 . - -# Copy Documentation -if [ -d $PACKAGE_PATH/Contents/Resources/documents ];then - cp -r $PACKAGE_PATH/Contents/Resources/documents $TARGET/documents -fi - -# Copy Uninstaller -if [ -f $PACKAGE_PATH/Contents/Resources/Tor_Uninstaller.applescript ]; then - cp $PACKAGE_PATH/Contents/Resources/Tor_Uninstaller.applescript $TARGET/Tor_Uninstaller.applescript - chmod 550 $TARGET/Tor_Uninstaller.applescript -fi - -if [ -f $PACKAGE_PATH/Contents/Resources/uninstall_tor_bundle.sh ]; then - cp $PACKAGE_PATH/Contents/Resources/uninstall_tor_bundle.sh $TARGET/uninstall_tor_bundle.sh - chmod 550 $TARGET/uninstall_tor_bundle.sh -fi - -if [ -f $PACKAGE_PATH/Contents/Resources/package_list.txt ]; then - cp $PACKAGE_PATH/Contents/Resources/package_list.txt $TARGET/package_list.txt -fi - -if [ -d /Library/StartupItems/Tor ]; then - rm -f /Library/StartupItems/Tor/Tor.loc - echo "$TARGET" > /Library/StartupItems/Tor/Tor.loc -fi diff --git a/contrib/osx/TorPreFlight b/contrib/osx/TorPreFlight deleted file mode 100644 index 69dff3c1f..000000000 --- a/contrib/osx/TorPreFlight +++ /dev/null @@ -1,51 +0,0 @@ -#!/bin/sh -# -# =================================================================== -# -# TorPreFlight is distributed under this license: -# -# Copyright (c) 2006 Andrew Lewman -# Copyright (c) 2008 The Tor Project, Inc. -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are -# met: -# -# * Redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer. -# -# * Redistributions in binary form must reproduce the above -# copyright notice, this list of conditions and the following disclaimer -# in the documentation and/or other materials provided with the -# distribution. -# -# * Neither the names of the copyright owners nor the names of its -# contributors may be used to endorse or promote products derived from -# this software without specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -#=============================================================================== - -# TorPreFlight is invoked before the install begins - -# Figure out where Tor is installed -if [ -f /Library/StartupItems/Tor/Tor.loc ]; then - TORPATH=`cat /Library/StartupItems/Tor/Tor.loc` -else - TORPATH="/Library/Tor/" -fi - -# Backup all of Tor, just in case -if [ -d $TORPATH ]; then - cp $TORPATH/torrc $TORPATH/torrc.installer-saved -fi diff --git a/contrib/osx/TorStartupDesc.plist.in b/contrib/osx/TorStartupDesc.plist.in deleted file mode 100644 index b0a48fafe..000000000 --- a/contrib/osx/TorStartupDesc.plist.in +++ /dev/null @@ -1,10 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>IFPkgDescriptionTitle</key> - <string>Tor Startup Script</string> - <key>IFPkgDescriptionVersion</key> - <string>@VERSION@</string> -</dict> -</plist> diff --git a/contrib/osx/TorStartupInfo.plist b/contrib/osx/TorStartupInfo.plist deleted file mode 100644 index 08d92742a..000000000 --- a/contrib/osx/TorStartupInfo.plist +++ /dev/null @@ -1,26 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>CFBundleIdentifier</key> - <string>Tor Startup Script</string> - <key>CFBundleGetInfoString</key> - <string>Tor Startup Script</string> - <key>CFBundleName</key> - <string>Tor Startup Script</string> - <key>CFBundleShortVersionString</key> - <string>0.1</string> - <key>IFPkgFlagAllowBackRev</key> - <true/> - <key>IFPkgFlagAuthorizationAction</key> - <string>RootAuthorization</string> - <key>IFPkgFlagRestartAction</key> - <string>NoRestart</string> - <key>IFPkgFlagFollowLinks</key> - <true/> - <key>IFPkgFlagIsRequired</key> - <false/> - <key>IFPkgFlagRootVolumeOnly</key> - <true/> -</dict> -</plist> diff --git a/contrib/osx/Tor_Uninstaller.applescript b/contrib/osx/Tor_Uninstaller.applescript deleted file mode 100644 index 246e265bb..000000000 --- a/contrib/osx/Tor_Uninstaller.applescript +++ /dev/null @@ -1,68 +0,0 @@ --- Tor Uninstaller.applescript --- Tor Uninstaller - --- =============================================================================== --- Tor Uninstaller is distributed under this license: --- --- Copyright (c) 2005 Andrew Lewman ( pgp key: 31B0974B ) --- --- Redistribution and use in source and binary forms, with or without --- modification, are permitted provided that the following conditions are --- met: --- --- * Redistributions of source code must retain the above copyright --- notice, this list of conditions and the following disclaimer. --- --- * Redistributions in binary form must reproduce the above --- copyright notice, this list of conditions and the following disclaimer --- in the documentation and/or other materials provided with the --- distribution. --- --- * Neither the names of the copyright owners nor the names of its --- contributors may be used to endorse or promote products derived from --- this software without specific prior written permission. --- --- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS --- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT --- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR --- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT --- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, --- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT --- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, --- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY --- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT --- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE --- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. --- =============================================================================== - -on run - - -- Validate & find disk paths - set boot_disk to (path to startup disk) as string - set default_tor_path to boot_disk & "Library:Tor" - set default_privoxy_path to boot_disk & "Library:Privoxy" - set default_tor_startup_path to boot_disk & "Library:StartupItems:Tor" - set default_privoxy_startup_path to boot_disk & "Library:StartupItems:Privoxy" - set shell_script to default_tor_path & ":uninstall_tor_bundle.sh" - set doomed_path_list to {default_tor_path, default_privoxy_path, default_tor_startup_path, default_privoxy_startup_path} - - -- Display what we're removing and ask for validation - -- this is the simplest way to do this - set remove_me to display dialog "Welcome to the Tor + Privoxy Uninstaller. This program will remove:" & return & return & POSIX path of default_tor_path & return & POSIX path of default_privoxy_path & return & POSIX path of default_tor_startup_path & return & POSIX path of default_privoxy_startup_path & return & return & "If this looks correct, choose Yes. Otherwise, choose No." buttons {"Yes", "No"} default button "No" - - -- Run a shell script to do all the unix work since applescript can't see it at all - if button returned of result is "Yes" then - try - do shell script (POSIX path of shell_script) with administrator privileges - on error - display dialog "Too many errors, quitting." buttons {"Quit"} default button "Quit" with icon stop giving up after 3 - quit - end try - -- So Long and Thanks for all the Fish! - display dialog "Thank you for using tor!" buttons {"Ok"} giving up after 3 - else - display dialog "Thank you for your continued use of Tor & Privoxy" buttons {"You're welcome."} giving up after 3 - end if - -end run --- We're done
\ No newline at end of file diff --git a/contrib/osx/addsysuser b/contrib/osx/addsysuser deleted file mode 100755 index 50797f1ab..000000000 --- a/contrib/osx/addsysuser +++ /dev/null @@ -1,83 +0,0 @@ -#!/bin/sh -# -# Original adduser 05 Feb 2002 by Jon L. Gardner -# -# Modified for Tor installer by Nick Mathewson -# 2007-06-12 Modified for leopard by Andrew Lewman -# Copyright (c) 2007 Andrew Lewman -# - - -ROOTPROP=/ - -if [ "`whoami`" != "root" ]; then - echo "You must be root to execute this script." - exit -fi -if [ "x$3" = "x" ]; then - echo 'Usage: addsysuser <username> "<full name>" <homedir>' - exit 0 -fi - -username=$1 -realname=$2 -homedir=$3 - -if [ -x /usr/bin/dscl ]; then - # Determine the gid of the daemon group - gid=`dscl . -read /groups/daemon gid` - if [ "x`dscl . -list /users|cut -f2 -d' '|grep $username`" != "x" ]; then - echo The account $username already exists. - exit 0 - fi - if [ -x /usr/bin/nidump ]; then - uiddef=`nidump passwd / | cut -d: -f3 | sort -n | grep -v '^[56789]..' |grep -v '^....$' | tail -n 1` - else - _tmp=/tmp/_dsexport_tmp.txt.$$ - rm -f $_tmp - dsexport $_tmp '/Local/Default' 'dsRecTypeStandard:Users' > /dev/null 2>&1 - uiddef=`cat $_tmp | sed 's/\\\://g' | cut -d: -f6 | grep '^[0-9]' | sort -n | grep -v '^[56789]..' | grep -v '^....$' | tail -n 1` - rm -f $_tmp - fi - uiddef=`echo $uiddef + 1 | bc` - dscl . -create /users/$username uid $uiddef - # home is the local path to the home directory - home=/Users/$username - echo Creating account for $username... - dscl . -create /users/$username - dscl . -create /users/$username _writers_tim_passwd $username - dscl . -create /users/$username realname $realname - dscl . -create /users/$username _writers_passwd $username - dscl . -create /users/$username gid $gid - dscl . -create /users/$username home $homedir - dscl . -create /users/$username name $username - dscl . -create /users/$username passwd '*' - dscl . -create /users/$username shell /dev/null -else - # Determine the gid of the daemon group - gid=`niutil -readprop $ROOTPROP /groups/daemon gid` - if [ "x`niutil -list $ROOTPROP /users|cut -f2 -d' '|grep $username`" != "x" ]; then - echo The account $username already exists. - exit 0 - fi - # home is the local path to the home directory - home=/Users/$username - # defhome is what goes into NetInfo - defhome="/Network/Servers/MyServer/Users" - #echo "Determining next available system uid (please be patient)..." - # Uids over 500 are for system users. - uiddef=`nidump passwd / | cut -d: -f3 | sort -n | grep -v '^[56789]..' |grep -v '^....$' | tail -n 1` - uiddef=`echo $uiddef + 1 |bc` - echo Creating account for $username... - niutil -create $ROOTPROP /users/$username - niutil -createprop $ROOTPROP /users/$username _writers_tim_passwd $username - niutil -createprop $ROOTPROP /users/$username realname $realname - niutil -createprop $ROOTPROP /users/$username _writers_passwd $username - niutil -createprop $ROOTPROP /users/$username uid $uiddef - #niutil -createprop $ROOTPROP /users/$username home_loc "<home_dir><url>afp://afp.server.com/Users/</url><path>$username</path></home_dir>" - niutil -createprop $ROOTPROP /users/$username gid $gid - niutil -createprop $ROOTPROP /users/$username home $homedir - niutil -createprop $ROOTPROP /users/$username name $username - niutil -createprop $ROOTPROP /users/$username passwd '*' - niutil -createprop $ROOTPROP /users/$username shell /dev/null -fi diff --git a/contrib/osx/org.torproject.tor.plist b/contrib/osx/org.torproject.tor.plist deleted file mode 100644 index 4cfab9f1b..000000000 --- a/contrib/osx/org.torproject.tor.plist +++ /dev/null @@ -1,26 +0,0 @@ -<?xml version=\"1.0\" encoding=\"UTF-8\"?> -<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\" - \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\"> -<plist version=\"1.0\"> -<dict> - <key>Label</key> - <string>org.torproject.tor</string> - <key>ProgramArguments</key> - <array> - <string>/usr/bin/tor</string> - <string>-f</string> - <string>/Library/Tor/torrc</string> - </array> - <key>RunAtLoad</key> - <true/> - <key>OnDemand</key> - <false/> - - <key>UserName</key> - <string>_tor</string> - <key>GroupName</key> - <string>daemon</string> - -</dict> - -</plist> diff --git a/contrib/osx/package.sh b/contrib/osx/package.sh deleted file mode 100644 index 040c7cd4c..000000000 --- a/contrib/osx/package.sh +++ /dev/null @@ -1,155 +0,0 @@ -#!/bin/sh -# $Id$ -# Copyright 2004-2005 Nick Mathewson. -# Copyright 2005-2007 Andrew Lewman -# Copyright 2008 The Tor Project, Inc. -# See LICENSE in Tor distribution for licensing information. - -# This script builds a Macintosh OS X metapackage containing 2 packages: -# - One for Tor. -# - One for Startup script for Tor. -# -# This script expects to be run from the toplevel makefile, with VERSION -# set to the latest Tor version, and Tor already built. -# - -# Read the documentation located in tor/doc/tor-osx-dmg-creation.txt on -# how to build Tor for OSX - -### -# Helpful info on OS X packaging: -# http://developer.apple.com/documentation/DeveloperTools/Conceptual/SoftwareDistribution/index.html -# man packagemaker - -# Make sure VERSION is set, so we don't name the package -# "Tor--$ARCH-Bundle.dmg" -if [ "XX$VERSION" = 'XX' ]; then - echo "VERSION not set." - exit 1 -fi - -## Determine OSX Version -# map version to name -if [ -x /usr/bin/sw_vers ]; then -# This is poor, yet functional. We don't care about the 3rd number in -# the OS version - OSVER=`/usr/bin/sw_vers | grep ProductVersion | cut -f2 | cut -d"." -f1,2` - case "$OSVER" in - "10.6") ARCH="universal";; - "10.5") ARCH="universal";; - "10.4") ARCH="universal";; - "10.3") ARCH="ppc";; - "10.2") ARCH="ppc";; - "10.1") ARCH="ppc";; - "10.0") ARCH="ppc";; - *) ARCH="unknown";; - esac -else - ARCH="unknown" -fi - -# Where will we put our temporary files? -BUILD_DIR=/tmp/tor-osx-$$ -# Path to PackageMaker app. -PACKAGEMAKER=/Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker - -umask 022 - -echo I might ask you for your password now, so you can sudo. - -sudo rm -rf $BUILD_DIR -mkdir $BUILD_DIR || exit 1 -for subdir in tor_packageroot tor_resources \ - torstartup_packageroot \ - torbundle_resources \ - output; do - mkdir $BUILD_DIR/$subdir -done - -### Make Tor package. - -make install DESTDIR=$BUILD_DIR/tor_packageroot -cp contrib/osx/ReadMe.rtf $BUILD_DIR/tor_resources -chmod 755 contrib/osx/TorPostflight -cp contrib/osx/TorPostflight $BUILD_DIR/tor_resources/postflight -cp contrib/osx/addsysuser $BUILD_DIR/tor_resources/addsysuser -cp contrib/osx/Tor_Uninstaller.applescript $BUILD_DIR/tor_resources/Tor_Uninstaller.applescript -cp contrib/osx/uninstall_tor_bundle.sh $BUILD_DIR/tor_resources/uninstall_tor_bundle.sh -cp contrib/osx/package_list.txt $BUILD_DIR/tor_resources/package_list.txt -cp contrib/osx/tor_logo.gif $BUILD_DIR/tor_resources/background.gif -cp src/config/geoip $BUILD_DIR/tor_resources/geoip -cat <<EOF > $BUILD_DIR/tor_resources/Welcome.txt -Tor: an anonymous Internet communication system - -Tor is a system for using the internet anonymously, and allowing -others to do so. -EOF - -### Assemble documentation - -DOC=$BUILD_DIR/tor_resources/documents -mkdir $DOC -mkdir $DOC/howto -cp AUTHORS $DOC/AUTHORS.txt -groff doc/tor.1.in -T ps -m man | pstopdf -i -o $DOC/tor-reference.pdf -groff doc/tor-resolve.1 -T ps -m man | pstopdf -i -o $DOC/tor-resolve.pdf -mkdir $DOC/Advanced -cp doc/spec/*.txt $DOC/Advanced -cp doc/HACKING $DOC/Advanced/HACKING.txt -cp ChangeLog $DOC/Advanced/ChangeLog.txt - -find $BUILD_DIR/tor_packageroot -print0 |sudo xargs -0 chown root:wheel - -$PACKAGEMAKER -build \ - -p $BUILD_DIR/output/Tor.pkg \ - -f $BUILD_DIR/tor_packageroot \ - -r $BUILD_DIR/tor_resources \ - -i contrib/osx/TorInfo.plist \ - -d contrib/osx/TorDesc.plist - -### Make Startup Script package - -mkdir -p $BUILD_DIR/torstartup_packageroot/Library/StartupItems/Tor -cp contrib/osx/Tor contrib/osx/StartupParameters.plist \ - $BUILD_DIR/torstartup_packageroot/Library/StartupItems/Tor - -find $BUILD_DIR/torstartup_packageroot -print0 | sudo xargs -0 chown root:wheel - -$PACKAGEMAKER -build \ - -p $BUILD_DIR/output/torstartup.pkg \ - -f $BUILD_DIR/torstartup_packageroot \ - -i contrib/osx/TorStartupInfo.plist \ - -d contrib/osx/TorStartupDesc.plist - -### Assemble the metapackage. Packagemaker won't buld metapackages from -# the command line, so we need to do it by hand. - -MPKG=$BUILD_DIR/output/Tor-$VERSION-$ARCH-Bundle.mpkg -mkdir -p "$MPKG/Contents/Resources" -echo -n "pmkrpkg1" > "$MPKG/Contents/PkgInfo" -cp contrib/osx/ReadMe.rtf "$MPKG/Contents/Resources" -cp contrib/osx/TorBundleInfo.plist "$MPKG/Contents/Info.plist" -cp contrib/osx/TorBundleWelcome.rtf "$MPKG/Contents/Resources/Welcome.rtf" -cp contrib/osx/TorBundleDesc.plist "$MPKG/Contents/Resources/Description.plist" -cp contrib/osx/tor_logo.gif "$MPKG/Contents/Resources/background.gif" - -# Move all the subpackages into place. -mkdir $BUILD_DIR/output/.contained_packages -mv $BUILD_DIR/output/*.pkg $BUILD_DIR/OUTPUT/.contained_packages -( cd $BUILD_DIR/output/.contained_packages ) - -### Copy readmes and licenses into toplevel. -cp contrib/osx/ReadMe.rtf $BUILD_DIR/output/Tor\ ReadMe.rtf -cp LICENSE $BUILD_DIR/output/Tor\ License.txt - -### Package it all into a DMG - -find $BUILD_DIR/output -print0 | sudo xargs -0 chown root:wheel - -mv $BUILD_DIR/output "$BUILD_DIR/Tor-$VERSION-$ARCH-Bundle" -rm -f "Tor-$VERSION-$ARCH-Bundle.dmg" -USER="`whoami`" -sudo hdiutil create -format UDZO -imagekey zlib-level=9 -srcfolder "$BUILD_DIR/Tor-$VERSION-$ARCH-Bundle" "Tor-$VERSION-$ARCH-Bundle.dmg" -sudo chown "$USER" "Tor-$VERSION-$ARCH-Bundle.dmg" - -sudo rm -rf $BUILD_DIR diff --git a/contrib/osx/package_list.txt b/contrib/osx/package_list.txt deleted file mode 100644 index 6285dd083..000000000 --- a/contrib/osx/package_list.txt +++ /dev/null @@ -1,2 +0,0 @@ -Tor -torstartup diff --git a/contrib/osx/tor_logo.gif b/contrib/osx/tor_logo.gif Binary files differdeleted file mode 100644 index 8eea7ee2e..000000000 --- a/contrib/osx/tor_logo.gif +++ /dev/null diff --git a/contrib/osx/uninstall_tor_bundle.sh b/contrib/osx/uninstall_tor_bundle.sh deleted file mode 100755 index b7c439429..000000000 --- a/contrib/osx/uninstall_tor_bundle.sh +++ /dev/null @@ -1,140 +0,0 @@ -#!/bin/sh -# -# not a psueber-pretty uninstall script for the Tor bundle package -# . this currently leaves ~/.tor, /var/log/tor and empty -# directories /Library/Tor, /Library/Privoxy (see comment below) -# . this relies on the fact that the startup items directories (if any) -# will be named the same as the package name (ie. Tor) -# -# -# version history -# initial version - 21 may, 2005 - loki der quaeler -# -# -# comments -# loki: because of the way the Tor package installs itself (the root directory -# is the filesystem root, as opposed to the much nicer way that privoxy -# installs itself with the privoxy home (/Library/Privoxy) as its -# install root directory), i thought it more prudent to leave empty -# directories laying around rather than try to incorrectly intuit from -# the bom contents what directories should exist and which ones could be -# deleted (ie, the Tor package has /Library listed in its bom -- -# obviously this is a Bad Thing(tm) to delete). -# + when the Tor installer is changed, this uninstaller could be modified. -# loki: /bin/ps, when run from a terminal window in osX, restricts information -# based on the width of the window. an 80 col window will stupidly cause -# the grep search for the privoxy pid to not find the pid, whereas the grep -# in a wider window succeeds. consider using killall. in the meantime, -# advise uninstall runners to drag wide their terminal window.. ugh -# - - -### this is the location of a file which contains all the actual package names -## (ie "Tor", "torstartup", ...) the list should be new-line-delimited. -PACKAGE_LIST_SRC=/Library/Tor/package_list.txt - -### this is the name of the user created in the install process of Tor -TOR_USER=_tor - -### these should be constant across all osX installs (so leave them be) -STARTUP_ITEMS_DIR=/Library/StartupItems -PKG_RCPT_BASE_DIR=/Library/Receipts -BOM_INTERMEDIATE_DIR=Contents/Resources -INFO_INTERMEDIATE_DIR=$BOM_INTERMEDIATE_DIR/English.lproj -TEMP_BOM_CONTENTS=/tmp/tor_uninst_scratch - -### make sure the script is being run as root, barf if not -if [ "`whoami`" != "root" ]; then - echo "Must be root to run the uninstall script." - exit -1 -fi - - -### check to see if tor is currently running, kill it if it is -## we grep on 'Tor/tor ' because 'tor' is too common (like in 'directory') -## -- this relies on the fact that tor has been started with command -## line arguments.. :-/ -TOR_PID=`ps -uax | grep 'Tor/tor ' | grep -v grep | awk '{print $2;}'` -if [ ${#TOR_PID} -gt 0 ]; then - echo ". Killing currently running tor process, pid is $TOR_PID" - kill -9 $TOR_PID -else - echo ". tor process appears to already be stopped" -fi - - -### check to see if privoxy is currently running, kill it if it is -PRIVOXY_PID=`ps -uax | grep privoxy | grep -v grep | awk '{print $2;}'` -if [ ${#PRIVOXY_PID} -gt 0 ]; then - echo ". Killing currently running privoxy process, pid is $PRIVOXY_PID" - kill -9 $PRIVOXY_PID -else - echo ". privoxy process appears to already be stopped" -fi - - -## grab each package name from the package list file -while read LINE; do - if [ ${#LINE} -gt 0 ]; then - PACKAGE_NAME=$LINE.pkg - PACKAGE_PATH=$PKG_RCPT_BASE_DIR/$PACKAGE_NAME - echo ". Uninstalling $PACKAGE_NAME" - if [ ! -d $PACKAGE_PATH ]; then - echo " . No receipt exists for this package -- skipping." - - continue - fi - - - ## get rid of the startup item if it exists - STARTUP_DIR=$STARTUP_ITEMS_DIR/$LINE - if [ -d $STARTUP_DIR ]; then - echo " . Deleting startup item $STARTUP_DIR" - rm -rf $STARTUP_DIR - fi - - - ## determine the root directory of the the relative paths specified in the bom - DEFAULT_LOC=`grep DefaultLocation $PACKAGE_PATH/$INFO_INTERMEDIATE_DIR/$LINE.info | awk '{print $2;}'` - if [ ${#DEFAULT_LOC} -eq 0 ]; then - echo "!! Could not find default location for $LINE package -- skipping package." - - continue - fi - - ## examine the list of installed items desribed in the bom - BOM_FILE=$PACKAGE_PATH/$BOM_INTERMEDIATE_DIR/$LINE.bom - lsbom $BOM_FILE > $TEMP_BOM_CONTENTS - while read BOM_ITEM; do - ## 3 column items describe just directories, 5 column items describe actual files - COL_COUNT=$(echo $BOM_ITEM | awk '{print NF;}') - if [ "$COL_COUNT" -eq 5 ]; then - FILE_NAME=$DEFAULT_LOC/$(echo $BOM_ITEM | awk '{print $1;}') - - echo " . Removing $FILE_NAME" - rm -rf $FILE_NAME - fi - done < $TEMP_BOM_CONTENTS - - ## remove package receipt - echo " . Removing package receipt $PACKAGE_PATH" - rm -rf $PACKAGE_PATH - fi -done < $PACKAGE_LIST_SRC - - -## nuke the user created by the install process. -echo ". Removing created user $TOR_USER" -if [ -x /usr/bin/dscl ]; then - dscl . -delete /users/$TOR_USER -else - niutil -destroy . /users/$TOR_USER -fi - -## clean up -echo ". Cleaning up" -rm -rf $TEMP_BOM_CONTENTS -rm -rf /Library/Privoxy/ /Library/StartupItems/Privoxy/ /Library/Tor/ /Library/StartupItems/Tor/ /Library/Torbutton/ /Library/Receipts/Privoxy.pkg /Library/Receipts/torbutton.pkg /Library/Receipts/Tor.pkg /Library/Receipts/Vidalia.pkg /Library/Receipts/TorStartup.pkg - -echo ". Finished" - diff --git a/contrib/polipo/Makefile.osx b/contrib/polipo/Makefile.osx deleted file mode 100644 index 55ed1c62f..000000000 --- a/contrib/polipo/Makefile.osx +++ /dev/null @@ -1,103 +0,0 @@ -PREFIX = /Library/Polipo -BINDIR = $(PREFIX)/bin -MANDIR = $(PREFIX)/man -INFODIR = $(PREFIX)/info -LOCAL_ROOT = $(PREFIX)/www -DISK_CACHE_ROOT = $(PREFIX)/cache - -CC = gcc -# CDEBUGFLAGS = -Os -g -Wall -std=gnu99 -CDEBUGFLAGS = -Os -g -Wall - -FILE_DEFINES = -DLOCAL_ROOT=\"$(LOCAL_ROOT)/\" \ - -DDISK_CACHE_ROOT=\"$(DISK_CACHE_ROOT)/\" - -# You may optionally also add any of the following to DEFINES: -# -# -DNO_DISK_CACHE to compile out the on-disk cache and local web server; -# -DNO_IPv6 to avoid using the RFC 3493 API and stick to stock -# Berkeley sockets; -# -DHAVE_IPv6 to force the use of the RFC 3493 API on systems other -# than GNU/Linux and BSD (let me know if it works); -# -DNO_FANCY_RESOLVER to compile out the asynchronous name resolution -# code; -# -DNO_STANDARD_RESOLVER to compile out the code that falls back to -# gethostbyname/getaddrinfo when DNS requests fail; -# -DNO_TUNNEL to compile out the code that handles CONNECT requests; -# -DNO_SOCKS to compile out the SOCKS gateway code. -# -DNO_FORBIDDEN to compile out the all of the forbidden URL code -# -DNO_REDIRECTOR to compile out the Squid-style redirector code - -DEFINES = $(FILE_DEFINES) $(PLATFORM_DEFINES) - -# Uncomment the UNIVERSAL, LDFLAGS, CFLAGS lines if you want universal binaries, otherwise -# you'll produce a binary only for your architecture and version of OSX -# UNIVERSAL = -O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc -# LDFLAGS = -Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk -# CFLAGS = $(MD5INCLUDES) $(CDEBUGFLAGS) $(DEFINES) $(EXTRA_DEFINES) $(UNIVERSAL) -# If you uncommented the above CFLAGS, remove this next one. -CFLAGS = $(MD5INCLUDES) $(CDEBUGFLAGS) $(DEFINES) $(EXTRA_DEFINES) - -SRCS = util.c event.c io.c chunk.c atom.c object.c log.c diskcache.c main.c \ - config.c local.c http.c client.c server.c auth.c tunnel.c \ - http_parse.c parse_time.c dns.c forbidden.c \ - md5import.c md5.c ftsimport.c fts_compat.c socks.c - -OBJS = util.o event.o io.o chunk.o atom.o object.o log.o diskcache.o main.o \ - config.o local.o http.o client.o server.o auth.o tunnel.o \ - http_parse.o parse_time.o dns.o forbidden.o \ - md5import.o ftsimport.o socks.o - -polipo: $(OBJS) - $(CC) $(CFLAGS) $(LDFLAGS) -o polipo $(OBJS) $(MD5LIBS) $(LDLIBS) - -ftsimport.o: ftsimport.c fts_compat.c - -md5import.o: md5import.c md5.c - -.PHONY: all install install.binary install.man - -all: polipo - -install: install.binary install.man - -install.binary: all - mkdir -p $(TARGET)$(BINDIR) - mkdir -p $(TARGET)$(LOCAL_ROOT) - mkdir -p $(TARGET)$(LOCAL_ROOT)/doc - mkdir -p $(TARGET)$(DISK_CACHE_ROOT) - cp -f polipo $(TARGET)$(BINDIR)/ - cp -f localindex.html $(TARGET)$(LOCAL_ROOT)/index.html - cp -f config.osx $(TARGET)$(LOCAL_ROOT)/config - texi2html polipo.texi && cp -f polipo.html $(TARGET)$(LOCAL_ROOT)/doc/index.html - groff polipo.man -T ps -m man | pstopdf -i -o $(TARGET)/polipo.pdf - mkdir -p /Library/StartupItems/Polipo - cp -f contrib/Polipo /Library/StartupItems/Polipo/ - cp -f contrib/StartupParameters.plist /Library/StartupItems/Polipo/ - echo "POLIPO=-YES-" >> /etc/hostconfig - -install.man: all - mkdir -p $(TARGET)$(MANDIR)/man1 - cp -f polipo.man $(TARGET)$(MANDIR)/man1/polipo.1 - -TAGS: $(SRCS) - etags $(SRCS) - -.PHONY: clean - -clean: - -rm -f polipo *.o *~ core TAGS gmon.out - -rm -f polipo.cp polipo.fn polipo.log polipo.vr - -rm -f polipo.cps polipo.info* polipo.pg polipo.toc polipo.vrs - -rm -f polipo.aux polipo.dvi polipo.ky polipo.ps polipo.tp - -rm -f polipo.dvi polipo.ps polipo.ps.gz polipo.pdf polipo.html - -rm -rf ./html/ - -rm -f polipo.man.html - -.PHONY: uninstall - -uninstall: - rm -rf /Library/Polipo - rm -rf /Library/StartupItems/Polipo - cp -f /etc/hostconfig /etc/hostconfig.polipo.saved - cat /etc/hostconfig.polipo.saved | grep -v POLIPO > /etc/hostconfig diff --git a/contrib/polipo/Makefile.osx-panther b/contrib/polipo/Makefile.osx-panther deleted file mode 100644 index d1d24d1f3..000000000 --- a/contrib/polipo/Makefile.osx-panther +++ /dev/null @@ -1,97 +0,0 @@ -PREFIX = /Library/Polipo -BINDIR = $(PREFIX)/bin -MANDIR = $(PREFIX)/man -INFODIR = $(PREFIX)/info -LOCAL_ROOT = $(PREFIX)/www -DISK_CACHE_ROOT = $(PREFIX)/cache - -CC = gcc -# CDEBUGFLAGS = -Os -g -Wall -std=gnu99 -CDEBUGFLAGS = -Os -g -Wall - -FILE_DEFINES = -DLOCAL_ROOT=\"$(LOCAL_ROOT)/\" \ - -DDISK_CACHE_ROOT=\"$(DISK_CACHE_ROOT)/\" - -# You may optionally also add any of the following to DEFINES: -# -# -DNO_DISK_CACHE to compile out the on-disk cache and local web server; -# -DNO_IPv6 to avoid using the RFC 3493 API and stick to stock -# Berkeley sockets; -# -DHAVE_IPv6 to force the use of the RFC 3493 API on systems other -# than GNU/Linux and BSD (let me know if it works); -# -DNO_FANCY_RESOLVER to compile out the asynchronous name resolution -# code; -# -DNO_STANDARD_RESOLVER to compile out the code that falls back to -# gethostbyname/getaddrinfo when DNS requests fail; -# -DNO_TUNNEL to compile out the code that handles CONNECT requests; -# -DNO_SOCKS to compile out the SOCKS gateway code. -# -DNO_FORBIDDEN to compile out the all of the forbidden URL code -# -DNO_REDIRECTOR to compile out the Squid-style redirector code - -DEFINES = $(FILE_DEFINES) $(PLATFORM_DEFINES) - -CFLAGS = $(MD5INCLUDES) $(CDEBUGFLAGS) $(DEFINES) $(EXTRA_DEFINES) - -SRCS = util.c event.c io.c chunk.c atom.c object.c log.c diskcache.c main.c \ - config.c local.c http.c client.c server.c auth.c tunnel.c \ - http_parse.c parse_time.c dns.c forbidden.c \ - md5import.c md5.c ftsimport.c fts_compat.c socks.c - -OBJS = util.o event.o io.o chunk.o atom.o object.o log.o diskcache.o main.o \ - config.o local.o http.o client.o server.o auth.o tunnel.o \ - http_parse.o parse_time.o dns.o forbidden.o \ - md5import.o ftsimport.o socks.o - -polipo: $(OBJS) - $(CC) $(CFLAGS) $(LDFLAGS) -o polipo $(OBJS) $(MD5LIBS) $(LDLIBS) - -ftsimport.o: ftsimport.c fts_compat.c - -md5import.o: md5import.c md5.c - -.PHONY: all install install.binary install.man - -all: polipo - -install: install.binary install.man - -install.binary: all - mkdir -p $(TARGET)$(BINDIR) - mkdir -p $(TARGET)$(LOCAL_ROOT) - mkdir -p $(TARGET)$(LOCAL_ROOT)/doc - mkdir -p $(TARGET)$(DISK_CACHE_ROOT) - cp -f polipo $(TARGET)$(BINDIR)/ - cp -f localindex.html $(TARGET)$(LOCAL_ROOT)/index.html - cp -f config.osx $(TARGET)$(LOCAL_ROOT)/config - texi2html polipo.texi && cp -f polipo.html $(TARGET)$(LOCAL_ROOT)/doc/index.html - groff polipo.man -T ps -m man | pstopdf -i -o $(TARGET)/polipo.pdf - mkdir -p /Library/StartupItems/Polipo - cp -f contrib/Polipo /Library/StartupItems/Polipo/ - cp -f contrib/StartupParameters.plist /Library/StartupItems/Polipo/ - echo "POLIPO=-YES-" >> /etc/hostconfig - -install.man: all - mkdir -p $(TARGET)$(MANDIR)/man1 - cp -f polipo.man $(TARGET)$(MANDIR)/man1/polipo.1 - -TAGS: $(SRCS) - etags $(SRCS) - -.PHONY: clean - -clean: - -rm -f polipo *.o *~ core TAGS gmon.out - -rm -f polipo.cp polipo.fn polipo.log polipo.vr - -rm -f polipo.cps polipo.info* polipo.pg polipo.toc polipo.vrs - -rm -f polipo.aux polipo.dvi polipo.ky polipo.ps polipo.tp - -rm -f polipo.dvi polipo.ps polipo.ps.gz polipo.pdf polipo.html - -rm -rf ./html/ - -rm -f polipo.man.html - -.PHONY: uninstall - -uninstall: - rm -rf /Library/Polipo - rm -rf /Library/StartupItems/Polipo - cp -f /etc/hostconfig /etc/hostconfig.polipo.saved - cat /etc/hostconfig.polipo.saved | grep -v POLIPO > /etc/hostconfig diff --git a/contrib/polipo/Polipo b/contrib/polipo/Polipo deleted file mode 100644 index 14589c84d..000000000 --- a/contrib/polipo/Polipo +++ /dev/null @@ -1,40 +0,0 @@ -#!/bin/sh - -## -# Polipo -## - -. /etc/rc.common - -StartService () -{ - if [ -f /Library/Polipo/bin/polipo ]; then - if pid=$(GetPID polipo); then - return 0 - else if [ "${POLIPO:=-NO-}" = "-YES-" ]; then - ConsoleMessage "Starting Polipo" - /Library/Polipo/bin/polipo -c /Library/Polipo/www/config - fi - fi - fi -} - -StopService () -{ - if pid=$(GetPID polipo); then - ConsoleMessage "Stopping Polipo" - kill -TERM "${pid}" - else - ConsoleMessage "Polipo not responding." -# Just for sanity (sometimes necessary.) - killall tor 2>/dev/null - fi -} - -RestartService () -{ - StopService - StartService -} - -RunService "$1" diff --git a/contrib/polipo/PolipoDesc.plist b/contrib/polipo/PolipoDesc.plist deleted file mode 100644 index 88b7612ff..000000000 --- a/contrib/polipo/PolipoDesc.plist +++ /dev/null @@ -1,10 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>IFPkgDescriptionTitle</key> - <string>Polipo</string> - <key>IFPkgDescriptionVersion</key> - <string>1.0.4</string> -</dict> -</plist> diff --git a/contrib/polipo/PolipoInfo.plist b/contrib/polipo/PolipoInfo.plist deleted file mode 100644 index aa018f2d9..000000000 --- a/contrib/polipo/PolipoInfo.plist +++ /dev/null @@ -1,28 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>CFBundleIdentifier</key> - <string>Polipo 1.0.4</string> - <key>CFBundleGetInfoString</key> - <string>Polipo 1.0.4</string> - <key>CFBundleName</key> - <string>Polipo</string> - <key>CFBundleSortVersionString</key> - <string>1.0.4</string> - <key>IFPkgFlagAllowBackRev</key> - <true/> - <key>IFPkgFlagAuthorizationAction</key> - <string>RootAuthorization</string> - <key>IFPkgFlagFollowLinks</key> - <true/> - <key>IFPkgFlagIsRequired</key> - <true/> - <key>IFPkgFlagRootVolumeOnly</key> - <true/> - <key>IfPkgFlagBackgroundScaling</key> - <string>proportional</string> - <key>IFPkgFlagBackgroundAlignment</key> - <string>bottomleft</string> -</dict> -</plist> diff --git a/contrib/polipo/PolipoPostflight b/contrib/polipo/PolipoPostflight deleted file mode 100644 index 0894caec6..000000000 --- a/contrib/polipo/PolipoPostflight +++ /dev/null @@ -1,50 +0,0 @@ -#!/bin/sh - -# PolipoPostflight gets invoked after any install or upgrade. - -ADDSYSUSER=$RECEIPT_PATH/addsysuser -if [ ! -x "$ADDSYSUSER" ]; then - echo "Could not find addsysuser script." - exit 1 -fi - -POLIPOUSER=_polipo -POLIPOGROUP=daemon -TARGET=/Library/Polipo -LOGDIR=$TARGET/log - -# Create user $POLIPOUSER in group daemon. If it's already there, great. -$ADDSYSUSER $POLIPOUSER "Polipo System user" $POLIPODIR - -# Create the polipo direcpolipoy, if it doesn't exist. -if [ ! -d $POLIPODIR ]; then - mkdir -p $POLIPODIR -fi -if [ ! -d $LOGDIR ]; then - mkdir -p $LOGDIR -fi -# Check its permissions. -chown $POLIPOUSER $POLIPODIR -chgrp daemon $POLIPODIR -chmod 700 $POLIPODIR -chown $POLIPOUSER $LOGDIR -chgrp daemon $LOGDIR -chmod 700 $LOGDIR - -# Create the configuration file only if there wasn't one already. -if [ ! -f $TARGET/config]; then - cp $TARGET/config.osx $TARGET/config -fi - -cd /usr/share/man/man1 -MAN1=$TARGET/share/man/man1 - -if [ -d /Library/StartupItems/Polipo ]; then - find /Library/StartupItems/Polipo -print0 | xargs -0 chown root:wheel -fi - -# Copy Uninstaller -if [ -f $PACKAGE_PATH/Contents/Resources/uninstall_polipo_bundle.sh ]; then - cp $PACKAGE_PATH/Contents/Resources/uninstall_polipo_bundle.sh $TARGET/uninstall_polipo_bundle.sh - chmod 755 $TARGET/uninstall_polipo_bundle.sh -fi diff --git a/contrib/polipo/README b/contrib/polipo/README index be3d4b63b..1110ca273 100644 --- a/contrib/polipo/README +++ b/contrib/polipo/README @@ -6,14 +6,14 @@ General Comments ---------------- These are some hacks for making polipo work and install a package native -to OSX or Windows. +to Windows. They need some work before they can be committed upstream: - - Merge the three makefiles into one with specific builds such as "make - dist-osx" or "make dist-win32" + - Change the Makefile so it has a specific build such as "make + dist-win32" - Configure the options for tor in polipo config, just leave them commented out for easy activation. - - Work out better polipo config options for Tor. + - Work out better polipo config options for Tor. As always, I'm happy to accept patches. @@ -45,12 +45,3 @@ choose Compile NSIS Script. You'll then create a polipo installer. The Polipo NSI installer assumes libgnurx-0.dll is in the same directory as polipo.exe. You'll need to copy libgnurx-0.dll into "./" in order to make the installation package. - ---------------------------------------------- -OSX Universal Binary and Installation package ---------------------------------------------- -1) Copy Makefile.osx over Makefile. -2) Run 'make'. -3) Copy the contents of this directory into a directory named "contrib". -4) Run './contrib/package.sh' -5) You should have a Polipo-version.dmg ready for installation. diff --git a/contrib/polipo/StartupParameters.plist b/contrib/polipo/StartupParameters.plist deleted file mode 100644 index 8309e930c..000000000 --- a/contrib/polipo/StartupParameters.plist +++ /dev/null @@ -1,11 +0,0 @@ -{ - Description = "Polipo"; - Provides = ("Polipo"); - Requires = ("NetworkExtensions","Resolver"); - OrderPreference = "Late"; - Messages = - { - start = "Starting Polipo"; - stop = "Stopping Polipo"; - }; -} diff --git a/contrib/polipo/addsysuser b/contrib/polipo/addsysuser deleted file mode 100644 index 7b167eac0..000000000 --- a/contrib/polipo/addsysuser +++ /dev/null @@ -1,68 +0,0 @@ -#!/bin/sh -# -# Original adduser 05 Feb 2002 by Jon L. Gardner -# -# Modified for Tor installer by Nick Mathewson -# 2007-06-12 Modified for leopard by Andrew Lewman - - -ROOTPROP=/ - -if [ "`whoami`" != "root" ]; then -echo "You must be root to execute this script." -exit -fi -if [ "x$3" = "x" ]; then -echo 'Usage: addsysuser <username> "<full name>" <homedir>' -exit 0 -fi -username=$1 -realname=$2 -homedir=$3 -if [ -x /usr/bin/dscl ]; then - # Determine the gid of the daemon group - gid=`dscl . -read /groups/daemon gid` - if [ "x`dscl . -list /users|cut -f2 -d' '|grep $username`" != "x" ]; then - echo The account $username already exists. - exit 0 - fi - # home is the local path to the home directory - home=/Users/$username - echo Creating account for $username... - dscl . -create /users/$username - dscl . -create /users/$username _writers_tim_passwd $username - dscl . -create /users/$username realname $realname - dscl . -create /users/$username _writers_passwd $username - dscl . -create /users/$username gid $gid - dscl . -create /users/$username home $homedir - dscl . -create /users/$username name $username - dscl . -create /users/$username passwd '*' - dscl . -create /users/$username shell /dev/null -else - # Determine the gid of the daemon group - gid=`niutil -readprop $ROOTPROP /groups/daemon gid` - if [ "x`niutil -list $ROOTPROP /users|cut -f2 -d' '|grep $username`" != "x" ]; then - echo The account $username already exists. - exit 0 - fi - # home is the local path to the home directory - home=/Users/$username - # defhome is what goes into NetInfo - defhome="/Network/Servers/MyServer/Users" - #echo "Determining next available system uid (please be patient)..." - # Uids over 500 are for system users. - uiddef=`nidump passwd / | cut -d: -f3 | sort -n | grep -v '^[56789]..' |grep -v '^....$' | tail -n 1` - uiddef=`echo $uiddef + 1 |bc` - echo Creating account for $username... - niutil -create $ROOTPROP /users/$username - niutil -createprop $ROOTPROP /users/$username _writers_tim_passwd $username - niutil -createprop $ROOTPROP /users/$username realname $realname - niutil -createprop $ROOTPROP /users/$username _writers_passwd $username - niutil -createprop $ROOTPROP /users/$username uid $uiddef - #niutil -createprop $ROOTPROP /users/$username home_loc "<home_dir><url>afp://afp.server.com/Users/</url><path>$username</path></home_dir>" - niutil -createprop $ROOTPROP /users/$username gid $gid - niutil -createprop $ROOTPROP /users/$username home $homedir - niutil -createprop $ROOTPROP /users/$username name $username - niutil -createprop $ROOTPROP /users/$username passwd '*' - niutil -createprop $ROOTPROP /users/$username shell /dev/null -fi diff --git a/contrib/polipo/package.sh b/contrib/polipo/package.sh deleted file mode 100644 index 83f74212b..000000000 --- a/contrib/polipo/package.sh +++ /dev/null @@ -1,87 +0,0 @@ -#!/bin/sh -# $Id: package.sh 8992 2006-12-23 03:12:09Z phobos $ -# Copyright 2004-2005 Nick Mathewson & Andrew Lewman. -# Copyright 2005-2008 Andrew Lewman -# This is licensed under the Modified BSD License. - -### -# Helpful info on OS X packaging: -# http://developer.apple.com/documentation/DeveloperTools/Conceptual/SoftwareDistribution/index.html -# man packagemaker - -VERSION="1.0.4" - -## Determine OSX Version -# map version to name -if [ -x /usr/bin/sw_vers ]; then -# This is poor, yet functional. We don't care about the 3rd number in -# the OS version - OSVER=`/usr/bin/sw_vers | grep ProductVersion | cut -f2 | cut -d"." -f1,2` - case "$OSVER" in - "10.5") ARCH="universal";; - "10.4") ARCH="universal";; - "10.3") ARCH="ppc";; - "10.2") ARCH="ppc";; - "10.1") ARCH="ppc";; - "10.0") ARCH="ppc";; - *) ARCH="unknown";; - esac -else - ARCH="unknown" -fi - -# Where will we put our temporary files? -BUILD_DIR=/tmp/polipo-osx-$$ -# Path to PackageMaker app. -PACKAGEMAKER=/Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker - -umask 022 - -echo I might ask you for your password now, so you can sudo. - -sudo rm -rf $BUILD_DIR -mkdir $BUILD_DIR || exit 1 -for subdir in polipo_packageroot output; do - mkdir $BUILD_DIR/$subdir -done - -### Make Polipo package. -chmod 755 contrib/PolipoPostflight -mkdir -p $BUILD_DIR/polipo_packageroot/Library/Polipo/ -cp polipo $BUILD_DIR/polipo_packageroot/polipo -cp config.sample $BUILD_DIR/polipo_packageroot/config -cp contrib/PolipoPostflight $BUILD_DIR/polipo_packageroot/postflight -cp contrib/addsysuser $BUILD_DIR/polipo_packageroot/addsysuser -cp contrib/uninstall_polipo_bundle.sh $BUILD_DIR/polipo_packageroot/uninstall_polipo_bundle.sh -cp localindex.html $BUILD_DIR/polipo_packageroot/index.html -cat <<EOF > $BUILD_DIR/polipo_packageroot/Welcome.txt -Polipo: a caching web proxy - -Polipo is a small and fast caching web proxy (a web cache, an HTTP -proxy, a proxy server). -EOF - -### Assemble documentation - -groff polipo.man -T ps -m man | pstopdf -i -o $BUILD_DIR/polipo_packageroot/polipo.pdf -texi2html polipo.texi && cp polipo.html $BUILD_DIR/polipo_packageroot/polipo.html - -find $BUILD_DIR/polipo_packageroot -print0 |sudo xargs -0 chown root:wheel - -$PACKAGEMAKER -build \ - -p $BUILD_DIR/output/Polipo.pkg \ - -f $BUILD_DIR/polipo_packageroot \ - -i contrib/PolipoInfo.plist \ - -d contrib/PolipoDesc.plist - -### Package it all into a DMG - -find $BUILD_DIR/output -print0 | sudo xargs -0 chown root:wheel - -mv $BUILD_DIR/output "$BUILD_DIR/Polipo-$VERSION-$ARCH" -rm -f "Polipo-$VERSION-$ARCH-Bundle.dmg" -USER="`whoami`" -sudo hdiutil create -format UDZO -srcfolder "$BUILD_DIR/Polipo-$VERSION-$ARCH" "Polipo-$VERSION-$ARCH.dmg" -sudo chown "$USER" "Polipo-$VERSION-$ARCH.dmg" - -#sudo rm -rf $BUILD_DIR diff --git a/contrib/polipo/uninstall_polipo_bundle.sh b/contrib/polipo/uninstall_polipo_bundle.sh deleted file mode 100644 index 73507701a..000000000 --- a/contrib/polipo/uninstall_polipo_bundle.sh +++ /dev/null @@ -1,100 +0,0 @@ -#!/bin/sh -# -# Original version 2005 by loki der quaeler -# Copyright 2007-2008 Andrew Lewman -# This is licensed under a Modified BSD license. - - -### this is the location of a file which contains all the actual package names -## (ie "Polipo", "polipostartup", ...) the list should be new-line-delimited. -PACKAGE_LIST_SRC="Polipo polipostartup" - -### this is the name of the user created in the install process of Polipo -POLIPO_USER=_polipo - -### these should be constant across all osX installs (so leave them be) -STARTUP_ITEMS_DIR=/Library/StartupItems -PKG_RCPT_BASE_DIR=/Library/Receipts -BOM_INTERMEDIATE_DIR=Contents/Resources -INFO_INTERMEDIATE_DIR=$BOM_INTERMEDIATE_DIR/English.lproj -TEMP_BOM_CONTENTS=/tmp/polipo_uninst_scratch - - -### make sure the script is being run as root, barf if not -if [ "`whoami`" != "root" ]; then - echo "Must be root to run the uninstall script." - exit -1 -fi - -### check to see if polipo is currently running, kill it if it is -## we grep on 'Polipo/polipo ' because 'polipo' is too common (like in 'direcpolipoy') -## -- this relies on the fact that polipo has been started with command -## line arguments.. :-/ -POLIPO_PID=`ps -uax | grep 'Polipo/polipo ' | grep -v grep | awk '{print $2;}'` -if [ ${#POLIPO_PID} -gt 0 ]; then - echo ". Killing currently running polipo process, pid is $POLIPO_PID" - kill -9 $POLIPO_PID -else - echo ". polipo process appears to already be stopped" -fi - - -## grab each package name from the package list file -while read LINE; do - if [ ${#LINE} -gt 0 ]; then - PACKAGE_NAME=$LINE.pkg - PACKAGE_PATH=$PKG_RCPT_BASE_DIR/$PACKAGE_NAME - echo ". Uninstalling $PACKAGE_NAME" - if [ ! -d $PACKAGE_PATH ]; then - echo " . No receipt exists for this package -- skipping." - - continue - fi - - - ## get rid of the startup item if it exists - STARTUP_DIR=$STARTUP_ITEMS_DIR/$LINE - if [ -d $STARTUP_DIR ]; then - echo " . Deleting startup item $STARTUP_DIR" - rm -rf $STARTUP_DIR - fi - - - ## determine the root direcpolipoy of the the relative paths specified in the bom - DEFAULT_LOC=`grep DefaultLocation $PACKAGE_PATH/$INFO_INTERMEDIATE_DIR/$LINE.info | awk '{print $2;}'` - if [ ${#DEFAULT_LOC} -eq 0 ]; then - echo "!! Could not find default location for $LINE package -- skipping package." - - continue - fi - - ## examine the list of installed items desribed in the bom - BOM_FILE=$PACKAGE_PATH/$BOM_INTERMEDIATE_DIR/$LINE.bom - lsbom $BOM_FILE > $TEMP_BOM_CONTENTS - while read BOM_ITEM; do - ## 3 column items describe just direcpolipoies, 5 column items describe actual files - COL_COUNT=$(echo $BOM_ITEM | awk '{print NF;}') - if [ "$COL_COUNT" -eq 5 ]; then - FILE_NAME=$DEFAULT_LOC/$(echo $BOM_ITEM | awk '{print $1;}') - - echo " . Removing $FILE_NAME" - rm -rf $FILE_NAME - fi - done < $TEMP_BOM_CONTENTS - - ## remove package receipt - echo " . Removing package receipt $PACKAGE_PATH" - rm -rf $PACKAGE_PATH - fi -done < $PACKAGE_LIST_SRC - -## nuke the user created by the install process. -echo ". Removing created user $POLIPO_USER" -niutil -destroy . /users/$POLIPO_USER - -## clean up -echo ". Cleaning up" -rm -rf $TEMP_BOM_CONTENTS -rm -rf /Library/Polipo/ /Library/StartupItems/Polipo/ -echo ". Finished" - diff --git a/contrib/privoxy-tor-toggle b/contrib/privoxy-tor-toggle deleted file mode 100644 index 8f9cd51bd..000000000 --- a/contrib/privoxy-tor-toggle +++ /dev/null @@ -1,73 +0,0 @@ -#!/bin/sh -# A script to turn Tor SOCKS4a in Privoxy on or off. - -CONFFILE=/etc/privoxy/config # privoxy config file. -TOR_REG="forward.*localhost:9050" # Regular expression to find Tor in privoxy -PRIVOXY="/etc/init.d/privoxy restart" # command to reload privoxy config file. -SED="/bin/sed" # sed command, of course. -GREP="/bin/grep" # grep command. - -usage () { -echo "\ -privoxy-tor-toggle: Change Privoxy's configuration to use/not use Tor. -Usage: - privoxy.tor <-- Switch Tor on or off. - privoxy.tor [on|off] <-- Set Tor on or off. - privoxy.tor status <-- Display Tor's current status. - privoxy.tor [-h|--help|-?] <-- Print usage. -" -} - -# Find out the current status of tor. Set $tor_status -get_status () { - gret=`$GREP -l -e "^$TOR_REG" $CONFFILE` - if [ x$gret = x ] ; then - tor_status=off; - else - tor_status=on; - fi - return -} - -# Turn tor on/off according to $1 -set_tor () { - tor_gate=$1 - get_status - if [ $tor_status = $tor_gate ] ; then - echo "Tor is already $1." - return - elif [ $tor_gate = flip ] ; then - if [ $tor_status = on ] ; then - tor_gate=off - elif [ $tor_status = off ] ; then - tor_gate=on - fi - fi - echo "Turning Tor $tor_gate..." - if [ $tor_gate = on ] ; then - reg=s/^#\($TOR_REG\)/\\1/ - $SED -i.bak -r "$reg" $CONFFILE - else - reg=s/^\($TOR_REG\)/#\\1/ - $SED -i.bak -r "$reg" $CONFFILE - fi - $PRIVOXY - return 0; -} - -if [ x$1 = x ] ; then - set_tor flip -elif [ $1 = on ] ; then - set_tor on -elif [ $1 = off ] ; then - set_tor off -elif [ $1 = status ] ; then - get_status - echo "Tor is $tor_status" -elif [ $1 = --help ] || [ $1 = -h ] || [ $1 = "-?" ] ; then - usage - exit 0 -else - echo "Unrecognized option: \"$1\"" -fi - diff --git a/contrib/proxy-some-domains b/contrib/proxy-some-domains deleted file mode 100644 index eb238a2fe..000000000 --- a/contrib/proxy-some-domains +++ /dev/null @@ -1,52 +0,0 @@ -Subject: -Re: Anonymous/Nonymous Communication Coexisting? -From: -Kristian Köhntopp <kris@xn--khntopp-90a.de> -Date: -Fri, 10 Jun 2005 08:56:19 +0200 -To: -or-talk@freehaven.net - -On Wednesday 08 June 2005 04:20, yancm@sdf.lonestar.org wrote: - ->> Is it possible to have a single application, such as a web ->> browser or a p2p client behave normally with normal url's but ->> use tor if the url is an xyz.onion address? Or is it ->> everything or nothing? - - -This is basically a question of using your proxy or not. You can -control the behaviour of your browser in great detail writing a -proxy.pac program in Javascript and setting that program as the -proxy autoconfiguration URL in your browser. - -An example: - -kris@jordan01:~> cat /srv/www/htdocs/proxy.pac - -function FindProxyForURL(url, host) -{ - var proxy_yes = "PROXY jordan01.int.cinetic.de:3128"; - var proxy_no = "DIRECT"; - - // Redirect all accesses to mlan hosts to the mlan proxy - if (dnsDomainIs(host, ".mlan.cinetic.de")) { - return proxy_yes; - } - - // Everything else is direct - return proxy_no; -} - -So here the program checks if the destination is a mlan-Host, and -if so, uses the appropriate proxy on jordan for the access, -while all other accesses are direct. - -You could do a similar thing with .onion accesses with a trivial -modification. - -Docs: -http://wp.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html - -Kristian - diff --git a/contrib/rc.subr b/contrib/rc.subr index 117ae71d4..d757e8952 100644 --- a/contrib/rc.subr +++ b/contrib/rc.subr @@ -1,5 +1,4 @@ #!/bin/sh -# $Id$ # $FreeBSD: ports/security/tor-devel/files/tor.in,v 1.1 2006/02/17 22:21:25 mnag Exp $ # # (rc.subr written by Peter Thoenen for Net/FreeBSD) diff --git a/contrib/suse/tor.sh.in b/contrib/suse/tor.sh.in index 5dad6499b..b7e9005eb 100644 --- a/contrib/suse/tor.sh.in +++ b/contrib/suse/tor.sh.in @@ -51,6 +51,8 @@ export TORUSER TORGROUP=@TORGROUP@ export TORGROUP +TOR_DAEMON_PID_DIR="@LOCALSTATEDIR@/run/tor" + if [ -x /bin/su ] ; then SUPROG=/bin/su elif [ -x /sbin/su ] ; then @@ -67,6 +69,12 @@ case "$1" in start) echo "Starting tor daemon" + + if [ ! -d $TOR_DAEMON_PID_DIR ] ; then + mkdir -p $TOR_DAEMON_PID_DIR + chown $TORUSER:$TORGROUP $TOR_DAEMON_PID_DIR + fi + ## Start daemon with startproc(8). If this fails ## the echo return value is set appropriate. diff --git a/contrib/tor-0.1.2.17.tar.gz.metalink.in b/contrib/tor-0.1.2.17.tar.gz.metalink.in deleted file mode 100644 index 559748865..000000000 --- a/contrib/tor-0.1.2.17.tar.gz.metalink.in +++ /dev/null @@ -1,41 +0,0 @@ -<?xml version="1.0" encoding="utf-8"?> -<metalink version="3.0" generator="Metalink Editor version 1.1.0" xmlns="http://www.metalinker.org/"> - <publisher> - <name>The Tor Project</name> - <url>https://www.torproject.org</url> - </publisher> - <license> - <name>BSD</name> - <url>http://opensource.org/licenses/bsd-license.php</url> - </license> - <identity>Tor</identity> - <version>@VERSION@</version> - <copyright>2007 The Tor Project, Inc.</copyright> - <description>Anonymity Online</description> - <files> - <file name="tor-@VERSION@.tar.gz"> - <size>1251636</size> - <language>en</language> - <os>Source</os> - <verification> - <hash type="md5">ef8fc7f45d167875c337063d437c9832</hash> - <hash type="sha1">01092fb75c407b5c1d7f33db069cf7641973d94d</hash> - <hash type="sha256">fc0fb0c2891ae09854a69512c6b4988964f2eaf62ce80ed6644cb21f87f6056a</hash> - <pieces type="sha1" length="262144"> - <hash piece="0">c778dd01e05734d57f769082545f9802386e42bb</hash> - <hash piece="1">39b172ed8b9290884c7bd129db633a79e28d5ae9</hash> - <hash piece="2">28d708e7489a1e9951e757443672535aedfa3abe</hash> - <hash piece="3">a7623e07081819a37300de0511bbdda0bdc960bd</hash> - <hash piece="4">f246021e55affe320a1f86eac5b049dd0caad828</hash> - </pieces> - </verification> - <resources> - <url type="http" location="at">http://tor.cypherpunks.at/dist/</url> - <url type="http" location="ca">http://tor.depthstrike.com/dist/</url> - <url type="http" location="ca">http://tor.hermetix.org/dist/</url> - <url type="http" location="ch">http://tor.boinc.ch/dist/</url> - <url type="http" location="cn">http://tor.anonymity.cn/dist/</url> - </resources> - </file> - </files> -</metalink> diff --git a/contrib/tor-mingw.nsi.in b/contrib/tor-mingw.nsi.in index 6deb8d96a..a44961b02 100644 --- a/contrib/tor-mingw.nsi.in +++ b/contrib/tor-mingw.nsi.in @@ -8,8 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters - -!define VERSION "0.2.1.25" +!define VERSION "0.2.2.19-alpha" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/contrib/tor-resolve.py b/contrib/tor-resolve.py index 919bc876c..47ae1a0c3 100755 --- a/contrib/tor-resolve.py +++ b/contrib/tor-resolve.py @@ -1,5 +1,4 @@ #!/usr/bin/python -#$Id$ import socket import struct diff --git a/contrib/tor.sh.in b/contrib/tor.sh.in index e169761a6..92f890681 100644 --- a/contrib/tor.sh.in +++ b/contrib/tor.sh.in @@ -16,6 +16,15 @@ # description: Onion Router - A low-latency anonymous proxy # +PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin +DAEMON=/usr/sbin/tor +NAME=tor +DESC="tor daemon" +TORPIDDIR=/var/run/tor +TORPID=$TORPIDDIR/tor.pid +WAITFORDAEMON=60 +ARGS="" + # Library functions if [ -f /etc/rc.d/init.d/functions ]; then . /etc/rc.d/init.d/functions @@ -23,9 +32,6 @@ elif [ -f /etc/init.d/functions ]; then . /etc/init.d/functions fi -# Increase open file descriptors a reasonable amount -ulimit -n 8192 - TORCTL=@BINDIR@/torctl # torctl will use these environment variables @@ -44,9 +50,47 @@ else SUPROG=/bin/su fi +# Raise ulimit based on number of file descriptors available (thanks, Debian) + +if [ -r /proc/sys/fs/file-max ]; then + system_max=`cat /proc/sys/fs/file-max` + if [ "$system_max" -gt "80000" ] ; then + MAX_FILEDESCRIPTORS=32768 + elif [ "$system_max" -gt "40000" ] ; then + MAX_FILEDESCRIPTORS=16384 + elif [ "$system_max" -gt "10000" ] ; then + MAX_FILEDESCRIPTORS=8192 + else + MAX_FILEDESCRIPTORS=1024 + cat << EOF + +Warning: Your system has very few filedescriptors available in total. + +Maybe you should try raising that by adding 'fs.file-max=100000' to your +/etc/sysctl.conf file. Feel free to pick any number that you deem appropriate. +Then run 'sysctl -p'. See /proc/sys/fs/file-max for the current value, and +file-nr in the same directory for how many of those are used at the moment. + +EOF + fi +else + MAX_FILEDESCRIPTORS=8192 +fi + +NICE="" + case "$1" in start) + if [ -n "$MAX_FILEDESCRIPTORS" ]; then + echo -n "Raising maximum number of filedescriptors (ulimit -n) to $MAX_FILEDESCRIPTORS" + if ulimit -n "$MAX_FILEDESCRIPTORS" ; then + echo "." + else + echo ": FAILED." + fi + fi + action $"Starting tor:" $TORCTL start RETVAL=$? ;; diff --git a/contrib/torify.1 b/contrib/torify.1 deleted file mode 100644 index b08d46845..000000000 --- a/contrib/torify.1 +++ /dev/null @@ -1,32 +0,0 @@ -.TH torify 1 "" Jan-2009 "" -.\" manual page by Peter Palfrader -.SH NAME -.LP -torify \- wrapper for tsocks and tor - -.SH SYNOPSIS -\fBtorify\fP\ \fIapplication\fP\ [\fIapplication's\ arguments\fP] - -.SH DESCRIPTION -\fBtorify\fR is a simple wrapper that calls tsocks with a tor specific -configuration file. - -tsocks itself is a wrapper between the tsocks library and the application -that you would like to run socksified. - -Please note that since tsocks uses LD_PRELOAD, torify cannot be applied -to suid binaries. - -You should also be aware that the way tsocks currently works only TCP -connections are socksified. Be aware that this will in most circumstances -not include hostname lookups which would still be routed through your -normal system resolver to your usual resolving nameservers. The -\fBtor-resolve\fR(1) tool can be useful as a workaround in some cases. -The Tor FAQ at https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ might -have further information on this subject. - -.SH SEE ALSO -.BR tor (1), -.BR tor-resolve (1), -.BR tsocks (1), -.BR tsocks.conf (5). diff --git a/contrib/torify.in b/contrib/torify.in index 05645fd07..d430da8ce 100755 --- a/contrib/torify.in +++ b/contrib/torify.in @@ -3,43 +3,69 @@ # Wrapper script for use of the tsocks(8) transparent socksification library # See the tsocks(1) and torify(1) manpages. -# Copyright (c) 2004, 2006 Peter Palfrader +# Copyright (c) 2004, 2006, 2009 Peter Palfrader # Modified by Jacob Appelbaum <jacob@appelbaum.net> April 16th 2006 # May be distributed under the same terms as Tor itself +# taken from Debian's Developer's Reference, 6.4 +pathfind() { + OLDIFS="$IFS" + IFS=: + for p in $PATH; do + if [ -x "$p/$*" ]; then + IFS="$OLDIFS" + return 0 + fi + done + IFS="$OLDIFS" + return 1 +} -# Define and ensure we have tsocks -# XXX: what if we don't have which? -TSOCKS="`which tsocks`" -if [ ! -x "$TSOCKS" ] -then - echo "$0: Can't find tsocks in PATH. Perhaps you haven't installed it?" >&2 - exit 1 +# Check for any argument list +if [ "$#" = 0 ]; then + echo "Usage: $0 [-hv] <command> [<options>...]" >&2 + exit 1 fi -# Check for any argument list -if [ "$#" = 0 ] -then - echo "Usage: $0 <command> [<options>...]" >&2 - exit 1 +if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] ); then + echo "Usage: $0 [-hv] <command> [<options>...]" + exit 0 fi -if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] ) -then - echo "Usage: $0 <command> [<options>...]" - exit 0 + +if [ "$1" = "-v" ] || [ "$1" = "--verbose" ]; then + verbose=1 + shift 1 +else + verbose=0 fi -# Define our tsocks config file -TSOCKS_CONF_FILE="@CONFDIR@/tor-tsocks.conf" -export TSOCKS_CONF_FILE +if pathfind torsocks; then + ! [ "$verbose" -ge 1 ] || echo "Using torsocks as socksifier." >&2 -# Check that we've got a tsocks config file -if [ -r "$TSOCKS_CONF_FILE" ] -then - exec tsocks "$@" - echo "$0: Failed to exec tsocks $@" >&2 + exec torsocks "$@" + echo "$0: Failed to exec torsocks $@" >&2 exit 1 + +elif pathfind tsocks; then + ! [ "$verbose" -ge 1 ] || echo "Using tsocks as socksifier." >&2 + + # Define our tsocks config file + TSOCKS_CONF_FILE="/etc/tor/tor-tsocks.conf" + export TSOCKS_CONF_FILE + + # Check that we've got a tsocks config file + if [ -r "$TSOCKS_CONF_FILE" ] + then + echo "WARNING: tsocks is known to leak DNS and UDP data. If you had torsocks we would use that." >&2 + exec tsocks "$@" + echo "$0: Failed to exec tsocks $@" >&2 + exit 1 + else + echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2 + exit 1 + fi + else - echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2 + echo "$0: Can't find either tsocks or torsocks in your PATH. Perhaps you haven't installed either?" >&2 exit 1 fi diff --git a/doc/FAQ b/doc/FAQ deleted file mode 100644 index 3a7db3ae6..000000000 --- a/doc/FAQ +++ /dev/null @@ -1,4 +0,0 @@ -This file is obsolete. Check out the online FAQ at the wiki -for more accurate and complete questions and answers: -http://wiki.noreply.org/wiki/TheOnionRouter/TorFAQ - diff --git a/doc/HACKING b/doc/HACKING index 50b5d80d1..7ff9c5f3c 100644 --- a/doc/HACKING +++ b/doc/HACKING @@ -1,41 +1,189 @@ +Hacking Tor: An Incomplete Guide +================================ -0. Useful tools. +Getting started +--------------- -0.0 The buildbot. +For full information on how Tor is supposed to work, look at the files in +https://gitweb.torproject.org/torspec.git/tree - http://tor-buildbot.freehaven.net:8010/ +For an explanation of how to change Tor's design to work differently, look at +https://gitweb.torproject.org/torspec.git/blob_plain/HEAD:/proposals/001-process.txt - - Down because nickm isn't running services at home any more. ioerror says - he will resurrect it. +For the latest version of the code, get a copy of git, and -0.1. Useful command-lines that are non-trivial to reproduce but can -help with tracking bugs or leaks. + git clone git://git.torproject.org/git/tor . -dmalloc -l ~/dmalloc.log -(run the commands it tells you) -./configure --with-dmalloc +We talk about Tor on the tor-talk mailing list. Design proposals and +discussion belong on the tor-dev mailing list. We hang around on +irc.oftc.net, with general discussion happening on #tor and development +happening on #tor-dev. + +How we use Git branches +----------------------- + +Each main development series (like 0.2.1, 0.2.2, etc) has its main work +applied to a single branch. At most one series can be the development series +at a time; all other series are maintenance series that get bug-fixes only. +The development series is built in a git branch called "master"; the +maintenance series are built in branches called "maint-0.2.0", "maint-0.2.1", +and so on. We regularly merge the active maint branches forward. + +For all series except the development series, we also have a "release" branch +(as in "release-0.2.1"). The release series is based on the corresponding +maintenance series, except that it deliberately lags the maint series for +most of its patches, so that bugfix patches are not typically included in a +maintenance release until they've been tested for a while in a development +release. Occasionally, we'll merge an urgent bugfix into the release branch +before it gets merged into maint, but that's rare. + +If you're working on a bugfix for a bug that occurs in a particular version, +base your bugfix branch on the "maint" branch for the first _actively +developed_ series that has that bug. (Right now, that's 0.2.1.) If you're +working on a new feature, base it on the master branch. + + +How we log changes +------------------ + +When you do a commit that needs a ChangeLog entry, add a new file to +the "changes" toplevel subdirectory. It should have the format of a +one-entry changelog section from the current ChangeLog file, as in + + o Major bugfixes: + - Fix a potential buffer overflow. Fixes bug 9999; bugfix on + 0.3.1.4-beta. + +To write a changes file, first categorize the change. Some common categories +are: Minor bugfixes, Major bugfixes, Minor features, Major features, Code +simplifications and refactoring. Then say what the change does. If +it's a bugfix, mention what bug it fixes and when the bug was +introduced. To find out which Git tag the change was introduced in, +you can use "git describe --contains <sha1 of commit>". + +If at all possible, try to create this file in the same commit where +you are making the change. Please give it a distinctive name that no +other branch will use for the lifetime of your change. + +When we go to make a release, we will concatenate all the entries +in changes to make a draft changelog, and clear the directory. We'll +then edit the draft changelog into a nice readable format. + +What needs a changes file?:: + A not-exhaustive list: Anything that might change user-visible + behavior. Anything that changes internals, documentation, or the build + system enough that somebody could notice. Big or interesting code + rewrites. Anything about which somebody might plausibly wonder "when + did that happen, and/or why did we do that" 6 months down the line. + +Why use changes files instead of Git commit messages?:: + Git commit messages are written for developers, not users, and they + are nigh-impossible to revise after the fact. + +Why use changes files instead of entries in the ChangeLog?:: + Having every single commit touch the ChangeLog file tended to create + zillions of merge conflicts. + +Useful tools +------------ + +These aren't strictly necessary for hacking on Tor, but they can help track +down bugs. + +The buildbot +~~~~~~~~~~~~ + +https://buildbot.vidalia-project.net/one_line_per_build + +Dmalloc +~~~~~~~ + +The dmalloc library will keep track of memory allocation, so you can find out +if we're leaking memory, doing any double-frees, or so on. + + dmalloc -l ~/dmalloc.log + (run the commands it tells you) + ./configure --with-dmalloc + +Valgrind +~~~~~~~~ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor -0.2. Running gcov for unit test coverage +(Note that if you get a zillion openssl warnings, you will also need to +pass --undef-value-errors=no to valgrind, or rebuild your openssl +with -DPURIFY.) + +Running gcov for unit test coverage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +----- make clean make CFLAGS='-g -fprofile-arcs -ftest-coverage' - ./src/or/test + ./src/test/test cd src/common; gcov *.[ch] cd ../or; gcov *.[ch] +----- + +Then, look at the .gcov files. '-' before a line means that the +compiler generated no code for that line. '######' means that the +line was never reached. Lines with numbers were called that number +of times. + +Profiling Tor with oprofile +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The oprofile tool runs (on Linux only!) to tell you what functions Tor is +spending its CPU time in, so we can identify berformance pottlenecks. + +Here are some basic instructions + + - Build tor with debugging symbols (you probably already have, unless + you messed with CFLAGS during the build process). + - Build all the libraries you care about with debugging symbols + (probably you only care about libssl, maybe zlib and Libevent). + - Copy this tor to a new directory + - Copy all the libraries it uses to that dir too (ldd ./tor will + tell you) + - Set LD_LIBRARY_PATH to include that dir. ldd ./tor should now + show you it's using the libs in that dir + - Run that tor + - Reset oprofiles counters/start it + * "opcontrol --reset; opcontrol --start", if Nick remembers right. + - After a while, have it dump the stats on tor and all the libs + in that dir you created. + * "opcontrol --dump;" + * "opreport -l that_dir/*" + - Profit + + +Coding conventions +------------------ + +Patch checklist +~~~~~~~~~~~~~~~ + +If possible, send your patch as one of these (in descending order of +preference) + + - A git branch we can pull from + - Patches generated by git format-patch + - A unified diff - Then, look at the .gcov files. '-' before a line means that the - compiler generated no code for that line. '######' means that the - line was never reached. Lines with numbers were called that number - of times. +Did you remember... -1. Coding conventions + - To build your code while configured with --enable-gcc-warnings? + - To run "make check-spaces" on your code? + - To write unit tests, as possible? + - To base your code on the appropriate branch? + - To include a file in the "changes" directory as appropriate? -1.0. Whitespace and C conformance +Whitespace and C conformance +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Invoke "make check-spaces" from time to time, so it can tell you about +deviations from our C whitespace style. Generally, we use: - Invoke "make check-spaces" from time to time, so it can tell you about - deviations from our C whitespace style. Generally, we use: - Unix-style line endings - K&R-style indentation - No space before newlines @@ -52,15 +200,17 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor "puts (x)". - Function declarations at the start of the line. - We try hard to build without warnings everywhere. In particular, if you're - using gcc, you should invoke the configure script with the option - "--enable-gcc-warnings". This will give a bunch of extra warning flags to - the compiler, and help us find divergences from our preferred C style. +We try hard to build without warnings everywhere. In particular, if you're +using gcc, you should invoke the configure script with the option +"--enable-gcc-warnings". This will give a bunch of extra warning flags to +the compiler, and help us find divergences from our preferred C style. + +Getting emacs to edit Tor source properly +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -1.0.1. Getting emacs to edit Tor source properly. +Nick likes to put the following snippet in his .emacs file: - Hi, folks! Nick here. I like to put the following snippet in my .emacs - file: +----- (add-hook 'c-mode-hook (lambda () (font-lock-mode 1) @@ -80,90 +230,99 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor (set-variable 'c-basic-offset 8) (set-variable 'tab-width 8)) )))) +----- - You'll note that it defaults to showing all trailing whitespace. The - "cond" test detects whether the file is one of a few C free software - projects that I often edit, and sets up the indentation level and tab - preferences to match what they want. +You'll note that it defaults to showing all trailing whitespace. The "cond" +test detects whether the file is one of a few C free software projects that I +often edit, and sets up the indentation level and tab preferences to match +what they want. - If you want to try this out, you'll need to change the filename regex - patterns to match where you keep your Tor files. +If you want to try this out, you'll need to change the filename regex +patterns to match where you keep your Tor files. - If you *only* use emacs to edit Tor, you could always just say: +If you use emacs for editing Tor and nothing else, you could always just say: - (add-hook 'c-mode-hook +----- + (add-hook 'c-mode-hook (lambda () (font-lock-mode 1) (set-variable 'show-trailing-whitespace t) (set-variable 'indent-tabs-mode nil) (set-variable 'c-basic-offset 2))) +----- - There is probably a better way to do this. No, we are probably not going - to clutter the files with emacs stuff. +There is probably a better way to do this. No, we are probably not going +to clutter the files with emacs stuff. -1.1. Details - Use tor_malloc, tor_free, tor_strdup, and tor_gettimeofday instead of their - generic equivalents. (They always succeed or exit.) +Functions to use +~~~~~~~~~~~~~~~~ - You can get a full list of the compatibility functions that Tor provides by - looking through src/common/util.h and src/common/compat.h. You can see the - available containers in src/common/containers.h. You should probably - familiarize yourself with these modules before you write too much code, - or else you'll wind up reinventing the wheel. +We have some wrapper functions like tor_malloc, tor_free, tor_strdup, and +tor_gettimeofday; use them instead of their generic equivalents. (They +always succeed or exit.) - Use 'INLINE' instead of 'inline', so that we work properly on Windows. +You can get a full list of the compatibility functions that Tor provides by +looking through src/common/util.h and src/common/compat.h. You can see the +available containers in src/common/containers.h. You should probably +familiarize yourself with these modules before you write too much code, or +else you'll wind up reinventing the wheel. -1.2. Calling and naming conventions +Use 'INLINE' instead of 'inline', so that we work properly on Windows. - Whenever possible, functions should return -1 on error and 0 on success. +Calling and naming conventions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - For multi-word identifiers, use lowercase words combined with - underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and - constants. +Whenever possible, functions should return -1 on error and 0 on success. - Typenames should end with "_t". +For multi-word identifiers, use lowercase words combined with +underscores. (e.g., "multi_word_identifier"). Use ALL_CAPS for macros and +constants. - Function names should be prefixed with a module name or object name. (In - general, code to manipulate an object should be a module with the same - name as the object, so it's hard to tell which convention is used.) +Typenames should end with "_t". - Functions that do things should have imperative-verb names - (e.g. buffer_clear, buffer_resize); functions that return booleans should - have predicate names (e.g. buffer_is_empty, buffer_needs_resizing). +Function names should be prefixed with a module name or object name. (In +general, code to manipulate an object should be a module with the same name +as the object, so it's hard to tell which convention is used.) - If you find that you have four or more possible return code values, it's - probably time to create an enum. If you find that you are passing three or - more flags to a function, it's probably time to create a flags argument - that takes a bitfield. +Functions that do things should have imperative-verb names +(e.g. buffer_clear, buffer_resize); functions that return booleans should +have predicate names (e.g. buffer_is_empty, buffer_needs_resizing). -1.3. What To Optimize +If you find that you have four or more possible return code values, it's +probably time to create an enum. If you find that you are passing three or +more flags to a function, it's probably time to create a flags argument that +takes a bitfield. - Don't optimize anything if it's not in the critical path. Right now, - the critical path seems to be AES, logging, and the network itself. - Feel free to do your own profiling to determine otherwise. +What To Optimize +~~~~~~~~~~~~~~~~ -1.4. Log conventions +Don't optimize anything if it's not in the critical path. Right now, the +critical path seems to be AES, logging, and the network itself. Feel free to +do your own profiling to determine otherwise. - http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#LogLevels +Log conventions +~~~~~~~~~~~~~~~ - No error or warning messages should be expected during normal OR or OP - operation. +https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#LogLevels - If a library function is currently called such that failure always - means ERR, then the library function should log WARN and let the caller - log ERR. +No error or warning messages should be expected during normal OR or OP +operation. - [XXX Proposed convention: every message of severity INFO or higher should - either (A) be intelligible to end-users who don't know the Tor source; or - (B) somehow inform the end-users that they aren't expected to understand - the message (perhaps with a string like "internal error"). Option (A) is - to be preferred to option (B). -NM] +If a library function is currently called such that failure always means ERR, +then the library function should log WARN and let the caller log ERR. -1.5. Doxygen +Every message of severity INFO or higher should either (A) be intelligible +to end-users who don't know the Tor source; or (B) somehow inform the +end-users that they aren't expected to understand the message (perhaps +with a string like "internal error"). Option (A) is to be preferred to +option (B). - We use the 'doxygen' utility to generate documentation from our - source code. Here's how to use it: +Doxygen +~~~~~~~~ + +We use the 'doxygen' utility to generate documentation from our +source code. Here's how to use it: 1. Begin every file that should be documented with /** @@ -214,11 +373,12 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor 6. See the Doxygen manual for more information; this summary just scratches the surface. -1.5.1. Doxygen comment conventions +Doxygen comment conventions +^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Say what functions do as a series of one or more imperative sentences, as - though you were telling somebody how to be the function. In other words, - DO NOT say: +Say what functions do as a series of one or more imperative sentences, as +though you were telling somebody how to be the function. In other words, DO +NOT say: /** The strtol function parses a number. * @@ -230,7 +390,7 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor */ long strtol(const char *nptr, char **nptr, int base); - Instead, please DO say: +Instead, please DO say: /** Parse a number in radix <b>base</b> from the string <b>nptr</b>, * and return the result. Skip all leading whitespace. If @@ -239,66 +399,102 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor **/ long strtol(const char *nptr, char **nptr, int base); - Doxygen comments are the contract in our abstraction-by-contract world: if - the functions that call your function rely on it doing something, then your - function should mention that it does that something in the documentation. - If you rely on a function doing something beyond what is in its - documentation, then you should watch out, or it might do something else - later. - -2. Code notes - -2.1. Dataflows - -2.1.1. How Incoming data is handled - -There are two paths for data arriving at Tor over the network: regular -TCP data, and DNS. - -2.1.1.1. TCP. - -When Tor takes information over the network, it uses the functions -read_to_buf() and read_to_buf_tls() in buffers.c. These read from a -socket or an SSL* into a buffer_t, which is an mbuf-style linkedlist -of memory chunks. - -read_to_buf() and read_to_buf_tls() are called only from -connection_read_to_buf() in connection.c. It takes a connection_t -pointer, and reads data into it over the network, up to the -connection's current bandwidth limits. It places that data into the -"inbuf" field of the connection, and then: - - Adjusts the connection's want-to-read/want-to-write status as - appropriate. - - Increments the read and written counts for the connection as - appropriate. - - Adjusts bandwidth buckets as appropriate. - -connection_read_to_buf() is called only from connection_handle_read(). -The connection_handle_read() function is called whenever libevent -decides (based on select, poll, epoll, kqueue, etc) that there is data -to read from a connection. If any data is read, -connection_handle_read() calls connection_process_inbuf() to see if -any of the data can be processed. If the connection was closed, -connection_handle_read() calls connection_reached_eof(). - -Connection_process_inbuf() and connection_reached_eof() both dispatch -based on the connection type to determine what to do with the data -that's just arrived on the connection's inbuf field. Each type of -connection has its own version of these functions. For example, -directory connections process incoming data in -connection_dir_process_inbuf(), while OR connections process incoming -data in connection_or_process_inbuf(). These -connection_*_process_inbuf() functions extract data from the -connection's inbuf field (a buffer_t), using functions from buffers.c. -Some of these accessor functions are straightforward data extractors -(like fetch_from_buf()); others do protocol-specific parsing. - - -2.1.1.2. DNS - -Tor launches (and optionally accepts) DNS requests using the code in -eventdns.c, which is a copy of libevent's evdns.c. (We don't use -libevent's version because it is not yet in the versions of libevent -all our users have.) DNS replies are read in nameserver_read(); -DNS queries are read in server_port_read(). +Doxygen comments are the contract in our abstraction-by-contract world: if +the functions that call your function rely on it doing something, then your +function should mention that it does that something in the documentation. If +you rely on a function doing something beyond what is in its documentation, +then you should watch out, or it might do something else later. + +Putting out a new release +------------------------- + +Here are the steps Roger takes when putting out a new Tor release: + +1) Use it for a while, as a client, as a relay, as a hidden service, +and as a directory authority. See if it has any obvious bugs, and +resolve those. + +1.5) As applicable, merge the maint-X branch into the release-X branch. + +2) Gather the changes/* files into a changelog entry, rewriting many +of them and reordering to focus on what users and funders would find +interesting and understandable. + + 2.1) Make sure that everything that wants a bug number has one. + 2.2) Concatenate them. + 2.3) Sort them by section. Within each section, try to make the + first entry or two and the last entry most interesting: they're + the ones that skimmers tend to read. + + 2.4) Clean them up + + Standard idioms: + "Fixes bug 9999; Bugfix on 0.3.3.3-alpha." + + One period after a space. + + Make stuff very terse + + Make sure each section name ends with a colon + + Describe the user-visible problem right away + + Mention relevant config options by name. If they're rare or unusual, + remind people what they're for + + Avoid starting lines with open-paren + + Present and imperative tense: not past. + + If a given changes stanza showed up in a different release (e.g. + maint-0.2.1), be sure to make the stanzas identical (so people can + distinguish if these are the same change). + + 2.5) Merge them in. + + 2.6) Clean everything one last time. + + 2.7) Run it through fmt to make it pretty. + + +3) Compose a short release blurb to highlight the user-facing +changes. Insert said release blurb into the ChangeLog stanza. If it's +a stable release, add it to the ReleaseNotes file too. If we're adding +to a release-0.2.x branch, manually commit the changelogs to the later +git branches too. + +4) Bump the version number in configure.in and rebuild. + +5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait +a while to see if anybody has problems building it. Try to get Sebastian +or somebody to try building it on Windows. + +6) Get at least two of weasel/arma/karsten to put the new version number +in their approved versions list. + +7) Sign and push the tarball to the website in the dist/ directory. Sign +and push the git tag. + (That's either "git tag -u <keyid> tor-0.2.x.y-status", then + "git push origin tag tor-0.2.x.y-status". To sign the + tarball, "gpg -ba <the_tarball>". Put the files in + /srv/www-master.torproject.org/htdocs/dist/ on vescum.) + +8) Edit include/versions.wmi to note the new version. From your website +checkout, run ./publish to build and publish the website. + +9) Email Erinn and weasel (cc'ing tor-assistants) that a new tarball +is up. This step should probably change to mailing more packagers. + +10) Add the version number to Trac. To do this, go to Trac, log in, +select "Admin" near the top of the screen, then select "Versions" from +the menu on the left. At the right, there will be an "Add version" +box. By convention, we enter the version in the form "Tor: +0.2.2.23-alpha" (or whatever the version is), and we select the date as +the date in the ChangeLog. + +11) Wait up to a day or two (for a development release), or until most +packages are up (for a stable release), and mail the release blurb and +changelog to tor-talk or tor-announce. + (We might be moving to faster announcements, but don't announce until + the website is at least updated.) diff --git a/doc/Makefile.am b/doc/Makefile.am index 295981375..bc3d8df47 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -1,12 +1,78 @@ +# We use a two-step process to generate documentation from asciidoc files. +# +# First, we use asciidoc/a2x to process the asciidoc files into .1.in and +# .html.in files (see the asciidoc-helper.sh script). These are the same as +# the regular .1 and .html files, except that they still have some autoconf +# variables set in them. +# +# Second, we use config.status to turn .1.in files into .1 files and +# .html.in files into .html files. +# +# We do the steps in this order so that we can ship the .*.in files as +# part of the source distribution, so that people without asciidoc can +# just use the .1 and .html files. -EXTRA_DIST = HACKING \ - tor-resolve.1 tor-gencert.1 \ - tor-osx-dmg-creation.txt tor-rpm-creation.txt \ +if USE_ASCIIDOC +asciidoc_files = tor tor-gencert tor-resolve torify +html_in = $(asciidoc_files:=.html.in) +man_in = $(asciidoc_files:=.1.in) +txt_in = $(asciidoc_files:=.1.txt) +nodist_man_MANS = $(asciidoc_files:=.1) +doc_DATA = $(asciidoc_files:=.html) +else +asciidoc_files = +html_in = +man_in = +txt_in = +nodist_man_MANS = +doc_DATA = +endif + +EXTRA_DIST = HACKING asciidoc-helper.sh \ + $(html_in) $(man_in) $(txt_in) \ + tor-rpm-creation.txt \ tor-win32-mingw-creation.txt spec/README -man_MANS = tor.1 tor-resolve.1 tor-gencert.1 +docdir = @docdir@ + +asciidoc_product = $(nodist_man_MANS) $(doc_DATA) + +# Generate the html documentation from asciidoc, but don't do +# machine-specific replacements yet +$(html_in) : + $(top_srcdir)/doc/asciidoc-helper.sh html @ASCIIDOC@ $(top_srcdir)/doc/$@ + +tor.html.in : tor.1.txt +torify.html.in : torify.1.txt +tor-gencert.html.in : tor-gencert.1.txt +tor-resolve.html.in : tor-resolve.1.txt + +# Generate the manpage from asciidoc, but don't do +# machine-specific replacements yet +$(man_in) : + $(top_srcdir)/doc/asciidoc-helper.sh man @A2X@ $(top_srcdir)/doc/$@ + +tor.1.in : tor.1.txt +torify.1.in : torify.1.txt +tor-gencert.1.in : tor-gencert.1.txt +tor-resolve.1.in : tor-resolve.1.txt -SUBDIRS = design-paper +# use ../config.status to swap all machine-specific magic strings +# in the asciidoc with their replacements. +$(asciidoc_product) : + if test -e $(top_srcdir)/doc/$@.in && ! test -e ./$@.in ; then \ + cp $(top_srcdir)/doc/$@.in .; \ + fi + ../config.status --file=$@; -DIST_SUBDIRS = design-paper +tor.1 : tor.1.in +torify.1 : torify.1.in +tor-gencert.1 : tor-gencert.1.in +tor-resolve.1 : tor-resolve.1.in +tor.html : tor.html.in +torify.html : torify.html.in +tor-gencert.html : tor-gencert.html.in +tor-resolve.html : tor-resolve.html.in +CLEANFILES = $(asciidoc_product) config.log +DISTCLEANFILES = $(html_in) $(man_in) @@ -4,8 +4,8 @@ We've split out our TODO into three files: TODO.02x is the list of items we're planning to get done in the next stable release. -TODO.external is the list of external constraints and deliverables that -we all need to keep in mind. +TODO.external lives in svn under /projects/todo/. It's the list of +external constraints and deliverables that we all need to keep in mind. TODO.future is the list of other items we plan to get to in later releases. diff --git a/doc/TODO.021 b/doc/TODO.021 index 881ba5ee4..37c5b9845 100644 --- a/doc/TODO.021 +++ b/doc/TODO.021 @@ -1,4 +1,3 @@ -$Id$ Legend: SPEC!! - Not specified SPEC - Spec not finalized diff --git a/doc/TODO.022 b/doc/TODO.022 index 3eeae006c..d83ed6e67 100644 --- a/doc/TODO.022 +++ b/doc/TODO.022 @@ -8,44 +8,85 @@ NOTE 2: It's easy to list stuff like this with no time estimates and 0.2.2, figure out how long the stuff we want will take, and triage accordingly, or vice versa. -- Design - - Begin design work for UDP transition; identify areas where we need to - make changes or instrument stuff early. - - Performance, mostly protocol-neutral. - - Work with Libevent 2.0's bufferevent interface - - Identify any performance stuff we need to push back into - libevent to make it as fast as we want. - - Revise how we do bandwidth limiting and round-robining between + o Revise how we do bandwidth limiting and round-robining between circuits on a connection. - - Revise how we do bandwidth limiting and round-robining between + . Revise how we do bandwidth limiting and round-robining between connections. - Better flow-control to avoid filling buffers on routers. - - Split AES across cores if possible. - - Split SSL across cores (reach; may require Libevent 2.1). - - Figure out good ways to instrument Tor internals so we can tell how well our bandwidth and flow-control stuff is actually working. + - What ports eat the bandwidth? + - How full do queues get? + - How much latency do queues get? + + - Rate limit at clients: + - Give clients an upper bound on how much they're willing to use + the network if they're not relaying? + - ... or group client circuits by IP at the server and rate-limit + like that. + + - Use if-modified-since to download consensuses -- Features + +- Other features - Proposals to implement: - - 146: reflect long-term stability + - 146: reflect long-term stability in consensuses - 147: Stop using v2 directories to generate v3 votes. + - Start pinging as soon as we learn about a relay, not on a + 22-minute cycle. Prioritize new and volatile relays for + testing. - Proposals to improve and implement - 158: microdescriptors + o Revise proposal + - Implement - Proposals to improve and implement if not broken - - IPv6 support. (Parts of 117, but figure out how to handle DNS + D IPv6 support. (Parts of 117, but figure out how to handle DNS requests.) - 140: Directory diffs + - Need a decent simple C diff implementation. + - Need a decent simple C ed patch implementation. - 149: learn info from netinfo cells. - - 134: handle authority fragmentation (Needs more analysis) + o Start discussion + - Revise proposal based on discussion. + X 134: handle authority fragmentation (Needs more analysis) + - 165: Easy migration for voting authority sets + - 163: Detect client-status better + o Write proposal + - Possibly implement, depending on discussion. + - 164: Have authorities report relay and voting status better: make it + easy to answer, "Why is my server not listed/not Guard/not + Running/etc" + o Write proposal + - Possibly implement, depending on discussion + - 162: Have consensuses come in multiple "flavours". + o Write proposal + - Possibly implement, depending on discussion. + + - Needs a proposal, or at least some design + - Weaken the requirements for being a Guard, based on K's + measurements. +K - Finish measurements +K? - Write proposal + - Adaptive timeouts for giving up on circuits and streams. +M - Revise proposal 151 + - Downweight guards more sensibly: be more forgiving about using + Guard nodes as non-first-hop. + - Write proposal. + - Lagged weight updates in consensuses: don't just move abruptly. +M? - Write proposal + d Don't kill a circuit on the first failed extend. + +- Installers + - Switch to MSI on win32 + - Use Thandy, perhaps? -- Deprecations - - Make .exit safe, or make it off-by-default. +o Deprecations + o Make .exit safe, or make it off-by-default. diff --git a/doc/TODO.external b/doc/TODO.external index c02d6aca5..2e7e536ef 100644 --- a/doc/TODO.external +++ b/doc/TODO.external @@ -1,196 +1,4 @@ -$Id$ -Legend: -SPEC!! - Not specified -SPEC - Spec not finalized -N - nick claims -R - arma claims -P - phobos claims -S - Steven claims -E - Matt claims -M - Mike claims -J - Jeff claims -I - ioerror claims -W - weasel claims -K - Karsten claims -C - coderman claims - - Not done - * Top priority - . Partially done - o Done - d Deferrable - D Deferred - X Abandoned -======================================================================= - -External constraints: - -For June/July: -NR - Work more on Paul's NRL research problem. - -For March 22: -I * Email auto-responder - * teach gettor how to ask for (and attach) split files. - -K . Metrics. - . With Mike's help, use Torflow to start doing monthly rudimentary - performance evaluations: - . Circuit throughput and latency - - Measure via Broadband and dialup - . Publish a report addressing key long-term metrics questions: - . What metrics should we present? - . What data are available for these metrics? - . What data are missing, and can collect them safely? Can we - publish them safely? - . What systems are available to present this data? - -E . Vidalia improvements - o Vidalia displays by-country user summary for bridge operators -? - write a help page for vidalia, "what is this" - -For mid August: - -Section 0, items that didn't make it into the original roadmap: - -0.1, installers and packaging -C . i18n for the msi bundle files -P . more consistent TBB builds -IC- get a buildbot up again. Have Linux and BSD build machines. - (Windows would be nice but realistically will come later.) -E - Get Tor to work properly on the iPhone. - -3.1, performance work. [Section numbers in here are from performance.pdf] - - High-priority items from performance.pdf -RS - 1.2, new circuit window sizes. make the default package window lower. -R+ - 2.1, squeeze loud circuits - - Evaluate the code to see what stats we can keep about circuit use. - - Write proposals for various meddling. Look at the research papers - that Juliusz pointed us to. Ask our systems friends. Plan to put - a lot of the parameters in the consensus, so we can tune it with - short turnaround times. -E+ - 2.5, Change Vidalia's default exit policy to not click "other - protocols". Or choose not to. Think this through first. -R+ - 2.6, Tell users not to file-share. - - Put statement on the Tor front page - - Put statement on the download pages too - - And the FAQ - - 3.1.2, Tor weather -I - Implement time-to-notification (immediate, a day, a week) -I - Get a relay operator mailing list going, with a plan and supporting - scripts and so on. -R - Link to them from the Tor relay page -R - and the torrc.sample? -SM - 4.1, balance traffic better - - Steven and Mike should decide if we should do Steven's plan - (rejigger the bandwidth numbers at the authorities based on - Steven's algorithm), or Mike's plan (relay scanning to identify - the unbalanced relays and fix them on the fly), or both. - - Figure out how to actually modify bandwidths in the consensus. We - may need to change the consensus voting algorithm to decide what - bandwidth to advertise based on something other than median: - if 7 authorities provide bandwidths, and 2 are doing scanning, - then the 5 that aren't scanning will outvote any changes. Should - all 7 scan? Should only some vote? Extra points if it doesn't - change all the numbers every new consensus, so consensus diffing - is still practical. -? - 4.5, Older entry guards are overloaded - - Pick a conservative timeout like a month, and implement. -M - 5.2, better timeouts for giving up on circuits/streams - - clients gather data about circuit timeouts, and then abandon - circuits that take more than a std dev above that. - -4.1, IOCP / libevent / windows / tor -N - get it working for nick -N - put out a release so other people can start testing it. -N - both the libevent buffer abstraction, and the - tor-uses-libevent-buffer-abstraction. Unless we think that's - unreachable for this milestone? - -4.2.1, risks from becoming a relay -S - Have a clear plan for how users who become relays will be safe, - and be confident that we can build this plan. - - evaluate all the various attacks that are made possible by relaying. - specifically, see "relaying-traffic attacks" in 6.6. - - identify and evaluate ways to make them not a big deal - - setting a low RelayBandwidth - - Nick Hopper's FC08 paper suggesting that we should do a modified - round-robin so we leak less about other circuits - - instructing clients to disable pings in their firewall, etc - - pick the promising ones, improve them so they're even better, and - spec them out so we know how to build them and how much effort is - involved in building them. - -4.5, clients download less directory info -N * deploy proposal 158. -N - decide whether to do proposal 140. if so, construct an implementation - plan for how we'll do it. if not, explain why not. - -5.1, Normalize TLS fingerprint -N o write a draft list of possible attacks for this section, with - estimates about difficulty of attack, difficulty of solution, etc -N - revisit the list and revise our plans as needed -NR- put up a blog post about the two contradictory conclusions: we can - discuss the theory of arms races, and our quandry, without revealing - any specific vulnerabilities. (or decide not to put up a blog post, - and explain why not.) - -5.5, email autoresponder -I . maintenance and keeping it running - -5.7.2, metrics - -XXX. - -6.2, Vidalia work -E - add breakpad support or similar for windows debugging -E o let vidalia change languages without needing a restart -E - Implement the status warning event interface started for the - phase one deliverables. -E - Work with Steve Tyree on building a Vidalia plugin API to enable - building Herdict and TBB plugins. - -6.3, Node scanning -M - Steps toward automation - - Set up email list for results - - Map failure types to potential BadExit lines -M - Improve the ability of SoaT to mimic various real web browsers - - randomizing user agents and locale strings - - caching, XMLHTTPRequest, form posting, content sniffing - - Investigate ideas like running Chrome/xulrunner in parallel -M - Other protocols - - SSH, IMAPS, POPS, SMTPS -M - Add ability to geolocalize exit selection based on scanner location - - Use this to rescan dynamic urls filtered by the URL filter - -6.4, Torbutton development -M - Resolve extension conflicts and other high priority bugs -M - Fix or hack around ugly firefox bugs, especially Timezone issue. - Definitely leaning towards "hack around" unless we see some - level of love from Mozilla. -M - Vidalia New Nym Integration - - Implement for Torbutton to pick up on Vidalia's NEWNYM and clear - cookies based on FoeBud's source - - Do this in such a way that we could adapt polipo to purge cache - if we were so inclined -M - Write up a summary of our options for dealing with the google - you-must-solve-a-captcha-to-search problem, and pick one as our - favorite option. - -6.6, Evaluate new anonymity attacks -S - relaying-traffic attacks - - original murdoch-danezis attack - - nick hopper's latency measurement attack - - columbia bandwidth measurement attack - - christian grothoff's long-circuit attack -S - client attacks - - website fingerprinting - -7.1, Tor VM Research, analysis, and prototyping -C . Get a working package out, meaning other people are testing it. - -7.2, Tor Browser Bundle -I - Port to one of OS X or Linux, and start the port to the other. -I . Make it the recommended Tor download on Windows -I - Make sure it's easy to un-brand TBB in case Firefox asks us to -I - Evaluate CCC's Freedom Stick +[This file moved to svn in /projects/todo/. More people can edit +it more easily there. -RD] diff --git a/doc/TODO.future b/doc/TODO.future index 64169ecfe..85e07aa92 100644 --- a/doc/TODO.future +++ b/doc/TODO.future @@ -1,4 +1,3 @@ -$Id$ Legend: SPEC!! - Not specified SPEC - Spec not finalized @@ -23,7 +22,6 @@ K - Karsten claims ======================================================================= Later, unless people want to implement them now: - - tor as a socks proxy should accept (and ignore) password auth - Actually use SSL_shutdown to close our TLS connections. - Include "v" line in networkstatus getinfo values. [Nick: bridge authorities output a networkstatus that is missing @@ -31,10 +29,6 @@ Later, unless people want to implement them now: bridgedb gives out bridges with certain characteristics. -RD] [Okay. Is this a separate item, or is it the same issue as the lack of a "v" line in response to the controller GETINFO command? -NM] - - Let tor dir mirrors proxy connections to the tor download site, so - if you know a bridge you can fetch the tor software. - - when somebody uses the controlport as an http proxy, give them - a "tor isn't an http proxy" error too like we do for the socks port. - MAYBE kill stalled circuits rather than stalled connections. This is possible thanks to cell queues, but we need to consider the anonymity implications. @@ -46,8 +40,6 @@ Later, unless people want to implement them now: online config documentation from a single source. - It would be potentially helpful to respond to https requests on the OR port by acting like an HTTPS server. - - Make the timestamp granularity on logs configurable, with default - of "1 second". This might make some kinds of after-the-fact attack harder. - We should get smarter about handling address resolve failures, or addresses that resolve to local IPs. It would be neat to retry @@ -273,7 +265,7 @@ Future versions: the RPM and other startup scripts should too? - add a "default.action" file to the tor/vidalia bundle so we can fix the https thing in the default configuration: - http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#PrivoxyWeirdSSLPort + https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#PrivoxyWeirdSSLPort ======================================================================= diff --git a/doc/asciidoc-helper.sh b/doc/asciidoc-helper.sh new file mode 100755 index 000000000..00f8b8d07 --- /dev/null +++ b/doc/asciidoc-helper.sh @@ -0,0 +1,65 @@ +#!/bin/sh + +# Copyright (c) The Tor Project, Inc. +# See LICENSE for licensing information +# Run this to generate .html.in or .1.in files from asciidoc files. +# Arguments: +# html|man asciidocpath outputfile + +set -e + +if [ $# != 3 ]; then + exit 1; +fi + +output=$3 + +if [ "$1" = "html" ]; then + input=${output%%.html.in}.1.txt + base=${output%%.html.in} + if [ "$2" != none ]; then + "$2" -d manpage -o $output $input; + else + echo "=================================="; + echo; + echo "You need asciidoc installed to be able to build the manpage."; + echo "To build without manpages, use the --disable-asciidoc argument"; + echo "when calling configure."; + echo; + echo "=================================="; + exit 1; + fi +elif [ "$1" = "man" ]; then + input=${output%%.1.in}.1.txt + base=${output%%.1.in} + + if test "$2" = none; then + echo "=================================="; + echo; + echo "You need asciidoc installed to be able to build the manpage."; + echo "To build without manpages, use the --disable-asciidoc argument"; + echo "when calling configure."; + echo; + echo "=================================="; + exit 1; + fi + if "$2" -f manpage $input; then + mv $base.1 $output; + else + cat<<EOF +================================== +You need a working asciidoc installed to be able to build the manpage. + +a2x is installed, but for some reason it isn't working. Sometimes +This happens because required docbook support files are missing. +Please install docbook-xsl, docbook-xml, and libxml2-utils (Debian) or +similar. + +Alternatively, to build without manpages, use the --disable-asciidoc +argument when calling configure. +================================== +EOF + exit 1; + fi +fi + diff --git a/doc/codecon04.mgp b/doc/codecon04.mgp deleted file mode 100644 index e9815fcb3..000000000 --- a/doc/codecon04.mgp +++ /dev/null @@ -1,357 +0,0 @@ -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%deffont "standard" xfont "comic sans ms-medium-r" -%%deffont "thick" xfont "arial black-medium-r" -%%deffont "typewriter" xfont "courier new-bold-r" -%%deffont "type2writer" xfont "arial narrow-bold-r" -%%deffont "standard" tfont "standard.ttf", tmfont "kochi-mincho.ttf" -%%deffont "thick" tfont "thick.ttf", tmfont "goth.ttf" -%%deffont "typewriter" tfont "typewriter.ttf", tmfont "goth.ttf" -%deffont "standard" xfont "helvetica-medium-r", tfont "arial.ttf", tmfont "times.ttf" -%deffont "thick" xfont "helvetica-bold-r", tfont "arialbd.ttf", tmfont "hoso6.ttf" -%deffont "italic" xfont "helvetica-italic-r", tfont "ariali.ttf", tmfont "hoso6.ttf" -%deffont "typewriter" xfont "courier-medium-r", tfont "typewriter.ttf", tmfont "hoso6.ttf" -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% -%% Default settings per each line numbers. -%% -%default 1 leftfill, size 8, fore "black", back "white", font "thick", hgap 1 -%default 2 size 8, vgap 10, prefix " ", ccolor "black" -%default 3 size 6, bar "gray70", vgap 0 -%default 4 size 6, fore "black", vgap 0, prefix " ", font "standard" -%% -%%default 1 area 90 90, leftfill, size 9, fore "yellow", back "blue", font "thick" -%%default 2 size 9, vgap 10, prefix " " -%%default 3 size 7, bar "gray70", vgap 10 -%%default 4 size 7, vgap 30, prefix " ", font "standard" -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% -%% Default settings that are applied to TAB-indented lines. -%% -%tab 1 size 5, vgap 40, prefix " ", icon arc "red" 50 -%tab 2 size 4, vgap 35, prefix " ", icon delta3 "blue" 40 -%tab 3 size 3, vgap 35, prefix " ", icon dia "DarkViolet" 40 -%% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page -%nodefault -%center, size 9, font "thick", back "white", fore "black" - -Tor: -%size 8 -Next-generation Onion Routing - - -%size 7 -Roger Dingledine -Nick Mathewson -Paul Syverson - -The Free Haven Project -%font "typewriter", fore "blue" -http://freehaven.net/ - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Low-latency anonymity system - -%leftfill -Deployed: 20 nodes, hundreds (?) of users - -Many improvements on earlier design - -Free software -- modified BSD license - -Design is not covered by earlier onion routing -patent - -Uses SOCKS to interface with client apps - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -We have working code - -(14 kloc of C) - -and a design document, -and a byte-level specification, -and a Debian package (in Unstable) - -Works on Linux, BSD, OSX, Cygwin, ... -User-space, doesn't need kernel mods or root - -%size 9 -http://freehaven.net/tor/ - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%page -%% -%%Talk Overview -%% -%%A bit about Onion Routing -%% -%%Improvements we've made -%% -%%Some related work -%% -%%Ask me questions -%% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Anonymity: Who needs it? - -Private citizens - advocacy, counseling, whistleblowing, reporting, ... -%size 6 -Higher-level protocols - voting, e-cash, auctions -%size 6 -Government applications - research, law enforcement -%size 6 -Business applications -%size 5 -(hide relationships and volumes of communication) - Who is visiting job sites? - Which groups are talking to patent lawyers? - Who are your suppliers and customers? - Is the CEO talking to a buyout partner? - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Anonymity is a network effect - - Systems need traffic (many low-sensitivity users) to attract the high-sensitivity users - Most users do not value anonymity much - Weak security (fast system) can mean more users - which can mean -%cont, font "italic" -stronger -%cont, font "standard" -anonymity - High-sensitivity agents have incentive to run nodes - so they can be certain first node in their path is good - to attract traffic for their messages - There can be an optimal level of free-riding - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Onion Routing is... - -An overlay network - -Users build virtual circuits through the network - -One layer of encryption at each hop - -Fixed-size cells - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Tor's goals - -Conservative design - minimize new design work needed - -%size 6 -Support testing of future research - -Design for deployment; deploy for use - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Threat model -- what we aim for - -Protect against somebody watching Alice - -Protect against curious Bob - -Protect against `some' curious nodes in the middle - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Differences / limitations - - -We're TCP-only, not all IP (but we're user-space and very portable) - -Not as strong as high-latency systems (Mixmaster, Mixminion) - -Not peer-to-peer - -No protocol normalization - -Not unobservable (no steg, etc) - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Perfect forward secrecy - - -Telescoping circuit - - negotiates keys at each hop - no more need for replay detection - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -No mixing, padding, traffic shaping (yet) - - -Please show us they're worth the usability tradeoff - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%page -%% -%%Many TCP streams can share one circuit -%% - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Many TCP streams share a circuit - -Previous designs built a new circuit for each stream - - lots of public key ops per request - plus anonymity dangers from making so many circuits - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Leaky-pipe circuit topology - -Alice can direct cells to any node in her circuit - - So we can support long-range padding, - have multiple streams exiting at different places in the circuit - etc - -%size 6 -Unclear whether this is dangerous or useful - -More research needed - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Congestion control - - -Simple rate limiting - -Plus have to keep internal nodes from overflowing - -(Can't use global state or inter-node control) - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Directory servers - -To solve the `introduction' problem - -Approve new servers - -Tell clients who's up right now - - plus their keys, location, etc - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Variable exit policies - - -Each server allows different outgoing connections - -E.g. no servers allow outgoing mail currently - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -End-to-end integrity checking - - -In previous onion routing, an insider could change -the text being transmitted: - -"dir" => "rm *" - -Even an external adversary could do this! - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Rendezvous points - -allow hidden services - -don't need (brittle) reply onions - - Access-controlled: Bob can control who he talks to - Robust: Bob's service is available even when some Tor nodes go down - Smear-resistant: Evil service can't frame a rendezvous router - Application-transparent: Don't need to modify Bob's apache - -%size 6 -(Not implemented yet) - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -How do we compare security? - -Assume adversary owns c of n nodes - can choose which -%size 6 -What's the chance for a random Alice and Bob that he wins? - -Freedom, Tor: (c/n)^2 -Peekabooty, six-four, etc: c/n -Jap (if no padding): 1 if c>1 -Anonymizer: 1 if c>0 - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Future work - -Threshold directory agreement - -Scalability: Morphmix/p2p extensions? -Restricted-route (non-clique topology) - -Non-TCP transport - -Implement rendezvous points - -Make it work better - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -We have working code - -Plus a design document, -and a byte-level specification -and a Debian package (in Unstable) - -%size 9 -http://freehaven.net/tor/ - -%size 6 -Privacy Enhancing Technologies workshop - -%size 9 -http://petworkshop.org/ - diff --git a/doc/design-paper/Makefile.am b/doc/design-paper/Makefile.am deleted file mode 100644 index fb94706d0..000000000 --- a/doc/design-paper/Makefile.am +++ /dev/null @@ -1,26 +0,0 @@ - -cell-struct.eps: cell-struct.fig - fig2dev -L eps $< $@ -interaction.eps: interaction.fig - fig2dev -L eps $< $@ -cell-struct.pdf: cell-struct.fig - fig2dev -L pdf $< $@ -interaction.pdf: interaction.fig - fig2dev -L pdf $< $@ - -tor-design.ps: cell-struct.eps interaction.eps tor-design.bib tor-design.tex usenix.sty latex8.bst - latex tor-design.tex - bibtex tor-design - latex tor-design.tex - latex tor-design.tex - dvips -o $@ tor-design.dvi - -tor-design.pdf: cell-struct.pdf interaction.pdf tor-design.bib tor-design.tex usenix.sty latex8.bst - pdflatex tor-design.tex - bibtex tor-design - pdflatex tor-design.tex - pdflatex tor-design.tex - -EXTRA_DIST = cell-struct.fig interaction.fig tor-design.bib usenix.sty latex8.bst tor-design.tex - -DISTCLEANFILES = cell-struct.eps interaction.eps cell-struct.pdf interaction.pdf tor-design.aux tor-design.bbl tor-design.blg tor-design.log tor-design.dvi tor-design.ps tor-design.pdf diff --git a/doc/design-paper/blocking.html b/doc/design-paper/blocking.html deleted file mode 100644 index 6028f5dc1..000000000 --- a/doc/design-paper/blocking.html +++ /dev/null @@ -1,2112 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" - "DTD/xhtml1-transitional.dtd"> -<html> -<meta name="GENERATOR" content="TtH 3.77"> -<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> - <style type="text/css"> div.p { margin-top: 7pt;}</style> - <style type="text/css"><!-- - td div.comp { margin-top: -0.6ex; margin-bottom: -1ex;} - td div.comb { margin-top: -0.6ex; margin-bottom: -.6ex;} - td div.hrcomp { line-height: 0.9; margin-top: -0.8ex; margin-bottom: -1ex;} - td div.norm {line-height:normal;} - span.roman {font-family: serif; font-style: normal; font-weight: normal;} - span.overacc2 {position: relative; left: .8em; top: -1.2ex;} - span.overacc1 {position: relative; left: .6em; top: -1.2ex;} --></style> - - -<title> Design of a blocking-resistant anonymity system\DRAFT</title> - -<h1 align="center">Design of a blocking-resistant anonymity system<br />DRAFT </h1> - -<div class="p"><!----></div> - -<h3 align="center">Roger Dingledine, Nick Mathewson </h3> - - -<div class="p"><!----></div> - -<h2> Abstract</h2> -Internet censorship is on the rise as websites around the world are -increasingly blocked by government-level firewalls. Although popular -anonymizing networks like Tor were originally designed to keep attackers from -tracing people's activities, many people are also using them to evade local -censorship. But if the censor simply denies access to the Tor network -itself, blocked users can no longer benefit from the security Tor offers. - -<div class="p"><!----></div> -Here we describe a design that builds upon the current Tor network -to provide an anonymizing network that resists blocking -by government-level attackers. - -<div class="p"><!----></div> - - <h2><a name="tth_sEc1"> -1</a> Introduction and Goals</h2> - -<div class="p"><!----></div> -Anonymizing networks like Tor [<a href="#tor-design" name="CITEtor-design">11</a>] bounce traffic around a -network of encrypting relays. Unlike encryption, which hides only <i>what</i> -is said, these networks also aim to hide who is communicating with whom, which -users are using which websites, and similar relations. These systems have a -broad range of users, including ordinary citizens who want to avoid being -profiled for targeted advertisements, corporations who don't want to reveal -information to their competitors, and law enforcement and government -intelligence agencies who need to do operations on the Internet without being -noticed. - -<div class="p"><!----></div> -Historical anonymity research has focused on an -attacker who monitors the user (call her Alice) and tries to discover her -activities, yet lets her reach any piece of the network. In more modern -threat models such as Tor's, the adversary is allowed to perform active -attacks such as modifying communications to trick Alice -into revealing her destination, or intercepting some connections -to run a man-in-the-middle attack. But these systems still assume that -Alice can eventually reach the anonymizing network. - -<div class="p"><!----></div> -An increasing number of users are using the Tor software -less for its anonymity properties than for its censorship -resistance properties — if they use Tor to access Internet sites like -Wikipedia -and Blogspot, they are no longer affected by local censorship -and firewall rules. In fact, an informal user study -showed China as the third largest user base -for Tor clients, with perhaps ten thousand people accessing the Tor -network from China each day. - -<div class="p"><!----></div> -The current Tor design is easy to block if the attacker controls Alice's -connection to the Tor network — by blocking the directory authorities, -by blocking all the server IP addresses in the directory, or by filtering -based on the fingerprint of the Tor TLS handshake. Here we describe an -extended design that builds upon the current Tor network to provide an -anonymizing -network that resists censorship as well as anonymity-breaking attacks. -In section <a href="#sec:adversary">2</a> we discuss our threat model — that is, -the assumptions we make about our adversary. Section <a href="#sec:current-tor">3</a> -describes the components of the current Tor design and how they can be -leveraged for a new blocking-resistant design. Section <a href="#sec:related">4</a> -explains the features and drawbacks of the currently deployed solutions. -In sections <a href="#sec:bridges">5</a> through <a href="#sec:discovery">7</a>, we explore the -components of our designs in detail. Section <a href="#sec:security">8</a> considers -security implications and Section <a href="#sec:reachability">9</a> presents other -issues with maintaining connectivity and sustainability for the design. -Section <a href="#sec:future">10</a> speculates about future more complex designs, -and finally Section <a href="#sec:conclusion">11</a> summarizes our next steps and -recommendations. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc2"> -<a name="sec:adversary"> -2</a> Adversary assumptions</h2> -</a> - -<div class="p"><!----></div> -To design an effective anti-censorship tool, we need a good model for the -goals and resources of the censors we are evading. Otherwise, we risk -spending our effort on keeping the adversaries from doing things they have no -interest in doing, and thwarting techniques they do not use. -The history of blocking-resistance designs is littered with conflicting -assumptions about what adversaries to expect and what problems are -in the critical path to a solution. Here we describe our best -understanding of the current situation around the world. - -<div class="p"><!----></div> -In the traditional security style, we aim to defeat a strong -attacker — if we can defend against this attacker, we inherit protection -against weaker attackers as well. After all, we want a general design -that will work for citizens of China, Thailand, and other censored -countries; for -whistleblowers in firewalled corporate networks; and for people in -unanticipated oppressive situations. In fact, by designing with -a variety of adversaries in mind, we can take advantage of the fact that -adversaries will be in different stages of the arms race at each location, -so a server blocked in one locale can still be useful in others. - -<div class="p"><!----></div> -We assume that the attackers' goals are somewhat complex. - -<dl compact="compact"> - - <dt><b></b></dt> - <dd><li>The attacker would like to restrict the flow of certain kinds of - information, particularly when this information is seen as embarrassing to - those in power (such as information about rights violations or corruption), - or when it enables or encourages others to oppose them effectively (such as - information about opposition movements or sites that are used to organize - protests).</dd> - <dt><b></b></dt> - <dd><li>As a second-order effect, censors aim to chill citizens' behavior by - creating an impression that their online activities are monitored.</dd> - <dt><b></b></dt> - <dd><li>In some cases, censors make a token attempt to block a few sites for - obscenity, blasphemy, and so on, but their efforts here are mainly for - show. In other cases, they really do try hard to block such content.</dd> - <dt><b></b></dt> - <dd><li>Complete blocking (where nobody at all can ever download censored - content) is not a - goal. Attackers typically recognize that perfect censorship is not only - impossible, but unnecessary: if "undesirable" information is known only - to a small few, further censoring efforts can be focused elsewhere.</dd> - <dt><b></b></dt> - <dd><li>Similarly, the censors are not attempting to shut down or block <i> - every</i> anti-censorship tool — merely the tools that are popular and - effective (because these tools impede the censors' information restriction - goals) and those tools that are highly visible (thus making the censors - look ineffectual to their citizens and their bosses).</dd> - <dt><b></b></dt> - <dd><li>Reprisal against <i>most</i> passive consumers of <i>most</i> kinds of - blocked information is also not a goal, given the broadness of most - censorship regimes. This seems borne out by fact.<a href="#tthFtNtAAB" name="tthFrefAAB"><sup>1</sup></a></dd> - <dt><b></b></dt> - <dd><li>Producers and distributors of targeted information are in much - greater danger than consumers; the attacker would like to not only block - their work, but identify them for reprisal.</dd> - <dt><b></b></dt> - <dd><li>The censors (or their governments) would like to have a working, useful - Internet. There are economic, political, and social factors that prevent - them from "censoring" the Internet by outlawing it entirely, or by - blocking access to all but a tiny list of sites. - Nevertheless, the censors <i>are</i> willing to block innocuous content - (like the bulk of a newspaper's reporting) in order to censor other content - distributed through the same channels (like that newspaper's coverage of - the censored country). -</dd> -</dl> - -<div class="p"><!----></div> -We assume there are three main technical network attacks in use by censors -currently [<a href="#clayton:pet2006" name="CITEclayton:pet2006">7</a>]: - -<div class="p"><!----></div> - -<dl compact="compact"> - - <dt><b></b></dt> - <dd><li>Block a destination or type of traffic by automatically searching for - certain strings or patterns in TCP packets. Offending packets can be - dropped, or can trigger a response like closing the - connection.</dd> - <dt><b></b></dt> - <dd><li>Block a destination by listing its IP address at a - firewall or other routing control point.</dd> - <dt><b></b></dt> - <dd><li>Intercept DNS requests and give bogus responses for certain - destination hostnames. -</dd> -</dl> - -<div class="p"><!----></div> -We assume the network firewall has limited CPU and memory per -connection [<a href="#clayton:pet2006" name="CITEclayton:pet2006">7</a>]. Against an adversary who could carefully -examine the contents of every packet and correlate the packets in every -stream on the network, we would need some stronger mechanism such as -steganography, which introduces its own -problems [<a href="#active-wardens" name="CITEactive-wardens">15</a>,<a href="#tcpstego" name="CITEtcpstego">26</a>]. But we make a "weak -steganography" assumption here: to remain unblocked, it is necessary to -remain unobservable only by computational resources on par with a modern -router, firewall, proxy, or IDS. - -<div class="p"><!----></div> -We assume that while various different regimes can coordinate and share -notes, there will be a time lag between one attacker learning how to overcome -a facet of our design and other attackers picking it up. (The most common -vector of transmission seems to be commercial providers of censorship tools: -once a provider adds a feature to meet one country's needs or requests, the -feature is available to all of the provider's customers.) Conversely, we -assume that insider attacks become a higher risk only after the early stages -of network development, once the system has reached a certain level of -success and visibility. - -<div class="p"><!----></div> -We do not assume that government-level attackers are always uniform -across the country. For example, users of different ISPs in China -experience different censorship policies and mechanisms. - -<div class="p"><!----></div> -We assume that the attacker may be able to use political and economic -resources to secure the cooperation of extraterritorial or multinational -corporations and entities in investigating information sources. -For example, the censors can threaten the service providers of -troublesome blogs with economic reprisals if they do not reveal the -authors' identities. - -<div class="p"><!----></div> -We assume that our users have control over their hardware and -software — they don't have any spyware installed, there are no -cameras watching their screens, etc. Unfortunately, in many situations -these threats are real [<a href="#zuckerman-threatmodels" name="CITEzuckerman-threatmodels">28</a>]; yet -software-based security systems like ours are poorly equipped to handle -a user who is entirely observed and controlled by the adversary. See -Section <a href="#subsec:cafes-and-livecds">8.4</a> for more discussion of what little -we can do about this issue. - -<div class="p"><!----></div> -Similarly, we assume that the user will be able to fetch a genuine -version of Tor, rather than one supplied by the adversary; see -Section <a href="#subsec:trust-chain">8.5</a> for discussion on helping the user -confirm that he has a genuine version and that he can connect to the -real Tor network. - -<div class="p"><!----></div> - <h2><a name="tth_sEc3"> -<a name="sec:current-tor"> -3</a> Adapting the current Tor design to anti-censorship</h2> -</a> - -<div class="p"><!----></div> -Tor is popular and sees a lot of use — it's the largest anonymity -network of its kind, and has -attracted more than 800 volunteer-operated routers from around the -world. Tor protects each user by routing their traffic through a multiply -encrypted "circuit" built of a few randomly selected servers, each of which -can remove only a single layer of encryption. Each server sees only the step -before it and the step after it in the circuit, and so no single server can -learn the connection between a user and her chosen communication partners. -In this section, we examine some of the reasons why Tor has become popular, -with particular emphasis to how we can take advantage of these properties -for a blocking-resistance design. - -<div class="p"><!----></div> -Tor aims to provide three security properties: - -<dl compact="compact"> - - <dt><b></b></dt> - <dd>1. A local network attacker can't learn, or influence, your -destination.</dd> - <dt><b></b></dt> - <dd>2. No single router in the Tor network can link you to your -destination.</dd> - <dt><b></b></dt> - <dd>3. The destination, or somebody watching the destination, -can't learn your location. -</dd> -</dl> - -<div class="p"><!----></div> -For blocking-resistance, we care most clearly about the first -property. But as the arms race progresses, the second property -will become important — for example, to discourage an adversary -from volunteering a relay in order to learn that Alice is reading -or posting to certain websites. The third property helps keep users safe from -collaborating websites: consider websites and other Internet services -that have been pressured -recently into revealing the identity of bloggers -or treating clients differently depending on their network -location [<a href="#goodell-syverson06" name="CITEgoodell-syverson06">17</a>]. - -<div class="p"><!----></div> -The Tor design provides other features as well that are not typically -present in manual or ad hoc circumvention techniques. - -<div class="p"><!----></div> -First, Tor has a well-analyzed and well-understood way to distribute -information about servers. -Tor directory authorities automatically aggregate, test, -and publish signed summaries of the available Tor routers. Tor clients -can fetch these summaries to learn which routers are available and -which routers are suitable for their needs. Directory information is cached -throughout the Tor network, so once clients have bootstrapped they never -need to interact with the authorities directly. (To tolerate a minority -of compromised directory authorities, we use a threshold trust scheme — -see Section <a href="#subsec:trust-chain">8.5</a> for details.) - -<div class="p"><!----></div> -Second, the list of directory authorities is not hard-wired. -Clients use the default authorities if no others are specified, -but it's easy to start a separate (or even overlapping) Tor network just -by running a different set of authorities and convincing users to prefer -a modified client. For example, we could launch a distinct Tor network -inside China; some users could even use an aggregate network made up of -both the main network and the China network. (But we should not be too -quick to create other Tor networks — part of Tor's anonymity comes from -users behaving like other users, and there are many unsolved anonymity -questions if different users know about different pieces of the network.) - -<div class="p"><!----></div> -Third, in addition to automatically learning from the chosen directories -which Tor routers are available and working, Tor takes care of building -paths through the network and rebuilding them as needed. So the user -never has to know how paths are chosen, never has to manually pick -working proxies, and so on. More generally, at its core the Tor protocol -is simply a tool that can build paths given a set of routers. Tor is -quite flexible about how it learns about the routers and how it chooses -the paths. Harvard's Blossom project [<a href="#blossom-thesis" name="CITEblossom-thesis">16</a>] makes this -flexibility more concrete: Blossom makes use of Tor not for its security -properties but for its reachability properties. It runs a separate set -of directory authorities, its own set of Tor routers (called the Blossom -network), and uses Tor's flexible path-building to let users view Internet -resources from any point in the Blossom network. - -<div class="p"><!----></div> -Fourth, Tor separates the role of <em>internal relay</em> from the -role of <em>exit relay</em>. That is, some volunteers choose just to relay -traffic between Tor users and Tor routers, and others choose to also allow -connections to external Internet resources. Because we don't force all -volunteers to play both roles, we end up with more relays. This increased -diversity in turn is what gives Tor its security: the more options the -user has for her first hop, and the more options she has for her last hop, -the less likely it is that a given attacker will be watching both ends -of her circuit [<a href="#tor-design" name="CITEtor-design">11</a>]. As a bonus, because our design attracts -more internal relays that want to help out but don't want to deal with -being an exit relay, we end up providing more options for the first -hop — the one most critical to being able to reach the Tor network. - -<div class="p"><!----></div> -Fifth, Tor is sustainable. Zero-Knowledge Systems offered the commercial -but now defunct Freedom Network [<a href="#freedom21-security" name="CITEfreedom21-security">2</a>], a design with -security comparable to Tor's, but its funding model relied on collecting -money from users to pay relay operators. Modern commercial proxy systems -similarly -need to keep collecting money to support their infrastructure. On the -other hand, Tor has built a self-sustaining community of volunteers who -donate their time and resources. This community trust is rooted in Tor's -open design: we tell the world exactly how Tor works, and we provide all -the source code. Users can decide for themselves, or pay any security -expert to decide, whether it is safe to use. Further, Tor's modularity -as described above, along with its open license, mean that its impact -will continue to grow. - -<div class="p"><!----></div> -Sixth, Tor has an established user base of hundreds of -thousands of people from around the world. This diversity of -users contributes to sustainability as above: Tor is used by -ordinary citizens, activists, corporations, law enforcement, and -even government and military users, -and they can -only achieve their security goals by blending together in the same -network [<a href="#econymics" name="CITEeconymics">1</a>,<a href="#usability:weis2006" name="CITEusability:weis2006">9</a>]. This user base also provides -something else: hundreds of thousands of different and often-changing -addresses that we can leverage for our blocking-resistance design. - -<div class="p"><!----></div> -Finally and perhaps most importantly, Tor provides anonymity and prevents any -single server from linking users to their communication partners. Despite -initial appearances, <i>distributed-trust anonymity is critical for -anti-censorship efforts</i>. If any single server can expose dissident bloggers -or compile a list of users' behavior, the censors can profitably compromise -that server's operator, perhaps by applying economic pressure to their -employers, -breaking into their computer, pressuring their family (if they have relatives -in the censored area), or so on. Furthermore, in designs where any relay can -expose its users, the censors can spread suspicion that they are running some -of the relays and use this belief to chill use of the network. - -<div class="p"><!----></div> -We discuss and adapt these components further in -Section <a href="#sec:bridges">5</a>. But first we examine the strengths and -weaknesses of other blocking-resistance approaches, so we can expand -our repertoire of building blocks and ideas. - -<div class="p"><!----></div> - <h2><a name="tth_sEc4"> -<a name="sec:related"> -4</a> Current proxy solutions</h2> -</a> - -<div class="p"><!----></div> -Relay-based blocking-resistance schemes generally have two main -components: a relay component and a discovery component. The relay part -encompasses the process of establishing a connection, sending traffic -back and forth, and so on — everything that's done once the user knows -where she's going to connect. Discovery is the step before that: the -process of finding one or more usable relays. - -<div class="p"><!----></div> -For example, we can divide the pieces of Tor in the previous section -into the process of building paths and sending -traffic over them (relay) and the process of learning from the directory -servers about what routers are available (discovery). With this distinction -in mind, we now examine several categories of relay-based schemes. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.1"> -4.1</a> Centrally-controlled shared proxies</h3> - -<div class="p"><!----></div> -Existing commercial anonymity solutions (like Anonymizer.com) are based -on a set of single-hop proxies. In these systems, each user connects to -a single proxy, which then relays traffic between the user and her -destination. These public proxy -systems are typically characterized by two features: they control and -operate the proxies centrally, and many different users get assigned -to each proxy. - -<div class="p"><!----></div> -In terms of the relay component, single proxies provide weak security -compared to systems that distribute trust over multiple relays, since a -compromised proxy can trivially observe all of its users' actions, and -an eavesdropper only needs to watch a single proxy to perform timing -correlation attacks against all its users' traffic and thus learn where -everyone is connecting. Worse, all users -need to trust the proxy company to have good security itself as well as -to not reveal user activities. - -<div class="p"><!----></div> -On the other hand, single-hop proxies are easier to deploy, and they -can provide better performance than distributed-trust designs like Tor, -since traffic only goes through one relay. They're also more convenient -from the user's perspective — since users entirely trust the proxy, -they can just use their web browser directly. - -<div class="p"><!----></div> -Whether public proxy schemes are more or less scalable than Tor is -still up for debate: commercial anonymity systems can use some of their -revenue to provision more bandwidth as they grow, whereas volunteer-based -anonymity systems can attract thousands of fast relays to spread the load. - -<div class="p"><!----></div> -The discovery piece can take several forms. Most commercial anonymous -proxies have one or a handful of commonly known websites, and their users -log in to those websites and relay their traffic through them. When -these websites get blocked (generally soon after the company becomes -popular), if the company cares about users in the blocked areas, they -start renting lots of disparate IP addresses and rotating through them -as they get blocked. They notify their users of new addresses (by email, -for example). It's an arms race, since attackers can sign up to receive the -email too, but operators have one nice trick available to them: because they -have a list of paying subscribers, they can notify certain subscribers -about updates earlier than others. - -<div class="p"><!----></div> -Access control systems on the proxy let them provide service only to -users with certain characteristics, such as paying customers or people -from certain IP address ranges. - -<div class="p"><!----></div> -Discovery in the face of a government-level firewall is a complex and -unsolved -topic, and we're stuck in this same arms race ourselves; we explore it -in more detail in Section <a href="#sec:discovery">7</a>. But first we examine the -other end of the spectrum — getting volunteers to run the proxies, -and telling only a few people about each proxy. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.2"> -4.2</a> Independent personal proxies</h3> - -<div class="p"><!----></div> -Personal proxies such as Circumventor [<a href="#circumventor" name="CITEcircumventor">18</a>] and -CGIProxy [<a href="#cgiproxy" name="CITEcgiproxy">23</a>] use the same technology as the public ones as -far as the relay component goes, but they use a different strategy for -discovery. Rather than managing a few centralized proxies and constantly -getting new addresses for them as the old addresses are blocked, they -aim to have a large number of entirely independent proxies, each managing -its own (much smaller) set of users. - -<div class="p"><!----></div> -As the Circumventor site explains, "You don't -actually install the Circumventor <em>on</em> the computer that is blocked -from accessing Web sites. You, or a friend of yours, has to install the -Circumventor on some <em>other</em> machine which is not censored." - -<div class="p"><!----></div> -This tactic has great advantages in terms of blocking-resistance — recall -our assumption in Section <a href="#sec:adversary">2</a> that the attention -a system attracts from the attacker is proportional to its number of -users and level of publicity. If each proxy only has a few users, and -there is no central list of proxies, most of them will never get noticed by -the censors. - -<div class="p"><!----></div> -On the other hand, there's a huge scalability question that so far has -prevented these schemes from being widely useful: how does the fellow -in China find a person in Ohio who will run a Circumventor for him? In -some cases he may know and trust some people on the outside, but in many -cases he's just out of luck. Just as hard, how does a new volunteer in -Ohio find a person in China who needs it? - -<div class="p"><!----></div> - -<div class="p"><!----></div> -This challenge leads to a hybrid design-centrally — distributed -personal proxies — which we will investigate in more detail in -Section <a href="#sec:discovery">7</a>. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.3"> -4.3</a> Open proxies</h3> - -<div class="p"><!----></div> -Yet another currently used approach to bypassing firewalls is to locate -open and misconfigured proxies on the Internet. A quick Google search -for "open proxy list" yields a wide variety of freely available lists -of HTTP, HTTPS, and SOCKS proxies. Many small companies have sprung up -providing more refined lists to paying customers. - -<div class="p"><!----></div> -There are some downsides to using these open proxies though. First, -the proxies are of widely varying quality in terms of bandwidth and -stability, and many of them are entirely unreachable. Second, unlike -networks of volunteers like Tor, the legality of routing traffic through -these proxies is questionable: it's widely believed that most of them -don't realize what they're offering, and probably wouldn't allow it if -they realized. Third, in many cases the connection to the proxy is -unencrypted, so firewalls that filter based on keywords in IP packets -will not be hindered. Fourth, in many countries (including China), the -firewall authorities hunt for open proxies as well, to preemptively -block them. And last, many users are suspicious that some -open proxies are a little <em>too</em> convenient: are they run by the -adversary, in which case they get to monitor all the user's requests -just as single-hop proxies can? - -<div class="p"><!----></div> -A distributed-trust design like Tor resolves each of these issues for -the relay component, but a constantly changing set of thousands of open -relays is clearly a useful idea for a discovery component. For example, -users might be able to make use of these proxies to bootstrap their -first introduction into the Tor network. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.4"> -4.4</a> Blocking resistance and JAP</h3> - -<div class="p"><!----></div> -Köpsell and Hilling's Blocking Resistance -design [<a href="#koepsell:wpes2004" name="CITEkoepsell:wpes2004">20</a>] is probably -the closest related work, and is the starting point for the design in this -paper. In this design, the JAP anonymity system [<a href="#web-mix" name="CITEweb-mix">3</a>] is used -as a base instead of Tor. Volunteers operate a large number of access -points that relay traffic to the core JAP -network, which in turn anonymizes users' traffic. The software to run these -relays is, as in our design, included in the JAP client software and enabled -only when the user decides to enable it. Discovery is handled with a -CAPTCHA-based mechanism; users prove that they aren't an automated process, -and are given the address of an access point. (The problem of a determined -attacker with enough manpower to launch many requests and enumerate all the -access points is not considered in depth.) There is also some suggestion -that information about access points could spread through existing social -networks. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.5"> -4.5</a> Infranet</h3> - -<div class="p"><!----></div> -The Infranet design [<a href="#infranet" name="CITEinfranet">14</a>] uses one-hop relays to deliver web -content, but disguises its communications as ordinary HTTP traffic. Requests -are split into multiple requests for URLs on the relay, which then encodes -its responses in the content it returns. The relay needs to be an actual -website with plausible content and a number of URLs which the user might want -to access — if the Infranet software produced its own cover content, it would -be far easier for censors to identify. To keep the censors from noticing -that cover content changes depending on what data is embedded, Infranet needs -the cover content to have an innocuous reason for changing frequently: the -paper recommends watermarked images and webcams. - -<div class="p"><!----></div> -The attacker and relay operators in Infranet's threat model are significantly -different than in ours. Unlike our attacker, Infranet's censor can't be -bypassed with encrypted traffic (presumably because the censor blocks -encrypted traffic, or at least considers it suspicious), and has more -computational resources to devote to each connection than ours (so it can -notice subtle patterns over time). Unlike our bridge operators, Infranet's -operators (and users) have more bandwidth to spare; the overhead in typical -steganography schemes is far higher than Tor's. - -<div class="p"><!----></div> -The Infranet design does not include a discovery element. Discovery, -however, is a critical point: if whatever mechanism allows users to learn -about relays also allows the censor to do so, he can trivially discover and -block their addresses, even if the steganography would prevent mere traffic -observation from revealing the relays' addresses. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.6"> -4.6</a> RST-evasion and other packet-level tricks</h3> - -<div class="p"><!----></div> -In their analysis of China's firewall's content-based blocking, Clayton, -Murdoch and Watson discovered that rather than blocking all packets in a TCP -streams once a forbidden word was noticed, the firewall was simply forging -RST packets to make the communicating parties believe that the connection was -closed [<a href="#clayton:pet2006" name="CITEclayton:pet2006">7</a>]. They proposed altering operating systems -to ignore forged RST packets. This approach might work in some cases, but -in practice it appears that many firewalls start filtering by IP address -once a sufficient number of RST packets have been sent. - -<div class="p"><!----></div> -Other packet-level responses to filtering include splitting -sensitive words across multiple TCP packets, so that the censors' -firewalls can't notice them without performing expensive stream -reconstruction [<a href="#ptacek98insertion" name="CITEptacek98insertion">27</a>]. This technique relies on the -same insight as our weak steganography assumption. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.7"> -4.7</a> Internal caching networks</h3> - -<div class="p"><!----></div> -Freenet [<a href="#freenet-pets00" name="CITEfreenet-pets00">6</a>] is an anonymous peer-to-peer data store. -Analyzing Freenet's security can be difficult, as its design is in flux as -new discovery and routing mechanisms are proposed, and no complete -specification has (to our knowledge) been written. Freenet servers relay -requests for specific content (indexed by a digest of the content) -"toward" the server that hosts it, and then cache the content as it -follows the same path back to -the requesting user. If Freenet's routing mechanism is successful in -allowing nodes to learn about each other and route correctly even as some -node-to-node links are blocked by firewalls, then users inside censored areas -can ask a local Freenet server for a piece of content, and get an answer -without having to connect out of the country at all. Of course, operators of -servers inside the censored area can still be targeted, and the addresses of -external servers can still be blocked. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.8"> -4.8</a> Skype</h3> - -<div class="p"><!----></div> -The popular Skype voice-over-IP software uses multiple techniques to tolerate -restrictive networks, some of which allow it to continue operating in the -presence of censorship. By switching ports and using encryption, Skype -attempts to resist trivial blocking and content filtering. Even if no -encryption were used, it would still be expensive to scan all voice -traffic for sensitive words. Also, most current keyloggers are unable to -store voice traffic. Nevertheless, Skype can still be blocked, especially at -its central login server. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.9"> -4.9</a> Tor itself</h3> - -<div class="p"><!----></div> -And last, we include Tor itself in the list of current solutions -to firewalls. Tens of thousands of people use Tor from countries that -routinely filter their Internet. Tor's website has been blocked in most -of them. But why hasn't the Tor network been blocked yet? - -<div class="p"><!----></div> -We have several theories. The first is the most straightforward: tens of -thousands of people are simply too few to matter. It may help that Tor is -perceived to be for experts only, and thus not worth attention yet. The -more subtle variant on this theory is that we've positioned Tor in the -public eye as a tool for retaining civil liberties in more free countries, -so perhaps blocking authorities don't view it as a threat. (We revisit -this idea when we consider whether and how to publicize a Tor variant -that improves blocking-resistance — see Section <a href="#subsec:publicity">9.5</a> -for more discussion.) - -<div class="p"><!----></div> -The broader explanation is that the maintenance of most government-level -filters is aimed at stopping widespread information flow and appearing to be -in control, not by the impossible goal of blocking all possible ways to bypass -censorship. Censors realize that there will always -be ways for a few people to get around the firewall, and as long as Tor -has not publically threatened their control, they see no urgent need to -block it yet. - -<div class="p"><!----></div> -We should recognize that we're <em>already</em> in the arms race. These -constraints can give us insight into the priorities and capabilities of -our various attackers. - -<div class="p"><!----></div> - <h2><a name="tth_sEc5"> -<a name="sec:bridges"> -5</a> The relay component of our blocking-resistant design</h2> -</a> - -<div class="p"><!----></div> -Section <a href="#sec:current-tor">3</a> describes many reasons why Tor is -well-suited as a building block in our context, but several changes will -allow the design to resist blocking better. The most critical changes are -to get more relay addresses, and to distribute them to users differently. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc5.1"> -5.1</a> Bridge relays</h3> - -<div class="p"><!----></div> -Today, Tor servers operate on less than a thousand distinct IP addresses; -an adversary -could enumerate and block them all with little trouble. To provide a -means of ingress to the network, we need a larger set of entry points, most -of which an adversary won't be able to enumerate easily. Fortunately, we -have such a set: the Tor users. - -<div class="p"><!----></div> -Hundreds of thousands of people around the world use Tor. We can leverage -our already self-selected user base to produce a list of thousands of -frequently-changing IP addresses. Specifically, we can give them a little -button in the GUI that says "Tor for Freedom", and users who click -the button will turn into <em>bridge relays</em> (or just <em>bridges</em> -for short). They can rate limit relayed connections to 10 KB/s (almost -nothing for a broadband user in a free country, but plenty for a user -who otherwise has no access at all), and since they are just relaying -bytes back and forth between blocked users and the main Tor network, they -won't need to make any external connections to Internet sites. Because -of this separation of roles, and because we're making use of software -that the volunteers have already installed for their own use, we expect -our scheme to attract and maintain more volunteers than previous schemes. - -<div class="p"><!----></div> -As usual, there are new anonymity and security implications from running a -bridge relay, particularly from letting people relay traffic through your -Tor client; but we leave this discussion for Section <a href="#sec:security">8</a>. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc5.2"> -5.2</a> The bridge directory authority</h3> - -<div class="p"><!----></div> -How do the bridge relays advertise their existence to the world? We -introduce a second new component of the design: a specialized directory -authority that aggregates and tracks bridges. Bridge relays periodically -publish server descriptors (summaries of their keys, locations, etc, -signed by their long-term identity key), just like the relays in the -"main" Tor network, but in this case they publish them only to the -bridge directory authorities. - -<div class="p"><!----></div> -The main difference between bridge authorities and the directory -authorities for the main Tor network is that the main authorities provide -a list of every known relay, but the bridge authorities only give -out a server descriptor if you already know its identity key. That is, -you can keep up-to-date on a bridge's location and other information -once you know about it, but you can't just grab a list of all the bridges. - -<div class="p"><!----></div> -The identity key, IP address, and directory port for each bridge -authority ship by default with the Tor software, so the bridge relays -can be confident they're publishing to the right location, and the -blocked users can establish an encrypted authenticated channel. See -Section <a href="#subsec:trust-chain">8.5</a> for more discussion of the public key -infrastructure and trust chain. - -<div class="p"><!----></div> -Bridges use Tor to publish their descriptors privately and securely, -so even an attacker monitoring the bridge directory authority's network -can't make a list of all the addresses contacting the authority. -Bridges may publish to only a subset of the -authorities, to limit the potential impact of an authority compromise. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc5.3"> -<a name="subsec:relay-together"> -5.3</a> Putting them together</h3> -</a> - -<div class="p"><!----></div> -If a blocked user knows the identity keys of a set of bridge relays, and -he has correct address information for at least one of them, he can use -that one to make a secure connection to the bridge authority and update -his knowledge about the other bridge relays. He can also use it to make -secure connections to the main Tor network and directory servers, so he -can build circuits and connect to the rest of the Internet. All of these -updates happen in the background: from the blocked user's perspective, -he just accesses the Internet via his Tor client like always. - -<div class="p"><!----></div> -So now we've reduced the problem from how to circumvent the firewall -for all transactions (and how to know that the pages you get have not -been modified by the local attacker) to how to learn about a working -bridge relay. - -<div class="p"><!----></div> -There's another catch though. We need to make sure that the network -traffic we generate by simply connecting to a bridge relay doesn't stand -out too much. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc6"> -<a name="sec:network-fingerprint"> -<a name="subsec:enclave-dirs"> -6</a> Hiding Tor's network fingerprint</h2> -</a> -</a> - -<div class="p"><!----></div> -Currently, Tor uses two protocols for its network communications. The -main protocol uses TLS for encrypted and authenticated communication -between Tor instances. The second protocol is standard HTTP, used for -fetching directory information. All Tor servers listen on their "ORPort" -for TLS connections, and some of them opt to listen on their "DirPort" -as well, to serve directory information. Tor servers choose whatever port -numbers they like; the server descriptor they publish to the directory -tells users where to connect. - -<div class="p"><!----></div> -One format for communicating address information about a bridge relay is -its IP address and DirPort. From there, the user can ask the bridge's -directory cache for an up-to-date copy of its server descriptor, and -learn its current circuit keys, its ORPort, and so on. - -<div class="p"><!----></div> -However, connecting directly to the directory cache involves a plaintext -HTTP request. A censor could create a network fingerprint (known as a -<em>signature</em> in the intrusion detection field) for the request -and/or its response, thus preventing these connections. To resolve this -vulnerability, we've modified the Tor protocol so that users can connect -to the directory cache via the main Tor port — they establish a TLS -connection with the bridge as normal, and then send a special "begindir" -relay command to establish an internal connection to its directory cache. - -<div class="p"><!----></div> -Therefore a better way to summarize a bridge's address is by its IP -address and ORPort, so all communications between the client and the -bridge will use ordinary TLS. But there are other details that need -more investigation. - -<div class="p"><!----></div> -What port should bridges pick for their ORPort? We currently recommend -that they listen on port 443 (the default HTTPS port) if they want to -be most useful, because clients behind standard firewalls will have -the best chance to reach them. Is this the best choice in all cases, -or should we encourage some fraction of them pick random ports, or other -ports commonly permitted through firewalls like 53 (DNS) or 110 -(POP)? Or perhaps we should use other ports where TLS traffic is -expected, like 993 (IMAPS) or 995 (POP3S). We need more research on our -potential users, and their current and anticipated firewall restrictions. - -<div class="p"><!----></div> -Furthermore, we need to look at the specifics of Tor's TLS handshake. -Right now Tor uses some predictable strings in its TLS handshakes. For -example, it sets the X.509 organizationName field to "Tor", and it puts -the Tor server's nickname in the certificate's commonName field. We -should tweak the handshake protocol so it doesn't rely on any unusual details -in the certificate, yet it remains secure; the certificate itself -should be made to resemble an ordinary HTTPS certificate. We should also try -to make our advertised cipher-suites closer to what an ordinary web server -would support. - -<div class="p"><!----></div> -Tor's TLS handshake uses two-certificate chains: one certificate -contains the self-signed identity key for -the router, and the second contains a current TLS key, signed by the -identity key. We use these to authenticate that we're talking to the right -router, and to limit the impact of TLS-key exposure. Most (though far from -all) consumer-oriented HTTPS services provide only a single certificate. -These extra certificates may help identify Tor's TLS handshake; instead, -bridges should consider using only a single TLS key certificate signed by -their identity key, and providing the full value of the identity key in an -early handshake cell. More significantly, Tor currently has all clients -present certificates, so that clients are harder to distinguish from servers. -But in a blocking-resistance environment, clients should not present -certificates at all. - -<div class="p"><!----></div> -Last, what if the adversary starts observing the network traffic even -more closely? Even if our TLS handshake looks innocent, our traffic timing -and volume still look different than a user making a secure web connection -to his bank. The same techniques used in the growing trend to build tools -to recognize encrypted Bittorrent traffic -could be used to identify Tor communication and recognize bridge -relays. Rather than trying to look like encrypted web traffic, we may be -better off trying to blend with some other encrypted network protocol. The -first step is to compare typical network behavior for a Tor client to -typical network behavior for various other protocols. This statistical -cat-and-mouse game is made more complex by the fact that Tor transports a -variety of protocols, and we'll want to automatically handle web browsing -differently from, say, instant messaging. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc6.1"> -<a name="subsec:id-address"> -6.1</a> Identity keys as part of addressing information</h3> -</a> - -<div class="p"><!----></div> -We have described a way for the blocked user to bootstrap into the -network once he knows the IP address and ORPort of a bridge. What about -local spoofing attacks? That is, since we never learned an identity -key fingerprint for the bridge, a local attacker could intercept our -connection and pretend to be the bridge we had in mind. It turns out -that giving false information isn't that bad — since the Tor client -ships with trusted keys for the bridge directory authority and the Tor -network directory authorities, the user can learn whether he's being -given a real connection to the bridge authorities or not. (After all, -if the adversary intercepts every connection the user makes and gives -him a bad connection each time, there's nothing we can do.) - -<div class="p"><!----></div> -What about anonymity-breaking attacks from observing traffic, if the -blocked user doesn't start out knowing the identity key of his intended -bridge? The vulnerabilities aren't so bad in this case either — the -adversary could do similar attacks just by monitoring the network -traffic. - -<div class="p"><!----></div> -Once the Tor client has fetched the bridge's server descriptor, it should -remember the identity key fingerprint for that bridge relay. Thus if -the bridge relay moves to a new IP address, the client can query the -bridge directory authority to look up a fresh server descriptor using -this fingerprint. - -<div class="p"><!----></div> -So we've shown that it's <em>possible</em> to bootstrap into the network -just by learning the IP address and ORPort of a bridge, but are there -situations where it's more convenient or more secure to learn the bridge's -identity fingerprint as well as instead, while bootstrapping? We keep -that question in mind as we next investigate bootstrapping and discovery. - -<div class="p"><!----></div> - <h2><a name="tth_sEc7"> -<a name="sec:discovery"> -7</a> Discovering working bridge relays</h2> -</a> - -<div class="p"><!----></div> -Tor's modular design means that we can develop a better relay component -independently of developing the discovery component. This modularity's -great promise is that we can pick any discovery approach we like; but the -unfortunate fact is that we have no magic bullet for discovery. We're -in the same arms race as all the other designs we described in -Section <a href="#sec:related">4</a>. - -<div class="p"><!----></div> -In this section we describe a variety of approaches to adding discovery -components for our design. - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.1"> -<a name="subsec:first-bridge"> -7.1</a> Bootstrapping: finding your first bridge.</h3> -</a> - -<div class="p"><!----></div> -In Section <a href="#subsec:relay-together">5.3</a>, we showed that a user who knows -a working bridge address can use it to reach the bridge authority and -to stay connected to the Tor network. But how do new users reach the -bridge authority in the first place? After all, the bridge authority -will be one of the first addresses that a censor blocks. - -<div class="p"><!----></div> -First, we should recognize that most government firewalls are not -perfect. That is, they may allow connections to Google cache or some -open proxy servers, or they let file-sharing traffic, Skype, instant -messaging, or World-of-Warcraft connections through. Different users will -have different mechanisms for bypassing the firewall initially. Second, -we should remember that most people don't operate in a vacuum; users will -hopefully know other people who are in other situations or have other -resources available. In the rest of this section we develop a toolkit -of different options and mechanisms, so that we can enable users in a -diverse set of contexts to bootstrap into the system. - -<div class="p"><!----></div> -(For users who can't use any of these techniques, hopefully they know -a friend who can — for example, perhaps the friend already knows some -bridge relay addresses. If they can't get around it at all, then we -can't help them — they should go meet more people or learn more about -the technology running the firewall in their area.) - -<div class="p"><!----></div> -By deploying all the schemes in the toolkit at once, we let bridges and -blocked users employ the discovery approach that is most appropriate -for their situation. - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.2"> -7.2</a> Independent bridges, no central discovery</h3> - -<div class="p"><!----></div> -The first design is simply to have no centralized discovery component at -all. Volunteers run bridges, and we assume they have some blocked users -in mind and communicate their address information to them out-of-band -(for example, through Gmail). This design allows for small personal -bridges that have only one or a handful of users in mind, but it can -also support an entire community of users. For example, Citizen Lab's -upcoming Psiphon single-hop proxy tool [<a href="#psiphon" name="CITEpsiphon">13</a>] plans to use this -<em>social network</em> approach as its discovery component. - -<div class="p"><!----></div> -There are several ways to do bootstrapping in this design. In the simple -case, the operator of the bridge informs each chosen user about his -bridge's address information and/or keys. A different approach involves -blocked users introducing new blocked users to the bridges they know. -That is, somebody in the blocked area can pass along a bridge's address to -somebody else they trust. This scheme brings in appealing but complex game -theoretic properties: the blocked user making the decision has an incentive -only to delegate to trustworthy people, since an adversary who learns -the bridge's address and filters it makes it unavailable for both of them. -Also, delegating known bridges to members of your social network can be -dangerous: an the adversary who can learn who knows which bridges may -be able to reconstruct the social network. - -<div class="p"><!----></div> -Note that a central set of bridge directory authorities can still be -compatible with a decentralized discovery process. That is, how users -first learn about bridges is entirely up to the bridges, but the process -of fetching up-to-date descriptors for them can still proceed as described -in Section <a href="#sec:bridges">5</a>. Of course, creating a central place that -knows about all the bridges may not be smart, especially if every other -piece of the system is decentralized. Further, if a user only knows -about one bridge and he loses track of it, it may be quite a hassle to -reach the bridge authority. We address these concerns next. - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.3"> -7.3</a> Families of bridges, no central discovery</h3> - -<div class="p"><!----></div> -Because the blocked users are running our software too, we have many -opportunities to improve usability or robustness. Our second design builds -on the first by encouraging volunteers to run several bridges at once -(or coordinate with other bridge volunteers), such that some -of the bridges are likely to be available at any given time. - -<div class="p"><!----></div> -The blocked user's Tor client would periodically fetch an updated set of -recommended bridges from any of the working bridges. Now the client can -learn new additions to the bridge pool, and can expire abandoned bridges -or bridges that the adversary has blocked, without the user ever needing -to care. To simplify maintenance of the community's bridge pool, each -community could run its own bridge directory authority — reachable via -the available bridges, and also mirrored at each bridge. - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.4"> -7.4</a> Public bridges with central discovery</h3> - -<div class="p"><!----></div> -What about people who want to volunteer as bridges but don't know any -suitable blocked users? What about people who are blocked but don't -know anybody on the outside? Here we describe how to make use of these -<em>public bridges</em> in a way that still makes it hard for the attacker -to learn all of them. - -<div class="p"><!----></div> -The basic idea is to divide public bridges into a set of pools based on -identity key. Each pool corresponds to a <em>distribution strategy</em>: -an approach to distributing its bridge addresses to users. Each strategy -is designed to exercise a different scarce resource or property of -the user. - -<div class="p"><!----></div> -How do we divide bridges between these strategy pools such that they're -evenly distributed and the allocation is hard to influence or predict, -but also in a way that's amenable to creating more strategies later -on without reshuffling all the pools? We assign a given bridge -to a strategy pool by hashing the bridge's identity key along with a -secret that only the bridge authority knows: the first n bits of this -hash dictate the strategy pool number, where n is a parameter that -describes how many strategy pools we want at this point. We choose n=3 -to start, so we divide bridges between 8 pools; but as we later invent -new distribution strategies, we can increment n to split the 8 into -16. Since a bridge can't predict the next bit in its hash, it can't -anticipate which identity key will correspond to a certain new pool -when the pools are split. Further, since the bridge authority doesn't -provide any feedback to the bridge about which strategy pool it's in, -an adversary who signs up bridges with the goal of filling a certain -pool [<a href="#casc-rep" name="CITEcasc-rep">12</a>] will be hindered. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -The first distribution strategy (used for the first pool) publishes bridge -addresses in a time-release fashion. The bridge authority divides the -available bridges into partitions, and each partition is deterministically -available only in certain time windows. That is, over the course of a -given time slot (say, an hour), each requester is given a random bridge -from within that partition. When the next time slot arrives, a new set -of bridges from the pool are available for discovery. Thus some bridge -address is always available when a new -user arrives, but to learn about all bridges the attacker needs to fetch -all new addresses at every new time slot. By varying the length of the -time slots, we can make it harder for the attacker to guess when to check -back. We expect these bridges will be the first to be blocked, but they'll -help the system bootstrap until they <em>do</em> get blocked. Further, -remember that we're dealing with different blocking regimes around the -world that will progress at different rates — so this pool will still -be useful to some users even as the arms races progress. - -<div class="p"><!----></div> -The second distribution strategy publishes bridge addresses based on the IP -address of the requesting user. Specifically, the bridge authority will -divide the available bridges in the pool into a bunch of partitions -(as in the first distribution scheme), hash the requester's IP address -with a secret of its own (as in the above allocation scheme for creating -pools), and give the requester a random bridge from the appropriate -partition. To raise the bar, we should discard the last octet of the -IP address before inputting it to the hash function, so an attacker -who only controls a single "/24" network only counts as one user. A -large attacker like China will still be able to control many addresses, -but the hassle of establishing connections from each network (or spoofing -TCP connections) may still slow them down. Similarly, as a special case, -we should treat IP addresses that are Tor exit nodes as all being on -the same network. - -<div class="p"><!----></div> -The third strategy combines the time-based and location-based -strategies to further constrain and rate-limit the available bridge -addresses. Specifically, the bridge address provided in a given time -slot to a given network location is deterministic within the partition, -rather than chosen randomly each time from the partition. Thus, repeated -requests during that time slot from a given network are given the same -bridge address as the first request. - -<div class="p"><!----></div> -The fourth strategy is based on Circumventor's discovery strategy. -The Circumventor project, realizing that its adoption will remain limited -if it has no central coordination mechanism, has started a mailing list to -distribute new proxy addresses every few days. From experimentation it -seems they have concluded that sending updates every three or four days -is sufficient to stay ahead of the current attackers. - -<div class="p"><!----></div> -The fifth strategy provides an alternative approach to a mailing list: -users provide an email address and receive an automated response -listing an available bridge address. We could limit one response per -email address. To further rate limit queries, we could require a CAPTCHA -solution -in each case too. In fact, we wouldn't need to -implement the CAPTCHA on our side: if we only deliver bridge addresses -to Yahoo or GMail addresses, we can leverage the rate-limiting schemes -that other parties already impose for account creation. - -<div class="p"><!----></div> -The sixth strategy ties in the social network design with public -bridges and a reputation system. We pick some seeds — trusted people in -blocked areas — and give them each a few dozen bridge addresses and a few -<em>delegation tokens</em>. We run a website next to the bridge authority, -where users can log in (they connect via Tor, and they don't need to -provide actual identities, just persistent pseudonyms). Users can delegate -trust to other people they know by giving them a token, which can be -exchanged for a new account on the website. Accounts in "good standing" -then accrue new bridge addresses and new tokens. As usual, reputation -schemes bring in a host of new complexities [<a href="#rep-anon" name="CITErep-anon">10</a>]: how do we -decide that an account is in good standing? We could tie reputation -to whether the bridges they're told about have been blocked — see -Section <a href="#subsec:geoip">7.7</a> below for initial thoughts on how to discover -whether bridges have been blocked. We could track reputation between -accounts (if you delegate to somebody who screws up, it impacts you too), -or we could use blinded delegation tokens [<a href="#chaum-blind" name="CITEchaum-blind">5</a>] to prevent -the website from mapping the seeds' social network. We put off deeper -discussion of the social network reputation strategy for future work. - -<div class="p"><!----></div> -Pools seven and eight are held in reserve, in case our currently deployed -tricks all fail at once and the adversary blocks all those bridges — so -we can adapt and move to new approaches quickly, and have some bridges -immediately available for the new schemes. New strategies might be based -on some other scarce resource, such as relaying traffic for others or -other proof of energy spent. (We might also worry about the incentives -for bridges that sign up and get allocated to the reserve pools: will they -be unhappy that they're not being used? But this is a transient problem: -if Tor users are bridges by default, nobody will mind not being used yet. -See also Section <a href="#subsec:incentives">9.4</a>.) - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.5"> -7.5</a> Public bridges with coordinated discovery</h3> - -<div class="p"><!----></div> -We presented the above discovery strategies in the context of a single -bridge directory authority, but in practice we will want to distribute the -operations over several bridge authorities — a single point of failure -or attack is a bad move. The first answer is to run several independent -bridge directory authorities, and bridges gravitate to one based on -their identity key. The better answer would be some federation of bridge -authorities that work together to provide redundancy but don't introduce -new security issues. We could even imagine designs where the bridge -authorities have encrypted versions of the bridge's server descriptors, -and the users learn a decryption key that they keep private when they -first hear about the bridge — this way the bridge authorities would not -be able to learn the IP address of the bridges. - -<div class="p"><!----></div> -We leave this design question for future work. - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.6"> -7.6</a> Assessing whether bridges are useful</h3> - -<div class="p"><!----></div> -Learning whether a bridge is useful is important in the bridge authority's -decision to include it in responses to blocked users. For example, if -we end up with a list of thousands of bridges and only a few dozen of -them are reachable right now, most blocked users will not end up knowing -about working bridges. - -<div class="p"><!----></div> -There are three components for assessing how useful a bridge is. First, -is it reachable from the public Internet? Second, what proportion of -the time is it available? Third, is it blocked in certain jurisdictions? - -<div class="p"><!----></div> -The first component can be tested just as we test reachability of -ordinary Tor servers. Specifically, the bridges do a self-test — connect -to themselves via the Tor network — before they are willing to -publish their descriptor, to make sure they're not obviously broken or -misconfigured. Once the bridges publish, the bridge authority also tests -reachability to make sure they're not confused or outright lying. - -<div class="p"><!----></div> -The second component can be measured and tracked by the bridge authority. -By doing periodic reachability tests, we can get a sense of how often the -bridge is available. More complex tests will involve bandwidth-intensive -checks to force the bridge to commit resources in order to be counted as -available. We need to evaluate how the relationship of uptime percentage -should weigh into our choice of which bridges to advertise. We leave -this to future work. - -<div class="p"><!----></div> -The third component is perhaps the trickiest: with many different -adversaries out there, how do we keep track of which adversaries have -blocked which bridges, and how do we learn about new blocks as they -occur? We examine this problem next. - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.7"> -<a name="subsec:geoip"> -7.7</a> How do we know if a bridge relay has been blocked?</h3> -</a> - -<div class="p"><!----></div> -There are two main mechanisms for testing whether bridges are reachable -from inside each blocked area: active testing via users, and passive -testing via bridges. - -<div class="p"><!----></div> -In the case of active testing, certain users inside each area -sign up as testing relays. The bridge authorities can then use a -Blossom-like [<a href="#blossom-thesis" name="CITEblossom-thesis">16</a>] system to build circuits through them -to each bridge and see if it can establish the connection. But how do -we pick the users? If we ask random users to do the testing (or if we -solicit volunteers from the users), the adversary should sign up so he -can enumerate the bridges we test. Indeed, even if we hand-select our -testers, the adversary might still discover their location and monitor -their network activity to learn bridge addresses. - -<div class="p"><!----></div> -Another answer is not to measure directly, but rather let the bridges -report whether they're being used. -Specifically, bridges should install a GeoIP database such as the public -IP-To-Country list [<a href="#ip-to-country" name="CITEip-to-country">19</a>], and then periodically report to the -bridge authorities which countries they're seeing use from. This data -would help us track which countries are making use of the bridge design, -and can also let us learn about new steps the adversary has taken in -the arms race. (The compressed GeoIP database is only several hundred -kilobytes, and we could even automate the update process by serving it -from the bridge authorities.) -More analysis of this passive reachability -testing design is needed to resolve its many edge cases: for example, -if a bridge stops seeing use from a certain area, does that mean the -bridge is blocked or does that mean those users are asleep? - -<div class="p"><!----></div> -There are many more problems with the general concept of detecting whether -bridges are blocked. First, different zones of the Internet are blocked -in different ways, and the actual firewall jurisdictions do not match -country borders. Our bridge scheme could help us map out the topology -of the censored Internet, but this is a huge task. More generally, -if a bridge relay isn't reachable, is that because of a network block -somewhere, because of a problem at the bridge relay, or just a temporary -outage somewhere in between? And last, an attacker could poison our -bridge database by signing up already-blocked bridges. In this case, -if we're stingy giving out bridge addresses, users in that country won't -learn working bridges. - -<div class="p"><!----></div> -All of these issues are made more complex when we try to integrate this -testing into our social network reputation system above. -Since in that case we punish or reward users based on whether bridges -get blocked, the adversary has new attacks to trick or bog down the -reputation tracking. Indeed, the bridge authority doesn't even know -what zone the blocked user is in, so do we blame him for any possible -censored zone, or what? - -<div class="p"><!----></div> -Clearly more analysis is required. The eventual solution will probably -involve a combination of passive measurement via GeoIP and active -measurement from trusted testers. More generally, we can use the passive -feedback mechanism to track usage of the bridge network as a whole — which -would let us respond to attacks and adapt the design, and it would also -let the general public track the progress of the project. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc7.8"> -7.8</a> Advantages of deploying all solutions at once</h3> - -<div class="p"><!----></div> -For once, we're not in the position of the defender: we don't have to -defend against every possible filtering scheme; we just have to defend -against at least one. On the flip side, the attacker is forced to guess -how to allocate his resources to defend against each of these discovery -strategies. So by deploying all of our strategies at once, we not only -increase our chances of finding one that the adversary has difficulty -blocking, but we actually make <em>all</em> of the strategies more robust -in the face of an adversary with limited resources. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc8"> -<a name="sec:security"> -8</a> Security considerations</h2> -</a> - -<div class="p"><!----></div> - <h3><a name="tth_sEc8.1"> -8.1</a> Possession of Tor in oppressed areas</h3> - -<div class="p"><!----></div> -Many people speculate that installing and using a Tor client in areas with -particularly extreme firewalls is a high risk — and the risk increases -as the firewall gets more restrictive. This notion certainly has merit, but -there's -a counter pressure as well: as the firewall gets more restrictive, more -ordinary people behind it end up using Tor for more mainstream activities, -such as learning -about Wall Street prices or looking at pictures of women's ankles. So -as the restrictive firewall pushes up the number of Tor users, the -"typical" Tor user becomes more mainstream, and therefore mere -use or possession of the Tor software is not so surprising. - -<div class="p"><!----></div> -It's hard to say which of these pressures will ultimately win out, -but we should keep both sides of the issue in mind. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc8.2"> -<a name="subsec:upload-padding"> -8.2</a> Observers can tell who is publishing and who is reading</h3> -</a> - -<div class="p"><!----></div> -Tor encrypts traffic on the local network, and it obscures the eventual -destination of the communication, but it doesn't do much to obscure the -traffic volume. In particular, a user publishing a home video will have a -different network fingerprint than a user reading an online news article. -Based on our assumption in Section <a href="#sec:adversary">2</a> that users who -publish material are in more danger, should we work to improve Tor's -security in this situation? - -<div class="p"><!----></div> -In the general case this is an extremely challenging task: -effective <em>end-to-end traffic confirmation attacks</em> -are known where the adversary observes the origin and the -destination of traffic and confirms that they are part of the -same communication [<a href="#danezis:pet2004" name="CITEdanezis:pet2004">8</a>,<a href="#e2e-traffic" name="CITEe2e-traffic">24</a>]. Related are -<em>website fingerprinting attacks</em>, where the adversary downloads -a few hundred popular websites, makes a set of "fingerprints" for each -site, and then observes the target Tor client's traffic to look for -a match [<a href="#pet05-bissias" name="CITEpet05-bissias">4</a>,<a href="#defensive-dropping" name="CITEdefensive-dropping">21</a>]. But can we do better -against a limited adversary who just does coarse-grained sweeps looking -for unusually prolific publishers? - -<div class="p"><!----></div> -One answer is for bridge users to automatically send bursts of padding -traffic periodically. (This traffic can be implemented in terms of -long-range drop cells, which are already part of the Tor specification.) -Of course, convincingly simulating an actual human publishing interesting -content is a difficult arms race, but it may be worthwhile to at least -start the race. More research remains. - -<div class="p"><!----></div> - <h3><a name="tth_sEc8.3"> -8.3</a> Anonymity effects from acting as a bridge relay</h3> - -<div class="p"><!----></div> -Against some attacks, relaying traffic for others can improve -anonymity. The simplest example is an attacker who owns a small number -of Tor servers. He will see a connection from the bridge, but he won't -be able to know whether the connection originated there or was relayed -from somebody else. More generally, the mere uncertainty of whether the -traffic originated from that user may be helpful. - -<div class="p"><!----></div> -There are some cases where it doesn't seem to help: if an attacker can -watch all of the bridge's incoming and outgoing traffic, then it's easy -to learn which connections were relayed and which started there. (In this -case he still doesn't know the final destinations unless he is watching -them too, but in this case bridges are no better off than if they were -an ordinary client.) - -<div class="p"><!----></div> -There are also some potential downsides to running a bridge. First, while -we try to make it hard to enumerate all bridges, it's still possible to -learn about some of them, and for some people just the fact that they're -running one might signal to an attacker that they place a higher value -on their anonymity. Second, there are some more esoteric attacks on Tor -relays that are not as well-understood or well-tested — for example, an -attacker may be able to "observe" whether the bridge is sending traffic -even if he can't actually watch its network, by relaying traffic through -it and noticing changes in traffic timing [<a href="#attack-tor-oak05" name="CITEattack-tor-oak05">25</a>]. On -the other hand, it may be that limiting the bandwidth the bridge is -willing to relay will allow this sort of attacker to determine if it's -being used as a bridge but not easily learn whether it is adding traffic -of its own. - -<div class="p"><!----></div> -We also need to examine how entry guards fit in. Entry guards -(a small set of nodes that are always used for the first -step in a circuit) help protect against certain attacks -where the attacker runs a few Tor servers and waits for -the user to choose these servers as the beginning and end of her -circuit<a href="#tthFtNtAAC" name="tthFrefAAC"><sup>2</sup></a>. -If the blocked user doesn't use the bridge's entry guards, then the bridge -doesn't gain as much cover benefit. On the other hand, what design changes -are needed for the blocked user to use the bridge's entry guards without -learning what they are (this seems hard), and even if we solve that, -do they then need to use the guards' guards and so on down the line? - -<div class="p"><!----></div> -It is an open research question whether the benefits of running a bridge -outweigh the risks. A lot of the decision rests on which attacks the -users are most worried about. For most users, we don't think running a -bridge relay will be that damaging, and it could help quite a bit. - -<div class="p"><!----></div> - <h3><a name="tth_sEc8.4"> -<a name="subsec:cafes-and-livecds"> -8.4</a> Trusting local hardware: Internet cafes and LiveCDs</h3> -</a> - -<div class="p"><!----></div> -Assuming that users have their own trusted hardware is not -always reasonable. - -<div class="p"><!----></div> -For Internet cafe Windows computers that let you attach your own USB key, -a USB-based Tor image would be smart. There's Torpark, and hopefully -there will be more thoroughly analyzed and trustworthy options down the -road. Worries remain about hardware or software keyloggers and other -spyware, as well as physical surveillance. - -<div class="p"><!----></div> -If the system lets you boot from a CD or from a USB key, you can gain -a bit more security by bringing a privacy LiveCD with you. (This -approach isn't foolproof either of course, since hardware -keyloggers and physical surveillance are still a worry). - -<div class="p"><!----></div> -In fact, LiveCDs are also useful if it's your own hardware, since it's -easier to avoid leaving private data and logs scattered around the -system. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc8.5"> -<a name="subsec:trust-chain"> -8.5</a> The trust chain</h3> -</a> - -<div class="p"><!----></div> -Tor's "public key infrastructure" provides a chain of trust to -let users verify that they're actually talking to the right servers. -There are four pieces to this trust chain. - -<div class="p"><!----></div> -First, when Tor clients are establishing circuits, at each step -they demand that the next Tor server in the path prove knowledge of -its private key [<a href="#tor-design" name="CITEtor-design">11</a>]. This step prevents the first node -in the path from just spoofing the rest of the path. Second, the -Tor directory authorities provide a signed list of servers along with -their public keys — so unless the adversary can control a threshold -of directory authorities, he can't trick the Tor client into using other -Tor servers. Third, the location and keys of the directory authorities, -in turn, is hard-coded in the Tor source code — so as long as the user -got a genuine version of Tor, he can know that he is using the genuine -Tor network. And last, the source code and other packages are signed -with the GPG keys of the Tor developers, so users can confirm that they -did in fact download a genuine version of Tor. - -<div class="p"><!----></div> -In the case of blocked users contacting bridges and bridge directory -authorities, the same logic applies in parallel: the blocked users fetch -information from both the bridge authorities and the directory authorities -for the `main' Tor network, and they combine this information locally. - -<div class="p"><!----></div> -How can a user in an oppressed country know that he has the correct -key fingerprints for the developers? As with other security systems, it -ultimately comes down to human interaction. The keys are signed by dozens -of people around the world, and we have to hope that our users have met -enough people in the PGP web of trust -that they can learn -the correct keys. For users that aren't connected to the global security -community, though, this question remains a critical weakness. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc9"> -<a name="sec:reachability"> -9</a> Maintaining reachability</h2> -</a> - -<div class="p"><!----></div> - <h3><a name="tth_sEc9.1"> -9.1</a> How many bridge relays should you know about?</h3> - -<div class="p"><!----></div> -The strategies described in Section <a href="#sec:discovery">7</a> talked about -learning one bridge address at a time. But if most bridges are ordinary -Tor users on cable modem or DSL connection, many of them will disappear -and/or move periodically. How many bridge relays should a blocked user -know about so that she is likely to have at least one reachable at any -given point? This is already a challenging problem if we only consider -natural churn: the best approach is to see what bridges we attract in -reality and measure their churn. We may also need to factor in a parameter -for how quickly bridges get discovered and blocked by the attacker; -we leave this for future work after we have more deployment experience. - -<div class="p"><!----></div> -A related question is: if the bridge relays change IP addresses -periodically, how often does the blocked user need to fetch updates in -order to keep from being cut out of the loop? - -<div class="p"><!----></div> -Once we have more experience and intuition, we should explore technical -solutions to this problem too. For example, if the discovery strategies -give out k bridge addresses rather than a single bridge address, perhaps -we can improve robustness from the user perspective without significantly -aiding the adversary. Rather than giving out a new random subset of k -addresses at each point, we could bind them together into <em>bridge -families</em>, so all users that learn about one member of the bridge family -are told about the rest as well. - -<div class="p"><!----></div> -This scheme may also help defend against attacks to map the set of -bridges. That is, if all blocked users learn a random subset of bridges, -the attacker should learn about a few bridges, monitor the country-level -firewall for connections to them, then watch those users to see what -other bridges they use, and repeat. By segmenting the bridge address -space, we can limit the exposure of other users. - -<div class="p"><!----></div> - <h3><a name="tth_sEc9.2"> -<a name="subsec:block-cable"> -9.2</a> Cablemodem users don't usually provide important websites</h3> -</a> - -<div class="p"><!----></div> -Another attacker we might be concerned about is that the attacker could -just block all DSL and cablemodem network addresses, on the theory that -they don't run any important services anyway. If most of our bridges -are on these networks, this attack could really hurt. - -<div class="p"><!----></div> -The first answer is to aim to get volunteers both from traditionally -"consumer" networks and also from traditionally "producer" networks. -Since bridges don't need to be Tor exit nodes, as we improve our usability -it seems quite feasible to get a lot of websites helping out. - -<div class="p"><!----></div> -The second answer (not as practical) would be to encourage more use of -consumer networks for popular and useful Internet services. - -<div class="p"><!----></div> -A related attack we might worry about is based on large countries putting -economic pressure on companies that want to expand their business. For -example, what happens if Verizon wants to sell services in China, and -China pressures Verizon to discourage its users in the free world from -running bridges? - -<div class="p"><!----></div> - <h3><a name="tth_sEc9.3"> -9.3</a> Scanning resistance: making bridges more subtle</h3> - -<div class="p"><!----></div> -If it's trivial to verify that a given address is operating as a bridge, -and most bridges run on a predictable port, then it's conceivable our -attacker could scan the whole Internet looking for bridges. (In fact, -he can just concentrate on scanning likely networks like cablemodem -and DSL services — see Section <a href="#subsec:block-cable">9.2</a> -above for -related attacks.) It would be nice to slow down this attack. It would -be even nicer to make it hard to learn whether we're a bridge without -first knowing some secret. We call this general property <em>scanning -resistance</em>, and it goes along with normalizing Tor's TLS handshake and -network fingerprint. - -<div class="p"><!----></div> -We could provide a password to the blocked user, and she (or her Tor -client) provides a nonced hash of this password when she connects. We'd -need to give her an ID key for the bridge too (in addition to the IP -address and port — see Section <a href="#subsec:id-address">6.1</a>), and wait to -present the password until we've finished the TLS handshake, else it -would look unusual. If Alice can authenticate the bridge before she -tries to send her password, we can resist an adversary who pretends -to be the bridge and launches a man-in-the-middle attack to learn the -password. But even if she can't, we still resist against widespread -scanning. - -<div class="p"><!----></div> -How should the bridge behave if accessed without the correct -authorization? Perhaps it should act like an unconfigured HTTPS server -("welcome to the default Apache page"), or maybe it should mirror -and act like common websites, or websites randomly chosen from Google. - -<div class="p"><!----></div> -We might assume that the attacker can recognize HTTPS connections that -use self-signed certificates. (This process would be resource-intensive -but not out of the realm of possibility.) But even in this case, many -popular websites around the Internet use self-signed or just plain broken -SSL certificates. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc9.4"> -<a name="subsec:incentives"> -9.4</a> How to motivate people to run bridge relays</h3> -</a> - -<div class="p"><!----></div> -One of the traditional ways to get people to run software that benefits -others is to give them motivation to install it themselves. An often -suggested approach is to install it as a stunning screensaver so everybody -will be pleased to run it. We take a similar approach here, by leveraging -the fact that these users are already interested in protecting their -own Internet traffic, so they will install and run the software. - -<div class="p"><!----></div> -Eventually, we may be able to make all Tor users become bridges if they -pass their self-reachability tests — the software and installers need -more work on usability first, but we're making progress. - -<div class="p"><!----></div> -In the mean time, we can make a snazzy network graph with -Vidalia<a href="#tthFtNtAAD" name="tthFrefAAD"><sup>3</sup></a> that -emphasizes the connections the bridge user is currently relaying. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h3><a name="tth_sEc9.5"> -<a name="subsec:publicity"> -9.5</a> Publicity attracts attention</h3> -</a> - -<div class="p"><!----></div> -Many people working on this field want to publicize the existence -and extent of censorship concurrently with the deployment of their -circumvention software. The easy reason for this two-pronged push is -to attract volunteers for running proxies in their systems; but in many -cases their main goal is not to focus on actually allowing individuals -to circumvent the firewall, but rather to educate the world about the -censorship. The media also tries to do its part by broadcasting the -existence of each new circumvention system. - -<div class="p"><!----></div> -But at the same time, this publicity attracts the attention of the -censors. We can slow down the arms race by not attracting as much -attention, and just spreading by word of mouth. If our goal is to -establish a solid social network of bridges and bridge users before -the adversary gets involved, does this extra attention work to our -disadvantage? - -<div class="p"><!----></div> - <h3><a name="tth_sEc9.6"> -9.6</a> The Tor website: how to get the software</h3> - -<div class="p"><!----></div> -One of the first censoring attacks against a system like ours is to -block the website and make the software itself hard to find. Our system -should work well once the user is running an authentic -copy of Tor and has found a working bridge, but to get to that point -we rely on their individual skills and ingenuity. - -<div class="p"><!----></div> -Right now, most countries that block access to Tor block only the main -website and leave mirrors and the network itself untouched. -Falling back on word-of-mouth is always a good last resort, but we should -also take steps to make sure it's relatively easy for users to get a copy, -such as publicizing the mirrors more and making copies available through -other media. We might also mirror the latest version of the software on -each bridge, so users who hear about an honest bridge can get a good -copy. -See Section <a href="#subsec:first-bridge">7.1</a> for more discussion. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc10"> -<a name="sec:future"> -10</a> Future designs</h2> -</a> - -<div class="p"><!----></div> - <h3><a name="tth_sEc10.1"> -10.1</a> Bridges inside the blocked network too</h3> - -<div class="p"><!----></div> -Assuming actually crossing the firewall is the risky part of the -operation, can we have some bridge relays inside the blocked area too, -and more established users can use them as relays so they don't need to -communicate over the firewall directly at all? A simple example here is -to make new blocked users into internal bridges also — so they sign up -on the bridge authority as part of doing their query, and we give out -their addresses -rather than (or along with) the external bridge addresses. This design -is a lot trickier because it brings in the complexity of whether the -internal bridges will remain available, can maintain reachability with -the outside world, etc. - -<div class="p"><!----></div> -More complex future designs involve operating a separate Tor network -inside the blocked area, and using <em>hidden service bridges</em> — bridges -that can be accessed by users of the internal Tor network but whose -addresses are not published or findable, even by these users — to get -from inside the firewall to the rest of the Internet. But this design -requires directory authorities to run inside the blocked area too, -and they would be a fine target to take down the network. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc11"> -<a name="sec:conclusion"> -11</a> Next Steps</h2> -</a> - -<div class="p"><!----></div> -Technical solutions won't solve the whole censorship problem. After all, -the firewalls in places like China are <em>socially</em> very -successful, even if technologies and tricks exist to get around them. -However, having a strong technical solution is still necessary as one -important piece of the puzzle. - -<div class="p"><!----></div> -In this paper, we have shown that Tor provides a great set of building -blocks to start from. The next steps are to deploy prototype bridges and -bridge authorities, implement some of the proposed discovery strategies, -and then observe the system in operation and get more intuition about -the actual requirements and adversaries we're up against. - -<div class="p"><!----></div> - -<h2>References</h2> - -<dl compact="compact"> - <dt><a href="#CITEeconymics" name="econymics">[1]</a></dt><dd> -Alessandro Acquisti, Roger Dingledine, and Paul Syverson. - On the economics of anonymity. - In Rebecca N. Wright, editor, <em>Financial Cryptography</em>. - Springer-Verlag, LNCS 2742, 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEfreedom21-security" name="freedom21-security">[2]</a></dt><dd> -Adam Back, Ian Goldberg, and Adam Shostack. - Freedom systems 2.1 security issues and analysis. - White paper, Zero Knowledge Systems, Inc., May 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEweb-mix" name="web-mix">[3]</a></dt><dd> -Oliver Berthold, Hannes Federrath, and Stefan Köpsell. - Web MIXes: A system for anonymous and unobservable Internet - access. - In H. Federrath, editor, <em>Designing Privacy Enhancing - Technologies: Workshop on Design Issue in Anonymity and Unobservability</em>. - Springer-Verlag, LNCS 2009, 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEpet05-bissias" name="pet05-bissias">[4]</a></dt><dd> -George Dean Bissias, Marc Liberatore, and Brian Neil Levine. - Privacy vulnerabilities in encrypted http streams. - In <em>Proceedings of Privacy Enhancing Technologies workshop (PET - 2005)</em>, May 2005. - - <a href="http://prisms.cs.umass.edu/brian/pubs/bissias.liberatore.pet.2005.pdf"><tt>http://prisms.cs.umass.edu/brian/pubs/bissias.liberatore.pet.2005.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEchaum-blind" name="chaum-blind">[5]</a></dt><dd> -David Chaum. - Blind signatures for untraceable payments. - In D. Chaum, R.L. Rivest, and A.T. Sherman, editors, <em>Advances in - Cryptology: Proceedings of Crypto 82</em>, pages 199-203. Plenum Press, 1983. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEfreenet-pets00" name="freenet-pets00">[6]</a></dt><dd> -Ian Clarke, Oskar Sandberg, Brandon Wiley, and Theodore W. Hong. - Freenet: A distributed anonymous information storage and retrieval - system. - In H. Federrath, editor, <em>Designing Privacy Enhancing - Technologies: Workshop on Design Issue in Anonymity and Unobservability</em>, - pages 46-66. Springer-Verlag, LNCS 2009, July 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEclayton:pet2006" name="clayton:pet2006">[7]</a></dt><dd> -Richard Clayton, Steven J. Murdoch, and Robert N. M. Watson. - Ignoring the great firewall of china. - In <em>Proceedings of the Sixth Workshop on Privacy Enhancing - Technologies (PET 2006)</em>, Cambridge, UK, June 2006. Springer. - <a href="http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf"><tt>http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEdanezis:pet2004" name="danezis:pet2004">[8]</a></dt><dd> -George Danezis. - The traffic analysis of continuous-time mixes. - In David Martin and Andrei Serjantov, editors, <em>Privacy Enhancing - Technologies (PET 2004)</em>, LNCS, May 2004. - <a href="http://www.cl.cam.ac.uk/users/gd216/cmm2.pdf"><tt>http://www.cl.cam.ac.uk/users/gd216/cmm2.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEusability:weis2006" name="usability:weis2006">[9]</a></dt><dd> -Roger Dingledine and Nick Mathewson. - Anonymity loves company: Usability and the network effect. - In <em>Proceedings of the Fifth Workshop on the Economics of - Information Security (WEIS 2006)</em>, Cambridge, UK, June 2006. - <a href="http://freehaven.net/doc/wupss04/usability.pdf"><tt>http://freehaven.net/doc/wupss04/usability.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITErep-anon" name="rep-anon">[10]</a></dt><dd> -Roger Dingledine, Nick Mathewson, and Paul Syverson. - Reputation in P2P Anonymity Systems. - In <em>Proceedings of Workshop on Economics of Peer-to-Peer - Systems</em>, June 2003. - <a href="http://freehaven.net/doc/econp2p03/econp2p03.pdf"><tt>http://freehaven.net/doc/econp2p03/econp2p03.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEtor-design" name="tor-design">[11]</a></dt><dd> -Roger Dingledine, Nick Mathewson, and Paul Syverson. - Tor: The second-generation onion router. - In <em>Proceedings of the 13th USENIX Security Symposium</em>, August - 2004. - <a href="http://tor.eff.org/tor-design.pdf"><tt>http://tor.eff.org/tor-design.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEcasc-rep" name="casc-rep">[12]</a></dt><dd> -Roger Dingledine and Paul Syverson. - Reliable MIX Cascade Networks through Reputation. - In Matt Blaze, editor, <em>Financial Cryptography</em>. Springer-Verlag, - LNCS 2357, 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEpsiphon" name="psiphon">[13]</a></dt><dd> -Ronald Deibert et al. - Psiphon. - <a href="http://psiphon.civisec.org/"><tt>http://psiphon.civisec.org/</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEinfranet" name="infranet">[14]</a></dt><dd> -Nick Feamster, Magdalena Balazinska, Greg Harfst, Hari Balakrishnan, and David - Karger. - Infranet: Circumventing web censorship and surveillance. - In <em>Proceedings of the 11th USENIX Security Symposium</em>, August - 2002. - <a href="http://nms.lcs.mit.edu/~feamster/papers/usenixsec2002.pdf"><tt>http://nms.lcs.mit.edu/~feamster/papers/usenixsec2002.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEactive-wardens" name="active-wardens">[15]</a></dt><dd> -Gina Fisk, Mike Fisk, Christos Papadopoulos, and Joshua Neil. - Eliminating steganography in internet traffic with active wardens. - In Fabien Petitcolas, editor, <em>Information Hiding Workshop (IH - 2002)</em>. Springer-Verlag, LNCS 2578, October 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEblossom-thesis" name="blossom-thesis">[16]</a></dt><dd> -Geoffrey Goodell. - <em>Perspective Access Networks</em>. - PhD thesis, Harvard University, July 2006. - <a href="http://afs.eecs.harvard.edu/~goodell/thesis.pdf"><tt>http://afs.eecs.harvard.edu/~goodell/thesis.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEgoodell-syverson06" name="goodell-syverson06">[17]</a></dt><dd> -Geoffrey Goodell and Paul Syverson. - The right place at the right time: The use of network location in - authentication and abuse prevention, 2006. - Submitted. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEcircumventor" name="circumventor">[18]</a></dt><dd> -Bennett Haselton. - How to install the Circumventor program. - - <a href="http://www.peacefire.org/circumventor/simple-circumventor-instructions.html"><tt>http://www.peacefire.org/circumventor/simple-circumventor-instructions.html</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEip-to-country" name="ip-to-country">[19]</a></dt><dd> -Ip-to-country database. - <a href="http://ip-to-country.webhosting.info/"><tt>http://ip-to-country.webhosting.info/</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEkoepsell:wpes2004" name="koepsell:wpes2004">[20]</a></dt><dd> -Stefan Köpsell and Ulf Hilling. - How to achieve blocking resistance for existing systems enabling - anonymous web surfing. - In <em>Proceedings of the Workshop on Privacy in the Electronic - Society (WPES 2004)</em>, Washington, DC, USA, October 2004. - <a href="http://freehaven.net/anonbib/papers/p103-koepsell.pdf"><tt>http://freehaven.net/anonbib/papers/p103-koepsell.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEdefensive-dropping" name="defensive-dropping">[21]</a></dt><dd> -Brian N. Levine, Michael K. Reiter, Chenxi Wang, and Matthew Wright. - Timing analysis in low-latency mix-based systems. - In Ari Juels, editor, <em>Financial Cryptography</em>. Springer-Verlag, - LNCS (forthcoming), 2004. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEmackinnon-personal" name="mackinnon-personal">[22]</a></dt><dd> -Rebecca MacKinnon. - Private communication, 2006. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEcgiproxy" name="cgiproxy">[23]</a></dt><dd> -James Marshall. - CGIProxy: HTTP/FTP Proxy in a CGI Script. - <a href="http://www.jmarshall.com/tools/cgiproxy/"><tt>http://www.jmarshall.com/tools/cgiproxy/</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEe2e-traffic" name="e2e-traffic">[24]</a></dt><dd> -Nick Mathewson and Roger Dingledine. - Practical traffic analysis: Extending and resisting statistical - disclosure. - In David Martin and Andrei Serjantov, editors, <em>Privacy Enhancing - Technologies (PET 2004)</em>, LNCS, May 2004. - <a href="http://freehaven.net/doc/e2e-traffic/e2e-traffic.pdf"><tt>http://freehaven.net/doc/e2e-traffic/e2e-traffic.pdf</tt></a>. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEattack-tor-oak05" name="attack-tor-oak05">[25]</a></dt><dd> -Steven J. Murdoch and George Danezis. - Low-cost traffic analysis of tor. - In <em>IEEE Symposium on Security and Privacy</em>. IEEE CS, May 2005. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEtcpstego" name="tcpstego">[26]</a></dt><dd> -Steven J. Murdoch and Stephen Lewis. - Embedding covert channels into TCP/IP. - In Mauro Barni, Jordi Herrera-Joancomartí, Stefan Katzenbeisser, - and Fernando Pérez-González, editors, <em>Information Hiding: 7th - International Workshop</em>, volume 3727 of <em>LNCS</em>, pages 247-261, - Barcelona, Catalonia (Spain), June 2005. Springer-Verlag. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEptacek98insertion" name="ptacek98insertion">[27]</a></dt><dd> -Thomas H. Ptacek and Timothy N. Newsham. - Insertion, evasion, and denial of service: Eluding network intrusion - detection. - Technical report, Secure Networks, Inc., Suite 330, 1201 5th Street - S.W, Calgary, Alberta, Canada, T2R-0Y6, 1998. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEzuckerman-threatmodels" name="zuckerman-threatmodels">[28]</a></dt><dd> -Ethan Zuckerman. - We've got to adjust some of our threat models. - <a href="http://www.ethanzuckerman.com/blog/?p=1019"><tt>http://www.ethanzuckerman.com/blog/?p=1019</tt></a>.</dd> -</dl> - - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> -<hr /><h3>Footnotes:</h3> - -<div class="p"><!----></div> -<a name="tthFtNtAAB"></a><a href="#tthFrefAAB"><sup>1</sup></a>So far in places - like China, the authorities mainly go after people who publish materials - and coordinate organized movements [<a href="#mackinnon-personal" name="CITEmackinnon-personal">22</a>]. - If they find that a - user happens to be reading a site that should be blocked, the typical - response is simply to block the site. Of course, even with an encrypted - connection, the adversary may be able to distinguish readers from - publishers by observing whether Alice is mostly downloading bytes or mostly - uploading them — we discuss this issue more in - Section <a href="#subsec:upload-padding">8.2</a>. -<div class="p"><!----></div> -<a name="tthFtNtAAC"></a><a href="#tthFrefAAC"><sup>2</sup></a><a href="http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ\#EntryGuards"><tt>http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#EntryGuards</tt></a> -<div class="p"><!----></div> -<a name="tthFtNtAAD"></a><a href="#tthFrefAAD"><sup>3</sup></a><a href="http://vidalia-project.net/"><tt>http://vidalia-project.net/</tt></a> -<br /><br /><hr /><small>File translated from -T<sub><font size="-1">E</font></sub>X -by <a href="http://hutchinson.belmont.ma.us/tth/"> -T<sub><font size="-1">T</font></sub>H</a>, -version 3.77.<br />On 11 May 2007, 21:49.</small> -</html> - diff --git a/doc/design-paper/blocking.pdf b/doc/design-paper/blocking.pdf Binary files differdeleted file mode 100644 index 1ee0eb0bb..000000000 --- a/doc/design-paper/blocking.pdf +++ /dev/null diff --git a/doc/design-paper/blocking.tex b/doc/design-paper/blocking.tex deleted file mode 100644 index 3b7d05ca5..000000000 --- a/doc/design-paper/blocking.tex +++ /dev/null @@ -1,1894 +0,0 @@ -%\documentclass{llncs} -\documentclass{usenixsubmit} -%\documentclass[twocolumn]{article} -%usepackage{usenix} - -\usepackage{url} -\usepackage{amsmath} -\usepackage{epsfig} - -\setlength{\textwidth}{6.0in} -\setlength{\textheight}{8.5in} -\setlength{\topmargin}{.5cm} -\setlength{\oddsidemargin}{1cm} -\setlength{\evensidemargin}{1cm} - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} - -\newcommand{\workingnote}[1]{} % The version that hides the note. -%\newcommand{\workingnote}[1]{(**#1)} % makes the note visible. - -\date{} -\title{Design of a blocking-resistant anonymity system\\DRAFT} - -%\author{Roger Dingledine\inst{1} \and Nick Mathewson\inst{1}} -\author{Roger Dingledine \\ The Tor Project \\ arma@torproject.org \and -Nick Mathewson \\ The Tor Project \\ nickm@torproject.org} - -\begin{document} -\maketitle -\pagestyle{plain} - -\begin{abstract} - -Internet censorship is on the rise as websites around the world are -increasingly blocked by government-level firewalls. Although popular -anonymizing networks like Tor were originally designed to keep attackers from -tracing people's activities, many people are also using them to evade local -censorship. But if the censor simply denies access to the Tor network -itself, blocked users can no longer benefit from the security Tor offers. - -Here we describe a design that builds upon the current Tor network -to provide an anonymizing network that resists blocking -by government-level attackers. We have implemented and deployed this -design, and talk briefly about early use. - -\end{abstract} - -\section{Introduction} - -Anonymizing networks like Tor~\cite{tor-design} bounce traffic around a -network of encrypting relays. Unlike encryption, which hides only {\it what} -is said, these networks also aim to hide who is communicating with whom, which -users are using which websites, and so on. These systems have a -broad range of users, including ordinary citizens who want to avoid being -profiled for targeted advertisements, corporations who don't want to reveal -information to their competitors, and law enforcement and government -intelligence agencies who need to do operations on the Internet without being -noticed. - -Historical anonymity research has focused on an -attacker who monitors the user (call her Alice) and tries to discover her -activities, yet lets her reach any piece of the network. In more modern -threat models such as Tor's, the adversary is allowed to perform active -attacks such as modifying communications to trick Alice -into revealing her destination, or intercepting some connections -to run a man-in-the-middle attack. But these systems still assume that -Alice can eventually reach the anonymizing network. - -An increasing number of users are using the Tor software -less for its anonymity properties than for its censorship -resistance properties---if they use Tor to access Internet sites like -Wikipedia -and Blogspot, they are no longer affected by local censorship -and firewall rules. In fact, an informal user study -%(described in Appendix~\ref{app:geoip}) -showed that a few hundred thousand users people access the Tor network -each day, with about 20\% of them coming from China~\cite{something}. - -The current Tor design is easy to block if the attacker controls Alice's -connection to the Tor network---by blocking the directory authorities, -by blocking all the relay IP addresses in the directory, or by filtering -based on the network fingerprint of the Tor TLS handshake. Here we -describe an -extended design that builds upon the current Tor network to provide an -anonymizing -network that resists censorship as well as anonymity-breaking attacks. -In section~\ref{sec:adversary} we discuss our threat model---that is, -the assumptions we make about our adversary. Section~\ref{sec:current-tor} -describes the components of the current Tor design and how they can be -leveraged for a new blocking-resistant design. Section~\ref{sec:related} -explains the features and drawbacks of the currently deployed solutions. -In sections~\ref{sec:bridges} through~\ref{sec:discovery}, we explore the -components of our designs in detail. Section~\ref{sec:security} considers -security implications and Section~\ref{sec:reachability} presents other -issues with maintaining connectivity and sustainability for the design. -%Section~\ref{sec:future} speculates about future more complex designs, -Finally section~\ref{sec:conclusion} summarizes our next steps and -recommendations. - -% The other motivation is for places where we're concerned they will -% try to enumerate a list of Tor users. So even if they're not blocking -% the Tor network, it may be smart to not be visible as connecting to it. - -%And adding more different classes of users and goals to the Tor network -%improves the anonymity for all Tor users~\cite{econymics,usability:weis2006}. - -% Adding use classes for countering blocking as well as anonymity has -% benefits too. Should add something about how providing undetected -% access to Tor would facilitate people talking to, e.g., govt. authorities -% about threats to public safety etc. in an environment where Tor use -% is not otherwise widespread and would make one stand out. - -\section{Adversary assumptions} -\label{sec:adversary} - -To design an effective anti-censorship tool, we need a good model for the -goals and resources of the censors we are evading. Otherwise, we risk -spending our effort on keeping the adversaries from doing things they have no -interest in doing, and thwarting techniques they do not use. -The history of blocking-resistance designs is littered with conflicting -assumptions about what adversaries to expect and what problems are -in the critical path to a solution. Here we describe our best -understanding of the current situation around the world. - -In the traditional security style, we aim to defeat a strong -attacker---if we can defend against this attacker, we inherit protection -against weaker attackers as well. After all, we want a general design -that will work for citizens of China, Thailand, and other censored -countries; for -whistleblowers in firewalled corporate networks; and for people in -unanticipated oppressive situations. In fact, by designing with -a variety of adversaries in mind, we can take advantage of the fact that -adversaries will be in different stages of the arms race at each location, -so an address blocked in one locale can still be useful in others. -We focus on an attacker with somewhat complex goals: - -\begin{tightlist} -\item The attacker would like to restrict the flow of certain kinds of - information, particularly when this information is seen as embarrassing to - those in power (such as information about rights violations or corruption), - or when it enables or encourages others to oppose them effectively (such as - information about opposition movements or sites that are used to organize - protests). -\item As a second-order effect, censors aim to chill citizens' behavior by - creating an impression that their online activities are monitored. -\item In some cases, censors make a token attempt to block a few sites for - obscenity, blasphemy, and so on, but their efforts here are mainly for - show. In other cases, they really do try hard to block such content. -\item Complete blocking (where nobody at all can ever download censored - content) is not a - goal. Attackers typically recognize that perfect censorship is not only - impossible, it is unnecessary: if ``undesirable'' information is known only - to a small few, further censoring efforts can be focused elsewhere. -\item Similarly, the censors do not attempt to shut down or block {\it - every} anti-censorship tool---merely the tools that are popular and - effective (because these tools impede the censors' information restriction - goals) and those tools that are highly visible (thus making the censors - look ineffectual to their citizens and their bosses). -\item Reprisal against {\it most} passive consumers of {\it most} kinds of - blocked information is also not a goal, given the broadness of most - censorship regimes. This seems borne out by fact.\footnote{So far in places - like China, the authorities mainly go after people who publish materials - and coordinate organized movements~\cite{mackinnon-personal}. - If they find that a - user happens to be reading a site that should be blocked, the typical - response is simply to block the site. Of course, even with an encrypted - connection, the adversary may be able to distinguish readers from - publishers by observing whether Alice is mostly downloading bytes or mostly - uploading them---we discuss this issue more in - Section~\ref{subsec:upload-padding}.} -\item Producers and distributors of targeted information are in much - greater danger than consumers; the attacker would like to not only block - their work, but identify them for reprisal. -\item The censors (or their governments) would like to have a working, useful - Internet. There are economic, political, and social factors that prevent - them from ``censoring'' the Internet by outlawing it entirely, or by - blocking access to all but a tiny list of sites. - Nevertheless, the censors {\it are} willing to block innocuous content - (like the bulk of a newspaper's reporting) in order to censor other content - distributed through the same channels (like that newspaper's coverage of - the censored country). -\end{tightlist} - -We assume there are three main technical network attacks in use by censors -currently~\cite{clayton:pet2006}: - -\begin{tightlist} -\item Block a destination or type of traffic by automatically searching for - certain strings or patterns in TCP packets. Offending packets can be - dropped, or can trigger a response like closing the - connection. -\item Block certain IP addresses or destination ports at a - firewall or other routing control point. -\item Intercept DNS requests and give bogus responses for certain - destination hostnames. -\end{tightlist} - -We assume the network firewall has limited CPU and memory per -connection~\cite{clayton:pet2006}. Against an adversary who could carefully -examine the contents of every packet and correlate the packets in every -stream on the network, we would need some stronger mechanism such as -steganography, which introduces its own -problems~\cite{active-wardens,tcpstego}. But we make a ``weak -steganography'' assumption here: to remain unblocked, it is necessary to -remain unobservable only by computational resources on par with a modern -router, firewall, proxy, or IDS. - -We assume that while various different regimes can coordinate and share -notes, there will be a time lag between one attacker learning how to overcome -a facet of our design and other attackers picking it up. (The most common -vector of transmission seems to be commercial providers of censorship tools: -once a provider adds a feature to meet one country's needs or requests, the -feature is available to all of the provider's customers.) Conversely, we -assume that insider attacks become a higher risk only after the early stages -of network development, once the system has reached a certain level of -success and visibility. - -We do not assume that government-level attackers are always uniform -across the country. For example, users of different ISPs in China -experience different censorship policies and mechanisms~\cite{china-ccs07}. -%there is no single centralized place in China -%that coordinates its specific censorship decisions and steps. - -We assume that the attacker may be able to use political and economic -resources to secure the cooperation of extraterritorial or multinational -corporations and entities in investigating information sources. -For example, the censors can threaten the service providers of -troublesome blogs with economic reprisals if they do not reveal the -authors' identities. - -We assume that our users have control over their hardware and -software---they don't have any spyware installed, there are no -cameras watching their screens, etc. Unfortunately, in many situations -these threats are real~\cite{zuckerman-threatmodels}; yet -software-based security systems like ours are poorly equipped to handle -a user who is entirely observed and controlled by the adversary. See -Section~\ref{subsec:cafes-and-livecds} for more discussion of what little -we can do about this issue. - -Similarly, we assume that the user will be able to fetch a genuine -version of Tor, rather than one supplied by the adversary; see -Section~\ref{subsec:trust-chain} for discussion on helping the user -confirm that he has a genuine version and that he can connect to the -real Tor network. - -\section{Adapting the current Tor design to anti-censorship} -\label{sec:current-tor} - -Tor is popular and sees a lot of use---it's the largest anonymity -network of its kind, and has -attracted more than 1500 volunteer-operated routers from around the -world. Tor protects each user by routing their traffic through a multiply -encrypted ``circuit'' built of a few randomly selected relay, each of which -can remove only a single layer of encryption. Each relay sees only the step -before it and the step after it in the circuit, and so no single relay can -learn the connection between a user and her chosen communication partners. -In this section, we examine some of the reasons why Tor has become popular, -with particular emphasis to how we can take advantage of these properties -for a blocking-resistance design. - -Tor aims to provide three security properties: -\begin{tightlist} -\item 1. A local network attacker can't learn, or influence, your -destination. -\item 2. No single router in the Tor network can link you to your -destination. -\item 3. The destination, or somebody watching the destination, -can't learn your location. -\end{tightlist} - -For blocking-resistance, we care most clearly about the first -property. But as the arms race progresses, the second property -will become important---for example, to discourage an adversary -from volunteering a relay in order to learn that Alice is reading -or posting to certain websites. The third property helps keep users safe from -collaborating websites: consider websites and other Internet services -that have been pressured -recently into revealing the identity of bloggers -%~\cite{arrested-bloggers} -or treating clients differently depending on their network -location~\cite{netauth}. -%~\cite{google-geolocation}. - -The Tor design provides other features as well that are not typically -present in manual or ad hoc circumvention techniques. - -First, Tor has a well-analyzed and well-understood way to distribute -information about relay. -Tor directory authorities automatically aggregate, test, -and publish signed summaries of the available Tor routers. Tor clients -can fetch these summaries to learn which routers are available and -which routers are suitable for their needs. Directory information is cached -throughout the Tor network, so once clients have bootstrapped they never -need to interact with the authorities directly. (To tolerate a minority -of compromised directory authorities, we use a threshold trust scheme--- -see Section~\ref{subsec:trust-chain} for details.) - -Second, the list of directory authorities is not hard-wired. -Clients use the default authorities if no others are specified, -but it's easy to start a separate (or even overlapping) Tor network just -by running a different set of authorities and convincing users to prefer -a modified client. For example, we could launch a distinct Tor network -inside China; some users could even use an aggregate network made up of -both the main network and the China network. (But we should not be too -quick to create other Tor networks---part of Tor's anonymity comes from -users behaving like other users, and there are many unsolved anonymity -questions if different users know about different pieces of the network.) - -Third, in addition to automatically learning from the chosen directories -which Tor routers are available and working, Tor takes care of building -paths through the network and rebuilding them as needed. So the user -never has to know how paths are chosen, never has to manually pick -working proxies, and so on. More generally, at its core the Tor protocol -is simply a tool that can build paths given a set of routers. Tor is -quite flexible about how it learns about the routers and how it chooses -the paths. Harvard's Blossom project~\cite{blossom-thesis} makes this -flexibility more concrete: Blossom makes use of Tor not for its security -properties but for its reachability properties. It runs a separate set -of directory authorities, its own set of Tor routers (called the Blossom -network), and uses Tor's flexible path-building to let users view Internet -resources from any point in the Blossom network. - -Fourth, Tor separates the role of \emph{internal relay} from the -role of \emph{exit relay}. That is, some volunteers choose just to relay -traffic between Tor users and Tor routers, and others choose to also allow -connections to external Internet resources. Because we don't force all -volunteers to play both roles, we end up with more relays. This increased -diversity in turn is what gives Tor its security: the more options the -user has for her first hop, and the more options she has for her last hop, -the less likely it is that a given attacker will be watching both ends -of her circuit~\cite{tor-design}. As a bonus, because our design attracts -more internal relays that want to help out but don't want to deal with -being an exit relay, we end up providing more options for the first -hop---the one most critical to being able to reach the Tor network. - -Fifth, Tor is sustainable. Zero-Knowledge Systems offered the commercial -but now defunct Freedom Network~\cite{freedom21-security}, a design with -security comparable to Tor's, but its funding model relied on collecting -money from users to pay relay operators. Modern commercial proxy systems -similarly -need to keep collecting money to support their infrastructure. On the -other hand, Tor has built a self-sustaining community of volunteers who -donate their time and resources. This community trust is rooted in Tor's -open design: we tell the world exactly how Tor works, and we provide all -the source code. Users can decide for themselves, or pay any security -expert to decide, whether it is safe to use. Further, Tor's modularity -as described above, along with its open license, mean that its impact -will continue to grow. - -Sixth, Tor has an established user base of hundreds of -thousands of people from around the world. This diversity of -users contributes to sustainability as above: Tor is used by -ordinary citizens, activists, corporations, law enforcement, and -even government and military users, -%\footnote{\url{https://www.torproject.org/overview}} -and they can -only achieve their security goals by blending together in the same -network~\cite{econymics,usability:weis2006}. This user base also provides -something else: hundreds of thousands of different and often-changing -addresses that we can leverage for our blocking-resistance design. - -Finally and perhaps most importantly, Tor provides anonymity and prevents any -single relay from linking users to their communication partners. Despite -initial appearances, {\it distributed-trust anonymity is critical for -anti-censorship efforts}. If any single relay can expose dissident bloggers -or compile a list of users' behavior, the censors can profitably compromise -that relay's operator, perhaps by applying economic pressure to their -employers, -breaking into their computer, pressuring their family (if they have relatives -in the censored area), or so on. Furthermore, in designs where any relay can -expose its users, the censors can spread suspicion that they are running some -of the relays and use this belief to chill use of the network. - -We discuss and adapt these components further in -Section~\ref{sec:bridges}. But first we examine the strengths and -weaknesses of other blocking-resistance approaches, so we can expand -our repertoire of building blocks and ideas. - -\section{Current proxy solutions} -\label{sec:related} - -Relay-based blocking-resistance schemes generally have two main -components: a relay component and a discovery component. The relay part -encompasses the process of establishing a connection, sending traffic -back and forth, and so on---everything that's done once the user knows -where she's going to connect. Discovery is the step before that: the -process of finding one or more usable relays. - -For example, we can divide the pieces of Tor in the previous section -into the process of building paths and sending -traffic over them (relay) and the process of learning from the directory -authorities about what routers are available (discovery). With this -distinction -in mind, we now examine several categories of relay-based schemes. - -\subsection{Centrally-controlled shared proxies} - -Existing commercial anonymity solutions (like Anonymizer.com) are based -on a set of single-hop proxies. In these systems, each user connects to -a single proxy, which then relays traffic between the user and her -destination. These public proxy -systems are typically characterized by two features: they control and -operate the proxies centrally, and many different users get assigned -to each proxy. - -In terms of the relay component, single proxies provide weak security -compared to systems that distribute trust over multiple relays, since a -compromised proxy can trivially observe all of its users' actions, and -an eavesdropper only needs to watch a single proxy to perform timing -correlation attacks against all its users' traffic and thus learn where -everyone is connecting. Worse, all users -need to trust the proxy company to have good security itself as well as -to not reveal user activities. - -On the other hand, single-hop proxies are easier to deploy, and they -can provide better performance than distributed-trust designs like Tor, -since traffic only goes through one relay. They're also more convenient -from the user's perspective---since users entirely trust the proxy, -they can just use their web browser directly. - -Whether public proxy schemes are more or less scalable than Tor is -still up for debate: commercial anonymity systems can use some of their -revenue to provision more bandwidth as they grow, whereas volunteer-based -anonymity systems can attract thousands of fast relays to spread the load. - -The discovery piece can take several forms. Most commercial anonymous -proxies have one or a handful of commonly known websites, and their users -log in to those websites and relay their traffic through them. When -these websites get blocked (generally soon after the company becomes -popular), if the company cares about users in the blocked areas, they -start renting lots of disparate IP addresses and rotating through them -as they get blocked. They notify their users of new addresses (by email, -for example). It's an arms race, since attackers can sign up to receive the -email too, but operators have one nice trick available to them: because they -have a list of paying subscribers, they can notify certain subscribers -about updates earlier than others. - -Access control systems on the proxy let them provide service only to -users with certain characteristics, such as paying customers or people -from certain IP address ranges. - -Discovery in the face of a government-level firewall is a complex and -unsolved -topic, and we're stuck in this same arms race ourselves; we explore it -in more detail in Section~\ref{sec:discovery}. But first we examine the -other end of the spectrum---getting volunteers to run the proxies, -and telling only a few people about each proxy. - -\subsection{Independent personal proxies} - -Personal proxies such as Circumventor~\cite{circumventor} and -CGIProxy~\cite{cgiproxy} use the same technology as the public ones as -far as the relay component goes, but they use a different strategy for -discovery. Rather than managing a few centralized proxies and constantly -getting new addresses for them as the old addresses are blocked, they -aim to have a large number of entirely independent proxies, each managing -its own (much smaller) set of users. - -As the Circumventor site explains, ``You don't -actually install the Circumventor \emph{on} the computer that is blocked -from accessing Web sites. You, or a friend of yours, has to install the -Circumventor on some \emph{other} machine which is not censored.'' - -This tactic has great advantages in terms of blocking-resistance---recall -our assumption in Section~\ref{sec:adversary} that the attention -a system attracts from the attacker is proportional to its number of -users and level of publicity. If each proxy only has a few users, and -there is no central list of proxies, most of them will never get noticed by -the censors. - -On the other hand, there's a huge scalability question that so far has -prevented these schemes from being widely useful: how does the fellow -in China find a person in Ohio who will run a Circumventor for him? In -some cases he may know and trust some people on the outside, but in many -cases he's just out of luck. Just as hard, how does a new volunteer in -Ohio find a person in China who needs it? - -% another key feature of a proxy run by your uncle is that you -% self-censor, so you're unlikely to bring abuse complaints onto -% your uncle. self-censoring clearly has a downside too, though. - -This challenge leads to a hybrid design---centrally-distributed -personal proxies---which we will investigate in more detail in -Section~\ref{sec:discovery}. - -\subsection{Open proxies} - -Yet another currently used approach to bypassing firewalls is to locate -open and misconfigured proxies on the Internet. A quick Google search -for ``open proxy list'' yields a wide variety of freely available lists -of HTTP, HTTPS, and SOCKS proxies. Many small companies have sprung up -providing more refined lists to paying customers. - -There are some downsides to using these open proxies though. First, -the proxies are of widely varying quality in terms of bandwidth and -stability, and many of them are entirely unreachable. Second, unlike -networks of volunteers like Tor, the legality of routing traffic through -these proxies is questionable: it's widely believed that most of them -don't realize what they're offering, and probably wouldn't allow it if -they realized. Third, in many cases the connection to the proxy is -unencrypted, so firewalls that filter based on keywords in IP packets -will not be hindered. Fourth, in many countries (including China), the -firewall authorities hunt for open proxies as well, to preemptively -block them. And last, many users are suspicious that some -open proxies are a little \emph{too} convenient: are they run by the -adversary, in which case they get to monitor all the user's requests -just as single-hop proxies can? - -A distributed-trust design like Tor resolves each of these issues for -the relay component, but a constantly changing set of thousands of open -relays is clearly a useful idea for a discovery component. For example, -users might be able to make use of these proxies to bootstrap their -first introduction into the Tor network. - -\subsection{Blocking resistance and JAP} - -K\"{o}psell and Hilling's Blocking Resistance -design~\cite{koepsell:wpes2004} is probably -the closest related work, and is the starting point for the design in this -paper. In this design, the JAP anonymity system~\cite{web-mix} is used -as a base instead of Tor. Volunteers operate a large number of access -points that relay traffic to the core JAP -network, which in turn anonymizes users' traffic. The software to run these -relays is, as in our design, included in the JAP client software and enabled -only when the user decides to enable it. Discovery is handled with a -CAPTCHA-based mechanism; users prove that they aren't an automated process, -and are given the address of an access point. (The problem of a determined -attacker with enough manpower to launch many requests and enumerate all the -access points is not considered in depth.) There is also some suggestion -that information about access points could spread through existing social -networks. - -\subsection{Infranet} - -The Infranet design~\cite{infranet} uses one-hop relays to deliver web -content, but disguises its communications as ordinary HTTP traffic. Requests -are split into multiple requests for URLs on the relay, which then encodes -its responses in the content it returns. The relay needs to be an actual -website with plausible content and a number of URLs which the user might want -to access---if the Infranet software produced its own cover content, it would -be far easier for censors to identify. To keep the censors from noticing -that cover content changes depending on what data is embedded, Infranet needs -the cover content to have an innocuous reason for changing frequently: the -paper recommends watermarked images and webcams. - -The attacker and relay operators in Infranet's threat model are significantly -different than in ours. Unlike our attacker, Infranet's censor can't be -bypassed with encrypted traffic (presumably because the censor blocks -encrypted traffic, or at least considers it suspicious), and has more -computational resources to devote to each connection than ours (so it can -notice subtle patterns over time). Unlike our bridge operators, Infranet's -operators (and users) have more bandwidth to spare; the overhead in typical -steganography schemes is far higher than Tor's. - -The Infranet design does not include a discovery element. Discovery, -however, is a critical point: if whatever mechanism allows users to learn -about relays also allows the censor to do so, he can trivially discover and -block their addresses, even if the steganography would prevent mere traffic -observation from revealing the relays' addresses. - -\subsection{RST-evasion and other packet-level tricks} - -In their analysis of China's firewall's content-based blocking, Clayton, -Murdoch and Watson discovered that rather than blocking all packets in a TCP -streams once a forbidden word was noticed, the firewall was simply forging -RST packets to make the communicating parties believe that the connection was -closed~\cite{clayton:pet2006}. They proposed altering operating systems -to ignore forged RST packets. This approach might work in some cases, but -in practice it appears that many firewalls start filtering by IP address -once a sufficient number of RST packets have been sent. - -Other packet-level responses to filtering include splitting -sensitive words across multiple TCP packets, so that the censors' -firewalls can't notice them without performing expensive stream -reconstruction~\cite{ptacek98insertion}. This technique relies on the -same insight as our weak steganography assumption. - -%\subsection{Internal caching networks} - -%Freenet~\cite{freenet-pets00} is an anonymous peer-to-peer data store. -%Analyzing Freenet's security can be difficult, as its design is in flux as -%new discovery and routing mechanisms are proposed, and no complete -%specification has (to our knowledge) been written. Freenet servers relay -%requests for specific content (indexed by a digest of the content) -%``toward'' the server that hosts it, and then cache the content as it -%follows the same path back to -%the requesting user. If Freenet's routing mechanism is successful in -%allowing nodes to learn about each other and route correctly even as some -%node-to-node links are blocked by firewalls, then users inside censored areas -%can ask a local Freenet server for a piece of content, and get an answer -%without having to connect out of the country at all. Of course, operators of -%servers inside the censored area can still be targeted, and the addresses of -%external servers can still be blocked. - -%\subsection{Skype} - -%The popular Skype voice-over-IP software uses multiple techniques to tolerate -%restrictive networks, some of which allow it to continue operating in the -%presence of censorship. By switching ports and using encryption, Skype -%attempts to resist trivial blocking and content filtering. Even if no -%encryption were used, it would still be expensive to scan all voice -%traffic for sensitive words. Also, most current keyloggers are unable to -%store voice traffic. Nevertheless, Skype can still be blocked, especially at -%its central login server. - -%*sjmurdoch* "we consider the login server to be the only central component in -%the Skype p2p network." -%*sjmurdoch* http://www1.cs.columbia.edu/~salman/publications/skype1_4.pdf -%-> *sjmurdoch* ok. what is the login server's role? -%-> *sjmurdoch* and do you need to reach it directly to use skype? -%*sjmurdoch* It checks the username and password -%*sjmurdoch* It is necessary in the current implementation, but I don't know if -%it is a fundemental limitation of the architecture - -\subsection{Tor itself} - -And last, we include Tor itself in the list of current solutions -to firewalls. Tens of thousands of people use Tor from countries that -routinely filter their Internet. Tor's website has been blocked in most -of them. But why hasn't the Tor network been blocked yet? - -We have several theories. The first is the most straightforward: tens of -thousands of people are simply too few to matter. It may help that Tor is -perceived to be for experts only, and thus not worth attention yet. The -more subtle variant on this theory is that we've positioned Tor in the -public eye as a tool for retaining civil liberties in more free countries, -so perhaps blocking authorities don't view it as a threat. (We revisit -this idea when we consider whether and how to publicize a Tor variant -that improves blocking-resistance---see Section~\ref{subsec:publicity} -for more discussion.) - -The broader explanation is that the maintenance of most government-level -filters is aimed at stopping widespread information flow and appearing to be -in control, not by the impossible goal of blocking all possible ways to bypass -censorship. Censors realize that there will always -be ways for a few people to get around the firewall, and as long as Tor -has not publically threatened their control, they see no urgent need to -block it yet. - -We should recognize that we're \emph{already} in the arms race. These -constraints can give us insight into the priorities and capabilities of -our various attackers. - -\section{The relay component of our blocking-resistant design} -\label{sec:bridges} - -Section~\ref{sec:current-tor} describes many reasons why Tor is -well-suited as a building block in our context, but several changes will -allow the design to resist blocking better. The most critical changes are -to get more relay addresses, and to distribute them to users differently. - -%We need to address three problems: -%- adapting the relay component of Tor so it resists blocking better. -%- Discovery. -%- Tor's network fingerprint. - -%Here we describe the new pieces we need to add to the current Tor design. - -\subsection{Bridge relays} - -Today, Tor relays operate on a few thousand distinct IP addresses; -an adversary -could enumerate and block them all with little trouble. To provide a -means of ingress to the network, we need a larger set of entry points, most -of which an adversary won't be able to enumerate easily. Fortunately, we -have such a set: the Tor users. - -Hundreds of thousands of people around the world use Tor. We can leverage -our already self-selected user base to produce a list of thousands of -frequently-changing IP addresses. Specifically, we can give them a little -button in the GUI that says ``Tor for Freedom'', and users who click -the button will turn into \emph{bridge relays} (or just \emph{bridges} -for short). They can rate limit relayed connections to 10 KB/s (almost -nothing for a broadband user in a free country, but plenty for a user -who otherwise has no access at all), and since they are just relaying -bytes back and forth between blocked users and the main Tor network, they -won't need to make any external connections to Internet sites. Because -of this separation of roles, and because we're making use of software -that the volunteers have already installed for their own use, we expect -our scheme to attract and maintain more volunteers than previous schemes. - -As usual, there are new anonymity and security implications from running a -bridge relay, particularly from letting people relay traffic through your -Tor client; but we leave this discussion for Section~\ref{sec:security}. - -%...need to outline instructions for a Tor config that will publish -%to an alternate directory authority, and for controller commands -%that will do this cleanly. - -\subsection{The bridge directory authority} - -How do the bridge relays advertise their existence to the world? We -introduce a second new component of the design: a specialized directory -authority that aggregates and tracks bridges. Bridge relays periodically -publish relay descriptors (summaries of their keys, locations, etc, -signed by their long-term identity key), just like the relays in the -``main'' Tor network, but in this case they publish them only to the -bridge directory authorities. - -The main difference between bridge authorities and the directory -authorities for the main Tor network is that the main authorities provide -a list of every known relay, but the bridge authorities only give -out a relay descriptor if you already know its identity key. That is, -you can keep up-to-date on a bridge's location and other information -once you know about it, but you can't just grab a list of all the bridges. - -The identity key, IP address, and directory port for each bridge -authority ship by default with the Tor software, so the bridge relays -can be confident they're publishing to the right location, and the -blocked users can establish an encrypted authenticated channel. See -Section~\ref{subsec:trust-chain} for more discussion of the public key -infrastructure and trust chain. - -Bridges use Tor to publish their descriptors privately and securely, -so even an attacker monitoring the bridge directory authority's network -can't make a list of all the addresses contacting the authority. -Bridges may publish to only a subset of the -authorities, to limit the potential impact of an authority compromise. - - -%\subsection{A simple matter of engineering} -% -%Although we've described bridges and bridge authorities in simple terms -%above, some design modifications and features are needed in the Tor -%codebase to add them. We describe the four main changes here. -% -%Firstly, we need to get smarter about rate limiting: -%Bandwidth classes -% -%Secondly, while users can in fact configure which directory authorities -%they use, we need to add a new type of directory authority and teach -%bridges to fetch directory information from the main authorities while -%publishing relay descriptors to the bridge authorities. We're most of -%the way there, since we can already specify attributes for directory -%authorities: -%add a separate flag named ``blocking''. -% -%Thirdly, need to build paths using bridges as the first -%hop. One more hole in the non-clique assumption. -% -%Lastly, since bridge authorities don't answer full network statuses, -%we need to add a new way for users to learn the current status for a -%single relay or a small set of relays---to answer such questions as -%``is it running?'' or ``is it behaving correctly?'' We describe in -%Section~\ref{subsec:enclave-dirs} a way for the bridge authority to -%publish this information without resorting to signing each answer -%individually. - -\subsection{Putting them together} -\label{subsec:relay-together} - -If a blocked user knows the identity keys of a set of bridge relays, and -he has correct address information for at least one of them, he can use -that one to make a secure connection to the bridge authority and update -his knowledge about the other bridge relays. He can also use it to make -secure connections to the main Tor network and directory authorities, so he -can build circuits and connect to the rest of the Internet. All of these -updates happen in the background: from the blocked user's perspective, -he just accesses the Internet via his Tor client like always. - -So now we've reduced the problem from how to circumvent the firewall -for all transactions (and how to know that the pages you get have not -been modified by the local attacker) to how to learn about a working -bridge relay. - -There's another catch though. We need to make sure that the network -traffic we generate by simply connecting to a bridge relay doesn't stand -out too much. - -%The following section describes ways to bootstrap knowledge of your first -%bridge relay, and ways to maintain connectivity once you know a few -%bridge relays. - -% (See Section~\ref{subsec:first-bridge} for a discussion -%of exactly what information is sufficient to characterize a bridge relay.) - - - -\section{Hiding Tor's network fingerprint} -\label{sec:network-fingerprint} -\label{subsec:enclave-dirs} - -Currently, Tor uses two protocols for its network communications. The -main protocol uses TLS for encrypted and authenticated communication -between Tor instances. The second protocol is standard HTTP, used for -fetching directory information. All Tor relays listen on their ``ORPort'' -for TLS connections, and some of them opt to listen on their ``DirPort'' -as well, to serve directory information. Tor relays choose whatever port -numbers they like; the relay descriptor they publish to the directory -tells users where to connect. - -One format for communicating address information about a bridge relay is -its IP address and DirPort. From there, the user can ask the bridge's -directory cache for an up-to-date copy of its relay descriptor, and -learn its current circuit keys, its ORPort, and so on. - -However, connecting directly to the directory cache involves a plaintext -HTTP request. A censor could create a network fingerprint (known as a -\emph{signature} in the intrusion detection field) for the request -and/or its response, thus preventing these connections. To resolve this -vulnerability, we've modified the Tor protocol so that users can connect -to the directory cache via the main Tor port---they establish a TLS -connection with the bridge as normal, and then send a special ``begindir'' -relay command to establish an internal connection to its directory cache. - -Therefore a better way to summarize a bridge's address is by its IP -address and ORPort, so all communications between the client and the -bridge will use ordinary TLS. But there are other details that need -more investigation. - -What port should bridges pick for their ORPort? We currently recommend -that they listen on port 443 (the default HTTPS port) if they want to -be most useful, because clients behind standard firewalls will have -the best chance to reach them. Is this the best choice in all cases, -or should we encourage some fraction of them pick random ports, or other -ports commonly permitted through firewalls like 53 (DNS) or 110 -(POP)? Or perhaps we should use other ports where TLS traffic is -expected, like 993 (IMAPS) or 995 (POP3S). We need more research on our -potential users, and their current and anticipated firewall restrictions. - -Furthermore, we need to look at the specifics of Tor's TLS handshake. -Right now Tor uses some predictable strings in its TLS handshakes. For -example, it sets the X.509 organizationName field to ``Tor'', and it puts -the Tor relay's nickname in the certificate's commonName field. We -should tweak the handshake protocol so it doesn't rely on any unusual details -in the certificate, yet it remains secure; the certificate itself -should be made to resemble an ordinary HTTPS certificate. We should also try -to make our advertised cipher-suites closer to what an ordinary web server -would support. - -Tor's TLS handshake uses two-certificate chains: one certificate -contains the self-signed identity key for -the router, and the second contains a current TLS key, signed by the -identity key. We use these to authenticate that we're talking to the right -router, and to limit the impact of TLS-key exposure. Most (though far from -all) consumer-oriented HTTPS services provide only a single certificate. -These extra certificates may help identify Tor's TLS handshake; instead, -bridges should consider using only a single TLS key certificate signed by -their identity key, and providing the full value of the identity key in an -early handshake cell. More significantly, Tor currently has all clients -present certificates, so that clients are harder to distinguish from relays. -But in a blocking-resistance environment, clients should not present -certificates at all. - -Last, what if the adversary starts observing the network traffic even -more closely? Even if our TLS handshake looks innocent, our traffic timing -and volume still look different than a user making a secure web connection -to his bank. The same techniques used in the growing trend to build tools -to recognize encrypted Bittorrent traffic -%~\cite{bt-traffic-shaping} -could be used to identify Tor communication and recognize bridge -relays. Rather than trying to look like encrypted web traffic, we may be -better off trying to blend with some other encrypted network protocol. The -first step is to compare typical network behavior for a Tor client to -typical network behavior for various other protocols. This statistical -cat-and-mouse game is made more complex by the fact that Tor transports a -variety of protocols, and we'll want to automatically handle web browsing -differently from, say, instant messaging. - -% Tor cells are 512 bytes each. So TLS records will be roughly -% multiples of this size? How bad is this? -RD -% Look at ``Inferring the Source of Encrypted HTTP Connections'' -% by Marc Liberatore and Brian Neil Levine (CCS 2006) -% They substantially flesh out the numbers for the web fingerprinting -% attack. -PS -% Yes, but I meant detecting the fingerprint of Tor traffic itself, not -% learning what websites we're going to. I wouldn't be surprised to -% learn that these are related problems, but it's not obvious to me. -RD - -\subsection{Identity keys as part of addressing information} -\label{subsec:id-address} - -We have described a way for the blocked user to bootstrap into the -network once he knows the IP address and ORPort of a bridge. What about -local spoofing attacks? That is, since we never learned an identity -key fingerprint for the bridge, a local attacker could intercept our -connection and pretend to be the bridge we had in mind. It turns out -that giving false information isn't that bad---since the Tor client -ships with trusted keys for the bridge directory authority and the Tor -network directory authorities, the user can learn whether he's being -given a real connection to the bridge authorities or not. (After all, -if the adversary intercepts every connection the user makes and gives -him a bad connection each time, there's nothing we can do.) - -What about anonymity-breaking attacks from observing traffic, if the -blocked user doesn't start out knowing the identity key of his intended -bridge? The vulnerabilities aren't so bad in this case either---the -adversary could do similar attacks just by monitoring the network -traffic. -% cue paper by steven and george - -Once the Tor client has fetched the bridge's relay descriptor, it should -remember the identity key fingerprint for that bridge relay. Thus if -the bridge relay moves to a new IP address, the client can query the -bridge directory authority to look up a fresh relay descriptor using -this fingerprint. - -So we've shown that it's \emph{possible} to bootstrap into the network -just by learning the IP address and ORPort of a bridge, but are there -situations where it's more convenient or more secure to learn the bridge's -identity fingerprint as well as instead, while bootstrapping? We keep -that question in mind as we next investigate bootstrapping and discovery. - -\section{Discovering working bridge relays} -\label{sec:discovery} - -Tor's modular design means that we can develop a better relay component -independently of developing the discovery component. This modularity's -great promise is that we can pick any discovery approach we like; but the -unfortunate fact is that we have no magic bullet for discovery. We're -in the same arms race as all the other designs we described in -Section~\ref{sec:related}. - -In this section we describe a variety of approaches to adding discovery -components for our design. - -\subsection{Bootstrapping: finding your first bridge.} -\label{subsec:first-bridge} - -In Section~\ref{subsec:relay-together}, we showed that a user who knows -a working bridge address can use it to reach the bridge authority and -to stay connected to the Tor network. But how do new users reach the -bridge authority in the first place? After all, the bridge authority -will be one of the first addresses that a censor blocks. - -First, we should recognize that most government firewalls are not -perfect. That is, they may allow connections to Google cache or some -open proxy servers, or they let file-sharing traffic, Skype, instant -messaging, or World-of-Warcraft connections through. Different users will -have different mechanisms for bypassing the firewall initially. Second, -we should remember that most people don't operate in a vacuum; users will -hopefully know other people who are in other situations or have other -resources available. In the rest of this section we develop a toolkit -of different options and mechanisms, so that we can enable users in a -diverse set of contexts to bootstrap into the system. - -(For users who can't use any of these techniques, hopefully they know -a friend who can---for example, perhaps the friend already knows some -bridge relay addresses. If they can't get around it at all, then we -can't help them---they should go meet more people or learn more about -the technology running the firewall in their area.) - -By deploying all the schemes in the toolkit at once, we let bridges and -blocked users employ the discovery approach that is most appropriate -for their situation. - -\subsection{Independent bridges, no central discovery} - -The first design is simply to have no centralized discovery component at -all. Volunteers run bridges, and we assume they have some blocked users -in mind and communicate their address information to them out-of-band -(for example, through Gmail). This design allows for small personal -bridges that have only one or a handful of users in mind, but it can -also support an entire community of users. For example, Citizen Lab's -upcoming Psiphon single-hop proxy tool~\cite{psiphon} plans to use this -\emph{social network} approach as its discovery component. - -There are several ways to do bootstrapping in this design. In the simple -case, the operator of the bridge informs each chosen user about his -bridge's address information and/or keys. A different approach involves -blocked users introducing new blocked users to the bridges they know. -That is, somebody in the blocked area can pass along a bridge's address to -somebody else they trust. This scheme brings in appealing but complex game -theoretic properties: the blocked user making the decision has an incentive -only to delegate to trustworthy people, since an adversary who learns -the bridge's address and filters it makes it unavailable for both of them. -Also, delegating known bridges to members of your social network can be -dangerous: an the adversary who can learn who knows which bridges may -be able to reconstruct the social network. - -Note that a central set of bridge directory authorities can still be -compatible with a decentralized discovery process. That is, how users -first learn about bridges is entirely up to the bridges, but the process -of fetching up-to-date descriptors for them can still proceed as described -in Section~\ref{sec:bridges}. Of course, creating a central place that -knows about all the bridges may not be smart, especially if every other -piece of the system is decentralized. Further, if a user only knows -about one bridge and he loses track of it, it may be quite a hassle to -reach the bridge authority. We address these concerns next. - -\subsection{Families of bridges, no central discovery} - -Because the blocked users are running our software too, we have many -opportunities to improve usability or robustness. Our second design builds -on the first by encouraging volunteers to run several bridges at once -(or coordinate with other bridge volunteers), such that some -of the bridges are likely to be available at any given time. - -The blocked user's Tor client would periodically fetch an updated set of -recommended bridges from any of the working bridges. Now the client can -learn new additions to the bridge pool, and can expire abandoned bridges -or bridges that the adversary has blocked, without the user ever needing -to care. To simplify maintenance of the community's bridge pool, each -community could run its own bridge directory authority---reachable via -the available bridges, and also mirrored at each bridge. - -\subsection{Public bridges with central discovery} - -What about people who want to volunteer as bridges but don't know any -suitable blocked users? What about people who are blocked but don't -know anybody on the outside? Here we describe how to make use of these -\emph{public bridges} in a way that still makes it hard for the attacker -to learn all of them. - -The basic idea is to divide public bridges into a set of pools based on -identity key. Each pool corresponds to a \emph{distribution strategy}: -an approach to distributing its bridge addresses to users. Each strategy -is designed to exercise a different scarce resource or property of -the user. - -How do we divide bridges between these strategy pools such that they're -evenly distributed and the allocation is hard to influence or predict, -but also in a way that's amenable to creating more strategies later -on without reshuffling all the pools? We assign a given bridge -to a strategy pool by hashing the bridge's identity key along with a -secret that only the bridge authority knows: the first $n$ bits of this -hash dictate the strategy pool number, where $n$ is a parameter that -describes how many strategy pools we want at this point. We choose $n=3$ -to start, so we divide bridges between 8 pools; but as we later invent -new distribution strategies, we can increment $n$ to split the 8 into -16. Since a bridge can't predict the next bit in its hash, it can't -anticipate which identity key will correspond to a certain new pool -when the pools are split. Further, since the bridge authority doesn't -provide any feedback to the bridge about which strategy pool it's in, -an adversary who signs up bridges with the goal of filling a certain -pool~\cite{casc-rep} will be hindered. - -% This algorithm is not ideal. When we split pools, each existing -% pool is cut in half, where half the bridges remain with the -% old distribution policy, and half will be under what the new one -% is. So the new distribution policy inherits a bunch of blocked -% bridges if the old policy was too loose, or a bunch of unblocked -% bridges if its policy was still secure. -RD -% -% I think it should be more chordlike. -% Bridges are allocated to wherever on the ring which is divided -% into arcs (buckets). -% If a bucket gets too full, you can just split it. -% More on this below. -PFS - -The first distribution strategy (used for the first pool) publishes bridge -addresses in a time-release fashion. The bridge authority divides the -available bridges into partitions, and each partition is deterministically -available only in certain time windows. That is, over the course of a -given time slot (say, an hour), each requester is given a random bridge -from within that partition. When the next time slot arrives, a new set -of bridges from the pool are available for discovery. Thus some bridge -address is always available when a new -user arrives, but to learn about all bridges the attacker needs to fetch -all new addresses at every new time slot. By varying the length of the -time slots, we can make it harder for the attacker to guess when to check -back. We expect these bridges will be the first to be blocked, but they'll -help the system bootstrap until they \emph{do} get blocked. Further, -remember that we're dealing with different blocking regimes around the -world that will progress at different rates---so this pool will still -be useful to some users even as the arms races progress. - -The second distribution strategy publishes bridge addresses based on the IP -address of the requesting user. Specifically, the bridge authority will -divide the available bridges in the pool into a bunch of partitions -(as in the first distribution scheme), hash the requester's IP address -with a secret of its own (as in the above allocation scheme for creating -pools), and give the requester a random bridge from the appropriate -partition. To raise the bar, we should discard the last octet of the -IP address before inputting it to the hash function, so an attacker -who only controls a single ``/24'' network only counts as one user. A -large attacker like China will still be able to control many addresses, -but the hassle of establishing connections from each network (or spoofing -TCP connections) may still slow them down. Similarly, as a special case, -we should treat IP addresses that are Tor exit nodes as all being on -the same network. - -The third strategy combines the time-based and location-based -strategies to further constrain and rate-limit the available bridge -addresses. Specifically, the bridge address provided in a given time -slot to a given network location is deterministic within the partition, -rather than chosen randomly each time from the partition. Thus, repeated -requests during that time slot from a given network are given the same -bridge address as the first request. - -The fourth strategy is based on Circumventor's discovery strategy. -The Circumventor project, realizing that its adoption will remain limited -if it has no central coordination mechanism, has started a mailing list to -distribute new proxy addresses every few days. From experimentation it -seems they have concluded that sending updates every three or four days -is sufficient to stay ahead of the current attackers. - -The fifth strategy provides an alternative approach to a mailing list: -users provide an email address and receive an automated response -listing an available bridge address. We could limit one response per -email address. To further rate limit queries, we could require a CAPTCHA -solution -%~\cite{captcha} -in each case too. In fact, we wouldn't need to -implement the CAPTCHA on our side: if we only deliver bridge addresses -to Yahoo or GMail addresses, we can leverage the rate-limiting schemes -that other parties already impose for account creation. - -The sixth strategy ties in the social network design with public -bridges and a reputation system. We pick some seeds---trusted people in -blocked areas---and give them each a few dozen bridge addresses and a few -\emph{delegation tokens}. We run a website next to the bridge authority, -where users can log in (they connect via Tor, and they don't need to -provide actual identities, just persistent pseudonyms). Users can delegate -trust to other people they know by giving them a token, which can be -exchanged for a new account on the website. Accounts in ``good standing'' -then accrue new bridge addresses and new tokens. As usual, reputation -schemes bring in a host of new complexities~\cite{rep-anon}: how do we -decide that an account is in good standing? We could tie reputation -to whether the bridges they're told about have been blocked---see -Section~\ref{subsec:geoip} below for initial thoughts on how to discover -whether bridges have been blocked. We could track reputation between -accounts (if you delegate to somebody who screws up, it impacts you too), -or we could use blinded delegation tokens~\cite{chaum-blind} to prevent -the website from mapping the seeds' social network. We put off deeper -discussion of the social network reputation strategy for future work. - -Pools seven and eight are held in reserve, in case our currently deployed -tricks all fail at once and the adversary blocks all those bridges---so -we can adapt and move to new approaches quickly, and have some bridges -immediately available for the new schemes. New strategies might be based -on some other scarce resource, such as relaying traffic for others or -other proof of energy spent. (We might also worry about the incentives -for bridges that sign up and get allocated to the reserve pools: will they -be unhappy that they're not being used? But this is a transient problem: -if Tor users are bridges by default, nobody will mind not being used yet. -See also Section~\ref{subsec:incentives}.) - -%Is it useful to load balance which bridges are handed out? The above -%pool concept makes some bridges wildly popular and others less so. -%But I guess that's the point. - -\subsection{Public bridges with coordinated discovery} - -We presented the above discovery strategies in the context of a single -bridge directory authority, but in practice we will want to distribute the -operations over several bridge authorities---a single point of failure -or attack is a bad move. The first answer is to run several independent -bridge directory authorities, and bridges gravitate to one based on -their identity key. The better answer would be some federation of bridge -authorities that work together to provide redundancy but don't introduce -new security issues. We could even imagine designs where the bridge -authorities have encrypted versions of the bridge's relay descriptors, -and the users learn a decryption key that they keep private when they -first hear about the bridge---this way the bridge authorities would not -be able to learn the IP address of the bridges. - -We leave this design question for future work. - -\subsection{Assessing whether bridges are useful} - -Learning whether a bridge is useful is important in the bridge authority's -decision to include it in responses to blocked users. For example, if -we end up with a list of thousands of bridges and only a few dozen of -them are reachable right now, most blocked users will not end up knowing -about working bridges. - -There are three components for assessing how useful a bridge is. First, -is it reachable from the public Internet? Second, what proportion of -the time is it available? Third, is it blocked in certain jurisdictions? - -The first component can be tested just as we test reachability of -ordinary Tor relays. Specifically, the bridges do a self-test---connect -to themselves via the Tor network---before they are willing to -publish their descriptor, to make sure they're not obviously broken or -misconfigured. Once the bridges publish, the bridge authority also tests -reachability to make sure they're not confused or outright lying. - -The second component can be measured and tracked by the bridge authority. -By doing periodic reachability tests, we can get a sense of how often the -bridge is available. More complex tests will involve bandwidth-intensive -checks to force the bridge to commit resources in order to be counted as -available. We need to evaluate how the relationship of uptime percentage -should weigh into our choice of which bridges to advertise. We leave -this to future work. - -The third component is perhaps the trickiest: with many different -adversaries out there, how do we keep track of which adversaries have -blocked which bridges, and how do we learn about new blocks as they -occur? We examine this problem next. - -\subsection{How do we know if a bridge relay has been blocked?} -\label{subsec:geoip} - -There are two main mechanisms for testing whether bridges are reachable -from inside each blocked area: active testing via users, and passive -testing via bridges. - -In the case of active testing, certain users inside each area -sign up as testing relays. The bridge authorities can then use a -Blossom-like~\cite{blossom-thesis} system to build circuits through them -to each bridge and see if it can establish the connection. But how do -we pick the users? If we ask random users to do the testing (or if we -solicit volunteers from the users), the adversary should sign up so he -can enumerate the bridges we test. Indeed, even if we hand-select our -testers, the adversary might still discover their location and monitor -their network activity to learn bridge addresses. - -Another answer is not to measure directly, but rather let the bridges -report whether they're being used. -%If they periodically report to their -%bridge directory authority how much use they're seeing, perhaps the -%authority can make smart decisions from there. -Specifically, bridges should install a GeoIP database such as the public -IP-To-Country list~\cite{ip-to-country}, and then periodically report to the -bridge authorities which countries they're seeing use from. This data -would help us track which countries are making use of the bridge design, -and can also let us learn about new steps the adversary has taken in -the arms race. (The compressed GeoIP database is only several hundred -kilobytes, and we could even automate the update process by serving it -from the bridge authorities.) -More analysis of this passive reachability -testing design is needed to resolve its many edge cases: for example, -if a bridge stops seeing use from a certain area, does that mean the -bridge is blocked or does that mean those users are asleep? - -There are many more problems with the general concept of detecting whether -bridges are blocked. First, different zones of the Internet are blocked -in different ways, and the actual firewall jurisdictions do not match -country borders. Our bridge scheme could help us map out the topology -of the censored Internet, but this is a huge task. More generally, -if a bridge relay isn't reachable, is that because of a network block -somewhere, because of a problem at the bridge relay, or just a temporary -outage somewhere in between? And last, an attacker could poison our -bridge database by signing up already-blocked bridges. In this case, -if we're stingy giving out bridge addresses, users in that country won't -learn working bridges. - -All of these issues are made more complex when we try to integrate this -testing into our social network reputation system above. -Since in that case we punish or reward users based on whether bridges -get blocked, the adversary has new attacks to trick or bog down the -reputation tracking. Indeed, the bridge authority doesn't even know -what zone the blocked user is in, so do we blame him for any possible -censored zone, or what? - -Clearly more analysis is required. The eventual solution will probably -involve a combination of passive measurement via GeoIP and active -measurement from trusted testers. More generally, we can use the passive -feedback mechanism to track usage of the bridge network as a whole---which -would let us respond to attacks and adapt the design, and it would also -let the general public track the progress of the project. - -%Worry: the adversary could choose not to block bridges but just record -%connections to them. So be it, I guess. - -\subsection{Advantages of deploying all solutions at once} - -For once, we're not in the position of the defender: we don't have to -defend against every possible filtering scheme; we just have to defend -against at least one. On the flip side, the attacker is forced to guess -how to allocate his resources to defend against each of these discovery -strategies. So by deploying all of our strategies at once, we not only -increase our chances of finding one that the adversary has difficulty -blocking, but we actually make \emph{all} of the strategies more robust -in the face of an adversary with limited resources. - -%\subsection{Remaining unsorted notes} - -%In the first subsection we describe how to find a first bridge. - -%Going to be an arms race. Need a bag of tricks. Hard to say -%which ones will work. Don't spend them all at once. - -%Some techniques are sufficient to get us an IP address and a port, -%and others can get us IP:port:key. Lay out some plausible options -%for how users can bootstrap into learning their first bridge. - -%\section{The account / reputation system} -%\section{Social networks with directory-side support} -%\label{sec:accounts} - -%One answer is to measure based on whether the bridge addresses -%we give it end up blocked. But how do we decide if they get blocked? - -%Perhaps each bridge should be known by a single bridge directory -%authority. This makes it easier to trace which users have learned about -%it, so easier to blame or reward. It also makes things more brittle, -%since loss of that authority means its bridges aren't advertised until -%they switch, and means its bridge users are sad too. -%(Need a slick hash algorithm that will map our identity key to a -%bridge authority, in a way that's sticky even when we add bridge -%directory authorities, but isn't sticky when our authority goes -%away. Does this exist?) -% [[Ian says: What about just using something like hash table chaining? -% This should work, so long as the client knows which authorities currently -% exist.]] - -%\subsection{Discovery based on social networks} - -%A token that can be exchanged at the bridge authority (assuming you -%can reach it) for a new bridge address. - -%The account server runs as a Tor controller for the bridge authority. - -%Users can establish reputations, perhaps based on social network -%connectivity, perhaps based on not getting their bridge relays blocked, - -%Probably the most critical lesson learned in past work on reputation -%systems in privacy-oriented environments~\cite{rep-anon} is the need for -%verifiable transactions. That is, the entity computing and advertising -%reputations for participants needs to actually learn in a convincing -%way that a given transaction was successful or unsuccessful. - -%(Lesson from designing reputation systems~\cite{rep-anon}: easy to -%reward good behavior, hard to punish bad behavior. - -\section{Security considerations} -\label{sec:security} - -\subsection{Possession of Tor in oppressed areas} - -Many people speculate that installing and using a Tor client in areas with -particularly extreme firewalls is a high risk---and the risk increases -as the firewall gets more restrictive. This notion certainly has merit, but -there's -a counter pressure as well: as the firewall gets more restrictive, more -ordinary people behind it end up using Tor for more mainstream activities, -such as learning -about Wall Street prices or looking at pictures of women's ankles. So -as the restrictive firewall pushes up the number of Tor users, the -``typical'' Tor user becomes more mainstream, and therefore mere -use or possession of the Tor software is not so surprising. - -It's hard to say which of these pressures will ultimately win out, -but we should keep both sides of the issue in mind. - -%Nick, want to rewrite/elaborate on this section? - -%Ian suggests: -% Possession of Tor: this is totally of-the-cuff, and there are lots of -% security issues to think about, but what about an ActiveX version of -% Tor? The magic you learn (as opposed to a bridge address) is a plain -% old HTTPS server, which feeds you an ActiveX applet pre-configured with -% some bridge address (possibly on the same machine). For bonus points, -% somehow arrange that (a) the applet is signed in some way the user can -% reliably check, but (b) don't end up with anything like an incriminating -% long-term cert stored on the user's computer. This may be marginally -% useful in some Internet-cafe situations as well, though (a) is even -% harder to get right there. - - -\subsection{Observers can tell who is publishing and who is reading} -\label{subsec:upload-padding} - -Tor encrypts traffic on the local network, and it obscures the eventual -destination of the communication, but it doesn't do much to obscure the -traffic volume. In particular, a user publishing a home video will have a -different network fingerprint than a user reading an online news article. -Based on our assumption in Section~\ref{sec:adversary} that users who -publish material are in more danger, should we work to improve Tor's -security in this situation? - -In the general case this is an extremely challenging task: -effective \emph{end-to-end traffic confirmation attacks} -are known where the adversary observes the origin and the -destination of traffic and confirms that they are part of the -same communication~\cite{danezis:pet2004,e2e-traffic}. Related are -\emph{website fingerprinting attacks}, where the adversary downloads -a few hundred popular websites, makes a set of "fingerprints" for each -site, and then observes the target Tor client's traffic to look for -a match~\cite{pet05-bissias,defensive-dropping}. But can we do better -against a limited adversary who just does coarse-grained sweeps looking -for unusually prolific publishers? - -One answer is for bridge users to automatically send bursts of padding -traffic periodically. (This traffic can be implemented in terms of -long-range drop cells, which are already part of the Tor specification.) -Of course, convincingly simulating an actual human publishing interesting -content is a difficult arms race, but it may be worthwhile to at least -start the race. More research remains. - -\subsection{Anonymity effects from acting as a bridge relay} - -Against some attacks, relaying traffic for others can improve -anonymity. The simplest example is an attacker who owns a small number -of Tor relays. He will see a connection from the bridge, but he won't -be able to know whether the connection originated there or was relayed -from somebody else. More generally, the mere uncertainty of whether the -traffic originated from that user may be helpful. - -There are some cases where it doesn't seem to help: if an attacker can -watch all of the bridge's incoming and outgoing traffic, then it's easy -to learn which connections were relayed and which started there. (In this -case he still doesn't know the final destinations unless he is watching -them too, but in this case bridges are no better off than if they were -an ordinary client.) - -There are also some potential downsides to running a bridge. First, while -we try to make it hard to enumerate all bridges, it's still possible to -learn about some of them, and for some people just the fact that they're -running one might signal to an attacker that they place a higher value -on their anonymity. Second, there are some more esoteric attacks on Tor -relays that are not as well-understood or well-tested---for example, an -attacker may be able to ``observe'' whether the bridge is sending traffic -even if he can't actually watch its network, by relaying traffic through -it and noticing changes in traffic timing~\cite{attack-tor-oak05}. On -the other hand, it may be that limiting the bandwidth the bridge is -willing to relay will allow this sort of attacker to determine if it's -being used as a bridge but not easily learn whether it is adding traffic -of its own. - -We also need to examine how entry guards fit in. Entry guards -(a small set of nodes that are always used for the first -step in a circuit) help protect against certain attacks -where the attacker runs a few Tor relays and waits for -the user to choose these relays as the beginning and end of her -circuit\footnote{\url{http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#EntryGuards}}. -If the blocked user doesn't use the bridge's entry guards, then the bridge -doesn't gain as much cover benefit. On the other hand, what design changes -are needed for the blocked user to use the bridge's entry guards without -learning what they are (this seems hard), and even if we solve that, -do they then need to use the guards' guards and so on down the line? - -It is an open research question whether the benefits of running a bridge -outweigh the risks. A lot of the decision rests on which attacks the -users are most worried about. For most users, we don't think running a -bridge relay will be that damaging, and it could help quite a bit. - -\subsection{Trusting local hardware: Internet cafes and LiveCDs} -\label{subsec:cafes-and-livecds} - -Assuming that users have their own trusted hardware is not -always reasonable. - -For Internet cafe Windows computers that let you attach your own USB key, -a USB-based Tor image would be smart. There's Torpark, and hopefully -there will be more thoroughly analyzed and trustworthy options down the -road. Worries remain about hardware or software keyloggers and other -spyware, as well as physical surveillance. - -If the system lets you boot from a CD or from a USB key, you can gain -a bit more security by bringing a privacy LiveCD with you. (This -approach isn't foolproof either of course, since hardware -keyloggers and physical surveillance are still a worry). - -In fact, LiveCDs are also useful if it's your own hardware, since it's -easier to avoid leaving private data and logs scattered around the -system. - -%\subsection{Forward compatibility and retiring bridge authorities} -% -%Eventually we'll want to change the identity key and/or location -%of a bridge authority. How do we do this mostly cleanly? - -\subsection{The trust chain} -\label{subsec:trust-chain} - -Tor's ``public key infrastructure'' provides a chain of trust to -let users verify that they're actually talking to the right relays. -There are four pieces to this trust chain. - -First, when Tor clients are establishing circuits, at each step -they demand that the next Tor relay in the path prove knowledge of -its private key~\cite{tor-design}. This step prevents the first node -in the path from just spoofing the rest of the path. Second, the -Tor directory authorities provide a signed list of relays along with -their public keys---so unless the adversary can control a threshold -of directory authorities, he can't trick the Tor client into using other -Tor relays. Third, the location and keys of the directory authorities, -in turn, is hard-coded in the Tor source code---so as long as the user -got a genuine version of Tor, he can know that he is using the genuine -Tor network. And last, the source code and other packages are signed -with the GPG keys of the Tor developers, so users can confirm that they -did in fact download a genuine version of Tor. - -In the case of blocked users contacting bridges and bridge directory -authorities, the same logic applies in parallel: the blocked users fetch -information from both the bridge authorities and the directory authorities -for the `main' Tor network, and they combine this information locally. - -How can a user in an oppressed country know that he has the correct -key fingerprints for the developers? As with other security systems, it -ultimately comes down to human interaction. The keys are signed by dozens -of people around the world, and we have to hope that our users have met -enough people in the PGP web of trust -%~\cite{pgp-wot} -that they can learn -the correct keys. For users that aren't connected to the global security -community, though, this question remains a critical weakness. - -%\subsection{Security through obscurity: publishing our design} - -%Many other schemes like dynaweb use the typical arms race strategy of -%not publishing their plans. Our goal here is to produce a design---a -%framework---that can be public and still secure. Where's the tradeoff? - -%\section{Performance improvements} -%\label{sec:performance} -% -%\subsection{Fetch relay descriptors just-in-time} -% -%I guess we should encourage most places to do this, so blocked -%users don't stand out. -% -% -%network-status and directory optimizations. caching better. partitioning -%issues? - -\section{Maintaining reachability} -\label{sec:reachability} - -\subsection{How many bridge relays should you know about?} - -The strategies described in Section~\ref{sec:discovery} talked about -learning one bridge address at a time. But if most bridges are ordinary -Tor users on cable modem or DSL connection, many of them will disappear -and/or move periodically. How many bridge relays should a blocked user -know about so that she is likely to have at least one reachable at any -given point? This is already a challenging problem if we only consider -natural churn: the best approach is to see what bridges we attract in -reality and measure their churn. We may also need to factor in a parameter -for how quickly bridges get discovered and blocked by the attacker; -we leave this for future work after we have more deployment experience. - -A related question is: if the bridge relays change IP addresses -periodically, how often does the blocked user need to fetch updates in -order to keep from being cut out of the loop? - -Once we have more experience and intuition, we should explore technical -solutions to this problem too. For example, if the discovery strategies -give out $k$ bridge addresses rather than a single bridge address, perhaps -we can improve robustness from the user perspective without significantly -aiding the adversary. Rather than giving out a new random subset of $k$ -addresses at each point, we could bind them together into \emph{bridge -families}, so all users that learn about one member of the bridge family -are told about the rest as well. - -This scheme may also help defend against attacks to map the set of -bridges. That is, if all blocked users learn a random subset of bridges, -the attacker should learn about a few bridges, monitor the country-level -firewall for connections to them, then watch those users to see what -other bridges they use, and repeat. By segmenting the bridge address -space, we can limit the exposure of other users. - -\subsection{Cablemodem users don't usually provide important websites} -\label{subsec:block-cable} - -Another attacker we might be concerned about is that the attacker could -just block all DSL and cablemodem network addresses, on the theory that -they don't run any important services anyway. If most of our bridges -are on these networks, this attack could really hurt. - -The first answer is to aim to get volunteers both from traditionally -``consumer'' networks and also from traditionally ``producer'' networks. -Since bridges don't need to be Tor exit nodes, as we improve our usability -it seems quite feasible to get a lot of websites helping out. - -The second answer (not as practical) would be to encourage more use of -consumer networks for popular and useful Internet services. -%(But P2P exists; -%minor websites exist; gaming exists; IM exists; ...) - -A related attack we might worry about is based on large countries putting -economic pressure on companies that want to expand their business. For -example, what happens if Verizon wants to sell services in China, and -China pressures Verizon to discourage its users in the free world from -running bridges? - -\subsection{Scanning resistance: making bridges more subtle} - -If it's trivial to verify that a given address is operating as a bridge, -and most bridges run on a predictable port, then it's conceivable our -attacker could scan the whole Internet looking for bridges. (In fact, -he can just concentrate on scanning likely networks like cablemodem -and DSL services---see Section~\ref{subsec:block-cable} above for -related attacks.) It would be nice to slow down this attack. It would -be even nicer to make it hard to learn whether we're a bridge without -first knowing some secret. We call this general property \emph{scanning -resistance}, and it goes along with normalizing Tor's TLS handshake and -network fingerprint. - -We could provide a password to the blocked user, and she (or her Tor -client) provides a nonced hash of this password when she connects. We'd -need to give her an ID key for the bridge too (in addition to the IP -address and port---see Section~\ref{subsec:id-address}), and wait to -present the password until we've finished the TLS handshake, else it -would look unusual. If Alice can authenticate the bridge before she -tries to send her password, we can resist an adversary who pretends -to be the bridge and launches a man-in-the-middle attack to learn the -password. But even if she can't, we still resist against widespread -scanning. - -How should the bridge behave if accessed without the correct -authorization? Perhaps it should act like an unconfigured HTTPS server -(``welcome to the default Apache page''), or maybe it should mirror -and act like common websites, or websites randomly chosen from Google. - -We might assume that the attacker can recognize HTTPS connections that -use self-signed certificates. (This process would be resource-intensive -but not out of the realm of possibility.) But even in this case, many -popular websites around the Internet use self-signed or just plain broken -SSL certificates. - -%to unknown servers. It can then attempt to connect to them and block -%connections to servers that seem suspicious. It may be that password -%protected web sites will not be suspicious in general, in which case -%that may be the easiest way to give controlled access to the bridge. -%If such sites that have no other overt features are automatically -%blocked when detected, then we may need to be more subtle. -%Possibilities include serving an innocuous web page if a TLS encrypted -%request is received without the authorization needed to access the Tor -%network and only responding to a requested access to the Tor network -%of proper authentication is given. If an unauthenticated request to -%access the Tor network is sent, the bridge should respond as if -%it has received a message it does not understand (as would be the -%case were it not a bridge). - -% Ian suggests a ``socialist millionaires'' protocol here, for something. - -% Did we once mention knocking here? it's a good idea, but we should clarify -% what we mean. Ian also notes that knocking itself is very fingerprintable, -% and we should beware. - -\subsection{How to motivate people to run bridge relays} -\label{subsec:incentives} - -One of the traditional ways to get people to run software that benefits -others is to give them motivation to install it themselves. An often -suggested approach is to install it as a stunning screensaver so everybody -will be pleased to run it. We take a similar approach here, by leveraging -the fact that these users are already interested in protecting their -own Internet traffic, so they will install and run the software. - -Eventually, we may be able to make all Tor users become bridges if they -pass their self-reachability tests---the software and installers need -more work on usability first, but we're making progress. - -In the mean time, we can make a snazzy network graph with -Vidalia\footnote{\url{http://vidalia-project.net/}} that -emphasizes the connections the bridge user is currently relaying. -%(Minor -%anonymity implications, but hey.) (In many cases there won't be much -%activity, so this may backfire. Or it may be better suited to full-fledged -%Tor relay.) - -% Also consider everybody-a-relay. Many of the scalability questions -% are easier when you're talking about making everybody a bridge. - -%\subsection{What if the clients can't install software?} - -%[this section should probably move to the related work section, -%or just disappear entirely.] - -%Bridge users without Tor software - -%Bridge relays could always open their socks proxy. This is bad though, -%first -%because bridges learn the bridge users' destinations, and second because -%we've learned that open socks proxies tend to attract abusive users who -%have no idea they're using Tor. - -%Bridges could require passwords in the socks handshake (not supported -%by most software including Firefox). Or they could run web proxies -%that require authentication and then pass the requests into Tor. This -%approach is probably a good way to help bootstrap the Psiphon network, -%if one of its barriers to deployment is a lack of volunteers willing -%to exit directly to websites. But it clearly drops some of the nice -%anonymity and security features Tor provides. - -%A hybrid approach where the user gets his anonymity from Tor but his -%software-less use from a web proxy running on a trusted machine on the -%free side. - -\subsection{Publicity attracts attention} -\label{subsec:publicity} - -Many people working on this field want to publicize the existence -and extent of censorship concurrently with the deployment of their -circumvention software. The easy reason for this two-pronged push is -to attract volunteers for running proxies in their systems; but in many -cases their main goal is not to focus on getting more users signed up, -but rather to educate the rest of the world about the -censorship. The media also tries to do its part by broadcasting the -existence of each new circumvention system. - -But at the same time, this publicity attracts the attention of the -censors. We can slow down the arms race by not attracting as much -attention, and just spreading by word of mouth. If our goal is to -establish a solid social network of bridges and bridge users before -the adversary gets involved, does this extra attention work to our -disadvantage? - -\subsection{The Tor website: how to get the software} - -One of the first censoring attacks against a system like ours is to -block the website and make the software itself hard to find. Our system -should work well once the user is running an authentic -copy of Tor and has found a working bridge, but to get to that point -we rely on their individual skills and ingenuity. - -Right now, most countries that block access to Tor block only the main -website and leave mirrors and the network itself untouched. -Falling back on word-of-mouth is always a good last resort, but we should -also take steps to make sure it's relatively easy for users to get a copy, -such as publicizing the mirrors more and making copies available through -other media. We might also mirror the latest version of the software on -each bridge, so users who hear about an honest bridge can get a good -copy. -See Section~\ref{subsec:first-bridge} for more discussion. - -% Ian suggests that we have every tor relay distribute a signed copy of the -% software. - -\section{Next Steps} -\label{sec:conclusion} - -Technical solutions won't solve the whole censorship problem. After all, -the firewalls in places like China are \emph{socially} very -successful, even if technologies and tricks exist to get around them. -However, having a strong technical solution is still necessary as one -important piece of the puzzle. - -In this paper, we have shown that Tor provides a great set of building -blocks to start from. The next steps are to deploy prototype bridges and -bridge authorities, implement some of the proposed discovery strategies, -and then observe the system in operation and get more intuition about -the actual requirements and adversaries we're up against. - -\bibliographystyle{plain} \bibliography{tor-design} - -%\appendix - -%\section{Counting Tor users by country} -%\label{app:geoip} - -\end{document} - - - -\section{Future designs} -\label{sec:future} - -\subsection{Bridges inside the blocked network too} - -Assuming actually crossing the firewall is the risky part of the -operation, can we have some bridge relays inside the blocked area too, -and more established users can use them as relays so they don't need to -communicate over the firewall directly at all? A simple example here is -to make new blocked users into internal bridges also---so they sign up -on the bridge authority as part of doing their query, and we give out -their addresses -rather than (or along with) the external bridge addresses. This design -is a lot trickier because it brings in the complexity of whether the -internal bridges will remain available, can maintain reachability with -the outside world, etc. - -More complex future designs involve operating a separate Tor network -inside the blocked area, and using \emph{hidden service bridges}---bridges -that can be accessed by users of the internal Tor network but whose -addresses are not published or findable, even by these users---to get -from inside the firewall to the rest of the Internet. But this design -requires directory authorities to run inside the blocked area too, -and they would be a fine target to take down the network. - -% Hidden services as bridge directory authorities. - - ------------------------------------------- - -ship geoip db to bridges. they look up users who tls to them in the db, -and upload a signed list of countries and number-of-users each day. the -bridge authority aggregates them and publishes stats. - -bridge relays have buddies -they ask a user to test the reachability of their buddy. -leaks O(1) bridges, but not O(n). - -we should not be blockable by ordinary cisco censorship features. -that is, if they want to block our new design, they will need to -add a feature to block exactly this. -strategically speaking, this may come in handy. - -Bridges come in clumps of 4 or 8 or whatever. If you know one bridge -in a clump, the authority will tell you the rest. Now bridges can -ask users to test reachability of their buddies. - -Giving out clumps helps with dynamic IP addresses too. Whether it -should be 4 or 8 depends on our churn. - -the account server. let's call it a database, it doesn't have to -be a thing that human interacts with. - -so how do we reward people for being good? - -\subsubsection{Public Bridges with Coordinated Discovery} - -****Pretty much this whole subsubsection will probably need to be -deferred until ``later'' and moved to after end document, but I'm leaving -it here for now in case useful.****** - -Rather than be entirely centralized, we can have a coordinated -collection of bridge authorities, analogous to how Tor network -directory authorities now work. - -Key components -``Authorities'' will distribute caches of what they know to overlapping -collections of nodes so that no one node is owned by one authority. -Also so that it is impossible to DoS info maintained by one authority -simply by making requests to it. - -Where a bridge gets assigned is not predictable by the bridge? - -If authorities don't know the IP addresses of the bridges they -are responsible for, they can't abuse that info (or be attacked for -having it). But, they also can't, e.g., control being sent massive -lists of nodes that were never good. This raises another question. -We generally decry use of IP address for location, etc. but we -need to do that to limit the introduction of functional but useless -IP addresses because, e.g., they are in China and the adversary -owns massive chunks of the IP space there. - -We don't want an arbitrary someone to be able to contact the -authorities and say an IP address is bad because it would be easy -for an adversary to take down all the suspicious bridges -even if they provide good cover websites, etc. Only the bridge -itself and/or the directory authority can declare a bridge blocked -from somewhere. - - -9. Bridge directories must not simply be a handful of nodes that -provide the list of bridges. They must flood or otherwise distribute -information out to other Tor nodes as mirrors. That way it becomes -difficult for censors to flood the bridge directory authorities with -requests, effectively denying access for others. But, there's lots of -churn and a much larger size than Tor directories. We are forced to -handle the directory scaling problem here much sooner than for the -network in general. Authorities can pass their bridge directories -(and policy info) to some moderate number of unidentified Tor nodes. -Anyone contacting one of those nodes can get bridge info. the nodes -must remain somewhat synched to prevent the adversary from abusing, -e.g., a timed release policy or the distribution to those nodes must -be resilient even if they are not coordinating. - -I think some kind of DHT like scheme would work here. A Tor node is -assigned a chunk of the directory. Lookups in the directory should be -via hashes of keys (fingerprints) and that should determine the Tor -nodes responsible. Ordinary directories can publish lists of Tor nodes -responsible for fingerprint ranges. Clients looking to update info on -some bridge will make a Tor connection to one of the nodes responsible -for that address. Instead of shutting down a circuit after getting -info on one address, extend it to another that is responsible for that -address (the node from which you are extending knows you are doing so -anyway). Keep going. This way you can amortize the Tor connection. - -10. We need some way to give new identity keys out to those who need -them without letting those get immediately blocked by authorities. One -way is to give a fingerprint that gets you more fingerprints, as -already described. These are meted out/updated periodically but allow -us to keep track of which sources are compromised: if a distribution -fingerprint repeatedly leads to quickly blocked bridges, it should be -suspect, dropped, etc. Since we're using hashes, there shouldn't be a -correlation with bridge directory mirrors, bridges, portions of the -network observed, etc. It should just be that the authorities know -about that key that leads to new addresses. - -This last point is very much like the issues in the valet nodes paper, -which is essentially about blocking resistance wrt exiting the Tor network, -while this paper is concerned with blocking the entering to the Tor network. -In fact the tickets used to connect to the IPo (Introduction Point), -could serve as an example, except that instead of authorizing -a connection to the Hidden Service, it's authorizing the downloading -of more fingerprints. - -Also, the fingerprints can follow the hash(q + '1' + cookie) scheme of -that paper (where q = hash(PK + salt) gave the q.onion address). This -allows us to control and track which fingerprint was causing problems. - -Note that, unlike many settings, the reputation problem should not be -hard here. If a bridge says it is blocked, then it might as well be. -If an adversary can say that the bridge is blocked wrt -$\mathit{censor}_i$, then it might as well be, since -$\mathit{censor}_i$ can presumably then block that bridge if it so -chooses. - -11. How much damage can the adversary do by running nodes in the Tor -network and watching for bridge nodes connecting to it? (This is -analogous to an Introduction Point watching for Valet Nodes connecting -to it.) What percentage of the network do you need to own to do how -much damage. Here the entry-guard design comes in helpfully. So we -need to have bridges use entry-guards, but (cf. 3 above) not use -bridges as entry-guards. Here's a serious tradeoff (again akin to the -ratio of valets to IPos) the more bridges/client the worse the -anonymity of that client. The fewer bridges/client the worse the -blocking resistance of that client. - - - diff --git a/doc/design-paper/cell-struct.eps b/doc/design-paper/cell-struct.eps deleted file mode 100644 index eb9fcb864..000000000 --- a/doc/design-paper/cell-struct.eps +++ /dev/null @@ -1,189 +0,0 @@ -%!PS-Adobe-2.0 EPSF-2.0 -%%Title: cell-struct.fig -%%Creator: fig2dev Version 3.2 Patchlevel 4 -%%CreationDate: Mon May 17 00:04:58 2004 -%%For: root@localhost.localdomain (root) -%%BoundingBox: 0 0 254 73 -%%Magnification: 1.0000 -%%EndComments -/$F2psDict 200 dict def -$F2psDict begin -$F2psDict /mtrx matrix put -/col-1 {0 setgray} bind def -/col0 {0.000 0.000 0.000 srgb} bind def -/col1 {0.000 0.000 1.000 srgb} bind def -/col2 {0.000 1.000 0.000 srgb} bind def -/col3 {0.000 1.000 1.000 srgb} bind def -/col4 {1.000 0.000 0.000 srgb} bind def -/col5 {1.000 0.000 1.000 srgb} bind def -/col6 {1.000 1.000 0.000 srgb} bind def -/col7 {1.000 1.000 1.000 srgb} bind def -/col8 {0.000 0.000 0.560 srgb} bind def -/col9 {0.000 0.000 0.690 srgb} bind def -/col10 {0.000 0.000 0.820 srgb} bind def -/col11 {0.530 0.810 1.000 srgb} bind def -/col12 {0.000 0.560 0.000 srgb} bind def -/col13 {0.000 0.690 0.000 srgb} bind def -/col14 {0.000 0.820 0.000 srgb} bind def -/col15 {0.000 0.560 0.560 srgb} bind def -/col16 {0.000 0.690 0.690 srgb} bind def -/col17 {0.000 0.820 0.820 srgb} bind def -/col18 {0.560 0.000 0.000 srgb} bind def -/col19 {0.690 0.000 0.000 srgb} bind def -/col20 {0.820 0.000 0.000 srgb} bind def -/col21 {0.560 0.000 0.560 srgb} bind def -/col22 {0.690 0.000 0.690 srgb} bind def -/col23 {0.820 0.000 0.820 srgb} bind def -/col24 {0.500 0.190 0.000 srgb} bind def -/col25 {0.630 0.250 0.000 srgb} bind def -/col26 {0.750 0.380 0.000 srgb} bind def -/col27 {1.000 0.500 0.500 srgb} bind def -/col28 {1.000 0.630 0.630 srgb} bind def -/col29 {1.000 0.750 0.750 srgb} bind def -/col30 {1.000 0.880 0.880 srgb} bind def -/col31 {1.000 0.840 0.000 srgb} bind def - -end -save -newpath 0 73 moveto 0 0 lineto 254 0 lineto 254 73 lineto closepath clip newpath --35.3 77.2 translate -1 -1 scale - -/cp {closepath} bind def -/ef {eofill} bind def -/gr {grestore} bind def -/gs {gsave} bind def -/sa {save} bind def -/rs {restore} bind def -/l {lineto} bind def -/m {moveto} bind def -/rm {rmoveto} bind def -/n {newpath} bind def -/s {stroke} bind def -/sh {show} bind def -/slc {setlinecap} bind def -/slj {setlinejoin} bind def -/slw {setlinewidth} bind def -/srgb {setrgbcolor} bind def -/rot {rotate} bind def -/sc {scale} bind def -/sd {setdash} bind def -/ff {findfont} bind def -/sf {setfont} bind def -/scf {scalefont} bind def -/sw {stringwidth} bind def -/tr {translate} bind def -/tnt {dup dup currentrgbcolor - 4 -2 roll dup 1 exch sub 3 -1 roll mul add - 4 -2 roll dup 1 exch sub 3 -1 roll mul add - 4 -2 roll dup 1 exch sub 3 -1 roll mul add srgb} - bind def -/shd {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul - 4 -2 roll mul srgb} bind def -/$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def -/$F2psEnd {$F2psEnteredState restore end} def - -$F2psBegin -10 setmiterlimit -0 slj 0 slc - 0.06000 0.06000 sc -% -% Fig objects follow -% -% -% here starts figure with depth 50 -% Polyline -7.500 slw -n 1200 975 m - 1200 1275 l gs col0 s gr -% Polyline -n 1725 975 m - 1725 1275 l gs col0 s gr -% Polyline -n 600 975 m 4800 975 l 4800 1275 l 600 1275 l - cp gs col0 s gr -% Polyline -n 1200 300 m - 1200 600 l gs col0 s gr -% Polyline -n 1725 300 m - 1725 600 l gs col0 s gr -% Polyline -n 600 300 m 4800 300 l 4800 600 l 600 600 l - cp gs col0 s gr -% Polyline -n 2550 975 m - 2550 1275 l gs col0 s gr -% Polyline -n 3150 975 m - 3150 1275 l gs col0 s gr -% Polyline -n 3450 975 m - 3450 1275 l gs col0 s gr -% Polyline -n 3900 975 m - 3900 1275 l gs col0 s gr -/Times-Roman ff 180.00 scf sf -675 1200 m -gs 1 -1 sc (CircID) col0 sh gr -/Times-Roman ff 180.00 scf sf -900 900 m -gs 1 -1 sc (2) col0 sh gr -/Times-Roman ff 180.00 scf sf -1275 1200 m -gs 1 -1 sc (Relay) col0 sh gr -/Times-Roman ff 180.00 scf sf -1800 1200 m -gs 1 -1 sc (StreamID) col0 sh gr -/Times-Roman ff 180.00 scf sf -2625 1200 m -gs 1 -1 sc (Digest) col0 sh gr -/Times-Roman ff 180.00 scf sf -3150 1200 m -gs 1 -1 sc (Len) col0 sh gr -/Times-Roman ff 180.00 scf sf -4200 1200 m -gs 1 -1 sc (DATA) col0 sh gr -/Times-Roman ff 180.00 scf sf -675 525 m -gs 1 -1 sc (CircID) col0 sh gr -/Times-Roman ff 180.00 scf sf -1275 525 m -gs 1 -1 sc (CMD) col0 sh gr -/Times-Roman ff 180.00 scf sf -900 225 m -gs 1 -1 sc (2) col0 sh gr -/Times-Roman ff 180.00 scf sf -1425 225 m -gs 1 -1 sc (1) col0 sh gr -/Times-Roman ff 180.00 scf sf -3225 525 m -gs 1 -1 sc (DATA) col0 sh gr -/Times-Roman ff 180.00 scf sf -3225 900 m -gs 1 -1 sc (2) col0 sh gr -/Times-Roman ff 180.00 scf sf -3450 1200 m -gs 1 -1 sc (CMD) col0 sh gr -/Times-Roman ff 180.00 scf sf -3600 900 m -gs 1 -1 sc (1) col0 sh gr -/Times-Roman ff 180.00 scf sf -3300 225 m -gs 1 -1 sc (509 bytes) col0 sh gr -/Times-Roman ff 180.00 scf sf -1425 900 m -gs 1 -1 sc (1) col0 sh gr -/Times-Roman ff 180.00 scf sf -2100 900 m -gs 1 -1 sc (2) col0 sh gr -/Times-Roman ff 180.00 scf sf -2850 900 m -gs 1 -1 sc (6) col0 sh gr -/Times-Roman ff 180.00 scf sf -4350 900 m -gs 1 -1 sc (498) col0 sh gr -% here ends figure; -$F2psEnd -rs -showpage diff --git a/doc/design-paper/cell-struct.fig b/doc/design-paper/cell-struct.fig deleted file mode 100644 index 3490673ca..000000000 --- a/doc/design-paper/cell-struct.fig +++ /dev/null @@ -1,49 +0,0 @@ -#FIG 3.2 -Landscape -Center -Inches -Letter -100.00 -Single --2 -1200 2 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 1200 975 1200 1275 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 1725 975 1725 1275 -2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 - 600 975 4800 975 4800 1275 600 1275 600 975 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 1200 300 1200 600 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 1725 300 1725 600 -2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 - 600 300 4800 300 4800 600 600 600 600 300 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 2550 975 2550 1275 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 3150 975 3150 1275 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 3450 975 3450 1275 -2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2 - 3900 975 3900 1275 -4 0 0 50 -1 0 12 0.0000 4 135 510 675 1200 CircID\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 900 900 2\001 -4 0 0 50 -1 0 12 0.0000 4 180 435 1275 1200 Relay\001 -4 0 0 50 -1 0 12 0.0000 4 135 735 1800 1200 StreamID\001 -4 0 0 50 -1 0 12 0.0000 4 180 510 2625 1200 Digest\001 -4 0 0 50 -1 0 12 0.0000 4 135 285 3150 1200 Len\001 -4 0 0 50 -1 0 12 0.0000 4 135 510 4200 1200 DATA\001 -4 0 0 50 -1 0 12 0.0000 4 135 510 675 525 CircID\001 -4 0 0 50 -1 0 12 0.0000 4 135 420 1275 525 CMD\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 900 225 2\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 1425 225 1\001 -4 0 0 50 -1 0 12 0.0000 4 135 510 3225 525 DATA\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 3225 900 2\001 -4 0 0 50 -1 0 12 0.0000 4 135 420 3450 1200 CMD\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 3600 900 1\001 -4 0 0 50 -1 0 12 0.0000 4 180 735 3300 225 509 bytes\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 1425 900 1\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 2100 900 2\001 -4 0 0 50 -1 0 12 0.0000 4 135 90 2850 900 6\001 -4 0 0 50 -1 0 12 0.0000 4 135 270 4350 900 498\001 diff --git a/doc/design-paper/cell-struct.pdf b/doc/design-paper/cell-struct.pdf Binary files differdeleted file mode 100644 index 8ca52deeb..000000000 --- a/doc/design-paper/cell-struct.pdf +++ /dev/null diff --git a/doc/design-paper/cell-struct.png b/doc/design-paper/cell-struct.png Binary files differdeleted file mode 100644 index 799bcc8c1..000000000 --- a/doc/design-paper/cell-struct.png +++ /dev/null diff --git a/doc/design-paper/challenges.pdf b/doc/design-paper/challenges.pdf Binary files differdeleted file mode 100644 index d0a3e0923..000000000 --- a/doc/design-paper/challenges.pdf +++ /dev/null diff --git a/doc/design-paper/challenges.tex b/doc/design-paper/challenges.tex deleted file mode 100644 index 6949693bf..000000000 --- a/doc/design-paper/challenges.tex +++ /dev/null @@ -1,1505 +0,0 @@ -\documentclass{llncs} - -\usepackage{url} -\usepackage{amsmath} -\usepackage{epsfig} - -\setlength{\textwidth}{5.9in} -\setlength{\textheight}{8.4in} -\setlength{\topmargin}{.5cm} -\setlength{\oddsidemargin}{1cm} -\setlength{\evensidemargin}{1cm} - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} - -\begin{document} - -\title{Challenges in deploying low-latency anonymity (DRAFT)} - -\author{Roger Dingledine\inst{1} \and -Nick Mathewson\inst{1} \and -Paul Syverson\inst{2}} -\institute{The Free Haven Project \email{<\{arma,nickm\}@freehaven.net>} \and -Naval Research Laboratory \email{<syverson@itd.nrl.navy.mil>}} - -\maketitle -\pagestyle{plain} - -\begin{abstract} - There are many unexpected or unexpectedly difficult obstacles to - deploying anonymous communications. Drawing on our experiences deploying - Tor (the second-generation onion routing network), we describe social - challenges and technical issues that must be faced - in building, deploying, and sustaining a scalable, distributed, low-latency - anonymity network. -\end{abstract} - -\section{Introduction} -% Your network is not practical unless it is sustainable and distributed. -Anonymous communication is full of surprises. This paper discusses some -unexpected challenges arising from our experiences deploying Tor, a -low-latency general-purpose anonymous communication system. We will discuss -some of the difficulties we have experienced and how we have met them (or how -we plan to meet them, if we know). We also discuss some less -troublesome open problems that we must nevertheless eventually address. -%We will describe both those future challenges that we intend to explore and -%those that we have decided not to explore and why. - -Tor is an overlay network for anonymizing TCP streams over the -Internet~\cite{tor-design}. It addresses limitations in earlier Onion -Routing designs~\cite{or-ih96,or-jsac98,or-discex00,or-pet00} by adding -perfect forward secrecy, congestion control, directory servers, data -integrity, configurable exit policies, and location-hidden services using -rendezvous points. Tor works on the real-world Internet, requires no special -privileges or kernel modifications, requires little synchronization or -coordination between nodes, and provides a reasonable trade-off between -anonymity, usability, and efficiency. - -We deployed the public Tor network in October 2003; since then it has -grown to over a hundred volunteer-operated nodes -and as much as 80 megabits of -average traffic per second. Tor's research strategy has focused on deploying -a network to as many users as possible; thus, we have resisted designs that -would compromise deployability by imposing high resource demands on node -operators, and designs that would compromise usability by imposing -unacceptable restrictions on which applications we support. Although this -strategy has -drawbacks (including a weakened threat model, as discussed below), it has -made it possible for Tor to serve many thousands of users and attract -funding from diverse sources whose goals range from security on a -national scale down to individual liberties. - -In~\cite{tor-design} we gave an overall view of Tor's -design and goals. Here we describe some policy, social, and technical -issues that we face as we continue deployment. -Rather than providing complete solutions to every problem, we -instead lay out the challenges and constraints that we have observed while -deploying Tor. In doing so, we aim to provide a research agenda -of general interest to projects attempting to build -and deploy practical, usable anonymity networks in the wild. - -%While the Tor design paper~\cite{tor-design} gives an overall view its -%design and goals, -%this paper describes the policy and technical issues that Tor faces as -%we continue deployment. Rather than trying to provide complete solutions -%to every problem here, we lay out the assumptions and constraints -%that we have observed through deploying Tor in the wild. In doing so, we -%aim to create a research agenda for others to -%help in addressing these issues. -% Section~\ref{sec:what-is-tor} gives an -%overview of the Tor -%design and ours goals. Sections~\ref{sec:crossroads-policy} -%and~\ref{sec:crossroads-design} go on to describe the practical challenges, -%both policy and technical respectively, -%that stand in the way of moving -%from a practical useful network to a practical useful anonymous network. - -%\section{What Is Tor} -\section{Background} -Here we give a basic overview of the Tor design and its properties, and -compare Tor to other low-latency anonymity designs. - -\subsection{Tor, threat models, and distributed trust} -\label{sec:what-is-tor} - -%Here we give a basic overview of the Tor design and its properties. For -%details on the design, assumptions, and security arguments, we refer -%the reader to the Tor design paper~\cite{tor-design}. - -Tor provides \emph{forward privacy}, so that users can connect to -Internet sites without revealing their logical or physical locations -to those sites or to observers. It also provides \emph{location-hidden -services}, so that servers can support authorized users without -giving an effective vector for physical or online attackers. -Tor provides these protections even when a portion of its -infrastructure is compromised. - -To connect to a remote server via Tor, the client software learns a signed -list of Tor nodes from one of several central \emph{directory servers}, and -incrementally creates a private pathway or \emph{circuit} of encrypted -connections through authenticated Tor nodes on the network, negotiating a -separate set of encryption keys for each hop along the circuit. The circuit -is extended one node at a time, and each node along the way knows only the -immediately previous and following nodes in the circuit, so no individual Tor -node knows the complete path that each fixed-sized data packet (or -\emph{cell}) will take. -%Because each node sees no more than one hop in the -%circuit, -Thus, neither an eavesdropper nor a compromised node can -see both the connection's source and destination. Later requests use a new -circuit, to complicate long-term linkability between different actions by -a single user. - -Tor also helps servers hide their locations while -providing services such as web publishing or instant -messaging. Using ``rendezvous points'', other Tor users can -connect to these authenticated hidden services, neither one learning the -other's network identity. - -Tor attempts to anonymize the transport layer, not the application layer. -This approach is useful for applications such as SSH -where authenticated communication is desired. However, when anonymity from -those with whom we communicate is desired, -application protocols that include personally identifying information need -additional application-level scrubbing proxies, such as -Privoxy~\cite{privoxy} for HTTP\@. Furthermore, Tor does not relay arbitrary -IP packets; it only anonymizes TCP streams and DNS requests -%, and only supports -%connections via SOCKS -(but see Section~\ref{subsec:tcp-vs-ip}). - -Most node operators do not want to allow arbitrary TCP traffic. % to leave -%their server. -To address this, Tor provides \emph{exit policies} so -each exit node can block the IP addresses and ports it is unwilling to allow. -Tor nodes advertise their exit policies to the directory servers, so that -clients can tell which nodes will support their connections. - -As of January 2005, the Tor network has grown to around a hundred nodes -on four continents, with a total capacity exceeding 1Gbit/s. Appendix A -shows a graph of the number of working nodes over time, as well as a -graph of the number of bytes being handled by the network over time. -The network is now sufficiently diverse for further development -and testing; but of course we always encourage new nodes -to join. - -Tor research and development has been funded by ONR and DARPA -for use in securing government -communications, and by the Electronic Frontier Foundation for use -in maintaining civil liberties for ordinary citizens online. The Tor -protocol is one of the leading choices -for the anonymizing layer in the European Union's PRIME directive to -help maintain privacy in Europe. -The AN.ON project in Germany -has integrated an independent implementation of the Tor protocol into -their popular Java Anon Proxy anonymizing client. -% This wide variety of -%interests helps maintain both the stability and the security of the -%network. - -\medskip -\noindent -{\bf Threat models and design philosophy.} -The ideal Tor network would be practical, useful and anonymous. When -trade-offs arise between these properties, Tor's research strategy has been -to remain useful enough to attract many users, -and practical enough to support them. Only subject to these -constraints do we try to maximize -anonymity.\footnote{This is not the only possible -direction in anonymity research: designs exist that provide more anonymity -than Tor at the expense of significantly increased resource requirements, or -decreased flexibility in application support (typically because of increased -latency). Such research does not typically abandon aspirations toward -deployability or utility, but instead tries to maximize deployability and -utility subject to a certain degree of structural anonymity (structural because -usability and practicality affect usage which affects the actual anonymity -provided by the network \cite{econymics,back01}).} -%{We believe that these -%approaches can be promising and useful, but that by focusing on deploying a -%usable system in the wild, Tor helps us experiment with the actual parameters -%of what makes a system ``practical'' for volunteer operators and ``useful'' -%for home users, and helps illuminate undernoticed issues which any deployed -%volunteer anonymity network will need to address.} -Because of our strategy, Tor has a weaker threat model than many designs in -the literature. In particular, because we -support interactive communications without impractically expensive padding, -we fall prey to a variety -of intra-network~\cite{back01,attack-tor-oak05,flow-correlation04} and -end-to-end~\cite{danezis:pet2004,SS03} anonymity-breaking attacks. - -Tor does not attempt to defend against a global observer. In general, an -attacker who can measure both ends of a connection through the Tor network -% I say 'measure' rather than 'observe', to encompass murdoch-danezis -% style attacks. -RD -can correlate the timing and volume of data on that connection as it enters -and leaves the network, and so link communication partners. -Known solutions to this attack would seem to require introducing a -prohibitive degree of traffic padding between the user and the network, or -introducing an unacceptable degree of latency (but see Section -\ref{subsec:mid-latency}). Also, it is not clear that these methods would -work at all against a minimally active adversary who could introduce timing -patterns or additional traffic. Thus, Tor only attempts to defend against -external observers who cannot observe both sides of a user's connections. - - -Against internal attackers who sign up Tor nodes, the situation is more -complicated. In the simplest case, if an adversary has compromised $c$ of -$n$ nodes on the Tor network, then the adversary will be able to compromise -a random circuit with probability $\frac{c^2}{n^2}$ (since the circuit -initiator chooses hops randomly). But there are -complicating factors: -(1)~If the user continues to build random circuits over time, an adversary - is pretty certain to see a statistical sample of the user's traffic, and - thereby can build an increasingly accurate profile of her behavior. (See - Section~\ref{subsec:helper-nodes} for possible solutions.) -(2)~An adversary who controls a popular service outside the Tor network - can be certain to observe all connections to that service; he - can therefore trace connections to that service with probability - $\frac{c}{n}$. -(3)~Users do not in fact choose nodes with uniform probability; they - favor nodes with high bandwidth or uptime, and exit nodes that - permit connections to their favorite services. -(See Section~\ref{subsec:routing-zones} for discussion of larger -adversaries and our dispersal goals.) - -% I'm trying to make this paragraph work without reference to the -% analysis/confirmation distinction, which we haven't actually introduced -% yet, and which we realize isn't very stable anyway. Also, I don't want to -% deprecate these attacks if we can't demonstrate that they don't work, since -% in case they *do* turn out to work well against Tor, we'll look pretty -% foolish. -NM -More powerful attacks may exist. In \cite{hintz-pet02} it was -shown that an attacker who can catalog data volumes of popular -responder destinations (say, websites with consistent data volumes) may not -need to -observe both ends of a stream to learn source-destination links for those -responders. -Similarly, latencies of going through various routes can be -cataloged~\cite{back01} to connect endpoints. -% Also, \cite{kesdogan:pet2002} takes the -% attack another level further, to narrow down where you could be -% based on an intersection attack on subpages in a website. -RD -It has not yet been shown whether these attacks will succeed or fail -in the presence of the variability and volume quantization introduced by the -Tor network, but it seems likely that these factors will at best delay -rather than halt the attacks in the cases where they succeed. -Along similar lines, the same paper suggests a ``clogging -attack'' in which the throughput on a circuit is observed to slow -down when an adversary clogs the right nodes with his own traffic. -To determine the nodes in a circuit this attack requires the ability -to continuously monitor the traffic exiting the network on a circuit -that is up long enough to probe all network nodes in binary fashion. -% Though somewhat related, clogging and interference are really different -% attacks with different assumptions about adversary distribution and -% capabilities as well as different techniques. -pfs -Murdoch and Danezis~\cite{attack-tor-oak05} show a practical -interference attack against portions of -the fifty node Tor network as deployed in mid 2004. -An outside attacker can actively trace a circuit through the Tor network -by observing changes in the latency of his -own traffic sent through various Tor nodes. This can be done -simultaneously at multiple nodes; however, like clogging, -this attack only reveals -the Tor nodes in the circuit, not initiator and responder addresses, -so it is still necessary to discover the endpoints to complete an -effective attack. Increasing the size and diversity of the Tor network may -help counter these attacks. - -%discuss $\frac{c^2}{n^2}$, except how in practice the chance of owning -%the last hop is not $c/n$ since that doesn't take the destination (website) -%into account. so in cases where the adversary does not also control the -%final destination we're in good shape, but if he *does* then we'd be better -%off with a system that lets each hop choose a path. -% -%Isn't it more accurate to say ``If the adversary _always_ controls the final -% dest, we would be just as well off with such as system.'' ? If not, why -% not? -nm -% Sure. In fact, better off, since they seem to scale more easily. -rd - -%Murdoch and Danezis describe an attack -%\cite{attack-tor-oak05} that lets an attacker determine the nodes used -%in a circuit; yet s/he cannot identify the initiator or responder, -%e.g., client or web server, through this attack. So the endpoints -%remain secure, which is the goal. It is conceivable that an -%adversary could attack or set up observation of all connections -%to an arbitrary Tor node in only a few minutes. If such an adversary -%were to exist, s/he could use this probing to remotely identify a node -%for further attack. Of more likely immediate practical concern -%an adversary with active access to the responder traffic -%wants to keep a circuit alive long enough to attack an identified -%node. Thus it is important to prevent the responding end of the circuit -%from keeping it open indefinitely. -%Also, someone could identify nodes in this way and if in their -%jurisdiction, immediately get a subpoena (if they even need one) -%telling the node operator(s) that she must retain all the active -%circuit data she now has. -%Further, the enclave model, which had previously looked to be the most -%generally secure, seems particularly threatened by this attack, since -%it identifies endpoints when they're also nodes in the Tor network: -%see Section~\ref{subsec:helper-nodes} for discussion of some ways to -%address this issue. - -\medskip -\noindent -{\bf Distributed trust.} -In practice Tor's threat model is based on -dispersal and diversity. -Our defense lies in having a diverse enough set of nodes -to prevent most real-world -adversaries from being in the right places to attack users, -by distributing each transaction -over several nodes in the network. This ``distributed trust'' approach -means the Tor network can be safely operated and used by a wide variety -of mutually distrustful users, providing sustainability and security. -%than some previous attempts at anonymizing networks. - -No organization can achieve this security on its own. If a single -corporation or government agency were to build a private network to -protect its operations, any connections entering or leaving that network -would be obviously linkable to the controlling organization. The members -and operations of that agency would be easier, not harder, to distinguish. - -Instead, to protect our networks from traffic analysis, we must -collaboratively blend the traffic from many organizations and private -citizens, so that an eavesdropper can't tell which users are which, -and who is looking for what information. %By bringing more users onto -%the network, all users become more secure~\cite{econymics}. -%[XXX I feel uncomfortable saying this last sentence now. -RD] -%[So, I took it out. I think we can do without it. -PFS] -The Tor network has a broad range of users, including ordinary citizens -concerned about their privacy, corporations -who don't want to reveal information to their competitors, and law -enforcement and government intelligence agencies who need -to do operations on the Internet without being noticed. -Naturally, organizations will not want to depend on others for their -security. If most participating providers are reliable, Tor tolerates -some hostile infiltration of the network. For maximum protection, -the Tor design includes an enclave approach that lets data be encrypted -(and authenticated) end-to-end, so high-sensitivity users can be sure it -hasn't been read or modified. This even works for Internet services that -don't have built-in encryption and authentication, such as unencrypted -HTTP or chat, and it requires no modification of those services. - -\subsection{Related work} -Tor differs from other deployed systems for traffic analysis resistance -in its security and flexibility. Mix networks such as -Mixmaster~\cite{mixmaster-spec} or its successor Mixminion~\cite{minion-design} -gain the highest degrees of anonymity at the expense of introducing highly -variable delays, making them unsuitable for applications such as web -browsing. Commercial single-hop -proxies~\cite{anonymizer} can provide good performance, but -a single compromise can expose all users' traffic, and a single-point -eavesdropper can perform traffic analysis on the entire network. -%Also, their proprietary implementations place any infrastructure that -%depends on these single-hop solutions at the mercy of their providers' -%financial health as well as network security. -The Java -Anon Proxy~\cite{web-mix} provides similar functionality to Tor but -handles only web browsing rather than all TCP\@. -%Some peer-to-peer file-sharing overlay networks such as -%Freenet~\cite{freenet} and Mute~\cite{mute} -The Freedom -network from Zero-Knowledge Systems~\cite{freedom21-security} -was even more flexible than Tor in -transporting arbitrary IP packets, and also supported -pseudonymity in addition to anonymity; but it had -a different approach to sustainability (collecting money from users -and paying ISPs to run Tor nodes), and was eventually shut down due to financial -load. Finally, %potentially more scalable -% [I had added 'potentially' because the scalability of these designs -% is not established, and I am uncomfortable making the -% bolder unmodified assertion. Roger took 'potentially' out. -% Here's an attempt at more neutral wording -pfs] -peer-to-peer designs that are intended to be more scalable, -for example Tarzan~\cite{tarzan:ccs02} and -MorphMix~\cite{morphmix:fc04}, have been proposed in the literature but -have not been fielded. These systems differ somewhat -in threat model and presumably practical resistance to threats. -Note that MorphMix differs from Tor only in -node discovery and circuit setup; so Tor's architecture is flexible -enough to contain a MorphMix experiment. -We direct the interested reader -to~\cite{tor-design} for a more in-depth review of related work. - -%XXXX six-four. crowds. i2p. - -%XXXX -%have a serious discussion of morphmix's assumptions, since they would -%seem to be the direct competition. in fact tor is a flexible architecture -%that would encompass morphmix, and they're nearly identical except for -%path selection and node discovery. and the trust system morphmix has -%seems overkill (and/or insecure) based on the threat model we've picked. -% this para should probably move to the scalability / directory system. -RD -% Nope. Cut for space, except for small comment added above -PFS - -\section{Social challenges} - -Many of the issues the Tor project needs to address extend beyond -system design and technology development. In particular, the -Tor project's \emph{image} with respect to its users and the rest of -the Internet impacts the security it can provide. -With this image issue in mind, this section discusses the Tor user base and -Tor's interaction with other services on the Internet. - -\subsection{Communicating security} - -Usability for anonymity systems -contributes to their security, because usability -affects the possible anonymity set~\cite{econymics,back01}. -Conversely, an unusable system attracts few users and thus can't provide -much anonymity. - -This phenomenon has a second-order effect: knowing this, users should -choose which anonymity system to use based in part on how usable -and secure -\emph{others} will find it, in order to get the protection of a larger -anonymity set. Thus we might supplement the adage ``usability is a security -parameter''~\cite{back01} with a new one: ``perceived usability is a -security parameter.'' From here we can better understand the effects -of publicity on security: the more convincing your -advertising, the more likely people will believe you have users, and thus -the more users you will attract. Perversely, over-hyped systems (if they -are not too broken) may be a better choice than modestly promoted ones, -if the hype attracts more users~\cite{usability-network-effect}. - -So it follows that we should come up with ways to accurately communicate -the available security levels to the user, so she can make informed -decisions. JAP aims to do this by including a -comforting `anonymity meter' dial in the software's graphical interface, -giving the user an impression of the level of protection for her current -traffic. - -However, there's a catch. For users to share the same anonymity set, -they need to act like each other. An attacker who can distinguish -a given user's traffic from the rest of the traffic will not be -distracted by anonymity set size. For high-latency systems like -Mixminion, where the threat model is based on mixing messages with each -other, there's an arms race between end-to-end statistical attacks and -counter-strategies~\cite{statistical-disclosure,minion-design,e2e-traffic,trickle02}. -But for low-latency systems like Tor, end-to-end \emph{traffic -correlation} attacks~\cite{danezis:pet2004,defensive-dropping,SS03} -allow an attacker who can observe both ends of a communication -to correlate packet timing and volume, quickly linking -the initiator to her destination. - -Like Tor, the current JAP implementation does not pad connections -apart from using small fixed-size cells for transport. In fact, -JAP's cascade-based network topology may be more vulnerable to these -attacks, because its network has fewer edges. JAP was born out of -the ISDN mix design~\cite{isdn-mixes}, where padding made sense because -every user had a fixed bandwidth allocation and altering the timing -pattern of packets could be immediately detected. But in its current context -as an Internet web anonymizer, adding sufficient padding to JAP -would probably be prohibitively expensive and ineffective against a -minimally active attacker.\footnote{Even if JAP could -fund higher-capacity nodes indefinitely, our experience -suggests that many users would not accept the increased per-user -bandwidth requirements, leading to an overall much smaller user base. But -see Section~\ref{subsec:mid-latency}.} Therefore, since under this threat -model the number of concurrent users does not seem to have much impact -on the anonymity provided, we suggest that JAP's anonymity meter is not -accurately communicating security levels to its users. - -On the other hand, while the number of active concurrent users may not -matter as much as we'd like, it still helps to have some other users -on the network. We investigate this issue next. - -\subsection{Reputability and perceived social value} -Another factor impacting the network's security is its reputability: -the perception of its social value based on its current user base. If Alice is -the only user who has ever downloaded the software, it might be socially -accepted, but she's not getting much anonymity. Add a thousand -activists, and she's anonymous, but everyone thinks she's an activist too. -Add a thousand -diverse citizens (cancer survivors, privacy enthusiasts, and so on) -and now she's harder to profile. - -Furthermore, the network's reputability affects its operator base: more people -are willing to run a service if they believe it will be used by human rights -workers than if they believe it will be used exclusively for disreputable -ends. This effect becomes stronger if node operators themselves think they -will be associated with their users' disreputable ends. - -So the more cancer survivors on Tor, the better for the human rights -activists. The more malicious hackers, the worse for the normal users. Thus, -reputability is an anonymity issue for two reasons. First, it impacts -the sustainability of the network: a network that's always about to be -shut down has difficulty attracting and keeping adequate nodes. -Second, a disreputable network is more vulnerable to legal and -political attacks, since it will attract fewer supporters. - -While people therefore have an incentive for the network to be used for -``more reputable'' activities than their own, there are still trade-offs -involved when it comes to anonymity. To follow the above example, a -network used entirely by cancer survivors might welcome file sharers -onto the network, though of course they'd prefer a wider -variety of users. - -Reputability becomes even more tricky in the case of privacy networks, -since the good uses of the network (such as publishing by journalists in -dangerous countries) are typically kept private, whereas network abuses -or other problems tend to be more widely publicized. - -The impact of public perception on security is especially important -during the bootstrapping phase of the network, where the first few -widely publicized uses of the network can dictate the types of users it -attracts next. -As an example, some U.S.~Department of Energy -penetration testing engineers are tasked with compromising DoE computers -from the outside. They only have a limited number of ISPs from which to -launch their attacks, and they found that the defenders were recognizing -attacks because they came from the same IP space. These engineers wanted -to use Tor to hide their tracks. First, from a technical standpoint, -Tor does not support the variety of IP packets one would like to use in -such attacks (see Section~\ref{subsec:tcp-vs-ip}). But aside from this, -we also decided that it would probably be poor precedent to encourage -such use---even legal use that improves national security---and managed -to dissuade them. - -%% "outside of academia, jap has just lost, permanently". (That is, -%% even though the crime detection issues are resolved and are unlikely -%% to go down the same way again, public perception has not been kind.) - -\subsection{Sustainability and incentives} -One of the unsolved problems in low-latency anonymity designs is -how to keep the nodes running. ZKS's Freedom network -depended on paying third parties to run its servers; the JAP project's -bandwidth depends on grants to pay for its bandwidth and -administrative expenses. In Tor, bandwidth and administrative costs are -distributed across the volunteers who run Tor nodes, so we at least have -reason to think that the Tor network could survive without continued research -funding.\footnote{It also helps that Tor is implemented with free and open - source software that can be maintained by anybody with the ability and - inclination.} But why are these volunteers running nodes, and what can we -do to encourage more volunteers to do so? - -We have not formally surveyed Tor node operators to learn why they are -running nodes, but -from the information they have provided, it seems that many of them run Tor -nodes for reasons of personal interest in privacy issues. It is possible -that others are running Tor nodes to protect their own -anonymity, but of course they are -hardly likely to tell us specifics if they are. -%Significantly, Tor's threat model changes the anonymity incentives for running -%a node. In a high-latency mix network, users can receive additional -%anonymity by running their own node, since doing so obscures when they are -%injecting messages into the network. But, anybody observing all I/O to a Tor -%node can tell when the node is generating traffic that corresponds to -%none of its incoming traffic. -% -%I didn't buy the above for reason's subtle enough that I just cut it -PFS -Tor exit node operators do attain a degree of -``deniability'' for traffic that originates at that exit node. For - example, it is likely in practice that HTTP requests from a Tor node's IP - will be assumed to be from the Tor network. - More significantly, people and organizations who use Tor for - anonymity depend on the - continued existence of the Tor network to do so; running a node helps to - keep the network operational. -%\item Local Tor entry and exit nodes allow users on a network to run in an -% `enclave' configuration. [XXXX need to resolve this. They would do this -% for E2E encryption + auth?] - - -%We must try to make the costs of running a Tor node easily minimized. -Since Tor is run by volunteers, the most crucial software usability issue is -usability by operators: when an operator leaves, the network becomes less -usable by everybody. To keep operators pleased, we must try to keep Tor's -resource and administrative demands as low as possible. - -Because of ISP billing structures, many Tor operators have underused capacity -that they are willing to donate to the network, at no additional monetary -cost to them. Features to limit bandwidth have been essential to adoption. -Also useful has been a ``hibernation'' feature that allows a Tor node that -wants to provide high bandwidth, but no more than a certain amount in a -giving billing cycle, to become dormant once its bandwidth is exhausted, and -to reawaken at a random offset into the next billing cycle. This feature has -interesting policy implications, however; see -the next section below. -Exit policies help to limit administrative costs by limiting the frequency of -abuse complaints (see Section~\ref{subsec:tor-and-blacklists}). We discuss -technical incentive mechanisms in Section~\ref{subsec:incentives-by-design}. - -%[XXXX say more. Why else would you run a node? What else can we do/do we -% already do to make running a node more attractive?] -%[We can enforce incentives; see Section 6.1. We can rate-limit clients. -% We can put "top bandwidth nodes lists" up a la seti@home.] - -\subsection{Bandwidth and file-sharing} -\label{subsec:bandwidth-and-file-sharing} -%One potentially problematical area with deploying Tor has been our response -%to file-sharing applications. -Once users have configured their applications to work with Tor, the largest -remaining usability issue is performance. Users begin to suffer -when websites ``feel slow.'' -Clients currently try to build their connections through nodes that they -guess will have enough bandwidth. But even if capacity is allocated -optimally, it seems unlikely that the current network architecture will have -enough capacity to provide every user with as much bandwidth as she would -receive if she weren't using Tor, unless far more nodes join the network. - -%Limited capacity does not destroy the network, however. Instead, usage tends -%towards an equilibrium: when performance suffers, users who value performance -%over anonymity tend to leave the system, thus freeing capacity until the -%remaining users on the network are exactly those willing to use that capacity -%there is. - -Much of Tor's recent bandwidth difficulties have come from file-sharing -applications. These applications provide two challenges to -any anonymizing network: their intensive bandwidth requirement, and the -degree to which they are associated (correctly or not) with copyright -infringement. - -High-bandwidth protocols can make the network unresponsive, -but tend to be somewhat self-correcting as lack of bandwidth drives away -users who need it. Issues of copyright violation, -however, are more interesting. Typical exit node operators want to help -people achieve private and anonymous speech, not to help people (say) host -Vin Diesel movies for download; and typical ISPs would rather not -deal with customers who draw menacing letters -from the MPAA\@. While it is quite likely that the operators are doing nothing -illegal, many ISPs have policies of dropping users who get repeated legal -threats regardless of the merits of those threats, and many operators would -prefer to avoid receiving even meritless legal threats. -So when letters arrive, operators are likely to face -pressure to block file-sharing applications entirely, in order to avoid the -hassle. - -But blocking file-sharing is not easy: popular -protocols have evolved to run on non-standard ports to -get around other port-based bans. Thus, exit node operators who want to -block file-sharing would have to find some way to integrate Tor with a -protocol-aware exit filter. This could be a technically expensive -undertaking, and one with poor prospects: it is unlikely that Tor exit nodes -would succeed where so many institutional firewalls have failed. Another -possibility for sensitive operators is to run a restrictive node that -only permits exit connections to a restricted range of ports that are -not frequently associated with file sharing. There are increasingly few such -ports. - -Other possible approaches might include rate-limiting connections, especially -long-lived connections or connections to file-sharing ports, so that -high-bandwidth connections do not flood the network. We might also want to -give priority to cells on low-bandwidth connections to keep them interactive, -but this could have negative anonymity implications. - -For the moment, it seems that Tor's bandwidth issues have rendered it -unattractive for bulk file-sharing traffic; this may continue to be so in the -future. Nevertheless, Tor will likely remain attractive for limited use in -file-sharing protocols that have separate control and data channels. - -%[We should say more -- but what? That we'll see a similar -% equilibriating effect as with bandwidth, where sensitive ops switch to -% middleman, and we become less useful for file-sharing, so the file-sharing -% people back off, so we get more ops since there's less file-sharing, so the -% file-sharers come back, etc.] - -%XXXX -%in practice, plausible deniability is hypothetical and doesn't seem very -%convincing. if ISPs find the activity antisocial, they don't care *why* -%your computer is doing that behavior. - -\subsection{Tor and blacklists} -\label{subsec:tor-and-blacklists} - -It was long expected that, alongside legitimate users, Tor would also -attract troublemakers who exploit Tor to abuse services on the -Internet with vandalism, rude mail, and so on. -Our initial answer to this situation was to use ``exit policies'' -to allow individual Tor nodes to block access to specific IP/port ranges. -This approach aims to make operators more willing to run Tor by allowing -them to prevent their nodes from being used for abusing particular -services. For example, all Tor nodes currently block SMTP (port 25), -to avoid being used for spam. - -Exit policies are useful, but they are insufficient: if not all nodes -block a given service, that service may try to block Tor instead. -While being blockable is important to being good netizens, we would like -to encourage services to allow anonymous access. Services should not -need to decide between blocking legitimate anonymous use and allowing -unlimited abuse. - -This is potentially a bigger problem than it may appear. -On the one hand, services should be allowed to refuse connections from -sources of possible abuse. -But when a Tor node administrator decides whether he prefers to be able -to post to Wikipedia from his IP address, or to allow people to read -Wikipedia anonymously through his Tor node, he is making the decision -for others as well. (For a while, Wikipedia -blocked all posting from all Tor nodes based on IP addresses.) If -the Tor node shares an address with a campus or corporate NAT, -then the decision can prevent the entire population from posting. -This is a loss for both Tor -and Wikipedia: we don't want to compete for (or divvy up) the -NAT-protected entities of the world. - -Worse, many IP blacklists are coarse-grained: they ignore Tor's exit -policies, partly because it's easier to implement and partly -so they can punish -all Tor nodes. One IP blacklist even bans -every class C network that contains a Tor node, and recommends banning SMTP -from these networks even though Tor does not allow SMTP at all. This -strategic decision aims to discourage the -operation of anything resembling an open proxy by encouraging its neighbors -to shut it down to get unblocked themselves. This pressure even -affects Tor nodes running in middleman mode (disallowing all exits) when -those nodes are blacklisted too. - -Problems of abuse occur mainly with services such as IRC networks and -Wikipedia, which rely on IP blocking to ban abusive users. While at first -blush this practice might seem to depend on the anachronistic assumption that -each IP is an identifier for a single user, it is actually more reasonable in -practice: it assumes that non-proxy IPs are a costly resource, and that an -abuser can not change IPs at will. By blocking IPs which are used by Tor -nodes, open proxies, and service abusers, these systems hope to make -ongoing abuse difficult. Although the system is imperfect, it works -tolerably well for them in practice. - -Of course, we would prefer that legitimate anonymous users be able to -access abuse-prone services. One conceivable approach would require -would-be IRC users, for instance, to register accounts if they want to -access the IRC network from Tor. In practice this would not -significantly impede abuse if creating new accounts were easily automatable; -this is why services use IP blocking. To deter abuse, pseudonymous -identities need to require a significant switching cost in resources or human -time. Some popular webmail applications -impose cost with Reverse Turing Tests, but this step may not deter all -abusers. Freedom used blind signatures to limit -the number of pseudonyms for each paying account, but Tor has neither the -ability nor the desire to collect payment. - -We stress that as far as we can tell, most Tor uses are not -abusive. Most services have not complained, and others are actively -working to find ways besides banning to cope with the abuse. For example, -the Freenode IRC network had a problem with a coordinated group of -abusers joining channels and subtly taking over the conversation; but -when they labelled all users coming from Tor IPs as ``anonymous users,'' -removing the ability of the abusers to blend in, the abuse stopped. - -%The use of squishy IP-based ``authentication'' and ``authorization'' -%has not broken down even to the level that SSNs used for these -%purposes have in commercial and public record contexts. Externalities -%and misplaced incentives cause a continued focus on fighting identity -%theft by protecting SSNs rather than developing better authentication -%and incentive schemes \cite{price-privacy}. Similarly we can expect a -%continued use of identification by IP number as long as there is no -%workable alternative. - -%[XXX Mention correct DNS-RBL implementation. -NM] - -\section{Design choices} - -In addition to social issues, Tor also faces some design trade-offs that must -be investigated as the network develops. - -\subsection{Transporting the stream vs transporting the packets} -\label{subsec:stream-vs-packet} -\label{subsec:tcp-vs-ip} - -Tor transports streams; it does not tunnel packets. -It has often been suggested that like the old Freedom -network~\cite{freedom21-security}, Tor should -``obviously'' anonymize IP traffic -at the IP layer. Before this could be done, many issues need to be resolved: - -\begin{enumerate} -\setlength{\itemsep}{0mm} -\setlength{\parsep}{0mm} -\item \emph{IP packets reveal OS characteristics.} We would still need to do -IP-level packet normalization, to stop things like TCP fingerprinting -attacks. %There likely exist libraries that can help with this. -This is unlikely to be a trivial task, given the diversity and complexity of -TCP stacks. -\item \emph{Application-level streams still need scrubbing.} We still need -Tor to be easy to integrate with user-level application-specific proxies -such as Privoxy. So it's not just a matter of capturing packets and -anonymizing them at the IP layer. -\item \emph{Certain protocols will still leak information.} For example, we -must rewrite DNS requests so they are delivered to an unlinkable DNS server -rather than the DNS server at a user's ISP; thus, we must understand the -protocols we are transporting. -\item \emph{The crypto is unspecified.} First we need a block-level encryption -approach that can provide security despite -packet loss and out-of-order delivery. Freedom allegedly had one, but it was -never publicly specified. -Also, TLS over UDP is not yet implemented or -specified, though some early work has begun~\cite{dtls}. -\item \emph{We'll still need to tune network parameters.} Since the above -encryption system will likely need sequence numbers (and maybe more) to do -replay detection, handle duplicate frames, and so on, we will be reimplementing -a subset of TCP anyway---a notoriously tricky path. -\item \emph{Exit policies for arbitrary IP packets mean building a secure -IDS\@.} Our node operators tell us that exit policies are one of -the main reasons they're willing to run Tor. -Adding an Intrusion Detection System to handle exit policies would -increase the security complexity of Tor, and would likely not work anyway, -as evidenced by the entire field of IDS and counter-IDS papers. Many -potential abuse issues are resolved by the fact that Tor only transports -valid TCP streams (as opposed to arbitrary IP including malformed packets -and IP floods), so exit policies become even \emph{more} important as -we become able to transport IP packets. We also need to compactly -describe exit policies so clients can predict -which nodes will allow which packets to exit. -\item \emph{The Tor-internal name spaces would need to be redesigned.} We -support hidden service {\tt{.onion}} addresses (and other special addresses, -like {\tt{.exit}} which lets the user request a particular exit node), -by intercepting the addresses when they are passed to the Tor client. -Doing so at the IP level would require a more complex interface between -Tor and the local DNS resolver. -\end{enumerate} - -This list is discouragingly long, but being able to transport more -protocols obviously has some advantages. It would be good to learn which -items are actual roadblocks and which are easier to resolve than we think. - -To be fair, Tor's stream-based approach has run into -stumbling blocks as well. While Tor supports the SOCKS protocol, -which provides a standardized interface for generic TCP proxies, many -applications do not support SOCKS\@. For them we already need to -replace the networking system calls with SOCKS-aware -versions, or run a SOCKS tunnel locally, neither of which is -easy for the average user. %---even with good instructions. -Even when applications can use SOCKS, they often make DNS requests -themselves before handing an IP address to Tor, which advertises -where the user is about to connect. -We are still working on more usable solutions. - -%So to actually provide good anonymity, we need to make sure that -%users have a practical way to use Tor anonymously. Possibilities include -%writing wrappers for applications to anonymize them automatically; improving -%the applications' support for SOCKS; writing libraries to help application -%writers use Tor properly; and implementing a local DNS proxy to reroute DNS -%requests to Tor so that applications can simply point their DNS resolvers at -%localhost and continue to use SOCKS for data only. - -\subsection{Mid-latency} -\label{subsec:mid-latency} - -Some users need to resist traffic correlation attacks. Higher-latency -mix-networks introduce variability into message -arrival times: as timing variance increases, timing correlation attacks -require increasingly more data~\cite{e2e-traffic}. Can we improve Tor's -resistance without losing too much usability? - -We need to learn whether we can trade a small increase in latency -for a large anonymity increase, or if we'd end up trading a lot of -latency for only a minimal security gain. A trade-off might be worthwhile -even if we -could only protect certain use cases, such as infrequent short-duration -transactions. % To answer this question -We might adapt the techniques of~\cite{e2e-traffic} to a lower-latency mix -network, where the messages are batches of cells in temporally clustered -connections. These large fixed-size batches can also help resist volume -signature attacks~\cite{hintz-pet02}. We could also experiment with traffic -shaping to get a good balance of throughput and security. -%Other padding regimens might supplement the -%mid-latency option; however, we should continue the caution with which -%we have always approached padding lest the overhead cost us too much -%performance or too many volunteers. - -We must keep usability in mind too. How much can latency increase -before we drive users away? We've already been forced to increase -latency slightly, as our growing network incorporates more DSL and -cable-modem nodes and more nodes in distant continents. Perhaps we can -harness this increased latency to improve anonymity rather than just -reduce usability. Further, if we let clients label certain circuits as -mid-latency as they are constructed, we could handle both types of traffic -on the same network, giving users a choice between speed and security---and -giving researchers a chance to experiment with parameters to improve the -quality of those choices. - -\subsection{Enclaves and helper nodes} -\label{subsec:helper-nodes} - -It has long been thought that users can improve their anonymity by -running their own node~\cite{tor-design,or-ih96,or-pet00}, and using -it in an \emph{enclave} configuration, where all their circuits begin -at the node under their control. Running Tor clients or servers at -the enclave perimeter is useful when policy or other requirements -prevent individual machines within the enclave from running Tor -clients~\cite{or-jsac98,or-discex00}. - -Of course, Tor's default path length of -three is insufficient for these enclaves, since the entry and/or exit -% [edit war: without the ``and/'' the natural reading here -% is aut rather than vel. And the use of the plural verb does not work -pfs] -themselves are sensitive. Tor thus increments path length by one -for each sensitive endpoint in the circuit. -Enclaves also help to protect against end-to-end attacks, since it's -possible that traffic coming from the node has simply been relayed from -elsewhere. However, if the node has recognizable behavior patterns, -an attacker who runs nodes in the network can triangulate over time to -gain confidence that it is in fact originating the traffic. Wright et -al.~\cite{wright03} introduce the notion of a \emph{helper node}---a -single fixed entry node for each user---to combat this \emph{predecessor -attack}. - -However, the attack in~\cite{attack-tor-oak05} shows that simply adding -to the path length, or using a helper node, may not protect an enclave -node. A hostile web server can send constant interference traffic to -all nodes in the network, and learn which nodes are involved in the -circuit (though at least in the current attack, he can't learn their -order). Using randomized path lengths may help some, since the attacker -will never be certain he has identified all nodes in the path unless -he probes the entire network, but as -long as the network remains small this attack will still be feasible. - -Helper nodes also aim to help Tor clients, because choosing entry and exit -points -randomly and changing them frequently allows an attacker who controls -even a few nodes to eventually link some of their destinations. The goal -is to take the risk once and for all about choosing a bad entry node, -rather than taking a new risk for each new circuit. (Choosing fixed -exit nodes is less useful, since even an honest exit node still doesn't -protect against a hostile website.) But obstacles remain before -we can implement helper nodes. -For one, the literature does not describe how to choose helpers from a list -of nodes that changes over time. If Alice is forced to choose a new entry -helper every $d$ days and $c$ of the $n$ nodes are bad, she can expect -to choose a compromised node around -every $dc/n$ days. Statistically over time this approach only helps -if she is better at choosing honest helper nodes than at choosing -honest nodes. Worse, an attacker with the ability to DoS nodes could -force users to switch helper nodes more frequently, or remove -other candidate helpers. - -%Do general DoS attacks have anonymity implications? See e.g. Adam -%Back's IH paper, but I think there's more to be pointed out here. -RD -% Not sure what you want to say here. -NM - -%Game theory for helper nodes: if Alice offers a hidden service on a -%server (enclave model), and nobody ever uses helper nodes, then against -%George+Steven's attack she's totally nailed. If only Alice uses a helper -%node, then she's still identified as the source of the data. If everybody -%uses a helper node (including Alice), then the attack identifies the -%helper node and also Alice, and knows which one is which. If everybody -%uses a helper node (but not Alice), then the attacker figures the real -%source was a client that is using Alice as a helper node. [How's my -%logic here?] -RD -% -% Not sure about the logic. For the attack to work with helper nodes, the -%attacker needs to guess that Alice is running the hidden service, right? -%Otherwise, how can he know to measure her traffic specifically? -NM -% -% In the Murdoch-Danezis attack, the adversary measures all servers. -RD - -%point to routing-zones section re: helper nodes to defend against -%big stuff. - -\subsection{Location-hidden services} -\label{subsec:hidden-services} - -% This section is first up against the wall when the revolution comes. - -Tor's \emph{rendezvous points} -let users provide TCP services to other Tor users without revealing -the service's location. Since this feature is relatively recent, we describe -here -a couple of our early observations from its deployment. - -First, our implementation of hidden services seems less hidden than we'd -like, since they build a different rendezvous circuit for each user, -and an external adversary can induce them to -produce traffic. This insecurity means that they may not be suitable as -a building block for Free Haven~\cite{freehaven-berk} or other anonymous -publishing systems that aim to provide long-term security, though helper -nodes, as discussed above, would seem to help. - -\emph{Hot-swap} hidden services, where more than one location can -provide the service and loss of any one location does not imply a -change in service, would help foil intersection and observation attacks -where an adversary monitors availability of a hidden service and also -monitors whether certain users or servers are online. The design -challenges in providing such services without otherwise compromising -the hidden service's anonymity remain an open problem; -however, see~\cite{move-ndss05}. - -In practice, hidden services are used for more than just providing private -access to a web server or IRC server. People are using hidden services -as a poor man's VPN and firewall-buster. Many people want to be able -to connect to the computers in their private network via secure shell, -and rather than playing with dyndns and trying to pierce holes in their -firewall, they run a hidden service on the inside and then rendezvous -with that hidden service externally. - -News sites like Bloggers Without Borders (www.b19s.org) are advertising -a hidden-service address on their front page. Doing this can provide -increased robustness if they use the dual-IP approach we describe -in~\cite{tor-design}, -but in practice they do it to increase visibility -of the Tor project and their support for privacy, and to offer -a way for their users, using unmodified software, to get end-to-end -encryption and authentication to their website. - -\subsection{Location diversity and ISP-class adversaries} -\label{subsec:routing-zones} - -Anonymity networks have long relied on diversity of node location for -protection against attacks---typically an adversary who can observe a -larger fraction of the network can launch a more effective attack. One -way to achieve dispersal involves growing the network so a given adversary -sees less. Alternately, we can arrange the topology so traffic can enter -or exit at many places (for example, by using a free-route network -like Tor rather than a cascade network like JAP). Lastly, we can use -distributed trust to spread each transaction over multiple jurisdictions. -But how do we decide whether two nodes are in related locations? - -Feamster and Dingledine defined a \emph{location diversity} metric -in~\cite{feamster:wpes2004}, and began investigating a variant of location -diversity based on the fact that the Internet is divided into thousands of -independently operated networks called {\em autonomous systems} (ASes). -The key insight from their paper is that while we typically think of a -connection as going directly from the Tor client to the first Tor node, -actually it traverses many different ASes on each hop. An adversary at -any of these ASes can monitor or influence traffic. Specifically, given -plausible initiators and recipients, and given random path selection, -some ASes in the simulation were able to observe 10\% to 30\% of the -transactions (that is, learn both the origin and the destination) on -the deployed Tor network (33 nodes as of June 2004). - -The paper concludes that for best protection against the AS-level -adversary, nodes should be in ASes that have the most links to other ASes: -Tier-1 ISPs such as AT\&T and Abovenet. Further, a given transaction -is safest when it starts or ends in a Tier-1 ISP\@. Therefore, assuming -initiator and responder are both in the U.S., it actually \emph{hurts} -our location diversity to use far-flung nodes in -continents like Asia or South America. -% it's not just entering or exiting from them. using them as the middle -% hop reduces your effective path length, which you presumably don't -% want because you chose that path length for a reason. -% -% Not sure I buy that argument. Two end nodes in the right ASs to -% discourage linking are still not known to each other. If some -% adversary in a single AS can bridge the middle node, it shouldn't -% therefore be able to identify initiator or responder; although it could -% contribute to further attacks given more assumptions. -% Nonetheless, no change to the actual text for now. - -Many open questions remain. First, it will be an immense engineering -challenge to get an entire BGP routing table to each Tor client, or to -summarize it sufficiently. Without a local copy, clients won't be -able to safely predict what ASes will be traversed on the various paths -through the Tor network to the final destination. Tarzan~\cite{tarzan:ccs02} -and MorphMix~\cite{morphmix:fc04} suggest that we compare IP prefixes to -determine location diversity; but the above paper showed that in practice -many of the Mixmaster nodes that share a single AS have entirely different -IP prefixes. When the network has scaled to thousands of nodes, does IP -prefix comparison become a more useful approximation? % Alternatively, can -%relevant parts of the routing tables be summarized centrally and delivered to -%clients in a less verbose format? -%% i already said "or to summarize is sufficiently" above. is that not -%% enough? -RD -% -Second, we can take advantage of caching certain content at the -exit nodes, to limit the number of requests that need to leave the -network at all. What about taking advantage of caches like Akamai or -Google~\cite{shsm03}? (Note that they're also well-positioned as global -adversaries.) -% -Third, if we follow the recommendations in~\cite{feamster:wpes2004} - and tailor path selection -to avoid choosing endpoints in similar locations, how much are we hurting -anonymity against larger real-world adversaries who can take advantage -of knowing our algorithm? -% -Fourth, can we use this knowledge to figure out which gaps in our network -most affect our robustness to this class of attack, and go recruit -new nodes with those ASes in mind? - -%Tor's security relies in large part on the dispersal properties of its -%network. We need to be more aware of the anonymity properties of various -%approaches so we can make better design decisions in the future. - -\subsection{The Anti-censorship problem} -\label{subsec:china} - -Citizens in a variety of countries, such as most recently China and -Iran, are blocked from accessing various sites outside -their country. These users try to find any tools available to allow -them to get around these firewalls. Some anonymity networks, such as -Six-Four~\cite{six-four}, are designed specifically with this goal in -mind; others like the Anonymizer~\cite{anonymizer} are paid by sponsors -such as Voice of America to encourage Internet -freedom. Even though Tor wasn't -designed with ubiquitous access to the network in mind, thousands of -users across the world are now using it for exactly this purpose. -% Academic and NGO organizations, peacefire, \cite{berkman}, etc - -Anti-censorship networks hoping to bridge country-level blocks face -a variety of challenges. One of these is that they need to find enough -exit nodes---servers on the `free' side that are willing to relay -traffic from users to their final destinations. Anonymizing -networks like Tor are well-suited to this task since we have -already gathered a set of exit nodes that are willing to tolerate some -political heat. - -The other main challenge is to distribute a list of reachable relays -to the users inside the country, and give them software to use those relays, -without letting the censors also enumerate this list and block each -relay. Anonymizer solves this by buying lots of seemingly-unrelated IP -addresses (or having them donated), abandoning old addresses as they are -`used up,' and telling a few users about the new ones. Distributed -anonymizing networks again have an advantage here, in that we already -have tens of thousands of separate IP addresses whose users might -volunteer to provide this service since they've already installed and use -the software for their own privacy~\cite{koepsell:wpes2004}. Because -the Tor protocol separates routing from network discovery \cite{tor-design}, -volunteers could configure their Tor clients -to generate node descriptors and send them to a special directory -server that gives them out to dissidents who need to get around blocks. - -Of course, this still doesn't prevent the adversary -from enumerating and preemptively blocking the volunteer relays. -Perhaps a tiered-trust system could be built where a few individuals are -given relays' locations. They could then recommend other individuals -by telling them -those addresses, thus providing a built-in incentive to avoid letting the -adversary intercept them. Max-flow trust algorithms~\cite{advogato} -might help to bound the number of IP addresses leaked to the adversary. Groups -like the W3C are looking into using Tor as a component in an overall system to -help address censorship; we wish them success. - -%\cite{infranet} - -\section{Scaling} -\label{sec:scaling} - -Tor is running today with hundreds of nodes and tens of thousands of -users, but it will certainly not scale to millions. -Scaling Tor involves four main challenges. First, to get a -large set of nodes, we must address incentives for -users to carry traffic for others. Next is safe node discovery, both -while bootstrapping (Tor clients must robustly find an initial -node list) and later (Tor clients must learn about a fair sample -of honest nodes and not let the adversary control circuits). -We must also detect and handle node speed and reliability as the network -becomes increasingly heterogeneous: since the speed and reliability -of a circuit is limited by its worst link, we must learn to track and -predict performance. Finally, we must stop assuming that all points on -the network can connect to all other points. - -\subsection{Incentives by Design} -\label{subsec:incentives-by-design} - -There are three behaviors we need to encourage for each Tor node: relaying -traffic; providing good throughput and reliability while doing it; -and allowing traffic to exit the network from that node. - -We encourage these behaviors through \emph{indirect} incentives: that -is, by designing the system and educating users in such a way that users -with certain goals will choose to relay traffic. One -main incentive for running a Tor node is social: volunteers -altruistically donate their bandwidth and time. We encourage this with -public rankings of the throughput and reliability of nodes, much like -seti@home. We further explain to users that they can get -deniability for any traffic emerging from the same address as a Tor -exit node, and they can use their own Tor node -as an entry or exit point with confidence that it's not run by an adversary. -Further, users may run a node simply because they need such a network -to be persistently available and usable, and the value of supporting this -exceeds any countervening costs. -Finally, we can encourage operators by improving the usability and feature -set of the software: -rate limiting support and easy packaging decrease the hassle of -maintaining a node, and our configurable exit policies allow each -operator to advertise a policy describing the hosts and ports to which -he feels comfortable connecting. - -To date these incentives appear to have been adequate. As the system scales -or as new issues emerge, however, we may also need to provide - \emph{direct} incentives: -providing payment or other resources in return for high-quality service. -Paying actual money is problematic: decentralized e-cash systems are -not yet practical, and a centralized collection system not only reduces -robustness, but also has failed in the past (the history of commercial -anonymizing networks is littered with failed attempts). A more promising -option is to use a tit-for-tat incentive scheme, where nodes provide better -service to nodes that have provided good service for them. - -Unfortunately, such an approach introduces new anonymity problems. -There are many surprising ways for nodes to game the incentive and -reputation system to undermine anonymity---such systems are typically -designed to encourage fairness in storage or bandwidth usage, not -fairness of provided anonymity. An adversary can attract more traffic -by performing well or can target individual users by selectively -performing, to undermine their anonymity. Typically a user who -chooses evenly from all nodes is most resistant to an adversary -targeting him, but that approach hampers the efficient use -of heterogeneous nodes. - -%When a node (call him Steve) performs well for Alice, does Steve gain -%reputation with the entire system, or just with Alice? If the entire -%system, how does Alice tell everybody about her experience in a way that -%prevents her from lying about it yet still protects her identity? If -%Steve's behavior only affects Alice's behavior, does this allow Steve to -%selectively perform only for Alice, and then break her anonymity later -%when somebody (presumably Alice) routes through his node? - -A possible solution is a simplified approach to the tit-for-tat -incentive scheme based on two rules: (1) each node should measure the -service it receives from adjacent nodes, and provide service relative -to the received service, but (2) when a node is making decisions that -affect its own security (such as building a circuit for its own -application connections), it should choose evenly from a sufficiently -large set of nodes that meet some minimum service -threshold~\cite{casc-rep}. This approach allows us to discourage -bad service -without opening Alice up as much to attacks. All of this requires -further study. - -\subsection{Trust and discovery} -\label{subsec:trust-and-discovery} - -The published Tor design is deliberately simplistic in how -new nodes are authorized and how clients are informed about Tor -nodes and their status. -All nodes periodically upload a signed description -of their locations, keys, and capabilities to each of several well-known {\it - directory servers}. These directory servers construct a signed summary -of all known Tor nodes (a ``directory''), and a signed statement of which -nodes they -believe to be operational then (a ``network status''). Clients -periodically download a directory to learn the latest nodes and -keys, and more frequently download a network status to learn which nodes are -likely to be running. Tor nodes also operate as directory caches, to -lighten the bandwidth on the directory servers. - -To prevent Sybil attacks (wherein an adversary signs up many -purportedly independent nodes to increase her network view), -this design -requires the directory server operators to manually -approve new nodes. Unapproved nodes are included in the directory, -but clients -do not use them at the start or end of their circuits. In practice, -directory administrators perform little actual verification, and tend to -approve any Tor node whose operator can compose a coherent email. -This procedure -may prevent trivial automated Sybil attacks, but will do little -against a clever and determined attacker. - -There are a number of flaws in this system that need to be addressed as we -move forward. First, -each directory server represents an independent point of failure: any -compromised directory server could start recommending only compromised -nodes. -Second, as more nodes join the network, %the more unreasonable it -%becomes to expect clients to know about them all. -directories -become infeasibly large, and downloading the list of nodes becomes -burdensome. -Third, the validation scheme may do as much harm as it does good. It -does not prevent clever attackers from mounting Sybil attacks, -and it may deter node operators from joining the network---if -they expect the validation process to be difficult, or they do not share -any languages in common with the directory server operators. - -We could try to move the system in several directions, depending on our -choice of threat model and requirements. If we did not need to increase -network capacity to support more users, we could simply - adopt even stricter validation requirements, and reduce the number of -nodes in the network to a trusted minimum. -But, we can only do that if we can simultaneously make node capacity -scale much more than we anticipate to be feasible soon, and if we can find -entities willing to run such nodes, an equally daunting prospect. - -In order to address the first two issues, it seems wise to move to a system -including a number of semi-trusted directory servers, no one of which can -compromise a user on its own. Ultimately, of course, we cannot escape the -problem of a first introducer: since most users will run Tor in whatever -configuration the software ships with, the Tor distribution itself will -remain a single point of failure so long as it includes the seed -keys for directory servers, a list of directory servers, or any other means -to learn which nodes are on the network. But omitting this information -from the Tor distribution would only delegate the trust problem to each -individual user. %, most of whom are presumably less informed about how to make -%trust decisions than the Tor developers. -A well publicized, widely available, authoritatively and independently -endorsed and signed list of initial directory servers and their keys -is a possible solution. But, setting that up properly is itself a large -bootstrapping task. - -%Network discovery, sybil, node admission, scaling. It seems that the code -%will ship with something and that's our trust root. We could try to get -%people to build a web of trust, but no. Where we go from here depends -%on what threats we have in mind. Really decentralized if your threat is -%RIAA; less so if threat is to application data or individuals or... - -\subsection{Measuring performance and capacity} -\label{subsec:performance} - -One of the paradoxes with engineering an anonymity network is that we'd like -to learn as much as we can about how traffic flows so we can improve the -network, but we want to prevent others from learning how traffic flows in -order to trace users' connections through the network. Furthermore, many -mechanisms that help Tor run efficiently -require measurements about the network. - -Currently, nodes try to deduce their own available bandwidth (based on how -much traffic they have been able to transfer recently) and include this -information in the descriptors they upload to the directory. Clients -choose servers weighted by their bandwidth, neglecting really slow -servers and capping the influence of really fast ones. -% -This is, of course, eminently cheatable. A malicious node can get a -disproportionate amount of traffic simply by claiming to have more bandwidth -than it does. But better mechanisms have their problems. If bandwidth data -is to be measured rather than self-reported, it is usually possible for -nodes to selectively provide better service for the measuring party, or -sabotage the measured value of other nodes. Complex solutions for -mix networks have been proposed, but do not address the issues -completely~\cite{mix-acc,casc-rep}. - -Even with no cheating, network measurement is complex. It is common -for views of a node's latency and/or bandwidth to vary wildly between -observers. Further, it is unclear whether total bandwidth is really -the right measure; perhaps clients should instead be considering nodes -based on unused bandwidth or observed throughput. -%How to measure performance without letting people selectively deny service -%by distinguishing pings. Heck, just how to measure performance at all. In -%practice people have funny firewalls that don't match up to their exit -%policies and Tor doesn't deal. -% -%Network investigation: Is all this bandwidth publishing thing a good idea? -%How can we collect stats better? Note weasel's smokeping, at -%http://seppia.noreply.org/cgi-bin/smokeping.cgi?target=Tor -%which probably gives george and steven enough info to break tor? -% -And even if we can collect and use this network information effectively, -we must ensure -that it is not more useful to attackers than to us. While it -seems plausible that bandwidth data alone is not enough to reveal -sender-recipient connections under most circumstances, it could certainly -reveal the path taken by large traffic flows under low-usage circumstances. - -\subsection{Non-clique topologies} - -Tor's comparatively weak threat model may allow easier scaling than -other -designs. High-latency mix networks need to avoid partitioning attacks, where -network splits let an attacker distinguish users in different partitions. -Since Tor assumes the adversary cannot cheaply observe nodes at will, -a network split may not decrease protection much. -Thus, one option when the scale of a Tor network -exceeds some size is simply to split it. Nodes could be allocated into -partitions while hampering collaborating hostile nodes from taking over -a single partition~\cite{casc-rep}. -Clients could switch between -networks, even on a per-circuit basis. -%Future analysis may uncover -%other dangers beyond those affecting mix-nets. - -More conservatively, we can try to scale a single Tor network. Likely -problems with adding more servers to a single Tor network include an -explosion in the number of sockets needed on each server as more servers -join, and increased coordination overhead to keep each users' view of -the network consistent. As we grow, we will also have more instances of -servers that can't reach each other simply due to Internet topology or -routing problems. - -%include restricting the number of sockets and the amount of bandwidth -%used by each node. The number of sockets is determined by the network's -%connectivity and the number of users, while bandwidth capacity is determined -%by the total bandwidth of nodes on the network. The simplest solution to -%bandwidth capacity is to add more nodes, since adding a Tor node of any -%feasible bandwidth will increase the traffic capacity of the network. So as -%a first step to scaling, we should focus on making the network tolerate more -%nodes, by reducing the interconnectivity of the nodes; later we can reduce -%overhead associated with directories, discovery, and so on. - -We can address these points by reducing the network's connectivity. -Danezis~\cite{danezis:pet2003} considers -the anonymity implications of restricting routes on mix networks and -recommends an approach based on expander graphs (where any subgraph is likely -to have many neighbors). It is not immediately clear that this approach will -extend to Tor, which has a weaker threat model but higher performance -requirements: instead of analyzing the -probability of an attacker's viewing whole paths, we will need to examine the -attacker's likelihood of compromising the endpoints. -% -Tor may not need an expander graph per se: it -may be enough to have a single central subnet that is highly connected, like -an Internet backbone. % As an -%example, assume fifty nodes of relatively high traffic capacity. This -%\emph{center} forms a clique. Assume each center node can -%handle 200 connections to other nodes (including the other ones in the -%center). Assume every noncenter node connects to three nodes in the -%center and anyone out of the center that they want to. Then the -%network easily scales to c. 2500 nodes with commensurate increase in -%bandwidth. -There are many open questions: how to distribute connectivity information -(presumably nodes will learn about the central nodes -when they download Tor), whether central nodes -will need to function as a `backbone', and so on. As above, -this could reduce the amount of anonymity available from a mix-net, -but for a low-latency network where anonymity derives largely from -the edges, it may be feasible. - -%In a sense, Tor already has a non-clique topology. -%Individuals can set up and run Tor nodes without informing the -%directory servers. This allows groups to run a -%local Tor network of private nodes that connects to the public Tor -%network. This network is hidden behind the Tor network, and its -%only visible connection to Tor is at those points where it connects. -%As far as the public network, or anyone observing it, is concerned, -%they are running clients. - -\section{The Future} -\label{sec:conclusion} - -Tor is the largest and most diverse low-latency anonymity network -available, but we are still in the beginning stages of deployment. Several -major questions remain. - -First, will our volunteer-based approach to sustainability work in the -long term? As we add more features and destabilize the network, the -developers spend a lot of time keeping the server operators happy. Even -though Tor is free software, the network would likely stagnate and die at -this stage if the developers stopped actively working on it. We may get -an unexpected boon from the fact that we're a general-purpose overlay -network: as Tor grows more popular, other groups who need an overlay -network on the Internet are starting to adapt Tor to their needs. -% -Second, Tor is only one of many components that preserve privacy online. -For applications where it is desirable to -keep identifying information out of application traffic, someone must build -more and better protocol-aware proxies that are usable by ordinary people. -% -Third, we need to gain a reputation for social good, and learn how to -coexist with the variety of Internet services and their established -authentication mechanisms. We can't just keep escalating the blacklist -standoff forever. -% -Fourth, the current Tor -architecture does not scale even to handle current user demand. We must -find designs and incentives to let some clients relay traffic too, without -sacrificing too much anonymity. - -These are difficult and open questions. Yet choosing not to solve them -means leaving most users to a less secure network or no anonymizing -network at all. - -\bibliographystyle{plain} \bibliography{tor-design} - -\clearpage -\appendix - -\begin{figure}[t] -%\unitlength=1in -\centering -%\begin{picture}(6.0,2.0) -%\put(3,1){\makebox(0,0)[c]{\epsfig{figure=graphnodes,width=6in}}} -%\end{picture} -\mbox{\epsfig{figure=graphnodes,width=5in}} -\caption{Number of Tor nodes over time, through January 2005. Lowest -line is number of exit -nodes that allow connections to port 80. Middle line is total number of -verified (registered) Tor nodes. The line above that represents nodes -that are running but not yet registered.} -\label{fig:graphnodes} -\end{figure} - -\begin{figure}[t] -\centering -\mbox{\epsfig{figure=graphtraffic,width=5in}} -\caption{The sum of traffic reported by each node over time, through -January 2005. The bottom -pair show average throughput, and the top pair represent the largest 15 -minute burst in each 4 hour period.} -\label{fig:graphtraffic} -\end{figure} - -\end{document} - -%Making use of nodes with little bandwidth, or high latency/packet loss. - -%Running Tor nodes behind NATs, behind great-firewalls-of-China, etc. -%Restricted routes. How to propagate to everybody the topology? BGP -%style doesn't work because we don't want just *one* path. Point to -%Geoff's stuff. - diff --git a/doc/design-paper/challenges2.tex b/doc/design-paper/challenges2.tex deleted file mode 100644 index a39b66cf7..000000000 --- a/doc/design-paper/challenges2.tex +++ /dev/null @@ -1,1612 +0,0 @@ -\documentclass{llncs} - -\usepackage{url} -\usepackage{amsmath} -\usepackage{epsfig} - -\setlength{\textwidth}{5.9in} -\setlength{\textheight}{8.4in} -\setlength{\topmargin}{.5cm} -\setlength{\oddsidemargin}{1cm} -\setlength{\evensidemargin}{1cm} - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} - - -\newcommand{\workingnote}[1]{} % The version that hides the note. -%\newcommand{\workingnote}[1]{(**#1)} % The version that makes the note visible. - - -\begin{document} - -\title{Design challenges and social factors in deploying low-latency anonymity} - -\author{Roger Dingledine\inst{1} \and -Nick Mathewson\inst{1} \and -Paul Syverson\inst{2}} -\institute{The Free Haven Project \email{<\{arma,nickm\}@freehaven.net>} \and -Naval Research Laboratory \email{<syverson@itd.nrl.navy.mil>}} - -\maketitle -\pagestyle{plain} - -\begin{abstract} - There are many unexpected or unexpectedly difficult obstacles to - deploying anonymous communications. We describe the design - philosophy of Tor (the third-generation onion routing network), and, - drawing on our experiences deploying Tor, we describe social - challenges and related technical issues that must be faced in - building, deploying, and sustaining a scalable, distributed, - low-latency anonymity network. -\end{abstract} - -\section{Introduction} -% Your network is not practical unless it is sustainable and distributed. -Anonymous communication is full of surprises. This article describes -Tor, a low-latency general-purpose anonymous communication system, and -discusses some unexpected challenges arising from our experiences -deploying Tor. We will discuss -some of the difficulties we have experienced and how we have met them (or how -we plan to meet them, if we know). -% We also discuss some less -% troublesome open problems that we must nevertheless eventually address. -%We will describe both those future challenges that we intend to explore and -%those that we have decided not to explore and why. - -Tor is an overlay network for anonymizing TCP streams over the -Internet~\cite{tor-design}. It addresses limitations in earlier Onion -Routing designs~\cite{or-ih96,or-jsac98,or-discex00,or-pet00} by adding -perfect forward secrecy, congestion control, directory servers, data -integrity, -%configurable exit policies, Huh? That was part of the gen. 1 design -PFS -and a revised design for location-hidden services using -rendezvous points. Tor works on the real-world Internet, requires no special -privileges or kernel modifications, requires little synchronization or -coordination between nodes, and provides a reasonable trade-off between -anonymity, usability, and efficiency. - -We deployed the public Tor network in October 2003; since then it has -grown to over nine hundred volunteer-operated nodes worldwide -and over 100 megabytes average traffic per second from hundreds of -thousands of concurrent users. -Tor's research strategy has focused on deploying -a network to as many users as possible; thus, we have resisted designs that -would compromise deployability by imposing high resource demands on node -operators, and designs that would compromise usability by imposing -unacceptable restrictions on which applications we support. Although this -strategy has drawbacks (including a weakened threat model, as -discussed below), it has made it possible for Tor to serve many -hundreds of thousands of users and attract funding from diverse -sources whose goals range from security on a national scale down to -individual liberties. - -In~\cite{tor-design} we gave an overall view of Tor's design and -goals. Here we review that design at a higher level and describe -some policy and social issues that we face as -we continue deployment. Though we will discuss technical responses to -these, we do not in this article discuss purely technical challenges -facing Tor (e.g., transport protocol, resource scaling issues, moving -to non-clique topologies, performance, etc.), nor do we even cover -all of the social issues: we simply touch on some of the most salient of these. -Also, rather than providing complete solutions to every problem, we -instead lay out the challenges and constraints that we have observed while -deploying Tor. In doing so, we aim to provide a research agenda -of general interest to projects attempting to build -and deploy practical, usable anonymity networks in the wild. - -%While the Tor design paper~\cite{tor-design} gives an overall view its -%design and goals, -%this paper describes the policy and technical issues that Tor faces as -%we continue deployment. Rather than trying to provide complete solutions -%to every problem here, we lay out the assumptions and constraints -%that we have observed through deploying Tor in the wild. In doing so, we -%aim to create a research agenda for others to -%help in addressing these issues. -% Section~\ref{sec:what-is-tor} gives an -%overview of the Tor -%design and ours goals. Sections~\ref{sec:crossroads-policy} -%and~\ref{sec:crossroads-design} go on to describe the practical challenges, -%both policy and technical respectively, -%that stand in the way of moving -%from a practical useful network to a practical useful anonymous network. - -%\section{What Is Tor} -\section{Background} -Here we give a basic overview of the Tor design and its properties, and -compare Tor to other low-latency anonymity designs. - -\subsection{Tor, threat models, and distributed trust} -\label{sec:what-is-tor} - -%Here we give a basic overview of the Tor design and its properties. For -%details on the design, assumptions, and security arguments, we refer -%the reader to the Tor design paper~\cite{tor-design}. - -Tor provides \emph{forward privacy}, so that users can connect to -Internet sites without revealing their logical or physical locations -to those sites or to observers. It also provides \emph{location-hidden -services}, so that servers can support authorized users without -giving an effective vector for physical or online attackers. -Tor provides these protections even when a portion of its -infrastructure is compromised. - -To connect to a remote server via Tor, the client software learns a signed -list of Tor nodes from one of several central \emph{directory servers}, and -incrementally creates a private pathway or \emph{circuit} of encrypted -connections through authenticated Tor nodes on the network, negotiating a -separate set of encryption keys for each hop along the circuit. The circuit -is extended one node at a time, and each node along the way knows only the -immediately previous and following nodes in the circuit, so no individual Tor -node knows the complete path that each fixed-sized data packet (or -\emph{cell}) will take. -%Because each node sees no more than one hop in the -%circuit, -Thus, neither an eavesdropper nor a compromised node can -see both the connection's source and destination. Later requests use a new -circuit, to complicate long-term linkability between different actions by -a single user. - -%Tor also helps servers hide their locations while -%providing services such as web publishing or instant -%messaging. Using ``rendezvous points'', other Tor users can -%connect to these authenticated hidden services, neither one learning the -%other's network identity. - -Tor attempts to anonymize the transport layer, not the application layer. -This approach is useful for applications such as SSH -where authenticated communication is desired. However, when anonymity from -those with whom we communicate is desired, -application protocols that include personally identifying information need -additional application-level scrubbing proxies, such as -Privoxy~\cite{privoxy} for HTTP\@. Furthermore, Tor does not relay arbitrary -IP packets; it only anonymizes TCP streams and DNS requests. -%, and only supports -%connections via SOCKS -%(but see Section~\ref{subsec:tcp-vs-ip}). - -%Most node operators do not want to allow arbitrary TCP traffic. % to leave -%their server. -%To address this, Tor provides \emph{exit policies} so -%each exit node can block the IP addresses and ports it is unwilling to allow. -%Tor nodes advertise their exit policies to the directory servers, so that -%client can tell which nodes will support their connections. -% -%***Covered in 3.4*** Matt Edman via -PFS -% -%As of this writing, the Tor network has grown to around nine hundred nodes -%on four continents, with a total average load exceeding 100 MB/s and -%a total capacity exceeding %1Gbit/s. -%\\***What's the current capacity? -PFS***\\ -% -%***Covered in intro*** Matt Edman via -PFS -% -%Appendix A -%shows a graph of the number of working nodes over time, as well as a -%graph of the number of bytes being handled by the network over time. -%The network is now sufficiently diverse for further development -%and testing; but of course we always encourage new nodes -%to join. - -Building from earlier versions of onion routing developed at NRL, -Tor was researched and developed by NRL and FreeHaven under -funding by ONR and DARPA for use in securing government -communications. Continuing development and deployment has also been -funded by the Omidyar Network, the Electronic Frontier Foundation for use -in maintaining civil liberties for ordinary citizens online, and the -International Broadcasting Bureau and Reporters without Borders to combat -blocking and censorship on the Internet. As we will see below, -this wide variety of interests helps maintain both the stability and -the security of the network. - -% The Tor -%protocol was chosen -%for the anonymizing layer in the European Union's PRIME directive to -%help maintain privacy in Europe. -%The AN.ON project in Germany -%has integrated an independent implementation of the Tor protocol into -%their popular Java Anon Proxy anonymizing client. - -\medskip -\noindent -{\bf Threat models and design philosophy.} -The ideal Tor network would be practical, useful and anonymous. When -trade-offs arise between these properties, Tor's research strategy has been -to remain useful enough to attract many users, -and practical enough to support them. Only subject to these -constraints do we try to maximize -anonymity.\footnote{This is not the only possible -direction in anonymity research: designs exist that provide more anonymity -than Tor at the expense of significantly increased resource requirements, or -decreased flexibility in application support (typically because of increased -latency). Such research does not typically abandon aspirations toward -deployability or utility, but instead tries to maximize deployability and -utility subject to a certain degree of structural anonymity (structural because -usability and practicality affect usage which affects the actual anonymity -provided by the network \cite{econymics,back01}).} -%{We believe that these -%approaches can be promising and useful, but that by focusing on deploying a -%usable system in the wild, Tor helps us experiment with the actual parameters -%of what makes a system ``practical'' for volunteer operators and ``useful'' -%for home users, and helps illuminate undernoticed issues which any deployed -%volunteer anonymity network will need to address.} -Because of our strategy, Tor has a weaker threat model than many designs in -the literature. In particular, because we -support interactive communications without impractically expensive padding, -we fall prey to a variety -of intra-network~\cite{back01,attack-tor-oak05,flow-correlation04,hs-attack} -and -end-to-end~\cite{danezis:pet2004,SS03} anonymity-breaking attacks. - -Tor does not attempt to defend against a global observer. In general, an -attacker who can measure both ends of a connection through the Tor network -% I say 'measure' rather than 'observe', to encompass murdoch-danezis -% style attacks. -RD -can correlate the timing and volume of data on that connection as it enters -and leaves the network, and so link communication partners. -Known solutions to this attack would seem to require introducing a -prohibitive degree of traffic padding between the user and the network, or -introducing an unacceptable degree of latency. -Also, it is not clear that these methods would -work at all against a minimally active adversary who could introduce timing -patterns or additional traffic. Thus, Tor only attempts to defend against -external observers who cannot observe both sides of a user's connections. - -Against internal attackers who sign up Tor nodes, the situation is more -complicated. In the simplest case, if an adversary has compromised $c$ of -$n$ nodes on the Tor network, then the adversary will be able to compromise -a random circuit with probability $\frac{c^2}{n^2}$~\cite{or-pet00} -(since the circuit -initiator chooses hops randomly). But there are -complicating factors: -(1)~If the user continues to build random circuits over time, an adversary - is pretty certain to see a statistical sample of the user's traffic, and - thereby can build an increasingly accurate profile of her behavior. -(2)~An adversary who controls a popular service outside the Tor network - can be certain to observe all connections to that service; he - can therefore trace connections to that service with probability - $\frac{c}{n}$. -(3)~Users do not in fact choose nodes with uniform probability; they - favor nodes with high bandwidth or uptime, and exit nodes that - permit connections to their favorite services. -We demonstrated the severity of these problems in experiments on the -live Tor network in 2006~\cite{hsattack} and introduced \emph{entry - guards} as a means to curtail them. By choosing entry guards from -a small persistent subset, it becomes difficult for an adversary to -increase the number of circuits observed entering the network from any -given client simply by causing -numerous connections or by watching compromised nodes over time.% (See -%also Section~\ref{subsec:routing-zones} for discussion of larger -%adversaries and our dispersal goals.) - - -% I'm trying to make this paragraph work without reference to the -% analysis/confirmation distinction, which we haven't actually introduced -% yet, and which we realize isn't very stable anyway. Also, I don't want to -% deprecate these attacks if we can't demonstrate that they don't work, since -% in case they *do* turn out to work well against Tor, we'll look pretty -% foolish. -NM -% -% Matt suggests maybe cutting the following paragraph -PFS -% -More powerful attacks may exist. In \cite{hintz-pet02} it was -shown that an attacker who can catalog data volumes of popular -responder destinations (say, websites with consistent data volumes) may not -need to -observe both ends of a stream to learn source-destination links for those -responders. Entry guards should complicate such attacks as well. -Similarly, latencies of going through various routes can be -cataloged~\cite{back01} to connect endpoints. -% Also, \cite{kesdogan:pet2002} takes the -% attack another level further, to narrow down where you could be -% based on an intersection attack on subpages in a website. -RD -It has not yet been shown whether these attacks will succeed or fail -in the presence of the variability and volume quantization introduced by the -Tor network, but it seems likely that these factors will at best delay -the time and data needed for success -rather than prevent the attacks completely. - -\workingnote{ -Along similar lines, the same paper suggests a ``clogging -attack'' in which the throughput on a circuit is observed to slow -down when an adversary clogs the right nodes with his own traffic. -To determine the nodes in a circuit this attack requires the ability -to continuously monitor the traffic exiting the network on a circuit -that is up long enough to probe all network nodes in binary fashion. -% Though somewhat related, clogging and interference are really different -% attacks with different assumptions about adversary distribution and -% capabilities as well as different techniques. -pfs -Murdoch and Danezis~\cite{attack-tor-oak05} show a practical -interference attack against portions of -the fifty node Tor network as deployed in mid 2004. -An outside attacker can actively trace a circuit through the Tor network -by observing changes in the latency of his -own traffic sent through various Tor nodes. This can be done -simultaneously at multiple nodes; however, like clogging, -this attack only reveals -the Tor nodes in the circuit, not initiator and responder addresses, -so it is still necessary to discover the endpoints to complete an -effective attack. The the size and diversity of the Tor network have -increased many fold since then, and it is unknown if the attacks -can scale to the current Tor network. -} - - -%discuss $\frac{c^2}{n^2}$, except how in practice the chance of owning -%the last hop is not $c/n$ since that doesn't take the destination (website) -%into account. so in cases where the adversary does not also control the -%final destination we're in good shape, but if he *does* then we'd be better -%off with a system that lets each hop choose a path. -% -%Isn't it more accurate to say ``If the adversary _always_ controls the final -% dest, we would be just as well off with such as system.'' ? If not, why -% not? -nm -% Sure. In fact, better off, since they seem to scale more easily. -rd - -%Murdoch and Danezis describe an attack -%\cite{attack-tor-oak05} that lets an attacker determine the nodes used -%in a circuit; yet s/he cannot identify the initiator or responder, -%e.g., client or web server, through this attack. So the endpoints -%remain secure, which is the goal. It is conceivable that an -%adversary could attack or set up observation of all connections -%to an arbitrary Tor node in only a few minutes. If such an adversary -%were to exist, s/he could use this probing to remotely identify a node -%for further attack. Of more likely immediate practical concern -%an adversary with active access to the responder traffic -%wants to keep a circuit alive long enough to attack an identified -%node. Thus it is important to prevent the responding end of the circuit -%from keeping it open indefinitely. -%Also, someone could identify nodes in this way and if in their -%jurisdiction, immediately get a subpoena (if they even need one) -%telling the node operator(s) that she must retain all the active -%circuit data she now has. -%Further, the enclave model, which had previously looked to be the most -%generally secure, seems particularly threatened by this attack, since -%it identifies endpoints when they're also nodes in the Tor network: -%see Section~\ref{subsec:helper-nodes} for discussion of some ways to -%address this issue. - -\medskip -\noindent -{\bf Distributed trust.} -In practice Tor's threat model is based on -dispersal and diversity. -Our defense lies in having a diverse enough set of nodes -to prevent most real-world -adversaries from being in the right places to attack users, -by distributing each transaction -over several nodes in the network. This ``distributed trust'' approach -means the Tor network can be safely operated and used by a wide variety -of mutually distrustful users, providing sustainability and security. -%than some previous attempts at anonymizing networks. - -%No organization can achieve this security on its own. If a single -%corporation or government agency were to build a private network to -%protect its operations, any connections entering or leaving that network -%would be obviously linkable to the controlling organization. The members -%and operations of that agency would be easier, not harder, to distinguish. - -To protect our networks from traffic analysis, we must -collaboratively blend the traffic from many organizations and private -citizens, so that an eavesdropper can't tell which users are which, -and who is looking for what information. %By bringing more users onto -%the network, all users become more secure~\cite{econymics}. -%[XXX I feel uncomfortable saying this last sentence now. -RD] -%[So, I took it out. I think we can do without it. -PFS] -The Tor network has a broad range of users, including ordinary citizens -concerned about their privacy, corporations -who don't want to reveal information to their competitors, and law -enforcement and government intelligence agencies who need -to do operations on the Internet without being noticed. -Naturally, organizations will not want to depend on others for their -security. If most participating providers are reliable, Tor tolerates -some hostile infiltration of the network. For maximum protection, -the Tor design includes an enclave approach that lets data be encrypted -(and authenticated) end-to-end, so high-sensitivity users can be sure it -hasn't been read or modified. This even works for Internet services that -don't have built-in encryption and authentication, such as unencrypted -HTTP or chat, and it requires no modification of those services. - -%\subsection{Related work} -Tor differs from other deployed systems for traffic analysis resistance -in its security and flexibility. Mix networks such as -Mixmaster~\cite{mixmaster-spec} or its successor Mixminion~\cite{minion-design} -gain the highest degrees of anonymity at the expense of introducing highly -variable delays, making them unsuitable for applications such as web -browsing. Commercial single-hop -proxies~\cite{anonymizer} can provide good performance, but -a single compromise can expose all users' traffic, and a single-point -eavesdropper can perform traffic analysis on the entire network. -%Also, their proprietary implementations place any infrastructure that -%depends on these single-hop solutions at the mercy of their providers' -%financial health as well as network security. -The Java -Anon Proxy (JAP)~\cite{web-mix} provides similar functionality to Tor but -handles only web browsing rather than all TCP\@. Because all traffic -passes through fixed ``cascades'' for which the endpoints are predictable, -an adversary can know where to watch for traffic analysis from particular -clients or to particular web servers. The design calls for padding to -complicate this, although it does not appear to be implemented. -%Some peer-to-peer file-sharing overlay networks such as -%Freenet~\cite{freenet} and Mute~\cite{mute} -The Freedom -network from Zero-Knowledge Systems~\cite{freedom21-security} -was even more flexible than Tor in -transporting arbitrary IP packets, and also supported -pseudonymity in addition to anonymity; but it had -a different approach to sustainability (collecting money from users -and paying ISPs to run Tor nodes), and was eventually shut down due to financial -load. Finally, %potentially more scalable -% [I had added 'potentially' because the scalability of these designs -% is not established, and I am uncomfortable making the -% bolder unmodified assertion. Roger took 'potentially' out. -% Here's an attempt at more neutral wording -pfs] -peer-to-peer designs that are intended to be more scalable, -for example Tarzan~\cite{tarzan:ccs02} and -MorphMix~\cite{morphmix:fc04}, have been proposed in the literature but -have not been fielded. These systems differ somewhat -in threat model and presumably practical resistance to threats. -% -% Matt suggests cutting some or all of the rest of this paragraph. -PFS -% -Note that MorphMix differs from Tor only in -node discovery and circuit setup; so Tor's architecture is flexible -enough to contain a MorphMix experiment. Recently, -Tor has adopted from MorphMix the approach of making it harder to -own both ends of a circuit by requiring that nodes be chosen from -different /16 subnets. This requires -an adversary to own nodes in multiple address ranges to even have the -possibility of observing both ends of a circuit. We direct the -interested reader to~\cite{tor-design} for a more in-depth review of -related work. - -%XXXX six-four. crowds. i2p. - -%XXXX -%have a serious discussion of morphmix's assumptions, since they would -%seem to be the direct competition. in fact tor is a flexible architecture -%that would encompass morphmix, and they're nearly identical except for -%path selection and node discovery. and the trust system morphmix has -%seems overkill (and/or insecure) based on the threat model we've picked. -% this para should probably move to the scalability / directory system. -RD -% Nope. Cut for space, except for small comment added above -PFS - -\section{Social challenges} - -Many of the issues the Tor project needs to address extend beyond -system design and technology development. In particular, the -Tor project's \emph{image} with respect to its users and the rest of -the Internet impacts the security it can provide. -With this image issue in mind, this section discusses the Tor user base and -Tor's interaction with other services on the Internet. - -\subsection{Communicating security} - -Usability for anonymity systems -contributes to their security, because usability -affects the possible anonymity set~\cite{econymics,back01}. -Conversely, an unusable system attracts few users and thus can't provide -much anonymity. - -This phenomenon has a second-order effect: knowing this, users should -choose which anonymity system to use based in part on how usable -and secure -\emph{others} will find it, in order to get the protection of a larger -anonymity set. Thus we might supplement the adage ``usability is a security -parameter''~\cite{back01} with a new one: ``perceived usability is a -security parameter.''~\cite{usability-network-effect}. -% From here we can better understand the effects -%of publicity on security: the more convincing your -%advertising, the more likely people will believe you have users, and thus -%the more users you will attract. Perversely, over-hyped systems (if they -%are not too broken) may be a better choice than modestly promoted ones, -%if the hype attracts more users~\cite{usability-network-effect}. - -%So it follows that we should come up with ways to accurately communicate -%the available security levels to the user, so she can make informed -%decisions. -%JAP aims to do this by including a -%comforting `anonymity meter' dial in the software's graphical interface, -%giving the user an impression of the level of protection for her current -%traffic. - -However, there's a catch. For users to share the same anonymity set, -they need to act like each other. An attacker who can distinguish -a given user's traffic from the rest of the traffic will not be -distracted by anonymity set size. For high-latency systems like -Mixminion, where the threat model is based on mixing messages with each -other, there's an arms race between end-to-end statistical attacks and -counter-strategies~\cite{statistical-disclosure,minion-design,e2e-traffic,trickle02}. -But for low-latency systems like Tor, end-to-end \emph{traffic -correlation} attacks~\cite{danezis:pet2004,defensive-dropping,SS03,hs-attack} -allow an attacker who can observe both ends of a communication -to correlate packet timing and volume, quickly linking -the initiator to her destination. - -\workingnote{ -Like Tor, the current JAP implementation does not pad connections -apart from using small fixed-size cells for transport. In fact, -JAP's cascade-based network topology may be more vulnerable to these -attacks, because its network has fewer edges. JAP was born out of -the ISDN mix design~\cite{isdn-mixes}, where padding made sense because -every user had a fixed bandwidth allocation and altering the timing -pattern of packets could be immediately detected. But in its current context -as an Internet web anonymizer, adding sufficient padding to JAP -would probably be prohibitively expensive and ineffective against a -minimally active attacker.\footnote{Even if JAP could -fund higher-capacity nodes indefinitely, our experience -suggests that many users would not accept the increased per-user -bandwidth requirements, leading to an overall much smaller user base.} -Therefore, since under this threat -model the number of concurrent users does not seem to have much impact -on the anonymity provided, we suggest that JAP's anonymity meter is not -accurately communicating security levels to its users. - -On the other hand, while the number of active concurrent users may not -matter as much as we'd like, it still helps to have some other users -on the network, in particular different types of users. -We investigate this issue next. -} -\subsection{Reputability and perceived social value} -Another factor impacting the network's security is its reputability: -the perception of its social value based on its current user base. If Alice is -the only user who has ever downloaded the software, it might be socially -accepted, but she's not getting much anonymity. Add a thousand -activists, and she's anonymous, but everyone thinks she's an activist too. -Add a thousand -diverse citizens (cancer survivors, privacy enthusiasts, and so on) -and now she's harder to profile. - -Furthermore, the network's reputability affects its operator base: more people -are willing to run a service if they believe it will be used by human rights -workers than if they believe it will be used exclusively for disreputable -ends. This effect becomes stronger if node operators themselves think they -will be associated with their users' disreputable ends. - -So the more cancer survivors on Tor, the better for the human rights -activists. The more malicious hackers, the worse for the normal users. Thus, -reputability is an anonymity issue for two reasons. First, it impacts -the sustainability of the network: a network that's always about to be -shut down has difficulty attracting and keeping adequate nodes. -Second, a disreputable network is more vulnerable to legal and -political attacks, since it will attract fewer supporters. - -\workingnote{ -While people therefore have an incentive for the network to be used for -``more reputable'' activities than their own, there are still trade-offs -involved when it comes to anonymity. To follow the above example, a -network used entirely by cancer survivors might welcome file sharers -onto the network, though of course they'd prefer a wider -variety of users. -} -Reputability becomes even more tricky in the case of privacy networks, -since the good uses of the network (such as publishing by journalists in -dangerous countries) are typically kept private, whereas network abuses -or other problems tend to be more widely publicized. - -\workingnote{ -The impact of public perception on security is especially important -during the bootstrapping phase of the network, where the first few -widely publicized uses of the network can dictate the types of users it -attracts next. -As an example, some U.S.~Department of Energy -penetration testing engineers are tasked with compromising DoE computers -from the outside. They only have a limited number of ISPs from which to -launch their attacks, and they found that the defenders were recognizing -attacks because they came from the same IP space. These engineers wanted -to use Tor to hide their tracks. First, from a technical standpoint, -Tor does not support the variety of IP packets one would like to use in -such attacks.% (see Section~\ref{subsec:tcp-vs-ip}). -But aside from this, we also decided that it would probably be poor -precedent to encourage such use---even legal use that improves -national security---and managed to dissuade them. -} -%% "outside of academia, jap has just lost, permanently". (That is, -%% even though the crime detection issues are resolved and are unlikely -%% to go down the same way again, public perception has not been kind.) - -\subsection{Sustainability and incentives} -One of the unsolved problems in low-latency anonymity designs is -how to keep the nodes running. ZKS's Freedom network -depended on paying third parties to run its servers; the JAP project's -bandwidth depends on grants to pay for its bandwidth and -administrative expenses. In Tor, bandwidth and administrative costs are -distributed across the volunteers who run Tor nodes, so we at least have -reason to think that the Tor network could survive without continued research -funding.\footnote{It also helps that Tor is implemented with free and open - source software that can be maintained by anybody with the ability and - inclination.} But why are these volunteers running nodes, and what can we -do to encourage more volunteers to do so? - -We have not formally surveyed Tor node operators to learn why they are -running nodes, but -from the information they have provided, it seems that many of them run Tor -nodes for reasons of personal interest in privacy issues. It is possible -that others are running Tor nodes to protect their own -anonymity, but of course they are -hardly likely to tell us specifics if they are. -%Significantly, Tor's threat model changes the anonymity incentives for running -%a node. In a high-latency mix network, users can receive additional -%anonymity by running their own node, since doing so obscures when they are -%injecting messages into the network. But, anybody observing all I/O to a Tor -%node can tell when the node is generating traffic that corresponds to -%none of its incoming traffic. -% -%I didn't buy the above for reason's subtle enough that I just cut it -PFS -Tor exit node operators do attain a degree of -``deniability'' for traffic that originates at that exit node. For - example, it is likely in practice that HTTP requests from a Tor node's IP - will be assumed to be from the Tor network. - More significantly, people and organizations who use Tor for - anonymity depend on the - continued existence of the Tor network to do so; running a node helps to - keep the network operational. -%\item Local Tor entry and exit nodes allow users on a network to run in an -% `enclave' configuration. [XXXX need to resolve this. They would do this -% for E2E encryption + auth?] - - -%We must try to make the costs of running a Tor node easily minimized. -Since Tor is run by volunteers, the most crucial software usability issue is -usability by operators: when an operator leaves, the network becomes less -usable by everybody. To keep operators pleased, we must try to keep Tor's -resource and administrative demands as low as possible. - -Because of ISP billing structures, many Tor operators have underused capacity -that they are willing to donate to the network, at no additional monetary -cost to them. Features to limit bandwidth have been essential to adoption. -Also useful has been a ``hibernation'' feature that allows a Tor node that -wants to provide high bandwidth, but no more than a certain amount in a -given billing cycle, to become dormant once its bandwidth is exhausted, and -to reawaken at a random offset into the next billing cycle. -Exit policies help to limit administrative costs by limiting the frequency of -abuse complaints (see Section~\ref{subsec:tor-and-blacklists}). -% We discuss -%technical incentive mechanisms in Section~\ref{subsec:incentives-by-design}. - -%[XXXX say more. Why else would you run a node? What else can we do/do we -% already do to make running a node more attractive?] -%[We can enforce incentives; see Section 6.1. We can rate-limit clients. -% We can put "top bandwidth nodes lists" up a la seti@home.] - -\workingnote{ -\subsection{Bandwidth and file-sharing} -\label{subsec:bandwidth-and-file-sharing} -%One potentially problematical area with deploying Tor has been our response -%to file-sharing applications. -Once users have configured their applications to work with Tor, the largest -remaining usability issue is performance. Users begin to suffer -when websites ``feel slow.'' -Clients currently try to build their connections through nodes that they -guess will have enough bandwidth. But even if capacity is allocated -optimally, it seems unlikely that the current network architecture will have -enough capacity to provide every user with as much bandwidth as she would -receive if she weren't using Tor, unless far more nodes join the network. - -%Limited capacity does not destroy the network, however. Instead, usage tends -%towards an equilibrium: when performance suffers, users who value performance -%over anonymity tend to leave the system, thus freeing capacity until the -%remaining users on the network are exactly those willing to use that capacity -%there is. - -Much of Tor's recent bandwidth difficulties have come from file-sharing -applications. These applications provide two challenges to -any anonymizing network: their intensive bandwidth requirement, and the -degree to which they are associated (correctly or not) with copyright -infringement. - -High-bandwidth protocols can make the network unresponsive, -but tend to be somewhat self-correcting as lack of bandwidth drives away -users who need it. Issues of copyright violation, -however, are more interesting. Typical exit node operators want to help -people achieve private and anonymous speech, not to help people (say) host -Vin Diesel movies for download; and typical ISPs would rather not -deal with customers who draw menacing letters -from the MPAA\@. While it is quite likely that the operators are doing nothing -illegal, many ISPs have policies of dropping users who get repeated legal -threats regardless of the merits of those threats, and many operators would -prefer to avoid receiving even meritless legal threats. -So when letters arrive, operators are likely to face -pressure to block file-sharing applications entirely, in order to avoid the -hassle. - -But blocking file-sharing is not easy: popular -protocols have evolved to run on non-standard ports to -get around other port-based bans. Thus, exit node operators who want to -block file-sharing would have to find some way to integrate Tor with a -protocol-aware exit filter. This could be a technically expensive -undertaking, and one with poor prospects: it is unlikely that Tor exit nodes -would succeed where so many institutional firewalls have failed. Another -possibility for sensitive operators is to run a restrictive node that -only permits exit connections to a restricted range of ports that are -not frequently associated with file sharing. There are increasingly few such -ports. - -Other possible approaches might include rate-limiting connections, especially -long-lived connections or connections to file-sharing ports, so that -high-bandwidth connections do not flood the network. We might also want to -give priority to cells on low-bandwidth connections to keep them interactive, -but this could have negative anonymity implications. - -For the moment, it seems that Tor's bandwidth issues have rendered it -unattractive for bulk file-sharing traffic; this may continue to be so in the -future. Nevertheless, Tor will likely remain attractive for limited use in -file-sharing protocols that have separate control and data channels. - -%[We should say more -- but what? That we'll see a similar -% equilibriating effect as with bandwidth, where sensitive ops switch to -% middleman, and we become less useful for file-sharing, so the file-sharing -% people back off, so we get more ops since there's less file-sharing, so the -% file-sharers come back, etc.] - -%XXXX -%in practice, plausible deniability is hypothetical and doesn't seem very -%convincing. if ISPs find the activity antisocial, they don't care *why* -%your computer is doing that behavior. -} - -\subsection{Tor and blacklists} -\label{subsec:tor-and-blacklists} - -It was long expected that, alongside legitimate users, Tor would also -attract troublemakers who exploit Tor to abuse services on the -Internet with vandalism, rude mail, and so on. -Our initial answer to this situation was to use ``exit policies'' -to allow individual Tor nodes to block access to specific IP/port ranges. -This approach aims to make operators more willing to run Tor by allowing -them to prevent their nodes from being used for abusing particular -services. For example, by default Tor nodes block SMTP (port 25), -to avoid the issue of spam. -\workingnote{ -Note that for spammers, Tor would be -a step back, a much less effective means of distributing spam than -those currently available. This is thus primarily an unmistakable -answer to those confused about Internet communication who might raise -spam as an issue. -} - -Exit policies are useful, but they are insufficient: if not all nodes -block a given service, that service may try to block Tor instead. -While being blockable is important to being good netizens, we would like -to encourage services to allow anonymous access. Services should not -need to decide between blocking legitimate anonymous use and allowing -unlimited abuse. For the time being, blocking by IP address is -an expedient strategy, even if it undermines Internet stability and -functionality in the long run~\cite{netauth} - -This is potentially a bigger problem than it may appear. -On the one hand, services should be allowed to refuse connections from -sources of possible abuse. -But when a Tor node administrator decides whether he prefers to be able -to post to Wikipedia from his IP address, or to allow people to read -Wikipedia anonymously through his Tor node, he is making the decision -for others as well. (For a while, Wikipedia -blocked all posting from all Tor nodes based on IP addresses.) If -the Tor node shares an address with a campus or corporate NAT, -then the decision can prevent the entire population from posting. -Similarly, whether intended or not, such blocking supports -repression of free speech. In many locations where Internet access -of various kinds is censored or even punished by imprisonment, -Tor is a path both to the outside world and to others inside. -Blocking posts from Tor makes the job of censoring authorities easier. -This is a loss for both Tor -and Wikipedia: we don't want to compete for (or divvy up) the -NAT-protected entities of the world. -This is also unfortunate because there are relatively simple technical -solutions. -Various schemes for escrowing anonymous posts until they are reviewed -by editors would both prevent abuse and remove incentives for attempts -to abuse. Further, pseudonymous reputation tracking of posters through Tor -would allow those who establish adequate reputation to post without -escrow. -\workingnote{ -Software to support pseudonymous access via Tor designed precisely -to interact with Wikipedia's access mechanism has even been developed -and proposed to Wikimedia by Jason Holt~\cite{nym}, but has not been taken up. - - -Perhaps worse, many IP blacklists are coarse-grained: they ignore Tor's exit -policies, partly because it's easier to implement and partly -so they can punish -all Tor nodes. One IP blacklist even bans -every class C network that contains a Tor node, and recommends banning SMTP -from these networks even though Tor does not allow SMTP at all. This -strategic decision aims to discourage the -operation of anything resembling an open proxy by encouraging its neighbors -to shut it down to get unblocked themselves. This pressure even -affects Tor nodes running in middleman mode (disallowing all exits) when -those nodes are blacklisted too. -% Perception of Tor as an abuse vector -%is also partly driven by multiple base-rate fallacies~\cite{axelsson00}. -} - -Problems of abuse occur mainly with services such as IRC networks and -Wikipedia, which rely on IP blocking to ban abusive users. While at first -blush this practice might seem to depend on the anachronistic assumption that -each IP is an identifier for a single user, it is actually more reasonable in -practice: it assumes that non-proxy IPs are a costly resource, and that an -abuser can not change IPs at will. By blocking IPs which are used by Tor -nodes, open proxies, and service abusers, these systems hope to make -ongoing abuse difficult. Although the system is imperfect, it works -tolerably well for them in practice. - -Of course, we would prefer that legitimate anonymous users be able to -access abuse-prone services. -\workingnote{ - One conceivable approach would require -would-be IRC users, for instance, to register accounts if they want to -access the IRC network from Tor. In practice this would not -significantly impede abuse if creating new accounts were easily automatable; -this is why services use IP blocking. To deter abuse, pseudonymous -identities need to require a significant switching cost in resources or human -time. Some popular webmail applications -impose cost with Reverse Turing Tests, but this step may not deter all -abusers. Freedom used blind signatures to limit -the number of pseudonyms for each paying account, but Tor has neither the -ability nor the desire to collect payment. -} -We stress that as far as we can tell, most Tor uses are not -abusive. Most services have not complained, and others are actively -working to find ways besides banning to cope with the abuse. For example, -the Freenode IRC network had a problem with a coordinated group of -abusers joining channels and subtly taking over the conversation; but -when they labelled all users coming from Tor IPs as ``anonymous users,'' -removing the ability of the abusers to blend in, the abuse stopped. -This is an illustration of how simple technical mechanisms can remove -the ability to abuse anonymously without undermining the ability -to communicate anonymously and can thus remove the incentive to attempt -abusing in this way. - -%The use of squishy IP-based ``authentication'' and ``authorization'' -%has not broken down even to the level that SSNs used for these -%purposes have in commercial and public record contexts. Externalities -%and misplaced incentives cause a continued focus on fighting identity -%theft by protecting SSNs rather than developing better authentication -%and incentive schemes \cite{price-privacy}. Similarly we can expect a -%continued use of identification by IP number as long as there is no -%workable alternative. - -%[XXX Mention correct DNS-RBL implementation. -NM] - -\workingnote{ -\section{Design choices} - -In addition to social issues, Tor also faces some design trade-offs that must -be investigated as the network develops. - -\subsection{Transporting the stream vs transporting the packets} -\label{subsec:stream-vs-packet} -\label{subsec:tcp-vs-ip} - -Tor transports streams; it does not tunnel packets. -It has often been suggested that like the old Freedom -network~\cite{freedom21-security}, Tor should -``obviously'' anonymize IP traffic -at the IP layer. Before this could be done, many issues need to be resolved: - -\begin{enumerate} -\setlength{\itemsep}{0mm} -\setlength{\parsep}{0mm} -\item \emph{IP packets reveal OS characteristics.} We would still need to do -IP-level packet normalization, to stop things like TCP fingerprinting -attacks. %There likely exist libraries that can help with this. -This is unlikely to be a trivial task, given the diversity and complexity of -TCP stacks. -\item \emph{Application-level streams still need scrubbing.} We still need -Tor to be easy to integrate with user-level application-specific proxies -such as Privoxy. So it's not just a matter of capturing packets and -anonymizing them at the IP layer. -\item \emph{Certain protocols will still leak information.} For example, we -must rewrite DNS requests so they are delivered to an unlinkable DNS server -rather than the DNS server at a user's ISP; thus, we must understand the -protocols we are transporting. -\item \emph{The crypto is unspecified.} First we need a block-level encryption -approach that can provide security despite -packet loss and out-of-order delivery. Freedom allegedly had one, but it was -never publicly specified. -Also, TLS over UDP is not yet implemented or -specified, though some early work has begun~\cite{dtls}. -\item \emph{We'll still need to tune network parameters.} Since the above -encryption system will likely need sequence numbers (and maybe more) to do -replay detection, handle duplicate frames, and so on, we will be reimplementing -a subset of TCP anyway---a notoriously tricky path. -\item \emph{Exit policies for arbitrary IP packets mean building a secure -IDS\@.} Our node operators tell us that exit policies are one of -the main reasons they're willing to run Tor. -Adding an Intrusion Detection System to handle exit policies would -increase the security complexity of Tor, and would likely not work anyway, -as evidenced by the entire field of IDS and counter-IDS papers. Many -potential abuse issues are resolved by the fact that Tor only transports -valid TCP streams (as opposed to arbitrary IP including malformed packets -and IP floods), so exit policies become even \emph{more} important as -we become able to transport IP packets. We also need to compactly -describe exit policies so clients can predict -which nodes will allow which packets to exit. -\item \emph{The Tor-internal name spaces would need to be redesigned.} We -support hidden service {\tt{.onion}} addresses (and other special addresses, -like {\tt{.exit}} which lets the user request a particular exit node), -by intercepting the addresses when they are passed to the Tor client. -Doing so at the IP level would require a more complex interface between -Tor and the local DNS resolver. -\end{enumerate} - -This list is discouragingly long, but being able to transport more -protocols obviously has some advantages. It would be good to learn which -items are actual roadblocks and which are easier to resolve than we think. - -To be fair, Tor's stream-based approach has run into -stumbling blocks as well. While Tor supports the SOCKS protocol, -which provides a standardized interface for generic TCP proxies, many -applications do not support SOCKS\@. For them we already need to -replace the networking system calls with SOCKS-aware -versions, or run a SOCKS tunnel locally, neither of which is -easy for the average user. %---even with good instructions. -Even when applications can use SOCKS, they often make DNS requests -themselves before handing an IP address to Tor, which advertises -where the user is about to connect. -We are still working on more usable solutions. - -%So to actually provide good anonymity, we need to make sure that -%users have a practical way to use Tor anonymously. Possibilities include -%writing wrappers for applications to anonymize them automatically; improving -%the applications' support for SOCKS; writing libraries to help application -%writers use Tor properly; and implementing a local DNS proxy to reroute DNS -%requests to Tor so that applications can simply point their DNS resolvers at -%localhost and continue to use SOCKS for data only. - -\subsection{Mid-latency} -\label{subsec:mid-latency} - -Some users need to resist traffic correlation attacks. Higher-latency -mix-networks introduce variability into message -arrival times: as timing variance increases, timing correlation attacks -require increasingly more data~\cite{e2e-traffic}. Can we improve Tor's -resistance without losing too much usability? - -We need to learn whether we can trade a small increase in latency -for a large anonymity increase, or if we'd end up trading a lot of -latency for only a minimal security gain. A trade-off might be worthwhile -even if we -could only protect certain use cases, such as infrequent short-duration -transactions. % To answer this question -We might adapt the techniques of~\cite{e2e-traffic} to a lower-latency mix -network, where the messages are batches of cells in temporally clustered -connections. These large fixed-size batches can also help resist volume -signature attacks~\cite{hintz-pet02}. We could also experiment with traffic -shaping to get a good balance of throughput and security. -%Other padding regimens might supplement the -%mid-latency option; however, we should continue the caution with which -%we have always approached padding lest the overhead cost us too much -%performance or too many volunteers. - -We must keep usability in mind too. How much can latency increase -before we drive users away? We've already been forced to increase -latency slightly, as our growing network incorporates more DSL and -cable-modem nodes and more nodes in distant continents. Perhaps we can -harness this increased latency to improve anonymity rather than just -reduce usability. Further, if we let clients label certain circuits as -mid-latency as they are constructed, we could handle both types of traffic -on the same network, giving users a choice between speed and security---and -giving researchers a chance to experiment with parameters to improve the -quality of those choices. - -\subsection{Enclaves and helper nodes} -\label{subsec:helper-nodes} - -It has long been thought that users can improve their anonymity by -running their own node~\cite{tor-design,or-ih96,or-pet00}, and using -it in an \emph{enclave} configuration, where all their circuits begin -at the node under their control. Running Tor clients or servers at -the enclave perimeter is useful when policy or other requirements -prevent individual machines within the enclave from running Tor -clients~\cite{or-jsac98,or-discex00}. - -Of course, Tor's default path length of -three is insufficient for these enclaves, since the entry and/or exit -% [edit war: without the ``and/'' the natural reading here -% is aut rather than vel. And the use of the plural verb does not work -pfs] -themselves are sensitive. Tor thus increments path length by one -for each sensitive endpoint in the circuit. -Enclaves also help to protect against end-to-end attacks, since it's -possible that traffic coming from the node has simply been relayed from -elsewhere. However, if the node has recognizable behavior patterns, -an attacker who runs nodes in the network can triangulate over time to -gain confidence that it is in fact originating the traffic. Wright et -al.~\cite{wright03} introduce the notion of a \emph{helper node}---a -single fixed entry node for each user---to combat this \emph{predecessor -attack}. - -However, the attack in~\cite{attack-tor-oak05} shows that simply adding -to the path length, or using a helper node, may not protect an enclave -node. A hostile web server can send constant interference traffic to -all nodes in the network, and learn which nodes are involved in the -circuit (though at least in the current attack, he can't learn their -order). Using randomized path lengths may help some, since the attacker -will never be certain he has identified all nodes in the path unless -he probes the entire network, but as -long as the network remains small this attack will still be feasible. - -Helper nodes also aim to help Tor clients, because choosing entry and exit -points -randomly and changing them frequently allows an attacker who controls -even a few nodes to eventually link some of their destinations. The goal -is to take the risk once and for all about choosing a bad entry node, -rather than taking a new risk for each new circuit. (Choosing fixed -exit nodes is less useful, since even an honest exit node still doesn't -protect against a hostile website.) But obstacles remain before -we can implement helper nodes. -For one, the literature does not describe how to choose helpers from a list -of nodes that changes over time. If Alice is forced to choose a new entry -helper every $d$ days and $c$ of the $n$ nodes are bad, she can expect -to choose a compromised node around -every $dc/n$ days. Statistically over time this approach only helps -if she is better at choosing honest helper nodes than at choosing -honest nodes. Worse, an attacker with the ability to DoS nodes could -force users to switch helper nodes more frequently, or remove -other candidate helpers. - -%Do general DoS attacks have anonymity implications? See e.g. Adam -%Back's IH paper, but I think there's more to be pointed out here. -RD -% Not sure what you want to say here. -NM - -%Game theory for helper nodes: if Alice offers a hidden service on a -%server (enclave model), and nobody ever uses helper nodes, then against -%George+Steven's attack she's totally nailed. If only Alice uses a helper -%node, then she's still identified as the source of the data. If everybody -%uses a helper node (including Alice), then the attack identifies the -%helper node and also Alice, and knows which one is which. If everybody -%uses a helper node (but not Alice), then the attacker figures the real -%source was a client that is using Alice as a helper node. [How's my -%logic here?] -RD -% -% Not sure about the logic. For the attack to work with helper nodes, the -%attacker needs to guess that Alice is running the hidden service, right? -%Otherwise, how can he know to measure her traffic specifically? -NM -% -% In the Murdoch-Danezis attack, the adversary measures all servers. -RD - -%point to routing-zones section re: helper nodes to defend against -%big stuff. - -\subsection{Location-hidden services} -\label{subsec:hidden-services} - -% This section is first up against the wall when the revolution comes. - -Tor's \emph{rendezvous points} -let users provide TCP services to other Tor users without revealing -the service's location. Since this feature is relatively recent, we describe -here -a couple of our early observations from its deployment. - -First, our implementation of hidden services seems less hidden than we'd -like, since they build a different rendezvous circuit for each user, -and an external adversary can induce them to -produce traffic. This insecurity means that they may not be suitable as -a building block for Free Haven~\cite{freehaven-berk} or other anonymous -publishing systems that aim to provide long-term security, though helper -nodes, as discussed above, would seem to help. - -\emph{Hot-swap} hidden services, where more than one location can -provide the service and loss of any one location does not imply a -change in service, would help foil intersection and observation attacks -where an adversary monitors availability of a hidden service and also -monitors whether certain users or servers are online. The design -challenges in providing such services without otherwise compromising -the hidden service's anonymity remain an open problem; -however, see~\cite{move-ndss05}. - -In practice, hidden services are used for more than just providing private -access to a web server or IRC server. People are using hidden services -as a poor man's VPN and firewall-buster. Many people want to be able -to connect to the computers in their private network via secure shell, -and rather than playing with dyndns and trying to pierce holes in their -firewall, they run a hidden service on the inside and then rendezvous -with that hidden service externally. - -News sites like Bloggers Without Borders (www.b19s.org) are advertising -a hidden-service address on their front page. Doing this can provide -increased robustness if they use the dual-IP approach we describe -in~\cite{tor-design}, -but in practice they do it to increase visibility -of the Tor project and their support for privacy, and to offer -a way for their users, using unmodified software, to get end-to-end -encryption and authentication to their website. - -\subsection{Location diversity and ISP-class adversaries} -\label{subsec:routing-zones} - -Anonymity networks have long relied on diversity of node location for -protection against attacks---typically an adversary who can observe a -larger fraction of the network can launch a more effective attack. One -way to achieve dispersal involves growing the network so a given adversary -sees less. Alternately, we can arrange the topology so traffic can enter -or exit at many places (for example, by using a free-route network -like Tor rather than a cascade network like JAP). Lastly, we can use -distributed trust to spread each transaction over multiple jurisdictions. -But how do we decide whether two nodes are in related locations? - -Feamster and Dingledine defined a \emph{location diversity} metric -in~\cite{feamster:wpes2004}, and began investigating a variant of location -diversity based on the fact that the Internet is divided into thousands of -independently operated networks called {\em autonomous systems} (ASes). -The key insight from their paper is that while we typically think of a -connection as going directly from the Tor client to the first Tor node, -actually it traverses many different ASes on each hop. An adversary at -any of these ASes can monitor or influence traffic. Specifically, given -plausible initiators and recipients, and given random path selection, -some ASes in the simulation were able to observe 10\% to 30\% of the -transactions (that is, learn both the origin and the destination) on -the deployed Tor network (33 nodes as of June 2004). - -The paper concludes that for best protection against the AS-level -adversary, nodes should be in ASes that have the most links to other ASes: -Tier-1 ISPs such as AT\&T and Abovenet. Further, a given transaction -is safest when it starts or ends in a Tier-1 ISP\@. Therefore, assuming -initiator and responder are both in the U.S., it actually \emph{hurts} -our location diversity to use far-flung nodes in -continents like Asia or South America. -% it's not just entering or exiting from them. using them as the middle -% hop reduces your effective path length, which you presumably don't -% want because you chose that path length for a reason. -% -% Not sure I buy that argument. Two end nodes in the right ASs to -% discourage linking are still not known to each other. If some -% adversary in a single AS can bridge the middle node, it shouldn't -% therefore be able to identify initiator or responder; although it could -% contribute to further attacks given more assumptions. -% Nonetheless, no change to the actual text for now. - -Many open questions remain. First, it will be an immense engineering -challenge to get an entire BGP routing table to each Tor client, or to -summarize it sufficiently. Without a local copy, clients won't be -able to safely predict what ASes will be traversed on the various paths -through the Tor network to the final destination. Tarzan~\cite{tarzan:ccs02} -and MorphMix~\cite{morphmix:fc04} suggest that we compare IP prefixes to -determine location diversity; but the above paper showed that in practice -many of the Mixmaster nodes that share a single AS have entirely different -IP prefixes. When the network has scaled to thousands of nodes, does IP -prefix comparison become a more useful approximation? % Alternatively, can -%relevant parts of the routing tables be summarized centrally and delivered to -%clients in a less verbose format? -%% i already said "or to summarize is sufficiently" above. is that not -%% enough? -RD -% -Second, we can take advantage of caching certain content at the -exit nodes, to limit the number of requests that need to leave the -network at all. What about taking advantage of caches like Akamai or -Google~\cite{shsm03}? (Note that they're also well-positioned as global -adversaries.) -% -Third, if we follow the recommendations in~\cite{feamster:wpes2004} - and tailor path selection -to avoid choosing endpoints in similar locations, how much are we hurting -anonymity against larger real-world adversaries who can take advantage -of knowing our algorithm? -% -Fourth, can we use this knowledge to figure out which gaps in our network -most affect our robustness to this class of attack, and go recruit -new nodes with those ASes in mind? - -%Tor's security relies in large part on the dispersal properties of its -%network. We need to be more aware of the anonymity properties of various -%approaches so we can make better design decisions in the future. - -\subsection{The Anti-censorship problem} -\label{subsec:china} - -Citizens in a variety of countries, such as most recently China and -Iran, are blocked from accessing various sites outside -their country. These users try to find any tools available to allow -them to get-around these firewalls. Some anonymity networks, such as -Six-Four~\cite{six-four}, are designed specifically with this goal in -mind; others like the Anonymizer~\cite{anonymizer} are paid by sponsors -such as Voice of America to encourage Internet -freedom. Even though Tor wasn't -designed with ubiquitous access to the network in mind, thousands of -users across the world are now using it for exactly this purpose. -% Academic and NGO organizations, peacefire, \cite{berkman}, etc - -Anti-censorship networks hoping to bridge country-level blocks face -a variety of challenges. One of these is that they need to find enough -exit nodes---servers on the `free' side that are willing to relay -traffic from users to their final destinations. Anonymizing -networks like Tor are well-suited to this task since we have -already gathered a set of exit nodes that are willing to tolerate some -political heat. - -The other main challenge is to distribute a list of reachable relays -to the users inside the country, and give them software to use those relays, -without letting the censors also enumerate this list and block each -relay. Anonymizer solves this by buying lots of seemingly-unrelated IP -addresses (or having them donated), abandoning old addresses as they are -`used up,' and telling a few users about the new ones. Distributed -anonymizing networks again have an advantage here, in that we already -have tens of thousands of separate IP addresses whose users might -volunteer to provide this service since they've already installed and use -the software for their own privacy~\cite{koepsell:wpes2004}. Because -the Tor protocol separates routing from network discovery \cite{tor-design}, -volunteers could configure their Tor clients -to generate node descriptors and send them to a special directory -server that gives them out to dissidents who need to get around blocks. - -Of course, this still doesn't prevent the adversary -from enumerating and preemptively blocking the volunteer relays. -Perhaps a tiered-trust system could be built where a few individuals are -given relays' locations. They could then recommend other individuals -by telling them -those addresses, thus providing a built-in incentive to avoid letting the -adversary intercept them. Max-flow trust algorithms~\cite{advogato} -might help to bound the number of IP addresses leaked to the adversary. Groups -like the W3C are looking into using Tor as a component in an overall system to -help address censorship; we wish them success. - -%\cite{infranet} - - -\section{Scaling} -\label{sec:scaling} - -Tor is running today with hundreds of nodes and hundreds of thousands of -users, but it will certainly not scale to millions. -Scaling Tor involves four main challenges. First, to get a -large set of nodes, we must address incentives for -users to carry traffic for others. Next is safe node discovery, both -while bootstrapping (Tor clients must robustly find an initial -node list) and later (Tor clients must learn about a fair sample -of honest nodes and not let the adversary control circuits). -We must also detect and handle node speed and reliability as the network -becomes increasingly heterogeneous: since the speed and reliability -of a circuit is limited by its worst link, we must learn to track and -predict performance. Finally, we must stop assuming that all points on -the network can connect to all other points. - -\subsection{Incentives by Design} -\label{subsec:incentives-by-design} - -There are three behaviors we need to encourage for each Tor node: relaying -traffic; providing good throughput and reliability while doing it; -and allowing traffic to exit the network from that node. - -We encourage these behaviors through \emph{indirect} incentives: that -is, by designing the system and educating users in such a way that users -with certain goals will choose to relay traffic. One -main incentive for running a Tor node is social: volunteers -altruistically donate their bandwidth and time. We encourage this with -public rankings of the throughput and reliability of nodes, much like -seti@home. We further explain to users that they can get -deniability for any traffic emerging from the same address as a Tor -exit node, and they can use their own Tor node -as an entry or exit point with confidence that it's not run by an adversary. -Further, users may run a node simply because they need such a network -to be persistently available and usable, and the value of supporting this -exceeds any countervening costs. -Finally, we can encourage operators by improving the usability and feature -set of the software: -rate limiting support and easy packaging decrease the hassle of -maintaining a node, and our configurable exit policies allow each -operator to advertise a policy describing the hosts and ports to which -he feels comfortable connecting. - -To date these incentives appear to have been adequate. As the system scales -or as new issues emerge, however, we may also need to provide - \emph{direct} incentives: -providing payment or other resources in return for high-quality service. -Paying actual money is problematic: decentralized e-cash systems are -not yet practical, and a centralized collection system not only reduces -robustness, but also has failed in the past (the history of commercial -anonymizing networks is littered with failed attempts). A more promising -option is to use a tit-for-tat incentive scheme, where nodes provide better -service to nodes that have provided good service for them. - -Unfortunately, such an approach introduces new anonymity problems. -There are many surprising ways for nodes to game the incentive and -reputation system to undermine anonymity---such systems are typically -designed to encourage fairness in storage or bandwidth usage, not -fairness of provided anonymity. An adversary can attract more traffic -by performing well or can target individual users by selectively -performing, to undermine their anonymity. Typically a user who -chooses evenly from all nodes is most resistant to an adversary -targeting him, but that approach hampers the efficient use -of heterogeneous nodes. - -%When a node (call him Steve) performs well for Alice, does Steve gain -%reputation with the entire system, or just with Alice? If the entire -%system, how does Alice tell everybody about her experience in a way that -%prevents her from lying about it yet still protects her identity? If -%Steve's behavior only affects Alice's behavior, does this allow Steve to -%selectively perform only for Alice, and then break her anonymity later -%when somebody (presumably Alice) routes through his node? - -A possible solution is a simplified approach to the tit-for-tat -incentive scheme based on two rules: (1) each node should measure the -service it receives from adjacent nodes, and provide service relative -to the received service, but (2) when a node is making decisions that -affect its own security (such as building a circuit for its own -application connections), it should choose evenly from a sufficiently -large set of nodes that meet some minimum service -threshold~\cite{casc-rep}. This approach allows us to discourage -bad service -without opening Alice up as much to attacks. All of this requires -further study. - - -\subsection{Trust and discovery} -\label{subsec:trust-and-discovery} - -The published Tor design is deliberately simplistic in how -new nodes are authorized and how clients are informed about Tor -nodes and their status. -All nodes periodically upload a signed description -of their locations, keys, and capabilities to each of several well-known {\it - directory servers}. These directory servers construct a signed summary -of all known Tor nodes (a ``directory''), and a signed statement of which -nodes they -believe to be operational then (a ``network status''). Clients -periodically download a directory to learn the latest nodes and -keys, and more frequently download a network status to learn which nodes are -likely to be running. Tor nodes also operate as directory caches, to -lighten the bandwidth on the directory servers. - -To prevent Sybil attacks (wherein an adversary signs up many -purportedly independent nodes to increase her network view), -this design -requires the directory server operators to manually -approve new nodes. Unapproved nodes are included in the directory, -but clients -do not use them at the start or end of their circuits. In practice, -directory administrators perform little actual verification, and tend to -approve any Tor node whose operator can compose a coherent email. -This procedure -may prevent trivial automated Sybil attacks, but will do little -against a clever and determined attacker. - -There are a number of flaws in this system that need to be addressed as we -move forward. First, -each directory server represents an independent point of failure: any -compromised directory server could start recommending only compromised -nodes. -Second, as more nodes join the network, %the more unreasonable it -%becomes to expect clients to know about them all. -directories -become infeasibly large, and downloading the list of nodes becomes -burdensome. -Third, the validation scheme may do as much harm as it does good. It -does not prevent clever attackers from mounting Sybil attacks, -and it may deter node operators from joining the network---if -they expect the validation process to be difficult, or they do not share -any languages in common with the directory server operators. - -We could try to move the system in several directions, depending on our -choice of threat model and requirements. If we did not need to increase -network capacity to support more users, we could simply - adopt even stricter validation requirements, and reduce the number of -nodes in the network to a trusted minimum. -But, we can only do that if can simultaneously make node capacity -scale much more than we anticipate to be feasible soon, and if we can find -entities willing to run such nodes, an equally daunting prospect. - -In order to address the first two issues, it seems wise to move to a system -including a number of semi-trusted directory servers, no one of which can -compromise a user on its own. Ultimately, of course, we cannot escape the -problem of a first introducer: since most users will run Tor in whatever -configuration the software ships with, the Tor distribution itself will -remain a single point of failure so long as it includes the seed -keys for directory servers, a list of directory servers, or any other means -to learn which nodes are on the network. But omitting this information -from the Tor distribution would only delegate the trust problem to each -individual user. %, most of whom are presumably less informed about how to make -%trust decisions than the Tor developers. -A well publicized, widely available, authoritatively and independently -endorsed and signed list of initial directory servers and their keys -is a possible solution. But, setting that up properly is itself a large -bootstrapping task. - -%Network discovery, sybil, node admission, scaling. It seems that the code -%will ship with something and that's our trust root. We could try to get -%people to build a web of trust, but no. Where we go from here depends -%on what threats we have in mind. Really decentralized if your threat is -%RIAA; less so if threat is to application data or individuals or... - - -\subsection{Measuring performance and capacity} -\label{subsec:performance} - -One of the paradoxes with engineering an anonymity network is that we'd like -to learn as much as we can about how traffic flows so we can improve the -network, but we want to prevent others from learning how traffic flows in -order to trace users' connections through the network. Furthermore, many -mechanisms that help Tor run efficiently -require measurements about the network. - -Currently, nodes try to deduce their own available bandwidth (based on how -much traffic they have been able to transfer recently) and include this -information in the descriptors they upload to the directory. Clients -choose servers weighted by their bandwidth, neglecting really slow -servers and capping the influence of really fast ones. -% -This is, of course, eminently cheatable. A malicious node can get a -disproportionate amount of traffic simply by claiming to have more bandwidth -than it does. But better mechanisms have their problems. If bandwidth data -is to be measured rather than self-reported, it is usually possible for -nodes to selectively provide better service for the measuring party, or -sabotage the measured value of other nodes. Complex solutions for -mix networks have been proposed, but do not address the issues -completely~\cite{mix-acc,casc-rep}. - -Even with no cheating, network measurement is complex. It is common -for views of a node's latency and/or bandwidth to vary wildly between -observers. Further, it is unclear whether total bandwidth is really -the right measure; perhaps clients should instead be considering nodes -based on unused bandwidth or observed throughput. -%How to measure performance without letting people selectively deny service -%by distinguishing pings. Heck, just how to measure performance at all. In -%practice people have funny firewalls that don't match up to their exit -%policies and Tor doesn't deal. -% -%Network investigation: Is all this bandwidth publishing thing a good idea? -%How can we collect stats better? Note weasel's smokeping, at -%http://seppia.noreply.org/cgi-bin/smokeping.cgi?target=Tor -%which probably gives george and steven enough info to break tor? -% -And even if we can collect and use this network information effectively, -we must ensure -that it is not more useful to attackers than to us. While it -seems plausible that bandwidth data alone is not enough to reveal -sender-recipient connections under most circumstances, it could certainly -reveal the path taken by large traffic flows under low-usage circumstances. - -\subsection{Non-clique topologies} - -Tor's comparatively weak threat model may allow easier scaling than -other -designs. High-latency mix networks need to avoid partitioning attacks, where -network splits let an attacker distinguish users in different partitions. -Since Tor assumes the adversary cannot cheaply observe nodes at will, -a network split may not decrease protection much. -Thus, one option when the scale of a Tor network -exceeds some size is simply to split it. Nodes could be allocated into -partitions while hampering collaborating hostile nodes from taking over -a single partition~\cite{casc-rep}. -Clients could switch between -networks, even on a per-circuit basis. -%Future analysis may uncover -%other dangers beyond those affecting mix-nets. - -More conservatively, we can try to scale a single Tor network. Likely -problems with adding more servers to a single Tor network include an -explosion in the number of sockets needed on each server as more servers -join, and increased coordination overhead to keep each users' view of -the network consistent. As we grow, we will also have more instances of -servers that can't reach each other simply due to Internet topology or -routing problems. - -%include restricting the number of sockets and the amount of bandwidth -%used by each node. The number of sockets is determined by the network's -%connectivity and the number of users, while bandwidth capacity is determined -%by the total bandwidth of nodes on the network. The simplest solution to -%bandwidth capacity is to add more nodes, since adding a Tor node of any -%feasible bandwidth will increase the traffic capacity of the network. So as -%a first step to scaling, we should focus on making the network tolerate more -%nodes, by reducing the interconnectivity of the nodes; later we can reduce -%overhead associated with directories, discovery, and so on. - -We can address these points by reducing the network's connectivity. -Danezis~\cite{danezis:pet2003} considers -the anonymity implications of restricting routes on mix networks and -recommends an approach based on expander graphs (where any subgraph is likely -to have many neighbors). It is not immediately clear that this approach will -extend to Tor, which has a weaker threat model but higher performance -requirements: instead of analyzing the -probability of an attacker's viewing whole paths, we will need to examine the -attacker's likelihood of compromising the endpoints. -% -Tor may not need an expander graph per se: it -may be enough to have a single central subnet that is highly connected, like -an Internet backbone. % As an -%example, assume fifty nodes of relatively high traffic capacity. This -%\emph{center} forms a clique. Assume each center node can -%handle 200 connections to other nodes (including the other ones in the -%center). Assume every noncenter node connects to three nodes in the -%center and anyone out of the center that they want to. Then the -%network easily scales to c. 2500 nodes with commensurate increase in -%bandwidth. -There are many open questions: how to distribute connectivity information -(presumably nodes will learn about the central nodes -when they download Tor), whether central nodes -will need to function as a `backbone', and so on. As above, -this could reduce the amount of anonymity available from a mix-net, -but for a low-latency network where anonymity derives largely from -the edges, it may be feasible. - -%In a sense, Tor already has a non-clique topology. -%Individuals can set up and run Tor nodes without informing the -%directory servers. This allows groups to run a -%local Tor network of private nodes that connects to the public Tor -%network. This network is hidden behind the Tor network, and its -%only visible connection to Tor is at those points where it connects. -%As far as the public network, or anyone observing it, is concerned, -%they are running clients. -} - -\section{The Future} -\label{sec:conclusion} - -Tor is the largest and most diverse low-latency anonymity network -available, but we are still in the beginning stages of deployment. Several -major questions remain. - -First, will our volunteer-based approach to sustainability work in the -long term? As we add more features and destabilize the network, the -developers spend a lot of time keeping the server operators happy. Even -though Tor is free software, the network would likely stagnate and die at -this stage if the developers stopped actively working on it. We may get -an unexpected boon from the fact that we're a general-purpose overlay -network: as Tor grows more popular, other groups who need an overlay -network on the Internet are starting to adapt Tor to their needs. -% -Second, Tor is only one of many components that preserve privacy online. -For applications where it is desirable to -keep identifying information out of application traffic, someone must build -more and better protocol-aware proxies that are usable by ordinary people. -% -Third, we need to gain a reputation for social good, and learn how to -coexist with the variety of Internet services and their established -authentication mechanisms. We can't just keep escalating the blacklist -standoff forever. -% -Fourth, the current Tor -architecture does not scale even to handle current user demand. We must -find designs and incentives to let some clients relay traffic too, without -sacrificing too much anonymity. - -These are difficult and open questions. Yet choosing not to solve them -means leaving most users to a less secure network or no anonymizing -network at all. - -\bibliographystyle{plain} \bibliography{tor-design} - -\end{document} - -\clearpage -\appendix - -\begin{figure}[t] -%\unitlength=1in -\centering -%\begin{picture}(6.0,2.0) -%\put(3,1){\makebox(0,0)[c]{\epsfig{figure=graphnodes,width=6in}}} -%\end{picture} -\mbox{\epsfig{figure=graphnodes,width=5in}} -\caption{Number of Tor nodes over time, through January 2005. Lowest -line is number of exit -nodes that allow connections to port 80. Middle line is total number of -verified (registered) Tor nodes. The line above that represents nodes -that are running but not yet registered.} -\label{fig:graphnodes} -\end{figure} - -\begin{figure}[t] -\centering -\mbox{\epsfig{figure=graphtraffic,width=5in}} -\caption{The sum of traffic reported by each node over time, through -January 2005. The bottom -pair show average throughput, and the top pair represent the largest 15 -minute burst in each 4 hour period.} -\label{fig:graphtraffic} -\end{figure} - - - -%Making use of nodes with little bandwidth, or high latency/packet loss. - -%Running Tor nodes behind NATs, behind great-firewalls-of-China, etc. -%Restricted routes. How to propagate to everybody the topology? BGP -%style doesn't work because we don't want just *one* path. Point to -%Geoff's stuff. - diff --git a/doc/design-paper/graphnodes.eps b/doc/design-paper/graphnodes.eps deleted file mode 100644 index c1ccf44ff..000000000 --- a/doc/design-paper/graphnodes.eps +++ /dev/null @@ -1,168 +0,0 @@ -%!PS-Adobe-3.0 EPSF-3.0 -%%BoundingBox: 0 0 594 282 -% -% created by bmeps 1.1.0 (SCCS=1.73) -% -/pstr - 1782 string -def -/inputf - currentfile - /ASCII85Decode filter - /FlateDecode filter - /RunLengthDecode filter -def -gsave -0 282 translate -594 282 scale -594 282 8 [594 0 0 -282 0 0] -{ inputf pstr readstring pop } -false -3 -colorimage -GhVOjDi>]<FajlqSJie3"Aig\TXkI[MM+kbD;&*L=t/GMV-hs)5>n9_WC$R38uPNS -:\/!h(->g0&o'd+8jO"#,u#\]-qJ1e";8Kq`509MM1b+?CGo=.n_,W;mpme-e![aj -B("Hbs6'BNh5d)*o60!7giMmDpRL,(pWU;Y7i/+RPG?h0oNsh*1qc+:\e*?imYf?h -re5U2P]Er&[l@U7MGMBC<a&dq32GfDGWpC0c6`qhbLO?+4%u@8Z/Vdnq"treSqVDa -\U#EJUY%e5Ph#d,:B[]`lVll3`Q9BVa#HgAoB(Bt*4*aZo7("N(W4Q'r4i65/`js/ -061>D:!%,B]fqbFk0:H<3\DZWUJuk]n`SCZd8H)<CDVC+qoL]1fjpXph0Q![-O?Oq -n2%`,NU$PO:5qJ*XDRK*P<t":(jqY4cZFDZM5+?Fc#ZM$-E"'\bBrOCFfhh@jSBjM -f:%bZ^h\g@?;f;.19A;qa6-IE5bGYGqlO2:J$;Gj6N""i>pE_P!2XOs^VlZn5=9gq -SE9TkJS0@]8s0%XFBHtSSdhZ7mZ"DR\ETm=>(Yd+lps3F+2>#X8RI>kQHh'^kE]G/ -a-5/<G.p/S\TAfTXlOXK5G)$?o9\G+,^,uSYE+&$4BjjYqN7SgY7t'?'`N!Q^[T)g -/]C$ZS1Y)G<UX2C0U&MVY3<'G=.rD^)@r<3<ggnVGFjRkYJXjJq:Y)V,*4CW$M@nY -NL*V(Qd.j5TJKEVZ`/?XB$ZD&"QGh-OD_k&OP;`"JT]DO9<1iMLL>?j@AaZ!H*6[I -W$g(]g^4*1cPI-WK.qdDQoH*_jR`3(kp3I@FTFUH.,QhO4Pd\`)8Xm2MW:P73+qe\ -l(RWFgm5I>8e&2oY10M02D"Bgl>$6gA8)eEL"!-DKD@0"<oXfT1(9!LgNJujgbDB8 -lc6]'n#IO/CtkdTgkJ%iQsJ=e777##4S;<q4[KHA&kX:>#m>K/.3VuT5IN3;!3&q6 -9.PR*SN*mGlIajjqQ8^eneFI#aI<?j/=R07$/0'ElO9To55d+AlhXG%89U!ZlcAtO -S^+cHqB;RVg</;?YrJof2t`YLp8K6JL$7'B=mp27ff([#\Wm,[0\m!:a1-7_X">h) -%BW3%i[Ym_R]4`UTdSXfJ3)6W4TfuNFO]o*I_.]iijHIdht#$U)$0f3eA.uk[0r'b -<A"ZaLu8ql1t,nPYn04AUs2kUP*SgT5[l%shgKb2=^:R=o.U-77ntMWVsgE(;dCQ4 -'qE2$X</F>#Aj=';Rad""$Q7lM$q$i%F(T&OXpc=%`R%j9CL[no72iYSO_=(6@q]E -J.C(`=q/trfLN`&@=,6Ocg0I1TGQL%CC&iZ8+6Rqj*#<^Q;eI=.Y2TO+Q_EAcn?Wa -D`X6FY=JerlZAa_r)+9m)OZ)5&nU[lbp5nB+FEJ'*7hr1Vt+RqGW-&sK7DXdY\rA. -'@"E3Fso6;ktS/,81ZP$Ch'):.7*M!9j[YpCV#.'e9%,qS"bg%l<IZ1%_jaOckeR2 -F!CA@j9?J!%DX7aLW-KB&G]IqK+S7pmr+T0\AHnIN=TM_haXFkc8VpA*cjH6oRT1R -W1!q;(eP6bqGjNuGV]RF$erJQ2pbF?Q(;q*7rI^=CRR[[m@s#RH)V]9i8ALDoJrab -U(6UuF1M;D7$&]1@qm4$=.4CHWM4($&XXX%_1PuaeN.338X0\">3;Y@#Fe)[1)cT^ -`W@OtV$9,#QRWFc7:7PH<%8!uAtNHE*VKHXBJ>-B?pFj<+>3,,V&^Q/bcCQL_^]X4 -bg"PN1V4F<Qeta">h%#hd1CBhK/Y<d"f0UX.ttb9Ot@,t\E<1SL3^WGX4.65Z+/%6 ->Rs%7SBu)&b[oN_ERW$FaNtClNe,n\X+=I)**h430XhM!i&c6kAcE#+[h7e2>D>?0 -:uV.oZ`qRK\fpLsKWeTG4SfYLl+;^NC8O9Z^=5h>Q1P[)*+(FKTT)tKb&s^-s7``i -c%(Q68^3oO+_G6h/*VT(D&iLWh^,rI5G]u>F2HJY`_6:tF?Yp-q"-$shiEhNke'kW -?15Lfc3(-;jGpX*W7<ihq>/'WjnVs:NM6oShm"ppNbghEDn2Oh6Mt+Xq&1!Np>/HE -a(AjV,f?ssgoh)WR@=u%!BsirW&BkYpp=-Q'udI76+k%AbLNn65G6ZLjjlWDnjA@R -#D_*9#PS'?X9(kKN)=MlV$]sthLR%c=V_H=H'1s\:"BorK7OP%g>'r;Zp?h\ecr6+ -KI(MQe&(6N]\Y(0cBC/q-G&`k-V#S!@2;3cFXT-:klfY]6;&!-[]gF(4Z-WZ0bt"M ->!pLt&ksVXea3>AW*0WX`WL?;DD5D9=<+rK)=5e#C8&qCf8Xbrk(;fL>lFk.*84PT -Pc?uRaumCeIj/2&fEH,Z^'mU".9EH[X_,,FaK$GM(mQMO'"lZF`OflCE5K;^M+3pJ -TMKOa+ST^jX;!s>c``QXaL\O'kFdTYfS=n-]:T+-4bN[0RgNB$\GoD"P^$WT*;%h] -a&ih_/Z/le''^/Krf>WE+ZMnnm"(k)7DVB.8L-uOfPEb)f,Rc0>KligeqWmJ'[Rq+ -#8kT>ZqA\8e5`d]+n6/\QP#Q!R:ZOcnV:ddH4A8R6Q4fN*Ms-2i%+Jj3sTq#crqmi -RD(mbp`cS0J^/m8H=+2Ola/7],0-c)E)XnZd3sd91#00]bu09lpk6t2r^[HGTL0:G -Ta=p`4_c^is3:9]ac?s6k<9-:dM302pL3'#"'TMb%LhL)-gGp`?V:/T\9RK$P4qW= -T.M2s_>mEl=M[TMaJm0<n?jX%m>(N1Unl\'lA!?J7@\#]Bsg?7;&b;cYhlT<:QtJc -Q:>a-,<\\*?hq%f[7(2\>1bW9>B2e!1g!_G[u,K;F75r4,c?gtaL7\L2Q`t>/!/`\ -qp4jE9$Slnh=1\33IF]rI<YAjMm=tm@pt$d:7^j-[=>(Mmmj30032FeK0R#7IuHDT -W31!/?O_U>:WUXK[S+\Y%5qe)%-%NQ?ubB`Kb'42F_(B]Tp5DfXO\iNLXRe/EP#1r -QOiu)T=%iK+H'dl*,]1-E!S)tFOQac@e^.YWJZsh_qWS<'VC-!DbG<^G/.33fu/lJ -5i\eM/]De)!Dpg@6#Dn%Hk1=WYJh[6T(:kXhY)9pf=[-)0!AWII(:PD)j9]sp=(DQ -GrF=X6aHC`2`?l)g@5e^"8t"=j>YVrfm[5H>Ab5iY?*qqmaXG)='VfSUX`)B\(;!9 -$)?lDAdn(AYuN5EdV\LW7G3oXM-P1c9g&;7qmh&i>Hh/Z>Li(%[8clI0dEpQ;3IQY -GPk*_Md2IK"&Fa;iMd]haBnt*]aqRbS,#8s^$AA(=DD#f$JBQ5?KPuGYmCij_Z&[o -$ebL%T/>93"`*K<;+%dJ_'hu[=/nc.UIohOEsXt_*8>_Fi./Vf#i1QmIFLnd5qiq[ -#YHVTJkBBa`Jn]"^PfU4`b*[dq_c&U<F83o;V#-WN+b-][EaehDA2@(I=1(2.IpX> -`+S0GR2Ye1=mWTAc>DC"Pg+[8;33K<F%rV.,"YLA]L8hOEp^!&5&4?n602DW:45oP -Vr!!Y.HK[cRW%A58]dU5<ff_epn_$fs3kZuBf0C=Y0o58Mt@P5D*qjU,dP=Y1$<kL -*<_T!"J9Z(.ZE2mG!dsI'u\Q4.'2j2"o'i(4@T^CWeVDte#=0jAfS!5B+hH_cl?.L -A3Eb:]@Xr.n$$>lZfH[9,Ui7=q@)WJMhZ2AAsVoS9eUsl+2B.Fc_&>ZT"`4+dH*X" -!\gu-rk(WQ59f,^Cpb,M\rh,Q1GtdaY_)0']Ig<a`P<bY0r'VDs"]aR,lMjK`VV54 -g&,=.G^^M`k_`WdmtZ;$dM?jB5)N1l]=35oe^.V8RKAZ.NeTC"BS`s-=AWk3@h('O -?>ei"NO@L^nSW54&'ND\_h&dP2YMDpa/nuQfHR[h`n4K1-%AeG`&:&*S"`IB[V:U8 -@Hcf(IoHWI5]8SaK0I`)'oI[hoZ^%,cmC!NZeJ)jHNfo^)uJ"#TrAM#fn?V2T?&52 -[!#:WS6f%44N+.-r38gZ)s!LPcqCQBZ8%(,9q.I\9N*)281j0&.(2a7gKqF@n101$ -O>Ud.5U6;eNnP-U)P[VCQ/Y+d7^/d2'N"Lr%)ut]e"akJ6[jKb+cOo_M+edEp%mDo ->EGpE4&?$'L*qBjc8L3u1UKGe2l>=Z]%nrY[GXRp;'.FTTD2`-N;YZYonrD_c[i1> -R6YfR#T>DODc/Argc&WeSF-c)g.W'^b&kEg)`;iOgX+Gr9fq(Y]RktkJTiLeo!2tN -hgI<GS;la?+Lre.1U55lTnXSU=N!X;]9^2/I(R$_Qj,20=VG0PRJL[^4u-8k^l$Pr -%F]g+mr&u:?"DdX*@3[rrQN?X7AjR8c2A+L"j+S;?n((;=OVf&3!ZCEYsm'Y`jI!` -@#q`:&/_^#Q+X2[kuHh]FCKpESJdp#=IORAG],fVCnKM6_;@]=g9ui'J<@:BCHEXM -`/n]fUZoikC`^!$#919lc\87e$pNH*+&6S6(\'2\AS5mJ7p+itN0Pm@i$3#>M7gN' -,Z4#UAEMInh;cT00<+g8aH>j6CRc\1E[M=OhBq$nl-hAZL9GrTfZrfCf3t-$Z'^b! -N$u.t(5$J5aBX-j"i:Mr^7/ASRst,m_L%?1G$,mukB#e8G(3?ZL5VbG*ZVl9Msb4] -`SnPXjHnliZ)0=TDX&IX$[Y5QnEY*hQ<F?T>bg<>RY&WC!!m#F0tc!(aa<mph?!A# -n)aH*"4t]P"Z<)mh=&b=c^Q"a'u#`jeX#)-MGj1q>!,_?d%?1(N3NZe,OHo%oFfB\ -MMWYuQ\A8'mQGUCS"f2S[J7kI9$Ul7AYItiZVHaf[7-$rU-t:o^t+jfIf63"cJC7b -4LnS56phbs;DYb1gAu)ZN#f^=6./Ht8pT)]Y;PXLE%bW?6q!_DmS&hE#d/%P;J2J+ -s3d]t>_,i*35GqB"!JjZO^Dg7JjsWga>A]&k0.PXjk$BXo,TuFbt^?>*[pGJg*08$ -ppf_6a1/h4>2F[>I'jDE_1N$?i_!Bk4lOHnAe:fV/U3Aa%6/gXaKqTQ'oHen\:0W@ -_i^n<PuWqFHL+[<Mo])Tjo02af5M/P^M`db9>/*V:]WlQ#pr$Par%"/!@R^[8&N`+ -o"t,GKG3H.\)gDTnG"GacVY06Ll7jC1o,4aS>O#PgFsNsO#^.0P[Sg:RV(skjjL\O -V?rj!3rlYR,GW'%Md->._2@hE3mH)r^:1OlT-d&bBnh:X*lS<=S=,5dN2c(T[/D;q -#tE-\b_r_O>=>::F<n?'-/)3%2n"Xr^Hqot4ANiFK?,%4*+VT&6Qor7,>h/KE)jX! -!'>&Z<SAS.pg*jiiWK1'eq6tLZ5qZC,m1EKYDYL9'1DIe[WFFhUJ<9N:G]O=QZ]lH -m,5E)Eku<Q&B?q$I9hU/W4>^mqp!-+;o/R"Ocb3#1\'3EDQq#,Lsqb),T?rT.^-+n -NEGl(!ZUUGA^<fiKZKc`r(JqhB`l;eO*=#WPk98ZT;VV<2*%.<64adS0+r0.m&)W@ -=:uJjC',j;Ark;:]XZYB('G3U+A09iWg=X8",l50ijF]^om94g9%b0aEu+B%<F+&u -'QN:<-8(Xi6]o[d9T`-URcD6s\"8C&S'[RB`Tja^`g%PHPq6)`(JP;K;NehVBFJ^M -+JPjLpn5cQ]BYq4luTcg*KSV66$p=!g%!;YGnIld(:c.9=s&lb]2_=j`:)2bM'DQ9 -I,^\+,(j0@%Q#A>f7<%E!B4eL6f2XA!?r2+;8Og-/Gd5t:<eMT)E$VU!JLuVm*%"> -9nJbA_l4TaKA[>BJODXt#\R;_B-!qZGA4$DXn\hQ9%tt7F[q&OIAJ*/+F$PTpN8ge -ehJr_Ns'=n#UcQJ9@`_K-[eP7K+p$R>$"uCqm>[Gb0'.ASQ%l]`R+?#1J^q^_]ki7 -H"W/LDqImO9'=au>Jmc8'hAe!(u\dnD?&*^Xc>V/8g9CqGE(/&V]j/J4)^u2)L(n= -B^DY[Qq2OlNbUbKSZU3NBi@Oe,n'r=pY?8Dmoc0sNf4T_jp12igW[OU%=MJ:P0ESq --.H6KOW+Dk%,grA#GnG%3osd^\*&H^Ir;DuqZ4)4!_/'M5gPfhEtJnE7[B3pUlCVL -kTPk2UX_VY3G"@LN(TPh_&IrfdQS*-bOEMY)1=P+](St7[s^P]cZE@9BiKIA8B[,0 -8HX:iE#U,*4@4M$G;*fI?sY7qrAKG<LdXYbk;HjlJe&$F_h+<!';$0.d.h7imUM"I -'K!S8IMn/]?IIns\1biuVrjE/P9'(P=^A#J@Zig0Le(=(2!j]0aPP34K<4I8QEf)o -RhW'<[D+\Q5r<ih-YU@6)44qViie5RGL;*?M_$o%;C5SCJ,3<I\OOVIi68m_(HI.` -[LCs53'gRfr6P;)g2l>omojQi]:2EI:ZoAl]A'gk_YnHFVt-9DOM/9n*XcPP9<*UZ -Q;_=ZjmSEg[RVggGBB3+A@35u7gE8tWjGJ?3,'WG&ro.eb*R:F5\MDe@+i$3,fh!i -:"6?O7_.]FCY'p4!8><</oJVZQ*]El>?2V2RJf!n5HRusha^CDCAVU^6h0"rEYT0X -&VQ<94-0Bps,QQC@C7t;5BC#@?ekB%8s:c"&&uilDNk%>%2OJ2-_L]h*[9Ul"0<5n -pa?P@^62jXoboK0K9G>&KbkW4P^^Hkh?rl29aBVm*_"5"U8cd=0QPrQeq;%m?c6NR -&0po_:>/n[Q=#/S&iZ(?bP1aKqn-n-%5DNRGkRf>JKSa(*'_e*MBAR.,NGBmL=p6P -l/1=&GASg*(+(g)R*)I\nuUq317rQp6CEF>M]*t-kQ9#OHEO5N:YC\TNXiV:RZSPJ -@BB^<$&u<6E7>?\b]:X+]Cd.3Y'oWh-.e:4mLE97+9PV6o4o8hfuO[0=I)H*dt5aQ -*iV%!8K0]J1g+TZ+f"(hS%^e@7BLdCm(KWso*U#6q+f8uDb'I+25,,mpIZ6\8r^/Q -]/+E+q'&['C?#bm!(P!:7V*D8$XbfA;(ON;%mYEo[9;Wm4X%8NTWr&./*!%^P\bYT -5m&)E*VUK%2osk3p.:$NduA%2It0[INLn8GeBeR<-p9jl&i=IPTQAhBPi8*_Fb3L9 -2sA1l,@_;MV1H%O#Ea^8J9M4++kI5,.ln+bR)1Eo\G#'8ok.Q)+c#R&R>,a[:?iYh -;e+9$#0dJOWTYUj-kK/F42b!3mfEKE:K*Qh[-%h#7sm84)TWKElE-C]F"WFPNCOD: -\R8R&qZ)V"&N%L%DO01ZOLBgGl/Q0+]#[/j+5Pd@.3=>4!$[sOh@TS=8_iX:]g;A( -0(/Vq+Z?9KrObJ1mZR\^d(M8FM^(1!ERiDG#)Xrs9Rhi$NLM%"F8P=ZBr/I]5L*T$ -grD+i+.3U@+I@2n96l3(_<';d;h2-V`],*8^jY-ikAi4iN/%f_:T'O0I(W5=qV/?& -2`%"KJB.a8:W(DW!@%CIOoKZ^<M#@c1H:Rgifd>'cukK(%fue=g^!6LpG1LlkQrFD -29:2U(0<J@hSmSHL<E!CV`EeMqqrhoDLe>[I_(-'1RlXj[iKRQGcW&KZBN/[9h4?I -_sTm<VF[G(8=9n`YRc(RaiJt+.fdtFaEBH,=lHJ\Y`rqAJ>uY6JCT1Y0.:$:!B<JN -"h>lB!#n7VOjP5oG_.#TJG.3-J$uuFWB$73DN3%QifIP?H<\9oJ^G5h<@0js.KfEr -qq]<k>t-TAn'R&3Z"Psrjs98P.s`R')SjoZ!G:C$=S^B[*A5DVa1ZX_,L\fB;-@_i -H8)\YbUcn5ZgUTX#8M[*!8:.j5X9nt^jY.Tfl$?3A-7q<K%HU6f.<c'\;d]F;d;)g -4uQVl0Q:V/AUA!q>&qYlZ;#%c:F;R%^c*<Qlat7K81SX$\.:"BLf`+\QU*mi!G)M# -*DlI48(ka^frd*$V^%Vd2b`HTp0f':cEac._="QSO4%Vsfhokd!Hn_#*FO'sDXWr8 -QfIgf6@^k2+1MitD$RoEn;:73f`>XhPT<SD5hQ0R"RZBdJbsu'"3M*%4.tqjZ3/8C -#A#:Q<V^0G)B/#kj(u"6N-?u4_0?4F*K7K-`qu2O0k)#hn>?U,Se[mJ@)dg:%q[qk -nR0AWJ\fh1C1,.bTdB_V@E"Gi'?B@u\q3n'b9g:^mN]NKTfq%1f0gsD(YI&e.5eAk -bZ1_YpSR[qEda(`JVl<6HVNSH*)q8XY)fltV[^,W<#9<*$U52(DIF"G"q0tmEG<f: -ig8Wo,^,uSYE*V+h2RErqV@>.l/]U\'8NSL0sn!B,Om8;2>(>[mI.dOpT91Y+'8?+ -X6rniI!ZNu9oAmFMu@D`s'*b@5E4\pa`fXG6CsJY[&9ImXYfd;>b*<-Mp4%aL#:J> -2/+3Pn8+>hV%0mNr1jKjBli*7eJ[iu\>#d.:Q1eJ(KRI!WtP2t6cF$I5-MWj#kD2' -n9G:;L6fB+>a&-poO`6CIEGWJFO-)TEOnFbL\O9ha-*Y0]7$[!+=_o#n<?Aae%o</ -aKluW.8F#/)-%(ONL`)+YS?m:ARn/[M)fB:(TT>DH%"]"B;Cg4#W'h/U4r@L,%<S8 -C"5aXfjq=C"M#H_NtXil+K]/VT",-q9!NE&Yk%uUlH<s+Y)/i&<SA/.B..1"7oH@o -\[,9*)#dn"E<J:'[pSbk+HD&eUsgG&iW&#ALChSHF`#(IcUiP/^YkF-k8Ns(LmZ,` -8`"-`I^+2EpoDJ>M#WN^1!aHehs%nZ`2aOo*cQK0*o"G*_s=VM]?#EK8M/fboBE'[ -D8XD3:Yn[1$"7&LYckl4<pJW-ij0-&Bt;YM.M8:]hF8`M<S2e#h/-Pu^0,h)W6X"S -U\n?T1@m29h8cO."3\Mhj:Lu")ABJErjhfO#D.d;+RhWj&n#16bJ_'Rr.3s#@&8Fs -9</"JV#1oe<s+$fFJe4<;G"828ZO[R9W]KE#WS!W\gD-_R1o^YiNYP42Oqe;q6&m6 -s42i,P@8th5r:O:4PDh"0qeND5+WB!=PDu&=]0Kr1fa*-enX3iF1^52_;>knBlLX] -BLYm2@F(WF!\al%Rs4(m6,N3LP/lTl/J5a?Gd2W>%PMD$3$o1eOVFHAW'86@Lq/ae -PoG]<!?0ND3p&uD,_aDaI4)seP\M_3-6QSfXhedD_F\16HC/)@KkRTj!)>$O["7%9 -D6B5)D40SQUj"t>'sP,GBNHcO_@C;#b4Y/s#/"Q=JK#Z6aoL"/UN22;/ICc%Ue"uB -]?c8;r^8O2Gn1@s^['02cR?WTXYaq/ld"*4j3n6hI:ur.qDG(lF-1K[aj&iIf8Zb6 -Or[i<?g[<aqu7aROUer -~> -grestore -currentdict /inputf undef -currentdict /pstr undef diff --git a/doc/design-paper/graphnodes.pdf b/doc/design-paper/graphnodes.pdf Binary files differdeleted file mode 100644 index 68e98dc8d..000000000 --- a/doc/design-paper/graphnodes.pdf +++ /dev/null diff --git a/doc/design-paper/graphtraffic.eps b/doc/design-paper/graphtraffic.eps deleted file mode 100644 index a84383cf2..000000000 --- a/doc/design-paper/graphtraffic.eps +++ /dev/null @@ -1,143 +0,0 @@ -%!PS-Adobe-3.0 EPSF-3.0 -%%BoundingBox: 0 0 594 282 -% -% created by bmeps 1.1.0 (SCCS=1.73) -% -/pstr - 1782 string -def -/inputf - currentfile - /ASCII85Decode filter - /FlateDecode filter - /RunLengthDecode filter -def -gsave -0 282 translate -594 282 scale -594 282 8 [594 0 0 -282 0 0] -{ inputf pstr readstring pop } -false -3 -colorimage -GhVOJCMt3EF[JG%)$pqO7S(n%G(K^F;`(G8(XOELP.48L(.b+8Ob*4)ImDCW62R>Y -5HGO<Fr1H]>/u'-DC`Yb)[DB4)jR6jITs'/g5KaoH;L?XMWU?Y2nS3s=$,-(cJ7Tn -YBfKkRG!i0]/-cTO*_rBhK-t*kM//,dl$1+3UP^>^A=DLhS%e1RVVcTmH/coI^_2p --/A)fY!:s%S!R6CL[OJmLso/T9kVT@SOM-<O.4GdDT?Ibc$/qg%i@"6-bO49Co>.P -"#a4/jnkLEcag/sj%jS#:OH52=nG\2FV0B#(d/l[r=F$/4Qhlj2FFL=Hhc&;&J`$g -e/#mQQ%%cA4[bE&c?USiST&@qHQeW6Q<1<?3l70B^)I">&A7>i*.RhB&kZqmpiuqT -;(;:1\^[8F`<fe<)dDm.L73akH"7:CV3Kg[r._&+4FZ(9HW8l3<+74k&%JL^0ok15 -[,a]$bn\sQcP$*%^M7Lam>q*%ek]FmNu1&pA5S"U,JuZG85EA52-4j+Fa)>Q]WpUj --c\0AUZjm(rgK(!Q'CJUfn"L/+)^cpr0ER@>F\KtL:]+M%t+>D*CI0*(eUl!(XEHC -$I9(6CE)"1Ous8io0`ntb7#Cg$&"&X9J(K:5Un!<:QB?a8(2a0*")gtAsRsn%p7_Y -jHp%:/!9qS_<='I%A-l7LCc&1FrQqV[ZM!=1G^WceO%6YV*a`XLa9B7m^LNMqdY3' -Q4<UR>^+0us6$rO^61F>EZnab:b1so.b)bf>Uo!B#Y'gi7FN_'.$fHM3n_"O`l(oW -(Ph(J+H_ZdYbETV;*h>J]RD]U)MoC-)+9`C6jcNt=$"H,Yfb*#M/"S=ag+N;<)?)& -/9*T#H2)>tq!dT0(po?TZI>;oH"-D60R!510A=+28p4MPmT29j@+ZGDPQChf'`4_7 -=89;sc_'deo<oChZ)I/0[+cegDPUat-1,bme6"/p$>N#m[0q+I,atN[9.Q94$\_&3 -g,5;nh9*RoFNa_4=gkG/akK[bc9hf.L^N%j_+MVk($!c^@@GEhD5&>b2'm@d[B;nR -%]d"=L=Bg0GjGk))]]eG0VW]DA4A/7>a9V=6G]9H!<d(q')["DlRGP3ED_BA->`q6 -1*$1JTu:\fnZ8ssn>ri7:2H]'6LgW:%,TmWcA.k.S@0X)P)*g&E>k`Lc'B-oD;ASY -\eH:KVUM`<Q;^gr))B8ii/#ab0\*N+iNOB"GcJ0%LpC%*TY,;f<5MddGepP%O+%Y9 -AjN/MkB<YK*'Ljm]G`d5$q;`&[Nf;5#^_2U>Y=D7fEPr[<7W\W<onQqh6!I-?lObi -Y]k,^2CKMYm4=J!Z'tQIOdNT8UZjP"TO4$g7Ba!ZqVP7IOU>hlk=@qY%Z_W$65f7> ->bumq/d5R@&L^V)(kp'eRQC%t"`ZMg$nP/:[me8R6oURfnU^pdf*J<%JP-E.(n'\G -&T6V'%gFZ8,#.H4JlulYG"<7("BC^WH]j3$J(>cAdkn-JkXrB*N2'8P&Tc6U9!),j -.lrFf.T_4J\7QJEm0ZgoF>BOm/j;ipaTXksiF>E+>bPYX[9Q-!s"\QRfG?fmX?D@O -$U*EpT?\k`J#-&`7pJ2>9]dJEI8:*F+]#k+^("B5?%lIB$$MG(=I<++YF&4(1YTjR -IY1'5c,%T(]:3sl,,Ptq-Q2],6>cB_d(G(@ppOkK!Lb?Sl_&malpO"Ur>r<LlqCbE -Y,t<i=&:D.LPi]7!L('\JhlJF!'p(4GJoN0ZY$U.`C,KS33a#@!XF%gT=1&SqgY>0 -`I@cjN53uiGuEuEIF?[28aa$G3<"j_WhreS'6pH%:mVSSN\6D?VPEA<enH"s"<et. -`O(hZVPZVYBNJ-pZJ-9o7o/`rh%])C0XqtXX%5H>AL*-kX<o3N4+`p@,+l!eEWrkF -dj!:`Wf\]:KA+jCf;[[ZMV+_lQ\d!7$'ulaoWfTm`d1=%P-_3`Zsi,>"p9EB_gWuC -"9mEX22$/=k[#nS+-W!M_FduS+a6&ciEji'TPX%VI>F0$3:N%,KSVuOL,^1T8WeiS -Eq.?kX!_aE]YOp$pN7\l/!Op+0=+>"F7uQ"5_hT+WO.%D-h.d#R>AM:XF%Bf+hN8g -H^:'D-Ai+9b@(JuGR:>-,Xtj=FK;"KKsS-21jJhh#:2]rHL\m\3Na&hE`G+o<UNk@ -D8k%@&DMUa#>ga1QmU]E/%/S(_Yq;!=4%>MIF;'6-+R]2iFo(s8#V<^h/g[75cqOP -_-WdW[-#A\QLk&eQ)G&,lO6Y2];anIYbMt#i&a"U@OHYclJ\>1$@(,1/;ip=,lhea -rY<T\=,YQ\>X]#uraYLc@4LLXW)YUtb`6@l$"'pt9q'U_%<jH;_^l::>_(n6^=\mO -^aMLr5t>ff79.tWd"ot@R>OP`OC97]Xk"G7em4.\YfVn#Xb%?gW4$HQei'Z!8b#mT -'->:W8(Bp&^nJNnX1E\Aeq!ae=_6G3RoB/=]&=>Z>_qW2`=O"X2m_f`!XhS'"ThQ: -mGNVkdDWb)#&q]OjM6@dpS*NB%*5:(#+0eS3O2upb1PR*B$)/X:j\1rc)*T/*#Sl> -PAg#j*b7l<m#u1WB@/Xg9uY2jN]ll(=M=?!01\0Ij$>Ei3?;TnTNlJ9aEF)DF!ggJ -R:F&'IIpSi1K(6Tg9,i[Vf:jOE?go3\ngpJ*lo9WnR!ZA/e]m4Gki4(N?hB/Ms?Le -l)kEO,RsL@$&'9S,d]?(A+>W]nA1;K&im@`E+JF1\AmWZUgTKnmU-=]`%68V`W`[V -i`7;mVI^:ke=D\U-nR5jn:'T+\?LoHne/(&j4d;fWH<R*#;[&DPH3E;XZr"A]q[PC -/C(>[pZanm"(s1<R8r=Qfoj3?<k@&*8O,tL4WiGj0=8Ag*1Q&X?E1uh8=EK?,;ka3 -_]6kg#.GodeMLWji]/jmpm$CIX='YmVIsi*A8f\%]/5n^mMs#mqV$J38&FZ=i6P:N -XXpuaTerLh^cVfMJp\mZ>_T?dkDF+rB$U\=OE%To=WP.=b1sKmKW,D$SjIXsJ*mUQ -B`R:NmJrLqfPt;-+b;+`p).=o!nVR.,Mc72>s(A+4"e$kna]1,+tp#ugD$MOcEQij -R!mceFd;q6Rf^_ZR^D$aqQ@+,Y:sPe\5Y'4FM5Yn@&i4]eQ,hLl(9Y@iQ&l+NE9rL -Z+eUh9s^1alX^fNbd_soRSMH:9;\G"CbSNHIu%c"A4;3L\%*Pfka]S9ZIo^nY`_,f -qp++X>?X3#`N1$aVUgZek:EO'Rc^e.8he$G4@a"QO_0ZW7.-\#DP\CY2?4[u_F4"I -KIIloF1bT8I)1C?a]:pP6peqn!V6r-Xt;Vf3^\o\$/#DtFEaK*MIWZhNP!;]g3;0c -cR`Y.-*G,YVPEW9^S+.+\DaVe6N$psR,S98OWCQtGH[M>&J`sHA?`Iog[gLb!qG_- -MfFZIVn"coLZU(T9(SuRhAL%O&CIa8dDfWaIpm?H'LSS,`cnkk4oo5Y31l\a%O2Y9 -3%J+2XFY7oiC^hCPR6o?r&?H(Mih*<C1$@`I4C5%.SBR!Zj3!APAS03QH:fMq[`;H -![kG;.JZa#Ic:4?Hg0k1!VE(7Sk`J&pa0:Tie4i(F-D;_J^:o#o%Th&\%n8?P9mq/ -:&'2.9<T]j5l0JWG*Sl).PJe:iq]j`(Ai*6)g!Oo%EOOLc$Z2Pb2#Bh>-J=e.PLcN -([EM%1X`_!GF:CRc.Rl7Xr$'DVhsFQ]>5n)Rs>@..p-XfB;J"36e^UmJ9AEN)>[+f -1pR'Y)$a9O_%o$69ANMFJL(]4Tr7#"^:D6X;dH"IP&YCj$g_MEM+fgQC`5JVlE%3u -SIj:Ui@Z>BZYsX5iWXUhf*jiRif9(#l6D6NW4g@^7nNr(p&:?@a+G%5q'<6GYdF19 -:Lom#ZBENTHrO_+-.T9jdA5_E(fbuGn+LDEO!$<uS]L*\Bt0'8/(?B'![QPs8R%oG -C\3d%`b%=-GW`Hs)*u`^L4chPL=m?pU[X9qLoIOr_Kkkq=2f$AJND]4a&=['\/<k2 -e(dS4&iK0F]rQghWrfk^Qi#7d`5A4=cSl*nA*]r(rdBm#9;r8m<ng.M3_a=3]%'!N -VWGd%Ol!?/rauRiMUOUp;I3TV<1YM,SBTUbrV#s`L0rhlG3L&+H:!d_XTdP2Y2&10 -QSkJ%dYU@V-`frU-b3=.ER\X-$?'_gs'>H;>A%gB`EUC56"r)IO&;3,Oc>22+HgbD ->/%M_hi*AbnX_Qc!4DrBZ27ZkV?U:09J.TEgnuL$+a8e`NmdEB.RT\8NRd%_At,pq -^H#Bi!kSp)?W(*&a&H_@\+)kIAHOm%EOuiWQI9.)pQe_30/en\?FIt-3BMrEMmi97 -7XXnOd(T&A^pg5&*3M)(pWN!@)Xl;K(U[%r^1/9pf+j%g\,.np")7LRE?T[3(:(Jj -G-LGeaqVeoW(VI%.M^;G(P6*f*?BT`Y8;M</Y8e3mU2<R>r#o<@$*5:c2:Qj1VP@L -RK;"n;!ubUIGH=[$\>Db[0@H0lUK!Ur9L0s*#E>-l86`K?^KZodfuor^4E[D/G=&o -$0Wg`4<O_O7?-Z.CK"B_T*)IpY?1.QC)`!llng@<M0CHM7n6&C6f&^8nhL_I$UX'D -Le+,d3'-r<D@e/SWSr8kqa9(R11CdUUUmbWH:97+f*"cnE8[l,Ii\k($i"O*X]pCE -s(5]"P#H=+,tJu?>U-sno8knk:U#AW7i$c3bJMh^8"d_Q_3Ni.')k9[::#iF9%?kI -C%UTM]49^WmlkccB4VHs'RAf&'B;DE#_[ceHF^[umAE8<5Rrg2*emIfXp<<4/_dZN -qq8FcG"A2;k5tB[ieJ02Cl<5&&o4KL\lgW[V)fARTVBsErZDT@BbcsrmeG@03'Bhd -_h%fir'JA>E.YYdBeB$d4SBD[90p`);^QV"%-Hu2Im;D;YQGqEX0n>/UQpuBZNOod -81QBU(W;HYE8cmaXJb:LG/'r1CoX5UpW)W:IdB22\!J"@pig/8F=h`E&.eqr.kf?J -`h`CSMWb(kabRo>58.d/e'W'@dQqoQ`Cq3;p-Ljp0*pmY:QCJD1]3`:5F>/\jI&gp ->bCD7Nqb"cpS1<WSaUrK88E$pqV>"d$Q^\c$utg#f7XXQ=4\#Kb>WH>Qia>mShr-B -C-b3AoaQnN<_l;0'7M;:$RG#O+/tcRUZW2p^AsS``%irLO<I+Bq;.jt9m''60J)'P -=7+HtocBFK`>I,UMf?0VAs@4"B2VZg#e(6A4m,'*cNPU+;j;gPhl24@(@q4N*,W>9 -iDO"j;PNZ8o!W"9.SEQ>RAp:iJeN/5mL5LPV(;sn[cB!).AoT*,IbhGGMboXX(d-Q -E\J_MK=a0P5?F-to@[5l>pq(PZPs=fY`l$oQNRo`1l]OeK3_OV^P.OFX"C(WF:T*C -b^5X'R?4kn"q?oq&KcqRmpR@?oQOPBp8oj!o!u4uXQ0o4+M/f?M0AC#`k0c'5'ptk -qBCHb,[:hE2"#aCL%Q43U55so`)f0q'iu^clK0sGqd"4u=J6m%6-S[og4T;aQ#3Mf -Bg([7(%`"^2lraoO4T;+H@^sD\pimLfhN-#\'><mYO,^>p_P'8FOBjfUidOC8rsuK -hBbt<mAYZa5776%8GNG>7SKc:<h;PC(nOt;gmgH[309@U3%cO!q^;i9q:(5;jQG[) -WbemcS"dr`89+;#)ThXeD6(h-Z$kd[ED=mi51c\;<fAgMR(Ao"PKeu&+\``%Ia,lO -+-\)bW/[*5ba<A2hYY@(iH>g9L7om2s+r&^b'+=uQYLf"7gJ]@(\RaMdcVC:_J^n> -P,DFkJr2fUV$KZ,$)!imU(ZmHYD\rF5Up,lEcYRS00f9!9D_AADDCO^$BP\n.a9D/ -'/=8^%ms_kQi1]kg<=Vmq5Vd+Uus5s:k`"^I7`V<S>><!)KiLI,jIERs$ME+$opeg -d$FdH+J2=Z@(2t@a$ITnCrYJ`/;aba9U=lM.<@n9d@Db25X$t^UE,6))-MAP@VOg2 -[7t*Dkt;,o:2hFbn%Os*7iu;@s7nb^2WCOM-,[Tfb*!C_(B!tgdh%8G\$`:9IARMs -U,!K54=u%r>%=i5!s;J?"5PWnj)-o=UH#>Hs!7A<HT^Tt?VI@#b4$cY')fD:rX4!5 -b@YaG`ZJeDIN>F&*+`-_[!QAjl%\u)U^O2OaD?N(Aa#\18DhP%D=#[7(549JWT$EQ -_Y:*V`DS>JR7*Z<3tdr2]]-n[T<8l3V9l7?UguVM(]87eA'/CZl8m6#8L!-f9cc@O -AA_jAMUtpuFjXJl7!JD0;4aoKVY'i!j$jn4k=@7Ub4V=UR%D$J*.T<3>XBXM<,(cT -IN#V27&Pq/6,Tm3$Z635V92_PC\;gTr`Z",dd<Z<Y=BLK\LFTUTf(d)7's$*L=)Pc -GrJBF.uj9Jj75a#k`GGKP#UaH3@8LgrQ]r]?Pgk%C@ZX&8Ut.m9&KmDSgi:aS13^! -H8o'%\G]CT9f\t+Bs"mo&=k;Bi<$SnW->>Wb;%^HAUjNN<?ro4TIE$u;`Eh?4u;jl -0K)_l/YP7/&?U8#s7XH!_=P-DUKJlq;@@V=)q<YE?/u$*c]dB.da<0.*oDrY2`:Z7 -,8sn"ro4B(%m'\Eh)u"u,GogdUX\qHRpGJ<gtkHroAMmo9).]Z3SOl7Dh\'Go`$DQ -s"2iF,.`28#*ajLq$p&%A^<eE/VVq9/)tLG3_=FS(KLtt)e(ZL?qXk!l\.<Dm(TOa -%/u%NLp._;_.e9A3^rN&OE0bS04`,b78MbDY'D1]G7_t@b\;#l#BAu83l=U97#Q6s -^m=N1;!pkDRN<]l3n!%E$A#D_Hqc'=L=5Tj%8cb.9#HM<iu>%m@j7g^!9Sh_OH;p9 -,$LfBr"bj^kc)Y@1-F&q;A5]]U71s#W+%O`*pP#`DL;M[".mNRbD>?gn5.s!18i5m -Sci[q3j0]ZhG2f[7E4,]49dL-TM>o9\jked-_i$VP5k02]>-G\J)PIY3^WBu6AM'X -s.`*Q!&`q;DRenNLH_gAk,=k[icc;s6r>3gf\Hq68Gkbei>4TADbcJKeTLKV<Heml -[7QCFaRNp(Ld)E3H[F',BSN?H;SlH$<M,N1F$9R>b$\SKJFf=a@%A&3oeW8g$6/'L -jAj`mid;ea-F@GF>=7F_hjG.L187EmeAG>_0IifNMHC>fcc;+jK:CCRO3(X!,SQM$ -S!hdF/7`cDOh'unOb/XJ>8f75ag:KICueG\TgNA5-IAeq<-%NX,,W;+/.6i6o2*OL -;G')Pl&"biOff@qN"9`q%O22D>;"YnAsfr'YW7`6[/o2tX!T=]><jiZ&k52>-^t+_ -e6G[MD7k1":h"T]M:JOLW_SG&L+*;<$HsODi9CmGge:_J-e/D2BrfiS`[.un2`4'a -MseCj-YdUAWI*PMUTKfejB.g$,:g+C6Tr<t8"%"+Lb.QPcI2;,)2#^uhs4\1KM:-" -N*KV.*F,6*UO"WnaX,"$.ML5cb91TkX"(1P#D"C+f,Et3*p])_+qFE/hJ6=6Ts]^! -_la#r2go;KYO]p4l0?%d68`)T^q,?BHcZm?aSTJ!iq7tu)7:;"9Zchp:#s$5OPt`i -]B#Q0_K-7,^oRWLlUdkF9dHIkFn)pIT="^BKY51IZKDf)^W1fFT@;lX=3/j'^fZPF -VQYR0O+4`aDh"C[p\mCpac"F -~> -grestore -currentdict /inputf undef -currentdict /pstr undef diff --git a/doc/design-paper/graphtraffic.pdf b/doc/design-paper/graphtraffic.pdf Binary files differdeleted file mode 100644 index e37382f6e..000000000 --- a/doc/design-paper/graphtraffic.pdf +++ /dev/null diff --git a/doc/design-paper/interaction.eps b/doc/design-paper/interaction.eps deleted file mode 100644 index 9b4e3db61..000000000 --- a/doc/design-paper/interaction.eps +++ /dev/null @@ -1,463 +0,0 @@ -%!PS-Adobe-2.0 EPSF-2.0 -%%Title: interaction.fig -%%Creator: fig2dev Version 3.2 Patchlevel 3d -%%CreationDate: Sat Jan 31 05:25:23 2004 -%%For: nickm@totoro.wangafu.net () -%%BoundingBox: 0 0 449 235 -%%Magnification: 1.0000 -%%EndComments -/$F2psDict 200 dict def -$F2psDict begin -$F2psDict /mtrx matrix put -/col-1 {0 setgray} bind def -/col0 {0.000 0.000 0.000 srgb} bind def -/col1 {0.000 0.000 1.000 srgb} bind def -/col2 {0.000 1.000 0.000 srgb} bind def -/col3 {0.000 1.000 1.000 srgb} bind def -/col4 {1.000 0.000 0.000 srgb} bind def -/col5 {1.000 0.000 1.000 srgb} bind def -/col6 {1.000 1.000 0.000 srgb} bind def -/col7 {1.000 1.000 1.000 srgb} bind def -/col8 {0.000 0.000 0.560 srgb} bind def -/col9 {0.000 0.000 0.690 srgb} bind def -/col10 {0.000 0.000 0.820 srgb} bind def -/col11 {0.530 0.810 1.000 srgb} bind def -/col12 {0.000 0.560 0.000 srgb} bind def -/col13 {0.000 0.690 0.000 srgb} bind def -/col14 {0.000 0.820 0.000 srgb} bind def -/col15 {0.000 0.560 0.560 srgb} bind def -/col16 {0.000 0.690 0.690 srgb} bind def -/col17 {0.000 0.820 0.820 srgb} bind def -/col18 {0.560 0.000 0.000 srgb} bind def -/col19 {0.690 0.000 0.000 srgb} bind def -/col20 {0.820 0.000 0.000 srgb} bind def -/col21 {0.560 0.000 0.560 srgb} bind def -/col22 {0.690 0.000 0.690 srgb} bind def -/col23 {0.820 0.000 0.820 srgb} bind def -/col24 {0.500 0.190 0.000 srgb} bind def -/col25 {0.630 0.250 0.000 srgb} bind def -/col26 {0.750 0.380 0.000 srgb} bind def -/col27 {1.000 0.500 0.500 srgb} bind def -/col28 {1.000 0.630 0.630 srgb} bind def -/col29 {1.000 0.750 0.750 srgb} bind def -/col30 {1.000 0.880 0.880 srgb} bind def -/col31 {1.000 0.840 0.000 srgb} bind def - -end -save -newpath 0 235 moveto 0 0 lineto 449 0 lineto 449 235 lineto closepath clip newpath --62.3 239.8 translate -1 -1 scale - -/cp {closepath} bind def -/ef {eofill} bind def -/gr {grestore} bind def -/gs {gsave} bind def -/sa {save} bind def -/rs {restore} bind def -/l {lineto} bind def -/m {moveto} bind def -/rm {rmoveto} bind def -/n {newpath} bind def -/s {stroke} bind def -/sh {show} bind def -/slc {setlinecap} bind def -/slj {setlinejoin} bind def -/slw {setlinewidth} bind def -/srgb {setrgbcolor} bind def -/rot {rotate} bind def -/sc {scale} bind def -/sd {setdash} bind def -/ff {findfont} bind def -/sf {setfont} bind def -/scf {scalefont} bind def -/sw {stringwidth} bind def -/tr {translate} bind def -/tnt {dup dup currentrgbcolor - 4 -2 roll dup 1 exch sub 3 -1 roll mul add - 4 -2 roll dup 1 exch sub 3 -1 roll mul add - 4 -2 roll dup 1 exch sub 3 -1 roll mul add srgb} - bind def -/shd {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul - 4 -2 roll mul srgb} bind def -/reencdict 12 dict def /ReEncode { reencdict begin -/newcodesandnames exch def /newfontname exch def /basefontname exch def -/basefontdict basefontname findfont def /newfont basefontdict maxlength dict def -basefontdict { exch dup /FID ne { dup /Encoding eq -{ exch dup length array copy newfont 3 1 roll put } -{ exch newfont 3 1 roll put } ifelse } { pop pop } ifelse } forall -newfont /FontName newfontname put newcodesandnames aload pop -128 1 255 { newfont /Encoding get exch /.notdef put } for -newcodesandnames length 2 idiv { newfont /Encoding get 3 1 roll put } repeat -newfontname newfont definefont pop end } def -/isovec [ -8#055 /minus 8#200 /grave 8#201 /acute 8#202 /circumflex 8#203 /tilde -8#204 /macron 8#205 /breve 8#206 /dotaccent 8#207 /dieresis -8#210 /ring 8#211 /cedilla 8#212 /hungarumlaut 8#213 /ogonek 8#214 /caron -8#220 /dotlessi 8#230 /oe 8#231 /OE -8#240 /space 8#241 /exclamdown 8#242 /cent 8#243 /sterling -8#244 /currency 8#245 /yen 8#246 /brokenbar 8#247 /section 8#250 /dieresis -8#251 /copyright 8#252 /ordfeminine 8#253 /guillemotleft 8#254 /logicalnot -8#255 /hyphen 8#256 /registered 8#257 /macron 8#260 /degree 8#261 /plusminus -8#262 /twosuperior 8#263 /threesuperior 8#264 /acute 8#265 /mu 8#266 /paragraph -8#267 /periodcentered 8#270 /cedilla 8#271 /onesuperior 8#272 /ordmasculine -8#273 /guillemotright 8#274 /onequarter 8#275 /onehalf -8#276 /threequarters 8#277 /questiondown 8#300 /Agrave 8#301 /Aacute -8#302 /Acircumflex 8#303 /Atilde 8#304 /Adieresis 8#305 /Aring -8#306 /AE 8#307 /Ccedilla 8#310 /Egrave 8#311 /Eacute -8#312 /Ecircumflex 8#313 /Edieresis 8#314 /Igrave 8#315 /Iacute -8#316 /Icircumflex 8#317 /Idieresis 8#320 /Eth 8#321 /Ntilde 8#322 /Ograve -8#323 /Oacute 8#324 /Ocircumflex 8#325 /Otilde 8#326 /Odieresis 8#327 /multiply -8#330 /Oslash 8#331 /Ugrave 8#332 /Uacute 8#333 /Ucircumflex -8#334 /Udieresis 8#335 /Yacute 8#336 /Thorn 8#337 /germandbls 8#340 /agrave -8#341 /aacute 8#342 /acircumflex 8#343 /atilde 8#344 /adieresis 8#345 /aring -8#346 /ae 8#347 /ccedilla 8#350 /egrave 8#351 /eacute -8#352 /ecircumflex 8#353 /edieresis 8#354 /igrave 8#355 /iacute -8#356 /icircumflex 8#357 /idieresis 8#360 /eth 8#361 /ntilde 8#362 /ograve -8#363 /oacute 8#364 /ocircumflex 8#365 /otilde 8#366 /odieresis 8#367 /divide -8#370 /oslash 8#371 /ugrave 8#372 /uacute 8#373 /ucircumflex -8#374 /udieresis 8#375 /yacute 8#376 /thorn 8#377 /ydieresis] def -/Times-Bold /Times-Bold-iso isovec ReEncode -/Times-Roman /Times-Roman-iso isovec ReEncode -/$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def -/$F2psEnd {$F2psEnteredState restore end} def - -$F2psBegin -10 setmiterlimit - 0.06000 0.06000 sc -% -% Fig objects follow -% -% Polyline -15.000 slw -n 6000 300 m - 6000 3975 l gs col0 s gr -% Polyline -7.500 slw -gs clippath -3615 555 m 3615 495 l 3464 495 l 3584 525 l 3464 555 l cp -eoclip -n 1200 525 m - 3600 525 l gs col0 s gr gr - -% arrowhead -n 3464 555 m 3584 525 l 3464 495 l 3464 555 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -1185 795 m 1185 855 l 1336 855 l 1216 825 l 1336 795 l cp -eoclip -n 3600 825 m - 1200 825 l gs col0 s gr gr - -% arrowhead -n 1336 795 m 1216 825 l 1336 855 l 1336 795 l cp gs 0.00 setgray ef gr col0 s -% Polyline -15.000 slw -n 1200 300 m - 1200 3975 l gs col0 s gr -% Polyline -7.500 slw -gs clippath -3615 1155 m 3615 1095 l 3464 1095 l 3584 1125 l 3464 1155 l cp -eoclip -n 1200 1125 m - 3600 1125 l gs col0 s gr gr - -% arrowhead -n 3464 1155 m 3584 1125 l 3464 1095 l 3464 1155 l cp gs 0.00 setgray ef gr col0 s -% Polyline -15.000 slw -n 3600 300 m - 3600 3975 l gs col0 s gr -% Polyline -7.500 slw -gs clippath -6015 1230 m 6015 1170 l 5864 1170 l 5984 1200 l 5864 1230 l cp -eoclip -n 3600 1200 m - 6000 1200 l gs col0 s gr gr - -% arrowhead -n 5864 1230 m 5984 1200 l 5864 1170 l 5864 1230 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -3585 1470 m 3585 1530 l 3736 1530 l 3616 1500 l 3736 1470 l cp -eoclip -n 6000 1500 m - 3600 1500 l gs col0 s gr gr - -% arrowhead -n 3736 1470 m 3616 1500 l 3736 1530 l 3736 1470 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -1185 1545 m 1185 1605 l 1336 1605 l 1216 1575 l 1336 1545 l cp -eoclip -n 3600 1575 m - 1200 1575 l gs col0 s gr gr - -% arrowhead -n 1336 1545 m 1216 1575 l 1336 1605 l 1336 1545 l cp gs 0.00 setgray ef gr col0 s -% Polyline - [15 45] 45 sd -n 1050 1800 m - 8325 1800 l gs col0 s gr [] 0 sd -% Polyline -gs clippath -3615 2130 m 3615 2070 l 3464 2070 l 3584 2100 l 3464 2130 l cp -eoclip -n 1200 2100 m - 3600 2100 l gs col0 s gr gr - -% arrowhead -n 3464 2130 m 3584 2100 l 3464 2070 l 3464 2130 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -6015 2205 m 6015 2145 l 5864 2145 l 5984 2175 l 5864 2205 l cp -eoclip -n 3600 2175 m - 6000 2175 l gs col0 s gr gr - -% arrowhead -n 5864 2205 m 5984 2175 l 5864 2145 l 5864 2205 l cp gs 0.00 setgray ef gr col0 s -% Polyline - [60] 0 sd -gs clippath -8190 2430 m 8190 2370 l 8039 2370 l 8159 2400 l 8039 2430 l cp -5985 2370 m 5985 2430 l 6136 2430 l 6016 2400 l 6136 2370 l cp -eoclip -n 6000 2400 m - 8175 2400 l gs col0 s gr gr - [] 0 sd -% arrowhead -n 6136 2370 m 6016 2400 l 6136 2430 l 6136 2370 l cp gs 0.00 setgray ef gr col0 s -% arrowhead -n 8039 2430 m 8159 2400 l 8039 2370 l 8039 2430 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -3585 2520 m 3585 2580 l 3736 2580 l 3616 2550 l 3736 2520 l cp -eoclip -n 6000 2550 m - 3600 2550 l gs col0 s gr gr - -% arrowhead -n 3736 2520 m 3616 2550 l 3736 2580 l 3736 2520 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -1185 2595 m 1185 2655 l 1336 2655 l 1216 2625 l 1336 2595 l cp -eoclip -n 3600 2625 m - 1200 2625 l gs col0 s gr gr - -% arrowhead -n 1336 2595 m 1216 2625 l 1336 2655 l 1336 2595 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -3615 3030 m 3615 2970 l 3464 2970 l 3584 3000 l 3464 3030 l cp -eoclip -n 1200 3000 m - 3600 3000 l gs col0 s gr gr - -% arrowhead -n 3464 3030 m 3584 3000 l 3464 2970 l 3464 3030 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -6015 3105 m 6015 3045 l 5864 3045 l 5984 3075 l 5864 3105 l cp -eoclip -n 3600 3075 m - 6000 3075 l gs col0 s gr gr - -% arrowhead -n 5864 3105 m 5984 3075 l 5864 3045 l 5864 3105 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -8190 3180 m 8190 3120 l 8039 3120 l 8159 3150 l 8039 3180 l cp -eoclip -n 6000 3150 m - 8175 3150 l gs col0 s gr gr - -% arrowhead -n 8039 3180 m 8159 3150 l 8039 3120 l 8039 3180 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -5985 3420 m 5985 3480 l 6136 3480 l 6016 3450 l 6136 3420 l cp -eoclip -n 8175 3450 m - 6000 3450 l gs col0 s gr gr - -% arrowhead -n 6136 3420 m 6016 3450 l 6136 3480 l 6136 3420 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -5985 3495 m 5985 3555 l 6136 3555 l 6016 3525 l 6136 3495 l cp -eoclip -n 8175 3525 m - 6000 3525 l gs col0 s gr gr - -% arrowhead -n 6136 3495 m 6016 3525 l 6136 3555 l 6136 3495 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -5985 3570 m 5985 3630 l 6136 3630 l 6016 3600 l 6136 3570 l cp -eoclip -n 8175 3600 m - 6000 3600 l gs col0 s gr gr - -% arrowhead -n 6136 3570 m 6016 3600 l 6136 3630 l 6136 3570 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -3585 3495 m 3585 3555 l 3736 3555 l 3616 3525 l 3736 3495 l cp -eoclip -n 6000 3525 m - 3600 3525 l gs col0 s gr gr - -% arrowhead -n 3736 3495 m 3616 3525 l 3736 3555 l 3736 3495 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -3585 3645 m 3585 3705 l 3736 3705 l 3616 3675 l 3736 3645 l cp -eoclip -n 6000 3675 m - 3600 3675 l gs col0 s gr gr - -% arrowhead -n 3736 3645 m 3616 3675 l 3736 3705 l 3736 3645 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -3585 3570 m 3585 3630 l 3736 3630 l 3616 3600 l 3736 3570 l cp -eoclip -n 6000 3600 m - 3600 3600 l gs col0 s gr gr - -% arrowhead -n 3736 3570 m 3616 3600 l 3736 3630 l 3736 3570 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -1185 3645 m 1185 3705 l 1336 3705 l 1216 3675 l 1336 3645 l cp -eoclip -n 3600 3675 m - 1200 3675 l gs col0 s gr gr - -% arrowhead -n 1336 3645 m 1216 3675 l 1336 3705 l 1336 3645 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -1185 3720 m 1185 3780 l 1336 3780 l 1216 3750 l 1336 3720 l cp -eoclip -n 3600 3750 m - 1200 3750 l gs col0 s gr gr - -% arrowhead -n 1336 3720 m 1216 3750 l 1336 3780 l 1336 3720 l cp gs 0.00 setgray ef gr col0 s -% Polyline -gs clippath -1185 3795 m 1185 3855 l 1336 3855 l 1216 3825 l 1336 3795 l cp -eoclip -n 3600 3825 m - 1200 3825 l gs col0 s gr gr - -% arrowhead -n 1336 3795 m 1216 3825 l 1336 3855 l 1336 3795 l cp gs 0.00 setgray ef gr col0 s -% Polyline -15.000 slw -n 8175 300 m - 8175 3975 l gs col0 s gr -% Polyline -7.500 slw -n 6300 825 m 7950 825 l 7950 1725 l 6300 1725 l - cp gs col7 1.00 shd ef gr gs col0 s gr -/Times-Bold-iso ff 180.00 scf sf -3375 225 m -gs 1 -1 sc (OR 1) col0 sh gr -/Times-Bold-iso ff 180.00 scf sf -1050 225 m -gs 1 -1 sc (Alice) col0 sh gr -/Times-Bold-iso ff 180.00 scf sf -5775 225 m -gs 1 -1 sc (OR 2) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -6075 3075 m -gs 1 -1 sc ("HTTP GET...") col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -4800 3975 m -gs 1 -1 sc (. . .) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7125 3975 m -gs 1 -1 sc (. . .) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -2400 3975 m -gs 1 -1 sc (. . .) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7125 2325 m -gs 1 -1 sc (\(TCP handshake\)) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Bold-iso ff 180.00 scf sf -7875 225 m -gs 1 -1 sc (website) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7125 1425 m -gs 1 -1 sc ({X}--AES encryption) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7125 1200 m -gs 1 -1 sc (E\(x\)--RSA encryption) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7125 975 m -gs 1 -1 sc (Legend:) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -2400 225 m -gs 1 -1 sc (\(link is TLS-encrypted\)) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -1275 1050 m -gs 1 -1 sc (Relay c1{Extend, OR2, E\(g^x2\)}) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -1275 2025 m -gs 1 -1 sc (Relay c1{{Begin <website>:80}}) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3525 1500 m -gs 1 -1 sc (Relay c1{Extended, g^y2, H\(K2\)}) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3675 2100 m -gs 1 -1 sc (Relay c2{Begin <website>:80}) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3525 2550 m -gs 1 -1 sc (Relay c1{{Connected}}) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -5925 2475 m -gs 1 -1 sc (Relay c2{Connected}) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -1275 2925 m -gs 1 -1 sc (Relay c1{{Data, "HTTP GET..."}}) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3675 3000 m -gs 1 -1 sc (Relay c2{Data, "HTTP GET..."}) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -4800 225 m -gs 1 -1 sc (\(link is TLS-encryped\)) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7050 225 m -gs 1 -1 sc (\(unencrypted\)) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -7125 1650 m -gs 1 -1 sc (cN--a circID) dup sw pop 2 div neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3525 3600 m -gs 1 -1 sc (Relay c1{{Data, \(response\)}}) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -8100 3375 m -gs 1 -1 sc (\(response\)) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -5925 3450 m -gs 1 -1 sc (Relay c2{Data, \(response\)}) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -5925 1425 m -gs 1 -1 sc (Created c2, g^y2, H\(K2\)) dup sw pop neg 0 rm col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3675 1125 m -gs 1 -1 sc (Create c2, E\(g^x2\)) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -1275 450 m -gs 1 -1 sc (Create c1, E\(g^x1\)) col0 sh gr -/Times-Roman-iso ff 150.00 scf sf -3525 750 m -gs 1 -1 sc (Created c1, g^y1, H\(K1\)) dup sw pop neg 0 rm col0 sh gr -$F2psEnd -rs diff --git a/doc/design-paper/interaction.fig b/doc/design-paper/interaction.fig deleted file mode 100644 index a7b49e0a5..000000000 --- a/doc/design-paper/interaction.fig +++ /dev/null @@ -1,122 +0,0 @@ -#FIG 3.2 -Landscape -Center -Inches -Letter -100.00 -Single --2 -1200 2 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 2 - 6000 300 6000 3975 -2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 1200 525 3600 525 -2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 825 1200 825 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 2 - 1200 300 1200 3975 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 1200 1125 3600 1125 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 2 - 3600 300 3600 3975 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 1200 6000 1200 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 6000 1500 3600 1500 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 1575 1200 1575 -2 1 2 1 0 7 50 0 -1 3.000 0 0 -1 0 0 2 - 1050 1800 8325 1800 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 1200 2100 3600 2100 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 2175 6000 2175 -2 1 1 1 0 7 50 0 -1 4.000 0 0 -1 1 1 2 - 1 1 1.00 60.00 120.00 - 1 1 1.00 60.00 120.00 - 6000 2400 8175 2400 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 6000 2550 3600 2550 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 2625 1200 2625 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 1200 3000 3600 3000 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 3075 6000 3075 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 6000 3150 8175 3150 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 8175 3450 6000 3450 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 8175 3525 6000 3525 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 8175 3600 6000 3600 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 6000 3525 3600 3525 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 6000 3675 3600 3675 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 6000 3600 3600 3600 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 3675 1200 3675 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 3750 1200 3750 -2 1 0 1 0 7 50 0 -1 3.000 0 0 -1 1 0 2 - 1 1 1.00 60.00 120.00 - 3600 3825 1200 3825 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 2 - 8175 300 8175 3975 -2 2 0 1 0 7 50 0 20 3.000 0 0 -1 0 0 5 - 6300 825 7950 825 7950 1725 6300 1725 6300 825 -4 0 0 50 0 2 12 0.0000 4 135 450 3375 225 OR 1\001 -4 0 0 50 0 2 12 0.0000 4 135 420 1050 225 Alice\001 -4 0 0 50 0 2 12 0.0000 4 135 450 5775 225 OR 2\001 -4 0 0 50 0 0 10 0.0000 4 105 960 6075 3075 "HTTP GET..."\001 -4 1 0 50 0 0 10 0.0000 4 15 135 4800 3975 . . .\001 -4 1 0 50 0 0 10 0.0000 4 15 135 7125 3975 . . .\001 -4 1 0 50 0 0 10 0.0000 4 15 135 2400 3975 . . .\001 -4 1 0 50 0 0 10 0.0000 4 135 1050 7125 2325 (TCP handshake)\001 -4 0 0 50 0 2 12 0.0000 4 135 630 7875 225 website\001 -4 1 0 50 0 0 10 0.0000 4 135 1335 7125 1425 {X}--AES encryption\001 -4 1 0 50 0 0 10 0.0000 4 135 1410 7125 1200 E(x)--RSA encryption\001 -4 1 0 50 0 0 10 0.0000 4 135 480 7125 975 Legend:\001 -4 1 0 50 0 0 10 0.0000 4 135 1455 2400 225 (link is TLS-encrypted)\001 -4 0 0 50 0 0 10 0.0000 4 135 2085 1275 1050 Relay c1{Extend, OR2, E(g^x2)}\001 -4 0 0 50 0 0 10 0.0000 4 135 1965 1275 2025 Relay c1{{Begin <website>:80}}\001 -4 2 0 50 0 0 10 0.0000 4 135 2190 3525 1500 Relay c1{Extended, g^y2, H(K2)}\001 -4 0 0 50 0 0 10 0.0000 4 135 1845 3675 2100 Relay c2{Begin <website>:80}\001 -4 2 0 50 0 0 10 0.0000 4 135 1410 3525 2550 Relay c1{{Connected}}\001 -4 2 0 50 0 0 10 0.0000 4 135 1290 5925 2475 Relay c2{Connected}\001 -4 0 0 50 0 0 10 0.0000 4 135 2085 1275 2925 Relay c1{{Data, "HTTP GET..."}}\001 -4 0 0 50 0 0 10 0.0000 4 135 1965 3675 3000 Relay c2{Data, "HTTP GET..."}\001 -4 1 0 50 0 0 10 0.0000 4 135 1365 4800 225 (link is TLS-encryped)\001 -4 1 0 50 0 0 10 0.0000 4 135 870 7050 225 (unencrypted)\001 -4 1 0 50 0 0 10 0.0000 4 105 780 7125 1650 cN--a circID\001 -4 2 0 50 0 0 10 0.0000 4 135 1860 3525 3600 Relay c1{{Data, (response)}}\001 -4 2 0 50 0 0 10 0.0000 4 135 645 8100 3375 (response)\001 -4 2 0 50 0 0 10 0.0000 4 135 1650 5925 3450 Relay c2{Data, (response)}\001 -4 2 0 50 0 0 10 0.0000 4 135 1545 5925 1425 Created c2, g^y2, H(K2)\001 -4 0 0 50 0 0 10 0.0000 4 135 1170 3675 1125 Create c2, E(g^x2)\001 -4 0 0 50 0 0 10 0.0000 4 135 1170 1275 450 Create c1, E(g^x1)\001 -4 2 0 50 0 0 10 0.0000 4 135 1545 3525 750 Created c1, g^y1, H(K1)\001 diff --git a/doc/design-paper/interaction.pdf b/doc/design-paper/interaction.pdf Binary files differdeleted file mode 100644 index 8def0add5..000000000 --- a/doc/design-paper/interaction.pdf +++ /dev/null diff --git a/doc/design-paper/interaction.png b/doc/design-paper/interaction.png Binary files differdeleted file mode 100644 index 2bb904fcd..000000000 --- a/doc/design-paper/interaction.png +++ /dev/null diff --git a/doc/design-paper/latex8.bst b/doc/design-paper/latex8.bst deleted file mode 100644 index 2dd324963..000000000 --- a/doc/design-paper/latex8.bst +++ /dev/null @@ -1,1124 +0,0 @@ - -% --------------------------------------------------------------- -% -% $Id$ -% -% by Paolo.Ienne@di.epfl.ch -% - -% --------------------------------------------------------------- -% -% no guarantee is given that the format corresponds perfectly to -% IEEE 8.5" x 11" Proceedings, but most features should be ok. -% -% --------------------------------------------------------------- -% -% `latex8' from BibTeX standard bibliography style `abbrv' -% version 0.99a for BibTeX versions 0.99a or later, LaTeX version 2.09. -% Copyright (C) 1985, all rights reserved. -% Copying of this file is authorized only if either -% (1) you make absolutely no changes to your copy, including name, or -% (2) if you do make changes, you name it something other than -% btxbst.doc, plain.bst, unsrt.bst, alpha.bst, and abbrv.bst. -% This restriction helps ensure that all standard styles are identical. -% The file btxbst.doc has the documentation for this style. - -ENTRY - { address - author - booktitle - chapter - edition - editor - howpublished - institution - journal - key - month - note - number - organization - pages - publisher - school - series - title - type - volume - year - } - {} - { label } - -INTEGERS { output.state before.all mid.sentence after.sentence after.block } - -FUNCTION {init.state.consts} -{ #0 'before.all := - #1 'mid.sentence := - #2 'after.sentence := - #3 'after.block := -} - -STRINGS { s t } - -FUNCTION {output.nonnull} -{ 's := - output.state mid.sentence = - { ", " * write$ } - { output.state after.block = - { add.period$ write$ - newline$ - "\newblock " write$ - } - { output.state before.all = - 'write$ - { add.period$ " " * write$ } - if$ - } - if$ - mid.sentence 'output.state := - } - if$ - s -} - -FUNCTION {output} -{ duplicate$ empty$ - 'pop$ - 'output.nonnull - if$ -} - -FUNCTION {output.check} -{ 't := - duplicate$ empty$ - { pop$ "empty " t * " in " * cite$ * warning$ } - 'output.nonnull - if$ -} - -FUNCTION {output.bibitem} -{ newline$ - "\bibitem{" write$ - cite$ write$ - "}" write$ - newline$ - "" - before.all 'output.state := -} - -FUNCTION {fin.entry} -{ add.period$ - write$ - newline$ -} - -FUNCTION {new.block} -{ output.state before.all = - 'skip$ - { after.block 'output.state := } - if$ -} - -FUNCTION {new.sentence} -{ output.state after.block = - 'skip$ - { output.state before.all = - 'skip$ - { after.sentence 'output.state := } - if$ - } - if$ -} - -FUNCTION {not} -{ { #0 } - { #1 } - if$ -} - -FUNCTION {and} -{ 'skip$ - { pop$ #0 } - if$ -} - -FUNCTION {or} -{ { pop$ #1 } - 'skip$ - if$ -} - -FUNCTION {new.block.checka} -{ empty$ - 'skip$ - 'new.block - if$ -} - -FUNCTION {new.block.checkb} -{ empty$ - swap$ empty$ - and - 'skip$ - 'new.block - if$ -} - -FUNCTION {new.sentence.checka} -{ empty$ - 'skip$ - 'new.sentence - if$ -} - -FUNCTION {new.sentence.checkb} -{ empty$ - swap$ empty$ - and - 'skip$ - 'new.sentence - if$ -} - -FUNCTION {field.or.null} -{ duplicate$ empty$ - { pop$ "" } - 'skip$ - if$ -} - -FUNCTION {emphasize} -{ duplicate$ empty$ - { pop$ "" } - { "{\em " swap$ * "}" * } - if$ -} - -INTEGERS { nameptr namesleft numnames } - -FUNCTION {format.names} -{ 's := - #1 'nameptr := - s num.names$ 'numnames := - numnames 'namesleft := - { namesleft #0 > } - { s nameptr "{f.~}{vv~}{ll}{, jj}" format.name$ 't := - nameptr #1 > - { namesleft #1 > - { ", " * t * } - { numnames #2 > - { "," * } - 'skip$ - if$ - t "others" = - { " et~al." * } - { " and " * t * } - if$ - } - if$ - } - 't - if$ - nameptr #1 + 'nameptr := - - namesleft #1 - 'namesleft := - } - while$ -} - -FUNCTION {format.authors} -{ author empty$ - { "" } - { author format.names } - if$ -} - -FUNCTION {format.editors} -{ editor empty$ - { "" } - { editor format.names - editor num.names$ #1 > - { ", editors" * } - { ", editor" * } - if$ - } - if$ -} - -FUNCTION {format.title} -{ title empty$ - { "" } - { title "t" change.case$ } - if$ -} - -FUNCTION {n.dashify} -{ 't := - "" - { t empty$ not } - { t #1 #1 substring$ "-" = - { t #1 #2 substring$ "--" = not - { "--" * - t #2 global.max$ substring$ 't := - } - { { t #1 #1 substring$ "-" = } - { "-" * - t #2 global.max$ substring$ 't := - } - while$ - } - if$ - } - { t #1 #1 substring$ * - t #2 global.max$ substring$ 't := - } - if$ - } - while$ -} - -FUNCTION {format.date} -{ year empty$ - { month empty$ - { "" } - { "there's a month but no year in " cite$ * warning$ - month - } - if$ - } - { month empty$ - 'year - { month " " * year * } - if$ - } - if$ -} - -FUNCTION {format.btitle} -{ title emphasize -} - -FUNCTION {tie.or.space.connect} -{ duplicate$ text.length$ #3 < - { "~" } - { " " } - if$ - swap$ * * -} - -FUNCTION {either.or.check} -{ empty$ - 'pop$ - { "can't use both " swap$ * " fields in " * cite$ * warning$ } - if$ -} - -FUNCTION {format.bvolume} -{ volume empty$ - { "" } - { "volume" volume tie.or.space.connect - series empty$ - 'skip$ - { " of " * series emphasize * } - if$ - "volume and number" number either.or.check - } - if$ -} - -FUNCTION {format.number.series} -{ volume empty$ - { number empty$ - { series field.or.null } - { output.state mid.sentence = - { "number" } - { "Number" } - if$ - number tie.or.space.connect - series empty$ - { "there's a number but no series in " cite$ * warning$ } - { " in " * series * } - if$ - } - if$ - } - { "" } - if$ -} - -FUNCTION {format.edition} -{ edition empty$ - { "" } - { output.state mid.sentence = - { edition "l" change.case$ " edition" * } - { edition "t" change.case$ " edition" * } - if$ - } - if$ -} - -INTEGERS { multiresult } - -FUNCTION {multi.page.check} -{ 't := - #0 'multiresult := - { multiresult not - t empty$ not - and - } - { t #1 #1 substring$ - duplicate$ "-" = - swap$ duplicate$ "," = - swap$ "+" = - or or - { #1 'multiresult := } - { t #2 global.max$ substring$ 't := } - if$ - } - while$ - multiresult -} - -FUNCTION {format.pages} -{ pages empty$ - { "" } - { pages multi.page.check - { "pages" pages n.dashify tie.or.space.connect } - { "page" pages tie.or.space.connect } - if$ - } - if$ -} - -FUNCTION {format.vol.num.pages} -{ volume field.or.null - number empty$ - 'skip$ - { "(" number * ")" * * - volume empty$ - { "there's a number but no volume in " cite$ * warning$ } - 'skip$ - if$ - } - if$ - pages empty$ - 'skip$ - { duplicate$ empty$ - { pop$ format.pages } - { ":" * pages n.dashify * } - if$ - } - if$ -} - -FUNCTION {format.chapter.pages} -{ chapter empty$ - 'format.pages - { type empty$ - { "chapter" } - { type "l" change.case$ } - if$ - chapter tie.or.space.connect - pages empty$ - 'skip$ - { ", " * format.pages * } - if$ - } - if$ -} - -FUNCTION {format.in.ed.booktitle} -{ booktitle empty$ - { "" } - { editor empty$ - { "In " booktitle emphasize * } - { "In " format.editors * ", " * booktitle emphasize * } - if$ - } - if$ -} - -FUNCTION {empty.misc.check} - -{ author empty$ title empty$ howpublished empty$ - month empty$ year empty$ note empty$ - and and and and and - key empty$ not and - { "all relevant fields are empty in " cite$ * warning$ } - 'skip$ - if$ -} - -FUNCTION {format.thesis.type} -{ type empty$ - 'skip$ - { pop$ - type "t" change.case$ - } - if$ -} - -FUNCTION {format.tr.number} -{ type empty$ - { "Technical Report" } - 'type - if$ - number empty$ - { "t" change.case$ } - { number tie.or.space.connect } - if$ -} - -FUNCTION {format.article.crossref} -{ key empty$ - { journal empty$ - { "need key or journal for " cite$ * " to crossref " * crossref * - warning$ - "" - } - { "In {\em " journal * "\/}" * } - if$ - } - { "In " key * } - if$ - " \cite{" * crossref * "}" * -} - -FUNCTION {format.crossref.editor} -{ editor #1 "{vv~}{ll}" format.name$ - editor num.names$ duplicate$ - #2 > - { pop$ " et~al." * } - { #2 < - 'skip$ - { editor #2 "{ff }{vv }{ll}{ jj}" format.name$ "others" = - { " et~al." * } - { " and " * editor #2 "{vv~}{ll}" format.name$ * } - if$ - } - if$ - } - if$ -} - -FUNCTION {format.book.crossref} -{ volume empty$ - { "empty volume in " cite$ * "'s crossref of " * crossref * warning$ - "In " - } - { "Volume" volume tie.or.space.connect - " of " * - } - if$ - editor empty$ - editor field.or.null author field.or.null = - or - { key empty$ - { series empty$ - { "need editor, key, or series for " cite$ * " to crossref " * - crossref * warning$ - "" * - } - { "{\em " * series * "\/}" * } - if$ - } - { key * } - if$ - } - { format.crossref.editor * } - if$ - " \cite{" * crossref * "}" * -} - -FUNCTION {format.incoll.inproc.crossref} -{ editor empty$ - editor field.or.null author field.or.null = - or - { key empty$ - { booktitle empty$ - { "need editor, key, or booktitle for " cite$ * " to crossref " * - crossref * warning$ - "" - } - { "In {\em " booktitle * "\/}" * } - if$ - } - { "In " key * } - if$ - } - { "In " format.crossref.editor * } - if$ - " \cite{" * crossref * "}" * -} - -FUNCTION {article} -{ output.bibitem - format.authors "author" output.check - new.block - format.title "title" output.check - new.block - crossref missing$ - { journal emphasize "journal" output.check - format.vol.num.pages output - format.date "year" output.check - } - { format.article.crossref output.nonnull - format.pages output - } - if$ - new.block - note output - fin.entry -} - -FUNCTION {book} -{ output.bibitem - author empty$ - { format.editors "author and editor" output.check } - { format.authors output.nonnull - crossref missing$ - { "author and editor" editor either.or.check } - 'skip$ - if$ - } - if$ - new.block - format.btitle "title" output.check - crossref missing$ - { format.bvolume output - new.block - format.number.series output - new.sentence - publisher "publisher" output.check - address output - } - { new.block - format.book.crossref output.nonnull - } - if$ - format.edition output - format.date "year" output.check - new.block - note output - fin.entry -} - -FUNCTION {booklet} -{ output.bibitem - format.authors output - new.block - format.title "title" output.check - howpublished address new.block.checkb - howpublished output - address output - format.date output - new.block - note output - fin.entry -} - -FUNCTION {inbook} -{ output.bibitem - author empty$ - { format.editors "author and editor" output.check } - { format.authors output.nonnull - - crossref missing$ - { "author and editor" editor either.or.check } - 'skip$ - if$ - } - if$ - new.block - format.btitle "title" output.check - crossref missing$ - { format.bvolume output - format.chapter.pages "chapter and pages" output.check - new.block - format.number.series output - new.sentence - publisher "publisher" output.check - address output - } - { format.chapter.pages "chapter and pages" output.check - new.block - format.book.crossref output.nonnull - } - if$ - format.edition output - format.date "year" output.check - new.block - note output - fin.entry -} - -FUNCTION {incollection} -{ output.bibitem - format.authors "author" output.check - new.block - format.title "title" output.check - new.block - crossref missing$ - { format.in.ed.booktitle "booktitle" output.check - format.bvolume output - format.number.series output - format.chapter.pages output - new.sentence - publisher "publisher" output.check - address output - format.edition output - format.date "year" output.check - } - { format.incoll.inproc.crossref output.nonnull - format.chapter.pages output - } - if$ - new.block - note output - fin.entry -} - -FUNCTION {inproceedings} -{ output.bibitem - format.authors "author" output.check - new.block - format.title "title" output.check - new.block - crossref missing$ - { format.in.ed.booktitle "booktitle" output.check - format.bvolume output - format.number.series output - format.pages output - address empty$ - { organization publisher new.sentence.checkb - organization output - publisher output - format.date "year" output.check - } - { address output.nonnull - format.date "year" output.check - new.sentence - organization output - publisher output - } - if$ - } - { format.incoll.inproc.crossref output.nonnull - format.pages output - } - if$ - new.block - note output - fin.entry -} - -FUNCTION {conference} { inproceedings } - -FUNCTION {manual} -{ output.bibitem - author empty$ - { organization empty$ - 'skip$ - { organization output.nonnull - address output - } - if$ - } - { format.authors output.nonnull } - if$ - new.block - format.btitle "title" output.check - author empty$ - { organization empty$ - { address new.block.checka - address output - } - 'skip$ - if$ - } - { organization address new.block.checkb - organization output - address output - } - if$ - format.edition output - format.date output - new.block - note output - fin.entry -} - -FUNCTION {mastersthesis} -{ output.bibitem - format.authors "author" output.check - new.block - format.title "title" output.check - new.block - "Master's thesis" format.thesis.type output.nonnull - school "school" output.check - address output - format.date "year" output.check - new.block - note output - fin.entry -} - -FUNCTION {misc} -{ output.bibitem - format.authors output - title howpublished new.block.checkb - format.title output - howpublished new.block.checka - howpublished output - format.date output - new.block - note output - fin.entry - empty.misc.check -} - -FUNCTION {phdthesis} -{ output.bibitem - format.authors "author" output.check - new.block - format.btitle "title" output.check - new.block - "PhD thesis" format.thesis.type output.nonnull - school "school" output.check - address output - format.date "year" output.check - new.block - note output - fin.entry -} - -FUNCTION {proceedings} -{ output.bibitem - editor empty$ - { organization output } - { format.editors output.nonnull } - - if$ - new.block - format.btitle "title" output.check - format.bvolume output - format.number.series output - address empty$ - { editor empty$ - { publisher new.sentence.checka } - { organization publisher new.sentence.checkb - organization output - } - if$ - publisher output - format.date "year" output.check - } - { address output.nonnull - format.date "year" output.check - new.sentence - editor empty$ - 'skip$ - { organization output } - if$ - publisher output - } - if$ - new.block - note output - fin.entry -} - -FUNCTION {techreport} -{ output.bibitem - format.authors "author" output.check - new.block - format.title "title" output.check - new.block - format.tr.number output.nonnull - institution "institution" output.check - address output - format.date "year" output.check - new.block - note output - fin.entry -} - -FUNCTION {unpublished} -{ output.bibitem - format.authors "author" output.check - new.block - format.title "title" output.check - new.block - note "note" output.check - format.date output - fin.entry -} - -FUNCTION {default.type} { misc } - -MACRO {jan} {"Jan."} - -MACRO {feb} {"Feb."} - -MACRO {mar} {"Mar."} - -MACRO {apr} {"Apr."} - -MACRO {may} {"May"} - -MACRO {jun} {"June"} - -MACRO {jul} {"July"} - -MACRO {aug} {"Aug."} - -MACRO {sep} {"Sept."} - -MACRO {oct} {"Oct."} - -MACRO {nov} {"Nov."} - -MACRO {dec} {"Dec."} - -MACRO {acmcs} {"ACM Comput. Surv."} - -MACRO {acta} {"Acta Inf."} - -MACRO {cacm} {"Commun. ACM"} - -MACRO {ibmjrd} {"IBM J. Res. Dev."} - -MACRO {ibmsj} {"IBM Syst.~J."} - -MACRO {ieeese} {"IEEE Trans. Softw. Eng."} - -MACRO {ieeetc} {"IEEE Trans. Comput."} - -MACRO {ieeetcad} - {"IEEE Trans. Comput.-Aided Design Integrated Circuits"} - -MACRO {ipl} {"Inf. Process. Lett."} - -MACRO {jacm} {"J.~ACM"} - -MACRO {jcss} {"J.~Comput. Syst. Sci."} - -MACRO {scp} {"Sci. Comput. Programming"} - -MACRO {sicomp} {"SIAM J. Comput."} - -MACRO {tocs} {"ACM Trans. Comput. Syst."} - -MACRO {tods} {"ACM Trans. Database Syst."} - -MACRO {tog} {"ACM Trans. Gr."} - -MACRO {toms} {"ACM Trans. Math. Softw."} - -MACRO {toois} {"ACM Trans. Office Inf. Syst."} - -MACRO {toplas} {"ACM Trans. Prog. Lang. Syst."} - -MACRO {tcs} {"Theoretical Comput. Sci."} - -READ - -FUNCTION {sortify} -{ purify$ - "l" change.case$ -} - -INTEGERS { len } - -FUNCTION {chop.word} -{ 's := - 'len := - s #1 len substring$ = - { s len #1 + global.max$ substring$ } - 's - if$ -} - -FUNCTION {sort.format.names} -{ 's := - #1 'nameptr := - "" - s num.names$ 'numnames := - numnames 'namesleft := - { namesleft #0 > } - { nameptr #1 > - { " " * } - 'skip$ - if$ - s nameptr "{vv{ } }{ll{ }}{ f{ }}{ jj{ }}" format.name$ 't := - nameptr numnames = t "others" = and - { "et al" * } - { t sortify * } - if$ - nameptr #1 + 'nameptr := - namesleft #1 - 'namesleft := - } - while$ -} - -FUNCTION {sort.format.title} -{ 't := - "A " #2 - "An " #3 - "The " #4 t chop.word - chop.word - chop.word - sortify - #1 global.max$ substring$ -} - -FUNCTION {author.sort} -{ author empty$ - { key empty$ - { "to sort, need author or key in " cite$ * warning$ - "" - } - { key sortify } - if$ - } - { author sort.format.names } - if$ -} - -FUNCTION {author.editor.sort} -{ author empty$ - { editor empty$ - { key empty$ - { "to sort, need author, editor, or key in " cite$ * warning$ - "" - } - { key sortify } - if$ - } - { editor sort.format.names } - if$ - } - { author sort.format.names } - if$ -} - -FUNCTION {author.organization.sort} -{ author empty$ - - { organization empty$ - { key empty$ - { "to sort, need author, organization, or key in " cite$ * warning$ - "" - } - { key sortify } - if$ - } - { "The " #4 organization chop.word sortify } - if$ - } - { author sort.format.names } - if$ -} - -FUNCTION {editor.organization.sort} -{ editor empty$ - { organization empty$ - { key empty$ - { "to sort, need editor, organization, or key in " cite$ * warning$ - "" - } - { key sortify } - if$ - } - { "The " #4 organization chop.word sortify } - if$ - } - { editor sort.format.names } - if$ -} - -FUNCTION {presort} -{ type$ "book" = - type$ "inbook" = - or - 'author.editor.sort - { type$ "proceedings" = - 'editor.organization.sort - { type$ "manual" = - 'author.organization.sort - 'author.sort - if$ - } - if$ - } - if$ - " " - * - year field.or.null sortify - * - " " - * - title field.or.null - sort.format.title - * - #1 entry.max$ substring$ - 'sort.key$ := -} - -ITERATE {presort} - -SORT - -STRINGS { longest.label } - -INTEGERS { number.label longest.label.width } - -FUNCTION {initialize.longest.label} -{ "" 'longest.label := - #1 'number.label := - #0 'longest.label.width := -} - -FUNCTION {longest.label.pass} -{ number.label int.to.str$ 'label := - number.label #1 + 'number.label := - label width$ longest.label.width > - { label 'longest.label := - label width$ 'longest.label.width := - } - 'skip$ - if$ -} - -EXECUTE {initialize.longest.label} - -ITERATE {longest.label.pass} - -FUNCTION {begin.bib} -{ preamble$ empty$ - 'skip$ - { preamble$ write$ newline$ } - if$ - "\begin{thebibliography}{" longest.label * - "}\setlength{\itemsep}{-1ex}\small" * write$ newline$ -} - -EXECUTE {begin.bib} - -EXECUTE {init.state.consts} - -ITERATE {call.type$} - -FUNCTION {end.bib} -{ newline$ - "\end{thebibliography}" write$ newline$ -} - -EXECUTE {end.bib} - -% end of file latex8.bst -% --------------------------------------------------------------- - - - diff --git a/doc/design-paper/llncs.cls b/doc/design-paper/llncs.cls deleted file mode 100644 index 697dd774e..000000000 --- a/doc/design-paper/llncs.cls +++ /dev/null @@ -1,1016 +0,0 @@ -% LLNCS DOCUMENT CLASS -- version 2.8 -% for LaTeX2e -% -\NeedsTeXFormat{LaTeX2e}[1995/12/01] -\ProvidesClass{llncs}[2000/05/16 v2.8 -^^JLaTeX document class for Lecture Notes in Computer Science] -% Options -\let\if@envcntreset\iffalse -\DeclareOption{envcountreset}{\let\if@envcntreset\iftrue} -\DeclareOption{citeauthoryear}{\let\citeauthoryear=Y} -\DeclareOption{oribibl}{\let\oribibl=Y} -\let\if@custvec\iftrue -\DeclareOption{orivec}{\let\if@custvec\iffalse} -\let\if@envcntsame\iffalse -\DeclareOption{envcountsame}{\let\if@envcntsame\iftrue} -\let\if@envcntsect\iffalse -\DeclareOption{envcountsect}{\let\if@envcntsect\iftrue} -\let\if@runhead\iffalse -\DeclareOption{runningheads}{\let\if@runhead\iftrue} - -\let\if@openbib\iffalse -\DeclareOption{openbib}{\let\if@openbib\iftrue} - -\DeclareOption*{\PassOptionsToClass{\CurrentOption}{article}} - -\ProcessOptions - -\LoadClass[twoside]{article} -\RequirePackage{multicol} % needed for the list of participants, index - -\setlength{\textwidth}{12.2cm} -\setlength{\textheight}{19.3cm} - -% Ragged bottom for the actual page -\def\thisbottomragged{\def\@textbottom{\vskip\z@ plus.0001fil -\global\let\@textbottom\relax}} - -\renewcommand\small{% - \@setfontsize\small\@ixpt{11}% - \abovedisplayskip 8.5\p@ \@plus3\p@ \@minus4\p@ - \abovedisplayshortskip \z@ \@plus2\p@ - \belowdisplayshortskip 4\p@ \@plus2\p@ \@minus2\p@ - \def\@listi{\leftmargin\leftmargini - \parsep 0\p@ \@plus1\p@ \@minus\p@ - \topsep 8\p@ \@plus2\p@ \@minus4\p@ - \itemsep0\p@}% - \belowdisplayskip \abovedisplayskip -} - -\frenchspacing -\widowpenalty=10000 -\clubpenalty=10000 - -\setlength\oddsidemargin {63\p@} -\setlength\evensidemargin {63\p@} -\setlength\marginparwidth {90\p@} - -\setlength\headsep {16\p@} - -\setlength\footnotesep{7.7\p@} -\setlength\textfloatsep{8mm\@plus 2\p@ \@minus 4\p@} -\setlength\intextsep {8mm\@plus 2\p@ \@minus 2\p@} - -\setcounter{secnumdepth}{2} - -\newcounter {chapter} -\renewcommand\thechapter {\@arabic\c@chapter} - -\newif\if@mainmatter \@mainmattertrue -\newcommand\frontmatter{\cleardoublepage - \@mainmatterfalse\pagenumbering{Roman}} -\newcommand\mainmatter{\cleardoublepage - \@mainmattertrue\pagenumbering{arabic}} -\newcommand\backmatter{\if@openright\cleardoublepage\else\clearpage\fi - \@mainmatterfalse} - -\renewcommand\part{\cleardoublepage - \thispagestyle{empty}% - \if@twocolumn - \onecolumn - \@tempswatrue - \else - \@tempswafalse - \fi - \null\vfil - \secdef\@part\@spart} - -\def\@part[#1]#2{% - \ifnum \c@secnumdepth >-2\relax - \refstepcounter{part}% - \addcontentsline{toc}{part}{\thepart\hspace{1em}#1}% - \else - \addcontentsline{toc}{part}{#1}% - \fi - \markboth{}{}% - {\centering - \interlinepenalty \@M - \normalfont - \ifnum \c@secnumdepth >-2\relax - \huge\bfseries \partname~\thepart - \par - \vskip 20\p@ - \fi - \Huge \bfseries #2\par}% - \@endpart} -\def\@spart#1{% - {\centering - \interlinepenalty \@M - \normalfont - \Huge \bfseries #1\par}% - \@endpart} -\def\@endpart{\vfil\newpage - \if@twoside - \null - \thispagestyle{empty}% - \newpage - \fi - \if@tempswa - \twocolumn - \fi} - -\newcommand\chapter{\clearpage - \thispagestyle{empty}% - \global\@topnum\z@ - \@afterindentfalse - \secdef\@chapter\@schapter} -\def\@chapter[#1]#2{\ifnum \c@secnumdepth >\m@ne - \if@mainmatter - \refstepcounter{chapter}% - \typeout{\@chapapp\space\thechapter.}% - \addcontentsline{toc}{chapter}% - {\protect\numberline{\thechapter}#1}% - \else - \addcontentsline{toc}{chapter}{#1}% - \fi - \else - \addcontentsline{toc}{chapter}{#1}% - \fi - \chaptermark{#1}% - \addtocontents{lof}{\protect\addvspace{10\p@}}% - \addtocontents{lot}{\protect\addvspace{10\p@}}% - \if@twocolumn - \@topnewpage[\@makechapterhead{#2}]% - \else - \@makechapterhead{#2}% - \@afterheading - \fi} -\def\@makechapterhead#1{% -% \vspace*{50\p@}% - {\centering - \ifnum \c@secnumdepth >\m@ne - \if@mainmatter - \large\bfseries \@chapapp{} \thechapter - \par\nobreak - \vskip 20\p@ - \fi - \fi - \interlinepenalty\@M - \Large \bfseries #1\par\nobreak - \vskip 40\p@ - }} -\def\@schapter#1{\if@twocolumn - \@topnewpage[\@makeschapterhead{#1}]% - \else - \@makeschapterhead{#1}% - \@afterheading - \fi} -\def\@makeschapterhead#1{% -% \vspace*{50\p@}% - {\centering - \normalfont - \interlinepenalty\@M - \Large \bfseries #1\par\nobreak - \vskip 40\p@ - }} - -\renewcommand\section{\@startsection{section}{1}{\z@}% - {-18\p@ \@plus -4\p@ \@minus -4\p@}% - {12\p@ \@plus 4\p@ \@minus 4\p@}% - {\normalfont\large\bfseries\boldmath - \rightskip=\z@ \@plus 8em\pretolerance=10000 }} -\renewcommand\subsection{\@startsection{subsection}{2}{\z@}% - {-18\p@ \@plus -4\p@ \@minus -4\p@}% - {8\p@ \@plus 4\p@ \@minus 4\p@}% - {\normalfont\normalsize\bfseries\boldmath - \rightskip=\z@ \@plus 8em\pretolerance=10000 }} -\renewcommand\subsubsection{\@startsection{subsubsection}{3}{\z@}% - {-18\p@ \@plus -4\p@ \@minus -4\p@}% - {-0.5em \@plus -0.22em \@minus -0.1em}% - {\normalfont\normalsize\bfseries\boldmath}} -\renewcommand\paragraph{\@startsection{paragraph}{4}{\z@}% - {-12\p@ \@plus -4\p@ \@minus -4\p@}% - {-0.5em \@plus -0.22em \@minus -0.1em}% - {\normalfont\normalsize\itshape}} -\renewcommand\subparagraph[1]{\typeout{LLNCS warning: You should not use - \string\subparagraph\space with this class}\vskip0.5cm -You should not use \verb|\subparagraph| with this class.\vskip0.5cm} - -\DeclareMathSymbol{\Gamma}{\mathalpha}{letters}{"00} -\DeclareMathSymbol{\Delta}{\mathalpha}{letters}{"01} -\DeclareMathSymbol{\Theta}{\mathalpha}{letters}{"02} -\DeclareMathSymbol{\Lambda}{\mathalpha}{letters}{"03} -\DeclareMathSymbol{\Xi}{\mathalpha}{letters}{"04} -\DeclareMathSymbol{\Pi}{\mathalpha}{letters}{"05} -\DeclareMathSymbol{\Sigma}{\mathalpha}{letters}{"06} -\DeclareMathSymbol{\Upsilon}{\mathalpha}{letters}{"07} -\DeclareMathSymbol{\Phi}{\mathalpha}{letters}{"08} -\DeclareMathSymbol{\Psi}{\mathalpha}{letters}{"09} -\DeclareMathSymbol{\Omega}{\mathalpha}{letters}{"0A} - -\let\footnotesize\small - -\if@custvec -\def\vec#1{\mathchoice{\mbox{\boldmath$\displaystyle#1$}} -{\mbox{\boldmath$\textstyle#1$}} -{\mbox{\boldmath$\scriptstyle#1$}} -{\mbox{\boldmath$\scriptscriptstyle#1$}}} -\fi - -\def\squareforqed{\hbox{\rlap{$\sqcap$}$\sqcup$}} -\def\qed{\ifmmode\squareforqed\else{\unskip\nobreak\hfil -\penalty50\hskip1em\null\nobreak\hfil\squareforqed -\parfillskip=0pt\finalhyphendemerits=0\endgraf}\fi} - -\def\getsto{\mathrel{\mathchoice {\vcenter{\offinterlineskip -\halign{\hfil -$\displaystyle##$\hfil\cr\gets\cr\to\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\textstyle##$\hfil\cr\gets -\cr\to\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptstyle##$\hfil\cr\gets -\cr\to\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptscriptstyle##$\hfil\cr -\gets\cr\to\cr}}}}} -\def\lid{\mathrel{\mathchoice {\vcenter{\offinterlineskip\halign{\hfil -$\displaystyle##$\hfil\cr<\cr\noalign{\vskip1.2pt}=\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\textstyle##$\hfil\cr<\cr -\noalign{\vskip1.2pt}=\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptstyle##$\hfil\cr<\cr -\noalign{\vskip1pt}=\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptscriptstyle##$\hfil\cr -<\cr -\noalign{\vskip0.9pt}=\cr}}}}} -\def\gid{\mathrel{\mathchoice {\vcenter{\offinterlineskip\halign{\hfil -$\displaystyle##$\hfil\cr>\cr\noalign{\vskip1.2pt}=\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\textstyle##$\hfil\cr>\cr -\noalign{\vskip1.2pt}=\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptstyle##$\hfil\cr>\cr -\noalign{\vskip1pt}=\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptscriptstyle##$\hfil\cr ->\cr -\noalign{\vskip0.9pt}=\cr}}}}} -\def\grole{\mathrel{\mathchoice {\vcenter{\offinterlineskip -\halign{\hfil -$\displaystyle##$\hfil\cr>\cr\noalign{\vskip-1pt}<\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\textstyle##$\hfil\cr ->\cr\noalign{\vskip-1pt}<\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptstyle##$\hfil\cr ->\cr\noalign{\vskip-0.8pt}<\cr}}} -{\vcenter{\offinterlineskip\halign{\hfil$\scriptscriptstyle##$\hfil\cr ->\cr\noalign{\vskip-0.3pt}<\cr}}}}} -\def\bbbr{{\rm I\!R}} %reelle Zahlen -\def\bbbm{{\rm I\!M}} -\def\bbbn{{\rm I\!N}} %natuerliche Zahlen -\def\bbbf{{\rm I\!F}} -\def\bbbh{{\rm I\!H}} -\def\bbbk{{\rm I\!K}} -\def\bbbp{{\rm I\!P}} -\def\bbbone{{\mathchoice {\rm 1\mskip-4mu l} {\rm 1\mskip-4mu l} -{\rm 1\mskip-4.5mu l} {\rm 1\mskip-5mu l}}} -\def\bbbc{{\mathchoice {\setbox0=\hbox{$\displaystyle\rm C$}\hbox{\hbox -to0pt{\kern0.4\wd0\vrule height0.9\ht0\hss}\box0}} -{\setbox0=\hbox{$\textstyle\rm C$}\hbox{\hbox -to0pt{\kern0.4\wd0\vrule height0.9\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptstyle\rm C$}\hbox{\hbox -to0pt{\kern0.4\wd0\vrule height0.9\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptscriptstyle\rm C$}\hbox{\hbox -to0pt{\kern0.4\wd0\vrule height0.9\ht0\hss}\box0}}}} -\def\bbbq{{\mathchoice {\setbox0=\hbox{$\displaystyle\rm -Q$}\hbox{\raise -0.15\ht0\hbox to0pt{\kern0.4\wd0\vrule height0.8\ht0\hss}\box0}} -{\setbox0=\hbox{$\textstyle\rm Q$}\hbox{\raise -0.15\ht0\hbox to0pt{\kern0.4\wd0\vrule height0.8\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptstyle\rm Q$}\hbox{\raise -0.15\ht0\hbox to0pt{\kern0.4\wd0\vrule height0.7\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptscriptstyle\rm Q$}\hbox{\raise -0.15\ht0\hbox to0pt{\kern0.4\wd0\vrule height0.7\ht0\hss}\box0}}}} -\def\bbbt{{\mathchoice {\setbox0=\hbox{$\displaystyle\rm -T$}\hbox{\hbox to0pt{\kern0.3\wd0\vrule height0.9\ht0\hss}\box0}} -{\setbox0=\hbox{$\textstyle\rm T$}\hbox{\hbox -to0pt{\kern0.3\wd0\vrule height0.9\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptstyle\rm T$}\hbox{\hbox -to0pt{\kern0.3\wd0\vrule height0.9\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptscriptstyle\rm T$}\hbox{\hbox -to0pt{\kern0.3\wd0\vrule height0.9\ht0\hss}\box0}}}} -\def\bbbs{{\mathchoice -{\setbox0=\hbox{$\displaystyle \rm S$}\hbox{\raise0.5\ht0\hbox -to0pt{\kern0.35\wd0\vrule height0.45\ht0\hss}\hbox -to0pt{\kern0.55\wd0\vrule height0.5\ht0\hss}\box0}} -{\setbox0=\hbox{$\textstyle \rm S$}\hbox{\raise0.5\ht0\hbox -to0pt{\kern0.35\wd0\vrule height0.45\ht0\hss}\hbox -to0pt{\kern0.55\wd0\vrule height0.5\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptstyle \rm S$}\hbox{\raise0.5\ht0\hbox -to0pt{\kern0.35\wd0\vrule height0.45\ht0\hss}\raise0.05\ht0\hbox -to0pt{\kern0.5\wd0\vrule height0.45\ht0\hss}\box0}} -{\setbox0=\hbox{$\scriptscriptstyle\rm S$}\hbox{\raise0.5\ht0\hbox -to0pt{\kern0.4\wd0\vrule height0.45\ht0\hss}\raise0.05\ht0\hbox -to0pt{\kern0.55\wd0\vrule height0.45\ht0\hss}\box0}}}} -\def\bbbz{{\mathchoice {\hbox{$\mathsf\textstyle Z\kern-0.4em Z$}} -{\hbox{$\mathsf\textstyle Z\kern-0.4em Z$}} -{\hbox{$\mathsf\scriptstyle Z\kern-0.3em Z$}} -{\hbox{$\mathsf\scriptscriptstyle Z\kern-0.2em Z$}}}} - -\let\ts\, - -\setlength\leftmargini {17\p@} -\setlength\leftmargin {\leftmargini} -\setlength\leftmarginii {\leftmargini} -\setlength\leftmarginiii {\leftmargini} -\setlength\leftmarginiv {\leftmargini} -\setlength \labelsep {.5em} -\setlength \labelwidth{\leftmargini} -\addtolength\labelwidth{-\labelsep} - -\def\@listI{\leftmargin\leftmargini - \parsep 0\p@ \@plus1\p@ \@minus\p@ - \topsep 8\p@ \@plus2\p@ \@minus4\p@ - \itemsep0\p@} -\let\@listi\@listI -\@listi -\def\@listii {\leftmargin\leftmarginii - \labelwidth\leftmarginii - \advance\labelwidth-\labelsep - \topsep 0\p@ \@plus2\p@ \@minus\p@} -\def\@listiii{\leftmargin\leftmarginiii - \labelwidth\leftmarginiii - \advance\labelwidth-\labelsep - \topsep 0\p@ \@plus\p@\@minus\p@ - \parsep \z@ - \partopsep \p@ \@plus\z@ \@minus\p@} - -\renewcommand\labelitemi{\normalfont\bfseries --} -\renewcommand\labelitemii{$\m@th\bullet$} - -\setlength\arraycolsep{1.4\p@} -\setlength\tabcolsep{1.4\p@} - -\def\tableofcontents{\chapter*{\contentsname\@mkboth{{\contentsname}}% - {{\contentsname}}} - \def\authcount##1{\setcounter{auco}{##1}\setcounter{@auth}{1}} - \def\lastand{\ifnum\value{auco}=2\relax - \unskip{} \andname\ - \else - \unskip \lastandname\ - \fi}% - \def\and{\stepcounter{@auth}\relax - \ifnum\value{@auth}=\value{auco}% - \lastand - \else - \unskip, - \fi}% - \@starttoc{toc}\if@restonecol\twocolumn\fi} - -\def\l@part#1#2{\addpenalty{\@secpenalty}% - \addvspace{2em plus\p@}% % space above part line - \begingroup - \parindent \z@ - \rightskip \z@ plus 5em - \hrule\vskip5pt - \large % same size as for a contribution heading - \bfseries\boldmath % set line in boldface - \leavevmode % TeX command to enter horizontal mode. - #1\par - \vskip5pt - \hrule - \vskip1pt - \nobreak % Never break after part entry - \endgroup} - -\def\@dotsep{2} - -\def\hyperhrefextend{\ifx\hyper@anchor\@undefined\else -{chapter.\thechapter}\fi} - -\def\addnumcontentsmark#1#2#3{% -\addtocontents{#1}{\protect\contentsline{#2}{\protect\numberline - {\thechapter}#3}{\thepage}\hyperhrefextend}} -\def\addcontentsmark#1#2#3{% -\addtocontents{#1}{\protect\contentsline{#2}{#3}{\thepage}\hyperhrefextend}} -\def\addcontentsmarkwop#1#2#3{% -\addtocontents{#1}{\protect\contentsline{#2}{#3}{0}\hyperhrefextend}} - -\def\@adcmk[#1]{\ifcase #1 \or -\def\@gtempa{\addnumcontentsmark}% - \or \def\@gtempa{\addcontentsmark}% - \or \def\@gtempa{\addcontentsmarkwop}% - \fi\@gtempa{toc}{chapter}} -\def\addtocmark{\@ifnextchar[{\@adcmk}{\@adcmk[3]}} - -\def\l@chapter#1#2{\addpenalty{-\@highpenalty} - \vskip 1.0em plus 1pt \@tempdima 1.5em \begingroup - \parindent \z@ \rightskip \@pnumwidth - \parfillskip -\@pnumwidth - \leavevmode \advance\leftskip\@tempdima \hskip -\leftskip - {\large\bfseries\boldmath#1}\ifx0#2\hfil\null - \else - \nobreak - \leaders\hbox{$\m@th \mkern \@dotsep mu.\mkern - \@dotsep mu$}\hfill - \nobreak\hbox to\@pnumwidth{\hss #2}% - \fi\par - \penalty\@highpenalty \endgroup} - -\def\l@title#1#2{\addpenalty{-\@highpenalty} - \addvspace{8pt plus 1pt} - \@tempdima \z@ - \begingroup - \parindent \z@ \rightskip \@tocrmarg - \parfillskip -\@tocrmarg - \leavevmode \advance\leftskip\@tempdima \hskip -\leftskip - #1\nobreak - \leaders\hbox{$\m@th \mkern \@dotsep mu.\mkern - \@dotsep mu$}\hfill - \nobreak\hbox to\@pnumwidth{\hss #2}\par - \penalty\@highpenalty \endgroup} - -\setcounter{tocdepth}{0} -\newdimen\tocchpnum -\newdimen\tocsecnum -\newdimen\tocsectotal -\newdimen\tocsubsecnum -\newdimen\tocsubsectotal -\newdimen\tocsubsubsecnum -\newdimen\tocsubsubsectotal -\newdimen\tocparanum -\newdimen\tocparatotal -\newdimen\tocsubparanum -\tocchpnum=\z@ % no chapter numbers -\tocsecnum=15\p@ % section 88. plus 2.222pt -\tocsubsecnum=23\p@ % subsection 88.8 plus 2.222pt -\tocsubsubsecnum=27\p@ % subsubsection 88.8.8 plus 1.444pt -\tocparanum=35\p@ % paragraph 88.8.8.8 plus 1.666pt -\tocsubparanum=43\p@ % subparagraph 88.8.8.8.8 plus 1.888pt -\def\calctocindent{% -\tocsectotal=\tocchpnum -\advance\tocsectotal by\tocsecnum -\tocsubsectotal=\tocsectotal -\advance\tocsubsectotal by\tocsubsecnum -\tocsubsubsectotal=\tocsubsectotal -\advance\tocsubsubsectotal by\tocsubsubsecnum -\tocparatotal=\tocsubsubsectotal -\advance\tocparatotal by\tocparanum} -\calctocindent - -\def\l@section{\@dottedtocline{1}{\tocchpnum}{\tocsecnum}} -\def\l@subsection{\@dottedtocline{2}{\tocsectotal}{\tocsubsecnum}} -\def\l@subsubsection{\@dottedtocline{3}{\tocsubsectotal}{\tocsubsubsecnum}} -\def\l@paragraph{\@dottedtocline{4}{\tocsubsubsectotal}{\tocparanum}} -\def\l@subparagraph{\@dottedtocline{5}{\tocparatotal}{\tocsubparanum}} - -\def\listoffigures{\@restonecolfalse\if@twocolumn\@restonecoltrue\onecolumn - \fi\section*{\listfigurename\@mkboth{{\listfigurename}}{{\listfigurename}}} - \@starttoc{lof}\if@restonecol\twocolumn\fi} -\def\l@figure{\@dottedtocline{1}{0em}{1.5em}} - -\def\listoftables{\@restonecolfalse\if@twocolumn\@restonecoltrue\onecolumn - \fi\section*{\listtablename\@mkboth{{\listtablename}}{{\listtablename}}} - \@starttoc{lot}\if@restonecol\twocolumn\fi} -\let\l@table\l@figure - -\renewcommand\listoffigures{% - \section*{\listfigurename - \@mkboth{\listfigurename}{\listfigurename}}% - \@starttoc{lof}% - } - -\renewcommand\listoftables{% - \section*{\listtablename - \@mkboth{\listtablename}{\listtablename}}% - \@starttoc{lot}% - } - -\ifx\oribibl\undefined -\ifx\citeauthoryear\undefined -\renewenvironment{thebibliography}[1] - {\section*{\refname} - \def\@biblabel##1{##1.} - \small - \list{\@biblabel{\@arabic\c@enumiv}}% - {\settowidth\labelwidth{\@biblabel{#1}}% - \leftmargin\labelwidth - \advance\leftmargin\labelsep - \if@openbib - \advance\leftmargin\bibindent - \itemindent -\bibindent - \listparindent \itemindent - \parsep \z@ - \fi - \usecounter{enumiv}% - \let\p@enumiv\@empty - \renewcommand\theenumiv{\@arabic\c@enumiv}}% - \if@openbib - \renewcommand\newblock{\par}% - \else - \renewcommand\newblock{\hskip .11em \@plus.33em \@minus.07em}% - \fi - \sloppy\clubpenalty4000\widowpenalty4000% - \sfcode`\.=\@m} - {\def\@noitemerr - {\@latex@warning{Empty `thebibliography' environment}}% - \endlist} -\def\@lbibitem[#1]#2{\item[{[#1]}\hfill]\if@filesw - {\let\protect\noexpand\immediate - \write\@auxout{\string\bibcite{#2}{#1}}}\fi\ignorespaces} -\newcount\@tempcntc -\def\@citex[#1]#2{\if@filesw\immediate\write\@auxout{\string\citation{#2}}\fi - \@tempcnta\z@\@tempcntb\m@ne\def\@citea{}\@cite{\@for\@citeb:=#2\do - {\@ifundefined - {b@\@citeb}{\@citeo\@tempcntb\m@ne\@citea\def\@citea{,}{\bfseries - ?}\@warning - {Citation `\@citeb' on page \thepage \space undefined}}% - {\setbox\z@\hbox{\global\@tempcntc0\csname b@\@citeb\endcsname\relax}% - \ifnum\@tempcntc=\z@ \@citeo\@tempcntb\m@ne - \@citea\def\@citea{,}\hbox{\csname b@\@citeb\endcsname}% - \else - \advance\@tempcntb\@ne - \ifnum\@tempcntb=\@tempcntc - \else\advance\@tempcntb\m@ne\@citeo - \@tempcnta\@tempcntc\@tempcntb\@tempcntc\fi\fi}}\@citeo}{#1}} -\def\@citeo{\ifnum\@tempcnta>\@tempcntb\else - \@citea\def\@citea{,\,\hskip\z@skip}% - \ifnum\@tempcnta=\@tempcntb\the\@tempcnta\else - {\advance\@tempcnta\@ne\ifnum\@tempcnta=\@tempcntb \else - \def\@citea{--}\fi - \advance\@tempcnta\m@ne\the\@tempcnta\@citea\the\@tempcntb}\fi\fi} -\else -\renewenvironment{thebibliography}[1] - {\section*{\refname} - \small - \list{}% - {\settowidth\labelwidth{}% - \leftmargin\parindent - \itemindent=-\parindent - \labelsep=\z@ - \if@openbib - \advance\leftmargin\bibindent - \itemindent -\bibindent - \listparindent \itemindent - \parsep \z@ - \fi - \usecounter{enumiv}% - \let\p@enumiv\@empty - \renewcommand\theenumiv{}}% - \if@openbib - \renewcommand\newblock{\par}% - \else - \renewcommand\newblock{\hskip .11em \@plus.33em \@minus.07em}% - \fi - \sloppy\clubpenalty4000\widowpenalty4000% - \sfcode`\.=\@m} - {\def\@noitemerr - {\@latex@warning{Empty `thebibliography' environment}}% - \endlist} - \def\@cite#1{#1}% - \def\@lbibitem[#1]#2{\item[]\if@filesw - {\def\protect##1{\string ##1\space}\immediate - \write\@auxout{\string\bibcite{#2}{#1}}}\fi\ignorespaces} - \fi -\else -\@cons\@openbib@code{\noexpand\small} -\fi - -\def\idxquad{\hskip 10\p@}% space that divides entry from number - -\def\@idxitem{\par\hangindent 10\p@} - -\def\subitem{\par\setbox0=\hbox{--\enspace}% second order - \noindent\hangindent\wd0\box0}% index entry - -\def\subsubitem{\par\setbox0=\hbox{--\,--\enspace}% third - \noindent\hangindent\wd0\box0}% order index entry - -\def\indexspace{\par \vskip 10\p@ plus5\p@ minus3\p@\relax} - -\renewenvironment{theindex} - {\@mkboth{\indexname}{\indexname}% - \thispagestyle{empty}\parindent\z@ - \parskip\z@ \@plus .3\p@\relax - \let\item\par - \def\,{\relax\ifmmode\mskip\thinmuskip - \else\hskip0.2em\ignorespaces\fi}% - \normalfont\small - \begin{multicols}{2}[\@makeschapterhead{\indexname}]% - } - {\end{multicols}} - -\renewcommand\footnoterule{% - \kern-3\p@ - \hrule\@width 2truecm - \kern2.6\p@} - \newdimen\fnindent - \fnindent1em -\long\def\@makefntext#1{% - \parindent \fnindent% - \leftskip \fnindent% - \noindent - \llap{\hb@xt@1em{\hss\@makefnmark\ }}\ignorespaces#1} - -\long\def\@makecaption#1#2{% - \vskip\abovecaptionskip - \sbox\@tempboxa{{\bfseries #1.} #2}% - \ifdim \wd\@tempboxa >\hsize - {\bfseries #1.} #2\par - \else - \global \@minipagefalse - \hb@xt@\hsize{\hfil\box\@tempboxa\hfil}% - \fi - \vskip\belowcaptionskip} - -\def\fps@figure{htbp} -\def\fnum@figure{\figurename\thinspace\thefigure} -\def \@floatboxreset {% - \reset@font - \small - \@setnobreak - \@setminipage -} -\def\fps@table{htbp} -\def\fnum@table{\tablename~\thetable} -\renewenvironment{table} - {\setlength\abovecaptionskip{0\p@}% - \setlength\belowcaptionskip{10\p@}% - \@float{table}} - {\end@float} -\renewenvironment{table*} - {\setlength\abovecaptionskip{0\p@}% - \setlength\belowcaptionskip{10\p@}% - \@dblfloat{table}} - {\end@dblfloat} - -\long\def\@caption#1[#2]#3{\par\addcontentsline{\csname - ext@#1\endcsname}{#1}{\protect\numberline{\csname - the#1\endcsname}{\ignorespaces #2}}\begingroup - \@parboxrestore - \@makecaption{\csname fnum@#1\endcsname}{\ignorespaces #3}\par - \endgroup} - -% LaTeX does not provide a command to enter the authors institute -% addresses. The \institute command is defined here. - -\newcounter{@inst} -\newcounter{@auth} -\newcounter{auco} -\def\andname{and} -\def\lastandname{\unskip, and} -\newdimen\instindent -\newbox\authrun -\newtoks\authorrunning -\newtoks\tocauthor -\newbox\titrun -\newtoks\titlerunning -\newtoks\toctitle - -\def\clearheadinfo{\gdef\@author{No Author Given}% - \gdef\@title{No Title Given}% - \gdef\@subtitle{}% - \gdef\@institute{No Institute Given}% - \gdef\@thanks{}% - \global\titlerunning={}\global\authorrunning={}% - \global\toctitle={}\global\tocauthor={}} - -\def\institute#1{\gdef\@institute{#1}} - -\def\institutename{\par - \begingroup - \parskip=\z@ - \parindent=\z@ - \setcounter{@inst}{1}% - \def\and{\par\stepcounter{@inst}% - \noindent$^{\the@inst}$\enspace\ignorespaces}% - \setbox0=\vbox{\def\thanks##1{}\@institute}% - \ifnum\c@@inst=1\relax - \else - \setcounter{footnote}{\c@@inst}% - \setcounter{@inst}{1}% - \noindent$^{\the@inst}$\enspace - \fi - \ignorespaces - \@institute\par - \endgroup} - -\def\@fnsymbol#1{\ensuremath{\ifcase#1\or\star\or{\star\star}\or - {\star\star\star}\or \dagger\or \ddagger\or - \mathchar "278\or \mathchar "27B\or \|\or **\or \dagger\dagger - \or \ddagger\ddagger \else\@ctrerr\fi}} - -\def\inst#1{\unskip$^{#1}$} -\def\fnmsep{\unskip$^,$} -\def\email#1{{\tt#1}} -\AtBeginDocument{\@ifundefined{url}{\def\url#1{#1}}{}} -\def\homedir{\~{ }} - -\def\subtitle#1{\gdef\@subtitle{#1}} -\clearheadinfo - -\renewcommand\maketitle{\newpage - \refstepcounter{chapter}% - \stepcounter{section}% - \setcounter{section}{0}% - \setcounter{subsection}{0}% - \setcounter{figure}{0} - \setcounter{table}{0} - \setcounter{equation}{0} - \setcounter{footnote}{0}% - \begingroup - \parindent=\z@ - \renewcommand\thefootnote{\@fnsymbol\c@footnote}% - \if@twocolumn - \ifnum \col@number=\@ne - \@maketitle - \else - \twocolumn[\@maketitle]% - \fi - \else - \newpage - \global\@topnum\z@ % Prevents figures from going at top of page. - \@maketitle - \fi - \thispagestyle{empty}\@thanks -% - \def\\{\unskip\ \ignorespaces}\def\inst##1{\unskip{}}% - \def\thanks##1{\unskip{}}\def\fnmsep{\unskip}% - \instindent=\hsize - \advance\instindent by-\headlineindent - \if!\the\toctitle!\addcontentsline{toc}{title}{\@title}\else - \addcontentsline{toc}{title}{\the\toctitle}\fi - \if@runhead - \if!\the\titlerunning!\else - \edef\@title{\the\titlerunning}% - \fi - \global\setbox\titrun=\hbox{\small\rm\unboldmath\ignorespaces\@title}% - \ifdim\wd\titrun>\instindent - \typeout{Title too long for running head. Please supply}% - \typeout{a shorter form with \string\titlerunning\space prior to - \string\maketitle}% - \global\setbox\titrun=\hbox{\small\rm - Title Suppressed Due to Excessive Length}% - \fi - \xdef\@title{\copy\titrun}% - \fi -% - \if!\the\tocauthor!\relax - {\def\and{\noexpand\protect\noexpand\and}% - \protected@xdef\toc@uthor{\@author}}% - \else - \def\\{\noexpand\protect\noexpand\newline}% - \protected@xdef\scratch{\the\tocauthor}% - \protected@xdef\toc@uthor{\scratch}% - \fi - \addtocontents{toc}{{\protect\raggedright\protect\leftskip15\p@ - \protect\rightskip\@tocrmarg - \protect\itshape\toc@uthor\protect\endgraf}}% - \if@runhead - \if!\the\authorrunning! - \value{@inst}=\value{@auth}% - \setcounter{@auth}{1}% - \else - \edef\@author{\the\authorrunning}% - \fi - \global\setbox\authrun=\hbox{\small\unboldmath\@author\unskip}% - \ifdim\wd\authrun>\instindent - \typeout{Names of authors too long for running head. Please supply}% - \typeout{a shorter form with \string\authorrunning\space prior to - \string\maketitle}% - \global\setbox\authrun=\hbox{\small\rm - Authors Suppressed Due to Excessive Length}% - \fi - \xdef\@author{\copy\authrun}% - \markboth{\@author}{\@title}% - \fi - \endgroup - \setcounter{footnote}{0}% - \clearheadinfo} -% -\def\@maketitle{\newpage - \markboth{}{}% - \def\lastand{\ifnum\value{@inst}=2\relax - \unskip{} \andname\ - \else - \unskip \lastandname\ - \fi}% - \def\and{\stepcounter{@auth}\relax - \ifnum\value{@auth}=\value{@inst}% - \lastand - \else - \unskip, - \fi}% - \begin{center}% - {\Large \bfseries\boldmath - \pretolerance=10000 - \@title \par}\vskip .8cm -\if!\@subtitle!\else {\large \bfseries\boldmath - \vskip -.65cm - \pretolerance=10000 - \@subtitle \par}\vskip .8cm\fi - \setbox0=\vbox{\setcounter{@auth}{1}\def\and{\stepcounter{@auth}}% - \def\thanks##1{}\@author}% - \global\value{@inst}=\value{@auth}% - \global\value{auco}=\value{@auth}% - \setcounter{@auth}{1}% -{\lineskip .5em -\noindent\ignorespaces -\@author\vskip.35cm} - {\small\institutename} - \end{center}% - } - -% definition of the "\spnewtheorem" command. -% -% Usage: -% -% \spnewtheorem{env_nam}{caption}[within]{cap_font}{body_font} -% or \spnewtheorem{env_nam}[numbered_like]{caption}{cap_font}{body_font} -% or \spnewtheorem*{env_nam}{caption}{cap_font}{body_font} -% -% New is "cap_font" and "body_font". It stands for -% fontdefinition of the caption and the text itself. -% -% "\spnewtheorem*" gives a theorem without number. -% -% A defined spnewthoerem environment is used as described -% by Lamport. -% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\def\@thmcountersep{} -\def\@thmcounterend{.} - -\def\spnewtheorem{\@ifstar{\@sthm}{\@Sthm}} - -% definition of \spnewtheorem with number - -\def\@spnthm#1#2{% - \@ifnextchar[{\@spxnthm{#1}{#2}}{\@spynthm{#1}{#2}}} -\def\@Sthm#1{\@ifnextchar[{\@spothm{#1}}{\@spnthm{#1}}} - -\def\@spxnthm#1#2[#3]#4#5{\expandafter\@ifdefinable\csname #1\endcsname - {\@definecounter{#1}\@addtoreset{#1}{#3}% - \expandafter\xdef\csname the#1\endcsname{\expandafter\noexpand - \csname the#3\endcsname \noexpand\@thmcountersep \@thmcounter{#1}}% - \expandafter\xdef\csname #1name\endcsname{#2}% - \global\@namedef{#1}{\@spthm{#1}{\csname #1name\endcsname}{#4}{#5}}% - \global\@namedef{end#1}{\@endtheorem}}} - -\def\@spynthm#1#2#3#4{\expandafter\@ifdefinable\csname #1\endcsname - {\@definecounter{#1}% - \expandafter\xdef\csname the#1\endcsname{\@thmcounter{#1}}% - \expandafter\xdef\csname #1name\endcsname{#2}% - \global\@namedef{#1}{\@spthm{#1}{\csname #1name\endcsname}{#3}{#4}}% - \global\@namedef{end#1}{\@endtheorem}}} - -\def\@spothm#1[#2]#3#4#5{% - \@ifundefined{c@#2}{\@latexerr{No theorem environment `#2' defined}\@eha}% - {\expandafter\@ifdefinable\csname #1\endcsname - {\global\@namedef{the#1}{\@nameuse{the#2}}% - \expandafter\xdef\csname #1name\endcsname{#3}% - \global\@namedef{#1}{\@spthm{#2}{\csname #1name\endcsname}{#4}{#5}}% - \global\@namedef{end#1}{\@endtheorem}}}} - -\def\@spthm#1#2#3#4{\topsep 7\p@ \@plus2\p@ \@minus4\p@ -\refstepcounter{#1}% -\@ifnextchar[{\@spythm{#1}{#2}{#3}{#4}}{\@spxthm{#1}{#2}{#3}{#4}}} - -\def\@spxthm#1#2#3#4{\@spbegintheorem{#2}{\csname the#1\endcsname}{#3}{#4}% - \ignorespaces} - -\def\@spythm#1#2#3#4[#5]{\@spopargbegintheorem{#2}{\csname - the#1\endcsname}{#5}{#3}{#4}\ignorespaces} - -\def\@spbegintheorem#1#2#3#4{\trivlist - \item[\hskip\labelsep{#3#1\ #2\@thmcounterend}]#4} - -\def\@spopargbegintheorem#1#2#3#4#5{\trivlist - \item[\hskip\labelsep{#4#1\ #2}]{#4(#3)\@thmcounterend\ }#5} - -% definition of \spnewtheorem* without number - -\def\@sthm#1#2{\@Ynthm{#1}{#2}} - -\def\@Ynthm#1#2#3#4{\expandafter\@ifdefinable\csname #1\endcsname - {\global\@namedef{#1}{\@Thm{\csname #1name\endcsname}{#3}{#4}}% - \expandafter\xdef\csname #1name\endcsname{#2}% - \global\@namedef{end#1}{\@endtheorem}}} - -\def\@Thm#1#2#3{\topsep 7\p@ \@plus2\p@ \@minus4\p@ -\@ifnextchar[{\@Ythm{#1}{#2}{#3}}{\@Xthm{#1}{#2}{#3}}} - -\def\@Xthm#1#2#3{\@Begintheorem{#1}{#2}{#3}\ignorespaces} - -\def\@Ythm#1#2#3[#4]{\@Opargbegintheorem{#1} - {#4}{#2}{#3}\ignorespaces} - -\def\@Begintheorem#1#2#3{#3\trivlist - \item[\hskip\labelsep{#2#1\@thmcounterend}]} - -\def\@Opargbegintheorem#1#2#3#4{#4\trivlist - \item[\hskip\labelsep{#3#1}]{#3(#2)\@thmcounterend\ }} - -\if@envcntsect - \def\@thmcountersep{.} - \spnewtheorem{theorem}{Theorem}[section]{\bfseries}{\itshape} -\else - \spnewtheorem{theorem}{Theorem}{\bfseries}{\itshape} - \if@envcntreset - \@addtoreset{theorem}{section} - \else - \@addtoreset{theorem}{chapter} - \fi -\fi - -%definition of divers theorem environments -\spnewtheorem*{claim}{Claim}{\itshape}{\rmfamily} -\spnewtheorem*{proof}{Proof}{\itshape}{\rmfamily} -\if@envcntsame % alle Umgebungen wie Theorem. - \def\spn@wtheorem#1#2#3#4{\@spothm{#1}[theorem]{#2}{#3}{#4}} -\else % alle Umgebungen mit eigenem Zaehler - \if@envcntsect % mit section numeriert - \def\spn@wtheorem#1#2#3#4{\@spxnthm{#1}{#2}[section]{#3}{#4}} - \else % nicht mit section numeriert - \if@envcntreset - \def\spn@wtheorem#1#2#3#4{\@spynthm{#1}{#2}{#3}{#4} - \@addtoreset{#1}{section}} - \else - \def\spn@wtheorem#1#2#3#4{\@spynthm{#1}{#2}{#3}{#4} - \@addtoreset{#1}{chapter}}% - \fi - \fi -\fi -\spn@wtheorem{case}{Case}{\itshape}{\rmfamily} -\spn@wtheorem{conjecture}{Conjecture}{\itshape}{\rmfamily} -\spn@wtheorem{corollary}{Corollary}{\bfseries}{\itshape} -\spn@wtheorem{definition}{Definition}{\bfseries}{\itshape} -\spn@wtheorem{example}{Example}{\itshape}{\rmfamily} -\spn@wtheorem{exercise}{Exercise}{\itshape}{\rmfamily} -\spn@wtheorem{lemma}{Lemma}{\bfseries}{\itshape} -\spn@wtheorem{note}{Note}{\itshape}{\rmfamily} -\spn@wtheorem{problem}{Problem}{\itshape}{\rmfamily} -\spn@wtheorem{property}{Property}{\itshape}{\rmfamily} -\spn@wtheorem{proposition}{Proposition}{\bfseries}{\itshape} -\spn@wtheorem{question}{Question}{\itshape}{\rmfamily} -\spn@wtheorem{solution}{Solution}{\itshape}{\rmfamily} -\spn@wtheorem{remark}{Remark}{\itshape}{\rmfamily} - -\def\@takefromreset#1#2{% - \def\@tempa{#1}% - \let\@tempd\@elt - \def\@elt##1{% - \def\@tempb{##1}% - \ifx\@tempa\@tempb\else - \@addtoreset{##1}{#2}% - \fi}% - \expandafter\expandafter\let\expandafter\@tempc\csname cl@#2\endcsname - \expandafter\def\csname cl@#2\endcsname{}% - \@tempc - \let\@elt\@tempd} - -\def\theopargself{\def\@spopargbegintheorem##1##2##3##4##5{\trivlist - \item[\hskip\labelsep{##4##1\ ##2}]{##4##3\@thmcounterend\ }##5} - \def\@Opargbegintheorem##1##2##3##4{##4\trivlist - \item[\hskip\labelsep{##3##1}]{##3##2\@thmcounterend\ }} - } - -\renewenvironment{abstract}{% - \list{}{\advance\topsep by0.35cm\relax\small - \leftmargin=1cm - \labelwidth=\z@ - \listparindent=\z@ - \itemindent\listparindent - \rightmargin\leftmargin}\item[\hskip\labelsep - \bfseries\abstractname]} - {\endlist} -\renewcommand{\abstractname}{Abstract.} -\renewcommand{\contentsname}{Table of Contents} -\renewcommand{\figurename}{Fig.} -\renewcommand{\tablename}{Table} - -\newdimen\headlineindent % dimension for space between -\headlineindent=1.166cm % number and text of headings. - -\def\ps@headings{\let\@mkboth\@gobbletwo - \let\@oddfoot\@empty\let\@evenfoot\@empty - \def\@evenhead{\normalfont\small\rlap{\thepage}\hspace{\headlineindent}% - \leftmark\hfil} - \def\@oddhead{\normalfont\small\hfil\rightmark\hspace{\headlineindent}% - \llap{\thepage}} - \def\chaptermark##1{}% - \def\sectionmark##1{}% - \def\subsectionmark##1{}} - -\def\ps@titlepage{\let\@mkboth\@gobbletwo - \let\@oddfoot\@empty\let\@evenfoot\@empty - \def\@evenhead{\normalfont\small\rlap{\thepage}\hspace{\headlineindent}% - \hfil} - \def\@oddhead{\normalfont\small\hfil\hspace{\headlineindent}% - \llap{\thepage}} - \def\chaptermark##1{}% - \def\sectionmark##1{}% - \def\subsectionmark##1{}} - -\if@runhead\ps@headings\else -\ps@empty\fi - -\setlength\arraycolsep{1.4\p@} -\setlength\tabcolsep{1.4\p@} - -\endinput - diff --git a/doc/design-paper/sptor.tex b/doc/design-paper/sptor.tex deleted file mode 100644 index 4b659eeda..000000000 --- a/doc/design-paper/sptor.tex +++ /dev/null @@ -1,353 +0,0 @@ -\documentclass{llncs} - -\usepackage{url} -\usepackage{amsmath} -\usepackage{epsfig} - -\setlength{\textwidth}{5.9in} -\setlength{\textheight}{8.4in} -\setlength{\topmargin}{.5cm} -\setlength{\oddsidemargin}{1cm} -\setlength{\evensidemargin}{1cm} - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} - - -\newcommand{\workingnote}[1]{} % The version that hides the note. -%\newcommand{\workingnote}[1]{(**#1)} % The version that makes the note visible. - - -\begin{document} - -\title{Design challenges and social factors in deploying low-latency anonymity} -% Could still use a better title -PFS - -\author{Roger Dingledine\inst{1} \and -Nick Mathewson\inst{1} \and -Paul Syverson\inst{2}} -\institute{The Tor Project \email{<\{arma,nickm\}@torproject.org>} \and -Naval Research Laboratory \email{<syverson@itd.nrl.navy.mil>}} - -\maketitle -\pagestyle{plain} - -\begin{abstract} - There are many unexpected or unexpectedly difficult obstacles to - deploying anonymous communications. We describe Tor (\emph{the} - onion routing), how to use it, our design philosophy, and some of - the challenges that we have faced and continue to face in building, - deploying, and sustaining a scalable, distributed, low-latency - anonymity network. -\end{abstract} - -\section{Introduction} -This article describes Tor, a widely-used low-latency general-purpose -anonymous communication system, and discusses some unexpected -challenges arising from our experiences deploying Tor. We will tell -you how to use it, who uses it, how it works, why we designed it the -way we did, and why this makes it usable and stable. - -Tor is an overlay network for anonymizing TCP streams over the -Internet~\cite{tor-design}. Tor works on the real-world Internet, -requires no special privileges or kernel modifications, requires -little synchronization or coordination between nodes, and provides a -reasonable trade-off between anonymity, usability, and efficiency. - -Since deployment in October 2003 the public Tor network has grown to -about a thousand volunteer-operated nodes worldwide and over 110 -megabytes average traffic per second from hundreds of thousands of -concurrent users. - -\section{Tor Design and Design Philosophy: Distributed Trust and Usability} - -Tor enables users to connect to Internet sites without revealing their -logical or physical locations to those sites or to observers. It -enables hosts to be publicly accessible yet have similar protection -against location through its \emph{location-hidden services}. - -To connect to a remote server via Tor the client software first learns -a %signed -list of Tor nodes from several central \emph{directory servers} via a -voting protocol (to avoid dependence on or complete trust in any one -of these servers). It then incrementally creates a private pathway or -\emph{circuit} across the network. This circuit consists of -encrypted connections through authenticated Tor nodes -whose public keys were obtained from the directory servers. The client -software negotiates a separate set of encryption keys for each hop along the -circuit. The nodes in the circuit are chosen at random by the client -subject to a preference for higher performing nodes to allocate -resources effectively and with a client-chosen preferred set of first -nodes called \emph{entry guards} to complicate profiling attacks by -internal adversaries~\cite{hs-attack}. -The circuit is extended one node at a time, tunneling extensions -through already established portions of the circuit, and each node -along the way knows only the immediately previous and following nodes -in the circuit, so no individual Tor node knows the complete path that -each fixed-sized data packet (or \emph{cell}) will take. Thus, -neither an eavesdropper nor a compromised node can see both the -connection's source and destination. Later requests use a new -circuit to complicate long-term linkability between different actions -by a single user. - -Tor attempts to anonymize the transport layer, not the application -layer. Thus, applications such as SSH can provide -authenticated communication that is hidden by Tor from outside observers. -When anonymity from communication partners is desired, -application-level protocols that transmit identifying -information need additional scrubbing proxies, such as -Privoxy~\cite{privoxy} for HTTP\@. Furthermore, Tor does not relay -arbitrary IP packets; it only anonymizes TCP streams and DNS requests. - -Tor, the third generation of deployed onion-routing -designs~\cite{or-ih96,or-jsac98,tor-design}, was researched, developed, -and deployed by the Naval Research Laboratory and the Free Haven -Project under ONR and DARPA funding for secure government -communications. In 2005, continuing work by Free Haven was funded by -the Electronic Frontier Foundation for maintaining civil liberties of -ordinary citizens online. In 2006, The Tor Project incorporated as a -non-profit and has received continued funding from the Omidyar Network, -the U.S. International Broadcasting Bureau, and other groups to combat -blocking and censorship on the Internet. This diversity of funding fits -Tor's overall philosophy: a wide variety of interests helps maintain -both the stability and the security of the network. - -Usability is also a central goal. Downloading and installing Tor is -easy. Simply go to\\ -http://www.torproject.org/ and download. Tor comes with install -wizards and a GUI for major operating systems: GNU/Linux, OS X, and -Windows. It also runs on various flavors of BSD and UNIX\@. Basic -instructions, documentation, FAQs, etc.\ are available in many -languages. The Tor GUI Vidalia makes server configuration easy, e.g., -choosing how much bandwidth to allocate to Tor, exit policy choices, -etc. And, the GUI Torbutton allows Firefox users a one-click toggle of -whether browsing goes through Tor or not. Tor is easily configured by -a site administrator to run at either individual desktops or just at a -site firewall or combinations of these. - -The ideal Tor network would be practical, useful and anonymous. When -trade-offs arise between these properties, Tor's research strategy has -been to remain useful enough to attract many users, and practical -enough to support them. Only subject to these constraints do we try -to maximize anonymity. Tor thus differs from other deployed systems -for traffic analysis resistance in its security and flexibility. Mix -networks such as -% Mixmaster~\cite{mixmaster-spec} or its successor -Mixminion~\cite{minion-design} gain the highest degrees of practical -anonymity at the expense of introducing highly variable delays, making -them unsuitable for applications such as web browsing. Commercial -single-hop proxies~\cite{anonymizer} can provide good performance, but -a single-point compromise can expose all users' traffic, and a -single-point eavesdropper can perform traffic analysis on the entire -network. Also, their proprietary implementations place any -infrastructure that depends on these single-hop solutions at the mercy -of their providers' financial health as well as network security. -There are numerous other designs for distributed anonymous low-latency -communication~\cite{crowds-tissec,web-mix,freedom21-security,i2p,tarzan:ccs02,morphmix:fc04}. -Some have been deployed or even commercialized; some exist only on -paper. Though each has something unique to offer, we feel Tor has -advantages over each of them that make it a superior choice for most -users and applications. For example, unlike purely P2P designs we -neither limit ordinary users to content and services available only -within our network nor require them to take on responsibility for -connections outside the network, unless they separately choose to run -server nodes. Nonetheless because we support low-latency interactive -communications, end-to-end \emph{traffic correlation} -attacks~\cite{danezis:pet2004,defensive-dropping,SS03,hs-attack,bauer:tr2007} -allow an attacker who can observe both ends of a communication to -correlate packet timing and volume, quickly linking the initiator to -her destination. - - -Our defense lies in having a diverse enough set of nodes to prevent -most real-world adversaries from being in the right places to attack -users, by distributing each transaction over several nodes in the -network. This ``distributed trust'' approach means the Tor network -can be safely operated and used by a wide variety of mutually -distrustful users, providing sustainability and security. - -The Tor network has a broad range of users, making it difficult for -eavesdroppers to track them or profile interests. These include -ordinary citizens concerned about their privacy, corporations who -don't want to reveal information to their competitors, and law -enforcement and government intelligence agencies who need to do -operations on the Internet without being noticed. Naturally, -organizations will not want to depend on others for their security. -If most participating providers are reliable, Tor tolerates some -hostile infiltration of the network. - -This distribution of trust is central to the Tor philosophy and -pervades Tor at all levels: Onion routing has been open source since -the mid-nineties (mistrusting users can inspect the code themselves); -Tor is free software (anyone could take up the development of Tor from -the current team); anyone can use Tor without license or charge (which -encourages a broad user base with diverse interests); Tor is designed to be -usable (also promotes a large, diverse user base) and configurable (so -users can easily set up and run server nodes); the Tor -infrastructure is run by volunteers (it is not dependent on the -economic viability or business strategy of any company) who are -scattered around the globe (not completely under the jurisdiction of -any single country); ongoing development and deployment has been -funded by diverse sources (development does not fully depend on -funding from any one source or even funding for any one primary -purpose or sources in any one jurisdiction). All of these contribute -to Tor's resilience and sustainability. - - -\section{Social challenges} - -Many of the issues the Tor project needs to address extend beyond -system design and technology development. In particular, the Tor -project's \emph{image} with respect to its users and the rest of the -Internet impacts the security it can provide. With this image issue -in mind, this section discusses the Tor user base and Tor's -interaction with other services on the Internet. - -\subsection{Communicating security} - -Usability for anonymity systems contributes to their security, because -usability affects the possible anonymity set~\cite{econymics,back01}. -Conversely, an unusable system attracts few users and thus can't -provide much anonymity. - -This phenomenon has a second-order effect: knowing this, users should -choose which anonymity system to use based in part on how usable and -secure \emph{others} will find it, in order to get the protection of a -larger anonymity set. Thus we might supplement the adage ``usability -is a security parameter''~\cite{back01} with a new one: ``perceived -usability is a security parameter.''~\cite{usability-network-effect}. - - - -\subsection{Reputability and perceived social value} -Another factor impacting the network's security is its reputability, -the perception of its social value based on its current user base. If -Alice is the only user who has ever downloaded the software, it might -be socially accepted, but she's not getting much anonymity. Add a -thousand activists, and she's anonymous, but everyone thinks she's an -activist too. Add a thousand diverse citizens (cancer survivors, -people concerned about identity theft, law enforcement agents, and so -on) and now she's harder to profile. - -Furthermore, the network's reputability affects its operator base: -more people are willing to run a service if they believe it will be -used by human rights workers than if they believe it will be used -exclusively for disreputable ends. This effect becomes stronger if -node operators themselves think they will be associated with their -users' ends. - -So the more cancer survivors on Tor, the better for the human rights -activists. The more malicious hackers, the worse for the normal -users. Thus, reputability is an anonymity issue for two -reasons. First, it impacts the sustainability of the network: a -network that's always about to be shut down has difficulty attracting -and keeping adequate nodes. Second, a disreputable network is more -vulnerable to legal and political attacks, since it will attract fewer -supporters. - -Reputability becomes even more tricky in the case of privacy networks, -since the good uses of the network (such as publishing by journalists -in dangerous countries, protecting road warriors from profiling and -potential physical harm, tracking of criminals by law enforcement, -protecting corporate research interests, etc.) are typically kept private, -whereas network abuses or other problems tend to be more widely -publicized. - - -\subsection{Abuse} -\label{subsec:tor-and-blacklists} - -For someone willing to be antisocial or even break the law, Tor is -usually a poor choice to hide bad behavior. For example, Tor nodes are -publicly identified, unlike the million-node botnets that are now -common on the Internet. Nonetheless, we always expected that, -alongside legitimate users, Tor would also attract troublemakers who -exploit Tor to abuse services on the Internet with vandalism, rude -mail, and so on. \emph{Exit policies} have allowed individual nodes -to block access to specific IP/port ranges. This approach aims to -make operators more willing to run Tor by allowing them to prevent -their nodes from being used for abusing particular services. For -example, by default Tor nodes block SMTP (port 25), to avoid the issue -of spam. - -Exit policies are useful but insufficient: if not all nodes block a -given service, that service may try to block Tor instead. While being -blockable is important to being good netizens, we would like to -encourage services to allow anonymous access. Services should not need -to decide between blocking legitimate anonymous use and allowing -unlimited abuse. Nonetheless, blocking IP addresses is a -course-grained solution~\cite{netauth}: entire apartment buildings, -campuses, and even countries sometimes share a single IP address. -Also, whether intended or not, such blocking supports repression of -free speech. In many locations where Internet access of various kinds -is censored or even punished by imprisonment, Tor is a path both to -the outside world and to others inside. Blocking posts from Tor makes -the job of censoring authorities easier. This is a loss for both Tor -and services that block, such as Wikipedia: we don't want to compete -for (or divvy up) the NAT-protected entities of the world. This is -also unfortunate because there are relatively simple technical -solutions~\cite{nym}. Various schemes for escrowing anonymous posts -until they are reviewed by editors would both prevent abuse and remove -incentives for attempts to abuse. Further, pseudonymous reputation -tracking of posters through Tor would allow those who establish -adequate reputation to post without escrow~\cite{nym,nymble}. - -We stress that as far as we can tell, most Tor uses are not -abusive. Most services have not complained, and others are actively -working to find ways besides banning to cope with the abuse. For -example, the Freenode IRC network had a problem with a coordinated -group of abusers joining channels and subtly taking over the -conversation; but when they labelled all users coming from Tor IP -addresses as ``anonymous users,'' removing the ability of the abusers -to blend in, the abusers stopped using Tor. This is an illustration of -how simple -technical mechanisms can remove the ability to abuse anonymously -without undermining the ability to communicate anonymously and can -thus remove the incentive to attempt abusing in this way. - - - -\section{The Future} -\label{sec:conclusion} - -Tor is the largest and most diverse low-latency anonymity network -available, but we are still in the early stages. Several major -questions remain. - -First, will our volunteer-based approach to sustainability continue to -work as well in the long term as it has the first several years? -Besides node operation, Tor research, deployment, maintainance, and -development is increasingly done by volunteers: package maintenance -for various OSes, document translation, GUI design and implementation, -live CDs, specification of new design changes, etc.\ -% -Second, Tor is only one of many components that preserve privacy -online. For applications where it is desirable to keep identifying -information out of application traffic, someone must build more and -better protocol-aware proxies that are usable by ordinary people. -% -Third, we need to maintain a reputation for social good, and learn how to -coexist with the variety of Internet services and their established -authentication mechanisms. We can't just keep escalating the blacklist -standoff forever. -% -Fourth, the current Tor architecture hardly scales even to handle -current user demand. We must deploy designs and incentives to further -encourage clients to relay traffic too, without thereby trading away -too much anonymity or other properties. - -These are difficult and open questions. Yet choosing not to solve them -means leaving most users to a less secure network or no anonymizing -network at all.\\ - -\noindent{\bf Acknowledgment:} Thanks to Matt Edman for many - helpful comments on a draft of this article. -\bibliographystyle{plain} \bibliography{tor-design} - -\end{document} - diff --git a/doc/design-paper/tor-design.bib b/doc/design-paper/tor-design.bib deleted file mode 100644 index 981761e94..000000000 --- a/doc/design-paper/tor-design.bib +++ /dev/null @@ -1,1493 +0,0 @@ -% hs-attack -@inproceedings{hs-attack, - title = {Locating Hidden Servers}, - author = {Lasse {\O}verlier and Paul Syverson}, - booktitle = {Proceedings of the 2006 IEEE Symposium on Security and Privacy}, - year = {2006}, - month = {May}, - publisher = {IEEE CS}, -} - - -@TechReport{bauer:tr2007, - author = {Kevin Bauer and Damon McCoy and Dirk Grunwald and Tadayoshi Kohno and Douglas Sicker}, - title = {Low-Resource Routing Attacks Against Anonymous Systems}, - institution = {University of Colorado at Boulder}, - year = 2007, - number = {CU-CS-1025-07} -} - -@inproceedings{bauer:wpes2007, - title = {Low-Resource Routing Attacks Against Tor}, - author = {Kevin Bauer and Damon McCoy and Dirk Grunwald and Tadayoshi Kohno and Douglas Sicker}, - booktitle = {{Proceedings of the Workshop on Privacy in the Electronic Society (WPES 2007)}}, - year = {2007}, - month = {October}, - address = {Washington, DC, USA}, -} - -% fix me -@misc{tannenbaum96, - author = "Andrew Tannenbaum", - title = "Computer Networks", - year = "1996", - publisher = "Prentice Hall, 3rd edition", -} - -@article{ meadows96, - author = "Catherine Meadows", - title = "The {NRL} Protocol Analyzer: An Overview", - journal = "Journal of Logic Programming", - volume = "26", - number = "2", - pages = "113--131", - year = "1996", -} -@inproceedings{kesdogan:pet2002, - title = {Unobservable Surfing on the World Wide Web: Is Private Information Retrieval an - alternative to the MIX based Approach?}, - author = {Dogan Kesdogan and Mark Borning and Michael Schmeink}, - booktitle = {Privacy Enhancing Technologies (PET 2002)}, - year = {2002}, - month = {April}, - editor = {Roger Dingledine and Paul Syverson}, - publisher = {Springer-Verlag, LNCS 2482}, -} - -@inproceedings{statistical-disclosure, - title = {Statistical Disclosure Attacks}, - author = {George Danezis}, - booktitle = {Security and Privacy in the Age of Uncertainty ({SEC2003})}, - organization = {{IFIP TC11}}, - year = {2003}, - month = {May}, - address = {Athens}, - pages = {421--426}, - publisher = {Kluwer}, -} - -@inproceedings{limits-open, - title = {Limits of Anonymity in Open Environments}, - author = {Dogan Kesdogan and Dakshi Agrawal and Stefan Penz}, - booktitle = {Information Hiding Workshop (IH 2002)}, - year = {2002}, - month = {October}, - editor = {Fabien Petitcolas}, - publisher = {Springer-Verlag, LNCS 2578}, -} - -@inproceedings{isdn-mixes, - title = {{ISDN-mixes: Untraceable communication with very small bandwidth overhead}}, - author = {Andreas Pfitzmann and Birgit Pfitzmann and Michael Waidner}, - booktitle = {GI/ITG Conference on Communication in Distributed Systems}, - year = {1991}, - month = {February}, - pages = {451-463}, -} - - -@Article{jerichow-jsac98, - author = {Anja Jerichow and Jan M\"{u}ller and Andreas - Pfitzmann and Birgit Pfitzmann and Michael Waidner}, - title = {Real-Time Mixes: A Bandwidth-Efficient Anonymity Protocol}, - journal = {IEEE Journal on Selected Areas in Communications}, - year = 1998, - volume = 16, - number = 4, - pages = {495--509}, - month = {May} -} - -@inproceedings{tarzan:ccs02, - title = {Tarzan: A Peer-to-Peer Anonymizing Network Layer}, - author = {Michael J. Freedman and Robert Morris}, - booktitle = {9th {ACM} {C}onference on {C}omputer and {C}ommunications - {S}ecurity ({CCS 2002})}, - year = {2002}, - month = {November}, - address = {Washington, DC}, -} - -@inproceedings{cebolla, - title = {{Cebolla: Pragmatic IP Anonymity}}, - author = {Zach Brown}, - booktitle = {Ottawa Linux Symposium}, - year = {2002}, - month = {June}, -} - -@inproceedings{eax, - author = "M. Bellare and P. Rogaway and D. Wagner", - title = {The {EAX} Mode of Operation: A Two-Pass Authenticated-Encryption Scheme Optimized for Simplicity and Efficiency}, - booktitle = {Fast Software Encryption 2004}, - month = {February}, - year = {2004}, -} - -@misc{darkside, - title = {{The Dark Side of the Web: An Open Proxy's View}}, - author = {Vivek S. Pai and Limin Wang and KyoungSoo Park and Ruoming Pang and Larry Peterson}, - note = {\newline \url{http://codeen.cs.princeton.edu/}}, -} -% note = {Submitted to HotNets-II. \url{http://codeen.cs.princeton.edu/}}, - -@Misc{anonymizer, - key = {anonymizer}, - title = {The {Anonymizer}}, - note = {\url{http://anonymizer.com/}} -} - -@Misc{privoxy, - key = {privoxy}, - title = {{Privoxy}}, - note = {\url{http://www.privoxy.org/}} -} - -@Misc{i2p, - key = {i2p}, - title = {{I2P}}, - note = {\url{http://www.i2p.net/}} -} - -@Misc{nym, - author = {Jason Holt}, - title = {nym: practical pseudonymity for anonymous networks}, - note = {Paper and source code at \url{http://www.lunkwill.org/src/nym/}} -} - -@InProceedings{nymble, - author = {Peter C. Johnson and Apu Kapadia and Patrick P. Tsang and Sean W. Smith}, - title = {Nymble: Anonymous {IP}-address Blocking}, - booktitle = {Privacy Enhancing Technologies (PET 2007)}, - year = 2007, - publisher = {Springer-Verlag, LNCS 4776} -} - -@inproceedings{anonnet, - title = {{Analysis of an Anonymity Network for Web Browsing}}, - author = {Marc Rennhard and Sandro Rafaeli and Laurent Mathy and Bernhard Plattner and - David Hutchison}, - booktitle = {{IEEE 7th Intl. Workshop on Enterprise Security (WET ICE - 2002)}}, - year = {2002}, - month = {June}, - address = {Pittsburgh, USA}, -} -% pages = {49--54}, - -@inproceedings{econymics, - title = {On the Economics of Anonymity}, - author = {Alessandro Acquisti and Roger Dingledine and Paul Syverson}, - booktitle = {Financial Cryptography}, - year = {2003}, - editor = {Rebecca N. Wright}, - publisher = {Springer-Verlag, LNCS 2742}, -} - -@inproceedings{defensive-dropping, - title = {Timing Analysis in Low-Latency Mix-Based Systems}, - author = {Brian N. Levine and Michael K. Reiter and Chenxi Wang and Matthew Wright}, - booktitle = {Financial Cryptography}, - year = {2004}, - editor = {Ari Juels}, - publisher = {Springer-Verlag, LNCS (forthcoming)}, -} - -@inproceedings{morphmix:fc04, - title = {Practical Anonymity for the Masses with MorphMix}, - author = {Marc Rennhard and Bernhard Plattner}, - booktitle = {Financial Cryptography}, - year = {2004}, - editor = {Ari Juels}, - publisher = {Springer-Verlag, LNCS (forthcoming)}, -} - -@inproceedings{eternity, - title = {The Eternity Service}, - author = {Ross Anderson}, - booktitle = {Pragocrypt '96}, - year = {1996}, -} - %note = {\url{http://www.cl.cam.ac.uk/users/rja14/eternity/eternity.html}}, - - -@inproceedings{minion-design, - title = {Mixminion: Design of a Type {III} Anonymous Remailer Protocol}, - author = {George Danezis and Roger Dingledine and Nick Mathewson}, - booktitle = {2003 IEEE Symposium on Security and Privacy}, - year = {2003}, - month = {May}, - publisher = {IEEE CS}, - pages = {2--15}, -} - %note = {\url{http://mixminion.net/minion-design.pdf}}, - -@inproceedings{ rao-pseudonymity, - author = "Josyula R. Rao and Pankaj Rohatgi", - title = "Can Pseudonymity Really Guarantee Privacy?", - booktitle = "Proceedings of the Ninth USENIX Security Symposium", - year = {2000}, - month = Aug, - publisher = {USENIX}, - pages = "85--96", -} - %note = {\url{http://www.usenix.org/publications/library/proceedings/sec2000/ -%full_papers/rao/rao.pdf}}, - -@InProceedings{pfitzmann90how, - author = "Birgit Pfitzmann and Andreas Pfitzmann", - title = "How to Break the Direct {RSA}-Implementation of {MIXes}", - booktitle = {Eurocrypt 89}, - publisher = {Springer-Verlag, LNCS 434}, - year = {1990}, - note = {\url{http://citeseer.nj.nec.com/pfitzmann90how.html}}, -} - -@Misc{tor-spec, - author = {Roger Dingledine and Nick Mathewson}, - title = {Tor Protocol Specifications}, - note = {\url{https://www.torproject.org/svn/trunk/doc/tor-spec.txt}}, -} - -@Misc{incentives-txt, - author = {Roger Dingledine and Nick Mathewson}, - title = {Tor Incentives Design Brainstorms}, - note = {\url{https://www.torproject.org/svn/trunk/doc/incentives.txt}}, -} - -@InProceedings{BM:mixencrypt, - author = {M{\"o}ller, Bodo}, - title = {Provably Secure Public-Key Encryption for Length-Preserving Chaumian Mixes}, - booktitle = {{CT-RSA} 2003}, - publisher = {Springer-Verlag, LNCS 2612}, - year = 2003, -} - -@InProceedings{back01, - author = {Adam Back and Ulf M\"oller and Anton Stiglic}, - title = {Traffic Analysis Attacks and Trade-Offs in Anonymity Providing Systems}, - booktitle = {Information Hiding (IH 2001)}, - pages = {245--257}, - year = 2001, - editor = {Ira S. Moskowitz}, - publisher = {Springer-Verlag, LNCS 2137}, -} - %note = {\newline \url{http://www.cypherspace.org/adam/pubs/traffic.pdf}}, - -@InProceedings{rackoff93cryptographic, - author = {Charles Rackoff and Daniel R. Simon}, - title = {Cryptographic Defense Against Traffic Analysis}, - booktitle = {{ACM} Symposium on Theory of Computing}, - pages = {672--681}, - year = {1993}, -} - %note = {\url{http://research.microsoft.com/crypto/dansimon/me.htm}}, - -@InProceedings{freehaven-berk, - author = {Roger Dingledine and Michael J. Freedman and David Molnar}, - title = {The Free Haven Project: Distributed Anonymous Storage Service}, - booktitle = {Designing Privacy Enhancing Technologies: Workshop - on Design Issue in Anonymity and Unobservability}, - year = 2000, - month = {July}, - editor = {H. Federrath}, - publisher = {Springer-Verlag, LNCS 2009}, -} - - @InProceedings{move-ndss05, - author = {Angelos Stavrou and Angelos D. Keromytis and Jason Nieh and Vishal Misra and Dan Rubenstein}, - title = {MOVE: An End-to-End Solution To Network Denial of Service}, - booktitle = {{ISOC Network and Distributed System Security Symposium (NDSS05)}}, - year = 2005, - month = {February}, - publisher = {Internet Society} -} - -%note = {\url{http://freehaven.net/papers.html}}, - - - - -@InProceedings{raymond00, - author = {J. F. Raymond}, - title = {{Traffic Analysis: Protocols, Attacks, Design Issues, - and Open Problems}}, - booktitle = {Designing Privacy Enhancing Technologies: Workshop - on Design Issue in Anonymity and Unobservability}, - year = 2000, - month = {July}, - pages = {10-29}, - editor = {H. Federrath}, - publisher = {Springer-Verlag, LNCS 2009}, -} - -@InProceedings{sybil, - author = "John Douceur", - title = {{The Sybil Attack}}, - booktitle = "Proceedings of the 1st International Peer To Peer Systems Workshop (IPTPS)", - month = Mar, - year = 2002, -} - - -@InCollection{price-privacy, - author = {Paul Syverson and Adam Shostack}, - editor = {L. Jean Camp and Stephen Lewis}, - title = {What Price Privacy? (and why identity theft is about neither identity nor theft)}, - booktitle = {Economics of Information Security}, - chapter = 10, - publisher = {Kluwer}, - year = 2004, - pages = {129--142} -} - - -@InProceedings{trickle02, - author = {Andrei Serjantov and Roger Dingledine and Paul Syverson}, - title = {From a Trickle to a Flood: Active Attacks on Several - Mix Types}, - booktitle = {Information Hiding (IH 2002)}, - year = {2002}, - editor = {Fabien Petitcolas}, - publisher = {Springer-Verlag, LNCS 2578}, -} - -@InProceedings{langos02, - author = {Oliver Berthold and Heinrich Langos}, - title = {Dummy Traffic Against Long Term Intersection Attacks}, - booktitle = {Privacy Enhancing Technologies (PET 2002)}, - year = {2002}, - editor = {Roger Dingledine and Paul Syverson}, - publisher = {Springer-Verlag, LNCS 2482} -} - - -@InProceedings{hintz-pet02, - author = {Andrew Hintz}, - title = {Fingerprinting Websites Using Traffic Analysis}, - booktitle = {Privacy Enhancing Technologies (PET 2002)}, - pages = {171--178}, - year = 2002, - editor = {Roger Dingledine and Paul Syverson}, - publisher = {Springer-Verlag, LNCS 2482} -} - -@InProceedings{or-discex00, - author = {Paul Syverson and Michael Reed and David Goldschlag}, - title = {{O}nion {R}outing Access Configurations}, - booktitle = {DARPA Information Survivability Conference and - Exposition (DISCEX 2000)}, - year = {2000}, - publisher = {IEEE CS Press}, - pages = {34--40}, - volume = {1}, -} - %note = {\newline \url{http://www.onion-router.net/Publications.html}}, - -@Inproceedings{or-pet00, - title = {{Towards an Analysis of Onion Routing Security}}, - author = {Paul Syverson and Gene Tsudik and Michael Reed and - Carl Landwehr}, - booktitle = {Designing Privacy Enhancing Technologies: Workshop - on Design Issue in Anonymity and Unobservability}, - year = 2000, - month = {July}, - pages = {96--114}, - editor = {H. Federrath}, - publisher = {Springer-Verlag, LNCS 2009}, -} - %note = {\url{http://www.onion-router.net/Publications/WDIAU-2000.ps.gz}}, - -@Inproceedings{freenet-pets00, - title = {Freenet: A Distributed Anonymous Information Storage - and Retrieval System}, - author = {Ian Clarke and Oskar Sandberg and Brandon Wiley and - Theodore W. Hong}, - booktitle = {Designing Privacy Enhancing Technologies: Workshop - on Design Issue in Anonymity and Unobservability}, - year = 2000, - month = {July}, - pages = {46--66}, - editor = {H. Federrath}, - publisher = {Springer-Verlag, LNCS 2009}, -} - %note = {\url{http://citeseer.nj.nec.com/clarke00freenet.html}}, - -@InProceedings{or-ih96, - author = {David M. Goldschlag and Michael G. Reed and Paul - F. Syverson}, - title = {Hiding Routing Information}, - booktitle = {Information Hiding, First International Workshop}, - pages = {137--150}, - year = 1996, - editor = {R. Anderson}, - month = {May}, - publisher = {Springer-Verlag, LNCS 1174}, -} - -@InProceedings{federrath-ih96, - author = {Hannes Federrath and Anja Jerichow and Andreas Pfitzmann}, - title = {{MIXes} in Mobile Communication Systems: Location - Management with Privacy}, - booktitle = {Information Hiding, First International Workshop}, - pages = {121--135}, - year = 1996, - editor = {R. Anderson}, - month = {May}, - publisher = {Springer-Verlag, LNCS 1174}, -} - - -@InProceedings{reed-protocols97, - author = {Michael G. Reed and Paul F. Syverson and David - M. Goldschlag}, - title = {Protocols Using Anonymous Connections: Mobile Applications}, - booktitle = {Security Protocols: 5th International Workshop}, - pages = {13--23}, - year = 1997, - editor = {Bruce Christianson and Bruno Crispo and Mark Lomas - and Michael Roe}, - month = {April}, - publisher = {Springer-Verlag, LNCS 1361} -} - - - -@Article{or-jsac98, - author = {Michael G. Reed and Paul F. Syverson and David - M. Goldschlag}, - title = {Anonymous Connections and Onion Routing}, - journal = {IEEE Journal on Selected Areas in Communications}, - year = 1998, - volume = 16, - number = 4, - pages = {482--494}, - month = {May}, -} - %note = {\url{http://www.onion-router.net/Publications/JSAC-1998.ps.gz}} - -@Misc{TLS, - author = {T. Dierks and C. Allen}, - title = {The {TLS} {P}rotocol --- {V}ersion 1.0}, - howpublished = {IETF RFC 2246}, - month = {January}, - year = {1999}, -} -%note = {\url{http://www.rfc-editor.org/rfc/rfc2246.txt}}, - -@Misc{SMTP, - author = {J. Postel}, - title = {Simple {M}ail {T}ransfer {P}rotocol}, - howpublished = {IETF RFC 2821 (also STD0010)}, - month = {April}, - year = {2001}, - note = {\url{http://www.rfc-editor.org/rfc/rfc2821.txt}}, -} - -@Misc{IMAP, - author = {M. Crispin}, - title = {Internet {M}essage {A}ccess {P}rotocol --- {V}ersion 4rev1}, - howpublished = {IETF RFC 2060}, - month = {December}, - year = {1996}, - note = {\url{http://www.rfc-editor.org/rfc/rfc2060.txt}}, -} - -@misc{pipenet, - title = {PipeNet 1.1}, - author = {Wei Dai}, - year = 1996, - month = {August}, - howpublished = {Usenet post}, - note = {\url{http://www.eskimo.com/~weidai/pipenet.txt} First mentioned - in a post to the cypherpunks list, Feb.\ 1995.}, -} - - -@Misc{POP3, - author = {J. Myers and M. Rose}, - title = {Post {O}ffice {P}rotocol --- {V}ersion 3}, - howpublished = {IETF RFC 1939 (also STD0053)}, - month = {May}, - year = {1996}, - note = {\url{http://www.rfc-editor.org/rfc/rfc1939.txt}}, -} - - -@InProceedings{shuffle, - author = {C. Andrew Neff}, - title = {A Verifiable Secret Shuffle and its Application to E-Voting}, - booktitle = {8th ACM Conference on Computer and Communications - Security (CCS-8)}, - pages = {116--125}, - year = 2001, - editor = {P. Samarati}, - month = {November}, - publisher = {ACM Press}, -} - %note = {\url{http://www.votehere.net/ada_compliant/ourtechnology/ - % technicaldocs/shuffle.pdf}}, - -@InProceedings{dolev91, - author = {Danny Dolev and Cynthia Dwork and Moni Naor}, - title = {Non-Malleable Cryptography}, - booktitle = {23rd ACM Symposium on the Theory of Computing (STOC)}, - pages = {542--552}, - year = 1991, - note = {Updated version at - \url{http://citeseer.nj.nec.com/dolev00nonmalleable.html}}, -} - -@TechReport{rsw96, - author = {Ronald L. Rivest and Adi Shamir and David A. Wagner}, - title = {Time-lock puzzles and timed-release Crypto}, - year = 1996, - type = {MIT LCS technical memo}, - number = {MIT/LCS/TR-684}, - month = {February}, - note = {\newline \url{http://citeseer.nj.nec.com/rivest96timelock.html}}, -} - -@InProceedings{web-mix, - author = {Oliver Berthold and Hannes Federrath and Stefan K\"opsell}, - title = {Web {MIX}es: A system for anonymous and unobservable - {I}nternet access}, - booktitle = {Designing Privacy Enhancing Technologies: Workshop - on Design Issue in Anonymity and Unobservability}, - editor = {H. Federrath}, - publisher = {Springer-Verlag, LNCS 2009}, - year = {2000}, -} -% pages = {115--129}, - -@InProceedings{disad-free-routes, - author = {Oliver Berthold and Andreas Pfitzmann and Ronny Standtke}, - title = {The disadvantages of free {MIX} routes and how to overcome - them}, - booktitle = {Designing Privacy Enhancing Technologies: Workshop - on Design Issue in Anonymity and Unobservability}, - pages = {30--45}, - year = 2000, - editor = {H. Federrath}, - publisher = {Springer-Verlag, LNCS 2009}, -} - %note = {\url{http://www.tik.ee.ethz.ch/~weiler/lehre/netsec/Unterlagen/anon/ - % disadvantages_berthold.pdf}}, - -@InProceedings{boneh00, - author = {Dan Boneh and Moni Naor}, - title = {Timed Commitments}, - booktitle = {Advances in Cryptology -- {CRYPTO} 2000}, - pages = {236--254}, - year = 2000, - publisher = {Springer-Verlag, LNCS 1880}, - note = {\newline \url{http://crypto.stanford.edu/~dabo/abstracts/timedcommit.html}}, -} - -@InProceedings{goldschlag98, - author = {David M. Goldschlag and Stuart G. Stubblebine}, - title = {Publicly Verifiable Lotteries: Applications of - Delaying Functions}, - booktitle = {Financial Cryptography}, - pages = {214--226}, - year = 1998, - publisher = {Springer-Verlag, LNCS 1465}, - note = {\newline \url{http://citeseer.nj.nec.com/goldschlag98publicly.html}}, -} - -@InProceedings{syverson98, - author = {Paul Syverson}, - title = {Weakly Secret Bit Commitment: Applications to - Lotteries and Fair Exchange}, - booktitle = {Computer Security Foundations Workshop (CSFW11)}, - pages = {2--13}, - year = 1998, - address = {Rockport Massachusetts}, - month = {June}, - publisher = {IEEE CS Press}, - note = {\newline \url{http://chacs.nrl.navy.mil/publications/CHACS/1998/}}, -} - -@Misc{shoup-iso, - author = {Victor Shoup}, - title = {A Proposal for an {ISO} {S}tandard for Public Key Encryption (version 2.1)}, - note = {Revised December 20, 2001. \url{http://www.shoup.net/papers/}}, -} - -@Misc{shoup-oaep, - author = {Victor Shoup}, - title = {{OAEP} Reconsidered}, - howpublished = {{IACR} e-print 2000/060}, - note = {\newline \url{http://eprint.iacr.org/2000/060/}}, -} - -@Misc{oaep-still-alive, - author = {E. Fujisaki and D. Pointcheval and T. Okamoto and J. Stern}, - title = {{RSA}-{OAEP} is Still Alive!}, - howpublished = {{IACR} e-print 2000/061}, - note = {\newline \url{http://eprint.iacr.org/2000/061/}}, -} - -@misc{echolot, - author = {Peter Palfrader}, - title = {Echolot: a pinger for anonymous remailers}, - note = {\url{http://www.palfrader.org/echolot/}}, -} - -@Misc{mixmaster-attacks, - author = {Lance Cottrell}, - title = {Mixmaster and Remailer Attacks}, - note = {\url{http://www.obscura.com/~loki/remailer/remailer-essay.html}}, -} - -@Misc{mixmaster-spec, - author = {Ulf M{\"o}ller and Lance Cottrell and Peter - Palfrader and Len Sassaman}, - title = {Mixmaster {P}rotocol --- {V}ersion 2}, - year = {2003}, - month = {July}, - howpublished = {Draft}, - note = {\url{http://www.abditum.com/mixmaster-spec.txt}}, -} - -@InProceedings{puzzles-tls, - author = "Drew Dean and Adam Stubblefield", - title = {{Using Client Puzzles to Protect TLS}}, - booktitle = "Proceedings of the 10th USENIX Security Symposium", - year = {2001}, - month = Aug, - publisher = {USENIX}, -} - -@InProceedings{breadpudding, - author = {Markus Jakobsson and Ari Juels}, - title = {Proofs of Work and Bread Pudding Protocols}, - booktitle = {Proceedings of the IFIP TC6 and TC11 Joint Working - Conference on Communications and Multimedia Security - (CMS '99)}, - year = 1999, - month = {September}, - publisher = {Kluwer} -} - -@Misc{hashcash, - author = {Adam Back}, - title = {Hash cash}, - note = {\newline \url{http://www.cypherspace.org/~adam/hashcash/}}, -} - -@InProceedings{oreilly-acc, - author = {Roger Dingledine and Michael J. Freedman and David Molnar}, - title = {Accountability}, - booktitle = {Peer-to-peer: Harnessing the Benefits of a Disruptive - Technology}, - year = {2001}, - publisher = {O'Reilly and Associates}, -} - - -@InProceedings{han, - author = {Yongfei Han}, - title = {Investigation of non-repudiation protocols}, - booktitle = {ACISP '96}, - year = 1996, - publisher = {Springer-Verlag}, -} - - -@Misc{socks5, - key = {socks5}, - title = {{SOCKS} {P}rotocol {V}ersion 5}, - howpublished= {IETF RFC 1928}, - month = {March}, - year = 1996, - note = {\url{http://www.ietf.org/rfc/rfc1928.txt}} -} - -@InProceedings{abe, - author = {Masayuki Abe}, - title = {Universally Verifiable {MIX} With Verification Work Independent of - The Number of {MIX} Servers}, - booktitle = {{EUROCRYPT} 1998}, - year = {1998}, - publisher = {Springer-Verlag, LNCS 1403}, -} - -@InProceedings{desmedt, - author = {Yvo Desmedt and Kaoru Kurosawa}, - title = {How To Break a Practical {MIX} and Design a New One}, - booktitle = {{EUROCRYPT} 2000}, - year = {2000}, - publisher = {Springer-Verlag, LNCS 1803}, - note = {\url{http://citeseer.nj.nec.com/447709.html}}, -} - -@InProceedings{mitkuro, - author = {M. Mitomo and K. Kurosawa}, - title = {{Attack for Flash MIX}}, - booktitle = {{ASIACRYPT} 2000}, - year = {2000}, - publisher = {Springer-Verlag, LNCS 1976}, - note = {\newline \url{http://citeseer.nj.nec.com/450148.html}}, -} - -@InProceedings{hybrid-mix, - author = {M. Ohkubo and M. Abe}, - title = {A {L}ength-{I}nvariant {H}ybrid {MIX}}, - booktitle = {Advances in Cryptology - {ASIACRYPT} 2000}, - year = {2000}, - publisher = {Springer-Verlag, LNCS 1976}, -} - -@InProceedings{PShuffle, - author = {Jun Furukawa and Kazue Sako}, - title = {An Efficient Scheme for Proving a Shuffle}, - editor = {Joe Kilian}, - booktitle = {CRYPTO 2001}, - year = {2001}, - publisher = {Springer-Verlag, LNCS 2139}, -} - - -@InProceedings{jakobsson-optimally, - author = "Markus Jakobsson and Ari Juels", - title = "An Optimally Robust Hybrid Mix Network (Extended Abstract)", - booktitle = {Principles of Distributed Computing - {PODC} '01}, - year = "2001", - publisher = {ACM Press}, - note = {\url{http://citeseer.nj.nec.com/492015.html}}, -} - -@InProceedings{kesdogan, - author = {D. Kesdogan and M. Egner and T. B\"uschkes}, - title = {Stop-and-Go {MIX}es Providing Probabilistic Anonymity in an Open - System}, - booktitle = {Information Hiding (IH 1998)}, - year = {1998}, - publisher = {Springer-Verlag, LNCS 1525}, -} - %note = {\url{http://www.cl.cam.ac.uk/~fapp2/ihw98/ihw98-sgmix.pdf}}, - -@InProceedings{socks4, - author = {David Koblas and Michelle R. Koblas}, - title = {{SOCKS}}, - booktitle = {UNIX Security III Symposium (1992 USENIX Security - Symposium)}, - pages = {77--83}, - year = 1992, - publisher = {USENIX}, -} - -@InProceedings{flash-mix, - author = {Markus Jakobsson}, - title = {Flash {M}ixing}, - booktitle = {Principles of Distributed Computing - {PODC} '99}, - year = {1999}, - publisher = {ACM Press}, - note = {\newline \url{http://citeseer.nj.nec.com/jakobsson99flash.html}}, -} - -@InProceedings{SK, - author = {Joe Kilian and Kazue Sako}, - title = {Receipt-Free {MIX}-Type Voting Scheme - A Practical Solution to - the Implementation of a Voting Booth}, - booktitle = {EUROCRYPT '95}, - year = {1995}, - publisher = {Springer-Verlag}, -} - -@InProceedings{OAEP, - author = {M. Bellare and P. Rogaway}, - year = {1994}, - booktitle = {EUROCRYPT '94}, - title = {Optimal {A}symmetric {E}ncryption {P}adding : How To Encrypt With - {RSA}}, - publisher = {Springer-Verlag}, - note = {\newline \url{http://www-cse.ucsd.edu/users/mihir/papers/oaep.html}}, -} -@inproceedings{babel, - title = {Mixing {E}-mail With {B}abel}, - author = {Ceki G\"ulc\"u and Gene Tsudik}, - booktitle = {{Network and Distributed Security Symposium (NDSS 96)}}, - year = 1996, - month = {February}, - pages = {2--16}, - publisher = {IEEE}, -} - %note = {\url{http://citeseer.nj.nec.com/2254.html}}, - -@Misc{rprocess, - author = {RProcess}, - title = {Selective Denial of Service Attacks}, - note = {\newline \url{http://www.eff.org/pub/Privacy/Anonymity/1999\_09\_DoS\_remail\_vuln.html}}, -} - -@Article{remailer-history, - author = {Sameer Parekh}, - title = {Prospects for Remailers}, - journal = {First Monday}, - volume = {1}, - number = {2}, - month = {August}, - year = {1996}, - note = {\url{http://www.firstmonday.dk/issues/issue2/remailers/}}, -} - -@Article{chaum-mix, - author = {David Chaum}, - title = {Untraceable electronic mail, return addresses, and digital pseudo-nyms}, - journal = {Communications of the ACM}, - year = {1981}, - volume = {4}, - number = {2}, - month = {February}, -} - %note = {\url{http://www.eskimo.com/~weidai/mix-net.txt}}, - -@InProceedings{nym-alias-net, - author = {David Mazi\`{e}res and M. Frans Kaashoek}, - title = {{The Design, Implementation and Operation of an Email - Pseudonym Server}}, - booktitle = {$5^{th}$ ACM Conference on Computer and - Communications Security (CCS'98)}, - year = 1998, - publisher = {ACM Press}, -} - %note = {\newline \url{http://www.scs.cs.nyu.edu/~dm/}}, - -@InProceedings{tangler, - author = {Marc Waldman and David Mazi\`{e}res}, - title = {Tangler: A Censorship-Resistant Publishing System - Based on Document Entanglements}, - booktitle = {$8^{th}$ ACM Conference on Computer and - Communications Security (CCS-8)}, - pages = {86--135}, - year = 2001, - publisher = {ACM Press}, -} - %note = {\url{http://www.scs.cs.nyu.edu/~dm/}} - -@misc{neochaum, - author = {Tim May}, - title = {Payment mixes for anonymity}, - howpublished = {E-mail archived at - \url{http://\newline www.inet-one.com/cypherpunks/dir.2000.02.28-2000.03.05/msg00334.html}}, -} - -@misc{helsingius, - author = {J. Helsingius}, - title = {{\tt anon.penet.fi} press release}, - note = {\newline \url{http://www.penet.fi/press-english.html}}, -} - -@InProceedings{garay97secure, - author = {J. Garay and R. Gennaro and C. Jutla and T. Rabin}, - title = {Secure distributed storage and retrieval}, - booktitle = {11th International Workshop, WDAG '97}, - pages = {275--289}, - year = {1997}, - publisher = {Springer-Verlag, LNCS 1320}, - note = {\newline \url{http://citeseer.nj.nec.com/garay97secure.html}}, -} - -@InProceedings{PIK, - author = {C. Park and K. Itoh and K. Kurosawa}, - title = {Efficient anonymous channel and all/nothing election scheme}, - booktitle = {Advances in Cryptology -- {EUROCRYPT} '93}, - pages = {248--259}, - publisher = {Springer-Verlag, LNCS 765}, -} - -@Misc{pgpfaq, - key = {PGP}, - title = {{PGP} {FAQ}}, - note = {\newline \url{http://www.faqs.org/faqs/pgp-faq/}}, -} - -@Article{riordan-schneier, - author = {James Riordan and Bruce Schneier}, - title = {A Certified E-mail Protocol with No Trusted Third Party}, - journal = {13th Annual Computer Security Applications Conference}, - month = {December}, - year = {1998}, - note = {\newline \url{http://www.counterpane.com/certified-email.html}}, -} - - -@Article{crowds-tissec, - author = {Michael K. Reiter and Aviel D. Rubin}, - title = {Crowds: Anonymity for Web Transactions}, - journal = {ACM TISSEC}, - year = 1998, - volume = 1, - number = 1, - pages = {66--92}, - month = {June}, -} - %note = {\url{http://citeseer.nj.nec.com/284739.html}} - -@Article{crowds-dimacs, - author = {Michael K. Reiter and Aviel D. Rubin}, - title = {Crowds: Anonymity for Web Transactions}, - journal = {{DIMACS} Technical Report (Revised)}, - volume = {97}, - number = {15}, - month = {August}, - year = {1997}, -} - -@Misc{advogato, - author = {Raph Levien}, - title = {Advogato's Trust Metric}, - note = {\newline \url{http://www.advogato.org/trust-metric.html}}, -} - -@InProceedings{publius, - author = {Marc Waldman and Aviel Rubin and Lorrie Cranor}, - title = {Publius: {A} robust, tamper-evident, censorship-resistant and - source-anonymous web publishing system}, - booktitle = {Proc. 9th USENIX Security Symposium}, - pages = {59--72}, - year = {2000}, - month = {August}, -} - %note = {\newline \url{http://citeseer.nj.nec.com/waldman00publius.html}}, - -@Misc{freedom-nyms, - author = {Russell Samuels}, - title = {Untraceable Nym Creation on the {F}reedom {N}etwork}, - year = {1999}, - month = {November}, - day = {21}, - note = {\newline \url{http://www.freedom.net/products/whitepapers/white11.html}}, -} - -@techreport{freedom2-arch, - title = {Freedom Systems 2.0 Architecture}, - author = {Philippe Boucher and Adam Shostack and Ian Goldberg}, - institution = {Zero Knowledge Systems, {Inc.}}, - year = {2000}, - month = {December}, - type = {White Paper}, - day = {18}, -} - -@techreport{freedom21-security, - title = {Freedom Systems 2.1 Security Issues and Analysis}, - author = {Adam Back and Ian Goldberg and Adam Shostack}, - institution = {Zero Knowledge Systems, {Inc.}}, - year = {2001}, - month = {May}, - type = {White Paper}, -} - -@inproceedings{cfs:sosp01, - title = {Wide-area cooperative storage with {CFS}}, - author = {Frank Dabek and M. Frans Kaashoek and David Karger and Robert Morris and Ion Stoica}, - booktitle = {18th {ACM} {S}ymposium on {O}perating {S}ystems {P}rinciples ({SOSP} '01)}, - year = {2001}, - month = {October}, - address = {Chateau Lake Louise, Banff, Canada}, -} - -@inproceedings{SS03, - title = {Passive Attack Analysis for Connection-Based Anonymity Systems}, - author = {Andrei Serjantov and Peter Sewell}, - booktitle = {Computer Security -- ESORICS 2003}, - publisher = {Springer-Verlag, LNCS 2808}, - year = {2003}, - month = {October}, -} - %note = {\url{http://www.cl.cam.ac.uk/users/aas23/papers_aas/conn_sys.ps}}, - -@Misc{pk-relations, - author = {M. Bellare and A. Desai and D. Pointcheval and P. Rogaway}, - title = {Relations Among Notions of Security for Public-Key Encryption - Schemes}, - howpublished = { - Extended abstract in {\em Advances in Cryptology - CRYPTO '98}, LNCS Vol. 1462. - Springer-Verlag, 1998. - Full version available from \newline \url{http://www-cse.ucsd.edu/users/mihir/}}, -} - - -@InProceedings{mix-acc, - author = {Roger Dingledine and Michael J. Freedman and David - Hopwood and David Molnar}, - title = {{A Reputation System to Increase MIX-net - Reliability}}, - booktitle = {Information Hiding (IH 2001)}, - pages = {126--141}, - year = 2001, - editor = {Ira S. Moskowitz}, - publisher = {Springer-Verlag, LNCS 2137}, -} - %note = {\url{http://www.freehaven.net/papers.html}}, - -@InProceedings{casc-rep, - author = {Roger Dingledine and Paul Syverson}, - title = {{Reliable MIX Cascade Networks through Reputation}}, - booktitle = {Financial Cryptography}, - year = 2002, - editor = {Matt Blaze}, - publisher = {Springer-Verlag, LNCS 2357}, -} - %note = {\newline \url{http://www.freehaven.net/papers.html}}, - -@InProceedings{zhou96certified, - author = {Zhou and Gollmann}, - title = {Certified Electronic Mail}, - booktitle = {{ESORICS: European Symposium on Research in Computer - Security}}, - publisher = {Springer-Verlag, LNCS 1146}, - year = {1996}, - note = {\newline \url{http://citeseer.nj.nec.com/zhou96certified.html}}, -} - -@Misc{realtime-mix, - author = {Anja Jerichow and Jan M\"uller and Andreas Pfitzmann and - Birgit Pfitzmann and Michael Waidner}, - title = {{Real-Time MIXes: A Bandwidth-Efficient Anonymity Protocol}}, - howpublished = {IEEE Journal on Selected Areas in Communications, 1998.}, - note = {\url{http://www.zurich.ibm.com/security/publications/1998.html}}, -} - -@InProceedings{danezis:pet2003, - author = {George Danezis}, - title = {Mix-networks with Restricted Routes}, - booktitle = {Privacy Enhancing Technologies (PET 2003)}, - year = 2003, - editor = {Roger Dingledine}, - publisher = {Springer-Verlag LNCS 2760} -} - -@InProceedings{gap-pets03, - author = {Krista Bennett and Christian Grothoff}, - title = {{GAP} -- practical anonymous networking}, - booktitle = {Privacy Enhancing Technologies (PET 2003)}, - year = 2003, - editor = {Roger Dingledine}, - publisher = {Springer-Verlag LNCS 2760} -} - -@Article{hordes-jcs, - author = {Brian Neal Levine and Clay Shields}, - title = {Hordes: A Multicast-Based Protocol for Anonymity}, - journal = {Journal of Computer Security}, - year = 2002, - volume = 10, - number = 3, - pages = {213--240} -} - -@TechReport{herbivore, - author = {Sharad Goel and Mark Robson and Milo Polte and Emin G\"{u}n Sirer}, - title = {Herbivore: A Scalable and Efficient Protocol for Anonymous Communication}, - institution = {Cornell University Computing and Information Science}, - year = 2003, - type = {Technical Report}, - number = {TR2003-1890}, - month = {February} -} - -@InProceedings{p5, - author = {Rob Sherwood and Bobby Bhattacharjee and Aravind Srinivasan}, - title = {$P^5$: A Protocol for Scalable Anonymous Communication}, - booktitle = {IEEE Symposium on Security and Privacy}, - pages = {58--70}, - year = 2002, - publisher = {IEEE CS} -} - -@phdthesis{ian-thesis, - title = {A Pseudonymous Communications Infrastructure for the Internet}, - author = {Ian Goldberg}, - school = {UC Berkeley}, - year = {2000}, - month = {Dec}, -} - -@Article{taz, - author = {Ian Goldberg and David Wagner}, - title = {TAZ Servers and the Rewebber Network: Enabling - Anonymous Publishing on the World Wide Web}, - journal = {First Monday}, - year = 1998, - volume = 3, - number = 4, - month = {August}, - note = {\url{http://www.firstmonday.dk/issues/issue3_4/goldberg/}} -} - -@Misc{tcp-over-tcp-is-bad, - key = {tcp-over-tcp-is-bad}, - title = {Why {TCP} Over {TCP} Is A Bad Idea}, - author = {Olaf Titz}, - note = {\url{http://sites.inka.de/sites/bigred/devel/tcp-tcp.html}} -} - -@inproceedings{wright02, - title = {An Analysis of the Degradation of Anonymous Protocols}, - author = {Matthew Wright and Micah Adler and Brian Neil Levine and Clay Shields}, - booktitle = {{Network and Distributed Security Symposium (NDSS 02)}}, - year = {2002}, - month = {February}, - publisher = {IEEE}, -} - -@inproceedings{wright03, - title = {Defending Anonymous Communication Against Passive Logging Attacks}, - author = {Matthew Wright and Micah Adler and Brian Neil Levine and Clay Shields}, - booktitle = {IEEE Symposium on Security and Privacy}, - pages= {28--41}, - year = {2003}, - month = {May}, - publisher = {IEEE CS}, -} - - -@InProceedings{attack-tor-oak05, - author = {Steven J. Murdoch and George Danezis}, - title = {Low-cost Traffic Analysis of {T}or}, - booktitle = {IEEE Symposium on Security and Privacy}, - year = 2005, - month = {May}, - publisher = {IEEE CS} -} - -@Misc{jap-backdoor, - author={{The AN.ON Project}}, - howpublished={Press release}, - year={2003}, - month={September}, - title={German Police proceeds against anonymity service}, - note={\url{http://www.datenschutzzentrum.de/material/themen/presse/anon-bka_e.htm}} -} - -@article{shsm03, - title = {Using Caching for Browsing Anonymity}, - author = {Anna Shubina and Sean Smith}, - journal = {ACM SIGEcom Exchanges}, - volume = {4}, - number = {2}, - year = {2003}, - month = {Sept}, - note = {\url{http://www.acm.org/sigs/sigecom/exchanges/volume_4_(03)/4.2-Shubina.pdf}}, -} - -@inproceedings{tor-design, - title = {Tor: The Second-Generation Onion Router}, - author = {Roger Dingledine and Nick Mathewson and Paul Syverson}, - booktitle = {Proceedings of the 13th USENIX Security Symposium}, - year = {2004}, - month = {August}, - note = {\url{https://www.torproject.org/tor-design.pdf}} -} - -@inproceedings{flow-correlation04, - title = {On Flow Correlation Attacks and Countermeasures in Mix Networks}, - author = {Ye Zhu and Xinwen Fu and Bryan Graham and Riccardo Bettati and Wei Zhao}, - booktitle = {Proceedings of Privacy Enhancing Technologies workshop (PET 2004)}, - year = {2004}, - month = {May}, - series = {LNCS}, - note = {\url{http://students.cs.tamu.edu/xinwenfu/paper/PET04.pdf}}, -} - -@InProceedings{danezis:pet2004, - author = "George Danezis", - title = "The Traffic Analysis of Continuous-Time Mixes", - booktitle= {Privacy Enhancing Technologies (PET 2004)}, - editor = {David Martin and Andrei Serjantov}, - month = {May}, - year = {2004}, - series = {LNCS}, - note = {\url{http://www.cl.cam.ac.uk/users/gd216/cmm2.pdf}}, -} - -@inproceedings{feamster:wpes2004, - title = {Location Diversity in Anonymity Networks}, - author = {Nick Feamster and Roger Dingledine}, - booktitle = {{Proceedings of the Workshop on Privacy in the Electronic Society (WPES 2004)}}, - year = {2004}, - month = {October}, - address = {Washington, DC, USA}, - note = {\url{http://freehaven.net/doc/routing-zones/routing-zones.ps}}, -} - -@inproceedings{koepsell:wpes2004, - title = {How to Achieve Blocking Resistance for Existing Systems Enabling Anonymous Web Surfing}, - author = {Stefan K\"opsell and Ulf Hilling}, - booktitle = {{Proceedings of the Workshop on Privacy in the Electronic Society (WPES 2004)}}, - year = {2004}, - month = {October}, - address = {Washington, DC, USA}, - note = {\url{http://freehaven.net/anonbib/papers/p103-koepsell.pdf}}, -} - -@inproceedings{sync-batching, - title = {Synchronous Batching: From Cascades to Free Routes}, - author = {Roger Dingledine and Vitaly Shmatikov and Paul Syverson}, - booktitle = {Proceedings of Privacy Enhancing Technologies workshop (PET 2004)}, - editor = {David Martin and Andrei Serjantov}, - year = {2004}, - month = {May}, - series = {LNCS}, - note = {\url{http://freehaven.net/doc/sync-batching/sync-batching.pdf}}, -} - -@InProceedings{e2e-traffic, - author = "Nick Mathewson and Roger Dingledine", - title = "Practical Traffic Analysis: Extending and Resisting Statistical Disclosure", - booktitle= {Privacy Enhancing Technologies (PET 2004)}, - editor = {David Martin and Andrei Serjantov}, - month = {May}, - year = {2004}, - series = {LNCS}, - note = {\url{http://freehaven.net/doc/e2e-traffic/e2e-traffic.pdf}}, -} - -@Misc{dtls, - author = {E. Rescorla and N. Modadugu}, - title = {{Datagram Transport Layer Security}}, - howpublished = {IETF Draft}, - month = {December}, - year = {2003}, - note = {\url{http://www.ietf.org/internet-drafts/draft-rescorla-dtls-02.txt}}, -} - -@InProceedings{usability-network-effect, - author={Roger Dingledine and Nick Mathewson}, - title={Anonymity Loves Company: Usability and the Network Effect}, - booktitle = {Designing Security Systems That People Can Use}, - year = {2005}, - publisher = {O'Reilly Media}, -} - -@inproceedings{usability:weis2006, - title = {Anonymity Loves Company: Usability and the Network Effect}, - author = {Roger Dingledine and Nick Mathewson}, - booktitle = {Proceedings of the Fifth Workshop on the Economics of Information Security - (WEIS 2006)}, - year = {2006}, - month = {June}, - address = {Cambridge, UK}, - bookurl = {http://weis2006.econinfosec.org/}, - note = {\url{http://freehaven.net/doc/wupss04/usability.pdf}}, -} - -@Misc{six-four, - key = {six-four}, - title = {{The Six/Four System}}, - note = {\url{http://sourceforge.net/projects/sixfour/}} -} - -@inproceedings{clayton:pet2006, - title = {Ignoring the Great Firewall of China}, - author = {Richard Clayton and Steven J. Murdoch and Robert N. M. Watson}, - booktitle = {Proceedings of the Sixth Workshop on Privacy Enhancing Technologies (PET 2006)}, - year = {2006}, - month = {June}, - address = {Cambridge, UK}, - publisher = {Springer}, - bookurl = {http://petworkshop.org/2006/}, - note = {\url{http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf}}, -} - -@Misc{zuckerman-threatmodels, - key = {zuckerman-threatmodels}, - title = {We've got to adjust some of our threat models}, - author = {Ethan Zuckerman}, - note = {\url{http://www.ethanzuckerman.com/blog/?p=1019}} -} - -@Misc{cgiproxy, - key = {cgiproxy}, - title = {{CGIProxy: HTTP/FTP Proxy in a CGI Script}}, - author = {James Marshall}, - note = {\url{http://www.jmarshall.com/tools/cgiproxy/}} -} - -@Misc{circumventor, - key = {circumventor}, - title = {{How to install the Circumventor program}}, - author = {Bennett Haselton}, - note = {\url{http://www.peacefire.org/circumventor/simple-circumventor-instructions.html}} -} - -@Misc{psiphon, - key = {psiphon}, - title = {Psiphon}, - author = {Ronald Deibert et al}, - note = {\url{http://psiphon.civisec.org/}} -} - -@InProceedings{tcpstego, author = {Steven J. Murdoch and Stephen Lewis}, - title = {Embedding Covert Channels into {TCP/IP}}, - booktitle = {Information Hiding: 7th International Workshop}, - pages = {247--261}, - year = {2005}, - editor = {Mauro Barni and Jordi Herrera-Joancomart\'{\i} and -Stefan Katzenbeisser and Fernando P\'{e}rez-Gonz\'{a}lez}, - volume = {3727}, - series = {LNCS}, - address = {Barcelona, Catalonia (Spain)}, - month = {June}, - publisher = {Springer-Verlag}, - url = {http://www.cl.cam.ac.uk/~sjm217/papers/ih05coverttcp.pdf} -} - -@phdthesis{blossom-thesis, - title = {Perspective Access Networks}, - author = {Geoffrey Goodell}, - school = {Harvard University}, - year = {2006}, - month = {July}, - note = {\url{http://afs.eecs.harvard.edu/~goodell/thesis.pdf}}, -} - -@inproceedings{tap:pet2006, - title = {On the Security of the Tor Authentication Protocol}, - author = {Ian Goldberg}, - booktitle = {Proceedings of the Sixth Workshop on Privacy Enhancing Technologies (PET 2006)}, - year = {2006}, - month = {June}, - address = {Cambridge, UK}, - publisher = {Springer}, - bookurl = {http://petworkshop.org/2006/}, - note = {\url{http://www.cypherpunks.ca/~iang/pubs/torsec.pdf}}, -} - -@inproceedings{rep-anon, - title = {{Reputation in P2P Anonymity Systems}}, - author = {Roger Dingledine and Nick Mathewson and Paul Syverson}, - booktitle = {Proceedings of Workshop on Economics of Peer-to-Peer Systems}, - year = {2003}, - month = {June}, - note = {\url{http://freehaven.net/doc/econp2p03/econp2p03.pdf}}, -} - -@misc{tor-challenges, - author = {Roger Dingledine and Nick Mathewson and Paul Syverson}, - title = {Challenges in deploying low-latency anonymity}, - year = {2005}, - note = {Manuscript} -} - -@InProceedings{chaum-blind, - author = {David Chaum}, - title = {Blind Signatures for Untraceable Payments}, - booktitle = {Advances in Cryptology: Proceedings of Crypto 82}, - pages = {199--203}, - year = 1983, - editor = {D. Chaum and R.L. Rivest and A.T. Sherman}, - publisher = {Plenum Press} -} - -@Article{netauth, - author = {Geoffrey Goodell and Paul Syverson}, - title = {The Right Place at the Right Time: Examining the use of network location in authentication and abuse prevention}, - journal = {Communications of the ACM}, - year = 2007, - volume = 50, - number = 5, - pages = {113--117}, - month = {May} -} - -@misc{ip-to-country, - key = {ip-to-country}, - title = {IP-to-country database}, - note = {\url{http://ip-to-country.webhosting.info/}}, -} - -@misc{mackinnon-personal, - author = {Rebecca MacKinnon}, - title = {Private communication}, - year = {2006}, -} - -@inproceedings{pet05-bissias, - title = {Privacy Vulnerabilities in Encrypted HTTP Streams}, - author = {George Dean Bissias and Marc Liberatore and Brian Neil Levine}, - booktitle = {Proceedings of Privacy Enhancing Technologies workshop (PET 2005)}, - year = {2005}, - month = {May}, - note = {\url{http://prisms.cs.umass.edu/brian/pubs/bissias.liberatore.pet.2005.pdf}}, -} - -@InProceedings{infranet, - author = {Nick Feamster and Magdalena Balazinska and Greg Harfst and Hari Balakrishnan and David Karger}, - title = {Infranet: Circumventing Web Censorship and Surveillance}, - booktitle = {Proceedings of the 11th USENIX Security Symposium}, - year = {2002}, - month = {August}, - note = {\url{http://nms.lcs.mit.edu/~feamster/papers/usenixsec2002.pdf}}, -} - -@techreport{ ptacek98insertion, - author = "Thomas H. Ptacek and Timothy N. Newsham", - title = "Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection", - institution = "Secure Networks, Inc.", - address = "Suite 330, 1201 5th Street S.W, Calgary, Alberta, Canada, T2R-0Y6", - year = "1998", - url = "citeseer.ist.psu.edu/ptacek98insertion.html", -} - -@inproceedings{active-wardens, - author = "Gina Fisk and Mike Fisk and Christos Papadopoulos and Joshua Neil", - title = "Eliminating Steganography in Internet Traffic with Active Wardens", - booktitle = {Information Hiding Workshop (IH 2002)}, - year = {2002}, - month = {October}, - editor = {Fabien Petitcolas}, - publisher = {Springer-Verlag, LNCS 2578}, -} - -@inproceedings{clog-the-queue, - title = {Don't Clog the Queue: Circuit Clogging and Mitigation in {P2P} anonymity schemes}, - author = {Jon McLachlan and Nicholas Hopper}, - booktitle = {Proceedings of Financial Cryptography (FC '08)}, - year = {2008}, - month = {January}, -} - -@inproceedings{snader08, - title = {A Tune-up for {Tor}: Improving Security and Performance in the {Tor} Network}, - author = {Robin Snader and Nikita Borisov}, - booktitle = {Proceedings of the Network and Distributed Security Symposium - {NDSS} '08}, - year = {2008}, - month = {February}, - publisher = {Internet Society}, -} - -@inproceedings{murdoch-pet2008, - title = {Metrics for Security and Performance in Low-Latency Anonymity Networks}, - author = {Steven J. Murdoch and Robert N. M. Watson}, - booktitle = {Proceedings of the Eighth International Symposium on Privacy Enhancing Technologies (PETS 2008)}, - year = {2008}, - month = {July}, - address = {Leuven, Belgium}, - pages = {115--132}, - editor = {Nikita Borisov and Ian Goldberg}, - publisher = {Springer}, - bookurl = {http://petsymposium.org/2008/}, -} - -@inproceedings{danezis-pet2008, - title = {Bridging and Fingerprinting: Epistemic Attacks on Route Selection}, - author = {George Danezis and Paul Syverson}, - booktitle = {Proceedings of the Eighth International Symposium on Privacy Enhancing Technologies (PETS 2008)}, - year = {2008}, - month = {July}, - address = {Leuven, Belgium}, - pages = {133--150}, - editor = {Nikita Borisov and Ian Goldberg}, - publisher = {Springer}, - bookurl = {http://petsymposium.org/2008/}, -} - -%%% Local Variables: -%%% mode: latex -%%% TeX-master: "tor-design" -%%% End: diff --git a/doc/design-paper/tor-design.html b/doc/design-paper/tor-design.html deleted file mode 100644 index 5fac644e6..000000000 --- a/doc/design-paper/tor-design.html +++ /dev/null @@ -1,2488 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" - "DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml"> -<head> -<meta name="GENERATOR" content="TtH 3.59" /> - <style type="text/css"> div.p { margin-top: 7pt;}</style> - <style type="text/css"><!-- - td div.comp { margin-top: -0.6ex; margin-bottom: -1ex;} - td div.comb { margin-top: -0.6ex; margin-bottom: -.6ex;} - td div.hrcomp { line-height: 0.9; margin-top: -0.8ex; margin-bottom: -1ex;} - td div.norm {line-height:normal;} - span.roman {font-family: serif; font-style: normal; font-weight: normal;} - span.overacc2 {position: relative; left: .8em; top: -1.2ex;} - span.overacc1 {position: relative; left: .6em; top: -1.2ex;} --></style> - - -<title> Tor: The Second-Generation Onion Router </title> -</head> -<body> - -<h1 align="center">Tor: The Second-Generation Onion Router </h1> -<div class="p"><!----></div> - -<h3 align="center"> -Roger Dingledine, The Free Haven Project, <tt>arma@freehaven.net</tt><br> -Nick Mathewson, The Free Haven Project, <tt>nickm@freehaven.net</tt><br> -Paul Syverson, Naval Research Lab, <tt>syverson@itd.nrl.navy.mil</tt> </h3> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<h2> Abstract</h2> -We present Tor, a circuit-based low-latency anonymous communication -service. This second-generation Onion Routing system addresses limitations -in the original design by adding perfect forward secrecy, congestion -control, directory servers, integrity checking, configurable exit policies, -and a practical design for location-hidden services via rendezvous -points. Tor works on the real-world -Internet, requires no special privileges or kernel modifications, requires -little synchronization or coordination between nodes, and provides a -reasonable tradeoff between anonymity, usability, and efficiency. -We briefly describe our experiences with an international network of -more than 30 nodes. We close with a list of open problems in anonymous communication. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc1"> -<a name="sec:intro"> -1</a> Overview</h2> -</a> - -<div class="p"><!----></div> -Onion Routing is a distributed overlay network designed to anonymize -TCP-based applications like web browsing, secure shell, -and instant messaging. Clients choose a path through the network and -build a <em>circuit</em>, in which each node (or "onion router" or "OR") -in the path knows its predecessor and successor, but no other nodes in -the circuit. Traffic flows down the circuit in fixed-size -<em>cells</em>, which are unwrapped by a symmetric key at each node -(like the layers of an onion) and relayed downstream. The -Onion Routing project published several design and analysis -papers [<a href="#or-ih96" name="CITEor-ih96">27</a>,<a href="#or-jsac98" name="CITEor-jsac98">41</a>,<a href="#or-discex00" name="CITEor-discex00">48</a>,<a href="#or-pet00" name="CITEor-pet00">49</a>]. While a wide area Onion -Routing network was deployed briefly, the only long-running -public implementation was a fragile -proof-of-concept that ran on a single machine. Even this simple deployment -processed connections from over sixty thousand distinct IP addresses from -all over the world at a rate of about fifty thousand per day. -But many critical design and deployment issues were never -resolved, and the design has not been updated in years. Here -we describe Tor, a protocol for asynchronous, loosely federated onion -routers that provides the following improvements over the old Onion -Routing design: - -<div class="p"><!----></div> -<b>Perfect forward secrecy:</b> In the original Onion Routing design, -a single hostile node could record traffic and -later compromise successive nodes in the circuit and force them -to decrypt it. Rather than using a single multiply encrypted data -structure (an <em>onion</em>) to lay each circuit, -Tor now uses an incremental or <em>telescoping</em> path-building design, -where the initiator negotiates session keys with each successive hop in -the circuit. Once these keys are deleted, subsequently compromised nodes -cannot decrypt old traffic. As a side benefit, onion replay detection -is no longer necessary, and the process of building circuits is more -reliable, since the initiator knows when a hop fails and can then try -extending to a new node. - -<div class="p"><!----></div> -<b>Separation of "protocol cleaning" from anonymity:</b> -Onion Routing originally required a separate "application -proxy" for each supported application protocol — most of which were -never written, so many applications were never supported. Tor uses the -standard and near-ubiquitous SOCKS [<a href="#socks4" name="CITEsocks4">32</a>] proxy interface, allowing -us to support most TCP-based programs without modification. Tor now -relies on the filtering features of privacy-enhancing -application-level proxies such as Privoxy [<a href="#privoxy" name="CITEprivoxy">39</a>], without trying -to duplicate those features itself. - -<div class="p"><!----></div> -<b>No mixing, padding, or traffic shaping (yet):</b> Onion -Routing originally called for batching and reordering cells as they arrived, -assumed padding between ORs, and in -later designs added padding between onion proxies (users) and -ORs [<a href="#or-ih96" name="CITEor-ih96">27</a>,<a href="#or-jsac98" name="CITEor-jsac98">41</a>]. Tradeoffs between padding protection -and cost were discussed, and <em>traffic shaping</em> algorithms were -theorized [<a href="#or-pet00" name="CITEor-pet00">49</a>] to provide good security without expensive -padding, but no concrete padding scheme was suggested. -Recent research [<a href="#econymics" name="CITEeconymics">1</a>] -and deployment experience [<a href="#freedom21-security" name="CITEfreedom21-security">4</a>] suggest that this -level of resource use is not practical or economical; and even full -link padding is still vulnerable [<a href="#defensive-dropping" name="CITEdefensive-dropping">33</a>]. Thus, -until we have a proven and convenient design for traffic shaping or -low-latency mixing that improves anonymity against a realistic -adversary, we leave these strategies out. - -<div class="p"><!----></div> -<b>Many TCP streams can share one circuit:</b> Onion Routing originally -built a separate circuit for each -application-level request, but this required -multiple public key operations for every request, and also presented -a threat to anonymity from building so many circuits; see -Section <a href="#sec:maintaining-anonymity">9</a>. Tor multiplexes multiple TCP -streams along each circuit to improve efficiency and anonymity. - -<div class="p"><!----></div> -<b>Leaky-pipe circuit topology:</b> Through in-band signaling -within the circuit, Tor initiators can direct traffic to nodes partway -down the circuit. This novel approach -allows traffic to exit the circuit from the middle — possibly -frustrating traffic shape and volume attacks based on observing the end -of the circuit. (It also allows for long-range padding if -future research shows this to be worthwhile.) - -<div class="p"><!----></div> -<b>Congestion control:</b> Earlier anonymity designs do not -address traffic bottlenecks. Unfortunately, typical approaches to -load balancing and flow control in overlay networks involve inter-node -control communication and global views of traffic. Tor's decentralized -congestion control uses end-to-end acks to maintain anonymity -while allowing nodes at the edges of the network to detect congestion -or flooding and send less data until the congestion subsides. - -<div class="p"><!----></div> -<b>Directory servers:</b> The earlier Onion Routing design -planned to flood state information through the network — an approach -that can be unreliable and complex. Tor takes a simplified view toward distributing this -information. Certain more trusted nodes act as <em>directory -servers</em>: they provide signed directories describing known -routers and their current state. Users periodically download them -via HTTP. - -<div class="p"><!----></div> -<b>Variable exit policies:</b> Tor provides a consistent mechanism -for each node to advertise a policy describing the hosts -and ports to which it will connect. These exit policies are critical -in a volunteer-based distributed infrastructure, because each operator -is comfortable with allowing different types of traffic to exit -from his node. - -<div class="p"><!----></div> -<b>End-to-end integrity checking:</b> The original Onion Routing -design did no integrity checking on data. Any node on the -circuit could change the contents of data cells as they passed by — for -example, to alter a connection request so it would connect -to a different webserver, or to `tag' encrypted traffic and look for -corresponding corrupted traffic at the network edges [<a href="#minion-design" name="CITEminion-design">15</a>]. -Tor hampers these attacks by verifying data integrity before it leaves -the network. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -<b>Rendezvous points and hidden services:</b> -Tor provides an integrated mechanism for responder anonymity via -location-protected servers. Previous Onion Routing designs included -long-lived "reply onions" that could be used to build circuits -to a hidden server, but these reply onions did not provide forward -security, and became useless if any node in the path went down -or rotated its keys. In Tor, clients negotiate <i>rendezvous points</i> -to connect with hidden servers; reply onions are no longer required. - -<div class="p"><!----></div> -Unlike Freedom [<a href="#freedom2-arch" name="CITEfreedom2-arch">8</a>], Tor does not require OS kernel -patches or network stack support. This prevents us from anonymizing -non-TCP protocols, but has greatly helped our portability and -deployability. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -We have implemented all of the above features, including rendezvous -points. Our source code is -available under a free license, and Tor -is not covered by the patent that affected distribution and use of -earlier versions of Onion Routing. -We have deployed a wide-area alpha network -to test the design, to get more experience with usability -and users, and to provide a research platform for experimentation. -As of this writing, the network stands at 32 nodes spread over two continents. - -<div class="p"><!----></div> -We review previous work in Section <a href="#sec:related-work">2</a>, describe -our goals and assumptions in Section <a href="#sec:assumptions">3</a>, -and then address the above list of improvements in -Sections <a href="#sec:design">4</a>, <a href="#sec:rendezvous">5</a>, and <a href="#sec:other-design">6</a>. -We summarize -in Section <a href="#sec:attacks">7</a> how our design stands up to -known attacks, and talk about our early deployment experiences in -Section <a href="#sec:in-the-wild">8</a>. We conclude with a list of open problems in -Section <a href="#sec:maintaining-anonymity">9</a> and future work for the Onion -Routing project in Section <a href="#sec:conclusion">10</a>. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc2"> -<a name="sec:related-work"> -2</a> Related work</h2> -</a> - -<div class="p"><!----></div> -Modern anonymity systems date to Chaum's <b>Mix-Net</b> -design [<a href="#chaum-mix" name="CITEchaum-mix">10</a>]. Chaum -proposed hiding the correspondence between sender and recipient by -wrapping messages in layers of public-key cryptography, and relaying them -through a path composed of "mixes." Each mix in turn -decrypts, delays, and re-orders messages before relaying them -onward. - -<div class="p"><!----></div> -Subsequent relay-based anonymity designs have diverged in two -main directions. Systems like <b>Babel</b> [<a href="#babel" name="CITEbabel">28</a>], -<b>Mixmaster</b> [<a href="#mixmaster-spec" name="CITEmixmaster-spec">36</a>], -and <b>Mixminion</b> [<a href="#minion-design" name="CITEminion-design">15</a>] have tried -to maximize anonymity at the cost of introducing comparatively large and -variable latencies. Because of this decision, these <em>high-latency</em> -networks resist strong global adversaries, -but introduce too much lag for interactive tasks like web browsing, -Internet chat, or SSH connections. - -<div class="p"><!----></div> -Tor belongs to the second category: <em>low-latency</em> designs that -try to anonymize interactive network traffic. These systems handle -a variety of bidirectional protocols. They also provide more convenient -mail delivery than the high-latency anonymous email -networks, because the remote mail server provides explicit and timely -delivery confirmation. But because these designs typically -involve many packets that must be delivered quickly, it is -difficult for them to prevent an attacker who can eavesdrop both ends of the -communication from correlating the timing and volume -of traffic entering the anonymity network with traffic leaving it [<a href="#SS03" name="CITESS03">45</a>]. -These -protocols are similarly vulnerable to an active adversary who introduces -timing patterns into traffic entering the network and looks -for correlated patterns among exiting traffic. -Although some work has been done to frustrate these attacks, most designs -protect primarily against traffic analysis rather than traffic -confirmation (see Section <a href="#subsec:threat-model">3.1</a>). - -<div class="p"><!----></div> -The simplest low-latency designs are single-hop proxies such as the -<b>Anonymizer</b> [<a href="#anonymizer" name="CITEanonymizer">3</a>]: a single trusted server strips the -data's origin before relaying it. These designs are easy to -analyze, but users must trust the anonymizing proxy. -Concentrating the traffic to this single point increases the anonymity set -(the people a given user is hiding among), but it is vulnerable if the -adversary can observe all traffic entering and leaving the proxy. - -<div class="p"><!----></div> -More complex are distributed-trust, circuit-based anonymizing systems. -In these designs, a user establishes one or more medium-term bidirectional -end-to-end circuits, and tunnels data in fixed-size cells. -Establishing circuits is computationally expensive and typically -requires public-key -cryptography, whereas relaying cells is comparatively inexpensive and -typically requires only symmetric encryption. -Because a circuit crosses several servers, and each server only knows -the adjacent servers in the circuit, no single server can link a -user to her communication partners. - -<div class="p"><!----></div> -The <b>Java Anon Proxy</b> (also known as JAP or Web MIXes) uses fixed shared -routes known as <em>cascades</em>. As with a single-hop proxy, this -approach aggregates users into larger anonymity sets, but again an -attacker only needs to observe both ends of the cascade to bridge all -the system's traffic. The Java Anon Proxy's design -calls for padding between end users and the head of the -cascade [<a href="#web-mix" name="CITEweb-mix">7</a>]. However, it is not demonstrated whether the current -implementation's padding policy improves anonymity. - -<div class="p"><!----></div> -<b>PipeNet</b> [<a href="#back01" name="CITEback01">5</a>,<a href="#pipenet" name="CITEpipenet">12</a>], another low-latency design proposed -around the same time as Onion Routing, gave -stronger anonymity but allowed a single user to shut -down the network by not sending. Systems like <b>ISDN -mixes</b> [<a href="#isdn-mixes" name="CITEisdn-mixes">38</a>] were designed for other environments with -different assumptions. - -<div class="p"><!----></div> -In P2P designs like <b>Tarzan</b> [<a href="#tarzan:ccs02" name="CITEtarzan:ccs02">24</a>] and -<b>MorphMix</b> [<a href="#morphmix:fc04" name="CITEmorphmix:fc04">43</a>], all participants both generate -traffic and relay traffic for others. These systems aim to conceal -whether a given peer originated a request -or just relayed it from another peer. While Tarzan and MorphMix use -layered encryption as above, <b>Crowds</b> [<a href="#crowds-tissec" name="CITEcrowds-tissec">42</a>] simply assumes -an adversary who cannot observe the initiator: it uses no public-key -encryption, so any node on a circuit can read users' traffic. - -<div class="p"><!----></div> -<b>Hordes</b> [<a href="#hordes-jcs" name="CITEhordes-jcs">34</a>] is based on Crowds but also uses multicast -responses to hide the initiator. <b>Herbivore</b> [<a href="#herbivore" name="CITEherbivore">25</a>] and -<b>P</b><sup><b>5</b></sup> [<a href="#p5" name="CITEp5">46</a>] go even further, requiring broadcast. -These systems are designed primarily for communication among peers, -although Herbivore users can make external connections by -requesting a peer to serve as a proxy. - -<div class="p"><!----></div> -Systems like <b>Freedom</b> and the original Onion Routing build circuits -all at once, using a layered "onion" of public-key encrypted messages, -each layer of which provides session keys and the address of the -next server in the circuit. Tor as described herein, Tarzan, MorphMix, -<b>Cebolla</b> [<a href="#cebolla" name="CITEcebolla">9</a>], and Rennhard's <b>Anonymity Network</b> [<a href="#anonnet" name="CITEanonnet">44</a>] -build circuits -in stages, extending them one hop at a time. -Section <a href="#subsubsec:constructing-a-circuit">4.2</a> describes how this -approach enables perfect forward secrecy. - -<div class="p"><!----></div> -Circuit-based designs must choose which protocol layer -to anonymize. They may intercept IP packets directly, and -relay them whole (stripping the source address) along the -circuit [<a href="#freedom2-arch" name="CITEfreedom2-arch">8</a>,<a href="#tarzan:ccs02" name="CITEtarzan:ccs02">24</a>]. Like -Tor, they may accept TCP streams and relay the data in those streams, -ignoring the breakdown of that data into TCP -segments [<a href="#morphmix:fc04" name="CITEmorphmix:fc04">43</a>,<a href="#anonnet" name="CITEanonnet">44</a>]. Finally, like Crowds, they may accept -application-level protocols such as HTTP and relay the application -requests themselves. -Making this protocol-layer decision requires a compromise between flexibility -and anonymity. For example, a system that understands HTTP -can strip -identifying information from requests, can take advantage of caching -to limit the number of requests that leave the network, and can batch -or encode requests to minimize the number of connections. -On the other hand, an IP-level anonymizer can handle nearly any protocol, -even ones unforeseen by its designers (though these systems require -kernel-level modifications to some operating systems, and so are more -complex and less portable). TCP-level anonymity networks like Tor present -a middle approach: they are application neutral (so long as the -application supports, or can be tunneled across, TCP), but by treating -application connections as data streams rather than raw TCP packets, -they avoid the inefficiencies of tunneling TCP over -TCP. - -<div class="p"><!----></div> -Distributed-trust anonymizing systems need to prevent attackers from -adding too many servers and thus compromising user paths. -Tor relies on a small set of well-known directory servers, run by -independent parties, to decide which nodes can -join. Tarzan and MorphMix allow unknown users to run servers, and use -a limited resource (like IP addresses) to prevent an attacker from -controlling too much of the network. Crowds suggests requiring -written, notarized requests from potential crowd members. - -<div class="p"><!----></div> -Anonymous communication is essential for censorship-resistant -systems like Eternity [<a href="#eternity" name="CITEeternity">2</a>], Free Haven [<a href="#freehaven-berk" name="CITEfreehaven-berk">19</a>], -Publius [<a href="#publius" name="CITEpublius">53</a>], and Tangler [<a href="#tangler" name="CITEtangler">52</a>]. Tor's rendezvous -points enable connections between mutually anonymous entities; they -are a building block for location-hidden servers, which are needed by -Eternity and Free Haven. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc3"> -<a name="sec:assumptions"> -3</a> Design goals and assumptions</h2> -</a> - -<div class="p"><!----></div> -<font size="+1"><b>Goals</b></font><br /> -Like other low-latency anonymity designs, Tor seeks to frustrate -attackers from linking communication partners, or from linking -multiple communications to or from a single user. Within this -main goal, however, several considerations have directed -Tor's evolution. - -<div class="p"><!----></div> -<b>Deployability:</b> The design must be deployed and used in the -real world. Thus it -must not be expensive to run (for example, by requiring more bandwidth -than volunteers are willing to provide); must not place a heavy -liability burden on operators (for example, by allowing attackers to -implicate onion routers in illegal activities); and must not be -difficult or expensive to implement (for example, by requiring kernel -patches, or separate proxies for every protocol). We also cannot -require non-anonymous parties (such as websites) -to run our software. (Our rendezvous point design does not meet -this goal for non-anonymous users talking to hidden servers, -however; see Section <a href="#sec:rendezvous">5</a>.) - -<div class="p"><!----></div> -<b>Usability:</b> A hard-to-use system has fewer users — and because -anonymity systems hide users among users, a system with fewer users -provides less anonymity. Usability is thus not only a convenience: -it is a security requirement [<a href="#econymics" name="CITEeconymics">1</a>,<a href="#back01" name="CITEback01">5</a>]. Tor should -therefore not -require modifying familiar applications; should not introduce prohibitive -delays; -and should require as few configuration decisions -as possible. Finally, Tor should be easily implementable on all common -platforms; we cannot require users to change their operating system -to be anonymous. (Tor currently runs on Win32, Linux, -Solaris, BSD-style Unix, MacOS X, and probably others.) - -<div class="p"><!----></div> -<b>Flexibility:</b> The protocol must be flexible and well-specified, -so Tor can serve as a test-bed for future research. -Many of the open problems in low-latency anonymity -networks, such as generating dummy traffic or preventing Sybil -attacks [<a href="#sybil" name="CITEsybil">22</a>], may be solvable independently from the issues -solved by -Tor. Hopefully future systems will not need to reinvent Tor's design. - -<div class="p"><!----></div> -<b>Simple design:</b> The protocol's design and security -parameters must be well-understood. Additional features impose implementation -and complexity costs; adding unproven techniques to the design threatens -deployability, readability, and ease of security analysis. Tor aims to -deploy a simple and stable system that integrates the best accepted -approaches to protecting anonymity.<br /> - -<div class="p"><!----></div> -<font size="+1"><b>Non-goals</b></font><a name="subsec:non-goals"> -</a><br /> -In favoring simple, deployable designs, we have explicitly deferred -several possible goals, either because they are solved elsewhere, or because -they are not yet solved. - -<div class="p"><!----></div> -<b>Not peer-to-peer:</b> Tarzan and MorphMix aim to scale to completely -decentralized peer-to-peer environments with thousands of short-lived -servers, many of which may be controlled by an adversary. This approach -is appealing, but still has many open -problems [<a href="#tarzan:ccs02" name="CITEtarzan:ccs02">24</a>,<a href="#morphmix:fc04" name="CITEmorphmix:fc04">43</a>]. - -<div class="p"><!----></div> -<b>Not secure against end-to-end attacks:</b> Tor does not claim -to completely solve end-to-end timing or intersection -attacks. Some approaches, such as having users run their own onion routers, -may help; -see Section <a href="#sec:maintaining-anonymity">9</a> for more discussion. - -<div class="p"><!----></div> -<b>No protocol normalization:</b> Tor does not provide <em>protocol -normalization</em> like Privoxy or the Anonymizer. If senders want anonymity from -responders while using complex and variable -protocols like HTTP, Tor must be layered with a filtering proxy such -as Privoxy to hide differences between clients, and expunge protocol -features that leak identity. -Note that by this separation Tor can also provide services that -are anonymous to the network yet authenticated to the responder, like -SSH. Similarly, Tor does not integrate -tunneling for non-stream-based protocols like UDP; this must be -provided by an external service if appropriate. - -<div class="p"><!----></div> -<b>Not steganographic:</b> Tor does not try to conceal who is connected -to the network. - -<div class="p"><!----></div> - <h3><a name="tth_sEc3.1"> -<a name="subsec:threat-model"> -3.1</a> Threat Model</h3> -</a> - -<div class="p"><!----></div> -A global passive adversary is the most commonly assumed threat when -analyzing theoretical anonymity designs. But like all practical -low-latency systems, Tor does not protect against such a strong -adversary. Instead, we assume an adversary who can observe some fraction -of network traffic; who can generate, modify, delete, or delay -traffic; who can operate onion routers of his own; and who can -compromise some fraction of the onion routers. - -<div class="p"><!----></div> -In low-latency anonymity systems that use layered encryption, the -adversary's typical goal is to observe both the initiator and the -responder. By observing both ends, passive attackers can confirm a -suspicion that Alice is -talking to Bob if the timing and volume patterns of the traffic on the -connection are distinct enough; active attackers can induce timing -signatures on the traffic to force distinct patterns. Rather -than focusing on these <em>traffic confirmation</em> attacks, -we aim to prevent <em>traffic -analysis</em> attacks, where the adversary uses traffic patterns to learn -which points in the network he should attack. - -<div class="p"><!----></div> -Our adversary might try to link an initiator Alice with her -communication partners, or try to build a profile of Alice's -behavior. He might mount passive attacks by observing the network edges -and correlating traffic entering and leaving the network — by -relationships in packet timing, volume, or externally visible -user-selected -options. The adversary can also mount active attacks by compromising -routers or keys; by replaying traffic; by selectively denying service -to trustworthy routers to move users to -compromised routers, or denying service to users to see if traffic -elsewhere in the -network stops; or by introducing patterns into traffic that can later be -detected. The adversary might subvert the directory servers to give users -differing views of network state. Additionally, he can try to decrease -the network's reliability by attacking nodes or by performing antisocial -activities from reliable nodes and trying to get them taken down — making -the network unreliable flushes users to other less anonymous -systems, where they may be easier to attack. We summarize -in Section <a href="#sec:attacks">7</a> how well the Tor design defends against -each of these attacks. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc4"> -<a name="sec:design"> -4</a> The Tor Design</h2> -</a> - -<div class="p"><!----></div> -The Tor network is an overlay network; each onion router (OR) -runs as a normal -user-level process without any special privileges. -Each onion router maintains a TLS [<a href="#TLS" name="CITETLS">17</a>] -connection to every other onion router. -Each user -runs local software called an onion proxy (OP) to fetch directories, -establish circuits across the network, -and handle connections from user applications. These onion proxies accept -TCP streams and multiplex them across the circuits. The onion -router on the other side -of the circuit connects to the requested destinations -and relays data. - -<div class="p"><!----></div> -Each onion router maintains a long-term identity key and a short-term -onion key. The identity -key is used to sign TLS certificates, to sign the OR's <em>router -descriptor</em> (a summary of its keys, address, bandwidth, exit policy, -and so on), and (by directory servers) to sign directories. The onion key is used to decrypt requests -from users to set up a circuit and negotiate ephemeral keys. -The TLS protocol also establishes a short-term link key when communicating -between ORs. Short-term keys are rotated periodically and -independently, to limit the impact of key compromise. - -<div class="p"><!----></div> -Section <a href="#subsec:cells">4.1</a> presents the fixed-size -<em>cells</em> that are the unit of communication in Tor. We describe -in Section <a href="#subsec:circuits">4.2</a> how circuits are -built, extended, truncated, and destroyed. Section <a href="#subsec:tcp">4.3</a> -describes how TCP streams are routed through the network. We address -integrity checking in Section <a href="#subsec:integrity-checking">4.4</a>, -and resource limiting in Section <a href="#subsec:rate-limit">4.5</a>. -Finally, -Section <a href="#subsec:congestion">4.6</a> talks about congestion control and -fairness issues. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.1"> -<a name="subsec:cells"> -4.1</a> Cells</h3> -</a> - -<div class="p"><!----></div> -Onion routers communicate with one another, and with users' OPs, via -TLS connections with ephemeral keys. Using TLS conceals the data on -the connection with perfect forward secrecy, and prevents an attacker -from modifying data on the wire or impersonating an OR. - -<div class="p"><!----></div> -Traffic passes along these connections in fixed-size cells. Each cell -is 512 bytes, and consists of a header and a payload. The header includes a circuit -identifier (circID) that specifies which circuit the cell refers to -(many circuits can be multiplexed over the single TLS connection), and -a command to describe what to do with the cell's payload. (Circuit -identifiers are connection-specific: each circuit has a different -circID on each OP/OR or OR/OR connection it traverses.) -Based on their command, cells are either <em>control</em> cells, which are -always interpreted by the node that receives them, or <em>relay</em> cells, -which carry end-to-end stream data. The control cell commands are: -<em>padding</em> (currently used for keepalive, but also usable for link -padding); <em>create</em> or <em>created</em> (used to set up a new circuit); -and <em>destroy</em> (to tear down a circuit). - -<div class="p"><!----></div> -Relay cells have an additional header (the relay header) at the front -of the payload, containing a streamID (stream identifier: many streams can -be multiplexed over a circuit); an end-to-end checksum for integrity -checking; the length of the relay payload; and a relay command. -The entire contents of the relay header and the relay cell payload -are encrypted or decrypted together as the relay cell moves along the -circuit, using the 128-bit AES cipher in counter mode to generate a -cipher stream. The relay commands are: <em>relay -data</em> (for data flowing down the stream), <em>relay begin</em> (to open a -stream), <em>relay end</em> (to close a stream cleanly), <em>relay -teardown</em> (to close a broken stream), <em>relay connected</em> -(to notify the OP that a relay begin has succeeded), <em>relay -extend</em> and <em>relay extended</em> (to extend the circuit by a hop, -and to acknowledge), <em>relay truncate</em> and <em>relay truncated</em> -(to tear down only part of the circuit, and to acknowledge), <em>relay -sendme</em> (used for congestion control), and <em>relay drop</em> (used to -implement long-range dummies). -We give a visual overview of cell structure plus the details of relay -cell structure, and then describe each of these cell types and commands -in more detail below. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> -<a name="tth_fIg1"> -</a> <center><img src="cell-struct.png" alt="cell-struct.png" /> -</center> -<div class="p"><!----></div> - <h3><a name="tth_sEc4.2"> -<a name="subsec:circuits"> -4.2</a> Circuits and streams</h3> -</a> - -<div class="p"><!----></div> -Onion Routing originally built one circuit for each -TCP stream. Because building a circuit can take several tenths of a -second (due to public-key cryptography and network latency), -this design imposed high costs on applications like web browsing that -open many TCP streams. - -<div class="p"><!----></div> -In Tor, each circuit can be shared by many TCP streams. To avoid -delays, users construct circuits preemptively. To limit linkability -among their streams, users' OPs build a new circuit -periodically if the previous ones have been used, -and expire old used circuits that no longer have any open streams. -OPs consider rotating to a new circuit once a minute: thus -even heavy users spend negligible time -building circuits, but a limited number of requests can be linked -to each other through a given exit node. Also, because circuits are built -in the background, OPs can recover from failed circuit creation -without harming user experience.<br /> - -<div class="p"><!----></div> - -<div class="p"><!----></div> -<a name="tth_fIg1"> -</a> <center><img src="interaction.png" alt="interaction.png" /> - -<center>Figure 1: Alice builds a two-hop circuit and begins fetching a web page.</center> -<a name="fig:interaction"> -</a> -</center> -<div class="p"><!----></div> -<a name="subsubsec:constructing-a-circuit"></a> -<font size="+1"><b>Constructing a circuit</b></font> -<br /> -A user's OP constructs circuits incrementally, negotiating a -symmetric key with each OR on the circuit, one hop at a time. To begin -creating a new circuit, the OP (call her Alice) sends a -<em>create</em> cell to the first node in her chosen path (call him Bob). -(She chooses a new -circID C<sub>AB</sub> not currently used on the connection from her to Bob.) -The <em>create</em> cell's -payload contains the first half of the Diffie-Hellman handshake -(g<sup>x</sup>), encrypted to the onion key of Bob. Bob -responds with a <em>created</em> cell containing g<sup>y</sup> -along with a hash of the negotiated key K=g<sup>xy</sup>. - -<div class="p"><!----></div> -Once the circuit has been established, Alice and Bob can send one -another relay cells encrypted with the negotiated -key.<a href="#tthFtNtAAB" name="tthFrefAAB"><sup>1</sup></a> More detail is given in -the next section. - -<div class="p"><!----></div> -To extend the circuit further, Alice sends a <em>relay extend</em> cell -to Bob, specifying the address of the next OR (call her Carol), and -an encrypted g<sup>x<sub>2</sub></sup> for her. Bob copies the half-handshake into a -<em>create</em> cell, and passes it to Carol to extend the circuit. -(Bob chooses a new circID C<sub>BC</sub> not currently used on the connection -between him and Carol. Alice never needs to know this circID; only Bob -associates C<sub>AB</sub> on his connection with Alice to C<sub>BC</sub> on -his connection with Carol.) -When Carol responds with a <em>created</em> cell, Bob wraps the payload -into a <em>relay extended</em> cell and passes it back to Alice. Now -the circuit is extended to Carol, and Alice and Carol share a common key -K<sub>2</sub> = g<sup>x<sub>2</sub> y<sub>2</sub></sup>. - -<div class="p"><!----></div> -To extend the circuit to a third node or beyond, Alice -proceeds as above, always telling the last node in the circuit to -extend one hop further. - -<div class="p"><!----></div> -This circuit-level handshake protocol achieves unilateral entity -authentication (Alice knows she's handshaking with the OR, but -the OR doesn't care who is opening the circuit — Alice uses no public key -and remains anonymous) and unilateral key authentication -(Alice and the OR agree on a key, and Alice knows only the OR learns -it). It also achieves forward -secrecy and key freshness. More formally, the protocol is as follows -(where E<sub>PK<sub>Bob</sub></sub>(·) is encryption with Bob's public key, -H is a secure hash function, and <font face="symbol">|</font -> is concatenation): - -<div class="p"><!----></div> -<a name="tth_tAb1"> -</a> -<table> -<tr><td align="right">Alice </td><td align="center">-> </td><td align="center">Bob </td><td>: E<sub>PK<sub>Bob</sub></sub>(g<sup>x</sup>) </td></tr> -<tr><td align="right">Bob </td><td align="center">-> </td><td align="center">Alice </td><td>: g<sup>y</sup>, H(K <font face="symbol">|</font -> "<span class="roman">handshake</span>") -</td></tr></table> - - -<div class="p"><!----></div> - In the second step, Bob proves that it was he who received g<sup>x</sup>, -and who chose y. We use PK encryption in the first step -(rather than, say, using the first two steps of STS, which has a -signature in the second step) because a single cell is too small to -hold both a public key and a signature. Preliminary analysis with the -NRL protocol analyzer [<a href="#meadows96" name="CITEmeadows96">35</a>] shows this protocol to be -secure (including perfect forward secrecy) under the -traditional Dolev-Yao model.<br /> - -<div class="p"><!----></div> -<font size="+1"><b>Relay cells</b></font><br /> -Once Alice has established the circuit (so she shares keys with each -OR on the circuit), she can send relay cells. -Upon receiving a relay -cell, an OR looks up the corresponding circuit, and decrypts the relay -header and payload with the session key for that circuit. -If the cell is headed away from Alice the OR then checks whether the -decrypted cell has a valid digest (as an optimization, the first -two bytes of the integrity check are zero, so in most cases we can avoid -computing the hash). -If valid, it accepts the relay cell and processes it as described -below. Otherwise, -the OR looks up the circID and OR for the -next step in the circuit, replaces the circID as appropriate, and -sends the decrypted relay cell to the next OR. (If the OR at the end -of the circuit receives an unrecognized relay cell, an error has -occurred, and the circuit is torn down.) - -<div class="p"><!----></div> -OPs treat incoming relay cells similarly: they iteratively unwrap the -relay header and payload with the session keys shared with each -OR on the circuit, from the closest to farthest. -If at any stage the digest is valid, the cell must have -originated at the OR whose encryption has just been removed. - -<div class="p"><!----></div> -To construct a relay cell addressed to a given OR, Alice assigns the -digest, and then iteratively -encrypts the cell payload (that is, the relay header and payload) with -the symmetric key of each hop up to that OR. Because the digest is -encrypted to a different value at each step, only at the targeted OR -will it have a meaningful value.<a href="#tthFtNtAAC" name="tthFrefAAC"><sup>2</sup></a> -This <em>leaky pipe</em> circuit topology -allows Alice's streams to exit at different ORs on a single circuit. -Alice may choose different exit points because of their exit policies, -or to keep the ORs from knowing that two streams -originate from the same person. - -<div class="p"><!----></div> -When an OR later replies to Alice with a relay cell, it -encrypts the cell's relay header and payload with the single key it -shares with Alice, and sends the cell back toward Alice along the -circuit. Subsequent ORs add further layers of encryption as they -relay the cell back to Alice. - -<div class="p"><!----></div> -To tear down a circuit, Alice sends a <em>destroy</em> control -cell. Each OR in the circuit receives the <em>destroy</em> cell, closes -all streams on that circuit, and passes a new <em>destroy</em> cell -forward. But just as circuits are built incrementally, they can also -be torn down incrementally: Alice can send a <em>relay -truncate</em> cell to a single OR on a circuit. That OR then sends a -<em>destroy</em> cell forward, and acknowledges with a -<em>relay truncated</em> cell. Alice can then extend the circuit to -different nodes, without signaling to the intermediate nodes (or -a limited observer) that she has changed her circuit. -Similarly, if a node on the circuit goes down, the adjacent -node can send a <em>relay truncated</em> cell back to Alice. Thus the -"break a node and see which circuits go down" -attack [<a href="#freedom21-security" name="CITEfreedom21-security">4</a>] is weakened. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.3"> -<a name="subsec:tcp"> -4.3</a> Opening and closing streams</h3> -</a> - -<div class="p"><!----></div> -When Alice's application wants a TCP connection to a given -address and port, it asks the OP (via SOCKS) to make the -connection. The OP chooses the newest open circuit (or creates one if -needed), and chooses a suitable OR on that circuit to be the -exit node (usually the last node, but maybe others due to exit policy -conflicts; see Section <a href="#subsec:exitpolicies">6.2</a>.) The OP then opens -the stream by sending a <em>relay begin</em> cell to the exit node, -using a new random streamID. Once the -exit node connects to the remote host, it responds -with a <em>relay connected</em> cell. Upon receipt, the OP sends a -SOCKS reply to notify the application of its success. The OP -now accepts data from the application's TCP stream, packaging it into -<em>relay data</em> cells and sending those cells along the circuit to -the chosen OR. - -<div class="p"><!----></div> -There's a catch to using SOCKS, however — some applications pass the -alphanumeric hostname to the Tor client, while others resolve it into -an IP address first and then pass the IP address to the Tor client. If -the application does DNS resolution first, Alice thereby reveals her -destination to the remote DNS server, rather than sending the hostname -through the Tor network to be resolved at the far end. Common applications -like Mozilla and SSH have this flaw. - -<div class="p"><!----></div> -With Mozilla, the flaw is easy to address: the filtering HTTP -proxy called Privoxy gives a hostname to the Tor client, so Alice's -computer never does DNS resolution. -But a portable general solution, such as is needed for -SSH, is -an open problem. Modifying or replacing the local nameserver -can be invasive, brittle, and unportable. Forcing the resolver -library to prefer TCP rather than UDP is hard, and also has -portability problems. Dynamically intercepting system calls to the -resolver library seems a promising direction. We could also provide -a tool similar to <em>dig</em> to perform a private lookup through the -Tor network. Currently, we encourage the use of privacy-aware proxies -like Privoxy wherever possible. - -<div class="p"><!----></div> -Closing a Tor stream is analogous to closing a TCP stream: it uses a -two-step handshake for normal operation, or a one-step handshake for -errors. If the stream closes abnormally, the adjacent node simply sends a -<em>relay teardown</em> cell. If the stream closes normally, the node sends -a <em>relay end</em> cell down the circuit, and the other side responds with -its own <em>relay end</em> cell. Because -all relay cells use layered encryption, only the destination OR knows -that a given relay cell is a request to close a stream. This two-step -handshake allows Tor to support TCP-based applications that use half-closed -connections. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.4"> -<a name="subsec:integrity-checking"> -4.4</a> Integrity checking on streams</h3> -</a> - -<div class="p"><!----></div> -Because the old Onion Routing design used a stream cipher without integrity -checking, traffic was -vulnerable to a malleability attack: though the attacker could not -decrypt cells, any changes to encrypted data -would create corresponding changes to the data leaving the network. -This weakness allowed an adversary who could guess the encrypted content -to change a padding cell to a destroy -cell; change the destination address in a <em>relay begin</em> cell to the -adversary's webserver; or change an FTP command from -<tt>dir</tt> to <tt>rm *</tt>. (Even an external -adversary could do this, because the link encryption similarly used a -stream cipher.) - -<div class="p"><!----></div> -Because Tor uses TLS on its links, external adversaries cannot modify -data. Addressing the insider malleability attack, however, is -more complex. - -<div class="p"><!----></div> -We could do integrity checking of the relay cells at each hop, either -by including hashes or by using an authenticating cipher mode like -EAX [<a href="#eax" name="CITEeax">6</a>], but there are some problems. First, these approaches -impose a message-expansion overhead at each hop, and so we would have to -either leak the path length or waste bytes by padding to a maximum -path length. Second, these solutions can only verify traffic coming -from Alice: ORs would not be able to produce suitable hashes for -the intermediate hops, since the ORs on a circuit do not know the -other ORs' session keys. Third, we have already accepted that our design -is vulnerable to end-to-end timing attacks; so tagging attacks performed -within the circuit provide no additional information to the attacker. - -<div class="p"><!----></div> -Thus, we check integrity only at the edges of each stream. (Remember that -in our leaky-pipe circuit topology, a stream's edge could be any hop -in the circuit.) When Alice -negotiates a key with a new hop, they each initialize a SHA-1 -digest with a derivative of that key, -thus beginning with randomness that only the two of them know. -Then they each incrementally add to the SHA-1 digest the contents of -all relay cells they create, and include with each relay cell the -first four bytes of the current digest. Each also keeps a SHA-1 -digest of data received, to verify that the received hashes are correct. - -<div class="p"><!----></div> -To be sure of removing or modifying a cell, the attacker must be able -to deduce the current digest state (which depends on all -traffic between Alice and Bob, starting with their negotiated key). -Attacks on SHA-1 where the adversary can incrementally add to a hash -to produce a new valid hash don't work, because all hashes are -end-to-end encrypted across the circuit. The computational overhead -of computing the digests is minimal compared to doing the AES -encryption performed at each hop of the circuit. We use only four -bytes per cell to minimize overhead; the chance that an adversary will -correctly guess a valid hash -is -acceptably low, given that the OP or OR tear down the circuit if they -receive a bad hash. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.5"> -<a name="subsec:rate-limit"> -4.5</a> Rate limiting and fairness</h3> -</a> - -<div class="p"><!----></div> -Volunteers are more willing to run services that can limit -their bandwidth usage. To accommodate them, Tor servers use a -token bucket approach [<a href="#tannenbaum96" name="CITEtannenbaum96">50</a>] to -enforce a long-term average rate of incoming bytes, while still -permitting short-term bursts above the allowed bandwidth. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -Because the Tor protocol outputs about the same number of bytes as it -takes in, it is sufficient in practice to limit only incoming bytes. -With TCP streams, however, the correspondence is not one-to-one: -relaying a single incoming byte can require an entire 512-byte cell. -(We can't just wait for more bytes, because the local application may -be awaiting a reply.) Therefore, we treat this case as if the entire -cell size had been read, regardless of the cell's fullness. - -<div class="p"><!----></div> -Further, inspired by Rennhard et al's design in [<a href="#anonnet" name="CITEanonnet">44</a>], a -circuit's edges can heuristically distinguish interactive streams from bulk -streams by comparing the frequency with which they supply cells. We can -provide good latency for interactive streams by giving them preferential -service, while still giving good overall throughput to the bulk -streams. Such preferential treatment presents a possible end-to-end -attack, but an adversary observing both -ends of the stream can already learn this information through timing -attacks. - -<div class="p"><!----></div> - <h3><a name="tth_sEc4.6"> -<a name="subsec:congestion"> -4.6</a> Congestion control</h3> -</a> - -<div class="p"><!----></div> -Even with bandwidth rate limiting, we still need to worry about -congestion, either accidental or intentional. If enough users choose the -same OR-to-OR connection for their circuits, that connection can become -saturated. For example, an attacker could send a large file -through the Tor network to a webserver he runs, and then -refuse to read any of the bytes at the webserver end of the -circuit. Without some congestion control mechanism, these bottlenecks -can propagate back through the entire network. We don't need to -reimplement full TCP windows (with sequence numbers, -the ability to drop cells when we're full and retransmit later, and so -on), -because TCP already guarantees in-order delivery of each -cell. -We describe our response below. - -<div class="p"><!----></div> -<b>Circuit-level throttling:</b> -To control a circuit's bandwidth usage, each OR keeps track of two -windows. The <em>packaging window</em> tracks how many relay data cells the OR is -allowed to package (from incoming TCP streams) for transmission back to the OP, -and the <em>delivery window</em> tracks how many relay data cells it is willing -to deliver to TCP streams outside the network. Each window is initialized -(say, to 1000 data cells). When a data cell is packaged or delivered, -the appropriate window is decremented. When an OR has received enough -data cells (currently 100), it sends a <em>relay sendme</em> cell towards the OP, -with streamID zero. When an OR receives a <em>relay sendme</em> cell with -streamID zero, it increments its packaging window. Either of these cells -increments the corresponding window by 100. If the packaging window -reaches 0, the OR stops reading from TCP connections for all streams -on the corresponding circuit, and sends no more relay data cells until -receiving a <em>relay sendme</em> cell. - -<div class="p"><!----></div> -The OP behaves identically, except that it must track a packaging window -and a delivery window for every OR in the circuit. If a packaging window -reaches 0, it stops reading from streams destined for that OR. - -<div class="p"><!----></div> -<b>Stream-level throttling</b>: -The stream-level congestion control mechanism is similar to the -circuit-level mechanism. ORs and OPs use <em>relay sendme</em> cells -to implement end-to-end flow control for individual streams across -circuits. Each stream begins with a packaging window (currently 500 cells), -and increments the window by a fixed value (50) upon receiving a <em>relay -sendme</em> cell. Rather than always returning a <em>relay sendme</em> cell as soon -as enough cells have arrived, the stream-level congestion control also -has to check whether data has been successfully flushed onto the TCP -stream; it sends the <em>relay sendme</em> cell only when the number of bytes pending -to be flushed is under some threshold (currently 10 cells' worth). - -<div class="p"><!----></div> - -<div class="p"><!----></div> -These arbitrarily chosen parameters seem to give tolerable throughput -and delay; see Section <a href="#sec:in-the-wild">8</a>. - -<div class="p"><!----></div> - <h2><a name="tth_sEc5"> -<a name="sec:rendezvous"> -5</a> Rendezvous Points and hidden services</h2> -</a> - -<div class="p"><!----></div> -Rendezvous points are a building block for <em>location-hidden -services</em> (also known as <em>responder anonymity</em>) in the Tor -network. Location-hidden services allow Bob to offer a TCP -service, such as a webserver, without revealing his IP address. -This type of anonymity protects against distributed DoS attacks: -attackers are forced to attack the onion routing network -because they do not know Bob's IP address. - -<div class="p"><!----></div> -Our design for location-hidden servers has the following goals. -<b>Access-control:</b> Bob needs a way to filter incoming requests, -so an attacker cannot flood Bob simply by making many connections to him. -<b>Robustness:</b> Bob should be able to maintain a long-term pseudonymous -identity even in the presence of router failure. Bob's service must -not be tied to a single OR, and Bob must be able to migrate his service -across ORs. <b>Smear-resistance:</b> -A social attacker -should not be able to "frame" a rendezvous router by -offering an illegal or disreputable location-hidden service and -making observers believe the router created that service. -<b>Application-transparency:</b> Although we require users -to run special software to access location-hidden servers, we must not -require them to modify their applications. - -<div class="p"><!----></div> -We provide location-hiding for Bob by allowing him to advertise -several onion routers (his <em>introduction points</em>) as contact -points. He may do this on any robust efficient -key-value lookup system with authenticated updates, such as a -distributed hash table (DHT) like CFS [<a href="#cfs:sosp01" name="CITEcfs:sosp01">11</a>].<a href="#tthFtNtAAD" name="tthFrefAAD"><sup>3</sup></a> Alice, the client, chooses an OR as her -<em>rendezvous point</em>. She connects to one of Bob's introduction -points, informs him of her rendezvous point, and then waits for him -to connect to the rendezvous point. This extra level of indirection -helps Bob's introduction points avoid problems associated with serving -unpopular files directly (for example, if Bob serves -material that the introduction point's community finds objectionable, -or if Bob's service tends to get attacked by network vandals). -The extra level of indirection also allows Bob to respond to some requests -and ignore others. - -<div class="p"><!----></div> - <h3><a name="tth_sEc5.1"> -5.1</a> Rendezvous points in Tor</h3> - -<div class="p"><!----></div> -The following steps are -performed on behalf of Alice and Bob by their local OPs; -application integration is described more fully below. - -<div class="p"><!----></div> - -<dl compact="compact"> - - <dt><b></b></dt> - <dd><li>Bob generates a long-term public key pair to identify his service.</dd> - <dt><b></b></dt> - <dd><li>Bob chooses some introduction points, and advertises them on - the lookup service, signing the advertisement with his public key. He - can add more later.</dd> - <dt><b></b></dt> - <dd><li>Bob builds a circuit to each of his introduction points, and tells - them to wait for requests.</dd> - <dt><b></b></dt> - <dd><li>Alice learns about Bob's service out of band (perhaps Bob told her, - or she found it on a website). She retrieves the details of Bob's - service from the lookup service. If Alice wants to access Bob's - service anonymously, she must connect to the lookup service via Tor.</dd> - <dt><b></b></dt> - <dd><li>Alice chooses an OR as the rendezvous point (RP) for her connection to - Bob's service. She builds a circuit to the RP, and gives it a - randomly chosen "rendezvous cookie" to recognize Bob.</dd> - <dt><b></b></dt> - <dd><li>Alice opens an anonymous stream to one of Bob's introduction - points, and gives it a message (encrypted with Bob's public key) - telling it about herself, - her RP and rendezvous cookie, and the - start of a DH - handshake. The introduction point sends the message to Bob.</dd> - <dt><b></b></dt> - <dd><li>If Bob wants to talk to Alice, he builds a circuit to Alice's - RP and sends the rendezvous cookie, the second half of the DH - handshake, and a hash of the session - key they now share. By the same argument as in - Section <a href="#subsubsec:constructing-a-circuit">4.2</a>, Alice knows she - shares the key only with Bob.</dd> - <dt><b></b></dt> - <dd><li>The RP connects Alice's circuit to Bob's. Note that RP can't - recognize Alice, Bob, or the data they transmit.</dd> - <dt><b></b></dt> - <dd><li>Alice sends a <em>relay begin</em> cell along the circuit. It - arrives at Bob's OP, which connects to Bob's - webserver.</dd> - <dt><b></b></dt> - <dd><li>An anonymous stream has been established, and Alice and Bob - communicate as normal. -</dd> -</dl> - -<div class="p"><!----></div> -When establishing an introduction point, Bob provides the onion router -with the public key identifying his service. Bob signs his -messages, so others cannot usurp his introduction point -in the future. He uses the same public key to establish the other -introduction points for his service, and periodically refreshes his -entry in the lookup service. - -<div class="p"><!----></div> -The message that Alice gives -the introduction point includes a hash of Bob's public key and an optional initial authorization token (the -introduction point can do prescreening, for example to block replays). Her -message to Bob may include an end-to-end authorization token so Bob -can choose whether to respond. -The authorization tokens can be used to provide selective access: -important users can get uninterrupted access. -During normal situations, Bob's service might simply be offered -directly from mirrors, while Bob gives out tokens to high-priority users. If -the mirrors are knocked down, -those users can switch to accessing Bob's service via -the Tor rendezvous system. - -<div class="p"><!----></div> -Bob's introduction points are themselves subject to DoS — he must -open many introduction points or risk such an attack. -He can provide selected users with a current list or future schedule of -unadvertised introduction points; -this is most practical -if there is a stable and large group of introduction points -available. Bob could also give secret public keys -for consulting the lookup service. All of these approaches -limit exposure even when -some selected users collude in the DoS. - -<div class="p"><!----></div> - <h3><a name="tth_sEc5.2"> -5.2</a> Integration with user applications</h3> - -<div class="p"><!----></div> -Bob configures his onion proxy to know the local IP address and port of his -service, a strategy for authorizing clients, and his public key. The onion -proxy anonymously publishes a signed statement of Bob's -public key, an expiration time, and -the current introduction points for his service onto the lookup service, -indexed -by the hash of his public key. Bob's webserver is unmodified, -and doesn't even know that it's hidden behind the Tor network. - -<div class="p"><!----></div> -Alice's applications also work unchanged — her client interface -remains a SOCKS proxy. We encode all of the necessary information -into the fully qualified domain name (FQDN) Alice uses when establishing her -connection. Location-hidden services use a virtual top level domain -called <tt>.onion</tt>: thus hostnames take the form <tt>x.y.onion</tt> where -<tt>x</tt> is the authorization cookie and <tt>y</tt> encodes the hash of -the public key. Alice's onion proxy -examines addresses; if they're destined for a hidden server, it decodes -the key and starts the rendezvous as described above. - -<div class="p"><!----></div> - <h3><a name="tth_sEc5.3"> -5.3</a> Previous rendezvous work</h3> - -<div class="p"><!----></div> -Rendezvous points in low-latency anonymity systems were first -described for use in ISDN telephony [<a href="#jerichow-jsac98" name="CITEjerichow-jsac98">30</a>,<a href="#isdn-mixes" name="CITEisdn-mixes">38</a>]. -Later low-latency designs used rendezvous points for hiding location -of mobile phones and low-power location -trackers [<a href="#federrath-ih96" name="CITEfederrath-ih96">23</a>,<a href="#reed-protocols97" name="CITEreed-protocols97">40</a>]. Rendezvous for -anonymizing low-latency -Internet connections was suggested in early Onion Routing -work [<a href="#or-ih96" name="CITEor-ih96">27</a>], but the first published design was by Ian -Goldberg [<a href="#ian-thesis" name="CITEian-thesis">26</a>]. His design differs from -ours in three ways. First, Goldberg suggests that Alice should manually -hunt down a current location of the service via Gnutella; our approach -makes lookup transparent to the user, as well as faster and more robust. -Second, in Tor the client and server negotiate session keys -with Diffie-Hellman, so plaintext is not exposed even at the rendezvous -point. Third, -our design minimizes the exposure from running the -service, to encourage volunteers to offer introduction and rendezvous -services. Tor's introduction points do not output any bytes to the -clients; the rendezvous points don't know the client or the server, -and can't read the data being transmitted. The indirection scheme is -also designed to include authentication/authorization — if Alice doesn't -include the right cookie with her request for service, Bob need not even -acknowledge his existence. - -<div class="p"><!----></div> - <h2><a name="tth_sEc6"> -<a name="sec:other-design"> -6</a> Other design decisions</h2> -</a> - -<div class="p"><!----></div> - <h3><a name="tth_sEc6.1"> -<a name="subsec:dos"> -6.1</a> Denial of service</h3> -</a> - -<div class="p"><!----></div> -Providing Tor as a public service creates many opportunities for -denial-of-service attacks against the network. While -flow control and rate limiting (discussed in -Section <a href="#subsec:congestion">4.6</a>) prevent users from consuming more -bandwidth than routers are willing to provide, opportunities remain for -users to -consume more network resources than their fair share, or to render the -network unusable for others. - -<div class="p"><!----></div> -First of all, there are several CPU-consuming denial-of-service -attacks wherein an attacker can force an OR to perform expensive -cryptographic operations. For example, an attacker can -fake the start of a TLS handshake, forcing the OR to carry out its -(comparatively expensive) half of the handshake at no real computational -cost to the attacker. - -<div class="p"><!----></div> -We have not yet implemented any defenses for these attacks, but several -approaches are possible. First, ORs can -require clients to solve a puzzle [<a href="#puzzles-tls" name="CITEpuzzles-tls">16</a>] while beginning new -TLS handshakes or accepting <em>create</em> cells. So long as these -tokens are easy to verify and computationally expensive to produce, this -approach limits the attack multiplier. Additionally, ORs can limit -the rate at which they accept <em>create</em> cells and TLS connections, -so that -the computational work of processing them does not drown out the -symmetric cryptography operations that keep cells -flowing. This rate limiting could, however, allow an attacker -to slow down other users when they build new circuits. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -Adversaries can also attack the Tor network's hosts and network -links. Disrupting a single circuit or link breaks all streams passing -along that part of the circuit. Users similarly lose service -when a router crashes or its operator restarts it. The current -Tor design treats such attacks as intermittent network failures, and -depends on users and applications to respond or recover as appropriate. A -future design could use an end-to-end TCP-like acknowledgment protocol, -so no streams are lost unless the entry or exit point is -disrupted. This solution would require more buffering at the network -edges, however, and the performance and anonymity implications from this -extra complexity still require investigation. - -<div class="p"><!----></div> - <h3><a name="tth_sEc6.2"> -<a name="subsec:exitpolicies"> -6.2</a> Exit policies and abuse</h3> -</a> - -<div class="p"><!----></div> - -<div class="p"><!----></div> -Exit abuse is a serious barrier to wide-scale Tor deployment. Anonymity -presents would-be vandals and abusers with an opportunity to hide -the origins of their activities. Attackers can harm the Tor network by -implicating exit servers for their abuse. Also, applications that commonly -use IP-based authentication (such as institutional mail or webservers) -can be fooled by the fact that anonymous connections appear to originate -at the exit OR. - -<div class="p"><!----></div> -We stress that Tor does not enable any new class of abuse. Spammers -and other attackers already have access to thousands of misconfigured -systems worldwide, and the Tor network is far from the easiest way -to launch attacks. -But because the -onion routers can be mistaken for the originators of the abuse, -and the volunteers who run them may not want to deal with the hassle of -explaining anonymity networks to irate administrators, we must block or limit -abuse through the Tor network. - -<div class="p"><!----></div> -To mitigate abuse issues, each onion router's <em>exit policy</em> -describes to which external addresses and ports the router will -connect. On one end of the spectrum are <em>open exit</em> -nodes that will connect anywhere. On the other end are <em>middleman</em> -nodes that only relay traffic to other Tor nodes, and <em>private exit</em> -nodes that only connect to a local host or network. A private -exit can allow a client to connect to a given host or -network more securely — an external adversary cannot eavesdrop traffic -between the private exit and the final destination, and so is less sure of -Alice's destination and activities. Most onion routers in the current -network function as -<em>restricted exits</em> that permit connections to the world at large, -but prevent access to certain abuse-prone addresses and services such -as SMTP. -The OR might also be able to authenticate clients to -prevent exit abuse without harming anonymity [<a href="#or-discex00" name="CITEor-discex00">48</a>]. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -Many administrators use port restrictions to support only a -limited set of services, such as HTTP, SSH, or AIM. -This is not a complete solution, of course, since abuse opportunities for these -protocols are still well known. - -<div class="p"><!----></div> -We have not yet encountered any abuse in the deployed network, but if -we do we should consider using proxies to clean traffic for certain -protocols as it leaves the network. For example, much abusive HTTP -behavior (such as exploiting buffer overflows or well-known script -vulnerabilities) can be detected in a straightforward manner. -Similarly, one could run automatic spam filtering software (such as -SpamAssassin) on email exiting the OR network. - -<div class="p"><!----></div> -ORs may also rewrite exiting traffic to append -headers or other information indicating that the traffic has passed -through an anonymity service. This approach is commonly used -by email-only anonymity systems. ORs can also -run on servers with hostnames like <tt>anonymous</tt> to further -alert abuse targets to the nature of the anonymous traffic. - -<div class="p"><!----></div> -A mixture of open and restricted exit nodes allows the most -flexibility for volunteers running servers. But while having many -middleman nodes provides a large and robust network, -having only a few exit nodes reduces the number of points -an adversary needs to monitor for traffic analysis, and places a -greater burden on the exit nodes. This tension can be seen in the -Java Anon Proxy -cascade model, wherein only one node in each cascade needs to handle -abuse complaints — but an adversary only needs to observe the entry -and exit of a cascade to perform traffic analysis on all that -cascade's users. The hydra model (many entries, few exits) presents a -different compromise: only a few exit nodes are needed, but an -adversary needs to work harder to watch all the clients; see -Section <a href="#sec:conclusion">10</a>. - -<div class="p"><!----></div> -Finally, we note that exit abuse must not be dismissed as a peripheral -issue: when a system's public image suffers, it can reduce the number -and diversity of that system's users, and thereby reduce the anonymity -of the system itself. Like usability, public perception is a -security parameter. Sadly, preventing abuse of open exit nodes is an -unsolved problem, and will probably remain an arms race for the -foreseeable future. The abuse problems faced by Princeton's CoDeeN -project [<a href="#darkside" name="CITEdarkside">37</a>] give us a glimpse of likely issues. - -<div class="p"><!----></div> - <h3><a name="tth_sEc6.3"> -<a name="subsec:dirservers"> -6.3</a> Directory Servers</h3> -</a> - -<div class="p"><!----></div> -First-generation Onion Routing designs [<a href="#freedom2-arch" name="CITEfreedom2-arch">8</a>,<a href="#or-jsac98" name="CITEor-jsac98">41</a>] used -in-band network status updates: each router flooded a signed statement -to its neighbors, which propagated it onward. But anonymizing networks -have different security goals than typical link-state routing protocols. -For example, delays (accidental or intentional) -that can cause different parts of the network to have different views -of link-state and topology are not only inconvenient: they give -attackers an opportunity to exploit differences in client knowledge. -We also worry about attacks to deceive a -client about the router membership list, topology, or current network -state. Such <em>partitioning attacks</em> on client knowledge help an -adversary to efficiently deploy resources -against a target [<a href="#minion-design" name="CITEminion-design">15</a>]. - -<div class="p"><!----></div> -Tor uses a small group of redundant, well-known onion routers to -track changes in network topology and node state, including keys and -exit policies. Each such <em>directory server</em> acts as an HTTP -server, so clients can fetch current network state -and router lists, and so other ORs can upload -state information. Onion routers periodically publish signed -statements of their state to each directory server. The directory servers -combine this information with their own views of network liveness, -and generate a signed description (a <em>directory</em>) of the entire -network state. Client software is -pre-loaded with a list of the directory servers and their keys, -to bootstrap each client's view of the network. - -<div class="p"><!----></div> -When a directory server receives a signed statement for an OR, it -checks whether the OR's identity key is recognized. Directory -servers do not advertise unrecognized ORs — if they did, -an adversary could take over the network by creating many -servers [<a href="#sybil" name="CITEsybil">22</a>]. Instead, new nodes must be approved by the -directory -server administrator before they are included. Mechanisms for automated -node approval are an area of active research, and are discussed more -in Section <a href="#sec:maintaining-anonymity">9</a>. - -<div class="p"><!----></div> -Of course, a variety of attacks remain. An adversary who controls -a directory server can track clients by providing them different -information — perhaps by listing only nodes under its control, or by -informing only certain clients about a given node. Even an external -adversary can exploit differences in client knowledge: clients who use -a node listed on one directory server but not the others are vulnerable. - -<div class="p"><!----></div> -Thus these directory servers must be synchronized and redundant, so -that they can agree on a common directory. Clients should only trust -this directory if it is signed by a threshold of the directory -servers. - -<div class="p"><!----></div> -The directory servers in Tor are modeled after those in -Mixminion [<a href="#minion-design" name="CITEminion-design">15</a>], but our situation is easier. First, -we make the -simplifying assumption that all participants agree on the set of -directory servers. Second, while Mixminion needs to predict node -behavior, Tor only needs a threshold consensus of the current -state of the network. Third, we assume that we can fall back to the -human administrators to discover and resolve problems when a consensus -directory cannot be reached. Since there are relatively few directory -servers (currently 3, but we expect as many as 9 as the network scales), -we can afford operations like broadcast to simplify the consensus-building -protocol. - -<div class="p"><!----></div> -To avoid attacks where a router connects to all the directory servers -but refuses to relay traffic from other routers, the directory servers -must also build circuits and use them to anonymously test router -reliability [<a href="#mix-acc" name="CITEmix-acc">18</a>]. Unfortunately, this defense is not yet -designed or -implemented. - -<div class="p"><!----></div> -Using directory servers is simpler and more flexible than flooding. -Flooding is expensive, and complicates the analysis when we -start experimenting with non-clique network topologies. Signed -directories can be cached by other -onion routers, -so directory servers are not a performance -bottleneck when we have many users, and do not aid traffic analysis by -forcing clients to announce their existence to any -central point. - -<div class="p"><!----></div> - <h2><a name="tth_sEc7"> -<a name="sec:attacks"> -7</a> Attacks and Defenses</h2> -</a> - -<div class="p"><!----></div> -Below we summarize a variety of attacks, and discuss how well our -design withstands them.<br /> - -<div class="p"><!----></div> -<font size="+1"><b>Passive attacks</b></font><br /> -<em>Observing user traffic patterns.</em> Observing a user's connection -will not reveal her destination or data, but it will -reveal traffic patterns (both sent and received). Profiling via user -connection patterns requires further processing, because multiple -application streams may be operating simultaneously or in series over -a single circuit. - -<div class="p"><!----></div> -<em>Observing user content.</em> While content at the user end is encrypted, -connections to responders may not be (indeed, the responding website -itself may be hostile). While filtering content is not a primary goal -of Onion Routing, Tor can directly use Privoxy and related -filtering services to anonymize application data streams. - -<div class="p"><!----></div> -<em>Option distinguishability.</em> We allow clients to choose -configuration options. For example, clients concerned about request -linkability should rotate circuits more often than those concerned -about traceability. Allowing choice may attract users with different -needs; but clients who are -in the minority may lose more anonymity by appearing distinct than they -gain by optimizing their behavior [<a href="#econymics" name="CITEeconymics">1</a>]. - -<div class="p"><!----></div> -<em>End-to-end timing correlation.</em> Tor only minimally hides -such correlations. An attacker watching patterns of -traffic at the initiator and the responder will be -able to confirm the correspondence with high probability. The -greatest protection currently available against such confirmation is to hide -the connection between the onion proxy and the first Tor node, -by running the OP on the Tor node or behind a firewall. This approach -requires an observer to separate traffic originating at the onion -router from traffic passing through it: a global observer can do this, -but it might be beyond a limited observer's capabilities. - -<div class="p"><!----></div> -<em>End-to-end size correlation.</em> Simple packet counting -will also be effective in confirming -endpoints of a stream. However, even without padding, we may have some -limited protection: the leaky pipe topology means different numbers -of packets may enter one end of a circuit than exit at the other. - -<div class="p"><!----></div> -<em>Website fingerprinting.</em> All the effective passive -attacks above are traffic confirmation attacks, -which puts them outside our design goals. There is also -a passive traffic analysis attack that is potentially effective. -Rather than searching exit connections for timing and volume -correlations, the adversary may build up a database of -"fingerprints" containing file sizes and access patterns for -targeted websites. He can later confirm a user's connection to a given -site simply by consulting the database. This attack has -been shown to be effective against SafeWeb [<a href="#hintz-pet02" name="CITEhintz-pet02">29</a>]. -It may be less effective against Tor, since -streams are multiplexed within the same circuit, and -fingerprinting will be limited to -the granularity of cells (currently 512 bytes). Additional -defenses could include -larger cell sizes, padding schemes to group websites -into large sets, and link -padding or long-range dummies.<a href="#tthFtNtAAE" name="tthFrefAAE"><sup>4</sup></a><br /> - -<div class="p"><!----></div> -<font size="+1"><b>Active attacks</b></font><br /> -<em>Compromise keys.</em> An attacker who learns the TLS session key can -see control cells and encrypted relay cells on every circuit on that -connection; learning a circuit -session key lets him unwrap one layer of the encryption. An attacker -who learns an OR's TLS private key can impersonate that OR for the TLS -key's lifetime, but he must -also learn the onion key to decrypt <em>create</em> cells (and because of -perfect forward secrecy, he cannot hijack already established circuits -without also compromising their session keys). Periodic key rotation -limits the window of opportunity for these attacks. On the other hand, -an attacker who learns a node's identity key can replace that node -indefinitely by sending new forged descriptors to the directory servers. - -<div class="p"><!----></div> -<em>Iterated compromise.</em> A roving adversary who can -compromise ORs (by system intrusion, legal coercion, or extralegal -coercion) could march down the circuit compromising the -nodes until he reaches the end. Unless the adversary can complete -this attack within the lifetime of the circuit, however, the ORs -will have discarded the necessary information before the attack can -be completed. (Thanks to the perfect forward secrecy of session -keys, the attacker cannot force nodes to decrypt recorded -traffic once the circuits have been closed.) Additionally, building -circuits that cross jurisdictions can make legal coercion -harder — this phenomenon is commonly called "jurisdictional -arbitrage." The Java Anon Proxy project recently experienced the -need for this approach, when -a German court forced them to add a backdoor to -their nodes [<a href="#jap-backdoor" name="CITEjap-backdoor">51</a>]. - -<div class="p"><!----></div> -<em>Run a recipient.</em> An adversary running a webserver -trivially learns the timing patterns of users connecting to it, and -can introduce arbitrary patterns in its responses. -End-to-end attacks become easier: if the adversary can induce -users to connect to his webserver (perhaps by advertising -content targeted to those users), he now holds one end of their -connection. There is also a danger that application -protocols and associated programs can be induced to reveal information -about the initiator. Tor depends on Privoxy and similar protocol cleaners -to solve this latter problem. - -<div class="p"><!----></div> -<em>Run an onion proxy.</em> It is expected that end users will -nearly always run their own local onion proxy. However, in some -settings, it may be necessary for the proxy to run -remotely — typically, in institutions that want -to monitor the activity of those connecting to the proxy. -Compromising an onion proxy compromises all future connections -through it. - -<div class="p"><!----></div> -<em>DoS non-observed nodes.</em> An observer who can only watch some -of the Tor network can increase the value of this traffic -by attacking non-observed nodes to shut them down, reduce -their reliability, or persuade users that they are not trustworthy. -The best defense here is robustness. - -<div class="p"><!----></div> -<em>Run a hostile OR.</em> In addition to being a local observer, -an isolated hostile node can create circuits through itself, or alter -traffic patterns to affect traffic at other nodes. Nonetheless, a hostile -node must be immediately adjacent to both endpoints to compromise the -anonymity of a circuit. If an adversary can -run multiple ORs, and can persuade the directory servers -that those ORs are trustworthy and independent, then occasionally -some user will choose one of those ORs for the start and another -as the end of a circuit. If an adversary -controls m > 1 of N nodes, he can correlate at most -([m/N])<sup>2</sup> of the traffic — although an -adversary -could still attract a disproportionately large amount of traffic -by running an OR with a permissive exit policy, or by -degrading the reliability of other routers. - -<div class="p"><!----></div> -<em>Introduce timing into messages.</em> This is simply a stronger -version of passive timing attacks already discussed earlier. - -<div class="p"><!----></div> -<em>Tagging attacks.</em> A hostile node could "tag" a -cell by altering it. If the -stream were, for example, an unencrypted request to a Web site, -the garbled content coming out at the appropriate time would confirm -the association. However, integrity checks on cells prevent -this attack. - -<div class="p"><!----></div> -<em>Replace contents of unauthenticated protocols.</em> When -relaying an unauthenticated protocol like HTTP, a hostile exit node -can impersonate the target server. Clients -should prefer protocols with end-to-end authentication. - -<div class="p"><!----></div> -<em>Replay attacks.</em> Some anonymity protocols are vulnerable -to replay attacks. Tor is not; replaying one side of a handshake -will result in a different negotiated session key, and so the rest -of the recorded session can't be used. - -<div class="p"><!----></div> -<em>Smear attacks.</em> An attacker could use the Tor network for -socially disapproved acts, to bring the -network into disrepute and get its operators to shut it down. -Exit policies reduce the possibilities for abuse, but -ultimately the network requires volunteers who can tolerate -some political heat. - -<div class="p"><!----></div> -<em>Distribute hostile code.</em> An attacker could trick users -into running subverted Tor software that did not, in fact, anonymize -their connections — or worse, could trick ORs into running weakened -software that provided users with less anonymity. We address this -problem (but do not solve it completely) by signing all Tor releases -with an official public key, and including an entry in the directory -that lists which versions are currently believed to be secure. To -prevent an attacker from subverting the official release itself -(through threats, bribery, or insider attacks), we provide all -releases in source code form, encourage source audits, and -frequently warn our users never to trust any software (even from -us) that comes without source.<br /> - -<div class="p"><!----></div> -<font size="+1"><b>Directory attacks</b></font><br /> -<em>Destroy directory servers.</em> If a few directory -servers disappear, the others still decide on a valid -directory. So long as any directory servers remain in operation, -they will still broadcast their views of the network and generate a -consensus directory. (If more than half are destroyed, this -directory will not, however, have enough signatures for clients to -use it automatically; human intervention will be necessary for -clients to decide whether to trust the resulting directory.) - -<div class="p"><!----></div> -<em>Subvert a directory server.</em> By taking over a directory server, -an attacker can partially influence the final directory. Since ORs -are included or excluded by majority vote, the corrupt directory can -at worst cast a tie-breaking vote to decide whether to include -marginal ORs. It remains to be seen how often such marginal cases -occur in practice. - -<div class="p"><!----></div> -<em>Subvert a majority of directory servers.</em> An adversary who controls -more than half the directory servers can include as many compromised -ORs in the final directory as he wishes. We must ensure that directory -server operators are independent and attack-resistant. - -<div class="p"><!----></div> -<em>Encourage directory server dissent.</em> The directory -agreement protocol assumes that directory server operators agree on -the set of directory servers. An adversary who can persuade some -of the directory server operators to distrust one another could -split the quorum into mutually hostile camps, thus partitioning -users based on which directory they use. Tor does not address -this attack. - -<div class="p"><!----></div> -<em>Trick the directory servers into listing a hostile OR.</em> -Our threat model explicitly assumes directory server operators will -be able to filter out most hostile ORs. - -<div class="p"><!----></div> -<em>Convince the directories that a malfunctioning OR is -working.</em> In the current Tor implementation, directory servers -assume that an OR is running correctly if they can start a TLS -connection to it. A hostile OR could easily subvert this test by -accepting TLS connections from ORs but ignoring all cells. Directory -servers must actively test ORs by building circuits and streams as -appropriate. The tradeoffs of a similar approach are discussed -in [<a href="#mix-acc" name="CITEmix-acc">18</a>].<br /> - -<div class="p"><!----></div> -<font size="+1"><b>Attacks against rendezvous points</b></font><br /> -<em>Make many introduction requests.</em> An attacker could -try to deny Bob service by flooding his introduction points with -requests. Because the introduction points can block requests that -lack authorization tokens, however, Bob can restrict the volume of -requests he receives, or require a certain amount of computation for -every request he receives. - -<div class="p"><!----></div> -<em>Attack an introduction point.</em> An attacker could -disrupt a location-hidden service by disabling its introduction -points. But because a service's identity is attached to its public -key, the service can simply re-advertise -itself at a different introduction point. Advertisements can also be -done secretly so that only high-priority clients know the address of -Bob's introduction points or so that different clients know of different -introduction points. This forces the attacker to disable all possible -introduction points. - -<div class="p"><!----></div> -<em>Compromise an introduction point.</em> An attacker who controls -Bob's introduction point can flood Bob with -introduction requests, or prevent valid introduction requests from -reaching him. Bob can notice a flood, and close the circuit. To notice -blocking of valid requests, however, he should periodically test the -introduction point by sending rendezvous requests and making -sure he receives them. - -<div class="p"><!----></div> -<em>Compromise a rendezvous point.</em> A rendezvous -point is no more sensitive than any other OR on -a circuit, since all data passing through the rendezvous is encrypted -with a session key shared by Alice and Bob. - -<div class="p"><!----></div> - <h2><a name="tth_sEc8"> -<a name="sec:in-the-wild"> -8</a> Early experiences: Tor in the Wild</h2> -</a> - -<div class="p"><!----></div> -As of mid-May 2004, the Tor network consists of 32 nodes -(24 in the US, 8 in Europe), and more are joining each week as the code -matures. (For comparison, the current remailer network -has about 40 nodes.) Each node has at least a 768Kb/768Kb connection, and -many have 10Mb. The number of users varies (and of course, it's hard to -tell for sure), but we sometimes have several hundred users — administrators at -several companies have begun sending their entire departments' web -traffic through Tor, to block other divisions of -their company from reading their traffic. Tor users have reported using -the network for web browsing, FTP, IRC, AIM, Kazaa, SSH, and -recipient-anonymous email via rendezvous points. One user has anonymously -set up a Wiki as a hidden service, where other users anonymously publish -the addresses of their hidden services. - -<div class="p"><!----></div> -Each Tor node currently processes roughly 800,000 relay -cells (a bit under half a gigabyte) per week. On average, about 80% -of each 498-byte payload is full for cells going back to the client, -whereas about 40% is full for cells coming from the client. (The difference -arises because most of the network's traffic is web browsing.) Interactive -traffic like SSH brings down the average a lot — once we have more -experience, and assuming we can resolve the anonymity issues, we may -partition traffic into two relay cell sizes: one to handle -bulk traffic and one for interactive traffic. - -<div class="p"><!----></div> -Based in part on our restrictive default exit policy (we -reject SMTP requests) and our low profile, we have had no abuse -issues since the network was deployed in October -2003. Our slow growth rate gives us time to add features, -resolve bugs, and get a feel for what users actually want from an -anonymity system. Even though having more users would bolster our -anonymity sets, we are not eager to attract the Kazaa or warez -communities — we feel that we must build a reputation for privacy, human -rights, research, and other socially laudable activities. - -<div class="p"><!----></div> -As for performance, profiling shows that Tor spends almost -all its CPU time in AES, which is fast. Current latency is attributable -to two factors. First, network latency is critical: we are -intentionally bouncing traffic around the world several times. Second, -our end-to-end congestion control algorithm focuses on protecting -volunteer servers from accidental DoS rather than on optimizing -performance. To quantify these effects, we did some informal tests using a network of 4 -nodes on the same machine (a heavily loaded 1GHz Athlon). We downloaded a 60 -megabyte file from <tt>debian.org</tt> every 30 minutes for 54 hours (108 sample -points). It arrived in about 300 seconds on average, compared to 210s for a -direct download. We ran a similar test on the production Tor network, -fetching the front page of <tt>cnn.com</tt> (55 kilobytes): -while a direct -download consistently took about 0.3s, the performance through Tor varied. -Some downloads were as fast as 0.4s, with a median at 2.8s, and -90% finishing within 5.3s. It seems that as the network expands, the chance -of building a slow circuit (one that includes a slow or heavily loaded node -or link) is increasing. On the other hand, as our users remain satisfied -with this increased latency, we can address our performance incrementally as we -proceed with development. -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> -Although Tor's clique topology and full-visibility directories present -scaling problems, we still expect the network to support a few hundred -nodes and maybe 10,000 users before we're forced to become -more distributed. With luck, the experience we gain running the current -topology will help us choose among alternatives when the time comes. - -<div class="p"><!----></div> - <h2><a name="tth_sEc9"> -<a name="sec:maintaining-anonymity"> -9</a> Open Questions in Low-latency Anonymity</h2> -</a> - -<div class="p"><!----></div> -In addition to the non-goals in -Section <a href="#subsec:non-goals">3</a>, many questions must be solved -before we can be confident of Tor's security. - -<div class="p"><!----></div> -Many of these open issues are questions of balance. For example, -how often should users rotate to fresh circuits? Frequent rotation -is inefficient, expensive, and may lead to intersection attacks and -predecessor attacks [<a href="#wright03" name="CITEwright03">54</a>], but infrequent rotation makes the -user's traffic linkable. Besides opening fresh circuits, clients can -also exit from the middle of the circuit, -or truncate and re-extend the circuit. More analysis is -needed to determine the proper tradeoff. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -How should we choose path lengths? If Alice always uses two hops, -then both ORs can be certain that by colluding they will learn about -Alice and Bob. In our current approach, Alice always chooses at least -three nodes unrelated to herself and her destination. -Should Alice choose a random path length (e.g. from a geometric -distribution) to foil an attacker who -uses timing to learn that he is the fifth hop and thus concludes that -both Alice and the responder are running ORs? - -<div class="p"><!----></div> -Throughout this paper, we have assumed that end-to-end traffic -confirmation will immediately and automatically defeat a low-latency -anonymity system. Even high-latency anonymity systems can be -vulnerable to end-to-end traffic confirmation, if the traffic volumes -are high enough, and if users' habits are sufficiently -distinct [<a href="#statistical-disclosure" name="CITEstatistical-disclosure">14</a>,<a href="#limits-open" name="CITElimits-open">31</a>]. Can anything be -done to -make low-latency systems resist these attacks as well as high-latency -systems? Tor already makes some effort to conceal the starts and ends of -streams by wrapping long-range control commands in identical-looking -relay cells. Link padding could frustrate passive observers who count -packets; long-range padding could work against observers who own the -first hop in a circuit. But more research remains to find an efficient -and practical approach. Volunteers prefer not to run constant-bandwidth -padding; but no convincing traffic shaping approach has been -specified. Recent work on long-range padding [<a href="#defensive-dropping" name="CITEdefensive-dropping">33</a>] -shows promise. One could also try to reduce correlation in packet timing -by batching and re-ordering packets, but it is unclear whether this could -improve anonymity without introducing so much latency as to render the -network unusable. - -<div class="p"><!----></div> -A cascade topology may better defend against traffic confirmation by -aggregating users, and making padding and -mixing more affordable. Does the hydra topology (many input nodes, -few output nodes) work better against some adversaries? Are we going -to get a hydra anyway because most nodes will be middleman nodes? - -<div class="p"><!----></div> -Common wisdom suggests that Alice should run her own OR for best -anonymity, because traffic coming from her node could plausibly have -come from elsewhere. How much mixing does this approach need? Is it -immediately beneficial because of real-world adversaries that can't -observe Alice's router, but can run routers of their own? - -<div class="p"><!----></div> -To scale to many users, and to prevent an attacker from observing the -whole network, it may be necessary -to support far more servers than Tor currently anticipates. -This introduces several issues. First, if approval by a central set -of directory servers is no longer feasible, what mechanism should be used -to prevent adversaries from signing up many colluding servers? Second, -if clients can no longer have a complete picture of the network, -how can they perform discovery while preventing attackers from -manipulating or exploiting gaps in their knowledge? Third, if there -are too many servers for every server to constantly communicate with -every other, which non-clique topology should the network use? -(Restricted-route topologies promise comparable anonymity with better -scalability [<a href="#danezis-pets03" name="CITEdanezis-pets03">13</a>], but whatever topology we choose, we -need some way to keep attackers from manipulating their position within -it [<a href="#casc-rep" name="CITEcasc-rep">21</a>].) Fourth, if no central authority is tracking -server reliability, how do we stop unreliable servers from making -the network unusable? Fifth, do clients receive so much anonymity -from running their own ORs that we should expect them all to do -so [<a href="#econymics" name="CITEeconymics">1</a>], or do we need another incentive structure to -motivate them? Tarzan and MorphMix present possible solutions. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -When a Tor node goes down, all its circuits (and thus streams) must break. -Will users abandon the system because of this brittleness? How well -does the method in Section <a href="#subsec:dos">6.1</a> allow streams to survive -node failure? If affected users rebuild circuits immediately, how much -anonymity is lost? It seems the problem is even worse in a peer-to-peer -environment — such systems don't yet provide an incentive for peers to -stay connected when they're done retrieving content, so we would expect -a higher churn rate. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - <h2><a name="tth_sEc10"> -<a name="sec:conclusion"> -10</a> Future Directions</h2> -</a> - -<div class="p"><!----></div> -Tor brings together many innovations into a unified deployable system. The -next immediate steps include: - -<div class="p"><!----></div> -<em>Scalability:</em> Tor's emphasis on deployability and design simplicity -has led us to adopt a clique topology, semi-centralized -directories, and a full-network-visibility model for client -knowledge. These properties will not scale past a few hundred servers. -Section <a href="#sec:maintaining-anonymity">9</a> describes some promising -approaches, but more deployment experience will be helpful in learning -the relative importance of these bottlenecks. - -<div class="p"><!----></div> -<em>Bandwidth classes:</em> This paper assumes that all ORs have -good bandwidth and latency. We should instead adopt the MorphMix model, -where nodes advertise their bandwidth level (DSL, T1, T3), and -Alice avoids bottlenecks by choosing nodes that match or -exceed her bandwidth. In this way DSL users can usefully join the Tor -network. - -<div class="p"><!----></div> -<em>Incentives:</em> Volunteers who run nodes are rewarded with publicity -and possibly better anonymity [<a href="#econymics" name="CITEeconymics">1</a>]. More nodes means increased -scalability, and more users can mean more anonymity. We need to continue -examining the incentive structures for participating in Tor. Further, -we need to explore more approaches to limiting abuse, and understand -why most people don't bother using privacy systems. - -<div class="p"><!----></div> -<em>Cover traffic:</em> Currently Tor omits cover traffic — its costs -in performance and bandwidth are clear but its security benefits are -not well understood. We must pursue more research on link-level cover -traffic and long-range cover traffic to determine whether some simple padding -method offers provable protection against our chosen adversary. - -<div class="p"><!----></div> - -<div class="p"><!----></div> -<em>Caching at exit nodes:</em> Perhaps each exit node should run a -caching web proxy [<a href="#shsm03" name="CITEshsm03">47</a>], to improve anonymity for cached pages -(Alice's request never -leaves the Tor network), to improve speed, and to reduce bandwidth cost. -On the other hand, forward security is weakened because caches -constitute a record of retrieved files. We must find the right -balance between usability and security. - -<div class="p"><!----></div> -<em>Better directory distribution:</em> -Clients currently download a description of -the entire network every 15 minutes. As the state grows larger -and clients more numerous, we may need a solution in which -clients receive incremental updates to directory state. -More generally, we must find more -scalable yet practical ways to distribute up-to-date snapshots of -network status without introducing new attacks. - -<div class="p"><!----></div> -<em>Further specification review:</em> Our public -byte-level specification [<a href="#tor-spec" name="CITEtor-spec">20</a>] needs -external review. We hope that as Tor -is deployed, more people will examine its -specification. - -<div class="p"><!----></div> -<em>Multisystem interoperability:</em> We are currently working with the -designer of MorphMix to unify the specification and implementation of -the common elements of our two systems. So far, this seems -to be relatively straightforward. Interoperability will allow testing -and direct comparison of the two designs for trust and scalability. - -<div class="p"><!----></div> -<em>Wider-scale deployment:</em> The original goal of Tor was to -gain experience in deploying an anonymizing overlay network, and -learn from having actual users. We are now at a point in design -and development where we can start deploying a wider network. Once -we have many actual users, we will doubtlessly be better -able to evaluate some of our design decisions, including our -robustness/latency tradeoffs, our performance tradeoffs (including -cell size), our abuse-prevention mechanisms, and -our overall usability. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<h2>Acknowledgments</h2> - We thank Peter Palfrader, Geoff Goodell, Adam Shostack, Joseph Sokol-Margolis, - John Bashinski, and Zack Brown - for editing and comments; - Matej Pfajfar, Andrei Serjantov, Marc Rennhard for design discussions; - Bram Cohen for congestion control discussions; - Adam Back for suggesting telescoping circuits; and - Cathy Meadows for formal analysis of the <em>extend</em> protocol. - This work has been supported by ONR and DARPA. - -<div class="p"><!----></div> - -<div class="p"><!----></div> - -<div class="p"><!----></div> -<h2>References</h2> - -<dl compact="compact"> -<font size="-1"></font> <dt><a href="#CITEeconymics" name="econymics">[1]</a></dt><dd> -A. Acquisti, R. Dingledine, and P. Syverson. - On the economics of anonymity. - In R. N. Wright, editor, <em>Financial Cryptography</em>. - Springer-Verlag, LNCS 2742, 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEeternity" name="eternity">[2]</a></dt><dd> -R. Anderson. - The eternity service. - In <em>Pragocrypt '96</em>, 1996. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEanonymizer" name="anonymizer">[3]</a></dt><dd> -The Anonymizer. - <tt><http://anonymizer.com/>. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEfreedom21-security" name="freedom21-security">[4]</a></dt><dd> -A. Back, I. Goldberg, and A. Shostack. - Freedom systems 2.1 security issues and analysis. - White paper, Zero Knowledge Systems, Inc., May 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEback01" name="back01">[5]</a></dt><dd> -A. Back, U. Möller, and A. Stiglic. - Traffic analysis attacks and trade-offs in anonymity providing - systems. - In I. S. Moskowitz, editor, <em>Information Hiding (IH 2001)</em>, pages - 245-257. Springer-Verlag, LNCS 2137, 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEeax" name="eax">[6]</a></dt><dd> -M. Bellare, P. Rogaway, and D. Wagner. - The EAX mode of operation: A two-pass authenticated-encryption - scheme optimized for simplicity and efficiency. - In <em>Fast Software Encryption 2004</em>, February 2004. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEweb-mix" name="web-mix">[7]</a></dt><dd> -O. Berthold, H. Federrath, and S. Köpsell. - Web MIXes: A system for anonymous and unobservable Internet - access. - In H. Federrath, editor, <em>Designing Privacy Enhancing - Technologies: Workshop on Design Issue in Anonymity and Unobservability</em>. - Springer-Verlag, LNCS 2009, 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEfreedom2-arch" name="freedom2-arch">[8]</a></dt><dd> -P. Boucher, A. Shostack, and I. Goldberg. - Freedom systems 2.0 architecture. - White paper, Zero Knowledge Systems, Inc., December 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEcebolla" name="cebolla">[9]</a></dt><dd> -Z. Brown. - Cebolla: Pragmatic IP Anonymity. - In <em>Ottawa Linux Symposium</em>, June 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEchaum-mix" name="chaum-mix">[10]</a></dt><dd> -D. Chaum. - Untraceable electronic mail, return addresses, and digital - pseudo-nyms. - <em>Communications of the ACM</em>, 4(2), February 1981. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEcfs:sosp01" name="cfs:sosp01">[11]</a></dt><dd> -F. Dabek, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. - Wide-area cooperative storage with CFS. - In <em>18th ACM Symposium on Operating Systems Principles - (SOSP '01)</em>, Chateau Lake Louise, Banff, Canada, October 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEpipenet" name="pipenet">[12]</a></dt><dd> -W. Dai. - Pipenet 1.1. - Usenet post, August 1996. - <tt><http://www.eskimo.com/ weidai/pipenet.txt> First mentioned in a - post to the cypherpunks list, Feb. 1995. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEdanezis-pets03" name="danezis-pets03">[13]</a></dt><dd> -G. Danezis. - Mix-networks with restricted routes. - In R. Dingledine, editor, <em>Privacy Enhancing Technologies (PET - 2003)</em>. Springer-Verlag LNCS 2760, 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEstatistical-disclosure" name="statistical-disclosure">[14]</a></dt><dd> -G. Danezis. - Statistical disclosure attacks. - In <em>Security and Privacy in the Age of Uncertainty (SEC2003)</em>, - pages 421-426, Athens, May 2003. IFIP TC11, Kluwer. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEminion-design" name="minion-design">[15]</a></dt><dd> -G. Danezis, R. Dingledine, and N. Mathewson. - Mixminion: Design of a type III anonymous remailer protocol. - In <em>2003 IEEE Symposium on Security and Privacy</em>, pages 2-15. - IEEE CS, May 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEpuzzles-tls" name="puzzles-tls">[16]</a></dt><dd> -D. Dean and A. Stubblefield. - Using Client Puzzles to Protect TLS. - In <em>Proceedings of the 10th USENIX Security Symposium</em>. USENIX, - Aug. 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITETLS" name="TLS">[17]</a></dt><dd> -T. Dierks and C. Allen. - The TLS Protocol - Version 1.0. - IETF RFC 2246, January 1999. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEmix-acc" name="mix-acc">[18]</a></dt><dd> -R. Dingledine, M. J. Freedman, D. Hopwood, and D. Molnar. - A Reputation System to Increase MIX-net Reliability. - In I. S. Moskowitz, editor, <em>Information Hiding (IH 2001)</em>, pages - 126-141. Springer-Verlag, LNCS 2137, 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEfreehaven-berk" name="freehaven-berk">[19]</a></dt><dd> -R. Dingledine, M. J. Freedman, and D. Molnar. - The free haven project: Distributed anonymous storage service. - In H. Federrath, editor, <em>Designing Privacy Enhancing - Technologies: Workshop on Design Issue in Anonymity and Unobservability</em>. - Springer-Verlag, LNCS 2009, July 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEtor-spec" name="tor-spec">[20]</a></dt><dd> -R. Dingledine and N. Mathewson. - Tor protocol specifications. - <tt><http://freehaven.net/tor/tor-spec.txt>. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEcasc-rep" name="casc-rep">[21]</a></dt><dd> -R. Dingledine and P. Syverson. - Reliable MIX Cascade Networks through Reputation. - In M. Blaze, editor, <em>Financial Cryptography</em>. Springer-Verlag, - LNCS 2357, 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEsybil" name="sybil">[22]</a></dt><dd> -J. Douceur. - The Sybil Attack. - In <em>Proceedings of the 1st International Peer To Peer Systems - Workshop (IPTPS)</em>, Mar. 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEfederrath-ih96" name="federrath-ih96">[23]</a></dt><dd> -H. Federrath, A. Jerichow, and A. Pfitzmann. - MIXes in mobile communication systems: Location management with - privacy. - In R. Anderson, editor, <em>Information Hiding, First International - Workshop</em>, pages 121-135. Springer-Verlag, LNCS 1174, May 1996. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEtarzan:ccs02" name="tarzan:ccs02">[24]</a></dt><dd> -M. J. Freedman and R. Morris. - Tarzan: A peer-to-peer anonymizing network layer. - In <em>9th ACM Conference on Computer and Communications - Security (CCS 2002)</em>, Washington, DC, November 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEherbivore" name="herbivore">[25]</a></dt><dd> -S. Goel, M. Robson, M. Polte, and E. G. Sirer. - Herbivore: A scalable and efficient protocol for anonymous - communication. - Technical Report TR2003-1890, Cornell University Computing and - Information Science, February 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEian-thesis" name="ian-thesis">[26]</a></dt><dd> -I. Goldberg. - <em>A Pseudonymous Communications Infrastructure for the Internet</em>. - PhD thesis, UC Berkeley, Dec 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEor-ih96" name="or-ih96">[27]</a></dt><dd> -D. M. Goldschlag, M. G. Reed, and P. F. Syverson. - Hiding routing information. - In R. Anderson, editor, <em>Information Hiding, First International - Workshop</em>, pages 137-150. Springer-Verlag, LNCS 1174, May 1996. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEbabel" name="babel">[28]</a></dt><dd> -C. Gülcü and G. Tsudik. - Mixing E-mail with Babel. - In <em>Network and Distributed Security Symposium (NDSS 96)</em>, - pages 2-16. IEEE, February 1996. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEhintz-pet02" name="hintz-pet02">[29]</a></dt><dd> -A. Hintz. - Fingerprinting websites using traffic analysis. - In R. Dingledine and P. Syverson, editors, <em>Privacy Enhancing - Technologies (PET 2002)</em>, pages 171-178. Springer-Verlag, LNCS 2482, 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEjerichow-jsac98" name="jerichow-jsac98">[30]</a></dt><dd> -A. Jerichow, J. Müller, A. Pfitzmann, B. Pfitzmann, and M. Waidner. - Real-time mixes: A bandwidth-efficient anonymity protocol. - <em>IEEE Journal on Selected Areas in Communications</em>, - 16(4):495-509, May 1998. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITElimits-open" name="limits-open">[31]</a></dt><dd> -D. Kesdogan, D. Agrawal, and S. Penz. - Limits of anonymity in open environments. - In F. Petitcolas, editor, <em>Information Hiding Workshop (IH - 2002)</em>. Springer-Verlag, LNCS 2578, October 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEsocks4" name="socks4">[32]</a></dt><dd> -D. Koblas and M. R. Koblas. - SOCKS. - In <em>UNIX Security III Symposium (1992 USENIX Security - Symposium)</em>, pages 77-83. USENIX, 1992. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEdefensive-dropping" name="defensive-dropping">[33]</a></dt><dd> -B. N. Levine, M. K. Reiter, C. Wang, and M. Wright. - Timing analysis in low-latency mix-based systems. - In A. Juels, editor, <em>Financial Cryptography</em>. Springer-Verlag, - LNCS (forthcoming), 2004. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEhordes-jcs" name="hordes-jcs">[34]</a></dt><dd> -B. N. Levine and C. Shields. - Hordes: A multicast-based protocol for anonymity. - <em>Journal of Computer Security</em>, 10(3):213-240, 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEmeadows96" name="meadows96">[35]</a></dt><dd> -C. Meadows. - The NRL protocol analyzer: An overview. - <em>Journal of Logic Programming</em>, 26(2):113-131, 1996. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEmixmaster-spec" name="mixmaster-spec">[36]</a></dt><dd> -U. Möller, L. Cottrell, P. Palfrader, and L. Sassaman. - Mixmaster Protocol - Version 2. - Draft, July 2003. - <tt><http://www.abditum.com/mixmaster-spec.txt>. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEdarkside" name="darkside">[37]</a></dt><dd> -V. S. Pai, L. Wang, K. Park, R. Pang, and L. Peterson. - The Dark Side of the Web: An Open Proxy's View. - <tt><http://codeen.cs.princeton.edu/>. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEisdn-mixes" name="isdn-mixes">[38]</a></dt><dd> -A. Pfitzmann, B. Pfitzmann, and M. Waidner. - ISDN-mixes: Untraceable communication with very small bandwidth - overhead. - In <em>GI/ITG Conference on Communication in Distributed Systems</em>, - pages 451-463, February 1991. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEprivoxy" name="privoxy">[39]</a></dt><dd> -Privoxy. - <tt><http://www.privoxy.org/>. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEreed-protocols97" name="reed-protocols97">[40]</a></dt><dd> -M. G. Reed, P. F. Syverson, and D. M. Goldschlag. - Protocols using anonymous connections: Mobile applications. - In B. Christianson, B. Crispo, M. Lomas, and M. Roe, editors, <em> - Security Protocols: 5th International Workshop</em>, pages 13-23. - Springer-Verlag, LNCS 1361, April 1997. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEor-jsac98" name="or-jsac98">[41]</a></dt><dd> -M. G. Reed, P. F. Syverson, and D. M. Goldschlag. - Anonymous connections and onion routing. - <em>IEEE Journal on Selected Areas in Communications</em>, - 16(4):482-494, May 1998. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEcrowds-tissec" name="crowds-tissec">[42]</a></dt><dd> -M. K. Reiter and A. D. Rubin. - Crowds: Anonymity for web transactions. - <em>ACM TISSEC</em>, 1(1):66-92, June 1998. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEmorphmix:fc04" name="morphmix:fc04">[43]</a></dt><dd> -M. Rennhard and B. Plattner. - Practical anonymity for the masses with morphmix. - In A. Juels, editor, <em>Financial Cryptography</em>. Springer-Verlag, - LNCS (forthcoming), 2004. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEanonnet" name="anonnet">[44]</a></dt><dd> -M. Rennhard, S. Rafaeli, L. Mathy, B. Plattner, and D. Hutchison. - Analysis of an Anonymity Network for Web Browsing. - In <em>IEEE 7th Intl. Workshop on Enterprise Security (WET ICE - 2002)</em>, Pittsburgh, USA, June 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITESS03" name="SS03">[45]</a></dt><dd> -A. Serjantov and P. Sewell. - Passive attack analysis for connection-based anonymity systems. - In <em>Computer Security - ESORICS 2003</em>. Springer-Verlag, LNCS - 2808, October 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEp5" name="p5">[46]</a></dt><dd> -R. Sherwood, B. Bhattacharjee, and A. Srinivasan. - p<sup>5</sup>: A protocol for scalable anonymous communication. - In <em>IEEE Symposium on Security and Privacy</em>, pages 58-70. IEEE - CS, 2002. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEshsm03" name="shsm03">[47]</a></dt><dd> -A. Shubina and S. Smith. - Using caching for browsing anonymity. - <em>ACM SIGEcom Exchanges</em>, 4(2), Sept 2003. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEor-discex00" name="or-discex00">[48]</a></dt><dd> -P. Syverson, M. Reed, and D. Goldschlag. - Onion Routing access configurations. - In <em>DARPA Information Survivability Conference and Exposition - (DISCEX 2000)</em>, volume 1, pages 34-40. IEEE CS Press, 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEor-pet00" name="or-pet00">[49]</a></dt><dd> -P. Syverson, G. Tsudik, M. Reed, and C. Landwehr. - Towards an Analysis of Onion Routing Security. - In H. Federrath, editor, <em>Designing Privacy Enhancing - Technologies: Workshop on Design Issue in Anonymity and Unobservability</em>, - pages 96-114. Springer-Verlag, LNCS 2009, July 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEtannenbaum96" name="tannenbaum96">[50]</a></dt><dd> -A. Tannenbaum. - Computer networks, 1996. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEjap-backdoor" name="jap-backdoor">[51]</a></dt><dd> -The AN.ON Project. - German police proceeds against anonymity service. - Press release, September 2003. - - <tt><http://www.datenschutzzentrum.de/material/themen/presse/anon-bka_e.htm>. - -<div class="p"><!----></div> -</tt></dd> - <dt><a href="#CITEtangler" name="tangler">[52]</a></dt><dd> -M. Waldman and D. Mazières. - Tangler: A censorship-resistant publishing system based on document - entanglements. - In <em>8<sup>th</sup> ACM Conference on Computer and Communications - Security (CCS-8)</em>, pages 86-135. ACM Press, 2001. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEpublius" name="publius">[53]</a></dt><dd> -M. Waldman, A. Rubin, and L. Cranor. - Publius: A robust, tamper-evident, censorship-resistant and - source-anonymous web publishing system. - In <em>Proc. 9th USENIX Security Symposium</em>, pages 59-72, August - 2000. - -<div class="p"><!----></div> -</dd> - <dt><a href="#CITEwright03" name="wright03">[54]</a></dt><dd> -M. Wright, M. Adler, B. N. Levine, and C. Shields. - Defending anonymous communication against passive logging attacks. - In <em>IEEE Symposium on Security and Privacy</em>, pages 28-41. IEEE - CS, May 2003.</dd> -</dl> - - -<div class="p"><!----></div> -<hr /><h3>Footnotes:</h3> - -<div class="p"><!----></div> -<a name="tthFtNtAAB"></a><a href="#tthFrefAAB"><sup>1</sup></a>Actually, the negotiated key is used to derive two - symmetric keys: one for each direction. -<div class="p"><!----></div> -<a name="tthFtNtAAC"></a><a href="#tthFrefAAC"><sup>2</sup></a> - With 48 bits of digest per cell, the probability of an accidental -collision is far lower than the chance of hardware failure. -<div class="p"><!----></div> -<a name="tthFtNtAAD"></a><a href="#tthFrefAAD"><sup>3</sup></a> -Rather than rely on an external infrastructure, the Onion Routing network -can run the lookup service itself. Our current implementation provides a -simple lookup system on the -directory servers. -<div class="p"><!----></div> -<a name="tthFtNtAAE"></a><a href="#tthFrefAAE"><sup>4</sup></a>Note that this fingerprinting -attack should not be confused with the much more complicated latency -attacks of [<a href="#back01" name="CITEback01">5</a>], which require a fingerprint of the latencies -of all circuits through the network, combined with those from the -network edges to the target user and the responder website. -<br /><br /><hr /><small>File translated from -T<sub><font size="-1">E</font></sub>X -by <a href="http://hutchinson.belmont.ma.us/tth/"> -T<sub><font size="-1">T</font></sub>H</a>, -version 3.59.<br />On 18 May 2004, 10:45.</small> -</body></html> - diff --git a/doc/design-paper/tor-design.pdf b/doc/design-paper/tor-design.pdf Binary files differdeleted file mode 100644 index 76a226515..000000000 --- a/doc/design-paper/tor-design.pdf +++ /dev/null diff --git a/doc/design-paper/tor-design.tex b/doc/design-paper/tor-design.tex deleted file mode 100644 index dff1b4068..000000000 --- a/doc/design-paper/tor-design.tex +++ /dev/null @@ -1,1988 +0,0 @@ -\documentclass[twocolumn]{article} -\usepackage{usenix} - -%\documentclass[times,10pt,twocolumn]{article} -%\usepackage{latex8} -%\usepackage{times} -\usepackage{url} -\usepackage{graphics} -\usepackage{amsmath} -\usepackage{epsfig} - -\pagestyle{empty} - -\renewcommand\url{\begingroup \def\UrlLeft{<}\def\UrlRight{>}\urlstyle{tt}\Url} -\newcommand\emailaddr{\begingroup \def\UrlLeft{<}\def\UrlRight{>}\urlstyle{tt}\Url} - -\newcommand{\workingnote}[1]{} % The version that hides the note. -%\newcommand{\workingnote}[1]{(**#1)} % The version that makes the note visible. - -% If an URL ends up with '%'s in it, that's because the line *in the .bib/.tex -% file* is too long, so break it there (it doesn't matter if the next line is -% indented with spaces). -DH - -%\newif\ifpdf -%\ifx\pdfoutput\undefined -% \pdffalse -%\else -% \pdfoutput=1 -% \pdftrue -%\fi - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} - -% Cut down on whitespace above and below figures displayed at head/foot of -% page. -\setlength{\textfloatsep}{3mm} -% Cut down on whitespace above and below figures displayed in middle of page -\setlength{\intextsep}{3mm} - -\begin{document} - -%% Use dvipdfm instead. --DH -%\ifpdf -% \pdfcompresslevel=9 -% \pdfpagewidth=\the\paperwidth -% \pdfpageheight=\the\paperheight -%\fi - -\title{Tor: The Second-Generation Onion Router} %\\DRAFT VERSION} -% Putting the 'Private' back in 'Virtual Private Network' - -\author{Roger Dingledine \\ The Free Haven Project \\ arma@freehaven.net \and -Nick Mathewson \\ The Free Haven Project \\ nickm@freehaven.net \and -Paul Syverson \\ Naval Research Lab \\ syverson@itd.nrl.navy.mil} - -\maketitle -\thispagestyle{empty} - -\begin{abstract} -We present Tor, a circuit-based low-latency anonymous communication -service. This second-generation Onion Routing system addresses limitations -in the original design by adding perfect forward secrecy, congestion -control, directory servers, integrity checking, configurable exit policies, -and a practical design for location-hidden services via rendezvous -points. Tor works on the real-world -Internet, requires no special privileges or kernel modifications, requires -little synchronization or coordination between nodes, and provides a -reasonable tradeoff between anonymity, usability, and efficiency. -We briefly describe our experiences with an international network of -more than 30 nodes. % that has been running for several months. -We close with a list of open problems in anonymous communication. -\end{abstract} - -%\begin{center} -%\textbf{Keywords:} anonymity, peer-to-peer, remailer, nymserver, reply block -%\end{center} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\section{Overview} -\label{sec:intro} - -Onion Routing is a distributed overlay network designed to anonymize -TCP-based applications like web browsing, secure shell, -and instant messaging. Clients choose a path through the network and -build a \emph{circuit}, in which each node (or ``onion router'' or ``OR'') -in the path knows its predecessor and successor, but no other nodes in -the circuit. Traffic flows down the circuit in fixed-size -\emph{cells}, which are unwrapped by a symmetric key at each node -(like the layers of an onion) and relayed downstream. The -Onion Routing project published several design and analysis -papers \cite{or-ih96,or-jsac98,or-discex00,or-pet00}. While a wide area Onion -Routing network was deployed briefly, the only long-running -public implementation was a fragile -proof-of-concept that ran on a single machine. Even this simple deployment -processed connections from over sixty thousand distinct IP addresses from -all over the world at a rate of about fifty thousand per day. -But many critical design and deployment issues were never -resolved, and the design has not been updated in years. Here -we describe Tor, a protocol for asynchronous, loosely federated onion -routers that provides the following improvements over the old Onion -Routing design: - -\textbf{Perfect forward secrecy:} In the original Onion Routing design, -a single hostile node could record traffic and -later compromise successive nodes in the circuit and force them -to decrypt it. Rather than using a single multiply encrypted data -structure (an \emph{onion}) to lay each circuit, -Tor now uses an incremental or \emph{telescoping} path-building design, -where the initiator negotiates session keys with each successive hop in -the circuit. Once these keys are deleted, subsequently compromised nodes -cannot decrypt old traffic. As a side benefit, onion replay detection -is no longer necessary, and the process of building circuits is more -reliable, since the initiator knows when a hop fails and can then try -extending to a new node. - -\textbf{Separation of ``protocol cleaning'' from anonymity:} -Onion Routing originally required a separate ``application -proxy'' for each supported application protocol---most of which were -never written, so many applications were never supported. Tor uses the -standard and near-ubiquitous SOCKS~\cite{socks4} proxy interface, allowing -us to support most TCP-based programs without modification. Tor now -relies on the filtering features of privacy-enhancing -application-level proxies such as Privoxy~\cite{privoxy}, without trying -to duplicate those features itself. - -\textbf{No mixing, padding, or traffic shaping (yet):} Onion -Routing originally called for batching and reordering cells as they arrived, -assumed padding between ORs, and in -later designs added padding between onion proxies (users) and -ORs~\cite{or-ih96,or-jsac98}. Tradeoffs between padding protection -and cost were discussed, and \emph{traffic shaping} algorithms were -theorized~\cite{or-pet00} to provide good security without expensive -padding, but no concrete padding scheme was suggested. -Recent research~\cite{econymics} -and deployment experience~\cite{freedom21-security} suggest that this -level of resource use is not practical or economical; and even full -link padding is still vulnerable~\cite{defensive-dropping}. Thus, -until we have a proven and convenient design for traffic shaping or -low-latency mixing that improves anonymity against a realistic -adversary, we leave these strategies out. - -\textbf{Many TCP streams can share one circuit:} Onion Routing originally -built a separate circuit for each -application-level request, but this required -multiple public key operations for every request, and also presented -a threat to anonymity from building so many circuits; see -Section~\ref{sec:maintaining-anonymity}. Tor multiplexes multiple TCP -streams along each circuit to improve efficiency and anonymity. - -\textbf{Leaky-pipe circuit topology:} Through in-band signaling -within the circuit, Tor initiators can direct traffic to nodes partway -down the circuit. This novel approach -allows traffic to exit the circuit from the middle---possibly -frustrating traffic shape and volume attacks based on observing the end -of the circuit. (It also allows for long-range padding if -future research shows this to be worthwhile.) - -\textbf{Congestion control:} Earlier anonymity designs do not -address traffic bottlenecks. Unfortunately, typical approaches to -load balancing and flow control in overlay networks involve inter-node -control communication and global views of traffic. Tor's decentralized -congestion control uses end-to-end acks to maintain anonymity -while allowing nodes at the edges of the network to detect congestion -or flooding and send less data until the congestion subsides. - -\textbf{Directory servers:} The earlier Onion Routing design -planned to flood state information through the network---an approach -that can be unreliable and complex. % open to partitioning attacks. -Tor takes a simplified view toward distributing this -information. Certain more trusted nodes act as \emph{directory -servers}: they provide signed directories describing known -routers and their current state. Users periodically download them -via HTTP. - -\textbf{Variable exit policies:} Tor provides a consistent mechanism -for each node to advertise a policy describing the hosts -and ports to which it will connect. These exit policies are critical -in a volunteer-based distributed infrastructure, because each operator -is comfortable with allowing different types of traffic to exit -from his node. - -\textbf{End-to-end integrity checking:} The original Onion Routing -design did no integrity checking on data. Any node on the -circuit could change the contents of data cells as they passed by---for -example, to alter a connection request so it would connect -to a different webserver, or to `tag' encrypted traffic and look for -corresponding corrupted traffic at the network edges~\cite{minion-design}. -Tor hampers these attacks by verifying data integrity before it leaves -the network. - -%\textbf{Improved robustness to failed nodes:} A failed node -%in the old design meant that circuit building failed, but thanks to -%Tor's step-by-step circuit building, users notice failed nodes -%while building circuits and route around them. Additionally, liveness -%information from directories allows users to avoid unreliable nodes in -%the first place. -%% Can't really claim this, now that we've found so many variants of -%% attack on partial-circuit-building. -RD - -\textbf{Rendezvous points and hidden services:} -Tor provides an integrated mechanism for responder anonymity via -location-protected servers. Previous Onion Routing designs included -long-lived ``reply onions'' that could be used to build circuits -to a hidden server, but these reply onions did not provide forward -security, and became useless if any node in the path went down -or rotated its keys. In Tor, clients negotiate {\it rendezvous points} -to connect with hidden servers; reply onions are no longer required. - -Unlike Freedom~\cite{freedom2-arch}, Tor does not require OS kernel -patches or network stack support. This prevents us from anonymizing -non-TCP protocols, but has greatly helped our portability and -deployability. - -%Unlike Freedom~\cite{freedom2-arch}, Tor only anonymizes -%TCP-based protocols---not requiring patches (or built-in support) in an -%operating system's network stack has been valuable to Tor's -%portability and deployability. - -We have implemented all of the above features, including rendezvous -points. Our source code is -available under a free license, and Tor -%, as far as we know, is unencumbered by patents. -is not covered by the patent that affected distribution and use of -earlier versions of Onion Routing. -We have deployed a wide-area alpha network -to test the design, to get more experience with usability -and users, and to provide a research platform for experimentation. -As of this writing, the network stands at 32 nodes %in thirteen -%distinct administrative domains -spread over two continents. - -We review previous work in Section~\ref{sec:related-work}, describe -our goals and assumptions in Section~\ref{sec:assumptions}, -and then address the above list of improvements in -Sections~\ref{sec:design},~\ref{sec:rendezvous}, and~\ref{sec:other-design}. -We summarize -in Section~\ref{sec:attacks} how our design stands up to -known attacks, and talk about our early deployment experiences in -Section~\ref{sec:in-the-wild}. We conclude with a list of open problems in -Section~\ref{sec:maintaining-anonymity} and future work for the Onion -Routing project in Section~\ref{sec:conclusion}. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\section{Related work} -\label{sec:related-work} - -Modern anonymity systems date to Chaum's {\bf Mix-Net} -design~\cite{chaum-mix}. Chaum -proposed hiding the correspondence between sender and recipient by -wrapping messages in layers of public-key cryptography, and relaying them -through a path composed of ``mixes.'' Each mix in turn -decrypts, delays, and re-orders messages before relaying them -onward. -%toward their destinations. - -Subsequent relay-based anonymity designs have diverged in two -main directions. Systems like {\bf Babel}~\cite{babel}, -{\bf Mixmaster}~\cite{mixmaster-spec}, -and {\bf Mixminion}~\cite{minion-design} have tried -to maximize anonymity at the cost of introducing comparatively large and -variable latencies. Because of this decision, these \emph{high-latency} -networks resist strong global adversaries, -but introduce too much lag for interactive tasks like web browsing, -Internet chat, or SSH connections. - -Tor belongs to the second category: \emph{low-latency} designs that -try to anonymize interactive network traffic. These systems handle -a variety of bidirectional protocols. They also provide more convenient -mail delivery than the high-latency anonymous email -networks, because the remote mail server provides explicit and timely -delivery confirmation. But because these designs typically -involve many packets that must be delivered quickly, it is -difficult for them to prevent an attacker who can eavesdrop both ends of the -communication from correlating the timing and volume -of traffic entering the anonymity network with traffic leaving it~\cite{SS03}. -These -protocols are similarly vulnerable to an active adversary who introduces -timing patterns into traffic entering the network and looks -for correlated patterns among exiting traffic. -Although some work has been done to frustrate these attacks, most designs -protect primarily against traffic analysis rather than traffic -confirmation (see Section~\ref{subsec:threat-model}). - -The simplest low-latency designs are single-hop proxies such as the -{\bf Anonymizer}~\cite{anonymizer}: a single trusted server strips the -data's origin before relaying it. These designs are easy to -analyze, but users must trust the anonymizing proxy. -Concentrating the traffic to this single point increases the anonymity set -(the people a given user is hiding among), but it is vulnerable if the -adversary can observe all traffic entering and leaving the proxy. - -More complex are distributed-trust, circuit-based anonymizing systems. -In these designs, a user establishes one or more medium-term bidirectional -end-to-end circuits, and tunnels data in fixed-size cells. -Establishing circuits is computationally expensive and typically -requires public-key -cryptography, whereas relaying cells is comparatively inexpensive and -typically requires only symmetric encryption. -Because a circuit crosses several servers, and each server only knows -the adjacent servers in the circuit, no single server can link a -user to her communication partners. - -The {\bf Java Anon Proxy} (also known as JAP or Web MIXes) uses fixed shared -routes known as \emph{cascades}. As with a single-hop proxy, this -approach aggregates users into larger anonymity sets, but again an -attacker only needs to observe both ends of the cascade to bridge all -the system's traffic. The Java Anon Proxy's design -calls for padding between end users and the head of the -cascade~\cite{web-mix}. However, it is not demonstrated whether the current -implementation's padding policy improves anonymity. - -{\bf PipeNet}~\cite{back01, pipenet}, another low-latency design proposed -around the same time as Onion Routing, gave -stronger anonymity but allowed a single user to shut -down the network by not sending. Systems like {\bf ISDN -mixes}~\cite{isdn-mixes} were designed for other environments with -different assumptions. -%XXX please can we fix this sentence to something less demeaning - -In P2P designs like {\bf Tarzan}~\cite{tarzan:ccs02} and -{\bf MorphMix}~\cite{morphmix:fc04}, all participants both generate -traffic and relay traffic for others. These systems aim to conceal -whether a given peer originated a request -or just relayed it from another peer. While Tarzan and MorphMix use -layered encryption as above, {\bf Crowds}~\cite{crowds-tissec} simply assumes -an adversary who cannot observe the initiator: it uses no public-key -encryption, so any node on a circuit can read users' traffic. - -{\bf Hordes}~\cite{hordes-jcs} is based on Crowds but also uses multicast -responses to hide the initiator. {\bf Herbivore}~\cite{herbivore} and -$\mbox{\bf P}^{\mathbf 5}$~\cite{p5} go even further, requiring broadcast. -These systems are designed primarily for communication among peers, -although Herbivore users can make external connections by -requesting a peer to serve as a proxy. - -Systems like {\bf Freedom} and the original Onion Routing build circuits -all at once, using a layered ``onion'' of public-key encrypted messages, -each layer of which provides session keys and the address of the -next server in the circuit. Tor as described herein, Tarzan, MorphMix, -{\bf Cebolla}~\cite{cebolla}, and Rennhard's {\bf Anonymity Network}~\cite{anonnet} -build circuits -in stages, extending them one hop at a time. -Section~\ref{subsubsec:constructing-a-circuit} describes how this -approach enables perfect forward secrecy. - -Circuit-based designs must choose which protocol layer -to anonymize. They may intercept IP packets directly, and -relay them whole (stripping the source address) along the -circuit~\cite{freedom2-arch,tarzan:ccs02}. Like -Tor, they may accept TCP streams and relay the data in those streams, -ignoring the breakdown of that data into TCP -segments~\cite{morphmix:fc04,anonnet}. Finally, like Crowds, they may accept -application-level protocols such as HTTP and relay the application -requests themselves. -Making this protocol-layer decision requires a compromise between flexibility -and anonymity. For example, a system that understands HTTP -can strip -identifying information from requests, can take advantage of caching -to limit the number of requests that leave the network, and can batch -or encode requests to minimize the number of connections. -On the other hand, an IP-level anonymizer can handle nearly any protocol, -even ones unforeseen by its designers (though these systems require -kernel-level modifications to some operating systems, and so are more -complex and less portable). TCP-level anonymity networks like Tor present -a middle approach: they are application neutral (so long as the -application supports, or can be tunneled across, TCP), but by treating -application connections as data streams rather than raw TCP packets, -they avoid the inefficiencies of tunneling TCP over -TCP. - -Distributed-trust anonymizing systems need to prevent attackers from -adding too many servers and thus compromising user paths. -Tor relies on a small set of well-known directory servers, run by -independent parties, to decide which nodes can -join. Tarzan and MorphMix allow unknown users to run servers, and use -a limited resource (like IP addresses) to prevent an attacker from -controlling too much of the network. Crowds suggests requiring -written, notarized requests from potential crowd members. - -Anonymous communication is essential for censorship-resistant -systems like Eternity~\cite{eternity}, Free~Haven~\cite{freehaven-berk}, -Publius~\cite{publius}, and Tangler~\cite{tangler}. Tor's rendezvous -points enable connections between mutually anonymous entities; they -are a building block for location-hidden servers, which are needed by -Eternity and Free~Haven. - -% didn't include rewebbers. No clear place to put them, so I'll leave -% them out for now. -RD - -\section{Design goals and assumptions} -\label{sec:assumptions} - -\noindent{\large\bf Goals}\\ -Like other low-latency anonymity designs, Tor seeks to frustrate -attackers from linking communication partners, or from linking -multiple communications to or from a single user. Within this -main goal, however, several considerations have directed -Tor's evolution. - -\textbf{Deployability:} The design must be deployed and used in the -real world. Thus it -must not be expensive to run (for example, by requiring more bandwidth -than volunteers are willing to provide); must not place a heavy -liability burden on operators (for example, by allowing attackers to -implicate onion routers in illegal activities); and must not be -difficult or expensive to implement (for example, by requiring kernel -patches, or separate proxies for every protocol). We also cannot -require non-anonymous parties (such as websites) -to run our software. (Our rendezvous point design does not meet -this goal for non-anonymous users talking to hidden servers, -however; see Section~\ref{sec:rendezvous}.) - -\textbf{Usability:} A hard-to-use system has fewer users---and because -anonymity systems hide users among users, a system with fewer users -provides less anonymity. Usability is thus not only a convenience: -it is a security requirement~\cite{econymics,back01}. Tor should -therefore not -require modifying familiar applications; should not introduce prohibitive -delays; -and should require as few configuration decisions -as possible. Finally, Tor should be easily implementable on all common -platforms; we cannot require users to change their operating system -to be anonymous. (Tor currently runs on Win32, Linux, -Solaris, BSD-style Unix, MacOS X, and probably others.) - -\textbf{Flexibility:} The protocol must be flexible and well-specified, -so Tor can serve as a test-bed for future research. -Many of the open problems in low-latency anonymity -networks, such as generating dummy traffic or preventing Sybil -attacks~\cite{sybil}, may be solvable independently from the issues -solved by -Tor. Hopefully future systems will not need to reinvent Tor's design. -%(But note that while a flexible design benefits researchers, -%there is a danger that differing choices of extensions will make users -%distinguishable. Experiments should be run on a separate network.) - -\textbf{Simple design:} The protocol's design and security -parameters must be well-understood. Additional features impose implementation -and complexity costs; adding unproven techniques to the design threatens -deployability, readability, and ease of security analysis. Tor aims to -deploy a simple and stable system that integrates the best accepted -approaches to protecting anonymity.\\ - -\noindent{\large\bf Non-goals}\label{subsec:non-goals}\\ -In favoring simple, deployable designs, we have explicitly deferred -several possible goals, either because they are solved elsewhere, or because -they are not yet solved. - -\textbf{Not peer-to-peer:} Tarzan and MorphMix aim to scale to completely -decentralized peer-to-peer environments with thousands of short-lived -servers, many of which may be controlled by an adversary. This approach -is appealing, but still has many open -problems~\cite{tarzan:ccs02,morphmix:fc04}. - -\textbf{Not secure against end-to-end attacks:} Tor does not claim -to completely solve end-to-end timing or intersection -attacks. Some approaches, such as having users run their own onion routers, -may help; -see Section~\ref{sec:maintaining-anonymity} for more discussion. - -\textbf{No protocol normalization:} Tor does not provide \emph{protocol -normalization} like Privoxy or the Anonymizer. If senders want anonymity from -responders while using complex and variable -protocols like HTTP, Tor must be layered with a filtering proxy such -as Privoxy to hide differences between clients, and expunge protocol -features that leak identity. -Note that by this separation Tor can also provide services that -are anonymous to the network yet authenticated to the responder, like -SSH. Similarly, Tor does not integrate -tunneling for non-stream-based protocols like UDP; this must be -provided by an external service if appropriate. - -\textbf{Not steganographic:} Tor does not try to conceal who is connected -to the network. - -\subsection{Threat Model} -\label{subsec:threat-model} - -A global passive adversary is the most commonly assumed threat when -analyzing theoretical anonymity designs. But like all practical -low-latency systems, Tor does not protect against such a strong -adversary. Instead, we assume an adversary who can observe some fraction -of network traffic; who can generate, modify, delete, or delay -traffic; who can operate onion routers of his own; and who can -compromise some fraction of the onion routers. - -In low-latency anonymity systems that use layered encryption, the -adversary's typical goal is to observe both the initiator and the -responder. By observing both ends, passive attackers can confirm a -suspicion that Alice is -talking to Bob if the timing and volume patterns of the traffic on the -connection are distinct enough; active attackers can induce timing -signatures on the traffic to force distinct patterns. Rather -than focusing on these \emph{traffic confirmation} attacks, -we aim to prevent \emph{traffic -analysis} attacks, where the adversary uses traffic patterns to learn -which points in the network he should attack. - -Our adversary might try to link an initiator Alice with her -communication partners, or try to build a profile of Alice's -behavior. He might mount passive attacks by observing the network edges -and correlating traffic entering and leaving the network---by -relationships in packet timing, volume, or externally visible -user-selected -options. The adversary can also mount active attacks by compromising -routers or keys; by replaying traffic; by selectively denying service -to trustworthy routers to move users to -compromised routers, or denying service to users to see if traffic -elsewhere in the -network stops; or by introducing patterns into traffic that can later be -detected. The adversary might subvert the directory servers to give users -differing views of network state. Additionally, he can try to decrease -the network's reliability by attacking nodes or by performing antisocial -activities from reliable nodes and trying to get them taken down---making -the network unreliable flushes users to other less anonymous -systems, where they may be easier to attack. We summarize -in Section~\ref{sec:attacks} how well the Tor design defends against -each of these attacks. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\section{The Tor Design} -\label{sec:design} - -The Tor network is an overlay network; each onion router (OR) -runs as a normal -user-level process without any special privileges. -Each onion router maintains a TLS~\cite{TLS} -connection to every other onion router. -%(We discuss alternatives to this clique-topology assumption in -%Section~\ref{sec:maintaining-anonymity}.) -% A subset of the ORs also act as -%directory servers, tracking which routers are in the network; -%see Section~\ref{subsec:dirservers} for directory server details. -Each user -runs local software called an onion proxy (OP) to fetch directories, -establish circuits across the network, -and handle connections from user applications. These onion proxies accept -TCP streams and multiplex them across the circuits. The onion -router on the other side -of the circuit connects to the requested destinations -and relays data. - -Each onion router maintains a long-term identity key and a short-term -onion key. The identity -key is used to sign TLS certificates, to sign the OR's \emph{router -descriptor} (a summary of its keys, address, bandwidth, exit policy, -and so on), and (by directory servers) to sign directories. %Changing -%the identity key of a router is considered equivalent to creating a -%new router. -The onion key is used to decrypt requests -from users to set up a circuit and negotiate ephemeral keys. -The TLS protocol also establishes a short-term link key when communicating -between ORs. Short-term keys are rotated periodically and -independently, to limit the impact of key compromise. - -Section~\ref{subsec:cells} presents the fixed-size -\emph{cells} that are the unit of communication in Tor. We describe -in Section~\ref{subsec:circuits} how circuits are -built, extended, truncated, and destroyed. Section~\ref{subsec:tcp} -describes how TCP streams are routed through the network. We address -integrity checking in Section~\ref{subsec:integrity-checking}, -and resource limiting in Section~\ref{subsec:rate-limit}. -Finally, -Section~\ref{subsec:congestion} talks about congestion control and -fairness issues. - -\subsection{Cells} -\label{subsec:cells} - -Onion routers communicate with one another, and with users' OPs, via -TLS connections with ephemeral keys. Using TLS conceals the data on -the connection with perfect forward secrecy, and prevents an attacker -from modifying data on the wire or impersonating an OR. - -Traffic passes along these connections in fixed-size cells. Each cell -is 512 bytes, %(but see Section~\ref{sec:conclusion} for a discussion of -%allowing large cells and small cells on the same network), -and consists of a header and a payload. The header includes a circuit -identifier (circID) that specifies which circuit the cell refers to -(many circuits can be multiplexed over the single TLS connection), and -a command to describe what to do with the cell's payload. (Circuit -identifiers are connection-specific: each circuit has a different -circID on each OP/OR or OR/OR connection it traverses.) -Based on their command, cells are either \emph{control} cells, which are -always interpreted by the node that receives them, or \emph{relay} cells, -which carry end-to-end stream data. The control cell commands are: -\emph{padding} (currently used for keepalive, but also usable for link -padding); \emph{create} or \emph{created} (used to set up a new circuit); -and \emph{destroy} (to tear down a circuit). - -Relay cells have an additional header (the relay header) at the front -of the payload, containing a streamID (stream identifier: many streams can -be multiplexed over a circuit); an end-to-end checksum for integrity -checking; the length of the relay payload; and a relay command. -The entire contents of the relay header and the relay cell payload -are encrypted or decrypted together as the relay cell moves along the -circuit, using the 128-bit AES cipher in counter mode to generate a -cipher stream. The relay commands are: \emph{relay -data} (for data flowing down the stream), \emph{relay begin} (to open a -stream), \emph{relay end} (to close a stream cleanly), \emph{relay -teardown} (to close a broken stream), \emph{relay connected} -(to notify the OP that a relay begin has succeeded), \emph{relay -extend} and \emph{relay extended} (to extend the circuit by a hop, -and to acknowledge), \emph{relay truncate} and \emph{relay truncated} -(to tear down only part of the circuit, and to acknowledge), \emph{relay -sendme} (used for congestion control), and \emph{relay drop} (used to -implement long-range dummies). -We give a visual overview of cell structure plus the details of relay -cell structure, and then describe each of these cell types and commands -in more detail below. - -%\begin{figure}[h] -%\unitlength=1cm -%\centering -%\begin{picture}(8.0,1.5) -%\put(4,.5){\makebox(0,0)[c]{\epsfig{file=cell-struct,width=7cm}}} -%\end{picture} -%\end{figure} - -\begin{figure}[h] -\centering -\mbox{\epsfig{figure=cell-struct,width=7cm}} -\end{figure} - -\subsection{Circuits and streams} -\label{subsec:circuits} - -Onion Routing originally built one circuit for each -TCP stream. Because building a circuit can take several tenths of a -second (due to public-key cryptography and network latency), -this design imposed high costs on applications like web browsing that -open many TCP streams. - -In Tor, each circuit can be shared by many TCP streams. To avoid -delays, users construct circuits preemptively. To limit linkability -among their streams, users' OPs build a new circuit -periodically if the previous ones have been used, -and expire old used circuits that no longer have any open streams. -OPs consider rotating to a new circuit once a minute: thus -even heavy users spend negligible time -building circuits, but a limited number of requests can be linked -to each other through a given exit node. Also, because circuits are built -in the background, OPs can recover from failed circuit creation -without harming user experience.\\ - -\begin{figure}[h] -\centering -\mbox{\epsfig{figure=interaction,width=8.75cm}} -\caption{Alice builds a two-hop circuit and begins fetching a web page.} -\label{fig:interaction} -\end{figure} - -\noindent{\large\bf Constructing a circuit}\label{subsubsec:constructing-a-circuit}\\ -%\subsubsection{Constructing a circuit} -A user's OP constructs circuits incrementally, negotiating a -symmetric key with each OR on the circuit, one hop at a time. To begin -creating a new circuit, the OP (call her Alice) sends a -\emph{create} cell to the first node in her chosen path (call him Bob). -(She chooses a new -circID $C_{AB}$ not currently used on the connection from her to Bob.) -The \emph{create} cell's -payload contains the first half of the Diffie-Hellman handshake -($g^x$), encrypted to the onion key of Bob. Bob -responds with a \emph{created} cell containing $g^y$ -along with a hash of the negotiated key $K=g^{xy}$. - -Once the circuit has been established, Alice and Bob can send one -another relay cells encrypted with the negotiated -key.\footnote{Actually, the negotiated key is used to derive two - symmetric keys: one for each direction.} More detail is given in -the next section. - -To extend the circuit further, Alice sends a \emph{relay extend} cell -to Bob, specifying the address of the next OR (call her Carol), and -an encrypted $g^{x_2}$ for her. Bob copies the half-handshake into a -\emph{create} cell, and passes it to Carol to extend the circuit. -(Bob chooses a new circID $C_{BC}$ not currently used on the connection -between him and Carol. Alice never needs to know this circID; only Bob -associates $C_{AB}$ on his connection with Alice to $C_{BC}$ on -his connection with Carol.) -When Carol responds with a \emph{created} cell, Bob wraps the payload -into a \emph{relay extended} cell and passes it back to Alice. Now -the circuit is extended to Carol, and Alice and Carol share a common key -$K_2 = g^{x_2 y_2}$. - -To extend the circuit to a third node or beyond, Alice -proceeds as above, always telling the last node in the circuit to -extend one hop further. - -This circuit-level handshake protocol achieves unilateral entity -authentication (Alice knows she's handshaking with the OR, but -the OR doesn't care who is opening the circuit---Alice uses no public key -and remains anonymous) and unilateral key authentication -(Alice and the OR agree on a key, and Alice knows only the OR learns -it). It also achieves forward -secrecy and key freshness. More formally, the protocol is as follows -(where $E_{PK_{Bob}}(\cdot)$ is encryption with Bob's public key, -$H$ is a secure hash function, and $|$ is concatenation): -\begin{equation*} -\begin{aligned} -\mathrm{Alice} \rightarrow \mathrm{Bob}&: E_{PK_{Bob}}(g^x) \\ -\mathrm{Bob} \rightarrow \mathrm{Alice}&: g^y, H(K | \mathrm{``handshake"}) \\ -\end{aligned} -\end{equation*} - -\noindent In the second step, Bob proves that it was he who received $g^x$, -and who chose $y$. We use PK encryption in the first step -(rather than, say, using the first two steps of STS, which has a -signature in the second step) because a single cell is too small to -hold both a public key and a signature. Preliminary analysis with the -NRL protocol analyzer~\cite{meadows96} shows this protocol to be -secure (including perfect forward secrecy) under the -traditional Dolev-Yao model.\\ - -\noindent{\large\bf Relay cells}\\ -%\subsubsection{Relay cells} -% -Once Alice has established the circuit (so she shares keys with each -OR on the circuit), she can send relay cells. -%Recall that every relay cell has a streamID that indicates to which -%stream the cell belongs. %This streamID allows a relay cell to be -%addressed to any OR on the circuit. -Upon receiving a relay -cell, an OR looks up the corresponding circuit, and decrypts the relay -header and payload with the session key for that circuit. -If the cell is headed away from Alice the OR then checks whether the -decrypted cell has a valid digest (as an optimization, the first -two bytes of the integrity check are zero, so in most cases we can avoid -computing the hash). -%is recognized---either because it -%corresponds to an open stream at this OR for the given circuit, or because -%it is the control streamID (zero). -If valid, it accepts the relay cell and processes it as described -below. Otherwise, -the OR looks up the circID and OR for the -next step in the circuit, replaces the circID as appropriate, and -sends the decrypted relay cell to the next OR. (If the OR at the end -of the circuit receives an unrecognized relay cell, an error has -occurred, and the circuit is torn down.) - -OPs treat incoming relay cells similarly: they iteratively unwrap the -relay header and payload with the session keys shared with each -OR on the circuit, from the closest to farthest. -If at any stage the digest is valid, the cell must have -originated at the OR whose encryption has just been removed. - -To construct a relay cell addressed to a given OR, Alice assigns the -digest, and then iteratively -encrypts the cell payload (that is, the relay header and payload) with -the symmetric key of each hop up to that OR. Because the digest is -encrypted to a different value at each step, only at the targeted OR -will it have a meaningful value.\footnote{ - % Should we just say that 2^56 is itself negligible? - % Assuming 4-hop circuits with 10 streams per hop, there are 33 - % possible bad streamIDs before the last circuit. This still - % gives an error only once every 2 million terabytes (approx). -With 48 bits of digest per cell, the probability of an accidental -collision is far lower than the chance of hardware failure.} -This \emph{leaky pipe} circuit topology -allows Alice's streams to exit at different ORs on a single circuit. -Alice may choose different exit points because of their exit policies, -or to keep the ORs from knowing that two streams -originate from the same person. - -When an OR later replies to Alice with a relay cell, it -encrypts the cell's relay header and payload with the single key it -shares with Alice, and sends the cell back toward Alice along the -circuit. Subsequent ORs add further layers of encryption as they -relay the cell back to Alice. - -To tear down a circuit, Alice sends a \emph{destroy} control -cell. Each OR in the circuit receives the \emph{destroy} cell, closes -all streams on that circuit, and passes a new \emph{destroy} cell -forward. But just as circuits are built incrementally, they can also -be torn down incrementally: Alice can send a \emph{relay -truncate} cell to a single OR on a circuit. That OR then sends a -\emph{destroy} cell forward, and acknowledges with a -\emph{relay truncated} cell. Alice can then extend the circuit to -different nodes, without signaling to the intermediate nodes (or -a limited observer) that she has changed her circuit. -Similarly, if a node on the circuit goes down, the adjacent -node can send a \emph{relay truncated} cell back to Alice. Thus the -``break a node and see which circuits go down'' -attack~\cite{freedom21-security} is weakened. - -\subsection{Opening and closing streams} -\label{subsec:tcp} - -When Alice's application wants a TCP connection to a given -address and port, it asks the OP (via SOCKS) to make the -connection. The OP chooses the newest open circuit (or creates one if -needed), and chooses a suitable OR on that circuit to be the -exit node (usually the last node, but maybe others due to exit policy -conflicts; see Section~\ref{subsec:exitpolicies}.) The OP then opens -the stream by sending a \emph{relay begin} cell to the exit node, -using a new random streamID. Once the -exit node connects to the remote host, it responds -with a \emph{relay connected} cell. Upon receipt, the OP sends a -SOCKS reply to notify the application of its success. The OP -now accepts data from the application's TCP stream, packaging it into -\emph{relay data} cells and sending those cells along the circuit to -the chosen OR. - -There's a catch to using SOCKS, however---some applications pass the -alphanumeric hostname to the Tor client, while others resolve it into -an IP address first and then pass the IP address to the Tor client. If -the application does DNS resolution first, Alice thereby reveals her -destination to the remote DNS server, rather than sending the hostname -through the Tor network to be resolved at the far end. Common applications -like Mozilla and SSH have this flaw. - -With Mozilla, the flaw is easy to address: the filtering HTTP -proxy called Privoxy gives a hostname to the Tor client, so Alice's -computer never does DNS resolution. -But a portable general solution, such as is needed for -SSH, is -an open problem. Modifying or replacing the local nameserver -can be invasive, brittle, and unportable. Forcing the resolver -library to prefer TCP rather than UDP is hard, and also has -portability problems. Dynamically intercepting system calls to the -resolver library seems a promising direction. We could also provide -a tool similar to \emph{dig} to perform a private lookup through the -Tor network. Currently, we encourage the use of privacy-aware proxies -like Privoxy wherever possible. - -Closing a Tor stream is analogous to closing a TCP stream: it uses a -two-step handshake for normal operation, or a one-step handshake for -errors. If the stream closes abnormally, the adjacent node simply sends a -\emph{relay teardown} cell. If the stream closes normally, the node sends -a \emph{relay end} cell down the circuit, and the other side responds with -its own \emph{relay end} cell. Because -all relay cells use layered encryption, only the destination OR knows -that a given relay cell is a request to close a stream. This two-step -handshake allows Tor to support TCP-based applications that use half-closed -connections. -% such as broken HTTP clients that close their side of the -%stream after writing but are still willing to read. - -\subsection{Integrity checking on streams} -\label{subsec:integrity-checking} - -Because the old Onion Routing design used a stream cipher without integrity -checking, traffic was -vulnerable to a malleability attack: though the attacker could not -decrypt cells, any changes to encrypted data -would create corresponding changes to the data leaving the network. -This weakness allowed an adversary who could guess the encrypted content -to change a padding cell to a destroy -cell; change the destination address in a \emph{relay begin} cell to the -adversary's webserver; or change an FTP command from -{\tt dir} to {\tt rm~*}. (Even an external -adversary could do this, because the link encryption similarly used a -stream cipher.) - -Because Tor uses TLS on its links, external adversaries cannot modify -data. Addressing the insider malleability attack, however, is -more complex. - -We could do integrity checking of the relay cells at each hop, either -by including hashes or by using an authenticating cipher mode like -EAX~\cite{eax}, but there are some problems. First, these approaches -impose a message-expansion overhead at each hop, and so we would have to -either leak the path length or waste bytes by padding to a maximum -path length. Second, these solutions can only verify traffic coming -from Alice: ORs would not be able to produce suitable hashes for -the intermediate hops, since the ORs on a circuit do not know the -other ORs' session keys. Third, we have already accepted that our design -is vulnerable to end-to-end timing attacks; so tagging attacks performed -within the circuit provide no additional information to the attacker. - -Thus, we check integrity only at the edges of each stream. (Remember that -in our leaky-pipe circuit topology, a stream's edge could be any hop -in the circuit.) When Alice -negotiates a key with a new hop, they each initialize a SHA-1 -digest with a derivative of that key, -thus beginning with randomness that only the two of them know. -Then they each incrementally add to the SHA-1 digest the contents of -all relay cells they create, and include with each relay cell the -first four bytes of the current digest. Each also keeps a SHA-1 -digest of data received, to verify that the received hashes are correct. - -To be sure of removing or modifying a cell, the attacker must be able -to deduce the current digest state (which depends on all -traffic between Alice and Bob, starting with their negotiated key). -Attacks on SHA-1 where the adversary can incrementally add to a hash -to produce a new valid hash don't work, because all hashes are -end-to-end encrypted across the circuit. The computational overhead -of computing the digests is minimal compared to doing the AES -encryption performed at each hop of the circuit. We use only four -bytes per cell to minimize overhead; the chance that an adversary will -correctly guess a valid hash -%, plus the payload the current cell, -is -acceptably low, given that the OP or OR tear down the circuit if they -receive a bad hash. - -\subsection{Rate limiting and fairness} -\label{subsec:rate-limit} - -Volunteers are more willing to run services that can limit -their bandwidth usage. To accommodate them, Tor servers use a -token bucket approach~\cite{tannenbaum96} to -enforce a long-term average rate of incoming bytes, while still -permitting short-term bursts above the allowed bandwidth. -% Current bucket sizes are set to ten seconds' worth of traffic. - -%Further, we want to avoid starving any Tor streams. Entire circuits -%could starve if we read greedily from connections and one connection -%uses all the remaining bandwidth. We solve this by dividing the number -%of tokens in the bucket by the number of connections that want to read, -%and reading at most that number of bytes from each connection. We iterate -%this procedure until the number of tokens in the bucket is under some -%threshold (currently 10KB), at which point we greedily read from connections. - -Because the Tor protocol outputs about the same number of bytes as it -takes in, it is sufficient in practice to limit only incoming bytes. -With TCP streams, however, the correspondence is not one-to-one: -relaying a single incoming byte can require an entire 512-byte cell. -(We can't just wait for more bytes, because the local application may -be awaiting a reply.) Therefore, we treat this case as if the entire -cell size had been read, regardless of the cell's fullness. - -Further, inspired by Rennhard et al's design in~\cite{anonnet}, a -circuit's edges can heuristically distinguish interactive streams from bulk -streams by comparing the frequency with which they supply cells. We can -provide good latency for interactive streams by giving them preferential -service, while still giving good overall throughput to the bulk -streams. Such preferential treatment presents a possible end-to-end -attack, but an adversary observing both -ends of the stream can already learn this information through timing -attacks. - -\subsection{Congestion control} -\label{subsec:congestion} - -Even with bandwidth rate limiting, we still need to worry about -congestion, either accidental or intentional. If enough users choose the -same OR-to-OR connection for their circuits, that connection can become -saturated. For example, an attacker could send a large file -through the Tor network to a webserver he runs, and then -refuse to read any of the bytes at the webserver end of the -circuit. Without some congestion control mechanism, these bottlenecks -can propagate back through the entire network. We don't need to -reimplement full TCP windows (with sequence numbers, -the ability to drop cells when we're full and retransmit later, and so -on), -because TCP already guarantees in-order delivery of each -cell. -%But we need to investigate further the effects of the current -%parameters on throughput and latency, while also keeping privacy in mind; -%see Section~\ref{sec:maintaining-anonymity} for more discussion. -We describe our response below. - -\textbf{Circuit-level throttling:} -To control a circuit's bandwidth usage, each OR keeps track of two -windows. The \emph{packaging window} tracks how many relay data cells the OR is -allowed to package (from incoming TCP streams) for transmission back to the OP, -and the \emph{delivery window} tracks how many relay data cells it is willing -to deliver to TCP streams outside the network. Each window is initialized -(say, to 1000 data cells). When a data cell is packaged or delivered, -the appropriate window is decremented. When an OR has received enough -data cells (currently 100), it sends a \emph{relay sendme} cell towards the OP, -with streamID zero. When an OR receives a \emph{relay sendme} cell with -streamID zero, it increments its packaging window. Either of these cells -increments the corresponding window by 100. If the packaging window -reaches 0, the OR stops reading from TCP connections for all streams -on the corresponding circuit, and sends no more relay data cells until -receiving a \emph{relay sendme} cell. - -The OP behaves identically, except that it must track a packaging window -and a delivery window for every OR in the circuit. If a packaging window -reaches 0, it stops reading from streams destined for that OR. - -\textbf{Stream-level throttling}: -The stream-level congestion control mechanism is similar to the -circuit-level mechanism. ORs and OPs use \emph{relay sendme} cells -to implement end-to-end flow control for individual streams across -circuits. Each stream begins with a packaging window (currently 500 cells), -and increments the window by a fixed value (50) upon receiving a \emph{relay -sendme} cell. Rather than always returning a \emph{relay sendme} cell as soon -as enough cells have arrived, the stream-level congestion control also -has to check whether data has been successfully flushed onto the TCP -stream; it sends the \emph{relay sendme} cell only when the number of bytes pending -to be flushed is under some threshold (currently 10 cells' worth). - -%% Maybe omit this next paragraph. -NM -%Currently, non-data relay cells do not affect the windows. Thus we -%avoid potential deadlock issues, for example, arising because a stream -%can't send a \emph{relay sendme} cell when its packaging window is empty. - -These arbitrarily chosen parameters seem to give tolerable throughput -and delay; see Section~\ref{sec:in-the-wild}. - -\section{Rendezvous Points and hidden services} -\label{sec:rendezvous} - -Rendezvous points are a building block for \emph{location-hidden -services} (also known as \emph{responder anonymity}) in the Tor -network. Location-hidden services allow Bob to offer a TCP -service, such as a webserver, without revealing his IP address. -This type of anonymity protects against distributed DoS attacks: -attackers are forced to attack the onion routing network -because they do not know Bob's IP address. - -Our design for location-hidden servers has the following goals. -\textbf{Access-control:} Bob needs a way to filter incoming requests, -so an attacker cannot flood Bob simply by making many connections to him. -\textbf{Robustness:} Bob should be able to maintain a long-term pseudonymous -identity even in the presence of router failure. Bob's service must -not be tied to a single OR, and Bob must be able to migrate his service -across ORs. \textbf{Smear-resistance:} -A social attacker -should not be able to ``frame'' a rendezvous router by -offering an illegal or disreputable location-hidden service and -making observers believe the router created that service. -\textbf{Application-transparency:} Although we require users -to run special software to access location-hidden servers, we must not -require them to modify their applications. - -We provide location-hiding for Bob by allowing him to advertise -several onion routers (his \emph{introduction points}) as contact -points. He may do this on any robust efficient -key-value lookup system with authenticated updates, such as a -distributed hash table (DHT) like CFS~\cite{cfs:sosp01}.\footnote{ -Rather than rely on an external infrastructure, the Onion Routing network -can run the lookup service itself. Our current implementation provides a -simple lookup system on the -directory servers.} Alice, the client, chooses an OR as her -\emph{rendezvous point}. She connects to one of Bob's introduction -points, informs him of her rendezvous point, and then waits for him -to connect to the rendezvous point. This extra level of indirection -helps Bob's introduction points avoid problems associated with serving -unpopular files directly (for example, if Bob serves -material that the introduction point's community finds objectionable, -or if Bob's service tends to get attacked by network vandals). -The extra level of indirection also allows Bob to respond to some requests -and ignore others. - -\subsection{Rendezvous points in Tor} - -The following steps are -%We give an overview of the steps of a rendezvous. These are -performed on behalf of Alice and Bob by their local OPs; -application integration is described more fully below. - -\begin{tightlist} -\item Bob generates a long-term public key pair to identify his service. -\item Bob chooses some introduction points, and advertises them on - the lookup service, signing the advertisement with his public key. He - can add more later. -\item Bob builds a circuit to each of his introduction points, and tells - them to wait for requests. -\item Alice learns about Bob's service out of band (perhaps Bob told her, - or she found it on a website). She retrieves the details of Bob's - service from the lookup service. If Alice wants to access Bob's - service anonymously, she must connect to the lookup service via Tor. -\item Alice chooses an OR as the rendezvous point (RP) for her connection to - Bob's service. She builds a circuit to the RP, and gives it a - randomly chosen ``rendezvous cookie'' to recognize Bob. -\item Alice opens an anonymous stream to one of Bob's introduction - points, and gives it a message (encrypted with Bob's public key) - telling it about herself, - her RP and rendezvous cookie, and the - start of a DH - handshake. The introduction point sends the message to Bob. -\item If Bob wants to talk to Alice, he builds a circuit to Alice's - RP and sends the rendezvous cookie, the second half of the DH - handshake, and a hash of the session - key they now share. By the same argument as in - Section~\ref{subsubsec:constructing-a-circuit}, Alice knows she - shares the key only with Bob. -\item The RP connects Alice's circuit to Bob's. Note that RP can't - recognize Alice, Bob, or the data they transmit. -\item Alice sends a \emph{relay begin} cell along the circuit. It - arrives at Bob's OP, which connects to Bob's - webserver. -\item An anonymous stream has been established, and Alice and Bob - communicate as normal. -\end{tightlist} - -When establishing an introduction point, Bob provides the onion router -with the public key identifying his service. Bob signs his -messages, so others cannot usurp his introduction point -in the future. He uses the same public key to establish the other -introduction points for his service, and periodically refreshes his -entry in the lookup service. - -The message that Alice gives -the introduction point includes a hash of Bob's public key % to identify -%the service, along with -and an optional initial authorization token (the -introduction point can do prescreening, for example to block replays). Her -message to Bob may include an end-to-end authorization token so Bob -can choose whether to respond. -The authorization tokens can be used to provide selective access: -important users can get uninterrupted access. -%important users get tokens to ensure uninterrupted access. %to the -%service. -During normal situations, Bob's service might simply be offered -directly from mirrors, while Bob gives out tokens to high-priority users. If -the mirrors are knocked down, -%by distributed DoS attacks or even -%physical attack, -those users can switch to accessing Bob's service via -the Tor rendezvous system. - -Bob's introduction points are themselves subject to DoS---he must -open many introduction points or risk such an attack. -He can provide selected users with a current list or future schedule of -unadvertised introduction points; -this is most practical -if there is a stable and large group of introduction points -available. Bob could also give secret public keys -for consulting the lookup service. All of these approaches -limit exposure even when -some selected users collude in the DoS\@. - -\subsection{Integration with user applications} - -Bob configures his onion proxy to know the local IP address and port of his -service, a strategy for authorizing clients, and his public key. The onion -proxy anonymously publishes a signed statement of Bob's -public key, an expiration time, and -the current introduction points for his service onto the lookup service, -indexed -by the hash of his public key. Bob's webserver is unmodified, -and doesn't even know that it's hidden behind the Tor network. - -Alice's applications also work unchanged---her client interface -remains a SOCKS proxy. We encode all of the necessary information -into the fully qualified domain name (FQDN) Alice uses when establishing her -connection. Location-hidden services use a virtual top level domain -called {\tt .onion}: thus hostnames take the form {\tt x.y.onion} where -{\tt x} is the authorization cookie and {\tt y} encodes the hash of -the public key. Alice's onion proxy -examines addresses; if they're destined for a hidden server, it decodes -the key and starts the rendezvous as described above. - -\subsection{Previous rendezvous work} -%XXXX Should this get integrated into the earlier related work section? -NM - -Rendezvous points in low-latency anonymity systems were first -described for use in ISDN telephony~\cite{jerichow-jsac98,isdn-mixes}. -Later low-latency designs used rendezvous points for hiding location -of mobile phones and low-power location -trackers~\cite{federrath-ih96,reed-protocols97}. Rendezvous for -anonymizing low-latency -Internet connections was suggested in early Onion Routing -work~\cite{or-ih96}, but the first published design was by Ian -Goldberg~\cite{ian-thesis}. His design differs from -ours in three ways. First, Goldberg suggests that Alice should manually -hunt down a current location of the service via Gnutella; our approach -makes lookup transparent to the user, as well as faster and more robust. -Second, in Tor the client and server negotiate session keys -with Diffie-Hellman, so plaintext is not exposed even at the rendezvous -point. Third, -our design minimizes the exposure from running the -service, to encourage volunteers to offer introduction and rendezvous -services. Tor's introduction points do not output any bytes to the -clients; the rendezvous points don't know the client or the server, -and can't read the data being transmitted. The indirection scheme is -also designed to include authentication/authorization---if Alice doesn't -include the right cookie with her request for service, Bob need not even -acknowledge his existence. - -\section{Other design decisions} -\label{sec:other-design} - -\subsection{Denial of service} -\label{subsec:dos} - -Providing Tor as a public service creates many opportunities for -denial-of-service attacks against the network. While -flow control and rate limiting (discussed in -Section~\ref{subsec:congestion}) prevent users from consuming more -bandwidth than routers are willing to provide, opportunities remain for -users to -consume more network resources than their fair share, or to render the -network unusable for others. - -First of all, there are several CPU-consuming denial-of-service -attacks wherein an attacker can force an OR to perform expensive -cryptographic operations. For example, an attacker can -%\emph{create} cell full of junk bytes can force an OR to perform an RSA -%decrypt. -%Similarly, an attacker can -fake the start of a TLS handshake, forcing the OR to carry out its -(comparatively expensive) half of the handshake at no real computational -cost to the attacker. - -We have not yet implemented any defenses for these attacks, but several -approaches are possible. First, ORs can -require clients to solve a puzzle~\cite{puzzles-tls} while beginning new -TLS handshakes or accepting \emph{create} cells. So long as these -tokens are easy to verify and computationally expensive to produce, this -approach limits the attack multiplier. Additionally, ORs can limit -the rate at which they accept \emph{create} cells and TLS connections, -so that -the computational work of processing them does not drown out the -symmetric cryptography operations that keep cells -flowing. This rate limiting could, however, allow an attacker -to slow down other users when they build new circuits. - -% What about link-to-link rate limiting? - -Adversaries can also attack the Tor network's hosts and network -links. Disrupting a single circuit or link breaks all streams passing -along that part of the circuit. Users similarly lose service -when a router crashes or its operator restarts it. The current -Tor design treats such attacks as intermittent network failures, and -depends on users and applications to respond or recover as appropriate. A -future design could use an end-to-end TCP-like acknowledgment protocol, -so no streams are lost unless the entry or exit point is -disrupted. This solution would require more buffering at the network -edges, however, and the performance and anonymity implications from this -extra complexity still require investigation. - -\subsection{Exit policies and abuse} -\label{subsec:exitpolicies} - -% originally, we planned to put the "users only know the hostname, -% not the IP, but exit policies are by IP" problem here too. Not -% worth putting in the submission, but worth thinking about putting -% in sometime somehow. -RD - -Exit abuse is a serious barrier to wide-scale Tor deployment. Anonymity -presents would-be vandals and abusers with an opportunity to hide -the origins of their activities. Attackers can harm the Tor network by -implicating exit servers for their abuse. Also, applications that commonly -use IP-based authentication (such as institutional mail or webservers) -can be fooled by the fact that anonymous connections appear to originate -at the exit OR. - -We stress that Tor does not enable any new class of abuse. Spammers -and other attackers already have access to thousands of misconfigured -systems worldwide, and the Tor network is far from the easiest way -to launch attacks. -%Indeed, because of its limited -%anonymity, Tor is probably not a good way to commit crimes. -But because the -onion routers can be mistaken for the originators of the abuse, -and the volunteers who run them may not want to deal with the hassle of -explaining anonymity networks to irate administrators, we must block or limit -abuse through the Tor network. - -To mitigate abuse issues, each onion router's \emph{exit policy} -describes to which external addresses and ports the router will -connect. On one end of the spectrum are \emph{open exit} -nodes that will connect anywhere. On the other end are \emph{middleman} -nodes that only relay traffic to other Tor nodes, and \emph{private exit} -nodes that only connect to a local host or network. A private -exit can allow a client to connect to a given host or -network more securely---an external adversary cannot eavesdrop traffic -between the private exit and the final destination, and so is less sure of -Alice's destination and activities. Most onion routers in the current -network function as -\emph{restricted exits} that permit connections to the world at large, -but prevent access to certain abuse-prone addresses and services such -as SMTP. -The OR might also be able to authenticate clients to -prevent exit abuse without harming anonymity~\cite{or-discex00}. - -%The abuse issues on closed (e.g. military) networks are different -%from the abuse on open networks like the Internet. While these IP-based -%access controls are still commonplace on the Internet, on closed networks, -%nearly all participants will be honest, and end-to-end authentication -%can be assumed for important traffic. - -Many administrators use port restrictions to support only a -limited set of services, such as HTTP, SSH, or AIM. -This is not a complete solution, of course, since abuse opportunities for these -protocols are still well known. - -We have not yet encountered any abuse in the deployed network, but if -we do we should consider using proxies to clean traffic for certain -protocols as it leaves the network. For example, much abusive HTTP -behavior (such as exploiting buffer overflows or well-known script -vulnerabilities) can be detected in a straightforward manner. -Similarly, one could run automatic spam filtering software (such as -SpamAssassin) on email exiting the OR network. - -ORs may also rewrite exiting traffic to append -headers or other information indicating that the traffic has passed -through an anonymity service. This approach is commonly used -by email-only anonymity systems. ORs can also -run on servers with hostnames like {\tt anonymous} to further -alert abuse targets to the nature of the anonymous traffic. - -A mixture of open and restricted exit nodes allows the most -flexibility for volunteers running servers. But while having many -middleman nodes provides a large and robust network, -having only a few exit nodes reduces the number of points -an adversary needs to monitor for traffic analysis, and places a -greater burden on the exit nodes. This tension can be seen in the -Java Anon Proxy -cascade model, wherein only one node in each cascade needs to handle -abuse complaints---but an adversary only needs to observe the entry -and exit of a cascade to perform traffic analysis on all that -cascade's users. The hydra model (many entries, few exits) presents a -different compromise: only a few exit nodes are needed, but an -adversary needs to work harder to watch all the clients; see -Section~\ref{sec:conclusion}. - -Finally, we note that exit abuse must not be dismissed as a peripheral -issue: when a system's public image suffers, it can reduce the number -and diversity of that system's users, and thereby reduce the anonymity -of the system itself. Like usability, public perception is a -security parameter. Sadly, preventing abuse of open exit nodes is an -unsolved problem, and will probably remain an arms race for the -foreseeable future. The abuse problems faced by Princeton's CoDeeN -project~\cite{darkside} give us a glimpse of likely issues. - -\subsection{Directory Servers} -\label{subsec:dirservers} - -First-generation Onion Routing designs~\cite{freedom2-arch,or-jsac98} used -in-band network status updates: each router flooded a signed statement -to its neighbors, which propagated it onward. But anonymizing networks -have different security goals than typical link-state routing protocols. -For example, delays (accidental or intentional) -that can cause different parts of the network to have different views -of link-state and topology are not only inconvenient: they give -attackers an opportunity to exploit differences in client knowledge. -We also worry about attacks to deceive a -client about the router membership list, topology, or current network -state. Such \emph{partitioning attacks} on client knowledge help an -adversary to efficiently deploy resources -against a target~\cite{minion-design}. - -Tor uses a small group of redundant, well-known onion routers to -track changes in network topology and node state, including keys and -exit policies. Each such \emph{directory server} acts as an HTTP -server, so clients can fetch current network state -and router lists, and so other ORs can upload -state information. Onion routers periodically publish signed -statements of their state to each directory server. The directory servers -combine this information with their own views of network liveness, -and generate a signed description (a \emph{directory}) of the entire -network state. Client software is -pre-loaded with a list of the directory servers and their keys, -to bootstrap each client's view of the network. -% XXX this means that clients will be forced to upgrade as the -% XXX dirservers change or get compromised. argue that this is ok. - -When a directory server receives a signed statement for an OR, it -checks whether the OR's identity key is recognized. Directory -servers do not advertise unrecognized ORs---if they did, -an adversary could take over the network by creating many -servers~\cite{sybil}. Instead, new nodes must be approved by the -directory -server administrator before they are included. Mechanisms for automated -node approval are an area of active research, and are discussed more -in Section~\ref{sec:maintaining-anonymity}. - -Of course, a variety of attacks remain. An adversary who controls -a directory server can track clients by providing them different -information---perhaps by listing only nodes under its control, or by -informing only certain clients about a given node. Even an external -adversary can exploit differences in client knowledge: clients who use -a node listed on one directory server but not the others are vulnerable. - -Thus these directory servers must be synchronized and redundant, so -that they can agree on a common directory. Clients should only trust -this directory if it is signed by a threshold of the directory -servers. - -The directory servers in Tor are modeled after those in -Mixminion~\cite{minion-design}, but our situation is easier. First, -we make the -simplifying assumption that all participants agree on the set of -directory servers. Second, while Mixminion needs to predict node -behavior, Tor only needs a threshold consensus of the current -state of the network. Third, we assume that we can fall back to the -human administrators to discover and resolve problems when a consensus -directory cannot be reached. Since there are relatively few directory -servers (currently 3, but we expect as many as 9 as the network scales), -we can afford operations like broadcast to simplify the consensus-building -protocol. - -To avoid attacks where a router connects to all the directory servers -but refuses to relay traffic from other routers, the directory servers -must also build circuits and use them to anonymously test router -reliability~\cite{mix-acc}. Unfortunately, this defense is not yet -designed or -implemented. - -Using directory servers is simpler and more flexible than flooding. -Flooding is expensive, and complicates the analysis when we -start experimenting with non-clique network topologies. Signed -directories can be cached by other -onion routers, -so directory servers are not a performance -bottleneck when we have many users, and do not aid traffic analysis by -forcing clients to announce their existence to any -central point. - -\section{Attacks and Defenses} -\label{sec:attacks} - -Below we summarize a variety of attacks, and discuss how well our -design withstands them.\\ - -\noindent{\large\bf Passive attacks}\\ -\emph{Observing user traffic patterns.} Observing a user's connection -will not reveal her destination or data, but it will -reveal traffic patterns (both sent and received). Profiling via user -connection patterns requires further processing, because multiple -application streams may be operating simultaneously or in series over -a single circuit. - -\emph{Observing user content.} While content at the user end is encrypted, -connections to responders may not be (indeed, the responding website -itself may be hostile). While filtering content is not a primary goal -of Onion Routing, Tor can directly use Privoxy and related -filtering services to anonymize application data streams. - -\emph{Option distinguishability.} We allow clients to choose -configuration options. For example, clients concerned about request -linkability should rotate circuits more often than those concerned -about traceability. Allowing choice may attract users with different -%There is economic incentive to attract users by -%allowing this choice; -needs; but clients who are -in the minority may lose more anonymity by appearing distinct than they -gain by optimizing their behavior~\cite{econymics}. - -\emph{End-to-end timing correlation.} Tor only minimally hides -such correlations. An attacker watching patterns of -traffic at the initiator and the responder will be -able to confirm the correspondence with high probability. The -greatest protection currently available against such confirmation is to hide -the connection between the onion proxy and the first Tor node, -by running the OP on the Tor node or behind a firewall. This approach -requires an observer to separate traffic originating at the onion -router from traffic passing through it: a global observer can do this, -but it might be beyond a limited observer's capabilities. - -\emph{End-to-end size correlation.} Simple packet counting -will also be effective in confirming -endpoints of a stream. However, even without padding, we may have some -limited protection: the leaky pipe topology means different numbers -of packets may enter one end of a circuit than exit at the other. - -\emph{Website fingerprinting.} All the effective passive -attacks above are traffic confirmation attacks, -which puts them outside our design goals. There is also -a passive traffic analysis attack that is potentially effective. -Rather than searching exit connections for timing and volume -correlations, the adversary may build up a database of -``fingerprints'' containing file sizes and access patterns for -targeted websites. He can later confirm a user's connection to a given -site simply by consulting the database. This attack has -been shown to be effective against SafeWeb~\cite{hintz-pet02}. -It may be less effective against Tor, since -streams are multiplexed within the same circuit, and -fingerprinting will be limited to -the granularity of cells (currently 512 bytes). Additional -defenses could include -larger cell sizes, padding schemes to group websites -into large sets, and link -padding or long-range dummies.\footnote{Note that this fingerprinting -attack should not be confused with the much more complicated latency -attacks of~\cite{back01}, which require a fingerprint of the latencies -of all circuits through the network, combined with those from the -network edges to the target user and the responder website.}\\ - -\noindent{\large\bf Active attacks}\\ -\emph{Compromise keys.} An attacker who learns the TLS session key can -see control cells and encrypted relay cells on every circuit on that -connection; learning a circuit -session key lets him unwrap one layer of the encryption. An attacker -who learns an OR's TLS private key can impersonate that OR for the TLS -key's lifetime, but he must -also learn the onion key to decrypt \emph{create} cells (and because of -perfect forward secrecy, he cannot hijack already established circuits -without also compromising their session keys). Periodic key rotation -limits the window of opportunity for these attacks. On the other hand, -an attacker who learns a node's identity key can replace that node -indefinitely by sending new forged descriptors to the directory servers. - -\emph{Iterated compromise.} A roving adversary who can -compromise ORs (by system intrusion, legal coercion, or extralegal -coercion) could march down the circuit compromising the -nodes until he reaches the end. Unless the adversary can complete -this attack within the lifetime of the circuit, however, the ORs -will have discarded the necessary information before the attack can -be completed. (Thanks to the perfect forward secrecy of session -keys, the attacker cannot force nodes to decrypt recorded -traffic once the circuits have been closed.) Additionally, building -circuits that cross jurisdictions can make legal coercion -harder---this phenomenon is commonly called ``jurisdictional -arbitrage.'' The Java Anon Proxy project recently experienced the -need for this approach, when -a German court forced them to add a backdoor to -their nodes~\cite{jap-backdoor}. - -\emph{Run a recipient.} An adversary running a webserver -trivially learns the timing patterns of users connecting to it, and -can introduce arbitrary patterns in its responses. -End-to-end attacks become easier: if the adversary can induce -users to connect to his webserver (perhaps by advertising -content targeted to those users), he now holds one end of their -connection. There is also a danger that application -protocols and associated programs can be induced to reveal information -about the initiator. Tor depends on Privoxy and similar protocol cleaners -to solve this latter problem. - -\emph{Run an onion proxy.} It is expected that end users will -nearly always run their own local onion proxy. However, in some -settings, it may be necessary for the proxy to run -remotely---typically, in institutions that want -to monitor the activity of those connecting to the proxy. -Compromising an onion proxy compromises all future connections -through it. - -\emph{DoS non-observed nodes.} An observer who can only watch some -of the Tor network can increase the value of this traffic -by attacking non-observed nodes to shut them down, reduce -their reliability, or persuade users that they are not trustworthy. -The best defense here is robustness. - -\emph{Run a hostile OR.} In addition to being a local observer, -an isolated hostile node can create circuits through itself, or alter -traffic patterns to affect traffic at other nodes. Nonetheless, a hostile -node must be immediately adjacent to both endpoints to compromise the -anonymity of a circuit. If an adversary can -run multiple ORs, and can persuade the directory servers -that those ORs are trustworthy and independent, then occasionally -some user will choose one of those ORs for the start and another -as the end of a circuit. If an adversary -controls $m>1$ of $N$ nodes, he can correlate at most -$\left(\frac{m}{N}\right)^2$ of the traffic---although an -adversary -could still attract a disproportionately large amount of traffic -by running an OR with a permissive exit policy, or by -degrading the reliability of other routers. - -\emph{Introduce timing into messages.} This is simply a stronger -version of passive timing attacks already discussed earlier. - -\emph{Tagging attacks.} A hostile node could ``tag'' a -cell by altering it. If the -stream were, for example, an unencrypted request to a Web site, -the garbled content coming out at the appropriate time would confirm -the association. However, integrity checks on cells prevent -this attack. - -\emph{Replace contents of unauthenticated protocols.} When -relaying an unauthenticated protocol like HTTP, a hostile exit node -can impersonate the target server. Clients -should prefer protocols with end-to-end authentication. - -\emph{Replay attacks.} Some anonymity protocols are vulnerable -to replay attacks. Tor is not; replaying one side of a handshake -will result in a different negotiated session key, and so the rest -of the recorded session can't be used. - -\emph{Smear attacks.} An attacker could use the Tor network for -socially disapproved acts, to bring the -network into disrepute and get its operators to shut it down. -Exit policies reduce the possibilities for abuse, but -ultimately the network requires volunteers who can tolerate -some political heat. - -\emph{Distribute hostile code.} An attacker could trick users -into running subverted Tor software that did not, in fact, anonymize -their connections---or worse, could trick ORs into running weakened -software that provided users with less anonymity. We address this -problem (but do not solve it completely) by signing all Tor releases -with an official public key, and including an entry in the directory -that lists which versions are currently believed to be secure. To -prevent an attacker from subverting the official release itself -(through threats, bribery, or insider attacks), we provide all -releases in source code form, encourage source audits, and -frequently warn our users never to trust any software (even from -us) that comes without source.\\ - -\noindent{\large\bf Directory attacks}\\ -\emph{Destroy directory servers.} If a few directory -servers disappear, the others still decide on a valid -directory. So long as any directory servers remain in operation, -they will still broadcast their views of the network and generate a -consensus directory. (If more than half are destroyed, this -directory will not, however, have enough signatures for clients to -use it automatically; human intervention will be necessary for -clients to decide whether to trust the resulting directory.) - -\emph{Subvert a directory server.} By taking over a directory server, -an attacker can partially influence the final directory. Since ORs -are included or excluded by majority vote, the corrupt directory can -at worst cast a tie-breaking vote to decide whether to include -marginal ORs. It remains to be seen how often such marginal cases -occur in practice. - -\emph{Subvert a majority of directory servers.} An adversary who controls -more than half the directory servers can include as many compromised -ORs in the final directory as he wishes. We must ensure that directory -server operators are independent and attack-resistant. - -\emph{Encourage directory server dissent.} The directory -agreement protocol assumes that directory server operators agree on -the set of directory servers. An adversary who can persuade some -of the directory server operators to distrust one another could -split the quorum into mutually hostile camps, thus partitioning -users based on which directory they use. Tor does not address -this attack. - -\emph{Trick the directory servers into listing a hostile OR.} -Our threat model explicitly assumes directory server operators will -be able to filter out most hostile ORs. -% If this is not true, an -% attacker can flood the directory with compromised servers. - -\emph{Convince the directories that a malfunctioning OR is -working.} In the current Tor implementation, directory servers -assume that an OR is running correctly if they can start a TLS -connection to it. A hostile OR could easily subvert this test by -accepting TLS connections from ORs but ignoring all cells. Directory -servers must actively test ORs by building circuits and streams as -appropriate. The tradeoffs of a similar approach are discussed -in~\cite{mix-acc}.\\ - -\noindent{\large\bf Attacks against rendezvous points}\\ -\emph{Make many introduction requests.} An attacker could -try to deny Bob service by flooding his introduction points with -requests. Because the introduction points can block requests that -lack authorization tokens, however, Bob can restrict the volume of -requests he receives, or require a certain amount of computation for -every request he receives. - -\emph{Attack an introduction point.} An attacker could -disrupt a location-hidden service by disabling its introduction -points. But because a service's identity is attached to its public -key, the service can simply re-advertise -itself at a different introduction point. Advertisements can also be -done secretly so that only high-priority clients know the address of -Bob's introduction points or so that different clients know of different -introduction points. This forces the attacker to disable all possible -introduction points. - -\emph{Compromise an introduction point.} An attacker who controls -Bob's introduction point can flood Bob with -introduction requests, or prevent valid introduction requests from -reaching him. Bob can notice a flood, and close the circuit. To notice -blocking of valid requests, however, he should periodically test the -introduction point by sending rendezvous requests and making -sure he receives them. - -\emph{Compromise a rendezvous point.} A rendezvous -point is no more sensitive than any other OR on -a circuit, since all data passing through the rendezvous is encrypted -with a session key shared by Alice and Bob. - -\section{Early experiences: Tor in the Wild} -\label{sec:in-the-wild} - -As of mid-May 2004, the Tor network consists of 32 nodes -(24 in the US, 8 in Europe), and more are joining each week as the code -matures. (For comparison, the current remailer network -has about 40 nodes.) % We haven't asked PlanetLab to provide -%Tor nodes, since their AUP wouldn't allow exit nodes (see -%also~\cite{darkside}) and because we aim to build a long-term community of -%node operators and developers.} -Each node has at least a 768Kb/768Kb connection, and -many have 10Mb. The number of users varies (and of course, it's hard to -tell for sure), but we sometimes have several hundred users---administrators at -several companies have begun sending their entire departments' web -traffic through Tor, to block other divisions of -their company from reading their traffic. Tor users have reported using -the network for web browsing, FTP, IRC, AIM, Kazaa, SSH, and -recipient-anonymous email via rendezvous points. One user has anonymously -set up a Wiki as a hidden service, where other users anonymously publish -the addresses of their hidden services. - -Each Tor node currently processes roughly 800,000 relay -cells (a bit under half a gigabyte) per week. On average, about 80\% -of each 498-byte payload is full for cells going back to the client, -whereas about 40\% is full for cells coming from the client. (The difference -arises because most of the network's traffic is web browsing.) Interactive -traffic like SSH brings down the average a lot---once we have more -experience, and assuming we can resolve the anonymity issues, we may -partition traffic into two relay cell sizes: one to handle -bulk traffic and one for interactive traffic. - -Based in part on our restrictive default exit policy (we -reject SMTP requests) and our low profile, we have had no abuse -issues since the network was deployed in October -2003. Our slow growth rate gives us time to add features, -resolve bugs, and get a feel for what users actually want from an -anonymity system. Even though having more users would bolster our -anonymity sets, we are not eager to attract the Kazaa or warez -communities---we feel that we must build a reputation for privacy, human -rights, research, and other socially laudable activities. - -As for performance, profiling shows that Tor spends almost -all its CPU time in AES, which is fast. Current latency is attributable -to two factors. First, network latency is critical: we are -intentionally bouncing traffic around the world several times. Second, -our end-to-end congestion control algorithm focuses on protecting -volunteer servers from accidental DoS rather than on optimizing -performance. % Right now the first $500 \times 500\mbox{B}=250\mbox{KB}$ -%of the stream arrives -%quickly, and after that throughput depends on the rate that \emph{relay -%sendme} acknowledgments arrive. -To quantify these effects, we did some informal tests using a network of 4 -nodes on the same machine (a heavily loaded 1GHz Athlon). We downloaded a 60 -megabyte file from {\tt debian.org} every 30 minutes for 54 hours (108 sample -points). It arrived in about 300 seconds on average, compared to 210s for a -direct download. We ran a similar test on the production Tor network, -fetching the front page of {\tt cnn.com} (55 kilobytes): -% every 20 seconds for 8952 data points -while a direct -download consistently took about 0.3s, the performance through Tor varied. -Some downloads were as fast as 0.4s, with a median at 2.8s, and -90\% finishing within 5.3s. It seems that as the network expands, the chance -of building a slow circuit (one that includes a slow or heavily loaded node -or link) is increasing. On the other hand, as our users remain satisfied -with this increased latency, we can address our performance incrementally as we -proceed with development. %\footnote{For example, we have just begun pushing -%a pipelining patch to the production network that seems to decrease -%latency for medium-to-large files; we will present revised benchmarks -%as they become available.} - -%With the current network's topology and load, users can typically get 1-2 -%megabits sustained transfer rate, which is good enough for now. -%Indeed, the Tor -%design aims foremost to provide a security research platform; performance -%only needs to be sufficient to retain users~\cite{econymics,back01}. -%We can tweak the congestion control -%parameters to provide faster throughput at the cost of -%larger buffers at each node; adding the heuristics mentioned in -%Section~\ref{subsec:rate-limit} to favor low-volume -%streams may also help. More research remains to find the -%right balance. -% We should say _HOW MUCH_ latency there is in these cases. -NM - -%performs badly on lossy networks. may need airhook or something else as -%transport alternative? - -Although Tor's clique topology and full-visibility directories present -scaling problems, we still expect the network to support a few hundred -nodes and maybe 10,000 users before we're forced to become -more distributed. With luck, the experience we gain running the current -topology will help us choose among alternatives when the time comes. - -\section{Open Questions in Low-latency Anonymity} -\label{sec:maintaining-anonymity} - -In addition to the non-goals in -Section~\ref{subsec:non-goals}, many questions must be solved -before we can be confident of Tor's security. - -Many of these open issues are questions of balance. For example, -how often should users rotate to fresh circuits? Frequent rotation -is inefficient, expensive, and may lead to intersection attacks and -predecessor attacks~\cite{wright03}, but infrequent rotation makes the -user's traffic linkable. Besides opening fresh circuits, clients can -also exit from the middle of the circuit, -or truncate and re-extend the circuit. More analysis is -needed to determine the proper tradeoff. - -%% Duplicated by 'Better directory distribution' in section 9. -% -%A similar question surrounds timing of directory operations: how often -%should directories be updated? Clients that update infrequently receive -%an inaccurate picture of the network, but frequent updates can overload -%the directory servers. More generally, we must find more -%decentralized yet practical ways to distribute up-to-date snapshots of -%network status without introducing new attacks. - -How should we choose path lengths? If Alice always uses two hops, -then both ORs can be certain that by colluding they will learn about -Alice and Bob. In our current approach, Alice always chooses at least -three nodes unrelated to herself and her destination. -%% This point is subtle, but not IMO necessary. Anybody who thinks -%% about it will see that it's implied by the above sentence; anybody -%% who doesn't think about it is safe in his ignorance. -% -%Thus normally she chooses -%three nodes, but if she is running an OR and her destination is on an OR, -%she uses five. -Should Alice choose a random path length (e.g.~from a geometric -distribution) to foil an attacker who -uses timing to learn that he is the fifth hop and thus concludes that -both Alice and the responder are running ORs? - -Throughout this paper, we have assumed that end-to-end traffic -confirmation will immediately and automatically defeat a low-latency -anonymity system. Even high-latency anonymity systems can be -vulnerable to end-to-end traffic confirmation, if the traffic volumes -are high enough, and if users' habits are sufficiently -distinct~\cite{statistical-disclosure,limits-open}. Can anything be -done to -make low-latency systems resist these attacks as well as high-latency -systems? Tor already makes some effort to conceal the starts and ends of -streams by wrapping long-range control commands in identical-looking -relay cells. Link padding could frustrate passive observers who count -packets; long-range padding could work against observers who own the -first hop in a circuit. But more research remains to find an efficient -and practical approach. Volunteers prefer not to run constant-bandwidth -padding; but no convincing traffic shaping approach has been -specified. Recent work on long-range padding~\cite{defensive-dropping} -shows promise. One could also try to reduce correlation in packet timing -by batching and re-ordering packets, but it is unclear whether this could -improve anonymity without introducing so much latency as to render the -network unusable. - -A cascade topology may better defend against traffic confirmation by -aggregating users, and making padding and -mixing more affordable. Does the hydra topology (many input nodes, -few output nodes) work better against some adversaries? Are we going -to get a hydra anyway because most nodes will be middleman nodes? - -Common wisdom suggests that Alice should run her own OR for best -anonymity, because traffic coming from her node could plausibly have -come from elsewhere. How much mixing does this approach need? Is it -immediately beneficial because of real-world adversaries that can't -observe Alice's router, but can run routers of their own? - -To scale to many users, and to prevent an attacker from observing the -whole network, it may be necessary -to support far more servers than Tor currently anticipates. -This introduces several issues. First, if approval by a central set -of directory servers is no longer feasible, what mechanism should be used -to prevent adversaries from signing up many colluding servers? Second, -if clients can no longer have a complete picture of the network, -how can they perform discovery while preventing attackers from -manipulating or exploiting gaps in their knowledge? Third, if there -are too many servers for every server to constantly communicate with -every other, which non-clique topology should the network use? -(Restricted-route topologies promise comparable anonymity with better -scalability~\cite{danezis:pet2003}, but whatever topology we choose, we -need some way to keep attackers from manipulating their position within -it~\cite{casc-rep}.) Fourth, if no central authority is tracking -server reliability, how do we stop unreliable servers from making -the network unusable? Fifth, do clients receive so much anonymity -from running their own ORs that we should expect them all to do -so~\cite{econymics}, or do we need another incentive structure to -motivate them? Tarzan and MorphMix present possible solutions. - -% advogato, captcha - -When a Tor node goes down, all its circuits (and thus streams) must break. -Will users abandon the system because of this brittleness? How well -does the method in Section~\ref{subsec:dos} allow streams to survive -node failure? If affected users rebuild circuits immediately, how much -anonymity is lost? It seems the problem is even worse in a peer-to-peer -environment---such systems don't yet provide an incentive for peers to -stay connected when they're done retrieving content, so we would expect -a higher churn rate. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\section{Future Directions} -\label{sec:conclusion} - -Tor brings together many innovations into a unified deployable system. The -next immediate steps include: - -\emph{Scalability:} Tor's emphasis on deployability and design simplicity -has led us to adopt a clique topology, semi-centralized -directories, and a full-network-visibility model for client -knowledge. These properties will not scale past a few hundred servers. -Section~\ref{sec:maintaining-anonymity} describes some promising -approaches, but more deployment experience will be helpful in learning -the relative importance of these bottlenecks. - -\emph{Bandwidth classes:} This paper assumes that all ORs have -good bandwidth and latency. We should instead adopt the MorphMix model, -where nodes advertise their bandwidth level (DSL, T1, T3), and -Alice avoids bottlenecks by choosing nodes that match or -exceed her bandwidth. In this way DSL users can usefully join the Tor -network. - -\emph{Incentives:} Volunteers who run nodes are rewarded with publicity -and possibly better anonymity~\cite{econymics}. More nodes means increased -scalability, and more users can mean more anonymity. We need to continue -examining the incentive structures for participating in Tor. Further, -we need to explore more approaches to limiting abuse, and understand -why most people don't bother using privacy systems. - -\emph{Cover traffic:} Currently Tor omits cover traffic---its costs -in performance and bandwidth are clear but its security benefits are -not well understood. We must pursue more research on link-level cover -traffic and long-range cover traffic to determine whether some simple padding -method offers provable protection against our chosen adversary. - -%%\emph{Offer two relay cell sizes:} Traffic on the Internet tends to be -%%large for bulk transfers and small for interactive traffic. One cell -%%size cannot be optimal for both types of traffic. -% This should go in the spec and todo, but not the paper yet. -RD - -\emph{Caching at exit nodes:} Perhaps each exit node should run a -caching web proxy~\cite{shsm03}, to improve anonymity for cached pages -(Alice's request never -leaves the Tor network), to improve speed, and to reduce bandwidth cost. -On the other hand, forward security is weakened because caches -constitute a record of retrieved files. We must find the right -balance between usability and security. - -\emph{Better directory distribution:} -Clients currently download a description of -the entire network every 15 minutes. As the state grows larger -and clients more numerous, we may need a solution in which -clients receive incremental updates to directory state. -More generally, we must find more -scalable yet practical ways to distribute up-to-date snapshots of -network status without introducing new attacks. - -\emph{Further specification review:} Our public -byte-level specification~\cite{tor-spec} needs -external review. We hope that as Tor -is deployed, more people will examine its -specification. - -\emph{Multisystem interoperability:} We are currently working with the -designer of MorphMix to unify the specification and implementation of -the common elements of our two systems. So far, this seems -to be relatively straightforward. Interoperability will allow testing -and direct comparison of the two designs for trust and scalability. - -\emph{Wider-scale deployment:} The original goal of Tor was to -gain experience in deploying an anonymizing overlay network, and -learn from having actual users. We are now at a point in design -and development where we can start deploying a wider network. Once -we have many actual users, we will doubtlessly be better -able to evaluate some of our design decisions, including our -robustness/latency tradeoffs, our performance tradeoffs (including -cell size), our abuse-prevention mechanisms, and -our overall usability. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -%% commented out for anonymous submission -\section*{Acknowledgments} - We thank Peter Palfrader, Geoff Goodell, Adam Shostack, Joseph Sokol-Margolis, - John Bashinski, and Zack Brown - for editing and comments; - Matej Pfajfar, Andrei Serjantov, Marc Rennhard for design discussions; - Bram Cohen for congestion control discussions; - Adam Back for suggesting telescoping circuits; and - Cathy Meadows for formal analysis of the \emph{extend} protocol. - This work has been supported by ONR and DARPA. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\bibliographystyle{latex8} -\bibliography{tor-design} - -\end{document} - -% Style guide: -% U.S. spelling -% avoid contractions (it's, can't, etc.) -% prefer ``for example'' or ``such as'' to e.g. -% prefer ``that is'' to i.e. -% 'mix', 'mixes' (as noun) -% 'mix-net' -% 'mix', 'mixing' (as verb) -% 'middleman' [Not with a hyphen; the hyphen has been optional -% since Middle English.] -% 'nymserver' -% 'Cypherpunk', 'Cypherpunks', 'Cypherpunk remailer' -% 'Onion Routing design', 'onion router' [note capitalization] -% 'SOCKS' -% Try not to use \cite as a noun. -% 'Authorizating' sounds great, but it isn't a word. -% 'First, second, third', not 'Firstly, secondly, thirdly'. -% 'circuit', not 'channel' -% Typography: no space on either side of an em dash---ever. -% Hyphens are for multi-part words; en dashs imply movement or -% opposition (The Alice--Bob connection); and em dashes are -% for punctuation---like that. -% A relay cell; a control cell; a \emph{create} cell; a -% \emph{relay truncated} cell. Never ``a \emph{relay truncated}.'' -% -% 'Substitute ``Damn'' every time you're inclined to write ``very;'' your -% editor will delete it and the writing will be just as it should be.' -% -- Mark Twain diff --git a/doc/design-paper/usenix.sty b/doc/design-paper/usenix.sty deleted file mode 100644 index 4442f1157..000000000 --- a/doc/design-paper/usenix.sty +++ /dev/null @@ -1,98 +0,0 @@ -% usenix-2e.sty - to be used with latex2e (the new one) for USENIX. -% To use this style file, do this: -% -% \documentclass[twocolumn]{article} -% \usepackage{usenix-2e} -% and put {\rm ....} around the author names. -% -% $Id$ -% -% The following definitions are modifications of standard article.sty -% definitions, arranged to do a better job of matching the USENIX -% guidelines. -% It will automatically select two-column mode and the Times-Roman -% font. - -% -% USENIX papers are two-column. -% Times-Roman font is nice if you can get it (requires NFSS, -% which is in latex2e. - -\if@twocolumn\else\input twocolumn.sty\fi -\usepackage{times} - -% -% USENIX wants margins of: 7/8" side, 1" bottom, and 3/4" top. -% 0.25" gutter between columns. -% Gives active areas of 6.75" x 9.25" -% -\setlength{\textheight}{9.0in} -\setlength{\columnsep}{0.25in} -%%\setlength{\textwidth}{6.75in} -\setlength{\textwidth}{7.00in} -%\setlength{\footheight}{0.0in} -\setlength{\topmargin}{-0.25in} -\setlength{\headheight}{0.0in} -\setlength{\headsep}{0.0in} -\setlength{\evensidemargin}{-0.125in} -\setlength{\oddsidemargin}{-0.125in} - -% -% Usenix wants no page numbers for submitted papers, so that they can -% number them themselves. -% -\pagestyle{empty} - -% -% Usenix titles are in 14-point bold type, with no date, and with no -% change in the empty page headers. The whol author section is 12 point -% italic--- you must use {\rm } around the actual author names to get -% them in roman. -% -\def\maketitle{\par - \begingroup - \renewcommand\thefootnote{\fnsymbol{footnote}}% - \def\@makefnmark{\hbox to\z@{$\m@th^{\@thefnmark}$\hss}}% - \long\def\@makefntext##1{\parindent 1em\noindent - \hbox to1.8em{\hss$\m@th^{\@thefnmark}$}##1}% - \if@twocolumn - \twocolumn[\@maketitle]% - \else \newpage - \global\@topnum\z@ - \@maketitle \fi\@thanks - \endgroup - \setcounter{footnote}{0}% - \let\maketitle\relax - \let\@maketitle\relax - \gdef\@thanks{}\gdef\@author{}\gdef\@title{}\let\thanks\relax} - -\def\@maketitle{\newpage - \vbox to 2.5in{ - \vspace*{\fill} - \vskip 2em - \begin{center}% - {\Large\bf \@title \par}% - \vskip 0.375in minus 0.300in - {\large\it - \lineskip .5em - \begin{tabular}[t]{c}\@author - \end{tabular}\par}% - \end{center}% - \par - \vspace*{\fill} -% \vskip 1.5em - } -} - -% -% The abstract is preceded by a 12-pt bold centered heading -\def\abstract{\begin{center}% -{\large\bf \abstractname\vspace{-.5em}\vspace{\z@}}% -\end{center}} -\def\endabstract{} - -% -% Main section titles are 12-pt bold. Others can be same or smaller. -% -\def\section{\@startsection {section}{1}{\z@}{-3.5ex plus-1ex minus - -.2ex}{2.3ex plus.2ex}{\reset@font\large\bf}} diff --git a/doc/design-paper/usenixsubmit.cls b/doc/design-paper/usenixsubmit.cls deleted file mode 100644 index 743ffcfe4..000000000 --- a/doc/design-paper/usenixsubmit.cls +++ /dev/null @@ -1,7 +0,0 @@ -% Created by Anil Somayaji - -\ProvidesClass{usenixsubmit} -\LoadClass[11pt,letterpaper]{article} -\usepackage{times} -\usepackage[margin=1in]{geometry} - diff --git a/doc/privoxy-osx-universal-binary.txt b/doc/privoxy-osx-universal-binary.txt deleted file mode 100644 index 02a726b83..000000000 --- a/doc/privoxy-osx-universal-binary.txt +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/sh -# working patch and options from pnx in #tor - -patch -N << "EOF" ---- GNUmakefile.in.orig 2007-11-15 02:39:01.000000000 +0100 -+++ GNUmakefile.in 2007-11-15 02:39:12.000000000 +0100 -@@ -246,7 +246,7 @@ - CFLAGS = @CFLAGS@ @CPPFLAGS@ $(OTHER_CFLAGS) $(SPECIAL_CFLAGS) -Wall \ - @STATIC_PCRE_ONLY@ -Ipcre - --LDFLAGS = $(DEBUG_CFLAGS) $(SPECIAL_CFLAGS) -+LDFLAGS = @LDFLAGS@ $(DEBUG_CFLAGS) $(SPECIAL_CFLAGS) - - - ############################################################################# -EOF - -autoheader && autoconf && CFLAGS="-O2 -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" LDFLAGS="-mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" ./configure --prefix=/Library/Privoxy/ --disable-dynamic-pcrs --sysconfdir=/Library/Privoxy/ --mandir=/Library/Privoxy/ --disable-dependency-tracking && make diff --git a/doc/roadmaps/2008-12-19-roadmap-full.pdf b/doc/roadmaps/2008-12-19-roadmap-full.pdf Binary files differdeleted file mode 100644 index d87171c2d..000000000 --- a/doc/roadmaps/2008-12-19-roadmap-full.pdf +++ /dev/null diff --git a/doc/roadmaps/2009-03-11-performance.pdf b/doc/roadmaps/2009-03-11-performance.pdf Binary files differdeleted file mode 100644 index 3af74ddca..000000000 --- a/doc/roadmaps/2009-03-11-performance.pdf +++ /dev/null diff --git a/doc/roadmaps/roadmap-2007.pdf b/doc/roadmaps/roadmap-2007.pdf Binary files differdeleted file mode 100644 index 2422c0588..000000000 --- a/doc/roadmaps/roadmap-2007.pdf +++ /dev/null diff --git a/doc/roadmaps/roadmap-2007.tex b/doc/roadmaps/roadmap-2007.tex deleted file mode 100644 index cebe4a590..000000000 --- a/doc/roadmaps/roadmap-2007.tex +++ /dev/null @@ -1,690 +0,0 @@ -\documentclass{article} - -\usepackage{url} - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} -\newcommand{\tmp}[1]{{\bf #1} [......] \\} -\newcommand{\plan}[1]{ {\bf (#1)}} - -\begin{document} - -\title{Tor Development Roadmap: Wishlist for Nov 2006--Dec 2007} -\author{Roger Dingledine \and Nick Mathewson \and Shava Nerad} - -\maketitle -\pagestyle{plain} - -% TO DO: -% add cites -% add time estimates - - -\section{Introduction} -%Hi, Roger! Hi, Shava. This paragraph should get deleted soon. Right now, -%this document goes into about as much detail as I'd like to go into for a -%technical audience, since that's the audience I know best. It doesn't have -%time estimates everywhere. It isn't well prioritized, and it doesn't -%distinguish well between things that need lots of research and things that -%don't. The breakdowns don't all make sense. There are lots of things where -%I don't make it clear how they fit into larger goals, and lots of larger -%goals that don't break down into little things. It isn't all stuff we can do -%for sure, and it isn't even all stuff we can do for sure in 2007. The -%tmp\{\} macro indicates stuff I haven't said enough about. That said, here -%plangoes... - -Tor (the software) and Tor (the overall software/network/support/document -suite) are now experiencing all the crises of success. Over the next year, -we're probably going to grow more in terms of users, developers, and funding -than before. This gives us the opportunity to perform long-neglected -maintenance tasks. - -\section{Code and design infrastructure} - -\subsection{Protocol revision} -To maintain backward compatibility, we've postponed major protocol -changes and redesigns for a long time. Because of this, there are a number -of sensible revisions we've been putting off until we could deploy several of -them at once. To do each of these, we first need to discuss design -alternatives with other cryptographers and outside collaborators to -make sure that our choices are secure. - -First of all, our protocol needs better {\bf versioning support} so that we -can make backward-incompatible changes to our core protocol. There are -difficult anonymity issues here, since many naive designs would make it easy -to tell clients apart (and then track them) based on their supported versions. - -With protocol versioning support would come the ability to {\bf future-proof - our ciphersuites}. For example, not only our OR protocol, but also our -directory protocol, is pretty firmly tied to the SHA-1 hash function, which -though not yet known to be insecure for our purposes, has begun to show -its age. We should -remove assumptions throughout our design based on the assumption that public -keys, secret keys, or digests will remain any particular size indefinitely. - -Our OR {\bf authentication protocol}, though provably -secure\cite{tap:pet2006}, relies more on particular aspects of RSA and our -implementation thereof than we had initially believed. To future-proof -against changes, we should replace it with a less delicate approach. - -\plan{For all the above: 2 person-months to specify, spread over several - months with time for interaction with external participants. One - person-month to implement. Start specifying in early 2007.} - -We might design a {\bf stream migration} feature so that streams tunneled -over Tor could be more resilient to dropped connections and changed IPs. -\plan{Not in 2007.} - -A new protocol could support {\bf multiple cell sizes}. Right now, all data -passes through the Tor network divided into 512-byte cells. This is -efficient for high-bandwidth protocols, but inefficient for protocols -like SSH or AIM that send information in small chunks. Of course, we need to -investigate the extent to which multiple sizes could make it easier for an -adversary to fingerprint a traffic pattern. \plan{Not in 2007.} - -As a part of our design, we should investigate possible {\bf cipher modes} -other than counter mode. For example, a mode with built-in integrity -checking, error propagation, and random access could simplify our protocol -significantly. Sadly, many of these are patented and unavailable for us. -\plan{Not in 2007.} - -\subsection{Scalability} - -\subsubsection{Improved directory efficiency} -Right now, clients download a statement of the {\bf network status} made by -each directory authority. We could reduce network bandwidth significantly by -having the authorities jointly sign a statement reflecting their vote on the -current network status. This would save clients up to 160K per hour, and -make their view of the network more uniform. Of course, we'd need to make -sure the voting process was secure and resilient to failures in the -network.\plan{Must do; specify in 2006. 2 weeks to specify, 3-4 weeks to - implement.} - -We should {\bf shorten router descriptors}, since the current format includes -a great deal of information that's only of interest to the directory -authorities, and not of interest to clients. We can do this by having each -router upload a short-form and a long-form signed descriptor, and having -clients download only the short form. Even a naive version of this would -save about 40\% of the bandwidth currently spent by clients downloading -descriptors.\plan{Must do; specify in 2006. 3-4 weeks.} - -We should {\bf have routers upload their descriptors even less often}, so -that clients do not need to download replacements every 18 hours whether any -information has changed or not. (As of Tor 0.1.2.3-alpha, clients tolerate -routers that don't upload often, but routers still upload at least every 18 -hours to support older clients.) \plan{Must do, but not until 0.1.1.x is -deprecated in mid 2007. 1 week.} - -\subsubsection{Non-clique topology} -Our current network design achieves a certain amount of its anonymity by -making clients act like each other through the simple expedient of making -sure that all clients know all servers, and that any server can talk to any -other server. But as the number of servers increases to serve an -ever-greater number of clients, these assumptions become impractical. - -At worst, if these scalability issues become troubling before a solution is -found, we can design and build a solution to {\bf split the network into -multiple slices} until a better solution comes along. This is not ideal, -since rather than looking like all other users from a point of view of path -selection, users would ``only'' look like 200,000--300,000 other -users.\plan{Not unless needed.} - -We are in the process of designing {\bf improved schemes for network - scalability}. Some approaches focus on limiting what an adversary can know -about what a user knows; others focus on reducing the extent to which an -adversary can exploit this knowledge. These are currently in their infancy, -and will probably not be needed in 2007, but they must be designed in 2007 if -they are to be deployed in 2008.\plan{Design in 2007; unknown difficulty. - Write a paper.} - -\subsubsection{Relay incentives} -To support more users on the network, we need to get more servers. So far, -we've relied on volunteerism to attract server operators, and so far it's -served us well. But in the long run, we need to {\bf design incentives for - users to run servers} and relay traffic for others. Most obviously, we -could try to build the network so that servers offered improved service for -other servers, but we would need to do so without weakening anonymity and -making it obvious which connections originate from users running servers. We -have some preliminary designs~\cite{incentives-txt,tor-challenges}, -but need to perform -some more research to make sure they would be safe and effective.\plan{Write - a draft paper; 2 person-months.} - -\subsection{Portability} -Our {\bf Windows implementation}, though much improved, continues to lag -behind Unix and Mac OS X, especially when running as a server. We hope to -merge promising patches from Mike Chiussi to address this point, and bring -Windows performance on par with other platforms.\plan{Do in 2007; 1.5 months - to integrate not counting Mike's work.} - -We should have {\bf better support for portable devices}, including modes of -operation that require less RAM, and that write to disk less frequently (to -avoid wearing out flash RAM).\plan{Optional; 2 weeks.} - -We should {\bf stop using socketpair on Windows}; instead, we can use -in-memory structures to communicate between cpuworkers and the main thread, -and between connections.\plan{Optional; 1 week.} - -\subsection{Performance: resource usage} -We've been working on {\bf using less RAM}, especially on servers. This has -paid off a lot for directory caches in the 0.1.2, which in some cases are -using 90\% less memory than they used to require. But we can do better, -especially in the area around our buffer management algorithms, by using an -approach more like the BSD and Linux kernels use instead of our current ring -buffer approach. (For OR connections, we can just use queues of cell-sized -chunks produced with a specialized allocator.) This could potentially save -around 25 to 50\% of the memory currently allocated for network buffers, and -make Tor a more attractive proposition for restricted-memory environments -like old computers, mobile devices, and the like.\plan{Do in 2007; 2-3 weeks - plus one week measurement.} - -We should improve our {\bf bandwidth limiting}. The current system has been -crucial in making users willing to run servers: nobody is willing to run a -server if it might use an unbounded amount of bandwidth, especially if they -are charged for their usage. We can make our system better by letting users -configure bandwidth limits independently for their own traffic and traffic -relayed for others; and by adding write limits for users running directory -servers.\plan{Do in 2006; 2-3 weeks.} - -On many hosts, sockets are still in short supply, and will be until we can -migrate our protocol to UDP. We can {\bf use fewer sockets} by making our -self-to-self connections happen internally to the code rather than involving -the operating system's socket implementation.\plan{Optional; 1 week.} - -\subsection{Performance: network usage} -We know too little about how well our current path -selection algorithms actually spread traffic around the network in practice. -We should {\bf research the efficacy of our traffic allocation} and either -assure ourselves that it is close enough to optimal as to need no improvement -(unlikely) or {\bf identify ways to improve network usage}, and get more -users' traffic delivered faster. Performing this research will require -careful thought about anonymity implications. - -We should also {\bf examine the efficacy of our congestion control - algorithm}, and see whether we can improve client performance in the -presence of a congested network through dynamic `sendme' window sizes or -other means. This will have anonymity implications too if we aren't careful. - -\plan{For both of the above: research, design and write - a measurement tool in 2007: 1 month. See if we can interest a graduate - student.} - -We should work on making Tor's cell-based protocol perform better on -networks with low bandwidth -and high packet loss.\plan{Do in 2007 if we're funded to do it; 4-6 weeks.} - -\subsection{Performance scenario: one Tor client, many users} -We should {\bf improve Tor's performance when a single Tor handles many - clients}. Many organizations want to manage a single Tor client on their -firewall for many users, rather than having each user install a separate -Tor client. We haven't optimized for this scenario, and it is likely that -there are some code paths in the current implementation that become -inefficient when a single Tor is servicing hundreds or thousands of client -connections. (Additionally, it is likely that such clients have interesting -anonymity requirements the we should investigate.) We should profile Tor -under appropriate loads, identify bottlenecks, and fix them.\plan{Do in 2007 - if we're funded to do it; 4-8 weeks.} - -\subsection{Tor servers on asymmetric bandwidth} - -Tor should work better on servers that have asymmetric connections like cable -or DSL. Because Tor has separate TCP connections between each -hop, if the incoming bytes are arriving just fine and the outgoing bytes are -all getting dropped on the floor, the TCP push-back mechanisms don't really -transmit this information back to the incoming streams.\plan{Do in 2007 since - related to bandwidth limiting. 3-4 weeks.} - -\subsection{Running Tor as both client and server} - -Many performance tradeoffs and balances that might need more attention. -We first need to track and fix whatever bottlenecks emerge; but we also -need to invent good algorithms for prioritizing the client's traffic -without starving the server's traffic too much.\plan{No idea; try -profiling and improving things in 2007.} - -\subsection{Protocol redesign for UDP} -Tor has relayed only TCP traffic since its first versions, and has used -TLS-over-TCP to do so. This approach has proved reliable and flexible, but -in the long term we will need to allow UDP traffic on the network, and switch -some or all of the network to using a UDP transport. {\bf Supporting UDP - traffic} will make Tor more suitable for protocols that require UDP, such -as many VOIP protocols. {\bf Using a UDP transport} could greatly reduce -resource limitations on servers, and make the network far less interruptible -by lossy connections. Either of these protocol changes would require a great -deal of design work, however. We hope to be able to enlist the aid of a few -talented graduate students to assist with the initial design and -specification, but the actual implementation will require significant testing -of different reliable transport approaches.\plan{Maybe do a design in 2007 if -we find an interested academic. Ian or Ben L might be good partners here.} - -\section{Blocking resistance} - -\subsection{Design for blocking resistance} -We have written a design document explaining our general approach to blocking -resistance. We should workshop it with other experts in the field to get -their ideas about how we can improve Tor's efficacy as an anti-censorship -tool. - -\subsection{Implementation: client-side and bridges-side} - -Our anticensorship design calls for some nodes to act as ``bridges'' -that are outside a national firewall, and others inside the firewall to -act as pure clients. This part of the design is quite clear-cut; we're -probably ready to begin implementing it. To {\bf implement bridges}, we -need to have servers publish themselves as limited-availability relays -to a special bridge authority if they judge they'd make good servers. -We will also need to help provide documentation for port forwarding, -and an easy configuration tool for running as a bridge. - -To {\bf implement clients}, we need to provide a flexible interface to -learn about bridges and to act on knowledge of bridges. We also need -to teach them how to know to use bridges as their first hop, and how to -fetch directory information from both classes of directory authority. - -Clients also need to {\bf use the encrypted directory variant} added in Tor -0.1.2.3-alpha. This will let them retrieve directory information over Tor -once they've got their initial bridges. We may want to get the rest of the -Tor user base to begin using this encrypted directory variant too, to -provide cover. - -Bridges will want to be able to {\bf listen on multiple addresses and ports} -if they can, to give the adversary more ports to block. - -\subsection{Research: anonymity implications from becoming a bridge} - -\subsection{Implementation: bridge authority} - -The design here is also reasonably clear-cut: we need to run some -directory authorities with a slightly modified protocol that doesn't leak -the entire list of bridges. Thus users can learn up-to-date information -for bridges they already know about, but they can't learn about arbitrary -new bridges. - -\subsection{Normalizing the Tor protocol on the wire} -Additionally, we should {\bf resist content-based filters}. Though an -adversary can't see what users are saying, some aspects of our protocol are -easy to fingerprint {\em as} Tor. We should correct this where possible. - -Look like Firefox; or look like nothing? -Future research: investigate timing similarities with other protocols. - -\subsection{Access control for bridges} -Design/impl: password-protecting bridges, in light of above. -And/or more general access control. - -\subsection{Research: scanning-resistance} - -\subsection{Research/Design/Impl: how users discover bridges} -Our design anticipates an arms race between discovery methods and censors. -We need to begin the infrastructure on our side quickly, preferably in a -flexible language like Python, so we can adapt quickly to censorship. - -phase one: personal bridges -phase two: families of personal bridges -phase three: more structured social network -phase four: bag of tricks -Research: phase five... - -Integration with Psiphon, etc? - -\subsection{Document best practices for users} -Document best practices for various activities common among -blocked users (e.g. WordPress use). - -\subsection{Research: how to know if a bridge has been blocked?} - -\subsection{GeoIP maintenance, and "private" user statistics} -How to know if the whole idea is working? - -\subsection{Research: hiding whether the user is reading or publishing?} - -\subsection{Research: how many bridges do you need to know to maintain -reachability?} - -\subsection{Resisting censorship of the Tor website, docs, and mirrors} - -We should take some effort to consider {\bf initial distribution of Tor and - related information} in countries where the Tor website and mirrors are -censored. (Right now, most countries that block access to Tor block only the -main website and leave mirrors and the network itself untouched.) Falling -back on word-of-mouth is always a good last resort, but we should also take -steps to make sure it's relatively easy for users to get ahold of a copy. - -\section{Security} - -\subsection{Security research projects} - -We should investigate approaches with some promise to help Tor resist -end-to-end traffic correlation attacks. It's an open research question -whether (and to what extent) {\bf mixed-latency} networks, {\bf low-volume - long-distance padding}, or other approaches can resist these attacks, which -are currently some of the most effective against careful Tor users. We -should research these questions and perform simulations to identify -opportunities for strengthening our design without dropping performance to -unacceptable levels. %Cite something -\plan{Start doing this in 2007; write a paper. 8-16 weeks.} - -We've got some preliminary results suggesting that {\bf a topology-aware - routing algorithm}~\cite{feamster:wpes2004} could reduce Tor users' -vulnerability against local or ISP-level adversaries, by ensuring that they -are never in a position to watch both ends of a connection. We need to -examine the effects of this approach in more detail and consider side-effects -on anonymity against other kinds of adversaries. If the approach still looks -promising, we should investigate ways for clients to implement it (or an -approximation of it) without having to download routing tables for the whole -Internet. \plan{Not in 2007 unless a graduate student wants to do it.} - -%\tmp{defenses against end-to-end correlation} We don't expect any to work -%right now, but it would be useful to learn that one did. Alternatively, -%proving that one didn't would free up researchers in the field to go work on -%other things. -% -% See above; I think I got this. - -We should research the efficacy of {\bf website fingerprinting} attacks, -wherein an adversary tries to match the distinctive traffic and timing -pattern of the resources constituting a given website to the traffic pattern -of a user's client. These attacks work great in simulations, but in -practice we hear they don't work nearly as well. We should get some actual -numbers to investigate the issue, and figure out what's going on. If we -resist these attacks, or can improve our design to resist them, we should. -% add cites -\plan{Possibly part of end-to-end correlation paper. Otherwise, not in 2007 - unless a graduate student is interested.} - -\subsection{Implementation security} -Right now, each Tor node stores its keys unencrypted. We should {\bf encrypt - more Tor keys} so that Tor authorities can require a startup password. We -should look into adding intermediary medium-term ``signing keys'' between -identity keys and onion keys, so that a password could be required to replace -a signing key, but not to start Tor. This would improve Tor's long-term -security, especially in its directory authority infrastructure.\plan{Design this - as a part of the revised ``v2.1'' directory protocol; implement it in - 2007. 3-4 weeks.} - -We should also {\bf mark RAM that holds key material as non-swappable} so -that there is no risk of recovering key material from a hard disk -compromise. This would require submitting patches upstream to OpenSSL, where -support for marking memory as sensitive is currently in a very preliminary -state.\plan{Nice to do, but not in immediate Tor scope.} - -There are numerous tools for identifying trouble spots in code (such as -Coverity or even VS2005's code analysis tool) and we should convince somebody -to run some of them against the Tor codebase. Ideally, we could figure out a -way to get our code checked periodically rather than just once.\plan{Almost - no time once we talk somebody into it.} - -We should try {\bf protocol fuzzing} to identify errors in our -implementation.\plan{Not in 2007 unless we find a grad student or - undergraduate who wants to try.} - -Our guard nodes help prevent an attacker from being able to become a chosen -client's entry point by having each client choose a few favorite entry points -as ``guards'' and stick to them. We should implement a {\bf directory - guards} feature to keep adversaries from enumerating Tor users by acting as -a directory cache.\plan{Do in 2007; 2 weeks.} - -\subsection{Detect corrupt exits and other servers} -With the success of our network, we've attracted servers in many locations, -operated by many kinds of people. Unfortunately, some of these locations -have compromised or defective networks, and some of these people are -untrustworthy or incompetent. Our current design relies on authority -administrators to identify bad nodes and mark them as nonfunctioning. We -should {\bf automate the process of identifying malfunctioning nodes} as -follows: - -We should create a generic {\bf feedback mechanism for add-on tools} like -Mike Perry's ``Snakes on a Tor'' to report failing nodes to authorities. -\plan{Do in 2006; 1-2 weeks.} - -We should write tools to {\bf detect more kinds of innocent node failure}, -such as nodes whose network providers intercept SSL, nodes whose network -providers censor popular websites, and so on. We should also try to detect -{\bf routers that snoop traffic}; we could do this by launching connections -to throwaway accounts, and seeing which accounts get used.\plan{Do in 2007; - ask Mike Perry if he's interested. 4-6 weeks.} - -We should add {\bf an efficient way for authorities to mark a set of servers - as probably collaborating} though not necessarily otherwise dishonest. -This happens when an administrator starts multiple routers, but doesn't mark -them as belonging to the same family.\plan{Do during v2.1 directory protocol - redesign; 1-2 weeks to implement.} - -To avoid attacks where an adversary claims good performance in order to -attract traffic, we should {\bf have authorities measure node performance} -(including stability and bandwidth) themselves, and not simply believe what -they're told. Measuring stability can be done by tracking MTBF. Measuring -bandwidth can be tricky, since it's hard to distinguish between a server with -low capacity, and a high-capacity server with most of its capacity in -use.\plan{Do ``Stable'' in 2007; 2-3 weeks. ``Fast'' will be harder; do it - if we can interest a grad student.} - -{\bf Operating a directory authority should be easier.} We rely on authority -operators to keep the network running well, but right now their job involves -too much busywork and administrative overhead. A better interface for them -to use could free their time to work on exception cases rather than on -adding named nodes to the network.\plan{Do in 2007; 4-5 weeks.} - -\subsection{Protocol security} - -In addition to other protocol changes discussed above, -% And should we move some of them down here? -NM -we should add {\bf hooks for denial-of-service resistance}; we have some -preliminary designs, but we shouldn't postpone them until we really need them. -If somebody tries a DDoS attack against the Tor network, we won't want to -wait for all the servers and clients to upgrade to a new -version.\plan{Research project; do this in 2007 if funded.} - -\section{Development infrastructure} - -\subsection{Build farm} -We've begun to deploy a cross-platform distributed build farm of hosts -that build and test the Tor source every time it changes in our development -repository. - -We need to {\bf get more participants}, so that we can test a larger variety -of platforms. (Previously, we've only found out when our code had broken on -obscure platforms when somebody got around to building it.) - -We need also to {\bf add our dependencies} to the build farm, so that we can -ensure that libraries we need (especially libevent) do not stop working on -any important platform between one release and the next. - -\plan{This is ongoing as more buildbots arrive.} - -\subsection{Improved testing harness} -Currently, our {\bf unit tests} cover only about 20\% of the code base. This -is uncomfortably low; we should write more and switch to a more flexible -testing framework.\plan{Ongoing basis, time permitting.} - -We should also write flexible {\bf automated single-host deployment tests} so -we can more easily verify that the current codebase works with the -network.\plan{Worthwhile in 2007; would save lots of time. 2-4 weeks.} - -We should build automated {\bf stress testing} frameworks so we can see which -realistic loads cause Tor to perform badly, and regularly profile Tor against -these loads. This would give us {\it in vitro} performance values to -supplement our deployment experience.\plan{Worthwhile in 2007; 2-6 weeks.} - -We should improve our memory profiling code.\plan{...} - - -\subsection{Centralized build system} -We currently rely on a separate packager to maintain the packaging system and -to build Tor on each platform for which we distribute binaries. Separate -package maintainers is sensible, but separate package builders has meant -long turnaround times between source releases and package releases. We -should create the necessary infrastructure for us to produce binaries for all -major packages within an hour or so of source release.\plan{We should - brainstorm this at least in 2007.} - -\subsection{Improved metrics} -We need a way to {\bf measure the network's health, capacity, and degree of - utilization}. Our current means for doing this are ad hoc and not -completely accurate - -We need better ways to {\bf tell which countries are users are coming from, - and how many there are}. A good perspective of the network helps us -allocate resources and identify trouble spots, but our current approaches -will work less and less well as we make it harder for adversaries to -enumerate users. We'll probably want to shift to a smarter, statistical -approach rather than our current ``count and extrapolate'' method. - -\plan{All of this in 2007 if funded; 4-8 weeks} - -% \tmp{We'd like to know how much of the network is getting used.} -% I think this is covered above -NM - -\subsection{Controller library} -We've done lots of design and development on our controller interface, which -allows UI applications and other tools to interact with Tor. We could -encourage the development of more such tools by releasing a {\bf - general-purpose controller library}, ideally with API support for several -popular programming languages.\plan{2006 or 2007; 1-2 weeks.} - -\section{User experience} - -\subsection{Get blocked less, get blocked less broadly} -Right now, some services block connections from the Tor network because -they don't have a better -way to keep vandals from abusing them than blocking IP addresses associated -with vandalism. Our approach so far has been to educate them about better -solutions that currently exist, but we should also {\bf create better -solutions for limiting vandalism by anonymous users} like credential and -blind-signature based implementations, and encourage their use. Other -promising starting points including writing a patch and explanation for -Wikipedia, and helping Freenode to document, maintain, and expand its -current Tor-friendly position.\plan{Do a writeup here in 2007; 1-2 weeks.} - -Those who do block Tor users also block overbroadly, sometimes blacklisting -operators of Tor servers that do not permit exit to their services. We could -obviate innocent reasons for doing so by designing a {\bf narrowly-targeted Tor - RBL service} so that those who wanted to overblock Tor could no longer -plead incompetence.\plan{Possibly in 2007 if we decide it's a good idea; 3 - weeks.} - -\subsection{All-in-one bundle} -We need a well-tested, well-documented bundle of Tor and supporting -applications configured to use it correctly. We have an initial -implementation well under way, but it will need additional work in -identifying requisite Firefox extensions, identifying security threats, -improving user experience, and so on. This will need significantly more work -before it's ready for a general public release. - -\subsection{LiveCD Tor} -We need a nice bootable livecd containing a minimal OS and a few applications -configured to use it correctly. The Anonym.OS project demonstrated that this -is quite feasible, but their project is not currently maintained. - -\subsection{A Tor client in a VM} -\tmp{a.k.a JanusVM} which is quite related to the firewall-level deployment -section below. JanusVM is a Linux kernel running in VMWare. It gets an IP -address from the network, and serves as a DHCP server for its host Windows -machine. It intercepts all outgoing traffic and redirects it into Privoxy, -Tor, etc. This Linux-in-Windows approach may help us with scalability in -the short term, and it may also be a good long-term solution rather than -accepting all security risks in Windows. - -%\subsection{Interface improvements} -%\tmp{Allow controllers to manipulate server status.} -% (Why is this in the User Experience section?) -RD -% I think it's better left to a generic ``make controller iface better'' item. - -\subsection{Firewall-level deployment} -Another useful deployment mode for some users is using {\bf Tor in a firewall - configuration}, and directing all their traffic through Tor. This can be a -little tricky to set up currently, but it's an effective way to make sure no -traffic leaves the host un-anonymized. To achieve this, we need to {\bf - improve and port our new TransPort} feature which allows Tor to be used -without SOCKS support; to {\bf add an anonymizing DNS proxy} feature to Tor; -and to {\bf construct a recommended set of firewall configurations} to redirect -traffic to Tor. - -This is an area where {\bf deployment via a livecd}, or an installation -targeted at specialized home routing hardware, could be useful. - -\subsection{Assess software and configurations for anonymity risks} -Right now, users and packagers are more or less on their own when selecting -Firefox extensions. We should {\bf assemble a recommended list of browser - extensions} through experiment, and include this in the application bundles -we distribute. - -We should also describe {\bf best practices for using Tor with each class of - application}. For example, Ethan Zuckerman has written a detailed -tutorial on how to use Tor, Firefox, GMail, and Wordpress to blog with -improved safety. There are many other cases on the Internet where anonymity -would be helpful, and there are a lot of ways to screw up using Tor. - -The Foxtor and Torbutton extensions serve similar purposes; we should pick a -favorite, and merge in the useful features of the other. - -%\tmp{clean up our own bundled software: -%E.g. Merge the good features of Foxtor into Torbutton} -% -% What else did you have in mind? -NM - -\subsection{Localization} -Right now, most of our user-facing code is internationalized. We need to -internationalize the last few hold-outs (like the Tor expert installer), and get -more translations for the parts that are already internationalized. - -Also, we should look into a {\bf unified translator's solution}. Currently, -since different tools have been internationalized using the -framework-appropriate method, different tools require translators to localize -them via different interfaces. Inasmuch as possible, we should make -translators only need to use a single tool to translate the whole Tor suite. - -\section{Support} - -It would be nice to set up some {\bf user support infrastructure} and -{\bf contributor support infrastructure}, especially focusing on server -operators and on coordinating volunteers. - -This includes intuitive and easy ticket systems for bug reports and -feature suggestions (not just mailing lists with a half dozen people -and no clear roles for who answers what), but it also includes a more -personalized and efficient framework for interaction so we keep the -attention and interest of the contributors, and so we make them feel -helpful and wanted. - -\section{Documentation} - -\subsection{Unified documentation scheme} - -We need to {\bf inventory our documentation.} Our documentation so far has -been mostly produced on an {\it ad hoc} basis, in response to particular -needs and requests. We should figure out what documentation we have, which of -it (if any) should get priority, and whether we can't put it all into a -single format. - -We could {\bf unify the docs} into a single book-like thing. This will also -help us identify what sections of the ``book'' are missing. - -\subsection{Missing technical documentation} - -We should {\bf revise our design paper} to reflect the new decisions and -research we've made since it was published in 2004. This will help other -researchers evaluate and suggest improvements to Tor's current design. - -Other projects sometimes implement the client side of our protocol. We -encourage this, but we should write {\bf a document about how to avoid -excessive resource use}, so we don't need to worry that they will do so -without regard to the effect of their choices on server resources. - -\subsection{Missing user documentation} - -Our documentation falls into two broad categories: some is `discoursive' and -explains in detail why users should take certain actions, and other -documentation is `comprehensive' and describes all of Tor's features. Right -now, we have no document that is both deep, readable, and thorough. We -should correct this by identifying missing spots in our design. - -\bibliographystyle{plain} \bibliography{tor-design} - -\end{document} - diff --git a/doc/roadmaps/roadmap-future.pdf b/doc/roadmaps/roadmap-future.pdf Binary files differdeleted file mode 100644 index 8300ce19c..000000000 --- a/doc/roadmaps/roadmap-future.pdf +++ /dev/null diff --git a/doc/roadmaps/roadmap-future.tex b/doc/roadmaps/roadmap-future.tex deleted file mode 100644 index 4ab240f97..000000000 --- a/doc/roadmaps/roadmap-future.tex +++ /dev/null @@ -1,895 +0,0 @@ -\documentclass{article} - -\usepackage{url} -\usepackage{fullpage} - -\newenvironment{tightlist}{\begin{list}{$\bullet$}{ - \setlength{\itemsep}{0mm} - \setlength{\parsep}{0mm} - % \setlength{\labelsep}{0mm} - % \setlength{\labelwidth}{0mm} - % \setlength{\topsep}{0mm} - }}{\end{list}} -\newcommand{\tmp}[1]{{\bf #1} [......] \\} -\newcommand{\plan}[1]{ {\bf (#1)}} - -\begin{document} - -\title{Tor Development Roadmap: Wishlist for 2008 and beyond} -\author{Roger Dingledine \and Nick Mathewson} -\date{} - -\maketitle -\pagestyle{plain} - -\section{Introduction} - -Tor (the software) and Tor (the overall software/network/support/document -suite) are now experiencing all the crises of success. Over the next -years, we're probably going to grow even more in terms of users, developers, -and funding than before. This document attempts to lay out all the -well-understood next steps that Tor needs to take. We should periodically -reorganize it to reflect current and intended priorities. - -\section{Everybody can be a relay} - -We've made a lot of progress towards letting an ordinary Tor client also -serve as a Tor relay. But these issues remain. - -\subsection{UPNP} - -We should teach Vidalia how to speak UPNP to automatically open and -forward ports on common (e.g. Linksys) routers. There are some promising -Qt-based UPNP libs out there, and in any case there are others (e.g. in -Perl) that we can base it on. - -\subsection{``ORPort auto'' to look for a reachable port} - -Vidalia defaults to port 443 on Windows and port 8080 elsewhere. But if -that port is already in use, or the ISP filters incoming connections -on that port (some cablemodem providers filter 443 inbound), the user -needs to learn how to notice this, and then pick a new one and type it -into Vidalia. - -We should add a new option ``auto'' that cycles through a set of preferred -ports, testing bindability and reachability for each of them, and only -complains to the user once it's given up on the common choices. - -\subsection{Incentives design} - -Roger has been working with researchers at Rice University to simulate -and analyze a new design where the directory authorities assign gold -stars to well-behaving relays, and then all the relays give priority -to traffic from gold-starred relays. The great feature of the design is -that not only does it provide the (explicit) incentive to run a relay, -but it also aims to grow the overall capacity of the network, so even -non-relays will benefit. - -It needs more analysis, and perhaps more design work, before we try -deploying it. - -\subsection{Windows libevent} - -Tor relays still don't work well or reliably on Windows XP or Windows -Vista, because we don't use the Windows-native ``overlapped IO'' -approach. Christian King made a good start at teaching libevent about -overlapped IO during Google Summer of Code 2007, and next steps are -to a) finish that, b) teach Tor to do openssl calls on buffers rather -than directly to the network, and c) teach Tor to use the new libevent -buffers approach. - -\subsection{Network scaling} - -If we attract many more relays, we will need to handle the growing pains -in terms of getting all the directory information to all the users. - -The first piece of this issue is a practical question: since the -directory size scales linearly with more relays, at some point it -will no longer be practical for every client to learn about every -relay. We can try to reduce the amount of information each client needs -to fetch (e.g. based on fetching less information preemptively as in -Section~\ref{subsec:fewer-descriptor-fetches} below), but eventually -clients will need to learn about only a subset of the network, and we -will need to design good ways to divide up the network information. - -The second piece is an anonymity question that arises from this -partitioning: if Tor's security comes from having all the clients -behaving in similar ways, yet we are now giving different clients -different directory information, how can we minimize the new anonymity -attacks we introduce? - -\subsection{Using fewer sockets} - -Since in the current network every Tor relay can reach every other Tor -relay, and we have many times more users than relays, pretty much every -possible link in the network is in use. That is, the current network -is a clique in practice. - -And since each of these connections requires a TCP socket, it's going -to be hard for the network to grow much larger: many systems come with -a default of 1024 file descriptors allowed per process, and raising -that ulimit is hard for end users. Worse, many low-end gateway/firewall -routers can't handle this many connections in their routing table. - -One approach is a restricted-route topology~\cite{danezis:pet2003}: -predefine which relays can reach which other relays, and communicate -these restrictions to the relays and the clients. We need to compute -which links are acceptable in a way that's decentralized yet scalable, -and in a way that achieves a small-worlds property; and we -need an efficient (compact) way to characterize the topology information -so all the users could keep up to date. - -Another approach would be to switch to UDP-based transport between -relays, so we don't need to keep the TCP sockets open at all. Needs more -investigation too. - -\subsection{Auto bandwidth detection and rate limiting, especially for - asymmetric connections.} - - -\subsection{Better algorithms for giving priority to local traffic} - -Proposal 111 made a lot of progress at separating local traffic from -relayed traffic, so Tor users can rate limit the relayed traffic at a -stricter level. But since we want to pass both traffic classes over the -same TCP connection, we can't keep them entirely separate. The current -compromise is that we treat all bytes to/from a given connectin as -local traffic if any of the bytes within the past N seconds were local -bytes. But a) we could use some more intelligent heuristics, and b) -this leaks information to an active attacker about when local traffic -was sent/received. - -\subsection{Tolerate absurdly wrong clocks, even for relays} - -Many of our users are on Windows, running with a clock several days or -even several years off from reality. Some of them are even intentionally -in this state so they can run software that will only run in the past. - -Before Tor 0.1.1.x, Tor clients would still function if their clock was -wildly off --- they simply got a copy of the directory and believed it. -Starting in Tor 0.1.1.x (and even moreso in Tor 0.2.0.x), the clients -only use networkstatus documents that they believe to be recent, so -clients with extremely wrong clocks no longer work. (This bug has been -an unending source of vague and confusing bug reports.) - -The first step is for clients to recognize when all the directory material -they're fetching has roughly the same offset from their current time, -and then automatically correct for it. - -Once that's working well, clients who opt to become bridge relays should -be able to use the same approach to serve accurate directory information -to their bridge users. - -\subsection{Risks from being a relay} - -Three different research -papers~\cite{back01,clog-the-queue,attack-tor-oak05} describe ways to -identify the nodes in a circuit by running traffic through candidate nodes -and looking for dips in the traffic while the circuit is active. These -clogging attacks are not that scary in the Tor context so long as relays -are never clients too. But if we're trying to encourage more clients to -turn on relay functionality too (whether as bridge relays or as normal -relays), then we need to understand this threat better and learn how to -mitigate it. - -One promising research direction is to investigate the RelayBandwidthRate -feature that lets Tor rate limit relayed traffic differently from local -traffic. Since the attacker's ``clogging'' traffic is not in the same -bandwidth class as the traffic initiated by the user, it may be harder -to detect interference. Or it may not be. - -\subsection{First a bridge, then a public relay?} - -Once enough of the items in this section are done, I want all clients -to start out automatically detecting their reachability and opting -to be bridge relays. - -Then if they realize they have enough consistency and bandwidth, they -should automatically upgrade to being non-exit relays. - -What metrics should we use for deciding when we're fast enough -and stable enough to switch? Given that the list of bridge relays needs -to be kept secret, it doesn't make much sense to switch back. - -\section{Tor on low resources / slow links} -\subsection{Reducing directory fetches further} -\label{subsec:fewer-descriptor-fetches} -\subsection{AvoidDiskWrites} -\subsection{Using less ram} -\subsection{Better DoS resistance for tor servers / authorities} -\section{Blocking resistance} -\subsection{Better bridge-address-distribution strategies} -\subsection{Get more volunteers running bridges} -\subsection{Handle multiple bridge authorities} -\subsection{Anonymity for bridge users: second layer of entry guards, etc?} -\subsection{More TLS normalization} -\subsection{Harder to block Tor software distribution} -\subsection{Integration with Psiphon} -\section{Packaging} -\subsection{Switch Privoxy out for Polipo} - - Make Vidalia able to launch more programs itself -\subsection{Continue Torbutton improvements} - especially better docs -\subsection{Vidalia and stability (especially wrt ongoing Windows problems)} - learn how to get useful crash reports (tracebacks) from Windows users -\subsection{Polipo support on Windows} -\subsection{Auto update for Tor, Vidalia, others} -\subsection{Tor browser bundle for USB and standalone use} -\subsection{LiveCD solution} -\subsection{VM-based solution} -\subsection{Tor-on-enclave-firewall configuration} -\subsection{General tutorials on what common applications are Tor-friendly} -\subsection{Controller libraries (torctl) plus documentation} -\subsection{Localization and translation (Vidalia, Torbutton, web pages)} -\section{Interacting better with Internet sites} -\subsection{Make tordnsel (tor exitlist) better and more well-known} -\subsection{Nymble} -\subsection{Work with Wikipedia, Slashdot, Google(, IRC networks)} -\subsection{IPv6 support for exit destinations} -\section{Network health} -\subsection{torflow / soat to detect bad relays} -\subsection{make authorities more automated} -\subsection{torstatus pages and better trend tracking} -\subsection{better metrics for assessing network health / growth} - - geoip usage-by-country reporting and aggregation - (Once that's working, switch to Directory guards) -\section{Performance research} -\subsection{Load balance better} -\subsection{Improve our congestion control algorithms} -\subsection{Two-hops vs Three-hops} -\subsection{Transport IP packets end-to-end} -\section{Outreach and user education} -\subsection{"Who uses Tor" use cases} -\subsection{Law enforcement contacts} - - "Was this IP address a Tor relay recently?" database -\subsection{Commercial/enterprise outreach. Help them use Tor well and - not fear it.} -\subsection{NGO outreach and training.} - - "How to be a safe blogger" -\subsection{More activist coordinators, more people to answer user questions} -\subsection{More people to hold hands of server operators} -\subsection{Teaching the media about Tor} -\subsection{The-dangers-of-plaintext awareness} -\subsection{check.torproject.org and other "privacy checkers"} -\subsection{Stronger legal FAQ for US} -\subsection{Legal FAQs for other countries} -\section{Anonymity research} -\subsection{estimate relay bandwidth more securely} -\subsection{website fingerprinting attacks} -\subsection{safer e2e defenses} -\subsection{Using Tor when you really need anonymity. Can you compose it - with other steps, like more trusted guards or separate proxies?} -\subsection{Topology-aware routing; routing-zones, steven's pet2007 paper.} -\subsection{Exactly what do guard nodes provide?} - -Entry guards seem to defend against all sorts of attacks. Can we work -through all the benefits they provide? Papers like Nikita's CCS 2007 -paper make me think their value is not well-understood by the research -community. - -\section{Organizational growth and stability} -\subsection{A contingency plan if Roger gets hit by a bus} - - Get a new executive director -\subsection{More diversity of funding} - - Don't rely on any one funder as much - - Don't rely on any sector or funder category as much -\subsection{More Tor-funded people who are skilled at peripheral apps like - Vidalia, Torbutton, Polipo, etc} -\subsection{More coordinated media handling and strategy} -\subsection{Clearer and more predictable trademark behavior} -\subsection{More outside funding for internships, etc e.g. GSoC.} -\section{Hidden services} -\subsection{Scaling: how to handle many hidden services} -\subsection{Performance: how to rendezvous with them quickly} -\subsection{Authentication/authorization: how to tolerate DoS / load} -\section{Tor as a general overlay network} -\subsection{Choose paths / exit by country} -\subsection{Easier to run your own private servers and have Tor use them - anywhere in the path} -\subsection{Easier to run an independent Tor network} -\section{Code security/correctness} -\subsection{veracode} -\subsection{code audit} -\subsection{more fuzzing tools} -\subsection{build farm, better testing harness} -\subsection{Long-overdue code refactoring and cleanup} -\section{Protocol security} -\subsection{safer circuit handshake} -\subsection{protocol versioning for future compatibility} -\subsection{cell sizes} -\subsection{adapt to new key sizes, etc} - -\bibliographystyle{plain} \bibliography{tor-design} - -\end{document} - - - - -\section{Code and design infrastructure} - -\subsection{Protocol revision} -To maintain backward compatibility, we've postponed major protocol -changes and redesigns for a long time. Because of this, there are a number -of sensible revisions we've been putting off until we could deploy several of -them at once. To do each of these, we first need to discuss design -alternatives with other cryptographers and outside collaborators to -make sure that our choices are secure. - -First of all, our protocol needs better {\bf versioning support} so that we -can make backward-incompatible changes to our core protocol. There are -difficult anonymity issues here, since many naive designs would make it easy -to tell clients apart (and then track them) based on their supported versions. - -With protocol versioning support would come the ability to {\bf future-proof - our ciphersuites}. For example, not only our OR protocol, but also our -directory protocol, is pretty firmly tied to the SHA-1 hash function, which -though not yet known to be insecure for our purposes, has begun to show -its age. We should -remove assumptions throughout our design based on the assumption that public -keys, secret keys, or digests will remain any particular size indefinitely. - -Our OR {\bf authentication protocol}, though provably -secure\cite{tap:pet2006}, relies more on particular aspects of RSA and our -implementation thereof than we had initially believed. To future-proof -against changes, we should replace it with a less delicate approach. - -\plan{For all the above: 2 person-months to specify, spread over several - months with time for interaction with external participants. One - person-month to implement. Start specifying in early 2007.} - -We might design a {\bf stream migration} feature so that streams tunneled -over Tor could be more resilient to dropped connections and changed IPs. -\plan{Not in 2007.} - -A new protocol could support {\bf multiple cell sizes}. Right now, all data -passes through the Tor network divided into 512-byte cells. This is -efficient for high-bandwidth protocols, but inefficient for protocols -like SSH or AIM that send information in small chunks. Of course, we need to -investigate the extent to which multiple sizes could make it easier for an -adversary to fingerprint a traffic pattern. \plan{Not in 2007.} - -As a part of our design, we should investigate possible {\bf cipher modes} -other than counter mode. For example, a mode with built-in integrity -checking, error propagation, and random access could simplify our protocol -significantly. Sadly, many of these are patented and unavailable for us. -\plan{Not in 2007.} - -\subsection{Scalability} - -\subsubsection{Improved directory efficiency} - -We should {\bf have routers upload their descriptors even less often}, so -that clients do not need to download replacements every 18 hours whether any -information has changed or not. (As of Tor 0.1.2.3-alpha, clients tolerate -routers that don't upload often, but routers still upload at least every 18 -hours to support older clients.) \plan{Must do, but not until 0.1.1.x is -deprecated in mid 2007. 1 week.} - -\subsubsection{Non-clique topology} -Our current network design achieves a certain amount of its anonymity by -making clients act like each other through the simple expedient of making -sure that all clients know all servers, and that any server can talk to any -other server. But as the number of servers increases to serve an -ever-greater number of clients, these assumptions become impractical. - -At worst, if these scalability issues become troubling before a solution is -found, we can design and build a solution to {\bf split the network into -multiple slices} until a better solution comes along. This is not ideal, -since rather than looking like all other users from a point of view of path -selection, users would ``only'' look like 200,000--300,000 other -users.\plan{Not unless needed.} - -We are in the process of designing {\bf improved schemes for network - scalability}. Some approaches focus on limiting what an adversary can know -about what a user knows; others focus on reducing the extent to which an -adversary can exploit this knowledge. These are currently in their infancy, -and will probably not be needed in 2007, but they must be designed in 2007 if -they are to be deployed in 2008.\plan{Design in 2007; unknown difficulty. - Write a paper.} - -\subsubsection{Relay incentives} -To support more users on the network, we need to get more servers. So far, -we've relied on volunteerism to attract server operators, and so far it's -served us well. But in the long run, we need to {\bf design incentives for - users to run servers} and relay traffic for others. Most obviously, we -could try to build the network so that servers offered improved service for -other servers, but we would need to do so without weakening anonymity and -making it obvious which connections originate from users running servers. We -have some preliminary designs~\cite{incentives-txt,tor-challenges}, -but need to perform -some more research to make sure they would be safe and effective.\plan{Write - a draft paper; 2 person-months.} -(XXX we did that) - -\subsection{Portability} -Our {\bf Windows implementation}, though much improved, continues to lag -behind Unix and Mac OS X, especially when running as a server. We hope to -merge promising patches from Christian King to address this point, and bring -Windows performance on par with other platforms.\plan{Do in 2007; 1.5 months - to integrate not counting Mike's work.} - -We should have {\bf better support for portable devices}, including modes of -operation that require less RAM, and that write to disk less frequently (to -avoid wearing out flash RAM).\plan{Optional; 2 weeks.} - -\subsection{Performance: resource usage} -We've been working on {\bf using less RAM}, especially on servers. This has -paid off a lot for directory caches in the 0.1.2, which in some cases are -using 90\% less memory than they used to require. But we can do better, -especially in the area around our buffer management algorithms, by using an -approach more like the BSD and Linux kernels use instead of our current ring -buffer approach. (For OR connections, we can just use queues of cell-sized -chunks produced with a specialized allocator.) This could potentially save -around 25 to 50\% of the memory currently allocated for network buffers, and -make Tor a more attractive proposition for restricted-memory environments -like old computers, mobile devices, and the like.\plan{Do in 2007; 2-3 weeks - plus one week measurement.} (XXX We did this, but we need to do something -more/else.) - -\subsection{Performance: network usage} -We know too little about how well our current path -selection algorithms actually spread traffic around the network in practice. -We should {\bf research the efficacy of our traffic allocation} and either -assure ourselves that it is close enough to optimal as to need no improvement -(unlikely) or {\bf identify ways to improve network usage}, and get more -users' traffic delivered faster. Performing this research will require -careful thought about anonymity implications. - -We should also {\bf examine the efficacy of our congestion control - algorithm}, and see whether we can improve client performance in the -presence of a congested network through dynamic `sendme' window sizes or -other means. This will have anonymity implications too if we aren't careful. - -\plan{For both of the above: research, design and write - a measurement tool in 2007: 1 month. See if we can interest a graduate - student.} - -We should work on making Tor's cell-based protocol perform better on -networks with low bandwidth -and high packet loss.\plan{Do in 2007 if we're funded to do it; 4-6 weeks.} - -\subsection{Performance scenario: one Tor client, many users} -We should {\bf improve Tor's performance when a single Tor handles many - clients}. Many organizations want to manage a single Tor client on their -firewall for many users, rather than having each user install a separate -Tor client. We haven't optimized for this scenario, and it is likely that -there are some code paths in the current implementation that become -inefficient when a single Tor is servicing hundreds or thousands of client -connections. (Additionally, it is likely that such clients have interesting -anonymity requirements the we should investigate.) We should profile Tor -under appropriate loads, identify bottlenecks, and fix them.\plan{Do in 2007 - if we're funded to do it; 4-8 weeks.} - -\subsection{Tor servers on asymmetric bandwidth} - -Tor should work better on servers that have asymmetric connections like cable -or DSL. Because Tor has separate TCP connections between each -hop, if the incoming bytes are arriving just fine and the outgoing bytes are -all getting dropped on the floor, the TCP push-back mechanisms don't really -transmit this information back to the incoming streams.\plan{Do in 2007 since - related to bandwidth limiting. 3-4 weeks.} - -\subsection{Running Tor as both client and server} - -Many performance tradeoffs and balances that might need more attention. -We first need to track and fix whatever bottlenecks emerge; but we also -need to invent good algorithms for prioritizing the client's traffic -without starving the server's traffic too much.\plan{No idea; try -profiling and improving things in 2007.} - -\subsection{Protocol redesign for UDP} -Tor has relayed only TCP traffic since its first versions, and has used -TLS-over-TCP to do so. This approach has proved reliable and flexible, but -in the long term we will need to allow UDP traffic on the network, and switch -some or all of the network to using a UDP transport. {\bf Supporting UDP - traffic} will make Tor more suitable for protocols that require UDP, such -as many VOIP protocols. {\bf Using a UDP transport} could greatly reduce -resource limitations on servers, and make the network far less interruptible -by lossy connections. Either of these protocol changes would require a great -deal of design work, however. We hope to be able to enlist the aid of a few -talented graduate students to assist with the initial design and -specification, but the actual implementation will require significant testing -of different reliable transport approaches.\plan{Maybe do a design in 2007 if -we find an interested academic. Ian or Ben L might be good partners here.} - -\section{Blocking resistance} - -\subsection{Design for blocking resistance} -We have written a design document explaining our general approach to blocking -resistance. We should workshop it with other experts in the field to get -their ideas about how we can improve Tor's efficacy as an anti-censorship -tool. - -\subsection{Implementation: client-side and bridges-side} - -Bridges will want to be able to {\bf listen on multiple addresses and ports} -if they can, to give the adversary more ports to block. - -\subsection{Research: anonymity implications from becoming a bridge} - -see arma's bridge proposal; e.g. should bridge users use a second layer of -entry guards? - -\subsection{Implementation: bridge authority} - -we run some -directory authorities with a slightly modified protocol that doesn't leak -the entire list of bridges. Thus users can learn up-to-date information -for bridges they already know about, but they can't learn about arbitrary -new bridges. - -we need a design for distributing the bridge authority over more than one -server - -\subsection{Normalizing the Tor protocol on the wire} -Additionally, we should {\bf resist content-based filters}. Though an -adversary can't see what users are saying, some aspects of our protocol are -easy to fingerprint {\em as} Tor. We should correct this where possible. - -Look like Firefox; or look like nothing? -Future research: investigate timing similarities with other protocols. - -\subsection{Research: scanning-resistance} - -\subsection{Research/Design/Impl: how users discover bridges} -Our design anticipates an arms race between discovery methods and censors. -We need to begin the infrastructure on our side quickly, preferably in a -flexible language like Python, so we can adapt quickly to censorship. - -phase one: personal bridges -phase two: families of personal bridges -phase three: more structured social network -phase four: bag of tricks -Research: phase five... - -Integration with Psiphon, etc? - -\subsection{Document best practices for users} -Document best practices for various activities common among -blocked users (e.g. WordPress use). - -\subsection{Research: how to know if a bridge has been blocked?} - -\subsection{GeoIP maintenance, and "private" user statistics} -How to know if the whole idea is working? - -\subsection{Research: hiding whether the user is reading or publishing?} - -\subsection{Research: how many bridges do you need to know to maintain -reachability?} - -\subsection{Resisting censorship of the Tor website, docs, and mirrors} - -We should take some effort to consider {\bf initial distribution of Tor and - related information} in countries where the Tor website and mirrors are -censored. (Right now, most countries that block access to Tor block only the -main website and leave mirrors and the network itself untouched.) Falling -back on word-of-mouth is always a good last resort, but we should also take -steps to make sure it's relatively easy for users to get ahold of a copy. - -\section{Security} - -\subsection{Security research projects} - -We should investigate approaches with some promise to help Tor resist -end-to-end traffic correlation attacks. It's an open research question -whether (and to what extent) {\bf mixed-latency} networks, {\bf low-volume - long-distance padding}, or other approaches can resist these attacks, which -are currently some of the most effective against careful Tor users. We -should research these questions and perform simulations to identify -opportunities for strengthening our design without dropping performance to -unacceptable levels. %Cite something -\plan{Start doing this in 2007; write a paper. 8-16 weeks.} - -We've got some preliminary results suggesting that {\bf a topology-aware - routing algorithm}~\cite{feamster:wpes2004} could reduce Tor users' -vulnerability against local or ISP-level adversaries, by ensuring that they -are never in a position to watch both ends of a connection. We need to -examine the effects of this approach in more detail and consider side-effects -on anonymity against other kinds of adversaries. If the approach still looks -promising, we should investigate ways for clients to implement it (or an -approximation of it) without having to download routing tables for the whole -Internet. \plan{Not in 2007 unless a graduate student wants to do it.} - -%\tmp{defenses against end-to-end correlation} We don't expect any to work -%right now, but it would be useful to learn that one did. Alternatively, -%proving that one didn't would free up researchers in the field to go work on -%other things. -% -% See above; I think I got this. - -We should research the efficacy of {\bf website fingerprinting} attacks, -wherein an adversary tries to match the distinctive traffic and timing -pattern of the resources constituting a given website to the traffic pattern -of a user's client. These attacks work great in simulations, but in -practice we hear they don't work nearly as well. We should get some actual -numbers to investigate the issue, and figure out what's going on. If we -resist these attacks, or can improve our design to resist them, we should. -% add cites -\plan{Possibly part of end-to-end correlation paper. Otherwise, not in 2007 - unless a graduate student is interested.} - -\subsection{Implementation security} - -We should also {\bf mark RAM that holds key material as non-swappable} so -that there is no risk of recovering key material from a hard disk -compromise. This would require submitting patches upstream to OpenSSL, where -support for marking memory as sensitive is currently in a very preliminary -state.\plan{Nice to do, but not in immediate Tor scope.} - -There are numerous tools for identifying trouble spots in code (such as -Coverity or even VS2005's code analysis tool) and we should convince somebody -to run some of them against the Tor codebase. Ideally, we could figure out a -way to get our code checked periodically rather than just once.\plan{Almost - no time once we talk somebody into it.} - -We should try {\bf protocol fuzzing} to identify errors in our -implementation.\plan{Not in 2007 unless we find a grad student or - undergraduate who wants to try.} - -Our guard nodes help prevent an attacker from being able to become a chosen -client's entry point by having each client choose a few favorite entry points -as ``guards'' and stick to them. We should implement a {\bf directory - guards} feature to keep adversaries from enumerating Tor users by acting as -a directory cache.\plan{Do in 2007; 2 weeks.} - -\subsection{Detect corrupt exits and other servers} -With the success of our network, we've attracted servers in many locations, -operated by many kinds of people. Unfortunately, some of these locations -have compromised or defective networks, and some of these people are -untrustworthy or incompetent. Our current design relies on authority -administrators to identify bad nodes and mark them as nonfunctioning. We -should {\bf automate the process of identifying malfunctioning nodes} as -follows: - -We should create a generic {\bf feedback mechanism for add-on tools} like -Mike Perry's ``Snakes on a Tor'' to report failing nodes to authorities. -\plan{Do in 2006; 1-2 weeks.} - -We should write tools to {\bf detect more kinds of innocent node failure}, -such as nodes whose network providers intercept SSL, nodes whose network -providers censor popular websites, and so on. We should also try to detect -{\bf routers that snoop traffic}; we could do this by launching connections -to throwaway accounts, and seeing which accounts get used.\plan{Do in 2007; - ask Mike Perry if he's interested. 4-6 weeks.} - -We should add {\bf an efficient way for authorities to mark a set of servers - as probably collaborating} though not necessarily otherwise dishonest. -This happens when an administrator starts multiple routers, but doesn't mark -them as belonging to the same family.\plan{Do during v2.1 directory protocol - redesign; 1-2 weeks to implement.} - -To avoid attacks where an adversary claims good performance in order to -attract traffic, we should {\bf have authorities measure node performance} -(including stability and bandwidth) themselves, and not simply believe what -they're told. We also measure stability by tracking MTBF. Measuring -bandwidth will be tricky, since it's hard to distinguish between a server with -low capacity, and a high-capacity server with most of its capacity in -use. See also Nikita's NDSS 2008 paper.\plan{Do it if we can interest -a grad student.} - -{\bf Operating a directory authority should be easier.} We rely on authority -operators to keep the network running well, but right now their job involves -too much busywork and administrative overhead. A better interface for them -to use could free their time to work on exception cases rather than on -adding named nodes to the network.\plan{Do in 2007; 4-5 weeks.} - -\subsection{Protocol security} - -In addition to other protocol changes discussed above, -% And should we move some of them down here? -NM -we should add {\bf hooks for denial-of-service resistance}; we have some -preliminary designs, but we shouldn't postpone them until we really need them. -If somebody tries a DDoS attack against the Tor network, we won't want to -wait for all the servers and clients to upgrade to a new -version.\plan{Research project; do this in 2007 if funded.} - -\section{Development infrastructure} - -\subsection{Build farm} -We've begun to deploy a cross-platform distributed build farm of hosts -that build and test the Tor source every time it changes in our development -repository. - -We need to {\bf get more participants}, so that we can test a larger variety -of platforms. (Previously, we've only found out when our code had broken on -obscure platforms when somebody got around to building it.) - -We need also to {\bf add our dependencies} to the build farm, so that we can -ensure that libraries we need (especially libevent) do not stop working on -any important platform between one release and the next. - -\plan{This is ongoing as more buildbots arrive.} - -\subsection{Improved testing harness} -Currently, our {\bf unit tests} cover only about 20\% of the code base. This -is uncomfortably low; we should write more and switch to a more flexible -testing framework.\plan{Ongoing basis, time permitting.} - -We should also write flexible {\bf automated single-host deployment tests} so -we can more easily verify that the current codebase works with the -network.\plan{Worthwhile in 2007; would save lots of time. 2-4 weeks.} - -We should build automated {\bf stress testing} frameworks so we can see which -realistic loads cause Tor to perform badly, and regularly profile Tor against -these loads. This would give us {\it in vitro} performance values to -supplement our deployment experience.\plan{Worthwhile in 2007; 2-6 weeks.} - -We should improve our memory profiling code.\plan{...} - - -\subsection{Centralized build system} -We currently rely on a separate packager to maintain the packaging system and -to build Tor on each platform for which we distribute binaries. Separate -package maintainers is sensible, but separate package builders has meant -long turnaround times between source releases and package releases. We -should create the necessary infrastructure for us to produce binaries for all -major packages within an hour or so of source release.\plan{We should - brainstorm this at least in 2007.} - -\subsection{Improved metrics} -We need a way to {\bf measure the network's health, capacity, and degree of - utilization}. Our current means for doing this are ad hoc and not -completely accurate - -We need better ways to {\bf tell which countries are users are coming from, - and how many there are}. A good perspective of the network helps us -allocate resources and identify trouble spots, but our current approaches -will work less and less well as we make it harder for adversaries to -enumerate users. We'll probably want to shift to a smarter, statistical -approach rather than our current ``count and extrapolate'' method. - -\plan{All of this in 2007 if funded; 4-8 weeks} - -% \tmp{We'd like to know how much of the network is getting used.} -% I think this is covered above -NM - -\subsection{Controller library} -We've done lots of design and development on our controller interface, which -allows UI applications and other tools to interact with Tor. We could -encourage the development of more such tools by releasing a {\bf - general-purpose controller library}, ideally with API support for several -popular programming languages.\plan{2006 or 2007; 1-2 weeks.} - -\section{User experience} - -\subsection{Get blocked less, get blocked less broadly} -Right now, some services block connections from the Tor network because -they don't have a better -way to keep vandals from abusing them than blocking IP addresses associated -with vandalism. Our approach so far has been to educate them about better -solutions that currently exist, but we should also {\bf create better -solutions for limiting vandalism by anonymous users} like credential and -blind-signature based implementations, and encourage their use. Other -promising starting points including writing a patch and explanation for -Wikipedia, and helping Freenode to document, maintain, and expand its -current Tor-friendly position.\plan{Do a writeup here in 2007; 1-2 weeks.} - -Those who do block Tor users also block overbroadly, sometimes blacklisting -operators of Tor servers that do not permit exit to their services. We could -obviate innocent reasons for doing so by designing a {\bf narrowly-targeted Tor - RBL service} so that those who wanted to overblock Tor could no longer -plead incompetence.\plan{Possibly in 2007 if we decide it's a good idea; 3 - weeks.} - -\subsection{All-in-one bundle} -We need a well-tested, well-documented bundle of Tor and supporting -applications configured to use it correctly. We have an initial -implementation well under way, but it will need additional work in -identifying requisite Firefox extensions, identifying security threats, -improving user experience, and so on. This will need significantly more work -before it's ready for a general public release. - -\subsection{LiveCD Tor} -We need a nice bootable livecd containing a minimal OS and a few applications -configured to use it correctly. The Anonym.OS project demonstrated that this -is quite feasible, but their project is not currently maintained. - -\subsection{A Tor client in a VM} -\tmp{a.k.a JanusVM} which is quite related to the firewall-level deployment -section below. JanusVM is a Linux kernel running in VMWare. It gets an IP -address from the network, and serves as a DHCP server for its host Windows -machine. It intercepts all outgoing traffic and redirects it into Privoxy, -Tor, etc. This Linux-in-Windows approach may help us with scalability in -the short term, and it may also be a good long-term solution rather than -accepting all security risks in Windows. - -%\subsection{Interface improvements} -%\tmp{Allow controllers to manipulate server status.} -% (Why is this in the User Experience section?) -RD -% I think it's better left to a generic ``make controller iface better'' item. - -\subsection{Firewall-level deployment} -Another useful deployment mode for some users is using {\bf Tor in a firewall - configuration}, and directing all their traffic through Tor. This can be a -little tricky to set up currently, but it's an effective way to make sure no -traffic leaves the host un-anonymized. To achieve this, we need to {\bf - improve and port our new TransPort} feature which allows Tor to be used -without SOCKS support; to {\bf add an anonymizing DNS proxy} feature to Tor; -and to {\bf construct a recommended set of firewall configurations} to redirect -traffic to Tor. - -This is an area where {\bf deployment via a livecd}, or an installation -targeted at specialized home routing hardware, could be useful. - -\subsection{Assess software and configurations for anonymity risks} -Right now, users and packagers are more or less on their own when selecting -Firefox extensions. We should {\bf assemble a recommended list of browser - extensions} through experiment, and include this in the application bundles -we distribute. - -We should also describe {\bf best practices for using Tor with each class of - application}. For example, Ethan Zuckerman has written a detailed -tutorial on how to use Tor, Firefox, GMail, and Wordpress to blog with -improved safety. There are many other cases on the Internet where anonymity -would be helpful, and there are a lot of ways to screw up using Tor. - -The Foxtor and Torbutton extensions serve similar purposes; we should pick a -favorite, and merge in the useful features of the other. - -%\tmp{clean up our own bundled software: -%E.g. Merge the good features of Foxtor into Torbutton} -% -% What else did you have in mind? -NM - -\subsection{Localization} -Right now, most of our user-facing code is internationalized. We need to -internationalize the last few hold-outs (like the Tor expert installer), and get -more translations for the parts that are already internationalized. - -Also, we should look into a {\bf unified translator's solution}. Currently, -since different tools have been internationalized using the -framework-appropriate method, different tools require translators to localize -them via different interfaces. Inasmuch as possible, we should make -translators only need to use a single tool to translate the whole Tor suite. - -\section{Support} - -It would be nice to set up some {\bf user support infrastructure} and -{\bf contributor support infrastructure}, especially focusing on server -operators and on coordinating volunteers. - -This includes intuitive and easy ticket systems for bug reports and -feature suggestions (not just mailing lists with a half dozen people -and no clear roles for who answers what), but it also includes a more -personalized and efficient framework for interaction so we keep the -attention and interest of the contributors, and so we make them feel -helpful and wanted. - -\section{Documentation} - -\subsection{Unified documentation scheme} - -We need to {\bf inventory our documentation.} Our documentation so far has -been mostly produced on an {\it ad hoc} basis, in response to particular -needs and requests. We should figure out what documentation we have, which of -it (if any) should get priority, and whether we can't put it all into a -single format. - -We could {\bf unify the docs} into a single book-like thing. This will also -help us identify what sections of the ``book'' are missing. - -\subsection{Missing technical documentation} - -We should {\bf revise our design paper} to reflect the new decisions and -research we've made since it was published in 2004. This will help other -researchers evaluate and suggest improvements to Tor's current design. - -Other projects sometimes implement the client side of our protocol. We -encourage this, but we should write {\bf a document about how to avoid -excessive resource use}, so we don't need to worry that they will do so -without regard to the effect of their choices on server resources. - -\subsection{Missing user documentation} - -Our documentation falls into two broad categories: some is `discoursive' and -explains in detail why users should take certain actions, and other -documentation is `comprehensive' and describes all of Tor's features. Right -now, we have no document that is both deep, readable, and thorough. We -should correct this by identifying missing spots in our design. - -\bibliographystyle{plain} \bibliography{tor-design} - -\end{document} - diff --git a/doc/rump-fc04.mgp b/doc/rump-fc04.mgp deleted file mode 100644 index efbf6c840..000000000 --- a/doc/rump-fc04.mgp +++ /dev/null @@ -1,175 +0,0 @@ -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%deffont "standard" xfont "comic sans ms-medium-r" -%%deffont "thick" xfont "arial black-medium-r" -%%deffont "typewriter" xfont "courier new-bold-r" -%%deffont "type2writer" xfont "arial narrow-bold-r" -%%deffont "standard" tfont "standard.ttf", tmfont "kochi-mincho.ttf" -%%deffont "thick" tfont "thick.ttf", tmfont "goth.ttf" -%%deffont "typewriter" tfont "typewriter.ttf", tmfont "goth.ttf" -%deffont "standard" xfont "helvetica-medium-r", tfont "arial.ttf", tmfont "times.ttf" -%deffont "thick" xfont "helvetica-bold-r", tfont "arialbd.ttf", tmfont "hoso6.ttf" -%deffont "italic" xfont "helvetica-italic-r", tfont "ariali.ttf", tmfont "hoso6.ttf" -%deffont "typewriter" xfont "courier-medium-r", tfont "typewriter.ttf", tmfont "hoso6.ttf" -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% -%% Default settings per each line numbers. -%% -%default 1 leftfill, size 8, fore "black", back "white", font "thick", hgap 1 -%default 2 size 8, vgap 10, prefix " ", ccolor "black" -%default 3 size 6, bar "gray70", vgap 0 -%default 4 size 6, fore "black", vgap 0, prefix " ", font "standard" -%% -%%default 1 area 90 90, leftfill, size 9, fore "yellow", back "blue", font "thick" -%%default 2 size 9, vgap 10, prefix " " -%%default 3 size 7, bar "gray70", vgap 10 -%%default 4 size 7, vgap 30, prefix " ", font "standard" -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% -%% Default settings that are applied to TAB-indented lines. -%% -%tab 1 size 5, vgap 40, prefix " ", icon arc "red" 50 -%tab 2 size 4, vgap 35, prefix " ", icon delta3 "blue" 40 -%tab 3 size 3, vgap 35, prefix " ", icon dia "DarkViolet" 40 -%% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page -%nodefault -%center, size 9, font "thick", back "white", fore "black" - - - -Tor: -%size 8 -Next-generation Onion Routing - - -%size 7 -Roger Dingledine -Nick Mathewson -Paul Syverson - -%%The Free Haven Project -%%%font "typewriter", fore "blue" -%%http://freehaven.net/ -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Low-latency anonymity system - -%leftfill -Deployed: 19 nodes, hundreds of users (?) - -Many improvements on earlier design - -Free software -- available source code - -Design is not covered by earlier onion routing -patent - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Perfect forward secrecy - - -Telescoping circuit - - negotiates keys at each hop - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%page -%% -%%Separation from "protocol cleaning" -%% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -No mixing, padding, traffic shaping (yet) - - -Please show us they're worth the usability tradeoff - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%page -%% -%%Many TCP streams can share one circuit -%% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Congestion control - - -Simple rate limiting - -Plus have to keep internal nodes from overflowing - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Directory servers - - -Approve new servers - -Tell clients who's up right now - - plus their keys, location, etc - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Variable exit policies - - -Each server allows different outgoing connections - -E.g. no servers allow outgoing mail currently - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -End-to-end integrity checking - - -In previous onion routing, an insider could change -the text being transmitted: - -"dir" => "rm *" - -Even an external adversary could do this! - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Rendezvous points - - -allow hidden services - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -Differences / limitations - - -We're TCP-only, not all IP (but we're user-space and very portable) - -Not peer-to-peer - -No protocol normalization - -%%Not unobservable - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%page - -We have working code - - -Plus a design document, -and a byte-level specification - -%size 9 -http://freehaven.net/tor/ - diff --git a/doc/tor-gencert.1 b/doc/tor-gencert.1 deleted file mode 100644 index 5bcb5f0c3..000000000 --- a/doc/tor-gencert.1 +++ /dev/null @@ -1,86 +0,0 @@ -.TH tor-gencert 1 "" Jan-2008 "" -.\" manual page by Nick Mathewson -.SH NAME -.LP -tor-gencert \- Generate certs and keys for Tor directory authorities - -.SH SYNOPSIS -\fBtor-gencert\fP\ [-h|--help] [-v] [-r|--reuse] [--create-identity-key] [-i \fIid_file\fP] [-c \fIcert_file\fP] [-m \fInum\fP] [-a \fIaddress\fP:\fIport\fP] - -.SH DESCRIPTION -\fBtor-gencert\fR generates certificates and private keys for use by Tor -directory authorities running the v3 Tor directory protocol, as used by Tor -0.2.0 and later. If you are not running a directory authority, you don't -need to use tor-gencert. -.PP -Every directory authority has a long term authority \fIidentity key\fP (which -is distinct from the identity key it uses as a Tor server); this key should -be kept offline in a secure location. It is used to certify shorter-lived -\fIsigning keys\fP, which are kept online and used by the directory authority -to sign votes and consensus documents. -.PP -After you use this program to generate a signing key and a certificate, copy -those files to the keys subdirectory of your Tor process, and send Tor a -SIGHUP signal. DO NOT COPY THE IDENTITY KEY. - -.SH OPTIONS -\fB-v\fP -Display verbose output. -.LP -.TP -\fB-h\fP or \fB--help\fP -Display help text and exit. -.LP -.TP -\fB-r\fP or \fB--reuse\fP -Generate a new certificate, but not a new signing key. This can be -used to change the address or lifetime associated with a given key. -.LP -.TP -\fB--create-identity-key\fP -Generate a new identity key. You should only use this option the first -time you run tor-gencert; in the future, you should use the identity -key that's already there. -.LP -.TP -\fB-i \fR\fIFILENAME\fP -Read the identity key from the specified file. If the file is not present -and --create-identity-key is provided, create the identity key in the -specified file. Default: "./authority_identity_key" -.LP -.TP -\fB-s \fR\fIFILENAME\fP -Write the signing key to the specified file. Default: -"./authority_signing_key" -.LP -.TP -\fB-c \fR\fIFILENAME\fP -Write the certificate to the specified file. -Default: "./authority_certificate" -.LP -.TP -\fB-m \fR\fINUM\fP -Number of months that the certificate should be valid. Default: 12. -.LP -.TP -\fB--passphrase-fd \fR\fIFILEDES\fP -Filedescriptor to read the file descriptor from. Ends at the first -NUL or newline. Default: read from the terminal. -.LP -.TP -\fB-a \fR\fIaddress\fR:\fIport\fP -If provided, advertise the address:port combination as this authority's -preferred directory port in its certificate. If the address is a hostname, -the hostname is resolved to an IP before it's published. - -.SH BUGS -This probably doesn't run on Windows. That's not a big issue, since we -don't really want authorities to be running on Windows anyway. - -.SH SEE ALSO -.BR tor (1) -.PP -See also the "dir-spec.txt" file, distributed with Tor. - -.SH AUTHORS -Roger Dingledine <arma@mit.edu>, Nick Mathewson <nickm@alum.mit.edu>. diff --git a/doc/tor-gencert.1.txt b/doc/tor-gencert.1.txt new file mode 100644 index 000000000..2a2d1179c --- /dev/null +++ b/doc/tor-gencert.1.txt @@ -0,0 +1,90 @@ +// Copyright (c) The Tor Project, Inc. +// See LICENSE for licensing information +// This is an asciidoc file used to generate the manpage/html reference. +// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +tor-gencert(1) +============== +Nick Mathewson + +NAME +---- +tor-gencert - Generate certs and keys for Tor directory authorities + +SYNOPSIS +-------- +**tor-gencert** [-h|--help] [-v] [-r|--reuse] [--create-identity-key] [-i __id_file__] [-c +__cert_file__] [-m __num__] [-a __address__:__port__] + +DESCRIPTION +----------- +**tor-gencert** generates certificates and private keys for use by Tor +directory authorities running the v3 Tor directory protocol, as used by +Tor 0.2.0 and later. If you are not running a directory authority, you +don't need to use tor-gencert. + + +Every directory authority has a long term authority __identity__ __key__ (which +is distinct from the identity key it uses as a Tor server); this key +should be kept offline in a secure location. It is used to certify +shorter-lived __signing__ __keys__, which are kept online and used by the +directory authority to sign votes and consensus documents. + + +After you use this program to generate a signing key and a certificate, +copy those files to the keys subdirectory of your Tor process, and send +Tor a SIGHUP signal. DO NOT COPY THE IDENTITY KEY. + +OPTIONS +------- +**-v**:: + Display verbose output. + +**-h** or **--help**:: + Display help text and exit. + +**-r** or **--reuse**:: + Generate a new certificate, but not a new signing key. This can be used to + change the address or lifetime associated with a given key. + +**--create-identity-key**:: + Generate a new identity key. You should only use this option the first time + you run tor-gencert; in the future, you should use the identity key that's + already there. + +**-i** __FILENAME__:: + Read the identity key from the specified file. If the file is not present + and --create-identity-key is provided, create the identity key in the + specified file. Default: "./authority_identity_key" + +**-s** __FILENAME__:: + Write the signing key to the specified file. Default: + "./authority_signing_key" + +**-c** __FILENAME__:: + Write the certificate to the specified file. Default: + "./authority_certificate" + +**-m** __NUM__:: + Number of months that the certificate should be valid. Default: 12. + +**--passphrase-fd** __FILEDES__:: + Filedescriptor to read the file descriptor from. Ends at the first NUL or + newline. Default: read from the terminal. + +**-a** __address__:__port__:: + If provided, advertise the address:port combination as this authority's + preferred directory port in its certificate. If the address is a hostname, + the hostname is resolved to an IP before it's published. + +BUGS +---- +This probably doesn't run on Windows. That's not a big issue, since we don't +really want authorities to be running on Windows anyway. + +SEE ALSO +-------- +**tor**(1) + + +See also the "dir-spec.txt" file, distributed with Tor. + +AUTHORS +------- + Roger Dingledine <arma@mit.edu>, Nick Mathewson <nickm@alum.mit.edu>. diff --git a/doc/tor-osx-dmg-creation.txt b/doc/tor-osx-dmg-creation.txt deleted file mode 100644 index 9a89e9875..000000000 --- a/doc/tor-osx-dmg-creation.txt +++ /dev/null @@ -1,145 +0,0 @@ -## Instructions for building the official dmgs for OSX. -## -## The loose table of contents: -## Summary -## Single Architecture Binaries for PPC or X86, not both. -## Backwards compatible single-architecture binaries for OSX x86 10.4 from newer versions of OS X. -## Universal Binaries for OSX PPC and X86 -## Each section is delineated by ###. - -The following steps are the exact steps used to produce the "official" -OSX builds of tor. - -### Summary: -1) Compile and install a static version of the latest release of -libevent. -2) Acquire and install your preferred version of tor. Extract. -3) "make dist-osx" -4) You now have a dmg from which you can install Tor. - -### Single Architecture Binaries for PPC or X86, not both. -### This method works in all versions of OSX 10.3 through 10.6 - -## Compiling libevent ## - -1) Download the latest stable libevent from -http://www.monkey.org/~provos/libevent/ - -2) The first step of compiling libevent is to configure it as -follows: - ./configure --enable-static --disable-shared - -3) Complete the "make" and "make install". You will need to be root, -or sudo -s, to complete the "make install". - -## Compiling Tor ## - -4) Get your preferred version of the tor source from https://www.torproject.org. Extract the -tarball. - -5) In the top level, this means /path/to/tor/, not tor/contrib/osx, -do a configure with these parameters: - CONFDIR=/Library/Tor ./configure --prefix=/Library/Tor \ - --bindir=/Library/Tor --sysconfdir=/Library - -6) In same top level dir, do a "make dist-osx". There now exists a -.dmg file in the same directory. Install from this dmg. - -### Backwards compatible single-architecture binaries for OSX x86 10.4 from newer versions of OS X. - -1) Install the latest XCode updates available from http://developer.apple.com. - -## Compiling libevent ## - -2) Download latest stable libevent from -http://www.monkey.org/~provos/libevent/ - -3) The first step of compiling libevent is to configure it as -follows: -CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386" \ -LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \ -./configure --enable-static --disable-shared --disable-dependency-tracking - -4) Complete the "make" and "make install". You will need to be root, -or sudo -s, to complete the "make install". - -5) Check for a successful universal binary of libevent.a in, by default, -/usr/local/lib by using the following command: - "file /usr/local/lib/libevent.a" - - Your output should be: -/usr/local/lib/libevent.a (for architecture i386): current ar archive random library - -6) Get your preferred version of the tor source from https://www.torproject.org/download. -Extract the tarball. - -7) In the top level, this means /path/to/tor/, not tor/contrib/osx, -do a configure with these parameters: -CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386" \ -LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \ -CONFDIR=/Library/Tor \ -./configure --prefix=/Library/Tor --bindir=/Library/Tor \ ---sysconfdir=/Library --disable-dependency-tracking - -8) "make dist-osx" - -9) Confirm you have created a universal binary by issuing the follow command: -"file src/or/tor". Its output should be as follows: -src/or/tor (for architecture i386): Mach-O executable i386 - -10) There should exist in the top-level directory a -Tor-$VERSION-universal-Bundle.dmg - -11) Congrats. You have a backwards-compatible binary. You are now ready to install Tor. - -### Universal Binaries for OSX PPC and X86 -### This method works in OSX 10.4 (Tiger) and newer OSX versions. - -1) Install the latest XCode updates available from http://developer.apple.com. - -## Compiling libevent ## - -2) Download latest stable libevent from -http://www.monkey.org/~provos/libevent/ - -3) The first step of compiling libevent is to configure it as -follows: -CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" \ -LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \ -./configure --enable-static --disable-shared --disable-dependency-tracking - -4) Complete the "make" and "make install". You will need to be root, -or sudo -s, to complete the "make install". - -5) Check for a successful universal binary of libevent.a in, by default, -/usr/local/lib by using the following command: - "file /usr/local/lib/libevent.a" - - Your output should be: -/usr/local/lib/libevent.a: Mach-O fat file with 2 architectures -/usr/local/lib/libevent.a (for architecture i386): current ar archive random library -/usr/local/lib/libevent.a (for architecture ppc): current ar archive - -6) Get your preferred version of the tor source from https://www.torproject.org/download. -Extract the tarball. - -7) In the top level, this means /path/to/tor/, not tor/contrib/osx, -do a configure with these parameters: -CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc" \ -LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \ -CONFDIR=/Library/Tor \ -./configure --prefix=/Library/Tor --bindir=/Library/Tor \ ---sysconfdir=/Library --disable-dependency-tracking - -8) "make dist-osx" - -9) Confirm you have created a universal binary by issuing the follow command: -"file src/or/tor". Its output should be as follows: -src/or/tor: Mach-O fat file with 2 architectures -src/or/tor (for architecture i386): Mach-O executable i386 -src/or/tor (for architecture ppc): Mach-O executable ppc - -10) There should exist in the top-level directory a -Tor-$VERSION-universal-Bundle.dmg - -11) Congrats. You have a universal binary. You are now ready to install Tor. diff --git a/doc/tor-resolve.1 b/doc/tor-resolve.1 deleted file mode 100644 index 3987095db..000000000 --- a/doc/tor-resolve.1 +++ /dev/null @@ -1,38 +0,0 @@ -.TH tor-resolve 1 "" Aug-2004 "" -.\" manual page by Peter Palfrader -.SH NAME -.LP -tor-resolve \- resolve a hostname to an IP address via tor - -.SH SYNOPSIS -\fBtor-resolve\fP\ [-4|-5] [-v] [-x] \fIhostname\fP\ [\fIsockshost\fP[:\fIsocksport]\fP] - -.SH DESCRIPTION -\fBtor-resolve\fR is a simple script to connect to a SOCKS proxy that -knows about the SOCKS RESOLVE command, hand it a hostname, and return -an IP address. -.SH OPTIONS -\fB-v \fP -Display verbose output. -.LP -.TP -\fB-x\fP -Perform a reverse lookup: get the PTR record for an IPv4 address. -.LP -.TP -\fB-5\fP -Use the SOCKS5 protocol. (Default) -.LP -.TP -\fB-4\fP -Use the SOCKS4a protocol rather than the default SOCKS5 protocol. Doesn't -support reverse DNS. - -.SH SEE ALSO -.BR tor (1), -.BR torify (1). -.PP -See doc/socks-extensions.txt in the Tor package for protocol details. - -.SH AUTHORS -Roger Dingledine <arma@mit.edu>, Nick Mathewson <nickm@alum.mit.edu>. diff --git a/doc/tor-resolve.1.txt b/doc/tor-resolve.1.txt new file mode 100644 index 000000000..bdc636b08 --- /dev/null +++ b/doc/tor-resolve.1.txt @@ -0,0 +1,49 @@ +// Copyright (c) The Tor Project, Inc. +// See LICENSE for licensing information +// This is an asciidoc file used to generate the manpage/html reference. +// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +tor-resolve(1) +============== +Peter Palfrader + +NAME +---- +tor-resolve - resolve a hostname to an IP address via tor + +SYNOPSIS +-------- +**tor-resolve** [-4|-5] [-v] [-x] __hostname__ [__sockshost__[:__socksport__]] + +DESCRIPTION +----------- +**tor-resolve** is a simple script to connect to a SOCKS proxy that knows about +the SOCKS RESOLVE command, hand it a hostname, and return an IP address. + +By default, **tor-resolve** uses the Tor server running on 127.0.0.1 on SOCKS +port 9050. If this isn't what you want, you should specify an explicit +__sockshost__ and/or __socksport__ on the command line. + +OPTIONS +------- +**-v**:: + Display verbose output. + +**-x**:: + Perform a reverse lookup: get the PTR record for an IPv4 address. + +**-5**:: + Use the SOCKS5 protocol. (Default) + +**-4**:: + Use the SOCKS4a protocol rather than the default SOCKS5 protocol. Doesn't + support reverse DNS. + +SEE ALSO +-------- +**tor**(1), **torify**(1). + + +See doc/socks-extensions.txt in the Tor package for protocol details. + +AUTHORS +------- +Roger Dingledine <arma@mit.edu>, Nick Mathewson <nickm@alum.mit.edu>. diff --git a/doc/tor-win32-mingw-creation.txt b/doc/tor-win32-mingw-creation.txt index f31d2741c..4a25e47a8 100644 --- a/doc/tor-win32-mingw-creation.txt +++ b/doc/tor-win32-mingw-creation.txt @@ -1,3 +1,4 @@ +## ## Instructions for building Tor with MinGW (http://www.mingw.org/) ## @@ -5,21 +6,21 @@ Stage One: Download and Install MinGW. --------------------------------------- Download mingw: -http://prdownloads.sf.net/mingw/MinGW-5.1.4.exe?download +http://prdownloads.sf.net/mingw/MinGW-5.1.6.exe?download Download msys: -http://prdownloads.sf.net/mingw/MSYS-1.0.10.exe?download +http://prdownloads.sf.net/ming/MSYS-1.0.11.exe?download + +Download msysDTK: +http://sourceforge.net/projects/mingw/files/MSYS%20Supplementary%20Tools/msysDTK-1.0.1/msysDTK-1.0.1.exe/download -Download the mingw developer tool kit: -http://prdownloads.sf.net/mingw/msysDTK-1.0.1.exe?download +Install MinGW, msysDTK, and MSYS in that order. -Download the mingw autoconf-2.59 update: -http://prdownloads.sf.net/mingw/msys-autoconf-2.59.tar.bz2?download +Make sure your PATH includes C:\MinGW\bin. You can verify this by right +clicking on "My Computer", choose "Properties", choose "Advanced", +choose "Environment Variables", select PATH. -Install mingw, msys and mingw-dtk. Extract msys-autoconf-2.59.tar.bz2 into -your mingw install location. For example, if you installed mingw into -/c/mingw/1.0/ you want to extract msys-autoconf-2.59.tar.bz2 into this -directory. +Start MSYS(rxvt). Create a directory called "tor-mingw". @@ -27,17 +28,17 @@ Stage Two: Download, extract, compile openssl ---------------------------------------------- Download openssl: -http://www.openssl.org/source/openssl-0.9.8k.tar.gz +http://www.openssl.org/source/openssl-0.9.8l.tar.gz Extract openssl: Copy the openssl tarball into the "tor-mingw" directory. Type "cd tor-mingw/" -Type "tar zxf openssl-0.9.8k.tar.gz" +Type "tar zxf openssl-0.9.8l.tar.gz" (Note: There are many symlink errors because Windows doesn't support symlinks. You can ignore these errors.) Make openssl libraries: -Type "cd tor-mingw/openssl-0.9.8k/" +Type "cd tor-mingw/openssl-0.9.8l/" Type "./Configure -no-idea -no-rc5 -no-mdc2 mingw" Edit Makefile and remove the "test:" and "tests:" sections. Type "rm -rf ./test" @@ -47,16 +48,11 @@ Type "cd ../ssl/" Type "find ./ -name "*.h" -exec cp {} ../include/openssl/ \;" Type "cd .." Type "cp *.h include/openssl/" -Type "cp fips/fips.h include/openssl/" +Type "find ./fips -type f -name "*.h" -exec cp {} include/openssl/ \;" # The next steps can take up to 30 minutes to complete. Type "make" Type "make install" -Alternatively: -Download the pre-compiled openssl for win32 from -http://gnuwin32.sourceforge.net/packages/openssl.htm -Install and proceed. - Stage Three: Download, extract, compile zlib --------------------------------------------- @@ -77,13 +73,6 @@ Type "./configure" Type "make" Type "make install" -OR - -Make zlib1.dll: -Type "cd tor-mingw/zlib-1.2.3/" -Type "./configure" -Type "make -f win32/Makefile.gcc" - Done. @@ -128,4 +117,3 @@ From the Tor build directory above, run: "./contrib/package_nsis-mingw.sh" The resulting Tor installer executable is in ./win_tmp/. - diff --git a/doc/tor.1.in b/doc/tor.1.in deleted file mode 100644 index 390e6c4ba..000000000 --- a/doc/tor.1.in +++ /dev/null @@ -1,1519 +0,0 @@ -.TH TOR 1 "January 2009" "TOR" -.SH NAME -tor \- The second-generation onion router -.SH SYNOPSIS -.B tor -[\fIOPTION value\fR]... -.SH DESCRIPTION -.I tor -is a connection-oriented anonymizing communication -service. Users choose a source-routed path through a set of nodes, and -negotiate a "virtual circuit" through the network, in which each node -knows its predecessor and successor, but no others. Traffic flowing down -the circuit is unwrapped by a symmetric key at each node, which reveals -the downstream node. -.PP -Basically \fItor\fR provides a distributed network of servers ("onion -routers"). Users bounce their TCP streams -- web traffic, ftp, ssh, etc -- -around the routers, and recipients, observers, and even the routers -themselves have difficulty tracking the source of the stream. -.SH OPTIONS -\fB-h, -help\fP -Display a short help message and exit. -.LP -.TP -\fB-f \fR\fIFILE\fP -FILE contains further "option value" pairs. (Default: @CONFDIR@/torrc) -.LP -.TP -\fB--hash-password\fP -Generates a hashed password for control port access. -.LP -.TP -\fB--list-fingerprint\fP -Generate your keys and output your nickname and fingerprint. -.LP -.TP -\fB--verify-config\fP -Verify the configuration file is valid. -.LP -.TP -\fB--nt-service\fP -\fB--service [install|remove|start|stop]\fP -Manage the Tor Windows NT/2000/XP service. Current instructions can -be found at http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#WinNTService -.LP -.TP -\fB--list-torrc-options\fP -List all valid options. -.LP -.TP -\fB--version\fP -Display Tor version and exit. -.LP -.TP -\fB--quiet\fP -Do not start Tor with a console log unless explicitly requested to do -so. (By default, Tor starts out logging messages at level "notice" or -higher to the console, until it has parsed its configuration.) -.LP -.TP -Other options can be specified either on the command-line (\fI--option -value\fR), or in the configuration file (\fIoption value\fR or -\fIoption "value"\fR). Options are case-insensitive. C-style escaped -characters are allowed inside quoted values. -.LP -.TP -\fBBandwidthRate \fR\fIN\fR \fBbytes\fR|\fBKB\fR|\fBMB\fR|\fBGB\fR|\fBTB\fP -A token bucket limits the average incoming bandwidth usage on this node -to the specified number of bytes per second, and the average outgoing -bandwidth usage to that same value. (Default: 5 MB) -.LP -.TP -\fBBandwidthBurst \fR\fIN\fR \fBbytes\fR|\fBKB\fR|\fBMB\fR|\fBGB\fR|\fBTB\fP -Limit the maximum token bucket size (also known as the burst) to the -given number of bytes in each direction. (Default: 10 MB) -.LP -.TP -\fBMaxAdvertisedBandwidth \fR\fIN\fR \fBbytes\fR|\fBKB\fR|\fBMB\fR|\fBGB\fR|\fBTB\fP -If set, we will not advertise more than this amount of bandwidth for our -BandwidthRate. Server operators who want to reduce the number of clients -who ask to build circuits through them (since this is proportional to -advertised bandwidth rate) can thus reduce the CPU demands on their -server without impacting network performance. -.LP -.TP -\fBRelayBandwidthRate \fR\fIN\fR \fBbytes\fR|\fBKB\fR|\fBMB\fR|\fBGB\fR|\fBTB\fP -If defined, a separate token bucket limits the average incoming bandwidth -usage for _relayed traffic_ on this node to the specified number of -bytes per second, and the average outgoing bandwidth usage to that same -value. Relayed traffic currently is calculated to include answers to directory -requests, but that may change in future versions. (Default: 0) -.LP -.TP -\fBRelayBandwidthBurst \fR\fIN\fR \fBbytes\fR|\fBKB\fR|\fBMB\fR|\fBGB\fR|\fBTB\fP -Limit the maximum token bucket size (also known as the burst) for -_relayed traffic_ to the -given number of bytes in each direction. (Default: 0) -.LP -.TP -\fBConnLimit \fR\fINUM\fP -The minimum number of file descriptors that must be available to -the Tor process before it will start. Tor will ask the OS for as -many file descriptors as the OS will allow (you can find this -by "ulimit -H -n"). If this number is less than ConnLimit, then -Tor will refuse to start. - -You probably don't need to adjust this. It has no effect on -Windows since that platform lacks getrlimit(). (Default: 1000) -.LP -.TP -\fBConstrainedSockets \fR\fB0\fR|\fB1\fR\fP -If set, Tor will tell the kernel to attempt to shrink the buffers for all -sockets to the size specified in \fBConstrainedSockSize\fP. This is useful -for virtual servers and other environments where system level TCP -buffers may be limited. If you're on a virtual server, and you -encounter the "Error creating network -socket: No buffer space available" message, you are likely experiencing -this problem. - -The preferred solution is to have the admin increase the buffer pool for -the host itself via /proc/sys/net/ipv4/tcp_mem or equivalent facility; this -configuration option is a second-resort. - -The DirPort option should also not be used if TCP buffers are scarce. The -cached directory requests consume additional sockets which exacerbates the -problem. - -You should \fBnot\fP enable this feature unless you encounter the "no buffer -space available" issue. Reducing the TCP buffers affects window size for -the TCP stream and will reduce throughput in proportion to round trip -time on long paths. (Default: 0.) -.LP -.TP -\fBConstrainedSockSize \fR\fIN\fR \fBbytes\fR|\fBKB\fP -When \fBConstrainedSockets\fP is enabled the receive and transmit buffers for -all sockets will be set to this limit. Must be a value between 2048 -and 262144, in 1024 byte increments. Default of 8192 is recommended. -.LP -.TP -\fBControlPort \fR\fIPort\fP -If set, Tor will accept connections on this port and allow those -connections to control the Tor process using the Tor Control Protocol -(described in control-spec.txt). Note: unless you also specify one of -\fBHashedControlPassword\fP or \fBCookieAuthentication\fP, setting -this option will cause Tor to allow any process on the local host to -control it. This option is required for many Tor controllers; most use -the value of 9051. -.LP -.TP -\fBControlListenAddress \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind the controller listener to this address. If you specify a port, -bind to this port rather than the one specified in ControlPort. We -strongly recommend that you leave this alone unless you know what you're -doing, since giving attackers access to your control listener is really -dangerous. (Default: 127.0.0.1) -This directive can be specified multiple times to bind to multiple -addresses/ports. -.LP -.TP -\fBControlSocket \fR\fIPath\fP -Like ControlPort, but listens on a Unix domain socket, rather than a TCP -socket. (Unix and Unix-like systems only.) -.LP -.TP -\fBHashedControlPassword \fR\fIhashed_password\fP -Don't allow any connections on the control port except when the other process -knows the password whose one-way hash is \fIhashed_password\fP. You can -compute the hash of a password by running "tor --hash-password -\fIpassword\fP". You can provide several acceptable passwords by using -more than HashedControlPassword line. -.LP -.TP -\fBCookieAuthentication \fR\fB0\fR|\fB1\fP -If this option is set to 1, don't allow any connections on the control port -except when the connecting process knows the contents of a file named -"control_auth_cookie", which Tor will create in its data directory. This -authentication method should only be used on systems with good filesystem -security. (Default: 0) -.LP -.TP -\fBCookieAuthFile \fR\fIPath\fP -If set, this option overrides the default location and file name for Tor's -cookie file. (See CookieAuthentication above.) -.LP -.TP -\fBCookieAuthFileGroupReadable \fR\fB0\fR|\fB1\fR|\fIGroupName\fP -If this option is set to 0, don't allow the filesystem group to read -the cookie file. If the option is set to 1, make the cookie file -readable by the default GID. [Making the file readable by other -groups is not yet implemented; let us know if you need this for some -reason.] (Default: 0). -.LP -.TP -\fBDataDirectory \fR\fIDIR\fP -Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) -.LP -.TP -\fBDirServer \fR[\fInickname\fR] [\fBflags\fR] \fIaddress\fR\fB:\fIport fingerprint\fP -Use a nonstandard authoritative directory server at the provided -address and port, with the specified key fingerprint. This option can -be repeated many times, for multiple authoritative directory -servers. Flags are separated by spaces, and determine what kind of an -authority this directory is. By default, every authority is authoritative -for current ("v2")-style directories, unless the "no-v2" flag is given. If the "v1" flags is provided, Tor will use this server as an -authority for old-style (v1) directories as well. (Only directory mirrors -care about this.) Tor will use this server as an authority for hidden -service information if the "hs" flag is set, or if the "v1" flag is set and -the "no-hs" flag is \fBnot\fP set. Tor will use this authority as a bridge -authoritative directory if the "bridge" flag is set. If a flag -"orport=\fBport\fR" is given, Tor will use the given port when opening -encrypted tunnels to the dirserver. Lastly, if a flag "v3ident=\fBfp\fR" is -given, the dirserver is a v3 directory authority whose v3 long-term -signing key has the fingerprint \fBfp\fR. - -If no \fBdirserver\fP line is given, Tor will use the default -directory servers. NOTE: this option is intended -for setting up a private Tor network with its own directory authorities. If -you use it, you will be distinguishable from other users, because you won't -believe the same authorities they do. -.LP -.TP -\fBAlternateDirAuthority \fR[\fInickname\fR] [\fBflags\fR] \fIaddress\fR\fB:\fIport fingerprint\fP -.LP -.TP -\fBAlternateHSAuthority \fR[\fInickname\fR] [\fBflags\fR] \fIaddress\fR\fB:\fIport fingerprint\fP -.LP -.TP -\fBAlternateBridgeAuthority \fR[\fInickname\fR] [\fBflags\fR] \fIaddress\fR\fB:\fIport fingerprint\fP -As DirServer, but replaces less of the default directory authorities. -Using AlternateDirAuthority replaces the default Tor directory -authorities, but leaves the hidden service authorities and bridge -authorities in place. Similarly, Using AlternateHSAuthority replaces -the default hidden service authorities, but not the directory or -bridge authorities. -.LP -.TP -\fBFetchDirInfoEarly \fR\fB0\fR|\fB1\fR\fP -If set to 1, Tor will always fetch directory information like other -directory caches, even if you don't meet the normal criteria for -fetching early. Normal users should leave it off. -(Default: 0) -.LP -.TP -\fBFetchHidServDescriptors \fR\fB0\fR|\fB1\fR\fP -If set to 0, Tor will never fetch any hidden service descriptors from -the rendezvous directories. This option is only useful if you're using -a Tor controller that handles hidden service fetches for you. -(Default: 1) -.LP -.TP -\fBFetchServerDescriptors \fR\fB0\fR|\fB1\fR\fP -If set to 0, Tor will never fetch any network status summaries or server -descriptors from the directory servers. This option is only useful if -you're using a Tor controller that handles directory fetches for you. -(Default: 1) -.LP -.TP -\fBFetchUselessDescriptors \fR\fB0\fR|\fB1\fR\fP -If set to 1, Tor will fetch every non-obsolete descriptor from the -authorities that it hears about. Otherwise, it will avoid fetching -useless descriptors, for example for routers that are not running. -This option is useful if you're using the contributed "exitlist" -script to enumerate Tor nodes that exit to certain addresses. -(Default: 0) -.LP -.TP -\fBHTTPProxy\fR \fIhost\fR[:\fIport\fR]\fP -Tor will make all its directory requests through this host:port -(or host:80 if port is not specified), -rather than connecting directly to any directory servers. -.LP -.TP -\fBHTTPProxyAuthenticator\fR \fIusername:password\fP -If defined, Tor will use this username:password for Basic HTTP proxy -authentication, as in RFC 2617. This is currently the only form of -HTTP proxy authentication that Tor supports; feel free to submit a -patch if you want it to support others. -.LP -.TP -\fBHTTPSProxy\fR \fIhost\fR[:\fIport\fR]\fP -Tor will make all its OR (SSL) connections through this host:port -(or host:443 if port is not specified), via HTTP CONNECT rather than -connecting directly to servers. You may want to set \fBFascistFirewall\fR -to restrict the set of ports you might try to connect to, if your HTTPS -proxy only allows connecting to certain ports. -.LP -.TP -\fBHTTPSProxyAuthenticator\fR \fIusername:password\fP -If defined, Tor will use this username:password for Basic HTTPS proxy -authentication, as in RFC 2617. This is currently the only form of -HTTPS proxy authentication that Tor supports; feel free to submit a -patch if you want it to support others. -.LP -.TP -\fBKeepalivePeriod \fR\fINUM\fP -To keep firewalls from expiring connections, send a padding keepalive -cell every NUM seconds on open connections that are in use. If the -connection has no open circuits, it will instead be closed after NUM -seconds of idleness. (Default: 5 minutes) -.LP -.TP -\fBLog \fR\fIminSeverity\fR[-\fImaxSeverity\fR] \fBstderr\fR|\fBstdout\fR|\fBsyslog\fR\fP -Send all messages between \fIminSeverity\fR and \fImaxSeverity\fR to -the standard output stream, the standard error stream, or to the system -log. (The "syslog" value is only supported on Unix.) Recognized -severity levels are debug, info, notice, warn, and err. We advise using -"notice" in most cases, since anything more verbose may provide sensitive -information to an attacker who obtains the logs. If only one -severity level is given, all messages of that level or higher will be -sent to the listed destination. -.LP -.TP -\fBLog \fR\fIminSeverity\fR[-\fImaxSeverity\fR] \fBfile\fR \fIFILENAME\fP -As above, but send log messages to the listed filename. The "Log" -option may appear more than once in a configuration file. Messages -are sent to all the logs that match their severity level. -.LP -.TP -\fBOutboundBindAddress \fR\fIIP\fP -Make all outbound connections originate from the IP address specified. This -is only useful when you have multiple network interfaces, and you want all -of Tor's outgoing connections to use a single one. -.LP -.TP -\fBPidFile \fR\fIFILE\fP -On startup, write our PID to FILE. On clean shutdown, remove FILE. -.LP -.TP -\fBProtocolWarnings \fR\fB0\fR|\fB1\fR\fP -If 1, Tor will log with severity 'warn' various cases of other parties -not following the Tor specification. Otherwise, they are logged with -severity 'info'. (Default: 0) -.LP -.TP -\fBRunAsDaemon \fR\fB0\fR|\fB1\fR\fP -If 1, Tor forks and daemonizes to the background. This option has -no effect on Windows; instead you should use the --service command-line -option. (Default: 0) -.LP -.TP -\fBSafeLogging \fR\fB0\fR|\fB1\fP -If 1, Tor replaces potentially sensitive strings in the logs -(e.g. addresses) with the string [scrubbed]. This way logs can still be -useful, but they don't leave behind personally identifying information -about what sites a user might have visited. (Default: 1) -.LP -.TP -\fBUser \fR\fIUID\fP -On startup, setuid to this user and setgid to their primary group. -.LP -.TP -\fBHardwareAccel \fR\fB0\fR|\fB1\fP -If non-zero, try to use crypto hardware acceleration when -available. This is untested and probably buggy. (Default: 0) -.LP -.TP -\fBAvoidDiskWrites \fR\fB0\fR|\fB1\fP -If non-zero, try to write to disk less frequently than we would otherwise. -This is useful when running on flash memory or other media that support only -a limited number of writes. (Default: 0) -.LP -.TP -\fBTunnelDirConns \fR\fB0\fR|\fB1\fP -If non-zero, when a directory server we contact supports it, we will -build a one-hop circuit and make an encrypted connection via its -ORPort. (Default: 1) -.LP -.TP -\fBPreferTunneledDirConns \fR\fB0\fR|\fB1\fP -If non-zero, we will avoid directory servers that don't support tunneled -directory connections, when possible. (Default: 1) - -.SH CLIENT OPTIONS -.PP -The following options are useful only for clients (that is, if \fBSocksPort\fP is non-zero): -.LP -.TP -\fBAllowInvalidNodes\fR \fBentry\fR|\fBexit\fR|\fBmiddle\fR|\fBintroduction\fR|\fBrendezvous\fR|...\fP -If some Tor servers are obviously not working right, the directory -authorities can manually mark them as invalid, meaning that it's not -recommended you use them for entry or exit positions in your circuits. You -can opt to use them in some circuit positions, though. The default is -"middle,rendezvous", and other choices are not advised. -.LP -.TP -\fBExcludeSingleHopRelays \fR\fB0\fR|\fB1\fR\fP -This option controls whether circuits built by Tor will include relays with -the AllowSingleHopExits flag set to true. If ExcludeSingleHopRelays is set to -0, these relays will be included. Note that these relays might be at higher -risk of being seized or observed, so they are not normally included. -(Default: 1) -.LP -.TP -\fBBridge \fR\fIIP:ORPort\fR [fingerprint]\fP -When set along with UseBridges, instructs Tor to use the relay at -"IP:ORPort" as a "bridge" relaying into the Tor network. If "fingerprint" -is provided (using the same format as for DirServer), we will verify that -the relay running at that location has the right fingerprint. We also use -fingerprint to look up the bridge descriptor at the bridge authority, if -it's provided and if UpdateBridgesFromAuthority is set too. -.LP -.TP -\fBCircuitBuildTimeout \fR\fINUM\fP -Try for at most NUM seconds when building circuits. If the circuit -isn't open in that time, give up on it. -(Default: 1 minute.) -.LP -.TP -\fBCircuitIdleTimeout \fR\fINUM\fP -If we have kept a clean (never used) circuit around for NUM seconds, -then close it. This way when the Tor client is entirely idle, it can -expire all of its circuits, and then expire its TLS connections. Also, -if we end up making a circuit that is not useful for exiting any of -the requests we're receiving, it won't forever take up a slot in the -circuit list. -(Default: 1 hour.) -.LP -.TP -\fBClientOnly \fR\fB0\fR|\fB1\fR\fP -If set to 1, Tor will under no circumstances run as a server or serve -directory requests. The default -is to run as a client unless ORPort is configured. (Usually, -you don't need to set this; Tor is pretty smart at figuring out whether -you are reliable and high-bandwidth enough to be a useful server.) -(Default: 0) -.LP -.TP -\fBExcludeNodes \fR\fInode\fR,\fInode\fR,\fI...\fP -A list of identity fingerprints, nicknames, country codes and address patterns -of nodes to never use when building a circuit. (Example: ExcludeNodes -SlowServer, $ABCDEFFFFFFFFFFFFFFF, {cc}, 255.254.0.0/8) -.LP -.TP -\fBExcludeExitNodes \fR\fInode\fR,\fInode\fR,\fI...\fP -A list of identity fingerprints, nicknames, country codes and address patterns -of nodes to never use when picking an exit node. Note that any node -listed in ExcludeNodes is automatically considered to be part of this -list. -.LP -.TP -\fBEntryNodes \fR\fInode\fR,\fInode\fR,\fI...\fP -A list of identity fingerprints, nicknames, country codes and address patterns -of nodes to use for the first hop in the circuit. -These are treated only as preferences unless StrictEntryNodes (see -below) is also set. -.LP -.TP -\fBExitNodes \fR\fInode\fR,\fInode\fR,\fI...\fP -A list of identity fingerprints, nicknames, country codes and address patterns -of nodes to use for the last hop in the circuit. -These are treated only as preferences unless StrictExitNodes (see -below) is also set. -.LP -.TP -\fBStrictEntryNodes \fR\fB0\fR|\fB1\fR\fP -If 1, Tor will never use any nodes besides those listed in "EntryNodes" for -the first hop of a circuit. -.LP -.TP -\fBStrictExitNodes \fR\fB0\fR|\fB1\fR\fP -If 1, Tor will never use any nodes besides those listed in "ExitNodes" for -the last hop of a circuit. -.LP -.TP -\fBFascistFirewall \fR\fB0\fR|\fB1\fR\fP -If 1, Tor will only create outgoing connections to ORs running on ports that -your firewall allows (defaults to 80 and 443; see \fBFirewallPorts\fR). This will -allow you to run Tor as a client behind a firewall with restrictive policies, -but will not allow you to run as a server behind such a firewall. -If you prefer more fine-grained control, use ReachableAddresses instead. -.LP -.TP -\fBFirewallPorts \fR\fIPORTS\fP -A list of ports that your firewall allows you to connect to. Only -used when \fBFascistFirewall\fR is set. This option is deprecated; use -ReachableAddresses instead. (Default: 80, 443) -.LP -.TP -\fBHidServAuth \fR\fIonion-address\fR \fIauth-cookie\fP [\fIservice-name\fR] -Client authorization for a hidden service. Valid onion addresses contain 16 -characters in a-z2-7 plus ".onion", and valid auth cookies contain 22 -characters in A-Za-z0-9+/. The service name is only used for internal -purposes, e.g., for Tor controllers. This option may be used multiple times -for different hidden services. If a hidden service uses authorization and -this option is not set, the hidden service is not accessible. Hidden -services can be configured to require authorization using the -\fBHiddenServiceAuthorizeClient\fR option. -.LP -.TP -\fBReachableAddresses \fR\fIADDR\fP[\fB/\fP\fIMASK\fP][:\fIPORT\fP]...\fP -A comma-separated list of IP addresses and ports that your firewall allows you -to connect to. The format is as -for the addresses in ExitPolicy, except that "accept" is understood -unless "reject" is explicitly provided. For example, 'ReachableAddresses -99.0.0.0/8, reject 18.0.0.0/8:80, accept *:80' means that your -firewall allows connections to everything inside net 99, rejects port -80 connections to net 18, and accepts connections to port 80 otherwise. -(Default: 'accept *:*'.) -.LP -.TP -\fBReachableDirAddresses \fR\fIADDR\fP[\fB/\fP\fIMASK\fP][:\fIPORT\fP]...\fP -Like \fBReachableAddresses\fP, a list of addresses and ports. Tor will obey -these restrictions when fetching directory information, using standard HTTP -GET requests. If not set explicitly then the value of \fBReachableAddresses\fP -is used. If \fBHTTPProxy\fR is set then these connections will go through that -proxy. -.LP -.TP -\fBReachableORAddresses \fR\fIADDR\fP[\fB/\fP\fIMASK\fP][:\fIPORT\fP]...\fP -Like \fBReachableAddresses\fP, a list of addresses and ports. Tor will obey -these restrictions when connecting to Onion Routers, using TLS/SSL. If not set -explicitly then the value of \fBReachableAddresses\fP is used. If -\fBHTTPSProxy\fR is set then these connections will go through that proxy. - -The separation between \fBReachableORAddresses\fP and -\fBReachableDirAddresses\fP is only interesting when you are connecting through -proxies (see \fBHTTPProxy\fR and \fBHTTPSProxy\fR). Most proxies limit TLS -connections (which Tor uses to connect to Onion Routers) to port 443, and some -limit HTTP GET requests (which Tor uses for fetching directory information) to -port 80. -.LP -.TP -\fBLongLivedPorts \fR\fIPORTS\fP -A list of ports for services that tend to have long-running connections -(e.g. chat and interactive shells). Circuits for streams that use these -ports will contain only high-uptime nodes, to reduce the chance that a -node will go down before the stream is finished. -(Default: 21, 22, 706, 1863, 5050, 5190, 5222, 5223, 6667, 6697, 8300) -.LP -.TP -\fBMapAddress\fR \fIaddress\fR \fInewaddress\fR -When a request for address arrives to Tor, it will rewrite it to -newaddress before processing it. For example, if you always want -connections to www.indymedia.org to exit via \fItorserver\fR (where -\fItorserver\fR is the nickname of the server), -use "MapAddress www.indymedia.org www.indymedia.org.torserver.exit". -.LP -.TP -\fBNewCircuitPeriod \fR\fINUM\fP -Every NUM seconds consider whether to build a new circuit. (Default: 30 seconds) -.LP -.TP -\fBMaxCircuitDirtiness \fR\fINUM\fP -Feel free to reuse a circuit that was first used at most NUM seconds ago, -but never attach a new stream to a circuit that is too old. -(Default: 10 minutes) -.LP -.TP -\fBNodeFamily \fR\fInode\fR,\fInode\fR,\fI...\fP -The Tor servers, defined by their identity fingerprints or nicknames, -constitute a "family" of similar or co-administered -servers, so never use any two of them in the same circuit. Defining a -NodeFamily is only needed when a server doesn't list the family itself -(with MyFamily). This option can be used multiple times. -.LP -.TP -\fBEnforceDistinctSubnets \fR\fB0\fR|\fB1\fR\fP -If 1, Tor will not put two servers whose IP addresses are "too -close" on the same circuit. Currently, two addresses are -"too close" if they lie in the same /16 range. (Default: 1) - -.\" \fBPathlenCoinWeight \fR\fI0.0-1.0\fP -.\" Paths are 3 hops plus a geometric distribution centered around this coinweight. -.\" Must be >=0.0 and <1.0. (Default: 0.3) NOT USED CURRENTLY -.\" .TP -.LP -.TP -\fBSocksPort \fR\fIPORT\fP -Advertise this port to listen for connections from Socks-speaking -applications. Set this to 0 if you don't want to allow application -connections. (Default: 9050) -.LP -.TP -\fBSocksListenAddress \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind to this address to listen for connections from Socks-speaking -applications. (Default: 127.0.0.1) You can also specify a port -(e.g. 192.168.0.1:9100). -This directive can be specified multiple times to bind to multiple -addresses/ports. -.LP -.TP -\fBSocksPolicy \fR\fIpolicy\fR,\fIpolicy\fR,\fI...\fP -Set an entrance policy for this server, to limit who can connect to the -SocksPort and DNSPort ports. -The policies have the same form as exit policies below. -.LP -.TP -\fBSocksTimeout \fR\fINUM\fP -Let a socks connection wait NUM seconds handshaking, and NUM seconds -unattached waiting for an appropriate circuit, before we fail it. -(Default: 2 minutes.) -.LP -.TP -\fBTrackHostExits \fR\fIhost\fR,\fI.domain\fR,\fI...\fR\fP -For each value in the comma separated list, Tor will track recent connections -to hosts that match this value and attempt to -reuse the same exit node for each. If the value is prepended with a '.', it is -treated as matching an entire domain. If one of the values is just a '.', it -means match everything. This option is useful if you frequently connect to -sites that will expire all your authentication cookies (i.e. log you out) if -your IP address changes. Note that this option does have the disadvantage of -making it more clear that a given history is -associated with a single user. However, most people who would wish to observe -this will observe it through cookies or other protocol-specific means anyhow. -.LP -.TP -\fBTrackHostExitsExpire \fR\fINUM\fP -Since exit servers go up and down, it is desirable to expire the association -between host and exit server after NUM seconds. The default -is 1800 seconds (30 minutes). -.LP -.TP -\fBUpdateBridgesFromAuthority \fR\fB0\fR|\fB1\fR\fP -When set (along with UseBridges), Tor will try to fetch bridge descriptors -from the configured bridge authorities when feasible. It will fall back -to a direct request if the authority responds with a 404. (Default: 0) -.LP -.TP -\fBUseBridges \fR\fB0\fR|\fB1\fR\fP -When set, Tor will fetch descriptors for each bridge listed in the -"Bridge" config lines, and use these relays as both entry guards and -directory guards. (Default: 0) -.LP -.TP -\fBUseEntryGuards \fR\fB0\fR|\fB1\fR\fP -If this option is set to 1, we pick a few long-term entry servers, and -try to stick with them. This is desirable because -constantly changing servers increases the odds that an adversary who owns -some servers will observe a fraction of your paths. -(Defaults to 1.) -.LP -.TP -\fBNumEntryGuards \fR\fINUM\fP -If UseEntryGuards is set to 1, we will try to pick a total of NUM routers -as long-term entries for our circuits. -(Defaults to 3.) -.LP -.TP -\fBSafeSocks \fR\fB0\fR|\fB1\fR\fP -When this option is enabled, Tor will reject application connections that -use unsafe variants of the socks protocol -- ones that only provide an -IP address, meaning the application is doing a DNS resolve first. -Specifically, these are socks4 and socks5 when not doing remote DNS. -(Defaults to 0.) -.LP -.TP -\fBTestSocks \fR\fB0\fR|\fB1\fR\fP -When this option is enabled, Tor will make a notice-level log entry for -each connection to the Socks port indicating whether the request used -a safe socks protocol or an unsafe one (see above entry on SafeSocks). -This helps to determine whether an application using Tor is possibly -leaking DNS requests. -(Default: 0) -.LP -.TP -\fBVirtualAddrNetwork \fR\fIAddress\fB/\fIbits\fP -When a controller asks for a virtual (unused) address with the -MAPADDRESS command, Tor picks an unassigned address from this range. -(Default: 127.192.0.0/10) - -When providing proxy server service to a network of computers using a tool like -dns-proxy-tor, -change this address to "10.192.0.0/10" or "172.16.0.0/12". -The default \fBVirtualAddrNetwork\fP address range on a -properly configured machine will route to the loopback interface. -For local use, no change to the -default \fBVirtualAddrNetwork\fP setting is needed. -.LP -.TP -\fBAllowNonRFC953Hostnames \fR\fB0\fR|\fB1\fR\fP -When this option is disabled, Tor blocks hostnames containing illegal -characters (like @ and :) rather than sending them to an exit node to be -resolved. This helps trap accidental attempts to resolve URLs and so on. -(Default: 0) -.LP -.TP -\fBFastFirstHopPK \fR\fB0\fR|\fB1\fR\fP -When this option is disabled, Tor uses the public key step for the first -hop of creating circuits. Skipping it is generally safe since we have -already used TLS to authenticate the relay and to establish forward-secure -keys. Turning this option off makes circuit building slower. - -Note that Tor will always use the public key step for the first hop if -it's operating as a relay, and it will never use the public key step if -it doesn't yet know the onion key of the first hop. -(Default: 1) -.LP -.TP -\fBTransPort\fP \fR\fIPORT\fP -If non-zero, enables transparent proxy support on \fR\fIPORT\fP (by -convention, 9040). -.\" This is required to enable support for \fBdns-proxy-tor\fP. -.\" ControlPort must be set when using \fBTransPort\fP. -Requires OS support for transparent proxies, such as BSDs' pf or -Linux's IPTables. -If you're planning -to use Tor as a transparent proxy for a network, you'll want to examine -and change VirtualAddrNetwork from the default setting. You'll also want -to set the TransListenAddress option for the network you'd like to proxy. -(Default: 0). -.LP -.TP -\fBTransListenAddress\fP \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind to this address to listen for transparent proxy connections. -(Default: 127.0.0.1). -This is useful for exporting a transparent proxy server -to an entire network. -.LP -.TP -\fBNATDPort\fP \fR\fIPORT\fP -Allow old versions of ipfw (as included in old versions of FreeBSD, -etc.) to send connections through Tor using the NATD protocol. -This option is only for people who cannot -use TransPort. -.LP -.TP -\fBNATDListenAddress\fP \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind to this address to listen for NATD connections. -(Default: 127.0.0.1). -.LP -.TP -\fBAutomapHostsOnResolve\fP \fR\fB0\fR|\fB1\fR\fP -When this option is enabled, and we get a request to resolve an -address that ends with one of the suffixes in -\fBAutomapHostsSuffixes\fP, we map an unused virtual address to that -address, and return the new virtual address. This is handy for making -".onion" addresses work with applications that resolve an address and -then connect to it. -(Default: 0). -.LP -.TP -\fBAutomapHostsSuffixes\fP \fR\fISUFFIX\fR,\fISUFFIX\fR,...\fP -A comma-separated list of suffixes to use with \fBAutomapHostsOnResolve\fP. -The "." suffix is equivalent to "all addresses." -(Default: .exit,.onion). -.LP -.TP -\fBDNSPort\fP \fR\fIPORT\fP -If non-zero, Tor listens for UDP DNS requests on this port and resolves them -anonymously. -(Default: 0). -.LP -.TP -\fBDNSListenAddress\fP \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind to this address to listen for DNS connections. -(Default: 127.0.0.1). -.LP -.TP -\fBClientDNSRejectInternalAddresses\fP \fR\fB0\fR|\fB1\fR\fP -If true, Tor does not believe any anonymously retrieved DNS answer that tells -it that an address resolves to an internal address (like 127.0.0.1 or -192.168.0.1). This option prevents certain browser-based attacks; don't turn -it off unless you know what you're doing. (Default: 1). -.LP -.TP -\fBDownloadExtraInfo\fP \fR\fB0\fR|\fB1\fR\fP -If true, Tor downloads and caches "extra-info" documents. These -documents contain information about servers other than the information -in their regular router descriptors. Tor does not use this information for -anything itself; to save bandwidth, leave this option turned off. -(Default: 0). -.LP -.TP -\fBFallbackNetworkstatusFile\fP \fIFILENAME\fP -If Tor doesn't have a cached networkstatus file, it starts out using -this one instead. Even if this file is out of date, Tor can still use -it to learn about directory mirrors, so it doesn't need to put load on -the authorities. (Default: None). -.LP -.TP -\fBWarnPlaintextPorts\fP \fR\fIport\fR,\fIport\fR,\fI...\fP -Tells Tor to issue a warnings whenever the user tries to make an -anonymous connection to one of these ports. This option is designed -to alert users to services that risk sending passwords in the clear. -(Default: 23,109,110,143). -.LP -.TP -\fBRejectPlaintextPorts\fP \fR\fIport\fR,\fIport\fR,\fI...\fP -Like WarnPlaintextPorts, but instead of warning about risky port uses, -Tor will instead refuse to make the connection. -(Default: None). - -.SH SERVER OPTIONS -.PP -The following options are useful only for servers (that is, if \fBORPort\fP is non-zero): -.LP -.TP -\fBAddress \fR\fIaddress\fP -The IP address or fully qualified domain name of this server (e.g. moria.mit.edu). You can -leave this unset, and Tor will guess your IP address. -.LP -.TP -\fBAllowSingleHopExits \fR\fB0\fR|\fB1\fR\fP -This option controls whether clients can use this server as a single hop -proxy. If set to 1, clients can use this server as an exit even if it is -the only hop in the circuit. (Default: 0) -.LP -.TP -\fBAssumeReachable \fR\fB0\fR|\fB1\fR\fP -This option is used when bootstrapping a new Tor network. If set to 1, -don't do self-reachability testing; just upload your server descriptor -immediately. If \fBAuthoritativeDirectory\fP is also set, this option -instructs the dirserver to bypass remote reachability testing too and -list all connected servers as running. -.LP -.TP -\fBBridgeRelay \fR\fB0\fR|\fB1\fR\fP -Sets the relay to act as a "bridge" with respect to relaying connections -from bridge users to the Tor network. Mainly it influences how the relay -will cache and serve directory information. Usually used in combination -with PublishServerDescriptor. -.LP -.TP -\fBContactInfo \fR\fIemail_address\fP -Administrative contact information for server. This line might get -picked up by spam harvesters, so you may want to obscure the fact -that it's an email address. -.LP -.TP -\fBExitPolicy \fR\fIpolicy\fR,\fIpolicy\fR,\fI...\fP -Set an exit policy for this server. Each policy is of the form -"\fBaccept\fP|\fBreject\fP \fIADDR\fP[\fB/\fP\fIMASK\fP]\fB[:\fP\fIPORT\fP]". -If \fB/\fP\fIMASK\fP is omitted then this policy just applies to the host -given. Instead of giving a host or network you can also use "\fB*\fP" to -denote the universe (0.0.0.0/0). \fIPORT\fP can be a single port number, -an interval of ports "\fIFROM_PORT\fP\fB-\fP\fITO_PORT\fP", or "\fB*\fP". -If \fIPORT\fP is omitted, that means "\fB*\fP". - -For example, "accept 18.7.22.69:*,reject 18.0.0.0/8:*,accept *:*" would -reject any traffic destined for MIT except for web.mit.edu, and -accept anything else. - -To specify all internal and link-local networks (including 0.0.0.0/8, -169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, and -172.16.0.0/12), you can use the "private" alias instead of an address. -These addresses are rejected by default (at the beginning of your -exit policy), along with your public IP address, unless you set the -ExitPolicyRejectPrivate config option -to 0. For example, once you've done that, you could allow HTTP to -127.0.0.1 and block all other connections to internal networks with -"accept 127.0.0.1:80,reject private:*", though that may also allow -connections to your own computer that are addressed to its public -(external) IP address. See RFC 1918 and RFC 3330 for more -details about internal and reserved IP address space. - -This directive can be specified multiple times so you don't have to put -it all on one line. - -Policies are considered first to last, and the first match wins. If -you want to _replace_ the default exit policy, end your exit policy with -either a reject *:* or an accept *:*. Otherwise, you're _augmenting_ -(prepending to) the default exit policy. The default exit policy is: -.PD 0 -.RS 12 -.IP "reject *:25" -.IP "reject *:119" -.IP "reject *:135-139" -.IP "reject *:445" -.IP "reject *:563" -.IP "reject *:1214" -.IP "reject *:4661-4666" -.IP "reject *:6346-6429" -.IP "reject *:6699" -.IP "reject *:6881-6999" -.IP "accept *:*" -.RE -.PD -.LP -.TP -\fBExitPolicyRejectPrivate \fR\fB0\fR|\fB1\fR\fP -Reject all private (local) networks, along with your own public IP -address, at the beginning of your exit -policy. See above entry on ExitPolicy. (Default: 1) -.LP -.TP -\fBMaxOnionsPending \fR\fINUM\fP -If you have more than this number of onionskins queued for decrypt, reject new ones. (Default: 100) -.LP -.TP -\fBMyFamily \fR\fInode\fR,\fInode\fR,\fI...\fP -Declare that this Tor server is controlled or administered by a group -or organization identical or similar to that of the other servers, defined by their identity fingerprints or nicknames. -When two servers both declare that they are in the same 'family', Tor clients -will not use them in the same circuit. (Each server only needs to list the -other servers in its family; it doesn't need to list itself, but it won't hurt.) -.LP -.TP -\fBNickname \fR\fIname\fP -Set the server's nickname to 'name'. Nicknames must be between 1 -and 19 characters inclusive, and must contain only the characters -[a-zA-Z0-9]. -.LP -.TP -\fBNumCPUs \fR\fInum\fP -How many processes to use at once for decrypting onionskins. (Default: 1) -.LP -.TP -\fBORPort \fR\fIPORT\fP -Advertise this port to listen for connections from Tor clients and servers. -.LP -.TP -\fBORListenAddress \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind to this IP address to listen for connections from Tor clients and -servers. If you specify a port, bind to this port rather than the one -specified in ORPort. (Default: 0.0.0.0) -This directive can be specified multiple times to bind to multiple -addresses/ports. -.LP -.TP -\fBPublishServerDescriptor \fR\fB0\fR|\fB1\fR|\fBv1\fR|\fBv2\fR|\fBv3\fR|\fBbridge\fR, ...\fP -This option is only considered if you have an ORPort defined. You can -choose multiple arguments, separated by commas. - -If set to 0, Tor will act as a server but it will not publish its -descriptor to the directory authorities. (This is useful if you're -testing out your server, or if you're using a Tor controller that handles -directory publishing for you.) Otherwise, Tor will publish its descriptor -to all directory authorities of the type(s) specified. The value "1" is -the default, which means "publish to the appropriate authorities". -.LP -.TP -\fBShutdownWaitLength\fR \fINUM\fP -When we get a SIGINT and we're a server, we begin shutting down: we close -listeners and start refusing new circuits. After \fBNUM\fP seconds, -we exit. If we get a second SIGINT, we exit immediately. (Default: -30 seconds) -.LP -.TP -\fBAccountingMax \fR\fIN\fR \fBbytes\fR|\fBKB\fR|\fBMB\fR|\fBGB\fR|\fBTB\fP -Never send more than the specified number of bytes in a given -accounting period, or receive more than that number in the period. -For example, with AccountingMax set to 1 GB, a server could send 900 MB -and receive 800 MB and continue running. It will only hibernate once one -of the two reaches 1 GB. -When the number of bytes is exhausted, Tor will hibernate until some -time in the next accounting period. To prevent all servers from -waking at the same time, Tor will also wait until a random point in -each period before waking up. If you have bandwidth cost issues, -enabling hibernation is preferable to setting a low bandwidth, since it -provides users with a collection of fast servers that are up some of -the time, which is more useful than a set of slow servers that are -always "available". -.LP -.TP -\fBAccountingStart \fR\fBday\fR|\fBweek\fR|\fBmonth\fR [\fIday\fR] \fIHH:MM\fR\fP -Specify how long accounting periods last. If \fBmonth\fP is given, -each accounting period runs from the time \fIHH:MM\fR on the -\fIday\fRth day of one month to the same day and time of the next. -(The day must be between 1 and 28.) If \fBweek\fP is given, each -accounting period runs from the time \fIHH:MM\fR of the \fIday\fRth -day of one week to the same day and time of the next week, with Monday -as day 1 and Sunday as day 7. If \fBday\fR is given, each accounting -period runs from the time \fIHH:MM\fR each day to the same time on the -next day. All times are local, and given in 24-hour time. (Defaults to -"month 1 0:00".) -.LP -.TP -\fBServerDNSResolvConfFile \fR\fIfilename\fP -Overrides the default DNS configuration with the configuration in -\fIfilename\fP. The file format is the same as the standard Unix -"\fBresolv.conf\fP" file (7). This option, like all other -ServerDNS options, only affects name lookups that your server does on -behalf of clients. (Defaults to use the system DNS configuration.) -.LP -.TP -\fBServerDNSAllowBrokenConfig \fR\fB0\fR|\fB1\fR\fP -If this option is false, Tor exits immediately if there are problems -parsing the system DNS configuration or connecting to nameservers. -Otherwise, Tor continues to periodically retry the system nameservers -until it eventually succeeds. -(Defaults to "1".) -.LP -.TP -\fBServerDNSSearchDomains \fR\fB0\fR|\fB1\fR\fP -If set to \fB1\fP, then we will search for addresses in the local search -domain. For example, if this system is configured to believe it is in -"example.com", and a client tries to connect to "www", the client will be -connected to "www.example.com". -This option only affects name lookups that your server does on -behalf of clients. -(Defaults to "0".) -.LP -.TP -\fBServerDNSDetectHijacking \fR\fB0\fR|\fB1\fR\fP -When this option is set to 1, we will test periodically to determine whether -our local nameservers have been configured to hijack failing DNS requests -(usually to an advertising site). If they are, we will attempt to correct -this. -This option only affects name lookups that your server does on -behalf of clients. -(Defaults to "1".) -.LP -.TP -\fBServerDNSTestAddresses \fR\fIaddress\fR,\fIaddress\fR,\fI...\fP -When we're detecting DNS hijacking, make sure that these \fIvalid\fP -addresses aren't getting redirected. If they are, then our DNS is -completely useless, and we'll reset our exit policy to "reject *:*". -This option only affects name lookups that your server does on -behalf of clients. -(Defaults to "www.google.com, www.mit.edu, www.yahoo.com, -www.slashdot.org".) -.LP -.TP -\fBServerDNSAllowNonRFC953Hostnames \fR\fB0\fR|\fB1\fR\fP -When this option is disabled, Tor does not try to resolve hostnames -containing illegal characters (like @ and :) rather than sending them to an -exit node to be resolved. This helps trap accidental attempts to resolve -URLs and so on. -This option only affects name lookups that your server does on -behalf of clients. -(Default: 0) -.LP -.TP -\fBBridgeRecordUsageByCountry \fR\fB0\fR|\fB1\fR\fP -When this option is enabled and BridgeRelay is also enabled, and we -have GeoIP data, Tor keeps a keep a per-country count of how many -client addresses have contacted it so that it can help the bridge -authority guess which countries have blocked access to it. (Default: 1) -.LP -.TP -\fBServerDNSRandomizeCase \fR\fB0\fR|\fB1\fR\fP -When this option is set, Tor sets the case of each character randomly in -outgoing DNS requests, and makes sure that the case matches in DNS replies. -This so-called "0x20 hack" helps resist some types of DNS poisoning attack. -For more information, see "Increased DNS Forgery Resistance through 0x20-Bit -Encoding". -This option only affects name lookups that your server does on -behalf of clients. -(Default: 1) -.LP -.TP -\fBGeoIPFile \fR\fIfilename\fP -A filename containing GeoIP data, for use with BridgeRecordUsageByCountry. - -.SH DIRECTORY SERVER OPTIONS -.PP -The following options are useful only for directory servers (that is, if \fBDirPort\fP is non-zero): -.LP -.TP -\fBAuthoritativeDirectory \fR\fB0\fR|\fB1\fR\fP -When this option is set to 1, Tor operates as an authoritative -directory server. Instead of caching the directory, it generates its -own list of good servers, signs it, and sends that to the clients. -Unless the clients already have you listed as a trusted directory, you -probably do not want to set this option. Please coordinate with the other -admins at tor-ops@freehaven.net if you think you should be a directory. -.LP -.TP -\fBDirPortFrontPage \fIFILENAME\fP -When this option is set, it takes an HTML file and publishes it as "/" on -the DirPort. Now relay operators can provide a disclaimer without needing -to set up a separate webserver. There's a sample disclaimer in -contrib/tor-exit-notice.html. -.LP -.TP -\fBV1AuthoritativeDirectory \fR\fB0\fR|\fB1\fR\fP -When this option is set in addition to \fBAuthoritativeDirectory\fP, Tor -generates version 1 directory and running-routers documents (for legacy -Tor clients up to 0.1.0.x). -.LP -.TP -\fBV2AuthoritativeDirectory \fR\fB0\fR|\fB1\fR\fP -When this option is set in addition to \fBAuthoritativeDirectory\fP, Tor -generates version 2 network statuses and serves descriptors, etc as -described in doc/spec/dir-spec-v2.txt (for Tor clients and servers -running 0.1.1.x and 0.1.2.x). -.LP -.TP -\fBV3AuthoritativeDirectory \fR\fB0\fR|\fB1\fR\fP -When this option is set in addition to \fBAuthoritativeDirectory\fP, Tor -generates version 3 network statuses and serves descriptors, etc as -described in doc/spec/dir-spec.txt (for Tor clients and servers -running at least 0.2.0.x). -.LP -.TP -\fBVersioningAuthoritativeDirectory \fR\fB0\fR|\fB1\fR\fP -When this option is set to 1, Tor adds information on -which versions of Tor are still believed safe for use to -the published directory. Each version 1 authority is -automatically a versioning authority; version 2 authorities -provide this service optionally. See \fBRecommendedVersions\fP, -\fBRecommendedClientVersions\fP, and \fBRecommendedServerVersions\fP. -.LP -.TP -\fBNamingAuthoritativeDirectory \fR\fB0\fR|\fB1\fR\fP -When this option is set to 1, then the server advertises that it has -opinions about nickname-to-fingerprint bindings. It will include these -opinions in its published network-status pages, by listing servers with -the flag "Named" if a correct binding between that nickname and -fingerprint has been registered with the dirserver. Naming dirservers -will refuse to accept or publish descriptors that contradict a -registered binding. See \fBapproved-routers\fP in the \fBFILES\fP -section below. -.LP -.TP -\fBHSAuthoritativeDir \fR\fB0\fR|\fB1\fR\fP -When this option is set in addition to \fBAuthoritativeDirectory\fP, Tor also -accepts and serves hidden service descriptors. (Default: 0) -.LP -.TP -\fBHSAuthorityRecordStats \fR\fB0\fR|\fB1\fR\fP -When this option is set in addition to \fBHSAuthoritativeDir\fP, Tor -periodically (every 15 minutes) writes statistics about hidden service -usage to a file \fBhsusage\fP in its data directory. (Default: 0) -.LP -.TP -\fBHidServDirectoryV2 \fR\fB0\fR|\fB1\fR\fP -When this option is set, Tor accepts and serves v2 hidden service -descriptors. Setting DirPort is not required for this, because clients -connect via the ORPort by default. (Default: 1) -.LP -.TP -\fBBridgeAuthoritativeDir \fR\fB0\fR|\fB1\fR\fP -When this option is set in addition to \fBAuthoritativeDirectory\fP, Tor -accepts and serves router descriptors, but it caches and serves the main -networkstatus documents rather than generating its own. (Default: 0) -.LP -.TP -\fBMinUptimeHidServDirectoryV2 \fR\fIN\fR \fBseconds\fR|\fBminutes\fR|\fBhours\fR|\fBdays\fR|\fBweeks\fP -Minimum uptime of a v2 hidden service directory to be accepted as such by -authoritative directories. (Default: 24 hours) -.LP -.TP -\fBDirPort \fR\fIPORT\fP -Advertise the directory service on this port. -.LP -.TP -\fBDirListenAddress \fR\fIIP\fR[:\fIPORT\fR]\fP -Bind the directory service to this address. If you specify a port, bind -to this port rather than the one specified in DirPort. (Default: 0.0.0.0) -This directive can be specified multiple times to bind to multiple -addresses/ports. -.LP -.TP -\fBDirPolicy \fR\fIpolicy\fR,\fIpolicy\fR,\fI...\fP -Set an entrance policy for this server, to limit who can connect to the -directory ports. -The policies have the same form as exit policies above. - -.SH DIRECTORY AUTHORITY SERVER OPTIONS -.PP -.LP -.TP -\fBRecommendedVersions \fR\fISTRING\fP -STRING is a comma-separated list of Tor versions currently believed -to be safe. The list is included in each directory, and nodes which -pull down the directory learn whether they need to upgrade. This -option can appear multiple times: the values from multiple lines are -spliced together. -When this is set then -\fBVersioningAuthoritativeDirectory\fP should be set too. -.LP -.TP -\fBRecommendedClientVersions \fR\fISTRING\fP -STRING is a comma-separated list of Tor versions currently believed -to be safe for clients to use. This information is included in version 2 -directories. If this is not set then the value of \fBRecommendedVersions\fR -is used. -When this is set then -\fBVersioningAuthoritativeDirectory\fP should be set too. -.LP -.TP -\fBRecommendedServerVersions \fR\fISTRING\fP -STRING is a comma-separated list of Tor versions currently believed -to be safe for servers to use. This information is included in version 2 -directories. If this is not set then the value of \fBRecommendedVersions\fR -is used. -When this is set then -\fBVersioningAuthoritativeDirectory\fP should be set too. -.LP -.TP -\fBDirAllowPrivateAddresses \fR\fB0\fR|\fB1\fR\fP -If set to 1, Tor will accept router descriptors with arbitrary "Address" -elements. Otherwise, if the address is not an IP address or is a private -IP address, it will reject the router descriptor. Defaults to 0. -.LP -.TP -\fBAuthDirBadDir \fR\fIAddressPattern\fR...\fP -Authoritative directories only. A set of address patterns for servers that -will be listed as bad directories in any network status document this authority -publishes, if \fBAuthDirListBadDirs\fR is set. -.LP -.TP -\fBAuthDirBadExit \fR\fIAddressPattern\fR...\fP -Authoritative directories only. A set of address patterns for servers that -will be listed as bad exits in any network status document this authority -publishes, if \fBAuthDirListBadExits\fR is set. -.LP -.TP -\fBAuthDirInvalid \fR\fIAddressPattern\fR...\fP -Authoritative directories only. A set of address patterns for servers that -will never be listed as "valid" in any network status document that this -authority publishes. -.LP -.TP -\fBAuthDirReject \fR\fIAddressPattern\fR...\fP -Authoritative directories only. A set of address patterns for servers that -will never be listed at all in any network status document that this -authority publishes, or accepted as an OR address in any descriptor submitted -for publication by this authority. -.LP -.TP -\fBAuthDirListBadDirs \fR\fB0\fR|\fB1\fR\fP -Authoritative directories only. If set to 1, this directory has -some opinion about which nodes are unsuitable as directory caches. (Do not -set this to 1 unless you plan to list non-functioning directories as bad; -otherwise, you are effectively voting in favor of every declared directory.) -.LP -.TP -\fBAuthDirListBadExits \fR\fB0\fR|\fB1\fR\fP -Authoritative directories only. If set to 1, this directory has -some opinion about which nodes are unsuitable as exit nodes. (Do not -set this to 1 unless you plan to list non-functioning exits as bad; -otherwise, you are effectively voting in favor of every declared exit -as an exit.) -.LP -.TP -\fBAuthDirRejectUnlisted \fR\fB0\fR|\fB1\fR\fP -Authoritative directories only. If set to 1, the directory server -rejects all uploaded server descriptors that aren't explicitly listed -in the fingerprints file. This acts as a "panic button" if we get -hit with a Sybil attack. (Default: 0) -.LP -.TP -\fBAuthDirMaxServersPerAddr\fR \fINUM\fP -Authoritative directories only. The maximum number of servers that we -will list as acceptable on a single IP address. Set this to "0" for -"no limit". (Default: 2) -.LP -.TP -\fBAuthDirMaxServersPerAuthAddr\fR \fINUM\fP -Authoritative directories only. Like AuthDirMaxServersPerAddr, but -applies to addresses shared with directory authorities. (Default: 5) -.LP -.TP -\fBV3AuthVotingInterval\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -V3 authoritative directories only. Configures the server's preferred -voting interval. Note that voting will \fIactually\fP happen at an -interval chosen by consensus from all the authorities' preferred -intervals. This time SHOULD divide evenly into a day. (Default: 1 hour) -.LP -.TP -\fBV3AuthVoteDelay\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -V3 authoritative directories only. Configures the server's preferred -delay between publishing its vote and assuming it has all the votes -from all the other authorities. Note that the actual time used is not -the server's preferred time, but the consensus of all preferences. -(Default: 5 minutes.) -.LP -.TP -\fBV3AuthDistDelay\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -V3 authoritative directories only. Configures the server's preferred -delay between publishing its consensus and signature and assuming it -has all the signatures from all the other authorities. Note that the -actual time used is not the server's preferred time, but the consensus -of all preferences. (Default: 5 minutes.) -.LP -.TP -\fBV3AuthNIntervalsValid\fR \fINUM\fP -V3 authoritative directories only. Configures the number of -VotingIntervals for which each consensus should be valid for. -Choosing high numbers increases network partitioning risks; choosing -low numbers increases directory traffic. Note that the actual number -of intervals used is not the server's preferred number, but the -consensus of all preferences. Must be at least 2. (Default: 3.) - - -.SH HIDDEN SERVICE OPTIONS -.PP -The following options are used to configure a hidden service. -.LP -.TP -\fBHiddenServiceDir \fR\fIDIRECTORY\fP -Store data files for a hidden service in DIRECTORY. Every hidden -service must have a separate directory. You may use this option multiple -times to specify multiple services. -.LP -.TP -\fBHiddenServicePort \fR\fIVIRTPORT \fR[\fITARGET\fR]\fP -Configure a virtual port VIRTPORT for a hidden service. You may use this -option multiple times; each time applies to the service using the most recent -hiddenservicedir. By default, this option maps the virtual port to the -same port on 127.0.0.1. You may override the target port, address, or both -by specifying a target of addr, port, or addr:port. You may also have -multiple lines with the same VIRTPORT: when a user connects to that VIRTPORT, -one of the TARGETs from those lines will be chosen at random. -.LP -.TP -\fBPublishHidServDescriptors \fR\fB0\fR|\fB1\fR\fP -If set to 0, Tor will run any hidden services you configure, but it won't -advertise them to the rendezvous directory. This option is only useful -if you're using a Tor controller that handles hidserv publishing for you. -(Default: 1) -.LP -.TP -\fBHiddenServiceVersion \fR\fIversion\fR,\fIversion\fR,\fI...\fP -A list of rendezvous service descriptor versions to publish for the hidden -service. Possible version numbers are 0 and 2. (Default: 0, 2) -.LP -.TP -\fBHiddenServiceAuthorizeClient \fR\fIauth-type\fR \fR\fIclient-name\fR,\fIclient-name\fR,\fI...\fP -If configured, the hidden service is accessible for authorized clients -only. The auth-type can either be 'basic' for a general-purpose -authorization protocol or 'stealth' for a less scalable protocol that also -hides service activity from unauthorized clients. Only clients that are -listed here are authorized to access the hidden service. Valid client names -are 1 to 19 characters long and only use characters in A-Za-z0-9+-_ -(no spaces). If this option is set, the hidden service is not accessible -for clients without authorization any more. Generated authorization data -can be found in the hostname file. Clients need to put this authorization -data in their configuration file using \fBHidServAuth\fR. -.LP -.TP -\fBRendPostPeriod \fR\fIN\fR \fBseconds\fR|\fBminutes\fR|\fBhours\fR|\fBdays\fR|\fBweeks\fP -Every time the specified period elapses, Tor uploads any rendezvous -service descriptors to the directory servers. This information is also -uploaded whenever it changes. (Default: 20 minutes) - -.SH TESTING NETWORK OPTIONS -.PP -The following options are used for running a testing Tor network. -.LP -.TP -\fBTestingTorNetwork \fR\fB0\fR|\fB1\fR\fP -If set to 1, Tor adjusts default values of the configuration options below, -so that it is easier to set up a testing Tor network. May only be set if -non-default set of DirServers is set. Cannot be unset while Tor is running. -(Default: 0) - -.PD 0 -.RS 12 -.IP "ServerDNSAllowBrokenConfig 1" -.IP "DirAllowPrivateAddresses 1" -.IP "EnforceDistinctSubnets 0" -.IP "AssumeReachable 1" -.IP "AuthDirMaxServersPerAddr 0" -.IP "AuthDirMaxServersPerAuthAddr 0" -.IP "ClientDNSRejectInternalAddresses 0" -.IP "ExitPolicyRejectPrivate 0" -.IP "V3AuthVotingInterval 5 minutes" -.IP "V3AuthVoteDelay 20 seconds" -.IP "V3AuthDistDelay 20 seconds" -.IP "TestingV3AuthInitialVotingInterval 5 minutes" -.IP "TestingV3AuthInitialVoteDelay 20 seconds" -.IP "TestingV3AuthInitialDistDelay 20 seconds" -.IP "TestingAuthDirTimeToLearnReachability 0 minutes" -.IP "TestingEstimatedDescriptorPropagationTime 0 minutes" -.RE -.PD -.LP -.TP -\fBTestingV3AuthInitialVotingInterval\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -Like \fBV3AuthVotingInterval\fR, but for initial voting interval before the -first consensus has been created. Changing this requires that -\fBTestingTorNetwork\fR is set. (Default: 30 minutes) -.LP -.TP -\fBTestingV3AuthInitialVoteDelay\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -Like \fBTestingV3AuthInitialVoteDelay\fR, but for initial voting interval -before the first consensus has been created. Changing this requires that -\fBTestingTorNetwork\fR is set. (Default: 5 minutes) -.LP -.TP -\fBTestingV3AuthInitialDistDelay\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -Like \fBTestingV3AuthInitialDistDelay\fR, but for initial voting interval -before the first consensus has been created. Changing this requires that -\fBTestingTorNetwork\fR is set. (Default: 5 minutes) -.LP -.TP -\fBTestingAuthDirTimeToLearnReachability\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -After starting as an authority, do not make claims about whether routers are -Running until this much time has passed. -Changing this requires that\fBTestingTorNetwork\fR is set. -(Default: 30 minutes) -.LP -.TP -\fBTestingEstimatedDescriptorPropagationTime\fR \fR\fIN\fR \fBminutes\fR|\fBhours\fP -Clients try downloading router descriptors from directory caches after this -time. Changing this requires that \fBTestingTorNetwork\fR is set. -(Default: 10 minutes) - -.\" UNDOCUMENTED -.\" ignoreversion - -.SH SIGNALS -Tor catches the following signals: -.LP -.TP -\fBSIGTERM\fR -Tor will catch this, clean up and sync to disk if necessary, and exit. -.LP -.TP -\fBSIGINT\fR -Tor clients behave as with SIGTERM; but Tor servers will do a controlled -slow shutdown, closing listeners and waiting 30 seconds before exiting. -(The delay can be configured with the ShutdownWaitLength config option.) -.LP -.TP -\fBSIGHUP\fR -The signal instructs Tor to reload its configuration (including closing -and reopening logs), fetch a new directory, and kill and restart its -helper processes if applicable. -.LP -.TP -\fBSIGUSR1\fR -Log statistics about current connections, past connections, and -throughput. -.LP -.TP -\fBSIGUSR2\fR -Switch all logs to loglevel debug. You can go back to the old loglevels -by sending a SIGHUP. -.LP -.TP -\fBSIGCHLD\fR -Tor receives this signal when one of its helper processes has exited, -so it can clean up. -.LP -.TP -\fBSIGPIPE\fR -Tor catches this signal and ignores it. -.LP -.TP -\fBSIGXFSZ\fR -If this signal exists on your platform, Tor catches and ignores it. - -.SH FILES -.LP -.TP -.B @CONFDIR@/torrc -The configuration file, which contains "option value" pairs. -.LP -.TP -.B @LOCALSTATEDIR@/lib/tor/ -The tor process stores keys and other data here. -.LP -.TP -.B \fIDataDirectory\fP/cached-status/* -The most recently downloaded network status document for each authority. Each file holds one such document; the filenames are the hexadecimal identity key fingerprints of the directory authorities. -.LP -.TP -.B \fIDataDirectory\fB/cached-descriptors\fR and \fBcached-descriptors.new\fR -These files hold downloaded router statuses. Some routers may appear more than once; if so, the most recently published descriptor is used. Lines beginning with @-signs are annotations that contain more information about a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-routers file. -.LP -.TP -.B \fIDataDirectory\fB/cached-routers\fR and \fBcached-routers.new\fR -Obsolete versions of cached-descriptors and cached-descriptors.new. When Tor can't find the newer files, it looks here instead. -.LP -.TP -.B \fIDataDirectory\fP/state -A set of persistent key-value mappings. These are documented in the file. These include: -.PD 0 -.RS 5 -.IP "- The current entry guards and their status." -.IP "- The current bandwidth accounting values (unused so far; see below)." -.IP "- When the file was last written" -.IP "- What version of Tor generated the state file" -.IP "- A short history of bandwidth usage, as produced in the router descriptors." -.RE -.PD -.LP -.TP -.B \fIDataDirectory\fP/bw_accounting -Used to track bandwidth accounting values (when the current period starts and ends; how much has been read and written so far this period). This file is obsolete, and the data is now stored in the 'state' file as well. Only used when bandwidth accounting is enabled. -.LP -.TP -.B \fIDataDirectory\fP/hsusage -Used to track hidden service usage in terms of fetch and publish -requests to this hidden service authoritative directory. Only used when -recording of statistics is enabled. -.LP -.TP -.B \fIDataDirectory\fP/control_auth_cookie -Used for cookie authentication with the controller. Location can be -overridden by the CookieAuthFile config option. Regenerated on startup. -See control-spec.txt for details. Only used when cookie authentication -is enabled. -.LP -.TP -.B \fIDataDirectory\fP/keys/* -Only used by servers. Holds identity keys and onion keys. -.LP -.TP -.B \fIDataDirectory\fP/fingerprint -Only used by servers. Holds the fingerprint of the server's identity key. -.LP -.TP -.B \fIDataDirectory\fP/approved-routers -Only for naming authoritative directory servers (see \fBNamingAuthoritativeDirectory\fP). This file lists nickname to identity bindings. Each line lists a nickname and a fingerprint separated by whitespace. See your \fBfingerprint\fP file in the \fIDataDirectory\fP for an example line. If the nickname is \fB!reject\fP then descriptors from the given identity (fingerprint) are rejected by this server. If it is \fB!invalid\fP then descriptors are accepted but marked in the directory as not valid, that is, not recommended. -.LP -.TP -.B \fIDataDirectory\fP/router-stability -Only used by authoritative directory servers. Tracks measurements for router mean-time-between-failures so that authorities have a good idea of how to set their Stable flags. -.LP -.TP -.B \fIHiddenServiceDirectory\fP/hostname -The <base32-encoded-fingerprint>.onion domain name for this hidden service. -If the hidden service is restricted to authorized clients only, this file -also contains authorization data for all clients. -.LP -.TP -.B \fIHiddenServiceDirectory\fP/private_key -The private key for this hidden service. -.LP -.TP -.B \fIHiddenServiceDirectory\fP/client_keys -Authorization data for a hidden service that is only accessible by authorized -clients. -.SH SEE ALSO -.BR privoxy (1), -.BR tsocks (1), -.BR torify (1) - -.BR https://www.torproject.org/ - -.SH BUGS -Plenty, probably. Tor is still in development. Please report them. -.SH AUTHORS -Roger Dingledine <arma@mit.edu>, Nick Mathewson <nickm@alum.mit.edu>. diff --git a/doc/tor.1.txt b/doc/tor.1.txt new file mode 100644 index 000000000..00d5066d0 --- /dev/null +++ b/doc/tor.1.txt @@ -0,0 +1,1541 @@ +// Copyright (c) The Tor Project, Inc. +// See LICENSE for licensing information +// This is an asciidoc file used to generate the manpage/html reference. +// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +TOR(1) +====== + +NAME +---- +tor - The second-generation onion router + + +SYNOPSIS +-------- +**tor** [__OPTION__ __value__]... + +DESCRIPTION +----------- +__tor__ is a connection-oriented anonymizing communication +service. Users choose a source-routed path through a set of nodes, and +negotiate a "virtual circuit" through the network, in which each node +knows its predecessor and successor, but no others. Traffic flowing down +the circuit is unwrapped by a symmetric key at each node, which reveals +the downstream node. + + +Basically __tor__ provides a distributed network of servers ("onion routers"). +Users bounce their TCP streams -- web traffic, ftp, ssh, etc -- around the +routers, and recipients, observers, and even the routers themselves have +difficulty tracking the source of the stream. + +OPTIONS +------- +**-h**, **-help**:: + Display a short help message and exit. + +**-f** __FILE__:: + FILE contains further "option value" pairs. (Default: @CONFDIR@/torrc) + +**--hash-password**:: + Generates a hashed password for control port access. + +**--list-fingerprint**:: + Generate your keys and output your nickname and fingerprint. + +**--verify-config**:: + Verify the configuration file is valid. + +**--nt-service**:: + **--service [install|remove|start|stop]** Manage the Tor Windows + NT/2000/XP service. Current instructions can be found at + https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#WinNTService + +**--list-torrc-options**:: + List all valid options. + +**--version**:: + Display Tor version and exit. + +**--quiet**:: + Do not start Tor with a console log unless explicitly requested to do so. + (By default, Tor starts out logging messages at level "notice" or higher to + the console, until it has parsed its configuration.) + +Other options can be specified either on the command-line (--option + value), or in the configuration file (option value or option "value"). + Options are case-insensitive. C-style escaped characters are allowed inside + quoted values. Options on the command line take precedence over + options found in the configuration file, except indicated otherwise. To + split one configuration entry into multiple lines, use a single \ before + the end of the line. Comments can be used in such multiline entries, but + they must start at the beginning of a line. + +**BandwidthRate** __N__ **bytes**|**KB**|**MB**|**GB**:: + A token bucket limits the average incoming bandwidth usage on this node to + the specified number of bytes per second, and the average outgoing + bandwidth usage to that same value. If you want to run a relay in the + public network, this needs to be _at the very least_ 20 KB (that is, + 20480 bytes). (Default: 5 MB) + +**BandwidthBurst** __N__ **bytes**|**KB**|**MB**|**GB**:: + Limit the maximum token bucket size (also known as the burst) to the given + number of bytes in each direction. (Default: 10 MB) + +**MaxAdvertisedBandwidth** __N__ **bytes**|**KB**|**MB**|**GB**:: + If set, we will not advertise more than this amount of bandwidth for our + BandwidthRate. Server operators who want to reduce the number of clients + who ask to build circuits through them (since this is proportional to + advertised bandwidth rate) can thus reduce the CPU demands on their server + without impacting network performance. + +**RelayBandwidthRate** __N__ **bytes**|**KB**|**MB**|**GB**:: + If not 0, a separate token bucket limits the average incoming bandwidth + usage for \_relayed traffic_ on this node to the specified number of bytes + per second, and the average outgoing bandwidth usage to that same value. + Relayed traffic currently is calculated to include answers to directory + requests, but that may change in future versions. (Default: 0) + +**RelayBandwidthBurst** __N__ **bytes**|**KB**|**MB**|**GB**:: + If not 0, limit the maximum token bucket size (also known as the burst) for + \_relayed traffic_ to the given number of bytes in each direction. + (Default: 0) + +**PerConnBWRate** __N__ **bytes**|**KB**|**MB**|**GB**:: + If set, do separate rate limiting for each connection from a non-relay. + You should never need to change this value, since a network-wide value is + published in the consensus and your relay will use that value. (Default: 0) + +**PerConnBWBurst** __N__ **bytes**|**KB**|**MB**|**GB**:: + If set, do separate rate limiting for each connection from a non-relay. + You should never need to change this value, since a network-wide value is + published in the consensus and your relay will use that value. (Default: 0) + +**ConnLimit** __NUM__:: + The minimum number of file descriptors that must be available to the Tor + process before it will start. Tor will ask the OS for as many file + descriptors as the OS will allow (you can find this by "ulimit -H -n"). + If this number is less than ConnLimit, then Tor will refuse to start. + + + + You probably don't need to adjust this. It has no effect on Windows + since that platform lacks getrlimit(). (Default: 1000) + +**ConstrainedSockets** **0**|**1**:: + If set, Tor will tell the kernel to attempt to shrink the buffers for all + sockets to the size specified in **ConstrainedSockSize**. This is useful for + virtual servers and other environments where system level TCP buffers may + be limited. If you're on a virtual server, and you encounter the "Error + creating network socket: No buffer space available" message, you are + likely experiencing this problem. + + + + The preferred solution is to have the admin increase the buffer pool for + the host itself via /proc/sys/net/ipv4/tcp_mem or equivalent facility; + this configuration option is a second-resort. + + + + The DirPort option should also not be used if TCP buffers are scarce. The + cached directory requests consume additional sockets which exacerbates + the problem. + + + + You should **not** enable this feature unless you encounter the "no buffer + space available" issue. Reducing the TCP buffers affects window size for + the TCP stream and will reduce throughput in proportion to round trip + time on long paths. (Default: 0.) + +**ConstrainedSockSize** __N__ **bytes**|**KB**:: + When **ConstrainedSockets** is enabled the receive and transmit buffers for + all sockets will be set to this limit. Must be a value between 2048 and + 262144, in 1024 byte increments. Default of 8192 is recommended. + +**ControlPort** __Port__:: + If set, Tor will accept connections on this port and allow those + connections to control the Tor process using the Tor Control Protocol + (described in control-spec.txt). Note: unless you also specify one of + **HashedControlPassword** or **CookieAuthentication**, setting this option will + cause Tor to allow any process on the local host to control it. This + option is required for many Tor controllers; most use the value of 9051. + +**ControlListenAddress** __IP__[:__PORT__]:: + Bind the controller listener to this address. If you specify a port, bind + to this port rather than the one specified in ControlPort. We strongly + recommend that you leave this alone unless you know what you're doing, + since giving attackers access to your control listener is really + dangerous. (Default: 127.0.0.1) This directive can be specified multiple + times to bind to multiple addresses/ports. + +**ControlSocket** __Path__:: + Like ControlPort, but listens on a Unix domain socket, rather than a TCP + socket. (Unix and Unix-like systems only.) + +**HashedControlPassword** __hashed_password__:: + Don't allow any connections on the control port except when the other + process knows the password whose one-way hash is __hashed_password__. You + can compute the hash of a password by running "tor --hash-password + __password__". You can provide several acceptable passwords by using more + than one HashedControlPassword line. + +**CookieAuthentication** **0**|**1**:: + If this option is set to 1, don't allow any connections on the control port + except when the connecting process knows the contents of a file named + "control_auth_cookie", which Tor will create in its data directory. This + authentication method should only be used on systems with good filesystem + security. (Default: 0) + +**CookieAuthFile** __Path__:: + If set, this option overrides the default location and file name + for Tor's cookie file. (See CookieAuthentication above.) + +**CookieAuthFileGroupReadable** **0**|**1**|__Groupname__:: + If this option is set to 0, don't allow the filesystem group to read the + cookie file. If the option is set to 1, make the cookie file readable by + the default GID. [Making the file readable by other groups is not yet + implemented; let us know if you need this for some reason.] (Default: 0). + +**DataDirectory** __DIR__:: + Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor) + +**DirServer** [__nickname__] [**flags**] __address__:__port__ __fingerprint__:: + Use a nonstandard authoritative directory server at the provided address + and port, with the specified key fingerprint. This option can be repeated + many times, for multiple authoritative directory servers. Flags are + separated by spaces, and determine what kind of an authority this directory + is. By default, every authority is authoritative for current ("v2")-style + directories, unless the "no-v2" flag is given. If the "v1" flags is + provided, Tor will use this server as an authority for old-style (v1) + directories as well. (Only directory mirrors care about this.) Tor will + use this server as an authority for hidden service information if the "hs" + flag is set, or if the "v1" flag is set and the "no-hs" flag is **not** set. + Tor will use this authority as a bridge authoritative directory if the + "bridge" flag is set. If a flag "orport=**port**" is given, Tor will use the + given port when opening encrypted tunnels to the dirserver. Lastly, if a + flag "v3ident=**fp**" is given, the dirserver is a v3 directory authority + whose v3 long-term signing key has the fingerprint **fp**. + + + + If no **dirserver** line is given, Tor will use the default directory + servers. NOTE: this option is intended for setting up a private Tor + network with its own directory authorities. If you use it, you will be + distinguishable from other users, because you won't believe the same + authorities they do. + +**AlternateDirAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + + +**AlternateHSAuthority** [__nickname__] [**flags**] __address__:__port__ __fingerprint__ + + +**AlternateBridgeAuthority** [__nickname__] [**flags**] __address__:__port__ __ fingerprint__:: + As DirServer, but replaces less of the default directory authorities. Using + AlternateDirAuthority replaces the default Tor directory authorities, but + leaves the hidden service authorities and bridge authorities in place. + Similarly, Using AlternateHSAuthority replaces the default hidden service + authorities, but not the directory or bridge authorities. + +**DisableAllSwap** **0**|**1**:: + If set to 1, Tor will attempt to lock all current and future memory pages, + so that memory cannot be paged out. Windows, OS X and Solaris are currently + not supported. We believe that this feature works on modern Gnu/Linux + distributions, and that it should work on *BSD systems (untested). This + option requires that you start your Tor as root, and you should use the + **User** option to properly reduce Tor's privileges. (Default: 0) + +**FetchDirInfoEarly** **0**|**1**:: + If set to 1, Tor will always fetch directory information like other + directory caches, even if you don't meet the normal criteria for fetching + early. Normal users should leave it off. (Default: 0) + +**FetchDirInfoExtraEarly** **0**|**1**:: + If set to 1, Tor will fetch directory information before other directory + caches. It will attempt to download directory information closer to the + start of the consensus period. Normal users should leave it off. + (Default: 0) + +**FetchHidServDescriptors** **0**|**1**:: + If set to 0, Tor will never fetch any hidden service descriptors from the + rendezvous directories. This option is only useful if you're using a Tor + controller that handles hidden service fetches for you. (Default: 1) + +**FetchServerDescriptors** **0**|**1**:: + If set to 0, Tor will never fetch any network status summaries or server + descriptors from the directory servers. This option is only useful if + you're using a Tor controller that handles directory fetches for you. + (Default: 1) + +**FetchUselessDescriptors** **0**|**1**:: + If set to 1, Tor will fetch every non-obsolete descriptor from the + authorities that it hears about. Otherwise, it will avoid fetching useless + descriptors, for example for routers that are not running. This option is + useful if you're using the contributed "exitlist" script to enumerate Tor + nodes that exit to certain addresses. (Default: 0) + +**HTTPProxy** __host__[:__port__]:: + Tor will make all its directory requests through this host:port (or host:80 + if port is not specified), rather than connecting directly to any directory + servers. + +**HTTPProxyAuthenticator** __username:password__:: + If defined, Tor will use this username:password for Basic HTTP proxy + authentication, as in RFC 2617. This is currently the only form of HTTP + proxy authentication that Tor supports; feel free to submit a patch if you + want it to support others. + +**HTTPSProxy** __host__[:__port__]:: + Tor will make all its OR (SSL) connections through this host:port (or + host:443 if port is not specified), via HTTP CONNECT rather than connecting + directly to servers. You may want to set **FascistFirewall** to restrict + the set of ports you might try to connect to, if your HTTPS proxy only + allows connecting to certain ports. + +**HTTPSProxyAuthenticator** __username:password__:: + If defined, Tor will use this username:password for Basic HTTPS proxy + authentication, as in RFC 2617. This is currently the only form of HTTPS + proxy authentication that Tor supports; feel free to submit a patch if you + want it to support others. + +**Socks4Proxy** __host__[:__port__]:: + Tor will make all OR connections through the SOCKS 4 proxy at host:port + (or host:1080 if port is not specified). + +**Socks5Proxy** __host__[:__port__]:: + Tor will make all OR connections through the SOCKS 5 proxy at host:port + (or host:1080 if port is not specified). + +**Socks5ProxyUsername** __username__ + + +**Socks5ProxyPassword** __password__:: + If defined, authenticate to the SOCKS 5 server using username and password + in accordance to RFC 1929. Both username and password must be between 1 and + 255 characters. + +**KeepalivePeriod** __NUM__:: + To keep firewalls from expiring connections, send a padding keepalive cell + every NUM seconds on open connections that are in use. If the connection + has no open circuits, it will instead be closed after NUM seconds of + idleness. (Default: 5 minutes) + +**Log** __minSeverity__[-__maxSeverity__] **stderr**|**stdout**|**syslog**:: + Send all messages between __minSeverity__ and __maxSeverity__ to the standard + output stream, the standard error stream, or to the system log. (The + "syslog" value is only supported on Unix.) Recognized severity levels are + debug, info, notice, warn, and err. We advise using "notice" in most cases, + since anything more verbose may provide sensitive information to an + attacker who obtains the logs. If only one severity level is given, all + messages of that level or higher will be sent to the listed destination. + +**Log** __minSeverity__[-__maxSeverity__] **file** __FILENAME__:: + As above, but send log messages to the listed filename. The + "Log" option may appear more than once in a configuration file. + Messages are sent to all the logs that match their severity + level. + +**Log** **[**__domain__,...**]**__minSeverity__[-__maxSeverity__] ... **file** __FILENAME__ + + +**Log** **[**__domain__,...**]**__minSeverity__[-__maxSeverity__] ... **stderr**|**stdout**|**syslog** :: + As above, but select messages by range of log severity __and__ by a + set of "logging domains". Each logging domain corresponds to an area of + functionality inside Tor. You can specify any number of severity ranges + for a single log statement, each of them prefixed by a comma-separated + list of logging domains. You can prefix a domain with $$~$$ to indicate + negation, and use * to indicate "all domains". If you specify a severity + range without a list of domains, it matches all domains. + + + + This is an advanced feature which is most useful for debugging one or two + of Tor's subsystems at a time. + + + + The currently recognized domains are: general, crypto, net, config, fs, + protocol, mm, http, app, control, circ, rend, bug, dir, dirserv, or, edge, + acct, hist, and handshake. Domain names are case-insensitive. + + + + For example, "`Log [handshake]debug [~net,~mm]info notice stdout`" sends + to stdout: all handshake messages of any severity, all info-and-higher + messages from domains other than networking and memory management, and all + messages of severity notice or higher. + +**LogMessageDomains** **0**|**1**:: + If 1, Tor includes message domains with each log message. Every log + message currently has at least one domain; most currently have exactly + one. This doesn't affect controller log messages. (Default: 0) + +**OutboundBindAddress** __IP__:: + Make all outbound connections originate from the IP address specified. This + is only useful when you have multiple network interfaces, and you want all + of Tor's outgoing connections to use a single one. This setting will be + ignored for connections to the loopback addresses (127.0.0.0/8 and ::1). + +**PidFile** __FILE__:: + On startup, write our PID to FILE. On clean shutdown, remove + FILE. + +**ProtocolWarnings** **0**|**1**:: + If 1, Tor will log with severity \'warn' various cases of other parties not + following the Tor specification. Otherwise, they are logged with severity + \'info'. (Default: 0) + +**RunAsDaemon** **0**|**1**:: + If 1, Tor forks and daemonizes to the background. This option has no effect + on Windows; instead you should use the --service command-line option. + (Default: 0) + + +**SafeLogging** **0**|**1**|**relay**:: + Tor can scrub potentially sensitive strings from log messages (e.g. + addresses) by replacing them with the string [scrubbed]. This way logs can + still be useful, but they don't leave behind personally identifying + information about what sites a user might have visited. + + + + If this option is set to 0, Tor will not perform any scrubbing, if it is + set to 1, all potentially sensitive strings are replaced. If it is set to + relay, all log messages generated when acting as a relay are sanitized, but + all messages generated when acting as a client are not. (Default: 1) + +**User** __UID__:: + On startup, setuid to this user and setgid to their primary group. + +**HardwareAccel** **0**|**1**:: + If non-zero, try to use built-in (static) crypto hardware acceleration when + available. (Default: 0) + +**AccelName** __NAME__:: + When using OpenSSL hardware crypto acceleration attempt to load the dynamic + engine of this name. This must be used for any dynamic hardware engine. + Names can be verified with the openssl engine command. + +**AccelDir** __DIR__:: + Specify this option if using dynamic hardware acceleration and the engine + implementation library resides somewhere other than the OpenSSL default. + +**AvoidDiskWrites** **0**|**1**:: + If non-zero, try to write to disk less frequently than we would otherwise. + This is useful when running on flash memory or other media that support + only a limited number of writes. (Default: 0) + +**TunnelDirConns** **0**|**1**:: + If non-zero, when a directory server we contact supports it, we will build + a one-hop circuit and make an encrypted connection via its ORPort. + (Default: 1) + +**PreferTunneledDirConns** **0**|**1**:: + If non-zero, we will avoid directory servers that don't support tunneled + directory connections, when possible. (Default: 1) + +**CircuitPriorityHalflife** __NUM1__:: + If this value is set, we override the default algorithm for choosing which + circuit's cell to deliver or relay next. When the value is 0, we + round-robin between the active circuits on a connection, delivering one + cell from each in turn. When the value is positive, we prefer delivering + cells from whichever connection has the lowest weighted cell count, where + cells are weighted exponentially according to the supplied + CircuitPriorityHalflife value (in seconds). If this option is not set at + all, we use the behavior recommended in the current consensus + networkstatus. This is an advanced option; you generally shouldn't have + to mess with it. (Default: not set.) + +CLIENT OPTIONS +-------------- + +The following options are useful only for clients (that is, if +**SocksPort** is non-zero): + +**AllowInvalidNodes** **entry**|**exit**|**middle**|**introduction**|**rendezvous**|**...**:: + If some Tor servers are obviously not working right, the directory + authorities can manually mark them as invalid, meaning that it's not + recommended you use them for entry or exit positions in your circuits. You + can opt to use them in some circuit positions, though. The default is + "middle,rendezvous", and other choices are not advised. + +**ExcludeSingleHopRelays** **0**|**1**:: + This option controls whether circuits built by Tor will include relays with + the AllowSingleHopExits flag set to true. If ExcludeSingleHopRelays is set + to 0, these relays will be included. Note that these relays might be at + higher risk of being seized or observed, so they are not normally + included. Also note that relatively few clients turn off this option, + so using these relays might make your client stand out. + (Default: 1) + +**Bridge** __IP__:__ORPort__ [fingerprint]:: + When set along with UseBridges, instructs Tor to use the relay at + "IP:ORPort" as a "bridge" relaying into the Tor network. If "fingerprint" + is provided (using the same format as for DirServer), we will verify that + the relay running at that location has the right fingerprint. We also use + fingerprint to look up the bridge descriptor at the bridge authority, if + it's provided and if UpdateBridgesFromAuthority is set too. + +**LearnCircuitBuildTimeout** **0**|**1**:: + If 0, CircuitBuildTimeout adaptive learning is disabled. (Default: 1) + +**CircuitBuildTimeout** __NUM__:: + + Try for at most NUM seconds when building circuits. If the circuit isn't + open in that time, give up on it. If LearnCircuitBuildTimeout is 1, this + value serves as the initial value to use before a timeout is learned. If + LearnCircuitBuildTimeout is 0, this value is the only value used. + (Default: 60 seconds.) + +**CircuitIdleTimeout** __NUM__:: + If we have kept a clean (never used) circuit around for NUM seconds, then + close it. This way when the Tor client is entirely idle, it can expire all + of its circuits, and then expire its TLS connections. Also, if we end up + making a circuit that is not useful for exiting any of the requests we're + receiving, it won't forever take up a slot in the circuit list. (Default: 1 + hour.) + +**CircuitStreamTimeout** __NUM__:: + If non-zero, this option overrides our internal timeout schedule for how + many seconds until we detach a stream from a circuit and try a new circuit. + If your network is particularly slow, you might want to set this to a + number like 60. (Default: 0) + +**ClientOnly** **0**|**1**:: + If set to 1, Tor will under no circumstances run as a server or serve + directory requests. The default is to run as a client unless ORPort is + configured. (Usually, you don't need to set this; Tor is pretty smart at + figuring out whether you are reliable and high-bandwidth enough to be a + useful server.) (Default: 0) + +**ExcludeNodes** __node__,__node__,__...__:: + A list of identity fingerprints, nicknames, country codes and address + patterns of nodes to avoid when building a circuit. + (Example: + ExcludeNodes SlowServer, $ EFFFFFFFFFFFFFFF, \{cc}, 255.254.0.0/8) + ++ + By default, this option is treated as a preference that Tor is allowed + to override in order to keep working. + For example, if you try to connect to a hidden service, + but you have excluded all of the hidden service's introduction points, + Tor will connect to one of them anyway. If you do not want this + behavior, set the StrictNodes option (documented below). + ++ + Note also that if you are a relay, this (and the other node selection + options below) only affects your own circuits that Tor builds for you. + Clients can still build circuits through you to any node. Controllers + can tell Tor to build circuits through any node. + + +**ExcludeExitNodes** __node__,__node__,__...__:: + A list of identity fingerprints, nicknames, country codes and address + patterns of nodes to never use when picking an exit node---that is, a + node that delivers traffic for you outside the Tor network. Note that any + node listed in ExcludeNodes is automatically considered to be part of this + list too. See also the caveats on the "ExitNodes" option below. + + +**ExitNodes** __node__,__node__,__...__:: + A list of identity fingerprints, nicknames, country codes and address + patterns of nodes to use as exit node---that is, a + node that delivers traffic for you outside the Tor network. + ++ + Note that if you list too few nodes here, or if you exclude too many exit + nodes with ExcludeExitNodes, you can degrade functionality. For example, + if none of the exits you list allows traffic on port 80 or 443, you won't + be able to browse the web. + ++ + Note also that not every circuit is used to deliver traffic outside of + the Tor network. It is normal to see non-exit circuits (such as those + used to connect to hidden services, those that do directory fetches, + those used for relay reachability self-tests, and so on) that end + at a non-exit node. To + keep a node from being used entirely, see ExcludeNodes and StrictNodes. + ++ + The ExcludeNodes option overrides this option: any node listed in both + ExitNodes and ExcludeNodes is treated as excluded. + ++ + The .exit address notation, if enabled via AllowDotExit, overrides + this option. + +**EntryNodes** __node__,__node__,__...__:: + A list of identity fingerprints and nicknames of nodes + to use for the first hop in your normal circuits. (Country codes and + address patterns are not yet supported.) Normal circuits include all + circuits except for direct connections to directory servers. The Bridge + option overrides this option; if you have configured bridges and + UseBridges is 1, the Bridges are used as your entry nodes. + ++ + The ExcludeNodes option overrides this option: any node listed in both + EntryNodes and ExcludeNodes is treated as excluded. + +**StrictNodes** **0**|**1**:: + If StrictNodes is set to 1, Tor will treat the ExcludeNodes option as a + requirement to follow for all the circuits you generate, even if doing so + will break functionality for you. If StrictNodes is set to 0, Tor will + still try to avoid nodes in the ExcludeNodes list, but it will err on the + side of avoiding unexpected errors. Specifically, StrictNodes 0 tells + Tor that it is okay to use an excluded node when it is *necessary* to + perform relay reachability self-tests, connect to + a hidden service, provide a hidden service to a client, fulfill a .exit + request, upload directory information, or download directory information. + (Default: 0) + +**FascistFirewall** **0**|**1**:: + If 1, Tor will only create outgoing connections to ORs running on ports + that your firewall allows (defaults to 80 and 443; see **FirewallPorts**). + This will allow you to run Tor as a client behind a firewall with + restrictive policies, but will not allow you to run as a server behind such + a firewall. If you prefer more fine-grained control, use + ReachableAddresses instead. + +**FirewallPorts** __PORTS__:: + A list of ports that your firewall allows you to connect to. Only used when + **FascistFirewall** is set. This option is deprecated; use ReachableAddresses + instead. (Default: 80, 443) + +**HidServAuth** __onion-address__ __auth-cookie__ [__service-name__]:: + Client authorization for a hidden service. Valid onion addresses contain 16 + characters in a-z2-7 plus ".onion", and valid auth cookies contain 22 + characters in A-Za-z0-9+/. The service name is only used for internal + purposes, e.g., for Tor controllers. This option may be used multiple times + for different hidden services. If a hidden service uses authorization and + this option is not set, the hidden service is not accessible. Hidden + services can be configured to require authorization using the + **HiddenServiceAuthorizeClient** option. + +**ReachableAddresses** __ADDR__[/__MASK__][:__PORT__]...:: + A comma-separated list of IP addresses and ports that your firewall allows + you to connect to. The format is as for the addresses in ExitPolicy, except + that "accept" is understood unless "reject" is explicitly provided. For + example, \'ReachableAddresses 99.0.0.0/8, reject 18.0.0.0/8:80, accept + \*:80' means that your firewall allows connections to everything inside net + 99, rejects port 80 connections to net 18, and accepts connections to port + 80 otherwise. (Default: \'accept \*:*'.) + +**ReachableDirAddresses** __ADDR__[/__MASK__][:__PORT__]...:: + Like **ReachableAddresses**, a list of addresses and ports. Tor will obey + these restrictions when fetching directory information, using standard HTTP + GET requests. If not set explicitly then the value of + **ReachableAddresses** is used. If **HTTPProxy** is set then these + connections will go through that proxy. + +**ReachableORAddresses** __ADDR__[/__MASK__][:__PORT__]...:: + Like **ReachableAddresses**, a list of addresses and ports. Tor will obey + these restrictions when connecting to Onion Routers, using TLS/SSL. If not + set explicitly then the value of **ReachableAddresses** is used. If + **HTTPSProxy** is set then these connections will go through that proxy. + + + + The separation between **ReachableORAddresses** and + **ReachableDirAddresses** is only interesting when you are connecting + through proxies (see **HTTPProxy** and **HTTPSProxy**). Most proxies limit + TLS connections (which Tor uses to connect to Onion Routers) to port 443, + and some limit HTTP GET requests (which Tor uses for fetching directory + information) to port 80. + +**LongLivedPorts** __PORTS__:: + A list of ports for services that tend to have long-running connections + (e.g. chat and interactive shells). Circuits for streams that use these + ports will contain only high-uptime nodes, to reduce the chance that a node + will go down before the stream is finished. (Default: 21, 22, 706, 1863, + 5050, 5190, 5222, 5223, 6667, 6697, 8300) + +**MapAddress** __address__ __newaddress__:: + When a request for address arrives to Tor, it will rewrite it to newaddress + before processing it. For example, if you always want connections to + www.indymedia.org to exit via __torserver__ (where __torserver__ is the + nickname of the server), use "MapAddress www.indymedia.org + www.indymedia.org.torserver.exit". + +**NewCircuitPeriod** __NUM__:: + Every NUM seconds consider whether to build a new circuit. (Default: 30 + seconds) + +**MaxCircuitDirtiness** __NUM__:: + Feel free to reuse a circuit that was first used at most NUM seconds ago, + but never attach a new stream to a circuit that is too old. (Default: 10 + minutes) + +**NodeFamily** __node__,__node__,__...__:: + The Tor servers, defined by their identity fingerprints or nicknames, + constitute a "family" of similar or co-administered servers, so never use + any two of them in the same circuit. Defining a NodeFamily is only needed + when a server doesn't list the family itself (with MyFamily). This option + can be used multiple times. + +**EnforceDistinctSubnets** **0**|**1**:: + If 1, Tor will not put two servers whose IP addresses are "too close" on + the same circuit. Currently, two addresses are "too close" if they lie in + the same /16 range. (Default: 1) + +**SocksPort** __PORT__:: + Advertise this port to listen for connections from Socks-speaking + applications. Set this to 0 if you don't want to allow application + connections. (Default: 9050) + +**SocksListenAddress** __IP__[:__PORT__]:: + Bind to this address to listen for connections from Socks-speaking + applications. (Default: 127.0.0.1) You can also specify a port (e.g. + 192.168.0.1:9100). This directive can be specified multiple times to bind + to multiple addresses/ports. + +**SocksPolicy** __policy__,__policy__,__...__:: + Set an entrance policy for this server, to limit who can connect to the + SocksPort and DNSPort ports. The policies have the same form as exit + policies below. + +**SocksTimeout** __NUM__:: + Let a socks connection wait NUM seconds handshaking, and NUM seconds + unattached waiting for an appropriate circuit, before we fail it. (Default: + 2 minutes.) + +**TrackHostExits** __host__,__.domain__,__...__:: + For each value in the comma separated list, Tor will track recent + connections to hosts that match this value and attempt to reuse the same + exit node for each. If the value is prepended with a \'.\', it is treated as + matching an entire domain. If one of the values is just a \'.', it means + match everything. This option is useful if you frequently connect to sites + that will expire all your authentication cookies (i.e. log you out) if + your IP address changes. Note that this option does have the disadvantage + of making it more clear that a given history is associated with a single + user. However, most people who would wish to observe this will observe it + through cookies or other protocol-specific means anyhow. + +**TrackHostExitsExpire** __NUM__:: + Since exit servers go up and down, it is desirable to expire the + association between host and exit server after NUM seconds. The default is + 1800 seconds (30 minutes). + +**UpdateBridgesFromAuthority** **0**|**1**:: + When set (along with UseBridges), Tor will try to fetch bridge descriptors + from the configured bridge authorities when feasible. It will fall back to + a direct request if the authority responds with a 404. (Default: 0) + +**UseBridges** **0**|**1**:: + When set, Tor will fetch descriptors for each bridge listed in the "Bridge" + config lines, and use these relays as both entry guards and directory + guards. (Default: 0) + +**UseEntryGuards** **0**|**1**:: + If this option is set to 1, we pick a few long-term entry servers, and try + to stick with them. This is desirable because constantly changing servers + increases the odds that an adversary who owns some servers will observe a + fraction of your paths. (Defaults to 1.) + +**NumEntryGuards** __NUM__:: + If UseEntryGuards is set to 1, we will try to pick a total of NUM routers + as long-term entries for our circuits. (Defaults to 3.) + +**SafeSocks** **0**|**1**:: + When this option is enabled, Tor will reject application connections that + use unsafe variants of the socks protocol -- ones that only provide an IP + address, meaning the application is doing a DNS resolve first. + Specifically, these are socks4 and socks5 when not doing remote DNS. + (Defaults to 0.) + +**TestSocks** **0**|**1**:: + When this option is enabled, Tor will make a notice-level log entry for + each connection to the Socks port indicating whether the request used a + safe socks protocol or an unsafe one (see above entry on SafeSocks). This + helps to determine whether an application using Tor is possibly leaking + DNS requests. (Default: 0) + +**WarnUnsafeSocks** **0**|**1**:: + When this option is enabled, Tor will warn whenever a request is + received that only contains an IP address instead of a hostname. Allowing + applications to do DNS resolves themselves is usually a bad idea and + can leak your location to attackers. (Default: 1) + +**VirtualAddrNetwork** __Address__/__bits__:: + When Tor needs to assign a virtual (unused) address because of a MAPADDRESS + command from the controller or the AutomapHostsOnResolve feature, Tor + picks an unassigned address from this range. (Default: + 127.192.0.0/10) + + + + When providing proxy server service to a network of computers using a tool + like dns-proxy-tor, change this address to "10.192.0.0/10" or + "172.16.0.0/12". The default **VirtualAddrNetwork** address range on a + properly configured machine will route to the loopback interface. For + local use, no change to the default VirtualAddrNetwork setting is needed. + +**AllowNonRFC953Hostnames** **0**|**1**:: + When this option is disabled, Tor blocks hostnames containing illegal + characters (like @ and :) rather than sending them to an exit node to be + resolved. This helps trap accidental attempts to resolve URLs and so on. + (Default: 0) + +**AllowDotExit** **0**|**1**:: + If enabled, we convert "www.google.com.foo.exit" addresses on the + SocksPort/TransPort/NATDPort into "www.google.com" addresses that exit from + the node "foo". Disabled by default since attacking websites and exit + relays can use it to manipulate your path selection. (Default: 0) + +**FastFirstHopPK** **0**|**1**:: + When this option is disabled, Tor uses the public key step for the first + hop of creating circuits. Skipping it is generally safe since we have + already used TLS to authenticate the relay and to establish forward-secure + keys. Turning this option off makes circuit building slower. + + + + Note that Tor will always use the public key step for the first hop if it's + operating as a relay, and it will never use the public key step if it + doesn't yet know the onion key of the first hop. (Default: 1) + +**TransPort** __PORT__:: + If non-zero, enables transparent proxy support on __PORT__ (by convention, + 9040). Requires OS support for transparent proxies, such as BSDs' pf or + Linux's IPTables. If you're planning to use Tor as a transparent proxy for + a network, you'll want to examine and change VirtualAddrNetwork from the + default setting. You'll also want to set the TransListenAddress option for + the network you'd like to proxy. (Default: 0). + +**TransListenAddress** __IP__[:__PORT__]:: + Bind to this address to listen for transparent proxy connections. (Default: + 127.0.0.1). This is useful for exporting a transparent proxy server to an + entire network. + +**NATDPort** __PORT__:: + Allow old versions of ipfw (as included in old versions of FreeBSD, etc.) + to send connections through Tor using the NATD protocol. This option is + only for people who cannot use TransPort. + +**NATDListenAddress** __IP__[:__PORT__]:: + Bind to this address to listen for NATD connections. (Default: 127.0.0.1). + +**AutomapHostsOnResolve** **0**|**1**:: + When this option is enabled, and we get a request to resolve an address + that ends with one of the suffixes in **AutomapHostsSuffixes**, we map an + unused virtual address to that address, and return the new virtual address. + This is handy for making ".onion" addresses work with applications that + resolve an address and then connect to it. (Default: 0). + +**AutomapHostsSuffixes** __SUFFIX__,__SUFFIX__,__...__:: + A comma-separated list of suffixes to use with **AutomapHostsOnResolve**. + The "." suffix is equivalent to "all addresses." (Default: .exit,.onion). + +**DNSPort** __PORT__:: + If non-zero, Tor listens for UDP DNS requests on this port and resolves + them anonymously. (Default: 0). + +**DNSListenAddress** __IP__[:__PORT__]:: + Bind to this address to listen for DNS connections. (Default: 127.0.0.1). + +**ClientDNSRejectInternalAddresses** **0**|**1**:: + If true, Tor does not believe any anonymously retrieved DNS answer that + tells it that an address resolves to an internal address (like 127.0.0.1 or + 192.168.0.1). This option prevents certain browser-based attacks; don't + turn it off unless you know what you're doing. (Default: 1). + +**ClientRejectInternalAddresses** **0**|**1**:: + If true, Tor does not try to fulfill requests to connect to an internal + address (like 127.0.0.1 or 192.168.0.1) __unless a exit node is + specifically requested__ (for example, via a .exit hostname, or a + controller request). (Default: 1). + +**DownloadExtraInfo** **0**|**1**:: + If true, Tor downloads and caches "extra-info" documents. These documents + contain information about servers other than the information in their + regular router descriptors. Tor does not use this information for anything + itself; to save bandwidth, leave this option turned off. (Default: 0). + +**FallbackNetworkstatusFile** __FILENAME__:: + If Tor doesn't have a cached networkstatus file, it starts out using this + one instead. Even if this file is out of date, Tor can still use it to + learn about directory mirrors, so it doesn't need to put load on the + authorities. (Default: None). + +**WarnPlaintextPorts** __port__,__port__,__...__:: + Tells Tor to issue a warnings whenever the user tries to make an anonymous + connection to one of these ports. This option is designed to alert users + to services that risk sending passwords in the clear. (Default: + 23,109,110,143). + +**RejectPlaintextPorts** __port__,__port__,__...__:: + Like WarnPlaintextPorts, but instead of warning about risky port uses, Tor + will instead refuse to make the connection. (Default: None). + +**AllowSingleHopCircuits** **0**|**1**:: + When this option is set, the attached Tor controller can use relays + that have the **AllowSingleHopExits** option turned on to build + one-hop Tor connections. (Default: 0) + +SERVER OPTIONS +-------------- + +The following options are useful only for servers (that is, if ORPort +is non-zero): + +**Address** __address__:: + The IP address or fully qualified domain name of this server (e.g. + moria.mit.edu). You can leave this unset, and Tor will guess your IP + address. This IP address is the one used to tell clients and other + servers where to find your Tor server; it doesn't affect the IP that your + Tor client binds to. To bind to a different address, use the + *ListenAddress and OutboundBindAddress options. + +**AllowSingleHopExits** **0**|**1**:: + This option controls whether clients can use this server as a single hop + proxy. If set to 1, clients can use this server as an exit even if it is + the only hop in the circuit. Note that most clients will refuse to use + servers that set this option, since most clients have + ExcludeSingleHopRelays set. (Default: 0) + +**AssumeReachable** **0**|**1**:: + This option is used when bootstrapping a new Tor network. If set to 1, + don't do self-reachability testing; just upload your server descriptor + immediately. If **AuthoritativeDirectory** is also set, this option + instructs the dirserver to bypass remote reachability testing too and list + all connected servers as running. + +**BridgeRelay** **0**|**1**:: + Sets the relay to act as a "bridge" with respect to relaying connections + from bridge users to the Tor network. It mainly causes Tor to publish a + server descriptor to the bridge database, rather than publishing a relay + descriptor to the public directory authorities. + +**ContactInfo** __email_address__:: + Administrative contact information for server. This line might get picked + up by spam harvesters, so you may want to obscure the fact that it's an + email address. + +**ExitPolicy** __policy__,__policy__,__...__:: + Set an exit policy for this server. Each policy is of the form + "**accept**|**reject** __ADDR__[/__MASK__][:__PORT__]". If /__MASK__ is + omitted then this policy just applies to the host given. Instead of giving + a host or network you can also use "\*" to denote the universe (0.0.0.0/0). + __PORT__ can be a single port number, an interval of ports + "__FROM_PORT__-__TO_PORT__", or "\*". If __PORT__ is omitted, that means + "\*". + + + + For example, "accept 18.7.22.69:\*,reject 18.0.0.0/8:\*,accept \*:\*" would + reject any traffic destined for MIT except for web.mit.edu, and accept + anything else. + + + + To specify all internal and link-local networks (including 0.0.0.0/8, + 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, and + 172.16.0.0/12), you can use the "private" alias instead of an address. + These addresses are rejected by default (at the beginning of your exit + policy), along with your public IP address, unless you set the + ExitPolicyRejectPrivate config option to 0. For example, once you've done + that, you could allow HTTP to 127.0.0.1 and block all other connections to + internal networks with "accept 127.0.0.1:80,reject private:\*", though that + may also allow connections to your own computer that are addressed to its + public (external) IP address. See RFC 1918 and RFC 3330 for more details + about internal and reserved IP address space. + + + + This directive can be specified multiple times so you don't have to put it + all on one line. + + + + Policies are considered first to last, and the first match wins. If you + want to \_replace_ the default exit policy, end your exit policy with + either a reject \*:* or an accept \*:*. Otherwise, you're \_augmenting_ + (prepending to) the default exit policy. The default exit policy is: + + + reject *:25 + reject *:119 + reject *:135-139 + reject *:445 + reject *:563 + reject *:1214 + reject *:4661-4666 + reject *:6346-6429 + reject *:6699 + reject *:6881-6999 + accept *:* + +**ExitPolicyRejectPrivate** **0**|**1**:: + Reject all private (local) networks, along with your own public IP address, + at the beginning of your exit policy. See above entry on ExitPolicy. + (Default: 1) + +**MaxOnionsPending** __NUM__:: + If you have more than this number of onionskins queued for decrypt, reject + new ones. (Default: 100) + +**MyFamily** __node__,__node__,__...__:: + Declare that this Tor server is controlled or administered by a group or + organization identical or similar to that of the other servers, defined by + their identity fingerprints or nicknames. When two servers both declare + that they are in the same \'family', Tor clients will not use them in the + same circuit. (Each server only needs to list the other servers in its + family; it doesn't need to list itself, but it won't hurt.) + +**Nickname** __name__:: + Set the server's nickname to \'name'. Nicknames must be between 1 and 19 + characters inclusive, and must contain only the characters [a-zA-Z0-9]. + +**NumCPUs** __num__:: + How many processes to use at once for decrypting onionskins. (Default: 1) + +**ORPort** __PORT__:: + Advertise this port to listen for connections from Tor clients and servers. + +**ORListenAddress** __IP__[:__PORT__]:: + Bind to this IP address to listen for connections from Tor clients and + servers. If you specify a port, bind to this port rather than the one + specified in ORPort. (Default: 0.0.0.0) This directive can be specified + multiple times to bind to multiple addresses/ports. + +**PublishServerDescriptor** **0**|**1**|**v1**|**v2**|**v3**|**bridge**,**...**:: + This option specifies which descriptors Tor will publish when acting as + a relay. You can + choose multiple arguments, separated by commas. + + + If this option is set to 0, Tor will not publish its + descriptors to any directories. (This is useful if you're testing + out your server, or if you're using a Tor controller that handles directory + publishing for you.) Otherwise, Tor will publish its descriptors of all + type(s) specified. The default is "1", + which means "if running as a server, publish the + appropriate descriptors to the authorities". + +**ShutdownWaitLength** __NUM__:: + When we get a SIGINT and we're a server, we begin shutting down: + we close listeners and start refusing new circuits. After **NUM** + seconds, we exit. If we get a second SIGINT, we exit immedi- + ately. (Default: 30 seconds) + + +**AccountingMax** __N__ **bytes**|**KB**|**MB**|**GB**|**TB**:: + Never send more than the specified number of bytes in a given accounting + period, or receive more than that number in the period. For example, with + AccountingMax set to 1 GB, a server could send 900 MB and receive 800 MB + and continue running. It will only hibernate once one of the two reaches 1 + GB. When the number of bytes gets low, Tor will stop accepting new + connections and circuits. When the number of bytes + is exhausted, Tor will hibernate until some + time in the next accounting period. To prevent all servers from waking at + the same time, Tor will also wait until a random point in each period + before waking up. If you have bandwidth cost issues, enabling hibernation + is preferable to setting a low bandwidth, since it provides users with a + collection of fast servers that are up some of the time, which is more + useful than a set of slow servers that are always "available". + +**AccountingStart** **day**|**week**|**month** [__day__] __HH:MM__:: + Specify how long accounting periods last. If **month** is given, each + accounting period runs from the time __HH:MM__ on the __dayth__ day of one + month to the same day and time of the next. (The day must be between 1 and + 28.) If **week** is given, each accounting period runs from the time __HH:MM__ + of the __dayth__ day of one week to the same day and time of the next week, + with Monday as day 1 and Sunday as day 7. If **day** is given, each + accounting period runs from the time __HH:MM__ each day to the same time on + the next day. All times are local, and given in 24-hour time. (Defaults to + "month 1 0:00".) + +**RefuseUnknownExits** **0**|**1**|**auto**:: + Prevent nodes that don't appear in the consensus from exiting using this + relay. If the option is 1, we always block exit attempts from such + nodes; if it's 0, we never do, and if the option is "auto", then we do + whatever the authorities suggest in the consensus. (Defaults to auto.) + +**ServerDNSResolvConfFile** __filename__:: + Overrides the default DNS configuration with the configuration in + __filename__. The file format is the same as the standard Unix + "**resolv.conf**" file (7). This option, like all other ServerDNS options, + only affects name lookups that your server does on behalf of clients. + (Defaults to use the system DNS configuration.) + +**ServerDNSAllowBrokenConfig** **0**|**1**:: + If this option is false, Tor exits immediately if there are problems + parsing the system DNS configuration or connecting to nameservers. + Otherwise, Tor continues to periodically retry the system nameservers until + it eventually succeeds. (Defaults to "1".) + +**ServerDNSSearchDomains** **0**|**1**:: + If set to 1, then we will search for addresses in the local search domain. + For example, if this system is configured to believe it is in + "example.com", and a client tries to connect to "www", the client will be + connected to "www.example.com". This option only affects name lookups that + your server does on behalf of clients. (Defaults to "0".) + +**ServerDNSDetectHijacking** **0**|**1**:: + When this option is set to 1, we will test periodically to determine + whether our local nameservers have been configured to hijack failing DNS + requests (usually to an advertising site). If they are, we will attempt to + correct this. This option only affects name lookups that your server does + on behalf of clients. (Defaults to "1".) + +**ServerDNSTestAddresses** __address__,__address__,__...__:: + When we're detecting DNS hijacking, make sure that these __valid__ addresses + aren't getting redirected. If they are, then our DNS is completely useless, + and we'll reset our exit policy to "reject *:*". This option only affects + name lookups that your server does on behalf of clients. (Defaults to + "www.google.com, www.mit.edu, www.yahoo.com, www.slashdot.org".) + +**ServerDNSAllowNonRFC953Hostnames** **0**|**1**:: + When this option is disabled, Tor does not try to resolve hostnames + containing illegal characters (like @ and :) rather than sending them to an + exit node to be resolved. This helps trap accidental attempts to resolve + URLs and so on. This option only affects name lookups that your server does + on behalf of clients. (Default: 0) + +**BridgeRecordUsageByCountry** **0**|**1**:: + When this option is enabled and BridgeRelay is also enabled, and we have + GeoIP data, Tor keeps a keep a per-country count of how many client + addresses have contacted it so that it can help the bridge authority guess + which countries have blocked access to it. (Default: 1) + +**ServerDNSRandomizeCase** **0**|**1**:: + When this option is set, Tor sets the case of each character randomly in + outgoing DNS requests, and makes sure that the case matches in DNS replies. + This so-called "0x20 hack" helps resist some types of DNS poisoning attack. + For more information, see "Increased DNS Forgery Resistance through + 0x20-Bit Encoding". This option only affects name lookups that your server + does on behalf of clients. (Default: 1) + +**GeoIPFile** __filename__:: + A filename containing GeoIP data, for use with BridgeRecordUsageByCountry. + +**CellStatistics** **0**|**1**:: + When this option is enabled, Tor writes statistics on the mean time that + cells spend in circuit queues to disk every 24 hours. (Default: 0) + +**DirReqStatistics** **0**|**1**:: + When this option is enabled, Tor writes statistics on the number and + response time of network status requests to disk every 24 hours. + (Default: 0) + +**EntryStatistics** **0**|**1**:: + When this option is enabled, Tor writes statistics on the number of + directly connecting clients to disk every 24 hours. (Default: 0) + +**ExitPortStatistics** **0**|**1**:: + When this option is enabled, Tor writes statistics on the number of relayed + bytes and opened stream per exit port to disk every 24 hours. (Default: 0) + +**ExtraInfoStatistics** **0**|**1**:: + When this option is enabled, Tor includes previously gathered statistics in + its extra-info documents that it uploads to the directory authorities. + (Default: 0) + +DIRECTORY SERVER OPTIONS +------------------------ + +The following options are useful only for directory servers (that is, +if DirPort is non-zero): + +**AuthoritativeDirectory** **0**|**1**:: + When this option is set to 1, Tor operates as an authoritative directory + server. Instead of caching the directory, it generates its own list of + good servers, signs it, and sends that to the clients. Unless the clients + already have you listed as a trusted directory, you probably do not want + to set this option. Please coordinate with the other admins at + tor-ops@torproject.org if you think you should be a directory. + +**DirPortFrontPage** __FILENAME__:: + When this option is set, it takes an HTML file and publishes it as "/" on + the DirPort. Now relay operators can provide a disclaimer without needing + to set up a separate webserver. There's a sample disclaimer in + contrib/tor-exit-notice.html. + +**V1AuthoritativeDirectory** **0**|**1**:: + When this option is set in addition to **AuthoritativeDirectory**, Tor + generates version 1 directory and running-routers documents (for legacy + Tor clients up to 0.1.0.x). + +**V2AuthoritativeDirectory** **0**|**1**:: + When this option is set in addition to **AuthoritativeDirectory**, Tor + generates version 2 network statuses and serves descriptors, etc as + described in doc/spec/dir-spec-v2.txt (for Tor clients and servers running + 0.1.1.x and 0.1.2.x). + +**V3AuthoritativeDirectory** **0**|**1**:: + When this option is set in addition to **AuthoritativeDirectory**, Tor + generates version 3 network statuses and serves descriptors, etc as + described in doc/spec/dir-spec.txt (for Tor clients and servers running at + least 0.2.0.x). + +**VersioningAuthoritativeDirectory** **0**|**1**:: + When this option is set to 1, Tor adds information on which versions of + Tor are still believed safe for use to the published directory. Each + version 1 authority is automatically a versioning authority; version 2 + authorities provide this service optionally. See **RecommendedVersions**, + **RecommendedClientVersions**, and **RecommendedServerVersions**. + +**NamingAuthoritativeDirectory** **0**|**1**:: + When this option is set to 1, then the server advertises that it has + opinions about nickname-to-fingerprint bindings. It will include these + opinions in its published network-status pages, by listing servers with + the flag "Named" if a correct binding between that nickname and fingerprint + has been registered with the dirserver. Naming dirservers will refuse to + accept or publish descriptors that contradict a registered binding. See + **approved-routers** in the **FILES** section below. + +**HSAuthoritativeDir** **0**|**1**:: + When this option is set in addition to **AuthoritativeDirectory**, Tor also + accepts and serves v0 hidden service descriptors, + which are produced and used by Tor 0.2.1.x and older. (Default: 0) + +**HidServDirectoryV2** **0**|**1**:: + When this option is set, Tor accepts and serves v2 hidden service + descriptors. Setting DirPort is not required for this, because clients + connect via the ORPort by default. (Default: 1) + +**BridgeAuthoritativeDir** **0**|**1**:: + When this option is set in addition to **AuthoritativeDirectory**, Tor + accepts and serves router descriptors, but it caches and serves the main + networkstatus documents rather than generating its own. (Default: 0) + +**MinUptimeHidServDirectoryV2** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: + Minimum uptime of a v2 hidden service directory to be accepted as such by + authoritative directories. (Default: 24 hours) + +**DirPort** __PORT__:: + Advertise the directory service on this port. + +**DirListenAddress** __IP__[:__PORT__]:: + Bind the directory service to this address. If you specify a port, bind to + this port rather than the one specified in DirPort. (Default: 0.0.0.0) + This directive can be specified multiple times to bind to multiple + addresses/ports. + +**DirPolicy** __policy__,__policy__,__...__:: + Set an entrance policy for this server, to limit who can connect to the + directory ports. The policies have the same form as exit policies above. + +**FetchV2Networkstatus** **0**|**1**:: + If set, we try to fetch the (obsolete, unused) version 2 network status + consensus documents from the directory authorities. No currently + supported Tor version uses them. (Default: 0.) + + +DIRECTORY AUTHORITY SERVER OPTIONS +---------------------------------- + +**RecommendedVersions** __STRING__:: + STRING is a comma-separated list of Tor versions currently believed to be + safe. The list is included in each directory, and nodes which pull down the + directory learn whether they need to upgrade. This option can appear + multiple times: the values from multiple lines are spliced together. When + this is set then **VersioningAuthoritativeDirectory** should be set too. + +**RecommendedClientVersions** __STRING__:: + STRING is a comma-separated list of Tor versions currently believed to be + safe for clients to use. This information is included in version 2 + directories. If this is not set then the value of **RecommendedVersions** + is used. When this is set then **VersioningAuthoritativeDirectory** should + be set too. + +**RecommendedServerVersions** __STRING__:: + STRING is a comma-separated list of Tor versions currently believed to be + safe for servers to use. This information is included in version 2 + directories. If this is not set then the value of **RecommendedVersions** + is used. When this is set then **VersioningAuthoritativeDirectory** should + be set too. + +**ConsensusParams** __STRING__:: + STRING is a space-separated list of key=value pairs that Tor will include + in the "params" line of its networkstatus vote. + +**DirAllowPrivateAddresses** **0**|**1**:: + If set to 1, Tor will accept router descriptors with arbitrary "Address" + elements. Otherwise, if the address is not an IP address or is a private IP + address, it will reject the router descriptor. Defaults to 0. + +**AuthDirBadDir** __AddressPattern...__:: + Authoritative directories only. A set of address patterns for servers that + will be listed as bad directories in any network status document this + authority publishes, if **AuthDirListBadDirs** is set. + +**AuthDirBadExit** __AddressPattern...__:: + Authoritative directories only. A set of address patterns for servers that + will be listed as bad exits in any network status document this authority + publishes, if **AuthDirListBadExits** is set. + +**AuthDirInvalid** __AddressPattern...__:: + Authoritative directories only. A set of address patterns for servers that + will never be listed as "valid" in any network status document that this + authority publishes. + +**AuthDirReject** __AddressPattern__...:: + Authoritative directories only. A set of address patterns for servers that + will never be listed at all in any network status document that this + authority publishes, or accepted as an OR address in any descriptor + submitted for publication by this authority. + +**AuthDirListBadDirs** **0**|**1**:: + Authoritative directories only. If set to 1, this directory has some + opinion about which nodes are unsuitable as directory caches. (Do not set + this to 1 unless you plan to list non-functioning directories as bad; + otherwise, you are effectively voting in favor of every declared + directory.) + +**AuthDirListBadExits** **0**|**1**:: + Authoritative directories only. If set to 1, this directory has some + opinion about which nodes are unsuitable as exit nodes. (Do not set this to + 1 unless you plan to list non-functioning exits as bad; otherwise, you are + effectively voting in favor of every declared exit as an exit.) + +**AuthDirRejectUnlisted** **0**|**1**:: + Authoritative directories only. If set to 1, the directory server rejects + all uploaded server descriptors that aren't explicitly listed in the + fingerprints file. This acts as a "panic button" if we get hit with a Sybil + attack. (Default: 0) + +**AuthDirMaxServersPerAddr** __NUM__:: + Authoritative directories only. The maximum number of servers that we will + list as acceptable on a single IP address. Set this to "0" for "no limit". + (Default: 2) + +**AuthDirMaxServersPerAuthAddr** __NUM__:: + Authoritative directories only. Like AuthDirMaxServersPerAddr, but applies + to addresses shared with directory authorities. (Default: 5) + +**BridgePassword** __Password__:: + If set, contains an HTTP authenticator that tells a bridge authority to + serve all requested bridge information. Used for debugging. (Default: + not set.) + +**V3AuthVotingInterval** __N__ **minutes**|**hours**:: + V3 authoritative directories only. Configures the server's preferred voting + interval. Note that voting will __actually__ happen at an interval chosen + by consensus from all the authorities' preferred intervals. This time + SHOULD divide evenly into a day. (Default: 1 hour) + +**V3AuthVoteDelay** __N__ **minutes**|**hours**:: + V3 authoritative directories only. Configures the server's preferred delay + between publishing its vote and assuming it has all the votes from all the + other authorities. Note that the actual time used is not the server's + preferred time, but the consensus of all preferences. (Default: 5 minutes.) + +**V3AuthDistDelay** __N__ **minutes**|**hours**:: + V3 authoritative directories only. Configures the server's preferred delay + between publishing its consensus and signature and assuming it has all the + signatures from all the other authorities. Note that the actual time used + is not the server's preferred time, but the consensus of all preferences. + (Default: 5 minutes.) + +**V3AuthNIntervalsValid** __NUM__:: + V3 authoritative directories only. Configures the number of VotingIntervals + for which each consensus should be valid for. Choosing high numbers + increases network partitioning risks; choosing low numbers increases + directory traffic. Note that the actual number of intervals used is not the + server's preferred number, but the consensus of all preferences. Must be at + least 2. (Default: 3.) + +**V3BandwidthsFile** __FILENAME__:: + V3 authoritative directories only. Configures the location of the + bandiwdth-authority generated file storing information on relays' measured + bandwidth capacities. (Default: unset.) + +**V3AuthUseLegacyKey** **0**|**1**:: + If set, the directory authority will sign consensuses not only with its + own signing key, but also with a "legacy" key and certificate with a + different identity. This feature is used to migrate directory authority + keys in the event of a compromise. (Default: 0.) + +**RephistTrackTime** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: + Tells an authority, or other node tracking node reliability and history, + that fine-grained information about nodes can be discarded when it hasn't + changed for a given amount of time. (Default: 24 hours) + +HIDDEN SERVICE OPTIONS +---------------------- + +The following options are used to configure a hidden service. + +**HiddenServiceDir** __DIRECTORY__:: + Store data files for a hidden service in DIRECTORY. Every hidden service + must have a separate directory. You may use this option multiple times to + specify multiple services. + +**HiddenServicePort** __VIRTPORT__ [__TARGET__]:: + Configure a virtual port VIRTPORT for a hidden service. You may use this + option multiple times; each time applies to the service using the most + recent hiddenservicedir. By default, this option maps the virtual port to + the same port on 127.0.0.1. You may override the target port, address, or + both by specifying a target of addr, port, or addr:port. You may also have + multiple lines with the same VIRTPORT: when a user connects to that + VIRTPORT, one of the TARGETs from those lines will be chosen at random. + +**PublishHidServDescriptors** **0**|**1**:: + If set to 0, Tor will run any hidden services you configure, but it won't + advertise them to the rendezvous directory. This option is only useful if + you're using a Tor controller that handles hidserv publishing for you. + (Default: 1) + +**HiddenServiceVersion** __version__,__version__,__...__:: + A list of rendezvous service descriptor versions to publish for the hidden + service. Currently, only version 2 is supported. (Default: 2) + +**HiddenServiceAuthorizeClient** __auth-type__ __client-name__,__client-name__,__...__:: + If configured, the hidden service is accessible for authorized clients + only. The auth-type can either be \'basic' for a general-purpose + authorization protocol or \'stealth' for a less scalable protocol that also + hides service activity from unauthorized clients. Only clients that are + listed here are authorized to access the hidden service. Valid client names + are 1 to 19 characters long and only use characters in A-Za-z0-9+-_ (no + spaces). If this option is set, the hidden service is not accessible for + clients without authorization any more. Generated authorization data can be + found in the hostname file. Clients need to put this authorization data in + their configuration file using **HidServAuth**. + +**RendPostPeriod** __N__ **seconds**|**minutes**|**hours**|**days**|**weeks**:: + Every time the specified period elapses, Tor uploads any rendezvous + service descriptors to the directory servers. This information is also + uploaded whenever it changes. (Default: 1 hour) + +TESTING NETWORK OPTIONS +----------------------- + +The following options are used for running a testing Tor network. + +**TestingTorNetwork** **0**|**1**:: + If set to 1, Tor adjusts default values of the configuration options below, + so that it is easier to set up a testing Tor network. May only be set if + non-default set of DirServers is set. Cannot be unset while Tor is running. + (Default: 0) + + + ServerDNSAllowBrokenConfig 1 + DirAllowPrivateAddresses 1 + EnforceDistinctSubnets 0 + AssumeReachable 1 + AuthDirMaxServersPerAddr 0 + AuthDirMaxServersPerAuthAddr 0 + ClientDNSRejectInternalAddresses 0 + ClientRejectInternalAddresses 0 + ExitPolicyRejectPrivate 0 + V3AuthVotingInterval 5 minutes + V3AuthVoteDelay 20 seconds + V3AuthDistDelay 20 seconds + MinUptimeHidServDirectoryV2 0 seconds + TestingV3AuthInitialVotingInterval 5 minutes + TestingV3AuthInitialVoteDelay 20 seconds + TestingV3AuthInitialDistDelay 20 seconds + TestingAuthDirTimeToLearnReachability 0 minutes + TestingEstimatedDescriptorPropagationTime 0 minutes + +**TestingV3AuthInitialVotingInterval** __N__ **minutes**|**hours**:: + Like V3AuthVotingInterval, but for initial voting interval before the first + consensus has been created. Changing this requires that + **TestingTorNetwork** is set. (Default: 30 minutes) + +**TestingV3AuthInitialVoteDelay** __N__ **minutes**|**hours**:: + Like TestingV3AuthInitialVoteDelay, but for initial voting interval before + the first consensus has been created. Changing this requires that + **TestingTorNetwork** is set. (Default: 5 minutes) + +**TestingV3AuthInitialDistDelay** __N__ **minutes**|**hours**:: + Like TestingV3AuthInitialDistDelay, but for initial voting interval before + the first consensus has been created. Changing this requires that + **TestingTorNetwork** is set. (Default: 5 minutes) + +**TestingAuthDirTimeToLearnReachability** __N__ **minutes**|**hours**:: + After starting as an authority, do not make claims about whether routers + are Running until this much time has passed. Changing this requires + that **TestingTorNetwork** is set. (Default: 30 minutes) + +**TestingEstimatedDescriptorPropagationTime** __N__ **minutes**|**hours**:: + Clients try downloading router descriptors from directory caches after this + time. Changing this requires that **TestingTorNetwork** is set. (Default: + 10 minutes) + +SIGNALS +------- + +Tor catches the following signals: + +**SIGTERM**:: + Tor will catch this, clean up and sync to disk if necessary, and exit. + +**SIGINT**:: + Tor clients behave as with SIGTERM; but Tor servers will do a controlled + slow shutdown, closing listeners and waiting 30 seconds before exiting. + (The delay can be configured with the ShutdownWaitLength config option.) + +**SIGHUP**:: + The signal instructs Tor to reload its configuration (including closing and + reopening logs), and kill and restart its helper processes if applicable. + +**SIGUSR1**:: + Log statistics about current connections, past connections, and throughput. + +**SIGUSR2**:: + Switch all logs to loglevel debug. You can go back to the old loglevels by + sending a SIGHUP. + +**SIGCHLD**:: + Tor receives this signal when one of its helper processes has exited, so it + can clean up. + +**SIGPIPE**:: + Tor catches this signal and ignores it. + +**SIGXFSZ**:: + If this signal exists on your platform, Tor catches and ignores it. + +FILES +----- + +**@CONFDIR@/torrc**:: + The configuration file, which contains "option value" pairs. + +**@LOCALSTATEDIR@/lib/tor/**:: + The tor process stores keys and other data here. + +__DataDirectory__**/cached-status/**:: + The most recently downloaded network status document for each authority. + Each file holds one such document; the filenames are the hexadecimal + identity key fingerprints of the directory authorities. + +__DataDirectory__**/cached-descriptors** and **cached-descriptors.new**:: + These files hold downloaded router statuses. Some routers may appear more + than once; if so, the most recently published descriptor is used. Lines + beginning with @-signs are annotations that contain more information about + a given router. The ".new" file is an append-only journal; when it gets + too large, all entries are merged into a new cached-descriptors file. + +__DataDirectory__**/cached-routers** and **cached-routers.new**:: + Obsolete versions of cached-descriptors and cached-descriptors.new. When + Tor can't find the newer files, it looks here instead. + +__DataDirectory__**/state**:: + A set of persistent key-value mappings. These are documented in + the file. These include: + - The current entry guards and their status. + - The current bandwidth accounting values (unused so far; see + below). + - When the file was last written + - What version of Tor generated the state file + - A short history of bandwidth usage, as produced in the router + descriptors. + +__DataDirectory__**/bw_accounting**:: + Used to track bandwidth accounting values (when the current period starts + and ends; how much has been read and written so far this period). This file + is obsolete, and the data is now stored in the \'state' file as well. Only + used when bandwidth accounting is enabled. + +__DataDirectory__**/control_auth_cookie**:: + Used for cookie authentication with the controller. Location can be + overridden by the CookieAuthFile config option. Regenerated on startup. See + control-spec.txt for details. Only used when cookie authentication is + enabled. + +__DataDirectory__**/keys/***:: + Only used by servers. Holds identity keys and onion keys. + +__DataDirectory__**/fingerprint**:: + Only used by servers. Holds the fingerprint of the server's identity key. + +__DataDirectory__**/approved-routers**:: + Only for naming authoritative directory servers (see + **NamingAuthoritativeDirectory**). This file lists nickname to identity + bindings. Each line lists a nickname and a fingerprint separated by + whitespace. See your **fingerprint** file in the __DataDirectory__ for an + example line. If the nickname is **!reject** then descriptors from the + given identity (fingerprint) are rejected by this server. If it is + **!invalid** then descriptors are accepted but marked in the directory as + not valid, that is, not recommended. + +__DataDirectory__**/router-stability**:: + Only used by authoritative directory servers. Tracks measurements for + router mean-time-between-failures so that authorities have a good idea of + how to set their Stable flags. + +__HiddenServiceDirectory__**/hostname**:: + The <base32-encoded-fingerprint>.onion domain name for this hidden service. + If the hidden service is restricted to authorized clients only, this file + also contains authorization data for all clients. + +__HiddenServiceDirectory__**/private_key**:: + The private key for this hidden service. + +__HiddenServiceDirectory__**/client_keys**:: + Authorization data for a hidden service that is only accessible by + authorized clients. + +SEE ALSO +-------- +**privoxy**(1), **tsocks**(1), **torify**(1) + + +**https://www.torproject.org/** + + +BUGS +---- + +Plenty, probably. Tor is still in development. Please report them. + +AUTHORS +------- +Roger Dingledine [arma at mit.edu], Nick Mathewson [nickm at alum.mit.edu]. + diff --git a/doc/torify.1.txt b/doc/torify.1.txt new file mode 100644 index 000000000..ca2c385c9 --- /dev/null +++ b/doc/torify.1.txt @@ -0,0 +1,50 @@ +// Copyright (c) The Tor Project, Inc. +// See LICENSE for licensing information +// This is an asciidoc file used to generate the manpage/html reference. +// Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +torify(1) +========= +Peter Palfrader +Jacob Appelbaum + +NAME +---- +torify - wrapper for torsocks or tsocks and tor + +SYNOPSIS +-------- +**torify** __application__ [__application's__ __arguments__] + +DESCRIPTION +----------- +**torify** is a simple wrapper that attempts to find the best underlying Tor +wrapper available on a system. It calls torsocks or tsocks with a tor specific +configuration file. + + +torsocks is an improved wrapper that explictly rejects UDP, safely resolves DNS +lookups and properly socksifies your TCP connections. + + +tsocks itself is a wrapper between the tsocks library and the application that +you would like to run socksified. + + +Please note that since both method use LD_PRELOAD, torify cannot be applied to +suid binaries. + +WARNING +------- +You should also be aware that the way tsocks currently works only TCP +connections are socksified. Be aware that this will in most circumstances not +include hostname lookups which would still be routed through your normal system +resolver to your usual resolving nameservers. The **tor-resolve**(1) tool can be +useful as a workaround in some cases. The Tor FAQ at +https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ might have further +information on this subject. + + +When used with torsocks, torify should not leak DNS requests or UDP data. + + +Both will leak ICMP data. + +SEE ALSO +-------- +**tor**(1), **tor-resolve**(1), **torsocks**(1), **tsocks**(1), +**tsocks.conf**(5). diff --git a/doc/translations.txt b/doc/translations.txt index 874abe1bc..06d16f446 100644 --- a/doc/translations.txt +++ b/doc/translations.txt @@ -31,7 +31,7 @@ The current pootle configuration is checked into subversion as well: TorCheck uses our translation portal to accept translations. Users use the portal to check in their changes. To make use of the translations -that users have commited to the translations/ subversion module, you'll +that users have committed to the translations/ subversion module, you'll need to ensure that you have a current checked out copy of TorCheck: cd check/trunk/i18n/ @@ -75,59 +75,32 @@ And finally check in the changes: Torbutton uses our translation portal to accept translations. Users use the portal to check in their changes. -To make use of the translations that users have commited to the translations/ +To make use of the translations that users have committed to the translations/ subversion module, you'll need to ensure that you have a current checked out -copy of Torbutton: +copy of them in your torbutton git checkout: - cd torbutton/trans_tools - torbutton/trans_tools$ svn up + cd torbutton.git/trans_tools + torbutton.git/trans_tools$ svn co https://tor-svn.freehaven.net/svn/translation/trunk/projects/torbutton pootle You should see something like the following: - Fetching external item into 'pootle' - External at revision 15300. - - At revision 15300. - -Now if you had changes, you need to convert from .po and move -the newly updated mozilla files into the current stable locale -directory. First convert them with the 'mkmoz.sh' script and then -move the proper mozilla files from 'torbutton/trans_tools/moz/' into -'torbutton/src/chrome/locale/' directory while properly naming the files -for their respective locale. - -Here's an example of how to move all of the current pootle translations into -the svn trunk area of Torbutton: - - cd torbutton/trans_tools - ./mkmoz.sh - for locale in `ls -1 moz/`; - do - mv -v moz/$locale/*.{dtd,properties} ../src/chrome/locale/$locale/; - done - -Now check the differences (ensure the output looks reasonable): + Checked out revision 21092. - svn diff +If you made changes to strings in Torbutton, you need to rebuild the +templates in torbutton.git/trans_tools/pootle/templates. This is done with +the following command from within the torbutton.git checkout directory: -And finally check in the changes: - - svn commit - - -If you make changes to strings in Torbutton, you need to rebuild the -templates in torbutton/trans_tools/pootle/templates. This is done via: - - moz2po -P -i torbutton/src/chrome/locale/en/ -o torbutton/trans_tools/templates/ + moz2po -P -i src/chrome/locale/en/ -o trans_tools/pootle/templates/ You now have two options: -Option 1 (The Pootle Web UI Way): +Option 1 (The [shitty] Pootle Web UI Way): View then commit the changes to the template with: - svn diff torbutton/trans_tools/templates/ - svn commit torbutton/trans_tools/templates/ + cd trans_tools/pootle + svn diff templates + svn commit templates Then poke Jake to 'svn up' on the Pootle side. If you do this enough times, he may give you a button to click to update templates in Pootle, @@ -150,7 +123,7 @@ Option 2 (Use your own msgmerge: YMMV, may change .po flags and formatting): Run msgmerge yourself for each language: - cd torbutton/trans_tools + cd trans_tools for i in `ls -1 pootle` do msgmerge -U ./pootle/$i/torbutton.dtd.po ./pootle/templates/torbutton.dtd.pot @@ -171,6 +144,36 @@ breaks :) After this process is done, you then need to regenerate the mozilla .dtd and .properties files as specified above. + +Regardless of whether or not you had changes in the torbutton strings, if there +were updated strings in pootle that you checked out from svn you now need to +convert from .po and move the newly updated mozilla files into the current +stable locale directory. First convert them with the 'mkmoz.sh' script and +then move the proper mozilla files from 'torbutton.git/trans_tools/moz/' into +'torbutton.git/src/chrome/locale/' directory while properly naming the files +for their respective locale. + +Here's an example of how to move all of the current pootle translations into +the svn trunk area of Torbutton: + + cd trans_tools + ./mkmoz.sh + for locale in `ls -1 moz/`; + do + mv -v moz/$locale/*.{dtd,properties} ../src/chrome/locale/$locale/ + done + +Now check the differences to your git branch to ensure the output looks +reasonable: + + cd .. + git diff + +And finally check in the changes: + + cd src/chrome/locale + git commit . + ---------------------------- Vidalia ------------------------------- Vidalia uses our translation portal to accept translations. Users use the diff --git a/src/Makefile.am b/src/Makefile.am index ae647b2d6..fa2dd560a 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1,5 +1,5 @@ # leave in dependency order, since common must be built first -SUBDIRS = common or tools win32 config -DIST_SUBDIRS = common or tools win32 config +SUBDIRS = common or test tools win32 config +DIST_SUBDIRS = common or test tools win32 config diff --git a/src/common/Makefile.am b/src/common/Makefile.am index 105c41334..b1e03cd71 100644 --- a/src/common/Makefile.am +++ b/src/common/Makefile.am @@ -1,5 +1,7 @@ -noinst_LIBRARIES = libor.a libor-crypto.a +noinst_LIBRARIES = libor.a libor-crypto.a libor-event.a + +EXTRA_DIST = common_sha1.i sha256.c #CFLAGS = -Wall -Wpointer-arith -O2 @@ -10,7 +12,21 @@ libor_extra_source= endif libor_a_SOURCES = address.c log.c util.c compat.c container.c mempool.c \ - memarea.c $(libor_extra_source) + memarea.c util_codedigest.c $(libor_extra_source) libor_crypto_a_SOURCES = crypto.c aes.c tortls.c torgzip.c +libor_event_a_SOURCES = compat_libevent.c + +noinst_HEADERS = address.h torlog.h crypto.h util.h compat.h aes.h torint.h tortls.h strlcpy.c strlcat.c torgzip.h container.h ht.h mempool.h memarea.h ciphers.inc compat_libevent.h tortls_states.h + +common_sha1.i: $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) + if test "@SHA1SUM@" != none; then \ + @SHA1SUM@ $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) | @SED@ -n 's/^\(.*\)$$/"\1\\n"/p' > common_sha1.i; \ + elif test "@OPENSSL@" != none; then \ + @OPENSSL@ sha1 $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) | @SED@ -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > common_sha1.i; \ + else \ + rm common_sha1.i; \ + touch common_sha1.i; \ + fi -noinst_HEADERS = address.h log.h crypto.h test.h util.h compat.h aes.h torint.h tortls.h strlcpy.c strlcat.c torgzip.h container.h ht.h mempool.h memarea.h ciphers.inc +util_codedigest.o: common_sha1.i +crypto.o: sha256.c diff --git a/src/common/address.c b/src/common/address.c index 82a709601..aff517ca5 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -12,7 +12,7 @@ #include "compat.h" #include "util.h" #include "address.h" -#include "log.h" +#include "torlog.h" #ifdef MS_WINDOWS #include <process.h> @@ -50,11 +50,13 @@ #include <assert.h> /** Convert the tor_addr_t in <b>a</b>, with port in <b>port</b>, into a - * socklen object in *<b>sa_out</b> of object size <b>len</b>. If not enough - * room is free, or on error, return -1. Else return the length of the - * sockaddr. */ -/* XXXX021 This returns socklen_t. socklen_t is sometimes unsigned. This - * function claims to return -1 sometimes. Problematic! */ + * sockaddr object in *<b>sa_out</b> of object size <b>len</b>. If not enough + * room is available in sa_out, or on error, return 0. On success, return + * the length of the sockaddr. + * + * Interface note: ordinarily, we return -1 for error. We can't do that here, + * since socklen_t is unsigned on some platforms. + **/ socklen_t tor_addr_to_sockaddr(const tor_addr_t *a, uint16_t port, @@ -65,7 +67,7 @@ tor_addr_to_sockaddr(const tor_addr_t *a, if (family == AF_INET) { struct sockaddr_in *sin; if (len < (int)sizeof(struct sockaddr_in)) - return -1; + return 0; sin = (struct sockaddr_in *)sa_out; memset(sin, 0, sizeof(struct sockaddr_in)); #ifdef HAVE_STRUCT_SOCKADDR_IN_SIN_LEN @@ -78,7 +80,7 @@ tor_addr_to_sockaddr(const tor_addr_t *a, } else if (family == AF_INET6) { struct sockaddr_in6 *sin6; if (len < (int)sizeof(struct sockaddr_in6)) - return -1; + return 0; sin6 = (struct sockaddr_in6 *)sa_out; memset(sin6, 0, sizeof(struct sockaddr_in6)); #ifdef HAVE_STRUCT_SOCKADDR_IN6_SIN6_LEN @@ -89,7 +91,7 @@ tor_addr_to_sockaddr(const tor_addr_t *a, memcpy(&sin6->sin6_addr, tor_addr_to_in6(a), sizeof(struct in6_addr)); return sizeof(struct sockaddr_in6); } else { - return -1; + return 0; } } @@ -508,7 +510,8 @@ tor_addr_parse_mask_ports(const char *s, tor_addr_t *addr_out, tor_assert(s); tor_assert(addr_out); - /* IP, [], /mask, ports */ + /** Longest possible length for an address, mask, and port-range combination. + * Includes IP, [], /mask, :, ports */ #define MAX_ADDRESS_LENGTH (TOR_ADDR_BUF_LEN+2+(1+INET_NTOA_BUF_LEN)+12+1) if (strlen(s) > MAX_ADDRESS_LENGTH) { @@ -601,7 +604,7 @@ tor_addr_parse_mask_ports(const char *s, tor_addr_t *addr_out, if (family == AF_INET6 && v4map) { if (bits > 32 && bits < 96) { /* Crazy */ log_warn(LD_GENERAL, - "Bad mask bits %i for V4-mapped V6 address; rejecting.", + "Bad mask bits %d for V4-mapped V6 address; rejecting.", bits); goto err; } @@ -726,7 +729,7 @@ tor_addr_is_loopback(const tor_addr_t *addr) } /** Set <b>dest</b> to equal the IPv4 address in <b>v4addr</b> (given in - * network order. */ + * network order). */ void tor_addr_from_ipv4n(tor_addr_t *dest, uint32_t v4addr) { @@ -760,6 +763,8 @@ tor_addr_from_in6(tor_addr_t *dest, const struct in6_addr *in6) void tor_addr_copy(tor_addr_t *dest, const tor_addr_t *src) { + if (src == dest) + return; tor_assert(src); tor_assert(dest); memcpy(dest, src, sizeof(tor_addr_t)); @@ -786,7 +791,7 @@ tor_addr_compare(const tor_addr_t *addr1, const tor_addr_t *addr2, * Reduce over-specific masks (>128 for ipv6, >32 for ipv4) to 128 or 32. * * The mask is interpreted relative to <b>addr1</b>, so that if a is - * ::ffff:1.2.3.4, and b is 3.4.5.6, + * \::ffff:1.2.3.4, and b is 3.4.5.6, * tor_addr_compare_masked(a,b,100,CMP_SEMANTIC) is the same as * -tor_addr_compare_masked(b,a,4,CMP_SEMANTIC). * @@ -798,6 +803,8 @@ int tor_addr_compare_masked(const tor_addr_t *addr1, const tor_addr_t *addr2, maskbits_t mbits, tor_addr_comparison_t how) { + /** Helper: Evaluates to -1 if a is less than b, 0 if a equals b, or 1 if a + * is greater than b. May evaluate a and b more than once. */ #define TRISTATE(a,b) (((a)<(b))?-1: (((a)==(b))?0:1)) sa_family_t family1, family2, v_family1, v_family2; @@ -912,13 +919,6 @@ tor_dup_addr(const tor_addr_t *addr) return tor_strdup(buf); } -/** Copy the address in <b>src</b> to <b>dest</b> */ -void -tor_addr_assign(tor_addr_t *dest, const tor_addr_t *src) -{ - memcpy(dest, src, sizeof(tor_addr_t)); -} - /** Return a string representing the address <b>addr</b>. This string is * statically allocated, and must not be freed. Each call to * <b>fmt_addr</b> invalidates the last result of the function. This @@ -1030,27 +1030,28 @@ get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr) { int sock=-1, r=-1; struct sockaddr_storage my_addr, target_addr; - socklen_t my_addr_len; + socklen_t addr_len; tor_assert(addr); memset(addr, 0, sizeof(tor_addr_t)); memset(&target_addr, 0, sizeof(target_addr)); - my_addr_len = (socklen_t)sizeof(my_addr); - /* Use the "discard" service port */ - ((struct sockaddr_in*)&target_addr)->sin_port = 9; /* Don't worry: no packets are sent. We just need to use a real address * on the actual Internet. */ if (family == AF_INET6) { struct sockaddr_in6 *sin6 = (struct sockaddr_in6*)&target_addr; + /* Use the "discard" service port */ + sin6->sin6_port = htons(9); sock = tor_open_socket(PF_INET6,SOCK_DGRAM,IPPROTO_UDP); - my_addr_len = (socklen_t)sizeof(struct sockaddr_in6); + addr_len = (socklen_t)sizeof(struct sockaddr_in6); sin6->sin6_family = AF_INET6; S6_ADDR16(sin6->sin6_addr)[0] = htons(0x2002); /* 2002:: */ } else if (family == AF_INET) { struct sockaddr_in *sin = (struct sockaddr_in*)&target_addr; + /* Use the "discard" service port */ + sin->sin_port = htons(9); sock = tor_open_socket(PF_INET,SOCK_DGRAM,IPPROTO_UDP); - my_addr_len = (socklen_t)sizeof(struct sockaddr_in); + addr_len = (socklen_t)sizeof(struct sockaddr_in); sin->sin_family = AF_INET; sin->sin_addr.s_addr = htonl(0x12000001); /* 18.0.0.1 */ } else { @@ -1063,14 +1064,13 @@ get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr) goto err; } - if (connect(sock,(struct sockaddr *)&target_addr, - (socklen_t)sizeof(target_addr))<0) { + if (connect(sock,(struct sockaddr *)&target_addr, addr_len) < 0) { int e = tor_socket_errno(sock); log_fn(severity, LD_NET, "connect() failed: %s", tor_socket_strerror(e)); goto err; } - if (getsockname(sock,(struct sockaddr*)&my_addr, &my_addr_len)) { + if (getsockname(sock,(struct sockaddr*)&my_addr, &addr_len)) { int e = tor_socket_errno(sock); log_fn(severity, LD_NET, "getsockname() to determine interface failed: %s", tor_socket_strerror(e)); @@ -1087,7 +1087,7 @@ get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr) /* ====== * IPv4 helpers - * XXXX022 IPv6 deprecate some of these. + * XXXX023 IPv6 deprecate some of these. */ /** Return true iff <b>ip</b> (in host order) is an IP reserved to localhost, diff --git a/src/common/address.h b/src/common/address.h index cb2438791..d05a3de2e 100644 --- a/src/common/address.h +++ b/src/common/address.h @@ -53,8 +53,20 @@ tor_addr_to_in6(const tor_addr_t *a) return a->family == AF_INET6 ? &a->addr.in6_addr : NULL; } +/** Given an IPv6 address <b>x</b>, yield it as an array of uint8_t. + * + * Requires that <b>x</b> is actually an IPv6 address. + */ #define tor_addr_to_in6_addr8(x) tor_addr_to_in6(x)->s6_addr +/** Given an IPv6 address <b>x</b>, yield it as an array of uint16_t. + * + * Requires that <b>x</b> is actually an IPv6 address. + */ #define tor_addr_to_in6_addr16(x) S6_ADDR16(*tor_addr_to_in6(x)) +/** Given an IPv6 address <b>x</b>, yield it as an array of uint32_t. + * + * Requires that <b>x</b> is actually an IPv6 address. + */ #define tor_addr_to_in6_addr32(x) S6_ADDR32(*tor_addr_to_in6(x)) /** Return an IPv4 address in network order for <b>a</b>, or 0 if @@ -71,7 +83,7 @@ tor_addr_to_ipv4h(const tor_addr_t *a) { return ntohl(tor_addr_to_ipv4n(a)); } -/* Given an IPv6 address, return its mapped IPv4 address in host order, or +/** Given an IPv6 address, return its mapped IPv4 address in host order, or * 0 if <b>a</b> is not an IPv6 address. * * (Does not check whether the address is really a mapped address */ @@ -102,12 +114,17 @@ tor_addr_eq_ipv4h(const tor_addr_t *a, uint32_t u) return a->family == AF_INET ? (tor_addr_to_ipv4h(a) == u) : 0; } -#define TOR_ADDR_BUF_LEN 48 /* [ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255] - */ +/** Length of a buffer that you need to allocate to be sure you can encode + * any tor_addr_t. + * + * This allows enough space for + * "[ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255]", + * plus a terminating NUL. + */ +#define TOR_ADDR_BUF_LEN 48 int tor_addr_lookup(const char *name, uint16_t family, tor_addr_t *addr_out); char *tor_dup_addr(const tor_addr_t *addr) ATTR_MALLOC; -void tor_addr_assign(tor_addr_t *dest, const tor_addr_t *src); const char *fmt_addr(const tor_addr_t *addr); int get_interface_address6(int severity, sa_family_t family, tor_addr_t *addr); @@ -155,6 +172,7 @@ void tor_addr_from_ipv4n(tor_addr_t *dest, uint32_t v4addr); #define tor_addr_from_ipv4h(dest, v4addr) \ tor_addr_from_ipv4n((dest), htonl(v4addr)) void tor_addr_from_ipv6_bytes(tor_addr_t *dest, const char *bytes); +/** Set <b>dest</b> to the IPv4 address incoded in <b>in</b>. */ #define tor_addr_from_in(dest, in) \ tor_addr_from_ipv4n((dest), (in)->s_addr); void tor_addr_from_in6(tor_addr_t *dest, const struct in6_addr *in6); diff --git a/src/common/aes.c b/src/common/aes.c index 7e332b050..c2fdeb594 100644 --- a/src/common/aes.c +++ b/src/common/aes.c @@ -21,7 +21,7 @@ #include "compat.h" #include "aes.h" #include "util.h" -#include "log.h" +#include "torlog.h" /* We have 3 strategies for getting AES: Via OpenSSL's AES_encrypt function, * via OpenSSL's EVP_EncryptUpdate function, or via the built-in AES @@ -100,6 +100,9 @@ /* Figure out which AES optimizations to use. */ #ifdef USE_BUILTIN_AES +/** If this is defined, we take advantage of the fact that AES treats its + * input as a set of 4 32-bit words, so that there is no need to encode and + * decode the 128-bit counter before every block encryption */ # define USE_RIJNDAEL_COUNTER_OPTIMIZATION # if 0 && (defined(__powerpc__) || defined(__powerpc64__)) /* XXXX do more experimentation before concluding this is actually @@ -263,7 +266,8 @@ aes_set_key(aes_cnt_cipher_t *cipher, const char *key, int key_bits) void aes_free_cipher(aes_cnt_cipher_t *cipher) { - tor_assert(cipher); + if (!cipher) + return; #ifdef USE_OPENSSL_EVP EVP_CIPHER_CTX_cleanup(&cipher->key); #endif diff --git a/src/common/compat.c b/src/common/compat.c index 333ef5b0f..5797374c4 100644 --- a/src/common/compat.c +++ b/src/common/compat.c @@ -15,6 +15,8 @@ /* This is required on rh7 to make strptime not complain. * We also need it to make memmem get defined (where available) */ +/* XXXX023 We should just use AC_USE_SYSTEM_EXTENSIONS in our autoconf, + * and get this (and other important stuff!) automatically */ #define _GNU_SOURCE #include "compat.h" @@ -88,7 +90,7 @@ #include <sys/prctl.h> #endif -#include "log.h" +#include "torlog.h" #include "util.h" #include "container.h" #include "address.h" @@ -170,12 +172,17 @@ tor_munmap_file(tor_mmap_t *handle) tor_mmap_t * tor_mmap_file(const char *filename) { + TCHAR tfilename[MAX_PATH]= {0}; tor_mmap_t *res = tor_malloc_zero(sizeof(tor_mmap_t)); int empty = 0; res->file_handle = INVALID_HANDLE_VALUE; res->mmap_handle = NULL; - - res->file_handle = CreateFile(filename, +#ifdef UNICODE + mbstowcs(tfilename,filename,MAX_PATH); +#else + strlcpy(tfilename,filename,MAX_PATH); +#endif + res->file_handle = CreateFile(tfilename, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, @@ -308,6 +315,100 @@ tor_vsnprintf(char *str, size_t size, const char *format, va_list args) return r; } +/** + * Portable asprintf implementation. Does a printf() into a newly malloc'd + * string. Sets *<b>strp</b> to this string, and returns its length (not + * including the terminating NUL character). + * + * You can treat this function as if its implementation were something like + <pre> + char buf[_INFINITY_]; + tor_snprintf(buf, sizeof(buf), fmt, args); + *strp = tor_strdup(buf); + return strlen(*strp): + </pre> + * Where _INFINITY_ is an imaginary constant so big that any string can fit + * into it. + */ +int +tor_asprintf(char **strp, const char *fmt, ...) +{ + int r; + va_list args; + va_start(args, fmt); + r = tor_vasprintf(strp, fmt, args); + va_end(args); + if (!*strp || r < 0) { + log_err(LD_BUG, "Internal error in asprintf"); + tor_assert(0); + } + return r; +} + +/** + * Portable vasprintf implementation. Does a printf() into a newly malloc'd + * string. Differs from regular vasprintf in the same ways that + * tor_asprintf() differs from regular asprintf. + */ +int +tor_vasprintf(char **strp, const char *fmt, va_list args) +{ + /* use a temporary variable in case *strp is in args. */ + char *strp_tmp=NULL; +#ifdef HAVE_VASPRINTF + /* If the platform gives us one, use it. */ + int r = vasprintf(&strp_tmp, fmt, args); + if (r < 0) + *strp = NULL; + else + *strp = strp_tmp; + return r; +#elif defined(_MSC_VER) + /* On Windows, _vsnprintf won't tell us the length of the string if it + * overflows, so we need to use _vcsprintf to tell how much to allocate */ + int len, r; + char *res; + len = _vscprintf(fmt, args); + if (len < 0) { + *strp = NULL; + return -1; + } + strp_tmp = tor_malloc(len + 1); + r = _vsnprintf(strp_tmp, len+1, fmt, args); + if (r != len) { + tor_free(strp_tmp); + *strp = NULL; + return -1; + } + *strp = strp_tmp; + return len; +#else + /* Everywhere else, we have a decent vsnprintf that tells us how many + * characters we need. We give it a try on a short buffer first, since + * it might be nice to avoid the second vsnprintf call. + */ + char buf[128]; + int len, r; + va_list tmp_args; + va_copy(tmp_args, args); + len = vsnprintf(buf, sizeof(buf), fmt, tmp_args); + va_end(tmp_args); + if (len < (int)sizeof(buf)) { + *strp = tor_strdup(buf); + return len; + } + strp_tmp = tor_malloc(len+1); + r = vsnprintf(strp_tmp, len+1, fmt, args); + if (r != len) { + tor_free(strp_tmp); + *strp = NULL; + return -1; + } + *strp = strp_tmp; + return len; +#endif +} + /** Given <b>hlen</b> bytes at <b>haystack</b> and <b>nlen</b> bytes at * <b>needle</b>, return a pointer to the first occurrence of the needle * within the haystack, or NULL if there is no such occurrence. @@ -399,6 +500,37 @@ const char TOR_TOLOWER_TABLE[256] = { 240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255, }; +/** Implementation of strtok_r for platforms whose coders haven't figured out + * how to write one. Hey guys! You can use this code here for free! */ +char * +tor_strtok_r_impl(char *str, const char *sep, char **lasts) +{ + char *cp, *start; + if (str) + start = cp = *lasts = str; + else if (!*lasts) + return NULL; + else + start = cp = *lasts; + + tor_assert(*sep); + if (sep[1]) { + while (*cp && !strchr(sep, *cp)) + ++cp; + } else { + tor_assert(strlen(sep) == 1); + cp = strchr(cp, *sep); + } + + if (!cp || !*cp) { + *lasts = NULL; + } else { + *cp++ = '\0'; + *lasts = cp; + } + return start; +} + #ifdef MS_WINDOWS /** Take a filename and return a pointer to its final element. This * function is called on __FILE__ to fix a MSVC nit where __FILE__ @@ -450,8 +582,8 @@ get_uint32(const void *cp) return v; } /** - * Read a 32-bit value beginning at <b>cp</b>. Equivalent to - * *(uint32_t*)(cp), but will not cause segfaults on platforms that forbid + * Read a 64-bit value beginning at <b>cp</b>. Equivalent to + * *(uint64_t*)(cp), but will not cause segfaults on platforms that forbid * unaligned memory access. */ uint64_t @@ -530,7 +662,9 @@ touch_file(const char *fname) /** Represents a lockfile on which we hold the lock. */ struct tor_lockfile_t { + /** Name of the file */ char *filename; + /** File descriptor used to hold the file open */ int fd; }; @@ -546,7 +680,10 @@ struct tor_lockfile_t { * * (Implementation note: because we need to fall back to fcntl on some * platforms, these locks are per-process, not per-thread. If you want - * to do in-process locking, use tor_mutex_t like a normal person.) + * to do in-process locking, use tor_mutex_t like a normal person. + * On Windows, when <b>blocking</b> is true, the maximum time that + * is actually waited is 10 seconds, after which NULL is returned + * and <b>locked_out</b> is set to 1.) */ tor_lockfile_t * tor_lockfile_lock(const char *filename, int blocking, int *locked_out) @@ -566,7 +703,7 @@ tor_lockfile_lock(const char *filename, int blocking, int *locked_out) #ifdef WIN32 _lseek(fd, 0, SEEK_SET); if (_locking(fd, blocking ? _LK_LOCK : _LK_NBLCK, 1) < 0) { - if (errno != EDEADLOCK) + if (errno != EACCES && errno != EDEADLOCK) log_warn(LD_FS,"Couldn't lock \"%s\": %s", filename, strerror(errno)); else *locked_out = 1; @@ -633,7 +770,8 @@ tor_lockfile_unlock(tor_lockfile_t *lockfile) tor_free(lockfile); } -/* Some old versions of Unix didn't define constants for these values, +/** @{ */ +/** Some old versions of Unix didn't define constants for these values, * and instead expect you to say 0, 1, or 2. */ #ifndef SEEK_CUR #define SEEK_CUR 1 @@ -641,6 +779,7 @@ tor_lockfile_unlock(tor_lockfile_t *lockfile) #ifndef SEEK_END #define SEEK_END 2 #endif +/** @} */ /** Return the position of <b>fd</b> with respect to the start of the file. */ off_t @@ -680,6 +819,7 @@ static int n_sockets_open = 0; /** Mutex to protect open_sockets, max_socket, and n_sockets_open. */ static tor_mutex_t *socket_accounting_mutex = NULL; +/** Helper: acquire the socket accounting lock. */ static INLINE void socket_accounting_lock(void) { @@ -688,6 +828,7 @@ socket_accounting_lock(void) tor_mutex_acquire(socket_accounting_mutex); } +/** Helper: release the socket accounting lock. */ static INLINE void socket_accounting_unlock(void) { @@ -746,6 +887,7 @@ tor_close_socket(int s) return r; } +/** @{ */ #ifdef DEBUG_SOCKET_COUNTING /** Helper: if DEBUG_SOCKET_COUNTING is enabled, remember that <b>s</b> is * now an open socket. */ @@ -770,6 +912,7 @@ mark_socket_open(int s) #else #define mark_socket_open(s) STMT_NIL #endif +/** @} */ /** As socket(), but counts the number of open sockets. */ int @@ -957,6 +1100,8 @@ tor_socketpair(int family, int type, int protocol, int fd[2]) #endif } +/** Number of extra file descriptors to keep in reserve beyond those that we + * tell Tor it's allowed to use. */ #define ULIMIT_BUFFER 32 /* keep 32 extra fd's beyond _ConnLimit */ /** Learn the maximum allowed number of file descriptors. (Some systems @@ -979,9 +1124,6 @@ set_max_file_descriptors(rlim_t limit, int *max_out) #if defined(CYGWIN) || defined(__CYGWIN__) const char *platform = "Cygwin"; const unsigned long MAX_CONNECTIONS = 3200; -#elif defined(IPHONE) - const char *platform = "iPhone"; - const unsigned long MAX_CONNECTIONS = 9999; #elif defined(MS_WINDOWS) const char *platform = "Windows"; const unsigned long MAX_CONNECTIONS = 15000; @@ -1072,6 +1214,7 @@ set_max_file_descriptors(rlim_t limit, int *max_out) static int log_credential_status(void) { +/** Log level to use when describing non-error UID/GID status. */ #define CREDENTIAL_LOG_LEVEL LOG_INFO /* Real, effective and saved UIDs */ uid_t ruid, euid, suid; @@ -1553,6 +1696,30 @@ tor_lookup_hostname(const char *name, uint32_t *addr) return -1; } +/** Initialize the insecure libc RNG. */ +void +tor_init_weak_random(unsigned seed) +{ +#ifdef MS_WINDOWS + srand(seed); +#else + srandom(seed); +#endif +} + +/** Return a randomly chosen value in the range 0..TOR_RAND_MAX. This + * entropy will not be cryptographically strong; do not rely on it + * for anything an adversary should not be able to predict. */ +long +tor_weak_random(void) +{ +#ifdef MS_WINDOWS + return rand(); +#else + return random(); +#endif +} + /** Hold the result of our call to <b>uname</b>. */ static char uname_result[256]; /** True iff uname_result is set. */ @@ -1578,13 +1745,14 @@ get_uname(void) #ifdef MS_WINDOWS OSVERSIONINFOEX info; int i; - unsigned int leftover_mask; const char *plat = NULL; const char *extra = NULL; + char acsd[MAX_PATH] = {0}; static struct { unsigned major; unsigned minor; const char *version; } win_version_table[] = { - { 6, 0, "Windows \"Longhorn\"" }, + { 6, 1, "Windows 7" }, + { 6, 0, "Windows Vista" }, { 5, 2, "Windows Server 2003" }, { 5, 1, "Windows XP" }, { 5, 0, "Windows 2000" }, @@ -1595,25 +1763,6 @@ get_uname(void) { 3, 51, "Windows NT 3.51" }, { 0, 0, NULL } }; -#ifdef VER_SUITE_BACKOFFICE - static struct { - unsigned int mask; const char *str; - } win_mask_table[] = { - { VER_SUITE_BACKOFFICE, " {backoffice}" }, - { VER_SUITE_BLADE, " {\"blade\" (2003, web edition)}" }, - { VER_SUITE_DATACENTER, " {datacenter}" }, - { VER_SUITE_ENTERPRISE, " {enterprise}" }, - { VER_SUITE_EMBEDDEDNT, " {embedded}" }, - { VER_SUITE_PERSONAL, " {personal}" }, - { VER_SUITE_SINGLEUSERTS, - " {terminal services, single user}" }, - { VER_SUITE_SMALLBUSINESS, " {small business}" }, - { VER_SUITE_SMALLBUSINESS_RESTRICTED, - " {small business, restricted}" }, - { VER_SUITE_TERMINAL, " {terminal services}" }, - { 0, NULL }, - }; -#endif memset(&info, 0, sizeof(info)); info.dwOSVersionInfoSize = sizeof(info); if (! GetVersionEx((LPOSVERSIONINFO)&info)) { @@ -1622,14 +1771,19 @@ get_uname(void) uname_result_is_set = 1; return uname_result; } +#ifdef UNICODE + wcstombs(acsd, info.szCSDVersion, MAX_PATH); +#else + strlcpy(acsd, info.szCSDVersion, sizeof(acsd)); +#endif if (info.dwMajorVersion == 4 && info.dwMinorVersion == 0) { if (info.dwPlatformId == VER_PLATFORM_WIN32_NT) plat = "Windows NT 4.0"; else plat = "Windows 95"; - if (info.szCSDVersion[1] == 'B') + if (acsd[1] == 'B') extra = "OSR2 (B)"; - else if (info.szCSDVersion[1] == 'C') + else if (acsd[1] == 'C') extra = "OSR2 (C)"; } else { for (i=0; win_version_table[i].major>0; ++i) { @@ -1641,29 +1795,30 @@ get_uname(void) } } if (plat && !strcmp(plat, "Windows 98")) { - if (info.szCSDVersion[1] == 'A') + if (acsd[1] == 'A') extra = "SE (A)"; - else if (info.szCSDVersion[1] == 'B') + else if (acsd[1] == 'B') extra = "SE (B)"; } if (plat) { if (!extra) - extra = info.szCSDVersion; + extra = acsd; tor_snprintf(uname_result, sizeof(uname_result), "%s %s", plat, extra); } else { if (info.dwMajorVersion > 6 || - (info.dwMajorVersion==6 && info.dwMinorVersion>0)) + (info.dwMajorVersion==6 && info.dwMinorVersion>1)) tor_snprintf(uname_result, sizeof(uname_result), "Very recent version of Windows [major=%d,minor=%d] %s", (int)info.dwMajorVersion,(int)info.dwMinorVersion, - info.szCSDVersion); + acsd); else tor_snprintf(uname_result, sizeof(uname_result), "Unrecognized version of Windows [major=%d,minor=%d] %s", (int)info.dwMajorVersion,(int)info.dwMinorVersion, - info.szCSDVersion); + acsd); } +#if !defined (WINCE) #ifdef VER_SUITE_BACKOFFICE if (info.wProductType == VER_NT_DOMAIN_CONTROLLER) { strlcat(uname_result, " [domain controller]", sizeof(uname_result)); @@ -1672,18 +1827,7 @@ get_uname(void) } else if (info.wProductType == VER_NT_WORKSTATION) { strlcat(uname_result, " [workstation]", sizeof(uname_result)); } - leftover_mask = info.wSuiteMask; - for (i = 0; win_mask_table[i].mask; ++i) { - if (info.wSuiteMask & win_mask_table[i].mask) { - strlcat(uname_result, win_mask_table[i].str, sizeof(uname_result)); - leftover_mask &= ~win_mask_table[i].mask; - } - } - if (leftover_mask) { - size_t len = strlen(uname_result); - tor_snprintf(uname_result+len, sizeof(uname_result)-len, - " {0x%x}", info.wSuiteMask); - } +#endif #endif #else strlcpy(uname_result, "Unknown platform", sizeof(uname_result)); @@ -1793,7 +1937,6 @@ spawn_exit(void) * call _exit, not exit, from child processes. */ _exit(0); #endif - } /** Set *timeval to the current time of day. On error, log and terminate. @@ -1813,8 +1956,15 @@ tor_gettimeofday(struct timeval *timeval) uint64_t ft_64; FILETIME ft_ft; } ft; +#if defined (WINCE) + /* wince do not have GetSystemTimeAsFileTime */ + SYSTEMTIME stime; + GetSystemTime(&stime); + SystemTimeToFileTime(&stime,&ft.ft_ft); +#else /* number of 100-nsec units since Jan 1, 1601 */ GetSystemTimeAsFileTime(&ft.ft_ft); +#endif if (ft.ft_64 < EPOCH_BIAS) { log_err(LD_GENERAL,"System time is before 1970; failing."); exit(1); @@ -1846,8 +1996,87 @@ tor_gettimeofday(struct timeval *timeval) #define TIME_FNS_NEED_LOCKS #endif -#ifndef HAVE_LOCALTIME_R -#ifdef TIME_FNS_NEED_LOCKS +static struct tm * +correct_tm(int islocal, const time_t *timep, struct tm *resultbuf, + struct tm *r) +{ + const char *outcome; + + if (PREDICT_LIKELY(r)) { + if (r->tm_year > 8099) { /* We can't strftime dates after 9999 CE. */ + r->tm_year = 8099; + r->tm_mon = 11; + r->tm_mday = 31; + r->tm_yday = 365; + r->tm_hour = 23; + r->tm_min = 59; + r->tm_sec = 59; + } + return r; + } + + /* If we get here, gmtime or localtime returned NULL. It might have done + * this because of overrun or underrun, or it might have done it because of + * some other weird issue. */ + if (timep) { + if (*timep < 0) { + r = resultbuf; + r->tm_year = 70; /* 1970 CE */ + r->tm_mon = 0; + r->tm_mday = 1; + r->tm_yday = 1; + r->tm_hour = 0; + r->tm_min = 0 ; + r->tm_sec = 0; + outcome = "Rounding up to 1970"; + goto done; + } else if (*timep >= INT32_MAX) { + /* Rounding down to INT32_MAX isn't so great, but keep in mind that we + * only do it if gmtime/localtime tells us NULL. */ + r = resultbuf; + r->tm_year = 137; /* 2037 CE */ + r->tm_mon = 11; + r->tm_mday = 31; + r->tm_yday = 365; + r->tm_hour = 23; + r->tm_min = 59; + r->tm_sec = 59; + outcome = "Rounding down to 2037"; + goto done; + } + } + + /* If we get here, then gmtime/localtime failed without getting an extreme + * value for *timep */ + + tor_fragile_assert(); + r = resultbuf; + memset(resultbuf, 0, sizeof(struct tm)); + outcome="can't recover"; + done: + log_warn(LD_BUG, "%s("I64_FORMAT") failed with error %s: %s", + islocal?"localtime":"gmtime", + timep?I64_PRINTF_ARG(*timep):0, + strerror(errno), + outcome); + return r; +} + +/** @{ */ +/** As localtime_r, but defined for platforms that don't have it: + * + * Convert *<b>timep</b> to a struct tm in local time, and store the value in + * *<b>result</b>. Return the result on success, or NULL on failure. + */ +#ifdef HAVE_LOCALTIME_R +struct tm * +tor_localtime_r(const time_t *timep, struct tm *result) +{ + struct tm *r; + r = localtime_r(timep, result); + return correct_tm(1, timep, result, r); +} +#elif defined(TIME_FNS_NEED_LOCKS) struct tm * tor_localtime_r(const time_t *timep, struct tm *result) { @@ -1857,9 +2086,10 @@ tor_localtime_r(const time_t *timep, struct tm *result) tor_assert(result); tor_mutex_acquire(m); r = localtime(timep); - memcpy(result, r, sizeof(struct tm)); + if (r) + memcpy(result, r, sizeof(struct tm)); tor_mutex_release(m); - return result; + return correct_tm(1, timep, result, r); } #else struct tm * @@ -1868,14 +2098,28 @@ tor_localtime_r(const time_t *timep, struct tm *result) struct tm *r; tor_assert(result); r = localtime(timep); - memcpy(result, r, sizeof(struct tm)); - return result; + if (r) + memcpy(result, r, sizeof(struct tm)); + return correct_tm(1, timep, result, r); } #endif -#endif +/** @} */ -#ifndef HAVE_GMTIME_R -#ifdef TIME_FNS_NEED_LOCKS +/** @{ */ +/** As gmtimee_r, but defined for platforms that don't have it: + * + * Convert *<b>timep</b> to a struct tm in UTC, and store the value in + * *<b>result</b>. Return the result on success, or NULL on failure. + */ +#ifdef HAVE_GMTIME_R +struct tm * +tor_gmtime_r(const time_t *timep, struct tm *result) +{ + struct tm *r; + r = gmtime_r(timep, result); + return correct_tm(0, timep, result, r); +} +#elif defined(TIME_FNS_NEED_LOCKS) struct tm * tor_gmtime_r(const time_t *timep, struct tm *result) { @@ -1885,9 +2129,10 @@ tor_gmtime_r(const time_t *timep, struct tm *result) tor_assert(result); tor_mutex_acquire(m); r = gmtime(timep); - memcpy(result, r, sizeof(struct tm)); + if (r) + memcpy(result, r, sizeof(struct tm)); tor_mutex_release(m); - return result; + return correct_tm(0, timep, result, r); } #else struct tm * @@ -1896,11 +2141,11 @@ tor_gmtime_r(const time_t *timep, struct tm *result) struct tm *r; tor_assert(result); r = gmtime(timep); - memcpy(result, r, sizeof(struct tm)); - return result; + if (r) + memcpy(result, r, sizeof(struct tm)); + return correct_tm(0, timep, result, r); } #endif -#endif #if defined(USE_WIN32_THREADS) void @@ -2014,6 +2259,8 @@ tor_mutex_new(void) void tor_mutex_free(tor_mutex_t *m) { + if (!m) + return; tor_mutex_uninit(m); tor_free(m); } @@ -2041,7 +2288,8 @@ tor_cond_new(void) void tor_cond_free(tor_cond_t *cond) { - tor_assert(cond); + if (!cond) + return; if (pthread_cond_destroy(&cond->cond)) { log_warn(LD_GENERAL,"Error freeing condition: %s", strerror(errno)); return; @@ -2098,7 +2346,8 @@ tor_cond_new(void) void tor_cond_free(tor_cond_t *cond) { - tor_assert(cond); + if (!cond) + return; DeleteCriticalSection(&cond->mutex); /* XXXX notify? */ smartlist_free(cond->events); @@ -2174,6 +2423,89 @@ tor_threads_init(void) } #endif +#if defined(HAVE_MLOCKALL) && HAVE_DECL_MLOCKALL && defined(RLIMIT_MEMLOCK) +/** Attempt to raise the current and max rlimit to infinity for our process. + * This only needs to be done once and can probably only be done when we have + * not already dropped privileges. + */ +static int +tor_set_max_memlock(void) +{ + /* Future consideration for Windows is probably SetProcessWorkingSetSize + * This is similar to setting the memory rlimit of RLIMIT_MEMLOCK + * http://msdn.microsoft.com/en-us/library/ms686234(VS.85).aspx + */ + + struct rlimit limit; + + /* RLIM_INFINITY is -1 on some platforms. */ + limit.rlim_cur = RLIM_INFINITY; + limit.rlim_max = RLIM_INFINITY; + + if (setrlimit(RLIMIT_MEMLOCK, &limit) == -1) { + if (errno == EPERM) { + log_warn(LD_GENERAL, "You appear to lack permissions to change memory " + "limits. Are you root?"); + } + log_warn(LD_GENERAL, "Unable to raise RLIMIT_MEMLOCK: %s", + strerror(errno)); + return -1; + } + + return 0; +} +#endif + +/** Attempt to lock all current and all future memory pages. + * This should only be called once and while we're privileged. + * Like mlockall() we return 0 when we're successful and -1 when we're not. + * Unlike mlockall() we return 1 if we've already attempted to lock memory. + */ +int +tor_mlockall(void) +{ + static int memory_lock_attempted = 0; + + if (memory_lock_attempted) { + return 1; + } + + memory_lock_attempted = 1; + + /* + * Future consideration for Windows may be VirtualLock + * VirtualLock appears to implement mlock() but not mlockall() + * + * http://msdn.microsoft.com/en-us/library/aa366895(VS.85).aspx + */ + +#if defined(HAVE_MLOCKALL) && HAVE_DECL_MLOCKALL && defined(RLIMIT_MEMLOCK) + if (tor_set_max_memlock() == 0) { + log_debug(LD_GENERAL, "RLIMIT_MEMLOCK is now set to RLIM_INFINITY."); + } + + if (mlockall(MCL_CURRENT|MCL_FUTURE) == 0) { + log_info(LD_GENERAL, "Insecure OS paging is effectively disabled."); + return 0; + } else { + if (errno == ENOSYS) { + /* Apple - it's 2009! I'm looking at you. Grrr. */ + log_notice(LD_GENERAL, "It appears that mlockall() is not available on " + "your platform."); + } else if (errno == EPERM) { + log_notice(LD_GENERAL, "It appears that you lack the permissions to " + "lock memory. Are you root?"); + } + log_notice(LD_GENERAL, "Unable to lock all current and future memory " + "pages: %s", strerror(errno)); + return -1; + } +#else + log_warn(LD_GENERAL, "Unable to lock memory pages. mlockall() unsupported?"); + return -1; +#endif +} + /** Identity of the "main" thread */ static unsigned long main_thread_id = -1; @@ -2325,20 +2657,26 @@ network_init(void) char * format_win32_error(DWORD err) { - LPVOID str = NULL; + TCHAR *str = NULL; char *result; /* Somebody once decided that this interface was better than strerror(). */ - FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | + FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), - (LPTSTR) &str, + (LPVOID)&str, 0, NULL); if (str) { - result = tor_strdup((char*)str); +#ifdef UNICODE + char abuf[1024] = {0}; + wcstombs(abuf,str,1024); + result = tor_strdup(abuf); +#else + result = tor_strdup(str); +#endif LocalFree(str); /* LocalFree != free() */ } else { result = tor_strdup("<unformattable error>"); diff --git a/src/common/compat.h b/src/common/compat.h index c7ed32b0d..af795ffba 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -51,6 +51,22 @@ #include <netinet6/in6.h> #endif +#if defined (WINCE) +#include <fcntl.h> +#include <io.h> +#include <math.h> +#include <projects.h> +#define snprintf _snprintf +/* this is not exported as W .... */ +#define SHGetPathFromIDListW SHGetPathFromIDList +/* wcecompat has vasprintf */ +#define HAVE_VASPRINTF +/* no service here */ +#ifdef NT_SERVICE +#undef NT_SERVICE +#endif +#endif // WINCE + #ifndef NULL_REP_IS_ZERO_BYTES #error "It seems your platform does not represent NULL as zero. We can't cope." #endif @@ -177,8 +193,8 @@ extern INLINE double U64_TO_DBL(uint64_t x) { /* ===== String compatibility */ #ifdef MS_WINDOWS /* Windows names string functions differently from most other platforms. */ -#define strncasecmp strnicmp -#define strcasecmp stricmp +#define strncasecmp _strnicmp +#define strcasecmp _stricmp #endif #ifndef HAVE_STRLCAT size_t strlcat(char *dst, const char *src, size_t siz) ATTR_NONNULL((1,2)); @@ -196,18 +212,26 @@ size_t strlcpy(char *dst, const char *src, size_t siz) ATTR_NONNULL((1,2)); #define U64_SCANF_ARG(a) (a) /** Expands to a literal uint64_t-typed constant for the value <b>n</b>. */ #define U64_LITERAL(n) (n ## ui64) +#define I64_PRINTF_ARG(a) (a) +#define I64_SCANF_ARG(a) (a) +#define I64_LITERAL(n) (n ## i64) #else #define U64_PRINTF_ARG(a) ((long long unsigned int)(a)) #define U64_SCANF_ARG(a) ((long long unsigned int*)(a)) #define U64_LITERAL(n) (n ## llu) +#define I64_PRINTF_ARG(a) ((long long signed int)(a)) +#define I64_SCANF_ARG(a) ((long long signed int*)(a)) +#define I64_LITERAL(n) (n ## ll) #endif #if defined(_MSC_VER) || defined(__MINGW32__) || defined(__MINGW64__) /** The formatting string used to put a uint64_t value in a printf() or * scanf() function. See also U64_PRINTF_ARG and U64_SCANF_ARG. */ #define U64_FORMAT "%I64u" +#define I64_FORMAT "%I64d" #else #define U64_FORMAT "%llu" +#define I64_FORMAT "%lld" #endif /** Represents an mmaped file. Allocated via tor_mmap_file; freed with @@ -235,6 +259,10 @@ int tor_snprintf(char *str, size_t size, const char *format, ...) int tor_vsnprintf(char *str, size_t size, const char *format, va_list args) ATTR_NONNULL((1,3)); +int tor_asprintf(char **strp, const char *fmt, ...) + CHECK_PRINTF(2,3); +int tor_vasprintf(char **strp, const char *fmt, va_list args); + const void *tor_memmem(const void *haystack, size_t hlen, const void *needle, size_t nlen) ATTR_PURE ATTR_NONNULL((1,3)); static const void *tor_memstr(const void *haystack, size_t hlen, @@ -267,6 +295,13 @@ extern const char TOR_TOLOWER_TABLE[]; #define TOR_TOLOWER(c) (TOR_TOLOWER_TABLE[(uint8_t)c]) #define TOR_TOUPPER(c) (TOR_TOUPPER_TABLE[(uint8_t)c]) +char *tor_strtok_r_impl(char *str, const char *sep, char **lasts); +#ifdef HAVE_STRTOK_R +#define tor_strtok_r(str, sep, lasts) strtok_r(str, sep, lasts) +#else +#define tor_strtok_r(str, sep, lasts) tor_strtok_r_impl(str, sep, lasts) +#endif + #ifdef MS_WINDOWS #define _SHORT_FILE_ (tor_fix_source_file(__FILE__)) const char *tor_fix_source_file(const char *fname); @@ -277,6 +312,7 @@ const char *tor_fix_source_file(const char *fname); /* ===== Time compatibility */ #if !defined(HAVE_GETTIMEOFDAY) && !defined(HAVE_STRUCT_TIMEVAL_TV_SEC) +/** Implementation of timeval for platforms that don't have it. */ struct timeval { time_t tv_sec; unsigned int tv_usec; @@ -285,16 +321,49 @@ struct timeval { void tor_gettimeofday(struct timeval *timeval); -#ifdef HAVE_LOCALTIME_R -#define tor_localtime_r localtime_r -#else struct tm *tor_localtime_r(const time_t *timep, struct tm *result); -#endif - -#ifdef HAVE_GMTIME_R -#define tor_gmtime_r gmtime_r -#else struct tm *tor_gmtime_r(const time_t *timep, struct tm *result); + +#ifndef timeradd +/** Replacement for timeradd on platforms that do not have it: sets tvout to + * the sum of tv1 and tv2. */ +#define timeradd(tv1,tv2,tvout) \ + do { \ + (tvout)->tv_sec = (tv1)->tv_sec + (tv2)->tv_sec; \ + (tvout)->tv_usec = (tv2)->tv_usec + (tv2)->tv_usec; \ + if ((tvout)->tv_usec >= 1000000) { \ + (tvout)->tv_usec -= 1000000; \ + (tvout)->tv_sec++; \ + } \ + } while (0) +#endif + +#ifndef timersub +/** Replacement for timersub on platforms that do not have it: sets tvout to + * tv1 minus tv2. */ +#define timersub(tv1,tv2,tvout) \ + do { \ + (tvout)->tv_sec = (tv1)->tv_sec - (tv2)->tv_sec; \ + (tvout)->tv_usec = (tv2)->tv_usec - (tv2)->tv_usec; \ + if ((tvout)->tv_usec < 0) { \ + (tvout)->tv_usec += 1000000; \ + (tvout)->tv_sec--; \ + } \ + } while (0) +#endif + +#ifndef timercmp +/** Replacement for timersub on platforms that do not have it: returns true + * iff the relational operator "op" makes the expression tv1 op tv2 true. + * + * Note that while this definition should work for all boolean opeators, some + * platforms' native timercmp definitions do not support >=, <=, or ==. So + * don't use those. + */ +#define timercmp(tv1,tv2,op) \ + (((tv1)->tv_sec == (tv2)->tv_sec) ? \ + ((tv1)->tv_usec op (tv2)->tv_usec) : \ + ((tv1)->tv_sec op (tv2)->tv_sec)) #endif /* ===== File compatibility */ @@ -329,9 +398,9 @@ int get_n_open_sockets(void); #define tor_socket_send(s, buf, len, flags) send(s, buf, len, flags) #define tor_socket_recv(s, buf, len, flags) recv(s, buf, len, flags) -/* Define struct in6_addr on platforms that do not have it. Generally, - * these platforms are ones without IPv6 support, but we want to have - * a working in6_addr there anyway, so we can use it to parse IPv6 +/** Implementatino of struct in6_addr for platforms that do not have it. + * Generally, these platforms are ones without IPv6 support, but we want to + * have a working in6_addr there anyway, so we can use it to parse IPv6 * addresses. */ #if !defined(HAVE_STRUCT_IN6_ADDR) struct in6_addr @@ -347,9 +416,10 @@ struct in6_addr }; #endif +/** @{ */ +/** Many BSD variants seem not to define these. */ #if defined(__APPLE__) || defined(__darwin__) || defined(__FreeBSD__) \ || defined(__NetBSD__) || defined(__OpenBSD__) -/* Many BSD variants seem not to define these. */ #ifndef s6_addr16 #define s6_addr16 __u6_addr.__u6_addr16 #endif @@ -357,12 +427,15 @@ struct in6_addr #define s6_addr32 __u6_addr.__u6_addr32 #endif #endif +/** @} */ #ifndef HAVE_SA_FAMILY_T typedef uint16_t sa_family_t; #endif -/* Apparently, MS and Solaris don't define s6_addr16 or s6_addr32. */ +/** @{ */ +/** Apparently, MS and Solaris don't define s6_addr16 or s6_addr32; these + * macros get you a pointer to s6_addr32 or local equivalent. */ #ifdef HAVE_STRUCT_IN6_ADDR_S6_ADDR32 #define S6_ADDR32(x) ((uint32_t*)(x).s6_addr32) #else @@ -373,9 +446,10 @@ typedef uint16_t sa_family_t; #else #define S6_ADDR16(x) ((uint16_t*)((char*)&(x).s6_addr)) #endif +/** @} */ -/* Define struct sockaddr_in6 on platforms that do not have it. See notes - * on struct in6_addr. */ +/** Implementation of struct sockaddr_in6 on platforms that do not have + * it. See notes on struct in6_addr. */ #if !defined(HAVE_STRUCT_SOCKADDR_IN6) struct sockaddr_in6 { sa_family_t sin6_family; @@ -445,6 +519,11 @@ typedef enum { SOCKS5_ADDRESS_TYPE_NOT_SUPPORTED = 0x08, } socks5_reply_status_t; +/* ===== Insecure rng */ +void tor_init_weak_random(unsigned seed); +long tor_weak_random(void); +#define TOR_RAND_MAX (RAND_MAX) + /* ===== OS compatibility */ const char *get_uname(void); @@ -494,14 +573,20 @@ void spawn_exit(void) ATTR_NORETURN; /** A generic lock structure for multithreaded builds. */ typedef struct tor_mutex_t { #if defined(USE_WIN32_THREADS) + /** Windows-only: on windows, we implement locks with CRITICAL_SECTIONS. */ CRITICAL_SECTION mutex; #elif defined(USE_PTHREADS) + /** Pthreads-only: with pthreads, we implement locks with + * pthread_mutex_t. */ pthread_mutex_t mutex; #else + /** No-threads only: Dummy variable so that tor_mutex_t takes up space. */ int _unused; #endif } tor_mutex_t; +int tor_mlockall(void); + #ifdef TOR_IS_MULTITHREADED tor_mutex_t *tor_mutex_new(void); void tor_mutex_init(tor_mutex_t *m); @@ -536,6 +621,19 @@ void tor_cond_signal_all(tor_cond_t *cond); #endif #endif +/** Macros for MIN/MAX. Never use these when the arguments could have + * side-effects. + * {With GCC extensions we could probably define a safer MIN/MAX. But + * depending on that safety would be dangerous, since not every platform + * has it.} + **/ +#ifndef MAX +#define MAX(a,b) ( ((a)<(b)) ? (b) : (a) ) +#endif +#ifndef MIN +#define MIN(a,b) ( ((a)>(b)) ? (b) : (a) ) +#endif + /* Platform-specific helpers. */ #ifdef MS_WINDOWS char *format_win32_error(DWORD err); diff --git a/src/common/compat_libevent.c b/src/common/compat_libevent.c new file mode 100644 index 000000000..3ad9be145 --- /dev/null +++ b/src/common/compat_libevent.c @@ -0,0 +1,554 @@ +/* Copyright (c) 2009-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file compat_libevent.c + * \brief Wrappers to handle porting between different versions of libevent. + * + * In an ideal world, we'd just use Libevent 2.0 from now on. But as of June + * 2009, Libevent 2.0 is still in alpha, and we will have old versions of + * Libevent for the forseeable future. + **/ + +#include "orconfig.h" +#include "compat.h" +#include "compat_libevent.h" + +#include "util.h" +#include "torlog.h" + +#ifdef HAVE_EVENT2_EVENT_H +#include <event2/event.h> +#else +#include <event.h> +#endif + +/** A number representing a version of Libevent. + + This is a 4-byte number, with the first three bytes representing the + major, minor, and patchlevel respectively of the library. The fourth + byte is unused. + + This is equivalent to the format of LIBEVENT_VERSION_NUMBER on Libevent + 2.0.1 or later. For versions of Libevent before 1.4.0, which followed the + format of "1.0, 1.0a, 1.0b", we define 1.0 to be equivalent to 1.0.0, 1.0a + to be equivalent to 1.0.1, and so on. +*/ +typedef uint32_t le_version_t; + +/** @{ */ +/** Macros: returns the number of a libevent version as a le_version_t */ +#define V(major, minor, patch) \ + (((major) << 24) | ((minor) << 16) | ((patch) << 8)) +#define V_OLD(major, minor, patch) \ + V((major), (minor), (patch)-'a'+1) +/** @} */ + +/** Represetns a version of libevent so old we can't figure out what version + * it is. */ +#define LE_OLD V(0,0,0) +/** Represents a version of libevent so weird we can't figure out what version + * it it. */ +#define LE_OTHER V(0,0,99) + +static le_version_t tor_get_libevent_version(const char **v_out); + +#ifdef HAVE_EVENT_SET_LOG_CALLBACK +/** A string which, if it appears in a libevent log, should be ignored. */ +static const char *suppress_msg = NULL; +/** Callback function passed to event_set_log() so we can intercept + * log messages from libevent. */ +static void +libevent_logging_callback(int severity, const char *msg) +{ + char buf[1024]; + size_t n; + if (suppress_msg && strstr(msg, suppress_msg)) + return; + n = strlcpy(buf, msg, sizeof(buf)); + if (n && n < sizeof(buf) && buf[n-1] == '\n') { + buf[n-1] = '\0'; + } + switch (severity) { + case _EVENT_LOG_DEBUG: + log(LOG_DEBUG, LD_NOCB|LD_NET, "Message from libevent: %s", buf); + break; + case _EVENT_LOG_MSG: + log(LOG_INFO, LD_NOCB|LD_NET, "Message from libevent: %s", buf); + break; + case _EVENT_LOG_WARN: + log(LOG_WARN, LD_NOCB|LD_GENERAL, "Warning from libevent: %s", buf); + break; + case _EVENT_LOG_ERR: + log(LOG_ERR, LD_NOCB|LD_GENERAL, "Error from libevent: %s", buf); + break; + default: + log(LOG_WARN, LD_NOCB|LD_GENERAL, "Message [%d] from libevent: %s", + severity, buf); + break; + } +} +/** Set hook to intercept log messages from libevent. */ +void +configure_libevent_logging(void) +{ + event_set_log_callback(libevent_logging_callback); +} +/** Ignore any libevent log message that contains <b>msg</b>. */ +void +suppress_libevent_log_msg(const char *msg) +{ + suppress_msg = msg; +} +#else +void +configure_libevent_logging(void) +{ +} +void +suppress_libevent_log_msg(const char *msg) +{ + (void)msg; +} +#endif + +#ifndef HAVE_EVENT2_EVENT_H +/** Work-alike replacement for event_new() on pre-Libevent-2.0 systems. */ +struct event * +tor_event_new(struct event_base *base, int sock, short what, + void (*cb)(int, short, void *), void *arg) +{ + struct event *e = tor_malloc_zero(sizeof(struct event)); + event_set(e, sock, what, cb, arg); + if (! base) + base = tor_libevent_get_base(); + event_base_set(base, e); + return e; +} +/** Work-alike replacement for evtimer_new() on pre-Libevent-2.0 systems. */ +struct event * +tor_evtimer_new(struct event_base *base, + void (*cb)(int, short, void *), void *arg) +{ + return tor_event_new(base, -1, 0, cb, arg); +} +/** Work-alike replacement for evsignal_new() on pre-Libevent-2.0 systems. */ +struct event * +tor_evsignal_new(struct event_base * base, int sig, + void (*cb)(int, short, void *), void *arg) +{ + return tor_event_new(base, sig, EV_SIGNAL|EV_PERSIST, cb, arg); +} +/** Work-alike replacement for event_free() on pre-Libevent-2.0 systems. */ +void +tor_event_free(struct event *ev) +{ + event_del(ev); + tor_free(ev); +} +#endif + +/** Global event base for use by the main thread. */ +struct event_base *the_event_base = NULL; + +/* This is what passes for version detection on OSX. We set + * MACOSX_KQUEUE_IS_BROKEN to true iff we're on a version of OSX before + * 10.4.0 (aka 1040). */ +#ifdef __APPLE__ +#ifdef __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ +#define MACOSX_KQUEUE_IS_BROKEN \ + (__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 1040) +#else +#define MACOSX_KQUEUE_IS_BROKEN 0 +#endif +#endif + +/** Initialize the Libevent library and set up the event base. */ +void +tor_libevent_initialize(void) +{ + tor_assert(the_event_base == NULL); + +#ifdef __APPLE__ + if (MACOSX_KQUEUE_IS_BROKEN || + tor_get_libevent_version(NULL) < V_OLD(1,1,'b')) { + setenv("EVENT_NOKQUEUE","1",1); + } +#endif + +#ifdef HAVE_EVENT2_EVENT_H + the_event_base = event_base_new(); +#else + the_event_base = event_init(); +#endif + +#if defined(HAVE_EVENT_GET_VERSION) && defined(HAVE_EVENT_GET_METHOD) + /* Making this a NOTICE for now so we can link bugs to a libevent versions + * or methods better. */ + log(LOG_NOTICE, LD_GENERAL, + "Initialized libevent version %s using method %s. Good.", + event_get_version(), tor_libevent_get_method()); +#else + log(LOG_NOTICE, LD_GENERAL, + "Initialized old libevent (version 1.0b or earlier)."); + log(LOG_WARN, LD_GENERAL, + "You have a *VERY* old version of libevent. It is likely to be buggy; " + "please build Tor with a more recent version."); +#endif +} + +/** Return the current Libevent event base that we're set up to use. */ +struct event_base * +tor_libevent_get_base(void) +{ + return the_event_base; +} + +#ifndef HAVE_EVENT_BASE_LOOPEXIT +/** Replacement for event_base_loopexit on some very old versions of Libevent + * that we are not yet brave enough to deprecate. */ +int +tor_event_base_loopexit(struct event_base *base, struct timeval *tv) +{ + tor_assert(base == the_event_base); + return event_loopexit(tv); +} +#endif + +/** Return the name of the Libevent backend we're using. */ +const char * +tor_libevent_get_method(void) +{ +#ifdef HAVE_EVENT2_EVENT_H + return event_base_get_method(the_event_base); +#elif defined(HAVE_EVENT_GET_METHOD) + return event_get_method(); +#else + return "<unknown>"; +#endif +} + +/** Return the le_version_t for the version of libevent specified in the + * string <b>v</b>. If the version is very new or uses an unrecognized + * version, format, return LE_OTHER. */ +static le_version_t +tor_decode_libevent_version(const char *v) +{ + unsigned major, minor, patchlevel; + char c, e, extra; + int fields; + + /* Try the new preferred "1.4.11-stable" format. + * Also accept "1.4.14b-stable". */ + fields = sscanf(v, "%u.%u.%u%c%c", &major, &minor, &patchlevel, &c, &e); + if (fields == 3 || + ((fields == 4 || fields == 5 ) && (c == '-' || c == '_')) || + (fields == 5 && TOR_ISALPHA(c) && (e == '-' || e == '_'))) { + return V(major,minor,patchlevel); + } + + /* Try the old "1.3e" format. */ + fields = sscanf(v, "%u.%u%c%c", &major, &minor, &c, &extra); + if (fields == 3 && TOR_ISALPHA(c)) { + return V_OLD(major, minor, c); + } else if (fields == 2) { + return V(major, minor, 0); + } + + return LE_OTHER; +} + +/** Return an integer representing the binary interface of a Libevent library. + * Two different versions with different numbers are sure not to be binary + * compatible. Two different versions with the same numbers have a decent + * chance of binary compatibility.*/ +static int +le_versions_compatibility(le_version_t v) +{ + if (v == LE_OTHER) + return 0; + if (v < V_OLD(1,0,'c')) + return 1; + else if (v < V(1,4,0)) + return 2; + else if (v < V(1,4,99)) + return 3; + else if (v < V(2,0,1)) + return 4; + else /* Everything 2.0 and later should be compatible. */ + return 5; +} + +/** Return the version number of the currently running version of Libevent. + * See le_version_t for info on the format. + */ +static le_version_t +tor_get_libevent_version(const char **v_out) +{ + const char *v; + le_version_t r; +#if defined(HAVE_EVENT_GET_VERSION_NUMBER) + v = event_get_version(); + r = event_get_version_number(); +#elif defined (HAVE_EVENT_GET_VERSION) + v = event_get_version(); + r = tor_decode_libevent_version(v); +#else + v = "pre-1.0c"; + r = LE_OLD; +#endif + if (v_out) + *v_out = v; + return r; +} + +/** Return a string representation of the version of the currently running + * version of Libevent. */ +const char * +tor_libevent_get_version_str(void) +{ +#ifdef HAVE_EVENT_GET_VERSION + return event_get_version(); +#else + return "pre-1.0c"; +#endif +} + +/** + * Compare the current Libevent method and version to a list of versions + * which are known not to work. Warn the user as appropriate. + */ +void +tor_check_libevent_version(const char *m, int server, + const char **badness_out) +{ + int buggy = 0, iffy = 0, slow = 0, thread_unsafe = 0; + le_version_t version; + const char *v = NULL; + const char *badness = NULL; + const char *sad_os = ""; + + version = tor_get_libevent_version(&v); + + /* It would be better to disable known-buggy methods rather than warning + * about them. But the problem is that with older versions of Libevent, + * it's not trivial to get them to change their methods once they're + * initialized... and with newer versions of Libevent, they aren't actually + * broken. But we should revisit this if we ever find a post-1.4 version + * of Libevent where we need to disable a given method. */ + if (!strcmp(m, "kqueue")) { + if (version < V_OLD(1,1,'b')) + buggy = 1; + } else if (!strcmp(m, "epoll")) { + if (version < V(1,1,0)) + iffy = 1; + } else if (!strcmp(m, "poll")) { + if (version < V_OLD(1,0,'e')) + buggy = 1; + if (version < V(1,1,0)) + slow = 1; + } else if (!strcmp(m, "select")) { + if (version < V(1,1,0)) + slow = 1; + } else if (!strcmp(m, "win32")) { + if (version < V_OLD(1,1,'b')) + buggy = 1; + } + + /* Libevent versions before 1.3b do very badly on operating systems with + * user-space threading implementations. */ +#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) + if (server && version < V_OLD(1,3,'b')) { + thread_unsafe = 1; + sad_os = "BSD variants"; + } +#elif defined(__APPLE__) || defined(__darwin__) + if (server && version < V_OLD(1,3,'b')) { + thread_unsafe = 1; + sad_os = "Mac OS X"; + } +#endif + + if (thread_unsafe) { + log(LOG_WARN, LD_GENERAL, + "Libevent version %s often crashes when running a Tor server with %s. " + "Please use the latest version of libevent (1.3b or later)",v,sad_os); + badness = "BROKEN"; + } else if (buggy) { + log(LOG_WARN, LD_GENERAL, + "There are serious bugs in using %s with libevent %s. " + "Please use the latest version of libevent.", m, v); + badness = "BROKEN"; + } else if (iffy) { + log(LOG_WARN, LD_GENERAL, + "There are minor bugs in using %s with libevent %s. " + "You may want to use the latest version of libevent.", m, v); + badness = "BUGGY"; + } else if (slow && server) { + log(LOG_WARN, LD_GENERAL, + "libevent %s can be very slow with %s. " + "When running a server, please use the latest version of libevent.", + v,m); + badness = "SLOW"; + } + + *badness_out = badness; +} + +#if defined(LIBEVENT_VERSION) +#define HEADER_VERSION LIBEVENT_VERSION +#elif defined(_EVENT_VERSION) +#define HEADER_VERSION _EVENT_VERSION +#endif + +/** See whether the headers we were built against differ from the library we + * linked against so much that we're likely to crash. If so, warn the + * user. */ +void +tor_check_libevent_header_compatibility(void) +{ + (void) le_versions_compatibility; + (void) tor_decode_libevent_version; + + /* In libevent versions before 2.0, it's hard to keep binary compatibility + * between upgrades, and unpleasant to detect when the version we compiled + * against is unlike the version we have linked against. Here's how. */ +#if defined(HEADER_VERSION) && defined(HAVE_EVENT_GET_VERSION) + /* We have a header-file version and a function-call version. Easy. */ + if (strcmp(HEADER_VERSION, event_get_version())) { + le_version_t v1, v2; + int compat1 = -1, compat2 = -1; + int verybad; + v1 = tor_decode_libevent_version(HEADER_VERSION); + v2 = tor_decode_libevent_version(event_get_version()); + compat1 = le_versions_compatibility(v1); + compat2 = le_versions_compatibility(v2); + + verybad = compat1 != compat2; + + log(verybad ? LOG_WARN : LOG_NOTICE, + LD_GENERAL, "We were compiled with headers from version %s " + "of Libevent, but we're using a Libevent library that says it's " + "version %s.", HEADER_VERSION, event_get_version()); + if (verybad) + log_warn(LD_GENERAL, "This will almost certainly make Tor crash."); + else + log_info(LD_GENERAL, "I think these versions are binary-compatible."); + } +#elif defined(HAVE_EVENT_GET_VERSION) + /* event_get_version but no _EVENT_VERSION. We might be in 1.4.0-beta or + earlier, where that's normal. To see whether we were compiled with an + earlier version, let's see whether the struct event defines MIN_HEAP_IDX. + */ +#ifdef HAVE_STRUCT_EVENT_MIN_HEAP_IDX + /* The header files are 1.4.0-beta or later. If the version is not + * 1.4.0-beta, we are incompatible. */ + { + if (strcmp(event_get_version(), "1.4.0-beta")) { + log_warn(LD_GENERAL, "It's a little hard to tell, but you seem to have " + "Libevent 1.4.0-beta header files, whereas you have linked " + "against Libevent %s. This will probably make Tor crash.", + event_get_version()); + } + } +#else + /* Our headers are 1.3e or earlier. If the library version is not 1.4.x or + later, we're probably fine. */ + { + const char *v = event_get_version(); + if ((v[0] == '1' && v[2] == '.' && v[3] > '3') || v[0] > '1') { + log_warn(LD_GENERAL, "It's a little hard to tell, but you seem to have " + "Libevent header file from 1.3e or earlier, whereas you have " + "linked against Libevent %s. This will probably make Tor " + "crash.", event_get_version()); + } + } +#endif + +#elif defined(HEADER_VERSION) +#warn "_EVENT_VERSION is defined but not get_event_version(): Libevent is odd." +#else + /* Your libevent is ancient. */ +#endif +} + +/* + If possible, we're going to try to use Libevent's periodic timer support, + since it does a pretty good job of making sure that periodic events get + called exactly M seconds apart, rather than starting each one exactly M + seconds after the time that the last one was run. + */ +#ifdef HAVE_EVENT2_EVENT_H +#define HAVE_PERIODIC +#define PERIODIC_FLAGS EV_PERSIST +#else +#define PERIODIC_FLAGS 0 +#endif + +/** Represents a timer that's run every N microseconds by Libevent. */ +struct periodic_timer_t { + /** Underlying event used to implement this periodic event. */ + struct event *ev; + /** The callback we'll be invoking whenever the event triggers */ + void (*cb)(struct periodic_timer_t *, void *); + /** User-supplied data for the callback */ + void *data; +#ifndef HAVE_PERIODIC + /** If Libevent doesn't know how to invoke events every N microseconds, + * we'll need to remember the timeout interval here. */ + struct timeval tv; +#endif +}; + +/** Libevent callback to implement a periodic event. */ +static void +periodic_timer_cb(evutil_socket_t fd, short what, void *arg) +{ + periodic_timer_t *timer = arg; + (void) what; + (void) fd; +#ifndef HAVE_PERIODIC + /** reschedule the event as needed. */ + event_add(timer->ev, &timer->tv); +#endif + timer->cb(timer, timer->data); +} + +/** Create and schedule a new timer that will run every <b>tv</b> in + * the event loop of <b>base</b>. When the timer fires, it will + * run the timer in <b>cb</b> with the user-supplied data in <b>data</b>. */ +periodic_timer_t * +periodic_timer_new(struct event_base *base, + const struct timeval *tv, + void (*cb)(periodic_timer_t *timer, void *data), + void *data) +{ + periodic_timer_t *timer; + tor_assert(base); + tor_assert(tv); + tor_assert(cb); + timer = tor_malloc_zero(sizeof(periodic_timer_t)); + if (!(timer->ev = tor_event_new(base, -1, PERIODIC_FLAGS, + periodic_timer_cb, timer))) { + tor_free(timer); + return NULL; + } + timer->cb = cb; + timer->data = data; +#ifndef HAVE_PERIODIC + memcpy(&timer->tv, tv, sizeof(struct timeval)); +#endif + event_add(timer->ev, (struct timeval *)tv); /*drop const for old libevent*/ + return timer; +} + +/** Stop and free a periodic timer */ +void +periodic_timer_free(periodic_timer_t *timer) +{ + if (!timer) + return; + tor_event_free(timer->ev); + tor_free(timer); +} + diff --git a/src/common/compat_libevent.h b/src/common/compat_libevent.h new file mode 100644 index 000000000..fdf5e0a18 --- /dev/null +++ b/src/common/compat_libevent.h @@ -0,0 +1,65 @@ +/* Copyright (c) 2009, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef _TOR_COMPAT_LIBEVENT_H +#define _TOR_COMPAT_LIBEVENT_H + +#include "orconfig.h" + +struct event; +struct event_base; + +#ifdef HAVE_EVENT2_EVENT_H +#include <event2/util.h> +#else +#define evutil_socket_t int +#endif + +void configure_libevent_logging(void); +void suppress_libevent_log_msg(const char *msg); + +#ifdef HAVE_EVENT2_EVENT_H +#define tor_event_new event_new +#define tor_evtimer_new evtimer_new +#define tor_evsignal_new evsignal_new +#define tor_event_free event_free +#define tor_evdns_add_server_port(sock, tcp, cb, data) \ + evdns_add_server_port_with_base(tor_libevent_get_base(), \ + (sock),(tcp),(cb),(data)); + +#else +struct event *tor_event_new(struct event_base * base, evutil_socket_t sock, + short what, void (*cb)(evutil_socket_t, short, void *), void *arg); +struct event *tor_evtimer_new(struct event_base * base, + void (*cb)(evutil_socket_t, short, void *), void *arg); +struct event *tor_evsignal_new(struct event_base * base, int sig, + void (*cb)(evutil_socket_t, short, void *), void *arg); +void tor_event_free(struct event *ev); +#define tor_evdns_add_server_port evdns_add_server_port +#endif + +typedef struct periodic_timer_t periodic_timer_t; + +periodic_timer_t *periodic_timer_new(struct event_base *base, + const struct timeval *tv, + void (*cb)(periodic_timer_t *timer, void *data), + void *data); +void periodic_timer_free(periodic_timer_t *); + +#ifdef HAVE_EVENT_BASE_LOOPEXIT +#define tor_event_base_loopexit event_base_loopexit +#else +struct timeval; +int tor_event_base_loopexit(struct event_base *base, struct timeval *tv); +#endif + +void tor_libevent_initialize(void); +struct event_base *tor_libevent_get_base(void); +const char *tor_libevent_get_method(void); +void tor_check_libevent_version(const char *m, int server, + const char **badness_out); +void tor_check_libevent_header_compatibility(void); +const char *tor_libevent_get_version_str(void); + +#endif + diff --git a/src/common/container.c b/src/common/container.c index 977d60407..7208d3680 100644 --- a/src/common/container.c +++ b/src/common/container.c @@ -13,7 +13,7 @@ #include "compat.h" #include "util.h" -#include "log.h" +#include "torlog.h" #include "container.h" #include "crypto.h" @@ -44,7 +44,8 @@ smartlist_create(void) void smartlist_free(smartlist_t *sl) { - tor_assert(sl != NULL); + if (!sl) + return; tor_free(sl->list); tor_free(sl); } @@ -459,6 +460,42 @@ smartlist_sort(smartlist_t *sl, int (*compare)(const void **a, const void **b)) (int (*)(const void *,const void*))compare); } +/** Given a smartlist <b>sl</b> sorted with the function <b>compare</b>, + * return the most frequent member in the list. Break ties in favor of + * later elements. If the list is empty, return NULL. + */ +void * +smartlist_get_most_frequent(const smartlist_t *sl, + int (*compare)(const void **a, const void **b)) +{ + const void *most_frequent = NULL; + int most_frequent_count = 0; + + const void *cur = NULL; + int i, count=0; + + if (!sl->num_used) + return NULL; + for (i = 0; i < sl->num_used; ++i) { + const void *item = sl->list[i]; + if (cur && 0 == compare(&cur, &item)) { + ++count; + } else { + if (cur && count >= most_frequent_count) { + most_frequent = cur; + most_frequent_count = count; + } + cur = item; + count = 1; + } + } + if (cur && count >= most_frequent_count) { + most_frequent = cur; + most_frequent_count = count; + } + return (void*)most_frequent; +} + /** Given a sorted smartlist <b>sl</b> and the comparison function used to * sort it, remove all duplicate members. If free_fn is provided, calls * free_fn on each duplicate. Otherwise, just removes them. Preserves order. @@ -550,6 +587,13 @@ smartlist_sort_strings(smartlist_t *sl) smartlist_sort(sl, _compare_string_ptrs); } +/** Return the most frequent string in the sorted list <b>sl</b> */ +char * +smartlist_get_most_frequent_string(smartlist_t *sl) +{ + return smartlist_get_most_frequent(sl, _compare_string_ptrs); +} + /** Remove duplicate strings from a sorted list, and free them with tor_free(). */ void @@ -561,16 +605,70 @@ smartlist_uniq_strings(smartlist_t *sl) /* Heap-based priority queue implementation for O(lg N) insert and remove. * Recall that the heap property is that, for every index I, h[I] < * H[LEFT_CHILD[I]] and h[I] < H[RIGHT_CHILD[I]]. + * + * For us to remove items other than the topmost item, each item must store + * its own index within the heap. When calling the pqueue functions, tell + * them about the offset of the field that stores the index within the item. + * + * Example: + * + * typedef struct timer_t { + * struct timeval tv; + * int heap_index; + * } timer_t; + * + * static int compare(const void *p1, const void *p2) { + * const timer_t *t1 = p1, *t2 = p2; + * if (t1->tv.tv_sec < t2->tv.tv_sec) { + * return -1; + * } else if (t1->tv.tv_sec > t2->tv.tv_sec) { + * return 1; + * } else { + * return t1->tv.tv_usec - t2->tv_usec; + * } + * } + * + * void timer_heap_insert(smartlist_t *heap, timer_t *timer) { + * smartlist_pqueue_add(heap, compare, STRUCT_OFFSET(timer_t, heap_index), + * timer); + * } + * + * void timer_heap_pop(smartlist_t *heap) { + * return smartlist_pqueue_pop(heap, compare, + * STRUCT_OFFSET(timer_t, heap_index)); + * } */ -/* For a 1-indexed array, we would use LEFT_CHILD[x] = 2*x and RIGHT_CHILD[x] - * = 2*x + 1. But this is C, so we have to adjust a little. */ +/** @{ */ +/** Functions to manipulate heap indices to find a node's parent and children. + * + * For a 1-indexed array, we would use LEFT_CHILD[x] = 2*x and RIGHT_CHILD[x] + * = 2*x + 1. But this is C, so we have to adjust a little. */ //#define LEFT_CHILD(i) ( ((i)+1)*2 - 1) //#define RIGHT_CHILD(i) ( ((i)+1)*2 ) //#define PARENT(i) ( ((i)+1)/2 - 1) #define LEFT_CHILD(i) ( 2*(i) + 1 ) #define RIGHT_CHILD(i) ( 2*(i) + 2 ) #define PARENT(i) ( ((i)-1) / 2 ) +/** }@ */ + +/** @{ */ +/** Helper macros for heaps: Given a local variable <b>idx_field_offset</b> + * set to the offset of an integer index within the heap element structure, + * IDX_OF_ITEM(p) gives you the index of p, and IDXP(p) gives you a pointer to + * where p's index is stored. Given additionally a local smartlist <b>sl</b>, + * UPDATE_IDX(i) sets the index of the element at <b>i</b> to the correct + * value (that is, to <b>i</b>). + */ +#define IDXP(p) ((int*)STRUCT_VAR_P(p, idx_field_offset)) + +#define UPDATE_IDX(i) do { \ + void *updated = sl->list[i]; \ + *IDXP(updated) = i; \ + } while (0) + +#define IDX_OF_ITEM(p) (*IDXP(p)) +/** @} */ /** Helper. <b>sl</b> may have at most one violation of the heap property: * the item at <b>idx</b> may be greater than one or both of its children. @@ -578,6 +676,7 @@ smartlist_uniq_strings(smartlist_t *sl) static INLINE void smartlist_heapify(smartlist_t *sl, int (*compare)(const void *a, const void *b), + int idx_field_offset, int idx) { while (1) { @@ -600,21 +699,28 @@ smartlist_heapify(smartlist_t *sl, void *tmp = sl->list[idx]; sl->list[idx] = sl->list[best_idx]; sl->list[best_idx] = tmp; + UPDATE_IDX(idx); + UPDATE_IDX(best_idx); idx = best_idx; } } } -/** Insert <b>item</b> into the heap stored in <b>sl</b>, where order - * is determined by <b>compare</b>. */ +/** Insert <b>item</b> into the heap stored in <b>sl</b>, where order is + * determined by <b>compare</b> and the offset of the item in the heap is + * stored in an int-typed field at position <b>idx_field_offset</b> within + * item. + */ void smartlist_pqueue_add(smartlist_t *sl, int (*compare)(const void *a, const void *b), + int idx_field_offset, void *item) { int idx; smartlist_add(sl,item); + UPDATE_IDX(sl->num_used-1); for (idx = sl->num_used - 1; idx; ) { int parent = PARENT(idx); @@ -622,6 +728,8 @@ smartlist_pqueue_add(smartlist_t *sl, void *tmp = sl->list[parent]; sl->list[parent] = sl->list[idx]; sl->list[idx] = tmp; + UPDATE_IDX(parent); + UPDATE_IDX(idx); idx = parent; } else { return; @@ -630,32 +738,63 @@ smartlist_pqueue_add(smartlist_t *sl, } /** Remove and return the top-priority item from the heap stored in <b>sl</b>, - * where order is determined by <b>compare</b>. <b>sl</b> must not be - * empty. */ + * where order is determined by <b>compare</b> and the item's position is + * stored at position <b>idx_field_offset</b> within the item. <b>sl</b> must + * not be empty. */ void * smartlist_pqueue_pop(smartlist_t *sl, - int (*compare)(const void *a, const void *b)) + int (*compare)(const void *a, const void *b), + int idx_field_offset) { void *top; tor_assert(sl->num_used); top = sl->list[0]; + *IDXP(top)=-1; if (--sl->num_used) { sl->list[0] = sl->list[sl->num_used]; - smartlist_heapify(sl, compare, 0); + UPDATE_IDX(0); + smartlist_heapify(sl, compare, idx_field_offset, 0); } return top; } +/** Remove the item <b>item</b> from the heap stored in <b>sl</b>, + * where order is determined by <b>compare</b> and the item's position is + * stored at position <b>idx_field_offset</b> within the item. <b>sl</b> must + * not be empty. */ +void +smartlist_pqueue_remove(smartlist_t *sl, + int (*compare)(const void *a, const void *b), + int idx_field_offset, + void *item) +{ + int idx = IDX_OF_ITEM(item); + tor_assert(idx >= 0); + tor_assert(sl->list[idx] == item); + --sl->num_used; + *IDXP(item) = -1; + if (idx == sl->num_used) { + return; + } else { + sl->list[idx] = sl->list[sl->num_used]; + UPDATE_IDX(idx); + smartlist_heapify(sl, compare, idx_field_offset, idx); + } +} + /** Assert that the heap property is correctly maintained by the heap stored * in <b>sl</b>, where order is determined by <b>compare</b>. */ void smartlist_pqueue_assert_ok(smartlist_t *sl, - int (*compare)(const void *a, const void *b)) + int (*compare)(const void *a, const void *b), + int idx_field_offset) { int i; - for (i = sl->num_used - 1; i > 0; --i) { - tor_assert(compare(sl->list[PARENT(i)], sl->list[i]) <= 0); + for (i = sl->num_used - 1; i >= 0; --i) { + if (i>0) + tor_assert(compare(sl->list[PARENT(i)], sl->list[i]) <= 0); + tor_assert(IDX_OF_ITEM(sl->list[i]) == i); } } @@ -681,6 +820,37 @@ smartlist_uniq_digests(smartlist_t *sl) smartlist_uniq(sl, _compare_digests, _tor_free); } +/** Helper: compare two DIGEST256_LEN digests. */ +static int +_compare_digests256(const void **_a, const void **_b) +{ + return memcmp((const char*)*_a, (const char*)*_b, DIGEST256_LEN); +} + +/** Sort the list of DIGEST256_LEN-byte digests into ascending order. */ +void +smartlist_sort_digests256(smartlist_t *sl) +{ + smartlist_sort(sl, _compare_digests256); +} + +/** Return the most frequent member of the sorted list of DIGEST256_LEN + * digests in <b>sl</b> */ +char * +smartlist_get_most_frequent_digest256(smartlist_t *sl) +{ + return smartlist_get_most_frequent(sl, _compare_digests256); +} + +/** Remove duplicate 256-bit digests from a sorted list, and free them with + * tor_free(). + */ +void +smartlist_uniq_digests256(smartlist_t *sl) +{ + smartlist_uniq(sl, _compare_digests256, _tor_free); +} + /** Helper: Declare an entry type and a map type to implement a mapping using * ht.h. The map type will be called <b>maptype</b>. The key part of each * entry is declared using the C declaration <b>keydecl</b>. All functions @@ -1113,6 +1283,9 @@ void strmap_free(strmap_t *map, void (*free_val)(void*)) { strmap_entry_t **ent, **next, *this; + if (!map) + return; + for (ent = HT_START(strmap_impl, &map->head); ent != NULL; ent = next) { this = *ent; next = HT_NEXT_RMV(strmap_impl, &map->head, ent); @@ -1134,6 +1307,8 @@ void digestmap_free(digestmap_t *map, void (*free_val)(void*)) { digestmap_entry_t **ent, **next, *this; + if (!map) + return; for (ent = HT_START(digestmap_impl, &map->head); ent != NULL; ent = next) { this = *ent; next = HT_NEXT_RMV(digestmap_impl, &map->head, ent); @@ -1220,6 +1395,7 @@ IMPLEMENT_ORDER_FUNC(find_nth_int, int) IMPLEMENT_ORDER_FUNC(find_nth_time, time_t) IMPLEMENT_ORDER_FUNC(find_nth_double, double) IMPLEMENT_ORDER_FUNC(find_nth_uint32, uint32_t) +IMPLEMENT_ORDER_FUNC(find_nth_int32, int32_t) IMPLEMENT_ORDER_FUNC(find_nth_long, long) /** Return a newly allocated digestset_t, optimized to hold a total of @@ -1248,6 +1424,8 @@ digestset_new(int max_elements) void digestset_free(digestset_t *set) { + if (!set) + return; bitarray_free(set->ba); tor_free(set); } diff --git a/src/common/container.h b/src/common/container.h index 2b9f964f7..8a3a40527 100644 --- a/src/common/container.h +++ b/src/common/container.h @@ -15,6 +15,7 @@ * and macros defined here. **/ typedef struct smartlist_t { + /** @{ */ /** <b>list</b> has enough capacity to store exactly <b>capacity</b> elements * before it needs to be resized. Only the first <b>num_used</b> (\<= * capacity) elements point to valid data. @@ -22,6 +23,7 @@ typedef struct smartlist_t { void **list; int num_used; int capacity; + /** @} */ } smartlist_t; smartlist_t *smartlist_create(void); @@ -93,13 +95,22 @@ void smartlist_del_keeporder(smartlist_t *sl, int idx); void smartlist_insert(smartlist_t *sl, int idx, void *val); void smartlist_sort(smartlist_t *sl, int (*compare)(const void **a, const void **b)); +void *smartlist_get_most_frequent(const smartlist_t *sl, + int (*compare)(const void **a, const void **b)); void smartlist_uniq(smartlist_t *sl, int (*compare)(const void **a, const void **b), void (*free_fn)(void *elt)); + void smartlist_sort_strings(smartlist_t *sl); void smartlist_sort_digests(smartlist_t *sl); +void smartlist_sort_digests256(smartlist_t *sl); + +char *smartlist_get_most_frequent_string(smartlist_t *sl); +char *smartlist_get_most_frequent_digest256(smartlist_t *sl); + void smartlist_uniq_strings(smartlist_t *sl); void smartlist_uniq_digests(smartlist_t *sl); +void smartlist_uniq_digests256(smartlist_t *sl); void *smartlist_bsearch(smartlist_t *sl, const void *key, int (*compare)(const void *key, const void **member)) ATTR_PURE; @@ -109,11 +120,18 @@ int smartlist_bsearch_idx(const smartlist_t *sl, const void *key, void smartlist_pqueue_add(smartlist_t *sl, int (*compare)(const void *a, const void *b), + int idx_field_offset, void *item); void *smartlist_pqueue_pop(smartlist_t *sl, - int (*compare)(const void *a, const void *b)); + int (*compare)(const void *a, const void *b), + int idx_field_offset); +void smartlist_pqueue_remove(smartlist_t *sl, + int (*compare)(const void *a, const void *b), + int idx_field_offset, + void *item); void smartlist_pqueue_assert_ok(smartlist_t *sl, - int (*compare)(const void *a, const void *b)); + int (*compare)(const void *a, const void *b), + int idx_field_offset); #define SPLIT_SKIP_SPACE 0x01 #define SPLIT_IGNORE_BLANK 0x02 @@ -579,9 +597,9 @@ bitarray_is_set(bitarray_t *b, int bit) /** A set of digests, implemented as a Bloom filter. */ typedef struct { - int mask; /* One less than the number of bits in <b>ba</b>; always one less + int mask; /**< One less than the number of bits in <b>ba</b>; always one less * than a power of two. */ - bitarray_t *ba; /* A bit array to implement the Bloom filter. */ + bitarray_t *ba; /**< A bit array to implement the Bloom filter. */ } digestset_t; #define BIT(n) ((n) & set->mask) @@ -627,6 +645,7 @@ void digestset_free(digestset_t* set); int find_nth_int(int *array, int n_elements, int nth); time_t find_nth_time(time_t *array, int n_elements, int nth); double find_nth_double(double *array, int n_elements, int nth); +int32_t find_nth_int32(int32_t *array, int n_elements, int nth); uint32_t find_nth_uint32(uint32_t *array, int n_elements, int nth); long find_nth_long(long *array, int n_elements, int nth); static INLINE int @@ -649,6 +668,11 @@ median_uint32(uint32_t *array, int n_elements) { return find_nth_uint32(array, n_elements, (n_elements-1)/2); } +static INLINE int32_t +median_int32(int32_t *array, int n_elements) +{ + return find_nth_int32(array, n_elements, (n_elements-1)/2); +} static INLINE long median_long(long *array, int n_elements) { diff --git a/src/common/crypto.c b/src/common/crypto.c index 838347e20..8d17a3dae 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -27,6 +27,7 @@ #include <openssl/rsa.h> #include <openssl/pem.h> #include <openssl/evp.h> +#include <openssl/engine.h> #include <openssl/rand.h> #include <openssl/opensslv.h> #include <openssl/bn.h> @@ -49,9 +50,9 @@ #define CRYPTO_PRIVATE #include "crypto.h" -#include "log.h" +#include "../common/torlog.h" #include "aes.h" -#include "util.h" +#include "../common/util.h" #include "container.h" #include "compat.h" @@ -61,6 +62,35 @@ #include <openssl/engine.h> +#ifdef ANDROID +/* Android's OpenSSL seems to have removed all of its Engine support. */ +#define DISABLE_ENGINES +#endif + +#if OPENSSL_VERSION_NUMBER < 0x00908000l +/** @{ */ +/** On OpenSSL versions before 0.9.8, there is no working SHA256 + * implementation, so we use Tom St Denis's nice speedy one, slightly adapted + * to our needs. These macros make it usable by us. */ +#define SHA256_CTX sha256_state +#define SHA256_Init sha256_init +#define SHA256_Update sha256_process +#define LTC_ARGCHK(x) tor_assert(x) +/** @} */ +#include "sha256.c" +#define SHA256_Final(a,b) sha256_done(b,a) + +static unsigned char * +SHA256(const unsigned char *m, size_t len, unsigned char *d) +{ + SHA256_CTX ctx; + SHA256_Init(&ctx); + SHA256_Update(&ctx, m, len); + SHA256_Final(d, &ctx); + return d; +} +#endif + /** Macro: is k a valid RSA public or private key? */ #define PUBLIC_KEY_OK(k) ((k) && (k)->key && (k)->key->n) /** Macro: is k a valid RSA private key? */ @@ -76,25 +106,26 @@ static int _n_openssl_mutexes = 0; /** A public key, or a public/private key-pair. */ struct crypto_pk_env_t { - int refs; /* reference counting so we don't have to copy keys */ - RSA *key; + int refs; /**< reference count, so we don't have to copy keys */ + RSA *key; /**< The key itself */ }; /** Key and stream information for a stream cipher. */ struct crypto_cipher_env_t { - char key[CIPHER_KEY_LEN]; - aes_cnt_cipher_t *cipher; + char key[CIPHER_KEY_LEN]; /**< The raw key. */ + aes_cnt_cipher_t *cipher; /**< The key in format usable for counter-mode AES + * encryption */ }; /** A structure to hold the first half (x, g^x) of a Diffie-Hellman handshake * while we're waiting for the second.*/ struct crypto_dh_env_t { - DH *dh; + DH *dh; /**< The openssl DH object */ }; static int setup_openssl_threading(void); -static int tor_check_dh_key(BIGNUM *bn); +static int tor_check_dh_key(int severity, BIGNUM *bn); /** Return the number of bytes added by padding method <b>padding</b>. */ @@ -151,6 +182,7 @@ crypto_log_errors(int severity, const char *doing) } } +#ifndef DISABLE_ENGINES /** Log any OpenSSL engines we're using at NOTICE. */ static void log_engine(const char *fn, ENGINE *e) @@ -165,37 +197,82 @@ log_engine(const char *fn, ENGINE *e) log(LOG_INFO, LD_CRYPTO, "Using default implementation for %s", fn); } } +#endif + +#ifndef DISABLE_ENGINES +/** Try to load an engine in a shared library via fully qualified path. + */ +static ENGINE * +try_load_engine(const char *path, const char *engine) +{ + ENGINE *e = ENGINE_by_id("dynamic"); + if (e) { + if (!ENGINE_ctrl_cmd_string(e, "ID", engine, 0) || + !ENGINE_ctrl_cmd_string(e, "DIR_LOAD", "2", 0) || + !ENGINE_ctrl_cmd_string(e, "DIR_ADD", path, 0) || + !ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0)) { + ENGINE_free(e); + e = NULL; + } + } + return e; +} +#endif /** Initialize the crypto library. Return 0 on success, -1 on failure. */ int -crypto_global_init(int useAccel) +crypto_global_init(int useAccel, const char *accelName, const char *accelDir) { if (!_crypto_global_initialized) { ERR_load_crypto_strings(); OpenSSL_add_all_algorithms(); _crypto_global_initialized = 1; setup_openssl_threading(); - /* XXX the below is a bug, since we can't know if we're supposed - * to be using hardware acceleration or not. we should arrange - * for this function to be called before init_keys. But make it - * not complain loudly, at least until we make acceleration work. */ - if (useAccel < 0) { - log_info(LD_CRYPTO, "Initializing OpenSSL via tor_tls_init()."); - } if (useAccel > 0) { +#ifdef DISABLE_ENGINES + (void)accelName; + (void)accelDir; + log_warn(LD_CRYPTO, "No OpenSSL hardware acceleration support enabled."); +#else + ENGINE *e = NULL; + log_info(LD_CRYPTO, "Initializing OpenSSL engine support."); ENGINE_load_builtin_engines(); - if (!ENGINE_register_all_complete()) - return -1; - - /* XXXX make sure this isn't leaking. */ + ENGINE_register_all_complete(); + + if (accelName) { + if (accelDir) { + log_info(LD_CRYPTO, "Trying to load dynamic OpenSSL engine \"%s\"" + " via path \"%s\".", accelName, accelDir); + e = try_load_engine(accelName, accelDir); + } else { + log_info(LD_CRYPTO, "Initializing dynamic OpenSSL engine \"%s\"" + " acceleration support.", accelName); + e = ENGINE_by_id(accelName); + } + if (!e) { + log_warn(LD_CRYPTO, "Unable to load dynamic OpenSSL engine \"%s\".", + accelName); + } else { + log_info(LD_CRYPTO, "Loaded dynamic OpenSSL engine \"%s\".", + accelName); + } + } + if (e) { + log_info(LD_CRYPTO, "Loaded OpenSSL hardware acceleration engine," + " setting default ciphers."); + ENGINE_set_default(e, ENGINE_METHOD_ALL); + } log_engine("RSA", ENGINE_get_default_RSA()); log_engine("DH", ENGINE_get_default_DH()); log_engine("RAND", ENGINE_get_default_RAND()); log_engine("SHA1", ENGINE_get_digest_engine(NID_sha1)); log_engine("3DES", ENGINE_get_cipher_engine(NID_des_ede3_ecb)); log_engine("AES", ENGINE_get_cipher_engine(NID_aes_128_ecb)); +#endif + } else { + log_info(LD_CRYPTO, "NOT using OpenSSL engine support."); } return crypto_seed_rng(1); } @@ -217,7 +294,11 @@ crypto_global_cleanup(void) EVP_cleanup(); ERR_remove_state(0); ERR_free_strings(); + +#ifndef DISABLE_ENGINES ENGINE_cleanup(); +#endif + CONF_modules_unload(1); CRYPTO_cleanup_all_ex_data(); #ifdef TOR_IS_MULTITHREADED @@ -248,18 +329,8 @@ _crypto_new_pk_env_rsa(RSA *rsa) return env; } -/** used by tortls.c: wrap the RSA from an evp_pkey in a crypto_pk_env_t. - * returns NULL if this isn't an RSA key. */ -crypto_pk_env_t * -_crypto_new_pk_env_evp_pkey(EVP_PKEY *pkey) -{ - RSA *rsa; - if (!(rsa = EVP_PKEY_get1_RSA(pkey))) - return NULL; - return _crypto_new_pk_env_rsa(rsa); -} - -/** Helper, used by tor-checkkey.c. Return the RSA from a crypto_pk_env_t. */ +/** Helper, used by tor-checkkey.c and tor-gencert.c. Return the RSA from a + * crypto_pk_env_t. */ RSA * _crypto_pk_env_get_rsa(crypto_pk_env_t *env) { @@ -311,7 +382,7 @@ crypto_new_pk_env(void) RSA *rsa; rsa = RSA_new(); - if (!rsa) return NULL; + tor_assert(rsa); return _crypto_new_pk_env_rsa(rsa); } @@ -321,10 +392,12 @@ crypto_new_pk_env(void) void crypto_free_pk_env(crypto_pk_env_t *env) { - tor_assert(env); + if (!env) + return; if (--env->refs > 0) return; + tor_assert(env->refs == 0); if (env->key) RSA_free(env->key); @@ -347,10 +420,7 @@ crypto_create_init_cipher(const char *key, int encrypt_mode) return NULL; } - if (crypto_cipher_set_key(crypto, key)) { - crypto_log_errors(LOG_WARN, "setting symmetric key"); - goto error; - } + crypto_cipher_set_key(crypto, key); if (encrypt_mode) r = crypto_cipher_encrypt_init_cipher(crypto); @@ -384,7 +454,8 @@ crypto_new_cipher_env(void) void crypto_free_cipher_env(crypto_cipher_env_t *env) { - tor_assert(env); + if (!env) + return; tor_assert(env->cipher); aes_free_cipher(env->cipher); @@ -394,11 +465,11 @@ crypto_free_cipher_env(crypto_cipher_env_t *env) /* public key crypto */ -/** Generate a new public/private keypair in <b>env</b>. Return 0 on - * success, -1 on failure. +/** Generate a <b>bits</b>-bit new public/private keypair in <b>env</b>. + * Return 0 on success, -1 on failure. */ int -crypto_pk_generate_key(crypto_pk_env_t *env) +crypto_pk_generate_key_with_bits(crypto_pk_env_t *env, int bits) { tor_assert(env); @@ -406,7 +477,7 @@ crypto_pk_generate_key(crypto_pk_env_t *env) RSA_free(env->key); #if OPENSSL_VERSION_NUMBER < 0x00908000l /* In OpenSSL 0.9.7, RSA_generate_key is all we have. */ - env->key = RSA_generate_key(PK_BYTES*8,65537, NULL, NULL); + env->key = RSA_generate_key(bits, 65537, NULL, NULL); #else /* In OpenSSL 0.9.8, RSA_generate_key is deprecated. */ { @@ -419,7 +490,7 @@ crypto_pk_generate_key(crypto_pk_env_t *env) r = RSA_new(); if (!r) goto done; - if (RSA_generate_key_ex(r, PK_BYTES*8, e, NULL) == -1) + if (RSA_generate_key_ex(r, bits, e, NULL) == -1) goto done; env->key = r; @@ -456,6 +527,8 @@ crypto_pk_read_private_key_from_string(crypto_pk_env_t *env, /* Create a read-only memory BIO, backed by the string 's' */ b = BIO_new_mem_buf((char*)s, (int)len); + if (!b) + return -1; if (env->key) RSA_free(env->key); @@ -516,6 +589,8 @@ crypto_pk_write_key_to_string_impl(crypto_pk_env_t *env, char **dest, tor_assert(dest); b = BIO_new(BIO_s_mem()); /* Create a memory BIO */ + if (!b) + return -1; /* Now you can treat b as if it were a file. Just use the * PEM_*_bio_* functions instead of the non-bio variants. @@ -583,6 +658,8 @@ crypto_pk_read_public_key_from_string(crypto_pk_env_t *env, const char *src, tor_assert(len<INT_MAX); b = BIO_new(BIO_s_mem()); /* Create a memory BIO */ + if (!b) + return -1; BIO_write(b, src, (int)len); @@ -705,14 +782,25 @@ crypto_pk_env_t * crypto_pk_copy_full(crypto_pk_env_t *env) { RSA *new_key; + int privatekey = 0; tor_assert(env); tor_assert(env->key); if (PRIVATE_KEY_OK(env)) { new_key = RSAPrivateKey_dup(env->key); + privatekey = 1; } else { new_key = RSAPublicKey_dup(env->key); } + if (!new_key) { + log_err(LD_CRYPTO, "Unable to duplicate a %s key: openssl failed.", + privatekey?"private":"public"); + crypto_log_errors(LOG_ERR, + privatekey ? "Duplicating a private key" : + "Duplicating a public key"); + tor_fragile_assert(); + return NULL; + } return _crypto_new_pk_env_rsa(new_key); } @@ -1213,19 +1301,14 @@ crypto_cipher_generate_key(crypto_cipher_env_t *env) /** Set the symmetric key for the cipher in <b>env</b> to the first * CIPHER_KEY_LEN bytes of <b>key</b>. Does not initialize the cipher. - * Return 0 on success, -1 on failure. */ -int +void crypto_cipher_set_key(crypto_cipher_env_t *env, const char *key) { tor_assert(env); tor_assert(key); - if (!env->key) - return -1; - memcpy(env->key, key, CIPHER_KEY_LEN); - return 0; } /** Generate an initialization vector for our AES-CTR cipher; store it @@ -1390,7 +1473,7 @@ crypto_cipher_decrypt_with_iv(crypto_cipher_env_t *cipher, /* SHA-1 */ -/** Compute the SHA1 digest of <b>len</b> bytes in data stored in +/** Compute the SHA1 digest of the <b>len</b> bytes on data stored in * <b>m</b>. Write the DIGEST_LEN byte result into <b>digest</b>. * Return 0 on success, -1 on failure. */ @@ -1402,19 +1485,97 @@ crypto_digest(char *digest, const char *m, size_t len) return (SHA1((const unsigned char*)m,len,(unsigned char*)digest) == NULL); } +/** Compute a 256-bit digest of <b>len</b> bytes in data stored in <b>m</b>, + * using the algorithm <b>algorithm</b>. Write the DIGEST_LEN256-byte result + * into <b>digest</b>. Return 0 on success, -1 on failure. */ +int +crypto_digest256(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm) +{ + tor_assert(m); + tor_assert(digest); + tor_assert(algorithm == DIGEST_SHA256); + return (SHA256((const unsigned char*)m,len,(unsigned char*)digest) == NULL); +} + +/** Set the digests_t in <b>ds_out</b> to contain every digest on the + * <b>len</b> bytes in <b>m</b> that we know how to compute. Return 0 on + * success, -1 on failure. */ +int +crypto_digest_all(digests_t *ds_out, const char *m, size_t len) +{ + digest_algorithm_t i; + tor_assert(ds_out); + memset(ds_out, 0, sizeof(*ds_out)); + if (crypto_digest(ds_out->d[DIGEST_SHA1], m, len) < 0) + return -1; + for (i = DIGEST_SHA256; i < N_DIGEST_ALGORITHMS; ++i) { + if (crypto_digest256(ds_out->d[i], m, len, i) < 0) + return -1; + } + return 0; +} + +/** Return the name of an algorithm, as used in directory documents. */ +const char * +crypto_digest_algorithm_get_name(digest_algorithm_t alg) +{ + switch (alg) { + case DIGEST_SHA1: + return "sha1"; + case DIGEST_SHA256: + return "sha256"; + default: + tor_fragile_assert(); + return "??unknown_digest??"; + } +} + +/** Given the name of a digest algorithm, return its integer value, or -1 if + * the name is not recognized. */ +int +crypto_digest_algorithm_parse_name(const char *name) +{ + if (!strcmp(name, "sha1")) + return DIGEST_SHA1; + else if (!strcmp(name, "sha256")) + return DIGEST_SHA256; + else + return -1; +} + /** Intermediate information about the digest of a stream of data. */ struct crypto_digest_env_t { - SHA_CTX d; + union { + SHA_CTX sha1; /**< state for SHA1 */ + SHA256_CTX sha2; /**< state for SHA256 */ + } d; /**< State for the digest we're using. Only one member of the + * union is usable, depending on the value of <b>algorithm</b>. */ + digest_algorithm_t algorithm : 8; /**< Which algorithm is in use? */ }; -/** Allocate and return a new digest object. +/** Allocate and return a new digest object to compute SHA1 digests. */ crypto_digest_env_t * crypto_new_digest_env(void) { crypto_digest_env_t *r; r = tor_malloc(sizeof(crypto_digest_env_t)); - SHA1_Init(&r->d); + SHA1_Init(&r->d.sha1); + r->algorithm = DIGEST_SHA1; + return r; +} + +/** Allocate and return a new digest object to compute 256-bit digests + * using <b>algorithm</b>. */ +crypto_digest_env_t * +crypto_new_digest256_env(digest_algorithm_t algorithm) +{ + crypto_digest_env_t *r; + tor_assert(algorithm == DIGEST_SHA256); + r = tor_malloc(sizeof(crypto_digest_env_t)); + SHA256_Init(&r->d.sha2); + r->algorithm = algorithm; return r; } @@ -1423,6 +1584,8 @@ crypto_new_digest_env(void) void crypto_free_digest_env(crypto_digest_env_t *digest) { + if (!digest) + return; memset(digest, 0, sizeof(crypto_digest_env_t)); tor_free(digest); } @@ -1435,30 +1598,51 @@ crypto_digest_add_bytes(crypto_digest_env_t *digest, const char *data, { tor_assert(digest); tor_assert(data); - /* Using the SHA1_*() calls directly means we don't support doing - * SHA1 in hardware. But so far the delay of getting the question + /* Using the SHA*_*() calls directly means we don't support doing + * SHA in hardware. But so far the delay of getting the question * to the hardware, and hearing the answer, is likely higher than * just doing it ourselves. Hashes are fast. */ - SHA1_Update(&digest->d, (void*)data, len); + switch (digest->algorithm) { + case DIGEST_SHA1: + SHA1_Update(&digest->d.sha1, (void*)data, len); + break; + case DIGEST_SHA256: + SHA256_Update(&digest->d.sha2, (void*)data, len); + break; + default: + tor_fragile_assert(); + break; + } } /** Compute the hash of the data that has been passed to the digest * object; write the first out_len bytes of the result to <b>out</b>. - * <b>out_len</b> must be \<= DIGEST_LEN. + * <b>out_len</b> must be \<= DIGEST256_LEN. */ void crypto_digest_get_digest(crypto_digest_env_t *digest, char *out, size_t out_len) { - unsigned char r[DIGEST_LEN]; - SHA_CTX tmpctx; + unsigned char r[DIGEST256_LEN]; + crypto_digest_env_t tmpenv; tor_assert(digest); tor_assert(out); - tor_assert(out_len <= DIGEST_LEN); - /* memcpy into a temporary ctx, since SHA1_Final clears the context */ - memcpy(&tmpctx, &digest->d, sizeof(SHA_CTX)); - SHA1_Final(r, &tmpctx); + /* memcpy into a temporary ctx, since SHA*_Final clears the context */ + memcpy(&tmpenv, digest, sizeof(crypto_digest_env_t)); + switch (digest->algorithm) { + case DIGEST_SHA1: + tor_assert(out_len <= DIGEST_LEN); + SHA1_Final(r, &tmpenv.d.sha1); + break; + case DIGEST_SHA256: + tor_assert(out_len <= DIGEST256_LEN); + SHA256_Final(r, &tmpenv.d.sha2); + break; + default: + tor_fragile_assert(); + break; + } memcpy(out, r, out_len); memset(r, 0, sizeof(r)); } @@ -1557,6 +1741,10 @@ init_dh_param(void) dh_param_g = g; } +/** Number of bits to use when choosing the x or y value in a Diffie-Hellman + * handshake. Since we exponentiate by this value, choosing a smaller one + * lets our handhake go faster. + */ #define DH_PRIVATE_KEY_BITS 320 /** Allocate and return a new DH object for a key exchange. @@ -1616,7 +1804,7 @@ crypto_dh_generate_public(crypto_dh_env_t *dh) crypto_log_errors(LOG_WARN, "generating DH key"); return -1; } - if (tor_check_dh_key(dh->dh->pub_key)<0) { + if (tor_check_dh_key(LOG_WARN, dh->dh->pub_key)<0) { log_warn(LD_CRYPTO, "Weird! Our own DH key was invalid. I guess once-in-" "the-universe chances really do happen. Trying again."); /* Free and clear the keys, so OpenSSL will actually try again. */ @@ -1663,7 +1851,7 @@ crypto_dh_get_public(crypto_dh_env_t *dh, char *pubkey, size_t pubkey_len) * See http://www.cl.cam.ac.uk/ftp/users/rja14/psandqs.ps.gz for some tips. */ static int -tor_check_dh_key(BIGNUM *bn) +tor_check_dh_key(int severity, BIGNUM *bn) { BIGNUM *x; char *s; @@ -1674,13 +1862,13 @@ tor_check_dh_key(BIGNUM *bn) init_dh_param(); BN_set_word(x, 1); if (BN_cmp(bn,x)<=0) { - log_warn(LD_CRYPTO, "DH key must be at least 2."); + log_fn(severity, LD_CRYPTO, "DH key must be at least 2."); goto err; } BN_copy(x,dh_param_p); BN_sub_word(x, 1); if (BN_cmp(bn,x)>=0) { - log_warn(LD_CRYPTO, "DH key must be at most p-2."); + log_fn(severity, LD_CRYPTO, "DH key must be at most p-2."); goto err; } BN_free(x); @@ -1688,7 +1876,7 @@ tor_check_dh_key(BIGNUM *bn) err: BN_free(x); s = BN_bn2hex(bn); - log_warn(LD_CRYPTO, "Rejecting insecure DH key [%s]", s); + log_fn(severity, LD_CRYPTO, "Rejecting insecure DH key [%s]", s); OPENSSL_free(s); return -1; } @@ -1706,7 +1894,7 @@ tor_check_dh_key(BIGNUM *bn) * where || is concatenation.) */ ssize_t -crypto_dh_compute_secret(crypto_dh_env_t *dh, +crypto_dh_compute_secret(int severity, crypto_dh_env_t *dh, const char *pubkey, size_t pubkey_len, char *secret_out, size_t secret_bytes_out) { @@ -1721,9 +1909,9 @@ crypto_dh_compute_secret(crypto_dh_env_t *dh, if (!(pubkey_bn = BN_bin2bn((const unsigned char*)pubkey, (int)pubkey_len, NULL))) goto error; - if (tor_check_dh_key(pubkey_bn)<0) { + if (tor_check_dh_key(severity, pubkey_bn)<0) { /* Check for invalid public keys. */ - log_warn(LD_CRYPTO,"Rejected invalid g^x"); + log_fn(severity, LD_CRYPTO,"Rejected invalid g^x"); goto error; } secret_tmp_len = crypto_dh_get_bytes(dh); @@ -1799,7 +1987,8 @@ crypto_expand_key_material(const char *key_in, size_t key_in_len, void crypto_dh_free(crypto_dh_env_t *dh) { - tor_assert(dh); + if (!dh) + return; tor_assert(dh->dh); DH_free(dh->dh); tor_free(dh); @@ -1807,15 +1996,22 @@ crypto_dh_free(crypto_dh_env_t *dh) /* random numbers */ -/* This is how much entropy OpenSSL likes to add right now, so maybe it will +/** How many bytes of entropy we add at once. + * + * This is how much entropy OpenSSL likes to add right now, so maybe it will * work for us too. */ #define ADD_ENTROPY 32 -/* Use RAND_poll if OpenSSL is 0.9.6 release or later. (The "f" means - "release".) */ +/** True iff we should use OpenSSL's RAND_poll function to add entropy to its + * pool. + * + * Use RAND_poll if OpenSSL is 0.9.6 release or later. (The "f" means + *"release".) */ #define HAVE_RAND_POLL (OPENSSL_VERSION_NUMBER >= 0x0090600fl) -/* Versions of OpenSSL prior to 0.9.7k and 0.9.8c had a bug where RAND_poll +/** True iff it's safe to use RAND_poll after setup. + * + * Versions of OpenSSL prior to 0.9.7k and 0.9.8c had a bug where RAND_poll * would allocate an fd_set on the stack, open a new file, and try to FD_SET * that fd without checking whether it fit in the fd_set. Thus, if the * system has not just been started up, it is unsafe to call */ @@ -1824,6 +2020,15 @@ crypto_dh_free(crypto_dh_env_t *dh) OPENSSL_VERSION_NUMBER <= 0x00907fffl) || \ (OPENSSL_VERSION_NUMBER >= 0x0090803fl)) +/** Set the seed of the weak RNG to a random value. */ +static void +seed_weak_rng(void) +{ + unsigned seed; + crypto_rand((void*)&seed, sizeof(seed)); + tor_init_weak_random(seed); +} + /** Seed OpenSSL's random number generator with bytes from the operating * system. <b>startup</b> should be true iff we have just started Tor and * have not yet allocated a bunch of fds. Return 0 on success, -1 on failure. @@ -1831,14 +2036,15 @@ crypto_dh_free(crypto_dh_env_t *dh) int crypto_seed_rng(int startup) { - char buf[ADD_ENTROPY]; int rand_poll_status = 0; /* local variables */ #ifdef MS_WINDOWS + unsigned char buf[ADD_ENTROPY]; static int provider_set = 0; static HCRYPTPROV provider; #else + char buf[ADD_ENTROPY]; static const char *filenames[] = { "/dev/srandom", "/dev/urandom", "/dev/random", NULL }; @@ -1874,6 +2080,7 @@ crypto_seed_rng(int startup) } RAND_seed(buf, sizeof(buf)); memset(buf, 0, sizeof(buf)); + seed_weak_rng(); return 0; #else for (i = 0; filenames[i]; ++i) { @@ -1890,6 +2097,7 @@ crypto_seed_rng(int startup) } RAND_seed(buf, (int)sizeof(buf)); memset(buf, 0, sizeof(buf)); + seed_weak_rng(); return 0; } @@ -1957,6 +2165,26 @@ crypto_rand_uint64(uint64_t max) } } +/** Return a pseudorandom double d, chosen uniformly from the range + * 0.0 <= d < 1.0. + */ +double +crypto_rand_double(void) +{ + /* We just use an unsigned int here; we don't really care about getting + * more than 32 bits of resolution */ + unsigned int uint; + crypto_rand((char*)&uint, sizeof(uint)); +#if SIZEOF_INT == 4 +#define UINT_MAX_AS_DOUBLE 4294967296.0 +#elif SIZEOF_INT == 8 +#define UINT_MAX_AS_DOUBLE 1.8446744073709552e+19 +#else +#error SIZEOF_INT is neither 4 nor 8 +#endif + return ((double)uint) / UINT_MAX_AS_DOUBLE; +} + /** Generate and return a new random hostname starting with <b>prefix</b>, * ending with <b>suffix</b>, and containing no less than * <b>min_rand_len</b> and no more than <b>max_rand_len</b> random base32 @@ -2046,9 +2274,12 @@ base64_encode(char *dest, size_t destlen, const char *src, size_t srclen) return ret; } +/** @{ */ +/** Special values used for the base64_decode_table */ #define X 255 #define SP 64 #define PAD 65 +/** @} */ /** Internal table mapping byte values to what they represent in base64. * Numbers 0..63 are 6-bit integers. SPs are spaces, and should be * skipped. Xs are invalid and must not appear in base64. PAD indicates @@ -2217,15 +2448,54 @@ digest_from_base64(char *digest, const char *d64) #endif } +/** Base-64 encode DIGEST256_LINE bytes from <b>digest</b>, remove the + * trailing = and newline characters, and store the nul-terminated result in + * the first BASE64_DIGEST256_LEN+1 bytes of <b>d64</b>. */ +int +digest256_to_base64(char *d64, const char *digest) +{ + char buf[256]; + base64_encode(buf, sizeof(buf), digest, DIGEST256_LEN); + buf[BASE64_DIGEST256_LEN] = '\0'; + memcpy(d64, buf, BASE64_DIGEST256_LEN+1); + return 0; +} + +/** Given a base-64 encoded, nul-terminated digest in <b>d64</b> (without + * trailing newline or = characters), decode it and store the result in the + * first DIGEST256_LEN bytes at <b>digest</b>. */ +int +digest256_from_base64(char *digest, const char *d64) +{ +#ifdef USE_OPENSSL_BASE64 + char buf_in[BASE64_DIGEST256_LEN+3]; + char buf[256]; + if (strlen(d64) != BASE64_DIGEST256_LEN) + return -1; + memcpy(buf_in, d64, BASE64_DIGEST256_LEN); + memcpy(buf_in+BASE64_DIGEST256_LEN, "=\n\0", 3); + if (base64_decode(buf, sizeof(buf), buf_in, strlen(buf_in)) != DIGEST256_LEN) + return -1; + memcpy(digest, buf, DIGEST256_LEN); + return 0; +#else + if (base64_decode(digest, DIGEST256_LEN, d64, strlen(d64)) == DIGEST256_LEN) + return 0; + else + return -1; +#endif +} + /** Implements base32 encoding as in rfc3548. Limitation: Requires * that srclen*8 is a multiple of 5. */ void base32_encode(char *dest, size_t destlen, const char *src, size_t srclen) { - unsigned int i, bit, v, u; - size_t nbits = srclen * 8; + unsigned int i, v, u; + size_t nbits = srclen * 8, bit; + tor_assert(srclen < SIZE_T_CEILING/8); tor_assert((nbits%5) == 0); /* We need an even multiple of 5 bits. */ tor_assert((nbits/5)+1 <= destlen); /* We need enough space. */ tor_assert(destlen < SIZE_T_CEILING); @@ -2249,11 +2519,12 @@ base32_decode(char *dest, size_t destlen, const char *src, size_t srclen) { /* XXXX we might want to rewrite this along the lines of base64_decode, if * it ever shows up in the profile. */ - unsigned int i, j, bit; - size_t nbits; + unsigned int i; + size_t nbits, j, bit; char *tmp; nbits = srclen * 5; + tor_assert(srclen < SIZE_T_CEILING / 5); tor_assert((nbits%8) == 0); /* We need an even multiple of 8 bits. */ tor_assert((nbits/8) <= destlen); /* We need enough space. */ tor_assert(destlen < SIZE_T_CEILING); @@ -2412,6 +2683,7 @@ _openssl_dynlock_destroy_cb(struct CRYPTO_dynlock_value *v, tor_free(v); } +/** @{ */ /** Helper: Construct mutexes, and set callbacks to help OpenSSL handle being * multithreaded. */ static int @@ -2437,4 +2709,5 @@ setup_openssl_threading(void) return 0; } #endif +/** @} */ diff --git a/src/common/crypto.h b/src/common/crypto.h index 576c03dc3..05185f3f1 100644 --- a/src/common/crypto.h +++ b/src/common/crypto.h @@ -18,6 +18,9 @@ /** Length of the output of our message digest. */ #define DIGEST_LEN 20 +/** Length of the output of our second (improved) message digests. (For now + * this is just sha256, but any it can be any other 256-byte digest). */ +#define DIGEST256_LEN 32 /** Length of our symmetric cipher's keys. */ #define CIPHER_KEY_LEN 16 /** Length of our symmetric cipher's IV. */ @@ -27,9 +30,12 @@ /** Length of our DH keys. */ #define DH_BYTES (1024/8) -/** Length of a message digest when encoded in base64 with trailing = signs - * removed. */ +/** Length of a sha1 message digest when encoded in base64 with trailing = + * signs removed. */ #define BASE64_DIGEST_LEN 27 +/** Length of a sha256 message digest when encoded in base64 with trailing = + * signs removed. */ +#define BASE64_DIGEST256_LEN 43 /** Constants used to indicate no padding for public-key encryption */ #define PK_NO_PADDING 60000 @@ -48,6 +54,26 @@ #define FINGERPRINT_LEN 49 /** Length of hex encoding of SHA1 digest, not including final NUL. */ #define HEX_DIGEST_LEN 40 +/** Length of hex encoding of SHA256 digest, not including final NUL. */ +#define HEX_DIGEST256_LEN 64 + +typedef enum { + DIGEST_SHA1 = 0, + DIGEST_SHA256 = 1, +} digest_algorithm_t; +#define N_DIGEST_ALGORITHMS (DIGEST_SHA256+1) + +/** A set of all the digests we know how to compute, taken on a single + * string. Any digests that are shorter than 256 bits are right-padded + * with 0 bits. + * + * Note that this representation wastes 12 bytes for the SHA1 case, so + * don't use it for anything where we need to allocate a whole bunch at + * once. + **/ +typedef struct { + char d[N_DIGEST_ALGORITHMS][DIGEST256_LEN]; +} digests_t; typedef struct crypto_pk_env_t crypto_pk_env_t; typedef struct crypto_cipher_env_t crypto_cipher_env_t; @@ -55,7 +81,9 @@ typedef struct crypto_digest_env_t crypto_digest_env_t; typedef struct crypto_dh_env_t crypto_dh_env_t; /* global state */ -int crypto_global_init(int hardwareAccel); +int crypto_global_init(int hardwareAccel, + const char *accelName, + const char *accelPath); void crypto_thread_cleanup(void); int crypto_global_cleanup(void); @@ -71,7 +99,9 @@ crypto_cipher_env_t *crypto_new_cipher_env(void); void crypto_free_cipher_env(crypto_cipher_env_t *env); /* public key crypto */ -int crypto_pk_generate_key(crypto_pk_env_t *env); +int crypto_pk_generate_key_with_bits(crypto_pk_env_t *env, int bits); +#define crypto_pk_generate_key(env) \ + crypto_pk_generate_key_with_bits((env), (PK_BYTES*8)) int crypto_pk_read_private_key_from_filename(crypto_pk_env_t *env, const char *keyfile); @@ -123,7 +153,7 @@ int crypto_pk_check_fingerprint_syntax(const char *s); /* symmetric crypto */ int crypto_cipher_generate_key(crypto_cipher_env_t *env); -int crypto_cipher_set_key(crypto_cipher_env_t *env, const char *key); +void crypto_cipher_set_key(crypto_cipher_env_t *env, const char *key); void crypto_cipher_generate_iv(char *iv_out); int crypto_cipher_set_iv(crypto_cipher_env_t *env, const char *iv); const char *crypto_cipher_get_key(crypto_cipher_env_t *env); @@ -143,9 +173,15 @@ int crypto_cipher_decrypt_with_iv(crypto_cipher_env_t *env, char *to, size_t tolen, const char *from, size_t fromlen); -/* SHA-1 */ +/* SHA-1 and other digests. */ int crypto_digest(char *digest, const char *m, size_t len); +int crypto_digest256(char *digest, const char *m, size_t len, + digest_algorithm_t algorithm); +int crypto_digest_all(digests_t *ds_out, const char *m, size_t len); +const char *crypto_digest_algorithm_get_name(digest_algorithm_t alg); +int crypto_digest_algorithm_parse_name(const char *name); crypto_digest_env_t *crypto_new_digest_env(void); +crypto_digest_env_t *crypto_new_digest256_env(digest_algorithm_t algorithm); void crypto_free_digest_env(crypto_digest_env_t *digest); void crypto_digest_add_bytes(crypto_digest_env_t *digest, const char *data, size_t len); @@ -167,7 +203,7 @@ int crypto_dh_get_bytes(crypto_dh_env_t *dh); int crypto_dh_generate_public(crypto_dh_env_t *dh); int crypto_dh_get_public(crypto_dh_env_t *dh, char *pubkey_out, size_t pubkey_out_len); -ssize_t crypto_dh_compute_secret(crypto_dh_env_t *dh, +ssize_t crypto_dh_compute_secret(int severity, crypto_dh_env_t *dh, const char *pubkey, size_t pubkey_len, char *secret_out, size_t secret_out_len); void crypto_dh_free(crypto_dh_env_t *dh); @@ -179,6 +215,7 @@ int crypto_seed_rng(int startup); int crypto_rand(char *to, size_t n); int crypto_rand_int(unsigned int max); uint64_t crypto_rand_uint64(uint64_t max); +double crypto_rand_double(void); char *crypto_random_hostname(int min_rand_len, int max_rand_len, const char *prefix, const char *suffix); @@ -196,6 +233,8 @@ int base32_decode(char *dest, size_t destlen, const char *src, size_t srclen); int digest_to_base64(char *d64, const char *digest); int digest_from_base64(char *digest, const char *d64); +int digest256_to_base64(char *d64, const char *digest); +int digest256_from_base64(char *digest, const char *d64); /** Length of RFC2440-style S2K specifier: the first 8 bytes are a salt, the * 9th describes how much iteration to do. */ @@ -210,7 +249,6 @@ struct evp_pkey_st; struct dh_st; struct rsa_st *_crypto_pk_env_get_rsa(crypto_pk_env_t *env); crypto_pk_env_t *_crypto_new_pk_env_rsa(struct rsa_st *rsa); -crypto_pk_env_t *_crypto_new_pk_env_evp_pkey(struct evp_pkey_st *pkey); struct evp_pkey_st *_crypto_pk_env_get_evp_pkey(crypto_pk_env_t *env, int private); struct dh_st *_crypto_dh_env_get_dh(crypto_dh_env_t *dh); diff --git a/src/common/ht.h b/src/common/ht.h index 033cd89ab..0850c0709 100644 --- a/src/common/ht.h +++ b/src/common/ht.h @@ -42,6 +42,10 @@ #define HT_SIZE(head) \ ((head)->hth_n_entries) +/* Return memory usage for a hashtable (not counting the entries themselves) */ +#define HT_MEM_USAGE(head) \ + (sizeof(*head) + (head)->hth_table_length * sizeof(void*)) + #define HT_FIND(name, head, elm) name##_HT_FIND((head), (elm)) #define HT_INSERT(name, head, elm) name##_HT_INSERT((head), (elm)) #define HT_REPLACE(name, head, elm) name##_HT_REPLACE((head), (elm)) diff --git a/src/common/log.c b/src/common/log.c index fe93bd7f5..d14563c88 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -33,13 +33,15 @@ #include "compat.h" #include "util.h" #define LOG_PRIVATE -#include "log.h" +#include "torlog.h" #include "container.h" -#include <event.h> - +/** @{ */ +/** The string we stick at the end of a log message when it is too long, + * and its length. */ #define TRUNCATED_STR "[...truncated]" #define TRUNCATED_STR_LEN 14 +/** @} */ /** Information for a single logfile; only used in log.c */ typedef struct logfile_t { @@ -83,7 +85,7 @@ should_log_function_name(log_domain_mask_t domain, int severity) /* All debugging messages occur in interesting places. */ return 1; case LOG_NOTICE: - case LOG_WARN: + case LOG_WARN: case LOG_ERR: /* We care about places where bugs occur. */ return (domain == LD_BUG); @@ -99,15 +101,31 @@ static int log_mutex_initialized = 0; /** Linked list of logfile_t. */ static logfile_t *logfiles = NULL; +/** Boolean: do we report logging domains? */ +static int log_domains_are_logged = 0; + #ifdef HAVE_SYSLOG_H /** The number of open syslog log handlers that we have. When this reaches 0, * we can close our connection to the syslog facility. */ static int syslog_count = 0; #endif +/** Represents a log message that we are going to send to callback-driven + * loggers once we can do so in a non-reentrant way. */ +typedef struct pending_cb_message_t { + int severity; /**< The severity of the message */ + log_domain_mask_t domain; /**< The domain of the message */ + char *msg; /**< The content of the message */ +} pending_cb_message_t; + +/** Log messages waiting to be replayed onto callback-based logs */ +static smartlist_t *pending_cb_messages = NULL; + +/** Lock the log_mutex to prevent others from changing the logfile_t list */ #define LOCK_LOGS() STMT_BEGIN \ tor_mutex_acquire(&log_mutex); \ STMT_END +/** Unlock the log_mutex */ #define UNLOCK_LOGS() STMT_BEGIN tor_mutex_release(&log_mutex); STMT_END /** What's the lowest log level anybody cares about? Checking this lets us @@ -117,6 +135,9 @@ int _log_global_min_severity = LOG_NOTICE; static void delete_log(logfile_t *victim); static void close_log(logfile_t *victim); +static char *domain_to_string(log_domain_mask_t domain, + char *buf, size_t buflen); + /** Name of the application: used to generate the message we write at the * start of each new log. */ static char *appname = NULL; @@ -208,13 +229,34 @@ format_msg(char *buf, size_t buf_len, size_t n; int r; char *end_of_prefix; + char *buf_end; - assert(buf_len >= 2); /* prevent integer underflow */ + assert(buf_len >= 16); /* prevent integer underflow and general stupidity */ buf_len -= 2; /* subtract 2 characters so we have room for \n\0 */ + buf_end = buf+buf_len; /* point *after* the last char we can write to */ n = _log_prefix(buf, buf_len, severity); end_of_prefix = buf+n; + if (log_domains_are_logged) { + char *cp = buf+n; + if (cp == buf_end) goto format_msg_no_room_for_domains; + *cp++ = '{'; + if (cp == buf_end) goto format_msg_no_room_for_domains; + cp = domain_to_string(domain, cp, (buf+buf_len-cp)); + if (cp == buf_end) goto format_msg_no_room_for_domains; + *cp++ = '}'; + if (cp == buf_end) goto format_msg_no_room_for_domains; + *cp++ = ' '; + if (cp == buf_end) goto format_msg_no_room_for_domains; + end_of_prefix = cp; + n = cp-buf; + format_msg_no_room_for_domains: + /* This will leave end_of_prefix and n unchanged, and thus cause + * whatever log domain string we had written to be clobbered. */ + ; + } + if (funcname && should_log_function_name(domain, severity)) { r = tor_snprintf(buf+n, buf_len-n, "%s(): ", funcname); if (r<0) @@ -263,6 +305,7 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, int formatted = 0; logfile_t *lf; char *end_of_prefix=NULL; + int callbacks_deferred = 0; /* Call assert, not tor_assert, since tor_assert calls log on failure. */ assert(format); @@ -270,6 +313,10 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, * interesting and hard to diagnose effects */ assert(severity >= LOG_ERR && severity <= LOG_DEBUG); LOCK_LOGS(); + + if ((! (domain & LD_NOCB)) && smartlist_len(pending_cb_messages)) + flush_pending_log_callbacks(); + lf = logfiles; while (lf) { if (! (lf->severities->masks[SEVERITY_MASK_IDX(severity)] & domain)) { @@ -280,10 +327,6 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, lf = lf->next; continue; } - if (lf->callback && (domain & LD_NOCB)) { - lf = lf->next; - continue; - } if (lf->seems_dead) { lf = lf->next; continue; @@ -295,6 +338,7 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, &msg_len); formatted = 1; } + if (lf->is_syslog) { #ifdef HAVE_SYSLOG_H char *m = end_of_prefix; @@ -318,7 +362,19 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, lf = lf->next; continue; } else if (lf->callback) { - lf->callback(severity, domain, end_of_prefix); + if (domain & LD_NOCB) { + if (!callbacks_deferred && pending_cb_messages) { + pending_cb_message_t *msg = tor_malloc(sizeof(pending_cb_message_t)); + msg->severity = severity; + msg->domain = domain; + msg->msg = tor_strdup(end_of_prefix); + smartlist_add(pending_cb_messages, msg); + + callbacks_deferred = 1; + } + } else { + lf->callback(severity, domain, end_of_prefix); + } lf = lf->next; continue; } @@ -332,9 +388,12 @@ logv(int severity, log_domain_mask_t domain, const char *funcname, UNLOCK_LOGS(); } -/** Output a message to the log. */ +/** Output a message to the log. It gets logged to all logfiles that + * care about messages with <b>severity</b> in <b>domain</b>. The content + * is formatted printf style basedc on <b>format</b> and extra arguments. + * */ void -_log(int severity, log_domain_mask_t domain, const char *format, ...) +tor_log(int severity, log_domain_mask_t domain, const char *format, ...) { va_list ap; if (severity > _log_global_min_severity) @@ -346,6 +405,9 @@ _log(int severity, log_domain_mask_t domain, const char *format, ...) /** Output a message to the log, prefixed with a function name <b>fn</b>. */ #ifdef __GNUC__ +/** GCC-based implementation of the log_fn backend, used when we have + * variadic macros. All arguments are as for log_fn, except for + * <b>fn</b>, which is the name of the calling functions. */ void _log_fn(int severity, log_domain_mask_t domain, const char *fn, const char *format, ...) @@ -358,6 +420,11 @@ _log_fn(int severity, log_domain_mask_t domain, const char *fn, va_end(ap); } #else +/** @{ */ +/** Variant implementation of log_fn, log_debug, log_info,... for C compilers + * without variadic macros. In this case, the calling function sets + * _log_fn_function_name to the name of the function, then invokes the + * appropriate _log_fn, _log_debug, etc. */ const char *_log_fn_function_name=NULL; void _log_fn(int severity, log_domain_mask_t domain, const char *format, ...) @@ -426,12 +493,15 @@ _log_err(log_domain_mask_t domain, const char *format, ...) va_end(ap); _log_fn_function_name = NULL; } +/** @} */ #endif /** Free all storage held by <b>victim</b>. */ static void log_free(logfile_t *victim) { + if (!victim) + return; tor_free(victim->severities); tor_free(victim->filename); tor_free(victim); @@ -442,9 +512,12 @@ void logs_free_all(void) { logfile_t *victim, *next; + smartlist_t *messages; LOCK_LOGS(); next = logfiles; logfiles = NULL; + messages = pending_cb_messages; + pending_cb_messages = NULL; UNLOCK_LOGS(); while (next) { victim = next; @@ -454,6 +527,12 @@ logs_free_all(void) } tor_free(appname); + SMARTLIST_FOREACH(messages, pending_cb_message_t *, msg, { + tor_free(msg->msg); + tor_free(msg); + }); + smartlist_free(messages); + /* We _could_ destroy the log mutex here, but that would screw up any logs * that happened between here and the end of execution. */ } @@ -539,8 +618,7 @@ add_stream_log_impl(const log_severity_list_t *severity, * to <b>fd</b>. Steals a reference to <b>severity</b>; the caller must * not use it after calling this function. */ void -add_stream_log(const log_severity_list_t *severity, - const char *name, int fd) +add_stream_log(const log_severity_list_t *severity, const char *name, int fd) { LOCK_LOGS(); add_stream_log_impl(severity, name, fd); @@ -555,6 +633,18 @@ init_logging(void) tor_mutex_init(&log_mutex); log_mutex_initialized = 1; } + if (pending_cb_messages == NULL) + pending_cb_messages = smartlist_create(); +} + +/** Set whether we report logging domains as a part of our log messages. + */ +void +logs_set_domain_logging(int enabled) +{ + LOCK_LOGS(); + log_domains_are_logged = enabled; + UNLOCK_LOGS(); } /** Add a log handler to receive messages during startup (before the real @@ -613,6 +703,48 @@ change_callback_log_severity(int loglevelMin, int loglevelMax, UNLOCK_LOGS(); } +/** If there are any log messages that were genered with LD_NOCB waiting to + * be sent to callback-based loggers, send them now. */ +void +flush_pending_log_callbacks(void) +{ + logfile_t *lf; + smartlist_t *messages, *messages_tmp; + + LOCK_LOGS(); + if (0 == smartlist_len(pending_cb_messages)) { + UNLOCK_LOGS(); + return; + } + + messages = pending_cb_messages; + pending_cb_messages = smartlist_create(); + do { + SMARTLIST_FOREACH_BEGIN(messages, pending_cb_message_t *, msg) { + const int severity = msg->severity; + const int domain = msg->domain; + for (lf = logfiles; lf; lf = lf->next) { + if (! lf->callback || lf->seems_dead || + ! (lf->severities->masks[SEVERITY_MASK_IDX(severity)] & domain)) { + continue; + } + lf->callback(severity, domain, msg->msg); + } + tor_free(msg->msg); + tor_free(msg); + } SMARTLIST_FOREACH_END(msg); + smartlist_clear(messages); + + messages_tmp = pending_cb_messages; + pending_cb_messages = messages; + messages = messages_tmp; + } while (smartlist_len(messages)); + + smartlist_free(messages); + + UNLOCK_LOGS(); +} + /** Close any log handlers added by add_temp_log() or marked by * mark_logs_temp(). */ void @@ -682,13 +814,11 @@ add_file_log(const log_severity_list_t *severity, const char *filename) logfiles->needs_close = 1; lf = logfiles; _log_global_min_severity = get_min_log_level(); - UNLOCK_LOGS(); if (log_tor_version(lf, 0) < 0) { - LOCK_LOGS(); delete_log(lf); - UNLOCK_LOGS(); } + UNLOCK_LOGS(); return 0; } @@ -709,7 +839,6 @@ add_syslog_log(const log_severity_list_t *severity) lf->fd = -1; lf->severities = tor_memdup(severity, sizeof(log_severity_list_t)); lf->filename = tor_strdup("<syslog>"); - lf->is_syslog = 1; LOCK_LOGS(); @@ -751,7 +880,7 @@ log_level_to_string(int level) static const char *domain_list[] = { "GENERAL", "CRYPTO", "NET", "CONFIG", "FS", "PROTOCOL", "MM", "HTTP", "APP", "CONTROL", "CIRC", "REND", "BUG", "DIR", "DIRSERV", - "OR", "EDGE", "ACCT", "HIST", NULL + "OR", "EDGE", "ACCT", "HIST", "HANDSHAKE", NULL }; /** Return a bitmask for the log domain for which <b>domain</b> is the name, @@ -766,18 +895,41 @@ parse_log_domain(const char *domain) } return 0; } -#if 0 -/** Translate a bitmask of log domains to a string, or NULL if the bitmask - * is undecodable. */ -static const char * -domain_to_string(log_domain_mask_t domain) + +/** Translate a bitmask of log domains to a string. */ +static char * +domain_to_string(log_domain_mask_t domain, char *buf, size_t buflen) { - int bit = tor_log2(domain); - if ((bit == 0 && domain == 0) || bit >= N_LOGGING_DOMAINS) - return NULL; - return domain_list[bit]; + char *cp = buf; + char *eos = buf+buflen; + + buf[0] = '\0'; + if (! domain) + return buf; + while (1) { + const char *d; + int bit = tor_log2(domain); + size_t n; + if (bit >= N_LOGGING_DOMAINS) { + tor_snprintf(buf, buflen, "<BUG:Unknown domain %lx>", (long)domain); + return buf+strlen(buf); + } + d = domain_list[bit]; + n = strlcpy(cp, d, eos-cp); + if (n >= buflen) { + tor_snprintf(buf, buflen, "<BUG:Truncating domain %lx>", (long)domain); + return buf+strlen(buf); + } + cp += n; + domain &= ~(1<<bit); + + if (domain == 0 || (eos-cp) < 2) + return cp; + + memcpy(cp, ",", 2); /*Nul-terminated ,"*/ + cp++; + } } -#endif /** Parse a log severity pattern in *<b>cfg_ptr</b>. Advance cfg_ptr after * the end of the severityPattern. Set the value of <b>severity_out</b> to @@ -853,7 +1005,10 @@ parse_log_severity_config(const char **cfg_ptr, smartlist_free(domains_list); if (err) return -1; - domains &= ~neg_domains; + if (domains == 0 && neg_domains) + domains = ~neg_domains; + else + domains &= ~neg_domains; cfg = eat_whitespace(closebracket+1); } else { ++got_an_unqualified_range; @@ -929,65 +1084,6 @@ switch_logs_debug(void) UNLOCK_LOGS(); } -#ifdef HAVE_EVENT_SET_LOG_CALLBACK -/** A string which, if it appears in a libevent log, should be ignored. */ -static const char *suppress_msg = NULL; -/** Callback function passed to event_set_log() so we can intercept - * log messages from libevent. */ -static void -libevent_logging_callback(int severity, const char *msg) -{ - char buf[1024]; - size_t n; - if (suppress_msg && strstr(msg, suppress_msg)) - return; - n = strlcpy(buf, msg, sizeof(buf)); - if (n && n < sizeof(buf) && buf[n-1] == '\n') { - buf[n-1] = '\0'; - } - switch (severity) { - case _EVENT_LOG_DEBUG: - log(LOG_DEBUG, LD_NOCB|LD_NET, "Message from libevent: %s", buf); - break; - case _EVENT_LOG_MSG: - log(LOG_INFO, LD_NOCB|LD_NET, "Message from libevent: %s", buf); - break; - case _EVENT_LOG_WARN: - log(LOG_WARN, LD_NOCB|LD_GENERAL, "Warning from libevent: %s", buf); - break; - case _EVENT_LOG_ERR: - log(LOG_ERR, LD_NOCB|LD_GENERAL, "Error from libevent: %s", buf); - break; - default: - log(LOG_WARN, LD_NOCB|LD_GENERAL, "Message [%d] from libevent: %s", - severity, buf); - break; - } -} -/** Set hook to intercept log messages from libevent. */ -void -configure_libevent_logging(void) -{ - event_set_log_callback(libevent_logging_callback); -} -/** Ignore any libevent log message that contains <b>msg</b>. */ -void -suppress_libevent_log_msg(const char *msg) -{ - suppress_msg = msg; -} -#else -void -configure_libevent_logging(void) -{ -} -void -suppress_libevent_log_msg(const char *msg) -{ - (void)msg; -} -#endif - #if 0 static void dump_log_info(logfile_t *lf) diff --git a/src/common/memarea.c b/src/common/memarea.c index ac26c5fd9..a6b8c4ee9 100644 --- a/src/common/memarea.c +++ b/src/common/memarea.c @@ -11,7 +11,11 @@ #include "memarea.h" #include "util.h" #include "compat.h" -#include "log.h" +#include "torlog.h" + +/** If true, we try to detect any attempts to write beyond the length of a + * memarea. */ +#define USE_SENTINELS /** All returned pointers should be aligned to the nearest multiple of this * value. */ @@ -25,13 +29,39 @@ #error "void* is neither 4 nor 8 bytes long. I don't know how to align stuff." #endif +#ifdef USE_SENTINELS +/** Magic value that we stick at the end of a memarea so we can make sure + * there are no run-off-the-end bugs. */ +#define SENTINEL_VAL 0x90806622u +/** How many bytes per area do we devote to the sentinel? */ +#define SENTINEL_LEN sizeof(uint32_t) +/** Given a mem_area_chunk_t with SENTINEL_LEN extra bytes allocated at the + * end, set those bytes. */ +#define SET_SENTINEL(chunk) \ + STMT_BEGIN \ + set_uint32( &(chunk)->u.mem[chunk->mem_size], SENTINEL_VAL ); \ + STMT_END +/** Assert that the sentinel on a memarea is set correctly. */ +#define CHECK_SENTINEL(chunk) \ + STMT_BEGIN \ + uint32_t sent_val = get_uint32(&(chunk)->u.mem[chunk->mem_size]); \ + tor_assert(sent_val == SENTINEL_VAL); \ + STMT_END +#else +#define SENTINEL_LEN 0 +#define SET_SENTINEL(chunk) STMT_NIL +#define CHECK_SENTINEL(chunk) STMT_NIL +#endif + /** Increment <b>ptr</b> until it is aligned to MEMAREA_ALIGN. */ static INLINE void * realign_pointer(void *ptr) { uintptr_t x = (uintptr_t)ptr; x = (x+MEMAREA_ALIGN_MASK) & ~MEMAREA_ALIGN_MASK; - tor_assert(((void*)x) >= ptr); // XXXX021 remove this once bug 930 is solved + /* Reinstate this if bug 930 ever reappears + tor_assert(((void*)x) >= ptr); + */ return (void*)x; } @@ -51,8 +81,11 @@ typedef struct memarea_chunk_t { } u; } memarea_chunk_t; +/** How many bytes are needed for overhead before we get to the memory part + * of a chunk? */ #define CHUNK_HEADER_SIZE STRUCT_OFFSET(memarea_chunk_t, u) +/** What's the smallest that we'll allocate a chunk? */ #define CHUNK_SIZE 4096 /** A memarea_t is an allocation region for a set of small memory requests @@ -79,15 +112,20 @@ alloc_chunk(size_t sz, int freelist_ok) freelist = res->next_chunk; res->next_chunk = NULL; --freelist_len; + CHECK_SENTINEL(res); return res; } else { size_t chunk_size = freelist_ok ? CHUNK_SIZE : sz; - memarea_chunk_t *res = tor_malloc_roundup(&chunk_size); + memarea_chunk_t *res; + chunk_size += SENTINEL_LEN; + res = tor_malloc_roundup(&chunk_size); res->next_chunk = NULL; - res->mem_size = chunk_size - CHUNK_HEADER_SIZE; + res->mem_size = chunk_size - CHUNK_HEADER_SIZE - SENTINEL_LEN; res->next_mem = res->u.mem; - tor_assert(res->next_mem+res->mem_size == ((char*)res)+chunk_size); + tor_assert(res->next_mem+res->mem_size+SENTINEL_LEN == + ((char*)res)+chunk_size); tor_assert(realign_pointer(res->next_mem) == res->next_mem); + SET_SENTINEL(res); return res; } } @@ -95,8 +133,9 @@ alloc_chunk(size_t sz, int freelist_ok) /** Release <b>chunk</b> from a memarea, either by adding it to the freelist * or by freeing it if the freelist is already too big. */ static void -chunk_free(memarea_chunk_t *chunk) +chunk_free_unchecked(memarea_chunk_t *chunk) { + CHECK_SENTINEL(chunk); if (freelist_len < MAX_FREELIST_LEN) { ++freelist_len; chunk->next_chunk = freelist; @@ -124,7 +163,7 @@ memarea_drop_all(memarea_t *area) memarea_chunk_t *chunk, *next; for (chunk = area->first; chunk; chunk = next) { next = chunk->next_chunk; - chunk_free(chunk); + chunk_free_unchecked(chunk); } area->first = NULL; /*fail fast on */ tor_free(area); @@ -140,7 +179,7 @@ memarea_clear(memarea_t *area) if (area->first->next_chunk) { for (chunk = area->first->next_chunk; chunk; chunk = next) { next = chunk->next_chunk; - chunk_free(chunk); + chunk_free_unchecked(chunk); } area->first->next_chunk = NULL; } @@ -183,6 +222,7 @@ memarea_alloc(memarea_t *area, size_t sz) memarea_chunk_t *chunk = area->first; char *result; tor_assert(chunk); + CHECK_SENTINEL(chunk); tor_assert(sz < SIZE_T_CEILING); if (sz == 0) sz = 1; @@ -203,9 +243,10 @@ memarea_alloc(memarea_t *area, size_t sz) } result = chunk->next_mem; chunk->next_mem = chunk->next_mem + sz; - // XXXX021 remove these once bug 930 is solved. + /* Reinstate these if bug 930 ever comes back tor_assert(chunk->next_mem >= chunk->u.mem); tor_assert(chunk->next_mem <= chunk->u.mem+chunk->mem_size); + */ chunk->next_mem = realign_pointer(chunk->next_mem); return result; } @@ -261,6 +302,7 @@ memarea_get_stats(memarea_t *area, size_t *allocated_out, size_t *used_out) size_t a = 0, u = 0; memarea_chunk_t *chunk; for (chunk = area->first; chunk; chunk = chunk->next_chunk) { + CHECK_SENTINEL(chunk); a += CHUNK_HEADER_SIZE + chunk->mem_size; tor_assert(chunk->next_mem >= chunk->u.mem); u += CHUNK_HEADER_SIZE + (chunk->next_mem - chunk->u.mem); @@ -277,6 +319,7 @@ memarea_assert_ok(memarea_t *area) tor_assert(area->first); for (chunk = area->first; chunk; chunk = chunk->next_chunk) { + CHECK_SENTINEL(chunk); tor_assert(chunk->next_mem >= chunk->u.mem); tor_assert(chunk->next_mem <= (char*) realign_pointer(chunk->u.mem+chunk->mem_size)); diff --git a/src/common/mempool.c b/src/common/mempool.c index 9538a0548..c44492318 100644 --- a/src/common/mempool.c +++ b/src/common/mempool.c @@ -65,7 +65,7 @@ #include "orconfig.h" #include "util.h" #include "compat.h" -#include "log.h" +#include "torlog.h" #define ALLOC(x) tor_malloc(x) #define FREE(x) tor_free(x) #define ASSERT(x) tor_assert(x) diff --git a/src/common/sha256.c b/src/common/sha256.c new file mode 100644 index 000000000..258b7e062 --- /dev/null +++ b/src/common/sha256.c @@ -0,0 +1,331 @@ +/* Copyright (c) 2009-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ +/* This SHA256 implementation is adapted from the public domain one in + LibTomCrypt, version 1.6. Tor uses it on platforms where OpenSSL doesn't + have a SHA256. */ + + +typedef struct sha256_state { + uint64_t length; + uint32_t state[8], curlen; + unsigned char buf[64]; +} sha256_state; + +#define CRYPT_OK 0 +#define CRYPT_NOP -1 +#define CRYPT_INVALID_ARG -2 + +#define LOAD32H(x,y) STMT_BEGIN x = ntohl(get_uint32((const char*)y)); STMT_END +#define STORE32H(x,y) STMT_BEGIN set_uint32((char*)y, htonl(x)); STMT_END +#define STORE64H(x,y) STMT_BEGIN \ + set_uint32((char*)y, htonl((uint32_t)((x)>>32))); \ + set_uint32(((char*)y)+4, htonl((uint32_t)((x)&0xffffffff))); \ + STMT_END +#define RORc(x, y) ( ((((unsigned long)(x)&0xFFFFFFFFUL)>>(unsigned long)((y)&31)) | ((unsigned long)(x)<<(unsigned long)(32-((y)&31)))) & 0xFFFFFFFFUL) +#ifndef MIN + #define MIN(x, y) ( ((x)<(y))?(x):(y) ) +#endif + + +/* LibTomCrypt, modular cryptographic library -- Tom St Denis + * + * LibTomCrypt is a library that provides various cryptographic + * algorithms in a highly modular and flexible manner. + * + * The library is free for all purposes without any express + * guarantee it works. + * + * Tom St Denis, tomstdenis@gmail.com, http://libtomcrypt.com + */ + +/** + @file sha256.c + SHA256 by Tom St Denis +*/ + + +#ifdef LTC_SMALL_CODE +/* the K array */ +static const uint32_t K[64] = { + 0x428a2f98UL, 0x71374491UL, 0xb5c0fbcfUL, 0xe9b5dba5UL, 0x3956c25bUL, + 0x59f111f1UL, 0x923f82a4UL, 0xab1c5ed5UL, 0xd807aa98UL, 0x12835b01UL, + 0x243185beUL, 0x550c7dc3UL, 0x72be5d74UL, 0x80deb1feUL, 0x9bdc06a7UL, + 0xc19bf174UL, 0xe49b69c1UL, 0xefbe4786UL, 0x0fc19dc6UL, 0x240ca1ccUL, + 0x2de92c6fUL, 0x4a7484aaUL, 0x5cb0a9dcUL, 0x76f988daUL, 0x983e5152UL, + 0xa831c66dUL, 0xb00327c8UL, 0xbf597fc7UL, 0xc6e00bf3UL, 0xd5a79147UL, + 0x06ca6351UL, 0x14292967UL, 0x27b70a85UL, 0x2e1b2138UL, 0x4d2c6dfcUL, + 0x53380d13UL, 0x650a7354UL, 0x766a0abbUL, 0x81c2c92eUL, 0x92722c85UL, + 0xa2bfe8a1UL, 0xa81a664bUL, 0xc24b8b70UL, 0xc76c51a3UL, 0xd192e819UL, + 0xd6990624UL, 0xf40e3585UL, 0x106aa070UL, 0x19a4c116UL, 0x1e376c08UL, + 0x2748774cUL, 0x34b0bcb5UL, 0x391c0cb3UL, 0x4ed8aa4aUL, 0x5b9cca4fUL, + 0x682e6ff3UL, 0x748f82eeUL, 0x78a5636fUL, 0x84c87814UL, 0x8cc70208UL, + 0x90befffaUL, 0xa4506cebUL, 0xbef9a3f7UL, 0xc67178f2UL +}; +#endif + +/* Various logical functions */ +#define Ch(x,y,z) (z ^ (x & (y ^ z))) +#define Maj(x,y,z) (((x | y) & z) | (x & y)) +#define S(x, n) RORc((x),(n)) +#define R(x, n) (((x)&0xFFFFFFFFUL)>>(n)) +#define Sigma0(x) (S(x, 2) ^ S(x, 13) ^ S(x, 22)) +#define Sigma1(x) (S(x, 6) ^ S(x, 11) ^ S(x, 25)) +#define Gamma0(x) (S(x, 7) ^ S(x, 18) ^ R(x, 3)) +#define Gamma1(x) (S(x, 17) ^ S(x, 19) ^ R(x, 10)) + +/* compress 512-bits */ +#ifdef LTC_CLEAN_STACK +static int _sha256_compress(sha256_state * md, unsigned char *buf) +#else +static int sha256_compress(sha256_state * md, unsigned char *buf) +#endif +{ + uint32_t S[8], W[64], t0, t1; +#ifdef LTC_SMALL_CODE + uint32_t t; +#endif + int i; + + /* copy state into S */ + for (i = 0; i < 8; i++) { + S[i] = md->state[i]; + } + + /* copy the state into 512-bits into W[0..15] */ + for (i = 0; i < 16; i++) { + LOAD32H(W[i], buf + (4*i)); + } + + /* fill W[16..63] */ + for (i = 16; i < 64; i++) { + W[i] = Gamma1(W[i - 2]) + W[i - 7] + Gamma0(W[i - 15]) + W[i - 16]; + } + + /* Compress */ +#ifdef LTC_SMALL_CODE +#define RND(a,b,c,d,e,f,g,h,i) \ + t0 = h + Sigma1(e) + Ch(e, f, g) + K[i] + W[i]; \ + t1 = Sigma0(a) + Maj(a, b, c); \ + d += t0; \ + h = t0 + t1; + + for (i = 0; i < 64; ++i) { + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],i); + t = S[7]; S[7] = S[6]; S[6] = S[5]; S[5] = S[4]; + S[4] = S[3]; S[3] = S[2]; S[2] = S[1]; S[1] = S[0]; S[0] = t; + } +#else +#define RND(a,b,c,d,e,f,g,h,i,ki) \ + t0 = h + Sigma1(e) + Ch(e, f, g) + ki + W[i]; \ + t1 = Sigma0(a) + Maj(a, b, c); \ + d += t0; \ + h = t0 + t1; + + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],0,0x428a2f98); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],1,0x71374491); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],2,0xb5c0fbcf); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],3,0xe9b5dba5); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],4,0x3956c25b); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],5,0x59f111f1); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],6,0x923f82a4); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],7,0xab1c5ed5); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],8,0xd807aa98); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],9,0x12835b01); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],10,0x243185be); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],11,0x550c7dc3); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],12,0x72be5d74); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],13,0x80deb1fe); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],14,0x9bdc06a7); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],15,0xc19bf174); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],16,0xe49b69c1); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],17,0xefbe4786); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],18,0x0fc19dc6); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],19,0x240ca1cc); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],20,0x2de92c6f); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],21,0x4a7484aa); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],22,0x5cb0a9dc); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],23,0x76f988da); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],24,0x983e5152); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],25,0xa831c66d); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],26,0xb00327c8); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],27,0xbf597fc7); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],28,0xc6e00bf3); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],29,0xd5a79147); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],30,0x06ca6351); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],31,0x14292967); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],32,0x27b70a85); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],33,0x2e1b2138); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],34,0x4d2c6dfc); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],35,0x53380d13); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],36,0x650a7354); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],37,0x766a0abb); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],38,0x81c2c92e); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],39,0x92722c85); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],40,0xa2bfe8a1); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],41,0xa81a664b); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],42,0xc24b8b70); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],43,0xc76c51a3); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],44,0xd192e819); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],45,0xd6990624); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],46,0xf40e3585); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],47,0x106aa070); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],48,0x19a4c116); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],49,0x1e376c08); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],50,0x2748774c); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],51,0x34b0bcb5); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],52,0x391c0cb3); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],53,0x4ed8aa4a); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],54,0x5b9cca4f); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],55,0x682e6ff3); + RND(S[0],S[1],S[2],S[3],S[4],S[5],S[6],S[7],56,0x748f82ee); + RND(S[7],S[0],S[1],S[2],S[3],S[4],S[5],S[6],57,0x78a5636f); + RND(S[6],S[7],S[0],S[1],S[2],S[3],S[4],S[5],58,0x84c87814); + RND(S[5],S[6],S[7],S[0],S[1],S[2],S[3],S[4],59,0x8cc70208); + RND(S[4],S[5],S[6],S[7],S[0],S[1],S[2],S[3],60,0x90befffa); + RND(S[3],S[4],S[5],S[6],S[7],S[0],S[1],S[2],61,0xa4506ceb); + RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],62,0xbef9a3f7); + RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],63,0xc67178f2); + +#undef RND + +#endif + + /* feedback */ + for (i = 0; i < 8; i++) { + md->state[i] = md->state[i] + S[i]; + } + return CRYPT_OK; +} + +#ifdef LTC_CLEAN_STACK +static int sha256_compress(sha256_state * md, unsigned char *buf) +{ + int err; + err = _sha256_compress(md, buf); + burn_stack(sizeof(uint32_t) * 74); + return err; +} +#endif + +/** + Initialize the hash state + @param md The hash state you wish to initialize + @return CRYPT_OK if successful +*/ +static int sha256_init(sha256_state * md) +{ + LTC_ARGCHK(md != NULL); + + md->curlen = 0; + md->length = 0; + md->state[0] = 0x6A09E667UL; + md->state[1] = 0xBB67AE85UL; + md->state[2] = 0x3C6EF372UL; + md->state[3] = 0xA54FF53AUL; + md->state[4] = 0x510E527FUL; + md->state[5] = 0x9B05688CUL; + md->state[6] = 0x1F83D9ABUL; + md->state[7] = 0x5BE0CD19UL; + return CRYPT_OK; +} + +/** + Process a block of memory though the hash + @param md The hash state + @param in The data to hash + @param inlen The length of the data (octets) + @return CRYPT_OK if successful +*/ +static int sha256_process (sha256_state * md, const unsigned char *in, unsigned long inlen) +{ + unsigned long n; + int err; + LTC_ARGCHK(md != NULL); + LTC_ARGCHK(in != NULL); + if (md->curlen > sizeof(md->buf)) { + return CRYPT_INVALID_ARG; + } + while (inlen > 0) { + if (md->curlen == 0 && inlen >= 64) { + if ((err = sha256_compress (md, (unsigned char *)in)) != CRYPT_OK) { + return err; + } + md->length += 64 * 8; + in += 64; + inlen -= 64; + } else { + n = MIN(inlen, (64 - md->curlen)); + memcpy(md->buf + md->curlen, in, (size_t)n); + md->curlen += n; + in += n; + inlen -= n; + if (md->curlen == 64) { + if ((err = sha256_compress (md, md->buf)) != CRYPT_OK) { + return err; + } + md->length += 8*64; + md->curlen = 0; + } + } + } + return CRYPT_OK; +} + +/** + Terminate the hash to get the digest + @param md The hash state + @param out [out] The destination of the hash (32 bytes) + @return CRYPT_OK if successful +*/ +static int sha256_done(sha256_state * md, unsigned char *out) +{ + int i; + + LTC_ARGCHK(md != NULL); + LTC_ARGCHK(out != NULL); + + if (md->curlen >= sizeof(md->buf)) { + return CRYPT_INVALID_ARG; + } + + + /* increase the length of the message */ + md->length += md->curlen * 8; + + /* append the '1' bit */ + md->buf[md->curlen++] = (unsigned char)0x80; + + /* if the length is currently above 56 bytes we append zeros + * then compress. Then we can fall back to padding zeros and length + * encoding like normal. + */ + if (md->curlen > 56) { + while (md->curlen < 64) { + md->buf[md->curlen++] = (unsigned char)0; + } + sha256_compress(md, md->buf); + md->curlen = 0; + } + + /* pad upto 56 bytes of zeroes */ + while (md->curlen < 56) { + md->buf[md->curlen++] = (unsigned char)0; + } + + /* store length */ + STORE64H(md->length, md->buf+56); + sha256_compress(md, md->buf); + + /* copy output */ + for (i = 0; i < 8; i++) { + STORE32H(md->state[i], out+(4*i)); + } +#ifdef LTC_CLEAN_STACK + zeromem(md, sizeof(sha256_state)); +#endif + return CRYPT_OK; +} + +/* $Source: /cvs/libtom/libtomcrypt/src/hashes/sha2/sha256.c,v $ */ +/* $Revision: 1.9 $ */ +/* $Date: 2006/11/01 09:28:17 $ */ diff --git a/src/common/test.h b/src/common/test.h deleted file mode 100644 index c5995d145..000000000 --- a/src/common/test.h +++ /dev/null @@ -1,184 +0,0 @@ -/* Copyright (c) 2001-2003, Roger Dingledine. - * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ -/* See LICENSE for licensing information */ - -#ifndef _TOR_TEST_H -#define _TOR_TEST_H - -/** - * \file test.h - * \brief Macros used by unit tests. - */ - -#include "compat.h" - -#ifdef __GNUC__ -#define PRETTY_FUNCTION __PRETTY_FUNCTION__ -#else -#define PRETTY_FUNCTION "" -#endif - -#define test_fail_msg(msg) \ - STMT_BEGIN \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): %s", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - msg); \ - goto done; \ - STMT_END - -#define test_fail() test_fail_msg("Assertion failed.") - -#define test_assert(expr) \ - STMT_BEGIN \ - if (expr) { printf("."); fflush(stdout); } else { \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): assertion failed: (%s)\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr); \ - goto done; \ - } STMT_END - -#define test_eq_type(tp, fmt, expr1, expr2) \ - STMT_BEGIN \ - tp _test_v1=(tp)(expr1); \ - tp _test_v2=(tp)(expr2); \ - if (_test_v1==_test_v2) { printf("."); fflush(stdout); } else { \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s==%s)\n" \ - " "fmt "!="fmt"\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, #expr2, \ - _test_v1, _test_v2); \ - goto done; \ - } STMT_END - -#define test_eq(expr1, expr2) \ - test_eq_type(long, "%ld", expr1, expr2) - -#define test_eq_ptr(expr1, expr2) \ - test_eq_type(void*, "%p", expr1, expr2) - -#define test_neq_type(tp, fmt, expr1, expr2) \ - STMT_BEGIN \ - tp _test_v1=(tp)(expr1); \ - tp _test_v2=(tp)(expr2); \ - if (_test_v1!=_test_v2) { printf("."); fflush(stdout); } else { \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s!=%s)\n" \ - " ("fmt" == "fmt")\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, #expr2, \ - _test_v1, _test_v2); \ - goto done; \ - } STMT_END - -#define test_neq(expr1, expr2) \ - test_neq_type(long, "%ld", expr1, expr2) - -#define test_neq_ptr(expr1, expr2) \ - test_neq_type(void *, "%p", expr1, expr2) - -#define test_streq(expr1, expr2) \ - STMT_BEGIN \ - const char *_test_v1=(expr1), *_test_v2=(expr2); \ - if (!strcmp(_test_v1,_test_v2)) { printf("."); fflush(stdout); } else { \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s==%s)\n"\ - " (\"%s\" != \"%s\")\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, #expr2, \ - _test_v1, _test_v2); \ - goto done; \ - } STMT_END - -#define test_strneq(expr1, expr2) \ - STMT_BEGIN \ - const char *_test_v1=(expr1), *_test_v2=(expr2); \ - if (strcmp(_test_v1,_test_v2)) { printf("."); fflush(stdout); } else { \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s!=%s)\n"\ - " (\"%s\" == \"%s\")\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, #expr2, \ - _test_v1, _test_v2); \ - goto done; \ - } STMT_END - -#define test_memeq(expr1, expr2, len) \ - STMT_BEGIN \ - const void *_test_v1=(expr1), *_test_v2=(expr2); \ - char *mem1, *mem2; \ - if (!memcmp(_test_v1,_test_v2,(len))) { \ - printf("."); fflush(stdout); } else { \ - have_failed = 1; \ - mem1 = tor_malloc(len*2+1); \ - mem2 = tor_malloc(len*2+1); \ - base16_encode(mem1, len*2+1, _test_v1, len); \ - base16_encode(mem2, len*2+1, _test_v2, len); \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s==%s)\n" \ - " %s != %s\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, #expr2, mem1, mem2); \ - tor_free(mem1); \ - tor_free(mem2); \ - goto done; \ - } STMT_END - -#define test_memeq_hex(expr1, hex) \ - STMT_BEGIN \ - const char *_test_v1 = (char*)(expr1); \ - const char *_test_v2 = (hex); \ - size_t _len_v2 = strlen(_test_v2); \ - char *_mem2 = tor_malloc(_len_v2/2); \ - tor_assert((_len_v2 & 1) == 0); \ - base16_decode(_mem2, _len_v2/2, _test_v2, _len_v2); \ - if (!memcmp(_mem2, _test_v1, _len_v2/2)) { \ - printf("."); fflush(stdout); } else { \ - char *_mem1 = tor_malloc(_len_v2+1); \ - base16_encode(_mem1, _len_v2+1, _test_v1, _len_v2/2); \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s==%s)\n" \ - " %s != %s\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, _test_v2, _mem1, _test_v2); \ - tor_free(_mem1); \ - tor_free(_mem2); \ - goto done; \ - } \ - tor_free(_mem2); \ - STMT_END - -#define test_memneq(expr1, expr2, len) \ - STMT_BEGIN \ - void *_test_v1=(expr1), *_test_v2=(expr2); \ - if (memcmp(_test_v1,_test_v2,(len))) { \ - printf("."); fflush(stdout); \ - } else { \ - have_failed = 1; \ - printf("\nFile %s: line %d (%s): Assertion failed: (%s!=%s)\n", \ - _SHORT_FILE_, \ - __LINE__, \ - PRETTY_FUNCTION, \ - #expr1, #expr2); \ - goto done; \ - } STMT_END - -#endif - diff --git a/src/common/torgzip.c b/src/common/torgzip.c index 249151cc9..e9079e0ba 100644 --- a/src/common/torgzip.c +++ b/src/common/torgzip.c @@ -13,20 +13,42 @@ #include <stdlib.h> #include <stdio.h> #include <assert.h> -#ifdef _MSC_VER -#include "..\..\contrib\zlib\zlib.h" -#else -#include <zlib.h> -#endif #include <string.h> #ifdef HAVE_NETINET_IN_H #include <netinet/in.h> #endif +#include "torint.h" #include "util.h" -#include "log.h" +#include "torlog.h" #include "torgzip.h" +/* zlib 1.2.4 and 1.2.5 do some "clever" things with macros. Instead of + saying "(defined(FOO) ? FOO : 0)" they like to say "FOO-0", on the theory + that nobody will care if the compile outputs a no-such-identifier warning. + + Sorry, but we like -Werror over here, so I guess we need to define these. + I hope that zlib 1.2.6 doesn't break these too. +*/ +#ifndef _LARGEFILE64_SOURCE +#define _LARGEFILE64_SOURCE 0 +#endif +#ifndef _LFS64_LARGEFILE +#define _LFS64_LARGEFILE 0 +#endif +#ifndef _FILE_OFFSET_BITS +#define _FILE_OFFSET_BITS 0 +#endif +#ifndef off64_t +#define off64_t int64_t +#endif + +#ifdef _MSC_VER +#include "..\..\contrib\zlib\zlib.h" +#else +#include <zlib.h> +#endif + /** Set to 1 if zlib is a version that supports gzip; set to 0 if it doesn't; * set to -1 if we haven't checked yet. */ static int gzip_is_supported = -1; @@ -57,6 +79,7 @@ method_bits(compress_method_t method) return method == GZIP_METHOD ? 15+16 : 15; } +/** @{ */ /* These macros define the maximum allowable compression factor. Anything of * size greater than CHECK_FOR_COMPRESSION_BOMB_AFTER is not allowed to * have an uncompression factor (uncompressed size:compressed size ratio) of @@ -72,6 +95,7 @@ method_bits(compress_method_t method) */ #define MAX_UNCOMPRESSION_FACTOR 25 #define CHECK_FOR_COMPRESSION_BOMB_AFTER (1024*64) +/** @} */ /** Return true if uncompressing an input of size <b>in_size</b> to an input * of size at least <b>size_out</b> looks like a compression bomb. */ @@ -198,9 +222,7 @@ tor_gzip_compress(char **out, size_t *out_len, deflateEnd(stream); tor_free(stream); } - if (*out) { - tor_free(*out); - } + tor_free(*out); return -1; } @@ -369,12 +391,12 @@ detect_compression_method(const char *in, size_t in_len) /** Internal state for an incremental zlib compression/decompression. The * body of this struct is not exposed. */ struct tor_zlib_state_t { - struct z_stream_s stream; - int compress; + struct z_stream_s stream; /**< The zlib stream */ + int compress; /**< True if we are compressing; false if we are inflating */ - /* Number of bytes read so far. Used to detect zlib bombs. */ + /** Number of bytes read so far. Used to detect zlib bombs. */ size_t input_so_far; - /* Number of bytes written so far. Used to detect zlib bombs. */ + /** Number of bytes written so far. Used to detect zlib bombs. */ size_t output_so_far; }; @@ -479,7 +501,8 @@ tor_zlib_process(tor_zlib_state_t *state, void tor_zlib_free(tor_zlib_state_t *state) { - tor_assert(state); + if (!state) + return; if (state->compress) deflateEnd(&state->stream); diff --git a/src/common/torint.h b/src/common/torint.h index d48968465..0b5c29adc 100644 --- a/src/common/torint.h +++ b/src/common/torint.h @@ -117,11 +117,10 @@ typedef unsigned int uint32_t; #ifndef INT32_MAX #define INT32_MAX 0x7fffffff #endif -#endif - #ifndef INT32_MIN #define INT32_MIN (-2147483647-1) #endif +#endif #if (SIZEOF_LONG == 4) #ifndef HAVE_INT32_T @@ -330,9 +329,9 @@ typedef uint32_t uintptr_t; #endif #endif -/* Any ssize_t larger than this amount is likely to be an underflow. */ +/** Any ssize_t larger than this amount is likely to be an underflow. */ #define SSIZE_T_CEILING ((ssize_t)(SSIZE_T_MAX-16)) -/* Any size_t larger than this amount is likely to be an underflow. */ +/** Any size_t larger than this amount is likely to be an underflow. */ #define SIZE_T_CEILING ((size_t)(SSIZE_T_MAX-16)) #endif /* __TORINT_H */ diff --git a/src/common/log.h b/src/common/torlog.h index 2520ed163..000e32dda 100644 --- a/src/common/log.h +++ b/src/common/torlog.h @@ -5,7 +5,7 @@ /* See LICENSE for licensing information */ /** - * \file log.h + * \file torlog.h * * \brief Headers for log.c **/ @@ -90,14 +90,16 @@ #define LD_ACCT (1u<<17) /** Router history */ #define LD_HIST (1u<<18) - +/** OR handshaking */ +#define LD_HANDSHAKE (1u<<19) /** Number of logging domains in the code. */ -#define N_LOGGING_DOMAINS 19 +#define N_LOGGING_DOMAINS 20 -/** This log message is not safe to send to a callback-based logger. - * Used as a flag, not a log domain. */ +/** This log message is not safe to send to a callback-based logger + * immediately. Used as a flag, not a log domain. */ #define LD_NOCB (1u<<31) +/** Mask of zero or more log domains, OR'd together. */ typedef uint32_t log_domain_mask_t; /** Configures which severities are logged for each logging domain for a given @@ -131,6 +133,7 @@ int add_file_log(const log_severity_list_t *severity, const char *filename); int add_syslog_log(const log_severity_list_t *severity); #endif int add_callback_log(const log_severity_list_t *severity, log_callback cb); +void logs_set_domain_logging(int enabled); int get_min_log_level(void); void switch_logs_debug(void); void logs_free_all(void); @@ -138,16 +141,15 @@ void add_temp_log(int min_severity); void close_temp_logs(void); void rollback_log_changes(void); void mark_logs_temp(void); -void configure_libevent_logging(void); -void suppress_libevent_log_msg(const char *msg); void change_callback_log_severity(int loglevelMin, int loglevelMax, log_callback cb); +void flush_pending_log_callbacks(void); void log_set_application_name(const char *name); /* Outputs a message to stdout */ -void _log(int severity, log_domain_mask_t domain, const char *format, ...) +void tor_log(int severity, log_domain_mask_t domain, const char *format, ...) CHECK_PRINTF(3,4); -#define log _log /* hack it so we don't conflict with log() as much */ +#define log tor_log /* hack it so we don't conflict with log() as much */ #ifdef __GNUC__ extern int _log_global_min_severity; diff --git a/src/common/tortls.c b/src/common/tortls.c index 7735618ea..10f4440cb 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -16,6 +16,10 @@ #include "orconfig.h" +#if defined (WINCE) +#include <WinSock2.h> +#endif + #include <assert.h> #ifdef MS_WINDOWS /*wrkard for dtls1.h >= 0.9.8m of "#include <winsock.h>"*/ #define WIN32_WINNT 0x400 @@ -45,7 +49,7 @@ #include "crypto.h" #include "tortls.h" #include "util.h" -#include "log.h" +#include "torlog.h" #include "container.h" #include "ht.h" #include <string.h> @@ -82,7 +86,9 @@ static int use_unsafe_renegotiation_op = 0; * SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION? */ static int use_unsafe_renegotiation_flag = 0; -/** Structure holding the TLS state for a single connection. */ +/** Holds a SSL_CTX object and related state used to configure TLS + * connections. + */ typedef struct tor_tls_context_t { int refcnt; SSL_CTX *ctx; @@ -184,10 +190,16 @@ static X509* tor_tls_create_certificate(crypto_pk_env_t *rsa, const char *cname_sign, unsigned int lifetime); static void tor_tls_unblock_renegotiation(tor_tls_t *tls); - -/** Global tls context. We keep it here because nobody else needs to - * touch it. */ -static tor_tls_context_t *global_tls_context = NULL; +static int tor_tls_context_init_one(tor_tls_context_t **ppcontext, + crypto_pk_env_t *identity, + unsigned int key_lifetime); +static tor_tls_context_t *tor_tls_context_new(crypto_pk_env_t *identity, + unsigned int key_lifetime); + +/** Global TLS contexts. We keep them here because nobody else needs + * to touch them. */ +static tor_tls_context_t *server_tls_context = NULL; +static tor_tls_context_t *client_tls_context = NULL; /** True iff tor_tls_init() has been called. */ static int tls_library_is_initialized = 0; @@ -195,30 +207,51 @@ static int tls_library_is_initialized = 0; #define _TOR_TLS_SYSCALL (_MIN_TOR_TLS_ERROR_VAL - 2) #define _TOR_TLS_ZERORETURN (_MIN_TOR_TLS_ERROR_VAL - 1) +#include "tortls_states.h" + +/** Return the symbolic name of an OpenSSL state. */ +static const char * +ssl_state_to_string(int ssl_state) +{ + static char buf[40]; + int i; + for (i = 0; state_map[i].name; ++i) { + if (state_map[i].state == ssl_state) + return state_map[i].name; + } + tor_snprintf(buf, sizeof(buf), "Unknown state %d", ssl_state); + return buf; +} + /** Log all pending tls errors at level <b>severity</b>. Use * <b>doing</b> to describe our current activities. */ static void -tls_log_errors(tor_tls_t *tls, int severity, const char *doing) +tls_log_errors(tor_tls_t *tls, int severity, int domain, const char *doing) { + const char *state = NULL; + int st; unsigned long err; const char *msg, *lib, *func, *addr; addr = tls ? tls->address : NULL; + st = (tls && tls->ssl) ? tls->ssl->state : -1; while ((err = ERR_get_error()) != 0) { msg = (const char*)ERR_reason_error_string(err); lib = (const char*)ERR_lib_error_string(err); func = (const char*)ERR_func_error_string(err); + if (!state) + state = (st>=0)?ssl_state_to_string(st):"---"; if (!msg) msg = "(null)"; if (!lib) lib = "(null)"; if (!func) func = "(null)"; if (doing) { - log(severity, LD_NET, "TLS error while %s%s%s: %s (in %s:%s)", + log(severity, domain, "TLS error while %s%s%s: %s (in %s:%s:%s)", doing, addr?" with ":"", addr?addr:"", - msg, lib, func); + msg, lib, func, state); } else { - log(severity, LD_NET, "TLS error%s%s: %s (in %s:%s)", + log(severity, domain, "TLS error%s%s: %s (in %s:%s:%s)", addr?" with ":"", addr?addr:"", - msg, lib, func); + msg, lib, func, state); } } } @@ -294,7 +327,7 @@ tor_tls_err_to_string(int err) */ static int tor_tls_get_error(tor_tls_t *tls, int r, int extra, - const char *doing, int severity) + const char *doing, int severity, int domain) { int err = SSL_get_error(tls->ssl, r); int tor_error = TOR_TLS_ERROR_MISC; @@ -309,25 +342,28 @@ tor_tls_get_error(tor_tls_t *tls, int r, int extra, if (extra&CATCH_SYSCALL) return _TOR_TLS_SYSCALL; if (r == 0) { - log(severity, LD_NET, "TLS error: unexpected close while %s", doing); + log(severity, LD_NET, "TLS error: unexpected close while %s (%s)", + doing, ssl_state_to_string(tls->ssl->state)); tor_error = TOR_TLS_ERROR_IO; } else { int e = tor_socket_errno(tls->socket); log(severity, LD_NET, - "TLS error: <syscall error while %s> (errno=%d: %s)", - doing, e, tor_socket_strerror(e)); + "TLS error: <syscall error while %s> (errno=%d: %s; state=%s)", + doing, e, tor_socket_strerror(e), + ssl_state_to_string(tls->ssl->state)); tor_error = tor_errno_to_tls_error(e); } - tls_log_errors(tls, severity, doing); + tls_log_errors(tls, severity, domain, doing); return tor_error; case SSL_ERROR_ZERO_RETURN: if (extra&CATCH_ZERO) return _TOR_TLS_ZERORETURN; - log(severity, LD_NET, "TLS connection closed while %s", doing); - tls_log_errors(tls, severity, doing); + log(severity, LD_NET, "TLS connection closed while %s in state %s", + doing, ssl_state_to_string(tls->ssl->state)); + tls_log_errors(tls, severity, domain, doing); return TOR_TLS_CLOSE; default: - tls_log_errors(tls, severity, doing); + tls_log_errors(tls, severity, domain, doing); return TOR_TLS_ERROR_MISC; } } @@ -341,11 +377,10 @@ tor_tls_init(void) long version; SSL_library_init(); SSL_load_error_strings(); - crypto_global_init(-1); version = SSLeay(); - /* OpenSSL 0.9.8l introdeced SSL3_FLAGS_ALLOW_UNSAGE_LEGACY_RENEGOTIATION + /* OpenSSL 0.9.8l introduced SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION * here, but without thinking too hard about it: it turns out that the * flag in question needed to be set at the last minute, and that it * conflicted with an existing flag number that had already been added @@ -364,8 +399,8 @@ tor_tls_init(void) * leave their headers out of sync with their libraries. * * Yes, it _is_ almost as if the OpenSSL developers decided that no - * program should be allowed to use renegotiation its first passed an - * test of intelligence and determination. + * program should be allowed to use renegotiation unless it first passed + * a test of intelligence and determination. */ if (version >= 0x009080c0L && version < 0x009080d0L) { log_notice(LD_GENERAL, "OpenSSL %s looks like version 0.9.8l; " @@ -400,9 +435,15 @@ tor_tls_init(void) void tor_tls_free_all(void) { - if (global_tls_context) { - tor_tls_context_decref(global_tls_context); - global_tls_context = NULL; + if (server_tls_context) { + tor_tls_context_t *ctx = server_tls_context; + server_tls_context = NULL; + tor_tls_context_decref(ctx); + } + if (client_tls_context) { + tor_tls_context_t *ctx = client_tls_context; + client_tls_context = NULL; + tor_tls_context_decref(ctx); } if (!HT_EMPTY(&tlsmap_root)) { log_warn(LD_MM, "Still have entries in the tlsmap at shutdown."); @@ -511,7 +552,7 @@ tor_tls_create_certificate(crypto_pk_env_t *rsa, x509 = NULL; } done: - tls_log_errors(NULL, LOG_WARN, "generating certificate"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "generating certificate"); if (sign_pkey) EVP_PKEY_free(sign_pkey); if (pkey) @@ -528,9 +569,9 @@ tor_tls_create_certificate(crypto_pk_env_t *rsa, (TLS1_TXT_DHE_RSA_WITH_AES_256_SHA ":" \ TLS1_TXT_DHE_RSA_WITH_AES_128_SHA ":" \ SSL3_TXT_EDH_RSA_DES_192_CBC3_SHA) -/* Note: for setting up your own private testing network with link crypto - * disabled, set the cipher lists to your cipher list to - * SSL3_TXT_RSA_NULL_SHA. If you do this, you won't be able to communicate +/* Note: to set up your own private testing network with link crypto + * disabled, set your Tors' cipher list to + * (SSL3_TXT_RSA_NULL_SHA). If you do this, you won't be able to communicate * with any of the "real" Tors, though. */ #ifdef V2_HANDSHAKE_CLIENT @@ -589,15 +630,97 @@ tor_tls_context_incref(tor_tls_context_t *ctx) ++ctx->refcnt; } -/** Create a new TLS context for use with Tor TLS handshakes. - * <b>identity</b> should be set to the identity key used to sign the - * certificate, and <b>nickname</b> set to the nickname to use. +/** Create new global client and server TLS contexts. + * + * If <b>server_identity</b> is NULL, this will not generate a server + * TLS context. If <b>is_public_server</b> is non-zero, this will use + * the same TLS context for incoming and outgoing connections, and + * ignore <b>client_identity</b>. */ +int +tor_tls_context_init(int is_public_server, + crypto_pk_env_t *client_identity, + crypto_pk_env_t *server_identity, + unsigned int key_lifetime) +{ + int rv1 = 0; + int rv2 = 0; + + if (is_public_server) { + tor_tls_context_t *new_ctx; + tor_tls_context_t *old_ctx; + + tor_assert(server_identity != NULL); + + rv1 = tor_tls_context_init_one(&server_tls_context, + server_identity, + key_lifetime); + + if (rv1 >= 0) { + new_ctx = server_tls_context; + tor_tls_context_incref(new_ctx); + old_ctx = client_tls_context; + client_tls_context = new_ctx; + + if (old_ctx != NULL) { + tor_tls_context_decref(old_ctx); + } + } + } else { + if (server_identity != NULL) { + rv1 = tor_tls_context_init_one(&server_tls_context, + server_identity, + key_lifetime); + } else { + tor_tls_context_t *old_ctx = server_tls_context; + server_tls_context = NULL; + + if (old_ctx != NULL) { + tor_tls_context_decref(old_ctx); + } + } + + rv2 = tor_tls_context_init_one(&client_tls_context, + client_identity, + key_lifetime); + } + + return MIN(rv1, rv2); +} + +/** Create a new global TLS context. * * You can call this function multiple times. Each time you call it, * it generates new certificates; all new connections will use * the new SSL context. */ -int +static int +tor_tls_context_init_one(tor_tls_context_t **ppcontext, + crypto_pk_env_t *identity, + unsigned int key_lifetime) +{ + tor_tls_context_t *new_ctx = tor_tls_context_new(identity, + key_lifetime); + tor_tls_context_t *old_ctx = *ppcontext; + + if (new_ctx != NULL) { + *ppcontext = new_ctx; + + /* Free the old context if one existed. */ + if (old_ctx != NULL) { + /* This is safe even if there are open connections: we reference- + * count tor_tls_context_t objects. */ + tor_tls_context_decref(old_ctx); + } + } + + return ((new_ctx != NULL) ? 0 : -1); +} + +/** Create a new TLS context for use with Tor TLS handshakes. + * <b>identity</b> should be set to the identity key used to sign the + * certificate. + */ +static tor_tls_context_t * tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime) { crypto_pk_env_t *rsa = NULL; @@ -685,6 +808,7 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime) goto error; { crypto_dh_env_t *dh = crypto_dh_new(DH_TYPE_TLS); + tor_assert(dh); SSL_CTX_set_tmp_dh(result->ctx, _crypto_dh_env_get_dh(dh)); crypto_dh_free(dh); } @@ -692,21 +816,15 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime) always_accept_verify_cb); /* let us realloc bufs that we're writing from */ SSL_CTX_set_mode(result->ctx, SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER); - /* Free the old context if one exists. */ - if (global_tls_context) { - /* This is safe even if there are open connections: OpenSSL does - * reference counting with SSL and SSL_CTX objects. */ - tor_tls_context_decref(global_tls_context); - } - global_tls_context = result; + if (rsa) crypto_free_pk_env(rsa); tor_free(nickname); tor_free(nn2); - return 0; + return result; error: - tls_log_errors(NULL, LOG_WARN, "creating TLS context"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "creating TLS context"); tor_free(nickname); tor_free(nn2); if (pkey) @@ -719,7 +837,7 @@ tor_tls_context_new(crypto_pk_env_t *identity, unsigned int key_lifetime) X509_free(cert); if (idcert) X509_free(idcert); - return -1; + return NULL; } #ifdef V2_HANDSHAKE_SERVER @@ -734,11 +852,11 @@ tor_tls_client_is_using_v2_ciphers(const SSL *ssl, const char *address) /* If we reached this point, we just got a client hello. See if there is * a cipher list. */ if (!(session = SSL_get_session((SSL *)ssl))) { - log_warn(LD_NET, "No session on TLS?"); + log_info(LD_NET, "No session on TLS?"); return 0; } if (!session->ciphers) { - log_warn(LD_NET, "No ciphers on session"); + log_info(LD_NET, "No ciphers on session"); return 0; } /* Now we need to see if there are any ciphers whose presence means we're @@ -750,8 +868,7 @@ tor_tls_client_is_using_v2_ciphers(const SSL *ssl, const char *address) strcmp(ciphername, TLS1_TXT_DHE_RSA_WITH_AES_256_SHA) && strcmp(ciphername, SSL3_TXT_EDH_RSA_DES_192_CBC3_SHA) && strcmp(ciphername, "(NONE)")) { - /* XXXX should be ld_debug */ - log_info(LD_NET, "Got a non-version-1 cipher called '%s'", ciphername); + log_debug(LD_NET, "Got a non-version-1 cipher called '%s'", ciphername); // return 1; goto dump_list; } @@ -767,8 +884,8 @@ tor_tls_client_is_using_v2_ciphers(const SSL *ssl, const char *address) smartlist_add(elts, (char*)ciphername); } s = smartlist_join_strings(elts, ":", 0, NULL); - log_info(LD_NET, "Got a non-version-1 cipher list from %s. It is: '%s'", - address, s); + log_debug(LD_NET, "Got a non-version-1 cipher list from %s. It is: '%s'", + address, s); tor_free(s); smartlist_free(elts); } @@ -899,10 +1016,12 @@ tor_tls_new(int sock, int isServer) { BIO *bio = NULL; tor_tls_t *result = tor_malloc_zero(sizeof(tor_tls_t)); + tor_tls_context_t *context = isServer ? server_tls_context : + client_tls_context; - tor_assert(global_tls_context); /* make sure somebody made it first */ - if (!(result->ssl = SSL_new(global_tls_context->ctx))) { - tls_log_errors(NULL, LOG_WARN, "generating TLS context"); + tor_assert(context); /* make sure somebody made it first */ + if (!(result->ssl = SSL_new(context->ctx))) { + tls_log_errors(NULL, LOG_WARN, LD_NET, "creating SSL object"); tor_free(result); return NULL; } @@ -918,7 +1037,7 @@ tor_tls_new(int sock, int isServer) if (!SSL_set_cipher_list(result->ssl, isServer ? SERVER_CIPHER_LIST : CLIENT_CIPHER_LIST)) { - tls_log_errors(NULL, LOG_WARN, "setting ciphers"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "setting ciphers"); #ifdef SSL_set_tlsext_host_name SSL_set_tlsext_host_name(result->ssl, NULL); #endif @@ -931,7 +1050,7 @@ tor_tls_new(int sock, int isServer) result->socket = sock; bio = BIO_new_socket(sock, BIO_NOCLOSE); if (! bio) { - tls_log_errors(NULL, LOG_WARN, "opening BIO"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "opening BIO"); #ifdef SSL_set_tlsext_host_name SSL_set_tlsext_host_name(result->ssl, NULL); #endif @@ -941,8 +1060,8 @@ tor_tls_new(int sock, int isServer) } HT_INSERT(tlsmap, &tlsmap_root, result); SSL_set_bio(result->ssl, bio, bio); - tor_tls_context_incref(global_tls_context); - result->context = global_tls_context; + tor_tls_context_incref(context); + result->context = context; result->state = TOR_TLS_ST_HANDSHAKE; result->isServer = isServer; result->wantwrite_n = 0; @@ -959,7 +1078,7 @@ tor_tls_new(int sock, int isServer) #endif /* Not expected to get called. */ - tls_log_errors(NULL, LOG_WARN, "generating TLS context"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "creating tor_tls_t object"); return result; } @@ -1038,7 +1157,9 @@ void tor_tls_free(tor_tls_t *tls) { tor_tls_t *removed; - tor_assert(tls && tls->ssl); + if (!tls) + return; + tor_assert(tls->ssl); removed = HT_REMOVE(tlsmap, &tlsmap_root, tls); if (!removed) { log_warn(LD_BUG, "Freeing a TLS that was not in the ssl->tls map."); @@ -1081,7 +1202,7 @@ tor_tls_read(tor_tls_t *tls, char *cp, size_t len) #endif return r; } - err = tor_tls_get_error(tls, r, CATCH_ZERO, "reading", LOG_DEBUG); + err = tor_tls_get_error(tls, r, CATCH_ZERO, "reading", LOG_DEBUG, LD_NET); if (err == _TOR_TLS_ZERORETURN || err == TOR_TLS_CLOSE) { log_debug(LD_NET,"read returned r=%d; TLS is closed",r); tls->state = TOR_TLS_ST_CLOSED; @@ -1117,7 +1238,7 @@ tor_tls_write(tor_tls_t *tls, const char *cp, size_t n) tls->wantwrite_n = 0; } r = SSL_write(tls->ssl, cp, (int)n); - err = tor_tls_get_error(tls, r, 0, "writing", LOG_INFO); + err = tor_tls_get_error(tls, r, 0, "writing", LOG_INFO, LD_NET); if (err == TOR_TLS_DONE) { return r; } @@ -1135,21 +1256,30 @@ int tor_tls_handshake(tor_tls_t *tls) { int r; + int oldstate; tor_assert(tls); tor_assert(tls->ssl); tor_assert(tls->state == TOR_TLS_ST_HANDSHAKE); check_no_tls_errors(); + oldstate = tls->ssl->state; if (tls->isServer) { + log_debug(LD_HANDSHAKE, "About to call SSL_accept on %p (%s)", tls, + ssl_state_to_string(tls->ssl->state)); r = SSL_accept(tls->ssl); } else { + log_debug(LD_HANDSHAKE, "About to call SSL_connect on %p (%s)", tls, + ssl_state_to_string(tls->ssl->state)); r = SSL_connect(tls->ssl); } + if (oldstate != tls->ssl->state) + log_debug(LD_HANDSHAKE, "After call, %p was in state %s", + tls, ssl_state_to_string(tls->ssl->state)); /* We need to call this here and not earlier, since OpenSSL has a penchant * for clearing its flags when you say accept or connect. */ tor_tls_unblock_renegotiation(tls); - r = tor_tls_get_error(tls,r,0, "handshaking", LOG_INFO); + r = tor_tls_get_error(tls,r,0, "handshaking", LOG_INFO, LD_HANDSHAKE); if (ERR_peek_error() != 0) { - tls_log_errors(tls, tls->isServer ? LOG_INFO : LOG_WARN, + tls_log_errors(tls, tls->isServer ? LOG_INFO : LOG_WARN, LD_HANDSHAKE, "handshaking"); return TOR_TLS_ERROR_MISC; } @@ -1170,7 +1300,8 @@ tor_tls_handshake(tor_tls_t *tls) " get set. Fixing that."); } tls->wasV2Handshake = 1; - log_debug(LD_NET, "Completed V2 TLS handshake with client; waiting " + log_debug(LD_HANDSHAKE, + "Completed V2 TLS handshake with client; waiting " "for renegotiation."); } else { tls->wasV2Handshake = 0; @@ -1182,10 +1313,13 @@ tor_tls_handshake(tor_tls_t *tls) X509 *cert = SSL_get_peer_certificate(tls->ssl); STACK_OF(X509) *chain = SSL_get_peer_cert_chain(tls->ssl); int n_certs = sk_X509_num(chain); - if (n_certs > 1 || (n_certs == 1 && cert != sk_X509_value(chain, 0))) + if (n_certs > 1 || (n_certs == 1 && cert != sk_X509_value(chain, 0))) { + log_debug(LD_HANDSHAKE, "Server sent back multiple certificates; it " + "looks like a v1 handshake on %p", tls); tls->wasV2Handshake = 0; - else { - log_debug(LD_NET, "Server sent back a single certificate; looks like " + } else { + log_debug(LD_HANDSHAKE, + "Server sent back a single certificate; looks like " "a v2 handshake on %p.", tls); tls->wasV2Handshake = 1; } @@ -1193,7 +1327,7 @@ tor_tls_handshake(tor_tls_t *tls) X509_free(cert); #endif if (SSL_set_cipher_list(tls->ssl, SERVER_CIPHER_LIST) == 0) { - tls_log_errors(NULL, LOG_WARN, "re-setting ciphers"); + tls_log_errors(NULL, LOG_WARN, LD_HANDSHAKE, "re-setting ciphers"); r = TOR_TLS_ERROR_MISC; } } @@ -1216,7 +1350,8 @@ tor_tls_renegotiate(tor_tls_t *tls) if (tls->state != TOR_TLS_ST_RENEGOTIATE) { int r = SSL_renegotiate(tls->ssl); if (r <= 0) { - return tor_tls_get_error(tls, r, 0, "renegotiating", LOG_WARN); + return tor_tls_get_error(tls, r, 0, "renegotiating", LOG_WARN, + LD_HANDSHAKE); } tls->state = TOR_TLS_ST_RENEGOTIATE; } @@ -1225,7 +1360,8 @@ tor_tls_renegotiate(tor_tls_t *tls) tls->state = TOR_TLS_ST_OPEN; return TOR_TLS_DONE; } else - return tor_tls_get_error(tls, r, 0, "renegotiating handshake", LOG_INFO); + return tor_tls_get_error(tls, r, 0, "renegotiating handshake", LOG_INFO, + LD_HANDSHAKE); } /** Shut down an open tls connection <b>tls</b>. When finished, returns @@ -1249,7 +1385,7 @@ tor_tls_shutdown(tor_tls_t *tls) r = SSL_read(tls->ssl, buf, 128); } while (r>0); err = tor_tls_get_error(tls, r, CATCH_ZERO, "reading to shut down", - LOG_INFO); + LOG_INFO, LD_NET); if (err == _TOR_TLS_ZERORETURN) { tls->state = TOR_TLS_ST_GOTCLOSE; /* fall through... */ @@ -1265,7 +1401,7 @@ tor_tls_shutdown(tor_tls_t *tls) return TOR_TLS_DONE; } err = tor_tls_get_error(tls, r, CATCH_SYSCALL|CATCH_ZERO, "shutting down", - LOG_INFO); + LOG_INFO, LD_NET); if (err == _TOR_TLS_SYSCALL) { /* The underlying TCP connection closed while we were shutting down. */ tls->state = TOR_TLS_ST_CLOSED; @@ -1297,7 +1433,7 @@ tor_tls_peer_has_cert(tor_tls_t *tls) { X509 *cert; cert = SSL_get_peer_certificate(tls->ssl); - tls_log_errors(tls, LOG_WARN, "getting peer certificate"); + tls_log_errors(tls, LOG_WARN, LD_HANDSHAKE, "getting peer certificate"); if (!cert) return 0; X509_free(cert); @@ -1324,7 +1460,7 @@ log_cert_lifetime(X509 *cert, const char *problem) log_warn(LD_GENERAL, "Couldn't allocate BIO!"); goto end; } if (!(ASN1_TIME_print(bio, X509_get_notBefore(cert)))) { - tls_log_errors(NULL, LOG_WARN, "printing certificate lifetime"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "printing certificate lifetime"); goto end; } BIO_get_mem_ptr(bio, &buf); @@ -1332,7 +1468,7 @@ log_cert_lifetime(X509 *cert, const char *problem) (void)BIO_reset(bio); if (!(ASN1_TIME_print(bio, X509_get_notAfter(cert)))) { - tls_log_errors(NULL, LOG_WARN, "printing certificate lifetime"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "printing certificate lifetime"); goto end; } BIO_get_mem_ptr(bio, &buf); @@ -1346,13 +1482,11 @@ log_cert_lifetime(X509 *cert, const char *problem) end: /* Not expected to get invoked */ - tls_log_errors(NULL, LOG_WARN, "getting certificate lifetime"); + tls_log_errors(NULL, LOG_WARN, LD_NET, "getting certificate lifetime"); if (bio) BIO_free(bio); - if (s1) - tor_free(s1); - if (s2) - tor_free(s2); + tor_free(s1); + tor_free(s2); } /** Helper function: try to extract a link certificate and an identity @@ -1420,7 +1554,7 @@ tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity_key) if (!(id_pkey = X509_get_pubkey(id_cert)) || X509_verify(cert, id_pkey) <= 0) { log_fn(severity,LD_PROTOCOL,"X509_verify on cert and pkey returned <= 0"); - tls_log_errors(tls, severity,"verifying certificate"); + tls_log_errors(tls, severity, LD_HANDSHAKE, "verifying certificate"); goto done; } @@ -1439,7 +1573,7 @@ tor_tls_verify(int severity, tor_tls_t *tls, crypto_pk_env_t **identity_key) /* This should never get invoked, but let's make sure in case OpenSSL * acts unexpectedly. */ - tls_log_errors(tls, LOG_WARN, "finishing tor_tls_verify"); + tls_log_errors(tls, LOG_WARN, LD_HANDSHAKE, "finishing tor_tls_verify"); return r; } @@ -1478,7 +1612,7 @@ tor_tls_check_lifetime(tor_tls_t *tls, int tolerance) if (cert) X509_free(cert); /* Not expected to get invoked */ - tls_log_errors(tls, LOG_WARN, "checking certificate lifetime"); + tls_log_errors(tls, LOG_WARN, LD_NET, "checking certificate lifetime"); return r; } @@ -1546,7 +1680,7 @@ _check_no_tls_errors(const char *fname, int line) return; log(LOG_WARN, LD_CRYPTO, "Unhandled OpenSSL errors found at %s:%d: ", tor_fix_source_file(fname), line); - tls_log_errors(NULL, LOG_WARN, NULL); + tls_log_errors(NULL, LOG_WARN, LD_NET, NULL); } /** Return true iff the initial TLS connection at <b>tls</b> did not use a v2 diff --git a/src/common/tortls.h b/src/common/tortls.h index 9644b87f2..55fee81ae 100644 --- a/src/common/tortls.h +++ b/src/common/tortls.h @@ -50,7 +50,10 @@ typedef struct tor_tls_t tor_tls_t; const char *tor_tls_err_to_string(int err); void tor_tls_free_all(void); -int tor_tls_context_new(crypto_pk_env_t *rsa, unsigned int key_lifetime); +int tor_tls_context_init(int is_public_server, + crypto_pk_env_t *client_identity, + crypto_pk_env_t *server_identity, + unsigned int key_lifetime); tor_tls_t *tor_tls_new(int sock, int is_server); void tor_tls_set_logged_address(tor_tls_t *tls, const char *address); void tor_tls_set_renegotiate_callback(tor_tls_t *tls, diff --git a/src/common/tortls_states.h b/src/common/tortls_states.h new file mode 100644 index 000000000..dcff2479f --- /dev/null +++ b/src/common/tortls_states.h @@ -0,0 +1,414 @@ +/* Copyright (c) 2003, Roger Dingledine + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/* Helper file: included only in tortls.c */ + +#ifndef _TORTLS_STATES_H +#define _TORTLS_STATES_H + +/* The main body of this file was mechanically generated with this + perl script: + + my %keys = (); + for $fn (@ARGV) { + open(F, $fn); + while (<F>) { + next unless /^#define ((?:SSL|DTLS)\w*_ST_\w*)/; + $keys{$1} = 1; + } + close(F); + } + for $k (sort keys %keys) { + print "#ifdef $k\n S($k),\n#endif\n" + } +*/ + +/** Mapping from allowed value of SSL.state to the name of C macro for that + * state. Used for debugging an openssl connection. */ +static const struct { int state; const char *name; } state_map[] = { +#define S(state) { state, #state } +#ifdef DTLS1_ST_CR_HELLO_VERIFY_REQUEST_A + S(DTLS1_ST_CR_HELLO_VERIFY_REQUEST_A), +#endif +#ifdef DTLS1_ST_CR_HELLO_VERIFY_REQUEST_B + S(DTLS1_ST_CR_HELLO_VERIFY_REQUEST_B), +#endif +#ifdef DTLS1_ST_SW_HELLO_VERIFY_REQUEST_A + S(DTLS1_ST_SW_HELLO_VERIFY_REQUEST_A), +#endif +#ifdef DTLS1_ST_SW_HELLO_VERIFY_REQUEST_B + S(DTLS1_ST_SW_HELLO_VERIFY_REQUEST_B), +#endif +#ifdef SSL23_ST_CR_SRVR_HELLO_A + S(SSL23_ST_CR_SRVR_HELLO_A), +#endif +#ifdef SSL23_ST_CR_SRVR_HELLO_B + S(SSL23_ST_CR_SRVR_HELLO_B), +#endif +#ifdef SSL23_ST_CW_CLNT_HELLO_A + S(SSL23_ST_CW_CLNT_HELLO_A), +#endif +#ifdef SSL23_ST_CW_CLNT_HELLO_B + S(SSL23_ST_CW_CLNT_HELLO_B), +#endif +#ifdef SSL23_ST_SR_CLNT_HELLO_A + S(SSL23_ST_SR_CLNT_HELLO_A), +#endif +#ifdef SSL23_ST_SR_CLNT_HELLO_B + S(SSL23_ST_SR_CLNT_HELLO_B), +#endif +#ifdef SSL2_ST_CLIENT_START_ENCRYPTION + S(SSL2_ST_CLIENT_START_ENCRYPTION), +#endif +#ifdef SSL2_ST_GET_CLIENT_FINISHED_A + S(SSL2_ST_GET_CLIENT_FINISHED_A), +#endif +#ifdef SSL2_ST_GET_CLIENT_FINISHED_B + S(SSL2_ST_GET_CLIENT_FINISHED_B), +#endif +#ifdef SSL2_ST_GET_CLIENT_HELLO_A + S(SSL2_ST_GET_CLIENT_HELLO_A), +#endif +#ifdef SSL2_ST_GET_CLIENT_HELLO_B + S(SSL2_ST_GET_CLIENT_HELLO_B), +#endif +#ifdef SSL2_ST_GET_CLIENT_HELLO_C + S(SSL2_ST_GET_CLIENT_HELLO_C), +#endif +#ifdef SSL2_ST_GET_CLIENT_MASTER_KEY_A + S(SSL2_ST_GET_CLIENT_MASTER_KEY_A), +#endif +#ifdef SSL2_ST_GET_CLIENT_MASTER_KEY_B + S(SSL2_ST_GET_CLIENT_MASTER_KEY_B), +#endif +#ifdef SSL2_ST_GET_SERVER_FINISHED_A + S(SSL2_ST_GET_SERVER_FINISHED_A), +#endif +#ifdef SSL2_ST_GET_SERVER_FINISHED_B + S(SSL2_ST_GET_SERVER_FINISHED_B), +#endif +#ifdef SSL2_ST_GET_SERVER_HELLO_A + S(SSL2_ST_GET_SERVER_HELLO_A), +#endif +#ifdef SSL2_ST_GET_SERVER_HELLO_B + S(SSL2_ST_GET_SERVER_HELLO_B), +#endif +#ifdef SSL2_ST_GET_SERVER_VERIFY_A + S(SSL2_ST_GET_SERVER_VERIFY_A), +#endif +#ifdef SSL2_ST_GET_SERVER_VERIFY_B + S(SSL2_ST_GET_SERVER_VERIFY_B), +#endif +#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_A + S(SSL2_ST_SEND_CLIENT_CERTIFICATE_A), +#endif +#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_B + S(SSL2_ST_SEND_CLIENT_CERTIFICATE_B), +#endif +#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_C + S(SSL2_ST_SEND_CLIENT_CERTIFICATE_C), +#endif +#ifdef SSL2_ST_SEND_CLIENT_CERTIFICATE_D + S(SSL2_ST_SEND_CLIENT_CERTIFICATE_D), +#endif +#ifdef SSL2_ST_SEND_CLIENT_FINISHED_A + S(SSL2_ST_SEND_CLIENT_FINISHED_A), +#endif +#ifdef SSL2_ST_SEND_CLIENT_FINISHED_B + S(SSL2_ST_SEND_CLIENT_FINISHED_B), +#endif +#ifdef SSL2_ST_SEND_CLIENT_HELLO_A + S(SSL2_ST_SEND_CLIENT_HELLO_A), +#endif +#ifdef SSL2_ST_SEND_CLIENT_HELLO_B + S(SSL2_ST_SEND_CLIENT_HELLO_B), +#endif +#ifdef SSL2_ST_SEND_CLIENT_MASTER_KEY_A + S(SSL2_ST_SEND_CLIENT_MASTER_KEY_A), +#endif +#ifdef SSL2_ST_SEND_CLIENT_MASTER_KEY_B + S(SSL2_ST_SEND_CLIENT_MASTER_KEY_B), +#endif +#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_A + S(SSL2_ST_SEND_REQUEST_CERTIFICATE_A), +#endif +#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_B + S(SSL2_ST_SEND_REQUEST_CERTIFICATE_B), +#endif +#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_C + S(SSL2_ST_SEND_REQUEST_CERTIFICATE_C), +#endif +#ifdef SSL2_ST_SEND_REQUEST_CERTIFICATE_D + S(SSL2_ST_SEND_REQUEST_CERTIFICATE_D), +#endif +#ifdef SSL2_ST_SEND_SERVER_FINISHED_A + S(SSL2_ST_SEND_SERVER_FINISHED_A), +#endif +#ifdef SSL2_ST_SEND_SERVER_FINISHED_B + S(SSL2_ST_SEND_SERVER_FINISHED_B), +#endif +#ifdef SSL2_ST_SEND_SERVER_HELLO_A + S(SSL2_ST_SEND_SERVER_HELLO_A), +#endif +#ifdef SSL2_ST_SEND_SERVER_HELLO_B + S(SSL2_ST_SEND_SERVER_HELLO_B), +#endif +#ifdef SSL2_ST_SEND_SERVER_VERIFY_A + S(SSL2_ST_SEND_SERVER_VERIFY_A), +#endif +#ifdef SSL2_ST_SEND_SERVER_VERIFY_B + S(SSL2_ST_SEND_SERVER_VERIFY_B), +#endif +#ifdef SSL2_ST_SEND_SERVER_VERIFY_C + S(SSL2_ST_SEND_SERVER_VERIFY_C), +#endif +#ifdef SSL2_ST_SERVER_START_ENCRYPTION + S(SSL2_ST_SERVER_START_ENCRYPTION), +#endif +#ifdef SSL2_ST_X509_GET_CLIENT_CERTIFICATE + S(SSL2_ST_X509_GET_CLIENT_CERTIFICATE), +#endif +#ifdef SSL2_ST_X509_GET_SERVER_CERTIFICATE + S(SSL2_ST_X509_GET_SERVER_CERTIFICATE), +#endif +#ifdef SSL3_ST_CR_CERT_A + S(SSL3_ST_CR_CERT_A), +#endif +#ifdef SSL3_ST_CR_CERT_B + S(SSL3_ST_CR_CERT_B), +#endif +#ifdef SSL3_ST_CR_CERT_REQ_A + S(SSL3_ST_CR_CERT_REQ_A), +#endif +#ifdef SSL3_ST_CR_CERT_REQ_B + S(SSL3_ST_CR_CERT_REQ_B), +#endif +#ifdef SSL3_ST_CR_CERT_STATUS_A + S(SSL3_ST_CR_CERT_STATUS_A), +#endif +#ifdef SSL3_ST_CR_CERT_STATUS_B + S(SSL3_ST_CR_CERT_STATUS_B), +#endif +#ifdef SSL3_ST_CR_CHANGE_A + S(SSL3_ST_CR_CHANGE_A), +#endif +#ifdef SSL3_ST_CR_CHANGE_B + S(SSL3_ST_CR_CHANGE_B), +#endif +#ifdef SSL3_ST_CR_FINISHED_A + S(SSL3_ST_CR_FINISHED_A), +#endif +#ifdef SSL3_ST_CR_FINISHED_B + S(SSL3_ST_CR_FINISHED_B), +#endif +#ifdef SSL3_ST_CR_KEY_EXCH_A + S(SSL3_ST_CR_KEY_EXCH_A), +#endif +#ifdef SSL3_ST_CR_KEY_EXCH_B + S(SSL3_ST_CR_KEY_EXCH_B), +#endif +#ifdef SSL3_ST_CR_SESSION_TICKET_A + S(SSL3_ST_CR_SESSION_TICKET_A), +#endif +#ifdef SSL3_ST_CR_SESSION_TICKET_B + S(SSL3_ST_CR_SESSION_TICKET_B), +#endif +#ifdef SSL3_ST_CR_SRVR_DONE_A + S(SSL3_ST_CR_SRVR_DONE_A), +#endif +#ifdef SSL3_ST_CR_SRVR_DONE_B + S(SSL3_ST_CR_SRVR_DONE_B), +#endif +#ifdef SSL3_ST_CR_SRVR_HELLO_A + S(SSL3_ST_CR_SRVR_HELLO_A), +#endif +#ifdef SSL3_ST_CR_SRVR_HELLO_B + S(SSL3_ST_CR_SRVR_HELLO_B), +#endif +#ifdef SSL3_ST_CW_CERT_A + S(SSL3_ST_CW_CERT_A), +#endif +#ifdef SSL3_ST_CW_CERT_B + S(SSL3_ST_CW_CERT_B), +#endif +#ifdef SSL3_ST_CW_CERT_C + S(SSL3_ST_CW_CERT_C), +#endif +#ifdef SSL3_ST_CW_CERT_D + S(SSL3_ST_CW_CERT_D), +#endif +#ifdef SSL3_ST_CW_CERT_VRFY_A + S(SSL3_ST_CW_CERT_VRFY_A), +#endif +#ifdef SSL3_ST_CW_CERT_VRFY_B + S(SSL3_ST_CW_CERT_VRFY_B), +#endif +#ifdef SSL3_ST_CW_CHANGE_A + S(SSL3_ST_CW_CHANGE_A), +#endif +#ifdef SSL3_ST_CW_CHANGE_B + S(SSL3_ST_CW_CHANGE_B), +#endif +#ifdef SSL3_ST_CW_CLNT_HELLO_A + S(SSL3_ST_CW_CLNT_HELLO_A), +#endif +#ifdef SSL3_ST_CW_CLNT_HELLO_B + S(SSL3_ST_CW_CLNT_HELLO_B), +#endif +#ifdef SSL3_ST_CW_FINISHED_A + S(SSL3_ST_CW_FINISHED_A), +#endif +#ifdef SSL3_ST_CW_FINISHED_B + S(SSL3_ST_CW_FINISHED_B), +#endif +#ifdef SSL3_ST_CW_FLUSH + S(SSL3_ST_CW_FLUSH), +#endif +#ifdef SSL3_ST_CW_KEY_EXCH_A + S(SSL3_ST_CW_KEY_EXCH_A), +#endif +#ifdef SSL3_ST_CW_KEY_EXCH_B + S(SSL3_ST_CW_KEY_EXCH_B), +#endif +#ifdef SSL3_ST_SR_CERT_A + S(SSL3_ST_SR_CERT_A), +#endif +#ifdef SSL3_ST_SR_CERT_B + S(SSL3_ST_SR_CERT_B), +#endif +#ifdef SSL3_ST_SR_CERT_VRFY_A + S(SSL3_ST_SR_CERT_VRFY_A), +#endif +#ifdef SSL3_ST_SR_CERT_VRFY_B + S(SSL3_ST_SR_CERT_VRFY_B), +#endif +#ifdef SSL3_ST_SR_CHANGE_A + S(SSL3_ST_SR_CHANGE_A), +#endif +#ifdef SSL3_ST_SR_CHANGE_B + S(SSL3_ST_SR_CHANGE_B), +#endif +#ifdef SSL3_ST_SR_CLNT_HELLO_A + S(SSL3_ST_SR_CLNT_HELLO_A), +#endif +#ifdef SSL3_ST_SR_CLNT_HELLO_B + S(SSL3_ST_SR_CLNT_HELLO_B), +#endif +#ifdef SSL3_ST_SR_CLNT_HELLO_C + S(SSL3_ST_SR_CLNT_HELLO_C), +#endif +#ifdef SSL3_ST_SR_FINISHED_A + S(SSL3_ST_SR_FINISHED_A), +#endif +#ifdef SSL3_ST_SR_FINISHED_B + S(SSL3_ST_SR_FINISHED_B), +#endif +#ifdef SSL3_ST_SR_KEY_EXCH_A + S(SSL3_ST_SR_KEY_EXCH_A), +#endif +#ifdef SSL3_ST_SR_KEY_EXCH_B + S(SSL3_ST_SR_KEY_EXCH_B), +#endif +#ifdef SSL3_ST_SW_CERT_A + S(SSL3_ST_SW_CERT_A), +#endif +#ifdef SSL3_ST_SW_CERT_B + S(SSL3_ST_SW_CERT_B), +#endif +#ifdef SSL3_ST_SW_CERT_REQ_A + S(SSL3_ST_SW_CERT_REQ_A), +#endif +#ifdef SSL3_ST_SW_CERT_REQ_B + S(SSL3_ST_SW_CERT_REQ_B), +#endif +#ifdef SSL3_ST_SW_CERT_STATUS_A + S(SSL3_ST_SW_CERT_STATUS_A), +#endif +#ifdef SSL3_ST_SW_CERT_STATUS_B + S(SSL3_ST_SW_CERT_STATUS_B), +#endif +#ifdef SSL3_ST_SW_CHANGE_A + S(SSL3_ST_SW_CHANGE_A), +#endif +#ifdef SSL3_ST_SW_CHANGE_B + S(SSL3_ST_SW_CHANGE_B), +#endif +#ifdef SSL3_ST_SW_FINISHED_A + S(SSL3_ST_SW_FINISHED_A), +#endif +#ifdef SSL3_ST_SW_FINISHED_B + S(SSL3_ST_SW_FINISHED_B), +#endif +#ifdef SSL3_ST_SW_FLUSH + S(SSL3_ST_SW_FLUSH), +#endif +#ifdef SSL3_ST_SW_HELLO_REQ_A + S(SSL3_ST_SW_HELLO_REQ_A), +#endif +#ifdef SSL3_ST_SW_HELLO_REQ_B + S(SSL3_ST_SW_HELLO_REQ_B), +#endif +#ifdef SSL3_ST_SW_HELLO_REQ_C + S(SSL3_ST_SW_HELLO_REQ_C), +#endif +#ifdef SSL3_ST_SW_KEY_EXCH_A + S(SSL3_ST_SW_KEY_EXCH_A), +#endif +#ifdef SSL3_ST_SW_KEY_EXCH_B + S(SSL3_ST_SW_KEY_EXCH_B), +#endif +#ifdef SSL3_ST_SW_SESSION_TICKET_A + S(SSL3_ST_SW_SESSION_TICKET_A), +#endif +#ifdef SSL3_ST_SW_SESSION_TICKET_B + S(SSL3_ST_SW_SESSION_TICKET_B), +#endif +#ifdef SSL3_ST_SW_SRVR_DONE_A + S(SSL3_ST_SW_SRVR_DONE_A), +#endif +#ifdef SSL3_ST_SW_SRVR_DONE_B + S(SSL3_ST_SW_SRVR_DONE_B), +#endif +#ifdef SSL3_ST_SW_SRVR_HELLO_A + S(SSL3_ST_SW_SRVR_HELLO_A), +#endif +#ifdef SSL3_ST_SW_SRVR_HELLO_B + S(SSL3_ST_SW_SRVR_HELLO_B), +#endif +#ifdef SSL_ST_ACCEPT + S(SSL_ST_ACCEPT), +#endif +#ifdef SSL_ST_BEFORE + S(SSL_ST_BEFORE), +#endif +#ifdef SSL_ST_CONNECT + S(SSL_ST_CONNECT), +#endif +#ifdef SSL_ST_INIT + S(SSL_ST_INIT), +#endif +#ifdef SSL_ST_MASK + S(SSL_ST_MASK), +#endif +#ifdef SSL_ST_OK + S(SSL_ST_OK), +#endif +#ifdef SSL_ST_READ_BODY + S(SSL_ST_READ_BODY), +#endif +#ifdef SSL_ST_READ_DONE + S(SSL_ST_READ_DONE), +#endif +#ifdef SSL_ST_READ_HEADER + S(SSL_ST_READ_HEADER), +#endif +#ifdef SSL_ST_RENEGOTIATE + S(SSL_ST_RENEGOTIATE), +#endif + { 0, NULL } +}; + +#endif + diff --git a/src/common/util.c b/src/common/util.c index f206d00c4..38c0ad05e 100644 --- a/src/common/util.c +++ b/src/common/util.c @@ -15,7 +15,8 @@ #include "orconfig.h" #include "util.h" -#include "log.h" +#include "torlog.h" +#undef log #include "crypto.h" #include "torint.h" #include "container.h" @@ -25,11 +26,17 @@ #include <io.h> #include <direct.h> #include <process.h> +#include <tchar.h> #else #include <dirent.h> #include <pwd.h> #endif +/* math.h needs this on Linux */ +#ifndef __USE_ISOC99 +#define __USE_ISOC99 1 +#endif +#include <math.h> #include <stdlib.h> #include <stdio.h> #include <string.h> @@ -287,7 +294,7 @@ tor_log_mallinfo(int severity) struct mallinfo mi; memset(&mi, 0, sizeof(mi)); mi = mallinfo(); - log(severity, LD_MM, + tor_log(severity, LD_MM, "mallinfo() said: arena=%d, ordblks=%d, smblks=%d, hblks=%d, " "hblkhd=%d, usmblks=%d, fsmblks=%d, uordblks=%d, fordblks=%d, " "keepcost=%d", @@ -310,6 +317,25 @@ tor_log_mallinfo(int severity) * Math * ===== */ +/** + * Returns the natural logarithm of d base 2. We define this wrapper here so + * as to make it easier not to conflict with Tor's log() macro. + */ +double +tor_mathlog(double d) +{ + return log(d); +} + +/** Return the long integer closest to d. We define this wrapper here so + * that not all users of math.h need to use the right incancations to get + * the c99 functions. */ +long +tor_lround(double d) +{ + return lround(d); +} + /** Returns floor(log2(u64)). If u64 is 0, (incorrectly) returns 0. */ int tor_log2(uint64_t u64) @@ -354,6 +380,36 @@ round_to_power_of_2(uint64_t u64) return low; } +/** Return the lowest x such that x is at least <b>number</b>, and x modulo + * <b>divisor</b> == 0. */ +unsigned +round_to_next_multiple_of(unsigned number, unsigned divisor) +{ + number += divisor - 1; + number -= number % divisor; + return number; +} + +/** Return the lowest x such that x is at least <b>number</b>, and x modulo + * <b>divisor</b> == 0. */ +uint32_t +round_uint32_to_next_multiple_of(uint32_t number, uint32_t divisor) +{ + number += divisor - 1; + number -= number % divisor; + return number; +} + +/** Return the lowest x such that x is at least <b>number</b>, and x modulo + * <b>divisor</b> == 0. */ +uint64_t +round_uint64_to_next_multiple_of(uint64_t number, uint64_t divisor) +{ + number += divisor - 1; + number -= number % divisor; + return number; +} + /* ===== * String manipulation * ===== */ @@ -636,6 +692,29 @@ find_whitespace_eos(const char *s, const char *eos) return s; } +/** Return the first occurrence of <b>needle</b> in <b>haystack</b> that + * occurs at the start of a line (that is, at the beginning of <b>haystack</b> + * or immediately after a newline). Return NULL if no such string is found. + */ +const char * +find_str_at_start_of_line(const char *haystack, const char *needle) +{ + size_t needle_len = strlen(needle); + + do { + if (!strncmp(haystack, needle, needle_len)) + return haystack; + + haystack = strchr(haystack, '\n'); + if (!haystack) + return NULL; + else + ++haystack; + } while (*haystack); + + return NULL; +} + /** Return true iff the 'len' bytes at 'mem' are all zero. */ int tor_mem_is_zero(const char *mem, size_t len) @@ -663,6 +742,13 @@ tor_digest_is_zero(const char *digest) return tor_mem_is_zero(digest, DIGEST_LEN); } +/** Return true iff the DIGEST256_LEN bytes in digest are all zero. */ +int +tor_digest256_is_zero(const char *digest) +{ + return tor_mem_is_zero(digest, DIGEST256_LEN); +} + /* Helper: common code to check whether the result of a strtol or strtoul or * strtoll is correct. */ #define CHECK_STRTOX_RESULT() \ @@ -714,7 +800,18 @@ tor_parse_ulong(const char *s, int base, unsigned long min, CHECK_STRTOX_RESULT(); } -/** As tor_parse_log, but return a unit64_t. Only base 10 is guaranteed to +/** As tor_parse_long(), but return a double. */ +double +tor_parse_double(const char *s, double min, double max, int *ok, char **next) +{ + char *endptr; + double r; + + r = strtod(s, &endptr); + CHECK_STRTOX_RESULT(); +} + +/** As tor_parse_long, but return a uint64_t. Only base 10 is guaranteed to * work for now. */ uint64_t tor_parse_uint64(const char *s, int base, uint64_t min, @@ -853,6 +950,9 @@ esc_for_log(const char *s) case '\\': case '\"': case '\'': + case '\r': + case '\n': + case '\t': len += 2; break; default: @@ -914,8 +1014,7 @@ const char * escaped(const char *s) { static char *_escaped_val = NULL; - if (_escaped_val) - tor_free(_escaped_val); + tor_free(_escaped_val); if (s) _escaped_val = esc_for_log(s); @@ -1002,6 +1101,42 @@ wrap_string(smartlist_t *out, const char *string, size_t width, * Time * ===== */ +/** + * Converts struct timeval to a double value. + * Preserves microsecond precision, but just barely. + * Error is approx +/- 0.1 usec when dealing with epoch values. + */ +double +tv_to_double(const struct timeval *tv) +{ + double conv = tv->tv_sec; + conv += tv->tv_usec/1000000.0; + return conv; +} + +/** + * Converts timeval to milliseconds. + */ +int64_t +tv_to_msec(const struct timeval *tv) +{ + int64_t conv = ((int64_t)tv->tv_sec)*1000L; + /* Round ghetto-style */ + conv += ((int64_t)tv->tv_usec+500)/1000L; + return conv; +} + +/** + * Converts timeval to microseconds. + */ +int64_t +tv_to_usec(const struct timeval *tv) +{ + int64_t conv = ((int64_t)tv->tv_sec)*1000000L; + conv += tv->tv_usec; + return conv; +} + /** Return the number of microseconds elapsed between *start and *end. */ long @@ -1011,7 +1146,8 @@ tv_udiff(const struct timeval *start, const struct timeval *end) long secdiff = end->tv_sec - start->tv_sec; if (labs(secdiff+1) > LONG_MAX/1000000) { - log_warn(LD_GENERAL, "comparing times too far apart."); + log_warn(LD_GENERAL, "comparing times on microsecond detail too far " + "apart: %ld seconds", secdiff); return LONG_MAX; } @@ -1019,6 +1155,26 @@ tv_udiff(const struct timeval *start, const struct timeval *end) return udiff; } +/** Return the number of milliseconds elapsed between *start and *end. + */ +long +tv_mdiff(const struct timeval *start, const struct timeval *end) +{ + long mdiff; + long secdiff = end->tv_sec - start->tv_sec; + + if (labs(secdiff+1) > LONG_MAX/1000) { + log_warn(LD_GENERAL, "comparing times on millisecond detail too far " + "apart: %ld seconds", secdiff); + return LONG_MAX; + } + + /* Subtract and round */ + mdiff = secdiff*1000L + + ((long)end->tv_usec - (long)start->tv_usec + 500L) / 1000L; + return mdiff; +} + /** Yield true iff <b>y</b> is a leap-year. */ #define IS_LEAPYEAR(y) (!(y % 4) && ((y % 100) || !(y % 400))) /** Helper: Return the number of leap-days between Jan 1, y1 and Jan 1, y2. */ @@ -1094,7 +1250,7 @@ format_rfc1123_time(char *buf, time_t t) memcpy(buf+8, MONTH_NAMES[tm.tm_mon], 3); } -/** Parse the the RFC1123 encoding of some time (in GMT) from <b>buf</b>, +/** Parse the RFC1123 encoding of some time (in GMT) from <b>buf</b>, * and store the result in *<b>t</b>. * * Return 0 on success, -1 on failure. @@ -1355,80 +1511,46 @@ update_approx_time(time_t now) #endif /* ===== - * Fuzzy time - * XXXX022 Use this consistently or rip most of it out. + * Rate limiting * ===== */ -/* In a perfect world, everybody would run NTP, and NTP would be perfect, so - * if we wanted to know "Is the current time before time X?" we could just say - * "time(NULL) < X". - * - * But unfortunately, many users are running Tor in an imperfect world, on - * even more imperfect computers. Hence, we need to track time oddly. We - * model the user's computer as being "skewed" from accurate time by - * -<b>ftime_skew</b> seconds, such that our best guess of the current time is - * time(NULL)+ftime_skew. We also assume that our measurements of time may - * have up to <b>ftime_slop</b> seconds of inaccuracy; IOW, our window of - * estimate for the current time is now + ftime_skew +/- ftime_slop. - */ -/** Our current estimate of our skew, such that we think the current time is - * closest to time(NULL)+ftime_skew. */ -static int ftime_skew = 0; -/** Tolerance during time comparisons, in seconds. */ -static int ftime_slop = 60; -/** Set the largest amount of sloppiness we'll allow in fuzzy time - * comparisons. */ -void -ftime_set_maximum_sloppiness(int seconds) -{ - tor_assert(seconds >= 0); - ftime_slop = seconds; -} -/** Set the amount by which we believe our system clock to differ from - * real time. */ -void -ftime_set_estimated_skew(int seconds) -{ - ftime_skew = seconds; -} -#if 0 -void -ftime_get_window(time_t now, ftime_t *ft_out) -{ - ft_out->earliest = now + ftime_skew - ftime_slop; - ft_out->latest = now + ftime_skew + ftime_slop; -} -#endif -/** Return true iff we think that <b>now</b> might be after <b>when</b>. */ -int -ftime_maybe_after(time_t now, time_t when) -{ - /* It may be after when iff the latest possible current time is after when */ - return (now + ftime_skew + ftime_slop) >= when; -} -/** Return true iff we think that <b>now</b> might be before <b>when</b>. */ -int -ftime_maybe_before(time_t now, time_t when) -{ - /* It may be before when iff the earliest possible current time is before */ - return (now + ftime_skew - ftime_slop) < when; -} -/** Return true if we think that <b>now</b> is definitely after <b>when</b>. */ -int -ftime_definitely_after(time_t now, time_t when) +/** If the rate-limiter <b>lim</b> is ready at <b>now</b>, return the number + * of calls to rate_limit_is_ready (including this one!) since the last time + * rate_limit_is_ready returned nonzero. Otherwise return 0. */ +static int +rate_limit_is_ready(ratelim_t *lim, time_t now) { - /* It is definitely after when if the earliest time it could be is still - * after when. */ - return (now + ftime_skew - ftime_slop) >= when; + if (lim->rate + lim->last_allowed <= now) { + int res = lim->n_calls_since_last_time + 1; + lim->last_allowed = now; + lim->n_calls_since_last_time = 0; + return res; + } else { + ++lim->n_calls_since_last_time; + return 0; + } } -/** Return true if we think that <b>now</b> is definitely before <b>when</b>. - */ -int -ftime_definitely_before(time_t now, time_t when) + +/** If the rate-limiter <b>lim</b> is ready at <b>now</b>, return a newly + * allocated string indicating how many messages were suppressed, suitable to + * append to a log message. Otherwise return NULL. */ +char * +rate_limit_log(ratelim_t *lim, time_t now) { - /* It is definitely before when if the latest time it could be is still - * before when. */ - return (now + ftime_skew + ftime_slop) < when; + int n; + if ((n = rate_limit_is_ready(lim, now))) { + if (n == 1) { + return tor_strdup(""); + } else { + char *cp=NULL; + tor_asprintf(&cp, + " [%d similar message(s) suppressed in last %d seconds]", + n-1, lim->rate); + return cp; + } + } else { + return NULL; + } } /* ===== @@ -1555,22 +1677,22 @@ check_private_dir(const char *dirname, cpd_check_t check) tor_free(f); if (r) { if (errno != ENOENT) { - log(LOG_WARN, LD_FS, "Directory %s cannot be read: %s", dirname, - strerror(errno)); + log_warn(LD_FS, "Directory %s cannot be read: %s", dirname, + strerror(errno)); return -1; } if (check == CPD_NONE) { - log(LOG_WARN, LD_FS, "Directory %s does not exist.", dirname); + log_warn(LD_FS, "Directory %s does not exist.", dirname); return -1; } else if (check == CPD_CREATE) { log_info(LD_GENERAL, "Creating directory %s", dirname); -#ifdef MS_WINDOWS +#if defined (MS_WINDOWS) && !defined (WINCE) r = mkdir(dirname); #else r = mkdir(dirname, 0700); #endif if (r) { - log(LOG_WARN, LD_FS, "Error creating directory %s: %s", dirname, + log_warn(LD_FS, "Error creating directory %s: %s", dirname, strerror(errno)); return -1; } @@ -1580,7 +1702,7 @@ check_private_dir(const char *dirname, cpd_check_t check) return 0; } if (!(st.st_mode & S_IFDIR)) { - log(LOG_WARN, LD_FS, "%s is not a directory", dirname); + log_warn(LD_FS, "%s is not a directory", dirname); return -1; } #ifndef MS_WINDOWS @@ -1593,7 +1715,7 @@ check_private_dir(const char *dirname, cpd_check_t check) pw = getpwuid(st.st_uid); - log(LOG_WARN, LD_FS, "%s is not owned by this user (%s, %d) but by " + log_warn(LD_FS, "%s is not owned by this user (%s, %d) but by " "%s (%d). Perhaps you are running Tor as the wrong user?", dirname, process_ownername, (int)getuid(), pw ? pw->pw_name : "<unknown>", (int)st.st_uid); @@ -1602,9 +1724,9 @@ check_private_dir(const char *dirname, cpd_check_t check) return -1; } if (st.st_mode & 0077) { - log(LOG_WARN, LD_FS, "Fixing permissions on directory %s", dirname); + log_warn(LD_FS, "Fixing permissions on directory %s", dirname); if (chmod(dirname, 0700)) { - log(LOG_WARN, LD_FS, "Could not chmod directory %s: %s", dirname, + log_warn(LD_FS, "Could not chmod directory %s: %s", dirname, strerror(errno)); return -1; } else { @@ -1635,12 +1757,13 @@ write_str_to_file(const char *fname, const char *str, int bin) } /** Represents a file that we're writing to, with support for atomic commit: - * we can write into a a temporary file, and either remove the file on + * we can write into a temporary file, and either remove the file on * failure, or replace the original file on success. */ struct open_file_t { char *tempname; /**< Name of the temporary file. */ char *filename; /**< Name of the original file. */ - int rename_on_close; /**< Are we using the temporary file or not? */ + unsigned rename_on_close:1; /**< Are we using the temporary file or not? */ + unsigned binary:1; /**< Did we open in binary mode? */ int fd; /**< fd for the open file. */ FILE *stdio_file; /**< stdio wrapper for <b>fd</b>. */ }; @@ -1688,7 +1811,7 @@ start_writing_to_file(const char *fname, int open_flags, int mode, } else { open_name = new_file->tempname = tor_malloc(tempname_len); if (tor_snprintf(new_file->tempname, tempname_len, "%s.tmp", fname)<0) { - log(LOG_WARN, LD_GENERAL, "Failed to generate filename"); + log_warn(LD_GENERAL, "Failed to generate filename"); goto err; } /* We always replace an existing temporary file if there is one. */ @@ -1696,9 +1819,12 @@ start_writing_to_file(const char *fname, int open_flags, int mode, open_flags &= ~O_EXCL; new_file->rename_on_close = 1; } + if (open_flags & O_BINARY) + new_file->binary = 1; - if ((new_file->fd = open(open_name, open_flags, mode)) < 0) { - log(LOG_WARN, LD_FS, "Couldn't open \"%s\" (%s) for writing: %s", + new_file->fd = open(open_name, open_flags, mode); + if (new_file->fd < 0) { + log_warn(LD_FS, "Couldn't open \"%s\" (%s) for writing: %s", open_name, fname, strerror(errno)); goto err; } @@ -1734,7 +1860,8 @@ fdopen_file(open_file_t *file_data) if (file_data->stdio_file) return file_data->stdio_file; tor_assert(file_data->fd >= 0); - if (!(file_data->stdio_file = fdopen(file_data->fd, "a"))) { + if (!(file_data->stdio_file = fdopen(file_data->fd, + file_data->binary?"ab":"a"))) { log_warn(LD_FS, "Couldn't fdopen \"%s\" [%d]: %s", file_data->filename, file_data->fd, strerror(errno)); } @@ -1834,7 +1961,7 @@ write_chunks_to_file_impl(const char *fname, const smartlist_t *chunks, { result = write_all(fd, chunk->bytes, chunk->len, 0); if (result < 0) { - log(LOG_WARN, LD_FS, "Error writing to \"%s\": %s", fname, + log_warn(LD_FS, "Error writing to \"%s\": %s", fname, strerror(errno)); goto err; } @@ -2090,7 +2217,40 @@ unescape_string(const char *s, char **result, size_t *size_out) const char * parse_config_line_from_str(const char *line, char **key_out, char **value_out) { + /* I believe the file format here is supposed to be: + FILE = (EMPTYLINE | LINE)* (EMPTYLASTLINE | LASTLINE)? + + EMPTYLASTLINE = SPACE* | COMMENT + EMPTYLINE = EMPTYLASTLINE NL + SPACE = ' ' | '\r' | '\t' + COMMENT = '#' NOT-NL* + NOT-NL = Any character except '\n' + NL = '\n' + + LASTLINE = SPACE* KEY SPACE* VALUES + LINE = LASTLINE NL + KEY = KEYCHAR+ + KEYCHAR = Any character except ' ', '\r', '\n', '\t', '#', "\" + + VALUES = QUOTEDVALUE | NORMALVALUE + QUOTEDVALUE = QUOTE QVITEM* QUOTE EOLSPACE? + QUOTE = '"' + QVCHAR = KEYCHAR | ESC ('n' | 't' | 'r' | '"' | ESC |'\'' | OCTAL | HEX) + ESC = "\\" + OCTAL = ODIGIT (ODIGIT ODIGIT?)? + HEX = ('x' | 'X') HEXDIGIT HEXDIGIT + ODIGIT = '0' .. '7' + HEXDIGIT = '0'..'9' | 'a' .. 'f' | 'A' .. 'F' + EOLSPACE = SPACE* COMMENT? + + NORMALVALUE = (VALCHAR | ESC ESC_IGNORE | CONTINUATION)* EOLSPACE? + VALCHAR = Any character except ESC, '#', and '\n' + ESC_IGNORE = Any character except '#' or '\n' + CONTINUATION = ESC NL ( COMMENT NL )* + */ + const char *key, *val, *cp; + int continuation = 0; tor_assert(key_out); tor_assert(value_out); @@ -2114,9 +2274,10 @@ parse_config_line_from_str(const char *line, char **key_out, char **value_out) return line; } - /* Skip until the next space. */ + /* Skip until the next space or \ followed by newline. */ key = line; - while (*line && !TOR_ISSPACE(*line) && *line != '#') + while (*line && !TOR_ISSPACE(*line) && *line != '#' && + ! (line[0] == '\\' && line[1] == '\n')) ++line; *key_out = tor_strndup(key, line-key); @@ -2127,7 +2288,7 @@ parse_config_line_from_str(const char *line, char **key_out, char **value_out) val = line; /* Find the end of the line. */ - if (*line == '\"') { + if (*line == '\"') { // XXX No continuation handling is done here if (!(line = unescape_string(line, value_out, NULL))) return NULL; while (*line == ' ' || *line == '\t') @@ -2135,18 +2296,53 @@ parse_config_line_from_str(const char *line, char **key_out, char **value_out) if (*line && *line != '#' && *line != '\n') return NULL; } else { - while (*line && *line != '\n' && *line != '#') - ++line; + /* Look for the end of the line. */ + while (*line && *line != '\n' && (*line != '#' || continuation)) { + if (*line == '\\' && line[1] == '\n') { + continuation = 1; + line += 2; + } else if (*line == '#') { + do { + ++line; + } while (*line && *line != '\n'); + if (*line == '\n') + ++line; + } else { + ++line; + } + } + if (*line == '\n') { cp = line++; } else { cp = line; } + /* Now back cp up to be the last nonspace character */ while (cp>val && TOR_ISSPACE(*(cp-1))) --cp; tor_assert(cp >= val); + + /* Now copy out and decode the value. */ *value_out = tor_strndup(val, cp-val); + if (continuation) { + char *v_out, *v_in; + v_out = v_in = *value_out; + while (*v_in) { + if (*v_in == '#') { + do { + ++v_in; + } while (*v_in && *v_in != '\n'); + if (*v_in == '\n') + ++v_in; + } else if (v_in[0] == '\\' && v_in[1] == '\n') { + v_in += 2; + } else { + *v_out++ = *v_in++; + } + } + *v_out = '\0'; + } } if (*line == '#') { @@ -2165,19 +2361,22 @@ char * expand_filename(const char *filename) { tor_assert(filename); +#ifdef MS_WINDOWS + return tor_strdup(filename); +#else if (*filename == '~') { - size_t len; - char *home, *result; + char *home, *result=NULL; const char *rest; if (filename[1] == '/' || filename[1] == '\0') { home = getenv("HOME"); if (!home) { log_warn(LD_CONFIG, "Couldn't find $HOME environment variable while " - "expanding \"%s\"", filename); - return NULL; + "expanding \"%s\"; defaulting to \"\".", filename); + home = tor_strdup(""); + } else { + home = tor_strdup(home); } - home = tor_strdup(home); rest = strlen(filename)>=2?(filename+2):""; } else { #ifdef HAVE_PWD_H @@ -2204,21 +2403,19 @@ expand_filename(const char *filename) if (strlen(home)>1 && !strcmpend(home,PATH_SEPARATOR)) { home[strlen(home)-1] = '\0'; } - /* Plus one for /, plus one for NUL. - * Round up to 16 in case we can't do math. */ - len = strlen(home)+strlen(rest)+16; - result = tor_malloc(len); - tor_snprintf(result,len,"%s"PATH_SEPARATOR"%s",home,rest); + tor_asprintf(&result,"%s"PATH_SEPARATOR"%s",home,rest); tor_free(home); return result; } else { return tor_strdup(filename); } +#endif } #define MAX_SCANF_WIDTH 9999 -/** DOCDOC */ +/** Helper: given an ASCII-encoded decimal digit, return its numeric value. + * NOTE: requires that its input be in-bounds. */ static int digit_to_num(char d) { @@ -2227,7 +2424,10 @@ digit_to_num(char d) return num; } -/** DOCDOC */ +/** Helper: Read an unsigned int from *<b>bufp</b> of up to <b>width</b> + * characters. (Handle arbitrary width if <b>width</b> is less than 0.) On + * success, store the result in <b>out</b>, advance bufp to the next + * character, and return 0. On failure, return -1. */ static int scan_unsigned(const char **bufp, unsigned *out, int width) { @@ -2254,7 +2454,9 @@ scan_unsigned(const char **bufp, unsigned *out, int width) return 0; } -/** DOCDOC */ +/** Helper: copy up to <b>width</b> non-space characters from <b>bufp</b> to + * <b>out</b>. Make sure <b>out</b> is nul-terminated. Advance <b>bufp</b> + * to the next non-space character or the EOS. */ static int scan_string(const char **bufp, char *out, int width) { @@ -2343,7 +2545,12 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) * and store the results in the corresponding argument fields. Differs from * sscanf in that it: Only handles %u and %Ns. Does not handle arbitrarily * long widths. %u does not consume any space. Is locale-independent. - * Returns -1 on malformed patterns. */ + * Returns -1 on malformed patterns. + * + * (As with other locale-independent functions, we need this to parse data that + * is in ASCII without worrying that the C library's locale-handling will make + * miscellaneous characters look like numbers, spaces, and so on.) + */ int tor_sscanf(const char *buf, const char *pattern, ...) { @@ -2364,20 +2571,32 @@ tor_listdir(const char *dirname) smartlist_t *result; #ifdef MS_WINDOWS char *pattern; + TCHAR tpattern[MAX_PATH] = {0}; + char name[MAX_PATH] = {0}; HANDLE handle; WIN32_FIND_DATA findData; size_t pattern_len = strlen(dirname)+16; pattern = tor_malloc(pattern_len); tor_snprintf(pattern, pattern_len, "%s\\*", dirname); - if (INVALID_HANDLE_VALUE == (handle = FindFirstFile(pattern, &findData))) { +#ifdef UNICODE + mbstowcs(tpattern,pattern,MAX_PATH); +#else + strlcpy(tpattern, pattern, MAX_PATH); +#endif + if (INVALID_HANDLE_VALUE == (handle = FindFirstFile(tpattern, &findData))) { tor_free(pattern); return NULL; } result = smartlist_create(); while (1) { - if (strcmp(findData.cFileName, ".") && - strcmp(findData.cFileName, "..")) { - smartlist_add(result, tor_strdup(findData.cFileName)); +#ifdef UNICODE + wcstombs(name,findData.cFileName,MAX_PATH); +#else + strlcpy(name,findData.cFileName,sizeof(name)); +#endif + if (strcmp(name, ".") && + strcmp(name, "..")) { + smartlist_add(result, tor_strdup(name)); } if (!FindNextFile(handle, &findData)) { DWORD err; @@ -2576,3 +2795,18 @@ write_pidfile(char *filename) } } +#ifdef MS_WINDOWS +HANDLE +load_windows_system_library(const TCHAR *library_name) +{ + TCHAR path[MAX_PATH]; + unsigned n; + n = GetSystemDirectory(path, MAX_PATH); + if (n == 0 || n + _tcslen(library_name) + 2 >= MAX_PATH) + return 0; + _tcscat(path, TEXT("\\")); + _tcscat(path, library_name); + return LoadLibrary(path); +} +#endif + diff --git a/src/common/util.h b/src/common/util.h index fcea9c5b8..6b5485674 100644 --- a/src/common/util.h +++ b/src/common/util.h @@ -43,7 +43,7 @@ * stderr. */ #define tor_assert(expr) STMT_BEGIN \ if (PREDICT_UNLIKELY(!(expr))) { \ - log(LOG_ERR, LD_BUG, "%s:%d: %s: Assertion %s failed; aborting.", \ + log_err(LD_BUG, "%s:%d: %s: Assertion %s failed; aborting.", \ _SHORT_FILE_, __LINE__, __func__, #expr); \ fprintf(stderr,"%s:%d %s: Assertion %s failed; aborting.\n", \ _SHORT_FILE_, __LINE__, __func__, #expr); \ @@ -152,8 +152,18 @@ void tor_log_mallinfo(int severity); #define bool_neq(a,b) (!(a)!=!(b)) /* Math functions */ +double tor_mathlog(double d) ATTR_CONST; +long tor_lround(double d) ATTR_CONST; int tor_log2(uint64_t u64) ATTR_CONST; uint64_t round_to_power_of_2(uint64_t u64); +unsigned round_to_next_multiple_of(unsigned number, unsigned divisor); +uint32_t round_uint32_to_next_multiple_of(uint32_t number, uint32_t divisor); +uint64_t round_uint64_to_next_multiple_of(uint64_t number, uint64_t divisor); + +/* Compute the CEIL of <b>a</b> divided by <b>b</b>, for nonnegative <b>a</b> + * and positive <b>b</b>. Works on integer types only. Not defined if a+b can + * overflow. */ +#define CEIL_DIV(a,b) (((a)+(b)-1)/(b)) /* String manipulation */ @@ -179,6 +189,8 @@ long tor_parse_long(const char *s, int base, long min, long max, int *ok, char **next); unsigned long tor_parse_ulong(const char *s, int base, unsigned long min, unsigned long max, int *ok, char **next); +double tor_parse_double(const char *s, double min, double max, int *ok, + char **next); uint64_t tor_parse_uint64(const char *s, int base, uint64_t min, uint64_t max, int *ok, char **next); const char *hex_str(const char *from, size_t fromlen) ATTR_NONNULL((1)); @@ -188,8 +200,11 @@ const char *eat_whitespace_no_nl(const char *s) ATTR_PURE; const char *eat_whitespace_eos_no_nl(const char *s, const char *eos) ATTR_PURE; const char *find_whitespace(const char *s) ATTR_PURE; const char *find_whitespace_eos(const char *s, const char *eos) ATTR_PURE; +const char *find_str_at_start_of_line(const char *haystack, const char *needle) + ATTR_PURE; int tor_mem_is_zero(const char *mem, size_t len) ATTR_PURE; int tor_digest_is_zero(const char *digest) ATTR_PURE; +int tor_digest256_is_zero(const char *digest) ATTR_PURE; char *esc_for_log(const char *string) ATTR_MALLOC; const char *escaped(const char *string); struct smartlist_t; @@ -207,7 +222,11 @@ void base16_encode(char *dest, size_t destlen, const char *src, size_t srclen); int base16_decode(char *dest, size_t destlen, const char *src, size_t srclen); /* Time helpers */ +double tv_to_double(const struct timeval *tv); +int64_t tv_to_msec(const struct timeval *tv); +int64_t tv_to_usec(const struct timeval *tv); long tv_udiff(const struct timeval *start, const struct timeval *end); +long tv_mdiff(const struct timeval *start, const struct timeval *end); time_t tor_timegm(struct tm *tm); #define RFC1123_TIME_LEN 29 void format_rfc1123_time(char *buf, time_t t); @@ -228,15 +247,32 @@ time_t approx_time(void); void update_approx_time(time_t now); #endif -/* Fuzzy time. */ -void ftime_set_maximum_sloppiness(int seconds); -void ftime_set_estimated_skew(int seconds); -/* typedef struct ftime_t { time_t earliest; time_t latest; } ftime_t; */ -/* void ftime_get_window(time_t now, ftime_t *ft_out); */ -int ftime_maybe_after(time_t now, time_t when); -int ftime_maybe_before(time_t now, time_t when); -int ftime_definitely_after(time_t now, time_t when); -int ftime_definitely_before(time_t now, time_t when); +/* Rate-limiter */ + +/** A ratelim_t remembers how often an event is occurring, and how often + * it's allowed to occur. Typical usage is something like: + * + <pre> + if (possibly_very_frequent_event()) { + const int INTERVAL = 300; + static ratelim_t warning_limit = RATELIM_INIT(INTERVAL); + char *m; + if ((m = rate_limit_log(&warning_limit, approx_time()))) { + log_warn(LD_GENERAL, "The event occurred!%s", m); + tor_free(m); + } + } + </pre> + */ +typedef struct ratelim_t { + int rate; + time_t last_allowed; + int n_calls_since_last_time; +} ratelim_t; + +#define RATELIM_INIT(r) { (r), 0, 0 } + +char *rate_limit_log(ratelim_t *lim, time_t now); /* File helpers */ ssize_t write_all(int fd, const char *buf, size_t count, int isSocket); @@ -294,5 +330,11 @@ void start_daemon(void); void finish_daemon(const char *desired_cwd); void write_pidfile(char *filename); +#ifdef MS_WINDOWS +HANDLE load_windows_system_library(const TCHAR *library_name); +#endif + +const char *libor_get_digests(void); + #endif diff --git a/src/common/util_codedigest.c b/src/common/util_codedigest.c new file mode 100644 index 000000000..88fe508b9 --- /dev/null +++ b/src/common/util_codedigest.c @@ -0,0 +1,11 @@ + +#include "util.h" + +const char * +libor_get_digests(void) +{ + return "" +#include "common_sha1.i" + ; +} + diff --git a/src/config/torrc.bridge.in b/src/config/torrc.bridge.in new file mode 100644 index 000000000..557b7adf4 --- /dev/null +++ b/src/config/torrc.bridge.in @@ -0,0 +1,171 @@ +## Configuration file for a typical Tor user +## Last updated 16 July 2009 for Tor 0.2.2.1-alpha. +## (May or may not work for much older or much newer versions of Tor.) +## +## Lines that begin with "## " try to explain what's going on. Lines +## that begin with just "#" are disabled commands: you can enable them +## by removing the "#" symbol. +## +## See 'man tor', or https://www.torproject.org/tor-manual.html, +## for more options you can use in this file. +## +## Tor will look for this file in various places based on your platform: +## https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#torrc + + +## Replace this with "SocksPort 0" if you plan to run Tor only as a +## relay, and not make any local application connections yourself. +SocksPort 9050 # what port to open for local application connections +SocksListenAddress 127.0.0.1 # accept connections only from localhost +#SocksListenAddress 192.168.0.1:9100 # listen on this IP:port also + +## Entry policies to allow/deny SOCKS requests based on IP address. +## First entry that matches wins. If no SocksPolicy is set, we accept +## all (and only) requests from SocksListenAddress. +#SocksPolicy accept 192.168.0.0/16 +#SocksPolicy reject * + +## Logs go to stdout at level "notice" unless redirected by something +## else, like one of the below lines. You can have as many Log lines as +## you want. +## +## We advise using "notice" in most cases, since anything more verbose +## may provide sensitive information to an attacker who obtains the logs. +## +## Send all messages of level 'notice' or higher to @LOCALSTATEDIR@/log/tor/notices.log +#Log notice file @LOCALSTATEDIR@/log/tor/notices.log +## Send every possible message to @LOCALSTATEDIR@/log/tor/debug.log +#Log debug file @LOCALSTATEDIR@/log/tor/debug.log +## Use the system log instead of Tor's logfiles +#Log notice syslog +## To send all messages to stderr: +#Log debug stderr + +## Uncomment this to start the process in the background... or use +## --runasdaemon 1 on the command line. This is ignored on Windows; +## see the FAQ entry if you want Tor to run as an NT service. +#RunAsDaemon 1 + +## The directory for keeping all the keys/etc. By default, we store +## things in $HOME/.tor on Unix, and in Application Data\tor on Windows. +#DataDirectory @LOCALSTATEDIR@/lib/tor + +## The port on which Tor will listen for local connections from Tor +## controller applications, as documented in control-spec.txt. +#ControlPort 9051 +## If you enable the controlport, be sure to enable one of these +## authentication methods, to prevent attackers from accessing it. +#HashedControlPassword 16:872860B76453A77D60CA2BB8C1A7042072093276A3D701AD684053EC4C +#CookieAuthentication 1 + +############### This section is just for location-hidden services ### + +## Once you have configured a hidden service, you can look at the +## contents of the file ".../hidden_service/hostname" for the address +## to tell people. +## +## HiddenServicePort x y:z says to redirect requests on port x to the +## address y:z. + +#HiddenServiceDir @LOCALSTATEDIR@/lib/tor/hidden_service/ +#HiddenServicePort 80 127.0.0.1:80 + +#HiddenServiceDir @LOCALSTATEDIR@/lib/tor/other_hidden_service/ +#HiddenServicePort 80 127.0.0.1:80 +#HiddenServicePort 22 127.0.0.1:22 + +################ This section is just for relays ##################### +# +## See https://www.torproject.org/docs/tor-doc-relay for details. + +## Required: what port to advertise for incoming Tor connections. +ORPort 9001 +## If you want to listen on a port other than the one advertised +## in ORPort (e.g. to advertise 443 but bind to 9090), uncomment the +## line below too. You'll need to do ipchains or other port forwarding +## yourself to make this work. +#ORListenAddress 0.0.0.0:9090 + +## A handle for your relay, so people don't have to refer to it by key. +Nickname Unnamed + +## The IP address or full DNS name for your relay. Leave commented out +## and Tor will guess. +#Address noname.example.com + +## Define these to limit how much relayed traffic you will allow. Your +## own traffic is still unthrottled. Note that RelayBandwidthRate must +## be at least 20 KB. +#RelayBandwidthRate 100 KB # Throttle traffic to 100KB/s (800Kbps) +#RelayBandwidthBurst 200 KB # But allow bursts up to 200KB/s (1600Kbps) +RelayBandwidthBurst 10485760 +RelayBandwidthRate 5242880 + +## Use these to restrict the maximum traffic per day, week, or month. +## Note that this threshold applies to sent _and_ to received bytes, +## not to their sum: Setting "4 GB" may allow up to 8 GB +## total before hibernating. +## +## Set a maximum of 4 gigabytes each way per period. +#AccountingMax 4 GB +## Each period starts daily at midnight (AccountingMax is per day) +#AccountingStart day 00:00 +## Each period starts on the 3rd of the month at 15:00 (AccountingMax +## is per month) +#AccountingStart month 3 15:00 + +## Contact info to be published in the directory, so we can contact you +## if your relay is misconfigured or something else goes wrong. Google +## indexes this, so spammers might also collect it. +#ContactInfo Random Person <nobody AT example dot com> +## You might also include your PGP or GPG fingerprint if you have one: +#ContactInfo 1234D/FFFFFFFF Random Person <nobody AT example dot com> + +## Uncomment this to mirror directory information for others. Please do +## if you have enough bandwidth. +DirPort 9030 # what port to advertise for directory connections +## If you want to listen on a port other than the one advertised +## in DirPort (e.g. to advertise 80 but bind to 9091), uncomment the line +## below too. You'll need to do ipchains or other port forwarding yourself +## to make this work. +#DirListenAddress 0.0.0.0:9091 +## Uncomment to return an arbitrary blob of html on your DirPort. Now you +## can explain what Tor is if anybody wonders why your IP address is +## contacting them. See contrib/tor-exit-notice.html in Tor's source +## distribution for a sample. +#DirPortFrontPage @CONFDIR@/tor-exit-notice.html + +## Uncomment this if you run more than one Tor relay, and add the identity +## key fingerprint of each Tor relay you control, even if they're on +## different networks. You declare it here so Tor clients can avoid +## using more than one of your relays in a single circuit. See +## https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#MultipleServers +#MyFamily $keyid,$keyid,... + +## A comma-separated list of exit policies. They're considered first +## to last, and the first match wins. If you want to _replace_ +## the default exit policy, end this with either a reject *:* or an +## accept *:*. Otherwise, you're _augmenting_ (prepending to) the +## default exit policy. Leave commented to just use the default, which is +## described in the man page or at +## https://www.torproject.org/documentation.html +## +## Look at https://www.torproject.org/faq-abuse.html#TypicalAbuses +## for issues you might encounter if you use the default exit policy. +## +## If certain IPs and ports are blocked externally, e.g. by your firewall, +## you should update your exit policy to reflect this -- otherwise Tor +## users will be told that those destinations are down. +## +#ExitPolicy accept *:6660-6667,reject *:* # allow irc ports but no more +#ExitPolicy accept *:119 # accept nntp as well as default exit policy +#ExitPolicy reject *:* # no exits allowed +# +## Bridge relays (or "bridges") are Tor relays that aren't listed in the +## main directory. Since there is no complete public list of them, even if an +## ISP is filtering connections to all the known Tor relays, they probably +## won't be able to block all the bridges. Also, websites won't treat you +## differently because they won't know you're running Tor. If you can +## be a real relay, please do; but if not, be a bridge! +BridgeRelay 1 +ExitPolicy reject *:* diff --git a/src/config/torrc.complete.in b/src/config/torrc.complete.in deleted file mode 100644 index 310458a5c..000000000 --- a/src/config/torrc.complete.in +++ /dev/null @@ -1,533 +0,0 @@ -# $Id$ -# Last updated on $Date$ -#################################################################### -## This config file is divided into four sections. They are: -## 1. Global Options (clients and servers) -## 2. Client Options Only -## 3. Server Options Only -## 4. Directory Server Options (for running your own Tor network) -## 5. Hidden Service Options (clients and servers) -## -## The conventions used are: -## double hash (##) is for summary text about the config option; -## single hash (#) is for the config option; and, -## the config option is always after the text. -#################################################################### - - -## Section 1: Global Options (clients and servers) - -## A token bucket limits the average incoming bandwidth on this node -## to the specified number of bytes per second. (Default: 2MB) -#BandwidthRate N bytes|KB|MB|GB|TB - -## Limit the maximum token bucket size (also known as the burst) to -## the given number of bytes. (Default: 5 MB) -#BandwidthBurst N bytes|KB|MB|GB|TB - -## If set, we will not advertise more than this amount of bandwidth -## for our BandwidthRate. Server operators who want to reduce the -## number of clients who ask to build circuits through them (since -## this is proportional to advertised bandwidth rate) can thus -## reduce the CPU demands on their server without impacting -## network performance. -#MaxAdvertisedBandwidth N bytes|KB|MB|GB|TB - -## If set, Tor will accept connections from the same machine -## (localhost only) on this port, and allow those connections to -## control the Tor process using the Tor Control Protocol -## (described in control-spec.txt). Note: unless you also specify -## one of HashedControlPassword or CookieAuthentication, setting -## this option will cause Tor to allow any process on the local -## host to control it. -#ControlPort Port - -## Don’t allow any connections on the control port except when the -## other process knows the password whose one-way hash is -## hashed_password. You can compute the hash of a password by -## running "tor --hash-password password". -#HashedControlPassword hashed_password - -## If this option is set to 1, don’t allow any connections on the -## control port except when the connecting process knows the -## contents of a file named "control_auth_cookie", which Tor will -## create in its data directory. This authentication method -## should only be used on systems with good filesystem security. -## (Default: 0) -#CookieAuthentication 0|1 - -## Store working data in DIR (Default: /usr/local/var/lib/tor) -#DataDirectory DIR - -## Every time the specified period elapses, Tor downloads a direc- -## tory. A directory contains a signed list of all known servers -## as well as their current liveness status. A value of "0 sec- -## onds" tells Tor to choose an appropriate default. -## (Default: 1 hour for clients, 20 minutes for servers) -#DirFetchPeriod N seconds|minutes|hours|days|weeks - -## Tor only trusts directories signed with one of these keys, and -## uses the given addresses to connect to the trusted directory -## servers. If no DirServer lines are specified, Tor uses the built-in -## defaults (moria1, moria2, tor26), so you can leave this alone unless -## you need to change it. -## -## WARNING! Changing these options will make your Tor behave -## differently from everyone else's, and hurt your anonymity. Even -## uncommenting these lines is a bad idea. They are the defaults now, -## but the defaults may change in the future, leaving you behind. -## -#DirServer moria1 v1 18.244.0.188:9031 FFCB 46DB 1339 DA84 674C 70D7 CB58 6434 C437 0441 -#DirServer moria2 v1 18.244.0.114:80 719B E45D E224 B607 C537 07D0 E214 3E2D 423E 74CF -#DirServer tor26 v1 86.59.21.38:80 847B 1F85 0344 D787 6491 A548 92F9 0493 4E4E B85D - -## On startup, setgid to this user. -#Group GID - -## Tor will make all its directory requests through this host:port -## (or host:80 if port is not specified), rather than connecting -## directly to any directory servers. -#HttpProxy host[:port] - -## If defined, Tor will use this username:password for Basic Http -## proxy authentication, as in RFC 2617. This is currently the -## only form of Http proxy authentication that Tor supports; feel -## free to submit a patch if you want it to support others. -#HttpProxyAuthenticator username:password - -## Tor will make all its OR (SSL) connections through this -## host:port (or host:443 if port is not specified), via HTTP CON- -## NECT rather than connecting directly to servers. You may want -## to set FascistFirewall to restrict the set of ports you might -## try to connect to, if your Https proxy only allows connecting -## to certain ports. -#HttpsProxy host[:port] - -## If defined, Tor will use this username:password for Basic Https -## proxy authentication, as in RFC 2617. This is currently the -## only form of Https proxy authentication that Tor supports; feel -## free to submit a patch if you want it to support others. -#HttpsProxyAuthenticator username:password - -## To keep firewalls from expiring connections, send a padding -## keepalive cell every NUM seconds on open connections that are -## in use. If the connection has no open circuits, it will instead -## be closed after NUM seconds of idleness. (Default: 5 minutes) -#KeepalivePeriod NUM - -## Send all messages between minSeverity and maxSeverity to the -## standard output stream, the standard error stream, or to the -## system log. (The "syslog" value is only supported on Unix.) -## Recognized severity levels are debug, info, notice, warn, and -## err. If only one severity level is given, all messages of that -## level or higher will be sent to the listed destination. -#Log minSeverity[-maxSeverity] stderr|stdout|syslog - -## As above, but send log messages to the listed filename. The -## "Log" option may appear more than once in a configuration file. -## Messages are sent to all the logs that match their severity -## level. -#Log minSeverity[-maxSeverity] file FILENAME - -## Maximum number of simultaneous sockets allowed. You probably -## don’t need to adjust this. (Default: 1024) -#MaxConn NUM - -## Make all outbound connections originate from the IP address -## specified. This is only useful when you have multiple network -## interfaces, and you want all of Tor’s outgoing connections to -## use a single one. -#OutboundBindAddress IP - -## On startup, write our PID to FILE. On clean shutdown, remove -## FILE. -#PIDFile FILE - -## If 1, Tor forks and daemonizes to the background. (Default: 0) -#RunAsDaemon 0|1 - -## If 1, Tor replaces potentially sensitive strings in the logs -## (e.g. addresses) with the string [scrubbed]. This way logs can -## still be useful, but they don’t leave behind personally identi- -## fying information about what sites a user might have visited. -## (Default: 1) -#SafeLogging 0|1 - -## Every time the specified period elapses, Tor downloads signed -## status information about the current state of known servers. A -## value of "0 seconds" tells Tor to choose an appropriate -## default. (Default: 30 minutes for clients, 15 minutes for -## servers) -#StatusFetchPeriod N seconds|minutes|hours|days|weeks - -## On startup, setuid to this user. -#User UID - -## If non-zero, try to use crypto hardware acceleration when -## available. (Default: 1) -#HardwareAccel 0|1 - - -## Section 2: Client Options Only - -## Where on our circuits should we allow Tor servers that the -## directory servers haven’t authenticated as "verified"? -## (Default: middle,rendezvous) -#AllowUnverifiedNodes entry|exit|middle|introduction|rendezvous|... - -## If set to 1, Tor will under no circumstances run as a server. -## The default is to run as a client unless ORPort is configured. -## (Usually, you don’t need to set this; Tor is pretty smart at -## figuring out whether you are reliable and high-bandwidth enough -## to be a useful server.) -## This option will likely be deprecated in the future; see the -## NoPublish option below. (Default: 0) -#ClientOnly 0|1 - -## A list of preferred nodes to use for the first hop in the -## circuit, if possible. -#EntryNodes nickname,nickname,... - -## A list of preferred nodes to use for the last hop in the -## circuit, if possible. -#ExitNodes nickname,nickname,... - -## A list of nodes to never use when building a circuit. -#ExcludeNodes nickname,nickname,... - -## If 1, Tor will never use any nodes besides those listed in -## "exitnodes" for the last hop of a circuit. -#StrictExitNodes 0|1 - -## If 1, Tor will never use any nodes besides those listed in -## "entrynodes" for the first hop of a circuit. -#StrictEntryNodes 0|1 - -## If 1, Tor will only create outgoing connections to ORs running -## on ports that your firewall allows (defaults to 80 and 443; see -## FirewallPorts). This will allow you to run Tor as a client -## behind a firewall with restrictive policies, but will not allow -## you to run as a server behind such a firewall. -#FascistFirewall 0|1 - -## A list of ports that your firewall allows you to connect to. -## Only used when FascistFirewall is set. (Default: 80, 443) -#FirewallPorts PORTS - -## A comma-separated list of IPs that your firewall allows you to -## connect to. Only used when FascistFirewall is set. The format -## is as for the addresses in ExitPolicy. -## For example, ’FirewallIPs 99.0.0.0/8, *:80’ means that your -## firewall allows connections to everything inside net 99, and -## to port 80 outside. -#FirewallIPs ADDR[/MASK][:PORT]... - -## A list of ports for services that tend to have long-running -## connections (e.g. chat and interactive shells). Circuits for -## streams that use these ports will contain only high-uptime -## nodes, to reduce the chance that a node will go down before the -## stream is finished. (Default: 21, 22, 706, 1863, 5050, 5190, -## 5222, 5223, 6667, 8300, 8888) -#LongLivedPorts PORTS - -## When a request for address arrives to Tor, it will rewrite it -## to newaddress before processing it. For example, if you always -## want connections to www.indymedia.org to exit via torserver -## (where torserver is the nickname of the server), -## use "MapAddress www.indymedia.org www.indymedia.org.torserver.exit". -#MapAddress address newaddress - -## Every NUM seconds consider whether to build a new circuit. -## (Default: 30 seconds) -#NewCircuitPeriod NUM - -## Feel free to reuse a circuit that was first used at most NUM -## seconds ago, but never attach a new stream to a circuit that is -## too old. (Default: 10 minutes) -#MaxCircuitDirtiness NUM - -## The named Tor servers constitute a "family" of similar or co- -## administered servers, so never use any two of them in the same -## circuit. Defining a NodeFamily is only needed when a server -## doesn’t list the family itself (with MyFamily). This option can -## be used multiple times. -#NodeFamily nickname,nickname,... - -## A list of preferred nodes to use for the rendezvous point, if -## possible. -#RendNodes nickname,nickname,... - -## A list of nodes to never use when choosing a rendezvous point. -#RendExcludeNodes nickname,nickname,... - -## Advertise this port to listen for connections from SOCKS-speak- -## ing applications. Set this to 0 if you don’t want to allow -## application connections. (Default: 9050) -#SOCKSPort PORT - -## Bind to this address to listen for connections from SOCKS- -## speaking applications. (Default: 127.0.0.1) You can also spec- -## ify a port (e.g. 192.168.0.1:9100). This directive can be spec- -## ified multiple times to bind to multiple addresses/ports. -#SOCKSBindAddress IP[:PORT] - -## Set an entrance policy for this server, to limit who can con- -## nect to the SOCKS ports. The policies have the same form as -## exit policies below. -#SOCKSPolicy policy,policy,... - -## For each value in the comma separated list, Tor will track -## recent connections to hosts that match this value and attempt -## to reuse the same exit node for each. If the value is prepended -## with a ’.’, it is treated as matching an entire domain. If one -## of the values is just a ’.’, it means match everything. This -## option is useful if you frequently connect to sites that will -## expire all your authentication cookies (ie log you out) if your -## IP address changes. Note that this option does have the disad- -## vantage of making it more clear that a given history is associ- -## ated with a single user. However, most people who would wish to -## observe this will observe it through cookies or other protocol- -## specific means anyhow. -#TrackHostExits host,.domain,... - -## Since exit servers go up and down, it is desirable to expire -## the association between host and exit server after NUM seconds. -## The default is 1800 seconds (30 minutes). -#TrackHostExitsExpire NUM - -## If this option is set to 1, we pick a few entry servers as our -## "helpers", and try to use only those fixed entry servers. This -## is desirable, because constantly changing servers increases the -## odds that an adversary who owns some servers will observe a -## fraction of your paths. (Defaults to 0; will eventually -## default to 1.) -#UseHelperNodes 0|1 - -## If UseHelperNodes is set to 1, we will try to pick a total of -## NUM helper nodes as entries for our circuits. (Defaults to 3.) -#NumHelperNodes NUM - - -## Section 3: Server Options Only - -## The IP or fqdn of this server (e.g. moria.mit.edu). You can -## leave this unset, and Tor will guess your IP. -#Address address - -## Administrative contact information for server. -#ContactInfo email_address - -## Set an exit policy for this server. Each policy is of the form -## "accept|reject ADDR[/MASK][:PORT]". If /MASK is omitted then -## this policy just applies to the host given. Instead of giving -## a host or network you can also use "*" to denote the universe -## (0.0.0.0/0). PORT can be a single port number, an interval of -## ports "FROM_PORT-TO_PORT", or "*". If PORT is omitted, that -## means "*". -## -## For example, "reject 127.0.0.1:*,reject 192.168.1.0/24:*,accept -## *:*" would reject any traffic destined for localhost and any -## 192.168.1.* address, but accept anything else. -## -## This directive can be specified multiple times so you don’t -## have to put it all on one line. -## -## See RFC 3330 for more details about internal and reserved IP -## address space. Policies are considered first to last, and the -## first match wins. If you want to _replace_ the default exit -## policy, end your exit policy with either a reject *:* or an -## accept *:*. Otherwise, you’re _augmenting_ (prepending to) the -## default exit policy. The default exit policy is: -## reject 0.0.0.0/8 -## reject 169.254.0.0/16 -## reject 127.0.0.0/8 -## reject 192.168.0.0/16 -## reject 10.0.0.0/8 -## reject 172.16.0.0/12 -## reject *:25 -## reject *:119 -## reject *:135-139 -## reject *:445 -## reject *:1214 -## reject *:4661-4666 -## reject *:6346-6429 -## reject *:6699 -## reject *:6881-6999 -## accept *:* -#ExitPolicy policy,policy,... - -## If you have more than this number of onionskins queued for -## decrypt, reject new ones. (Default: 100) -#MaxOnionsPending NUM - -## Declare that this Tor server is controlled or administered by a -## group or organization identical or similar to that of the other -## named servers. When two servers both declare that they are in -## the same ’family’, Tor clients will not use them in the same -## circuit. (Each server only needs to list the other servers in -## its family; it doesn’t need to list itself, but it won’t hurt.) -#MyFamily nickname,nickname,... - -## Set the server’s nickname to ’name’. -#Nickname name - -## If you set NoPublish 1, Tor will act as a server if you have an -## ORPort defined, but it will not publish its descriptor to the -## dirservers. This option is useful if you're testing out your -## server, or if you're using alternate dirservers (e.g. for other -## Tor networks such as Blossom). (Default: 0) -#NoPublish 0|1 - -## How many processes to use at once for decrypting onionskins. -## (Default: 1) -#NumCPUs num - -## Advertise this port to listen for connections from Tor clients -## and servers. -#ORPort PORT - -## Bind to this IP address to listen for connections from Tor -## clients and servers. If you specify a port, bind to this port -## rather than the one specified in ORPort. (Default: 0.0.0.0) -#ORBindAddress IP[:PORT] - -## Whenever an outgoing connection tries to connect to one of a -## given set of addresses, connect to target (an address:port -## pair) instead. The address pattern is given in the same format -## as for an exit policy. The address translation applies after -## exit policies are applied. Multiple RedirectExit options can -## be used: once any one has matched successfully, no subsequent -## rules are considered. You can specify that no redirection is -## to be performed on a given set of addresses by using the spe- -## cial target string "pass", which prevents subsequent rules from -## being considered. -#RedirectExit pattern target - -## When we get a SIGINT and we're a server, we begin shutting -## down: we close listeners and start refusing new circuits. After -## NUM seconds, we exit. If we get a second SIGINT, we exit imme- -## diately. (Default: 30 seconds) -#ShutdownWaitLengthNUM - -## Every time the specified period elapses, Tor uploads its server -## descriptors to the directory servers. This information is also -## uploaded whenever it changes. (Default: 20 minutes) -#DirPostPeriod N seconds|minutes|hours|days|weeks - -## A token bucket limits the average relayed bandwidth (server -## traffic only, not client traffic) on this node to the specified -## number of bytes per second. -#RelayBandwidthRate N bytes|KB|MB|GB|TB - -## Limit the maximum token bucket size (also known as the burst) for -## relayed traffic (server traffic only, not client traffic) to the -## given number of bytes. -#RelayBandwidthBurst N bytes|KB|MB|GB|TB - -## Never send more than the specified number of bytes in a given -## accounting period, or receive more than that number in the -## period. For example, with AccountingMax set to 1 GB, a server -## could send 900 MB and receive 800 MB and continue running. It -## will only hibernate once one of the two reaches 1 GB. When the -## number of bytes is exhausted, Tor will hibernate until some -## time in the next accounting period. To prevent all servers -## from waking at the same time, Tor will also wait until a random -## point in each period before waking up. If you have bandwidth -## cost issues, enabling hibernation is preferable to setting a -## low bandwidth, since it provides users with a collection of -## fast servers that are up some of the time, which is more useful -## than a set of slow servers that are always "available". -#AccountingMax N bytes|KB|MB|GB|TB - -## Specify how long accounting periods last. If month is given, -## each accounting period runs from the time HH:MM on the dayth -## day of one month to the same day and time of the next. (The -## day must be between 1 and 28.) If week is given, each account- -## ing period runs from the time HH:MM of the dayth day of one -## week to the same day and time of the next week, with Monday as -## day 1 and Sunday as day 7. If day is given, each accounting -## period runs from the time HH:MM each day to the same time on -## the next day. All times are local, and given in 24-hour time. -## (Defaults to "month 1 0:00".) -#AccountingStart day|week|month [day] HH:MM - - -## Section 4: Directory Server Options (for running your own Tor -## network) - -## When this option is set to 1, Tor operates as an authoritative -## directory server. Instead of caching the directory, it gener- -## ates its own list of good servers, signs it, and sends that to -## the clients. Unless the clients already have you listed as a -## trusted directory, you probably do not want to set this option. -## Please coordinate with the other admins at -## tor-ops@freehaven.net if you think you should be a directory. -#AuthoritativeDirectory 0|1 - -## Advertise the directory service on this port. -#DirPort PORT - -## Bind the directory service to this address. If you specify a -## port, bind to this port rather than the one specified in DirPort. -## (Default: 0.0.0.0) -#DirBindAddress IP[:PORT] - -## Set an entrance policy for this server, to limit who can con- -## nect to the directory ports. The policies have the same form -## as exit policies above. -#DirPolicy policy,policy,... - -## STRING is a command-separated list of Tor versions currently -## believed to be safe. The list is included in each directory, -## and nodes which pull down the directory learn whether they need -## to upgrade. This option can appear multiple times: the values -## from multiple lines are spliced together. -#RecommendedVersions STRING - - -## If set to 1, Tor will accept router descriptors with arbitrary -## "Address" elements. Otherwise, if the address is not an IP or -## is a private IP, it will reject the router descriptor. Defaults -## to 0. -#DirAllowPrivateAddresses 0|1 - -## If set to 1, Tor tries to build circuits through all of the -## servers it knows about, so it can tell which are up and which -## are down. This option is only useful for authoritative direc- -## tories, so you probably don't want to use it. -#RunTesting 0|1 - -## Section 5: Hidden Service Options (clients and servers) - -## Store data files for a hidden service in DIRECTORY. Every hid- -## den service must have a separate directory. You may use this -## option multiple times to specify multiple services. -#HiddenServiceDir DIRECTORY - -## Configure a virtual port VIRTPORT for a hidden service. You -## may use this option multiple times; each time applies to the -## service using the most recent hiddenservicedir. By default, -## this option maps the virtual port to the same port on -## 127.0.0.1. You may override the target port, address, or both -## by specifying a target of addr, port, or addr:port. -#HiddenServicePort VIRTPORT [TARGET] - -## If possible, use the specified nodes as introduction points for -## the hidden service. If this is left unset, Tor will be smart -## and pick some reasonable ones; most people can leave this unset. -#HiddenServiceNodes nickname,nickname,... - -## Do not use the specified nodes as introduction points for the -## hidden service. In normal use there is no reason to set this. -#HiddenServiceExcludeNodes nickname,nickname,... - -## Publish the given rendezvous service descriptor versions for the -## hidden service. -#HiddenServiceVersion 0,2 - -## Every time the specified period elapses, Tor uploads any ren- -## dezvous service descriptors to the directory servers. This -## information is also uploaded whenever it changes. -## (Default: 20 minutes) -#RendPostPeriod N seconds|minutes|hours|days|weeks -# diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in index d0b1ee159..f0c78ce5a 100644 --- a/src/config/torrc.sample.in +++ b/src/config/torrc.sample.in @@ -1,5 +1,5 @@ ## Configuration file for a typical Tor user -## Last updated 12 April 2009 for Tor 0.2.1.14-rc. +## Last updated 16 July 2009 for Tor 0.2.2.1-alpha. ## (May or may not work for much older or much newer versions of Tor.) ## ## Lines that begin with "## " try to explain what's going on. Lines @@ -95,9 +95,22 @@ SocksListenAddress 127.0.0.1 # accept connections only from localhost ## Define these to limit how much relayed traffic you will allow. Your ## own traffic is still unthrottled. Note that RelayBandwidthRate must -## be at least 20 KBytes. -#RelayBandwidthRate 100 KBytes # Throttle traffic to 100KB/s (800Kbps) -#RelayBandwidthBurst 200 KBytes # But allow bursts up to 200KB/s (1600Kbps) +## be at least 20 KB. +#RelayBandwidthRate 100 KB # Throttle traffic to 100KB/s (800Kbps) +#RelayBandwidthBurst 200 KB # But allow bursts up to 200KB/s (1600Kbps) + +## Use these to restrict the maximum traffic per day, week, or month. +## Note that this threshold applies to sent _and_ to received bytes, +## not to their sum: Setting "4 GB" may allow up to 8 GB +## total before hibernating. +## +## Set a maximum of 4 gigabytes each way per period. +#AccountingMax 4 GB +## Each period starts daily at midnight (AccountingMax is per day) +#AccountingStart day 00:00 +## Each period starts on the 3rd of the month at 15:00 (AccountingMax +## is per month) +#AccountingStart month 3 15:00 ## Contact info to be published in the directory, so we can contact you ## if your relay is misconfigured or something else goes wrong. Google @@ -116,8 +129,9 @@ SocksListenAddress 127.0.0.1 # accept connections only from localhost #DirListenAddress 0.0.0.0:9091 ## Uncomment to return an arbitrary blob of html on your DirPort. Now you ## can explain what Tor is if anybody wonders why your IP address is -## contacting them. See contrib/tor-exit-notice.html for a sample. -#DirPortFrontPage /etc/tor/exit-notice.html +## contacting them. See contrib/tor-exit-notice.html in Tor's source +## distribution for a sample. +#DirPortFrontPage @CONFDIR@/tor-exit-notice.html ## Uncomment this if you run more than one Tor relay, and add the identity ## key fingerprint of each Tor relay you control, even if they're on diff --git a/src/or/Makefile.am b/src/or/Makefile.am index b56d49643..a9ac3cdee 100644 --- a/src/or/Makefile.am +++ b/src/or/Makefile.am @@ -1,8 +1,5 @@ -TESTS = test - -noinst_PROGRAMS = test - bin_PROGRAMS = tor +noinst_LIBRARIES = libtor.a if BUILD_NT_SERVICES tor_platform_source=ntmain.c @@ -10,18 +7,30 @@ else tor_platform_source= endif -EXTRA_DIST=ntmain.c +EXTRA_DIST=ntmain.c or_sha1.i + +if USE_EXTERNAL_EVDNS +evdns_source= +else +evdns_source=eventdns.c +endif -tor_SOURCES = buffers.c circuitbuild.c circuitlist.c \ +libtor_a_SOURCES = buffers.c circuitbuild.c circuitlist.c \ circuituse.c command.c config.c \ connection.c connection_edge.c connection_or.c control.c \ cpuworker.c directory.c dirserv.c dirvote.c \ dns.c dnsserv.c geoip.c hibernate.c main.c $(tor_platform_source) \ + microdesc.c \ networkstatus.c onion.c policies.c \ reasons.c relay.c rendcommon.c rendclient.c rendmid.c \ rendservice.c rephist.c router.c routerlist.c routerparse.c \ - eventdns.c \ - tor_main.c + $(evdns_source) config_codedigest.c + +#libtor_a_LIBADD = ../common/libor.a ../common/libor-crypto.a \ +# ../common/libor-event.a + + +tor_SOURCES = tor_main.c AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ -DLOCALSTATEDIR="\"$(localstatedir)\"" \ @@ -32,42 +41,48 @@ AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ # matters a lot there, and is quite hard to debug if you forget to do it. tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@ -tor_LDADD = ../common/libor.a ../common/libor-crypto.a \ - -lz @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ -test_SOURCES = buffers.c circuitbuild.c circuitlist.c \ - circuituse.c command.c config.c \ - connection.c connection_edge.c connection_or.c control.c \ - cpuworker.c directory.c dirserv.c dirvote.c \ - dns.c dnsserv.c geoip.c hibernate.c main.c $(tor_platform_source) \ - networkstatus.c onion.c policies.c \ - reasons.c relay.c rendcommon.c rendclient.c rendmid.c \ - rendservice.c rephist.c router.c routerlist.c routerparse.c \ - eventdns.c \ - test_data.c test.c +tor_LDADD = ./libtor.a ../common/libor.a ../common/libor-crypto.a \ + ../common/libor-event.a \ + @TOR_ZLIB_LIBS@ -lm @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ -test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ - @TOR_LDFLAGS_libevent@ -test_LDADD = ../common/libor.a ../common/libor-crypto.a \ - -lz @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ +noinst_HEADERS = buffers.h circuitbuild.h circuitlist.h circuituse.h \ + command.h config.h connection_edge.h connection.h connection_or.h \ + control.h cpuworker.h directory.h dirserv.h dirvote.h dns.h \ + dnsserv.h geoip.h hibernate.h main.h microdesc.h networkstatus.h \ + ntmain.h onion.h policies.h reasons.h relay.h rendclient.h \ + rendcommon.h rendmid.h rendservice.h rephist.h router.h routerlist.h \ + routerparse.h or.h eventdns.h eventdns_tor.h micro-revision.i -noinst_HEADERS = or.h eventdns.h eventdns_tor.h micro-revision.i +config_codedigest.o: or_sha1.i tor_main.o: micro-revision.i micro-revision.i: FORCE @rm -f micro-revision.tmp; \ - if test -d ../../.git && test -x "`which git 2>&1;true`"; then \ - HASH="`git rev-parse --short=16 HEAD`"; \ - echo \"$$HASH\" > micro-revision.tmp; \ - fi; \ + if test -d ../../.git && test -x "`which git 2>&1;true`"; then \ + HASH="`git rev-parse --short=16 HEAD`"; \ + echo \"$$HASH\" > micro-revision.tmp; \ + fi; \ if test ! -f micro-revision.tmp ; then \ if test ! -f micro-revision.i ; then \ echo '""' > micro-revision.i; \ fi; \ elif test ! -f micro-revision.i || \ - test x"`cat micro-revision.tmp`" != x"`cat micro-revision.i`"; then \ + test x"`cat micro-revision.tmp`" != x"`cat micro-revision.i`"; then \ mv micro-revision.tmp micro-revision.i; \ fi; true +or_sha1.i: $(tor_SOURCES) + if test "@SHA1SUM@" != none; then \ + @SHA1SUM@ $(tor_SOURCES) | @SED@ -n 's/^\(.*\)$$/"\1\\n"/p' > or_sha1.i; \ + elif test "@OPENSSL@" != none; then \ + @OPENSSL@ sha1 $(tor_SOURCES) | @SED@ -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > or_sha1.i; \ + else \ + rm or_sha1.i; \ + touch or_sha1.i; \ + fi + +CLEANFILES = micro-revision.i + #Dummy target to ensure that micro-revision.i _always_ gets built. FORCE: diff --git a/src/or/buffers.c b/src/or/buffers.c index 710374638..db926955b 100644 --- a/src/or/buffers.c +++ b/src/or/buffers.c @@ -12,6 +12,14 @@ **/ #define BUFFERS_PRIVATE #include "or.h" +#include "buffers.h" +#include "config.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "reasons.h" +#include "../common/util.h" +#include "../common/torlog.h" #ifdef HAVE_UNISTD_H #include <unistd.h> #endif @@ -145,10 +153,13 @@ get_freelist(size_t alloc) /** Deallocate a chunk or put it on a freelist */ static void -chunk_free(chunk_t *chunk) +chunk_free_unchecked(chunk_t *chunk) { - size_t alloc = CHUNK_ALLOC_SIZE(chunk->memlen); - chunk_freelist_t *freelist = get_freelist(alloc); + size_t alloc; + chunk_freelist_t *freelist; + + alloc = CHUNK_ALLOC_SIZE(chunk->memlen); + freelist = get_freelist(alloc); if (freelist && freelist->cur_length < freelist->max_length) { chunk->next = freelist->head; freelist->head = chunk; @@ -193,7 +204,7 @@ chunk_new_with_alloc_size(size_t alloc) } #else static void -chunk_free(chunk_t *chunk) +chunk_free_unchecked(chunk_t *chunk) { tor_free(chunk); } @@ -259,13 +270,22 @@ buf_shrink_freelists(int free_all) int n_to_free = free_all ? freelists[i].cur_length : (freelists[i].lowest_length - slack); int n_to_skip = freelists[i].cur_length - n_to_free; + int orig_length = freelists[i].cur_length; int orig_n_to_free = n_to_free, n_freed=0; int orig_n_to_skip = n_to_skip; int new_length = n_to_skip; chunk_t **chp = &freelists[i].head; chunk_t *chunk; while (n_to_skip) { - tor_assert((*chp)->next); + if (! (*chp)->next) { + log_warn(LD_BUG, "I wanted to skip %d chunks in the freelist for " + "%d-byte chunks, but only found %d. (Length %d)", + orig_n_to_skip, (int)freelists[i].alloc_size, + orig_n_to_skip-n_to_skip, freelists[i].cur_length); + assert_freelist_ok(&freelists[i]); + goto done; + } + // tor_assert((*chp)->next); chp = &(*chp)->next; --n_to_skip; } @@ -290,13 +310,15 @@ buf_shrink_freelists(int free_all) } // tor_assert(!n_to_free); freelists[i].cur_length = new_length; - log_info(LD_MM, "Cleaned freelist for %d-byte chunks: kept %d, " - "dropped %d.", - (int)freelists[i].alloc_size, orig_n_to_skip, orig_n_to_free); + log_info(LD_MM, "Cleaned freelist for %d-byte chunks: original " + "length %d, kept %d, dropped %d.", + (int)freelists[i].alloc_size, orig_length, + orig_n_to_skip, orig_n_to_free); } freelists[i].lowest_length = freelists[i].cur_length; assert_freelist_ok(&freelists[i]); } + done: enable_control_logging(); #else (void) free_all; @@ -404,7 +426,7 @@ buf_pullup(buf_t *buf, size_t bytes, int nulterminate) dest->next = src->next; if (buf->tail == src) buf->tail = dest; - chunk_free(src); + chunk_free_unchecked(src); } else { memcpy(CHUNK_WRITE_PTR(dest), src->data, n); dest->datalen += n; @@ -450,7 +472,7 @@ buf_remove_from_front(buf_t *buf, size_t n) buf->head = victim->next; if (buf->tail == victim) buf->tail = NULL; - chunk_free(victim); + chunk_free_unchecked(victim); } } check(); @@ -484,7 +506,7 @@ buf_clear(buf_t *buf) buf->datalen = 0; for (chunk = buf->head; chunk; chunk = next) { next = chunk->next; - chunk_free(chunk); + chunk_free_unchecked(chunk); } buf->head = buf->tail = NULL; } @@ -523,6 +545,8 @@ buf_slack(const buf_t *buf) void buf_free(buf_t *buf) { + if (!buf) + return; buf_clear(buf); buf->magic = 0xdeadbeef; tor_free(buf); @@ -642,12 +666,12 @@ read_to_chunk_tls(buf_t *buf, chunk_t *chunk, tor_tls_t *tls, * (because of EOF), set *<b>reached_eof</b> to 1 and return 0. Return -1 on * error; else return the number of bytes read. */ -/* XXXX021 indicate "read blocked" somehow? */ +/* XXXX023 indicate "read blocked" somehow? */ int read_to_buf(int s, size_t at_most, buf_t *buf, int *reached_eof, int *socket_error) { - /* XXXX021 It's stupid to overload the return values for these functions: + /* XXXX023 It's stupid to overload the return values for these functions: * "error status" and "number of bytes read" are not mutually exclusive. */ int r = 0; @@ -832,7 +856,7 @@ flush_chunk_tls(tor_tls_t *tls, buf_t *buf, chunk_t *chunk, int flush_buf(int s, buf_t *buf, size_t sz, size_t *buf_flushlen) { - /* XXXX021 It's stupid to overload the return values for these functions: + /* XXXX023 It's stupid to overload the return values for these functions: * "error status" and "number of bytes flushed" are not mutually exclusive. */ int r; @@ -1275,6 +1299,47 @@ fetch_from_buf_http(buf_t *buf, return 1; } +/** + * Wait this many seconds before warning the user about using SOCKS unsafely + * again (requires that WarnUnsafeSocks is turned on). */ +#define SOCKS_WARN_INTERVAL 5 + +/** Warn that the user application has made an unsafe socks request using + * protocol <b>socks_protocol</b> on port <b>port</b>. Don't warn more than + * once per SOCKS_WARN_INTERVAL, unless <b>safe_socks</b> is set. */ +static void +log_unsafe_socks_warning(int socks_protocol, const char *address, + uint16_t port, int safe_socks) +{ + static ratelim_t socks_ratelim = RATELIM_INIT(SOCKS_WARN_INTERVAL); + + or_options_t *options = get_options(); + char *m = NULL; + if (! options->WarnUnsafeSocks) + return; + if (safe_socks || (m = rate_limit_log(&socks_ratelim, approx_time()))) { + log_warn(LD_APP, + "Your application (using socks%d to port %d) is giving " + "Tor only an IP address. Applications that do DNS resolves " + "themselves may leak information. Consider using Socks4A " + "(e.g. via privoxy or socat) instead. For more information, " + "please see https://wiki.torproject.org/TheOnionRouter/" + "TorFAQ#SOCKSAndDNS.%s%s", + socks_protocol, + (int)port, + safe_socks ? " Rejecting." : "", + m ? m : ""); + tor_free(m); + } + control_event_client_status(LOG_WARN, + "DANGEROUS_SOCKS PROTOCOL=SOCKS%d ADDRESS=%s:%d", + socks_protocol, address, (int)port); +} + +/** Do not attempt to parse socks messages longer than this. This value is + * actually significantly higher than the longest possible socks message. */ +#define MAX_SOCKS_MESSAGE_LEN 512 + /** There is a (possibly incomplete) socks handshake on <b>buf</b>, of one * of the forms * - socks4: "socksheader username\\0" @@ -1313,14 +1378,10 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, char *next, *startaddr; struct in_addr in; - /* If the user connects with socks4 or the wrong variant of socks5, - * then log a warning to let him know that it might be unwise. */ - static int have_warned_about_unsafe_socks = 0; - if (buf->datalen < 2) /* version and another byte */ return 0; - buf_pullup(buf, 128, 0); + buf_pullup(buf, MAX_SOCKS_MESSAGE_LEN, 0); tor_assert(buf->head && buf->head->datalen >= 2); socksver = *buf->head->data; @@ -1396,21 +1457,8 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, req->port = ntohs(get_uint16(buf->head->data+4+addrlen)); buf_remove_from_front(buf, 6+addrlen); if (req->command != SOCKS_COMMAND_RESOLVE_PTR && - !addressmap_have_mapping(req->address,0) && - !have_warned_about_unsafe_socks) { - log_warn(LD_APP, - "Your application (using socks5 to port %d) is giving " - "Tor only an IP address. Applications that do DNS resolves " - "themselves may leak information. Consider using Socks4A " - "(e.g. via privoxy or socat) instead. For more information, " - "please see http://wiki.noreply.org/noreply/TheOnionRouter/" - "TorFAQ#SOCKSAndDNS.%s", req->port, - safe_socks ? " Rejecting." : ""); - /*have_warned_about_unsafe_socks = 1;*/ - /*(for now, warn every time)*/ - control_event_client_status(LOG_WARN, - "DANGEROUS_SOCKS PROTOCOL=SOCKS5 ADDRESS=%s:%d", - req->address, req->port); + !addressmap_have_mapping(req->address,0)) { + log_unsafe_socks_warning(5, req->address, req->port, safe_socks); if (safe_socks) return -1; } @@ -1475,8 +1523,8 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, return -1; } - req->port = ntohs(*(uint16_t*)(buf->head->data+2)); - destip = ntohl(*(uint32_t*)(buf->head->data+4)); + req->port = ntohs(get_uint16(buf->head->data+2)); + destip = ntohl(get_uint32(buf->head->data+4)); if ((!req->port && req->command!=SOCKS_COMMAND_RESOLVE) || !destip) { log_warn(LD_APP,"socks4: Port or DestIP is zero. Rejecting."); return -1; @@ -1491,7 +1539,8 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, return -1; } log_debug(LD_APP, - "socks4: successfully read destip (%s)", safe_str(tmpbuf)); + "socks4: successfully read destip (%s)", + safe_str_client(tmpbuf)); socks4_prot = socks4; } @@ -1509,20 +1558,9 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, startaddr = NULL; if (socks4_prot != socks4a && - !addressmap_have_mapping(tmpbuf,0) && - !have_warned_about_unsafe_socks) { - log_warn(LD_APP, - "Your application (using socks4 to port %d) is giving Tor " - "only an IP address. Applications that do DNS resolves " - "themselves may leak information. Consider using Socks4A " - "(e.g. via privoxy or socat) instead. For more information, " - "please see http://wiki.noreply.org/noreply/TheOnionRouter/" - "TorFAQ#SOCKSAndDNS.%s", req->port, - safe_socks ? " Rejecting." : ""); - /*have_warned_about_unsafe_socks = 1;*/ /*(for now, warn every time)*/ - control_event_client_status(LOG_WARN, - "DANGEROUS_SOCKS PROTOCOL=SOCKS4 ADDRESS=%s:%d", - tmpbuf, req->port); + !addressmap_have_mapping(tmpbuf,0)) { + log_unsafe_socks_warning(4, tmpbuf, req->port, safe_socks); + if (safe_socks) return -1; } @@ -1614,6 +1652,132 @@ fetch_from_buf_socks(buf_t *buf, socks_request_t *req, } } +/** Inspect a reply from SOCKS server stored in <b>buf</b> according + * to <b>state</b>, removing the protocol data upon success. Return 0 on + * incomplete response, 1 on success and -1 on error, in which case + * <b>reason</b> is set to a descriptive message (free() when finished + * with it). + * + * As a special case, 2 is returned when user/pass is required + * during SOCKS5 handshake and user/pass is configured. + */ +int +fetch_from_buf_socks_client(buf_t *buf, int state, char **reason) +{ + unsigned char *data; + size_t addrlen; + + if (buf->datalen < 2) + return 0; + + buf_pullup(buf, MAX_SOCKS_MESSAGE_LEN, 0); + tor_assert(buf->head && buf->head->datalen >= 2); + + data = (unsigned char *) buf->head->data; + + switch (state) { + case PROXY_SOCKS4_WANT_CONNECT_OK: + /* Wait for the complete response */ + if (buf->head->datalen < 8) + return 0; + + if (data[1] != 0x5a) { + *reason = tor_strdup(socks4_response_code_to_string(data[1])); + return -1; + } + + /* Success */ + buf_remove_from_front(buf, 8); + return 1; + + case PROXY_SOCKS5_WANT_AUTH_METHOD_NONE: + /* we don't have any credentials */ + if (data[1] != 0x00) { + *reason = tor_strdup("server doesn't support any of our " + "available authentication methods"); + return -1; + } + + log_info(LD_NET, "SOCKS 5 client: continuing without authentication"); + buf_clear(buf); + return 1; + + case PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929: + /* we have a username and password. return 1 if we can proceed without + * providing authentication, or 2 otherwise. */ + switch (data[1]) { + case 0x00: + log_info(LD_NET, "SOCKS 5 client: we have auth details but server " + "doesn't require authentication."); + buf_clear(buf); + return 1; + case 0x02: + log_info(LD_NET, "SOCKS 5 client: need authentication."); + buf_clear(buf); + return 2; + /* fall through */ + } + + *reason = tor_strdup("server doesn't support any of our available " + "authentication methods"); + return -1; + + case PROXY_SOCKS5_WANT_AUTH_RFC1929_OK: + /* handle server reply to rfc1929 authentication */ + if (data[1] != 0x00) { + *reason = tor_strdup("authentication failed"); + return -1; + } + + log_info(LD_NET, "SOCKS 5 client: authentication successful."); + buf_clear(buf); + return 1; + + case PROXY_SOCKS5_WANT_CONNECT_OK: + /* response is variable length. BND.ADDR, etc, isn't needed + * (don't bother with buf_pullup()), but make sure to eat all + * the data used */ + + /* wait for address type field to arrive */ + if (buf->datalen < 4) + return 0; + + switch (data[3]) { + case 0x01: /* ip4 */ + addrlen = 4; + break; + case 0x04: /* ip6 */ + addrlen = 16; + break; + case 0x03: /* fqdn (can this happen here?) */ + if (buf->datalen < 5) + return 0; + addrlen = 1 + data[4]; + break; + default: + *reason = tor_strdup("invalid response to connect request"); + return -1; + } + + /* wait for address and port */ + if (buf->datalen < 6 + addrlen) + return 0; + + if (data[1] != 0x00) { + *reason = tor_strdup(socks5_response_code_to_string(data[1])); + return -1; + } + + buf_remove_from_front(buf, 6 + addrlen); + return 1; + } + + /* shouldn't get here... */ + tor_assert(0); + + return -1; +} + /** Return 1 iff buf looks more like it has an (obsolete) v0 controller * command on it than any valid v1 controller command. */ int diff --git a/src/or/buffers.h b/src/or/buffers.h new file mode 100644 index 000000000..e50b9ff6f --- /dev/null +++ b/src/or/buffers.h @@ -0,0 +1,58 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file buffers.h + * \brief Header file for buffers.c. + **/ + +#ifndef _TOR_BUFFERS_H +#define _TOR_BUFFERS_H + +buf_t *buf_new(void); +buf_t *buf_new_with_capacity(size_t size); +void buf_free(buf_t *buf); +void buf_clear(buf_t *buf); +void buf_shrink(buf_t *buf); +void buf_shrink_freelists(int free_all); +void buf_dump_freelist_sizes(int severity); + +size_t buf_datalen(const buf_t *buf); +size_t buf_allocation(const buf_t *buf); +size_t buf_slack(const buf_t *buf); + +int read_to_buf(int s, size_t at_most, buf_t *buf, int *reached_eof, + int *socket_error); +int read_to_buf_tls(tor_tls_t *tls, size_t at_most, buf_t *buf); + +int flush_buf(int s, buf_t *buf, size_t sz, size_t *buf_flushlen); +int flush_buf_tls(tor_tls_t *tls, buf_t *buf, size_t sz, size_t *buf_flushlen); + +int write_to_buf(const char *string, size_t string_len, buf_t *buf); +int write_to_buf_zlib(buf_t *buf, tor_zlib_state_t *state, + const char *data, size_t data_len, int done); +int move_buf_to_buf(buf_t *buf_out, buf_t *buf_in, size_t *buf_flushlen); +int fetch_from_buf(char *string, size_t string_len, buf_t *buf); +int fetch_var_cell_from_buf(buf_t *buf, var_cell_t **out, int linkproto); +int fetch_from_buf_http(buf_t *buf, + char **headers_out, size_t max_headerlen, + char **body_out, size_t *body_used, size_t max_bodylen, + int force_complete); +int fetch_from_buf_socks(buf_t *buf, socks_request_t *req, + int log_sockstype, int safe_socks); +int fetch_from_buf_socks_client(buf_t *buf, int state, char **reason); +int fetch_from_buf_line(buf_t *buf, char *data_out, size_t *data_len); + +int peek_buf_has_control0_command(buf_t *buf); + +void assert_buf_ok(buf_t *buf); + +#ifdef BUFFERS_PRIVATE +int buf_find_string_offset(const buf_t *buf, const char *s, size_t n); +#endif + +#endif + diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 76713e676..2189e0e55 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -9,9 +9,46 @@ * \brief The actual details of building circuits. **/ +#define CIRCUIT_PRIVATE + #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "directory.h" +#include "main.h" +#include "networkstatus.h" +#include "onion.h" +#include "policies.h" +#include "relay.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" +#include "crypto.h" +#undef log +#include <math.h> + +#ifndef MIN +#define MIN(a,b) ((a)<(b)?(a):(b)) +#endif + +#define CBT_BIN_TO_MS(bin) ((bin)*CBT_BIN_WIDTH + (CBT_BIN_WIDTH/2)) /********* START VARIABLES **********/ +/** Global list of circuit build times */ +// XXXX023: Add this as a member for entry_guard_t instead of global? +// Then we could do per-guard statistics, as guards are likely to +// vary in their own latency. The downside of this is that guards +// can change frequently, so we'd be building a lot more circuits +// most likely. +/* XXXX023 Make this static; add accessor functions. */ +circuit_build_times_t circ_times; /** A global list of all circuits at this hop. */ extern circuit_t *global_circuitlist; @@ -47,6 +84,10 @@ static smartlist_t *entry_guards = NULL; * and those changes need to be flushed to disk. */ static int entry_guards_dirty = 0; +/** If set, we're running the unit tests: we should avoid clobbering + * our state file or accessing get_options() or get_or_state() */ +static int unit_tests = 0; + /********* END VARIABLES ************/ static int circuit_deliver_create_cell(circuit_t *circ, @@ -59,6 +100,1358 @@ static int onion_append_hop(crypt_path_t **head_ptr, extend_info_t *choice); static void entry_guards_changed(void); +/** + * This function decides if CBT learning should be disabled. It returns + * true if one or more of the following four conditions are met: + * + * 1. If the cbtdisabled consensus parameter is set. + * 2. If the torrc option LearnCircuitBuildTimeout is false. + * 3. If we are a directory authority + * 4. If we fail to write circuit build time history to our state file. + */ +static int +circuit_build_times_disabled(void) +{ + if (unit_tests) { + return 0; + } else { + int consensus_disabled = networkstatus_get_param(NULL, "cbtdisabled", + 0, 0, 1); + int config_disabled = !get_options()->LearnCircuitBuildTimeout; + int dirauth_disabled = get_options()->AuthoritativeDir; + int state_disabled = did_last_state_file_write_fail() ? 1 : 0; + + if (consensus_disabled || config_disabled || dirauth_disabled || + state_disabled) { + log_info(LD_CIRC, + "CircuitBuildTime learning is disabled. " + "Consensus=%d, Config=%d, AuthDir=%d, StateFile=%d", + consensus_disabled, config_disabled, dirauth_disabled, + state_disabled); + return 1; + } else { + return 0; + } + } +} + +/** + * Retrieve and bounds-check the cbtmaxtimeouts consensus paramter. + * + * Effect: When this many timeouts happen in the last 'cbtrecentcount' + * circuit attempts, the client should discard all of its history and + * begin learning a fresh timeout value. + */ +static int32_t +circuit_build_times_max_timeouts(void) +{ + return networkstatus_get_param(NULL, "cbtmaxtimeouts", + CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT, + CBT_MIN_MAX_RECENT_TIMEOUT_COUNT, + CBT_MAX_MAX_RECENT_TIMEOUT_COUNT); +} + +/** + * Retrieve and bounds-check the cbtnummodes consensus paramter. + * + * Effect: This value governs how many modes to use in the weighted + * average calculation of Pareto parameter Xm. A value of 3 introduces + * some bias (2-5% of CDF) under ideal conditions, but allows for better + * performance in the event that a client chooses guard nodes of radically + * different performance characteristics. + */ +static int32_t +circuit_build_times_default_num_xm_modes(void) +{ + int32_t num = networkstatus_get_param(NULL, "cbtnummodes", + CBT_DEFAULT_NUM_XM_MODES, + CBT_MIN_NUM_XM_MODES, + CBT_MAX_NUM_XM_MODES); + return num; +} + +/** + * Retrieve and bounds-check the cbtmincircs consensus paramter. + * + * Effect: This is the minimum number of circuits to build before + * computing a timeout. + */ +static int32_t +circuit_build_times_min_circs_to_observe(void) +{ + int32_t num = networkstatus_get_param(NULL, "cbtmincircs", + CBT_DEFAULT_MIN_CIRCUITS_TO_OBSERVE, + CBT_MIN_MIN_CIRCUITS_TO_OBSERVE, + CBT_MAX_MIN_CIRCUITS_TO_OBSERVE); + return num; +} + +/** Return true iff <b>cbt</b> has recorded enough build times that we + * want to start acting on the timeout it implies. */ +int +circuit_build_times_enough_to_compute(circuit_build_times_t *cbt) +{ + return cbt->total_build_times >= circuit_build_times_min_circs_to_observe(); +} + +/** + * Retrieve and bounds-check the cbtquantile consensus paramter. + * + * Effect: This is the position on the quantile curve to use to set the + * timeout value. It is a percent (10-99). + */ +double +circuit_build_times_quantile_cutoff(void) +{ + int32_t num = networkstatus_get_param(NULL, "cbtquantile", + CBT_DEFAULT_QUANTILE_CUTOFF, + CBT_MIN_QUANTILE_CUTOFF, + CBT_MAX_QUANTILE_CUTOFF); + return num/100.0; +} + +int +circuit_build_times_get_bw_scale(networkstatus_t *ns) +{ + return networkstatus_get_param(ns, "bwweightscale", + BW_WEIGHT_SCALE, + BW_MIN_WEIGHT_SCALE, + BW_MAX_WEIGHT_SCALE); +} + +/** + * Retrieve and bounds-check the cbtclosequantile consensus paramter. + * + * Effect: This is the position on the quantile curve to use to set the + * timeout value to use to actually close circuits. It is a percent + * (0-99). + */ +static double +circuit_build_times_close_quantile(void) +{ + int32_t param; + /* Cast is safe - circuit_build_times_quantile_cutoff() is capped */ + int32_t min = (int)tor_lround(100*circuit_build_times_quantile_cutoff()); + param = networkstatus_get_param(NULL, "cbtclosequantile", + CBT_DEFAULT_CLOSE_QUANTILE, + CBT_MIN_CLOSE_QUANTILE, + CBT_MAX_CLOSE_QUANTILE); + if (param < min) { + log_warn(LD_DIR, "Consensus parameter cbtclosequantile is " + "too small, raising to %d", min); + param = min; + } + return param / 100.0; +} + +/** + * Retrieve and bounds-check the cbttestfreq consensus paramter. + * + * Effect: Describes how often in seconds to build a test circuit to + * gather timeout values. Only applies if less than 'cbtmincircs' + * have been recorded. + */ +static int32_t +circuit_build_times_test_frequency(void) +{ + int32_t num = networkstatus_get_param(NULL, "cbttestfreq", + CBT_DEFAULT_TEST_FREQUENCY, + CBT_MIN_TEST_FREQUENCY, + CBT_MAX_TEST_FREQUENCY); + return num; +} + +/** + * Retrieve and bounds-check the cbtmintimeout consensus paramter. + * + * Effect: This is the minimum allowed timeout value in milliseconds. + * The minimum is to prevent rounding to 0 (we only check once + * per second). + */ +static int32_t +circuit_build_times_min_timeout(void) +{ + int32_t num = networkstatus_get_param(NULL, "cbtmintimeout", + CBT_DEFAULT_TIMEOUT_MIN_VALUE, + CBT_MIN_TIMEOUT_MIN_VALUE, + CBT_MAX_TIMEOUT_MIN_VALUE); + return num; +} + +/** + * Retrieve and bounds-check the cbtinitialtimeout consensus paramter. + * + * Effect: This is the timeout value to use before computing a timeout, + * in milliseconds. + */ +int32_t +circuit_build_times_initial_timeout(void) +{ + int32_t min = circuit_build_times_min_timeout(); + int32_t param = networkstatus_get_param(NULL, "cbtinitialtimeout", + CBT_DEFAULT_TIMEOUT_INITIAL_VALUE, + CBT_MIN_TIMEOUT_INITIAL_VALUE, + CBT_MAX_TIMEOUT_INITIAL_VALUE); + if (param < min) { + log_warn(LD_DIR, "Consensus parameter cbtinitialtimeout is too small, " + "raising to %d", min); + param = min; + } + return param; +} + +/** + * Retrieve and bounds-check the cbtrecentcount consensus paramter. + * + * Effect: This is the number of circuit build times to keep track of + * for deciding if we hit cbtmaxtimeouts and need to reset our state + * and learn a new timeout. + */ +static int32_t +circuit_build_times_recent_circuit_count(networkstatus_t *ns) +{ + return networkstatus_get_param(ns, "cbtrecentcount", + CBT_DEFAULT_RECENT_CIRCUITS, + CBT_MIN_RECENT_CIRCUITS, + CBT_MAX_RECENT_CIRCUITS); +} + +/** + * This function is called when we get a consensus update. + * + * It checks to see if we have changed any consensus parameters + * that require reallocation or discard of previous stats. + */ +void +circuit_build_times_new_consensus_params(circuit_build_times_t *cbt, + networkstatus_t *ns) +{ + int32_t num = circuit_build_times_recent_circuit_count(ns); + + if (num > 0 && num != cbt->liveness.num_recent_circs) { + int8_t *recent_circs; + log_notice(LD_CIRC, "The Tor Directory Consensus has changed how many " + "circuits we must track to detect network failures from %d " + "to %d.", cbt->liveness.num_recent_circs, num); + + tor_assert(cbt->liveness.timeouts_after_firsthop); + + /* + * Technically this is a circular array that we are reallocating + * and memcopying. However, since it only consists of either 1s + * or 0s, and is only used in a statistical test to determine when + * we should discard our history after a sufficient number of 1's + * have been reached, it is fine if order is not preserved or + * elements are lost. + * + * cbtrecentcount should only be changing in cases of severe network + * distress anyway, so memory correctness here is paramount over + * doing acrobatics to preserve the array. + */ + recent_circs = tor_malloc_zero(sizeof(int8_t)*num); + memcpy(recent_circs, cbt->liveness.timeouts_after_firsthop, + sizeof(int8_t)*MIN(num, cbt->liveness.num_recent_circs)); + + // Adjust the index if it needs it. + if (num < cbt->liveness.num_recent_circs) { + cbt->liveness.after_firsthop_idx = MIN(num-1, + cbt->liveness.after_firsthop_idx); + } + + tor_free(cbt->liveness.timeouts_after_firsthop); + cbt->liveness.timeouts_after_firsthop = recent_circs; + cbt->liveness.num_recent_circs = num; + } +} + +/** Make a note that we're running unit tests (rather than running Tor + * itself), so we avoid clobbering our state file. */ +void +circuitbuild_running_unit_tests(void) +{ + unit_tests = 1; +} + +/** + * Return the initial default or configured timeout in milliseconds + */ +static double +circuit_build_times_get_initial_timeout(void) +{ + double timeout; + if (!unit_tests && get_options()->CircuitBuildTimeout) { + timeout = get_options()->CircuitBuildTimeout*1000; + if (timeout < circuit_build_times_min_timeout()) { + log_warn(LD_CIRC, "Config CircuitBuildTimeout too low. Setting to %ds", + circuit_build_times_min_timeout()/1000); + timeout = circuit_build_times_min_timeout(); + } + } else { + timeout = circuit_build_times_initial_timeout(); + } + return timeout; +} + +/** + * Reset the build time state. + * + * Leave estimated parameters, timeout and network liveness intact + * for future use. + */ +void +circuit_build_times_reset(circuit_build_times_t *cbt) +{ + memset(cbt->circuit_build_times, 0, sizeof(cbt->circuit_build_times)); + cbt->total_build_times = 0; + cbt->build_times_idx = 0; + cbt->have_computed_timeout = 0; +} + +/** + * Initialize the buildtimes structure for first use. + * + * Sets the initial timeout values based on either the config setting, + * the consensus param, or the default (CBT_DEFAULT_TIMEOUT_INITIAL_VALUE). + */ +void +circuit_build_times_init(circuit_build_times_t *cbt) +{ + memset(cbt, 0, sizeof(*cbt)); + cbt->liveness.num_recent_circs = + circuit_build_times_recent_circuit_count(NULL); + cbt->liveness.timeouts_after_firsthop = tor_malloc_zero(sizeof(int8_t)* + cbt->liveness.num_recent_circs); + cbt->close_ms = cbt->timeout_ms = circuit_build_times_get_initial_timeout(); + control_event_buildtimeout_set(cbt, BUILDTIMEOUT_SET_EVENT_RESET); +} + +#if 0 +/** + * Rewind our build time history by n positions. + */ +static void +circuit_build_times_rewind_history(circuit_build_times_t *cbt, int n) +{ + int i = 0; + + cbt->build_times_idx -= n; + cbt->build_times_idx %= CBT_NCIRCUITS_TO_OBSERVE; + + for (i = 0; i < n; i++) { + cbt->circuit_build_times[(i+cbt->build_times_idx) + %CBT_NCIRCUITS_TO_OBSERVE]=0; + } + + if (cbt->total_build_times > n) { + cbt->total_build_times -= n; + } else { + cbt->total_build_times = 0; + } + + log_info(LD_CIRC, + "Rewound history by %d places. Current index: %d. " + "Total: %d", n, cbt->build_times_idx, cbt->total_build_times); +} +#endif + +/** + * Add a new build time value <b>time</b> to the set of build times. Time + * units are milliseconds. + * + * circuit_build_times <b>cbt</b> is a circular array, so loop around when + * array is full. + */ +int +circuit_build_times_add_time(circuit_build_times_t *cbt, build_time_t time) +{ + if (time <= 0 || time > CBT_BUILD_TIME_MAX) { + log_warn(LD_BUG, "Circuit build time is too large (%u)." + "This is probably a bug.", time); + tor_fragile_assert(); + return -1; + } + + log_debug(LD_CIRC, "Adding circuit build time %u", time); + + cbt->circuit_build_times[cbt->build_times_idx] = time; + cbt->build_times_idx = (cbt->build_times_idx + 1) % CBT_NCIRCUITS_TO_OBSERVE; + if (cbt->total_build_times < CBT_NCIRCUITS_TO_OBSERVE) + cbt->total_build_times++; + + if ((cbt->total_build_times % CBT_SAVE_STATE_EVERY) == 0) { + /* Save state every n circuit builds */ + if (!unit_tests && !get_options()->AvoidDiskWrites) + or_state_mark_dirty(get_or_state(), 0); + } + + return 0; +} + +/** + * Return maximum circuit build time + */ +static build_time_t +circuit_build_times_max(circuit_build_times_t *cbt) +{ + int i = 0; + build_time_t max_build_time = 0; + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] > max_build_time + && cbt->circuit_build_times[i] != CBT_BUILD_ABANDONED) + max_build_time = cbt->circuit_build_times[i]; + } + return max_build_time; +} + +#if 0 +/** Return minimum circuit build time */ +build_time_t +circuit_build_times_min(circuit_build_times_t *cbt) +{ + int i = 0; + build_time_t min_build_time = CBT_BUILD_TIME_MAX; + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] && /* 0 <-> uninitialized */ + cbt->circuit_build_times[i] < min_build_time) + min_build_time = cbt->circuit_build_times[i]; + } + if (min_build_time == CBT_BUILD_TIME_MAX) { + log_warn(LD_CIRC, "No build times less than CBT_BUILD_TIME_MAX!"); + } + return min_build_time; +} +#endif + +/** + * Calculate and return a histogram for the set of build times. + * + * Returns an allocated array of histrogram bins representing + * the frequency of index*CBT_BIN_WIDTH millisecond + * build times. Also outputs the number of bins in nbins. + * + * The return value must be freed by the caller. + */ +static uint32_t * +circuit_build_times_create_histogram(circuit_build_times_t *cbt, + build_time_t *nbins) +{ + uint32_t *histogram; + build_time_t max_build_time = circuit_build_times_max(cbt); + int i, c; + + *nbins = 1 + (max_build_time / CBT_BIN_WIDTH); + histogram = tor_malloc_zero(*nbins * sizeof(build_time_t)); + + // calculate histogram + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] == 0 + || cbt->circuit_build_times[i] == CBT_BUILD_ABANDONED) + continue; /* 0 <-> uninitialized */ + + c = (cbt->circuit_build_times[i] / CBT_BIN_WIDTH); + histogram[c]++; + } + + return histogram; +} + +/** + * Return the Pareto start-of-curve parameter Xm. + * + * Because we are not a true Pareto curve, we compute this as the + * weighted average of the N most frequent build time bins. N is either + * 1 if we don't have enough circuit build time data collected, or + * determined by the consensus parameter cbtnummodes (default 3). + */ +static build_time_t +circuit_build_times_get_xm(circuit_build_times_t *cbt) +{ + build_time_t i, nbins; + build_time_t *nth_max_bin; + int32_t bin_counts=0; + build_time_t ret = 0; + uint32_t *histogram = circuit_build_times_create_histogram(cbt, &nbins); + int n=0; + int num_modes = circuit_build_times_default_num_xm_modes(); + + tor_assert(nbins > 0); + tor_assert(num_modes > 0); + + // Only use one mode if < 1000 buildtimes. Not enough data + // for multiple. + if (cbt->total_build_times < CBT_NCIRCUITS_TO_OBSERVE) + num_modes = 1; + + nth_max_bin = (build_time_t*)tor_malloc_zero(num_modes*sizeof(build_time_t)); + + /* Determine the N most common build times */ + for (i = 0; i < nbins; i++) { + if (histogram[i] >= histogram[nth_max_bin[0]]) { + nth_max_bin[0] = i; + } + + for (n = 1; n < num_modes; n++) { + if (histogram[i] >= histogram[nth_max_bin[n]] && + (!histogram[nth_max_bin[n-1]] + || histogram[i] < histogram[nth_max_bin[n-1]])) { + nth_max_bin[n] = i; + } + } + } + + for (n = 0; n < num_modes; n++) { + bin_counts += histogram[nth_max_bin[n]]; + ret += CBT_BIN_TO_MS(nth_max_bin[n])*histogram[nth_max_bin[n]]; + log_info(LD_CIRC, "Xm mode #%d: %u %u", n, CBT_BIN_TO_MS(nth_max_bin[n]), + histogram[nth_max_bin[n]]); + } + + /* The following assert is safe, because we don't get called when we + * haven't observed at least CBT_MIN_MIN_CIRCUITS_TO_OBSERVE circuits. */ + tor_assert(bin_counts > 0); + + ret /= bin_counts; + tor_free(histogram); + tor_free(nth_max_bin); + + return ret; +} + +/** + * Output a histogram of current circuit build times to + * the or_state_t state structure. + */ +void +circuit_build_times_update_state(circuit_build_times_t *cbt, + or_state_t *state) +{ + uint32_t *histogram; + build_time_t i = 0; + build_time_t nbins = 0; + config_line_t **next, *line; + + histogram = circuit_build_times_create_histogram(cbt, &nbins); + // write to state + config_free_lines(state->BuildtimeHistogram); + next = &state->BuildtimeHistogram; + *next = NULL; + + state->TotalBuildTimes = cbt->total_build_times; + state->CircuitBuildAbandonedCount = 0; + + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] == CBT_BUILD_ABANDONED) + state->CircuitBuildAbandonedCount++; + } + + for (i = 0; i < nbins; i++) { + // compress the histogram by skipping the blanks + if (histogram[i] == 0) continue; + *next = line = tor_malloc_zero(sizeof(config_line_t)); + line->key = tor_strdup("CircuitBuildTimeBin"); + line->value = tor_malloc(25); + tor_snprintf(line->value, 25, "%d %d", + CBT_BIN_TO_MS(i), histogram[i]); + next = &(line->next); + } + + if (!unit_tests) { + if (!get_options()->AvoidDiskWrites) + or_state_mark_dirty(get_or_state(), 0); + } + + tor_free(histogram); +} + +/** + * Shuffle the build times array. + * + * Adapted from http://en.wikipedia.org/wiki/Fisher-Yates_shuffle + */ +static void +circuit_build_times_shuffle_and_store_array(circuit_build_times_t *cbt, + build_time_t *raw_times, + uint32_t num_times) +{ + uint32_t n = num_times; + if (num_times > CBT_NCIRCUITS_TO_OBSERVE) { + log_notice(LD_CIRC, "The number of circuit times that this Tor version " + "uses to calculate build times is less than the number stored " + "in your state file. Decreasing the circuit time history from " + "%d to %d.", num_times, CBT_NCIRCUITS_TO_OBSERVE); + } + + /* This code can only be run on a compact array */ + while (n-- > 1) { + int k = crypto_rand_int(n + 1); /* 0 <= k <= n. */ + build_time_t tmp = raw_times[k]; + raw_times[k] = raw_times[n]; + raw_times[n] = tmp; + } + + /* Since the times are now shuffled, take a random CBT_NCIRCUITS_TO_OBSERVE + * subset (ie the first CBT_NCIRCUITS_TO_OBSERVE values) */ + for (n = 0; n < MIN(num_times, CBT_NCIRCUITS_TO_OBSERVE); n++) { + circuit_build_times_add_time(cbt, raw_times[n]); + } +} + +/** + * Filter old synthetic timeouts that were created before the + * new right-censored Pareto calculation was deployed. + * + * Once all clients before 0.2.1.13-alpha are gone, this code + * will be unused. + */ +static int +circuit_build_times_filter_timeouts(circuit_build_times_t *cbt) +{ + int num_filtered=0, i=0; + double timeout_rate = 0; + build_time_t max_timeout = 0; + + timeout_rate = circuit_build_times_timeout_rate(cbt); + max_timeout = (build_time_t)cbt->close_ms; + + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] > max_timeout) { + build_time_t replaced = cbt->circuit_build_times[i]; + num_filtered++; + cbt->circuit_build_times[i] = CBT_BUILD_ABANDONED; + + log_debug(LD_CIRC, "Replaced timeout %d with %d", replaced, + cbt->circuit_build_times[i]); + } + } + + log_info(LD_CIRC, + "We had %d timeouts out of %d build times, " + "and filtered %d above the max of %u", + (int)(cbt->total_build_times*timeout_rate), + cbt->total_build_times, num_filtered, max_timeout); + + return num_filtered; +} + +/** + * Load histogram from <b>state</b>, shuffling the resulting array + * after we do so. Use this result to estimate parameters and + * calculate the timeout. + * + * Return -1 on error. + */ +int +circuit_build_times_parse_state(circuit_build_times_t *cbt, + or_state_t *state) +{ + int tot_values = 0; + uint32_t loaded_cnt = 0, N = 0; + config_line_t *line; + unsigned int i; + build_time_t *loaded_times; + int err = 0; + circuit_build_times_init(cbt); + + if (circuit_build_times_disabled()) { + return 0; + } + + /* build_time_t 0 means uninitialized */ + loaded_times = tor_malloc_zero(sizeof(build_time_t)*state->TotalBuildTimes); + + for (line = state->BuildtimeHistogram; line; line = line->next) { + smartlist_t *args = smartlist_create(); + smartlist_split_string(args, line->value, " ", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + if (smartlist_len(args) < 2) { + log_warn(LD_GENERAL, "Unable to parse circuit build times: " + "Too few arguments to CircuitBuildTime"); + err = 1; + SMARTLIST_FOREACH(args, char*, cp, tor_free(cp)); + smartlist_free(args); + break; + } else { + const char *ms_str = smartlist_get(args,0); + const char *count_str = smartlist_get(args,1); + uint32_t count, k; + build_time_t ms; + int ok; + ms = (build_time_t)tor_parse_ulong(ms_str, 0, 0, + CBT_BUILD_TIME_MAX, &ok, NULL); + if (!ok) { + log_warn(LD_GENERAL, "Unable to parse circuit build times: " + "Unparsable bin number"); + err = 1; + SMARTLIST_FOREACH(args, char*, cp, tor_free(cp)); + smartlist_free(args); + break; + } + count = (uint32_t)tor_parse_ulong(count_str, 0, 0, + UINT32_MAX, &ok, NULL); + if (!ok) { + log_warn(LD_GENERAL, "Unable to parse circuit build times: " + "Unparsable bin count"); + err = 1; + SMARTLIST_FOREACH(args, char*, cp, tor_free(cp)); + smartlist_free(args); + break; + } + + if (loaded_cnt+count+state->CircuitBuildAbandonedCount + > state->TotalBuildTimes) { + log_warn(LD_CIRC, + "Too many build times in state file. " + "Stopping short before %d", + loaded_cnt+count); + SMARTLIST_FOREACH(args, char*, cp, tor_free(cp)); + smartlist_free(args); + break; + } + + for (k = 0; k < count; k++) { + loaded_times[loaded_cnt++] = ms; + } + N++; + SMARTLIST_FOREACH(args, char*, cp, tor_free(cp)); + smartlist_free(args); + } + } + + log_info(LD_CIRC, + "Adding %d timeouts.", state->CircuitBuildAbandonedCount); + for (i=0; i < state->CircuitBuildAbandonedCount; i++) { + loaded_times[loaded_cnt++] = CBT_BUILD_ABANDONED; + } + + if (loaded_cnt != state->TotalBuildTimes) { + log_warn(LD_CIRC, + "Corrupt state file? Build times count mismatch. " + "Read %d times, but file says %d", loaded_cnt, + state->TotalBuildTimes); + err = 1; + circuit_build_times_reset(cbt); + goto done; + } + + circuit_build_times_shuffle_and_store_array(cbt, loaded_times, loaded_cnt); + + /* Verify that we didn't overwrite any indexes */ + for (i=0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (!cbt->circuit_build_times[i]) + break; + tot_values++; + } + log_info(LD_CIRC, + "Loaded %d/%d values from %d lines in circuit time histogram", + tot_values, cbt->total_build_times, N); + + if (cbt->total_build_times != tot_values + || cbt->total_build_times > CBT_NCIRCUITS_TO_OBSERVE) { + log_warn(LD_CIRC, + "Corrupt state file? Shuffled build times mismatch. " + "Read %d times, but file says %d", tot_values, + state->TotalBuildTimes); + err = 1; + circuit_build_times_reset(cbt); + goto done; + } + + circuit_build_times_set_timeout(cbt); + + if (!state->CircuitBuildAbandonedCount && cbt->total_build_times) { + circuit_build_times_filter_timeouts(cbt); + } + + done: + tor_free(loaded_times); + return err ? -1 : 0; +} + +/** + * Estimates the Xm and Alpha parameters using + * http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation + * + * The notable difference is that we use mode instead of min to estimate Xm. + * This is because our distribution is frechet-like. We claim this is + * an acceptable approximation because we are only concerned with the + * accuracy of the CDF of the tail. + */ +int +circuit_build_times_update_alpha(circuit_build_times_t *cbt) +{ + build_time_t *x=cbt->circuit_build_times; + double a = 0; + int n=0,i=0,abandoned_count=0; + build_time_t max_time=0; + + /* http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation */ + /* We sort of cheat here and make our samples slightly more pareto-like + * and less frechet-like. */ + cbt->Xm = circuit_build_times_get_xm(cbt); + + tor_assert(cbt->Xm > 0); + + for (i=0; i< CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (!x[i]) { + continue; + } + + if (x[i] < cbt->Xm) { + a += tor_mathlog(cbt->Xm); + } else if (x[i] == CBT_BUILD_ABANDONED) { + abandoned_count++; + } else { + a += tor_mathlog(x[i]); + if (x[i] > max_time) + max_time = x[i]; + } + n++; + } + + /* + * We are erring and asserting here because this can only happen + * in codepaths other than startup. The startup state parsing code + * performs this same check, and resets state if it hits it. If we + * hit it at runtime, something serious has gone wrong. + */ + if (n!=cbt->total_build_times) { + log_err(LD_CIRC, "Discrepancy in build times count: %d vs %d", n, + cbt->total_build_times); + } + tor_assert(n==cbt->total_build_times); + + if (max_time <= 0) { + /* This can happen if Xm is actually the *maximum* value in the set. + * It can also happen if we've abandoned every single circuit somehow. + * In either case, tell the caller not to compute a new build timeout. */ + log_warn(LD_BUG, + "Could not determine largest build time (%d). " + "Xm is %dms and we've abandoned %d out of %d circuits.", max_time, + cbt->Xm, abandoned_count, n); + return 0; + } + + a += abandoned_count*tor_mathlog(max_time); + + a -= n*tor_mathlog(cbt->Xm); + // Estimator comes from Eq #4 in: + // "Bayesian estimation based on trimmed samples from Pareto populations" + // by Arturo J. Fernández. We are right-censored only. + a = (n-abandoned_count)/a; + + cbt->alpha = a; + + return 1; +} + +/** + * This is the Pareto Quantile Function. It calculates the point x + * in the distribution such that F(x) = quantile (ie quantile*100% + * of the mass of the density function is below x on the curve). + * + * We use it to calculate the timeout and also to generate synthetic + * values of time for circuits that timeout before completion. + * + * See http://en.wikipedia.org/wiki/Quantile_function, + * http://en.wikipedia.org/wiki/Inverse_transform_sampling and + * http://en.wikipedia.org/wiki/Pareto_distribution#Generating_a_ + * random_sample_from_Pareto_distribution + * That's right. I'll cite wikipedia all day long. + * + * Return value is in milliseconds. + */ +double +circuit_build_times_calculate_timeout(circuit_build_times_t *cbt, + double quantile) +{ + double ret; + tor_assert(quantile >= 0); + tor_assert(1.0-quantile > 0); + tor_assert(cbt->Xm > 0); + + ret = cbt->Xm/pow(1.0-quantile,1.0/cbt->alpha); + if (ret > INT32_MAX) { + ret = INT32_MAX; + } + tor_assert(ret > 0); + return ret; +} + +/** Pareto CDF */ +double +circuit_build_times_cdf(circuit_build_times_t *cbt, double x) +{ + double ret; + tor_assert(cbt->Xm > 0); + ret = 1.0-pow(cbt->Xm/x,cbt->alpha); + tor_assert(0 <= ret && ret <= 1.0); + return ret; +} + +/** + * Generate a synthetic time using our distribution parameters. + * + * The return value will be within the [q_lo, q_hi) quantile points + * on the CDF. + */ +build_time_t +circuit_build_times_generate_sample(circuit_build_times_t *cbt, + double q_lo, double q_hi) +{ + double randval = crypto_rand_double(); + build_time_t ret; + double u; + + /* Generate between [q_lo, q_hi) */ + /*XXXX This is what nextafter is supposed to be for; we should use it on the + * platforms that support it. */ + q_hi -= 1.0/(INT32_MAX); + + tor_assert(q_lo >= 0); + tor_assert(q_hi < 1); + tor_assert(q_lo < q_hi); + + u = q_lo + (q_hi-q_lo)*randval; + + tor_assert(0 <= u && u < 1.0); + /* circuit_build_times_calculate_timeout returns <= INT32_MAX */ + ret = (build_time_t) + tor_lround(circuit_build_times_calculate_timeout(cbt, u)); + tor_assert(ret > 0); + return ret; +} + +/** + * Estimate an initial alpha parameter by solving the quantile + * function with a quantile point and a specific timeout value. + */ +void +circuit_build_times_initial_alpha(circuit_build_times_t *cbt, + double quantile, double timeout_ms) +{ + // Q(u) = Xm/((1-u)^(1/a)) + // Q(0.8) = Xm/((1-0.8))^(1/a)) = CircBuildTimeout + // CircBuildTimeout = Xm/((1-0.8))^(1/a)) + // CircBuildTimeout = Xm*((1-0.8))^(-1/a)) + // ln(CircBuildTimeout) = ln(Xm)+ln(((1-0.8)))*(-1/a) + // -ln(1-0.8)/(ln(CircBuildTimeout)-ln(Xm))=a + tor_assert(quantile >= 0); + tor_assert(cbt->Xm > 0); + cbt->alpha = tor_mathlog(1.0-quantile)/ + (tor_mathlog(cbt->Xm)-tor_mathlog(timeout_ms)); + tor_assert(cbt->alpha > 0); +} + +/** + * Returns true if we need circuits to be built + */ +int +circuit_build_times_needs_circuits(circuit_build_times_t *cbt) +{ + /* Return true if < MIN_CIRCUITS_TO_OBSERVE */ + return !circuit_build_times_enough_to_compute(cbt); +} + +/** + * Returns true if we should build a timeout test circuit + * right now. + */ +int +circuit_build_times_needs_circuits_now(circuit_build_times_t *cbt) +{ + return circuit_build_times_needs_circuits(cbt) && + approx_time()-cbt->last_circ_at > circuit_build_times_test_frequency(); +} + +/** + * Called to indicate that the network showed some signs of liveness, + * i.e. we received a cell. + * + * This is used by circuit_build_times_network_check_live() to decide + * if we should record the circuit build timeout or not. + * + * This function is called every time we receive a cell. Avoid + * syscalls, events, and other high-intensity work. + */ +void +circuit_build_times_network_is_live(circuit_build_times_t *cbt) +{ + time_t now = approx_time(); + if (cbt->liveness.nonlive_timeouts > 0) { + log_notice(LD_CIRC, + "Tor now sees network activity. Restoring circuit build " + "timeout recording. Network was down for %d seconds " + "during %d circuit attempts.", + (int)(now - cbt->liveness.network_last_live), + cbt->liveness.nonlive_timeouts); + } + cbt->liveness.network_last_live = now; + cbt->liveness.nonlive_timeouts = 0; +} + +/** + * Called to indicate that we completed a circuit. Because this circuit + * succeeded, it doesn't count as a timeout-after-the-first-hop. + * + * This is used by circuit_build_times_network_check_changed() to determine + * if we had too many recent timeouts and need to reset our learned timeout + * to something higher. + */ +void +circuit_build_times_network_circ_success(circuit_build_times_t *cbt) +{ + cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx] = 0; + cbt->liveness.after_firsthop_idx++; + cbt->liveness.after_firsthop_idx %= cbt->liveness.num_recent_circs; +} + +/** + * A circuit just timed out. If it failed after the first hop, record it + * in our history for later deciding if the network speed has changed. + * + * This is used by circuit_build_times_network_check_changed() to determine + * if we had too many recent timeouts and need to reset our learned timeout + * to something higher. + */ +static void +circuit_build_times_network_timeout(circuit_build_times_t *cbt, + int did_onehop) +{ + if (did_onehop) { + cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx]=1; + cbt->liveness.after_firsthop_idx++; + cbt->liveness.after_firsthop_idx %= cbt->liveness.num_recent_circs; + } +} + +/** + * A circuit was just forcibly closed. If there has been no recent network + * activity at all, but this circuit was launched back when we thought the + * network was live, increment the number of "nonlive" circuit timeouts. + * + * This is used by circuit_build_times_network_check_live() to decide + * if we should record the circuit build timeout or not. + */ +static void +circuit_build_times_network_close(circuit_build_times_t *cbt, + int did_onehop, time_t start_time) +{ + time_t now = time(NULL); + /* + * Check if this is a timeout that was for a circuit that spent its + * entire existence during a time where we have had no network activity. + */ + if (cbt->liveness.network_last_live < start_time) { + if (did_onehop) { + char last_live_buf[ISO_TIME_LEN+1]; + char start_time_buf[ISO_TIME_LEN+1]; + char now_buf[ISO_TIME_LEN+1]; + format_local_iso_time(last_live_buf, cbt->liveness.network_last_live); + format_local_iso_time(start_time_buf, start_time); + format_local_iso_time(now_buf, now); + log_warn(LD_BUG, + "Circuit somehow completed a hop while the network was " + "not live. Network was last live at %s, but circuit launched " + "at %s. It's now %s.", last_live_buf, start_time_buf, + now_buf); + } + cbt->liveness.nonlive_timeouts++; + if (cbt->liveness.nonlive_timeouts == 1) { + log_notice(LD_CIRC, + "Tor has not observed any network activity for the past %d " + "seconds. Disabling circuit build timeout recording.", + (int)(now - cbt->liveness.network_last_live)); + } else { + log_info(LD_CIRC, + "Got non-live timeout. Current count is: %d", + cbt->liveness.nonlive_timeouts); + } + } +} + +/** + * When the network is not live, we do not record circuit build times. + * + * The network is considered not live if there has been at least one + * circuit build that began and ended (had its close_ms measurement + * period expire) since we last received a cell. + * + * Also has the side effect of rewinding the circuit time history + * in the case of recent liveness changes. + */ +int +circuit_build_times_network_check_live(circuit_build_times_t *cbt) +{ + if (cbt->liveness.nonlive_timeouts > 0) { + return 0; + } + + return 1; +} + +/** + * Returns true if we have seen more than MAX_RECENT_TIMEOUT_COUNT of + * the past RECENT_CIRCUITS time out after the first hop. Used to detect + * if the network connection has changed significantly, and if so, + * resets our circuit build timeout to the default. + * + * Also resets the entire timeout history in this case and causes us + * to restart the process of building test circuits and estimating a + * new timeout. + */ +int +circuit_build_times_network_check_changed(circuit_build_times_t *cbt) +{ + int total_build_times = cbt->total_build_times; + int timeout_count=0; + int i; + + /* how many of our recent circuits made it to the first hop but then + * timed out? */ + for (i = 0; i < cbt->liveness.num_recent_circs; i++) { + timeout_count += cbt->liveness.timeouts_after_firsthop[i]; + } + + /* If 80% of our recent circuits are timing out after the first hop, + * we need to re-estimate a new initial alpha and timeout. */ + if (timeout_count < circuit_build_times_max_timeouts()) { + return 0; + } + + circuit_build_times_reset(cbt); + memset(cbt->liveness.timeouts_after_firsthop, 0, + sizeof(*cbt->liveness.timeouts_after_firsthop)* + cbt->liveness.num_recent_circs); + cbt->liveness.after_firsthop_idx = 0; + + /* Check to see if this has happened before. If so, double the timeout + * to give people on abysmally bad network connections a shot at access */ + if (cbt->timeout_ms >= circuit_build_times_get_initial_timeout()) { + if (cbt->timeout_ms > INT32_MAX/2 || cbt->close_ms > INT32_MAX/2) { + log_warn(LD_CIRC, "Insanely large circuit build timeout value. " + "(timeout = %lfmsec, close = %lfmsec)", + cbt->timeout_ms, cbt->close_ms); + } else { + cbt->timeout_ms *= 2; + cbt->close_ms *= 2; + } + } else { + cbt->close_ms = cbt->timeout_ms + = circuit_build_times_get_initial_timeout(); + } + + control_event_buildtimeout_set(cbt, BUILDTIMEOUT_SET_EVENT_RESET); + + log_notice(LD_CIRC, + "Your network connection speed appears to have changed. Resetting " + "timeout to %lds after %d timeouts and %d buildtimes.", + tor_lround(cbt->timeout_ms/1000), timeout_count, + total_build_times); + + return 1; +} + +/** + * Count the number of timeouts in a set of cbt data. + */ +double +circuit_build_times_timeout_rate(const circuit_build_times_t *cbt) +{ + int i=0,timeouts=0; + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] >= cbt->timeout_ms) { + timeouts++; + } + } + + if (!cbt->total_build_times) + return 0; + + return ((double)timeouts)/cbt->total_build_times; +} + +/** + * Count the number of closed circuits in a set of cbt data. + */ +double +circuit_build_times_close_rate(const circuit_build_times_t *cbt) +{ + int i=0,closed=0; + for (i = 0; i < CBT_NCIRCUITS_TO_OBSERVE; i++) { + if (cbt->circuit_build_times[i] == CBT_BUILD_ABANDONED) { + closed++; + } + } + + if (!cbt->total_build_times) + return 0; + + return ((double)closed)/cbt->total_build_times; +} + +/** + * Store a timeout as a synthetic value. + * + * Returns true if the store was successful and we should possibly + * update our timeout estimate. + */ +int +circuit_build_times_count_close(circuit_build_times_t *cbt, + int did_onehop, + time_t start_time) +{ + if (circuit_build_times_disabled()) { + cbt->close_ms = cbt->timeout_ms + = circuit_build_times_get_initial_timeout(); + return 0; + } + + /* Record this force-close to help determine if the network is dead */ + circuit_build_times_network_close(cbt, did_onehop, start_time); + + /* Only count timeouts if network is live.. */ + if (!circuit_build_times_network_check_live(cbt)) { + return 0; + } + + circuit_build_times_add_time(cbt, CBT_BUILD_ABANDONED); + return 1; +} + +/** + * Update timeout counts to determine if we need to expire + * our build time history due to excessive timeouts. + * + * We do not record any actual time values at this stage; + * we are only interested in recording the fact that a timeout + * happened. We record the time values via + * circuit_build_times_count_close() and circuit_build_times_add_time(). + */ +void +circuit_build_times_count_timeout(circuit_build_times_t *cbt, + int did_onehop) +{ + if (circuit_build_times_disabled()) { + cbt->close_ms = cbt->timeout_ms + = circuit_build_times_get_initial_timeout(); + return; + } + + /* Register the fact that a timeout just occurred. */ + circuit_build_times_network_timeout(cbt, did_onehop); + + /* If there are a ton of timeouts, we should reset + * the circuit build timeout. */ + circuit_build_times_network_check_changed(cbt); +} + +/** + * Estimate a new timeout based on history and set our timeout + * variable accordingly. + */ +static int +circuit_build_times_set_timeout_worker(circuit_build_times_t *cbt) +{ + build_time_t max_time; + if (!circuit_build_times_enough_to_compute(cbt)) + return 0; + + if (!circuit_build_times_update_alpha(cbt)) + return 0; + + cbt->timeout_ms = circuit_build_times_calculate_timeout(cbt, + circuit_build_times_quantile_cutoff()); + + cbt->close_ms = circuit_build_times_calculate_timeout(cbt, + circuit_build_times_close_quantile()); + + max_time = circuit_build_times_max(cbt); + + /* Sometimes really fast guard nodes give us such a steep curve + * that this ends up being not that much greater than timeout_ms. + * Make it be at least 1 min to handle this case. */ + cbt->close_ms = MAX(cbt->close_ms, circuit_build_times_initial_timeout()); + + if (cbt->timeout_ms > max_time) { + log_notice(LD_CIRC, + "Circuit build timeout of %dms is beyond the maximum build " + "time we have ever observed. Capping it to %dms.", + (int)cbt->timeout_ms, max_time); + cbt->timeout_ms = max_time; + } + + if (max_time < INT32_MAX/2 && cbt->close_ms > 2*max_time) { + log_info(LD_CIRC, + "Circuit build measurement period of %dms is more than twice " + "the maximum build time we have ever observed. Capping it to " + "%dms.", (int)cbt->close_ms, 2*max_time); + cbt->close_ms = 2*max_time; + } + + cbt->have_computed_timeout = 1; + return 1; +} + +/** + * Exposed function to compute a new timeout. Dispatches events and + * also filters out extremely high timeout values. + */ +void +circuit_build_times_set_timeout(circuit_build_times_t *cbt) +{ + long prev_timeout = tor_lround(cbt->timeout_ms/1000); + double timeout_rate; + + if (!circuit_build_times_set_timeout_worker(cbt)) + return; + + if (cbt->timeout_ms < circuit_build_times_min_timeout()) { + log_warn(LD_CIRC, "Set buildtimeout to low value %lfms. Setting to %dms", + cbt->timeout_ms, circuit_build_times_min_timeout()); + cbt->timeout_ms = circuit_build_times_min_timeout(); + if (cbt->close_ms < cbt->timeout_ms) { + /* This shouldn't happen because of MAX() in timeout_worker above, + * but doing it just in case */ + cbt->close_ms = circuit_build_times_initial_timeout(); + } + } + + control_event_buildtimeout_set(cbt, BUILDTIMEOUT_SET_EVENT_COMPUTED); + + timeout_rate = circuit_build_times_timeout_rate(cbt); + + if (prev_timeout > tor_lround(cbt->timeout_ms/1000)) { + log_notice(LD_CIRC, + "Based on %d circuit times, it looks like we don't need to " + "wait so long for circuits to finish. We will now assume a " + "circuit is too slow to use after waiting %ld seconds.", + cbt->total_build_times, + tor_lround(cbt->timeout_ms/1000)); + log_info(LD_CIRC, + "Circuit timeout data: %lfms, %lfms, Xm: %d, a: %lf, r: %lf", + cbt->timeout_ms, cbt->close_ms, cbt->Xm, cbt->alpha, + timeout_rate); + } else if (prev_timeout < tor_lround(cbt->timeout_ms/1000)) { + log_notice(LD_CIRC, + "Based on %d circuit times, it looks like we need to wait " + "longer for circuits to finish. We will now assume a " + "circuit is too slow to use after waiting %ld seconds.", + cbt->total_build_times, + tor_lround(cbt->timeout_ms/1000)); + log_info(LD_CIRC, + "Circuit timeout data: %lfms, %lfms, Xm: %d, a: %lf, r: %lf", + cbt->timeout_ms, cbt->close_ms, cbt->Xm, cbt->alpha, + timeout_rate); + } else { + log_info(LD_CIRC, + "Set circuit build timeout to %lds (%lfms, %lfms, Xm: %d, a: %lf," + " r: %lf) based on %d circuit times", + tor_lround(cbt->timeout_ms/1000), + cbt->timeout_ms, cbt->close_ms, cbt->Xm, cbt->alpha, timeout_rate, + cbt->total_build_times); + } +} + /** Iterate over values of circ_id, starting from conn-\>next_circ_id, * and with the high bit specified by conn-\>circ_id_type, until we get * a circ_id that is not in use by any other circuit on that conn. @@ -112,21 +1505,21 @@ circuit_list_path_impl(origin_circuit_t *circ, int verbose, int verbose_names) crypt_path_t *hop; smartlist_t *elements; const char *states[] = {"closed", "waiting for keys", "open"}; - char buf[128]; char *s; elements = smartlist_create(); if (verbose) { const char *nickname = build_state_get_exit_nickname(circ->build_state); - tor_snprintf(buf, sizeof(buf), "%s%s circ (length %d%s%s):", + char *cp; + tor_asprintf(&cp, "%s%s circ (length %d%s%s):", circ->build_state->is_internal ? "internal" : "exit", circ->build_state->need_uptime ? " (high-uptime)" : "", circ->build_state->desired_path_len, - circ->_base.state == CIRCUIT_STATE_OPEN ? "" : ", exit ", + circ->_base.state == CIRCUIT_STATE_OPEN ? "" : ", last hop ", circ->_base.state == CIRCUIT_STATE_OPEN ? "" : (nickname?nickname:"*unnamed*")); - smartlist_add(elements, tor_strdup(buf)); + smartlist_add(elements, cp); } hop = circ->cpath; @@ -148,8 +1541,7 @@ circuit_list_path_impl(origin_circuit_t *circ, int verbose, int verbose_names) router_get_verbose_nickname(elt, ri); } else if ((rs = router_get_consensus_status_by_id(id))) { routerstatus_get_verbose_nickname(elt, rs); - } else if (hop->extend_info->nickname && - is_legal_nickname(hop->extend_info->nickname)) { + } else if (is_legal_nickname(hop->extend_info->nickname)) { elt[0] = '$'; base16_encode(elt+1, HEX_DIGEST_LEN+1, id, DIGEST_LEN); elt[HEX_DIGEST_LEN+1]= '~'; @@ -217,7 +1609,7 @@ void circuit_log_path(int severity, unsigned int domain, origin_circuit_t *circ) { char *s = circuit_list_path(circ,1); - log(severity,domain,"%s",s); + tor_log(severity,domain,"%s",s); tor_free(s); } @@ -269,7 +1661,7 @@ static int onion_populate_cpath(origin_circuit_t *circ) { int r; -again: + again: r = onion_extend_cpath(circ); if (r < 0) { log_info(LD_CIRC,"Generating cpath hop failed."); @@ -361,9 +1753,10 @@ circuit_handle_first_hop(origin_circuit_t *circ) if (!n_conn) { /* not currently connected in a useful way. */ - const char *name = firsthop->extend_info->nickname ? + const char *name = strlen(firsthop->extend_info->nickname) ? firsthop->extend_info->nickname : fmt_addr(&firsthop->extend_info->addr); - log_info(LD_CIRC, "Next router is %s: %s ", safe_str(name), msg?msg:"???"); + log_info(LD_CIRC, "Next router is %s: %s ", + safe_str_client(name), msg?msg:"???"); circ->_base.n_hop = extend_info_dup(firsthop->extend_info); if (should_launch) { @@ -506,7 +1899,8 @@ circuit_deliver_create_cell(circuit_t *circ, uint8_t cell_type, cell.circ_id = circ->n_circ_id; memcpy(cell.payload, payload, ONIONSKIN_CHALLENGE_LEN); - append_cell_to_circuit_queue(circ, circ->n_conn, &cell, CELL_DIRECTION_OUT); + append_cell_to_circuit_queue(circ, circ->n_conn, &cell, + CELL_DIRECTION_OUT, 0); if (CIRCUIT_IS_ORIGIN(circ)) { /* mark it so it gets better rate limiting treatment. */ @@ -536,7 +1930,7 @@ inform_testing_reachability(void) "CHECKING_REACHABILITY DIRADDRESS=%s:%d", me->address, me->dir_port); } - log(LOG_NOTICE, LD_OR, "Now checking whether ORPort %s:%d%s %s reachable... " + log_notice(LD_OR, "Now checking whether ORPort %s:%d%s %s reachable... " "(this may take up to %d minutes -- look for log " "messages indicating success)", me->address, me->or_port, @@ -569,6 +1963,18 @@ should_use_create_fast_for_circuit(origin_circuit_t *circ) return 1; } +/** Return true if <b>circ</b> is the type of circuit we want to count + * timeouts from. In particular, we want it to have not completed yet + * (already completing indicates we cannibalized it), and we want it to + * have exactly three hops. + */ +int +circuit_timeout_want_to_count_circ(origin_circuit_t *circ) +{ + return !circ->has_opened + && circ->build_state->desired_path_len == DEFAULT_ROUTE_LEN; +} + /** This is the backbone function for building circuits. * * If circ's first hop is closed, then we need to build a create @@ -642,15 +2048,43 @@ circuit_send_next_onion_skin(origin_circuit_t *circ) if (!hop) { /* done building the circuit. whew. */ circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_OPEN); + if (circuit_timeout_want_to_count_circ(circ)) { + struct timeval end; + long timediff; + tor_gettimeofday(&end); + timediff = tv_mdiff(&circ->_base.timestamp_created, &end); + + /* + * If the circuit build time is much greater than we would have cut + * it off at, we probably had a suspend event along this codepath, + * and we should discard the value. + */ + if (timediff < 0 || timediff > 2*circ_times.close_ms+1000) { + log_notice(LD_CIRC, "Strange value for circuit build time: %ldmsec. " + "Assuming clock jump. Purpose %d (%s)", timediff, + circ->_base.purpose, + circuit_purpose_to_string(circ->_base.purpose)); + } else if (!circuit_build_times_disabled()) { + /* Only count circuit times if the network is live */ + if (circuit_build_times_network_check_live(&circ_times)) { + circuit_build_times_add_time(&circ_times, (build_time_t)timediff); + circuit_build_times_set_timeout(&circ_times); + } + + if (circ->_base.purpose != CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) { + circuit_build_times_network_circ_success(&circ_times); + } + } + } log_info(LD_CIRC,"circuit built!"); circuit_reset_failure_count(0); if (circ->build_state->onehop_tunnel) control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_STATUS, 0); - if (!has_completed_circuit && !circ->build_state->onehop_tunnel) { + if (!can_complete_circuit && !circ->build_state->onehop_tunnel) { or_options_t *options = get_options(); - has_completed_circuit=1; + can_complete_circuit=1; /* FFFF Log a count of known routers here */ - log(LOG_NOTICE, LD_GENERAL, + log_notice(LD_GENERAL, "Tor has successfully opened a circuit. " "Looks like client functionality is working."); control_event_bootstrap(BOOTSTRAP_STATUS_DONE, 0); @@ -662,6 +2096,10 @@ circuit_send_next_onion_skin(origin_circuit_t *circ) } circuit_rep_hist_note_result(circ); circuit_has_opened(circ); /* do other actions as necessary */ + + /* We're done with measurement circuits here. Just close them */ + if (circ->_base.purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) + circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_FINISHED); return 0; } @@ -705,13 +2143,13 @@ void circuit_note_clock_jumped(int seconds_elapsed) { int severity = server_mode(get_options()) ? LOG_WARN : LOG_NOTICE; - log(severity, LD_GENERAL, "Your system clock just jumped %d seconds %s; " + tor_log(severity, LD_GENERAL, "Your system clock just jumped %d seconds %s; " "assuming established circuits no longer work.", seconds_elapsed >=0 ? seconds_elapsed : -seconds_elapsed, seconds_elapsed >=0 ? "forward" : "backward"); control_event_general_status(LOG_WARN, "CLOCK_JUMPED TIME=%d", seconds_elapsed); - has_completed_circuit=0; /* so it'll log when it works again */ + can_complete_circuit=0; /* so it'll log when it works again */ control_event_client_status(severity, "CIRCUIT_NOT_ESTABLISHED REASON=%s", "CLOCK_JUMPED"); circuit_mark_all_unused_circs(); @@ -944,10 +2382,9 @@ circuit_finish_handshake(origin_circuit_t *circ, uint8_t reply_type, return -END_CIRC_REASON_TORPROTOCOL; } - if (hop->dh_handshake_state) { - crypto_dh_free(hop->dh_handshake_state); /* don't need it anymore */ - hop->dh_handshake_state = NULL; - } + crypto_dh_free(hop->dh_handshake_state); /* don't need it anymore */ + hop->dh_handshake_state = NULL; + memset(hop->fast_handshake_state, 0, sizeof(hop->fast_handshake_state)); if (circuit_init_cpath_crypto(hop, keys, 0)<0) { @@ -1035,8 +2472,8 @@ onionskin_answer(or_circuit_t *circ, uint8_t cell_type, const char *payload, cell_type == CELL_CREATED ? ONIONSKIN_REPLY_LEN : DIGEST_LEN*2); log_debug(LD_CIRC,"init digest forward 0x%.8x, backward 0x%.8x.", - (unsigned int)*(uint32_t*)(keys), - (unsigned int)*(uint32_t*)(keys+20)); + (unsigned int)get_uint32(keys), + (unsigned int)get_uint32(keys+20)); if (circuit_init_cpath_crypto(tmp_cpath, keys, 0)<0) { log_warn(LD_BUG,"Circuit initialization failed"); tor_free(tmp_cpath); @@ -1057,7 +2494,7 @@ onionskin_answer(or_circuit_t *circ, uint8_t cell_type, const char *payload, circ->is_first_hop = (cell_type == CELL_CREATED_FAST); append_cell_to_circuit_queue(TO_CIRCUIT(circ), - circ->p_conn, &cell, CELL_DIRECTION_IN); + circ->p_conn, &cell, CELL_DIRECTION_IN, 0); log_debug(LD_CIRC,"Finished sending 'created' cell."); if (!is_local_addr(&circ->p_conn->_base.addr) && @@ -1086,7 +2523,7 @@ new_route_len(uint8_t purpose, extend_info_t *exit, tor_assert(routers); - routelen = 3; + routelen = DEFAULT_ROUTE_LEN; if (exit && purpose != CIRCUIT_PURPOSE_TESTING && purpose != CIRCUIT_PURPOSE_S_ESTABLISH_INTRO) @@ -1151,6 +2588,8 @@ circuit_all_predicted_ports_handled(time_t now, int *need_uptime, smartlist_t *LongLivedServices = get_options()->LongLivedPorts; tor_assert(need_uptime); tor_assert(need_capacity); + // Always predict need_capacity + *need_capacity = 1; enough = (smartlist_len(sl) == 0); for (i = 0; i < smartlist_len(sl); ++i) { port = smartlist_get(sl, i); @@ -1173,6 +2612,8 @@ router_handles_some_port(routerinfo_t *router, smartlist_t *needed_ports) for (i = 0; i < smartlist_len(needed_ports); ++i) { addr_policy_result_t r; + /* alignment issues aren't a worry for this dereference, since + needed_ports is explicitly a smartlist of uint16_t's */ port = *(uint16_t *)smartlist_get(needed_ports, i); tor_assert(port); r = compare_addr_to_addr_policy(0, port, router->exit_policy); @@ -1255,9 +2696,24 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, n_supported[i] = -1; continue; /* skip routers that are known to be down or bad exits */ } + + if (options->_ExcludeExitNodesUnion && + routerset_contains_router(options->_ExcludeExitNodesUnion, router)) { + n_supported[i] = -1; + continue; /* user asked us not to use it, no matter what */ + } + if (options->ExitNodes && + !routerset_contains_router(options->ExitNodes, router)) { + n_supported[i] = -1; + continue; /* not one of our chosen exit nodes */ + } + if (router_is_unreliable(router, need_uptime, need_capacity, 0)) { n_supported[i] = -1; - continue; /* skip routers that are not suitable */ + continue; /* skip routers that are not suitable. Don't worry if + * this makes us reject all the possible routers: if so, + * we'll retry later in this function with need_update and + * need_capacity set to 0. */ } if (!(router->is_valid || options->_AllowInvalid & ALLOW_INVALID_EXIT)) { /* if it's invalid and we don't want it */ @@ -1316,28 +2772,21 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, /* If any routers definitely support any pending connections, choose one * at random. */ if (best_support > 0) { - smartlist_t *supporting = smartlist_create(), *use = smartlist_create(); + smartlist_t *supporting = smartlist_create(); for (i = 0; i < smartlist_len(dir->routers); i++) if (n_supported[i] == best_support) smartlist_add(supporting, smartlist_get(dir->routers, i)); - routersets_get_disjunction(use, supporting, options->ExitNodes, - options->_ExcludeExitNodesUnion, 1); - if (smartlist_len(use) == 0 && !options->StrictExitNodes) { - routersets_get_disjunction(use, supporting, NULL, - options->_ExcludeExitNodesUnion, 1); - } - router = routerlist_sl_choose_by_bandwidth(use, WEIGHT_FOR_EXIT); - smartlist_free(use); + router = routerlist_sl_choose_by_bandwidth(supporting, WEIGHT_FOR_EXIT); smartlist_free(supporting); } else { /* Either there are no pending connections, or no routers even seem to * possibly support any of them. Choose a router at random that satisfies * at least one predicted exit port. */ - int try; - smartlist_t *needed_ports, *supporting, *use; + int attempt; + smartlist_t *needed_ports, *supporting; if (best_support == -1) { if (need_uptime || need_capacity) { @@ -1349,42 +2798,32 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, tor_free(n_supported); return choose_good_exit_server_general(dir, 0, 0); } - log_notice(LD_CIRC, "All routers are down or won't exit -- choosing a " - "doomed exit at random."); + log_notice(LD_CIRC, "All routers are down or won't exit%s -- " + "choosing a doomed exit at random.", + options->_ExcludeExitNodesUnion ? " or are Excluded" : ""); } supporting = smartlist_create(); - use = smartlist_create(); needed_ports = circuit_get_unhandled_ports(time(NULL)); - for (try = 0; try < 2; try++) { + for (attempt = 0; attempt < 2; attempt++) { /* try once to pick only from routers that satisfy a needed port, * then if there are none, pick from any that support exiting. */ for (i = 0; i < smartlist_len(dir->routers); i++) { router = smartlist_get(dir->routers, i); if (n_supported[i] != -1 && - (try || router_handles_some_port(router, needed_ports))) { + (attempt || router_handles_some_port(router, needed_ports))) { // log_fn(LOG_DEBUG,"Try %d: '%s' is a possibility.", // try, router->nickname); smartlist_add(supporting, router); } } - routersets_get_disjunction(use, supporting, options->ExitNodes, - options->_ExcludeExitNodesUnion, 1); - if (smartlist_len(use) == 0 && !options->StrictExitNodes) { - routersets_get_disjunction(use, supporting, NULL, - options->_ExcludeExitNodesUnion, 1); - } - /* XXX sometimes the above results in null, when the requested - * exit node is down. we should pick it anyway. */ - router = routerlist_sl_choose_by_bandwidth(use, WEIGHT_FOR_EXIT); + router = routerlist_sl_choose_by_bandwidth(supporting, WEIGHT_FOR_EXIT); if (router) break; smartlist_clear(supporting); - smartlist_clear(use); } SMARTLIST_FOREACH(needed_ports, uint16_t *, cp, tor_free(cp)); smartlist_free(needed_ports); - smartlist_free(use); smartlist_free(supporting); } @@ -1393,10 +2832,11 @@ choose_good_exit_server_general(routerlist_t *dir, int need_uptime, log_info(LD_CIRC, "Chose exit server '%s'", router->nickname); return router; } - if (options->StrictExitNodes) { + if (options->ExitNodes) { log_warn(LD_CIRC, - "No specified exit routers seem to be running, and " - "StrictExitNodes is set: can't choose an exit."); + "No specified %sexit routers seem to be running: " + "can't choose an exit.", + options->_ExcludeExitNodesUnion ? "non-excluded " : ""); } return NULL; } @@ -1427,15 +2867,13 @@ choose_good_exit_server(uint8_t purpose, routerlist_t *dir, if (options->_AllowInvalid & ALLOW_INVALID_MIDDLE) flags |= CRN_ALLOW_INVALID; if (is_internal) /* pick it like a middle hop */ - return router_choose_random_node(NULL, NULL, - options->ExcludeNodes, flags); + return router_choose_random_node(NULL, options->ExcludeNodes, flags); else return choose_good_exit_server_general(dir,need_uptime,need_capacity); case CIRCUIT_PURPOSE_C_ESTABLISH_REND: if (options->_AllowInvalid & ALLOW_INVALID_RENDEZVOUS) flags |= CRN_ALLOW_INVALID; - return router_choose_random_node(NULL, NULL, - options->ExcludeNodes, flags); + return router_choose_random_node(NULL, options->ExcludeNodes, flags); } log_warn(LD_BUG,"Unhandled purpose %d", purpose); tor_fragile_assert(); @@ -1450,7 +2888,6 @@ warn_if_last_router_excluded(origin_circuit_t *circ, const extend_info_t *exit) or_options_t *options = get_options(); routerset_t *rs = options->ExcludeNodes; const char *description; - int domain = LD_CIRC; uint8_t purpose = circ->_base.purpose; if (circ->build_state->onehop_tunnel) @@ -1463,13 +2900,14 @@ warn_if_last_router_excluded(origin_circuit_t *circ, const extend_info_t *exit) case CIRCUIT_PURPOSE_INTRO_POINT: case CIRCUIT_PURPOSE_REND_POINT_WAITING: case CIRCUIT_PURPOSE_REND_ESTABLISHED: - log_warn(LD_BUG, "Called on non-origin circuit (purpose %d)", - (int)purpose); + log_warn(LD_BUG, "Called on non-origin circuit (purpose %d, %s)", + (int)purpose, + circuit_purpose_to_string(purpose)); return; case CIRCUIT_PURPOSE_C_GENERAL: if (circ->build_state->is_internal) return; - description = "Requested exit node"; + description = "requested exit node"; rs = options->_ExcludeExitNodesUnion; break; case CIRCUIT_PURPOSE_C_INTRODUCING: @@ -1484,22 +2922,34 @@ warn_if_last_router_excluded(origin_circuit_t *circ, const extend_info_t *exit) case CIRCUIT_PURPOSE_C_REND_READY: case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: case CIRCUIT_PURPOSE_C_REND_JOINED: - description = "Chosen rendezvous point"; - domain = LD_BUG; + description = "chosen rendezvous point"; break; case CIRCUIT_PURPOSE_CONTROLLER: rs = options->_ExcludeExitNodesUnion; - description = "Controller-selected circuit target"; + description = "controller-selected circuit target"; break; } if (routerset_contains_extendinfo(rs, exit)) { - log_fn(LOG_WARN, domain, "%s '%s' is in ExcludeNodes%s. Using anyway " - "(circuit purpose %d).", - description,exit->nickname, - rs==options->ExcludeNodes?"":" or ExcludeExitNodes", - (int)purpose); - circuit_log_path(LOG_WARN, domain, circ); + /* We should never get here if StrictNodes is set to 1. */ + if (options->StrictNodes) { + log_warn(LD_BUG, "Using %s '%s' which is listed in ExcludeNodes%s, " + "even though StrictNodes is set. Please report. " + "(Circuit purpose: %s)", + description, exit->nickname, + rs==options->ExcludeNodes?"":" or ExcludeExitNodes", + circuit_purpose_to_string(purpose)); + } else { + log_warn(LD_CIRC, "Using %s '%s' which is listed in " + "ExcludeNodes%s, because no better options were available. To " + "prevent this (and possibly break your Tor functionality), " + "set the StrictNodes configuration option. " + "(Circuit purpose: %s)", + description, exit->nickname, + rs==options->ExcludeNodes?"":" or ExcludeExitNodes", + circuit_purpose_to_string(purpose)); + } + circuit_log_path(LOG_WARN, LD_CIRC, circ); } return; @@ -1555,8 +3005,7 @@ circuit_append_new_exit(origin_circuit_t *circ, extend_info_t *exit) state = circ->build_state; tor_assert(state); - if (state->chosen_exit) - extend_info_free(state->chosen_exit); + extend_info_free(state->chosen_exit); state->chosen_exit = extend_info_dup(exit); ++circ->build_state->desired_path_len; @@ -1678,8 +3127,7 @@ choose_good_middle_server(uint8_t purpose, flags |= CRN_NEED_CAPACITY; if (options->_AllowInvalid & ALLOW_INVALID_MIDDLE) flags |= CRN_ALLOW_INVALID; - choice = router_choose_random_node(NULL, - excluded, options->ExcludeNodes, flags); + choice = router_choose_random_node(excluded, options->ExcludeNodes, flags); smartlist_free(excluded); return choice; } @@ -1743,11 +3191,7 @@ choose_good_entry_server(uint8_t purpose, cpath_build_state_t *state) if (options->_AllowInvalid & ALLOW_INVALID_ENTRY) flags |= CRN_ALLOW_INVALID; - choice = router_choose_random_node( - NULL, - excluded, - options->ExcludeNodes, - flags); + choice = router_choose_random_node(excluded, options->ExcludeNodes, flags); smartlist_free(excluded); return choice; } @@ -1868,9 +3312,9 @@ extend_info_from_router(routerinfo_t *r) void extend_info_free(extend_info_t *info) { - tor_assert(info); - if (info->onion_key) - crypto_free_pk_env(info->onion_key); + if (!info) + return; + crypto_free_pk_env(info->onion_key); tor_free(info); } @@ -1929,8 +3373,6 @@ entry_guard_set_status(entry_guard_t *e, routerinfo_t *ri, char buf[HEX_DIGEST_LEN+1]; int changed = 0; - tor_assert(options); - *reason = NULL; /* Do we want to mark this guard as bad? */ @@ -1993,35 +3435,58 @@ entry_is_time_to_retry(entry_guard_t *e, time_t now) * - Listed as either up or never yet contacted; * - Present in the routerlist; * - Listed as 'stable' or 'fast' by the current dirserver consensus, - * if demanded by <b>need_uptime</b> or <b>need_capacity</b>; - * (This check is currently redundant with the Guard flag, but in - * the future that might change. Best to leave it in for now.) + * if demanded by <b>need_uptime</b> or <b>need_capacity</b> + * (unless it's a configured EntryNode); * - Allowed by our current ReachableORAddresses config option; and - * - Currently thought to be reachable by us (unless assume_reachable + * - Currently thought to be reachable by us (unless <b>assume_reachable</b> * is true). + * + * If the answer is no, set *<b>msg</b> to an explanation of why. */ static INLINE routerinfo_t * entry_is_live(entry_guard_t *e, int need_uptime, int need_capacity, - int assume_reachable) + int assume_reachable, const char **msg) { routerinfo_t *r; - if (e->bad_since) + or_options_t *options = get_options(); + tor_assert(msg); + + if (e->bad_since) { + *msg = "bad"; return NULL; + } /* no good if it's unreachable, unless assume_unreachable or can_retry. */ if (!assume_reachable && !e->can_retry && - e->unreachable_since && !entry_is_time_to_retry(e, time(NULL))) + e->unreachable_since && !entry_is_time_to_retry(e, time(NULL))) { + *msg = "unreachable"; return NULL; + } r = router_get_by_digest(e->identity); - if (!r) + if (!r) { + *msg = "no descriptor"; return NULL; - if (get_options()->UseBridges && r->purpose != ROUTER_PURPOSE_BRIDGE) + } + if (get_options()->UseBridges && r->purpose != ROUTER_PURPOSE_BRIDGE) { + *msg = "not a bridge"; return NULL; - if (!get_options()->UseBridges && r->purpose != ROUTER_PURPOSE_GENERAL) + } + if (!get_options()->UseBridges && r->purpose != ROUTER_PURPOSE_GENERAL) { + *msg = "not general-purpose"; return NULL; - if (router_is_unreliable(r, need_uptime, need_capacity, 0)) + } + if (options->EntryNodes && + routerset_contains_router(options->EntryNodes, r)) { + /* they asked for it, they get it */ + need_uptime = need_capacity = 0; + } + if (router_is_unreliable(r, need_uptime, need_capacity, 0)) { + *msg = "not fast/stable"; return NULL; - if (!fascist_firewall_allows_or(r)) + } + if (!fascist_firewall_allows_or(r)) { + *msg = "unreachable by config"; return NULL; + } return r; } @@ -2030,11 +3495,12 @@ static int num_live_entry_guards(void) { int n = 0; + const char *msg; if (! entry_guards) return 0; SMARTLIST_FOREACH(entry_guards, entry_guard_t *, entry, { - if (entry_is_live(entry, 0, 1, 0)) + if (entry_is_live(entry, 0, 1, 0, &msg)) ++n; }); return n; @@ -2058,16 +3524,21 @@ static void log_entry_guards(int severity) { smartlist_t *elements = smartlist_create(); - char buf[1024]; char *s; SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, { - tor_snprintf(buf, sizeof(buf), "%s (%s%s)", - e->nickname, - entry_is_live(e, 0, 1, 0) ? "up " : "down ", - e->made_contact ? "made-contact" : "never-contacted"); - smartlist_add(elements, tor_strdup(buf)); + const char *msg = NULL; + char *cp; + if (entry_is_live(e, 0, 1, 0, &msg)) + tor_asprintf(&cp, "%s (up %s)", + e->nickname, + e->made_contact ? "made-contact" : "never-contacted"); + else + tor_asprintf(&cp, "%s (%s, %s)", + e->nickname, msg, + e->made_contact ? "made-contact" : "never-contacted"); + smartlist_add(elements, cp); }); s = smartlist_join_strings(elements, ",", 0, NULL); @@ -2091,12 +3562,13 @@ control_event_guard_deferred(void) **/ #if 0 int n = 0; + const char *msg; or_options_t *options = get_options(); if (!entry_guards) return; SMARTLIST_FOREACH(entry_guards, entry_guard_t *, entry, { - if (entry_is_live(entry, 0, 1, 0)) { + if (entry_is_live(entry, 0, 1, 0, &msg)) { if (n++ == options->NumEntryGuards) { control_event_guard(entry->nickname, entry->identity, "DEFERRED"); return; @@ -2158,9 +3630,8 @@ add_an_entry_guard(routerinfo_t *chosen, int reset_status) /** If the use of entry guards is configured, choose more entry guards * until we have enough in the list. */ static void -pick_entry_guards(void) +pick_entry_guards(or_options_t *options) { - or_options_t *options = get_options(); int changed = 0; tor_assert(entry_guards); @@ -2182,7 +3653,8 @@ pick_entry_guards(void) static void entry_guard_free(entry_guard_t *e) { - tor_assert(e); + if (!e) + return; tor_free(e->chosen_by_version); tor_free(e); } @@ -2191,10 +3663,9 @@ entry_guard_free(entry_guard_t *e) * or which was selected by a version of Tor that's known to select * entry guards badly. */ static int -remove_obsolete_entry_guards(void) +remove_obsolete_entry_guards(time_t now) { int changed = 0, i; - time_t now = time(NULL); for (i = 0; i < smartlist_len(entry_guards); ++i) { entry_guard_t *entry = smartlist_get(entry_guards, i); @@ -2254,11 +3725,10 @@ remove_obsolete_entry_guards(void) * long that we don't think they'll come up again. Return 1 if we * removed any, or 0 if we did nothing. */ static int -remove_dead_entry_guards(void) +remove_dead_entry_guards(time_t now) { char dbuf[HEX_DIGEST_LEN+1]; char tbuf[ISO_TIME_LEN+1]; - time_t now = time(NULL); int i; int changed = 0; @@ -2293,19 +3763,17 @@ remove_dead_entry_guards(void) * think that things are unlisted. */ void -entry_guards_compute_status(void) +entry_guards_compute_status(or_options_t *options, time_t now) { - time_t now; int changed = 0; int severity = LOG_DEBUG; - or_options_t *options; digestmap_t *reasons; + if (! entry_guards) return; - options = get_options(); - - now = time(NULL); + if (options->EntryNodes) /* reshuffle the entry guard list if needed */ + entry_nodes_should_be_added(); reasons = digestmap_new(); SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, entry) @@ -2322,7 +3790,7 @@ entry_guards_compute_status(void) } SMARTLIST_FOREACH_END(entry); - if (remove_dead_entry_guards()) + if (remove_dead_entry_guards(now)) changed = 1; severity = changed ? LOG_DEBUG : LOG_INFO; @@ -2330,13 +3798,16 @@ entry_guards_compute_status(void) if (changed) { SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, entry) { const char *reason = digestmap_get(reasons, entry->identity); - log_info(LD_CIRC, "Summary: Entry '%s' is %s, %s%s%s, and %s.", + const char *live_msg = ""; + routerinfo_t *r = entry_is_live(entry, 0, 1, 0, &live_msg); + log_info(LD_CIRC, "Summary: Entry '%s' is %s, %s%s%s, and %s%s.", entry->nickname, entry->unreachable_since ? "unreachable" : "reachable", entry->bad_since ? "unusable" : "usable", reason ? ", ": "", reason ? reason : "", - entry_is_live(entry, 0, 1, 0) ? "live" : "not live"); + r ? "live" : "not live / ", + r ? "" : live_msg); } SMARTLIST_FOREACH_END(entry); log_info(LD_CIRC, " (%d/%d entry guards are usable/new)", num_live_entry_guards(), smartlist_len(entry_guards)); @@ -2355,7 +3826,7 @@ entry_guards_compute_status(void) * If <b>mark_relay_status</b>, also call router_set_status() on this * relay. * - * XXX022 change succeeded and mark_relay_status into 'int flags'. + * XXX023 change succeeded and mark_relay_status into 'int flags'. */ int entry_guard_register_connect_status(const char *digest, int succeeded, @@ -2407,6 +3878,7 @@ entry_guard_register_connect_status(const char *digest, int succeeded, "Removing from the list. %d/%d entry guards usable/new.", entry->nickname, buf, num_live_entry_guards()-1, smartlist_len(entry_guards)-1); + control_event_guard(entry->nickname, entry->identity, "DROPPED"); entry_guard_free(entry); smartlist_del_keeporder(entry_guards, idx); log_entry_guards(LOG_INFO); @@ -2443,7 +3915,8 @@ entry_guard_register_connect_status(const char *digest, int succeeded, if (e == entry) break; if (e->made_contact) { - routerinfo_t *r = entry_is_live(e, 0, 1, 1); + const char *msg; + routerinfo_t *r = entry_is_live(e, 0, 1, 1, &msg); if (r && e->unreachable_since) { refuse_conn = 1; e->can_retry = 1; @@ -2474,16 +3947,16 @@ static int should_add_entry_nodes = 0; void entry_nodes_should_be_added(void) { - log_info(LD_CIRC, "New EntryNodes config option detected. Will use."); + log_info(LD_CIRC, "EntryNodes config option set. Putting configured " + "relays at the front of the entry guard list."); should_add_entry_nodes = 1; } /** Add all nodes in EntryNodes that aren't currently guard nodes to the list * of guard nodes, at the front. */ static void -entry_guards_prepend_from_config(void) +entry_guards_prepend_from_config(or_options_t *options) { - or_options_t *options = get_options(); smartlist_t *entry_routers, *entry_fps; smartlist_t *old_entry_guards_on_list, *old_entry_guards_not_on_list; tor_assert(entry_guards); @@ -2498,7 +3971,7 @@ entry_guards_prepend_from_config(void) return; } - if (options->EntryNodes) { + { char *string = routerset_to_string(options->EntryNodes); log_info(LD_CIRC,"Adding configured EntryNodes '%s'.", string); tor_free(string); @@ -2511,13 +3984,14 @@ entry_guards_prepend_from_config(void) /* Split entry guards into those on the list and those not. */ - /* XXXX022 Now that we allow countries and IP ranges in EntryNodes, this is + /* XXXX023 Now that we allow countries and IP ranges in EntryNodes, this is * potentially an enormous list. For now, we disable such values for * EntryNodes in options_validate(); really, this wants a better solution. * Perhaps we should do this calculation once whenever the list of routers * changes or the entrynodes setting changes. */ - routerset_get_all_routers(entry_routers, options->EntryNodes, 0); + routerset_get_all_routers(entry_routers, options->EntryNodes, + options->ExcludeNodes, 0); SMARTLIST_FOREACH(entry_routers, routerinfo_t *, ri, smartlist_add(entry_fps,ri->cache_info.identity_digest)); SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, { @@ -2542,13 +4016,10 @@ entry_guards_prepend_from_config(void) SMARTLIST_FOREACH(entry_routers, routerinfo_t *, ri, { add_an_entry_guard(ri, 0); }); - /* Finally, the remaining EntryNodes, unless we're strict */ - if (options->StrictEntryNodes) { - SMARTLIST_FOREACH(old_entry_guards_not_on_list, entry_guard_t *, e, - entry_guard_free(e)); - } else { - smartlist_add_all(entry_guards, old_entry_guards_not_on_list); - } + /* Finally, free the remaining previously configured guards that are not in + * EntryNodes. */ + SMARTLIST_FOREACH(old_entry_guards_not_on_list, entry_guard_t *, e, + entry_guard_free(e)); smartlist_free(entry_routers); smartlist_free(entry_fps); @@ -2557,16 +4028,18 @@ entry_guards_prepend_from_config(void) entry_guards_changed(); } -/** Return 1 if we're fine adding arbitrary routers out of the - * directory to our entry guard list. Else return 0. */ +/** Return 0 if we're fine adding arbitrary routers out of the + * directory to our entry guard list, or return 1 if we have a + * list already and we must stick to it. + */ int -entry_list_can_grow(or_options_t *options) +entry_list_is_constrained(or_options_t *options) { - if (options->StrictEntryNodes) - return 0; + if (options->EntryNodes) + return 1; if (options->UseBridges) - return 0; - return 1; + return 1; + return 0; } /** Pick a live (up and listed) entry guard from entry_guards. If @@ -2584,10 +4057,9 @@ choose_random_entry(cpath_build_state_t *state) routerinfo_t *r = NULL; int need_uptime = state ? state->need_uptime : 0; int need_capacity = state ? state->need_capacity : 0; - int consider_exit_family = 0; + int preferred_min, consider_exit_family = 0; if (chosen_exit) { - smartlist_add(exit_family, chosen_exit); routerlist_add_family(exit_family, chosen_exit); consider_exit_family = 1; } @@ -2596,38 +4068,66 @@ choose_random_entry(cpath_build_state_t *state) entry_guards = smartlist_create(); if (should_add_entry_nodes) - entry_guards_prepend_from_config(); + entry_guards_prepend_from_config(options); - if (entry_list_can_grow(options) && - (! entry_guards || - smartlist_len(entry_guards) < options->NumEntryGuards)) - pick_entry_guards(); + if (!entry_list_is_constrained(options) && + smartlist_len(entry_guards) < options->NumEntryGuards) + pick_entry_guards(options); retry: smartlist_clear(live_entry_guards); SMARTLIST_FOREACH(entry_guards, entry_guard_t *, entry, { - r = entry_is_live(entry, need_uptime, need_capacity, 0); - if (r && (!consider_exit_family || !smartlist_isin(exit_family, r))) { - smartlist_add(live_entry_guards, r); - if (!entry->made_contact) { - /* Always start with the first not-yet-contacted entry - * guard. Otherwise we might add several new ones, pick - * the second new one, and now we've expanded our entry - * guard list without needing to. */ - goto choose_and_finish; + const char *msg; + r = entry_is_live(entry, need_uptime, need_capacity, 0, &msg); + if (!r) + continue; /* down, no point */ + if (r == chosen_exit) + continue; /* don't pick the same node for entry and exit */ + if (consider_exit_family && smartlist_isin(exit_family, r)) + continue; /* avoid relays that are family members of our exit */ +#if 0 /* since EntryNodes is always strict now, this clause is moot */ + if (options->EntryNodes && + !routerset_contains_router(options->EntryNodes, r)) { + /* We've come to the end of our preferred entry nodes. */ + if (smartlist_len(live_entry_guards)) + goto choose_and_finish; /* only choose from the ones we like */ + if (options->StrictNodes) { + /* in theory this case should never happen, since + * entry_guards_prepend_from_config() drops unwanted relays */ + tor_fragile_assert(); + } else { + log_info(LD_CIRC, + "No relays from EntryNodes available. Using others."); } - if (smartlist_len(live_entry_guards) >= options->NumEntryGuards) - break; /* we have enough */ } +#endif + smartlist_add(live_entry_guards, r); + if (!entry->made_contact) { + /* Always start with the first not-yet-contacted entry + * guard. Otherwise we might add several new ones, pick + * the second new one, and now we've expanded our entry + * guard list without needing to. */ + goto choose_and_finish; + } + if (smartlist_len(live_entry_guards) >= options->NumEntryGuards) + break; /* we have enough */ }); - /* Try to have at least 2 choices available. This way we don't - * get stuck with a single live-but-crummy entry and just keep - * using him. - * (We might get 2 live-but-crummy entry guards, but so be it.) */ - if (smartlist_len(live_entry_guards) < 2) { - if (entry_list_can_grow(options)) { + if (entry_list_is_constrained(options)) { + /* If we prefer the entry nodes we've got, and we have at least + * one choice, that's great. Use it. */ + preferred_min = 1; + } else { + /* Try to have at least 2 choices available. This way we don't + * get stuck with a single live-but-crummy entry and just keep + * using him. + * (We might get 2 live-but-crummy entry guards, but so be it.) */ + preferred_min = 2; + } + + if (smartlist_len(live_entry_guards) < preferred_min) { + if (!entry_list_is_constrained(options)) { /* still no? try adding a new entry then */ /* XXX if guard doesn't imply fast and stable, then we need * to tell add_an_entry_guard below what we want, or it might @@ -2652,26 +4152,31 @@ choose_random_entry(cpath_build_state_t *state) need_capacity = 0; goto retry; } - if (!r && !entry_list_can_grow(options) && consider_exit_family) { - /* still no? if we're using bridges or have strictentrynodes - * set, and our chosen exit is in the same family as all our - * bridges/entry guards, then be flexible about families. */ +#if 0 + /* Removing this retry logic: if we only allow one exit, and it is in the + same family as all our entries, then we are just plain not going to win + here. */ + if (!r && entry_list_is_constrained(options) && consider_exit_family) { + /* still no? if we're using bridges, + * and our chosen exit is in the same family as all our + * bridges, then be flexible about families. */ consider_exit_family = 0; goto retry; } +#endif /* live_entry_guards may be empty below. Oh well, we tried. */ } choose_and_finish: - if (entry_list_can_grow(options)) { + if (entry_list_is_constrained(options)) { + /* We need to weight by bandwidth, because our bridges or entryguards + * were not already selected proportional to their bandwidth. */ + r = routerlist_sl_choose_by_bandwidth(live_entry_guards, WEIGHT_FOR_GUARD); + } else { /* We choose uniformly at random here, because choose_good_entry_server() * already weights its choices by bandwidth, so we don't want to * *double*-weight our guard selection. */ r = smartlist_choose(live_entry_guards); - } else { - /* We need to weight by bandwidth, because our bridges or entryguards - * were not already selected proportional to their bandwidth. */ - r = routerlist_sl_choose_by_bandwidth(live_entry_guards, WEIGHT_FOR_GUARD); } smartlist_free(live_entry_guards); smartlist_free(exit_family); @@ -2803,9 +4308,9 @@ entry_guards_parse_state(or_state_t *state, int set, char **msg) } entry_guards = new_entry_guards; entry_guards_dirty = 0; - /* XXX022 hand new_entry_guards to this func, and move it up a + /* XXX023 hand new_entry_guards to this func, and move it up a * few lines, so we don't have to re-dirty it */ - if (remove_obsolete_entry_guards()) + if (remove_obsolete_entry_guards(now)) entry_guards_dirty = 1; } digestmap_free(added_by, _tor_free); @@ -2904,9 +4409,11 @@ entry_guards_update_state(or_state_t *state) * */ int getinfo_helper_entry_guards(control_connection_t *conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { - int use_long_names = conn->use_long_names; + (void) conn; + (void) errmsg; if (!strcmp(question,"entry-guards") || !strcmp(question,"helper-nodes")) { @@ -2915,12 +4422,13 @@ getinfo_helper_entry_guards(control_connection_t *conn, char nbuf[MAX_VERBOSE_NICKNAME_LEN+1]; if (!entry_guards) entry_guards = smartlist_create(); - SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, - { + SMARTLIST_FOREACH_BEGIN(entry_guards, entry_guard_t *, e) { size_t len = MAX_VERBOSE_NICKNAME_LEN+ISO_TIME_LEN+32; char *c = tor_malloc(len); const char *status = NULL; time_t when = 0; + routerinfo_t *ri; + if (!e->made_contact) { status = "never-connected"; } else if (e->bad_since) { @@ -2929,19 +4437,17 @@ getinfo_helper_entry_guards(control_connection_t *conn, } else { status = "up"; } - if (use_long_names) { - routerinfo_t *ri = router_get_by_digest(e->identity); - if (ri) { - router_get_verbose_nickname(nbuf, ri); - } else { - nbuf[0] = '$'; - base16_encode(nbuf+1, sizeof(nbuf)-1, e->identity, DIGEST_LEN); - /* e->nickname field is not very reliable if we don't know about - * this router any longer; don't include it. */ - } + + ri = router_get_by_digest(e->identity); + if (ri) { + router_get_verbose_nickname(nbuf, ri); } else { - base16_encode(nbuf, sizeof(nbuf), e->identity, DIGEST_LEN); + nbuf[0] = '$'; + base16_encode(nbuf+1, sizeof(nbuf)-1, e->identity, DIGEST_LEN); + /* e->nickname field is not very reliable if we don't know about + * this router any longer; don't include it. */ } + if (when) { format_iso_time(tbuf, when); tor_snprintf(c, len, "%s %s %s\n", nbuf, status, tbuf); @@ -2949,7 +4455,7 @@ getinfo_helper_entry_guards(control_connection_t *conn, tor_snprintf(c, len, "%s %s\n", nbuf, status); } smartlist_add(sl, c); - }); + } SMARTLIST_FOREACH_END(e); *answer = smartlist_join_strings(sl, "", 0, NULL); SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); smartlist_free(sl); @@ -2990,29 +4496,56 @@ clear_bridge_list(void) * (either by comparing keys if possible, else by comparing addr/port). * Else return NULL. */ static bridge_info_t * -routerinfo_get_configured_bridge(routerinfo_t *ri) +get_configured_bridge_by_addr_port_digest(tor_addr_t *addr, uint16_t port, + const char *digest) { if (!bridge_list) return NULL; SMARTLIST_FOREACH_BEGIN(bridge_list, bridge_info_t *, bridge) { if (tor_digest_is_zero(bridge->identity) && - tor_addr_eq_ipv4h(&bridge->addr, ri->addr) && - bridge->port == ri->or_port) + !tor_addr_compare(&bridge->addr, addr, CMP_EXACT) && + bridge->port == port) return bridge; - if (!memcmp(bridge->identity, ri->cache_info.identity_digest, - DIGEST_LEN)) + if (!memcmp(bridge->identity, digest, DIGEST_LEN)) return bridge; } SMARTLIST_FOREACH_END(bridge); return NULL; } +/** Wrapper around get_configured_bridge_by_addr_port_digest() to look + * it up via router descriptor <b>ri</b>. */ +static bridge_info_t * +get_configured_bridge_by_routerinfo(routerinfo_t *ri) +{ + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, ri->addr); + return get_configured_bridge_by_addr_port_digest(&addr, + ri->or_port, ri->cache_info.identity_digest); +} + /** Return 1 if <b>ri</b> is one of our known bridges, else 0. */ int routerinfo_is_a_configured_bridge(routerinfo_t *ri) { - return routerinfo_get_configured_bridge(ri) ? 1 : 0; + return get_configured_bridge_by_routerinfo(ri) ? 1 : 0; +} + +/** We made a connection to a router at <b>addr</b>:<b>port</b> + * without knowing its digest. Its digest turned out to be <b>digest</b>. + * If it was a bridge, and we still don't know its digest, record it. + */ +void +learned_router_identity(tor_addr_t *addr, uint16_t port, const char *digest) +{ + bridge_info_t *bridge = + get_configured_bridge_by_addr_port_digest(addr, port, digest); + if (bridge && tor_digest_is_zero(bridge->identity)) { + memcpy(bridge->identity, digest, DIGEST_LEN); + log_notice(LD_DIR, "Learned fingerprint %s for bridge %s:%d", + hex_str(digest, DIGEST_LEN), fmt_addr(addr), port); + } } /** Remember a new bridge at <b>addr</b>:<b>port</b>. If <b>digest</b> @@ -3031,6 +4564,24 @@ bridge_add_from_config(const tor_addr_t *addr, uint16_t port, char *digest) smartlist_add(bridge_list, b); } +/** Return true iff <b>routerset</b> contains the bridge <b>bridge</b>. */ +static int +routerset_contains_bridge(const routerset_t *routerset, + const bridge_info_t *bridge) +{ + int result; + extend_info_t *extinfo; + tor_assert(bridge); + if (!routerset) + return 0; + + extinfo = extend_info_alloc( + NULL, bridge->identity, NULL, &bridge->addr, bridge->port); + result = routerset_contains_extendinfo(routerset, extinfo); + extend_info_free(extinfo); + return result; +} + /** If <b>digest</b> is one of our known bridges, return it. */ static bridge_info_t * find_bridge_by_digest(const char *digest) @@ -3043,12 +4594,12 @@ find_bridge_by_digest(const char *digest) return NULL; } -/** We need to ask <b>bridge</b> for its server descriptor. <b>address</b> - * is a helpful string describing this bridge. */ +/** We need to ask <b>bridge</b> for its server descriptor. */ static void launch_direct_bridge_descriptor_fetch(bridge_info_t *bridge) { char *address; + or_options_t *options = get_options(); if (connection_get_by_type_addr_port_purpose( CONN_TYPE_DIR, &bridge->addr, bridge->port, @@ -3056,6 +4607,13 @@ launch_direct_bridge_descriptor_fetch(bridge_info_t *bridge) return; /* it's already on the way */ address = tor_dup_addr(&bridge->addr); + if (routerset_contains_bridge(options->ExcludeNodes, bridge)) { + download_status_mark_impossible(&bridge->fetch_status); + log_warn(LD_APP, "Not using bridge at %s: it is in ExcludeNodes.", + safe_str_client(fmt_addr(&bridge->addr))); + return; + } + directory_initiate_command(address, &bridge->addr, bridge->port, 0, 0, /* does not matter */ @@ -3082,9 +4640,8 @@ retry_bridge_descriptor_fetch_directly(const char *digest) * descriptor, fetch a new copy of its descriptor -- either directly * from the bridge or via a bridge authority. */ void -fetch_bridge_descriptors(time_t now) +fetch_bridge_descriptors(or_options_t *options, time_t now) { - or_options_t *options = get_options(); int num_bridge_auths = get_n_authorities(BRIDGE_AUTHORITY); int ask_bridge_directly; int can_use_bridge_authority; @@ -3097,6 +4654,12 @@ fetch_bridge_descriptors(time_t now) if (!download_status_is_ready(&bridge->fetch_status, now, IMPOSSIBLE_TO_DOWNLOAD)) continue; /* don't bother, no need to retry yet */ + if (routerset_contains_bridge(options->ExcludeNodes, bridge)) { + download_status_mark_impossible(&bridge->fetch_status); + log_warn(LD_APP, "Not using bridge at %s: it is in ExcludeNodes.", + safe_str_client(fmt_addr(&bridge->addr))); + continue; + } /* schedule another fetch as if this one will fail, in case it does */ download_status_failed(&bridge->fetch_status, 0); @@ -3143,6 +4706,29 @@ fetch_bridge_descriptors(time_t now) SMARTLIST_FOREACH_END(bridge); } +/** If our <b>bridge</b> is configured to be a different address than + * the bridge gives in its routerinfo <b>ri</b>, rewrite the routerinfo + * we received to use the address we meant to use. Now we handle + * multihomed bridges better. + */ +static void +rewrite_routerinfo_address_for_bridge(bridge_info_t *bridge, routerinfo_t *ri) +{ + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, ri->addr); + + if (!tor_addr_compare(&bridge->addr, &addr, CMP_EXACT) && + bridge->port == ri->or_port) + return; /* they match, so no need to do anything */ + + ri->addr = tor_addr_to_ipv4h(&bridge->addr); + tor_free(ri->address); + ri->address = tor_dup_ip(ri->addr); + ri->or_port = bridge->port; + log_info(LD_DIR, "Adjusted bridge '%s' to match configured address %s:%d.", + ri->nickname, ri->address, ri->or_port); +} + /** We just learned a descriptor for a bridge. See if that * digest is in our entry guard list, and add it if not. */ void @@ -3152,7 +4738,7 @@ learned_bridge_descriptor(routerinfo_t *ri, int from_cache) tor_assert(ri->purpose == ROUTER_PURPOSE_BRIDGE); if (get_options()->UseBridges) { int first = !any_bridge_descriptors_known(); - bridge_info_t *bridge = routerinfo_get_configured_bridge(ri); + bridge_info_t *bridge = get_configured_bridge_by_routerinfo(ri); time_t now = time(NULL); ri->is_running = 1; @@ -3161,6 +4747,8 @@ learned_bridge_descriptor(routerinfo_t *ri, int from_cache) if (!from_cache) download_status_reset(&bridge->fetch_status); + rewrite_routerinfo_address_for_bridge(bridge, ri); + add_an_entry_guard(ri, 1); log_notice(LD_DIR, "new bridge descriptor '%s' (%s)", ri->nickname, from_cache ? "cached" : "fresh"); @@ -3209,26 +4797,38 @@ any_pending_bridge_descriptor_fetches(void) return 0; } -/** Return 1 if we have at least one descriptor for a bridge and - * all descriptors we know are down. Else return 0. If <b>act</b> is - * 1, then mark the down bridges up; else just observe and report. */ +/** Return 1 if we have at least one descriptor for an entry guard + * (bridge or member of EntryNodes) and all descriptors we know are + * down. Else return 0. If <b>act</b> is 1, then mark the down guards + * up; else just observe and report. */ static int -bridges_retry_helper(int act) +entries_retry_helper(or_options_t *options, int act) { routerinfo_t *ri; int any_known = 0; int any_running = 0; + int purpose = options->UseBridges ? + ROUTER_PURPOSE_BRIDGE : ROUTER_PURPOSE_GENERAL; if (!entry_guards) entry_guards = smartlist_create(); SMARTLIST_FOREACH(entry_guards, entry_guard_t *, e, { ri = router_get_by_digest(e->identity); - if (ri && ri->purpose == ROUTER_PURPOSE_BRIDGE) { + if (ri && ri->purpose == purpose) { any_known = 1; if (ri->is_running) - any_running = 1; /* some bridge is both known and running */ - else if (act) { /* mark it for retry */ - ri->is_running = 1; + any_running = 1; /* some entry is both known and running */ + else if (act) { + /* Mark all current connections to this OR as unhealthy, since + * otherwise there could be one that started 30 seconds + * ago, and in 30 seconds it will time out, causing us to mark + * the node down and undermine the retry attempt. We mark even + * the established conns, since if the network just came back + * we'll want to attach circuits to fresh conns. */ + connection_or_set_bad_connections(ri->cache_info.identity_digest, 1); + + /* mark this entry node for retry */ + router_set_status(ri->cache_info.identity_digest, 1); e->can_retry = 1; e->bad_since = 0; } @@ -3239,19 +4839,21 @@ bridges_retry_helper(int act) return any_known && !any_running; } -/** Do we know any descriptors for our bridges, and are they all - * down? */ +/** Do we know any descriptors for our bridges / entrynodes, and are + * all the ones we have descriptors for down? */ int -bridges_known_but_down(void) +entries_known_but_down(or_options_t *options) { - return bridges_retry_helper(0); + tor_assert(entry_list_is_constrained(options)); + return entries_retry_helper(options, 0); } -/** Mark all down known bridges up. */ +/** Mark all down known bridges / entrynodes up. */ void -bridges_retry_all(void) +entries_retry_all(or_options_t *options) { - bridges_retry_helper(1); + tor_assert(entry_list_is_constrained(options)); + entries_retry_helper(options, 1); } /** Release all storage held by the list of entry guards and related diff --git a/src/or/circuitbuild.h b/src/or/circuitbuild.h new file mode 100644 index 000000000..af2493187 --- /dev/null +++ b/src/or/circuitbuild.h @@ -0,0 +1,127 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file circuitbuild.h + * \brief Header file for circuitbuild.c. + **/ + +#ifndef _TOR_CIRCUITBUILD_H +#define _TOR_CIRCUITBUILD_H + +char *circuit_list_path(origin_circuit_t *circ, int verbose); +char *circuit_list_path_for_controller(origin_circuit_t *circ); +void circuit_log_path(int severity, unsigned int domain, + origin_circuit_t *circ); +void circuit_rep_hist_note_result(origin_circuit_t *circ); +origin_circuit_t *origin_circuit_init(uint8_t purpose, int flags); +origin_circuit_t *circuit_establish_circuit(uint8_t purpose, + extend_info_t *exit, + int flags); +int circuit_handle_first_hop(origin_circuit_t *circ); +void circuit_n_conn_done(or_connection_t *or_conn, int status); +int inform_testing_reachability(void); +int circuit_timeout_want_to_count_circ(origin_circuit_t *circ); +int circuit_send_next_onion_skin(origin_circuit_t *circ); +void circuit_note_clock_jumped(int seconds_elapsed); +int circuit_extend(cell_t *cell, circuit_t *circ); +int circuit_init_cpath_crypto(crypt_path_t *cpath, const char *key_data, + int reverse); +int circuit_finish_handshake(origin_circuit_t *circ, uint8_t cell_type, + const uint8_t *reply); +int circuit_truncated(origin_circuit_t *circ, crypt_path_t *layer); +int onionskin_answer(or_circuit_t *circ, uint8_t cell_type, + const char *payload, const char *keys); +int circuit_all_predicted_ports_handled(time_t now, int *need_uptime, + int *need_capacity); + +int circuit_append_new_exit(origin_circuit_t *circ, extend_info_t *info); +int circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *info); +void onion_append_to_cpath(crypt_path_t **head_ptr, crypt_path_t *new_hop); +extend_info_t *extend_info_alloc(const char *nickname, const char *digest, + crypto_pk_env_t *onion_key, + const tor_addr_t *addr, uint16_t port); +extend_info_t *extend_info_from_router(routerinfo_t *r); +extend_info_t *extend_info_dup(extend_info_t *info); +void extend_info_free(extend_info_t *info); +routerinfo_t *build_state_get_exit_router(cpath_build_state_t *state); +const char *build_state_get_exit_nickname(cpath_build_state_t *state); + +void entry_guards_compute_status(or_options_t *options, time_t now); +int entry_guard_register_connect_status(const char *digest, int succeeded, + int mark_relay_status, time_t now); +void entry_nodes_should_be_added(void); +int entry_list_is_constrained(or_options_t *options); +routerinfo_t *choose_random_entry(cpath_build_state_t *state); +int entry_guards_parse_state(or_state_t *state, int set, char **msg); +void entry_guards_update_state(or_state_t *state); +int getinfo_helper_entry_guards(control_connection_t *conn, + const char *question, char **answer, + const char **errmsg); + +void clear_bridge_list(void); +int routerinfo_is_a_configured_bridge(routerinfo_t *ri); +void +learned_router_identity(tor_addr_t *addr, uint16_t port, const char *digest); +void bridge_add_from_config(const tor_addr_t *addr, uint16_t port, + char *digest); +void retry_bridge_descriptor_fetch_directly(const char *digest); +void fetch_bridge_descriptors(or_options_t *options, time_t now); +void learned_bridge_descriptor(routerinfo_t *ri, int from_cache); +int any_bridge_descriptors_known(void); +int any_pending_bridge_descriptor_fetches(void); +int entries_known_but_down(or_options_t *options); +void entries_retry_all(or_options_t *options); + +void entry_guards_free_all(void); + +extern circuit_build_times_t circ_times; +int circuit_build_times_enough_to_compute(circuit_build_times_t *cbt); +void circuit_build_times_update_state(circuit_build_times_t *cbt, + or_state_t *state); +int circuit_build_times_parse_state(circuit_build_times_t *cbt, + or_state_t *state); +void circuit_build_times_count_timeout(circuit_build_times_t *cbt, + int did_onehop); +int circuit_build_times_count_close(circuit_build_times_t *cbt, + int did_onehop, time_t start_time); +void circuit_build_times_set_timeout(circuit_build_times_t *cbt); +int circuit_build_times_add_time(circuit_build_times_t *cbt, + build_time_t time); +int circuit_build_times_needs_circuits(circuit_build_times_t *cbt); + +int circuit_build_times_needs_circuits_now(circuit_build_times_t *cbt); +void circuit_build_times_init(circuit_build_times_t *cbt); +void circuit_build_times_new_consensus_params(circuit_build_times_t *cbt, + networkstatus_t *ns); +double circuit_build_times_timeout_rate(const circuit_build_times_t *cbt); +double circuit_build_times_close_rate(const circuit_build_times_t *cbt); + +#ifdef CIRCUIT_PRIVATE +double circuit_build_times_calculate_timeout(circuit_build_times_t *cbt, + double quantile); +build_time_t circuit_build_times_generate_sample(circuit_build_times_t *cbt, + double q_lo, double q_hi); +void circuit_build_times_initial_alpha(circuit_build_times_t *cbt, + double quantile, double time_ms); +int circuit_build_times_update_alpha(circuit_build_times_t *cbt); +double circuit_build_times_cdf(circuit_build_times_t *cbt, double x); +void circuitbuild_running_unit_tests(void); +void circuit_build_times_reset(circuit_build_times_t *cbt); + +/* Network liveness functions */ +int circuit_build_times_network_check_changed(circuit_build_times_t *cbt); +#endif + +/* Network liveness functions */ +void circuit_build_times_network_is_live(circuit_build_times_t *cbt); +int circuit_build_times_network_check_live(circuit_build_times_t *cbt); +void circuit_build_times_network_circ_success(circuit_build_times_t *cbt); + +int circuit_build_times_get_bw_scale(networkstatus_t *ns); + +#endif + diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index 524856875..33dc8f09a 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -10,6 +10,21 @@ **/ #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "connection.h" +#include "config.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "networkstatus.h" +#include "onion.h" +#include "relay.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rephist.h" +#include "routerlist.h" #include "ht.h" /********* START VARIABLES **********/ @@ -352,6 +367,8 @@ circuit_purpose_to_controller_string(uint8_t purpose) case CIRCUIT_PURPOSE_TESTING: return "TESTING"; + case CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT: + return "MEASURE_TIMEOUT"; case CIRCUIT_PURPOSE_CONTROLLER: return "CONTROLLER"; @@ -361,13 +378,71 @@ circuit_purpose_to_controller_string(uint8_t purpose) } } +/** Return a human-readable string for the circuit purpose <b>purpose</b>. */ +const char * +circuit_purpose_to_string(uint8_t purpose) +{ + static char buf[32]; + + switch (purpose) + { + case CIRCUIT_PURPOSE_OR: + return "Circuit at relay"; + case CIRCUIT_PURPOSE_INTRO_POINT: + return "Acting as intro point"; + case CIRCUIT_PURPOSE_REND_POINT_WAITING: + return "Acting as rendevous (pending)"; + case CIRCUIT_PURPOSE_REND_ESTABLISHED: + return "Acting as rendevous (established)"; + case CIRCUIT_PURPOSE_C_GENERAL: + return "General-purpose client"; + case CIRCUIT_PURPOSE_C_INTRODUCING: + return "Hidden service client: Connecting to intro point"; + case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: + return "Hidden service client: Waiting for ack from intro point"; + case CIRCUIT_PURPOSE_C_INTRODUCE_ACKED: + return "Hidden service client: Received ack from intro point"; + case CIRCUIT_PURPOSE_C_ESTABLISH_REND: + return "Hidden service client: Establishing rendezvous point"; + case CIRCUIT_PURPOSE_C_REND_READY: + return "Hidden service client: Pending rendezvous point"; + case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: + return "Hidden service client: Pending rendezvous point (ack received)"; + case CIRCUIT_PURPOSE_C_REND_JOINED: + return "Hidden service client: Active rendezvous point"; + case CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT: + return "Measuring circuit timeout"; + + case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: + return "Hidden service: Establishing introduction point"; + case CIRCUIT_PURPOSE_S_INTRO: + return "Hidden service: Introduction point"; + case CIRCUIT_PURPOSE_S_CONNECT_REND: + return "Hidden service: Connecting to rendezvous point"; + case CIRCUIT_PURPOSE_S_REND_JOINED: + return "Hidden service: Active rendezvous point"; + + case CIRCUIT_PURPOSE_TESTING: + return "Testing circuit"; + + case CIRCUIT_PURPOSE_CONTROLLER: + return "Circuit made by controller"; + + default: + tor_snprintf(buf, sizeof(buf), "UNKNOWN_%d", (int)purpose); + return buf; + } +} + /** Pick a reasonable package_window to start out for our circuits. * Originally this was hard-coded at 1000, but now the consensus votes * on the answer. See proposal 168. */ int32_t circuit_initial_package_window(void) { - int32_t num = networkstatus_get_param(NULL, "circwindow", CIRCWINDOW_START); + int32_t num = networkstatus_get_param(NULL, "circwindow", CIRCWINDOW_START, + CIRCWINDOW_START_MIN, + CIRCWINDOW_START_MAX); /* If the consensus tells us a negative number, we'd assert. */ if (num < 0) num = CIRCWINDOW_START; @@ -379,11 +454,17 @@ circuit_initial_package_window(void) static void init_circuit_base(circuit_t *circ) { - circ->timestamp_created = time(NULL); + tor_gettimeofday(&circ->timestamp_created); circ->package_window = circuit_initial_package_window(); circ->deliver_window = CIRCWINDOW_START; + /* Initialize the cell_ewma_t structure */ + circ->n_cell_ewma.last_adjusted_tick = cell_ewma_get_tick(); + circ->n_cell_ewma.cell_count = 0.0; + circ->n_cell_ewma.heap_index = -1; + circ->n_cell_ewma.is_for_p_conn = 0; + circuit_add(circ); } @@ -408,6 +489,8 @@ origin_circuit_new(void) init_circuit_base(TO_CIRCUIT(circ)); + circ_times.last_circ_at = approx_time(); + return circ; } @@ -429,6 +512,16 @@ or_circuit_new(circid_t p_circ_id, or_connection_t *p_conn) init_circuit_base(TO_CIRCUIT(circ)); + /* Initialize the cell_ewma_t structure */ + + /* Initialize the cell counts to 0 */ + circ->p_cell_ewma.cell_count = 0.0; + circ->p_cell_ewma.last_adjusted_tick = cell_ewma_get_tick(); + circ->p_cell_ewma.is_for_p_conn = 1; + + /* It's not in any heap yet. */ + circ->p_cell_ewma.heap_index = -1; + return circ; } @@ -439,39 +532,37 @@ circuit_free(circuit_t *circ) { void *mem; size_t memlen; - tor_assert(circ); + if (!circ) + return; + if (CIRCUIT_IS_ORIGIN(circ)) { origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); mem = ocirc; memlen = sizeof(origin_circuit_t); tor_assert(circ->magic == ORIGIN_CIRCUIT_MAGIC); if (ocirc->build_state) { - if (ocirc->build_state->chosen_exit) extend_info_free(ocirc->build_state->chosen_exit); - if (ocirc->build_state->pending_final_cpath) circuit_free_cpath_node(ocirc->build_state->pending_final_cpath); } tor_free(ocirc->build_state); circuit_free_cpath(ocirc->cpath); - if (ocirc->intro_key) - crypto_free_pk_env(ocirc->intro_key); - if (ocirc->rend_data) - rend_data_free(ocirc->rend_data); + + crypto_free_pk_env(ocirc->intro_key); + rend_data_free(ocirc->rend_data); } else { or_circuit_t *ocirc = TO_OR_CIRCUIT(circ); + /* Remember cell statistics for this circuit before deallocating. */ + if (get_options()->CellStatistics) + rep_hist_buffer_stats_add_circ(circ, time(NULL)); mem = ocirc; memlen = sizeof(or_circuit_t); tor_assert(circ->magic == OR_CIRCUIT_MAGIC); - if (ocirc->p_crypto) - crypto_free_cipher_env(ocirc->p_crypto); - if (ocirc->p_digest) - crypto_free_digest_env(ocirc->p_digest); - if (ocirc->n_crypto) - crypto_free_cipher_env(ocirc->n_crypto); - if (ocirc->n_digest) - crypto_free_digest_env(ocirc->n_digest); + crypto_free_cipher_env(ocirc->p_crypto); + crypto_free_digest_env(ocirc->p_digest); + crypto_free_cipher_env(ocirc->n_crypto); + crypto_free_digest_env(ocirc->n_digest); if (ocirc->rend_splice) { or_circuit_t *other = ocirc->rend_splice; @@ -487,8 +578,7 @@ circuit_free(circuit_t *circ) cell_queue_clear(ô->p_conn_cells); } - if (circ->n_hop) - extend_info_free(circ->n_hop); + extend_info_free(circ->n_hop); tor_free(circ->n_conn_onionskin); /* Remove from map. */ @@ -498,7 +588,7 @@ circuit_free(circuit_t *circ) * "active" checks will be violated. */ cell_queue_clear(&circ->n_conn_cells); - memset(circ, 0xAA, memlen); /* poison memory */ + memset(mem, 0xAA, memlen); /* poison memory */ tor_free(mem); } @@ -541,10 +631,10 @@ circuit_free_all(void) circuit_free(global_circuitlist); global_circuitlist = next; } - if (circuits_pending_or_conns) { - smartlist_free(circuits_pending_or_conns); - circuits_pending_or_conns = NULL; - } + + smartlist_free(circuits_pending_or_conns); + circuits_pending_or_conns = NULL; + HT_CLEAR(orconn_circid_map, &orconn_circid_circuit_map); } @@ -552,18 +642,15 @@ circuit_free_all(void) static void circuit_free_cpath_node(crypt_path_t *victim) { - if (victim->f_crypto) - crypto_free_cipher_env(victim->f_crypto); - if (victim->b_crypto) - crypto_free_cipher_env(victim->b_crypto); - if (victim->f_digest) - crypto_free_digest_env(victim->f_digest); - if (victim->b_digest) - crypto_free_digest_env(victim->b_digest); - if (victim->dh_handshake_state) - crypto_dh_free(victim->dh_handshake_state); - if (victim->extend_info) - extend_info_free(victim->extend_info); + if (!victim) + return; + + crypto_free_cipher_env(victim->f_crypto); + crypto_free_cipher_env(victim->b_crypto); + crypto_free_digest_env(victim->f_digest); + crypto_free_digest_env(victim->b_digest); + crypto_dh_free(victim->dh_handshake_state); + extend_info_free(victim->extend_info); memset(victim, 0xBB, sizeof(crypt_path_t)); /* poison memory */ tor_free(victim); @@ -577,9 +664,10 @@ circuit_dump_details(int severity, circuit_t *circ, int conn_array_index, const char *type, int this_circid, int other_circid) { log(severity, LD_CIRC, "Conn %d has %s circuit: circID %d (other side %d), " - "state %d (%s), born %d:", + "state %d (%s), born %ld:", conn_array_index, type, this_circid, other_circid, circ->state, - circuit_state_to_string(circ->state), (int)circ->timestamp_created); + circuit_state_to_string(circ->state), + (long)circ->timestamp_created.tv_sec); if (CIRCUIT_IS_ORIGIN(circ)) { /* circ starts at this node */ circuit_log_path(severity, LD_CIRC, TO_ORIGIN_CIRCUIT(circ)); } @@ -891,6 +979,11 @@ circuit_find_to_cannibalize(uint8_t purpose, extend_info_t *info, int need_uptime = (flags & CIRCLAUNCH_NEED_UPTIME) != 0; int need_capacity = (flags & CIRCLAUNCH_NEED_CAPACITY) != 0; int internal = (flags & CIRCLAUNCH_IS_INTERNAL) != 0; + or_options_t *options = get_options(); + + /* Make sure we're not trying to create a onehop circ by + * cannibalization. */ + tor_assert(!(flags & CIRCLAUNCH_ONEHOP_TUNNEL)); log_debug(LD_CIRC, "Hunting for a circ to cannibalize: purpose %d, uptime %d, " @@ -907,7 +1000,8 @@ circuit_find_to_cannibalize(uint8_t purpose, extend_info_t *info, if ((!need_uptime || circ->build_state->need_uptime) && (!need_capacity || circ->build_state->need_capacity) && (internal == circ->build_state->is_internal) && - circ->remaining_relay_early_cells) { + circ->remaining_relay_early_cells && + !circ->build_state->onehop_tunnel) { if (info) { /* need to make sure we don't duplicate hops */ crypt_path_t *hop = circ->cpath; @@ -924,6 +1018,19 @@ circuit_find_to_cannibalize(uint8_t purpose, extend_info_t *info, hop=hop->next; } while (hop!=circ->cpath); } + if (options->ExcludeNodes) { + /* Make sure no existing nodes in the circuit are excluded for + * general use. (This may be possible if StrictNodes is 0, and we + * thought we needed to use an otherwise excluded node for, say, a + * directory operation.) */ + crypt_path_t *hop = circ->cpath; + do { + if (routerset_contains_extendinfo(options->ExcludeNodes, + hop->extend_info)) + goto next; + hop = hop->next; + } while (hop != circ->cpath); + } if (!best || (best->build_state->need_uptime && !need_uptime)) best = circ; next: ; @@ -986,6 +1093,7 @@ circuit_mark_all_unused_circs(void) * This is useful for letting the user change pseudonyms, so new * streams will not be linkable to old streams. */ +/* XXX023 this is a bad name for what this function does */ void circuit_expire_all_dirty_circs(void) { @@ -996,6 +1104,8 @@ circuit_expire_all_dirty_circs(void) if (CIRCUIT_IS_ORIGIN(circ) && !circ->marked_for_close && circ->timestamp_dirty) + /* XXXX023 This is a screwed-up way to say "This is too dirty + * for new circuits. */ circ->timestamp_dirty -= options->MaxCircuitDirtiness; } } @@ -1083,14 +1193,16 @@ _circuit_mark_for_close(circuit_t *circ, int reason, int line, tor_assert(ocirc->rend_data); /* treat this like getting a nack from it */ log_info(LD_REND, "Failed intro circ %s to %s (awaiting ack). " - "Removing from descriptor.", - safe_str(ocirc->rend_data->onion_address), - safe_str(build_state_get_exit_nickname(ocirc->build_state))); + "Removing from descriptor.", + safe_str_client(ocirc->rend_data->onion_address), + safe_str_client(build_state_get_exit_nickname(ocirc->build_state))); rend_client_remove_intro_point(ocirc->build_state->chosen_exit, ocirc->rend_data); } - if (circ->n_conn) + if (circ->n_conn) { + circuit_clear_cell_queue(circ, circ->n_conn); connection_or_send_destroy(circ->n_circ_id, circ->n_conn, reason); + } if (! CIRCUIT_IS_ORIGIN(circ)) { or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); @@ -1114,8 +1226,10 @@ _circuit_mark_for_close(circuit_t *circ, int reason, int line, conn->on_circuit = NULL; } - if (or_circ->p_conn) + if (or_circ->p_conn) { + circuit_clear_cell_queue(circ, or_circ->p_conn); connection_or_send_destroy(or_circ->p_circ_id, or_circ->p_conn, reason); + } } else { origin_circuit_t *ocirc = TO_ORIGIN_CIRCUIT(circ); edge_connection_t *conn; @@ -1236,11 +1350,6 @@ assert_circuit_ok(const circuit_t *c) tor_assert(c == c2); } } -#if 0 /* false now that rendezvous exits are attached to p_streams */ - if (origin_circ) - for (conn = origin_circ->p_streams; conn; conn = conn->next_stream) - tor_assert(conn->_base.type == CONN_TYPE_AP); -#endif if (or_circ) for (conn = or_circ->n_streams; conn; conn = conn->next_stream) tor_assert(conn->_base.type == CONN_TYPE_EXIT); diff --git a/src/or/circuitlist.h b/src/or/circuitlist.h new file mode 100644 index 000000000..7b01ca3ae --- /dev/null +++ b/src/or/circuitlist.h @@ -0,0 +1,61 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file circuitlist.h + * \brief Header file for circuitlist.c. + **/ + +#ifndef _TOR_CIRCUITLIST_H +#define _TOR_CIRCUITLIST_H + +circuit_t * _circuit_get_global_list(void); +const char *circuit_state_to_string(int state); +const char *circuit_purpose_to_controller_string(uint8_t purpose); +const char *circuit_purpose_to_string(uint8_t purpose); +void circuit_dump_by_conn(connection_t *conn, int severity); +void circuit_set_p_circid_orconn(or_circuit_t *circ, circid_t id, + or_connection_t *conn); +void circuit_set_n_circid_orconn(circuit_t *circ, circid_t id, + or_connection_t *conn); +void circuit_set_state(circuit_t *circ, uint8_t state); +void circuit_close_all_marked(void); +int32_t circuit_initial_package_window(void); +origin_circuit_t *origin_circuit_new(void); +or_circuit_t *or_circuit_new(circid_t p_circ_id, or_connection_t *p_conn); +circuit_t *circuit_get_by_circid_orconn(circid_t circ_id, + or_connection_t *conn); +int circuit_id_in_use_on_orconn(circid_t circ_id, or_connection_t *conn); +circuit_t *circuit_get_by_edge_conn(edge_connection_t *conn); +void circuit_unlink_all_from_or_conn(or_connection_t *conn, int reason); +origin_circuit_t *circuit_get_by_global_id(uint32_t id); +origin_circuit_t *circuit_get_by_rend_query_and_purpose(const char *rend_query, + uint8_t purpose); +origin_circuit_t *circuit_get_next_by_pk_and_purpose(origin_circuit_t *start, + const char *digest, uint8_t purpose); +or_circuit_t *circuit_get_rendezvous(const char *cookie); +or_circuit_t *circuit_get_intro_point(const char *digest); +origin_circuit_t *circuit_find_to_cannibalize(uint8_t purpose, + extend_info_t *info, int flags); +void circuit_mark_all_unused_circs(void); +void circuit_expire_all_dirty_circs(void); +void _circuit_mark_for_close(circuit_t *circ, int reason, + int line, const char *file); +int circuit_get_cpath_len(origin_circuit_t *circ); +crypt_path_t *circuit_get_cpath_hop(origin_circuit_t *circ, int hopnum); +void circuit_get_all_pending_on_or_conn(smartlist_t *out, + or_connection_t *or_conn); +int circuit_count_pending_on_or_conn(or_connection_t *or_conn); + +#define circuit_mark_for_close(c, reason) \ + _circuit_mark_for_close((c), (reason), __LINE__, _SHORT_FILE_) + +void assert_cpath_layer_ok(const crypt_path_t *cp); +void assert_circuit_ok(const circuit_t *c); +void circuit_free_all(void); + +#endif + diff --git a/src/or/circuituse.c b/src/or/circuituse.c index 6a9c3975c..fbe2e459a 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -10,6 +10,20 @@ **/ #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "control.h" +#include "policies.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rendservice.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" /********* START VARIABLES **********/ @@ -17,7 +31,7 @@ extern circuit_t *global_circuitlist; /* from circuitlist.c */ /********* END VARIABLES ************/ -static void circuit_expire_old_circuits_clientside(time_t now); +static void circuit_expire_old_circuits_clientside(void); static void circuit_increment_failure_count(void); /** Return 1 if <b>circ</b> could be returned by circuit_get_best(). @@ -148,10 +162,14 @@ circuit_is_better(circuit_t *a, circuit_t *b, uint8_t purpose) return 1; } else { if (a->timestamp_dirty || - a->timestamp_created > b->timestamp_created) + timercmp(&a->timestamp_created, &b->timestamp_created, >)) return 1; if (CIRCUIT_IS_ORIGIN(b) && TO_ORIGIN_CIRCUIT(b)->build_state->is_internal) + /* XXX023 what the heck is this internal thing doing here. I + * think we can get rid of it. circuit_is_acceptable() already + * makes sure that is_internal is exactly what we need it to + * be. -RD */ return 1; } break; @@ -190,7 +208,7 @@ circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, int need_uptime, int need_internal) { circuit_t *circ, *best=NULL; - time_t now = time(NULL); + struct timeval now; int intro_going_on_but_too_old = 0; tor_assert(conn); @@ -199,17 +217,16 @@ circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT || purpose == CIRCUIT_PURPOSE_C_REND_JOINED); + tor_gettimeofday(&now); + for (circ=global_circuitlist;circ;circ = circ->next) { if (!circuit_is_acceptable(circ,conn,must_be_open,purpose, - need_uptime,need_internal,now)) + need_uptime,need_internal,now.tv_sec)) continue; -/* XXX022 make this 15 be a function of circuit finishing times we've - * seen lately, a la Fallon Chen's GSoC work -RD */ -#define REND_PARALLEL_INTRO_DELAY 15 if (purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT && - !must_be_open && circ->state != CIRCUIT_STATE_OPEN && - circ->timestamp_created + REND_PARALLEL_INTRO_DELAY < now) { + !must_be_open && circ->state != CIRCUIT_STATE_OPEN && + tv_mdiff(&now, &circ->timestamp_created) > circ_times.timeout_ms) { intro_going_on_but_too_old = 1; continue; } @@ -229,50 +246,73 @@ circuit_get_best(edge_connection_t *conn, int must_be_open, uint8_t purpose, return best ? TO_ORIGIN_CIRCUIT(best) : NULL; } +#if 0 /** Check whether, according to the policies in <b>options</b>, the * circuit <b>circ</b> makes sense. */ -/* XXXX currently only checks Exclude{Exit}Nodes. It should check more. */ +/* XXXX currently only checks Exclude{Exit}Nodes; it should check more. + * Also, it doesn't have the right definition of an exit circuit. Also, + * it's never called. */ int circuit_conforms_to_options(const origin_circuit_t *circ, const or_options_t *options) { const crypt_path_t *cpath, *cpath_next = NULL; - for (cpath = circ->cpath; cpath && cpath_next != circ->cpath; - cpath = cpath_next) { + /* first check if it includes any excluded nodes */ + for (cpath = circ->cpath; cpath_next != circ->cpath; cpath = cpath_next) { cpath_next = cpath->next; - if (routerset_contains_extendinfo(options->ExcludeNodes, cpath->extend_info)) return 0; + } - if (cpath->next == circ->cpath) { - /* This is apparently the exit node. */ + /* then consider the final hop */ + if (routerset_contains_extendinfo(options->ExcludeExitNodes, + circ->cpath->prev->extend_info)) + return 0; - if (routerset_contains_extendinfo(options->ExcludeExitNodes, - cpath->extend_info)) - return 0; - } - } return 1; } +#endif /** Close all circuits that start at us, aren't open, and were born * at least CircuitBuildTimeout seconds ago. */ void -circuit_expire_building(time_t now) +circuit_expire_building(void) { - circuit_t *victim, *circ = global_circuitlist; - time_t general_cutoff = now - get_options()->CircuitBuildTimeout; - time_t begindir_cutoff = now - get_options()->CircuitBuildTimeout/2; - time_t introcirc_cutoff = begindir_cutoff; + circuit_t *victim, *next_circ = global_circuitlist; + /* circ_times.timeout_ms and circ_times.close_ms are from + * circuit_build_times_get_initial_timeout() if we haven't computed + * custom timeouts yet */ + struct timeval general_cutoff, begindir_cutoff, fourhop_cutoff, + cannibalize_cutoff, close_cutoff, extremely_old_cutoff; + struct timeval now; + struct timeval introcirc_cutoff; cpath_build_state_t *build_state; - while (circ) { - time_t cutoff; - victim = circ; - circ = circ->next; + tor_gettimeofday(&now); +#define SET_CUTOFF(target, msec) do { \ + long ms = tor_lround(msec); \ + struct timeval diff; \ + diff.tv_sec = ms / 1000; \ + diff.tv_usec = (int)((ms % 1000) * 1000); \ + timersub(&now, &diff, &target); \ + } while (0) + + SET_CUTOFF(general_cutoff, circ_times.timeout_ms); + SET_CUTOFF(begindir_cutoff, circ_times.timeout_ms / 2.0); + SET_CUTOFF(fourhop_cutoff, circ_times.timeout_ms * (4/3.0)); + SET_CUTOFF(cannibalize_cutoff, circ_times.timeout_ms / 2.0); + SET_CUTOFF(close_cutoff, circ_times.close_ms); + SET_CUTOFF(extremely_old_cutoff, circ_times.close_ms*2 + 1000); + + introcirc_cutoff = begindir_cutoff; + + while (next_circ) { + struct timeval cutoff; + victim = next_circ; + next_circ = next_circ->next; if (!CIRCUIT_IS_ORIGIN(victim) || /* didn't originate here */ victim->marked_for_close) /* don't mess with marked circs */ continue; @@ -280,11 +320,19 @@ circuit_expire_building(time_t now) build_state = TO_ORIGIN_CIRCUIT(victim)->build_state; if (build_state && build_state->onehop_tunnel) cutoff = begindir_cutoff; + else if (build_state && build_state->desired_path_len == 4 + && !TO_ORIGIN_CIRCUIT(victim)->has_opened) + cutoff = fourhop_cutoff; + else if (TO_ORIGIN_CIRCUIT(victim)->has_opened) + cutoff = cannibalize_cutoff; else if (victim->purpose == CIRCUIT_PURPOSE_C_INTRODUCING) cutoff = introcirc_cutoff; + else if (victim->purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) + cutoff = close_cutoff; else cutoff = general_cutoff; - if (victim->timestamp_created > cutoff) + + if (timercmp(&victim->timestamp_created, &cutoff, >)) continue; /* it's still young, leave it alone */ #if 0 @@ -330,7 +378,7 @@ circuit_expire_building(time_t now) * because that's set when they switch purposes */ if (TO_ORIGIN_CIRCUIT(victim)->rend_data || - victim->timestamp_dirty > cutoff) + victim->timestamp_dirty > cutoff.tv_sec) continue; break; case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: @@ -339,10 +387,60 @@ circuit_expire_building(time_t now) * make an introduction attempt. so timestamp_dirty * will reflect the time since the last attempt. */ - if (victim->timestamp_dirty > cutoff) + if (victim->timestamp_dirty > cutoff.tv_sec) continue; break; } + } else { /* circuit not open, consider recording failure as timeout */ + int first_hop_succeeded = TO_ORIGIN_CIRCUIT(victim)->cpath && + TO_ORIGIN_CIRCUIT(victim)->cpath->state == CPATH_STATE_OPEN; + + if (TO_ORIGIN_CIRCUIT(victim)->p_streams != NULL) { + log_warn(LD_BUG, "Circuit %d (purpose %d, %s) has timed out, " + "yet has attached streams!", + TO_ORIGIN_CIRCUIT(victim)->global_identifier, + victim->purpose, + circuit_purpose_to_string(victim->purpose)); + tor_fragile_assert(); + continue; + } + + if (circuit_timeout_want_to_count_circ(TO_ORIGIN_CIRCUIT(victim)) && + circuit_build_times_enough_to_compute(&circ_times)) { + /* Circuits are allowed to last longer for measurement. + * Switch their purpose and wait. */ + if (victim->purpose != CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) { + control_event_circuit_status(TO_ORIGIN_CIRCUIT(victim), + CIRC_EVENT_FAILED, + END_CIRC_REASON_TIMEOUT); + victim->purpose = CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT; + /* Record this failure to check for too many timeouts + * in a row. This function does not record a time value yet + * (we do that later); it only counts the fact that we did + * have a timeout. */ + circuit_build_times_count_timeout(&circ_times, + first_hop_succeeded); + continue; + } + + /* + * If the circuit build time is much greater than we would have cut + * it off at, we probably had a suspend event along this codepath, + * and we should discard the value. + */ + if (timercmp(&victim->timestamp_created, &extremely_old_cutoff, <)) { + log_notice(LD_CIRC, + "Extremely large value for circuit build timeout: %lds. " + "Assuming clock jump. Purpose %d (%s)", + (long)(now.tv_sec - victim->timestamp_created.tv_sec), + victim->purpose, + circuit_purpose_to_string(victim->purpose)); + } else if (circuit_build_times_count_close(&circ_times, + first_hop_succeeded, + victim->timestamp_created.tv_sec)) { + circuit_build_times_set_timeout(&circ_times); + } + } } if (victim->n_conn) @@ -357,7 +455,10 @@ circuit_expire_building(time_t now) circuit_state_to_string(victim->state), victim->purpose); circuit_log_path(LOG_INFO,LD_CIRC,TO_ORIGIN_CIRCUIT(victim)); - circuit_mark_for_close(victim, END_CIRC_REASON_TIMEOUT); + if (victim->purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) + circuit_mark_for_close(victim, END_CIRC_REASON_MEASUREMENT_EXPIRED); + else + circuit_mark_for_close(victim, END_CIRC_REASON_TIMEOUT); } } @@ -431,11 +532,11 @@ circuit_stream_is_being_handled(edge_connection_t *conn, } /** Don't keep more than this many unused open circuits around. */ -#define MAX_UNUSED_OPEN_CIRCUITS 12 +#define MAX_UNUSED_OPEN_CIRCUITS 14 /** Figure out how many circuits we have open that are clean. Make * sure it's enough for all the upcoming behaviors we predict we'll have. - * But if we have too many, close the not-so-useful ones. + * But put an upper bound on the total number of circuits. */ static void circuit_predict_and_launch_new(void) @@ -517,6 +618,19 @@ circuit_predict_and_launch_new(void) circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); return; } + + /* Finally, check to see if we still need more circuits to learn + * a good build timeout. But if we're close to our max number we + * want, don't do another -- we want to leave a few slots open so + * we can still build circuits preemptively as needed. */ + if (num < MAX_UNUSED_OPEN_CIRCUITS-2 && + circuit_build_times_needs_circuits_now(&circ_times)) { + flags = CIRCLAUNCH_NEED_CAPACITY; + log_info(LD_CIRC, + "Have %d clean circs need another buildtime test circ.", num); + circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); + return; + } } /** Build a new test circuit every 5 minutes */ @@ -544,7 +658,7 @@ circuit_build_needed_circs(time_t now) time_to_new_circuit = now + options->NewCircuitPeriod; if (proxy_mode(get_options())) addressmap_clean(now); - circuit_expire_old_circuits_clientside(now); + circuit_expire_old_circuits_clientside(); #if 0 /* disable for now, until predict-and-launch-new can cull leftovers */ circ = circuit_get_youngest_clean_open(CIRCUIT_PURPOSE_C_GENERAL); @@ -624,37 +738,77 @@ circuit_detach_stream(circuit_t *circ, edge_connection_t *conn) tor_fragile_assert(); } +/** If we haven't yet decided on a good timeout value for circuit + * building, we close idles circuits aggressively so we can get more + * data points. */ +#define IDLE_TIMEOUT_WHILE_LEARNING (10*60) + /** Find each circuit that has been unused for too long, or dirty * for too long and has no streams on it: mark it for close. */ static void -circuit_expire_old_circuits_clientside(time_t now) +circuit_expire_old_circuits_clientside(void) { circuit_t *circ; - time_t cutoff = now - get_options()->CircuitIdleTimeout; + struct timeval cutoff, now; + + tor_gettimeofday(&now); + cutoff = now; + + if (circuit_build_times_needs_circuits(&circ_times)) { + /* Circuits should be shorter lived if we need more of them + * for learning a good build timeout */ + cutoff.tv_sec -= IDLE_TIMEOUT_WHILE_LEARNING; + } else { + cutoff.tv_sec -= get_options()->CircuitIdleTimeout; + } for (circ = global_circuitlist; circ; circ = circ->next) { - if (circ->marked_for_close || ! CIRCUIT_IS_ORIGIN(circ)) + if (circ->marked_for_close || !CIRCUIT_IS_ORIGIN(circ)) continue; /* If the circuit has been dirty for too long, and there are no streams * on it, mark it for close. */ if (circ->timestamp_dirty && - circ->timestamp_dirty + get_options()->MaxCircuitDirtiness < now && + circ->timestamp_dirty + get_options()->MaxCircuitDirtiness < + now.tv_sec && !TO_ORIGIN_CIRCUIT(circ)->p_streams /* nothing attached */ ) { - log_debug(LD_CIRC, "Closing n_circ_id %d (dirty %d secs ago, " + log_debug(LD_CIRC, "Closing n_circ_id %d (dirty %ld sec ago, " "purpose %d)", - circ->n_circ_id, (int)(now - circ->timestamp_dirty), + circ->n_circ_id, (long)(now.tv_sec - circ->timestamp_dirty), circ->purpose); circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); - } else if (!circ->timestamp_dirty && - circ->state == CIRCUIT_STATE_OPEN && - circ->purpose == CIRCUIT_PURPOSE_C_GENERAL) { - if (circ->timestamp_created < cutoff) { - log_debug(LD_CIRC, - "Closing circuit that has been unused for %d seconds.", - (int)(now - circ->timestamp_created)); - circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); + } else if (!circ->timestamp_dirty && circ->state == CIRCUIT_STATE_OPEN) { + if (timercmp(&circ->timestamp_created, &cutoff, <)) { + if (circ->purpose == CIRCUIT_PURPOSE_C_GENERAL || + circ->purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT || + circ->purpose == CIRCUIT_PURPOSE_S_ESTABLISH_INTRO || + circ->purpose == CIRCUIT_PURPOSE_TESTING || + (circ->purpose >= CIRCUIT_PURPOSE_C_INTRODUCING && + circ->purpose <= CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED) || + circ->purpose == CIRCUIT_PURPOSE_S_CONNECT_REND) { + log_debug(LD_CIRC, + "Closing circuit that has been unused for %ld msec.", + tv_mdiff(&circ->timestamp_created, &now)); + circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); + } else if (!TO_ORIGIN_CIRCUIT(circ)->is_ancient) { + /* Server-side rend joined circuits can end up really old, because + * they are reused by clients for longer than normal. The client + * controls their lifespan. (They never become dirty, because + * connection_exit_begin_conn() never marks anything as dirty.) + * Similarly, server-side intro circuits last a long time. */ + if (circ->purpose != CIRCUIT_PURPOSE_S_REND_JOINED && + circ->purpose != CIRCUIT_PURPOSE_S_INTRO) { + log_notice(LD_CIRC, + "Ancient non-dirty circuit %d is still around after " + "%ld milliseconds. Purpose: %d (%s)", + TO_ORIGIN_CIRCUIT(circ)->global_identifier, + tv_mdiff(&circ->timestamp_created, &now), + circ->purpose, + circuit_purpose_to_string(circ->purpose)); + TO_ORIGIN_CIRCUIT(circ)->is_ancient = 1; + } + } } } } @@ -794,6 +948,11 @@ circuit_has_opened(origin_circuit_t *circ) { control_event_circuit_status(circ, CIRC_EVENT_BUILT, 0); + /* Remember that this circuit has finished building. Now if we start + * it building again later (e.g. by extending it), we will know not + * to consider its build time. */ + circ->has_opened = 1; + switch (TO_CIRCUIT(circ)->purpose) { case CIRCUIT_PURPOSE_C_ESTABLISH_REND: rend_client_rendcirc_has_opened(circ); @@ -846,15 +1005,29 @@ circuit_build_failed(origin_circuit_t *circ) * to blame, blame it. Also, avoid this relay for a while, and * fail any one-hop directory fetches destined for it. */ const char *n_conn_id = circ->cpath->extend_info->identity_digest; + int already_marked = 0; if (circ->_base.n_conn) { or_connection_t *n_conn = circ->_base.n_conn; + if (n_conn->is_bad_for_new_circs) { + /* We only want to blame this router when a fresh healthy + * connection fails. So don't mark this router as newly failed, + * since maybe this was just an old circuit attempt that's + * finally timing out now. Also, there's no need to blow away + * circuits/streams/etc, since the failure of an unhealthy conn + * doesn't tell us much about whether a healthy conn would + * succeed. */ + already_marked = 1; + } log_info(LD_OR, "Our circuit failed to get a response from the first hop " "(%s:%d). I'm going to try to rotate to a better connection.", n_conn->_base.address, n_conn->_base.port); n_conn->is_bad_for_new_circs = 1; + } else { + log_info(LD_OR, + "Our circuit died before the first hop with no connection"); } - if (n_conn_id) { + if (n_conn_id && !already_marked) { entry_guard_register_connect_status(n_conn_id, 0, 1, time(NULL)); /* if there are any one-hop streams waiting on this circuit, fail * them now so they can retry elsewhere. */ @@ -936,8 +1109,8 @@ circuit_launch_by_router(uint8_t purpose, if (exit) info = extend_info_from_router(exit); circ = circuit_launch_by_extend_info(purpose, info, flags); - if (info) - extend_info_free(info); + + extend_info_free(info); return circ; } @@ -969,13 +1142,14 @@ circuit_launch_by_extend_info(uint8_t purpose, * internal circs rather than exit circs? -RD */ circ = circuit_find_to_cannibalize(purpose, extend_info, flags); if (circ) { - log_info(LD_CIRC,"Cannibalizing circ '%s' for purpose %d", - build_state_get_exit_nickname(circ->build_state), purpose); + log_info(LD_CIRC,"Cannibalizing circ '%s' for purpose %d (%s)", + build_state_get_exit_nickname(circ->build_state), purpose, + circuit_purpose_to_string(purpose)); circ->_base.purpose = purpose; /* reset the birth date of this circ, else expire_building * will see it and think it's been trying to build since it * began. */ - circ->_base.timestamp_created = time(NULL); + tor_gettimeofday(&circ->_base.timestamp_created); switch (purpose) { case CIRCUIT_PURPOSE_C_ESTABLISH_REND: case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: @@ -1080,15 +1254,17 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, int severity = LOG_NOTICE; /* FFFF if this is a tunneled directory fetch, don't yell * as loudly. the user doesn't even know it's happening. */ - if (options->UseBridges && bridges_known_but_down()) { + if (entry_list_is_constrained(options) && + entries_known_but_down(options)) { log_fn(severity, LD_APP|LD_DIR, - "Application request when we're believed to be " - "offline. Optimistically trying known bridges again."); - bridges_retry_all(); + "Application request when we haven't used client functionality " + "lately. Optimistically trying known %s again.", + options->UseBridges ? "bridges" : "entrynodes"); + entries_retry_all(options); } else if (!options->UseBridges || any_bridge_descriptors_known()) { log_fn(severity, LD_APP|LD_DIR, - "Application request when we're believed to be " - "offline. Optimistically trying directory fetches again."); + "Application request when we haven't used client functionality " + "lately. Optimistically trying directory fetches again."); routerlist_retry_directory_downloads(time(NULL)); } } @@ -1111,17 +1287,19 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, need_uptime)) { log_notice(LD_APP, "No Tor server allows exit to %s:%d. Rejecting.", - safe_str(conn->socks_request->address), + safe_str_client(conn->socks_request->address), conn->socks_request->port); return -1; } } else { - /* XXXX022 Duplicates checks in connection_ap_handshake_attach_circuit */ + /* XXXX023 Duplicates checks in connection_ap_handshake_attach_circuit: + * refactor into a single function? */ routerinfo_t *router = router_get_by_nickname(conn->chosen_exit_name, 1); int opt = conn->chosen_exit_optional; if (router && !connection_ap_can_use_exit(conn, router)) { log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, - "Requested exit point '%s' would refuse request. %s.", + "Requested exit point '%s' is excluded or " + "would refuse request. %s.", conn->chosen_exit_name, opt ? "Trying others" : "Closing"); if (opt) { conn->chosen_exit_optional = 0; @@ -1152,19 +1330,14 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, if (!extend_info) { log_info(LD_REND, "No intro points for '%s': re-fetching service descriptor.", - safe_str(conn->rend_data->onion_address)); - /* Fetch both, v0 and v2 rend descriptors in parallel. Use whichever - * arrives first. Exception: When using client authorization, only - * fetch v2 descriptors.*/ + safe_str_client(conn->rend_data->onion_address)); rend_client_refetch_v2_renddesc(conn->rend_data); - if (conn->rend_data->auth_type == REND_NO_AUTH) - rend_client_refetch_renddesc(conn->rend_data->onion_address); conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT; return 0; } log_info(LD_REND,"Chose '%s' as intro point for '%s'.", extend_info->nickname, - safe_str(conn->rend_data->onion_address)); + safe_str_client(conn->rend_data->onion_address)); } /* If we have specified a particular exit node for our @@ -1193,7 +1366,7 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, } if (tor_addr_from_str(&addr, conn->socks_request->address) < 0) { log_info(LD_DIR, "Broken address %s on tunnel conn. Closing.", - escaped_safe_str(conn->socks_request->address)); + escaped_safe_str_client(conn->socks_request->address)); return -1; } extend_info = extend_info_alloc(conn->chosen_exit_name+1, @@ -1235,10 +1408,20 @@ circuit_get_open_circ_or_launch(edge_connection_t *conn, flags); } - if (extend_info) - extend_info_free(extend_info); + extend_info_free(extend_info); - if (desired_circuit_purpose != CIRCUIT_PURPOSE_C_GENERAL) { + if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_GENERAL) { + /* We just caused a circuit to get built because of this stream. + * If this stream has caused a _lot_ of circuits to be built, that's + * a bad sign: we should tell the user. */ + if (conn->num_circuits_launched < NUM_CIRCUITS_LAUNCHED_THRESHOLD && + ++conn->num_circuits_launched == NUM_CIRCUITS_LAUNCHED_THRESHOLD) + log_warn(LD_BUG, "The application request to %s:%d has launched " + "%d circuits without finding one it likes.", + escaped_safe_str_client(conn->socks_request->address), + conn->socks_request->port, + conn->num_circuits_launched); + } else { /* help predict this next time */ rep_hist_note_used_internal(time(NULL), need_uptime, 1); if (circ) { @@ -1418,7 +1601,7 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) LOG_INFO : LOG_NOTICE; log_fn(severity, LD_APP, "Tried for %d seconds to get a connection to %s:%d. Giving up.", - conn_age, safe_str(conn->socks_request->address), + conn_age, safe_str_client(conn->socks_request->address), conn->socks_request->port); return -1; } @@ -1447,7 +1630,8 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) } if (router && !connection_ap_can_use_exit(conn, router)) { log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, - "Requested exit point '%s' would refuse request. %s.", + "Requested exit point '%s' is excluded or " + "would refuse request. %s.", conn->chosen_exit_name, opt ? "Trying others" : "Closing"); if (opt) { conn->chosen_exit_optional = 0; @@ -1461,7 +1645,7 @@ connection_ap_handshake_attach_circuit(edge_connection_t *conn) /* find the circuit that we should use, if there is one. */ retval = circuit_get_open_circ_or_launch( conn, CIRCUIT_PURPOSE_C_GENERAL, &circ); - if (retval < 1) // XXX021 if we totally fail, this still returns 0 -RD + if (retval < 1) // XXX022 if we totally fail, this still returns 0 -RD return retval; log_debug(LD_APP|LD_CIRC, diff --git a/src/or/circuituse.h b/src/or/circuituse.h new file mode 100644 index 000000000..9f393ab37 --- /dev/null +++ b/src/or/circuituse.h @@ -0,0 +1,55 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file circuituse.h + * \brief Header file for circuituse.c. + **/ + +#ifndef _TOR_CIRCUITUSE_H +#define _TOR_CIRCUITUSE_H + +void circuit_expire_building(void); +void circuit_remove_handled_ports(smartlist_t *needed_ports); +int circuit_stream_is_being_handled(edge_connection_t *conn, uint16_t port, + int min); +#if 0 +int circuit_conforms_to_options(const origin_circuit_t *circ, + const or_options_t *options); +#endif +void circuit_build_needed_circs(time_t now); +void circuit_detach_stream(circuit_t *circ, edge_connection_t *conn); + +void circuit_expire_old_circuits_serverside(time_t now); + +void reset_bandwidth_test(void); +int circuit_enough_testing_circs(void); + +void circuit_has_opened(origin_circuit_t *circ); +void circuit_build_failed(origin_circuit_t *circ); + +/** Flag to set when a circuit should have only a single hop. */ +#define CIRCLAUNCH_ONEHOP_TUNNEL (1<<0) +/** Flag to set when a circuit needs to be built of high-uptime nodes */ +#define CIRCLAUNCH_NEED_UPTIME (1<<1) +/** Flag to set when a circuit needs to be built of high-capacity nodes */ +#define CIRCLAUNCH_NEED_CAPACITY (1<<2) +/** Flag to set when the last hop of a circuit doesn't need to be an + * exit node. */ +#define CIRCLAUNCH_IS_INTERNAL (1<<3) +origin_circuit_t *circuit_launch_by_extend_info(uint8_t purpose, + extend_info_t *info, + int flags); +origin_circuit_t *circuit_launch_by_router(uint8_t purpose, + routerinfo_t *exit, int flags); +void circuit_reset_failure_count(int timeout); +int connection_ap_handshake_attach_chosen_circuit(edge_connection_t *conn, + origin_circuit_t *circ, + crypt_path_t *cpath); +int connection_ap_handshake_attach_circuit(edge_connection_t *conn); + +#endif + diff --git a/src/or/command.c b/src/or/command.c index 926945680..00d9af33f 100644 --- a/src/or/command.c +++ b/src/or/command.c @@ -16,6 +16,19 @@ */ #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "command.h" +#include "connection.h" +#include "connection_or.h" +#include "config.h" +#include "control.h" +#include "cpuworker.h" +#include "hibernate.h" +#include "onion.h" +#include "relay.h" +#include "router.h" +#include "routerlist.h" /** How many CELL_PADDING cells have we received, ever? */ uint64_t stats_n_padding_cells_processed = 0; @@ -275,7 +288,14 @@ command_process_create_cell(cell_t *cell, or_connection_t *conn) /* hand it off to the cpuworkers, and then return. */ if (assign_onionskin_to_cpuworker(NULL, circ, onionskin) < 0) { - log_warn(LD_GENERAL,"Failed to hand off onionskin. Closing."); +#define WARN_HANDOFF_FAILURE_INTERVAL (6*60*60) + static ratelim_t handoff_warning = + RATELIM_INIT(WARN_HANDOFF_FAILURE_INTERVAL); + char *m; + if ((m = rate_limit_log(&handoff_warning, approx_time()))) { + log_warn(LD_GENERAL,"Failed to hand off onionskin. Closing.%s",m); + tor_free(m); + } circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_INTERNAL); return; } @@ -396,15 +416,18 @@ command_process_relay_cell(cell_t *cell, or_connection_t *conn) * gotten no more than MAX_RELAY_EARLY_CELLS_PER_CIRCUIT of them. */ if (cell->command == CELL_RELAY_EARLY) { if (direction == CELL_DIRECTION_IN) { - /* XXX Allow an unlimited number of inbound relay_early cells for - * now, for hidden service compatibility. See bug 1038. -RD */ + /* Allow an unlimited number of inbound relay_early cells, + * for hidden service compatibility. There isn't any way to make + * a long circuit through inbound relay_early cells anyway. See + * bug 1038. -RD */ } else { or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); if (or_circ->remaining_relay_early_cells == 0) { log_fn(LOG_PROTOCOL_WARN, LD_OR, "Received too many RELAY_EARLY cells on circ %d from %s:%d." " Closing circuit.", - cell->circ_id, safe_str(conn->_base.address), conn->_base.port); + cell->circ_id, safe_str(conn->_base.address), + conn->_base.port); circuit_mark_for_close(circ, END_CIRC_REASON_TORPROTOCOL); return; } @@ -512,7 +535,8 @@ command_process_versions_cell(var_cell_t *cell, or_connection_t *conn) conn->handshake_state->received_versions = 1; log_info(LD_OR, "Negotiated version %d with %s:%d; sending NETINFO.", - highest_supported_version, safe_str(conn->_base.address), + highest_supported_version, + safe_str_client(conn->_base.address), conn->_base.port); tor_assert(conn->link_proto >= 2); @@ -627,8 +651,8 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) else log_info(LD_OR, "Got good NETINFO cell from %s:%d; OR connection is now " "open, using protocol version %d", - safe_str(conn->_base.address), conn->_base.port, - (int)conn->link_proto); + safe_str_client(conn->_base.address), + conn->_base.port, (int)conn->link_proto); assert_connection_ok(TO_CONN(conn),time(NULL)); } diff --git a/src/or/command.h b/src/or/command.h new file mode 100644 index 000000000..95b0f3a93 --- /dev/null +++ b/src/or/command.h @@ -0,0 +1,25 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file command.h + * \brief Header file for command.c. + **/ + +#ifndef _TOR_COMMAND_H +#define _TOR_COMMAND_H + +void command_process_cell(cell_t *cell, or_connection_t *conn); +void command_process_var_cell(var_cell_t *cell, or_connection_t *conn); + +extern uint64_t stats_n_padding_cells_processed; +extern uint64_t stats_n_create_cells_processed; +extern uint64_t stats_n_created_cells_processed; +extern uint64_t stats_n_relay_cells_processed; +extern uint64_t stats_n_destroy_cells_processed; + +#endif + diff --git a/src/or/config.c b/src/or/config.c index fbfa771ed..9f94ef1af 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -12,6 +12,28 @@ #define CONFIG_PRIVATE #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "cpuworker.h" +#include "dirserv.h" +#include "dirvote.h" +#include "dns.h" +#include "geoip.h" +#include "hibernate.h" +#include "main.h" +#include "networkstatus.h" +#include "policies.h" +#include "relay.h" +#include "rendclient.h" +#include "rendservice.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" #ifdef MS_WINDOWS #include <shlobj.h> #endif @@ -61,11 +83,12 @@ static config_abbrev_t _option_abbrevs[] = { PLURAL(LongLivedPort), PLURAL(HiddenServiceNode), PLURAL(HiddenServiceExcludeNode), - PLURAL(NumCpu), + PLURAL(NumCPU), PLURAL(RendNode), PLURAL(RendExcludeNode), PLURAL(StrictEntryNode), PLURAL(StrictExitNode), + PLURAL(StrictNode), { "l", "Log", 1, 0}, { "AllowUnverifiedNodes", "AllowInvalidNodes", 0, 0}, { "AutomapHostSuffixes", "AutomapHostsSuffixes", 0, 0}, @@ -83,10 +106,12 @@ static config_abbrev_t _option_abbrevs[] = { { "NumEntryNodes", "NumEntryGuards", 0, 0}, { "ResolvConf", "ServerDNSResolvConfFile", 0, 1}, { "SearchDomains", "ServerDNSSearchDomains", 0, 1}, - { "ServerDNSAllowBrokenResolvConf", "ServerDNSAllowBrokenConfig", 0, 0 }, + { "ServerDNSAllowBrokenResolvConf", "ServerDNSAllowBrokenConfig", 0, 0}, { "PreferTunnelledDirConns", "PreferTunneledDirConns", 0, 0}, { "BridgeAuthoritativeDirectory", "BridgeAuthoritativeDir", 0, 0}, { "HashedControlPassword", "__HashedControlSessionPassword", 1, 0}, + { "StrictEntryNodes", "StrictNodes", 0, 1}, + { "StrictExitNodes", "StrictNodes", 0, 1}, { NULL, NULL, 0, 0}, }; @@ -134,6 +159,7 @@ static config_var_t _option_vars[] = { V(AccountingMax, MEMUNIT, "0 bytes"), V(AccountingStart, STRING, NULL), V(Address, STRING, NULL), + V(AllowDotExit, BOOL, "0"), V(AllowInvalidNodes, CSV, "middle,rendezvous"), V(AllowNonRFC953Hostnames, BOOL, "0"), V(AllowSingleHopCircuits, BOOL, "0"), @@ -162,10 +188,16 @@ static config_var_t _option_vars[] = { V(BridgePassword, STRING, NULL), V(BridgeRecordUsageByCountry, BOOL, "1"), V(BridgeRelay, BOOL, "0"), - V(CircuitBuildTimeout, INTERVAL, "1 minute"), + V(CellStatistics, BOOL, "0"), + V(LearnCircuitBuildTimeout, BOOL, "1"), + V(CircuitBuildTimeout, INTERVAL, "0"), V(CircuitIdleTimeout, INTERVAL, "1 hour"), + V(CircuitStreamTimeout, INTERVAL, "0"), + V(CircuitPriorityHalflife, DOUBLE, "-100.0"), /*negative:'Use default'*/ V(ClientDNSRejectInternalAddresses, BOOL,"1"), + V(ClientRejectInternalAddresses, BOOL, "1"), V(ClientOnly, BOOL, "0"), + V(ConsensusParams, STRING, NULL), V(ConnLimit, UINT, "1000"), V(ConstrainedSockets, BOOL, "0"), V(ConstrainedSockSize, MEMUNIT, "8192"), @@ -186,18 +218,19 @@ static config_var_t _option_vars[] = { V(DirPort, UINT, "0"), V(DirPortFrontPage, FILENAME, NULL), OBSOLETE("DirPostPeriod"), -#ifdef ENABLE_GEOIP_STATS - V(DirRecordUsageByCountry, BOOL, "0"), - V(DirRecordUsageGranularity, UINT, "4"), - V(DirRecordUsageRetainIPs, INTERVAL, "14 days"), - V(DirRecordUsageSaveInterval, INTERVAL, "6 hours"), -#endif + OBSOLETE("DirRecordUsageByCountry"), + OBSOLETE("DirRecordUsageGranularity"), + OBSOLETE("DirRecordUsageRetainIPs"), + OBSOLETE("DirRecordUsageSaveInterval"), + V(DirReqStatistics, BOOL, "0"), VAR("DirServer", LINELIST, DirServers, NULL), + V(DisableAllSwap, BOOL, "0"), V(DNSPort, UINT, "0"), V(DNSListenAddress, LINELIST, NULL), V(DownloadExtraInfo, BOOL, "0"), V(EnforceDistinctSubnets, BOOL, "1"), V(EntryNodes, ROUTERSET, NULL), + V(EntryStatistics, BOOL, "0"), V(TestingEstimatedDescriptorPropagationTime, INTERVAL, "10 minutes"), V(ExcludeNodes, ROUTERSET, NULL), V(ExcludeExitNodes, ROUTERSET, NULL), @@ -205,15 +238,24 @@ static config_var_t _option_vars[] = { V(ExitNodes, ROUTERSET, NULL), V(ExitPolicy, LINELIST, NULL), V(ExitPolicyRejectPrivate, BOOL, "1"), + V(ExitPortStatistics, BOOL, "0"), + V(ExtraInfoStatistics, BOOL, "0"), + +#if defined (WINCE) + V(FallbackNetworkstatusFile, FILENAME, "fallback-consensus"), +#else V(FallbackNetworkstatusFile, FILENAME, SHARE_DATADIR PATH_SEPARATOR "tor" PATH_SEPARATOR "fallback-consensus"), +#endif V(FascistFirewall, BOOL, "0"), V(FirewallPorts, CSV, ""), V(FastFirstHopPK, BOOL, "1"), V(FetchDirInfoEarly, BOOL, "0"), + V(FetchDirInfoExtraEarly, BOOL, "0"), V(FetchServerDescriptors, BOOL, "1"), V(FetchHidServDescriptors, BOOL, "1"), V(FetchUselessDescriptors, BOOL, "0"), + V(FetchV2Networkstatus, BOOL, "0"), #ifdef WIN32 V(GeoIPFile, FILENAME, "<default>"), #else @@ -222,6 +264,8 @@ static config_var_t _option_vars[] = { #endif OBSOLETE("Group"), V(HardwareAccel, BOOL, "0"), + V(AccelName, STRING, NULL), + V(AccelDir, FILENAME, NULL), V(HashedControlPassword, LINELIST, NULL), V(HidServDirectoryV2, BOOL, "1"), VAR("HiddenServiceDir", LINELIST_S, RendConfigLines, NULL), @@ -233,14 +277,19 @@ static config_var_t _option_vars[] = { VAR("HiddenServiceAuthorizeClient",LINELIST_S,RendConfigLines, NULL), V(HidServAuth, LINELIST, NULL), V(HSAuthoritativeDir, BOOL, "0"), - V(HSAuthorityRecordStats, BOOL, "0"), - V(HttpProxy, STRING, NULL), - V(HttpProxyAuthenticator, STRING, NULL), - V(HttpsProxy, STRING, NULL), - V(HttpsProxyAuthenticator, STRING, NULL), + OBSOLETE("HSAuthorityRecordStats"), + V(HTTPProxy, STRING, NULL), + V(HTTPProxyAuthenticator, STRING, NULL), + V(HTTPSProxy, STRING, NULL), + V(HTTPSProxyAuthenticator, STRING, NULL), + V(Socks4Proxy, STRING, NULL), + V(Socks5Proxy, STRING, NULL), + V(Socks5ProxyUsername, STRING, NULL), + V(Socks5ProxyPassword, STRING, NULL), OBSOLETE("IgnoreVersion"), V(KeepalivePeriod, INTERVAL, "5 minutes"), VAR("Log", LINELIST, Logs, NULL), + V(LogMessageDomains, BOOL, "0"), OBSOLETE("LinkPadding"), OBSOLETE("LogLevel"), OBSOLETE("LogFile"), @@ -254,17 +303,20 @@ static config_var_t _option_vars[] = { V(MyFamily, STRING, NULL), V(NewCircuitPeriod, INTERVAL, "30 seconds"), VAR("NamingAuthoritativeDirectory",BOOL, NamingAuthoritativeDir, "0"), - V(NatdListenAddress, LINELIST, NULL), - V(NatdPort, UINT, "0"), + V(NATDListenAddress, LINELIST, NULL), + V(NATDPort, UINT, "0"), V(Nickname, STRING, NULL), - V(NoPublish, BOOL, "0"), + V(WarnUnsafeSocks, BOOL, "1"), + OBSOLETE("NoPublish"), VAR("NodeFamily", LINELIST, NodeFamilies, NULL), - V(NumCpus, UINT, "1"), + V(NumCPUs, UINT, "1"), V(NumEntryGuards, UINT, "3"), V(ORListenAddress, LINELIST, NULL), V(ORPort, UINT, "0"), V(OutboundBindAddress, STRING, NULL), OBSOLETE("PathlenCoinWeight"), + V(PerConnBWBurst, MEMUNIT, "0"), + V(PerConnBWRate, MEMUNIT, "0"), V(PidFile, STRING, NULL), V(TestingTorNetwork, BOOL, "0"), V(PreferTunneledDirConns, BOOL, "1"), @@ -278,6 +330,7 @@ static config_var_t _option_vars[] = { V(RecommendedClientVersions, LINELIST, NULL), V(RecommendedServerVersions, LINELIST, NULL), OBSOLETE("RedirectExit"), + V(RefuseUnknownExits, STRING, "auto"), V(RejectPlaintextPorts, CSV, ""), V(RelayBandwidthBurst, MEMUNIT, "0"), V(RelayBandwidthRate, MEMUNIT, "0"), @@ -287,8 +340,9 @@ static config_var_t _option_vars[] = { V(RephistTrackTime, INTERVAL, "24 hours"), OBSOLETE("RouterFile"), V(RunAsDaemon, BOOL, "0"), - V(RunTesting, BOOL, "0"), - V(SafeLogging, BOOL, "1"), +// V(RunTesting, BOOL, "0"), + OBSOLETE("RunTesting"), // currently unused + V(SafeLogging, STRING, "1"), V(SafeSocks, BOOL, "0"), V(ServerDNSAllowBrokenConfig, BOOL, "1"), V(ServerDNSAllowNonRFC953Hostnames, BOOL,"0"), @@ -304,8 +358,7 @@ static config_var_t _option_vars[] = { V(SocksPort, UINT, "9050"), V(SocksTimeout, INTERVAL, "2 minutes"), OBSOLETE("StatusFetchPeriod"), - V(StrictEntryNodes, BOOL, "0"), - V(StrictExitNodes, BOOL, "0"), + V(StrictNodes, BOOL, "0"), OBSOLETE("SysLog"), V(TestSocks, BOOL, "0"), OBSOLETE("TestVia"), @@ -330,6 +383,7 @@ static config_var_t _option_vars[] = { V(V3AuthDistDelay, INTERVAL, "5 minutes"), V(V3AuthNIntervalsValid, UINT, "3"), V(V3AuthUseLegacyKey, BOOL, "0"), + V(V3BandwidthsFile, FILENAME, NULL), VAR("VersioningAuthoritativeDirectory",BOOL,VersioningAuthoritativeDir, "0"), V(VirtualAddrNetwork, STRING, "127.192.0.0/10"), V(WarnPlaintextPorts, CSV, "23,109,110,143"), @@ -340,6 +394,8 @@ static config_var_t _option_vars[] = { VAR("__HashedControlSessionPassword", LINELIST, HashedControlSessionPassword, NULL), V(MinUptimeHidServDirectoryV2, INTERVAL, "24 hours"), + V(_UsingTestNetworkDefaults, BOOL, "0"), + { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; @@ -353,6 +409,7 @@ static config_var_t testing_tor_network_defaults[] = { V(AuthDirMaxServersPerAddr, UINT, "0"), V(AuthDirMaxServersPerAuthAddr,UINT, "0"), V(ClientDNSRejectInternalAddresses, BOOL,"0"), + V(ClientRejectInternalAddresses, BOOL, "0"), V(ExitPolicyRejectPrivate, BOOL, "0"), V(V3AuthVotingInterval, INTERVAL, "5 minutes"), V(V3AuthVoteDelay, INTERVAL, "20 seconds"), @@ -362,6 +419,8 @@ static config_var_t testing_tor_network_defaults[] = { V(TestingV3AuthInitialDistDelay, INTERVAL, "20 seconds"), V(TestingAuthDirTimeToLearnReachability, INTERVAL, "0 minutes"), V(TestingEstimatedDescriptorPropagationTime, INTERVAL, "0 minutes"), + V(MinUptimeHidServDirectoryV2, INTERVAL, "0 minutes"), + V(_UsingTestNetworkDefaults, BOOL, "1"), { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; #undef VAR @@ -377,6 +436,9 @@ static config_var_t _state_vars[] = { V(AccountingExpectedUsage, MEMUNIT, NULL), V(AccountingIntervalStart, ISOTIME, NULL), V(AccountingSecondsActive, INTERVAL, NULL), + V(AccountingSecondsToReachSoftLimit,INTERVAL, NULL), + V(AccountingSoftLimitHitAt, ISOTIME, NULL), + V(AccountingBytesAtSoftLimit, MEMUNIT, NULL), VAR("EntryGuard", LINELIST_S, EntryGuards, NULL), VAR("EntryGuardDownSince", LINELIST_S, EntryGuards, NULL), @@ -387,15 +449,30 @@ static config_var_t _state_vars[] = { V(BWHistoryReadEnds, ISOTIME, NULL), V(BWHistoryReadInterval, UINT, "900"), V(BWHistoryReadValues, CSV, ""), + V(BWHistoryReadMaxima, CSV, ""), V(BWHistoryWriteEnds, ISOTIME, NULL), V(BWHistoryWriteInterval, UINT, "900"), V(BWHistoryWriteValues, CSV, ""), + V(BWHistoryWriteMaxima, CSV, ""), + V(BWHistoryDirReadEnds, ISOTIME, NULL), + V(BWHistoryDirReadInterval, UINT, "900"), + V(BWHistoryDirReadValues, CSV, ""), + V(BWHistoryDirReadMaxima, CSV, ""), + V(BWHistoryDirWriteEnds, ISOTIME, NULL), + V(BWHistoryDirWriteInterval, UINT, "900"), + V(BWHistoryDirWriteValues, CSV, ""), + V(BWHistoryDirWriteMaxima, CSV, ""), V(TorVersion, STRING, NULL), V(LastRotatedOnionKey, ISOTIME, NULL), V(LastWritten, ISOTIME, NULL), + V(TotalBuildTimes, UINT, NULL), + V(CircuitBuildAbandonedCount, UINT, "0"), + VAR("CircuitBuildTimeBin", LINELIST_S, BuildtimeHistogram, NULL), + VAR("BuildtimeHistogram", LINELIST_V, BuildtimeHistogram, NULL), + { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; @@ -410,213 +487,6 @@ typedef struct config_var_description_t { const char *description; } config_var_description_t; -/** Descriptions of the configuration options, to be displayed by online - * option browsers */ -/* XXXX022 did anybody want this? at all? If not, kill it.*/ -static config_var_description_t options_description[] = { - /* ==== general options */ - { "AvoidDiskWrites", "If non-zero, try to write to disk less frequently than" - " we would otherwise." }, - { "BandwidthRate", "A token bucket limits the average incoming bandwidth on " - "this node to the specified number of bytes per second." }, - { "BandwidthBurst", "Limit the maximum token buffer size (also known as " - "burst) to the given number of bytes." }, - { "ConnLimit", "Minimum number of simultaneous sockets we must have." }, - { "ConstrainedSockets", "Shrink tx and rx buffers for sockets to avoid " - "system limits on vservers and related environments. See man page for " - "more information regarding this option." }, - { "ConstrainedSockSize", "Limit socket buffers to this size when " - "ConstrainedSockets is enabled." }, - /* ControlListenAddress */ - { "ControlPort", "If set, Tor will accept connections from the same machine " - "(localhost only) on this port, and allow those connections to control " - "the Tor process using the Tor Control Protocol (described in " - "control-spec.txt).", }, - { "CookieAuthentication", "If this option is set to 1, don't allow any " - "connections to the control port except when the connecting process " - "can read a file that Tor creates in its data directory." }, - { "DataDirectory", "Store working data, state, keys, and caches here." }, - { "DirServer", "Tor only trusts directories signed with one of these " - "servers' keys. Used to override the standard list of directory " - "authorities." }, - /* { "FastFirstHopPK", "" }, */ - /* FetchServerDescriptors, FetchHidServDescriptors, - * FetchUselessDescriptors */ - { "HardwareAccel", "If set, Tor tries to use hardware crypto accelerators " - "when it can." }, - /* HashedControlPassword */ - { "HTTPProxy", "Force Tor to make all HTTP directory requests through this " - "host:port (or host:80 if port is not set)." }, - { "HTTPProxyAuthenticator", "A username:password pair to be used with " - "HTTPProxy." }, - { "HTTPSProxy", "Force Tor to make all TLS (SSL) connections through this " - "host:port (or host:80 if port is not set)." }, - { "HTTPSProxyAuthenticator", "A username:password pair to be used with " - "HTTPSProxy." }, - { "KeepalivePeriod", "Send a padding cell every N seconds to keep firewalls " - "from closing our connections while Tor is not in use." }, - { "Log", "Where to send logging messages. Format is " - "minSeverity[-maxSeverity] (stderr|stdout|syslog|file FILENAME)." }, - { "OutboundBindAddress", "Make all outbound connections originate from the " - "provided IP address (only useful for multiple network interfaces)." }, - { "PIDFile", "On startup, write our PID to this file. On clean shutdown, " - "remove the file." }, - { "PreferTunneledDirConns", "If non-zero, avoid directory servers that " - "don't support tunneled connections." }, - /* PreferTunneledDirConns */ - /* ProtocolWarnings */ - /* RephistTrackTime */ - { "RunAsDaemon", "If set, Tor forks and daemonizes to the background when " - "started. Unix only." }, - { "SafeLogging", "If set to 0, Tor logs potentially sensitive strings " - "rather than replacing them with the string [scrubbed]." }, - { "TunnelDirConns", "If non-zero, when a directory server we contact " - "supports it, we will build a one-hop circuit and make an encrypted " - "connection via its ORPort." }, - { "User", "On startup, setuid to this user." }, - - /* ==== client options */ - { "AllowInvalidNodes", "Where on our circuits should Tor allow servers " - "that the directory authorities haven't called \"valid\"?" }, - { "AllowNonRFC953Hostnames", "If set to 1, we don't automatically reject " - "hostnames for having invalid characters." }, - /* CircuitBuildTimeout, CircuitIdleTimeout */ - { "ClientOnly", "If set to 1, Tor will under no circumstances run as a " - "server, even if ORPort is enabled." }, - { "EntryNodes", "A list of preferred entry nodes to use for the first hop " - "in circuits, when possible." }, - /* { "EnforceDistinctSubnets" , "" }, */ - { "ExitNodes", "A list of preferred nodes to use for the last hop in " - "circuits, when possible." }, - { "ExcludeNodes", "A list of nodes never to use when building a circuit." }, - { "FascistFirewall", "If set, Tor will only create outgoing connections to " - "servers running on the ports listed in FirewallPorts." }, - { "FirewallPorts", "A list of ports that we can connect to. Only used " - "when FascistFirewall is set." }, - { "LongLivedPorts", "A list of ports for services that tend to require " - "high-uptime connections." }, - { "MapAddress", "Force Tor to treat all requests for one address as if " - "they were for another." }, - { "NewCircuitPeriod", "Force Tor to consider whether to build a new circuit " - "every NUM seconds." }, - { "MaxCircuitDirtiness", "Do not attach new streams to a circuit that has " - "been used more than this many seconds ago." }, - /* NatdPort, NatdListenAddress */ - { "NodeFamily", "A list of servers that constitute a 'family' and should " - "never be used in the same circuit." }, - { "NumEntryGuards", "How many entry guards should we keep at a time?" }, - /* PathlenCoinWeight */ - { "ReachableAddresses", "Addresses we can connect to, as IP/bits:port-port. " - "By default, we assume all addresses are reachable." }, - /* reachablediraddresses, reachableoraddresses. */ - /* SafeSOCKS */ - { "SOCKSPort", "The port where we listen for SOCKS connections from " - "applications." }, - { "SOCKSListenAddress", "Bind to this address to listen to connections from " - "SOCKS-speaking applications." }, - { "SOCKSPolicy", "Set an entry policy to limit which addresses can connect " - "to the SOCKSPort." }, - /* SocksTimeout */ - { "StrictExitNodes", "If set, Tor will fail to operate when none of the " - "configured ExitNodes can be used." }, - { "StrictEntryNodes", "If set, Tor will fail to operate when none of the " - "configured EntryNodes can be used." }, - /* TestSocks */ - { "TrackHostsExit", "Hosts and domains which should, if possible, be " - "accessed from the same exit node each time we connect to them." }, - { "TrackHostsExitExpire", "Time after which we forget which exit we were " - "using to connect to hosts in TrackHostsExit." }, - /* "TransPort", "TransListenAddress */ - { "UseEntryGuards", "Set to 0 if we want to pick from the whole set of " - "servers for the first position in each circuit, rather than picking a " - "set of 'Guards' to prevent profiling attacks." }, - - /* === server options */ - { "Address", "The advertised (external) address we should use." }, - /* Accounting* options. */ - /* AssumeReachable */ - { "ContactInfo", "Administrative contact information to advertise for this " - "server." }, - { "ExitPolicy", "Address/port ranges for which to accept or reject outgoing " - "connections on behalf of Tor users." }, - /* { "ExitPolicyRejectPrivate, "" }, */ - { "MaxAdvertisedBandwidth", "If set, we will not advertise more than this " - "amount of bandwidth for our bandwidth rate, regardless of how much " - "bandwidth we actually detect." }, - { "MaxOnionsPending", "Reject new attempts to extend circuits when we " - "already have this many pending." }, - { "MyFamily", "Declare a list of other servers as belonging to the same " - "family as this one, so that clients will not use two from the same " - "family in the same circuit." }, - { "Nickname", "Set the server nickname." }, - { "NoPublish", "{DEPRECATED}" }, - { "NumCPUs", "How many processes to use at once for public-key crypto." }, - { "ORPort", "Advertise this port to listen for connections from Tor clients " - "and servers." }, - { "ORListenAddress", "Bind to this address to listen for connections from " - "clients and servers, instead of the default 0.0.0.0:ORPort." }, - { "PublishServerDescriptor", "Set to 0 to keep the server from " - "uploading info to the directory authorities." }, - /* ServerDNS: DetectHijacking, ResolvConfFile, SearchDomains */ - { "ShutdownWaitLength", "Wait this long for clients to finish when " - "shutting down because of a SIGINT." }, - - /* === directory cache options */ - { "DirPort", "Serve directory information from this port, and act as a " - "directory cache." }, - { "DirPortFrontPage", "Serve a static html disclaimer on DirPort." }, - { "DirListenAddress", "Bind to this address to listen for connections from " - "clients and servers, instead of the default 0.0.0.0:DirPort." }, - { "DirPolicy", "Set a policy to limit who can connect to the directory " - "port." }, - - /* Authority options: AuthDirBadExit, AuthDirInvalid, AuthDirReject, - * AuthDirRejectUnlisted, AuthDirListBadExits, AuthoritativeDirectory, - * DirAllowPrivateAddresses, HSAuthoritativeDir, - * NamingAuthoritativeDirectory, RecommendedVersions, - * RecommendedClientVersions, RecommendedServerVersions, RendPostPeriod, - * RunTesting, V1AuthoritativeDirectory, VersioningAuthoritativeDirectory, */ - - /* Hidden service options: HiddenService: dir,excludenodes, nodes, - * options, port. PublishHidServDescriptor */ - - /* Nonpersistent options: __LeaveStreamsUnattached, __AllDirActionsPrivate */ - { NULL, NULL }, -}; - -/** Online description of state variables. */ -static config_var_description_t state_description[] = { - { "AccountingBytesReadInInterval", - "How many bytes have we read in this accounting period?" }, - { "AccountingBytesWrittenInInterval", - "How many bytes have we written in this accounting period?" }, - { "AccountingExpectedUsage", - "How many bytes did we expect to use per minute? (0 for no estimate.)" }, - { "AccountingIntervalStart", "When did this accounting period begin?" }, - { "AccountingSecondsActive", "How long have we been awake in this period?" }, - - { "BWHistoryReadEnds", "When does the last-recorded read-interval end?" }, - { "BWHistoryReadInterval", "How long is each read-interval (in seconds)?" }, - { "BWHistoryReadValues", "Number of bytes read in each interval." }, - { "BWHistoryWriteEnds", "When does the last-recorded write-interval end?" }, - { "BWHistoryWriteInterval", "How long is each write-interval (in seconds)?"}, - { "BWHistoryWriteValues", "Number of bytes written in each interval." }, - - { "EntryGuard", "One of the nodes we have chosen as a fixed entry" }, - { "EntryGuardDownSince", - "The last entry guard has been unreachable since this time." }, - { "EntryGuardUnlistedSince", - "The last entry guard has been unusable since this time." }, - - { "LastRotatedOnionKey", - "The last time at which we changed the medium-term private key used for " - "building circuits." }, - { "LastWritten", "When was this state file last regenerated?" }, - - { "TorVersion", "Which version of Tor generated this state file?" }, - { NULL, NULL }, -}; - /** Type of a callback to validate whether a given configuration is * well-formed and consistent. See options_trial_assign() for documentation * of arguments. */ @@ -635,8 +505,6 @@ typedef struct { config_var_t *vars; /**< List of variables we recognize, their default * values, and where we stick them in the structure. */ validate_fn_t validate_fn; /**< Function to validate config. */ - /** Documentation for configuration variables. */ - config_var_description_t *descriptions; /** If present, extra is a LINELIST variable for unrecognized * lines. Otherwise, unrecognized lines are an error. */ config_var_t *extra; @@ -700,20 +568,6 @@ static uint64_t config_parse_memunit(const char *s, int *ok); static int config_parse_interval(const char *s, int *ok); static void init_libevent(void); static int opt_streq(const char *s1, const char *s2); -/** Versions of libevent. */ -typedef enum { - /* Note: we compare these, so it's important that "old" precede everything, - * and that "other" come last. */ - LE_OLD=0, LE_10C, LE_10D, LE_10E, LE_11, LE_11A, LE_11B, LE_12, LE_12A, - LE_13, LE_13A, LE_13B, LE_13C, LE_13D, LE_13E, - LE_140, LE_141, LE_142, LE_143, LE_144, LE_145, LE_146, LE_147, LE_148, - LE_1499, - LE_OTHER -} le_version_t; -static le_version_t decode_libevent_version(const char *v, int *bincompat_out); -#if defined(HAVE_EVENT_GET_VERSION) && defined(HAVE_EVENT_GET_METHOD) -static void check_libevent_version(const char *m, int server); -#endif /** Magic value for or_options_t. */ #define OR_OPTIONS_MAGIC 9090909 @@ -726,7 +580,6 @@ static config_format_t options_format = { _option_abbrevs, _option_vars, (validate_fn_t)options_validate, - options_description, NULL }; @@ -747,7 +600,6 @@ static config_format_t state_format = { _state_abbrevs, _state_vars, (validate_fn_t)or_state_validate, - state_description, &state_extra_var, }; @@ -812,13 +664,13 @@ set_options(or_options_t *new_val, char **msg) "Acting on config options left us in a broken state. Dying."); exit(1); } - if (old_options) - config_free(&options_format, old_options); + + config_free(&options_format, old_options); return 0; } -extern const char tor_svn_revision[]; /* from tor_main.c */ +extern const char tor_git_revision[]; /* from tor_main.c */ /** The version of this Tor process, as parsed. */ static char *_version = NULL; @@ -828,10 +680,10 @@ const char * get_version(void) { if (_version == NULL) { - if (strlen(tor_svn_revision)) { - size_t len = strlen(VERSION)+strlen(tor_svn_revision)+16; + if (strlen(tor_git_revision)) { + size_t len = strlen(VERSION)+strlen(tor_git_revision)+16; _version = tor_malloc(len); - tor_snprintf(_version, len, "%s (git-%s)", VERSION, tor_svn_revision); + tor_snprintf(_version, len, "%s (git-%s)", VERSION, tor_git_revision); } else { _version = tor_strdup(VERSION); } @@ -844,8 +696,10 @@ get_version(void) static void or_options_free(or_options_t *options) { - if (options->_ExcludeExitNodesUnion) - routerset_free(options->_ExcludeExitNodesUnion); + if (!options) + return; + + routerset_free(options->_ExcludeExitNodesUnion); config_free(&options_format, options); } @@ -854,43 +708,72 @@ or_options_free(or_options_t *options) void config_free_all(void) { - if (global_options) { - or_options_free(global_options); - global_options = NULL; - } - if (global_state) { - config_free(&state_format, global_state); - global_state = NULL; - } - if (global_cmdline_options) { - config_free_lines(global_cmdline_options); - global_cmdline_options = NULL; - } + or_options_free(global_options); + global_options = NULL; + + config_free(&state_format, global_state); + global_state = NULL; + + config_free_lines(global_cmdline_options); + global_cmdline_options = NULL; + tor_free(torrc_fname); tor_free(_version); tor_free(global_dirfrontpagecontents); } -/** If options->SafeLogging is on, return a not very useful string, - * else return address. +/** Make <b>address</b> -- a piece of information related to our operation as + * a client -- safe to log according to the settings in options->SafeLogging, + * and return it. + * + * (We return "[scrubbed]" if SafeLogging is "1", and address otherwise.) + */ +const char * +safe_str_client(const char *address) +{ + tor_assert(address); + if (get_options()->_SafeLogging == SAFELOG_SCRUB_ALL) + return "[scrubbed]"; + else + return address; +} + +/** Make <b>address</b> -- a piece of information of unspecified sensitivity + * -- safe to log according to the settings in options->SafeLogging, and + * return it. + * + * (We return "[scrubbed]" if SafeLogging is anything besides "0", and address + * otherwise.) */ const char * safe_str(const char *address) { tor_assert(address); - if (get_options()->SafeLogging) + if (get_options()->_SafeLogging != SAFELOG_SCRUB_NONE) return "[scrubbed]"; else return address; } +/** Equivalent to escaped(safe_str_client(address)). See reentrancy note on + * escaped(): don't use this outside the main thread, or twice in the same + * log statement. */ +const char * +escaped_safe_str_client(const char *address) +{ + if (get_options()->_SafeLogging == SAFELOG_SCRUB_ALL) + return "[scrubbed]"; + else + return escaped(address); +} + /** Equivalent to escaped(safe_str(address)). See reentrancy note on * escaped(): don't use this outside the main thread, or twice in the same * log statement. */ const char * escaped_safe_str(const char *address) { - if (get_options()->SafeLogging) + if (get_options()->_SafeLogging != SAFELOG_SCRUB_NONE) return "[scrubbed]"; else return escaped(address); @@ -1089,10 +972,12 @@ options_act_reversible(or_options_t *old_options, char **msg) } /* Launch the listeners. (We do this before we setuid, so we can bind to - * ports under 1024.) */ - if (retry_all_listeners(replaced_listeners, new_listeners) < 0) { - *msg = tor_strdup("Failed to bind one of the listener ports."); - goto rollback; + * ports under 1024.) We don't want to rebind if we're hibernating. */ + if (!we_are_hibernating()) { + if (retry_all_listeners(replaced_listeners, new_listeners) < 0) { + *msg = tor_strdup("Failed to bind one of the listener ports."); + goto rollback; + } } } @@ -1106,6 +991,15 @@ options_act_reversible(or_options_t *old_options, char **msg) } #endif + /* Attempt to lock all current and future memory with mlockall() only once */ + if (options->DisableAllSwap) { + if (tor_mlockall() == -1) { + *msg = tor_strdup("DisableAllSwap failure. Do you have proper " + "permissions?"); + goto done; + } + } + /* Setuid/setgid as appropriate */ if (options->User) { if (switch_id(options->User) != 0) { @@ -1118,11 +1012,9 @@ options_act_reversible(or_options_t *old_options, char **msg) /* Ensure data directory is private; create if possible. */ if (check_private_dir(options->DataDirectory, running_tor ? CPD_CREATE : CPD_CHECK)<0) { - char buf[1024]; - int tmp = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Couldn't access/create private data directory \"%s\"", options->DataDirectory); - *msg = tor_strdup(tmp >= 0 ? buf : "internal error"); goto done; /* No need to roll back, since you can't change the value. */ } @@ -1133,10 +1025,8 @@ options_act_reversible(or_options_t *old_options, char **msg) tor_snprintf(fn, len, "%s"PATH_SEPARATOR"cached-status", options->DataDirectory); if (check_private_dir(fn, running_tor ? CPD_CREATE : CPD_CHECK) < 0) { - char buf[1024]; - int tmp = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Couldn't access/create private data directory \"%s\"", fn); - *msg = tor_strdup(tmp >= 0 ? buf : "internal error"); tor_free(fn); goto done; } @@ -1236,7 +1126,6 @@ get_effective_bwrate(or_options_t *options) bw = options->MaxAdvertisedBandwidth; if (options->RelayBandwidthRate > 0 && bw > options->RelayBandwidthRate) bw = options->RelayBandwidthRate; - /* ensure_bandwidth_cap() makes sure that this cast can't overflow. */ return (uint32_t)bw; } @@ -1314,14 +1203,14 @@ options_act(or_options_t *old_options) return 0; /* Finish backgrounding the process */ - if (running_tor && options->RunAsDaemon) { + if (options->RunAsDaemon) { /* We may be calling this for the n'th time (on SIGHUP), but it's safe. */ finish_daemon(options->DataDirectory); } /* Write our PID to the PID file. If we do not have write permissions we * will log a warning */ - if (running_tor && options->PidFile) + if (options->PidFile) write_pidfile(options->PidFile); /* Register addressmap directives */ @@ -1354,30 +1243,75 @@ options_act(or_options_t *old_options) if (accounting_is_enabled(options)) configure_accounting(time(NULL)); + /* parse RefuseUnknownExits tristate */ + if (!strcmp(options->RefuseUnknownExits, "0")) + options->RefuseUnknownExits_ = 0; + else if (!strcmp(options->RefuseUnknownExits, "1")) + options->RefuseUnknownExits_ = 1; + else if (!strcmp(options->RefuseUnknownExits, "auto")) + options->RefuseUnknownExits_ = -1; + else { + /* Should have caught this in options_validate */ + return -1; + } + + /* Change the cell EWMA settings */ + cell_ewma_set_scale_factor(options, networkstatus_get_latest_consensus()); + /* Check for transitions that need action. */ if (old_options) { - if (options->UseEntryGuards && !old_options->UseEntryGuards) { + if ((options->UseEntryGuards && !old_options->UseEntryGuards) || + !routerset_equal(old_options->ExcludeNodes,options->ExcludeNodes) || + !routerset_equal(old_options->ExcludeExitNodes, + options->ExcludeExitNodes) || + !routerset_equal(old_options->EntryNodes, options->EntryNodes) || + !routerset_equal(old_options->ExitNodes, options->ExitNodes) || + options->StrictNodes != old_options->StrictNodes) { log_info(LD_CIRC, - "Switching to entry guards; abandoning previous circuits"); + "Changed to using entry guards, or changed preferred or " + "excluded node lists. Abandoning previous circuits."); circuit_mark_all_unused_circs(); circuit_expire_all_dirty_circs(); + addressmap_clear_excluded_trackexithosts(options); } +/* How long should we delay counting bridge stats after becoming a bridge? + * We use this so we don't count people who used our bridge thinking it is + * a relay. If you change this, don't forget to change the log message + * below. It's 4 hours (the time it takes to stop being used by clients) + * plus some extra time for clock skew. */ +#define RELAY_BRIDGE_STATS_DELAY (6 * 60 * 60) + if (! bool_eq(options->BridgeRelay, old_options->BridgeRelay)) { - log_info(LD_GENERAL, "Bridge status changed. Forgetting GeoIP stats."); - geoip_remove_old_clients(time(NULL)+(2*60*60)); + int was_relay = 0; + if (options->BridgeRelay) { + time_t int_start = time(NULL); + if (old_options->ORPort == options->ORPort) { + int_start += RELAY_BRIDGE_STATS_DELAY; + was_relay = 1; + } + geoip_bridge_stats_init(int_start); + log_info(LD_CONFIG, "We are acting as a bridge now. Starting new " + "GeoIP stats interval%s.", was_relay ? " in 6 " + "hours from now" : ""); + } else { + geoip_bridge_stats_term(); + log_info(LD_GENERAL, "We are no longer acting as a bridge. " + "Forgetting GeoIP stats."); + } } if (options_transition_affects_workers(old_options, options)) { log_info(LD_GENERAL, "Worker-related options changed. Rotating workers."); + + if (init_keys() < 0) { + log_warn(LD_BUG,"Error initializing keys; exiting"); + return -1; + } if (server_mode(options) && !server_mode(old_options)) { - if (init_keys() < 0) { - log_warn(LD_BUG,"Error initializing keys; exiting"); - return -1; - } ip_address_changed(0); - if (has_completed_circuit || !any_predicted_circuits(time(NULL))) + if (can_complete_circuit || !any_predicted_circuits(time(NULL))) inform_testing_reachability(); } cpuworkers_rotate(); @@ -1390,6 +1324,10 @@ options_act(or_options_t *old_options) if (options->V3AuthoritativeDir && !old_options->V3AuthoritativeDir) init_keys(); + + if (options->PerConnBWRate != old_options->PerConnBWRate || + options->PerConnBWBurst != old_options->PerConnBWBurst) + connection_or_update_token_buckets(get_connection_array(), options); } /* Maybe load geoip file */ @@ -1398,7 +1336,7 @@ options_act(or_options_t *old_options) || !geoip_is_loaded())) { /* XXXX Don't use this "<default>" junk; make our filename options * understand prefixes somehow. -NM */ - /* XXXX021 Reload GeoIPFile on SIGHUP. -NM */ + /* XXXX023 Reload GeoIPFile on SIGHUP. -NM */ char *actual_fname = tor_strdup(options->GeoIPFile); #ifdef WIN32 if (!strcmp(actual_fname, "<default>")) { @@ -1412,17 +1350,68 @@ options_act(or_options_t *old_options) geoip_load_file(actual_fname, options); tor_free(actual_fname); } -#ifdef ENABLE_GEOIP_STATS - log_warn(LD_CONFIG, "We are configured to measure GeoIP statistics, but " - "the way these statistics are measured has changed " - "significantly in later versions of Tor. The results may not be " - "as expected if you are used to later versions. Be sure you " - "know what you are doing."); -#endif + + if (options->DirReqStatistics && !geoip_is_loaded()) { + /* Check if GeoIP database could be loaded. */ + log_warn(LD_CONFIG, "Configured to measure directory request " + "statistics, but no GeoIP database found!"); + return -1; + } + + if (options->EntryStatistics) { + if (should_record_bridge_info(options)) { + /* Don't allow measuring statistics on entry guards when configured + * as bridge. */ + log_warn(LD_CONFIG, "Bridges cannot be configured to measure " + "additional GeoIP statistics as entry guards."); + return -1; + } else if (!geoip_is_loaded()) { + /* Check if GeoIP database could be loaded. */ + log_warn(LD_CONFIG, "Configured to measure entry node statistics, " + "but no GeoIP database found!"); + return -1; + } + } + + if (options->CellStatistics || options->DirReqStatistics || + options->EntryStatistics || options->ExitPortStatistics) { + time_t now = time(NULL); + if ((!old_options || !old_options->CellStatistics) && + options->CellStatistics) + rep_hist_buffer_stats_init(now); + if ((!old_options || !old_options->DirReqStatistics) && + options->DirReqStatistics) + geoip_dirreq_stats_init(now); + if ((!old_options || !old_options->EntryStatistics) && + options->EntryStatistics) + geoip_entry_stats_init(now); + if ((!old_options || !old_options->ExitPortStatistics) && + options->ExitPortStatistics) + rep_hist_exit_stats_init(now); + if (!old_options) + log_notice(LD_CONFIG, "Configured to measure statistics. Look for " + "the *-stats files that will first be written to the " + "data directory in 24 hours from now."); + } + + if (old_options && old_options->CellStatistics && + !options->CellStatistics) + rep_hist_buffer_stats_term(); + if (old_options && old_options->DirReqStatistics && + !options->DirReqStatistics) + geoip_dirreq_stats_term(); + if (old_options && old_options->EntryStatistics && + !options->EntryStatistics) + geoip_entry_stats_term(); + if (old_options && old_options->ExitPortStatistics && + !options->ExitPortStatistics) + rep_hist_exit_stats_term(); + /* Check if we need to parse and add the EntryNodes config option. */ if (options->EntryNodes && (!old_options || - (!routerset_equal(old_options->EntryNodes,options->EntryNodes)))) + !routerset_equal(old_options->EntryNodes,options->EntryNodes) || + !routerset_equal(old_options->ExcludeNodes,options->ExcludeNodes))) entry_nodes_should_be_added(); /* Since our options changed, we might need to regenerate and upload our @@ -1491,7 +1480,10 @@ expand_abbrev(config_format_t *fmt, const char *option, int command_line, fmt->abbrevs[i].abbreviated, fmt->abbrevs[i].full); } - return fmt->abbrevs[i].full; + /* Keep going through the list in case we want to rewrite it more. + * (We could imagine recursing here, but I don't want to get the + * user into an infinite loop if we craft our list wrong.) */ + option = fmt->abbrevs[i].full; } } return option; @@ -1536,7 +1528,10 @@ config_get_commandlines(int argc, char **argv, config_line_t **result) *new = tor_malloc_zero(sizeof(config_line_t)); s = argv[i]; - while (*s == '-') + /* Each keyword may be prefixed with one or two dashes. */ + if (*s == '-') + s++; + if (*s == '-') s++; (*new)->key = tor_strdup(expand_abbrev(&options_format, s, 1, 1)); @@ -1628,19 +1623,6 @@ config_free_lines(config_line_t *front) } } -/** Return the description for a given configuration variable, or NULL if no - * description exists. */ -static const char * -config_find_description(config_format_t *fmt, const char *name) -{ - int i; - for (i=0; fmt->descriptions[i].name; ++i) { - if (!strcasecmp(name, fmt->descriptions[i].name)) - return fmt->descriptions[i].description; - } - return NULL; -} - /** If <b>key</b> is a configuration option, return the corresponding * config_var_t. Otherwise, if <b>key</b> is a non-standard abbreviation, * warn, and return the corresponding config_var_t. Otherwise return NULL. @@ -1671,6 +1653,16 @@ config_find_option(config_format_t *fmt, const char *key) return NULL; } +/** Return the number of option entries in <b>fmt</b>. */ +static int +config_count_options(config_format_t *fmt) +{ + int i; + for (i=0; fmt->vars[i].name; ++i) + ; + return i; +} + /* * Functions to assign config options. */ @@ -1684,8 +1676,7 @@ static int config_assign_value(config_format_t *fmt, or_options_t *options, config_line_t *c, char **msg) { - int i, r, ok; - char buf[1024]; + int i, ok; config_var_t *var; void *lvalue; @@ -1701,10 +1692,9 @@ config_assign_value(config_format_t *fmt, or_options_t *options, case CONFIG_TYPE_UINT: i = (int)tor_parse_long(c->value, 10, 0, INT_MAX, &ok, NULL); if (!ok) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Int keyword '%s %s' is malformed or out of bounds.", c->key, c->value); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } *(int *)lvalue = i; @@ -1713,10 +1703,9 @@ config_assign_value(config_format_t *fmt, or_options_t *options, case CONFIG_TYPE_INTERVAL: { i = config_parse_interval(c->value, &ok); if (!ok) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Interval '%s %s' is malformed or out of bounds.", c->key, c->value); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } *(int *)lvalue = i; @@ -1726,10 +1715,9 @@ config_assign_value(config_format_t *fmt, or_options_t *options, case CONFIG_TYPE_MEMUNIT: { uint64_t u64 = config_parse_memunit(c->value, &ok); if (!ok) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Value '%s %s' is malformed or out of bounds.", c->key, c->value); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } *(uint64_t *)lvalue = u64; @@ -1739,10 +1727,9 @@ config_assign_value(config_format_t *fmt, or_options_t *options, case CONFIG_TYPE_BOOL: i = (int)tor_parse_long(c->value, 10, 0, 1, &ok, NULL); if (!ok) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Boolean '%s %s' expects 0 or 1.", c->key, c->value); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } *(int *)lvalue = i; @@ -1760,9 +1747,8 @@ config_assign_value(config_format_t *fmt, or_options_t *options, case CONFIG_TYPE_ISOTIME: if (parse_iso_time(c->value, (time_t *)lvalue)) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Invalid time '%s' for keyword '%s'", c->value, c->key); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } break; @@ -1773,9 +1759,8 @@ config_assign_value(config_format_t *fmt, or_options_t *options, } *(routerset_t**)lvalue = routerset_new(); if (routerset_parse(*(routerset_t**)lvalue, c->value, c->key)<0) { - tor_snprintf(buf, sizeof(buf), "Invalid exit list '%s' for option '%s'", + tor_asprintf(msg, "Invalid exit list '%s' for option '%s'", c->value, c->key); - *msg = tor_strdup(buf); return -1; } break; @@ -1800,9 +1785,8 @@ config_assign_value(config_format_t *fmt, or_options_t *options, log_warn(LD_CONFIG, "Skipping obsolete configuration option '%s'", c->key); break; case CONFIG_TYPE_LINELIST_V: - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "You may not provide a value for virtual option '%s'", c->key); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; default: tor_assert(0); @@ -1823,7 +1807,7 @@ config_assign_value(config_format_t *fmt, or_options_t *options, static int config_assign_line(config_format_t *fmt, or_options_t *options, config_line_t *c, int use_defaults, - int clear_first, char **msg) + int clear_first, bitarray_t *options_seen, char **msg) { config_var_t *var; @@ -1838,13 +1822,12 @@ config_assign_line(config_format_t *fmt, or_options_t *options, config_line_append((config_line_t**)lvalue, c->key, c->value); return 0; } else { - char buf[1024]; - int tmp = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Unknown option '%s'. Failing.", c->key); - *msg = tor_strdup(tmp >= 0 ? buf : "internal error"); return -1; } } + /* Put keyword into canonical case. */ if (strcmp(var->name, c->key)) { tor_free(c->key); @@ -1867,6 +1850,18 @@ config_assign_line(config_format_t *fmt, or_options_t *options, return 0; } + if (options_seen && (var->type != CONFIG_TYPE_LINELIST && + var->type != CONFIG_TYPE_LINELIST_S)) { + /* We're tracking which options we've seen, and this option is not + * supposed to occur more than once. */ + int var_index = (int)(var - fmt->vars); + if (bitarray_is_set(options_seen, var_index)) { + log_warn(LD_CONFIG, "Option '%s' used more than once; all but the last " + "value will be ignored.", var->name); + } + bitarray_set(options_seen, var_index); + } + if (config_assign_value(fmt, options, c, msg) < 0) return -2; return 0; @@ -1967,7 +1962,6 @@ get_assigned_option(config_format_t *fmt, void *options, { config_var_t *var; const void *value; - char buf[32]; config_line_t *result; tor_assert(options && key); @@ -2008,19 +2002,16 @@ get_assigned_option(config_format_t *fmt, void *options, case CONFIG_TYPE_UINT: /* This means every or_options_t uint or bool element * needs to be an int. Not, say, a uint16_t or char. */ - tor_snprintf(buf, sizeof(buf), "%d", *(int*)value); - result->value = tor_strdup(buf); + tor_asprintf(&result->value, "%d", *(int*)value); escape_val = 0; /* Can't need escape. */ break; case CONFIG_TYPE_MEMUNIT: - tor_snprintf(buf, sizeof(buf), U64_FORMAT, + tor_asprintf(&result->value, U64_FORMAT, U64_PRINTF_ARG(*(uint64_t*)value)); - result->value = tor_strdup(buf); escape_val = 0; /* Can't need escape. */ break; case CONFIG_TYPE_DOUBLE: - tor_snprintf(buf, sizeof(buf), "%f", *(double*)value); - result->value = tor_strdup(buf); + tor_asprintf(&result->value, "%f", *(double*)value); escape_val = 0; /* Can't need escape. */ break; case CONFIG_TYPE_BOOL: @@ -2139,6 +2130,8 @@ config_assign(config_format_t *fmt, void *options, config_line_t *list, int use_defaults, int clear_first, char **msg) { config_line_t *p; + bitarray_t *options_seen; + const int n_options = config_count_options(fmt); CHECK(fmt, options); @@ -2158,14 +2151,18 @@ config_assign(config_format_t *fmt, void *options, config_line_t *list, config_reset_line(fmt, options, p->key, use_defaults); } + options_seen = bitarray_init_zero(n_options); /* pass 3: assign. */ while (list) { int r; if ((r=config_assign_line(fmt, options, list, use_defaults, - clear_first, msg))) + clear_first, options_seen, msg))) { + bitarray_free(options_seen); return r; + } list = list->next; } + bitarray_free(options_seen); return 0; } @@ -2308,20 +2305,10 @@ list_torrc_options(void) smartlist_t *lines = smartlist_create(); for (i = 0; _option_vars[i].name; ++i) { config_var_t *var = &_option_vars[i]; - const char *desc; if (var->type == CONFIG_TYPE_OBSOLETE || var->type == CONFIG_TYPE_LINELIST_V) continue; - desc = config_find_description(&options_format, var->name); printf("%s\n", var->name); - if (desc) { - wrap_string(lines, desc, 76, " ", " "); - SMARTLIST_FOREACH(lines, char *, cp, { - printf("%s", cp); - tor_free(cp); - }); - smartlist_clear(lines); - } } smartlist_free(lines); } @@ -2340,7 +2327,7 @@ resolve_my_address(int warn_severity, or_options_t *options, uint32_t *addr_out, char **hostname_out) { struct in_addr in; - uint32_t addr; + uint32_t addr; /* host order */ char hostname[256]; int explicit_ip=1; int explicit_hostname=1; @@ -2370,8 +2357,8 @@ resolve_my_address(int warn_severity, or_options_t *options, if (tor_inet_aton(hostname, &in) == 0) { /* then we have to resolve it */ explicit_ip = 0; - if (tor_lookup_hostname(hostname, &addr)) { - uint32_t interface_ip; + if (tor_lookup_hostname(hostname, &addr)) { /* failed to resolve */ + uint32_t interface_ip; /* host order */ if (explicit_hostname) { log_fn(warn_severity, LD_CONFIG, @@ -2392,7 +2379,7 @@ resolve_my_address(int warn_severity, or_options_t *options, log_fn(notice_severity, LD_CONFIG, "Learned IP address '%s' for " "local interface. Using that.", tmpbuf); strlcpy(hostname, "<guessed from interfaces>", sizeof(hostname)); - } else { + } else { /* resolved hostname into addr */ in.s_addr = htonl(addr); if (!explicit_hostname && @@ -2495,7 +2482,7 @@ is_local_addr(const tor_addr_t *addr) if (get_options()->EnforceDistinctSubnets == 0) return 0; if (tor_addr_family(addr) == AF_INET) { - /*XXXX022 IP6 what corresponds to an /24? */ + /*XXXX023 IP6 what corresponds to an /24? */ uint32_t ip = tor_addr_to_ipv4h(addr); /* It's possible that this next check will hit before the first time @@ -2512,61 +2499,16 @@ is_local_addr(const tor_addr_t *addr) return 0; } -/** Called when we don't have a nickname set. Try to guess a good nickname - * based on the hostname, and return it in a newly allocated string. If we - * can't, return NULL and let the caller warn if it wants to. */ -static char * -get_default_nickname(void) -{ - static const char * const bad_default_nicknames[] = { - "localhost", - NULL, - }; - char localhostname[256]; - char *cp, *out, *outp; - int i; - - if (gethostname(localhostname, sizeof(localhostname)) < 0) - return NULL; - - /* Put it in lowercase; stop at the first dot. */ - if ((cp = strchr(localhostname, '.'))) - *cp = '\0'; - tor_strlower(localhostname); - - /* Strip invalid characters. */ - cp = localhostname; - out = outp = tor_malloc(strlen(localhostname) + 1); - while (*cp) { - if (strchr(LEGAL_NICKNAME_CHARACTERS, *cp)) - *outp++ = *cp++; - else - cp++; - } - *outp = '\0'; - - /* Enforce length. */ - if (strlen(out) > MAX_NICKNAME_LEN) - out[MAX_NICKNAME_LEN]='\0'; - - /* Check for dumb names. */ - for (i = 0; bad_default_nicknames[i]; ++i) { - if (!strcmp(out, bad_default_nicknames[i])) { - tor_free(out); - return NULL; - } - } - - return out; -} - /** Release storage held by <b>options</b>. */ static void config_free(config_format_t *fmt, void *options) { int i; - tor_assert(options); + if (!options) + return; + + tor_assert(fmt); for (i=0; fmt->vars[i].name; ++i) option_clear(fmt, options, &(fmt->vars[i])); @@ -2668,6 +2610,8 @@ is_listening_on_low_port(uint16_t port_option, const config_line_t *listen_options) { #ifdef MS_WINDOWS + (void) port_option; + (void) listen_options; return 0; /* No port is too low for windows. */ #else const config_line_t *l; @@ -2717,7 +2661,6 @@ config_dump(config_format_t *fmt, void *options, int minimal, config_line_t *line, *assigned; char *result; int i; - const char *desc; char *msg = NULL; defaults = config_alloc(fmt); @@ -2745,24 +2688,13 @@ config_dump(config_format_t *fmt, void *options, int minimal, option_is_same(fmt, options, defaults, fmt->vars[i].name)) comment_option = 1; - desc = config_find_description(fmt, fmt->vars[i].name); line = assigned = get_assigned_option(fmt, options, fmt->vars[i].name, 1); - if (line && desc) { - /* Only dump the description if there's something to describe. */ - wrap_string(elements, desc, 78, "# ", "# "); - } - for (; line; line = line->next) { - size_t len = strlen(line->key) + strlen(line->value) + 5; char *tmp; - tmp = tor_malloc(len); - if (tor_snprintf(tmp, len, "%s%s %s\n", - comment_option ? "# " : "", - line->key, line->value)<0) { - log_err(LD_BUG,"Internal error writing option value"); - tor_assert(0); - } + tor_asprintf(&tmp, "%s%s %s\n", + comment_option ? "# " : "", + line->key, line->value); smartlist_add(elements, tmp); } config_free_lines(assigned); @@ -2771,13 +2703,8 @@ config_dump(config_format_t *fmt, void *options, int minimal, if (fmt->extra) { line = *(config_line_t**)STRUCT_VAR_P(options, fmt->extra->var_offset); for (; line; line = line->next) { - size_t len = strlen(line->key) + strlen(line->value) + 3; char *tmp; - tmp = tor_malloc(len); - if (tor_snprintf(tmp, len, "%s %s\n", line->key, line->value)<0) { - log_err(LD_BUG,"Internal error writing option value"); - tor_assert(0); - } + tor_asprintf(&tmp, "%s %s\n", line->key, line->value); smartlist_add(elements, tmp); } } @@ -2793,7 +2720,7 @@ config_dump(config_format_t *fmt, void *options, int minimal, * the configuration in <b>options</b>. If <b>minimal</b> is true, do not * include options that are the same as Tor's defaults. */ -static char * +char * options_dump(or_options_t *options, int minimal) { return config_dump(&options_format, options, minimal, 0); @@ -2806,7 +2733,6 @@ static int validate_ports_csv(smartlist_t *sl, const char *name, char **msg) { int i; - char buf[1024]; tor_assert(name); if (!sl) @@ -2816,9 +2742,7 @@ validate_ports_csv(smartlist_t *sl, const char *name, char **msg) { i = atoi(cp); if (i < 1 || i > 65535) { - int r = tor_snprintf(buf, sizeof(buf), - "Port '%s' out of range in %s", cp, name); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); + tor_asprintf(msg, "Port '%s' out of range in %s", cp, name); return -1; } }); @@ -2832,18 +2756,15 @@ validate_ports_csv(smartlist_t *sl, const char *name, char **msg) static int ensure_bandwidth_cap(uint64_t *value, const char *desc, char **msg) { - int r; - char buf[1024]; if (*value > ROUTER_MAX_DECLARED_BANDWIDTH) { /* This handles an understandable special case where somebody says "2gb" * whereas our actual maximum is 2gb-1 (INT_MAX) */ --*value; } if (*value > ROUTER_MAX_DECLARED_BANDWIDTH) { - r = tor_snprintf(buf, sizeof(buf), "%s ("U64_FORMAT") must be at most %d", - desc, U64_PRINTF_ARG(*value), - ROUTER_MAX_DECLARED_BANDWIDTH); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); + tor_asprintf(msg, "%s ("U64_FORMAT") must be at most %d", + desc, U64_PRINTF_ARG(*value), + ROUTER_MAX_DECLARED_BANDWIDTH); return -1; } return 0; @@ -2896,15 +2817,14 @@ compute_publishserverdescriptor(or_options_t *options) /** Highest allowable value for RendPostPeriod. */ #define MAX_DIR_PERIOD (MIN_ONION_KEY_LIFETIME/2) -/** Lowest allowable value for CircuitBuildTimeout; values too low will - * increase network load because of failing connections being retried, and - * might prevent users from connecting to the network at all. */ -#define MIN_CIRCUIT_BUILD_TIMEOUT 30 - /** Lowest allowable value for MaxCircuitDirtiness; if this is too low, Tor * will generate too many circuits and potentially overload the network. */ #define MIN_MAX_CIRCUIT_DIRTINESS 10 +/** Lowest allowable value for CircuitStreamTimeout; if this is too low, Tor + * will generate too many circuits and potentially overload the network. */ +#define MIN_CIRCUIT_STREAM_TIMEOUT 10 + /** Return 0 if every setting in <b>options</b> is reasonable, and a * permissible transition from <b>old_options</b>. Else return -1. * Should have no side effects, except for normalizing the contents of @@ -2921,10 +2841,9 @@ static int options_validate(or_options_t *old_options, or_options_t *options, int from_setconf, char **msg) { - int i, r; + int i; config_line_t *cl; const char *uname = get_uname(); - char buf[1024]; #define REJECT(arg) \ STMT_BEGIN *msg = tor_strdup(arg); return -1; STMT_END #define COMPLAIN(arg) STMT_BEGIN log(LOG_WARN, LD_CONFIG, arg); STMT_END @@ -2941,7 +2860,7 @@ options_validate(or_options_t *old_options, or_options_t *options, !strcmpstart(uname, "Windows Me"))) { log(LOG_WARN, LD_CONFIG, "Tor is running as a server, but you are " "running %s; this probably won't work. See " - "http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#ServerOS " + "https://wiki.torproject.org/TheOnionRouter/TorFAQ#ServerOS " "for details.", uname); } @@ -2960,8 +2879,8 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->TransPort == 0 && options->TransListenAddress != NULL) REJECT("TransPort must be defined if TransListenAddress is defined."); - if (options->NatdPort == 0 && options->NatdListenAddress != NULL) - REJECT("NatdPort must be defined if NatdListenAddress is defined."); + if (options->NATDPort == 0 && options->NATDListenAddress != NULL) + REJECT("NATDPort must be defined if NATDListenAddress is defined."); /* Don't gripe about SocksPort 0 with SocksListenAddress set; a standard * configuration does this. */ @@ -2980,8 +2899,8 @@ options_validate(or_options_t *old_options, or_options_t *options, old = old_options ? old_options->TransListenAddress : NULL; tp = "transparent proxy"; } else { - opt = options->NatdListenAddress; - old = old_options ? old_options->NatdListenAddress : NULL; + opt = options->NATDListenAddress; + old = old_options ? old_options->NATDListenAddress : NULL; tp = "natd proxy"; } @@ -3008,21 +2927,13 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->Nickname == NULL) { if (server_mode(options)) { - if (!(options->Nickname = get_default_nickname())) { - log_notice(LD_CONFIG, "Couldn't pick a nickname based on " - "our hostname; using %s instead.", UNNAMED_ROUTER_NICKNAME); options->Nickname = tor_strdup(UNNAMED_ROUTER_NICKNAME); - } else { - log_notice(LD_CONFIG, "Choosing default nickname '%s'", - options->Nickname); - } } } else { if (!is_legal_nickname(options->Nickname)) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Nickname '%s' is wrong length or contains illegal characters.", options->Nickname); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } } @@ -3039,14 +2950,6 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options_init_logs(options, 1)<0) /* Validate the log(s) */ REJECT("Failed to validate Log options. See logs for details."); - if (options->NoPublish) { - log(LOG_WARN, LD_CONFIG, - "NoPublish is obsolete. Use PublishServerDescriptor instead."); - SMARTLIST_FOREACH(options->PublishServerDescriptor, char *, s, - tor_free(s)); - smartlist_clear(options->PublishServerDescriptor); - } - if (authdir_mode(options)) { /* confirm that our address isn't broken, so we can complain now */ uint32_t tmp; @@ -3054,6 +2957,12 @@ options_validate(or_options_t *old_options, or_options_t *options, REJECT("Failed to resolve/guess local address. See logs for details."); } + if (strcmp(options->RefuseUnknownExits, "0") && + strcmp(options->RefuseUnknownExits, "1") && + strcmp(options->RefuseUnknownExits, "auto")) { + REJECT("RefuseUnknownExits must be 0, 1, or auto"); + } + #ifndef MS_WINDOWS if (options->RunAsDaemon && torrc_fname && path_is_relative(torrc_fname)) REJECT("Can't use a relative path to torrc when RunAsDaemon is set."); @@ -3068,14 +2977,14 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->TransPort < 0 || options->TransPort > 65535) REJECT("TransPort option out of bounds."); - if (options->NatdPort < 0 || options->NatdPort > 65535) - REJECT("NatdPort option out of bounds."); + if (options->NATDPort < 0 || options->NATDPort > 65535) + REJECT("NATDPort option out of bounds."); if (options->SocksPort == 0 && options->TransPort == 0 && - options->NatdPort == 0 && options->ORPort == 0 && + options->NATDPort == 0 && options->ORPort == 0 && options->DNSPort == 0 && !options->RendConfigLines) log(LOG_WARN, LD_CONFIG, - "SocksPort, TransPort, NatdPort, DNSPort, and ORPort are all " + "SocksPort, TransPort, NATDPort, DNSPort, and ORPort are all " "undefined, and there aren't any hidden services configured. " "Tor will still run, but probably won't do anything."); @@ -3109,19 +3018,11 @@ options_validate(or_options_t *old_options, or_options_t *options, routerset_union(options->_ExcludeExitNodesUnion,options->ExcludeNodes); } - if (options->StrictExitNodes && - (!options->ExitNodes) && - (!old_options || - (old_options->StrictExitNodes != options->StrictExitNodes) || - (!routerset_equal(old_options->ExitNodes,options->ExitNodes)))) - COMPLAIN("StrictExitNodes set, but no ExitNodes listed."); - - if (options->StrictEntryNodes && - (!options->EntryNodes) && - (!old_options || - (old_options->StrictEntryNodes != options->StrictEntryNodes) || - (!routerset_equal(old_options->EntryNodes,options->EntryNodes)))) - COMPLAIN("StrictEntryNodes set, but no EntryNodes listed."); + if (options->ExcludeNodes && options->StrictNodes) { + COMPLAIN("You have asked to exclude certain relays from all positions " + "in your circuits. Expect hidden services and other Tor " + "features to be broken in unpredictable ways."); + } if (options->EntryNodes && !routerset_is_list(options->EntryNodes)) { /* XXXX fix this; see entry_guards_prepend_from_config(). */ @@ -3159,6 +3060,10 @@ options_validate(or_options_t *old_options, or_options_t *options, options->V3AuthoritativeDir)) REJECT("AuthoritativeDir is set, but none of " "(Bridge/HS/V1/V2/V3)AuthoritativeDir is set."); + /* If we have a v3bandwidthsfile and it's broken, complain on startup */ + if (options->V3BandwidthsFile && !old_options) { + dirserv_read_measured_bandwidths(options->V3BandwidthsFile, NULL); + } } if (options->AuthoritativeDir && !options->DirPort) @@ -3170,15 +3075,18 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->AuthoritativeDir && options->ClientOnly) REJECT("Running as authoritative directory, but ClientOnly also set."); - if (options->HSAuthorityRecordStats && !options->HSAuthoritativeDir) - REJECT("HSAuthorityRecordStats is set but we're not running as " - "a hidden service authority."); + if (options->FetchDirInfoExtraEarly && !options->FetchDirInfoEarly) + REJECT("FetchDirInfoExtraEarly requires that you also set " + "FetchDirInfoEarly"); + + if (options->HSAuthoritativeDir && proxy_mode(options)) + REJECT("Running as authoritative v0 HS directory, but also configured " + "as a client."); if (options->ConnLimit <= 0) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "ConnLimit must be greater than 0, but was set to %d", options->ConnLimit); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } @@ -3283,6 +3191,12 @@ options_validate(or_options_t *old_options, or_options_t *options, REJECT("Servers must be able to freely connect to the rest " "of the Internet, so they must not set UseBridges."); + /* If both of these are set, we'll end up with funny behavior where we + * demand enough entrynodes be up and running else we won't build + * circuits, yet we never actually use them. */ + if (options->UseBridges && options->EntryNodes) + REJECT("You cannot set both UseBridges and EntryNodes."); + options->_AllowInvalid = 0; if (options->AllowInvalidNodes) { SMARTLIST_FOREACH(options->AllowInvalidNodes, const char *, cp, { @@ -3297,18 +3211,29 @@ options_validate(or_options_t *old_options, or_options_t *options, else if (!strcasecmp(cp, "rendezvous")) options->_AllowInvalid |= ALLOW_INVALID_RENDEZVOUS; else { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "Unrecognized value '%s' in AllowInvalidNodes", cp); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } }); } + if (!options->SafeLogging || + !strcasecmp(options->SafeLogging, "0")) { + options->_SafeLogging = SAFELOG_SCRUB_NONE; + } else if (!strcasecmp(options->SafeLogging, "relay")) { + options->_SafeLogging = SAFELOG_SCRUB_RELAY; + } else if (!strcasecmp(options->SafeLogging, "1")) { + options->_SafeLogging = SAFELOG_SCRUB_ALL; + } else { + tor_asprintf(msg, + "Unrecognized value '%s' in SafeLogging", + escaped(options->SafeLogging)); + return -1; + } + if (compute_publishserverdescriptor(options) < 0) { - r = tor_snprintf(buf, sizeof(buf), - "Unrecognized value in PublishServerDescriptor"); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); + tor_asprintf(msg, "Unrecognized value in PublishServerDescriptor"); return -1; } @@ -3321,6 +3246,12 @@ options_validate(or_options_t *old_options, or_options_t *options, "PublishServerDescriptor line."); } + if (options->BridgeRelay && options->DirPort) { + log_warn(LD_CONFIG, "Can't set a DirPort on a bridge relay; disabling " + "DirPort"); + options->DirPort = 0; + } + if (options->MinUptimeHidServDirectoryV2 < 0) { log_warn(LD_CONFIG, "MinUptimeHidServDirectoryV2 option must be at " "least 0 seconds. Changing to 0."); @@ -3328,29 +3259,30 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->RendPostPeriod < MIN_REND_POST_PERIOD) { - log(LOG_WARN,LD_CONFIG,"RendPostPeriod option is too short; " - "raising to %d seconds.", MIN_REND_POST_PERIOD); + log_warn(LD_CONFIG, "RendPostPeriod option is too short; " + "raising to %d seconds.", MIN_REND_POST_PERIOD); options->RendPostPeriod = MIN_REND_POST_PERIOD; } if (options->RendPostPeriod > MAX_DIR_PERIOD) { - log(LOG_WARN, LD_CONFIG, "RendPostPeriod is too large; clipping to %ds.", - MAX_DIR_PERIOD); + log_warn(LD_CONFIG, "RendPostPeriod is too large; clipping to %ds.", + MAX_DIR_PERIOD); options->RendPostPeriod = MAX_DIR_PERIOD; } - if (options->CircuitBuildTimeout < MIN_CIRCUIT_BUILD_TIMEOUT) { - log(LOG_WARN, LD_CONFIG, "CircuitBuildTimeout option is too short; " - "raising to %d seconds.", MIN_CIRCUIT_BUILD_TIMEOUT); - options->CircuitBuildTimeout = MIN_CIRCUIT_BUILD_TIMEOUT; - } - if (options->MaxCircuitDirtiness < MIN_MAX_CIRCUIT_DIRTINESS) { - log(LOG_WARN, LD_CONFIG, "MaxCircuitDirtiness option is too short; " - "raising to %d seconds.", MIN_MAX_CIRCUIT_DIRTINESS); + log_warn(LD_CONFIG, "MaxCircuitDirtiness option is too short; " + "raising to %d seconds.", MIN_MAX_CIRCUIT_DIRTINESS); options->MaxCircuitDirtiness = MIN_MAX_CIRCUIT_DIRTINESS; } + if (options->CircuitStreamTimeout && + options->CircuitStreamTimeout < MIN_CIRCUIT_STREAM_TIMEOUT) { + log_warn(LD_CONFIG, "CircuitStreamTimeout option is too short; " + "raising to %d seconds.", MIN_CIRCUIT_STREAM_TIMEOUT); + options->CircuitStreamTimeout = MIN_CIRCUIT_STREAM_TIMEOUT; + } + if (options->KeepalivePeriod < 1) REJECT("KeepalivePeriod option must be positive."); @@ -3369,6 +3301,12 @@ options_validate(or_options_t *old_options, or_options_t *options, if (ensure_bandwidth_cap(&options->RelayBandwidthBurst, "RelayBandwidthBurst", msg) < 0) return -1; + if (ensure_bandwidth_cap(&options->PerConnBWRate, + "PerConnBWRate", msg) < 0) + return -1; + if (ensure_bandwidth_cap(&options->PerConnBWBurst, + "PerConnBWBurst", msg) < 0) + return -1; if (options->RelayBandwidthRate && !options->RelayBandwidthBurst) options->RelayBandwidthBurst = options->RelayBandwidthRate; @@ -3377,31 +3315,28 @@ options_validate(or_options_t *old_options, or_options_t *options, if (server_mode(options)) { if (options->BandwidthRate < ROUTER_REQUIRED_MIN_BANDWIDTH) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "BandwidthRate is set to %d bytes/second. " "For servers, it must be at least %d.", (int)options->BandwidthRate, ROUTER_REQUIRED_MIN_BANDWIDTH); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } else if (options->MaxAdvertisedBandwidth < ROUTER_REQUIRED_MIN_BANDWIDTH/2) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "MaxAdvertisedBandwidth is set to %d bytes/second. " "For servers, it must be at least %d.", (int)options->MaxAdvertisedBandwidth, ROUTER_REQUIRED_MIN_BANDWIDTH/2); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } if (options->RelayBandwidthRate && options->RelayBandwidthRate < ROUTER_REQUIRED_MIN_BANDWIDTH) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "RelayBandwidthRate is set to %d bytes/second. " "For servers, it must be at least %d.", (int)options->RelayBandwidthRate, ROUTER_REQUIRED_MIN_BANDWIDTH); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } } @@ -3423,34 +3358,73 @@ options_validate(or_options_t *old_options, or_options_t *options, if (accounting_parse_options(options, 1)<0) REJECT("Failed to parse accounting options. See logs for details."); - if (options->HttpProxy) { /* parse it now */ - if (parse_addr_port(LOG_WARN, options->HttpProxy, NULL, - &options->HttpProxyAddr, &options->HttpProxyPort) < 0) - REJECT("HttpProxy failed to parse or resolve. Please fix."); - if (options->HttpProxyPort == 0) { /* give it a default */ - options->HttpProxyPort = 80; + if (options->HTTPProxy) { /* parse it now */ + if (tor_addr_port_parse(options->HTTPProxy, + &options->HTTPProxyAddr, &options->HTTPProxyPort) < 0) + REJECT("HTTPProxy failed to parse or resolve. Please fix."); + if (options->HTTPProxyPort == 0) { /* give it a default */ + options->HTTPProxyPort = 80; } } - if (options->HttpProxyAuthenticator) { - if (strlen(options->HttpProxyAuthenticator) >= 48) - REJECT("HttpProxyAuthenticator is too long (>= 48 chars)."); + if (options->HTTPProxyAuthenticator) { + if (strlen(options->HTTPProxyAuthenticator) >= 48) + REJECT("HTTPProxyAuthenticator is too long (>= 48 chars)."); } - if (options->HttpsProxy) { /* parse it now */ - if (parse_addr_port(LOG_WARN, options->HttpsProxy, NULL, - &options->HttpsProxyAddr, &options->HttpsProxyPort) <0) - REJECT("HttpsProxy failed to parse or resolve. Please fix."); - if (options->HttpsProxyPort == 0) { /* give it a default */ - options->HttpsProxyPort = 443; + if (options->HTTPSProxy) { /* parse it now */ + if (tor_addr_port_parse(options->HTTPSProxy, + &options->HTTPSProxyAddr, &options->HTTPSProxyPort) <0) + REJECT("HTTPSProxy failed to parse or resolve. Please fix."); + if (options->HTTPSProxyPort == 0) { /* give it a default */ + options->HTTPSProxyPort = 443; } } - if (options->HttpsProxyAuthenticator) { - if (strlen(options->HttpsProxyAuthenticator) >= 48) - REJECT("HttpsProxyAuthenticator is too long (>= 48 chars)."); + if (options->HTTPSProxyAuthenticator) { + if (strlen(options->HTTPSProxyAuthenticator) >= 48) + REJECT("HTTPSProxyAuthenticator is too long (>= 48 chars)."); } + if (options->Socks4Proxy) { /* parse it now */ + if (tor_addr_port_parse(options->Socks4Proxy, + &options->Socks4ProxyAddr, + &options->Socks4ProxyPort) <0) + REJECT("Socks4Proxy failed to parse or resolve. Please fix."); + if (options->Socks4ProxyPort == 0) { /* give it a default */ + options->Socks4ProxyPort = 1080; + } + } + + if (options->Socks5Proxy) { /* parse it now */ + if (tor_addr_port_parse(options->Socks5Proxy, + &options->Socks5ProxyAddr, + &options->Socks5ProxyPort) <0) + REJECT("Socks5Proxy failed to parse or resolve. Please fix."); + if (options->Socks5ProxyPort == 0) { /* give it a default */ + options->Socks5ProxyPort = 1080; + } + } + + if (options->Socks4Proxy && options->Socks5Proxy) + REJECT("You cannot specify both Socks4Proxy and SOCKS5Proxy"); + + if (options->Socks5ProxyUsername) { + size_t len; + + len = strlen(options->Socks5ProxyUsername); + if (len < 1 || len > 255) + REJECT("Socks5ProxyUsername must be between 1 and 255 characters."); + + if (!options->Socks5ProxyPassword) + REJECT("Socks5ProxyPassword must be included with Socks5ProxyUsername."); + + len = strlen(options->Socks5ProxyPassword); + if (len < 1 || len > 255) + REJECT("Socks5ProxyPassword must be between 1 and 255 characters."); + } else if (options->Socks5ProxyPassword) + REJECT("Socks5ProxyPassword must be included with Socks5ProxyUsername."); + if (options->HashedControlPassword) { smartlist_t *sl = decode_hashed_passwords(options->HashedControlPassword); if (!sl) { @@ -3514,6 +3488,12 @@ options_validate(or_options_t *old_options, or_options_t *options, "upgrade your Tor controller as soon as possible."); } + if (options->CookieAuthFileGroupReadable && !options->CookieAuthFile) { + log_warn(LD_CONFIG, "CookieAuthFileGroupReadable is set, but will have " + "no effect: you must specify an explicit CookieAuthFile to " + "have it group-readable."); + } + if (options->UseEntryGuards && ! options->NumEntryGuards) REJECT("Cannot enable UseEntryGuards with NumEntryGuards set to 0"); @@ -3547,11 +3527,10 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->ConstrainedSockSize < MIN_CONSTRAINED_TCP_BUFFER || options->ConstrainedSockSize > MAX_CONSTRAINED_TCP_BUFFER || options->ConstrainedSockSize % 1024) { - r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "ConstrainedSockSize is invalid. Must be a value between %d and %d " "in 1024 byte increments.", MIN_CONSTRAINED_TCP_BUFFER, MAX_CONSTRAINED_TCP_BUFFER); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } if (options->DirPort) { @@ -3599,6 +3578,12 @@ options_validate(or_options_t *old_options, or_options_t *options, if (options->PreferTunneledDirConns && !options->TunnelDirConns) REJECT("Must set TunnelDirConns if PreferTunneledDirConns is set."); + if ((options->Socks4Proxy || options->Socks5Proxy) && + !options->HTTPProxy && !options->PreferTunneledDirConns) + REJECT("When Socks4Proxy or Socks5Proxy is configured, " + "PreferTunneledDirConns and TunnelDirConns must both be " + "set to 1, or HTTPProxy must be configured."); + if (options->AutomapHostsSuffixes) { SMARTLIST_FOREACH(options->AutomapHostsSuffixes, char *, suf, { @@ -3613,12 +3598,19 @@ options_validate(or_options_t *old_options, or_options_t *options, "a non-default set of DirServers."); } - /*XXXX022 checking for defaults manually like this is a bit fragile.*/ + if (options->AllowSingleHopExits && !options->DirServers) { + COMPLAIN("You have set AllowSingleHopExits; now your relay will allow " + "others to make one-hop exits. However, since by default most " + "clients avoid relays that set this option, most clients will " + "ignore you."); + } + + /*XXXX023 checking for defaults manually like this is a bit fragile.*/ /* Keep changes to hard-coded values synchronous to man page and default * values table. */ if (options->TestingV3AuthInitialVotingInterval != 30*60 && - !options->TestingTorNetwork) { + !options->TestingTorNetwork && !options->_UsingTestNetworkDefaults) { REJECT("TestingV3AuthInitialVotingInterval may only be changed in testing " "Tor networks!"); } else if (options->TestingV3AuthInitialVotingInterval < MIN_VOTE_INTERVAL) { @@ -3629,7 +3621,8 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->TestingV3AuthInitialVoteDelay != 5*60 && - !options->TestingTorNetwork) { + !options->TestingTorNetwork && !options->_UsingTestNetworkDefaults) { + REJECT("TestingV3AuthInitialVoteDelay may only be changed in testing " "Tor networks!"); } else if (options->TestingV3AuthInitialVoteDelay < MIN_VOTE_SECONDS) { @@ -3637,7 +3630,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->TestingV3AuthInitialDistDelay != 5*60 && - !options->TestingTorNetwork) { + !options->TestingTorNetwork && !options->_UsingTestNetworkDefaults) { REJECT("TestingV3AuthInitialDistDelay may only be changed in testing " "Tor networks!"); } else if (options->TestingV3AuthInitialDistDelay < MIN_DIST_SECONDS) { @@ -3652,7 +3645,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->TestingAuthDirTimeToLearnReachability != 30*60 && - !options->TestingTorNetwork) { + !options->TestingTorNetwork && !options->_UsingTestNetworkDefaults) { REJECT("TestingAuthDirTimeToLearnReachability may only be changed in " "testing Tor networks!"); } else if (options->TestingAuthDirTimeToLearnReachability < 0) { @@ -3662,7 +3655,7 @@ options_validate(or_options_t *old_options, or_options_t *options, } if (options->TestingEstimatedDescriptorPropagationTime != 10*60 && - !options->TestingTorNetwork) { + !options->TestingTorNetwork && !options->_UsingTestNetworkDefaults) { REJECT("TestingEstimatedDescriptorPropagationTime may only be changed in " "testing Tor networks!"); } else if (options->TestingEstimatedDescriptorPropagationTime < 0) { @@ -3678,6 +3671,26 @@ options_validate(or_options_t *old_options, or_options_t *options, "testing Tor network!"); } + if (options->AccelName && !options->HardwareAccel) + options->HardwareAccel = 1; + if (options->AccelDir && !options->AccelName) + REJECT("Can't use hardware crypto accelerator dir without engine name."); + + if (options->PublishServerDescriptor) + SMARTLIST_FOREACH(options->PublishServerDescriptor, const char *, pubdes, { + if (!strcmp(pubdes, "1") || !strcmp(pubdes, "0")) + if (smartlist_len(options->PublishServerDescriptor) > 1) { + COMPLAIN("You have passed a list of multiple arguments to the " + "PublishServerDescriptor option that includes 0 or 1. " + "0 or 1 should only be used as the sole argument. " + "This configuration will be rejected in a future release."); + break; + } + }); + + if (options->BridgeRelay == 1 && options->ORPort == 0) + REJECT("BridgeRelay is 1, ORPort is 0. This is an invalid combination."); + return 0; #undef REJECT #undef COMPLAIN @@ -3716,12 +3729,10 @@ options_transition_allowed(or_options_t *old, or_options_t *new_val, } if (strcmp(old->DataDirectory,new_val->DataDirectory)!=0) { - char buf[1024]; - int r = tor_snprintf(buf, sizeof(buf), + tor_asprintf(msg, "While Tor is running, changing DataDirectory " "(\"%s\"->\"%s\") is not allowed.", old->DataDirectory, new_val->DataDirectory); - *msg = tor_strdup(r >= 0 ? buf : "internal error"); return -1; } @@ -3730,19 +3741,22 @@ options_transition_allowed(or_options_t *old, or_options_t *new_val, return -1; } - if (!opt_streq(old->Group, new_val->Group)) { - *msg = tor_strdup("While Tor is running, changing Group is not allowed."); + if ((old->HardwareAccel != new_val->HardwareAccel) + || !opt_streq(old->AccelName, new_val->AccelName) + || !opt_streq(old->AccelDir, new_val->AccelDir)) { + *msg = tor_strdup("While Tor is running, changing OpenSSL hardware " + "acceleration engine is not allowed."); return -1; } - if (old->HardwareAccel != new_val->HardwareAccel) { - *msg = tor_strdup("While Tor is running, changing HardwareAccel is " - "not allowed."); + if (old->TestingTorNetwork != new_val->TestingTorNetwork) { + *msg = tor_strdup("While Tor is running, changing TestingTorNetwork " + "is not allowed."); return -1; } - if (old->TestingTorNetwork != new_val->TestingTorNetwork) { - *msg = tor_strdup("While Tor is running, changing TestingTorNetwork " + if (old->DisableAllSwap != new_val->DisableAllSwap) { + *msg = tor_strdup("While Tor is running, changing DisableAllSwap " "is not allowed."); return -1; } @@ -3757,13 +3771,15 @@ options_transition_affects_workers(or_options_t *old_options, or_options_t *new_options) { if (!opt_streq(old_options->DataDirectory, new_options->DataDirectory) || - old_options->NumCpus != new_options->NumCpus || + old_options->NumCPUs != new_options->NumCPUs || old_options->ORPort != new_options->ORPort || old_options->ServerDNSSearchDomains != new_options->ServerDNSSearchDomains || old_options->SafeLogging != new_options->SafeLogging || old_options->ClientOnly != new_options->ClientOnly || - !config_lines_eq(old_options->Logs, new_options->Logs)) + public_server_mode(old_options) != public_server_mode(new_options) || + !config_lines_eq(old_options->Logs, new_options->Logs) || + old_options->LogMessageDomains != new_options->LogMessageDomains) return 1; /* Check whether log options match. */ @@ -3789,7 +3805,6 @@ options_transition_affects_descriptor(or_options_t *old_options, old_options->ORPort != new_options->ORPort || old_options->DirPort != new_options->DirPort || old_options->ClientOnly != new_options->ClientOnly || - old_options->NoPublish != new_options->NoPublish || old_options->_PublishServerDescriptor != new_options->_PublishServerDescriptor || get_effective_bwrate(old_options) != get_effective_bwrate(new_options) || @@ -3812,6 +3827,7 @@ get_windows_conf_root(void) { static int is_set = 0; static char path[MAX_PATH+1]; + TCHAR tpath[MAX_PATH] = {0}; LPITEMIDLIST idl; IMalloc *m; @@ -3829,7 +3845,7 @@ get_windows_conf_root(void) #define APPDATA_PATH CSIDL_APPDATA #endif if (!SUCCEEDED(SHGetSpecialFolderLocation(NULL, APPDATA_PATH, &idl))) { - GetCurrentDirectory(MAX_PATH, path); + getcwd(path,MAX_PATH); is_set = 1; log_warn(LD_CONFIG, "I couldn't find your application data folder: are you " @@ -3838,8 +3854,15 @@ get_windows_conf_root(void) return path; } /* Convert the path from an "ID List" (whatever that is!) to a path. */ - result = SHGetPathFromIDList(idl, path); - /* Now we need to free the */ + result = SHGetPathFromIDList(idl, tpath); +#ifdef UNICODE + wcstombs(path,tpath,MAX_PATH); +#else + strlcpy(path,tpath,sizeof(path)); +#endif + + /* Now we need to free the memory that the path-idl was stored in. In + * typical Windows fashion, we can't just call 'free()' on it. */ SHGetMalloc(&m); if (m) { m->lpVtbl->Free(m, idl); @@ -3887,10 +3910,7 @@ check_nickname_list(const char *lst, const char *name, char **msg) SMARTLIST_FOREACH(sl, const char *, s, { if (!is_legal_nickname_or_hexdigest(s)) { - char buf[1024]; - int tmp = tor_snprintf(buf, sizeof(buf), - "Invalid nickname '%s' in %s line", s, name); - *msg = tor_strdup(tmp >= 0 ? buf : "internal error"); + tor_asprintf(msg, "Invalid nickname '%s' in %s line", s, name); r = -1; break; } @@ -3914,13 +3934,7 @@ find_torrc_filename(int argc, char **argv, log(LOG_WARN, LD_CONFIG, "Duplicate -f options on command line."); tor_free(fname); } -#ifdef MS_WINDOWS - /* XXX one day we might want to extend expand_filename to work - * under Windows as well. */ - fname = tor_strdup(argv[i+1]); -#else fname = expand_filename(argv[i+1]); -#endif *using_default_torrc = 0; ++i; } else if (!strcmp(argv[i],"--ignore-missing-torrc")) { @@ -4026,6 +4040,12 @@ options_init_from_torrc(int argc, char **argv) printf("Tor version %s.\n",get_version()); exit(0); } + if (argc > 1 && (!strcmp(argv[1],"--digests"))) { + printf("Tor version %s.\n",get_version()); + printf("%s", libor_get_digests()); + printf("%s", tor_get_digests()); + exit(0); + } /* Go through command-line variables */ if (!global_cmdline_options) { @@ -4193,12 +4213,9 @@ options_init_from_string(const char *cf, err: config_free(&options_format, newoptions); if (*msg) { - int len = (int)strlen(*msg)+256; - char *newmsg = tor_malloc(len); - - tor_snprintf(newmsg, len, "Failed to parse/validate config: %s", *msg); - tor_free(*msg); - *msg = newmsg; + char *old_msg = *msg; + tor_asprintf(msg, "Failed to parse/validate config: %s", old_msg); + tor_free(old_msg); } return err; } @@ -4318,11 +4335,13 @@ options_init_logs(or_options_t *options, int validate_only) if (smartlist_len(elts) == 2 && !strcasecmp(smartlist_get(elts,0), "file")) { if (!validate_only) { - if (add_file_log(severity, smartlist_get(elts, 1)) < 0) { + char *fname = expand_filename(smartlist_get(elts, 1)); + if (add_file_log(severity, fname) < 0) { log_warn(LD_CONFIG, "Couldn't open file for 'Log %s': %s", opt->value, strerror(errno)); ok = 0; } + tor_free(fname); } goto cleanup; } @@ -4338,6 +4357,9 @@ options_init_logs(or_options_t *options, int validate_only) } smartlist_free(elts); + if (ok && !validate_only) + logs_set_domain_logging(options->LogMessageDomains); + return ok?0:-1; } @@ -4586,7 +4608,7 @@ normalize_data_directory(or_options_t *options) } /** Check and normalize the value of options->DataDirectory; return 0 if it - * sane, -1 otherwise. */ + * is sane, -1 otherwise. */ static int validate_data_directory(or_options_t *options) { @@ -4618,7 +4640,6 @@ write_configuration_file(const char *fname, or_options_t *options) { char *old_val=NULL, *new_val=NULL, *new_conf=NULL; int rename_old = 0, r; - size_t len; tor_assert(fname); @@ -4645,9 +4666,7 @@ write_configuration_file(const char *fname, or_options_t *options) goto err; } - len = strlen(new_conf)+256; - new_val = tor_malloc(len); - tor_snprintf(new_val, len, "%s\n%s\n\n%s", + tor_asprintf(&new_val, "%s\n%s\n\n%s", GENERATED_FILE_PREFIX, GENERATED_FILE_COMMENT, new_conf); if (rename_old) { @@ -4697,22 +4716,19 @@ write_configuration_file(const char *fname, or_options_t *options) int options_save_current(void) { - if (torrc_fname) { - /* This fails if we can't write to our configuration file. - * - * If we try falling back to datadirectory or something, we have a better - * chance of saving the configuration, but a better chance of doing - * something the user never expected. Let's just warn instead. */ - return write_configuration_file(torrc_fname, get_options()); - } - return write_configuration_file(get_default_conf_file(), get_options()); + /* This fails if we can't write to our configuration file. + * + * If we try falling back to datadirectory or something, we have a better + * chance of saving the configuration, but a better chance of doing + * something the user never expected. */ + return write_configuration_file(get_torrc_fname(), get_options()); } /** Mapping from a unit name to a multiplier for converting that unit into a - * base unit. */ + * base unit. Used by config_parse_unit. */ struct unit_table_t { - const char *unit; - uint64_t multiplier; + const char *unit; /**< The name of the unit */ + uint64_t multiplier; /**< How many of the base unit appear in this unit */ }; /** Table to map the names of memory units to the number of bytes they @@ -4770,30 +4786,47 @@ static struct unit_table_t time_units[] = { static uint64_t config_parse_units(const char *val, struct unit_table_t *u, int *ok) { - uint64_t v; + uint64_t v = 0; + double d = 0; + int use_float = 0; char *cp; tor_assert(ok); v = tor_parse_uint64(val, 10, 0, UINT64_MAX, ok, &cp); - if (!*ok) - return 0; + if (!*ok || (cp && *cp == '.')) { + d = tor_parse_double(val, 0, UINT64_MAX, ok, &cp); + if (!*ok) + goto done; + use_float = 1; + } + if (!cp) { *ok = 1; - return v; + v = use_float ? DBL_TO_U64(d) : v; + goto done; } - while (TOR_ISSPACE(*cp)) - ++cp; + + cp = (char*) eat_whitespace(cp); + for ( ;u->unit;++u) { if (!strcasecmp(u->unit, cp)) { - v *= u->multiplier; + if (use_float) + v = u->multiplier * d; + else + v *= u->multiplier; *ok = 1; - return v; + goto done; } } log_warn(LD_CONFIG, "Unknown unit '%s'.", cp); *ok = 0; - return 0; + done: + + if (*ok) + return v; + else + return 0; } /** Parse a string in the format "number unit", where unit is a unit of @@ -4803,7 +4836,8 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok) static uint64_t config_parse_memunit(const char *s, int *ok) { - return config_parse_units(s, memory_units, ok); + uint64_t u = config_parse_units(s, memory_units, ok); + return u; } /** Parse a string in the format "number unit", where unit is a unit of time. @@ -4825,256 +4859,37 @@ config_parse_interval(const char *s, int *ok) return (int)r; } -/* This is what passes for version detection on OSX. We set - * MACOSX_KQUEUE_IS_BROKEN to true iff we're on a version of OSX before - * 10.4.0 (aka 1040). */ -#ifdef __APPLE__ -#ifdef __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ -#define MACOSX_KQUEUE_IS_BROKEN \ - (__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 1040) -#else -#define MACOSX_KQUEUE_IS_BROKEN 0 -#endif -#endif - /** * Initialize the libevent library. */ static void init_libevent(void) { + const char *badness=NULL; + configure_libevent_logging(); /* If the kernel complains that some method (say, epoll) doesn't * exist, we don't care about it, since libevent will cope. */ suppress_libevent_log_msg("Function not implemented"); -#ifdef __APPLE__ - if (MACOSX_KQUEUE_IS_BROKEN || - decode_libevent_version(event_get_version(), NULL) < LE_11B) { - setenv("EVENT_NOKQUEUE","1",1); - } -#endif - /* In libevent versions before 2.0, it's hard to keep binary compatibility - * between upgrades, and unpleasant to detect when the version we compiled - * against is unlike the version we have linked against. Here's how. */ -#if defined(_EVENT_VERSION) && defined(HAVE_EVENT_GET_VERSION) - /* We have a header-file version and a function-call version. Easy. */ - if (strcmp(_EVENT_VERSION, event_get_version())) { - int compat1 = -1, compat2 = -1; - int verybad, prettybad ; - decode_libevent_version(_EVENT_VERSION, &compat1); - decode_libevent_version(event_get_version(), &compat2); - verybad = compat1 != compat2; - prettybad = (compat1 == -1 || compat2 == -1) && compat1 != compat2; - - log(verybad ? LOG_WARN : (prettybad ? LOG_NOTICE : LOG_INFO), - LD_GENERAL, "We were compiled with headers from version %s " - "of Libevent, but we're using a Libevent library that says it's " - "version %s.", _EVENT_VERSION, event_get_version()); - if (verybad) - log_warn(LD_GENERAL, "This will almost certainly make Tor crash."); - else if (prettybad) - log_notice(LD_GENERAL, "If Tor crashes, this might be why."); - else - log_info(LD_GENERAL, "I think these versions are binary-compatible."); - } -#elif defined(HAVE_EVENT_GET_VERSION) - /* event_get_version but no _EVENT_VERSION. We might be in 1.4.0-beta or - earlier, where that's normal. To see whether we were compiled with an - earlier version, let's see whether the struct event defines MIN_HEAP_IDX. - */ -#ifdef HAVE_STRUCT_EVENT_MIN_HEAP_IDX - /* The header files are 1.4.0-beta or later. If the version is not - * 1.4.0-beta, we are incompatible. */ - { - if (strcmp(event_get_version(), "1.4.0-beta")) { - log_warn(LD_GENERAL, "It's a little hard to tell, but you seem to have " - "Libevent 1.4.0-beta header files, whereas you have linked " - "against Libevent %s. This will probably make Tor crash.", - event_get_version()); - } - } -#else - /* Our headers are 1.3e or earlier. If the library version is not 1.4.x or - later, we're probably fine. */ - { - const char *v = event_get_version(); - if ((v[0] == '1' && v[2] == '.' && v[3] > '3') || v[0] > '1') { - log_warn(LD_GENERAL, "It's a little hard to tell, but you seem to have " - "Libevent header file from 1.3e or earlier, whereas you have " - "linked against Libevent %s. This will probably make Tor " - "crash.", event_get_version()); - } - } -#endif + tor_check_libevent_header_compatibility(); -#elif defined(_EVENT_VERSION) -#warn "_EVENT_VERSION is defined but not get_event_version(): Libevent is odd." -#else - /* Your libevent is ancient. */ -#endif + tor_libevent_initialize(); - event_init(); suppress_libevent_log_msg(NULL); -#if defined(HAVE_EVENT_GET_VERSION) && defined(HAVE_EVENT_GET_METHOD) - /* Making this a NOTICE for now so we can link bugs to a libevent versions - * or methods better. */ - log(LOG_NOTICE, LD_GENERAL, - "Initialized libevent version %s using method %s. Good.", - event_get_version(), event_get_method()); - check_libevent_version(event_get_method(), get_options()->ORPort != 0); -#else - log(LOG_NOTICE, LD_GENERAL, - "Initialized old libevent (version 1.0b or earlier)."); - log(LOG_WARN, LD_GENERAL, - "You have a *VERY* old version of libevent. It is likely to be buggy; " - "please build Tor with a more recent version."); -#endif -} - -/** Table mapping return value of event_get_version() to le_version_t. */ -static const struct { - const char *name; le_version_t version; int bincompat; -} le_version_table[] = { - /* earlier versions don't have get_version. */ - { "1.0c", LE_10C, 1}, - { "1.0d", LE_10D, 1}, - { "1.0e", LE_10E, 1}, - { "1.1", LE_11, 1 }, - { "1.1a", LE_11A, 1 }, - { "1.1b", LE_11B, 1 }, - { "1.2", LE_12, 1 }, - { "1.2a", LE_12A, 1 }, - { "1.3", LE_13, 1 }, - { "1.3a", LE_13A, 1 }, - { "1.3b", LE_13B, 1 }, - { "1.3c", LE_13C, 1 }, - { "1.3d", LE_13D, 1 }, - { "1.3e", LE_13E, 1 }, - { "1.4.0-beta", LE_140, 2 }, - { "1.4.1-beta", LE_141, 2 }, - { "1.4.2-rc", LE_142, 2 }, - { "1.4.3-stable", LE_143, 2 }, - { "1.4.4-stable", LE_144, 2 }, - { "1.4.5-stable", LE_145, 2 }, - { "1.4.6-stable", LE_146, 2 }, - { "1.4.7-stable", LE_147, 2 }, - { "1.4.8-stable", LE_148, 2 }, - { "1.4.99-trunk", LE_1499, 3 }, - { NULL, LE_OTHER, 0 } -}; -/** Return the le_version_t for the current version of libevent. If the - * version is very new, return LE_OTHER. If the version is so old that it - * doesn't support event_get_version(), return LE_OLD. */ -static le_version_t -decode_libevent_version(const char *v, int *bincompat_out) -{ - int i; - for (i=0; le_version_table[i].name; ++i) { - if (!strcmp(le_version_table[i].name, v)) { - if (bincompat_out) - *bincompat_out = le_version_table[i].bincompat; - return le_version_table[i].version; - } - } - if (v[0] != '1' && bincompat_out) - *bincompat_out = 100; - else if (!strcmpstart(v, "1.4") && bincompat_out) - *bincompat_out = 2; - return LE_OTHER; -} - -#if defined(HAVE_EVENT_GET_VERSION) && defined(HAVE_EVENT_GET_METHOD) -/** - * Compare the given libevent method and version to a list of versions - * which are known not to work. Warn the user as appropriate. - */ -static void -check_libevent_version(const char *m, int server) -{ - int buggy = 0, iffy = 0, slow = 0, thread_unsafe = 0; - le_version_t version; - const char *v = event_get_version(); - const char *badness = NULL; - const char *sad_os = ""; - - version = decode_libevent_version(v, NULL); - - /* XXX Would it be worthwhile disabling the methods that we know - * are buggy, rather than just warning about them and then proceeding - * to use them? If so, we should probably not wrap this whole thing - * in HAVE_EVENT_GET_VERSION and HAVE_EVENT_GET_METHOD. -RD */ - /* XXXX The problem is that it's not trivial to get libevent to change it's - * method once it's initialized, and it's not trivial to tell what method it - * will use without initializing it. I guess we could preemptively disable - * buggy libevent modes based on the version _before_ initializing it, - * though, but then there's no good way (afaict) to warn "I would have used - * kqueue, but instead I'm using select." -NM */ - if (!strcmp(m, "kqueue")) { - if (version < LE_11B) - buggy = 1; - } else if (!strcmp(m, "epoll")) { - if (version < LE_11) - iffy = 1; - } else if (!strcmp(m, "poll")) { - if (version < LE_10E) - buggy = 1; - else if (version < LE_11) - slow = 1; - } else if (!strcmp(m, "select")) { - if (version < LE_11) - slow = 1; - } else if (!strcmp(m, "win32")) { - if (version < LE_11B) - buggy = 1; - } - - /* Libevent versions before 1.3b do very badly on operating systems with - * user-space threading implementations. */ -#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(__NetBSD__) - if (server && version < LE_13B) { - thread_unsafe = 1; - sad_os = "BSD variants"; - } -#elif defined(__APPLE__) || defined(__darwin__) - if (server && version < LE_13B) { - thread_unsafe = 1; - sad_os = "Mac OS X"; - } -#endif - - if (thread_unsafe) { - log(LOG_WARN, LD_GENERAL, - "Libevent version %s often crashes when running a Tor server with %s. " - "Please use the latest version of libevent (1.3b or later)",v,sad_os); - badness = "BROKEN"; - } else if (buggy) { - log(LOG_WARN, LD_GENERAL, - "There are serious bugs in using %s with libevent %s. " - "Please use the latest version of libevent.", m, v); - badness = "BROKEN"; - } else if (iffy) { - log(LOG_WARN, LD_GENERAL, - "There are minor bugs in using %s with libevent %s. " - "You may want to use the latest version of libevent.", m, v); - badness = "BUGGY"; - } else if (slow && server) { - log(LOG_WARN, LD_GENERAL, - "libevent %s can be very slow with %s. " - "When running a server, please use the latest version of libevent.", - v,m); - badness = "SLOW"; - } + tor_check_libevent_version(tor_libevent_get_method(), + get_options()->ORPort != 0, + &badness); if (badness) { + const char *v = tor_libevent_get_version_str(); + const char *m = tor_libevent_get_method(); control_event_general_status(LOG_WARN, "BAD_LIBEVENT VERSION=%s METHOD=%s BADNESS=%s RECOVERED=NO", v, m, badness); } - } -#endif /** Return the persistent state struct for this Tor. */ or_state_t * @@ -5156,22 +4971,61 @@ or_state_validate(or_state_t *old_state, or_state_t *state, } /** Replace the current persistent state with <b>new_state</b> */ -static void +static int or_state_set(or_state_t *new_state) { char *err = NULL; + int ret = 0; tor_assert(new_state); - if (global_state) - config_free(&state_format, global_state); + config_free(&state_format, global_state); global_state = new_state; if (entry_guards_parse_state(global_state, 1, &err)<0) { log_warn(LD_GENERAL,"%s",err); tor_free(err); + ret = -1; } if (rep_hist_load_state(global_state, &err)<0) { log_warn(LD_GENERAL,"Unparseable bandwidth history state: %s",err); tor_free(err); + ret = -1; } + if (circuit_build_times_parse_state(&circ_times, global_state) < 0) { + ret = -1; + } + return ret; +} + +/** + * Save a broken state file to a backup location. + */ +static void +or_state_save_broken(char *fname) +{ + int i; + file_status_t status; + size_t len = strlen(fname)+16; + char *fname2 = tor_malloc(len); + for (i = 0; i < 100; ++i) { + tor_snprintf(fname2, len, "%s.%d", fname, i); + status = file_status(fname2); + if (status == FN_NOENT) + break; + } + if (i == 100) { + log_warn(LD_BUG, "Unable to parse state in \"%s\"; too many saved bad " + "state files to move aside. Discarding the old state file.", + fname); + unlink(fname); + } else { + log_warn(LD_BUG, "Unable to parse state in \"%s\". Moving it aside " + "to \"%s\". This could be a bug in Tor; please tell " + "the developers.", fname, fname2); + if (rename(fname, fname2) < 0) { + log_warn(LD_BUG, "Weirdly, I couldn't even move the state aside. The " + "OS gave an error of %s", strerror(errno)); + } + } + tor_free(fname2); } /** Reload the persistent state from disk, generating a new state as needed. @@ -5233,31 +5087,8 @@ or_state_load(void) " This is a bug in Tor."); goto done; } else if (badstate && contents) { - int i; - file_status_t status; - size_t len = strlen(fname)+16; - char *fname2 = tor_malloc(len); - for (i = 0; i < 100; ++i) { - tor_snprintf(fname2, len, "%s.%d", fname, i); - status = file_status(fname2); - if (status == FN_NOENT) - break; - } - if (i == 100) { - log_warn(LD_BUG, "Unable to parse state in \"%s\"; too many saved bad " - "state files to move aside. Discarding the old state file.", - fname); - unlink(fname); - } else { - log_warn(LD_BUG, "Unable to parse state in \"%s\". Moving it aside " - "to \"%s\". This could be a bug in Tor; please tell " - "the developers.", fname, fname2); - if (rename(fname, fname2) < 0) { - log_warn(LD_BUG, "Weirdly, I couldn't even move the state aside. The " - "OS gave an error of %s", strerror(errno)); - } - } - tor_free(fname2); + or_state_save_broken(fname); + tor_free(contents); config_free(&state_format, new_state); @@ -5269,7 +5100,9 @@ or_state_load(void) } else { log_info(LD_GENERAL, "Initialized state"); } - or_state_set(new_state); + if (or_state_set(new_state) == -1) { + or_state_save_broken(fname); + } new_state = NULL; if (!contents) { global_state->next_write = 0; @@ -5286,13 +5119,32 @@ or_state_load(void) return r; } +/** Did the last time we tried to write the state file fail? If so, we + * should consider disabling such features as preemptive circuit generation + * to compute circuit-build-time. */ +static int last_state_file_write_failed = 0; + +/** Return whether the state file failed to write last time we tried. */ +int +did_last_state_file_write_fail(void) +{ + return last_state_file_write_failed; +} + +/** If writing the state to disk fails, try again after this many seconds. */ +#define STATE_WRITE_RETRY_INTERVAL 3600 + +/** If we're a relay, how often should we checkpoint our state file even + * if nothing else dirties it? This will checkpoint ongoing stats like + * bandwidth used, per-country user stats, etc. */ +#define STATE_RELAY_CHECKPOINT_INTERVAL (12*60*60) + /** Write the persistent state to disk. Return 0 for success, <0 on failure. */ int or_state_save(time_t now) { char *state, *contents; char tbuf[ISO_TIME_LEN+1]; - size_t len; char *fname; tor_assert(global_state); @@ -5304,20 +5156,18 @@ or_state_save(time_t now) * to avoid redundant writes. */ entry_guards_update_state(global_state); rep_hist_update_state(global_state); + circuit_build_times_update_state(&circ_times, global_state); if (accounting_is_enabled(get_options())) accounting_run_housekeeping(now); - global_state->LastWritten = time(NULL); + global_state->LastWritten = now; + tor_free(global_state->TorVersion); - len = strlen(get_version())+8; - global_state->TorVersion = tor_malloc(len); - tor_snprintf(global_state->TorVersion, len, "Tor %s", get_version()); + tor_asprintf(&global_state->TorVersion, "Tor %s", get_version()); state = config_dump(&state_format, global_state, 1, 0); - len = strlen(state)+256; - contents = tor_malloc(len); - format_local_iso_time(tbuf, time(NULL)); - tor_snprintf(contents, len, + format_local_iso_time(tbuf, now); + tor_asprintf(&contents, "# Tor state file last generated on %s local time\n" "# Other times below are in GMT\n" "# You *do not* need to edit this file.\n\n%s", @@ -5327,15 +5177,25 @@ or_state_save(time_t now) if (write_str_to_file(fname, contents, 0)<0) { log_warn(LD_FS, "Unable to write state to file \"%s\"; " "will try again later", fname); + last_state_file_write_failed = 1; tor_free(fname); tor_free(contents); + /* Try again after STATE_WRITE_RETRY_INTERVAL (or sooner, if the state + * changes sooner). */ + global_state->next_write = now + STATE_WRITE_RETRY_INTERVAL; return -1; } + + last_state_file_write_failed = 0; log_info(LD_GENERAL, "Saved state to \"%s\"", fname); tor_free(fname); tor_free(contents); - global_state->next_write = TIME_MAX; + if (server_mode(get_options())) + global_state->next_write = now + STATE_RELAY_CHECKPOINT_INTERVAL; + else + global_state->next_write = TIME_MAX; + return 0; } @@ -5362,18 +5222,18 @@ remove_file_if_very_old(const char *fname, time_t now) * types. */ int getinfo_helper_config(control_connection_t *conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { (void) conn; + (void) errmsg; if (!strcmp(question, "config/names")) { smartlist_t *sl = smartlist_create(); int i; for (i = 0; _option_vars[i].name; ++i) { config_var_t *var = &_option_vars[i]; - const char *type, *desc; + const char *type; char *line; - size_t len; - desc = config_find_description(&options_format, var->name); switch (var->type) { case CONFIG_TYPE_STRING: type = "String"; break; case CONFIG_TYPE_FILENAME: type = "Filename"; break; @@ -5394,14 +5254,7 @@ getinfo_helper_config(control_connection_t *conn, } if (!type) continue; - len = strlen(var->name)+strlen(type)+16; - if (desc) - len += strlen(desc); - line = tor_malloc(len); - if (desc) - tor_snprintf(line, len, "%s %s %s\n",var->name,type,desc); - else - tor_snprintf(line, len, "%s %s\n",var->name,type); + tor_asprintf(&line, "%s %s\n",var->name,type); smartlist_add(sl, line); } *answer = smartlist_join_strings(sl, "", 0, NULL); diff --git a/src/or/config.h b/src/or/config.h new file mode 100644 index 000000000..78a67dddf --- /dev/null +++ b/src/or/config.h @@ -0,0 +1,79 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file config.h + * \brief Header file for config.c. + **/ + +#ifndef _TOR_CONFIG_H +#define _TOR_CONFIG_H + +const char *get_dirportfrontpage(void); +or_options_t *get_options(void); +int set_options(or_options_t *new_val, char **msg); +void config_free_all(void); +const char *safe_str_client(const char *address); +const char *safe_str(const char *address); +const char *escaped_safe_str_client(const char *address); +const char *escaped_safe_str(const char *address); +const char *get_version(void); + +int config_get_lines(const char *string, config_line_t **result); +void config_free_lines(config_line_t *front); +setopt_err_t options_trial_assign(config_line_t *list, int use_defaults, + int clear_first, char **msg); +int resolve_my_address(int warn_severity, or_options_t *options, + uint32_t *addr, char **hostname_out); +int is_local_addr(const tor_addr_t *addr) ATTR_PURE; +void options_init(or_options_t *options); +char *options_dump(or_options_t *options, int minimal); +int options_init_from_torrc(int argc, char **argv); +setopt_err_t options_init_from_string(const char *cf, + int command, const char *command_arg, char **msg); +int option_is_recognized(const char *key); +const char *option_get_canonical_name(const char *key); +config_line_t *option_get_assignment(or_options_t *options, + const char *key); +int options_save_current(void); +const char *get_torrc_fname(void); +char *options_get_datadir_fname2_suffix(or_options_t *options, + const char *sub1, const char *sub2, + const char *suffix); +#define get_datadir_fname2_suffix(sub1, sub2, suffix) \ + options_get_datadir_fname2_suffix(get_options(), (sub1), (sub2), (suffix)) +/** Return a newly allocated string containing datadir/sub1. See + * get_datadir_fname2_suffix. */ +#define get_datadir_fname(sub1) get_datadir_fname2_suffix((sub1), NULL, NULL) +/** Return a newly allocated string containing datadir/sub1/sub2. See + * get_datadir_fname2_suffix. */ +#define get_datadir_fname2(sub1,sub2) \ + get_datadir_fname2_suffix((sub1), (sub2), NULL) +/** Return a newly allocated string containing datadir/sub1suffix. See + * get_datadir_fname2_suffix. */ +#define get_datadir_fname_suffix(sub1, suffix) \ + get_datadir_fname2_suffix((sub1), NULL, (suffix)) + +or_state_t *get_or_state(void); +int did_last_state_file_write_fail(void); +int or_state_save(time_t now); + +int options_need_geoip_info(or_options_t *options, const char **reason_out); +int getinfo_helper_config(control_connection_t *conn, + const char *question, char **answer, + const char **errmsg); + +const char *tor_get_digests(void); +uint32_t get_effective_bwrate(or_options_t *options); +uint32_t get_effective_bwburst(or_options_t *options); + +#ifdef CONFIG_PRIVATE +/* Used only by config.c and test.c */ +or_options_t *options_new(void); +#endif + +#endif + diff --git a/src/or/config_codedigest.c b/src/or/config_codedigest.c new file mode 100644 index 000000000..be9eaa331 --- /dev/null +++ b/src/or/config_codedigest.c @@ -0,0 +1,11 @@ + +const char *tor_get_digests(void); + +const char * +tor_get_digests(void) +{ + return "" +#include "or_sha1.i" + ; +} + diff --git a/src/or/connection.c b/src/or/connection.c index 4869a2439..fc2097f9a 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -11,6 +11,30 @@ **/ #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "cpuworker.h" +#include "directory.h" +#include "dirserv.h" +#include "dns.h" +#include "dnsserv.h" +#include "geoip.h" +#include "main.h" +#include "policies.h" +#include "reasons.h" +#include "relay.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rephist.h" +#include "router.h" +#include "routerparse.h" static connection_t *connection_create_listener( struct sockaddr *listensockaddr, @@ -21,17 +45,22 @@ static void connection_init(time_t now, connection_t *conn, int type, static int connection_init_accepted_conn(connection_t *conn, uint8_t listener_type); static int connection_handle_listener_read(connection_t *conn, int new_type); -static int connection_read_bucket_should_increase(or_connection_t *conn); +static int connection_bucket_should_increase(int bucket, + or_connection_t *conn); static int connection_finished_flushing(connection_t *conn); static int connection_flushed_some(connection_t *conn); static int connection_finished_connecting(connection_t *conn); static int connection_reached_eof(connection_t *conn); -static int connection_read_to_buf(connection_t *conn, int *max_to_read, +static int connection_read_to_buf(connection_t *conn, ssize_t *max_to_read, int *socket_error); static int connection_process_inbuf(connection_t *conn, int package_partial); static void client_check_address_changed(int sock); static void set_constrained_socket_buffers(int sock, int size); +static const char *connection_proxy_state_to_string(int state); +static int connection_read_https_proxy_response(connection_t *conn); +static void connection_send_socks5_connect(connection_t *conn); + /** The last IPv4 address that our network interface seemed to have been * binding to, in host order. We use this to detect when our IP changes. */ static uint32_t last_interface_ip = 0; @@ -92,8 +121,7 @@ conn_state_to_string(int type, int state) case CONN_TYPE_OR: switch (state) { case OR_CONN_STATE_CONNECTING: return "connect()ing"; - case OR_CONN_STATE_PROXY_FLUSHING: return "proxy flushing"; - case OR_CONN_STATE_PROXY_READING: return "proxy reading"; + case OR_CONN_STATE_PROXY_HANDSHAKING: return "handshaking (proxy)"; case OR_CONN_STATE_TLS_HANDSHAKING: return "handshaking (TLS)"; case OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING: return "renegotiating (TLS)"; @@ -177,6 +205,9 @@ or_connection_new(int socket_family) or_conn->timestamp_last_added_nonpadding = time(NULL); or_conn->next_circ_id = crypto_rand_int(1<<15); + or_conn->active_circuit_pqueue = smartlist_create(); + or_conn->active_circuit_pqueue_last_recalibrated = cell_ewma_get_tick(); + return or_conn; } @@ -202,6 +233,7 @@ control_connection_new(int socket_family) tor_malloc_zero(sizeof(control_connection_t)); connection_init(time(NULL), TO_CONN(control_conn), CONN_TYPE_CONTROL, socket_family); + log_notice(LD_CONTROL, "New control connection opened."); return control_conn; } @@ -299,25 +331,6 @@ connection_link_connections(connection_t *conn_a, connection_t *conn_b) conn_b->linked_conn = conn_a; } -/** Tell libevent that we don't care about <b>conn</b> any more. */ -void -connection_unregister_events(connection_t *conn) -{ - if (conn->read_event) { - if (event_del(conn->read_event)) - log_warn(LD_BUG, "Error removing read event for %d", conn->s); - tor_free(conn->read_event); - } - if (conn->write_event) { - if (event_del(conn->write_event)) - log_warn(LD_BUG, "Error removing write event for %d", conn->s); - tor_free(conn->write_event); - } - if (conn->dns_server_port) { - dnsserv_close_listener(conn); - } -} - /** Deallocate memory used by <b>conn</b>. Deallocate its buffers if * necessary, close its socket if necessary, and mark the directory as dirty * if <b>conn</b> is an OR or OP connection. @@ -327,6 +340,9 @@ _connection_free(connection_t *conn) { void *mem; size_t memlen; + if (!conn) + return; + switch (conn->type) { case CONN_TYPE_OR: tor_assert(conn->magic == OR_CONNECTION_MAGIC); @@ -384,14 +400,11 @@ _connection_free(connection_t *conn) if (connection_speaks_cells(conn)) { or_connection_t *or_conn = TO_OR_CONN(conn); - if (or_conn->tls) { - tor_tls_free(or_conn->tls); - or_conn->tls = NULL; - } - if (or_conn->handshake_state) { - or_handshake_state_free(or_conn->handshake_state); - or_conn->handshake_state = NULL; - } + tor_tls_free(or_conn->tls); + or_conn->tls = NULL; + or_handshake_state_free(or_conn->handshake_state); + or_conn->handshake_state = NULL; + smartlist_free(or_conn->active_circuit_pqueue); tor_free(or_conn->nickname); } if (CONN_IS_EDGE(conn)) { @@ -401,8 +414,8 @@ _connection_free(connection_t *conn) memset(edge_conn->socks_request, 0xcc, sizeof(socks_request_t)); tor_free(edge_conn->socks_request); } - if (edge_conn->rend_data) - rend_data_free(edge_conn->rend_data); + + rend_data_free(edge_conn->rend_data); } if (conn->type == CONN_TYPE_CONTROL) { control_connection_t *control_conn = TO_CONTROL_CONN(conn); @@ -415,16 +428,15 @@ _connection_free(connection_t *conn) if (conn->type == CONN_TYPE_DIR) { dir_connection_t *dir_conn = TO_DIR_CONN(conn); tor_free(dir_conn->requested_resource); - if (dir_conn->zlib_state) - tor_zlib_free(dir_conn->zlib_state); + + tor_zlib_free(dir_conn->zlib_state); if (dir_conn->fingerprint_stack) { SMARTLIST_FOREACH(dir_conn->fingerprint_stack, char *, cp, tor_free(cp)); smartlist_free(dir_conn->fingerprint_stack); } - if (dir_conn->cached_dir) - cached_dir_decref(dir_conn->cached_dir); - if (dir_conn->rend_data) - rend_data_free(dir_conn->rend_data); + + cached_dir_decref(dir_conn->cached_dir); + rend_data_free(dir_conn->rend_data); } if (conn->s >= 0) { @@ -439,7 +451,7 @@ _connection_free(connection_t *conn) connection_or_remove_from_identity_map(TO_OR_CONN(conn)); } - memset(conn, 0xAA, memlen); /* poison memory */ + memset(mem, 0xCC, memlen); /* poison memory */ tor_free(mem); } @@ -448,7 +460,8 @@ _connection_free(connection_t *conn) void connection_free(connection_t *conn) { - tor_assert(conn); + if (!conn) + return; tor_assert(!connection_is_on_closeable_list(conn)); tor_assert(!connection_in_array(conn)); if (conn->linked_conn) { @@ -544,13 +557,6 @@ connection_about_to_close_connection(connection_t *conn) * failed: forget about this router, and maybe try again. */ connection_dir_request_failed(dir_conn); } - if (conn->purpose == DIR_PURPOSE_FETCH_RENDDESC && dir_conn->rend_data) { - /* Give it a try. However, there is no re-fetching for v0 rend - * descriptors; if the response is empty or the descriptor is - * unusable, close pending connections (unless a v2 request is - * still in progress). */ - rend_client_desc_trynow(dir_conn->rend_data->onion_address, 0); - } /* If we were trying to fetch a v2 rend desc and did not succeed, * retry as needed. (If a fetch is successful, the connection state * is changed to DIR_PURPOSE_HAS_FETCHED_RENDDESC to mark that @@ -573,7 +579,7 @@ connection_about_to_close_connection(connection_t *conn) or_options_t *options = get_options(); rep_hist_note_connect_failed(or_conn->identity_digest, now); entry_guard_register_connect_status(or_conn->identity_digest,0, - !options->HttpsProxy, now); + !options->HTTPSProxy, now); if (conn->state >= OR_CONN_STATE_TLS_HANDSHAKING) { int reason = tls_error_to_orconn_end_reason(or_conn->tls_error); control_event_or_conn_status(or_conn, OR_CONN_EVENT_FAILED, @@ -589,7 +595,7 @@ connection_about_to_close_connection(connection_t *conn) rep_hist_note_disconnect(or_conn->identity_digest, now); control_event_or_conn_status(or_conn, OR_CONN_EVENT_CLOSED, tls_error_to_orconn_end_reason(or_conn->tls_error)); - } else if (or_conn->identity_digest) { + } else if (!tor_digest_is_zero(or_conn->identity_digest)) { rep_hist_note_connection_died(or_conn->identity_digest, now); control_event_or_conn_status(or_conn, OR_CONN_EVENT_CLOSED, tls_error_to_orconn_end_reason(or_conn->tls_error)); @@ -833,16 +839,16 @@ static void warn_too_many_conns(void) { #define WARN_TOO_MANY_CONNS_INTERVAL (6*60*60) - static time_t last_warned = 0; - time_t now = time(NULL); - int n_conns = get_n_open_sockets(); - if (last_warned + WARN_TOO_MANY_CONNS_INTERVAL < now) { + static ratelim_t last_warned = RATELIM_INIT(WARN_TOO_MANY_CONNS_INTERVAL); + char *m; + if ((m = rate_limit_log(&last_warned, approx_time()))) { + int n_conns = get_n_open_sockets(); log_warn(LD_NET,"Failing because we have %d connections already. Please " - "raise your ulimit -n.", n_conns); - last_warned = now; + "raise your ulimit -n.%s", n_conns, m); + tor_free(m); + control_event_general_status(LOG_WARN, "TOO_MANY_CONNECTIONS CURRENT=%d", + n_conns); } - control_event_general_status(LOG_WARN, "TOO_MANY_CONNECTIONS CURRENT=%d", - n_conns); } /** Bind a new non-blocking socket listening to the socket described @@ -1172,7 +1178,8 @@ connection_handle_listener_read(connection_t *conn, int new_type) } if (connection_init_accepted_conn(newconn, conn->type) < 0) { - connection_mark_for_close(newconn); + if (! newconn->marked_for_close) + connection_mark_for_close(newconn); return 0; } return 0; @@ -1198,9 +1205,11 @@ connection_init_accepted_conn(connection_t *conn, uint8_t listener_type) conn->state = AP_CONN_STATE_SOCKS_WAIT; break; case CONN_TYPE_AP_TRANS_LISTENER: + TO_EDGE_CONN(conn)->is_transparent_ap = 1; conn->state = AP_CONN_STATE_CIRCUIT_WAIT; return connection_ap_process_transparent(TO_EDGE_CONN(conn)); case CONN_TYPE_AP_NATD_LISTENER: + TO_EDGE_CONN(conn)->is_transparent_ap = 1; conn->state = AP_CONN_STATE_NATD_WAIT; break; } @@ -1231,7 +1240,7 @@ connection_connect(connection_t *conn, const char *address, { int s, inprogress = 0; char addrbuf[256]; - struct sockaddr *dest_addr = (struct sockaddr*) addrbuf; + struct sockaddr *dest_addr; socklen_t dest_addr_len; or_options_t *options = get_options(); int protocol_family; @@ -1254,7 +1263,7 @@ connection_connect(connection_t *conn, const char *address, return -1; } - if (options->OutboundBindAddress) { + if (options->OutboundBindAddress && !tor_addr_is_loopback(addr)) { struct sockaddr_in ext_addr; memset(&ext_addr, 0, sizeof(ext_addr)); @@ -1285,7 +1294,8 @@ connection_connect(connection_t *conn, const char *address, dest_addr_len = tor_addr_to_sockaddr(addr, port, dest_addr, sizeof(addrbuf)); tor_assert(dest_addr_len > 0); - log_debug(LD_NET,"Connecting to %s:%u.",escaped_safe_str(address),port); + log_debug(LD_NET, "Connecting to %s:%u.", + escaped_safe_str_client(address), port); if (connect(s, dest_addr, dest_addr_len) < 0) { int e = tor_socket_errno(s); @@ -1293,7 +1303,8 @@ connection_connect(connection_t *conn, const char *address, /* yuck. kill it. */ *socket_error = e; log_info(LD_NET, - "connect() to %s:%u failed: %s",escaped_safe_str(address), + "connect() to %s:%u failed: %s", + escaped_safe_str_client(address), port, tor_socket_strerror(e)); tor_close_socket(s); return -1; @@ -1307,7 +1318,8 @@ connection_connect(connection_t *conn, const char *address, /* it succeeded. we're connected. */ log_fn(inprogress?LOG_DEBUG:LOG_INFO, LD_NET, - "Connection to %s:%u %s (sock %d).",escaped_safe_str(address), + "Connection to %s:%u %s (sock %d).", + escaped_safe_str_client(address), port, inprogress?"in progress":"established", s); conn->s = s; if (connection_add(conn) < 0) /* no space, forget it */ @@ -1315,6 +1327,353 @@ connection_connect(connection_t *conn, const char *address, return inprogress ? 0 : 1; } +/** Convert state number to string representation for logging purposes. + */ +static const char * +connection_proxy_state_to_string(int state) +{ + static const char *unknown = "???"; + static const char *states[] = { + "PROXY_NONE", + "PROXY_HTTPS_WANT_CONNECT_OK", + "PROXY_SOCKS4_WANT_CONNECT_OK", + "PROXY_SOCKS5_WANT_AUTH_METHOD_NONE", + "PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929", + "PROXY_SOCKS5_WANT_AUTH_RFC1929_OK", + "PROXY_SOCKS5_WANT_CONNECT_OK", + "PROXY_CONNECTED", + }; + + if (state < PROXY_NONE || state > PROXY_CONNECTED) + return unknown; + + return states[state]; +} + +/** Write a proxy request of <b>type</b> (socks4, socks5, https) to conn + * for conn->addr:conn->port, authenticating with the auth details given + * in the configuration (if available). SOCKS 5 and HTTP CONNECT proxies + * support authentication. + * + * Returns -1 if conn->addr is incompatible with the proxy protocol, and + * 0 otherwise. + * + * Use connection_read_proxy_handshake() to complete the handshake. + */ +int +connection_proxy_connect(connection_t *conn, int type) +{ + or_options_t *options; + + tor_assert(conn); + + options = get_options(); + + switch (type) { + case PROXY_CONNECT: { + char buf[1024]; + char *base64_authenticator=NULL; + const char *authenticator = options->HTTPSProxyAuthenticator; + + /* Send HTTP CONNECT and authentication (if available) in + * one request */ + + if (authenticator) { + base64_authenticator = alloc_http_authenticator(authenticator); + if (!base64_authenticator) + log_warn(LD_OR, "Encoding https authenticator failed"); + } + + if (base64_authenticator) { + tor_snprintf(buf, sizeof(buf), "CONNECT %s:%d HTTP/1.1\r\n" + "Proxy-Authorization: Basic %s\r\n\r\n", + fmt_addr(&conn->addr), + conn->port, base64_authenticator); + tor_free(base64_authenticator); + } else { + tor_snprintf(buf, sizeof(buf), "CONNECT %s:%d HTTP/1.0\r\n\r\n", + fmt_addr(&conn->addr), conn->port); + } + + connection_write_to_buf(buf, strlen(buf), conn); + conn->proxy_state = PROXY_HTTPS_WANT_CONNECT_OK; + break; + } + + case PROXY_SOCKS4: { + unsigned char buf[9]; + uint16_t portn; + uint32_t ip4addr; + + /* Send a SOCKS4 connect request with empty user id */ + + if (tor_addr_family(&conn->addr) != AF_INET) { + log_warn(LD_NET, "SOCKS4 client is incompatible with IPv6"); + return -1; + } + + ip4addr = tor_addr_to_ipv4n(&conn->addr); + portn = htons(conn->port); + + buf[0] = 4; /* version */ + buf[1] = SOCKS_COMMAND_CONNECT; /* command */ + memcpy(buf + 2, &portn, 2); /* port */ + memcpy(buf + 4, &ip4addr, 4); /* addr */ + buf[8] = 0; /* userid (empty) */ + + connection_write_to_buf((char *)buf, sizeof(buf), conn); + conn->proxy_state = PROXY_SOCKS4_WANT_CONNECT_OK; + break; + } + + case PROXY_SOCKS5: { + unsigned char buf[4]; /* fields: vers, num methods, method list */ + + /* Send a SOCKS5 greeting (connect request must wait) */ + + buf[0] = 5; /* version */ + + /* number of auth methods */ + if (options->Socks5ProxyUsername) { + buf[1] = 2; + buf[2] = 0x00; /* no authentication */ + buf[3] = 0x02; /* rfc1929 Username/Passwd auth */ + conn->proxy_state = PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929; + } else { + buf[1] = 1; + buf[2] = 0x00; /* no authentication */ + conn->proxy_state = PROXY_SOCKS5_WANT_AUTH_METHOD_NONE; + } + + connection_write_to_buf((char *)buf, 2 + buf[1], conn); + break; + } + + default: + log_err(LD_BUG, "Invalid proxy protocol, %d", type); + tor_fragile_assert(); + return -1; + } + + log_debug(LD_NET, "set state %s", + connection_proxy_state_to_string(conn->proxy_state)); + + return 0; +} + +/** Read conn's inbuf. If the http response from the proxy is all + * here, make sure it's good news, then return 1. If it's bad news, + * return -1. Else return 0 and hope for better luck next time. + */ +static int +connection_read_https_proxy_response(connection_t *conn) +{ + char *headers; + char *reason=NULL; + int status_code; + time_t date_header; + + switch (fetch_from_buf_http(conn->inbuf, + &headers, MAX_HEADERS_SIZE, + NULL, NULL, 10000, 0)) { + case -1: /* overflow */ + log_warn(LD_PROTOCOL, + "Your https proxy sent back an oversized response. Closing."); + return -1; + case 0: + log_info(LD_NET,"https proxy response not all here yet. Waiting."); + return 0; + /* case 1, fall through */ + } + + if (parse_http_response(headers, &status_code, &date_header, + NULL, &reason) < 0) { + log_warn(LD_NET, + "Unparseable headers from proxy (connecting to '%s'). Closing.", + conn->address); + tor_free(headers); + return -1; + } + if (!reason) reason = tor_strdup("[no reason given]"); + + if (status_code == 200) { + log_info(LD_NET, + "HTTPS connect to '%s' successful! (200 %s) Starting TLS.", + conn->address, escaped(reason)); + tor_free(reason); + return 1; + } + /* else, bad news on the status code */ + log_warn(LD_NET, + "The https proxy sent back an unexpected status code %d (%s). " + "Closing.", + status_code, escaped(reason)); + tor_free(reason); + return -1; +} + +/** Send SOCKS5 CONNECT command to <b>conn</b>, copying <b>conn->addr</b> + * and <b>conn->port</b> into the request. + */ +static void +connection_send_socks5_connect(connection_t *conn) +{ + unsigned char buf[1024]; + size_t reqsize = 6; + uint16_t port = htons(conn->port); + + buf[0] = 5; /* version */ + buf[1] = SOCKS_COMMAND_CONNECT; /* command */ + buf[2] = 0; /* reserved */ + + if (tor_addr_family(&conn->addr) == AF_INET) { + uint32_t addr = tor_addr_to_ipv4n(&conn->addr); + + buf[3] = 1; + reqsize += 4; + memcpy(buf + 4, &addr, 4); + memcpy(buf + 8, &port, 2); + } else { /* AF_INET6 */ + buf[3] = 4; + reqsize += 16; + memcpy(buf + 4, tor_addr_to_in6(&conn->addr), 16); + memcpy(buf + 20, &port, 2); + } + + connection_write_to_buf((char *)buf, reqsize, conn); + + conn->proxy_state = PROXY_SOCKS5_WANT_CONNECT_OK; +} + +/** Call this from connection_*_process_inbuf() to advance the proxy + * handshake. + * + * No matter what proxy protocol is used, if this function returns 1, the + * handshake is complete, and the data remaining on inbuf may contain the + * start of the communication with the requested server. + * + * Returns 0 if the current buffer contains an incomplete response, and -1 + * on error. + */ +int +connection_read_proxy_handshake(connection_t *conn) +{ + int ret = 0; + char *reason = NULL; + + log_debug(LD_NET, "enter state %s", + connection_proxy_state_to_string(conn->proxy_state)); + + switch (conn->proxy_state) { + case PROXY_HTTPS_WANT_CONNECT_OK: + ret = connection_read_https_proxy_response(conn); + if (ret == 1) + conn->proxy_state = PROXY_CONNECTED; + break; + + case PROXY_SOCKS4_WANT_CONNECT_OK: + ret = fetch_from_buf_socks_client(conn->inbuf, + conn->proxy_state, + &reason); + if (ret == 1) + conn->proxy_state = PROXY_CONNECTED; + break; + + case PROXY_SOCKS5_WANT_AUTH_METHOD_NONE: + ret = fetch_from_buf_socks_client(conn->inbuf, + conn->proxy_state, + &reason); + /* no auth needed, do connect */ + if (ret == 1) { + connection_send_socks5_connect(conn); + ret = 0; + } + break; + + case PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929: + ret = fetch_from_buf_socks_client(conn->inbuf, + conn->proxy_state, + &reason); + + /* send auth if needed, otherwise do connect */ + if (ret == 1) { + connection_send_socks5_connect(conn); + ret = 0; + } else if (ret == 2) { + unsigned char buf[1024]; + size_t reqsize, usize, psize; + const char *user, *pass; + + user = get_options()->Socks5ProxyUsername; + pass = get_options()->Socks5ProxyPassword; + tor_assert(user && pass); + + /* XXX len of user and pass must be <= 255 !!! */ + usize = strlen(user); + psize = strlen(pass); + tor_assert(usize <= 255 && psize <= 255); + reqsize = 3 + usize + psize; + + buf[0] = 1; /* negotiation version */ + buf[1] = usize; + memcpy(buf + 2, user, usize); + buf[2 + usize] = psize; + memcpy(buf + 3 + usize, pass, psize); + + connection_write_to_buf((char *)buf, reqsize, conn); + + conn->proxy_state = PROXY_SOCKS5_WANT_AUTH_RFC1929_OK; + ret = 0; + } + break; + + case PROXY_SOCKS5_WANT_AUTH_RFC1929_OK: + ret = fetch_from_buf_socks_client(conn->inbuf, + conn->proxy_state, + &reason); + /* send the connect request */ + if (ret == 1) { + connection_send_socks5_connect(conn); + ret = 0; + } + break; + + case PROXY_SOCKS5_WANT_CONNECT_OK: + ret = fetch_from_buf_socks_client(conn->inbuf, + conn->proxy_state, + &reason); + if (ret == 1) + conn->proxy_state = PROXY_CONNECTED; + break; + + default: + log_err(LD_BUG, "Invalid proxy_state for reading, %d", + conn->proxy_state); + tor_fragile_assert(); + ret = -1; + break; + } + + log_debug(LD_NET, "leaving state %s", + connection_proxy_state_to_string(conn->proxy_state)); + + if (ret < 0) { + if (reason) { + log_warn(LD_NET, "Proxy Client: unable to connect to %s:%d (%s)", + conn->address, conn->port, escaped(reason)); + tor_free(reason); + } else { + log_warn(LD_NET, "Proxy Client: unable to connect to %s:%d", + conn->address, conn->port); + } + } else if (ret == 1) { + log_info(LD_NET, "Proxy Client: connection to %s:%d successful", + conn->address, conn->port); + } + + return ret; +} + /** * Launch any configured listener connections of type <b>type</b>. (A * listener is configured if <b>port_option</b> is non-zero. If any @@ -1506,8 +1865,8 @@ retry_all_listeners(smartlist_t *replaced_conns, replaced_conns, new_conns, 0, AF_INET)<0) return -1; - if (retry_listeners(CONN_TYPE_AP_NATD_LISTENER, options->NatdListenAddress, - options->NatdPort, "127.0.0.1", + if (retry_listeners(CONN_TYPE_AP_NATD_LISTENER, options->NATDListenAddress, + options->NATDPort, "127.0.0.1", replaced_conns, new_conns, 0, AF_INET)<0) return -1; @@ -1643,6 +2002,7 @@ connection_bucket_write_limit(connection_t *conn, time_t now) int base = connection_speaks_cells(conn) ? CELL_NETWORK_SIZE : RELAY_PAYLOAD_SIZE; int priority = conn->type != CONN_TYPE_DIR; + int conn_bucket = (int)conn->outbuf_flushlen; int global_bucket = global_write_bucket; if (!connection_is_rate_limited(conn)) { @@ -1650,12 +2010,22 @@ connection_bucket_write_limit(connection_t *conn, time_t now) return conn->outbuf_flushlen; } + if (connection_speaks_cells(conn)) { + /* use the per-conn write limit if it's lower, but if it's less + * than zero just use zero */ + or_connection_t *or_conn = TO_OR_CONN(conn); + if (conn->state == OR_CONN_STATE_OPEN) + if (or_conn->write_bucket < conn_bucket) + conn_bucket = or_conn->write_bucket >= 0 ? + or_conn->write_bucket : 0; + } + if (connection_counts_as_relayed_traffic(conn, now) && global_relayed_write_bucket <= global_write_bucket) global_bucket = global_relayed_write_bucket; - return connection_bucket_round_robin(base, priority, global_bucket, - conn->outbuf_flushlen); + return connection_bucket_round_robin(base, priority, + global_bucket, conn_bucket); } /** Return 1 if the global write buckets are low enough that we @@ -1709,14 +2079,12 @@ global_write_bucket_low(connection_t *conn, size_t attempt, int priority) return 0; } -/** We just read num_read and wrote num_written onto conn. - * Decrement buckets appropriately. */ +/** We just read <b>num_read</b> and wrote <b>num_written</b> bytes + * onto <b>conn</b>. Decrement buckets appropriately. */ static void connection_buckets_decrement(connection_t *conn, time_t now, size_t num_read, size_t num_written) { - if (!connection_is_rate_limited(conn)) - return; /* local IPs are free */ if (num_written >= INT_MAX || num_read >= INT_MAX) { log_err(LD_BUG, "Value out of range. num_read=%lu, num_written=%lu, " "connection type=%s, state=%s", @@ -1728,10 +2096,24 @@ connection_buckets_decrement(connection_t *conn, time_t now, tor_fragile_assert(); } - if (num_read > 0) + /* Count bytes of answering direct and tunneled directory requests */ + if (conn->type == CONN_TYPE_DIR && conn->purpose == DIR_PURPOSE_SERVER) { + if (num_read > 0) + rep_hist_note_dir_bytes_read(num_read, now); + if (num_written > 0) + rep_hist_note_dir_bytes_written(num_written, now); + } + + if (!connection_is_rate_limited(conn)) + return; /* local IPs are free */ + if (num_read > 0) { rep_hist_note_bytes_read(num_read, now); - if (num_written > 0) + } + if (num_written > 0) { rep_hist_note_bytes_written(num_written, now); + } + if (conn->type == CONN_TYPE_EXIT) + rep_hist_note_exit_bytes(conn->port, num_written, num_read); if (connection_counts_as_relayed_traffic(conn, now)) { global_relayed_read_bucket -= (int)num_read; @@ -1739,8 +2121,10 @@ connection_buckets_decrement(connection_t *conn, time_t now, } global_read_bucket -= (int)num_read; global_write_bucket -= (int)num_written; - if (connection_speaks_cells(conn) && conn->state == OR_CONN_STATE_OPEN) + if (connection_speaks_cells(conn) && conn->state == OR_CONN_STATE_OPEN) { TO_OR_CONN(conn)->read_bucket -= (int)num_read; + TO_OR_CONN(conn)->write_bucket -= (int)num_written; + } } /** If we have exhausted our global buckets, or the buckets for conn, @@ -1779,12 +2163,10 @@ connection_consider_empty_write_buckets(connection_t *conn) } else if (connection_counts_as_relayed_traffic(conn, approx_time()) && global_relayed_write_bucket <= 0) { reason = "global relayed write bucket exhausted. Pausing."; -#if 0 } else if (connection_speaks_cells(conn) && conn->state == OR_CONN_STATE_OPEN && TO_OR_CONN(conn)->write_bucket <= 0) { reason = "connection write bucket exhausted. Pausing."; -#endif } else return; /* all good, no need to stop it */ @@ -1880,14 +2262,19 @@ connection_bucket_refill(int seconds_elapsed, time_t now) { if (connection_speaks_cells(conn)) { or_connection_t *or_conn = TO_OR_CONN(conn); - if (connection_read_bucket_should_increase(or_conn)) { + if (connection_bucket_should_increase(or_conn->read_bucket, or_conn)) { connection_bucket_refill_helper(&or_conn->read_bucket, or_conn->bandwidthrate, or_conn->bandwidthburst, seconds_elapsed, "or_conn->read_bucket"); - //log_fn(LOG_DEBUG,"Receiver bucket %d now %d.", i, - // conn->read_bucket); + } + if (connection_bucket_should_increase(or_conn->write_bucket, or_conn)) { + connection_bucket_refill_helper(&or_conn->write_bucket, + or_conn->bandwidthrate, + or_conn->bandwidthburst, + seconds_elapsed, + "or_conn->write_bucket"); } } @@ -1908,8 +2295,10 @@ connection_bucket_refill(int seconds_elapsed, time_t now) if (conn->write_blocked_on_bw == 1 && global_write_bucket > 0 /* and we're allowed to write */ && (!connection_counts_as_relayed_traffic(conn, now) || - global_relayed_write_bucket > 0)) { - /* even if we're relayed traffic */ + global_relayed_write_bucket > 0) /* even if it's relayed traffic */ + && (!connection_speaks_cells(conn) || + conn->state != OR_CONN_STATE_OPEN || + TO_OR_CONN(conn)->write_bucket > 0)) { LOG_FN_CONN(conn, (LOG_DEBUG,LD_NET, "waking up conn (fd %d) for write", conn->s)); conn->write_blocked_on_bw = 0; @@ -1918,17 +2307,17 @@ connection_bucket_refill(int seconds_elapsed, time_t now) }); } -/** Is the receiver bucket for connection <b>conn</b> low enough that we +/** Is the <b>bucket</b> for connection <b>conn</b> low enough that we * should add another pile of tokens to it? */ static int -connection_read_bucket_should_increase(or_connection_t *conn) +connection_bucket_should_increase(int bucket, or_connection_t *conn) { tor_assert(conn); if (conn->_base.state != OR_CONN_STATE_OPEN) return 0; /* only open connections play the rate limiting game */ - if (conn->read_bucket >= conn->bandwidthburst) + if (bucket >= conn->bandwidthburst) return 0; return 1; @@ -1946,10 +2335,10 @@ connection_read_bucket_should_increase(or_connection_t *conn) * Mark the connection and return -1 if you want to close it, else * return 0. */ -int -connection_handle_read(connection_t *conn) +static int +connection_handle_read_impl(connection_t *conn) { - int max_to_read=-1, try_to_read; + ssize_t max_to_read=-1, try_to_read; size_t before, n_read = 0; int socket_error = 0; @@ -1975,7 +2364,7 @@ connection_handle_read(connection_t *conn) return 0; } -loop_again: + loop_again: try_to_read = max_to_read; tor_assert(!conn->marked_for_close); @@ -2016,12 +2405,16 @@ loop_again: return -1; } if (conn->linked_conn) { - /* The other side's handle_write will never actually get called, so + /* The other side's handle_write() will never actually get called, so * we need to invoke the appropriate callbacks ourself. */ connection_t *linked = conn->linked_conn; if (n_read) { - /* Probably a no-op, but hey. */ + /* Probably a no-op, since linked conns typically don't count for + * bandwidth rate limiting. But do it anyway so we can keep stats + * accurately. Note that since we read the bytes from conn, and + * we're writing the bytes onto the linked connection, we count + * these as <i>written</i> bytes. */ connection_buckets_decrement(linked, approx_time(), 0, n_read); if (connection_flushed_some(linked) < 0) @@ -2033,7 +2426,7 @@ loop_again: if (!buf_datalen(linked->outbuf) && conn->active_on_link) connection_stop_reading_from_linked_conn(conn); } - /* If we hit the EOF, call connection_reached_eof. */ + /* If we hit the EOF, call connection_reached_eof(). */ if (!conn->marked_for_close && conn->inbuf_reached_eof && connection_reached_eof(conn) < 0) { @@ -2042,6 +2435,16 @@ loop_again: return 0; } +int +connection_handle_read(connection_t *conn) +{ + int res; + + tor_gettimeofday_cache_clear(); + res = connection_handle_read_impl(conn); + return res; +} + /** Pull in new bytes from conn-\>s or conn-\>linked_conn onto conn-\>inbuf, * either directly or via TLS. Reduce the token buckets by the number of bytes * read. @@ -2053,7 +2456,8 @@ loop_again: * Return -1 if we want to break conn, else return 0. */ static int -connection_read_to_buf(connection_t *conn, int *max_to_read, int *socket_error) +connection_read_to_buf(connection_t *conn, ssize_t *max_to_read, + int *socket_error) { int result; ssize_t at_most = *max_to_read; @@ -2075,7 +2479,7 @@ connection_read_to_buf(connection_t *conn, int *max_to_read, int *socket_error) } if (connection_speaks_cells(conn) && - conn->state > OR_CONN_STATE_PROXY_READING) { + conn->state > OR_CONN_STATE_PROXY_HANDSHAKING) { int pending; or_connection_t *or_conn = TO_OR_CONN(conn); size_t initial_size; @@ -2171,15 +2575,19 @@ connection_read_to_buf(connection_t *conn, int *max_to_read, int *socket_error) n_read = (size_t) result; } - if (n_read > 0) { /* change *max_to_read */ - /*XXXX021 check for overflow*/ - *max_to_read = (int)(at_most - n_read); - } + if (n_read > 0) { + /* change *max_to_read */ + *max_to_read = at_most - n_read; - if (conn->type == CONN_TYPE_AP) { - edge_connection_t *edge_conn = TO_EDGE_CONN(conn); - /*XXXX021 check for overflow*/ - edge_conn->n_read += (int)n_read; + /* Update edge_conn->n_read */ + if (conn->type == CONN_TYPE_AP) { + edge_connection_t *edge_conn = TO_EDGE_CONN(conn); + /* Check for overflow: */ + if (PREDICT_LIKELY(UINT32_MAX - edge_conn->n_read > n_read)) + edge_conn->n_read += (int)n_read; + else + edge_conn->n_read = UINT32_MAX; + } } connection_buckets_decrement(conn, approx_time(), n_read, n_written); @@ -2243,8 +2651,8 @@ connection_outbuf_too_full(connection_t *conn) * Mark the connection and return -1 if you want to close it, else * return 0. */ -int -connection_handle_write(connection_t *conn, int force) +static int +connection_handle_write_impl(connection_t *conn, int force) { int e; socklen_t len=(socklen_t)sizeof(e); @@ -2259,7 +2667,7 @@ connection_handle_write(connection_t *conn, int force) return 0; /* do nothing */ if (conn->in_flushed_some) { - log_warn(LD_BUG, "called recursively from inside conn->in_flushed_some()"); + log_warn(LD_BUG, "called recursively from inside conn->in_flushed_some"); return 0; } @@ -2303,7 +2711,7 @@ connection_handle_write(connection_t *conn, int force) : connection_bucket_write_limit(conn, now); if (connection_speaks_cells(conn) && - conn->state > OR_CONN_STATE_PROXY_READING) { + conn->state > OR_CONN_STATE_PROXY_HANDSHAKING) { or_connection_t *or_conn = TO_OR_CONN(conn); if (conn->state == OR_CONN_STATE_TLS_HANDSHAKING || conn->state == OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING) { @@ -2322,6 +2730,13 @@ connection_handle_write(connection_t *conn, int force) /* else open, or closing */ result = flush_buf_tls(or_conn->tls, conn->outbuf, max_to_write, &conn->outbuf_flushlen); + + /* If we just flushed the last bytes, check if this tunneled dir + * request is done. */ + if (buf_datalen(conn->outbuf) == 0 && conn->dirreq_id) + geoip_change_dirreq_state(conn->dirreq_id, DIRREQ_TUNNELED, + DIRREQ_OR_CONN_BUFFER_FLUSHED); + switch (result) { CASE_TOR_TLS_ERROR_ANY: case TOR_TLS_CLOSE: @@ -2341,8 +2756,8 @@ connection_handle_write(connection_t *conn, int force) if (!connection_is_reading(conn)) { connection_stop_writing(conn); conn->write_blocked_on_bw = 1; - /* we'll start reading again when the next second arrives, - * and then also start writing again. + /* we'll start reading again when we get more tokens in our + * read bucket; then we'll start writing again too. */ } /* else no problem, we're already reading */ @@ -2371,10 +2786,13 @@ connection_handle_write(connection_t *conn, int force) n_written = (size_t) result; } - if (conn->type == CONN_TYPE_AP) { + if (n_written && conn->type == CONN_TYPE_AP) { edge_connection_t *edge_conn = TO_EDGE_CONN(conn); - /*XXXX021 check for overflow.*/ - edge_conn->n_written += (int)n_written; + /* Check for overflow: */ + if (PREDICT_LIKELY(UINT32_MAX - edge_conn->n_written > n_written)) + edge_conn->n_written += (int)n_written; + else + edge_conn->n_written = UINT32_MAX; } connection_buckets_decrement(conn, approx_time(), n_read, n_written); @@ -2404,6 +2822,15 @@ connection_handle_write(connection_t *conn, int force) return 0; } +int +connection_handle_write(connection_t *conn, int force) +{ + int res; + tor_gettimeofday_cache_clear(); + res = connection_handle_write_impl(conn, force); + return res; +} + /** OpenSSL TLS record size is 16383; this is close. The goal here is to * push data out as soon as we know there's enough for a TLS record, so * during periods of high load we won't read entire megabytes from @@ -2577,13 +3004,11 @@ connection_get_by_type_state(int type, int state) /** Return a connection of type <b>type</b> that has rendquery equal * to <b>rendquery</b>, and that is not marked for close. If state - * is non-zero, conn must be of that state too. If rendversion is - * nonnegative, conn must be fetching that rendversion, too. + * is non-zero, conn must be of that state too. */ connection_t * connection_get_by_type_state_rendquery(int type, int state, - const char *rendquery, - int rendversion) + const char *rendquery) { smartlist_t *conns = get_connection_array(); @@ -2598,8 +3023,6 @@ connection_get_by_type_state_rendquery(int type, int state, (!state || state == conn->state)) { if (type == CONN_TYPE_DIR && TO_DIR_CONN(conn)->rend_data && - (rendversion < 0 || - rendversion == TO_DIR_CONN(conn)->rend_data->rend_desc_version) && !rend_cmp_service_ids(rendquery, TO_DIR_CONN(conn)->rend_data->onion_address)) return conn; @@ -2717,10 +3140,10 @@ alloc_http_authenticator(const char *authenticator) static void client_check_address_changed(int sock) { - uint32_t iface_ip, ip_out; + uint32_t iface_ip, ip_out; /* host order */ struct sockaddr_in out_addr; socklen_t out_addr_len = (socklen_t) sizeof(out_addr); - uint32_t *ip; + uint32_t *ip; /* host order */ if (!last_interface_ip) get_interface_address(LOG_INFO, &last_interface_ip); @@ -2734,7 +3157,7 @@ client_check_address_changed(int sock) return; } - /* Okay. If we've used this address previously, we're okay. */ + /* If we've used this address previously, we're okay. */ ip_out = ntohl(out_addr.sin_addr.s_addr); SMARTLIST_FOREACH(outgoing_addrs, uint32_t*, ip_ptr, if (*ip_ptr == ip_out) return; @@ -2830,6 +3253,8 @@ connection_flushed_some(connection_t *conn) r = connection_dirserv_flushed_some(TO_DIR_CONN(conn)); } else if (conn->type == CONN_TYPE_OR) { r = connection_or_flushed_some(TO_OR_CONN(conn)); + } else if (CONN_IS_EDGE(conn)) { + r = connection_edge_flushed_some(TO_EDGE_CONN(conn)); } conn->in_flushed_some = 0; return r; @@ -3033,7 +3458,7 @@ assert_connection_ok(connection_t *conn, time_t now) } // tor_assert(conn->addr && conn->port); tor_assert(conn->address); - if (conn->state > OR_CONN_STATE_PROXY_READING) + if (conn->state > OR_CONN_STATE_PROXY_HANDSHAKING) tor_assert(or_conn->tls); } diff --git a/src/or/connection.h b/src/or/connection.h new file mode 100644 index 000000000..576d3a63e --- /dev/null +++ b/src/or/connection.h @@ -0,0 +1,100 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file connection.h + * \brief Header file for connection.c. + **/ + +#ifndef _TOR_CONNECTION_H +#define _TOR_CONNECTION_H + +const char *conn_type_to_string(int type); +const char *conn_state_to_string(int type, int state); + +dir_connection_t *dir_connection_new(int socket_family); +or_connection_t *or_connection_new(int socket_family); +edge_connection_t *edge_connection_new(int type, int socket_family); +control_connection_t *control_connection_new(int socket_family); +connection_t *connection_new(int type, int socket_family); + +void connection_link_connections(connection_t *conn_a, connection_t *conn_b); +void connection_free(connection_t *conn); +void connection_free_all(void); +void connection_about_to_close_connection(connection_t *conn); +void connection_close_immediate(connection_t *conn); +void _connection_mark_for_close(connection_t *conn,int line, const char *file); + +#define connection_mark_for_close(c) \ + _connection_mark_for_close((c), __LINE__, _SHORT_FILE_) + +void connection_expire_held_open(void); + +int connection_connect(connection_t *conn, const char *address, + const tor_addr_t *addr, + uint16_t port, int *socket_error); + +int connection_proxy_connect(connection_t *conn, int type); +int connection_read_proxy_handshake(connection_t *conn); + +int retry_all_listeners(smartlist_t *replaced_conns, + smartlist_t *new_conns); + +ssize_t connection_bucket_write_limit(connection_t *conn, time_t now); +int global_write_bucket_low(connection_t *conn, size_t attempt, int priority); +void connection_bucket_init(void); +void connection_bucket_refill(int seconds_elapsed, time_t now); + +int connection_handle_read(connection_t *conn); + +int connection_fetch_from_buf(char *string, size_t len, connection_t *conn); + +int connection_wants_to_flush(connection_t *conn); +int connection_outbuf_too_full(connection_t *conn); +int connection_handle_write(connection_t *conn, int force); +void _connection_write_to_buf_impl(const char *string, size_t len, + connection_t *conn, int zlib); +static void connection_write_to_buf(const char *string, size_t len, + connection_t *conn); +static void connection_write_to_buf_zlib(const char *string, size_t len, + dir_connection_t *conn, int done); +static INLINE void +connection_write_to_buf(const char *string, size_t len, connection_t *conn) +{ + _connection_write_to_buf_impl(string, len, conn, 0); +} +static INLINE void +connection_write_to_buf_zlib(const char *string, size_t len, + dir_connection_t *conn, int done) +{ + _connection_write_to_buf_impl(string, len, TO_CONN(conn), done ? -1 : 1); +} + +connection_t *connection_get_by_global_id(uint64_t id); + +connection_t *connection_get_by_type(int type); +connection_t *connection_get_by_type_purpose(int type, int purpose); +connection_t *connection_get_by_type_addr_port_purpose(int type, + const tor_addr_t *addr, + uint16_t port, int purpose); +connection_t *connection_get_by_type_state(int type, int state); +connection_t *connection_get_by_type_state_rendquery(int type, int state, + const char *rendquery); + +#define connection_speaks_cells(conn) ((conn)->type == CONN_TYPE_OR) +int connection_is_listener(connection_t *conn); +int connection_state_is_open(connection_t *conn); +int connection_state_is_connecting(connection_t *conn); + +char *alloc_http_authenticator(const char *authenticator); + +void assert_connection_ok(connection_t *conn, time_t now); +int connection_or_nonopen_was_started_here(or_connection_t *conn); +void connection_dump_buffer_mem_stats(int severity); +void remove_file_if_very_old(const char *fname, time_t now); + +#endif + diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index 5609c1df4..2c1196c0c 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -10,6 +10,28 @@ **/ #include "or.h" +#include "buffers.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "dns.h" +#include "dnsserv.h" +#include "dirserv.h" +#include "hibernate.h" +#include "main.h" +#include "policies.h" +#include "reasons.h" +#include "relay.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rendservice.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" #ifdef HAVE_LINUX_TYPES_H #include <linux/types.h> @@ -125,7 +147,7 @@ connection_edge_process_inbuf(edge_connection_t *conn, int package_partial) return 0; case AP_CONN_STATE_OPEN: case EXIT_CONN_STATE_OPEN: - if (connection_edge_package_raw_inbuf(conn, package_partial) < 0) { + if (connection_edge_package_raw_inbuf(conn, package_partial, NULL) < 0) { /* (We already sent an end cell if possible) */ connection_mark_for_close(TO_CONN(conn)); return -1; @@ -279,6 +301,23 @@ connection_edge_end_errno(edge_connection_t *conn) return connection_edge_end(conn, reason); } +/** We just wrote some data to <b>conn</b>; act appropriately. + * + * (That is, if it's open, consider sending a stream-level sendme cell if we + * have just flushed enough.) + */ +int +connection_edge_flushed_some(edge_connection_t *conn) +{ + switch (conn->_base.state) { + case AP_CONN_STATE_OPEN: + case EXIT_CONN_STATE_OPEN: + connection_edge_consider_sending_sendme(conn); + break; + } + return 0; +} + /** Connection <b>conn</b> has finished writing and has no bytes left on * its outbuf. * @@ -330,11 +369,13 @@ connection_edge_finished_connecting(edge_connection_t *edge_conn) tor_assert(conn->state == EXIT_CONN_STATE_CONNECTING); log_info(LD_EXIT,"Exit connection to %s:%u (%s) established.", - escaped_safe_str(conn->address),conn->port, + escaped_safe_str(conn->address), conn->port, safe_str(fmt_addr(&conn->addr))); + rep_hist_note_exit_stream_opened(conn->port); + conn->state = EXIT_CONN_STATE_OPEN; - connection_watch_events(conn, EV_READ); /* stop writing, continue reading */ + connection_watch_events(conn, READ_EVENT); /* stop writing, keep reading */ if (connection_wants_to_flush(conn)) /* in case there are any queued relay * cells */ connection_start_writing(conn); @@ -375,13 +416,16 @@ connection_edge_finished_connecting(edge_connection_t *edge_conn) static int compute_retry_timeout(edge_connection_t *conn) { + int timeout = get_options()->CircuitStreamTimeout; + if (timeout) /* if our config options override the default, use them */ + return timeout; if (conn->num_socks_retries < 2) /* try 0 and try 1 */ return 10; return 15; } /** Find all general-purpose AP streams waiting for a response that sent their - * begin/resolve cell >=15 seconds ago. Detach from their current circuit, and + * begin/resolve cell too long ago. Detach from their current circuit, and * mark their current circuit as unsuitable for new streams. Then call * connection_ap_handshake_attach_circuit() to attach to a new circuit (if * available) or launch a new one. @@ -423,7 +467,8 @@ connection_ap_expire_beginning(void) log_fn(severity, LD_APP, "Tried for %d seconds to get a connection to %s:%d. " "Giving up. (%s)", - seconds_since_born, safe_str(conn->socks_request->address), + seconds_since_born, + safe_str_client(conn->socks_request->address), conn->socks_request->port, conn_state_to_string(CONN_TYPE_AP, conn->_base.state)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TIMEOUT); @@ -440,7 +485,7 @@ connection_ap_expire_beginning(void) circ = circuit_get_by_edge_conn(conn); if (!circ) { /* it's vanished? */ log_info(LD_APP,"Conn is waiting (address %s), but lost its circ.", - safe_str(conn->socks_request->address)); + safe_str_client(conn->socks_request->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TIMEOUT); continue; } @@ -450,7 +495,7 @@ connection_ap_expire_beginning(void) "Rend stream is %d seconds late. Giving up on address" " '%s.onion'.", seconds_idle, - safe_str(conn->socks_request->address)); + safe_str_client(conn->socks_request->address)); connection_edge_end(conn, END_STREAM_REASON_TIMEOUT); connection_mark_unattached_ap(conn, END_STREAM_REASON_TIMEOUT); } @@ -460,7 +505,8 @@ connection_ap_expire_beginning(void) log_fn(cutoff < 15 ? LOG_INFO : severity, LD_APP, "We tried for %d seconds to connect to '%s' using exit '%s'." " Retrying on a new circuit.", - seconds_idle, safe_str(conn->socks_request->address), + seconds_idle, + safe_str_client(conn->socks_request->address), conn->cpath_layer ? conn->cpath_layer->extend_info->nickname : "*unnamed*"); /* send an end down the circuit */ @@ -471,6 +517,7 @@ connection_ap_expire_beginning(void) /* kludge to make us not try this circuit again, yet to allow * current streams on it to survive if they can: make it * unattractive to use for new streams */ + /* XXXX023 this is a kludgy way to do this. */ tor_assert(circ->timestamp_dirty); circ->timestamp_dirty -= options->MaxCircuitDirtiness; /* give our stream another 'cutoff' seconds to try */ @@ -511,7 +558,7 @@ connection_ap_attach_pending(void) /** Tell any AP streams that are waiting for a one-hop tunnel to * <b>failed_digest</b> that they are going to fail. */ -/* XXX022 We should get rid of this function, and instead attach +/* XXX023 We should get rid of this function, and instead attach * one-hop streams to circ->p_streams so they get marked in * circuit_mark_for_close like normal p_streams. */ void @@ -577,8 +624,8 @@ circuit_discard_optional_exit_enclaves(extend_info_t *info) tor_assert(edge_conn->socks_request); if (edge_conn->chosen_exit_optional) { log_info(LD_APP, "Giving up on enclave exit '%s' for destination %s.", - safe_str(edge_conn->chosen_exit_name), - escaped_safe_str(edge_conn->socks_request->address)); + safe_str_client(edge_conn->chosen_exit_name), + escaped_safe_str_client(edge_conn->socks_request->address)); edge_conn->chosen_exit_optional = 0; tor_free(edge_conn->chosen_exit_name); /* clears it */ /* if this port is dangerous, warn or reject it now that we don't @@ -683,7 +730,11 @@ addressmap_init(void) static void addressmap_ent_free(void *_ent) { - addressmap_entry_t *ent = _ent; + addressmap_entry_t *ent; + if (!_ent) + return; + + ent = _ent; tor_free(ent->new_address); tor_free(ent); } @@ -692,7 +743,11 @@ addressmap_ent_free(void *_ent) static void addressmap_virtaddress_ent_free(void *_ent) { - virtaddress_entry_t *ent = _ent; + virtaddress_entry_t *ent; + if (!_ent) + return; + + ent = _ent; tor_free(ent->ipv4_address); tor_free(ent->hostname_address); tor_free(ent); @@ -744,7 +799,8 @@ clear_trackexithost_mappings(const char *exitname) tor_strlower(suffix); STRMAP_FOREACH_MODIFY(addressmap, address, addressmap_entry_t *, ent) { - if (ent->source == ADDRMAPSRC_TRACKEXIT && !strcmpend(address, suffix)) { + if (ent->source == ADDRMAPSRC_TRACKEXIT && + !strcmpend(ent->new_address, suffix)) { addressmap_ent_remove(address, ent); MAP_DEL_CURRENT(address); } @@ -753,6 +809,56 @@ clear_trackexithost_mappings(const char *exitname) tor_free(suffix); } +/** Remove all TRACKEXIT mappings from the addressmap for which the target + * host is unknown or no longer allowed. */ +void +addressmap_clear_excluded_trackexithosts(or_options_t *options) +{ + const routerset_t *allow_nodes = options->ExitNodes; + const routerset_t *exclude_nodes = options->_ExcludeExitNodesUnion; + + if (!addressmap) + return; + if (routerset_is_empty(allow_nodes)) + allow_nodes = NULL; + if (allow_nodes == NULL && routerset_is_empty(exclude_nodes)) + return; + + STRMAP_FOREACH_MODIFY(addressmap, address, addressmap_entry_t *, ent) { + size_t len; + const char *target = ent->new_address, *dot; + char *nodename; + routerinfo_t *ri; /* XXX023 Use node_t. */ + + if (strcmpend(target, ".exit")) { + /* Not a .exit mapping */ + continue; + } else if (ent->source != ADDRMAPSRC_TRACKEXIT) { + /* Not a trackexit mapping. */ + continue; + } + len = strlen(target); + if (len < 6) + continue; /* malformed. */ + dot = target + len - 6; /* dot now points to just before .exit */ + dot = strrchr(dot, '.'); /* dot now points to the . before .exit or NULL */ + if (!dot) { + nodename = tor_strndup(target, len-5); + } else { + nodename = tor_strndup(dot+1, strlen(dot+1)-5); + } + ri = router_get_by_nickname(nodename, 0); + tor_free(nodename); + if (!ri || + (allow_nodes && !routerset_contains_router(allow_nodes, ri)) || + routerset_contains_router(exclude_nodes, ri)) { + /* We don't know this one, or we want to be rid of it. */ + addressmap_ent_remove(address, ent); + MAP_DEL_CURRENT(address); + } + } STRMAP_FOREACH_END; +} + /** Remove all entries from the addressmap that were set via the * configuration file or the command line. */ void @@ -782,14 +888,11 @@ addressmap_clean(time_t now) void addressmap_free_all(void) { - if (addressmap) { - strmap_free(addressmap, addressmap_ent_free); - addressmap = NULL; - } - if (virtaddress_reversemap) { - strmap_free(virtaddress_reversemap, addressmap_virtaddress_ent_free); - virtaddress_reversemap = NULL; - } + strmap_free(addressmap, addressmap_ent_free); + addressmap = NULL; + + strmap_free(virtaddress_reversemap, addressmap_virtaddress_ent_free); + virtaddress_reversemap = NULL; } /** Look at address, and rewrite it until it doesn't want any @@ -816,9 +919,9 @@ addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out) return (rewrites > 0); /* done, no rewrite needed */ } - cp = tor_strdup(escaped_safe_str(ent->new_address)); + cp = tor_strdup(escaped_safe_str_client(ent->new_address)); log_info(LD_APP, "Addressmap: rewriting %s to %s", - escaped_safe_str(address), cp); + escaped_safe_str_client(address), cp); if (ent->expires > 1 && ent->expires < expires) expires = ent->expires; tor_free(cp); @@ -826,7 +929,7 @@ addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out) } log_warn(LD_CONFIG, "Loop detected: we've rewritten %s 16 times! Using it as-is.", - escaped_safe_str(address)); + escaped_safe_str_client(address)); /* it's fine to rewrite a rewrite, but don't loop forever */ if (expires_out) *expires_out = TIME_MAX; @@ -848,9 +951,9 @@ addressmap_rewrite_reverse(char *address, size_t maxlen, time_t *expires_out) tor_snprintf(s, len, "REVERSE[%s]", address); ent = strmap_get(addressmap, s); if (ent) { - cp = tor_strdup(escaped_safe_str(ent->new_address)); + cp = tor_strdup(escaped_safe_str_client(ent->new_address)); log_info(LD_APP, "Rewrote reverse lookup %s -> %s", - escaped_safe_str(s), cp); + escaped_safe_str_client(s), cp); tor_free(cp); strlcpy(address, ent->new_address, maxlen); r = 1; @@ -912,7 +1015,9 @@ addressmap_register(const char *address, char *new_address, time_t expires, if (expires > 1) { log_info(LD_APP,"Temporary addressmap ('%s' to '%s') not performed, " "since it's already mapped to '%s'", - safe_str(address), safe_str(new_address), safe_str(ent->new_address)); + safe_str_client(address), + safe_str_client(new_address), + safe_str_client(ent->new_address)); tor_free(new_address); return; } @@ -931,7 +1036,8 @@ addressmap_register(const char *address, char *new_address, time_t expires, ent->source = source; log_info(LD_CONFIG, "Addressmap: (re)mapped '%s' to '%s'", - safe_str(address), safe_str(ent->new_address)); + safe_str_client(address), + safe_str_client(ent->new_address)); control_event_address_mapped(address, ent->new_address, expires, NULL); } @@ -951,7 +1057,8 @@ client_dns_incr_failures(const char *address) if (ent->num_resolve_failures < SHORT_MAX) ++ent->num_resolve_failures; /* don't overflow */ log_info(LD_APP, "Address %s now has %d resolve failures.", - safe_str(address), ent->num_resolve_failures); + safe_str_client(address), + ent->num_resolve_failures); return ent->num_resolve_failures; } @@ -1247,8 +1354,10 @@ addressmap_register_virtual_address(int type, char *new_address) log_warn(LD_BUG, "Internal confusion: I thought that '%s' was mapped to by " "'%s', but '%s' really maps to '%s'. This is a harmless bug.", - safe_str(new_address), safe_str(*addrp), safe_str(*addrp), - ent?safe_str(ent->new_address):"(nothing)"); + safe_str_client(new_address), + safe_str_client(*addrp), + safe_str_client(*addrp), + ent?safe_str_client(ent->new_address):"(nothing)"); } tor_free(*addrp); @@ -1276,7 +1385,8 @@ addressmap_register_virtual_address(int type, char *new_address) (type == RESOLVED_TYPE_IPV4) ? vent->ipv4_address : vent->hostname_address)); log_info(LD_APP, "Map from %s to %s okay.", - safe_str(*addrp),safe_str(new_address)); + safe_str_client(*addrp), + safe_str_client(new_address)); } #endif @@ -1391,6 +1501,26 @@ consider_plaintext_ports(edge_connection_t *conn, uint16_t port) * different one? */ #define TRACKHOSTEXITS_RETRIES 5 +/** Call connection_ap_handshake_rewrite_and_attach() unless a controller + * asked us to leave streams unattached. Return 0 in that case. + * + * See connection_ap_handshake_rewrite_and_attach()'s + * documentation for arguments and return value. + */ +int +connection_ap_rewrite_and_attach_if_allowed(edge_connection_t *conn, + origin_circuit_t *circ, + crypt_path_t *cpath) +{ + or_options_t *options = get_options(); + + if (options->LeaveStreamsUnattached) { + conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; + return 0; + } + return connection_ap_handshake_rewrite_and_attach(conn, circ, cpath); +} + /** Connection <b>conn</b> just finished its socks handshake, or the * controller asked us to take care of it. If <b>circ</b> is defined, * then that's where we'll want to attach it. Otherwise we have to @@ -1415,16 +1545,20 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, hostname_type_t addresstype; or_options_t *options = get_options(); struct in_addr addr_tmp; + /* We set this to true if this is an address we should automatically + * remap to a local address in VirtualAddrNetwork */ int automap = 0; char orig_address[MAX_SOCKS_ADDR_LEN]; time_t map_expires = TIME_MAX; + /* This will be set to true iff the address starts out as a non-.exit + address, and we remap it to one because of an entry in the addressmap. */ int remapped_to_exit = 0; time_t now = time(NULL); tor_strlower(socks->address); /* normalize it */ strlcpy(orig_address, socks->address, sizeof(orig_address)); log_debug(LD_APP,"Client asked for %s:%d", - safe_str(socks->address), + safe_str_client(socks->address), socks->port); if (socks->command == SOCKS_COMMAND_RESOLVE && @@ -1446,7 +1580,8 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, return -1; } log_info(LD_APP, "Automapping %s to %s", - escaped_safe_str(socks->address), safe_str(new_addr)); + escaped_safe_str_client(socks->address), + safe_str_client(new_addr)); strlcpy(socks->address, new_addr, sizeof(socks->address)); } } @@ -1503,7 +1638,7 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, * information. */ log_warn(LD_APP,"Missing mapping for virtual address '%s'. Refusing.", - socks->address); /* don't safe_str() this yet. */ + safe_str_client(socks->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_INTERNAL); return -1; } @@ -1511,11 +1646,12 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, /* Parse the address provided by SOCKS. Modify it in-place if it * specifies a hidden-service (.onion) or particular exit node (.exit). */ - addresstype = parse_extended_hostname(socks->address); + addresstype = parse_extended_hostname(socks->address, + remapped_to_exit || options->AllowDotExit); if (addresstype == BAD_HOSTNAME) { log_warn(LD_APP, "Invalid onion hostname %s; rejecting", - safe_str(socks->address)); + safe_str_client(socks->address)); control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", escaped(socks->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); @@ -1524,38 +1660,61 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, if (addresstype == EXIT_HOSTNAME) { /* foo.exit -- modify conn->chosen_exit_node to specify the exit - * node, and conn->address to hold only the address portion.*/ + * node, and conn->address to hold only the address portion. */ char *s = strrchr(socks->address,'.'); + + /* If StrictNodes is not set, then .exit overrides ExcludeNodes. */ + routerset_t *excludeset = options->StrictNodes ? + options->_ExcludeExitNodesUnion : options->ExcludeExitNodes; + /*XXX023 make this a node_t. */ + routerinfo_t *router; + tor_assert(!automap); if (s) { + /* The address was of the form "(stuff).(name).exit */ if (s[1] != '\0') { conn->chosen_exit_name = tor_strdup(s+1); + router = router_get_by_nickname(conn->chosen_exit_name, 1); if (remapped_to_exit) /* 5 tries before it expires the addressmap */ conn->chosen_exit_retries = TRACKHOSTEXITS_RETRIES; *s = 0; } else { + /* Oops, the address was (stuff)..exit. That's not okay. */ log_warn(LD_APP,"Malformed exit address '%s.exit'. Refusing.", - safe_str(socks->address)); + safe_str_client(socks->address)); control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", escaped(socks->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); return -1; } } else { - routerinfo_t *r; + /* It looks like they just asked for "foo.exit". */ conn->chosen_exit_name = tor_strdup(socks->address); - r = router_get_by_nickname(conn->chosen_exit_name, 1); - *socks->address = 0; - if (r) { - strlcpy(socks->address, r->address, sizeof(socks->address)); - } else { - log_warn(LD_APP, - "Unrecognized server in exit address '%s.exit'. Refusing.", - safe_str(socks->address)); - connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); - return -1; + router = router_get_by_nickname(conn->chosen_exit_name, 1); + if (router) { + *socks->address = 0; + strlcpy(socks->address, router->address, sizeof(socks->address)); } } + /* Now make sure that the chosen exit exists... */ + if (!router) { + log_warn(LD_APP, + "Unrecognized relay in exit address '%s.exit'. Refusing.", + safe_str_client(socks->address)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + return -1; + } + /* ...and make sure that it isn't excluded. */ + if (routerset_contains_router(excludeset, router)) { + log_warn(LD_APP, + "Excluded relay in exit address '%s.exit'. Refusing.", + safe_str_client(socks->address)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); + return -1; + } + /* XXXX022-1090 Should we also allow foo.bar.exit if ExitNodes is set and + Bar is not listed in it? I say yes, but our revised manpage branch + implies no. */ } if (addresstype != ONION_HOSTNAME) { @@ -1565,7 +1724,7 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, escaped(socks->address)); log_warn(LD_APP, "Destination '%s' seems to be an invalid hostname. Failing.", - safe_str(socks->address)); + safe_str_client(socks->address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); return -1; } @@ -1574,18 +1733,6 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, uint32_t answer; struct in_addr in; /* Reply to resolves immediately if we can. */ - if (strlen(socks->address) > RELAY_PAYLOAD_SIZE) { - log_warn(LD_APP,"Address to be resolved is too large. Failing."); - control_event_client_status(LOG_WARN, "SOCKS_BAD_HOSTNAME HOSTNAME=%s", - escaped(socks->address)); - connection_ap_handshake_socks_resolved(conn, - RESOLVED_TYPE_ERROR_TRANSIENT, - 0,NULL,-1,TIME_MAX); - connection_mark_unattached_ap(conn, - END_STREAM_REASON_SOCKSPROTOCOL | - END_STREAM_REASON_FLAG_ALREADY_SOCKS_REPLIED); - return -1; - } if (tor_inet_aton(socks->address, &in)) { /* see if it's an IP already */ /* leave it in network order */ answer = in.s_addr; @@ -1608,6 +1755,28 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); return -1; } + if (options->ClientRejectInternalAddresses && + !conn->use_begindir && !conn->chosen_exit_name && !circ) { + tor_addr_t addr; + if (tor_addr_from_str(&addr, socks->address) >= 0 && + tor_addr_is_internal(&addr, 0)) { + /* If this is an explicit private address with no chosen exit node, + * then we really don't want to try to connect to it. That's + * probably an error. */ + if (conn->is_transparent_ap) { + log_warn(LD_NET, + "Rejecting request for anonymous connection to private " + "address %s on a TransPort or NATDPort. Possible loop " + "in your NAT rules?", safe_str_client(socks->address)); + } else { + log_warn(LD_NET, + "Rejecting SOCKS request for anonymous connection to " + "private address %s", safe_str_client(socks->address)); + } + connection_mark_unattached_ap(conn, END_STREAM_REASON_PRIVATE_ADDR); + return -1; + } + } if (!conn->use_begindir && !conn->chosen_exit_name && !circ) { /* see if we can find a suitable enclave exit */ @@ -1616,7 +1785,7 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, if (r) { log_info(LD_APP, "Redirecting address %s to exit at enclave router %s", - safe_str(socks->address), r->nickname); + safe_str_client(socks->address), r->nickname); /* use the hex digest, not nickname, in case there are two routers with this nickname */ conn->chosen_exit_name = @@ -1680,12 +1849,12 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, strlcpy(conn->rend_data->onion_address, socks->address, sizeof(conn->rend_data->onion_address)); log_info(LD_REND,"Got a hidden service request for ID '%s'", - safe_str(conn->rend_data->onion_address)); + safe_str_client(conn->rend_data->onion_address)); /* see if we already have it cached */ r = rend_cache_lookup_entry(conn->rend_data->onion_address, -1, &entry); if (r<0) { log_warn(LD_BUG,"Invalid service name '%s'", - safe_str(conn->rend_data->onion_address)); + safe_str_client(conn->rend_data->onion_address)); connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL); return -1; } @@ -1707,32 +1876,15 @@ connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, if (r==0) { conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT; log_info(LD_REND, "Unknown descriptor %s. Fetching.", - safe_str(conn->rend_data->onion_address)); - /* Fetch both, v0 and v2 rend descriptors in parallel. Use whichever - * arrives first. Exception: When using client authorization, only - * fetch v2 descriptors.*/ + safe_str_client(conn->rend_data->onion_address)); rend_client_refetch_v2_renddesc(conn->rend_data); - if (conn->rend_data->auth_type == REND_NO_AUTH) - rend_client_refetch_renddesc(conn->rend_data->onion_address); } else { /* r > 0 */ - if (now - entry->received < NUM_SECONDS_BEFORE_HS_REFETCH) { - conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; - log_info(LD_REND, "Descriptor is here and fresh enough. Great."); - if (connection_ap_handshake_attach_circuit(conn) < 0) { - if (!conn->_base.marked_for_close) - connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); - return -1; - } - } else { - conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT; - log_info(LD_REND, "Stale descriptor %s. Re-fetching.", - safe_str(conn->rend_data->onion_address)); - /* Fetch both, v0 and v2 rend descriptors in parallel. Use whichever - * arrives first. Exception: When using client authorization, only - * fetch v2 descriptors.*/ - rend_client_refetch_v2_renddesc(conn->rend_data); - if (conn->rend_data->auth_type == REND_NO_AUTH) - rend_client_refetch_renddesc(conn->rend_data->onion_address); + conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; + log_info(LD_REND, "Descriptor is here. Great."); + if (connection_ap_handshake_attach_circuit(conn) < 0) { + if (!conn->_base.marked_for_close) + connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); + return -1; } } return 0; @@ -1920,24 +2072,12 @@ connection_ap_handshake_process_socks(edge_connection_t *conn) return -1; } /* else socks handshake is done, continue processing */ - if (hostname_is_noconnect_address(socks->address)) - { - control_event_stream_status(conn, STREAM_EVENT_NEW, 0); - control_event_stream_status(conn, STREAM_EVENT_CLOSED, 0); - connection_mark_unattached_ap(conn, END_STREAM_REASON_DONE); - return -1; - } - if (SOCKS_COMMAND_IS_CONNECT(socks->command)) control_event_stream_status(conn, STREAM_EVENT_NEW, 0); else control_event_stream_status(conn, STREAM_EVENT_NEW_RESOLVE, 0); - if (options->LeaveStreamsUnattached) { - conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; - return 0; - } - return connection_ap_handshake_rewrite_and_attach(conn, NULL, NULL); + return connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); } /** connection_init_accepted_conn() found a new trans AP conn. @@ -1951,7 +2091,6 @@ int connection_ap_process_transparent(edge_connection_t *conn) { socks_request_t *socks; - or_options_t *options = get_options(); tor_assert(conn); tor_assert(conn->_base.type == CONN_TYPE_AP); @@ -1975,11 +2114,7 @@ connection_ap_process_transparent(edge_connection_t *conn) control_event_stream_status(conn, STREAM_EVENT_NEW, 0); - if (options->LeaveStreamsUnattached) { - conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; - return 0; - } - return connection_ap_handshake_rewrite_and_attach(conn, NULL, NULL); + return connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); } /** connection_edge_process_inbuf() found a conn in state natd_wait. See if @@ -2000,7 +2135,6 @@ connection_ap_process_natd(edge_connection_t *conn) size_t tlen = 30; int err, port_ok; socks_request_t *socks; - or_options_t *options = get_options(); tor_assert(conn); tor_assert(conn->_base.type == CONN_TYPE_AP); @@ -2016,13 +2150,13 @@ connection_ap_process_natd(edge_connection_t *conn) if (err == 0) return 0; if (err < 0) { - log_warn(LD_APP,"Natd handshake failed (DEST too long). Closing"); + log_warn(LD_APP,"NATD handshake failed (DEST too long). Closing"); connection_mark_unattached_ap(conn, END_STREAM_REASON_INVALID_NATD_DEST); return -1; } if (strcmpstart(tmp_buf, "[DEST ")) { - log_warn(LD_APP,"Natd handshake was ill-formed; closing. The client " + log_warn(LD_APP,"NATD handshake was ill-formed; closing. The client " "said: %s", escaped(tmp_buf)); connection_mark_unattached_ap(conn, END_STREAM_REASON_INVALID_NATD_DEST); @@ -2031,7 +2165,7 @@ connection_ap_process_natd(edge_connection_t *conn) daddr = tbuf = &tmp_buf[0] + 6; /* after end of "[DEST " */ if (!(tbuf = strchr(tbuf, ' '))) { - log_warn(LD_APP,"Natd handshake was ill-formed; closing. The client " + log_warn(LD_APP,"NATD handshake was ill-formed; closing. The client " "said: %s", escaped(tmp_buf)); connection_mark_unattached_ap(conn, END_STREAM_REASON_INVALID_NATD_DEST); @@ -2045,7 +2179,7 @@ connection_ap_process_natd(edge_connection_t *conn) socks->port = (uint16_t) tor_parse_long(tbuf, 10, 1, 65535, &port_ok, &daddr); if (!port_ok) { - log_warn(LD_APP,"Natd handshake failed; port %s is ill-formed or out " + log_warn(LD_APP,"NATD handshake failed; port %s is ill-formed or out " "of range.", escaped(tbuf)); connection_mark_unattached_ap(conn, END_STREAM_REASON_INVALID_NATD_DEST); return -1; @@ -2056,13 +2190,9 @@ connection_ap_process_natd(edge_connection_t *conn) control_event_stream_status(conn, STREAM_EVENT_NEW, 0); - if (options->LeaveStreamsUnattached) { - conn->_base.state = AP_CONN_STATE_CONTROLLER_WAIT; - return 0; - } conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT; - return connection_ap_handshake_rewrite_and_attach(conn, NULL, NULL); + return connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); } /** Iterate over the two bytes of stream_id until we get one that is not @@ -2075,7 +2205,7 @@ get_unique_stream_id_by_circ(origin_circuit_t *circ) streamid_t test_stream_id; uint32_t attempts=0; -again: + again: test_stream_id = circ->next_stream_id++; if (++attempts > 1<<16) { /* Make sure we don't loop forever if all stream_id's are used. */ @@ -2112,8 +2242,14 @@ connection_ap_handshake_send_begin(edge_connection_t *ap_conn) ap_conn->stream_id = get_unique_stream_id_by_circ(circ); if (ap_conn->stream_id==0) { + /* XXXX023 Instead of closing this stream, we should make it get + * retried on another circuit. */ connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); - circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_RESOURCELIMIT); + + /* Mark this circuit "unusable for new streams". */ + /* XXXX023 this is a kludgy way to do this. */ + tor_assert(circ->_base.timestamp_dirty); + circ->_base.timestamp_dirty -= get_options()->MaxCircuitDirtiness; return -1; } @@ -2171,9 +2307,14 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) ap_conn->stream_id = get_unique_stream_id_by_circ(circ); if (ap_conn->stream_id==0) { + /* XXXX023 Instead of closing this stream, we should make it get + * retried on another circuit. */ connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); - /*XXXX022 _close_ the circuit because it's full? That sounds dumb. */ - circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_RESOURCELIMIT); + + /* Mark this circuit "unusable for new streams". */ + /* XXXX023 this is a kludgy way to do this. */ + tor_assert(circ->_base.timestamp_dirty); + circ->_base.timestamp_dirty -= get_options()->MaxCircuitDirtiness; return -1; } @@ -2191,7 +2332,7 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) r = tor_addr_parse_reverse_lookup_name(&addr, a, AF_INET, 1); if (r <= 0) { log_warn(LD_APP, "Rejecting ill-formed reverse lookup of %s", - safe_str(a)); + safe_str_client(a)); connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); return -1; } @@ -2199,7 +2340,7 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) r = tor_addr_to_reverse_lookup_name(inaddr_buf, sizeof(inaddr_buf), &addr); if (r < 0) { log_warn(LD_BUG, "Couldn't generate reverse lookup hostname of %s", - safe_str(a)); + safe_str_client(a)); connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); return -1; } @@ -2209,12 +2350,6 @@ connection_ap_handshake_send_resolve(edge_connection_t *ap_conn) tor_assert(payload_len <= (int)sizeof(inaddr_buf)); } - if (payload_len > RELAY_PAYLOAD_SIZE) { - /* This should be impossible: we don't accept addresses this big. */ - connection_mark_unattached_ap(ap_conn, END_STREAM_REASON_INTERNAL); - return -1; - } - log_debug(LD_APP, "Sending relay cell to begin stream %d.", ap_conn->stream_id); @@ -2246,7 +2381,8 @@ connection_ap_make_link(char *address, uint16_t port, edge_connection_t *conn; log_info(LD_APP,"Making internal %s tunnel to %s:%d ...", - want_onehop ? "direct" : "anonymized" , safe_str(address),port); + want_onehop ? "direct" : "anonymized", + safe_str_client(address), port); conn = edge_connection_new(CONN_TYPE_AP, AF_INET); conn->_base.linked = 1; /* so that we can add it safely below. */ @@ -2336,7 +2472,7 @@ tell_controller_about_resolved_result(edge_connection_t *conn, * certain errors or for values that didn't come via DNS. <b>expires</b> is * a time when the answer expires, or -1 or TIME_MAX if there's a good TTL. **/ -/* XXXX022 the use of the ttl and expires fields is nutty. Let's make this +/* XXXX023 the use of the ttl and expires fields is nutty. Let's make this * interface and those that use it less ugly. */ void connection_ap_handshake_socks_resolved(edge_connection_t *conn, @@ -2512,6 +2648,7 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) char *address=NULL; uint16_t port; or_circuit_t *or_circ = NULL; + or_options_t *options = get_options(); assert_circuit_ok(circ); if (!CIRCUIT_IS_ORIGIN(circ)) @@ -2526,7 +2663,7 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) * that we have a stream connected to a circuit, and we don't connect to a * circuit until we have a pending/successful resolve. */ - if (!server_mode(get_options()) && + if (!server_mode(options) && circ->purpose != CIRCUIT_PURPOSE_S_REND_JOINED) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Relay begin cell at non-server. Closing."); @@ -2560,21 +2697,30 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) tor_free(address); return 0; } - if (or_circ && or_circ->is_first_hop && - !get_options()->AllowSingleHopExits) { + if (or_circ && or_circ->p_conn && !options->AllowSingleHopExits && + (or_circ->is_first_hop || + (!connection_or_digest_is_known_relay( + or_circ->p_conn->identity_digest) && + should_refuse_unknown_exits(options)))) { /* Don't let clients use us as a single-hop proxy, unless the user - * has explicitly allowed that in the config. It attracts attackers + * has explicitly allowed that in the config. It attracts attackers * and users who'd be better off with, well, single-hop proxies. */ log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, - "Attempt to open a stream on first hop of circuit. Closing."); + "Attempt by %s to open a stream %s. Closing.", + safe_str(or_circ->p_conn->_base.address), + or_circ->is_first_hop ? "on first hop of circuit" : + "from unknown relay"); relay_send_end_cell_from_edge(rh.stream_id, circ, - END_STREAM_REASON_TORPROTOCOL, NULL); + or_circ->is_first_hop ? + END_STREAM_REASON_TORPROTOCOL : + END_STREAM_REASON_MISC, + NULL); tor_free(address); return 0; } } else if (rh.command == RELAY_COMMAND_BEGIN_DIR) { - if (!directory_permits_begindir_requests(get_options()) || + if (!directory_permits_begindir_requests(options) || circ->purpose != CIRCUIT_PURPOSE_OR) { relay_send_end_cell_from_edge(rh.stream_id, circ, END_STREAM_REASON_NOTDIRECTORY, NULL); @@ -2600,6 +2746,11 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) log_debug(LD_EXIT,"Creating new exit connection."); n_stream = edge_connection_new(CONN_TYPE_EXIT, AF_INET); + + /* Remember the tunneled request ID in the new edge connection, so that + * we can measure download times. */ + TO_CONN(n_stream)->dirreq_id = circ->dirreq_id; + n_stream->_base.purpose = EXIT_PURPOSE_CONNECT; n_stream->stream_id = rh.stream_id; @@ -2630,7 +2781,7 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) log_debug(LD_REND,"Finished assigning addr/port"); n_stream->cpath_layer = origin_circ->cpath->prev; /* link it */ - /* add it into the linked list of n_streams on this circuit */ + /* add it into the linked list of p_streams on this circuit */ n_stream->next_stream = origin_circ->p_streams; n_stream->on_circuit = circ; origin_circ->p_streams = n_stream; @@ -2657,7 +2808,7 @@ connection_exit_begin_conn(cell_t *cell, circuit_t *circ) if (rh.command == RELAY_COMMAND_BEGIN_DIR) { tor_assert(or_circ); if (or_circ->p_conn && !tor_addr_is_null(&or_circ->p_conn->real_addr)) - tor_addr_assign(&n_stream->_base.addr, &or_circ->p_conn->real_addr); + tor_addr_copy(&n_stream->_base.addr, &or_circ->p_conn->real_addr); return connection_exit_connect_dir(n_stream); } @@ -2750,7 +2901,7 @@ connection_exit_connect(edge_connection_t *edge_conn) if (!connection_edge_is_rendezvous_stream(edge_conn) && router_compare_to_my_exit_policy(edge_conn)) { log_info(LD_EXIT,"%s:%d failed exit policy. Closing.", - escaped_safe_str(conn->address), conn->port); + escaped_safe_str_client(conn->address), conn->port); connection_edge_end(edge_conn, END_STREAM_REASON_EXITPOLICY); circuit_detach_stream(circuit_get_by_edge_conn(edge_conn), edge_conn); connection_free(conn); @@ -2762,17 +2913,17 @@ connection_exit_connect(edge_connection_t *edge_conn) log_debug(LD_EXIT,"about to try connecting"); switch (connection_connect(conn, conn->address, addr, port, &socket_error)) { - case -1: - /* XXX021 use socket_error below rather than trying to piece things - * together from the current errno, which may have been clobbered. */ - connection_edge_end_errno(edge_conn); + case -1: { + int reason = errno_to_stream_end_reason(socket_error); + connection_edge_end(edge_conn, reason); circuit_detach_stream(circuit_get_by_edge_conn(edge_conn), edge_conn); connection_free(conn); return; + } case 0: conn->state = EXIT_CONN_STATE_CONNECTING; - connection_watch_events(conn, EV_WRITE | EV_READ); + connection_watch_events(conn, READ_EVENT | WRITE_EVENT); /* writable indicates finish; * readable/error indicates broken link in windows-land. */ return; @@ -2785,7 +2936,7 @@ connection_exit_connect(edge_connection_t *edge_conn) log_warn(LD_BUG,"newly connected conn had data waiting!"); // connection_start_writing(conn); } - connection_watch_events(conn, EV_READ); + connection_watch_events(conn, READ_EVENT); /* also, deliver a 'connected' cell back through the circuit. */ if (connection_edge_is_rendezvous_stream(edge_conn)) { @@ -2832,13 +2983,17 @@ connection_exit_connect_dir(edge_connection_t *exitconn) dirconn = dir_connection_new(AF_INET); - tor_addr_assign(&dirconn->_base.addr, &exitconn->_base.addr); + tor_addr_copy(&dirconn->_base.addr, &exitconn->_base.addr); dirconn->_base.port = 0; dirconn->_base.address = tor_strdup(exitconn->_base.address); dirconn->_base.type = CONN_TYPE_DIR; dirconn->_base.purpose = DIR_PURPOSE_SERVER; dirconn->_base.state = DIR_CONN_STATE_SERVER_COMMAND_WAIT; + /* Note that the new dir conn belongs to the same tunneled request as + * the edge conn, so that we can measure download times. */ + TO_CONN(dirconn)->dirreq_id = TO_CONN(exitconn)->dirreq_id; + connection_link_connections(TO_CONN(dirconn), TO_CONN(exitconn)); if (connection_add(TO_CONN(exitconn))<0) { @@ -2893,6 +3048,8 @@ connection_edge_is_rendezvous_stream(edge_connection_t *conn) int connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) { + or_options_t *options = get_options(); + tor_assert(conn); tor_assert(conn->_base.type == CONN_TYPE_AP); tor_assert(conn->socks_request); @@ -2938,20 +3095,26 @@ connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit) if (!conn->chosen_exit_name && policy_is_reject_star(exit->exit_policy)) return 0; } + if (options->_ExcludeExitNodesUnion && + routerset_contains_router(options->_ExcludeExitNodesUnion, exit)) { + /* Not a suitable exit. Refuse it. */ + return 0; + } + return 1; } /** If address is of the form "y.onion" with a well-formed handle y: * Put a NUL after y, lower-case it, and return ONION_HOSTNAME. * - * If address is of the form "y.exit": + * If address is of the form "y.exit" and <b>allowdotexit</b> is true: * Put a NUL after y and return EXIT_HOSTNAME. * * Otherwise: * Return NORMAL_HOSTNAME and change nothing. */ hostname_type_t -parse_extended_hostname(char *address) +parse_extended_hostname(char *address, int allowdotexit) { char *s; char query[REND_SERVICE_ID_LEN_BASE32+1]; @@ -2960,8 +3123,16 @@ parse_extended_hostname(char *address) if (!s) return NORMAL_HOSTNAME; /* no dot, thus normal */ if (!strcmp(s+1,"exit")) { - *s = 0; /* NUL-terminate it */ - return EXIT_HOSTNAME; /* .exit */ + if (allowdotexit) { + *s = 0; /* NUL-terminate it */ + return EXIT_HOSTNAME; /* .exit */ + } else { + log_warn(LD_APP, "The \".exit\" notation is disabled in Tor due to " + "security risks. Set AllowDotExit in your torrc to enable " + "it."); + /* FFFF send a controller event too to notify Vidalia users */ + return BAD_HOSTNAME; + } } if (strcmp(s+1,"onion")) return NORMAL_HOSTNAME; /* neither .exit nor .onion, thus normal */ @@ -2974,17 +3145,9 @@ parse_extended_hostname(char *address) if (rend_valid_service_id(query)) { return ONION_HOSTNAME; /* success */ } -failed: + failed: /* otherwise, return to previous state and return 0 */ *s = '.'; return BAD_HOSTNAME; } -/** Check if the address is of the form "y.noconnect" - */ -int -hostname_is_noconnect_address(const char *address) -{ - return ! strcasecmpend(address, ".noconnect"); -} - diff --git a/src/or/connection_edge.h b/src/or/connection_edge.h new file mode 100644 index 000000000..70d0dd271 --- /dev/null +++ b/src/or/connection_edge.h @@ -0,0 +1,101 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file connection_edge.h + * \brief Header file for connection_edge.c. + **/ + +#ifndef _TOR_CONNECTION_EDGE_H +#define _TOR_CONNECTION_EDGE_H + +#define connection_mark_unattached_ap(conn, endreason) \ + _connection_mark_unattached_ap((conn), (endreason), __LINE__, _SHORT_FILE_) + +void _connection_mark_unattached_ap(edge_connection_t *conn, int endreason, + int line, const char *file); +int connection_edge_reached_eof(edge_connection_t *conn); +int connection_edge_process_inbuf(edge_connection_t *conn, + int package_partial); +int connection_edge_destroy(circid_t circ_id, edge_connection_t *conn); +int connection_edge_end(edge_connection_t *conn, uint8_t reason); +int connection_edge_end_errno(edge_connection_t *conn); +int connection_edge_flushed_some(edge_connection_t *conn); +int connection_edge_finished_flushing(edge_connection_t *conn); +int connection_edge_finished_connecting(edge_connection_t *conn); + +int connection_ap_handshake_send_begin(edge_connection_t *ap_conn); +int connection_ap_handshake_send_resolve(edge_connection_t *ap_conn); + +edge_connection_t *connection_ap_make_link(char *address, uint16_t port, + const char *digest, + int use_begindir, int want_onehop); +void connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, + size_t replylen, + int endreason); +void connection_ap_handshake_socks_resolved(edge_connection_t *conn, + int answer_type, + size_t answer_len, + const uint8_t *answer, + int ttl, + time_t expires); + +int connection_exit_begin_conn(cell_t *cell, circuit_t *circ); +int connection_exit_begin_resolve(cell_t *cell, or_circuit_t *circ); +void connection_exit_connect(edge_connection_t *conn); +int connection_edge_is_rendezvous_stream(edge_connection_t *conn); +int connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit); +void connection_ap_expire_beginning(void); +void connection_ap_attach_pending(void); +void connection_ap_fail_onehop(const char *failed_digest, + cpath_build_state_t *build_state); +void circuit_discard_optional_exit_enclaves(extend_info_t *info); +int connection_ap_detach_retriable(edge_connection_t *conn, + origin_circuit_t *circ, + int reason); +int connection_ap_process_transparent(edge_connection_t *conn); + +int address_is_invalid_destination(const char *address, int client); + +void addressmap_init(void); +void addressmap_clear_excluded_trackexithosts(or_options_t *options); +void addressmap_clean(time_t now); +void addressmap_clear_configured(void); +void addressmap_clear_transient(void); +void addressmap_free_all(void); +int addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out); +int addressmap_have_mapping(const char *address, int update_timeout); + +void addressmap_register(const char *address, char *new_address, + time_t expires, addressmap_entry_source_t source); +int parse_virtual_addr_network(const char *val, int validate_only, + char **msg); +int client_dns_incr_failures(const char *address); +void client_dns_clear_failures(const char *address); +void client_dns_set_addressmap(const char *address, uint32_t val, + const char *exitname, int ttl); +const char *addressmap_register_virtual_address(int type, char *new_address); +void addressmap_get_mappings(smartlist_t *sl, time_t min_expires, + time_t max_expires, int want_expiry); +int connection_ap_rewrite_and_attach_if_allowed(edge_connection_t *conn, + origin_circuit_t *circ, + crypt_path_t *cpath); +int connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, + origin_circuit_t *circ, + crypt_path_t *cpath); + +/** Possible return values for parse_extended_hostname. */ +typedef enum hostname_type_t { + NORMAL_HOSTNAME, ONION_HOSTNAME, EXIT_HOSTNAME, BAD_HOSTNAME +} hostname_type_t; +hostname_type_t parse_extended_hostname(char *address, int allowdotexit); + +#if defined(HAVE_NET_IF_H) && defined(HAVE_NET_PFVAR_H) +int get_pf_socket(void); +#endif + +#endif + diff --git a/src/or/connection_or.c b/src/or/connection_or.c index e70edc7c8..c6049d51f 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -11,6 +11,22 @@ **/ #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "command.h" +#include "config.h" +#include "connection.h" +#include "connection_or.h" +#include "control.h" +#include "dirserv.h" +#include "geoip.h" +#include "main.h" +#include "networkstatus.h" +#include "reasons.h" +#include "relay.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" static int connection_tls_finish_handshake(or_connection_t *conn); static int connection_or_process_cells_from_inbuf(or_connection_t *conn); @@ -80,10 +96,8 @@ connection_or_clear_identity_map(void) } }); - if (orconn_identity_map) { - digestmap_free(orconn_identity_map, NULL); - orconn_identity_map = NULL; - } + digestmap_free(orconn_identity_map, NULL); + orconn_identity_map = NULL; } /** Change conn->identity_digest to digest, and add conn into @@ -133,7 +147,7 @@ void cell_pack(packed_cell_t *dst, const cell_t *src) { char *dest = dst->body; - *(uint16_t*)dest = htons(src->circ_id); + set_uint16(dest, htons(src->circ_id)); *(uint8_t*)(dest+2) = src->command; memcpy(dest+3, src->payload, CELL_PAYLOAD_SIZE); } @@ -144,7 +158,7 @@ cell_pack(packed_cell_t *dst, const cell_t *src) static void cell_unpack(cell_t *dest, const char *src) { - dest->circ_id = ntohs(*(uint16_t*)(src)); + dest->circ_id = ntohs(get_uint16(src)); dest->command = *(uint8_t*)(src+2); memcpy(dest->payload, src+3, CELL_PAYLOAD_SIZE); } @@ -187,66 +201,6 @@ connection_or_reached_eof(or_connection_t *conn) return 0; } -/** Read conn's inbuf. If the http response from the proxy is all - * here, make sure it's good news, and begin the tls handshake. If - * it's bad news, close the connection and return -1. Else return 0 - * and hope for better luck next time. - */ -static int -connection_or_read_proxy_response(or_connection_t *or_conn) -{ - char *headers; - char *reason=NULL; - int status_code; - time_t date_header; - connection_t *conn = TO_CONN(or_conn); - - switch (fetch_from_buf_http(conn->inbuf, - &headers, MAX_HEADERS_SIZE, - NULL, NULL, 10000, 0)) { - case -1: /* overflow */ - log_warn(LD_PROTOCOL, - "Your https proxy sent back an oversized response. Closing."); - return -1; - case 0: - log_info(LD_OR,"https proxy response not all here yet. Waiting."); - return 0; - /* case 1, fall through */ - } - - if (parse_http_response(headers, &status_code, &date_header, - NULL, &reason) < 0) { - log_warn(LD_OR, - "Unparseable headers from proxy (connecting to '%s'). Closing.", - conn->address); - tor_free(headers); - return -1; - } - if (!reason) reason = tor_strdup("[no reason given]"); - - if (status_code == 200) { - log_info(LD_OR, - "HTTPS connect to '%s' successful! (200 %s) Starting TLS.", - conn->address, escaped(reason)); - tor_free(reason); - if (connection_tls_start_handshake(or_conn, 0) < 0) { - /* TLS handshaking error of some kind. */ - connection_mark_for_close(conn); - - return -1; - } - return 0; - } - /* else, bad news on the status code */ - log_warn(LD_OR, - "The https proxy sent back an unexpected status code %d (%s). " - "Closing.", - status_code, escaped(reason)); - tor_free(reason); - connection_mark_for_close(conn); - return -1; -} - /** Handle any new bytes that have come in on connection <b>conn</b>. * If conn is in 'open' state, hand it to * connection_or_process_cells_from_inbuf() @@ -255,11 +209,24 @@ connection_or_read_proxy_response(or_connection_t *or_conn) int connection_or_process_inbuf(or_connection_t *conn) { + int ret; tor_assert(conn); switch (conn->_base.state) { - case OR_CONN_STATE_PROXY_READING: - return connection_or_read_proxy_response(conn); + case OR_CONN_STATE_PROXY_HANDSHAKING: + ret = connection_read_proxy_handshake(TO_CONN(conn)); + + /* start TLS after handshake completion, or deal with error */ + if (ret == 1) { + tor_assert(TO_CONN(conn)->proxy_state == PROXY_CONNECTED); + if (connection_tls_start_handshake(conn, 0) < 0) + ret = -1; + } + if (ret < 0) { + connection_mark_for_close(TO_CONN(conn)); + } + + return ret; case OR_CONN_STATE_OPEN: case OR_CONN_STATE_OR_HANDSHAKING: return connection_or_process_cells_from_inbuf(conn); @@ -285,8 +252,7 @@ connection_or_flushed_some(or_connection_t *conn) /* If we're under the low water mark, add cells until we're just over the * high water mark. */ if (datalen < OR_CONN_LOWWATER) { - ssize_t n = (OR_CONN_HIGHWATER - datalen + CELL_NETWORK_SIZE-1) - / CELL_NETWORK_SIZE; + ssize_t n = CEIL_DIV(OR_CONN_HIGHWATER - datalen, CELL_NETWORK_SIZE); time_t now = approx_time(); while (conn->active_circuits && n > 0) { int flushed; @@ -312,11 +278,7 @@ connection_or_finished_flushing(or_connection_t *conn) assert_connection_ok(TO_CONN(conn),0); switch (conn->_base.state) { - case OR_CONN_STATE_PROXY_FLUSHING: - log_debug(LD_OR,"finished sending CONNECT to proxy."); - conn->_base.state = OR_CONN_STATE_PROXY_READING; - connection_stop_writing(TO_CONN(conn)); - break; + case OR_CONN_STATE_PROXY_HANDSHAKING: case OR_CONN_STATE_OPEN: case OR_CONN_STATE_OR_HANDSHAKING: connection_stop_writing(TO_CONN(conn)); @@ -334,37 +296,34 @@ connection_or_finished_flushing(or_connection_t *conn) int connection_or_finished_connecting(or_connection_t *or_conn) { + int proxy_type; connection_t *conn; tor_assert(or_conn); conn = TO_CONN(or_conn); tor_assert(conn->state == OR_CONN_STATE_CONNECTING); - log_debug(LD_OR,"OR connect() to router at %s:%u finished.", + log_debug(LD_HANDSHAKE,"OR connect() to router at %s:%u finished.", conn->address,conn->port); control_event_bootstrap(BOOTSTRAP_STATUS_HANDSHAKE, 0); - if (get_options()->HttpsProxy) { - char buf[1024]; - char *base64_authenticator=NULL; - const char *authenticator = get_options()->HttpsProxyAuthenticator; + proxy_type = PROXY_NONE; - if (authenticator) { - base64_authenticator = alloc_http_authenticator(authenticator); - if (!base64_authenticator) - log_warn(LD_OR, "Encoding https authenticator failed"); - } - if (base64_authenticator) { - tor_snprintf(buf, sizeof(buf), "CONNECT %s:%d HTTP/1.1\r\n" - "Proxy-Authorization: Basic %s\r\n\r\n", - fmt_addr(&conn->addr), - conn->port, base64_authenticator); - tor_free(base64_authenticator); - } else { - tor_snprintf(buf, sizeof(buf), "CONNECT %s:%d HTTP/1.0\r\n\r\n", - fmt_addr(&conn->addr), conn->port); + if (get_options()->HTTPSProxy) + proxy_type = PROXY_CONNECT; + else if (get_options()->Socks4Proxy) + proxy_type = PROXY_SOCKS4; + else if (get_options()->Socks5Proxy) + proxy_type = PROXY_SOCKS5; + + if (proxy_type != PROXY_NONE) { + /* start proxy handshake */ + if (connection_proxy_connect(conn, proxy_type) < 0) { + connection_mark_for_close(conn); + return -1; } - connection_write_to_buf(buf, strlen(buf), conn); - conn->state = OR_CONN_STATE_PROXY_FLUSHING; + + connection_start_reading(conn); + conn->state = OR_CONN_STATE_PROXY_HANDSHAKING; return 0; } @@ -376,6 +335,78 @@ connection_or_finished_connecting(or_connection_t *or_conn) return 0; } +/** Return 1 if identity digest <b>id_digest</b> is known to be a + * currently or recently running relay. Otherwise return 0. */ +int +connection_or_digest_is_known_relay(const char *id_digest) +{ + if (router_get_consensus_status_by_id(id_digest)) + return 1; /* It's in the consensus: "yes" */ + if (router_get_by_digest(id_digest)) + return 1; /* Not in the consensus, but we have a descriptor for + * it. Probably it was in a recent consensus. "Yes". */ + return 0; +} + +/** Set the per-conn read and write limits for <b>conn</b>. If it's a known + * relay, we will rely on the global read and write buckets, so give it + * per-conn limits that are big enough they'll never matter. But if it's + * not a known relay, first check if we set PerConnBwRate/Burst, then + * check if the consensus sets them, else default to 'big enough'. + * + * If <b>reset</b> is true, set the bucket to be full. Otherwise, just + * clip the bucket if it happens to be <em>too</em> full. + */ +static void +connection_or_update_token_buckets_helper(or_connection_t *conn, int reset, + or_options_t *options) +{ + int rate, burst; /* per-connection rate limiting params */ + if (connection_or_digest_is_known_relay(conn->identity_digest)) { + /* It's in the consensus, or we have a descriptor for it meaning it + * was probably in a recent consensus. It's a recognized relay: + * give it full bandwidth. */ + rate = (int)options->BandwidthRate; + burst = (int)options->BandwidthBurst; + } else { + /* Not a recognized relay. Squeeze it down based on the suggested + * bandwidth parameters in the consensus, but allow local config + * options to override. */ + rate = options->PerConnBWRate ? (int)options->PerConnBWRate : + networkstatus_get_param(NULL, "perconnbwrate", + (int)options->BandwidthRate, 1, INT32_MAX); + burst = options->PerConnBWBurst ? (int)options->PerConnBWBurst : + networkstatus_get_param(NULL, "perconnbwburst", + (int)options->BandwidthBurst, 1, INT32_MAX); + } + + conn->bandwidthrate = rate; + conn->bandwidthburst = burst; + if (reset) { /* set up the token buckets to be full */ + conn->read_bucket = conn->write_bucket = burst; + return; + } + /* If the new token bucket is smaller, take out the extra tokens. + * (If it's larger, don't -- the buckets can grow to reach the cap.) */ + if (conn->read_bucket > burst) + conn->read_bucket = burst; + if (conn->write_bucket > burst) + conn->write_bucket = burst; +} + +/** Either our set of relays or our per-conn rate limits have changed. + * Go through all the OR connections and update their token buckets to make + * sure they don't exceed their maximum values. */ +void +connection_or_update_token_buckets(smartlist_t *conns, or_options_t *options) +{ + SMARTLIST_FOREACH(conns, connection_t *, conn, + { + if (connection_speaks_cells(conn)) + connection_or_update_token_buckets_helper(TO_OR_CONN(conn), 0, options); + }); +} + /** If we don't necessarily know the router we're connecting to, but we * have an addr/port/id_digest, then fill in as much as we can. Start * by checking to see if this describes a router we know. */ @@ -385,11 +416,9 @@ connection_or_init_conn_from_address(or_connection_t *conn, const char *id_digest, int started_here) { - or_options_t *options = get_options(); routerinfo_t *r = router_get_by_digest(id_digest); - conn->bandwidthrate = (int)options->BandwidthRate; - conn->read_bucket = conn->bandwidthburst = (int)options->BandwidthBurst; connection_or_set_identity_digest(conn, id_digest); + connection_or_update_token_buckets_helper(conn, 1, get_options()); conn->_base.port = port; tor_addr_copy(&conn->_base.addr, addr); @@ -442,7 +471,7 @@ connection_or_init_conn_from_address(or_connection_t *conn, * Requires that both input connections are open; not is_bad_for_new_circs, * and not impossibly non-canonical. * - * If </b>forgive_new_connections</b> is true, then we do not call + * If <b>forgive_new_connections</b> is true, then we do not call * <b>a</b>better than <b>b</b> simply because b has no circuits, * unless b is also relatively old. */ @@ -581,11 +610,24 @@ connection_or_get_for_extend(const char *digest, #define TIME_BEFORE_OR_CONN_IS_TOO_OLD (60*60*24*7) /** Given the head of the linked list for all the or_connections with a given - * identity, set elements of that list as is_bad_for_new_circs() as - * appropriate. Helper for connection_or_set_bad_connections(). + * identity, set elements of that list as is_bad_for_new_circs as + * appropriate. Helper for connection_or_set_bad_connections(). + * + * Specifically, we set the is_bad_for_new_circs flag on: + * - all connections if <b>force</b> is true. + * - all connections that are too old. + * - all open non-canonical connections for which a canonical connection + * exists to the same router. + * - all open canonical connections for which a 'better' canonical + * connection exists to the same router. + * - all open non-canonical connections for which a 'better' non-canonical + * connection exists to the same router at the same address. + * + * See connection_or_is_better() for our idea of what makes one OR connection + * better than another. */ static void -connection_or_group_set_badness(or_connection_t *head) +connection_or_group_set_badness(or_connection_t *head, int force) { or_connection_t *or_conn = NULL, *best = NULL; int n_old = 0, n_inprogress = 0, n_canonical = 0, n_other = 0; @@ -597,8 +639,9 @@ connection_or_group_set_badness(or_connection_t *head) if (or_conn->_base.marked_for_close || or_conn->is_bad_for_new_circs) continue; - if (or_conn->_base.timestamp_created + TIME_BEFORE_OR_CONN_IS_TOO_OLD - < now) { + if (force || + or_conn->_base.timestamp_created + TIME_BEFORE_OR_CONN_IS_TOO_OLD + < now) { log_info(LD_OR, "Marking OR conn to %s:%d as too old for new circuits " "(fd %d, %d secs old).", @@ -631,7 +674,7 @@ connection_or_group_set_badness(or_connection_t *head) /* We have at least one open canonical connection to this router, * and this one is open but not canonical. Mark it bad. */ log_info(LD_OR, - "Marking OR conn to %s:%d as too old for new circuits: " + "Marking OR conn to %s:%d as unsuitable for new circuits: " "(fd %d, %d secs old). It is not canonical, and we have " "another connection to that OR that is.", or_conn->_base.address, or_conn->_base.port, or_conn->_base.s, @@ -671,7 +714,7 @@ connection_or_group_set_badness(or_connection_t *head) even when we're being forgiving. */ if (best->is_canonical) { log_info(LD_OR, - "Marking OR conn to %s:%d as too old for new circuits: " + "Marking OR conn to %s:%d as unsuitable for new circuits: " "(fd %d, %d secs old). We have a better canonical one " "(fd %d; %d secs old).", or_conn->_base.address, or_conn->_base.port, or_conn->_base.s, @@ -681,9 +724,9 @@ connection_or_group_set_badness(or_connection_t *head) } else if (!tor_addr_compare(&or_conn->real_addr, &best->real_addr, CMP_EXACT)) { log_info(LD_OR, - "Marking OR conn to %s:%d as too old for new circuits: " - "(fd %d, %d secs old). We have a better one " - "(fd %d; %d secs old).", + "Marking OR conn to %s:%d as unsuitable for new circuits: " + "(fd %d, %d secs old). We have a better one with the " + "same address (fd %d; %d secs old).", or_conn->_base.address, or_conn->_base.port, or_conn->_base.s, (int)(now - or_conn->_base.timestamp_created), best->_base.s, (int)(now - best->_base.timestamp_created)); @@ -693,27 +736,20 @@ connection_or_group_set_badness(or_connection_t *head) } } -/** Go through all the OR connections, and set the is_bad_for_new_circs - * flag on: - * - all connections that are too old. - * - all open non-canonical connections for which a canonical connection - * exists to the same router. - * - all open canonical connections for which a 'better' canonical - * connection exists to the same router. - * - all open non-canonical connections for which a 'better' non-canonical - * connection exists to the same router at the same address. - * - * See connection_or_is_better() for our idea of what makes one OR connection - * better than another. +/** Go through all the OR connections (or if <b>digest</b> is non-NULL, just + * the OR connections with that digest), and set the is_bad_for_new_circs + * flag based on the rules in connection_or_group_set_badness() (or just + * always set it if <b>force</b> is true). */ void -connection_or_set_bad_connections(void) +connection_or_set_bad_connections(const char *digest, int force) { if (!orconn_identity_map) return; DIGESTMAP_FOREACH(orconn_identity_map, identity, or_connection_t *, conn) { - connection_or_group_set_badness(conn); + if (!digest || !memcmp(digest, conn->identity_digest, DIGEST_LEN)) + connection_or_group_set_badness(conn, force); } DIGESTMAP_FOREACH_END; } @@ -753,6 +789,7 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, or_connection_t *conn; or_options_t *options = get_options(); int socket_error = 0; + int using_proxy = 0; tor_addr_t addr; tor_assert(_addr); @@ -771,19 +808,27 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, conn->_base.state = OR_CONN_STATE_CONNECTING; control_event_or_conn_status(conn, OR_CONN_EVENT_LAUNCHED, 0); - if (options->HttpsProxy) { - /* we shouldn't connect directly. use the https proxy instead. */ - tor_addr_from_ipv4h(&addr, options->HttpsProxyAddr); - port = options->HttpsProxyPort; + /* use a proxy server if available */ + if (options->HTTPSProxy) { + using_proxy = 1; + tor_addr_copy(&addr, &options->HTTPSProxyAddr); + port = options->HTTPSProxyPort; + } else if (options->Socks4Proxy) { + using_proxy = 1; + tor_addr_copy(&addr, &options->Socks4ProxyAddr); + port = options->Socks4ProxyPort; + } else if (options->Socks5Proxy) { + using_proxy = 1; + tor_addr_copy(&addr, &options->Socks5ProxyAddr); + port = options->Socks5ProxyPort; } switch (connection_connect(TO_CONN(conn), conn->_base.address, &addr, port, &socket_error)) { case -1: /* If the connection failed immediately, and we're using - * an https proxy, our https proxy is down. Don't blame the - * Tor server. */ - if (!options->HttpsProxy) + * a proxy, our proxy is down. Don't blame the Tor server. */ + if (!using_proxy) entry_guard_register_connect_status(conn->identity_digest, 0, 1, time(NULL)); connection_or_connect_failed(conn, @@ -792,7 +837,7 @@ connection_or_connect(const tor_addr_t *_addr, uint16_t port, connection_free(TO_CONN(conn)); return NULL; case 0: - connection_watch_events(TO_CONN(conn), EV_READ | EV_WRITE); + connection_watch_events(TO_CONN(conn), READ_EVENT | WRITE_EVENT); /* writable indicates finish, readable indicates broken link, error indicates broken link on windows */ return conn; @@ -819,13 +864,14 @@ connection_tls_start_handshake(or_connection_t *conn, int receiving) { conn->_base.state = OR_CONN_STATE_TLS_HANDSHAKING; conn->tls = tor_tls_new(conn->_base.s, receiving); - tor_tls_set_logged_address(conn->tls, escaped_safe_str(conn->_base.address)); + tor_tls_set_logged_address(conn->tls, // XXX client and relay? + escaped_safe_str(conn->_base.address)); if (!conn->tls) { log_warn(LD_BUG,"tor_tls_new failed. Closing."); return -1; } connection_start_reading(TO_CONN(conn)); - log_debug(LD_OR,"starting TLS handshake on fd %d", conn->_base.s); + log_debug(LD_HANDSHAKE,"starting TLS handshake on fd %d", conn->_base.s); note_crypto_pk_op(receiving ? TLS_HANDSHAKE_S : TLS_HANDSHAKE_C); if (connection_tls_continue_handshake(conn) < 0) { @@ -932,23 +978,26 @@ connection_or_nonopen_was_started_here(or_connection_t *conn) * return -1 if he is lying, broken, or otherwise something is wrong. * * If we initiated this connection (<b>started_here</b> is true), make sure - * the other side sent sent a correctly formed certificate. If I initiated the + * the other side sent a correctly formed certificate. If I initiated the * connection, make sure it's the right guy. * * Otherwise (if we _didn't_ initiate this connection), it's okay for * the certificate to be weird or absent. * * If we return 0, and the certificate is as expected, write a hash of the - * identity key into digest_rcvd, which must have DIGEST_LEN space in it. (If - * we return -1 this buffer is undefined.) If the certificate is invalid - * or missing on an incoming connection, we return 0 and set digest_rcvd to - * DIGEST_LEN 0 bytes. + * identity key into <b>digest_rcvd_out</b>, which must have DIGEST_LEN + * space in it. + * If the certificate is invalid or missing on an incoming connection, + * we return 0 and set <b>digest_rcvd_out</b> to DIGEST_LEN NUL bytes. + * (If we return -1, the contents of this buffer are undefined.) * * As side effects, * 1) Set conn->circ_id_type according to tor-spec.txt. * 2) If we're an authdirserver and we initiated the connection: drop all * descriptors that claim to be on that IP/port but that aren't * this guy; and note that this guy is reachable. + * 3) If this is a bridge and we didn't configure its identity + * fingerprint, remember the keyid we just learned. */ static int connection_or_check_valid_tls_handshake(or_connection_t *conn, @@ -959,19 +1008,23 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, or_options_t *options = get_options(); int severity = server_mode(options) ? LOG_PROTOCOL_WARN : LOG_WARN; const char *safe_address = - started_here ? conn->_base.address : safe_str(conn->_base.address); + started_here ? conn->_base.address : + safe_str_client(conn->_base.address); const char *conn_type = started_here ? "outgoing" : "incoming"; + crypto_pk_env_t *our_identity = + started_here ? get_tlsclient_identity_key() : + get_server_identity_key(); int has_cert = 0, has_identity=0; check_no_tls_errors(); has_cert = tor_tls_peer_has_cert(conn->tls); if (started_here && !has_cert) { - log_info(LD_PROTOCOL,"Tried connecting to router at %s:%d, but it didn't " + log_info(LD_HANDSHAKE,"Tried connecting to router at %s:%d, but it didn't " "send a cert! Closing.", safe_address, conn->_base.port); return -1; } else if (!has_cert) { - log_debug(LD_PROTOCOL,"Got incoming connection with no certificate. " + log_debug(LD_HANDSHAKE,"Got incoming connection with no certificate. " "That's ok."); } check_no_tls_errors(); @@ -980,15 +1033,16 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, int v = tor_tls_verify(started_here?severity:LOG_INFO, conn->tls, &identity_rcvd); if (started_here && v<0) { - log_fn(severity,LD_OR,"Tried connecting to router at %s:%d: It" + log_fn(severity,LD_HANDSHAKE,"Tried connecting to router at %s:%d: It" " has a cert but it's invalid. Closing.", safe_address, conn->_base.port); return -1; } else if (v<0) { - log_info(LD_PROTOCOL,"Incoming connection gave us an invalid cert " + log_info(LD_HANDSHAKE,"Incoming connection gave us an invalid cert " "chain; ignoring."); } else { - log_debug(LD_OR,"The certificate seems to be valid on %s connection " + log_debug(LD_HANDSHAKE, + "The certificate seems to be valid on %s connection " "with %s:%d", conn_type, safe_address, conn->_base.port); } check_no_tls_errors(); @@ -997,7 +1051,7 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, if (identity_rcvd) { has_identity = 1; crypto_pk_get_digest(identity_rcvd, digest_rcvd_out); - if (crypto_pk_cmp_keys(get_identity_key(), identity_rcvd)<0) { + if (crypto_pk_cmp_keys(our_identity, identity_rcvd)<0) { conn->circ_id_type = CIRC_ID_TYPE_LOWER; } else { conn->circ_id_type = CIRC_ID_TYPE_HIGHER; @@ -1015,9 +1069,13 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, conn->nickname[0] = '$'; base16_encode(conn->nickname+1, HEX_DIGEST_LEN+1, conn->identity_digest, DIGEST_LEN); - log_info(LD_OR, "Connected to router %s at %s:%d without knowing " + log_info(LD_HANDSHAKE, "Connected to router %s at %s:%d without knowing " "its key. Hoping for the best.", conn->nickname, conn->_base.address, conn->_base.port); + /* if it's a bridge and we didn't know its identity fingerprint, now + * we do -- remember it for future attempts. */ + learned_router_identity(&conn->_base.addr, conn->_base.port, + digest_rcvd_out); } if (started_here) { @@ -1031,7 +1089,7 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, base16_encode(seen, sizeof(seen), digest_rcvd_out, DIGEST_LEN); base16_encode(expected, sizeof(expected), conn->identity_digest, DIGEST_LEN); - log_fn(severity, LD_OR, + log_fn(severity, LD_HANDSHAKE, "Tried connecting to router at %s:%d, but identity key was not " "as expected: wanted %s but got %s.", conn->_base.address, conn->_base.port, expected, seen); @@ -1044,9 +1102,6 @@ connection_or_check_valid_tls_handshake(or_connection_t *conn, as_advertised = 0; } if (authdir_mode_tests_reachability(options)) { - /* We initiated this connection to address:port. Drop all routers - * with the same address:port and a different key. - */ dirserv_orconn_tls_done(conn->_base.address, conn->_base.port, digest_rcvd_out, as_advertised); } @@ -1073,8 +1128,8 @@ connection_tls_finish_handshake(or_connection_t *conn) char digest_rcvd[DIGEST_LEN]; int started_here = connection_or_nonopen_was_started_here(conn); - log_debug(LD_OR,"tls handshake with %s done. verifying.", - safe_str(conn->_base.address)); + log_debug(LD_HANDSHAKE,"tls handshake with %s done. verifying.", + safe_str_client(conn->_base.address)); directory_set_dirty(); @@ -1082,6 +1137,8 @@ connection_tls_finish_handshake(or_connection_t *conn) digest_rcvd) < 0) return -1; + circuit_build_times_network_is_live(&circ_times); + if (tor_tls_used_v1_handshake(conn->tls)) { conn->link_proto = 1; if (!started_here) { @@ -1117,7 +1174,8 @@ connection_init_or_handshake_state(or_connection_t *conn, int started_here) void or_handshake_state_free(or_handshake_state_t *state) { - tor_assert(state); + if (!state) + return; memset(state, 0xBE, sizeof(or_handshake_state_t)); tor_free(state); } @@ -1134,6 +1192,7 @@ connection_or_set_state_open(or_connection_t *conn) control_event_or_conn_status(conn, OR_CONN_EVENT_CONNECTED, 0); if (started_here) { + circuit_build_times_network_is_live(&circ_times); rep_hist_note_connect_succeeded(conn->identity_digest, now); if (entry_guard_register_connect_status(conn->identity_digest, 1, 0, now) < 0) { @@ -1158,10 +1217,10 @@ connection_or_set_state_open(or_connection_t *conn) } } } - if (conn->handshake_state) { - or_handshake_state_free(conn->handshake_state); - conn->handshake_state = NULL; - } + + or_handshake_state_free(conn->handshake_state); + conn->handshake_state = NULL; + connection_start_reading(TO_CONN(conn)); circuit_n_conn_done(conn, 1); /* send the pending creates, if any. */ @@ -1235,6 +1294,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn) if (connection_fetch_var_cell_from_buf(conn, &var_cell)) { if (!var_cell) return 0; /* not yet. */ + circuit_build_times_network_is_live(&circ_times); command_process_var_cell(var_cell, conn); var_cell_free(var_cell); } else { @@ -1244,6 +1304,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn) available? */ return 0; /* not yet */ + circuit_build_times_network_is_live(&circ_times); connection_fetch_from_buf(buf, CELL_NETWORK_SIZE, TO_CONN(conn)); /* retrieve cell info from buf (create the host-order struct from the @@ -1274,10 +1335,6 @@ connection_or_send_destroy(circid_t circ_id, or_connection_t *conn, int reason) cell.payload[0] = (uint8_t) reason; log_debug(LD_OR,"Sending destroy (circID %d).", circ_id); - /* XXXX It's possible that under some circumstances, we want the destroy - * to take precedence over other data waiting on the circuit's cell queue. - */ - connection_or_write_cell_to_buf(&cell, conn); return 0; } @@ -1357,9 +1414,8 @@ connection_or_send_netinfo(or_connection_t *conn) len = append_address_to_payload(out, &my_addr); if (len < 0) return -1; - out += len; } else { - *out++ = 0; + *out = 0; } connection_or_write_cell_to_buf(&cell, conn); diff --git a/src/or/connection_or.h b/src/or/connection_or.h new file mode 100644 index 000000000..70ef96a33 --- /dev/null +++ b/src/or/connection_or.h @@ -0,0 +1,57 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file connection_or.h + * \brief Header file for connection_or.c. + **/ + +#ifndef _TOR_CONNECTION_OR_H +#define _TOR_CONNECTION_OR_H + +void connection_or_remove_from_identity_map(or_connection_t *conn); +void connection_or_clear_identity_map(void); +or_connection_t *connection_or_get_for_extend(const char *digest, + const tor_addr_t *target_addr, + const char **msg_out, + int *launch_out); +void connection_or_set_bad_connections(const char *digest, int force); + +int connection_or_reached_eof(or_connection_t *conn); +int connection_or_process_inbuf(or_connection_t *conn); +int connection_or_flushed_some(or_connection_t *conn); +int connection_or_finished_flushing(or_connection_t *conn); +int connection_or_finished_connecting(or_connection_t *conn); +int connection_or_digest_is_known_relay(const char *id_digest); +void connection_or_update_token_buckets(smartlist_t *conns, + or_options_t *options); + +void connection_or_connect_failed(or_connection_t *conn, + int reason, const char *msg); +or_connection_t *connection_or_connect(const tor_addr_t *addr, uint16_t port, + const char *id_digest); + +int connection_tls_start_handshake(or_connection_t *conn, int receiving); +int connection_tls_continue_handshake(or_connection_t *conn); + +void or_handshake_state_free(or_handshake_state_t *state); +int connection_or_set_state_open(or_connection_t *conn); +void connection_or_write_cell_to_buf(const cell_t *cell, + or_connection_t *conn); +void connection_or_write_var_cell_to_buf(const var_cell_t *cell, + or_connection_t *conn); +int connection_or_send_destroy(circid_t circ_id, or_connection_t *conn, + int reason); +int connection_or_send_netinfo(or_connection_t *conn); +int is_or_protocol_version_known(uint16_t version); + +void cell_pack(packed_cell_t *dest, const cell_t *src); +void var_cell_pack_header(const var_cell_t *cell, char *hdr_out); +var_cell_t *var_cell_new(uint16_t payload_len); +void var_cell_free(var_cell_t *cell); + +#endif + diff --git a/src/or/control.c b/src/or/control.c index eae617036..926a46520 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -11,6 +11,26 @@ #define CONTROL_PRIVATE #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "control.h" +#include "directory.h" +#include "dirserv.h" +#include "dnsserv.h" +#include "geoip.h" +#include "hibernate.h" +#include "main.h" +#include "networkstatus.h" +#include "policies.h" +#include "reasons.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" /** Yield true iff <b>s</b> is the state of a control_connection_t that has * finished authentication and is accepting commands. */ @@ -43,7 +63,8 @@ #define EVENT_STREAM_BANDWIDTH_USED 0x0014 #define EVENT_CLIENTS_SEEN 0x0015 #define EVENT_NEWCONSENSUS 0x0016 -#define _EVENT_MAX 0x0016 +#define EVENT_BUILDTIMEOUT_SET 0x0017 +#define _EVENT_MAX 0x0017 /* If _EVENT_MAX ever hits 0x0020, we need to make the mask wider. */ /** Bitfield: The bit 1<<e is set if <b>any</b> open control @@ -54,13 +75,9 @@ **/ typedef uint32_t event_mask_t; -/** An event mask of all the events that controller with the LONG_NAMES option - * set is interested in receiving. */ -static event_mask_t global_event_mask1long = 0; - -/** An event mask of all the events that controller with the SHORT_NAMES option - * set is interested in receiving. */ -static event_mask_t global_event_mask1short = 0; +/** An event mask of all the events that any controller is interested in + * receiving. */ +static event_mask_t global_event_mask = 0; /** True iff we have disabled log messages from being sent to the controller */ static int disable_log_messages = 0; @@ -68,13 +85,7 @@ static int disable_log_messages = 0; /** Macro: true if any control connection is interested in events of type * <b>e</b>. */ #define EVENT_IS_INTERESTING(e) \ - ((global_event_mask1long|global_event_mask1short) & (1<<(e))) -/** Macro: true if any control connection with the LONG_NAMES option is - * interested in events of type <b>e</b>. */ -#define EVENT_IS_INTERESTING1L(e) (global_event_mask1long & (1<<(e))) -/** Macro: true if any control connection with the SHORT_NAMES option is - * interested in events of type <b>e</b>. */ -#define EVENT_IS_INTERESTING1S(e) (global_event_mask1short & (1<<(e))) + (global_event_mask & (1<<(e))) /** If we're using cookie-type authentication, how long should our cookies be? */ @@ -95,25 +106,13 @@ static char authentication_cookie[AUTHENTICATION_COOKIE_LEN]; * of this so we can respond to getinfo status/bootstrap-phase queries. */ static char last_sent_bootstrap_message[BOOTSTRAP_MSG_LEN]; -/** Flag for event_format_t. Indicates that we should use the old - * name format of nickname|hexdigest - */ -#define SHORT_NAMES 1 -/** Flag for event_format_t. Indicates that we should use the new - * name format of $hexdigest[=~]nickname +/** Flag for event_format_t. Indicates that we should use the one standard + format. */ -#define LONG_NAMES 2 -#define ALL_NAMES (SHORT_NAMES|LONG_NAMES) -/** Flag for event_format_t. Indicates that we should use the new event - * format where extra event fields are allowed using a NAME=VAL format. */ -#define EXTENDED_FORMAT 4 -/** Flag for event_format_t. Indicates that we are using the old event format - * where extra fields aren't allowed. */ -#define NONEXTENDED_FORMAT 8 -#define ALL_FORMATS (EXTENDED_FORMAT|NONEXTENDED_FORMAT) +#define ALL_FORMATS 1 /** Bit field of flags to select how to format a controller event. Recognized - * flags are SHORT_NAMES, LONG_NAMES, EXTENDED_FORMAT, NONEXTENDED_FORMAT. */ + * flag is ALL_FORMATS. */ typedef int event_format_t; static void connection_printf_to_buf(control_connection_t *conn, @@ -123,9 +122,6 @@ static void send_control_done(control_connection_t *conn); static void send_control_event(uint16_t event, event_format_t which, const char *format, ...) CHECK_PRINTF(3,4); -static void send_control_event_extended(uint16_t event, event_format_t which, - const char *format, ...) - CHECK_PRINTF(3,4); static int handle_control_setconf(control_connection_t *conn, uint32_t len, char *body); static int handle_control_resetconf(control_connection_t *conn, uint32_t len, @@ -139,8 +135,6 @@ static int handle_control_setevents(control_connection_t *conn, uint32_t len, static int handle_control_authenticate(control_connection_t *conn, uint32_t len, const char *body); -static int handle_control_saveconf(control_connection_t *conn, uint32_t len, - const char *body); static int handle_control_signal(control_connection_t *conn, uint32_t len, const char *body); static int handle_control_mapaddress(control_connection_t *conn, uint32_t len, @@ -174,7 +168,7 @@ static int handle_control_usefeature(control_connection_t *conn, const char *body); static int write_stream_target_to_buf(edge_connection_t *conn, char *buf, size_t len); -static void orconn_target_get_name(int long_names, char *buf, size_t len, +static void orconn_target_get_name(char *buf, size_t len, or_connection_t *conn); static char *get_cookie_file(void); @@ -214,25 +208,19 @@ control_update_global_event_mask(void) { smartlist_t *conns = get_connection_array(); event_mask_t old_mask, new_mask; - old_mask = global_event_mask1short; - old_mask |= global_event_mask1long; + old_mask = global_event_mask; - global_event_mask1short = 0; - global_event_mask1long = 0; + global_event_mask = 0; SMARTLIST_FOREACH(conns, connection_t *, _conn, { if (_conn->type == CONN_TYPE_CONTROL && STATE_IS_OPEN(_conn->state)) { control_connection_t *conn = TO_CONTROL_CONN(_conn); - if (conn->use_long_names) - global_event_mask1long |= conn->event_mask; - else - global_event_mask1short |= conn->event_mask; + global_event_mask |= conn->event_mask; } }); - new_mask = global_event_mask1short; - new_mask |= global_event_mask1long; + new_mask = global_event_mask; /* Handle the aftermath. Set up the log callback to tell us only what * we want to hear...*/ @@ -312,7 +300,7 @@ connection_write_str_to_buf(const char *s, control_connection_t *conn) /** Given a <b>len</b>-character string in <b>data</b>, made of lines * terminated by CRLF, allocate a new string in *<b>out</b>, and copy the * contents of <b>data</b> into *<b>out</b>, adding a period before any period - * that that appears at the start of a line, and adding a period-CRLF line at + * that appears at the start of a line, and adding a period-CRLF line at * the end. Replace all LF characters sequences with CRLF. Return the number * of bytes in *<b>out</b>. */ @@ -542,28 +530,15 @@ send_control_event_string(uint16_t event, event_format_t which, const char *msg) { smartlist_t *conns = get_connection_array(); + (void)which; tor_assert(event >= _EVENT_MIN && event <= _EVENT_MAX); - SMARTLIST_FOREACH(conns, connection_t *, conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { if (conn->type == CONN_TYPE_CONTROL && !conn->marked_for_close && conn->state == CONTROL_CONN_STATE_OPEN) { control_connection_t *control_conn = TO_CONTROL_CONN(conn); - if (control_conn->use_long_names) { - if (!(which & LONG_NAMES)) - continue; - } else { - if (!(which & SHORT_NAMES)) - continue; - } - if (control_conn->use_extended_events) { - if (!(which & EXTENDED_FORMAT)) - continue; - } else { - if (!(which & NONEXTENDED_FORMAT)) - continue; - } + if (control_conn->event_mask & (1<<event)) { int is_err = 0; connection_write_to_buf(msg, strlen(msg), TO_CONN(control_conn)); @@ -579,7 +554,7 @@ send_control_event_string(uint16_t event, event_format_t which, connection_handle_write(TO_CONN(control_conn), 1); } } - }); + } SMARTLIST_FOREACH_END(conn); } /** Helper for send_control1_event and send_control1_event_extended: @@ -587,22 +562,17 @@ send_control_event_string(uint16_t event, event_format_t which, * <b>event</b>. The event's body is created by the printf-style format in * <b>format</b>, and other arguments as provided. * - * If <b>extended</b> is true, and the format contains a single '@' character, - * it will be replaced with a space and all text after that character will be - * sent only to controllers that have enabled extended events. - * * Currently the length of the message is limited to 1024 (including the * ending \\r\\n\\0). */ static void -send_control_event_impl(uint16_t event, event_format_t which, int extended, - const char *format, va_list ap) +send_control_event_impl(uint16_t event, event_format_t which, + const char *format, va_list ap) { /* This is just a little longer than the longest allowed log message */ #define SEND_CONTROL1_EVENT_BUFFERSIZE 10064 int r; char buf[SEND_CONTROL1_EVENT_BUFFERSIZE]; size_t len; - char *cp; r = tor_vsnprintf(buf, sizeof(buf), format, ap); if (r<0) { @@ -618,15 +588,7 @@ send_control_event_impl(uint16_t event, event_format_t which, int extended, buf[SEND_CONTROL1_EVENT_BUFFERSIZE-3] = '\r'; } - if (extended && (cp = strchr(buf, '@'))) { - which &= ~ALL_FORMATS; - *cp = ' '; - send_control_event_string(event, which|EXTENDED_FORMAT, buf); - memcpy(cp, "\r\n\0", 3); - send_control_event_string(event, which|NONEXTENDED_FORMAT, buf); - } else { - send_control_event_string(event, which|ALL_FORMATS, buf); - } + send_control_event_string(event, which|ALL_FORMATS, buf); } /** Send an event to all v1 controllers that are listening for code @@ -641,27 +603,7 @@ send_control_event(uint16_t event, event_format_t which, { va_list ap; va_start(ap, format); - send_control_event_impl(event, which, 0, format, ap); - va_end(ap); -} - -/** Send an event to all v1 controllers that are listening for code - * <b>event</b>. The event's body is created by the printf-style format in - * <b>format</b>, and other arguments as provided. - * - * If the format contains a single '@' character, it will be replaced with a - * space and all text after that character will be sent only to controllers - * that have enabled extended events. - * - * Currently the length of the message is limited to 1024 (including the - * ending \\n\\r\\0. */ -static void -send_control_event_extended(uint16_t event, event_format_t which, - const char *format, ...) -{ - va_list ap; - va_start(ap, format); - send_control_event_impl(event, which, 1, format, ap); + send_control_event_impl(event, which, format, ap); va_end(ap); } @@ -907,36 +849,37 @@ handle_control_loadconf(control_connection_t *conn, uint32_t len, retval = options_init_from_string(body, CMD_RUN_TOR, NULL, &errstring); - if (retval != SETOPT_OK) { + if (retval != SETOPT_OK) log_warn(LD_CONTROL, "Controller gave us config file that didn't validate: %s", errstring); - switch (retval) { - case SETOPT_ERR_PARSE: - msg = "552 Invalid config file"; - break; - case SETOPT_ERR_TRANSITION: - msg = "553 Transition not allowed"; - break; - case SETOPT_ERR_SETTING: - msg = "553 Unable to set option"; - break; - case SETOPT_ERR_MISC: - default: - msg = "550 Unable to load config"; - break; - case SETOPT_OK: - tor_fragile_assert(); - break; - } + + switch (retval) { + case SETOPT_ERR_PARSE: + msg = "552 Invalid config file"; + break; + case SETOPT_ERR_TRANSITION: + msg = "553 Transition not allowed"; + break; + case SETOPT_ERR_SETTING: + msg = "553 Unable to set option"; + break; + case SETOPT_ERR_MISC: + default: + msg = "550 Unable to load config"; + break; + case SETOPT_OK: + break; + } + if (msg) { if (errstring) connection_printf_to_buf(conn, "%s: %s\r\n", msg, errstring); else connection_printf_to_buf(conn, "%s\r\n", msg); - tor_free(errstring); - return 0; + } else { + send_control_done(conn); } - send_control_done(conn); + tor_free(errstring); return 0; } @@ -948,7 +891,6 @@ handle_control_setevents(control_connection_t *conn, uint32_t len, { uint16_t event_code; uint32_t event_mask = 0; - unsigned int extended = 0; smartlist_t *events = smartlist_create(); (void) len; @@ -958,7 +900,6 @@ handle_control_setevents(control_connection_t *conn, uint32_t len, SMARTLIST_FOREACH_BEGIN(events, const char *, ev) { if (!strcasecmp(ev, "EXTENDED")) { - extended = 1; continue; } else if (!strcasecmp(ev, "CIRC")) event_code = EVENT_CIRCUIT_STATUS; @@ -1002,6 +943,8 @@ handle_control_setevents(control_connection_t *conn, uint32_t len, event_code = EVENT_CLIENTS_SEEN; else if (!strcasecmp(ev, "NEWCONSENSUS")) event_code = EVENT_NEWCONSENSUS; + else if (!strcasecmp(ev, "BUILDTIMEOUT_SET")) + event_code = EVENT_BUILDTIMEOUT_SET; else { connection_printf_to_buf(conn, "552 Unrecognized event \"%s\"\r\n", ev); @@ -1016,8 +959,6 @@ handle_control_setevents(control_connection_t *conn, uint32_t len, smartlist_free(events); conn->event_mask = event_mask; - if (extended) - conn->use_extended_events = 1; control_update_global_event_mask(); send_control_done(conn); @@ -1281,7 +1222,9 @@ handle_control_signal(control_connection_t *conn, uint32_t len, /* Flush the "done" first if the signal might make us shut down. */ if (sig == SIGTERM || sig == SIGINT) connection_handle_write(TO_CONN(conn), 1); - control_signal_act(sig); + + process_signal(sig); + return 0; } @@ -1328,7 +1271,7 @@ handle_control_mapaddress(control_connection_t *conn, uint32_t len, smartlist_add(reply, ans); log_warn(LD_CONTROL, "Unable to allocate address for '%s' in MapAddress msg", - safe_str(line)); + safe_str_client(line)); } else { tor_snprintf(ans, anslen, "250-%s=%s", address, to); smartlist_add(reply, ans); @@ -1345,7 +1288,8 @@ handle_control_mapaddress(control_connection_t *conn, uint32_t len, "not of expected form 'foo=bar'.", line); smartlist_add(reply, ans); log_info(LD_CONTROL, "Skipping MapAddress '%s': wrong " - "number of items.", safe_str(line)); + "number of items.", + safe_str_client(line)); } SMARTLIST_FOREACH(elts, char *, cp, tor_free(cp)); smartlist_clear(elts); @@ -1374,13 +1318,15 @@ handle_control_mapaddress(control_connection_t *conn, uint32_t len, * trivial-to-implement questions. */ static int getinfo_helper_misc(control_connection_t *conn, const char *question, - char **answer) + char **answer, const char **errmsg) { (void) conn; if (!strcmp(question, "version")) { *answer = tor_strdup(get_version()); } else if (!strcmp(question, "config-file")) { *answer = tor_strdup(get_torrc_fname()); + } else if (!strcmp(question, "config-text")) { + *answer = options_dump(get_options(), 1); } else if (!strcmp(question, "info/names")) { *answer = list_getinfo_options(); } else if (!strcmp(question, "events/names")) { @@ -1392,15 +1338,19 @@ getinfo_helper_misc(control_connection_t *conn, const char *question, *answer = tor_strdup("VERBOSE_NAMES EXTENDED_EVENTS"); } else if (!strcmp(question, "address")) { uint32_t addr; - if (router_pick_published_address(get_options(), &addr) < 0) + if (router_pick_published_address(get_options(), &addr) < 0) { + *errmsg = "Address unknown"; return -1; + } *answer = tor_dup_ip(addr); } else if (!strcmp(question, "dir-usage")) { *answer = directory_dump_request_log(); } else if (!strcmp(question, "fingerprint")) { routerinfo_t *me = router_get_my_routerinfo(); - if (!me) + if (!me) { + *errmsg = "No routerdesc known; am I really a server?"; return -1; + } *answer = tor_malloc(HEX_DIGEST_LEN+1); base16_encode(*answer, HEX_DIGEST_LEN+1, me->cache_info.identity_digest, DIGEST_LEN); @@ -1461,8 +1411,10 @@ munge_extrainfo_into_routerinfo(const char *ri_body, signed_descriptor_t *ri, * directory information. */ static int getinfo_helper_dir(control_connection_t *control_conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { + (void) control_conn; if (!strcmpstart(question, "desc/id/")) { routerinfo_t *ri = router_get_by_hexdigest(question+strlen("desc/id/")); if (ri) { @@ -1537,6 +1489,7 @@ getinfo_helper_dir(control_connection_t *control_conn, log_warn(LD_CONTROL, "getinfo '%s': %s", question, msg); smartlist_free(descs); tor_free(url); + *errmsg = msg; return -1; } SMARTLIST_FOREACH(descs, signed_descriptor_t *, sd, @@ -1587,7 +1540,7 @@ getinfo_helper_dir(control_connection_t *control_conn, } } else if (!strcmp(question, "dir/status-vote/current/consensus")) { /* v3 */ if (directory_caches_dir_info(get_options())) { - const cached_dir_t *consensus = dirserv_get_consensus(); + const cached_dir_t *consensus = dirserv_get_consensus("ns"); if (consensus) *answer = tor_strdup(consensus->dir); } @@ -1598,10 +1551,8 @@ getinfo_helper_dir(control_connection_t *control_conn, } } else if (!strcmp(question, "network-status")) { /* v1 */ routerlist_t *routerlist = router_get_routerlist(); - int verbose = control_conn->use_long_names; if (!routerlist || !routerlist->routers || - list_server_status_v1(routerlist->routers, answer, - verbose ? 2 : 1) < 0) { + list_server_status_v1(routerlist->routers, answer, 1) < 0) { return -1; } } else if (!strcmpstart(question, "extra-info/digest/")) { @@ -1635,8 +1586,10 @@ getinfo_helper_dir(control_connection_t *control_conn, * current states of things we send events about. */ static int getinfo_helper_events(control_connection_t *control_conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { + (void) control_conn; if (!strcmp(question, "circuit-status")) { circuit_t *circ; smartlist_t *status = smartlist_create(); @@ -1647,10 +1600,9 @@ getinfo_helper_events(control_connection_t *control_conn, const char *purpose; if (! CIRCUIT_IS_ORIGIN(circ) || circ->marked_for_close) continue; - if (control_conn->use_long_names) - path = circuit_list_path_for_controller(TO_ORIGIN_CIRCUIT(circ)); - else - path = circuit_list_path(TO_ORIGIN_CIRCUIT(circ),0); + + path = circuit_list_path_for_controller(TO_ORIGIN_CIRCUIT(circ)); + if (circ->state == CIRCUIT_STATE_OPEN) state = "BUILT"; else if (strlen(path)) @@ -1728,8 +1680,7 @@ getinfo_helper_events(control_connection_t *control_conn, } else if (!strcmp(question, "orconn-status")) { smartlist_t *conns = get_connection_array(); smartlist_t *status = smartlist_create(); - SMARTLIST_FOREACH(conns, connection_t *, base_conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, base_conn) { const char *state; char *s; char name[128]; @@ -1744,29 +1695,19 @@ getinfo_helper_events(control_connection_t *control_conn, state = "LAUNCHED"; else state = "NEW"; - orconn_target_get_name(control_conn->use_long_names, name, sizeof(name), - conn); + orconn_target_get_name(name, sizeof(name), conn); slen = strlen(name)+strlen(state)+2; s = tor_malloc(slen+1); tor_snprintf(s, slen, "%s %s", name, state); smartlist_add(status, s); - }); + } SMARTLIST_FOREACH_END(base_conn); *answer = smartlist_join_strings(status, "\r\n", 0, NULL); SMARTLIST_FOREACH(status, char *, cp, tor_free(cp)); smartlist_free(status); - } else if (!strcmpstart(question, "addr-mappings/") || - !strcmpstart(question, "address-mappings/")) { + } else if (!strcmpstart(question, "address-mappings/")) { time_t min_e, max_e; smartlist_t *mappings; - int want_expiry = !strcmpstart(question, "address-mappings/"); - if (!strcmpstart(question, "addr-mappings/")) { - /* XXXX022 This has been deprecated since 0.2.0.3-alpha, and has - generated a warning since 0.2.1.10-alpha; remove late in 0.2.2.x. */ - log_warn(LD_CONTROL, "Controller used obsolete addr-mappings/ GETINFO " - "key; use address-mappings/ instead."); - } - question += strlen(want_expiry ? "address-mappings/" - : "addr-mappings/"); + question += strlen("address-mappings/"); if (!strcmp(question, "all")) { min_e = 0; max_e = TIME_MAX; } else if (!strcmp(question, "cache")) { @@ -1779,7 +1720,7 @@ getinfo_helper_events(control_connection_t *control_conn, return 0; } mappings = smartlist_create(); - addressmap_get_mappings(mappings, min_e, max_e, want_expiry); + addressmap_get_mappings(mappings, min_e, max_e, 1); *answer = smartlist_join_strings(mappings, "\r\n", 0, NULL); SMARTLIST_FOREACH(mappings, char *, cp, tor_free(cp)); smartlist_free(mappings); @@ -1787,7 +1728,7 @@ getinfo_helper_events(control_connection_t *control_conn, /* Note that status/ is not a catch-all for events; there's only supposed * to be a status GETINFO if there's a corresponding STATUS event. */ if (!strcmp(question, "status/circuit-established")) { - *answer = tor_strdup(has_completed_circuit ? "1" : "0"); + *answer = tor_strdup(can_complete_circuit ? "1" : "0"); } else if (!strcmp(question, "status/enough-dir-info")) { *answer = tor_strdup(router_have_minimum_dir_info() ? "1" : "0"); } else if (!strcmp(question, "status/good-server-descriptor") || @@ -1846,21 +1787,12 @@ getinfo_helper_events(control_connection_t *control_conn, "information", question); } } else if (!strcmp(question, "status/clients-seen")) { - char geoip_start[ISO_TIME_LEN+1]; - size_t answer_len; - char *geoip_summary = extrainfo_get_client_geoip_summary(time(NULL)); - - if (!geoip_summary) + char *bridge_stats = geoip_get_bridge_stats_controller(time(NULL)); + if (!bridge_stats) { + *errmsg = "No bridge-client stats available"; return -1; - - answer_len = strlen("TimeStarted=\"\" CountrySummary=") + - ISO_TIME_LEN + strlen(geoip_summary) + 1; - *answer = tor_malloc(answer_len); - format_iso_time(geoip_start, geoip_get_history_start()); - tor_snprintf(*answer, answer_len, - "TimeStarted=\"%s\" CountrySummary=%s", - geoip_start, geoip_summary); - tor_free(geoip_summary); + } + *answer = bridge_stats; } else { return 0; } @@ -1870,11 +1802,14 @@ getinfo_helper_events(control_connection_t *control_conn, /** Callback function for GETINFO: on a given control connection, try to * answer the question <b>q</b> and store the newly-allocated answer in - * *<b>a</b>. If there's no answer, or an error occurs, just don't set - * <b>a</b>. Return 0. + * *<b>a</b>. If an internal error occurs, return -1 and optionally set + * *<b>error_out</b> to point to an error message to be delivered to the + * controller. On success, _or if the key is not recognized_, return 0. Do not + * set <b>a</b> if the key is not recognized. */ typedef int (*getinfo_helper_t)(control_connection_t *, - const char *q, char **a); + const char *q, char **a, + const char **error_out); /** A single item for the GETINFO question-to-answer-function table. */ typedef struct getinfo_item_t { @@ -1894,6 +1829,8 @@ typedef struct getinfo_item_t { static const getinfo_item_t getinfo_items[] = { ITEM("version", misc, "The current version of Tor."), ITEM("config-file", misc, "Current location of the \"torrc\" file."), + ITEM("config-text", misc, + "Return the string that would be written by a saveconf command."), ITEM("accounting/bytes", accounting, "Number of bytes read/written so far in the accounting interval."), ITEM("accounting/bytes-left", accounting, @@ -1933,7 +1870,6 @@ static const getinfo_item_t getinfo_items[] = { PREFIX("ns/purpose/", networkstatus, "Brief summary of router status by purpose (v2 directory format)."), - PREFIX("unregistered-servers-", dirserv_unregistered, NULL), ITEM("network-status", dir, "Brief summary of router status (v1 directory format)"), ITEM("circuit-status", events, "List of current circuits originating here."), @@ -1945,14 +1881,6 @@ static const getinfo_item_t getinfo_items[] = { DOC("address-mappings/config", "Current address mappings from configuration."), DOC("address-mappings/control", "Current address mappings from controller."), - PREFIX("addr-mappings/", events, NULL), - DOC("addr-mappings/all", "Current address mappings without expiry times."), - DOC("addr-mappings/cache", - "Current cached DNS replies without expiry times."), - DOC("addr-mappings/config", - "Current address mappings from configuration without expiry times."), - DOC("addr-mappings/control", - "Current address mappings from controller without expiry times."), PREFIX("status/", events, NULL), DOC("status/circuit-established", "Whether we think client functionality is working."), @@ -1988,18 +1916,18 @@ static char * list_getinfo_options(void) { int i; - char buf[300]; + char *buf=NULL; smartlist_t *lines = smartlist_create(); char *ans; for (i = 0; getinfo_items[i].varname; ++i) { if (!getinfo_items[i].desc) continue; - tor_snprintf(buf, sizeof(buf), "%s%s -- %s\n", + tor_asprintf(&buf, "%s%s -- %s\n", getinfo_items[i].varname, getinfo_items[i].is_prefix ? "*" : "", getinfo_items[i].desc); - smartlist_add(lines, tor_strdup(buf)); + smartlist_add(lines, buf); } smartlist_sort_strings(lines); @@ -2016,7 +1944,8 @@ list_getinfo_options(void) * internal error. */ static int handle_getinfo_helper(control_connection_t *control_conn, - const char *question, char **answer) + const char *question, char **answer, + const char **err_out) { int i; *answer = NULL; /* unrecognized key by default */ @@ -2029,7 +1958,7 @@ handle_getinfo_helper(control_connection_t *control_conn, match = !strcmp(question, getinfo_items[i].varname); if (match) { tor_assert(getinfo_items[i].fn); - return getinfo_items[i].fn(control_conn, question, answer); + return getinfo_items[i].fn(control_conn, question, answer, err_out); } } @@ -2051,10 +1980,12 @@ handle_control_getinfo(control_connection_t *conn, uint32_t len, smartlist_split_string(questions, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH(questions, const char *, q, - { - if (handle_getinfo_helper(conn, q, &ans) < 0) { - connection_write_str_to_buf("551 Internal error\r\n", conn); + SMARTLIST_FOREACH_BEGIN(questions, const char *, q) { + const char *errmsg = NULL; + if (handle_getinfo_helper(conn, q, &ans, &errmsg) < 0) { + if (!errmsg) + errmsg = "Internal error"; + connection_printf_to_buf(conn, "551 %s\r\n", errmsg); goto done; } if (!ans) { @@ -2063,7 +1994,7 @@ handle_control_getinfo(control_connection_t *conn, uint32_t len, smartlist_add(answers, tor_strdup(q)); smartlist_add(answers, ans); } - }); + } SMARTLIST_FOREACH_END(q); if (smartlist_len(unrecognized)) { for (i=0; i < smartlist_len(unrecognized)-1; ++i) connection_printf_to_buf(conn, @@ -2108,12 +2039,12 @@ handle_control_getinfo(control_connection_t *conn, uint32_t len, static uint8_t circuit_purpose_from_string(const char *string) { - if (!strcmpstart(string, "purpose=")) + if (!strcasecmpstart(string, "purpose=")) string += strlen("purpose="); - if (!strcmp(string, "general")) + if (!strcasecmp(string, "general")) return CIRCUIT_PURPOSE_C_GENERAL; - else if (!strcmp(string, "controller")) + else if (!strcasecmp(string, "controller")) return CIRCUIT_PURPOSE_CONTROLLER; else return CIRCUIT_PURPOSE_UNKNOWN; @@ -2145,6 +2076,31 @@ getargs_helper(const char *command, control_connection_t *conn, return NULL; } +/** Helper. Return the first element of <b>sl</b> at index <b>start_at</b> or + * higher that starts with <b>prefix</b>, case-insensitive. Return NULL if no + * such element exists. */ +static const char * +find_element_starting_with(smartlist_t *sl, int start_at, const char *prefix) +{ + int i; + for (i = start_at; i < smartlist_len(sl); ++i) { + const char *elt = smartlist_get(sl, i); + if (!strcasecmpstart(elt, prefix)) + return elt; + } + return NULL; +} + +/** Helper. Return true iff s is an argument that we should treat as a + * key-value pair. */ +static int +is_keyval_pair(const char *s) +{ + /* An argument is a key-value pair if it has an =, and it isn't of the form + * $fingeprint=name */ + return strchr(s, '=') && s[0] != '$'; +} + /** Called when we get an EXTENDCIRCUIT message. Try to extend the listed * circuit, and report success or failure. */ static int @@ -2160,33 +2116,57 @@ handle_control_extendcircuit(control_connection_t *conn, uint32_t len, router_nicknames = smartlist_create(); - args = getargs_helper("EXTENDCIRCUIT", conn, body, 2, -1); + args = getargs_helper("EXTENDCIRCUIT", conn, body, 1, -1); if (!args) goto done; zero_circ = !strcmp("0", (char*)smartlist_get(args,0)); - if (!zero_circ && !(circ = get_circ(smartlist_get(args,0)))) { - connection_printf_to_buf(conn, "552 Unknown circuit \"%s\"\r\n", - (char*)smartlist_get(args, 0)); - } - smartlist_split_string(router_nicknames, smartlist_get(args,1), ",", 0, 0); - if (zero_circ && smartlist_len(args)>2) { - char *purp = smartlist_get(args,2); - intended_purpose = circuit_purpose_from_string(purp); - if (intended_purpose == CIRCUIT_PURPOSE_UNKNOWN) { - connection_printf_to_buf(conn, "552 Unknown purpose \"%s\"\r\n", purp); + if (zero_circ) { + const char *purp = find_element_starting_with(args, 1, "PURPOSE="); + + if (purp) { + intended_purpose = circuit_purpose_from_string(purp); + if (intended_purpose == CIRCUIT_PURPOSE_UNKNOWN) { + connection_printf_to_buf(conn, "552 Unknown purpose \"%s\"\r\n", purp); + SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); + smartlist_free(args); + goto done; + } + } + + if ((smartlist_len(args) == 1) || + (smartlist_len(args) >= 2 && is_keyval_pair(smartlist_get(args, 1)))) { + // "EXTENDCIRCUIT 0" || EXTENDCIRCUIT 0 foo=bar" + circ = circuit_launch_by_router(intended_purpose, NULL, + CIRCLAUNCH_NEED_CAPACITY); + if (!circ) { + connection_write_str_to_buf("551 Couldn't start circuit\r\n", conn); + } else { + connection_printf_to_buf(conn, "250 EXTENDED %lu\r\n", + (unsigned long)circ->global_identifier); + } SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); smartlist_free(args); goto done; } + // "EXTENDCIRCUIT 0 router1,router2" || + // "EXTENDCIRCUIT 0 router1,router2 PURPOSE=foo" } - SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); - smartlist_free(args); - if (!zero_circ && !circ) { + + if (!zero_circ && !(circ = get_circ(smartlist_get(args,0)))) { + connection_printf_to_buf(conn, "552 Unknown circuit \"%s\"\r\n", + (char*)smartlist_get(args, 0)); + SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); + smartlist_free(args); goto done; } + smartlist_split_string(router_nicknames, smartlist_get(args,1), ",", 0, 0); + + SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); + smartlist_free(args); + routers = smartlist_create(); SMARTLIST_FOREACH(router_nicknames, const char *, n, { @@ -2244,8 +2224,7 @@ handle_control_extendcircuit(control_connection_t *conn, uint32_t len, done: SMARTLIST_FOREACH(router_nicknames, char *, n, tor_free(n)); smartlist_free(router_nicknames); - if (routers) - smartlist_free(routers); + smartlist_free(routers); return 0; } @@ -2271,7 +2250,7 @@ handle_control_setcircuitpurpose(control_connection_t *conn, } { - char *purp = smartlist_get(args,1); + const char *purp = find_element_starting_with(args,1,"PURPOSE="); new_purpose = circuit_purpose_from_string(purp); if (new_purpose == CIRCUIT_PURPOSE_UNKNOWN) { connection_printf_to_buf(conn, "552 Unknown purpose \"%s\"\r\n", purp); @@ -2282,7 +2261,7 @@ handle_control_setcircuitpurpose(control_connection_t *conn, circ->_base.purpose = new_purpose; connection_write_str_to_buf("250 OK\r\n", conn); -done: + done: if (args) { SMARTLIST_FOREACH(args, char *, cp, tor_free(cp)); smartlist_free(args); @@ -2316,9 +2295,9 @@ handle_control_attachstream(control_connection_t *conn, uint32_t len, } else if (!zero_circ && !(circ = get_circ(smartlist_get(args, 1)))) { connection_printf_to_buf(conn, "552 Unknown circuit \"%s\"\r\n", (char*)smartlist_get(args, 1)); - } else if (circ && smartlist_len(args) > 2) { - char *hopstring = smartlist_get(args, 2); - if (!strcasecmpstart(hopstring, "HOP=")) { + } else if (circ) { + const char *hopstring = find_element_starting_with(args,2,"HOP="); + if (hopstring) { hopstring += strlen("HOP="); hop = (int) tor_parse_ulong(hopstring, 10, 0, INT_MAX, &hop_line_ok, NULL); @@ -2365,7 +2344,7 @@ handle_control_attachstream(control_connection_t *conn, uint32_t len, char* exit_digest; if (circ->build_state && circ->build_state->chosen_exit && - circ->build_state->chosen_exit->identity_digest) { + !tor_digest_is_zero(circ->build_state->chosen_exit->identity_digest)) { exit_digest = circ->build_state->chosen_exit->identity_digest; r = router_get_by_digest(exit_digest); } @@ -2426,9 +2405,9 @@ handle_control_postdescriptor(control_connection_t *conn, uint32_t len, } } else if (!strcasecmpstart(option, "cache=")) { option += strlen("cache="); - if (!strcmp(option, "no")) + if (!strcasecmp(option, "no")) cache = 0; - else if (!strcmp(option, "yes")) + else if (!strcasecmp(option, "yes")) cache = 1; else { connection_printf_to_buf(conn, "552 Unknown cache request \"%s\"\r\n", @@ -2610,17 +2589,17 @@ handle_control_resolve(control_connection_t *conn, uint32_t len, args = smartlist_create(); smartlist_split_string(args, body, " ", SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - if (smartlist_len(args) && - !strcasecmp(smartlist_get(args, 0), "mode=reverse")) { - char *cp = smartlist_get(args, 0); - smartlist_del_keeporder(args, 0); - tor_free(cp); - is_reverse = 1; + { + const char *modearg = find_element_starting_with(args, 0, "mode="); + if (modearg && !strcasecmp(modearg, "mode=reverse")) + is_reverse = 1; } failed = smartlist_create(); SMARTLIST_FOREACH(args, const char *, arg, { - if (dnsserv_launch_request(arg, is_reverse)<0) - smartlist_add(failed, (char*)arg); + if (!is_keyval_pair(arg)) { + if (dnsserv_launch_request(arg, is_reverse)<0) + smartlist_add(failed, (char*)arg); + } }); send_control_done(conn); @@ -2710,7 +2689,6 @@ handle_control_usefeature(control_connection_t *conn, const char *body) { smartlist_t *args; - int verbose_names = 0, extended_events = 0; int bad = 0; (void) len; /* body is nul-terminated; it's safe to ignore the length */ args = smartlist_create(); @@ -2718,9 +2696,9 @@ handle_control_usefeature(control_connection_t *conn, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); SMARTLIST_FOREACH(args, const char *, arg, { if (!strcasecmp(arg, "VERBOSE_NAMES")) - verbose_names = 1; + ; else if (!strcasecmp(arg, "EXTENDED_EVENTS")) - extended_events = 1; + ; else { connection_printf_to_buf(conn, "552 Unrecognized feature \"%s\"\r\n", arg); @@ -2730,12 +2708,6 @@ handle_control_usefeature(control_connection_t *conn, }); if (!bad) { - if (verbose_names) { - conn->use_long_names = 1; - control_update_global_event_mask(); - } - if (extended_events) - conn->use_extended_events = 1; send_control_done(conn); } @@ -2885,9 +2857,10 @@ connection_control_process_inbuf(control_connection_t *conn) && !TOR_ISSPACE(conn->incoming_cmd[cmd_len])) ++cmd_len; - data_len -= cmd_len; conn->incoming_cmd[cmd_len]='\0'; args = conn->incoming_cmd+cmd_len+1; + tor_assert(data_len>(size_t)cmd_len); + data_len -= (cmd_len+1); /* skip the command and NUL we added after it */ while (*args == ' ' || *args == '\t') { ++args; --data_len; @@ -3039,20 +3012,11 @@ control_event_circuit_status(origin_circuit_t *circ, circuit_status_event_t tp, tor_free(reason); } - if (EVENT_IS_INTERESTING1S(EVENT_CIRCUIT_STATUS)) { - char *path = circuit_list_path(circ,0); - const char *sp = strlen(path) ? " " : ""; - send_control_event_extended(EVENT_CIRCUIT_STATUS, SHORT_NAMES, - "650 CIRC %lu %s%s%s@%s\r\n", - (unsigned long)circ->global_identifier, - status, sp, path, extended_buf); - tor_free(path); - } - if (EVENT_IS_INTERESTING1L(EVENT_CIRCUIT_STATUS)) { + { char *vpath = circuit_list_path_for_controller(circ); const char *sp = strlen(vpath) ? " " : ""; - send_control_event_extended(EVENT_CIRCUIT_STATUS, LONG_NAMES, - "650 CIRC %lu %s%s%s@%s\r\n", + send_control_event(EVENT_CIRCUIT_STATUS, ALL_FORMATS, + "650 CIRC %lu %s%s%s %s\r\n", (unsigned long)circ->global_identifier, status, sp, vpath, extended_buf); tor_free(vpath); @@ -3131,26 +3095,26 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, char *r = NULL; if (!reason_str) { r = tor_malloc(16); - tor_snprintf(r, 16, "UNKNOWN_%d", reason_code); + tor_snprintf(r, 16, " UNKNOWN_%d", reason_code); reason_str = r; } if (reason_code & END_STREAM_REASON_FLAG_REMOTE) tor_snprintf(reason_buf, sizeof(reason_buf), - "REASON=END REMOTE_REASON=%s", reason_str); + " REASON=END REMOTE_REASON=%s", reason_str); else tor_snprintf(reason_buf, sizeof(reason_buf), - "REASON=%s", reason_str); + " REASON=%s", reason_str); tor_free(r); } else if (reason_code && tp == STREAM_EVENT_REMAP) { switch (reason_code) { case REMAP_STREAM_SOURCE_CACHE: - strlcpy(reason_buf, "SOURCE=CACHE", sizeof(reason_buf)); + strlcpy(reason_buf, " SOURCE=CACHE", sizeof(reason_buf)); break; case REMAP_STREAM_SOURCE_EXIT: - strlcpy(reason_buf, "SOURCE=EXIT", sizeof(reason_buf)); + strlcpy(reason_buf, " SOURCE=EXIT", sizeof(reason_buf)); break; default: - tor_snprintf(reason_buf, sizeof(reason_buf), "REASON=UNKNOWN_%d", + tor_snprintf(reason_buf, sizeof(reason_buf), " REASON=UNKNOWN_%d", reason_code); /* XXX do we want SOURCE=UNKNOWN_%d above instead? -RD */ break; @@ -3158,8 +3122,7 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, } if (tp == STREAM_EVENT_NEW) { - tor_snprintf(addrport_buf,sizeof(addrport_buf), "%sSOURCE_ADDR=%s:%d", - strlen(reason_buf) ? " " : "", + tor_snprintf(addrport_buf,sizeof(addrport_buf), " SOURCE_ADDR=%s:%d", TO_CONN(conn)->address, TO_CONN(conn)->port ); } else { addrport_buf[0] = '\0'; @@ -3188,8 +3151,8 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, circ = circuit_get_by_edge_conn(conn); if (circ && CIRCUIT_IS_ORIGIN(circ)) origin_circ = TO_ORIGIN_CIRCUIT(circ); - send_control_event_extended(EVENT_STREAM_STATUS, ALL_NAMES, - "650 STREAM "U64_FORMAT" %s %lu %s@%s%s%s\r\n", + send_control_event(EVENT_STREAM_STATUS, ALL_FORMATS, + "650 STREAM "U64_FORMAT" %s %lu %s%s%s%s\r\n", U64_PRINTF_ARG(conn->_base.global_identifier), status, origin_circ? (unsigned long)origin_circ->global_identifier : 0ul, @@ -3202,30 +3165,21 @@ control_event_stream_status(edge_connection_t *conn, stream_status_event_t tp, /** Figure out the best name for the target router of an OR connection * <b>conn</b>, and write it into the <b>len</b>-character buffer - * <b>name</b>. Use verbose names if <b>long_names</b> is set. */ + * <b>name</b>. */ static void -orconn_target_get_name(int long_names, - char *name, size_t len, or_connection_t *conn) -{ - if (! long_names) { - if (conn->nickname) - strlcpy(name, conn->nickname, len); - else - tor_snprintf(name, len, "%s:%d", - conn->_base.address, conn->_base.port); +orconn_target_get_name(char *name, size_t len, or_connection_t *conn) +{ + routerinfo_t *ri = router_get_by_digest(conn->identity_digest); + if (ri) { + tor_assert(len > MAX_VERBOSE_NICKNAME_LEN); + router_get_verbose_nickname(name, ri); + } else if (! tor_digest_is_zero(conn->identity_digest)) { + name[0] = '$'; + base16_encode(name+1, len-1, conn->identity_digest, + DIGEST_LEN); } else { - routerinfo_t *ri = router_get_by_digest(conn->identity_digest); - if (ri) { - tor_assert(len > MAX_VERBOSE_NICKNAME_LEN); - router_get_verbose_nickname(name, ri); - } else if (! tor_digest_is_zero(conn->identity_digest)) { - name[0] = '$'; - base16_encode(name+1, len-1, conn->identity_digest, - DIGEST_LEN); - } else { - tor_snprintf(name, len, "%s:%d", - conn->_base.address, conn->_base.port); - } + tor_snprintf(name, len, "%s:%d", + conn->_base.address, conn->_base.port); } } @@ -3264,24 +3218,13 @@ control_event_or_conn_status(or_connection_t *conn, or_conn_status_event_t tp, reason ? " " : "", ncircs); } - if (EVENT_IS_INTERESTING1S(EVENT_OR_CONN_STATUS)) { - orconn_target_get_name(0, name, sizeof(name), conn); - send_control_event_extended(EVENT_OR_CONN_STATUS, SHORT_NAMES, - "650 ORCONN %s %s@%s%s%s\r\n", - name, status, - reason ? "REASON=" : "", - orconn_end_reason_to_control_string(reason), - ncircs_buf); - } - if (EVENT_IS_INTERESTING1L(EVENT_OR_CONN_STATUS)) { - orconn_target_get_name(1, name, sizeof(name), conn); - send_control_event_extended(EVENT_OR_CONN_STATUS, LONG_NAMES, - "650 ORCONN %s %s@%s%s%s\r\n", - name, status, - reason ? "REASON=" : "", - orconn_end_reason_to_control_string(reason), - ncircs_buf); - } + orconn_target_get_name(name, sizeof(name), conn); + send_control_event(EVENT_OR_CONN_STATUS, ALL_FORMATS, + "650 ORCONN %s %s %s%s%s\r\n", + name, status, + reason ? "REASON=" : "", + orconn_end_reason_to_control_string(reason), + ncircs_buf); return 0; } @@ -3296,7 +3239,7 @@ control_event_stream_bandwidth(edge_connection_t *edge_conn) if (!edge_conn->n_read && !edge_conn->n_written) return 0; - send_control_event(EVENT_STREAM_BANDWIDTH_USED, ALL_NAMES, + send_control_event(EVENT_STREAM_BANDWIDTH_USED, ALL_FORMATS, "650 STREAM_BW "U64_FORMAT" %lu %lu\r\n", U64_PRINTF_ARG(edge_conn->_base.global_identifier), (unsigned long)edge_conn->n_read, @@ -3325,7 +3268,7 @@ control_event_stream_bandwidth_used(void) if (!edge_conn->n_read && !edge_conn->n_written) continue; - send_control_event(EVENT_STREAM_BANDWIDTH_USED, ALL_NAMES, + send_control_event(EVENT_STREAM_BANDWIDTH_USED, ALL_FORMATS, "650 STREAM_BW "U64_FORMAT" %lu %lu\r\n", U64_PRINTF_ARG(edge_conn->_base.global_identifier), (unsigned long)edge_conn->n_read, @@ -3345,7 +3288,7 @@ int control_event_bandwidth_used(uint32_t n_read, uint32_t n_written) { if (EVENT_IS_INTERESTING(EVENT_BANDWIDTH_USED)) { - send_control_event(EVENT_BANDWIDTH_USED, ALL_NAMES, + send_control_event(EVENT_BANDWIDTH_USED, ALL_FORMATS, "650 BW %lu %lu\r\n", (unsigned long)n_read, (unsigned long)n_written); @@ -3415,7 +3358,7 @@ control_event_logmsg(int severity, uint32_t domain, const char *msg) default: s = "UnknownLogSeverity"; break; } ++disable_log_messages; - send_control_event(event, ALL_NAMES, "650 %s %s\r\n", s, b?b:msg); + send_control_event(event, ALL_FORMATS, "650 %s %s\r\n", s, b?b:msg); --disable_log_messages; tor_free(b); } @@ -3428,31 +3371,12 @@ control_event_logmsg(int severity, uint32_t domain, const char *msg) int control_event_descriptors_changed(smartlist_t *routers) { - size_t len; char *msg; - smartlist_t *identities = NULL; - char buf[HEX_DIGEST_LEN+1]; if (!EVENT_IS_INTERESTING(EVENT_NEW_DESC)) return 0; - if (EVENT_IS_INTERESTING1S(EVENT_NEW_DESC)) { - identities = smartlist_create(); - SMARTLIST_FOREACH(routers, routerinfo_t *, r, - { - base16_encode(buf,sizeof(buf),r->cache_info.identity_digest,DIGEST_LEN); - smartlist_add(identities, tor_strdup(buf)); - }); - } - if (EVENT_IS_INTERESTING1S(EVENT_NEW_DESC)) { - char *ids = smartlist_join_strings(identities, " ", 0, &len); - size_t ids_len = strlen(ids)+32; - msg = tor_malloc(ids_len); - tor_snprintf(msg, ids_len, "650 NEWDESC %s\r\n", ids); - send_control_event_string(EVENT_NEW_DESC, SHORT_NAMES|ALL_FORMATS, msg); - tor_free(ids); - tor_free(msg); - } - if (EVENT_IS_INTERESTING1L(EVENT_NEW_DESC)) { + + { smartlist_t *names = smartlist_create(); char *ids; size_t names_len; @@ -3465,16 +3389,12 @@ control_event_descriptors_changed(smartlist_t *routers) names_len = strlen(ids)+32; msg = tor_malloc(names_len); tor_snprintf(msg, names_len, "650 NEWDESC %s\r\n", ids); - send_control_event_string(EVENT_NEW_DESC, LONG_NAMES|ALL_FORMATS, msg); + send_control_event_string(EVENT_NEW_DESC, ALL_FORMATS, msg); tor_free(ids); tor_free(msg); SMARTLIST_FOREACH(names, char *, cp, tor_free(cp)); smartlist_free(names); } - if (identities) { - SMARTLIST_FOREACH(identities, char *, cp, tor_free(cp)); - smartlist_free(identities); - } return 0; } @@ -3491,17 +3411,17 @@ control_event_address_mapped(const char *from, const char *to, time_t expires, return 0; if (expires < 3 || expires == TIME_MAX) - send_control_event_extended(EVENT_ADDRMAP, ALL_NAMES, - "650 ADDRMAP %s %s NEVER@%s\r\n", from, to, + send_control_event(EVENT_ADDRMAP, ALL_FORMATS, + "650 ADDRMAP %s %s NEVER %s\r\n", from, to, error?error:""); else { char buf[ISO_TIME_LEN+1]; char buf2[ISO_TIME_LEN+1]; format_local_iso_time(buf,expires); format_iso_time(buf2,expires); - send_control_event_extended(EVENT_ADDRMAP, ALL_NAMES, + send_control_event(EVENT_ADDRMAP, ALL_FORMATS, "650 ADDRMAP %s %s \"%s\"" - "@%s%sEXPIRES=\"%s\"\r\n", + " %s%sEXPIRES=\"%s\"\r\n", from, to, buf, error?error:"", error?" ":"", buf2); @@ -3541,9 +3461,9 @@ control_event_or_authdir_new_descriptor(const char *action, buf = tor_malloc(totallen); strlcpy(buf, firstline, totallen); strlcpy(buf+strlen(firstline), esc, totallen); - send_control_event_string(EVENT_AUTHDIR_NEWDESCS, ALL_NAMES|ALL_FORMATS, + send_control_event_string(EVENT_AUTHDIR_NEWDESCS, ALL_FORMATS, buf); - send_control_event_string(EVENT_AUTHDIR_NEWDESCS, ALL_NAMES|ALL_FORMATS, + send_control_event_string(EVENT_AUTHDIR_NEWDESCS, ALL_FORMATS, "650 OK\r\n"); tor_free(esc); tor_free(buf); @@ -3581,8 +3501,8 @@ control_event_networkstatus_changed_helper(smartlist_t *statuses, SMARTLIST_FOREACH(strs, char *, cp, tor_free(cp)); smartlist_free(strs); tor_free(s); - send_control_event_string(event, ALL_NAMES|ALL_FORMATS, esc); - send_control_event_string(event, ALL_NAMES|ALL_FORMATS, + send_control_event_string(event, ALL_FORMATS, esc); + send_control_event_string(event, ALL_FORMATS, "650 OK\r\n"); tor_free(esc); @@ -3608,6 +3528,55 @@ control_event_newconsensus(const networkstatus_t *consensus) consensus->routerstatus_list, EVENT_NEWCONSENSUS, "NEWCONSENSUS"); } +/** Called when we compute a new circuitbuildtimeout */ +int +control_event_buildtimeout_set(const circuit_build_times_t *cbt, + buildtimeout_set_event_t type) +{ + const char *type_string = NULL; + double qnt = circuit_build_times_quantile_cutoff(); + + if (!control_event_is_interesting(EVENT_BUILDTIMEOUT_SET)) + return 0; + + switch (type) { + case BUILDTIMEOUT_SET_EVENT_COMPUTED: + type_string = "COMPUTED"; + break; + case BUILDTIMEOUT_SET_EVENT_RESET: + type_string = "RESET"; + qnt = 1.0; + break; + case BUILDTIMEOUT_SET_EVENT_SUSPENDED: + type_string = "SUSPENDED"; + qnt = 1.0; + break; + case BUILDTIMEOUT_SET_EVENT_DISCARD: + type_string = "DISCARD"; + qnt = 1.0; + break; + case BUILDTIMEOUT_SET_EVENT_RESUME: + type_string = "RESUME"; + break; + default: + type_string = "UNKNOWN"; + break; + } + + send_control_event(EVENT_BUILDTIMEOUT_SET, ALL_FORMATS, + "650 BUILDTIMEOUT_SET %s TOTAL_TIMES=%lu " + "TIMEOUT_MS=%lu XM=%lu ALPHA=%lf CUTOFF_QUANTILE=%lf " + "TIMEOUT_RATE=%lf CLOSE_MS=%lu CLOSE_RATE=%lf\r\n", + type_string, (unsigned long)cbt->total_build_times, + (unsigned long)cbt->timeout_ms, + (unsigned long)cbt->Xm, cbt->alpha, qnt, + circuit_build_times_timeout_rate(cbt), + (unsigned long)cbt->close_ms, + circuit_build_times_close_rate(cbt)); + + return 0; +} + /** Called when a single local_routerstatus_t has changed: Sends an NS event * to any controller that cares. */ int @@ -3631,7 +3600,7 @@ control_event_networkstatus_changed_single(routerstatus_t *rs) int control_event_my_descriptor_changed(void) { - send_control_event(EVENT_DESCCHANGED, ALL_NAMES, "650 DESCCHANGED\r\n"); + send_control_event(EVENT_DESCCHANGED, ALL_FORMATS, "650 DESCCHANGED\r\n"); return 0; } @@ -3679,7 +3648,7 @@ control_event_status(int type, int severity, const char *format, va_list args) return -1; } - send_control_event_impl(type, ALL_NAMES|ALL_FORMATS, 0, format_buf, args); + send_control_event_impl(type, ALL_FORMATS, format_buf, args); return 0; } @@ -3743,7 +3712,7 @@ control_event_guard(const char *nickname, const char *digest, if (!EVENT_IS_INTERESTING(EVENT_GUARD)) return 0; - if (EVENT_IS_INTERESTING1L(EVENT_GUARD)) { + { char buf[MAX_VERBOSE_NICKNAME_LEN+1]; routerinfo_t *ri = router_get_by_digest(digest); if (ri) { @@ -3751,13 +3720,9 @@ control_event_guard(const char *nickname, const char *digest, } else { tor_snprintf(buf, sizeof(buf), "$%s~%s", hbuf, nickname); } - send_control_event(EVENT_GUARD, LONG_NAMES, + send_control_event(EVENT_GUARD, ALL_FORMATS, "650 GUARD ENTRY %s %s\r\n", buf, status); } - if (EVENT_IS_INTERESTING1S(EVENT_GUARD)) { - send_control_event(EVENT_GUARD, SHORT_NAMES, - "650 GUARD ENTRY $%s %s\r\n", hbuf, status); - } return 0; } @@ -3910,7 +3875,7 @@ static int bootstrap_problems = 0; * information and initial circuits. * * <b>status</b> is the new status, that is, what task we will be doing - * next. <b>percent</b> is zero if we just started this task, else it + * next. <b>progress</b> is zero if we just started this task, else it * represents progress on the task. */ void control_event_bootstrap(bootstrap_status_t status, int progress) @@ -3966,6 +3931,9 @@ control_event_bootstrap_problem(const char *warn, int reason) char buf[BOOTSTRAP_MSG_LEN]; const char *recommendation = "ignore"; + /* bootstrap_percent must not be in "undefined" state here. */ + tor_assert(status >= 0); + if (bootstrap_percent == 100) return; /* already bootstrapped; nothing to be done here. */ @@ -4008,10 +3976,9 @@ control_event_bootstrap_problem(const char *warn, int reason) * from recently. Send a copy to the controller in case it wants to * display it for the user. */ void -control_event_clients_seen(const char *timestarted, const char *countries) +control_event_clients_seen(const char *controller_str) { send_control_event(EVENT_CLIENTS_SEEN, 0, - "650 CLIENTS_SEEN TimeStarted=\"%s\" CountrySummary=%s\r\n", - timestarted, countries); + "650 CLIENTS_SEEN %s\r\n", controller_str); } diff --git a/src/or/control.h b/src/or/control.h new file mode 100644 index 000000000..2ae96b4b8 --- /dev/null +++ b/src/or/control.h @@ -0,0 +1,85 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file control.h + * \brief Header file for control.c. + **/ + +#ifndef _TOR_CONTROL_H +#define _TOR_CONTROL_H + +void control_update_global_event_mask(void); +void control_adjust_event_log_severity(void); + +/** Log information about the connection <b>conn</b>, protecting it as with + * CONN_LOG_PROTECT. Example: + * + * LOG_FN_CONN(conn, (LOG_DEBUG, "Socket %d wants to write", conn->s)); + **/ +#define LOG_FN_CONN(conn, args) \ + CONN_LOG_PROTECT(conn, log_fn args) + +int connection_control_finished_flushing(control_connection_t *conn); +int connection_control_reached_eof(control_connection_t *conn); +int connection_control_process_inbuf(control_connection_t *conn); + +#define EVENT_AUTHDIR_NEWDESCS 0x000D +#define EVENT_NS 0x000F +int control_event_is_interesting(int event); + +int control_event_circuit_status(origin_circuit_t *circ, + circuit_status_event_t e, int reason); +int control_event_stream_status(edge_connection_t *conn, + stream_status_event_t e, + int reason); +int control_event_or_conn_status(or_connection_t *conn, + or_conn_status_event_t e, int reason); +int control_event_bandwidth_used(uint32_t n_read, uint32_t n_written); +int control_event_stream_bandwidth(edge_connection_t *edge_conn); +int control_event_stream_bandwidth_used(void); +void control_event_logmsg(int severity, unsigned int domain, const char *msg); +int control_event_descriptors_changed(smartlist_t *routers); +int control_event_address_mapped(const char *from, const char *to, + time_t expires, const char *error); +int control_event_or_authdir_new_descriptor(const char *action, + const char *desc, + size_t desclen, + const char *msg); +int control_event_my_descriptor_changed(void); +int control_event_networkstatus_changed(smartlist_t *statuses); + +int control_event_newconsensus(const networkstatus_t *consensus); +int control_event_networkstatus_changed_single(routerstatus_t *rs); +int control_event_general_status(int severity, const char *format, ...) + CHECK_PRINTF(2,3); +int control_event_client_status(int severity, const char *format, ...) + CHECK_PRINTF(2,3); +int control_event_server_status(int severity, const char *format, ...) + CHECK_PRINTF(2,3); +int control_event_guard(const char *nickname, const char *digest, + const char *status); +int control_event_buildtimeout_set(const circuit_build_times_t *cbt, + buildtimeout_set_event_t type); + +int init_cookie_authentication(int enabled); +smartlist_t *decode_hashed_passwords(config_line_t *passwords); +void disable_control_logging(void); +void enable_control_logging(void); + +void control_event_bootstrap(bootstrap_status_t status, int progress); +void control_event_bootstrap_problem(const char *warn, int reason); + +void control_event_clients_seen(const char *controller_str); + +#ifdef CONTROL_PRIVATE +/* Used only by control.c and test.c */ +size_t write_escaped_data(const char *data, size_t len, char **out); +size_t read_escaped_data(const char *data, size_t len, char **out); +#endif + +#endif + diff --git a/src/or/cpuworker.c b/src/or/cpuworker.c index 91af5f1d6..7cbc19133 100644 --- a/src/or/cpuworker.c +++ b/src/or/cpuworker.c @@ -13,6 +13,15 @@ **/ #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "config.h" +#include "connection.h" +#include "cpuworker.h" +#include "main.h" +#include "onion.h" +#include "router.h" /** The maximum number of cpuworker processes we will keep around. */ #define MAX_CPUWORKERS 16 @@ -183,7 +192,7 @@ connection_cpu_process_inbuf(connection_t *conn) tor_assert(0); /* don't ask me to do handshakes yet */ } -done_processing: + done_processing: conn->state = CPUWORKER_STATE_IDLE; num_cpuworkers_busy--; if (conn->timestamp_created < last_rotation_time) { @@ -241,7 +250,7 @@ cpuworker_main(void *data) for (;;) { ssize_t r; - if ((r = recv(fd, &question_type, 1, 0)) != 1) { + if ((r = recv(fd, (void *)&question_type, 1, 0)) != 1) { // log_fn(LOG_ERR,"read type failed. Exiting."); if (r == 0) { log_info(LD_OR, @@ -358,7 +367,7 @@ spawn_cpuworker(void) static void spawn_enough_cpuworkers(void) { - int num_cpuworkers_needed = get_options()->NumCpus; + int num_cpuworkers_needed = get_options()->NumCPUs; if (num_cpuworkers_needed < MIN_CPUWORKERS) num_cpuworkers_needed = MIN_CPUWORKERS; diff --git a/src/or/cpuworker.h b/src/or/cpuworker.h new file mode 100644 index 000000000..04e37ee45 --- /dev/null +++ b/src/or/cpuworker.h @@ -0,0 +1,25 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file cpuworker.h + * \brief Header file for cpuworker.c. + **/ + +#ifndef _TOR_CPUWORKER_H +#define _TOR_CPUWORKER_H + +void cpu_init(void); +void cpuworkers_rotate(void); +int connection_cpu_finished_flushing(connection_t *conn); +int connection_cpu_reached_eof(connection_t *conn); +int connection_cpu_process_inbuf(connection_t *conn); +int assign_onionskin_to_cpuworker(connection_t *cpuworker, + or_circuit_t *circ, + char *onionskin); + +#endif + diff --git a/src/or/directory.c b/src/or/directory.c index 7150fce40..68734e604 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -4,6 +4,26 @@ /* See LICENSE for licensing information */ #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "control.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "geoip.h" +#include "main.h" +#include "networkstatus.h" +#include "policies.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" + #if defined(EXPORTMALLINFO) && defined(HAVE_MALLOC_H) && defined(HAVE_MALLINFO) #ifndef OPENBSD #include <malloc.h> @@ -47,8 +67,10 @@ static void http_set_address_origin(const char *headers, connection_t *conn); static void connection_dir_download_networkstatus_failed( dir_connection_t *conn, int status_code); static void connection_dir_download_routerdesc_failed(dir_connection_t *conn); +static void connection_dir_bridge_routerdesc_failed(dir_connection_t *conn); static void connection_dir_download_cert_failed( dir_connection_t *conn, int status_code); +static void connection_dir_retry_bridges(smartlist_t *descs); static void dir_networkstatus_download_failed(smartlist_t *failed, int status_code); static void dir_routerdesc_download_failed(smartlist_t *failed, @@ -92,17 +114,19 @@ static void directory_initiate_command_rend(const char *address, #define ROUTERDESC_CACHE_LIFETIME (30*60) #define ROUTERDESC_BY_DIGEST_CACHE_LIFETIME (48*60*60) #define ROBOTS_CACHE_LIFETIME (24*60*60) +#define MICRODESC_CACHE_LIFETIME (48*60*60) /********* END VARIABLES ************/ -/** Return true iff the directory purpose 'purpose' must use an - * anonymous connection to a directory. */ +/** Return true iff the directory purpose <b>dir_purpose</b> (and if it's + * fetching descriptors, it's fetching them for <b>router_purpose</b>) + * must use an anonymous connection to a directory. */ static int purpose_needs_anonymity(uint8_t dir_purpose, uint8_t router_purpose) { if (get_options()->AllDirActionsPrivate) return 1; - if (router_purpose == ROUTER_PURPOSE_BRIDGE && has_completed_circuit) + if (router_purpose == ROUTER_PURPOSE_BRIDGE && can_complete_circuit) return 1; /* if no circuits yet, we may need this info to bootstrap. */ if (dir_purpose == DIR_PURPOSE_UPLOAD_DIR || dir_purpose == DIR_PURPOSE_UPLOAD_VOTE || @@ -229,10 +253,13 @@ directories_have_accepted_server_descriptor(void) } /** Start a connection to every suitable directory authority, using - * connection purpose 'purpose' and uploading the payload 'payload' - * (length 'payload_len'). The purpose should be one of + * connection purpose <b>dir_purpose</b> and uploading <b>payload</b> + * (of length <b>payload_len</b>). The dir_purpose should be one of * 'DIR_PURPOSE_UPLOAD_DIR' or 'DIR_PURPOSE_UPLOAD_RENDDESC'. * + * <b>router_purpose</b> describes the type of descriptor we're + * publishing, if we're publishing a descriptor -- e.g. general or bridge. + * * <b>type</b> specifies what sort of dir authorities (V1, V2, * HIDSERV, BRIDGE) we should upload to. * @@ -248,6 +275,7 @@ directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, const char *payload, size_t payload_len, size_t extrainfo_len) { + or_options_t *options = get_options(); int post_via_tor; smartlist_t *dirservers = router_get_trusted_dir_servers(); int found = 0; @@ -263,6 +291,16 @@ directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, if ((type & ds->type) == 0) continue; + if (options->ExcludeNodes && options->StrictNodes && + routerset_contains_routerstatus(options->ExcludeNodes, rs)) { + log_warn(LD_DIR, "Wanted to contact authority '%s' for %s, but " + "it's in our ExcludedNodes list and StrictNodes is set. " + "Skipping.", + ds->nickname, + dir_conn_purpose_to_string(dir_purpose)); + continue; + } + found = 1; /* at least one authority of this type was listed */ if (dir_purpose == DIR_PURPOSE_UPLOAD_DIR) ds->has_accepted_serverdesc = 0; @@ -315,6 +353,7 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, break; case DIR_PURPOSE_FETCH_V2_NETWORKSTATUS: type = V2_AUTHORITY; + prefer_authority = 1; /* Only v2 authorities have these anyway. */ break; case DIR_PURPOSE_FETCH_SERVERDESC: type = (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_AUTHORITY : @@ -348,7 +387,7 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, if (!get_via_tor) { if (options->UseBridges && type != BRIDGE_AUTHORITY) { /* want to ask a running bridge for which we have a descriptor. */ - /* XXX022 we assume that all of our bridges can answer any + /* XXX023 we assume that all of our bridges can answer any * possible directory question. This won't be true forever. -RD */ /* It certainly is not true with conditional consensus downloading, * so, for now, never assume the server supports that. */ @@ -472,12 +511,14 @@ directory_initiate_command_routerstatus_rend(routerstatus_t *status, time_t if_modified_since, const rend_data_t *rend_query) { + or_options_t *options = get_options(); routerinfo_t *router; char address_buf[INET_NTOA_BUF_LEN+1]; struct in_addr in; const char *address; tor_addr_t addr; router = router_get_by_digest(status->identity_digest); + if (!router && anonymized_connection) { log_info(LD_DIR, "Not sending anonymized request to directory '%s'; we " "don't have its router descriptor.", status->nickname); @@ -490,6 +531,17 @@ directory_initiate_command_routerstatus_rend(routerstatus_t *status, address = address_buf; } tor_addr_from_ipv4h(&addr, status->addr); + + if (options->ExcludeNodes && options->StrictNodes && + routerset_contains_routerstatus(options->ExcludeNodes, status)) { + log_warn(LD_DIR, "Wanted to contact directory mirror '%s' for %s, but " + "it's in our ExcludedNodes list and StrictNodes is set. " + "Skipping. This choice might make your Tor not work.", + status->nickname, + dir_conn_purpose_to_string(dir_purpose)); + return; + } + directory_initiate_command_rend(address, &addr, status->or_port, status->dir_port, status->version_supports_conditional_consensus, @@ -560,7 +612,7 @@ connection_dir_request_failed(dir_connection_t *conn) if (directory_conn_is_self_reachability_test(conn)) { return; /* this was a test fetch. don't retry. */ } - if (entry_list_can_grow(get_options())) + if (!entry_list_is_constrained(get_options())) router_set_status(conn->identity_digest, 0); /* don't try him again */ if (conn->_base.purpose == DIR_PURPOSE_FETCH_V2_NETWORKSTATUS) { log_info(LD_DIR, "Giving up on directory server at '%s'; retrying", @@ -570,6 +622,8 @@ connection_dir_request_failed(dir_connection_t *conn) conn->_base.purpose == DIR_PURPOSE_FETCH_EXTRAINFO) { log_info(LD_DIR, "Giving up on directory server at '%s'; retrying", conn->_base.address); + if (conn->router_purpose == ROUTER_PURPOSE_BRIDGE) + connection_dir_bridge_routerdesc_failed(conn); connection_dir_download_routerdesc_failed(conn); } else if (conn->_base.purpose == DIR_PURPOSE_FETCH_CONSENSUS) { networkstatus_consensus_download_failed(0); @@ -614,7 +668,7 @@ connection_dir_download_networkstatus_failed(dir_connection_t *conn, * failed, and possibly retry them later.*/ smartlist_t *failed = smartlist_create(); dir_split_resource_into_fingerprints(conn->requested_resource+3, - failed, NULL, 0, 0); + failed, NULL, 0); if (smartlist_len(failed)) { dir_networkstatus_download_failed(failed, status_code); SMARTLIST_FOREACH(failed, char *, cp, tor_free(cp)); @@ -623,6 +677,24 @@ connection_dir_download_networkstatus_failed(dir_connection_t *conn, } } +/** Helper: Attempt to fetch directly the descriptors of each bridge + * listed in <b>failed</b>. + */ +static void +connection_dir_retry_bridges(smartlist_t *descs) +{ + char digest[DIGEST_LEN]; + SMARTLIST_FOREACH(descs, const char *, cp, + { + if (base16_decode(digest, DIGEST_LEN, cp, strlen(cp))<0) { + log_warn(LD_BUG, "Malformed fingerprint in list: %s", + escaped(cp)); + continue; + } + retry_bridge_descriptor_fetch_directly(digest); + }); +} + /** Called when an attempt to download one or more router descriptors * or extra-info documents on connection <b>conn</b> failed. */ @@ -640,6 +712,33 @@ connection_dir_download_routerdesc_failed(dir_connection_t *conn) (void) conn; } +/** Called when an attempt to download a bridge's routerdesc from + * one of the authorities failed due to a network error. If + * possible attempt to download descriptors from the bridge directly. + */ +static void +connection_dir_bridge_routerdesc_failed(dir_connection_t *conn) +{ + smartlist_t *which = NULL; + + /* Requests for bridge descriptors are in the form 'fp/', so ignore + anything else. */ + if (!conn->requested_resource || strcmpstart(conn->requested_resource,"fp/")) + return; + + which = smartlist_create(); + dir_split_resource_into_fingerprints(conn->requested_resource + + strlen("fp/"), + which, NULL, 0); + + tor_assert(conn->_base.purpose != DIR_PURPOSE_FETCH_EXTRAINFO); + if (smartlist_len(which)) { + connection_dir_retry_bridges(which); + SMARTLIST_FOREACH(which, char *, cp, tor_free(cp)); + } + smartlist_free(which); +} + /** Called when an attempt to fetch a certificate fails. */ static void connection_dir_download_cert_failed(dir_connection_t *conn, int status) @@ -651,7 +750,7 @@ connection_dir_download_cert_failed(dir_connection_t *conn, int status) return; failed = smartlist_create(); dir_split_resource_into_fingerprints(conn->requested_resource+3, - failed, NULL, 1, 0); + failed, NULL, DSR_HEX); SMARTLIST_FOREACH(failed, char *, cp, { authority_cert_dl_failed(cp, status); @@ -667,7 +766,7 @@ connection_dir_download_cert_failed(dir_connection_t *conn, int status) * 1) If or_port is 0, or it's a direct conn and or_port is firewalled * or we're a dir mirror, no. * 2) If we prefer to avoid begindir conns, and we're not fetching or - * publishing a bridge relay descriptor, no. + * publishing a bridge relay descriptor, no. * 3) Else yes. */ static int @@ -746,6 +845,15 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, log_debug(LD_DIR, "Initiating %s", dir_conn_purpose_to_string(dir_purpose)); + /* ensure that we don't make direct connections when a SOCKS server is + * configured. */ + if (!anonymized_connection && !use_begindir && !options->HTTPProxy && + (options->Socks4Proxy || options->Socks5Proxy)) { + log_warn(LD_DIR, "Cannot connect to a directory server through a " + "SOCKS proxy!"); + return; + } + conn = dir_connection_new(AF_INET); /* set up conn so it's got all the data we need to remember */ @@ -770,9 +878,9 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, if (!anonymized_connection && !use_begindir) { /* then we want to connect to dirport directly */ - if (options->HttpProxy) { - tor_addr_from_ipv4h(&addr, options->HttpProxyAddr); - dir_port = options->HttpProxyPort; + if (options->HTTPProxy) { + tor_addr_copy(&addr, &options->HTTPProxyAddr); + dir_port = options->HTTPProxyPort; } switch (connection_connect(TO_CONN(conn), conn->_base.address, &addr, @@ -793,7 +901,7 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, payload, payload_len, supports_conditional_consensus, if_modified_since); - connection_watch_events(TO_CONN(conn), EV_READ | EV_WRITE); + connection_watch_events(TO_CONN(conn), READ_EVENT | WRITE_EVENT); /* writable indicates finish, readable indicates broken link, error indicates broken link in windowsland. */ } @@ -832,7 +940,7 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, payload, payload_len, supports_conditional_consensus, if_modified_since); - connection_watch_events(TO_CONN(conn), EV_READ | EV_WRITE); + connection_watch_events(TO_CONN(conn), READ_EVENT|WRITE_EVENT); connection_start_reading(TO_CONN(linked_conn)); } } @@ -880,7 +988,7 @@ directory_get_consensus_url(int supports_conditional_consensus) if (supports_conditional_consensus) { char *authority_id_list; - smartlist_t *authority_digets = smartlist_create(); + smartlist_t *authority_digests = smartlist_create(); SMARTLIST_FOREACH(router_get_trusted_dir_servers(), trusted_dir_server_t *, ds, @@ -892,10 +1000,10 @@ directory_get_consensus_url(int supports_conditional_consensus) hex = tor_malloc(2*CONDITIONAL_CONSENSUS_FPR_LEN+1); base16_encode(hex, 2*CONDITIONAL_CONSENSUS_FPR_LEN+1, ds->v3_identity_digest, CONDITIONAL_CONSENSUS_FPR_LEN); - smartlist_add(authority_digets, hex); + smartlist_add(authority_digests, hex); }); - smartlist_sort(authority_digets, _compare_strs); - authority_id_list = smartlist_join_strings(authority_digets, + smartlist_sort(authority_digests, _compare_strs); + authority_id_list = smartlist_join_strings(authority_digests, "+", 0, NULL); len = strlen(authority_id_list)+64; @@ -903,8 +1011,8 @@ directory_get_consensus_url(int supports_conditional_consensus) tor_snprintf(url, len, "/tor/status-vote/current/consensus/%s.z", authority_id_list); - SMARTLIST_FOREACH(authority_digets, char *, cp, tor_free(cp)); - smartlist_free(authority_digets); + SMARTLIST_FOREACH(authority_digests, char *, cp, tor_free(cp)); + smartlist_free(authority_digests); tor_free(authority_id_list); } else { url = tor_strdup("/tor/status-vote/current/consensus.z"); @@ -913,7 +1021,7 @@ directory_get_consensus_url(int supports_conditional_consensus) } /** Queue an appropriate HTTP command on conn-\>outbuf. The other args - * are as in directory_initiate_command. + * are as in directory_initiate_command(). */ static void directory_send_command(dir_connection_t *conn, @@ -956,9 +1064,9 @@ directory_send_command(dir_connection_t *conn, } /* come up with some proxy lines, if we're using one. */ - if (direct && get_options()->HttpProxy) { + if (direct && get_options()->HTTPProxy) { char *base64_authenticator=NULL; - const char *authenticator = get_options()->HttpProxyAuthenticator; + const char *authenticator = get_options()->HTTPProxyAuthenticator; tor_snprintf(proxystring, sizeof(proxystring),"http://%s", hoststring); if (authenticator) { @@ -1049,31 +1157,10 @@ directory_send_command(dir_connection_t *conn, httpcommand = "POST"; url = tor_strdup("/tor/post/consensus-signature"); break; - case DIR_PURPOSE_FETCH_RENDDESC: - tor_assert(resource); - tor_assert(!payload); - - /* this must be true or we wouldn't be doing the lookup */ - tor_assert(strlen(resource) <= REND_SERVICE_ID_LEN_BASE32); - /* This breaks the function abstraction. */ - conn->rend_data = tor_malloc_zero(sizeof(rend_data_t)); - strlcpy(conn->rend_data->onion_address, resource, - sizeof(conn->rend_data->onion_address)); - conn->rend_data->rend_desc_version = 0; - - httpcommand = "GET"; - /* Request the most recent versioned descriptor. */ - // (XXXX We were going to switch this to fetch rendezvous1 descriptors, - // but that never got testing, and it wasn't a good design.) - len = strlen(resource)+32; - url = tor_malloc(len); - tor_snprintf(url, len, "/tor/rendezvous/%s", resource); - break; case DIR_PURPOSE_FETCH_RENDDESC_V2: tor_assert(resource); tor_assert(strlen(resource) <= REND_DESC_ID_V2_LEN_BASE32); tor_assert(!payload); - conn->rend_data->rend_desc_version = 2; httpcommand = "GET"; len = strlen(resource) + 32; url = tor_malloc(len); @@ -1162,7 +1249,7 @@ parse_http_url(const char *headers, char **url) if (s-tmp >= 3 && !strcmpstart(tmp,"://")) { tmp = strchr(tmp+3, '/'); if (tmp && tmp < s) { - log_debug(LD_DIR,"Skipping over 'http[s]://hostname' string"); + log_debug(LD_DIR,"Skipping over 'http[s]://hostname/' string"); start = tmp; } } @@ -1479,7 +1566,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) } (void) skewed; /* skewed isn't used yet. */ - if (status_code == 503 && body_len < 16) { + if (status_code == 503) { routerstatus_t *rs; trusted_dir_server_t *ds; log_info(LD_DIR,"Received http status code %d (%s) from server " @@ -1493,12 +1580,6 @@ connection_dir_client_reached_eof(dir_connection_t *conn) tor_free(body); tor_free(headers); tor_free(reason); return -1; - } else if (status_code == 503) { - /* XXXX022 Remove this once every server with bug 539 is obsolete. */ - log_info(LD_DIR, "Server at '%s:%d' sent us a 503 response, but included " - "a body anyway. We'll pretend it gave us a 200.", - conn->_base.address, conn->_base.port); - status_code = 200; } plausible = body_is_plausible(body, body_len, conn->_base.purpose); @@ -1565,7 +1646,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) v2_networkstatus_source_t source; char *cp; log_info(LD_DIR,"Received networkstatus objects (size %d) from server " - "'%s:%d'",(int) body_len, conn->_base.address, conn->_base.port); + "'%s:%d'", (int)body_len, conn->_base.address, conn->_base.port); if (status_code != 200) { log_warn(LD_DIR, "Received http status code %d (%s) from server " @@ -1581,7 +1662,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) source = NS_FROM_DIR_BY_FP; which = smartlist_create(); dir_split_resource_into_fingerprints(conn->requested_resource+3, - which, NULL, 0, 0); + which, NULL, 0); } else if (conn->requested_resource && !strcmpstart(conn->requested_resource, "all")) { source = NS_FROM_DIR_ALL; @@ -1639,8 +1720,8 @@ connection_dir_client_reached_eof(dir_connection_t *conn) return -1; } log_info(LD_DIR,"Received consensus directory (size %d) from server " - "'%s:%d'",(int) body_len, conn->_base.address, conn->_base.port); - if ((r=networkstatus_set_current_consensus(body, 0))<0) { + "'%s:%d'", (int)body_len, conn->_base.address, conn->_base.port); + if ((r=networkstatus_set_current_consensus(body, "ns", 0))<0) { log_fn(r<-1?LOG_WARN:LOG_INFO, LD_DIR, "Unable to load consensus directory downloaded from " "server '%s:%d'. I'll try again soon.", @@ -1667,9 +1748,11 @@ connection_dir_client_reached_eof(dir_connection_t *conn) return -1; } log_info(LD_DIR,"Received authority certificates (size %d) from server " - "'%s:%d'",(int) body_len, conn->_base.address, conn->_base.port); + "'%s:%d'", (int)body_len, conn->_base.address, conn->_base.port); if (trusted_dirs_load_certs_from_string(body, 0, 1)<0) { log_warn(LD_DIR, "Unable to parse fetched certificates"); + /* if we fetched more than one and only some failed, the successful + * ones got flushed to disk so it's safe to call this on them */ connection_dir_download_cert_failed(conn, status_code); } else { directory_info_has_arrived(now, 0); @@ -1680,7 +1763,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) const char *msg; int st; log_info(LD_DIR,"Got votes (size %d) from server %s:%d", - (int) body_len, conn->_base.address, conn->_base.port); + (int)body_len, conn->_base.address, conn->_base.port); if (status_code != 200) { log_warn(LD_DIR, "Received http status code %d (%s) from server " @@ -1700,11 +1783,11 @@ connection_dir_client_reached_eof(dir_connection_t *conn) if (conn->_base.purpose == DIR_PURPOSE_FETCH_DETACHED_SIGNATURES) { const char *msg = NULL; log_info(LD_DIR,"Got detached signatures (size %d) from server %s:%d", - (int) body_len, conn->_base.address, conn->_base.port); + (int)body_len, conn->_base.address, conn->_base.port); if (status_code != 200) { log_warn(LD_DIR, - "Received http status code %d (%s) from server " - "'%s:%d' while fetching \"/tor/status-vote/consensus-signatures.z\".", + "Received http status code %d (%s) from server '%s:%d' while fetching " + "\"/tor/status-vote/next/consensus-signatures.z\".", status_code, escaped(reason), conn->_base.address, conn->_base.port); tor_free(body); tor_free(headers); tor_free(reason); @@ -1732,7 +1815,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) which = smartlist_create(); dir_split_resource_into_fingerprints(conn->requested_resource + (descriptor_digests ? 2 : 3), - which, NULL, 0, 0); + which, NULL, 0); n_asked_for = smartlist_len(which); } if (status_code != 200) { @@ -1814,7 +1897,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) ds->nickname); /* XXXX use this information; be sure to upload next one * sooner. -NM */ - /* XXXX021 On further thought, the task above implies that we're + /* XXXX023 On further thought, the task above implies that we're * basing our regenerate-descriptor time on when we uploaded the * last descriptor, not on the published time of the last * descriptor. If those are different, that's a bad thing to @@ -1920,7 +2003,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) /* Success, or at least there's a v2 descriptor already * present. Notify pending connections about this. */ conn->_base.purpose = DIR_PURPOSE_HAS_FETCHED_RENDDESC; - rend_client_desc_trynow(conn->rend_data->onion_address, -1); + rend_client_desc_trynow(conn->rend_data->onion_address); } break; case 404: @@ -1967,7 +2050,7 @@ connection_dir_client_reached_eof(dir_connection_t *conn) log_info(LD_REND, "Successfully fetched v2 rendezvous " "descriptor."); conn->_base.purpose = DIR_PURPOSE_HAS_FETCHED_RENDDESC; - rend_client_desc_trynow(conn->rend_data->onion_address, -1); + rend_client_desc_trynow(conn->rend_data->onion_address); break; } break; @@ -2010,12 +2093,6 @@ connection_dir_client_reached_eof(dir_connection_t *conn) "'%s:%d'. Malformed rendezvous descriptor?", escaped(reason), conn->_base.address, conn->_base.port); break; - case 503: - log_info(LD_REND,"http status 503 (%s) response from dirserver " - "'%s:%d'. Node is (currently) not acting as v2 hidden " - "service directory.", - escaped(reason), conn->_base.address, conn->_base.port); - break; default: log_warn(LD_REND,"http status %d (%s) response unexpected (server " "'%s:%d').", @@ -2322,7 +2399,7 @@ directory_dump_request_log(void) } #endif -/** Decide whether a client would accept the consensus we have +/** Decide whether a client would accept the consensus we have. * * Clients can say they only want a consensus if it's signed by more * than half the authorities in a list. They pass this list in @@ -2343,31 +2420,32 @@ client_likes_consensus(networkstatus_t *v, const char *want_url) int need_at_least; int have = 0; - dir_split_resource_into_fingerprints(want_url, want_authorities, NULL, 0, 0); + dir_split_resource_into_fingerprints(want_url, want_authorities, NULL, 0); need_at_least = smartlist_len(want_authorities)/2+1; - SMARTLIST_FOREACH(want_authorities, const char *, d, { + SMARTLIST_FOREACH_BEGIN(want_authorities, const char *, d) { char want_digest[DIGEST_LEN]; size_t want_len = strlen(d)/2; if (want_len > DIGEST_LEN) want_len = DIGEST_LEN; if (base16_decode(want_digest, DIGEST_LEN, d, want_len*2) < 0) { - log_warn(LD_DIR,"Failed to decode requested authority digest %s.", d); + log_fn(LOG_PROTOCOL_WARN, LD_DIR, + "Failed to decode requested authority digest %s.", d); continue; }; - SMARTLIST_FOREACH(v->voters, networkstatus_voter_info_t *, vi, { - if (vi->signature && + SMARTLIST_FOREACH_BEGIN(v->voters, networkstatus_voter_info_t *, vi) { + if (smartlist_len(vi->sigs) && !memcmp(vi->identity_digest, want_digest, want_len)) { have++; break; }; - }); + } SMARTLIST_FOREACH_END(vi); /* early exit, if we already have enough */ if (have >= need_at_least) break; - }); + } SMARTLIST_FOREACH_END(d); SMARTLIST_FOREACH(want_authorities, char *, d, tor_free(d)); smartlist_free(want_authorities); @@ -2514,9 +2592,12 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, /* v2 or v3 network status fetch. */ smartlist_t *dir_fps = smartlist_create(); int is_v3 = !strcmpstart(url, "/tor/status-vote"); + geoip_client_action_t act = + is_v3 ? GEOIP_CLIENT_NETWORKSTATUS : GEOIP_CLIENT_NETWORKSTATUS_V2; const char *request_type = NULL; const char *key = url + strlen("/tor/status/"); long lifetime = NETWORKSTATUS_CACHE_LIFETIME; + if (!is_v3) { dirserv_get_networkstatus_v2_fingerprints(dir_fps, key); if (!strcmpstart(key, "fp/")) @@ -2531,18 +2612,44 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } else { networkstatus_t *v = networkstatus_get_latest_consensus(); time_t now = time(NULL); + const char *want_fps = NULL; + char *flavor = NULL; #define CONSENSUS_URL_PREFIX "/tor/status-vote/current/consensus/" - if (v && - !strcmpstart(url, CONSENSUS_URL_PREFIX) && - !client_likes_consensus(v, url + strlen(CONSENSUS_URL_PREFIX))) { + #define CONSENSUS_FLAVORED_PREFIX "/tor/status-vote/current/consensus-" + /* figure out the flavor if any, and who we wanted to sign the thing */ + if (!strcmpstart(url, CONSENSUS_FLAVORED_PREFIX)) { + const char *f, *cp; + f = url + strlen(CONSENSUS_FLAVORED_PREFIX); + cp = strchr(f, '/'); + if (cp) { + want_fps = cp+1; + flavor = tor_strndup(f, cp-f); + } else { + flavor = tor_strdup(f); + } + } else { + if (!strcmpstart(url, CONSENSUS_URL_PREFIX)) + want_fps = url+strlen(CONSENSUS_URL_PREFIX); + } + + /* XXXX MICRODESC NM NM should check document of correct flavor */ + if (v && want_fps && + !client_likes_consensus(v, want_fps)) { write_http_status_line(conn, 404, "Consensus not signed by sufficient " "number of requested authorities"); smartlist_free(dir_fps); + geoip_note_ns_response(act, GEOIP_REJECT_NOT_ENOUGH_SIGS); + tor_free(flavor); goto done; } - smartlist_add(dir_fps, tor_memdup("\0\0\0\0\0\0\0\0\0\0" - "\0\0\0\0\0\0\0\0\0\0", 20)); + { + char *fp = tor_malloc_zero(DIGEST_LEN); + if (flavor) + strlcpy(fp, flavor, DIGEST_LEN); + tor_free(flavor); + smartlist_add(dir_fps, fp); + } request_type = compressed?"v3.z":"v3"; lifetime = (v && v->fresh_until > now) ? v->fresh_until - now : 0; } @@ -2550,6 +2657,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, if (!smartlist_len(dir_fps)) { /* we failed to create/cache cp */ write_http_status_line(conn, 503, "Network status object unavailable"); smartlist_free(dir_fps); + geoip_note_ns_response(act, GEOIP_REJECT_UNAVAILABLE); goto done; } @@ -2557,11 +2665,13 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, write_http_status_line(conn, 404, "Not found"); SMARTLIST_FOREACH(dir_fps, char *, cp, tor_free(cp)); smartlist_free(dir_fps); + geoip_note_ns_response(act, GEOIP_REJECT_NOT_FOUND); goto done; } else if (!smartlist_len(dir_fps)) { write_http_status_line(conn, 304, "Not modified"); SMARTLIST_FOREACH(dir_fps, char *, cp, tor_free(cp)); smartlist_free(dir_fps); + geoip_note_ns_response(act, GEOIP_REJECT_NOT_MODIFIED); goto done; } @@ -2573,18 +2683,25 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, write_http_status_line(conn, 503, "Directory busy, try again later"); SMARTLIST_FOREACH(dir_fps, char *, fp, tor_free(fp)); smartlist_free(dir_fps); + geoip_note_ns_response(act, GEOIP_REJECT_BUSY); goto done; } -#ifdef ENABLE_GEOIP_STATS { - geoip_client_action_t act = - is_v3 ? GEOIP_CLIENT_NETWORKSTATUS : GEOIP_CLIENT_NETWORKSTATUS_V2; struct in_addr in; - if (tor_inet_aton((TO_CONN(conn))->address, &in)) + if (tor_inet_aton((TO_CONN(conn))->address, &in)) { geoip_note_client_seen(act, ntohl(in.s_addr), time(NULL)); + geoip_note_ns_response(act, GEOIP_SUCCESS); + /* Note that a request for a network status has started, so that we + * can measure the download time later on. */ + if (TO_CONN(conn)->dirreq_id) + geoip_start_dirreq(TO_CONN(conn)->dirreq_id, dlen, act, + DIRREQ_TUNNELED); + else + geoip_start_dirreq(TO_CONN(conn)->global_identifier, dlen, act, + DIRREQ_DIRECT); + } } -#endif // note_request(request_type,dlen); (void) request_type; @@ -2610,7 +2727,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, ssize_t estimated_len = 0; smartlist_t *items = smartlist_create(); smartlist_t *dir_items = smartlist_create(); - int lifetime = 60; /* XXXX022 should actually use vote intervals. */ + int lifetime = 60; /* XXXX023 should actually use vote intervals. */ url += strlen("/tor/status-vote/"); current = !strcmpstart(url, "current/"); url = strchr(url, '/'); @@ -2620,7 +2737,7 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, const char *item; tor_assert(!current); /* we handle current consensus specially above, * since it wants to be spooled. */ - if ((item = dirvote_get_pending_consensus())) + if ((item = dirvote_get_pending_consensus(FLAV_NS))) smartlist_add(items, (char*)item); } else if (!current && !strcmp(url, "consensus-signatures")) { /* XXXX the spec says that we should implement @@ -2646,7 +2763,8 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, flags = DGV_BY_ID | (current ? DGV_INCLUDE_PREVIOUS : DGV_INCLUDE_PENDING); } - dir_split_resource_into_fingerprints(url, fps, NULL, 1, 1); + dir_split_resource_into_fingerprints(url, fps, NULL, + DSR_HEX|DSR_SORT_UNIQ); SMARTLIST_FOREACH(fps, char *, fp, { if ((d = dirvote_get_vote(fp, flags))) smartlist_add(dir_items, (cached_dir_t*)d); @@ -2699,6 +2817,41 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, goto done; } + if (!strcmpstart(url, "/tor/micro/d/")) { + smartlist_t *fps = smartlist_create(); + + dir_split_resource_into_fingerprints(url+strlen("/tor/micro/d/"), + fps, NULL, + DSR_DIGEST256|DSR_BASE64|DSR_SORT_UNIQ); + + if (!dirserv_have_any_microdesc(fps)) { + write_http_status_line(conn, 404, "Not found"); + SMARTLIST_FOREACH(fps, char *, fp, tor_free(fp)); + smartlist_free(fps); + goto done; + } + dlen = dirserv_estimate_microdesc_size(fps, compressed); + if (global_write_bucket_low(TO_CONN(conn), dlen, 2)) { + log_info(LD_DIRSERV, + "Client asked for server descriptors, but we've been " + "writing too many bytes lately. Sending 503 Dir busy."); + write_http_status_line(conn, 503, "Directory busy, try again later"); + SMARTLIST_FOREACH(fps, char *, fp, tor_free(fp)); + smartlist_free(fps); + goto done; + } + + write_http_response_header(conn, -1, compressed, MICRODESC_CACHE_LIFETIME); + conn->dir_spool_src = DIR_SPOOL_MICRODESC; + conn->fingerprint_stack = fps; + + if (compressed) + conn->zlib_state = tor_zlib_new(1, ZLIB_METHOD); + + connection_dirserv_flushed_some(conn); + goto done; + } + if (!strcmpstart(url,"/tor/server/") || (!options->BridgeAuthoritativeDir && !options->BridgeRelay && !strcmpstart(url,"/tor/extra/"))) { @@ -2780,7 +2933,8 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } else if (!strcmpstart(url, "/tor/keys/fp/")) { smartlist_t *fps = smartlist_create(); dir_split_resource_into_fingerprints(url+strlen("/tor/keys/fp/"), - fps, NULL, 1, 1); + fps, NULL, + DSR_HEX|DSR_SORT_UNIQ); SMARTLIST_FOREACH(fps, char *, d, { authority_cert_t *c = authority_cert_get_newest_by_id(d); if (c) smartlist_add(certs, c); @@ -2790,7 +2944,8 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, } else if (!strcmpstart(url, "/tor/keys/sk/")) { smartlist_t *fps = smartlist_create(); dir_split_resource_into_fingerprints(url+strlen("/tor/keys/sk/"), - fps, NULL, 1, 1); + fps, NULL, + DSR_HEX|DSR_SORT_UNIQ); SMARTLIST_FOREACH(fps, char *, d, { authority_cert_t *c = authority_cert_get_by_sk_digest(d); if (c) smartlist_add(certs, c); @@ -2892,18 +3047,9 @@ directory_handle_command_get(dir_connection_t *conn, const char *headers, note_request("/tor/rendezvous?/", desc_len); /* need to send descp separately, because it may include NULs */ connection_write_to_buf(descp, desc_len, TO_CONN(conn)); - /* report successful fetch to statistic */ - if (options->HSAuthorityRecordStats) { - hs_usage_note_fetch_total(query, time(NULL)); - hs_usage_note_fetch_successful(query, time(NULL)); - } break; case 0: /* well-formed but not present */ write_http_status_line(conn, 404, "Not found"); - /* report (unsuccessful) fetch to statistic */ - if (options->HSAuthorityRecordStats) { - hs_usage_note_fetch_total(query, time(NULL)); - } break; case -1: /* not well-formed */ write_http_status_line(conn, 400, "Bad request"); @@ -3134,6 +3280,8 @@ directory_handle_command_post(dir_connection_t *conn, const char *headers, write_http_status_line(conn, status, "Vote stored"); } else { tor_assert(msg); + log_warn(LD_DIRSERV, "Rejected vote from %s (\"%s\").", + conn->_base.address, msg); write_http_status_line(conn, status, msg); } goto done; @@ -3180,8 +3328,8 @@ directory_handle_command(dir_connection_t *conn) &body, &body_len, MAX_DIR_UL_SIZE, 0)) { case -1: /* overflow */ log_warn(LD_DIRSERV, - "Invalid input from address '%s'. Closing.", - conn->_base.address); + "Request too large from address '%s' to DirPort. Closing.", + safe_str(conn->_base.address)); return -1; case 0: log_debug(LD_DIRSERV,"command not all here yet."); @@ -3217,6 +3365,16 @@ connection_dir_finished_flushing(dir_connection_t *conn) tor_assert(conn); tor_assert(conn->_base.type == CONN_TYPE_DIR); + /* Note that we have finished writing the directory response. For direct + * connections this means we're done, for tunneled connections its only + * an intermediate step. */ + if (TO_CONN(conn)->dirreq_id) + geoip_change_dirreq_state(TO_CONN(conn)->dirreq_id, DIRREQ_TUNNELED, + DIRREQ_FLUSHING_DIR_CONN_FINISHED); + else + geoip_change_dirreq_state(TO_CONN(conn)->global_identifier, + DIRREQ_DIRECT, + DIRREQ_FLUSHING_DIR_CONN_FINISHED); switch (conn->_base.state) { case DIR_CONN_STATE_CLIENT_SENDING: log_debug(LD_DIR,"client finished sending command."); @@ -3399,6 +3557,14 @@ download_status_reset(download_status_t *dls) dls->next_attempt_at = time(NULL) + schedule[0]; } +/** Return the number of failures on <b>dls</b> since the last success (if + * any). */ +int +download_status_get_n_failures(const download_status_t *dls) +{ + return dls->n_download_failures; +} + /** Called when one or more routerdesc (or extrainfo, if <b>was_extrainfo</b>) * fetches have failed (with uppercase fingerprints listed in <b>failed</b>, * either as descriptor digests or as identity digests based on @@ -3414,16 +3580,8 @@ dir_routerdesc_download_failed(smartlist_t *failed, int status_code, int server = directory_fetches_from_authorities(get_options()); if (!was_descriptor_digests) { if (router_purpose == ROUTER_PURPOSE_BRIDGE) { - tor_assert(!was_extrainfo); /* not supported yet */ - SMARTLIST_FOREACH(failed, const char *, cp, - { - if (base16_decode(digest, DIGEST_LEN, cp, strlen(cp))<0) { - log_warn(LD_BUG, "Malformed fingerprint in list: %s", - escaped(cp)); - continue; - } - retry_bridge_descriptor_fetch_directly(digest); - }); + tor_assert(!was_extrainfo); + connection_dir_retry_bridges(failed); } return; /* FFFF should implement for other-than-router-purpose someday */ } @@ -3516,19 +3674,37 @@ dir_split_resource_into_fingerprint_pairs(const char *res, /** Given a directory <b>resource</b> request, containing zero * or more strings separated by plus signs, followed optionally by ".z", store * the strings, in order, into <b>fp_out</b>. If <b>compressed_out</b> is - * non-NULL, set it to 1 if the resource ends in ".z", else set it to 0. If - * decode_hex is true, then delete all elements that aren't hex digests, and - * decode the rest. If sort_uniq is true, then sort the list and remove - * all duplicates. + * non-NULL, set it to 1 if the resource ends in ".z", else set it to 0. + * + * If (flags & DSR_HEX), then delete all elements that aren't hex digests, and + * decode the rest. If (flags & DSR_BASE64), then use "-" rather than "+" as + * a separator, delete all the elements that aren't base64-encoded digests, + * and decode the rest. If (flags & DSR_DIGEST256), these digests should be + * 256 bits long; else they should be 160. + * + * If (flags & DSR_SORT_UNIQ), then sort the list and remove all duplicates. */ int dir_split_resource_into_fingerprints(const char *resource, smartlist_t *fp_out, int *compressed_out, - int decode_hex, int sort_uniq) + int flags) { + const int decode_hex = flags & DSR_HEX; + const int decode_base64 = flags & DSR_BASE64; + const int digests_are_256 = flags & DSR_DIGEST256; + const int sort_uniq = flags & DSR_SORT_UNIQ; + + const int digest_len = digests_are_256 ? DIGEST256_LEN : DIGEST_LEN; + const int hex_digest_len = digests_are_256 ? + HEX_DIGEST256_LEN : HEX_DIGEST_LEN; + const int base64_digest_len = digests_are_256 ? + BASE64_DIGEST256_LEN : BASE64_DIGEST_LEN; smartlist_t *fp_tmp = smartlist_create(); + + tor_assert(!(decode_hex && decode_base64)); tor_assert(fp_out); - smartlist_split_string(fp_tmp, resource, "+", 0, 0); + + smartlist_split_string(fp_tmp, resource, decode_base64?"-":"+", 0, 0); if (compressed_out) *compressed_out = 0; if (smartlist_len(fp_tmp)) { @@ -3540,22 +3716,25 @@ dir_split_resource_into_fingerprints(const char *resource, *compressed_out = 1; } } - if (decode_hex) { + if (decode_hex || decode_base64) { + const size_t encoded_len = decode_hex ? hex_digest_len : base64_digest_len; int i; char *cp, *d = NULL; for (i = 0; i < smartlist_len(fp_tmp); ++i) { cp = smartlist_get(fp_tmp, i); - if (strlen(cp) != HEX_DIGEST_LEN) { + if (strlen(cp) != encoded_len) { log_info(LD_DIR, "Skipping digest %s with non-standard length.", escaped(cp)); smartlist_del_keeporder(fp_tmp, i--); goto again; } - d = tor_malloc_zero(DIGEST_LEN); - if (base16_decode(d, DIGEST_LEN, cp, HEX_DIGEST_LEN)<0) { - log_info(LD_DIR, "Skipping non-decodable digest %s", escaped(cp)); - smartlist_del_keeporder(fp_tmp, i--); - goto again; + d = tor_malloc_zero(digest_len); + if (decode_hex ? + (base16_decode(d, digest_len, cp, hex_digest_len)<0) : + (base64_decode(d, digest_len, cp, base64_digest_len)<0)) { + log_info(LD_DIR, "Skipping non-decodable digest %s", escaped(cp)); + smartlist_del_keeporder(fp_tmp, i--); + goto again; } smartlist_set(fp_tmp, i, d); d = NULL; @@ -3565,26 +3744,18 @@ dir_split_resource_into_fingerprints(const char *resource, } } if (sort_uniq) { - smartlist_t *fp_tmp2 = smartlist_create(); - int i; - if (decode_hex) - smartlist_sort_digests(fp_tmp); - else + if (decode_hex || decode_base64) { + if (digests_are_256) { + smartlist_sort_digests256(fp_tmp); + smartlist_uniq_digests256(fp_tmp); + } else { + smartlist_sort_digests(fp_tmp); + smartlist_uniq_digests(fp_tmp); + } + } else { smartlist_sort_strings(fp_tmp); - if (smartlist_len(fp_tmp)) - smartlist_add(fp_tmp2, smartlist_get(fp_tmp, 0)); - for (i = 1; i < smartlist_len(fp_tmp); ++i) { - char *cp = smartlist_get(fp_tmp, i); - char *last = smartlist_get(fp_tmp2, smartlist_len(fp_tmp2)-1); - - if ((decode_hex && memcmp(cp, last, DIGEST_LEN)) - || (!decode_hex && strcasecmp(cp, last))) - smartlist_add(fp_tmp2, cp); - else - tor_free(cp); + smartlist_uniq_strings(fp_tmp); } - smartlist_free(fp_tmp); - fp_tmp = fp_tmp2; } smartlist_add_all(fp_out, fp_tmp); smartlist_free(fp_tmp); diff --git a/src/or/directory.h b/src/or/directory.h new file mode 100644 index 000000000..94dfb1764 --- /dev/null +++ b/src/or/directory.h @@ -0,0 +1,110 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file directory.h + * \brief Header file for directory.c. + **/ + +#ifndef _TOR_DIRECTORY_H +#define _TOR_DIRECTORY_H + +int directories_have_accepted_server_descriptor(void); +char *authority_type_to_string(authority_type_t auth); +void directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, + authority_type_t type, const char *payload, + size_t payload_len, size_t extrainfo_len); +void directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, + const char *resource, + int pds_flags); +void directory_get_from_all_authorities(uint8_t dir_purpose, + uint8_t router_purpose, + const char *resource); +void directory_initiate_command_routerstatus(routerstatus_t *status, + uint8_t dir_purpose, + uint8_t router_purpose, + int anonymized_connection, + const char *resource, + const char *payload, + size_t payload_len, + time_t if_modified_since); +void directory_initiate_command_routerstatus_rend(routerstatus_t *status, + uint8_t dir_purpose, + uint8_t router_purpose, + int anonymized_connection, + const char *resource, + const char *payload, + size_t payload_len, + time_t if_modified_since, + const rend_data_t *rend_query); + +int parse_http_response(const char *headers, int *code, time_t *date, + compress_method_t *compression, char **response); + +int connection_dir_is_encrypted(dir_connection_t *conn); +int connection_dir_reached_eof(dir_connection_t *conn); +int connection_dir_process_inbuf(dir_connection_t *conn); +int connection_dir_finished_flushing(dir_connection_t *conn); +int connection_dir_finished_connecting(dir_connection_t *conn); +void connection_dir_request_failed(dir_connection_t *conn); +void directory_initiate_command(const char *address, const tor_addr_t *addr, + uint16_t or_port, uint16_t dir_port, + int supports_conditional_consensus, + int supports_begindir, const char *digest, + uint8_t dir_purpose, uint8_t router_purpose, + int anonymized_connection, + const char *resource, + const char *payload, size_t payload_len, + time_t if_modified_since); + +#define DSR_HEX (1<<0) +#define DSR_BASE64 (1<<1) +#define DSR_DIGEST256 (1<<2) +#define DSR_SORT_UNIQ (1<<3) +int dir_split_resource_into_fingerprints(const char *resource, + smartlist_t *fp_out, int *compressed_out, + int flags); + +int dir_split_resource_into_fingerprint_pairs(const char *res, + smartlist_t *pairs_out); +char *directory_dump_request_log(void); +void note_request(const char *key, size_t bytes); +int router_supports_extrainfo(const char *identity_digest, int is_authority); + +time_t download_status_increment_failure(download_status_t *dls, + int status_code, const char *item, + int server, time_t now); +/** Increment the failure count of the download_status_t <b>dls</b>, with + * the optional status code <b>sc</b>. */ +#define download_status_failed(dls, sc) \ + download_status_increment_failure((dls), (sc), NULL, \ + get_options()->DirPort, time(NULL)) + +void download_status_reset(download_status_t *dls); +static int download_status_is_ready(download_status_t *dls, time_t now, + int max_failures); +/** Return true iff, as of <b>now</b>, the resource tracked by <b>dls</b> is + * ready to get its download reattempted. */ +static INLINE int +download_status_is_ready(download_status_t *dls, time_t now, + int max_failures) +{ + return (dls->n_download_failures <= max_failures + && dls->next_attempt_at <= now); +} + +static void download_status_mark_impossible(download_status_t *dl); +/** Mark <b>dl</b> as never downloadable. */ +static INLINE void +download_status_mark_impossible(download_status_t *dl) +{ + dl->n_download_failures = IMPOSSIBLE_TO_DOWNLOAD; +} + +int download_status_get_n_failures(const download_status_t *dls); + +#endif + diff --git a/src/or/dirserv.c b/src/or/dirserv.c index 7db6c19a3..860ac1f70 100644 --- a/src/or/dirserv.c +++ b/src/or/dirserv.c @@ -5,6 +5,22 @@ #define DIRSERV_PRIVATE #include "or.h" +#include "buffers.h" +#include "config.h" +#include "connection.h" +#include "connection_or.h" +#include "control.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "hibernate.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "policies.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" /** * \file dirserv.c @@ -27,6 +43,8 @@ extern time_t time_of_process_start; /* from main.c */ +extern long stats_n_seconds_working; /* from main.c */ + /** Do we need to regenerate the v1 directory when someone asks for it? */ static time_t the_directory_is_dirty = 1; /** Do we need to regenerate the v1 runningrouters document when somebody @@ -41,7 +59,7 @@ static time_t the_v2_networkstatus_is_dirty = 1; static cached_dir_t *the_directory = NULL; /** For authoritative directories: the current (v1) network status. */ -static cached_dir_t the_runningrouters = { NULL, NULL, 0, 0, 0, -1 }; +static cached_dir_t the_runningrouters; static void directory_remove_invalid(void); static cached_dir_t *dirserv_regenerate_directory(void); @@ -63,13 +81,16 @@ static signed_descriptor_t *get_signed_descriptor_by_fp(const char *fp, time_t publish_cutoff); static int dirserv_add_extrainfo(extrainfo_t *ei, const char **msg); +/************** Measured Bandwidth parsing code ******/ +#define MAX_MEASUREMENT_AGE (3*24*60*60) /* 3 days */ + /************** Fingerprint handling code ************/ #define FP_NAMED 1 /**< Listed in fingerprint file. */ #define FP_INVALID 2 /**< Believed invalid. */ #define FP_REJECT 4 /**< We will not publish this router. */ #define FP_BADDIR 8 /**< We'll tell clients to avoid using this as a dir. */ -#define FP_BADEXIT 16 /**< We'll tell clients not to use this as an exit. */ +#define FP_BADEXIT 16 /**< We'll tell clients not to use this as an exit. */ #define FP_UNNAMED 32 /**< Another router has this name in fingerprint file. */ /** Encapsulate a nickname and an FP_* status; target of status_by_digest @@ -99,7 +120,7 @@ authdir_config_new(void) return list; } -/** Add the fingerprint <b>fp</b> for the nickname <b>nickname</b> to +/** Add the fingerprint <b>fp</b> for <b>nickname</b> to * the smartlist of fingerprint_entry_t's <b>list</b>. Return 0 if it's * new, or 1 if we replaced the old value. */ @@ -181,8 +202,7 @@ dirserv_add_own_fingerprint(const char *nickname, crypto_pk_env_t *pk) * file. The file format is line-based, with each non-blank holding one * nickname, some space, and a fingerprint for that nickname. On success, * replace the current fingerprint list with the new list and return 0. On - * failure, leave the current fingerprint list untouched, and - * return -1. */ + * failure, leave the current fingerprint list untouched, and return -1. */ int dirserv_load_fingerprint_file(void) { @@ -368,13 +388,19 @@ dirserv_get_status_impl(const char *id_digest, const char *nickname, strmap_size(fingerprint_list->fp_by_name), digestmap_size(fingerprint_list->status_by_digest)); - /* 0.1.1.17-rc was the first version that claimed to be stable, doesn't - * crash and drop circuits all the time, and is even vaguely compatible with - * the current network */ - if (platform && !tor_version_as_new_as(platform,"0.1.1.17-rc")) { + /* Tor 0.2.0.26-rc is the oldest version that currently caches the right + * directory information. Once more of them die off, we should raise this + * minimum. */ + if (platform && !tor_version_as_new_as(platform,"0.2.0.26-rc")) { if (msg) *msg = "Tor version is far too old to work."; return FP_REJECT; + } else if (platform && tor_version_as_new_as(platform,"0.2.1.3-alpha") + && !tor_version_as_new_as(platform, "0.2.1.19")) { + /* These versions mishandled RELAY_EARLY cells on rend circuits. */ + if (msg) + *msg = "Tor version is too buggy to work."; + return FP_REJECT; } result = dirserv_get_name_status(id_digest, nickname); @@ -520,7 +546,7 @@ authdir_wants_to_reject_router(routerinfo_t *ri, const char **msg, /* Okay. Now check whether the fingerprint is recognized. */ uint32_t status = dirserv_router_get_status(ri, msg); time_t now; - int severity = complain ? LOG_NOTICE : LOG_INFO; + int severity = (complain && ri->contact_info) ? LOG_NOTICE : LOG_INFO; tor_assert(msg); if (status & FP_REJECT) return -1; /* msg is already set. */ @@ -712,6 +738,10 @@ dirserv_add_descriptor(routerinfo_t *ri, const char **msg, const char *source) desc = tor_strndup(ri->cache_info.signed_descriptor_body, desclen); nickname = tor_strdup(ri->nickname); + /* Tell if we're about to need to launch a test if we add this. */ + ri->needs_retest_if_added = + dirserv_should_launch_reachability_test(ri, ri_old); + r = router_add_to_routerlist(ri, msg, 0, 0); if (!WRA_WAS_ADDED(r)) { /* unless the routerinfo was fine, just out-of-date */ @@ -726,7 +756,7 @@ dirserv_add_descriptor(routerinfo_t *ri, const char **msg, const char *source) changed = smartlist_create(); smartlist_add(changed, ri); - control_event_descriptors_changed(changed); + routerlist_descriptors_added(changed, 0); smartlist_free(changed); if (!*msg) { *msg = ri->is_valid ? "Descriptor for valid server accepted" : @@ -835,46 +865,6 @@ directory_remove_invalid(void) routerlist_assert_ok(rl); } -/** Write a list of unregistered descriptors into a newly allocated - * string and return it. Used by dirserv operators to keep track of - * fast nodes that haven't registered. - */ -int -getinfo_helper_dirserv_unregistered(control_connection_t *control_conn, - const char *question, char **answer_out) -{ - smartlist_t *answerlist; - char buf[1024]; - char *answer; - int min_bw = atoi(question); - routerlist_t *rl = router_get_routerlist(); - - (void) control_conn; - - if (strcmpstart(question, "unregistered-servers-")) - return 0; - question += strlen("unregistered-servers-"); - - answerlist = smartlist_create(); - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ent, { - uint32_t r = dirserv_router_get_status(ent, NULL); - if (router_get_advertised_bandwidth(ent) >= (size_t)min_bw && - !(r & FP_NAMED)) { - /* then log this one */ - tor_snprintf(buf, sizeof(buf), - "%s: BW %d on '%s'.", - ent->nickname, router_get_advertised_bandwidth(ent), - ent->platform ? ent->platform : ""); - smartlist_add(answerlist, tor_strdup(buf)); - } - }); - answer = smartlist_join_strings(answerlist, "\r\n", 0, NULL); - SMARTLIST_FOREACH(answerlist, char *, cp, tor_free(cp)); - smartlist_free(answerlist); - *answer_out = answer; - return 0; -} - /** Mark the directory as <b>dirty</b> -- when we're next asked for a * directory, we will rebuild it instead of reusing the most recently * generated one. @@ -933,28 +923,66 @@ list_single_server_status(routerinfo_t *desc, int is_live) return tor_strdup(buf); } +static INLINE int +running_long_enough_to_decide_unreachable(void) +{ + return time_of_process_start + + get_options()->TestingAuthDirTimeToLearnReachability < approx_time(); +} + /** Each server needs to have passed a reachability test no more * than this number of seconds ago, or he is listed as down in * the directory. */ #define REACHABLE_TIMEOUT (45*60) +/** If we tested a router and found it reachable _at least this long_ after it + * declared itself hibernating, it is probably done hibernating and we just + * missed a descriptor from it. */ +#define HIBERNATION_PUBLICATION_SKEW (60*60) + /** Treat a router as alive if * - It's me, and I'm not hibernating. * or - We've found it reachable recently. */ void dirserv_set_router_is_running(routerinfo_t *router, time_t now) { + /*XXXX023 This function is a mess. Separate out the part that calculates + whether it's reachable and the part that tells rephist that the router was + unreachable. + */ int answer; - if (router_is_me(router) && !we_are_hibernating()) + if (router_is_me(router)) { + /* We always know if we are down ourselves. */ + answer = ! we_are_hibernating(); + } else if (router->is_hibernating && + (router->cache_info.published_on + + HIBERNATION_PUBLICATION_SKEW) > router->last_reachable) { + /* A hibernating router is down unless we (somehow) had contact with it + * since it declared itself to be hibernating. */ + answer = 0; + } else if (get_options()->AssumeReachable) { + /* If AssumeReachable, everybody is up unless they say they are down! */ answer = 1; - else - answer = get_options()->AssumeReachable || - now < router->last_reachable + REACHABLE_TIMEOUT; + } else { + /* Otherwise, a router counts as up if we found it reachable in the last + REACHABLE_TIMEOUT seconds. */ + answer = (now < router->last_reachable + REACHABLE_TIMEOUT); + } + + if (!answer && running_long_enough_to_decide_unreachable()) { + /* Not considered reachable. tell rephist about that. - if (!answer) { - /* not considered reachable. tell rephist. */ - rep_hist_note_router_unreachable(router->cache_info.identity_digest, now); + Because we launch a reachability test for each router every + REACHABILITY_TEST_CYCLE_PERIOD seconds, then the router has probably + been down since at least that time after we last successfully reached + it. + */ + time_t when = now; + if (router->last_reachable && + router->last_reachable + REACHABILITY_TEST_CYCLE_PERIOD < now) + when = router->last_reachable + REACHABILITY_TEST_CYCLE_PERIOD; + rep_hist_note_router_unreachable(router->cache_info.identity_digest, when); } router->is_running = answer; @@ -965,7 +993,6 @@ dirserv_set_router_is_running(routerinfo_t *router, time_t now) * *<b>router_status_out</b>. Return 0 on success, -1 on failure. * * If for_controller is true, include the routers with very old descriptors. - * If for_controller is >1, use the verbose nickname format. */ int list_server_status_v1(smartlist_t *routers, char **router_status_out, @@ -985,23 +1012,22 @@ list_server_status_v1(smartlist_t *routers, char **router_status_out, rs_entries = smartlist_create(); - SMARTLIST_FOREACH(routers, routerinfo_t *, ri, - { + SMARTLIST_FOREACH_BEGIN(routers, routerinfo_t *, ri) { if (authdir) { /* Update router status in routerinfo_t. */ dirserv_set_router_is_running(ri, now); } - if (for_controller == 1 || ri->cache_info.published_on >= cutoff) - smartlist_add(rs_entries, list_single_server_status(ri, ri->is_running)); - else if (for_controller > 2) { + if (for_controller) { char name_buf[MAX_VERBOSE_NICKNAME_LEN+2]; char *cp = name_buf; if (!ri->is_running) *cp++ = '!'; router_get_verbose_nickname(cp, ri); smartlist_add(rs_entries, tor_strdup(name_buf)); + } else if (ri->cache_info.published_on >= cutoff) { + smartlist_add(rs_entries, list_single_server_status(ri, ri->is_running)); } - }); + } SMARTLIST_FOREACH_END(ri); *router_status_out = smartlist_join_strings(rs_entries, " ", 0, NULL); @@ -1119,7 +1145,8 @@ dirserv_dump_directory_to_string(char **dir_out, return -1; } note_crypto_pk_op(SIGN_DIR); - if (router_append_dirobj_signature(buf,buf_len,digest,private_key)<0) { + if (router_append_dirobj_signature(buf,buf_len,digest,DIGEST_LEN, + private_key)<0) { tor_free(buf); return -1; } @@ -1144,18 +1171,21 @@ directory_fetches_from_authorities(or_options_t *options) { routerinfo_t *me; uint32_t addr; + int refuseunknown; if (options->FetchDirInfoEarly) return 1; if (options->BridgeRelay == 1) return 0; if (server_mode(options) && router_pick_published_address(options, &addr)<0) return 1; /* we don't know our IP address; ask an authority. */ - if (options->DirPort == 0) + refuseunknown = ! router_my_exit_policy_is_reject_star() && + should_refuse_unknown_exits(options); + if (options->DirPort == 0 && !refuseunknown) return 0; if (!server_mode(options) || !advertised_server_mode()) return 0; me = router_get_my_routerinfo(); - if (!me || !me->dir_port) + if (!me || (!me->dir_port && !refuseunknown)) return 0; /* if dirport not advertised, return 0 too */ return 1; } @@ -1195,7 +1225,14 @@ directory_caches_v2_dir_info(or_options_t *options) int directory_caches_dir_info(or_options_t *options) { - return options->BridgeRelay != 0 || options->DirPort != 0; + if (options->BridgeRelay || options->DirPort) + return 1; + if (!server_mode(options) || !advertised_server_mode()) + return 0; + /* We need an up-to-date view of network info if we're going to try to + * block exit attempts from unknown relays. */ + return ! router_my_exit_policy_is_reject_star() && + should_refuse_unknown_exits(options); } /** Return 1 if we want to allow remote people to ask us directory @@ -1238,14 +1275,14 @@ directory_too_idle_to_fetch_descriptors(or_options_t *options, time_t now) static cached_dir_t *cached_directory = NULL; /** The v1 runningrouters document we'll serve (as a cache or as an authority) * if requested. */ -static cached_dir_t cached_runningrouters = { NULL, NULL, 0, 0, 0, -1 }; +static cached_dir_t cached_runningrouters; /** Used for other dirservers' v2 network statuses. Map from hexdigest to * cached_dir_t. */ static digestmap_t *cached_v2_networkstatus = NULL; -/** The v3 consensus network status that we're currently serving. */ -static cached_dir_t *cached_v3_networkstatus = NULL; +/** Map from flavor name to the v3 consensuses that we're currently serving. */ +static strmap_t *cached_consensuses = NULL; /** Possibly replace the contents of <b>d</b> with the value of * <b>directory</b> published on <b>when</b>, unless <b>when</b> is older than @@ -1319,7 +1356,11 @@ clear_cached_dir(cached_dir_t *d) static void _free_cached_dir(void *_d) { - cached_dir_t *d = (cached_dir_t *)_d; + cached_dir_t *d; + if (!_d) + return; + + d = (cached_dir_t *)_d; cached_dir_decref(d); } @@ -1413,17 +1454,26 @@ dirserv_set_cached_networkstatus_v2(const char *networkstatus, } } -/** Replace the v3 consensus networkstatus that we're serving with - * <b>networkstatus</b>, published at <b>published</b>. No validation is - * performed. */ +/** Replace the v3 consensus networkstatus of type <b>flavor_name</b> that + * we're serving with <b>networkstatus</b>, published at <b>published</b>. No + * validation is performed. */ void -dirserv_set_cached_networkstatus_v3(const char *networkstatus, - time_t published) +dirserv_set_cached_consensus_networkstatus(const char *networkstatus, + const char *flavor_name, + const digests_t *digests, + time_t published) { - if (cached_v3_networkstatus) - cached_dir_decref(cached_v3_networkstatus); - cached_v3_networkstatus = new_cached_dir( - tor_strdup(networkstatus), published); + cached_dir_t *new_networkstatus; + cached_dir_t *old_networkstatus; + if (!cached_consensuses) + cached_consensuses = strmap_new(); + + new_networkstatus = new_cached_dir(tor_strdup(networkstatus), published); + memcpy(&new_networkstatus->digests, digests, sizeof(digests_t)); + old_networkstatus = strmap_set(cached_consensuses, flavor_name, + new_networkstatus); + if (old_networkstatus) + cached_dir_decref(old_networkstatus); } /** Remove any v2 networkstatus from the directory cache that was published @@ -1520,7 +1570,8 @@ dirserv_regenerate_directory(void) { char *new_directory=NULL; - if (dirserv_dump_directory_to_string(&new_directory, get_identity_key())) { + if (dirserv_dump_directory_to_string(&new_directory, + get_server_identity_key())) { log_warn(LD_BUG, "Error creating directory."); tor_free(new_directory); return NULL; @@ -1550,7 +1601,7 @@ generate_runningrouters(void) char digest[DIGEST_LEN]; char published[ISO_TIME_LEN+1]; size_t len; - crypto_pk_env_t *private_key = get_identity_key(); + crypto_pk_env_t *private_key = get_server_identity_key(); char *identity_pkey; /* Identity key, DER64-encoded. */ size_t identity_pkey_len; @@ -1577,7 +1628,8 @@ generate_runningrouters(void) goto err; } note_crypto_pk_op(SIGN_DIR); - if (router_append_dirobj_signature(s, len, digest, private_key)<0) + if (router_append_dirobj_signature(s, len, digest, DIGEST_LEN, + private_key)<0) goto err; set_cached_dir(&the_runningrouters, s, time(NULL)); @@ -1605,9 +1657,11 @@ dirserv_get_runningrouters(void) /** Return the latest downloaded consensus networkstatus in encoded, signed, * optionally compressed format, suitable for sending to clients. */ cached_dir_t * -dirserv_get_consensus(void) +dirserv_get_consensus(const char *flavor_name) { - return cached_v3_networkstatus; + if (!cached_consensuses) + return NULL; + return strmap_get(cached_consensuses, flavor_name); } /** For authoritative directories: the current (v2) network status. */ @@ -1645,7 +1699,7 @@ should_generate_v2_networkstatus(void) #define TIME_KNOWN_TO_GUARANTEE_FAMILIAR (8*24*60*60) /** Similarly, every node with sufficient WFU is around enough to be a guard. */ -#define WFU_TO_GUARANTEE_GUARD (0.995) +#define WFU_TO_GUARANTEE_GUARD (0.98) /* Thresholds for server performance: set by * dirserv_compute_performance_thresholds, and used by @@ -1700,9 +1754,12 @@ dirserv_thinks_router_is_unreliable(time_t now, { if (need_uptime) { if (!enough_mtbf_info) { - /* XXX022 Once most authorities are on v3, we should change the rule from + /* XXX023 Once most authorities are on v3, we should change the rule from * "use uptime if we don't have mtbf data" to "don't advertise Stable on - * v3 if we don't have enough mtbf data." */ + * v3 if we don't have enough mtbf data." Or maybe not, since if we ever + * hit a point where we need to reset a lot of authorities at once, + * none of them would be in a position to declare Stable. + */ long uptime = real_uptime(router, now); if ((unsigned)uptime < stable_uptime && (unsigned)uptime < UPTIME_TO_GUARANTEE_STABLE) @@ -1725,7 +1782,7 @@ dirserv_thinks_router_is_unreliable(time_t now, /** Return true iff <b>router</b> should be assigned the "HSDir" flag. * Right now this means it advertises support for it, it has a high - * uptime, and it's currently considered Running. + * uptime, it has a DirPort open, and it's currently considered Running. * * This function needs to be called after router-\>is_running has * been set. @@ -1733,9 +1790,31 @@ dirserv_thinks_router_is_unreliable(time_t now, static int dirserv_thinks_router_is_hs_dir(routerinfo_t *router, time_t now) { - long uptime = real_uptime(router, now); - return (router->wants_to_be_hs_dir && + long uptime; + + /* If we haven't been running for at least + * get_options()->MinUptimeHidServDirectoryV2 seconds, we can't + * have accurate data telling us a relay has been up for at least + * that long. We also want to allow a bit of slack: Reachability + * tests aren't instant. If we haven't been running long enough, + * trust the relay. */ + + if (stats_n_seconds_working > + get_options()->MinUptimeHidServDirectoryV2 * 1.1) + uptime = MIN(rep_hist_get_uptime(router->cache_info.identity_digest, now), + real_uptime(router, now)); + else + uptime = real_uptime(router, now); + + /* XXX We shouldn't need to check dir_port, but we do because of + * bug 1693. In the future, once relays set wants_to_be_hs_dir + * correctly, we can revert to only checking dir_port if router's + * version is too old. */ + /* XXX Unfortunately, we need to keep checking dir_port until all + * *clients* suffering from bug 2722 are obsolete. The first version + * to fix the bug was 0.2.2.25-alpha. */ + return (router->wants_to_be_hs_dir && router->dir_port && uptime > get_options()->MinUptimeHidServDirectoryV2 && router->is_running); } @@ -1790,7 +1869,8 @@ dirserv_compute_performance_thresholds(routerlist_t *rl) if (router_is_active(ri, now)) { const char *id = ri->cache_info.identity_digest; uint32_t bw; - ri->is_exit = exit_policy_is_general_exit(ri->exit_policy); + ri->is_exit = (!router_exit_policy_rejects_all(ri) && + exit_policy_is_general_exit(ri->exit_policy)); uptimes[n_active] = (uint32_t)real_uptime(ri, now); mtbfs[n_active] = rep_hist_get_stability(id, now); tks [n_active] = rep_hist_get_weighted_time_known(id, now); @@ -1897,16 +1977,20 @@ version_from_platform(const char *platform) * which has at least <b>buf_len</b> free characters. Do NUL-termination. * Use the same format as in network-status documents. If <b>version</b> is * non-NULL, add a "v" line for the platform. Return 0 on success, -1 on - * failure. If <b>first_line_only</b> is true, don't include any flags - * or version line. + * failure. + * + * The format argument has three possible values: + * NS_V2 - Output an entry suitable for a V2 NS opinion document + * NS_V3_CONSENSUS - Output the first portion of a V3 NS consensus entry + * NS_V3_CONSENSUS_MICRODESC - Output the first portion of a V3 microdesc + * consensus entry. + * NS_V3_VOTE - Output a complete V3 NS vote + * NS_CONTROL_PORT - Output a NS document for the control port */ int routerstatus_format_entry(char *buf, size_t buf_len, routerstatus_t *rs, const char *version, - int first_line_only, int v2_format) -/* XXX: first_line_only and v2_format should probably be be both - * replaced by a single purpose parameter. - */ + routerstatus_format_type_t format) { int r; struct in_addr in; @@ -1925,10 +2009,11 @@ routerstatus_format_entry(char *buf, size_t buf_len, tor_inet_ntoa(&in, ipaddr, sizeof(ipaddr)); r = tor_snprintf(buf, buf_len, - "r %s %s %s %s %s %d %d\n", + "r %s %s %s%s%s %s %d %d\n", rs->nickname, identity64, - digest64, + (format==NS_V3_CONSENSUS_MICRODESC)?"":digest64, + (format==NS_V3_CONSENSUS_MICRODESC)?"":" ", published, ipaddr, (int)rs->or_port, @@ -1937,7 +2022,12 @@ routerstatus_format_entry(char *buf, size_t buf_len, log_warn(LD_BUG, "Not enough space in buffer."); return -1; } - if (first_line_only) + + /* TODO: Maybe we want to pass in what we need to build the rest of + * this here, instead of in the caller. Then we could use the + * networkstatus_type_t values, with an additional control port value + * added -MP */ + if (format == NS_V3_CONSENSUS || format == NS_V3_CONSENSUS_MICRODESC) return 0; cp = buf + strlen(buf); @@ -1974,62 +2064,87 @@ routerstatus_format_entry(char *buf, size_t buf_len, cp += strlen(cp); } - if (!v2_format) { + if (format != NS_V2) { routerinfo_t* desc = router_get_by_digest(rs->identity_digest); + uint32_t bw; + + if (format != NS_CONTROL_PORT) { + /* Blow up more or less nicely if we didn't get anything or not the + * thing we expected. + */ + if (!desc) { + char id[HEX_DIGEST_LEN+1]; + char dd[HEX_DIGEST_LEN+1]; + + base16_encode(id, sizeof(id), rs->identity_digest, DIGEST_LEN); + base16_encode(dd, sizeof(dd), rs->descriptor_digest, DIGEST_LEN); + log_warn(LD_BUG, "Cannot get any descriptor for %s " + "(wanted descriptor %s).", + id, dd); + return -1; + }; + + /* This assert can fire for the control port, because + * it can request NS documents before all descriptors + * have been fetched. */ + if (memcmp(desc->cache_info.signed_descriptor_digest, + rs->descriptor_digest, + DIGEST_LEN)) { + char rl_d[HEX_DIGEST_LEN+1]; + char rs_d[HEX_DIGEST_LEN+1]; + char id[HEX_DIGEST_LEN+1]; + + base16_encode(rl_d, sizeof(rl_d), + desc->cache_info.signed_descriptor_digest, DIGEST_LEN); + base16_encode(rs_d, sizeof(rs_d), rs->descriptor_digest, DIGEST_LEN); + base16_encode(id, sizeof(id), rs->identity_digest, DIGEST_LEN); + log_err(LD_BUG, "descriptor digest in routerlist does not match " + "the one in routerstatus: %s vs %s " + "(router %s)\n", + rl_d, rs_d, id); + + tor_assert(!memcmp(desc->cache_info.signed_descriptor_digest, + rs->descriptor_digest, + DIGEST_LEN)); + }; + } - /* Blow up more or less nicely if we didn't get anything or not the - * thing we expected. - */ - if (!desc) { - char id[HEX_DIGEST_LEN+1]; - char dd[HEX_DIGEST_LEN+1]; - - base16_encode(id, sizeof(id), rs->identity_digest, DIGEST_LEN); - base16_encode(dd, sizeof(dd), rs->descriptor_digest, DIGEST_LEN); - log_warn(LD_BUG, "Cannot get any descriptor for %s " - "(wanted descriptor %s).", - id, dd); - return -1; - }; - if (memcmp(desc->cache_info.signed_descriptor_digest, - rs->descriptor_digest, - DIGEST_LEN)) { - char rl_d[HEX_DIGEST_LEN+1]; - char rs_d[HEX_DIGEST_LEN+1]; - char id[HEX_DIGEST_LEN+1]; - - base16_encode(rl_d, sizeof(rl_d), - desc->cache_info.signed_descriptor_digest, DIGEST_LEN); - base16_encode(rs_d, sizeof(rs_d), rs->descriptor_digest, DIGEST_LEN); - base16_encode(id, sizeof(id), rs->identity_digest, DIGEST_LEN); - log_err(LD_BUG, "descriptor digest in routerlist does not match " - "the one in routerstatus: %s vs %s " - "(router %s)\n", - rl_d, rs_d, id); - - tor_assert(!memcmp(desc->cache_info.signed_descriptor_digest, - rs->descriptor_digest, - DIGEST_LEN)); - }; - + if (format == NS_CONTROL_PORT && rs->has_bandwidth) { + bw = rs->bandwidth; + } else { + tor_assert(desc); + bw = router_get_advertised_bandwidth_capped(desc) / 1000; + } r = tor_snprintf(cp, buf_len - (cp-buf), - "w Bandwidth=%d\n", - router_get_advertised_bandwidth_capped(desc) / 1024); + "w Bandwidth=%d\n", bw); + if (r<0) { log_warn(LD_BUG, "Not enough space in buffer."); return -1; } cp += strlen(cp); + if (format == NS_V3_VOTE && rs->has_measured_bw) { + *--cp = '\0'; /* Kill "\n" */ + r = tor_snprintf(cp, buf_len - (cp-buf), + " Measured=%d\n", rs->measured_bw); + if (r<0) { + log_warn(LD_BUG, "Not enough space in buffer for weight line."); + return -1; + } + cp += strlen(cp); + } - summary = policy_summarize(desc->exit_policy); - r = tor_snprintf(cp, buf_len - (cp-buf), "p %s\n", summary); - if (r<0) { - log_warn(LD_BUG, "Not enough space in buffer."); + if (desc) { + summary = policy_summarize(desc->exit_policy); + r = tor_snprintf(cp, buf_len - (cp-buf), "p %s\n", summary); + if (r<0) { + log_warn(LD_BUG, "Not enough space in buffer."); + tor_free(summary); + return -1; + } + cp += strlen(cp); tor_free(summary); - return -1; } - cp += strlen(cp); - tor_free(summary); } return 0; @@ -2132,9 +2247,7 @@ get_possible_sybil_list(const smartlist_t *routers) /** Extract status information from <b>ri</b> and from other authority * functions and store it in <b>rs</b>>. If <b>naming</b>, consider setting - * the named flag in <b>rs</b>. If not <b>exits_can_be_guards</b>, never mark - * an exit as a guard. If <b>listbadexits</b>, consider setting the badexit - * flag. + * the named flag in <b>rs</b>. * * We assume that ri-\>is_running has already been set, e.g. by * dirserv_set_router_is_running(ri, now); @@ -2142,8 +2255,8 @@ get_possible_sybil_list(const smartlist_t *routers) void set_routerstatus_from_routerinfo(routerstatus_t *rs, routerinfo_t *ri, time_t now, - int naming, int exits_can_be_guards, - int listbadexits, int listbaddirs) + int naming, int listbadexits, + int listbaddirs) { int unstable_version = !tor_version_as_new_as(ri->platform,"0.1.1.16-rc-cvs"); @@ -2172,11 +2285,10 @@ set_routerstatus_from_routerinfo(routerstatus_t *rs, rs->is_valid = ri->is_valid; if (rs->is_fast && - (!rs->is_exit || exits_can_be_guards) && (router_get_advertised_bandwidth(ri) >= BANDWIDTH_TO_GUARANTEE_GUARD || router_get_advertised_bandwidth(ri) >= - (exits_can_be_guards ? guard_bandwidth_including_exits : - guard_bandwidth_excluding_exits))) { + MIN(guard_bandwidth_including_exits, + guard_bandwidth_excluding_exits))) { long tk = rep_hist_get_weighted_time_known( ri->cache_info.identity_digest, now); double wfu = rep_hist_get_weighted_fractional_uptime( @@ -2232,6 +2344,177 @@ router_clear_status_flags(routerinfo_t *router) router->is_bad_exit = router->is_bad_directory = 0; } +/** + * Helper function to parse out a line in the measured bandwidth file + * into a measured_bw_line_t output structure. Returns -1 on failure + * or 0 on success. + */ +int +measured_bw_line_parse(measured_bw_line_t *out, const char *orig_line) +{ + char *line = tor_strdup(orig_line); + char *cp = line; + int got_bw = 0; + int got_node_id = 0; + char *strtok_state; /* lame sauce d'jour */ + cp = tor_strtok_r(cp, " \t", &strtok_state); + + if (!cp) { + log_warn(LD_DIRSERV, "Invalid line in bandwidth file: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } + + if (orig_line[strlen(orig_line)-1] != '\n') { + log_warn(LD_DIRSERV, "Incomplete line in bandwidth file: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } + + do { + if (strcmpstart(cp, "bw=") == 0) { + int parse_ok = 0; + char *endptr; + if (got_bw) { + log_warn(LD_DIRSERV, "Double bw= in bandwidth file line: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } + cp+=strlen("bw="); + + out->bw = tor_parse_long(cp, 0, 0, LONG_MAX, &parse_ok, &endptr); + if (!parse_ok || (*endptr && !TOR_ISSPACE(*endptr))) { + log_warn(LD_DIRSERV, "Invalid bandwidth in bandwidth file line: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } + got_bw=1; + } else if (strcmpstart(cp, "node_id=$") == 0) { + if (got_node_id) { + log_warn(LD_DIRSERV, "Double node_id= in bandwidth file line: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } + cp+=strlen("node_id=$"); + + if (strlen(cp) != HEX_DIGEST_LEN || + base16_decode(out->node_id, DIGEST_LEN, cp, HEX_DIGEST_LEN)) { + log_warn(LD_DIRSERV, "Invalid node_id in bandwidth file line: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } + strncpy(out->node_hex, cp, sizeof(out->node_hex)); + got_node_id=1; + } + } while ((cp = tor_strtok_r(NULL, " \t", &strtok_state))); + + if (got_bw && got_node_id) { + tor_free(line); + return 0; + } else { + log_warn(LD_DIRSERV, "Incomplete line in bandwidth file: %s", + escaped(orig_line)); + tor_free(line); + return -1; + } +} + +/** + * Helper function to apply a parsed measurement line to a list + * of bandwidth statuses. Returns true if a line is found, + * false otherwise. + */ +int +measured_bw_line_apply(measured_bw_line_t *parsed_line, + smartlist_t *routerstatuses) +{ + routerstatus_t *rs = NULL; + if (!routerstatuses) + return 0; + + rs = smartlist_bsearch(routerstatuses, parsed_line->node_id, + compare_digest_to_routerstatus_entry); + + if (rs) { + rs->has_measured_bw = 1; + rs->measured_bw = (uint32_t)parsed_line->bw; + } else { + log_info(LD_DIRSERV, "Node ID %s not found in routerstatus list", + parsed_line->node_hex); + } + + return rs != NULL; +} + +/** + * Read the measured bandwidth file and apply it to the list of + * routerstatuses. Returns -1 on error, 0 otherwise. + */ +int +dirserv_read_measured_bandwidths(const char *from_file, + smartlist_t *routerstatuses) +{ + char line[256]; + FILE *fp = fopen(from_file, "r"); + int applied_lines = 0; + time_t file_time; + int ok; + if (fp == NULL) { + log_warn(LD_CONFIG, "Can't open bandwidth file at configured location: %s", + from_file); + return -1; + } + + if (!fgets(line, sizeof(line), fp) + || !strlen(line) || line[strlen(line)-1] != '\n') { + log_warn(LD_DIRSERV, "Long or truncated time in bandwidth file: %s", + escaped(line)); + fclose(fp); + return -1; + } + + line[strlen(line)-1] = '\0'; + file_time = tor_parse_ulong(line, 10, 0, ULONG_MAX, &ok, NULL); + if (!ok) { + log_warn(LD_DIRSERV, "Non-integer time in bandwidth file: %s", + escaped(line)); + fclose(fp); + return -1; + } + + if ((time(NULL) - file_time) > MAX_MEASUREMENT_AGE) { + log_warn(LD_DIRSERV, "Bandwidth measurement file stale. Age: %u", + (unsigned)(time(NULL) - file_time)); + fclose(fp); + return -1; + } + + if (routerstatuses) + smartlist_sort(routerstatuses, compare_routerstatus_entries); + + while (!feof(fp)) { + measured_bw_line_t parsed_line; + if (fgets(line, sizeof(line), fp) && strlen(line)) { + if (measured_bw_line_parse(&parsed_line, line) != -1) { + if (measured_bw_line_apply(&parsed_line, routerstatuses) > 0) + applied_lines++; + } + } + } + + fclose(fp); + log_info(LD_DIRSERV, + "Bandwidth measurement file successfully read. " + "Applied %d measurements.", applied_lines); + return 0; +} + /** Return a new networkstatus_t* containing our current opinion. (For v3 * authorities) */ networkstatus_t * @@ -2249,22 +2532,18 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, int naming = options->NamingAuthoritativeDir; int listbadexits = options->AuthDirListBadExits; int listbaddirs = options->AuthDirListBadDirs; - int exits_can_be_guards; routerlist_t *rl = router_get_routerlist(); time_t now = time(NULL); time_t cutoff = now - ROUTER_MAX_AGE_TO_PUBLISH; networkstatus_voter_info_t *voter = NULL; vote_timing_t timing; digestmap_t *omit_as_sybil = NULL; - int vote_on_reachability = 1; + const int vote_on_reachability = running_long_enough_to_decide_unreachable(); + smartlist_t *microdescriptors = NULL; tor_assert(private_key); tor_assert(cert); - if (now - time_of_process_start < - options->TestingAuthDirTimeToLearnReachability) - vote_on_reachability = 0; - if (resolve_my_address(LOG_WARN, options, &addr, &hostname)<0) { log_warn(LD_NET, "Couldn't resolve my hostname"); return NULL; @@ -2299,27 +2578,24 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, dirserv_compute_performance_thresholds(rl); - /* XXXX We should take steps to keep this from oscillating if - * total_exit_bandwidth is close to total_bandwidth/3. */ - exits_can_be_guards = total_exit_bandwidth >= (total_bandwidth / 3); - routers = smartlist_create(); smartlist_add_all(routers, rl->routers); routers_sort_by_identity(routers); omit_as_sybil = get_possible_sybil_list(routers); routerstatuses = smartlist_create(); + microdescriptors = smartlist_create(); - SMARTLIST_FOREACH(routers, routerinfo_t *, ri, { + SMARTLIST_FOREACH_BEGIN(routers, routerinfo_t *, ri) { if (ri->cache_info.published_on >= cutoff) { routerstatus_t *rs; vote_routerstatus_t *vrs; + microdesc_t *md; vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; set_routerstatus_from_routerinfo(rs, ri, now, - naming, exits_can_be_guards, - listbadexits, listbaddirs); + naming, listbadexits, listbaddirs); if (digestmap_get(omit_as_sybil, ri->cache_info.identity_digest)) clear_status_flags_on_sybil(rs); @@ -2328,12 +2604,39 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, rs->is_running = 0; vrs->version = version_from_platform(ri->platform); + md = dirvote_create_microdescriptor(ri); + if (md) { + char buf[128]; + vote_microdesc_hash_t *h; + dirvote_format_microdesc_vote_line(buf, sizeof(buf), md); + h = tor_malloc(sizeof(vote_microdesc_hash_t)); + h->microdesc_hash_line = tor_strdup(buf); + h->next = NULL; + vrs->microdesc = h; + md->last_listed = now; + smartlist_add(microdescriptors, md); + } + smartlist_add(routerstatuses, vrs); } - }); + } SMARTLIST_FOREACH_END(ri); + + { + smartlist_t *added = + microdescs_add_list_to_cache(get_microdesc_cache(), + microdescriptors, SAVED_NOWHERE, 0); + smartlist_free(added); + smartlist_free(microdescriptors); + } + smartlist_free(routers); digestmap_free(omit_as_sybil, NULL); + if (options->V3BandwidthsFile) { + dirserv_read_measured_bandwidths(options->V3BandwidthsFile, + routerstatuses); + } + v3_out = tor_malloc_zero(sizeof(networkstatus_t)); v3_out->type = NS_TYPE_VOTE; @@ -2383,19 +2686,29 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, } smartlist_sort_strings(v3_out->known_flags); + if (options->ConsensusParams) { + v3_out->net_params = smartlist_create(); + smartlist_split_string(v3_out->net_params, + options->ConsensusParams, NULL, 0, 0); + smartlist_sort_strings(v3_out->net_params); + } + voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); voter->nickname = tor_strdup(options->Nickname); memcpy(voter->identity_digest, identity_digest, DIGEST_LEN); + voter->sigs = smartlist_create(); voter->address = hostname; voter->addr = addr; voter->dir_port = options->DirPort; voter->or_port = options->ORPort; voter->contact = tor_strdup(contact); - memcpy(voter->signing_key_digest, signing_key_digest, DIGEST_LEN); if (options->V3AuthUseLegacyKey) { authority_cert_t *c = get_my_v3_legacy_cert(); if (c) { - crypto_pk_get_digest(c->identity_key, voter->legacy_id_digest); + if (crypto_pk_get_digest(c->identity_key, voter->legacy_id_digest)) { + log_warn(LD_BUG, "Unable to compute digest of legacy v3 identity key"); + memset(voter->legacy_id_digest, 0, DIGEST_LEN); + } } } @@ -2435,13 +2748,12 @@ generate_v2_networkstatus_opinion(void) int versioning = options->VersioningAuthoritativeDir; int listbaddirs = options->AuthDirListBadDirs; int listbadexits = options->AuthDirListBadExits; - int exits_can_be_guards; const char *contact; char *version_lines = NULL; smartlist_t *routers = NULL; digestmap_t *omit_as_sybil = NULL; - private_key = get_identity_key(); + private_key = get_server_identity_key(); if (resolve_my_address(LOG_WARN, options, &addr, &hostname)<0) { log_warn(LD_NET, "Couldn't resolve my hostname"); @@ -2515,10 +2827,6 @@ generate_v2_networkstatus_opinion(void) dirserv_compute_performance_thresholds(rl); - /* XXXX We should take steps to keep this from oscillating if - * total_exit_bandwidth is close to total_bandwidth/3. */ - exits_can_be_guards = total_exit_bandwidth >= (total_bandwidth / 3); - routers = smartlist_create(); smartlist_add_all(routers, rl->routers); routers_sort_by_identity(routers); @@ -2531,13 +2839,12 @@ generate_v2_networkstatus_opinion(void) char *version = version_from_platform(ri->platform); set_routerstatus_from_routerinfo(&rs, ri, now, - naming, exits_can_be_guards, - listbadexits, listbaddirs); + naming, listbadexits, listbaddirs); if (digestmap_get(omit_as_sybil, ri->cache_info.identity_digest)) clear_status_flags_on_sybil(&rs); - if (routerstatus_format_entry(outp, endp-outp, &rs, version, 0, 1)) { + if (routerstatus_format_entry(outp, endp-outp, &rs, version, NS_V2)) { log_warn(LD_BUG, "Unable to print router status."); tor_free(version); goto done; @@ -2559,7 +2866,8 @@ generate_v2_networkstatus_opinion(void) outp += strlen(outp); note_crypto_pk_op(SIGN_DIR); - if (router_append_dirobj_signature(outp,endp-outp,digest,private_key)<0) { + if (router_append_dirobj_signature(outp,endp-outp,digest,DIGEST_LEN, + private_key)<0) { log_warn(LD_BUG, "Unable to sign router status."); goto done; } @@ -2591,10 +2899,8 @@ generate_v2_networkstatus_opinion(void) tor_free(status); tor_free(hostname); tor_free(identity_pkey); - if (routers) - smartlist_free(routers); - if (omit_as_sybil) - digestmap_free(omit_as_sybil, NULL); + smartlist_free(routers); + digestmap_free(omit_as_sybil, NULL); return r; } @@ -2642,7 +2948,8 @@ dirserv_get_networkstatus_v2_fingerprints(smartlist_t *result, log_info(LD_DIRSERV, "Client requested 'all' network status objects; we have none."); } else if (!strcmpstart(key, "fp/")) { - dir_split_resource_into_fingerprints(key+3, result, NULL, 1, 1); + dir_split_resource_into_fingerprints(key+3, result, NULL, + DSR_HEX|DSR_SORT_UNIQ); } } @@ -2709,10 +3016,12 @@ dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, } else if (!strcmpstart(key, "d/")) { by_id = 0; key += strlen("d/"); - dir_split_resource_into_fingerprints(key, fps_out, NULL, 1, 1); + dir_split_resource_into_fingerprints(key, fps_out, NULL, + DSR_HEX|DSR_SORT_UNIQ); } else if (!strcmpstart(key, "fp/")) { key += strlen("fp/"); - dir_split_resource_into_fingerprints(key, fps_out, NULL, 1, 1); + dir_split_resource_into_fingerprints(key, fps_out, NULL, + DSR_HEX|DSR_SORT_UNIQ); } else { *msg = "Key not recognized"; return -1; @@ -2777,7 +3086,8 @@ dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, } else if (!strcmpstart(key, "/tor/server/d/")) { smartlist_t *digests = smartlist_create(); key += strlen("/tor/server/d/"); - dir_split_resource_into_fingerprints(key, digests, NULL, 1, 1); + dir_split_resource_into_fingerprints(key, digests, NULL, + DSR_HEX|DSR_SORT_UNIQ); SMARTLIST_FOREACH(digests, const char *, d, { signed_descriptor_t *sd = router_get_by_descriptor_digest(d); @@ -2790,7 +3100,8 @@ dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, smartlist_t *digests = smartlist_create(); time_t cutoff = time(NULL) - ROUTER_MAX_AGE_TO_PUBLISH; key += strlen("/tor/server/fp/"); - dir_split_resource_into_fingerprints(key, digests, NULL, 1, 1); + dir_split_resource_into_fingerprints(key, digests, NULL, + DSR_HEX|DSR_SORT_UNIQ); SMARTLIST_FOREACH(digests, const char *, d, { if (router_digest_is_me(d)) { @@ -2841,34 +3152,82 @@ dirserv_orconn_tls_done(const char *address, tor_assert(address); tor_assert(digest_rcvd); - SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri, { + /* XXX023 Doing a loop like this is stupid. We should just look up the + * router by digest_rcvd, and see if address, orport, and as_advertised + * match up. -NM */ + SMARTLIST_FOREACH_BEGIN(rl->routers, routerinfo_t *, ri) { if (!strcasecmp(address, ri->address) && or_port == ri->or_port && as_advertised && !memcmp(ri->cache_info.identity_digest, digest_rcvd, DIGEST_LEN)) { /* correct digest. mark this router reachable! */ if (!bridge_auth || ri->purpose == ROUTER_PURPOSE_BRIDGE) { - log_info(LD_DIRSERV, "Found router %s to be reachable. Yay.", - ri->nickname); - rep_hist_note_router_reachable(digest_rcvd, now); + tor_addr_t addr, *addrp=NULL; + log_info(LD_DIRSERV, "Found router %s to be reachable at %s:%d. Yay.", + ri->nickname, address, ri->or_port ); + if (tor_addr_from_str(&addr, ri->address) != -1) + addrp = &addr; + else + log_warn(LD_BUG, "Couldn't parse IP address \"%s\"", ri->address); + rep_hist_note_router_reachable(digest_rcvd, addrp, or_port, now); ri->last_reachable = now; } } - }); + } SMARTLIST_FOREACH_END(ri); /* FFFF Maybe we should reinstate the code that dumps routers with the same * addr/port but with nonmatching keys, but instead of dumping, we should * skip testing. */ } -/** Auth dir server only: if <b>try_all</b> is 1, launch connections to - * all known routers; else we want to load balance such that we only +/** Called when we, as an authority, receive a new router descriptor either as + * an upload or a download. Used to decide whether to relaunch reachability + * testing for the server. */ +int +dirserv_should_launch_reachability_test(routerinfo_t *ri, routerinfo_t *ri_old) +{ + if (!authdir_mode_handles_descs(get_options(), ri->purpose)) + return 0; + if (!ri_old) { + /* New router: Launch an immediate reachability test, so we will have an + * opinion soon in case we're generating a consensus soon */ + return 1; + } + if (ri_old->is_hibernating && !ri->is_hibernating) { + /* It just came out of hibernation; launch a reachability test */ + return 1; + } + if (! routers_have_same_or_addr(ri, ri_old)) { + /* Address or port changed; launch a reachability test */ + return 1; + } + return 0; +} + +/** Helper function for dirserv_test_reachability(). Start a TLS + * connection to <b>router</b>, and annotate it with when we started + * the test. */ +void +dirserv_single_reachability_test(time_t now, routerinfo_t *router) +{ + tor_addr_t router_addr; + log_debug(LD_OR,"Testing reachability of %s at %s:%u.", + router->nickname, router->address, router->or_port); + /* Remember when we started trying to determine reachability */ + if (!router->testing_since) + router->testing_since = now; + tor_addr_from_ipv4h(&router_addr, router->addr); + connection_or_connect(&router_addr, router->or_port, + router->cache_info.identity_digest); +} + +/** Auth dir server only: load balance such that we only * try a few connections per call. * * The load balancing is such that if we get called once every ten - * seconds, we will cycle through all the tests in 1280 seconds (a - * bit over 20 minutes). + * seconds, we will cycle through all the tests in + * REACHABILITY_TEST_CYCLE_PERIOD seconds (a bit over 20 minutes). */ void -dirserv_test_reachability(time_t now, int try_all) +dirserv_test_reachability(time_t now) { /* XXX decide what to do here; see or-talk thread "purging old router * information, revocation." -NM @@ -2885,38 +3244,33 @@ dirserv_test_reachability(time_t now, int try_all) SMARTLIST_FOREACH_BEGIN(rl->routers, routerinfo_t *, router) { const char *id_digest = router->cache_info.identity_digest; - tor_addr_t router_addr; if (router_is_me(router)) continue; if (bridge_auth && router->purpose != ROUTER_PURPOSE_BRIDGE) continue; /* bridge authorities only test reachability on bridges */ // if (router->cache_info.published_on > cutoff) // continue; - if (try_all || (((uint8_t)id_digest[0]) % 128) == ctr) { - log_debug(LD_OR,"Testing reachability of %s at %s:%u.", - router->nickname, router->address, router->or_port); - /* Remember when we started trying to determine reachability */ - if (!router->testing_since) - router->testing_since = now; - tor_addr_from_ipv4h(&router_addr, router->addr); - connection_or_connect(&router_addr, router->or_port, id_digest); + if ((((uint8_t)id_digest[0]) % REACHABILITY_MODULO_PER_TEST) == ctr) { + dirserv_single_reachability_test(now, router); } } SMARTLIST_FOREACH_END(router); - if (!try_all) /* increment ctr */ - ctr = (ctr + 1) % 128; + ctr = (ctr + 1) % REACHABILITY_MODULO_PER_TEST; /* increment ctr */ } -/** Given a fingerprint <b>fp</b> which is either set if we're looking - * for a v2 status, or zeroes if we're looking for a v3 status, return - * a pointer to the appropriate cached dir object, or NULL if there isn't - * one available. */ +/** Given a fingerprint <b>fp</b> which is either set if we're looking for a + * v2 status, or zeroes if we're looking for a v3 status, or a NUL-padded + * flavor name if we want a flavored v3 status, return a pointer to the + * appropriate cached dir object, or NULL if there isn't one available. */ static cached_dir_t * lookup_cached_dir_by_fp(const char *fp) { cached_dir_t *d = NULL; - if (tor_digest_is_zero(fp) && cached_v3_networkstatus) - d = cached_v3_networkstatus; - else if (router_digest_is_me(fp) && the_v2_networkstatus) + if (tor_digest_is_zero(fp) && cached_consensuses) + d = strmap_get(cached_consensuses, "ns"); + else if (memchr(fp, '\0', DIGEST_LEN) && cached_consensuses && + (d = strmap_get(cached_consensuses, fp))) { + /* this here interface is a nasty hack XXXX023 */; + } else if (router_digest_is_me(fp) && the_v2_networkstatus) d = the_v2_networkstatus; else if (cached_v2_networkstatus) d = digestmap_get(cached_v2_networkstatus, fp); @@ -3002,6 +3356,18 @@ dirserv_have_any_serverdesc(smartlist_t *fps, int spool_src) return 0; } +/** Return true iff any of the 256-bit elements in <b>fps</b> is the digest of + * a microdescriptor we have. */ +int +dirserv_have_any_microdesc(const smartlist_t *fps) +{ + microdesc_cache_t *cache = get_microdesc_cache(); + SMARTLIST_FOREACH(fps, const char *, fp, + if (microdesc_cache_lookup_by_digest256(cache, fp)) + return 1); + return 0; +} + /** Return an approximate estimate of the number of bytes that will * be needed to transmit the server descriptors (if is_serverdescs -- * they can be either d/ or fp/ queries) or networkstatus objects (if @@ -3033,6 +3399,17 @@ dirserv_estimate_data_size(smartlist_t *fps, int is_serverdescs, return result; } +/** Given a list of microdescriptor hashes, guess how many bytes will be + * needed to transmit them, and return the guess. */ +size_t +dirserv_estimate_microdesc_size(const smartlist_t *fps, int compressed) +{ + size_t result = smartlist_len(fps) * microdesc_average_size(NULL); + if (compressed) + result /= 2; + return result; +} + /** When we're spooling data onto our outbuf, add more whenever we dip * below this threshold. */ #define DIRSERV_BUFFER_MIN 16384 @@ -3096,6 +3473,8 @@ connection_dirserv_add_servers_to_outbuf(dir_connection_t *conn) #endif body = signed_descriptor_get_body(sd); if (conn->zlib_state) { + /* XXXX022 This 'last' business should actually happen on the last + * routerinfo, not on the last fingerprint. */ int last = ! smartlist_len(conn->fingerprint_stack); connection_write_to_buf_zlib(body, sd->signed_descriptor_len, conn, last); @@ -3119,6 +3498,44 @@ connection_dirserv_add_servers_to_outbuf(dir_connection_t *conn) return 0; } +/** Spooling helper: called when we're sending a bunch of microdescriptors, + * and the outbuf has become too empty. Pulls some entries from + * fingerprint_stack, and writes the corresponding microdescs onto outbuf. If + * we run out of entries, flushes the zlib state and sets the spool source to + * NONE. Returns 0 on success, negative on failure. + */ +static int +connection_dirserv_add_microdescs_to_outbuf(dir_connection_t *conn) +{ + microdesc_cache_t *cache = get_microdesc_cache(); + while (smartlist_len(conn->fingerprint_stack) && + buf_datalen(conn->_base.outbuf) < DIRSERV_BUFFER_MIN) { + char *fp256 = smartlist_pop_last(conn->fingerprint_stack); + microdesc_t *md = microdesc_cache_lookup_by_digest256(cache, fp256); + tor_free(fp256); + if (!md) + continue; + if (conn->zlib_state) { + /* XXXX022 This 'last' business should actually happen on the last + * routerinfo, not on the last fingerprint. */ + int last = !smartlist_len(conn->fingerprint_stack); + connection_write_to_buf_zlib(md->body, md->bodylen, conn, last); + if (last) { + tor_zlib_free(conn->zlib_state); + conn->zlib_state = NULL; + } + } else { + connection_write_to_buf(md->body, md->bodylen, TO_CONN(conn)); + } + } + if (!smartlist_len(conn->fingerprint_stack)) { + conn->dir_spool_src = DIR_SPOOL_NONE; + smartlist_free(conn->fingerprint_stack); + conn->fingerprint_stack = NULL; + } + return 0; +} + /** Spooling helper: Called when we're sending a directory or networkstatus, * and the outbuf has become too empty. Pulls some bytes from * <b>conn</b>-\>cached_dir-\>dir_z, uncompresses them if appropriate, and @@ -3201,8 +3618,7 @@ connection_dirserv_add_networkstatus_bytes_to_outbuf(dir_connection_t *conn) } } else { connection_dirserv_finish_spooling(conn); - if (conn->fingerprint_stack) - smartlist_free(conn->fingerprint_stack); + smartlist_free(conn->fingerprint_stack); conn->fingerprint_stack = NULL; return 0; } @@ -3226,6 +3642,8 @@ connection_dirserv_flushed_some(dir_connection_t *conn) case DIR_SPOOL_SERVER_BY_DIGEST: case DIR_SPOOL_SERVER_BY_FP: return connection_dirserv_add_servers_to_outbuf(conn); + case DIR_SPOOL_MICRODESC: + return connection_dirserv_add_microdescs_to_outbuf(conn); case DIR_SPOOL_CACHED_DIR: return connection_dirserv_add_dir_bytes_to_outbuf(conn); case DIR_SPOOL_NETWORKSTATUS: @@ -3247,10 +3665,10 @@ dirserv_free_all(void) cached_dir_decref(the_v2_networkstatus); cached_dir_decref(cached_directory); clear_cached_dir(&cached_runningrouters); - if (cached_v2_networkstatus) { - digestmap_free(cached_v2_networkstatus, _free_cached_dir); - cached_v2_networkstatus = NULL; - } - cached_dir_decref(cached_v3_networkstatus); + + digestmap_free(cached_v2_networkstatus, _free_cached_dir); + cached_v2_networkstatus = NULL; + strmap_free(cached_consensuses, _free_cached_dir); + cached_consensuses = NULL; } diff --git a/src/or/dirserv.h b/src/or/dirserv.h new file mode 100644 index 000000000..569abfca2 --- /dev/null +++ b/src/or/dirserv.h @@ -0,0 +1,146 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file dirserv.h + * \brief Header file for dirserv.c. + **/ + +#ifndef _TOR_DIRSERV_H +#define _TOR_DIRSERV_H + +/** What fraction (1 over this number) of the relay ID space do we + * (as a directory authority) launch connections to at each reachability + * test? */ +#define REACHABILITY_MODULO_PER_TEST 128 + +/** How often (in seconds) do we launch reachability tests? */ +#define REACHABILITY_TEST_INTERVAL 10 + +/** How many seconds apart are the reachability tests for a given relay? */ +#define REACHABILITY_TEST_CYCLE_PERIOD \ + (REACHABILITY_TEST_INTERVAL*REACHABILITY_MODULO_PER_TEST) + +/** Maximum length of an exit policy summary. */ +#define MAX_EXITPOLICY_SUMMARY_LEN 1000 + +/** Maximum allowable length of a version line in a networkstatus. */ +#define MAX_V_LINE_LEN 128 +/** Length of "r Authority BadDirectory BadExit Exit Fast Guard HSDir Named + * Running Stable Unnamed V2Dir Valid\n". */ +#define MAX_FLAG_LINE_LEN 96 +/** Length of "w" line for weighting. Currently at most + * "w Bandwidth=<uint32t> Measured=<uint32t>\n" */ +#define MAX_WEIGHT_LINE_LEN (12+10+10+10+1) +/** Maximum length of an exit policy summary line. */ +#define MAX_POLICY_LINE_LEN (3+MAX_EXITPOLICY_SUMMARY_LEN) +/** Amount of space to allocate for each entry: r, s, and v lines. */ +#define RS_ENTRY_LEN \ + ( /* first line */ \ + MAX_NICKNAME_LEN+BASE64_DIGEST_LEN*2+ISO_TIME_LEN+INET_NTOA_BUF_LEN+ \ + 5*2 /* ports */ + 10 /* punctuation */ + \ + /* second line */ \ + MAX_FLAG_LINE_LEN + \ + /* weight line */ \ + MAX_WEIGHT_LINE_LEN + \ + /* p line. */ \ + MAX_POLICY_LINE_LEN + \ + /* v line. */ \ + MAX_V_LINE_LEN \ + ) + +#define UNNAMED_ROUTER_NICKNAME "Unnamed" + +int connection_dirserv_flushed_some(dir_connection_t *conn); + +int dirserv_add_own_fingerprint(const char *nickname, crypto_pk_env_t *pk); +int dirserv_load_fingerprint_file(void); +void dirserv_free_fingerprint_list(void); +const char *dirserv_get_nickname_by_digest(const char *digest); +enum was_router_added_t dirserv_add_multiple_descriptors( + const char *desc, uint8_t purpose, + const char *source, + const char **msg); +enum was_router_added_t dirserv_add_descriptor(routerinfo_t *ri, + const char **msg, + const char *source); +void dirserv_set_router_is_running(routerinfo_t *router, time_t now); +int list_server_status_v1(smartlist_t *routers, char **router_status_out, + int for_controller); +int dirserv_dump_directory_to_string(char **dir_out, + crypto_pk_env_t *private_key); + +int directory_fetches_from_authorities(or_options_t *options); +int directory_fetches_dir_info_early(or_options_t *options); +int directory_fetches_dir_info_later(or_options_t *options); +int directory_caches_v2_dir_info(or_options_t *options); +#define directory_caches_v1_dir_info(o) directory_caches_v2_dir_info(o) +int directory_caches_dir_info(or_options_t *options); +int directory_permits_begindir_requests(or_options_t *options); +int directory_permits_controller_requests(or_options_t *options); +int directory_too_idle_to_fetch_descriptors(or_options_t *options, time_t now); + +void directory_set_dirty(void); +cached_dir_t *dirserv_get_directory(void); +cached_dir_t *dirserv_get_runningrouters(void); +cached_dir_t *dirserv_get_consensus(const char *flavor_name); +void dirserv_set_cached_directory(const char *directory, time_t when, + int is_running_routers); +void dirserv_set_cached_networkstatus_v2(const char *directory, + const char *identity, + time_t published); +void dirserv_set_cached_consensus_networkstatus(const char *consensus, + const char *flavor_name, + const digests_t *digests, + time_t published); +void dirserv_clear_old_networkstatuses(time_t cutoff); +void dirserv_clear_old_v1_info(time_t now); +void dirserv_get_networkstatus_v2(smartlist_t *result, const char *key); +void dirserv_get_networkstatus_v2_fingerprints(smartlist_t *result, + const char *key); +int dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, + const char **msg, + int for_unencrypted_conn, + int is_extrainfo); +int dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, + const char **msg); +void dirserv_orconn_tls_done(const char *address, + uint16_t or_port, + const char *digest_rcvd, + int as_advertised); +int dirserv_should_launch_reachability_test(routerinfo_t *ri, + routerinfo_t *ri_old); +void dirserv_single_reachability_test(time_t now, routerinfo_t *router); +void dirserv_test_reachability(time_t now); +int authdir_wants_to_reject_router(routerinfo_t *ri, const char **msg, + int complain); +int dirserv_would_reject_router(routerstatus_t *rs); +int dirserv_remove_old_statuses(smartlist_t *fps, time_t cutoff); +int dirserv_have_any_serverdesc(smartlist_t *fps, int spool_src); +int dirserv_have_any_microdesc(const smartlist_t *fps); +size_t dirserv_estimate_data_size(smartlist_t *fps, int is_serverdescs, + int compressed); +size_t dirserv_estimate_microdesc_size(const smartlist_t *fps, int compressed); + +int routerstatus_format_entry(char *buf, size_t buf_len, + routerstatus_t *rs, const char *platform, + routerstatus_format_type_t format); +void dirserv_free_all(void); +void cached_dir_decref(cached_dir_t *d); +cached_dir_t *new_cached_dir(char *s, time_t published); + +#ifdef DIRSERV_PRIVATE +int measured_bw_line_parse(measured_bw_line_t *out, const char *line); + +int measured_bw_line_apply(measured_bw_line_t *parsed_line, + smartlist_t *routerstatuses); +#endif + +int dirserv_read_measured_bandwidths(const char *from_file, + smartlist_t *routerstatuses); + +#endif + diff --git a/src/or/dirvote.c b/src/or/dirvote.c index 656cb8a56..750c649f5 100644 --- a/src/or/dirvote.c +++ b/src/or/dirvote.c @@ -5,27 +5,72 @@ #define DIRVOTE_PRIVATE #include "or.h" +#include "config.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "policies.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" /** * \file dirvote.c * \brief Functions to compute directory consensus, and schedule voting. **/ -static int dirvote_add_signatures_to_pending_consensus( +/** A consensus that we have built and are appending signatures to. Once it's + * time to publish it, it will become an active consensus if it accumulates + * enough signatures. */ +typedef struct pending_consensus_t { + /** The body of the consensus that we're currently building. Once we + * have it built, it goes into dirserv.c */ + char *body; + /** The parsed in-progress consensus document. */ + networkstatus_t *consensus; +} pending_consensus_t; + +static int dirvote_add_signatures_to_all_pending_consensuses( const char *detached_signatures_body, const char **msg_out); +static int dirvote_add_signatures_to_pending_consensus( + pending_consensus_t *pc, + ns_detached_signatures_t *sigs, + const char **msg_out); static char *list_v3_auth_ids(void); static void dirvote_fetch_missing_votes(void); static void dirvote_fetch_missing_signatures(void); static int dirvote_perform_vote(void); static void dirvote_clear_votes(int all_votes); -static int dirvote_compute_consensus(void); +static int dirvote_compute_consensuses(void); static int dirvote_publish_consensus(void); +static char *make_consensus_method_list(int low, int high, const char *sep); + +/** The highest consensus method that we currently support. */ +#define MAX_SUPPORTED_CONSENSUS_METHOD 11 + +/** Lowest consensus method that contains a 'directory-footer' marker */ +#define MIN_METHOD_FOR_FOOTER 9 + +/** Lowest consensus method that contains bandwidth weights */ +#define MIN_METHOD_FOR_BW_WEIGHTS 9 + +/** Lowest consensus method that contains consensus params */ +#define MIN_METHOD_FOR_PARAMS 7 + +/** Lowest consensus method that generates microdescriptors */ +#define MIN_METHOD_FOR_MICRODESC 8 /* ===== * Voting * =====*/ +/* Overestimated. */ +#define MICRODESC_LINE_LEN 80 + /** Return a new string containing the string representation of the vote in * <b>v3_ns</b>, signed with our v3 signing key <b>private_signing_key</b>. * For v3 authorities. */ @@ -44,6 +89,7 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, uint32_t addr; routerlist_t *rl = router_get_routerlist(); char *version_lines = NULL; + int r; networkstatus_voter_info_t *voter; tor_assert(private_signing_key); @@ -70,20 +116,30 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, version_lines = tor_malloc(v_len); cp = version_lines; if (client_versions) { - tor_snprintf(cp, v_len-(cp-version_lines), + r = tor_snprintf(cp, v_len-(cp-version_lines), "client-versions %s\n", client_versions); + if (r < 0) { + log_err(LD_BUG, "Insufficient memory for client-versions line"); + tor_assert(0); + } cp += strlen(cp); } - if (server_versions) - tor_snprintf(cp, v_len-(cp-version_lines), + if (server_versions) { + r = tor_snprintf(cp, v_len-(cp-version_lines), "server-versions %s\n", server_versions); + if (r < 0) { + log_err(LD_BUG, "Insufficient memory for server-versions line"); + tor_assert(0); + } + } } else { version_lines = tor_strdup(""); } len = 8192; len += strlen(version_lines); - len += (RS_ENTRY_LEN)*smartlist_len(rl->routers); + len += (RS_ENTRY_LEN+MICRODESC_LINE_LEN)*smartlist_len(rl->routers); + len += strlen("\ndirectory-footer\n"); len += v3_ns->cert->cache_info.signed_descriptor_len; status = tor_malloc(len); @@ -93,17 +149,25 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, char fu[ISO_TIME_LEN+1]; char vu[ISO_TIME_LEN+1]; char *flags = smartlist_join_strings(v3_ns->known_flags, " ", 0, NULL); + char *params; authority_cert_t *cert = v3_ns->cert; + char *methods = + make_consensus_method_list(1, MAX_SUPPORTED_CONSENSUS_METHOD, " "); format_iso_time(published, v3_ns->published); format_iso_time(va, v3_ns->valid_after); format_iso_time(fu, v3_ns->fresh_until); format_iso_time(vu, v3_ns->valid_until); + if (v3_ns->net_params) + params = smartlist_join_strings(v3_ns->net_params, " ", 0, NULL); + else + params = tor_strdup(""); + tor_assert(cert); - tor_snprintf(status, len, + r = tor_snprintf(status, len, "network-status-version 3\n" "vote-status %s\n" - "consensus-methods 1 2 3 4 5\n" + "consensus-methods %s\n" "published %s\n" "valid-after %s\n" "fresh-until %s\n" @@ -111,24 +175,38 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, "voting-delay %d %d\n" "%s" /* versions */ "known-flags %s\n" + "params %s\n" "dir-source %s %s %s %s %d %d\n" "contact %s\n", v3_ns->type == NS_TYPE_VOTE ? "vote" : "opinion", + methods, published, va, fu, vu, v3_ns->vote_seconds, v3_ns->dist_seconds, version_lines, flags, + params, voter->nickname, fingerprint, voter->address, - ipaddr, voter->dir_port, voter->or_port, voter->contact); + ipaddr, voter->dir_port, voter->or_port, voter->contact); + if (r < 0) { + log_err(LD_BUG, "Insufficient memory for network status line"); + tor_assert(0); + } + + tor_free(params); tor_free(flags); + tor_free(methods); outp = status + strlen(status); endp = status + len; if (!tor_digest_is_zero(voter->legacy_id_digest)) { char fpbuf[HEX_DIGEST_LEN+1]; base16_encode(fpbuf, sizeof(fpbuf), voter->legacy_id_digest, DIGEST_LEN); - tor_snprintf(outp, endp-outp, "legacy-dir-key %s\n", fpbuf); + r = tor_snprintf(outp, endp-outp, "legacy-dir-key %s\n", fpbuf); + if (r < 0) { + log_err(LD_BUG, "Insufficient memory for legacy-dir-key line"); + tor_assert(0); + } outp += strlen(outp); } @@ -139,15 +217,32 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, outp += cert->cache_info.signed_descriptor_len; } - SMARTLIST_FOREACH(v3_ns->routerstatus_list, vote_routerstatus_t *, vrs, - { + SMARTLIST_FOREACH_BEGIN(v3_ns->routerstatus_list, vote_routerstatus_t *, + vrs) { + vote_microdesc_hash_t *h; if (routerstatus_format_entry(outp, endp-outp, &vrs->status, - vrs->version, 0, 0) < 0) { + vrs->version, NS_V3_VOTE) < 0) { log_warn(LD_BUG, "Unable to print router status."); goto err; } outp += strlen(outp); - }); + + for (h = vrs->microdesc; h; h = h->next) { + size_t mlen = strlen(h->microdesc_hash_line); + if (outp+mlen >= endp) { + log_warn(LD_BUG, "Can't fit microdesc line in vote."); + } + memcpy(outp, h->microdesc_hash_line, mlen+1); + outp += strlen(outp); + } + } SMARTLIST_FOREACH_END(vrs); + + r = tor_snprintf(outp, endp-outp, "directory-footer\n"); + if (r < 0) { + log_err(LD_BUG, "Insufficient memory for directory-footer line"); + tor_assert(0); + } + outp += strlen(outp); { char signing_key_fingerprint[FINGERPRINT_LEN+1]; @@ -170,10 +265,10 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key, outp += strlen(outp); } - if (router_get_networkstatus_v3_hash(status, digest)<0) + if (router_get_networkstatus_v3_hash(status, digest, DIGEST_SHA1)<0) goto err; note_crypto_pk_op(SIGN_DIR); - if (router_append_dirobj_signature(outp,endp-outp,digest, + if (router_append_dirobj_signature(outp,endp-outp,digest, DIGEST_LEN, private_signing_key)<0) { log_warn(LD_BUG, "Unable to sign networkstatus vote."); goto err; @@ -216,6 +311,20 @@ get_voter(const networkstatus_t *vote) return smartlist_get(vote->voters, 0); } +/** Return the signature made by <b>voter</b> using the algorithm + * <b>alg</b>, or NULL if none is found. */ +document_signature_t * +voter_get_sig_by_algorithm(const networkstatus_voter_info_t *voter, + digest_algorithm_t alg) +{ + if (!voter->sigs) + return NULL; + SMARTLIST_FOREACH(voter->sigs, document_signature_t *, sig, + if (sig->alg == alg) + return sig); + return NULL; +} + /** Temporary structure used in constructing a list of dir-source entries * for a consensus. One of these is generated for every vote, and one more * for every legacy key in each vote. */ @@ -275,34 +384,8 @@ get_frequent_members(smartlist_t *out, smartlist_t *in, int min) /** Given a sorted list of strings <b>lst</b>, return the member that appears * most. Break ties in favor of later-occurring members. */ -static const char * -get_most_frequent_member(smartlist_t *lst) -{ - const char *most_frequent = NULL; - int most_frequent_count = 0; - - const char *cur = NULL; - int count = 0; - - SMARTLIST_FOREACH(lst, const char *, s, - { - if (cur && !strcmp(s, cur)) { - ++count; - } else { - if (count >= most_frequent_count) { - most_frequent = cur; - most_frequent_count = count; - } - cur = s; - count = 1; - } - }); - if (count >= most_frequent_count) { - most_frequent = cur; - most_frequent_count = count; - } - return most_frequent; -} +#define get_most_frequent_member(lst) \ + smartlist_get_most_frequent_string(lst) /** Return 0 if and only if <b>a</b> and <b>b</b> are routerstatuses * that come from the same routerinfo, with the same derived elements. @@ -344,7 +427,8 @@ _compare_vote_rs(const void **_a, const void **_b) * in favor of smaller descriptor digest. */ static vote_routerstatus_t * -compute_routerstatus_consensus(smartlist_t *votes) +compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, + char *microdesc_digest256_out) { vote_routerstatus_t *most = NULL, *cur = NULL; int most_n = 0, cur_n = 0; @@ -360,9 +444,9 @@ compute_routerstatus_consensus(smartlist_t *votes) if (cur && !compare_vote_rs(cur, rs)) { ++cur_n; } else { - if (cur_n > most_n || - (cur && cur_n == most_n && - cur->status.published_on > most_published)) { + if (cur && (cur_n > most_n || + (cur_n == most_n && + cur->status.published_on > most_published))) { most = cur; most_n = cur_n; most_published = cur->status.published_on; @@ -380,18 +464,45 @@ compute_routerstatus_consensus(smartlist_t *votes) } tor_assert(most); + + if (consensus_method >= MIN_METHOD_FOR_MICRODESC && + microdesc_digest256_out) { + smartlist_t *digests = smartlist_create(); + const char *best_microdesc_digest; + SMARTLIST_FOREACH_BEGIN(votes, vote_routerstatus_t *, rs) { + char d[DIGEST256_LEN]; + if (compare_vote_rs(rs, most)) + continue; + if (!vote_routerstatus_find_microdesc_hash(d, rs, consensus_method, + DIGEST_SHA256)) + smartlist_add(digests, tor_memdup(d, sizeof(d))); + } SMARTLIST_FOREACH_END(rs); + smartlist_sort_digests256(digests); + best_microdesc_digest = smartlist_get_most_frequent_digest256(digests); + if (best_microdesc_digest) + memcpy(microdesc_digest256_out, best_microdesc_digest, DIGEST256_LEN); + SMARTLIST_FOREACH(digests, char *, cp, tor_free(cp)); + smartlist_free(digests); + } + return most; } -/** Given a list of strings in <b>lst</b>, set the DIGEST_LEN-byte digest at - * <b>digest_out</b> to the hash of the concatenation of those strings. */ +/** Given a list of strings in <b>lst</b>, set the <b>len_out</b>-byte digest + * at <b>digest_out</b> to the hash of the concatenation of those strings, + * computed with the algorithm <b>alg</b>. */ static void -hash_list_members(char *digest_out, smartlist_t *lst) +hash_list_members(char *digest_out, size_t len_out, + smartlist_t *lst, digest_algorithm_t alg) { - crypto_digest_env_t *d = crypto_new_digest_env(); + crypto_digest_env_t *d; + if (alg == DIGEST_SHA1) + d = crypto_new_digest_env(); + else + d = crypto_new_digest256_env(alg); SMARTLIST_FOREACH(lst, const char *, cp, crypto_digest_add_bytes(d, cp, strlen(cp))); - crypto_digest_get_digest(d, digest_out, DIGEST_LEN); + crypto_digest_get_digest(d, digest_out, len_out); crypto_free_digest_env(d); } @@ -455,7 +566,31 @@ compute_consensus_method(smartlist_t *votes) static int consensus_method_is_supported(int method) { - return (method >= 1) && (method <= 5); + return (method >= 1) && (method <= MAX_SUPPORTED_CONSENSUS_METHOD); +} + +/** Return a newly allocated string holding the numbers between low and high + * (inclusive) that are supported consensus methods. */ +static char * +make_consensus_method_list(int low, int high, const char *separator) +{ + char *list; + + char b[32]; + int i; + smartlist_t *lst; + lst = smartlist_create(); + for (i = low; i <= high; ++i) { + if (!consensus_method_is_supported(i)) + continue; + tor_snprintf(b, sizeof(b), "%d", i); + smartlist_add(lst, tor_strdup(b)); + } + list = smartlist_join_strings(lst, separator, 0, NULL); + tor_assert(list); + SMARTLIST_FOREACH(lst, char *, cp, tor_free(cp)); + smartlist_free(lst); + return list; } /** Helper: given <b>lst</b>, a list of version strings such that every @@ -475,6 +610,728 @@ compute_consensus_versions_list(smartlist_t *lst, int n_versioning) return result; } +/** Helper: given a list of valid networkstatus_t, return a new string + * containing the contents of the consensus network parameter set. + */ +/* private */ char * +dirvote_compute_params(smartlist_t *votes) +{ + int i; + int32_t *vals; + + int cur_param_len; + const char *cur_param; + const char *eq; + char *result; + + const int n_votes = smartlist_len(votes); + smartlist_t *output; + smartlist_t *param_list = smartlist_create(); + + /* We require that the parameter lists in the votes are well-formed: that + is, that their keywords are unique and sorted, and that their values are + between INT32_MIN and INT32_MAX inclusive. This should be guaranteed by + the parsing code. */ + + vals = tor_malloc(sizeof(int)*n_votes); + + SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { + if (!v->net_params) + continue; + smartlist_add_all(param_list, v->net_params); + } SMARTLIST_FOREACH_END(v); + + if (smartlist_len(param_list) == 0) { + tor_free(vals); + smartlist_free(param_list); + return NULL; + } + + smartlist_sort_strings(param_list); + i = 0; + cur_param = smartlist_get(param_list, 0); + eq = strchr(cur_param, '='); + tor_assert(eq); + cur_param_len = (int)(eq+1 - cur_param); + + output = smartlist_create(); + + SMARTLIST_FOREACH_BEGIN(param_list, const char *, param) { + const char *next_param; + int ok=0; + eq = strchr(param, '='); + tor_assert(i<n_votes); + vals[i++] = (int32_t) + tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL); + tor_assert(ok); + + if (param_sl_idx+1 == smartlist_len(param_list)) + next_param = NULL; + else + next_param = smartlist_get(param_list, param_sl_idx+1); + if (!next_param || strncmp(next_param, param, cur_param_len)) { + /* We've reached the end of a series. */ + int32_t median = median_int32(vals, i); + char *out_string = tor_malloc(64+cur_param_len); + memcpy(out_string, param, cur_param_len); + tor_snprintf(out_string+cur_param_len,64, "%ld", (long)median); + smartlist_add(output, out_string); + + i = 0; + if (next_param) { + eq = strchr(next_param, '='); + cur_param_len = (int)(eq+1 - next_param); + } + } + } SMARTLIST_FOREACH_END(param); + + result = smartlist_join_strings(output, " ", 0, NULL); + SMARTLIST_FOREACH(output, char *, cp, tor_free(cp)); + smartlist_free(output); + smartlist_free(param_list); + tor_free(vals); + return result; +} + +#define RANGE_CHECK(a,b,c,d,e,f,g,mx) \ + ((a) >= 0 && (a) <= (mx) && (b) >= 0 && (b) <= (mx) && \ + (c) >= 0 && (c) <= (mx) && (d) >= 0 && (d) <= (mx) && \ + (e) >= 0 && (e) <= (mx) && (f) >= 0 && (f) <= (mx) && \ + (g) >= 0 && (g) <= (mx)) + +#define CHECK_EQ(a, b, margin) \ + ((a)-(b) >= 0 ? (a)-(b) <= (margin) : (b)-(a) <= (margin)) + +typedef enum { + BW_WEIGHTS_NO_ERROR = 0, + BW_WEIGHTS_RANGE_ERROR = 1, + BW_WEIGHTS_SUMG_ERROR = 2, + BW_WEIGHTS_SUME_ERROR = 3, + BW_WEIGHTS_SUMD_ERROR = 4, + BW_WEIGHTS_BALANCE_MID_ERROR = 5, + BW_WEIGHTS_BALANCE_EG_ERROR = 6 +} bw_weights_error_t; + +/** + * Verify that any weightings satisfy the balanced formulas. + */ +static bw_weights_error_t +networkstatus_check_weights(int64_t Wgg, int64_t Wgd, int64_t Wmg, + int64_t Wme, int64_t Wmd, int64_t Wee, + int64_t Wed, int64_t scale, int64_t G, + int64_t M, int64_t E, int64_t D, int64_t T, + int64_t margin, int do_balance) { + bw_weights_error_t berr = BW_WEIGHTS_NO_ERROR; + + // Wed + Wmd + Wgd == 1 + if (!CHECK_EQ(Wed + Wmd + Wgd, scale, margin)) { + berr = BW_WEIGHTS_SUMD_ERROR; + goto out; + } + + // Wmg + Wgg == 1 + if (!CHECK_EQ(Wmg + Wgg, scale, margin)) { + berr = BW_WEIGHTS_SUMG_ERROR; + goto out; + } + + // Wme + Wee == 1 + if (!CHECK_EQ(Wme + Wee, scale, margin)) { + berr = BW_WEIGHTS_SUME_ERROR; + goto out; + } + + // Verify weights within range 0->1 + if (!RANGE_CHECK(Wgg, Wgd, Wmg, Wme, Wmd, Wed, Wee, scale)) { + berr = BW_WEIGHTS_RANGE_ERROR; + goto out; + } + + if (do_balance) { + // Wgg*G + Wgd*D == Wee*E + Wed*D, already scaled + if (!CHECK_EQ(Wgg*G + Wgd*D, Wee*E + Wed*D, (margin*T)/3)) { + berr = BW_WEIGHTS_BALANCE_EG_ERROR; + goto out; + } + + // Wgg*G + Wgd*D == M*scale + Wmd*D + Wme*E + Wmg*G, already scaled + if (!CHECK_EQ(Wgg*G + Wgd*D, M*scale + Wmd*D + Wme*E + Wmg*G, + (margin*T)/3)) { + berr = BW_WEIGHTS_BALANCE_MID_ERROR; + goto out; + } + } + + out: + if (berr) { + log_info(LD_DIR, + "Bw weight mismatch %d. G="I64_FORMAT" M="I64_FORMAT + " E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT + " Wmd=%d Wme=%d Wmg=%d Wed=%d Wee=%d" + " Wgd=%d Wgg=%d Wme=%d Wmg=%d", + berr, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + (int)Wmd, (int)Wme, (int)Wmg, (int)Wed, (int)Wee, + (int)Wgd, (int)Wgg, (int)Wme, (int)Wmg); + } + + return berr; +} + +/** + * This function computes the bandwidth weights for consensus method 10. + * + * It returns true if weights could be computed, false otherwise. + */ +static int +networkstatus_compute_bw_weights_v10(smartlist_t *chunks, int64_t G, + int64_t M, int64_t E, int64_t D, + int64_t T, int64_t weight_scale) +{ + bw_weights_error_t berr = 0; + int64_t Wgg = -1, Wgd = -1; + int64_t Wmg = -1, Wme = -1, Wmd = -1; + int64_t Wed = -1, Wee = -1; + const char *casename; + char buf[512]; + int r; + + if (G <= 0 || M <= 0 || E <= 0 || D <= 0) { + log_warn(LD_DIR, "Consensus with empty bandwidth: " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + return 0; + } + + /* + * Computed from cases in 3.4.3 of dir-spec.txt + * + * 1. Neither are scarce + * 2. Both Guard and Exit are scarce + * a. R+D <= S + * b. R+D > S + * 3. One of Guard or Exit is scarce + * a. S+D < T/3 + * b. S+D >= T/3 + */ + if (3*E >= T && 3*G >= T) { // E >= T/3 && G >= T/3 + /* Case 1: Neither are scarce. */ + casename = "Case 1 (Wgd=Wmd=Wed)"; + Wgd = weight_scale/3; + Wed = weight_scale/3; + Wmd = weight_scale/3; + Wee = (weight_scale*(E+G+M))/(3*E); + Wme = weight_scale - Wee; + Wmg = (weight_scale*(2*G-E-M))/(3*G); + Wgg = weight_scale - Wmg; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed, + weight_scale, G, M, E, D, T, 10, 1); + + if (berr) { + log_warn(LD_DIR, + "Bw Weights error %d for %s v10. G="I64_FORMAT" M="I64_FORMAT + " E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT + " Wmd=%d Wme=%d Wmg=%d Wed=%d Wee=%d" + " Wgd=%d Wgg=%d Wme=%d Wmg=%d weight_scale=%d", + berr, casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + (int)Wmd, (int)Wme, (int)Wmg, (int)Wed, (int)Wee, + (int)Wgd, (int)Wgg, (int)Wme, (int)Wmg, (int)weight_scale); + return 0; + } + } else if (3*E < T && 3*G < T) { // E < T/3 && G < T/3 + int64_t R = MIN(E, G); + int64_t S = MAX(E, G); + /* + * Case 2: Both Guards and Exits are scarce + * Balance D between E and G, depending upon + * D capacity and scarcity. + */ + if (R+D < S) { // Subcase a + Wgg = weight_scale; + Wee = weight_scale; + Wmg = 0; + Wme = 0; + Wmd = 0; + if (E < G) { + casename = "Case 2a (E scarce)"; + Wed = weight_scale; + Wgd = 0; + } else { /* E >= G */ + casename = "Case 2a (G scarce)"; + Wed = 0; + Wgd = weight_scale; + } + } else { // Subcase b: R+D >= S + casename = "Case 2b1 (Wgg=1, Wmd=Wgd)"; + Wee = (weight_scale*(E - G + M))/E; + Wed = (weight_scale*(D - 2*E + 4*G - 2*M))/(3*D); + Wme = (weight_scale*(G-M))/E; + Wmg = 0; + Wgg = weight_scale; + Wmd = (weight_scale - Wed)/2; + Wgd = (weight_scale - Wed)/2; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed, + weight_scale, G, M, E, D, T, 10, 1); + + if (berr) { + casename = "Case 2b2 (Wgg=1, Wee=1)"; + Wgg = weight_scale; + Wee = weight_scale; + Wed = (weight_scale*(D - 2*E + G + M))/(3*D); + Wmd = (weight_scale*(D - 2*M + G + E))/(3*D); + Wme = 0; + Wmg = 0; + + if (Wmd < 0) { // Can happen if M > T/3 + casename = "Case 2b3 (Wmd=0)"; + Wmd = 0; + log_warn(LD_DIR, + "Too much Middle bandwidth on the network to calculate " + "balanced bandwidth-weights. Consider increasing the " + "number of Guard nodes by lowering the requirements."); + } + Wgd = weight_scale - Wed - Wmd; + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, + Wed, weight_scale, G, M, E, D, T, 10, 1); + } + if (berr != BW_WEIGHTS_NO_ERROR && + berr != BW_WEIGHTS_BALANCE_MID_ERROR) { + log_warn(LD_DIR, + "Bw Weights error %d for %s v10. G="I64_FORMAT" M="I64_FORMAT + " E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT + " Wmd=%d Wme=%d Wmg=%d Wed=%d Wee=%d" + " Wgd=%d Wgg=%d Wme=%d Wmg=%d weight_scale=%d", + berr, casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + (int)Wmd, (int)Wme, (int)Wmg, (int)Wed, (int)Wee, + (int)Wgd, (int)Wgg, (int)Wme, (int)Wmg, (int)weight_scale); + return 0; + } + } + } else { // if (E < T/3 || G < T/3) { + int64_t S = MIN(E, G); + // Case 3: Exactly one of Guard or Exit is scarce + if (!(3*E < T || 3*G < T) || !(3*G >= T || 3*E >= T)) { + log_warn(LD_BUG, + "Bw-Weights Case 3 v10 but with G="I64_FORMAT" M=" + I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + } + + if (3*(S+D) < T) { // Subcase a: S+D < T/3 + if (G < E) { + casename = "Case 3a (G scarce)"; + Wgg = Wgd = weight_scale; + Wmd = Wed = Wmg = 0; + // Minor subcase, if E is more scarce than M, + // keep its bandwidth in place. + if (E < M) Wme = 0; + else Wme = (weight_scale*(E-M))/(2*E); + Wee = weight_scale-Wme; + } else { // G >= E + casename = "Case 3a (E scarce)"; + Wee = Wed = weight_scale; + Wmd = Wgd = Wme = 0; + // Minor subcase, if G is more scarce than M, + // keep its bandwidth in place. + if (G < M) Wmg = 0; + else Wmg = (weight_scale*(G-M))/(2*G); + Wgg = weight_scale-Wmg; + } + } else { // Subcase b: S+D >= T/3 + // D != 0 because S+D >= T/3 + if (G < E) { + casename = "Case 3bg (G scarce, Wgg=1, Wmd == Wed)"; + Wgg = weight_scale; + Wgd = (weight_scale*(D - 2*G + E + M))/(3*D); + Wmg = 0; + Wee = (weight_scale*(E+M))/(2*E); + Wme = weight_scale - Wee; + Wmd = (weight_scale - Wgd)/2; + Wed = (weight_scale - Wgd)/2; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, + Wed, weight_scale, G, M, E, D, T, 10, 1); + } else { // G >= E + casename = "Case 3be (E scarce, Wee=1, Wmd == Wgd)"; + Wee = weight_scale; + Wed = (weight_scale*(D - 2*E + G + M))/(3*D); + Wme = 0; + Wgg = (weight_scale*(G+M))/(2*G); + Wmg = weight_scale - Wgg; + Wmd = (weight_scale - Wed)/2; + Wgd = (weight_scale - Wed)/2; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, + Wed, weight_scale, G, M, E, D, T, 10, 1); + } + if (berr) { + log_warn(LD_DIR, + "Bw Weights error %d for %s v10. G="I64_FORMAT" M="I64_FORMAT + " E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT + " Wmd=%d Wme=%d Wmg=%d Wed=%d Wee=%d" + " Wgd=%d Wgg=%d Wme=%d Wmg=%d weight_scale=%d", + berr, casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + (int)Wmd, (int)Wme, (int)Wmg, (int)Wed, (int)Wee, + (int)Wgd, (int)Wgg, (int)Wme, (int)Wmg, (int)weight_scale); + return 0; + } + } + } + + /* We cast down the weights to 32 bit ints on the assumption that + * weight_scale is ~= 10000. We need to ensure a rogue authority + * doesn't break this assumption to rig our weights */ + tor_assert(0 < weight_scale && weight_scale < INT32_MAX); + + /* + * Provide Wgm=Wgg, Wmm=1, Wem=Wee, Weg=Wed. May later determine + * that middle nodes need different bandwidth weights for dirport traffic, + * or that weird exit policies need special weight, or that bridges + * need special weight. + * + * NOTE: This list is sorted. + */ + r = tor_snprintf(buf, sizeof(buf), + "bandwidth-weights Wbd=%d Wbe=%d Wbg=%d Wbm=%d " + "Wdb=%d " + "Web=%d Wed=%d Wee=%d Weg=%d Wem=%d " + "Wgb=%d Wgd=%d Wgg=%d Wgm=%d " + "Wmb=%d Wmd=%d Wme=%d Wmg=%d Wmm=%d\n", + (int)Wmd, (int)Wme, (int)Wmg, (int)weight_scale, + (int)weight_scale, + (int)weight_scale, (int)Wed, (int)Wee, (int)Wed, (int)Wee, + (int)weight_scale, (int)Wgd, (int)Wgg, (int)Wgg, + (int)weight_scale, (int)Wmd, (int)Wme, (int)Wmg, (int)weight_scale); + if (r<0) { + log_warn(LD_BUG, + "Not enough space in buffer for bandwidth-weights line."); + *buf = '\0'; + return 0; + } + smartlist_add(chunks, tor_strdup(buf)); + + log_notice(LD_CIRC, "Computed bandwidth weights for %s with v10: " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + return 1; +} +/** + * This function computes the bandwidth weights for consensus method 9. + * + * It has been obsoleted in favor of consensus method 10. + */ +static void +networkstatus_compute_bw_weights_v9(smartlist_t *chunks, int64_t G, int64_t M, + int64_t E, int64_t D, int64_t T, + int64_t weight_scale) +{ + int64_t Wgg = -1, Wgd = -1; + int64_t Wmg = -1, Wme = -1, Wmd = -1; + int64_t Wed = -1, Wee = -1; + const char *casename; + char buf[512]; + int r; + + if (G <= 0 || M <= 0 || E <= 0 || D <= 0) { + log_warn(LD_DIR, "Consensus with empty bandwidth: " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + return; + } + + /* + * Computed from cases in 3.4.3 of dir-spec.txt + * + * 1. Neither are scarce + * 2. Both Guard and Exit are scarce + * a. R+D <= S + * b. R+D > S + * 3. One of Guard or Exit is scarce + * a. S+D < T/3 + * b. S+D >= T/3 + */ + if (3*E >= T && 3*G >= T) { // E >= T/3 && G >= T/3 + bw_weights_error_t berr = 0; + /* Case 1: Neither are scarce. + * + * Attempt to ensure that we have a large amount of exit bandwidth + * in the middle position. + */ + casename = "Case 1 (Wme*E = Wmd*D)"; + Wgg = (weight_scale*(D+E+G+M))/(3*G); + if (D==0) Wmd = 0; + else Wmd = (weight_scale*(2*D + 2*E - G - M))/(6*D); + Wme = (weight_scale*(2*D + 2*E - G - M))/(6*E); + Wee = (weight_scale*(-2*D + 4*E + G + M))/(6*E); + Wgd = 0; + Wmg = weight_scale - Wgg; + Wed = weight_scale - Wmd; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed, + weight_scale, G, M, E, D, T, 10, 1); + + if (berr) { + log_warn(LD_DIR, "Bw Weights error %d for case %s. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT, + berr, casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + } + } else if (3*E < T && 3*G < T) { // E < T/3 && G < T/3 + int64_t R = MIN(E, G); + int64_t S = MAX(E, G); + /* + * Case 2: Both Guards and Exits are scarce + * Balance D between E and G, depending upon + * D capacity and scarcity. + */ + if (R+D < S) { // Subcase a + Wgg = weight_scale; + Wee = weight_scale; + Wmg = 0; + Wme = 0; + Wmd = 0; + if (E < G) { + casename = "Case 2a (E scarce)"; + Wed = weight_scale; + Wgd = 0; + } else { /* E >= G */ + casename = "Case 2a (G scarce)"; + Wed = 0; + Wgd = weight_scale; + } + } else { // Subcase b: R+D > S + bw_weights_error_t berr = 0; + casename = "Case 2b (Wme*E == Wmd*D)"; + if (D != 0) { + Wgg = weight_scale; + Wgd = (weight_scale*(D + E - 2*G + M))/(3*D); // T/3 >= G (Ok) + Wmd = (weight_scale*(D + E + G - 2*M))/(6*D); // T/3 >= M + Wme = (weight_scale*(D + E + G - 2*M))/(6*E); + Wee = (weight_scale*(-D + 5*E - G + 2*M))/(6*E); // 2E+M >= T/3 + Wmg = 0; + Wed = weight_scale - Wgd - Wmd; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed, + weight_scale, G, M, E, D, T, 10, 1); + } + + if (D == 0 || berr) { // Can happen if M > T/3 + casename = "Case 2b (E=G)"; + Wgg = weight_scale; + Wee = weight_scale; + Wmg = 0; + Wme = 0; + Wmd = 0; + if (D == 0) Wgd = 0; + else Wgd = (weight_scale*(D+E-G))/(2*D); + Wed = weight_scale - Wgd; + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, + Wed, weight_scale, G, M, E, D, T, 10, 1); + } + if (berr != BW_WEIGHTS_NO_ERROR && + berr != BW_WEIGHTS_BALANCE_MID_ERROR) { + log_warn(LD_DIR, "Bw Weights error %d for case %s. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT, + berr, casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + } + } + } else { // if (E < T/3 || G < T/3) { + int64_t S = MIN(E, G); + // Case 3: Exactly one of Guard or Exit is scarce + if (!(3*E < T || 3*G < T) || !(3*G >= T || 3*E >= T)) { + log_warn(LD_BUG, + "Bw-Weights Case 3 but with G="I64_FORMAT" M=" + I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + } + + if (3*(S+D) < T) { // Subcase a: S+D < T/3 + if (G < E) { + casename = "Case 3a (G scarce)"; + Wgg = Wgd = weight_scale; + Wmd = Wed = Wmg = 0; + // Minor subcase, if E is more scarce than M, + // keep its bandwidth in place. + if (E < M) Wme = 0; + else Wme = (weight_scale*(E-M))/(2*E); + Wee = weight_scale-Wme; + } else { // G >= E + casename = "Case 3a (E scarce)"; + Wee = Wed = weight_scale; + Wmd = Wgd = Wme = 0; + // Minor subcase, if G is more scarce than M, + // keep its bandwidth in place. + if (G < M) Wmg = 0; + else Wmg = (weight_scale*(G-M))/(2*G); + Wgg = weight_scale-Wmg; + } + } else { // Subcase b: S+D >= T/3 + bw_weights_error_t berr = 0; + // D != 0 because S+D >= T/3 + if (G < E) { + casename = "Case 3b (G scarce, Wme*E == Wmd*D)"; + Wgd = (weight_scale*(D + E - 2*G + M))/(3*D); + Wmd = (weight_scale*(D + E + G - 2*M))/(6*D); + Wme = (weight_scale*(D + E + G - 2*M))/(6*E); + Wee = (weight_scale*(-D + 5*E - G + 2*M))/(6*E); + Wgg = weight_scale; + Wmg = 0; + Wed = weight_scale - Wgd - Wmd; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, + Wed, weight_scale, G, M, E, D, T, 10, 1); + } else { // G >= E + casename = "Case 3b (E scarce, Wme*E == Wmd*D)"; + Wgg = (weight_scale*(D + E + G + M))/(3*G); + Wmd = (weight_scale*(2*D + 2*E - G - M))/(6*D); + Wme = (weight_scale*(2*D + 2*E - G - M))/(6*E); + Wee = (weight_scale*(-2*D + 4*E + G + M))/(6*E); + Wgd = 0; + Wmg = weight_scale - Wgg; + Wed = weight_scale - Wmd; + + berr = networkstatus_check_weights(Wgg, Wgd, Wmg, Wme, Wmd, Wee, + Wed, weight_scale, G, M, E, D, T, 10, 1); + } + if (berr) { + log_warn(LD_DIR, "Bw Weights error %d for case %s. " + "G="I64_FORMAT" M="I64_FORMAT + " E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT, + berr, casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + } + } + } + + /* We cast down the weights to 32 bit ints on the assumption that + * weight_scale is ~= 10000. We need to ensure a rogue authority + * doesn't break this assumption to rig our weights */ + tor_assert(0 < weight_scale && weight_scale < INT32_MAX); + + if (Wgg < 0 || Wgg > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wgg="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wgg), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + + Wgg = MAX(MIN(Wgg, weight_scale), 0); + } + if (Wgd < 0 || Wgd > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wgd="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wgd), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + Wgd = MAX(MIN(Wgd, weight_scale), 0); + } + if (Wmg < 0 || Wmg > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wmg="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wmg), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + Wmg = MAX(MIN(Wmg, weight_scale), 0); + } + if (Wme < 0 || Wme > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wme="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wme), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + Wme = MAX(MIN(Wme, weight_scale), 0); + } + if (Wmd < 0 || Wmd > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wmd="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wmd), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + Wmd = MAX(MIN(Wmd, weight_scale), 0); + } + if (Wee < 0 || Wee > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wee="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wee), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + Wee = MAX(MIN(Wee, weight_scale), 0); + } + if (Wed < 0 || Wed > weight_scale) { + log_warn(LD_DIR, "Bw %s: Wed="I64_FORMAT"! G="I64_FORMAT + " M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, I64_PRINTF_ARG(Wed), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); + Wed = MAX(MIN(Wed, weight_scale), 0); + } + + // Add consensus weight keywords + smartlist_add(chunks, tor_strdup("bandwidth-weights ")); + /* + * Provide Wgm=Wgg, Wmm=1, Wem=Wee, Weg=Wed. May later determine + * that middle nodes need different bandwidth weights for dirport traffic, + * or that weird exit policies need special weight, or that bridges + * need special weight. + * + * NOTE: This list is sorted. + */ + r = tor_snprintf(buf, sizeof(buf), + "Wbd=%d Wbe=%d Wbg=%d Wbm=%d " + "Wdb=%d " + "Web=%d Wed=%d Wee=%d Weg=%d Wem=%d " + "Wgb=%d Wgd=%d Wgg=%d Wgm=%d " + "Wmb=%d Wmd=%d Wme=%d Wmg=%d Wmm=%d\n", + (int)Wmd, (int)Wme, (int)Wmg, (int)weight_scale, + (int)weight_scale, + (int)weight_scale, (int)Wed, (int)Wee, (int)Wed, (int)Wee, + (int)weight_scale, (int)Wgd, (int)Wgg, (int)Wgg, + (int)weight_scale, (int)Wmd, (int)Wme, (int)Wmg, (int)weight_scale); + if (r<0) { + log_warn(LD_BUG, + "Not enough space in buffer for bandwidth-weights line."); + *buf = '\0'; + } + smartlist_add(chunks, tor_strdup(buf)); + log_notice(LD_CIRC, "Computed bandwidth weights for %s with v9: " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT, + casename, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T)); +} + /** Given a list of vote networkstatus_t in <b>votes</b>, our public * authority <b>identity_key</b>, our private authority <b>signing_key</b>, * and the number of <b>total_authorities</b> that we believe exist in our @@ -489,18 +1346,27 @@ networkstatus_compute_consensus(smartlist_t *votes, crypto_pk_env_t *identity_key, crypto_pk_env_t *signing_key, const char *legacy_id_key_digest, - crypto_pk_env_t *legacy_signing_key) + crypto_pk_env_t *legacy_signing_key, + consensus_flavor_t flavor) { smartlist_t *chunks; char *result = NULL; int consensus_method; - time_t valid_after, fresh_until, valid_until; int vote_seconds, dist_seconds; char *client_versions = NULL, *server_versions = NULL; smartlist_t *flags; + const char *flavor_name; + int64_t G=0, M=0, E=0, D=0, T=0; /* For bandwidth weights */ + const routerstatus_format_type_t rs_format = + flavor == FLAV_NS ? NS_V3_CONSENSUS : NS_V3_CONSENSUS_MICRODESC; + char *params = NULL; + int added_weights = 0; + tor_assert(flavor == FLAV_NS || flavor == FLAV_MICRODESC); tor_assert(total_authorities >= smartlist_len(votes)); + flavor_name = networkstatus_get_flavor_name(flavor); + if (!smartlist_len(votes)) { log_warn(LD_DIR, "Can't compute a consensus from no votes."); return NULL; @@ -593,7 +1459,7 @@ networkstatus_compute_consensus(smartlist_t *votes, chunks = smartlist_create(); { - char buf[1024]; + char *buf=NULL; char va_buf[ISO_TIME_LEN+1], fu_buf[ISO_TIME_LEN+1], vu_buf[ISO_TIME_LEN+1]; char *flaglist; @@ -602,16 +1468,20 @@ networkstatus_compute_consensus(smartlist_t *votes, format_iso_time(vu_buf, valid_until); flaglist = smartlist_join_strings(flags, " ", 0, NULL); - smartlist_add(chunks, tor_strdup("network-status-version 3\n" - "vote-status consensus\n")); + tor_asprintf(&buf, "network-status-version 3%s%s\n" + "vote-status consensus\n", + flavor == FLAV_NS ? "" : " ", + flavor == FLAV_NS ? "" : flavor_name); + + smartlist_add(chunks, buf); if (consensus_method >= 2) { - tor_snprintf(buf, sizeof(buf), "consensus-method %d\n", + tor_asprintf(&buf, "consensus-method %d\n", consensus_method); - smartlist_add(chunks, tor_strdup(buf)); + smartlist_add(chunks, buf); } - tor_snprintf(buf, sizeof(buf), + tor_asprintf(&buf, "valid-after %s\n" "fresh-until %s\n" "valid-until %s\n" @@ -622,18 +1492,26 @@ networkstatus_compute_consensus(smartlist_t *votes, va_buf, fu_buf, vu_buf, vote_seconds, dist_seconds, client_versions, server_versions, flaglist); - smartlist_add(chunks, tor_strdup(buf)); + smartlist_add(chunks, buf); tor_free(flaglist); } + if (consensus_method >= MIN_METHOD_FOR_PARAMS) { + params = dirvote_compute_params(votes); + if (params) { + smartlist_add(chunks, tor_strdup("params ")); + smartlist_add(chunks, params); + smartlist_add(chunks, tor_strdup("\n")); + } + } + /* Sort the votes. */ smartlist_sort(votes, _compare_votes_by_authority_id); /* Add the authority sections. */ { smartlist_t *dir_sources = smartlist_create(); - SMARTLIST_FOREACH(votes, networkstatus_t *, v, - { + SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { dir_src_ent_t *e = tor_malloc_zero(sizeof(dir_src_ent_t)); e->v = v; e->digest = get_voter(v)->identity_digest; @@ -647,18 +1525,17 @@ networkstatus_compute_consensus(smartlist_t *votes, e_legacy->is_legacy = 1; smartlist_add(dir_sources, e_legacy); } - }); + } SMARTLIST_FOREACH_END(v); smartlist_sort(dir_sources, _compare_dir_src_ents_by_authority_id); - SMARTLIST_FOREACH(dir_sources, const dir_src_ent_t *, e, - { - char buf[1024]; + SMARTLIST_FOREACH_BEGIN(dir_sources, const dir_src_ent_t *, e) { struct in_addr in; char ip[INET_NTOA_BUF_LEN]; char fingerprint[HEX_DIGEST_LEN+1]; char votedigest[HEX_DIGEST_LEN+1]; networkstatus_t *v = e->v; networkstatus_voter_info_t *voter = get_voter(v); + char *buf = NULL; if (e->is_legacy) tor_assert(consensus_method >= 2); @@ -669,22 +1546,22 @@ networkstatus_compute_consensus(smartlist_t *votes, base16_encode(votedigest, sizeof(votedigest), voter->vote_digest, DIGEST_LEN); - tor_snprintf(buf, sizeof(buf), + tor_asprintf(&buf, "dir-source %s%s %s %s %s %d %d\n", voter->nickname, e->is_legacy ? "-legacy" : "", fingerprint, voter->address, ip, voter->dir_port, voter->or_port); - smartlist_add(chunks, tor_strdup(buf)); + smartlist_add(chunks, buf); if (! e->is_legacy) { - tor_snprintf(buf, sizeof(buf), + tor_asprintf(&buf, "contact %s\n" "vote-digest %s\n", voter->contact, votedigest); - smartlist_add(chunks, tor_strdup(buf)); + smartlist_add(chunks, buf); } - }); + } SMARTLIST_FOREACH_END(e); SMARTLIST_FOREACH(dir_sources, dir_src_ent_t *, e, tor_free(e)); smartlist_free(dir_sources); } @@ -701,7 +1578,10 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_t *versions = smartlist_create(); smartlist_t *exitsummaries = smartlist_create(); uint32_t *bandwidths = tor_malloc(sizeof(uint32_t) * smartlist_len(votes)); + uint32_t *measured_bws = tor_malloc(sizeof(uint32_t) * + smartlist_len(votes)); int num_bandwidths; + int num_mbws; int *n_voter_flags; /* n_voter_flags[j] is the number of flags that * votes[j] knows about. */ @@ -813,10 +1693,12 @@ networkstatus_compute_consensus(smartlist_t *votes, const char *chosen_name = NULL; int exitsummary_disagreement = 0; int is_named = 0, is_unnamed = 0, is_running = 0; + int is_guard = 0, is_exit = 0, is_bad_exit = 0; int naming_conflict = 0; int n_listing = 0; int i; - char buf[256]; + char *buf=NULL; + char microdesc_digest[DIGEST256_LEN]; /* Of the next-to-be-considered digest in each voter, which is first? */ SMARTLIST_FOREACH(votes, networkstatus_t *, v, { @@ -835,6 +1717,7 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_clear(chosen_flags); smartlist_clear(versions); num_bandwidths = 0; + num_mbws = 0; /* Okay, go through all the entries for this digest. */ SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) { @@ -868,6 +1751,9 @@ networkstatus_compute_consensus(smartlist_t *votes, } /* count bandwidths */ + if (rs->status.has_measured_bw) + measured_bws[num_mbws++] = rs->status.measured_bw; + if (rs->status.has_bandwidth) bandwidths[num_bandwidths++] = rs->status.bandwidth; } SMARTLIST_FOREACH_END(v); @@ -879,7 +1765,9 @@ networkstatus_compute_consensus(smartlist_t *votes, /* Figure out the most popular opinion of what the most recent * routerinfo and its contents are. */ - rs = compute_routerstatus_consensus(matching_descs); + memset(microdesc_digest, 0, sizeof(microdesc_digest)); + rs = compute_routerstatus_consensus(matching_descs, consensus_method, + microdesc_digest); /* Copy bits of that into rs_out. */ tor_assert(!memcmp(lowest_id, rs->status.identity_digest, DIGEST_LEN)); memcpy(rs_out.identity_digest, lowest_id, DIGEST_LEN); @@ -925,8 +1813,14 @@ networkstatus_compute_consensus(smartlist_t *votes, } else { if (flag_counts[fl_sl_idx] > n_flag_voters[fl_sl_idx]/2) { smartlist_add(chosen_flags, (char*)fl); - if (!strcmp(fl, "Running")) + if (!strcmp(fl, "Exit")) + is_exit = 1; + else if (!strcmp(fl, "Guard")) + is_guard = 1; + else if (!strcmp(fl, "Running")) is_running = 1; + else if (!strcmp(fl, "BadExit")) + is_bad_exit = 1; } } }); @@ -945,11 +1839,36 @@ networkstatus_compute_consensus(smartlist_t *votes, } /* Pick a bandwidth */ - if (consensus_method >= 5 && num_bandwidths > 0) { + if (consensus_method >= 6 && num_mbws > 2) { + rs_out.has_bandwidth = 1; + rs_out.bandwidth = median_uint32(measured_bws, num_mbws); + } else if (consensus_method >= 5 && num_bandwidths > 0) { rs_out.has_bandwidth = 1; rs_out.bandwidth = median_uint32(bandwidths, num_bandwidths); } + /* Fix bug 2203: Do not count BadExit nodes as Exits for bw weights */ + if (consensus_method >= 11) { + is_exit = is_exit && !is_bad_exit; + } + + if (consensus_method >= MIN_METHOD_FOR_BW_WEIGHTS) { + if (rs_out.has_bandwidth) { + T += rs_out.bandwidth; + if (is_exit && is_guard) + D += rs_out.bandwidth; + else if (is_exit) + E += rs_out.bandwidth; + else if (is_guard) + G += rs_out.bandwidth; + else + M += rs_out.bandwidth; + } else { + log_warn(LD_BUG, "Missing consensus bandwidth for router %s", + rs_out.nickname); + } + } + /* Ok, we already picked a descriptor digest we want to list * previously. Now we want to use the exit policy summary from * that descriptor. If everybody plays nice all the voters who @@ -1034,11 +1953,23 @@ networkstatus_compute_consensus(smartlist_t *votes, } } - /* Okay!! Now we can write the descriptor... */ - /* First line goes into "buf". */ - routerstatus_format_entry(buf, sizeof(buf), &rs_out, NULL, 1, 0); - smartlist_add(chunks, tor_strdup(buf)); - /* Second line is all flags. The "\n" is missing. */ + { + char buf[4096]; + /* Okay!! Now we can write the descriptor... */ + /* First line goes into "buf". */ + routerstatus_format_entry(buf, sizeof(buf), &rs_out, NULL, + rs_format); + smartlist_add(chunks, tor_strdup(buf)); + } + /* Now an m line, if applicable. */ + if (flavor == FLAV_MICRODESC && + !tor_digest256_is_zero(microdesc_digest)) { + char m[BASE64_DIGEST256_LEN+1], *cp; + digest256_to_base64(m, microdesc_digest); + tor_asprintf(&cp, "m %s\n", m); + smartlist_add(chunks, cp); + } + /* Next line is all flags. The "\n" is missing. */ smartlist_add(chunks, smartlist_join_strings(chosen_flags, " ", 0, NULL)); /* Now the version line. */ @@ -1049,24 +1980,16 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_add(chunks, tor_strdup("\n")); /* Now the weight line. */ if (rs_out.has_bandwidth) { - int r = tor_snprintf(buf, sizeof(buf), - "w Bandwidth=%d\n", rs_out.bandwidth); - if (r<0) { - log_warn(LD_BUG, "Not enough space in buffer for weight line."); - *buf = '\0'; - } - smartlist_add(chunks, tor_strdup(buf)); - }; + char *cp=NULL; + tor_asprintf(&cp, "w Bandwidth=%d\n", rs_out.bandwidth); + smartlist_add(chunks, cp); + } + /* Now the exitpolicy summary line. */ - if (rs_out.has_exitsummary) { - char buf[MAX_POLICY_LINE_LEN+1]; - int r = tor_snprintf(buf, sizeof(buf), "p %s\n", rs_out.exitsummary); - if (r<0) { - log_warn(LD_BUG, "Not enough space in buffer for exitpolicy line."); - *buf = '\0'; - } - smartlist_add(chunks, tor_strdup(buf)); - }; + if (rs_out.has_exitsummary && flavor == FLAV_NS) { + tor_asprintf(&buf, "p %s\n", rs_out.exitsummary); + smartlist_add(chunks, buf); + } /* And the loop is over and we move on to the next router */ } @@ -1087,34 +2010,97 @@ networkstatus_compute_consensus(smartlist_t *votes, smartlist_free(versions); smartlist_free(exitsummaries); tor_free(bandwidths); + tor_free(measured_bws); + } + + if (consensus_method >= MIN_METHOD_FOR_FOOTER) { + /* Starting with consensus method 9, we clearly mark the directory + * footer region */ + smartlist_add(chunks, tor_strdup("directory-footer\n")); + } + + if (consensus_method >= MIN_METHOD_FOR_BW_WEIGHTS) { + int64_t weight_scale = BW_WEIGHT_SCALE; + char *bw_weight_param = NULL; + + // Parse params, extract BW_WEIGHT_SCALE if present + // DO NOT use consensus_param_bw_weight_scale() in this code! + // The consensus is not formed yet! + if (params) { + if (strcmpstart(params, "bwweightscale=") == 0) + bw_weight_param = params; + else + bw_weight_param = strstr(params, " bwweightscale="); + } + + if (bw_weight_param) { + int ok=0; + char *eq = strchr(bw_weight_param, '='); + if (eq) { + weight_scale = tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, + NULL); + if (!ok) { + log_warn(LD_DIR, "Bad element '%s' in bw weight param", + escaped(bw_weight_param)); + weight_scale = BW_WEIGHT_SCALE; + } + } else { + log_warn(LD_DIR, "Bad element '%s' in bw weight param", + escaped(bw_weight_param)); + weight_scale = BW_WEIGHT_SCALE; + } + } + + if (consensus_method < 10) { + networkstatus_compute_bw_weights_v9(chunks, G, M, E, D, T, weight_scale); + added_weights = 1; + } else { + added_weights = networkstatus_compute_bw_weights_v10(chunks, G, M, E, D, + T, weight_scale); + } } /* Add a signature. */ { - char digest[DIGEST_LEN]; + char digest[DIGEST256_LEN]; char fingerprint[HEX_DIGEST_LEN+1]; char signing_key_fingerprint[HEX_DIGEST_LEN+1]; + digest_algorithm_t digest_alg = + flavor == FLAV_NS ? DIGEST_SHA1 : DIGEST_SHA256; + size_t digest_len = + flavor == FLAV_NS ? DIGEST_LEN : DIGEST256_LEN; + const char *algname = crypto_digest_algorithm_get_name(digest_alg); + char *buf = NULL; + char sigbuf[4096]; - char buf[4096]; smartlist_add(chunks, tor_strdup("directory-signature ")); /* Compute the hash of the chunks. */ - hash_list_members(digest, chunks); + hash_list_members(digest, digest_len, chunks, digest_alg); /* Get the fingerprints */ crypto_pk_get_fingerprint(identity_key, fingerprint, 0); crypto_pk_get_fingerprint(signing_key, signing_key_fingerprint, 0); /* add the junk that will go at the end of the line. */ - tor_snprintf(buf, sizeof(buf), "%s %s\n", fingerprint, - signing_key_fingerprint); + if (flavor == FLAV_NS) { + tor_asprintf(&buf, "%s %s\n", fingerprint, + signing_key_fingerprint); + } else { + tor_asprintf(&buf, "%s %s %s\n", + algname, fingerprint, + signing_key_fingerprint); + } + smartlist_add(chunks, buf); /* And the signature. */ - if (router_append_dirobj_signature(buf, sizeof(buf), digest, + sigbuf[0] = '\0'; + if (router_append_dirobj_signature(sigbuf, sizeof(sigbuf), + digest, digest_len, signing_key)) { log_warn(LD_BUG, "Couldn't sign consensus networkstatus."); return NULL; /* This leaks, but it should never happen. */ } - smartlist_add(chunks, tor_strdup(buf)); + smartlist_add(chunks, tor_strdup(sigbuf)); if (legacy_id_key_digest && legacy_signing_key && consensus_method >= 3) { smartlist_add(chunks, tor_strdup("directory-signature ")); @@ -1122,14 +2108,23 @@ networkstatus_compute_consensus(smartlist_t *votes, legacy_id_key_digest, DIGEST_LEN); crypto_pk_get_fingerprint(legacy_signing_key, signing_key_fingerprint, 0); - tor_snprintf(buf, sizeof(buf), "%s %s\n", fingerprint, - signing_key_fingerprint); - if (router_append_dirobj_signature(buf, sizeof(buf), digest, + if (flavor == FLAV_NS) { + tor_asprintf(&buf, "%s %s\n", fingerprint, + signing_key_fingerprint); + } else { + tor_asprintf(&buf, "%s %s %s\n", + algname, fingerprint, + signing_key_fingerprint); + } + smartlist_add(chunks, buf); + sigbuf[0] = '\0'; + if (router_append_dirobj_signature(sigbuf, sizeof(sigbuf), + digest, digest_len, legacy_signing_key)) { log_warn(LD_BUG, "Couldn't sign consensus networkstatus."); return NULL; /* This leaks, but it should never happen. */ } - smartlist_add(chunks, tor_strdup(buf)); + smartlist_add(chunks, tor_strdup(sigbuf)); } } @@ -1146,11 +2141,15 @@ networkstatus_compute_consensus(smartlist_t *votes, networkstatus_t *c; if (!(c = networkstatus_parse_vote_from_string(result, NULL, NS_TYPE_CONSENSUS))) { - log_err(LD_BUG,"Generated a networkstatus consensus we couldn't " + log_err(LD_BUG, "Generated a networkstatus consensus we couldn't " "parse."); tor_free(result); return NULL; } + // Verify balancing parameters + if (consensus_method >= MIN_METHOD_FOR_BW_WEIGHTS && added_weights) { + networkstatus_verify_bw_weights(c); + } networkstatus_vote_free(c); } @@ -1171,10 +2170,14 @@ networkstatus_add_detached_signatures(networkstatus_t *target, const char **msg_out) { int r = 0; + const char *flavor; + smartlist_t *siglist; tor_assert(sigs); tor_assert(target); tor_assert(target->type == NS_TYPE_CONSENSUS); + flavor = networkstatus_get_flavor_name(target->flavor); + /* Do the times seem right? */ if (target->valid_after != sigs->valid_after) { *msg_out = "Valid-After times do not match " @@ -1191,79 +2194,179 @@ networkstatus_add_detached_signatures(networkstatus_t *target, "when adding detached signatures to consensus"; return -1; } - /* Are they the same consensus? */ - if (memcmp(target->networkstatus_digest, sigs->networkstatus_digest, - DIGEST_LEN)) { - *msg_out = "Digest mismatch when adding detached signatures to consensus"; + siglist = strmap_get(sigs->signatures, flavor); + if (!siglist) { + *msg_out = "No signatures for given consensus flavor"; return -1; } - /* For each voter in src... */ - SMARTLIST_FOREACH_BEGIN(sigs->signatures, networkstatus_voter_info_t *, - src_voter) { - char voter_identity[HEX_DIGEST_LEN+1]; - networkstatus_voter_info_t *target_voter = - networkstatus_get_voter_by_id(target, src_voter->identity_digest); - authority_cert_t *cert = NULL; - - base16_encode(voter_identity, sizeof(voter_identity), - src_voter->identity_digest, DIGEST_LEN); - log_info(LD_DIR, "Looking at signature from %s", voter_identity); - /* If the target doesn't know about this voter, then forget it. */ - if (!target_voter) { - log_info(LD_DIR, "We do not know about %s", voter_identity); - continue; + /** Make sure all the digests we know match, and at least one matches. */ + { + digests_t *digests = strmap_get(sigs->digests, flavor); + int n_matches = 0; + digest_algorithm_t alg; + if (!digests) { + *msg_out = "No digests for given consensus flavor"; + return -1; + } + for (alg = DIGEST_SHA1; alg < N_DIGEST_ALGORITHMS; ++alg) { + if (!tor_mem_is_zero(digests->d[alg], DIGEST256_LEN)) { + if (!memcmp(target->digests.d[alg], digests->d[alg], DIGEST256_LEN)) { + ++n_matches; + } else { + *msg_out = "Mismatched digest."; + return -1; + } } + } + if (!n_matches) { + *msg_out = "No regognized digests for given consensus flavor"; + } + } - /* If the target already has a good signature from this voter, then skip - * this one. */ - if (target_voter->good_signature) { - log_info(LD_DIR, "We already have a good signature from %s", - voter_identity); - continue; - } + /* For each voter in src... */ + SMARTLIST_FOREACH_BEGIN(siglist, document_signature_t *, sig) { + char voter_identity[HEX_DIGEST_LEN+1]; + networkstatus_voter_info_t *target_voter = + networkstatus_get_voter_by_id(target, sig->identity_digest); + authority_cert_t *cert = NULL; + const char *algorithm; + document_signature_t *old_sig = NULL; + + algorithm = crypto_digest_algorithm_get_name(sig->alg); + + base16_encode(voter_identity, sizeof(voter_identity), + sig->identity_digest, DIGEST_LEN); + log_info(LD_DIR, "Looking at signature from %s using %s", voter_identity, + algorithm); + /* If the target doesn't know about this voter, then forget it. */ + if (!target_voter) { + log_info(LD_DIR, "We do not know any voter with ID %s", voter_identity); + continue; + } - /* Try checking the signature if we haven't already. */ - if (!src_voter->good_signature && !src_voter->bad_signature) { - cert = authority_cert_get_by_digests(src_voter->identity_digest, - src_voter->signing_key_digest); - if (cert) { - networkstatus_check_voter_signature(target, src_voter, cert); - } + old_sig = voter_get_sig_by_algorithm(target_voter, sig->alg); + + /* If the target already has a good signature from this voter, then skip + * this one. */ + if (old_sig && old_sig->good_signature) { + log_info(LD_DIR, "We already have a good signature from %s using %s", + voter_identity, algorithm); + continue; + } + + /* Try checking the signature if we haven't already. */ + if (!sig->good_signature && !sig->bad_signature) { + cert = authority_cert_get_by_digests(sig->identity_digest, + sig->signing_key_digest); + if (cert) + networkstatus_check_document_signature(target, sig, cert); + } + + /* If this signature is good, or we don't have any signature yet, + * then maybe add it. */ + if (sig->good_signature || !old_sig || old_sig->bad_signature) { + log_info(LD_DIR, "Adding signature from %s with %s", voter_identity, + algorithm); + ++r; + if (old_sig) { + smartlist_remove(target_voter->sigs, old_sig); + document_signature_free(old_sig); } + smartlist_add(target_voter->sigs, document_signature_dup(sig)); + } else { + log_info(LD_DIR, "Not adding signature from %s", voter_identity); + } + } SMARTLIST_FOREACH_END(sig); + + return r; +} + +/** Return a newly allocated string containing all the signatures on + * <b>consensus</b> by all voters. If <b>for_detached_signatures</b> is true, + * then the signatures will be put in a detached signatures document, so + * prefix any non-NS-flavored signatures with "additional-signature" rather + * than "directory-signature". */ +static char * +networkstatus_format_signatures(networkstatus_t *consensus, + int for_detached_signatures) +{ + smartlist_t *elements; + char buf[4096]; + char *result = NULL; + int n_sigs = 0; + const consensus_flavor_t flavor = consensus->flavor; + const char *flavor_name = networkstatus_get_flavor_name(flavor); + const char *keyword; + + if (for_detached_signatures && flavor != FLAV_NS) + keyword = "additional-signature"; + else + keyword = "directory-signature"; - /* If this signature is good, or we don't have any signature yet, - * then add it. */ - if (src_voter->good_signature || !target_voter->signature) { - log_info(LD_DIR, "Adding signature from %s", voter_identity); - ++r; - tor_free(target_voter->signature); - target_voter->signature = - tor_memdup(src_voter->signature, src_voter->signature_len); - memcpy(target_voter->signing_key_digest, src_voter->signing_key_digest, - DIGEST_LEN); - target_voter->signature_len = src_voter->signature_len; - target_voter->good_signature = src_voter->good_signature; - target_voter->bad_signature = src_voter->bad_signature; + elements = smartlist_create(); + + SMARTLIST_FOREACH_BEGIN(consensus->voters, networkstatus_voter_info_t *, v) { + SMARTLIST_FOREACH_BEGIN(v->sigs, document_signature_t *, sig) { + char sk[HEX_DIGEST_LEN+1]; + char id[HEX_DIGEST_LEN+1]; + if (!sig->signature || sig->bad_signature) + continue; + ++n_sigs; + base16_encode(sk, sizeof(sk), sig->signing_key_digest, DIGEST_LEN); + base16_encode(id, sizeof(id), sig->identity_digest, DIGEST_LEN); + if (flavor == FLAV_NS) { + tor_snprintf(buf, sizeof(buf), + "%s %s %s\n-----BEGIN SIGNATURE-----\n", + keyword, id, sk); } else { - log_info(LD_DIR, "Not adding signature from %s", voter_identity); + const char *digest_name = + crypto_digest_algorithm_get_name(sig->alg); + tor_snprintf(buf, sizeof(buf), + "%s%s%s %s %s %s\n-----BEGIN SIGNATURE-----\n", + keyword, + for_detached_signatures ? " " : "", + for_detached_signatures ? flavor_name : "", + digest_name, id, sk); } - } SMARTLIST_FOREACH_END(src_voter); + smartlist_add(elements, tor_strdup(buf)); + base64_encode(buf, sizeof(buf), sig->signature, sig->signature_len); + strlcat(buf, "-----END SIGNATURE-----\n", sizeof(buf)); + smartlist_add(elements, tor_strdup(buf)); + } SMARTLIST_FOREACH_END(sig); + } SMARTLIST_FOREACH_END(v); - return r; + result = smartlist_join_strings(elements, "", 0, NULL); + SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); + smartlist_free(elements); + if (!n_sigs) + tor_free(result); + return result; } /** Return a newly allocated string holding the detached-signatures document - * corresponding to the signatures on <b>consensus</b>. */ + * corresponding to the signatures on <b>consensuses</b>, which must contain + * exactly one FLAV_NS consensus, and no more than one consensus for each + * other flavor. */ char * -networkstatus_get_detached_signatures(networkstatus_t *consensus) +networkstatus_get_detached_signatures(smartlist_t *consensuses) { smartlist_t *elements; char buf[4096]; - char *result = NULL; - int n_sigs = 0; - tor_assert(consensus); - tor_assert(consensus->type == NS_TYPE_CONSENSUS); + char *result = NULL, *sigs = NULL; + networkstatus_t *consensus_ns = NULL; + tor_assert(consensuses); + + SMARTLIST_FOREACH(consensuses, networkstatus_t *, ns, { + tor_assert(ns); + tor_assert(ns->type == NS_TYPE_CONSENSUS); + if (ns && ns->flavor == FLAV_NS) + consensus_ns = ns; + }); + if (!consensus_ns) { + log_warn(LD_BUG, "No NS consensus given."); + return NULL; + } elements = smartlist_create(); @@ -1272,10 +2375,11 @@ networkstatus_get_detached_signatures(networkstatus_t *consensus) vu_buf[ISO_TIME_LEN+1]; char d[HEX_DIGEST_LEN+1]; - base16_encode(d, sizeof(d), consensus->networkstatus_digest, DIGEST_LEN); - format_iso_time(va_buf, consensus->valid_after); - format_iso_time(fu_buf, consensus->fresh_until); - format_iso_time(vu_buf, consensus->valid_until); + base16_encode(d, sizeof(d), + consensus_ns->digests.d[DIGEST_SHA1], DIGEST_LEN); + format_iso_time(va_buf, consensus_ns->valid_after); + format_iso_time(fu_buf, consensus_ns->fresh_until); + format_iso_time(vu_buf, consensus_ns->valid_until); tor_snprintf(buf, sizeof(buf), "consensus-digest %s\n" @@ -1285,45 +2389,89 @@ networkstatus_get_detached_signatures(networkstatus_t *consensus) smartlist_add(elements, tor_strdup(buf)); } - SMARTLIST_FOREACH(consensus->voters, networkstatus_voter_info_t *, v, - { - char sk[HEX_DIGEST_LEN+1]; - char id[HEX_DIGEST_LEN+1]; - if (!v->signature || v->bad_signature) + /* Get all the digests for the non-FLAV_NS consensuses */ + SMARTLIST_FOREACH_BEGIN(consensuses, networkstatus_t *, ns) { + const char *flavor_name = networkstatus_get_flavor_name(ns->flavor); + int alg; + if (ns->flavor == FLAV_NS) + continue; + + /* start with SHA256; we don't include SHA1 for anything but the basic + * consensus. */ + for (alg = DIGEST_SHA256; alg < N_DIGEST_ALGORITHMS; ++alg) { + char d[HEX_DIGEST256_LEN+1]; + const char *alg_name = + crypto_digest_algorithm_get_name(alg); + if (tor_mem_is_zero(ns->digests.d[alg], DIGEST256_LEN)) continue; - ++n_sigs; - base16_encode(sk, sizeof(sk), v->signing_key_digest, DIGEST_LEN); - base16_encode(id, sizeof(id), v->identity_digest, DIGEST_LEN); - tor_snprintf(buf, sizeof(buf), - "directory-signature %s %s\n-----BEGIN SIGNATURE-----\n", - id, sk); + base16_encode(d, sizeof(d), ns->digests.d[alg], DIGEST256_LEN); + tor_snprintf(buf, sizeof(buf), "additional-digest %s %s %s\n", + flavor_name, alg_name, d); smartlist_add(elements, tor_strdup(buf)); - base64_encode(buf, sizeof(buf), v->signature, v->signature_len); - strlcat(buf, "-----END SIGNATURE-----\n", sizeof(buf)); - smartlist_add(elements, tor_strdup(buf)); - }); + } + } SMARTLIST_FOREACH_END(ns); + + /* Now get all the sigs for non-FLAV_NS consensuses */ + SMARTLIST_FOREACH_BEGIN(consensuses, networkstatus_t *, ns) { + char *sigs; + if (ns->flavor == FLAV_NS) + continue; + sigs = networkstatus_format_signatures(ns, 1); + if (!sigs) { + log_warn(LD_DIR, "Couldn't format signatures"); + goto err; + } + smartlist_add(elements, sigs); + } SMARTLIST_FOREACH_END(ns); - result = smartlist_join_strings(elements, "", 0, NULL); + /* Now add the FLAV_NS consensus signatrures. */ + sigs = networkstatus_format_signatures(consensus_ns, 1); + if (!sigs) + goto err; + smartlist_add(elements, sigs); + result = smartlist_join_strings(elements, "", 0, NULL); + err: SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); smartlist_free(elements); - if (!n_sigs) - tor_free(result); return result; } +/** Return a newly allocated string holding a detached-signatures document for + * all of the in-progress consensuses in the <b>n_flavors</b>-element array at + * <b>pending</b>. */ +static char * +get_detached_signatures_from_pending_consensuses(pending_consensus_t *pending, + int n_flavors) +{ + int flav; + char *signatures; + smartlist_t *c = smartlist_create(); + for (flav = 0; flav < n_flavors; ++flav) { + if (pending[flav].consensus) + smartlist_add(c, pending[flav].consensus); + } + signatures = networkstatus_get_detached_signatures(c); + smartlist_free(c); + return signatures; +} + /** Release all storage held in <b>s</b>. */ void ns_detached_signatures_free(ns_detached_signatures_t *s) { + if (!s) + return; if (s->signatures) { - SMARTLIST_FOREACH(s->signatures, networkstatus_voter_info_t *, v, - { - tor_free(v->signature); - tor_free(v); - }); - smartlist_free(s->signatures); + STRMAP_FOREACH(s->signatures, flavor, smartlist_t *, sigs) { + SMARTLIST_FOREACH(sigs, document_signature_t *, sig, + document_signature_free(sig)); + smartlist_free(sigs); + } STRMAP_FOREACH_END; + strmap_free(s->signatures, NULL); + strmap_free(s->digests, _tor_free); } + tor_free(s); } @@ -1513,7 +2661,7 @@ dirvote_act(or_options_t *options, time_t now) if (voting_schedule.voting_ends < now && !voting_schedule.have_built_consensus) { log_notice(LD_DIR, "Time to compute a consensus."); - dirvote_compute_consensus(); + dirvote_compute_consensuses(); /* XXXX We will want to try again later if we haven't got enough * votes yet. Implement this if it turns out to ever happen. */ voting_schedule.have_built_consensus = 1; @@ -1550,14 +2698,13 @@ static smartlist_t *pending_vote_list = NULL; /** List of pending_vote_t for the previous vote. After we've used them to * build a consensus, the votes go here for the next period. */ static smartlist_t *previous_vote_list = NULL; -/** The body of the consensus that we're currently building. Once we - * have it built, it goes into dirserv.c */ -static char *pending_consensus_body = NULL; + +static pending_consensus_t pending_consensuses[N_CONSENSUS_FLAVORS]; + /** The detached signatures for the consensus that we're currently * building. */ static char *pending_consensus_signatures = NULL; -/** The parsed in-progress consensus document. */ -static networkstatus_t *pending_consensus = NULL; + /** List of ns_detached_signatures_t: hold signatures that get posted to us * before we have generated the consensus on our own. */ static smartlist_t *pending_consensus_signature_list = NULL; @@ -1651,15 +2798,39 @@ dirvote_fetch_missing_votes(void) static void dirvote_fetch_missing_signatures(void) { - if (!pending_consensus) + int need_any = 0; + int i; + for (i=0; i < N_CONSENSUS_FLAVORS; ++i) { + networkstatus_t *consensus = pending_consensuses[i].consensus; + if (!consensus || + networkstatus_check_consensus_signature(consensus, -1) == 1) { + /* We have no consensus, or we have one that's signed by everybody. */ + continue; + } + need_any = 1; + } + if (!need_any) return; - if (networkstatus_check_consensus_signature(pending_consensus, -1) == 1) - return; /* we have a signature from everybody. */ directory_get_from_all_authorities(DIR_PURPOSE_FETCH_DETACHED_SIGNATURES, 0, NULL); } +/** Release all storage held by pending consensuses (those waiting for + * signatures). */ +static void +dirvote_clear_pending_consensuses(void) +{ + int i; + for (i = 0; i < N_CONSENSUS_FLAVORS; ++i) { + pending_consensus_t *pc = &pending_consensuses[i]; + tor_free(pc->body); + + networkstatus_vote_free(pc->consensus); + pc->consensus = NULL; + } +} + /** Drop all currently pending votes, consensus, and detached signatures. */ static void dirvote_clear_votes(int all_votes) @@ -1697,12 +2868,8 @@ dirvote_clear_votes(int all_votes) tor_free(cp)); smartlist_clear(pending_consensus_signature_list); } - tor_free(pending_consensus_body); tor_free(pending_consensus_signatures); - if (pending_consensus) { - networkstatus_vote_free(pending_consensus); - pending_consensus = NULL; - } + dirvote_clear_pending_consensuses(); } /** Return a newly allocated string containing the hex-encoded v3 authority @@ -1760,7 +2927,13 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) } tor_assert(smartlist_len(vote->voters) == 1); vi = get_voter(vote); - tor_assert(vi->good_signature == 1); + { + int any_sig_good = 0; + SMARTLIST_FOREACH(vi->sigs, document_signature_t *, sig, + if (sig->good_signature) + any_sig_good = 1); + tor_assert(any_sig_good); + } ds = trusteddirserver_get_by_v3_auth_digest(vi->identity_digest); if (!ds) { char *keys = list_v3_auth_ids(); @@ -1797,6 +2970,9 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) goto err; } + /* Fetch any new router descriptors we just learned about */ + update_consensus_router_descriptor_downloads(time(NULL), 1, vote); + /* Now see whether we already have a vote from this authority. */ SMARTLIST_FOREACH(pending_vote_list, pending_vote_t *, v, { if (! memcmp(v->vote->cert->cache_info.identity_digest, @@ -1805,7 +2981,8 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) networkstatus_voter_info_t *vi_old = get_voter(v->vote); if (!memcmp(vi_old->vote_digest, vi->vote_digest, DIGEST_LEN)) { /* Ah, it's the same vote. Not a problem. */ - log_info(LD_DIR, "Discarding a vote we already have."); + log_info(LD_DIR, "Discarding a vote we already have (from %s).", + vi->address); if (*status_out < 200) *status_out = 200; goto discard; @@ -1832,7 +3009,7 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) goto err; } } - }); + }); pending_vote = tor_malloc_zero(sizeof(pending_vote_t)); pending_vote->vote_body = new_cached_dir(tor_strndup(vote_body, @@ -1856,8 +3033,7 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) *status_out = 400; discard: - if (vote) - networkstatus_vote_free(vote); + networkstatus_vote_free(vote); if (end_of_vote && !strcmpstart(end_of_vote, "network-status-version ")) { vote_body = end_of_vote; @@ -1884,14 +3060,18 @@ dirvote_add_vote(const char *vote_body, const char **msg_out, int *status_out) * pending_consensus: it won't be ready to be published until we have * everybody else's signatures collected too. (V3 Authority only) */ static int -dirvote_compute_consensus(void) +dirvote_compute_consensuses(void) { /* Have we got enough votes to try? */ - int n_votes, n_voters; + int n_votes, n_voters, n_vote_running = 0; smartlist_t *votes = NULL, *votestrings = NULL; char *consensus_body = NULL, *signatures = NULL, *votefile; networkstatus_t *consensus = NULL; authority_cert_t *my_cert; + pending_consensus_t pending[N_CONSENSUS_FLAVORS]; + int flav; + + memset(pending, 0, sizeof(pending)); if (!pending_vote_list) pending_vote_list = smartlist_create(); @@ -1900,7 +3080,20 @@ dirvote_compute_consensus(void) n_votes = smartlist_len(pending_vote_list); if (n_votes <= n_voters/2) { log_warn(LD_DIR, "We don't have enough votes to generate a consensus: " - "%d of %d", n_votes, n_voters/2); + "%d of %d", n_votes, n_voters/2+1); + goto err; + } + tor_assert(pending_vote_list); + SMARTLIST_FOREACH(pending_vote_list, pending_vote_t *, v, { + if (smartlist_string_isin(v->vote->known_flags, "Running")) + n_vote_running++; + }); + if (!n_vote_running) { + /* See task 1066. */ + log_warn(LD_DIR, "Nobody has voted on the Running flag. Generating " + "and publishing a consensus without Running nodes " + "would make many clients stop working. Not " + "generating a consensus!"); goto err; } @@ -1931,47 +3124,71 @@ dirvote_compute_consensus(void) char legacy_dbuf[DIGEST_LEN]; crypto_pk_env_t *legacy_sign=NULL; char *legacy_id_digest = NULL; + int n_generated = 0; if (get_options()->V3AuthUseLegacyKey) { authority_cert_t *cert = get_my_v3_legacy_cert(); legacy_sign = get_my_v3_legacy_signing_key(); if (cert) { - crypto_pk_get_digest(cert->identity_key, legacy_dbuf); - legacy_id_digest = legacy_dbuf; + if (crypto_pk_get_digest(cert->identity_key, legacy_dbuf)) { + log_warn(LD_BUG, + "Unable to compute digest of legacy v3 identity key"); + } else { + legacy_id_digest = legacy_dbuf; + } } } - consensus_body = networkstatus_compute_consensus( + + for (flav = 0; flav < N_CONSENSUS_FLAVORS; ++flav) { + const char *flavor_name = networkstatus_get_flavor_name(flav); + consensus_body = networkstatus_compute_consensus( votes, n_voters, my_cert->identity_key, - get_my_v3_authority_signing_key(), legacy_id_digest, legacy_sign); - } - if (!consensus_body) { - log_warn(LD_DIR, "Couldn't generate a consensus at all!"); - goto err; - } - consensus = networkstatus_parse_vote_from_string(consensus_body, NULL, - NS_TYPE_CONSENSUS); - if (!consensus) { - log_warn(LD_DIR, "Couldn't parse consensus we generated!"); - goto err; + get_my_v3_authority_signing_key(), legacy_id_digest, legacy_sign, + flav); + + if (!consensus_body) { + log_warn(LD_DIR, "Couldn't generate a %s consensus at all!", + flavor_name); + continue; + } + consensus = networkstatus_parse_vote_from_string(consensus_body, NULL, + NS_TYPE_CONSENSUS); + if (!consensus) { + log_warn(LD_DIR, "Couldn't parse %s consensus we generated!", + flavor_name); + tor_free(consensus_body); + continue; + } + + /* 'Check' our own signature, to mark it valid. */ + networkstatus_check_consensus_signature(consensus, -1); + + pending[flav].body = consensus_body; + pending[flav].consensus = consensus; + n_generated++; + consensus_body = NULL; + consensus = NULL; + } + if (!n_generated) { + log_warn(LD_DIR, "Couldn't generate any consensus flavors at all."); + goto err; + } } - /* 'Check' our own signature, to mark it valid. */ - networkstatus_check_consensus_signature(consensus, -1); - signatures = networkstatus_get_detached_signatures(consensus); + signatures = get_detached_signatures_from_pending_consensuses( + pending, N_CONSENSUS_FLAVORS); + if (!signatures) { log_warn(LD_DIR, "Couldn't extract signatures."); goto err; } - tor_free(pending_consensus_body); - pending_consensus_body = consensus_body; + dirvote_clear_pending_consensuses(); + memcpy(pending_consensuses, pending, sizeof(pending)); + tor_free(pending_consensus_signatures); pending_consensus_signatures = signatures; - if (pending_consensus) - networkstatus_vote_free(pending_consensus); - pending_consensus = consensus; - if (pending_consensus_signature_list) { int n_sigs = 0; /* we may have gotten signatures for this consensus before we built @@ -1979,7 +3196,7 @@ dirvote_compute_consensus(void) SMARTLIST_FOREACH(pending_consensus_signature_list, char *, sig, { const char *msg = NULL; - int r = dirvote_add_signatures_to_pending_consensus(sig, &msg); + int r = dirvote_add_signatures_to_all_pending_consensuses(sig, &msg); if (r >= 0) n_sigs += r; else @@ -2003,10 +3220,10 @@ dirvote_compute_consensus(void) strlen(pending_consensus_signatures), 0); log_notice(LD_DIR, "Signature(s) posted."); + smartlist_free(votes); return 0; err: - if (votes) - smartlist_free(votes); + smartlist_free(votes); tor_free(consensus_body); tor_free(signatures); networkstatus_vote_free(consensus); @@ -2015,76 +3232,62 @@ dirvote_compute_consensus(void) } /** Helper: we just got the <b>detached_signatures_body</b> sent to us as - * signatures on the currently pending consensus. Add them to the consensus + * signatures on the currently pending consensus. Add them to <b>pc</b> * as appropriate. Return the number of signatures added. (?) */ static int dirvote_add_signatures_to_pending_consensus( - const char *detached_signatures_body, + pending_consensus_t *pc, + ns_detached_signatures_t *sigs, const char **msg_out) { - ns_detached_signatures_t *sigs = NULL; + const char *flavor_name; int r = -1; - tor_assert(detached_signatures_body); - tor_assert(msg_out); - /* Only call if we have a pending consensus right now. */ - tor_assert(pending_consensus); - tor_assert(pending_consensus_body); + tor_assert(pc->consensus); + tor_assert(pc->body); tor_assert(pending_consensus_signatures); + flavor_name = networkstatus_get_flavor_name(pc->consensus->flavor); *msg_out = NULL; - if (!(sigs = networkstatus_parse_detached_signatures( - detached_signatures_body, NULL))) { - *msg_out = "Couldn't parse detached signatures."; - goto err; + { + smartlist_t *sig_list = strmap_get(sigs->signatures, flavor_name); + log_info(LD_DIR, "Have %d signatures for adding to %s consensus.", + sig_list ? smartlist_len(sig_list) : 0, flavor_name); } - - log_info(LD_DIR, "Have %d signatures for adding to consensus.", - smartlist_len(sigs->signatures)); - r = networkstatus_add_detached_signatures(pending_consensus, - sigs, msg_out); + r = networkstatus_add_detached_signatures(pc->consensus, sigs, msg_out); log_info(LD_DIR,"Added %d signatures to consensus.", r); if (r >= 1) { - char *new_detached = - networkstatus_get_detached_signatures(pending_consensus); - const char *src; + char *new_signatures = + networkstatus_format_signatures(pc->consensus, 0); char *dst, *dst_end; size_t new_consensus_len; - if (!new_detached) { + if (!new_signatures) { *msg_out = "No signatures to add"; goto err; } new_consensus_len = - strlen(pending_consensus_body) + strlen(new_detached) + 1; - pending_consensus_body = tor_realloc(pending_consensus_body, - new_consensus_len); - dst_end = pending_consensus_body + new_consensus_len; - dst = strstr(pending_consensus_body, "directory-signature "); + strlen(pc->body) + strlen(new_signatures) + 1; + pc->body = tor_realloc(pc->body, new_consensus_len); + dst_end = pc->body + new_consensus_len; + dst = strstr(pc->body, "directory-signature "); tor_assert(dst); - src = strstr(new_detached, "directory-signature "); - tor_assert(src); - strlcpy(dst, src, dst_end-dst); + strlcpy(dst, new_signatures, dst_end-dst); /* We remove this block once it has failed to crash for a while. But * unless it shows up in profiles, we're probably better leaving it in, * just in case we break detached signature processing at some point. */ { - ns_detached_signatures_t *sigs = - networkstatus_parse_detached_signatures(new_detached, NULL); networkstatus_t *v = networkstatus_parse_vote_from_string( - pending_consensus_body, NULL, + pc->body, NULL, NS_TYPE_CONSENSUS); - tor_assert(sigs); - ns_detached_signatures_free(sigs); tor_assert(v); networkstatus_vote_free(v); } - tor_free(pending_consensus_signatures); - pending_consensus_signatures = new_detached; *msg_out = "Signatures added"; + tor_free(new_signatures); } else if (r == 0) { *msg_out = "Signatures ignored"; } else { @@ -2096,8 +3299,62 @@ dirvote_add_signatures_to_pending_consensus( if (!*msg_out) *msg_out = "Unrecognized error while adding detached signatures."; done: - if (sigs) - ns_detached_signatures_free(sigs); + return r; +} + +static int +dirvote_add_signatures_to_all_pending_consensuses( + const char *detached_signatures_body, + const char **msg_out) +{ + int r=0, i, n_added = 0, errors = 0; + ns_detached_signatures_t *sigs; + tor_assert(detached_signatures_body); + tor_assert(msg_out); + tor_assert(pending_consensus_signatures); + + if (!(sigs = networkstatus_parse_detached_signatures( + detached_signatures_body, NULL))) { + *msg_out = "Couldn't parse detached signatures."; + goto err; + } + + for (i = 0; i < N_CONSENSUS_FLAVORS; ++i) { + int res; + pending_consensus_t *pc = &pending_consensuses[i]; + if (!pc->consensus) + continue; + res = dirvote_add_signatures_to_pending_consensus(pc, sigs, msg_out); + if (res < 0) + errors++; + else + n_added += res; + } + + if (errors && !n_added) { + r = -1; + goto err; + } + + if (n_added && pending_consensuses[FLAV_NS].consensus) { + char *new_detached = + get_detached_signatures_from_pending_consensuses( + pending_consensuses, N_CONSENSUS_FLAVORS); + if (new_detached) { + tor_free(pending_consensus_signatures); + pending_consensus_signatures = new_detached; + } + } + + r = n_added; + goto done; + err: + if (!*msg_out) + *msg_out = "Unrecognized error while adding detached signatures."; + done: + ns_detached_signatures_free(sigs); + /* XXXX NM Check how return is used. We can now have an error *and* + signatures added. */ return r; } @@ -2110,10 +3367,10 @@ dirvote_add_signatures(const char *detached_signatures_body, const char *source, const char **msg) { - if (pending_consensus) { + if (pending_consensuses[FLAV_NS].consensus) { log_notice(LD_DIR, "Got a signature from %s. " "Adding it to the pending consensus.", source); - return dirvote_add_signatures_to_pending_consensus( + return dirvote_add_signatures_to_all_pending_consensuses( detached_signatures_body, msg); } else { log_notice(LD_DIR, "Got a signature from %s. " @@ -2132,17 +3389,25 @@ dirvote_add_signatures(const char *detached_signatures_body, static int dirvote_publish_consensus(void) { - /* Can we actually publish it yet? */ - if (!pending_consensus || - networkstatus_check_consensus_signature(pending_consensus, 1)<0) { - log_warn(LD_DIR, "Not enough info to publish pending consensus"); - return -1; - } + int i; + + /* Now remember all the other consensuses as if we were a directory cache. */ + for (i = 0; i < N_CONSENSUS_FLAVORS; ++i) { + pending_consensus_t *pending = &pending_consensuses[i]; + const char *name; + name = networkstatus_get_flavor_name(i); + tor_assert(name); + if (!pending->consensus || + networkstatus_check_consensus_signature(pending->consensus, 1)<0) { + log_warn(LD_DIR, "Not enough info to publish pending %s consensus",name); + continue; + } - if (networkstatus_set_current_consensus(pending_consensus_body, 0)) - log_warn(LD_DIR, "Error publishing consensus"); - else - log_notice(LD_DIR, "Consensus published."); + if (networkstatus_set_current_consensus(pending->body, name, 0)) + log_warn(LD_DIR, "Error publishing %s consensus", name); + else + log_notice(LD_DIR, "Published %s consensus", name); + } return 0; } @@ -2152,20 +3417,16 @@ void dirvote_free_all(void) { dirvote_clear_votes(1); - /* now empty as a result of clear_pending_votes. */ + /* now empty as a result of dirvote_clear_votes(). */ smartlist_free(pending_vote_list); pending_vote_list = NULL; smartlist_free(previous_vote_list); previous_vote_list = NULL; - tor_free(pending_consensus_body); + dirvote_clear_pending_consensuses(); tor_free(pending_consensus_signatures); - if (pending_consensus) { - networkstatus_vote_free(pending_consensus); - pending_consensus = NULL; - } if (pending_consensus_signature_list) { - /* now empty as a result of clear_pending_votes. */ + /* now empty as a result of dirvote_clear_votes(). */ smartlist_free(pending_consensus_signature_list); pending_consensus_signature_list = NULL; } @@ -2177,13 +3438,14 @@ dirvote_free_all(void) /** Return the body of the consensus that we're currently trying to build. */ const char * -dirvote_get_pending_consensus(void) +dirvote_get_pending_consensus(consensus_flavor_t flav) { - return pending_consensus_body; + tor_assert(((int)flav) >= 0 && flav < N_CONSENSUS_FLAVORS); + return pending_consensuses[flav].body; } /** Return the signatures that we know for the consensus that we're currently - * trying to build */ + * trying to build. */ const char * dirvote_get_pending_detached_signatures(void) { @@ -2229,15 +3491,147 @@ dirvote_get_vote(const char *fp, int flags) } else { if (pending_vote_list && include_pending) { SMARTLIST_FOREACH(pending_vote_list, pending_vote_t *, pv, - if (!memcmp(pv->vote->networkstatus_digest, fp, DIGEST_LEN)) + if (!memcmp(pv->vote->digests.d[DIGEST_SHA1], fp, DIGEST_LEN)) return pv->vote_body); } if (previous_vote_list && include_previous) { SMARTLIST_FOREACH(previous_vote_list, pending_vote_t *, pv, - if (!memcmp(pv->vote->networkstatus_digest, fp, DIGEST_LEN)) + if (!memcmp(pv->vote->digests.d[DIGEST_SHA1], fp, DIGEST_LEN)) return pv->vote_body); } } return NULL; } +/** Construct and return a new microdescriptor from a routerinfo <b>ri</b>. + * + * XXX Right now, there is only one way to generate microdescriptors from + * router descriptors. This may change in future consensus methods. If so, + * we'll need an internal way to remember which method we used, and ask for a + * particular method. + **/ +microdesc_t * +dirvote_create_microdescriptor(const routerinfo_t *ri) +{ + microdesc_t *result = NULL; + char *key = NULL, *summary = NULL, *family = NULL; + char buf[1024]; + size_t keylen; + char *out = buf, *end = buf+sizeof(buf); + + if (crypto_pk_write_public_key_to_string(ri->onion_pkey, &key, &keylen)<0) + goto done; + summary = policy_summarize(ri->exit_policy); + if (ri->declared_family) + family = smartlist_join_strings(ri->declared_family, " ", 0, NULL); + + if (tor_snprintf(out, end-out, "onion-key\n%s", key)<0) + goto done; + out += strlen(out); + if (family) { + if (tor_snprintf(out, end-out, "family %s\n", family)<0) + goto done; + out += strlen(out); + } + if (summary && strcmp(summary, "reject 1-65535")) { + if (tor_snprintf(out, end-out, "p %s\n", summary)<0) + goto done; + out += strlen(out); + } + *out = '\0'; /* Make sure it's nul-terminated. This should be a no-op */ + + { + smartlist_t *lst = microdescs_parse_from_string(buf, out, 0, 1); + if (smartlist_len(lst) != 1) { + log_warn(LD_DIR, "We generated a microdescriptor we couldn't parse."); + SMARTLIST_FOREACH(lst, microdesc_t *, md, microdesc_free(md)); + smartlist_free(lst); + goto done; + } + result = smartlist_get(lst, 0); + smartlist_free(lst); + } + + done: + tor_free(key); + tor_free(summary); + tor_free(family); + return result; +} + +/** Cached space-separated string to hold */ +static char *microdesc_consensus_methods = NULL; + +/** Format the appropriate vote line to describe the microdescriptor <b>md</b> + * in a consensus vote document. Write it into the <b>out_len</b>-byte buffer + * in <b>out</b>. Return -1 on failure and the number of characters written + * on success. */ +ssize_t +dirvote_format_microdesc_vote_line(char *out, size_t out_len, + const microdesc_t *md) +{ + char d64[BASE64_DIGEST256_LEN+1]; + if (!microdesc_consensus_methods) { + microdesc_consensus_methods = + make_consensus_method_list(MIN_METHOD_FOR_MICRODESC, + MAX_SUPPORTED_CONSENSUS_METHOD, + ","); + tor_assert(microdesc_consensus_methods); + } + if (digest256_to_base64(d64, md->digest)<0) + return -1; + + if (tor_snprintf(out, out_len, "m %s sha256=%s\n", + microdesc_consensus_methods, d64)<0) + return -1; + + return strlen(out); +} + +/** If <b>vrs</b> has a hash made for the consensus method <b>method</b> with + * the digest algorithm <b>alg</b>, decode it and copy it into + * <b>digest256_out</b> and return 0. Otherwise return -1. */ +int +vote_routerstatus_find_microdesc_hash(char *digest256_out, + const vote_routerstatus_t *vrs, + int method, + digest_algorithm_t alg) +{ + /* XXXX only returns the sha256 method. */ + const vote_microdesc_hash_t *h; + char mstr[64]; + size_t mlen; + char dstr[64]; + + tor_snprintf(mstr, sizeof(mstr), "%d", method); + mlen = strlen(mstr); + tor_snprintf(dstr, sizeof(dstr), " %s=", + crypto_digest_algorithm_get_name(alg)); + + for (h = vrs->microdesc; h; h = h->next) { + const char *cp = h->microdesc_hash_line; + size_t num_len; + /* cp looks like \d+(,\d+)* (digesttype=val )+ . Let's hunt for mstr in + * the first part. */ + while (1) { + num_len = strspn(cp, "1234567890"); + if (num_len == mlen && !memcmp(mstr, cp, mlen)) { + /* This is the line. */ + char buf[BASE64_DIGEST256_LEN+1]; + /* XXXX ignores extraneous stuff if the digest is too long. This + * seems harmless enough, right? */ + cp = strstr(cp, dstr); + if (!cp) + return -1; + cp += strlen(dstr); + strlcpy(buf, cp, sizeof(buf)); + return digest256_from_base64(digest256_out, buf); + } + if (num_len == 0 || cp[num_len] != ',') + break; + cp += num_len + 1; + } + } + return -1; +} + diff --git a/src/or/dirvote.h b/src/or/dirvote.h new file mode 100644 index 000000000..67540a37f --- /dev/null +++ b/src/or/dirvote.h @@ -0,0 +1,90 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file dirvote.h + * \brief Header file for dirvote.c. + **/ + +#ifndef _TOR_DIRVOTE_H +#define _TOR_DIRVOTE_H + +/** Lowest allowable value for VoteSeconds. */ +#define MIN_VOTE_SECONDS 20 +/** Lowest allowable value for DistSeconds. */ +#define MIN_DIST_SECONDS 20 +/** Smallest allowable voting interval. */ +#define MIN_VOTE_INTERVAL 300 + +void dirvote_free_all(void); + +/* vote manipulation */ +char *networkstatus_compute_consensus(smartlist_t *votes, + int total_authorities, + crypto_pk_env_t *identity_key, + crypto_pk_env_t *signing_key, + const char *legacy_identity_key_digest, + crypto_pk_env_t *legacy_signing_key, + consensus_flavor_t flavor); +int networkstatus_add_detached_signatures(networkstatus_t *target, + ns_detached_signatures_t *sigs, + const char **msg_out); +char *networkstatus_get_detached_signatures(smartlist_t *consensuses); +void ns_detached_signatures_free(ns_detached_signatures_t *s); + +/* cert manipulation */ +authority_cert_t *authority_cert_dup(authority_cert_t *cert); + +/* vote scheduling */ +void dirvote_get_preferred_voting_intervals(vote_timing_t *timing_out); +time_t dirvote_get_start_of_next_interval(time_t now, int interval); +void dirvote_recalculate_timing(or_options_t *options, time_t now); +void dirvote_act(or_options_t *options, time_t now); + +/* invoked on timers and by outside triggers. */ +struct pending_vote_t * dirvote_add_vote(const char *vote_body, + const char **msg_out, + int *status_out); +int dirvote_add_signatures(const char *detached_signatures_body, + const char *source, + const char **msg_out); + +/* Item access */ +const char *dirvote_get_pending_consensus(consensus_flavor_t flav); +const char *dirvote_get_pending_detached_signatures(void); +#define DGV_BY_ID 1 +#define DGV_INCLUDE_PENDING 2 +#define DGV_INCLUDE_PREVIOUS 4 +const cached_dir_t *dirvote_get_vote(const char *fp, int flags); +void set_routerstatus_from_routerinfo(routerstatus_t *rs, + routerinfo_t *ri, time_t now, + int naming, int listbadexits, + int listbaddirs); +void router_clear_status_flags(routerinfo_t *ri); +networkstatus_t * +dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, + authority_cert_t *cert); + +microdesc_t *dirvote_create_microdescriptor(const routerinfo_t *ri); +ssize_t dirvote_format_microdesc_vote_line(char *out, size_t out_len, + const microdesc_t *md); + +int vote_routerstatus_find_microdesc_hash(char *digest256_out, + const vote_routerstatus_t *vrs, + int method, + digest_algorithm_t alg); +document_signature_t *voter_get_sig_by_algorithm( + const networkstatus_voter_info_t *voter, + digest_algorithm_t alg); + +#ifdef DIRVOTE_PRIVATE +char *format_networkstatus_vote(crypto_pk_env_t *private_key, + networkstatus_t *v3_ns); +char *dirvote_compute_params(smartlist_t *votes); +#endif + +#endif + diff --git a/src/or/dns.c b/src/or/dns.c index 3bb9c3489..61c8f32c9 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -12,8 +12,70 @@ **/ #include "or.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "control.h" +#include "dns.h" +#include "main.h" +#include "policies.h" +#include "relay.h" +#include "router.h" #include "ht.h" +#ifdef HAVE_EVENT2_DNS_H +#include <event2/event.h> +#include <event2/dns.h> +#else +#include <event.h> #include "eventdns.h" +#ifndef HAVE_EVDNS_SET_DEFAULT_OUTGOING_BIND_ADDRESS +#define HAVE_EVDNS_SET_DEFAULT_OUTGOING_BIND_ADDRESS +#endif +#endif + +#ifndef HAVE_EVENT2_DNS_H +struct evdns_base; +struct evdns_request; +#define evdns_base_new(x,y) tor_malloc(1) +#define evdns_base_clear_nameservers_and_suspend(base) \ + evdns_clear_nameservers_and_suspend() +#define evdns_base_search_clear(base) evdns_search_clear() +#define evdns_base_set_default_outgoing_bind_address(base, a, len) \ + evdns_set_default_outgoing_bind_address((a),(len)) +#define evdns_base_resolv_conf_parse(base, options, fname) \ + evdns_resolv_conf_parse((options), (fname)) +#define evdns_base_count_nameservers(base) \ + evdns_count_nameservers() +#define evdns_base_resume(base) \ + evdns_resume() +#define evdns_base_config_windows_nameservers(base) \ + evdns_config_windows_nameservers() +#define evdns_base_set_option_(base, opt, val) \ + evdns_set_option((opt),(val),DNS_OPTIONS_ALL) +/* Note: our internal eventdns.c, plus Libevent 1.4, used a 1 return to + * signify failure to launch a resolve. Libevent 2.0 uses a -1 return to + * signify a failure on a resolve, though if we're on Libevent 2.0, we should + * have event2/dns.h and never hit these macros. Regardless, 0 is success. */ +#define evdns_base_resolve_ipv4(base, addr, options, cb, ptr) \ + ((evdns_resolve_ipv4((addr), (options), (cb), (ptr))!=0) \ + ? NULL : ((void*)1)) +#define evdns_base_resolve_reverse(base, addr, options, cb, ptr) \ + ((evdns_resolve_reverse((addr), (options), (cb), (ptr))!=0) \ + ? NULL : ((void*)1)) +#define evdns_base_resolve_reverse_ipv6(base, addr, options, cb, ptr) \ + ((evdns_resolve_reverse_ipv6((addr), (options), (cb), (ptr))!=0) \ + ? NULL : ((void*)1)) + +#elif defined(LIBEVENT_VERSION_NUMBER) && LIBEVENT_VERSION_NUMBER < 0x02000303 +#define evdns_base_set_option_(base, opt, val) \ + evdns_base_set_option((base), (opt),(val),DNS_OPTIONS_ALL) + +#else +#define evdns_base_set_option_ evdns_base_set_option + +#endif /** Longest hostname we're willing to resolve. */ #define MAX_ADDRESSLEN 256 @@ -28,6 +90,9 @@ #define DNS_RESOLVE_FAILED_PERMANENT 2 #define DNS_RESOLVE_SUCCEEDED 3 +/** Our evdns_base; this structure handles all our name lookups. */ +static struct evdns_base *the_evdns_base = NULL; + /** Have we currently configured nameservers with eventdns? */ static int nameservers_configured = 0; /** Did our most recent attempt to configure nameservers with eventdns fail? */ @@ -89,6 +154,8 @@ typedef struct cached_resolve_t { uint32_t ttl; /**< What TTL did the nameserver tell us? */ /** Connections that want to know when we get an answer for this resolve. */ pending_connection_t *pending_connections; + /** Position of this element in the heap*/ + int minheap_idx; } cached_resolve_t; static void purge_expired_resolves(time_t now); @@ -211,8 +278,16 @@ dns_reset(void) { or_options_t *options = get_options(); if (! server_mode(options)) { - evdns_clear_nameservers_and_suspend(); - evdns_search_clear(); + + if (!the_evdns_base) { + if (!(the_evdns_base = evdns_base_new(tor_libevent_get_base(), 0))) { + log_err(LD_BUG, "Couldn't create an evdns_base"); + return -1; + } + } + + evdns_base_clear_nameservers_and_suspend(the_evdns_base); + evdns_base_search_clear(the_evdns_base); nameservers_configured = 0; tor_free(resolv_conf_fname); resolv_conf_mtime = 0; @@ -262,6 +337,8 @@ dns_get_expiry_ttl(uint32_t ttl) static void _free_cached_resolve(cached_resolve_t *r) { + if (!r) + return; while (r->pending_connections) { pending_connection_t *victim = r->pending_connections; r->pending_connections = victim->next; @@ -303,6 +380,7 @@ set_expiry(cached_resolve_t *resolve, time_t expires) resolve->expire = expires; smartlist_pqueue_add(cached_resolve_pqueue, _compare_cached_resolves_by_expiry, + STRUCT_OFFSET(cached_resolve_t, minheap_idx), resolve); } @@ -325,8 +403,7 @@ dns_free_all(void) _free_cached_resolve(item); } HT_CLEAR(cache_map, &cache_root); - if (cached_resolve_pqueue) - smartlist_free(cached_resolve_pqueue); + smartlist_free(cached_resolve_pqueue); cached_resolve_pqueue = NULL; tor_free(resolv_conf_fname); } @@ -349,7 +426,8 @@ purge_expired_resolves(time_t now) if (resolve->expire > now) break; smartlist_pqueue_pop(cached_resolve_pqueue, - _compare_cached_resolves_by_expiry); + _compare_cached_resolves_by_expiry, + STRUCT_OFFSET(cached_resolve_t, minheap_idx)); if (resolve->state == CACHE_STATE_PENDING) { log_debug(LD_EXIT, @@ -393,7 +471,7 @@ purge_expired_resolves(time_t now) log_err(LD_BUG, "The expired resolve we purged didn't match any in" " the cache. Tried to purge %s (%p); instead got %s (%p).", resolve->address, (void*)resolve, - removed ? removed->address : "NULL", (void*)remove); + removed ? removed->address : "NULL", (void*)removed); } tor_assert(removed == resolve); } else { @@ -611,7 +689,7 @@ dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, * know the answer. */ if (tor_addr_from_str(&addr, exitconn->_base.address) >= 0) { if (tor_addr_family(&addr) == AF_INET) { - tor_addr_assign(&exitconn->_base.addr, &addr); + tor_addr_copy(&exitconn->_base.addr, &addr); exitconn->address_ttl = DEFAULT_DNS_TTL; return 1; } else { @@ -711,6 +789,7 @@ dns_resolve_impl(edge_connection_t *exitconn, int is_resolve, resolve = tor_malloc_zero(sizeof(cached_resolve_t)); resolve->magic = CACHED_RESOLVE_MAGIC; resolve->state = CACHE_STATE_PENDING; + resolve->minheap_idx = -1; resolve->is_reverse = is_reverse; strlcpy(resolve->address, exitconn->_base.address, sizeof(resolve->address)); @@ -807,7 +886,8 @@ connection_dns_remove(edge_connection_t *conn) tor_free(pend); log_debug(LD_EXIT, "First connection (fd %d) no longer waiting " "for resolve of %s", - conn->_base.s, escaped_safe_str(conn->_base.address)); + conn->_base.s, + escaped_safe_str(conn->_base.address)); return; } else { for ( ; pend->next; pend = pend->next) { @@ -1108,6 +1188,14 @@ configure_nameservers(int force) conf_fname = "/etc/resolv.conf"; #endif + if (!the_evdns_base) { + if (!(the_evdns_base = evdns_base_new(tor_libevent_get_base(), 0))) { + log_err(LD_BUG, "Couldn't create an evdns_base"); + return -1; + } + } + +#ifdef HAVE_EVDNS_SET_DEFAULT_OUTGOING_BIND_ADDRESS if (options->OutboundBindAddress) { tor_addr_t addr; if (tor_addr_from_str(&addr, options->OutboundBindAddress) < 0) { @@ -1118,20 +1206,17 @@ configure_nameservers(int force) struct sockaddr_storage ss; socklen = tor_addr_to_sockaddr(&addr, 0, (struct sockaddr *)&ss, sizeof(ss)); - if (socklen < 0) { + if (socklen <= 0) { log_warn(LD_BUG, "Couldn't convert outbound bind address to sockaddr." " Ignoring."); } else { - evdns_set_default_outgoing_bind_address((struct sockaddr *)&ss, - socklen); + evdns_base_set_default_outgoing_bind_address(the_evdns_base, + (struct sockaddr *)&ss, + socklen); } } } - - if (options->ServerDNSRandomizeCase) - evdns_set_option("randomize-case:", "1", DNS_OPTIONS_ALL); - else - evdns_set_option("randomize-case:", "0", DNS_OPTIONS_ALL); +#endif evdns_set_log_fn(evdns_log_cb); if (conf_fname) { @@ -1146,16 +1231,17 @@ configure_nameservers(int force) return 0; } if (nameservers_configured) { - evdns_search_clear(); - evdns_clear_nameservers_and_suspend(); + evdns_base_search_clear(the_evdns_base); + evdns_base_clear_nameservers_and_suspend(the_evdns_base); } log_info(LD_EXIT, "Parsing resolver configuration in '%s'", conf_fname); - if ((r = evdns_resolv_conf_parse(DNS_OPTIONS_ALL, conf_fname))) { + if ((r = evdns_base_resolv_conf_parse(the_evdns_base, + DNS_OPTIONS_ALL, conf_fname))) { log_warn(LD_EXIT, "Unable to parse '%s', or no nameservers in '%s' (%d)", conf_fname, conf_fname, r); goto err; } - if (evdns_count_nameservers() == 0) { + if (evdns_base_count_nameservers(the_evdns_base) == 0) { log_warn(LD_EXIT, "Unable to find any nameservers in '%s'.", conf_fname); goto err; } @@ -1163,38 +1249,47 @@ configure_nameservers(int force) resolv_conf_fname = tor_strdup(conf_fname); resolv_conf_mtime = st.st_mtime; if (nameservers_configured) - evdns_resume(); + evdns_base_resume(the_evdns_base); } #ifdef MS_WINDOWS else { if (nameservers_configured) { - evdns_search_clear(); - evdns_clear_nameservers_and_suspend(); + evdns_base_search_clear(the_evdns_base); + evdns_base_clear_nameservers_and_suspend(the_evdns_base); } - if (evdns_config_windows_nameservers()) { + if (evdns_base_config_windows_nameservers(the_evdns_base)) { log_warn(LD_EXIT,"Could not config nameservers."); goto err; } - if (evdns_count_nameservers() == 0) { + if (evdns_base_count_nameservers(the_evdns_base) == 0) { log_warn(LD_EXIT, "Unable to find any platform nameservers in " "your Windows configuration."); goto err; } if (nameservers_configured) - evdns_resume(); + evdns_base_resume(the_evdns_base); tor_free(resolv_conf_fname); resolv_conf_mtime = 0; } #endif - if (evdns_count_nameservers() == 1) { - evdns_set_option("max-timeouts:", "16", DNS_OPTIONS_ALL); - evdns_set_option("timeout:", "10", DNS_OPTIONS_ALL); +#define SET(k,v) evdns_base_set_option_(the_evdns_base, (k), (v)) + + if (evdns_base_count_nameservers(the_evdns_base) == 1) { + SET("max-timeouts:", "16"); + SET("timeout:", "10"); } else { - evdns_set_option("max-timeouts:", "3", DNS_OPTIONS_ALL); - evdns_set_option("timeout:", "5", DNS_OPTIONS_ALL); + SET("max-timeouts:", "3"); + SET("timeout:", "5"); } + if (options->ServerDNSRandomizeCase) + SET("randomize-case:", "1"); + else + SET("randomize-case:", "0"); + +#undef SET + dns_servers_relaunch_checks(); nameservers_configured = 1; @@ -1292,6 +1387,7 @@ static int launch_resolve(edge_connection_t *exitconn) { char *addr = tor_strdup(exitconn->_base.address); + struct evdns_request *req = NULL; tor_addr_t a; int r; int options = get_options()->ServerDNSSearchDomains ? 0 @@ -1307,28 +1403,34 @@ launch_resolve(edge_connection_t *exitconn) r = tor_addr_parse_reverse_lookup_name( &a, exitconn->_base.address, AF_UNSPEC, 0); + + tor_assert(the_evdns_base); if (r == 0) { log_info(LD_EXIT, "Launching eventdns request for %s", escaped_safe_str(exitconn->_base.address)); - r = evdns_resolve_ipv4(exitconn->_base.address, options, - evdns_callback, addr); + req = evdns_base_resolve_ipv4(the_evdns_base, + exitconn->_base.address, options, + evdns_callback, addr); } else if (r == 1) { log_info(LD_EXIT, "Launching eventdns reverse request for %s", escaped_safe_str(exitconn->_base.address)); if (tor_addr_family(&a) == AF_INET) - r = evdns_resolve_reverse(tor_addr_to_in(&a), DNS_QUERY_NO_SEARCH, + req = evdns_base_resolve_reverse(the_evdns_base, + tor_addr_to_in(&a), DNS_QUERY_NO_SEARCH, evdns_callback, addr); else - r = evdns_resolve_reverse_ipv6(tor_addr_to_in6(&a), DNS_QUERY_NO_SEARCH, + req = evdns_base_resolve_reverse_ipv6(the_evdns_base, + tor_addr_to_in6(&a), DNS_QUERY_NO_SEARCH, evdns_callback, addr); } else if (r == -1) { log_warn(LD_BUG, "Somehow a malformed in-addr.arpa address reached here."); } - if (r) { - log_warn(LD_EXIT, "eventdns rejected address %s: error %d.", - escaped_safe_str(addr), r); - r = evdns_err_is_transient(r) ? -2 : -1; + r = 0; + if (!req) { + log_warn(LD_EXIT, "eventdns rejected address %s.", + escaped_safe_str(addr)); + r = -1; tor_free(addr); /* There is no evdns request in progress; stop * addr from getting leaked. */ } @@ -1449,8 +1551,8 @@ evdns_wildcard_check_callback(int result, char type, int count, int ttl, } log(dns_wildcard_one_notice_given ? LOG_INFO : LOG_NOTICE, LD_EXIT, "Your DNS provider gave an answer for \"%s\", which " - "is not supposed to exist. Apparently they are hijacking " - "DNS failures. Trying to correct for this. We've noticed %d " + "is not supposed to exist. Apparently they are hijacking " + "DNS failures. Trying to correct for this. We've noticed %d " "possibly bad address%s so far.", string_address, strmap_size(dns_wildcard_response_count), (strmap_size(dns_wildcard_response_count) == 1) ? "" : "es"); @@ -1466,17 +1568,20 @@ static void launch_wildcard_check(int min_len, int max_len, const char *suffix) { char *addr; - int r; + struct evdns_request *req; addr = crypto_random_hostname(min_len, max_len, "", suffix); log_info(LD_EXIT, "Testing whether our DNS server is hijacking nonexistent " "domains with request for bogus hostname \"%s\"", addr); - r = evdns_resolve_ipv4(/* This "addr" tells us which address to resolve */ + tor_assert(the_evdns_base); + req = evdns_base_resolve_ipv4( + the_evdns_base, + /* This "addr" tells us which address to resolve */ addr, DNS_QUERY_NO_SEARCH, evdns_wildcard_check_callback, /* This "addr" is an argument to the callback*/ addr); - if (r) { + if (!req) { /* There is no evdns request in progress; stop addr from getting leaked */ tor_free(addr); } @@ -1488,6 +1593,7 @@ static void launch_test_addresses(int fd, short event, void *args) { or_options_t *options = get_options(); + struct evdns_request *req; (void)fd; (void)event; (void)args; @@ -1499,14 +1605,19 @@ launch_test_addresses(int fd, short event, void *args) * be an exit server.*/ if (!options->ServerDNSTestAddresses) return; - SMARTLIST_FOREACH(options->ServerDNSTestAddresses, const char *, address, - { - int r = evdns_resolve_ipv4(address, DNS_QUERY_NO_SEARCH, evdns_callback, - tor_strdup(address)); - if (r) - log_info(LD_EXIT, "eventdns rejected test address %s: error %d", - escaped_safe_str(address), r); - }); + tor_assert(the_evdns_base); + SMARTLIST_FOREACH_BEGIN(options->ServerDNSTestAddresses, + const char *, address) { + char *a = tor_strdup(address); + req = evdns_base_resolve_ipv4(the_evdns_base, + address, DNS_QUERY_NO_SEARCH, evdns_callback, a); + + if (!req) { + log_info(LD_EXIT, "eventdns rejected test address %s", + escaped_safe_str(address)); + tor_free(a); + } + } SMARTLIST_FOREACH_END(address); } #define N_WILDCARD_CHECKS 2 @@ -1547,7 +1658,7 @@ dns_launch_wildcard_checks(void) void dns_launch_correctness_checks(void) { - static struct event launch_event; + static struct event *launch_event = NULL; struct timeval timeout; if (!get_options()->ServerDNSDetectHijacking) return; @@ -1555,10 +1666,12 @@ dns_launch_correctness_checks(void) /* Wait a while before launching requests for test addresses, so we can * get the results from checking for wildcarding. */ - evtimer_set(&launch_event, launch_test_addresses, NULL); + if (! launch_event) + launch_event = tor_evtimer_new(tor_libevent_get_base(), + launch_test_addresses, NULL); timeout.tv_sec = 30; timeout.tv_usec = 0; - if (evtimer_add(&launch_event, &timeout)<0) { + if (evtimer_add(launch_event, &timeout)<0) { log_warn(LD_BUG, "Couldn't add timer for checking for dns hijacking"); } } @@ -1574,10 +1687,9 @@ dns_seems_to_be_broken(void) void dns_reset_correctness_checks(void) { - if (dns_wildcard_response_count) { - strmap_free(dns_wildcard_response_count, _tor_free); - dns_wildcard_response_count = NULL; - } + strmap_free(dns_wildcard_response_count, _tor_free); + dns_wildcard_response_count = NULL; + n_wildcard_requests = 0; if (dns_wildcard_list) { @@ -1622,6 +1734,30 @@ assert_resolve_ok(cached_resolve_t *resolve) } } +/** Return the number of DNS cache entries as an int */ +static int +dns_cache_entry_count(void) +{ + return HT_SIZE(&cache_root); +} + +/** Log memory information about our internal DNS cache at level 'severity'. */ +void +dump_dns_mem_usage(int severity) +{ + /* This should never be larger than INT_MAX. */ + int hash_count = dns_cache_entry_count(); + size_t hash_mem = sizeof(struct cached_resolve_t) * hash_count; + hash_mem += HT_MEM_USAGE(&cache_root); + + /* Print out the count and estimated size of our &cache_root. It undercounts + hostnames in cached reverse resolves. + */ + log(severity, LD_MM, "Our DNS cache has %d entries.", hash_count); + log(severity, LD_MM, "Our DNS cache size is approximately %u bytes.", + (unsigned)hash_mem); +} + #ifdef DEBUG_DNS_CACHE /** Exit with an assertion if the DNS cache is corrupt. */ static void @@ -1642,7 +1778,8 @@ _assert_cache_ok(void) return; smartlist_pqueue_assert_ok(cached_resolve_pqueue, - _compare_cached_resolves_by_expiry); + _compare_cached_resolves_by_expiry, + STRUCT_OFFSET(cached_resolve_t, minheap_idx)); SMARTLIST_FOREACH(cached_resolve_pqueue, cached_resolve_t *, res, { diff --git a/src/or/dns.h b/src/or/dns.h new file mode 100644 index 000000000..25ff86e2c --- /dev/null +++ b/src/or/dns.h @@ -0,0 +1,31 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file dns.h + * \brief Header file for dns.c. + **/ + +#ifndef _TOR_DNS_H +#define _TOR_DNS_H + +int dns_init(void); +int has_dns_init_failed(void); +void dns_free_all(void); +uint32_t dns_clip_ttl(uint32_t ttl); +int dns_reset(void); +void connection_dns_remove(edge_connection_t *conn); +void assert_connection_edge_not_dns_pending(edge_connection_t *conn); +void assert_all_pending_dns_resolves_ok(void); +void dns_cancel_pending_resolve(const char *question); +int dns_resolve(edge_connection_t *exitconn); +void dns_launch_correctness_checks(void); +int dns_seems_to_be_broken(void); +void dns_reset_correctness_checks(void); +void dump_dns_mem_usage(int severity); + +#endif + diff --git a/src/or/dnsserv.c b/src/or/dnsserv.c index f8b5c3cdd..243b730cb 100644 --- a/src/or/dnsserv.c +++ b/src/or/dnsserv.c @@ -9,7 +9,21 @@ **/ #include "or.h" +#include "dnsserv.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "control.h" +#include "main.h" +#include "policies.h" +#ifdef HAVE_EVENT2_DNS_H +#include <event2/dns.h> +#include <event2/dns_compat.h> +/* XXXX023 this implies we want an improved evdns */ +#include <event2/dns_struct.h> +#else #include "eventdns.h" +#endif /** Helper function: called by evdns whenever the client sends a request to our * DNSPort. We need to eventually answer the request <b>req</b>. @@ -85,12 +99,7 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) evdns_server_request_respond(req, DNS_ERR_NONE); return; } - if (q->type == EVDNS_TYPE_A) { - /* Refuse any attempt to resolve a noconnect address, right now. */ - if (hostname_is_noconnect_address(q->name)) { - err = DNS_ERR_REFUSED; - } - } else { + if (q->type != EVDNS_TYPE_A) { tor_assert(q->type == EVDNS_TYPE_PTR); } @@ -132,17 +141,18 @@ evdns_server_callback(struct evdns_server_request *req, void *_data) control_event_stream_status(conn, STREAM_EVENT_NEW, 0); - /* Now, throw the connection over to get rewritten (which will answer it - * immediately if it's in the cache, or completely bogus, or automapped), - * and then attached to a circuit. */ + /* Now, unless a controller asked us to leave streams unattached, + * throw the connection over to get rewritten (which will + * answer it immediately if it's in the cache, or completely bogus, or + * automapped), and then attached to a circuit. */ log_info(LD_APP, "Passing request for %s to rewrite_and_attach.", - escaped_safe_str(q->name)); + escaped_safe_str_client(q->name)); q_name = tor_strdup(q->name); /* q could be freed in rewrite_and_attach */ - connection_ap_handshake_rewrite_and_attach(conn, NULL, NULL); + connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); /* Now, the connection is marked if it was bad. */ - log_info(LD_APP, "Passed request for %s to rewrite_and_attach.", - escaped_safe_str(q_name)); + log_info(LD_APP, "Passed request for %s to rewrite_and_attach_if_allowed.", + escaped_safe_str_client(q_name)); tor_free(q_name); } @@ -177,17 +187,18 @@ dnsserv_launch_request(const char *name, int reverse) return -1; } - /* Now, throw the connection over to get rewritten (which will answer it - * immediately if it's in the cache, or completely bogus, or automapped), - * and then attached to a circuit. */ + /* Now, unless a controller asked us to leave streams unattached, + * throw the connection over to get rewritten (which will + * answer it immediately if it's in the cache, or completely bogus, or + * automapped), and then attached to a circuit. */ log_info(LD_APP, "Passing request for %s to rewrite_and_attach.", - escaped_safe_str(name)); + escaped_safe_str_client(name)); q_name = tor_strdup(name); /* q could be freed in rewrite_and_attach */ - connection_ap_handshake_rewrite_and_attach(conn, NULL, NULL); + connection_ap_rewrite_and_attach_if_allowed(conn, NULL, NULL); /* Now, the connection is marked if it was bad. */ - log_info(LD_APP, "Passed request for %s to rewrite_and_attach.", - escaped_safe_str(q_name)); + log_info(LD_APP, "Passed request for %s to rewrite_and_attach_if_allowed.", + escaped_safe_str_client(q_name)); tor_free(q_name); return 0; } @@ -269,7 +280,7 @@ dnsserv_resolved(edge_connection_t *conn, conn->socks_request->command == SOCKS_COMMAND_RESOLVE) { evdns_server_request_add_a_reply(req, name, - 1, (char*)answer, ttl); + 1, answer, ttl); } else if (answer_type == RESOLVED_TYPE_HOSTNAME && answer_len < 256 && conn->socks_request->command == SOCKS_COMMAND_RESOLVE_PTR) { @@ -298,8 +309,8 @@ dnsserv_configure_listener(connection_t *conn) tor_assert(conn->s >= 0); tor_assert(conn->type == CONN_TYPE_AP_DNS_LISTENER); - conn->dns_server_port = evdns_add_server_port(conn->s, 0, - evdns_server_callback, NULL); + conn->dns_server_port = + tor_evdns_add_server_port(conn->s, 0, evdns_server_callback, NULL); } /** Free the evdns server port for <b>conn</b>, which must be an diff --git a/src/or/dnsserv.h b/src/or/dnsserv.h new file mode 100644 index 000000000..fcca86888 --- /dev/null +++ b/src/or/dnsserv.h @@ -0,0 +1,26 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file dnsserv.h + * \brief Header file for dnsserv.c. + **/ + +#ifndef _TOR_DNSSERV_H +#define _TOR_DNSSERV_H + +void dnsserv_configure_listener(connection_t *conn); +void dnsserv_close_listener(connection_t *conn); +void dnsserv_resolved(edge_connection_t *conn, + int answer_type, + size_t answer_len, + const char *answer, + int ttl); +void dnsserv_reject_request(edge_connection_t *conn); +int dnsserv_launch_request(const char *name, int is_reverse); + +#endif + diff --git a/src/or/eventdns.c b/src/or/eventdns.c index 2777f9098..cf583d068 100644 --- a/src/or/eventdns.c +++ b/src/or/eventdns.c @@ -31,6 +31,7 @@ */ #include "eventdns_tor.h" +#include "../common/util.h" #include <sys/types.h> /* #define NDEBUG */ @@ -89,6 +90,7 @@ #include <stdarg.h> #include "eventdns.h" + #ifdef WIN32 #include <windows.h> #include <winsock2.h> @@ -173,8 +175,6 @@ struct evdns_request { /* these objects are kept in a circular list */ struct evdns_request *next, *prev; - u16 timeout_event_deleted; /**< Debugging: where was timeout_event - * deleted? 0 for "it's added." */ struct event timeout_event; u16 trans_id; /* the transaction id */ @@ -214,8 +214,6 @@ struct nameserver { struct event event; /* these objects are kept in a circular list */ struct nameserver *next, *prev; - u16 timeout_event_deleted; /**< Debugging: where was timeout_event - * deleted? 0 for "it's added." */ struct event timeout_event; /* used to keep the timeout for */ /* when we next probe this server. */ /* Valid if state == 0 */ @@ -474,51 +472,10 @@ sockaddr_eq(const struct sockaddr *sa1, const struct sockaddr *sa2, return 1; } -/* for debugging bug 929. XXXX022 */ -static int -_add_timeout_event(u16 *lineno, struct event *ev, struct timeval *to) -{ - *lineno = 0; - return evtimer_add(ev, to); -} -#define add_timeout_event(s, to) \ - (_add_timeout_event(&(s)->timeout_event_deleted, &(s)->timeout_event, (to))) - -/* for debugging bug 929. XXXX022 */ -static int -_del_timeout_event(u16 *lineno, struct event *ev, int line) -{ - if (*lineno) { - log(EVDNS_LOG_DEBUG, - "Duplicate timeout event_del from line %d: first call " - "was at %d.", line, (int)*lineno); - return 0; - } else { - *lineno = (u16)line; - return event_del(ev); - } -} -#define del_timeout_event(s) \ - (_del_timeout_event(&(s)->timeout_event_deleted, &(s)->timeout_event, \ - __LINE__)) -/* For debugging bug 929/957. XXXX022 */ -static int -_del_timeout_event_if_set(u16 *lineno, struct event *ev, int line) -{ - if (*lineno == 0) { - log(EVDNS_LOG_DEBUG, - "Event that I thought was non-added as of line %d " - "was actually added on line %d", - line, (int)*lineno); - *lineno = line; - return event_del(ev); - } - return 0; -} -#define del_timeout_event_if_set(s) \ - _del_timeout_event_if_set(&(s)->timeout_event_deleted, \ - &(s)->timeout_event, \ - __LINE__) +#define add_timeout_event(s, to) \ + (event_add(&(s)->timeout_event, (to))) +#define del_timeout_event(s) \ + (event_del(&(s)->timeout_event)) /* This walks the list of inflight requests to find the */ /* one with a matching transaction id. Returns NULL on */ @@ -555,7 +512,7 @@ static void nameserver_probe_failed(struct nameserver *const ns) { const struct timeval * timeout; del_timeout_event(ns); - CLEAR(&ns->timeout_event); + if (ns->state == 1) { /* This can happen if the nameserver acts in a way which makes us mark */ /* it as bad and then starts sending good replies. */ @@ -567,8 +524,6 @@ nameserver_probe_failed(struct nameserver *const ns) { global_nameserver_timeouts_length - 1)]; ns->failed_times++; - del_timeout_event_if_set(ns); - evtimer_set(&ns->timeout_event, nameserver_prod_callback, ns); if (add_timeout_event(ns, (struct timeval *) timeout) < 0) { log(EVDNS_LOG_WARN, "Error from libevent when adding timer event for %s", @@ -597,8 +552,6 @@ nameserver_failed(struct nameserver *const ns, const char *msg) { ns->state = 0; ns->failed_times = 1; - del_timeout_event_if_set(ns); - evtimer_set(&ns->timeout_event, nameserver_prod_callback, ns); if (add_timeout_event(ns, (struct timeval *) &global_nameserver_timeouts[0]) < 0) { log(EVDNS_LOG_WARN, "Error from libevent when adding timer event for %s", @@ -634,7 +587,6 @@ nameserver_up(struct nameserver *const ns) { log(EVDNS_LOG_WARN, "Nameserver %s is back up", debug_ntop((struct sockaddr *)&ns->address)); del_timeout_event(ns); - CLEAR(&ns->timeout_event); ns->state = 1; ns->failed_times = 0; ns->timedout = 0; @@ -666,7 +618,6 @@ request_finished(struct evdns_request *const req, struct evdns_request **head) { log(EVDNS_LOG_DEBUG, "Removing timeout for request %lx", (unsigned long) req); del_timeout_event(req); - CLEAR(&req->timeout_event); search_request_finished(req); global_requests_inflight--; @@ -1294,7 +1245,8 @@ nameserver_read(struct nameserver *ns) { for (;;) { const int r = - (int)recvfrom(ns->socket, packet, (socklen_t)sizeof(packet), 0, + (int)recvfrom(ns->socket, (void*)packet, + (socklen_t)sizeof(packet), 0, sa, &addrlen); if (r < 0) { int err = last_error(ns->socket); @@ -1325,7 +1277,7 @@ server_port_read(struct evdns_server_port *s) { for (;;) { addrlen = (socklen_t)sizeof(struct sockaddr_storage); - r = recvfrom(s->socket, packet, sizeof(packet), 0, + r = recvfrom(s->socket, (void*)packet, sizeof(packet), 0, (struct sockaddr*) &addr, &addrlen); if (r < 0) { int err = last_error(s->socket); @@ -1342,8 +1294,8 @@ server_port_read(struct evdns_server_port *s) { static void server_port_flush(struct evdns_server_port *port) { - while (port->pending_replies) { - struct server_request *req = port->pending_replies; + struct server_request *req = port->pending_replies; + while (req) { ssize_t r = sendto(port->socket, req->response, req->response_len, 0, (struct sockaddr*) &req->addr, (socklen_t)req->addrlen); if (r < 0) { @@ -1355,6 +1307,9 @@ server_port_flush(struct evdns_server_port *port) if (server_request_free(req)) { /* we released the last reference to req->port. */ return; + } else { + assert(port->pending_replies != req); + req = port->pending_replies; } } @@ -1713,7 +1668,7 @@ evdns_server_request_add_reply(struct evdns_server_request *_req, int section, c /* exported function */ int -evdns_server_request_add_a_reply(struct evdns_server_request *req, const char *name, int n, void *addrs, int ttl) +evdns_server_request_add_a_reply(struct evdns_server_request *req, const char *name, int n, const void *addrs, int ttl) { return evdns_server_request_add_reply( req, EVDNS_ANSWER_SECTION, name, TYPE_A, CLASS_INET, @@ -1722,7 +1677,7 @@ evdns_server_request_add_a_reply(struct evdns_server_request *req, const char *n /* exported function */ int -evdns_server_request_add_aaaa_reply(struct evdns_server_request *req, const char *name, int n, void *addrs, int ttl) +evdns_server_request_add_aaaa_reply(struct evdns_server_request *req, const char *name, int n, const void *addrs, int ttl) { return evdns_server_request_add_reply( req, EVDNS_ANSWER_SECTION, name, TYPE_AAAA, CLASS_INET, @@ -2044,9 +1999,8 @@ evdns_request_timeout_callback(int fd, short events, void *arg) { /* retransmit it */ /* Stop waiting for the timeout. No need to do this in * request_finished; that one already deletes the timeout event. - * XXXX021 port this change to libevent. */ + * XXXX023 port this change to libevent. */ del_timeout_event(req); - CLEAR(&req->timeout_event); evdns_request_transmit(req); } } @@ -2059,7 +2013,8 @@ evdns_request_timeout_callback(int fd, short events, void *arg) { /* 2 other failure */ static int evdns_request_transmit_to(struct evdns_request *req, struct nameserver *server) { - const ssize_t r = send(server->socket, req->request, req->request_len, 0); + const ssize_t r = send(server->socket, (void*)req->request, + req->request_len, 0); if (r < 0) { int err = last_error(server->socket); if (error_is_eagain(err)) return 1; @@ -2109,8 +2064,7 @@ evdns_request_transmit(struct evdns_request *req) { /* transmitted; we need to check for timeout. */ log(EVDNS_LOG_DEBUG, "Setting timeout for request %lx", (unsigned long) req); - del_timeout_event_if_set(req); - evtimer_set(&req->timeout_event, evdns_request_timeout_callback, req); + if (add_timeout_event(req, &global_timeout) < 0) { log(EVDNS_LOG_WARN, "Error from libevent when adding timer for request %lx", @@ -2225,7 +2179,6 @@ evdns_clear_nameservers_and_suspend(void) (void) event_del(&server->event); CLEAR(&server->event); del_timeout_event(server); - CLEAR(&server->timeout_event); if (server->socket >= 0) CLOSE_SOCKET(server->socket); CLEAR(server); @@ -2243,7 +2196,6 @@ evdns_clear_nameservers_and_suspend(void) req->ns = NULL; /* ???? What to do about searches? */ del_timeout_event(req); - CLEAR(&req->timeout_event); req->trans_id = 0; req->transmit_me = 0; @@ -2292,6 +2244,21 @@ evdns_resume(void) } static int +sockaddr_is_loopback(const struct sockaddr *addr) +{ + static const char LOOPBACK_S6[16] = + "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1"; + if (addr->sa_family == AF_INET) { + struct sockaddr_in *sin = (struct sockaddr_in *)addr; + return (ntohl(sin->sin_addr.s_addr) & 0xff000000) == 0x7f000000; + } else if (addr->sa_family == AF_INET6) { + struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)addr; + return !memcmp(sin6->sin6_addr.s6_addr, LOOPBACK_S6, 16); + } + return 0; +} + +static int _evdns_nameserver_add_impl(const struct sockaddr *address, socklen_t addrlen) { /* first check to see if we already have this nameserver */ @@ -2318,7 +2285,8 @@ _evdns_nameserver_add_impl(const struct sockaddr *address, if (!ns) return -1; memset(ns, 0, sizeof(struct nameserver)); - ns->timeout_event_deleted = __LINE__; + + evtimer_set(&ns->timeout_event, nameserver_prod_callback, ns); ns->socket = socket(PF_INET, SOCK_DGRAM, 0); if (ns->socket < 0) { err = 1; goto out1; } @@ -2331,7 +2299,8 @@ _evdns_nameserver_add_impl(const struct sockaddr *address, fcntl(ns->socket, F_SETFL, O_NONBLOCK); #endif - if (global_bind_addr_is_set) { + if (global_bind_addr_is_set && + !sockaddr_is_loopback((struct sockaddr*)&global_bind_address)) { if (bind(ns->socket, (struct sockaddr *)&global_bind_address, global_bind_addrlen) < 0) { log(EVDNS_LOG_DEBUG, "Couldn't bind to outgoing address."); @@ -2553,7 +2522,8 @@ request_new(int type, const char *name, int flags, } memset(req, 0, sizeof(struct evdns_request)); - req->timeout_event_deleted = __LINE__; + + evtimer_set(&req->timeout_event, evdns_request_timeout_callback, req); if (global_randomize_case) { unsigned i; @@ -2941,14 +2911,6 @@ evdns_resolv_set_defaults(int flags) { if (flags & DNS_OPTION_NAMESERVERS) evdns_nameserver_ip_add("127.0.0.1"); } -#ifndef HAVE_STRTOK_R -static char * -strtok_r(char *s, const char *delim, char **state) { - (void)state; - return strtok(s, delim); -} -#endif - /* helper version of atoi which returns -1 on error */ static int strtoint(const char *const str) { @@ -3025,9 +2987,9 @@ static void resolv_conf_parse_line(char *const start, int flags) { char *strtok_state; static const char *const delims = " \t"; -#define NEXT_TOKEN strtok_r(NULL, delims, &strtok_state) +#define NEXT_TOKEN tor_strtok_r(NULL, delims, &strtok_state) - char *const first_token = strtok_r(start, delims, &strtok_state); + char *const first_token = tor_strtok_r(start, delims, &strtok_state); if (!first_token) return; if (!strcmp(first_token, "nameserver") && (flags & DNS_OPTION_NAMESERVERS)) { @@ -3171,14 +3133,13 @@ load_nameservers_with_getnetworkparams(void) IP_ADDR_STRING *ns; GetNetworkParams_fn_t fn; - /* XXXX Possibly, we should hardcode the location of this DLL. */ - if (!(handle = LoadLibrary("iphlpapi.dll"))) { + if (!(handle = load_windows_system_library(TEXT("iphlpapi.dll")))) { log(EVDNS_LOG_WARN, "Could not open iphlpapi.dll"); /* right now status = 0, doesn't that mean "good" - mikec */ status = -1; goto done; } - if (!(fn = (GetNetworkParams_fn_t) GetProcAddress(handle, "GetNetworkParams"))) { + if (!(fn = (GetNetworkParams_fn_t) GetProcAddress(handle, TEXT("GetNetworkParams")))) { log(EVDNS_LOG_WARN, "Could not get address of function."); /* same as above */ status = -1; @@ -3241,9 +3202,10 @@ load_nameservers_with_getnetworkparams(void) } static int -config_nameserver_from_reg_key(HKEY key, const char *subkey) +config_nameserver_from_reg_key(HKEY key, const TCHAR *subkey) { char *buf; + char ansibuf[MAX_PATH] = {0}; DWORD bufsz = 0, type = 0; int status = 0; @@ -3255,24 +3217,30 @@ config_nameserver_from_reg_key(HKEY key, const char *subkey) if (RegQueryValueEx(key, subkey, 0, &type, (LPBYTE)buf, &bufsz) == ERROR_SUCCESS && bufsz > 1) { - status = evdns_nameserver_ip_add_line(buf); + wcstombs(ansibuf,(wchar_t*)buf,MAX_PATH);/*XXXX UNICODE */ + status = evdns_nameserver_ip_add_line(ansibuf); } mm_free(buf); return status; } -#define SERVICES_KEY "System\\CurrentControlSet\\Services\\" -#define WIN_NS_9X_KEY SERVICES_KEY "VxD\\MSTCP" -#define WIN_NS_NT_KEY SERVICES_KEY "Tcpip\\Parameters" +#define SERVICES_KEY TEXT("System\\CurrentControlSet\\Services\\") +#define WIN_NS_9X_KEY SERVICES_KEY TEXT("VxD\\MSTCP") +#define WIN_NS_NT_KEY SERVICES_KEY TEXT("Tcpip\\Parameters") static int load_nameservers_from_registry(void) { int found = 0; int r; + OSVERSIONINFO info; + memset(&info, 0, sizeof(info)); + info.dwOSVersionInfoSize = sizeof (info); + GetVersionEx(&info); + #define TRY(k, name) \ - if (!found && config_nameserver_from_reg_key(k,name) == 0) { \ + if (!found && config_nameserver_from_reg_key(k,TEXT(name)) == 0) { \ log(EVDNS_LOG_DEBUG,"Found nameservers in %s/%s",#k,name); \ found = 1; \ } else if (!found) { \ @@ -3280,7 +3248,7 @@ load_nameservers_from_registry(void) #k,#name); \ } - if (((int)GetVersion()) > 0) { /* NT */ + if (info.dwMajorVersion >= 5) { /* NT */ HKEY nt_key = 0, interfaces_key = 0; if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, WIN_NS_NT_KEY, 0, @@ -3288,7 +3256,7 @@ load_nameservers_from_registry(void) log(EVDNS_LOG_DEBUG,"Couldn't open nt key, %d",(int)GetLastError()); return -1; } - r = RegOpenKeyEx(nt_key, "Interfaces", 0, + r = RegOpenKeyEx(nt_key, TEXT("Interfaces"), 0, KEY_QUERY_VALUE|KEY_ENUMERATE_SUB_KEYS, &interfaces_key); if (r != ERROR_SUCCESS) { @@ -3384,8 +3352,7 @@ evdns_shutdown(int fail_requests) if (server->socket >= 0) CLOSE_SOCKET(server->socket); (void) event_del(&server->event); - if (server->state == 0) - del_timeout_event(server); + del_timeout_event(server); CLEAR(server); mm_free(server); if (server_next == server_head) diff --git a/src/or/eventdns.h b/src/or/eventdns.h index bf3b64d08..2fe4ac937 100644 --- a/src/or/eventdns.h +++ b/src/or/eventdns.h @@ -323,8 +323,8 @@ struct evdns_server_port *evdns_add_server_port(int socket, int is_tcp, evdns_re void evdns_close_server_port(struct evdns_server_port *port); int evdns_server_request_add_reply(struct evdns_server_request *req, int section, const char *name, int type, int class, int ttl, int datalen, int is_name, const char *data); -int evdns_server_request_add_a_reply(struct evdns_server_request *req, const char *name, int n, void *addrs, int ttl); -int evdns_server_request_add_aaaa_reply(struct evdns_server_request *req, const char *name, int n, void *addrs, int ttl); +int evdns_server_request_add_a_reply(struct evdns_server_request *req, const char *name, int n, const void *addrs, int ttl); +int evdns_server_request_add_aaaa_reply(struct evdns_server_request *req, const char *name, int n, const void *addrs, int ttl); int evdns_server_request_add_ptr_reply(struct evdns_server_request *req, struct in_addr *in, const char *inaddr_name, const char *hostname, int ttl); int evdns_server_request_add_cname_reply(struct evdns_server_request *req, const char *name, const char *cname, int ttl); diff --git a/src/or/geoip.c b/src/or/geoip.c index 360b65f41..5bb2410a7 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -3,15 +3,23 @@ /** * \file geoip.c - * \brief Functions related to maintaining an IP-to-country database and to - * summarizing client connections by country. + * \brief Functions related to maintaining an IP-to-country database; + * to summarizing client connections by country to entry guards, bridges, + * and directory servers; and for statistics on answering network status + * requests. */ #define GEOIP_PRIVATE #include "or.h" #include "ht.h" +#include "config.h" +#include "control.h" +#include "dnsserv.h" +#include "geoip.h" +#include "routerlist.h" static void clear_geoip_db(void); +static void init_geoip_countries(void); /** An entry from the GeoIP file: maps an IP range to a country. */ typedef struct geoip_entry_t { @@ -20,16 +28,11 @@ typedef struct geoip_entry_t { intptr_t country; /**< An index into geoip_countries */ } geoip_entry_t; -/** For how many periods should we remember per-country request history? */ -#define REQUEST_HIST_LEN 3 -/** How long are the periods for which we should remember request history? */ -#define REQUEST_HIST_PERIOD (8*60*60) - /** A per-country record for GeoIP request history. */ typedef struct geoip_country_t { char countrycode[3]; - uint32_t n_v2_ns_requests[REQUEST_HIST_LEN]; - uint32_t n_v3_ns_requests[REQUEST_HIST_LEN]; + uint32_t n_v2_ns_requests; + uint32_t n_v3_ns_requests; } geoip_country_t; /** A list of geoip_country_t */ @@ -42,7 +45,7 @@ static strmap_t *country_idxplus1_by_lc_code = NULL; static smartlist_t *geoip_entries = NULL; /** Return the index of the <b>country</b>'s entry in the GeoIP DB - * if it is a valid 2-letter country code, otherwise return zero. + * if it is a valid 2-letter country code, otherwise return -1. */ country_t geoip_get_country(const char *country) @@ -101,11 +104,11 @@ geoip_parse_entry(const char *line) { unsigned int low, high; char b[3]; - if (!geoip_countries) { - geoip_countries = smartlist_create(); + if (!geoip_countries) + init_geoip_countries(); + if (!geoip_entries) geoip_entries = smartlist_create(); - country_idxplus1_by_lc_code = strmap_new(); - } + while (TOR_ISSPACE(*line)) ++line; if (*line == '#') @@ -142,6 +145,7 @@ _geoip_compare_entries(const void **_a, const void **_b) static int _geoip_compare_key_to_entry(const void *_key, const void **_member) { + /* No alignment issue here, since _key really is a pointer to uint32_t */ const uint32_t addr = *(uint32_t *)_key; const geoip_entry_t *entry = *_member; if (addr < entry->ip_low) @@ -160,6 +164,24 @@ should_record_bridge_info(or_options_t *options) return options->BridgeRelay && options->BridgeRecordUsageByCountry; } +/** Set up a new list of geoip countries with no countries (yet) set in it, + * except for the unknown country. + */ +static void +init_geoip_countries(void) +{ + geoip_country_t *geoip_unresolved; + geoip_countries = smartlist_create(); + /* Add a geoip_country_t for requests that could not be resolved to a + * country as first element (index 0) to geoip_countries. */ + geoip_unresolved = tor_malloc_zero(sizeof(geoip_country_t)); + strlcpy(geoip_unresolved->countrycode, "??", + sizeof(geoip_unresolved->countrycode)); + smartlist_add(geoip_countries, geoip_unresolved); + country_idxplus1_by_lc_code = strmap_new(); + strmap_set_lc(country_idxplus1_by_lc_code, "??", (void*)(1)); +} + /** Clear the GeoIP database and reload it from the file * <b>filename</b>. Return 0 on success, -1 on failure. * @@ -185,16 +207,14 @@ geoip_load_file(const char *filename, or_options_t *options) filename, msg); return -1; } - if (!geoip_countries) { - geoip_countries = smartlist_create(); - country_idxplus1_by_lc_code = strmap_new(); - } + if (!geoip_countries) + init_geoip_countries(); if (geoip_entries) { SMARTLIST_FOREACH(geoip_entries, geoip_entry_t *, e, tor_free(e)); smartlist_free(geoip_entries); } geoip_entries = smartlist_create(); - log_notice(LD_GENERAL, "Parsing GEOIP file."); + log_notice(LD_GENERAL, "Parsing GEOIP file %s.", filename); while (!feof(f)) { char buf[512]; if (fgets(buf, (int)sizeof(buf), f) == NULL) @@ -215,9 +235,10 @@ geoip_load_file(const char *filename, or_options_t *options) } /** Given an IP address in host order, return a number representing the - * country to which that address belongs, or -1 for unknown. The return value - * will always be less than geoip_get_n_countries(). To decode it, - * call geoip_get_country_name(). + * country to which that address belongs, -1 for "No geoip information + * available", or 0 for the 'unknown country'. The return value will always + * be less than geoip_get_n_countries(). To decode it, call + * geoip_get_country_name(). */ int geoip_get_country_by_ip(uint32_t ipaddr) @@ -226,13 +247,15 @@ geoip_get_country_by_ip(uint32_t ipaddr) if (!geoip_entries) return -1; ent = smartlist_bsearch(geoip_entries, &ipaddr, _geoip_compare_key_to_entry); - return ent ? (int)ent->country : -1; + return ent ? (int)ent->country : 0; } /** Return the number of countries recognized by the GeoIP database. */ int geoip_get_n_countries(void) { + if (!geoip_countries) + init_geoip_countries(); return (int) smartlist_len(geoip_countries); } @@ -261,23 +284,22 @@ geoip_is_loaded(void) typedef struct clientmap_entry_t { HT_ENTRY(clientmap_entry_t) node; uint32_t ipaddr; - time_t last_seen; /* The last 2 bits of this value hold the client - * operation. */ + /** Time when we last saw this IP address, in MINUTES since the epoch. + * + * (This will run out of space around 4011 CE. If Tor is still in use around + * 4000 CE, please remember to add more bits to last_seen_in_minutes.) */ + unsigned int last_seen_in_minutes:30; + unsigned int action:2; } clientmap_entry_t; -#define ACTION_MASK 3 +/** Largest allowable value for last_seen_in_minutes. (It's a 30-bit field, + * so it can hold up to (1u<<30)-1, or 0x3fffffffu. + */ +#define MAX_LAST_SEEN_IN_MINUTES 0X3FFFFFFFu /** Map from client IP address to last time seen. */ static HT_HEAD(clientmap, clientmap_entry_t) client_history = HT_INITIALIZER(); -/** Time at which we started tracking client IP history. */ -static time_t client_history_starts = 0; - -/** When did the current period of checking per-country request history - * start? */ -static time_t current_request_period_starts = 0; -/** How many older request periods are we remembering? */ -static int n_old_request_periods = 0; /** Hashtable helper: compute a hash of a clientmap_entry_t. */ static INLINE unsigned @@ -289,7 +311,7 @@ clientmap_entry_hash(const clientmap_entry_t *a) static INLINE int clientmap_entries_eq(const clientmap_entry_t *a, const clientmap_entry_t *b) { - return a->ipaddr == b->ipaddr; + return a->ipaddr == b->ipaddr && a->action == b->action; } HT_PROTOTYPE(clientmap, clientmap_entry_t, node, clientmap_entry_hash, @@ -297,8 +319,87 @@ HT_PROTOTYPE(clientmap, clientmap_entry_t, node, clientmap_entry_hash, HT_GENERATE(clientmap, clientmap_entry_t, node, clientmap_entry_hash, clientmap_entries_eq, 0.6, malloc, realloc, free); +/** Clear history of connecting clients used by entry and bridge stats. */ +static void +client_history_clear(void) +{ + clientmap_entry_t **ent, **next, *this; + for (ent = HT_START(clientmap, &client_history); ent != NULL; + ent = next) { + if ((*ent)->action == GEOIP_CLIENT_CONNECT) { + this = *ent; + next = HT_NEXT_RMV(clientmap, &client_history, ent); + tor_free(this); + } else { + next = HT_NEXT(clientmap, &client_history, ent); + } + } +} + +/** How often do we update our estimate which share of v2 and v3 directory + * requests is sent to us? We could as well trigger updates of shares from + * network status updates, but that means adding a lot of calls into code + * that is independent from geoip stats (and keeping them up-to-date). We + * are perfectly fine with an approximation of 15-minute granularity. */ +#define REQUEST_SHARE_INTERVAL (15 * 60) + +/** When did we last determine which share of v2 and v3 directory requests + * is sent to us? */ +static time_t last_time_determined_shares = 0; + +/** Sum of products of v2 shares times the number of seconds for which we + * consider these shares as valid. */ +static double v2_share_times_seconds; + +/** Sum of products of v3 shares times the number of seconds for which we + * consider these shares as valid. */ +static double v3_share_times_seconds; + +/** Number of seconds we are determining v2 and v3 shares. */ +static int share_seconds; + +/** Try to determine which fraction of v2 and v3 directory requests aimed at + * caches will be sent to us at time <b>now</b> and store that value in + * order to take a mean value later on. */ +static void +geoip_determine_shares(time_t now) +{ + double v2_share = 0.0, v3_share = 0.0; + if (router_get_my_share_of_directory_requests(&v2_share, &v3_share) < 0) + return; + if (last_time_determined_shares) { + v2_share_times_seconds += v2_share * + ((double) (now - last_time_determined_shares)); + v3_share_times_seconds += v3_share * + ((double) (now - last_time_determined_shares)); + share_seconds += (int)(now - last_time_determined_shares); + } + last_time_determined_shares = now; +} + +/** Calculate which fraction of v2 and v3 directory requests aimed at caches + * have been sent to us since the last call of this function up to time + * <b>now</b>. Set *<b>v2_share_out</b> and *<b>v3_share_out</b> to the + * fractions of v2 and v3 protocol shares we expect to have seen. Reset + * counters afterwards. Return 0 on success, -1 on failure (e.g. when zero + * seconds have passed since the last call).*/ +static int +geoip_get_mean_shares(time_t now, double *v2_share_out, + double *v3_share_out) +{ + geoip_determine_shares(now); + if (!share_seconds) + return -1; + *v2_share_out = v2_share_times_seconds / ((double) share_seconds); + *v3_share_out = v3_share_times_seconds / ((double) share_seconds); + v2_share_times_seconds = v3_share_times_seconds = 0.0; + share_seconds = 0; + return 0; +} + /** Note that we've seen a client connect from the IP <b>addr</b> (host order) - * at time <b>now</b>. Ignored by all but bridges. */ + * at time <b>now</b>. Ignored by all but bridges and directories if + * configured accordingly. */ void geoip_note_client_seen(geoip_client_action_t action, uint32_t addr, time_t now) @@ -306,72 +407,46 @@ geoip_note_client_seen(geoip_client_action_t action, or_options_t *options = get_options(); clientmap_entry_t lookup, *ent; if (action == GEOIP_CLIENT_CONNECT) { - if (!(options->BridgeRelay && options->BridgeRecordUsageByCountry)) - return; - /* Did we recently switch from bridge to relay or back? */ - if (client_history_starts > now) + /* Only remember statistics as entry guard or as bridge. */ + if (!options->EntryStatistics && + (!(options->BridgeRelay && options->BridgeRecordUsageByCountry))) return; } else { -#ifndef ENABLE_GEOIP_STATS - return; -#else if (options->BridgeRelay || options->BridgeAuthoritativeDir || - !options->DirRecordUsageByCountry) + !options->DirReqStatistics) return; -#endif } - /* Rotate the current request period. */ - while (current_request_period_starts + REQUEST_HIST_PERIOD < now) { - if (!geoip_countries) - geoip_countries = smartlist_create(); - if (!current_request_period_starts) { - current_request_period_starts = now; - break; - } - SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { - memmove(&c->n_v2_ns_requests[0], &c->n_v2_ns_requests[1], - sizeof(uint32_t)*(REQUEST_HIST_LEN-1)); - memmove(&c->n_v3_ns_requests[0], &c->n_v3_ns_requests[1], - sizeof(uint32_t)*(REQUEST_HIST_LEN-1)); - c->n_v2_ns_requests[REQUEST_HIST_LEN-1] = 0; - c->n_v3_ns_requests[REQUEST_HIST_LEN-1] = 0; - }); - current_request_period_starts += REQUEST_HIST_PERIOD; - if (n_old_request_periods < REQUEST_HIST_LEN-1) - ++n_old_request_periods; - } - - /* We use the low 3 bits of the time to encode the action. Since we're - * potentially remembering tons of clients, we don't want to make - * clientmap_entry_t larger than it has to be. */ - now = (now & ~ACTION_MASK) | (((int)action) & ACTION_MASK); lookup.ipaddr = addr; + lookup.action = (int)action; ent = HT_FIND(clientmap, &client_history, &lookup); - if (ent) { - ent->last_seen = now; - } else { + if (! ent) { ent = tor_malloc_zero(sizeof(clientmap_entry_t)); ent->ipaddr = addr; - ent->last_seen = now; + ent->action = (int)action; HT_INSERT(clientmap, &client_history, ent); } + if (now / 60 <= (int)MAX_LAST_SEEN_IN_MINUTES && now >= 0) + ent->last_seen_in_minutes = (unsigned)(now/60); + else + ent->last_seen_in_minutes = 0; if (action == GEOIP_CLIENT_NETWORKSTATUS || action == GEOIP_CLIENT_NETWORKSTATUS_V2) { int country_idx = geoip_get_country_by_ip(addr); + if (country_idx < 0) + country_idx = 0; /** unresolved requests are stored at index 0. */ if (country_idx >= 0 && country_idx < smartlist_len(geoip_countries)) { geoip_country_t *country = smartlist_get(geoip_countries, country_idx); if (action == GEOIP_CLIENT_NETWORKSTATUS) - ++country->n_v3_ns_requests[REQUEST_HIST_LEN-1]; + ++country->n_v3_ns_requests; else - ++country->n_v2_ns_requests[REQUEST_HIST_LEN-1]; + ++country->n_v2_ns_requests; } - } - if (!client_history_starts) { - client_history_starts = now; - current_request_period_starts = now; + /* Periodically determine share of requests that we should see */ + if (last_time_determined_shares + REQUEST_SHARE_INTERVAL < now) + geoip_determine_shares(now); } } @@ -380,8 +455,8 @@ geoip_note_client_seen(geoip_client_action_t action, static int _remove_old_client_helper(struct clientmap_entry_t *ent, void *_cutoff) { - time_t cutoff = *(time_t*)_cutoff; - if (ent->last_seen < cutoff) { + time_t cutoff = *(time_t*)_cutoff / 60; + if (ent->last_seen_in_minutes < cutoff) { tor_free(ent); return 1; } else { @@ -389,18 +464,45 @@ _remove_old_client_helper(struct clientmap_entry_t *ent, void *_cutoff) } } -/** Forget about all clients that haven't connected since <b>cutoff</b>. - * If <b>cutoff</b> is in the future, clients won't be added to the history - * until this time is reached. This is useful to prevent relays that switch - * to bridges from reporting unbelievable numbers of clients. */ +/** Forget about all clients that haven't connected since <b>cutoff</b>. */ void geoip_remove_old_clients(time_t cutoff) { clientmap_HT_FOREACH_FN(&client_history, _remove_old_client_helper, &cutoff); - if (client_history_starts < cutoff) - client_history_starts = cutoff; +} + +/** How many responses are we giving to clients requesting v2 network + * statuses? */ +static uint32_t ns_v2_responses[GEOIP_NS_RESPONSE_NUM]; + +/** How many responses are we giving to clients requesting v3 network + * statuses? */ +static uint32_t ns_v3_responses[GEOIP_NS_RESPONSE_NUM]; + +/** Note that we've rejected a client's request for a v2 or v3 network + * status, encoded in <b>action</b> for reason <b>reason</b> at time + * <b>now</b>. */ +void +geoip_note_ns_response(geoip_client_action_t action, + geoip_ns_response_t response) +{ + static int arrays_initialized = 0; + if (!get_options()->DirReqStatistics) + return; + if (!arrays_initialized) { + memset(ns_v2_responses, 0, sizeof(ns_v2_responses)); + memset(ns_v3_responses, 0, sizeof(ns_v3_responses)); + arrays_initialized = 1; + } + tor_assert(action == GEOIP_CLIENT_NETWORKSTATUS || + action == GEOIP_CLIENT_NETWORKSTATUS_V2); + tor_assert(response < GEOIP_NS_RESPONSE_NUM); + if (action == GEOIP_CLIENT_NETWORKSTATUS) + ns_v3_responses[response]++; + else + ns_v2_responses[response]++; } /** Do not mention any country from which fewer than this number of IPs have @@ -414,13 +516,6 @@ geoip_remove_old_clients(time_t cutoff) * multiple of this value. */ #define IP_GRANULARITY 8 -/** Return the time at which we started recording geoip data. */ -time_t -geoip_get_history_start(void) -{ - return client_history_starts; -} - /** Helper type: used to sort per-country totals by value. */ typedef struct c_hist_t { char country[3]; /**< Two-letter country code. */ @@ -442,97 +537,320 @@ _c_hist_compare(const void **_a, const void **_b) return strcmp(a->country, b->country); } -/** How long do we have to have observed per-country request history before we - * are willing to talk about it? */ -#define GEOIP_MIN_OBSERVATION_TIME (12*60*60) +/** When there are incomplete directory requests at the end of a 24-hour + * period, consider those requests running for longer than this timeout as + * failed, the others as still running. */ +#define DIRREQ_TIMEOUT (10*60) -/** Return the lowest x such that x is at least <b>number</b>, and x modulo - * <b>divisor</b> == 0. */ -static INLINE unsigned -round_to_next_multiple_of(unsigned number, unsigned divisor) +/** Entry in a map from either conn->global_identifier for direct requests + * or a unique circuit identifier for tunneled requests to request time, + * response size, and completion time of a network status request. Used to + * measure download times of requests to derive average client + * bandwidths. */ +typedef struct dirreq_map_entry_t { + HT_ENTRY(dirreq_map_entry_t) node; + /** Unique identifier for this network status request; this is either the + * conn->global_identifier of the dir conn (direct request) or a new + * locally unique identifier of a circuit (tunneled request). This ID is + * only unique among other direct or tunneled requests, respectively. */ + uint64_t dirreq_id; + unsigned int state:3; /**< State of this directory request. */ + unsigned int type:1; /**< Is this a direct or a tunneled request? */ + unsigned int completed:1; /**< Is this request complete? */ + unsigned int action:2; /**< Is this a v2 or v3 request? */ + /** When did we receive the request and started sending the response? */ + struct timeval request_time; + size_t response_size; /**< What is the size of the response in bytes? */ + struct timeval completion_time; /**< When did the request succeed? */ +} dirreq_map_entry_t; + +/** Map of all directory requests asking for v2 or v3 network statuses in + * the current geoip-stats interval. Values are + * of type *<b>dirreq_map_entry_t</b>. */ +static HT_HEAD(dirreqmap, dirreq_map_entry_t) dirreq_map = + HT_INITIALIZER(); + +static int +dirreq_map_ent_eq(const dirreq_map_entry_t *a, + const dirreq_map_entry_t *b) { - number += divisor - 1; - number -= number % divisor; - return number; + return a->dirreq_id == b->dirreq_id && a->type == b->type; } -/** Return a newly allocated comma-separated string containing entries for all - * the countries from which we've seen enough clients connect. The entry - * format is cc=num where num is the number of IPs we've seen connecting from - * that country, and cc is a lowercased country code. Returns NULL if we don't - * want to export geoip data yet. */ -char * -geoip_get_client_history(time_t now, geoip_client_action_t action) +static unsigned +dirreq_map_ent_hash(const dirreq_map_entry_t *entry) +{ + unsigned u = (unsigned) entry->dirreq_id; + u += entry->type << 20; + return u; +} + +HT_PROTOTYPE(dirreqmap, dirreq_map_entry_t, node, dirreq_map_ent_hash, + dirreq_map_ent_eq); +HT_GENERATE(dirreqmap, dirreq_map_entry_t, node, dirreq_map_ent_hash, + dirreq_map_ent_eq, 0.6, malloc, realloc, free); + +/** Helper: Put <b>entry</b> into map of directory requests using + * <b>type</b> and <b>dirreq_id</b> as key parts. If there is + * already an entry for that key, print out a BUG warning and return. */ +static void +_dirreq_map_put(dirreq_map_entry_t *entry, dirreq_type_t type, + uint64_t dirreq_id) +{ + dirreq_map_entry_t *old_ent; + tor_assert(entry->type == type); + tor_assert(entry->dirreq_id == dirreq_id); + + /* XXXX we could switch this to HT_INSERT some time, since it seems that + * this bug doesn't happen. But since this function doesn't seem to be + * critical-path, it's sane to leave it alone. */ + old_ent = HT_REPLACE(dirreqmap, &dirreq_map, entry); + if (old_ent && old_ent != entry) { + log_warn(LD_BUG, "Error when putting directory request into local " + "map. There was already an entry for the same identifier."); + return; + } +} + +/** Helper: Look up and return an entry in the map of directory requests + * using <b>type</b> and <b>dirreq_id</b> as key parts. If there + * is no such entry, return NULL. */ +static dirreq_map_entry_t * +_dirreq_map_get(dirreq_type_t type, uint64_t dirreq_id) +{ + dirreq_map_entry_t lookup; + lookup.type = type; + lookup.dirreq_id = dirreq_id; + return HT_FIND(dirreqmap, &dirreq_map, &lookup); +} + +/** Note that an either direct or tunneled (see <b>type</b>) directory + * request for a network status with unique ID <b>dirreq_id</b> of size + * <b>response_size</b> and action <b>action</b> (either v2 or v3) has + * started. */ +void +geoip_start_dirreq(uint64_t dirreq_id, size_t response_size, + geoip_client_action_t action, dirreq_type_t type) +{ + dirreq_map_entry_t *ent; + if (!get_options()->DirReqStatistics) + return; + ent = tor_malloc_zero(sizeof(dirreq_map_entry_t)); + ent->dirreq_id = dirreq_id; + tor_gettimeofday(&ent->request_time); + ent->response_size = response_size; + ent->action = action; + ent->type = type; + _dirreq_map_put(ent, type, dirreq_id); +} + +/** Change the state of the either direct or tunneled (see <b>type</b>) + * directory request with <b>dirreq_id</b> to <b>new_state</b> and + * possibly mark it as completed. If no entry can be found for the given + * key parts (e.g., if this is a directory request that we are not + * measuring, or one that was started in the previous measurement period), + * or if the state cannot be advanced to <b>new_state</b>, do nothing. */ +void +geoip_change_dirreq_state(uint64_t dirreq_id, dirreq_type_t type, + dirreq_state_t new_state) +{ + dirreq_map_entry_t *ent; + if (!get_options()->DirReqStatistics) + return; + ent = _dirreq_map_get(type, dirreq_id); + if (!ent) + return; + if (new_state == DIRREQ_IS_FOR_NETWORK_STATUS) + return; + if (new_state - 1 != ent->state) + return; + ent->state = new_state; + if ((type == DIRREQ_DIRECT && + new_state == DIRREQ_FLUSHING_DIR_CONN_FINISHED) || + (type == DIRREQ_TUNNELED && + new_state == DIRREQ_OR_CONN_BUFFER_FLUSHED)) { + tor_gettimeofday(&ent->completion_time); + ent->completed = 1; + } +} + +/** Return a newly allocated comma-separated string containing statistics + * on network status downloads. The string contains the number of completed + * requests, timeouts, and still running requests as well as the download + * times by deciles and quartiles. Return NULL if we have not observed + * requests for long enough. */ +static char * +geoip_get_dirreq_history(geoip_client_action_t action, + dirreq_type_t type) { char *result = NULL; - if (!geoip_is_loaded()) + smartlist_t *dirreq_completed = NULL; + uint32_t complete = 0, timeouts = 0, running = 0; + int bufsize = 1024, written; + dirreq_map_entry_t **ptr, **next, *ent; + struct timeval now; + + tor_gettimeofday(&now); + if (action != GEOIP_CLIENT_NETWORKSTATUS && + action != GEOIP_CLIENT_NETWORKSTATUS_V2) return NULL; - if (client_history_starts < (now - GEOIP_MIN_OBSERVATION_TIME)) { - char buf[32]; - smartlist_t *chunks = NULL; - smartlist_t *entries = NULL; - int n_countries = geoip_get_n_countries(); - int i; - clientmap_entry_t **ent; - unsigned *counts = tor_malloc_zero(sizeof(unsigned)*n_countries); - unsigned total = 0; - unsigned granularity = IP_GRANULARITY; -#ifdef ENABLE_GEOIP_STATS - if (get_options()->DirRecordUsageByCountry) - granularity = get_options()->DirRecordUsageGranularity; -#endif - HT_FOREACH(ent, clientmap, &client_history) { - int country; - if (((*ent)->last_seen & ACTION_MASK) != (int)action) - continue; - country = geoip_get_country_by_ip((*ent)->ipaddr); - if (country < 0) - continue; - tor_assert(0 <= country && country < n_countries); - ++counts[country]; - ++total; - } - /* Don't record anything if we haven't seen enough IPs. */ - if (total < MIN_IPS_TO_NOTE_ANYTHING) - goto done; - /* Make a list of c_hist_t */ - entries = smartlist_create(); - for (i = 0; i < n_countries; ++i) { - unsigned c = counts[i]; - const char *countrycode; - c_hist_t *ent; - /* Only report a country if it has a minimum number of IPs. */ - if (c >= MIN_IPS_TO_NOTE_COUNTRY) { - c = round_to_next_multiple_of(c, granularity); - countrycode = geoip_get_country_name(i); - ent = tor_malloc(sizeof(c_hist_t)); - strlcpy(ent->country, countrycode, sizeof(ent->country)); - ent->total = c; - smartlist_add(entries, ent); + dirreq_completed = smartlist_create(); + for (ptr = HT_START(dirreqmap, &dirreq_map); ptr; ptr = next) { + ent = *ptr; + if (ent->action != action || ent->type != type) { + next = HT_NEXT(dirreqmap, &dirreq_map, ptr); + continue; + } else { + if (ent->completed) { + smartlist_add(dirreq_completed, ent); + complete++; + next = HT_NEXT_RMV(dirreqmap, &dirreq_map, ptr); + } else { + if (tv_mdiff(&ent->request_time, &now) / 1000 > DIRREQ_TIMEOUT) + timeouts++; + else + running++; + next = HT_NEXT_RMV(dirreqmap, &dirreq_map, ptr); + tor_free(ent); } } - /* Sort entries. Note that we must do this _AFTER_ rounding, or else - * the sort order could leak info. */ - smartlist_sort(entries, _c_hist_compare); - - /* Build the result. */ - chunks = smartlist_create(); - SMARTLIST_FOREACH(entries, c_hist_t *, ch, { - tor_snprintf(buf, sizeof(buf), "%s=%u", ch->country, ch->total); - smartlist_add(chunks, tor_strdup(buf)); - }); - result = smartlist_join_strings(chunks, ",", 0, NULL); - done: - tor_free(counts); - if (chunks) { - SMARTLIST_FOREACH(chunks, char *, c, tor_free(c)); - smartlist_free(chunks); - } - if (entries) { - SMARTLIST_FOREACH(entries, c_hist_t *, c, tor_free(c)); - smartlist_free(entries); + } +#define DIR_REQ_GRANULARITY 4 + complete = round_uint32_to_next_multiple_of(complete, + DIR_REQ_GRANULARITY); + timeouts = round_uint32_to_next_multiple_of(timeouts, + DIR_REQ_GRANULARITY); + running = round_uint32_to_next_multiple_of(running, + DIR_REQ_GRANULARITY); + result = tor_malloc_zero(bufsize); + written = tor_snprintf(result, bufsize, "complete=%u,timeout=%u," + "running=%u", complete, timeouts, running); + if (written < 0) { + tor_free(result); + goto done; + } + +#define MIN_DIR_REQ_RESPONSES 16 + if (complete >= MIN_DIR_REQ_RESPONSES) { + uint32_t *dltimes; + /* We may have rounded 'completed' up. Here we want to use the + * real value. */ + complete = smartlist_len(dirreq_completed); + dltimes = tor_malloc_zero(sizeof(uint32_t) * complete); + SMARTLIST_FOREACH_BEGIN(dirreq_completed, dirreq_map_entry_t *, ent) { + uint32_t bytes_per_second; + uint32_t time_diff = (uint32_t) tv_mdiff(&ent->request_time, + &ent->completion_time); + if (time_diff == 0) + time_diff = 1; /* Avoid DIV/0; "instant" answers are impossible + * by law of nature or something, but a milisecond + * is a bit greater than "instantly" */ + bytes_per_second = (uint32_t)(1000 * ent->response_size / time_diff); + dltimes[ent_sl_idx] = bytes_per_second; + } SMARTLIST_FOREACH_END(ent); + median_uint32(dltimes, complete); /* sorts as a side effect. */ + written = tor_snprintf(result + written, bufsize - written, + ",min=%u,d1=%u,d2=%u,q1=%u,d3=%u,d4=%u,md=%u," + "d6=%u,d7=%u,q3=%u,d8=%u,d9=%u,max=%u", + dltimes[0], + dltimes[1*complete/10-1], + dltimes[2*complete/10-1], + dltimes[1*complete/4-1], + dltimes[3*complete/10-1], + dltimes[4*complete/10-1], + dltimes[5*complete/10-1], + dltimes[6*complete/10-1], + dltimes[7*complete/10-1], + dltimes[3*complete/4-1], + dltimes[8*complete/10-1], + dltimes[9*complete/10-1], + dltimes[complete-1]); + if (written<0) + tor_free(result); + tor_free(dltimes); + } + done: + SMARTLIST_FOREACH(dirreq_completed, dirreq_map_entry_t *, ent, + tor_free(ent)); + smartlist_free(dirreq_completed); + return result; +} + +/** Return a newly allocated comma-separated string containing entries for + * all the countries from which we've seen enough clients connect as a + * bridge, directory server, or entry guard. The entry format is cc=num + * where num is the number of IPs we've seen connecting from that country, + * and cc is a lowercased country code. Returns NULL if we don't want + * to export geoip data yet. */ +char * +geoip_get_client_history(geoip_client_action_t action) +{ + char *result = NULL; + unsigned granularity = IP_GRANULARITY; + smartlist_t *chunks = NULL; + smartlist_t *entries = NULL; + int n_countries = geoip_get_n_countries(); + int i; + clientmap_entry_t **ent; + unsigned *counts = NULL; + unsigned total = 0; + + if (!geoip_is_loaded()) + return NULL; + + counts = tor_malloc_zero(sizeof(unsigned)*n_countries); + HT_FOREACH(ent, clientmap, &client_history) { + int country; + if ((*ent)->action != (int)action) + continue; + country = geoip_get_country_by_ip((*ent)->ipaddr); + if (country < 0) + country = 0; /** unresolved requests are stored at index 0. */ + tor_assert(0 <= country && country < n_countries); + ++counts[country]; + ++total; + } + /* Don't record anything if we haven't seen enough IPs. */ + if (total < MIN_IPS_TO_NOTE_ANYTHING) + goto done; + /* Make a list of c_hist_t */ + entries = smartlist_create(); + for (i = 0; i < n_countries; ++i) { + unsigned c = counts[i]; + const char *countrycode; + c_hist_t *ent; + /* Only report a country if it has a minimum number of IPs. */ + if (c >= MIN_IPS_TO_NOTE_COUNTRY) { + c = round_to_next_multiple_of(c, granularity); + countrycode = geoip_get_country_name(i); + ent = tor_malloc(sizeof(c_hist_t)); + strlcpy(ent->country, countrycode, sizeof(ent->country)); + ent->total = c; + smartlist_add(entries, ent); } } + /* Sort entries. Note that we must do this _AFTER_ rounding, or else + * the sort order could leak info. */ + smartlist_sort(entries, _c_hist_compare); + + /* Build the result. */ + chunks = smartlist_create(); + SMARTLIST_FOREACH(entries, c_hist_t *, ch, { + char *buf=NULL; + tor_asprintf(&buf, "%s=%u", ch->country, ch->total); + smartlist_add(chunks, buf); + }); + result = smartlist_join_strings(chunks, ",", 0, NULL); + done: + tor_free(counts); + if (chunks) { + SMARTLIST_FOREACH(chunks, char *, c, tor_free(c)); + smartlist_free(chunks); + } + if (entries) { + SMARTLIST_FOREACH(entries, c_hist_t *, c, tor_free(c)); + smartlist_free(entries); + } return result; } @@ -540,18 +858,12 @@ geoip_get_client_history(time_t now, geoip_client_action_t action) * for <b>action</b> in a format suitable for an extra-info document, or NULL * on failure. */ char * -geoip_get_request_history(time_t now, geoip_client_action_t action) +geoip_get_request_history(geoip_client_action_t action) { smartlist_t *entries, *strings; char *result; unsigned granularity = IP_GRANULARITY; -#ifdef ENABLE_GEOIP_STATS - if (get_options()->DirRecordUsageByCountry) - granularity = get_options()->DirRecordUsageGranularity; -#endif - if (client_history_starts >= (now - GEOIP_MIN_OBSERVATION_TIME)) - return NULL; if (action != GEOIP_CLIENT_NETWORKSTATUS && action != GEOIP_CLIENT_NETWORKSTATUS_V2) return NULL; @@ -560,13 +872,10 @@ geoip_get_request_history(time_t now, geoip_client_action_t action) entries = smartlist_create(); SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { - uint32_t *n = (action == GEOIP_CLIENT_NETWORKSTATUS) - ? c->n_v3_ns_requests : c->n_v2_ns_requests; uint32_t tot = 0; - int i; c_hist_t *ent; - for (i=0; i < REQUEST_HIST_LEN; ++i) - tot += n[i]; + tot = (action == GEOIP_CLIENT_NETWORKSTATUS) ? + c->n_v3_ns_requests : c->n_v2_ns_requests; if (!tot) continue; ent = tor_malloc_zero(sizeof(c_hist_t)); @@ -578,9 +887,9 @@ geoip_get_request_history(time_t now, geoip_client_action_t action) strings = smartlist_create(); SMARTLIST_FOREACH(entries, c_hist_t *, ent, { - char buf[32]; - tor_snprintf(buf, sizeof(buf), "%s=%u", ent->country, ent->total); - smartlist_add(strings, tor_strdup(buf)); + char *buf = NULL; + tor_asprintf(&buf, "%s=%u", ent->country, ent->total); + smartlist_add(strings, buf); }); result = smartlist_join_strings(strings, ",", 0, NULL); SMARTLIST_FOREACH(strings, char *, cp, tor_free(cp)); @@ -590,69 +899,450 @@ geoip_get_request_history(time_t now, geoip_client_action_t action) return result; } -/** Store all our geoip statistics into $DATADIR/geoip-stats. */ +/** Start time of directory request stats or 0 if we're not collecting + * directory request statistics. */ +static time_t start_of_dirreq_stats_interval; + +/** Initialize directory request stats. */ void -dump_geoip_stats(void) +geoip_dirreq_stats_init(time_t now) +{ + start_of_dirreq_stats_interval = now; +} + +/** Stop collecting directory request stats in a way that we can re-start + * doing so in geoip_dirreq_stats_init(). */ +void +geoip_dirreq_stats_term(void) +{ + SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { + c->n_v2_ns_requests = c->n_v3_ns_requests = 0; + }); + { + clientmap_entry_t **ent, **next, *this; + for (ent = HT_START(clientmap, &client_history); ent != NULL; + ent = next) { + if ((*ent)->action == GEOIP_CLIENT_NETWORKSTATUS || + (*ent)->action == GEOIP_CLIENT_NETWORKSTATUS_V2) { + this = *ent; + next = HT_NEXT_RMV(clientmap, &client_history, ent); + tor_free(this); + } else { + next = HT_NEXT(clientmap, &client_history, ent); + } + } + } + v2_share_times_seconds = v3_share_times_seconds = 0.0; + last_time_determined_shares = 0; + share_seconds = 0; + memset(ns_v2_responses, 0, sizeof(ns_v2_responses)); + memset(ns_v3_responses, 0, sizeof(ns_v3_responses)); + { + dirreq_map_entry_t **ent, **next, *this; + for (ent = HT_START(dirreqmap, &dirreq_map); ent != NULL; ent = next) { + this = *ent; + next = HT_NEXT_RMV(dirreqmap, &dirreq_map, ent); + tor_free(this); + } + } + start_of_dirreq_stats_interval = 0; +} + +/** Write dirreq statistics to $DATADIR/stats/dirreq-stats and return when + * we would next want to write. */ +time_t +geoip_dirreq_stats_write(time_t now) { -#ifdef ENABLE_GEOIP_STATS - time_t now = time(NULL); - time_t request_start; - char *filename = get_datadir_fname("geoip-stats"); + char *statsdir = NULL, *filename = NULL; char *data_v2 = NULL, *data_v3 = NULL; - char since[ISO_TIME_LEN+1], written[ISO_TIME_LEN+1]; + char written[ISO_TIME_LEN+1]; open_file_t *open_file = NULL; double v2_share = 0.0, v3_share = 0.0; FILE *out; + int i; - data_v2 = geoip_get_client_history(now, GEOIP_CLIENT_NETWORKSTATUS_V2); - data_v3 = geoip_get_client_history(now, GEOIP_CLIENT_NETWORKSTATUS); - format_iso_time(since, geoip_get_history_start()); + if (!start_of_dirreq_stats_interval) + return 0; /* Not initialized. */ + if (start_of_dirreq_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write. */ + + /* Discard all items in the client history that are too old. */ + geoip_remove_old_clients(start_of_dirreq_stats_interval); + + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE) < 0) + goto done; + filename = get_datadir_fname2("stats", "dirreq-stats"); + data_v2 = geoip_get_client_history(GEOIP_CLIENT_NETWORKSTATUS_V2); + data_v3 = geoip_get_client_history(GEOIP_CLIENT_NETWORKSTATUS); format_iso_time(written, now); - out = start_writing_to_stdio_file(filename, OPEN_FLAGS_REPLACE, + out = start_writing_to_stdio_file(filename, OPEN_FLAGS_APPEND, 0600, &open_file); if (!out) goto done; - if (fprintf(out, "written %s\nstarted-at %s\nns-ips %s\nns-v2-ips %s\n", - written, since, + if (fprintf(out, "dirreq-stats-end %s (%d s)\ndirreq-v3-ips %s\n" + "dirreq-v2-ips %s\n", written, + (unsigned) (now - start_of_dirreq_stats_interval), data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) goto done; tor_free(data_v2); tor_free(data_v3); - request_start = current_request_period_starts - - (n_old_request_periods * REQUEST_HIST_PERIOD); - format_iso_time(since, request_start); - data_v2 = geoip_get_request_history(now, GEOIP_CLIENT_NETWORKSTATUS_V2); - data_v3 = geoip_get_request_history(now, GEOIP_CLIENT_NETWORKSTATUS); - if (fprintf(out, "requests-start %s\nn-ns-reqs %s\nn-v2-ns-reqs %s\n", - since, + data_v2 = geoip_get_request_history(GEOIP_CLIENT_NETWORKSTATUS_V2); + data_v3 = geoip_get_request_history(GEOIP_CLIENT_NETWORKSTATUS); + if (fprintf(out, "dirreq-v3-reqs %s\ndirreq-v2-reqs %s\n", data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) goto done; - if (!router_get_my_share_of_directory_requests(&v2_share, &v3_share)) { - if (fprintf(out, "v2-ns-share %0.2lf%%\n", v2_share*100) < 0) + tor_free(data_v2); + tor_free(data_v3); + SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, { + c->n_v2_ns_requests = c->n_v3_ns_requests = 0; + }); +#define RESPONSE_GRANULARITY 8 + for (i = 0; i < GEOIP_NS_RESPONSE_NUM; i++) { + ns_v2_responses[i] = round_uint32_to_next_multiple_of( + ns_v2_responses[i], RESPONSE_GRANULARITY); + ns_v3_responses[i] = round_uint32_to_next_multiple_of( + ns_v3_responses[i], RESPONSE_GRANULARITY); + } +#undef RESPONSE_GRANULARITY + if (fprintf(out, "dirreq-v3-resp ok=%u,not-enough-sigs=%u,unavailable=%u," + "not-found=%u,not-modified=%u,busy=%u\n", + ns_v3_responses[GEOIP_SUCCESS], + ns_v3_responses[GEOIP_REJECT_NOT_ENOUGH_SIGS], + ns_v3_responses[GEOIP_REJECT_UNAVAILABLE], + ns_v3_responses[GEOIP_REJECT_NOT_FOUND], + ns_v3_responses[GEOIP_REJECT_NOT_MODIFIED], + ns_v3_responses[GEOIP_REJECT_BUSY]) < 0) + goto done; + if (fprintf(out, "dirreq-v2-resp ok=%u,unavailable=%u," + "not-found=%u,not-modified=%u,busy=%u\n", + ns_v2_responses[GEOIP_SUCCESS], + ns_v2_responses[GEOIP_REJECT_UNAVAILABLE], + ns_v2_responses[GEOIP_REJECT_NOT_FOUND], + ns_v2_responses[GEOIP_REJECT_NOT_MODIFIED], + ns_v2_responses[GEOIP_REJECT_BUSY]) < 0) + goto done; + memset(ns_v2_responses, 0, sizeof(ns_v2_responses)); + memset(ns_v3_responses, 0, sizeof(ns_v3_responses)); + if (!geoip_get_mean_shares(now, &v2_share, &v3_share)) { + if (fprintf(out, "dirreq-v2-share %0.2lf%%\n", v2_share*100) < 0) goto done; - if (fprintf(out, "v3-ns-share %0.2lf%%\n", v3_share*100) < 0) + if (fprintf(out, "dirreq-v3-share %0.2lf%%\n", v3_share*100) < 0) goto done; } + data_v2 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS_V2, + DIRREQ_DIRECT); + data_v3 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS, + DIRREQ_DIRECT); + if (fprintf(out, "dirreq-v3-direct-dl %s\ndirreq-v2-direct-dl %s\n", + data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) + goto done; + tor_free(data_v2); + tor_free(data_v3); + data_v2 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS_V2, + DIRREQ_TUNNELED); + data_v3 = geoip_get_dirreq_history(GEOIP_CLIENT_NETWORKSTATUS, + DIRREQ_TUNNELED); + if (fprintf(out, "dirreq-v3-tunneled-dl %s\ndirreq-v2-tunneled-dl %s\n", + data_v3 ? data_v3 : "", data_v2 ? data_v2 : "") < 0) + goto done; + finish_writing_to_file(open_file); open_file = NULL; + + start_of_dirreq_stats_interval = now; + done: if (open_file) abort_writing_to_file(open_file); tor_free(filename); + tor_free(statsdir); tor_free(data_v2); tor_free(data_v3); -#endif + return start_of_dirreq_stats_interval + WRITE_STATS_INTERVAL; +} + +/** Start time of bridge stats or 0 if we're not collecting bridge + * statistics. */ +static time_t start_of_bridge_stats_interval; + +/** Initialize bridge stats. */ +void +geoip_bridge_stats_init(time_t now) +{ + start_of_bridge_stats_interval = now; +} + +/** Stop collecting bridge stats in a way that we can re-start doing so in + * geoip_bridge_stats_init(). */ +void +geoip_bridge_stats_term(void) +{ + client_history_clear(); + start_of_bridge_stats_interval = 0; +} + +/** Validate a bridge statistics string as it would be written to a + * current extra-info descriptor. Return 1 if the string is valid and + * recent enough, or 0 otherwise. */ +static int +validate_bridge_stats(const char *stats_str, time_t now) +{ + char stats_end_str[ISO_TIME_LEN+1], stats_start_str[ISO_TIME_LEN+1], + *eos; + + const char *BRIDGE_STATS_END = "bridge-stats-end "; + const char *BRIDGE_IPS = "bridge-ips "; + const char *BRIDGE_IPS_EMPTY_LINE = "bridge-ips\n"; + const char *tmp; + time_t stats_end_time; + int seconds; + tor_assert(stats_str); + + /* Parse timestamp and number of seconds from + "bridge-stats-end YYYY-MM-DD HH:MM:SS (N s)" */ + tmp = find_str_at_start_of_line(stats_str, BRIDGE_STATS_END); + if (!tmp) + return 0; + tmp += strlen(BRIDGE_STATS_END); + + if (strlen(tmp) < ISO_TIME_LEN + 6) + return 0; + strlcpy(stats_end_str, tmp, sizeof(stats_end_str)); + if (parse_iso_time(stats_end_str, &stats_end_time) < 0) + return 0; + if (stats_end_time < now - (25*60*60) || + stats_end_time > now + (1*60*60)) + return 0; + seconds = (int)strtol(tmp + ISO_TIME_LEN + 2, &eos, 10); + if (!eos || seconds < 23*60*60) + return 0; + format_iso_time(stats_start_str, stats_end_time - seconds); + + /* Parse: "bridge-ips CC=N,CC=N,..." */ + tmp = find_str_at_start_of_line(stats_str, BRIDGE_IPS); + if (!tmp) { + /* Look if there is an empty "bridge-ips" line */ + tmp = find_str_at_start_of_line(stats_str, BRIDGE_IPS_EMPTY_LINE); + if (!tmp) + return 0; + } + + return 1; +} + +/** Most recent bridge statistics formatted to be written to extra-info + * descriptors. */ +static char *bridge_stats_extrainfo = NULL; + +/** Return a newly allocated string holding our bridge usage stats by country + * in a format suitable for inclusion in an extrainfo document. Return NULL on + * failure. */ +static char * +format_bridge_stats_extrainfo(time_t now) +{ + char *out = NULL, *data = NULL; + long duration = now - start_of_bridge_stats_interval; + char written[ISO_TIME_LEN+1]; + + if (duration < 0) + return NULL; + + format_iso_time(written, now); + data = geoip_get_client_history(GEOIP_CLIENT_CONNECT); + + tor_asprintf(&out, + "bridge-stats-end %s (%ld s)\n" + "bridge-ips %s\n", + written, duration, + data ? data : ""); + tor_free(data); + + return out; +} + +/** Return a newly allocated string holding our bridge usage stats by country + * in a format suitable for the answer to a controller request. Return NULL on + * failure. */ +static char * +format_bridge_stats_controller(time_t now) +{ + char *out = NULL, *data = NULL; + char started[ISO_TIME_LEN+1]; + (void) now; + + format_iso_time(started, start_of_bridge_stats_interval); + data = geoip_get_client_history(GEOIP_CLIENT_CONNECT); + + tor_asprintf(&out, + "TimeStarted=\"%s\" CountrySummary=%s", + started, data ? data : ""); + tor_free(data); + return out; +} + +/** Write bridge statistics to $DATADIR/stats/bridge-stats and return + * when we should next try to write statistics. */ +time_t +geoip_bridge_stats_write(time_t now) +{ + char *filename = NULL, *val = NULL, *statsdir = NULL; + + /* Check if 24 hours have passed since starting measurements. */ + if (now < start_of_bridge_stats_interval + WRITE_STATS_INTERVAL) + return start_of_bridge_stats_interval + WRITE_STATS_INTERVAL; + + /* Discard all items in the client history that are too old. */ + geoip_remove_old_clients(start_of_bridge_stats_interval); + + /* Generate formatted string */ + val = format_bridge_stats_extrainfo(now); + if (val == NULL) + goto done; + + /* Update the stored value. */ + tor_free(bridge_stats_extrainfo); + bridge_stats_extrainfo = val; + start_of_bridge_stats_interval = now; + + /* Write it to disk. */ + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE) < 0) + goto done; + filename = get_datadir_fname2("stats", "bridge-stats"); + + write_str_to_file(filename, bridge_stats_extrainfo, 0); + + /* Tell the controller, "hey, there are clients!" */ + { + char *controller_str = format_bridge_stats_controller(now); + if (controller_str) + control_event_clients_seen(controller_str); + tor_free(controller_str); + } + done: + tor_free(filename); + tor_free(statsdir); + + return start_of_bridge_stats_interval + WRITE_STATS_INTERVAL; +} + +/** Try to load the most recent bridge statistics from disk, unless we + * have finished a measurement interval lately, and check whether they + * are still recent enough. */ +static void +load_bridge_stats(time_t now) +{ + char *fname, *contents; + if (bridge_stats_extrainfo) + return; + + fname = get_datadir_fname2("stats", "bridge-stats"); + contents = read_file_to_str(fname, RFTS_IGNORE_MISSING, NULL); + if (contents && validate_bridge_stats(contents, now)) + bridge_stats_extrainfo = contents; + + tor_free(fname); +} + +/** Return most recent bridge statistics for inclusion in extra-info + * descriptors, or NULL if we don't have recent bridge statistics. */ +const char * +geoip_get_bridge_stats_extrainfo(time_t now) +{ + load_bridge_stats(now); + return bridge_stats_extrainfo; +} + +/** Return a new string containing the recent bridge statistics to be returned + * to controller clients, or NULL if we don't have any bridge statistics. */ +char * +geoip_get_bridge_stats_controller(time_t now) +{ + return format_bridge_stats_controller(now); +} + +/** Start time of entry stats or 0 if we're not collecting entry + * statistics. */ +static time_t start_of_entry_stats_interval; + +/** Initialize entry stats. */ +void +geoip_entry_stats_init(time_t now) +{ + start_of_entry_stats_interval = now; +} + +/** Stop collecting entry stats in a way that we can re-start doing so in + * geoip_entry_stats_init(). */ +void +geoip_entry_stats_term(void) +{ + client_history_clear(); + start_of_entry_stats_interval = 0; +} + +/** Write entry statistics to $DATADIR/stats/entry-stats and return time + * when we would next want to write. */ +time_t +geoip_entry_stats_write(time_t now) +{ + char *statsdir = NULL, *filename = NULL; + char *data = NULL; + char written[ISO_TIME_LEN+1]; + open_file_t *open_file = NULL; + FILE *out; + + if (!start_of_entry_stats_interval) + return 0; /* Not initialized. */ + if (start_of_entry_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write. */ + + /* Discard all items in the client history that are too old. */ + geoip_remove_old_clients(start_of_entry_stats_interval); + + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE) < 0) + goto done; + filename = get_datadir_fname2("stats", "entry-stats"); + data = geoip_get_client_history(GEOIP_CLIENT_CONNECT); + format_iso_time(written, now); + out = start_writing_to_stdio_file(filename, OPEN_FLAGS_APPEND, + 0600, &open_file); + if (!out) + goto done; + if (fprintf(out, "entry-stats-end %s (%u s)\nentry-ips %s\n", + written, (unsigned) (now - start_of_entry_stats_interval), + data ? data : "") < 0) + goto done; + + start_of_entry_stats_interval = now; + + finish_writing_to_file(open_file); + open_file = NULL; + done: + if (open_file) + abort_writing_to_file(open_file); + tor_free(filename); + tor_free(statsdir); + tor_free(data); + return start_of_entry_stats_interval + WRITE_STATS_INTERVAL; } /** Helper used to implement GETINFO ip-to-country/... controller command. */ int getinfo_helper_geoip(control_connection_t *control_conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { (void)control_conn; - if (geoip_is_loaded() && !strcmpstart(question, "ip-to-country/")) { + if (!geoip_is_loaded()) { + *errmsg = "GeoIP data not loaded"; + return -1; + } + if (!strcmpstart(question, "ip-to-country/")) { int c; uint32_t ip; struct in_addr in; @@ -674,8 +1364,8 @@ clear_geoip_db(void) SMARTLIST_FOREACH(geoip_countries, geoip_country_t *, c, tor_free(c)); smartlist_free(geoip_countries); } - if (country_idxplus1_by_lc_code) - strmap_free(country_idxplus1_by_lc_code, NULL); + + strmap_free(country_idxplus1_by_lc_code, NULL); if (geoip_entries) { SMARTLIST_FOREACH(geoip_entries, geoip_entry_t *, ent, tor_free(ent)); smartlist_free(geoip_entries); @@ -689,13 +1379,24 @@ clear_geoip_db(void) void geoip_free_all(void) { - clientmap_entry_t **ent, **next, *this; - for (ent = HT_START(clientmap, &client_history); ent != NULL; ent = next) { - this = *ent; - next = HT_NEXT_RMV(clientmap, &client_history, ent); - tor_free(this); + { + clientmap_entry_t **ent, **next, *this; + for (ent = HT_START(clientmap, &client_history); ent != NULL; ent = next) { + this = *ent; + next = HT_NEXT_RMV(clientmap, &client_history, ent); + tor_free(this); + } + HT_CLEAR(clientmap, &client_history); + } + { + dirreq_map_entry_t **ent, **next, *this; + for (ent = HT_START(dirreqmap, &dirreq_map); ent != NULL; ent = next) { + this = *ent; + next = HT_NEXT_RMV(dirreqmap, &dirreq_map, ent); + tor_free(this); + } + HT_CLEAR(dirreqmap, &dirreq_map); } - HT_CLEAR(clientmap, &client_history); clear_geoip_db(); } diff --git a/src/or/geoip.h b/src/or/geoip.h new file mode 100644 index 000000000..24f7c5b93 --- /dev/null +++ b/src/or/geoip.h @@ -0,0 +1,57 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file geoip.h + * \brief Header file for geoip.c. + **/ + +#ifndef _TOR_GEOIP_H +#define _TOR_GEOIP_H + +#ifdef GEOIP_PRIVATE +int geoip_parse_entry(const char *line); +#endif +int should_record_bridge_info(or_options_t *options); +int geoip_load_file(const char *filename, or_options_t *options); +int geoip_get_country_by_ip(uint32_t ipaddr); +int geoip_get_n_countries(void); +const char *geoip_get_country_name(country_t num); +int geoip_is_loaded(void); +country_t geoip_get_country(const char *countrycode); + +void geoip_note_client_seen(geoip_client_action_t action, + uint32_t addr, time_t now); +void geoip_remove_old_clients(time_t cutoff); + +void geoip_note_ns_response(geoip_client_action_t action, + geoip_ns_response_t response); +char *geoip_get_client_history(geoip_client_action_t action); +char *geoip_get_request_history(geoip_client_action_t action); +int getinfo_helper_geoip(control_connection_t *control_conn, + const char *question, char **answer, + const char **errmsg); +void geoip_free_all(void); + +void geoip_start_dirreq(uint64_t dirreq_id, size_t response_size, + geoip_client_action_t action, dirreq_type_t type); +void geoip_change_dirreq_state(uint64_t dirreq_id, dirreq_type_t type, + dirreq_state_t new_state); + +void geoip_dirreq_stats_init(time_t now); +time_t geoip_dirreq_stats_write(time_t now); +void geoip_dirreq_stats_term(void); +void geoip_entry_stats_init(time_t now); +time_t geoip_entry_stats_write(time_t now); +void geoip_entry_stats_term(void); +void geoip_bridge_stats_init(time_t now); +time_t geoip_bridge_stats_write(time_t now); +void geoip_bridge_stats_term(void); +const char *geoip_get_bridge_stats_extrainfo(time_t); +char *geoip_get_bridge_stats_controller(time_t); + +#endif + diff --git a/src/or/hibernate.c b/src/or/hibernate.c index 9ec63756f..aebce4cc8 100644 --- a/src/or/hibernate.c +++ b/src/or/hibernate.c @@ -22,6 +22,12 @@ hibernating, phase 2: */ #include "or.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "hibernate.h" +#include "main.h" +#include "router.h" /** Possible values of hibernate_state */ typedef enum { @@ -89,6 +95,13 @@ static uint64_t n_bytes_read_in_interval = 0; static uint64_t n_bytes_written_in_interval = 0; /** How many seconds have we been running this interval? */ static uint32_t n_seconds_active_in_interval = 0; +/** How many seconds were we active in this interval before we hit our soft + * limit? */ +static int n_seconds_to_hit_soft_limit = 0; +/** When in this interval was the soft limit hit. */ +static time_t soft_limit_hit_at = 0; +/** How many bytes had we read/written when we hit the soft limit? */ +static uint64_t n_bytes_at_soft_limit = 0; /** When did this accounting interval start? */ static time_t interval_start_time = 0; /** When will this accounting interval end? */ @@ -182,6 +195,9 @@ accounting_parse_options(or_options_t *options, int validate_only) case UNIT_DAY: d = 0; break; + /* Coverity dislikes unreachable default cases; some compilers warn on + * switch statements missing a case. Tell Coverity not to worry. */ + /* coverity[dead_error_begin] */ default: tor_assert(0); } @@ -332,29 +348,58 @@ start_of_accounting_period_after(time_t now) return edge_of_accounting_period_containing(now, 1); } +/** Return the length of the accounting period containing the time + * <b>now</b>. */ +static long +length_of_accounting_period_containing(time_t now) +{ + return edge_of_accounting_period_containing(now, 1) - + edge_of_accounting_period_containing(now, 0); +} + /** Initialize the accounting subsystem. */ void configure_accounting(time_t now) { + time_t s_now; /* Try to remember our recorded usage. */ if (!interval_start_time) read_bandwidth_usage(); /* If we fail, we'll leave values at zero, and * reset below.*/ - if (!interval_start_time || - start_of_accounting_period_after(interval_start_time) <= now) { - /* We didn't have recorded usage, or we don't have recorded usage - * for this interval. Start a new interval. */ + + s_now = start_of_accounting_period_containing(now); + + if (!interval_start_time) { + /* We didn't have recorded usage; Start a new interval. */ log_info(LD_ACCT, "Starting new accounting interval."); reset_accounting(now); - } else if (interval_start_time == - start_of_accounting_period_containing(interval_start_time)) { + } else if (s_now == interval_start_time) { log_info(LD_ACCT, "Continuing accounting interval."); /* We are in the interval we thought we were in. Do nothing.*/ interval_end_time = start_of_accounting_period_after(interval_start_time); } else { - log_warn(LD_ACCT, - "Mismatched accounting interval; starting a fresh one."); - reset_accounting(now); + long duration = + length_of_accounting_period_containing(interval_start_time); + double delta = ((double)(s_now - interval_start_time)) / duration; + if (-0.50 <= delta && delta <= 0.50) { + /* The start of the period is now a little later or earlier than we + * remembered. That's fine; we might lose some bytes we could otherwise + * have written, but better to err on the side of obeying people's + * accounting settings. */ + log_info(LD_ACCT, "Accounting interval moved by %.02f%%; " + "that's fine.", delta*100); + interval_end_time = start_of_accounting_period_after(now); + } else if (delta >= 0.99) { + /* This is the regular time-moved-forward case; don't be too noisy + * about it or people will complain */ + log_info(LD_ACCT, "Accounting interval elapsed; starting a new one"); + reset_accounting(now); + } else { + log_warn(LD_ACCT, + "Mismatched accounting interval: moved by %.02f%%. " + "Starting a fresh one.", delta*100); + reset_accounting(now); + } } accounting_set_wakeup_time(); } @@ -365,23 +410,42 @@ configure_accounting(time_t now) static void update_expected_bandwidth(void) { - uint64_t used, expected; - uint64_t max_configured = (get_options()->BandwidthRate * 60); - - if (n_seconds_active_in_interval < 1800) { + uint64_t expected; + or_options_t *options= get_options(); + uint64_t max_configured = (options->RelayBandwidthRate > 0 ? + options->RelayBandwidthRate : + options->BandwidthRate) * 60; + +#define MIN_TIME_FOR_MEASUREMENT (1800) + + if (soft_limit_hit_at > interval_start_time && n_bytes_at_soft_limit && + (soft_limit_hit_at - interval_start_time) > MIN_TIME_FOR_MEASUREMENT) { + /* If we hit our soft limit last time, only count the bytes up to that + * time. This is a better predictor of our actual bandwidth than + * considering the entirety of the last interval, since we likely started + * using bytes very slowly once we hit our soft limit. */ + expected = n_bytes_at_soft_limit / + (soft_limit_hit_at - interval_start_time); + expected /= 60; + } else if (n_seconds_active_in_interval >= MIN_TIME_FOR_MEASUREMENT) { + /* Otherwise, we either measured enough time in the last interval but + * never hit our soft limit, or we're using a state file from a Tor that + * doesn't know to store soft-limit info. Just take rate at which + * we were reading/writing in the last interval as our expected rate. + */ + uint64_t used = MAX(n_bytes_written_in_interval, + n_bytes_read_in_interval); + expected = used / (n_seconds_active_in_interval / 60); + } else { /* If we haven't gotten enough data last interval, set 'expected' * to 0. This will set our wakeup to the start of the interval. * Next interval, we'll choose our starting time based on how much * we sent this interval. */ expected = 0; - } else { - used = n_bytes_written_in_interval < n_bytes_read_in_interval ? - n_bytes_read_in_interval : n_bytes_written_in_interval; - expected = used / (n_seconds_active_in_interval / 60); - if (expected > max_configured) - expected = max_configured; } + if (expected > max_configured) + expected = max_configured; expected_bandwidth_usage = expected; } @@ -399,6 +463,9 @@ reset_accounting(time_t now) n_bytes_read_in_interval = 0; n_bytes_written_in_interval = 0; n_seconds_active_in_interval = 0; + n_bytes_at_soft_limit = 0; + soft_limit_hit_at = 0; + n_seconds_to_hit_soft_limit = 0; } /** Return true iff we should save our bandwidth usage to disk. */ @@ -449,35 +516,39 @@ accounting_run_housekeeping(time_t now) static void accounting_set_wakeup_time(void) { - char buf[ISO_TIME_LEN+1]; char digest[DIGEST_LEN]; crypto_digest_env_t *d_env; int time_in_interval; uint64_t time_to_exhaust_bw; int time_to_consider; - if (! identity_key_is_set()) { + if (! server_identity_key_is_set()) { if (init_keys() < 0) { log_err(LD_BUG, "Error initializing keys"); tor_assert(0); } } - format_iso_time(buf, interval_start_time); - crypto_pk_get_digest(get_identity_key(), digest); + if (server_identity_key_is_set()) { + char buf[ISO_TIME_LEN+1]; + format_iso_time(buf, interval_start_time); - d_env = crypto_new_digest_env(); - crypto_digest_add_bytes(d_env, buf, ISO_TIME_LEN); - crypto_digest_add_bytes(d_env, digest, DIGEST_LEN); - crypto_digest_get_digest(d_env, digest, DIGEST_LEN); - crypto_free_digest_env(d_env); + crypto_pk_get_digest(get_server_identity_key(), digest); + + d_env = crypto_new_digest_env(); + crypto_digest_add_bytes(d_env, buf, ISO_TIME_LEN); + crypto_digest_add_bytes(d_env, digest, DIGEST_LEN); + crypto_digest_get_digest(d_env, digest, DIGEST_LEN); + crypto_free_digest_env(d_env); + } else { + crypto_rand(digest, DIGEST_LEN); + } if (!expected_bandwidth_usage) { char buf1[ISO_TIME_LEN+1]; char buf2[ISO_TIME_LEN+1]; format_local_iso_time(buf1, interval_start_time); format_local_iso_time(buf2, interval_end_time); - time_to_exhaust_bw = GUESS_TIME_TO_USE_BANDWIDTH; interval_wakeup_time = interval_start_time; log_notice(LD_ACCT, @@ -492,8 +563,8 @@ accounting_set_wakeup_time(void) time_to_exhaust_bw = (get_options()->AccountingMax/expected_bandwidth_usage)*60; - if (time_to_exhaust_bw > TIME_MAX) { - time_to_exhaust_bw = TIME_MAX; + if (time_to_exhaust_bw > INT_MAX) { + time_to_exhaust_bw = INT_MAX; time_to_consider = 0; } else { time_to_consider = time_in_interval - (int)time_to_exhaust_bw; @@ -511,8 +582,6 @@ accounting_set_wakeup_time(void) * to be chosen than the last half. */ interval_wakeup_time = interval_start_time + (get_uint32(digest) % time_to_consider); - - format_iso_time(buf, interval_wakeup_time); } { @@ -559,6 +628,10 @@ accounting_record_bandwidth_usage(time_t now, or_state_t *state) state->AccountingSecondsActive = n_seconds_active_in_interval; state->AccountingExpectedUsage = expected_bandwidth_usage; + state->AccountingSecondsToReachSoftLimit = n_seconds_to_hit_soft_limit; + state->AccountingSoftLimitHitAt = soft_limit_hit_at; + state->AccountingBytesAtSoftLimit = n_bytes_at_soft_limit; + or_state_mark_dirty(state, now+(get_options()->AvoidDiskWrites ? 7200 : 60)); @@ -582,10 +655,6 @@ read_bandwidth_usage(void) if (!state) return -1; - /* Okay; it looks like the state file is more up-to-date than the - * bw_accounting file, or the bw_accounting file is nonexistent, - * or the bw_accounting file is corrupt. - */ log_info(LD_ACCT, "Reading bandwidth accounting data from state file"); n_bytes_read_in_interval = state->AccountingBytesReadInInterval; n_bytes_written_in_interval = state->AccountingBytesWrittenInInterval; @@ -593,6 +662,21 @@ read_bandwidth_usage(void) interval_start_time = state->AccountingIntervalStart; expected_bandwidth_usage = state->AccountingExpectedUsage; + /* Older versions of Tor (before 0.2.2.17-alpha or so) didn't generate these + * fields. If you switch back and forth, you might get an + * AccountingSoftLimitHitAt value from long before the most recent + * interval_start_time. If that's so, then ignore the softlimit-related + * values. */ + if (state->AccountingSoftLimitHitAt > interval_start_time) { + soft_limit_hit_at = state->AccountingSoftLimitHitAt; + n_bytes_at_soft_limit = state->AccountingBytesAtSoftLimit; + n_seconds_to_hit_soft_limit = state->AccountingSecondsToReachSoftLimit; + } else { + soft_limit_hit_at = 0; + n_bytes_at_soft_limit = 0; + n_seconds_to_hit_soft_limit = 0; + } + { char tbuf1[ISO_TIME_LEN+1]; char tbuf2[ISO_TIME_LEN+1]; @@ -632,8 +716,27 @@ hibernate_hard_limit_reached(void) static int hibernate_soft_limit_reached(void) { - uint64_t soft_limit = DBL_TO_U64(U64_TO_DBL(get_options()->AccountingMax) - * .95); + const uint64_t acct_max = get_options()->AccountingMax; +#define SOFT_LIM_PCT (.95) +#define SOFT_LIM_BYTES (500*1024*1024) +#define SOFT_LIM_MINUTES (3*60) + /* The 'soft limit' is a fair bit more complicated now than once it was. + * We want to stop accepting connections when ALL of the following are true: + * - We expect to use up the remaining bytes in under 3 hours + * - We have used up 95% of our bytes. + * - We have less than 500MB of bytes left. + */ + uint64_t soft_limit = DBL_TO_U64(U64_TO_DBL(acct_max) * SOFT_LIM_PCT); + if (acct_max > SOFT_LIM_BYTES && acct_max - SOFT_LIM_BYTES > soft_limit) { + soft_limit = acct_max - SOFT_LIM_BYTES; + } + if (expected_bandwidth_usage) { + const uint64_t expected_usage = + expected_bandwidth_usage * SOFT_LIM_MINUTES; + if (acct_max > expected_usage && acct_max - expected_usage > soft_limit) + soft_limit = acct_max - expected_usage; + } + if (!soft_limit) return 0; return n_bytes_read_in_interval >= soft_limit @@ -658,6 +761,14 @@ hibernate_begin(hibernate_state_t new_state, time_t now) exit(0); } + if (new_state == HIBERNATE_STATE_LOWBANDWIDTH && + hibernate_state == HIBERNATE_STATE_LIVE) { + soft_limit_hit_at = now; + n_seconds_to_hit_soft_limit = n_seconds_active_in_interval; + n_bytes_at_soft_limit = MAX(n_bytes_read_in_interval, + n_bytes_written_in_interval); + } + /* close listeners. leave control listener(s). */ while ((conn = connection_get_by_type(CONN_TYPE_OR_LISTENER)) || (conn = connection_get_by_type(CONN_TYPE_AP_LISTENER)) || @@ -673,7 +784,8 @@ hibernate_begin(hibernate_state_t new_state, time_t now) /* XXX upload rendezvous service descriptors with no intro points */ if (new_state == HIBERNATE_STATE_EXITING) { - log_notice(LD_GENERAL,"Interrupt: will shut down in %d seconds. Interrupt " + log_notice(LD_GENERAL,"Interrupt: we have stopped accepting new " + "connections, and will shut down in %d seconds. Interrupt " "again to exit now.", options->ShutdownWaitLength); shutdown_time = time(NULL) + options->ShutdownWaitLength; } else { /* soft limit reached */ @@ -830,7 +942,8 @@ consider_hibernation(time_t now) if (hibernate_state == HIBERNATE_STATE_LIVE) { if (hibernate_soft_limit_reached()) { log_notice(LD_ACCT, - "Bandwidth soft limit reached; commencing hibernation."); + "Bandwidth soft limit reached; commencing hibernation. " + "No new conncetions will be accepted"); hibernate_begin(HIBERNATE_STATE_LOWBANDWIDTH, now); } else if (accounting_enabled && now < interval_wakeup_time) { format_local_iso_time(buf,interval_wakeup_time); @@ -860,9 +973,11 @@ consider_hibernation(time_t now) * NULL. */ int getinfo_helper_accounting(control_connection_t *conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { (void) conn; + (void) errmsg; if (!strcmp(question, "accounting/enabled")) { *answer = tor_strdup(accounting_is_enabled(get_options()) ? "1" : "0"); } else if (!strcmp(question, "accounting/hibernating")) { diff --git a/src/or/hibernate.h b/src/or/hibernate.h new file mode 100644 index 000000000..2aea0fab0 --- /dev/null +++ b/src/or/hibernate.h @@ -0,0 +1,29 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file hibernate.h + * \brief Header file for hibernate.c. + **/ + +#ifndef _TOR_HIBERNATE_H +#define _TOR_HIBERNATE_H + +int accounting_parse_options(or_options_t *options, int validate_only); +int accounting_is_enabled(or_options_t *options); +void configure_accounting(time_t now); +void accounting_run_housekeeping(time_t now); +void accounting_add_bytes(size_t n_read, size_t n_written, int seconds); +int accounting_record_bandwidth_usage(time_t now, or_state_t *state); +void hibernate_begin_shutdown(void); +int we_are_hibernating(void); +void consider_hibernation(time_t now); +int getinfo_helper_accounting(control_connection_t *conn, + const char *question, char **answer, + const char **errmsg); + +#endif + diff --git a/src/or/main.c b/src/or/main.c index e44fd4946..7bae59ce0 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -12,12 +12,50 @@ #define MAIN_PRIVATE #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "command.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "cpuworker.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "dns.h" +#include "dnsserv.h" +#include "geoip.h" +#include "hibernate.h" +#include "main.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "ntmain.h" +#include "onion.h" +#include "policies.h" +#include "relay.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rendservice.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" #ifdef USE_DMALLOC #include <dmalloc.h> #include <openssl/crypto.h> #endif #include "memarea.h" +#ifdef HAVE_EVENT2_EVENT_H +#include <event2/event.h> +#else +#include <event.h> +#endif + void evdns_shutdown(int); /********* PROTOTYPES **********/ @@ -26,8 +64,7 @@ static void dumpmemusage(int severity); static void dumpstats(int severity); /* log stats */ static void conn_read_callback(int fd, short event, void *_conn); static void conn_write_callback(int fd, short event, void *_conn); -static void signal_callback(int fd, short events, void *arg); -static void second_elapsed_callback(int fd, short event, void *args); +static void second_elapsed_callback(periodic_timer_t *timer, void *args); static int conn_close_if_marked(int i); static void connection_start_reading_from_linked_conn(connection_t *conn); static int connection_should_read_from_linked_conn(connection_t *conn); @@ -81,8 +118,12 @@ static smartlist_t *active_linked_connection_lst = NULL; static int called_loop_once = 0; /** We set this to 1 when we've opened a circuit, so we can print a log - * entry to inform the user that Tor is working. */ -int has_completed_circuit=0; + * entry to inform the user that Tor is working. We set it to 0 when + * we think the fact that we once opened a circuit doesn't mean we can do so + * any longer (a big time jump happened, when we notice our directory is + * heinously out-of-date, etc. + */ +int can_complete_circuit=0; /** How often do we check for router descriptors that we should download * when we have too little directory info? */ @@ -127,12 +168,10 @@ connection_add(connection_t *conn) smartlist_add(connection_array, conn); if (conn->s >= 0 || conn->linked) { - conn->read_event = tor_malloc_zero(sizeof(struct event)); - conn->write_event = tor_malloc_zero(sizeof(struct event)); - event_set(conn->read_event, conn->s, EV_READ|EV_PERSIST, - conn_read_callback, conn); - event_set(conn->write_event, conn->s, EV_WRITE|EV_PERSIST, - conn_write_callback, conn); + conn->read_event = tor_event_new(tor_libevent_get_base(), + conn->s, EV_READ|EV_PERSIST, conn_read_callback, conn); + conn->write_event = tor_event_new(tor_libevent_get_base(), + conn->s, EV_WRITE|EV_PERSIST, conn_write_callback, conn); } log_debug(LD_NET,"new conn type %s, socket %d, address %s, n_conns %d.", @@ -142,6 +181,25 @@ connection_add(connection_t *conn) return 0; } +/** Tell libevent that we don't care about <b>conn</b> any more. */ +void +connection_unregister_events(connection_t *conn) +{ + if (conn->read_event) { + if (event_del(conn->read_event)) + log_warn(LD_BUG, "Error removing read event for %d", conn->s); + tor_free(conn->read_event); + } + if (conn->write_event) { + if (event_del(conn->write_event)) + log_warn(LD_BUG, "Error removing write event for %d", conn->s); + tor_free(conn->write_event); + } + if (conn->dns_server_port) { + dnsserv_close_listener(conn); + } +} + /** Remove the connection from the global list, and remove the * corresponding poll entry. Calling this function will shift the last * connection (if any) into the position occupied by conn. @@ -246,17 +304,17 @@ get_connection_array(void) } /** Set the event mask on <b>conn</b> to <b>events</b>. (The event - * mask is a bitmask whose bits are EV_READ and EV_WRITE.) + * mask is a bitmask whose bits are READ_EVENT and WRITE_EVENT) */ void -connection_watch_events(connection_t *conn, short events) +connection_watch_events(connection_t *conn, watchable_events_t events) { - if (events & EV_READ) + if (events & READ_EVENT) connection_start_reading(conn); else connection_stop_reading(conn); - if (events & EV_WRITE) + if (events & WRITE_EVENT) connection_start_writing(conn); else connection_stop_writing(conn); @@ -393,11 +451,11 @@ connection_start_reading_from_linked_conn(connection_t *conn) smartlist_add(active_linked_connection_lst, conn); if (!called_loop_once) { /* This is the first event on the list; we won't be in LOOP_ONCE mode, - * so we need to make sure that the event_loop() actually exits at the - * end of its run through the current connections and - * lets us activate read events for linked connections. */ + * so we need to make sure that the event_base_loop() actually exits at + * the end of its run through the current connections and lets us + * activate read events for linked connections. */ struct timeval tv = { 0, 0 }; - event_loopexit(&tv); + tor_event_base_loopexit(tor_libevent_get_base(), &tv); } } else { tor_assert(smartlist_isin(active_linked_connection_lst, conn)); @@ -540,7 +598,7 @@ conn_close_if_marked(int i) log_info(LD_NET, "Conn (addr %s, fd %d, type %s, state %d) marked, but wants " "to flush %d bytes. (Marked at %s:%d)", - escaped_safe_str(conn->address), + escaped_safe_str_client(conn->address), conn->s, conn_type_to_string(conn->type), conn->state, (int)conn->outbuf_flushlen, conn->marked_for_close_file, conn->marked_for_close); @@ -593,8 +651,8 @@ conn_close_if_marked(int i) "something is wrong with theirs. " "(fd %d, type %s, state %d, marked at %s:%d).", (int)buf_datalen(conn->outbuf), - escaped_safe_str(conn->address), conn->s, - conn_type_to_string(conn->type), conn->state, + escaped_safe_str_client(conn->address), + conn->s, conn_type_to_string(conn->type), conn->state, conn->marked_for_close_file, conn->marked_for_close); } @@ -623,7 +681,7 @@ directory_all_unreachable(time_t now) log_notice(LD_NET, "Is your network connection down? " "Failing connection to '%s:%d'.", - safe_str(edge_conn->socks_request->address), + safe_str_client(edge_conn->socks_request->address), edge_conn->socks_request->port); connection_mark_unattached_ap(edge_conn, END_STREAM_REASON_NET_UNREACHABLE); @@ -651,7 +709,7 @@ directory_info_has_arrived(time_t now, int from_cache) /* if we have enough dir info, then update our guard status with * whatever we just learned. */ - entry_guards_compute_status(); + entry_guards_compute_status(options, now); /* Don't even bother trying to get extrainfo until the rest of our * directory info is up-to-date */ if (options->DownloadExtraInfo) @@ -659,7 +717,7 @@ directory_info_has_arrived(time_t now, int from_cache) } if (server_mode(options) && !we_are_hibernating() && !from_cache && - (has_completed_circuit || !any_predicted_circuits(now))) + (can_complete_circuit || !any_predicted_circuits(now))) consider_testing_reachability(1, 1); } @@ -722,6 +780,7 @@ run_connection_housekeeping(int i, time_t now) the connection or send a keepalive, depending. */ or_conn = TO_OR_CONN(conn); + tor_assert(conn->outbuf); if (or_conn->is_bad_for_new_circs && !or_conn->n_circuits) { /* It's bad for new circuits, and has no unmarked circuits on it: @@ -778,11 +837,18 @@ run_connection_housekeeping(int i, time_t now) } } -/** Honor a NEWNYM request: make future requests unlinkability to past +/** Honor a NEWNYM request: make future requests unlinkable to past * requests. */ static void signewnym_impl(time_t now) { + or_options_t *options = get_options(); + if (!proxy_mode(options)) { + log_info(LD_CONTROL, "Ignoring SIGNAL NEWNYM because client functionality " + "is disabled."); + return; + } + circuit_expire_all_dirty_circs(); addressmap_clear_transient(); rend_cache_purge(); @@ -806,16 +872,19 @@ run_scheduled_events(time_t now) static time_t time_to_try_getting_descriptors = 0; static time_t time_to_reset_descriptor_failures = 0; static time_t time_to_add_entropy = 0; - static time_t time_to_write_hs_statistics = 0; static time_t time_to_write_bridge_status_file = 0; static time_t time_to_downrate_stability = 0; static time_t time_to_save_stability = 0; static time_t time_to_clean_caches = 0; static time_t time_to_recheck_bandwidth = 0; static time_t time_to_check_for_expired_networkstatus = 0; - static time_t time_to_dump_geoip_stats = 0; + static time_t time_to_write_stats_files = 0; + static time_t time_to_write_bridge_stats = 0; + static time_t time_to_launch_reachability_tests = 0; + static int should_init_bridge_stats = 1; static time_t time_to_retry_dns_init = 0; or_options_t *options = get_options(); + int is_server = server_mode(options); int i; int have_dir_info; @@ -833,11 +902,14 @@ run_scheduled_events(time_t now) signewnym_impl(now); } + /* 0c. If we've deferred log messages for the controller, handle them now */ + flush_pending_log_callbacks(); + /** 1a. Every MIN_ONION_KEY_LIFETIME seconds, rotate the onion keys, * shut down and restart all cpuworkers, and update the directory if * necessary. */ - if (server_mode(options) && + if (is_server && get_onion_key_set_at()+MIN_ONION_KEY_LIFETIME < now) { log_info(LD_GENERAL,"Rotating onion key."); rotate_onion_key(); @@ -853,7 +925,7 @@ run_scheduled_events(time_t now) update_router_descriptor_downloads(now); update_extrainfo_downloads(now); if (options->UseBridges) - fetch_bridge_descriptors(now); + fetch_bridge_descriptors(options, now); if (router_have_minimum_dir_info()) time_to_try_getting_descriptors = now + LAZY_DESCRIPTOR_RETRY_INTERVAL; else @@ -871,7 +943,10 @@ run_scheduled_events(time_t now) last_rotated_x509_certificate = now; if (last_rotated_x509_certificate+MAX_SSL_KEY_LIFETIME < now) { log_info(LD_GENERAL,"Rotating tls context."); - if (tor_tls_context_new(get_identity_key(), MAX_SSL_KEY_LIFETIME) < 0) { + if (tor_tls_context_init(public_server_mode(options), + get_tlsclient_identity_key(), + is_server ? get_server_identity_key() : NULL, + MAX_SSL_KEY_LIFETIME) < 0) { log_warn(LD_BUG, "Error reinitializing TLS context"); /* XXX is it a bug here, that we just keep going? -RD */ } @@ -896,10 +971,12 @@ run_scheduled_events(time_t now) if (accounting_is_enabled(options)) accounting_run_housekeeping(now); - if (now % 10 == 0 && (authdir_mode_tests_reachability(options)) && - !we_are_hibernating()) { + if (time_to_launch_reachability_tests < now && + (authdir_mode_tests_reachability(options)) && + !we_are_hibernating()) { + time_to_launch_reachability_tests = now + REACHABILITY_TEST_INTERVAL; /* try to determine reachability of the other Tor relays */ - dirserv_test_reachability(now, 0); + dirserv_test_reachability(now); } /** 1d. Periodically, we discount older stability information so that new @@ -941,11 +1018,56 @@ run_scheduled_events(time_t now) time_to_check_for_expired_networkstatus = now + CHECK_EXPIRED_NS_INTERVAL; } - if (time_to_dump_geoip_stats < now) { -#define DUMP_GEOIP_STATS_INTERVAL (60*60); - if (time_to_dump_geoip_stats) - dump_geoip_stats(); - time_to_dump_geoip_stats = now + DUMP_GEOIP_STATS_INTERVAL; + /* 1g. Check whether we should write statistics to disk. + */ + if (time_to_write_stats_files < now) { +#define CHECK_WRITE_STATS_INTERVAL (60*60) + time_t next_time_to_write_stats_files = (time_to_write_stats_files > 0 ? + time_to_write_stats_files : now) + CHECK_WRITE_STATS_INTERVAL; + if (options->CellStatistics) { + time_t next_write = + rep_hist_buffer_stats_write(time_to_write_stats_files); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->DirReqStatistics) { + time_t next_write = geoip_dirreq_stats_write(time_to_write_stats_files); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->EntryStatistics) { + time_t next_write = geoip_entry_stats_write(time_to_write_stats_files); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + if (options->ExitPortStatistics) { + time_t next_write = rep_hist_exit_stats_write(time_to_write_stats_files); + if (next_write && next_write < next_time_to_write_stats_files) + next_time_to_write_stats_files = next_write; + } + time_to_write_stats_files = next_time_to_write_stats_files; + } + + /* 1h. Check whether we should write bridge statistics to disk. + */ + if (should_record_bridge_info(options)) { + if (time_to_write_bridge_stats < now) { + if (should_init_bridge_stats) { + /* (Re-)initialize bridge statistics. */ + geoip_bridge_stats_init(now); + time_to_write_bridge_stats = now + WRITE_STATS_INTERVAL; + should_init_bridge_stats = 0; + } else { + /* Possibly write bridge statistics to disk and ask when to write + * them next time. */ + time_to_write_bridge_stats = geoip_bridge_stats_write( + time_to_write_bridge_stats); + } + } + } else if (!should_init_bridge_stats) { + /* Bridge mode was turned off. Ensure that stats are re-initialized + * next time bridge mode is turned on. */ + should_init_bridge_stats = 1; } /* Remove old information from rephist and the rend cache. */ @@ -953,6 +1075,8 @@ run_scheduled_events(time_t now) rep_history_clean(now - options->RephistTrackTime); rend_cache_clean(); rend_cache_clean_v2_descs_as_dir(); + if (authdir_mode_v3(options)) + microdesc_cache_rebuild(NULL, 0); #define CLEAN_CACHES_INTERVAL (30*60) time_to_clean_caches = now + CLEAN_CACHES_INTERVAL; } @@ -993,7 +1117,7 @@ run_scheduled_events(time_t now) /* also, check religiously for reachability, if it's within the first * 20 minutes of our uptime. */ if (server_mode(options) && - (has_completed_circuit || !any_predicted_circuits(now)) && + (can_complete_circuit || !any_predicted_circuits(now)) && !we_are_hibernating()) { if (stats_n_seconds_working < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { consider_testing_reachability(1, dirport_reachability_count==0); @@ -1036,7 +1160,9 @@ run_scheduled_events(time_t now) * We do this before step 4, so it can try building more if * it's not comfortable with the number of available circuits. */ - circuit_expire_building(now); + /* XXXX022 If our circuit build timeout is much lower than a second, maybe + we should do this more often? */ + circuit_expire_building(); /** 3b. Also look at pending streams and prune the ones that 'began' * a long time ago but haven't gotten a 'connected' yet. @@ -1069,7 +1195,7 @@ run_scheduled_events(time_t now) circuit_expire_old_circuits_serverside(now); /** 5. We do housekeeping for each connection... */ - connection_or_set_bad_connections(); + connection_or_set_bad_connections(NULL, 0); for (i=0;i<smartlist_len(connection_array);i++) { run_connection_housekeeping(i, now); } @@ -1092,7 +1218,7 @@ run_scheduled_events(time_t now) circuit_close_all_marked(); /** 7. And upload service descriptors if necessary. */ - if (has_completed_circuit && !we_are_hibernating()) { + if (can_complete_circuit && !we_are_hibernating()) { rend_consider_services_upload(now); rend_consider_descriptor_republication(); } @@ -1109,7 +1235,7 @@ run_scheduled_events(time_t now) /** 9. and if we're a server, check whether our DNS is telling stories to * us. */ - if (server_mode(options) && time_to_check_for_correct_dns < now) { + if (is_server && time_to_check_for_correct_dns < now) { if (!time_to_check_for_correct_dns) { time_to_check_for_correct_dns = now + 60 + crypto_rand_int(120); } else { @@ -1119,12 +1245,6 @@ run_scheduled_events(time_t now) } } - /** 10. write hidden service usage statistic to disk */ - if (options->HSAuthorityRecordStats && time_to_write_hs_statistics < now) { - hs_usage_write_statistics_to_file(now); -#define WRITE_HSUSAGE_INTERVAL (30*60) - time_to_write_hs_statistics = now+WRITE_HSUSAGE_INTERVAL; - } /** 10b. write bridge networkstatus file to disk */ if (options->BridgeAuthoritativeDir && time_to_write_bridge_status_file < now) { @@ -1134,39 +1254,30 @@ run_scheduled_events(time_t now) } } -/** Libevent timer: used to invoke second_elapsed_callback() once per - * second. */ -static struct event *timeout_event = NULL; +/** Timer: used to invoke second_elapsed_callback() once per second. */ +static periodic_timer_t *second_timer = NULL; /** Number of libevent errors in the last second: we die if we get too many. */ static int n_libevent_errors = 0; /** Libevent callback: invoked once every second. */ static void -second_elapsed_callback(int fd, short event, void *args) +second_elapsed_callback(periodic_timer_t *timer, void *arg) { /* XXXX This could be sensibly refactored into multiple callbacks, and we * could use Libevent's timers for this rather than checking the current * time against a bunch of timeouts every second. */ - static struct timeval one_second; static time_t current_second = 0; time_t now; size_t bytes_written; size_t bytes_read; int seconds_elapsed; or_options_t *options = get_options(); - (void)fd; - (void)event; - (void)args; - if (!timeout_event) { - timeout_event = tor_malloc_zero(sizeof(struct event)); - evtimer_set(timeout_event, second_elapsed_callback, NULL); - one_second.tv_sec = 1; - one_second.tv_usec = 0; - } + (void)timer; + (void)arg; n_libevent_errors = 0; - /* log_fn(LOG_NOTICE, "Tick."); */ + /* log_notice(LD_GENERAL, "Tick."); */ now = time(NULL); update_approx_time(now); @@ -1189,7 +1300,7 @@ second_elapsed_callback(int fd, short event, void *args) if (server_mode(options) && !we_are_hibernating() && seconds_elapsed > 0 && - has_completed_circuit && + can_complete_circuit && stats_n_seconds_working / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT != (stats_n_seconds_working+seconds_elapsed) / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { @@ -1231,18 +1342,6 @@ second_elapsed_callback(int fd, short event, void *args) run_scheduled_events(now); current_second = now; /* remember which second it is, for next time */ - -#if 0 - if (current_second % 300 == 0) { - rep_history_clean(current_second - options->RephistTrackTime); - dumpmemusage(get_min_log_level()<LOG_INFO ? - get_min_log_level() : LOG_INFO); - } -#endif - - if (evtimer_add(timeout_event, &one_second)) - log_err(LD_NET, - "Error from libevent when setting one-second timeout event"); } #ifndef MS_WINDOWS @@ -1385,7 +1484,7 @@ do_main_loop(void) /* load the private keys, if we're supposed to have them, and set up the * TLS context. */ - if (! identity_key_is_set()) { + if (! client_identity_key_is_set()) { if (init_keys() < 0) { log_err(LD_BUG,"Error initializing keys; exiting"); return -1; @@ -1403,8 +1502,10 @@ do_main_loop(void) /* initialize the bootstrap status events to know we're starting up */ control_event_bootstrap(BOOTSTRAP_STATUS_STARTING, 0); - if (trusted_dirs_reload_certs()) - return -1; + if (trusted_dirs_reload_certs()) { + log_warn(LD_DIR, + "Couldn't load all cached v3 certificates. Starting anyway."); + } if (router_reload_v2_networkstatus()) { return -1; } @@ -1421,18 +1522,23 @@ do_main_loop(void) now = time(NULL); directory_info_has_arrived(now, 1); - if (authdir_mode_tests_reachability(get_options())) { - /* the directory is already here, run startup things */ - dirserv_test_reachability(now, 1); - } - if (server_mode(get_options())) { /* launch cpuworkers. Need to do this *after* we've read the onion key. */ cpu_init(); } /* set up once-a-second callback. */ - second_elapsed_callback(0,0,NULL); + if (! second_timer) { + struct timeval one_second; + one_second.tv_sec = 1; + one_second.tv_usec = 0; + + second_timer = periodic_timer_new(tor_libevent_get_base(), + &one_second, + second_elapsed_callback, + NULL); + tor_assert(second_timer); + } for (;;) { if (nt_service_is_stopping()) @@ -1451,20 +1557,16 @@ do_main_loop(void) /* poll until we have an event, or the second ends, or until we have * some active linked connections to trigger events for. */ - loop_result = event_loop(called_loop_once ? EVLOOP_ONCE : 0); + loop_result = event_base_loop(tor_libevent_get_base(), + called_loop_once ? EVLOOP_ONCE : 0); /* let catch() handle things like ^c, and otherwise don't worry about it */ if (loop_result < 0) { int e = tor_socket_errno(-1); /* let the program survive things like ^z */ if (e != EINTR && !ERRNO_IS_EINPROGRESS(e)) { -#ifdef HAVE_EVENT_GET_METHOD log_err(LD_NET,"libevent call with %s failed: %s [%d]", - event_get_method(), tor_socket_strerror(e), e); -#else - log_err(LD_NET,"libevent call failed: %s [%d]", - tor_socket_strerror(e), e); -#endif + tor_libevent_get_method(), tor_socket_strerror(e), e); return -1; #ifndef MS_WINDOWS } else if (e == EINVAL) { @@ -1485,46 +1587,7 @@ do_main_loop(void) } } -/** Used to implement the SIGNAL control command: if we accept - * <b>the_signal</b> as a remote pseudo-signal, act on it. */ -/* We don't re-use catch() here because: - * 1. We handle a different set of signals than those allowed in catch. - * 2. Platforms without signal() are unlikely to define SIGfoo. - * 3. The control spec is defined to use fixed numeric signal values - * which just happen to match the Unix values. - */ -void -control_signal_act(int the_signal) -{ - switch (the_signal) - { - case 1: - signal_callback(0,0,(void*)(uintptr_t)SIGHUP); - break; - case 2: - signal_callback(0,0,(void*)(uintptr_t)SIGINT); - break; - case 10: - signal_callback(0,0,(void*)(uintptr_t)SIGUSR1); - break; - case 12: - signal_callback(0,0,(void*)(uintptr_t)SIGUSR2); - break; - case 15: - signal_callback(0,0,(void*)(uintptr_t)SIGTERM); - break; - case SIGNEWNYM: - signal_callback(0,0,(void*)(uintptr_t)SIGNEWNYM); - break; - case SIGCLEARDNSCACHE: - signal_callback(0,0,(void*)(uintptr_t)SIGCLEARDNSCACHE); - break; - default: - log_warn(LD_BUG, "Unrecognized signal number %d.", the_signal); - break; - } -} - +#ifndef MS_WINDOWS /* Only called when we're willing to use signals */ /** Libevent callback: invoked when we get a signal. */ static void @@ -1533,6 +1596,15 @@ signal_callback(int fd, short events, void *arg) uintptr_t sig = (uintptr_t)arg; (void)fd; (void)events; + + process_signal(sig); +} +#endif + +/** Do the work of acting on a signal received in <b>sig</b> */ +void +process_signal(uintptr_t sig) +{ switch (sig) { case SIGTERM: @@ -1607,6 +1679,7 @@ dumpmemusage(int severity) U64_PRINTF_ARG(rephist_total_alloc), rephist_total_num); dump_routerlist_mem_usage(severity); dump_cell_pool_usage(severity); + dump_dns_mem_usage(severity); buf_dump_freelist_sizes(severity); tor_log_mallinfo(severity); } @@ -1633,7 +1706,8 @@ dumpstats(int severity) if (!connection_is_listener(conn)) { log(severity,LD_GENERAL, "Conn %d is to %s:%d.", i, - safe_str(conn->address), conn->port); + safe_str_client(conn->address), + conn->port); log(severity,LD_GENERAL, "Conn %d: %d bytes waiting on inbuf (len %d, last read %d secs ago)", i, @@ -1730,7 +1804,7 @@ handle_signals(int is_parent) { #ifndef MS_WINDOWS /* do signal stuff only on Unix */ int i; - static int signals[] = { + static const int signals[] = { SIGINT, /* do a controlled slow shutdown */ SIGTERM, /* to terminate now */ SIGPIPE, /* otherwise SIGPIPE kills us */ @@ -1742,12 +1816,13 @@ handle_signals(int is_parent) #endif SIGCHLD, /* handle dns/cpu workers that exit */ -1 }; - static struct event signal_events[16]; /* bigger than it has to be. */ + static struct event *signal_events[16]; /* bigger than it has to be. */ if (is_parent) { for (i = 0; signals[i] >= 0; ++i) { - signal_set(&signal_events[i], signals[i], signal_callback, - (void*)(uintptr_t)signals[i]); - if (signal_add(&signal_events[i], NULL)) + signal_events[i] = tor_evsignal_new( + tor_libevent_get_base(), signals[i], signal_callback, + (void*)(uintptr_t)signals[i]); + if (event_add(signal_events[i], NULL)) log_warn(LD_BUG, "Error from libevent when adding event for signal %d", signals[i]); } @@ -1836,7 +1911,9 @@ tor_init(int argc, char *argv[]) "and you probably shouldn't."); #endif - if (crypto_global_init(get_options()->HardwareAccel)) { + if (crypto_global_init(get_options()->HardwareAccel, + get_options()->AccelName, + get_options()->AccelDir)) { log_err(LD_BUG, "Unable to initialize OpenSSL. Exiting."); return -1; } @@ -1930,7 +2007,6 @@ tor_free_all(int postfork) rend_cache_free_all(); rend_service_authorization_free_all(); rep_hist_free_all(); - hs_usage_free_all(); dns_free_all(); clear_pending_onions(); circuit_free_all(); @@ -1938,6 +2014,7 @@ tor_free_all(int postfork) connection_free_all(); buf_shrink_freelists(1); memarea_clear_freelist(); + microdesc_free_all(); if (!postfork) { config_free_all(); router_free_all(); @@ -1948,13 +2025,11 @@ tor_free_all(int postfork) tor_tls_free_all(); } /* stuff in main.c */ - if (connection_array) - smartlist_free(connection_array); - if (closeable_connection_lst) - smartlist_free(closeable_connection_lst); - if (active_linked_connection_lst) - smartlist_free(active_linked_connection_lst); - tor_free(timeout_event); + + smartlist_free(connection_array); + smartlist_free(closeable_connection_lst); + smartlist_free(active_linked_connection_lst); + periodic_timer_free(second_timer); if (!postfork) { release_lockfile(); } @@ -2015,7 +2090,7 @@ do_list_fingerprint(void) log_err(LD_BUG,"Error initializing keys; can't display fingerprint"); return -1; } - if (!(k = get_identity_key())) { + if (!(k = get_server_identity_key())) { log_err(LD_GENERAL,"Error: missing identity key."); return -1; } @@ -2045,6 +2120,31 @@ do_hash_password(void) printf("16:%s\n",output); } +#if defined (WINCE) +int +find_flashcard_path(PWCHAR path, size_t size) +{ + WIN32_FIND_DATA d = {0}; + HANDLE h = NULL; + + if (!path) + return -1; + + h = FindFirstFlashCard(&d); + if (h == INVALID_HANDLE_VALUE) + return -1; + + if (wcslen(d.cFileName) == 0) { + FindClose(h); + return -1; + } + + wcsncpy(path,d.cFileName,size); + FindClose(h); + return 0; +} +#endif + /** Main entry point for the Tor process. Called from main(). */ /* This function is distinct from main() only so we can link main.c into * the unittest binary without conflicting with the unittests' main. */ @@ -2052,6 +2152,46 @@ int tor_main(int argc, char *argv[]) { int result = 0; +#if defined (WINCE) + WCHAR path [MAX_PATH] = {0}; + WCHAR fullpath [MAX_PATH] = {0}; + PWCHAR p = NULL; + FILE* redir = NULL; + FILE* redirdbg = NULL; + + // this is to facilitate debugging by opening + // a file on a folder shared by the wm emulator. + // if no flashcard (real or emulated) is present, + // log files will be written in the root folder + if (find_flashcard_path(path,MAX_PATH) == -1) + { + redir = _wfreopen( L"\\stdout.log", L"w", stdout ); + redirdbg = _wfreopen( L"\\stderr.log", L"w", stderr ); + } else { + swprintf(fullpath,L"\\%s\\tor",path); + CreateDirectory(fullpath,NULL); + + swprintf(fullpath,L"\\%s\\tor\\stdout.log",path); + redir = _wfreopen( fullpath, L"w", stdout ); + + swprintf(fullpath,L"\\%s\\tor\\stderr.log",path); + redirdbg = _wfreopen( fullpath, L"w", stderr ); + } +#endif + +#ifdef MS_WINDOWS + /* Call SetProcessDEPPolicy to permanently enable DEP. + The function will not resolve on earlier versions of Windows, + and failure is not dangerous. */ + HMODULE hMod = GetModuleHandleA("Kernel32.dll"); + if (hMod) { + typedef BOOL (WINAPI *PSETDEP)(DWORD); + PSETDEP setdeppolicy = (PSETDEP)GetProcAddress(hMod, + "SetProcessDEPPolicy"); + if (setdeppolicy) setdeppolicy(1); /* PROCESS_DEP_ENABLE */ + } +#endif + update_approx_time(time(NULL)); tor_threads_init(); init_logging(); diff --git a/src/or/main.h b/src/or/main.h new file mode 100644 index 000000000..0551f7aaf --- /dev/null +++ b/src/or/main.h @@ -0,0 +1,69 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file main.h + * \brief Header file for main.c. + **/ + +#ifndef _TOR_MAIN_H +#define _TOR_MAIN_H + +extern int can_complete_circuit; + +int connection_add(connection_t *conn); +int connection_remove(connection_t *conn); +void connection_unregister_events(connection_t *conn); +int connection_in_array(connection_t *conn); +void add_connection_to_closeable_list(connection_t *conn); +int connection_is_on_closeable_list(connection_t *conn); + +smartlist_t *get_connection_array(void); + +/** Bitmask for events that we can turn on and off with + * connection_watch_events. */ +typedef enum watchable_events { + READ_EVENT=0x02, /**< We want to know when a connection is readable */ + WRITE_EVENT=0x04 /**< We want to know when a connection is writable */ +} watchable_events_t; +void connection_watch_events(connection_t *conn, watchable_events_t events); +int connection_is_reading(connection_t *conn); +void connection_stop_reading(connection_t *conn); +void connection_start_reading(connection_t *conn); + +int connection_is_writing(connection_t *conn); +void connection_stop_writing(connection_t *conn); +void connection_start_writing(connection_t *conn); + +void connection_stop_reading_from_linked_conn(connection_t *conn); + +void directory_all_unreachable(time_t now); +void directory_info_has_arrived(time_t now, int from_cache); + +void ip_address_changed(int at_interface); +void dns_servers_relaunch_checks(void); + +void handle_signals(int is_parent); +void process_signal(uintptr_t sig); + +int try_locking(or_options_t *options, int err_if_locked); +int have_lockfile(void); +void release_lockfile(void); + +void tor_cleanup(void); +void tor_free_all(int postfork); + +int tor_main(int argc, char *argv[]); + +#ifdef MAIN_PRIVATE +int do_main_loop(void); +int do_list_fingerprint(void); +void do_hash_password(void); +int tor_init(int argc, char **argv); +#endif + +#endif + diff --git a/src/or/microdesc.c b/src/or/microdesc.c new file mode 100644 index 000000000..5740c40d5 --- /dev/null +++ b/src/or/microdesc.c @@ -0,0 +1,492 @@ +/* Copyright (c) 2009-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "config.h" +#include "microdesc.h" +#include "routerparse.h" + +/** A data structure to hold a bunch of cached microdescriptors. There are + * two active files in the cache: a "cache file" that we mmap, and a "journal + * file" that we append to. Periodically, we rebuild the cache file to hold + * only the microdescriptors that we want to keep */ +struct microdesc_cache_t { + /** Map from sha256-digest to microdesc_t for every microdesc_t in the + * cache. */ + HT_HEAD(microdesc_map, microdesc_t) map; + + /** Name of the cache file. */ + char *cache_fname; + /** Name of the journal file. */ + char *journal_fname; + /** Mmap'd contents of the cache file, or NULL if there is none. */ + tor_mmap_t *cache_content; + /** Number of bytes used in the journal file. */ + size_t journal_len; + /** Number of bytes in descriptors removed as too old. */ + size_t bytes_dropped; + + /** Total bytes of microdescriptor bodies we have added to this cache */ + uint64_t total_len_seen; + /** Total number of microdescriptors we have added to this cache */ + unsigned n_seen; +}; + +/** Helper: computes a hash of <b>md</b> to place it in a hash table. */ +static INLINE unsigned int +_microdesc_hash(microdesc_t *md) +{ + unsigned *d = (unsigned*)md->digest; +#if SIZEOF_INT == 4 + return d[0] ^ d[1] ^ d[2] ^ d[3] ^ d[4] ^ d[5] ^ d[6] ^ d[7]; +#else + return d[0] ^ d[1] ^ d[2] ^ d[3]; +#endif +} + +/** Helper: compares <b>a</b> and </b> for equality for hash-table purposes. */ +static INLINE int +_microdesc_eq(microdesc_t *a, microdesc_t *b) +{ + return !memcmp(a->digest, b->digest, DIGEST256_LEN); +} + +HT_PROTOTYPE(microdesc_map, microdesc_t, node, + _microdesc_hash, _microdesc_eq); +HT_GENERATE(microdesc_map, microdesc_t, node, + _microdesc_hash, _microdesc_eq, 0.6, + malloc, realloc, free); + +/** Write the body of <b>md</b> into <b>f</b>, with appropriate annotations. + * On success, return the total number of bytes written, and set + * *<b>annotation_len_out</b> to the number of bytes written as + * annotations. */ +static ssize_t +dump_microdescriptor(FILE *f, microdesc_t *md, size_t *annotation_len_out) +{ + ssize_t r = 0; + size_t written; + /* XXXX drops unkown annotations. */ + if (md->last_listed) { + char buf[ISO_TIME_LEN+1]; + char annotation[ISO_TIME_LEN+32]; + format_iso_time(buf, md->last_listed); + tor_snprintf(annotation, sizeof(annotation), "@last-listed %s\n", buf); + fputs(annotation, f); + r += strlen(annotation); + *annotation_len_out = r; + } else { + *annotation_len_out = 0; + } + + md->off = (off_t) ftell(f); + written = fwrite(md->body, 1, md->bodylen, f); + if (written != md->bodylen) { + log_warn(LD_DIR, + "Couldn't dump microdescriptor (wrote %lu out of %lu): %s", + (unsigned long)written, (unsigned long)md->bodylen, + strerror(ferror(f))); + return -1; + } + r += md->bodylen; + return r; +} + +/** Holds a pointer to the current microdesc_cache_t object, or NULL if no + * such object has been allocated. */ +static microdesc_cache_t *the_microdesc_cache = NULL; + +/** Return a pointer to the microdescriptor cache, loading it if necessary. */ +microdesc_cache_t * +get_microdesc_cache(void) +{ + if (PREDICT_UNLIKELY(the_microdesc_cache==NULL)) { + microdesc_cache_t *cache = tor_malloc_zero(sizeof(microdesc_cache_t)); + HT_INIT(microdesc_map, &cache->map); + cache->cache_fname = get_datadir_fname("cached-microdescs"); + cache->journal_fname = get_datadir_fname("cached-microdescs.new"); + microdesc_cache_reload(cache); + the_microdesc_cache = cache; + } + return the_microdesc_cache; +} + +/* There are three sources of microdescriptors: + 1) Generated by us while acting as a directory authority. + 2) Loaded from the cache on disk. + 3) Downloaded. +*/ + +/** Decode the microdescriptors from the string starting at <b>s</b> and + * ending at <b>eos</b>, and store them in <b>cache</b>. If <b>no-save</b>, + * mark them as non-writable to disk. If <b>where</b> is SAVED_IN_CACHE, + * leave their bodies as pointers to the mmap'd cache. If where is + * <b>SAVED_NOWHERE</b>, do not allow annotations. Return a list of the added + * microdescriptors. */ +smartlist_t * +microdescs_add_to_cache(microdesc_cache_t *cache, + const char *s, const char *eos, saved_location_t where, + int no_save) +{ + /*XXXX need an argument that sets last_listed as appropriate. */ + + smartlist_t *descriptors, *added; + const int allow_annotations = (where != SAVED_NOWHERE); + const int copy_body = (where != SAVED_IN_CACHE); + + descriptors = microdescs_parse_from_string(s, eos, + allow_annotations, + copy_body); + + added = microdescs_add_list_to_cache(cache, descriptors, where, no_save); + smartlist_free(descriptors); + return added; +} + +/* As microdescs_add_to_cache, but takes a list of micrdescriptors instead of + * a string to encode. Frees any members of <b>descriptors</b> that it does + * not add. */ +smartlist_t * +microdescs_add_list_to_cache(microdesc_cache_t *cache, + smartlist_t *descriptors, saved_location_t where, + int no_save) +{ + smartlist_t *added; + open_file_t *open_file = NULL; + FILE *f = NULL; + // int n_added = 0; + ssize_t size = 0; + + if (where == SAVED_NOWHERE && !no_save) { + f = start_writing_to_stdio_file(cache->journal_fname, + OPEN_FLAGS_APPEND|O_BINARY, + 0600, &open_file); + if (!f) { + log_warn(LD_DIR, "Couldn't append to journal in %s: %s", + cache->journal_fname, strerror(errno)); + return NULL; + } + } + + added = smartlist_create(); + SMARTLIST_FOREACH_BEGIN(descriptors, microdesc_t *, md) { + microdesc_t *md2; + md2 = HT_FIND(microdesc_map, &cache->map, md); + if (md2) { + /* We already had this one. */ + if (md2->last_listed < md->last_listed) + md2->last_listed = md->last_listed; + microdesc_free(md); + if (where != SAVED_NOWHERE) + cache->bytes_dropped += size; + continue; + } + + /* Okay, it's a new one. */ + if (f) { + size_t annotation_len; + size = dump_microdescriptor(f, md, &annotation_len); + if (size < 0) { + /* XXX handle errors from dump_microdescriptor() */ + /* log? return -1? die? coredump the universe? */ + continue; + } + md->saved_location = SAVED_IN_JOURNAL; + cache->journal_len += size; + } else { + md->saved_location = where; + } + + md->no_save = no_save; + + HT_INSERT(microdesc_map, &cache->map, md); + smartlist_add(added, md); + ++cache->n_seen; + cache->total_len_seen += md->bodylen; + } SMARTLIST_FOREACH_END(md); + + if (f) + finish_writing_to_file(open_file); /*XXX Check me.*/ + + return added; +} + +/** Remove every microdescriptor in <b>cache</b>. */ +void +microdesc_cache_clear(microdesc_cache_t *cache) +{ + microdesc_t **entry, **next; + for (entry = HT_START(microdesc_map, &cache->map); entry; entry = next) { + microdesc_t *md = *entry; + next = HT_NEXT_RMV(microdesc_map, &cache->map, entry); + microdesc_free(md); + } + HT_CLEAR(microdesc_map, &cache->map); + if (cache->cache_content) { + tor_munmap_file(cache->cache_content); + cache->cache_content = NULL; + } + cache->total_len_seen = 0; + cache->n_seen = 0; + cache->bytes_dropped = 0; +} + +/** Reload the contents of <b>cache</b> from disk. If it is empty, load it + * for the first time. Return 0 on success, -1 on failure. */ +int +microdesc_cache_reload(microdesc_cache_t *cache) +{ + struct stat st; + char *journal_content; + smartlist_t *added; + tor_mmap_t *mm; + int total = 0; + + microdesc_cache_clear(cache); + + mm = cache->cache_content = tor_mmap_file(cache->cache_fname); + if (mm) { + added = microdescs_add_to_cache(cache, mm->data, mm->data+mm->size, + SAVED_IN_CACHE, 0); + if (added) { + total += smartlist_len(added); + smartlist_free(added); + } + } + + journal_content = read_file_to_str(cache->journal_fname, + RFTS_IGNORE_MISSING, &st); + if (journal_content) { + cache->journal_len = (size_t) st.st_size; + added = microdescs_add_to_cache(cache, journal_content, + journal_content+st.st_size, + SAVED_IN_JOURNAL, 0); + if (added) { + total += smartlist_len(added); + smartlist_free(added); + } + tor_free(journal_content); + } + log_notice(LD_DIR, "Reloaded microdescriptor cache. Found %d descriptors.", + total); + + microdesc_cache_rebuild(cache, 0 /* don't force */); + + return 0; +} + +/** By default, we remove any microdescriptors that have gone at least this + * long without appearing in a current consensus. */ +#define TOLERATE_MICRODESC_AGE (7*24*60*60) + +/** Remove all microdescriptors from <b>cache</b> that haven't been listed for + * a long time. Does not rebuild the cache on disk. If <b>cutoff</b> is + * positive, specifically remove microdescriptors that have been unlisted + * since <b>cutoff</b>. If <b>force</b> is true, remove microdescriptors even + * if we have no current live microdescriptor consensus. + */ +void +microdesc_cache_clean(microdesc_cache_t *cache, time_t cutoff, int force) +{ + microdesc_t **mdp, *victim; + int dropped=0, kept=0; + size_t bytes_dropped = 0; + time_t now = time(NULL); + + (void) force; + /* In 0.2.2, we let this proceed unconditionally: only authorities have + * microdesc caches. */ + + if (cutoff <= 0) + cutoff = now - TOLERATE_MICRODESC_AGE; + + for (mdp = HT_START(microdesc_map, &cache->map); mdp != NULL; ) { + if ((*mdp)->last_listed < cutoff) { + ++dropped; + victim = *mdp; + mdp = HT_NEXT_RMV(microdesc_map, &cache->map, mdp); + bytes_dropped += victim->bodylen; + microdesc_free(victim); + } else { + ++kept; + mdp = HT_NEXT(microdesc_map, &cache->map, mdp); + } + } + + if (dropped) { + log_notice(LD_DIR, "Removed %d/%d microdescriptors as old.", + dropped,dropped+kept); + cache->bytes_dropped += bytes_dropped; + } +} + +static int +should_rebuild_md_cache(microdesc_cache_t *cache) +{ + const size_t old_len = + cache->cache_content ? cache->cache_content->size : 0; + const size_t journal_len = cache->journal_len; + const size_t dropped = cache->bytes_dropped; + + if (journal_len < 16384) + return 0; /* Don't bother, not enough has happened yet. */ + if (dropped > (journal_len + old_len) / 3) + return 1; /* We could save 1/3 or more of the currently used space. */ + if (journal_len > old_len / 2) + return 1; /* We should append to the regular file */ + + return 0; +} + +/** Regenerate the main cache file for <b>cache</b>, clear the journal file, + * and update every microdesc_t in the cache with pointers to its new + * location. If <b>force</b> is true, do this unconditionally. If + * <b>force</b> is false, do it only if we expect to save space on disk. */ +int +microdesc_cache_rebuild(microdesc_cache_t *cache, int force) +{ + open_file_t *open_file; + FILE *f; + microdesc_t **mdp; + smartlist_t *wrote; + ssize_t size; + off_t off = 0; + int orig_size, new_size; + + if (cache == NULL) { + cache = the_microdesc_cache; + if (cache == NULL) + return 0; + } + + /* Remove dead descriptors */ + microdesc_cache_clean(cache, 0/*cutoff*/, 0/*force*/); + + if (!force && !should_rebuild_md_cache(cache)) + return 0; + + log_info(LD_DIR, "Rebuilding the microdescriptor cache..."); + + orig_size = (int)(cache->cache_content ? cache->cache_content->size : 0); + orig_size += (int)cache->journal_len; + + f = start_writing_to_stdio_file(cache->cache_fname, + OPEN_FLAGS_REPLACE|O_BINARY, + 0600, &open_file); + if (!f) + return -1; + + wrote = smartlist_create(); + + HT_FOREACH(mdp, microdesc_map, &cache->map) { + microdesc_t *md = *mdp; + size_t annotation_len; + if (md->no_save) + continue; + + size = dump_microdescriptor(f, md, &annotation_len); + if (size < 0) { + /* XXX handle errors from dump_microdescriptor() */ + /* log? return -1? die? coredump the universe? */ + continue; + } + md->off = off + annotation_len; + off += size; + if (md->saved_location != SAVED_IN_CACHE) { + tor_free(md->body); + md->saved_location = SAVED_IN_CACHE; + } + smartlist_add(wrote, md); + } + + finish_writing_to_file(open_file); /*XXX Check me.*/ + + if (cache->cache_content) + tor_munmap_file(cache->cache_content); + cache->cache_content = tor_mmap_file(cache->cache_fname); + + if (!cache->cache_content && smartlist_len(wrote)) { + log_err(LD_DIR, "Couldn't map file that we just wrote to %s!", + cache->cache_fname); + smartlist_free(wrote); + return -1; + } + SMARTLIST_FOREACH_BEGIN(wrote, microdesc_t *, md) { + tor_assert(md->saved_location == SAVED_IN_CACHE); + md->body = (char*)cache->cache_content->data + md->off; + tor_assert(!memcmp(md->body, "onion-key", 9)); + } SMARTLIST_FOREACH_END(md); + + smartlist_free(wrote); + + write_str_to_file(cache->journal_fname, "", 1); + cache->journal_len = 0; + cache->bytes_dropped = 0; + + new_size = cache->cache_content ? (int)cache->cache_content->size : 0; + log_info(LD_DIR, "Done rebuilding microdesc cache. " + "Saved %d bytes; %d still used.", + orig_size-new_size, new_size); + + return 0; +} + +/** Deallocate a single microdescriptor. Note: the microdescriptor MUST have + * previously been removed from the cache if it had ever been inserted. */ +void +microdesc_free(microdesc_t *md) +{ + if (!md) + return; + /* Must be removed from hash table! */ + if (md->onion_pkey) + crypto_free_pk_env(md->onion_pkey); + if (md->body && md->saved_location != SAVED_IN_CACHE) + tor_free(md->body); + + if (md->family) { + SMARTLIST_FOREACH(md->family, char *, cp, tor_free(cp)); + smartlist_free(md->family); + } + tor_free(md->exitsummary); + + tor_free(md); +} + +/** Free all storage held in the microdesc.c module. */ +void +microdesc_free_all(void) +{ + if (the_microdesc_cache) { + microdesc_cache_clear(the_microdesc_cache); + tor_free(the_microdesc_cache->cache_fname); + tor_free(the_microdesc_cache->journal_fname); + tor_free(the_microdesc_cache); + } +} + +/** If there is a microdescriptor in <b>cache</b> whose sha256 digest is + * <b>d</b>, return it. Otherwise return NULL. */ +microdesc_t * +microdesc_cache_lookup_by_digest256(microdesc_cache_t *cache, const char *d) +{ + microdesc_t *md, search; + if (!cache) + cache = get_microdesc_cache(); + memcpy(search.digest, d, DIGEST256_LEN); + md = HT_FIND(microdesc_map, &cache->map, &search); + return md; +} + +/** Return the mean size of decriptors added to <b>cache</b> since it was last + * cleared. Used to estimate the size of large downloads. */ +size_t +microdesc_average_size(microdesc_cache_t *cache) +{ + if (!cache) + cache = get_microdesc_cache(); + if (!cache->n_seen) + return 512; + return (size_t)(cache->total_len_seen / cache->n_seen); +} + diff --git a/src/or/microdesc.h b/src/or/microdesc.h new file mode 100644 index 000000000..77ce8536b --- /dev/null +++ b/src/or/microdesc.h @@ -0,0 +1,38 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file microdesc.h + * \brief Header file for microdesc.c. + **/ + +#ifndef _TOR_MICRODESC_H +#define _TOR_MICRODESC_H + +microdesc_cache_t *get_microdesc_cache(void); + +smartlist_t *microdescs_add_to_cache(microdesc_cache_t *cache, + const char *s, const char *eos, saved_location_t where, + int no_save); +smartlist_t *microdescs_add_list_to_cache(microdesc_cache_t *cache, + smartlist_t *descriptors, saved_location_t where, + int no_save); + +void microdesc_cache_clean(microdesc_cache_t *cache, time_t cutoff, int force); +int microdesc_cache_rebuild(microdesc_cache_t *cache, int force); +int microdesc_cache_reload(microdesc_cache_t *cache); +void microdesc_cache_clear(microdesc_cache_t *cache); + +microdesc_t *microdesc_cache_lookup_by_digest256(microdesc_cache_t *cache, + const char *d); + +size_t microdesc_average_size(microdesc_cache_t *cache); + +void microdesc_free(microdesc_t *md); +void microdesc_free_all(void); + +#endif + diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index 7106294d5..a50d3ca07 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -11,6 +11,20 @@ */ #include "or.h" +#include "circuitbuild.h" +#include "config.h" +#include "connection.h" +#include "connection_or.h" +#include "control.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "main.h" +#include "networkstatus.h" +#include "relay.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" /* For tracking v2 networkstatus documents. Only caches do this now. */ @@ -35,16 +49,22 @@ static networkstatus_t *current_consensus = NULL; /** A v3 consensus networkstatus that we've received, but which we don't * have enough certificates to be happy about. */ -static networkstatus_t *consensus_waiting_for_certs = NULL; -/** The encoded version of consensus_waiting_for_certs. */ -static char *consensus_waiting_for_certs_body = NULL; -/** When did we set the current value of consensus_waiting_for_certs? If this - * is too recent, we shouldn't try to fetch a new consensus for a little while, - * to give ourselves time to get certificates for this one. */ -static time_t consensus_waiting_for_certs_set_at = 0; -/** Set to 1 if we've been holding on to consensus_waiting_for_certs so long - * that we should treat it as maybe being bad. */ -static int consensus_waiting_for_certs_dl_failed = 0; +typedef struct consensus_waiting_for_certs_t { + /** The consensus itself. */ + networkstatus_t *consensus; + /** The encoded version of the consensus, nul-terminated. */ + char *body; + /** When did we set the current value of consensus_waiting_for_certs? If + * this is too recent, we shouldn't try to fetch a new consensus for a + * little while, to give ourselves time to get certificates for this one. */ + time_t set_at; + /** Set to 1 if we've been holding on to it for so long we should maybe + * treat it as being bad. */ + int dl_failed; +} consensus_waiting_for_certs_t; + +static consensus_waiting_for_certs_t + consensus_waiting_for_certs[N_CONSENSUS_FLAVORS]; /** The last time we tried to download a networkstatus, or 0 for "never". We * use this to rate-limit download attempts for directory caches (including @@ -56,7 +76,7 @@ static time_t last_networkstatus_download_attempted = 0; * before the current consensus becomes invalid. */ static time_t time_to_download_next_consensus = 0; /** Download status for the current consensus networkstatus. */ -static download_status_t consensus_dl_status = { 0, 0, DL_SCHED_CONSENSUS }; +static download_status_t consensus_dl_status[N_CONSENSUS_FLAVORS]; /** True iff we have logged a warning about this OR's version being older than * listed by the authorities. */ @@ -89,6 +109,7 @@ networkstatus_reset_warnings(void) void networkstatus_reset_download_failures(void) { + int i; const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, SMARTLIST_FOREACH(ns->entries, routerstatus_t *, rs, @@ -97,7 +118,8 @@ networkstatus_reset_download_failures(void) rs->need_to_mirror = 1; }));; - download_status_reset(&consensus_dl_status); + for (i=0; i < N_CONSENSUS_FLAVORS; ++i) + download_status_reset(&consensus_dl_status[i]); if (v2_download_status_map) { digestmap_iter_t *iter; digestmap_t *map = v2_download_status_map; @@ -170,7 +192,7 @@ router_reload_v2_networkstatus(void) return 0; } -/** Read the cached v3 consensus networkstatus from the disk. */ +/** Read every cached v3 consensus networkstatus from the disk. */ int router_reload_consensus_networkstatus(void) { @@ -179,31 +201,46 @@ router_reload_consensus_networkstatus(void) struct stat st; or_options_t *options = get_options(); const unsigned int flags = NSSET_FROM_CACHE | NSSET_DONT_DOWNLOAD_CERTS; + int flav; /* FFFF Suppress warnings if cached consensus is bad? */ + for (flav = 0; flav < N_CONSENSUS_FLAVORS; ++flav) { + char buf[128]; + const char *flavor = networkstatus_get_flavor_name(flav); + if (flav == FLAV_NS) { + filename = get_datadir_fname("cached-consensus"); + } else { + tor_snprintf(buf, sizeof(buf), "cached-%s-consensus", flavor); + filename = get_datadir_fname(buf); + } + s = read_file_to_str(filename, RFTS_IGNORE_MISSING, NULL); + if (s) { + if (networkstatus_set_current_consensus(s, flavor, flags) < -1) { + log_warn(LD_FS, "Couldn't load consensus %s networkstatus from \"%s\"", + flavor, filename); + } + tor_free(s); + } + tor_free(filename); - filename = get_datadir_fname("cached-consensus"); - s = read_file_to_str(filename, RFTS_IGNORE_MISSING, NULL); - if (s) { - if (networkstatus_set_current_consensus(s, flags) < -1) { - log_warn(LD_FS, "Couldn't load consensus networkstatus from \"%s\"", - filename); + if (flav == FLAV_NS) { + filename = get_datadir_fname("unverified-consensus"); + } else { + tor_snprintf(buf, sizeof(buf), "unverified-%s-consensus", flavor); + filename = get_datadir_fname(buf); } - tor_free(s); - } - tor_free(filename); - filename = get_datadir_fname("unverified-consensus"); - s = read_file_to_str(filename, RFTS_IGNORE_MISSING, NULL); - if (s) { - if (networkstatus_set_current_consensus(s, + s = read_file_to_str(filename, RFTS_IGNORE_MISSING, NULL); + if (s) { + if (networkstatus_set_current_consensus(s, flavor, flags|NSSET_WAS_WAITING_FOR_CERTS)) { - log_info(LD_FS, "Couldn't load consensus networkstatus from \"%s\"", - filename); + log_info(LD_FS, "Couldn't load consensus %s networkstatus from \"%s\"", + flavor, filename); } - tor_free(s); + tor_free(s); + } + tor_free(filename); } - tor_free(filename); if (!current_consensus || (stat(options->FallbackNetworkstatusFile, &st)==0 && @@ -211,7 +248,7 @@ router_reload_consensus_networkstatus(void) s = read_file_to_str(options->FallbackNetworkstatusFile, RFTS_IGNORE_MISSING, NULL); if (s) { - if (networkstatus_set_current_consensus(s, + if (networkstatus_set_current_consensus(s, "ns", flags|NSSET_ACCEPT_OBSOLETE)) { log_info(LD_FS, "Couldn't load consensus networkstatus from \"%s\"", options->FallbackNetworkstatusFile); @@ -242,8 +279,16 @@ router_reload_consensus_networkstatus(void) static void vote_routerstatus_free(vote_routerstatus_t *rs) { + vote_microdesc_hash_t *h, *next; + if (!rs) + return; tor_free(rs->version); tor_free(rs->status.exitsummary); + for (h = rs->microdesc; h; h = next) { + tor_free(h->microdesc_hash_line); + next = h->next; + tor_free(h); + } tor_free(rs); } @@ -251,6 +296,8 @@ vote_routerstatus_free(vote_routerstatus_t *rs) void routerstatus_free(routerstatus_t *rs) { + if (!rs) + return; tor_free(rs->exitsummary); tor_free(rs); } @@ -259,6 +306,8 @@ routerstatus_free(routerstatus_t *rs) void networkstatus_v2_free(networkstatus_v2_t *ns) { + if (!ns) + return; tor_free(ns->source_address); tor_free(ns->contact); if (ns->signing_key) @@ -273,7 +322,25 @@ networkstatus_v2_free(networkstatus_v2_t *ns) tor_free(ns); } -/** Clear all storage held in <b>ns</b>. */ +/** Free all storage held in <b>sig</b> */ +void +document_signature_free(document_signature_t *sig) +{ + tor_free(sig->signature); + tor_free(sig); +} + +/** Return a newly allocated copy of <b>sig</b> */ +document_signature_t * +document_signature_dup(const document_signature_t *sig) +{ + document_signature_t *r = tor_memdup(sig, sizeof(document_signature_t)); + if (r->signature) + r->signature = tor_memdup(sig->signature, sig->signature_len); + return r; +} + +/** Free all storage held in <b>ns</b>. */ void networkstatus_vote_free(networkstatus_t *ns) { @@ -286,6 +353,10 @@ networkstatus_vote_free(networkstatus_t *ns) SMARTLIST_FOREACH(ns->known_flags, char *, c, tor_free(c)); smartlist_free(ns->known_flags); } + if (ns->weight_params) { + SMARTLIST_FOREACH(ns->weight_params, char *, c, tor_free(c)); + smartlist_free(ns->weight_params); + } if (ns->net_params) { SMARTLIST_FOREACH(ns->net_params, char *, c, tor_free(c)); smartlist_free(ns->net_params); @@ -295,18 +366,20 @@ networkstatus_vote_free(networkstatus_t *ns) smartlist_free(ns->supported_methods); } if (ns->voters) { - SMARTLIST_FOREACH(ns->voters, networkstatus_voter_info_t *, voter, - { + SMARTLIST_FOREACH_BEGIN(ns->voters, networkstatus_voter_info_t *, voter) { tor_free(voter->nickname); tor_free(voter->address); tor_free(voter->contact); - tor_free(voter->signature); + if (voter->sigs) { + SMARTLIST_FOREACH(voter->sigs, document_signature_t *, sig, + document_signature_free(sig)); + smartlist_free(voter->sigs); + } tor_free(voter); - }); + } SMARTLIST_FOREACH_END(voter); smartlist_free(ns->voters); } - if (ns->cert) - authority_cert_free(ns->cert); + authority_cert_free(ns->cert); if (ns->routerstatus_list) { if (ns->type == NS_TYPE_VOTE || ns->type == NS_TYPE_OPINION) { @@ -319,8 +392,8 @@ networkstatus_vote_free(networkstatus_t *ns) smartlist_free(ns->routerstatus_list); } - if (ns->desc_digest_map) - digestmap_free(ns->desc_digest_map, NULL); + + digestmap_free(ns->desc_digest_map, NULL); memset(ns, 11, sizeof(*ns)); tor_free(ns); @@ -341,35 +414,39 @@ networkstatus_get_voter_by_id(networkstatus_t *vote, return NULL; } -/** Check whether the signature on <b>voter</b> is correctly signed by - * the signing key of <b>cert</b>. Return -1 if <b>cert</b> doesn't match the +/** Check whether the signature <b>sig</b> is correctly signed with the + * signing key in <b>cert</b>. Return -1 if <b>cert</b> doesn't match the * signing key; otherwise set the good_signature or bad_signature flag on * <b>voter</b>, and return 0. */ -/* (private; exposed for testing.) */ int -networkstatus_check_voter_signature(networkstatus_t *consensus, - networkstatus_voter_info_t *voter, - authority_cert_t *cert) +networkstatus_check_document_signature(const networkstatus_t *consensus, + document_signature_t *sig, + const authority_cert_t *cert) { - char d[DIGEST_LEN]; + char key_digest[DIGEST_LEN]; + const int dlen = sig->alg == DIGEST_SHA1 ? DIGEST_LEN : DIGEST256_LEN; char *signed_digest; size_t signed_digest_len; - if (crypto_pk_get_digest(cert->signing_key, d)<0) + + if (crypto_pk_get_digest(cert->signing_key, key_digest)<0) return -1; - if (memcmp(voter->signing_key_digest, d, DIGEST_LEN)) + if (memcmp(sig->signing_key_digest, key_digest, DIGEST_LEN) || + memcmp(sig->identity_digest, cert->cache_info.identity_digest, + DIGEST_LEN)) return -1; + signed_digest_len = crypto_pk_keysize(cert->signing_key); signed_digest = tor_malloc(signed_digest_len); if (crypto_pk_public_checksig(cert->signing_key, signed_digest, signed_digest_len, - voter->signature, - voter->signature_len) != DIGEST_LEN || - memcmp(signed_digest, consensus->networkstatus_digest, DIGEST_LEN)) { + sig->signature, + sig->signature_len) < dlen || + memcmp(signed_digest, consensus->digests.d[sig->alg], dlen)) { log_warn(LD_DIR, "Got a bad signature on a networkstatus vote"); - voter->bad_signature = 1; + sig->bad_signature = 1; } else { - voter->good_signature = 1; + sig->good_signature = 1; } tor_free(signed_digest); return 0; @@ -388,7 +465,7 @@ networkstatus_check_consensus_signature(networkstatus_t *consensus, int warn) { int n_good = 0; - int n_missing_key = 0; + int n_missing_key = 0, n_dl_failed_key = 0; int n_bad = 0; int n_unknown = 0; int n_no_signature = 0; @@ -402,37 +479,62 @@ networkstatus_check_consensus_signature(networkstatus_t *consensus, tor_assert(consensus->type == NS_TYPE_CONSENSUS); - SMARTLIST_FOREACH(consensus->voters, networkstatus_voter_info_t *, voter, - { - if (!voter->good_signature && !voter->bad_signature && voter->signature) { - /* we can try to check the signature. */ - int is_v3_auth = trusteddirserver_get_by_v3_auth_digest( - voter->identity_digest) != NULL; - authority_cert_t *cert = - authority_cert_get_by_digests(voter->identity_digest, - voter->signing_key_digest); - if (!is_v3_auth) { - smartlist_add(unrecognized, voter); - ++n_unknown; - continue; - } else if (!cert || cert->expires < now) { - smartlist_add(need_certs_from, voter); - ++n_missing_key; - continue; - } - if (networkstatus_check_voter_signature(consensus, voter, cert) < 0) { - smartlist_add(need_certs_from, voter); - ++n_missing_key; - continue; + SMARTLIST_FOREACH_BEGIN(consensus->voters, networkstatus_voter_info_t *, + voter) { + int good_here = 0; + int bad_here = 0; + int unknown_here = 0; + int missing_key_here = 0, dl_failed_key_here = 0; + SMARTLIST_FOREACH_BEGIN(voter->sigs, document_signature_t *, sig) { + if (!sig->good_signature && !sig->bad_signature && + sig->signature) { + /* we can try to check the signature. */ + int is_v3_auth = trusteddirserver_get_by_v3_auth_digest( + sig->identity_digest) != NULL; + authority_cert_t *cert = + authority_cert_get_by_digests(sig->identity_digest, + sig->signing_key_digest); + tor_assert(!memcmp(sig->identity_digest, voter->identity_digest, + DIGEST_LEN)); + + if (!is_v3_auth) { + smartlist_add(unrecognized, voter); + ++unknown_here; + continue; + } else if (!cert || cert->expires < now) { + smartlist_add(need_certs_from, voter); + ++missing_key_here; + if (authority_cert_dl_looks_uncertain(sig->identity_digest)) + ++dl_failed_key_here; + continue; + } + if (networkstatus_check_document_signature(consensus, sig, cert) < 0) { + smartlist_add(need_certs_from, voter); + ++missing_key_here; + if (authority_cert_dl_looks_uncertain(sig->identity_digest)) + ++dl_failed_key_here; + continue; + } } - } - if (voter->good_signature) + if (sig->good_signature) + ++good_here; + else if (sig->bad_signature) + ++bad_here; + } SMARTLIST_FOREACH_END(sig); + if (good_here) ++n_good; - else if (voter->bad_signature) + else if (bad_here) ++n_bad; - else + else if (missing_key_here) { + ++n_missing_key; + if (dl_failed_key_here) + ++n_dl_failed_key; + } else if (unknown_here) { + ++n_unknown; + } else { ++n_no_signature; - }); + } + } SMARTLIST_FOREACH_END(voter); /* Now see whether we're missing any voters entirely. */ SMARTLIST_FOREACH(router_get_trusted_dir_servers(), @@ -443,39 +545,71 @@ networkstatus_check_consensus_signature(networkstatus_t *consensus, smartlist_add(missing_authorities, ds); }); - if (warn > 1 || (warn >= 0 && n_good < n_required)) + if (warn > 1 || (warn >= 0 && + (n_good + n_missing_key - n_dl_failed_key < n_required))) { severity = LOG_WARN; - else + } else { severity = LOG_INFO; + } if (warn >= 0) { SMARTLIST_FOREACH(unrecognized, networkstatus_voter_info_t *, voter, { - log_info(LD_DIR, "Consensus includes unrecognized authority '%s' " - "at %s:%d (contact %s; identity %s)", + log(severity, LD_DIR, "Consensus includes unrecognized authority " + "'%s' at %s:%d (contact %s; identity %s)", voter->nickname, voter->address, (int)voter->dir_port, voter->contact?voter->contact:"n/a", hex_str(voter->identity_digest, DIGEST_LEN)); }); SMARTLIST_FOREACH(need_certs_from, networkstatus_voter_info_t *, voter, { - log_info(LD_DIR, "Looks like we need to download a new certificate " - "from authority '%s' at %s:%d (contact %s; identity %s)", + log(severity, LD_DIR, "Looks like we need to download a new " + "certificate from authority '%s' at %s:%d (contact %s; " + "identity %s)", voter->nickname, voter->address, (int)voter->dir_port, voter->contact?voter->contact:"n/a", hex_str(voter->identity_digest, DIGEST_LEN)); }); SMARTLIST_FOREACH(missing_authorities, trusted_dir_server_t *, ds, { - log_info(LD_DIR, "Consensus does not include configured " + log(severity, LD_DIR, "Consensus does not include configured " "authority '%s' at %s:%d (identity %s)", ds->nickname, ds->address, (int)ds->dir_port, hex_str(ds->v3_identity_digest, DIGEST_LEN)); }); - log(severity, LD_DIR, - "%d unknown, %d missing key, %d good, %d bad, %d no signature, " - "%d required", n_unknown, n_missing_key, n_good, n_bad, - n_no_signature, n_required); + { + smartlist_t *sl = smartlist_create(); + char *cp; + tor_asprintf(&cp, "A consensus needs %d good signatures from recognized " + "authorities for us to accept it. This one has %d.", + n_required, n_good); + smartlist_add(sl,cp); + if (n_no_signature) { + tor_asprintf(&cp, "%d of the authorities we know didn't sign it.", + n_no_signature); + smartlist_add(sl,cp); + } + if (n_unknown) { + tor_asprintf(&cp, "It has %d signatures from authorities we don't " + "recognize.", n_unknown); + smartlist_add(sl,cp); + } + if (n_bad) { + tor_asprintf(&cp, "%d of the signatures on it didn't verify " + "correctly.", n_bad); + smartlist_add(sl,cp); + } + if (n_missing_key) { + tor_asprintf(&cp, "We were unable to check %d of the signatures, " + "because we were missing the keys.", n_missing_key); + smartlist_add(sl,cp); + } + cp = smartlist_join_strings(sl, " ", 0, NULL); + log(severity, LD_DIR, "%s", cp); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); + smartlist_free(sl); + } } smartlist_free(unrecognized); @@ -785,8 +919,8 @@ networkstatus_v2_list_clean(time_t now) /** Helper for bsearching a list of routerstatus_t pointers: compare a * digest in the key to the identity digest of a routerstatus_t. */ -static int -_compare_digest_to_routerstatus_entry(const void *_key, const void **_member) +int +compare_digest_to_routerstatus_entry(const void *_key, const void **_member) { const char *key = _key; const routerstatus_t *rs = *_member; @@ -799,7 +933,7 @@ routerstatus_t * networkstatus_v2_find_entry(networkstatus_v2_t *ns, const char *digest) { return smartlist_bsearch(ns->entries, digest, - _compare_digest_to_routerstatus_entry); + compare_digest_to_routerstatus_entry); } /** Return the entry in <b>ns</b> for the identity digest <b>digest</b>, or @@ -808,7 +942,7 @@ routerstatus_t * networkstatus_vote_find_entry(networkstatus_t *ns, const char *digest) { return smartlist_bsearch(ns->routerstatus_list, digest, - _compare_digest_to_routerstatus_entry); + compare_digest_to_routerstatus_entry); } /*XXXX make this static once functions are moved into this file. */ @@ -820,7 +954,7 @@ networkstatus_vote_find_entry_idx(networkstatus_t *ns, const char *digest, int *found_out) { return smartlist_bsearch_idx(ns->routerstatus_list, digest, - _compare_digest_to_routerstatus_entry, + compare_digest_to_routerstatus_entry, found_out); } @@ -873,7 +1007,7 @@ router_get_consensus_status_by_id(const char *digest) if (!current_consensus) return NULL; return smartlist_bsearch(current_consensus->routerstatus_list, digest, - _compare_digest_to_routerstatus_entry); + compare_digest_to_routerstatus_entry); } /** Given a nickname (possibly verbose, possibly a hexadecimal digest), return @@ -1077,28 +1211,30 @@ update_v2_networkstatus_cache_downloads(time_t now) static void update_consensus_networkstatus_downloads(time_t now) { - or_options_t *options = get_options(); + int i; if (!networkstatus_get_live_consensus(now)) time_to_download_next_consensus = now; /* No live consensus? Get one now!*/ if (time_to_download_next_consensus > now) return; /* Wait until the current consensus is older. */ - if (authdir_mode_v3(options)) - return; /* Authorities never fetch a consensus */ - if (!download_status_is_ready(&consensus_dl_status, now, + /* XXXXNM Microdescs: may need to download more types. */ + if (!download_status_is_ready(&consensus_dl_status[FLAV_NS], now, CONSENSUS_NETWORKSTATUS_MAX_DL_TRIES)) return; /* We failed downloading a consensus too recently. */ if (connection_get_by_type_purpose(CONN_TYPE_DIR, DIR_PURPOSE_FETCH_CONSENSUS)) return; /* There's an in-progress download.*/ - if (consensus_waiting_for_certs) { - /* XXXX make sure this doesn't delay sane downloads. */ - if (consensus_waiting_for_certs_set_at + DELAY_WHILE_FETCHING_CERTS > now) - return; /* We're still getting certs for this one. */ - else { - if (!consensus_waiting_for_certs_dl_failed) { - download_status_failed(&consensus_dl_status, 0); - consensus_waiting_for_certs_dl_failed=1; + for (i=0; i < N_CONSENSUS_FLAVORS; ++i) { + consensus_waiting_for_certs_t *waiting = &consensus_waiting_for_certs[i]; + if (waiting->consensus) { + /* XXXX make sure this doesn't delay sane downloads. */ + if (waiting->set_at + DELAY_WHILE_FETCHING_CERTS > now) + return; /* We're still getting certs for this one. */ + else { + if (!waiting->dl_failed) { + download_status_failed(&consensus_dl_status[FLAV_NS], 0); + waiting->dl_failed=1; + } } } } @@ -1114,7 +1250,8 @@ update_consensus_networkstatus_downloads(time_t now) void networkstatus_consensus_download_failed(int status_code) { - download_status_failed(&consensus_dl_status, status_code); + /* XXXXNM Microdescs: may need to handle more types. */ + download_status_failed(&consensus_dl_status[FLAV_NS], status_code); /* Retry immediately, if appropriate. */ update_consensus_networkstatus_downloads(time(NULL)); } @@ -1148,8 +1285,15 @@ update_consensus_networkstatus_fetch_time(time_t now) /* We want to cache the next one at some point after this one * is no longer fresh... */ start = c->fresh_until + min_sec_before_caching; - /* But only in the first half-interval after that. */ - dl_interval = interval/2; + /* Some clients may need the consensus sooner than others. */ + if (options->FetchDirInfoExtraEarly || authdir_mode_v3(options)) { + dl_interval = 60; + if (min_sec_before_caching + dl_interval > interval) + dl_interval = interval/2; + } else { + /* But only in the first half-interval after that. */ + dl_interval = interval/2; + } } else { /* We're an ordinary client or a bridge. Give all the caches enough * time to download the consensus. */ @@ -1168,7 +1312,7 @@ update_consensus_networkstatus_fetch_time(time_t now) } if (dl_interval < 1) dl_interval = 1; - /* We must not try to replace c while it's still the most valid: */ + /* We must not try to replace c while it's still fresh: */ tor_assert(c->fresh_until < start); /* We must download the next one before c is invalid: */ tor_assert(start+dl_interval < c->valid_until); @@ -1189,7 +1333,6 @@ update_consensus_networkstatus_fetch_time(time_t now) time_to_download_next_consensus = now; log_info(LD_DIR, "No live consensus; we should fetch one immediately."); } - } /** Return 1 if there's a reason we shouldn't try any directory @@ -1213,7 +1356,7 @@ update_networkstatus_downloads(time_t now) or_options_t *options = get_options(); if (should_delay_dir_fetches(options)) return; - if (directory_fetches_dir_info_early(options)) + if (authdir_mode_any_main(options) || options->FetchV2Networkstatus) update_v2_networkstatus_cache_downloads(now); update_consensus_networkstatus_downloads(now); update_certificate_downloads(now); @@ -1224,10 +1367,14 @@ update_networkstatus_downloads(time_t now) void update_certificate_downloads(time_t now) { - if (consensus_waiting_for_certs) - authority_certs_fetch_missing(consensus_waiting_for_certs, now); - else - authority_certs_fetch_missing(current_consensus, now); + int i; + for (i = 0; i < N_CONSENSUS_FLAVORS; ++i) { + if (consensus_waiting_for_certs[i].consensus) + authority_certs_fetch_missing(consensus_waiting_for_certs[i].consensus, + now); + } + + authority_certs_fetch_missing(current_consensus, now); } /** Return 1 if we have a consensus but we don't have enough certificates @@ -1235,7 +1382,8 @@ update_certificate_downloads(time_t now) int consensus_is_waiting_for_certs(void) { - return consensus_waiting_for_certs ? 1 : 0; + return consensus_waiting_for_certs[USABLE_CONSENSUS_FLAVOR].consensus + ? 1 : 0; } /** Return the network status with a given identity digest. */ @@ -1404,16 +1552,31 @@ networkstatus_copy_old_consensus_info(networkstatus_t *new_c, * user, and -2 for more serious problems. */ int -networkstatus_set_current_consensus(const char *consensus, unsigned flags) +networkstatus_set_current_consensus(const char *consensus, + const char *flavor, + unsigned flags) { - networkstatus_t *c; + networkstatus_t *c=NULL; int r, result = -1; time_t now = time(NULL); + or_options_t *options = get_options(); char *unverified_fname = NULL, *consensus_fname = NULL; + int flav = networkstatus_parse_flavor_name(flavor); const unsigned from_cache = flags & NSSET_FROM_CACHE; const unsigned was_waiting_for_certs = flags & NSSET_WAS_WAITING_FOR_CERTS; const unsigned dl_certs = !(flags & NSSET_DONT_DOWNLOAD_CERTS); const unsigned accept_obsolete = flags & NSSET_ACCEPT_OBSOLETE; + const unsigned require_flavor = flags & NSSET_REQUIRE_FLAVOR; + const digests_t *current_digests = NULL; + consensus_waiting_for_certs_t *waiting = NULL; + time_t current_valid_after = 0; + int free_consensus = 1; /* Free 'c' at the end of the function */ + + if (flav < 0) { + /* XXXX we don't handle unrecognized flavors yet. */ + log_warn(LD_BUG, "Unrecognized consensus flavor %s", flavor); + return -2; + } /* Make sure it's parseable. */ c = networkstatus_parse_vote_from_string(consensus, NULL, NS_TYPE_CONSENSUS); @@ -1423,33 +1586,69 @@ networkstatus_set_current_consensus(const char *consensus, unsigned flags) goto done; } + if ((int)c->flavor != flav) { + /* This wasn't the flavor we thought we were getting. */ + if (require_flavor) { + log_warn(LD_DIR, "Got consensus with unexpected flavor %s (wanted %s)", + networkstatus_get_flavor_name(c->flavor), flavor); + goto done; + } + flav = c->flavor; + flavor = networkstatus_get_flavor_name(flav); + } + + if (flav != USABLE_CONSENSUS_FLAVOR && + !directory_caches_dir_info(options)) { + /* This consensus is totally boring to us: we won't use it, and we won't + * serve it. Drop it. */ + goto done; + } + if (from_cache && !accept_obsolete && c->valid_until < now-OLD_ROUTER_DESC_MAX_AGE) { - /* XXX022 when we try to make fallbackconsensus work again, we should + /* XXXX If we try to make fallbackconsensus work again, we should * consider taking this out. Until then, believing obsolete consensuses * is causing more harm than good. See also bug 887. */ - log_info(LD_DIR, "Loaded an obsolete consensus. Discarding."); + log_info(LD_DIR, "Loaded an expired consensus. Discarding."); goto done; } - if (current_consensus && - !memcmp(c->networkstatus_digest, current_consensus->networkstatus_digest, - DIGEST_LEN)) { + if (!strcmp(flavor, "ns")) { + consensus_fname = get_datadir_fname("cached-consensus"); + unverified_fname = get_datadir_fname("unverified-consensus"); + if (current_consensus) { + current_digests = ¤t_consensus->digests; + current_valid_after = current_consensus->valid_after; + } + } else { + cached_dir_t *cur; + char buf[128]; + tor_snprintf(buf, sizeof(buf), "cached-%s-consensus", flavor); + consensus_fname = get_datadir_fname(buf); + tor_snprintf(buf, sizeof(buf), "unverified-%s-consensus", flavor); + unverified_fname = get_datadir_fname(buf); + cur = dirserv_get_consensus(flavor); + if (cur) { + current_digests = &cur->digests; + current_valid_after = cur->published; + } + } + + if (current_digests && + !memcmp(&c->digests, current_digests, sizeof(c->digests))) { /* We already have this one. That's a failure. */ - log_info(LD_DIR, "Got a consensus we already have"); + log_info(LD_DIR, "Got a %s consensus we already have", flavor); goto done; } - if (current_consensus && c->valid_after <= current_consensus->valid_after) { + if (current_valid_after && c->valid_after <= current_valid_after) { /* We have a newer one. There's no point in accepting this one, * even if it's great. */ - log_info(LD_DIR, "Got a consensus at least as old as the one we have"); + log_info(LD_DIR, "Got a %s consensus at least as old as the one we have", + flavor); goto done; } - consensus_fname = get_datadir_fname("cached-consensus"); - unverified_fname = get_datadir_fname("unverified-consensus"); - /* Make sure it's signed enough. */ if ((r=networkstatus_check_consensus_signature(c, 1))<0) { if (r == -1) { @@ -1458,16 +1657,16 @@ networkstatus_set_current_consensus(const char *consensus, unsigned flags) log_info(LD_DIR, "Not enough certificates to check networkstatus consensus"); } - if (!current_consensus || - c->valid_after > current_consensus->valid_after) { - if (consensus_waiting_for_certs) - networkstatus_vote_free(consensus_waiting_for_certs); - tor_free(consensus_waiting_for_certs_body); - consensus_waiting_for_certs = c; - c = NULL; /* Prevent free. */ - consensus_waiting_for_certs_body = tor_strdup(consensus); - consensus_waiting_for_certs_set_at = now; - consensus_waiting_for_certs_dl_failed = 0; + if (!current_valid_after || + c->valid_after > current_valid_after) { + waiting = &consensus_waiting_for_certs[flav]; + networkstatus_vote_free(waiting->consensus); + tor_free(waiting->body); + waiting->consensus = c; + free_consensus = 0; + waiting->body = tor_strdup(consensus); + waiting->set_at = now; + waiting->dl_failed = 0; if (!from_cache) { write_str_to_file(unverified_fname, consensus, 0); } @@ -1496,57 +1695,81 @@ networkstatus_set_current_consensus(const char *consensus, unsigned flags) } } - if (!from_cache) + if (!from_cache && flav == USABLE_CONSENSUS_FLAVOR) control_event_client_status(LOG_NOTICE, "CONSENSUS_ARRIVED"); /* Are we missing any certificates at all? */ if (r != 1 && dl_certs) authority_certs_fetch_missing(c, now); - notify_control_networkstatus_changed(current_consensus, c); + if (flav == USABLE_CONSENSUS_FLAVOR) { + notify_control_networkstatus_changed(current_consensus, c); - if (current_consensus) { - networkstatus_copy_old_consensus_info(c, current_consensus); - networkstatus_vote_free(current_consensus); + if (current_consensus) { + networkstatus_copy_old_consensus_info(c, current_consensus); + networkstatus_vote_free(current_consensus); + /* Defensive programming : we should set current_consensus very soon, + * but we're about to call some stuff in the meantime, and leaving this + * dangling pointer around has proven to be trouble. */ + current_consensus = NULL; + } } - if (consensus_waiting_for_certs && - consensus_waiting_for_certs->valid_after <= c->valid_after) { - networkstatus_vote_free(consensus_waiting_for_certs); - consensus_waiting_for_certs = NULL; - if (consensus != consensus_waiting_for_certs_body) - tor_free(consensus_waiting_for_certs_body); + waiting = &consensus_waiting_for_certs[flav]; + if (waiting->consensus && + waiting->consensus->valid_after <= c->valid_after) { + networkstatus_vote_free(waiting->consensus); + waiting->consensus = NULL; + if (consensus != waiting->body) + tor_free(waiting->body); else - consensus_waiting_for_certs_body = NULL; - consensus_waiting_for_certs_set_at = 0; - consensus_waiting_for_certs_dl_failed = 0; + waiting->body = NULL; + waiting->set_at = 0; + waiting->dl_failed = 0; unlink(unverified_fname); } /* Reset the failure count only if this consensus is actually valid. */ if (c->valid_after <= now && now <= c->valid_until) { - download_status_reset(&consensus_dl_status); + download_status_reset(&consensus_dl_status[flav]); } else { if (!from_cache) - download_status_failed(&consensus_dl_status, 0); + download_status_failed(&consensus_dl_status[flav], 0); } - current_consensus = c; - c = NULL; /* Prevent free. */ + if (flav == USABLE_CONSENSUS_FLAVOR) { + current_consensus = c; + free_consensus = 0; /* Prevent free. */ + + /* XXXXNM Microdescs: needs a non-ns variant. */ + update_consensus_networkstatus_fetch_time(now); + dirvote_recalculate_timing(options, now); + routerstatus_list_update_named_server_map(); + cell_ewma_set_scale_factor(options, current_consensus); - update_consensus_networkstatus_fetch_time(now); - dirvote_recalculate_timing(get_options(), now); - routerstatus_list_update_named_server_map(); + /* XXXX023 this call might be unnecessary here: can changing the + * current consensus really alter our view of any OR's rate limits? */ + connection_or_update_token_buckets(get_connection_array(), options); + + circuit_build_times_new_consensus_params(&circ_times, current_consensus); + } + + if (directory_caches_dir_info(options)) { + dirserv_set_cached_consensus_networkstatus(consensus, + flavor, + &c->digests, + c->valid_after); + } if (!from_cache) { write_str_to_file(consensus_fname, consensus, 0); } - if (directory_caches_dir_info(get_options())) - dirserv_set_cached_networkstatus_v3(consensus, - current_consensus->valid_after); +/** If a consensus appears more than this many seconds before its declared + * valid-after time, declare that our clock is skewed. */ +#define EARLY_CONSENSUS_NOTICE_SKEW 60 - if (ftime_definitely_before(now, current_consensus->valid_after)) { + if (now < current_consensus->valid_after - EARLY_CONSENSUS_NOTICE_SKEW) { char tbuf[ISO_TIME_LEN+1]; char dbuf[64]; long delta = now - current_consensus->valid_after; @@ -1564,7 +1787,7 @@ networkstatus_set_current_consensus(const char *consensus, unsigned flags) result = 0; done: - if (c) + if (free_consensus) networkstatus_vote_free(c); tor_free(consensus_fname); tor_free(unverified_fname); @@ -1576,13 +1799,17 @@ networkstatus_set_current_consensus(const char *consensus, unsigned flags) void networkstatus_note_certs_arrived(void) { - if (consensus_waiting_for_certs) { - if (networkstatus_check_consensus_signature( - consensus_waiting_for_certs, 0)>=0) { + int i; + for (i=0; i<N_CONSENSUS_FLAVORS; ++i) { + consensus_waiting_for_certs_t *waiting = &consensus_waiting_for_certs[i]; + if (!waiting->consensus) + continue; + if (networkstatus_check_consensus_signature(waiting->consensus, 0)>=0) { if (!networkstatus_set_current_consensus( - consensus_waiting_for_certs_body, + waiting->body, + networkstatus_get_flavor_name(i), NSSET_WAS_WAITING_FOR_CERTS)) { - tor_free(consensus_waiting_for_certs_body); + tor_free(waiting->body); } } } @@ -1668,10 +1895,8 @@ download_status_map_update_from_v2_networkstatus(void) v2_download_status_map = digestmap_new(); dl_status = digestmap_new(); - SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, - { - SMARTLIST_FOREACH(ns->entries, routerstatus_t *, rs, - { + SMARTLIST_FOREACH_BEGIN(networkstatus_v2_list, networkstatus_v2_t *, ns) { + SMARTLIST_FOREACH_BEGIN(ns->entries, routerstatus_t *, rs) { const char *d = rs->descriptor_digest; download_status_t *s; if (digestmap_get(dl_status, d)) @@ -1680,8 +1905,8 @@ download_status_map_update_from_v2_networkstatus(void) s = tor_malloc_zero(sizeof(download_status_t)); } digestmap_set(dl_status, d, s); - }); - }); + } SMARTLIST_FOREACH_END(rs); + } SMARTLIST_FOREACH_END(ns); digestmap_free(v2_download_status_map, _tor_free); v2_download_status_map = dl_status; networkstatus_v2_list_has_changed = 0; @@ -1695,11 +1920,9 @@ routerstatus_list_update_named_server_map(void) if (!current_consensus) return; - if (named_server_map) - strmap_free(named_server_map, _tor_free); + strmap_free(named_server_map, _tor_free); named_server_map = strmap_new(); - if (unnamed_server_map) - strmap_free(unnamed_server_map, NULL); + strmap_free(unnamed_server_map, NULL); unnamed_server_map = strmap_new(); SMARTLIST_FOREACH(current_consensus->routerstatus_list, routerstatus_t *, rs, { @@ -1774,6 +1997,15 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, router->is_bad_directory = rs->is_bad_directory; router->is_bad_exit = rs->is_bad_exit; router->is_hs_dir = rs->is_hs_dir; + } else { + /* If we _are_ an authority, we should check whether this router + * is one that will cause us to need a reachability test. */ + routerinfo_t *old_router = + router_get_by_digest(router->cache_info.identity_digest); + if (old_router != router) { + router->needs_retest_if_added = + dirserv_should_launch_reachability_test(router, old_router); + } } if (router->is_running && ds) { download_status_reset(&ds->v2_ns_dl_status); @@ -1836,7 +2068,7 @@ char * networkstatus_getinfo_helper_single(routerstatus_t *rs) { char buf[RS_ENTRY_LEN+1]; - routerstatus_format_entry(buf, sizeof(buf), rs, NULL, 0, 1); + routerstatus_format_entry(buf, sizeof(buf), rs, NULL, NS_CONTROL_PORT); return tor_strdup(buf); } @@ -1873,7 +2105,7 @@ networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now) if (bridge_auth && ri->purpose == ROUTER_PURPOSE_BRIDGE) dirserv_set_router_is_running(ri, now); /* then generate and write out status lines for each of them */ - set_routerstatus_from_routerinfo(&rs, ri, now, 0, 0, 0, 0); + set_routerstatus_from_routerinfo(&rs, ri, now, 0, 0, 0); smartlist_add(statuses, networkstatus_getinfo_helper_single(&rs)); }); @@ -1898,35 +2130,118 @@ networkstatus_dump_bridge_status_to_file(time_t now) tor_free(status); } +static int32_t +get_net_param_from_list(smartlist_t *net_params, const char *param_name, + int32_t default_val, int32_t min_val, int32_t max_val) +{ + int32_t res = default_val; + size_t name_len = strlen(param_name); + + tor_assert(max_val > min_val); + tor_assert(min_val <= default_val); + tor_assert(max_val >= default_val); + + SMARTLIST_FOREACH_BEGIN(net_params, const char *, p) { + if (!strcmpstart(p, param_name) && p[name_len] == '=') { + int ok=0; + long v = tor_parse_long(p+name_len+1, 10, INT32_MIN, + INT32_MAX, &ok, NULL); + if (ok) { + res = (int32_t) v; + break; + } + } + } SMARTLIST_FOREACH_END(p); + + if (res < min_val) { + log_warn(LD_DIR, "Consensus parameter %s is too small. Got %d, raising to " + "%d.", param_name, res, min_val); + res = min_val; + } else if (res > max_val) { + log_warn(LD_DIR, "Consensus parameter %s is too large. Got %d, capping to " + "%d.", param_name, res, max_val); + res = max_val; + } + + return res; +} + /** Return the value of a integer parameter from the networkstatus <b>ns</b> * whose name is <b>param_name</b>. If <b>ns</b> is NULL, try loading the * latest consensus ourselves. Return <b>default_val</b> if no latest - * consensus, or if it has no parameter called <b>param_name</b>. */ + * consensus, or if it has no parameter called <b>param_name</b>. + * Make sure the value parsed from the consensus is at least + * <b>min_val</b> and at most <b>max_val</b> and raise/cap the parsed value + * if necessary. */ int32_t networkstatus_get_param(networkstatus_t *ns, const char *param_name, - int32_t default_val) + int32_t default_val, int32_t min_val, int32_t max_val) { - size_t name_len; - if (!ns) /* if they pass in null, go find it ourselves */ ns = networkstatus_get_latest_consensus(); if (!ns || !ns->net_params) return default_val; - name_len = strlen(param_name); + return get_net_param_from_list(ns->net_params, param_name, + default_val, min_val, max_val); +} - SMARTLIST_FOREACH_BEGIN(ns->net_params, const char *, p) { - if (!strcmpstart(p, param_name) && p[name_len] == '=') { - int ok=0; - long v = tor_parse_long(p+name_len+1, 10, INT32_MIN, INT32_MAX, &ok, - NULL); - if (ok) - return (int32_t) v; - } - } SMARTLIST_FOREACH_END(p); +/** Return the value of a integer bw weight parameter from the networkstatus + * <b>ns</b> whose name is <b>weight_name</b>. If <b>ns</b> is NULL, try + * loading the latest consensus ourselves. Return <b>default_val</b> if no + * latest consensus, or if it has no parameter called <b>weight_name</b>. */ +int32_t +networkstatus_get_bw_weight(networkstatus_t *ns, const char *weight_name, + int32_t default_val) +{ + int32_t param; + int max; + if (!ns) /* if they pass in null, go find it ourselves */ + ns = networkstatus_get_latest_consensus(); + + if (!ns || !ns->weight_params) + return default_val; + + max = circuit_build_times_get_bw_scale(ns); + param = get_net_param_from_list(ns->weight_params, weight_name, + default_val, -1, + BW_MAX_WEIGHT_SCALE); + if (param > max) { + log_warn(LD_DIR, "Value of consensus weight %s was too large, capping " + "to %d", weight_name, max); + param = max; + } + return param; +} + +/** Return the name of the consensus flavor <b>flav</b> as used to identify + * the flavor in directory documents. */ +const char * +networkstatus_get_flavor_name(consensus_flavor_t flav) +{ + switch (flav) { + case FLAV_NS: + return "ns"; + case FLAV_MICRODESC: + return "microdesc"; + default: + tor_fragile_assert(); + return "??"; + } +} - return default_val; +/** Return the consensus_flavor_t value for the flavor called <b>flavname</b>, + * or -1 if the flavor is not recognized. */ +int +networkstatus_parse_flavor_name(const char *flavname) +{ + if (!strcmp(flavname, "ns")) + return FLAV_NS; + else if (!strcmp(flavname, "microdesc")) + return FLAV_MICRODESC; + else + return -1; } /** If <b>question</b> is a string beginning with "ns/" in a format the @@ -1935,7 +2250,8 @@ networkstatus_get_param(networkstatus_t *ns, const char *param_name, * ORs. Return 0 on success, -1 on unrecognized question format. */ int getinfo_helper_networkstatus(control_connection_t *conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { routerstatus_t *status; (void) conn; @@ -1959,8 +2275,10 @@ getinfo_helper_networkstatus(control_connection_t *conn, } else if (!strcmpstart(question, "ns/id/")) { char d[DIGEST_LEN]; - if (base16_decode(d, DIGEST_LEN, question+6, strlen(question+6))) + if (base16_decode(d, DIGEST_LEN, question+6, strlen(question+6))) { + *errmsg = "Data not decodeable as hex"; return -1; + } status = router_get_consensus_status_by_id(d); } else if (!strcmpstart(question, "ns/name/")) { status = router_get_consensus_status_by_nickname(question+8, 0); @@ -1968,7 +2286,7 @@ getinfo_helper_networkstatus(control_connection_t *conn, *answer = networkstatus_getinfo_by_purpose(question+11, time(NULL)); return *answer ? 0 : -1; } else { - return -1; + return 0; } if (status) @@ -1980,30 +2298,29 @@ getinfo_helper_networkstatus(control_connection_t *conn, void networkstatus_free_all(void) { + int i; if (networkstatus_v2_list) { SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, networkstatus_v2_free(ns)); smartlist_free(networkstatus_v2_list); networkstatus_v2_list = NULL; } - if (v2_download_status_map) { - digestmap_free(v2_download_status_map, _tor_free); - v2_download_status_map = NULL; - } - if (current_consensus) { - networkstatus_vote_free(current_consensus); - current_consensus = NULL; - } - if (consensus_waiting_for_certs) { - networkstatus_vote_free(consensus_waiting_for_certs); - consensus_waiting_for_certs = NULL; - } - tor_free(consensus_waiting_for_certs_body); - if (named_server_map) { - strmap_free(named_server_map, _tor_free); - } - if (unnamed_server_map) { - strmap_free(unnamed_server_map, NULL); + + digestmap_free(v2_download_status_map, _tor_free); + v2_download_status_map = NULL; + networkstatus_vote_free(current_consensus); + current_consensus = NULL; + + for (i=0; i < N_CONSENSUS_FLAVORS; ++i) { + consensus_waiting_for_certs_t *waiting = &consensus_waiting_for_certs[i]; + if (waiting->consensus) { + networkstatus_vote_free(waiting->consensus); + waiting->consensus = NULL; + } + tor_free(waiting->body); } + + strmap_free(named_server_map, _tor_free); + strmap_free(unnamed_server_map, NULL); } diff --git a/src/or/networkstatus.h b/src/or/networkstatus.h new file mode 100644 index 000000000..ec2e8f884 --- /dev/null +++ b/src/or/networkstatus.h @@ -0,0 +1,99 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file networkstatus.h + * \brief Header file for networkstatus.c. + **/ + +#ifndef _TOR_NETWORKSTATUS_H +#define _TOR_NETWORKSTATUS_H + +/** How old do we allow a v2 network-status to get before removing it + * completely? */ +#define MAX_NETWORKSTATUS_AGE (10*24*60*60) + +void networkstatus_reset_warnings(void); +void networkstatus_reset_download_failures(void); +int router_reload_v2_networkstatus(void); +int router_reload_consensus_networkstatus(void); +void routerstatus_free(routerstatus_t *rs); +void networkstatus_v2_free(networkstatus_v2_t *ns); +void networkstatus_vote_free(networkstatus_t *ns); +networkstatus_voter_info_t *networkstatus_get_voter_by_id( + networkstatus_t *vote, + const char *identity); +int networkstatus_check_consensus_signature(networkstatus_t *consensus, + int warn); +int networkstatus_check_document_signature(const networkstatus_t *consensus, + document_signature_t *sig, + const authority_cert_t *cert); +char *networkstatus_get_cache_filename(const char *identity_digest); +int router_set_networkstatus_v2(const char *s, time_t arrived_at, + v2_networkstatus_source_t source, + smartlist_t *requested_fingerprints); +void networkstatus_v2_list_clean(time_t now); +int compare_digest_to_routerstatus_entry(const void *_key, + const void **_member); +routerstatus_t *networkstatus_v2_find_entry(networkstatus_v2_t *ns, + const char *digest); +routerstatus_t *networkstatus_vote_find_entry(networkstatus_t *ns, + const char *digest); +int networkstatus_vote_find_entry_idx(networkstatus_t *ns, + const char *digest, int *found_out); +const smartlist_t *networkstatus_get_v2_list(void); +download_status_t *router_get_dl_status_by_descriptor_digest(const char *d); +routerstatus_t *router_get_consensus_status_by_id(const char *digest); +routerstatus_t *router_get_consensus_status_by_descriptor_digest( + const char *digest); +routerstatus_t *router_get_consensus_status_by_nickname(const char *nickname, + int warn_if_unnamed); +const char *networkstatus_get_router_digest_by_nickname(const char *nickname); +int networkstatus_nickname_is_unnamed(const char *nickname); +void networkstatus_consensus_download_failed(int status_code); +void update_consensus_networkstatus_fetch_time(time_t now); +int should_delay_dir_fetches(or_options_t *options); +void update_networkstatus_downloads(time_t now); +void update_certificate_downloads(time_t now); +int consensus_is_waiting_for_certs(void); +networkstatus_v2_t *networkstatus_v2_get_by_digest(const char *digest); +networkstatus_t *networkstatus_get_latest_consensus(void); +networkstatus_t *networkstatus_get_live_consensus(time_t now); +networkstatus_t *networkstatus_get_reasonably_live_consensus(time_t now); +#define NSSET_FROM_CACHE 1 +#define NSSET_WAS_WAITING_FOR_CERTS 2 +#define NSSET_DONT_DOWNLOAD_CERTS 4 +#define NSSET_ACCEPT_OBSOLETE 8 +#define NSSET_REQUIRE_FLAVOR 16 +int networkstatus_set_current_consensus(const char *consensus, + const char *flavor, + unsigned flags); +void networkstatus_note_certs_arrived(void); +void routers_update_all_from_networkstatus(time_t now, int dir_version); +void routers_update_status_from_consensus_networkstatus(smartlist_t *routers, + int reset_failures); +void signed_descs_update_status_from_consensus_networkstatus( + smartlist_t *descs); + +char *networkstatus_getinfo_helper_single(routerstatus_t *rs); +char *networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now); +void networkstatus_dump_bridge_status_to_file(time_t now); +int32_t networkstatus_get_param(networkstatus_t *ns, const char *param_name, + int32_t default_val, int32_t min_val, + int32_t max_val); +int getinfo_helper_networkstatus(control_connection_t *conn, + const char *question, char **answer, + const char **errmsg); +int32_t networkstatus_get_bw_weight(networkstatus_t *ns, const char *weight, + int32_t default_val); +const char *networkstatus_get_flavor_name(consensus_flavor_t flav); +int networkstatus_parse_flavor_name(const char *flavname); +void document_signature_free(document_signature_t *sig); +document_signature_t *document_signature_dup(const document_signature_t *sig); +void networkstatus_free_all(void); + +#endif + diff --git a/src/or/ntmain.c b/src/or/ntmain.c index 4dcf9b376..b2fee648c 100644 --- a/src/or/ntmain.c +++ b/src/or/ntmain.c @@ -5,13 +5,22 @@ #define MAIN_PRIVATE #include "or.h" - -#include <tchar.h> -#define GENSRV_SERVICENAME TEXT("tor") -#define GENSRV_DISPLAYNAME TEXT("Tor Win32 Service") +#include "config.h" +#include "main.h" +#include "ntmain.h" + +#ifdef HAVE_EVENT2_EVENT_H +#include <event2/event.h> +#else +#include <event.h> +#endif + +#include <windows.h> +#define GENSRV_SERVICENAME "tor" +#define GENSRV_DISPLAYNAME "Tor Win32 Service" #define GENSRV_DESCRIPTION \ - TEXT("Provides an anonymous Internet communication system") -#define GENSRV_USERACCT TEXT("NT AUTHORITY\\LocalService") + "Provides an anonymous Internet communication system" +#define GENSRV_USERACCT "NT AUTHORITY\\LocalService" // Cheating: using the pre-defined error codes, tricks Windows into displaying // a semi-related human-readable error message if startup fails as @@ -27,7 +36,6 @@ static SERVICE_STATUS_HANDLE hStatus; * to the NT service functions. */ static char **backup_argv; static int backup_argc; -static char* nt_strerror(uint32_t errnum); static void nt_service_control(DWORD request); static void nt_service_body(int argc, char **argv); @@ -46,6 +54,11 @@ static int nt_service_cmd_stop(void); struct service_fns { int loaded; + /** @{ */ + /** Function pointers for Windows API functions related to service + * management. These are NULL, or they point to the . They're set by + * calling the LOAD macro below. */ + BOOL (WINAPI *ChangeServiceConfig2A_fn)( SC_HANDLE hService, DWORD dwInfoLevel, @@ -61,30 +74,30 @@ struct service_fns { SC_HANDLE (WINAPI *CreateServiceA_fn)( SC_HANDLE hSCManager, - LPCTSTR lpServiceName, - LPCTSTR lpDisplayName, + LPCSTR lpServiceName, + LPCSTR lpDisplayName, DWORD dwDesiredAccess, DWORD dwServiceType, DWORD dwStartType, DWORD dwErrorControl, - LPCTSTR lpBinaryPathName, - LPCTSTR lpLoadOrderGroup, + LPCSTR lpBinaryPathName, + LPCSTR lpLoadOrderGroup, LPDWORD lpdwTagId, - LPCTSTR lpDependencies, - LPCTSTR lpServiceStartName, - LPCTSTR lpPassword); + LPCSTR lpDependencies, + LPCSTR lpServiceStartName, + LPCSTR lpPassword); BOOL (WINAPI *DeleteService_fn)( SC_HANDLE hService); SC_HANDLE (WINAPI *OpenSCManagerA_fn)( - LPCTSTR lpMachineName, - LPCTSTR lpDatabaseName, + LPCSTR lpMachineName, + LPCSTR lpDatabaseName, DWORD dwDesiredAccess); SC_HANDLE (WINAPI *OpenServiceA_fn)( SC_HANDLE hSCManager, - LPCTSTR lpServiceName, + LPCSTR lpServiceName, DWORD dwDesiredAccess); BOOL (WINAPI *QueryServiceStatus_fn)( @@ -92,28 +105,29 @@ struct service_fns { LPSERVICE_STATUS lpServiceStatus); SERVICE_STATUS_HANDLE (WINAPI *RegisterServiceCtrlHandlerA_fn)( - LPCTSTR lpServiceName, + LPCSTR lpServiceName, LPHANDLER_FUNCTION lpHandlerProc); BOOL (WINAPI *SetServiceStatus_fn)(SERVICE_STATUS_HANDLE, LPSERVICE_STATUS); BOOL (WINAPI *StartServiceCtrlDispatcherA_fn)( - const SERVICE_TABLE_ENTRY* lpServiceTable); + const SERVICE_TABLE_ENTRYA* lpServiceTable); BOOL (WINAPI *StartServiceA_fn)( SC_HANDLE hService, DWORD dwNumServiceArgs, - LPCTSTR* lpServiceArgVectors); + LPCSTR* lpServiceArgVectors); BOOL (WINAPI *LookupAccountNameA_fn)( - LPCTSTR lpSystemName, - LPCTSTR lpAccountName, + LPCSTR lpSystemName, + LPCSTR lpAccountName, PSID Sid, LPDWORD cbSid, LPTSTR ReferencedDomainName, LPDWORD cchReferencedDomainName, PSID_NAME_USE peUse); + /** @} */ } service_fns = { 0, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, @@ -130,13 +144,16 @@ nt_service_loadlibrary(void) if (service_fns.loaded) return; - /* XXXX Possibly, we should hardcode the location of this DLL. */ - if (!(library = LoadLibrary("advapi32.dll"))) { + if (!(library = load_windows_system_library(TEXT("advapi32.dll")))) { log_err(LD_GENERAL, "Couldn't open advapi32.dll. Are you trying to use " "NT services on Windows 98? That doesn't work."); goto err; } +/* Helper macro: try to load a function named <b>f</b> from "library" into + * service_functions.<b>f</b>_fn. On failure, log an error message, and goto + * err. + */ #define LOAD(f) STMT_BEGIN \ if (!(fn = GetProcAddress(library, #f))) { \ log_err(LD_BUG, \ @@ -218,7 +235,7 @@ nt_service_control(DWORD request) log_notice(LD_GENERAL, "Got stop/shutdown request; shutting down cleanly."); service_status.dwCurrentState = SERVICE_STOP_PENDING; - event_loopexit(&exit_now); + event_base_loopexit(tor_libevent_get_base(), &exit_now); return; } service_fns.SetServiceStatus_fn(hStatus, &service_status); @@ -275,20 +292,20 @@ nt_service_body(int argc, char **argv) static void nt_service_main(void) { - SERVICE_TABLE_ENTRY table[2]; + SERVICE_TABLE_ENTRYA table[2]; DWORD result = 0; char *errmsg; nt_service_loadlibrary(); table[0].lpServiceName = (char*)GENSRV_SERVICENAME; - table[0].lpServiceProc = (LPSERVICE_MAIN_FUNCTION)nt_service_body; + table[0].lpServiceProc = (LPSERVICE_MAIN_FUNCTIONA)nt_service_body; table[1].lpServiceName = NULL; table[1].lpServiceProc = NULL; if (!service_fns.StartServiceCtrlDispatcherA_fn(table)) { result = GetLastError(); - errmsg = nt_strerror(result); + errmsg = format_win32_error(result); printf("Service error %d : %s\n", (int) result, errmsg); - LocalFree(errmsg); + tor_free(errmsg); if (result == ERROR_FAILED_SERVICE_CONTROLLER_CONNECT) { if (tor_init(backup_argc, backup_argv) < 0) return; @@ -323,9 +340,9 @@ nt_service_open_scm(void) nt_service_loadlibrary(); if ((hSCManager = service_fns.OpenSCManagerA_fn( NULL, NULL, SC_MANAGER_CREATE_SERVICE)) == NULL) { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("OpenSCManager() failed : %s\n", errmsg); - LocalFree(errmsg); + tor_free(errmsg); } return hSCManager; } @@ -340,9 +357,9 @@ nt_service_open(SC_HANDLE hSCManager) nt_service_loadlibrary(); if ((hService = service_fns.OpenServiceA_fn(hSCManager, GENSRV_SERVICENAME, SERVICE_ALL_ACCESS)) == NULL) { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("OpenService() failed : %s\n", errmsg); - LocalFree(errmsg); + tor_free(errmsg); } return hService; } @@ -374,14 +391,14 @@ nt_service_start(SC_HANDLE hService) printf("Service started successfully\n"); return 0; } else { - errmsg = nt_strerror(service_status.dwWin32ExitCode); + errmsg = format_win32_error(service_status.dwWin32ExitCode); printf("Service failed to start : %s\n", errmsg); - LocalFree(errmsg); + tor_free(errmsg); } } else { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("StartService() failed : %s\n", errmsg); - LocalFree(errmsg); + tor_free(errmsg); } return -1; } @@ -418,14 +435,14 @@ nt_service_stop(SC_HANDLE hService) } else if (wait_time == MAX_SERVICE_WAIT_TIME) { printf("Service did not stop within %d seconds.\n", wait_time); } else { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("QueryServiceStatus() failed : %s\n",errmsg); - LocalFree(errmsg); + tor_free(errmsg); } } else { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("ControlService() failed : %s\n", errmsg); - LocalFree(errmsg); + tor_free(errmsg); } return -1; } @@ -439,6 +456,7 @@ static char * nt_service_command_line(int *using_default_torrc) { TCHAR tor_exe[MAX_PATH+1]; + char tor_exe_ascii[MAX_PATH+1]; char *command, *options=NULL; smartlist_t *sl; int i, cmdlen; @@ -464,18 +482,25 @@ nt_service_command_line(int *using_default_torrc) options = smartlist_join_strings(sl,"\" \"",0,NULL); smartlist_free(sl); +#ifdef UNICODE + wcstombs(tor_exe_ascii, tor_exe, sizeof(tor_exe_ascii)); +#else + strlcpy(tor_exe_ascii, tor_exe, sizeof(tor_exe_ascii)); +#endif + /* Allocate a string for the NT service command line */ - cmdlen = strlen(tor_exe) + (options?strlen(options):0) + 32; + cmdlen = strlen(tor_exe_ascii) + (options?strlen(options):0) + 32; command = tor_malloc(cmdlen); /* Format the service command */ if (options) { if (tor_snprintf(command, cmdlen, "\"%s\" --nt-service \"%s\"", - tor_exe, options)<0) { + tor_exe_ascii, options)<0) { tor_free(command); /* sets command to NULL. */ } } else { /* ! options */ - if (tor_snprintf(command, cmdlen, "\"%s\" --nt-service", tor_exe)<0) { + if (tor_snprintf(command, cmdlen, "\"%s\" --nt-service", + tor_exe_ascii)<0) { tor_free(command); /* sets command to NULL. */ } } @@ -500,10 +525,10 @@ nt_service_install(int argc, char **argv) SC_HANDLE hSCManager = NULL; SC_HANDLE hService = NULL; - SERVICE_DESCRIPTION sdBuff; + SERVICE_DESCRIPTIONA sdBuff; char *command; char *errmsg; - const char *user_acct = GENSRV_USERACCT; + const char *user_acct = NULL; const char *password = ""; int i; OSVERSIONINFOEX info; @@ -547,13 +572,12 @@ nt_service_install(int argc, char **argv) is_win2k_or_worse = 1; } - if (user_acct == GENSRV_USERACCT) { + if (!user_acct) { if (is_win2k_or_worse) { /* On Win2k, there is no LocalService account, so we actually need to * fall back on NULL (the system account). */ printf("Running on Win2K or earlier, so the LocalService account " "doesn't exist. Falling back to SYSTEM account.\n"); - user_acct = NULL; } else { /* Genericity is apparently _so_ last year in Redmond, where some * accounts are accounts that you can look up, and some accounts @@ -562,6 +586,7 @@ nt_service_install(int argc, char **argv) */ printf("Running on a Post-Win2K OS, so we'll assume that the " "LocalService account exists.\n"); + user_acct = GENSRV_USERACCT; } } else if (0 && service_fns.LookupAccountNameA_fn(NULL, // On this system user_acct, @@ -590,10 +615,10 @@ nt_service_install(int argc, char **argv) SERVICE_AUTO_START, SERVICE_ERROR_IGNORE, command, NULL, NULL, NULL, user_acct, password)) == NULL) { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("CreateService() failed : %s\n", errmsg); service_fns.CloseServiceHandle_fn(hSCManager); - LocalFree(errmsg); + tor_free(errmsg); tor_free(command); return -1; } @@ -634,9 +659,9 @@ nt_service_remove(void) nt_service_stop(hService); if (service_fns.DeleteService_fn(hService) == FALSE) { - errmsg = nt_strerror(GetLastError()); + errmsg = format_win32_error(GetLastError()); printf("DeleteService() failed : %s\n", errmsg); - LocalFree(errmsg); + tor_free(errmsg); service_fns.CloseServiceHandle_fn(hService); service_fns.CloseServiceHandle_fn(hSCManager); return -1; @@ -693,20 +718,6 @@ nt_service_cmd_stop(void) return stop; } -/** Given a Win32 error code, this attempts to make Windows - * return a human-readable error message. The char* returned - * is allocated by Windows, but should be freed with LocalFree() - * when finished with it. */ -static char* -nt_strerror(uint32_t errnum) -{ - char *msgbuf; - FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM, - NULL, errnum, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), - (LPSTR)&msgbuf, 0, NULL); - return msgbuf; -} - int nt_service_parse_options(int argc, char **argv, int *should_exit) { diff --git a/src/or/ntmain.h b/src/or/ntmain.h new file mode 100644 index 000000000..acd0e1d7e --- /dev/null +++ b/src/or/ntmain.h @@ -0,0 +1,30 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file ntmain.h + * \brief Header file for ntmain.c. + **/ + +#ifndef _TOR_NTMAIN_H +#define _TOR_NTMAIN_H + +#ifdef MS_WINDOWS +#if !defined (WINCE) +#define NT_SERVICE +#endif +#endif + +#ifdef NT_SERVICE +int nt_service_parse_options(int argc, char **argv, int *should_exit); +int nt_service_is_stopping(void); +void nt_service_set_state(DWORD state); +#else +#define nt_service_is_stopping() 0 +#endif + +#endif + diff --git a/src/or/onion.c b/src/or/onion.c index e455a5263..e1d10a60b 100644 --- a/src/or/onion.c +++ b/src/or/onion.c @@ -11,6 +11,10 @@ **/ #include "or.h" +#include "circuitlist.h" +#include "config.h" +#include "onion.h" +#include "rephist.h" /** Type for a linked list of circuits that are waiting for a free CPU worker * to process a waiting onion handshake. */ @@ -58,11 +62,18 @@ onion_pending_add(or_circuit_t *circ, char *onionskin) tor_assert(!ol_tail->next); if (ol_length >= get_options()->MaxOnionsPending) { - log_warn(LD_GENERAL, - "Your computer is too slow to handle this many circuit " - "creation requests! Please consider using the " - "MaxAdvertisedBandwidth config option or choosing a more " - "restricted exit policy."); +#define WARN_TOO_MANY_CIRC_CREATIONS_INTERVAL (60) + static ratelim_t last_warned = + RATELIM_INIT(WARN_TOO_MANY_CIRC_CREATIONS_INTERVAL); + char *m; + if ((m = rate_limit_log(&last_warned, approx_time()))) { + log_warn(LD_GENERAL, + "Your computer is too slow to handle this many circuit " + "creation requests! Please consider using the " + "MaxAdvertisedBandwidth config option or choosing a more " + "restricted exit policy.%s",m); + tor_free(m); + } tor_free(tmp); return -1; } @@ -248,6 +259,10 @@ onion_skin_server_handshake(const char *onion_skin, /*ONIONSKIN_CHALLENGE_LEN*/ } dh = crypto_dh_new(DH_TYPE_CIRCUIT); + if (!dh) { + log_warn(LD_BUG, "Couldn't allocate DH key"); + goto err; + } if (crypto_dh_get_public(dh, handshake_reply_out, DH_KEY_LEN)) { log_info(LD_GENERAL, "crypto_dh_get_public failed."); goto err; @@ -255,8 +270,9 @@ onion_skin_server_handshake(const char *onion_skin, /*ONIONSKIN_CHALLENGE_LEN*/ key_material_len = DIGEST_LEN+key_out_len; key_material = tor_malloc(key_material_len); - len = crypto_dh_compute_secret(dh, challenge, DH_KEY_LEN, - key_material, key_material_len); + len = crypto_dh_compute_secret(LOG_PROTOCOL_WARN, dh, challenge, + DH_KEY_LEN, key_material, + key_material_len); if (len < 0) { log_info(LD_GENERAL, "crypto_dh_compute_secret failed."); goto err; @@ -306,8 +322,9 @@ onion_skin_client_handshake(crypto_dh_env_t *handshake_state, key_material_len = DIGEST_LEN + key_out_len; key_material = tor_malloc(key_material_len); - len = crypto_dh_compute_secret(handshake_state, handshake_reply, DH_KEY_LEN, - key_material, key_material_len); + len = crypto_dh_compute_secret(LOG_PROTOCOL_WARN, handshake_state, + handshake_reply, DH_KEY_LEN, key_material, + key_material_len); if (len < 0) goto err; diff --git a/src/or/onion.h b/src/or/onion.h new file mode 100644 index 000000000..7f603b814 --- /dev/null +++ b/src/or/onion.h @@ -0,0 +1,48 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file onion.h + * \brief Header file for onion.c. + **/ + +#ifndef _TOR_ONION_H +#define _TOR_ONION_H + +int onion_pending_add(or_circuit_t *circ, char *onionskin); +or_circuit_t *onion_next_task(char **onionskin_out); +void onion_pending_remove(or_circuit_t *circ); + +int onion_skin_create(crypto_pk_env_t *router_key, + crypto_dh_env_t **handshake_state_out, + char *onion_skin_out); + +int onion_skin_server_handshake(const char *onion_skin, + crypto_pk_env_t *private_key, + crypto_pk_env_t *prev_private_key, + char *handshake_reply_out, + char *key_out, + size_t key_out_len); + +int onion_skin_client_handshake(crypto_dh_env_t *handshake_state, + const char *handshake_reply, + char *key_out, + size_t key_out_len); + +int fast_server_handshake(const uint8_t *key_in, + uint8_t *handshake_reply_out, + uint8_t *key_out, + size_t key_out_len); + +int fast_client_handshake(const uint8_t *handshake_state, + const uint8_t *handshake_reply_out, + uint8_t *key_out, + size_t key_out_len); + +void clear_pending_onions(void); + +#endif + diff --git a/src/or/or.h b/src/or/or.h index 897ad32a4..d667358eb 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -20,9 +20,6 @@ #ifndef INSTRUMENT_DOWNLOADS #define INSTRUMENT_DOWNLOADS 1 #endif -#ifndef ENABLE_GEOIP_STATS -#define ENABLE_GEOIP_STATS 1 -#endif #endif #ifdef MS_WINDOWS @@ -62,6 +59,9 @@ #ifdef HAVE_SYS_STAT_H #include <sys/stat.h> #endif +#ifdef HAVE_NETINET_IN_H +#include <netinet/in.h> +#endif #ifdef HAVE_ARPA_INET_H #include <arpa/inet.h> #endif @@ -83,18 +83,15 @@ #define snprintf _snprintf #endif -#include "crypto.h" #include "tortls.h" -#include "log.h" -#include "compat.h" +#include "../common/torlog.h" #include "container.h" -#include "util.h" #include "torgzip.h" #include "address.h" +#include "compat_libevent.h" +#include "ht.h" -#include <event.h> - -/* These signals are defined to help control_signal_act work. +/* These signals are defined to help handle_control_signal work. */ #ifndef SIGHUP #define SIGHUP 1 @@ -161,7 +158,7 @@ #define MAX_DNS_TTL (3*60*60) /** How small can a TTL be before we stop believing it? Provides rudimentary * pinning. */ -#define MIN_DNS_TTL (60) +#define MIN_DNS_TTL 60 /** How often do we rotate onion keys? */ #define MIN_ONION_KEY_LIFETIME (7*24*60*60) @@ -221,6 +218,21 @@ typedef enum { /* !!!! If _CONN_TYPE_MAX is ever over 15, we must grow the type field in * connection_t. */ +/* Proxy client types */ +#define PROXY_NONE 0 +#define PROXY_CONNECT 1 +#define PROXY_SOCKS4 2 +#define PROXY_SOCKS5 3 + +/* Proxy client handshake states */ +#define PROXY_HTTPS_WANT_CONNECT_OK 1 +#define PROXY_SOCKS4_WANT_CONNECT_OK 2 +#define PROXY_SOCKS5_WANT_AUTH_METHOD_NONE 3 +#define PROXY_SOCKS5_WANT_AUTH_METHOD_RFC1929 4 +#define PROXY_SOCKS5_WANT_AUTH_RFC1929_OK 5 +#define PROXY_SOCKS5_WANT_CONNECT_OK 6 +#define PROXY_CONNECTED 7 + /** True iff <b>x</b> is an edge connection. */ #define CONN_IS_EDGE(x) \ ((x)->type == CONN_TYPE_EXIT || (x)->type == CONN_TYPE_AP) @@ -241,26 +253,24 @@ typedef enum { #define _OR_CONN_STATE_MIN 1 /** State for a connection to an OR: waiting for connect() to finish. */ #define OR_CONN_STATE_CONNECTING 1 -/** State for a connection to an OR: waiting for proxy command to flush. */ -#define OR_CONN_STATE_PROXY_FLUSHING 2 -/** State for a connection to an OR: waiting for proxy response. */ -#define OR_CONN_STATE_PROXY_READING 3 +/** State for a connection to an OR: waiting for proxy handshake to complete */ +#define OR_CONN_STATE_PROXY_HANDSHAKING 2 /** State for a connection to an OR or client: SSL is handshaking, not done * yet. */ -#define OR_CONN_STATE_TLS_HANDSHAKING 4 +#define OR_CONN_STATE_TLS_HANDSHAKING 3 /** State for a connection to an OR: We're doing a second SSL handshake for * renegotiation purposes. */ -#define OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING 5 +#define OR_CONN_STATE_TLS_CLIENT_RENEGOTIATING 4 /** State for a connection at an OR: We're waiting for the client to * renegotiate. */ -#define OR_CONN_STATE_TLS_SERVER_RENEGOTIATING 6 +#define OR_CONN_STATE_TLS_SERVER_RENEGOTIATING 5 /** State for a connection to an OR: We're done with our SSL handshake, but we * haven't yet negotiated link protocol versions and sent a netinfo cell. */ -#define OR_CONN_STATE_OR_HANDSHAKING 7 +#define OR_CONN_STATE_OR_HANDSHAKING 6 /** State for a connection to an OR: Ready to send/receive cells. */ -#define OR_CONN_STATE_OPEN 8 -#define _OR_CONN_STATE_MAX 8 +#define OR_CONN_STATE_OPEN 7 +#define _OR_CONN_STATE_MAX 7 #define _EXIT_CONN_STATE_MIN 1 /** State for an exit connection: waiting for response from DNS farm. */ @@ -457,23 +467,23 @@ typedef enum { #define CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED 11 /** Client-side circuit purpose: at Alice, rendezvous established. */ #define CIRCUIT_PURPOSE_C_REND_JOINED 12 - -#define _CIRCUIT_PURPOSE_C_MAX 12 - +/** This circuit is used for build time measurement only */ +#define CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT 13 +#define _CIRCUIT_PURPOSE_C_MAX 13 /** Hidden-service-side circuit purpose: at Bob, waiting for introductions. */ -#define CIRCUIT_PURPOSE_S_ESTABLISH_INTRO 13 +#define CIRCUIT_PURPOSE_S_ESTABLISH_INTRO 14 /** Hidden-service-side circuit purpose: at Bob, successfully established * intro. */ -#define CIRCUIT_PURPOSE_S_INTRO 14 +#define CIRCUIT_PURPOSE_S_INTRO 15 /** Hidden-service-side circuit purpose: at Bob, connecting to rend point. */ -#define CIRCUIT_PURPOSE_S_CONNECT_REND 15 +#define CIRCUIT_PURPOSE_S_CONNECT_REND 16 /** Hidden-service-side circuit purpose: at Bob, rendezvous established. */ -#define CIRCUIT_PURPOSE_S_REND_JOINED 16 +#define CIRCUIT_PURPOSE_S_REND_JOINED 17 /** A testing circuit; not meant to be used for actual traffic. */ -#define CIRCUIT_PURPOSE_TESTING 17 +#define CIRCUIT_PURPOSE_TESTING 18 /** A controller made this circuit and Tor should not use it. */ -#define CIRCUIT_PURPOSE_CONTROLLER 18 -#define _CIRCUIT_PURPOSE_MAX 18 +#define CIRCUIT_PURPOSE_CONTROLLER 19 +#define _CIRCUIT_PURPOSE_MAX 19 /** A catch-all for unrecognized purposes. Currently we don't expect * to make or see any circuits with this purpose. */ #define CIRCUIT_PURPOSE_UNKNOWN 255 @@ -483,7 +493,7 @@ typedef enum { #define CIRCUIT_PURPOSE_IS_ORIGIN(p) ((p)>_CIRCUIT_PURPOSE_OR_MAX) /** True iff the circuit purpose <b>p</b> is for a circuit that originated * here to serve as a client. (Hidden services don't count here.) */ -#define CIRCUIT_PURPOSE_IS_CLIENT(p) \ +#define CIRCUIT_PURPOSE_IS_CLIENT(p) \ ((p)> _CIRCUIT_PURPOSE_OR_MAX && \ (p)<=_CIRCUIT_PURPOSE_C_MAX) /** True iff the circuit_t <b>c</b> is actually an origin_circuit_t. */ @@ -573,6 +583,9 @@ typedef enum { /** This is a connection on the NATD port, and the destination IP:Port was * either ill-formed or out-of-range. */ #define END_STREAM_REASON_INVALID_NATD_DEST 261 +/** The target address is in a private network (like 127.0.0.1 or 10.0.0.1); + * you don't want to do that over a randomly chosen exit */ +#define END_STREAM_REASON_PRIVATE_ADDR 262 /** Bitwise-and this value with endreason to mask out all flags. */ #define END_STREAM_REASON_MASK 511 @@ -604,6 +617,10 @@ typedef enum { /* Negative reasons are internal: we never send them in a DESTROY or TRUNCATE * call; they only go to the controller for tracking */ +/** Our post-timeout circuit time measurement period expired. + * We must give up now */ +#define END_CIRC_REASON_MEASUREMENT_EXPIRED -3 + /** We couldn't build a path for this circuit. */ #define END_CIRC_REASON_NOPATH -2 /** Catch-all "other" reason for closing origin circuits. */ @@ -641,10 +658,6 @@ typedef enum { /** Length of a binary-encoded rendezvous service ID. */ #define REND_SERVICE_ID_LEN 10 -/** How long after we receive a hidden service descriptor do we consider - * it fresh? */ -#define NUM_SECONDS_BEFORE_HS_REFETCH (60*15) - /** Time period for which a v2 descriptor will be valid. */ #define REND_TIME_PERIOD_V2_DESC_VALIDITY (24*60*60) @@ -739,12 +752,6 @@ typedef struct rend_data_t { /** Rendezvous cookie used by both, client and service. */ char rend_cookie[REND_COOKIE_LEN]; - - /** Rendezvous descriptor version that is used by a service. Used to - * distinguish introduction and rendezvous points belonging to the same - * rendezvous service ID, but different descriptor versions. - */ - uint8_t rend_desc_version; } rend_data_t; /** Time interval for tracking possible replays of INTRODUCE2 cells. @@ -761,6 +768,8 @@ typedef enum { /** Initial value for both sides of a circuit transmission window when the * circuit is initialized. Measured in cells. */ #define CIRCWINDOW_START 1000 +#define CIRCWINDOW_START_MIN 100 +#define CIRCWINDOW_START_MAX 1000 /** Amount to increment a circuit window when we get a circuit SENDME. */ #define CIRCWINDOW_INCREMENT 100 /** Initial value on both sides of a stream transmission window when the @@ -839,9 +848,13 @@ typedef struct cell_t { /** Parsed variable-length onion routing cell. */ typedef struct var_cell_t { + /** Type of the cell: CELL_VERSIONS, etc. */ uint8_t command; + /** Circuit thich received the cell */ circid_t circ_id; + /** Number of bytes actually stored in <b>payload</b> */ uint16_t payload_len; + /** Payload of this cell */ uint8_t payload[1]; } var_cell_t; @@ -851,12 +864,28 @@ typedef struct packed_cell_t { char body[CELL_NETWORK_SIZE]; /**< Cell as packed for network. */ } packed_cell_t; +/** Number of cells added to a circuit queue including their insertion + * time on 10 millisecond detail; used for buffer statistics. */ +typedef struct insertion_time_elem_t { + struct insertion_time_elem_t *next; /**< Next element in queue. */ + uint32_t insertion_time; /**< When were cells inserted (in 10 ms steps + * starting at 0:00 of the current day)? */ + unsigned counter; /**< How many cells were inserted? */ +} insertion_time_elem_t; + +/** Queue of insertion times. */ +typedef struct insertion_time_queue_t { + struct insertion_time_elem_t *first; /**< First element in queue. */ + struct insertion_time_elem_t *last; /**< Last element in queue. */ +} insertion_time_queue_t; + /** A queue of cells on a circuit, waiting to be added to the * or_connection_t's outbuf. */ typedef struct cell_queue_t { packed_cell_t *head; /**< The first cell, or NULL if the queue is empty. */ packed_cell_t *tail; /**< The last cell, or NULL if the queue is empty. */ int n; /**< The number of cells in the queue. */ + insertion_time_queue_t *insertion_times; /**< Insertion times of cells. */ } cell_queue_t; /** Beginning of a RELAY cell payload. */ @@ -912,7 +941,7 @@ typedef struct connection_t { * again once the bandwidth throttler allows it? */ unsigned int write_blocked_on_bw:1; /**< Boolean: should we start writing * again once the bandwidth throttler allows - * reads? */ + * writes? */ unsigned int hold_open_until_flushed:1; /**< Despite this connection's being * marked for close, do we flush it * before closing it? */ @@ -937,8 +966,11 @@ typedef struct connection_t { * connection. */ unsigned int linked_conn_is_closed:1; - int s; /**< Our socket; -1 if this connection is closed, or has no - * socket. */ + /** CONNECT/SOCKS proxy client handshake state (for outgoing connections). */ + unsigned int proxy_state:4; + + /** Our socket; -1 if this connection is closed, or has no socket. */ + evutil_socket_t s; int conn_array_index; /**< Index into the global connection array. */ struct event *read_event; /**< Libevent event structure. */ struct event *write_event; /**< Libevent event structure. */ @@ -974,12 +1006,14 @@ typedef struct connection_t { /** Unique identifier for this connection on this Tor instance. */ uint64_t global_identifier; - /* XXXX022 move this field, and all the listener-only fields (just + /* XXXX023 move this field, and all the listener-only fields (just socket_family, I think), into a new listener_connection_t subtype. */ /** If the connection is a CONN_TYPE_AP_DNS_LISTENER, this field points * to the evdns_server_port is uses to listen to and answer connections. */ struct evdns_server_port *dns_server_port; + /** Unique ID for measuring tunneled network status requests. */ + uint64_t dirreq_id; } connection_t; /** Stores flags and information related to the portion of a v2 Tor OR @@ -1026,7 +1060,10 @@ typedef struct or_connection_t { * NETINFO cell listed the address we're connected to as recognized. */ unsigned int is_canonical:1; /** True iff this connection shouldn't get any new circs attached to it, - * because the connection is too old, or because there's a better one, etc. + * because the connection is too old, or because there's a better one. + * More generally, this flag is used to note an unhealthy connection; + * for example, if a bad connection fails we shouldn't assume that the + * router itself has a problem. */ unsigned int is_bad_for_new_circs:1; uint8_t link_proto; /**< What protocol version are we using? 0 for @@ -1041,12 +1078,13 @@ typedef struct or_connection_t { time_t timestamp_last_added_nonpadding; /** When did we last add a * non-padding cell to the outbuf? */ - /* bandwidth* and read_bucket only used by ORs in OPEN state: */ + /* bandwidth* and *_bucket only used by ORs in OPEN state: */ int bandwidthrate; /**< Bytes/s added to the bucket. (OPEN ORs only.) */ int bandwidthburst; /**< Max bucket size for this conn. (OPEN ORs only.) */ int read_bucket; /**< When this hits 0, stop receiving. Every second we * add 'bandwidthrate' to this, capping it at * bandwidthburst. (OPEN ORs only) */ + int write_bucket; /**< When this hits 0, stop writing. Like read_bucket. */ int n_circuits; /**< How many circuits use this connection as p_conn or * n_conn ? */ @@ -1054,6 +1092,17 @@ typedef struct or_connection_t { * free up on this connection's outbuf. Every time we pull cells from a * circuit, we advance this pointer to the next circuit in the ring. */ struct circuit_t *active_circuits; + /** Priority queue of cell_ewma_t for circuits with queued cells waiting for + * room to free up on this connection's outbuf. Kept in heap order + * according to EWMA. + * + * This is redundant with active_circuits; if we ever decide only to use the + * cell_ewma algorithm for choosing circuits, we can remove active_circuits. + */ + smartlist_t *active_circuit_pqueue; + /** The tick on which the cell_ewma_ts in active_circuit_pqueue last had + * their ewma values rescaled. */ + unsigned active_circuit_pqueue_last_recalibrated; struct or_connection_t *next_with_same_id; /**< Next connection with same * identity digest as this one. */ } or_connection_t; @@ -1102,6 +1151,13 @@ typedef struct edge_connection_t { * already retried several times. */ uint8_t num_socks_retries; +#define NUM_CIRCUITS_LAUNCHED_THRESHOLD 10 + /** Number of times we've launched a circuit to handle this stream. If + * it gets too high, that could indicate an inconsistency between our + * "launch a circuit to handle this stream" logic and our "attach our + * stream to one of the available circuits" logic. */ + unsigned int num_circuits_launched:4; + /** True iff this connection is for a DNS request only. */ unsigned int is_dns_request:1; @@ -1128,6 +1184,10 @@ typedef struct edge_connection_t { * zero, abandon the associated mapaddress. */ unsigned int chosen_exit_retries:3; + /** True iff this is an AP connection that came from a transparent or + * NATd connection */ + unsigned int is_transparent_ap:1; + /** If this is a DNSPort connection, this field holds the pending DNS * request that we're going to try to answer. */ struct evdns_server_request *dns_server_request; @@ -1150,7 +1210,8 @@ typedef struct dir_connection_t { enum { DIR_SPOOL_NONE=0, DIR_SPOOL_SERVER_BY_DIGEST, DIR_SPOOL_SERVER_BY_FP, DIR_SPOOL_EXTRA_BY_DIGEST, DIR_SPOOL_EXTRA_BY_FP, - DIR_SPOOL_CACHED_DIR, DIR_SPOOL_NETWORKSTATUS + DIR_SPOOL_CACHED_DIR, DIR_SPOOL_NETWORKSTATUS, + DIR_SPOOL_MICRODESC, /* NOTE: if we add another entry, add another bit. */ } dir_spool_src : 3; /** If we're fetching descriptors, what router purpose shall we assign * to them? */ @@ -1178,12 +1239,6 @@ typedef struct control_connection_t { uint32_t event_mask; /**< Bitfield: which events does this controller * care about? */ - unsigned int use_long_names:1; /**< True if we should use long nicknames - * on this (v1) connection. Only settable - * via v1 controllers. */ - /** For control connections only. If set, we send extended info with control - * events as appropriate. */ - unsigned int use_extended_events:1; /** True if we have sent a protocolinfo reply on this connection. */ unsigned int have_sent_protocolinfo:1; @@ -1267,6 +1322,7 @@ typedef struct cached_dir_t { size_t dir_len; /**< Length of <b>dir</b> (not counting its NUL). */ size_t dir_z_len; /**< Length of <b>dir_z</b>. */ time_t published; /**< When was this object published. */ + digests_t digests; /**< Digests of this object (networkstatus only) */ int refcnt; /**< Reference count for this cached_dir_t. */ } cached_dir_t; @@ -1427,6 +1483,9 @@ typedef struct { * directory according to the authorities. */ unsigned int policy_is_reject_star:1; /**< True iff the exit policy for this * router rejects everything. */ + /** True if, after we have added this router, we should re-launch + * tests for it. */ + unsigned int needs_retest_if_added:1; /** Tor can use this router for general positions in circuits. */ #define ROUTER_PURPOSE_GENERAL 0 @@ -1516,6 +1575,9 @@ typedef struct routerstatus_t { unsigned int has_bandwidth:1; /**< The vote/consensus had bw info */ unsigned int has_exitsummary:1; /**< The vote/consensus had exit summaries */ + unsigned int has_measured_bw:1; /**< The vote/consensus had a measured bw */ + + uint32_t measured_bw; /**< Measured bandwidth (capacity) of the router */ uint32_t bandwidth; /**< Bandwidth (capacity) of the router as reported in * the vote/consensus, in kilobytes/sec. */ @@ -1540,6 +1602,52 @@ typedef struct routerstatus_t { } routerstatus_t; +/** A microdescriptor is the smallest amount of information needed to build a + * circuit through a router. They are generated by the directory authorities, + * using information from the uploaded routerinfo documents. They are not + * self-signed, but are rather authenticated by having their hash in a signed + * networkstatus document. */ +typedef struct microdesc_t { + /** Hashtable node, used to look up the microdesc by its digest. */ + HT_ENTRY(microdesc_t) node; + + /* Cache information */ + + /** When was this microdescriptor last listed in a consensus document? + * Once a microdesc has been unlisted long enough, we can drop it. + */ + time_t last_listed; + /** Where is this microdescriptor currently stored? */ + saved_location_t saved_location : 3; + /** If true, do not attempt to cache this microdescriptor on disk. */ + unsigned int no_save : 1; + /** If saved_location == SAVED_IN_CACHE, this field holds the offset of the + * microdescriptor in the cache. */ + off_t off; + + /* The string containing the microdesc. */ + + /** A pointer to the encoded body of the microdescriptor. If the + * saved_location is SAVED_IN_CACHE, then the body is a pointer into an + * mmap'd region. Otherwise, it is a malloc'd string. The string might not + * be NUL-terminated; take the length from <b>bodylen</b>. */ + char *body; + /** The length of the microdescriptor in <b>body</b>. */ + size_t bodylen; + /** A SHA256-digest of the microdescriptor. */ + char digest[DIGEST256_LEN]; + + /* Fields in the microdescriptor. */ + + /** As routerinfo_t.onion_pkey */ + crypto_pk_env_t *onion_pkey; + /** As routerinfo_t.family */ + smartlist_t *family; + /** Encoded exit policy summary */ + char *exitsummary; /**< exit policy summary - + * XXX this probably should not stay a string. */ +} microdesc_t; + /** How many times will we try to download a router's descriptor before giving * up? */ #define MAX_ROUTERDESC_DOWNLOAD_FAILURES 8 @@ -1582,6 +1690,16 @@ typedef struct networkstatus_v2_t { * sorted by identity_digest. */ } networkstatus_v2_t; +/** Linked list of microdesc hash lines for a single router in a directory + * vote. + */ +typedef struct vote_microdesc_hash_t { + /** Next element in the list, or NULL. */ + struct vote_microdesc_hash_t *next; + /** The raw contents of the microdesc hash line, excluding the "m". */ + char *microdesc_hash_line; +} vote_microdesc_hash_t; + /** The claim about a single router, made in a vote. */ typedef struct vote_routerstatus_t { routerstatus_t status; /**< Underlying 'status' object for this router. @@ -1590,31 +1708,46 @@ typedef struct vote_routerstatus_t { * networkstatus_t.known_flags. */ char *version; /**< The version that the authority says this router is * running. */ + /** The hash or hashes that the authority claims this microdesc has. */ + vote_microdesc_hash_t *microdesc; } vote_routerstatus_t; +/** A signature of some document by an authority. */ +typedef struct document_signature_t { + /** Declared SHA-1 digest of this voter's identity key */ + char identity_digest[DIGEST_LEN]; + /** Declared SHA-1 digest of signing key used by this voter. */ + char signing_key_digest[DIGEST_LEN]; + /** Algorithm used to compute the digest of the document. */ + digest_algorithm_t alg; + /** Signature of the signed thing. */ + char *signature; + /** Length of <b>signature</b> */ + int signature_len; + unsigned int bad_signature : 1; /**< Set to true if we've tried to verify + * the sig, and we know it's bad. */ + unsigned int good_signature : 1; /**< Set to true if we've verified the sig + * as good. */ +} document_signature_t; + /** Information about a single voter in a vote or a consensus. */ typedef struct networkstatus_voter_info_t { + /** Declared SHA-1 digest of this voter's identity key */ + char identity_digest[DIGEST_LEN]; char *nickname; /**< Nickname of this voter */ - char identity_digest[DIGEST_LEN]; /**< Digest of this voter's identity key */ + /** Digest of this voter's "legacy" identity key, if any. In vote only; for + * consensuses, we treat legacy keys as additional signers. */ + char legacy_id_digest[DIGEST_LEN]; char *address; /**< Address of this voter, in string format. */ uint32_t addr; /**< Address of this voter, in IPv4, in host order. */ uint16_t dir_port; /**< Directory port of this voter */ uint16_t or_port; /**< OR port of this voter */ char *contact; /**< Contact information for this voter. */ char vote_digest[DIGEST_LEN]; /**< Digest of this voter's vote, as signed. */ - /** Digest of this voter's "legacy" identity key, if any. In vote only; for - * consensuses, we treat legacy keys as additional signers. */ - char legacy_id_digest[DIGEST_LEN]; /* Nothing from here on is signed. */ - char signing_key_digest[DIGEST_LEN]; /**< Declared digest of signing key - * used by this voter. */ - char *signature; /**< Signature from this voter. */ - int signature_len; /**< Length of <b>signature</b> */ - unsigned int bad_signature : 1; /**< Set to true if we've tried to verify - * the sig, and we know it's bad. */ - unsigned int good_signature : 1; /**< Set to true if we've verified the sig - * as good. */ + /** The signature of the document and the signature's status. */ + smartlist_t *sigs; } networkstatus_voter_info_t; /** Enumerates the possible seriousness values of a networkstatus document. */ @@ -1624,10 +1757,25 @@ typedef enum { NS_TYPE_OPINION, } networkstatus_type_t; +/** Enumerates recognized flavors of a consensus networkstatus document. All + * flavors of a consensus are generated from the same set of votes, but they + * present different types information to different versions of Tor. */ +typedef enum { + FLAV_NS = 0, + FLAV_MICRODESC = 1, +} consensus_flavor_t; + +/** Which consensus flavor do we actually want to use to build circuits? */ +#define USABLE_CONSENSUS_FLAVOR FLAV_NS + +/** How many different consensus flavors are there? */ +#define N_CONSENSUS_FLAVORS ((int)(FLAV_MICRODESC)+1) + /** A common structure to hold a v3 network status vote, or a v3 network * status consensus. */ typedef struct networkstatus_t { - networkstatus_type_t type; /**< Vote, consensus, or opinion? */ + networkstatus_type_t type : 8; /**< Vote, consensus, or opinion? */ + consensus_flavor_t flavor : 8; /**< If a consensus, what kind? */ time_t published; /**< Vote only: Time when vote was written. */ time_t valid_after; /**< Time after which this vote or consensus applies. */ time_t fresh_until; /**< Time before which this is the most recent vote or @@ -1659,6 +1807,10 @@ typedef struct networkstatus_t { * consensus, sorted by key. */ smartlist_t *net_params; + /** List of key=value strings for the bw weight parameters in the + * consensus. */ + smartlist_t *weight_params; + /** List of networkstatus_voter_info_t. For a vote, only one element * is included. For a consensus, one element is included for every voter * whose vote contributed to the consensus. */ @@ -1666,8 +1818,8 @@ typedef struct networkstatus_t { struct authority_cert_t *cert; /**< Vote only: the voter's certificate. */ - /** Digest of this document, as signed. */ - char networkstatus_digest[DIGEST_LEN]; + /** Digests of this document, as signed. */ + digests_t digests; /** List of router statuses, sorted by identity digest. For a vote, * the elements are vote_routerstatus_t; for a consensus, the elements @@ -1679,14 +1831,15 @@ typedef struct networkstatus_t { digestmap_t *desc_digest_map; } networkstatus_t; -/** A set of signatures for a networkstatus consensus. All fields are as for - * networkstatus_t. */ +/** A set of signatures for a networkstatus consensus. Unless otherwise + * noted, all fields are as for networkstatus_t. */ typedef struct ns_detached_signatures_t { time_t valid_after; time_t fresh_until; time_t valid_until; - char networkstatus_digest[DIGEST_LEN]; - smartlist_t *signatures; /* list of networkstatus_voter_info_t */ + strmap_t *digests; /**< Map from flavor name to digestset_t */ + strmap_t *signatures; /**< Map from flavor name to list of + * document_signature_t */ } ns_detached_signatures_t; /** Allowable types of desc_store_t. */ @@ -1891,6 +2044,29 @@ typedef struct { time_t expiry_time; } cpath_build_state_t; +/** + * The cell_ewma_t structure keeps track of how many cells a circuit has + * transferred recently. It keeps an EWMA (exponentially weighted moving + * average) of the number of cells flushed from the circuit queue onto a + * connection in connection_or_flush_from_first_active_circuit(). + */ +typedef struct { + /** The last 'tick' at which we recalibrated cell_count. + * + * A cell sent at exactly the start of this tick has weight 1.0. Cells sent + * since the start of this tick have weight greater than 1.0; ones sent + * earlier have less weight. */ + unsigned last_adjusted_tick; + /** The EWMA of the cell count. */ + double cell_count; + /** True iff this is the cell count for a circuit's previous + * connection. */ + unsigned int is_for_p_conn : 1; + /** The position of the circuit within the OR connection's priority + * queue. */ + int heap_index; +} cell_ewma_t; + #define ORIGIN_CIRCUIT_MAGIC 0x35315243u #define OR_CIRCUIT_MAGIC 0x98ABC04Fu @@ -1957,9 +2133,15 @@ typedef struct circuit_t { * length ONIONSKIN_CHALLENGE_LEN. */ char *n_conn_onionskin; - time_t timestamp_created; /**< When was this circuit created? */ - time_t timestamp_dirty; /**< When the circuit was first used, or 0 if the - * circuit is clean. */ + struct timeval timestamp_created; /**< When was the circuit created? */ + /** When the circuit was first used, or 0 if the circuit is clean. + * + * XXXX023 Note that some code will artifically adjust this value backward + * in time in order to indicate that a circuit shouldn't be used for new + * streams, but that it can stay alive as long as it has streams on it. + * That's a kludge we should fix. + */ + time_t timestamp_dirty; uint16_t marked_for_close; /**< Should we close this circuit at the end of * the main loop? (If true, holds the line number @@ -1976,6 +2158,14 @@ typedef struct circuit_t { * linked to an OR connection. */ struct circuit_t *prev_active_on_n_conn; struct circuit_t *next; /**< Next circuit in linked list of all circuits. */ + + /** Unique ID for measuring tunneled network status requests. */ + uint64_t dirreq_id; + + /** The EWMA count for the number of cells flushed from the + * n_conn_cells queue. Used to determine which circuit to flush from next. + */ + cell_ewma_t n_cell_ewma; } circuit_t; /** Largest number of relay_early cells that we can send on a given @@ -2008,6 +2198,13 @@ typedef struct origin_circuit_t { * to the specification? */ unsigned int remaining_relay_early_cells : 4; + /** Set if this circuit is insanely old and we already informed the user */ + unsigned int is_ancient : 1; + + /** Set if this circuit has already been opened. Used to detect + * cannibalized circuits. */ + unsigned int has_opened : 1; + /** What commands were sent over this circuit that decremented the * RELAY_EARLY counter? This is for debugging task 878. */ uint8_t relay_early_commands[MAX_RELAY_EARLY_CELLS_PER_CIRCUIT]; @@ -2098,6 +2295,19 @@ typedef struct or_circuit_t { /** True iff this circuit was made with a CREATE_FAST cell. */ unsigned int is_first_hop : 1; + + /** Number of cells that were removed from circuit queue; reset every + * time when writing buffer stats to disk. */ + uint32_t processed_cells; + + /** Total time in milliseconds that cells spent in both app-ward and + * exit-ward queues of this circuit; reset every time when writing + * buffer stats to disk. */ + uint64_t total_cell_waiting_time; + + /** The EWMA count for the number of cells flushed from the + * p_conn_cells queue. */ + cell_ewma_t p_cell_ewma; } or_circuit_t; /** Convert a circuit subtype to a circuit_t.*/ @@ -2157,6 +2367,9 @@ typedef struct { config_line_t *Logs; /**< New-style list of configuration lines * for logs */ + int LogMessageDomains; /**< Boolean: Should we log the domain(s) in which + * each log message occurs? */ + char *DebugLogFile; /**< Where to send verbose log messages. */ char *DataDirectory; /**< OR only: where to store long-term data. */ char *Nickname; /**< OR only: nickname of this onion router. */ @@ -2169,19 +2382,22 @@ typedef struct { routerset_t *EntryNodes;/**< Structure containing nicknames, digests, * country codes and IP address patterns of ORs to * consider as entry points. */ - int StrictExitNodes; /**< Boolean: When none of our ExitNodes are up, do we - * stop building circuits? */ - int StrictEntryNodes; /**< Boolean: When none of our EntryNodes are up, do we - * stop building circuits? */ + int StrictNodes; /**< Boolean: When none of our EntryNodes or ExitNodes + * are up, or we need to access a node in ExcludeNodes, + * do we just fail instead? */ routerset_t *ExcludeNodes;/**< Structure containing nicknames, digests, * country codes and IP address patterns of ORs - * not to use in circuits. */ + * not to use in circuits. But see StrictNodes + * above. */ routerset_t *ExcludeExitNodes;/**< Structure containing nicknames, digests, * country codes and IP address patterns of * ORs not to consider as exits. */ /** Union of ExcludeNodes and ExcludeExitNodes */ - struct routerset_t *_ExcludeExitNodesUnion; + routerset_t *_ExcludeExitNodesUnion; + + int DisableAllSwap; /**< Boolean: Attempt to call mlockall() on our + * process for all current and future memory. */ /** List of "entry", "middle", "exit", "introduction", "rendezvous". */ smartlist_t *AllowInvalidNodes; @@ -2197,7 +2413,7 @@ typedef struct { * connections. */ config_line_t *TransListenAddress; /** Addresses to bind for listening for transparent natd connections */ - config_line_t *NatdListenAddress; + config_line_t *NATDListenAddress; /** Addresses to bind for listening for SOCKS connections. */ config_line_t *DNSListenAddress; /** Addresses to bind for listening for OR connections. */ @@ -2221,7 +2437,7 @@ typedef struct { int SocksPort; /**< Port to listen on for SOCKS connections. */ /** Port to listen on for transparent pf/netfilter connections. */ int TransPort; - int NatdPort; /**< Port to listen on for transparent natd connections. */ + int NATDPort; /**< Port to listen on for transparent natd connections. */ int ControlPort; /**< Port to listen on for control connections. */ config_line_t *ControlSocket; /**< List of Unix Domain Sockets to listen on * for control connections. */ @@ -2237,8 +2453,6 @@ typedef struct { * for version 3 directories? */ int HSAuthoritativeDir; /**< Boolean: does this an authoritative directory * handle hidden service requests? */ - int HSAuthorityRecordStats; /**< Boolean: does this HS authoritative - * directory record statistics? */ int NamingAuthoritativeDir; /**< Boolean: is this an authoritative directory * that's willing to bind names? */ int VersioningAuthoritativeDir; /**< Boolean: is this an authoritative @@ -2267,8 +2481,6 @@ typedef struct { int AvoidDiskWrites; /**< Boolean: should we never cache things to disk? * Not used yet. */ int ClientOnly; /**< Boolean: should we never evolve into a server role? */ - /** Boolean: should we never publish a descriptor? Deprecated. */ - int NoPublish; /** To what authority types do we publish our descriptor? Choices are * "v1", "v2", "v3", "bridge", or "". */ smartlist_t *PublishServerDescriptor; @@ -2277,7 +2489,9 @@ typedef struct { /** Boolean: do we publish hidden service descriptors to the HS auths? */ int PublishHidServDescriptors; int FetchServerDescriptors; /**< Do we fetch server descriptors as normal? */ - int FetchHidServDescriptors; /** and hidden service descriptors? */ + int FetchHidServDescriptors; /**< and hidden service descriptors? */ + int FetchV2Networkstatus; /**< Do we fetch v2 networkstatus documents when + * we don't need to? */ int HidServDirectoryV2; /**< Do we participate in the HS DHT? */ int MinUptimeHidServDirectoryV2; /**< As directory authority, accept hidden @@ -2299,6 +2513,14 @@ typedef struct { int ConstrainedSockets; /**< Shrink xmit and recv socket buffers. */ uint64_t ConstrainedSockSize; /**< Size of constrained buffers. */ + /** Whether we should drop exit streams from Tors that we don't know are + * relays. One of "0" (never refuse), "1" (always refuse), or "auto" (do + * what the consensus says, defaulting to 'refuse' if the consensus says + * nothing). */ + char *RefuseUnknownExits; + /** Parsed version of RefuseUnknownExits. -1 for auto. */ + int RefuseUnknownExits_; + /** Application ports that require all nodes in circ to have sufficient * uptime. */ smartlist_t *LongLivedPorts; @@ -2328,10 +2550,18 @@ typedef struct { * connections alive? */ int SocksTimeout; /**< How long do we let a socks connection wait * unattached before we fail it? */ - int CircuitBuildTimeout; /**< Cull non-open circuits that were born - * at least this many seconds ago. */ + int LearnCircuitBuildTimeout; /**< If non-zero, we attempt to learn a value + * for CircuitBuildTimeout based on timeout + * history */ + int CircuitBuildTimeout; /**< Cull non-open circuits that were born at + * least this many seconds ago. Used until + * adaptive algorithm learns a new value. */ int CircuitIdleTimeout; /**< Cull open clean circuits that were born * at least this many seconds ago. */ + int CircuitStreamTimeout; /**< If non-zero, detach streams from circuits + * and try a new circuit if the stream has been + * waiting for this many seconds. If zero, use + * our default internal timeout schedule. */ int MaxOnionsPending; /**< How many circuit CREATE requests do we allow * to wait simultaneously before we start dropping * them? */ @@ -2349,24 +2579,36 @@ typedef struct { * willing to use for all relayed conns? */ uint64_t RelayBandwidthBurst; /**< How much bandwidth, at maximum, will we * use in a second for all relayed conns? */ - int NumCpus; /**< How many CPUs should we try to use? */ - int RunTesting; /**< If true, create testing circuits to measure how well the - * other ORs are running. */ + uint64_t PerConnBWRate; /**< Long-term bw on a single TLS conn, if set. */ + uint64_t PerConnBWBurst; /**< Allowed burst on a single TLS conn, if set. */ + int NumCPUs; /**< How many CPUs should we try to use? */ +//int RunTesting; /**< If true, create testing circuits to measure how well the +// * other ORs are running. */ config_line_t *RendConfigLines; /**< List of configuration lines * for rendezvous services. */ config_line_t *HidServAuth; /**< List of configuration lines for client-side * authorizations for hidden services */ char *ContactInfo; /**< Contact info to be published in the directory. */ - char *HttpProxy; /**< hostname[:port] to use as http proxy, if any. */ - uint32_t HttpProxyAddr; /**< Parsed IPv4 addr for http proxy, if any. */ - uint16_t HttpProxyPort; /**< Parsed port for http proxy, if any. */ - char *HttpProxyAuthenticator; /**< username:password string, if any. */ + char *HTTPProxy; /**< hostname[:port] to use as http proxy, if any. */ + tor_addr_t HTTPProxyAddr; /**< Parsed IPv4 addr for http proxy, if any. */ + uint16_t HTTPProxyPort; /**< Parsed port for http proxy, if any. */ + char *HTTPProxyAuthenticator; /**< username:password string, if any. */ + + char *HTTPSProxy; /**< hostname[:port] to use as https proxy, if any. */ + tor_addr_t HTTPSProxyAddr; /**< Parsed addr for https proxy, if any. */ + uint16_t HTTPSProxyPort; /**< Parsed port for https proxy, if any. */ + char *HTTPSProxyAuthenticator; /**< username:password string, if any. */ - char *HttpsProxy; /**< hostname[:port] to use as https proxy, if any. */ - uint32_t HttpsProxyAddr; /**< Parsed IPv4 addr for https proxy, if any. */ - uint16_t HttpsProxyPort; /**< Parsed port for https proxy, if any. */ - char *HttpsProxyAuthenticator; /**< username:password string, if any. */ + char *Socks4Proxy; /**< hostname:port to use as a SOCKS4 proxy, if any. */ + tor_addr_t Socks4ProxyAddr; /**< Derived from Socks4Proxy. */ + uint16_t Socks4ProxyPort; /**< Derived from Socks4Proxy. */ + + char *Socks5Proxy; /**< hostname:port to use as a SOCKS5 proxy, if any. */ + tor_addr_t Socks5ProxyAddr; /**< Derived from Sock5Proxy. */ + uint16_t Socks5ProxyPort; /**< Derived from Socks5Proxy. */ + char *Socks5ProxyUsername; /**< Username for SOCKS5 authentication, if any */ + char *Socks5ProxyPassword; /**< Password for SOCKS5 authentication, if any */ /** List of configuration lines for replacement directory authorities. * If you just want to replace one class of authority at a time, @@ -2429,8 +2671,13 @@ typedef struct { * or not (1)? */ int ShutdownWaitLength; /**< When we get a SIGINT and we're a server, how * long do we wait before exiting? */ - int SafeLogging; /**< Boolean: are we allowed to log sensitive strings - * such as addresses (0), or do we scrub them first (1)? */ + char *SafeLogging; /**< Contains "relay", "1", "0" (meaning no scrubbing). */ + + /* Derived from SafeLogging */ + enum { + SAFELOG_SCRUB_ALL, SAFELOG_SCRUB_RELAY, SAFELOG_SCRUB_NONE + } _SafeLogging; + int SafeSocks; /**< Boolean: should we outright refuse application * connections that use socks4 or socks5-with-local-dns? */ #define LOG_PROTOCOL_WARN (get_options()->ProtocolWarnings ? \ @@ -2441,6 +2688,8 @@ typedef struct { * log whether it was DNS-leaking or not? */ int HardwareAccel; /**< Boolean: Should we enable OpenSSL hardware * acceleration where available? */ + char *AccelName; /**< Optional hardware acceleration engine name. */ + char *AccelDir; /**< Optional hardware acceleration engine search dir. */ int UseEntryGuards; /**< Boolean: Do we try to enter from a smallish number * of fixed nodes? */ int NumEntryGuards; /**< How many entry guards do we try to establish? */ @@ -2451,6 +2700,9 @@ typedef struct { * means directly from the authorities) no matter our other config? */ int FetchDirInfoEarly; + /** Should we fetch our dir info at the start of the consensus period? */ + int FetchDirInfoExtraEarly; + char *VirtualAddrNetwork; /**< Address and mask to hand out for virtual * MAPADDRESS requests. */ int ServerDNSSearchDomains; /**< Boolean: If set, we don't force exit @@ -2499,11 +2751,42 @@ typedef struct { * exit allows it, we use it. */ int AllowSingleHopCircuits; + /** If true, we convert "www.google.com.foo.exit" addresses on the + * socks/trans/natd ports into "www.google.com" addresses that + * exit from the node "foo". Disabled by default since attacking + * websites and exit relays can use it to manipulate your path + * selection. */ + int AllowDotExit; + + /** If true, we will warn if a user gives us only an IP address + * instead of a hostname. */ + int WarnUnsafeSocks; + + /** If true, the user wants us to collect statistics on clients + * requesting network statuses from us as directory. */ + int DirReqStatistics; + + /** If true, the user wants us to collect statistics on port usage. */ + int ExitPortStatistics; + + /** If true, the user wants us to collect cell statistics. */ + int CellStatistics; + + /** If true, the user wants us to collect statistics as entry node. */ + int EntryStatistics; + + /** If true, include statistics file contents in extra-info documents. */ + int ExtraInfoStatistics; + /** If true, do not believe anybody who tells us that a domain resolves * to an internal address, or that an internal address has a PTR mapping. * Helps avoid some cross-site attacks. */ int ClientDNSRejectInternalAddresses; + /** If true, do not accept any requests to connect to internal addresses + * over randomly chosen exits. */ + int ClientRejectInternalAddresses; + /** The length of time that we think a consensus should be fresh. */ int V3AuthVotingInterval; /** The length of time we think it will take to distribute votes. */ @@ -2517,6 +2800,13 @@ typedef struct { * migration purposes? */ int V3AuthUseLegacyKey; + /** Location of bandwidth measurement file */ + char *V3BandwidthsFile; + + /** Authority only: key=value pairs that we add to our networkstatus + * consensus vote on the 'params' line. */ + char *ConsensusParams; + /** The length of time that we think an initial consensus should be fresh. * Only altered on testing networks. */ int TestingV3AuthInitialVotingInterval; @@ -2553,19 +2843,6 @@ typedef struct { * the bridge authority guess which countries have blocked access to us. */ int BridgeRecordUsageByCountry; -#ifdef ENABLE_GEOIP_STATS - /** If true, and Tor is built with GEOIP_STATS support, and we're a - * directory, record how many directory requests we get from each country. */ - int DirRecordUsageByCountry; - /** Round all GeoIP results to the next multiple of this value, to avoid - * leaking information. */ - int DirRecordUsageGranularity; - /** Time interval: purge geoip stats after this long. */ - int DirRecordUsageRetainIPs; - /** Time interval: Flush geoip data to disk this often. */ - int DirRecordUsageSaveInterval; -#endif - /** Optionally, a file with GeoIP data. */ char *GeoIPFile; @@ -2573,6 +2850,26 @@ typedef struct { * to make this false. */ int ReloadTorrcOnSIGHUP; + /* The main parameter for picking circuits within a connection. + * + * If this value is positive, when picking a cell to relay on a connection, + * we always relay from the circuit whose weighted cell count is lowest. + * Cells are weighted exponentially such that if one cell is sent + * 'CircuitPriorityHalflife' seconds before another, it counts for half as + * much. + * + * If this value is zero, we're disabling the cell-EWMA algorithm. + * + * If this value is negative, we're using the default approach + * according to either Tor or a parameter set in the consensus. + */ + double CircuitPriorityHalflife; + + /** Set to true if the TestingTorNetwork configuration option is set. + * This is used so that options_validate() has a chance to realize that + * the defaults have changed. */ + int _UsingTestNetworkDefaults; + } or_options_t; /** Persistent state for an onion router, as saved to disk. */ @@ -2591,6 +2888,9 @@ typedef struct { uint64_t AccountingBytesReadInInterval; uint64_t AccountingBytesWrittenInInterval; int AccountingSecondsActive; + int AccountingSecondsToReachSoftLimit; + time_t AccountingSoftLimitHitAt; + uint64_t AccountingBytesAtSoftLimit; uint64_t AccountingExpectedUsage; /** A list of Entry Guard-related configuration lines. */ @@ -2601,13 +2901,30 @@ typedef struct { * bandwidth usage. The "Interval" fields hold the granularity, in seconds, * of the entries of Values. The "Values" lists hold decimal string * representations of the number of bytes read or written in each - * interval. */ + * interval. The "Maxima" list holds decimal strings describing the highest + * rate achieved during the interval. + */ time_t BWHistoryReadEnds; int BWHistoryReadInterval; smartlist_t *BWHistoryReadValues; + smartlist_t *BWHistoryReadMaxima; time_t BWHistoryWriteEnds; int BWHistoryWriteInterval; smartlist_t *BWHistoryWriteValues; + smartlist_t *BWHistoryWriteMaxima; + time_t BWHistoryDirReadEnds; + int BWHistoryDirReadInterval; + smartlist_t *BWHistoryDirReadValues; + smartlist_t *BWHistoryDirReadMaxima; + time_t BWHistoryDirWriteEnds; + int BWHistoryDirWriteInterval; + smartlist_t *BWHistoryDirWriteValues; + smartlist_t *BWHistoryDirWriteMaxima; + + /** Build time histogram */ + config_line_t * BuildtimeHistogram; + unsigned int TotalBuildTimes; + unsigned int CircuitBuildAbandonedCount; /** What version of Tor wrote this state file? */ char *TorVersion; @@ -2669,209 +2986,143 @@ struct socks_request_t { /* all the function prototypes go here */ -/********************************* buffers.c ***************************/ - -buf_t *buf_new(void); -buf_t *buf_new_with_capacity(size_t size); -void buf_free(buf_t *buf); -void buf_clear(buf_t *buf); -void buf_shrink(buf_t *buf); -void buf_shrink_freelists(int free_all); -void buf_dump_freelist_sizes(int severity); - -size_t buf_datalen(const buf_t *buf); -size_t buf_allocation(const buf_t *buf); -size_t buf_slack(const buf_t *buf); -const char *_buf_peek_raw_buffer(const buf_t *buf); - -int read_to_buf(int s, size_t at_most, buf_t *buf, int *reached_eof, - int *socket_error); -int read_to_buf_tls(tor_tls_t *tls, size_t at_most, buf_t *buf); - -int flush_buf(int s, buf_t *buf, size_t sz, size_t *buf_flushlen); -int flush_buf_tls(tor_tls_t *tls, buf_t *buf, size_t sz, size_t *buf_flushlen); - -int write_to_buf(const char *string, size_t string_len, buf_t *buf); -int write_to_buf_zlib(buf_t *buf, tor_zlib_state_t *state, - const char *data, size_t data_len, int done); -int move_buf_to_buf(buf_t *buf_out, buf_t *buf_in, size_t *buf_flushlen); -int fetch_from_buf(char *string, size_t string_len, buf_t *buf); -int fetch_var_cell_from_buf(buf_t *buf, var_cell_t **out, int linkproto); -int fetch_from_buf_http(buf_t *buf, - char **headers_out, size_t max_headerlen, - char **body_out, size_t *body_used, size_t max_bodylen, - int force_complete); -int fetch_from_buf_socks(buf_t *buf, socks_request_t *req, - int log_sockstype, int safe_socks); -int fetch_from_buf_line(buf_t *buf, char *data_out, size_t *data_len); - -int peek_buf_has_control0_command(buf_t *buf); - -void assert_buf_ok(buf_t *buf); - -#ifdef BUFFERS_PRIVATE -int buf_find_string_offset(const buf_t *buf, const char *s, size_t n); -#endif - /********************************* circuitbuild.c **********************/ -char *circuit_list_path(origin_circuit_t *circ, int verbose); -char *circuit_list_path_for_controller(origin_circuit_t *circ); -void circuit_log_path(int severity, unsigned int domain, - origin_circuit_t *circ); -void circuit_rep_hist_note_result(origin_circuit_t *circ); -origin_circuit_t *origin_circuit_init(uint8_t purpose, int flags); -origin_circuit_t *circuit_establish_circuit(uint8_t purpose, - extend_info_t *exit, - int flags); -int circuit_handle_first_hop(origin_circuit_t *circ); -void circuit_n_conn_done(or_connection_t *or_conn, int status); -int inform_testing_reachability(void); -int circuit_send_next_onion_skin(origin_circuit_t *circ); -void circuit_note_clock_jumped(int seconds_elapsed); -int circuit_extend(cell_t *cell, circuit_t *circ); -int circuit_init_cpath_crypto(crypt_path_t *cpath, const char *key_data, - int reverse); -int circuit_finish_handshake(origin_circuit_t *circ, uint8_t cell_type, - const uint8_t *reply); -int circuit_truncated(origin_circuit_t *circ, crypt_path_t *layer); -int onionskin_answer(or_circuit_t *circ, uint8_t cell_type, - const char *payload, const char *keys); -int circuit_all_predicted_ports_handled(time_t now, int *need_uptime, - int *need_capacity); - -int circuit_append_new_exit(origin_circuit_t *circ, extend_info_t *info); -int circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *info); -void onion_append_to_cpath(crypt_path_t **head_ptr, crypt_path_t *new_hop); -extend_info_t *extend_info_alloc(const char *nickname, const char *digest, - crypto_pk_env_t *onion_key, - const tor_addr_t *addr, uint16_t port); -extend_info_t *extend_info_from_router(routerinfo_t *r); -extend_info_t *extend_info_dup(extend_info_t *info); -void extend_info_free(extend_info_t *info); -routerinfo_t *build_state_get_exit_router(cpath_build_state_t *state); -const char *build_state_get_exit_nickname(cpath_build_state_t *state); - -void entry_guards_compute_status(void); -int entry_guard_register_connect_status(const char *digest, int succeeded, - int mark_relay_status, time_t now); -void entry_nodes_should_be_added(void); -int entry_list_can_grow(or_options_t *options); -routerinfo_t *choose_random_entry(cpath_build_state_t *state); -int entry_guards_parse_state(or_state_t *state, int set, char **msg); -void entry_guards_update_state(or_state_t *state); -int getinfo_helper_entry_guards(control_connection_t *conn, - const char *question, char **answer); - -void clear_bridge_list(void); -int routerinfo_is_a_configured_bridge(routerinfo_t *ri); -void bridge_add_from_config(const tor_addr_t *addr, uint16_t port, - char *digest); -void retry_bridge_descriptor_fetch_directly(const char *digest); -void fetch_bridge_descriptors(time_t now); -void learned_bridge_descriptor(routerinfo_t *ri, int from_cache); -int any_bridge_descriptors_known(void); -int any_pending_bridge_descriptor_fetches(void); -int bridges_known_but_down(void); -void bridges_retry_all(void); - -void entry_guards_free_all(void); - -/********************************* circuitlist.c ***********************/ - -circuit_t * _circuit_get_global_list(void); -const char *circuit_state_to_string(int state); -const char *circuit_purpose_to_controller_string(uint8_t purpose); -void circuit_dump_by_conn(connection_t *conn, int severity); -void circuit_set_p_circid_orconn(or_circuit_t *circ, circid_t id, - or_connection_t *conn); -void circuit_set_n_circid_orconn(circuit_t *circ, circid_t id, - or_connection_t *conn); -void circuit_set_state(circuit_t *circ, uint8_t state); -void circuit_close_all_marked(void); -int32_t circuit_initial_package_window(void); -origin_circuit_t *origin_circuit_new(void); -or_circuit_t *or_circuit_new(circid_t p_circ_id, or_connection_t *p_conn); -circuit_t *circuit_get_by_circid_orconn(circid_t circ_id, - or_connection_t *conn); -int circuit_id_in_use_on_orconn(circid_t circ_id, or_connection_t *conn); -circuit_t *circuit_get_by_edge_conn(edge_connection_t *conn); -void circuit_unlink_all_from_or_conn(or_connection_t *conn, int reason); -origin_circuit_t *circuit_get_by_global_id(uint32_t id); -origin_circuit_t *circuit_get_by_rend_query_and_purpose(const char *rend_query, - uint8_t purpose); -origin_circuit_t *circuit_get_next_by_pk_and_purpose(origin_circuit_t *start, - const char *digest, uint8_t purpose); -or_circuit_t *circuit_get_rendezvous(const char *cookie); -or_circuit_t *circuit_get_intro_point(const char *digest); -origin_circuit_t *circuit_find_to_cannibalize(uint8_t purpose, - extend_info_t *info, int flags); -void circuit_mark_all_unused_circs(void); -void circuit_expire_all_dirty_circs(void); -void _circuit_mark_for_close(circuit_t *circ, int reason, - int line, const char *file); -int circuit_get_cpath_len(origin_circuit_t *circ); -crypt_path_t *circuit_get_cpath_hop(origin_circuit_t *circ, int hopnum); -void circuit_get_all_pending_on_or_conn(smartlist_t *out, - or_connection_t *or_conn); -int circuit_count_pending_on_or_conn(or_connection_t *or_conn); - -#define circuit_mark_for_close(c, reason) \ - _circuit_mark_for_close((c), (reason), __LINE__, _SHORT_FILE_) - -void assert_cpath_layer_ok(const crypt_path_t *cp); -void assert_circuit_ok(const circuit_t *c); -void circuit_free_all(void); - -/********************************* circuituse.c ************************/ - -void circuit_expire_building(time_t now); -void circuit_remove_handled_ports(smartlist_t *needed_ports); -int circuit_stream_is_being_handled(edge_connection_t *conn, uint16_t port, - int min); -int circuit_conforms_to_options(const origin_circuit_t *circ, - const or_options_t *options); -void circuit_build_needed_circs(time_t now); -void circuit_detach_stream(circuit_t *circ, edge_connection_t *conn); - -void circuit_expire_old_circuits_serverside(time_t now); - -void reset_bandwidth_test(void); -int circuit_enough_testing_circs(void); - -void circuit_has_opened(origin_circuit_t *circ); -void circuit_build_failed(origin_circuit_t *circ); - -/** Flag to set when a circuit should have only a single hop. */ -#define CIRCLAUNCH_ONEHOP_TUNNEL (1<<0) -/** Flag to set when a circuit needs to be built of high-uptime nodes */ -#define CIRCLAUNCH_NEED_UPTIME (1<<1) -/** Flag to set when a circuit needs to be built of high-capacity nodes */ -#define CIRCLAUNCH_NEED_CAPACITY (1<<2) -/** Flag to set when the last hop of a circuit doesn't need to be an - * exit node. */ -#define CIRCLAUNCH_IS_INTERNAL (1<<3) -origin_circuit_t *circuit_launch_by_extend_info(uint8_t purpose, - extend_info_t *info, - int flags); -origin_circuit_t *circuit_launch_by_router(uint8_t purpose, - routerinfo_t *exit, int flags); -void circuit_reset_failure_count(int timeout); -int connection_ap_handshake_attach_chosen_circuit(edge_connection_t *conn, - origin_circuit_t *circ, - crypt_path_t *cpath); -int connection_ap_handshake_attach_circuit(edge_connection_t *conn); - -/********************************* command.c ***************************/ - -void command_process_cell(cell_t *cell, or_connection_t *conn); -void command_process_var_cell(var_cell_t *cell, or_connection_t *conn); - -extern uint64_t stats_n_padding_cells_processed; -extern uint64_t stats_n_create_cells_processed; -extern uint64_t stats_n_created_cells_processed; -extern uint64_t stats_n_relay_cells_processed; -extern uint64_t stats_n_destroy_cells_processed; +/** How many hops does a general-purpose circuit have by default? */ +#define DEFAULT_ROUTE_LEN 3 + +/* Circuit Build Timeout "public" structures. */ + +/** Precision multiplier for the Bw weights */ +#define BW_WEIGHT_SCALE 10000 +#define BW_MIN_WEIGHT_SCALE 1 +#define BW_MAX_WEIGHT_SCALE INT32_MAX + +/** Total size of the circuit timeout history to accumulate. + * 1000 is approx 2.5 days worth of continual-use circuits. */ +#define CBT_NCIRCUITS_TO_OBSERVE 1000 + +/** Width of the histogram bins in milliseconds */ +#define CBT_BIN_WIDTH ((build_time_t)50) + +/** Number of modes to use in the weighted-avg computation of Xm */ +#define CBT_DEFAULT_NUM_XM_MODES 3 +#define CBT_MIN_NUM_XM_MODES 1 +#define CBT_MAX_NUM_XM_MODES 20 + +/** A build_time_t is milliseconds */ +typedef uint32_t build_time_t; + +/** + * CBT_BUILD_ABANDONED is our flag value to represent a force-closed + * circuit (Aka a 'right-censored' pareto value). + */ +#define CBT_BUILD_ABANDONED ((build_time_t)(INT32_MAX-1)) +#define CBT_BUILD_TIME_MAX ((build_time_t)(INT32_MAX)) + +/** Save state every 10 circuits */ +#define CBT_SAVE_STATE_EVERY 10 + +/* Circuit build times consensus parameters */ + +/** + * How long to wait before actually closing circuits that take too long to + * build in terms of CDF quantile. + */ +#define CBT_DEFAULT_CLOSE_QUANTILE 95 +#define CBT_MIN_CLOSE_QUANTILE CBT_MIN_QUANTILE_CUTOFF +#define CBT_MAX_CLOSE_QUANTILE CBT_MAX_QUANTILE_CUTOFF + +/** + * How many circuits count as recent when considering if the + * connection has gone gimpy or changed. + */ +#define CBT_DEFAULT_RECENT_CIRCUITS 20 +#define CBT_MIN_RECENT_CIRCUITS 3 +#define CBT_MAX_RECENT_CIRCUITS 1000 + +/** + * Maximum count of timeouts that finish the first hop in the past + * RECENT_CIRCUITS before calculating a new timeout. + * + * This tells us whether to abandon timeout history and set + * the timeout back to whatever circuit_build_times_get_initial_timeout() + * gives us. + */ +#define CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT (CBT_DEFAULT_RECENT_CIRCUITS*9/10) +#define CBT_MIN_MAX_RECENT_TIMEOUT_COUNT 3 +#define CBT_MAX_MAX_RECENT_TIMEOUT_COUNT 10000 + +/** Minimum circuits before estimating a timeout */ +#define CBT_DEFAULT_MIN_CIRCUITS_TO_OBSERVE 100 +#define CBT_MIN_MIN_CIRCUITS_TO_OBSERVE 1 +#define CBT_MAX_MIN_CIRCUITS_TO_OBSERVE 10000 + +/** Cutoff percentile on the CDF for our timeout estimation. */ +#define CBT_DEFAULT_QUANTILE_CUTOFF 80 +#define CBT_MIN_QUANTILE_CUTOFF 10 +#define CBT_MAX_QUANTILE_CUTOFF 99 +double circuit_build_times_quantile_cutoff(void); + +/** How often in seconds should we build a test circuit */ +#define CBT_DEFAULT_TEST_FREQUENCY 60 +#define CBT_MIN_TEST_FREQUENCY 1 +#define CBT_MAX_TEST_FREQUENCY INT32_MAX + +/** Lowest allowable value for CircuitBuildTimeout in milliseconds */ +#define CBT_DEFAULT_TIMEOUT_MIN_VALUE (1500) +#define CBT_MIN_TIMEOUT_MIN_VALUE 500 +#define CBT_MAX_TIMEOUT_MIN_VALUE INT32_MAX + +/** Initial circuit build timeout in milliseconds */ +#define CBT_DEFAULT_TIMEOUT_INITIAL_VALUE (60*1000) +#define CBT_MIN_TIMEOUT_INITIAL_VALUE CBT_MIN_TIMEOUT_MIN_VALUE +#define CBT_MAX_TIMEOUT_INITIAL_VALUE INT32_MAX +int32_t circuit_build_times_initial_timeout(void); + +#if CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT < CBT_MIN_MAX_RECENT_TIMEOUT_COUNT +#error "RECENT_CIRCUITS is set too low." +#endif + +/** Information about the state of our local network connection */ +typedef struct { + /** The timestamp we last completed a TLS handshake or received a cell */ + time_t network_last_live; + /** If the network is not live, how many timeouts has this caused? */ + int nonlive_timeouts; + /** Circular array of circuits that have made it to the first hop. Slot is + * 1 if circuit timed out, 0 if circuit succeeded */ + int8_t *timeouts_after_firsthop; + /** Number of elements allocated for the above array */ + int num_recent_circs; + /** Index into circular array. */ + int after_firsthop_idx; +} network_liveness_t; + +/** Structure for circuit build times history */ +typedef struct { + /** The circular array of recorded build times in milliseconds */ + build_time_t circuit_build_times[CBT_NCIRCUITS_TO_OBSERVE]; + /** Current index in the circuit_build_times circular array */ + int build_times_idx; + /** Total number of build times accumulated. Max CBT_NCIRCUITS_TO_OBSERVE */ + int total_build_times; + /** Information about the state of our local network connection */ + network_liveness_t liveness; + /** Last time we built a circuit. Used to decide to build new test circs */ + time_t last_circ_at; + /** "Minimum" value of our pareto distribution (actually mode) */ + build_time_t Xm; + /** alpha exponent for pareto dist. */ + double alpha; + /** Have we computed a timeout? */ + int have_computed_timeout; + /** The exact value for that timeout in milliseconds. Stored as a double + * to maintain precision from calculations to and from quantile value. */ + double timeout_ms; + /** How long we wait before actually closing the circuit. */ + double close_ms; +} circuit_build_times_t; /********************************* config.c ***************************/ @@ -2884,203 +3135,8 @@ typedef enum setopt_err_t { SETOPT_ERR_SETTING = -4, } setopt_err_t; -const char *get_dirportfrontpage(void); -or_options_t *get_options(void); -int set_options(or_options_t *new_val, char **msg); -void config_free_all(void); -const char *safe_str(const char *address); -const char *escaped_safe_str(const char *address); -const char *get_version(void); - -int config_get_lines(const char *string, config_line_t **result); -void config_free_lines(config_line_t *front); -setopt_err_t options_trial_assign(config_line_t *list, int use_defaults, - int clear_first, char **msg); -int resolve_my_address(int warn_severity, or_options_t *options, - uint32_t *addr, char **hostname_out); -int is_local_addr(const tor_addr_t *addr) ATTR_PURE; -void options_init(or_options_t *options); -int options_init_from_torrc(int argc, char **argv); -setopt_err_t options_init_from_string(const char *cf, - int command, const char *command_arg, char **msg); -int option_is_recognized(const char *key); -const char *option_get_canonical_name(const char *key); -config_line_t *option_get_assignment(or_options_t *options, - const char *key); -int options_save_current(void); -const char *get_torrc_fname(void); -char *options_get_datadir_fname2_suffix(or_options_t *options, - const char *sub1, const char *sub2, - const char *suffix); -#define get_datadir_fname2_suffix(sub1, sub2, suffix) \ - options_get_datadir_fname2_suffix(get_options(), (sub1), (sub2), (suffix)) -/** Return a newly allocated string containing datadir/sub1. See - * get_datadir_fname2_suffix. */ -#define get_datadir_fname(sub1) get_datadir_fname2_suffix((sub1), NULL, NULL) -/** Return a newly allocated string containing datadir/sub1/sub2. See - * get_datadir_fname2_suffix. */ -#define get_datadir_fname2(sub1,sub2) \ - get_datadir_fname2_suffix((sub1), (sub2), NULL) -/** Return a newly allocated string containing datadir/sub1suffix. See - * get_datadir_fname2_suffix. */ -#define get_datadir_fname_suffix(sub1, suffix) \ - get_datadir_fname2_suffix((sub1), NULL, (suffix)) - -or_state_t *get_or_state(void); -int or_state_save(time_t now); - -int options_need_geoip_info(or_options_t *options, const char **reason_out); -int getinfo_helper_config(control_connection_t *conn, - const char *question, char **answer); - -uint32_t get_effective_bwrate(or_options_t *options); -uint32_t get_effective_bwburst(or_options_t *options); - -#ifdef CONFIG_PRIVATE -/* Used only by config.c and test.c */ -or_options_t *options_new(void); -#endif - -/********************************* connection.c ***************************/ - -const char *conn_type_to_string(int type); -const char *conn_state_to_string(int type, int state); - -dir_connection_t *dir_connection_new(int socket_family); -or_connection_t *or_connection_new(int socket_family); -edge_connection_t *edge_connection_new(int type, int socket_family); -control_connection_t *control_connection_new(int socket_family); -connection_t *connection_new(int type, int socket_family); - -void connection_link_connections(connection_t *conn_a, connection_t *conn_b); -void connection_unregister_events(connection_t *conn); -void connection_free(connection_t *conn); -void connection_free_all(void); -void connection_about_to_close_connection(connection_t *conn); -void connection_close_immediate(connection_t *conn); -void _connection_mark_for_close(connection_t *conn,int line, const char *file); - -#define connection_mark_for_close(c) \ - _connection_mark_for_close((c), __LINE__, _SHORT_FILE_) - -void connection_expire_held_open(void); - -int connection_connect(connection_t *conn, const char *address, - const tor_addr_t *addr, - uint16_t port, int *socket_error); -int retry_all_listeners(smartlist_t *replaced_conns, - smartlist_t *new_conns); - -ssize_t connection_bucket_write_limit(connection_t *conn, time_t now); -int global_write_bucket_low(connection_t *conn, size_t attempt, int priority); -void connection_bucket_init(void); -void connection_bucket_refill(int seconds_elapsed, time_t now); - -int connection_handle_read(connection_t *conn); - -int connection_fetch_from_buf(char *string, size_t len, connection_t *conn); - -int connection_wants_to_flush(connection_t *conn); -int connection_outbuf_too_full(connection_t *conn); -int connection_handle_write(connection_t *conn, int force); -void _connection_write_to_buf_impl(const char *string, size_t len, - connection_t *conn, int zlib); -static void connection_write_to_buf(const char *string, size_t len, - connection_t *conn); -static void connection_write_to_buf_zlib(const char *string, size_t len, - dir_connection_t *conn, int done); -static INLINE void -connection_write_to_buf(const char *string, size_t len, connection_t *conn) -{ - _connection_write_to_buf_impl(string, len, conn, 0); -} -static INLINE void -connection_write_to_buf_zlib(const char *string, size_t len, - dir_connection_t *conn, int done) -{ - _connection_write_to_buf_impl(string, len, TO_CONN(conn), done ? -1 : 1); -} - -connection_t *connection_get_by_global_id(uint64_t id); - -connection_t *connection_get_by_type(int type); -connection_t *connection_get_by_type_purpose(int type, int purpose); -connection_t *connection_get_by_type_addr_port_purpose(int type, - const tor_addr_t *addr, - uint16_t port, int purpose); -connection_t *connection_get_by_type_state(int type, int state); -connection_t *connection_get_by_type_state_rendquery(int type, int state, - const char *rendquery, - int rendversion); - -#define connection_speaks_cells(conn) ((conn)->type == CONN_TYPE_OR) -int connection_is_listener(connection_t *conn); -int connection_state_is_open(connection_t *conn); -int connection_state_is_connecting(connection_t *conn); - -char *alloc_http_authenticator(const char *authenticator); - -void assert_connection_ok(connection_t *conn, time_t now); -int connection_or_nonopen_was_started_here(or_connection_t *conn); -void connection_dump_buffer_mem_stats(int severity); -void remove_file_if_very_old(const char *fname, time_t now); - /********************************* connection_edge.c *************************/ -#define connection_mark_unattached_ap(conn, endreason) \ - _connection_mark_unattached_ap((conn), (endreason), __LINE__, _SHORT_FILE_) - -void _connection_mark_unattached_ap(edge_connection_t *conn, int endreason, - int line, const char *file); -int connection_edge_reached_eof(edge_connection_t *conn); -int connection_edge_process_inbuf(edge_connection_t *conn, - int package_partial); -int connection_edge_destroy(circid_t circ_id, edge_connection_t *conn); -int connection_edge_end(edge_connection_t *conn, uint8_t reason); -int connection_edge_end_errno(edge_connection_t *conn); -int connection_edge_finished_flushing(edge_connection_t *conn); -int connection_edge_finished_connecting(edge_connection_t *conn); - -int connection_ap_handshake_send_begin(edge_connection_t *ap_conn); -int connection_ap_handshake_send_resolve(edge_connection_t *ap_conn); - -edge_connection_t *connection_ap_make_link(char *address, uint16_t port, - const char *digest, - int use_begindir, int want_onehop); -void connection_ap_handshake_socks_reply(edge_connection_t *conn, char *reply, - size_t replylen, - int endreason); -void connection_ap_handshake_socks_resolved(edge_connection_t *conn, - int answer_type, - size_t answer_len, - const uint8_t *answer, - int ttl, - time_t expires); - -int connection_exit_begin_conn(cell_t *cell, circuit_t *circ); -int connection_exit_begin_resolve(cell_t *cell, or_circuit_t *circ); -void connection_exit_connect(edge_connection_t *conn); -int connection_edge_is_rendezvous_stream(edge_connection_t *conn); -int connection_ap_can_use_exit(edge_connection_t *conn, routerinfo_t *exit); -void connection_ap_expire_beginning(void); -void connection_ap_attach_pending(void); -void connection_ap_fail_onehop(const char *failed_digest, - cpath_build_state_t *build_state); -void circuit_discard_optional_exit_enclaves(extend_info_t *info); -int connection_ap_detach_retriable(edge_connection_t *conn, - origin_circuit_t *circ, - int reason); -int connection_ap_process_transparent(edge_connection_t *conn); - -int address_is_invalid_destination(const char *address, int client); - -void addressmap_init(void); -void addressmap_clean(time_t now); -void addressmap_clear_configured(void); -void addressmap_clear_transient(void); -void addressmap_free_all(void); -int addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out); -int addressmap_have_mapping(const char *address, int update_timeout); /** Enumerates possible origins of a client-side address mapping. */ typedef enum { /** We're remapping this address because the controller told us to. */ @@ -3095,75 +3151,6 @@ typedef enum { * Tor server that told us what its value was. */ ADDRMAPSRC_DNS, } addressmap_entry_source_t; -void addressmap_register(const char *address, char *new_address, - time_t expires, addressmap_entry_source_t source); -int parse_virtual_addr_network(const char *val, int validate_only, - char **msg); -int client_dns_incr_failures(const char *address); -void client_dns_clear_failures(const char *address); -void client_dns_set_addressmap(const char *address, uint32_t val, - const char *exitname, int ttl); -const char *addressmap_register_virtual_address(int type, char *new_address); -void addressmap_get_mappings(smartlist_t *sl, time_t min_expires, - time_t max_expires, int want_expiry); -int connection_ap_handshake_rewrite_and_attach(edge_connection_t *conn, - origin_circuit_t *circ, - crypt_path_t *cpath); -int hostname_is_noconnect_address(const char *address); - -/** Possible return values for parse_extended_hostname. */ -typedef enum hostname_type_t { - NORMAL_HOSTNAME, ONION_HOSTNAME, EXIT_HOSTNAME, BAD_HOSTNAME -} hostname_type_t; -hostname_type_t parse_extended_hostname(char *address); - -#if defined(HAVE_NET_IF_H) && defined(HAVE_NET_PFVAR_H) -int get_pf_socket(void); -#endif - -/********************************* connection_or.c ***************************/ - -void connection_or_remove_from_identity_map(or_connection_t *conn); -void connection_or_clear_identity_map(void); -or_connection_t *connection_or_get_for_extend(const char *digest, - const tor_addr_t *target_addr, - const char **msg_out, - int *launch_out); -void connection_or_set_bad_connections(void); - -int connection_or_reached_eof(or_connection_t *conn); -int connection_or_process_inbuf(or_connection_t *conn); -int connection_or_flushed_some(or_connection_t *conn); -int connection_or_finished_flushing(or_connection_t *conn); -int connection_or_finished_connecting(or_connection_t *conn); - -void connection_or_connect_failed(or_connection_t *conn, - int reason, const char *msg); -or_connection_t *connection_or_connect(const tor_addr_t *addr, uint16_t port, - const char *id_digest); - -int connection_tls_start_handshake(or_connection_t *conn, int receiving); -int connection_tls_continue_handshake(or_connection_t *conn); - -void or_handshake_state_free(or_handshake_state_t *state); -int connection_or_set_state_open(or_connection_t *conn); -void connection_or_write_cell_to_buf(const cell_t *cell, - or_connection_t *conn); -void connection_or_write_var_cell_to_buf(const var_cell_t *cell, - or_connection_t *conn); -int connection_or_send_destroy(circid_t circ_id, or_connection_t *conn, - int reason); -int connection_or_send_netinfo(or_connection_t *conn); -int connection_or_send_cert(or_connection_t *conn); -int connection_or_send_link_auth(or_connection_t *conn); -int connection_or_compute_link_auth_hmac(or_connection_t *conn, - char *hmac_out); -int is_or_protocol_version_known(uint16_t version); - -void cell_pack(packed_cell_t *dest, const cell_t *src); -void var_cell_pack_header(const var_cell_t *cell, char *hdr_out); -var_cell_t *var_cell_new(uint16_t payload_len); -void var_cell_free(var_cell_t *cell); /********************************* control.c ***************************/ @@ -3201,8 +3188,14 @@ typedef enum or_conn_status_event_t { OR_CONN_EVENT_NEW = 4, } or_conn_status_event_t; -void control_update_global_event_mask(void); -void control_adjust_event_log_severity(void); +/** Used to indicate the type of a buildtime event */ +typedef enum buildtimeout_set_event_t { + BUILDTIMEOUT_SET_EVENT_COMPUTED = 0, + BUILDTIMEOUT_SET_EVENT_RESET = 1, + BUILDTIMEOUT_SET_EVENT_SUSPENDED = 2, + BUILDTIMEOUT_SET_EVENT_DISCARD = 3, + BUILDTIMEOUT_SET_EVENT_RESUME = 4 +} buildtimeout_set_event_t; /** Execute the statement <b>stmt</b>, which may log events concerning the * connection <b>conn</b>. To prevent infinite loops, disable log messages @@ -3212,7 +3205,9 @@ void control_adjust_event_log_severity(void); */ #define CONN_LOG_PROTECT(conn, stmt) \ STMT_BEGIN \ - int _log_conn_is_control = (conn && conn->type == CONN_TYPE_CONTROL); \ + int _log_conn_is_control; \ + tor_assert(conn); \ + _log_conn_is_control = (conn->type == CONN_TYPE_CONTROL); \ if (_log_conn_is_control) \ disable_control_logging(); \ STMT_BEGIN stmt; STMT_END; \ @@ -3220,58 +3215,6 @@ void control_adjust_event_log_severity(void); enable_control_logging(); \ STMT_END -/** Log information about the connection <b>conn</b>, protecting it as with - * CONN_LOG_PROTECT. Example: - * - * LOG_FN_CONN(conn, (LOG_DEBUG, "Socket %d wants to write", conn->s)); - **/ -#define LOG_FN_CONN(conn, args) \ - CONN_LOG_PROTECT(conn, log_fn args) - -int connection_control_finished_flushing(control_connection_t *conn); -int connection_control_reached_eof(control_connection_t *conn); -int connection_control_process_inbuf(control_connection_t *conn); - -#define EVENT_AUTHDIR_NEWDESCS 0x000D -#define EVENT_NS 0x000F -int control_event_is_interesting(int event); - -int control_event_circuit_status(origin_circuit_t *circ, - circuit_status_event_t e, int reason); -int control_event_stream_status(edge_connection_t *conn, - stream_status_event_t e, - int reason); -int control_event_or_conn_status(or_connection_t *conn, - or_conn_status_event_t e, int reason); -int control_event_bandwidth_used(uint32_t n_read, uint32_t n_written); -int control_event_stream_bandwidth(edge_connection_t *edge_conn); -int control_event_stream_bandwidth_used(void); -void control_event_logmsg(int severity, unsigned int domain, const char *msg); -int control_event_descriptors_changed(smartlist_t *routers); -int control_event_address_mapped(const char *from, const char *to, - time_t expires, const char *error); -int control_event_or_authdir_new_descriptor(const char *action, - const char *desc, - size_t desclen, - const char *msg); -int control_event_my_descriptor_changed(void); -int control_event_networkstatus_changed(smartlist_t *statuses); -int control_event_newconsensus(const networkstatus_t *consensus); -int control_event_networkstatus_changed_single(routerstatus_t *rs); -int control_event_general_status(int severity, const char *format, ...) - CHECK_PRINTF(2,3); -int control_event_client_status(int severity, const char *format, ...) - CHECK_PRINTF(2,3); -int control_event_server_status(int severity, const char *format, ...) - CHECK_PRINTF(2,3); -int control_event_guard(const char *nickname, const char *digest, - const char *status); - -int init_cookie_authentication(int enabled); -smartlist_t *decode_hashed_passwords(config_line_t *passwords); -void disable_control_logging(void); -void enable_control_logging(void); - /** Enum describing various stages of bootstrapping, for use with controller * bootstrap status events. The values range from 0 to 100. */ typedef enum { @@ -3292,427 +3235,133 @@ typedef enum { BOOTSTRAP_STATUS_DONE=100 } bootstrap_status_t; -void control_event_bootstrap(bootstrap_status_t status, int progress); -void control_event_bootstrap_problem(const char *warn, int reason); - -void control_event_clients_seen(const char *timestarted, - const char *countries); - -#ifdef CONTROL_PRIVATE -/* Used only by control.c and test.c */ -size_t write_escaped_data(const char *data, size_t len, char **out); -size_t read_escaped_data(const char *data, size_t len, char **out); -#endif - -/********************************* cpuworker.c *****************************/ - -void cpu_init(void); -void cpuworkers_rotate(void); -int connection_cpu_finished_flushing(connection_t *conn); -int connection_cpu_reached_eof(connection_t *conn); -int connection_cpu_process_inbuf(connection_t *conn); -int assign_onionskin_to_cpuworker(connection_t *cpuworker, - or_circuit_t *circ, - char *onionskin); - /********************************* directory.c ***************************/ -int directories_have_accepted_server_descriptor(void); -char *authority_type_to_string(authority_type_t auth); -void directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose, - authority_type_t type, const char *payload, - size_t payload_len, size_t extrainfo_len); -void directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, - const char *resource, - int pds_flags); -void directory_get_from_all_authorities(uint8_t dir_purpose, - uint8_t router_purpose, - const char *resource); -void directory_initiate_command_routerstatus(routerstatus_t *status, - uint8_t dir_purpose, - uint8_t router_purpose, - int anonymized_connection, - const char *resource, - const char *payload, - size_t payload_len, - time_t if_modified_since); -void directory_initiate_command_routerstatus_rend(routerstatus_t *status, - uint8_t dir_purpose, - uint8_t router_purpose, - int anonymized_connection, - const char *resource, - const char *payload, - size_t payload_len, - time_t if_modified_since, - const rend_data_t *rend_query); - -int parse_http_response(const char *headers, int *code, time_t *date, - compress_method_t *compression, char **response); - -int connection_dir_is_encrypted(dir_connection_t *conn); -int connection_dir_reached_eof(dir_connection_t *conn); -int connection_dir_process_inbuf(dir_connection_t *conn); -int connection_dir_finished_flushing(dir_connection_t *conn); -int connection_dir_finished_connecting(dir_connection_t *conn); -void connection_dir_request_failed(dir_connection_t *conn); -void directory_initiate_command(const char *address, const tor_addr_t *addr, - uint16_t or_port, uint16_t dir_port, - int supports_conditional_consensus, - int supports_begindir, const char *digest, - uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, - const char *resource, - const char *payload, size_t payload_len, - time_t if_modified_since); - -int dir_split_resource_into_fingerprints(const char *resource, - smartlist_t *fp_out, int *compresseed_out, - int decode_hex, int sort_uniq); /** A pair of digests created by dir_split_resource_info_fingerprint_pairs() */ typedef struct { char first[DIGEST_LEN]; char second[DIGEST_LEN]; } fp_pair_t; -int dir_split_resource_into_fingerprint_pairs(const char *res, - smartlist_t *pairs_out); -char *directory_dump_request_log(void); -void note_request(const char *key, size_t bytes); -int router_supports_extrainfo(const char *identity_digest, int is_authority); - -time_t download_status_increment_failure(download_status_t *dls, - int status_code, const char *item, - int server, time_t now); -/** Increment the failure count of the download_status_t <b>dls</b>, with - * the optional status code <b>sc</b>. */ -#define download_status_failed(dls, sc) \ - download_status_increment_failure((dls), (sc), NULL, \ - get_options()->DirPort, time(NULL)) - -void download_status_reset(download_status_t *dls); -static int download_status_is_ready(download_status_t *dls, time_t now, - int max_failures); -/** Return true iff, as of <b>now</b>, the resource tracked by <b>dls</b> is - * ready to get its download reattempted. */ -static INLINE int -download_status_is_ready(download_status_t *dls, time_t now, - int max_failures) -{ - return (dls->n_download_failures <= max_failures - && dls->next_attempt_at <= now); -} - -static void download_status_mark_impossible(download_status_t *dl); -/** Mark <b>dl</b> as never downloadable. */ -static INLINE void -download_status_mark_impossible(download_status_t *dl) -{ - dl->n_download_failures = IMPOSSIBLE_TO_DOWNLOAD; -} /********************************* dirserv.c ***************************/ -/** Maximum length of an exit policy summary. */ -#define MAX_EXITPOLICY_SUMMARY_LEN (1000) - -/** Maximum allowable length of a version line in a networkstatus. */ -#define MAX_V_LINE_LEN 128 -/** Length of "r Authority BadDirectory BadExit Exit Fast Guard HSDir Named - * Running Stable Unnamed V2Dir Valid\n". */ -#define MAX_FLAG_LINE_LEN 96 -/** Length of "w" line for weighting. Currently at most - * "w Bandwidth=<uint32t>\n" */ -#define MAX_WEIGHT_LINE_LEN (13+10) -/** Maximum length of an exit policy summary line. */ -#define MAX_POLICY_LINE_LEN (3+MAX_EXITPOLICY_SUMMARY_LEN) -/** Amount of space to allocate for each entry: r, s, and v lines. */ -#define RS_ENTRY_LEN \ - ( /* first line */ \ - MAX_NICKNAME_LEN+BASE64_DIGEST_LEN*2+ISO_TIME_LEN+INET_NTOA_BUF_LEN+ \ - 5*2 /* ports */ + 10 /* punctuation */ + \ - /* second line */ \ - MAX_FLAG_LINE_LEN + \ - /* weight line */ \ - MAX_WEIGHT_LINE_LEN + \ - /* p line. */ \ - MAX_POLICY_LINE_LEN + \ - /* v line. */ \ - MAX_V_LINE_LEN \ - ) -#define UNNAMED_ROUTER_NICKNAME "Unnamed" - -int connection_dirserv_flushed_some(dir_connection_t *conn); - -int dirserv_add_own_fingerprint(const char *nickname, crypto_pk_env_t *pk); -int dirserv_load_fingerprint_file(void); -void dirserv_free_fingerprint_list(void); -const char *dirserv_get_nickname_by_digest(const char *digest); -enum was_router_added_t dirserv_add_multiple_descriptors( - const char *desc, uint8_t purpose, - const char *source, - const char **msg); -enum was_router_added_t dirserv_add_descriptor(routerinfo_t *ri, - const char **msg, - const char *source); -int getinfo_helper_dirserv_unregistered(control_connection_t *conn, - const char *question, char **answer); -void dirserv_free_descriptors(void); -void dirserv_set_router_is_running(routerinfo_t *router, time_t now); -int list_server_status_v1(smartlist_t *routers, char **router_status_out, - int for_controller); -int dirserv_dump_directory_to_string(char **dir_out, - crypto_pk_env_t *private_key); - -int directory_fetches_from_authorities(or_options_t *options); -int directory_fetches_dir_info_early(or_options_t *options); -int directory_fetches_dir_info_later(or_options_t *options); -int directory_caches_v2_dir_info(or_options_t *options); -#define directory_caches_v1_dir_info(o) directory_caches_v2_dir_info(o) -int directory_caches_dir_info(or_options_t *options); -int directory_permits_begindir_requests(or_options_t *options); -int directory_permits_controller_requests(or_options_t *options); -int directory_too_idle_to_fetch_descriptors(or_options_t *options, time_t now); - -void directory_set_dirty(void); -cached_dir_t *dirserv_get_directory(void); -cached_dir_t *dirserv_get_runningrouters(void); -cached_dir_t *dirserv_get_consensus(void); -void dirserv_set_cached_directory(const char *directory, time_t when, - int is_running_routers); -void dirserv_set_cached_networkstatus_v2(const char *directory, - const char *identity, - time_t published); -void dirserv_set_cached_networkstatus_v3(const char *consensus, - time_t published); -void dirserv_clear_old_networkstatuses(time_t cutoff); -void dirserv_clear_old_v1_info(time_t now); -void dirserv_get_networkstatus_v2(smartlist_t *result, const char *key); -void dirserv_get_networkstatus_v2_fingerprints(smartlist_t *result, - const char *key); -int dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, - const char **msg, - int for_unencrypted_conn, - int is_extrainfo); -int dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, - const char **msg); -void dirserv_orconn_tls_done(const char *address, - uint16_t or_port, - const char *digest_rcvd, - int as_advertised); -void dirserv_test_reachability(time_t now, int try_all); -int authdir_wants_to_reject_router(routerinfo_t *ri, const char **msg, - int complain); -int dirserv_would_reject_router(routerstatus_t *rs); -int dirserv_remove_old_statuses(smartlist_t *fps, time_t cutoff); -int dirserv_have_any_serverdesc(smartlist_t *fps, int spool_src); -size_t dirserv_estimate_data_size(smartlist_t *fps, int is_serverdescs, - int compressed); -int routerstatus_format_entry(char *buf, size_t buf_len, - routerstatus_t *rs, const char *platform, - int first_line_only, int v2_format); -void dirserv_free_all(void); -void cached_dir_decref(cached_dir_t *d); -cached_dir_t *new_cached_dir(char *s, time_t published); -/********************************* dirvote.c ************************/ +/** An enum to describe what format we're generating a routerstatus line in. + */ +typedef enum { + /** For use in a v2 opinion */ + NS_V2, + /** For use in a consensus networkstatus document (ns flavor) */ + NS_V3_CONSENSUS, + /** For use in a vote networkstatus document */ + NS_V3_VOTE, + /** For passing to the controlport in response to a GETINFO request */ + NS_CONTROL_PORT, + /** For use in a consensus networkstatus document (microdesc flavor) */ + NS_V3_CONSENSUS_MICRODESC +} routerstatus_format_type_t; + +#ifdef DIRSERV_PRIVATE +typedef struct measured_bw_line_t { + char node_id[DIGEST_LEN]; + char node_hex[MAX_HEX_NICKNAME_LEN+1]; + long int bw; +} measured_bw_line_t; -/** Lowest allowable value for VoteSeconds. */ -#define MIN_VOTE_SECONDS 20 -/** Lowest allowable value for DistSeconds. */ -#define MIN_DIST_SECONDS 20 -/** Smallest allowable voting interval. */ -#define MIN_VOTE_INTERVAL 300 - -void dirvote_free_all(void); - -/* vote manipulation */ -char *networkstatus_compute_consensus(smartlist_t *votes, - int total_authorities, - crypto_pk_env_t *identity_key, - crypto_pk_env_t *signing_key, - const char *legacy_identity_key_digest, - crypto_pk_env_t *legacy_signing_key); -int networkstatus_add_detached_signatures(networkstatus_t *target, - ns_detached_signatures_t *sigs, - const char **msg_out); -char *networkstatus_get_detached_signatures(networkstatus_t *consensus); -void ns_detached_signatures_free(ns_detached_signatures_t *s); - -/* cert manipulation */ -authority_cert_t *authority_cert_dup(authority_cert_t *cert); +#endif + +/********************************* dirvote.c ************************/ /** Describes the schedule by which votes should be generated. */ typedef struct vote_timing_t { + /** Length in seconds between one consensus becoming valid and the next + * becoming valid. */ int vote_interval; + /** For how many intervals is a consensus valid? */ int n_intervals_valid; + /** Time in seconds allowed to propagate votes */ int vote_delay; + /** Time in seconds allowed to propagate signatures */ int dist_delay; } vote_timing_t; -/* vote scheduling */ -void dirvote_get_preferred_voting_intervals(vote_timing_t *timing_out); -time_t dirvote_get_start_of_next_interval(time_t now, int interval); -void dirvote_recalculate_timing(or_options_t *options, time_t now); -void dirvote_act(or_options_t *options, time_t now); - -/* invoked on timers and by outside triggers. */ -struct pending_vote_t * dirvote_add_vote(const char *vote_body, - const char **msg_out, - int *status_out); -int dirvote_add_signatures(const char *detached_signatures_body, - const char *source, - const char **msg_out); - -/* Item access */ -const char *dirvote_get_pending_consensus(void); -const char *dirvote_get_pending_detached_signatures(void); -#define DGV_BY_ID 1 -#define DGV_INCLUDE_PENDING 2 -#define DGV_INCLUDE_PREVIOUS 4 -const cached_dir_t *dirvote_get_vote(const char *fp, int flags); -void set_routerstatus_from_routerinfo(routerstatus_t *rs, - routerinfo_t *ri, time_t now, - int naming, int exits_can_be_guards, - int listbadexits, int listbaddirs); -void router_clear_status_flags(routerinfo_t *ri); -networkstatus_t * -dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key, - authority_cert_t *cert); - -#ifdef DIRVOTE_PRIVATE -char *format_networkstatus_vote(crypto_pk_env_t *private_key, - networkstatus_t *v3_ns); -char *dirvote_compute_params(smartlist_t *votes); -#endif - -/********************************* dns.c ***************************/ - -int dns_init(void); -int has_dns_init_failed(void); -void dns_free_all(void); -uint32_t dns_clip_ttl(uint32_t ttl); -int dns_reset(void); -void connection_dns_remove(edge_connection_t *conn); -void assert_connection_edge_not_dns_pending(edge_connection_t *conn); -void assert_all_pending_dns_resolves_ok(void); -void dns_cancel_pending_resolve(const char *question); -int dns_resolve(edge_connection_t *exitconn); -void dns_launch_correctness_checks(void); -int dns_seems_to_be_broken(void); -void dns_reset_correctness_checks(void); - -/********************************* dnsserv.c ************************/ - -void dnsserv_configure_listener(connection_t *conn); -void dnsserv_close_listener(connection_t *conn); -void dnsserv_resolved(edge_connection_t *conn, - int answer_type, - size_t answer_len, - const char *answer, - int ttl); -void dnsserv_reject_request(edge_connection_t *conn); -int dnsserv_launch_request(const char *name, int is_reverse); /********************************* geoip.c **************************/ -#ifdef GEOIP_PRIVATE -int geoip_parse_entry(const char *line); -#endif -int should_record_bridge_info(or_options_t *options); -int geoip_load_file(const char *filename, or_options_t *options); -int geoip_get_country_by_ip(uint32_t ipaddr); -int geoip_get_n_countries(void); -const char *geoip_get_country_name(country_t num); -int geoip_is_loaded(void); -country_t geoip_get_country(const char *countrycode); +/** Round all GeoIP results to the next multiple of this value, to avoid + * leaking information. */ +#define DIR_RECORD_USAGE_GRANULARITY 8 +/** Time interval: Flush geoip data to disk this often. */ +#define DIR_ENTRY_RECORD_USAGE_RETAIN_IPS (24*60*60) +/** How long do we have to have observed per-country request history before + * we are willing to talk about it? */ +#define DIR_RECORD_USAGE_MIN_OBSERVATION_TIME (12*60*60) + /** Indicates an action that we might be noting geoip statistics on. * Note that if we're noticing CONNECT, we're a bridge, and if we're noticing * the others, we're not. */ typedef enum { - /** We've noticed a connection as a bridge relay. */ + /** We've noticed a connection as a bridge relay or entry guard. */ GEOIP_CLIENT_CONNECT = 0, /** We've served a networkstatus consensus as a directory server. */ GEOIP_CLIENT_NETWORKSTATUS = 1, /** We've served a v2 networkstatus consensus as a directory server. */ GEOIP_CLIENT_NETWORKSTATUS_V2 = 2, } geoip_client_action_t; -void geoip_note_client_seen(geoip_client_action_t action, - uint32_t addr, time_t now); -void geoip_remove_old_clients(time_t cutoff); -time_t geoip_get_history_start(void); -char *geoip_get_client_history(time_t now, geoip_client_action_t action); -char *geoip_get_request_history(time_t now, geoip_client_action_t action); -int getinfo_helper_geoip(control_connection_t *control_conn, - const char *question, char **answer); -void geoip_free_all(void); -void dump_geoip_stats(void); - -/********************************* hibernate.c **********************/ - -int accounting_parse_options(or_options_t *options, int validate_only); -int accounting_is_enabled(or_options_t *options); -void configure_accounting(time_t now); -void accounting_run_housekeeping(time_t now); -void accounting_add_bytes(size_t n_read, size_t n_written, int seconds); -int accounting_record_bandwidth_usage(time_t now, or_state_t *state); -void hibernate_begin_shutdown(void); -int we_are_hibernating(void); -void consider_hibernation(time_t now); -int getinfo_helper_accounting(control_connection_t *conn, - const char *question, char **answer); -void accounting_set_bandwidth_usage_from_state(or_state_t *state); - -/********************************* main.c ***************************/ - -extern int has_completed_circuit; - -int connection_add(connection_t *conn); -int connection_remove(connection_t *conn); -int connection_in_array(connection_t *conn); -void add_connection_to_closeable_list(connection_t *conn); -int connection_is_on_closeable_list(connection_t *conn); - -smartlist_t *get_connection_array(void); - -void connection_watch_events(connection_t *conn, short events); -int connection_is_reading(connection_t *conn); -void connection_stop_reading(connection_t *conn); -void connection_start_reading(connection_t *conn); - -int connection_is_writing(connection_t *conn); -void connection_stop_writing(connection_t *conn); -void connection_start_writing(connection_t *conn); - -void connection_stop_reading_from_linked_conn(connection_t *conn); - -void directory_all_unreachable(time_t now); -void directory_info_has_arrived(time_t now, int from_cache); - -void ip_address_changed(int at_interface); -void dns_servers_relaunch_checks(void); - -void control_signal_act(int the_signal); -void handle_signals(int is_parent); - -int try_locking(or_options_t *options, int err_if_locked); -int have_lockfile(void); -void release_lockfile(void); - -void tor_cleanup(void); -void tor_free_all(int postfork); - -int tor_main(int argc, char *argv[]); - -#ifdef MAIN_PRIVATE -int do_main_loop(void); -int do_list_fingerprint(void); -void do_hash_password(void); -int tor_init(int argc, char **argv); -#endif +/** Indicates either a positive reply or a reason for rejectng a network + * status request that will be included in geoip statistics. */ +typedef enum { + /** Request is answered successfully. */ + GEOIP_SUCCESS = 0, + /** V3 network status is not signed by a sufficient number of requested + * authorities. */ + GEOIP_REJECT_NOT_ENOUGH_SIGS = 1, + /** Requested network status object is unavailable. */ + GEOIP_REJECT_UNAVAILABLE = 2, + /** Requested network status not found. */ + GEOIP_REJECT_NOT_FOUND = 3, + /** Network status has not been modified since If-Modified-Since time. */ + GEOIP_REJECT_NOT_MODIFIED = 4, + /** Directory is busy. */ + GEOIP_REJECT_BUSY = 5, +} geoip_ns_response_t; +#define GEOIP_NS_RESPONSE_NUM 6 + +/** Directory requests that we are measuring can be either direct or + * tunneled. */ +typedef enum { + DIRREQ_DIRECT = 0, + DIRREQ_TUNNELED = 1, +} dirreq_type_t; -/********************************* networkstatus.c *********************/ +/** Possible states for either direct or tunneled directory requests that + * are relevant for determining network status download times. */ +typedef enum { + /** Found that the client requests a network status; applies to both + * direct and tunneled requests; initial state of a request that we are + * measuring. */ + DIRREQ_IS_FOR_NETWORK_STATUS = 0, + /** Finished writing a network status to the directory connection; + * applies to both direct and tunneled requests; completes a direct + * request. */ + DIRREQ_FLUSHING_DIR_CONN_FINISHED = 1, + /** END cell sent to circuit that initiated a tunneled request. */ + DIRREQ_END_CELL_SENT = 2, + /** Flushed last cell from queue of the circuit that initiated a + * tunneled request to the outbuf of the OR connection. */ + DIRREQ_CIRC_QUEUE_FLUSHED = 3, + /** Flushed last byte from buffer of the OR connection belonging to the + * circuit that initiated a tunneled request; completes a tunneled + * request. */ + DIRREQ_OR_CONN_BUFFER_FLUSHED = 4 +} dirreq_state_t; + +#define WRITE_STATS_INTERVAL (24*60*60) + +/********************************* microdesc.c *************************/ + +typedef struct microdesc_cache_t microdesc_cache_t; -/** How old do we allow a v2 network-status to get before removing it - * completely? */ -#define MAX_NETWORKSTATUS_AGE (10*24*60*60) +/********************************* networkstatus.c *********************/ /** Location where we found a v2 networkstatus. */ typedef enum { @@ -3733,127 +3382,8 @@ typedef enum version_status_t { VS_UNKNOWN, /**< We have no idea. */ } version_status_t; -void networkstatus_reset_warnings(void); -void networkstatus_reset_download_failures(void); -int router_reload_v2_networkstatus(void); -int router_reload_consensus_networkstatus(void); -void routerstatus_free(routerstatus_t *rs); -void networkstatus_v2_free(networkstatus_v2_t *ns); -void networkstatus_vote_free(networkstatus_t *ns); -networkstatus_voter_info_t *networkstatus_get_voter_by_id( - networkstatus_t *vote, - const char *identity); -int networkstatus_check_consensus_signature(networkstatus_t *consensus, - int warn); -int networkstatus_check_voter_signature(networkstatus_t *consensus, - networkstatus_voter_info_t *voter, - authority_cert_t *cert); -char *networkstatus_get_cache_filename(const char *identity_digest); -int router_set_networkstatus_v2(const char *s, time_t arrived_at, - v2_networkstatus_source_t source, - smartlist_t *requested_fingerprints); -void networkstatus_v2_list_clean(time_t now); -routerstatus_t *networkstatus_v2_find_entry(networkstatus_v2_t *ns, - const char *digest); -routerstatus_t *networkstatus_vote_find_entry(networkstatus_t *ns, - const char *digest); -int networkstatus_vote_find_entry_idx(networkstatus_t *ns, - const char *digest, int *found_out); -const smartlist_t *networkstatus_get_v2_list(void); -download_status_t *router_get_dl_status_by_descriptor_digest(const char *d); -routerstatus_t *router_get_consensus_status_by_id(const char *digest); -routerstatus_t *router_get_consensus_status_by_descriptor_digest( - const char *digest); -routerstatus_t *router_get_consensus_status_by_nickname(const char *nickname, - int warn_if_unnamed); -const char *networkstatus_get_router_digest_by_nickname(const char *nickname); -int networkstatus_nickname_is_unnamed(const char *nickname); -void networkstatus_consensus_download_failed(int status_code); -void update_consensus_networkstatus_fetch_time(time_t now); -int should_delay_dir_fetches(or_options_t *options); -void update_networkstatus_downloads(time_t now); -void update_certificate_downloads(time_t now); -int consensus_is_waiting_for_certs(void); -networkstatus_v2_t *networkstatus_v2_get_by_digest(const char *digest); -networkstatus_t *networkstatus_get_latest_consensus(void); -networkstatus_t *networkstatus_get_live_consensus(time_t now); -networkstatus_t *networkstatus_get_reasonably_live_consensus(time_t now); -#define NSSET_FROM_CACHE 1 -#define NSSET_WAS_WAITING_FOR_CERTS 2 -#define NSSET_DONT_DOWNLOAD_CERTS 4 -#define NSSET_ACCEPT_OBSOLETE 8 -int networkstatus_set_current_consensus(const char *consensus, unsigned flags); -void networkstatus_note_certs_arrived(void); -void routers_update_all_from_networkstatus(time_t now, int dir_version); -void routerstatus_list_update_from_consensus_networkstatus(time_t now); -void routers_update_status_from_consensus_networkstatus(smartlist_t *routers, - int reset_failures); -void signed_descs_update_status_from_consensus_networkstatus( - smartlist_t *descs); - -char *networkstatus_getinfo_helper_single(routerstatus_t *rs); -char *networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now); -void networkstatus_dump_bridge_status_to_file(time_t now); -int32_t networkstatus_get_param(networkstatus_t *ns, const char *param_name, - int32_t default_val); -int getinfo_helper_networkstatus(control_connection_t *conn, - const char *question, char **answer); -void networkstatus_free_all(void); - -/********************************* ntmain.c ***************************/ -#ifdef MS_WINDOWS -#define NT_SERVICE -#endif - -#ifdef NT_SERVICE -int nt_service_parse_options(int argc, char **argv, int *should_exit); -int nt_service_is_stopping(void); -void nt_service_set_state(DWORD state); -#else -#define nt_service_is_stopping() (0) -#endif - -/********************************* onion.c ***************************/ - -int onion_pending_add(or_circuit_t *circ, char *onionskin); -or_circuit_t *onion_next_task(char **onionskin_out); -void onion_pending_remove(or_circuit_t *circ); - -int onion_skin_create(crypto_pk_env_t *router_key, - crypto_dh_env_t **handshake_state_out, - char *onion_skin_out); - -int onion_skin_server_handshake(const char *onion_skin, - crypto_pk_env_t *private_key, - crypto_pk_env_t *prev_private_key, - char *handshake_reply_out, - char *key_out, - size_t key_out_len); - -int onion_skin_client_handshake(crypto_dh_env_t *handshake_state, - const char *handshake_reply, - char *key_out, - size_t key_out_len); - -int fast_server_handshake(const uint8_t *key_in, - uint8_t *handshake_reply_out, - uint8_t *key_out, - size_t key_out_len); - -int fast_client_handshake(const uint8_t *handshake_state, - const uint8_t *handshake_reply_out, - uint8_t *key_out, - size_t key_out_len); - -void clear_pending_onions(void); - /********************************* policies.c ************************/ -/* (length of "accept 255.255.255.255/255.255.255.255:65535-65535\n" plus a - * NUL.) - */ -#define POLICY_BUF_LEN 52 - /** Outcome of applying an address policy to an address. */ typedef enum { /** The address was accepted */ @@ -3868,146 +3398,8 @@ typedef enum { ADDR_POLICY_PROBABLY_REJECTED=2 } addr_policy_result_t; -int firewall_is_fascist_or(void); -int fascist_firewall_allows_address_or(const tor_addr_t *addr, uint16_t port); -int fascist_firewall_allows_or(routerinfo_t *ri); -int fascist_firewall_allows_address_dir(const tor_addr_t *addr, uint16_t port); -int dir_policy_permits_address(const tor_addr_t *addr); -int socks_policy_permits_address(const tor_addr_t *addr); -int authdir_policy_permits_address(uint32_t addr, uint16_t port); -int authdir_policy_valid_address(uint32_t addr, uint16_t port); -int authdir_policy_baddir_address(uint32_t addr, uint16_t port); -int authdir_policy_badexit_address(uint32_t addr, uint16_t port); - -int validate_addr_policies(or_options_t *options, char **msg); -void policy_expand_private(smartlist_t **policy); -int policies_parse_from_options(or_options_t *options); - -addr_policy_t *addr_policy_get_canonical_entry(addr_policy_t *ent); -int cmp_addr_policies(smartlist_t *a, smartlist_t *b); -addr_policy_result_t compare_tor_addr_to_addr_policy(const tor_addr_t *addr, - uint16_t port, const smartlist_t *policy); -addr_policy_result_t compare_addr_to_addr_policy(uint32_t addr, - uint16_t port, const smartlist_t *policy); -int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, - int rejectprivate, const char *local_address); -void policies_set_router_exitpolicy_to_reject_all(routerinfo_t *exitrouter); -int exit_policy_is_general_exit(smartlist_t *policy); -int policy_is_reject_star(const smartlist_t *policy); -int getinfo_helper_policies(control_connection_t *conn, - const char *question, char **answer); -int policy_write_item(char *buf, size_t buflen, addr_policy_t *item, - int format_for_desc); - -void addr_policy_list_free(smartlist_t *p); -void addr_policy_free(addr_policy_t *p); -void policies_free_all(void); - -char *policy_summarize(smartlist_t *policy); - -/********************************* reasons.c ***************************/ - -const char *stream_end_reason_to_control_string(int reason); -const char *stream_end_reason_to_string(int reason); -socks5_reply_status_t stream_end_reason_to_socks5_response(int reason); -uint8_t errno_to_stream_end_reason(int e); - -const char *orconn_end_reason_to_control_string(int r); -int tls_error_to_orconn_end_reason(int e); -int errno_to_orconn_end_reason(int e); - -const char *circuit_end_reason_to_control_string(int reason); - -/********************************* relay.c ***************************/ - -extern uint64_t stats_n_relay_cells_relayed; -extern uint64_t stats_n_relay_cells_delivered; - -int circuit_receive_relay_cell(cell_t *cell, circuit_t *circ, - cell_direction_t cell_direction); - -void relay_header_pack(uint8_t *dest, const relay_header_t *src); -void relay_header_unpack(relay_header_t *dest, const uint8_t *src); -int relay_send_command_from_edge(streamid_t stream_id, circuit_t *circ, - uint8_t relay_command, const char *payload, - size_t payload_len, crypt_path_t *cpath_layer); -int connection_edge_send_command(edge_connection_t *fromconn, - uint8_t relay_command, const char *payload, - size_t payload_len); -int connection_edge_package_raw_inbuf(edge_connection_t *conn, - int package_partial); -void connection_edge_consider_sending_sendme(edge_connection_t *conn); - -extern uint64_t stats_n_data_cells_packaged; -extern uint64_t stats_n_data_bytes_packaged; -extern uint64_t stats_n_data_cells_received; -extern uint64_t stats_n_data_bytes_received; - -void init_cell_pool(void); -void free_cell_pool(void); -void clean_cell_pool(void); -void dump_cell_pool_usage(int severity); - -void cell_queue_clear(cell_queue_t *queue); -void cell_queue_append(cell_queue_t *queue, packed_cell_t *cell); -void cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell); - -void append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, - cell_t *cell, cell_direction_t direction); -void connection_or_unlink_all_active_circs(or_connection_t *conn); -int connection_or_flush_from_first_active_circuit(or_connection_t *conn, - int max, time_t now); -void assert_active_circuits_ok(or_connection_t *orconn); -void make_circuit_inactive_on_conn(circuit_t *circ, or_connection_t *conn); -void make_circuit_active_on_conn(circuit_t *circ, or_connection_t *conn); - -int append_address_to_payload(uint8_t *payload_out, const tor_addr_t *addr); -const uint8_t *decode_address_from_payload(tor_addr_t *addr_out, - const uint8_t *payload, - int payload_len); - /********************************* rephist.c ***************************/ -void rep_hist_init(void); -void rep_hist_note_connect_failed(const char* nickname, time_t when); -void rep_hist_note_connect_succeeded(const char* nickname, time_t when); -void rep_hist_note_disconnect(const char* nickname, time_t when); -void rep_hist_note_connection_died(const char* nickname, time_t when); -void rep_hist_note_extend_succeeded(const char *from_name, - const char *to_name); -void rep_hist_note_extend_failed(const char *from_name, const char *to_name); -void rep_hist_dump_stats(time_t now, int severity); -void rep_hist_note_bytes_read(size_t num_bytes, time_t when); -void rep_hist_note_bytes_written(size_t num_bytes, time_t when); -int rep_hist_bandwidth_assess(void); -char *rep_hist_get_bandwidth_lines(int for_extrainfo); -void rep_hist_update_state(or_state_t *state); -int rep_hist_load_state(or_state_t *state, char **err); -void rep_history_clean(time_t before); - -void rep_hist_note_router_reachable(const char *id, time_t when); -void rep_hist_note_router_unreachable(const char *id, time_t when); -int rep_hist_record_mtbf_data(time_t now, int missing_means_down); -int rep_hist_load_mtbf_data(time_t now); - -time_t rep_hist_downrate_old_runs(time_t now); -double rep_hist_get_stability(const char *id, time_t when); -double rep_hist_get_weighted_fractional_uptime(const char *id, time_t when); -long rep_hist_get_weighted_time_known(const char *id, time_t when); -int rep_hist_have_measured_enough_stability(void); -const char *rep_hist_get_router_stability_doc(time_t now); - -void rep_hist_note_used_port(time_t now, uint16_t port); -smartlist_t *rep_hist_get_predicted_ports(time_t now); -void rep_hist_note_used_resolve(time_t now); -void rep_hist_note_used_internal(time_t now, int need_uptime, - int need_capacity); -int rep_hist_get_predicted_internal(time_t now, int *need_uptime, - int *need_capacity); - -int any_predicted_circuits(time_t now); -int rep_hist_circbuilding_dormant(time_t now); - /** Possible public/private key operations in Tor: used to keep track of where * we're spending our time. */ typedef enum { @@ -4017,49 +3409,6 @@ typedef enum { TLS_HANDSHAKE_C, TLS_HANDSHAKE_S, REND_CLIENT, REND_MID, REND_SERVER, } pk_op_t; -void note_crypto_pk_op(pk_op_t operation); -void dump_pk_ops(int severity); - -void rep_hist_free_all(void); - -/* for hidden service usage statistics */ -void hs_usage_note_publish_total(const char *service_id, time_t now); -void hs_usage_note_publish_novel(const char *service_id, time_t now); -void hs_usage_note_fetch_total(const char *service_id, time_t now); -void hs_usage_note_fetch_successful(const char *service_id, time_t now); -void hs_usage_write_statistics_to_file(time_t now); -void hs_usage_free_all(void); - -/********************************* rendclient.c ***************************/ - -void rend_client_introcirc_has_opened(origin_circuit_t *circ); -void rend_client_rendcirc_has_opened(origin_circuit_t *circ); -int rend_client_introduction_acked(origin_circuit_t *circ, - const uint8_t *request, - size_t request_len); -void rend_client_refetch_renddesc(const char *query); -void rend_client_refetch_v2_renddesc(const rend_data_t *rend_query); -void rend_client_cancel_descriptor_fetches(void); -int rend_client_remove_intro_point(extend_info_t *failed_intro, - const rend_data_t *rend_query); -int rend_client_rendezvous_acked(origin_circuit_t *circ, - const uint8_t *request, - size_t request_len); -int rend_client_receive_rendezvous(origin_circuit_t *circ, - const uint8_t *request, - size_t request_len); -void rend_client_desc_trynow(const char *query, int rend_version); - -extend_info_t *rend_client_get_random_intro(const rend_data_t *rend_query); - -int rend_client_send_introduction(origin_circuit_t *introcirc, - origin_circuit_t *rendcirc); -int rend_parse_service_authorization(or_options_t *options, - int validate_only); -rend_service_authorization_t *rend_client_lookup_service_authorization( - const char *onion_address); -void rend_service_authorization_free_all(void); -rend_data_t *rend_data_dup(const rend_data_t *request); /********************************* rendcommon.c ***************************/ @@ -4102,31 +3451,6 @@ typedef struct rend_service_descriptor_t { smartlist_t *successful_uploads; } rend_service_descriptor_t; -/** Free all storage associated with <b>data</b> */ -static INLINE void -rend_data_free(rend_data_t *data) -{ - tor_free(data); -} - -int rend_cmp_service_ids(const char *one, const char *two); - -void rend_process_relay_cell(circuit_t *circ, const crypt_path_t *layer_hint, - int command, size_t length, - const uint8_t *payload); - -void rend_service_descriptor_free(rend_service_descriptor_t *desc); -int rend_encode_service_descriptor(rend_service_descriptor_t *desc, - crypto_pk_env_t *key, - char **str_out, - size_t *len_out); -rend_service_descriptor_t *rend_parse_service_descriptor(const char *str, - size_t len); -int rend_get_service_id(crypto_pk_env_t *pk, char *out); -void rend_encoded_v2_service_descriptor_free( - rend_encoded_v2_service_descriptor_t *desc); -void rend_intro_point_free(rend_intro_point_t *intro); - /** A cached rendezvous descriptor. */ typedef struct rend_cache_entry_t { size_t len; /**< Length of <b>desc</b> */ @@ -4135,151 +3459,6 @@ typedef struct rend_cache_entry_t { rend_service_descriptor_t *parsed; /**< Parsed value of 'desc' */ } rend_cache_entry_t; -void rend_cache_init(void); -void rend_cache_clean(void); -void rend_cache_clean_v2_descs_as_dir(void); -void rend_cache_purge(void); -void rend_cache_free_all(void); -int rend_valid_service_id(const char *query); -int rend_cache_lookup_desc(const char *query, int version, const char **desc, - size_t *desc_len); -int rend_cache_lookup_entry(const char *query, int version, - rend_cache_entry_t **entry_out); -int rend_cache_lookup_v2_desc_as_dir(const char *query, const char **desc); -int rend_cache_store(const char *desc, size_t desc_len, int published); -int rend_cache_store_v2_desc_as_client(const char *desc, - const rend_data_t *rend_query); -int rend_cache_store_v2_desc_as_dir(const char *desc); -int rend_cache_size(void); -int rend_encode_v2_descriptors(smartlist_t *descs_out, - rend_service_descriptor_t *desc, time_t now, - uint8_t period, rend_auth_type_t auth_type, - crypto_pk_env_t *client_key, - smartlist_t *client_cookies); -int rend_compute_v2_desc_id(char *desc_id_out, const char *service_id, - const char *descriptor_cookie, - time_t now, uint8_t replica); -int rend_id_is_in_interval(const char *a, const char *b, const char *c); -void rend_get_descriptor_id_bytes(char *descriptor_id_out, - const char *service_id, - const char *secret_id_part); - -/********************************* rendservice.c ***************************/ - -int num_rend_services(void); -int rend_config_services(or_options_t *options, int validate_only); -int rend_service_load_keys(void); -void rend_services_init(void); -void rend_services_introduce(void); -void rend_consider_services_upload(time_t now); -void rend_hsdir_routers_changed(void); -void rend_consider_descriptor_republication(void); - -void rend_service_intro_has_opened(origin_circuit_t *circuit); -int rend_service_intro_established(origin_circuit_t *circuit, - const uint8_t *request, - size_t request_len); -void rend_service_rendezvous_has_opened(origin_circuit_t *circuit); -int rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, - size_t request_len); -void rend_service_relaunch_rendezvous(origin_circuit_t *oldcirc); -int rend_service_set_connection_addr_port(edge_connection_t *conn, - origin_circuit_t *circ); -void rend_service_dump_stats(int severity); -void rend_service_free_all(void); - -/********************************* rendmid.c *******************************/ -int rend_mid_establish_intro(or_circuit_t *circ, const uint8_t *request, - size_t request_len); -int rend_mid_introduce(or_circuit_t *circ, const uint8_t *request, - size_t request_len); -int rend_mid_establish_rendezvous(or_circuit_t *circ, const uint8_t *request, - size_t request_len); -int rend_mid_rendezvous(or_circuit_t *circ, const uint8_t *request, - size_t request_len); - -/********************************* router.c ***************************/ - -crypto_pk_env_t *get_onion_key(void); -time_t get_onion_key_set_at(void); -void set_identity_key(crypto_pk_env_t *k); -crypto_pk_env_t *get_identity_key(void); -int identity_key_is_set(void); -authority_cert_t *get_my_v3_authority_cert(void); -crypto_pk_env_t *get_my_v3_authority_signing_key(void); -authority_cert_t *get_my_v3_legacy_cert(void); -crypto_pk_env_t *get_my_v3_legacy_signing_key(void); -void dup_onion_keys(crypto_pk_env_t **key, crypto_pk_env_t **last); -void rotate_onion_key(void); -crypto_pk_env_t *init_key_from_file(const char *fname, int generate, - int severity); -void v3_authority_check_key_expiry(void); - -int init_keys(void); - -int check_whether_orport_reachable(void); -int check_whether_dirport_reachable(void); -void consider_testing_reachability(int test_or, int test_dir); -void router_orport_found_reachable(void); -void router_dirport_found_reachable(void); -void router_perform_bandwidth_test(int num_circs, time_t now); - -int authdir_mode(or_options_t *options); -int authdir_mode_v1(or_options_t *options); -int authdir_mode_v2(or_options_t *options); -int authdir_mode_v3(or_options_t *options); -int authdir_mode_any_main(or_options_t *options); -int authdir_mode_any_nonhidserv(or_options_t *options); -int authdir_mode_handles_descs(or_options_t *options, int purpose); -int authdir_mode_publishes_statuses(or_options_t *options); -int authdir_mode_tests_reachability(or_options_t *options); -int authdir_mode_bridge(or_options_t *options); - -int server_mode(or_options_t *options); -int advertised_server_mode(void); -int proxy_mode(or_options_t *options); -void consider_publishable_server(int force); - -void router_upload_dir_desc_to_dirservers(int force); -void mark_my_descriptor_dirty_if_older_than(time_t when); -void mark_my_descriptor_dirty(void); -void check_descriptor_bandwidth_changed(time_t now); -void check_descriptor_ipaddress_changed(time_t now); -void router_new_address_suggestion(const char *suggestion, - const dir_connection_t *d_conn); -int router_compare_to_my_exit_policy(edge_connection_t *conn); -routerinfo_t *router_get_my_routerinfo(void); -extrainfo_t *router_get_my_extrainfo(void); -const char *router_get_my_descriptor(void); -int router_digest_is_me(const char *digest); -int router_extrainfo_digest_is_me(const char *digest); -int router_is_me(routerinfo_t *router); -int router_fingerprint_is_me(const char *fp); -int router_pick_published_address(or_options_t *options, uint32_t *addr); -int router_rebuild_descriptor(int force); -int router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, - crypto_pk_env_t *ident_key); -int extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo, - crypto_pk_env_t *ident_key); -char *extrainfo_get_client_geoip_summary(time_t); -int is_legal_nickname(const char *s); -int is_legal_nickname_or_hexdigest(const char *s); -int is_legal_hexdigest(const char *s); -void router_get_verbose_nickname(char *buf, const routerinfo_t *router); -void routerstatus_get_verbose_nickname(char *buf, - const routerstatus_t *router); -void router_reset_warnings(void); -void router_reset_reachability(void); -void router_free_all(void); - -const char *router_purpose_to_string(uint8_t p); -uint8_t router_purpose_from_string(const char *s); - -#ifdef ROUTER_PRIVATE -/* Used only by router.c and test.c */ -void get_platform_str(char *platform, size_t len); -#endif - /********************************* routerlist.c ***************************/ /** Represents information about a single trusted directory server. */ @@ -4319,22 +3498,7 @@ typedef struct trusted_dir_server_t { #define ROUTER_MAX_DECLARED_BANDWIDTH INT32_MAX -int get_n_authorities(authority_type_t type); -int trusted_dirs_reload_certs(void); -int trusted_dirs_load_certs_from_string(const char *contents, int from_store, - int flush); -void trusted_dirs_flush_certs_to_disk(void); -authority_cert_t *authority_cert_get_newest_by_id(const char *id_digest); -authority_cert_t *authority_cert_get_by_sk_digest(const char *sk_digest); -authority_cert_t *authority_cert_get_by_digests(const char *id_digest, - const char *sk_digest); -void authority_cert_get_all(smartlist_t *certs_out); -void authority_cert_dl_failed(const char *id_digest, int status); -void authority_certs_fetch_missing(networkstatus_t *status, time_t now); -int router_reload_router_list(void); -smartlist_t *router_get_trusted_dir_servers(void); - -/* Flags for pick_directory_server and pick_trusteddirserver. */ +/* Flags for pick_directory_server() and pick_trusteddirserver(). */ /** Flag to indicate that we should not automatically be willing to use * ourself to answer a directory request. * Passed to router_pick_directory_server (et al).*/ @@ -4363,34 +3527,13 @@ smartlist_t *router_get_trusted_dir_servers(void); */ #define PDS_NO_EXISTING_SERVERDESC_FETCH (1<<3) #define _PDS_PREFER_TUNNELED_DIR_CONNS (1<<16) -routerstatus_t *router_pick_directory_server(authority_type_t type, int flags); -trusted_dir_server_t *router_get_trusteddirserver_by_digest(const char *d); -trusted_dir_server_t *trusteddirserver_get_by_v3_auth_digest(const char *d); -routerstatus_t *router_pick_trusteddirserver(authority_type_t type, int flags); -int router_get_my_share_of_directory_requests(double *v2_share_out, - double *v3_share_out); -void router_reset_status_download_failures(void); -void routerlist_add_family(smartlist_t *sl, routerinfo_t *router); -int routers_in_same_family(routerinfo_t *r1, routerinfo_t *r2); -void add_nickname_list_to_smartlist(smartlist_t *sl, const char *list, - int must_be_running); -int router_nickname_is_in_list(routerinfo_t *router, const char *list); -routerinfo_t *routerlist_find_my_routerinfo(void); -routerinfo_t *router_find_exact_exit_enclave(const char *address, - uint16_t port); -int router_is_unreliable(routerinfo_t *router, int need_uptime, - int need_capacity, int need_guard); -uint32_t router_get_advertised_bandwidth(routerinfo_t *router); -uint32_t router_get_advertised_bandwidth_capped(routerinfo_t *router); /** Possible ways to weight routers when choosing one randomly. See * routerlist_sl_choose_by_bandwidth() for more information.*/ -typedef enum { - NO_WEIGHTING, WEIGHT_FOR_EXIT, WEIGHT_FOR_GUARD +typedef enum bandwidth_weight_rule_t { + NO_WEIGHTING, WEIGHT_FOR_EXIT, WEIGHT_FOR_MID, WEIGHT_FOR_GUARD, + WEIGHT_FOR_DIR } bandwidth_weight_rule_t; -routerinfo_t *routerlist_sl_choose_by_bandwidth(smartlist_t *sl, - bandwidth_weight_rule_t rule); -routerstatus_t *routerstatus_sl_choose_by_bandwidth(smartlist_t *sl); /** Flags to be passed to control router_choose_random_node() to indicate what * kind of nodes to pick according to what algorithm. */ @@ -4400,44 +3543,9 @@ typedef enum { CRN_NEED_GUARD = 1<<2, CRN_ALLOW_INVALID = 1<<3, /* XXXX not used, apparently. */ - CRN_STRICT_PREFERRED = 1<<4, - /* XXXX not used, apparently. */ CRN_WEIGHT_AS_EXIT = 1<<5 } router_crn_flags_t; -routerinfo_t *router_choose_random_node(const char *preferred, - smartlist_t *excludedsmartlist, - struct routerset_t *excludedset, - router_crn_flags_t flags); - -routerinfo_t *router_get_by_nickname(const char *nickname, - int warn_if_unnamed); -int router_digest_version_as_new_as(const char *digest, const char *cutoff); -int router_digest_is_trusted_dir_type(const char *digest, - authority_type_t type); -#define router_digest_is_trusted_dir(d) \ - router_digest_is_trusted_dir_type((d), NO_AUTHORITY) - -int router_addr_is_trusted_dir(uint32_t addr); -int hexdigest_to_digest(const char *hexdigest, char *digest); -routerinfo_t *router_get_by_hexdigest(const char *hexdigest); -routerinfo_t *router_get_by_digest(const char *digest); -signed_descriptor_t *router_get_by_descriptor_digest(const char *digest); -signed_descriptor_t *router_get_by_extrainfo_digest(const char *digest); -signed_descriptor_t *extrainfo_get_by_descriptor_digest(const char *digest); -const char *signed_descriptor_get_body(signed_descriptor_t *desc); -const char *signed_descriptor_get_annotations(signed_descriptor_t *desc); -routerlist_t *router_get_routerlist(void); -void routerinfo_free(routerinfo_t *router); -void extrainfo_free(extrainfo_t *extrainfo); -void routerlist_free(routerlist_t *rl); -void dump_routerlist_mem_usage(int severity); -void routerlist_remove(routerlist_t *rl, routerinfo_t *ri, int make_old, - time_t now); -void routerlist_free_all(void); -void routerlist_reset_warnings(void); -void router_set_status(const char *digest, int up); - /** Return value for router_add_to_routerlist() and dirserv_add_descriptor() */ typedef enum was_router_added_t { ROUTER_ADDED_SUCCESSFULLY = 1, @@ -4447,111 +3555,9 @@ typedef enum was_router_added_t { ROUTER_NOT_IN_CONSENSUS = -3, ROUTER_NOT_IN_CONSENSUS_OR_NETWORKSTATUS = -4, ROUTER_AUTHDIR_REJECTS = -5, + ROUTER_WAS_NOT_WANTED = -6 } was_router_added_t; -static int WRA_WAS_ADDED(was_router_added_t s); -static int WRA_WAS_OUTDATED(was_router_added_t s); -static int WRA_WAS_REJECTED(was_router_added_t s); -/** Return true iff the descriptor was added. It might still be necessary to - * check whether the descriptor generator should be notified. - */ -static INLINE int -WRA_WAS_ADDED(was_router_added_t s) { - return s == ROUTER_ADDED_SUCCESSFULLY || s == ROUTER_ADDED_NOTIFY_GENERATOR; -} -/** Return true iff the descriptor was not added because it was either: - * - not in the consensus - * - neither in the consensus nor in any networkstatus document - * - it was outdated. - */ -static INLINE int WRA_WAS_OUTDATED(was_router_added_t s) -{ - return (s == ROUTER_WAS_NOT_NEW || - s == ROUTER_NOT_IN_CONSENSUS || - s == ROUTER_NOT_IN_CONSENSUS_OR_NETWORKSTATUS); -} -/** Return true iff the descriptor rejected because it was malformed. */ -static INLINE int WRA_WAS_REJECTED(was_router_added_t s) -{ - return (s == ROUTER_AUTHDIR_REJECTS); -} -was_router_added_t router_add_to_routerlist(routerinfo_t *router, - const char **msg, - int from_cache, - int from_fetch); -was_router_added_t router_add_extrainfo_to_routerlist( - extrainfo_t *ei, const char **msg, - int from_cache, int from_fetch); -void routerlist_remove_old_routers(void); -int router_load_single_router(const char *s, uint8_t purpose, int cache, - const char **msg); -int router_load_routers_from_string(const char *s, const char *eos, - saved_location_t saved_location, - smartlist_t *requested_fingerprints, - int descriptor_digests, - const char *prepend_annotations); -void router_load_extrainfo_from_string(const char *s, const char *eos, - saved_location_t saved_location, - smartlist_t *requested_fingerprints, - int descriptor_digests); -void routerlist_retry_directory_downloads(time_t now); -int router_exit_policy_all_routers_reject(uint32_t addr, uint16_t port, - int need_uptime); -int router_exit_policy_rejects_all(routerinfo_t *router); -trusted_dir_server_t *add_trusted_dir_server(const char *nickname, - const char *address, - uint16_t dir_port, uint16_t or_port, - const char *digest, const char *v3_auth_digest, - authority_type_t type); -void authority_cert_free(authority_cert_t *cert); -void clear_trusted_dir_servers(void); -int any_trusted_dir_is_v1_authority(void); -void update_router_descriptor_downloads(time_t now); -void update_extrainfo_downloads(time_t now); -int router_have_minimum_dir_info(void); -void router_dir_info_changed(void); -const char *get_dir_info_status_string(void); -int count_loading_descriptors_progress(void); -void router_reset_descriptor_download_failures(void); -int router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2); -int routerinfo_incompatible_with_extrainfo(routerinfo_t *ri, extrainfo_t *ei, - signed_descriptor_t *sd, - const char **msg); -void routerlist_assert_ok(routerlist_t *rl); -const char *esc_router_info(routerinfo_t *router); -void routers_sort_by_identity(smartlist_t *routers); - -routerset_t *routerset_new(void); -int routerset_parse(routerset_t *target, const char *s, - const char *description); -void routerset_union(routerset_t *target, const routerset_t *source); -int routerset_is_list(const routerset_t *set); -int routerset_needs_geoip(const routerset_t *set); -int routerset_contains_router(const routerset_t *set, routerinfo_t *ri); -int routerset_contains_routerstatus(const routerset_t *set, - routerstatus_t *rs); -int routerset_contains_extendinfo(const routerset_t *set, - const extend_info_t *ei); -void routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, - int running_only); -void routersets_get_disjunction(smartlist_t *target, const smartlist_t *source, - const routerset_t *include, - const routerset_t *exclude, int running_only); -void routerset_subtract_routers(smartlist_t *out, - const routerset_t *routerset); -char *routerset_to_string(const routerset_t *routerset); -void routerset_refresh_countries(routerset_t *target); -int routerset_equal(const routerset_t *old, const routerset_t *new); -void routerset_free(routerset_t *routerset); -void routerinfo_set_country(routerinfo_t *ri); -void routerlist_refresh_countries(void); -void refresh_all_country_info(void); - -int hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, - const char *id); -int hid_serv_acting_as_directory(void); -int hid_serv_responsible_for_desc_id(const char *id); - /********************************* routerparse.c ************************/ #define MAX_STATUS_TAG_LEN 32 @@ -4570,71 +3576,10 @@ typedef struct tor_version_t { int patchlevel; char status_tag[MAX_STATUS_TAG_LEN]; int svn_revision; -} tor_version_t; -int router_get_router_hash(const char *s, size_t s_len, char *digest); -int router_get_dir_hash(const char *s, char *digest); -int router_get_runningrouters_hash(const char *s, char *digest); -int router_get_networkstatus_v2_hash(const char *s, char *digest); -int router_get_networkstatus_v3_hash(const char *s, char *digest); -int router_get_extrainfo_hash(const char *s, char *digest); -int router_append_dirobj_signature(char *buf, size_t buf_len, - const char *digest, - crypto_pk_env_t *private_key); -int router_parse_list_from_string(const char **s, const char *eos, - smartlist_t *dest, - saved_location_t saved_location, - int is_extrainfo, - int allow_annotations, - const char *prepend_annotations); -int router_parse_routerlist_from_directory(const char *s, - routerlist_t **dest, - crypto_pk_env_t *pkey, - int check_version, - int write_to_cache); -int router_parse_runningrouters(const char *str); -int router_parse_directory(const char *str); -routerinfo_t *router_parse_entry_from_string(const char *s, const char *end, - int cache_copy, - int allow_annotations, - const char *prepend_annotations); -extrainfo_t *extrainfo_parse_entry_from_string(const char *s, const char *end, - int cache_copy, struct digest_ri_map_t *routermap); -addr_policy_t *router_parse_addr_policy_item_from_string(const char *s, - int assume_action); -version_status_t tor_version_is_obsolete(const char *myversion, - const char *versionlist); -int tor_version_parse(const char *s, tor_version_t *out); -int tor_version_as_new_as(const char *platform, const char *cutoff); -int tor_version_compare(tor_version_t *a, tor_version_t *b); -void sort_version_list(smartlist_t *lst, int remove_duplicates); -void assert_addr_policy_ok(smartlist_t *t); -void dump_distinct_digest_count(int severity); - -networkstatus_v2_t *networkstatus_v2_parse_from_string(const char *s); -networkstatus_t *networkstatus_parse_vote_from_string(const char *s, - const char **eos_out, - networkstatus_type_t ns_type); -ns_detached_signatures_t *networkstatus_parse_detached_signatures( - const char *s, const char *eos); - -authority_cert_t *authority_cert_parse_from_string(const char *s, - const char **end_of_string); -int rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, - char *desc_id_out, - char **intro_points_encrypted_out, - size_t *intro_points_encrypted_size_out, - size_t *encoded_size_out, - const char **next_out, const char *desc); -int rend_decrypt_introduction_points(char **ipos_decrypted, - size_t *ipos_decrypted_size, - const char *descriptor_cookie, - const char *ipos_encrypted, - size_t ipos_encrypted_size); -int rend_parse_introduction_points(rend_service_descriptor_t *parsed, - const char *intro_points_encoded, - size_t intro_points_encoded_size); -int rend_parse_client_keys(strmap_t *parsed_clients, const char *str); + int git_tag_len; + char git_tag[DIGEST_LEN]; +} tor_version_t; #endif diff --git a/src/or/policies.c b/src/or/policies.c index f8c36c784..e48f42058 100644 --- a/src/or/policies.c +++ b/src/or/policies.c @@ -9,6 +9,10 @@ **/ #include "or.h" +#include "config.h" +#include "dirserv.h" +#include "policies.h" +#include "routerparse.h" #include "ht.h" /** Policy that addresses for incoming SOCKS connections must match. */ @@ -344,7 +348,8 @@ validate_addr_policies(or_options_t *options, char **msg) *msg = NULL; if (policies_parse_exit_policy(options->ExitPolicy, &addr_policy, - options->ExitPolicyRejectPrivate, NULL)) + options->ExitPolicyRejectPrivate, NULL, + !options->BridgeRelay)) REJECT("Error in ExitPolicy entry."); /* The rest of these calls *append* to addr_policy. So don't actually @@ -375,14 +380,8 @@ validate_addr_policies(or_options_t *options, char **msg) if (parse_addr_policy(options->ReachableDirAddresses, &addr_policy, ADDR_POLICY_ACCEPT)) REJECT("Error in ReachableDirAddresses entry."); - if (parse_addr_policy(options->AuthDirReject, &addr_policy, - ADDR_POLICY_REJECT)) - REJECT("Error in AuthDirReject entry."); - if (parse_addr_policy(options->AuthDirInvalid, &addr_policy, - ADDR_POLICY_REJECT)) - REJECT("Error in AuthDirInvalid entry."); -err: + err: addr_policy_list_free(addr_policy); return *msg ? -1 : 0; #undef REJECT @@ -829,14 +828,16 @@ exit_policy_remove_redundancies(smartlist_t *dest) "reject *:6346-6429,reject *:6699,reject *:6881-6999,accept *:*" /** Parse the exit policy <b>cfg</b> into the linked list *<b>dest</b>. If - * cfg doesn't end in an absolute accept or reject, add the default exit + * cfg doesn't end in an absolute accept or reject and if + * <b>add_default_policy</b> is true, add the default exit * policy afterwards. If <b>rejectprivate</b> is true, prepend * "reject private:*" to the policy. Return -1 if we can't parse cfg, * else return 0. */ int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, - int rejectprivate, const char *local_address) + int rejectprivate, const char *local_address, + int add_default_policy) { if (rejectprivate) { append_exit_policy_string(dest, "reject private:*"); @@ -848,13 +849,23 @@ policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, } if (parse_addr_policy(cfg, dest, -1)) return -1; - append_exit_policy_string(dest, DEFAULT_EXIT_POLICY); - + if (add_default_policy) + append_exit_policy_string(dest, DEFAULT_EXIT_POLICY); + else + append_exit_policy_string(dest, "reject *:*"); exit_policy_remove_redundancies(*dest); return 0; } +/** Add "reject *:*" to the end of the policy in *<b>dest</b>, allocating + * *<b>dest</b> as needed. */ +void +policies_exit_policy_append_reject_star(smartlist_t **dest) +{ + append_exit_policy_string(dest, "reject *:*"); +} + /** Replace the exit policy of <b>r</b> with reject *:*. */ void policies_set_router_exitpolicy_to_reject_all(routerinfo_t *r) @@ -866,6 +877,51 @@ policies_set_router_exitpolicy_to_reject_all(routerinfo_t *r) smartlist_add(r->exit_policy, item); } +/** Return 1 if there is at least one /8 subnet in <b>policy</b> that + * allows exiting to <b>port</b>. Otherwise, return 0. */ +static int +exit_policy_is_general_exit_helper(smartlist_t *policy, int port) +{ + uint32_t mask, ip, i; + /* Is this /8 rejected (1), or undecided (0)? */ + char subnet_status[256]; + + memset(subnet_status, 0, sizeof(subnet_status)); + SMARTLIST_FOREACH(policy, addr_policy_t *, p, { + if (tor_addr_family(&p->addr) != AF_INET) + continue; /* IPv4 only for now */ + if (p->prt_min > port || p->prt_max < port) + continue; /* Doesn't cover our port. */ + mask = 0; + tor_assert(p->maskbits <= 32); + + if (p->maskbits) + mask = UINT32_MAX<<(32-p->maskbits); + ip = tor_addr_to_ipv4h(&p->addr); + + /* Calculate the first and last subnet that this exit policy touches + * and set it as loop boundaries. */ + for (i = ((mask & ip)>>24); i <= (~((mask & ip) ^ mask)>>24); ++i) { + tor_addr_t addr; + if (subnet_status[i] != 0) + continue; /* We already reject some part of this /8 */ + tor_addr_from_ipv4h(&addr, i<<24); + if (tor_addr_is_internal(&addr, 0)) + continue; /* Local or non-routable addresses */ + if (p->policy_type == ADDR_POLICY_ACCEPT) { + if (p->maskbits > 8) + continue; /* Narrower than a /8. */ + /* We found an allowed subnet of at least size /8. Done + * for this port! */ + return 1; + } else if (p->policy_type == ADDR_POLICY_REJECT) { + subnet_status[i] = 1; + } + } + }); + return 0; +} + /** Return true iff <b>ri</b> is "useful as an exit node", meaning * it allows exit to at least one /8 address space for at least * two of ports 80, 443, and 6667. */ @@ -879,21 +935,7 @@ exit_policy_is_general_exit(smartlist_t *policy) return 0; for (i = 0; i < 3; ++i) { - SMARTLIST_FOREACH(policy, addr_policy_t *, p, { - if (tor_addr_family(&p->addr) != AF_INET) - continue; /* IPv4 only for now */ - if (p->prt_min > ports[i] || p->prt_max < ports[i]) - continue; /* Doesn't cover our port. */ - if (p->maskbits > 8) - continue; /* Narrower than a /8. */ - if (tor_addr_is_loopback(&p->addr)) - continue; /* 127.x or ::1. */ - /* We have a match that is at least a /8. */ - if (p->policy_type == ADDR_POLICY_ACCEPT) { - ++n_allowed; - break; /* stop considering this port */ - } - }); + n_allowed += exit_policy_is_general_exit_helper(policy, ports[i]); } return n_allowed >= 2; } @@ -1240,7 +1282,7 @@ policy_summarize(smartlist_t *policy) result = tor_malloc(final_size); tor_snprintf(result, final_size, "%s %s", prefix, shorter_str); -cleanup: + cleanup: /* cleanup */ SMARTLIST_FOREACH(summary, policy_summary_item_t *, s, tor_free(s)); smartlist_free(summary); @@ -1260,9 +1302,11 @@ cleanup: * about "exit-policy/..." */ int getinfo_helper_policies(control_connection_t *conn, - const char *question, char **answer) + const char *question, char **answer, + const char **errmsg) { (void) conn; + (void) errmsg; if (!strcmp(question, "exit-policy/default")) { *answer = tor_strdup(DEFAULT_EXIT_POLICY); } @@ -1273,7 +1317,8 @@ getinfo_helper_policies(control_connection_t *conn, void addr_policy_list_free(smartlist_t *lst) { - if (!lst) return; + if (!lst) + return; SMARTLIST_FOREACH(lst, addr_policy_t *, policy, addr_policy_free(policy)); smartlist_free(lst); } @@ -1282,19 +1327,20 @@ addr_policy_list_free(smartlist_t *lst) void addr_policy_free(addr_policy_t *p) { - if (p) { - if (--p->refcnt <= 0) { - if (p->is_canonical) { - policy_map_ent_t search, *found; - search.policy = p; - found = HT_REMOVE(policy_map, &policy_root, &search); - if (found) { - tor_assert(p == found->policy); - tor_free(found); - } + if (!p) + return; + + if (--p->refcnt <= 0) { + if (p->is_canonical) { + policy_map_ent_t search, *found; + search.policy = p; + found = HT_REMOVE(policy_map, &policy_root, &search); + if (found) { + tor_assert(p == found->policy); + tor_free(found); } - tor_free(p); } + tor_free(p); } } diff --git a/src/or/policies.h b/src/or/policies.h new file mode 100644 index 000000000..b2947c67e --- /dev/null +++ b/src/or/policies.h @@ -0,0 +1,61 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file policies.h + * \brief Header file for policies.c. + **/ + +#ifndef _TOR_POLICIES_H +#define _TOR_POLICIES_H + +/* (length of "accept 255.255.255.255/255.255.255.255:65535-65535\n" plus a + * NUL.) + */ +#define POLICY_BUF_LEN 52 + +int firewall_is_fascist_or(void); +int fascist_firewall_allows_address_or(const tor_addr_t *addr, uint16_t port); +int fascist_firewall_allows_or(routerinfo_t *ri); +int fascist_firewall_allows_address_dir(const tor_addr_t *addr, uint16_t port); +int dir_policy_permits_address(const tor_addr_t *addr); +int socks_policy_permits_address(const tor_addr_t *addr); +int authdir_policy_permits_address(uint32_t addr, uint16_t port); +int authdir_policy_valid_address(uint32_t addr, uint16_t port); +int authdir_policy_baddir_address(uint32_t addr, uint16_t port); +int authdir_policy_badexit_address(uint32_t addr, uint16_t port); + +int validate_addr_policies(or_options_t *options, char **msg); +void policy_expand_private(smartlist_t **policy); +int policies_parse_from_options(or_options_t *options); + +addr_policy_t *addr_policy_get_canonical_entry(addr_policy_t *ent); +int cmp_addr_policies(smartlist_t *a, smartlist_t *b); +addr_policy_result_t compare_tor_addr_to_addr_policy(const tor_addr_t *addr, + uint16_t port, const smartlist_t *policy); +addr_policy_result_t compare_addr_to_addr_policy(uint32_t addr, + uint16_t port, const smartlist_t *policy); +int policies_parse_exit_policy(config_line_t *cfg, smartlist_t **dest, + int rejectprivate, const char *local_address, + int add_default_policy); +void policies_exit_policy_append_reject_star(smartlist_t **dest); +void policies_set_router_exitpolicy_to_reject_all(routerinfo_t *exitrouter); +int exit_policy_is_general_exit(smartlist_t *policy); +int policy_is_reject_star(const smartlist_t *policy); +int getinfo_helper_policies(control_connection_t *conn, + const char *question, char **answer, + const char **errmsg); +int policy_write_item(char *buf, size_t buflen, addr_policy_t *item, + int format_for_desc); + +void addr_policy_list_free(smartlist_t *p); +void addr_policy_free(addr_policy_t *p); +void policies_free_all(void); + +char *policy_summarize(smartlist_t *policy); + +#endif + diff --git a/src/or/reasons.c b/src/or/reasons.c index 38eb4078c..319e6c055 100644 --- a/src/or/reasons.c +++ b/src/or/reasons.c @@ -9,6 +9,8 @@ **/ #include "or.h" +#include "config.h" +#include "reasons.h" /***************************** Edge (stream) reasons **********************/ @@ -38,6 +40,8 @@ stream_end_reason_to_control_string(int reason) case END_STREAM_REASON_NET_UNREACHABLE: return "NET_UNREACHABLE"; case END_STREAM_REASON_SOCKSPROTOCOL: return "SOCKS_PROTOCOL"; + case END_STREAM_REASON_PRIVATE_ADDR: return "PRIVATE_ADDR"; + default: return NULL; } } @@ -123,6 +127,9 @@ stream_end_reason_to_socks5_response(int reason) return SOCKS5_NET_UNREACHABLE; case END_STREAM_REASON_SOCKSPROTOCOL: return SOCKS5_GENERAL_ERROR; + case END_STREAM_REASON_PRIVATE_ADDR: + return SOCKS5_GENERAL_ERROR; + default: log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, "Reason for ending (%d) not recognized; " @@ -167,13 +174,7 @@ errno_to_stream_end_reason(int e) S_CASE(ENETUNREACH): return END_STREAM_REASON_INTERNAL; S_CASE(EHOSTUNREACH): - /* XXXX022 - * The correct behavior is END_STREAM_REASON_NOROUTE, but older - * clients don't recognize it. So we're going to continue sending - * "MISC" until 0.2.1.27 or later is "well established". - */ - /* return END_STREAM_REASON_NOROUTE; */ - return END_STREAM_REASON_MISC; + return END_STREAM_REASON_NOROUTE; S_CASE(ECONNREFUSED): return END_STREAM_REASON_CONNECTREFUSED; S_CASE(ECONNRESET): @@ -332,9 +333,78 @@ circuit_end_reason_to_control_string(int reason) return "NOPATH"; case END_CIRC_REASON_NOSUCHSERVICE: return "NOSUCHSERVICE"; + case END_CIRC_REASON_MEASUREMENT_EXPIRED: + return "MEASUREMENT_EXPIRED"; default: log_warn(LD_BUG, "Unrecognized reason code %d", (int)reason); return NULL; } } +/** Return a string corresponding to a SOCKS4 reponse code. */ +const char * +socks4_response_code_to_string(uint8_t code) +{ + switch (code) { + case 0x5a: + return "connection accepted"; + case 0x5b: + return "server rejected connection"; + case 0x5c: + return "server cannot connect to identd on this client"; + case 0x5d: + return "user id does not match identd"; + default: + return "invalid SOCKS 4 response code"; + } +} + +/** Return a string corresponding to a SOCKS5 reponse code. */ +const char * +socks5_response_code_to_string(uint8_t code) +{ + switch (code) { + case 0x00: + return "connection accepted"; + case 0x01: + return "general SOCKS server failure"; + case 0x02: + return "connection not allowed by ruleset"; + case 0x03: + return "Network unreachable"; + case 0x04: + return "Host unreachable"; + case 0x05: + return "Connection refused"; + case 0x06: + return "TTL expired"; + case 0x07: + return "Command not supported"; + case 0x08: + return "Address type not supported"; + default: + return "unknown reason"; + } +} + +/** Return a string corresponding to a bandwidht_weight_rule_t */ +const char * +bandwidth_weight_rule_to_string(bandwidth_weight_rule_t rule) +{ + switch (rule) + { + case NO_WEIGHTING: + return "no weighting"; + case WEIGHT_FOR_EXIT: + return "weight as exit"; + case WEIGHT_FOR_MID: + return "weight as middle node"; + case WEIGHT_FOR_GUARD: + return "weight as guard"; + case WEIGHT_FOR_DIR: + return "weight as directory"; + default: + return "unknown rule"; + } +} + diff --git a/src/or/reasons.h b/src/or/reasons.h new file mode 100644 index 000000000..01f971794 --- /dev/null +++ b/src/or/reasons.h @@ -0,0 +1,31 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file reasons.h + * \brief Header file for reasons.c. + **/ + +#ifndef _TOR_REASONS_H +#define _TOR_REASONS_H + +const char *stream_end_reason_to_control_string(int reason); +const char *stream_end_reason_to_string(int reason); +socks5_reply_status_t stream_end_reason_to_socks5_response(int reason); +uint8_t errno_to_stream_end_reason(int e); + +const char *orconn_end_reason_to_control_string(int r); +int tls_error_to_orconn_end_reason(int e); +int errno_to_orconn_end_reason(int e); + +const char *circuit_end_reason_to_control_string(int reason); +const char *socks4_response_code_to_string(uint8_t code); +const char *socks5_response_code_to_string(uint8_t code); + +const char *bandwidth_weight_rule_to_string(enum bandwidth_weight_rule_t rule); + +#endif + diff --git a/src/or/relay.c b/src/or/relay.c index b40d7ad24..807cb3d43 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -10,8 +10,26 @@ * receiving from circuits, plus queuing on circuits. **/ +#include <math.h> #include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "connection_or.h" +#include "control.h" +#include "geoip.h" +#include "main.h" #include "mempool.h" +#include "networkstatus.h" +#include "policies.h" +#include "reasons.h" +#include "relay.h" +#include "rendcommon.h" +#include "routerlist.h" +#include "routerparse.h" static int relay_crypt(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, @@ -20,20 +38,46 @@ static edge_connection_t *relay_lookup_conn(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, crypt_path_t *layer_hint); -static int -connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, - edge_connection_t *conn, - crypt_path_t *layer_hint); -static void -circuit_consider_sending_sendme(circuit_t *circ, crypt_path_t *layer_hint); +static int connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, + edge_connection_t *conn, + crypt_path_t *layer_hint); +static void circuit_consider_sending_sendme(circuit_t *circ, + crypt_path_t *layer_hint); +static void circuit_resume_edge_reading(circuit_t *circ, + crypt_path_t *layer_hint); +static int circuit_resume_edge_reading_helper(edge_connection_t *conn, + circuit_t *circ, + crypt_path_t *layer_hint); +static int circuit_consider_stop_edge_reading(circuit_t *circ, + crypt_path_t *layer_hint); +static int circuit_queue_streams_are_blocked(circuit_t *circ); + +/* XXXX023 move this all to compat_libevent */ +/** Cache the current hi-res time; the cache gets reset when libevent + * calls us. */ +static struct timeval cached_time_hires = {0, 0}; + +/** Stop reading on edge connections when we have this many cells + * waiting on the appropriate queue. */ +#define CELL_QUEUE_HIGHWATER_SIZE 256 +/** Start reading from edge connections again when we get down to this many + * cells. */ +#define CELL_QUEUE_LOWWATER_SIZE 64 + static void -circuit_resume_edge_reading(circuit_t *circ, crypt_path_t *layer_hint); -static int -circuit_resume_edge_reading_helper(edge_connection_t *conn, - circuit_t *circ, - crypt_path_t *layer_hint); -static int -circuit_consider_stop_edge_reading(circuit_t *circ, crypt_path_t *layer_hint); +tor_gettimeofday_cached(struct timeval *tv) +{ + if (cached_time_hires.tv_sec == 0) { + tor_gettimeofday(&cached_time_hires); + } + *tv = cached_time_hires; +} + +void +tor_gettimeofday_cache_clear(void) +{ + cached_time_hires.tv_sec = 0; +} /** Stats: how many relay cells have originated at this hop, or have * been relayed onward (not recognized at this hop)? @@ -230,7 +274,7 @@ circuit_receive_relay_cell(cell_t *cell, circuit_t *circ, * we might kill the circ before we relay * the cells. */ - append_cell_to_circuit_queue(circ, or_conn, cell, cell_direction); + append_cell_to_circuit_queue(circ, or_conn, cell, cell_direction, 0); return 0; } @@ -327,7 +371,7 @@ relay_crypt(circuit_t *circ, cell_t *cell, cell_direction_t cell_direction, static int circuit_package_relay_cell(cell_t *cell, circuit_t *circ, cell_direction_t cell_direction, - crypt_path_t *layer_hint) + crypt_path_t *layer_hint, streamid_t on_stream) { or_connection_t *conn; /* where to send the cell */ @@ -371,7 +415,7 @@ circuit_package_relay_cell(cell_t *cell, circuit_t *circ, } ++stats_n_relay_cells_relayed; - append_cell_to_circuit_queue(circ, conn, cell, cell_direction); + append_cell_to_circuit_queue(circ, conn, cell, cell_direction, on_stream); return 0; } @@ -496,7 +540,7 @@ relay_command_to_string(uint8_t command) * return 0. */ int -relay_send_command_from_edge(uint16_t stream_id, circuit_t *circ, +relay_send_command_from_edge(streamid_t stream_id, circuit_t *circ, uint8_t relay_command, const char *payload, size_t payload_len, crypt_path_t *cpath_layer) { @@ -531,6 +575,12 @@ relay_send_command_from_edge(uint16_t stream_id, circuit_t *circ, log_debug(LD_OR,"delivering %d cell %s.", relay_command, cell_direction == CELL_DIRECTION_OUT ? "forward" : "backward"); + /* If we are sending an END cell and this circuit is used for a tunneled + * directory request, advance its state. */ + if (relay_command == RELAY_COMMAND_END && circ->dirreq_id) + geoip_change_dirreq_state(circ->dirreq_id, DIRREQ_TUNNELED, + DIRREQ_END_CELL_SENT); + if (cell_direction == CELL_DIRECTION_OUT && circ->n_conn) { /* if we're using relaybandwidthrate, this conn wants priority */ circ->n_conn->client_used = approx_time(); @@ -540,17 +590,11 @@ relay_send_command_from_edge(uint16_t stream_id, circuit_t *circ, origin_circuit_t *origin_circ = TO_ORIGIN_CIRCUIT(circ); if (origin_circ->remaining_relay_early_cells > 0 && (relay_command == RELAY_COMMAND_EXTEND || - (cpath_layer != origin_circ->cpath && - !CIRCUIT_PURPOSE_IS_ESTABLISHED_REND(circ->purpose)))) { - /* If we've got any relay_early cells left, and we're sending - * an extend cell or (we're not talking to the first hop and we're - * not talking to a rendezvous circuit), use one of them. - * Don't worry about the conn protocol version: + cpath_layer != origin_circ->cpath)) { + /* If we've got any relay_early cells left and (we're sending + * an extend cell or we're not talking to the first hop), use + * one of them. Don't worry about the conn protocol version: * append_cell_to_circuit_queue will fix it up. */ - /* XXX For now, clients don't use RELAY_EARLY cells when sending - * relay cells on rendezvous circuits. See bug 1038. Eventually, - * we can take this behavior away in favor of having clients avoid - * rendezvous points running 0.2.1.3-alpha through 0.2.1.18. -RD */ cell.command = CELL_RELAY_EARLY; --origin_circ->remaining_relay_early_cells; log_debug(LD_OR, "Sending a RELAY_EARLY cell; %d remaining.", @@ -578,8 +622,8 @@ relay_send_command_from_edge(uint16_t stream_id, circuit_t *circ, } } - if (circuit_package_relay_cell(&cell, circ, cell_direction, cpath_layer) - < 0) { + if (circuit_package_relay_cell(&cell, circ, cell_direction, cpath_layer, + stream_id) < 0) { log_warn(LD_BUG,"circuit_package_relay_cell failed. Closing."); circuit_mark_for_close(circ, END_CIRC_REASON_INTERNAL); return -1; @@ -747,6 +791,8 @@ connection_ap_process_end_not_open( < MAX_RESOLVE_FAILURES) { /* We haven't retried too many times; reattach the connection. */ circuit_log_path(LOG_INFO,LD_APP,circ); + /* Mark this circuit "unusable for new streams". */ + /* XXXX023 this is a kludgy way to do this. */ tor_assert(circ->_base.timestamp_dirty); circ->_base.timestamp_dirty -= get_options()->MaxCircuitDirtiness; @@ -901,7 +947,7 @@ connection_edge_process_relay_cell_not_open( } /* handle anything that might have queued */ - if (connection_edge_package_raw_inbuf(conn, 1) < 0) { + if (connection_edge_package_raw_inbuf(conn, 1, NULL) < 0) { /* (We already sent an end cell if possible) */ connection_mark_for_close(TO_CONN(conn)); return 0; @@ -999,7 +1045,8 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, relay_header_unpack(&rh, cell->payload); // log_fn(LOG_DEBUG,"command %d stream %d", rh.command, rh.stream_id); num_seen++; - log_debug(domain, "Now seen %d relay cells here.", num_seen); + log_debug(domain, "Now seen %d relay cells here (command %d, stream %d).", + num_seen, rh.command, rh.stream_id); if (rh.length > RELAY_PAYLOAD_SIZE) { log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, @@ -1038,6 +1085,16 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, "Begin cell for known stream. Dropping."); return 0; } + if (rh.command == RELAY_COMMAND_BEGIN_DIR) { + /* Assign this circuit and its app-ward OR connection a unique ID, + * so that we can measure download times. The local edge and dir + * connection will be assigned the same ID when they are created + * and linked. */ + static uint64_t next_id = 0; + circ->dirreq_id = ++next_id; + TO_CONN(TO_OR_CIRCUIT(circ)->p_conn)->dirreq_id = circ->dirreq_id; + } + return connection_exit_begin_conn(cell, circ); case RELAY_COMMAND_DATA: ++stats_n_data_cells_received; @@ -1131,6 +1188,7 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, } if (circ->n_conn) { uint8_t trunc_reason = *(uint8_t*)(cell->payload + RELAY_HEADER_SIZE); + circuit_clear_cell_queue(circ, circ->n_conn); connection_or_send_destroy(circ->n_circ_id, circ->n_conn, trunc_reason); circuit_set_n_circid_orconn(circ, 0, NULL); @@ -1179,9 +1237,13 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, conn->package_window += STREAMWINDOW_INCREMENT; log_debug(domain,"stream-level sendme, packagewindow now %d.", conn->package_window); + if (circuit_queue_streams_are_blocked(circ)) { + /* Still waiting for queue to flush; don't touch conn */ + return 0; + } connection_start_reading(TO_CONN(conn)); /* handle whatever might still be on the inbuf */ - if (connection_edge_package_raw_inbuf(conn, 1) < 0) { + if (connection_edge_package_raw_inbuf(conn, 1, NULL) < 0) { /* (We already sent an end cell if possible) */ connection_mark_for_close(TO_CONN(conn)); return 0; @@ -1247,15 +1309,19 @@ uint64_t stats_n_data_cells_received = 0; * ever received were completely full of data. */ uint64_t stats_n_data_bytes_received = 0; -/** While conn->inbuf has an entire relay payload of bytes on it, - * and the appropriate package windows aren't empty, grab a cell - * and send it down the circuit. +/** If <b>conn</b> has an entire relay payload of bytes on its inbuf (or + * <b>package_partial</b> is true), and the appropriate package windows aren't + * empty, grab a cell and send it down the circuit. + * + * If *<b>max_cells</b> is given, package no more than max_cells. Decrement + * *<b>max_cells</b> by the number of cells packaged. * * Return -1 (and send a RELAY_COMMAND_END cell if necessary) if conn should * be marked for close, else return 0. */ int -connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial) +connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial, + int *max_cells) { size_t amount_to_process, length; char payload[CELL_PAYLOAD_SIZE]; @@ -1271,7 +1337,10 @@ connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial) return 0; } -repeat_connection_edge_package_raw_inbuf: + if (max_cells && *max_cells <= 0) + return 0; + + repeat_connection_edge_package_raw_inbuf: circ = circuit_get_by_edge_conn(conn); if (!circ) { @@ -1332,12 +1401,19 @@ repeat_connection_edge_package_raw_inbuf: } log_debug(domain,"conn->package_window is now %d",conn->package_window); + if (max_cells) { + *max_cells -= 1; + if (*max_cells <= 0) + return 0; + } + /* handle more if there's more, or return 0 if there isn't */ goto repeat_connection_edge_package_raw_inbuf; } -/** Called when we've just received a relay data cell, or when - * we've just finished flushing all bytes to stream <b>conn</b>. +/** Called when we've just received a relay data cell, when + * we've just finished flushing all bytes to stream <b>conn</b>, + * or when we've flushed *some* bytes to the stream <b>conn</b>. * * If conn->outbuf is not too full, and our deliver window is * low, send back a suitable number of stream-level sendme cells. @@ -1379,7 +1455,10 @@ connection_edge_consider_sending_sendme(edge_connection_t *conn) static void circuit_resume_edge_reading(circuit_t *circ, crypt_path_t *layer_hint) { - + if (circuit_queue_streams_are_blocked(circ)) { + log_debug(layer_hint?LD_APP:LD_EXIT,"Too big queue, no resuming"); + return; + } log_debug(layer_hint?LD_APP:LD_EXIT,"resuming"); if (CIRCUIT_IS_ORIGIN(circ)) @@ -1395,31 +1474,136 @@ circuit_resume_edge_reading(circuit_t *circ, crypt_path_t *layer_hint) * of a linked list of edge streams that should each be considered. */ static int -circuit_resume_edge_reading_helper(edge_connection_t *conn, +circuit_resume_edge_reading_helper(edge_connection_t *first_conn, circuit_t *circ, crypt_path_t *layer_hint) { - for ( ; conn; conn=conn->next_stream) { - if (conn->_base.marked_for_close) + edge_connection_t *conn; + int n_packaging_streams, n_streams_left; + int packaged_this_round; + int cells_on_queue; + int cells_per_conn; + edge_connection_t *chosen_stream = NULL; + + /* How many cells do we have space for? It will be the minimum of + * the number needed to exhaust the package window, and the minimum + * needed to fill the cell queue. */ + int max_to_package = circ->package_window; + if (CIRCUIT_IS_ORIGIN(circ)) { + cells_on_queue = circ->n_conn_cells.n; + } else { + or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); + cells_on_queue = or_circ->p_conn_cells.n; + } + if (CELL_QUEUE_HIGHWATER_SIZE - cells_on_queue < max_to_package) + max_to_package = CELL_QUEUE_HIGHWATER_SIZE - cells_on_queue; + + /* Once we used to start listening on the streams in the order they + * appeared in the linked list. That leads to starvation on the + * streams that appeared later on the list, since the first streams + * would always get to read first. Instead, we just pick a random + * stream on the list, and enable reading for streams starting at that + * point (and wrapping around as if the list were circular). It would + * probably be better to actually remember which streams we've + * serviced in the past, but this is simple and effective. */ + + /* Select a stream uniformly at random from the linked list. We + * don't need cryptographic randomness here. */ + { + int num_streams = 0; + for (conn = first_conn; conn; conn = conn->next_stream) { + num_streams++; + if ((tor_weak_random() % num_streams)==0) + chosen_stream = conn; + /* Invariant: chosen_stream has been chosen uniformly at random from + * among the first num_streams streams on first_conn. */ + } + } + + /* Count how many non-marked streams there are that have anything on + * their inbuf, and enable reading on all of the connections. */ + n_packaging_streams = 0; + /* Activate reading starting from the chosen stream */ + for (conn=chosen_stream; conn; conn = conn->next_stream) { + /* Start reading for the streams starting from here */ + if (conn->_base.marked_for_close || conn->package_window <= 0) continue; - if ((!layer_hint && conn->package_window > 0) || - (layer_hint && conn->package_window > 0 && - conn->cpath_layer == layer_hint)) { + if (!layer_hint || conn->cpath_layer == layer_hint) { connection_start_reading(TO_CONN(conn)); + + if (buf_datalen(conn->_base.inbuf) > 0) + ++n_packaging_streams; + } + } + /* Go back and do the ones we skipped, circular-style */ + for (conn = first_conn; conn != chosen_stream; conn = conn->next_stream) { + if (conn->_base.marked_for_close || conn->package_window <= 0) + continue; + if (!layer_hint || conn->cpath_layer == layer_hint) { + connection_start_reading(TO_CONN(conn)); + + if (buf_datalen(conn->_base.inbuf) > 0) + ++n_packaging_streams; + } + } + + if (n_packaging_streams == 0) /* avoid divide-by-zero */ + return 0; + + again: + + cells_per_conn = CEIL_DIV(max_to_package, n_packaging_streams); + + packaged_this_round = 0; + n_streams_left = 0; + + /* Iterate over all connections. Package up to cells_per_conn cells on + * each. Update packaged_this_round with the total number of cells + * packaged, and n_streams_left with the number that still have data to + * package. + */ + for (conn=first_conn; conn; conn=conn->next_stream) { + if (conn->_base.marked_for_close || conn->package_window <= 0) + continue; + if (!layer_hint || conn->cpath_layer == layer_hint) { + int n = cells_per_conn, r; /* handle whatever might still be on the inbuf */ - if (connection_edge_package_raw_inbuf(conn, 1)<0) { - /* (We already sent an end cell if possible) */ + r = connection_edge_package_raw_inbuf(conn, 1, &n); + + /* Note how many we packaged */ + packaged_this_round += (cells_per_conn-n); + + if (r<0) { + /* Problem while packaging. (We already sent an end cell if + * possible) */ connection_mark_for_close(TO_CONN(conn)); continue; } + /* If there's still data to read, we'll be coming back to this stream. */ + if (buf_datalen(conn->_base.inbuf)) + ++n_streams_left; + /* If the circuit won't accept any more data, return without looking * at any more of the streams. Any connections that should be stopped * have already been stopped by connection_edge_package_raw_inbuf. */ if (circuit_consider_stop_edge_reading(circ, layer_hint)) return -1; + /* XXXX should we also stop immediately if we fill up the cell queue? + * Probably. */ } } + + /* If we made progress, and we are willing to package more, and there are + * any streams left that want to package stuff... try again! + */ + if (packaged_this_round && packaged_this_round < max_to_package && + n_streams_left) { + max_to_package -= packaged_this_round; + n_packaging_streams = n_streams_left; + goto again; + } + return 0; } @@ -1488,13 +1672,6 @@ circuit_consider_sending_sendme(circuit_t *circ, crypt_path_t *layer_hint) } } -/** Stop reading on edge connections when we have this many cells - * waiting on the appropriate queue. */ -#define CELL_QUEUE_HIGHWATER_SIZE 256 -/** Start reading from edge connections again when we get down to this many - * cells. */ -#define CELL_QUEUE_LOWWATER_SIZE 64 - #ifdef ACTIVE_CIRCUITS_PARANOIA #define assert_active_circuits_ok_paranoid(conn) \ assert_active_circuits_ok(conn) @@ -1508,6 +1685,10 @@ static int total_cells_allocated = 0; /** A memory pool to allocate packed_cell_t objects. */ static mp_pool_t *cell_pool = NULL; +/** Memory pool to allocate insertion_time_elem_t objects used for cell + * statistics. */ +static mp_pool_t *it_pool = NULL; + /** Allocate structures to hold cells. */ void init_cell_pool(void) @@ -1516,7 +1697,8 @@ init_cell_pool(void) cell_pool = mp_pool_new(sizeof(packed_cell_t), 128*1024); } -/** Free all storage used to hold cells. */ +/** Free all storage used to hold cells (and insertion times if we measure + * cell statistics). */ void free_cell_pool(void) { @@ -1525,6 +1707,10 @@ free_cell_pool(void) mp_pool_destroy(cell_pool); cell_pool = NULL; } + if (it_pool) { + mp_pool_destroy(it_pool); + it_pool = NULL; + } } /** Free excess storage in cell pool. */ @@ -1537,7 +1723,7 @@ clean_cell_pool(void) /** Release storage held by <b>cell</b>. */ static INLINE void -packed_cell_free(packed_cell_t *cell) +packed_cell_free_unchecked(packed_cell_t *cell) { --total_cells_allocated; mp_pool_release(cell); @@ -1599,7 +1785,38 @@ cell_queue_append(cell_queue_t *queue, packed_cell_t *cell) void cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell) { - cell_queue_append(queue, packed_cell_copy(cell)); + packed_cell_t *copy = packed_cell_copy(cell); + /* Remember the time when this cell was put in the queue. */ + if (get_options()->CellStatistics) { + struct timeval now; + uint32_t added; + insertion_time_queue_t *it_queue = queue->insertion_times; + if (!it_pool) + it_pool = mp_pool_new(sizeof(insertion_time_elem_t), 1024); + tor_gettimeofday_cached(&now); +#define SECONDS_IN_A_DAY 86400L + added = (uint32_t)(((now.tv_sec % SECONDS_IN_A_DAY) * 100L) + + ((uint32_t)now.tv_usec / (uint32_t)10000L)); + if (!it_queue) { + it_queue = tor_malloc_zero(sizeof(insertion_time_queue_t)); + queue->insertion_times = it_queue; + } + if (it_queue->last && it_queue->last->insertion_time == added) { + it_queue->last->counter++; + } else { + insertion_time_elem_t *elem = mp_pool_get(it_pool); + elem->next = NULL; + elem->insertion_time = added; + elem->counter = 1; + if (it_queue->last) { + it_queue->last->next = elem; + it_queue->last = elem; + } else { + it_queue->first = it_queue->last = elem; + } + } + } + cell_queue_append(queue, copy); } /** Remove and free every cell in <b>queue</b>. */ @@ -1610,11 +1827,19 @@ cell_queue_clear(cell_queue_t *queue) cell = queue->head; while (cell) { next = cell->next; - packed_cell_free(cell); + packed_cell_free_unchecked(cell); cell = next; } queue->head = queue->tail = NULL; queue->n = 0; + if (queue->insertion_times) { + while (queue->insertion_times->first) { + insertion_time_elem_t *elem = queue->insertion_times->first; + queue->insertion_times->first = elem->next; + mp_pool_release(elem); + } + tor_free(queue->insertion_times); + } } /** Extract and return the cell at the head of <b>queue</b>; return NULL if @@ -1666,8 +1891,226 @@ prev_circ_on_conn_p(circuit_t *circ, or_connection_t *conn) } } +/** Helper for sorting cell_ewma_t values in their priority queue. */ +static int +compare_cell_ewma_counts(const void *p1, const void *p2) +{ + const cell_ewma_t *e1=p1, *e2=p2; + if (e1->cell_count < e2->cell_count) + return -1; + else if (e1->cell_count > e2->cell_count) + return 1; + else + return 0; +} + +/** Given a cell_ewma_t, return a pointer to the circuit containing it. */ +static circuit_t * +cell_ewma_to_circuit(cell_ewma_t *ewma) +{ + if (ewma->is_for_p_conn) { + /* This is an or_circuit_t's p_cell_ewma. */ + or_circuit_t *orcirc = SUBTYPE_P(ewma, or_circuit_t, p_cell_ewma); + return TO_CIRCUIT(orcirc); + } else { + /* This is some circuit's n_cell_ewma. */ + return SUBTYPE_P(ewma, circuit_t, n_cell_ewma); + } +} + +/* ==== Functions for scaling cell_ewma_t ==== + + When choosing which cells to relay first, we favor circuits that have been + quiet recently. This gives better latency on connections that aren't + pushing lots of data, and makes the network feel more interactive. + + Conceptually, we take an exponentially weighted mean average of the number + of cells a circuit has sent, and allow active circuits (those with cells to + relay) to send cells in reverse order of their exponentially-weighted mean + average (EWMA) cell count. [That is, a cell sent N seconds ago 'counts' + F^N times as much as a cell sent now, for 0<F<1.0, and we favor the + circuit that has sent the fewest cells] + + If 'double' had infinite precision, we could do this simply by counting a + cell sent at startup as having weight 1.0, and a cell sent N seconds later + as having weight F^-N. This way, we would never need to re-scale + any already-sent cells. + + To prevent double from overflowing, we could count a cell sent now as + having weight 1.0 and a cell sent N seconds ago as having weight F^N. + This, however, would mean we'd need to re-scale *ALL* old circuits every + time we wanted to send a cell. + + So as a compromise, we divide time into 'ticks' (currently, 10-second + increments) and say that a cell sent at the start of a current tick is + worth 1.0, a cell sent N seconds before the start of the current tick is + worth F^N, and a cell sent N seconds after the start of the current tick is + worth F^-N. This way we don't overflow, and we don't need to constantly + rescale. + */ + +/** How long does a tick last (seconds)? */ +#define EWMA_TICK_LEN 10 + +/** The default per-tick scale factor, if it hasn't been overridden by a + * consensus or a configuration setting. zero means "disabled". */ +#define EWMA_DEFAULT_HALFLIFE 0.0 + +/** Given a timeval <b>now</b>, compute the cell_ewma tick in which it occurs + * and the fraction of the tick that has elapsed between the start of the tick + * and <b>now</b>. Return the former and store the latter in + * *<b>remainder_out</b>. + * + * These tick values are not meant to be shared between Tor instances, or used + * for other purposes. */ +static unsigned +cell_ewma_tick_from_timeval(const struct timeval *now, + double *remainder_out) +{ + unsigned res = (unsigned) (now->tv_sec / EWMA_TICK_LEN); + /* rem */ + double rem = (now->tv_sec % EWMA_TICK_LEN) + + ((double)(now->tv_usec)) / 1.0e6; + *remainder_out = rem / EWMA_TICK_LEN; + return res; +} + +/** Compute and return the current cell_ewma tick. */ +unsigned +cell_ewma_get_tick(void) +{ + return ((unsigned)approx_time() / EWMA_TICK_LEN); +} + +/** The per-tick scale factor to be used when computing cell-count EWMA + * values. (A cell sent N ticks before the start of the current tick + * has value ewma_scale_factor ** N.) + */ +static double ewma_scale_factor = 0.1; +static int ewma_enabled = 0; + +#define EPSILON 0.00001 +#define LOG_ONEHALF -0.69314718055994529 + +/** Adjust the global cell scale factor based on <b>options</b> */ +void +cell_ewma_set_scale_factor(or_options_t *options, networkstatus_t *consensus) +{ + int32_t halflife_ms; + double halflife; + const char *source; + if (options && options->CircuitPriorityHalflife >= -EPSILON) { + halflife = options->CircuitPriorityHalflife; + source = "CircuitPriorityHalflife in configuration"; + } else if (consensus && (halflife_ms = networkstatus_get_param( + consensus, "CircuitPriorityHalflifeMsec", + -1, -1, INT32_MAX)) >= 0) { + halflife = ((double)halflife_ms)/1000.0; + source = "CircuitPriorityHalflifeMsec in consensus"; + } else { + halflife = EWMA_DEFAULT_HALFLIFE; + source = "Default value"; + } + + if (halflife <= EPSILON) { + /* The cell EWMA algorithm is disabled. */ + ewma_scale_factor = 0.1; + ewma_enabled = 0; + log_info(LD_OR, + "Disabled cell_ewma algorithm because of value in %s", + source); + } else { + /* convert halflife into halflife-per-tick. */ + halflife /= EWMA_TICK_LEN; + /* compute per-tick scale factor. */ + ewma_scale_factor = exp( LOG_ONEHALF / halflife ); + ewma_enabled = 1; + log_info(LD_OR, + "Enabled cell_ewma algorithm because of value in %s; " + "scale factor is %lf per %d seconds", + source, ewma_scale_factor, EWMA_TICK_LEN); + } +} + +/** Return the multiplier necessary to convert the value of a cell sent in + * 'from_tick' to one sent in 'to_tick'. */ +static INLINE double +get_scale_factor(unsigned from_tick, unsigned to_tick) +{ + /* This math can wrap around, but that's okay: unsigned overflow is + well-defined */ + int diff = (int)(to_tick - from_tick); + return pow(ewma_scale_factor, diff); +} + +/** Adjust the cell count of <b>ewma</b> so that it is scaled with respect to + * <b>cur_tick</b> */ +static void +scale_single_cell_ewma(cell_ewma_t *ewma, unsigned cur_tick) +{ + double factor = get_scale_factor(ewma->last_adjusted_tick, cur_tick); + ewma->cell_count *= factor; + ewma->last_adjusted_tick = cur_tick; +} + +/** Adjust the cell count of every active circuit on <b>conn</b> so + * that they are scaled with respect to <b>cur_tick</b> */ +static void +scale_active_circuits(or_connection_t *conn, unsigned cur_tick) +{ + + double factor = get_scale_factor( + conn->active_circuit_pqueue_last_recalibrated, + cur_tick); + /** Ordinarily it isn't okay to change the value of an element in a heap, + * but it's okay here, since we are preserving the order. */ + SMARTLIST_FOREACH(conn->active_circuit_pqueue, cell_ewma_t *, e, { + tor_assert(e->last_adjusted_tick == + conn->active_circuit_pqueue_last_recalibrated); + e->cell_count *= factor; + e->last_adjusted_tick = cur_tick; + }); + conn->active_circuit_pqueue_last_recalibrated = cur_tick; +} + +/** Rescale <b>ewma</b> to the same scale as <b>conn</b>, and add it to + * <b>conn</b>'s priority queue of active circuits */ +static void +add_cell_ewma_to_conn(or_connection_t *conn, cell_ewma_t *ewma) +{ + tor_assert(ewma->heap_index == -1); + scale_single_cell_ewma(ewma, + conn->active_circuit_pqueue_last_recalibrated); + + smartlist_pqueue_add(conn->active_circuit_pqueue, + compare_cell_ewma_counts, + STRUCT_OFFSET(cell_ewma_t, heap_index), + ewma); +} + +/** Remove <b>ewma</b> from <b>conn</b>'s priority queue of active circuits */ +static void +remove_cell_ewma_from_conn(or_connection_t *conn, cell_ewma_t *ewma) +{ + tor_assert(ewma->heap_index != -1); + smartlist_pqueue_remove(conn->active_circuit_pqueue, + compare_cell_ewma_counts, + STRUCT_OFFSET(cell_ewma_t, heap_index), + ewma); +} + +/** Remove and return the first cell_ewma_t from conn's priority queue of + * active circuits. Requires that the priority queue is nonempty. */ +static cell_ewma_t * +pop_first_cell_ewma_from_conn(or_connection_t *conn) +{ + return smartlist_pqueue_pop(conn->active_circuit_pqueue, + compare_cell_ewma_counts, + STRUCT_OFFSET(cell_ewma_t, heap_index)); +} + /** Add <b>circ</b> to the list of circuits with pending cells on - * <b>conn</b>. No effect if <b>circ</b> is already unlinked. */ + * <b>conn</b>. No effect if <b>circ</b> is already linked. */ void make_circuit_active_on_conn(circuit_t *circ, or_connection_t *conn) { @@ -1679,6 +2122,8 @@ make_circuit_active_on_conn(circuit_t *circ, or_connection_t *conn) return; } + assert_active_circuits_ok_paranoid(conn); + if (! conn->active_circuits) { conn->active_circuits = circ; *prevp = *nextp = circ; @@ -1690,10 +2135,19 @@ make_circuit_active_on_conn(circuit_t *circ, or_connection_t *conn) *prev_circ_on_conn_p(head, conn) = circ; *prevp = old_tail; } + + if (circ->n_conn == conn) { + add_cell_ewma_to_conn(conn, &circ->n_cell_ewma); + } else { + or_circuit_t *orcirc = TO_OR_CIRCUIT(circ); + tor_assert(conn == orcirc->p_conn); + add_cell_ewma_to_conn(conn, &orcirc->p_cell_ewma); + } + assert_active_circuits_ok_paranoid(conn); } -/** Remove <b>circ</b> to the list of circuits with pending cells on +/** Remove <b>circ</b> from the list of circuits with pending cells on * <b>conn</b>. No effect if <b>circ</b> is already unlinked. */ void make_circuit_inactive_on_conn(circuit_t *circ, or_connection_t *conn) @@ -1707,6 +2161,8 @@ make_circuit_inactive_on_conn(circuit_t *circ, or_connection_t *conn) return; } + assert_active_circuits_ok_paranoid(conn); + tor_assert(next && prev); tor_assert(*prev_circ_on_conn_p(next, conn) == circ); tor_assert(*next_circ_on_conn_p(prev, conn) == circ); @@ -1720,6 +2176,15 @@ make_circuit_inactive_on_conn(circuit_t *circ, or_connection_t *conn) conn->active_circuits = next; } *prevp = *nextp = NULL; + + if (circ->n_conn == conn) { + remove_cell_ewma_from_conn(conn, &circ->n_cell_ewma); + } else { + or_circuit_t *orcirc = TO_OR_CIRCUIT(circ); + tor_assert(conn == orcirc->p_conn); + remove_cell_ewma_from_conn(conn, &orcirc->p_cell_ewma); + } + assert_active_circuits_ok_paranoid(conn); } @@ -1739,16 +2204,27 @@ connection_or_unlink_all_active_circs(or_connection_t *orconn) cur = next; } while (cur != head); orconn->active_circuits = NULL; + + SMARTLIST_FOREACH(orconn->active_circuit_pqueue, cell_ewma_t *, e, + e->heap_index = -1); + smartlist_clear(orconn->active_circuit_pqueue); } /** Block (if <b>block</b> is true) or unblock (if <b>block</b> is false) * every edge connection that is using <b>circ</b> to write to <b>orconn</b>, - * and start or stop reading as appropriate. */ -static void + * and start or stop reading as appropriate. + * + * If <b>stream_id</b> is nonzero, block only the edge connection whose + * stream_id matches it. + * + * Returns the number of streams whose status we changed. + */ +static int set_streams_blocked_on_circ(circuit_t *circ, or_connection_t *orconn, - int block) + int block, streamid_t stream_id) { edge_connection_t *edge = NULL; + int n = 0; if (circ->n_conn == orconn) { circ->streams_blocked_on_n_conn = block; if (CIRCUIT_IS_ORIGIN(circ)) @@ -1761,7 +2237,13 @@ set_streams_blocked_on_circ(circuit_t *circ, or_connection_t *orconn, for (; edge; edge = edge->next_stream) { connection_t *conn = TO_CONN(edge); - edge->edge_blocked_on_circ = block; + if (stream_id && edge->stream_id != stream_id) + continue; + + if (edge->edge_blocked_on_circ != block) { + ++n; + edge->edge_blocked_on_circ = block; + } if (!conn->read_event) { /* This connection is a placeholder for something; probably a DNS @@ -1778,10 +2260,12 @@ set_streams_blocked_on_circ(circuit_t *circ, or_connection_t *orconn, connection_start_reading(conn); } } + + return n; } /** Pull as many cells as possible (but no more than <b>max</b>) from the - * queue of the first active circuit on <b>conn</b>, and write then to + * queue of the first active circuit on <b>conn</b>, and write them to * <b>conn</b>->outbuf. Return the number of cells written. Advance * the active circuit pointer to the next active circuit in the ring. */ int @@ -1792,9 +2276,35 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max, cell_queue_t *queue; circuit_t *circ; int streams_blocked; + + /* The current (hi-res) time */ + struct timeval now_hires; + + /* The EWMA cell counter for the circuit we're flushing. */ + cell_ewma_t *cell_ewma = NULL; + double ewma_increment = -1; + circ = conn->active_circuits; if (!circ) return 0; assert_active_circuits_ok_paranoid(conn); + + /* See if we're doing the ewma circuit selection algorithm. */ + if (ewma_enabled) { + unsigned tick; + double fractional_tick; + tor_gettimeofday_cached(&now_hires); + tick = cell_ewma_tick_from_timeval(&now_hires, &fractional_tick); + + if (tick != conn->active_circuit_pqueue_last_recalibrated) { + scale_active_circuits(conn, tick); + } + + ewma_increment = pow(ewma_scale_factor, -fractional_tick); + + cell_ewma = smartlist_get(conn->active_circuit_pqueue, 0); + circ = cell_ewma_to_circuit(cell_ewma); + } + if (circ->n_conn == conn) { queue = &circ->n_conn_cells; streams_blocked = circ->streams_blocked_on_n_conn; @@ -1808,10 +2318,60 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max, packed_cell_t *cell = cell_queue_pop(queue); tor_assert(*next_circ_on_conn_p(circ,conn)); + /* Calculate the exact time that this cell has spent in the queue. */ + if (get_options()->CellStatistics && !CIRCUIT_IS_ORIGIN(circ)) { + struct timeval now; + uint32_t flushed; + uint32_t cell_waiting_time; + insertion_time_queue_t *it_queue = queue->insertion_times; + tor_gettimeofday_cached(&now); + flushed = (uint32_t)((now.tv_sec % SECONDS_IN_A_DAY) * 100L + + (uint32_t)now.tv_usec / (uint32_t)10000L); + if (!it_queue || !it_queue->first) { + log_info(LD_GENERAL, "Cannot determine insertion time of cell. " + "Looks like the CellStatistics option was " + "recently enabled."); + } else { + or_circuit_t *orcirc = TO_OR_CIRCUIT(circ); + insertion_time_elem_t *elem = it_queue->first; + cell_waiting_time = + (uint32_t)((flushed * 10L + SECONDS_IN_A_DAY * 1000L - + elem->insertion_time * 10L) % + (SECONDS_IN_A_DAY * 1000L)); +#undef SECONDS_IN_A_DAY + elem->counter--; + if (elem->counter < 1) { + it_queue->first = elem->next; + if (elem == it_queue->last) + it_queue->last = NULL; + mp_pool_release(elem); + } + orcirc->total_cell_waiting_time += cell_waiting_time; + orcirc->processed_cells++; + } + } + + /* If we just flushed our queue and this circuit is used for a + * tunneled directory request, possibly advance its state. */ + if (queue->n == 0 && TO_CONN(conn)->dirreq_id) + geoip_change_dirreq_state(TO_CONN(conn)->dirreq_id, + DIRREQ_TUNNELED, + DIRREQ_CIRC_QUEUE_FLUSHED); + connection_write_to_buf(cell->body, CELL_NETWORK_SIZE, TO_CONN(conn)); - packed_cell_free(cell); + packed_cell_free_unchecked(cell); ++n_flushed; + if (cell_ewma) { + cell_ewma_t *tmp; + cell_ewma->cell_count += ewma_increment; + /* We pop and re-add the cell_ewma_t here, not above, since we need to + * re-add it immediately to keep the priority queue consistent with + * the linked-list implementation */ + tmp = pop_first_cell_ewma_from_conn(conn); + tor_assert(tmp == cell_ewma); + add_cell_ewma_to_conn(conn, cell_ewma); + } if (circ != conn->active_circuits) { /* If this happens, the current circuit just got made inactive by * a call in connection_write_to_buf(). That's nothing to worry about: @@ -1829,9 +2389,9 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max, /* Is the cell queue low enough to unblock all the streams that are waiting * to write to this circuit? */ if (streams_blocked && queue->n <= CELL_QUEUE_LOWWATER_SIZE) - set_streams_blocked_on_circ(circ, conn, 0); /* unblock streams */ + set_streams_blocked_on_circ(circ, conn, 0, 0); /* unblock streams */ - /* Did we just ran out of cells on this queue? */ + /* Did we just run out of cells on this circuit's queue? */ if (queue->n == 0) { log_debug(LD_GENERAL, "Made a circuit inactive."); make_circuit_inactive_on_conn(circ, conn); @@ -1846,10 +2406,14 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max, * transmitting in <b>direction</b>. */ void append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, - cell_t *cell, cell_direction_t direction) + cell_t *cell, cell_direction_t direction, + streamid_t fromstream) { cell_queue_t *queue; int streams_blocked; + if (circ->marked_for_close) + return; + if (direction == CELL_DIRECTION_OUT) { queue = &circ->n_conn_cells; streams_blocked = circ->streams_blocked_on_n_conn; @@ -1868,7 +2432,12 @@ append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, /* If we have too many cells on the circuit, we should stop reading from * the edge streams for a while. */ if (!streams_blocked && queue->n >= CELL_QUEUE_HIGHWATER_SIZE) - set_streams_blocked_on_circ(circ, orconn, 1); /* block streams */ + set_streams_blocked_on_circ(circ, orconn, 1, 0); /* block streams */ + + if (streams_blocked && fromstream) { + /* This edge connection is apparently not blocked; block it. */ + set_streams_blocked_on_circ(circ, orconn, 1, fromstream); + } if (queue->n == 1) { /* This was the first cell added to the queue. We need to make this @@ -1947,6 +2516,25 @@ decode_address_from_payload(tor_addr_t *addr_out, const uint8_t *payload, return payload + 2 + payload[1]; } +/** Remove all the cells queued on <b>circ</b> for <b>orconn</b>. */ +void +circuit_clear_cell_queue(circuit_t *circ, or_connection_t *orconn) +{ + cell_queue_t *queue; + if (circ->n_conn == orconn) { + queue = &circ->n_conn_cells; + } else { + or_circuit_t *orcirc = TO_OR_CIRCUIT(circ); + tor_assert(orcirc->p_conn == orconn); + queue = &orcirc->p_conn_cells; + } + + if (queue->n) + make_circuit_inactive_on_conn(circ,orconn); + + cell_queue_clear(queue); +} + /** Fail with an assert if the active circuits ring on <b>orconn</b> is * corrupt. */ void @@ -1954,16 +2542,44 @@ assert_active_circuits_ok(or_connection_t *orconn) { circuit_t *head = orconn->active_circuits; circuit_t *cur = head; + int n = 0; if (! head) return; do { circuit_t *next = *next_circ_on_conn_p(cur, orconn); circuit_t *prev = *prev_circ_on_conn_p(cur, orconn); + cell_ewma_t *ewma; tor_assert(next); tor_assert(prev); tor_assert(*next_circ_on_conn_p(prev, orconn) == cur); tor_assert(*prev_circ_on_conn_p(next, orconn) == cur); + if (orconn == cur->n_conn) { + ewma = &cur->n_cell_ewma; + tor_assert(!ewma->is_for_p_conn); + } else { + ewma = &TO_OR_CIRCUIT(cur)->p_cell_ewma; + tor_assert(ewma->is_for_p_conn); + } + tor_assert(ewma->heap_index != -1); + tor_assert(ewma == smartlist_get(orconn->active_circuit_pqueue, + ewma->heap_index)); + n++; cur = next; } while (cur != head); + + tor_assert(n == smartlist_len(orconn->active_circuit_pqueue)); +} + +/** Return 1 if we shouldn't restart reading on this circuit, even if + * we get a SENDME. Else return 0. +*/ +static int +circuit_queue_streams_are_blocked(circuit_t *circ) +{ + if (CIRCUIT_IS_ORIGIN(circ)) { + return circ->streams_blocked_on_n_conn; + } else { + return circ->streams_blocked_on_p_conn; + } } diff --git a/src/or/relay.h b/src/or/relay.h new file mode 100644 index 000000000..f64752da5 --- /dev/null +++ b/src/or/relay.h @@ -0,0 +1,70 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file relay.h + * \brief Header file for relay.c. + **/ + +#ifndef _TOR_RELAY_H +#define _TOR_RELAY_H + +extern uint64_t stats_n_relay_cells_relayed; +extern uint64_t stats_n_relay_cells_delivered; + +int circuit_receive_relay_cell(cell_t *cell, circuit_t *circ, + cell_direction_t cell_direction); + +void relay_header_pack(uint8_t *dest, const relay_header_t *src); +void relay_header_unpack(relay_header_t *dest, const uint8_t *src); +int relay_send_command_from_edge(streamid_t stream_id, circuit_t *circ, + uint8_t relay_command, const char *payload, + size_t payload_len, crypt_path_t *cpath_layer); +int connection_edge_send_command(edge_connection_t *fromconn, + uint8_t relay_command, const char *payload, + size_t payload_len); +int connection_edge_package_raw_inbuf(edge_connection_t *conn, + int package_partial, + int *max_cells); +void connection_edge_consider_sending_sendme(edge_connection_t *conn); + +extern uint64_t stats_n_data_cells_packaged; +extern uint64_t stats_n_data_bytes_packaged; +extern uint64_t stats_n_data_cells_received; +extern uint64_t stats_n_data_bytes_received; + +void init_cell_pool(void); +void free_cell_pool(void); +void clean_cell_pool(void); +void dump_cell_pool_usage(int severity); + +void cell_queue_clear(cell_queue_t *queue); +void cell_queue_append(cell_queue_t *queue, packed_cell_t *cell); +void cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell); + +void append_cell_to_circuit_queue(circuit_t *circ, or_connection_t *orconn, + cell_t *cell, cell_direction_t direction, + streamid_t fromstream); +void connection_or_unlink_all_active_circs(or_connection_t *conn); +int connection_or_flush_from_first_active_circuit(or_connection_t *conn, + int max, time_t now); +void assert_active_circuits_ok(or_connection_t *orconn); +void make_circuit_inactive_on_conn(circuit_t *circ, or_connection_t *conn); +void make_circuit_active_on_conn(circuit_t *circ, or_connection_t *conn); + +int append_address_to_payload(uint8_t *payload_out, const tor_addr_t *addr); +const uint8_t *decode_address_from_payload(tor_addr_t *addr_out, + const uint8_t *payload, + int payload_len); +unsigned cell_ewma_get_tick(void); +void cell_ewma_set_scale_factor(or_options_t *options, + networkstatus_t *consensus); +void circuit_clear_cell_queue(circuit_t *circ, or_connection_t *orconn); + +void tor_gettimeofday_cache_clear(void); + +#endif + diff --git a/src/or/rendclient.c b/src/or/rendclient.c index fb95efbdc..e4a6d09f8 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -8,6 +8,23 @@ **/ #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "connection_edge.h" +#include "directory.h" +#include "main.h" +#include "relay.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rephist.h" +#include "routerlist.h" + +static extend_info_t *rend_client_get_random_intro_impl( + const rend_cache_entry_t *rend_query, + const int strict, const int warnings); /** Called when we've established a circuit to an introduction point: * send the introduction request. */ @@ -49,6 +66,50 @@ rend_client_send_establish_rendezvous(origin_circuit_t *circ) return 0; } +/** Extend the introduction circuit <b>circ</b> to another valid + * introduction point for the hidden service it is trying to connect + * to, or mark it and launch a new circuit if we can't extend it. + * Return 0 on success. Return -1 and mark the introduction + * circuit on failure. + * + * On failure, the caller is responsible for marking the associated + * rendezvous circuit for close. */ +static int +rend_client_reextend_intro_circuit(origin_circuit_t *circ) +{ + extend_info_t *extend_info; + int result; + extend_info = rend_client_get_random_intro(circ->rend_data); + if (!extend_info) { + log_warn(LD_REND, + "No usable introduction points left for %s. Closing.", + safe_str_client(circ->rend_data->onion_address)); + circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_INTERNAL); + return -1; + } + if (circ->remaining_relay_early_cells) { + log_info(LD_REND, + "Re-extending circ %d, this time to %s.", + circ->_base.n_circ_id, extend_info->nickname); + result = circuit_extend_to_new_exit(circ, extend_info); + } else { + log_info(LD_REND, + "Building a new introduction circuit, this time to %s.", + extend_info->nickname); + circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_FINISHED); + if (!circuit_launch_by_extend_info(CIRCUIT_PURPOSE_C_INTRODUCING, + extend_info, + CIRCLAUNCH_IS_INTERNAL)) { + log_warn(LD_REND, "Building introduction circuit failed."); + result = -1; + } else { + result = 0; + } + } + extend_info_free(extend_info); + return result; +} + /** Called when we're trying to connect an ap conn; sends an INTRODUCE1 cell * down introcirc if possible. */ @@ -63,7 +124,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc, rend_cache_entry_t *entry; crypt_path_t *cpath; off_t dh_offset; - crypto_pk_env_t *intro_key; /* either Bob's public key or an intro key. */ + crypto_pk_env_t *intro_key = NULL; tor_assert(introcirc->_base.purpose == CIRCUIT_PURPOSE_C_INTRODUCING); tor_assert(rendcirc->_base.purpose == CIRCUIT_PURPOSE_C_REND_READY); @@ -77,19 +138,14 @@ rend_client_send_introduction(origin_circuit_t *introcirc, log_info(LD_REND, "query %s didn't have valid rend desc in cache. " "Refetching descriptor.", - safe_str(introcirc->rend_data->onion_address)); - /* Fetch both v0 and v2 rend descriptors in parallel. Use whichever - * arrives first. Exception: When using client authorization, only - * fetch v2 descriptors.*/ + safe_str_client(introcirc->rend_data->onion_address)); rend_client_refetch_v2_renddesc(introcirc->rend_data); - if (introcirc->rend_data->auth_type == REND_NO_AUTH) - rend_client_refetch_renddesc(introcirc->rend_data->onion_address); { connection_t *conn; while ((conn = connection_get_by_type_state_rendquery(CONN_TYPE_AP, AP_CONN_STATE_CIRCUIT_WAIT, - introcirc->rend_data->onion_address, -1))) { + introcirc->rend_data->onion_address))) { conn->state = AP_CONN_STATE_RENDDESC_WAIT; } } @@ -98,42 +154,27 @@ rend_client_send_introduction(origin_circuit_t *introcirc, } /* first 20 bytes of payload are the hash of Bob's pk */ - if (entry->parsed->version == 0) { /* un-versioned descriptor */ - intro_key = entry->parsed->pk; - } else { /* versioned descriptor */ - intro_key = NULL; - SMARTLIST_FOREACH(entry->parsed->intro_nodes, rend_intro_point_t *, - intro, { - if (!memcmp(introcirc->build_state->chosen_exit->identity_digest, - intro->extend_info->identity_digest, DIGEST_LEN)) { - intro_key = intro->intro_key; - break; - } - }); - if (!intro_key) { - /** XXX This case probably means that the intro point vanished while - * we were building a circuit to it. In the future, we should find - * out how that happened and whether we should kill the circuits to - * removed intro points immediately. See task 1073. */ - int num_intro_points = smartlist_len(entry->parsed->intro_nodes); - if (rend_cache_lookup_entry(introcirc->rend_data->onion_address, - 0, &entry) > 0) { - log_info(LD_REND, "We have both a v0 and a v2 rend desc for this " - "service. The v2 desc doesn't contain the introduction " - "point (and key) to send an INTRODUCE1/2 cell to this " - "introduction point. Assuming the introduction point " - "is for v0 rend clients and using the service key " - "from the v0 desc instead. (This is probably a bug, " - "because we shouldn't even have both a v0 and a v2 " - "descriptor for the same service.)"); - /* See flyspray task 1024. */ - intro_key = entry->parsed->pk; - } else { - log_info(LD_REND, "Internal error: could not find intro key; we " - "only have a v2 rend desc with %d intro points.", - num_intro_points); - goto perm_err; - } + intro_key = NULL; + SMARTLIST_FOREACH(entry->parsed->intro_nodes, rend_intro_point_t *, + intro, { + if (!memcmp(introcirc->build_state->chosen_exit->identity_digest, + intro->extend_info->identity_digest, DIGEST_LEN)) { + intro_key = intro->intro_key; + break; + } + }); + if (!intro_key) { + log_info(LD_REND, "Could not find intro key for %s at %s; we " + "have a v2 rend desc with %d intro points. " + "Trying a different intro point...", + safe_str_client(introcirc->rend_data->onion_address), + introcirc->build_state->chosen_exit->nickname, + smartlist_len(entry->parsed->intro_nodes)); + + if (rend_client_reextend_intro_circuit(introcirc)) { + goto perm_err; + } else { + return -1; } } if (crypto_pk_get_digest(intro_key, payload)<0) { @@ -236,8 +277,9 @@ rend_client_send_introduction(origin_circuit_t *introcirc, introcirc->_base.purpose = CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT; return 0; -perm_err: - circuit_mark_for_close(TO_CIRCUIT(introcirc), END_CIRC_REASON_INTERNAL); + perm_err: + if (!introcirc->_base.marked_for_close) + circuit_mark_for_close(TO_CIRCUIT(introcirc), END_CIRC_REASON_INTERNAL); circuit_mark_for_close(TO_CIRCUIT(rendcirc), END_CIRC_REASON_INTERNAL); return -2; } @@ -300,45 +342,16 @@ rend_client_introduction_acked(origin_circuit_t *circ, * points. If any remain, extend to a new one and try again. * If none remain, refetch the service descriptor. */ + log_info(LD_REND, "Got nack for %s from %s...", + safe_str_client(circ->rend_data->onion_address), + circ->build_state->chosen_exit->nickname); if (rend_client_remove_intro_point(circ->build_state->chosen_exit, circ->rend_data) > 0) { /* There are introduction points left. Re-extend the circuit to * another intro point and try again. */ - extend_info_t *extend_info; - int result; - extend_info = rend_client_get_random_intro(circ->rend_data); - if (!extend_info) { - log_warn(LD_REND, "No introduction points left for %s. Closing.", - escaped_safe_str(circ->rend_data->onion_address)); - circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_INTERNAL); - return -1; - } - if (circ->remaining_relay_early_cells) { - log_info(LD_REND, - "Got nack for %s from %s. Re-extending circ %d, " - "this time to %s.", - escaped_safe_str(circ->rend_data->onion_address), - circ->build_state->chosen_exit->nickname, - circ->_base.n_circ_id, extend_info->nickname); - result = circuit_extend_to_new_exit(circ, extend_info); - } else { - log_info(LD_REND, - "Got nack for %s from %s. Building a new introduction " - "circuit, this time to %s.", - escaped_safe_str(circ->rend_data->onion_address), - circ->build_state->chosen_exit->nickname, - extend_info->nickname); - circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_FINISHED); - if (!circuit_launch_by_extend_info(CIRCUIT_PURPOSE_C_INTRODUCING, - extend_info, - CIRCLAUNCH_IS_INTERNAL)) { - log_warn(LD_REND, "Building introduction circuit failed."); - result = -1; - } else { - result = 0; - } - } - extend_info_free(extend_info); + int result = rend_client_reextend_intro_circuit(circ); + /* XXXX If that call failed, should we close the rend circuit, + * too? */ return result; } } @@ -488,45 +501,21 @@ directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) rend_query->onion_address, desc_id_base32, rend_query->auth_type, (rend_query->auth_type == REND_NO_AUTH ? "[none]" : - escaped_safe_str(descriptor_cookie_base64)), + escaped_safe_str_client(descriptor_cookie_base64)), hs_dir->nickname, hs_dir->dir_port); return 1; } -/** If we are not currently fetching a rendezvous service descriptor - * for the service ID <b>query</b>, start a directory connection to fetch a - * new one. - */ -void -rend_client_refetch_renddesc(const char *query) -{ - if (!get_options()->FetchHidServDescriptors) - return; - log_info(LD_REND, "Fetching rendezvous descriptor for service %s", - escaped_safe_str(query)); - if (connection_get_by_type_state_rendquery(CONN_TYPE_DIR, 0, query, 0)) { - log_info(LD_REND,"Would fetch a new renddesc here (for %s), but one is " - "already in progress.", escaped_safe_str(query)); - } else { - /* not one already; initiate a dir rend desc lookup */ - directory_get_from_dirserver(DIR_PURPOSE_FETCH_RENDDESC, - ROUTER_PURPOSE_GENERAL, query, - PDS_RETRY_IF_NO_SERVERS); - } -} - -/** Start a connection to a hidden service directory to fetch a v2 - * rendezvous service descriptor for the base32-encoded service ID - * <b>query</b>. - */ +/** Unless we already have a descriptor for <b>rend_query</b> with at least + * one (possibly) working introduction point in it, start a connection to a + * hidden service directory to fetch a v2 rendezvous service descriptor. */ void rend_client_refetch_v2_renddesc(const rend_data_t *rend_query) { char descriptor_id[DIGEST_LEN]; int replicas_left_to_try[REND_NUMBER_OF_NON_CONSECUTIVE_REPLICAS]; - int i, tries_left, r; + int i, tries_left; rend_cache_entry_t *e = NULL; - time_t now = time(NULL); tor_assert(rend_query); /* Are we configured to fetch descriptors? */ if (!get_options()->FetchHidServDescriptors) { @@ -535,15 +524,13 @@ rend_client_refetch_v2_renddesc(const rend_data_t *rend_query) return; } /* Before fetching, check if we already have the descriptor here. */ - r = rend_cache_lookup_entry(rend_query->onion_address, -1, &e); - if (r > 0 && now - e->received < NUM_SECONDS_BEFORE_HS_REFETCH) { + if (rend_cache_lookup_entry(rend_query->onion_address, -1, &e) > 0) { log_info(LD_REND, "We would fetch a v2 rendezvous descriptor, but we " - "already have a fresh copy of that descriptor here. " - "Not fetching."); + "already have that descriptor here. Not fetching."); return; } log_debug(LD_REND, "Fetching v2 rendezvous descriptor for service %s", - safe_str(rend_query->onion_address)); + safe_str_client(rend_query->onion_address)); /* Randomly iterate over the replicas until a descriptor can be fetched * from one of the consecutive nodes, or no options are left. */ tries_left = REND_NUMBER_OF_NON_CONSECUTIVE_REPLICAS; @@ -569,8 +556,8 @@ rend_client_refetch_v2_renddesc(const rend_data_t *rend_query) log_info(LD_REND, "Could not pick one of the responsible hidden " "service directories to fetch descriptors, because " "we already tried them all unsuccessfully."); - /* Close pending connections (unless a v0 request is still going on). */ - rend_client_desc_trynow(rend_query->onion_address, 2); + /* Close pending connections. */ + rend_client_desc_trynow(rend_query->onion_address); return; } @@ -600,9 +587,8 @@ rend_client_cancel_descriptor_fetches(void) "Marking for close dir conn fetching rendezvous " "descriptor for unknown service!"); } else { - log_debug(LD_REND, "Marking for close dir conn fetching v%d " + log_debug(LD_REND, "Marking for close dir conn fetching " "rendezvous descriptor for service %s", - (int)(rd->rend_desc_version), safe_str(rd->onion_address)); } connection_mark_for_close(conn); @@ -628,18 +614,13 @@ rend_client_remove_intro_point(extend_info_t *failed_intro, r = rend_cache_lookup_entry(rend_query->onion_address, -1, &ent); if (r<0) { log_warn(LD_BUG, "Malformed service ID %s.", - escaped_safe_str(rend_query->onion_address)); + escaped_safe_str_client(rend_query->onion_address)); return -1; } if (r==0) { log_info(LD_REND, "Unknown service %s. Re-fetching descriptor.", - escaped_safe_str(rend_query->onion_address)); - /* Fetch both, v0 and v2 rend descriptors in parallel. Use whichever - * arrives first. Exception: When using client authorization, only - * fetch v2 descriptors.*/ + escaped_safe_str_client(rend_query->onion_address)); rend_client_refetch_v2_renddesc(rend_query); - if (rend_query->auth_type == REND_NO_AUTH) - rend_client_refetch_renddesc(rend_query->onion_address); return 0; } @@ -653,21 +634,16 @@ rend_client_remove_intro_point(extend_info_t *failed_intro, } } - if (smartlist_len(ent->parsed->intro_nodes) == 0) { + if (! rend_client_any_intro_points_usable(ent)) { log_info(LD_REND, "No more intro points remain for %s. Re-fetching descriptor.", - escaped_safe_str(rend_query->onion_address)); - /* Fetch both, v0 and v2 rend descriptors in parallel. Use whichever - * arrives first. Exception: When using client authorization, only - * fetch v2 descriptors.*/ + escaped_safe_str_client(rend_query->onion_address)); rend_client_refetch_v2_renddesc(rend_query); - if (rend_query->auth_type == REND_NO_AUTH) - rend_client_refetch_renddesc(rend_query->onion_address); /* move all pending streams back to renddesc_wait */ while ((conn = connection_get_by_type_state_rendquery(CONN_TYPE_AP, AP_CONN_STATE_CIRCUIT_WAIT, - rend_query->onion_address, -1))) { + rend_query->onion_address))) { conn->state = AP_CONN_STATE_RENDDESC_WAIT; } @@ -675,7 +651,7 @@ rend_client_remove_intro_point(extend_info_t *failed_intro, } log_info(LD_REND,"%d options left for %s.", smartlist_len(ent->parsed->intro_nodes), - escaped_safe_str(rend_query->onion_address)); + escaped_safe_str_client(rend_query->onion_address)); return 1; } @@ -698,7 +674,7 @@ rend_client_rendezvous_acked(origin_circuit_t *circ, const uint8_t *request, log_info(LD_REND,"Got rendezvous ack. This circuit is now ready for " "rendezvous."); circ->_base.purpose = CIRCUIT_PURPOSE_C_REND_READY; - /* XXXX022 This is a pretty brute-force approach. It'd be better to + /* XXXX023 This is a pretty brute-force approach. It'd be better to * attach only the connections that are waiting on this circuit, rather * than trying to attach them all. See comments bug 743. */ /* If we already have the introduction circuit built, make sure we send @@ -737,7 +713,8 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const uint8_t *request, tor_assert(circ->build_state->pending_final_cpath); hop = circ->build_state->pending_final_cpath; tor_assert(hop->dh_handshake_state); - if (crypto_dh_compute_secret(hop->dh_handshake_state, (char*)request, + if (crypto_dh_compute_secret(LOG_PROTOCOL_WARN, + hop->dh_handshake_state, (char*)request, DH_KEY_LEN, keys, DIGEST_LEN+CPATH_KEY_MATERIAL_LEN)<0) { log_warn(LD_GENERAL, "Couldn't complete DH handshake."); @@ -767,7 +744,7 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const uint8_t *request, onion_append_to_cpath(&circ->cpath, hop); circ->build_state->pending_final_cpath = NULL; /* prevent double-free */ - /* XXXX022 This is a pretty brute-force approach. It'd be better to + /* XXXX023 This is a pretty brute-force approach. It'd be better to * attach only the connections that are waiting on this circuit, rather * than trying to attach them all. See comments bug 743. */ connection_ap_attach_pending(); @@ -779,24 +756,18 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const uint8_t *request, return -1; } -/** Find all the apconns in state AP_CONN_STATE_RENDDESC_WAIT that - * are waiting on query. If there's a working cache entry here - * with at least one intro point, move them to the next state. If - * <b>rend_version</b> is non-negative, fail connections that have - * requested <b>query</b> unless there are still descriptor fetch - * requests in progress for other descriptor versions than - * <b>rend_version</b>. - */ +/** Find all the apconns in state AP_CONN_STATE_RENDDESC_WAIT that are + * waiting on <b>query</b>. If there's a working cache entry here with at + * least one intro point, move them to the next state. */ void -rend_client_desc_trynow(const char *query, int rend_version) +rend_client_desc_trynow(const char *query) { edge_connection_t *conn; rend_cache_entry_t *entry; time_t now = time(NULL); smartlist_t *conns = get_connection_array(); - SMARTLIST_FOREACH(conns, connection_t *, _conn, - { + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, _conn) { if (_conn->type != CONN_TYPE_AP || _conn->state != AP_CONN_STATE_RENDDESC_WAIT || _conn->marked_for_close) @@ -809,7 +780,7 @@ rend_client_desc_trynow(const char *query, int rend_version) assert_connection_ok(TO_CONN(conn), now); if (rend_cache_lookup_entry(conn->rend_data->onion_address, -1, &entry) == 1 && - smartlist_len(entry->parsed->intro_nodes) > 0) { + rend_client_any_intro_points_usable(entry)) { /* either this fetch worked, or it failed but there was a * valid entry from before which we should reuse */ log_info(LD_REND,"Rend desc is usable. Launching circuits."); @@ -828,17 +799,12 @@ rend_client_desc_trynow(const char *query, int rend_version) connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH); } } else { /* 404, or fetch didn't get that far */ - /* Unless there are requests for another descriptor version pending, - * close the connection. */ - if (rend_version >= 0 && - !connection_get_by_type_state_rendquery(CONN_TYPE_DIR, 0, query, - rend_version == 0 ? 2 : 0)) { - log_notice(LD_REND,"Closing stream for '%s.onion': hidden service is " - "unavailable (try again later).", safe_str(query)); - connection_mark_unattached_ap(conn, END_STREAM_REASON_RESOLVEFAILED); - } + log_notice(LD_REND,"Closing stream for '%s.onion': hidden service is " + "unavailable (try again later).", + safe_str_client(query)); + connection_mark_unattached_ap(conn, END_STREAM_REASON_RESOLVEFAILED); } - }); + } SMARTLIST_FOREACH_END(_conn); } /** Return a newly allocated extend_info_t* for a randomly chosen introduction @@ -848,24 +814,63 @@ rend_client_desc_trynow(const char *query, int rend_version) extend_info_t * rend_client_get_random_intro(const rend_data_t *rend_query) { - int i; + extend_info_t *result; rend_cache_entry_t *entry; - rend_intro_point_t *intro; - routerinfo_t *router; if (rend_cache_lookup_entry(rend_query->onion_address, -1, &entry) < 1) { - log_warn(LD_REND, - "Query '%s' didn't have valid rend desc in cache. Failing.", - safe_str(rend_query->onion_address)); + log_warn(LD_REND, + "Query '%s' didn't have valid rend desc in cache. Failing.", + safe_str_client(rend_query->onion_address)); return NULL; } + /* See if we can get a node that complies with ExcludeNodes */ + if ((result = rend_client_get_random_intro_impl(entry, 1, 1))) + return result; + /* If not, and StrictNodes is not set, see if we can return any old node + */ + if (!get_options()->StrictNodes) + return rend_client_get_random_intro_impl(entry, 0, 1); + return NULL; +} + +/** As rend_client_get_random_intro, except assume that StrictNodes is set + * iff <b>strict</b> is true. If <b>warnings</b> is false, don't complain + * to the user when we're out of nodes, even if StrictNodes is true. + */ +static extend_info_t * +rend_client_get_random_intro_impl(const rend_cache_entry_t *entry, + const int strict, + const int warnings) +{ + int i; + + rend_intro_point_t *intro; + routerinfo_t *router; + or_options_t *options = get_options(); + smartlist_t *usable_nodes; + int n_excluded = 0; + + /* We'll keep a separate list of the usable nodes. If this becomes empty, + * no nodes are usable. */ + usable_nodes = smartlist_create(); + smartlist_add_all(usable_nodes, entry->parsed->intro_nodes); + again: - if (smartlist_len(entry->parsed->intro_nodes) == 0) + if (smartlist_len(usable_nodes) == 0) { + if (n_excluded && get_options()->StrictNodes && warnings) { + /* We only want to warn if StrictNodes is really set. Otherwise + * we're just about to retry anyways. + */ + log_warn(LD_REND, "All introduction points for hidden service are " + "at excluded relays, and StrictNodes is set. Skipping."); + } + smartlist_free(usable_nodes); return NULL; + } - i = crypto_rand_int(smartlist_len(entry->parsed->intro_nodes)); - intro = smartlist_get(entry->parsed->intro_nodes, i); + i = crypto_rand_int(smartlist_len(usable_nodes)); + intro = smartlist_get(usable_nodes, i); /* Do we need to look up the router or is the extend info complete? */ if (!intro->extend_info->onion_key) { if (tor_digest_is_zero(intro->extend_info->identity_digest)) @@ -875,16 +880,34 @@ rend_client_get_random_intro(const rend_data_t *rend_query) if (!router) { log_info(LD_REND, "Unknown router with nickname '%s'; trying another.", intro->extend_info->nickname); - rend_intro_point_free(intro); - smartlist_del(entry->parsed->intro_nodes, i); + smartlist_del(usable_nodes, i); goto again; } extend_info_free(intro->extend_info); intro->extend_info = extend_info_from_router(router); } + /* Check if we should refuse to talk to this router. */ + if (options->ExcludeNodes && strict && + routerset_contains_extendinfo(options->ExcludeNodes, + intro->extend_info)) { + n_excluded++; + smartlist_del(usable_nodes, i); + goto again; + } + + smartlist_free(usable_nodes); return extend_info_dup(intro->extend_info); } +/** Return true iff any introduction points still listed in <b>entry</b> are + * usable. */ +int +rend_client_any_intro_points_usable(const rend_cache_entry_t *entry) +{ + return rend_client_get_random_intro_impl( + entry, get_options()->StrictNodes, 0) != NULL; +} + /** Client-side authorizations for hidden services; map of onion address to * rend_service_authorization_t*. */ static strmap_t *auth_hid_servs = NULL; @@ -1009,8 +1032,7 @@ rend_parse_service_authorization(or_options_t *options, int validate_only) err: res = -1; done: - if (auth) - rend_service_authorization_free(auth); + rend_service_authorization_free(auth); SMARTLIST_FOREACH(sl, char *, c, tor_free(c);); smartlist_free(sl); if (!validate_only && res == 0) { diff --git a/src/or/rendclient.h b/src/or/rendclient.h new file mode 100644 index 000000000..6910c1a97 --- /dev/null +++ b/src/or/rendclient.h @@ -0,0 +1,45 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file rendclient.h + * \brief Header file for rendclient.c. + **/ + +#ifndef _TOR_RENDCLIENT_H +#define _TOR_RENDCLIENT_H + +void rend_client_introcirc_has_opened(origin_circuit_t *circ); +void rend_client_rendcirc_has_opened(origin_circuit_t *circ); +int rend_client_introduction_acked(origin_circuit_t *circ, + const uint8_t *request, + size_t request_len); +void rend_client_refetch_v2_renddesc(const rend_data_t *rend_query); +void rend_client_cancel_descriptor_fetches(void); +int rend_client_remove_intro_point(extend_info_t *failed_intro, + const rend_data_t *rend_query); +int rend_client_rendezvous_acked(origin_circuit_t *circ, + const uint8_t *request, + size_t request_len); +int rend_client_receive_rendezvous(origin_circuit_t *circ, + const uint8_t *request, + size_t request_len); +void rend_client_desc_trynow(const char *query); + +extend_info_t *rend_client_get_random_intro(const rend_data_t *rend_query); +int rend_client_any_intro_points_usable(const rend_cache_entry_t *entry); + +int rend_client_send_introduction(origin_circuit_t *introcirc, + origin_circuit_t *rendcirc); +int rend_parse_service_authorization(or_options_t *options, + int validate_only); +rend_service_authorization_t *rend_client_lookup_service_authorization( + const char *onion_address); +void rend_service_authorization_free_all(void); +rend_data_t *rend_data_dup(const rend_data_t *request); + +#endif + diff --git a/src/or/rendcommon.c b/src/or/rendcommon.c index ba28ccabd..f8e7f9bbb 100644 --- a/src/or/rendcommon.c +++ b/src/or/rendcommon.c @@ -9,6 +9,15 @@ **/ #include "or.h" +#include "circuitbuild.h" +#include "config.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rendmid.h" +#include "rendservice.h" +#include "rephist.h" +#include "routerlist.h" +#include "routerparse.h" /** Return 0 if one and two are the same service ids, else -1 or 1 */ int @@ -22,6 +31,8 @@ rend_cmp_service_ids(const char *one, const char *two) void rend_service_descriptor_free(rend_service_descriptor_t *desc) { + if (!desc) + return; if (desc->pk) crypto_free_pk_env(desc->pk); if (desc->intro_nodes) { @@ -125,7 +136,8 @@ rend_compute_v2_desc_id(char *desc_id_out, const char *service_id, if (!service_id || strlen(service_id) != REND_SERVICE_ID_LEN_BASE32) { log_warn(LD_REND, "Could not compute v2 descriptor ID: " - "Illegal service ID: %s", safe_str(service_id)); + "Illegal service ID: %s", + safe_str(service_id)); return -1; } if (replica >= REND_NUMBER_OF_NON_CONSECUTIVE_REPLICAS) { @@ -138,7 +150,7 @@ rend_compute_v2_desc_id(char *desc_id_out, const char *service_id, service_id, REND_SERVICE_ID_LEN_BASE32) < 0) { log_warn(LD_REND, "Could not compute v2 descriptor ID: " "Illegal characters in service ID: %s", - safe_str(service_id)); + safe_str_client(service_id)); return -1; } /* Calculate current time-period. */ @@ -403,8 +415,7 @@ rend_desc_v2_is_parsable(rend_encoded_v2_service_descriptor_t *desc) &test_intro_size, &test_encoded_size, &test_next, desc->desc_str); - if (test_parsed) - rend_service_descriptor_free(test_parsed); + rend_service_descriptor_free(test_parsed); tor_free(test_intro_content); return (res >= 0); } @@ -414,6 +425,8 @@ void rend_encoded_v2_service_descriptor_free( rend_encoded_v2_service_descriptor_t *desc) { + if (!desc) + return; tor_free(desc->desc_str); tor_free(desc); } @@ -422,10 +435,11 @@ rend_encoded_v2_service_descriptor_free( void rend_intro_point_free(rend_intro_point_t *intro) { - if (intro->extend_info) - extend_info_free(intro->extend_info); - if (intro->intro_key) - crypto_free_pk_env(intro->intro_key); + if (!intro) + return; + + extend_info_free(intro->extend_info); + crypto_free_pk_env(intro->intro_key); tor_free(intro); } @@ -618,7 +632,8 @@ rend_encode_v2_descriptors(smartlist_t *descs_out, } if (router_append_dirobj_signature(desc_str + written, desc_len - written, - desc_digest, service_key) < 0) { + desc_digest, DIGEST_LEN, + service_key) < 0) { log_warn(LD_BUG, "Couldn't sign desc."); rend_encoded_v2_service_descriptor_free(enc); goto err; @@ -655,63 +670,6 @@ rend_encode_v2_descriptors(smartlist_t *descs_out, return seconds_valid; } -/** Encode a service descriptor for <b>desc</b>, and sign it with - * <b>key</b>. Store the descriptor in *<b>str_out</b>, and set - * *<b>len_out</b> to its length. - */ -int -rend_encode_service_descriptor(rend_service_descriptor_t *desc, - crypto_pk_env_t *key, - char **str_out, size_t *len_out) -{ - char *cp; - char *end; - int i, r; - size_t asn1len; - size_t buflen = - PK_BYTES*2*(smartlist_len(desc->intro_nodes)+2);/*Too long, but ok*/ - cp = *str_out = tor_malloc(buflen); - end = cp + PK_BYTES*2*(smartlist_len(desc->intro_nodes)+1); - r = crypto_pk_asn1_encode(desc->pk, cp+2, end-(cp+2)); - if (r < 0) { - tor_free(*str_out); - return -1; - } - asn1len = r; - set_uint16(cp, htons((uint16_t)asn1len)); - cp += 2+asn1len; - set_uint32(cp, htonl((uint32_t)desc->timestamp)); - cp += 4; - set_uint16(cp, htons((uint16_t)smartlist_len(desc->intro_nodes))); - cp += 2; - for (i=0; i < smartlist_len(desc->intro_nodes); ++i) { - rend_intro_point_t *intro = smartlist_get(desc->intro_nodes, i); - char ipoint[HEX_DIGEST_LEN+2]; - const size_t ipoint_len = HEX_DIGEST_LEN+1; - ipoint[0] = '$'; - base16_encode(ipoint+1, HEX_DIGEST_LEN+1, - intro->extend_info->identity_digest, - DIGEST_LEN); - tor_assert(strlen(ipoint) == ipoint_len); - /* Assert that appending ipoint and its NUL won't over overrun the - * buffer. */ - tor_assert(cp + ipoint_len+1 < *str_out + buflen); - memcpy(cp, ipoint, ipoint_len+1); - cp += ipoint_len+1; - } - note_crypto_pk_op(REND_SERVER); - r = crypto_pk_private_sign_digest(key, - cp, buflen - (cp - *str_out), - *str_out, cp-*str_out); - if (r<0) { - tor_free(*str_out); - return -1; - } - cp += r; - *len_out = (size_t)(cp-*str_out); - return 0; -} - /** Parse a service descriptor at <b>str</b> (<b>len</b> bytes). On * success, return a newly alloced service_descriptor_t. On failure, * return NULL. @@ -828,22 +786,27 @@ rend_cache_init(void) /** Helper: free storage held by a single service descriptor cache entry. */ static void -_rend_cache_entry_free(void *p) +rend_cache_entry_free(rend_cache_entry_t *e) { - rend_cache_entry_t *e = p; + if (!e) + return; rend_service_descriptor_free(e->parsed); tor_free(e->desc); tor_free(e); } +static void +_rend_cache_entry_free(void *p) +{ + rend_cache_entry_free(p); +} + /** Free all storage held by the service descriptor cache. */ void rend_cache_free_all(void) { - if (rend_cache) - strmap_free(rend_cache, _rend_cache_entry_free); - if (rend_cache_v2_dir) - digestmap_free(rend_cache_v2_dir, _rend_cache_entry_free); + strmap_free(rend_cache, _rend_cache_entry_free); + digestmap_free(rend_cache_v2_dir, _rend_cache_entry_free); rend_cache = NULL; rend_cache_v2_dir = NULL; } @@ -864,7 +827,7 @@ rend_cache_clean(void) ent = (rend_cache_entry_t*)val; if (ent->parsed->timestamp < cutoff) { iter = strmap_iter_next_rmv(rend_cache, iter); - _rend_cache_entry_free(ent); + rend_cache_entry_free(ent); } else { iter = strmap_iter_next(rend_cache, iter); } @@ -900,9 +863,9 @@ rend_cache_clean_v2_descs_as_dir(void) char key_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; base32_encode(key_base32, sizeof(key_base32), key, DIGEST_LEN); log_info(LD_REND, "Removing descriptor with ID '%s' from cache", - safe_str(key_base32)); + safe_str_client(key_base32)); iter = digestmap_iter_next_rmv(rend_cache_v2_dir, iter); - _rend_cache_entry_free(ent); + rend_cache_entry_free(ent); } else { iter = digestmap_iter_next(rend_cache_v2_dir, iter); } @@ -978,6 +941,11 @@ rend_cache_lookup_entry(const char *query, int version, rend_cache_entry_t **e) } if (!*e) return 0; + tor_assert((*e)->parsed && (*e)->parsed->intro_nodes); + /* XXX023 hack for now, to return "not found" if there are no intro + * points remaining. See bug 997. */ + if (! rend_client_any_intro_points_usable(*e)) + return 0; return 1; } @@ -1056,7 +1024,6 @@ rend_cache_store(const char *desc, size_t desc_len, int published) char query[REND_SERVICE_ID_LEN_BASE32+1]; char key[REND_SERVICE_ID_LEN_BASE32+2]; /* 0<query>\0 */ time_t now; - or_options_t *options = get_options(); tor_assert(rend_cache); parsed = rend_parse_service_descriptor(desc,desc_len); if (!parsed) { @@ -1071,13 +1038,15 @@ rend_cache_store(const char *desc, size_t desc_len, int published) now = time(NULL); if (parsed->timestamp < now-REND_CACHE_MAX_AGE-REND_CACHE_MAX_SKEW) { log_fn(LOG_PROTOCOL_WARN, LD_REND, - "Service descriptor %s is too old.", safe_str(query)); + "Service descriptor %s is too old.", + safe_str_client(query)); rend_service_descriptor_free(parsed); return -2; } if (parsed->timestamp > now+REND_CACHE_MAX_SKEW) { log_fn(LOG_PROTOCOL_WARN, LD_REND, - "Service descriptor %s is too far in the future.", safe_str(query)); + "Service descriptor %s is too far in the future.", + safe_str_client(query)); rend_service_descriptor_free(parsed); return -2; } @@ -1085,25 +1054,22 @@ rend_cache_store(const char *desc, size_t desc_len, int published) tor_snprintf(key, sizeof(key), "2%s", query); if (!published && strmap_get_lc(rend_cache, key)) { log_info(LD_REND, "We already have a v2 descriptor for service %s.", - safe_str(query)); + safe_str_client(query)); rend_service_descriptor_free(parsed); return -1; } - /* report novel publication to statistics */ - if (published && options->HSAuthorityRecordStats) { - hs_usage_note_publish_total(query, now); - } tor_snprintf(key, sizeof(key), "0%s", query); e = (rend_cache_entry_t*) strmap_get_lc(rend_cache, key); if (e && e->parsed->timestamp > parsed->timestamp) { log_info(LD_REND,"We already have a newer service descriptor %s with the " - "same ID and version.", safe_str(query)); + "same ID and version.", + safe_str_client(query)); rend_service_descriptor_free(parsed); return 0; } if (e && e->len == desc_len && !memcmp(desc,e->desc,desc_len)) { log_info(LD_REND,"We already have this service descriptor %s.", - safe_str(query)); + safe_str_client(query)); e->received = time(NULL); rend_service_descriptor_free(parsed); return 0; @@ -1111,10 +1077,6 @@ rend_cache_store(const char *desc, size_t desc_len, int published) if (!e) { e = tor_malloc_zero(sizeof(rend_cache_entry_t)); strmap_set_lc(rend_cache, key, e); - /* report novel publication to statistics */ - if (published && options->HSAuthorityRecordStats) { - hs_usage_note_publish_novel(query, now); - } } else { rend_service_descriptor_free(e->parsed); tor_free(e->desc); @@ -1126,7 +1088,7 @@ rend_cache_store(const char *desc, size_t desc_len, int published) memcpy(e->desc, desc, desc_len); log_debug(LD_REND,"Successfully stored rend desc '%s', len %d.", - safe_str(query), (int)desc_len); + safe_str_client(query), (int)desc_len); return 1; } @@ -1177,7 +1139,7 @@ rend_cache_store_v2_desc_as_dir(const char *desc) if (!hid_serv_responsible_for_desc_id(desc_id)) { log_info(LD_REND, "Service descriptor with desc ID %s is not in " "interval that we are responsible for.", - safe_str(desc_id_base32)); + safe_str_client(desc_id_base32)); goto skip; } /* Is descriptor too old? */ @@ -1306,7 +1268,8 @@ rend_cache_store_v2_desc_as_client(const char *desc, /* Decode/decrypt introduction points. */ if (intro_content) { if (rend_query->auth_type != REND_NO_AUTH && - rend_query->descriptor_cookie) { + !tor_mem_is_zero(rend_query->descriptor_cookie, + sizeof(rend_query->descriptor_cookie))) { char *ipos_decrypted = NULL; size_t ipos_decrypted_size; if (rend_decrypt_introduction_points(&ipos_decrypted, @@ -1341,14 +1304,14 @@ rend_cache_store_v2_desc_as_client(const char *desc, /* Is descriptor too old? */ if (parsed->timestamp < now - REND_CACHE_MAX_AGE-REND_CACHE_MAX_SKEW) { log_warn(LD_REND, "Service descriptor with service ID %s is too old.", - safe_str(service_id)); + safe_str_client(service_id)); retval = -2; goto err; } /* Is descriptor too far in the future? */ if (parsed->timestamp > now + REND_CACHE_MAX_SKEW) { log_warn(LD_REND, "Service descriptor with service ID %s is too far in " - "the future.", safe_str(service_id)); + "the future.", safe_str_client(service_id)); retval = -2; goto err; } @@ -1356,7 +1319,7 @@ rend_cache_store_v2_desc_as_client(const char *desc, tor_snprintf(key, sizeof(key), "0%s", service_id); if (strmap_get_lc(rend_cache, key)) { log_info(LD_REND, "We already have a v0 descriptor for service ID %s.", - safe_str(service_id)); + safe_str_client(service_id)); retval = -1; goto err; } @@ -1366,14 +1329,14 @@ rend_cache_store_v2_desc_as_client(const char *desc, if (e && e->parsed->timestamp > parsed->timestamp) { log_info(LD_REND, "We already have a newer service descriptor for " "service ID %s with the same desc ID and version.", - safe_str(service_id)); + safe_str_client(service_id)); retval = 0; goto err; } /* Do we already have this descriptor? */ if (e && !strcmp(desc, e->desc)) { log_info(LD_REND,"We already have this service descriptor %s.", - safe_str(service_id)); + safe_str_client(service_id)); e->received = time(NULL); retval = 0; goto err; @@ -1391,12 +1354,11 @@ rend_cache_store_v2_desc_as_client(const char *desc, strlcpy(e->desc, desc, encoded_size + 1); e->len = encoded_size; log_debug(LD_REND,"Successfully stored rend desc '%s', len %d.", - safe_str(service_id), (int)encoded_size); + safe_str_client(service_id), (int)encoded_size); return 1; err: - if (parsed) - rend_service_descriptor_free(parsed); + rend_service_descriptor_free(parsed); tor_free(intro_content); return retval; } diff --git a/src/or/rendcommon.h b/src/or/rendcommon.h new file mode 100644 index 000000000..44b5227cf --- /dev/null +++ b/src/or/rendcommon.h @@ -0,0 +1,66 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file rendcommon.h + * \brief Header file for rendcommon.c. + **/ + +#ifndef _TOR_RENDCOMMON_H +#define _TOR_RENDCOMMON_H + +/** Free all storage associated with <b>data</b> */ +static INLINE void +rend_data_free(rend_data_t *data) +{ + tor_free(data); +} + +int rend_cmp_service_ids(const char *one, const char *two); + +void rend_process_relay_cell(circuit_t *circ, const crypt_path_t *layer_hint, + int command, size_t length, + const uint8_t *payload); + +void rend_service_descriptor_free(rend_service_descriptor_t *desc); +rend_service_descriptor_t *rend_parse_service_descriptor(const char *str, + size_t len); +int rend_get_service_id(crypto_pk_env_t *pk, char *out); +void rend_encoded_v2_service_descriptor_free( + rend_encoded_v2_service_descriptor_t *desc); +void rend_intro_point_free(rend_intro_point_t *intro); + +void rend_cache_init(void); +void rend_cache_clean(void); +void rend_cache_clean_v2_descs_as_dir(void); +void rend_cache_purge(void); +void rend_cache_free_all(void); +int rend_valid_service_id(const char *query); +int rend_cache_lookup_desc(const char *query, int version, const char **desc, + size_t *desc_len); +int rend_cache_lookup_entry(const char *query, int version, + rend_cache_entry_t **entry_out); +int rend_cache_lookup_v2_desc_as_dir(const char *query, const char **desc); +int rend_cache_store(const char *desc, size_t desc_len, int published); +int rend_cache_store_v2_desc_as_client(const char *desc, + const rend_data_t *rend_query); +int rend_cache_store_v2_desc_as_dir(const char *desc); +int rend_cache_size(void); +int rend_encode_v2_descriptors(smartlist_t *descs_out, + rend_service_descriptor_t *desc, time_t now, + uint8_t period, rend_auth_type_t auth_type, + crypto_pk_env_t *client_key, + smartlist_t *client_cookies); +int rend_compute_v2_desc_id(char *desc_id_out, const char *service_id, + const char *descriptor_cookie, + time_t now, uint8_t replica); +int rend_id_is_in_interval(const char *a, const char *b, const char *c); +void rend_get_descriptor_id_bytes(char *descriptor_id_out, + const char *service_id, + const char *secret_id_part); + +#endif + diff --git a/src/or/rendmid.c b/src/or/rendmid.c index 064f1be2e..5e2dd986b 100644 --- a/src/or/rendmid.c +++ b/src/or/rendmid.c @@ -8,6 +8,11 @@ **/ #include "or.h" +#include "circuitlist.h" +#include "config.h" +#include "relay.h" +#include "rendmid.h" +#include "rephist.h" /** Respond to an ESTABLISH_INTRO cell by checking the signed data and * setting the circuit's purpose and service pk digest. diff --git a/src/or/rendmid.h b/src/or/rendmid.h new file mode 100644 index 000000000..5ed87fd2b --- /dev/null +++ b/src/or/rendmid.h @@ -0,0 +1,25 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file rendmid.h + * \brief Header file for rendmid.c. + **/ + +#ifndef _TOR_RENDMID_H +#define _TOR_RENDMID_H + +int rend_mid_establish_intro(or_circuit_t *circ, const uint8_t *request, + size_t request_len); +int rend_mid_introduce(or_circuit_t *circ, const uint8_t *request, + size_t request_len); +int rend_mid_establish_rendezvous(or_circuit_t *circ, const uint8_t *request, + size_t request_len); +int rend_mid_rendezvous(or_circuit_t *circ, const uint8_t *request, + size_t request_len); + +#endif + diff --git a/src/or/rendservice.c b/src/or/rendservice.c index 33e8d3e7e..cd8f9eabe 100644 --- a/src/or/rendservice.c +++ b/src/or/rendservice.c @@ -8,10 +8,23 @@ **/ #include "or.h" +#include "circuitbuild.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "directory.h" +#include "networkstatus.h" +#include "rendclient.h" +#include "rendcommon.h" +#include "rendservice.h" +#include "router.h" +#include "relay.h" +#include "rephist.h" +#include "routerlist.h" +#include "routerparse.h" static origin_circuit_t *find_intro_circuit(rend_intro_point_t *intro, - const char *pk_digest, - int desc_version); + const char *pk_digest); /** Represents the mapping from a virtual port of a rendezvous service to * a real port on some IP. @@ -42,8 +55,6 @@ typedef struct rend_service_t { /* Fields specified in config file */ char *directory; /**< where in the filesystem it stores it */ smartlist_t *ports; /**< List of rend_service_port_config_t */ - int descriptor_version; /**< Rendezvous descriptor version that will be - * published. */ rend_auth_type_t auth_type; /**< Client authorization type or 0 if no client * authorization is performed. */ smartlist_t *clients; /**< List of rend_authorized_client_t's of @@ -58,7 +69,7 @@ typedef struct rend_service_t { * or are trying to establish. */ time_t intro_period_started; /**< Start of the current period to build * introduction points. */ - int n_intro_circuits_launched; /**< count of intro circuits we have + int n_intro_circuits_launched; /**< Count of intro circuits we have * established in this period. */ rend_service_descriptor_t *desc; /**< Current hidden service descriptor. */ time_t desc_is_dirty; /**< Time at which changes to the hidden service @@ -90,7 +101,8 @@ num_rend_services(void) static void rend_authorized_client_free(rend_authorized_client_t *client) { - if (!client) return; + if (!client) + return; if (client->client_key) crypto_free_pk_env(client->client_key); tor_free(client->client_name); @@ -109,7 +121,9 @@ rend_authorized_client_strmap_item_free(void *authorized_client) static void rend_service_free(rend_service_t *service) { - if (!service) return; + if (!service) + return; + tor_free(service->directory); SMARTLIST_FOREACH(service->ports, void*, p, tor_free(p)); smartlist_free(service->ports); @@ -120,15 +134,14 @@ rend_service_free(rend_service_t *service) rend_intro_point_free(intro);); smartlist_free(service->intro_nodes); } - if (service->desc) - rend_service_descriptor_free(service->desc); + + rend_service_descriptor_free(service->desc); if (service->clients) { SMARTLIST_FOREACH(service->clients, rend_authorized_client_t *, c, rend_authorized_client_free(c);); smartlist_free(service->clients); } - if (service->accepted_intros) - digestmap_free(service->accepted_intros, _tor_free); + digestmap_free(service->accepted_intros, _tor_free); tor_free(service); } @@ -137,9 +150,9 @@ rend_service_free(rend_service_t *service) void rend_service_free_all(void) { - if (!rend_service_list) { + if (!rend_service_list) return; - } + SMARTLIST_FOREACH(rend_service_list, rend_service_t*, ptr, rend_service_free(ptr)); smartlist_free(rend_service_list); @@ -156,36 +169,6 @@ rend_add_service(rend_service_t *service) service->intro_nodes = smartlist_create(); - /* If the service is configured to publish unversioned (v0) and versioned - * descriptors (v2 or higher), split it up into two separate services - * (unless it is configured to perform client authorization). */ - if (service->descriptor_version == -1) { - if (service->auth_type == REND_NO_AUTH) { - rend_service_t *v0_service = tor_malloc_zero(sizeof(rend_service_t)); - v0_service->directory = tor_strdup(service->directory); - v0_service->ports = smartlist_create(); - SMARTLIST_FOREACH(service->ports, rend_service_port_config_t *, p, { - rend_service_port_config_t *copy = - tor_malloc_zero(sizeof(rend_service_port_config_t)); - memcpy(copy, p, sizeof(rend_service_port_config_t)); - smartlist_add(v0_service->ports, copy); - }); - v0_service->intro_period_started = service->intro_period_started; - v0_service->descriptor_version = 0; /* Unversioned descriptor. */ - v0_service->auth_type = REND_NO_AUTH; - rend_add_service(v0_service); - } - - service->descriptor_version = 2; /* Versioned descriptor. */ - } - - if (service->auth_type != REND_NO_AUTH && !service->descriptor_version) { - log_warn(LD_CONFIG, "Hidden service with client authorization and " - "version 0 descriptors configured; ignoring."); - rend_service_free(service); - return; - } - if (service->auth_type != REND_NO_AUTH && smartlist_len(service->clients) == 0) { log_warn(LD_CONFIG, "Hidden service with client authorization but no " @@ -297,7 +280,7 @@ rend_config_services(or_options_t *options, int validate_only) for (line = options->RendConfigLines; line; line = line->next) { if (!strcasecmp(line->key, "HiddenServiceDir")) { - if (service) { + if (service) { /* register the one we just finished parsing */ if (validate_only) rend_service_free(service); else @@ -307,7 +290,6 @@ rend_config_services(or_options_t *options, int validate_only) service->directory = tor_strdup(line->value); service->ports = smartlist_create(); service->intro_period_started = time(NULL); - service->descriptor_version = -1; /**< All descriptor versions. */ continue; } if (!service) { @@ -433,35 +415,13 @@ rend_config_services(or_options_t *options, int validate_only) return -1; } } else { - smartlist_t *versions; - char *version_str; - int i, version, ver_ok=1, versions_bitmask = 0; tor_assert(!strcasecmp(line->key, "HiddenServiceVersion")); - versions = smartlist_create(); - smartlist_split_string(versions, line->value, ",", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - for (i = 0; i < smartlist_len(versions); i++) { - version_str = smartlist_get(versions, i); - if (strlen(version_str) != 1 || strspn(version_str, "02") != 1) { - log_warn(LD_CONFIG, - "HiddenServiceVersion can only be 0 and/or 2."); - SMARTLIST_FOREACH(versions, char *, cp, tor_free(cp)); - smartlist_free(versions); - rend_service_free(service); - return -1; - } - version = (int)tor_parse_long(version_str, 10, 0, INT_MAX, &ver_ok, - NULL); - if (!ver_ok) - continue; - versions_bitmask |= 1 << version; + if (strcmp(line->value, "2")) { + log_warn(LD_CONFIG, + "The only supported HiddenServiceVersion is 2."); + rend_service_free(service); + return -1; } - /* If exactly one version is set, change descriptor_version to that - * value; otherwise leave it at -1. */ - if (versions_bitmask == 1 << 0) service->descriptor_version = 0; - if (versions_bitmask == 1 << 2) service->descriptor_version = 2; - SMARTLIST_FOREACH(versions, char *, cp, tor_free(cp)); - smartlist_free(versions); } } if (service) { @@ -483,8 +443,7 @@ rend_config_services(or_options_t *options, int validate_only) * probably ok? */ SMARTLIST_FOREACH(rend_service_list, rend_service_t *, new, { SMARTLIST_FOREACH(old_service_list, rend_service_t *, old, { - if (!strcmp(old->directory, new->directory) && - old->descriptor_version == new->descriptor_version) { + if (!strcmp(old->directory, new->directory)) { smartlist_add_all(new->intro_nodes, old->intro_nodes); smartlist_clear(old->intro_nodes); smartlist_add(surviving_services, old); @@ -507,18 +466,16 @@ rend_config_services(or_options_t *options, int validate_only) tor_assert(oc->rend_data); SMARTLIST_FOREACH(surviving_services, rend_service_t *, ptr, { if (!memcmp(ptr->pk_digest, oc->rend_data->rend_pk_digest, - DIGEST_LEN) && - ptr->descriptor_version == oc->rend_data->rend_desc_version) { + DIGEST_LEN)) { keep_it = 1; break; } }); if (keep_it) continue; - log_info(LD_REND, "Closing intro point %s for service %s version %d.", - safe_str(oc->build_state->chosen_exit->nickname), - oc->rend_data->onion_address, - oc->rend_data->rend_desc_version); + log_info(LD_REND, "Closing intro point %s for service %s.", + safe_str_client(oc->build_state->chosen_exit->nickname), + oc->rend_data->onion_address); circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); /* XXXX Is there another reason we should use here? */ } @@ -541,14 +498,13 @@ rend_service_update_descriptor(rend_service_t *service) rend_service_descriptor_t *d; origin_circuit_t *circ; int i; - if (service->desc) { - rend_service_descriptor_free(service->desc); - service->desc = NULL; - } + + rend_service_descriptor_free(service->desc); + service->desc = NULL; + d = service->desc = tor_malloc_zero(sizeof(rend_service_descriptor_t)); d->pk = crypto_pk_dup_key(service->private_key); d->timestamp = time(NULL); - d->version = service->descriptor_version; d->intro_nodes = smartlist_create(); /* Support intro protocols 2 and 3. */ d->protocols = (1 << 2) + (1 << 3); @@ -556,7 +512,7 @@ rend_service_update_descriptor(rend_service_t *service) for (i = 0; i < smartlist_len(service->intro_nodes); ++i) { rend_intro_point_t *intro_svc = smartlist_get(service->intro_nodes, i); rend_intro_point_t *intro_desc; - circ = find_intro_circuit(intro_svc, service->pk_digest, d->version); + circ = find_intro_circuit(intro_svc, service->pk_digest); if (!circ || circ->_base.purpose != CIRCUIT_PURPOSE_S_INTRO) continue; @@ -797,17 +753,15 @@ rend_service_load_keys(void) return r; } -/** Return the service whose public key has a digest of <b>digest</b> and - * which publishes the given descriptor <b>version</b>. Return NULL if no - * such service exists. +/** Return the service whose public key has a digest of <b>digest</b>, or + * NULL if no such service exists. */ static rend_service_t * -rend_service_get_by_pk_digest_and_version(const char* digest, - uint8_t version) +rend_service_get_by_pk_digest(const char* digest) { SMARTLIST_FOREACH(rend_service_list, rend_service_t*, s, - if (!memcmp(s->pk_digest,digest,DIGEST_LEN) && - s->descriptor_version == version) return s); + if (!memcmp(s->pk_digest,digest,DIGEST_LEN)) + return s); return NULL; } @@ -894,6 +848,7 @@ clean_accepted_intros(rend_service_t *service, time_t now) /** Respond to an INTRODUCE2 cell by launching a circuit to the chosen * rendezvous point. */ + /* XXX022 this function sure could use some organizing. -RD */ int rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, size_t request_len) @@ -921,6 +876,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, time_t now = time(NULL); char diffie_hellman_hash[DIGEST_LEN]; time_t *access_time; + or_options_t *options = get_options(); + tor_assert(circuit->rend_data); base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, @@ -944,21 +901,16 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, } /* look up service depending on circuit. */ - service = rend_service_get_by_pk_digest_and_version( - circuit->rend_data->rend_pk_digest, - circuit->rend_data->rend_desc_version); + service = rend_service_get_by_pk_digest( + circuit->rend_data->rend_pk_digest); if (!service) { log_warn(LD_REND, "Got an INTRODUCE2 cell for an unrecognized service %s.", escaped(serviceid)); return -1; } - /* if descriptor version is 2, use intro key instead of service key. */ - if (circuit->rend_data->rend_desc_version == 0) { - intro_key = service->private_key; - } else { - intro_key = circuit->intro_key; - } + /* use intro key instead of service key. */ + intro_key = circuit->intro_key; /* first DIGEST_LEN bytes of request is intro or service pk digest */ crypto_pk_get_digest(intro_key, intro_key_digest); @@ -989,7 +941,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, len = r; if (*buf == 3) { /* Version 3 INTRODUCE2 cell. */ - time_t ts = 0, now = time(NULL); + time_t ts = 0; v3_shift = 1; auth_type = buf[1]; switch (auth_type) { @@ -1083,7 +1035,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, router = router_get_by_nickname(rp_nickname, 0); if (!router) { log_info(LD_REND, "Couldn't find router %s named in introduce2 cell.", - escaped_safe_str(rp_nickname)); + escaped_safe_str_client(rp_nickname)); /* XXXX Add a no-such-router reason? */ reason = END_CIRC_REASON_TORPROTOCOL; goto err; @@ -1098,6 +1050,15 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, goto err; } + /* Check if we'd refuse to talk to this router */ + if (options->ExcludeNodes && options->StrictNodes && + routerset_contains_extendinfo(options->ExcludeNodes, extend_info)) { + log_warn(LD_REND, "Client asked to rendezvous at a relay that we " + "exclude, and StrictNodes is set. Refusing service."); + reason = END_CIRC_REASON_INTERNAL; /* XXX might leak why we refused */ + goto err; + } + r_cookie = ptr; base16_encode(hexcookie,9,r_cookie,4); @@ -1158,7 +1119,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, reason = END_CIRC_REASON_INTERNAL; goto err; } - if (crypto_dh_compute_secret(dh, ptr+REND_COOKIE_LEN, DH_KEY_LEN, keys, + if (crypto_dh_compute_secret(LOG_PROTOCOL_WARN, dh, ptr+REND_COOKIE_LEN, + DH_KEY_LEN, keys, DIGEST_LEN+CPATH_KEY_MATERIAL_LEN)<0) { log_warn(LD_BUG, "Internal error: couldn't complete DH handshake"); reason = END_CIRC_REASON_INTERNAL; @@ -1168,7 +1130,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, circ_needs_uptime = rend_service_requires_uptime(service); /* help predict this next time */ - rep_hist_note_used_internal(time(NULL), circ_needs_uptime, 1); + rep_hist_note_used_internal(now, circ_needs_uptime, 1); /* Launch a circuit to alice's chosen rendezvous point. */ @@ -1184,14 +1146,16 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, if (!launched) { /* give up */ log_warn(LD_REND, "Giving up launching first hop of circuit to rendezvous " "point %s for service %s.", - escaped_safe_str(extend_info->nickname), serviceid); + escaped_safe_str_client(extend_info->nickname), + serviceid); reason = END_CIRC_REASON_CONNECTFAILED; goto err; } log_info(LD_REND, "Accepted intro; launching circuit to %s " "(cookie %s) for service %s.", - escaped_safe_str(extend_info->nickname), hexcookie, serviceid); + escaped_safe_str_client(extend_info->nickname), + hexcookie, serviceid); tor_assert(launched->build_state); /* Fill in the circuit's state. */ launched->rend_data = tor_malloc_zero(sizeof(rend_data_t)); @@ -1201,11 +1165,10 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, memcpy(launched->rend_data->rend_cookie, r_cookie, REND_COOKIE_LEN); strlcpy(launched->rend_data->onion_address, service->service_id, sizeof(launched->rend_data->onion_address)); - launched->rend_data->rend_desc_version = service->descriptor_version; launched->build_state->pending_final_cpath = cpath = tor_malloc_zero(sizeof(crypt_path_t)); cpath->magic = CRYPT_PATH_MAGIC; - launched->build_state->expiry_time = time(NULL) + MAX_REND_TIMEOUT; + launched->build_state->expiry_time = now + MAX_REND_TIMEOUT; cpath->dh_handshake_state = dh; dh = NULL; @@ -1289,7 +1252,7 @@ rend_service_launch_establish_intro(rend_service_t *service, log_info(LD_REND, "Launching circuit to introduction point %s for service %s", - escaped_safe_str(intro->extend_info->nickname), + escaped_safe_str_client(intro->extend_info->nickname), service->service_id); rep_hist_note_used_internal(time(NULL), 1, 0); @@ -1302,7 +1265,7 @@ rend_service_launch_establish_intro(rend_service_t *service, if (!launched) { log_info(LD_REND, "Can't launch circuit to establish introduction at %s.", - escaped_safe_str(intro->extend_info->nickname)); + escaped_safe_str_client(intro->extend_info->nickname)); return -1; } @@ -1325,18 +1288,16 @@ rend_service_launch_establish_intro(rend_service_t *service, strlcpy(launched->rend_data->onion_address, service->service_id, sizeof(launched->rend_data->onion_address)); memcpy(launched->rend_data->rend_pk_digest, service->pk_digest, DIGEST_LEN); - launched->rend_data->rend_desc_version = service->descriptor_version; - if (service->descriptor_version == 2) - launched->intro_key = crypto_pk_dup_key(intro->intro_key); + launched->intro_key = crypto_pk_dup_key(intro->intro_key); if (launched->_base.state == CIRCUIT_STATE_OPEN) rend_service_intro_has_opened(launched); return 0; } /** Return the number of introduction points that are or have been - * established for the given service address and rendezvous version. */ + * established for the given service address in <b>query</b>. */ static int -count_established_intro_points(const char *query, int rend_version) +count_established_intro_points(const char *query) { int num_ipos = 0; circuit_t *circ; @@ -1347,7 +1308,6 @@ count_established_intro_points(const char *query, int rend_version) circ->purpose == CIRCUIT_PURPOSE_S_INTRO)) { origin_circuit_t *oc = TO_ORIGIN_CIRCUIT(circ); if (oc->rend_data && - oc->rend_data->rend_desc_version == rend_version && !rend_cmp_service_ids(query, oc->rend_data->onion_address)) num_ipos++; } @@ -1377,9 +1337,8 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, circuit->rend_data->rend_pk_digest, REND_SERVICE_ID_LEN); - service = rend_service_get_by_pk_digest_and_version( - circuit->rend_data->rend_pk_digest, - circuit->rend_data->rend_desc_version); + service = rend_service_get_by_pk_digest( + circuit->rend_data->rend_pk_digest); if (!service) { log_warn(LD_REND, "Unrecognized service ID %s on introduction circuit %d.", serviceid, circuit->_base.n_circ_id); @@ -1388,28 +1347,34 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) } /* If we already have enough introduction circuits for this service, - * redefine this one as a general circuit. */ - if (count_established_intro_points(serviceid, - circuit->rend_data->rend_desc_version) > NUM_INTRO_POINTS) { - log_info(LD_CIRC|LD_REND, "We have just finished an introduction " - "circuit, but we already have enough. Redefining purpose to " - "general."); - TO_CIRCUIT(circuit)->purpose = CIRCUIT_PURPOSE_C_GENERAL; - circuit_has_opened(circuit); - return; + * redefine this one as a general circuit or close it, depending. */ + if (count_established_intro_points(serviceid) > NUM_INTRO_POINTS) { + or_options_t *options = get_options(); + if (options->ExcludeNodes) { + /* XXXX in some future version, we can test whether the transition is + allowed or not given the actual nodes in the circuit. But for now, + this case, we might as well close the thing. */ + log_info(LD_CIRC|LD_REND, "We have just finished an introduction " + "circuit, but we already have enough. Closing it."); + circuit_mark_for_close(TO_CIRCUIT(circuit), END_CIRC_REASON_NONE); + return; + } else { + tor_assert(circuit->build_state->is_internal); + log_info(LD_CIRC|LD_REND, "We have just finished an introduction " + "circuit, but we already have enough. Redefining purpose to " + "general; leaving as internal."); + TO_CIRCUIT(circuit)->purpose = CIRCUIT_PURPOSE_C_GENERAL; + circuit_has_opened(circuit); + return; + } } log_info(LD_REND, "Established circuit %d as introduction point for service %s", circuit->_base.n_circ_id, serviceid); - /* If the introduction point will not be used in an unversioned - * descriptor, use the intro key instead of the service key in - * ESTABLISH_INTRO. */ - if (service->descriptor_version == 0) - intro_key = service->private_key; - else - intro_key = circuit->intro_key; + /* Use the intro key instead of the service key in ESTABLISH_INTRO. */ + intro_key = circuit->intro_key; /* Build the payload for a RELAY_ESTABLISH_INTRO cell. */ r = crypto_pk_asn1_encode(intro_key, buf+2, RELAY_PAYLOAD_SIZE-2); @@ -1453,7 +1418,7 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) /** Called when we get an INTRO_ESTABLISHED cell; mark the circuit as a * live introduction point, and note that the service descriptor is - * now out-of-date.*/ + * now out-of-date. */ int rend_service_intro_established(origin_circuit_t *circuit, const uint8_t *request, @@ -1470,9 +1435,8 @@ rend_service_intro_established(origin_circuit_t *circuit, goto err; } tor_assert(circuit->rend_data); - service = rend_service_get_by_pk_digest_and_version( - circuit->rend_data->rend_pk_digest, - circuit->rend_data->rend_desc_version); + service = rend_service_get_by_pk_digest( + circuit->rend_data->rend_pk_digest); if (!service) { log_warn(LD_REND, "Unknown service on introduction circuit %d.", circuit->_base.n_circ_id); @@ -1522,9 +1486,8 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit) "cookie %s for service %s", circuit->_base.n_circ_id, hexcookie, serviceid); - service = rend_service_get_by_pk_digest_and_version( - circuit->rend_data->rend_pk_digest, - circuit->rend_data->rend_desc_version); + service = rend_service_get_by_pk_digest( + circuit->rend_data->rend_pk_digest); if (!service) { log_warn(LD_GENERAL, "Internal error: unrecognized service ID on " "introduction circuit."); @@ -1580,13 +1543,12 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit) */ /** Return the (possibly non-open) introduction circuit ending at - * <b>intro</b> for the service whose public key is <b>pk_digest</b> and - * which publishes descriptor of version <b>desc_version</b>. Return - * NULL if no such service is found. + * <b>intro</b> for the service whose public key is <b>pk_digest</b>. + * (<b>desc_version</b> is ignored). Return NULL if no such service is + * found. */ static origin_circuit_t * -find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest, - int desc_version) +find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest) { origin_circuit_t *circ = NULL; @@ -1595,8 +1557,7 @@ find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest, CIRCUIT_PURPOSE_S_INTRO))) { if (!memcmp(circ->build_state->chosen_exit->identity_digest, intro->extend_info->identity_digest, DIGEST_LEN) && - circ->rend_data && - circ->rend_data->rend_desc_version == desc_version) { + circ->rend_data) { return circ; } } @@ -1606,8 +1567,7 @@ find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest, CIRCUIT_PURPOSE_S_ESTABLISH_INTRO))) { if (!memcmp(circ->build_state->chosen_exit->identity_digest, intro->extend_info->identity_digest, DIGEST_LEN) && - circ->rend_data && - circ->rend_data->rend_desc_version == desc_version) { + circ->rend_data) { return circ; } } @@ -1640,6 +1600,7 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, } for (j = 0; j < smartlist_len(responsible_dirs); j++) { char desc_id_base32[REND_DESC_ID_V2_LEN_BASE32 + 1]; + char *hs_dir_ip; hs_dir = smartlist_get(responsible_dirs, j); if (smartlist_digest_isin(renddesc->successful_uploads, hs_dir->identity_digest)) @@ -1661,15 +1622,18 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, strlen(desc->desc_str), 0); base32_encode(desc_id_base32, sizeof(desc_id_base32), desc->desc_id, DIGEST_LEN); + hs_dir_ip = tor_dup_ip(hs_dir->addr); log_info(LD_REND, "Sending publish request for v2 descriptor for " "service '%s' with descriptor ID '%s' with validity " "of %d seconds to hidden service directory '%s' on " - "port %d.", - safe_str(service_id), - safe_str(desc_id_base32), + "%s:%d.", + safe_str_client(service_id), + safe_str_client(desc_id_base32), seconds_valid, hs_dir->nickname, - hs_dir->dir_port); + hs_dir_ip, + hs_dir->or_port); + tor_free(hs_dir_ip); /* Remember successful upload to this router for next time. */ if (!smartlist_digest_isin(successful_uploads, hs_dir->identity_digest)) smartlist_add(successful_uploads, hs_dir->identity_digest); @@ -1699,9 +1663,8 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, smartlist_free(successful_uploads); } -/** Encode and sign up-to-date v0 and/or v2 service descriptors for - * <b>service</b>, and upload it/them to all the dirservers/to the - * responsible hidden service directories. +/** Encode and sign an up-to-date service descriptor for <b>service</b>, + * and upload it/them to the responsible hidden service directories. */ static void upload_service_descriptor(rend_service_t *service) @@ -1713,35 +1676,8 @@ upload_service_descriptor(rend_service_t *service) rendpostperiod = get_options()->RendPostPeriod; - /* Upload unversioned (v0) descriptor? */ - if (service->descriptor_version == 0 && - get_options()->PublishHidServDescriptors) { - char *desc; - size_t desc_len; - /* Encode the descriptor. */ - if (rend_encode_service_descriptor(service->desc, - service->private_key, - &desc, &desc_len)<0) { - log_warn(LD_BUG, "Internal error: couldn't encode service descriptor; " - "not uploading."); - return; - } - - /* Post it to the dirservers */ - rend_get_service_id(service->desc->pk, serviceid); - log_info(LD_REND, "Sending publish request for hidden service %s", - serviceid); - directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_RENDDESC, - ROUTER_PURPOSE_GENERAL, - HIDSERV_AUTHORITY, desc, desc_len, 0); - tor_free(desc); - service->next_upload_time = now + rendpostperiod; - uploaded = 1; - } - - /* Upload v2 descriptor? */ - if (service->descriptor_version == 2 && - get_options()->PublishHidServDescriptors) { + /* Upload descriptor? */ + if (get_options()->PublishHidServDescriptors) { networkstatus_t *c = networkstatus_get_latest_consensus(); if (c && smartlist_len(c->routerstatus_list) > 0) { int seconds_valid, i, j, num_descs; @@ -1880,8 +1816,7 @@ rend_services_introduce(void) for (j=0; j < smartlist_len(service->intro_nodes); ++j) { intro = smartlist_get(service->intro_nodes, j); router = router_get_by_digest(intro->extend_info->identity_digest); - if (!router || !find_intro_circuit(intro, service->pk_digest, - service->descriptor_version)) { + if (!router || !find_intro_circuit(intro, service->pk_digest)) { log_info(LD_REND,"Giving up on %s as intro point for %s.", intro->extend_info->nickname, service->service_id); if (service->desc) { @@ -1933,7 +1868,7 @@ rend_services_introduce(void) router_crn_flags_t flags = CRN_NEED_UPTIME; if (get_options()->_AllowInvalid & ALLOW_INVALID_INTRODUCTION) flags |= CRN_ALLOW_INVALID; - router = router_choose_random_node(NULL, intro_routers, + router = router_choose_random_node(intro_routers, options->ExcludeNodes, flags); if (!router) { log_warn(LD_REND, @@ -1945,10 +1880,8 @@ rend_services_introduce(void) smartlist_add(intro_routers, router); intro = tor_malloc_zero(sizeof(rend_intro_point_t)); intro->extend_info = extend_info_from_router(router); - if (service->descriptor_version == 2) { - intro->intro_key = crypto_new_pk_env(); - tor_assert(!crypto_pk_generate_key(intro->intro_key)); - } + intro->intro_key = crypto_new_pk_env(); + tor_assert(!crypto_pk_generate_key(intro->intro_key)); smartlist_add(service->intro_nodes, intro); log_info(LD_REND, "Picked router %s as an intro point for %s.", router->nickname, service->service_id); @@ -2041,8 +1974,7 @@ rend_consider_descriptor_republication(void) for (i=0; i < smartlist_len(rend_service_list); ++i) { service = smartlist_get(rend_service_list, i); - if (service->descriptor_version && service->desc && - !service->desc->all_uploads_performed) { + if (service->desc && !service->desc->all_uploads_performed) { /* If we failed in uploading a descriptor last time, try again *without* * updating the descriptor's contents. */ upload_service_descriptor(service); @@ -2068,10 +2000,9 @@ rend_service_dump_stats(int severity) service->directory); for (j=0; j < smartlist_len(service->intro_nodes); ++j) { intro = smartlist_get(service->intro_nodes, j); - safe_name = safe_str(intro->extend_info->nickname); + safe_name = safe_str_client(intro->extend_info->nickname); - circ = find_intro_circuit(intro, service->pk_digest, - service->descriptor_version); + circ = find_intro_circuit(intro, service->pk_digest); if (!circ) { log(severity, LD_GENERAL, " Intro point %d at %s: no circuit", j, safe_name); @@ -2102,9 +2033,8 @@ rend_service_set_connection_addr_port(edge_connection_t *conn, log_debug(LD_REND,"beginning to hunt for addr/port"); base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, circ->rend_data->rend_pk_digest, REND_SERVICE_ID_LEN); - service = rend_service_get_by_pk_digest_and_version( - circ->rend_data->rend_pk_digest, - circ->rend_data->rend_desc_version); + service = rend_service_get_by_pk_digest( + circ->rend_data->rend_pk_digest); if (!service) { log_warn(LD_REND, "Couldn't find any service associated with pk %s on " "rendezvous circuit %d; closing.", diff --git a/src/or/rendservice.h b/src/or/rendservice.h new file mode 100644 index 000000000..70389afe9 --- /dev/null +++ b/src/or/rendservice.h @@ -0,0 +1,37 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file rendservice.h + * \brief Header file for rendservice.c. + **/ + +#ifndef _TOR_RENDSERVICE_H +#define _TOR_RENDSERVICE_H + +int num_rend_services(void); +int rend_config_services(or_options_t *options, int validate_only); +int rend_service_load_keys(void); +void rend_services_introduce(void); +void rend_consider_services_upload(time_t now); +void rend_hsdir_routers_changed(void); +void rend_consider_descriptor_republication(void); + +void rend_service_intro_has_opened(origin_circuit_t *circuit); +int rend_service_intro_established(origin_circuit_t *circuit, + const uint8_t *request, + size_t request_len); +void rend_service_rendezvous_has_opened(origin_circuit_t *circuit); +int rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, + size_t request_len); +void rend_service_relaunch_rendezvous(origin_circuit_t *oldcirc); +int rend_service_set_connection_addr_port(edge_connection_t *conn, + origin_circuit_t *circ); +void rend_service_dump_stats(int severity); +void rend_service_free_all(void); + +#endif + diff --git a/src/or/rephist.c b/src/or/rephist.c index dee74ed5c..02db24738 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -6,15 +6,22 @@ * \file rephist.c * \brief Basic history and "reputation" functionality to remember * which servers have worked in the past, how much bandwidth we've - * been using, which ports we tend to want, and so on. + * been using, which ports we tend to want, and so on; further, + * exit port statistics and cell statistics. **/ #include "or.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "networkstatus.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" #include "ht.h" static void bw_arrays_init(void); static void predicted_ports_init(void); -static void hs_usage_init(void); /** Total number of bytes currently allocated in fields used by rephist.c. */ uint64_t rephist_total_alloc=0; @@ -67,6 +74,13 @@ typedef struct or_history_t { /** If nonzero, we have been unable to connect since this time. */ time_t down_since; + /** The address at which we most recently connected to this OR + * successfully. */ + tor_addr_t last_reached_addr; + + /** The port at which we most recently connected to this OR successfully */ + uint16_t last_reached_port; + /* === For MTBF tracking: */ /** Weighted sum total of all times that this router has been online. */ @@ -113,6 +127,7 @@ get_or_history(const char* id) rephist_total_num++; hist->link_history_map = digestmap_new(); hist->since = hist->changed = time(NULL); + tor_addr_make_unspec(&hist->last_reached_addr); digestmap_set(history_map, id, hist); } return hist; @@ -185,7 +200,6 @@ rep_hist_init(void) history_map = digestmap_new(); bw_arrays_init(); predicted_ports_init(); - hs_usage_init(); } /** Helper: note that we are no longer connected to the router with history @@ -284,13 +298,20 @@ rep_hist_note_connection_died(const char* id, time_t when) /** We have just decided that this router with identity digest <b>id</b> is * reachable, meaning we will give it a "Running" flag for the next while. */ void -rep_hist_note_router_reachable(const char *id, time_t when) +rep_hist_note_router_reachable(const char *id, const tor_addr_t *at_addr, + const uint16_t at_port, time_t when) { or_history_t *hist = get_or_history(id); int was_in_run = 1; char tbuf[ISO_TIME_LEN+1]; + int addr_changed, port_changed; tor_assert(hist); + tor_assert((!at_addr && !at_port) || (at_addr && at_port)); + + addr_changed = at_addr && + tor_addr_compare(at_addr, &hist->last_reached_addr, CMP_EXACT) != 0; + port_changed = at_port && at_port != hist->last_reached_port; if (!started_tracking_stability) started_tracking_stability = time(NULL); @@ -310,6 +331,27 @@ rep_hist_note_router_reachable(const char *id, time_t when) down_length = when - hist->start_of_downtime; hist->total_weighted_time += down_length; hist->start_of_downtime = 0; + } else if (addr_changed || port_changed) { + /* If we're reachable, but the address changed, treat this as some + * downtime. */ + int penalty = get_options()->TestingTorNetwork ? 240 : 3600; + networkstatus_t *ns; + + if ((ns = networkstatus_get_latest_consensus())) { + int fresh_interval = (int)(ns->fresh_until - ns->valid_after); + int live_interval = (int)(ns->valid_until - ns->valid_after); + /* on average, a descriptor addr change takes .5 intervals to make it + * into a consensus, and half a liveness period to make it to + * clients. */ + penalty = (int)(fresh_interval + live_interval) / 2; + } + format_local_iso_time(tbuf, hist->start_of_run); + log_info(LD_HIST,"Router %s still seems Running, but its address appears " + "to have changed since the last time it was reachable. I'm " + "going to treat it as having been down for %d seconds", + hex_str(id, DIGEST_LEN), penalty); + rep_hist_note_router_unreachable(id, when-penalty); + rep_hist_note_router_reachable(id, NULL, 0, when); } else { format_local_iso_time(tbuf, hist->start_of_run); if (was_in_run) @@ -319,6 +361,10 @@ rep_hist_note_router_reachable(const char *id, time_t when) log_info(LD_HIST,"Router %s is now Running; it was previously untracked", hex_str(id, DIGEST_LEN)); } + if (at_addr) + tor_addr_copy(&hist->last_reached_addr, at_addr); + if (at_port) + hist->last_reached_port = at_port; } /** We have just decided that this router is unreachable, meaning @@ -339,12 +385,20 @@ rep_hist_note_router_unreachable(const char *id, time_t when) long run_length = when - hist->start_of_run; format_local_iso_time(tbuf, hist->start_of_run); - hist->weighted_run_length += run_length; hist->total_run_weights += 1.0; hist->start_of_run = 0; - hist->weighted_uptime += run_length; - hist->total_weighted_time += run_length; + if (run_length < 0) { + unsigned long penalty = -run_length; +#define SUBTRACT_CLAMPED(var, penalty) \ + do { (var) = (var) < (penalty) ? 0 : (var) - (penalty); } while (0) + SUBTRACT_CLAMPED(hist->weighted_run_length, penalty); + SUBTRACT_CLAMPED(hist->weighted_uptime, penalty); + } else { + hist->weighted_run_length += run_length; + hist->weighted_uptime += run_length; + hist->total_weighted_time += run_length; + } was_running = 1; log_info(LD_HIST, "Router %s is now non-Running: it had previously been " "Running since %s. Its total weighted uptime is %lu/%lu.", @@ -417,7 +471,7 @@ rep_hist_downrate_old_runs(time_t now) static double get_stability(or_history_t *hist, time_t when) { - unsigned long total = hist->weighted_run_length; + long total = hist->weighted_run_length; double total_weights = hist->total_run_weights; if (hist->start_of_run) { @@ -453,8 +507,8 @@ get_total_weighted_time(or_history_t *hist, time_t when) static double get_weighted_fractional_uptime(or_history_t *hist, time_t when) { - unsigned long total = hist->total_weighted_time; - unsigned long up = hist->weighted_uptime; + long total = hist->total_weighted_time; + long up = hist->weighted_uptime; if (hist->start_of_run) { long run_length = (when - hist->start_of_run); @@ -474,6 +528,20 @@ get_weighted_fractional_uptime(or_history_t *hist, time_t when) return ((double) up) / total; } +/** Return how long the router whose identity digest is <b>id</b> has + * been reachable. Return 0 if the router is unknown or currently deemed + * unreachable. */ +long +rep_hist_get_uptime(const char *id, time_t when) +{ + or_history_t *hist = get_or_history(id); + if (!hist) + return 0; + if (!hist->start_of_run || when < hist->start_of_run) + return 0; + return when - hist->start_of_run; +} + /** Return an estimated MTBF for the router whose identity digest is * <b>id</b>. Return 0 if the router is unknown. */ double @@ -519,7 +587,7 @@ rep_hist_get_weighted_time_known(const char *id, time_t when) int rep_hist_have_measured_enough_stability(void) { - /* XXXX021 This doesn't do so well when we change our opinion + /* XXXX022 This doesn't do so well when we change our opinion * as to whether we're tracking router stability. */ return started_tracking_stability < time(NULL) - 4*60*60; } @@ -795,12 +863,12 @@ rep_hist_record_mtbf_data(time_t now, int missing_means_down) static char * rep_hist_format_router_status(or_history_t *hist, time_t now) { - char buf[1024]; char sor_buf[ISO_TIME_LEN+1]; char sod_buf[ISO_TIME_LEN+1]; double wfu; double mtbf; int up = 0, down = 0; + char *cp = NULL; if (hist->start_of_run) { format_iso_time(sor_buf, hist->start_of_run); @@ -813,7 +881,7 @@ rep_hist_format_router_status(or_history_t *hist, time_t now) wfu = get_weighted_fractional_uptime(hist, now); mtbf = get_stability(hist, now); - tor_snprintf(buf, sizeof(buf), + tor_asprintf(&cp, "%s%s%s" "%s%s%s" "wfu %0.3lf\n" @@ -831,8 +899,7 @@ rep_hist_format_router_status(or_history_t *hist, time_t now) hist->weighted_run_length, hist->total_run_weights ); - - return tor_strdup(buf); + return cp; } /** The last stability analysis document that we created, or NULL if we never @@ -925,10 +992,10 @@ find_next_with(smartlist_t *sl, int i, const char *prefix) return -1; } -/** How many bad times has parse_possibly_bad_iso_time parsed? */ +/** How many bad times has parse_possibly_bad_iso_time() parsed? */ static int n_bogus_times = 0; /** Parse the ISO-formatted time in <b>s</b> into *<b>time_out</b>, but - * rounds any pre-1970 date to Jan 1, 1970. */ + * round any pre-1970 date to Jan 1, 1970. */ static int parse_possibly_bad_iso_time(const char *s, time_t *time_out) { @@ -1161,6 +1228,8 @@ rep_hist_load_mtbf_data(time_t now) * totals? */ #define NUM_SECS_ROLLING_MEASURE 10 /** How large are the intervals for which we track and report bandwidth use? */ +/* XXXX Watch out! Before Tor 0.2.2.21-alpha, using any other value here would + * generate an unparseable state file. */ #define NUM_SECS_BW_SUM_INTERVAL (15*60) /** How far in the past do we remember and publish bandwidth use? */ #define NUM_SECS_BW_SUM_IS_VALID (24*60*60) @@ -1219,7 +1288,7 @@ commit_max(bw_array_t *b) b->total_in_period = 0; } -/** Shift the current observation time of 'b' forward by one second. */ +/** Shift the current observation time of <b>b</b> forward by one second. */ static INLINE void advance_obs(bw_array_t *b) { @@ -1249,14 +1318,18 @@ advance_obs(bw_array_t *b) static INLINE void add_obs(bw_array_t *b, time_t when, uint64_t n) { - /* Don't record data in the past. */ - if (when<b->cur_obs_time) - return; + if (when < b->cur_obs_time) + return; /* Don't record data in the past. */ + /* If we're currently adding observations for an earlier second than * 'when', advance b->cur_obs_time and b->cur_obs_idx by an - * appropriate number of seconds, and do all the other housekeeping */ - while (when>b->cur_obs_time) + * appropriate number of seconds, and do all the other housekeeping. */ + while (when > b->cur_obs_time) { + /* Doing this one second at a time is potentially inefficient, if we start + with a state file that is very old. Fortunately, it doesn't seem to + show up in profiles, so we can just ignore it for now. */ advance_obs(b); + } b->obs[b->cur_obs_idx] += n; b->total_in_period += n; @@ -1280,16 +1353,29 @@ bw_array_new(void) static bw_array_t *read_array = NULL; /** Recent history of bandwidth observations for write operations. */ static bw_array_t *write_array = NULL; - -/** Set up read_array and write_array. */ +/** Recent history of bandwidth observations for read operations for the + directory protocol. */ +static bw_array_t *dir_read_array = NULL; +/** Recent history of bandwidth observations for write operations for the + directory protocol. */ +static bw_array_t *dir_write_array = NULL; + +/** Set up [dir-]read_array and [dir-]write_array, freeing them if they + * already exist. */ static void bw_arrays_init(void) { + tor_free(read_array); + tor_free(write_array); + tor_free(dir_read_array); + tor_free(dir_write_array); read_array = bw_array_new(); write_array = bw_array_new(); + dir_read_array = bw_array_new(); + dir_write_array = bw_array_new(); } -/** We read <b>num_bytes</b> more bytes in second <b>when</b>. +/** Remember that we read <b>num_bytes</b> bytes in second <b>when</b>. * * Add num_bytes to the current running total for <b>when</b>. * @@ -1310,7 +1396,7 @@ rep_hist_note_bytes_written(size_t num_bytes, time_t when) add_obs(write_array, when, num_bytes); } -/** We wrote <b>num_bytes</b> more bytes in second <b>when</b>. +/** Remember that we wrote <b>num_bytes</b> bytes in second <b>when</b>. * (like rep_hist_note_bytes_written() above) */ void @@ -1320,6 +1406,24 @@ rep_hist_note_bytes_read(size_t num_bytes, time_t when) add_obs(read_array, when, num_bytes); } +/** Remember that we wrote <b>num_bytes</b> directory bytes in second + * <b>when</b>. (like rep_hist_note_bytes_written() above) + */ +void +rep_hist_note_dir_bytes_written(size_t num_bytes, time_t when) +{ + add_obs(dir_write_array, when, num_bytes); +} + +/** Remember that we read <b>num_bytes</b> directory bytes in second + * <b>when</b>. (like rep_hist_note_bytes_written() above) + */ +void +rep_hist_note_dir_bytes_read(size_t num_bytes, time_t when) +{ + add_obs(dir_read_array, when, num_bytes); +} + /** Helper: Return the largest value in b->maxima. (This is equal to the * most bandwidth used in any NUM_SECS_ROLLING_MEASURE period for the last * NUM_SECS_BW_SUM_IS_VALID seconds.) @@ -1355,14 +1459,14 @@ rep_hist_bandwidth_assess(void) return (int)(U64_TO_DBL(r)/NUM_SECS_ROLLING_MEASURE); } -/** Print the bandwidth history of b (either read_array or write_array) - * into the buffer pointed to by buf. The format is simply comma - * separated numbers, from oldest to newest. +/** Print the bandwidth history of b (either [dir-]read_array or + * [dir-]write_array) into the buffer pointed to by buf. The format is + * simply comma separated numbers, from oldest to newest. * * It returns the number of bytes written. */ static size_t -rep_hist_fill_bandwidth_history(char *buf, size_t len, bw_array_t *b) +rep_hist_fill_bandwidth_history(char *buf, size_t len, const bw_array_t *b) { char *cp = buf; int i, n; @@ -1407,151 +1511,254 @@ rep_hist_fill_bandwidth_history(char *buf, size_t len, bw_array_t *b) } /** Allocate and return lines for representing this server's bandwidth - * history in its descriptor. + * history in its descriptor. We publish these lines in our extra-info + * descriptor. */ char * -rep_hist_get_bandwidth_lines(int for_extrainfo) +rep_hist_get_bandwidth_lines(void) { char *buf, *cp; char t[ISO_TIME_LEN+1]; int r; - bw_array_t *b; + bw_array_t *b = NULL; + const char *desc = NULL; size_t len; - /* opt (read|write)-history yyyy-mm-dd HH:MM:SS (n s) n,n,n,n,n... */ - len = (60+20*NUM_TOTALS)*2; + /* opt [dirreq-](read|write)-history yyyy-mm-dd HH:MM:SS (n s) n,n,n... */ +/* The n,n,n part above. Largest representation of a uint64_t is 20 chars + * long, plus the comma. */ +#define MAX_HIST_VALUE_LEN 21*NUM_TOTALS + len = (67+MAX_HIST_VALUE_LEN)*4; buf = tor_malloc_zero(len); cp = buf; - for (r=0;r<2;++r) { - b = r?read_array:write_array; + for (r=0;r<4;++r) { + char tmp[MAX_HIST_VALUE_LEN]; + size_t slen; + switch (r) { + case 0: + b = write_array; + desc = "write-history"; + break; + case 1: + b = read_array; + desc = "read-history"; + break; + case 2: + b = dir_write_array; + desc = "dirreq-write-history"; + break; + case 3: + b = dir_read_array; + desc = "dirreq-read-history"; + break; + } tor_assert(b); + slen = rep_hist_fill_bandwidth_history(tmp, MAX_HIST_VALUE_LEN, b); + /* If we don't have anything to write, skip to the next entry. */ + if (slen == 0) + continue; format_iso_time(t, b->next_period-NUM_SECS_BW_SUM_INTERVAL); - tor_snprintf(cp, len-(cp-buf), "%s%s %s (%d s) ", - for_extrainfo ? "" : "opt ", - r ? "read-history" : "write-history", t, - NUM_SECS_BW_SUM_INTERVAL); + tor_snprintf(cp, len-(cp-buf), "%s %s (%d s) ", + desc, t, NUM_SECS_BW_SUM_INTERVAL); cp += strlen(cp); - cp += rep_hist_fill_bandwidth_history(cp, len-(cp-buf), b); + strlcat(cp, tmp, len-(cp-buf)); + cp += slen; strlcat(cp, "\n", len-(cp-buf)); ++cp; } return buf; } -/** Update <b>state</b> with the newest bandwidth history. */ +/** Write a single bw_array_t into the Values, Ends, Interval, and Maximum + * entries of an or_state_t. Done before writing out a new state file. */ +static void +rep_hist_update_bwhist_state_section(or_state_t *state, + const bw_array_t *b, + smartlist_t **s_values, + smartlist_t **s_maxima, + time_t *s_begins, + int *s_interval) +{ + char *cp; + int i,j; + uint64_t maxval; + + if (*s_values) { + SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); + smartlist_free(*s_values); + } + if (*s_maxima) { + SMARTLIST_FOREACH(*s_maxima, char *, val, tor_free(val)); + smartlist_free(*s_maxima); + } + if (! server_mode(get_options())) { + /* Clients don't need to store bandwidth history persistently; + * force these values to the defaults. */ + /* FFFF we should pull the default out of config.c's state table, + * so we don't have two defaults. */ + if (*s_begins != 0 || *s_interval != 900) { + time_t now = time(NULL); + time_t save_at = get_options()->AvoidDiskWrites ? now+3600 : now+600; + or_state_mark_dirty(state, save_at); + } + *s_begins = 0; + *s_interval = 900; + *s_values = smartlist_create(); + *s_maxima = smartlist_create(); + return; + } + *s_begins = b->next_period; + *s_interval = NUM_SECS_BW_SUM_INTERVAL; + + *s_values = smartlist_create(); + *s_maxima = smartlist_create(); + /* Set i to first position in circular array */ + i = (b->num_maxes_set <= b->next_max_idx) ? 0 : b->next_max_idx; + for (j=0; j < b->num_maxes_set; ++j,++i) { + if (i >= NUM_TOTALS) + i = 0; + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->totals[i] & ~0x3ff)); + smartlist_add(*s_values, cp); + maxval = b->maxima[i] / NUM_SECS_ROLLING_MEASURE; + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(maxval & ~0x3ff)); + smartlist_add(*s_maxima, cp); + } + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(b->total_in_period & ~0x3ff)); + smartlist_add(*s_values, cp); + maxval = b->max_total / NUM_SECS_ROLLING_MEASURE; + tor_asprintf(&cp, U64_FORMAT, U64_PRINTF_ARG(maxval & ~0x3ff)); + smartlist_add(*s_maxima, cp); +} + +/** Update <b>state</b> with the newest bandwidth history. Done before + * writing out a new state file. */ void rep_hist_update_state(or_state_t *state) { - int len, r; - char *buf, *cp; - smartlist_t **s_values; - time_t *s_begins; - int *s_interval; - bw_array_t *b; - - len = 20*NUM_TOTALS+1; - buf = tor_malloc_zero(len); +#define UPDATE(arrname,st) \ + rep_hist_update_bwhist_state_section(state,\ + (arrname),\ + &state->BWHistory ## st ## Values, \ + &state->BWHistory ## st ## Maxima, \ + &state->BWHistory ## st ## Ends, \ + &state->BWHistory ## st ## Interval) - for (r=0;r<2;++r) { - b = r?read_array:write_array; - s_begins = r?&state->BWHistoryReadEnds :&state->BWHistoryWriteEnds; - s_interval= r?&state->BWHistoryReadInterval:&state->BWHistoryWriteInterval; - s_values = r?&state->BWHistoryReadValues :&state->BWHistoryWriteValues; + UPDATE(write_array, Write); + UPDATE(read_array, Read); + UPDATE(dir_write_array, DirWrite); + UPDATE(dir_read_array, DirRead); - if (*s_values) { - SMARTLIST_FOREACH(*s_values, char *, val, tor_free(val)); - smartlist_free(*s_values); - } - if (! server_mode(get_options())) { - /* Clients don't need to store bandwidth history persistently; - * force these values to the defaults. */ - /* FFFF we should pull the default out of config.c's state table, - * so we don't have two defaults. */ - if (*s_begins != 0 || *s_interval != 900) { - time_t now = time(NULL); - time_t save_at = get_options()->AvoidDiskWrites ? now+3600 : now+600; - or_state_mark_dirty(state, save_at); - } - *s_begins = 0; - *s_interval = 900; - *s_values = smartlist_create(); - continue; - } - *s_begins = b->next_period; - *s_interval = NUM_SECS_BW_SUM_INTERVAL; - cp = buf; - cp += rep_hist_fill_bandwidth_history(cp, len, b); - tor_snprintf(cp, len-(cp-buf), cp == buf ? U64_FORMAT : ","U64_FORMAT, - U64_PRINTF_ARG(b->total_in_period)); - *s_values = smartlist_create(); - if (server_mode(get_options())) - smartlist_split_string(*s_values, buf, ",", SPLIT_SKIP_SPACE, 0); - } - tor_free(buf); if (server_mode(get_options())) { - or_state_mark_dirty(get_or_state(), time(NULL)+(2*3600)); + or_state_mark_dirty(state, time(NULL)+(2*3600)); } +#undef UPDATE } -/** Set bandwidth history from our saved state. */ -int -rep_hist_load_state(or_state_t *state, char **err) +/** Load a single bw_array_t from its Values, Ends, Maxima, and Interval + * entries in an or_state_t. Done while reading the state file. */ +static int +rep_hist_load_bwhist_state_section(bw_array_t *b, + const smartlist_t *s_values, + const smartlist_t *s_maxima, + const time_t s_begins, + const int s_interval) { - time_t s_begins, start; time_t now = time(NULL); - uint64_t v; - int r,i,ok; - int all_ok = 1; - int s_interval; - smartlist_t *s_values; - bw_array_t *b; - - /* Assert they already have been malloced */ - tor_assert(read_array && write_array); + int retval = 0; + time_t start; - for (r=0;r<2;++r) { - b = r?read_array:write_array; - s_begins = r?state->BWHistoryReadEnds:state->BWHistoryWriteEnds; - s_interval = r?state->BWHistoryReadInterval:state->BWHistoryWriteInterval; - s_values = r?state->BWHistoryReadValues:state->BWHistoryWriteValues; - if (s_values && s_begins >= now - NUM_SECS_BW_SUM_INTERVAL*NUM_TOTALS) { - start = s_begins - s_interval*(smartlist_len(s_values)); - if (start > now) - continue; - b->cur_obs_time = start; - b->next_period = start + NUM_SECS_BW_SUM_INTERVAL; - SMARTLIST_FOREACH(s_values, char *, cp, { + uint64_t v, mv; + int i,ok,ok_m; + int have_maxima = (smartlist_len(s_values) == smartlist_len(s_maxima)); + + if (s_values && s_begins >= now - NUM_SECS_BW_SUM_INTERVAL*NUM_TOTALS) { + start = s_begins - s_interval*(smartlist_len(s_values)); + if (start > now) + return 0; + b->cur_obs_time = start; + b->next_period = start + NUM_SECS_BW_SUM_INTERVAL; + SMARTLIST_FOREACH_BEGIN(s_values, const char *, cp) { + const char *maxstr = NULL; v = tor_parse_uint64(cp, 10, 0, UINT64_MAX, &ok, NULL); + if (have_maxima) { + maxstr = smartlist_get(s_maxima, cp_sl_idx); + mv = tor_parse_uint64(maxstr, 10, 0, UINT64_MAX, &ok_m, NULL); + mv *= NUM_SECS_ROLLING_MEASURE; + } else { + /* No maxima known; guess average rate to be conservative. */ + mv = (v / s_interval) * NUM_SECS_ROLLING_MEASURE; + } if (!ok) { - all_ok=0; - log_notice(LD_HIST, "Could not parse '%s' into a number.'", cp); + retval = -1; + log_notice(LD_HIST, "Could not parse value '%s' into a number.'",cp); } + if (maxstr && !ok_m) { + retval = -1; + log_notice(LD_HIST, "Could not parse maximum '%s' into a number.'", + maxstr); + } + if (start < now) { - add_obs(b, start, v); - start += NUM_SECS_BW_SUM_INTERVAL; + time_t cur_start = start; + time_t actual_interval_len = s_interval; + uint64_t cur_val = 0; + /* Calculate the average per second. This is the best we can do + * because our state file doesn't have per-second resolution. */ + if (start + s_interval > now) + actual_interval_len = now - start; + cur_val = v / actual_interval_len; + /* This is potentially inefficient, but since we don't do it very + * often it should be ok. */ + while (cur_start < start + actual_interval_len) { + add_obs(b, cur_start, cur_val); + ++cur_start; + } + b->max_total = mv; + /* This will result in some fairly choppy history if s_interval + * is not the same as NUM_SECS_BW_SUM_INTERVAL. XXXX */ + start += actual_interval_len; } - }); - } + } SMARTLIST_FOREACH_END(cp); + } - /* Clean up maxima and observed */ - /* Do we really want to zero this for the purpose of max capacity? */ - for (i=0; i<NUM_SECS_ROLLING_MEASURE; ++i) { - b->obs[i] = 0; - } - b->total_obs = 0; - for (i=0; i<NUM_TOTALS; ++i) { - b->maxima[i] = 0; - } - b->max_total = 0; + /* Clean up maxima and observed */ + for (i=0; i<NUM_SECS_ROLLING_MEASURE; ++i) { + b->obs[i] = 0; } + b->total_obs = 0; + + return retval; +} + +/** Set bandwidth history from the state file we just loaded. */ +int +rep_hist_load_state(or_state_t *state, char **err) +{ + int all_ok = 1; + /* Assert they already have been malloced */ + tor_assert(read_array && write_array); + tor_assert(dir_read_array && dir_write_array); + +#define LOAD(arrname,st) \ + if (rep_hist_load_bwhist_state_section( \ + (arrname), \ + state->BWHistory ## st ## Values, \ + state->BWHistory ## st ## Maxima, \ + state->BWHistory ## st ## Ends, \ + state->BWHistory ## st ## Interval)<0) \ + all_ok = 0 + + LOAD(write_array, Write); + LOAD(read_array, Read); + LOAD(dir_write_array, DirWrite); + LOAD(dir_read_array, DirRead); + +#undef LOAD if (!all_ok) { *err = tor_strdup("Parsing of bandwidth history values failed"); /* and create fresh arrays */ - tor_free(read_array); - tor_free(write_array); - read_array = bw_array_new(); - write_array = bw_array_new(); + bw_arrays_init(); return -1; } return 0; @@ -1571,7 +1778,7 @@ static smartlist_t *predicted_ports_times=NULL; static void add_predicted_port(time_t now, uint16_t port) { - /* XXXX we could just use uintptr_t here, I think. */ + /* XXXX we could just use uintptr_t here, I think. -NM */ uint16_t *tmp_port = tor_malloc(sizeof(uint16_t)); time_t *tmp_time = tor_malloc(sizeof(time_t)); *tmp_port = port; @@ -1715,8 +1922,8 @@ rep_hist_get_predicted_internal(time_t now, int *need_uptime, return 0; /* too long ago */ if (predicted_internal_uptime_time + PREDICTED_CIRCS_RELEVANCE_TIME >= now) *need_uptime = 1; - if (predicted_internal_capacity_time + PREDICTED_CIRCS_RELEVANCE_TIME >= now) - *need_capacity = 1; + // Always predict that we need capacity. + *need_capacity = 1; return 1; } @@ -1845,564 +2052,530 @@ dump_pk_ops(int severity) pk_op_counts.n_rend_server_ops); } -/** Free all storage held by the OR/link history caches, by the - * bandwidth history arrays, or by the port history. */ +/*** Exit port statistics ***/ + +/* Some constants */ +/** To what multiple should byte numbers be rounded up? */ +#define EXIT_STATS_ROUND_UP_BYTES 1024 +/** To what multiple should stream counts be rounded up? */ +#define EXIT_STATS_ROUND_UP_STREAMS 4 +/** Number of TCP ports */ +#define EXIT_STATS_NUM_PORTS 65536 +/** Top n ports that will be included in exit stats. */ +#define EXIT_STATS_TOP_N_PORTS 10 + +/* The following data structures are arrays and no fancy smartlists or maps, + * so that all write operations can be done in constant time. This comes at + * the price of some memory (1.25 MB) and linear complexity when writing + * stats for measuring relays. */ +/** Number of bytes read in current period by exit port */ +static uint64_t *exit_bytes_read = NULL; +/** Number of bytes written in current period by exit port */ +static uint64_t *exit_bytes_written = NULL; +/** Number of streams opened in current period by exit port */ +static uint32_t *exit_streams = NULL; + +/** Start time of exit stats or 0 if we're not collecting exit stats. */ +static time_t start_of_exit_stats_interval; + +/** Initialize exit port stats. */ void -rep_hist_free_all(void) +rep_hist_exit_stats_init(time_t now) { - digestmap_free(history_map, free_or_history); - tor_free(read_array); - tor_free(write_array); - tor_free(last_stability_doc); - built_last_stability_doc_at = 0; - predicted_ports_free(); + start_of_exit_stats_interval = now; + exit_bytes_read = tor_malloc_zero(EXIT_STATS_NUM_PORTS * + sizeof(uint64_t)); + exit_bytes_written = tor_malloc_zero(EXIT_STATS_NUM_PORTS * + sizeof(uint64_t)); + exit_streams = tor_malloc_zero(EXIT_STATS_NUM_PORTS * + sizeof(uint32_t)); } -/****************** hidden service usage statistics ******************/ - -/** How large are the intervals for which we track and report hidden service - * use? */ -#define NUM_SECS_HS_USAGE_SUM_INTERVAL (15*60) -/** How far in the past do we remember and publish hidden service use? */ -#define NUM_SECS_HS_USAGE_SUM_IS_VALID (24*60*60) -/** How many hidden service usage intervals do we remember? (derived) */ -#define NUM_TOTALS_HS_USAGE (NUM_SECS_HS_USAGE_SUM_IS_VALID/ \ - NUM_SECS_HS_USAGE_SUM_INTERVAL) - -/** List element containing a service id and the count. */ -typedef struct hs_usage_list_elem_t { - /** Service id of this elem. */ - char service_id[REND_SERVICE_ID_LEN_BASE32+1]; - /** Number of occurrences for the given service id. */ - uint32_t count; - /* Pointer to next list elem */ - struct hs_usage_list_elem_t *next; -} hs_usage_list_elem_t; - -/** Ordered list that stores service ids and the number of observations. It is - * ordered by the number of occurrences in descending order. Its purpose is to - * calculate the frequency distribution when the period is over. */ -typedef struct hs_usage_list_t { - /* Pointer to the first element in the list. */ - hs_usage_list_elem_t *start; - /* Number of total occurrences for all list elements. */ - uint32_t total_count; - /* Number of service ids, i.e. number of list elements. */ - uint32_t total_service_ids; -} hs_usage_list_t; - -/** Tracks service-related observations in the current period and their - * history. */ -typedef struct hs_usage_service_related_observation_t { - /** Ordered list that stores service ids and the number of observations in - * the current period. It is ordered by the number of occurrences in - * descending order. Its purpose is to calculate the frequency distribution - * when the period is over. */ - hs_usage_list_t *list; - /** Circular arrays that store the history of observations. totals stores all - * observations, twenty (ten, five) the number of observations related to a - * service id being accounted for the top 20 (10, 5) percent of all - * observations. */ - uint32_t totals[NUM_TOTALS_HS_USAGE]; - uint32_t five[NUM_TOTALS_HS_USAGE]; - uint32_t ten[NUM_TOTALS_HS_USAGE]; - uint32_t twenty[NUM_TOTALS_HS_USAGE]; -} hs_usage_service_related_observation_t; - -/** Tracks the history of general period-related observations, i.e. those that - * cannot be related to a specific service id. */ -typedef struct hs_usage_general_period_related_observations_t { - /** Circular array that stores the history of observations. */ - uint32_t totals[NUM_TOTALS_HS_USAGE]; -} hs_usage_general_period_related_observations_t; - -/** Keeps information about the current observation period and its relation to - * the histories of observations. */ -typedef struct hs_usage_current_observation_period_t { - /** Where do we write the next history entry? */ - int next_idx; - /** How many values in history have been set ever? (upper bound!) */ - int num_set; - /** When did this period begin? */ - time_t start_of_current_period; - /** When does the next period begin? */ - time_t start_of_next_period; -} hs_usage_current_observation_period_t; - -/** Usage statistics for the current observation period. */ -static hs_usage_current_observation_period_t *current_period = NULL; - -/** Total number of descriptor publish requests in the current observation - * period. */ -static hs_usage_service_related_observation_t *publish_total = NULL; - -/** Number of descriptor publish requests for services that have not been - * seen before in the current observation period. */ -static hs_usage_service_related_observation_t *publish_novel = NULL; - -/** Total number of descriptor fetch requests in the current observation - * period. */ -static hs_usage_service_related_observation_t *fetch_total = NULL; - -/** Number of successful descriptor fetch requests in the current - * observation period. */ -static hs_usage_service_related_observation_t *fetch_successful = NULL; - -/** Number of descriptors stored in the current observation period. */ -static hs_usage_general_period_related_observations_t *descs = NULL; - -/** Creates an empty ordered list element. */ -static hs_usage_list_elem_t * -hs_usage_list_elem_new(void) +/** Reset counters for exit port statistics. */ +void +rep_hist_reset_exit_stats(time_t now) { - hs_usage_list_elem_t *e; - e = tor_malloc_zero(sizeof(hs_usage_list_elem_t)); - rephist_total_alloc += sizeof(hs_usage_list_elem_t); - e->count = 1; - e->next = NULL; - return e; + start_of_exit_stats_interval = now; + memset(exit_bytes_read, 0, EXIT_STATS_NUM_PORTS * sizeof(uint64_t)); + memset(exit_bytes_written, 0, EXIT_STATS_NUM_PORTS * sizeof(uint64_t)); + memset(exit_streams, 0, EXIT_STATS_NUM_PORTS * sizeof(uint32_t)); } -/** Creates an empty ordered list. */ -static hs_usage_list_t * -hs_usage_list_new(void) +/** Stop collecting exit port stats in a way that we can re-start doing + * so in rep_hist_exit_stats_init(). */ +void +rep_hist_exit_stats_term(void) { - hs_usage_list_t *l; - l = tor_malloc_zero(sizeof(hs_usage_list_t)); - rephist_total_alloc += sizeof(hs_usage_list_t); - l->start = NULL; - l->total_count = 0; - l->total_service_ids = 0; - return l; + start_of_exit_stats_interval = 0; + tor_free(exit_bytes_read); + tor_free(exit_bytes_written); + tor_free(exit_streams); } -/** Creates an empty structure for storing service-related observations. */ -static hs_usage_service_related_observation_t * -hs_usage_service_related_observation_new(void) +/** Helper for qsort: compare two ints. */ +static int +_compare_int(const void *x, const void *y) { - hs_usage_service_related_observation_t *h; - h = tor_malloc_zero(sizeof(hs_usage_service_related_observation_t)); - rephist_total_alloc += sizeof(hs_usage_service_related_observation_t); - h->list = hs_usage_list_new(); - return h; + return (*(int*)x - *(int*)y); } -/** Creates an empty structure for storing general period-related - * observations. */ -static hs_usage_general_period_related_observations_t * -hs_usage_general_period_related_observations_new(void) -{ - hs_usage_general_period_related_observations_t *p; - p = tor_malloc_zero(sizeof(hs_usage_general_period_related_observations_t)); - rephist_total_alloc+= sizeof(hs_usage_general_period_related_observations_t); - return p; -} +/** Return a newly allocated string containing the exit port statistics + * until <b>now</b>, or NULL if we're not collecting exit stats. */ +char * +rep_hist_format_exit_stats(time_t now) +{ + int i, j, top_elements = 0, cur_min_idx = 0, cur_port; + uint64_t top_bytes[EXIT_STATS_TOP_N_PORTS]; + int top_ports[EXIT_STATS_TOP_N_PORTS]; + uint64_t cur_bytes = 0, other_read = 0, other_written = 0, + total_read = 0, total_written = 0; + uint32_t total_streams = 0, other_streams = 0; + char *buf; + smartlist_t *written_strings, *read_strings, *streams_strings; + char *written_string, *read_string, *streams_string; + char t[ISO_TIME_LEN+1]; + char *result; -/** Creates an empty structure for storing period-specific information. */ -static hs_usage_current_observation_period_t * -hs_usage_current_observation_period_new(void) -{ - hs_usage_current_observation_period_t *c; - time_t now; - c = tor_malloc_zero(sizeof(hs_usage_current_observation_period_t)); - rephist_total_alloc += sizeof(hs_usage_current_observation_period_t); - now = time(NULL); - c->start_of_current_period = now; - c->start_of_next_period = now + NUM_SECS_HS_USAGE_SUM_INTERVAL; - return c; -} + if (!start_of_exit_stats_interval) + return NULL; /* Not initialized. */ -/** Initializes the structures for collecting hidden service usage data. */ -static void -hs_usage_init(void) -{ - current_period = hs_usage_current_observation_period_new(); - publish_total = hs_usage_service_related_observation_new(); - publish_novel = hs_usage_service_related_observation_new(); - fetch_total = hs_usage_service_related_observation_new(); - fetch_successful = hs_usage_service_related_observation_new(); - descs = hs_usage_general_period_related_observations_new(); -} + /* Go through all ports to find the n ports that saw most written and + * read bytes. + * + * Invariant: at the end of the loop for iteration i, + * total_read is the sum of all exit_bytes_read[0..i] + * total_written is the sum of all exit_bytes_written[0..i] + * total_stream is the sum of all exit_streams[0..i] + * + * top_elements = MAX(EXIT_STATS_TOP_N_PORTS, + * #{j | 0 <= j <= i && volume(i) > 0}) + * + * For all 0 <= j < top_elements, + * top_bytes[j] > 0 + * 0 <= top_ports[j] <= 65535 + * top_bytes[j] = volume(top_ports[j]) + * + * There is no j in 0..i and k in 0..top_elements such that: + * volume(j) > top_bytes[k] AND j is not in top_ports[0..top_elements] + * + * There is no j!=cur_min_idx in 0..top_elements such that: + * top_bytes[j] < top_bytes[cur_min_idx] + * + * where volume(x) == exit_bytes_read[x]+exit_bytes_written[x] + * + * Worst case: O(EXIT_STATS_NUM_PORTS * EXIT_STATS_TOP_N_PORTS) + */ + for (i = 1; i < EXIT_STATS_NUM_PORTS; i++) { + total_read += exit_bytes_read[i]; + total_written += exit_bytes_written[i]; + total_streams += exit_streams[i]; + cur_bytes = exit_bytes_read[i] + exit_bytes_written[i]; + if (cur_bytes == 0) { + continue; + } + if (top_elements < EXIT_STATS_TOP_N_PORTS) { + top_bytes[top_elements] = cur_bytes; + top_ports[top_elements++] = i; + } else if (cur_bytes > top_bytes[cur_min_idx]) { + top_bytes[cur_min_idx] = cur_bytes; + top_ports[cur_min_idx] = i; + } else { + continue; + } + cur_min_idx = 0; + for (j = 1; j < top_elements; j++) { + if (top_bytes[j] < top_bytes[cur_min_idx]) { + cur_min_idx = j; + } + } + } -/** Clears the given ordered list by resetting its attributes and releasing - * the memory allocated by its elements. */ -static void -hs_usage_list_clear(hs_usage_list_t *lst) -{ - /* walk through elements and free memory */ - hs_usage_list_elem_t *current = lst->start; - hs_usage_list_elem_t *tmp; - while (current != NULL) { - tmp = current->next; - rephist_total_alloc -= sizeof(hs_usage_list_elem_t); - tor_free(current); - current = tmp; + /* Add observations of top ports to smartlists. */ + written_strings = smartlist_create(); + read_strings = smartlist_create(); + streams_strings = smartlist_create(); + other_read = total_read; + other_written = total_written; + other_streams = total_streams; + /* Sort the ports; this puts them out of sync with top_bytes, but we + * won't be using top_bytes again anyway */ + qsort(top_ports, top_elements, sizeof(int), _compare_int); + for (j = 0; j < top_elements; j++) { + cur_port = top_ports[j]; + if (exit_bytes_written[cur_port] > 0) { + uint64_t num = round_uint64_to_next_multiple_of( + exit_bytes_written[cur_port], + EXIT_STATS_ROUND_UP_BYTES); + num /= 1024; + buf = NULL; + tor_asprintf(&buf, "%d="U64_FORMAT, cur_port, U64_PRINTF_ARG(num)); + smartlist_add(written_strings, buf); + other_written -= exit_bytes_written[cur_port]; + } + if (exit_bytes_read[cur_port] > 0) { + uint64_t num = round_uint64_to_next_multiple_of( + exit_bytes_read[cur_port], + EXIT_STATS_ROUND_UP_BYTES); + num /= 1024; + buf = NULL; + tor_asprintf(&buf, "%d="U64_FORMAT, cur_port, U64_PRINTF_ARG(num)); + smartlist_add(read_strings, buf); + other_read -= exit_bytes_read[cur_port]; + } + if (exit_streams[cur_port] > 0) { + uint32_t num = round_uint32_to_next_multiple_of( + exit_streams[cur_port], + EXIT_STATS_ROUND_UP_STREAMS); + buf = NULL; + tor_asprintf(&buf, "%d=%u", cur_port, num); + smartlist_add(streams_strings, buf); + other_streams -= exit_streams[cur_port]; + } } - /* reset attributes */ - lst->start = NULL; - lst->total_count = 0; - lst->total_service_ids = 0; - return; -} -/** Frees the memory used by the given list. */ -static void -hs_usage_list_free(hs_usage_list_t *lst) -{ - if (!lst) - return; - hs_usage_list_clear(lst); - rephist_total_alloc -= sizeof(hs_usage_list_t); - tor_free(lst); + /* Add observations of other ports in a single element. */ + other_written = round_uint64_to_next_multiple_of(other_written, + EXIT_STATS_ROUND_UP_BYTES); + other_written /= 1024; + buf = NULL; + tor_asprintf(&buf, "other="U64_FORMAT, U64_PRINTF_ARG(other_written)); + smartlist_add(written_strings, buf); + other_read = round_uint64_to_next_multiple_of(other_read, + EXIT_STATS_ROUND_UP_BYTES); + other_read /= 1024; + buf = NULL; + tor_asprintf(&buf, "other="U64_FORMAT, U64_PRINTF_ARG(other_read)); + smartlist_add(read_strings, buf); + other_streams = round_uint32_to_next_multiple_of(other_streams, + EXIT_STATS_ROUND_UP_STREAMS); + buf = NULL; + tor_asprintf(&buf, "other=%u", other_streams); + smartlist_add(streams_strings, buf); + + /* Join all observations in single strings. */ + written_string = smartlist_join_strings(written_strings, ",", 0, NULL); + read_string = smartlist_join_strings(read_strings, ",", 0, NULL); + streams_string = smartlist_join_strings(streams_strings, ",", 0, NULL); + SMARTLIST_FOREACH(written_strings, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(read_strings, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(streams_strings, char *, cp, tor_free(cp)); + smartlist_free(written_strings); + smartlist_free(read_strings); + smartlist_free(streams_strings); + + /* Put everything together. */ + format_iso_time(t, now); + tor_asprintf(&result, "exit-stats-end %s (%d s)\n" + "exit-kibibytes-written %s\n" + "exit-kibibytes-read %s\n" + "exit-streams-opened %s\n", + t, (unsigned) (now - start_of_exit_stats_interval), + written_string, + read_string, + streams_string); + tor_free(written_string); + tor_free(read_string); + tor_free(streams_string); + return result; } -/** Frees the memory used by the given service-related observations. */ -static void -hs_usage_service_related_observation_free( - hs_usage_service_related_observation_t *s) +/** If 24 hours have passed since the beginning of the current exit port + * stats period, write exit stats to $DATADIR/stats/exit-stats (possibly + * overwriting an existing file) and reset counters. Return when we would + * next want to write exit stats or 0 if we never want to write. */ +time_t +rep_hist_exit_stats_write(time_t now) { - if (!s) - return; - hs_usage_list_free(s->list); - rephist_total_alloc -= sizeof(hs_usage_service_related_observation_t); - tor_free(s); -} + char *statsdir = NULL, *filename = NULL, *str = NULL; -/** Frees the memory used by the given period-specific observations. */ -static void -hs_usage_general_period_related_observations_free( - hs_usage_general_period_related_observations_t *s) -{ - rephist_total_alloc-=sizeof(hs_usage_general_period_related_observations_t); - tor_free(s); -} + if (!start_of_exit_stats_interval) + return 0; /* Not initialized. */ + if (start_of_exit_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write. */ -/** Frees the memory used by period-specific information. */ -static void -hs_usage_current_observation_period_free( - hs_usage_current_observation_period_t *s) -{ - rephist_total_alloc -= sizeof(hs_usage_current_observation_period_t); - tor_free(s); -} + log_info(LD_HIST, "Writing exit port statistics to disk."); -/** Frees all memory that was used for collecting hidden service usage data. */ -void -hs_usage_free_all(void) -{ - hs_usage_general_period_related_observations_free(descs); - descs = NULL; - hs_usage_service_related_observation_free(fetch_successful); - hs_usage_service_related_observation_free(fetch_total); - hs_usage_service_related_observation_free(publish_novel); - hs_usage_service_related_observation_free(publish_total); - fetch_successful = fetch_total = publish_novel = publish_total = NULL; - hs_usage_current_observation_period_free(current_period); - current_period = NULL; -} + /* Generate history string. */ + str = rep_hist_format_exit_stats(now); -/** Inserts a new occurrence for the given service id to the given ordered - * list. */ -static void -hs_usage_insert_value(hs_usage_list_t *lst, const char *service_id) -{ - /* search if there is already an elem with same service_id in list */ - hs_usage_list_elem_t *current = lst->start; - hs_usage_list_elem_t *previous = NULL; - while (current != NULL && strcasecmp(current->service_id,service_id)) { - previous = current; - current = current->next; - } - /* found an element with same service_id? */ - if (current == NULL) { - /* not found! append to end (which could also be the end of a zero-length - * list), don't need to sort (1 is smallest value). */ - /* create elem */ - hs_usage_list_elem_t *e = hs_usage_list_elem_new(); - /* update list attributes (one new elem, one new occurrence) */ - lst->total_count++; - lst->total_service_ids++; - /* copy service id to elem */ - strlcpy(e->service_id,service_id,sizeof(e->service_id)); - /* let either l->start or previously last elem point to new elem */ - if (lst->start == NULL) { - /* this is the first elem */ - lst->start = e; - } else { - /* there were elems in the list before */ - previous->next = e; - } - } else { - /* found! add occurrence to elem and consider resorting */ - /* update list attributes (no new elem, but one new occurrence) */ - lst->total_count++; - /* add occurrence to elem */ - current->count++; - /* is it another than the first list elem? and has previous elem fewer - * count than current? then we need to resort */ - if (previous != NULL && previous->count < current->count) { - /* yes! we need to resort */ - /* remove current elem first */ - previous->next = current->next; - /* can we prepend elem to all other elements? */ - if (lst->start->count <= current->count) { - /* yes! prepend elem */ - current->next = lst->start; - lst->start = current; - } else { - /* no! walk through list a second time and insert at correct place */ - hs_usage_list_elem_t *insert_current = lst->start->next; - hs_usage_list_elem_t *insert_previous = lst->start; - while (insert_current != NULL && - insert_current->count > current->count) { - insert_previous = insert_current; - insert_current = insert_current->next; - } - /* insert here */ - current->next = insert_current; - insert_previous->next = current; - } - } - } -} + /* Reset counters. */ + rep_hist_reset_exit_stats(now); -/** Writes the current service-related observations to the history array and - * clears the observations of the current period. */ -static void -hs_usage_write_service_related_observations_to_history( - hs_usage_current_observation_period_t *p, - hs_usage_service_related_observation_t *h) -{ - /* walk through the first 20 % of list elements and calculate frequency - * distributions */ - /* maximum indices for the three frequencies */ - int five_percent_idx = h->list->total_service_ids/20; - int ten_percent_idx = h->list->total_service_ids/10; - int twenty_percent_idx = h->list->total_service_ids/5; - /* temp values */ - uint32_t five_percent = 0; - uint32_t ten_percent = 0; - uint32_t twenty_percent = 0; - /* walk through list */ - hs_usage_list_elem_t *current = h->list->start; - int i=0; - while (current != NULL && i <= twenty_percent_idx) { - twenty_percent += current->count; - if (i <= ten_percent_idx) - ten_percent += current->count; - if (i <= five_percent_idx) - five_percent += current->count; - current = current->next; - i++; - } - /* copy frequencies */ - h->twenty[p->next_idx] = twenty_percent; - h->ten[p->next_idx] = ten_percent; - h->five[p->next_idx] = five_percent; - /* copy total number of observations */ - h->totals[p->next_idx] = h->list->total_count; - /* free memory of old list */ - hs_usage_list_clear(h->list); -} - -/** Advances to next observation period. */ -static void -hs_usage_advance_current_observation_period(void) -{ - /* aggregate observations to history, including frequency distribution - * arrays */ - hs_usage_write_service_related_observations_to_history( - current_period, publish_total); - hs_usage_write_service_related_observations_to_history( - current_period, publish_novel); - hs_usage_write_service_related_observations_to_history( - current_period, fetch_total); - hs_usage_write_service_related_observations_to_history( - current_period, fetch_successful); - /* write current number of descriptors to descs history */ - descs->totals[current_period->next_idx] = rend_cache_size(); - /* advance to next period */ - current_period->next_idx++; - if (current_period->next_idx == NUM_TOTALS_HS_USAGE) - current_period->next_idx = 0; - if (current_period->num_set < NUM_TOTALS_HS_USAGE) - ++current_period->num_set; - current_period->start_of_current_period=current_period->start_of_next_period; - current_period->start_of_next_period += NUM_SECS_HS_USAGE_SUM_INTERVAL; -} - -/** Checks if the current period is up to date, and if not, advances it. */ -static void -hs_usage_check_if_current_period_is_up_to_date(time_t now) -{ - while (now > current_period->start_of_next_period) { - hs_usage_advance_current_observation_period(); + /* Try to write to disk. */ + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE) < 0) { + log_warn(LD_HIST, "Unable to create stats/ directory!"); + goto done; } -} + filename = get_datadir_fname2("stats", "exit-stats"); + if (write_str_to_file(filename, str, 0) < 0) + log_warn(LD_HIST, "Unable to write exit port statistics to disk!"); -/** Adds a service-related observation, maybe after advancing to next - * observation period. */ -static void -hs_usage_add_service_related_observation( - hs_usage_service_related_observation_t *h, - time_t now, - const char *service_id) -{ - if (now < current_period->start_of_current_period) { - /* don't record old data */ - return; - } - /* check if we are up-to-date */ - hs_usage_check_if_current_period_is_up_to_date(now); - /* add observation */ - hs_usage_insert_value(h->list, service_id); + done: + tor_free(str); + tor_free(statsdir); + tor_free(filename); + return start_of_exit_stats_interval + WRITE_STATS_INTERVAL; } -/** Adds the observation of storing a rendezvous service descriptor to our - * cache in our role as HS authoritative directory. */ +/** Note that we wrote <b>num_written</b> bytes and read <b>num_read</b> + * bytes to/from an exit connection to <b>port</b>. */ void -hs_usage_note_publish_total(const char *service_id, time_t now) +rep_hist_note_exit_bytes(uint16_t port, size_t num_written, + size_t num_read) { - hs_usage_add_service_related_observation(publish_total, now, service_id); + if (!start_of_exit_stats_interval) + return; /* Not initialized. */ + exit_bytes_written[port] += num_written; + exit_bytes_read[port] += num_read; + log_debug(LD_HIST, "Written %lu bytes and read %lu bytes to/from an " + "exit connection to port %d.", + (unsigned long)num_written, (unsigned long)num_read, port); } -/** Adds the observation of storing a novel rendezvous service descriptor to - * our cache in our role as HS authoritative directory. */ +/** Note that we opened an exit stream to <b>port</b>. */ void -hs_usage_note_publish_novel(const char *service_id, time_t now) +rep_hist_note_exit_stream_opened(uint16_t port) { - hs_usage_add_service_related_observation(publish_novel, now, service_id); + if (!start_of_exit_stats_interval) + return; /* Not initialized. */ + exit_streams[port]++; + log_debug(LD_HIST, "Opened exit stream to port %d", port); } -/** Adds the observation of being requested for a rendezvous service descriptor - * in our role as HS authoritative directory. */ +/*** cell statistics ***/ + +/** Start of the current buffer stats interval or 0 if we're not + * collecting buffer statistics. */ +static time_t start_of_buffer_stats_interval; + +/** Initialize buffer stats. */ void -hs_usage_note_fetch_total(const char *service_id, time_t now) +rep_hist_buffer_stats_init(time_t now) { - hs_usage_add_service_related_observation(fetch_total, now, service_id); + start_of_buffer_stats_interval = now; } -/** Adds the observation of being requested for a rendezvous service descriptor - * in our role as HS authoritative directory and being able to answer that - * request successfully. */ +/** Statistics from a single circuit. Collected when the circuit closes, or + * when we flush statistics to disk. */ +typedef struct circ_buffer_stats_t { + /** Average number of cells in the circuit's queue */ + double mean_num_cells_in_queue; + /** Average time a cell waits in the queue. */ + double mean_time_cells_in_queue; + /** Total number of cells sent over this circuit */ + uint32_t processed_cells; +} circ_buffer_stats_t; + +/** List of circ_buffer_stats_t. */ +static smartlist_t *circuits_for_buffer_stats = NULL; + +/** Remember cell statistics for circuit <b>circ</b> at time + * <b>end_of_interval</b> and reset cell counters in case the circuit + * remains open in the next measurement interval. */ void -hs_usage_note_fetch_successful(const char *service_id, time_t now) +rep_hist_buffer_stats_add_circ(circuit_t *circ, time_t end_of_interval) +{ + circ_buffer_stats_t *stat; + time_t start_of_interval; + int interval_length; + or_circuit_t *orcirc; + if (CIRCUIT_IS_ORIGIN(circ)) + return; + orcirc = TO_OR_CIRCUIT(circ); + if (!orcirc->processed_cells) + return; + if (!circuits_for_buffer_stats) + circuits_for_buffer_stats = smartlist_create(); + start_of_interval = circ->timestamp_created.tv_sec > + start_of_buffer_stats_interval ? + circ->timestamp_created.tv_sec : + start_of_buffer_stats_interval; + interval_length = (int) (end_of_interval - start_of_interval); + if (interval_length <= 0) + return; + stat = tor_malloc_zero(sizeof(circ_buffer_stats_t)); + stat->processed_cells = orcirc->processed_cells; + /* 1000.0 for s -> ms; 2.0 because of app-ward and exit-ward queues */ + stat->mean_num_cells_in_queue = interval_length == 0 ? 0.0 : + (double) orcirc->total_cell_waiting_time / + (double) interval_length / 1000.0 / 2.0; + stat->mean_time_cells_in_queue = + (double) orcirc->total_cell_waiting_time / + (double) orcirc->processed_cells; + smartlist_add(circuits_for_buffer_stats, stat); + orcirc->total_cell_waiting_time = 0; + orcirc->processed_cells = 0; +} + +/** Sorting helper: return -1, 1, or 0 based on comparison of two + * circ_buffer_stats_t */ +static int +_buffer_stats_compare_entries(const void **_a, const void **_b) { - hs_usage_add_service_related_observation(fetch_successful, now, service_id); + const circ_buffer_stats_t *a = *_a, *b = *_b; + if (a->processed_cells < b->processed_cells) + return 1; + else if (a->processed_cells > b->processed_cells) + return -1; + else + return 0; } -/** Writes the given circular array to a string. */ -static size_t -hs_usage_format_history(char *buf, size_t len, uint32_t *data) +/** Stop collecting cell stats in a way that we can re-start doing so in + * rep_hist_buffer_stats_init(). */ +void +rep_hist_buffer_stats_term(void) { - char *cp = buf; /* pointer where we are in the buffer */ - int i, n; - if (current_period->num_set <= current_period->next_idx) { - i = 0; /* not been through circular array */ - } else { - i = current_period->next_idx; - } - for (n = 0; n < current_period->num_set; ++n,++i) { - if (i >= NUM_TOTALS_HS_USAGE) - i -= NUM_TOTALS_HS_USAGE; - tor_assert(i < NUM_TOTALS_HS_USAGE); - if (n == (current_period->num_set-1)) - tor_snprintf(cp, len-(cp-buf), "%d", data[i]); - else - tor_snprintf(cp, len-(cp-buf), "%d,", data[i]); - cp += strlen(cp); - } - return cp-buf; + start_of_buffer_stats_interval = 0; + if (!circuits_for_buffer_stats) + circuits_for_buffer_stats = smartlist_create(); + SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, + stat, tor_free(stat)); + smartlist_clear(circuits_for_buffer_stats); } -/** Writes the complete usage history as hidden service authoritative directory - * to a string. */ -static char * -hs_usage_format_statistics(void) +/** Write buffer statistics to $DATADIR/stats/buffer-stats and return when + * we would next want to write exit stats. */ +time_t +rep_hist_buffer_stats_write(time_t now) { - char *buf, *cp, *s = NULL; - char t[ISO_TIME_LEN+1]; - int r; - uint32_t *data = NULL; - size_t len; - len = (70+20*NUM_TOTALS_HS_USAGE)*11; - buf = tor_malloc_zero(len); - cp = buf; - for (r = 0; r < 11; ++r) { - switch (r) { - case 0: - s = (char*) "publish-total-history"; - data = publish_total->totals; - break; - case 1: - s = (char*) "publish-novel-history"; - data = publish_novel->totals; - break; - case 2: - s = (char*) "publish-top-5-percent-history"; - data = publish_total->five; - break; - case 3: - s = (char*) "publish-top-10-percent-history"; - data = publish_total->ten; - break; - case 4: - s = (char*) "publish-top-20-percent-history"; - data = publish_total->twenty; - break; - case 5: - s = (char*) "fetch-total-history"; - data = fetch_total->totals; - break; - case 6: - s = (char*) "fetch-successful-history"; - data = fetch_successful->totals; - break; - case 7: - s = (char*) "fetch-top-5-percent-history"; - data = fetch_total->five; - break; - case 8: - s = (char*) "fetch-top-10-percent-history"; - data = fetch_total->ten; - break; - case 9: - s = (char*) "fetch-top-20-percent-history"; - data = fetch_total->twenty; - break; - case 10: - s = (char*) "desc-total-history"; - data = descs->totals; - break; - } - format_iso_time(t, current_period->start_of_current_period); - tor_snprintf(cp, len-(cp-buf), "%s %s (%d s) ", s, t, - NUM_SECS_HS_USAGE_SUM_INTERVAL); - cp += strlen(cp); - cp += hs_usage_format_history(cp, len-(cp-buf), data); - strlcat(cp, "\n", len-(cp-buf)); - ++cp; + char *statsdir = NULL, *filename = NULL; + char written[ISO_TIME_LEN+1]; + open_file_t *open_file = NULL; + FILE *out; +#define SHARES 10 + int processed_cells[SHARES], circs_in_share[SHARES], + number_of_circuits, i; + double queued_cells[SHARES], time_in_queue[SHARES]; + smartlist_t *str_build = smartlist_create(); + char *str = NULL, *buf=NULL; + circuit_t *circ; + + if (!start_of_buffer_stats_interval) + return 0; /* Not initialized. */ + if (start_of_buffer_stats_interval + WRITE_STATS_INTERVAL > now) + goto done; /* Not ready to write */ + + /* add current circuits to stats */ + for (circ = _circuit_get_global_list(); circ; circ = circ->next) + rep_hist_buffer_stats_add_circ(circ, now); + /* calculate deciles */ + memset(processed_cells, 0, SHARES * sizeof(int)); + memset(circs_in_share, 0, SHARES * sizeof(int)); + memset(queued_cells, 0, SHARES * sizeof(double)); + memset(time_in_queue, 0, SHARES * sizeof(double)); + if (!circuits_for_buffer_stats) + circuits_for_buffer_stats = smartlist_create(); + smartlist_sort(circuits_for_buffer_stats, + _buffer_stats_compare_entries); + number_of_circuits = smartlist_len(circuits_for_buffer_stats); + if (number_of_circuits < 1) { + log_info(LD_HIST, "Attempt to write cell statistics to disk failed. " + "We haven't seen a single circuit to report about."); + goto done; } - return buf; + i = 0; + SMARTLIST_FOREACH_BEGIN(circuits_for_buffer_stats, + circ_buffer_stats_t *, stat) + { + int share = i++ * SHARES / number_of_circuits; + processed_cells[share] += stat->processed_cells; + queued_cells[share] += stat->mean_num_cells_in_queue; + time_in_queue[share] += stat->mean_time_cells_in_queue; + circs_in_share[share]++; + } + SMARTLIST_FOREACH_END(stat); + /* clear buffer stats history */ + SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, + stat, tor_free(stat)); + smartlist_clear(circuits_for_buffer_stats); + /* write to file */ + statsdir = get_datadir_fname("stats"); + if (check_private_dir(statsdir, CPD_CREATE) < 0) + goto done; + filename = get_datadir_fname2("stats", "buffer-stats"); + out = start_writing_to_stdio_file(filename, OPEN_FLAGS_APPEND, + 0600, &open_file); + if (!out) + goto done; + format_iso_time(written, now); + if (fprintf(out, "cell-stats-end %s (%d s)\n", written, + (unsigned) (now - start_of_buffer_stats_interval)) < 0) + goto done; + for (i = 0; i < SHARES; i++) { + tor_asprintf(&buf,"%d", !circs_in_share[i] ? 0 : + processed_cells[i] / circs_in_share[i]); + smartlist_add(str_build, buf); + } + str = smartlist_join_strings(str_build, ",", 0, NULL); + if (fprintf(out, "cell-processed-cells %s\n", str) < 0) + goto done; + tor_free(str); + SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); + smartlist_clear(str_build); + for (i = 0; i < SHARES; i++) { + tor_asprintf(&buf, "%.2f", circs_in_share[i] == 0 ? 0.0 : + queued_cells[i] / (double) circs_in_share[i]); + smartlist_add(str_build, buf); + } + str = smartlist_join_strings(str_build, ",", 0, NULL); + if (fprintf(out, "cell-queued-cells %s\n", str) < 0) + goto done; + tor_free(str); + SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); + smartlist_clear(str_build); + for (i = 0; i < SHARES; i++) { + tor_asprintf(&buf, "%.0f", circs_in_share[i] == 0 ? 0.0 : + time_in_queue[i] / (double) circs_in_share[i]); + smartlist_add(str_build, buf); + } + str = smartlist_join_strings(str_build, ",", 0, NULL); + if (fprintf(out, "cell-time-in-queue %s\n", str) < 0) + goto done; + tor_free(str); + SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); + smartlist_free(str_build); + str_build = NULL; + if (fprintf(out, "cell-circuits-per-decile %d\n", + (number_of_circuits + SHARES - 1) / SHARES) < 0) + goto done; + finish_writing_to_file(open_file); + open_file = NULL; + start_of_buffer_stats_interval = now; + done: + if (open_file) + abort_writing_to_file(open_file); + tor_free(filename); + tor_free(statsdir); + if (str_build) { + SMARTLIST_FOREACH(str_build, char *, c, tor_free(c)); + smartlist_free(str_build); + } + tor_free(str); +#undef SHARES + return start_of_buffer_stats_interval + WRITE_STATS_INTERVAL; } -/** Write current statistics about hidden service usage to file. */ +/** Free all storage held by the OR/link history caches, by the + * bandwidth history arrays, by the port history, or by statistics . */ void -hs_usage_write_statistics_to_file(time_t now) +rep_hist_free_all(void) { - char *buf; - size_t len; - char *fname; - or_options_t *options = get_options(); - /* check if we are up-to-date */ - hs_usage_check_if_current_period_is_up_to_date(now); - buf = hs_usage_format_statistics(); - len = strlen(options->DataDirectory) + 16; - fname = tor_malloc(len); - tor_snprintf(fname, len, "%s"PATH_SEPARATOR"hsusage", - options->DataDirectory); - write_str_to_file(fname,buf,0); - tor_free(buf); - tor_free(fname); + digestmap_free(history_map, free_or_history); + tor_free(read_array); + tor_free(write_array); + tor_free(last_stability_doc); + tor_free(exit_bytes_read); + tor_free(exit_bytes_written); + tor_free(exit_streams); + built_last_stability_doc_at = 0; + predicted_ports_free(); + if (circuits_for_buffer_stats) { + SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, s, + tor_free(s)); + smartlist_free(circuits_for_buffer_stats); + circuits_for_buffer_stats = NULL; + } } diff --git a/src/or/rephist.h b/src/or/rephist.h new file mode 100644 index 000000000..b06a39ed5 --- /dev/null +++ b/src/or/rephist.h @@ -0,0 +1,82 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file rephist.h + * \brief Header file for rephist.c. + **/ + +#ifndef _TOR_REPHIST_H +#define _TOR_REPHIST_H + +void rep_hist_init(void); +void rep_hist_note_connect_failed(const char* nickname, time_t when); +void rep_hist_note_connect_succeeded(const char* nickname, time_t when); +void rep_hist_note_disconnect(const char* nickname, time_t when); +void rep_hist_note_connection_died(const char* nickname, time_t when); +void rep_hist_note_extend_succeeded(const char *from_name, + const char *to_name); +void rep_hist_note_extend_failed(const char *from_name, const char *to_name); +void rep_hist_dump_stats(time_t now, int severity); +void rep_hist_note_bytes_read(size_t num_bytes, time_t when); +void rep_hist_note_bytes_written(size_t num_bytes, time_t when); + +void rep_hist_note_dir_bytes_read(size_t num_bytes, time_t when); +void rep_hist_note_dir_bytes_written(size_t num_bytes, time_t when); + +int rep_hist_bandwidth_assess(void); +char *rep_hist_get_bandwidth_lines(void); +void rep_hist_update_state(or_state_t *state); +int rep_hist_load_state(or_state_t *state, char **err); +void rep_history_clean(time_t before); + +void rep_hist_note_router_reachable(const char *id, const tor_addr_t *at_addr, + uint16_t at_port, time_t when); +void rep_hist_note_router_unreachable(const char *id, time_t when); +int rep_hist_record_mtbf_data(time_t now, int missing_means_down); +int rep_hist_load_mtbf_data(time_t now); + +time_t rep_hist_downrate_old_runs(time_t now); +long rep_hist_get_uptime(const char *id, time_t when); +double rep_hist_get_stability(const char *id, time_t when); +double rep_hist_get_weighted_fractional_uptime(const char *id, time_t when); +long rep_hist_get_weighted_time_known(const char *id, time_t when); +int rep_hist_have_measured_enough_stability(void); +const char *rep_hist_get_router_stability_doc(time_t now); + +void rep_hist_note_used_port(time_t now, uint16_t port); +smartlist_t *rep_hist_get_predicted_ports(time_t now); +void rep_hist_note_used_resolve(time_t now); +void rep_hist_note_used_internal(time_t now, int need_uptime, + int need_capacity); +int rep_hist_get_predicted_internal(time_t now, int *need_uptime, + int *need_capacity); + +int any_predicted_circuits(time_t now); +int rep_hist_circbuilding_dormant(time_t now); + +void note_crypto_pk_op(pk_op_t operation); +void dump_pk_ops(int severity); + +void rep_hist_free_all(void); + +void rep_hist_exit_stats_init(time_t now); +void rep_hist_reset_exit_stats(time_t now); +void rep_hist_exit_stats_term(void); +char *rep_hist_format_exit_stats(time_t now); +time_t rep_hist_exit_stats_write(time_t now); +void rep_hist_note_exit_bytes(uint16_t port, size_t num_written, + size_t num_read); +void rep_hist_note_exit_stream_opened(uint16_t port); + +void rep_hist_buffer_stats_init(time_t now); +void rep_hist_buffer_stats_add_circ(circuit_t *circ, + time_t end_of_interval); +time_t rep_hist_buffer_stats_write(time_t now); +void rep_hist_buffer_stats_term(void); + +#endif + diff --git a/src/or/router.c b/src/or/router.c index 52bfa64e4..65afd49f7 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -7,6 +7,24 @@ #define ROUTER_PRIVATE #include "or.h" +#include "circuitlist.h" +#include "circuituse.h" +#include "config.h" +#include "connection.h" +#include "control.h" +#include "directory.h" +#include "dirserv.h" +#include "dns.h" +#include "geoip.h" +#include "hibernate.h" +#include "main.h" +#include "networkstatus.h" +#include "policies.h" +#include "relay.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" /** * \file router.c @@ -31,11 +49,15 @@ static crypto_pk_env_t *onionkey=NULL; /** Previous private onionskin decryption key: used to decode CREATE cells * generated by clients that have an older version of our descriptor. */ static crypto_pk_env_t *lastonionkey=NULL; -/** Private "identity key": used to sign directory info and TLS +/** Private server "identity key": used to sign directory info and TLS * certificates. Never changes. */ -static crypto_pk_env_t *identitykey=NULL; -/** Digest of identitykey. */ -static char identitykey_digest[DIGEST_LEN]; +static crypto_pk_env_t *server_identitykey=NULL; +/** Digest of server_identitykey. */ +static char server_identitykey_digest[DIGEST_LEN]; +/** Private client "identity key": used to sign bridges' and clients' + * outbound TLS certificates. Regenerated on startup and on IP address + * change. */ +static crypto_pk_env_t *client_identitykey=NULL; /** Signing key used for v3 directory material; only set for authorities. */ static crypto_pk_env_t *authority_signing_key = NULL; /** Key certificate to authenticate v3 directory material; only set for @@ -61,8 +83,7 @@ static void set_onion_key(crypto_pk_env_t *k) { tor_mutex_acquire(key_lock); - if (onionkey) - crypto_free_pk_env(onionkey); + crypto_free_pk_env(onionkey); onionkey = k; onionkey_set_at = time(NULL); tor_mutex_release(key_lock); @@ -106,32 +127,78 @@ get_onion_key_set_at(void) return onionkey_set_at; } -/** Set the current identity key to k. +/** Set the current server identity key to <b>k</b>. */ void -set_identity_key(crypto_pk_env_t *k) +set_server_identity_key(crypto_pk_env_t *k) { - if (identitykey) - crypto_free_pk_env(identitykey); - identitykey = k; - crypto_pk_get_digest(identitykey, identitykey_digest); + crypto_free_pk_env(server_identitykey); + server_identitykey = k; + crypto_pk_get_digest(server_identitykey, server_identitykey_digest); } -/** Returns the current identity key; requires that the identity key has been - * set. +/** Make sure that we have set up our identity keys to match or not match as + * appropriate, and die with an assertion if we have not. */ +static void +assert_identity_keys_ok(void) +{ + tor_assert(client_identitykey); + if (public_server_mode(get_options())) { + /* assert that we have set the client and server keys to be equal */ + tor_assert(server_identitykey); + tor_assert(0==crypto_pk_cmp_keys(client_identitykey, server_identitykey)); + } else { + /* assert that we have set the client and server keys to be unequal */ + if (server_identitykey) + tor_assert(0!=crypto_pk_cmp_keys(client_identitykey, + server_identitykey)); + } +} + +/** Returns the current server identity key; requires that the key has + * been set, and that we are running as a Tor server. */ crypto_pk_env_t * -get_identity_key(void) +get_server_identity_key(void) { - tor_assert(identitykey); - return identitykey; + tor_assert(server_identitykey); + tor_assert(server_mode(get_options())); + assert_identity_keys_ok(); + return server_identitykey; } -/** Return true iff the identity key has been set. */ +/** Return true iff the server identity key has been set. */ int -identity_key_is_set(void) +server_identity_key_is_set(void) +{ + return server_identitykey != NULL; +} + +/** Set the current client identity key to <b>k</b>. + */ +void +set_client_identity_key(crypto_pk_env_t *k) +{ + crypto_free_pk_env(client_identitykey); + client_identitykey = k; +} + +/** Returns the current client identity key for use on outgoing TLS + * connections; requires that the key has been set. + */ +crypto_pk_env_t * +get_tlsclient_identity_key(void) { - return identitykey != NULL; + tor_assert(client_identitykey); + assert_identity_keys_ok(); + return client_identitykey; +} + +/** Return true iff the client identity key has been set. */ +int +client_identity_key_is_set(void) +{ + return client_identitykey != NULL; } /** Return the key certificate for this v3 (voting) authority, or NULL @@ -201,8 +268,7 @@ rotate_onion_key(void) } log_info(LD_GENERAL, "Rotating onion key"); tor_mutex_acquire(key_lock); - if (lastonionkey) - crypto_free_pk_env(lastonionkey); + crypto_free_pk_env(lastonionkey); lastonionkey = onionkey; onionkey = prkey; now = time(NULL); @@ -331,10 +397,9 @@ load_authority_keyset(int legacy, crypto_pk_env_t **key_out, goto done; } - if (*key_out) - crypto_free_pk_env(*key_out); - if (*cert_out) - authority_cert_free(*cert_out); + crypto_free_pk_env(*key_out); + authority_cert_free(*cert_out); + *key_out = signing_key; *cert_out = parsed; r = 0; @@ -344,10 +409,8 @@ load_authority_keyset(int legacy, crypto_pk_env_t **key_out, done: tor_free(fname); tor_free(cert); - if (signing_key) - crypto_free_pk_env(signing_key); - if (parsed) - authority_cert_free(parsed); + crypto_free_pk_env(signing_key); + authority_cert_free(parsed); return r; } @@ -442,7 +505,9 @@ init_keys(void) key_lock = tor_mutex_new(); /* There are a couple of paths that put us here before */ - if (crypto_global_init(get_options()->HardwareAccel)) { + if (crypto_global_init(get_options()->HardwareAccel, + get_options()->AccelName, + get_options()->AccelDir)) { log_err(LD_BUG, "Unable to initialize OpenSSL. Exiting."); return -1; } @@ -456,9 +521,12 @@ init_keys(void) crypto_free_pk_env(prkey); return -1; } - set_identity_key(prkey); - /* Create a TLS context; default the client nickname to "client". */ - if (tor_tls_context_new(get_identity_key(), MAX_SSL_KEY_LIFETIME) < 0) { + set_client_identity_key(prkey); + /* Create a TLS context. */ + if (tor_tls_context_init(0, + get_tlsclient_identity_key(), + NULL, + MAX_SSL_KEY_LIFETIME) < 0) { log_err(LD_GENERAL,"Error creating TLS context for Tor client."); return -1; } @@ -493,13 +561,28 @@ init_keys(void) } } - /* 1. Read identity key. Make it if none is found. */ + /* 1b. Read identity key. Make it if none is found. */ keydir = get_datadir_fname2("keys", "secret_id_key"); log_info(LD_GENERAL,"Reading/making identity key \"%s\"...",keydir); prkey = init_key_from_file(keydir, 1, LOG_ERR); tor_free(keydir); if (!prkey) return -1; - set_identity_key(prkey); + set_server_identity_key(prkey); + + /* 1c. If we are configured as a bridge, generate a client key; + * otherwise, set the server identity key as our client identity + * key. */ + if (public_server_mode(options)) { + set_client_identity_key(crypto_pk_dup_key(prkey)); /* set above */ + } else { + if (!(prkey = crypto_new_pk_env())) + return -1; + if (crypto_pk_generate_key(prkey)) { + crypto_free_pk_env(prkey); + return -1; + } + set_client_identity_key(prkey); + } /* 2. Read onion key. Make it if none is found. */ keydir = get_datadir_fname2("keys", "secret_onion_key"); @@ -536,18 +619,22 @@ init_keys(void) tor_free(keydir); /* 3. Initialize link key and TLS context. */ - if (tor_tls_context_new(get_identity_key(), MAX_SSL_KEY_LIFETIME) < 0) { + if (tor_tls_context_init(public_server_mode(options), + get_tlsclient_identity_key(), + get_server_identity_key(), + MAX_SSL_KEY_LIFETIME) < 0) { log_err(LD_GENERAL,"Error initializing TLS context"); return -1; } /* 4. Build our router descriptor. */ /* Must be called after keys are initialized. */ mydesc = router_get_my_descriptor(); - if (authdir_mode(options)) { + if (authdir_mode_handles_descs(options, ROUTER_PURPOSE_GENERAL)) { const char *m = NULL; routerinfo_t *ri; /* We need to add our own fingerprint so it gets recognized. */ - if (dirserv_add_own_fingerprint(options->Nickname, get_identity_key())) { + if (dirserv_add_own_fingerprint(options->Nickname, + get_server_identity_key())) { log_err(LD_GENERAL,"Error adding own fingerprint to approved set"); return -1; } @@ -568,7 +655,8 @@ init_keys(void) /* 5. Dump fingerprint to 'fingerprint' */ keydir = get_datadir_fname("fingerprint"); log_info(LD_GENERAL,"Dumping fingerprint to \"%s\"...",keydir); - if (crypto_pk_get_fingerprint(get_identity_key(), fingerprint, 0)<0) { + if (crypto_pk_get_fingerprint(get_server_identity_key(), + fingerprint, 0) < 0) { log_err(LD_GENERAL,"Error computing fingerprint"); tor_free(keydir); return -1; @@ -606,7 +694,7 @@ init_keys(void) return -1; } /* 6b. [authdirserver only] add own key to approved directories. */ - crypto_pk_get_digest(get_identity_key(), digest); + crypto_pk_get_digest(get_server_identity_key(), digest); type = ((options->V1AuthoritativeDir ? V1_AUTHORITY : NO_AUTHORITY) | (options->V2AuthoritativeDir ? V2_AUTHORITY : NO_AUTHORITY) | (options->V3AuthoritativeDir ? V3_AUTHORITY : NO_AUTHORITY) | @@ -762,9 +850,29 @@ consider_testing_reachability(int test_or, int test_dir) routerinfo_t *me = router_get_my_routerinfo(); int orport_reachable = check_whether_orport_reachable(); tor_addr_t addr; + or_options_t *options = get_options(); if (!me) return; + if (routerset_contains_router(options->ExcludeNodes, me) && + options->StrictNodes) { + /* If we've excluded ourself, and StrictNodes is set, we can't test + * ourself. */ + if (test_or || test_dir) { +#define SELF_EXCLUDED_WARN_INTERVAL 3600 + static ratelim_t warning_limit=RATELIM_INIT(SELF_EXCLUDED_WARN_INTERVAL); + char *msg; + if ((msg = rate_limit_log(&warning_limit, approx_time()))) { + log_warn(LD_CIRC, "Can't peform self-tests for this relay: we have " + "listed ourself in ExcludeNodes, and StrictNodes is set. " + "We cannot learn whether we are usable, and will not " + "be able to advertise ourself.%s", msg); + tor_free(msg); + } + } + return; + } + if (test_or && (!orport_reachable || !circuit_enough_testing_circs())) { log_info(LD_CIRC, "Testing %s of my ORPort: %s:%d.", !orport_reachable ? "reachability" : "bandwidth", @@ -952,6 +1060,28 @@ server_mode(or_options_t *options) return (options->ORPort != 0 || options->ORListenAddress); } +/** Return true iff we are trying to be a non-bridge server. + */ +int +public_server_mode(or_options_t *options) +{ + if (!server_mode(options)) return 0; + return (!options->BridgeRelay); +} + +/** Return true iff the combination of options in <b>options</b> and parameters + * in the consensus mean that we don't want to allow exits from circuits + * we got from addresses not known to be servers. */ +int +should_refuse_unknown_exits(or_options_t *options) +{ + if (options->RefuseUnknownExits_ != -1) { + return options->RefuseUnknownExits_; + } else { + return networkstatus_get_param(NULL, "refuseunknownexits", 1, 0, 1); + } +} + /** Remember if we've advertised ourselves to the dirservers. */ static int server_is_advertised=0; @@ -977,10 +1107,10 @@ set_server_advertised(int s) int proxy_mode(or_options_t *options) { - return (options->SocksPort != 0 || options->SocksListenAddress || - options->TransPort != 0 || options->TransListenAddress || - options->NatdPort != 0 || options->NatdListenAddress || - options->DNSPort != 0 || options->DNSListenAddress); + return (options->SocksPort != 0 || + options->TransPort != 0 || + options->NATDPort != 0 || + options->DNSPort != 0); } /** Decide if we're a publishable server. We are a publishable server if: @@ -1114,12 +1244,24 @@ router_compare_to_my_exit_policy(edge_connection_t *conn) desc_routerinfo->exit_policy) != ADDR_POLICY_ACCEPTED; } +/** Return true iff my exit policy is reject *:*. Return -1 if we don't + * have a descriptor */ +int +router_my_exit_policy_is_reject_star(void) +{ + if (!router_get_my_routerinfo()) /* make sure desc_routerinfo exists */ + return -1; + + return desc_routerinfo->policy_is_reject_star; +} + /** Return true iff I'm a server and <b>digest</b> is equal to - * my identity digest. */ + * my server identity key digest. */ int router_digest_is_me(const char *digest) { - return identitykey && !memcmp(identitykey_digest, digest, DIGEST_LEN); + return (server_identitykey && + !memcmp(server_identitykey_digest, digest, DIGEST_LEN)); } /** Return true iff I'm a server and <b>digest</b> is equal to @@ -1209,6 +1351,8 @@ static int router_guess_address_from_dir_headers(uint32_t *guess); int router_pick_published_address(or_options_t *options, uint32_t *addr) { + char buf[INET_NTOA_BUF_LEN]; + struct in_addr a; if (resolve_my_address(LOG_INFO, options, addr, NULL) < 0) { log_info(LD_CONFIG, "Could not determine our address locally. " "Checking if directory headers provide any hints."); @@ -1218,6 +1362,9 @@ router_pick_published_address(or_options_t *options, uint32_t *addr) return -1; } } + a.s_addr = htonl(*addr); + tor_inet_ntoa(&a, buf, sizeof(buf)); + log_info(LD_CONFIG,"Success: chose address '%s'.", buf); return 0; } @@ -1240,7 +1387,7 @@ router_rebuild_descriptor(int force) if (router_pick_published_address(options, &addr) < 0) { /* Stop trying to rebuild our descriptor every second. We'll - * learn that it's time to try again when server_has_changed_ip() + * learn that it's time to try again when ip_address_changed() * marks it dirty. */ desc_clean_since = time(NULL); return -1; @@ -1256,7 +1403,7 @@ router_rebuild_descriptor(int force) ri->cache_info.published_on = time(NULL); ri->onion_pkey = crypto_pk_dup_key(get_onion_key()); /* must invoke from * main thread */ - ri->identity_pkey = crypto_pk_dup_key(get_identity_key()); + ri->identity_pkey = crypto_pk_dup_key(get_server_identity_key()); if (crypto_pk_get_digest(ri->identity_pkey, ri->cache_info.identity_digest)<0) { routerinfo_free(ri); @@ -1273,9 +1420,16 @@ router_rebuild_descriptor(int force) ri->bandwidthcapacity = hibernating ? 0 : rep_hist_bandwidth_assess(); - policies_parse_exit_policy(options->ExitPolicy, &ri->exit_policy, - options->ExitPolicyRejectPrivate, - ri->address); + if (dns_seems_to_be_broken() || has_dns_init_failed()) { + /* DNS is screwed up; don't claim to be an exit. */ + policies_exit_policy_append_reject_star(&ri->exit_policy); + } else { + policies_parse_exit_policy(options->ExitPolicy, &ri->exit_policy, + options->ExitPolicyRejectPrivate, + ri->address, !options->BridgeRelay); + } + ri->policy_is_reject_star = + policy_is_reject_star(ri->exit_policy); if (desc_routerinfo) { /* inherit values */ ri->is_valid = desc_routerinfo->is_valid; @@ -1346,26 +1500,30 @@ router_rebuild_descriptor(int force) ei->cache_info.published_on = ri->cache_info.published_on; memcpy(ei->cache_info.identity_digest, ri->cache_info.identity_digest, DIGEST_LEN); - ei->cache_info.signed_descriptor_body = tor_malloc(8192); - if (extrainfo_dump_to_string(ei->cache_info.signed_descriptor_body, 8192, - ei, get_identity_key()) < 0) { + if (extrainfo_dump_to_string(&ei->cache_info.signed_descriptor_body, + ei, get_server_identity_key()) < 0) { log_warn(LD_BUG, "Couldn't generate extra-info descriptor."); - routerinfo_free(ri); extrainfo_free(ei); - return -1; + ei = NULL; + } else { + ei->cache_info.signed_descriptor_len = + strlen(ei->cache_info.signed_descriptor_body); + router_get_extrainfo_hash(ei->cache_info.signed_descriptor_body, + ei->cache_info.signed_descriptor_digest); } - ei->cache_info.signed_descriptor_len = - strlen(ei->cache_info.signed_descriptor_body); - router_get_extrainfo_hash(ei->cache_info.signed_descriptor_body, - ei->cache_info.signed_descriptor_digest); /* Now finish the router descriptor. */ - memcpy(ri->cache_info.extra_info_digest, - ei->cache_info.signed_descriptor_digest, - DIGEST_LEN); + if (ei) { + memcpy(ri->cache_info.extra_info_digest, + ei->cache_info.signed_descriptor_digest, + DIGEST_LEN); + } else { + /* ri was allocated with tor_malloc_zero, so there is no need to + * zero ri->cache_info.extra_info_digest here. */ + } ri->cache_info.signed_descriptor_body = tor_malloc(8192); if (router_dump_router_to_string(ri->cache_info.signed_descriptor_body, 8192, - ri, get_identity_key())<0) { + ri, get_server_identity_key()) < 0) { log_warn(LD_BUG, "Couldn't generate router descriptor."); routerinfo_free(ri); extrainfo_free(ei); @@ -1380,7 +1538,7 @@ router_rebuild_descriptor(int force) /* Let bridges serve their own descriptors unencrypted, so they can * pass reachability testing. (If they want to be harder to notice, * they can always leave the DirPort off). */ - if (!options->BridgeRelay) + if (ei && !options->BridgeRelay) ei->cache_info.send_unencrypted = 1; router_get_router_hash(ri->cache_info.signed_descriptor_body, @@ -1389,13 +1547,13 @@ router_rebuild_descriptor(int force) routerinfo_set_country(ri); - tor_assert(! routerinfo_incompatible_with_extrainfo(ri, ei, NULL, NULL)); + if (ei) { + tor_assert(! routerinfo_incompatible_with_extrainfo(ri, ei, NULL, NULL)); + } - if (desc_routerinfo) - routerinfo_free(desc_routerinfo); + routerinfo_free(desc_routerinfo); desc_routerinfo = ri; - if (desc_extrainfo) - extrainfo_free(desc_extrainfo); + extrainfo_free(desc_extrainfo); desc_extrainfo = ei; desc_clean_since = time(NULL); @@ -1606,6 +1764,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, char digest[DIGEST_LEN]; char published[ISO_TIME_LEN+1]; char fingerprint[FINGERPRINT_LEN+1]; + int has_extra_info_digest; char extra_info_digest[HEX_DIGEST_LEN+1]; size_t onion_pkeylen, identity_pkeylen; size_t written; @@ -1634,7 +1793,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, return -1; } - /* PEM-encode the identity key key */ + /* PEM-encode the identity key */ if (crypto_pk_write_public_key_to_string(router->identity_pkey, &identity_pkey,&identity_pkeylen)<0) { log_warn(LD_BUG,"write identity_pkey to string failed!"); @@ -1656,8 +1815,13 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, family_line = tor_strdup(""); } - base16_encode(extra_info_digest, sizeof(extra_info_digest), - router->cache_info.extra_info_digest, DIGEST_LEN); + has_extra_info_digest = + ! tor_digest_is_zero(router->cache_info.extra_info_digest); + + if (has_extra_info_digest) { + base16_encode(extra_info_digest, sizeof(extra_info_digest), + router->cache_info.extra_info_digest, DIGEST_LEN); + } /* Generate the easy portion of the router descriptor. */ result = tor_snprintf(s, maxlen, @@ -1668,7 +1832,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, "opt fingerprint %s\n" "uptime %ld\n" "bandwidth %d %d %d\n" - "opt extra-info-digest %s\n%s" + "%s%s%s%s" "onion-key\n%s" "signing-key\n%s" "%s%s%s%s", @@ -1683,7 +1847,9 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, (int) router->bandwidthrate, (int) router->bandwidthburst, (int) router->bandwidthcapacity, - extra_info_digest, + has_extra_info_digest ? "opt extra-info-digest " : "", + has_extra_info_digest ? extra_info_digest : "", + has_extra_info_digest ? "\n" : "", options->DownloadExtraInfo ? "opt caches-extra-info\n" : "", onion_pkey, identity_pkey, family_line, @@ -1715,9 +1881,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, } /* Write the exit policy to the end of 's'. */ - if (dns_seems_to_be_broken() || has_dns_init_failed() || - !router->exit_policy || !smartlist_len(router->exit_policy)) { - /* DNS is screwed up; don't claim to be an exit. */ + if (!router->exit_policy || !smartlist_len(router->exit_policy)) { strlcat(s+written, "reject *:*\n", maxlen-written); written += strlen("reject *:*\n"); tmpe = NULL; @@ -1740,7 +1904,8 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, } } - if (written+256 > maxlen) { /* Not enough room for signature. */ + if (written + DIROBJ_MAX_SIG_LEN > maxlen) { + /* Not enough room for signature. */ log_warn(LD_BUG,"not enough room left in descriptor for signature!"); return -1; } @@ -1755,7 +1920,7 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, note_crypto_pk_op(SIGN_RTR); if (router_append_dirobj_signature(s+written,maxlen-written, - digest,ident_key)<0) { + digest,DIGEST_LEN,ident_key)<0) { log_warn(LD_BUG, "Couldn't sign router descriptor"); return -1; } @@ -1790,11 +1955,62 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, return (int)written+1; } -/** Write the contents of <b>extrainfo</b> to the <b>maxlen</b>-byte string - * <b>s</b>, signing them with <b>ident_key</b>. Return 0 on success, - * negative on failure. */ +/** Load the contents of <b>filename</b>, find the last line starting with + * <b>end_line</b>, ensure that its timestamp is not more than 25 hours in + * the past or more than 1 hour in the future with respect to <b>now</b>, + * and write the file contents starting with that line to *<b>out</b>. + * Return 1 for success, 0 if the file does not exist, or -1 if the file + * does not contain a line matching these criteria or other failure. */ +static int +load_stats_file(const char *filename, const char *end_line, time_t now, + char **out) +{ + int r = -1; + char *fname = get_datadir_fname(filename); + char *contents, *start = NULL, *tmp, timestr[ISO_TIME_LEN+1]; + time_t written; + switch (file_status(fname)) { + case FN_FILE: + /* X022 Find an alternative to reading the whole file to memory. */ + if ((contents = read_file_to_str(fname, 0, NULL))) { + tmp = strstr(contents, end_line); + /* Find last block starting with end_line */ + while (tmp) { + start = tmp; + tmp = strstr(tmp + 1, end_line); + } + if (!start) + goto notfound; + if (strlen(start) < strlen(end_line) + 1 + sizeof(timestr)) + goto notfound; + strlcpy(timestr, start + 1 + strlen(end_line), sizeof(timestr)); + if (parse_iso_time(timestr, &written) < 0) + goto notfound; + if (written < now - (25*60*60) || written > now + (1*60*60)) + goto notfound; + *out = tor_strdup(start); + r = 1; + } + notfound: + tor_free(contents); + break; + case FN_NOENT: + r = 0; + break; + case FN_ERROR: + case FN_DIR: + default: + break; + } + tor_free(fname); + return r; +} + +/** Write the contents of <b>extrainfo</b> and aggregated statistics to + * *<b>s_out</b>, signing them with <b>ident_key</b>. Return 0 on + * success, negative on failure. */ int -extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo, +extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, crypto_pk_env_t *ident_key) { or_options_t *options = get_options(); @@ -1803,87 +2019,128 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo, char digest[DIGEST_LEN]; char *bandwidth_usage; int result; - size_t len; + static int write_stats_to_extrainfo = 1; + char sig[DIROBJ_MAX_SIG_LEN+1]; + char *s, *pre, *contents, *cp, *s_dup = NULL; + time_t now = time(NULL); + smartlist_t *chunks = smartlist_create(); + extrainfo_t *ei_tmp = NULL; base16_encode(identity, sizeof(identity), extrainfo->cache_info.identity_digest, DIGEST_LEN); format_iso_time(published, extrainfo->cache_info.published_on); - bandwidth_usage = rep_hist_get_bandwidth_lines(1); + bandwidth_usage = rep_hist_get_bandwidth_lines(); - result = tor_snprintf(s, maxlen, - "extra-info %s %s\n" - "published %s\n%s", - extrainfo->nickname, identity, - published, bandwidth_usage); + tor_asprintf(&pre, "extra-info %s %s\npublished %s\n%s", + extrainfo->nickname, identity, + published, bandwidth_usage); tor_free(bandwidth_usage); - if (result<0) - return -1; + smartlist_add(chunks, pre); + + if (options->ExtraInfoStatistics && write_stats_to_extrainfo) { + log_info(LD_GENERAL, "Adding stats to extra-info descriptor."); + if (options->DirReqStatistics && + load_stats_file("stats"PATH_SEPARATOR"dirreq-stats", + "dirreq-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->EntryStatistics && + load_stats_file("stats"PATH_SEPARATOR"entry-stats", + "entry-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->CellStatistics && + load_stats_file("stats"PATH_SEPARATOR"buffer-stats", + "cell-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->ExitPortStatistics && + load_stats_file("stats"PATH_SEPARATOR"exit-stats", + "exit-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + } - if (should_record_bridge_info(options)) { - char *geoip_summary = extrainfo_get_client_geoip_summary(time(NULL)); - if (geoip_summary) { - char geoip_start[ISO_TIME_LEN+1]; - format_iso_time(geoip_start, geoip_get_history_start()); - result = tor_snprintf(s+strlen(s), maxlen-strlen(s), - "geoip-start-time %s\n" - "geoip-client-origins %s\n", - geoip_start, geoip_summary); - control_event_clients_seen(geoip_start, geoip_summary); - tor_free(geoip_summary); - if (result<0) - return -1; + if (should_record_bridge_info(options) && write_stats_to_extrainfo) { + const char *bridge_stats = geoip_get_bridge_stats_extrainfo(now); + if (bridge_stats) { + smartlist_add(chunks, tor_strdup(bridge_stats)); } } - len = strlen(s); - strlcat(s+len, "router-signature\n", maxlen-len); - len += strlen(s+len); - if (router_get_extrainfo_hash(s, digest)<0) - return -1; - if (router_append_dirobj_signature(s+len, maxlen-len, digest, ident_key)<0) - return -1; + smartlist_add(chunks, tor_strdup("router-signature\n")); + s = smartlist_join_strings(chunks, "", 0, NULL); + + while (strlen(s) > MAX_EXTRAINFO_UPLOAD_SIZE - DIROBJ_MAX_SIG_LEN) { + /* So long as there are at least two chunks (one for the initial + * extra-info line and one for the router-signature), we can keep removing + * things. */ + if (smartlist_len(chunks) > 2) { + /* We remove the next-to-last element (remember, len-1 is the last + element), since we need to keep the router-signature element. */ + int idx = smartlist_len(chunks) - 2; + char *e = smartlist_get(chunks, idx); + smartlist_del_keeporder(chunks, idx); + log_warn(LD_GENERAL, "We just generated an extra-info descriptor " + "with statistics that exceeds the 50 KB " + "upload limit. Removing last added " + "statistics."); + tor_free(e); + tor_free(s); + s = smartlist_join_strings(chunks, "", 0, NULL); + } else { + log_warn(LD_BUG, "We just generated an extra-info descriptors that " + "exceeds the 50 KB upload limit."); + goto err; + } + } -#ifdef DEBUG_ROUTER_DUMP_ROUTER_TO_STRING - { - char *cp, *s_dup; - extrainfo_t *ei_tmp; - cp = s_dup = tor_strdup(s); - ei_tmp = extrainfo_parse_entry_from_string(cp, NULL, 1, NULL); - if (!ei_tmp) { - log_err(LD_BUG, - "We just generated an extrainfo descriptor we can't parse."); - log_err(LD_BUG, "Descriptor was: <<%s>>", s); - tor_free(s_dup); - return -1; + memset(sig, 0, sizeof(sig)); + if (router_get_extrainfo_hash(s, digest) < 0 || + router_append_dirobj_signature(sig, sizeof(sig), digest, DIGEST_LEN, + ident_key) < 0) { + log_warn(LD_BUG, "Could not append signature to extra-info " + "descriptor."); + goto err; + } + smartlist_add(chunks, tor_strdup(sig)); + tor_free(s); + s = smartlist_join_strings(chunks, "", 0, NULL); + + cp = s_dup = tor_strdup(s); + ei_tmp = extrainfo_parse_entry_from_string(cp, NULL, 1, NULL); + if (!ei_tmp) { + if (write_stats_to_extrainfo) { + log_warn(LD_GENERAL, "We just generated an extra-info descriptor " + "with statistics that we can't parse. Not " + "adding statistics to this or any future " + "extra-info descriptors."); + write_stats_to_extrainfo = 0; + result = extrainfo_dump_to_string(s_out, extrainfo, ident_key); + goto done; + } else { + log_warn(LD_BUG, "We just generated an extrainfo descriptor we " + "can't parse."); + goto err; } - tor_free(s_dup); - extrainfo_free(ei_tmp); } -#endif - return (int)strlen(s)+1; -} + *s_out = s; + s = NULL; /* prevent free */ + result = 0; + goto done; -/** Wrapper function for geoip_get_client_history(). It first discards - * any items in the client history that are too old -- it dumps anything - * more than 48 hours old, but it only considers whether to dump at most - * once per 48 hours, so we aren't too precise to an observer (see also - * r14780). - */ -char * -extrainfo_get_client_geoip_summary(time_t now) -{ - static time_t last_purged_at = 0; - int geoip_purge_interval = 48*60*60; -#ifdef ENABLE_GEOIP_STATS - if (get_options()->DirRecordUsageByCountry) - geoip_purge_interval = get_options()->DirRecordUsageRetainIPs; -#endif - if (now > last_purged_at+geoip_purge_interval) { - geoip_remove_old_clients(now-geoip_purge_interval); - last_purged_at = now; - } - return geoip_get_client_history(now, GEOIP_CLIENT_CONNECT); + err: + result = -1; + + done: + tor_free(s); + SMARTLIST_FOREACH(chunks, char *, cp, tor_free(cp)); + smartlist_free(chunks); + tor_free(s_dup); + extrainfo_free(ei_tmp); + + return result; } /** Return true iff <b>s</b> is a legally valid server nickname. */ @@ -2010,26 +2267,17 @@ router_purpose_from_string(const char *s) void router_free_all(void) { - if (onionkey) - crypto_free_pk_env(onionkey); - if (lastonionkey) - crypto_free_pk_env(lastonionkey); - if (identitykey) - crypto_free_pk_env(identitykey); - if (key_lock) - tor_mutex_free(key_lock); - if (desc_routerinfo) - routerinfo_free(desc_routerinfo); - if (desc_extrainfo) - extrainfo_free(desc_extrainfo); - if (authority_signing_key) - crypto_free_pk_env(authority_signing_key); - if (authority_key_certificate) - authority_cert_free(authority_key_certificate); - if (legacy_signing_key) - crypto_free_pk_env(legacy_signing_key); - if (legacy_key_certificate) - authority_cert_free(legacy_key_certificate); + crypto_free_pk_env(onionkey); + crypto_free_pk_env(lastonionkey); + crypto_free_pk_env(server_identitykey); + crypto_free_pk_env(client_identitykey); + tor_mutex_free(key_lock); + routerinfo_free(desc_routerinfo); + extrainfo_free(desc_extrainfo); + crypto_free_pk_env(authority_signing_key); + authority_cert_free(authority_key_certificate); + crypto_free_pk_env(legacy_signing_key); + authority_cert_free(legacy_key_certificate); if (warned_nonexistent_family) { SMARTLIST_FOREACH(warned_nonexistent_family, char *, cp, tor_free(cp)); diff --git a/src/or/router.h b/src/or/router.h new file mode 100644 index 000000000..5e021f6fe --- /dev/null +++ b/src/or/router.h @@ -0,0 +1,101 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file router.h + * \brief Header file for router.c. + **/ + +#ifndef _TOR_ROUTER_H +#define _TOR_ROUTER_H + +crypto_pk_env_t *get_onion_key(void); +time_t get_onion_key_set_at(void); +void set_server_identity_key(crypto_pk_env_t *k); +crypto_pk_env_t *get_server_identity_key(void); +int server_identity_key_is_set(void); +void set_client_identity_key(crypto_pk_env_t *k); +crypto_pk_env_t *get_tlsclient_identity_key(void); +int client_identity_key_is_set(void); +authority_cert_t *get_my_v3_authority_cert(void); +crypto_pk_env_t *get_my_v3_authority_signing_key(void); +authority_cert_t *get_my_v3_legacy_cert(void); +crypto_pk_env_t *get_my_v3_legacy_signing_key(void); +void dup_onion_keys(crypto_pk_env_t **key, crypto_pk_env_t **last); +void rotate_onion_key(void); +crypto_pk_env_t *init_key_from_file(const char *fname, int generate, + int severity); +void v3_authority_check_key_expiry(void); + +int init_keys(void); + +int check_whether_orport_reachable(void); +int check_whether_dirport_reachable(void); +void consider_testing_reachability(int test_or, int test_dir); +void router_orport_found_reachable(void); +void router_dirport_found_reachable(void); +void router_perform_bandwidth_test(int num_circs, time_t now); + +int authdir_mode(or_options_t *options); +int authdir_mode_v1(or_options_t *options); +int authdir_mode_v2(or_options_t *options); +int authdir_mode_v3(or_options_t *options); +int authdir_mode_any_main(or_options_t *options); +int authdir_mode_any_nonhidserv(or_options_t *options); +int authdir_mode_handles_descs(or_options_t *options, int purpose); +int authdir_mode_publishes_statuses(or_options_t *options); +int authdir_mode_tests_reachability(or_options_t *options); +int authdir_mode_bridge(or_options_t *options); + +int server_mode(or_options_t *options); +int public_server_mode(or_options_t *options); +int advertised_server_mode(void); +int proxy_mode(or_options_t *options); +void consider_publishable_server(int force); +int should_refuse_unknown_exits(or_options_t *options); + +void router_upload_dir_desc_to_dirservers(int force); +void mark_my_descriptor_dirty_if_older_than(time_t when); +void mark_my_descriptor_dirty(void); +void check_descriptor_bandwidth_changed(time_t now); +void check_descriptor_ipaddress_changed(time_t now); +void router_new_address_suggestion(const char *suggestion, + const dir_connection_t *d_conn); +int router_compare_to_my_exit_policy(edge_connection_t *conn); +int router_my_exit_policy_is_reject_star(void); +routerinfo_t *router_get_my_routerinfo(void); +extrainfo_t *router_get_my_extrainfo(void); +const char *router_get_my_descriptor(void); +int router_digest_is_me(const char *digest); +int router_extrainfo_digest_is_me(const char *digest); +int router_is_me(routerinfo_t *router); +int router_fingerprint_is_me(const char *fp); +int router_pick_published_address(or_options_t *options, uint32_t *addr); +int router_rebuild_descriptor(int force); +int router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, + crypto_pk_env_t *ident_key); +int extrainfo_dump_to_string(char **s, extrainfo_t *extrainfo, + crypto_pk_env_t *ident_key); +int is_legal_nickname(const char *s); +int is_legal_nickname_or_hexdigest(const char *s); +int is_legal_hexdigest(const char *s); +void router_get_verbose_nickname(char *buf, const routerinfo_t *router); +void routerstatus_get_verbose_nickname(char *buf, + const routerstatus_t *router); +void router_reset_warnings(void); +void router_reset_reachability(void); +void router_free_all(void); + +const char *router_purpose_to_string(uint8_t p); +uint8_t router_purpose_from_string(const char *s); + +#ifdef ROUTER_PRIVATE +/* Used only by router.c and test.c */ +void get_platform_str(char *platform, size_t len); +#endif + +#endif + diff --git a/src/or/routerlist.c b/src/or/routerlist.c index fb8fb8815..f567ccdf3 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -12,6 +12,25 @@ **/ #include "or.h" +#include "circuitbuild.h" +#include "config.h" +#include "connection.h" +#include "control.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "geoip.h" +#include "hibernate.h" +#include "main.h" +#include "networkstatus.h" +#include "policies.h" +#include "reasons.h" +#include "rendcommon.h" +#include "rendservice.h" +#include "rephist.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" // #define DEBUG_ROUTERLIST @@ -26,8 +45,8 @@ static void mark_all_trusteddirservers_up(void); static int router_nickname_matches(routerinfo_t *router, const char *nickname); static void trusted_dir_server_free(trusted_dir_server_t *ds); static void launch_router_descriptor_downloads(smartlist_t *downloadable, + routerstatus_t *source, time_t now); -static void update_consensus_router_descriptor_downloads(time_t now); static int signed_desc_digest_is_recognized(signed_descriptor_t *desc); static void update_router_have_minimum_dir_info(void); static const char *signed_descriptor_get_body_impl(signed_descriptor_t *desc, @@ -156,21 +175,24 @@ already_have_cert(authority_cert_t *cert) /** Load a bunch of new key certificates from the string <b>contents</b>. If * <b>from_store</b> is true, the certificates are from the cache, and we - * don't need to flush them to disk. If <b>from_store</b> is false, we need - * to flush any changed certificates to disk. Return 0 on success, -1 on - * failure. */ + * don't need to flush them to disk. If <b>flush</b> is true, we need + * to flush any changed certificates to disk now. Return 0 on success, -1 + * if any certs fail to parse. */ int trusted_dirs_load_certs_from_string(const char *contents, int from_store, int flush) { trusted_dir_server_t *ds; const char *s, *eos; + int failure_code = 0; for (s = contents; *s; s = eos) { authority_cert_t *cert = authority_cert_parse_from_string(s, &eos); cert_list_t *cl; - if (!cert) + if (!cert) { + failure_code = -1; break; + } ds = trusteddirserver_get_by_v3_auth_digest( cert->cache_info.identity_digest); log_debug(LD_DIR, "Parsed certificate for %s", @@ -181,15 +203,15 @@ trusted_dirs_load_certs_from_string(const char *contents, int from_store, log_info(LD_DIR, "Skipping %s certificate for %s that we " "already have.", from_store ? "cached" : "downloaded", - ds ? ds->nickname : "??"); + ds ? ds->nickname : "an old or new authority"); /* a duplicate on a download should be treated as a failure, since it * probably means we wanted a different secret key or we are trying to * replace an expired cert that has not in fact been updated. */ if (!from_store) { - log_warn(LD_DIR, "Got a certificate for %s that we already have. " - "Maybe they haven't updated it. Waiting for a while.", - ds ? ds->nickname : "??"); + log_warn(LD_DIR, "Got a certificate for %s, but we already have it. " + "Maybe they haven't updated it. Waiting for a while.", + ds ? ds->nickname : "an old or new authority"); authority_cert_dl_failed(cert->cache_info.identity_digest, 404); } @@ -224,7 +246,7 @@ trusted_dirs_load_certs_from_string(const char *contents, int from_store, ds->dir_port != cert->dir_port)) { char *a = tor_dup_ip(cert->addr); log_notice(LD_DIR, "Updating address for directory authority %s " - "from %s:%d to %s:%d based on in certificate.", + "from %s:%d to %s:%d based on certificate.", ds->nickname, ds->address, (int)ds->dir_port, a, cert->dir_port); tor_free(a); @@ -241,8 +263,11 @@ trusted_dirs_load_certs_from_string(const char *contents, int from_store, if (flush) trusted_dirs_flush_certs_to_disk(); + /* call this even if failure_code is <0, since some certs might have + * succeeded. */ networkstatus_note_certs_arrived(); - return 0; + + return failure_code; } /** Save all v3 key certificates to the cached-certs file. */ @@ -303,7 +328,7 @@ trusted_dirs_remove_old_certs(void) time_t cert_published; if (newest == cert) continue; - expired = ftime_definitely_after(now, cert->expires); + expired = now > cert->expires; cert_published = cert->cache_info.published_on; /* Store expired certs for 48 hours after a newer arrives; */ @@ -415,6 +440,23 @@ authority_cert_dl_failed(const char *id_digest, int status) download_status_failed(&cl->dl_status, status); } +/** Return true iff when we've been getting enough failures when trying to + * download the certificate with ID digest <b>id_digest</b> that we're willing + * to start bugging the user about it. */ +int +authority_cert_dl_looks_uncertain(const char *id_digest) +{ +#define N_AUTH_CERT_DL_FAILURES_TO_BUG_USER 2 + cert_list_t *cl; + int n_failures; + if (!trusted_dir_certs || + !(cl = digestmap_get(trusted_dir_certs, id_digest))) + return 0; + + n_failures = download_status_get_n_failures(&cl->dl_status); + return n_failures >= N_AUTH_CERT_DL_FAILURES_TO_BUG_USER; +} + /** How many times will we try to fetch a certificate before giving up? */ #define MAX_CERT_DL_FAILURES 8 @@ -442,17 +484,18 @@ authority_certs_fetch_missing(networkstatus_t *status, time_t now) list_pending_downloads(pending, DIR_PURPOSE_FETCH_CERTIFICATE, "fp/"); if (status) { - SMARTLIST_FOREACH(status->voters, networkstatus_voter_info_t *, voter, - { - if (tor_digest_is_zero(voter->signing_key_digest)) - continue; /* This authority never signed this consensus, so don't - * go looking for a cert with key digest 0000000000. */ - if (!cache && - !trusteddirserver_get_by_v3_auth_digest(voter->identity_digest)) - continue; /* We are not a cache, and we don't know this authority.*/ - cl = get_cert_list(voter->identity_digest); + SMARTLIST_FOREACH_BEGIN(status->voters, networkstatus_voter_info_t *, + voter) { + if (!smartlist_len(voter->sigs)) + continue; /* This authority never signed this consensus, so don't + * go looking for a cert with key digest 0000000000. */ + if (!cache && + !trusteddirserver_get_by_v3_auth_digest(voter->identity_digest)) + continue; /* We are not a cache, and we don't know this authority.*/ + cl = get_cert_list(voter->identity_digest); + SMARTLIST_FOREACH_BEGIN(voter->sigs, document_signature_t *, sig) { cert = authority_cert_get_by_digests(voter->identity_digest, - voter->signing_key_digest); + sig->signing_key_digest); if (cert) { if (now < cert->expires) download_status_reset(&cl->dl_status); @@ -463,37 +506,36 @@ authority_certs_fetch_missing(networkstatus_t *status, time_t now) !digestmap_get(pending, voter->identity_digest)) { log_notice(LD_DIR, "We're missing a certificate from authority " "with signing key %s: launching request.", - hex_str(voter->signing_key_digest, DIGEST_LEN)); - smartlist_add(missing_digests, voter->identity_digest); + hex_str(sig->signing_key_digest, DIGEST_LEN)); + smartlist_add(missing_digests, sig->identity_digest); } - }); + } SMARTLIST_FOREACH_END(sig); + } SMARTLIST_FOREACH_END(voter); } - SMARTLIST_FOREACH(trusted_dir_servers, trusted_dir_server_t *, ds, - { - int found = 0; - if (!(ds->type & V3_AUTHORITY)) - continue; - if (smartlist_digest_isin(missing_digests, ds->v3_identity_digest)) - continue; - cl = get_cert_list(ds->v3_identity_digest); - SMARTLIST_FOREACH(cl->certs, authority_cert_t *, cert, - { - if (!ftime_definitely_after(now, cert->expires)) { - /* It's not expired, and we weren't looking for something to - * verify a consensus with. Call it done. */ - download_status_reset(&cl->dl_status); - found = 1; - break; - } - }); - if (!found && - download_status_is_ready(&cl->dl_status, now,MAX_CERT_DL_FAILURES) && - !digestmap_get(pending, ds->v3_identity_digest)) { - log_notice(LD_DIR, "No current certificate known for authority %s; " - "launching request.", ds->nickname); - smartlist_add(missing_digests, ds->v3_identity_digest); + SMARTLIST_FOREACH_BEGIN(trusted_dir_servers, trusted_dir_server_t *, ds) { + int found = 0; + if (!(ds->type & V3_AUTHORITY)) + continue; + if (smartlist_digest_isin(missing_digests, ds->v3_identity_digest)) + continue; + cl = get_cert_list(ds->v3_identity_digest); + SMARTLIST_FOREACH(cl->certs, authority_cert_t *, cert, { + if (now < cert->expires) { + /* It's not expired, and we weren't looking for something to + * verify a consensus with. Call it done. */ + download_status_reset(&cl->dl_status); + found = 1; + break; } }); + if (!found && + download_status_is_ready(&cl->dl_status, now,MAX_CERT_DL_FAILURES) && + !digestmap_get(pending, ds->v3_identity_digest)) { + log_info(LD_DIR, "No current certificate known for authority %s; " + "launching request.", ds->nickname); + smartlist_add(missing_digests, ds->v3_identity_digest); + } + } SMARTLIST_FOREACH_END(ds); if (!smartlist_len(missing_digests)) { goto done; @@ -749,8 +791,7 @@ router_rebuild_store(int flags, desc_store_t *store) store->journal_len = 0; store->bytes_dropped = 0; done: - if (signed_descriptors) - smartlist_free(signed_descriptors); + smartlist_free(signed_descriptors); tor_free(fname); tor_free(fname_tmp); if (chunk_list) { @@ -968,8 +1009,9 @@ router_get_trusteddirserver_by_digest(const char *digest) return NULL; } -/** Return the trusted_dir_server_t for the directory authority whose identity - * key hashes to <b>digest</b>, or NULL if no such authority is known. +/** Return the trusted_dir_server_t for the directory authority whose + * v3 identity key hashes to <b>digest</b>, or NULL if no such authority + * is known. */ trusted_dir_server_t * trusteddirserver_get_by_v3_auth_digest(const char *digest) @@ -1028,6 +1070,7 @@ router_pick_trusteddirserver(authority_type_t type, int flags) static routerstatus_t * router_pick_directory_server_impl(authority_type_t type, int flags) { + or_options_t *options = get_options(); routerstatus_t *result; smartlist_t *direct, *tunnel; smartlist_t *trusted_direct, *trusted_tunnel; @@ -1037,10 +1080,13 @@ router_pick_directory_server_impl(authority_type_t type, int flags) int requireother = ! (flags & PDS_ALLOW_SELF); int fascistfirewall = ! (flags & PDS_IGNORE_FASCISTFIREWALL); int prefer_tunnel = (flags & _PDS_PREFER_TUNNELED_DIR_CONNS); + int try_excluding = 1, n_excluded = 0; if (!consensus) return NULL; + retry_without_exclude: + direct = smartlist_create(); tunnel = smartlist_create(); trusted_direct = smartlist_create(); @@ -1072,6 +1118,11 @@ router_pick_directory_server_impl(authority_type_t type, int flags) if ((type & EXTRAINFO_CACHE) && !router_supports_extrainfo(status->identity_digest, 0)) continue; + if (try_excluding && options->ExcludeNodes && + routerset_contains_routerstatus(options->ExcludeNodes, status)) { + ++n_excluded; + continue; + } /* XXXX IP6 proposal 118 */ tor_addr_from_ipv4h(&addr, status->addr); @@ -1089,9 +1140,10 @@ router_pick_directory_server_impl(authority_type_t type, int flags) } SMARTLIST_FOREACH_END(status); if (smartlist_len(tunnel)) { - result = routerstatus_sl_choose_by_bandwidth(tunnel); + result = routerstatus_sl_choose_by_bandwidth(tunnel, WEIGHT_FOR_DIR); } else if (smartlist_len(overloaded_tunnel)) { - result = routerstatus_sl_choose_by_bandwidth(overloaded_tunnel); + result = routerstatus_sl_choose_by_bandwidth(overloaded_tunnel, + WEIGHT_FOR_DIR); } else if (smartlist_len(trusted_tunnel)) { /* FFFF We don't distinguish between trusteds and overloaded trusteds * yet. Maybe one day we should. */ @@ -1099,9 +1151,10 @@ router_pick_directory_server_impl(authority_type_t type, int flags) * is a feature, but it could easily be a bug. -RD */ result = smartlist_choose(trusted_tunnel); } else if (smartlist_len(direct)) { - result = routerstatus_sl_choose_by_bandwidth(direct); + result = routerstatus_sl_choose_by_bandwidth(direct, WEIGHT_FOR_DIR); } else if (smartlist_len(overloaded_direct)) { - result = routerstatus_sl_choose_by_bandwidth(overloaded_direct); + result = routerstatus_sl_choose_by_bandwidth(overloaded_direct, + WEIGHT_FOR_DIR); } else { result = smartlist_choose(trusted_direct); } @@ -1111,6 +1164,15 @@ router_pick_directory_server_impl(authority_type_t type, int flags) smartlist_free(trusted_tunnel); smartlist_free(overloaded_direct); smartlist_free(overloaded_tunnel); + + if (result == NULL && try_excluding && !options->StrictNodes && n_excluded) { + /* If we got no result, and we are excluding nodes, and StrictNodes is + * not set, try again without excluding nodes. */ + try_excluding = 0; + n_excluded = 0; + goto retry_without_exclude; + } + return result; } @@ -1121,6 +1183,7 @@ static routerstatus_t * router_pick_trusteddirserver_impl(authority_type_t type, int flags, int *n_busy_out) { + or_options_t *options = get_options(); smartlist_t *direct, *tunnel; smartlist_t *overloaded_direct, *overloaded_tunnel; routerinfo_t *me = router_get_my_routerinfo(); @@ -1131,10 +1194,13 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, const int prefer_tunnel = (flags & _PDS_PREFER_TUNNELED_DIR_CONNS); const int no_serverdesc_fetching =(flags & PDS_NO_EXISTING_SERVERDESC_FETCH); int n_busy = 0; + int try_excluding = 1, n_excluded = 0; if (!trusted_dir_servers) return NULL; + retry_without_exclude: + direct = smartlist_create(); tunnel = smartlist_create(); overloaded_direct = smartlist_create(); @@ -1153,6 +1219,12 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, continue; if (requireother && me && router_digest_is_me(d->digest)) continue; + if (try_excluding && options->ExcludeNodes && + routerset_contains_routerstatus(options->ExcludeNodes, + &d->fake_status)) { + ++n_excluded; + continue; + } /* XXXX IP6 proposal 118 */ tor_addr_from_ipv4h(&addr, d->addr); @@ -1199,6 +1271,15 @@ router_pick_trusteddirserver_impl(authority_type_t type, int flags, smartlist_free(tunnel); smartlist_free(overloaded_direct); smartlist_free(overloaded_tunnel); + + if (result == NULL && try_excluding && !options->StrictNodes && n_excluded) { + /* If we got no result, and we are excluding nodes, and StrictNodes is + * not set, try again without excluding nodes. */ + try_excluding = 0; + n_excluded = 0; + goto retry_without_exclude; + } + return result; } @@ -1230,6 +1311,13 @@ mark_all_trusteddirservers_up(void) router_dir_info_changed(); } +/** Return true iff r1 and r2 have the same address and OR port. */ +int +routers_have_same_or_addr(const routerinfo_t *r1, const routerinfo_t *r2) +{ + return r1->addr == r2->addr && r1->or_port == r2->or_port; +} + /** Reset all internal variables used to count failed downloads of network * status objects. */ void @@ -1450,6 +1538,8 @@ routerlist_find_my_routerinfo(void) /** Find a router that's up, that has this IP address, and * that allows exit to this address:port, or return NULL if there * isn't a good one. + * Don't exit enclave to excluded relays -- it wouldn't actually + * hurt anything, but this way there are fewer confused users. */ routerinfo_t * router_find_exact_exit_enclave(const char *address, uint16_t port) @@ -1457,6 +1547,7 @@ router_find_exact_exit_enclave(const char *address, uint16_t port) uint32_t addr; struct in_addr in; tor_addr_t a; + or_options_t *options = get_options(); if (!tor_inet_aton(address, &in)) return NULL; /* it's not an IP already */ @@ -1469,7 +1560,8 @@ router_find_exact_exit_enclave(const char *address, uint16_t port) if (router->addr == addr && router->is_running && compare_tor_addr_to_addr_policy(&a, port, router->exit_policy) == - ADDR_POLICY_ACCEPTED) + ADDR_POLICY_ACCEPTED && + !routerset_contains_router(options->_ExcludeExitNodesUnion, router)) return router; }); return NULL; @@ -1521,6 +1613,29 @@ router_get_advertised_bandwidth_capped(routerinfo_t *router) return result; } +/** When weighting bridges, enforce these values as lower and upper + * bound for believable bandwidth, because there is no way for us + * to verify a bridge's bandwidth currently. */ +#define BRIDGE_MIN_BELIEVABLE_BANDWIDTH 20000 /* 20 kB/sec */ +#define BRIDGE_MAX_BELIEVABLE_BANDWIDTH 100000 /* 100 kB/sec */ + +/** Return the smaller of the router's configured BandwidthRate + * and its advertised capacity, making sure to stay within the + * interval between bridge-min-believe-bw and + * bridge-max-believe-bw. */ +static uint32_t +bridge_get_advertised_bandwidth_bounded(routerinfo_t *router) +{ + uint32_t result = router->bandwidthcapacity; + if (result > router->bandwidthrate) + result = router->bandwidthrate; + if (result > BRIDGE_MAX_BELIEVABLE_BANDWIDTH) + result = BRIDGE_MAX_BELIEVABLE_BANDWIDTH; + else if (result < BRIDGE_MIN_BELIEVABLE_BANDWIDTH) + result = BRIDGE_MIN_BELIEVABLE_BANDWIDTH; + return result; +} + /** Return bw*1000, unless bw*1000 would overflow, in which case return * INT32_MAX. */ static INLINE int32_t @@ -1531,6 +1646,218 @@ kb_to_bytes(uint32_t bw) /** Helper function: * choose a random element of smartlist <b>sl</b>, weighted by + * the advertised bandwidth of each element using the consensus + * bandwidth weights. + * + * If <b>statuses</b> is zero, then <b>sl</b> is a list of + * routerinfo_t's. Otherwise it's a list of routerstatus_t's. + * + * If <b>rule</b>==WEIGHT_FOR_EXIT. we're picking an exit node: consider all + * nodes' bandwidth equally regardless of their Exit status, since there may + * be some in the list because they exit to obscure ports. If + * <b>rule</b>==NO_WEIGHTING, we're picking a non-exit node: weight + * exit-node's bandwidth less depending on the smallness of the fraction of + * Exit-to-total bandwidth. If <b>rule</b>==WEIGHT_FOR_GUARD, we're picking a + * guard node: consider all guard's bandwidth equally. Otherwise, weight + * guards proportionally less. + */ +static void * +smartlist_choose_by_bandwidth_weights(smartlist_t *sl, + bandwidth_weight_rule_t rule, + int statuses) +{ + int64_t weight_scale; + int64_t rand_bw; + double Wg = -1, Wm = -1, We = -1, Wd = -1; + double Wgb = -1, Wmb = -1, Web = -1, Wdb = -1; + double weighted_bw = 0; + double *bandwidths; + double tmp = 0; + unsigned int i; + int have_unknown = 0; /* true iff sl contains element not in consensus. */ + + /* Can't choose exit and guard at same time */ + tor_assert(rule == NO_WEIGHTING || + rule == WEIGHT_FOR_EXIT || + rule == WEIGHT_FOR_GUARD || + rule == WEIGHT_FOR_MID || + rule == WEIGHT_FOR_DIR); + + if (smartlist_len(sl) == 0) { + log_info(LD_CIRC, + "Empty routerlist passed in to consensus weight node " + "selection for rule %s", + bandwidth_weight_rule_to_string(rule)); + return NULL; + } + + weight_scale = circuit_build_times_get_bw_scale(NULL); + + if (rule == WEIGHT_FOR_GUARD) { + Wg = networkstatus_get_bw_weight(NULL, "Wgg", -1); + Wm = networkstatus_get_bw_weight(NULL, "Wgm", -1); /* Bridges */ + We = 0; + Wd = networkstatus_get_bw_weight(NULL, "Wgd", -1); + + Wgb = networkstatus_get_bw_weight(NULL, "Wgb", -1); + Wmb = networkstatus_get_bw_weight(NULL, "Wmb", -1); + Web = networkstatus_get_bw_weight(NULL, "Web", -1); + Wdb = networkstatus_get_bw_weight(NULL, "Wdb", -1); + } else if (rule == WEIGHT_FOR_MID) { + Wg = networkstatus_get_bw_weight(NULL, "Wmg", -1); + Wm = networkstatus_get_bw_weight(NULL, "Wmm", -1); + We = networkstatus_get_bw_weight(NULL, "Wme", -1); + Wd = networkstatus_get_bw_weight(NULL, "Wmd", -1); + + Wgb = networkstatus_get_bw_weight(NULL, "Wgb", -1); + Wmb = networkstatus_get_bw_weight(NULL, "Wmb", -1); + Web = networkstatus_get_bw_weight(NULL, "Web", -1); + Wdb = networkstatus_get_bw_weight(NULL, "Wdb", -1); + } else if (rule == WEIGHT_FOR_EXIT) { + // Guards CAN be exits if they have weird exit policies + // They are d then I guess... + We = networkstatus_get_bw_weight(NULL, "Wee", -1); + Wm = networkstatus_get_bw_weight(NULL, "Wem", -1); /* Odd exit policies */ + Wd = networkstatus_get_bw_weight(NULL, "Wed", -1); + Wg = networkstatus_get_bw_weight(NULL, "Weg", -1); /* Odd exit policies */ + + Wgb = networkstatus_get_bw_weight(NULL, "Wgb", -1); + Wmb = networkstatus_get_bw_weight(NULL, "Wmb", -1); + Web = networkstatus_get_bw_weight(NULL, "Web", -1); + Wdb = networkstatus_get_bw_weight(NULL, "Wdb", -1); + } else if (rule == WEIGHT_FOR_DIR) { + We = networkstatus_get_bw_weight(NULL, "Wbe", -1); + Wm = networkstatus_get_bw_weight(NULL, "Wbm", -1); + Wd = networkstatus_get_bw_weight(NULL, "Wbd", -1); + Wg = networkstatus_get_bw_weight(NULL, "Wbg", -1); + + Wgb = Wmb = Web = Wdb = weight_scale; + } else if (rule == NO_WEIGHTING) { + Wg = Wm = We = Wd = weight_scale; + Wgb = Wmb = Web = Wdb = weight_scale; + } + + if (Wg < 0 || Wm < 0 || We < 0 || Wd < 0 || Wgb < 0 || Wmb < 0 || Wdb < 0 + || Web < 0) { + log_debug(LD_CIRC, + "Got negative bandwidth weights. Defaulting to old selection" + " algorithm."); + return NULL; // Use old algorithm. + } + + Wg /= weight_scale; + Wm /= weight_scale; + We /= weight_scale; + Wd /= weight_scale; + + Wgb /= weight_scale; + Wmb /= weight_scale; + Web /= weight_scale; + Wdb /= weight_scale; + + bandwidths = tor_malloc_zero(sizeof(double)*smartlist_len(sl)); + + // Cycle through smartlist and total the bandwidth. + for (i = 0; i < (unsigned)smartlist_len(sl); ++i) { + int is_exit = 0, is_guard = 0, is_dir = 0, this_bw = 0, is_me = 0; + double weight = 1; + if (statuses) { + routerstatus_t *status = smartlist_get(sl, i); + is_exit = status->is_exit && !status->is_bad_exit; + is_guard = status->is_possible_guard; + is_dir = (status->dir_port != 0); + if (!status->has_bandwidth) { + tor_free(bandwidths); + /* This should never happen, unless all the authorites downgrade + * to 0.2.0 or rogue routerstatuses get inserted into our consensus. */ + log_warn(LD_BUG, + "Consensus is not listing bandwidths. Defaulting back to " + "old router selection algorithm."); + return NULL; + } + this_bw = kb_to_bytes(status->bandwidth); + if (router_digest_is_me(status->identity_digest)) + is_me = 1; + } else { + routerstatus_t *rs; + routerinfo_t *router = smartlist_get(sl, i); + rs = router_get_consensus_status_by_id( + router->cache_info.identity_digest); + is_exit = router->is_exit && !router->is_bad_exit; + is_guard = router->is_possible_guard; + is_dir = (router->dir_port != 0); + if (rs && rs->has_bandwidth) { + this_bw = kb_to_bytes(rs->bandwidth); + } else { /* bridge or other descriptor not in our consensus */ + this_bw = bridge_get_advertised_bandwidth_bounded(router); + have_unknown = 1; + } + if (router_digest_is_me(router->cache_info.identity_digest)) + is_me = 1; + } + if (is_guard && is_exit) { + weight = (is_dir ? Wdb*Wd : Wd); + } else if (is_guard) { + weight = (is_dir ? Wgb*Wg : Wg); + } else if (is_exit) { + weight = (is_dir ? Web*We : We); + } else { // middle + weight = (is_dir ? Wmb*Wm : Wm); + } + + bandwidths[i] = weight*this_bw; + weighted_bw += weight*this_bw; + if (is_me) + sl_last_weighted_bw_of_me = weight*this_bw; + } + + /* XXXX023 this is a kludge to expose these values. */ + sl_last_total_weighted_bw = weighted_bw; + + log_debug(LD_CIRC, "Choosing node for rule %s based on weights " + "Wg=%lf Wm=%lf We=%lf Wd=%lf with total bw %lf", + bandwidth_weight_rule_to_string(rule), + Wg, Wm, We, Wd, weighted_bw); + + /* If there is no bandwidth, choose at random */ + if (DBL_TO_U64(weighted_bw) == 0) { + /* Don't warn when using bridges/relays not in the consensus */ + if (!have_unknown) + log_warn(LD_CIRC, + "Weighted bandwidth is %lf in node selection for rule %s", + weighted_bw, bandwidth_weight_rule_to_string(rule)); + tor_free(bandwidths); + return smartlist_choose(sl); + } + + rand_bw = crypto_rand_uint64(DBL_TO_U64(weighted_bw)); + rand_bw++; /* crypto_rand_uint64() counts from 0, and we need to count + * from 1 below. See bug 1203 for details. */ + + /* Last, count through sl until we get to the element we picked */ + tmp = 0.0; + for (i=0; i < (unsigned)smartlist_len(sl); i++) { + tmp += bandwidths[i]; + if (tmp >= rand_bw) + break; + } + + if (i == (unsigned)smartlist_len(sl)) { + /* This was once possible due to round-off error, but shouldn't be able + * to occur any longer. */ + tor_fragile_assert(); + --i; + log_warn(LD_BUG, "Round-off error in computing bandwidth had an effect on " + " which router we chose. Please tell the developers. " + "%lf " U64_FORMAT " %lf", tmp, U64_PRINTF_ARG(rand_bw), + weighted_bw); + } + tor_free(bandwidths); + return smartlist_get(sl, i); +} + +/** Helper function: + * choose a random element of smartlist <b>sl</b>, weighted by * the advertised bandwidth of each element. * * If <b>statuses</b> is zero, then <b>sl</b> is a list of @@ -1565,11 +1892,24 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, bitarray_t *guard_bits; int me_idx = -1; + // This function does not support WEIGHT_FOR_DIR + // or WEIGHT_FOR_MID + if (rule == WEIGHT_FOR_DIR || rule == WEIGHT_FOR_MID) { + rule = NO_WEIGHTING; + } + /* Can't choose exit and guard at same time */ tor_assert(rule == NO_WEIGHTING || rule == WEIGHT_FOR_EXIT || rule == WEIGHT_FOR_GUARD); + if (smartlist_len(sl) == 0) { + log_info(LD_CIRC, + "Empty routerlist passed in to old node selection for rule %s", + bandwidth_weight_rule_to_string(rule)); + return NULL; + } + /* First count the total bandwidth weight, and make a list * of each value. <0 means "unknown; no routerinfo." We use the * bits of negative values to remember whether the router was fast (-x)&1 @@ -1594,7 +1934,7 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, if (status->has_bandwidth) { this_bw = kb_to_bytes(status->bandwidth); } else { /* guess */ - /* XXX022 once consensuses always list bandwidths, we can take + /* XXX023 once consensuses always list bandwidths, we can take * this guessing business out. -RD */ is_known = 0; flags = status->is_fast ? 1 : 0; @@ -1613,14 +1953,14 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, if (rs && rs->has_bandwidth) { this_bw = kb_to_bytes(rs->bandwidth); } else if (rs) { /* guess; don't trust the descriptor */ - /* XXX022 once consensuses always list bandwidths, we can take + /* XXX023 once consensuses always list bandwidths, we can take * this guessing business out. -RD */ is_known = 0; flags = router->is_fast ? 1 : 0; flags |= is_exit ? 2 : 0; flags |= is_guard ? 4 : 0; } else /* bridge or other descriptor not in our consensus */ - this_bw = router_get_advertised_bandwidth_capped(router); + this_bw = bridge_get_advertised_bandwidth_bounded(router); } if (is_exit) bitarray_set(exit_bits, i); @@ -1628,6 +1968,8 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, bitarray_set(guard_bits, i); if (is_known) { bandwidths[i] = (int32_t) this_bw; // safe since MAX_BELIEVABLE<INT32_MAX + // XXX this is no longer true! We don't always cap the bw anymore. Can + // a consensus make us overflow?-sh tor_assert(bandwidths[i] >= 0); if (is_guard) total_guard_bw += this_bw; @@ -1691,12 +2033,12 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, * For detailed derivation of this formula, see * http://archives.seul.org/or/dev/Jul-2007/msg00056.html */ - if (rule == WEIGHT_FOR_EXIT) + if (rule == WEIGHT_FOR_EXIT || !total_exit_bw) exit_weight = 1.0; else exit_weight = 1.0 - all_bw/(3.0*exit_bw); - if (rule == WEIGHT_FOR_GUARD) + if (rule == WEIGHT_FOR_GUARD || !total_guard_bw) guard_weight = 1.0; else guard_weight = 1.0 - all_bw/(3.0*guard_bw); @@ -1727,7 +2069,7 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, } } - /* XXXX022 this is a kludge to expose these values. */ + /* XXXX023 this is a kludge to expose these values. */ sl_last_total_weighted_bw = total_bw; log_debug(LD_CIRC, "Total weighted bw = "U64_FORMAT @@ -1745,6 +2087,8 @@ smartlist_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule, /* Almost done: choose a random value from the bandwidth weights. */ rand_bw = crypto_rand_uint64(total_bw); + rand_bw++; /* crypto_rand_uint64() counts from 0, and we need to count + * from 1 below. See bug 1203 for details. */ /* Last, count through sl until we get to the element we picked */ tmp = 0; @@ -1788,26 +2132,34 @@ routerinfo_t * routerlist_sl_choose_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule) { - return smartlist_choose_by_bandwidth(sl, rule, 0); + routerinfo_t *ret; + if ((ret = smartlist_choose_by_bandwidth_weights(sl, rule, 0))) { + return ret; + } else { + return smartlist_choose_by_bandwidth(sl, rule, 0); + } } /** Choose a random element of status list <b>sl</b>, weighted by * the advertised bandwidth of each status. */ routerstatus_t * -routerstatus_sl_choose_by_bandwidth(smartlist_t *sl) +routerstatus_sl_choose_by_bandwidth(smartlist_t *sl, + bandwidth_weight_rule_t rule) { /* We are choosing neither exit nor guard here. Weight accordingly. */ - return smartlist_choose_by_bandwidth(sl, NO_WEIGHTING, 1); + routerstatus_t *ret; + if ((ret = smartlist_choose_by_bandwidth_weights(sl, rule, 1))) { + return ret; + } else { + return smartlist_choose_by_bandwidth(sl, rule, 1); + } } -/** Return a random running router from the routerlist. If any node - * named in <b>preferred</b> is available, pick one of those. Never +/** Return a random running router from the routerlist. Never * pick a node whose routerinfo is in * <b>excludedsmartlist</b>, or whose routerinfo matches <b>excludedset</b>, - * even if they are the only nodes - * available. If <b>CRN_STRICT_PREFERRED</b> is set in flags, never pick - * any node besides those in <b>preferred</b>. + * even if they are the only nodes available. * If <b>CRN_NEED_UPTIME</b> is set in flags and any router has more than * a minimum uptime, return one of those. * If <b>CRN_NEED_CAPACITY</b> is set in flags, weight your choice by the @@ -1820,8 +2172,7 @@ routerstatus_sl_choose_by_bandwidth(smartlist_t *sl) * node (that is, possibly discounting exit nodes). */ routerinfo_t * -router_choose_random_node(const char *preferred, - smartlist_t *excludedsmartlist, +router_choose_random_node(smartlist_t *excludedsmartlist, routerset_t *excludedset, router_crn_flags_t flags) { @@ -1829,18 +2180,16 @@ router_choose_random_node(const char *preferred, const int need_capacity = (flags & CRN_NEED_CAPACITY) != 0; const int need_guard = (flags & CRN_NEED_GUARD) != 0; const int allow_invalid = (flags & CRN_ALLOW_INVALID) != 0; - const int strict = (flags & CRN_STRICT_PREFERRED) != 0; const int weight_for_exit = (flags & CRN_WEIGHT_AS_EXIT) != 0; - smartlist_t *sl, *excludednodes; + smartlist_t *sl=smartlist_create(), + *excludednodes=smartlist_create(); routerinfo_t *choice = NULL, *r; bandwidth_weight_rule_t rule; tor_assert(!(weight_for_exit && need_guard)); rule = weight_for_exit ? WEIGHT_FOR_EXIT : - (need_guard ? WEIGHT_FOR_GUARD : NO_WEIGHTING); - - excludednodes = smartlist_create(); + (need_guard ? WEIGHT_FOR_GUARD : WEIGHT_FOR_MID); /* Exclude relays that allow single hop exit circuits, if the user * wants to (such relays might be risky) */ @@ -1857,60 +2206,35 @@ router_choose_random_node(const char *preferred, routerlist_add_family(excludednodes, r); } - /* Try the preferred nodes first. Ignore need_uptime and need_capacity - * and need_guard, since the user explicitly asked for these nodes. */ - if (preferred) { - sl = smartlist_create(); - add_nickname_list_to_smartlist(sl,preferred,1); - smartlist_subtract(sl,excludednodes); - if (excludedsmartlist) - smartlist_subtract(sl,excludedsmartlist); - if (excludedset) - routerset_subtract_routers(sl,excludedset); - choice = smartlist_choose(sl); - smartlist_free(sl); - } - if (!choice && !strict) { - /* Then give up on our preferred choices: any node - * will do that has the required attributes. */ - sl = smartlist_create(); - router_add_running_routers_to_smartlist(sl, allow_invalid, - need_uptime, need_capacity, - need_guard); - smartlist_subtract(sl,excludednodes); - if (excludedsmartlist) - smartlist_subtract(sl,excludedsmartlist); - if (excludedset) - routerset_subtract_routers(sl,excludedset); - - if (need_capacity || need_guard) - choice = routerlist_sl_choose_by_bandwidth(sl, rule); - else - choice = smartlist_choose(sl); - - smartlist_free(sl); - if (!choice && (need_uptime || need_capacity || need_guard)) { - /* try once more -- recurse but with fewer restrictions. */ - log_info(LD_CIRC, - "We couldn't find any live%s%s%s routers; falling back " - "to list of all routers.", - need_capacity?", fast":"", - need_uptime?", stable":"", - need_guard?", guard":""); - flags &= ~ (CRN_NEED_UPTIME|CRN_NEED_CAPACITY|CRN_NEED_GUARD); - choice = router_choose_random_node( - NULL, excludedsmartlist, excludedset, flags); - } + router_add_running_routers_to_smartlist(sl, allow_invalid, + need_uptime, need_capacity, + need_guard); + smartlist_subtract(sl,excludednodes); + if (excludedsmartlist) + smartlist_subtract(sl,excludedsmartlist); + if (excludedset) + routerset_subtract_routers(sl,excludedset); + + // Always weight by bandwidth + choice = routerlist_sl_choose_by_bandwidth(sl, rule); + + smartlist_free(sl); + if (!choice && (need_uptime || need_capacity || need_guard)) { + /* try once more -- recurse but with fewer restrictions. */ + log_info(LD_CIRC, + "We couldn't find any live%s%s%s routers; falling back " + "to list of all routers.", + need_capacity?", fast":"", + need_uptime?", stable":"", + need_guard?", guard":""); + flags &= ~ (CRN_NEED_UPTIME|CRN_NEED_CAPACITY|CRN_NEED_GUARD); + choice = router_choose_random_node( + excludedsmartlist, excludedset, flags); } smartlist_free(excludednodes); if (!choice) { - if (strict) { - log_warn(LD_CIRC, "All preferred nodes were down when trying to choose " - "node, and the Strict[...]Nodes option is set. Failing."); - } else { - log_warn(LD_CIRC, - "No available nodes when trying to choose node. Failing."); - } + log_warn(LD_CIRC, + "No available nodes when trying to choose node. Failing."); } return choice; } @@ -2367,6 +2691,9 @@ extrainfo_free(extrainfo_t *extrainfo) static void signed_descriptor_free(signed_descriptor_t *sd) { + if (!sd) + return; + tor_free(sd->signed_descriptor_body); /* XXXX remove this once more bugs go away. */ @@ -2374,12 +2701,15 @@ signed_descriptor_free(signed_descriptor_t *sd) tor_free(sd); } -/** Extract a signed_descriptor_t from a routerinfo, and free the routerinfo. +/** Extract a signed_descriptor_t from a general routerinfo, and free the + * routerinfo. */ static signed_descriptor_t * signed_descriptor_from_routerinfo(routerinfo_t *ri) { - signed_descriptor_t *sd = tor_malloc_zero(sizeof(signed_descriptor_t)); + signed_descriptor_t *sd; + tor_assert(ri->purpose == ROUTER_PURPOSE_GENERAL); + sd = tor_malloc_zero(sizeof(signed_descriptor_t)); memcpy(sd, &(ri->cache_info), sizeof(signed_descriptor_t)); sd->routerlist_index = -1; ri->cache_info.signed_descriptor_body = NULL; @@ -2398,7 +2728,8 @@ _extrainfo_free(void *e) void routerlist_free(routerlist_t *rl) { - tor_assert(rl); + if (!rl) + return; rimap_free(rl->identity_map, NULL); sdmap_free(rl->desc_digest_map, NULL); sdmap_free(rl->desc_by_eid_map, NULL); @@ -2437,46 +2768,6 @@ dump_routerlist_mem_usage(int severity) "In %d old descriptors: "U64_FORMAT" bytes.", smartlist_len(routerlist->routers), U64_PRINTF_ARG(livedescs), smartlist_len(routerlist->old_routers), U64_PRINTF_ARG(olddescs)); - -#if 0 - { - const smartlist_t *networkstatus_v2_list = networkstatus_get_v2_list(); - networkstatus_t *consensus = networkstatus_get_latest_consensus(); - log(severity, LD_DIR, "Now let's look through old_descriptors!"); - SMARTLIST_FOREACH(routerlist->old_routers, signed_descriptor_t *, sd, { - int in_v2 = 0; - int in_v3 = 0; - char published[ISO_TIME_LEN+1]; - char last_valid_until[ISO_TIME_LEN+1]; - char last_served_at[ISO_TIME_LEN+1]; - char id[HEX_DIGEST_LEN+1]; - routerstatus_t *rs; - format_iso_time(published, sd->published_on); - format_iso_time(last_valid_until, sd->last_listed_as_valid_until); - format_iso_time(last_served_at, sd->last_served_at); - base16_encode(id, sizeof(id), sd->identity_digest, DIGEST_LEN); - SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, - { - rs = networkstatus_v2_find_entry(ns, sd->identity_digest); - if (rs && !memcmp(rs->descriptor_digest, - sd->signed_descriptor_digest, DIGEST_LEN)) { - in_v2 = 1; break; - } - }); - if (consensus) { - rs = networkstatus_vote_find_entry(consensus, sd->identity_digest); - if (rs && !memcmp(rs->descriptor_digest, - sd->signed_descriptor_digest, DIGEST_LEN)) - in_v3 = 1; - } - log(severity, LD_DIR, - "Old descriptor for %s (published %s) %sin v2 ns, %sin v3 " - "consensus. Last valid until %s; last served at %s.", - id, published, in_v2 ? "" : "not ", in_v3 ? "" : "not ", - last_valid_until, last_served_at); - }); - } -#endif } /** Debugging helper: If <b>idx</b> is nonnegative, assert that <b>ri</b> is @@ -2509,6 +2800,7 @@ static void routerlist_insert(routerlist_t *rl, routerinfo_t *ri) { routerinfo_t *ri_old; + signed_descriptor_t *sd_old; { /* XXXX Remove if this slows us down. */ routerinfo_t *ri_generated = router_get_my_routerinfo(); @@ -2518,8 +2810,16 @@ routerlist_insert(routerlist_t *rl, routerinfo_t *ri) ri_old = rimap_set(rl->identity_map, ri->cache_info.identity_digest, ri); tor_assert(!ri_old); - sdmap_set(rl->desc_digest_map, ri->cache_info.signed_descriptor_digest, - &(ri->cache_info)); + + sd_old = sdmap_set(rl->desc_digest_map, + ri->cache_info.signed_descriptor_digest, + &(ri->cache_info)); + if (sd_old) { + rl->desc_store.bytes_dropped += sd_old->signed_descriptor_len; + sdmap_remove(rl->desc_by_eid_map, sd_old->extra_info_digest); + signed_descriptor_free(sd_old); + } + if (!tor_digest_is_zero(ri->cache_info.extra_info_digest)) sdmap_set(rl->desc_by_eid_map, ri->cache_info.extra_info_digest, &ri->cache_info); @@ -2738,6 +3038,7 @@ routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old, routerinfo_t *ri_new) { int idx; + int same_descriptors; routerinfo_t *ri_tmp; extrainfo_t *ei_tmp; @@ -2782,8 +3083,15 @@ routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old, &ri_new->cache_info); } + same_descriptors = ! memcmp(ri_old->cache_info.signed_descriptor_digest, + ri_new->cache_info.signed_descriptor_digest, + DIGEST_LEN); + if (should_cache_old_descriptors() && - ri_old->purpose == ROUTER_PURPOSE_GENERAL) { + ri_old->purpose == ROUTER_PURPOSE_GENERAL && + !same_descriptors) { + /* ri_old is going to become a signed_descriptor_t and go into + * old_routers */ signed_descriptor_t *sd = signed_descriptor_from_routerinfo(ri_old); smartlist_add(rl->old_routers, sd); sd->routerlist_index = smartlist_len(rl->old_routers)-1; @@ -2791,24 +3099,27 @@ routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old, if (!tor_digest_is_zero(sd->extra_info_digest)) sdmap_set(rl->desc_by_eid_map, sd->extra_info_digest, sd); } else { - if (memcmp(ri_old->cache_info.signed_descriptor_digest, - ri_new->cache_info.signed_descriptor_digest, - DIGEST_LEN)) { - /* digests don't match; digestmap_set didn't replace */ + /* We're dropping ri_old. */ + if (!same_descriptors) { + /* digests don't match; The sdmap_set above didn't replace */ sdmap_remove(rl->desc_digest_map, ri_old->cache_info.signed_descriptor_digest); - } - ei_tmp = eimap_remove(rl->extra_info_map, - ri_old->cache_info.extra_info_digest); - if (ei_tmp) { - rl->extrainfo_store.bytes_dropped += - ei_tmp->cache_info.signed_descriptor_len; - extrainfo_free(ei_tmp); - } - if (!tor_digest_is_zero(ri_old->cache_info.extra_info_digest)) { - sdmap_remove(rl->desc_by_eid_map, - ri_old->cache_info.extra_info_digest); + if (memcmp(ri_old->cache_info.extra_info_digest, + ri_new->cache_info.extra_info_digest, DIGEST_LEN)) { + ei_tmp = eimap_remove(rl->extra_info_map, + ri_old->cache_info.extra_info_digest); + if (ei_tmp) { + rl->extrainfo_store.bytes_dropped += + ei_tmp->cache_info.signed_descriptor_len; + extrainfo_free(ei_tmp); + } + } + + if (!tor_digest_is_zero(ri_old->cache_info.extra_info_digest)) { + sdmap_remove(rl->desc_by_eid_map, + ri_old->cache_info.extra_info_digest); + } } rl->desc_store.bytes_dropped += ri_old->cache_info.signed_descriptor_len; routerinfo_free(ri_old); @@ -2846,8 +3157,7 @@ routerlist_reparse_old(routerlist_t *rl, signed_descriptor_t *sd) void routerlist_free_all(void) { - if (routerlist) - routerlist_free(routerlist); + routerlist_free(routerlist); routerlist = NULL; if (warned_nicknames) { SMARTLIST_FOREACH(warned_nicknames, char *, cp, tor_free(cp)); @@ -2940,7 +3250,8 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, int from_cache, int from_fetch) { const char *id_digest; - int authdir = authdir_mode_handles_descs(get_options(), router->purpose); + or_options_t *options = get_options(); + int authdir = authdir_mode_handles_descs(options, router->purpose); int authdir_believes_valid = 0; routerinfo_t *old_router; networkstatus_t *consensus = networkstatus_get_latest_consensus(); @@ -2954,15 +3265,33 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, id_digest = router->cache_info.identity_digest; + old_router = router_get_by_digest(id_digest); + /* Make sure that we haven't already got this exact descriptor. */ if (sdmap_get(routerlist->desc_digest_map, router->cache_info.signed_descriptor_digest)) { - log_info(LD_DIR, - "Dropping descriptor that we already have for router '%s'", - router->nickname); - *msg = "Router descriptor was not new."; - routerinfo_free(router); - return ROUTER_WAS_NOT_NEW; + /* If we have this descriptor already and the new descriptor is a bridge + * descriptor, replace it. If we had a bridge descriptor before and the + * new one is not a bridge descriptor, don't replace it. */ + + /* Only members of routerlist->identity_map can be bridges; we don't + * put bridges in old_routers. */ + const int was_bridge = old_router && + old_router->purpose == ROUTER_PURPOSE_BRIDGE; + + if (routerinfo_is_a_configured_bridge(router) && + router->purpose == ROUTER_PURPOSE_BRIDGE && + !was_bridge) { + log_info(LD_DIR, "Replacing non-bridge descriptor with bridge " + "descriptor for router '%s'", router->nickname); + } else { + log_info(LD_DIR, + "Dropping descriptor that we already have for router '%s'", + router->nickname); + *msg = "Router descriptor was not new."; + routerinfo_free(router); + return ROUTER_WAS_NOT_NEW; + } } if (authdir) { @@ -2999,15 +3328,14 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, SMARTLIST_FOREACH(networkstatus_v2_list, networkstatus_v2_t *, ns, { routerstatus_t *rs = - networkstatus_v2_find_entry(ns, router->cache_info.identity_digest); + networkstatus_v2_find_entry(ns, id_digest); if (rs && !memcmp(rs->descriptor_digest, router->cache_info.signed_descriptor_digest, DIGEST_LEN)) rs->need_to_mirror = 0; }); if (consensus) { - routerstatus_t *rs = networkstatus_vote_find_entry(consensus, - router->cache_info.identity_digest); + routerstatus_t *rs = networkstatus_vote_find_entry(consensus, id_digest); if (rs && !memcmp(rs->descriptor_digest, router->cache_info.signed_descriptor_digest, DIGEST_LEN)) { @@ -3028,14 +3356,26 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, return ROUTER_NOT_IN_CONSENSUS; } + /* If we're reading a bridge descriptor from our cache, and we don't + * recognize it as one of our currently configured bridges, drop the + * descriptor. Otherwise we could end up using it as one of our entry + * guards even if it isn't in our Bridge config lines. */ + if (router->purpose == ROUTER_PURPOSE_BRIDGE && from_cache && + !authdir_mode_bridge(options) && + !routerinfo_is_a_configured_bridge(router)) { + log_info(LD_DIR, "Dropping bridge descriptor for '%s' because we have " + "no bridge configured at that address.", router->nickname); + *msg = "Router descriptor was not a configured bridge."; + routerinfo_free(router); + return ROUTER_WAS_NOT_WANTED; + } + /* If we have a router with the same identity key, choose the newer one. */ - old_router = rimap_get(routerlist->identity_map, - router->cache_info.identity_digest); if (old_router) { if (!in_consensus && (router->cache_info.published_on <= old_router->cache_info.published_on)) { /* Same key, but old. This one is not listed in the consensus. */ - log_debug(LD_DIR, "Skipping not-new descriptor for router '%s'", + log_debug(LD_DIR, "Not-new descriptor for router '%s'", router->nickname); /* Only journal this desc if we'll be serving it. */ if (!from_cache && should_cache_old_descriptors()) @@ -3049,8 +3389,7 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, log_debug(LD_DIR, "Replacing entry for router '%s/%s' [%s]", router->nickname, old_router->nickname, hex_str(id_digest,DIGEST_LEN)); - if (router->addr == old_router->addr && - router->or_port == old_router->or_port) { + if (routers_have_same_or_addr(router, old_router)) { /* these carry over when the address and orport are unchanged. */ router->last_reachable = old_router->last_reachable; router->testing_since = old_router->testing_since; @@ -3078,9 +3417,10 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, /* We haven't seen a router with this identity before. Add it to the end of * the list. */ routerlist_insert(routerlist, router); - if (!from_cache) + if (!from_cache) { signed_desc_append_to_journal(&router->cache_info, &routerlist->desc_store); + } directory_set_dirty(); return ROUTER_ADDED_SUCCESSFULLY; } @@ -3096,7 +3436,7 @@ router_add_extrainfo_to_routerlist(extrainfo_t *ei, const char **msg, int inserted; (void)from_fetch; if (msg) *msg = NULL; - /*XXXX022 Do something with msg */ + /*XXXX023 Do something with msg */ inserted = extrainfo_insert(router_get_routerlist(), ei); @@ -3397,15 +3737,19 @@ routerlist_remove_old_routers(void) /** We just added a new set of descriptors. Take whatever extra steps * we need. */ -static void +void routerlist_descriptors_added(smartlist_t *sl, int from_cache) { tor_assert(sl); control_event_descriptors_changed(sl); - SMARTLIST_FOREACH(sl, routerinfo_t *, ri, + SMARTLIST_FOREACH_BEGIN(sl, routerinfo_t *, ri) { if (ri->purpose == ROUTER_PURPOSE_BRIDGE) learned_bridge_descriptor(ri, from_cache); - ); + if (ri->needs_retest_if_added) { + ri->needs_retest_if_added = 0; + dirserv_single_reachability_test(approx_time(), ri); + } + } SMARTLIST_FOREACH_END(ri); } /** @@ -3736,12 +4080,8 @@ add_trusted_dir_server(const char *nickname, const char *address, if (ent->or_port) ent->fake_status.version_supports_begindir = 1; -/* XX021 - wait until authorities are upgraded */ -#if 0 + ent->fake_status.version_supports_conditional_consensus = 1; -#else - ent->fake_status.version_supports_conditional_consensus = 0; -#endif smartlist_add(trusted_dir_servers, ent); router_dir_info_changed(); @@ -3756,10 +4096,8 @@ authority_cert_free(authority_cert_t *cert) return; tor_free(cert->cache_info.signed_descriptor_body); - if (cert->signing_key) - crypto_free_pk_env(cert->signing_key); - if (cert->identity_key) - crypto_free_pk_env(cert->identity_key); + crypto_free_pk_env(cert->signing_key); + crypto_free_pk_env(cert->identity_key); tor_free(cert); } @@ -3768,6 +4106,9 @@ authority_cert_free(authority_cert_t *cert) static void trusted_dir_server_free(trusted_dir_server_t *ds) { + if (!ds) + return; + tor_free(ds->nickname); tor_free(ds->description); tor_free(ds->address); @@ -3821,7 +4162,7 @@ list_pending_downloads(digestmap_t *result, const char *resource = TO_DIR_CONN(conn)->requested_resource; if (!strcmpstart(resource, prefix)) dir_split_resource_into_fingerprints(resource + p_len, - tmp, NULL, 1, 0); + tmp, NULL, DSR_HEX); } }); SMARTLIST_FOREACH(tmp, char *, d, @@ -3923,7 +4264,7 @@ client_would_use_router(routerstatus_t *rs, time_t now, or_options_t *options) * this number per server. */ #define MIN_DL_PER_REQUEST 4 /** To prevent a single screwy cache from confusing us by selective reply, - * try to split our requests into at least this this many requests. */ + * try to split our requests into at least this many requests. */ #define MIN_REQUESTS 3 /** If we want fewer than this many descriptors, wait until we * want more, or until MAX_CLIENT_INTERVAL_WITHOUT_REQUEST has @@ -3937,7 +4278,8 @@ client_would_use_router(routerstatus_t *rs, time_t now, or_options_t *options) * whether to delay fetching until we have more. If we don't want to delay, * launch one or more requests to the appropriate directory authorities. */ static void -launch_router_descriptor_downloads(smartlist_t *downloadable, time_t now) +launch_router_descriptor_downloads(smartlist_t *downloadable, + routerstatus_t *source, time_t now) { int should_delay = 0, n_downloadable; or_options_t *options = get_options(); @@ -3990,7 +4332,7 @@ launch_router_descriptor_downloads(smartlist_t *downloadable, time_t now) pds_flags |= PDS_NO_EXISTING_SERVERDESC_FETCH; } - n_per_request = (n_downloadable+MIN_REQUESTS-1) / MIN_REQUESTS; + n_per_request = CEIL_DIV(n_downloadable, MIN_REQUESTS); if (n_per_request > MAX_DL_PER_REQUEST) n_per_request = MAX_DL_PER_REQUEST; if (n_per_request < MIN_DL_PER_REQUEST) @@ -4003,11 +4345,11 @@ launch_router_descriptor_downloads(smartlist_t *downloadable, time_t now) log_info(LD_DIR, "Launching %d request%s for %d router%s, %d at a time", - (n_downloadable+n_per_request-1)/n_per_request, + CEIL_DIV(n_downloadable, n_per_request), req_plural, n_downloadable, rtr_plural, n_per_request); smartlist_sort_digests(downloadable); for (i=0; i < n_downloadable; i += n_per_request) { - initiate_descriptor_downloads(NULL, DIR_PURPOSE_FETCH_SERVERDESC, + initiate_descriptor_downloads(source, DIR_PURPOSE_FETCH_SERVERDESC, downloadable, i, i+n_per_request, pds_flags); } @@ -4163,18 +4505,18 @@ update_router_descriptor_cache_downloads_v2(time_t now) digestmap_free(map,NULL); } -/** For any descriptor that we want that's currently listed in the live - * consensus, download it as appropriate. */ -static void -update_consensus_router_descriptor_downloads(time_t now) +/** For any descriptor that we want that's currently listed in + * <b>consensus</b>, download it as appropriate. */ +void +update_consensus_router_descriptor_downloads(time_t now, int is_vote, + networkstatus_t *consensus) { or_options_t *options = get_options(); digestmap_t *map = NULL; smartlist_t *no_longer_old = smartlist_create(); smartlist_t *downloadable = smartlist_create(); + routerstatus_t *source = NULL; int authdir = authdir_mode(options); - networkstatus_t *consensus = - networkstatus_get_reasonably_live_consensus(now); int n_delayed=0, n_have=0, n_would_reject=0, n_wouldnt_use=0, n_inprogress=0, n_in_oldrouters=0; @@ -4183,10 +4525,24 @@ update_consensus_router_descriptor_downloads(time_t now) if (!consensus) goto done; + if (is_vote) { + /* where's it from, so we know whom to ask for descriptors */ + trusted_dir_server_t *ds; + networkstatus_voter_info_t *voter = smartlist_get(consensus->voters, 0); + tor_assert(voter); + ds = trusteddirserver_get_by_v3_auth_digest(voter->identity_digest); + if (ds) + source = &(ds->fake_status); + else + log_warn(LD_DIR, "couldn't lookup source from vote?"); + } + map = digestmap_new(); list_pending_descriptor_downloads(map, 0); - SMARTLIST_FOREACH(consensus->routerstatus_list, routerstatus_t *, rs, + SMARTLIST_FOREACH(consensus->routerstatus_list, void *, rsp, { + routerstatus_t *rs = + is_vote ? &(((vote_routerstatus_t *)rsp)->status) : rsp; signed_descriptor_t *sd; if ((sd = router_get_by_descriptor_digest(rs->descriptor_digest))) { routerinfo_t *ri; @@ -4221,6 +4577,18 @@ update_consensus_router_descriptor_downloads(time_t now) ++n_wouldnt_use; continue; /* We would never use it ourself. */ } + if (is_vote && source) { + char time_bufnew[ISO_TIME_LEN+1]; + char time_bufold[ISO_TIME_LEN+1]; + routerinfo_t *oldrouter = router_get_by_digest(rs->identity_digest); + format_iso_time(time_bufnew, rs->published_on); + if (oldrouter) + format_iso_time(time_bufold, oldrouter->cache_info.published_on); + log_info(LD_DIR, "Learned about %s (%s vs %s) from %s's vote (%s)", + rs->nickname, time_bufnew, + oldrouter ? time_bufold : "none", + source->nickname, oldrouter ? "known" : "unknown"); + } smartlist_add(downloadable, rs->descriptor_digest); }); @@ -4254,7 +4622,7 @@ update_consensus_router_descriptor_downloads(time_t now) smartlist_len(downloadable), n_delayed, n_have, n_in_oldrouters, n_would_reject, n_wouldnt_use, n_inprogress); - launch_router_descriptor_downloads(downloadable, now); + launch_router_descriptor_downloads(downloadable, source, now); digestmap_free(map, NULL); done: @@ -4264,7 +4632,7 @@ update_consensus_router_descriptor_downloads(time_t now) /** How often should we launch a server/authority request to be sure of getting * a guess for our IP? */ -/*XXXX021 this info should come from netinfo cells or something, or we should +/*XXXX023 this info should come from netinfo cells or something, or we should * do this only when we aren't seeing incoming data. see bug 652. */ #define DUMMY_DOWNLOAD_INTERVAL (20*60) @@ -4279,9 +4647,10 @@ update_router_descriptor_downloads(time_t now) if (directory_fetches_dir_info_early(options)) { update_router_descriptor_cache_downloads_v2(now); } - update_consensus_router_descriptor_downloads(now); + update_consensus_router_descriptor_downloads(now, 0, + networkstatus_get_reasonably_live_consensus(now)); - /* XXXX021 we could be smarter here; see notes on bug 652. */ + /* XXXX023 we could be smarter here; see notes on bug 652. */ /* If we're a server that doesn't have a configured address, we rely on * directory fetches to learn when our address changes. So if we haven't * tried to get any routerdescs in a long time, try a dummy fetch now. */ @@ -4416,16 +4785,21 @@ get_dir_info_status_string(void) /** Iterate over the servers listed in <b>consensus</b>, and count how many of * them seem like ones we'd use, and how many of <em>those</em> we have * descriptors for. Store the former in *<b>num_usable</b> and the latter in - * *<b>num_present</b>. */ + * *<b>num_present</b>. If <b>in_set</b> is non-NULL, only consider those + * routers in <b>in_set</b>. + */ static void count_usable_descriptors(int *num_present, int *num_usable, const networkstatus_t *consensus, - or_options_t *options, time_t now) + or_options_t *options, time_t now, + routerset_t *in_set) { *num_present = 0, *num_usable=0; SMARTLIST_FOREACH(consensus->routerstatus_list, routerstatus_t *, rs, { + if (in_set && ! routerset_contains_routerstatus(in_set, rs)) + continue; if (client_would_use_router(rs, now, options)) { ++*num_usable; /* the consensus says we want it. */ if (router_get_by_descriptor_digest(rs->descriptor_digest)) { @@ -4454,7 +4828,7 @@ count_loading_descriptors_progress(void) return 0; /* can't count descriptors if we have no list of them */ count_usable_descriptors(&num_present, &num_usable, - consensus, get_options(), now); + consensus, get_options(), now, NULL); if (num_usable == 0) return 0; /* don't div by 0 */ @@ -4498,22 +4872,39 @@ update_router_have_minimum_dir_info(void) goto done; } - count_usable_descriptors(&num_present, &num_usable, consensus, options, now); + count_usable_descriptors(&num_present, &num_usable, consensus, options, now, + NULL); if (num_present < num_usable/4) { tor_snprintf(dir_info_status, sizeof(dir_info_status), "We have only %d/%d usable descriptors.", num_present, num_usable); res = 0; control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_DESCRIPTORS, 0); + goto done; } else if (num_present < 2) { tor_snprintf(dir_info_status, sizeof(dir_info_status), "Only %d descriptor%s here and believed reachable!", num_present, num_present ? "" : "s"); res = 0; - } else { - res = 1; + goto done; } + /* Check for entry nodes. */ + if (options->EntryNodes) { + count_usable_descriptors(&num_present, &num_usable, consensus, options, + now, options->EntryNodes); + + if (!num_usable || !num_present) { + tor_snprintf(dir_info_status, sizeof(dir_info_status), + "We have only %d/%d usable entry node descriptors.", + num_present, num_usable); + res = 0; + goto done; + } + } + + res = 1; + done: if (res && !have_min_dir_info) { log(LOG_NOTICE, LD_DIR, @@ -4526,6 +4917,13 @@ update_router_have_minimum_dir_info(void) log(quiet ? LOG_INFO : LOG_NOTICE, LD_DIR, "Our directory information is no longer up-to-date " "enough to build circuits: %s", dir_info_status); + + /* a) make us log when we next complete a circuit, so we know when Tor + * is back up and usable, and b) disable some activities that Tor + * should only do while circuits are working, like reachability tests + * and fetching bridge descriptors only over circuits. */ + can_complete_circuit = 0; + control_event_client_status(LOG_NOTICE, "NOT_ENOUGH_DIR_INFO"); } have_min_dir_info = res; @@ -4812,8 +5210,8 @@ esc_router_info(routerinfo_t *router) static char *info=NULL; char *esc_contact, *esc_platform; size_t len; - if (info) - tor_free(info); + tor_free(info); + if (!router) return NULL; /* we're exiting; just free the memory we use */ @@ -4948,9 +5346,8 @@ void routerset_refresh_countries(routerset_t *target) { int cc; - if (target->countries) { - bitarray_free(target->countries); - } + bitarray_free(target->countries); + if (!geoip_is_loaded()) { target->countries = NULL; target->n_countries = 0; @@ -5024,7 +5421,9 @@ routerset_parse(routerset_t *target, const char *s, const char *description) return r; } -/** DOCDOC */ +/** Called when we change a node set, or when we reload the geoip list: + * recompute all country info in all configuration node sets and in the + * routerlist. */ void refresh_all_country_info(void) { @@ -5075,7 +5474,7 @@ routerset_needs_geoip(const routerset_t *set) } /** Return true iff there are no entries in <b>set</b>. */ -static int +int routerset_is_empty(const routerset_t *set) { return !set || smartlist_len(set->list) == 0; @@ -5158,10 +5557,11 @@ routerset_contains_routerstatus(const routerset_t *set, routerstatus_t *rs) } /** Add every known routerinfo_t that is a member of <b>routerset</b> to - * <b>out</b>. If <b>running_only</b>, only add the running ones. */ + * <b>out</b>, but never add any that are part of <b>excludeset</b>. + * If <b>running_only</b>, only add the running ones. */ void routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, - int running_only) + const routerset_t *excludeset, int running_only) { tor_assert(out); if (!routerset || !routerset->list) @@ -5171,12 +5571,13 @@ routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, if (routerset_is_list(routerset)) { /* No routers are specified by type; all are given by name or digest. - * we can do a lookup in O(len(list)). */ + * we can do a lookup in O(len(routerset)). */ SMARTLIST_FOREACH(routerset->list, const char *, name, { routerinfo_t *router = router_get_by_nickname(name, 1); if (router) { if (!running_only || router->is_running) - smartlist_add(out, router); + if (!routerset_contains_router(excludeset, router)) + smartlist_add(out, router); } }); } else { @@ -5186,15 +5587,21 @@ routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, SMARTLIST_FOREACH(rl->routers, routerinfo_t *, router, { if (running_only && !router->is_running) continue; - if (routerset_contains_router(routerset, router)) + if (routerset_contains_router(routerset, router) && + !routerset_contains_router(excludeset, router)) smartlist_add(out, router); }); } } -/** Add to <b>target</b> every routerinfo_t from <b>source</b> that is in - * <b>include</b>, but not excluded in a more specific fashion by - * <b>exclude</b>. If <b>running_only</b>, only include running routers. +#if 0 +/** Add to <b>target</b> every routerinfo_t from <b>source</b> except: + * + * 1) Don't add it if <b>include</b> is non-empty and the relay isn't in + * <b>include</b>; and + * 2) Don't add it if <b>exclude</b> is non-empty and the relay is + * excluded in a more specific fashion by <b>exclude</b>. + * 3) If <b>running_only</b>, don't add non-running routers. */ void routersets_get_disjunction(smartlist_t *target, @@ -5218,6 +5625,7 @@ routersets_get_disjunction(smartlist_t *target, } }); } +#endif /** Remove every routerinfo_t from <b>lst</b> that is in <b>routerset</b>. */ void @@ -5249,10 +5657,15 @@ routerset_to_string(const routerset_t *set) int routerset_equal(const routerset_t *old, const routerset_t *new) { - if (old == NULL && new == NULL) + if (routerset_is_empty(old) && routerset_is_empty(new)) { + /* Two empty sets are equal */ return 1; - else if (old == NULL || new == NULL) + } else if (routerset_is_empty(old) || routerset_is_empty(new)) { + /* An empty set is equal to nothing else. */ return 0; + } + tor_assert(old != NULL); + tor_assert(new != NULL); if (smartlist_len(old->list) != smartlist_len(new->list)) return 0; @@ -5264,35 +5677,15 @@ routerset_equal(const routerset_t *old, const routerset_t *new) }); return 1; - -#if 0 - /* XXXX: This won't work if the names/digests are identical but in a - different order. Checking for exact equality would be heavy going, - is it worth it? -RH*/ - /* This code is totally bogus; sizeof doesn't work even remotely like this - * code seems to think. Let's revert to a string-based comparison for - * now. -NM*/ - if (sizeof(old->names) != sizeof(new->names)) - return 0; - - if (memcmp(old->names,new->names,sizeof(new->names))) - return 0; - if (sizeof(old->digests) != sizeof(new->digests)) - return 0; - if (memcmp(old->digests,new->digests,sizeof(new->digests))) - return 0; - if (sizeof(old->countries) != sizeof(new->countries)) - return 0; - if (memcmp(old->countries,new->countries,sizeof(new->countries))) - return 0; - return 1; -#endif } /** Free all storage held in <b>routerset</b>. */ void routerset_free(routerset_t *routerset) { + if (!routerset) + return; + SMARTLIST_FOREACH(routerset->list, char *, cp, tor_free(cp)); smartlist_free(routerset->list); SMARTLIST_FOREACH(routerset->policies, addr_policy_t *, p, @@ -5303,8 +5696,7 @@ routerset_free(routerset_t *routerset) strmap_free(routerset->names, NULL); digestmap_free(routerset->digests, NULL); - if (routerset->countries) - bitarray_free(routerset->countries); + bitarray_free(routerset->countries); tor_free(routerset); } @@ -5335,7 +5727,6 @@ hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, { int start, found, n_added = 0, i; networkstatus_t *c = networkstatus_get_latest_consensus(); - int use_begindir = get_options()->TunnelDirConns; if (!c || !smartlist_len(c->routerstatus_list)) { log_warn(LD_REND, "We don't have a consensus, so we can't perform v2 " "rendezvous operations."); @@ -5348,14 +5739,9 @@ hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, do { routerstatus_t *r = smartlist_get(c->routerstatus_list, i); if (r->is_hs_dir) { - if (r->dir_port || use_begindir) - smartlist_add(responsible_dirs, r); - else - log_info(LD_REND, "Not adding router '%s' to list of responsible " - "hidden service directories, because we have no way of " - "reaching it.", r->nickname); + smartlist_add(responsible_dirs, r); if (++n_added == REND_NUMBER_OF_CONSECUTIVE_REPLICAS) - break; + return 0; } if (++i == smartlist_len(c->routerstatus_list)) i = 0; diff --git a/src/or/routerlist.h b/src/or/routerlist.h new file mode 100644 index 000000000..fec18705b --- /dev/null +++ b/src/or/routerlist.h @@ -0,0 +1,200 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file routerlist.h + * \brief Header file for routerlist.c. + **/ + +#ifndef _TOR_ROUTERLIST_H +#define _TOR_ROUTERLIST_H + +int get_n_authorities(authority_type_t type); +int trusted_dirs_reload_certs(void); +int trusted_dirs_load_certs_from_string(const char *contents, int from_store, + int flush); +void trusted_dirs_flush_certs_to_disk(void); +authority_cert_t *authority_cert_get_newest_by_id(const char *id_digest); +authority_cert_t *authority_cert_get_by_sk_digest(const char *sk_digest); +authority_cert_t *authority_cert_get_by_digests(const char *id_digest, + const char *sk_digest); +void authority_cert_get_all(smartlist_t *certs_out); +void authority_cert_dl_failed(const char *id_digest, int status); +void authority_certs_fetch_missing(networkstatus_t *status, time_t now); +int router_reload_router_list(void); +int authority_cert_dl_looks_uncertain(const char *id_digest); +smartlist_t *router_get_trusted_dir_servers(void); + +routerstatus_t *router_pick_directory_server(authority_type_t type, int flags); +trusted_dir_server_t *router_get_trusteddirserver_by_digest(const char *d); +trusted_dir_server_t *trusteddirserver_get_by_v3_auth_digest(const char *d); +routerstatus_t *router_pick_trusteddirserver(authority_type_t type, int flags); +int router_get_my_share_of_directory_requests(double *v2_share_out, + double *v3_share_out); +void router_reset_status_download_failures(void); +void routerlist_add_family(smartlist_t *sl, routerinfo_t *router); +int routers_in_same_family(routerinfo_t *r1, routerinfo_t *r2); +int routers_have_same_or_addr(const routerinfo_t *r1, const routerinfo_t *r2); +void add_nickname_list_to_smartlist(smartlist_t *sl, const char *list, + int must_be_running); +int router_nickname_is_in_list(routerinfo_t *router, const char *list); +routerinfo_t *routerlist_find_my_routerinfo(void); +routerinfo_t *router_find_exact_exit_enclave(const char *address, + uint16_t port); +int router_is_unreliable(routerinfo_t *router, int need_uptime, + int need_capacity, int need_guard); +uint32_t router_get_advertised_bandwidth(routerinfo_t *router); +uint32_t router_get_advertised_bandwidth_capped(routerinfo_t *router); + +routerinfo_t *routerlist_sl_choose_by_bandwidth(smartlist_t *sl, + bandwidth_weight_rule_t rule); +routerstatus_t *routerstatus_sl_choose_by_bandwidth(smartlist_t *sl, + bandwidth_weight_rule_t rule); + +routerinfo_t *router_choose_random_node(smartlist_t *excludedsmartlist, + struct routerset_t *excludedset, + router_crn_flags_t flags); + +routerinfo_t *router_get_by_nickname(const char *nickname, + int warn_if_unnamed); +int router_digest_version_as_new_as(const char *digest, const char *cutoff); +int router_digest_is_trusted_dir_type(const char *digest, + authority_type_t type); +#define router_digest_is_trusted_dir(d) \ + router_digest_is_trusted_dir_type((d), NO_AUTHORITY) + +int router_addr_is_trusted_dir(uint32_t addr); +int hexdigest_to_digest(const char *hexdigest, char *digest); +routerinfo_t *router_get_by_hexdigest(const char *hexdigest); +routerinfo_t *router_get_by_digest(const char *digest); +signed_descriptor_t *router_get_by_descriptor_digest(const char *digest); +signed_descriptor_t *router_get_by_extrainfo_digest(const char *digest); +signed_descriptor_t *extrainfo_get_by_descriptor_digest(const char *digest); +const char *signed_descriptor_get_body(signed_descriptor_t *desc); +const char *signed_descriptor_get_annotations(signed_descriptor_t *desc); +routerlist_t *router_get_routerlist(void); +void routerinfo_free(routerinfo_t *router); +void extrainfo_free(extrainfo_t *extrainfo); +void routerlist_free(routerlist_t *rl); +void dump_routerlist_mem_usage(int severity); +void routerlist_remove(routerlist_t *rl, routerinfo_t *ri, int make_old, + time_t now); +void routerlist_free_all(void); +void routerlist_reset_warnings(void); +void router_set_status(const char *digest, int up); + +static int WRA_WAS_ADDED(was_router_added_t s); +static int WRA_WAS_OUTDATED(was_router_added_t s); +static int WRA_WAS_REJECTED(was_router_added_t s); +/** Return true iff the descriptor was added. It might still be necessary to + * check whether the descriptor generator should be notified. + */ +static INLINE int +WRA_WAS_ADDED(was_router_added_t s) { + return s == ROUTER_ADDED_SUCCESSFULLY || s == ROUTER_ADDED_NOTIFY_GENERATOR; +} +/** Return true iff the descriptor was not added because it was either: + * - not in the consensus + * - neither in the consensus nor in any networkstatus document + * - it was outdated. + */ +static INLINE int WRA_WAS_OUTDATED(was_router_added_t s) +{ + return (s == ROUTER_WAS_NOT_NEW || + s == ROUTER_NOT_IN_CONSENSUS || + s == ROUTER_NOT_IN_CONSENSUS_OR_NETWORKSTATUS); +} +static INLINE int WRA_WAS_REJECTED(was_router_added_t s) +{ + return (s == ROUTER_AUTHDIR_REJECTS); +} +was_router_added_t router_add_to_routerlist(routerinfo_t *router, + const char **msg, + int from_cache, + int from_fetch); +was_router_added_t router_add_extrainfo_to_routerlist( + extrainfo_t *ei, const char **msg, + int from_cache, int from_fetch); +void routerlist_descriptors_added(smartlist_t *sl, int from_cache); +void routerlist_remove_old_routers(void); +int router_load_single_router(const char *s, uint8_t purpose, int cache, + const char **msg); +int router_load_routers_from_string(const char *s, const char *eos, + saved_location_t saved_location, + smartlist_t *requested_fingerprints, + int descriptor_digests, + const char *prepend_annotations); +void router_load_extrainfo_from_string(const char *s, const char *eos, + saved_location_t saved_location, + smartlist_t *requested_fingerprints, + int descriptor_digests); + +void routerlist_retry_directory_downloads(time_t now); +int router_exit_policy_all_routers_reject(uint32_t addr, uint16_t port, + int need_uptime); +int router_exit_policy_rejects_all(routerinfo_t *router); +trusted_dir_server_t *add_trusted_dir_server(const char *nickname, + const char *address, + uint16_t dir_port, uint16_t or_port, + const char *digest, const char *v3_auth_digest, + authority_type_t type); +void authority_cert_free(authority_cert_t *cert); +void clear_trusted_dir_servers(void); +int any_trusted_dir_is_v1_authority(void); +void update_consensus_router_descriptor_downloads(time_t now, int is_vote, + networkstatus_t *consensus); +void update_router_descriptor_downloads(time_t now); +void update_extrainfo_downloads(time_t now); +int router_have_minimum_dir_info(void); +void router_dir_info_changed(void); +const char *get_dir_info_status_string(void); +int count_loading_descriptors_progress(void); +void router_reset_descriptor_download_failures(void); +int router_differences_are_cosmetic(routerinfo_t *r1, routerinfo_t *r2); +int routerinfo_incompatible_with_extrainfo(routerinfo_t *ri, extrainfo_t *ei, + signed_descriptor_t *sd, + const char **msg); + +void routerlist_assert_ok(routerlist_t *rl); +const char *esc_router_info(routerinfo_t *router); +void routers_sort_by_identity(smartlist_t *routers); + +routerset_t *routerset_new(void); +int routerset_parse(routerset_t *target, const char *s, + const char *description); +void routerset_union(routerset_t *target, const routerset_t *source); +int routerset_is_list(const routerset_t *set); +int routerset_needs_geoip(const routerset_t *set); +int routerset_is_empty(const routerset_t *set); +int routerset_contains_router(const routerset_t *set, routerinfo_t *ri); +int routerset_contains_routerstatus(const routerset_t *set, + routerstatus_t *rs); +int routerset_contains_extendinfo(const routerset_t *set, + const extend_info_t *ei); +void routerset_get_all_routers(smartlist_t *out, const routerset_t *routerset, + const routerset_t *excludeset, + int running_only); +#if 0 +void routersets_get_disjunction(smartlist_t *target, const smartlist_t *source, + const routerset_t *include, + const routerset_t *exclude, int running_only); +#endif +void routerset_subtract_routers(smartlist_t *out, + const routerset_t *routerset); +char *routerset_to_string(const routerset_t *routerset); +void routerset_refresh_countries(routerset_t *target); +int routerset_equal(const routerset_t *old, const routerset_t *new); +void routerset_free(routerset_t *routerset); +void routerinfo_set_country(routerinfo_t *ri); +void routerlist_refresh_countries(void); +void refresh_all_country_info(void); + +int hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, + const char *id); +int hid_serv_acting_as_directory(void); +int hid_serv_responsible_for_desc_id(const char *id); + +#endif + diff --git a/src/or/routerparse.c b/src/or/routerparse.c index 3d73f8de6..d0138e638 100644 --- a/src/or/routerparse.c +++ b/src/or/routerparse.c @@ -10,7 +10,21 @@ **/ #include "or.h" +#include "config.h" +#include "circuitbuild.h" +#include "dirserv.h" +#include "dirvote.h" +#include "policies.h" +#include "rendcommon.h" +#include "router.h" +#include "routerlist.h" #include "memarea.h" +#include "microdesc.h" +#include "networkstatus.h" +#include "rephist.h" +#include "routerparse.h" +#undef log +#include <math.h> /****************************************************************************/ @@ -55,6 +69,7 @@ typedef enum { K_S, K_V, K_W, + K_M, K_EVENTDNS, K_EXTRA_INFO, K_EXTRA_INFO_DIGEST, @@ -62,6 +77,31 @@ typedef enum { K_HIDDEN_SERVICE_DIR, K_ALLOW_SINGLE_HOP_EXITS, + K_DIRREQ_END, + K_DIRREQ_V2_IPS, + K_DIRREQ_V3_IPS, + K_DIRREQ_V2_REQS, + K_DIRREQ_V3_REQS, + K_DIRREQ_V2_SHARE, + K_DIRREQ_V3_SHARE, + K_DIRREQ_V2_RESP, + K_DIRREQ_V3_RESP, + K_DIRREQ_V2_DIR, + K_DIRREQ_V3_DIR, + K_DIRREQ_V2_TUN, + K_DIRREQ_V3_TUN, + K_ENTRY_END, + K_ENTRY_IPS, + K_CELL_END, + K_CELL_PROCESSED, + K_CELL_QUEUED, + K_CELL_TIME, + K_CELL_CIRCS, + K_EXIT_END, + K_EXIT_WRITTEN, + K_EXIT_READ, + K_EXIT_OPENED, + K_DIR_KEY_CERTIFICATE_VERSION, K_DIR_IDENTITY_KEY, K_DIR_KEY_PUBLISHED, @@ -78,13 +118,18 @@ typedef enum { K_KNOWN_FLAGS, K_PARAMS, + K_BW_WEIGHTS, K_VOTE_DIGEST, K_CONSENSUS_DIGEST, + K_ADDITIONAL_DIGEST, + K_ADDITIONAL_SIGNATURE, K_CONSENSUS_METHODS, K_CONSENSUS_METHOD, K_LEGACY_DIR_KEY, + K_DIRECTORY_FOOTER, A_PURPOSE, + A_LAST_LISTED, _A_UNKNOWN, R_RENDEZVOUS_SERVICE_DESCRIPTOR, @@ -122,7 +167,7 @@ typedef enum { * type. * * This structure is only allocated in memareas; do not allocate it on - * the heap, or token_free() won't work. + * the heap, or token_clear() won't work. */ typedef struct directory_token_t { directory_keyword tp; /**< Type of the token. */ @@ -258,6 +303,31 @@ static token_rule_t extrainfo_token_table[] = { T0N("opt", K_OPT, CONCAT_ARGS, OBJ_OK ), T01("read-history", K_READ_HISTORY, ARGS, NO_OBJ ), T01("write-history", K_WRITE_HISTORY, ARGS, NO_OBJ ), + T01("dirreq-stats-end", K_DIRREQ_END, ARGS, NO_OBJ ), + T01("dirreq-v2-ips", K_DIRREQ_V2_IPS, ARGS, NO_OBJ ), + T01("dirreq-v3-ips", K_DIRREQ_V3_IPS, ARGS, NO_OBJ ), + T01("dirreq-v2-reqs", K_DIRREQ_V2_REQS, ARGS, NO_OBJ ), + T01("dirreq-v3-reqs", K_DIRREQ_V3_REQS, ARGS, NO_OBJ ), + T01("dirreq-v2-share", K_DIRREQ_V2_SHARE, ARGS, NO_OBJ ), + T01("dirreq-v3-share", K_DIRREQ_V3_SHARE, ARGS, NO_OBJ ), + T01("dirreq-v2-resp", K_DIRREQ_V2_RESP, ARGS, NO_OBJ ), + T01("dirreq-v3-resp", K_DIRREQ_V3_RESP, ARGS, NO_OBJ ), + T01("dirreq-v2-direct-dl", K_DIRREQ_V2_DIR, ARGS, NO_OBJ ), + T01("dirreq-v3-direct-dl", K_DIRREQ_V3_DIR, ARGS, NO_OBJ ), + T01("dirreq-v2-tunneled-dl", K_DIRREQ_V2_TUN, ARGS, NO_OBJ ), + T01("dirreq-v3-tunneled-dl", K_DIRREQ_V3_TUN, ARGS, NO_OBJ ), + T01("entry-stats-end", K_ENTRY_END, ARGS, NO_OBJ ), + T01("entry-ips", K_ENTRY_IPS, ARGS, NO_OBJ ), + T01("cell-stats-end", K_CELL_END, ARGS, NO_OBJ ), + T01("cell-processed-cells", K_CELL_PROCESSED, ARGS, NO_OBJ ), + T01("cell-queued-cells", K_CELL_QUEUED, ARGS, NO_OBJ ), + T01("cell-time-in-queue", K_CELL_TIME, ARGS, NO_OBJ ), + T01("cell-circuits-per-decile", K_CELL_CIRCS, ARGS, NO_OBJ ), + T01("exit-stats-end", K_EXIT_END, ARGS, NO_OBJ ), + T01("exit-kibibytes-written", K_EXIT_WRITTEN, ARGS, NO_OBJ ), + T01("exit-kibibytes-read", K_EXIT_READ, ARGS, NO_OBJ ), + T01("exit-streams-opened", K_EXIT_OPENED, ARGS, NO_OBJ ), + T1_START( "extra-info", K_EXTRA_INFO, GE(2), NO_OBJ ), END_OF_TABLE @@ -267,10 +337,11 @@ static token_rule_t extrainfo_token_table[] = { * documents. */ static token_rule_t rtrstatus_token_table[] = { T01("p", K_P, CONCAT_ARGS, NO_OBJ ), - T1( "r", K_R, GE(8), NO_OBJ ), + T1( "r", K_R, GE(7), NO_OBJ ), T1( "s", K_S, ARGS, NO_OBJ ), T01("v", K_V, CONCAT_ARGS, NO_OBJ ), T01("w", K_W, ARGS, NO_OBJ ), + T0N("m", K_M, CONCAT_ARGS, NO_OBJ ), T0N("opt", K_OPT, CONCAT_ARGS, OBJ_OK ), END_OF_TABLE }; @@ -375,7 +446,7 @@ static token_rule_t client_keys_token_table[] = { /** List of tokens allowed in V3 networkstatus votes. */ static token_rule_t networkstatus_token_table[] = { - T1("network-status-version", K_NETWORK_STATUS_VERSION, + T1_START("network-status-version", K_NETWORK_STATUS_VERSION, GE(1), NO_OBJ ), T1("vote-status", K_VOTE_STATUS, GE(1), NO_OBJ ), T1("published", K_PUBLISHED, CONCAT_ARGS, NO_OBJ ), @@ -403,7 +474,7 @@ static token_rule_t networkstatus_token_table[] = { /** List of tokens allowed in V3 networkstatus consensuses. */ static token_rule_t networkstatus_consensus_token_table[] = { - T1("network-status-version", K_NETWORK_STATUS_VERSION, + T1_START("network-status-version", K_NETWORK_STATUS_VERSION, GE(1), NO_OBJ ), T1("vote-status", K_VOTE_STATUS, GE(1), NO_OBJ ), T1("valid-after", K_VALID_AFTER, CONCAT_ARGS, NO_OBJ ), @@ -430,17 +501,29 @@ static token_rule_t networkstatus_consensus_token_table[] = { /** List of tokens allowable in the footer of v1/v2 directory/networkstatus * footers. */ static token_rule_t networkstatus_vote_footer_token_table[] = { - T( "directory-signature", K_DIRECTORY_SIGNATURE, GE(2), NEED_OBJ ), + T01("directory-footer", K_DIRECTORY_FOOTER, NO_ARGS, NO_OBJ ), + T01("bandwidth-weights", K_BW_WEIGHTS, ARGS, NO_OBJ ), + T( "directory-signature", K_DIRECTORY_SIGNATURE, GE(2), NEED_OBJ ), END_OF_TABLE }; /** List of tokens allowable in detached networkstatus signature documents. */ static token_rule_t networkstatus_detached_signature_token_table[] = { T1_START("consensus-digest", K_CONSENSUS_DIGEST, GE(1), NO_OBJ ), + T("additional-digest", K_ADDITIONAL_DIGEST,GE(3), NO_OBJ ), T1("valid-after", K_VALID_AFTER, CONCAT_ARGS, NO_OBJ ), T1("fresh-until", K_FRESH_UNTIL, CONCAT_ARGS, NO_OBJ ), T1("valid-until", K_VALID_UNTIL, CONCAT_ARGS, NO_OBJ ), - T1N("directory-signature", K_DIRECTORY_SIGNATURE, GE(2), NEED_OBJ ), + T("additional-signature", K_ADDITIONAL_SIGNATURE, GE(4), NEED_OBJ ), + T1N("directory-signature", K_DIRECTORY_SIGNATURE, GE(2), NEED_OBJ ), + END_OF_TABLE +}; + +static token_rule_t microdesc_token_table[] = { + T1_START("onion-key", K_ONION_KEY, NO_ARGS, NEED_KEY_1024), + T01("family", K_FAMILY, ARGS, NO_OBJ ), + T01("p", K_P, CONCAT_ARGS, NO_OBJ ), + A01("@last-listed", A_LAST_LISTED, CONCAT_ARGS, NO_OBJ ), END_OF_TABLE }; @@ -453,9 +536,13 @@ static addr_policy_t *router_parse_addr_policy_private(directory_token_t *tok); static int router_get_hash_impl(const char *s, size_t s_len, char *digest, const char *start_str, const char *end_str, - char end_char); - -static void token_free(directory_token_t *tok); + char end_char, + digest_algorithm_t alg); +static int router_get_hashes_impl(const char *s, size_t s_len, + digests_t *digests, + const char *start_str, const char *end_str, + char end_char); +static void token_clear(directory_token_t *tok); static smartlist_t *find_all_exitpolicy(smartlist_t *s); static directory_token_t *_find_by_keyword(smartlist_t *s, directory_keyword keyword, @@ -479,6 +566,7 @@ static directory_token_t *get_next_token(memarea_t *area, #define CST_CHECK_AUTHORITY (1<<0) #define CST_NO_CHECK_OBJTYPE (1<<1) static int check_signature_token(const char *digest, + ssize_t digest_len, directory_token_t *tok, crypto_pk_env_t *pkey, int flags, @@ -499,6 +587,34 @@ static int tor_version_same_series(tor_version_t *a, tor_version_t *b); #define DUMP_AREA(a,name) STMT_NIL #endif +/** Last time we dumped a descriptor to disk. */ +static time_t last_desc_dumped = 0; + +/** For debugging purposes, dump unparseable descriptor *<b>desc</b> of + * type *<b>type</b> to file $DATADIR/unparseable-desc. Do not write more + * than one descriptor to disk per minute. If there is already such a + * file in the data directory, overwrite it. */ +static void +dump_desc(const char *desc, const char *type) +{ + time_t now = time(NULL); + tor_assert(desc); + tor_assert(type); + if (!last_desc_dumped || last_desc_dumped + 60 < now) { + char *debugfile = get_datadir_fname("unparseable-desc"); + size_t filelen = 50 + strlen(type) + strlen(desc); + char *content = tor_malloc_zero(filelen); + tor_snprintf(content, filelen, "Unable to parse descriptor of type " + "%s:\n%s", type, desc); + write_str_to_file(debugfile, content, 0); + log_info(LD_DIR, "Unable to parse descriptor of type %s. See file " + "unparseable-desc in data directory for details.", type); + tor_free(content); + tor_free(debugfile); + last_desc_dumped = now; + } +} + /** Set <b>digest</b> to the SHA-1 digest of the hash of the directory in * <b>s</b>. Return 0 on success, -1 on failure. */ @@ -506,7 +622,8 @@ int router_get_dir_hash(const char *s, char *digest) { return router_get_hash_impl(s, strlen(s), digest, - "signed-directory","\ndirectory-signature",'\n'); + "signed-directory","\ndirectory-signature",'\n', + DIGEST_SHA1); } /** Set <b>digest</b> to the SHA-1 digest of the hash of the first router in @@ -516,7 +633,8 @@ int router_get_router_hash(const char *s, size_t s_len, char *digest) { return router_get_hash_impl(s, s_len, digest, - "router ","\nrouter-signature", '\n'); + "router ","\nrouter-signature", '\n', + DIGEST_SHA1); } /** Set <b>digest</b> to the SHA-1 digest of the hash of the running-routers @@ -526,7 +644,8 @@ int router_get_runningrouters_hash(const char *s, char *digest) { return router_get_hash_impl(s, strlen(s), digest, - "network-status","\ndirectory-signature", '\n'); + "network-status","\ndirectory-signature", '\n', + DIGEST_SHA1); } /** Set <b>digest</b> to the SHA-1 digest of the hash of the network-status @@ -536,18 +655,31 @@ router_get_networkstatus_v2_hash(const char *s, char *digest) { return router_get_hash_impl(s, strlen(s), digest, "network-status-version","\ndirectory-signature", - '\n'); + '\n', + DIGEST_SHA1); +} + +/** Set <b>digests</b> to all the digests of the consensus document in + * <b>s</b> */ +int +router_get_networkstatus_v3_hashes(const char *s, digests_t *digests) +{ + return router_get_hashes_impl(s,strlen(s),digests, + "network-status-version", + "\ndirectory-signature", + ' '); } /** Set <b>digest</b> to the SHA-1 digest of the hash of the network-status * string in <b>s</b>. Return 0 on success, -1 on failure. */ int -router_get_networkstatus_v3_hash(const char *s, char *digest) +router_get_networkstatus_v3_hash(const char *s, char *digest, + digest_algorithm_t alg) { return router_get_hash_impl(s, strlen(s), digest, "network-status-version", "\ndirectory-signature", - ' '); + ' ', alg); } /** Set <b>digest</b> to the SHA-1 digest of the hash of the extrainfo @@ -556,7 +688,7 @@ int router_get_extrainfo_hash(const char *s, char *digest) { return router_get_hash_impl(s, strlen(s), digest, "extra-info", - "\nrouter-signature",'\n'); + "\nrouter-signature",'\n', DIGEST_SHA1); } /** Helper: used to generate signatures for routers, directories and @@ -568,16 +700,17 @@ router_get_extrainfo_hash(const char *s, char *digest) */ int router_append_dirobj_signature(char *buf, size_t buf_len, const char *digest, - crypto_pk_env_t *private_key) + size_t digest_len, crypto_pk_env_t *private_key) { char *signature; size_t i, keysize; + int siglen; keysize = crypto_pk_keysize(private_key); signature = tor_malloc(keysize); - if (crypto_pk_private_sign(private_key, signature, keysize, - digest, DIGEST_LEN) < 0) { - + siglen = crypto_pk_private_sign(private_key, signature, keysize, + digest, digest_len); + if (siglen < 0) { log_warn(LD_BUG,"Couldn't sign digest."); goto err; } @@ -585,7 +718,7 @@ router_append_dirobj_signature(char *buf, size_t buf_len, const char *digest, goto truncated; i = strlen(buf); - if (base64_encode(buf+i, buf_len-i, signature, 128) < 0) { + if (base64_encode(buf+i, buf_len-i, signature, siglen) < 0) { log_warn(LD_BUG,"couldn't base64-encode signature"); goto err; } @@ -692,7 +825,7 @@ router_parse_directory(const char *str) char digest[DIGEST_LEN]; time_t published_on; int r; - const char *end, *cp; + const char *end, *cp, *str_dup = str; smartlist_t *tokens = NULL; crypto_pk_env_t *declared_key = NULL; memarea_t *area = memarea_new(); @@ -728,11 +861,11 @@ router_parse_directory(const char *str) } declared_key = find_dir_signing_key(str, str+strlen(str)); note_crypto_pk_op(VERIFY_DIR); - if (check_signature_token(digest, tok, declared_key, + if (check_signature_token(digest, DIGEST_LEN, tok, declared_key, CST_CHECK_AUTHORITY, "directory")<0) goto err; - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_clear(tokens); memarea_clear(area); @@ -765,11 +898,12 @@ router_parse_directory(const char *str) r = 0; goto done; err: + dump_desc(str_dup, "v1 directory"); r = -1; done: if (declared_key) crypto_free_pk_env(declared_key); if (tokens) { - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); } if (area) { @@ -791,7 +925,7 @@ router_parse_runningrouters(const char *str) int r = -1; crypto_pk_env_t *declared_key = NULL; smartlist_t *tokens = NULL; - const char *eos = str + strlen(str); + const char *eos = str + strlen(str), *str_dup = str; memarea_t *area = NULL; if (router_get_runningrouters_hash(str, digest)) { @@ -820,7 +954,7 @@ router_parse_runningrouters(const char *str) } declared_key = find_dir_signing_key(str, eos); note_crypto_pk_op(VERIFY_DIR); - if (check_signature_token(digest, tok, declared_key, + if (check_signature_token(digest, DIGEST_LEN, tok, declared_key, CST_CHECK_AUTHORITY, "running-routers") < 0) goto err; @@ -832,9 +966,10 @@ router_parse_runningrouters(const char *str) r = 0; err: + dump_desc(str_dup, "v1 running-routers"); if (declared_key) crypto_free_pk_env(declared_key); if (tokens) { - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); } if (area) { @@ -884,7 +1019,7 @@ find_dir_signing_key(const char *str, const char *eos) } done: - if (tok) token_free(tok); + if (tok) token_clear(tok); if (area) { DUMP_AREA(area, "dir-signing-key token"); memarea_drop_all(area); @@ -920,6 +1055,7 @@ dir_signing_key_is_trusted(crypto_pk_env_t *key) */ static int check_signature_token(const char *digest, + ssize_t digest_len, directory_token_t *tok, crypto_pk_env_t *pkey, int flags, @@ -952,14 +1088,14 @@ check_signature_token(const char *digest, signed_digest = tor_malloc(keysize); if (crypto_pk_public_checksig(pkey, signed_digest, keysize, tok->object_body, tok->object_size) - != DIGEST_LEN) { + < digest_len) { log_warn(LD_DIR, "Error reading %s: invalid signature.", doctype); tor_free(signed_digest); return -1; } // log_debug(LD_DIR,"Signed %s hash starts %s", doctype, // hex_str(signed_digest,4)); - if (memcmp(digest, signed_digest, DIGEST_LEN)) { + if (memcmp(digest, signed_digest, digest_len)) { log_warn(LD_DIR, "Error reading %s: signature does not match.", doctype); tor_free(signed_digest); return -1; @@ -1145,7 +1281,7 @@ router_parse_entry_from_string(const char *s, const char *end, smartlist_t *tokens = NULL, *exit_policy_tokens = NULL; directory_token_t *tok; struct in_addr in; - const char *start_of_annotations, *cp; + const char *start_of_annotations, *cp, *s_dup = s; size_t prepend_len = prepend_annotations ? strlen(prepend_annotations) : 0; int ok = 1; memarea_t *area = NULL; @@ -1429,7 +1565,7 @@ router_parse_entry_from_string(const char *s, const char *end, verified_digests = digestmap_new(); digestmap_set(verified_digests, signed_digest, (void*)(uintptr_t)1); #endif - if (check_signature_token(digest, tok, router->identity_pkey, 0, + if (check_signature_token(digest, DIGEST_LEN, tok, router->identity_pkey, 0, "router descriptor") < 0) goto err; @@ -1447,16 +1583,15 @@ router_parse_entry_from_string(const char *s, const char *end, goto done; err: + dump_desc(s_dup, "router descriptor"); routerinfo_free(router); router = NULL; done: if (tokens) { - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); } - if (exit_policy_tokens) { - smartlist_free(exit_policy_tokens); - } + smartlist_free(exit_policy_tokens); if (area) { DUMP_AREA(area, "routerinfo"); memarea_drop_all(area); @@ -1481,6 +1616,7 @@ extrainfo_parse_entry_from_string(const char *s, const char *end, crypto_pk_env_t *key = NULL; routerinfo_t *router = NULL; memarea_t *area = NULL; + const char *s_dup = s; if (!end) { end = s + strlen(s); @@ -1555,7 +1691,8 @@ extrainfo_parse_entry_from_string(const char *s, const char *end, if (key) { note_crypto_pk_op(VERIFY_RTR); - if (check_signature_token(digest, tok, key, 0, "extra-info") < 0) + if (check_signature_token(digest, DIGEST_LEN, tok, key, 0, + "extra-info") < 0) goto err; if (router) @@ -1569,12 +1706,12 @@ extrainfo_parse_entry_from_string(const char *s, const char *end, goto done; err: - if (extrainfo) - extrainfo_free(extrainfo); + dump_desc(s_dup, "extra-info descriptor"); + extrainfo_free(extrainfo); extrainfo = NULL; done: if (tokens) { - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); } if (area) { @@ -1602,6 +1739,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) size_t len; int found; memarea_t *area = NULL; + const char *s_dup = s; s = eat_whitespace(s); eos = strstr(s, "\ndir-key-certification"); @@ -1632,7 +1770,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) goto err; } if (router_get_hash_impl(s, strlen(s), digest, "dir-key-certificate-version", - "\ndir-key-certification", '\n') < 0) + "\ndir-key-certification", '\n', DIGEST_SHA1) < 0) goto err; tok = smartlist_get(tokens, 0); if (tok->tp != K_DIR_KEY_CERTIFICATE_VERSION || strcmp(tok->args[0], "3")) { @@ -1680,7 +1818,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) struct in_addr in; char *address = NULL; tor_assert(tok->n_args); - /* XXX021 use tor_addr_port_parse() below instead. -RD */ + /* XXX023 use tor_addr_port_parse() below instead. -RD */ if (parse_addr_port(LOG_WARN, tok->args[0], &address, NULL, &cert->dir_port)<0 || tor_inet_aton(address, &in) == 0) { @@ -1725,7 +1863,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) } } if (!found) { - if (check_signature_token(digest, tok, cert->identity_key, 0, + if (check_signature_token(digest, DIGEST_LEN, tok, cert->identity_key, 0, "key certificate")) { goto err; } @@ -1734,6 +1872,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) /* XXXX Once all authorities generate cross-certified certificates, * make this field mandatory. */ if (check_signature_token(cert->cache_info.identity_digest, + DIGEST_LEN, tok, cert->signing_key, CST_NO_CHECK_OBJTYPE, @@ -1753,7 +1892,7 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) if (end_of_string) { *end_of_string = eat_whitespace(eos); } - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); if (area) { DUMP_AREA(area, "authority cert"); @@ -1761,8 +1900,9 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) } return cert; err: + dump_desc(s_dup, "authority cert"); authority_cert_free(cert); - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); if (area) { DUMP_AREA(area, "authority cert"); @@ -1773,23 +1913,28 @@ authority_cert_parse_from_string(const char *s, const char **end_of_string) /** Helper: given a string <b>s</b>, return the start of the next router-status * object (starting with "r " at the start of a line). If none is found, - * return the start of the next directory signature. If none is found, return - * the end of the string. */ + * return the start of the directory footer, or the next directory signature. + * If none is found, return the end of the string. */ static INLINE const char * find_start_of_next_routerstatus(const char *s) { - const char *eos = strstr(s, "\nr "); - if (eos) { - const char *eos2 = tor_memstr(s, eos-s, "\ndirectory-signature"); - if (eos2 && eos2 < eos) - return eos2; - else - return eos+1; - } else { - if ((eos = strstr(s, "\ndirectory-signature"))) - return eos+1; - return s + strlen(s); - } + const char *eos, *footer, *sig; + if ((eos = strstr(s, "\nr "))) + ++eos; + else + eos = s + strlen(s); + + footer = tor_memstr(s, eos-s, "\ndirectory-footer"); + sig = tor_memstr(s, eos-s, "\ndirectory-signature"); + + if (footer && sig) + return MIN(footer, sig) + 1; + else if (footer) + return footer+1; + else if (sig) + return sig+1; + else + return eos; } /** Given a string at *<b>s</b>, containing a routerstatus object, and an @@ -1803,22 +1948,29 @@ find_start_of_next_routerstatus(const char *s) * If <b>consensus_method</b> is nonzero, this routerstatus is part of a * consensus, and we should parse it according to the method used to * make that consensus. + * + * Parse according to the syntax used by the consensus flavor <b>flav</b>. **/ static routerstatus_t * routerstatus_parse_entry_from_string(memarea_t *area, const char **s, smartlist_t *tokens, networkstatus_t *vote, vote_routerstatus_t *vote_rs, - int consensus_method) + int consensus_method, + consensus_flavor_t flav) { - const char *eos; + const char *eos, *s_dup = *s; routerstatus_t *rs = NULL; directory_token_t *tok; char timebuf[ISO_TIME_LEN+1]; struct in_addr in; + int offset = 0; tor_assert(tokens); tor_assert(bool_eq(vote, vote_rs)); + if (!consensus_method) + flav = FLAV_NS; + eos = find_start_of_next_routerstatus(*s); if (tokenize_string(area,*s, eos, tokens, rtrstatus_token_table,0)) { @@ -1830,7 +1982,15 @@ routerstatus_parse_entry_from_string(memarea_t *area, goto err; } tok = find_by_keyword(tokens, K_R); - tor_assert(tok->n_args >= 8); + tor_assert(tok->n_args >= 7); + if (flav == FLAV_NS) { + if (tok->n_args < 8) { + log_warn(LD_DIR, "Too few arguments to r"); + goto err; + } + } else { + offset = -1; + } if (vote_rs) { rs = &vote_rs->status; } else { @@ -1851,29 +2011,34 @@ routerstatus_parse_entry_from_string(memarea_t *area, goto err; } - if (digest_from_base64(rs->descriptor_digest, tok->args[2])) { - log_warn(LD_DIR, "Error decoding descriptor digest %s", - escaped(tok->args[2])); - goto err; + if (flav == FLAV_NS) { + if (digest_from_base64(rs->descriptor_digest, tok->args[2])) { + log_warn(LD_DIR, "Error decoding descriptor digest %s", + escaped(tok->args[2])); + goto err; + } } if (tor_snprintf(timebuf, sizeof(timebuf), "%s %s", - tok->args[3], tok->args[4]) < 0 || + tok->args[3+offset], tok->args[4+offset]) < 0 || parse_iso_time(timebuf, &rs->published_on)<0) { - log_warn(LD_DIR, "Error parsing time '%s %s'", - tok->args[3], tok->args[4]); + log_warn(LD_DIR, "Error parsing time '%s %s' [%d %d]", + tok->args[3+offset], tok->args[4+offset], + offset, (int)flav); goto err; } - if (tor_inet_aton(tok->args[5], &in) == 0) { + if (tor_inet_aton(tok->args[5+offset], &in) == 0) { log_warn(LD_DIR, "Error parsing router address in network-status %s", - escaped(tok->args[5])); + escaped(tok->args[5+offset])); goto err; } rs->addr = ntohl(in.s_addr); - rs->or_port =(uint16_t) tor_parse_long(tok->args[6],10,0,65535,NULL,NULL); - rs->dir_port = (uint16_t) tor_parse_long(tok->args[7],10,0,65535,NULL,NULL); + rs->or_port = (uint16_t) tor_parse_long(tok->args[6+offset], + 10,0,65535,NULL,NULL); + rs->dir_port = (uint16_t) tor_parse_long(tok->args[7+offset], + 10,0,65535,NULL,NULL); tok = find_opt_by_keyword(tokens, K_S); if (tok && vote) { @@ -1959,6 +2124,17 @@ routerstatus_parse_entry_from_string(memarea_t *area, goto err; } rs->has_bandwidth = 1; + } else if (!strcmpstart(tok->args[i], "Measured=")) { + int ok; + rs->measured_bw = + (uint32_t)tor_parse_ulong(strchr(tok->args[i], '=')+1, + 10, 0, UINT32_MAX, &ok, NULL); + if (!ok) { + log_warn(LD_DIR, "Invalid Measured Bandwidth %s", + escaped(tok->args[i])); + goto err; + } + rs->has_measured_bw = 1; } } } @@ -1980,16 +2156,29 @@ routerstatus_parse_entry_from_string(memarea_t *area, rs->has_exitsummary = 1; } + if (vote_rs) { + SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, t) { + if (t->tp == K_M && t->n_args) { + vote_microdesc_hash_t *line = + tor_malloc(sizeof(vote_microdesc_hash_t)); + line->next = vote_rs->microdesc; + line->microdesc_hash_line = tor_strdup(t->args[0]); + vote_rs->microdesc = line; + } + } SMARTLIST_FOREACH_END(t); + } + if (!strcasecmp(rs->nickname, UNNAMED_ROUTER_NICKNAME)) rs->is_named = 0; goto done; err: + dump_desc(s_dup, "routerstatus entry"); if (rs && !vote_rs) routerstatus_free(rs); rs = NULL; done: - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_clear(tokens); if (area) { DUMP_AREA(area, "routerstatus entry"); @@ -2001,8 +2190,8 @@ routerstatus_parse_entry_from_string(memarea_t *area, } /** Helper to sort a smartlist of pointers to routerstatus_t */ -static int -_compare_routerstatus_entries(const void **_a, const void **_b) +int +compare_routerstatus_entries(const void **_a, const void **_b) { const routerstatus_t *a = *_a, *b = *_b; return memcmp(a->identity_digest, b->identity_digest, DIGEST_LEN); @@ -2026,7 +2215,7 @@ _free_duplicate_routerstatus_entry(void *e) networkstatus_v2_t * networkstatus_v2_parse_from_string(const char *s) { - const char *eos; + const char *eos, *s_dup = s; smartlist_t *tokens = smartlist_create(); smartlist_t *footer_tokens = smartlist_create(); networkstatus_v2_t *ns = NULL; @@ -2140,17 +2329,17 @@ networkstatus_v2_parse_from_string(const char *s) ns->entries = smartlist_create(); s = eos; - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_clear(tokens); memarea_clear(area); while (!strcmpstart(s, "r ")) { routerstatus_t *rs; if ((rs = routerstatus_parse_entry_from_string(area, &s, tokens, - NULL, NULL, 0))) + NULL, NULL, 0, 0))) smartlist_add(ns->entries, rs); } - smartlist_sort(ns->entries, _compare_routerstatus_entries); - smartlist_uniq(ns->entries, _compare_routerstatus_entries, + smartlist_sort(ns->entries, compare_routerstatus_entries); + smartlist_uniq(ns->entries, compare_routerstatus_entries, _free_duplicate_routerstatus_entry); if (tokenize_string(area,s, NULL, footer_tokens, dir_footer_token_table,0)) { @@ -2169,19 +2358,19 @@ networkstatus_v2_parse_from_string(const char *s) } note_crypto_pk_op(VERIFY_DIR); - if (check_signature_token(ns_digest, tok, ns->signing_key, 0, + if (check_signature_token(ns_digest, DIGEST_LEN, tok, ns->signing_key, 0, "network-status") < 0) goto err; goto done; err: - if (ns) - networkstatus_v2_free(ns); + dump_desc(s_dup, "v2 networkstatus"); + networkstatus_v2_free(ns); ns = NULL; done: - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); - SMARTLIST_FOREACH(footer_tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(footer_tokens, directory_token_t *, t, token_clear(t)); smartlist_free(footer_tokens); if (area) { DUMP_AREA(area, "v2 networkstatus"); @@ -2190,6 +2379,395 @@ networkstatus_v2_parse_from_string(const char *s) return ns; } +/** Verify the bandwidth weights of a network status document */ +int +networkstatus_verify_bw_weights(networkstatus_t *ns) +{ + int64_t weight_scale; + int64_t G=0, M=0, E=0, D=0, T=0; + double Wgg, Wgm, Wgd, Wmg, Wmm, Wme, Wmd, Weg, Wem, Wee, Wed; + double Gtotal=0, Mtotal=0, Etotal=0; + const char *casename = NULL; + int valid = 1; + + weight_scale = circuit_build_times_get_bw_scale(ns); + Wgg = networkstatus_get_bw_weight(ns, "Wgg", -1); + Wgm = networkstatus_get_bw_weight(ns, "Wgm", -1); + Wgd = networkstatus_get_bw_weight(ns, "Wgd", -1); + Wmg = networkstatus_get_bw_weight(ns, "Wmg", -1); + Wmm = networkstatus_get_bw_weight(ns, "Wmm", -1); + Wme = networkstatus_get_bw_weight(ns, "Wme", -1); + Wmd = networkstatus_get_bw_weight(ns, "Wmd", -1); + Weg = networkstatus_get_bw_weight(ns, "Weg", -1); + Wem = networkstatus_get_bw_weight(ns, "Wem", -1); + Wee = networkstatus_get_bw_weight(ns, "Wee", -1); + Wed = networkstatus_get_bw_weight(ns, "Wed", -1); + + if (Wgg<0 || Wgm<0 || Wgd<0 || Wmg<0 || Wmm<0 || Wme<0 || Wmd<0 || Weg<0 + || Wem<0 || Wee<0 || Wed<0) { + log_warn(LD_BUG, "No bandwidth weights produced in consensus!"); + return 0; + } + + // First, sanity check basic summing properties that hold for all cases + // We use > 1 as the check for these because they are computed as integers. + // Sometimes there are rounding errors. + if (fabs(Wmm - weight_scale) > 1) { + log_warn(LD_BUG, "Wmm=%lf != "I64_FORMAT, + Wmm, I64_PRINTF_ARG(weight_scale)); + valid = 0; + } + + if (fabs(Wem - Wee) > 1) { + log_warn(LD_BUG, "Wem=%lf != Wee=%lf", Wem, Wee); + valid = 0; + } + + if (fabs(Wgm - Wgg) > 1) { + log_warn(LD_BUG, "Wgm=%lf != Wgg=%lf", Wgm, Wgg); + valid = 0; + } + + if (fabs(Weg - Wed) > 1) { + log_warn(LD_BUG, "Wed=%lf != Weg=%lf", Wed, Weg); + valid = 0; + } + + if (fabs(Wgg + Wmg - weight_scale) > 0.001*weight_scale) { + log_warn(LD_BUG, "Wgg=%lf != "I64_FORMAT" - Wmg=%lf", Wgg, + I64_PRINTF_ARG(weight_scale), Wmg); + valid = 0; + } + + if (fabs(Wee + Wme - weight_scale) > 0.001*weight_scale) { + log_warn(LD_BUG, "Wee=%lf != "I64_FORMAT" - Wme=%lf", Wee, + I64_PRINTF_ARG(weight_scale), Wme); + valid = 0; + } + + if (fabs(Wgd + Wmd + Wed - weight_scale) > 0.001*weight_scale) { + log_warn(LD_BUG, "Wgd=%lf + Wmd=%lf + Wed=%lf != "I64_FORMAT, + Wgd, Wmd, Wed, I64_PRINTF_ARG(weight_scale)); + valid = 0; + } + + Wgg /= weight_scale; + Wgm /= weight_scale; + Wgd /= weight_scale; + + Wmg /= weight_scale; + Wmm /= weight_scale; + Wme /= weight_scale; + Wmd /= weight_scale; + + Weg /= weight_scale; + Wem /= weight_scale; + Wee /= weight_scale; + Wed /= weight_scale; + + // Then, gather G, M, E, D, T to determine case + SMARTLIST_FOREACH_BEGIN(ns->routerstatus_list, routerstatus_t *, rs) { + if (rs->has_bandwidth) { + T += rs->bandwidth; + if (rs->is_exit && rs->is_possible_guard) { + D += rs->bandwidth; + Gtotal += Wgd*rs->bandwidth; + Mtotal += Wmd*rs->bandwidth; + Etotal += Wed*rs->bandwidth; + } else if (rs->is_exit) { + E += rs->bandwidth; + Mtotal += Wme*rs->bandwidth; + Etotal += Wee*rs->bandwidth; + } else if (rs->is_possible_guard) { + G += rs->bandwidth; + Gtotal += Wgg*rs->bandwidth; + Mtotal += Wmg*rs->bandwidth; + } else { + M += rs->bandwidth; + Mtotal += Wmm*rs->bandwidth; + } + } else { + log_warn(LD_BUG, "Missing consensus bandwidth for router %s", + rs->nickname); + } + } SMARTLIST_FOREACH_END(rs); + + // Finally, check equality conditions depending upon case 1, 2 or 3 + // Full equality cases: 1, 3b + // Partial equality cases: 2b (E=G), 3a (M=E) + // Fully unknown: 2a + if (3*E >= T && 3*G >= T) { + // Case 1: Neither are scarce + casename = "Case 1"; + if (fabs(Etotal-Mtotal) > 0.01*MAX(Etotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Mtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Mtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (fabs(Etotal-Gtotal) > 0.01*MAX(Etotal,Gtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (fabs(Gtotal-Mtotal) > 0.01*MAX(Gtotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Mtotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Mtotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } else if (3*E < T && 3*G < T) { + int64_t R = MIN(E, G); + int64_t S = MAX(E, G); + /* + * Case 2: Both Guards and Exits are scarce + * Balance D between E and G, depending upon + * D capacity and scarcity. Devote no extra + * bandwidth to middle nodes. + */ + if (R+D < S) { // Subcase a + double Rtotal, Stotal; + if (E < G) { + Rtotal = Etotal; + Stotal = Gtotal; + } else { + Rtotal = Gtotal; + Stotal = Etotal; + } + casename = "Case 2a"; + // Rtotal < Stotal + if (Rtotal > Stotal) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Rtotal %lf > Stotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Rtotal, Stotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + // Rtotal < T/3 + if (3*Rtotal > T) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: 3*Rtotal %lf > T " + I64_FORMAT". G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Rtotal*3, I64_PRINTF_ARG(T), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + // Stotal < T/3 + if (3*Stotal > T) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: 3*Stotal %lf > T " + I64_FORMAT". G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Stotal*3, I64_PRINTF_ARG(T), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + // Mtotal > T/3 + if (3*Mtotal < T) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: 3*Mtotal %lf < T " + I64_FORMAT". " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Mtotal*3, I64_PRINTF_ARG(T), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } else { // Subcase b: R+D > S + casename = "Case 2b"; + + /* Check the rare-M redirect case. */ + if (D != 0 && 3*M < T) { + casename = "Case 2b (balanced)"; + if (fabs(Etotal-Mtotal) > 0.01*MAX(Etotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Mtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Mtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (fabs(Etotal-Gtotal) > 0.01*MAX(Etotal,Gtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (fabs(Gtotal-Mtotal) > 0.01*MAX(Gtotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Mtotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Mtotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } else { + if (fabs(Etotal-Gtotal) > 0.01*MAX(Etotal,Gtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } + } + } else { // if (E < T/3 || G < T/3) { + int64_t S = MIN(E, G); + int64_t NS = MAX(E, G); + if (3*(S+D) < T) { // Subcase a: + double Stotal; + double NStotal; + if (G < E) { + casename = "Case 3a (G scarce)"; + Stotal = Gtotal; + NStotal = Etotal; + } else { // if (G >= E) { + casename = "Case 3a (E scarce)"; + NStotal = Gtotal; + Stotal = Etotal; + } + // Stotal < T/3 + if (3*Stotal > T) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: 3*Stotal %lf > T " + I64_FORMAT". G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT + " D="I64_FORMAT" T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Stotal*3, I64_PRINTF_ARG(T), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (NS >= M) { + if (fabs(NStotal-Mtotal) > 0.01*MAX(NStotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: NStotal %lf != Mtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, NStotal, Mtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } else { + // if NS < M, NStotal > T/3 because only one of G or E is scarce + if (3*NStotal < T) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: 3*NStotal %lf < T " + I64_FORMAT". G="I64_FORMAT" M="I64_FORMAT + " E="I64_FORMAT" D="I64_FORMAT" T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, NStotal*3, I64_PRINTF_ARG(T), + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } + } else { // Subcase b: S+D >= T/3 + casename = "Case 3b"; + if (fabs(Etotal-Mtotal) > 0.01*MAX(Etotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Mtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Mtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (fabs(Etotal-Gtotal) > 0.01*MAX(Etotal,Gtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Etotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Etotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + if (fabs(Gtotal-Mtotal) > 0.01*MAX(Gtotal,Mtotal)) { + log_warn(LD_DIR, + "Bw Weight Failure for %s: Mtotal %lf != Gtotal %lf. " + "G="I64_FORMAT" M="I64_FORMAT" E="I64_FORMAT" D="I64_FORMAT + " T="I64_FORMAT". " + "Wgg=%lf Wgd=%lf Wmg=%lf Wme=%lf Wmd=%lf Wee=%lf Wed=%lf", + casename, Mtotal, Gtotal, + I64_PRINTF_ARG(G), I64_PRINTF_ARG(M), I64_PRINTF_ARG(E), + I64_PRINTF_ARG(D), I64_PRINTF_ARG(T), + Wgg, Wgd, Wmg, Wme, Wmd, Wee, Wed); + valid = 0; + } + } + } + + if (valid) + log_notice(LD_DIR, "Bandwidth-weight %s is verified and valid.", + casename); + + return valid; +} + /** Parse a v3 networkstatus vote, opinion, or consensus (depending on * ns_type), from <b>s</b>, and return the result. Return NULL on failure. */ networkstatus_t * @@ -2200,19 +2778,21 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, smartlist_t *rs_tokens = NULL, *footer_tokens = NULL; networkstatus_voter_info_t *voter = NULL; networkstatus_t *ns = NULL; - char ns_digest[DIGEST_LEN]; - const char *cert, *end_of_header, *end_of_footer; + digests_t ns_digests; + const char *cert, *end_of_header, *end_of_footer, *s_dup = s; directory_token_t *tok; int ok; struct in_addr in; int i, inorder, n_signatures = 0; memarea_t *area = NULL, *rs_area = NULL; + consensus_flavor_t flav = FLAV_NS; + tor_assert(s); if (eos_out) *eos_out = NULL; - if (router_get_networkstatus_v3_hash(s, ns_digest)) { + if (router_get_networkstatus_v3_hashes(s, &ns_digests)) { log_warn(LD_DIR, "Unable to compute digest of network-status"); goto err; } @@ -2228,7 +2808,23 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, } ns = tor_malloc_zero(sizeof(networkstatus_t)); - memcpy(ns->networkstatus_digest, ns_digest, DIGEST_LEN); + memcpy(&ns->digests, &ns_digests, sizeof(ns_digests)); + + tok = find_by_keyword(tokens, K_NETWORK_STATUS_VERSION); + tor_assert(tok); + if (tok->n_args > 1) { + int flavor = networkstatus_parse_flavor_name(tok->args[1]); + if (flavor < 0) { + log_warn(LD_DIR, "Can't parse document with unknown flavor %s", + escaped(tok->args[2])); + goto err; + } + ns->flavor = flav = flavor; + } + if (flav != FLAV_NS && ns_type != NS_TYPE_CONSENSUS) { + log_warn(LD_DIR, "Flavor found on non-consensus networkstatus."); + goto err; + } if (ns_type != NS_TYPE_CONSENSUS) { const char *end_of_cert = NULL; @@ -2382,8 +2978,9 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, if (voter) smartlist_add(ns->voters, voter); voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); + voter->sigs = smartlist_create(); if (ns->type != NS_TYPE_CONSENSUS) - memcpy(voter->vote_digest, ns_digest, DIGEST_LEN); + memcpy(voter->vote_digest, ns_digests.d[DIGEST_SHA1], DIGEST_LEN); voter->nickname = tor_strdup(tok->args[0]); if (strlen(tok->args[1]) != HEX_DIGEST_LEN || @@ -2475,7 +3072,7 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, if (ns->type != NS_TYPE_CONSENSUS) { vote_routerstatus_t *rs = tor_malloc_zero(sizeof(vote_routerstatus_t)); if (routerstatus_parse_entry_from_string(rs_area, &s, rs_tokens, ns, - rs, 0)) + rs, 0, 0)) smartlist_add(ns->routerstatus_list, rs); else { tor_free(rs->version); @@ -2485,7 +3082,8 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, routerstatus_t *rs; if ((rs = routerstatus_parse_entry_from_string(rs_area, &s, rs_tokens, NULL, NULL, - ns->consensus_method))) + ns->consensus_method, + flav))) smartlist_add(ns->routerstatus_list, rs); } } @@ -2518,14 +3116,73 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, goto err; } - SMARTLIST_FOREACH(footer_tokens, directory_token_t *, _tok, { + int found_sig = 0; + SMARTLIST_FOREACH_BEGIN(footer_tokens, directory_token_t *, _tok) { + tok = _tok; + if (tok->tp == K_DIRECTORY_SIGNATURE) + found_sig = 1; + else if (found_sig) { + log_warn(LD_DIR, "Extraneous token after first directory-signature"); + goto err; + } + } SMARTLIST_FOREACH_END(_tok); + } + + if ((tok = find_opt_by_keyword(footer_tokens, K_DIRECTORY_FOOTER))) { + if (tok != smartlist_get(footer_tokens, 0)) { + log_warn(LD_DIR, "Misplaced directory-footer token"); + goto err; + } + } + + tok = find_opt_by_keyword(footer_tokens, K_BW_WEIGHTS); + if (tok) { + ns->weight_params = smartlist_create(); + for (i = 0; i < tok->n_args; ++i) { + int ok=0; + char *eq = strchr(tok->args[i], '='); + if (!eq) { + log_warn(LD_DIR, "Bad element '%s' in weight params", + escaped(tok->args[i])); + goto err; + } + tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL); + if (!ok) { + log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i])); + goto err; + } + smartlist_add(ns->weight_params, tor_strdup(tok->args[i])); + } + } + + SMARTLIST_FOREACH_BEGIN(footer_tokens, directory_token_t *, _tok) { char declared_identity[DIGEST_LEN]; networkstatus_voter_info_t *v; + document_signature_t *sig; + const char *id_hexdigest = NULL; + const char *sk_hexdigest = NULL; + digest_algorithm_t alg = DIGEST_SHA1; tok = _tok; if (tok->tp != K_DIRECTORY_SIGNATURE) continue; tor_assert(tok->n_args >= 2); + if (tok->n_args == 2) { + id_hexdigest = tok->args[0]; + sk_hexdigest = tok->args[1]; + } else { + const char *algname = tok->args[0]; + int a; + id_hexdigest = tok->args[1]; + sk_hexdigest = tok->args[2]; + a = crypto_digest_algorithm_parse_name(algname); + if (a<0) { + log_warn(LD_DIR, "Unknown digest algorithm %s; skipping", + escaped(algname)); + continue; + } + alg = a; + } if (!tok->object_type || strcmp(tok->object_type, "SIGNATURE") || @@ -2534,11 +3191,11 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, goto err; } - if (strlen(tok->args[0]) != HEX_DIGEST_LEN || + if (strlen(id_hexdigest) != HEX_DIGEST_LEN || base16_decode(declared_identity, sizeof(declared_identity), - tok->args[0], HEX_DIGEST_LEN) < 0) { + id_hexdigest, HEX_DIGEST_LEN) < 0) { log_warn(LD_DIR, "Error decoding declared identity %s in " - "network-status vote.", escaped(tok->args[0])); + "network-status vote.", escaped(id_hexdigest)); goto err; } if (!(v = networkstatus_get_voter_by_id(ns, declared_identity))) { @@ -2546,11 +3203,15 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, "any declared directory source."); goto err; } - if (strlen(tok->args[1]) != HEX_DIGEST_LEN || - base16_decode(v->signing_key_digest, sizeof(v->signing_key_digest), - tok->args[1], HEX_DIGEST_LEN) < 0) { - log_warn(LD_DIR, "Error decoding declared digest %s in " - "network-status vote.", escaped(tok->args[1])); + sig = tor_malloc_zero(sizeof(document_signature_t)); + memcpy(sig->identity_digest, v->identity_digest, DIGEST_LEN); + sig->alg = alg; + if (strlen(sk_hexdigest) != HEX_DIGEST_LEN || + base16_decode(sig->signing_key_digest, sizeof(sig->signing_key_digest), + sk_hexdigest, HEX_DIGEST_LEN) < 0) { + log_warn(LD_DIR, "Error decoding declared signing key digest %s in " + "network-status vote.", escaped(sk_hexdigest)); + tor_free(sig); goto err; } @@ -2559,35 +3220,49 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, DIGEST_LEN)) { log_warn(LD_DIR, "Digest mismatch between declared and actual on " "network-status vote."); + tor_free(sig); goto err; } } + if (voter_get_sig_by_algorithm(v, sig->alg)) { + /* We already parsed a vote with this algorithm from this voter. Use the + first one. */ + log_fn(LOG_PROTOCOL_WARN, LD_DIR, "We received a networkstatus " + "that contains two votes from the same voter with the same " + "algorithm. Ignoring the second vote."); + tor_free(sig); + continue; + } + if (ns->type != NS_TYPE_CONSENSUS) { - if (check_signature_token(ns_digest, tok, ns->cert->signing_key, 0, - "network-status vote")) + if (check_signature_token(ns_digests.d[DIGEST_SHA1], DIGEST_LEN, + tok, ns->cert->signing_key, 0, + "network-status vote")) { + tor_free(sig); goto err; - v->good_signature = 1; + } + sig->good_signature = 1; } else { - if (tok->object_size >= INT_MAX || tok->object_size >= SIZE_T_CEILING) + if (tok->object_size >= INT_MAX || tok->object_size >= SIZE_T_CEILING) { + tor_free(sig); goto err; - /* We already parsed a vote from this voter. Use the first one. */ - if (v->signature) { - log_fn(LOG_PROTOCOL_WARN, LD_DIR, "We received a networkstatus " - "that contains two votes from the same voter. Ignoring " - "the second vote."); - continue; } - - v->signature = tor_memdup(tok->object_body, tok->object_size); - v->signature_len = (int) tok->object_size; + sig->signature = tor_memdup(tok->object_body, tok->object_size); + sig->signature_len = (int) tok->object_size; } + smartlist_add(v->sigs, sig); + ++n_signatures; - }); + } SMARTLIST_FOREACH_END(_tok); if (! n_signatures) { log_warn(LD_DIR, "No signatures on networkstatus vote."); goto err; + } else if (ns->type == NS_TYPE_VOTE && n_signatures != 1) { + log_warn(LD_DIR, "Received more than one signature on a " + "network-status vote."); + goto err; } if (eos_out) @@ -2595,27 +3270,31 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, goto done; err: - if (ns) - networkstatus_vote_free(ns); + dump_desc(s_dup, "v3 networkstatus"); + networkstatus_vote_free(ns); ns = NULL; done: if (tokens) { - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); } if (voter) { + if (voter->sigs) { + SMARTLIST_FOREACH(voter->sigs, document_signature_t *, sig, + document_signature_free(sig)); + smartlist_free(voter->sigs); + } tor_free(voter->nickname); tor_free(voter->address); tor_free(voter->contact); - tor_free(voter->signature); tor_free(voter); } if (rs_tokens) { - SMARTLIST_FOREACH(rs_tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(rs_tokens, directory_token_t *, t, token_clear(t)); smartlist_free(rs_tokens); } if (footer_tokens) { - SMARTLIST_FOREACH(footer_tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(footer_tokens, directory_token_t *, t, token_clear(t)); smartlist_free(footer_tokens); } if (area) { @@ -2628,6 +3307,35 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, return ns; } +/** Return the digests_t that holds the digests of the + * <b>flavor_name</b>-flavored networkstatus according to the detached + * signatures document <b>sigs</b>, allocating a new digests_t as neeeded. */ +static digests_t * +detached_get_digests(ns_detached_signatures_t *sigs, const char *flavor_name) +{ + digests_t *d = strmap_get(sigs->digests, flavor_name); + if (!d) { + d = tor_malloc_zero(sizeof(digests_t)); + strmap_set(sigs->digests, flavor_name, d); + } + return d; +} + +/** Return the list of signatures of the <b>flavor_name</b>-flavored + * networkstatus according to the detached signatures document <b>sigs</b>, + * allocating a new digests_t as neeeded. */ +static smartlist_t * +detached_get_signatures(ns_detached_signatures_t *sigs, + const char *flavor_name) +{ + smartlist_t *sl = strmap_get(sigs->signatures, flavor_name); + if (!sl) { + sl = smartlist_create(); + strmap_set(sigs->signatures, flavor_name, sl); + } + return sl; +} + /** Parse a detached v3 networkstatus signature document between <b>s</b> and * <b>eos</b> and return the result. Return -1 on failure. */ ns_detached_signatures_t * @@ -2637,10 +3345,13 @@ networkstatus_parse_detached_signatures(const char *s, const char *eos) * networkstatus_parse_vote_from_string(). */ directory_token_t *tok; memarea_t *area = NULL; + digests_t *digests; smartlist_t *tokens = smartlist_create(); ns_detached_signatures_t *sigs = tor_malloc_zero(sizeof(ns_detached_signatures_t)); + sigs->digests = strmap_new(); + sigs->signatures = strmap_new(); if (!eos) eos = s + strlen(s); @@ -2652,18 +3363,57 @@ networkstatus_parse_detached_signatures(const char *s, const char *eos) goto err; } - tok = find_by_keyword(tokens, K_CONSENSUS_DIGEST); - if (strlen(tok->args[0]) != HEX_DIGEST_LEN) { - log_warn(LD_DIR, "Wrong length on consensus-digest in detached " - "networkstatus signatures"); - goto err; - } - if (base16_decode(sigs->networkstatus_digest, DIGEST_LEN, - tok->args[0], strlen(tok->args[0])) < 0) { - log_warn(LD_DIR, "Bad encoding on on consensus-digest in detached " - "networkstatus signatures"); - goto err; - } + /* Grab all the digest-like tokens. */ + SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, _tok) { + const char *algname; + digest_algorithm_t alg; + const char *flavor; + const char *hexdigest; + size_t expected_length; + + tok = _tok; + + if (tok->tp == K_CONSENSUS_DIGEST) { + algname = "sha1"; + alg = DIGEST_SHA1; + flavor = "ns"; + hexdigest = tok->args[0]; + } else if (tok->tp == K_ADDITIONAL_DIGEST) { + int a = crypto_digest_algorithm_parse_name(tok->args[1]); + if (a<0) { + log_warn(LD_DIR, "Unrecognized algorithm name %s", tok->args[0]); + continue; + } + alg = (digest_algorithm_t) a; + flavor = tok->args[0]; + algname = tok->args[1]; + hexdigest = tok->args[2]; + } else { + continue; + } + + expected_length = + (alg == DIGEST_SHA1) ? HEX_DIGEST_LEN : HEX_DIGEST256_LEN; + + if (strlen(hexdigest) != expected_length) { + log_warn(LD_DIR, "Wrong length on consensus-digest in detached " + "networkstatus signatures"); + goto err; + } + digests = detached_get_digests(sigs, flavor); + tor_assert(digests); + if (!tor_mem_is_zero(digests->d[alg], DIGEST256_LEN)) { + log_warn(LD_DIR, "Multiple digests for %s with %s on detached " + "signatures document", flavor, algname); + continue; + } + if (base16_decode(digests->d[alg], DIGEST256_LEN, + hexdigest, strlen(hexdigest)) < 0) { + log_warn(LD_DIR, "Bad encoding on consensus-digest in detached " + "networkstatus signatures"); + goto err; + } + } SMARTLIST_FOREACH_END(_tok); tok = find_by_keyword(tokens, K_VALID_AFTER); if (parse_iso_time(tok->args[0], &sigs->valid_after)) { @@ -2683,57 +3433,102 @@ networkstatus_parse_detached_signatures(const char *s, const char *eos) goto err; } - sigs->signatures = smartlist_create(); - SMARTLIST_FOREACH(tokens, directory_token_t *, _tok, - { - char id_digest[DIGEST_LEN]; - char sk_digest[DIGEST_LEN]; - networkstatus_voter_info_t *voter; + SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, _tok) { + const char *id_hexdigest; + const char *sk_hexdigest; + const char *algname; + const char *flavor; + digest_algorithm_t alg; + + char id_digest[DIGEST_LEN]; + char sk_digest[DIGEST_LEN]; + smartlist_t *siglist; + document_signature_t *sig; + int is_duplicate; - tok = _tok; - if (tok->tp != K_DIRECTORY_SIGNATURE) - continue; + tok = _tok; + if (tok->tp == K_DIRECTORY_SIGNATURE) { tor_assert(tok->n_args >= 2); + flavor = "ns"; + algname = "sha1"; + id_hexdigest = tok->args[0]; + sk_hexdigest = tok->args[1]; + } else if (tok->tp == K_ADDITIONAL_SIGNATURE) { + tor_assert(tok->n_args >= 4); + flavor = tok->args[0]; + algname = tok->args[1]; + id_hexdigest = tok->args[2]; + sk_hexdigest = tok->args[3]; + } else { + continue; + } - if (!tok->object_type || - strcmp(tok->object_type, "SIGNATURE") || - tok->object_size < 128 || tok->object_size > 512) { - log_warn(LD_DIR, "Bad object type or length on directory-signature"); - goto err; + { + int a = crypto_digest_algorithm_parse_name(algname); + if (a<0) { + log_warn(LD_DIR, "Unrecognized algorithm name %s", algname); + continue; } + alg = (digest_algorithm_t) a; + } - if (strlen(tok->args[0]) != HEX_DIGEST_LEN || - base16_decode(id_digest, sizeof(id_digest), - tok->args[0], HEX_DIGEST_LEN) < 0) { - log_warn(LD_DIR, "Error decoding declared identity %s in " - "network-status vote.", escaped(tok->args[0])); - goto err; - } - if (strlen(tok->args[1]) != HEX_DIGEST_LEN || - base16_decode(sk_digest, sizeof(sk_digest), - tok->args[1], HEX_DIGEST_LEN) < 0) { - log_warn(LD_DIR, "Error decoding declared digest %s in " - "network-status vote.", escaped(tok->args[1])); - goto err; - } + if (!tok->object_type || + strcmp(tok->object_type, "SIGNATURE") || + tok->object_size < 128 || tok->object_size > 512) { + log_warn(LD_DIR, "Bad object type or length on directory-signature"); + goto err; + } - voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); - memcpy(voter->identity_digest, id_digest, DIGEST_LEN); - memcpy(voter->signing_key_digest, sk_digest, DIGEST_LEN); - if (tok->object_size >= INT_MAX || tok->object_size >= SIZE_T_CEILING) - goto err; - voter->signature = tor_memdup(tok->object_body, tok->object_size); - voter->signature_len = (int) tok->object_size; + if (strlen(id_hexdigest) != HEX_DIGEST_LEN || + base16_decode(id_digest, sizeof(id_digest), + id_hexdigest, HEX_DIGEST_LEN) < 0) { + log_warn(LD_DIR, "Error decoding declared identity %s in " + "network-status vote.", escaped(id_hexdigest)); + goto err; + } + if (strlen(sk_hexdigest) != HEX_DIGEST_LEN || + base16_decode(sk_digest, sizeof(sk_digest), + sk_hexdigest, HEX_DIGEST_LEN) < 0) { + log_warn(LD_DIR, "Error decoding declared signing key digest %s in " + "network-status vote.", escaped(sk_hexdigest)); + goto err; + } - smartlist_add(sigs->signatures, voter); + siglist = detached_get_signatures(sigs, flavor); + is_duplicate = 0; + SMARTLIST_FOREACH(siglist, document_signature_t *, s, { + if (s->alg == alg && + !memcmp(id_digest, s->identity_digest, DIGEST_LEN) && + !memcmp(sk_digest, s->signing_key_digest, DIGEST_LEN)) { + is_duplicate = 1; + } }); + if (is_duplicate) { + log_warn(LD_DIR, "Two signatures with identical keys and algorithm " + "found."); + continue; + } + + sig = tor_malloc_zero(sizeof(document_signature_t)); + sig->alg = alg; + memcpy(sig->identity_digest, id_digest, DIGEST_LEN); + memcpy(sig->signing_key_digest, sk_digest, DIGEST_LEN); + if (tok->object_size >= INT_MAX || tok->object_size >= SIZE_T_CEILING) { + tor_free(sig); + goto err; + } + sig->signature = tor_memdup(tok->object_body, tok->object_size); + sig->signature_len = (int) tok->object_size; + + smartlist_add(siglist, sig); + } SMARTLIST_FOREACH_END(_tok); goto done; err: ns_detached_signatures_free(sigs); sigs = NULL; done: - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); if (area) { DUMP_AREA(area, "detached signatures"); @@ -2789,7 +3584,7 @@ router_parse_addr_policy_item_from_string(const char *s, int assume_action) err: r = NULL; done: - token_free(tok); + token_clear(tok); if (area) { DUMP_AREA(area, "policy item"); memarea_drop_all(area); @@ -2912,9 +3707,8 @@ assert_addr_policy_ok(smartlist_t *lst) /** Free all resources allocated for <b>tok</b> */ static void -token_free(directory_token_t *tok) +token_clear(directory_token_t *tok) { - tor_assert(tok); if (tok->key) crypto_free_pk_env(tok->key); } @@ -2926,7 +3720,7 @@ token_free(directory_token_t *tok) #define RET_ERR(msg) \ STMT_BEGIN \ - if (tok) token_free(tok); \ + if (tok) token_clear(tok); \ tok = ALLOC_ZERO(sizeof(directory_token_t)); \ tok->tp = _ERR; \ tok->error = STRDUP(msg); \ @@ -3222,7 +4016,7 @@ tokenize_string(memarea_t *area, tok = get_next_token(area, s, end, table); if (tok->tp == _ERR) { log_warn(LD_DIR, "parse error: %s", tok->error); - token_free(tok); + token_clear(tok); return -1; } ++counts[tok->tp]; @@ -3336,17 +4130,11 @@ find_all_exitpolicy(smartlist_t *s) return out; } -/** Compute the SHA-1 digest of the substring of <b>s</b> taken from the first - * occurrence of <b>start_str</b> through the first instance of c after the - * first subsequent occurrence of <b>end_str</b>; store the 20-byte result in - * <b>digest</b>; return 0 on success. - * - * If no such substring exists, return -1. - */ static int -router_get_hash_impl(const char *s, size_t s_len, char *digest, +router_get_hash_impl_helper(const char *s, size_t s_len, const char *start_str, - const char *end_str, char end_c) + const char *end_str, char end_c, + const char **start_out, const char **end_out) { const char *start, *end; start = tor_memstr(s, s_len, start_str); @@ -3373,14 +4161,215 @@ router_get_hash_impl(const char *s, size_t s_len, char *digest, } ++end; - if (crypto_digest(digest, start, end-start)) { - log_warn(LD_BUG,"couldn't compute digest"); + *start_out = start; + *end_out = end; + return 0; +} + +/** Compute the digest of the substring of <b>s</b> taken from the first + * occurrence of <b>start_str</b> through the first instance of c after the + * first subsequent occurrence of <b>end_str</b>; store the 20-byte result in + * <b>digest</b>; return 0 on success. + * + * If no such substring exists, return -1. + */ +static int +router_get_hash_impl(const char *s, size_t s_len, char *digest, + const char *start_str, + const char *end_str, char end_c, + digest_algorithm_t alg) +{ + const char *start=NULL, *end=NULL; + if (router_get_hash_impl_helper(s,s_len,start_str,end_str,end_c, + &start,&end)<0) return -1; + + if (alg == DIGEST_SHA1) { + if (crypto_digest(digest, start, end-start)) { + log_warn(LD_BUG,"couldn't compute digest"); + return -1; + } + } else { + if (crypto_digest256(digest, start, end-start, alg)) { + log_warn(LD_BUG,"couldn't compute digest"); + return -1; + } } return 0; } +/** As router_get_hash_impl, but compute all hashes. */ +static int +router_get_hashes_impl(const char *s, size_t s_len, digests_t *digests, + const char *start_str, + const char *end_str, char end_c) +{ + const char *start=NULL, *end=NULL; + if (router_get_hash_impl_helper(s,s_len,start_str,end_str,end_c, + &start,&end)<0) + return -1; + + if (crypto_digest_all(digests, start, end-start)) { + log_warn(LD_BUG,"couldn't compute digests"); + return -1; + } + + return 0; +} + +/** Assuming that s starts with a microdesc, return the start of the + * *NEXT* one. Return NULL on "not found." */ +static const char * +find_start_of_next_microdesc(const char *s, const char *eos) +{ + int started_with_annotations; + s = eat_whitespace_eos(s, eos); + if (!s) + return NULL; + +#define CHECK_LENGTH() STMT_BEGIN \ + if (s+32 > eos) \ + return NULL; \ + STMT_END + +#define NEXT_LINE() STMT_BEGIN \ + s = memchr(s, '\n', eos-s); \ + if (!s || s+1 >= eos) \ + return NULL; \ + s++; \ + STMT_END + + CHECK_LENGTH(); + + started_with_annotations = (*s == '@'); + + if (started_with_annotations) { + /* Start by advancing to the first non-annotation line. */ + while (*s == '@') + NEXT_LINE(); + } + CHECK_LENGTH(); + + /* Now we should be pointed at an onion-key line. If we are, then skip + * it. */ + if (!strcmpstart(s, "onion-key")) + NEXT_LINE(); + + /* Okay, now we're pointed at the first line of the microdescriptor which is + not an annotation or onion-key. The next line that _is_ an annotation or + onion-key is the start of the next microdescriptor. */ + while (s+32 < eos) { + if (*s == '@' || !strcmpstart(s, "onion-key")) + return s; + NEXT_LINE(); + } + return NULL; + +#undef CHECK_LENGTH +#undef NEXT_LINE +} + +/** Parse as many microdescriptors as are found from the string starting at + * <b>s</b> and ending at <b>eos</b>. If allow_annotations is set, read any + * annotations we recognize and ignore ones we don't. If <b>copy_body</b> is + * true, then strdup the bodies of the microdescriptors. Return all newly + * parsed microdescriptors in a newly allocated smartlist_t. */ +smartlist_t * +microdescs_parse_from_string(const char *s, const char *eos, + int allow_annotations, int copy_body) +{ + smartlist_t *tokens; + smartlist_t *result; + microdesc_t *md = NULL; + memarea_t *area; + const char *start = s; + const char *start_of_next_microdesc; + int flags = allow_annotations ? TS_ANNOTATIONS_OK : 0; + + directory_token_t *tok; + + if (!eos) + eos = s + strlen(s); + + s = eat_whitespace_eos(s, eos); + area = memarea_new(); + result = smartlist_create(); + tokens = smartlist_create(); + + while (s < eos) { + start_of_next_microdesc = find_start_of_next_microdesc(s, eos); + if (!start_of_next_microdesc) + start_of_next_microdesc = eos; + + if (tokenize_string(area, s, start_of_next_microdesc, tokens, + microdesc_token_table, flags)) { + log_warn(LD_DIR, "Unparseable microdescriptor"); + goto next; + } + + md = tor_malloc_zero(sizeof(microdesc_t)); + { + const char *cp = tor_memstr(s, start_of_next_microdesc-s, + "onion-key"); + tor_assert(cp); + + md->bodylen = start_of_next_microdesc - cp; + if (copy_body) + md->body = tor_strndup(cp, md->bodylen); + else + md->body = (char*)cp; + md->off = cp - start; + } + + if ((tok = find_opt_by_keyword(tokens, A_LAST_LISTED))) { + if (parse_iso_time(tok->args[0], &md->last_listed)) { + log_warn(LD_DIR, "Bad last-listed time in microdescriptor"); + goto next; + } + } + + tok = find_by_keyword(tokens, K_ONION_KEY); + md->onion_pkey = tok->key; + tok->key = NULL; + + if ((tok = find_opt_by_keyword(tokens, K_FAMILY))) { + int i; + md->family = smartlist_create(); + for (i=0;i<tok->n_args;++i) { + if (!is_legal_nickname_or_hexdigest(tok->args[i])) { + log_warn(LD_DIR, "Illegal nickname %s in family line", + escaped(tok->args[i])); + goto next; + } + smartlist_add(md->family, tor_strdup(tok->args[i])); + } + } + + if ((tok = find_opt_by_keyword(tokens, K_P))) { + md->exitsummary = tor_strdup(tok->args[0]); + } + + crypto_digest256(md->digest, md->body, md->bodylen, DIGEST_SHA256); + + smartlist_add(result, md); + + md = NULL; + next: + microdesc_free(md); + md = NULL; + + memarea_clear(area); + smartlist_clear(tokens); + s = start_of_next_microdesc; + } + + memarea_drop_all(area); + smartlist_free(tokens); + + return result; +} + /** Parse the Tor version of the platform string <b>platform</b>, * and compare it to the version in <b>cutoff</b>. Return 1 if * the router is at least as new as the cutoff, else return 0. @@ -3405,7 +4394,7 @@ tor_version_as_new_as(const char *platform, const char *cutoff) if (!*start) return 0; s = (char *)find_whitespace(start); /* also finds '\0', which is fine */ s2 = (char*)eat_whitespace(s); - if (!strcmpstart(s2, "(r")) + if (!strcmpstart(s2, "(r") || !strcmpstart(s2, "(git-")) s = (char*)find_whitespace(s2); if ((size_t)(s-start+1) >= sizeof(tmp)) /* too big, no */ @@ -3501,6 +4490,23 @@ tor_version_parse(const char *s, tor_version_t *out) if (!strcmpstart(cp, "(r")) { cp += 2; out->svn_revision = (int) strtol(cp,&eos,10); + } else if (!strcmpstart(cp, "(git-")) { + char *close_paren = strchr(cp, ')'); + int hexlen; + char digest[DIGEST_LEN]; + if (! close_paren) + return -1; + cp += 5; + if (close_paren-cp > HEX_DIGEST_LEN) + return -1; + hexlen = (int)(close_paren-cp); + memset(digest, 0, sizeof(digest)); + if ( hexlen == 0 || (hexlen % 2) == 1) + return -1; + if (base16_decode(digest, hexlen/2, cp, hexlen)) + return -1; + memcpy(out->git_tag, digest, hexlen/2); + out->git_tag_len = hexlen/2; } return 0; @@ -3526,8 +4532,14 @@ tor_version_compare(tor_version_t *a, tor_version_t *b) return i; else if ((i = strcmp(a->status_tag, b->status_tag))) return i; + else if ((i = a->svn_revision - b->svn_revision)) + return i; + else if ((i = a->git_tag_len - b->git_tag_len)) + return i; + else if (a->git_tag_len) + return memcmp(a->git_tag, b->git_tag, a->git_tag_len); else - return a->svn_revision - b->svn_revision; + return 0; } /** Return true iff versions <b>a</b> and <b>b</b> belong to the same series. @@ -3616,7 +4628,7 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, /* Compute descriptor hash for later validation. */ if (router_get_hash_impl(desc, strlen(desc), desc_hash, "rendezvous-service-descriptor ", - "\nsignature", '\n') < 0) { + "\nsignature", '\n', DIGEST_SHA1) < 0) { log_warn(LD_REND, "Couldn't compute descriptor hash."); goto err; } @@ -3627,12 +4639,12 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, else eos = eos + 1; /* Check length. */ - if (strlen(desc) > REND_DESC_MAX_SIZE) { + if (eos-desc > REND_DESC_MAX_SIZE) { /* XXX023 If we are parsing this descriptor as a server, this * should be a protocol warning. */ - log_warn(LD_REND, "Descriptor length is %i which exceeds " - "maximum rendezvous descriptor size of %i bytes.", - (int)strlen(desc), REND_DESC_MAX_SIZE); + log_warn(LD_REND, "Descriptor length is %d which exceeds " + "maximum rendezvous descriptor size of %d bytes.", + (int)(eos-desc), REND_DESC_MAX_SIZE); goto err; } /* Tokenize descriptor. */ @@ -3737,7 +4749,7 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, /* Parse and verify signature. */ tok = find_by_keyword(tokens, R_SIGNATURE); note_crypto_pk_op(VERIFY_RTR); - if (check_signature_token(desc_hash, tok, result->pk, 0, + if (check_signature_token(desc_hash, DIGEST_LEN, tok, result->pk, 0, "v2 rendezvous service descriptor") < 0) goto err; /* Verify that descriptor ID belongs to public key and secret ID part. */ @@ -3751,12 +4763,11 @@ rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, } goto done; err: - if (result) - rend_service_descriptor_free(result); + rend_service_descriptor_free(result); result = NULL; done: if (tokens) { - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); } if (area) @@ -3914,7 +4925,7 @@ rend_parse_introduction_points(rend_service_descriptor_t *parsed, eos = eos+1; tor_assert(eos <= intro_points_encoded+intro_points_encoded_size); /* Free tokens and clear token list. */ - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_clear(tokens); memarea_clear(area); /* Tokenize string. */ @@ -3987,7 +4998,7 @@ rend_parse_introduction_points(rend_service_descriptor_t *parsed, done: /* Free tokens and clear token list. */ - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); if (area) memarea_drop_all(area); @@ -4026,7 +5037,7 @@ rend_parse_client_keys(strmap_t *parsed_clients, const char *ckstr) else eos = eos + 1; /* Free tokens and clear token list. */ - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_clear(tokens); memarea_clear(area); /* Tokenize string. */ @@ -4098,7 +5109,7 @@ rend_parse_client_keys(strmap_t *parsed_clients, const char *ckstr) result = -1; done: /* Free tokens and clear token list. */ - SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_free(t)); + SMARTLIST_FOREACH(tokens, directory_token_t *, t, token_clear(t)); smartlist_free(tokens); if (area) memarea_drop_all(area); diff --git a/src/or/routerparse.h b/src/or/routerparse.h new file mode 100644 index 000000000..8b8cde25f --- /dev/null +++ b/src/or/routerparse.h @@ -0,0 +1,86 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file routerparse.h + * \brief Header file for routerparse.c. + **/ + +#ifndef _TOR_ROUTERPARSE_H +#define _TOR_ROUTERPARSE_H + +int router_get_router_hash(const char *s, size_t s_len, char *digest); +int router_get_dir_hash(const char *s, char *digest); +int router_get_runningrouters_hash(const char *s, char *digest); +int router_get_networkstatus_v2_hash(const char *s, char *digest); +int router_get_networkstatus_v3_hash(const char *s, char *digest, + digest_algorithm_t algorithm); +int router_get_networkstatus_v3_hashes(const char *s, digests_t *digests); +int router_get_extrainfo_hash(const char *s, char *digest); +#define DIROBJ_MAX_SIG_LEN 256 +int router_append_dirobj_signature(char *buf, size_t buf_len, + const char *digest, + size_t digest_len, + crypto_pk_env_t *private_key); +int router_parse_list_from_string(const char **s, const char *eos, + smartlist_t *dest, + saved_location_t saved_location, + int is_extrainfo, + int allow_annotations, + const char *prepend_annotations); +int router_parse_runningrouters(const char *str); +int router_parse_directory(const char *str); + +routerinfo_t *router_parse_entry_from_string(const char *s, const char *end, + int cache_copy, + int allow_annotations, + const char *prepend_annotations); +extrainfo_t *extrainfo_parse_entry_from_string(const char *s, const char *end, + int cache_copy, struct digest_ri_map_t *routermap); +addr_policy_t *router_parse_addr_policy_item_from_string(const char *s, + int assume_action); +version_status_t tor_version_is_obsolete(const char *myversion, + const char *versionlist); +int tor_version_parse(const char *s, tor_version_t *out); +int tor_version_as_new_as(const char *platform, const char *cutoff); +int tor_version_compare(tor_version_t *a, tor_version_t *b); +void sort_version_list(smartlist_t *lst, int remove_duplicates); +void assert_addr_policy_ok(smartlist_t *t); +void dump_distinct_digest_count(int severity); + +int compare_routerstatus_entries(const void **_a, const void **_b); +networkstatus_v2_t *networkstatus_v2_parse_from_string(const char *s); +int networkstatus_verify_bw_weights(networkstatus_t *ns); +networkstatus_t *networkstatus_parse_vote_from_string(const char *s, + const char **eos_out, + networkstatus_type_t ns_type); +ns_detached_signatures_t *networkstatus_parse_detached_signatures( + const char *s, const char *eos); + +smartlist_t *microdescs_parse_from_string(const char *s, const char *eos, + int allow_annotations, + int copy_body); + +authority_cert_t *authority_cert_parse_from_string(const char *s, + const char **end_of_string); +int rend_parse_v2_service_descriptor(rend_service_descriptor_t **parsed_out, + char *desc_id_out, + char **intro_points_encrypted_out, + size_t *intro_points_encrypted_size_out, + size_t *encoded_size_out, + const char **next_out, const char *desc); +int rend_decrypt_introduction_points(char **ipos_decrypted, + size_t *ipos_decrypted_size, + const char *descriptor_cookie, + const char *ipos_encrypted, + size_t ipos_encrypted_size); +int rend_parse_introduction_points(rend_service_descriptor_t *parsed, + const char *intro_points_encoded, + size_t intro_points_encoded_size); +int rend_parse_client_keys(strmap_t *parsed_clients, const char *str); + +#endif + diff --git a/src/or/test.c b/src/or/test.c deleted file mode 100644 index b08f202c2..000000000 --- a/src/or/test.c +++ /dev/null @@ -1,4918 +0,0 @@ -/* Copyright (c) 2001-2004, Roger Dingledine. - * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. - * Copyright (c) 2007-2011, The Tor Project, Inc. */ -/* See LICENSE for licensing information */ - -/* Ordinarily defined in tor_main.c; this bit is just here to provide one - * since we're not linking to tor_main.c */ -const char tor_svn_revision[] = ""; - -/** - * \file test.c - * \brief Unit tests for many pieces of the lower level Tor modules. - **/ - -#include "orconfig.h" - -#include <stdio.h> -#ifdef HAVE_FCNTL_H -#include <fcntl.h> -#endif - -#ifdef MS_WINDOWS -/* For mkdir() */ -#include <direct.h> -#else -#include <dirent.h> -#endif - -/* These macros pull in declarations for some functions and structures that - * are typically file-private. */ -#define BUFFERS_PRIVATE -#define CONFIG_PRIVATE -#define CONTROL_PRIVATE -#define CRYPTO_PRIVATE -#define DIRSERV_PRIVATE -#define DIRVOTE_PRIVATE -#define GEOIP_PRIVATE -#define MEMPOOL_PRIVATE -#define ROUTER_PRIVATE - -#include "or.h" -#include "test.h" -#include "torgzip.h" -#include "mempool.h" -#include "memarea.h" - -#ifdef USE_DMALLOC -#include <dmalloc.h> -#include <openssl/crypto.h> -#endif - -/** Set to true if any unit test has failed. Mostly, this is set by the macros - * in test.h */ -int have_failed = 0; - -/** Temporary directory (set up by setup_directory) under which we store all - * our files during testing. */ -static char temp_dir[256]; - -/** Select and create the temporary directory we'll use to run our unit tests. - * Store it in <b>temp_dir</b>. Exit immediately if we can't create it. - * idempotent. */ -static void -setup_directory(void) -{ - static int is_setup = 0; - int r; - if (is_setup) return; - -#ifdef MS_WINDOWS - // XXXX - tor_snprintf(temp_dir, sizeof(temp_dir), - "c:\\windows\\temp\\tor_test_%d", (int)getpid()); - r = mkdir(temp_dir); -#else - tor_snprintf(temp_dir, sizeof(temp_dir), "/tmp/tor_test_%d", (int) getpid()); - r = mkdir(temp_dir, 0700); -#endif - if (r) { - fprintf(stderr, "Can't create directory %s:", temp_dir); - perror(""); - exit(1); - } - is_setup = 1; -} - -/** Return a filename relative to our testing temporary directory */ -static const char * -get_fname(const char *name) -{ - static char buf[1024]; - setup_directory(); - tor_snprintf(buf,sizeof(buf),"%s/%s",temp_dir,name); - return buf; -} - -/** Remove all files stored under the temporary directory, and the directory - * itself. */ -static void -remove_directory(void) -{ - smartlist_t *elements = tor_listdir(temp_dir); - if (elements) { - SMARTLIST_FOREACH(elements, const char *, cp, - { - size_t len = strlen(cp)+strlen(temp_dir)+16; - char *tmp = tor_malloc(len); - tor_snprintf(tmp, len, "%s"PATH_SEPARATOR"%s", temp_dir, cp); - unlink(tmp); - tor_free(tmp); - }); - SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); - smartlist_free(elements); - } - rmdir(temp_dir); -} - -/** Define this if unit tests spend too much time generating public keys*/ -#undef CACHE_GENERATED_KEYS - -static crypto_pk_env_t *pregen_keys[5] = {NULL, NULL, NULL, NULL, NULL}; -#define N_PREGEN_KEYS ((int)(sizeof(pregen_keys)/sizeof(pregen_keys[0]))) - -/** Generate and return a new keypair for use in unit tests. If we're using - * the key cache optimization, we might reuse keys: we only guarantee that - * keys made with distinct values for <b>idx</b> are different. The value of - * <b>idx</b> must be at least 0, and less than N_PREGEN_KEYS. */ -static crypto_pk_env_t * -pk_generate(int idx) -{ -#ifdef CACHE_GENERATED_KEYS - tor_assert(idx < N_PREGEN_KEYS); - if (! pregen_keys[idx]) { - pregen_keys[idx] = crypto_new_pk_env(); - tor_assert(!crypto_pk_generate_key(pregen_keys[idx])); - } - return crypto_pk_dup_key(pregen_keys[idx]); -#else - crypto_pk_env_t *result; - (void) idx; - result = crypto_new_pk_env(); - tor_assert(!crypto_pk_generate_key(result)); - return result; -#endif -} - -/** Free all storage used for the cached key optimization. */ -static void -free_pregenerated_keys(void) -{ - unsigned idx; - for (idx = 0; idx < N_PREGEN_KEYS; ++idx) { - if (pregen_keys[idx]) { - crypto_free_pk_env(pregen_keys[idx]); - pregen_keys[idx] = NULL; - } - } -} - -/** Run unit tests for buffers.c */ -static void -test_buffers(void) -{ - char str[256]; - char str2[256]; - - buf_t *buf = NULL, *buf2 = NULL; - const char *cp; - - int j; - size_t r; - - /**** - * buf_new - ****/ - if (!(buf = buf_new())) - test_fail(); - - //test_eq(buf_capacity(buf), 4096); - test_eq(buf_datalen(buf), 0); - - /**** - * General pointer frobbing - */ - for (j=0;j<256;++j) { - str[j] = (char)j; - } - write_to_buf(str, 256, buf); - write_to_buf(str, 256, buf); - test_eq(buf_datalen(buf), 512); - fetch_from_buf(str2, 200, buf); - test_memeq(str, str2, 200); - test_eq(buf_datalen(buf), 312); - memset(str2, 0, sizeof(str2)); - - fetch_from_buf(str2, 256, buf); - test_memeq(str+200, str2, 56); - test_memeq(str, str2+56, 200); - test_eq(buf_datalen(buf), 56); - memset(str2, 0, sizeof(str2)); - /* Okay, now we should be 512 bytes into the 4096-byte buffer. If we add - * another 3584 bytes, we hit the end. */ - for (j=0;j<15;++j) { - write_to_buf(str, 256, buf); - } - assert_buf_ok(buf); - test_eq(buf_datalen(buf), 3896); - fetch_from_buf(str2, 56, buf); - test_eq(buf_datalen(buf), 3840); - test_memeq(str+200, str2, 56); - for (j=0;j<15;++j) { - memset(str2, 0, sizeof(str2)); - fetch_from_buf(str2, 256, buf); - test_memeq(str, str2, 256); - } - test_eq(buf_datalen(buf), 0); - buf_free(buf); - buf = NULL; - - /* Okay, now make sure growing can work. */ - buf = buf_new_with_capacity(16); - //test_eq(buf_capacity(buf), 16); - write_to_buf(str+1, 255, buf); - //test_eq(buf_capacity(buf), 256); - fetch_from_buf(str2, 254, buf); - test_memeq(str+1, str2, 254); - //test_eq(buf_capacity(buf), 256); - assert_buf_ok(buf); - write_to_buf(str, 32, buf); - //test_eq(buf_capacity(buf), 256); - assert_buf_ok(buf); - write_to_buf(str, 256, buf); - assert_buf_ok(buf); - //test_eq(buf_capacity(buf), 512); - test_eq(buf_datalen(buf), 33+256); - fetch_from_buf(str2, 33, buf); - test_eq(*str2, str[255]); - - test_memeq(str2+1, str, 32); - //test_eq(buf_capacity(buf), 512); - test_eq(buf_datalen(buf), 256); - fetch_from_buf(str2, 256, buf); - test_memeq(str, str2, 256); - - /* now try shrinking: case 1. */ - buf_free(buf); - buf = buf_new_with_capacity(33668); - for (j=0;j<67;++j) { - write_to_buf(str,255, buf); - } - //test_eq(buf_capacity(buf), 33668); - test_eq(buf_datalen(buf), 17085); - for (j=0; j < 40; ++j) { - fetch_from_buf(str2, 255,buf); - test_memeq(str2, str, 255); - } - - /* now try shrinking: case 2. */ - buf_free(buf); - buf = buf_new_with_capacity(33668); - for (j=0;j<67;++j) { - write_to_buf(str,255, buf); - } - for (j=0; j < 20; ++j) { - fetch_from_buf(str2, 255,buf); - test_memeq(str2, str, 255); - } - for (j=0;j<80;++j) { - write_to_buf(str,255, buf); - } - //test_eq(buf_capacity(buf),33668); - for (j=0; j < 120; ++j) { - fetch_from_buf(str2, 255,buf); - test_memeq(str2, str, 255); - } - - /* Move from buf to buf. */ - buf_free(buf); - buf = buf_new_with_capacity(4096); - buf2 = buf_new_with_capacity(4096); - for (j=0;j<100;++j) - write_to_buf(str, 255, buf); - test_eq(buf_datalen(buf), 25500); - for (j=0;j<100;++j) { - r = 10; - move_buf_to_buf(buf2, buf, &r); - test_eq(r, 0); - } - test_eq(buf_datalen(buf), 24500); - test_eq(buf_datalen(buf2), 1000); - for (j=0;j<3;++j) { - fetch_from_buf(str2, 255, buf2); - test_memeq(str2, str, 255); - } - r = 8192; /*big move*/ - move_buf_to_buf(buf2, buf, &r); - test_eq(r, 0); - r = 30000; /* incomplete move */ - move_buf_to_buf(buf2, buf, &r); - test_eq(r, 13692); - for (j=0;j<97;++j) { - fetch_from_buf(str2, 255, buf2); - test_memeq(str2, str, 255); - } - buf_free(buf); - buf_free(buf2); - buf = buf2 = NULL; - - buf = buf_new_with_capacity(5); - cp = "Testing. This is a moderately long Testing string."; - for (j = 0; cp[j]; j++) - write_to_buf(cp+j, 1, buf); - test_eq(0, buf_find_string_offset(buf, "Testing", 7)); - test_eq(1, buf_find_string_offset(buf, "esting", 6)); - test_eq(1, buf_find_string_offset(buf, "est", 3)); - test_eq(39, buf_find_string_offset(buf, "ing str", 7)); - test_eq(35, buf_find_string_offset(buf, "Testing str", 11)); - test_eq(32, buf_find_string_offset(buf, "ng ", 3)); - test_eq(43, buf_find_string_offset(buf, "string.", 7)); - test_eq(-1, buf_find_string_offset(buf, "shrdlu", 6)); - test_eq(-1, buf_find_string_offset(buf, "Testing thing", 13)); - test_eq(-1, buf_find_string_offset(buf, "ngx", 3)); - buf_free(buf); - buf = NULL; - -#if 0 - { - int s; - int eof; - int i; - buf_t *buf2; - /**** - * read_to_buf - ****/ - s = open(get_fname("data"), O_WRONLY|O_CREAT|O_TRUNC, 0600); - write(s, str, 256); - close(s); - - s = open(get_fname("data"), O_RDONLY, 0); - eof = 0; - errno = 0; /* XXXX */ - i = read_to_buf(s, 10, buf, &eof); - printf("%s\n", strerror(errno)); - test_eq(i, 10); - test_eq(eof, 0); - //test_eq(buf_capacity(buf), 4096); - test_eq(buf_datalen(buf), 10); - - test_memeq(str, (char*)_buf_peek_raw_buffer(buf), 10); - - /* Test reading 0 bytes. */ - i = read_to_buf(s, 0, buf, &eof); - //test_eq(buf_capacity(buf), 512*1024); - test_eq(buf_datalen(buf), 10); - test_eq(eof, 0); - test_eq(i, 0); - - /* Now test when buffer is filled exactly. */ - buf2 = buf_new_with_capacity(6); - i = read_to_buf(s, 6, buf2, &eof); - //test_eq(buf_capacity(buf2), 6); - test_eq(buf_datalen(buf2), 6); - test_eq(eof, 0); - test_eq(i, 6); - test_memeq(str+10, (char*)_buf_peek_raw_buffer(buf2), 6); - buf_free(buf2); - buf2 = NULL; - - /* Now test when buffer is filled with more data to read. */ - buf2 = buf_new_with_capacity(32); - i = read_to_buf(s, 128, buf2, &eof); - //test_eq(buf_capacity(buf2), 128); - test_eq(buf_datalen(buf2), 32); - test_eq(eof, 0); - test_eq(i, 32); - buf_free(buf2); - buf2 = NULL; - - /* Now read to eof. */ - test_assert(buf_capacity(buf) > 256); - i = read_to_buf(s, 1024, buf, &eof); - test_eq(i, (256-32-10-6)); - test_eq(buf_capacity(buf), MAX_BUF_SIZE); - test_eq(buf_datalen(buf), 256-6-32); - test_memeq(str, (char*)_buf_peek_raw_buffer(buf), 10); /* XXX Check rest. */ - test_eq(eof, 0); - - i = read_to_buf(s, 1024, buf, &eof); - test_eq(i, 0); - test_eq(buf_capacity(buf), MAX_BUF_SIZE); - test_eq(buf_datalen(buf), 256-6-32); - test_eq(eof, 1); - } -#endif - - done: - if (buf) - buf_free(buf); - if (buf2) - buf_free(buf2); -} - -/** Run unit tests for Diffie-Hellman functionality. */ -static void -test_crypto_dh(void) -{ - crypto_dh_env_t *dh1 = crypto_dh_new(DH_TYPE_CIRCUIT); - crypto_dh_env_t *dh2 = crypto_dh_new(DH_TYPE_CIRCUIT); - char p1[DH_BYTES]; - char p2[DH_BYTES]; - char s1[DH_BYTES]; - char s2[DH_BYTES]; - ssize_t s1len, s2len; - - test_eq(crypto_dh_get_bytes(dh1), DH_BYTES); - test_eq(crypto_dh_get_bytes(dh2), DH_BYTES); - - memset(p1, 0, DH_BYTES); - memset(p2, 0, DH_BYTES); - test_memeq(p1, p2, DH_BYTES); - test_assert(! crypto_dh_get_public(dh1, p1, DH_BYTES)); - test_memneq(p1, p2, DH_BYTES); - test_assert(! crypto_dh_get_public(dh2, p2, DH_BYTES)); - test_memneq(p1, p2, DH_BYTES); - - memset(s1, 0, DH_BYTES); - memset(s2, 0xFF, DH_BYTES); - s1len = crypto_dh_compute_secret(dh1, p2, DH_BYTES, s1, 50); - s2len = crypto_dh_compute_secret(dh2, p1, DH_BYTES, s2, 50); - test_assert(s1len > 0); - test_eq(s1len, s2len); - test_memeq(s1, s2, s1len); - - { - /* XXXX Now fabricate some bad values and make sure they get caught, - * Check 0, 1, N-1, >= N, etc. - */ - } - - done: - crypto_dh_free(dh1); - crypto_dh_free(dh2); -} - -/** Run unit tests for our random number generation function and its wrappers. - */ -static void -test_crypto_rng(void) -{ - int i, j, allok; - char data1[100], data2[100]; - - /* Try out RNG. */ - test_assert(! crypto_seed_rng(0)); - crypto_rand(data1, 100); - crypto_rand(data2, 100); - test_memneq(data1,data2,100); - allok = 1; - for (i = 0; i < 100; ++i) { - uint64_t big; - char *host; - j = crypto_rand_int(100); - if (i < 0 || i >= 100) - allok = 0; - big = crypto_rand_uint64(U64_LITERAL(1)<<40); - if (big >= (U64_LITERAL(1)<<40)) - allok = 0; - big = crypto_rand_uint64(U64_LITERAL(5)); - if (big >= 5) - allok = 0; - host = crypto_random_hostname(3,8,"www.",".onion"); - if (strcmpstart(host,"www.") || - strcmpend(host,".onion") || - strlen(host) < 13 || - strlen(host) > 18) - allok = 0; - tor_free(host); - } - test_assert(allok); - done: - ; -} - -/** Run unit tests for our AES functionality */ -static void -test_crypto_aes(void) -{ - char *data1 = NULL, *data2 = NULL, *data3 = NULL; - crypto_cipher_env_t *env1 = NULL, *env2 = NULL; - int i, j; - - data1 = tor_malloc(1024); - data2 = tor_malloc(1024); - data3 = tor_malloc(1024); - - /* Now, test encryption and decryption with stream cipher. */ - data1[0]='\0'; - for (i = 1023; i>0; i -= 35) - strncat(data1, "Now is the time for all good onions", i); - - memset(data2, 0, 1024); - memset(data3, 0, 1024); - env1 = crypto_new_cipher_env(); - test_neq(env1, 0); - env2 = crypto_new_cipher_env(); - test_neq(env2, 0); - j = crypto_cipher_generate_key(env1); - crypto_cipher_set_key(env2, crypto_cipher_get_key(env1)); - crypto_cipher_encrypt_init_cipher(env1); - crypto_cipher_decrypt_init_cipher(env2); - - /* Try encrypting 512 chars. */ - crypto_cipher_encrypt(env1, data2, data1, 512); - crypto_cipher_decrypt(env2, data3, data2, 512); - test_memeq(data1, data3, 512); - test_memneq(data1, data2, 512); - - /* Now encrypt 1 at a time, and get 1 at a time. */ - for (j = 512; j < 560; ++j) { - crypto_cipher_encrypt(env1, data2+j, data1+j, 1); - } - for (j = 512; j < 560; ++j) { - crypto_cipher_decrypt(env2, data3+j, data2+j, 1); - } - test_memeq(data1, data3, 560); - /* Now encrypt 3 at a time, and get 5 at a time. */ - for (j = 560; j < 1024-5; j += 3) { - crypto_cipher_encrypt(env1, data2+j, data1+j, 3); - } - for (j = 560; j < 1024-5; j += 5) { - crypto_cipher_decrypt(env2, data3+j, data2+j, 5); - } - test_memeq(data1, data3, 1024-5); - /* Now make sure that when we encrypt with different chunk sizes, we get - the same results. */ - crypto_free_cipher_env(env2); - env2 = NULL; - - memset(data3, 0, 1024); - env2 = crypto_new_cipher_env(); - test_neq(env2, 0); - crypto_cipher_set_key(env2, crypto_cipher_get_key(env1)); - crypto_cipher_encrypt_init_cipher(env2); - for (j = 0; j < 1024-16; j += 17) { - crypto_cipher_encrypt(env2, data3+j, data1+j, 17); - } - for (j= 0; j < 1024-16; ++j) { - if (data2[j] != data3[j]) { - printf("%d: %d\t%d\n", j, (int) data2[j], (int) data3[j]); - } - } - test_memeq(data2, data3, 1024-16); - crypto_free_cipher_env(env1); - env1 = NULL; - crypto_free_cipher_env(env2); - env2 = NULL; - - /* NIST test vector for aes. */ - env1 = crypto_new_cipher_env(); /* IV starts at 0 */ - crypto_cipher_set_key(env1, "\x80\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00"); - crypto_cipher_encrypt_init_cipher(env1); - crypto_cipher_encrypt(env1, data1, - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00", 16); - test_memeq_hex(data1, "0EDD33D3C621E546455BD8BA1418BEC8"); - - /* Now test rollover. All these values are originally from a python - * script. */ - crypto_cipher_set_iv(env1, "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xff\xff\xff\xff\xff\xff\xff\xff"); - memset(data2, 0, 1024); - crypto_cipher_encrypt(env1, data1, data2, 32); - test_memeq_hex(data1, "335fe6da56f843199066c14a00a40231" - "cdd0b917dbc7186908a6bfb5ffd574d3"); - - crypto_cipher_set_iv(env1, "\x00\x00\x00\x00\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); - memset(data2, 0, 1024); - crypto_cipher_encrypt(env1, data1, data2, 32); - test_memeq_hex(data1, "e627c6423fa2d77832a02b2794094b73" - "3e63c721df790d2c6469cc1953a3ffac"); - - crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); - memset(data2, 0, 1024); - crypto_cipher_encrypt(env1, data1, data2, 32); - test_memeq_hex(data1, "2aed2bff0de54f9328efd070bf48f70a" - "0EDD33D3C621E546455BD8BA1418BEC8"); - - /* Now check rollover on inplace cipher. */ - crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); - crypto_cipher_crypt_inplace(env1, data2, 64); - test_memeq_hex(data2, "2aed2bff0de54f9328efd070bf48f70a" - "0EDD33D3C621E546455BD8BA1418BEC8" - "93e2c5243d6839eac58503919192f7ae" - "1908e67cafa08d508816659c2e693191"); - crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" - "\xff\xff\xff\xff\xff\xff\xff\xff"); - crypto_cipher_crypt_inplace(env1, data2, 64); - test_assert(tor_mem_is_zero(data2, 64)); - - done: - if (env1) - crypto_free_cipher_env(env1); - if (env2) - crypto_free_cipher_env(env2); - tor_free(data1); - tor_free(data2); - tor_free(data3); -} - -/** Run unit tests for our SHA-1 functionality */ -static void -test_crypto_sha(void) -{ - crypto_digest_env_t *d1 = NULL, *d2 = NULL; - int i; - char key[80]; - char digest[20]; - char data[50]; - char d_out1[DIGEST_LEN], d_out2[DIGEST_LEN]; - - /* Test SHA-1 with a test vector from the specification. */ - i = crypto_digest(data, "abc", 3); - test_memeq_hex(data, "A9993E364706816ABA3E25717850C26C9CD0D89D"); - - /* Test HMAC-SHA-1 with test cases from RFC2202. */ - - /* Case 1. */ - memset(key, 0x0b, 20); - crypto_hmac_sha1(digest, key, 20, "Hi There", 8); - test_streq(hex_str(digest, 20), - "B617318655057264E28BC0B6FB378C8EF146BE00"); - /* Case 2. */ - crypto_hmac_sha1(digest, "Jefe", 4, "what do ya want for nothing?", 28); - test_streq(hex_str(digest, 20), - "EFFCDF6AE5EB2FA2D27416D5F184DF9C259A7C79"); - - /* Case 4. */ - base16_decode(key, 25, - "0102030405060708090a0b0c0d0e0f10111213141516171819", 50); - memset(data, 0xcd, 50); - crypto_hmac_sha1(digest, key, 25, data, 50); - test_streq(hex_str(digest, 20), - "4C9007F4026250C6BC8414F9BF50C86C2D7235DA"); - - /* Case . */ - memset(key, 0xaa, 80); - crypto_hmac_sha1(digest, key, 80, - "Test Using Larger Than Block-Size Key - Hash Key First", - 54); - test_streq(hex_str(digest, 20), - "AA4AE5E15272D00E95705637CE8A3B55ED402112"); - - /* Incremental digest code. */ - d1 = crypto_new_digest_env(); - test_assert(d1); - crypto_digest_add_bytes(d1, "abcdef", 6); - d2 = crypto_digest_dup(d1); - test_assert(d2); - crypto_digest_add_bytes(d2, "ghijkl", 6); - crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); - crypto_digest(d_out2, "abcdefghijkl", 12); - test_memeq(d_out1, d_out2, DIGEST_LEN); - crypto_digest_assign(d2, d1); - crypto_digest_add_bytes(d2, "mno", 3); - crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); - crypto_digest(d_out2, "abcdefmno", 9); - test_memeq(d_out1, d_out2, DIGEST_LEN); - crypto_digest_get_digest(d1, d_out1, sizeof(d_out1)); - crypto_digest(d_out2, "abcdef", 6); - test_memeq(d_out1, d_out2, DIGEST_LEN); - - done: - if (d1) - crypto_free_digest_env(d1); - if (d2) - crypto_free_digest_env(d2); -} - -/** Run unit tests for our public key crypto functions */ -static void -test_crypto_pk(void) -{ - crypto_pk_env_t *pk1 = NULL, *pk2 = NULL; - char *encoded = NULL; - char data1[1024], data2[1024], data3[1024]; - size_t size; - int i, j, p, len; - - /* Public-key ciphers */ - pk1 = pk_generate(0); - pk2 = crypto_new_pk_env(); - test_assert(pk1 && pk2); - test_assert(! crypto_pk_write_public_key_to_string(pk1, &encoded, &size)); - test_assert(! crypto_pk_read_public_key_from_string(pk2, encoded, size)); - test_eq(0, crypto_pk_cmp_keys(pk1, pk2)); - - test_eq(128, crypto_pk_keysize(pk1)); - test_eq(128, crypto_pk_keysize(pk2)); - - test_eq(128, crypto_pk_public_encrypt(pk2, data1, sizeof(data1), - "Hello whirled.", 15, - PK_PKCS1_OAEP_PADDING)); - test_eq(128, crypto_pk_public_encrypt(pk1, data2, sizeof(data2), - "Hello whirled.", 15, - PK_PKCS1_OAEP_PADDING)); - /* oaep padding should make encryption not match */ - test_memneq(data1, data2, 128); - test_eq(15, crypto_pk_private_decrypt(pk1, data3, sizeof(data3), data1, 128, - PK_PKCS1_OAEP_PADDING,1)); - test_streq(data3, "Hello whirled."); - memset(data3, 0, 1024); - test_eq(15, crypto_pk_private_decrypt(pk1, data3, sizeof(data3), data2, 128, - PK_PKCS1_OAEP_PADDING,1)); - test_streq(data3, "Hello whirled."); - /* Can't decrypt with public key. */ - test_eq(-1, crypto_pk_private_decrypt(pk2, data3, sizeof(data3), data2, 128, - PK_PKCS1_OAEP_PADDING,1)); - /* Try again with bad padding */ - memcpy(data2+1, "XYZZY", 5); /* This has fails ~ once-in-2^40 */ - test_eq(-1, crypto_pk_private_decrypt(pk1, data3, sizeof(data3), data2, 128, - PK_PKCS1_OAEP_PADDING,1)); - - /* File operations: save and load private key */ - test_assert(! crypto_pk_write_private_key_to_filename(pk1, - get_fname("pkey1"))); - /* failing case for read: can't read. */ - test_assert(crypto_pk_read_private_key_from_filename(pk2, - get_fname("xyzzy")) < 0); - write_str_to_file(get_fname("xyzzy"), "foobar", 6); - /* Failing case for read: no key. */ - test_assert(crypto_pk_read_private_key_from_filename(pk2, - get_fname("xyzzy")) < 0); - test_assert(! crypto_pk_read_private_key_from_filename(pk2, - get_fname("pkey1"))); - test_eq(15, crypto_pk_private_decrypt(pk2, data3, sizeof(data3), data1, 128, - PK_PKCS1_OAEP_PADDING,1)); - - /* Now try signing. */ - strlcpy(data1, "Ossifrage", 1024); - test_eq(128, crypto_pk_private_sign(pk1, data2, sizeof(data2), data1, 10)); - test_eq(10, crypto_pk_public_checksig(pk1, data3, sizeof(data3), data2, 128)); - test_streq(data3, "Ossifrage"); - /* Try signing digests. */ - test_eq(128, crypto_pk_private_sign_digest(pk1, data2, sizeof(data2), - data1, 10)); - test_eq(20, crypto_pk_public_checksig(pk1, data3, sizeof(data1), data2, 128)); - test_eq(0, crypto_pk_public_checksig_digest(pk1, data1, - 10, data2, 128)); - test_eq(-1, crypto_pk_public_checksig_digest(pk1, data1, - 11, data2, 128)); - /*XXXX test failed signing*/ - - /* Try encoding */ - crypto_free_pk_env(pk2); - pk2 = NULL; - i = crypto_pk_asn1_encode(pk1, data1, 1024); - test_assert(i>0); - pk2 = crypto_pk_asn1_decode(data1, i); - test_assert(crypto_pk_cmp_keys(pk1,pk2) == 0); - - /* Try with hybrid encryption wrappers. */ - crypto_rand(data1, 1024); - for (i = 0; i < 3; ++i) { - for (j = 85; j < 140; ++j) { - memset(data2,0,1024); - memset(data3,0,1024); - if (i == 0 && j < 129) - continue; - p = (i==0)?PK_NO_PADDING: - (i==1)?PK_PKCS1_PADDING:PK_PKCS1_OAEP_PADDING; - len = crypto_pk_public_hybrid_encrypt(pk1,data2,sizeof(data2), - data1,j,p,0); - test_assert(len>=0); - len = crypto_pk_private_hybrid_decrypt(pk1,data3,sizeof(data3), - data2,len,p,1); - test_eq(len,j); - test_memeq(data1,data3,j); - } - } - - /* Try copy_full */ - crypto_free_pk_env(pk2); - pk2 = crypto_pk_copy_full(pk1); - test_assert(pk2 != NULL); - test_neq_ptr(pk1, pk2); - test_assert(crypto_pk_cmp_keys(pk1,pk2) == 0); - - done: - if (pk1) - crypto_free_pk_env(pk1); - if (pk2) - crypto_free_pk_env(pk2); - tor_free(encoded); -} - -/** Run unit tests for misc crypto functionality. */ -static void -test_crypto(void) -{ - char *data1 = NULL, *data2 = NULL, *data3 = NULL; - int i, j, idx; - - data1 = tor_malloc(1024); - data2 = tor_malloc(1024); - data3 = tor_malloc(1024); - test_assert(data1 && data2 && data3); - - /* Base64 tests */ - memset(data1, 6, 1024); - for (idx = 0; idx < 10; ++idx) { - i = base64_encode(data2, 1024, data1, idx); - test_assert(i >= 0); - j = base64_decode(data3, 1024, data2, i); - test_eq(j,idx); - test_memeq(data3, data1, idx); - } - - strlcpy(data1, "Test string that contains 35 chars.", 1024); - strlcat(data1, " 2nd string that contains 35 chars.", 1024); - - i = base64_encode(data2, 1024, data1, 71); - j = base64_decode(data3, 1024, data2, i); - test_eq(j, 71); - test_streq(data3, data1); - test_assert(data2[i] == '\0'); - - crypto_rand(data1, DIGEST_LEN); - memset(data2, 100, 1024); - digest_to_base64(data2, data1); - test_eq(BASE64_DIGEST_LEN, strlen(data2)); - test_eq(100, data2[BASE64_DIGEST_LEN+2]); - memset(data3, 99, 1024); - test_eq(digest_from_base64(data3, data2), 0); - test_memeq(data1, data3, DIGEST_LEN); - test_eq(99, data3[DIGEST_LEN+1]); - - test_assert(digest_from_base64(data3, "###") < 0); - - /* Base32 tests */ - strlcpy(data1, "5chrs", 1024); - /* bit pattern is: [35 63 68 72 73] -> - * [00110101 01100011 01101000 01110010 01110011] - * By 5s: [00110 10101 10001 10110 10000 11100 10011 10011] - */ - base32_encode(data2, 9, data1, 5); - test_streq(data2, "gvrwq4tt"); - - strlcpy(data1, "\xFF\xF5\x6D\x44\xAE\x0D\x5C\xC9\x62\xC4", 1024); - base32_encode(data2, 30, data1, 10); - test_streq(data2, "772w2rfobvomsywe"); - - /* Base16 tests */ - strlcpy(data1, "6chrs\xff", 1024); - base16_encode(data2, 13, data1, 6); - test_streq(data2, "3663687273FF"); - - strlcpy(data1, "f0d678affc000100", 1024); - i = base16_decode(data2, 8, data1, 16); - test_eq(i,0); - test_memeq(data2, "\xf0\xd6\x78\xaf\xfc\x00\x01\x00",8); - - /* now try some failing base16 decodes */ - test_eq(-1, base16_decode(data2, 8, data1, 15)); /* odd input len */ - test_eq(-1, base16_decode(data2, 7, data1, 16)); /* dest too short */ - strlcpy(data1, "f0dz!8affc000100", 1024); - test_eq(-1, base16_decode(data2, 8, data1, 16)); - - tor_free(data1); - tor_free(data2); - tor_free(data3); - - /* Add spaces to fingerprint */ - { - data1 = tor_strdup("ABCD1234ABCD56780000ABCD1234ABCD56780000"); - test_eq(strlen(data1), 40); - data2 = tor_malloc(FINGERPRINT_LEN+1); - add_spaces_to_fp(data2, FINGERPRINT_LEN+1, data1); - test_streq(data2, "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 0000"); - tor_free(data1); - tor_free(data2); - } - - /* Check fingerprint */ - { - test_assert(crypto_pk_check_fingerprint_syntax( - "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 0000")); - test_assert(!crypto_pk_check_fingerprint_syntax( - "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 000")); - test_assert(!crypto_pk_check_fingerprint_syntax( - "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 00000")); - test_assert(!crypto_pk_check_fingerprint_syntax( - "ABCD 1234 ABCD 5678 0000 ABCD1234 ABCD 5678 0000")); - test_assert(!crypto_pk_check_fingerprint_syntax( - "ABCD 1234 ABCD 5678 0000 ABCD1234 ABCD 5678 00000")); - test_assert(!crypto_pk_check_fingerprint_syntax( - "ACD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 00000")); - } - - done: - tor_free(data1); - tor_free(data2); - tor_free(data3); -} - -/** Run unit tests for our secret-to-key passphrase hashing functionality. */ -static void -test_crypto_s2k(void) -{ - char buf[29]; - char buf2[29]; - char *buf3 = NULL; - int i; - - memset(buf, 0, sizeof(buf)); - memset(buf2, 0, sizeof(buf2)); - buf3 = tor_malloc(65536); - memset(buf3, 0, 65536); - - secret_to_key(buf+9, 20, "", 0, buf); - crypto_digest(buf2+9, buf3, 1024); - test_memeq(buf, buf2, 29); - - memcpy(buf,"vrbacrda",8); - memcpy(buf2,"vrbacrda",8); - buf[8] = 96; - buf2[8] = 96; - secret_to_key(buf+9, 20, "12345678", 8, buf); - for (i = 0; i < 65536; i += 16) { - memcpy(buf3+i, "vrbacrda12345678", 16); - } - crypto_digest(buf2+9, buf3, 65536); - test_memeq(buf, buf2, 29); - - done: - tor_free(buf3); -} - -/** Helper: return a tristate based on comparing the strings in *<b>a</b> and - * *<b>b</b>. */ -static int -_compare_strs(const void **a, const void **b) -{ - const char *s1 = *a, *s2 = *b; - return strcmp(s1, s2); -} - -/** Helper: return a tristate based on comparing the strings in *<b>a</b> and - * *<b>b</b>, excluding a's first character, and ignoring case. */ -static int -_compare_without_first_ch(const void *a, const void **b) -{ - const char *s1 = a, *s2 = *b; - return strcasecmp(s1+1, s2); -} - -/** Test basic utility functionality. */ -static void -test_util(void) -{ - struct timeval start, end; - struct tm a_time; - char timestr[RFC1123_TIME_LEN+1]; - char buf[1024]; - time_t t_res; - int i; - uint32_t u32; - uint16_t u16; - char *cp, *k, *v; - const char *str; - - start.tv_sec = 5; - start.tv_usec = 5000; - - end.tv_sec = 5; - end.tv_usec = 5000; - - test_eq(0L, tv_udiff(&start, &end)); - - end.tv_usec = 7000; - - test_eq(2000L, tv_udiff(&start, &end)); - - end.tv_sec = 6; - - test_eq(1002000L, tv_udiff(&start, &end)); - - end.tv_usec = 0; - - test_eq(995000L, tv_udiff(&start, &end)); - - end.tv_sec = 4; - - test_eq(-1005000L, tv_udiff(&start, &end)); - - end.tv_usec = 999990; - start.tv_sec = 1; - start.tv_usec = 500; - - /* The test values here are confirmed to be correct on a platform - * with a working timegm. */ - a_time.tm_year = 2003-1900; - a_time.tm_mon = 7; - a_time.tm_mday = 30; - a_time.tm_hour = 6; - a_time.tm_min = 14; - a_time.tm_sec = 55; - test_eq((time_t) 1062224095UL, tor_timegm(&a_time)); - a_time.tm_year = 2004-1900; /* Try a leap year, after feb. */ - test_eq((time_t) 1093846495UL, tor_timegm(&a_time)); - a_time.tm_mon = 1; /* Try a leap year, in feb. */ - a_time.tm_mday = 10; - test_eq((time_t) 1076393695UL, tor_timegm(&a_time)); - - format_rfc1123_time(timestr, 0); - test_streq("Thu, 01 Jan 1970 00:00:00 GMT", timestr); - format_rfc1123_time(timestr, (time_t)1091580502UL); - test_streq("Wed, 04 Aug 2004 00:48:22 GMT", timestr); - - t_res = 0; - i = parse_rfc1123_time(timestr, &t_res); - test_eq(i,0); - test_eq(t_res, (time_t)1091580502UL); - test_eq(-1, parse_rfc1123_time("Wed, zz Aug 2004 99-99x99 GMT", &t_res)); - tor_gettimeofday(&start); - - /* Tests for corner cases of strl operations */ - test_eq(5, strlcpy(buf, "Hello", 0)); - strlcpy(buf, "Hello", sizeof(buf)); - test_eq(10, strlcat(buf, "Hello", 5)); - - /* Test tor_strstrip() */ - strlcpy(buf, "Testing 1 2 3", sizeof(buf)); - tor_strstrip(buf, ",!"); - test_streq(buf, "Testing 1 2 3"); - strlcpy(buf, "!Testing 1 2 3?", sizeof(buf)); - tor_strstrip(buf, "!? "); - test_streq(buf, "Testing123"); - - /* Test parse_addr_port */ - cp = NULL; u32 = 3; u16 = 3; - test_assert(!parse_addr_port(LOG_WARN, "1.2.3.4", &cp, &u32, &u16)); - test_streq(cp, "1.2.3.4"); - test_eq(u32, 0x01020304u); - test_eq(u16, 0); - tor_free(cp); - test_assert(!parse_addr_port(LOG_WARN, "4.3.2.1:99", &cp, &u32, &u16)); - test_streq(cp, "4.3.2.1"); - test_eq(u32, 0x04030201u); - test_eq(u16, 99); - tor_free(cp); - test_assert(!parse_addr_port(LOG_WARN, "nonexistent.address:4040", - &cp, NULL, &u16)); - test_streq(cp, "nonexistent.address"); - test_eq(u16, 4040); - tor_free(cp); - test_assert(!parse_addr_port(LOG_WARN, "localhost:9999", &cp, &u32, &u16)); - test_streq(cp, "localhost"); - test_eq(u32, 0x7f000001u); - test_eq(u16, 9999); - tor_free(cp); - u32 = 3; - test_assert(!parse_addr_port(LOG_WARN, "localhost", NULL, &u32, &u16)); - test_eq(cp, NULL); - test_eq(u32, 0x7f000001u); - test_eq(u16, 0); - tor_free(cp); - test_eq(0, addr_mask_get_bits(0x0u)); - test_eq(32, addr_mask_get_bits(0xFFFFFFFFu)); - test_eq(16, addr_mask_get_bits(0xFFFF0000u)); - test_eq(31, addr_mask_get_bits(0xFFFFFFFEu)); - test_eq(1, addr_mask_get_bits(0x80000000u)); - - /* Test tor_parse_long. */ - test_eq(10L, tor_parse_long("10",10,0,100,NULL,NULL)); - test_eq(0L, tor_parse_long("10",10,50,100,NULL,NULL)); - test_eq(-50L, tor_parse_long("-50",10,-100,100,NULL,NULL)); - - /* Test tor_parse_ulong */ - test_eq(10UL, tor_parse_ulong("10",10,0,100,NULL,NULL)); - test_eq(0UL, tor_parse_ulong("10",10,50,100,NULL,NULL)); - - /* Test tor_parse_uint64. */ - test_assert(U64_LITERAL(10) == tor_parse_uint64("10 x",10,0,100, &i, &cp)); - test_assert(i == 1); - test_streq(cp, " x"); - test_assert(U64_LITERAL(12345678901) == - tor_parse_uint64("12345678901",10,0,UINT64_MAX, &i, &cp)); - test_assert(i == 1); - test_streq(cp, ""); - test_assert(U64_LITERAL(0) == - tor_parse_uint64("12345678901",10,500,INT32_MAX, &i, &cp)); - test_assert(i == 0); - - /* Test failing snprintf cases */ - test_eq(-1, tor_snprintf(buf, 0, "Foo")); - test_eq(-1, tor_snprintf(buf, 2, "Foo")); - - /* Test printf with uint64 */ - tor_snprintf(buf, sizeof(buf), "x!"U64_FORMAT"!x", - U64_PRINTF_ARG(U64_LITERAL(12345678901))); - test_streq(buf, "x!12345678901!x"); - - /* Test parse_config_line_from_str */ - strlcpy(buf, "k v\n" " key value with spaces \n" "keykey val\n" - "k2\n" - "k3 \n" "\n" " \n" "#comment\n" - "k4#a\n" "k5#abc\n" "k6 val #with comment\n" - "kseven \"a quoted 'string\"\n" - "k8 \"a \\x71uoted\\n\\\"str\\\\ing\\t\\001\\01\\1\\\"\"\n" - , sizeof(buf)); - str = buf; - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k"); - test_streq(v, "v"); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "key value with")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "key"); - test_streq(v, "value with spaces"); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "keykey")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "keykey"); - test_streq(v, "val"); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "k2\n")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k2"); - test_streq(v, ""); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "k3 \n")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k3"); - test_streq(v, ""); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "#comment")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k4"); - test_streq(v, ""); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "k5#abc")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k5"); - test_streq(v, ""); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "k6")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k6"); - test_streq(v, "val"); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "kseven")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "kseven"); - test_streq(v, "a quoted \'string"); - tor_free(k); tor_free(v); - test_assert(!strcmpstart(str, "k8 ")); - - str = parse_config_line_from_str(str, &k, &v); - test_streq(k, "k8"); - test_streq(v, "a quoted\n\"str\\ing\t\x01\x01\x01\""); - tor_free(k); tor_free(v); - test_streq(str, ""); - - /* Test for strcmpstart and strcmpend. */ - test_assert(strcmpstart("abcdef", "abcdef")==0); - test_assert(strcmpstart("abcdef", "abc")==0); - test_assert(strcmpstart("abcdef", "abd")<0); - test_assert(strcmpstart("abcdef", "abb")>0); - test_assert(strcmpstart("ab", "abb")<0); - - test_assert(strcmpend("abcdef", "abcdef")==0); - test_assert(strcmpend("abcdef", "def")==0); - test_assert(strcmpend("abcdef", "deg")<0); - test_assert(strcmpend("abcdef", "dee")>0); - test_assert(strcmpend("ab", "abb")<0); - - test_assert(strcasecmpend("AbcDEF", "abcdef")==0); - test_assert(strcasecmpend("abcdef", "dEF")==0); - test_assert(strcasecmpend("abcDEf", "deg")<0); - test_assert(strcasecmpend("abcdef", "DEE")>0); - test_assert(strcasecmpend("ab", "abB")<0); - - /* Test mem_is_zero */ - memset(buf,0,128); - buf[128] = 'x'; - test_assert(tor_digest_is_zero(buf)); - test_assert(tor_mem_is_zero(buf, 10)); - test_assert(tor_mem_is_zero(buf, 20)); - test_assert(tor_mem_is_zero(buf, 128)); - test_assert(!tor_mem_is_zero(buf, 129)); - buf[60] = (char)255; - test_assert(!tor_mem_is_zero(buf, 128)); - buf[0] = (char)1; - test_assert(!tor_mem_is_zero(buf, 10)); - - /* Test inet_ntop */ - { - char tmpbuf[TOR_ADDR_BUF_LEN]; - const char *ip = "176.192.208.224"; - struct in_addr in; - tor_inet_pton(AF_INET, ip, &in); - tor_inet_ntop(AF_INET, &in, tmpbuf, sizeof(tmpbuf)); - test_streq(tmpbuf, ip); - } - - /* Test 'escaped' */ - test_streq("\"\"", escaped("")); - test_streq("\"abcd\"", escaped("abcd")); - test_streq("\"\\\\\\n\\r\\t\\\"\\'\"", escaped("\\\n\r\t\"\'")); - test_streq("\"z\\001abc\\277d\"", escaped("z\001abc\277d")); - test_assert(NULL == escaped(NULL)); - - /* Test strndup and memdup */ - { - const char *s = "abcdefghijklmnopqrstuvwxyz"; - cp = tor_strndup(s, 30); - test_streq(cp, s); /* same string, */ - test_neq(cp, s); /* but different pointers. */ - tor_free(cp); - - cp = tor_strndup(s, 5); - test_streq(cp, "abcde"); - tor_free(cp); - - s = "a\0b\0c\0d\0e\0"; - cp = tor_memdup(s,10); - test_memeq(cp, s, 10); /* same ram, */ - test_neq(cp, s); /* but different pointers. */ - tor_free(cp); - } - - /* Test str-foo functions */ - cp = tor_strdup("abcdef"); - test_assert(tor_strisnonupper(cp)); - cp[3] = 'D'; - test_assert(!tor_strisnonupper(cp)); - tor_strupper(cp); - test_streq(cp, "ABCDEF"); - test_assert(tor_strisprint(cp)); - cp[3] = 3; - test_assert(!tor_strisprint(cp)); - tor_free(cp); - - /* Test eat_whitespace. */ - { - const char *s = " \n a"; - test_eq_ptr(eat_whitespace(s), s+4); - s = "abcd"; - test_eq_ptr(eat_whitespace(s), s); - s = "#xyz\nab"; - test_eq_ptr(eat_whitespace(s), s+5); - } - - /* Test memmem and memstr */ - { - const char *haystack = "abcde"; - tor_assert(!tor_memmem(haystack, 5, "ef", 2)); - test_eq_ptr(tor_memmem(haystack, 5, "cd", 2), haystack + 2); - test_eq_ptr(tor_memmem(haystack, 5, "cde", 3), haystack + 2); - haystack = "ababcad"; - test_eq_ptr(tor_memmem(haystack, 7, "abc", 3), haystack + 2); - test_eq_ptr(tor_memstr(haystack, 7, "abc"), haystack + 2); - test_assert(!tor_memstr(haystack, 7, "fe")); - test_assert(!tor_memstr(haystack, 7, "longerthantheoriginal")); - } - - /* Test wrap_string */ - { - smartlist_t *sl = smartlist_create(); - wrap_string(sl, "This is a test of string wrapping functionality: woot.", - 10, "", ""); - cp = smartlist_join_strings(sl, "", 0, NULL); - test_streq(cp, - "This is a\ntest of\nstring\nwrapping\nfunctional\nity: woot.\n"); - tor_free(cp); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - wrap_string(sl, "This is a test of string wrapping functionality: woot.", - 16, "### ", "# "); - cp = smartlist_join_strings(sl, "", 0, NULL); - test_streq(cp, - "### This is a\n# test of string\n# wrapping\n# functionality:\n" - "# woot.\n"); - - tor_free(cp); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_free(sl); - } - - tor_gettimeofday(&start); - /* now make sure time works. */ - tor_gettimeofday(&end); - /* We might've timewarped a little. */ - test_assert(tv_udiff(&start, &end) >= -5000); - - /* Test tor_log2(). */ - test_eq(tor_log2(64), 6); - test_eq(tor_log2(65), 6); - test_eq(tor_log2(63), 5); - test_eq(tor_log2(1), 0); - test_eq(tor_log2(2), 1); - test_eq(tor_log2(3), 1); - test_eq(tor_log2(4), 2); - test_eq(tor_log2(5), 2); - test_eq(tor_log2(U64_LITERAL(40000000000000000)), 55); - test_eq(tor_log2(UINT64_MAX), 63); - - /* Test round_to_power_of_2 */ - test_eq(round_to_power_of_2(120), 128); - test_eq(round_to_power_of_2(128), 128); - test_eq(round_to_power_of_2(130), 128); - test_eq(round_to_power_of_2(U64_LITERAL(40000000000000000)), - U64_LITERAL(1)<<55); - test_eq(round_to_power_of_2(0), 2); - - done: - ; -} - -/** Helper: assert that IPv6 addresses <b>a</b> and <b>b</b> are the same. On - * failure, reports an error, describing the addresses as <b>e1</b> and - * <b>e2</b>, and reporting the line number as <b>line</b>. */ -static void -_test_eq_ip6(struct in6_addr *a, struct in6_addr *b, const char *e1, - const char *e2, int line) -{ - int i; - int ok = 1; - for (i = 0; i < 16; ++i) { - if (a->s6_addr[i] != b->s6_addr[i]) { - ok = 0; - break; - } - } - if (ok) { - printf("."); fflush(stdout); - } else { - char buf1[128], *cp1; - char buf2[128], *cp2; - have_failed = 1; - cp1 = buf1; cp2 = buf2; - for (i=0; i<16; ++i) { - tor_snprintf(cp1, sizeof(buf1)-(cp1-buf1), "%02x", a->s6_addr[i]); - tor_snprintf(cp2, sizeof(buf2)-(cp2-buf2), "%02x", b->s6_addr[i]); - cp1 += 2; cp2 += 2; - if ((i%2)==1 && i != 15) { - *cp1++ = ':'; - *cp2++ = ':'; - } - } - *cp1 = *cp2 = '\0'; - printf("Line %d: assertion failed: (%s == %s)\n" - " %s != %s\n", line, e1, e2, buf1, buf2); - fflush(stdout); - } -} - -/** Helper: Assert that two strings both decode as IPv6 addresses with - * tor_inet_pton(), and both decode to the same address. */ -#define test_pton6_same(a,b) STMT_BEGIN \ - test_eq(tor_inet_pton(AF_INET6, a, &a1), 1); \ - test_eq(tor_inet_pton(AF_INET6, b, &a2), 1); \ - _test_eq_ip6(&a1,&a2,#a,#b,__LINE__); \ - STMT_END - -/** Helper: Assert that <b>a</b> is recognized as a bad IPv6 address by - * tor_inet_pton(). */ -#define test_pton6_bad(a) \ - test_eq(0, tor_inet_pton(AF_INET6, a, &a1)) - -/** Helper: assert that <b>a</b>, when parsed by tor_inet_pton() and displayed - * with tor_inet_ntop(), yields <b>b</b>. Also assert that <b>b</b> parses to - * the same value as <b>a</b>. */ -#define test_ntop6_reduces(a,b) STMT_BEGIN \ - test_eq(tor_inet_pton(AF_INET6, a, &a1), 1); \ - test_streq(tor_inet_ntop(AF_INET6, &a1, buf, sizeof(buf)), b); \ - test_eq(tor_inet_pton(AF_INET6, b, &a2), 1); \ - _test_eq_ip6(&a1, &a2, a, b, __LINE__); \ - STMT_END - -/** Helper: assert that <b>a</b> parses by tor_inet_pton() into a address that - * passes tor_addr_is_internal() with <b>for_listening</b>. */ -#define test_internal_ip(a,for_listening) STMT_BEGIN \ - test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ - t1.family = AF_INET6; \ - if (!tor_addr_is_internal(&t1, for_listening)) \ - test_fail_msg( a "was not internal."); \ - STMT_END - -/** Helper: assert that <b>a</b> parses by tor_inet_pton() into a address that - * does not pass tor_addr_is_internal() with <b>for_listening</b>. */ -#define test_external_ip(a,for_listening) STMT_BEGIN \ - test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ - t1.family = AF_INET6; \ - if (tor_addr_is_internal(&t1, for_listening)) \ - test_fail_msg(a "was not external."); \ - STMT_END - -/** Helper: Assert that <b>a</b> and <b>b</b>, when parsed by - * tor_inet_pton(), give addresses that compare in the order defined by - * <b>op</b> with tor_addr_compare(). */ -#define test_addr_compare(a, op, b) STMT_BEGIN \ - test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ - test_eq(tor_inet_pton(AF_INET6, b, &t2.addr.in6_addr), 1); \ - t1.family = t2.family = AF_INET6; \ - r = tor_addr_compare(&t1,&t2,CMP_SEMANTIC); \ - if (!(r op 0)) \ - test_fail_msg("failed: tor_addr_compare("a","b") "#op" 0"); \ - STMT_END - -/** Helper: Assert that <b>a</b> and <b>b</b>, when parsed by - * tor_inet_pton(), give addresses that compare in the order defined by - * <b>op</b> with tor_addr_compare_masked() with <b>m</b> masked. */ -#define test_addr_compare_masked(a, op, b, m) STMT_BEGIN \ - test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ - test_eq(tor_inet_pton(AF_INET6, b, &t2.addr.in6_addr), 1); \ - t1.family = t2.family = AF_INET6; \ - r = tor_addr_compare_masked(&t1,&t2,m,CMP_SEMANTIC); \ - if (!(r op 0)) \ - test_fail_msg("failed: tor_addr_compare_masked("a","b","#m") "#op" 0"); \ - STMT_END - -/** Helper: assert that <b>xx</b> is parseable as a masked IPv6 address with - * ports by tor_parse_mask_addr_ports(), with family <b>f</b>, IP address - * as 4 32-bit words <b>ip1...ip4</b>, mask bits as <b>mm</b>, and port range - * as <b>pt1..pt2</b>. */ -#define test_addr_mask_ports_parse(xx, f, ip1, ip2, ip3, ip4, mm, pt1, pt2) \ - STMT_BEGIN \ - test_eq(tor_addr_parse_mask_ports(xx, &t1, &mask, &port1, &port2), f); \ - p1=tor_inet_ntop(AF_INET6, &t1.addr.in6_addr, bug, sizeof(bug)); \ - test_eq(htonl(ip1), tor_addr_to_in6_addr32(&t1)[0]); \ - test_eq(htonl(ip2), tor_addr_to_in6_addr32(&t1)[1]); \ - test_eq(htonl(ip3), tor_addr_to_in6_addr32(&t1)[2]); \ - test_eq(htonl(ip4), tor_addr_to_in6_addr32(&t1)[3]); \ - test_eq(mask, mm); \ - test_eq(port1, pt1); \ - test_eq(port2, pt2); \ - STMT_END - -/** Run unit tests for IPv6 encoding/decoding/manipulation functions. */ -static void -test_util_ip6_helpers(void) -{ - char buf[TOR_ADDR_BUF_LEN], bug[TOR_ADDR_BUF_LEN]; - struct in6_addr a1, a2; - tor_addr_t t1, t2; - int r, i; - uint16_t port1, port2; - maskbits_t mask; - const char *p1; - struct sockaddr_storage sa_storage; - struct sockaddr_in *sin; - struct sockaddr_in6 *sin6; - - // struct in_addr b1, b2; - /* Test tor_inet_ntop and tor_inet_pton: IPv6 */ - - /* ==== Converting to and from sockaddr_t. */ - sin = (struct sockaddr_in *)&sa_storage; - sin->sin_family = AF_INET; - sin->sin_port = 9090; - sin->sin_addr.s_addr = htonl(0x7f7f0102); /*127.127.1.2*/ - tor_addr_from_sockaddr(&t1, (struct sockaddr *)sin, NULL); - test_eq(tor_addr_family(&t1), AF_INET); - test_eq(tor_addr_to_ipv4h(&t1), 0x7f7f0102); - - memset(&sa_storage, 0, sizeof(sa_storage)); - test_eq(sizeof(struct sockaddr_in), - tor_addr_to_sockaddr(&t1, 1234, (struct sockaddr *)&sa_storage, - sizeof(sa_storage))); - test_eq(1234, ntohs(sin->sin_port)); - test_eq(0x7f7f0102, ntohl(sin->sin_addr.s_addr)); - - memset(&sa_storage, 0, sizeof(sa_storage)); - sin6 = (struct sockaddr_in6 *)&sa_storage; - sin6->sin6_family = AF_INET6; - sin6->sin6_port = htons(7070); - sin6->sin6_addr.s6_addr[0] = 128; - tor_addr_from_sockaddr(&t1, (struct sockaddr *)sin6, NULL); - test_eq(tor_addr_family(&t1), AF_INET6); - p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 0); - test_streq(p1, "8000::"); - - memset(&sa_storage, 0, sizeof(sa_storage)); - test_eq(sizeof(struct sockaddr_in6), - tor_addr_to_sockaddr(&t1, 9999, (struct sockaddr *)&sa_storage, - sizeof(sa_storage))); - test_eq(AF_INET6, sin6->sin6_family); - test_eq(9999, ntohs(sin6->sin6_port)); - test_eq(0x80000000, ntohl(S6_ADDR32(sin6->sin6_addr)[0])); - - /* ==== tor_addr_lookup: static cases. (Can't test dns without knowing we - * have a good resolver. */ - test_eq(0, tor_addr_lookup("127.128.129.130", AF_UNSPEC, &t1)); - test_eq(AF_INET, tor_addr_family(&t1)); - test_eq(tor_addr_to_ipv4h(&t1), 0x7f808182); - - test_eq(0, tor_addr_lookup("9000::5", AF_UNSPEC, &t1)); - test_eq(AF_INET6, tor_addr_family(&t1)); - test_eq(0x90, tor_addr_to_in6_addr8(&t1)[0]); - test_assert(tor_mem_is_zero((char*)tor_addr_to_in6_addr8(&t1)+1, 14)); - test_eq(0x05, tor_addr_to_in6_addr8(&t1)[15]); - - /* === Test pton: valid af_inet6 */ - /* Simple, valid parsing. */ - r = tor_inet_pton(AF_INET6, - "0102:0304:0506:0708:090A:0B0C:0D0E:0F10", &a1); - test_assert(r==1); - for (i=0;i<16;++i) { test_eq(i+1, (int)a1.s6_addr[i]); } - /* ipv4 ending. */ - test_pton6_same("0102:0304:0506:0708:090A:0B0C:0D0E:0F10", - "0102:0304:0506:0708:090A:0B0C:13.14.15.16"); - /* shortened words. */ - test_pton6_same("0001:0099:BEEF:0000:0123:FFFF:0001:0001", - "1:99:BEEF:0:0123:FFFF:1:1"); - /* zeros at the beginning */ - test_pton6_same("0000:0000:0000:0000:0009:C0A8:0001:0001", - "::9:c0a8:1:1"); - test_pton6_same("0000:0000:0000:0000:0009:C0A8:0001:0001", - "::9:c0a8:0.1.0.1"); - /* zeros in the middle. */ - test_pton6_same("fe80:0000:0000:0000:0202:1111:0001:0001", - "fe80::202:1111:1:1"); - /* zeros at the end. */ - test_pton6_same("1000:0001:0000:0007:0000:0000:0000:0000", - "1000:1:0:7::"); - - /* === Test ntop: af_inet6 */ - test_ntop6_reduces("0:0:0:0:0:0:0:0", "::"); - - test_ntop6_reduces("0001:0099:BEEF:0006:0123:FFFF:0001:0001", - "1:99:beef:6:123:ffff:1:1"); - - //test_ntop6_reduces("0:0:0:0:0:0:c0a8:0101", "::192.168.1.1"); - test_ntop6_reduces("0:0:0:0:0:ffff:c0a8:0101", "::ffff:192.168.1.1"); - test_ntop6_reduces("002:0:0000:0:3::4", "2::3:0:0:4"); - test_ntop6_reduces("0:0::1:0:3", "::1:0:3"); - test_ntop6_reduces("008:0::0", "8::"); - test_ntop6_reduces("0:0:0:0:0:ffff::1", "::ffff:0.0.0.1"); - test_ntop6_reduces("abcd:0:0:0:0:0:7f00::", "abcd::7f00:0"); - test_ntop6_reduces("0000:0000:0000:0000:0009:C0A8:0001:0001", - "::9:c0a8:1:1"); - test_ntop6_reduces("fe80:0000:0000:0000:0202:1111:0001:0001", - "fe80::202:1111:1:1"); - test_ntop6_reduces("1000:0001:0000:0007:0000:0000:0000:0000", - "1000:1:0:7::"); - - /* === Test pton: invalid in6. */ - test_pton6_bad("foobar."); - test_pton6_bad("55555::"); - test_pton6_bad("9:-60::"); - test_pton6_bad("1:2:33333:4:0002:3::"); - //test_pton6_bad("1:2:3333:4:00002:3::");// BAD, but glibc doesn't say so. - test_pton6_bad("1:2:3333:4:fish:3::"); - test_pton6_bad("1:2:3:4:5:6:7:8:9"); - test_pton6_bad("1:2:3:4:5:6:7"); - test_pton6_bad("1:2:3:4:5:6:1.2.3.4.5"); - test_pton6_bad("1:2:3:4:5:6:1.2.3"); - test_pton6_bad("::1.2.3"); - test_pton6_bad("::1.2.3.4.5"); - test_pton6_bad("99"); - test_pton6_bad(""); - test_pton6_bad("1::2::3:4"); - test_pton6_bad("a:::b:c"); - test_pton6_bad(":::a:b:c"); - test_pton6_bad("a:b:c:::"); - - /* test internal checking */ - test_external_ip("fbff:ffff::2:7", 0); - test_internal_ip("fc01::2:7", 0); - test_internal_ip("fdff:ffff::f:f", 0); - test_external_ip("fe00::3:f", 0); - - test_external_ip("fe7f:ffff::2:7", 0); - test_internal_ip("fe80::2:7", 0); - test_internal_ip("febf:ffff::f:f", 0); - - test_internal_ip("fec0::2:7:7", 0); - test_internal_ip("feff:ffff::e:7:7", 0); - test_external_ip("ff00::e:7:7", 0); - - test_internal_ip("::", 0); - test_internal_ip("::1", 0); - test_internal_ip("::1", 1); - test_internal_ip("::", 0); - test_external_ip("::", 1); - test_external_ip("::2", 0); - test_external_ip("2001::", 0); - test_external_ip("ffff::", 0); - - test_external_ip("::ffff:0.0.0.0", 1); - test_internal_ip("::ffff:0.0.0.0", 0); - test_internal_ip("::ffff:0.255.255.255", 0); - test_external_ip("::ffff:1.0.0.0", 0); - - test_external_ip("::ffff:9.255.255.255", 0); - test_internal_ip("::ffff:10.0.0.0", 0); - test_internal_ip("::ffff:10.255.255.255", 0); - test_external_ip("::ffff:11.0.0.0", 0); - - test_external_ip("::ffff:126.255.255.255", 0); - test_internal_ip("::ffff:127.0.0.0", 0); - test_internal_ip("::ffff:127.255.255.255", 0); - test_external_ip("::ffff:128.0.0.0", 0); - - test_external_ip("::ffff:172.15.255.255", 0); - test_internal_ip("::ffff:172.16.0.0", 0); - test_internal_ip("::ffff:172.31.255.255", 0); - test_external_ip("::ffff:172.32.0.0", 0); - - test_external_ip("::ffff:192.167.255.255", 0); - test_internal_ip("::ffff:192.168.0.0", 0); - test_internal_ip("::ffff:192.168.255.255", 0); - test_external_ip("::ffff:192.169.0.0", 0); - - test_external_ip("::ffff:169.253.255.255", 0); - test_internal_ip("::ffff:169.254.0.0", 0); - test_internal_ip("::ffff:169.254.255.255", 0); - test_external_ip("::ffff:169.255.0.0", 0); - test_assert(is_internal_IP(0x7f000001, 0)); - - /* tor_addr_compare(tor_addr_t x2) */ - test_addr_compare("ffff::", ==, "ffff::0"); - test_addr_compare("0::3:2:1", <, "0::ffff:0.3.2.1"); - test_addr_compare("0::2:2:1", <, "0::ffff:0.3.2.1"); - test_addr_compare("0::ffff:0.3.2.1", >, "0::0:0:0"); - test_addr_compare("0::ffff:5.2.2.1", <, "::ffff:6.0.0.0"); /* XXXX wrong. */ - tor_addr_parse_mask_ports("[::ffff:2.3.4.5]", &t1, NULL, NULL, NULL); - tor_addr_parse_mask_ports("2.3.4.5", &t2, NULL, NULL, NULL); - test_assert(tor_addr_compare(&t1, &t2, CMP_SEMANTIC) == 0); - tor_addr_parse_mask_ports("[::ffff:2.3.4.4]", &t1, NULL, NULL, NULL); - tor_addr_parse_mask_ports("2.3.4.5", &t2, NULL, NULL, NULL); - test_assert(tor_addr_compare(&t1, &t2, CMP_SEMANTIC) < 0); - - /* test compare_masked */ - test_addr_compare_masked("ffff::", ==, "ffff::0", 128); - test_addr_compare_masked("ffff::", ==, "ffff::0", 64); - test_addr_compare_masked("0::2:2:1", <, "0::8000:2:1", 81); - test_addr_compare_masked("0::2:2:1", ==, "0::8000:2:1", 80); - - /* Test decorated addr_to_string. */ - test_eq(AF_INET6, tor_addr_from_str(&t1, "[123:45:6789::5005:11]")); - p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); - test_streq(p1, "[123:45:6789::5005:11]"); - test_eq(AF_INET, tor_addr_from_str(&t1, "18.0.0.1")); - p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); - test_streq(p1, "18.0.0.1"); - - /* Test tor_addr_parse_reverse_lookup_name */ - i = tor_addr_parse_reverse_lookup_name(&t1, "Foobar.baz", AF_UNSPEC, 0); - test_eq(0, i); - i = tor_addr_parse_reverse_lookup_name(&t1, "Foobar.baz", AF_UNSPEC, 1); - test_eq(0, i); - i = tor_addr_parse_reverse_lookup_name(&t1, "1.0.168.192.in-addr.arpa", - AF_UNSPEC, 1); - test_eq(1, i); - test_eq(tor_addr_family(&t1), AF_INET); - p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); - test_streq(p1, "192.168.0.1"); - i = tor_addr_parse_reverse_lookup_name(&t1, "192.168.0.99", AF_UNSPEC, 0); - test_eq(0, i); - i = tor_addr_parse_reverse_lookup_name(&t1, "192.168.0.99", AF_UNSPEC, 1); - test_eq(1, i); - p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); - test_streq(p1, "192.168.0.99"); - memset(&t1, 0, sizeof(t1)); - i = tor_addr_parse_reverse_lookup_name(&t1, - "0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f." - "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." - "ip6.ARPA", - AF_UNSPEC, 0); - test_eq(1, i); - p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); - test_streq(p1, "[9dee:effe:ebe1:beef:fedc:ba98:7654:3210]"); - /* Failing cases. */ - i = tor_addr_parse_reverse_lookup_name(&t1, - "6.7.8.9.a.b.c.d.e.f." - "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." - "ip6.ARPA", - AF_UNSPEC, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, - "6.7.8.9.a.b.c.d.e.f.a.b.c.d.e.f.0." - "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." - "ip6.ARPA", - AF_UNSPEC, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, - "6.7.8.9.a.b.c.d.e.f.X.0.0.0.0.9." - "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." - "ip6.ARPA", - AF_UNSPEC, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, "32.1.1.in-addr.arpa", - AF_UNSPEC, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, ".in-addr.arpa", - AF_UNSPEC, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, "1.2.3.4.5.in-addr.arpa", - AF_UNSPEC, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, "1.2.3.4.5.in-addr.arpa", - AF_INET6, 0); - test_eq(i, -1); - i = tor_addr_parse_reverse_lookup_name(&t1, - "6.7.8.9.a.b.c.d.e.f.a.b.c.d.e.0." - "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." - "ip6.ARPA", - AF_INET, 0); - test_eq(i, -1); - - /* test tor_addr_parse_mask_ports */ - test_addr_mask_ports_parse("[::f]/17:47-95", AF_INET6, - 0, 0, 0, 0x0000000f, 17, 47, 95); - //test_addr_parse("[::fefe:4.1.1.7/120]:999-1000"); - //test_addr_parse_check("::fefe:401:107", 120, 999, 1000); - test_addr_mask_ports_parse("[::ffff:4.1.1.7]/120:443", AF_INET6, - 0, 0, 0x0000ffff, 0x04010107, 120, 443, 443); - test_addr_mask_ports_parse("[abcd:2::44a:0]:2-65000", AF_INET6, - 0xabcd0002, 0, 0, 0x044a0000, 128, 2, 65000); - - r=tor_addr_parse_mask_ports("[fefef::]/112", &t1, NULL, NULL, NULL); - test_assert(r == -1); - r=tor_addr_parse_mask_ports("efef::/112", &t1, NULL, NULL, NULL); - test_assert(r == -1); - r=tor_addr_parse_mask_ports("[f:f:f:f:f:f:f:f::]", &t1, NULL, NULL, NULL); - test_assert(r == -1); - r=tor_addr_parse_mask_ports("[::f:f:f:f:f:f:f:f]", &t1, NULL, NULL, NULL); - test_assert(r == -1); - r=tor_addr_parse_mask_ports("[f:f:f:f:f:f:f:f:f]", &t1, NULL, NULL, NULL); - test_assert(r == -1); - /* Test for V4-mapped address with mask < 96. (arguably not valid) */ - r=tor_addr_parse_mask_ports("[::ffff:1.1.2.2/33]", &t1, &mask, NULL, NULL); - test_assert(r == -1); - r=tor_addr_parse_mask_ports("1.1.2.2/33", &t1, &mask, NULL, NULL); - test_assert(r == -1); - r=tor_addr_parse_mask_ports("1.1.2.2/31", &t1, &mask, NULL, NULL); - test_assert(r == AF_INET); - r=tor_addr_parse_mask_ports("[efef::]/112", &t1, &mask, &port1, &port2); - test_assert(r == AF_INET6); - test_assert(port1 == 1); - test_assert(port2 == 65535); - - /* make sure inet address lengths >= max */ - test_assert(INET_NTOA_BUF_LEN >= sizeof("255.255.255.255")); - test_assert(TOR_ADDR_BUF_LEN >= - sizeof("ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255")); - - test_assert(sizeof(tor_addr_t) >= sizeof(struct in6_addr)); - - /* get interface addresses */ - r = get_interface_address6(LOG_DEBUG, AF_INET, &t1); - i = get_interface_address6(LOG_DEBUG, AF_INET6, &t2); -#if 0 - tor_inet_ntop(AF_INET, &t1.sa.sin_addr, buf, sizeof(buf)); - printf("\nv4 address: %s (family=%i)", buf, IN_FAMILY(&t1)); - tor_inet_ntop(AF_INET6, &t2.sa6.sin6_addr, buf, sizeof(buf)); - printf("\nv6 address: %s (family=%i)", buf, IN_FAMILY(&t2)); -#endif - - done: - ; -} - -/** Run unit tests for basic dynamic-sized array functionality. */ -static void -test_util_smartlist_basic(void) -{ - smartlist_t *sl; - - /* XXXX test sort_digests, uniq_strings, uniq_digests */ - - /* Test smartlist add, del_keeporder, insert, get. */ - sl = smartlist_create(); - smartlist_add(sl, (void*)1); - smartlist_add(sl, (void*)2); - smartlist_add(sl, (void*)3); - smartlist_add(sl, (void*)4); - smartlist_del_keeporder(sl, 1); - smartlist_insert(sl, 1, (void*)22); - smartlist_insert(sl, 0, (void*)0); - smartlist_insert(sl, 5, (void*)555); - test_eq_ptr((void*)0, smartlist_get(sl,0)); - test_eq_ptr((void*)1, smartlist_get(sl,1)); - test_eq_ptr((void*)22, smartlist_get(sl,2)); - test_eq_ptr((void*)3, smartlist_get(sl,3)); - test_eq_ptr((void*)4, smartlist_get(sl,4)); - test_eq_ptr((void*)555, smartlist_get(sl,5)); - /* Try deleting in the middle. */ - smartlist_del(sl, 1); - test_eq_ptr((void*)555, smartlist_get(sl, 1)); - /* Try deleting at the end. */ - smartlist_del(sl, 4); - test_eq(4, smartlist_len(sl)); - - /* test isin. */ - test_assert(smartlist_isin(sl, (void*)3)); - test_assert(!smartlist_isin(sl, (void*)99)); - - done: - smartlist_free(sl); -} - -/** Run unit tests for smartlist-of-strings functionality. */ -static void -test_util_smartlist_strings(void) -{ - smartlist_t *sl = smartlist_create(); - char *cp=NULL, *cp_alloc=NULL; - size_t sz; - - /* Test split and join */ - test_eq(0, smartlist_len(sl)); - smartlist_split_string(sl, "abc", ":", 0, 0); - test_eq(1, smartlist_len(sl)); - test_streq("abc", smartlist_get(sl, 0)); - smartlist_split_string(sl, "a::bc::", "::", 0, 0); - test_eq(4, smartlist_len(sl)); - test_streq("a", smartlist_get(sl, 1)); - test_streq("bc", smartlist_get(sl, 2)); - test_streq("", smartlist_get(sl, 3)); - cp_alloc = smartlist_join_strings(sl, "", 0, NULL); - test_streq(cp_alloc, "abcabc"); - tor_free(cp_alloc); - cp_alloc = smartlist_join_strings(sl, "!", 0, NULL); - test_streq(cp_alloc, "abc!a!bc!"); - tor_free(cp_alloc); - cp_alloc = smartlist_join_strings(sl, "XY", 0, NULL); - test_streq(cp_alloc, "abcXYaXYbcXY"); - tor_free(cp_alloc); - cp_alloc = smartlist_join_strings(sl, "XY", 1, NULL); - test_streq(cp_alloc, "abcXYaXYbcXYXY"); - tor_free(cp_alloc); - cp_alloc = smartlist_join_strings(sl, "", 1, NULL); - test_streq(cp_alloc, "abcabc"); - tor_free(cp_alloc); - - smartlist_split_string(sl, "/def/ /ghijk", "/", 0, 0); - test_eq(8, smartlist_len(sl)); - test_streq("", smartlist_get(sl, 4)); - test_streq("def", smartlist_get(sl, 5)); - test_streq(" ", smartlist_get(sl, 6)); - test_streq("ghijk", smartlist_get(sl, 7)); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - smartlist_split_string(sl, "a,bbd,cdef", ",", SPLIT_SKIP_SPACE, 0); - test_eq(3, smartlist_len(sl)); - test_streq("a", smartlist_get(sl,0)); - test_streq("bbd", smartlist_get(sl,1)); - test_streq("cdef", smartlist_get(sl,2)); - smartlist_split_string(sl, " z <> zhasd <> <> bnud<> ", "<>", - SPLIT_SKIP_SPACE, 0); - test_eq(8, smartlist_len(sl)); - test_streq("z", smartlist_get(sl,3)); - test_streq("zhasd", smartlist_get(sl,4)); - test_streq("", smartlist_get(sl,5)); - test_streq("bnud", smartlist_get(sl,6)); - test_streq("", smartlist_get(sl,7)); - - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - smartlist_split_string(sl, " ab\tc \td ef ", NULL, - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - test_eq(4, smartlist_len(sl)); - test_streq("ab", smartlist_get(sl,0)); - test_streq("c", smartlist_get(sl,1)); - test_streq("d", smartlist_get(sl,2)); - test_streq("ef", smartlist_get(sl,3)); - smartlist_split_string(sl, "ghi\tj", NULL, - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - test_eq(6, smartlist_len(sl)); - test_streq("ghi", smartlist_get(sl,4)); - test_streq("j", smartlist_get(sl,5)); - - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - cp_alloc = smartlist_join_strings(sl, "XY", 0, NULL); - test_streq(cp_alloc, ""); - tor_free(cp_alloc); - cp_alloc = smartlist_join_strings(sl, "XY", 1, NULL); - test_streq(cp_alloc, "XY"); - tor_free(cp_alloc); - - smartlist_split_string(sl, " z <> zhasd <> <> bnud<> ", "<>", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - test_eq(3, smartlist_len(sl)); - test_streq("z", smartlist_get(sl, 0)); - test_streq("zhasd", smartlist_get(sl, 1)); - test_streq("bnud", smartlist_get(sl, 2)); - smartlist_split_string(sl, " z <> zhasd <> <> bnud<> ", "<>", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 2); - test_eq(5, smartlist_len(sl)); - test_streq("z", smartlist_get(sl, 3)); - test_streq("zhasd <> <> bnud<>", smartlist_get(sl, 4)); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - smartlist_split_string(sl, "abcd\n", "\n", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - test_eq(1, smartlist_len(sl)); - test_streq("abcd", smartlist_get(sl, 0)); - smartlist_split_string(sl, "efgh", "\n", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - test_eq(2, smartlist_len(sl)); - test_streq("efgh", smartlist_get(sl, 1)); - - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - /* Test swapping, shuffling, and sorting. */ - smartlist_split_string(sl, "the,onion,router,by,arma,and,nickm", ",", 0, 0); - test_eq(7, smartlist_len(sl)); - smartlist_sort(sl, _compare_strs); - cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); - test_streq(cp_alloc,"and,arma,by,nickm,onion,router,the"); - tor_free(cp_alloc); - smartlist_swap(sl, 1, 5); - cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); - test_streq(cp_alloc,"and,router,by,nickm,onion,arma,the"); - tor_free(cp_alloc); - smartlist_shuffle(sl); - test_eq(7, smartlist_len(sl)); - test_assert(smartlist_string_isin(sl, "and")); - test_assert(smartlist_string_isin(sl, "router")); - test_assert(smartlist_string_isin(sl, "by")); - test_assert(smartlist_string_isin(sl, "nickm")); - test_assert(smartlist_string_isin(sl, "onion")); - test_assert(smartlist_string_isin(sl, "arma")); - test_assert(smartlist_string_isin(sl, "the")); - - /* Test bsearch. */ - smartlist_sort(sl, _compare_strs); - test_streq("nickm", smartlist_bsearch(sl, "zNicKM", - _compare_without_first_ch)); - test_streq("and", smartlist_bsearch(sl, " AND", _compare_without_first_ch)); - test_eq_ptr(NULL, smartlist_bsearch(sl, " ANz", _compare_without_first_ch)); - - /* Test bsearch_idx */ - { - int f; - test_eq(0, smartlist_bsearch_idx(sl," aaa",_compare_without_first_ch,&f)); - test_eq(f, 0); - test_eq(0, smartlist_bsearch_idx(sl," and",_compare_without_first_ch,&f)); - test_eq(f, 1); - test_eq(1, smartlist_bsearch_idx(sl," arm",_compare_without_first_ch,&f)); - test_eq(f, 0); - test_eq(1, smartlist_bsearch_idx(sl," arma",_compare_without_first_ch,&f)); - test_eq(f, 1); - test_eq(2, smartlist_bsearch_idx(sl," armb",_compare_without_first_ch,&f)); - test_eq(f, 0); - test_eq(7, smartlist_bsearch_idx(sl," zzzz",_compare_without_first_ch,&f)); - test_eq(f, 0); - } - - /* Test reverse() and pop_last() */ - smartlist_reverse(sl); - cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); - test_streq(cp_alloc,"the,router,onion,nickm,by,arma,and"); - tor_free(cp_alloc); - cp_alloc = smartlist_pop_last(sl); - test_streq(cp_alloc, "and"); - tor_free(cp_alloc); - test_eq(smartlist_len(sl), 6); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - cp_alloc = smartlist_pop_last(sl); - test_eq(cp_alloc, NULL); - - /* Test uniq() */ - smartlist_split_string(sl, - "50,noon,radar,a,man,a,plan,a,canal,panama,radar,noon,50", - ",", 0, 0); - smartlist_sort(sl, _compare_strs); - smartlist_uniq(sl, _compare_strs, _tor_free); - cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); - test_streq(cp_alloc, "50,a,canal,man,noon,panama,plan,radar"); - tor_free(cp_alloc); - - /* Test string_isin and isin_case and num_isin */ - test_assert(smartlist_string_isin(sl, "noon")); - test_assert(!smartlist_string_isin(sl, "noonoon")); - test_assert(smartlist_string_isin_case(sl, "nOOn")); - test_assert(!smartlist_string_isin_case(sl, "nooNooN")); - test_assert(smartlist_string_num_isin(sl, 50)); - test_assert(!smartlist_string_num_isin(sl, 60)); - - /* Test smartlist_choose */ - { - int i; - int allsame = 1; - int allin = 1; - void *first = smartlist_choose(sl); - test_assert(smartlist_isin(sl, first)); - for (i = 0; i < 100; ++i) { - void *second = smartlist_choose(sl); - if (second != first) - allsame = 0; - if (!smartlist_isin(sl, second)) - allin = 0; - } - test_assert(!allsame); - test_assert(allin); - } - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_clear(sl); - - /* Test string_remove and remove and join_strings2 */ - smartlist_split_string(sl, - "Some say the Earth will end in ice and some in fire", - " ", 0, 0); - cp = smartlist_get(sl, 4); - test_streq(cp, "will"); - smartlist_add(sl, cp); - smartlist_remove(sl, cp); - tor_free(cp); - cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); - test_streq(cp_alloc, "Some,say,the,Earth,fire,end,in,ice,and,some,in"); - tor_free(cp_alloc); - smartlist_string_remove(sl, "in"); - cp_alloc = smartlist_join_strings2(sl, "+XX", 1, 0, &sz); - test_streq(cp_alloc, "Some+say+the+Earth+fire+end+some+ice+and"); - test_eq((int)sz, 40); - - done: - - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_free(sl); - tor_free(cp_alloc); -} - -/** Run unit tests for smartlist set manipulation functions. */ -static void -test_util_smartlist_overlap(void) -{ - smartlist_t *sl = smartlist_create(); - smartlist_t *ints = smartlist_create(); - smartlist_t *odds = smartlist_create(); - smartlist_t *evens = smartlist_create(); - smartlist_t *primes = smartlist_create(); - int i; - for (i=1; i < 10; i += 2) - smartlist_add(odds, (void*)(uintptr_t)i); - for (i=0; i < 10; i += 2) - smartlist_add(evens, (void*)(uintptr_t)i); - - /* add_all */ - smartlist_add_all(ints, odds); - smartlist_add_all(ints, evens); - test_eq(smartlist_len(ints), 10); - - smartlist_add(primes, (void*)2); - smartlist_add(primes, (void*)3); - smartlist_add(primes, (void*)5); - smartlist_add(primes, (void*)7); - - /* overlap */ - test_assert(smartlist_overlap(ints, odds)); - test_assert(smartlist_overlap(odds, primes)); - test_assert(smartlist_overlap(evens, primes)); - test_assert(!smartlist_overlap(odds, evens)); - - /* intersect */ - smartlist_add_all(sl, odds); - smartlist_intersect(sl, primes); - test_eq(smartlist_len(sl), 3); - test_assert(smartlist_isin(sl, (void*)3)); - test_assert(smartlist_isin(sl, (void*)5)); - test_assert(smartlist_isin(sl, (void*)7)); - - /* subtract */ - smartlist_add_all(sl, primes); - smartlist_subtract(sl, odds); - test_eq(smartlist_len(sl), 1); - test_assert(smartlist_isin(sl, (void*)2)); - - done: - smartlist_free(odds); - smartlist_free(evens); - smartlist_free(ints); - smartlist_free(primes); - smartlist_free(sl); -} - -/** Run unit tests for smartlist-of-digests functions. */ -static void -test_util_smartlist_digests(void) -{ - smartlist_t *sl = smartlist_create(); - - /* digest_isin. */ - smartlist_add(sl, tor_memdup("AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN)); - smartlist_add(sl, tor_memdup("\00090AAB2AAAAaasdAAAAA", DIGEST_LEN)); - smartlist_add(sl, tor_memdup("\00090AAB2AAAAaasdAAAAA", DIGEST_LEN)); - test_eq(0, smartlist_digest_isin(NULL, "AAAAAAAAAAAAAAAAAAAA")); - test_assert(smartlist_digest_isin(sl, "AAAAAAAAAAAAAAAAAAAA")); - test_assert(smartlist_digest_isin(sl, "\00090AAB2AAAAaasdAAAAA")); - test_eq(0, smartlist_digest_isin(sl, "\00090AAB2AAABaasdAAAAA")); - - /* sort digests */ - smartlist_sort_digests(sl); - test_memeq(smartlist_get(sl, 0), "\00090AAB2AAAAaasdAAAAA", DIGEST_LEN); - test_memeq(smartlist_get(sl, 1), "\00090AAB2AAAAaasdAAAAA", DIGEST_LEN); - test_memeq(smartlist_get(sl, 2), "AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN); - test_eq(3, smartlist_len(sl)); - - /* uniq_digests */ - smartlist_uniq_digests(sl); - test_eq(2, smartlist_len(sl)); - test_memeq(smartlist_get(sl, 0), "\00090AAB2AAAAaasdAAAAA", DIGEST_LEN); - test_memeq(smartlist_get(sl, 1), "AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN); - - done: - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_free(sl); -} - -/** Run unit tests for concatenate-a-smartlist-of-strings functions. */ -static void -test_util_smartlist_join(void) -{ - smartlist_t *sl = smartlist_create(); - smartlist_t *sl2 = smartlist_create(), *sl3 = smartlist_create(), - *sl4 = smartlist_create(); - char *joined=NULL; - /* unique, sorted. */ - smartlist_split_string(sl, - "Abashments Ambush Anchorman Bacon Banks Borscht " - "Bunks Inhumane Insurance Knish Know Manners " - "Maraschinos Stamina Sunbonnets Unicorns Wombats", - " ", 0, 0); - /* non-unique, sorted. */ - smartlist_split_string(sl2, - "Ambush Anchorman Anchorman Anemias Anemias Bacon " - "Crossbowmen Inhumane Insurance Knish Know Manners " - "Manners Maraschinos Wombats Wombats Work", - " ", 0, 0); - SMARTLIST_FOREACH_JOIN(sl, char *, cp1, - sl2, char *, cp2, - strcmp(cp1,cp2), - smartlist_add(sl3, cp2)) { - test_streq(cp1, cp2); - smartlist_add(sl4, cp1); - } SMARTLIST_FOREACH_JOIN_END(cp1, cp2); - - SMARTLIST_FOREACH(sl3, const char *, cp, - test_assert(smartlist_isin(sl2, cp) && - !smartlist_string_isin(sl, cp))); - SMARTLIST_FOREACH(sl4, const char *, cp, - test_assert(smartlist_isin(sl, cp) && - smartlist_string_isin(sl2, cp))); - joined = smartlist_join_strings(sl3, ",", 0, NULL); - test_streq(joined, "Anemias,Anemias,Crossbowmen,Work"); - tor_free(joined); - joined = smartlist_join_strings(sl4, ",", 0, NULL); - test_streq(joined, "Ambush,Anchorman,Anchorman,Bacon,Inhumane,Insurance," - "Knish,Know,Manners,Manners,Maraschinos,Wombats,Wombats"); - tor_free(joined); - - done: - smartlist_free(sl4); - smartlist_free(sl3); - SMARTLIST_FOREACH(sl2, char *, cp, tor_free(cp)); - smartlist_free(sl2); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - smartlist_free(sl); - tor_free(joined); -} - -/** Run unit tests for bitarray code */ -static void -test_util_bitarray(void) -{ - bitarray_t *ba = NULL; - int i, j, ok=1; - - ba = bitarray_init_zero(1); - test_assert(ba); - test_assert(! bitarray_is_set(ba, 0)); - bitarray_set(ba, 0); - test_assert(bitarray_is_set(ba, 0)); - bitarray_clear(ba, 0); - test_assert(! bitarray_is_set(ba, 0)); - bitarray_free(ba); - - ba = bitarray_init_zero(1023); - for (i = 1; i < 64; ) { - for (j = 0; j < 1023; ++j) { - if (j % i) - bitarray_set(ba, j); - else - bitarray_clear(ba, j); - } - for (j = 0; j < 1023; ++j) { - if (!bool_eq(bitarray_is_set(ba, j), j%i)) - ok = 0; - } - test_assert(ok); - if (i < 7) - ++i; - else if (i == 28) - i = 32; - else - i += 7; - } - - done: - if (ba) - bitarray_free(ba); -} - -/** Run unit tests for digest set code (implemented as a hashtable or as a - * bloom filter) */ -static void -test_util_digestset(void) -{ - smartlist_t *included = smartlist_create(); - char d[DIGEST_LEN]; - int i; - int ok = 1; - int false_positives = 0; - digestset_t *set = NULL; - - for (i = 0; i < 1000; ++i) { - crypto_rand(d, DIGEST_LEN); - smartlist_add(included, tor_memdup(d, DIGEST_LEN)); - } - set = digestset_new(1000); - SMARTLIST_FOREACH(included, const char *, cp, - if (digestset_isin(set, cp)) - ok = 0); - test_assert(ok); - SMARTLIST_FOREACH(included, const char *, cp, - digestset_add(set, cp)); - SMARTLIST_FOREACH(included, const char *, cp, - if (!digestset_isin(set, cp)) - ok = 0); - test_assert(ok); - for (i = 0; i < 1000; ++i) { - crypto_rand(d, DIGEST_LEN); - if (digestset_isin(set, d)) - ++false_positives; - } - test_assert(false_positives < 50); /* Should be far lower. */ - - done: - if (set) - digestset_free(set); - SMARTLIST_FOREACH(included, char *, cp, tor_free(cp)); - smartlist_free(included); -} - -/** mutex for thread test to stop the threads hitting data at the same time. */ -static tor_mutex_t *_thread_test_mutex = NULL; -/** mutexes for the thread test to make sure that the threads have to - * interleave somewhat. */ -static tor_mutex_t *_thread_test_start1 = NULL, - *_thread_test_start2 = NULL; -/** Shared strmap for the thread test. */ -static strmap_t *_thread_test_strmap = NULL; -/** The name of thread1 for the thread test */ -static char *_thread1_name = NULL; -/** The name of thread2 for the thread test */ -static char *_thread2_name = NULL; - -static void _thread_test_func(void* _s) ATTR_NORETURN; - -/** How many iterations have the threads in the unit test run? */ -static int t1_count = 0, t2_count = 0; - -/** Helper function for threading unit tests: This function runs in a - * subthread. It grabs its own mutex (start1 or start2) to make sure that it - * should start, then it repeatedly alters _test_thread_strmap protected by - * _thread_test_mutex. */ -static void -_thread_test_func(void* _s) -{ - char *s = _s; - int i, *count; - tor_mutex_t *m; - char buf[64]; - char **cp; - if (!strcmp(s, "thread 1")) { - m = _thread_test_start1; - cp = &_thread1_name; - count = &t1_count; - } else { - m = _thread_test_start2; - cp = &_thread2_name; - count = &t2_count; - } - tor_mutex_acquire(m); - - tor_snprintf(buf, sizeof(buf), "%lu", tor_get_thread_id()); - *cp = tor_strdup(buf); - - for (i=0; i<10000; ++i) { - tor_mutex_acquire(_thread_test_mutex); - strmap_set(_thread_test_strmap, "last to run", *cp); - ++*count; - tor_mutex_release(_thread_test_mutex); - } - tor_mutex_acquire(_thread_test_mutex); - strmap_set(_thread_test_strmap, s, *cp); - tor_mutex_release(_thread_test_mutex); - - tor_mutex_release(m); - - spawn_exit(); -} - -/** Run unit tests for threading logic. */ -static void -test_util_threads(void) -{ - char *s1 = NULL, *s2 = NULL; - int done = 0, timedout = 0; - time_t started; -#ifndef MS_WINDOWS - struct timeval tv; - tv.tv_sec=0; - tv.tv_usec=10; -#endif -#ifndef TOR_IS_MULTITHREADED - /* Skip this test if we aren't threading. We should be threading most - * everywhere by now. */ - if (1) - return; -#endif - _thread_test_mutex = tor_mutex_new(); - _thread_test_start1 = tor_mutex_new(); - _thread_test_start2 = tor_mutex_new(); - _thread_test_strmap = strmap_new(); - s1 = tor_strdup("thread 1"); - s2 = tor_strdup("thread 2"); - tor_mutex_acquire(_thread_test_start1); - tor_mutex_acquire(_thread_test_start2); - spawn_func(_thread_test_func, s1); - spawn_func(_thread_test_func, s2); - tor_mutex_release(_thread_test_start2); - tor_mutex_release(_thread_test_start1); - started = time(NULL); - while (!done) { - tor_mutex_acquire(_thread_test_mutex); - strmap_assert_ok(_thread_test_strmap); - if (strmap_get(_thread_test_strmap, "thread 1") && - strmap_get(_thread_test_strmap, "thread 2")) { - done = 1; - } else if (time(NULL) > started + 25) { - timedout = done = 1; - } - tor_mutex_release(_thread_test_mutex); -#ifndef MS_WINDOWS - /* Prevent the main thread from starving the worker threads. */ - select(0, NULL, NULL, NULL, &tv); -#endif - } - - tor_mutex_acquire(_thread_test_start1); - tor_mutex_release(_thread_test_start1); - tor_mutex_acquire(_thread_test_start2); - tor_mutex_release(_thread_test_start2); - - tor_mutex_free(_thread_test_mutex); - - if (timedout) { - printf("\nTimed out: %d %d", t1_count, t2_count); - test_assert(strmap_get(_thread_test_strmap, "thread 1")); - test_assert(strmap_get(_thread_test_strmap, "thread 2")); - test_assert(!timedout); - } - - /* different thread IDs. */ - test_assert(strcmp(strmap_get(_thread_test_strmap, "thread 1"), - strmap_get(_thread_test_strmap, "thread 2"))); - test_assert(!strcmp(strmap_get(_thread_test_strmap, "thread 1"), - strmap_get(_thread_test_strmap, "last to run")) || - !strcmp(strmap_get(_thread_test_strmap, "thread 2"), - strmap_get(_thread_test_strmap, "last to run"))); - - done: - tor_free(s1); - tor_free(s2); - tor_free(_thread1_name); - tor_free(_thread2_name); - if (_thread_test_strmap) - strmap_free(_thread_test_strmap, NULL); - if (_thread_test_start1) - tor_mutex_free(_thread_test_start1); - if (_thread_test_start2) - tor_mutex_free(_thread_test_start2); -} - -/** Helper: return a tristate based on comparing two strings. */ -static int -_compare_strings_for_pqueue(const void *s1, const void *s2) -{ - return strcmp((const char*)s1, (const char*)s2); -} - -/** Run unit tests for heap-based priority queue functions. */ -static void -test_util_pqueue(void) -{ - smartlist_t *sl = smartlist_create(); - int (*cmp)(const void *, const void*); -#define OK() smartlist_pqueue_assert_ok(sl, cmp) - - cmp = _compare_strings_for_pqueue; - - smartlist_pqueue_add(sl, cmp, (char*)"cows"); - smartlist_pqueue_add(sl, cmp, (char*)"zebras"); - smartlist_pqueue_add(sl, cmp, (char*)"fish"); - smartlist_pqueue_add(sl, cmp, (char*)"frogs"); - smartlist_pqueue_add(sl, cmp, (char*)"apples"); - smartlist_pqueue_add(sl, cmp, (char*)"squid"); - smartlist_pqueue_add(sl, cmp, (char*)"daschunds"); - smartlist_pqueue_add(sl, cmp, (char*)"eggplants"); - smartlist_pqueue_add(sl, cmp, (char*)"weissbier"); - smartlist_pqueue_add(sl, cmp, (char*)"lobsters"); - smartlist_pqueue_add(sl, cmp, (char*)"roquefort"); - - OK(); - - test_eq(smartlist_len(sl), 11); - test_streq(smartlist_get(sl, 0), "apples"); - test_streq(smartlist_pqueue_pop(sl, cmp), "apples"); - test_eq(smartlist_len(sl), 10); - OK(); - test_streq(smartlist_pqueue_pop(sl, cmp), "cows"); - test_streq(smartlist_pqueue_pop(sl, cmp), "daschunds"); - smartlist_pqueue_add(sl, cmp, (char*)"chinchillas"); - OK(); - smartlist_pqueue_add(sl, cmp, (char*)"fireflies"); - OK(); - test_streq(smartlist_pqueue_pop(sl, cmp), "chinchillas"); - test_streq(smartlist_pqueue_pop(sl, cmp), "eggplants"); - test_streq(smartlist_pqueue_pop(sl, cmp), "fireflies"); - OK(); - test_streq(smartlist_pqueue_pop(sl, cmp), "fish"); - test_streq(smartlist_pqueue_pop(sl, cmp), "frogs"); - test_streq(smartlist_pqueue_pop(sl, cmp), "lobsters"); - test_streq(smartlist_pqueue_pop(sl, cmp), "roquefort"); - OK(); - test_eq(smartlist_len(sl), 3); - test_streq(smartlist_pqueue_pop(sl, cmp), "squid"); - test_streq(smartlist_pqueue_pop(sl, cmp), "weissbier"); - test_streq(smartlist_pqueue_pop(sl, cmp), "zebras"); - test_eq(smartlist_len(sl), 0); - OK(); -#undef OK - - done: - - smartlist_free(sl); -} - -/** Run unit tests for compression functions */ -static void -test_util_gzip(void) -{ - char *buf1=NULL, *buf2=NULL, *buf3=NULL, *cp1, *cp2; - const char *ccp2; - size_t len1, len2; - tor_zlib_state_t *state = NULL; - - buf1 = tor_strdup("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ"); - test_assert(detect_compression_method(buf1, strlen(buf1)) == UNKNOWN_METHOD); - if (is_gzip_supported()) { - test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, - GZIP_METHOD)); - test_assert(buf2); - test_assert(!memcmp(buf2, "\037\213", 2)); /* Gzip magic. */ - test_assert(detect_compression_method(buf2, len1) == GZIP_METHOD); - - test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1, - GZIP_METHOD, 1, LOG_INFO)); - test_assert(buf3); - test_streq(buf1,buf3); - - tor_free(buf2); - tor_free(buf3); - } - - test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, - ZLIB_METHOD)); - test_assert(buf2); - test_assert(!memcmp(buf2, "\x78\xDA", 2)); /* deflate magic. */ - test_assert(detect_compression_method(buf2, len1) == ZLIB_METHOD); - - test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1, - ZLIB_METHOD, 1, LOG_INFO)); - test_assert(buf3); - test_streq(buf1,buf3); - - /* Check whether we can uncompress concatenated, compressed strings. */ - tor_free(buf3); - buf2 = tor_realloc(buf2, len1*2); - memcpy(buf2+len1, buf2, len1); - test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1*2, - ZLIB_METHOD, 1, LOG_INFO)); - test_eq(len2, (strlen(buf1)+1)*2); - test_memeq(buf3, - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ\0" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ\0", - (strlen(buf1)+1)*2); - - tor_free(buf1); - tor_free(buf2); - tor_free(buf3); - - /* Check whether we can uncompress partial strings. */ - buf1 = - tor_strdup("String with low redundancy that won't be compressed much."); - test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, - ZLIB_METHOD)); - tor_assert(len1>16); - /* when we allow an incomplete string, we should succeed.*/ - tor_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, - ZLIB_METHOD, 0, LOG_INFO)); - buf3[len2]='\0'; - tor_assert(len2 > 5); - tor_assert(!strcmpstart(buf1, buf3)); - - /* when we demand a complete string, this must fail. */ - tor_free(buf3); - tor_assert(tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, - ZLIB_METHOD, 1, LOG_INFO)); - tor_assert(!buf3); - - /* Now, try streaming compression. */ - tor_free(buf1); - tor_free(buf2); - tor_free(buf3); - state = tor_zlib_new(1, ZLIB_METHOD); - tor_assert(state); - cp1 = buf1 = tor_malloc(1024); - len1 = 1024; - ccp2 = "ABCDEFGHIJABCDEFGHIJ"; - len2 = 21; - test_assert(tor_zlib_process(state, &cp1, &len1, &ccp2, &len2, 0) - == TOR_ZLIB_OK); - test_eq(len2, 0); /* Make sure we compressed it all. */ - test_assert(cp1 > buf1); - - len2 = 0; - cp2 = cp1; - test_assert(tor_zlib_process(state, &cp1, &len1, &ccp2, &len2, 1) - == TOR_ZLIB_DONE); - test_eq(len2, 0); - test_assert(cp1 > cp2); /* Make sure we really added something. */ - - tor_assert(!tor_gzip_uncompress(&buf3, &len2, buf1, 1024-len1, - ZLIB_METHOD, 1, LOG_WARN)); - test_streq(buf3, "ABCDEFGHIJABCDEFGHIJ"); /*Make sure it compressed right.*/ - - done: - if (state) - tor_zlib_free(state); - tor_free(buf2); - tor_free(buf3); - tor_free(buf1); -} - -/** Run unit tests for string-to-void* map functions */ -static void -test_util_strmap(void) -{ - strmap_t *map; - strmap_iter_t *iter; - const char *k; - void *v; - char *visited = NULL; - smartlist_t *found_keys = NULL; - - map = strmap_new(); - test_assert(map); - test_eq(strmap_size(map), 0); - test_assert(strmap_isempty(map)); - v = strmap_set(map, "K1", (void*)99); - test_eq(v, NULL); - test_assert(!strmap_isempty(map)); - v = strmap_set(map, "K2", (void*)101); - test_eq(v, NULL); - v = strmap_set(map, "K1", (void*)100); - test_eq(v, (void*)99); - test_eq_ptr(strmap_get(map,"K1"), (void*)100); - test_eq_ptr(strmap_get(map,"K2"), (void*)101); - test_eq_ptr(strmap_get(map,"K-not-there"), NULL); - strmap_assert_ok(map); - - v = strmap_remove(map,"K2"); - strmap_assert_ok(map); - test_eq_ptr(v, (void*)101); - test_eq_ptr(strmap_get(map,"K2"), NULL); - test_eq_ptr(strmap_remove(map,"K2"), NULL); - - strmap_set(map, "K2", (void*)101); - strmap_set(map, "K3", (void*)102); - strmap_set(map, "K4", (void*)103); - test_eq(strmap_size(map), 4); - strmap_assert_ok(map); - strmap_set(map, "K5", (void*)104); - strmap_set(map, "K6", (void*)105); - strmap_assert_ok(map); - - /* Test iterator. */ - iter = strmap_iter_init(map); - found_keys = smartlist_create(); - while (!strmap_iter_done(iter)) { - strmap_iter_get(iter,&k,&v); - smartlist_add(found_keys, tor_strdup(k)); - test_eq_ptr(v, strmap_get(map, k)); - - if (!strcmp(k, "K2")) { - iter = strmap_iter_next_rmv(map,iter); - } else { - iter = strmap_iter_next(map,iter); - } - } - - /* Make sure we removed K2, but not the others. */ - test_eq_ptr(strmap_get(map, "K2"), NULL); - test_eq_ptr(strmap_get(map, "K5"), (void*)104); - /* Make sure we visited everyone once */ - smartlist_sort_strings(found_keys); - visited = smartlist_join_strings(found_keys, ":", 0, NULL); - test_streq(visited, "K1:K2:K3:K4:K5:K6"); - - strmap_assert_ok(map); - /* Clean up after ourselves. */ - strmap_free(map, NULL); - map = NULL; - - /* Now try some lc functions. */ - map = strmap_new(); - strmap_set_lc(map,"Ab.C", (void*)1); - test_eq_ptr(strmap_get(map,"ab.c"), (void*)1); - strmap_assert_ok(map); - test_eq_ptr(strmap_get_lc(map,"AB.C"), (void*)1); - test_eq_ptr(strmap_get(map,"AB.C"), NULL); - test_eq_ptr(strmap_remove_lc(map,"aB.C"), (void*)1); - strmap_assert_ok(map); - test_eq_ptr(strmap_get_lc(map,"AB.C"), NULL); - - done: - if (map) - strmap_free(map,NULL); - if (found_keys) { - SMARTLIST_FOREACH(found_keys, char *, cp, tor_free(cp)); - smartlist_free(found_keys); - } - tor_free(visited); -} - -/** Run unit tests for mmap() wrapper functionality. */ -static void -test_util_mmap(void) -{ - char *fname1 = tor_strdup(get_fname("mapped_1")); - char *fname2 = tor_strdup(get_fname("mapped_2")); - char *fname3 = tor_strdup(get_fname("mapped_3")); - const size_t buflen = 17000; - char *buf = tor_malloc(17000); - tor_mmap_t *mapping = NULL; - - crypto_rand(buf, buflen); - - mapping = tor_mmap_file(fname1); - test_assert(! mapping); - - write_str_to_file(fname1, "Short file.", 1); - write_bytes_to_file(fname2, buf, buflen, 1); - write_bytes_to_file(fname3, buf, 16384, 1); - - mapping = tor_mmap_file(fname1); - test_assert(mapping); - test_eq(mapping->size, strlen("Short file.")); - test_streq(mapping->data, "Short file."); -#ifdef MS_WINDOWS - tor_munmap_file(mapping); - mapping = NULL; - test_assert(unlink(fname1) == 0); -#else - /* make sure we can unlink. */ - test_assert(unlink(fname1) == 0); - test_streq(mapping->data, "Short file."); - tor_munmap_file(mapping); - mapping = NULL; -#endif - - /* Now a zero-length file. */ - write_str_to_file(fname1, "", 1); - mapping = tor_mmap_file(fname1); - test_eq(mapping, NULL); - test_eq(ERANGE, errno); - unlink(fname1); - - /* Make sure that we fail to map a no-longer-existent file. */ - mapping = tor_mmap_file(fname1); - test_assert(mapping == NULL); - - /* Now try a big file that stretches across a few pages and isn't aligned */ - mapping = tor_mmap_file(fname2); - test_assert(mapping); - test_eq(mapping->size, buflen); - test_memeq(mapping->data, buf, buflen); - tor_munmap_file(mapping); - mapping = NULL; - - /* Now try a big aligned file. */ - mapping = tor_mmap_file(fname3); - test_assert(mapping); - test_eq(mapping->size, 16384); - test_memeq(mapping->data, buf, 16384); - tor_munmap_file(mapping); - mapping = NULL; - - done: - unlink(fname1); - unlink(fname2); - unlink(fname3); - - tor_free(fname1); - tor_free(fname2); - tor_free(fname3); - tor_free(buf); - - if (mapping) - tor_munmap_file(mapping); -} - -/** Run unit tests for escaping/unescaping data for use by controllers. */ -static void -test_util_control_formats(void) -{ - char *out = NULL; - const char *inp = - "..This is a test\r\nof the emergency \nbroadcast\r\n..system.\r\nZ.\r\n"; - size_t sz; - - sz = read_escaped_data(inp, strlen(inp), &out); - test_streq(out, - ".This is a test\nof the emergency \nbroadcast\n.system.\nZ.\n"); - test_eq(sz, strlen(out)); - - done: - tor_free(out); -} - -static void -test_util_sscanf(void) -{ - unsigned u1, u2, u3; - char s1[10], s2[10], s3[10], ch; - int r; - - r = tor_sscanf("hello world", "hello world"); /* String match: success */ - test_eq(r, 0); - r = tor_sscanf("hello world 3", "hello worlb %u", &u1); /* String fail */ - test_eq(r, 0); - r = tor_sscanf("12345", "%u", &u1); /* Simple number */ - test_eq(r, 1); - test_eq(u1, 12345u); - r = tor_sscanf("", "%u", &u1); /* absent number */ - test_eq(r, 0); - r = tor_sscanf("A", "%u", &u1); /* bogus number */ - test_eq(r, 0); - r = tor_sscanf("4294967295", "%u", &u1); /* UINT32_MAX should work. */ - test_eq(r, 1); - test_eq(u1, 4294967295u); - r = tor_sscanf("4294967296", "%u", &u1); /* Always say -1 at 32 bits. */ - test_eq(r, 0); - r = tor_sscanf("123456", "%2u%u", &u1, &u2); /* Width */ - test_eq(r, 2); - test_eq(u1, 12u); - test_eq(u2, 3456u); - r = tor_sscanf("!12:3:456", "!%2u:%2u:%3u", &u1, &u2, &u3); /* separators */ - test_eq(r, 3); - test_eq(u1, 12u); - test_eq(u2, 3u); - test_eq(u3, 456u); - r = tor_sscanf("12:3:045", "%2u:%2u:%3u", &u1, &u2, &u3); /* 0s */ - test_eq(r, 3); - test_eq(u1, 12u); - test_eq(u2, 3u); - test_eq(u3, 45u); - /* %u does not match space.*/ - r = tor_sscanf("12:3: 45", "%2u:%2u:%3u", &u1, &u2, &u3); - test_eq(r, 2); - /* %u does not match negative numbers. */ - r = tor_sscanf("12:3:-4", "%2u:%2u:%3u", &u1, &u2, &u3); - test_eq(r, 2); - /* Arbitrary amounts of 0-padding are okay */ - r = tor_sscanf("12:03:000000000000000099", "%2u:%2u:%u", &u1, &u2, &u3); - test_eq(r, 3); - test_eq(u1, 12u); - test_eq(u2, 3u); - test_eq(u3, 99u); - - r = tor_sscanf("99% fresh", "%3u%% fresh", &u1); /* percents are scannable.*/ - test_eq(r, 1); - test_eq(u1, 99); - - r = tor_sscanf("hello", "%s", s1); /* %s needs a number. */ - test_eq(r, -1); - - r = tor_sscanf("hello", "%3s%7s", s1, s2); /* %s matches characters. */ - test_eq(r, 2); - test_streq(s1, "hel"); - test_streq(s2, "lo"); - r = tor_sscanf("WD40", "%2s%u", s3, &u1); /* %s%u */ - test_eq(r, 2); - test_streq(s3, "WD"); - test_eq(u1, 40); - r = tor_sscanf("76trombones", "%6u%9s", &u1, s1); /* %u%s */ - test_eq(r, 2); - test_eq(u1, 76); - test_streq(s1, "trombones"); - r = tor_sscanf("hello world", "%9s %9s", s1, s2); /* %s doesn't eat space. */ - test_eq(r, 2); - test_streq(s1, "hello"); - test_streq(s2, "world"); - r = tor_sscanf("hi", "%9s%9s%3s", s1, s2, s3); /* %s can be empty. */ - test_eq(r, 3); - test_streq(s1, "hi"); - test_streq(s2, ""); - test_streq(s3, ""); - - r = tor_sscanf("1.2.3", "%u.%u.%u%c", &u1, &u2, &u3, &ch); - test_eq(r, 3); - r = tor_sscanf("1.2.3 foobar", "%u.%u.%u%c", &u1, &u2, &u3, &ch); - test_eq(r, 4); - - done: - ; -} - -/** Run unit tests for the onion handshake code. */ -static void -test_onion_handshake(void) -{ - /* client-side */ - crypto_dh_env_t *c_dh = NULL; - char c_buf[ONIONSKIN_CHALLENGE_LEN]; - char c_keys[40]; - - /* server-side */ - char s_buf[ONIONSKIN_REPLY_LEN]; - char s_keys[40]; - - /* shared */ - crypto_pk_env_t *pk = NULL; - - pk = pk_generate(0); - - /* client handshake 1. */ - memset(c_buf, 0, ONIONSKIN_CHALLENGE_LEN); - test_assert(! onion_skin_create(pk, &c_dh, c_buf)); - - /* server handshake */ - memset(s_buf, 0, ONIONSKIN_REPLY_LEN); - memset(s_keys, 0, 40); - test_assert(! onion_skin_server_handshake(c_buf, pk, NULL, - s_buf, s_keys, 40)); - - /* client handshake 2 */ - memset(c_keys, 0, 40); - test_assert(! onion_skin_client_handshake(c_dh, s_buf, c_keys, 40)); - - if (memcmp(c_keys, s_keys, 40)) { - puts("Aiiiie"); - exit(1); - } - test_memeq(c_keys, s_keys, 40); - memset(s_buf, 0, 40); - test_memneq(c_keys, s_buf, 40); - - done: - if (c_dh) - crypto_dh_free(c_dh); - if (pk) - crypto_free_pk_env(pk); -} - -/** Run unit tests for router descriptor generation logic. */ -static void -test_dir_format(void) -{ - char buf[8192], buf2[8192]; - char platform[256]; - char fingerprint[FINGERPRINT_LEN+1]; - char *pk1_str = NULL, *pk2_str = NULL, *pk3_str = NULL, *cp; - size_t pk1_str_len, pk2_str_len, pk3_str_len; - routerinfo_t *r1=NULL, *r2=NULL; - crypto_pk_env_t *pk1 = NULL, *pk2 = NULL, *pk3 = NULL; - routerinfo_t *rp1 = NULL; - addr_policy_t *ex1, *ex2; - routerlist_t *dir1 = NULL, *dir2 = NULL; - tor_version_t ver1; - - pk1 = pk_generate(0); - pk2 = pk_generate(1); - pk3 = pk_generate(2); - - test_assert( is_legal_nickname("a")); - test_assert(!is_legal_nickname("")); - test_assert(!is_legal_nickname("abcdefghijklmnopqrst")); /* 20 chars */ - test_assert(!is_legal_nickname("hyphen-")); /* bad char */ - test_assert( is_legal_nickname("abcdefghijklmnopqrs")); /* 19 chars */ - test_assert(!is_legal_nickname("$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA")); - /* valid */ - test_assert( is_legal_nickname_or_hexdigest( - "$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA")); - test_assert( is_legal_nickname_or_hexdigest( - "$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA=fred")); - test_assert( is_legal_nickname_or_hexdigest( - "$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA~fred")); - /* too short */ - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")); - /* illegal char */ - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAzAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")); - /* hex part too long */ - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")); - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=fred")); - /* Bad nickname */ - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")); - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~")); - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~hyphen-")); - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~" - "abcdefghijklmnoppqrst")); - /* Bad extra char. */ - test_assert(!is_legal_nickname_or_hexdigest( - "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA!")); - test_assert(is_legal_nickname_or_hexdigest("xyzzy")); - test_assert(is_legal_nickname_or_hexdigest("abcdefghijklmnopqrs")); - test_assert(!is_legal_nickname_or_hexdigest("abcdefghijklmnopqrst")); - - get_platform_str(platform, sizeof(platform)); - r1 = tor_malloc_zero(sizeof(routerinfo_t)); - r1->address = tor_strdup("18.244.0.1"); - r1->addr = 0xc0a80001u; /* 192.168.0.1 */ - r1->cache_info.published_on = 0; - r1->or_port = 9000; - r1->dir_port = 9003; - r1->onion_pkey = crypto_pk_dup_key(pk1); - r1->identity_pkey = crypto_pk_dup_key(pk2); - r1->bandwidthrate = 1000; - r1->bandwidthburst = 5000; - r1->bandwidthcapacity = 10000; - r1->exit_policy = NULL; - r1->nickname = tor_strdup("Magri"); - r1->platform = tor_strdup(platform); - - ex1 = tor_malloc_zero(sizeof(addr_policy_t)); - ex2 = tor_malloc_zero(sizeof(addr_policy_t)); - ex1->policy_type = ADDR_POLICY_ACCEPT; - tor_addr_from_ipv4h(&ex1->addr, 0); - ex1->maskbits = 0; - ex1->prt_min = ex1->prt_max = 80; - ex2->policy_type = ADDR_POLICY_REJECT; - tor_addr_from_ipv4h(&ex2->addr, 18<<24); - ex2->maskbits = 8; - ex2->prt_min = ex2->prt_max = 24; - r2 = tor_malloc_zero(sizeof(routerinfo_t)); - r2->address = tor_strdup("1.1.1.1"); - r2->addr = 0x0a030201u; /* 10.3.2.1 */ - r2->platform = tor_strdup(platform); - r2->cache_info.published_on = 5; - r2->or_port = 9005; - r2->dir_port = 0; - r2->onion_pkey = crypto_pk_dup_key(pk2); - r2->identity_pkey = crypto_pk_dup_key(pk1); - r2->bandwidthrate = r2->bandwidthburst = r2->bandwidthcapacity = 3000; - r2->exit_policy = smartlist_create(); - smartlist_add(r2->exit_policy, ex2); - smartlist_add(r2->exit_policy, ex1); - r2->nickname = tor_strdup("Fred"); - - test_assert(!crypto_pk_write_public_key_to_string(pk1, &pk1_str, - &pk1_str_len)); - test_assert(!crypto_pk_write_public_key_to_string(pk2 , &pk2_str, - &pk2_str_len)); - test_assert(!crypto_pk_write_public_key_to_string(pk3 , &pk3_str, - &pk3_str_len)); - - memset(buf, 0, 2048); - test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); - - strlcpy(buf2, "router Magri 18.244.0.1 9000 0 9003\n" - "platform Tor "VERSION" on ", sizeof(buf2)); - strlcat(buf2, get_uname(), sizeof(buf2)); - strlcat(buf2, "\n" - "opt protocols Link 1 2 Circuit 1\n" - "published 1970-01-01 00:00:00\n" - "opt fingerprint ", sizeof(buf2)); - test_assert(!crypto_pk_get_fingerprint(pk2, fingerprint, 1)); - strlcat(buf2, fingerprint, sizeof(buf2)); - strlcat(buf2, "\nuptime 0\n" - /* XXX the "0" above is hard-coded, but even if we made it reflect - * uptime, that still wouldn't make it right, because the two - * descriptors might be made on different seconds... hm. */ - "bandwidth 1000 5000 10000\n" - "opt extra-info-digest 0000000000000000000000000000000000000000\n" - "onion-key\n", sizeof(buf2)); - strlcat(buf2, pk1_str, sizeof(buf2)); - strlcat(buf2, "signing-key\n", sizeof(buf2)); - strlcat(buf2, pk2_str, sizeof(buf2)); - strlcat(buf2, "opt hidden-service-dir\n", sizeof(buf2)); - strlcat(buf2, "reject *:*\nrouter-signature\n", sizeof(buf2)); - buf[strlen(buf2)] = '\0'; /* Don't compare the sig; it's never the same - * twice */ - - test_streq(buf, buf2); - - test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); - cp = buf; - rp1 = router_parse_entry_from_string((const char*)cp,NULL,1,0,NULL); - test_assert(rp1); - test_streq(rp1->address, r1->address); - test_eq(rp1->or_port, r1->or_port); - //test_eq(rp1->dir_port, r1->dir_port); - test_eq(rp1->bandwidthrate, r1->bandwidthrate); - test_eq(rp1->bandwidthburst, r1->bandwidthburst); - test_eq(rp1->bandwidthcapacity, r1->bandwidthcapacity); - test_assert(crypto_pk_cmp_keys(rp1->onion_pkey, pk1) == 0); - test_assert(crypto_pk_cmp_keys(rp1->identity_pkey, pk2) == 0); - //test_assert(rp1->exit_policy == NULL); - -#if 0 - /* XXX Once we have exit policies, test this again. XXX */ - strlcpy(buf2, "router tor.tor.tor 9005 0 0 3000\n", sizeof(buf2)); - strlcat(buf2, pk2_str, sizeof(buf2)); - strlcat(buf2, "signing-key\n", sizeof(buf2)); - strlcat(buf2, pk1_str, sizeof(buf2)); - strlcat(buf2, "accept *:80\nreject 18.*:24\n\n", sizeof(buf2)); - test_assert(router_dump_router_to_string(buf, 2048, &r2, pk2)>0); - test_streq(buf, buf2); - - cp = buf; - rp2 = router_parse_entry_from_string(&cp,1); - test_assert(rp2); - test_streq(rp2->address, r2.address); - test_eq(rp2->or_port, r2.or_port); - test_eq(rp2->dir_port, r2.dir_port); - test_eq(rp2->bandwidth, r2.bandwidth); - test_assert(crypto_pk_cmp_keys(rp2->onion_pkey, pk2) == 0); - test_assert(crypto_pk_cmp_keys(rp2->identity_pkey, pk1) == 0); - test_eq(rp2->exit_policy->policy_type, EXIT_POLICY_ACCEPT); - test_streq(rp2->exit_policy->string, "accept *:80"); - test_streq(rp2->exit_policy->address, "*"); - test_streq(rp2->exit_policy->port, "80"); - test_eq(rp2->exit_policy->next->policy_type, EXIT_POLICY_REJECT); - test_streq(rp2->exit_policy->next->string, "reject 18.*:24"); - test_streq(rp2->exit_policy->next->address, "18.*"); - test_streq(rp2->exit_policy->next->port, "24"); - test_assert(rp2->exit_policy->next->next == NULL); - - /* Okay, now for the directories. */ - { - fingerprint_list = smartlist_create(); - crypto_pk_get_fingerprint(pk2, buf, 1); - add_fingerprint_to_dir("Magri", buf, fingerprint_list); - crypto_pk_get_fingerprint(pk1, buf, 1); - add_fingerprint_to_dir("Fred", buf, fingerprint_list); - } - - { - char d[DIGEST_LEN]; - const char *m; - /* XXXX NM re-enable. */ - /* Make sure routers aren't too far in the past any more. */ - r1->cache_info.published_on = time(NULL); - r2->cache_info.published_on = time(NULL)-3*60*60; - test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); - test_eq(dirserv_add_descriptor(buf,&m,""), ROUTER_ADDED_NOTIFY_GENERATOR); - test_assert(router_dump_router_to_string(buf, 2048, r2, pk1)>0); - test_eq(dirserv_add_descriptor(buf,&m,""), ROUTER_ADDED_NOTIFY_GENERATOR); - get_options()->Nickname = tor_strdup("DirServer"); - test_assert(!dirserv_dump_directory_to_string(&cp,pk3, 0)); - crypto_pk_get_digest(pk3, d); - test_assert(!router_parse_directory(cp)); - test_eq(2, smartlist_len(dir1->routers)); - tor_free(cp); - } -#endif - dirserv_free_fingerprint_list(); - - /* Try out version parsing functionality */ - test_eq(0, tor_version_parse("0.3.4pre2-cvs", &ver1)); - test_eq(0, ver1.major); - test_eq(3, ver1.minor); - test_eq(4, ver1.micro); - test_eq(VER_PRE, ver1.status); - test_eq(2, ver1.patchlevel); - test_eq(0, tor_version_parse("0.3.4rc1", &ver1)); - test_eq(0, ver1.major); - test_eq(3, ver1.minor); - test_eq(4, ver1.micro); - test_eq(VER_RC, ver1.status); - test_eq(1, ver1.patchlevel); - test_eq(0, tor_version_parse("1.3.4", &ver1)); - test_eq(1, ver1.major); - test_eq(3, ver1.minor); - test_eq(4, ver1.micro); - test_eq(VER_RELEASE, ver1.status); - test_eq(0, ver1.patchlevel); - test_eq(0, tor_version_parse("1.3.4.999", &ver1)); - test_eq(1, ver1.major); - test_eq(3, ver1.minor); - test_eq(4, ver1.micro); - test_eq(VER_RELEASE, ver1.status); - test_eq(999, ver1.patchlevel); - test_eq(0, tor_version_parse("0.1.2.4-alpha", &ver1)); - test_eq(0, ver1.major); - test_eq(1, ver1.minor); - test_eq(2, ver1.micro); - test_eq(4, ver1.patchlevel); - test_eq(VER_RELEASE, ver1.status); - test_streq("alpha", ver1.status_tag); - test_eq(0, tor_version_parse("0.1.2.4", &ver1)); - test_eq(0, ver1.major); - test_eq(1, ver1.minor); - test_eq(2, ver1.micro); - test_eq(4, ver1.patchlevel); - test_eq(VER_RELEASE, ver1.status); - test_streq("", ver1.status_tag); - -#define test_eq_vs(vs1, vs2) test_eq_type(version_status_t, "%d", (vs1), (vs2)) -#define test_v_i_o(val, ver, lst) \ - test_eq_vs(val, tor_version_is_obsolete(ver, lst)) - - /* make sure tor_version_is_obsolete() works */ - test_v_i_o(VS_OLD, "0.0.1", "Tor 0.0.2"); - test_v_i_o(VS_OLD, "0.0.1", "0.0.2, Tor 0.0.3"); - test_v_i_o(VS_OLD, "0.0.1", "0.0.2,Tor 0.0.3"); - test_v_i_o(VS_OLD, "0.0.1","0.0.3,BetterTor 0.0.1"); - test_v_i_o(VS_RECOMMENDED, "0.0.2", "Tor 0.0.2,Tor 0.0.3"); - test_v_i_o(VS_NEW_IN_SERIES, "0.0.2", "Tor 0.0.2pre1,Tor 0.0.3"); - test_v_i_o(VS_OLD, "0.0.2", "Tor 0.0.2.1,Tor 0.0.3"); - test_v_i_o(VS_NEW, "0.1.0", "Tor 0.0.2,Tor 0.0.3"); - test_v_i_o(VS_RECOMMENDED, "0.0.7rc2", "0.0.7,Tor 0.0.7rc2,Tor 0.0.8"); - test_v_i_o(VS_OLD, "0.0.5.0", "0.0.5.1-cvs"); - test_v_i_o(VS_NEW_IN_SERIES, "0.0.5.1-cvs", "0.0.5, 0.0.6"); - /* Not on list, but newer than any in same series. */ - test_v_i_o(VS_NEW_IN_SERIES, "0.1.0.3", - "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); - /* Series newer than any on list. */ - test_v_i_o(VS_NEW, "0.1.2.3", "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); - /* Series older than any on list. */ - test_v_i_o(VS_OLD, "0.0.1.3", "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); - /* Not on list, not newer than any on same series. */ - test_v_i_o(VS_UNRECOMMENDED, "0.1.0.1", - "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); - /* On list, not newer than any on same series. */ - test_v_i_o(VS_UNRECOMMENDED, - "0.1.0.1", "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); - test_eq(0, tor_version_as_new_as("Tor 0.0.5", "0.0.9pre1-cvs")); - test_eq(1, tor_version_as_new_as( - "Tor 0.0.8 on Darwin 64-121-192-100.c3-0." - "sfpo-ubr1.sfrn-sfpo.ca.cable.rcn.com Power Macintosh", - "0.0.8rc2")); - test_eq(0, tor_version_as_new_as( - "Tor 0.0.8 on Darwin 64-121-192-100.c3-0." - "sfpo-ubr1.sfrn-sfpo.ca.cable.rcn.com Power Macintosh", "0.0.8.2")); - - /* Now try svn revisions. */ - test_eq(1, tor_version_as_new_as("Tor 0.2.1.0-dev (r100)", - "Tor 0.2.1.0-dev (r99)")); - test_eq(1, tor_version_as_new_as("Tor 0.2.1.0-dev (r100) on Banana Jr", - "Tor 0.2.1.0-dev (r99) on Hal 9000")); - test_eq(1, tor_version_as_new_as("Tor 0.2.1.0-dev (r100)", - "Tor 0.2.1.0-dev on Colossus")); - test_eq(0, tor_version_as_new_as("Tor 0.2.1.0-dev (r99)", - "Tor 0.2.1.0-dev (r100)")); - test_eq(0, tor_version_as_new_as("Tor 0.2.1.0-dev (r99) on MCP", - "Tor 0.2.1.0-dev (r100) on AM")); - test_eq(0, tor_version_as_new_as("Tor 0.2.1.0-dev", - "Tor 0.2.1.0-dev (r99)")); - test_eq(1, tor_version_as_new_as("Tor 0.2.1.1", - "Tor 0.2.1.0-dev (r99)")); - done: - if (r1) - routerinfo_free(r1); - if (r2) - routerinfo_free(r2); - - tor_free(pk1_str); - tor_free(pk2_str); - tor_free(pk3_str); - if (pk1) crypto_free_pk_env(pk1); - if (pk2) crypto_free_pk_env(pk2); - if (pk3) crypto_free_pk_env(pk3); - if (rp1) routerinfo_free(rp1); - tor_free(dir1); /* XXXX And more !*/ - tor_free(dir2); /* And more !*/ -} - -/** Run unit tests for misc directory functions. */ -static void -test_dirutil(void) -{ - smartlist_t *sl = smartlist_create(); - fp_pair_t *pair; - - dir_split_resource_into_fingerprint_pairs( - /* Two pairs, out of order, with one duplicate. */ - "73656372657420646174612E0000000000FFFFFF-" - "557365204145532d32353620696e73746561642e+" - "73656372657420646174612E0000000000FFFFFF-" - "557365204145532d32353620696e73746561642e+" - "48657861646563696d616c2069736e277420736f-" - "676f6f6420666f7220686964696e6720796f7572.z", sl); - - test_eq(smartlist_len(sl), 2); - pair = smartlist_get(sl, 0); - test_memeq(pair->first, "Hexadecimal isn't so", DIGEST_LEN); - test_memeq(pair->second, "good for hiding your", DIGEST_LEN); - pair = smartlist_get(sl, 1); - test_memeq(pair->first, "secret data.\0\0\0\0\0\xff\xff\xff", DIGEST_LEN); - test_memeq(pair->second, "Use AES-256 instead.", DIGEST_LEN); - - done: - SMARTLIST_FOREACH(sl, fp_pair_t *, pair, tor_free(pair)); - smartlist_free(sl); -} - -extern const char AUTHORITY_CERT_1[]; -extern const char AUTHORITY_SIGNKEY_1[]; -extern const char AUTHORITY_CERT_2[]; -extern const char AUTHORITY_SIGNKEY_2[]; -extern const char AUTHORITY_CERT_3[]; -extern const char AUTHORITY_SIGNKEY_3[]; - -/** Helper: Test that two networkstatus_voter_info_t do in fact represent the - * same voting authority, and that they do in fact have all the same - * information. */ -static void -test_same_voter(networkstatus_voter_info_t *v1, - networkstatus_voter_info_t *v2) -{ - test_streq(v1->nickname, v2->nickname); - test_memeq(v1->identity_digest, v2->identity_digest, DIGEST_LEN); - test_streq(v1->address, v2->address); - test_eq(v1->addr, v2->addr); - test_eq(v1->dir_port, v2->dir_port); - test_eq(v1->or_port, v2->or_port); - test_streq(v1->contact, v2->contact); - test_memeq(v1->vote_digest, v2->vote_digest, DIGEST_LEN); - done: - ; -} - -/** Run unit tests for getting the median of a list. */ -static void -test_util_order_functions(void) -{ - int lst[25], n = 0; - // int a=12,b=24,c=25,d=60,e=77; - -#define median() median_int(lst, n) - - lst[n++] = 12; - test_eq(12, median()); /* 12 */ - lst[n++] = 77; - //smartlist_shuffle(sl); - test_eq(12, median()); /* 12, 77 */ - lst[n++] = 77; - //smartlist_shuffle(sl); - test_eq(77, median()); /* 12, 77, 77 */ - lst[n++] = 24; - test_eq(24, median()); /* 12,24,77,77 */ - lst[n++] = 60; - lst[n++] = 12; - lst[n++] = 25; - //smartlist_shuffle(sl); - test_eq(25, median()); /* 12,12,24,25,60,77,77 */ -#undef median - - done: - ; -} - -/** Helper: Make a new routerinfo containing the right information for a - * given vote_routerstatus_t. */ -static routerinfo_t * -generate_ri_from_rs(const vote_routerstatus_t *vrs) -{ - routerinfo_t *r; - const routerstatus_t *rs = &vrs->status; - static time_t published = 0; - - r = tor_malloc_zero(sizeof(routerinfo_t)); - memcpy(r->cache_info.identity_digest, rs->identity_digest, DIGEST_LEN); - memcpy(r->cache_info.signed_descriptor_digest, rs->descriptor_digest, - DIGEST_LEN); - r->cache_info.do_not_cache = 1; - r->cache_info.routerlist_index = -1; - r->cache_info.signed_descriptor_body = - tor_strdup("123456789012345678901234567890123"); - r->cache_info.signed_descriptor_len = - strlen(r->cache_info.signed_descriptor_body); - r->exit_policy = smartlist_create(); - r->cache_info.published_on = ++published + time(NULL); - return r; -} - -/** Run unit tests for generating and parsing V3 consensus networkstatus - * documents. */ -static void -test_v3_networkstatus(void) -{ - authority_cert_t *cert1=NULL, *cert2=NULL, *cert3=NULL; - crypto_pk_env_t *sign_skey_1=NULL, *sign_skey_2=NULL, *sign_skey_3=NULL; - crypto_pk_env_t *sign_skey_leg1=NULL; - const char *msg=NULL; - - time_t now = time(NULL); - networkstatus_voter_info_t *voter; - networkstatus_t *vote=NULL, *v1=NULL, *v2=NULL, *v3=NULL, *con=NULL; - vote_routerstatus_t *vrs; - routerstatus_t *rs; - char *v1_text=NULL, *v2_text=NULL, *v3_text=NULL, *consensus_text=NULL, *cp; - smartlist_t *votes = smartlist_create(); - - /* For generating the two other consensuses. */ - char *detached_text1=NULL, *detached_text2=NULL; - char *consensus_text2=NULL, *consensus_text3=NULL; - networkstatus_t *con2=NULL, *con3=NULL; - ns_detached_signatures_t *dsig1=NULL, *dsig2=NULL; - - /* Parse certificates and keys. */ - cert1 = authority_cert_parse_from_string(AUTHORITY_CERT_1, NULL); - test_assert(cert1); - test_assert(cert1->is_cross_certified); - cert2 = authority_cert_parse_from_string(AUTHORITY_CERT_2, NULL); - test_assert(cert2); - cert3 = authority_cert_parse_from_string(AUTHORITY_CERT_3, NULL); - test_assert(cert3); - sign_skey_1 = crypto_new_pk_env(); - sign_skey_2 = crypto_new_pk_env(); - sign_skey_3 = crypto_new_pk_env(); - sign_skey_leg1 = pk_generate(4); - - test_assert(!crypto_pk_read_private_key_from_string(sign_skey_1, - AUTHORITY_SIGNKEY_1,-1)); - test_assert(!crypto_pk_read_private_key_from_string(sign_skey_2, - AUTHORITY_SIGNKEY_2,-1)); - test_assert(!crypto_pk_read_private_key_from_string(sign_skey_3, - AUTHORITY_SIGNKEY_3,-1)); - - test_assert(!crypto_pk_cmp_keys(sign_skey_1, cert1->signing_key)); - test_assert(!crypto_pk_cmp_keys(sign_skey_2, cert2->signing_key)); - - /* - * Set up a vote; generate it; try to parse it. - */ - vote = tor_malloc_zero(sizeof(networkstatus_t)); - vote->type = NS_TYPE_VOTE; - vote->published = now; - vote->valid_after = now+1000; - vote->fresh_until = now+2000; - vote->valid_until = now+3000; - vote->vote_seconds = 100; - vote->dist_seconds = 200; - vote->supported_methods = smartlist_create(); - smartlist_split_string(vote->supported_methods, "1 2 3", NULL, 0, -1); - vote->client_versions = tor_strdup("0.1.2.14,0.1.2.15"); - vote->server_versions = tor_strdup("0.1.2.14,0.1.2.15,0.1.2.16"); - vote->known_flags = smartlist_create(); - smartlist_split_string(vote->known_flags, - "Authority Exit Fast Guard Running Stable V2Dir Valid", - 0, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - vote->voters = smartlist_create(); - voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); - voter->nickname = tor_strdup("Voter1"); - voter->address = tor_strdup("1.2.3.4"); - voter->addr = 0x01020304; - voter->dir_port = 80; - voter->or_port = 9000; - voter->contact = tor_strdup("voter@example.com"); - crypto_pk_get_digest(cert1->identity_key, voter->identity_digest); - smartlist_add(vote->voters, voter); - vote->cert = authority_cert_dup(cert1); - vote->routerstatus_list = smartlist_create(); - /* add the first routerstatus. */ - vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); - rs = &vrs->status; - vrs->version = tor_strdup("0.1.2.14"); - rs->published_on = now-1500; - strlcpy(rs->nickname, "router2", sizeof(rs->nickname)); - memset(rs->identity_digest, 3, DIGEST_LEN); - memset(rs->descriptor_digest, 78, DIGEST_LEN); - rs->addr = 0x99008801; - rs->or_port = 443; - rs->dir_port = 8000; - /* all flags but running cleared */ - rs->is_running = 1; - smartlist_add(vote->routerstatus_list, vrs); - test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); - - /* add the second routerstatus. */ - vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); - rs = &vrs->status; - vrs->version = tor_strdup("0.2.0.5"); - rs->published_on = now-1000; - strlcpy(rs->nickname, "router1", sizeof(rs->nickname)); - memset(rs->identity_digest, 5, DIGEST_LEN); - memset(rs->descriptor_digest, 77, DIGEST_LEN); - rs->addr = 0x99009901; - rs->or_port = 443; - rs->dir_port = 0; - rs->is_exit = rs->is_stable = rs->is_fast = rs->is_running = - rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; - smartlist_add(vote->routerstatus_list, vrs); - test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); - - /* add the third routerstatus. */ - vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); - rs = &vrs->status; - vrs->version = tor_strdup("0.1.0.3"); - rs->published_on = now-1000; - strlcpy(rs->nickname, "router3", sizeof(rs->nickname)); - memset(rs->identity_digest, 33, DIGEST_LEN); - memset(rs->descriptor_digest, 79, DIGEST_LEN); - rs->addr = 0xAA009901; - rs->or_port = 400; - rs->dir_port = 9999; - rs->is_authority = rs->is_exit = rs->is_stable = rs->is_fast = - rs->is_running = rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; - smartlist_add(vote->routerstatus_list, vrs); - test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); - - /* add a fourth routerstatus that is not running. */ - vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); - rs = &vrs->status; - vrs->version = tor_strdup("0.1.6.3"); - rs->published_on = now-1000; - strlcpy(rs->nickname, "router4", sizeof(rs->nickname)); - memset(rs->identity_digest, 34, DIGEST_LEN); - memset(rs->descriptor_digest, 48, DIGEST_LEN); - rs->addr = 0xC0000203; - rs->or_port = 500; - rs->dir_port = 1999; - /* Running flag (and others) cleared */ - smartlist_add(vote->routerstatus_list, vrs); - test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); - - /* dump the vote and try to parse it. */ - v1_text = format_networkstatus_vote(sign_skey_1, vote); - test_assert(v1_text); - v1 = networkstatus_parse_vote_from_string(v1_text, NULL, NS_TYPE_VOTE); - test_assert(v1); - - /* Make sure the parsed thing was right. */ - test_eq(v1->type, NS_TYPE_VOTE); - test_eq(v1->published, vote->published); - test_eq(v1->valid_after, vote->valid_after); - test_eq(v1->fresh_until, vote->fresh_until); - test_eq(v1->valid_until, vote->valid_until); - test_eq(v1->vote_seconds, vote->vote_seconds); - test_eq(v1->dist_seconds, vote->dist_seconds); - test_streq(v1->client_versions, vote->client_versions); - test_streq(v1->server_versions, vote->server_versions); - test_assert(v1->voters && smartlist_len(v1->voters)); - voter = smartlist_get(v1->voters, 0); - test_streq(voter->nickname, "Voter1"); - test_streq(voter->address, "1.2.3.4"); - test_eq(voter->addr, 0x01020304); - test_eq(voter->dir_port, 80); - test_eq(voter->or_port, 9000); - test_streq(voter->contact, "voter@example.com"); - test_assert(v1->cert); - test_assert(!crypto_pk_cmp_keys(sign_skey_1, v1->cert->signing_key)); - cp = smartlist_join_strings(v1->known_flags, ":", 0, NULL); - test_streq(cp, "Authority:Exit:Fast:Guard:Running:Stable:V2Dir:Valid"); - tor_free(cp); - test_eq(smartlist_len(v1->routerstatus_list), 4); - /* Check the first routerstatus. */ - vrs = smartlist_get(v1->routerstatus_list, 0); - rs = &vrs->status; - test_streq(vrs->version, "0.1.2.14"); - test_eq(rs->published_on, now-1500); - test_streq(rs->nickname, "router2"); - test_memeq(rs->identity_digest, - "\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3", - DIGEST_LEN); - test_memeq(rs->descriptor_digest, "NNNNNNNNNNNNNNNNNNNN", DIGEST_LEN); - test_eq(rs->addr, 0x99008801); - test_eq(rs->or_port, 443); - test_eq(rs->dir_port, 8000); - test_eq(vrs->flags, U64_LITERAL(16)); // no flags except "running" - /* Check the second routerstatus. */ - vrs = smartlist_get(v1->routerstatus_list, 1); - rs = &vrs->status; - test_streq(vrs->version, "0.2.0.5"); - test_eq(rs->published_on, now-1000); - test_streq(rs->nickname, "router1"); - test_memeq(rs->identity_digest, - "\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5", - DIGEST_LEN); - test_memeq(rs->descriptor_digest, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); - test_eq(rs->addr, 0x99009901); - test_eq(rs->or_port, 443); - test_eq(rs->dir_port, 0); - test_eq(vrs->flags, U64_LITERAL(254)); // all flags except "authority." - - /* Generate second vote. It disagrees on some of the times, - * and doesn't list versions, and knows some crazy flags */ - vote->published = now+1; - vote->fresh_until = now+3005; - vote->dist_seconds = 300; - authority_cert_free(vote->cert); - vote->cert = authority_cert_dup(cert2); - tor_free(vote->client_versions); - tor_free(vote->server_versions); - voter = smartlist_get(vote->voters, 0); - tor_free(voter->nickname); - tor_free(voter->address); - voter->nickname = tor_strdup("Voter2"); - voter->address = tor_strdup("2.3.4.5"); - voter->addr = 0x02030405; - crypto_pk_get_digest(cert2->identity_key, voter->identity_digest); - smartlist_add(vote->known_flags, tor_strdup("MadeOfCheese")); - smartlist_add(vote->known_flags, tor_strdup("MadeOfTin")); - smartlist_sort_strings(vote->known_flags); - vrs = smartlist_get(vote->routerstatus_list, 2); - smartlist_del_keeporder(vote->routerstatus_list, 2); - tor_free(vrs->version); - tor_free(vrs); - vrs = smartlist_get(vote->routerstatus_list, 0); - vrs->status.is_fast = 1; - /* generate and parse. */ - v2_text = format_networkstatus_vote(sign_skey_2, vote); - test_assert(v2_text); - v2 = networkstatus_parse_vote_from_string(v2_text, NULL, NS_TYPE_VOTE); - test_assert(v2); - /* Check that flags come out right.*/ - cp = smartlist_join_strings(v2->known_flags, ":", 0, NULL); - test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:" - "Running:Stable:V2Dir:Valid"); - tor_free(cp); - vrs = smartlist_get(v2->routerstatus_list, 1); - /* 1023 - authority(1) - madeofcheese(16) - madeoftin(32) */ - test_eq(vrs->flags, U64_LITERAL(974)); - - /* Generate the third vote. */ - vote->published = now; - vote->fresh_until = now+2003; - vote->dist_seconds = 250; - authority_cert_free(vote->cert); - vote->cert = authority_cert_dup(cert3); - smartlist_add(vote->supported_methods, tor_strdup("4")); - vote->client_versions = tor_strdup("0.1.2.14,0.1.2.17"); - vote->server_versions = tor_strdup("0.1.2.10,0.1.2.15,0.1.2.16"); - voter = smartlist_get(vote->voters, 0); - tor_free(voter->nickname); - tor_free(voter->address); - voter->nickname = tor_strdup("Voter3"); - voter->address = tor_strdup("3.4.5.6"); - voter->addr = 0x03040506; - crypto_pk_get_digest(cert3->identity_key, voter->identity_digest); - /* This one has a legacy id. */ - memset(voter->legacy_id_digest, (int)'A', DIGEST_LEN); - vrs = smartlist_get(vote->routerstatus_list, 0); - smartlist_del_keeporder(vote->routerstatus_list, 0); - tor_free(vrs->version); - tor_free(vrs); - vrs = smartlist_get(vote->routerstatus_list, 0); - memset(vrs->status.descriptor_digest, (int)'Z', DIGEST_LEN); - test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); - - v3_text = format_networkstatus_vote(sign_skey_3, vote); - test_assert(v3_text); - - v3 = networkstatus_parse_vote_from_string(v3_text, NULL, NS_TYPE_VOTE); - test_assert(v3); - - /* Compute a consensus as voter 3. */ - smartlist_add(votes, v3); - smartlist_add(votes, v1); - smartlist_add(votes, v2); - consensus_text = networkstatus_compute_consensus(votes, 3, - cert3->identity_key, - sign_skey_3, - "AAAAAAAAAAAAAAAAAAAA", - sign_skey_leg1); - test_assert(consensus_text); - con = networkstatus_parse_vote_from_string(consensus_text, NULL, - NS_TYPE_CONSENSUS); - test_assert(con); - //log_notice(LD_GENERAL, "<<%s>>\n<<%s>>\n<<%s>>\n", - // v1_text, v2_text, v3_text); - - /* Check consensus contents. */ - test_assert(con->type == NS_TYPE_CONSENSUS); - test_eq(con->published, 0); /* this field only appears in votes. */ - test_eq(con->valid_after, now+1000); - test_eq(con->fresh_until, now+2003); /* median */ - test_eq(con->valid_until, now+3000); - test_eq(con->vote_seconds, 100); - test_eq(con->dist_seconds, 250); /* median */ - test_streq(con->client_versions, "0.1.2.14"); - test_streq(con->server_versions, "0.1.2.15,0.1.2.16"); - cp = smartlist_join_strings(v2->known_flags, ":", 0, NULL); - test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:" - "Running:Stable:V2Dir:Valid"); - tor_free(cp); - test_eq(4, smartlist_len(con->voters)); /*3 voters, 1 legacy key.*/ - /* The voter id digests should be in this order. */ - test_assert(memcmp(cert2->cache_info.identity_digest, - cert1->cache_info.identity_digest,DIGEST_LEN)<0); - test_assert(memcmp(cert1->cache_info.identity_digest, - cert3->cache_info.identity_digest,DIGEST_LEN)<0); - test_same_voter(smartlist_get(con->voters, 1), - smartlist_get(v2->voters, 0)); - test_same_voter(smartlist_get(con->voters, 2), - smartlist_get(v1->voters, 0)); - test_same_voter(smartlist_get(con->voters, 3), - smartlist_get(v3->voters, 0)); - - test_assert(!con->cert); - test_eq(2, smartlist_len(con->routerstatus_list)); - /* There should be two listed routers: one with identity 3, one with - * identity 5. */ - /* This one showed up in 2 digests. */ - rs = smartlist_get(con->routerstatus_list, 0); - test_memeq(rs->identity_digest, - "\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3", - DIGEST_LEN); - test_memeq(rs->descriptor_digest, "NNNNNNNNNNNNNNNNNNNN", DIGEST_LEN); - test_assert(!rs->is_authority); - test_assert(!rs->is_exit); - test_assert(!rs->is_fast); - test_assert(!rs->is_possible_guard); - test_assert(!rs->is_stable); - test_assert(rs->is_running); /* If it wasn't running it wouldn't be here */ - test_assert(!rs->is_v2_dir); - test_assert(!rs->is_valid); - test_assert(!rs->is_named); - /* XXXX check version */ - - rs = smartlist_get(con->routerstatus_list, 1); - /* This one showed up in 3 digests. Twice with ID 'M', once with 'Z'. */ - test_memeq(rs->identity_digest, - "\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5", - DIGEST_LEN); - test_streq(rs->nickname, "router1"); - test_memeq(rs->descriptor_digest, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); - test_eq(rs->published_on, now-1000); - test_eq(rs->addr, 0x99009901); - test_eq(rs->or_port, 443); - test_eq(rs->dir_port, 0); - test_assert(!rs->is_authority); - test_assert(rs->is_exit); - test_assert(rs->is_fast); - test_assert(rs->is_possible_guard); - test_assert(rs->is_stable); - test_assert(rs->is_running); - test_assert(rs->is_v2_dir); - test_assert(rs->is_valid); - test_assert(!rs->is_named); - /* XXXX check version */ - // x231 - // x213 - - /* Check signatures. the first voter is a pseudo-entry with a legacy key. - * The second one hasn't signed. The fourth one has signed: validate it. */ - voter = smartlist_get(con->voters, 1); - test_assert(!voter->signature); - test_assert(!voter->good_signature); - test_assert(!voter->bad_signature); - - voter = smartlist_get(con->voters, 3); - test_assert(voter->signature); - test_assert(!voter->good_signature); - test_assert(!voter->bad_signature); - test_assert(!networkstatus_check_voter_signature(con, - smartlist_get(con->voters, 3), - cert3)); - test_assert(voter->signature); - test_assert(voter->good_signature); - test_assert(!voter->bad_signature); - - { - const char *msg=NULL; - /* Compute the other two signed consensuses. */ - smartlist_shuffle(votes); - consensus_text2 = networkstatus_compute_consensus(votes, 3, - cert2->identity_key, - sign_skey_2, NULL,NULL); - smartlist_shuffle(votes); - consensus_text3 = networkstatus_compute_consensus(votes, 3, - cert1->identity_key, - sign_skey_1, NULL,NULL); - test_assert(consensus_text2); - test_assert(consensus_text3); - con2 = networkstatus_parse_vote_from_string(consensus_text2, NULL, - NS_TYPE_CONSENSUS); - con3 = networkstatus_parse_vote_from_string(consensus_text3, NULL, - NS_TYPE_CONSENSUS); - test_assert(con2); - test_assert(con3); - - /* All three should have the same digest. */ - test_memeq(con->networkstatus_digest, con2->networkstatus_digest, - DIGEST_LEN); - test_memeq(con->networkstatus_digest, con3->networkstatus_digest, - DIGEST_LEN); - - /* Extract a detached signature from con3. */ - detached_text1 = networkstatus_get_detached_signatures(con3); - tor_assert(detached_text1); - /* Try to parse it. */ - dsig1 = networkstatus_parse_detached_signatures(detached_text1, NULL); - tor_assert(dsig1); - - /* Are parsed values as expected? */ - test_eq(dsig1->valid_after, con3->valid_after); - test_eq(dsig1->fresh_until, con3->fresh_until); - test_eq(dsig1->valid_until, con3->valid_until); - test_memeq(dsig1->networkstatus_digest, con3->networkstatus_digest, - DIGEST_LEN); - test_eq(1, smartlist_len(dsig1->signatures)); - voter = smartlist_get(dsig1->signatures, 0); - test_memeq(voter->identity_digest, cert1->cache_info.identity_digest, - DIGEST_LEN); - - /* Try adding it to con2. */ - detached_text2 = networkstatus_get_detached_signatures(con2); - test_eq(1, networkstatus_add_detached_signatures(con2, dsig1, &msg)); - tor_free(detached_text2); - detached_text2 = networkstatus_get_detached_signatures(con2); - //printf("\n<%s>\n", detached_text2); - dsig2 = networkstatus_parse_detached_signatures(detached_text2, NULL); - test_assert(dsig2); - /* - printf("\n"); - SMARTLIST_FOREACH(dsig2->signatures, networkstatus_voter_info_t *, vi, { - char hd[64]; - base16_encode(hd, sizeof(hd), vi->identity_digest, DIGEST_LEN); - printf("%s\n", hd); - }); - */ - test_eq(2, smartlist_len(dsig2->signatures)); - - /* Try adding to con2 twice; verify that nothing changes. */ - test_eq(0, networkstatus_add_detached_signatures(con2, dsig1, &msg)); - - /* Add to con. */ - test_eq(2, networkstatus_add_detached_signatures(con, dsig2, &msg)); - /* Check signatures */ - test_assert(!networkstatus_check_voter_signature(con, - smartlist_get(con->voters, 1), - cert2)); - test_assert(!networkstatus_check_voter_signature(con, - smartlist_get(con->voters, 2), - cert1)); - - } - - done: - smartlist_free(votes); - tor_free(v1_text); - tor_free(v2_text); - tor_free(v3_text); - tor_free(consensus_text); - - if (vote) - networkstatus_vote_free(vote); - if (v1) - networkstatus_vote_free(v1); - if (v2) - networkstatus_vote_free(v2); - if (v3) - networkstatus_vote_free(v3); - if (con) - networkstatus_vote_free(con); - if (sign_skey_1) - crypto_free_pk_env(sign_skey_1); - if (sign_skey_2) - crypto_free_pk_env(sign_skey_2); - if (sign_skey_3) - crypto_free_pk_env(sign_skey_3); - if (sign_skey_leg1) - crypto_free_pk_env(sign_skey_leg1); - if (cert1) - authority_cert_free(cert1); - if (cert2) - authority_cert_free(cert2); - if (cert3) - authority_cert_free(cert3); - - tor_free(consensus_text2); - tor_free(consensus_text3); - tor_free(detached_text1); - tor_free(detached_text2); - if (con2) - networkstatus_vote_free(con2); - if (con3) - networkstatus_vote_free(con3); - if (dsig1) - ns_detached_signatures_free(dsig1); - if (dsig2) - ns_detached_signatures_free(dsig2); -} - -/** Helper: Parse the exit policy string in <b>policy_str</b>, and make sure - * that policies_summarize() produces the string <b>expected_summary</b> from - * it. */ -static void -test_policy_summary_helper(const char *policy_str, - const char *expected_summary) -{ - config_line_t line; - smartlist_t *policy = smartlist_create(); - char *summary = NULL; - int r; - - line.key = (char*)"foo"; - line.value = (char *)policy_str; - line.next = NULL; - - r = policies_parse_exit_policy(&line, &policy, 0, NULL); - test_eq(r, 0); - summary = policy_summarize(policy); - - test_assert(summary != NULL); - test_streq(summary, expected_summary); - - done: - tor_free(summary); - if (policy) - addr_policy_list_free(policy); -} - -/** Run unit tests for generating summary lines of exit policies */ -static void -test_policies(void) -{ - int i; - smartlist_t *policy = NULL, *policy2 = NULL; - addr_policy_t *p; - tor_addr_t tar; - config_line_t line; - smartlist_t *sm = NULL; - char *policy_str = NULL; - - policy = smartlist_create(); - - p = router_parse_addr_policy_item_from_string("reject 192.168.0.0/16:*",-1); - test_assert(p != NULL); - test_eq(ADDR_POLICY_REJECT, p->policy_type); - tor_addr_from_ipv4h(&tar, 0xc0a80000u); - test_eq(0, tor_addr_compare(&p->addr, &tar, CMP_EXACT)); - test_eq(16, p->maskbits); - test_eq(1, p->prt_min); - test_eq(65535, p->prt_max); - - smartlist_add(policy, p); - - test_assert(ADDR_POLICY_ACCEPTED == - compare_addr_to_addr_policy(0x01020304u, 2, policy)); - test_assert(ADDR_POLICY_PROBABLY_ACCEPTED == - compare_addr_to_addr_policy(0, 2, policy)); - test_assert(ADDR_POLICY_REJECTED == - compare_addr_to_addr_policy(0xc0a80102, 2, policy)); - - policy2 = NULL; - test_assert(0 == policies_parse_exit_policy(NULL, &policy2, 1, NULL)); - test_assert(policy2); - - test_assert(!exit_policy_is_general_exit(policy)); - test_assert(exit_policy_is_general_exit(policy2)); - test_assert(!exit_policy_is_general_exit(NULL)); - - test_assert(cmp_addr_policies(policy, policy2)); - test_assert(cmp_addr_policies(policy, NULL)); - test_assert(!cmp_addr_policies(policy2, policy2)); - test_assert(!cmp_addr_policies(NULL, NULL)); - - test_assert(!policy_is_reject_star(policy2)); - test_assert(policy_is_reject_star(policy)); - test_assert(policy_is_reject_star(NULL)); - - addr_policy_list_free(policy); - policy = NULL; - - /* make sure compacting logic works. */ - policy = NULL; - line.key = (char*)"foo"; - line.value = (char*)"accept *:80,reject private:*,reject *:*"; - line.next = NULL; - test_assert(0 == policies_parse_exit_policy(&line, &policy, 0, NULL)); - test_assert(policy); - //test_streq(policy->string, "accept *:80"); - //test_streq(policy->next->string, "reject *:*"); - test_eq(smartlist_len(policy), 2); - - /* test policy summaries */ - /* check if we properly ignore private IP addresses */ - test_policy_summary_helper("reject 192.168.0.0/16:*," - "reject 0.0.0.0/8:*," - "reject 10.0.0.0/8:*," - "accept *:10-30," - "accept *:90," - "reject *:*", - "accept 10-30,90"); - /* check all accept policies, and proper counting of rejects */ - test_policy_summary_helper("reject 11.0.0.0/9:80," - "reject 12.0.0.0/9:80," - "reject 13.0.0.0/9:80," - "reject 14.0.0.0/9:80," - "accept *:*", "accept 1-65535"); - test_policy_summary_helper("reject 11.0.0.0/9:80," - "reject 12.0.0.0/9:80," - "reject 13.0.0.0/9:80," - "reject 14.0.0.0/9:80," - "reject 15.0.0.0:81," - "accept *:*", "accept 1-65535"); - test_policy_summary_helper("reject 11.0.0.0/9:80," - "reject 12.0.0.0/9:80," - "reject 13.0.0.0/9:80," - "reject 14.0.0.0/9:80," - "reject 15.0.0.0:80," - "accept *:*", - "reject 80"); - /* no exits */ - test_policy_summary_helper("accept 11.0.0.0/9:80," - "reject *:*", - "reject 1-65535"); - /* port merging */ - test_policy_summary_helper("accept *:80," - "accept *:81," - "accept *:100-110," - "accept *:111," - "reject *:*", - "accept 80-81,100-111"); - /* border ports */ - test_policy_summary_helper("accept *:1," - "accept *:3," - "accept *:65535," - "reject *:*", - "accept 1,3,65535"); - /* holes */ - test_policy_summary_helper("accept *:1," - "accept *:3," - "accept *:5," - "accept *:7," - "reject *:*", - "accept 1,3,5,7"); - test_policy_summary_helper("reject *:1," - "reject *:3," - "reject *:5," - "reject *:7," - "accept *:*", - "reject 1,3,5,7"); - - /* truncation ports */ - sm = smartlist_create(); - for (i=1; i<2000; i+=2) { - char buf[POLICY_BUF_LEN]; - tor_snprintf(buf, sizeof(buf), "reject *:%d", i); - smartlist_add(sm, tor_strdup(buf)); - } - smartlist_add(sm, tor_strdup("accept *:*")); - policy_str = smartlist_join_strings(sm, ",", 0, NULL); - test_policy_summary_helper( policy_str, - "accept 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44," - "46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90," - "92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128," - "130,132,134,136,138,140,142,144,146,148,150,152,154,156,158,160,162,164," - "166,168,170,172,174,176,178,180,182,184,186,188,190,192,194,196,198,200," - "202,204,206,208,210,212,214,216,218,220,222,224,226,228,230,232,234,236," - "238,240,242,244,246,248,250,252,254,256,258,260,262,264,266,268,270,272," - "274,276,278,280,282,284,286,288,290,292,294,296,298,300,302,304,306,308," - "310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344," - "346,348,350,352,354,356,358,360,362,364,366,368,370,372,374,376,378,380," - "382,384,386,388,390,392,394,396,398,400,402,404,406,408,410,412,414,416," - "418,420,422,424,426,428,430,432,434,436,438,440,442,444,446,448,450,452," - "454,456,458,460,462,464,466,468,470,472,474,476,478,480,482,484,486,488," - "490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522"); - - done: - if (policy) - addr_policy_list_free(policy); - if (policy2) - addr_policy_list_free(policy2); - tor_free(policy_str); - if (sm) { - SMARTLIST_FOREACH(sm, char *, s, tor_free(s)); - smartlist_free(sm); - } -} - -/** Run unit tests for basic rendezvous functions. */ -static void -test_rend_fns(void) -{ - char address1[] = "fooaddress.onion"; - char address2[] = "aaaaaaaaaaaaaaaa.onion"; - char address3[] = "fooaddress.exit"; - char address4[] = "www.torproject.org"; - rend_service_descriptor_t *d1 = - tor_malloc_zero(sizeof(rend_service_descriptor_t)); - rend_service_descriptor_t *d2 = NULL; - char *encoded = NULL; - size_t len; - time_t now; - int i; - crypto_pk_env_t *pk1 = pk_generate(0), *pk2 = pk_generate(1); - - /* Test unversioned (v0) descriptor */ - d1->pk = crypto_pk_dup_key(pk1); - now = time(NULL); - d1->timestamp = now; - d1->version = 0; - d1->intro_nodes = smartlist_create(); - for (i = 0; i < 3; i++) { - rend_intro_point_t *intro = tor_malloc_zero(sizeof(rend_intro_point_t)); - intro->extend_info = tor_malloc_zero(sizeof(extend_info_t)); - crypto_rand(intro->extend_info->identity_digest, DIGEST_LEN); - intro->extend_info->nickname[0] = '$'; - base16_encode(intro->extend_info->nickname+1, HEX_DIGEST_LEN+1, - intro->extend_info->identity_digest, DIGEST_LEN); - smartlist_add(d1->intro_nodes, intro); - } - test_assert(! rend_encode_service_descriptor(d1, pk1, &encoded, &len)); - d2 = rend_parse_service_descriptor(encoded, len); - test_assert(d2); - - test_assert(!crypto_pk_cmp_keys(d1->pk, d2->pk)); - test_eq(d2->timestamp, now); - test_eq(d2->version, 0); - test_eq(d2->protocols, 1<<2); - test_eq(smartlist_len(d2->intro_nodes), 3); - for (i = 0; i < 3; i++) { - rend_intro_point_t *intro1 = smartlist_get(d1->intro_nodes, i); - rend_intro_point_t *intro2 = smartlist_get(d2->intro_nodes, i); - test_streq(intro1->extend_info->nickname, - intro2->extend_info->nickname); - } - - test_assert(BAD_HOSTNAME == parse_extended_hostname(address1)); - test_assert(ONION_HOSTNAME == parse_extended_hostname(address2)); - test_assert(EXIT_HOSTNAME == parse_extended_hostname(address3)); - test_assert(NORMAL_HOSTNAME == parse_extended_hostname(address4)); - - crypto_free_pk_env(pk1); - crypto_free_pk_env(pk2); - pk1 = pk2 = NULL; - rend_service_descriptor_free(d1); - rend_service_descriptor_free(d2); - d1 = d2 = NULL; - - done: - if (pk1) - crypto_free_pk_env(pk1); - if (pk2) - crypto_free_pk_env(pk2); - if (d1) - rend_service_descriptor_free(d1); - if (d2) - rend_service_descriptor_free(d2); - tor_free(encoded); -} - -/** Run AES performance benchmarks. */ -static void -bench_aes(void) -{ - int len, i; - char *b1, *b2; - crypto_cipher_env_t *c; - struct timeval start, end; - const int iters = 100000; - uint64_t nsec; - c = crypto_new_cipher_env(); - crypto_cipher_generate_key(c); - crypto_cipher_encrypt_init_cipher(c); - for (len = 1; len <= 8192; len *= 2) { - b1 = tor_malloc_zero(len); - b2 = tor_malloc_zero(len); - tor_gettimeofday(&start); - for (i = 0; i < iters; ++i) { - crypto_cipher_encrypt(c, b1, b2, len); - } - tor_gettimeofday(&end); - tor_free(b1); - tor_free(b2); - nsec = (uint64_t) tv_udiff(&start,&end); - nsec *= 1000; - nsec /= (iters*len); - printf("%d bytes: "U64_FORMAT" nsec per byte\n", len, - U64_PRINTF_ARG(nsec)); - } - crypto_free_cipher_env(c); -} - -/** Run digestmap_t performance benchmarks. */ -static void -bench_dmap(void) -{ - smartlist_t *sl = smartlist_create(); - smartlist_t *sl2 = smartlist_create(); - struct timeval start, end, pt2, pt3, pt4; - const int iters = 10000; - const int elts = 4000; - const int fpostests = 1000000; - char d[20]; - int i,n=0, fp = 0; - digestmap_t *dm = digestmap_new(); - digestset_t *ds = digestset_new(elts); - - for (i = 0; i < elts; ++i) { - crypto_rand(d, 20); - smartlist_add(sl, tor_memdup(d, 20)); - } - for (i = 0; i < elts; ++i) { - crypto_rand(d, 20); - smartlist_add(sl2, tor_memdup(d, 20)); - } - printf("nbits=%d\n", ds->mask+1); - - tor_gettimeofday(&start); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, digestmap_set(dm, cp, (void*)1)); - } - tor_gettimeofday(&pt2); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, digestmap_get(dm, cp)); - SMARTLIST_FOREACH(sl2, const char *, cp, digestmap_get(dm, cp)); - } - tor_gettimeofday(&pt3); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, digestset_add(ds, cp)); - } - tor_gettimeofday(&pt4); - for (i = 0; i < iters; ++i) { - SMARTLIST_FOREACH(sl, const char *, cp, n += digestset_isin(ds, cp)); - SMARTLIST_FOREACH(sl2, const char *, cp, n += digestset_isin(ds, cp)); - } - tor_gettimeofday(&end); - - for (i = 0; i < fpostests; ++i) { - crypto_rand(d, 20); - if (digestset_isin(ds, d)) ++fp; - } - - printf("%ld\n",(unsigned long)tv_udiff(&start, &pt2)); - printf("%ld\n",(unsigned long)tv_udiff(&pt2, &pt3)); - printf("%ld\n",(unsigned long)tv_udiff(&pt3, &pt4)); - printf("%ld\n",(unsigned long)tv_udiff(&pt4, &end)); - printf("-- %d\n", n); - printf("++ %f\n", fp/(double)fpostests); - digestmap_free(dm, NULL); - digestset_free(ds); - SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); - SMARTLIST_FOREACH(sl2, char *, cp, tor_free(cp)); - smartlist_free(sl); - smartlist_free(sl2); -} - -/** Run unittests for memory pool allocator */ -static void -test_util_mempool(void) -{ - mp_pool_t *pool = NULL; - smartlist_t *allocated = NULL; - int i; - - pool = mp_pool_new(1, 100); - test_assert(pool); - test_assert(pool->new_chunk_capacity >= 100); - test_assert(pool->item_alloc_size >= sizeof(void*)+1); - mp_pool_destroy(pool); - pool = NULL; - - pool = mp_pool_new(241, 2500); - test_assert(pool); - test_assert(pool->new_chunk_capacity >= 10); - test_assert(pool->item_alloc_size >= sizeof(void*)+241); - test_eq(pool->item_alloc_size & 0x03, 0); - test_assert(pool->new_chunk_capacity < 60); - - allocated = smartlist_create(); - for (i = 0; i < 20000; ++i) { - if (smartlist_len(allocated) < 20 || crypto_rand_int(2)) { - void *m = mp_pool_get(pool); - memset(m, 0x09, 241); - smartlist_add(allocated, m); - //printf("%d: %p\n", i, m); - //mp_pool_assert_ok(pool); - } else { - int idx = crypto_rand_int(smartlist_len(allocated)); - void *m = smartlist_get(allocated, idx); - //printf("%d: free %p\n", i, m); - smartlist_del(allocated, idx); - mp_pool_release(m); - //mp_pool_assert_ok(pool); - } - if (crypto_rand_int(777)==0) - mp_pool_clean(pool, 1, 1); - - if (i % 777) - mp_pool_assert_ok(pool); - } - - done: - if (allocated) { - SMARTLIST_FOREACH(allocated, void *, m, mp_pool_release(m)); - mp_pool_assert_ok(pool); - mp_pool_clean(pool, 0, 0); - mp_pool_assert_ok(pool); - smartlist_free(allocated); - } - - if (pool) - mp_pool_destroy(pool); -} - -/** Run unittests for memory area allocator */ -static void -test_util_memarea(void) -{ - memarea_t *area = memarea_new(); - char *p1, *p2, *p3, *p1_orig; - void *malloced_ptr = NULL; - int i; - - test_assert(area); - - p1_orig = p1 = memarea_alloc(area,64); - p2 = memarea_alloc_zero(area,52); - p3 = memarea_alloc(area,11); - - test_assert(memarea_owns_ptr(area, p1)); - test_assert(memarea_owns_ptr(area, p2)); - test_assert(memarea_owns_ptr(area, p3)); - /* Make sure we left enough space. */ - test_assert(p1+64 <= p2); - test_assert(p2+52 <= p3); - /* Make sure we aligned. */ - test_eq(((uintptr_t)p1) % sizeof(void*), 0); - test_eq(((uintptr_t)p2) % sizeof(void*), 0); - test_eq(((uintptr_t)p3) % sizeof(void*), 0); - test_assert(!memarea_owns_ptr(area, p3+8192)); - test_assert(!memarea_owns_ptr(area, p3+30)); - test_assert(tor_mem_is_zero(p2, 52)); - /* Make sure we don't overalign. */ - p1 = memarea_alloc(area, 1); - p2 = memarea_alloc(area, 1); - test_eq(p1+sizeof(void*), p2); - { - malloced_ptr = tor_malloc(64); - test_assert(!memarea_owns_ptr(area, malloced_ptr)); - tor_free(malloced_ptr); - } - - /* memarea_memdup */ - { - malloced_ptr = tor_malloc(64); - crypto_rand((char*)malloced_ptr, 64); - p1 = memarea_memdup(area, malloced_ptr, 64); - test_assert(p1 != malloced_ptr); - test_memeq(p1, malloced_ptr, 64); - tor_free(malloced_ptr); - } - - /* memarea_strdup. */ - p1 = memarea_strdup(area,""); - p2 = memarea_strdup(area, "abcd"); - test_assert(p1); - test_assert(p2); - test_streq(p1, ""); - test_streq(p2, "abcd"); - - /* memarea_strndup. */ - { - const char *s = "Ad ogni porta batte la morte e grida: il nome!"; - /* (From Turandot, act 3.) */ - size_t len = strlen(s); - p1 = memarea_strndup(area, s, 1000); - p2 = memarea_strndup(area, s, 10); - test_streq(p1, s); - test_assert(p2 >= p1 + len + 1); - test_memeq(s, p2, 10); - test_eq(p2[10], '\0'); - p3 = memarea_strndup(area, s, len); - test_streq(p3, s); - p3 = memarea_strndup(area, s, len-1); - test_memeq(s, p3, len-1); - test_eq(p3[len-1], '\0'); - } - - memarea_clear(area); - p1 = memarea_alloc(area, 1); - test_eq(p1, p1_orig); - memarea_clear(area); - - /* Check for running over an area's size. */ - for (i = 0; i < 512; ++i) { - p1 = memarea_alloc(area, crypto_rand_int(5)+1); - test_assert(memarea_owns_ptr(area, p1)); - } - memarea_assert_ok(area); - /* Make sure we can allocate a too-big object. */ - p1 = memarea_alloc_zero(area, 9000); - p2 = memarea_alloc_zero(area, 16); - test_assert(memarea_owns_ptr(area, p1)); - test_assert(memarea_owns_ptr(area, p2)); - - done: - memarea_drop_all(area); - tor_free(malloced_ptr); -} - -/** Run unit tests for utility functions to get file names relative to - * the data directory. */ -static void -test_util_datadir(void) -{ - char buf[1024]; - char *f = NULL; - - f = get_datadir_fname(NULL); - test_streq(f, temp_dir); - tor_free(f); - f = get_datadir_fname("state"); - tor_snprintf(buf, sizeof(buf), "%s"PATH_SEPARATOR"state", temp_dir); - test_streq(f, buf); - tor_free(f); - f = get_datadir_fname2("cache", "thingy"); - tor_snprintf(buf, sizeof(buf), - "%s"PATH_SEPARATOR"cache"PATH_SEPARATOR"thingy", temp_dir); - test_streq(f, buf); - tor_free(f); - f = get_datadir_fname2_suffix("cache", "thingy", ".foo"); - tor_snprintf(buf, sizeof(buf), - "%s"PATH_SEPARATOR"cache"PATH_SEPARATOR"thingy.foo", temp_dir); - test_streq(f, buf); - tor_free(f); - f = get_datadir_fname_suffix("cache", ".foo"); - tor_snprintf(buf, sizeof(buf), "%s"PATH_SEPARATOR"cache.foo", - temp_dir); - test_streq(f, buf); - - done: - tor_free(f); -} - -/** Test AES-CTR encryption and decryption with IV. */ -static void -test_crypto_aes_iv(void) -{ - crypto_cipher_env_t *cipher; - char *plain, *encrypted1, *encrypted2, *decrypted1, *decrypted2; - char plain_1[1], plain_15[15], plain_16[16], plain_17[17]; - char key1[16], key2[16]; - ssize_t encrypted_size, decrypted_size; - - plain = tor_malloc(4095); - encrypted1 = tor_malloc(4095 + 1 + 16); - encrypted2 = tor_malloc(4095 + 1 + 16); - decrypted1 = tor_malloc(4095 + 1); - decrypted2 = tor_malloc(4095 + 1); - - crypto_rand(plain, 4095); - crypto_rand(key1, 16); - crypto_rand(key2, 16); - crypto_rand(plain_1, 1); - crypto_rand(plain_15, 15); - crypto_rand(plain_16, 16); - crypto_rand(plain_17, 17); - key1[0] = key2[0] + 128; /* Make sure that contents are different. */ - /* Encrypt and decrypt with the same key. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 4095, - plain, 4095); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(encrypted_size, 16 + 4095); - tor_assert(encrypted_size > 0); /* This is obviously true, since 4111 is - * greater than 0, but its truth is not - * obvious to all analysis tools. */ - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 4095, - encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(decrypted_size, 4095); - tor_assert(decrypted_size > 0); - test_memeq(plain, decrypted1, 4095); - /* Encrypt a second time (with a new random initialization vector). */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted2, 16 + 4095, - plain, 4095); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(encrypted_size, 16 + 4095); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted2, 4095, - encrypted2, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(decrypted_size, 4095); - tor_assert(decrypted_size > 0); - test_memeq(plain, decrypted2, 4095); - test_memneq(encrypted1, encrypted2, encrypted_size); - /* Decrypt with the wrong key. */ - cipher = crypto_create_init_cipher(key2, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted2, 4095, - encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_memneq(plain, decrypted2, encrypted_size); - /* Alter the initialization vector. */ - encrypted1[0] += 42; - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 4095, - encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_memneq(plain, decrypted2, 4095); - /* Special length case: 1. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 1, - plain_1, 1); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(encrypted_size, 16 + 1); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 1, - encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(decrypted_size, 1); - tor_assert(decrypted_size > 0); - test_memeq(plain_1, decrypted1, 1); - /* Special length case: 15. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 15, - plain_15, 15); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(encrypted_size, 16 + 15); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 15, - encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(decrypted_size, 15); - tor_assert(decrypted_size > 0); - test_memeq(plain_15, decrypted1, 15); - /* Special length case: 16. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 16, - plain_16, 16); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(encrypted_size, 16 + 16); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 16, - encrypted1, encrypted_size); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(decrypted_size, 16); - tor_assert(decrypted_size > 0); - test_memeq(plain_16, decrypted1, 16); - /* Special length case: 17. */ - cipher = crypto_create_init_cipher(key1, 1); - encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 17, - plain_17, 17); - crypto_free_cipher_env(cipher); - cipher = NULL; - test_eq(encrypted_size, 16 + 17); - tor_assert(encrypted_size > 0); - cipher = crypto_create_init_cipher(key1, 0); - decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 17, - encrypted1, encrypted_size); - test_eq(decrypted_size, 17); - tor_assert(decrypted_size > 0); - test_memeq(plain_17, decrypted1, 17); - - done: - /* Free memory. */ - tor_free(plain); - tor_free(encrypted1); - tor_free(encrypted2); - tor_free(decrypted1); - tor_free(decrypted2); - if (cipher) - crypto_free_cipher_env(cipher); -} - -/** Test base32 decoding. */ -static void -test_crypto_base32_decode(void) -{ - char plain[60], encoded[96 + 1], decoded[60]; - int res; - crypto_rand(plain, 60); - /* Encode and decode a random string. */ - base32_encode(encoded, 96 + 1, plain, 60); - res = base32_decode(decoded, 60, encoded, 96); - test_eq(res, 0); - test_memeq(plain, decoded, 60); - /* Encode, uppercase, and decode a random string. */ - base32_encode(encoded, 96 + 1, plain, 60); - tor_strupper(encoded); - res = base32_decode(decoded, 60, encoded, 96); - test_eq(res, 0); - test_memeq(plain, decoded, 60); - /* Change encoded string and decode. */ - if (encoded[0] == 'A' || encoded[0] == 'a') - encoded[0] = 'B'; - else - encoded[0] = 'A'; - res = base32_decode(decoded, 60, encoded, 96); - test_eq(res, 0); - test_memneq(plain, decoded, 60); - /* Bad encodings. */ - encoded[0] = '!'; - res = base32_decode(decoded, 60, encoded, 96); - test_assert(res < 0); - - done: - ; -} - -/** Test encoding and parsing of v2 rendezvous service descriptors. */ -static void -test_rend_fns_v2(void) -{ - rend_service_descriptor_t *generated = NULL, *parsed = NULL; - char service_id[DIGEST_LEN]; - char service_id_base32[REND_SERVICE_ID_LEN_BASE32+1]; - const char *next_desc; - smartlist_t *descs = smartlist_create(); - char computed_desc_id[DIGEST_LEN]; - char parsed_desc_id[DIGEST_LEN]; - crypto_pk_env_t *pk1 = NULL, *pk2 = NULL; - time_t now; - char *intro_points_encrypted = NULL; - size_t intro_points_size; - size_t encoded_size; - int i; - pk1 = pk_generate(0); - pk2 = pk_generate(1); - generated = tor_malloc_zero(sizeof(rend_service_descriptor_t)); - generated->pk = crypto_pk_dup_key(pk1); - crypto_pk_get_digest(generated->pk, service_id); - base32_encode(service_id_base32, REND_SERVICE_ID_LEN_BASE32+1, - service_id, REND_SERVICE_ID_LEN); - now = time(NULL); - generated->timestamp = now; - generated->version = 2; - generated->protocols = 42; - generated->intro_nodes = smartlist_create(); - - for (i = 0; i < 3; i++) { - rend_intro_point_t *intro = tor_malloc_zero(sizeof(rend_intro_point_t)); - crypto_pk_env_t *okey = pk_generate(2 + i); - intro->extend_info = tor_malloc_zero(sizeof(extend_info_t)); - intro->extend_info->onion_key = okey; - crypto_pk_get_digest(intro->extend_info->onion_key, - intro->extend_info->identity_digest); - //crypto_rand(info->identity_digest, DIGEST_LEN); /* Would this work? */ - intro->extend_info->nickname[0] = '$'; - base16_encode(intro->extend_info->nickname + 1, - sizeof(intro->extend_info->nickname) - 1, - intro->extend_info->identity_digest, DIGEST_LEN); - /* Does not cover all IP addresses. */ - tor_addr_from_ipv4h(&intro->extend_info->addr, crypto_rand_int(65536)); - intro->extend_info->port = crypto_rand_int(65536); - intro->intro_key = crypto_pk_dup_key(pk2); - smartlist_add(generated->intro_nodes, intro); - } - test_assert(rend_encode_v2_descriptors(descs, generated, now, 0, - REND_NO_AUTH, NULL, NULL) > 0); - test_assert(rend_compute_v2_desc_id(computed_desc_id, service_id_base32, - NULL, now, 0) == 0); - test_memeq(((rend_encoded_v2_service_descriptor_t *) - smartlist_get(descs, 0))->desc_id, computed_desc_id, DIGEST_LEN); - test_assert(rend_parse_v2_service_descriptor(&parsed, parsed_desc_id, - &intro_points_encrypted, - &intro_points_size, - &encoded_size, - &next_desc, - ((rend_encoded_v2_service_descriptor_t *) - smartlist_get(descs, 0))->desc_str) == 0); - test_assert(parsed); - test_memeq(((rend_encoded_v2_service_descriptor_t *) - smartlist_get(descs, 0))->desc_id, parsed_desc_id, DIGEST_LEN); - test_eq(rend_parse_introduction_points(parsed, intro_points_encrypted, - intro_points_size), 3); - test_assert(!crypto_pk_cmp_keys(generated->pk, parsed->pk)); - test_eq(parsed->timestamp, now); - test_eq(parsed->version, 2); - test_eq(parsed->protocols, 42); - test_eq(smartlist_len(parsed->intro_nodes), 3); - for (i = 0; i < smartlist_len(parsed->intro_nodes); i++) { - rend_intro_point_t *par_intro = smartlist_get(parsed->intro_nodes, i), - *gen_intro = smartlist_get(generated->intro_nodes, i); - extend_info_t *par_info = par_intro->extend_info; - extend_info_t *gen_info = gen_intro->extend_info; - test_assert(!crypto_pk_cmp_keys(gen_info->onion_key, par_info->onion_key)); - test_memeq(gen_info->identity_digest, par_info->identity_digest, - DIGEST_LEN); - test_streq(gen_info->nickname, par_info->nickname); - test_assert(tor_addr_eq(&gen_info->addr, &par_info->addr)); - test_eq(gen_info->port, par_info->port); - } - - rend_service_descriptor_free(parsed); - rend_service_descriptor_free(generated); - parsed = generated = NULL; - - done: - if (descs) { - for (i = 0; i < smartlist_len(descs); i++) - rend_encoded_v2_service_descriptor_free(smartlist_get(descs, i)); - smartlist_free(descs); - } - if (parsed) - rend_service_descriptor_free(parsed); - if (generated) - rend_service_descriptor_free(generated); - if (pk1) - crypto_free_pk_env(pk1); - if (pk2) - crypto_free_pk_env(pk2); - tor_free(intro_points_encrypted); -} - -/** Run unit tests for GeoIP code. */ -static void -test_geoip(void) -{ - int i, j; - time_t now = time(NULL); - char *s = NULL; - - /* Populate the DB a bit. Add these in order, since we can't do the final - * 'sort' step. These aren't very good IP addresses, but they're perfectly - * fine uint32_t values. */ - test_eq(0, geoip_parse_entry("10,50,AB")); - test_eq(0, geoip_parse_entry("52,90,XY")); - test_eq(0, geoip_parse_entry("95,100,AB")); - test_eq(0, geoip_parse_entry("\"105\",\"140\",\"ZZ\"")); - test_eq(0, geoip_parse_entry("\"150\",\"190\",\"XY\"")); - test_eq(0, geoip_parse_entry("\"200\",\"250\",\"AB\"")); - - /* We should have 3 countries: ab, xy, zz. */ - test_eq(3, geoip_get_n_countries()); - /* Make sure that country ID actually works. */ -#define NAMEFOR(x) geoip_get_country_name(geoip_get_country_by_ip(x)) - test_streq("ab", NAMEFOR(32)); - test_streq("??", NAMEFOR(5)); - test_streq("??", NAMEFOR(51)); - test_streq("xy", NAMEFOR(150)); - test_streq("xy", NAMEFOR(190)); - test_streq("??", NAMEFOR(2000)); -#undef NAMEFOR - - get_options()->BridgeRelay = 1; - get_options()->BridgeRecordUsageByCountry = 1; - /* Put 9 observations in AB... */ - for (i=32; i < 40; ++i) - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now-7200); - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, 225, now-7200); - /* and 3 observations in XY, several times. */ - for (j=0; j < 10; ++j) - for (i=52; i < 55; ++i) - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now-3600); - /* and 17 observations in ZZ... */ - for (i=110; i < 127; ++i) - geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now); - s = geoip_get_client_history(now+5*24*60*60, GEOIP_CLIENT_CONNECT); - test_assert(s); - test_streq("zz=24,ab=16,xy=8", s); - tor_free(s); - - /* Now clear out all the AB observations. */ - geoip_remove_old_clients(now-6000); - s = geoip_get_client_history(now+5*24*60*60, GEOIP_CLIENT_CONNECT); - test_assert(s); - test_streq("zz=24,xy=8", s); - - done: - tor_free(s); -} - -/** For test_array. Declare an CLI-invocable off-by-default function in the - * unit tests, with function name and user-visible name <b>x</b>*/ -#define DISABLED(x) { #x, x, 0, 0, 0 } -/** For test_array. Declare an CLI-invocable unit test function, with function - * name test_<b>x</b>(), and user-visible name <b>x</b> */ -#define ENT(x) { #x, test_ ## x, 0, 0, 1 } -/** For test_array. Declare an CLI-invocable unit test function, with function - * name test_<b>x</b>_<b>y</b>(), and user-visible name - * <b>x</b>/<b>y</b>. This function will be treated as a subentry of <b>x</b>, - * so that invoking <b>x</b> from the CLI invokes this test too. */ -#define SUBENT(x,y) { #x "/" #y, test_ ## x ## _ ## y, 1, 0, 1 } - -/** An array of functions and information for all the unit tests we can run. */ -static struct { - const char *test_name; /**< How does the user refer to this test from the - * command line? */ - void (*test_fn)(void); /**< What function is called to run this test? */ - int is_subent; /**< Is this a subentry of a bigger set of related tests? */ - int selected; /**< Are we planning to run this one? */ - int is_default; /**< If the user doesn't say what tests they want, do they - * get this function by default? */ -} test_array[] = { - ENT(buffers), - ENT(crypto), - SUBENT(crypto, rng), - SUBENT(crypto, aes), - SUBENT(crypto, sha), - SUBENT(crypto, pk), - SUBENT(crypto, dh), - SUBENT(crypto, s2k), - SUBENT(crypto, aes_iv), - SUBENT(crypto, base32_decode), - ENT(util), - SUBENT(util, ip6_helpers), - SUBENT(util, gzip), - SUBENT(util, datadir), - SUBENT(util, smartlist_basic), - SUBENT(util, smartlist_strings), - SUBENT(util, smartlist_overlap), - SUBENT(util, smartlist_digests), - SUBENT(util, smartlist_join), - SUBENT(util, bitarray), - SUBENT(util, digestset), - SUBENT(util, mempool), - SUBENT(util, memarea), - SUBENT(util, strmap), - SUBENT(util, control_formats), - SUBENT(util, pqueue), - SUBENT(util, mmap), - SUBENT(util, threads), - SUBENT(util, order_functions), - SUBENT(util, sscanf), - ENT(onion_handshake), - ENT(dir_format), - ENT(dirutil), - ENT(v3_networkstatus), - ENT(policies), - ENT(rend_fns), - SUBENT(rend_fns, v2), - ENT(geoip), - - DISABLED(bench_aes), - DISABLED(bench_dmap), - { NULL, NULL, 0, 0, 0 }, -}; - -static void syntax(void) ATTR_NORETURN; - -/** Print a syntax usage message, and exit.*/ -static void -syntax(void) -{ - int i; - printf("Syntax:\n" - " test [-v|--verbose] [--warn|--notice|--info|--debug]\n" - " [testname...]\n" - "Recognized tests are:\n"); - for (i = 0; test_array[i].test_name; ++i) { - printf(" %s\n", test_array[i].test_name); - } - - exit(0); -} - -/** Main entry point for unit test code: parse the command line, and run - * some unit tests. */ -int -main(int c, char**v) -{ - or_options_t *options; - char *errmsg = NULL; - int i; - int verbose = 0, any_selected = 0; - int loglevel = LOG_ERR; - -#ifdef USE_DMALLOC - { - int r = CRYPTO_set_mem_ex_functions(_tor_malloc, _tor_realloc, _tor_free); - tor_assert(r); - } -#endif - - update_approx_time(time(NULL)); - options = options_new(); - tor_threads_init(); - init_logging(); - - for (i = 1; i < c; ++i) { - if (!strcmp(v[i], "-v") || !strcmp(v[i], "--verbose")) - verbose++; - else if (!strcmp(v[i], "--warn")) - loglevel = LOG_WARN; - else if (!strcmp(v[i], "--notice")) - loglevel = LOG_NOTICE; - else if (!strcmp(v[i], "--info")) - loglevel = LOG_INFO; - else if (!strcmp(v[i], "--debug")) - loglevel = LOG_DEBUG; - else if (!strcmp(v[i], "--help") || !strcmp(v[i], "-h") || v[i][0] == '-') - syntax(); - else { - int j, found=0; - for (j = 0; test_array[j].test_name; ++j) { - if (!strcmp(v[i], test_array[j].test_name) || - (test_array[j].is_subent && - !strcmpstart(test_array[j].test_name, v[i]) && - test_array[j].test_name[strlen(v[i])] == '/') || - (v[i][0] == '=' && !strcmp(v[i]+1, test_array[j].test_name))) { - test_array[j].selected = 1; - any_selected = 1; - found = 1; - } - } - if (!found) { - printf("Unknown test: %s\n", v[i]); - syntax(); - } - } - } - - if (!any_selected) { - for (i = 0; test_array[i].test_name; ++i) { - test_array[i].selected = test_array[i].is_default; - } - } - - { - log_severity_list_t s; - memset(&s, 0, sizeof(s)); - set_log_severity_config(loglevel, LOG_ERR, &s); - add_stream_log(&s, "", fileno(stdout)); - } - - options->command = CMD_RUN_UNITTESTS; - crypto_global_init(0); - rep_hist_init(); - network_init(); - setup_directory(); - options_init(options); - options->DataDirectory = tor_strdup(temp_dir); - if (set_options(options, &errmsg) < 0) { - printf("Failed to set initial options: %s\n", errmsg); - tor_free(errmsg); - return 1; - } - - crypto_seed_rng(1); - - atexit(remove_directory); - - printf("Running Tor unit tests on %s\n", get_uname()); - - for (i = 0; test_array[i].test_name; ++i) { - if (!test_array[i].selected) - continue; - if (!test_array[i].is_subent) { - printf("\n============================== %s\n",test_array[i].test_name); - } else if (test_array[i].is_subent && verbose) { - printf("\n%s", test_array[i].test_name); - } - test_array[i].test_fn(); - } - puts(""); - - free_pregenerated_keys(); -#ifdef USE_DMALLOC - tor_free_all(0); - dmalloc_log_unfreed(); -#endif - - if (have_failed) - return 1; - else - return 0; -} - diff --git a/src/or/tor_main.c b/src/or/tor_main.c index 8fd3804b2..1ce14ab76 100644 --- a/src/or/tor_main.c +++ b/src/or/tor_main.c @@ -7,7 +7,7 @@ * built from. This string is generated by a bit of shell kludging int * src/or/Makefile.am, and is usually right. */ -const char tor_svn_revision[] = +const char tor_git_revision[] = #ifndef _MSC_VER #include "micro-revision.i" #endif diff --git a/src/test/Makefile.am b/src/test/Makefile.am new file mode 100644 index 000000000..546fa2f4b --- /dev/null +++ b/src/test/Makefile.am @@ -0,0 +1,30 @@ +TESTS = test + +noinst_PROGRAMS = test + +AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ + -DLOCALSTATEDIR="\"$(localstatedir)\"" \ + -DBINDIR="\"$(bindir)\"" \ + -I"$(top_srcdir)/src/or" + +# -L flags need to go in LDFLAGS. -l flags need to go in LDADD. +# This seems to matter nowhere but on windows, but I assure you that it +# matters a lot there, and is quite hard to debug if you forget to do it. + +test_SOURCES = \ + test_data.c \ + test.c \ + test_addr.c \ + test_crypto.c \ + test_dir.c \ + test_containers.c \ + test_util.c \ + tinytest.c + +test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ + @TOR_LDFLAGS_libevent@ +test_LDADD = ../or/libtor.a ../common/libor.a ../common/libor-crypto.a \ + ../common/libor-event.a \ + @TOR_ZLIB_LIBS@ -lm @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +noinst_HEADERS = tinytest.h tinytest_macros.h test.h diff --git a/src/test/test.c b/src/test/test.c new file mode 100644 index 000000000..9b24a99b5 --- /dev/null +++ b/src/test/test.c @@ -0,0 +1,1292 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/* Ordinarily defined in tor_main.c; this bit is just here to provide one + * since we're not linking to tor_main.c */ +const char tor_git_revision[] = ""; + +/** + * \file test.c + * \brief Unit tests for many pieces of the lower level Tor modules. + **/ + +#include "orconfig.h" + +#include <stdio.h> +#ifdef HAVE_FCNTL_H +#include <fcntl.h> +#endif + +#ifdef MS_WINDOWS +/* For mkdir() */ +#include <direct.h> +#else +#include <dirent.h> +#endif + +/* These macros pull in declarations for some functions and structures that + * are typically file-private. */ +#define BUFFERS_PRIVATE +#define CONFIG_PRIVATE +#define GEOIP_PRIVATE +#define ROUTER_PRIVATE +#define CIRCUIT_PRIVATE + +/* + * Linux doesn't provide lround in math.h by default, but mac os does... + * It's best just to leave math.h out of the picture entirely. + */ +//#include <math.h> +long int lround(double x); +double fabs(double x); + +#include "or.h" +#include "buffers.h" +#include "circuitbuild.h" +#include "config.h" +#include "connection_edge.h" +#include "geoip.h" +#include "rendcommon.h" +#include "test.h" +#include "torgzip.h" +#include "mempool.h" +#include "memarea.h" +#include "onion.h" +#include "policies.h" +#include "rephist.h" +#include "routerparse.h" + +#ifdef USE_DMALLOC +#include <dmalloc.h> +#include <openssl/crypto.h> +#include "main.h" +#endif + +/** Set to true if any unit test has failed. Mostly, this is set by the macros + * in test.h */ +int have_failed = 0; + +/** Temporary directory (set up by setup_directory) under which we store all + * our files during testing. */ +static char temp_dir[256]; +static pid_t temp_dir_setup_in_pid = 0; + +/** Select and create the temporary directory we'll use to run our unit tests. + * Store it in <b>temp_dir</b>. Exit immediately if we can't create it. + * idempotent. */ +static void +setup_directory(void) +{ + static int is_setup = 0; + int r; + if (is_setup) return; + +#ifdef MS_WINDOWS + { + char buf[MAX_PATH]; + const char *tmp = buf; + /* If this fails, we're probably screwed anyway */ + if (!GetTempPath(sizeof(buf),buf)) + tmp = "c:\\windows\\temp"; + tor_snprintf(temp_dir, sizeof(temp_dir), + "%s\\tor_test_%d", tmp, (int)getpid()); + r = mkdir(temp_dir); + } +#else + tor_snprintf(temp_dir, sizeof(temp_dir), "/tmp/tor_test_%d", (int) getpid()); + r = mkdir(temp_dir, 0700); +#endif + if (r) { + fprintf(stderr, "Can't create directory %s:", temp_dir); + perror(""); + exit(1); + } + is_setup = 1; + temp_dir_setup_in_pid = getpid(); +} + +/** Return a filename relative to our testing temporary directory */ +const char * +get_fname(const char *name) +{ + static char buf[1024]; + setup_directory(); + if (!name) + return temp_dir; + tor_snprintf(buf,sizeof(buf),"%s/%s",temp_dir,name); + return buf; +} + +/** Remove all files stored under the temporary directory, and the directory + * itself. Called by atexit(). */ +static void +remove_directory(void) +{ + smartlist_t *elements; + if (getpid() != temp_dir_setup_in_pid) { + /* Only clean out the tempdir when the main process is exiting. */ + return; + } + elements = tor_listdir(temp_dir); + if (elements) { + SMARTLIST_FOREACH(elements, const char *, cp, + { + size_t len = strlen(cp)+strlen(temp_dir)+16; + char *tmp = tor_malloc(len); + tor_snprintf(tmp, len, "%s"PATH_SEPARATOR"%s", temp_dir, cp); + unlink(tmp); + tor_free(tmp); + }); + SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); + smartlist_free(elements); + } + rmdir(temp_dir); +} + +/** Define this if unit tests spend too much time generating public keys*/ +#undef CACHE_GENERATED_KEYS + +static crypto_pk_env_t *pregen_keys[5] = {NULL, NULL, NULL, NULL, NULL}; +#define N_PREGEN_KEYS ((int)(sizeof(pregen_keys)/sizeof(pregen_keys[0]))) + +/** Generate and return a new keypair for use in unit tests. If we're using + * the key cache optimization, we might reuse keys: we only guarantee that + * keys made with distinct values for <b>idx</b> are different. The value of + * <b>idx</b> must be at least 0, and less than N_PREGEN_KEYS. */ +crypto_pk_env_t * +pk_generate(int idx) +{ +#ifdef CACHE_GENERATED_KEYS + tor_assert(idx < N_PREGEN_KEYS); + if (! pregen_keys[idx]) { + pregen_keys[idx] = crypto_new_pk_env(); + tor_assert(!crypto_pk_generate_key(pregen_keys[idx])); + } + return crypto_pk_dup_key(pregen_keys[idx]); +#else + crypto_pk_env_t *result; + (void) idx; + result = crypto_new_pk_env(); + tor_assert(!crypto_pk_generate_key(result)); + return result; +#endif +} + +/** Free all storage used for the cached key optimization. */ +static void +free_pregenerated_keys(void) +{ + unsigned idx; + for (idx = 0; idx < N_PREGEN_KEYS; ++idx) { + if (pregen_keys[idx]) { + crypto_free_pk_env(pregen_keys[idx]); + pregen_keys[idx] = NULL; + } + } +} + +/** Run unit tests for buffers.c */ +static void +test_buffers(void) +{ + char str[256]; + char str2[256]; + + buf_t *buf = NULL, *buf2 = NULL; + const char *cp; + + int j; + size_t r; + + /**** + * buf_new + ****/ + if (!(buf = buf_new())) + test_fail(); + + //test_eq(buf_capacity(buf), 4096); + test_eq(buf_datalen(buf), 0); + + /**** + * General pointer frobbing + */ + for (j=0;j<256;++j) { + str[j] = (char)j; + } + write_to_buf(str, 256, buf); + write_to_buf(str, 256, buf); + test_eq(buf_datalen(buf), 512); + fetch_from_buf(str2, 200, buf); + test_memeq(str, str2, 200); + test_eq(buf_datalen(buf), 312); + memset(str2, 0, sizeof(str2)); + + fetch_from_buf(str2, 256, buf); + test_memeq(str+200, str2, 56); + test_memeq(str, str2+56, 200); + test_eq(buf_datalen(buf), 56); + memset(str2, 0, sizeof(str2)); + /* Okay, now we should be 512 bytes into the 4096-byte buffer. If we add + * another 3584 bytes, we hit the end. */ + for (j=0;j<15;++j) { + write_to_buf(str, 256, buf); + } + assert_buf_ok(buf); + test_eq(buf_datalen(buf), 3896); + fetch_from_buf(str2, 56, buf); + test_eq(buf_datalen(buf), 3840); + test_memeq(str+200, str2, 56); + for (j=0;j<15;++j) { + memset(str2, 0, sizeof(str2)); + fetch_from_buf(str2, 256, buf); + test_memeq(str, str2, 256); + } + test_eq(buf_datalen(buf), 0); + buf_free(buf); + buf = NULL; + + /* Okay, now make sure growing can work. */ + buf = buf_new_with_capacity(16); + //test_eq(buf_capacity(buf), 16); + write_to_buf(str+1, 255, buf); + //test_eq(buf_capacity(buf), 256); + fetch_from_buf(str2, 254, buf); + test_memeq(str+1, str2, 254); + //test_eq(buf_capacity(buf), 256); + assert_buf_ok(buf); + write_to_buf(str, 32, buf); + //test_eq(buf_capacity(buf), 256); + assert_buf_ok(buf); + write_to_buf(str, 256, buf); + assert_buf_ok(buf); + //test_eq(buf_capacity(buf), 512); + test_eq(buf_datalen(buf), 33+256); + fetch_from_buf(str2, 33, buf); + test_eq(*str2, str[255]); + + test_memeq(str2+1, str, 32); + //test_eq(buf_capacity(buf), 512); + test_eq(buf_datalen(buf), 256); + fetch_from_buf(str2, 256, buf); + test_memeq(str, str2, 256); + + /* now try shrinking: case 1. */ + buf_free(buf); + buf = buf_new_with_capacity(33668); + for (j=0;j<67;++j) { + write_to_buf(str,255, buf); + } + //test_eq(buf_capacity(buf), 33668); + test_eq(buf_datalen(buf), 17085); + for (j=0; j < 40; ++j) { + fetch_from_buf(str2, 255,buf); + test_memeq(str2, str, 255); + } + + /* now try shrinking: case 2. */ + buf_free(buf); + buf = buf_new_with_capacity(33668); + for (j=0;j<67;++j) { + write_to_buf(str,255, buf); + } + for (j=0; j < 20; ++j) { + fetch_from_buf(str2, 255,buf); + test_memeq(str2, str, 255); + } + for (j=0;j<80;++j) { + write_to_buf(str,255, buf); + } + //test_eq(buf_capacity(buf),33668); + for (j=0; j < 120; ++j) { + fetch_from_buf(str2, 255,buf); + test_memeq(str2, str, 255); + } + + /* Move from buf to buf. */ + buf_free(buf); + buf = buf_new_with_capacity(4096); + buf2 = buf_new_with_capacity(4096); + for (j=0;j<100;++j) + write_to_buf(str, 255, buf); + test_eq(buf_datalen(buf), 25500); + for (j=0;j<100;++j) { + r = 10; + move_buf_to_buf(buf2, buf, &r); + test_eq(r, 0); + } + test_eq(buf_datalen(buf), 24500); + test_eq(buf_datalen(buf2), 1000); + for (j=0;j<3;++j) { + fetch_from_buf(str2, 255, buf2); + test_memeq(str2, str, 255); + } + r = 8192; /*big move*/ + move_buf_to_buf(buf2, buf, &r); + test_eq(r, 0); + r = 30000; /* incomplete move */ + move_buf_to_buf(buf2, buf, &r); + test_eq(r, 13692); + for (j=0;j<97;++j) { + fetch_from_buf(str2, 255, buf2); + test_memeq(str2, str, 255); + } + buf_free(buf); + buf_free(buf2); + buf = buf2 = NULL; + + buf = buf_new_with_capacity(5); + cp = "Testing. This is a moderately long Testing string."; + for (j = 0; cp[j]; j++) + write_to_buf(cp+j, 1, buf); + test_eq(0, buf_find_string_offset(buf, "Testing", 7)); + test_eq(1, buf_find_string_offset(buf, "esting", 6)); + test_eq(1, buf_find_string_offset(buf, "est", 3)); + test_eq(39, buf_find_string_offset(buf, "ing str", 7)); + test_eq(35, buf_find_string_offset(buf, "Testing str", 11)); + test_eq(32, buf_find_string_offset(buf, "ng ", 3)); + test_eq(43, buf_find_string_offset(buf, "string.", 7)); + test_eq(-1, buf_find_string_offset(buf, "shrdlu", 6)); + test_eq(-1, buf_find_string_offset(buf, "Testing thing", 13)); + test_eq(-1, buf_find_string_offset(buf, "ngx", 3)); + buf_free(buf); + buf = NULL; + + done: + if (buf) + buf_free(buf); + if (buf2) + buf_free(buf2); +} + +/** Run unit tests for the onion handshake code. */ +static void +test_onion_handshake(void) +{ + /* client-side */ + crypto_dh_env_t *c_dh = NULL; + char c_buf[ONIONSKIN_CHALLENGE_LEN]; + char c_keys[40]; + + /* server-side */ + char s_buf[ONIONSKIN_REPLY_LEN]; + char s_keys[40]; + + /* shared */ + crypto_pk_env_t *pk = NULL; + + pk = pk_generate(0); + + /* client handshake 1. */ + memset(c_buf, 0, ONIONSKIN_CHALLENGE_LEN); + test_assert(! onion_skin_create(pk, &c_dh, c_buf)); + + /* server handshake */ + memset(s_buf, 0, ONIONSKIN_REPLY_LEN); + memset(s_keys, 0, 40); + test_assert(! onion_skin_server_handshake(c_buf, pk, NULL, + s_buf, s_keys, 40)); + + /* client handshake 2 */ + memset(c_keys, 0, 40); + test_assert(! onion_skin_client_handshake(c_dh, s_buf, c_keys, 40)); + + if (memcmp(c_keys, s_keys, 40)) { + puts("Aiiiie"); + exit(1); + } + test_memeq(c_keys, s_keys, 40); + memset(s_buf, 0, 40); + test_memneq(c_keys, s_buf, 40); + + done: + if (c_dh) + crypto_dh_free(c_dh); + if (pk) + crypto_free_pk_env(pk); +} + +static void +test_circuit_timeout(void) +{ + /* Plan: + * 1. Generate 1000 samples + * 2. Estimate parameters + * 3. If difference, repeat + * 4. Save state + * 5. load state + * 6. Estimate parameters + * 7. compare differences + */ + circuit_build_times_t initial; + circuit_build_times_t estimate; + circuit_build_times_t final; + double timeout1, timeout2; + or_state_t state; + int i, runs; + double close_ms; + circuit_build_times_init(&initial); + circuit_build_times_init(&estimate); + circuit_build_times_init(&final); + + memset(&state, 0, sizeof(or_state_t)); + + circuitbuild_running_unit_tests(); +#define timeout0 (build_time_t)(30*1000.0) + initial.Xm = 3000; + circuit_build_times_initial_alpha(&initial, + CBT_DEFAULT_QUANTILE_CUTOFF/100.0, + timeout0); + close_ms = MAX(circuit_build_times_calculate_timeout(&initial, + CBT_DEFAULT_CLOSE_QUANTILE/100.0), + CBT_DEFAULT_TIMEOUT_INITIAL_VALUE); + do { + for (i=0; i < CBT_DEFAULT_MIN_CIRCUITS_TO_OBSERVE; i++) { + build_time_t sample = circuit_build_times_generate_sample(&initial,0,1); + + if (sample > close_ms) { + circuit_build_times_add_time(&estimate, CBT_BUILD_ABANDONED); + } else { + circuit_build_times_add_time(&estimate, sample); + } + } + circuit_build_times_update_alpha(&estimate); + timeout1 = circuit_build_times_calculate_timeout(&estimate, + CBT_DEFAULT_QUANTILE_CUTOFF/100.0); + circuit_build_times_set_timeout(&estimate); + log_notice(LD_CIRC, "Timeout1 is %lf, Xm is %d", timeout1, estimate.Xm); + /* 2% error */ + } while (fabs(circuit_build_times_cdf(&initial, timeout0) - + circuit_build_times_cdf(&initial, timeout1)) > 0.02); + + test_assert(estimate.total_build_times <= CBT_NCIRCUITS_TO_OBSERVE); + + circuit_build_times_update_state(&estimate, &state); + test_assert(circuit_build_times_parse_state(&final, &state) == 0); + + circuit_build_times_update_alpha(&final); + timeout2 = circuit_build_times_calculate_timeout(&final, + CBT_DEFAULT_QUANTILE_CUTOFF/100.0); + + circuit_build_times_set_timeout(&final); + log_notice(LD_CIRC, "Timeout2 is %lf, Xm is %d", timeout2, final.Xm); + + /* 5% here because some accuracy is lost due to histogram conversion */ + test_assert(fabs(circuit_build_times_cdf(&initial, timeout0) - + circuit_build_times_cdf(&initial, timeout2)) < 0.05); + + for (runs = 0; runs < 50; runs++) { + int build_times_idx = 0; + int total_build_times = 0; + + final.close_ms = final.timeout_ms = CBT_DEFAULT_TIMEOUT_INITIAL_VALUE; + estimate.close_ms = estimate.timeout_ms + = CBT_DEFAULT_TIMEOUT_INITIAL_VALUE; + + for (i = 0; i < CBT_DEFAULT_RECENT_CIRCUITS*2; i++) { + circuit_build_times_network_circ_success(&estimate); + circuit_build_times_add_time(&estimate, + circuit_build_times_generate_sample(&estimate, 0, + CBT_DEFAULT_QUANTILE_CUTOFF/100.0)); + + circuit_build_times_network_circ_success(&estimate); + circuit_build_times_add_time(&final, + circuit_build_times_generate_sample(&final, 0, + CBT_DEFAULT_QUANTILE_CUTOFF/100.0)); + } + + test_assert(!circuit_build_times_network_check_changed(&estimate)); + test_assert(!circuit_build_times_network_check_changed(&final)); + + /* Reset liveness to be non-live */ + final.liveness.network_last_live = 0; + estimate.liveness.network_last_live = 0; + + build_times_idx = estimate.build_times_idx; + total_build_times = estimate.total_build_times; + + test_assert(circuit_build_times_network_check_live(&estimate)); + test_assert(circuit_build_times_network_check_live(&final)); + + circuit_build_times_count_close(&estimate, 0, + (time_t)(approx_time()-estimate.close_ms/1000.0-1)); + circuit_build_times_count_close(&final, 0, + (time_t)(approx_time()-final.close_ms/1000.0-1)); + + test_assert(!circuit_build_times_network_check_live(&estimate)); + test_assert(!circuit_build_times_network_check_live(&final)); + + log_info(LD_CIRC, "idx: %d %d, tot: %d %d", + build_times_idx, estimate.build_times_idx, + total_build_times, estimate.total_build_times); + + /* Check rollback index. Should match top of loop. */ + test_assert(build_times_idx == estimate.build_times_idx); + // This can fail if estimate.total_build_times == 1000, because + // in that case, rewind actually causes us to lose timeouts + if (total_build_times != CBT_NCIRCUITS_TO_OBSERVE) + test_assert(total_build_times == estimate.total_build_times); + + /* Now simulate that the network has become live and we need + * a change */ + circuit_build_times_network_is_live(&estimate); + circuit_build_times_network_is_live(&final); + + for (i = 0; i < CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT; i++) { + circuit_build_times_count_timeout(&estimate, 1); + + if (i < CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT-1) { + circuit_build_times_count_timeout(&final, 1); + } + } + + test_assert(estimate.liveness.after_firsthop_idx == 0); + test_assert(final.liveness.after_firsthop_idx == + CBT_DEFAULT_MAX_RECENT_TIMEOUT_COUNT-1); + + test_assert(circuit_build_times_network_check_live(&estimate)); + test_assert(circuit_build_times_network_check_live(&final)); + + circuit_build_times_count_timeout(&final, 1); + } + + done: + return; +} + +/** Helper: Parse the exit policy string in <b>policy_str</b>, and make sure + * that policies_summarize() produces the string <b>expected_summary</b> from + * it. */ +static void +test_policy_summary_helper(const char *policy_str, + const char *expected_summary) +{ + config_line_t line; + smartlist_t *policy = smartlist_create(); + char *summary = NULL; + int r; + + line.key = (char*)"foo"; + line.value = (char *)policy_str; + line.next = NULL; + + r = policies_parse_exit_policy(&line, &policy, 0, NULL, 1); + test_eq(r, 0); + summary = policy_summarize(policy); + + test_assert(summary != NULL); + test_streq(summary, expected_summary); + + done: + tor_free(summary); + if (policy) + addr_policy_list_free(policy); +} + +/** Run unit tests for generating summary lines of exit policies */ +static void +test_policies(void) +{ + int i; + smartlist_t *policy = NULL, *policy2 = NULL, *policy3 = NULL, + *policy4 = NULL, *policy5 = NULL, *policy6 = NULL, + *policy7 = NULL; + addr_policy_t *p; + tor_addr_t tar; + config_line_t line; + smartlist_t *sm = NULL; + char *policy_str = NULL; + + policy = smartlist_create(); + + p = router_parse_addr_policy_item_from_string("reject 192.168.0.0/16:*",-1); + test_assert(p != NULL); + test_eq(ADDR_POLICY_REJECT, p->policy_type); + tor_addr_from_ipv4h(&tar, 0xc0a80000u); + test_eq(0, tor_addr_compare(&p->addr, &tar, CMP_EXACT)); + test_eq(16, p->maskbits); + test_eq(1, p->prt_min); + test_eq(65535, p->prt_max); + + smartlist_add(policy, p); + + test_assert(ADDR_POLICY_ACCEPTED == + compare_addr_to_addr_policy(0x01020304u, 2, policy)); + test_assert(ADDR_POLICY_PROBABLY_ACCEPTED == + compare_addr_to_addr_policy(0, 2, policy)); + test_assert(ADDR_POLICY_REJECTED == + compare_addr_to_addr_policy(0xc0a80102, 2, policy)); + + test_assert(0 == policies_parse_exit_policy(NULL, &policy2, 1, NULL, 1)); + test_assert(policy2); + + policy3 = smartlist_create(); + p = router_parse_addr_policy_item_from_string("reject *:*",-1); + test_assert(p != NULL); + smartlist_add(policy3, p); + p = router_parse_addr_policy_item_from_string("accept *:*",-1); + test_assert(p != NULL); + smartlist_add(policy3, p); + + policy4 = smartlist_create(); + p = router_parse_addr_policy_item_from_string("accept *:443",-1); + test_assert(p != NULL); + smartlist_add(policy4, p); + p = router_parse_addr_policy_item_from_string("accept *:443",-1); + test_assert(p != NULL); + smartlist_add(policy4, p); + + policy5 = smartlist_create(); + p = router_parse_addr_policy_item_from_string("reject 0.0.0.0/8:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject 169.254.0.0/16:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject 127.0.0.0/8:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject 192.168.0.0/16:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject 10.0.0.0/8:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject 172.16.0.0/12:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject 80.190.250.90:*",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject *:1-65534",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("reject *:65535",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + p = router_parse_addr_policy_item_from_string("accept *:1-65535",-1); + test_assert(p != NULL); + smartlist_add(policy5, p); + + policy6 = smartlist_create(); + p = router_parse_addr_policy_item_from_string("accept 43.3.0.0/9:*",-1); + test_assert(p != NULL); + smartlist_add(policy6, p); + + policy7 = smartlist_create(); + p = router_parse_addr_policy_item_from_string("accept 0.0.0.0/8:*",-1); + test_assert(p != NULL); + smartlist_add(policy7, p); + + test_assert(!exit_policy_is_general_exit(policy)); + test_assert(exit_policy_is_general_exit(policy2)); + test_assert(!exit_policy_is_general_exit(NULL)); + test_assert(!exit_policy_is_general_exit(policy3)); + test_assert(!exit_policy_is_general_exit(policy4)); + test_assert(!exit_policy_is_general_exit(policy5)); + test_assert(!exit_policy_is_general_exit(policy6)); + test_assert(!exit_policy_is_general_exit(policy7)); + + test_assert(cmp_addr_policies(policy, policy2)); + test_assert(cmp_addr_policies(policy, NULL)); + test_assert(!cmp_addr_policies(policy2, policy2)); + test_assert(!cmp_addr_policies(NULL, NULL)); + + test_assert(!policy_is_reject_star(policy2)); + test_assert(policy_is_reject_star(policy)); + test_assert(policy_is_reject_star(NULL)); + + addr_policy_list_free(policy); + policy = NULL; + + /* make sure compacting logic works. */ + policy = NULL; + line.key = (char*)"foo"; + line.value = (char*)"accept *:80,reject private:*,reject *:*"; + line.next = NULL; + test_assert(0 == policies_parse_exit_policy(&line, &policy, 0, NULL, 1)); + test_assert(policy); + //test_streq(policy->string, "accept *:80"); + //test_streq(policy->next->string, "reject *:*"); + test_eq(smartlist_len(policy), 2); + + /* test policy summaries */ + /* check if we properly ignore private IP addresses */ + test_policy_summary_helper("reject 192.168.0.0/16:*," + "reject 0.0.0.0/8:*," + "reject 10.0.0.0/8:*," + "accept *:10-30," + "accept *:90," + "reject *:*", + "accept 10-30,90"); + /* check all accept policies, and proper counting of rejects */ + test_policy_summary_helper("reject 11.0.0.0/9:80," + "reject 12.0.0.0/9:80," + "reject 13.0.0.0/9:80," + "reject 14.0.0.0/9:80," + "accept *:*", "accept 1-65535"); + test_policy_summary_helper("reject 11.0.0.0/9:80," + "reject 12.0.0.0/9:80," + "reject 13.0.0.0/9:80," + "reject 14.0.0.0/9:80," + "reject 15.0.0.0:81," + "accept *:*", "accept 1-65535"); + test_policy_summary_helper("reject 11.0.0.0/9:80," + "reject 12.0.0.0/9:80," + "reject 13.0.0.0/9:80," + "reject 14.0.0.0/9:80," + "reject 15.0.0.0:80," + "accept *:*", + "reject 80"); + /* no exits */ + test_policy_summary_helper("accept 11.0.0.0/9:80," + "reject *:*", + "reject 1-65535"); + /* port merging */ + test_policy_summary_helper("accept *:80," + "accept *:81," + "accept *:100-110," + "accept *:111," + "reject *:*", + "accept 80-81,100-111"); + /* border ports */ + test_policy_summary_helper("accept *:1," + "accept *:3," + "accept *:65535," + "reject *:*", + "accept 1,3,65535"); + /* holes */ + test_policy_summary_helper("accept *:1," + "accept *:3," + "accept *:5," + "accept *:7," + "reject *:*", + "accept 1,3,5,7"); + test_policy_summary_helper("reject *:1," + "reject *:3," + "reject *:5," + "reject *:7," + "accept *:*", + "reject 1,3,5,7"); + + /* truncation ports */ + sm = smartlist_create(); + for (i=1; i<2000; i+=2) { + char buf[POLICY_BUF_LEN]; + tor_snprintf(buf, sizeof(buf), "reject *:%d", i); + smartlist_add(sm, tor_strdup(buf)); + } + smartlist_add(sm, tor_strdup("accept *:*")); + policy_str = smartlist_join_strings(sm, ",", 0, NULL); + test_policy_summary_helper( policy_str, + "accept 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44," + "46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90," + "92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128," + "130,132,134,136,138,140,142,144,146,148,150,152,154,156,158,160,162,164," + "166,168,170,172,174,176,178,180,182,184,186,188,190,192,194,196,198,200," + "202,204,206,208,210,212,214,216,218,220,222,224,226,228,230,232,234,236," + "238,240,242,244,246,248,250,252,254,256,258,260,262,264,266,268,270,272," + "274,276,278,280,282,284,286,288,290,292,294,296,298,300,302,304,306,308," + "310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344," + "346,348,350,352,354,356,358,360,362,364,366,368,370,372,374,376,378,380," + "382,384,386,388,390,392,394,396,398,400,402,404,406,408,410,412,414,416," + "418,420,422,424,426,428,430,432,434,436,438,440,442,444,446,448,450,452," + "454,456,458,460,462,464,466,468,470,472,474,476,478,480,482,484,486,488," + "490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522"); + + done: + addr_policy_list_free(policy); + addr_policy_list_free(policy2); + addr_policy_list_free(policy3); + addr_policy_list_free(policy4); + addr_policy_list_free(policy5); + addr_policy_list_free(policy6); + addr_policy_list_free(policy7); + tor_free(policy_str); + if (sm) { + SMARTLIST_FOREACH(sm, char *, s, tor_free(s)); + smartlist_free(sm); + } +} + +/** Run AES performance benchmarks. */ +static void +bench_aes(void) +{ + int len, i; + char *b1, *b2; + crypto_cipher_env_t *c; + struct timeval start, end; + const int iters = 100000; + uint64_t nsec; + c = crypto_new_cipher_env(); + crypto_cipher_generate_key(c); + crypto_cipher_encrypt_init_cipher(c); + for (len = 1; len <= 8192; len *= 2) { + b1 = tor_malloc_zero(len); + b2 = tor_malloc_zero(len); + tor_gettimeofday(&start); + for (i = 0; i < iters; ++i) { + crypto_cipher_encrypt(c, b1, b2, len); + } + tor_gettimeofday(&end); + tor_free(b1); + tor_free(b2); + nsec = (uint64_t) tv_udiff(&start,&end); + nsec *= 1000; + nsec /= (iters*len); + printf("%d bytes: "U64_FORMAT" nsec per byte\n", len, + U64_PRINTF_ARG(nsec)); + } + crypto_free_cipher_env(c); +} + +/** Run digestmap_t performance benchmarks. */ +static void +bench_dmap(void) +{ + smartlist_t *sl = smartlist_create(); + smartlist_t *sl2 = smartlist_create(); + struct timeval start, end, pt2, pt3, pt4; + const int iters = 10000; + const int elts = 4000; + const int fpostests = 1000000; + char d[20]; + int i,n=0, fp = 0; + digestmap_t *dm = digestmap_new(); + digestset_t *ds = digestset_new(elts); + + for (i = 0; i < elts; ++i) { + crypto_rand(d, 20); + smartlist_add(sl, tor_memdup(d, 20)); + } + for (i = 0; i < elts; ++i) { + crypto_rand(d, 20); + smartlist_add(sl2, tor_memdup(d, 20)); + } + printf("nbits=%d\n", ds->mask+1); + + tor_gettimeofday(&start); + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, digestmap_set(dm, cp, (void*)1)); + } + tor_gettimeofday(&pt2); + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, digestmap_get(dm, cp)); + SMARTLIST_FOREACH(sl2, const char *, cp, digestmap_get(dm, cp)); + } + tor_gettimeofday(&pt3); + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, digestset_add(ds, cp)); + } + tor_gettimeofday(&pt4); + for (i = 0; i < iters; ++i) { + SMARTLIST_FOREACH(sl, const char *, cp, n += digestset_isin(ds, cp)); + SMARTLIST_FOREACH(sl2, const char *, cp, n += digestset_isin(ds, cp)); + } + tor_gettimeofday(&end); + + for (i = 0; i < fpostests; ++i) { + crypto_rand(d, 20); + if (digestset_isin(ds, d)) ++fp; + } + + printf("%ld\n",(unsigned long)tv_udiff(&start, &pt2)); + printf("%ld\n",(unsigned long)tv_udiff(&pt2, &pt3)); + printf("%ld\n",(unsigned long)tv_udiff(&pt3, &pt4)); + printf("%ld\n",(unsigned long)tv_udiff(&pt4, &end)); + printf("-- %d\n", n); + printf("++ %f\n", fp/(double)fpostests); + digestmap_free(dm, NULL); + digestset_free(ds); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(sl2, char *, cp, tor_free(cp)); + smartlist_free(sl); + smartlist_free(sl2); +} + +/** Test encoding and parsing of rendezvous service descriptors. */ +static void +test_rend_fns(void) +{ + rend_service_descriptor_t *generated = NULL, *parsed = NULL; + char service_id[DIGEST_LEN]; + char service_id_base32[REND_SERVICE_ID_LEN_BASE32+1]; + const char *next_desc; + smartlist_t *descs = smartlist_create(); + char computed_desc_id[DIGEST_LEN]; + char parsed_desc_id[DIGEST_LEN]; + crypto_pk_env_t *pk1 = NULL, *pk2 = NULL; + time_t now; + char *intro_points_encrypted = NULL; + size_t intro_points_size; + size_t encoded_size; + int i; + char address1[] = "fooaddress.onion"; + char address2[] = "aaaaaaaaaaaaaaaa.onion"; + char address3[] = "fooaddress.exit"; + char address4[] = "www.torproject.org"; + + test_assert(BAD_HOSTNAME == parse_extended_hostname(address1, 1)); + test_assert(ONION_HOSTNAME == parse_extended_hostname(address2, 1)); + test_assert(EXIT_HOSTNAME == parse_extended_hostname(address3, 1)); + test_assert(NORMAL_HOSTNAME == parse_extended_hostname(address4, 1)); + + pk1 = pk_generate(0); + pk2 = pk_generate(1); + generated = tor_malloc_zero(sizeof(rend_service_descriptor_t)); + generated->pk = crypto_pk_dup_key(pk1); + crypto_pk_get_digest(generated->pk, service_id); + base32_encode(service_id_base32, REND_SERVICE_ID_LEN_BASE32+1, + service_id, REND_SERVICE_ID_LEN); + now = time(NULL); + generated->timestamp = now; + generated->version = 2; + generated->protocols = 42; + generated->intro_nodes = smartlist_create(); + + for (i = 0; i < 3; i++) { + rend_intro_point_t *intro = tor_malloc_zero(sizeof(rend_intro_point_t)); + crypto_pk_env_t *okey = pk_generate(2 + i); + intro->extend_info = tor_malloc_zero(sizeof(extend_info_t)); + intro->extend_info->onion_key = okey; + crypto_pk_get_digest(intro->extend_info->onion_key, + intro->extend_info->identity_digest); + //crypto_rand(info->identity_digest, DIGEST_LEN); /* Would this work? */ + intro->extend_info->nickname[0] = '$'; + base16_encode(intro->extend_info->nickname + 1, + sizeof(intro->extend_info->nickname) - 1, + intro->extend_info->identity_digest, DIGEST_LEN); + /* Does not cover all IP addresses. */ + tor_addr_from_ipv4h(&intro->extend_info->addr, crypto_rand_int(65536)); + intro->extend_info->port = 1 + crypto_rand_int(65535); + intro->intro_key = crypto_pk_dup_key(pk2); + smartlist_add(generated->intro_nodes, intro); + } + test_assert(rend_encode_v2_descriptors(descs, generated, now, 0, + REND_NO_AUTH, NULL, NULL) > 0); + test_assert(rend_compute_v2_desc_id(computed_desc_id, service_id_base32, + NULL, now, 0) == 0); + test_memeq(((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0))->desc_id, computed_desc_id, DIGEST_LEN); + test_assert(rend_parse_v2_service_descriptor(&parsed, parsed_desc_id, + &intro_points_encrypted, + &intro_points_size, + &encoded_size, + &next_desc, + ((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0))->desc_str) == 0); + test_assert(parsed); + test_memeq(((rend_encoded_v2_service_descriptor_t *) + smartlist_get(descs, 0))->desc_id, parsed_desc_id, DIGEST_LEN); + test_eq(rend_parse_introduction_points(parsed, intro_points_encrypted, + intro_points_size), 3); + test_assert(!crypto_pk_cmp_keys(generated->pk, parsed->pk)); + test_eq(parsed->timestamp, now); + test_eq(parsed->version, 2); + test_eq(parsed->protocols, 42); + test_eq(smartlist_len(parsed->intro_nodes), 3); + for (i = 0; i < smartlist_len(parsed->intro_nodes); i++) { + rend_intro_point_t *par_intro = smartlist_get(parsed->intro_nodes, i), + *gen_intro = smartlist_get(generated->intro_nodes, i); + extend_info_t *par_info = par_intro->extend_info; + extend_info_t *gen_info = gen_intro->extend_info; + test_assert(!crypto_pk_cmp_keys(gen_info->onion_key, par_info->onion_key)); + test_memeq(gen_info->identity_digest, par_info->identity_digest, + DIGEST_LEN); + test_streq(gen_info->nickname, par_info->nickname); + test_assert(tor_addr_eq(&gen_info->addr, &par_info->addr)); + test_eq(gen_info->port, par_info->port); + } + + rend_service_descriptor_free(parsed); + rend_service_descriptor_free(generated); + parsed = generated = NULL; + + done: + if (descs) { + for (i = 0; i < smartlist_len(descs); i++) + rend_encoded_v2_service_descriptor_free(smartlist_get(descs, i)); + smartlist_free(descs); + } + if (parsed) + rend_service_descriptor_free(parsed); + if (generated) + rend_service_descriptor_free(generated); + if (pk1) + crypto_free_pk_env(pk1); + if (pk2) + crypto_free_pk_env(pk2); + tor_free(intro_points_encrypted); +} + +/** Run unit tests for GeoIP code. */ +static void +test_geoip(void) +{ + int i, j; + time_t now = time(NULL); + char *s = NULL; + + /* Populate the DB a bit. Add these in order, since we can't do the final + * 'sort' step. These aren't very good IP addresses, but they're perfectly + * fine uint32_t values. */ + test_eq(0, geoip_parse_entry("10,50,AB")); + test_eq(0, geoip_parse_entry("52,90,XY")); + test_eq(0, geoip_parse_entry("95,100,AB")); + test_eq(0, geoip_parse_entry("\"105\",\"140\",\"ZZ\"")); + test_eq(0, geoip_parse_entry("\"150\",\"190\",\"XY\"")); + test_eq(0, geoip_parse_entry("\"200\",\"250\",\"AB\"")); + + /* We should have 4 countries: ??, ab, xy, zz. */ + test_eq(4, geoip_get_n_countries()); + /* Make sure that country ID actually works. */ +#define NAMEFOR(x) geoip_get_country_name(geoip_get_country_by_ip(x)) + test_streq("??", NAMEFOR(3)); + test_eq(0, geoip_get_country_by_ip(3)); + test_streq("ab", NAMEFOR(32)); + test_streq("??", NAMEFOR(5)); + test_streq("??", NAMEFOR(51)); + test_streq("xy", NAMEFOR(150)); + test_streq("xy", NAMEFOR(190)); + test_streq("??", NAMEFOR(2000)); +#undef NAMEFOR + + get_options()->BridgeRelay = 1; + get_options()->BridgeRecordUsageByCountry = 1; + /* Put 9 observations in AB... */ + for (i=32; i < 40; ++i) + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now-7200); + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, 225, now-7200); + /* and 3 observations in XY, several times. */ + for (j=0; j < 10; ++j) + for (i=52; i < 55; ++i) + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now-3600); + /* and 17 observations in ZZ... */ + for (i=110; i < 127; ++i) + geoip_note_client_seen(GEOIP_CLIENT_CONNECT, i, now); + s = geoip_get_client_history(GEOIP_CLIENT_CONNECT); + test_assert(s); + test_streq("zz=24,ab=16,xy=8", s); + tor_free(s); + + /* Now clear out all the AB observations. */ + geoip_remove_old_clients(now-6000); + s = geoip_get_client_history(GEOIP_CLIENT_CONNECT); + test_assert(s); + test_streq("zz=24,xy=8", s); + + done: + tor_free(s); +} + +/** Run unit tests for stats code. */ +static void +test_stats(void) +{ + time_t now = 1281533250; /* 2010-08-11 13:27:30 UTC */ + char *s = NULL; + int i; + + /* We shouldn't collect exit stats without initializing them. */ + rep_hist_note_exit_stream_opened(80); + rep_hist_note_exit_bytes(80, 100, 10000); + s = rep_hist_format_exit_stats(now + 86400); + test_assert(!s); + + /* Initialize stats, note some streams and bytes, and generate history + * string. */ + rep_hist_exit_stats_init(now); + rep_hist_note_exit_stream_opened(80); + rep_hist_note_exit_bytes(80, 100, 10000); + rep_hist_note_exit_stream_opened(443); + rep_hist_note_exit_bytes(443, 100, 10000); + rep_hist_note_exit_bytes(443, 100, 10000); + s = rep_hist_format_exit_stats(now + 86400); + test_streq("exit-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "exit-kibibytes-written 80=1,443=1,other=0\n" + "exit-kibibytes-read 80=10,443=20,other=0\n" + "exit-streams-opened 80=4,443=4,other=0\n", s); + tor_free(s); + + /* Add a few bytes on 10 more ports and ensure that only the top 10 + * ports are contained in the history string. */ + for (i = 50; i < 60; i++) { + rep_hist_note_exit_bytes(i, i, i); + rep_hist_note_exit_stream_opened(i); + } + s = rep_hist_format_exit_stats(now + 86400); + test_streq("exit-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "exit-kibibytes-written 52=1,53=1,54=1,55=1,56=1,57=1,58=1," + "59=1,80=1,443=1,other=1\n" + "exit-kibibytes-read 52=1,53=1,54=1,55=1,56=1,57=1,58=1," + "59=1,80=10,443=20,other=1\n" + "exit-streams-opened 52=4,53=4,54=4,55=4,56=4,57=4,58=4," + "59=4,80=4,443=4,other=4\n", s); + tor_free(s); + + /* Stop collecting stats, add some bytes, and ensure we don't generate + * a history string. */ + rep_hist_exit_stats_term(); + rep_hist_note_exit_bytes(80, 100, 10000); + s = rep_hist_format_exit_stats(now + 86400); + test_assert(!s); + + /* Re-start stats, add some bytes, reset stats, and see what history we + * get when observing no streams or bytes at all. */ + rep_hist_exit_stats_init(now); + rep_hist_note_exit_stream_opened(80); + rep_hist_note_exit_bytes(80, 100, 10000); + rep_hist_reset_exit_stats(now); + s = rep_hist_format_exit_stats(now + 86400); + test_streq("exit-stats-end 2010-08-12 13:27:30 (86400 s)\n" + "exit-kibibytes-written other=0\n" + "exit-kibibytes-read other=0\n" + "exit-streams-opened other=0\n", s); + + done: + tor_free(s); +} + +static void * +legacy_test_setup(const struct testcase_t *testcase) +{ + return testcase->setup_data; +} + +void +legacy_test_helper(void *data) +{ + void (*fn)(void) = data; + fn(); +} + +static int +legacy_test_cleanup(const struct testcase_t *testcase, void *ptr) +{ + (void)ptr; + (void)testcase; + return 1; +} + +const struct testcase_setup_t legacy_setup = { + legacy_test_setup, legacy_test_cleanup +}; + +#define ENT(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_ ## name } +#define SUBENT(group, name) \ + { #group "_" #name, legacy_test_helper, 0, &legacy_setup, \ + test_ ## group ## _ ## name } +#define DISABLED(name) \ + { #name, legacy_test_helper, TT_SKIP, &legacy_setup, name } +#define FORK(name) \ + { #name, legacy_test_helper, TT_FORK, &legacy_setup, test_ ## name } + +static struct testcase_t test_array[] = { + ENT(buffers), + ENT(onion_handshake), + ENT(circuit_timeout), + ENT(policies), + ENT(rend_fns), + ENT(geoip), + FORK(stats), + + DISABLED(bench_aes), + DISABLED(bench_dmap), + END_OF_TESTCASES +}; + +extern struct testcase_t addr_tests[]; +extern struct testcase_t crypto_tests[]; +extern struct testcase_t container_tests[]; +extern struct testcase_t util_tests[]; +extern struct testcase_t dir_tests[]; + +static struct testgroup_t testgroups[] = { + { "", test_array }, + { "addr/", addr_tests }, + { "crypto/", crypto_tests }, + { "container/", container_tests }, + { "util/", util_tests }, + { "dir/", dir_tests }, + END_OF_GROUPS +}; + +/** Main entry point for unit test code: parse the command line, and run + * some unit tests. */ +int +main(int c, const char **v) +{ + or_options_t *options; + char *errmsg = NULL; + int i, i_out; + int loglevel = LOG_ERR; + +#ifdef USE_DMALLOC + { + int r = CRYPTO_set_mem_ex_functions(_tor_malloc, _tor_realloc, _tor_free); + tor_assert(r); + } +#endif + + update_approx_time(time(NULL)); + options = options_new(); + tor_threads_init(); + init_logging(); + + for (i_out = i = 1; i < c; ++i) { + if (!strcmp(v[i], "--warn")) { + loglevel = LOG_WARN; + } else if (!strcmp(v[i], "--notice")) { + loglevel = LOG_NOTICE; + } else if (!strcmp(v[i], "--info")) { + loglevel = LOG_INFO; + } else if (!strcmp(v[i], "--debug")) { + loglevel = LOG_DEBUG; + } else { + v[i_out++] = v[i]; + } + } + c = i_out; + + { + log_severity_list_t s; + memset(&s, 0, sizeof(s)); + set_log_severity_config(loglevel, LOG_ERR, &s); + add_stream_log(&s, "", fileno(stdout)); + } + + options->command = CMD_RUN_UNITTESTS; + crypto_global_init(0, NULL, NULL); + rep_hist_init(); + network_init(); + setup_directory(); + options_init(options); + options->DataDirectory = tor_strdup(temp_dir); + options->EntryStatistics = 1; + if (set_options(options, &errmsg) < 0) { + printf("Failed to set initial options: %s\n", errmsg); + tor_free(errmsg); + return 1; + } + + crypto_seed_rng(1); + + atexit(remove_directory); + + have_failed = (tinytest_main(c, v, testgroups) != 0); + + free_pregenerated_keys(); +#ifdef USE_DMALLOC + tor_free_all(0); + dmalloc_log_unfreed(); +#endif + + if (have_failed) + return 1; + else + return 0; +} + diff --git a/src/test/test.h b/src/test/test.h new file mode 100644 index 000000000..f7ae46ce6 --- /dev/null +++ b/src/test/test.h @@ -0,0 +1,75 @@ +/* Copyright (c) 2001-2003, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef _TOR_TEST_H +#define _TOR_TEST_H + +/** + * \file test.h + * \brief Macros and functions used by unit tests. + */ + +#include "compat.h" +#include "tinytest.h" +#define TT_EXIT_TEST_FUNCTION STMT_BEGIN goto done; STMT_END +#include "tinytest_macros.h" + +#ifdef __GNUC__ +#define PRETTY_FUNCTION __PRETTY_FUNCTION__ +#else +#define PRETTY_FUNCTION "" +#endif + +#define test_fail_msg(msg) TT_DIE((msg)) + +#define test_fail() test_fail_msg("Assertion failed.") + +#define test_assert(expr) tt_assert(expr) + +#define test_eq(expr1, expr2) tt_int_op((expr1), ==, (expr2)) +#define test_eq_ptr(expr1, expr2) tt_ptr_op((expr1), ==, (expr2)) +#define test_neq(expr1, expr2) tt_int_op((expr1), !=, (expr2)) +#define test_neq_ptr(expr1, expr2) tt_ptr_op((expr1), !=, (expr2)) +#define test_streq(expr1, expr2) tt_str_op((expr1), ==, (expr2)) +#define test_strneq(expr1, expr2) tt_str_op((expr1), !=, (expr2)) +#define test_streq(expr1, expr2) tt_str_op((expr1), ==, (expr2)) + +#define test_mem_op(expr1, op, expr2, len) \ + tt_assert_test_fmt_type(expr1,expr2,#expr1" "#op" "#expr2, \ + const char *, \ + (memcmp(_val1, _val2, len) op 0), \ + char *, "%s", \ + { size_t printlen = (len)*2+1; \ + _print = tor_malloc(printlen); \ + base16_encode(_print, printlen, _value, \ + (len)); }, \ + { tor_free(_print); } \ + ); + +#define test_memeq(expr1, expr2, len) test_mem_op((expr1), ==, (expr2), len) +#define test_memneq(expr1, expr2, len) test_mem_op((expr1), !=, (expr2), len) + +/* As test_mem_op, but decodes 'hex' before comparing. There must be a + * local char* variable called mem_op_hex_tmp for this to work. */ +#define test_mem_op_hex(expr1, op, hex) \ + STMT_BEGIN \ + size_t length = strlen(hex); \ + tor_free(mem_op_hex_tmp); \ + mem_op_hex_tmp = tor_malloc(length/2); \ + tor_assert((length&1)==0); \ + base16_decode(mem_op_hex_tmp, length/2, hex, length); \ + test_mem_op(expr1, op, mem_op_hex_tmp, length/2); \ + STMT_END + +#define test_memeq_hex(expr1, hex) test_mem_op_hex(expr1, ==, hex) + +const char *get_fname(const char *name); +crypto_pk_env_t *pk_generate(int idx); + +void legacy_test_helper(void *data); +extern const struct testcase_setup_t legacy_setup; + +#endif + diff --git a/src/test/test_addr.c b/src/test/test_addr.c new file mode 100644 index 000000000..6db4ee248 --- /dev/null +++ b/src/test/test_addr.c @@ -0,0 +1,497 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "or.h" +#include "test.h" + +static void +test_addr_basic(void) +{ + uint32_t u32; + uint16_t u16; + char *cp; + + /* Test parse_addr_port */ + cp = NULL; u32 = 3; u16 = 3; + test_assert(!parse_addr_port(LOG_WARN, "1.2.3.4", &cp, &u32, &u16)); + test_streq(cp, "1.2.3.4"); + test_eq(u32, 0x01020304u); + test_eq(u16, 0); + tor_free(cp); + test_assert(!parse_addr_port(LOG_WARN, "4.3.2.1:99", &cp, &u32, &u16)); + test_streq(cp, "4.3.2.1"); + test_eq(u32, 0x04030201u); + test_eq(u16, 99); + tor_free(cp); + test_assert(!parse_addr_port(LOG_WARN, "nonexistent.address:4040", + &cp, NULL, &u16)); + test_streq(cp, "nonexistent.address"); + test_eq(u16, 4040); + tor_free(cp); + test_assert(!parse_addr_port(LOG_WARN, "localhost:9999", &cp, &u32, &u16)); + test_streq(cp, "localhost"); + test_eq(u32, 0x7f000001u); + test_eq(u16, 9999); + tor_free(cp); + u32 = 3; + test_assert(!parse_addr_port(LOG_WARN, "localhost", NULL, &u32, &u16)); + test_eq(cp, NULL); + test_eq(u32, 0x7f000001u); + test_eq(u16, 0); + tor_free(cp); + test_eq(0, addr_mask_get_bits(0x0u)); + test_eq(32, addr_mask_get_bits(0xFFFFFFFFu)); + test_eq(16, addr_mask_get_bits(0xFFFF0000u)); + test_eq(31, addr_mask_get_bits(0xFFFFFFFEu)); + test_eq(1, addr_mask_get_bits(0x80000000u)); + + /* Test inet_ntop */ + { + char tmpbuf[TOR_ADDR_BUF_LEN]; + const char *ip = "176.192.208.224"; + struct in_addr in; + tor_inet_pton(AF_INET, ip, &in); + tor_inet_ntop(AF_INET, &in, tmpbuf, sizeof(tmpbuf)); + test_streq(tmpbuf, ip); + } + + done: + ; +} + +#define _test_op_ip6(a,op,b,e1,e2) \ + STMT_BEGIN \ + tt_assert_test_fmt_type(a,b,e1" "#op" "e2,struct in6_addr*, \ + (memcmp(_val1->s6_addr, _val2->s6_addr, 16) op 0), \ + char *, "%s", \ + { int i; char *cp; \ + cp = _print = tor_malloc(64); \ + for (i=0;i<16;++i) { \ + tor_snprintf(cp, 3,"%02x", (unsigned)_value->s6_addr[i]);\ + cp += 2; \ + if (i != 15) *cp++ = ':'; \ + } \ + }, { tor_free(_print); } \ + ); \ + STMT_END + +/** Helper: Assert that two strings both decode as IPv6 addresses with + * tor_inet_pton(), and both decode to the same address. */ +#define test_pton6_same(a,b) STMT_BEGIN \ + test_eq(tor_inet_pton(AF_INET6, a, &a1), 1); \ + test_eq(tor_inet_pton(AF_INET6, b, &a2), 1); \ + _test_op_ip6(&a1,==,&a2,#a,#b); \ + STMT_END + +/** Helper: Assert that <b>a</b> is recognized as a bad IPv6 address by + * tor_inet_pton(). */ +#define test_pton6_bad(a) \ + test_eq(0, tor_inet_pton(AF_INET6, a, &a1)) + +/** Helper: assert that <b>a</b>, when parsed by tor_inet_pton() and displayed + * with tor_inet_ntop(), yields <b>b</b>. Also assert that <b>b</b> parses to + * the same value as <b>a</b>. */ +#define test_ntop6_reduces(a,b) STMT_BEGIN \ + test_eq(tor_inet_pton(AF_INET6, a, &a1), 1); \ + test_streq(tor_inet_ntop(AF_INET6, &a1, buf, sizeof(buf)), b); \ + test_eq(tor_inet_pton(AF_INET6, b, &a2), 1); \ + _test_op_ip6(&a1, ==, &a2, a, b); \ + STMT_END + +/** Helper: assert that <b>a</b> parses by tor_inet_pton() into a address that + * passes tor_addr_is_internal() with <b>for_listening</b>. */ +#define test_internal_ip(a,for_listening) STMT_BEGIN \ + test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ + t1.family = AF_INET6; \ + if (!tor_addr_is_internal(&t1, for_listening)) \ + test_fail_msg( a "was not internal."); \ + STMT_END + +/** Helper: assert that <b>a</b> parses by tor_inet_pton() into a address that + * does not pass tor_addr_is_internal() with <b>for_listening</b>. */ +#define test_external_ip(a,for_listening) STMT_BEGIN \ + test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ + t1.family = AF_INET6; \ + if (tor_addr_is_internal(&t1, for_listening)) \ + test_fail_msg(a "was not external."); \ + STMT_END + +/** Helper: Assert that <b>a</b> and <b>b</b>, when parsed by + * tor_inet_pton(), give addresses that compare in the order defined by + * <b>op</b> with tor_addr_compare(). */ +#define test_addr_compare(a, op, b) STMT_BEGIN \ + test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ + test_eq(tor_inet_pton(AF_INET6, b, &t2.addr.in6_addr), 1); \ + t1.family = t2.family = AF_INET6; \ + r = tor_addr_compare(&t1,&t2,CMP_SEMANTIC); \ + if (!(r op 0)) \ + test_fail_msg("failed: tor_addr_compare("a","b") "#op" 0"); \ + STMT_END + +/** Helper: Assert that <b>a</b> and <b>b</b>, when parsed by + * tor_inet_pton(), give addresses that compare in the order defined by + * <b>op</b> with tor_addr_compare_masked() with <b>m</b> masked. */ +#define test_addr_compare_masked(a, op, b, m) STMT_BEGIN \ + test_eq(tor_inet_pton(AF_INET6, a, &t1.addr.in6_addr), 1); \ + test_eq(tor_inet_pton(AF_INET6, b, &t2.addr.in6_addr), 1); \ + t1.family = t2.family = AF_INET6; \ + r = tor_addr_compare_masked(&t1,&t2,m,CMP_SEMANTIC); \ + if (!(r op 0)) \ + test_fail_msg("failed: tor_addr_compare_masked("a","b","#m") "#op" 0"); \ + STMT_END + +/** Helper: assert that <b>xx</b> is parseable as a masked IPv6 address with + * ports by tor_parse_mask_addr_ports(), with family <b>f</b>, IP address + * as 4 32-bit words <b>ip1...ip4</b>, mask bits as <b>mm</b>, and port range + * as <b>pt1..pt2</b>. */ +#define test_addr_mask_ports_parse(xx, f, ip1, ip2, ip3, ip4, mm, pt1, pt2) \ + STMT_BEGIN \ + test_eq(tor_addr_parse_mask_ports(xx, &t1, &mask, &port1, &port2), f); \ + p1=tor_inet_ntop(AF_INET6, &t1.addr.in6_addr, bug, sizeof(bug)); \ + test_eq(htonl(ip1), tor_addr_to_in6_addr32(&t1)[0]); \ + test_eq(htonl(ip2), tor_addr_to_in6_addr32(&t1)[1]); \ + test_eq(htonl(ip3), tor_addr_to_in6_addr32(&t1)[2]); \ + test_eq(htonl(ip4), tor_addr_to_in6_addr32(&t1)[3]); \ + test_eq(mask, mm); \ + test_eq(port1, pt1); \ + test_eq(port2, pt2); \ + STMT_END + +/** Run unit tests for IPv6 encoding/decoding/manipulation functions. */ +static void +test_addr_ip6_helpers(void) +{ + char buf[TOR_ADDR_BUF_LEN], bug[TOR_ADDR_BUF_LEN]; + struct in6_addr a1, a2; + tor_addr_t t1, t2; + int r, i; + uint16_t port1, port2; + maskbits_t mask; + const char *p1; + struct sockaddr_storage sa_storage; + struct sockaddr_in *sin; + struct sockaddr_in6 *sin6; + + // struct in_addr b1, b2; + /* Test tor_inet_ntop and tor_inet_pton: IPv6 */ + + /* ==== Converting to and from sockaddr_t. */ + sin = (struct sockaddr_in *)&sa_storage; + sin->sin_family = AF_INET; + sin->sin_port = 9090; + sin->sin_addr.s_addr = htonl(0x7f7f0102); /*127.127.1.2*/ + tor_addr_from_sockaddr(&t1, (struct sockaddr *)sin, NULL); + test_eq(tor_addr_family(&t1), AF_INET); + test_eq(tor_addr_to_ipv4h(&t1), 0x7f7f0102); + + memset(&sa_storage, 0, sizeof(sa_storage)); + test_eq(sizeof(struct sockaddr_in), + tor_addr_to_sockaddr(&t1, 1234, (struct sockaddr *)&sa_storage, + sizeof(sa_storage))); + test_eq(1234, ntohs(sin->sin_port)); + test_eq(0x7f7f0102, ntohl(sin->sin_addr.s_addr)); + + memset(&sa_storage, 0, sizeof(sa_storage)); + sin6 = (struct sockaddr_in6 *)&sa_storage; + sin6->sin6_family = AF_INET6; + sin6->sin6_port = htons(7070); + sin6->sin6_addr.s6_addr[0] = 128; + tor_addr_from_sockaddr(&t1, (struct sockaddr *)sin6, NULL); + test_eq(tor_addr_family(&t1), AF_INET6); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 0); + test_streq(p1, "8000::"); + + memset(&sa_storage, 0, sizeof(sa_storage)); + test_eq(sizeof(struct sockaddr_in6), + tor_addr_to_sockaddr(&t1, 9999, (struct sockaddr *)&sa_storage, + sizeof(sa_storage))); + test_eq(AF_INET6, sin6->sin6_family); + test_eq(9999, ntohs(sin6->sin6_port)); + test_eq(0x80000000, ntohl(S6_ADDR32(sin6->sin6_addr)[0])); + + /* ==== tor_addr_lookup: static cases. (Can't test dns without knowing we + * have a good resolver. */ + test_eq(0, tor_addr_lookup("127.128.129.130", AF_UNSPEC, &t1)); + test_eq(AF_INET, tor_addr_family(&t1)); + test_eq(tor_addr_to_ipv4h(&t1), 0x7f808182); + + test_eq(0, tor_addr_lookup("9000::5", AF_UNSPEC, &t1)); + test_eq(AF_INET6, tor_addr_family(&t1)); + test_eq(0x90, tor_addr_to_in6_addr8(&t1)[0]); + test_assert(tor_mem_is_zero((char*)tor_addr_to_in6_addr8(&t1)+1, 14)); + test_eq(0x05, tor_addr_to_in6_addr8(&t1)[15]); + + /* === Test pton: valid af_inet6 */ + /* Simple, valid parsing. */ + r = tor_inet_pton(AF_INET6, + "0102:0304:0506:0708:090A:0B0C:0D0E:0F10", &a1); + test_assert(r==1); + for (i=0;i<16;++i) { test_eq(i+1, (int)a1.s6_addr[i]); } + /* ipv4 ending. */ + test_pton6_same("0102:0304:0506:0708:090A:0B0C:0D0E:0F10", + "0102:0304:0506:0708:090A:0B0C:13.14.15.16"); + /* shortened words. */ + test_pton6_same("0001:0099:BEEF:0000:0123:FFFF:0001:0001", + "1:99:BEEF:0:0123:FFFF:1:1"); + /* zeros at the beginning */ + test_pton6_same("0000:0000:0000:0000:0009:C0A8:0001:0001", + "::9:c0a8:1:1"); + test_pton6_same("0000:0000:0000:0000:0009:C0A8:0001:0001", + "::9:c0a8:0.1.0.1"); + /* zeros in the middle. */ + test_pton6_same("fe80:0000:0000:0000:0202:1111:0001:0001", + "fe80::202:1111:1:1"); + /* zeros at the end. */ + test_pton6_same("1000:0001:0000:0007:0000:0000:0000:0000", + "1000:1:0:7::"); + + /* === Test ntop: af_inet6 */ + test_ntop6_reduces("0:0:0:0:0:0:0:0", "::"); + + test_ntop6_reduces("0001:0099:BEEF:0006:0123:FFFF:0001:0001", + "1:99:beef:6:123:ffff:1:1"); + + //test_ntop6_reduces("0:0:0:0:0:0:c0a8:0101", "::192.168.1.1"); + test_ntop6_reduces("0:0:0:0:0:ffff:c0a8:0101", "::ffff:192.168.1.1"); + test_ntop6_reduces("002:0:0000:0:3::4", "2::3:0:0:4"); + test_ntop6_reduces("0:0::1:0:3", "::1:0:3"); + test_ntop6_reduces("008:0::0", "8::"); + test_ntop6_reduces("0:0:0:0:0:ffff::1", "::ffff:0.0.0.1"); + test_ntop6_reduces("abcd:0:0:0:0:0:7f00::", "abcd::7f00:0"); + test_ntop6_reduces("0000:0000:0000:0000:0009:C0A8:0001:0001", + "::9:c0a8:1:1"); + test_ntop6_reduces("fe80:0000:0000:0000:0202:1111:0001:0001", + "fe80::202:1111:1:1"); + test_ntop6_reduces("1000:0001:0000:0007:0000:0000:0000:0000", + "1000:1:0:7::"); + + /* === Test pton: invalid in6. */ + test_pton6_bad("foobar."); + test_pton6_bad("55555::"); + test_pton6_bad("9:-60::"); + test_pton6_bad("1:2:33333:4:0002:3::"); + //test_pton6_bad("1:2:3333:4:00002:3::");// BAD, but glibc doesn't say so. + test_pton6_bad("1:2:3333:4:fish:3::"); + test_pton6_bad("1:2:3:4:5:6:7:8:9"); + test_pton6_bad("1:2:3:4:5:6:7"); + test_pton6_bad("1:2:3:4:5:6:1.2.3.4.5"); + test_pton6_bad("1:2:3:4:5:6:1.2.3"); + test_pton6_bad("::1.2.3"); + test_pton6_bad("::1.2.3.4.5"); + test_pton6_bad("99"); + test_pton6_bad(""); + test_pton6_bad("1::2::3:4"); + test_pton6_bad("a:::b:c"); + test_pton6_bad(":::a:b:c"); + test_pton6_bad("a:b:c:::"); + + /* test internal checking */ + test_external_ip("fbff:ffff::2:7", 0); + test_internal_ip("fc01::2:7", 0); + test_internal_ip("fdff:ffff::f:f", 0); + test_external_ip("fe00::3:f", 0); + + test_external_ip("fe7f:ffff::2:7", 0); + test_internal_ip("fe80::2:7", 0); + test_internal_ip("febf:ffff::f:f", 0); + + test_internal_ip("fec0::2:7:7", 0); + test_internal_ip("feff:ffff::e:7:7", 0); + test_external_ip("ff00::e:7:7", 0); + + test_internal_ip("::", 0); + test_internal_ip("::1", 0); + test_internal_ip("::1", 1); + test_internal_ip("::", 0); + test_external_ip("::", 1); + test_external_ip("::2", 0); + test_external_ip("2001::", 0); + test_external_ip("ffff::", 0); + + test_external_ip("::ffff:0.0.0.0", 1); + test_internal_ip("::ffff:0.0.0.0", 0); + test_internal_ip("::ffff:0.255.255.255", 0); + test_external_ip("::ffff:1.0.0.0", 0); + + test_external_ip("::ffff:9.255.255.255", 0); + test_internal_ip("::ffff:10.0.0.0", 0); + test_internal_ip("::ffff:10.255.255.255", 0); + test_external_ip("::ffff:11.0.0.0", 0); + + test_external_ip("::ffff:126.255.255.255", 0); + test_internal_ip("::ffff:127.0.0.0", 0); + test_internal_ip("::ffff:127.255.255.255", 0); + test_external_ip("::ffff:128.0.0.0", 0); + + test_external_ip("::ffff:172.15.255.255", 0); + test_internal_ip("::ffff:172.16.0.0", 0); + test_internal_ip("::ffff:172.31.255.255", 0); + test_external_ip("::ffff:172.32.0.0", 0); + + test_external_ip("::ffff:192.167.255.255", 0); + test_internal_ip("::ffff:192.168.0.0", 0); + test_internal_ip("::ffff:192.168.255.255", 0); + test_external_ip("::ffff:192.169.0.0", 0); + + test_external_ip("::ffff:169.253.255.255", 0); + test_internal_ip("::ffff:169.254.0.0", 0); + test_internal_ip("::ffff:169.254.255.255", 0); + test_external_ip("::ffff:169.255.0.0", 0); + test_assert(is_internal_IP(0x7f000001, 0)); + + /* tor_addr_compare(tor_addr_t x2) */ + test_addr_compare("ffff::", ==, "ffff::0"); + test_addr_compare("0::3:2:1", <, "0::ffff:0.3.2.1"); + test_addr_compare("0::2:2:1", <, "0::ffff:0.3.2.1"); + test_addr_compare("0::ffff:0.3.2.1", >, "0::0:0:0"); + test_addr_compare("0::ffff:5.2.2.1", <, "::ffff:6.0.0.0"); /* XXXX wrong. */ + tor_addr_parse_mask_ports("[::ffff:2.3.4.5]", &t1, NULL, NULL, NULL); + tor_addr_parse_mask_ports("2.3.4.5", &t2, NULL, NULL, NULL); + test_assert(tor_addr_compare(&t1, &t2, CMP_SEMANTIC) == 0); + tor_addr_parse_mask_ports("[::ffff:2.3.4.4]", &t1, NULL, NULL, NULL); + tor_addr_parse_mask_ports("2.3.4.5", &t2, NULL, NULL, NULL); + test_assert(tor_addr_compare(&t1, &t2, CMP_SEMANTIC) < 0); + + /* test compare_masked */ + test_addr_compare_masked("ffff::", ==, "ffff::0", 128); + test_addr_compare_masked("ffff::", ==, "ffff::0", 64); + test_addr_compare_masked("0::2:2:1", <, "0::8000:2:1", 81); + test_addr_compare_masked("0::2:2:1", ==, "0::8000:2:1", 80); + + /* Test decorated addr_to_string. */ + test_eq(AF_INET6, tor_addr_from_str(&t1, "[123:45:6789::5005:11]")); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); + test_streq(p1, "[123:45:6789::5005:11]"); + test_eq(AF_INET, tor_addr_from_str(&t1, "18.0.0.1")); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); + test_streq(p1, "18.0.0.1"); + + /* Test tor_addr_parse_reverse_lookup_name */ + i = tor_addr_parse_reverse_lookup_name(&t1, "Foobar.baz", AF_UNSPEC, 0); + test_eq(0, i); + i = tor_addr_parse_reverse_lookup_name(&t1, "Foobar.baz", AF_UNSPEC, 1); + test_eq(0, i); + i = tor_addr_parse_reverse_lookup_name(&t1, "1.0.168.192.in-addr.arpa", + AF_UNSPEC, 1); + test_eq(1, i); + test_eq(tor_addr_family(&t1), AF_INET); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); + test_streq(p1, "192.168.0.1"); + i = tor_addr_parse_reverse_lookup_name(&t1, "192.168.0.99", AF_UNSPEC, 0); + test_eq(0, i); + i = tor_addr_parse_reverse_lookup_name(&t1, "192.168.0.99", AF_UNSPEC, 1); + test_eq(1, i); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); + test_streq(p1, "192.168.0.99"); + memset(&t1, 0, sizeof(t1)); + i = tor_addr_parse_reverse_lookup_name(&t1, + "0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f." + "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." + "ip6.ARPA", + AF_UNSPEC, 0); + test_eq(1, i); + p1 = tor_addr_to_str(buf, &t1, sizeof(buf), 1); + test_streq(p1, "[9dee:effe:ebe1:beef:fedc:ba98:7654:3210]"); + /* Failing cases. */ + i = tor_addr_parse_reverse_lookup_name(&t1, + "6.7.8.9.a.b.c.d.e.f." + "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." + "ip6.ARPA", + AF_UNSPEC, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, + "6.7.8.9.a.b.c.d.e.f.a.b.c.d.e.f.0." + "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." + "ip6.ARPA", + AF_UNSPEC, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, + "6.7.8.9.a.b.c.d.e.f.X.0.0.0.0.9." + "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." + "ip6.ARPA", + AF_UNSPEC, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, "32.1.1.in-addr.arpa", + AF_UNSPEC, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, ".in-addr.arpa", + AF_UNSPEC, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, "1.2.3.4.5.in-addr.arpa", + AF_UNSPEC, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, "1.2.3.4.5.in-addr.arpa", + AF_INET6, 0); + test_eq(i, -1); + i = tor_addr_parse_reverse_lookup_name(&t1, + "6.7.8.9.a.b.c.d.e.f.a.b.c.d.e.0." + "f.e.e.b.1.e.b.e.e.f.f.e.e.e.d.9." + "ip6.ARPA", + AF_INET, 0); + test_eq(i, -1); + + /* test tor_addr_parse_mask_ports */ + test_addr_mask_ports_parse("[::f]/17:47-95", AF_INET6, + 0, 0, 0, 0x0000000f, 17, 47, 95); + //test_addr_parse("[::fefe:4.1.1.7/120]:999-1000"); + //test_addr_parse_check("::fefe:401:107", 120, 999, 1000); + test_addr_mask_ports_parse("[::ffff:4.1.1.7]/120:443", AF_INET6, + 0, 0, 0x0000ffff, 0x04010107, 120, 443, 443); + test_addr_mask_ports_parse("[abcd:2::44a:0]:2-65000", AF_INET6, + 0xabcd0002, 0, 0, 0x044a0000, 128, 2, 65000); + + r=tor_addr_parse_mask_ports("[fefef::]/112", &t1, NULL, NULL, NULL); + test_assert(r == -1); + r=tor_addr_parse_mask_ports("efef::/112", &t1, NULL, NULL, NULL); + test_assert(r == -1); + r=tor_addr_parse_mask_ports("[f:f:f:f:f:f:f:f::]", &t1, NULL, NULL, NULL); + test_assert(r == -1); + r=tor_addr_parse_mask_ports("[::f:f:f:f:f:f:f:f]", &t1, NULL, NULL, NULL); + test_assert(r == -1); + r=tor_addr_parse_mask_ports("[f:f:f:f:f:f:f:f:f]", &t1, NULL, NULL, NULL); + test_assert(r == -1); + /* Test for V4-mapped address with mask < 96. (arguably not valid) */ + r=tor_addr_parse_mask_ports("[::ffff:1.1.2.2/33]", &t1, &mask, NULL, NULL); + test_assert(r == -1); + r=tor_addr_parse_mask_ports("1.1.2.2/33", &t1, &mask, NULL, NULL); + test_assert(r == -1); + r=tor_addr_parse_mask_ports("1.1.2.2/31", &t1, &mask, NULL, NULL); + test_assert(r == AF_INET); + r=tor_addr_parse_mask_ports("[efef::]/112", &t1, &mask, &port1, &port2); + test_assert(r == AF_INET6); + test_assert(port1 == 1); + test_assert(port2 == 65535); + + /* make sure inet address lengths >= max */ + test_assert(INET_NTOA_BUF_LEN >= sizeof("255.255.255.255")); + test_assert(TOR_ADDR_BUF_LEN >= + sizeof("ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255")); + + test_assert(sizeof(tor_addr_t) >= sizeof(struct in6_addr)); + + /* get interface addresses */ + r = get_interface_address6(LOG_DEBUG, AF_INET, &t1); + i = get_interface_address6(LOG_DEBUG, AF_INET6, &t2); +#if 0 + tor_inet_ntop(AF_INET, &t1.sa.sin_addr, buf, sizeof(buf)); + printf("\nv4 address: %s (family=%d)", buf, IN_FAMILY(&t1)); + tor_inet_ntop(AF_INET6, &t2.sa6.sin6_addr, buf, sizeof(buf)); + printf("\nv6 address: %s (family=%d)", buf, IN_FAMILY(&t2)); +#endif + + done: + ; +} + +#define ADDR_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_addr_ ## name } + +struct testcase_t addr_tests[] = { + ADDR_LEGACY(basic), + ADDR_LEGACY(ip6_helpers), + END_OF_TESTCASES +}; + diff --git a/src/test/test_containers.c b/src/test/test_containers.c new file mode 100644 index 000000000..af9fb1c5c --- /dev/null +++ b/src/test/test_containers.c @@ -0,0 +1,765 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "or.h" +#include "test.h" + +/** Helper: return a tristate based on comparing the strings in *<b>a</b> and + * *<b>b</b>. */ +static int +_compare_strs(const void **a, const void **b) +{ + const char *s1 = *a, *s2 = *b; + return strcmp(s1, s2); +} + +/** Helper: return a tristate based on comparing the strings in *<b>a</b> and + * *<b>b</b>, excluding a's first character, and ignoring case. */ +static int +_compare_without_first_ch(const void *a, const void **b) +{ + const char *s1 = a, *s2 = *b; + return strcasecmp(s1+1, s2); +} + +/** Run unit tests for basic dynamic-sized array functionality. */ +static void +test_container_smartlist_basic(void) +{ + smartlist_t *sl; + + /* XXXX test sort_digests, uniq_strings, uniq_digests */ + + /* Test smartlist add, del_keeporder, insert, get. */ + sl = smartlist_create(); + smartlist_add(sl, (void*)1); + smartlist_add(sl, (void*)2); + smartlist_add(sl, (void*)3); + smartlist_add(sl, (void*)4); + smartlist_del_keeporder(sl, 1); + smartlist_insert(sl, 1, (void*)22); + smartlist_insert(sl, 0, (void*)0); + smartlist_insert(sl, 5, (void*)555); + test_eq_ptr((void*)0, smartlist_get(sl,0)); + test_eq_ptr((void*)1, smartlist_get(sl,1)); + test_eq_ptr((void*)22, smartlist_get(sl,2)); + test_eq_ptr((void*)3, smartlist_get(sl,3)); + test_eq_ptr((void*)4, smartlist_get(sl,4)); + test_eq_ptr((void*)555, smartlist_get(sl,5)); + /* Try deleting in the middle. */ + smartlist_del(sl, 1); + test_eq_ptr((void*)555, smartlist_get(sl, 1)); + /* Try deleting at the end. */ + smartlist_del(sl, 4); + test_eq(4, smartlist_len(sl)); + + /* test isin. */ + test_assert(smartlist_isin(sl, (void*)3)); + test_assert(!smartlist_isin(sl, (void*)99)); + + done: + smartlist_free(sl); +} + +/** Run unit tests for smartlist-of-strings functionality. */ +static void +test_container_smartlist_strings(void) +{ + smartlist_t *sl = smartlist_create(); + char *cp=NULL, *cp_alloc=NULL; + size_t sz; + + /* Test split and join */ + test_eq(0, smartlist_len(sl)); + smartlist_split_string(sl, "abc", ":", 0, 0); + test_eq(1, smartlist_len(sl)); + test_streq("abc", smartlist_get(sl, 0)); + smartlist_split_string(sl, "a::bc::", "::", 0, 0); + test_eq(4, smartlist_len(sl)); + test_streq("a", smartlist_get(sl, 1)); + test_streq("bc", smartlist_get(sl, 2)); + test_streq("", smartlist_get(sl, 3)); + cp_alloc = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp_alloc, "abcabc"); + tor_free(cp_alloc); + cp_alloc = smartlist_join_strings(sl, "!", 0, NULL); + test_streq(cp_alloc, "abc!a!bc!"); + tor_free(cp_alloc); + cp_alloc = smartlist_join_strings(sl, "XY", 0, NULL); + test_streq(cp_alloc, "abcXYaXYbcXY"); + tor_free(cp_alloc); + cp_alloc = smartlist_join_strings(sl, "XY", 1, NULL); + test_streq(cp_alloc, "abcXYaXYbcXYXY"); + tor_free(cp_alloc); + cp_alloc = smartlist_join_strings(sl, "", 1, NULL); + test_streq(cp_alloc, "abcabc"); + tor_free(cp_alloc); + + smartlist_split_string(sl, "/def/ /ghijk", "/", 0, 0); + test_eq(8, smartlist_len(sl)); + test_streq("", smartlist_get(sl, 4)); + test_streq("def", smartlist_get(sl, 5)); + test_streq(" ", smartlist_get(sl, 6)); + test_streq("ghijk", smartlist_get(sl, 7)); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + smartlist_split_string(sl, "a,bbd,cdef", ",", SPLIT_SKIP_SPACE, 0); + test_eq(3, smartlist_len(sl)); + test_streq("a", smartlist_get(sl,0)); + test_streq("bbd", smartlist_get(sl,1)); + test_streq("cdef", smartlist_get(sl,2)); + smartlist_split_string(sl, " z <> zhasd <> <> bnud<> ", "<>", + SPLIT_SKIP_SPACE, 0); + test_eq(8, smartlist_len(sl)); + test_streq("z", smartlist_get(sl,3)); + test_streq("zhasd", smartlist_get(sl,4)); + test_streq("", smartlist_get(sl,5)); + test_streq("bnud", smartlist_get(sl,6)); + test_streq("", smartlist_get(sl,7)); + + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + smartlist_split_string(sl, " ab\tc \td ef ", NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + test_eq(4, smartlist_len(sl)); + test_streq("ab", smartlist_get(sl,0)); + test_streq("c", smartlist_get(sl,1)); + test_streq("d", smartlist_get(sl,2)); + test_streq("ef", smartlist_get(sl,3)); + smartlist_split_string(sl, "ghi\tj", NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + test_eq(6, smartlist_len(sl)); + test_streq("ghi", smartlist_get(sl,4)); + test_streq("j", smartlist_get(sl,5)); + + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + cp_alloc = smartlist_join_strings(sl, "XY", 0, NULL); + test_streq(cp_alloc, ""); + tor_free(cp_alloc); + cp_alloc = smartlist_join_strings(sl, "XY", 1, NULL); + test_streq(cp_alloc, "XY"); + tor_free(cp_alloc); + + smartlist_split_string(sl, " z <> zhasd <> <> bnud<> ", "<>", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + test_eq(3, smartlist_len(sl)); + test_streq("z", smartlist_get(sl, 0)); + test_streq("zhasd", smartlist_get(sl, 1)); + test_streq("bnud", smartlist_get(sl, 2)); + smartlist_split_string(sl, " z <> zhasd <> <> bnud<> ", "<>", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 2); + test_eq(5, smartlist_len(sl)); + test_streq("z", smartlist_get(sl, 3)); + test_streq("zhasd <> <> bnud<>", smartlist_get(sl, 4)); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + smartlist_split_string(sl, "abcd\n", "\n", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + test_eq(1, smartlist_len(sl)); + test_streq("abcd", smartlist_get(sl, 0)); + smartlist_split_string(sl, "efgh", "\n", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + test_eq(2, smartlist_len(sl)); + test_streq("efgh", smartlist_get(sl, 1)); + + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Test swapping, shuffling, and sorting. */ + smartlist_split_string(sl, "the,onion,router,by,arma,and,nickm", ",", 0, 0); + test_eq(7, smartlist_len(sl)); + smartlist_sort(sl, _compare_strs); + cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); + test_streq(cp_alloc,"and,arma,by,nickm,onion,router,the"); + tor_free(cp_alloc); + smartlist_swap(sl, 1, 5); + cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); + test_streq(cp_alloc,"and,router,by,nickm,onion,arma,the"); + tor_free(cp_alloc); + smartlist_shuffle(sl); + test_eq(7, smartlist_len(sl)); + test_assert(smartlist_string_isin(sl, "and")); + test_assert(smartlist_string_isin(sl, "router")); + test_assert(smartlist_string_isin(sl, "by")); + test_assert(smartlist_string_isin(sl, "nickm")); + test_assert(smartlist_string_isin(sl, "onion")); + test_assert(smartlist_string_isin(sl, "arma")); + test_assert(smartlist_string_isin(sl, "the")); + + /* Test bsearch. */ + smartlist_sort(sl, _compare_strs); + test_streq("nickm", smartlist_bsearch(sl, "zNicKM", + _compare_without_first_ch)); + test_streq("and", smartlist_bsearch(sl, " AND", _compare_without_first_ch)); + test_eq_ptr(NULL, smartlist_bsearch(sl, " ANz", _compare_without_first_ch)); + + /* Test bsearch_idx */ + { + int f; + test_eq(0, smartlist_bsearch_idx(sl," aaa",_compare_without_first_ch,&f)); + test_eq(f, 0); + test_eq(0, smartlist_bsearch_idx(sl," and",_compare_without_first_ch,&f)); + test_eq(f, 1); + test_eq(1, smartlist_bsearch_idx(sl," arm",_compare_without_first_ch,&f)); + test_eq(f, 0); + test_eq(1, smartlist_bsearch_idx(sl," arma",_compare_without_first_ch,&f)); + test_eq(f, 1); + test_eq(2, smartlist_bsearch_idx(sl," armb",_compare_without_first_ch,&f)); + test_eq(f, 0); + test_eq(7, smartlist_bsearch_idx(sl," zzzz",_compare_without_first_ch,&f)); + test_eq(f, 0); + } + + /* Test reverse() and pop_last() */ + smartlist_reverse(sl); + cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); + test_streq(cp_alloc,"the,router,onion,nickm,by,arma,and"); + tor_free(cp_alloc); + cp_alloc = smartlist_pop_last(sl); + test_streq(cp_alloc, "and"); + tor_free(cp_alloc); + test_eq(smartlist_len(sl), 6); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + cp_alloc = smartlist_pop_last(sl); + test_eq(cp_alloc, NULL); + + /* Test uniq() */ + smartlist_split_string(sl, + "50,noon,radar,a,man,a,plan,a,canal,panama,radar,noon,50", + ",", 0, 0); + smartlist_sort(sl, _compare_strs); + smartlist_uniq(sl, _compare_strs, _tor_free); + cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); + test_streq(cp_alloc, "50,a,canal,man,noon,panama,plan,radar"); + tor_free(cp_alloc); + + /* Test string_isin and isin_case and num_isin */ + test_assert(smartlist_string_isin(sl, "noon")); + test_assert(!smartlist_string_isin(sl, "noonoon")); + test_assert(smartlist_string_isin_case(sl, "nOOn")); + test_assert(!smartlist_string_isin_case(sl, "nooNooN")); + test_assert(smartlist_string_num_isin(sl, 50)); + test_assert(!smartlist_string_num_isin(sl, 60)); + + /* Test smartlist_choose */ + { + int i; + int allsame = 1; + int allin = 1; + void *first = smartlist_choose(sl); + test_assert(smartlist_isin(sl, first)); + for (i = 0; i < 100; ++i) { + void *second = smartlist_choose(sl); + if (second != first) + allsame = 0; + if (!smartlist_isin(sl, second)) + allin = 0; + } + test_assert(!allsame); + test_assert(allin); + } + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Test string_remove and remove and join_strings2 */ + smartlist_split_string(sl, + "Some say the Earth will end in ice and some in fire", + " ", 0, 0); + cp = smartlist_get(sl, 4); + test_streq(cp, "will"); + smartlist_add(sl, cp); + smartlist_remove(sl, cp); + tor_free(cp); + cp_alloc = smartlist_join_strings(sl, ",", 0, NULL); + test_streq(cp_alloc, "Some,say,the,Earth,fire,end,in,ice,and,some,in"); + tor_free(cp_alloc); + smartlist_string_remove(sl, "in"); + cp_alloc = smartlist_join_strings2(sl, "+XX", 1, 0, &sz); + test_streq(cp_alloc, "Some+say+the+Earth+fire+end+some+ice+and"); + test_eq((int)sz, 40); + + done: + + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_free(sl); + tor_free(cp_alloc); +} + +/** Run unit tests for smartlist set manipulation functions. */ +static void +test_container_smartlist_overlap(void) +{ + smartlist_t *sl = smartlist_create(); + smartlist_t *ints = smartlist_create(); + smartlist_t *odds = smartlist_create(); + smartlist_t *evens = smartlist_create(); + smartlist_t *primes = smartlist_create(); + int i; + for (i=1; i < 10; i += 2) + smartlist_add(odds, (void*)(uintptr_t)i); + for (i=0; i < 10; i += 2) + smartlist_add(evens, (void*)(uintptr_t)i); + + /* add_all */ + smartlist_add_all(ints, odds); + smartlist_add_all(ints, evens); + test_eq(smartlist_len(ints), 10); + + smartlist_add(primes, (void*)2); + smartlist_add(primes, (void*)3); + smartlist_add(primes, (void*)5); + smartlist_add(primes, (void*)7); + + /* overlap */ + test_assert(smartlist_overlap(ints, odds)); + test_assert(smartlist_overlap(odds, primes)); + test_assert(smartlist_overlap(evens, primes)); + test_assert(!smartlist_overlap(odds, evens)); + + /* intersect */ + smartlist_add_all(sl, odds); + smartlist_intersect(sl, primes); + test_eq(smartlist_len(sl), 3); + test_assert(smartlist_isin(sl, (void*)3)); + test_assert(smartlist_isin(sl, (void*)5)); + test_assert(smartlist_isin(sl, (void*)7)); + + /* subtract */ + smartlist_add_all(sl, primes); + smartlist_subtract(sl, odds); + test_eq(smartlist_len(sl), 1); + test_assert(smartlist_isin(sl, (void*)2)); + + done: + smartlist_free(odds); + smartlist_free(evens); + smartlist_free(ints); + smartlist_free(primes); + smartlist_free(sl); +} + +/** Run unit tests for smartlist-of-digests functions. */ +static void +test_container_smartlist_digests(void) +{ + smartlist_t *sl = smartlist_create(); + + /* digest_isin. */ + smartlist_add(sl, tor_memdup("AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN)); + smartlist_add(sl, tor_memdup("\00090AAB2AAAAaasdAAAAA", DIGEST_LEN)); + smartlist_add(sl, tor_memdup("\00090AAB2AAAAaasdAAAAA", DIGEST_LEN)); + test_eq(0, smartlist_digest_isin(NULL, "AAAAAAAAAAAAAAAAAAAA")); + test_assert(smartlist_digest_isin(sl, "AAAAAAAAAAAAAAAAAAAA")); + test_assert(smartlist_digest_isin(sl, "\00090AAB2AAAAaasdAAAAA")); + test_eq(0, smartlist_digest_isin(sl, "\00090AAB2AAABaasdAAAAA")); + + /* sort digests */ + smartlist_sort_digests(sl); + test_memeq(smartlist_get(sl, 0), "\00090AAB2AAAAaasdAAAAA", DIGEST_LEN); + test_memeq(smartlist_get(sl, 1), "\00090AAB2AAAAaasdAAAAA", DIGEST_LEN); + test_memeq(smartlist_get(sl, 2), "AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN); + test_eq(3, smartlist_len(sl)); + + /* uniq_digests */ + smartlist_uniq_digests(sl); + test_eq(2, smartlist_len(sl)); + test_memeq(smartlist_get(sl, 0), "\00090AAB2AAAAaasdAAAAA", DIGEST_LEN); + test_memeq(smartlist_get(sl, 1), "AAAAAAAAAAAAAAAAAAAA", DIGEST_LEN); + + done: + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_free(sl); +} + +/** Run unit tests for concatenate-a-smartlist-of-strings functions. */ +static void +test_container_smartlist_join(void) +{ + smartlist_t *sl = smartlist_create(); + smartlist_t *sl2 = smartlist_create(), *sl3 = smartlist_create(), + *sl4 = smartlist_create(); + char *joined=NULL; + /* unique, sorted. */ + smartlist_split_string(sl, + "Abashments Ambush Anchorman Bacon Banks Borscht " + "Bunks Inhumane Insurance Knish Know Manners " + "Maraschinos Stamina Sunbonnets Unicorns Wombats", + " ", 0, 0); + /* non-unique, sorted. */ + smartlist_split_string(sl2, + "Ambush Anchorman Anchorman Anemias Anemias Bacon " + "Crossbowmen Inhumane Insurance Knish Know Manners " + "Manners Maraschinos Wombats Wombats Work", + " ", 0, 0); + SMARTLIST_FOREACH_JOIN(sl, char *, cp1, + sl2, char *, cp2, + strcmp(cp1,cp2), + smartlist_add(sl3, cp2)) { + test_streq(cp1, cp2); + smartlist_add(sl4, cp1); + } SMARTLIST_FOREACH_JOIN_END(cp1, cp2); + + SMARTLIST_FOREACH(sl3, const char *, cp, + test_assert(smartlist_isin(sl2, cp) && + !smartlist_string_isin(sl, cp))); + SMARTLIST_FOREACH(sl4, const char *, cp, + test_assert(smartlist_isin(sl, cp) && + smartlist_string_isin(sl2, cp))); + joined = smartlist_join_strings(sl3, ",", 0, NULL); + test_streq(joined, "Anemias,Anemias,Crossbowmen,Work"); + tor_free(joined); + joined = smartlist_join_strings(sl4, ",", 0, NULL); + test_streq(joined, "Ambush,Anchorman,Anchorman,Bacon,Inhumane,Insurance," + "Knish,Know,Manners,Manners,Maraschinos,Wombats,Wombats"); + tor_free(joined); + + done: + smartlist_free(sl4); + smartlist_free(sl3); + SMARTLIST_FOREACH(sl2, char *, cp, tor_free(cp)); + smartlist_free(sl2); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_free(sl); + tor_free(joined); +} + +/** Run unit tests for bitarray code */ +static void +test_container_bitarray(void) +{ + bitarray_t *ba = NULL; + int i, j, ok=1; + + ba = bitarray_init_zero(1); + test_assert(ba); + test_assert(! bitarray_is_set(ba, 0)); + bitarray_set(ba, 0); + test_assert(bitarray_is_set(ba, 0)); + bitarray_clear(ba, 0); + test_assert(! bitarray_is_set(ba, 0)); + bitarray_free(ba); + + ba = bitarray_init_zero(1023); + for (i = 1; i < 64; ) { + for (j = 0; j < 1023; ++j) { + if (j % i) + bitarray_set(ba, j); + else + bitarray_clear(ba, j); + } + for (j = 0; j < 1023; ++j) { + if (!bool_eq(bitarray_is_set(ba, j), j%i)) + ok = 0; + } + test_assert(ok); + if (i < 7) + ++i; + else if (i == 28) + i = 32; + else + i += 7; + } + + done: + if (ba) + bitarray_free(ba); +} + +/** Run unit tests for digest set code (implemented as a hashtable or as a + * bloom filter) */ +static void +test_container_digestset(void) +{ + smartlist_t *included = smartlist_create(); + char d[DIGEST_LEN]; + int i; + int ok = 1; + int false_positives = 0; + digestset_t *set = NULL; + + for (i = 0; i < 1000; ++i) { + crypto_rand(d, DIGEST_LEN); + smartlist_add(included, tor_memdup(d, DIGEST_LEN)); + } + set = digestset_new(1000); + SMARTLIST_FOREACH(included, const char *, cp, + if (digestset_isin(set, cp)) + ok = 0); + test_assert(ok); + SMARTLIST_FOREACH(included, const char *, cp, + digestset_add(set, cp)); + SMARTLIST_FOREACH(included, const char *, cp, + if (!digestset_isin(set, cp)) + ok = 0); + test_assert(ok); + for (i = 0; i < 1000; ++i) { + crypto_rand(d, DIGEST_LEN); + if (digestset_isin(set, d)) + ++false_positives; + } + test_assert(false_positives < 50); /* Should be far lower. */ + + done: + if (set) + digestset_free(set); + SMARTLIST_FOREACH(included, char *, cp, tor_free(cp)); + smartlist_free(included); +} + +typedef struct pq_entry_t { + const char *val; + int idx; +} pq_entry_t; + +/** Helper: return a tristate based on comparing two pq_entry_t values. */ +static int +_compare_strings_for_pqueue(const void *p1, const void *p2) +{ + const pq_entry_t *e1=p1, *e2=p2; + return strcmp(e1->val, e2->val); +} + +/** Run unit tests for heap-based priority queue functions. */ +static void +test_container_pqueue(void) +{ + smartlist_t *sl = smartlist_create(); + int (*cmp)(const void *, const void*); + const int offset = STRUCT_OFFSET(pq_entry_t, idx); +#define ENTRY(s) pq_entry_t s = { #s, -1 } + ENTRY(cows); + ENTRY(zebras); + ENTRY(fish); + ENTRY(frogs); + ENTRY(apples); + ENTRY(squid); + ENTRY(daschunds); + ENTRY(eggplants); + ENTRY(weissbier); + ENTRY(lobsters); + ENTRY(roquefort); + ENTRY(chinchillas); + ENTRY(fireflies); + +#define OK() smartlist_pqueue_assert_ok(sl, cmp, offset) + + cmp = _compare_strings_for_pqueue; + smartlist_pqueue_add(sl, cmp, offset, &cows); + smartlist_pqueue_add(sl, cmp, offset, &zebras); + smartlist_pqueue_add(sl, cmp, offset, &fish); + smartlist_pqueue_add(sl, cmp, offset, &frogs); + smartlist_pqueue_add(sl, cmp, offset, &apples); + smartlist_pqueue_add(sl, cmp, offset, &squid); + smartlist_pqueue_add(sl, cmp, offset, &daschunds); + smartlist_pqueue_add(sl, cmp, offset, &eggplants); + smartlist_pqueue_add(sl, cmp, offset, &weissbier); + smartlist_pqueue_add(sl, cmp, offset, &lobsters); + smartlist_pqueue_add(sl, cmp, offset, &roquefort); + + OK(); + + test_eq(smartlist_len(sl), 11); + test_eq_ptr(smartlist_get(sl, 0), &apples); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &apples); + test_eq(smartlist_len(sl), 10); + OK(); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &cows); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &daschunds); + smartlist_pqueue_add(sl, cmp, offset, &chinchillas); + OK(); + smartlist_pqueue_add(sl, cmp, offset, &fireflies); + OK(); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &chinchillas); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &eggplants); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &fireflies); + OK(); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &fish); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &frogs); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &lobsters); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &roquefort); + OK(); + test_eq(smartlist_len(sl), 3); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &squid); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &weissbier); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &zebras); + test_eq(smartlist_len(sl), 0); + OK(); + + /* Now test remove. */ + smartlist_pqueue_add(sl, cmp, offset, &cows); + smartlist_pqueue_add(sl, cmp, offset, &fish); + smartlist_pqueue_add(sl, cmp, offset, &frogs); + smartlist_pqueue_add(sl, cmp, offset, &apples); + smartlist_pqueue_add(sl, cmp, offset, &squid); + smartlist_pqueue_add(sl, cmp, offset, &zebras); + test_eq(smartlist_len(sl), 6); + OK(); + smartlist_pqueue_remove(sl, cmp, offset, &zebras); + test_eq(smartlist_len(sl), 5); + OK(); + smartlist_pqueue_remove(sl, cmp, offset, &cows); + test_eq(smartlist_len(sl), 4); + OK(); + smartlist_pqueue_remove(sl, cmp, offset, &apples); + test_eq(smartlist_len(sl), 3); + OK(); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &fish); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &frogs); + test_eq_ptr(smartlist_pqueue_pop(sl, cmp, offset), &squid); + test_eq(smartlist_len(sl), 0); + OK(); + +#undef OK + + done: + + smartlist_free(sl); +} + +/** Run unit tests for string-to-void* map functions */ +static void +test_container_strmap(void) +{ + strmap_t *map; + strmap_iter_t *iter; + const char *k; + void *v; + char *visited = NULL; + smartlist_t *found_keys = NULL; + + map = strmap_new(); + test_assert(map); + test_eq(strmap_size(map), 0); + test_assert(strmap_isempty(map)); + v = strmap_set(map, "K1", (void*)99); + test_eq(v, NULL); + test_assert(!strmap_isempty(map)); + v = strmap_set(map, "K2", (void*)101); + test_eq(v, NULL); + v = strmap_set(map, "K1", (void*)100); + test_eq(v, (void*)99); + test_eq_ptr(strmap_get(map,"K1"), (void*)100); + test_eq_ptr(strmap_get(map,"K2"), (void*)101); + test_eq_ptr(strmap_get(map,"K-not-there"), NULL); + strmap_assert_ok(map); + + v = strmap_remove(map,"K2"); + strmap_assert_ok(map); + test_eq_ptr(v, (void*)101); + test_eq_ptr(strmap_get(map,"K2"), NULL); + test_eq_ptr(strmap_remove(map,"K2"), NULL); + + strmap_set(map, "K2", (void*)101); + strmap_set(map, "K3", (void*)102); + strmap_set(map, "K4", (void*)103); + test_eq(strmap_size(map), 4); + strmap_assert_ok(map); + strmap_set(map, "K5", (void*)104); + strmap_set(map, "K6", (void*)105); + strmap_assert_ok(map); + + /* Test iterator. */ + iter = strmap_iter_init(map); + found_keys = smartlist_create(); + while (!strmap_iter_done(iter)) { + strmap_iter_get(iter,&k,&v); + smartlist_add(found_keys, tor_strdup(k)); + test_eq_ptr(v, strmap_get(map, k)); + + if (!strcmp(k, "K2")) { + iter = strmap_iter_next_rmv(map,iter); + } else { + iter = strmap_iter_next(map,iter); + } + } + + /* Make sure we removed K2, but not the others. */ + test_eq_ptr(strmap_get(map, "K2"), NULL); + test_eq_ptr(strmap_get(map, "K5"), (void*)104); + /* Make sure we visited everyone once */ + smartlist_sort_strings(found_keys); + visited = smartlist_join_strings(found_keys, ":", 0, NULL); + test_streq(visited, "K1:K2:K3:K4:K5:K6"); + + strmap_assert_ok(map); + /* Clean up after ourselves. */ + strmap_free(map, NULL); + map = NULL; + + /* Now try some lc functions. */ + map = strmap_new(); + strmap_set_lc(map,"Ab.C", (void*)1); + test_eq_ptr(strmap_get(map,"ab.c"), (void*)1); + strmap_assert_ok(map); + test_eq_ptr(strmap_get_lc(map,"AB.C"), (void*)1); + test_eq_ptr(strmap_get(map,"AB.C"), NULL); + test_eq_ptr(strmap_remove_lc(map,"aB.C"), (void*)1); + strmap_assert_ok(map); + test_eq_ptr(strmap_get_lc(map,"AB.C"), NULL); + + done: + if (map) + strmap_free(map,NULL); + if (found_keys) { + SMARTLIST_FOREACH(found_keys, char *, cp, tor_free(cp)); + smartlist_free(found_keys); + } + tor_free(visited); +} + +/** Run unit tests for getting the median of a list. */ +static void +test_container_order_functions(void) +{ + int lst[25], n = 0; + // int a=12,b=24,c=25,d=60,e=77; + +#define median() median_int(lst, n) + + lst[n++] = 12; + test_eq(12, median()); /* 12 */ + lst[n++] = 77; + //smartlist_shuffle(sl); + test_eq(12, median()); /* 12, 77 */ + lst[n++] = 77; + //smartlist_shuffle(sl); + test_eq(77, median()); /* 12, 77, 77 */ + lst[n++] = 24; + test_eq(24, median()); /* 12,24,77,77 */ + lst[n++] = 60; + lst[n++] = 12; + lst[n++] = 25; + //smartlist_shuffle(sl); + test_eq(25, median()); /* 12,12,24,25,60,77,77 */ +#undef median + + done: + ; +} + +#define CONTAINER_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_container_ ## name } + +struct testcase_t container_tests[] = { + CONTAINER_LEGACY(smartlist_basic), + CONTAINER_LEGACY(smartlist_strings), + CONTAINER_LEGACY(smartlist_overlap), + CONTAINER_LEGACY(smartlist_digests), + CONTAINER_LEGACY(smartlist_join), + CONTAINER_LEGACY(bitarray), + CONTAINER_LEGACY(digestset), + CONTAINER_LEGACY(strmap), + CONTAINER_LEGACY(pqueue), + CONTAINER_LEGACY(order_functions), + END_OF_TESTCASES +}; + diff --git a/src/test/test_crypto.c b/src/test/test_crypto.c new file mode 100644 index 000000000..781081a4a --- /dev/null +++ b/src/test/test_crypto.c @@ -0,0 +1,796 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#define CRYPTO_PRIVATE +#include "or.h" +#include "test.h" + +/** Run unit tests for Diffie-Hellman functionality. */ +static void +test_crypto_dh(void) +{ + crypto_dh_env_t *dh1 = crypto_dh_new(DH_TYPE_CIRCUIT); + crypto_dh_env_t *dh2 = crypto_dh_new(DH_TYPE_CIRCUIT); + char p1[DH_BYTES]; + char p2[DH_BYTES]; + char s1[DH_BYTES]; + char s2[DH_BYTES]; + ssize_t s1len, s2len; + + test_eq(crypto_dh_get_bytes(dh1), DH_BYTES); + test_eq(crypto_dh_get_bytes(dh2), DH_BYTES); + + memset(p1, 0, DH_BYTES); + memset(p2, 0, DH_BYTES); + test_memeq(p1, p2, DH_BYTES); + test_assert(! crypto_dh_get_public(dh1, p1, DH_BYTES)); + test_memneq(p1, p2, DH_BYTES); + test_assert(! crypto_dh_get_public(dh2, p2, DH_BYTES)); + test_memneq(p1, p2, DH_BYTES); + + memset(s1, 0, DH_BYTES); + memset(s2, 0xFF, DH_BYTES); + s1len = crypto_dh_compute_secret(LOG_WARN, dh1, p2, DH_BYTES, s1, 50); + s2len = crypto_dh_compute_secret(LOG_WARN, dh2, p1, DH_BYTES, s2, 50); + test_assert(s1len > 0); + test_eq(s1len, s2len); + test_memeq(s1, s2, s1len); + + { + /* XXXX Now fabricate some bad values and make sure they get caught, + * Check 0, 1, N-1, >= N, etc. + */ + } + + done: + crypto_dh_free(dh1); + crypto_dh_free(dh2); +} + +/** Run unit tests for our random number generation function and its wrappers. + */ +static void +test_crypto_rng(void) +{ + int i, j, allok; + char data1[100], data2[100]; + double d; + + /* Try out RNG. */ + test_assert(! crypto_seed_rng(0)); + crypto_rand(data1, 100); + crypto_rand(data2, 100); + test_memneq(data1,data2,100); + allok = 1; + for (i = 0; i < 100; ++i) { + uint64_t big; + char *host; + j = crypto_rand_int(100); + if (i < 0 || i >= 100) + allok = 0; + big = crypto_rand_uint64(U64_LITERAL(1)<<40); + if (big >= (U64_LITERAL(1)<<40)) + allok = 0; + big = crypto_rand_uint64(U64_LITERAL(5)); + if (big >= 5) + allok = 0; + d = crypto_rand_double(); + test_assert(d >= 0); + test_assert(d < 1.0); + host = crypto_random_hostname(3,8,"www.",".onion"); + if (strcmpstart(host,"www.") || + strcmpend(host,".onion") || + strlen(host) < 13 || + strlen(host) > 18) + allok = 0; + tor_free(host); + } + test_assert(allok); + done: + ; +} + +/** Run unit tests for our AES functionality */ +static void +test_crypto_aes(void) +{ + char *data1 = NULL, *data2 = NULL, *data3 = NULL; + crypto_cipher_env_t *env1 = NULL, *env2 = NULL; + int i, j; + char *mem_op_hex_tmp=NULL; + + data1 = tor_malloc(1024); + data2 = tor_malloc(1024); + data3 = tor_malloc(1024); + + /* Now, test encryption and decryption with stream cipher. */ + data1[0]='\0'; + for (i = 1023; i>0; i -= 35) + strncat(data1, "Now is the time for all good onions", i); + + memset(data2, 0, 1024); + memset(data3, 0, 1024); + env1 = crypto_new_cipher_env(); + test_neq(env1, 0); + env2 = crypto_new_cipher_env(); + test_neq(env2, 0); + j = crypto_cipher_generate_key(env1); + crypto_cipher_set_key(env2, crypto_cipher_get_key(env1)); + crypto_cipher_encrypt_init_cipher(env1); + crypto_cipher_decrypt_init_cipher(env2); + + /* Try encrypting 512 chars. */ + crypto_cipher_encrypt(env1, data2, data1, 512); + crypto_cipher_decrypt(env2, data3, data2, 512); + test_memeq(data1, data3, 512); + test_memneq(data1, data2, 512); + + /* Now encrypt 1 at a time, and get 1 at a time. */ + for (j = 512; j < 560; ++j) { + crypto_cipher_encrypt(env1, data2+j, data1+j, 1); + } + for (j = 512; j < 560; ++j) { + crypto_cipher_decrypt(env2, data3+j, data2+j, 1); + } + test_memeq(data1, data3, 560); + /* Now encrypt 3 at a time, and get 5 at a time. */ + for (j = 560; j < 1024-5; j += 3) { + crypto_cipher_encrypt(env1, data2+j, data1+j, 3); + } + for (j = 560; j < 1024-5; j += 5) { + crypto_cipher_decrypt(env2, data3+j, data2+j, 5); + } + test_memeq(data1, data3, 1024-5); + /* Now make sure that when we encrypt with different chunk sizes, we get + the same results. */ + crypto_free_cipher_env(env2); + env2 = NULL; + + memset(data3, 0, 1024); + env2 = crypto_new_cipher_env(); + test_neq(env2, 0); + crypto_cipher_set_key(env2, crypto_cipher_get_key(env1)); + crypto_cipher_encrypt_init_cipher(env2); + for (j = 0; j < 1024-16; j += 17) { + crypto_cipher_encrypt(env2, data3+j, data1+j, 17); + } + for (j= 0; j < 1024-16; ++j) { + if (data2[j] != data3[j]) { + printf("%d: %d\t%d\n", j, (int) data2[j], (int) data3[j]); + } + } + test_memeq(data2, data3, 1024-16); + crypto_free_cipher_env(env1); + env1 = NULL; + crypto_free_cipher_env(env2); + env2 = NULL; + + /* NIST test vector for aes. */ + env1 = crypto_new_cipher_env(); /* IV starts at 0 */ + crypto_cipher_set_key(env1, "\x80\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00"); + crypto_cipher_encrypt_init_cipher(env1); + crypto_cipher_encrypt(env1, data1, + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00", 16); + test_memeq_hex(data1, "0EDD33D3C621E546455BD8BA1418BEC8"); + + /* Now test rollover. All these values are originally from a python + * script. */ + crypto_cipher_set_iv(env1, "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xff\xff\xff\xff\xff\xff\xff\xff"); + memset(data2, 0, 1024); + crypto_cipher_encrypt(env1, data1, data2, 32); + test_memeq_hex(data1, "335fe6da56f843199066c14a00a40231" + "cdd0b917dbc7186908a6bfb5ffd574d3"); + + crypto_cipher_set_iv(env1, "\x00\x00\x00\x00\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); + memset(data2, 0, 1024); + crypto_cipher_encrypt(env1, data1, data2, 32); + test_memeq_hex(data1, "e627c6423fa2d77832a02b2794094b73" + "3e63c721df790d2c6469cc1953a3ffac"); + + crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); + memset(data2, 0, 1024); + crypto_cipher_encrypt(env1, data1, data2, 32); + test_memeq_hex(data1, "2aed2bff0de54f9328efd070bf48f70a" + "0EDD33D3C621E546455BD8BA1418BEC8"); + + /* Now check rollover on inplace cipher. */ + crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_crypt_inplace(env1, data2, 64); + test_memeq_hex(data2, "2aed2bff0de54f9328efd070bf48f70a" + "0EDD33D3C621E546455BD8BA1418BEC8" + "93e2c5243d6839eac58503919192f7ae" + "1908e67cafa08d508816659c2e693191"); + crypto_cipher_set_iv(env1, "\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff"); + crypto_cipher_crypt_inplace(env1, data2, 64); + test_assert(tor_mem_is_zero(data2, 64)); + + done: + tor_free(mem_op_hex_tmp); + if (env1) + crypto_free_cipher_env(env1); + if (env2) + crypto_free_cipher_env(env2); + tor_free(data1); + tor_free(data2); + tor_free(data3); +} + +/** Run unit tests for our SHA-1 functionality */ +static void +test_crypto_sha(void) +{ + crypto_digest_env_t *d1 = NULL, *d2 = NULL; + int i; + char key[80]; + char digest[32]; + char data[50]; + char d_out1[DIGEST_LEN], d_out2[DIGEST256_LEN]; + char *mem_op_hex_tmp=NULL; + + /* Test SHA-1 with a test vector from the specification. */ + i = crypto_digest(data, "abc", 3); + test_memeq_hex(data, "A9993E364706816ABA3E25717850C26C9CD0D89D"); + + /* Test SHA-256 with a test vector from the specification. */ + i = crypto_digest256(data, "abc", 3, DIGEST_SHA256); + test_memeq_hex(data, "BA7816BF8F01CFEA414140DE5DAE2223B00361A3" + "96177A9CB410FF61F20015AD"); + + /* Test HMAC-SHA-1 with test cases from RFC2202. */ + + /* Case 1. */ + memset(key, 0x0b, 20); + crypto_hmac_sha1(digest, key, 20, "Hi There", 8); + test_streq(hex_str(digest, 20), + "B617318655057264E28BC0B6FB378C8EF146BE00"); + /* Case 2. */ + crypto_hmac_sha1(digest, "Jefe", 4, "what do ya want for nothing?", 28); + test_streq(hex_str(digest, 20), + "EFFCDF6AE5EB2FA2D27416D5F184DF9C259A7C79"); + + /* Case 4. */ + base16_decode(key, 25, + "0102030405060708090a0b0c0d0e0f10111213141516171819", 50); + memset(data, 0xcd, 50); + crypto_hmac_sha1(digest, key, 25, data, 50); + test_streq(hex_str(digest, 20), + "4C9007F4026250C6BC8414F9BF50C86C2D7235DA"); + + /* Case 5. */ + memset(key, 0xaa, 80); + crypto_hmac_sha1(digest, key, 80, + "Test Using Larger Than Block-Size Key - Hash Key First", + 54); + test_streq(hex_str(digest, 20), + "AA4AE5E15272D00E95705637CE8A3B55ED402112"); + + /* Incremental digest code. */ + d1 = crypto_new_digest_env(); + test_assert(d1); + crypto_digest_add_bytes(d1, "abcdef", 6); + d2 = crypto_digest_dup(d1); + test_assert(d2); + crypto_digest_add_bytes(d2, "ghijkl", 6); + crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest(d_out2, "abcdefghijkl", 12); + test_memeq(d_out1, d_out2, DIGEST_LEN); + crypto_digest_assign(d2, d1); + crypto_digest_add_bytes(d2, "mno", 3); + crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest(d_out2, "abcdefmno", 9); + test_memeq(d_out1, d_out2, DIGEST_LEN); + crypto_digest_get_digest(d1, d_out1, sizeof(d_out1)); + crypto_digest(d_out2, "abcdef", 6); + test_memeq(d_out1, d_out2, DIGEST_LEN); + crypto_free_digest_env(d1); + crypto_free_digest_env(d2); + + /* Incremental digest code with sha256 */ + d1 = crypto_new_digest256_env(DIGEST_SHA256); + test_assert(d1); + crypto_digest_add_bytes(d1, "abcdef", 6); + d2 = crypto_digest_dup(d1); + test_assert(d2); + crypto_digest_add_bytes(d2, "ghijkl", 6); + crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest256(d_out2, "abcdefghijkl", 12, DIGEST_SHA256); + test_memeq(d_out1, d_out2, DIGEST_LEN); + crypto_digest_assign(d2, d1); + crypto_digest_add_bytes(d2, "mno", 3); + crypto_digest_get_digest(d2, d_out1, sizeof(d_out1)); + crypto_digest256(d_out2, "abcdefmno", 9, DIGEST_SHA256); + test_memeq(d_out1, d_out2, DIGEST_LEN); + crypto_digest_get_digest(d1, d_out1, sizeof(d_out1)); + crypto_digest256(d_out2, "abcdef", 6, DIGEST_SHA256); + test_memeq(d_out1, d_out2, DIGEST_LEN); + + done: + if (d1) + crypto_free_digest_env(d1); + if (d2) + crypto_free_digest_env(d2); + tor_free(mem_op_hex_tmp); +} + +/** Run unit tests for our public key crypto functions */ +static void +test_crypto_pk(void) +{ + crypto_pk_env_t *pk1 = NULL, *pk2 = NULL; + char *encoded = NULL; + char data1[1024], data2[1024], data3[1024]; + size_t size; + int i, j, p, len; + + /* Public-key ciphers */ + pk1 = pk_generate(0); + pk2 = crypto_new_pk_env(); + test_assert(pk1 && pk2); + test_assert(! crypto_pk_write_public_key_to_string(pk1, &encoded, &size)); + test_assert(! crypto_pk_read_public_key_from_string(pk2, encoded, size)); + test_eq(0, crypto_pk_cmp_keys(pk1, pk2)); + + test_eq(128, crypto_pk_keysize(pk1)); + test_eq(128, crypto_pk_keysize(pk2)); + + test_eq(128, crypto_pk_public_encrypt(pk2, data1, sizeof(data1), + "Hello whirled.", 15, + PK_PKCS1_OAEP_PADDING)); + test_eq(128, crypto_pk_public_encrypt(pk1, data2, sizeof(data1), + "Hello whirled.", 15, + PK_PKCS1_OAEP_PADDING)); + /* oaep padding should make encryption not match */ + test_memneq(data1, data2, 128); + test_eq(15, crypto_pk_private_decrypt(pk1, data3, sizeof(data3), data1, 128, + PK_PKCS1_OAEP_PADDING,1)); + test_streq(data3, "Hello whirled."); + memset(data3, 0, 1024); + test_eq(15, crypto_pk_private_decrypt(pk1, data3, sizeof(data3), data2, 128, + PK_PKCS1_OAEP_PADDING,1)); + test_streq(data3, "Hello whirled."); + /* Can't decrypt with public key. */ + test_eq(-1, crypto_pk_private_decrypt(pk2, data3, sizeof(data3), data2, 128, + PK_PKCS1_OAEP_PADDING,1)); + /* Try again with bad padding */ + memcpy(data2+1, "XYZZY", 5); /* This has fails ~ once-in-2^40 */ + test_eq(-1, crypto_pk_private_decrypt(pk1, data3, sizeof(data3), data2, 128, + PK_PKCS1_OAEP_PADDING,1)); + + /* File operations: save and load private key */ + test_assert(! crypto_pk_write_private_key_to_filename(pk1, + get_fname("pkey1"))); + /* failing case for read: can't read. */ + test_assert(crypto_pk_read_private_key_from_filename(pk2, + get_fname("xyzzy")) < 0); + write_str_to_file(get_fname("xyzzy"), "foobar", 6); + /* Failing case for read: no key. */ + test_assert(crypto_pk_read_private_key_from_filename(pk2, + get_fname("xyzzy")) < 0); + test_assert(! crypto_pk_read_private_key_from_filename(pk2, + get_fname("pkey1"))); + test_eq(15, crypto_pk_private_decrypt(pk2, data3, sizeof(data3), data1, 128, + PK_PKCS1_OAEP_PADDING,1)); + + /* Now try signing. */ + strlcpy(data1, "Ossifrage", 1024); + test_eq(128, crypto_pk_private_sign(pk1, data2, sizeof(data2), data1, 10)); + test_eq(10, + crypto_pk_public_checksig(pk1, data3, sizeof(data3), data2, 128)); + test_streq(data3, "Ossifrage"); + /* Try signing digests. */ + test_eq(128, crypto_pk_private_sign_digest(pk1, data2, sizeof(data2), + data1, 10)); + test_eq(20, + crypto_pk_public_checksig(pk1, data3, sizeof(data3), data2, 128)); + test_eq(0, crypto_pk_public_checksig_digest(pk1, data1, 10, data2, 128)); + test_eq(-1, crypto_pk_public_checksig_digest(pk1, data1, 11, data2, 128)); + + /*XXXX test failed signing*/ + + /* Try encoding */ + crypto_free_pk_env(pk2); + pk2 = NULL; + i = crypto_pk_asn1_encode(pk1, data1, 1024); + test_assert(i>0); + pk2 = crypto_pk_asn1_decode(data1, i); + test_assert(crypto_pk_cmp_keys(pk1,pk2) == 0); + + /* Try with hybrid encryption wrappers. */ + crypto_rand(data1, 1024); + for (i = 0; i < 3; ++i) { + for (j = 85; j < 140; ++j) { + memset(data2,0,1024); + memset(data3,0,1024); + if (i == 0 && j < 129) + continue; + p = (i==0)?PK_NO_PADDING: + (i==1)?PK_PKCS1_PADDING:PK_PKCS1_OAEP_PADDING; + len = crypto_pk_public_hybrid_encrypt(pk1,data2,sizeof(data2), + data1,j,p,0); + test_assert(len>=0); + len = crypto_pk_private_hybrid_decrypt(pk1,data3,sizeof(data3), + data2,len,p,1); + test_eq(len,j); + test_memeq(data1,data3,j); + } + } + + /* Try copy_full */ + crypto_free_pk_env(pk2); + pk2 = crypto_pk_copy_full(pk1); + test_assert(pk2 != NULL); + test_neq_ptr(pk1, pk2); + test_assert(crypto_pk_cmp_keys(pk1,pk2) == 0); + + done: + if (pk1) + crypto_free_pk_env(pk1); + if (pk2) + crypto_free_pk_env(pk2); + tor_free(encoded); +} + +/** Run unit tests for misc crypto formatting functionality (base64, base32, + * fingerprints, etc) */ +static void +test_crypto_formats(void) +{ + char *data1 = NULL, *data2 = NULL, *data3 = NULL; + int i, j, idx; + + data1 = tor_malloc(1024); + data2 = tor_malloc(1024); + data3 = tor_malloc(1024); + test_assert(data1 && data2 && data3); + + /* Base64 tests */ + memset(data1, 6, 1024); + for (idx = 0; idx < 10; ++idx) { + i = base64_encode(data2, 1024, data1, idx); + test_assert(i >= 0); + j = base64_decode(data3, 1024, data2, i); + test_eq(j,idx); + test_memeq(data3, data1, idx); + } + + strlcpy(data1, "Test string that contains 35 chars.", 1024); + strlcat(data1, " 2nd string that contains 35 chars.", 1024); + + i = base64_encode(data2, 1024, data1, 71); + test_assert(i >= 0); + j = base64_decode(data3, 1024, data2, i); + test_eq(j, 71); + test_streq(data3, data1); + test_assert(data2[i] == '\0'); + + crypto_rand(data1, DIGEST_LEN); + memset(data2, 100, 1024); + digest_to_base64(data2, data1); + test_eq(BASE64_DIGEST_LEN, strlen(data2)); + test_eq(100, data2[BASE64_DIGEST_LEN+2]); + memset(data3, 99, 1024); + test_eq(digest_from_base64(data3, data2), 0); + test_memeq(data1, data3, DIGEST_LEN); + test_eq(99, data3[DIGEST_LEN+1]); + + test_assert(digest_from_base64(data3, "###") < 0); + + /* Encoding SHA256 */ + crypto_rand(data2, DIGEST256_LEN); + memset(data2, 100, 1024); + digest256_to_base64(data2, data1); + test_eq(BASE64_DIGEST256_LEN, strlen(data2)); + test_eq(100, data2[BASE64_DIGEST256_LEN+2]); + memset(data3, 99, 1024); + test_eq(digest256_from_base64(data3, data2), 0); + test_memeq(data1, data3, DIGEST256_LEN); + test_eq(99, data3[DIGEST256_LEN+1]); + + /* Base32 tests */ + strlcpy(data1, "5chrs", 1024); + /* bit pattern is: [35 63 68 72 73] -> + * [00110101 01100011 01101000 01110010 01110011] + * By 5s: [00110 10101 10001 10110 10000 11100 10011 10011] + */ + base32_encode(data2, 9, data1, 5); + test_streq(data2, "gvrwq4tt"); + + strlcpy(data1, "\xFF\xF5\x6D\x44\xAE\x0D\x5C\xC9\x62\xC4", 1024); + base32_encode(data2, 30, data1, 10); + test_streq(data2, "772w2rfobvomsywe"); + + /* Base16 tests */ + strlcpy(data1, "6chrs\xff", 1024); + base16_encode(data2, 13, data1, 6); + test_streq(data2, "3663687273FF"); + + strlcpy(data1, "f0d678affc000100", 1024); + i = base16_decode(data2, 8, data1, 16); + test_eq(i,0); + test_memeq(data2, "\xf0\xd6\x78\xaf\xfc\x00\x01\x00",8); + + /* now try some failing base16 decodes */ + test_eq(-1, base16_decode(data2, 8, data1, 15)); /* odd input len */ + test_eq(-1, base16_decode(data2, 7, data1, 16)); /* dest too short */ + strlcpy(data1, "f0dz!8affc000100", 1024); + test_eq(-1, base16_decode(data2, 8, data1, 16)); + + tor_free(data1); + tor_free(data2); + tor_free(data3); + + /* Add spaces to fingerprint */ + { + data1 = tor_strdup("ABCD1234ABCD56780000ABCD1234ABCD56780000"); + test_eq(strlen(data1), 40); + data2 = tor_malloc(FINGERPRINT_LEN+1); + add_spaces_to_fp(data2, FINGERPRINT_LEN+1, data1); + test_streq(data2, "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 0000"); + tor_free(data1); + tor_free(data2); + } + + /* Check fingerprint */ + { + test_assert(crypto_pk_check_fingerprint_syntax( + "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 0000")); + test_assert(!crypto_pk_check_fingerprint_syntax( + "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 000")); + test_assert(!crypto_pk_check_fingerprint_syntax( + "ABCD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 00000")); + test_assert(!crypto_pk_check_fingerprint_syntax( + "ABCD 1234 ABCD 5678 0000 ABCD1234 ABCD 5678 0000")); + test_assert(!crypto_pk_check_fingerprint_syntax( + "ABCD 1234 ABCD 5678 0000 ABCD1234 ABCD 5678 00000")); + test_assert(!crypto_pk_check_fingerprint_syntax( + "ACD 1234 ABCD 5678 0000 ABCD 1234 ABCD 5678 00000")); + } + + done: + tor_free(data1); + tor_free(data2); + tor_free(data3); +} + +/** Run unit tests for our secret-to-key passphrase hashing functionality. */ +static void +test_crypto_s2k(void) +{ + char buf[29]; + char buf2[29]; + char *buf3 = NULL; + int i; + + memset(buf, 0, sizeof(buf)); + memset(buf2, 0, sizeof(buf2)); + buf3 = tor_malloc(65536); + memset(buf3, 0, 65536); + + secret_to_key(buf+9, 20, "", 0, buf); + crypto_digest(buf2+9, buf3, 1024); + test_memeq(buf, buf2, 29); + + memcpy(buf,"vrbacrda",8); + memcpy(buf2,"vrbacrda",8); + buf[8] = 96; + buf2[8] = 96; + secret_to_key(buf+9, 20, "12345678", 8, buf); + for (i = 0; i < 65536; i += 16) { + memcpy(buf3+i, "vrbacrda12345678", 16); + } + crypto_digest(buf2+9, buf3, 65536); + test_memeq(buf, buf2, 29); + + done: + tor_free(buf3); +} + +/** Test AES-CTR encryption and decryption with IV. */ +static void +test_crypto_aes_iv(void) +{ + crypto_cipher_env_t *cipher; + char *plain, *encrypted1, *encrypted2, *decrypted1, *decrypted2; + char plain_1[1], plain_15[15], plain_16[16], plain_17[17]; + char key1[16], key2[16]; + ssize_t encrypted_size, decrypted_size; + + plain = tor_malloc(4095); + encrypted1 = tor_malloc(4095 + 1 + 16); + encrypted2 = tor_malloc(4095 + 1 + 16); + decrypted1 = tor_malloc(4095 + 1); + decrypted2 = tor_malloc(4095 + 1); + + crypto_rand(plain, 4095); + crypto_rand(key1, 16); + crypto_rand(key2, 16); + crypto_rand(plain_1, 1); + crypto_rand(plain_15, 15); + crypto_rand(plain_16, 16); + crypto_rand(plain_17, 17); + key1[0] = key2[0] + 128; /* Make sure that contents are different. */ + /* Encrypt and decrypt with the same key. */ + cipher = crypto_create_init_cipher(key1, 1); + encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 4095, + plain, 4095); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(encrypted_size, 16 + 4095); + tor_assert(encrypted_size > 0); /* This is obviously true, since 4111 is + * greater than 0, but its truth is not + * obvious to all analysis tools. */ + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 4095, + encrypted1, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(decrypted_size, 4095); + tor_assert(decrypted_size > 0); + test_memeq(plain, decrypted1, 4095); + /* Encrypt a second time (with a new random initialization vector). */ + cipher = crypto_create_init_cipher(key1, 1); + encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted2, 16 + 4095, + plain, 4095); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(encrypted_size, 16 + 4095); + tor_assert(encrypted_size > 0); + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted2, 4095, + encrypted2, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(decrypted_size, 4095); + tor_assert(decrypted_size > 0); + test_memeq(plain, decrypted2, 4095); + test_memneq(encrypted1, encrypted2, encrypted_size); + /* Decrypt with the wrong key. */ + cipher = crypto_create_init_cipher(key2, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted2, 4095, + encrypted1, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_memneq(plain, decrypted2, encrypted_size); + /* Alter the initialization vector. */ + encrypted1[0] += 42; + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 4095, + encrypted1, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_memneq(plain, decrypted2, 4095); + /* Special length case: 1. */ + cipher = crypto_create_init_cipher(key1, 1); + encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 1, + plain_1, 1); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(encrypted_size, 16 + 1); + tor_assert(encrypted_size > 0); + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 1, + encrypted1, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(decrypted_size, 1); + tor_assert(decrypted_size > 0); + test_memeq(plain_1, decrypted1, 1); + /* Special length case: 15. */ + cipher = crypto_create_init_cipher(key1, 1); + encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 15, + plain_15, 15); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(encrypted_size, 16 + 15); + tor_assert(encrypted_size > 0); + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 15, + encrypted1, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(decrypted_size, 15); + tor_assert(decrypted_size > 0); + test_memeq(plain_15, decrypted1, 15); + /* Special length case: 16. */ + cipher = crypto_create_init_cipher(key1, 1); + encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 16, + plain_16, 16); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(encrypted_size, 16 + 16); + tor_assert(encrypted_size > 0); + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 16, + encrypted1, encrypted_size); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(decrypted_size, 16); + tor_assert(decrypted_size > 0); + test_memeq(plain_16, decrypted1, 16); + /* Special length case: 17. */ + cipher = crypto_create_init_cipher(key1, 1); + encrypted_size = crypto_cipher_encrypt_with_iv(cipher, encrypted1, 16 + 17, + plain_17, 17); + crypto_free_cipher_env(cipher); + cipher = NULL; + test_eq(encrypted_size, 16 + 17); + tor_assert(encrypted_size > 0); + cipher = crypto_create_init_cipher(key1, 0); + decrypted_size = crypto_cipher_decrypt_with_iv(cipher, decrypted1, 17, + encrypted1, encrypted_size); + test_eq(decrypted_size, 17); + tor_assert(decrypted_size > 0); + test_memeq(plain_17, decrypted1, 17); + + done: + /* Free memory. */ + tor_free(plain); + tor_free(encrypted1); + tor_free(encrypted2); + tor_free(decrypted1); + tor_free(decrypted2); + if (cipher) + crypto_free_cipher_env(cipher); +} + +/** Test base32 decoding. */ +static void +test_crypto_base32_decode(void) +{ + char plain[60], encoded[96 + 1], decoded[60]; + int res; + crypto_rand(plain, 60); + /* Encode and decode a random string. */ + base32_encode(encoded, 96 + 1, plain, 60); + res = base32_decode(decoded, 60, encoded, 96); + test_eq(res, 0); + test_memeq(plain, decoded, 60); + /* Encode, uppercase, and decode a random string. */ + base32_encode(encoded, 96 + 1, plain, 60); + tor_strupper(encoded); + res = base32_decode(decoded, 60, encoded, 96); + test_eq(res, 0); + test_memeq(plain, decoded, 60); + /* Change encoded string and decode. */ + if (encoded[0] == 'A' || encoded[0] == 'a') + encoded[0] = 'B'; + else + encoded[0] = 'A'; + res = base32_decode(decoded, 60, encoded, 96); + test_eq(res, 0); + test_memneq(plain, decoded, 60); + /* Bad encodings. */ + encoded[0] = '!'; + res = base32_decode(decoded, 60, encoded, 96); + test_assert(res < 0); + + done: + ; +} + +#define CRYPTO_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_crypto_ ## name } + +struct testcase_t crypto_tests[] = { + CRYPTO_LEGACY(formats), + CRYPTO_LEGACY(rng), + CRYPTO_LEGACY(aes), + CRYPTO_LEGACY(sha), + CRYPTO_LEGACY(pk), + CRYPTO_LEGACY(dh), + CRYPTO_LEGACY(s2k), + CRYPTO_LEGACY(aes_iv), + CRYPTO_LEGACY(base32_decode), + END_OF_TESTCASES +}; + diff --git a/src/or/test_data.c b/src/test/test_data.c index fc8585761..fc8585761 100644 --- a/src/or/test_data.c +++ b/src/test/test_data.c diff --git a/src/test/test_dir.c b/src/test/test_dir.c new file mode 100644 index 000000000..8fd94289a --- /dev/null +++ b/src/test/test_dir.c @@ -0,0 +1,1320 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#define DIRSERV_PRIVATE +#define DIRVOTE_PRIVATE +#define ROUTER_PRIVATE +#include "or.h" +#include "directory.h" +#include "dirserv.h" +#include "dirvote.h" +#include "networkstatus.h" +#include "router.h" +#include "routerlist.h" +#include "routerparse.h" +#include "test.h" + +static void +test_dir_nicknames(void) +{ + test_assert( is_legal_nickname("a")); + test_assert(!is_legal_nickname("")); + test_assert(!is_legal_nickname("abcdefghijklmnopqrst")); /* 20 chars */ + test_assert(!is_legal_nickname("hyphen-")); /* bad char */ + test_assert( is_legal_nickname("abcdefghijklmnopqrs")); /* 19 chars */ + test_assert(!is_legal_nickname("$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA")); + /* valid */ + test_assert( is_legal_nickname_or_hexdigest( + "$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA")); + test_assert( is_legal_nickname_or_hexdigest( + "$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA=fred")); + test_assert( is_legal_nickname_or_hexdigest( + "$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA~fred")); + /* too short */ + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")); + /* illegal char */ + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAzAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")); + /* hex part too long */ + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")); + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=fred")); + /* Bad nickname */ + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")); + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~")); + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~hyphen-")); + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~" + "abcdefghijklmnoppqrst")); + /* Bad extra char. */ + test_assert(!is_legal_nickname_or_hexdigest( + "$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA!")); + test_assert(is_legal_nickname_or_hexdigest("xyzzy")); + test_assert(is_legal_nickname_or_hexdigest("abcdefghijklmnopqrs")); + test_assert(!is_legal_nickname_or_hexdigest("abcdefghijklmnopqrst")); + done: + ; +} + +/** Run unit tests for router descriptor generation logic. */ +static void +test_dir_formats(void) +{ + char buf[8192], buf2[8192]; + char platform[256]; + char fingerprint[FINGERPRINT_LEN+1]; + char *pk1_str = NULL, *pk2_str = NULL, *pk3_str = NULL, *cp; + size_t pk1_str_len, pk2_str_len, pk3_str_len; + routerinfo_t *r1=NULL, *r2=NULL; + crypto_pk_env_t *pk1 = NULL, *pk2 = NULL, *pk3 = NULL; + routerinfo_t *rp1 = NULL; + addr_policy_t *ex1, *ex2; + routerlist_t *dir1 = NULL, *dir2 = NULL; + + pk1 = pk_generate(0); + pk2 = pk_generate(1); + pk3 = pk_generate(2); + + test_assert(pk1 && pk2 && pk3); + + get_platform_str(platform, sizeof(platform)); + r1 = tor_malloc_zero(sizeof(routerinfo_t)); + r1->address = tor_strdup("18.244.0.1"); + r1->addr = 0xc0a80001u; /* 192.168.0.1 */ + r1->cache_info.published_on = 0; + r1->or_port = 9000; + r1->dir_port = 9003; + r1->onion_pkey = crypto_pk_dup_key(pk1); + r1->identity_pkey = crypto_pk_dup_key(pk2); + r1->bandwidthrate = 1000; + r1->bandwidthburst = 5000; + r1->bandwidthcapacity = 10000; + r1->exit_policy = NULL; + r1->nickname = tor_strdup("Magri"); + r1->platform = tor_strdup(platform); + + ex1 = tor_malloc_zero(sizeof(addr_policy_t)); + ex2 = tor_malloc_zero(sizeof(addr_policy_t)); + ex1->policy_type = ADDR_POLICY_ACCEPT; + tor_addr_from_ipv4h(&ex1->addr, 0); + ex1->maskbits = 0; + ex1->prt_min = ex1->prt_max = 80; + ex2->policy_type = ADDR_POLICY_REJECT; + tor_addr_from_ipv4h(&ex2->addr, 18<<24); + ex2->maskbits = 8; + ex2->prt_min = ex2->prt_max = 24; + r2 = tor_malloc_zero(sizeof(routerinfo_t)); + r2->address = tor_strdup("1.1.1.1"); + r2->addr = 0x0a030201u; /* 10.3.2.1 */ + r2->platform = tor_strdup(platform); + r2->cache_info.published_on = 5; + r2->or_port = 9005; + r2->dir_port = 0; + r2->onion_pkey = crypto_pk_dup_key(pk2); + r2->identity_pkey = crypto_pk_dup_key(pk1); + r2->bandwidthrate = r2->bandwidthburst = r2->bandwidthcapacity = 3000; + r2->exit_policy = smartlist_create(); + smartlist_add(r2->exit_policy, ex2); + smartlist_add(r2->exit_policy, ex1); + r2->nickname = tor_strdup("Fred"); + + test_assert(!crypto_pk_write_public_key_to_string(pk1, &pk1_str, + &pk1_str_len)); + test_assert(!crypto_pk_write_public_key_to_string(pk2 , &pk2_str, + &pk2_str_len)); + test_assert(!crypto_pk_write_public_key_to_string(pk3 , &pk3_str, + &pk3_str_len)); + + memset(buf, 0, 2048); + test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); + + strlcpy(buf2, "router Magri 18.244.0.1 9000 0 9003\n" + "platform Tor "VERSION" on ", sizeof(buf2)); + strlcat(buf2, get_uname(), sizeof(buf2)); + strlcat(buf2, "\n" + "opt protocols Link 1 2 Circuit 1\n" + "published 1970-01-01 00:00:00\n" + "opt fingerprint ", sizeof(buf2)); + test_assert(!crypto_pk_get_fingerprint(pk2, fingerprint, 1)); + strlcat(buf2, fingerprint, sizeof(buf2)); + strlcat(buf2, "\nuptime 0\n" + /* XXX the "0" above is hard-coded, but even if we made it reflect + * uptime, that still wouldn't make it right, because the two + * descriptors might be made on different seconds... hm. */ + "bandwidth 1000 5000 10000\n" + "onion-key\n", sizeof(buf2)); + strlcat(buf2, pk1_str, sizeof(buf2)); + strlcat(buf2, "signing-key\n", sizeof(buf2)); + strlcat(buf2, pk2_str, sizeof(buf2)); + strlcat(buf2, "opt hidden-service-dir\n", sizeof(buf2)); + strlcat(buf2, "reject *:*\nrouter-signature\n", sizeof(buf2)); + buf[strlen(buf2)] = '\0'; /* Don't compare the sig; it's never the same + * twice */ + + test_streq(buf, buf2); + + test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); + cp = buf; + rp1 = router_parse_entry_from_string((const char*)cp,NULL,1,0,NULL); + test_assert(rp1); + test_streq(rp1->address, r1->address); + test_eq(rp1->or_port, r1->or_port); + //test_eq(rp1->dir_port, r1->dir_port); + test_eq(rp1->bandwidthrate, r1->bandwidthrate); + test_eq(rp1->bandwidthburst, r1->bandwidthburst); + test_eq(rp1->bandwidthcapacity, r1->bandwidthcapacity); + test_assert(crypto_pk_cmp_keys(rp1->onion_pkey, pk1) == 0); + test_assert(crypto_pk_cmp_keys(rp1->identity_pkey, pk2) == 0); + //test_assert(rp1->exit_policy == NULL); + +#if 0 + /* XXX Once we have exit policies, test this again. XXX */ + strlcpy(buf2, "router tor.tor.tor 9005 0 0 3000\n", sizeof(buf2)); + strlcat(buf2, pk2_str, sizeof(buf2)); + strlcat(buf2, "signing-key\n", sizeof(buf2)); + strlcat(buf2, pk1_str, sizeof(buf2)); + strlcat(buf2, "accept *:80\nreject 18.*:24\n\n", sizeof(buf2)); + test_assert(router_dump_router_to_string(buf, 2048, &r2, pk2)>0); + test_streq(buf, buf2); + + cp = buf; + rp2 = router_parse_entry_from_string(&cp,1); + test_assert(rp2); + test_streq(rp2->address, r2.address); + test_eq(rp2->or_port, r2.or_port); + test_eq(rp2->dir_port, r2.dir_port); + test_eq(rp2->bandwidth, r2.bandwidth); + test_assert(crypto_pk_cmp_keys(rp2->onion_pkey, pk2) == 0); + test_assert(crypto_pk_cmp_keys(rp2->identity_pkey, pk1) == 0); + test_eq(rp2->exit_policy->policy_type, EXIT_POLICY_ACCEPT); + test_streq(rp2->exit_policy->string, "accept *:80"); + test_streq(rp2->exit_policy->address, "*"); + test_streq(rp2->exit_policy->port, "80"); + test_eq(rp2->exit_policy->next->policy_type, EXIT_POLICY_REJECT); + test_streq(rp2->exit_policy->next->string, "reject 18.*:24"); + test_streq(rp2->exit_policy->next->address, "18.*"); + test_streq(rp2->exit_policy->next->port, "24"); + test_assert(rp2->exit_policy->next->next == NULL); + + /* Okay, now for the directories. */ + { + fingerprint_list = smartlist_create(); + crypto_pk_get_fingerprint(pk2, buf, 1); + add_fingerprint_to_dir("Magri", buf, fingerprint_list); + crypto_pk_get_fingerprint(pk1, buf, 1); + add_fingerprint_to_dir("Fred", buf, fingerprint_list); + } + + { + char d[DIGEST_LEN]; + const char *m; + /* XXXX NM re-enable. */ + /* Make sure routers aren't too far in the past any more. */ + r1->cache_info.published_on = time(NULL); + r2->cache_info.published_on = time(NULL)-3*60*60; + test_assert(router_dump_router_to_string(buf, 2048, r1, pk2)>0); + test_eq(dirserv_add_descriptor(buf,&m,""), ROUTER_ADDED_NOTIFY_GENERATOR); + test_assert(router_dump_router_to_string(buf, 2048, r2, pk1)>0); + test_eq(dirserv_add_descriptor(buf,&m,""), ROUTER_ADDED_NOTIFY_GENERATOR); + get_options()->Nickname = tor_strdup("DirServer"); + test_assert(!dirserv_dump_directory_to_string(&cp,pk3, 0)); + crypto_pk_get_digest(pk3, d); + test_assert(!router_parse_directory(cp)); + test_eq(2, smartlist_len(dir1->routers)); + tor_free(cp); + } +#endif + dirserv_free_fingerprint_list(); + + done: + if (r1) + routerinfo_free(r1); + if (r2) + routerinfo_free(r2); + + tor_free(pk1_str); + tor_free(pk2_str); + tor_free(pk3_str); + if (pk1) crypto_free_pk_env(pk1); + if (pk2) crypto_free_pk_env(pk2); + if (pk3) crypto_free_pk_env(pk3); + if (rp1) routerinfo_free(rp1); + tor_free(dir1); /* XXXX And more !*/ + tor_free(dir2); /* And more !*/ +} + +static void +test_dir_versions(void) +{ + tor_version_t ver1; + + /* Try out version parsing functionality */ + test_eq(0, tor_version_parse("0.3.4pre2-cvs", &ver1)); + test_eq(0, ver1.major); + test_eq(3, ver1.minor); + test_eq(4, ver1.micro); + test_eq(VER_PRE, ver1.status); + test_eq(2, ver1.patchlevel); + test_eq(0, tor_version_parse("0.3.4rc1", &ver1)); + test_eq(0, ver1.major); + test_eq(3, ver1.minor); + test_eq(4, ver1.micro); + test_eq(VER_RC, ver1.status); + test_eq(1, ver1.patchlevel); + test_eq(0, tor_version_parse("1.3.4", &ver1)); + test_eq(1, ver1.major); + test_eq(3, ver1.minor); + test_eq(4, ver1.micro); + test_eq(VER_RELEASE, ver1.status); + test_eq(0, ver1.patchlevel); + test_eq(0, tor_version_parse("1.3.4.999", &ver1)); + test_eq(1, ver1.major); + test_eq(3, ver1.minor); + test_eq(4, ver1.micro); + test_eq(VER_RELEASE, ver1.status); + test_eq(999, ver1.patchlevel); + test_eq(0, tor_version_parse("0.1.2.4-alpha", &ver1)); + test_eq(0, ver1.major); + test_eq(1, ver1.minor); + test_eq(2, ver1.micro); + test_eq(4, ver1.patchlevel); + test_eq(VER_RELEASE, ver1.status); + test_streq("alpha", ver1.status_tag); + test_eq(0, tor_version_parse("0.1.2.4", &ver1)); + test_eq(0, ver1.major); + test_eq(1, ver1.minor); + test_eq(2, ver1.micro); + test_eq(4, ver1.patchlevel); + test_eq(VER_RELEASE, ver1.status); + test_streq("", ver1.status_tag); + +#define tt_versionstatus_op(vs1, op, vs2) \ + tt_assert_test_type(vs1,vs2,#vs1" "#op" "#vs2,version_status_t, \ + (_val1 op _val2),"%d") +#define test_v_i_o(val, ver, lst) \ + tt_versionstatus_op(val, ==, tor_version_is_obsolete(ver, lst)) + + /* make sure tor_version_is_obsolete() works */ + test_v_i_o(VS_OLD, "0.0.1", "Tor 0.0.2"); + test_v_i_o(VS_OLD, "0.0.1", "0.0.2, Tor 0.0.3"); + test_v_i_o(VS_OLD, "0.0.1", "0.0.2,Tor 0.0.3"); + test_v_i_o(VS_OLD, "0.0.1","0.0.3,BetterTor 0.0.1"); + test_v_i_o(VS_RECOMMENDED, "0.0.2", "Tor 0.0.2,Tor 0.0.3"); + test_v_i_o(VS_NEW_IN_SERIES, "0.0.2", "Tor 0.0.2pre1,Tor 0.0.3"); + test_v_i_o(VS_OLD, "0.0.2", "Tor 0.0.2.1,Tor 0.0.3"); + test_v_i_o(VS_NEW, "0.1.0", "Tor 0.0.2,Tor 0.0.3"); + test_v_i_o(VS_RECOMMENDED, "0.0.7rc2", "0.0.7,Tor 0.0.7rc2,Tor 0.0.8"); + test_v_i_o(VS_OLD, "0.0.5.0", "0.0.5.1-cvs"); + test_v_i_o(VS_NEW_IN_SERIES, "0.0.5.1-cvs", "0.0.5, 0.0.6"); + /* Not on list, but newer than any in same series. */ + test_v_i_o(VS_NEW_IN_SERIES, "0.1.0.3", + "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); + /* Series newer than any on list. */ + test_v_i_o(VS_NEW, "0.1.2.3", "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); + /* Series older than any on list. */ + test_v_i_o(VS_OLD, "0.0.1.3", "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); + /* Not on list, not newer than any on same series. */ + test_v_i_o(VS_UNRECOMMENDED, "0.1.0.1", + "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); + /* On list, not newer than any on same series. */ + test_v_i_o(VS_UNRECOMMENDED, + "0.1.0.1", "Tor 0.1.0.2,Tor 0.0.9.5,Tor 0.1.1.0"); + test_eq(0, tor_version_as_new_as("Tor 0.0.5", "0.0.9pre1-cvs")); + test_eq(1, tor_version_as_new_as( + "Tor 0.0.8 on Darwin 64-121-192-100.c3-0." + "sfpo-ubr1.sfrn-sfpo.ca.cable.rcn.com Power Macintosh", + "0.0.8rc2")); + test_eq(0, tor_version_as_new_as( + "Tor 0.0.8 on Darwin 64-121-192-100.c3-0." + "sfpo-ubr1.sfrn-sfpo.ca.cable.rcn.com Power Macintosh", "0.0.8.2")); + + /* Now try svn revisions. */ + test_eq(1, tor_version_as_new_as("Tor 0.2.1.0-dev (r100)", + "Tor 0.2.1.0-dev (r99)")); + test_eq(1, tor_version_as_new_as("Tor 0.2.1.0-dev (r100) on Banana Jr", + "Tor 0.2.1.0-dev (r99) on Hal 9000")); + test_eq(1, tor_version_as_new_as("Tor 0.2.1.0-dev (r100)", + "Tor 0.2.1.0-dev on Colossus")); + test_eq(0, tor_version_as_new_as("Tor 0.2.1.0-dev (r99)", + "Tor 0.2.1.0-dev (r100)")); + test_eq(0, tor_version_as_new_as("Tor 0.2.1.0-dev (r99) on MCP", + "Tor 0.2.1.0-dev (r100) on AM")); + test_eq(0, tor_version_as_new_as("Tor 0.2.1.0-dev", + "Tor 0.2.1.0-dev (r99)")); + test_eq(1, tor_version_as_new_as("Tor 0.2.1.1", + "Tor 0.2.1.0-dev (r99)")); + + /* Now try git revisions */ + test_eq(0, tor_version_parse("0.5.6.7 (git-ff00ff)", &ver1)); + test_eq(0, ver1.major); + test_eq(5, ver1.minor); + test_eq(6, ver1.micro); + test_eq(7, ver1.patchlevel); + test_eq(3, ver1.git_tag_len); + test_memeq(ver1.git_tag, "\xff\x00\xff", 3); + test_eq(-1, tor_version_parse("0.5.6.7 (git-ff00xx)", &ver1)); + test_eq(-1, tor_version_parse("0.5.6.7 (git-ff00fff)", &ver1)); + test_eq(0, tor_version_parse("0.5.6.7 (git ff00fff)", &ver1)); + done: + ; +} + +/** Run unit tests for directory fp_pair functions. */ +static void +test_dir_fp_pairs(void) +{ + smartlist_t *sl = smartlist_create(); + fp_pair_t *pair; + + dir_split_resource_into_fingerprint_pairs( + /* Two pairs, out of order, with one duplicate. */ + "73656372657420646174612E0000000000FFFFFF-" + "557365204145532d32353620696e73746561642e+" + "73656372657420646174612E0000000000FFFFFF-" + "557365204145532d32353620696e73746561642e+" + "48657861646563696d616c2069736e277420736f-" + "676f6f6420666f7220686964696e6720796f7572.z", sl); + + test_eq(smartlist_len(sl), 2); + pair = smartlist_get(sl, 0); + test_memeq(pair->first, "Hexadecimal isn't so", DIGEST_LEN); + test_memeq(pair->second, "good for hiding your", DIGEST_LEN); + pair = smartlist_get(sl, 1); + test_memeq(pair->first, "secret data.\0\0\0\0\0\xff\xff\xff", DIGEST_LEN); + test_memeq(pair->second, "Use AES-256 instead.", DIGEST_LEN); + + done: + SMARTLIST_FOREACH(sl, fp_pair_t *, pair, tor_free(pair)); + smartlist_free(sl); +} + +static void +test_dir_split_fps(void *testdata) +{ + smartlist_t *sl = smartlist_create(); + char *mem_op_hex_tmp = NULL; + (void)testdata; + + /* Some example hex fingerprints and their base64 equivalents */ +#define HEX1 "Fe0daff89127389bc67558691231234551193EEE" +#define HEX2 "Deadbeef99999991111119999911111111f00ba4" +#define HEX3 "b33ff00db33ff00db33ff00db33ff00db33ff00d" +#define HEX256_1 \ + "f3f3f3f3fbbbbf3f3f3f3fbbbf3f3f3f3fbbbbf3f3f3f3fbbbf3f3f3f3fbbbbf" +#define HEX256_2 \ + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccCCc" +#define HEX256_3 \ + "0123456789ABCdef0123456789ABCdef0123456789ABCdef0123456789ABCdef" +#define B64_1 "/g2v+JEnOJvGdVhpEjEjRVEZPu4" +#define B64_2 "3q2+75mZmZERERmZmRERERHwC6Q" +#define B64_3 "sz/wDbM/8A2zP/ANsz/wDbM/8A0" +#define B64_256_1 "8/Pz8/u7vz8/Pz+7vz8/Pz+7u/Pz8/P7u/Pz8/P7u78" +#define B64_256_2 "zMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMw" +#define B64_256_3 "ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8" + + /* no flags set */ + dir_split_resource_into_fingerprints("A+C+B", sl, NULL, 0); + tt_int_op(smartlist_len(sl), ==, 3); + tt_str_op(smartlist_get(sl, 0), ==, "A"); + tt_str_op(smartlist_get(sl, 1), ==, "C"); + tt_str_op(smartlist_get(sl, 2), ==, "B"); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* uniq strings. */ + dir_split_resource_into_fingerprints("A+C+B+A+B+B", sl, NULL, DSR_SORT_UNIQ); + tt_int_op(smartlist_len(sl), ==, 3); + tt_str_op(smartlist_get(sl, 0), ==, "A"); + tt_str_op(smartlist_get(sl, 1), ==, "B"); + tt_str_op(smartlist_get(sl, 2), ==, "C"); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Decode hex. */ + dir_split_resource_into_fingerprints(HEX1"+"HEX2, sl, NULL, DSR_HEX); + tt_int_op(smartlist_len(sl), ==, 2); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX1); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX2); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* decode hex and drop weirdness. */ + dir_split_resource_into_fingerprints(HEX1"+bogus+"HEX2"+"HEX256_1, + sl, NULL, DSR_HEX); + tt_int_op(smartlist_len(sl), ==, 2); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX1); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX2); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Decode long hex */ + dir_split_resource_into_fingerprints(HEX256_1"+"HEX256_2"+"HEX2"+"HEX256_3, + sl, NULL, DSR_HEX|DSR_DIGEST256); + tt_int_op(smartlist_len(sl), ==, 3); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX256_1); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX256_2); + test_mem_op_hex(smartlist_get(sl, 2), ==, HEX256_3); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Decode hex and sort. */ + dir_split_resource_into_fingerprints(HEX1"+"HEX2"+"HEX3"+"HEX2, + sl, NULL, DSR_HEX|DSR_SORT_UNIQ); + tt_int_op(smartlist_len(sl), ==, 3); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX3); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX2); + test_mem_op_hex(smartlist_get(sl, 2), ==, HEX1); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Decode long hex and sort */ + dir_split_resource_into_fingerprints(HEX256_1"+"HEX256_2"+"HEX256_3 + "+"HEX256_1, + sl, NULL, + DSR_HEX|DSR_DIGEST256|DSR_SORT_UNIQ); + tt_int_op(smartlist_len(sl), ==, 3); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX256_3); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX256_2); + test_mem_op_hex(smartlist_get(sl, 2), ==, HEX256_1); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Decode base64 */ + dir_split_resource_into_fingerprints(B64_1"-"B64_2, sl, NULL, DSR_BASE64); + tt_int_op(smartlist_len(sl), ==, 2); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX1); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX2); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + /* Decode long base64 */ + dir_split_resource_into_fingerprints(B64_256_1"-"B64_256_2, + sl, NULL, DSR_BASE64|DSR_DIGEST256); + tt_int_op(smartlist_len(sl), ==, 2); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX256_1); + test_mem_op_hex(smartlist_get(sl, 1), ==, HEX256_2); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + dir_split_resource_into_fingerprints(B64_256_1, + sl, NULL, DSR_BASE64|DSR_DIGEST256); + tt_int_op(smartlist_len(sl), ==, 1); + test_mem_op_hex(smartlist_get(sl, 0), ==, HEX256_1); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + done: + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_free(sl); + tor_free(mem_op_hex_tmp); +} + +static void +test_dir_measured_bw(void) +{ + measured_bw_line_t mbwl; + int i; + const char *lines_pass[] = { + "node_id=$557365204145532d32353620696e73746561642e bw=1024\n", + "node_id=$557365204145532d32353620696e73746561642e\t bw=1024 \n", + " node_id=$557365204145532d32353620696e73746561642e bw=1024\n", + "\tnoise\tnode_id=$557365204145532d32353620696e73746561642e " + "bw=1024 junk=007\n", + "misc=junk node_id=$557365204145532d32353620696e73746561642e " + "bw=1024 junk=007\n", + "end" + }; + const char *lines_fail[] = { + /* Test possible python stupidity on input */ + "node_id=None bw=1024\n", + "node_id=$None bw=1024\n", + "node_id=$557365204145532d32353620696e73746561642e bw=None\n", + "node_id=$557365204145532d32353620696e73746561642e bw=1024.0\n", + "node_id=$557365204145532d32353620696e73746561642e bw=.1024\n", + "node_id=$557365204145532d32353620696e73746561642e bw=1.024\n", + "node_id=$557365204145532d32353620696e73746561642e bw=1024 bw=0\n", + "node_id=$557365204145532d32353620696e73746561642e bw=1024 bw=None\n", + "node_id=$557365204145532d32353620696e73746561642e bw=-1024\n", + /* Test incomplete writes due to race conditions, partial copies, etc */ + "node_i", + "node_i\n", + "node_id=", + "node_id=\n", + "node_id=$557365204145532d32353620696e73746561642e bw=", + "node_id=$557365204145532d32353620696e73746561642e bw=1024", + "node_id=$557365204145532d32353620696e73746561642e bw=\n", + "node_id=$557365204145532d32353620696e7374", + "node_id=$557365204145532d32353620696e7374\n", + "", + "\n", + " \n ", + " \n\n", + /* Test assorted noise */ + " node_id= ", + "node_id==$557365204145532d32353620696e73746561642e bw==1024\n", + "node_id=$55736520414552d32353620696e73746561642e bw=1024\n", + "node_id=557365204145532d32353620696e73746561642e bw=1024\n", + "node_id= $557365204145532d32353620696e73746561642e bw=0.23\n", + "end" + }; + + for (i = 0; strcmp(lines_fail[i], "end"); i++) { + //fprintf(stderr, "Testing: %s\n", lines_fail[i]); + test_assert(measured_bw_line_parse(&mbwl, lines_fail[i]) == -1); + } + + for (i = 0; strcmp(lines_pass[i], "end"); i++) { + //fprintf(stderr, "Testing: %s %d\n", lines_pass[i], TOR_ISSPACE('\n')); + test_assert(measured_bw_line_parse(&mbwl, lines_pass[i]) == 0); + test_assert(mbwl.bw == 1024); + test_assert(strcmp(mbwl.node_hex, + "557365204145532d32353620696e73746561642e") == 0); + } + + done: + return; +} + +static void +test_dir_param_voting(void) +{ + networkstatus_t vote1, vote2, vote3, vote4; + smartlist_t *votes = smartlist_create(); + char *res = NULL; + + /* dirvote_compute_params only looks at the net_params field of the votes, + so that's all we need to set. + */ + memset(&vote1, 0, sizeof(vote1)); + memset(&vote2, 0, sizeof(vote2)); + memset(&vote3, 0, sizeof(vote3)); + memset(&vote4, 0, sizeof(vote4)); + vote1.net_params = smartlist_create(); + vote2.net_params = smartlist_create(); + vote3.net_params = smartlist_create(); + vote4.net_params = smartlist_create(); + smartlist_split_string(vote1.net_params, + "ab=90 abcd=20 cw=50 x-yz=-99", NULL, 0, 0); + smartlist_split_string(vote2.net_params, + "ab=27 cw=5 x-yz=88", NULL, 0, 0); + smartlist_split_string(vote3.net_params, + "abcd=20 c=60 cw=500 x-yz=-9 zzzzz=101", NULL, 0, 0); + smartlist_split_string(vote4.net_params, + "ab=900 abcd=200 c=1 cw=51 x-yz=100", NULL, 0, 0); + test_eq(100, networkstatus_get_param(&vote4, "x-yz", 50, 0, 300)); + test_eq(222, networkstatus_get_param(&vote4, "foobar", 222, 0, 300)); + test_eq(80, networkstatus_get_param(&vote4, "ab", 12, 0, 80)); + test_eq(-8, networkstatus_get_param(&vote4, "ab", -12, -100, -8)); + test_eq(0, networkstatus_get_param(&vote4, "foobar", 0, -100, 8)); + + smartlist_add(votes, &vote1); + smartlist_add(votes, &vote2); + smartlist_add(votes, &vote3); + smartlist_add(votes, &vote4); + + res = dirvote_compute_params(votes); + test_streq(res, + "ab=90 abcd=20 c=1 cw=50 x-yz=-9 zzzzz=101"); + + done: + tor_free(res); + SMARTLIST_FOREACH(vote1.net_params, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(vote2.net_params, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(vote3.net_params, char *, cp, tor_free(cp)); + SMARTLIST_FOREACH(vote4.net_params, char *, cp, tor_free(cp)); + smartlist_free(vote1.net_params); + smartlist_free(vote2.net_params); + smartlist_free(vote3.net_params); + smartlist_free(vote4.net_params); + smartlist_free(votes); + + return; +} + +extern const char AUTHORITY_CERT_1[]; +extern const char AUTHORITY_SIGNKEY_1[]; +extern const char AUTHORITY_CERT_2[]; +extern const char AUTHORITY_SIGNKEY_2[]; +extern const char AUTHORITY_CERT_3[]; +extern const char AUTHORITY_SIGNKEY_3[]; + +/** Helper: Test that two networkstatus_voter_info_t do in fact represent the + * same voting authority, and that they do in fact have all the same + * information. */ +static void +test_same_voter(networkstatus_voter_info_t *v1, + networkstatus_voter_info_t *v2) +{ + test_streq(v1->nickname, v2->nickname); + test_memeq(v1->identity_digest, v2->identity_digest, DIGEST_LEN); + test_streq(v1->address, v2->address); + test_eq(v1->addr, v2->addr); + test_eq(v1->dir_port, v2->dir_port); + test_eq(v1->or_port, v2->or_port); + test_streq(v1->contact, v2->contact); + test_memeq(v1->vote_digest, v2->vote_digest, DIGEST_LEN); + done: + ; +} + +/** Helper: Make a new routerinfo containing the right information for a + * given vote_routerstatus_t. */ +static routerinfo_t * +generate_ri_from_rs(const vote_routerstatus_t *vrs) +{ + routerinfo_t *r; + const routerstatus_t *rs = &vrs->status; + static time_t published = 0; + + r = tor_malloc_zero(sizeof(routerinfo_t)); + memcpy(r->cache_info.identity_digest, rs->identity_digest, DIGEST_LEN); + memcpy(r->cache_info.signed_descriptor_digest, rs->descriptor_digest, + DIGEST_LEN); + r->cache_info.do_not_cache = 1; + r->cache_info.routerlist_index = -1; + r->cache_info.signed_descriptor_body = + tor_strdup("123456789012345678901234567890123"); + r->cache_info.signed_descriptor_len = + strlen(r->cache_info.signed_descriptor_body); + r->exit_policy = smartlist_create(); + r->cache_info.published_on = ++published + time(NULL); + return r; +} + +/** Helper: get a detached signatures document for one or two + * consensuses. */ +static char * +get_detached_sigs(networkstatus_t *ns, networkstatus_t *ns2) +{ + char *r; + smartlist_t *sl; + tor_assert(ns && ns->flavor == FLAV_NS); + sl = smartlist_create(); + smartlist_add(sl,ns); + if (ns2) + smartlist_add(sl,ns2); + r = networkstatus_get_detached_signatures(sl); + smartlist_free(sl); + return r; +} + +/** Run unit tests for generating and parsing V3 consensus networkstatus + * documents. */ +static void +test_dir_v3_networkstatus(void) +{ + authority_cert_t *cert1=NULL, *cert2=NULL, *cert3=NULL; + crypto_pk_env_t *sign_skey_1=NULL, *sign_skey_2=NULL, *sign_skey_3=NULL; + crypto_pk_env_t *sign_skey_leg1=NULL; + const char *msg=NULL; + + time_t now = time(NULL); + networkstatus_voter_info_t *voter; + document_signature_t *sig; + networkstatus_t *vote=NULL, *v1=NULL, *v2=NULL, *v3=NULL, *con=NULL, + *con_md=NULL; + vote_routerstatus_t *vrs; + routerstatus_t *rs; + char *v1_text=NULL, *v2_text=NULL, *v3_text=NULL, *consensus_text=NULL, *cp; + smartlist_t *votes = smartlist_create(); + + /* For generating the two other consensuses. */ + char *detached_text1=NULL, *detached_text2=NULL; + char *consensus_text2=NULL, *consensus_text3=NULL; + char *consensus_text_md2=NULL, *consensus_text_md3=NULL; + char *consensus_text_md=NULL; + networkstatus_t *con2=NULL, *con_md2=NULL, *con3=NULL, *con_md3=NULL; + ns_detached_signatures_t *dsig1=NULL, *dsig2=NULL; + + /* Parse certificates and keys. */ + cert1 = authority_cert_parse_from_string(AUTHORITY_CERT_1, NULL); + test_assert(cert1); + test_assert(cert1->is_cross_certified); + cert2 = authority_cert_parse_from_string(AUTHORITY_CERT_2, NULL); + test_assert(cert2); + cert3 = authority_cert_parse_from_string(AUTHORITY_CERT_3, NULL); + test_assert(cert3); + sign_skey_1 = crypto_new_pk_env(); + sign_skey_2 = crypto_new_pk_env(); + sign_skey_3 = crypto_new_pk_env(); + sign_skey_leg1 = pk_generate(4); + + test_assert(!crypto_pk_read_private_key_from_string(sign_skey_1, + AUTHORITY_SIGNKEY_1, -1)); + test_assert(!crypto_pk_read_private_key_from_string(sign_skey_2, + AUTHORITY_SIGNKEY_2, -1)); + test_assert(!crypto_pk_read_private_key_from_string(sign_skey_3, + AUTHORITY_SIGNKEY_3, -1)); + + test_assert(!crypto_pk_cmp_keys(sign_skey_1, cert1->signing_key)); + test_assert(!crypto_pk_cmp_keys(sign_skey_2, cert2->signing_key)); + + /* + * Set up a vote; generate it; try to parse it. + */ + vote = tor_malloc_zero(sizeof(networkstatus_t)); + vote->type = NS_TYPE_VOTE; + vote->published = now; + vote->valid_after = now+1000; + vote->fresh_until = now+2000; + vote->valid_until = now+3000; + vote->vote_seconds = 100; + vote->dist_seconds = 200; + vote->supported_methods = smartlist_create(); + smartlist_split_string(vote->supported_methods, "1 2 3", NULL, 0, -1); + vote->client_versions = tor_strdup("0.1.2.14,0.1.2.15"); + vote->server_versions = tor_strdup("0.1.2.14,0.1.2.15,0.1.2.16"); + vote->known_flags = smartlist_create(); + smartlist_split_string(vote->known_flags, + "Authority Exit Fast Guard Running Stable V2Dir Valid", + 0, SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + vote->voters = smartlist_create(); + voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); + voter->nickname = tor_strdup("Voter1"); + voter->address = tor_strdup("1.2.3.4"); + voter->addr = 0x01020304; + voter->dir_port = 80; + voter->or_port = 9000; + voter->contact = tor_strdup("voter@example.com"); + crypto_pk_get_digest(cert1->identity_key, voter->identity_digest); + smartlist_add(vote->voters, voter); + vote->cert = authority_cert_dup(cert1); + vote->net_params = smartlist_create(); + smartlist_split_string(vote->net_params, "circuitwindow=101 foo=990", + NULL, 0, 0); + vote->routerstatus_list = smartlist_create(); + /* add the first routerstatus. */ + vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); + rs = &vrs->status; + vrs->version = tor_strdup("0.1.2.14"); + rs->published_on = now-1500; + strlcpy(rs->nickname, "router2", sizeof(rs->nickname)); + memset(rs->identity_digest, 3, DIGEST_LEN); + memset(rs->descriptor_digest, 78, DIGEST_LEN); + rs->addr = 0x99008801; + rs->or_port = 443; + rs->dir_port = 8000; + /* all flags but running cleared */ + rs->is_running = 1; + smartlist_add(vote->routerstatus_list, vrs); + test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); + + /* add the second routerstatus. */ + vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); + rs = &vrs->status; + vrs->version = tor_strdup("0.2.0.5"); + rs->published_on = now-1000; + strlcpy(rs->nickname, "router1", sizeof(rs->nickname)); + memset(rs->identity_digest, 5, DIGEST_LEN); + memset(rs->descriptor_digest, 77, DIGEST_LEN); + rs->addr = 0x99009901; + rs->or_port = 443; + rs->dir_port = 0; + rs->is_exit = rs->is_stable = rs->is_fast = rs->is_running = + rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; + smartlist_add(vote->routerstatus_list, vrs); + test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); + + /* add the third routerstatus. */ + vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); + rs = &vrs->status; + vrs->version = tor_strdup("0.1.0.3"); + rs->published_on = now-1000; + strlcpy(rs->nickname, "router3", sizeof(rs->nickname)); + memset(rs->identity_digest, 33, DIGEST_LEN); + memset(rs->descriptor_digest, 79, DIGEST_LEN); + rs->addr = 0xAA009901; + rs->or_port = 400; + rs->dir_port = 9999; + rs->is_authority = rs->is_exit = rs->is_stable = rs->is_fast = + rs->is_running = rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; + smartlist_add(vote->routerstatus_list, vrs); + test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); + + /* add a fourth routerstatus that is not running. */ + vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); + rs = &vrs->status; + vrs->version = tor_strdup("0.1.6.3"); + rs->published_on = now-1000; + strlcpy(rs->nickname, "router4", sizeof(rs->nickname)); + memset(rs->identity_digest, 34, DIGEST_LEN); + memset(rs->descriptor_digest, 47, DIGEST_LEN); + rs->addr = 0xC0000203; + rs->or_port = 500; + rs->dir_port = 1999; + /* Running flag (and others) cleared */ + smartlist_add(vote->routerstatus_list, vrs); + test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); + + /* dump the vote and try to parse it. */ + v1_text = format_networkstatus_vote(sign_skey_1, vote); + test_assert(v1_text); + v1 = networkstatus_parse_vote_from_string(v1_text, NULL, NS_TYPE_VOTE); + test_assert(v1); + + /* Make sure the parsed thing was right. */ + test_eq(v1->type, NS_TYPE_VOTE); + test_eq(v1->published, vote->published); + test_eq(v1->valid_after, vote->valid_after); + test_eq(v1->fresh_until, vote->fresh_until); + test_eq(v1->valid_until, vote->valid_until); + test_eq(v1->vote_seconds, vote->vote_seconds); + test_eq(v1->dist_seconds, vote->dist_seconds); + test_streq(v1->client_versions, vote->client_versions); + test_streq(v1->server_versions, vote->server_versions); + test_assert(v1->voters && smartlist_len(v1->voters)); + voter = smartlist_get(v1->voters, 0); + test_streq(voter->nickname, "Voter1"); + test_streq(voter->address, "1.2.3.4"); + test_eq(voter->addr, 0x01020304); + test_eq(voter->dir_port, 80); + test_eq(voter->or_port, 9000); + test_streq(voter->contact, "voter@example.com"); + test_assert(v1->cert); + test_assert(!crypto_pk_cmp_keys(sign_skey_1, v1->cert->signing_key)); + cp = smartlist_join_strings(v1->known_flags, ":", 0, NULL); + test_streq(cp, "Authority:Exit:Fast:Guard:Running:Stable:V2Dir:Valid"); + tor_free(cp); + test_eq(smartlist_len(v1->routerstatus_list), 4); + /* Check the first routerstatus. */ + vrs = smartlist_get(v1->routerstatus_list, 0); + rs = &vrs->status; + test_streq(vrs->version, "0.1.2.14"); + test_eq(rs->published_on, now-1500); + test_streq(rs->nickname, "router2"); + test_memeq(rs->identity_digest, + "\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3", + DIGEST_LEN); + test_memeq(rs->descriptor_digest, "NNNNNNNNNNNNNNNNNNNN", DIGEST_LEN); + test_eq(rs->addr, 0x99008801); + test_eq(rs->or_port, 443); + test_eq(rs->dir_port, 8000); + test_eq(vrs->flags, U64_LITERAL(16)); // no flags except "running" + /* Check the second routerstatus. */ + vrs = smartlist_get(v1->routerstatus_list, 1); + rs = &vrs->status; + test_streq(vrs->version, "0.2.0.5"); + test_eq(rs->published_on, now-1000); + test_streq(rs->nickname, "router1"); + test_memeq(rs->identity_digest, + "\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5", + DIGEST_LEN); + test_memeq(rs->descriptor_digest, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); + test_eq(rs->addr, 0x99009901); + test_eq(rs->or_port, 443); + test_eq(rs->dir_port, 0); + test_eq(vrs->flags, U64_LITERAL(254)); // all flags except "authority." + + { + measured_bw_line_t mbw; + memset(mbw.node_id, 33, sizeof(mbw.node_id)); + mbw.bw = 1024; + test_assert(measured_bw_line_apply(&mbw, + v1->routerstatus_list) == 1); + vrs = smartlist_get(v1->routerstatus_list, 2); + test_assert(vrs->status.has_measured_bw && + vrs->status.measured_bw == 1024); + } + + /* Generate second vote. It disagrees on some of the times, + * and doesn't list versions, and knows some crazy flags */ + vote->published = now+1; + vote->fresh_until = now+3005; + vote->dist_seconds = 300; + authority_cert_free(vote->cert); + vote->cert = authority_cert_dup(cert2); + vote->net_params = smartlist_create(); + smartlist_split_string(vote->net_params, "bar=2000000000 circuitwindow=20", + NULL, 0, 0); + tor_free(vote->client_versions); + tor_free(vote->server_versions); + voter = smartlist_get(vote->voters, 0); + tor_free(voter->nickname); + tor_free(voter->address); + voter->nickname = tor_strdup("Voter2"); + voter->address = tor_strdup("2.3.4.5"); + voter->addr = 0x02030405; + crypto_pk_get_digest(cert2->identity_key, voter->identity_digest); + smartlist_add(vote->known_flags, tor_strdup("MadeOfCheese")); + smartlist_add(vote->known_flags, tor_strdup("MadeOfTin")); + smartlist_sort_strings(vote->known_flags); + vrs = smartlist_get(vote->routerstatus_list, 2); + smartlist_del_keeporder(vote->routerstatus_list, 2); + tor_free(vrs->version); + tor_free(vrs); + vrs = smartlist_get(vote->routerstatus_list, 0); + vrs->status.is_fast = 1; + /* generate and parse. */ + v2_text = format_networkstatus_vote(sign_skey_2, vote); + test_assert(v2_text); + v2 = networkstatus_parse_vote_from_string(v2_text, NULL, NS_TYPE_VOTE); + test_assert(v2); + /* Check that flags come out right.*/ + cp = smartlist_join_strings(v2->known_flags, ":", 0, NULL); + test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:" + "Running:Stable:V2Dir:Valid"); + tor_free(cp); + vrs = smartlist_get(v2->routerstatus_list, 1); + /* 1023 - authority(1) - madeofcheese(16) - madeoftin(32) */ + test_eq(vrs->flags, U64_LITERAL(974)); + + /* Generate the third vote. */ + vote->published = now; + vote->fresh_until = now+2003; + vote->dist_seconds = 250; + authority_cert_free(vote->cert); + vote->cert = authority_cert_dup(cert3); + vote->net_params = smartlist_create(); + smartlist_split_string(vote->net_params, "circuitwindow=80 foo=660", + NULL, 0, 0); + smartlist_add(vote->supported_methods, tor_strdup("4")); + vote->client_versions = tor_strdup("0.1.2.14,0.1.2.17"); + vote->server_versions = tor_strdup("0.1.2.10,0.1.2.15,0.1.2.16"); + voter = smartlist_get(vote->voters, 0); + tor_free(voter->nickname); + tor_free(voter->address); + voter->nickname = tor_strdup("Voter3"); + voter->address = tor_strdup("3.4.5.6"); + voter->addr = 0x03040506; + crypto_pk_get_digest(cert3->identity_key, voter->identity_digest); + /* This one has a legacy id. */ + memset(voter->legacy_id_digest, (int)'A', DIGEST_LEN); + vrs = smartlist_get(vote->routerstatus_list, 0); + smartlist_del_keeporder(vote->routerstatus_list, 0); + tor_free(vrs->version); + tor_free(vrs); + vrs = smartlist_get(vote->routerstatus_list, 0); + memset(vrs->status.descriptor_digest, (int)'Z', DIGEST_LEN); + test_assert(router_add_to_routerlist(generate_ri_from_rs(vrs), &msg,0,0)>=0); + + v3_text = format_networkstatus_vote(sign_skey_3, vote); + test_assert(v3_text); + + v3 = networkstatus_parse_vote_from_string(v3_text, NULL, NS_TYPE_VOTE); + test_assert(v3); + + /* Compute a consensus as voter 3. */ + smartlist_add(votes, v3); + smartlist_add(votes, v1); + smartlist_add(votes, v2); + consensus_text = networkstatus_compute_consensus(votes, 3, + cert3->identity_key, + sign_skey_3, + "AAAAAAAAAAAAAAAAAAAA", + sign_skey_leg1, + FLAV_NS); + test_assert(consensus_text); + con = networkstatus_parse_vote_from_string(consensus_text, NULL, + NS_TYPE_CONSENSUS); + test_assert(con); + //log_notice(LD_GENERAL, "<<%s>>\n<<%s>>\n<<%s>>\n", + // v1_text, v2_text, v3_text); + consensus_text_md = networkstatus_compute_consensus(votes, 3, + cert3->identity_key, + sign_skey_3, + "AAAAAAAAAAAAAAAAAAAA", + sign_skey_leg1, + FLAV_MICRODESC); + test_assert(consensus_text_md); + con_md = networkstatus_parse_vote_from_string(consensus_text_md, NULL, + NS_TYPE_CONSENSUS); + test_assert(con_md); + test_eq(con_md->flavor, FLAV_MICRODESC); + + /* Check consensus contents. */ + test_assert(con->type == NS_TYPE_CONSENSUS); + test_eq(con->published, 0); /* this field only appears in votes. */ + test_eq(con->valid_after, now+1000); + test_eq(con->fresh_until, now+2003); /* median */ + test_eq(con->valid_until, now+3000); + test_eq(con->vote_seconds, 100); + test_eq(con->dist_seconds, 250); /* median */ + test_streq(con->client_versions, "0.1.2.14"); + test_streq(con->server_versions, "0.1.2.15,0.1.2.16"); + cp = smartlist_join_strings(v2->known_flags, ":", 0, NULL); + test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:" + "Running:Stable:V2Dir:Valid"); + tor_free(cp); + cp = smartlist_join_strings(con->net_params, ":", 0, NULL); + test_streq(cp, "bar=2000000000:circuitwindow=80:foo=660"); + tor_free(cp); + + test_eq(4, smartlist_len(con->voters)); /*3 voters, 1 legacy key.*/ + /* The voter id digests should be in this order. */ + test_assert(memcmp(cert2->cache_info.identity_digest, + cert1->cache_info.identity_digest,DIGEST_LEN)<0); + test_assert(memcmp(cert1->cache_info.identity_digest, + cert3->cache_info.identity_digest,DIGEST_LEN)<0); + test_same_voter(smartlist_get(con->voters, 1), + smartlist_get(v2->voters, 0)); + test_same_voter(smartlist_get(con->voters, 2), + smartlist_get(v1->voters, 0)); + test_same_voter(smartlist_get(con->voters, 3), + smartlist_get(v3->voters, 0)); + + test_assert(!con->cert); + test_eq(2, smartlist_len(con->routerstatus_list)); + /* There should be two listed routers: one with identity 3, one with + * identity 5. */ + /* This one showed up in 2 digests. */ + rs = smartlist_get(con->routerstatus_list, 0); + test_memeq(rs->identity_digest, + "\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3", + DIGEST_LEN); + test_memeq(rs->descriptor_digest, "NNNNNNNNNNNNNNNNNNNN", DIGEST_LEN); + test_assert(!rs->is_authority); + test_assert(!rs->is_exit); + test_assert(!rs->is_fast); + test_assert(!rs->is_possible_guard); + test_assert(!rs->is_stable); + test_assert(rs->is_running); /* If it wasn't running it wouldn't be here */ + test_assert(!rs->is_v2_dir); + test_assert(!rs->is_valid); + test_assert(!rs->is_named); + /* XXXX check version */ + + rs = smartlist_get(con->routerstatus_list, 1); + /* This one showed up in 3 digests. Twice with ID 'M', once with 'Z'. */ + test_memeq(rs->identity_digest, + "\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5", + DIGEST_LEN); + test_streq(rs->nickname, "router1"); + test_memeq(rs->descriptor_digest, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); + test_eq(rs->published_on, now-1000); + test_eq(rs->addr, 0x99009901); + test_eq(rs->or_port, 443); + test_eq(rs->dir_port, 0); + test_assert(!rs->is_authority); + test_assert(rs->is_exit); + test_assert(rs->is_fast); + test_assert(rs->is_possible_guard); + test_assert(rs->is_stable); + test_assert(rs->is_running); + test_assert(rs->is_v2_dir); + test_assert(rs->is_valid); + test_assert(!rs->is_named); + /* XXXX check version */ + + /* Check signatures. the first voter is a pseudo-entry with a legacy key. + * The second one hasn't signed. The fourth one has signed: validate it. */ + voter = smartlist_get(con->voters, 1); + test_eq(smartlist_len(voter->sigs), 0); + + voter = smartlist_get(con->voters, 3); + test_eq(smartlist_len(voter->sigs), 1); + sig = smartlist_get(voter->sigs, 0); + test_assert(sig->signature); + test_assert(!sig->good_signature); + test_assert(!sig->bad_signature); + + test_assert(!networkstatus_check_document_signature(con, sig, cert3)); + test_assert(sig->signature); + test_assert(sig->good_signature); + test_assert(!sig->bad_signature); + + { + const char *msg=NULL; + /* Compute the other two signed consensuses. */ + smartlist_shuffle(votes); + consensus_text2 = networkstatus_compute_consensus(votes, 3, + cert2->identity_key, + sign_skey_2, NULL,NULL, + FLAV_NS); + consensus_text_md2 = networkstatus_compute_consensus(votes, 3, + cert2->identity_key, + sign_skey_2, NULL,NULL, + FLAV_MICRODESC); + smartlist_shuffle(votes); + consensus_text3 = networkstatus_compute_consensus(votes, 3, + cert1->identity_key, + sign_skey_1, NULL,NULL, + FLAV_NS); + consensus_text_md3 = networkstatus_compute_consensus(votes, 3, + cert1->identity_key, + sign_skey_1, NULL,NULL, + FLAV_MICRODESC); + test_assert(consensus_text2); + test_assert(consensus_text3); + test_assert(consensus_text_md2); + test_assert(consensus_text_md3); + con2 = networkstatus_parse_vote_from_string(consensus_text2, NULL, + NS_TYPE_CONSENSUS); + con3 = networkstatus_parse_vote_from_string(consensus_text3, NULL, + NS_TYPE_CONSENSUS); + con_md2 = networkstatus_parse_vote_from_string(consensus_text_md2, NULL, + NS_TYPE_CONSENSUS); + con_md3 = networkstatus_parse_vote_from_string(consensus_text_md3, NULL, + NS_TYPE_CONSENSUS); + test_assert(con2); + test_assert(con3); + test_assert(con_md2); + test_assert(con_md3); + + /* All three should have the same digest. */ + test_memeq(&con->digests, &con2->digests, sizeof(digests_t)); + test_memeq(&con->digests, &con3->digests, sizeof(digests_t)); + + test_memeq(&con_md->digests, &con_md2->digests, sizeof(digests_t)); + test_memeq(&con_md->digests, &con_md3->digests, sizeof(digests_t)); + + /* Extract a detached signature from con3. */ + detached_text1 = get_detached_sigs(con3, con_md3); + tor_assert(detached_text1); + /* Try to parse it. */ + dsig1 = networkstatus_parse_detached_signatures(detached_text1, NULL); + tor_assert(dsig1); + + /* Are parsed values as expected? */ + test_eq(dsig1->valid_after, con3->valid_after); + test_eq(dsig1->fresh_until, con3->fresh_until); + test_eq(dsig1->valid_until, con3->valid_until); + { + digests_t *dsig_digests = strmap_get(dsig1->digests, "ns"); + test_assert(dsig_digests); + test_memeq(dsig_digests->d[DIGEST_SHA1], con3->digests.d[DIGEST_SHA1], + DIGEST_LEN); + dsig_digests = strmap_get(dsig1->digests, "microdesc"); + test_assert(dsig_digests); + test_memeq(dsig_digests->d[DIGEST_SHA256], + con_md3->digests.d[DIGEST_SHA256], + DIGEST256_LEN); + } + { + smartlist_t *dsig_signatures = strmap_get(dsig1->signatures, "ns"); + test_assert(dsig_signatures); + test_eq(1, smartlist_len(dsig_signatures)); + sig = smartlist_get(dsig_signatures, 0); + test_memeq(sig->identity_digest, cert1->cache_info.identity_digest, + DIGEST_LEN); + test_eq(sig->alg, DIGEST_SHA1); + + dsig_signatures = strmap_get(dsig1->signatures, "microdesc"); + test_assert(dsig_signatures); + test_eq(1, smartlist_len(dsig_signatures)); + sig = smartlist_get(dsig_signatures, 0); + test_memeq(sig->identity_digest, cert1->cache_info.identity_digest, + DIGEST_LEN); + test_eq(sig->alg, DIGEST_SHA256); + } + + /* Try adding it to con2. */ + detached_text2 = get_detached_sigs(con2,con_md2); + test_eq(1, networkstatus_add_detached_signatures(con2, dsig1, &msg)); + tor_free(detached_text2); + test_eq(1, networkstatus_add_detached_signatures(con_md2, dsig1, &msg)); + tor_free(detached_text2); + detached_text2 = get_detached_sigs(con2,con_md2); + //printf("\n<%s>\n", detached_text2); + dsig2 = networkstatus_parse_detached_signatures(detached_text2, NULL); + test_assert(dsig2); + /* + printf("\n"); + SMARTLIST_FOREACH(dsig2->signatures, networkstatus_voter_info_t *, vi, { + char hd[64]; + base16_encode(hd, sizeof(hd), vi->identity_digest, DIGEST_LEN); + printf("%s\n", hd); + }); + */ + test_eq(2, + smartlist_len((smartlist_t*)strmap_get(dsig2->signatures, "ns"))); + test_eq(2, + smartlist_len((smartlist_t*)strmap_get(dsig2->signatures, + "microdesc"))); + + /* Try adding to con2 twice; verify that nothing changes. */ + test_eq(0, networkstatus_add_detached_signatures(con2, dsig1, &msg)); + + /* Add to con. */ + test_eq(2, networkstatus_add_detached_signatures(con, dsig2, &msg)); + /* Check signatures */ + voter = smartlist_get(con->voters, 1); + sig = smartlist_get(voter->sigs, 0); + test_assert(sig); + test_assert(!networkstatus_check_document_signature(con, sig, cert2)); + voter = smartlist_get(con->voters, 2); + sig = smartlist_get(voter->sigs, 0); + test_assert(sig); + test_assert(!networkstatus_check_document_signature(con, sig, cert1)); + } + + done: + smartlist_free(votes); + tor_free(v1_text); + tor_free(v2_text); + tor_free(v3_text); + tor_free(consensus_text); + tor_free(consensus_text_md); + + if (vote) + networkstatus_vote_free(vote); + if (v1) + networkstatus_vote_free(v1); + if (v2) + networkstatus_vote_free(v2); + if (v3) + networkstatus_vote_free(v3); + if (con) + networkstatus_vote_free(con); + if (con_md) + networkstatus_vote_free(con_md); + if (sign_skey_1) + crypto_free_pk_env(sign_skey_1); + if (sign_skey_2) + crypto_free_pk_env(sign_skey_2); + if (sign_skey_3) + crypto_free_pk_env(sign_skey_3); + if (sign_skey_leg1) + crypto_free_pk_env(sign_skey_leg1); + if (cert1) + authority_cert_free(cert1); + if (cert2) + authority_cert_free(cert2); + if (cert3) + authority_cert_free(cert3); + + tor_free(consensus_text2); + tor_free(consensus_text3); + tor_free(consensus_text_md2); + tor_free(consensus_text_md3); + tor_free(detached_text1); + tor_free(detached_text2); + if (con2) + networkstatus_vote_free(con2); + if (con3) + networkstatus_vote_free(con3); + if (con_md2) + networkstatus_vote_free(con_md2); + if (con_md3) + networkstatus_vote_free(con_md3); + if (dsig1) + ns_detached_signatures_free(dsig1); + if (dsig2) + ns_detached_signatures_free(dsig2); +} + +#define DIR_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_dir_ ## name } + +#define DIR(name) \ + { #name, test_dir_##name, 0, NULL, NULL } + +struct testcase_t dir_tests[] = { + DIR_LEGACY(nicknames), + DIR_LEGACY(formats), + DIR_LEGACY(versions), + DIR_LEGACY(fp_pairs), + DIR(split_fps), + DIR_LEGACY(measured_bw), + DIR_LEGACY(param_voting), + DIR_LEGACY(v3_networkstatus), + END_OF_TESTCASES +}; + diff --git a/src/test/test_util.c b/src/test/test_util.c new file mode 100644 index 000000000..b1fafc84b --- /dev/null +++ b/src/test/test_util.c @@ -0,0 +1,1240 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2011, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#define CONTROL_PRIVATE +#define MEMPOOL_PRIVATE +#include "or.h" +#include "config.h" +#include "control.h" +#include "test.h" +#include "mempool.h" +#include "memarea.h" + +static void +test_util_time(void) +{ + struct timeval start, end; + struct tm a_time; + char timestr[RFC1123_TIME_LEN+1]; + time_t t_res; + int i; + + start.tv_sec = 5; + start.tv_usec = 5000; + + end.tv_sec = 5; + end.tv_usec = 5000; + + test_eq(0L, tv_udiff(&start, &end)); + + end.tv_usec = 7000; + + test_eq(2000L, tv_udiff(&start, &end)); + + end.tv_sec = 6; + + test_eq(1002000L, tv_udiff(&start, &end)); + + end.tv_usec = 0; + + test_eq(995000L, tv_udiff(&start, &end)); + + end.tv_sec = 4; + + test_eq(-1005000L, tv_udiff(&start, &end)); + + end.tv_usec = 999990; + start.tv_sec = 1; + start.tv_usec = 500; + + /* The test values here are confirmed to be correct on a platform + * with a working timegm. */ + a_time.tm_year = 2003-1900; + a_time.tm_mon = 7; + a_time.tm_mday = 30; + a_time.tm_hour = 6; + a_time.tm_min = 14; + a_time.tm_sec = 55; + test_eq((time_t) 1062224095UL, tor_timegm(&a_time)); + a_time.tm_year = 2004-1900; /* Try a leap year, after feb. */ + test_eq((time_t) 1093846495UL, tor_timegm(&a_time)); + a_time.tm_mon = 1; /* Try a leap year, in feb. */ + a_time.tm_mday = 10; + test_eq((time_t) 1076393695UL, tor_timegm(&a_time)); + + format_rfc1123_time(timestr, 0); + test_streq("Thu, 01 Jan 1970 00:00:00 GMT", timestr); + format_rfc1123_time(timestr, (time_t)1091580502UL); + test_streq("Wed, 04 Aug 2004 00:48:22 GMT", timestr); + + t_res = 0; + i = parse_rfc1123_time(timestr, &t_res); + test_eq(i,0); + test_eq(t_res, (time_t)1091580502UL); + test_eq(-1, parse_rfc1123_time("Wed, zz Aug 2004 99-99x99 GMT", &t_res)); + + tor_gettimeofday(&start); + /* now make sure time works. */ + tor_gettimeofday(&end); + /* We might've timewarped a little. */ + tt_int_op(tv_udiff(&start, &end), >=, -5000); + + done: + ; +} + +static void +test_util_config_line(void) +{ + char buf[1024]; + char *k=NULL, *v=NULL; + const char *str; + + /* Test parse_config_line_from_str */ + strlcpy(buf, "k v\n" " key value with spaces \n" "keykey val\n" + "k2\n" + "k3 \n" "\n" " \n" "#comment\n" + "k4#a\n" "k5#abc\n" "k6 val #with comment\n" + "kseven \"a quoted 'string\"\n" + "k8 \"a \\x71uoted\\n\\\"str\\\\ing\\t\\001\\01\\1\\\"\"\n" + "k9 a line that\\\n spans two lines.\n\n" + "k10 more than\\\n one contin\\\nuation\n" + "k11 \\\ncontinuation at the start\n" + "k12 line with a\\\n#comment\n embedded\n" + "k13\\\ncontinuation at the very start\n" + "k14 a line that has a comment and # ends with a slash \\\n" + "k15 this should be the next new line\n" + "k16 a line that has a comment and # ends without a slash \n" + "k17 this should be the next new line\n" + , sizeof(buf)); + str = buf; + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k"); + test_streq(v, "v"); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "key value with")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "key"); + test_streq(v, "value with spaces"); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "keykey")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "keykey"); + test_streq(v, "val"); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "k2\n")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k2"); + test_streq(v, ""); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "k3 \n")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k3"); + test_streq(v, ""); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "#comment")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k4"); + test_streq(v, ""); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "k5#abc")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k5"); + test_streq(v, ""); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "k6")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k6"); + test_streq(v, "val"); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "kseven")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "kseven"); + test_streq(v, "a quoted \'string"); + tor_free(k); tor_free(v); + test_assert(!strcmpstart(str, "k8 ")); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k8"); + test_streq(v, "a quoted\n\"str\\ing\t\x01\x01\x01\""); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k9"); + test_streq(v, "a line that spans two lines."); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k10"); + test_streq(v, "more than one continuation"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k11"); + test_streq(v, "continuation at the start"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k12"); + test_streq(v, "line with a embedded"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k13"); + test_streq(v, "continuation at the very start"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k14"); + test_streq(v, "a line that has a comment and" ); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k15"); + test_streq(v, "this should be the next new line"); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k16"); + test_streq(v, "a line that has a comment and" ); + tor_free(k); tor_free(v); + + str = parse_config_line_from_str(str, &k, &v); + test_streq(k, "k17"); + test_streq(v, "this should be the next new line"); + tor_free(k); tor_free(v); + + test_streq(str, ""); + + done: + tor_free(k); + tor_free(v); +} + +/** Test basic string functionality. */ +static void +test_util_strmisc(void) +{ + char buf[1024]; + int i; + char *cp; + + /* Tests for corner cases of strl operations */ + test_eq(5, strlcpy(buf, "Hello", 0)); + strlcpy(buf, "Hello", sizeof(buf)); + test_eq(10, strlcat(buf, "Hello", 5)); + + /* Test tor_strstrip() */ + strlcpy(buf, "Testing 1 2 3", sizeof(buf)); + tor_strstrip(buf, ",!"); + test_streq(buf, "Testing 1 2 3"); + strlcpy(buf, "!Testing 1 2 3?", sizeof(buf)); + tor_strstrip(buf, "!? "); + test_streq(buf, "Testing123"); + + /* Test tor_parse_long. */ + test_eq(10L, tor_parse_long("10",10,0,100,NULL,NULL)); + test_eq(0L, tor_parse_long("10",10,50,100,NULL,NULL)); + test_eq(-50L, tor_parse_long("-50",10,-100,100,NULL,NULL)); + + /* Test tor_parse_ulong */ + test_eq(10UL, tor_parse_ulong("10",10,0,100,NULL,NULL)); + test_eq(0UL, tor_parse_ulong("10",10,50,100,NULL,NULL)); + + /* Test tor_parse_uint64. */ + test_assert(U64_LITERAL(10) == tor_parse_uint64("10 x",10,0,100, &i, &cp)); + test_assert(i == 1); + test_streq(cp, " x"); + test_assert(U64_LITERAL(12345678901) == + tor_parse_uint64("12345678901",10,0,UINT64_MAX, &i, &cp)); + test_assert(i == 1); + test_streq(cp, ""); + test_assert(U64_LITERAL(0) == + tor_parse_uint64("12345678901",10,500,INT32_MAX, &i, &cp)); + test_assert(i == 0); + + { + /* Test tor_parse_double. */ + double d = tor_parse_double("10", 0, UINT64_MAX,&i,NULL); + test_assert(i == 1); + test_assert(DBL_TO_U64(d) == 10); + d = tor_parse_double("0", 0, UINT64_MAX,&i,NULL); + test_assert(i == 1); + test_assert(DBL_TO_U64(d) == 0); + d = tor_parse_double(" ", 0, UINT64_MAX,&i,NULL); + test_assert(i == 0); + d = tor_parse_double(".0a", 0, UINT64_MAX,&i,NULL); + test_assert(i == 0); + d = tor_parse_double(".0a", 0, UINT64_MAX,&i,&cp); + test_assert(i == 1); + d = tor_parse_double("-.0", 0, UINT64_MAX,&i,NULL); + test_assert(i == 1); + } + + /* Test failing snprintf cases */ + test_eq(-1, tor_snprintf(buf, 0, "Foo")); + test_eq(-1, tor_snprintf(buf, 2, "Foo")); + + /* Test printf with uint64 */ + tor_snprintf(buf, sizeof(buf), "x!"U64_FORMAT"!x", + U64_PRINTF_ARG(U64_LITERAL(12345678901))); + test_streq(buf, "x!12345678901!x"); + + /* Test for strcmpstart and strcmpend. */ + test_assert(strcmpstart("abcdef", "abcdef")==0); + test_assert(strcmpstart("abcdef", "abc")==0); + test_assert(strcmpstart("abcdef", "abd")<0); + test_assert(strcmpstart("abcdef", "abb")>0); + test_assert(strcmpstart("ab", "abb")<0); + + test_assert(strcmpend("abcdef", "abcdef")==0); + test_assert(strcmpend("abcdef", "def")==0); + test_assert(strcmpend("abcdef", "deg")<0); + test_assert(strcmpend("abcdef", "dee")>0); + test_assert(strcmpend("ab", "abb")<0); + + test_assert(strcasecmpend("AbcDEF", "abcdef")==0); + test_assert(strcasecmpend("abcdef", "dEF")==0); + test_assert(strcasecmpend("abcDEf", "deg")<0); + test_assert(strcasecmpend("abcdef", "DEE")>0); + test_assert(strcasecmpend("ab", "abB")<0); + + /* Test mem_is_zero */ + memset(buf,0,128); + buf[128] = 'x'; + test_assert(tor_digest_is_zero(buf)); + test_assert(tor_mem_is_zero(buf, 10)); + test_assert(tor_mem_is_zero(buf, 20)); + test_assert(tor_mem_is_zero(buf, 128)); + test_assert(!tor_mem_is_zero(buf, 129)); + buf[60] = (char)255; + test_assert(!tor_mem_is_zero(buf, 128)); + buf[0] = (char)1; + test_assert(!tor_mem_is_zero(buf, 10)); + + /* Test 'escaped' */ + test_streq("\"\"", escaped("")); + test_streq("\"abcd\"", escaped("abcd")); + test_streq("\"\\\\\\n\\r\\t\\\"\\'\"", escaped("\\\n\r\t\"\'")); + test_streq("\"z\\001abc\\277d\"", escaped("z\001abc\277d")); + test_assert(NULL == escaped(NULL)); + + /* Test strndup and memdup */ + { + const char *s = "abcdefghijklmnopqrstuvwxyz"; + cp = tor_strndup(s, 30); + test_streq(cp, s); /* same string, */ + test_neq(cp, s); /* but different pointers. */ + tor_free(cp); + + cp = tor_strndup(s, 5); + test_streq(cp, "abcde"); + tor_free(cp); + + s = "a\0b\0c\0d\0e\0"; + cp = tor_memdup(s,10); + test_memeq(cp, s, 10); /* same ram, */ + test_neq(cp, s); /* but different pointers. */ + tor_free(cp); + } + + /* Test str-foo functions */ + cp = tor_strdup("abcdef"); + test_assert(tor_strisnonupper(cp)); + cp[3] = 'D'; + test_assert(!tor_strisnonupper(cp)); + tor_strupper(cp); + test_streq(cp, "ABCDEF"); + test_assert(tor_strisprint(cp)); + cp[3] = 3; + test_assert(!tor_strisprint(cp)); + tor_free(cp); + + /* Test eat_whitespace. */ + { + const char *s = " \n a"; + test_eq_ptr(eat_whitespace(s), s+4); + s = "abcd"; + test_eq_ptr(eat_whitespace(s), s); + s = "#xyz\nab"; + test_eq_ptr(eat_whitespace(s), s+5); + } + + /* Test memmem and memstr */ + { + const char *haystack = "abcde"; + tor_assert(!tor_memmem(haystack, 5, "ef", 2)); + test_eq_ptr(tor_memmem(haystack, 5, "cd", 2), haystack + 2); + test_eq_ptr(tor_memmem(haystack, 5, "cde", 3), haystack + 2); + haystack = "ababcad"; + test_eq_ptr(tor_memmem(haystack, 7, "abc", 3), haystack + 2); + test_eq_ptr(tor_memstr(haystack, 7, "abc"), haystack + 2); + test_assert(!tor_memstr(haystack, 7, "fe")); + test_assert(!tor_memstr(haystack, 7, "longerthantheoriginal")); + } + + /* Test wrap_string */ + { + smartlist_t *sl = smartlist_create(); + wrap_string(sl, "This is a test of string wrapping functionality: woot.", + 10, "", ""); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, + "This is a\ntest of\nstring\nwrapping\nfunctional\nity: woot.\n"); + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_clear(sl); + + wrap_string(sl, "This is a test of string wrapping functionality: woot.", + 16, "### ", "# "); + cp = smartlist_join_strings(sl, "", 0, NULL); + test_streq(cp, + "### This is a\n# test of string\n# wrapping\n# functionality:\n" + "# woot.\n"); + + tor_free(cp); + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_free(sl); + } + done: + ; +} + +static void +test_util_pow2(void) +{ + /* Test tor_log2(). */ + test_eq(tor_log2(64), 6); + test_eq(tor_log2(65), 6); + test_eq(tor_log2(63), 5); + test_eq(tor_log2(1), 0); + test_eq(tor_log2(2), 1); + test_eq(tor_log2(3), 1); + test_eq(tor_log2(4), 2); + test_eq(tor_log2(5), 2); + test_eq(tor_log2(U64_LITERAL(40000000000000000)), 55); + test_eq(tor_log2(UINT64_MAX), 63); + + /* Test round_to_power_of_2 */ + test_eq(round_to_power_of_2(120), 128); + test_eq(round_to_power_of_2(128), 128); + test_eq(round_to_power_of_2(130), 128); + test_eq(round_to_power_of_2(U64_LITERAL(40000000000000000)), + U64_LITERAL(1)<<55); + test_eq(round_to_power_of_2(0), 2); + + done: + ; +} + +/** mutex for thread test to stop the threads hitting data at the same time. */ +static tor_mutex_t *_thread_test_mutex = NULL; +/** mutexes for the thread test to make sure that the threads have to + * interleave somewhat. */ +static tor_mutex_t *_thread_test_start1 = NULL, + *_thread_test_start2 = NULL; +/** Shared strmap for the thread test. */ +static strmap_t *_thread_test_strmap = NULL; +/** The name of thread1 for the thread test */ +static char *_thread1_name = NULL; +/** The name of thread2 for the thread test */ +static char *_thread2_name = NULL; + +static void _thread_test_func(void* _s) ATTR_NORETURN; + +/** How many iterations have the threads in the unit test run? */ +static int t1_count = 0, t2_count = 0; + +/** Helper function for threading unit tests: This function runs in a + * subthread. It grabs its own mutex (start1 or start2) to make sure that it + * should start, then it repeatedly alters _test_thread_strmap protected by + * _thread_test_mutex. */ +static void +_thread_test_func(void* _s) +{ + char *s = _s; + int i, *count; + tor_mutex_t *m; + char buf[64]; + char **cp; + if (!strcmp(s, "thread 1")) { + m = _thread_test_start1; + cp = &_thread1_name; + count = &t1_count; + } else { + m = _thread_test_start2; + cp = &_thread2_name; + count = &t2_count; + } + + tor_snprintf(buf, sizeof(buf), "%lu", tor_get_thread_id()); + *cp = tor_strdup(buf); + + tor_mutex_acquire(m); + + for (i=0; i<10000; ++i) { + tor_mutex_acquire(_thread_test_mutex); + strmap_set(_thread_test_strmap, "last to run", *cp); + ++*count; + tor_mutex_release(_thread_test_mutex); + } + tor_mutex_acquire(_thread_test_mutex); + strmap_set(_thread_test_strmap, s, *cp); + tor_mutex_release(_thread_test_mutex); + + tor_mutex_release(m); + + spawn_exit(); +} + +/** Run unit tests for threading logic. */ +static void +test_util_threads(void) +{ + char *s1 = NULL, *s2 = NULL; + int done = 0, timedout = 0; + time_t started; +#ifndef MS_WINDOWS + struct timeval tv; + tv.tv_sec=0; + tv.tv_usec=10; +#endif +#ifndef TOR_IS_MULTITHREADED + /* Skip this test if we aren't threading. We should be threading most + * everywhere by now. */ + if (1) + return; +#endif + _thread_test_mutex = tor_mutex_new(); + _thread_test_start1 = tor_mutex_new(); + _thread_test_start2 = tor_mutex_new(); + _thread_test_strmap = strmap_new(); + s1 = tor_strdup("thread 1"); + s2 = tor_strdup("thread 2"); + tor_mutex_acquire(_thread_test_start1); + tor_mutex_acquire(_thread_test_start2); + spawn_func(_thread_test_func, s1); + spawn_func(_thread_test_func, s2); + tor_mutex_release(_thread_test_start2); + tor_mutex_release(_thread_test_start1); + started = time(NULL); + while (!done) { + tor_mutex_acquire(_thread_test_mutex); + strmap_assert_ok(_thread_test_strmap); + if (strmap_get(_thread_test_strmap, "thread 1") && + strmap_get(_thread_test_strmap, "thread 2")) { + done = 1; + } else if (time(NULL) > started + 25) { + timedout = done = 1; + } + tor_mutex_release(_thread_test_mutex); +#ifndef MS_WINDOWS + /* Prevent the main thread from starving the worker threads. */ + select(0, NULL, NULL, NULL, &tv); +#endif + } + tor_mutex_acquire(_thread_test_start1); + tor_mutex_release(_thread_test_start1); + tor_mutex_acquire(_thread_test_start2); + tor_mutex_release(_thread_test_start2); + + tor_mutex_free(_thread_test_mutex); + + if (timedout) { + printf("\nTimed out: %d %d", t1_count, t2_count); + test_assert(strmap_get(_thread_test_strmap, "thread 1")); + test_assert(strmap_get(_thread_test_strmap, "thread 2")); + test_assert(!timedout); + } + + /* different thread IDs. */ + test_assert(strcmp(strmap_get(_thread_test_strmap, "thread 1"), + strmap_get(_thread_test_strmap, "thread 2"))); + test_assert(!strcmp(strmap_get(_thread_test_strmap, "thread 1"), + strmap_get(_thread_test_strmap, "last to run")) || + !strcmp(strmap_get(_thread_test_strmap, "thread 2"), + strmap_get(_thread_test_strmap, "last to run"))); + + done: + tor_free(s1); + tor_free(s2); + tor_free(_thread1_name); + tor_free(_thread2_name); + if (_thread_test_strmap) + strmap_free(_thread_test_strmap, NULL); + if (_thread_test_start1) + tor_mutex_free(_thread_test_start1); + if (_thread_test_start2) + tor_mutex_free(_thread_test_start2); +} + +/** Run unit tests for compression functions */ +static void +test_util_gzip(void) +{ + char *buf1=NULL, *buf2=NULL, *buf3=NULL, *cp1, *cp2; + const char *ccp2; + size_t len1, len2; + tor_zlib_state_t *state = NULL; + + buf1 = tor_strdup("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ"); + test_assert(detect_compression_method(buf1, strlen(buf1)) == UNKNOWN_METHOD); + if (is_gzip_supported()) { + test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, + GZIP_METHOD)); + test_assert(buf2); + test_assert(!memcmp(buf2, "\037\213", 2)); /* Gzip magic. */ + test_assert(detect_compression_method(buf2, len1) == GZIP_METHOD); + + test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1, + GZIP_METHOD, 1, LOG_INFO)); + test_assert(buf3); + test_streq(buf1,buf3); + + tor_free(buf2); + tor_free(buf3); + } + + test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, + ZLIB_METHOD)); + test_assert(buf2); + test_assert(!memcmp(buf2, "\x78\xDA", 2)); /* deflate magic. */ + test_assert(detect_compression_method(buf2, len1) == ZLIB_METHOD); + + test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1, + ZLIB_METHOD, 1, LOG_INFO)); + test_assert(buf3); + test_streq(buf1,buf3); + + /* Check whether we can uncompress concatenated, compressed strings. */ + tor_free(buf3); + buf2 = tor_realloc(buf2, len1*2); + memcpy(buf2+len1, buf2, len1); + test_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1*2, + ZLIB_METHOD, 1, LOG_INFO)); + test_eq(len2, (strlen(buf1)+1)*2); + test_memeq(buf3, + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ\0" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAAAAAAAAAAAAAAAAZ\0", + (strlen(buf1)+1)*2); + + tor_free(buf1); + tor_free(buf2); + tor_free(buf3); + + /* Check whether we can uncompress partial strings. */ + buf1 = + tor_strdup("String with low redundancy that won't be compressed much."); + test_assert(!tor_gzip_compress(&buf2, &len1, buf1, strlen(buf1)+1, + ZLIB_METHOD)); + tor_assert(len1>16); + /* when we allow an incomplete string, we should succeed.*/ + tor_assert(!tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, + ZLIB_METHOD, 0, LOG_INFO)); + buf3[len2]='\0'; + tor_assert(len2 > 5); + tor_assert(!strcmpstart(buf1, buf3)); + + /* when we demand a complete string, this must fail. */ + tor_free(buf3); + tor_assert(tor_gzip_uncompress(&buf3, &len2, buf2, len1-16, + ZLIB_METHOD, 1, LOG_INFO)); + tor_assert(!buf3); + + /* Now, try streaming compression. */ + tor_free(buf1); + tor_free(buf2); + tor_free(buf3); + state = tor_zlib_new(1, ZLIB_METHOD); + tor_assert(state); + cp1 = buf1 = tor_malloc(1024); + len1 = 1024; + ccp2 = "ABCDEFGHIJABCDEFGHIJ"; + len2 = 21; + test_assert(tor_zlib_process(state, &cp1, &len1, &ccp2, &len2, 0) + == TOR_ZLIB_OK); + test_eq(len2, 0); /* Make sure we compressed it all. */ + test_assert(cp1 > buf1); + + len2 = 0; + cp2 = cp1; + test_assert(tor_zlib_process(state, &cp1, &len1, &ccp2, &len2, 1) + == TOR_ZLIB_DONE); + test_eq(len2, 0); + test_assert(cp1 > cp2); /* Make sure we really added something. */ + + tor_assert(!tor_gzip_uncompress(&buf3, &len2, buf1, 1024-len1, + ZLIB_METHOD, 1, LOG_WARN)); + test_streq(buf3, "ABCDEFGHIJABCDEFGHIJ"); /*Make sure it compressed right.*/ + + done: + if (state) + tor_zlib_free(state); + tor_free(buf2); + tor_free(buf3); + tor_free(buf1); +} + +/** Run unit tests for mmap() wrapper functionality. */ +static void +test_util_mmap(void) +{ + char *fname1 = tor_strdup(get_fname("mapped_1")); + char *fname2 = tor_strdup(get_fname("mapped_2")); + char *fname3 = tor_strdup(get_fname("mapped_3")); + const size_t buflen = 17000; + char *buf = tor_malloc(17000); + tor_mmap_t *mapping = NULL; + + crypto_rand(buf, buflen); + + mapping = tor_mmap_file(fname1); + test_assert(! mapping); + + write_str_to_file(fname1, "Short file.", 1); + write_bytes_to_file(fname2, buf, buflen, 1); + write_bytes_to_file(fname3, buf, 16384, 1); + + mapping = tor_mmap_file(fname1); + test_assert(mapping); + test_eq(mapping->size, strlen("Short file.")); + test_streq(mapping->data, "Short file."); +#ifdef MS_WINDOWS + tor_munmap_file(mapping); + mapping = NULL; + test_assert(unlink(fname1) == 0); +#else + /* make sure we can unlink. */ + test_assert(unlink(fname1) == 0); + test_streq(mapping->data, "Short file."); + tor_munmap_file(mapping); + mapping = NULL; +#endif + + /* Now a zero-length file. */ + write_str_to_file(fname1, "", 1); + mapping = tor_mmap_file(fname1); + test_eq(mapping, NULL); + test_eq(ERANGE, errno); + unlink(fname1); + + /* Make sure that we fail to map a no-longer-existent file. */ + mapping = tor_mmap_file(fname1); + test_assert(mapping == NULL); + + /* Now try a big file that stretches across a few pages and isn't aligned */ + mapping = tor_mmap_file(fname2); + test_assert(mapping); + test_eq(mapping->size, buflen); + test_memeq(mapping->data, buf, buflen); + tor_munmap_file(mapping); + mapping = NULL; + + /* Now try a big aligned file. */ + mapping = tor_mmap_file(fname3); + test_assert(mapping); + test_eq(mapping->size, 16384); + test_memeq(mapping->data, buf, 16384); + tor_munmap_file(mapping); + mapping = NULL; + + done: + unlink(fname1); + unlink(fname2); + unlink(fname3); + + tor_free(fname1); + tor_free(fname2); + tor_free(fname3); + tor_free(buf); + + if (mapping) + tor_munmap_file(mapping); +} + +/** Run unit tests for escaping/unescaping data for use by controllers. */ +static void +test_util_control_formats(void) +{ + char *out = NULL; + const char *inp = + "..This is a test\r\nof the emergency \nbroadcast\r\n..system.\r\nZ.\r\n"; + size_t sz; + + sz = read_escaped_data(inp, strlen(inp), &out); + test_streq(out, + ".This is a test\nof the emergency \nbroadcast\n.system.\nZ.\n"); + test_eq(sz, strlen(out)); + + done: + tor_free(out); +} + +static void +test_util_sscanf(void) +{ + unsigned u1, u2, u3; + char s1[10], s2[10], s3[10], ch; + int r; + + r = tor_sscanf("hello world", "hello world"); /* String match: success */ + test_eq(r, 0); + r = tor_sscanf("hello world 3", "hello worlb %u", &u1); /* String fail */ + test_eq(r, 0); + r = tor_sscanf("12345", "%u", &u1); /* Simple number */ + test_eq(r, 1); + test_eq(u1, 12345u); + r = tor_sscanf("", "%u", &u1); /* absent number */ + test_eq(r, 0); + r = tor_sscanf("A", "%u", &u1); /* bogus number */ + test_eq(r, 0); + r = tor_sscanf("4294967295", "%u", &u1); /* UINT32_MAX should work. */ + test_eq(r, 1); + test_eq(u1, 4294967295u); + r = tor_sscanf("4294967296", "%u", &u1); /* Always say -1 at 32 bits. */ + test_eq(r, 0); + r = tor_sscanf("123456", "%2u%u", &u1, &u2); /* Width */ + test_eq(r, 2); + test_eq(u1, 12u); + test_eq(u2, 3456u); + r = tor_sscanf("!12:3:456", "!%2u:%2u:%3u", &u1, &u2, &u3); /* separators */ + test_eq(r, 3); + test_eq(u1, 12u); + test_eq(u2, 3u); + test_eq(u3, 456u); + r = tor_sscanf("12:3:045", "%2u:%2u:%3u", &u1, &u2, &u3); /* 0s */ + test_eq(r, 3); + test_eq(u1, 12u); + test_eq(u2, 3u); + test_eq(u3, 45u); + /* %u does not match space.*/ + r = tor_sscanf("12:3: 45", "%2u:%2u:%3u", &u1, &u2, &u3); + test_eq(r, 2); + /* %u does not match negative numbers. */ + r = tor_sscanf("12:3:-4", "%2u:%2u:%3u", &u1, &u2, &u3); + test_eq(r, 2); + /* Arbitrary amounts of 0-padding are okay */ + r = tor_sscanf("12:03:000000000000000099", "%2u:%2u:%u", &u1, &u2, &u3); + test_eq(r, 3); + test_eq(u1, 12u); + test_eq(u2, 3u); + test_eq(u3, 99u); + + r = tor_sscanf("99% fresh", "%3u%% fresh", &u1); /* percents are scannable.*/ + test_eq(r, 1); + test_eq(u1, 99); + + r = tor_sscanf("hello", "%s", s1); /* %s needs a number. */ + test_eq(r, -1); + + r = tor_sscanf("hello", "%3s%7s", s1, s2); /* %s matches characters. */ + test_eq(r, 2); + test_streq(s1, "hel"); + test_streq(s2, "lo"); + r = tor_sscanf("WD40", "%2s%u", s3, &u1); /* %s%u */ + test_eq(r, 2); + test_streq(s3, "WD"); + test_eq(u1, 40); + r = tor_sscanf("76trombones", "%6u%9s", &u1, s1); /* %u%s */ + test_eq(r, 2); + test_eq(u1, 76); + test_streq(s1, "trombones"); + r = tor_sscanf("hello world", "%9s %9s", s1, s2); /* %s doesn't eat space. */ + test_eq(r, 2); + test_streq(s1, "hello"); + test_streq(s2, "world"); + r = tor_sscanf("hi", "%9s%9s%3s", s1, s2, s3); /* %s can be empty. */ + test_eq(r, 3); + test_streq(s1, "hi"); + test_streq(s2, ""); + test_streq(s3, ""); + + r = tor_sscanf("1.2.3", "%u.%u.%u%c", &u1, &u2, &u3, &ch); + test_eq(r, 3); + r = tor_sscanf("1.2.3 foobar", "%u.%u.%u%c", &u1, &u2, &u3, &ch); + test_eq(r, 4); + + done: + ; +} + +/** Run unittests for memory pool allocator */ +static void +test_util_mempool(void) +{ + mp_pool_t *pool = NULL; + smartlist_t *allocated = NULL; + int i; + + pool = mp_pool_new(1, 100); + test_assert(pool); + test_assert(pool->new_chunk_capacity >= 100); + test_assert(pool->item_alloc_size >= sizeof(void*)+1); + mp_pool_destroy(pool); + pool = NULL; + + pool = mp_pool_new(241, 2500); + test_assert(pool); + test_assert(pool->new_chunk_capacity >= 10); + test_assert(pool->item_alloc_size >= sizeof(void*)+241); + test_eq(pool->item_alloc_size & 0x03, 0); + test_assert(pool->new_chunk_capacity < 60); + + allocated = smartlist_create(); + for (i = 0; i < 20000; ++i) { + if (smartlist_len(allocated) < 20 || crypto_rand_int(2)) { + void *m = mp_pool_get(pool); + memset(m, 0x09, 241); + smartlist_add(allocated, m); + //printf("%d: %p\n", i, m); + //mp_pool_assert_ok(pool); + } else { + int idx = crypto_rand_int(smartlist_len(allocated)); + void *m = smartlist_get(allocated, idx); + //printf("%d: free %p\n", i, m); + smartlist_del(allocated, idx); + mp_pool_release(m); + //mp_pool_assert_ok(pool); + } + if (crypto_rand_int(777)==0) + mp_pool_clean(pool, 1, 1); + + if (i % 777) + mp_pool_assert_ok(pool); + } + + done: + if (allocated) { + SMARTLIST_FOREACH(allocated, void *, m, mp_pool_release(m)); + mp_pool_assert_ok(pool); + mp_pool_clean(pool, 0, 0); + mp_pool_assert_ok(pool); + smartlist_free(allocated); + } + + if (pool) + mp_pool_destroy(pool); +} + +/** Run unittests for memory area allocator */ +static void +test_util_memarea(void) +{ + memarea_t *area = memarea_new(); + char *p1, *p2, *p3, *p1_orig; + void *malloced_ptr = NULL; + int i; + + test_assert(area); + + p1_orig = p1 = memarea_alloc(area,64); + p2 = memarea_alloc_zero(area,52); + p3 = memarea_alloc(area,11); + + test_assert(memarea_owns_ptr(area, p1)); + test_assert(memarea_owns_ptr(area, p2)); + test_assert(memarea_owns_ptr(area, p3)); + /* Make sure we left enough space. */ + test_assert(p1+64 <= p2); + test_assert(p2+52 <= p3); + /* Make sure we aligned. */ + test_eq(((uintptr_t)p1) % sizeof(void*), 0); + test_eq(((uintptr_t)p2) % sizeof(void*), 0); + test_eq(((uintptr_t)p3) % sizeof(void*), 0); + test_assert(!memarea_owns_ptr(area, p3+8192)); + test_assert(!memarea_owns_ptr(area, p3+30)); + test_assert(tor_mem_is_zero(p2, 52)); + /* Make sure we don't overalign. */ + p1 = memarea_alloc(area, 1); + p2 = memarea_alloc(area, 1); + test_eq(p1+sizeof(void*), p2); + { + malloced_ptr = tor_malloc(64); + test_assert(!memarea_owns_ptr(area, malloced_ptr)); + tor_free(malloced_ptr); + } + + /* memarea_memdup */ + { + malloced_ptr = tor_malloc(64); + crypto_rand((char*)malloced_ptr, 64); + p1 = memarea_memdup(area, malloced_ptr, 64); + test_assert(p1 != malloced_ptr); + test_memeq(p1, malloced_ptr, 64); + tor_free(malloced_ptr); + } + + /* memarea_strdup. */ + p1 = memarea_strdup(area,""); + p2 = memarea_strdup(area, "abcd"); + test_assert(p1); + test_assert(p2); + test_streq(p1, ""); + test_streq(p2, "abcd"); + + /* memarea_strndup. */ + { + const char *s = "Ad ogni porta batte la morte e grida: il nome!"; + /* (From Turandot, act 3.) */ + size_t len = strlen(s); + p1 = memarea_strndup(area, s, 1000); + p2 = memarea_strndup(area, s, 10); + test_streq(p1, s); + test_assert(p2 >= p1 + len + 1); + test_memeq(s, p2, 10); + test_eq(p2[10], '\0'); + p3 = memarea_strndup(area, s, len); + test_streq(p3, s); + p3 = memarea_strndup(area, s, len-1); + test_memeq(s, p3, len-1); + test_eq(p3[len-1], '\0'); + } + + memarea_clear(area); + p1 = memarea_alloc(area, 1); + test_eq(p1, p1_orig); + memarea_clear(area); + + /* Check for running over an area's size. */ + for (i = 0; i < 512; ++i) { + p1 = memarea_alloc(area, crypto_rand_int(5)+1); + test_assert(memarea_owns_ptr(area, p1)); + } + memarea_assert_ok(area); + /* Make sure we can allocate a too-big object. */ + p1 = memarea_alloc_zero(area, 9000); + p2 = memarea_alloc_zero(area, 16); + test_assert(memarea_owns_ptr(area, p1)); + test_assert(memarea_owns_ptr(area, p2)); + + done: + memarea_drop_all(area); + tor_free(malloced_ptr); +} + +/** Run unit tests for utility functions to get file names relative to + * the data directory. */ +static void +test_util_datadir(void) +{ + char buf[1024]; + char *f = NULL; + char *temp_dir = NULL; + + temp_dir = get_datadir_fname(NULL); + f = get_datadir_fname("state"); + tor_snprintf(buf, sizeof(buf), "%s"PATH_SEPARATOR"state", temp_dir); + test_streq(f, buf); + tor_free(f); + f = get_datadir_fname2("cache", "thingy"); + tor_snprintf(buf, sizeof(buf), + "%s"PATH_SEPARATOR"cache"PATH_SEPARATOR"thingy", temp_dir); + test_streq(f, buf); + tor_free(f); + f = get_datadir_fname2_suffix("cache", "thingy", ".foo"); + tor_snprintf(buf, sizeof(buf), + "%s"PATH_SEPARATOR"cache"PATH_SEPARATOR"thingy.foo", temp_dir); + test_streq(f, buf); + tor_free(f); + f = get_datadir_fname_suffix("cache", ".foo"); + tor_snprintf(buf, sizeof(buf), "%s"PATH_SEPARATOR"cache.foo", + temp_dir); + test_streq(f, buf); + + done: + tor_free(f); + tor_free(temp_dir); +} + +static void +test_util_strtok(void) +{ + char buf[128]; + char buf2[128]; + char *cp1, *cp2; + strlcpy(buf, "Graved on the dark in gestures of descent", sizeof(buf)); + strlcpy(buf2, "they.seemed;their!own;most.perfect;monument", sizeof(buf2)); + /* -- "Year's End", Richard Wilbur */ + + test_streq("Graved", tor_strtok_r_impl(buf, " ", &cp1)); + test_streq("they", tor_strtok_r_impl(buf2, ".!..;!", &cp2)); +#define S1() tor_strtok_r_impl(NULL, " ", &cp1) +#define S2() tor_strtok_r_impl(NULL, ".!..;!", &cp2) + test_streq("on", S1()); + test_streq("the", S1()); + test_streq("dark", S1()); + test_streq("seemed", S2()); + test_streq("their", S2()); + test_streq("own", S2()); + test_streq("in", S1()); + test_streq("gestures", S1()); + test_streq("of", S1()); + test_streq("most", S2()); + test_streq("perfect", S2()); + test_streq("descent", S1()); + test_streq("monument", S2()); + test_assert(NULL == S1()); + test_assert(NULL == S2()); + done: + ; +} + +static void +test_util_find_str_at_start_of_line(void *ptr) +{ + const char *long_string = + "hello world. hello world. hello hello. howdy.\n" + "hello hello world\n"; + + (void)ptr; + + /* not-found case. */ + tt_assert(! find_str_at_start_of_line(long_string, "fred")); + + /* not-found case where haystack doesn't end with \n */ + tt_assert(! find_str_at_start_of_line("foobar\nbaz", "fred")); + + /* start-of-string case */ + tt_assert(long_string == + find_str_at_start_of_line(long_string, "hello world.")); + + /* start-of-line case */ + tt_assert(strchr(long_string,'\n')+1 == + find_str_at_start_of_line(long_string, "hello hello")); + done: + ; +} + +static void +test_util_asprintf(void *ptr) +{ +#define LOREMIPSUM \ + "Lorem ipsum dolor sit amet, consectetur adipisicing elit" + char *cp=NULL, *cp2=NULL; + int r; + (void)ptr; + + /* empty string. */ + r = tor_asprintf(&cp, "%s", ""); + tt_assert(cp); + tt_int_op(r, ==, strlen(cp)); + tt_str_op(cp, ==, ""); + + /* Short string with some printing in it. */ + r = tor_asprintf(&cp2, "First=%d, Second=%d", 101, 202); + tt_assert(cp2); + tt_int_op(r, ==, strlen(cp2)); + tt_str_op(cp2, ==, "First=101, Second=202"); + tt_assert(cp != cp2); + tor_free(cp); + tor_free(cp2); + + /* Glass-box test: a string exactly 128 characters long. */ + r = tor_asprintf(&cp, "Lorem1: %sLorem2: %s", LOREMIPSUM, LOREMIPSUM); + tt_assert(cp); + tt_int_op(r, ==, 128); + tt_assert(cp[128] == '\0'); + tt_str_op(cp, ==, + "Lorem1: "LOREMIPSUM"Lorem2: "LOREMIPSUM); + tor_free(cp); + + /* String longer than 128 characters */ + r = tor_asprintf(&cp, "1: %s 2: %s 3: %s", + LOREMIPSUM, LOREMIPSUM, LOREMIPSUM); + tt_assert(cp); + tt_int_op(r, ==, strlen(cp)); + tt_str_op(cp, ==, "1: "LOREMIPSUM" 2: "LOREMIPSUM" 3: "LOREMIPSUM); + + done: + tor_free(cp); + tor_free(cp2); +} + +static void +test_util_listdir(void *ptr) +{ + smartlist_t *dir_contents = NULL; + char *fname1=NULL, *fname2=NULL, *dirname=NULL; + (void)ptr; + + fname1 = tor_strdup(get_fname("hopscotch")); + fname2 = tor_strdup(get_fname("mumblety-peg")); + dirname = tor_strdup(get_fname(NULL)); + + tt_int_op(write_str_to_file(fname1, "X\n", 0), ==, 0); + tt_int_op(write_str_to_file(fname2, "Y\n", 0), ==, 0); + + dir_contents = tor_listdir(dirname); + tt_assert(dir_contents); + /* make sure that each filename is listed. */ + tt_assert(smartlist_string_isin_case(dir_contents, "hopscotch")); + tt_assert(smartlist_string_isin_case(dir_contents, "mumblety-peg")); + + tt_assert(!smartlist_string_isin(dir_contents, ".")); + tt_assert(!smartlist_string_isin(dir_contents, "..")); + + done: + tor_free(fname1); + tor_free(fname2); + tor_free(dirname); + if (dir_contents) { + SMARTLIST_FOREACH(dir_contents, char *, cp, tor_free(cp)); + smartlist_free(dir_contents); + } +} + +#ifdef MS_WINDOWS +static void +test_util_load_win_lib(void *ptr) +{ + HANDLE h = load_windows_system_library("advapi32.dll"); + (void) ptr; + + tt_assert(h); + done: + if (h) + CloseHandle(h); +} +#endif + +#define UTIL_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_util_ ## name } + +#define UTIL_TEST(name, flags) \ + { #name, test_util_ ## name, flags, NULL, NULL } + +struct testcase_t util_tests[] = { + UTIL_LEGACY(time), + UTIL_LEGACY(config_line), + UTIL_LEGACY(strmisc), + UTIL_LEGACY(pow2), + UTIL_LEGACY(gzip), + UTIL_LEGACY(datadir), + UTIL_LEGACY(mempool), + UTIL_LEGACY(memarea), + UTIL_LEGACY(control_formats), + UTIL_LEGACY(mmap), + UTIL_LEGACY(threads), + UTIL_LEGACY(sscanf), + UTIL_LEGACY(strtok), + UTIL_TEST(find_str_at_start_of_line, 0), + UTIL_TEST(asprintf, 0), + UTIL_TEST(listdir, 0), +#ifdef MS_WINDOWS + UTIL_TEST(load_win_lib, 0), +#endif + END_OF_TESTCASES +}; + diff --git a/src/test/tinytest.c b/src/test/tinytest.c new file mode 100644 index 000000000..11ffc2fe5 --- /dev/null +++ b/src/test/tinytest.c @@ -0,0 +1,380 @@ +/* tinytest.c -- Copyright 2009-2010 Nick Mathewson + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> + +#ifdef WIN32 +#include <windows.h> +#else +#include <sys/types.h> +#include <sys/wait.h> +#include <unistd.h> +#endif + +#ifndef __GNUC__ +#define __attribute__(x) +#endif + +#ifdef TINYTEST_LOCAL +#include "tinytest_local.h" +#endif +#include "tinytest.h" +#include "tinytest_macros.h" + +#define LONGEST_TEST_NAME 16384 + +static int in_tinytest_main = 0; /**< true if we're in tinytest_main().*/ +static int n_ok = 0; /**< Number of tests that have passed */ +static int n_bad = 0; /**< Number of tests that have failed. */ +static int n_skipped = 0; /**< Number of tests that have been skipped. */ + +static int opt_forked = 0; /**< True iff we're called from inside a win32 fork*/ +static int opt_nofork = 0; /**< Suppress calls to fork() for debugging. */ +static int opt_verbosity = 1; /**< -==quiet,0==terse,1==normal,2==verbose */ +const char *verbosity_flag = ""; + +enum outcome { SKIP=2, OK=1, FAIL=0 }; +static enum outcome cur_test_outcome = 0; +const char *cur_test_prefix = NULL; /**< prefix of the current test group */ +/** Name of the current test, if we haven't logged is yet. Used for --quiet */ +const char *cur_test_name = NULL; + +#ifdef WIN32 +/** Pointer to argv[0] for win32. */ +static const char *commandname = NULL; +#endif + +static void usage(struct testgroup_t *groups, int list_groups) + __attribute__((noreturn)); + +static enum outcome +_testcase_run_bare(const struct testcase_t *testcase) +{ + void *env = NULL; + int outcome; + if (testcase->setup) { + env = testcase->setup->setup_fn(testcase); + if (!env) + return FAIL; + else if (env == (void*)TT_SKIP) + return SKIP; + } + + cur_test_outcome = OK; + testcase->fn(env); + outcome = cur_test_outcome; + + if (testcase->setup) { + if (testcase->setup->cleanup_fn(testcase, env) == 0) + outcome = FAIL; + } + + return outcome; +} + +#define MAGIC_EXITCODE 42 + +static enum outcome +_testcase_run_forked(const struct testgroup_t *group, + const struct testcase_t *testcase) +{ +#ifdef WIN32 + /* Fork? On Win32? How primitive! We'll do what the smart kids do: + we'll invoke our own exe (whose name we recall from the command + line) with a command line that tells it to run just the test we + want, and this time without forking. + + (No, threads aren't an option. The whole point of forking is to + share no state between tests.) + */ + int ok; + char buffer[LONGEST_TEST_NAME+256]; + STARTUPINFOA si; + PROCESS_INFORMATION info; + DWORD exitcode; + + if (!in_tinytest_main) { + printf("\nERROR. On Windows, _testcase_run_forked must be" + " called from within tinytest_main.\n"); + abort(); + } + if (opt_verbosity>0) + printf("[forking] "); + + snprintf(buffer, sizeof(buffer), "%s --RUNNING-FORKED %s %s%s", + commandname, verbosity_flag, group->prefix, testcase->name); + + memset(&si, 0, sizeof(si)); + memset(&info, 0, sizeof(info)); + si.cb = sizeof(si); + + ok = CreateProcessA(commandname, buffer, NULL, NULL, 0, + 0, NULL, NULL, &si, &info); + if (!ok) { + printf("CreateProcess failed!\n"); + return 0; + } + WaitForSingleObject(info.hProcess, INFINITE); + GetExitCodeProcess(info.hProcess, &exitcode); + CloseHandle(info.hProcess); + CloseHandle(info.hThread); + if (exitcode == 0) + return OK; + else if (exitcode == MAGIC_EXITCODE) + return SKIP; + else + return FAIL; +#else + int outcome_pipe[2]; + pid_t pid; + (void)group; + + if (pipe(outcome_pipe)) + perror("opening pipe"); + + if (opt_verbosity>0) + printf("[forking] "); + pid = fork(); + if (!pid) { + /* child. */ + int test_r, write_r; + char b[1]; + close(outcome_pipe[0]); + test_r = _testcase_run_bare(testcase); + assert(0<=(int)test_r && (int)test_r<=2); + b[0] = "NYS"[test_r]; + write_r = (int)write(outcome_pipe[1], b, 1); + if (write_r != 1) { + perror("write outcome to pipe"); + exit(1); + } + exit(0); + } else { + /* parent */ + int status, r; + char b[1]; + /* Close this now, so that if the other side closes it, + * our read fails. */ + close(outcome_pipe[1]); + r = (int)read(outcome_pipe[0], b, 1); + if (r == 0) { + printf("[Lost connection!] "); + return 0; + } else if (r != 1) { + perror("read outcome from pipe"); + } + waitpid(pid, &status, 0); + close(outcome_pipe[0]); + return b[0]=='Y' ? OK : (b[0]=='S' ? SKIP : FAIL); + } +#endif +} + +int +testcase_run_one(const struct testgroup_t *group, + const struct testcase_t *testcase) +{ + enum outcome outcome; + + if (testcase->flags & TT_SKIP) { + if (opt_verbosity>0) + printf("%s%s: SKIPPED\n", + group->prefix, testcase->name); + ++n_skipped; + return SKIP; + } + + if (opt_verbosity>0 && !opt_forked) { + printf("%s%s: ", group->prefix, testcase->name); + } else { + if (opt_verbosity==0) printf("."); + cur_test_prefix = group->prefix; + cur_test_name = testcase->name; + } + + if ((testcase->flags & TT_FORK) && !(opt_forked||opt_nofork)) { + outcome = _testcase_run_forked(group, testcase); + } else { + outcome = _testcase_run_bare(testcase); + } + + if (outcome == OK) { + ++n_ok; + if (opt_verbosity>0 && !opt_forked) + puts(opt_verbosity==1?"OK":""); + } else if (outcome == SKIP) { + ++n_skipped; + if (opt_verbosity>0 && !opt_forked) + puts("SKIPPED"); + } else { + ++n_bad; + if (!opt_forked) + printf("\n [%s FAILED]\n", testcase->name); + } + + if (opt_forked) { + exit(outcome==OK ? 0 : (outcome==SKIP?MAGIC_EXITCODE : 1)); + } else { + return (int)outcome; + } +} + +int +_tinytest_set_flag(struct testgroup_t *groups, const char *arg, unsigned long flag) +{ + int i, j; + size_t length = LONGEST_TEST_NAME; + char fullname[LONGEST_TEST_NAME]; + int found=0; + if (strstr(arg, "..")) + length = strstr(arg,"..")-arg; + for (i=0; groups[i].prefix; ++i) { + for (j=0; groups[i].cases[j].name; ++j) { + snprintf(fullname, sizeof(fullname), "%s%s", + groups[i].prefix, groups[i].cases[j].name); + if (!flag) /* Hack! */ + printf(" %s\n", fullname); + if (!strncmp(fullname, arg, length)) { + groups[i].cases[j].flags |= flag; + ++found; + } + } + } + return found; +} + +static void +usage(struct testgroup_t *groups, int list_groups) +{ + puts("Options are: [--verbose|--quiet|--terse] [--no-fork]"); + puts(" Specify tests by name, or using a prefix ending with '..'"); + puts(" To skip a test, list give its name prefixed with a colon."); + puts(" Use --list-tests for a list of tests."); + if (list_groups) { + puts("Known tests are:"); + _tinytest_set_flag(groups, "..", 0); + } + exit(0); +} + +int +tinytest_main(int c, const char **v, struct testgroup_t *groups) +{ + int i, j, n=0; + +#ifdef WIN32 + commandname = v[0]; +#endif + for (i=1; i<c; ++i) { + if (v[i][0] == '-') { + if (!strcmp(v[i], "--RUNNING-FORKED")) { + opt_forked = 1; + } else if (!strcmp(v[i], "--no-fork")) { + opt_nofork = 1; + } else if (!strcmp(v[i], "--quiet")) { + opt_verbosity = -1; + verbosity_flag = "--quiet"; + } else if (!strcmp(v[i], "--verbose")) { + opt_verbosity = 2; + verbosity_flag = "--verbose"; + } else if (!strcmp(v[i], "--terse")) { + opt_verbosity = 0; + verbosity_flag = "--terse"; + } else if (!strcmp(v[i], "--help")) { + usage(groups, 0); + } else if (!strcmp(v[i], "--list-tests")) { + usage(groups, 1); + } else { + printf("Unknown option %s. Try --help\n",v[i]); + return -1; + } + } else { + const char *test = v[i]; + int flag = _TT_ENABLED; + if (test[0] == ':') { + ++test; + flag = TT_SKIP; + } else { + ++n; + } + if (!_tinytest_set_flag(groups, test, flag)) { + printf("No such test as %s!\n", v[i]); + return -1; + } + } + } + if (!n) + _tinytest_set_flag(groups, "..", _TT_ENABLED); + + setvbuf(stdout, NULL, _IONBF, 0); + + ++in_tinytest_main; + for (i=0; groups[i].prefix; ++i) + for (j=0; groups[i].cases[j].name; ++j) + if (groups[i].cases[j].flags & _TT_ENABLED) + testcase_run_one(&groups[i], + &groups[i].cases[j]); + + --in_tinytest_main; + + if (opt_verbosity==0) + puts(""); + + if (n_bad) + printf("%d/%d TESTS FAILED. (%d skipped)\n", n_bad, + n_bad+n_ok,n_skipped); + else if (opt_verbosity >= 1) + printf("%d tests ok. (%d skipped)\n", n_ok, n_skipped); + + return (n_bad == 0) ? 0 : 1; +} + +int +_tinytest_get_verbosity(void) +{ + return opt_verbosity; +} + +void +_tinytest_set_test_failed(void) +{ + if (opt_verbosity <= 0 && cur_test_name) { + if (opt_verbosity==0) puts(""); + printf("%s%s: ", cur_test_prefix, cur_test_name); + cur_test_name = NULL; + } + cur_test_outcome = 0; +} + +void +_tinytest_set_test_skipped(void) +{ + if (cur_test_outcome==OK) + cur_test_outcome = SKIP; +} + diff --git a/src/test/tinytest.h b/src/test/tinytest.h new file mode 100644 index 000000000..cbe28b7f5 --- /dev/null +++ b/src/test/tinytest.h @@ -0,0 +1,87 @@ +/* tinytest.h -- Copyright 2009-2010 Nick Mathewson + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _TINYTEST_H +#define _TINYTEST_H + +/** Flag for a test that needs to run in a subprocess. */ +#define TT_FORK (1<<0) +/** Runtime flag for a test we've decided to skip. */ +#define TT_SKIP (1<<1) +/** Internal runtime flag for a test we've decided to run. */ +#define _TT_ENABLED (1<<2) +/** If you add your own flags, make them start at this point. */ +#define TT_FIRST_USER_FLAG (1<<3) + +typedef void (*testcase_fn)(void *); + +struct testcase_t; + +/** Functions to initialize/teardown a structure for a testcase. */ +struct testcase_setup_t { + /** Return a new structure for use by a given testcase. */ + void *(*setup_fn)(const struct testcase_t *); + /** Clean/free a structure from setup_fn. Return 1 if ok, 0 on err. */ + int (*cleanup_fn)(const struct testcase_t *, void *); +}; + +/** A single test-case that you can run. */ +struct testcase_t { + const char *name; /**< An identifier for this case. */ + testcase_fn fn; /**< The function to run to implement this case. */ + unsigned long flags; /**< Bitfield of TT_* flags. */ + const struct testcase_setup_t *setup; /**< Optional setup/cleanup fns*/ + void *setup_data; /**< Extra data usable by setup function */ +}; +#define END_OF_TESTCASES { NULL, NULL, 0, NULL, NULL } + +/** A group of tests that are selectable together. */ +struct testgroup_t { + const char *prefix; /**< Prefix to prepend to testnames. */ + struct testcase_t *cases; /** Array, ending with END_OF_TESTCASES */ +}; +#define END_OF_GROUPS { NULL, NULL} + +/** Implementation: called from a test to indicate failure, before logging. */ +void _tinytest_set_test_failed(void); +/** Implementation: called from a test to indicate that we're skipping. */ +void _tinytest_set_test_skipped(void); +/** Implementation: return 0 for quiet, 1 for normal, 2 for loud. */ +int _tinytest_get_verbosity(void); +/** Implementation: Set a flag on tests matching a name; returns number + * of tests that matched. */ +int _tinytest_set_flag(struct testgroup_t *, const char *, unsigned long); + +/** Set all tests in 'groups' matching the name 'named' to be skipped. */ +#define tinytest_skip(groups, named) \ + _tinytest_set_flag(groups, named, TT_SKIP) + +/** Run a single testcase in a single group. */ +int testcase_run_one(const struct testgroup_t *,const struct testcase_t *); +/** Run a set of testcases from an END_OF_GROUPS-terminated array of groups, + as selected from the command line. */ +int tinytest_main(int argc, const char **argv, struct testgroup_t *groups); + +#endif diff --git a/src/test/tinytest_demo.c b/src/test/tinytest_demo.c new file mode 100644 index 000000000..bd33cc37f --- /dev/null +++ b/src/test/tinytest_demo.c @@ -0,0 +1,215 @@ +/* tinytest_demo.c -- Copyright 2009 Nick Mathewson + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +/* Welcome to the example file for tinytest! I'll show you how to set up + * some simple and not-so-simple testcases. */ + +/* Make sure you include these headers. */ +#include "tinytest.h" +#include "tinytest_macros.h" + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <errno.h> + +/* ============================================================ */ + +/* First, let's see if strcmp is working. (All your test cases should be + * functions declared to take a single void * as) an argument. */ +void +test_strcmp(void *data) +{ + (void)data; /* This testcase takes no data. */ + + /* Let's make sure the empty string is equal to itself */ + if (strcmp("","")) { + /* This macro tells tinytest to stop the current test + * and go straight to the "end" label. */ + tt_abort_msg("The empty string was not equal to itself"); + } + + /* Pretty often, calling tt_abort_msg to indicate failure is more + heavy-weight than you want. Instead, just say: */ + tt_assert(strcmp("testcase", "testcase") == 0); + + /* Occasionally, you don't want to stop the current testcase just + because a single assertion has failed. In that case, use + tt_want: */ + tt_want(strcmp("tinytest", "testcase") > 0); + + /* You can use the tt_*_op family of macros to compare values and to + fail unless they have the relationship you want. They produce + more useful output than tt_assert, since they display the actual + values of the failing things. + + Fail unless strcmp("abc, "abc") == 0 */ + tt_int_op(strcmp("abc", "abc"), ==, 0); + + /* Fail unless strcmp("abc, "abcd") is less than 0 */ + tt_int_op(strcmp("abc", "abcd"), < , 0); + + /* Incidentally, there's a test_str_op that uses strcmp internally. */ + tt_str_op("abc", <, "abcd"); + + + /* Every test-case function needs to finish with an "end:" + label and (optionally) code to clean up local variables. */ + end: + ; +} + +/* ============================================================ */ + +/* Now let's mess with setup and teardown functions! These are handy if + you have a bunch of tests that all need a similar environment, and you + want to reconstruct that environment freshly for each one. */ + +/* First you declare a type to hold the environment info, and functions to + set it up and tear it down. */ +struct data_buffer { + /* We're just going to have couple of character buffer. Using + setup/teardown functions is probably overkill for this case. + + You could also do file descriptors, complicated handles, temporary + files, etc. */ + char buffer1[512]; + char buffer2[512]; +}; +/* The setup function needs to take a const struct testcase_t and return + void* */ +void * +setup_data_buffer(const struct testcase_t *testcase) +{ + struct data_buffer *db = malloc(sizeof(struct data_buffer)); + + /* If you had a complicated set of setup rules, you might behave + differently here depending on testcase->flags or + testcase->setup_data or even or testcase->name. */ + + /* Returning a NULL here would mean that we couldn't set up for this + test, so we don't need to test db for null. */ + return db; +} +/* The clean function deallocates storage carefully and returns true on + success. */ +int +clean_data_buffer(const struct testcase_t *testcase, void *ptr) +{ + struct data_buffer *db = ptr; + + if (db) { + free(db); + return 1; + } + return 0; +} +/* Finally, declare a testcase_setup_t with these functions. */ +struct testcase_setup_t data_buffer_setup = { + setup_data_buffer, clean_data_buffer +}; + + +/* Now let's write our test. */ +void +test_memcpy(void *ptr) +{ + /* This time, we use the argument. */ + struct data_buffer *db = ptr; + + /* We'll also introduce a local variable that might need cleaning up. */ + char *mem = NULL; + + /* Let's make sure that memcpy does what we'd like. */ + strcpy(db->buffer1, "String 0"); + memcpy(db->buffer2, db->buffer1, sizeof(db->buffer1)); + tt_str_op(db->buffer1, ==, db->buffer2); + + /* Now we've allocated memory that's referenced by a local variable. + The end block of the function will clean it up. */ + mem = strdup("Hello world."); + tt_assert(mem); + + /* Another rather trivial test. */ + tt_str_op(db->buffer1, !=, mem); + + end: + /* This time our end block has something to do. */ + if (mem) + free(mem); +} + +/* ============================================================ */ + +/* Now we need to make sure that our tests get invoked. First, you take + a bunch of related tests and put them into an array of struct testcase_t. +*/ + +struct testcase_t demo_tests[] = { + /* Here's a really simple test: it has a name you can refer to it + with, and a function to invoke it. */ + { "strcmp", test_strcmp, }, + + /* The second test has a flag, "TT_FORK", to make it run in a + subprocess, and a pointer to the testcase_setup_t that configures + its environment. */ + { "memcpy", test_memcpy, TT_FORK, &data_buffer_setup }, + + /* The array has to end with END_OF_TESTCASES. */ + END_OF_TESTCASES +}; + +/* Next, we make an array of testgroups. This is mandatory. Unlike more + heavy-duty testing frameworks, groups can't nest. */ +struct testgroup_t groups[] = { + + /* Every group has a 'prefix', and an array of tests. That's it. */ + { "demo/", demo_tests }, + + END_OF_GROUPS +}; + + +int +main(int c, const char **v) +{ + /* Finally, just call tinytest_main(). It lets you specify verbose + or quiet output with --verbose and --quiet. You can list + specific tests: + + tinytest-demo demo/memcpy + + or use a ..-wildcard to select multiple tests with a common + prefix: + + tinytest-demo demo/.. + + If you list no tests, you get them all by default, so that + "tinytest-demo" and "tinytest-demo .." mean the same thing. + + */ + return tinytest_main(c, v, groups); +} diff --git a/src/test/tinytest_macros.h b/src/test/tinytest_macros.h new file mode 100644 index 000000000..a7fa64a82 --- /dev/null +++ b/src/test/tinytest_macros.h @@ -0,0 +1,167 @@ +/* tinytest_macros.h -- Copyright 2009-2010 Nick Mathewson + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _TINYTEST_MACROS_H +#define _TINYTEST_MACROS_H + +/* Helpers for defining statement-like macros */ +#define TT_STMT_BEGIN do { +#define TT_STMT_END } while (0) + +/* Redefine this if your test functions want to abort with something besides + * "goto end;" */ +#ifndef TT_EXIT_TEST_FUNCTION +#define TT_EXIT_TEST_FUNCTION TT_STMT_BEGIN goto end; TT_STMT_END +#endif + +/* Redefine this if you want to note success/failure in some different way. */ +#ifndef TT_DECLARE +#define TT_DECLARE(prefix, args) \ + TT_STMT_BEGIN \ + printf("\n %s %s:%d: ",prefix,__FILE__,__LINE__); \ + printf args ; \ + TT_STMT_END +#endif + +/* Announce a failure. Args are parenthesized printf args. */ +#define TT_GRIPE(args) TT_DECLARE("FAIL", args) + +/* Announce a non-failure if we're verbose. */ +#define TT_BLATHER(args) \ + TT_STMT_BEGIN \ + if (_tinytest_get_verbosity()>1) TT_DECLARE(" OK", args); \ + TT_STMT_END + +#define TT_DIE(args) \ + TT_STMT_BEGIN \ + _tinytest_set_test_failed(); \ + TT_GRIPE(args); \ + TT_EXIT_TEST_FUNCTION; \ + TT_STMT_END + +#define TT_FAIL(args) \ + TT_STMT_BEGIN \ + _tinytest_set_test_failed(); \ + TT_GRIPE(args); \ + TT_STMT_END + +/* Fail and abort the current test for the reason in msg */ +#define tt_abort_printf(msg) TT_DIE(msg) +#define tt_abort_perror(op) TT_DIE(("%s: %s [%d]",(op),strerror(errno), errno)) +#define tt_abort_msg(msg) TT_DIE(("%s", msg)) +#define tt_abort() TT_DIE(("%s", "(Failed.)")) + +/* Fail but do not abort the current test for the reason in msg. */ +#define tt_fail_printf(msg) TT_FAIL(msg) +#define tt_fail_perror(op) TT_FAIL(("%s: %s [%d]",(op),strerror(errno), errno)) +#define tt_fail_msg(msg) TT_FAIL(("%s", msg)) +#define tt_fail() TT_FAIL(("%s", "(Failed.)")) + +/* End the current test, and indicate we are skipping it. */ +#define tt_skip() \ + TT_STMT_BEGIN \ + _tinytest_set_test_skipped(); \ + TT_EXIT_TEST_FUNCTION; \ + TT_STMT_END + +#define _tt_want(b, msg, fail) \ + TT_STMT_BEGIN \ + if (!(b)) { \ + _tinytest_set_test_failed(); \ + TT_GRIPE((msg)); \ + fail; \ + } else { \ + TT_BLATHER((msg)); \ + } \ + TT_STMT_END + +/* Assert b, but do not stop the test if b fails. Log msg on failure. */ +#define tt_want_msg(b, msg) \ + _tt_want(b, msg, ); + +/* Assert b and stop the test if b fails. Log msg on failure. */ +#define tt_assert_msg(b, msg) \ + _tt_want(b, msg, TT_EXIT_TEST_FUNCTION); + +/* Assert b, but do not stop the test if b fails. */ +#define tt_want(b) tt_want_msg( (b), "want("#b")") +/* Assert b, and stop the test if b fails. */ +#define tt_assert(b) tt_assert_msg((b), "assert("#b")") + +#define tt_assert_test_fmt_type(a,b,str_test,type,test,printf_type,printf_fmt, \ + setup_block,cleanup_block) \ + TT_STMT_BEGIN \ + type _val1 = (type)(a); \ + type _val2 = (type)(b); \ + int _tt_status = (test); \ + if (!_tt_status || _tinytest_get_verbosity()>1) { \ + printf_type _print; \ + printf_type _print1; \ + printf_type _print2; \ + type _value = _val1; \ + setup_block; \ + _print1 = _print; \ + _value = _val2; \ + setup_block; \ + _print2 = _print; \ + TT_DECLARE(_tt_status?" OK":"FAIL", \ + ("assert(%s): "printf_fmt" vs "printf_fmt, \ + str_test, _print1, _print2)); \ + _print = _print1; \ + cleanup_block; \ + _print = _print2; \ + cleanup_block; \ + if (!_tt_status) { \ + _tinytest_set_test_failed(); \ + TT_EXIT_TEST_FUNCTION; \ + } \ + } \ + TT_STMT_END + +#define tt_assert_test_type(a,b,str_test,type,test,fmt) \ + tt_assert_test_fmt_type(a,b,str_test,type,test,type,fmt, \ + {_print=_value;},{}) + +/* Helper: assert that a op b, when cast to type. Format the values with + * printf format fmt on failure. */ +#define tt_assert_op_type(a,op,b,type,fmt) \ + tt_assert_test_type(a,b,#a" "#op" "#b,type,(_val1 op _val2),fmt) + +#define tt_int_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,long,(_val1 op _val2),"%ld") + +#define tt_uint_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,unsigned long, \ + (_val1 op _val2),"%lu") + +#define tt_ptr_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,void*, \ + (_val1 op _val2),"%p") + +#define tt_str_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,const char *, \ + (strcmp(_val1,_val2) op 0),"<%s>") + +#endif diff --git a/src/tools/Makefile.am b/src/tools/Makefile.am index 41786e437..1bb507684 100644 --- a/src/tools/Makefile.am +++ b/src/tools/Makefile.am @@ -3,16 +3,16 @@ noinst_PROGRAMS = tor-checkkey tor_resolve_SOURCES = tor-resolve.c tor_resolve_LDFLAGS = @TOR_LDFLAGS_libevent@ -tor_resolve_LDADD = ../common/libor.a @TOR_LIBEVENT_LIBS@ @TOR_LIB_WS32@ +tor_resolve_LDADD = ../common/libor.a -lm @TOR_LIBEVENT_LIBS@ @TOR_LIB_WS32@ tor_gencert_SOURCES = tor-gencert.c tor_gencert_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ @TOR_LDFLAGS_libevent@ tor_gencert_LDADD = ../common/libor.a ../common/libor-crypto.a \ - -lz @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + -lm @TOR_ZLIB_LIBS@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ tor_checkkey_SOURCES = tor-checkkey.c tor_checkkey_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ @TOR_LDFLAGS_libevent@ tor_checkkey_LDADD = ../common/libor.a ../common/libor-crypto.a \ - -lz @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + -lm @TOR_ZLIB_LIBS@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ diff --git a/src/tools/tor-checkkey.c b/src/tools/tor-checkkey.c index b29b52d8d..94c8cbd44 100644 --- a/src/tools/tor-checkkey.c +++ b/src/tools/tor-checkkey.c @@ -6,19 +6,21 @@ #include <stdio.h> #include <stdlib.h> #include "crypto.h" -#include "log.h" -#include "util.h" +#include "torlog.h" +#include "../common/util.h" #include "compat.h" #include <openssl/bn.h> #include <openssl/rsa.h> -int main(int c, char **v) +int +main(int c, char **v) { crypto_pk_env_t *env; char *str; RSA *rsa; int wantdigest=0; int fname_idx; + char *fname=NULL; init_logging(); if (c < 2) { @@ -29,7 +31,7 @@ int main(int c, char **v) return 1; } - if (crypto_global_init(0)) { + if (crypto_global_init(0, NULL, NULL)) { fprintf(stderr, "Couldn't initialize crypto library.\n"); return 1; } @@ -46,7 +48,9 @@ int main(int c, char **v) fname_idx = 1; } - str = read_file_to_str(v[fname_idx], 0, NULL); + fname = expand_filename(v[fname_idx]); + str = read_file_to_str(fname, 0, NULL); + tor_free(fname); if (!str) { fprintf(stderr, "Couldn't read %s\n", v[fname_idx]); return 1; diff --git a/src/tools/tor-gencert.c b/src/tools/tor-gencert.c index e6b09963b..a04eddafc 100644 --- a/src/tools/tor-gencert.c +++ b/src/tools/tor-gencert.c @@ -13,6 +13,7 @@ #include <openssl/evp.h> #include <openssl/pem.h> +#include <openssl/rsa.h> #include <openssl/objects.h> #include <openssl/obj_mac.h> #include <openssl/err.h> @@ -27,8 +28,8 @@ #define CRYPTO_PRIVATE #include "compat.h" -#include "util.h" -#include "log.h" +#include "../common/util.h" +#include "../common/torlog.h" #include "crypto.h" #include "address.h" @@ -63,7 +64,6 @@ show_help(void) "[-c certificate_file]\n" " [-m lifetime_in_months] [-a address:port] " "[--passphrase-fd <fd>]\n"); - } /* XXXX copied from crypto.c */ @@ -218,6 +218,20 @@ parse_commandline(int argc, char **argv) return 0; } +static RSA * +generate_key(int bits) +{ + RSA *rsa = NULL; + crypto_pk_env_t *env = crypto_new_pk_env(); + if (crypto_pk_generate_key_with_bits(env,bits)<0) + goto done; + rsa = _crypto_pk_env_get_rsa(env); + rsa = RSAPrivateKey_dup(rsa); + done: + crypto_free_pk_env(env); + return rsa; +} + /** Try to read the identity key from <b>identity_key_file</b>. If no such * file exists and create_identity_key is set, make a new identity key and * store it. Return 0 on success, nonzero on failure. @@ -238,7 +252,7 @@ load_identity_key(void) } log_notice(LD_GENERAL, "Generating %d-bit RSA identity key.", IDENTITY_KEY_BITS); - if (!(key = RSA_generate_key(IDENTITY_KEY_BITS, 65537, NULL, NULL))) { + if (!(key = generate_key(IDENTITY_KEY_BITS))) { log_err(LD_GENERAL, "Couldn't generate identity key."); crypto_log_errors(LOG_ERR, "Generating identity key"); return 1; @@ -323,7 +337,7 @@ generate_signing_key(void) RSA *key; log_notice(LD_GENERAL, "Generating %d-bit RSA signing key.", SIGNING_KEY_BITS); - if (!(key = RSA_generate_key(SIGNING_KEY_BITS, 65537, NULL, NULL))) { + if (!(key = generate_key(SIGNING_KEY_BITS))) { log_err(LD_GENERAL, "Couldn't generate signing key."); crypto_log_errors(LOG_ERR, "Generating signing key"); return 1; @@ -393,7 +407,6 @@ get_fingerprint(EVP_PKEY *pkey, char *out) return r; } - /** Set <b>out</b> to the hex-encoded fingerprint of <b>pkey</b>. */ static int get_digest(EVP_PKEY *pkey, char *out) @@ -487,7 +500,6 @@ generate_certificate(void) return 0; } - /** Entry point to tor-gencert */ int main(int argc, char **argv) @@ -496,7 +508,7 @@ main(int argc, char **argv) init_logging(); /* Don't bother using acceleration. */ - if (crypto_global_init(0)) { + if (crypto_global_init(0, NULL, NULL)) { fprintf(stderr, "Couldn't initialize crypto library.\n"); return 1; } diff --git a/src/tools/tor-resolve.c b/src/tools/tor-resolve.c index 4d9d57a31..12349d9d1 100644 --- a/src/tools/tor-resolve.c +++ b/src/tools/tor-resolve.c @@ -6,9 +6,9 @@ #include "orconfig.h" #include "compat.h" -#include "util.h" +#include "../common/util.h" #include "address.h" -#include "log.h" +#include "../common/torlog.h" #include <stdio.h> #include <stdlib.h> @@ -148,7 +148,7 @@ parse_socks4a_resolve_response(const char *hostname, static const char * socks5_reason_to_string(char reason) { - switch(reason) { + switch (reason) { case SOCKS5_SUCCEEDED: return "succeeded"; case SOCKS5_GENERAL_ERROR: @@ -251,7 +251,7 @@ do_resolve(const char *hostname, uint32_t sockshost, uint16_t socksport, } if (parse_socks4a_resolve_response(hostname, reply_buf, RESPONSE_LEN_4, - result_addr)<0){ + result_addr)<0) { return -1; } } else { diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index 65804fe19..b3cd1db50 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -88,11 +88,18 @@ #define HAVE_STRING_H /* Define to 1 if you have the `strlcat' function. */ +#if defined (WINCE) +#define HAVE_STRLCAT +#else #undef HAVE_STRLCAT +#endif /* Define to 1 if you have the `strlcpy' function. */ +#if defined (WINCE) +#define HAVE_STRLCPY +#else #undef HAVE_STRLCPY - +#endif /* Define to 1 if you have the `strptime' function. */ #undef HAVE_STRPTIME @@ -226,6 +233,5 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.2.1.25" - +#define VERSION "0.2.2.19-alpha" diff --git a/tor.spec.in b/tor.spec.in index 54b9be092..25557b73e 100644 --- a/tor.spec.in +++ b/tor.spec.in @@ -15,13 +15,6 @@ %define toruser @TORUSER@ %define torgroup @TORGROUP@ -## Target a specific arch and OS -# -# default is i386 linux - -## Override any system rpm macros -# - ## Version song and dance # # This should be the Tor version number, as it appears on the tarball, @@ -54,14 +47,11 @@ %if %{is_fc} %define ostag %(sed -e 's/^.*release /fc/' -e 's/ .*$//' -e 's/\\./_/g' < /etc/fedora-release) -%else +%endif + %if %{is_rh} %define ostag %(sed -e 's/^.*release /rh/' -e 's/ .*$//' -e 's/\\./_/g' < /etc/redhat-release) %endif -%endif - -# These are probably wrong... just placeholders should we actually -# end up supporting these distributions %if %{is_mdk} %define ostag mdk @@ -116,7 +106,7 @@ Group: System Environment/Daemons License: 3-clause BSD Vendor: The Tor Project (https://torproject.org) -Packager: Andrew Lewman <andrew@torproject.org> +Packager: Erinn Clark <erinn@torproject.org> %if %{is_suse} Requires: openssl >= 0.9.7 @@ -127,6 +117,7 @@ BuildRequires: openssl-devel >= 0.9.7 %endif %if %{is_fc} BuildRequires: rpm-build >= 4.0 +Conflicts: tor-core, tor-lsb, tor-upstart %endif Requires(pre): /usr/bin/id, /bin/date, /bin/sh Requires(pre): %{_sbindir}/useradd, %{_sbindir}/groupadd @@ -163,7 +154,7 @@ for high-stakes anonymity. %build %if %{is_suse} -%configure --with-tor-user=%{toruser} --with-tor-group=%{torgroup} +%configure --with-tor-user=%{toruser} --with-tor-group=%{torgroup} --docdir=%{_docdir}/%{name} %else %configure --with-tor-user=%{toruser} --with-tor-group=%{torgroup} %endif @@ -272,13 +263,20 @@ exit 0 %files %defattr(-,root,root) -%doc AUTHORS INSTALL LICENSE README ChangeLog doc/HACKING doc/TODO +%if %{is_suse} +%doc INSTALL LICENSE README ChangeLog doc/HACKING doc/TODO doc/*html +%endif +%doc INSTALL LICENSE README ChangeLog doc/HACKING doc/TODO %{_mandir}/man*/* %{_bindir}/tor %{_bindir}/torctl %{_bindir}/torify %{_bindir}/tor-resolve %{_bindir}/tor-gencert +%if %{is_suse} +%else +%{_docdir}/* +%endif %{_datadir}/tor/geoip %config %{_initrddir}/%{name} %config(noreplace) %attr(0644,root,root) %{_sysconfdir}/logrotate.d/%{name} @@ -289,6 +287,16 @@ exit 0 %attr(0750,%{toruser},%{torgroup}) %dir %{_localstatedir}/log/%{name} %changelog +* Fri Aug 20 2010 Erinn Clark <erinn@torproject.org> +- add conflicts for Fedora packages +- add logic for SuSE since it requires special doc handling + +* Mon Feb 22 2010 Erinn Clark <erinn@torproject.org> +- remove AUTHORS from %doc line since it no longer exists upstream +- switch maintainers + +* Fri May 01 2009 Andrew Lewman <andrew@torproject.org> +- clean up distro detection and remove dead comment blocks * Sun Feb 22 2009 Andrew Lewman <andrew@torproject.org> - update the description, vendor, and packager |
