diff options
156 files changed, 9158 insertions, 5992 deletions
diff --git a/.gitignore b/.gitignore index 3a28ecfd4..8034ae661 100644 --- a/.gitignore +++ b/.gitignore @@ -21,6 +21,7 @@ *.log # Autotools stuff .deps +.dirstamp # Stuff made by our makefiles *.bak @@ -41,6 +42,7 @@ /config.guess /config.sub /conftest* +/micro-revision.* /patch-stamp /stamp-h /stamp-h.in @@ -140,7 +142,6 @@ /src/or/Makefile /src/or/Makefile.in /src/or/or_sha1.i -/src/or/micro-revision.* /src/or/tor /src/or/tor.exe /src/or/libtor.a @@ -1,3 +1,418 @@ +Changes in version 0.2.3.22-rc - 2012-09-11 + Tor 0.2.3.22-rc fixes another opportunity for a remotely triggerable + assertion. + + o Security fixes: + - Fix an assertion failure in tor_timegm() that could be triggered + by a badly formatted directory object. Bug found by fuzzing with + Radamsa. Fixes bug 6811; bugfix on 0.2.0.20-rc. + + o Minor bugfixes: + - Avoid segfault when starting up having run with an extremely old + version of Tor and parsing its state file. Fixes bug 6801; bugfix + on 0.2.2.23-alpha. + + +Changes in version 0.2.2.39 - 2012-09-11 + Tor 0.2.2.39 fixes two more opportunities for remotely triggerable + assertions. + + o Security fixes: + - Fix an assertion failure in tor_timegm() that could be triggered + by a badly formatted directory object. Bug found by fuzzing with + Radamsa. Fixes bug 6811; bugfix on 0.2.0.20-rc. + - Do not crash when comparing an address with port value 0 to an + address policy. This bug could have been used to cause a remote + assertion failure by or against directory authorities, or to + allow some applications to crash clients. Fixes bug 6690; bugfix + on 0.2.1.10-alpha. + + +Changes in version 0.2.4.2-alpha - 2012-09-10 + Tor 0.2.4.2-alpha enables port forwarding for pluggable transports, + raises the default rate limiting even more, and makes the bootstrapping + log messages less noisy. + + o Major features: + - Automatically forward the TCP ports of pluggable transport + proxies using tor-fw-helper if PortForwarding is enabled. Implements + ticket 4567. + + o Major bugfixes: + - Raise the default BandwidthRate/BandwidthBurst values from 5MB/10MB + to 1GB/1GB. The previous defaults were intended to be "basically + infinite", but it turns out they're now limiting our 100mbit+ + relays and bridges. Fixes bug 6605; bugfix on 0.2.0.10-alpha (the + last time we raised it). + + o Minor features: + - Detect when we're running with a version of OpenSSL other than the + one we compiled with. This has occasionally given people hard-to- + track-down errors. + - Log fewer lines at level "notice" about our OpenSSL and Libevent + versions and capabilities when everything is going right. Resolves + part of ticket 6736. + - Directory authorities no long accept descriptors for any version of + Tor before 0.2.2.35, or for any 0.2.3 release before 0.2.3.10-alpha. + These versions are insecure, unsupported, or both. Implements + ticket 6789. + + o Minor bugfixes: + - Rename the (internal-use-only) UsingTestingNetworkDefaults option + to start with a triple-underscore so the controller won't touch it. + Patch by Meejah. Fixes bug 3155. Bugfix on 0.2.2.23-alpha. + - Avoid segfault when starting up having run with an extremely old + version of Tor and parsing its state file. Fixes bug 6801; bugfix + on 0.2.2.23-alpha. + - Rename the (testing-use-only) _UseFilteringSSLBufferevents option + so it doesn't start with _. Fixes bug 3155. Bugfix on 0.2.3.1-alpha. + - Don't follow the NULL pointer if microdescriptor generation fails. + (This does not appear to be triggerable, but it's best to be safe.) + Found by "f. tp.". Fixes bug 6797; bugfix on 0.2.4.1-alpha. + - Fix mis-declared dependencies on src/common/crypto.c and + src/or/tor_main.c that could break out-of-tree builds under some + circumstances. Fixes bug 6778; bugfix on 0.2.4.1-alpha. + - Avoid a warning when building common_sha1.i out of tree. Fixes bug + 6778; bugfix on 0.2.4.1-alpha. + - Fix a harmless (in this case) build warning for implicitly + converting a strlen() to an int. Bugfix on 0.2.4.1-alpha. + + o Removed features: + - Now that all versions before 0.2.2.x are disallowed, we no longer + need to work around their missing features. Thus we can remove a + bunch of compatibility code. + + o Code refactoring: + - Tweak tor-fw-helper to accept an arbitrary amount of arbitrary + TCP ports to forward. In the past it only accepted two ports: + the ORPort and the DirPort. + + +Changes in version 0.2.4.1-alpha - 2012-09-05 + Tor 0.2.4.1-alpha lets bridges publish their pluggable transports to + bridgedb; lets relays use IPv6 addresses and directory authorities + advertise them; and switches to a cleaner build interface. + + This is the first alpha release in a new series, so expect there to + be bugs. Users who would rather test out a more stable branch should + stay with 0.2.3.x for now. + + o Major features (bridges): + - Bridges now report the pluggable transports they support to the + bridge authority, so it can pass the supported transports on to + bridgedb and/or eventually do reachability testing. Implements + ticket 3589. + + o Major features (IPv6): + - Bridge authorities now accept IPv6 bridge addresses and include + them in network status documents. Implements ticket 5534. + - Clients who set "ClientUseIPv6 1" may connect to entry nodes over + IPv6. Set "ClientPreferIPv6ORPort 1" to make this even more likely + to happen. Implements ticket 5535. + - All kind of relays, not just bridges, can now advertise an IPv6 + OR port. Implements ticket 6362. + - Directory authorities vote on IPv6 OR ports using the new consensus + method 14. Implements ticket 6363. + + o Major features (build): + - Switch to a nonrecursive Makefile structure. Now instead of each + Makefile.am invoking other Makefile.am's, there is a master + Makefile.am that includes the others. This change makes our build + process slightly more maintainable, and improves parallelism for + building with make -j. Original patch by Stewart Smith; various + fixes by Jim Meyering. + - Where available, we now use automake's "silent" make rules by + default, so that warnings are easier to spot. You can get the old + behavior with "make V=1". Patch by Stewart Smith for ticket 6522. + + o Minor features (code security and spec conformance): + - Clear keys and key-derived material left on the stack in + rendservice.c and rendclient.c. Check return value of + crypto_pk_write_private_key_to_string() in end_service_load_keys(). + These fixes should make us more forward-secure against cold-boot + attacks and the like. Fixes bug 2385. + - Reject EXTEND cells sent to nonexistent streams. According to the + spec, an EXTEND cell sent to _any_ nonzero stream ID is invalid, but + we were only checking for stream IDs that were currently in use. + Found while hunting for more instances of bug 6271. Bugfix on + 0.0.2pre8, which introduced incremental circuit construction. + + o Minor features (streamlining); + - No longer include the "opt" prefix when generating routerinfos + or v2 directories: it has been needless since Tor 0.1.2. Closes + ticket 5124. + - Remove some now-needless code that tried to aggressively flush + OR connections as data was added to them. Since 0.2.0.1-alpha, our + cell queue logic has saved us from the failure mode that this code + was supposed to prevent. Removing this code will limit the number + of baroque control flow paths through Tor's network logic. Reported + pseudonymously on IRC. Fixes bug 6468; bugfix on 0.2.0.1-alpha. + + o Minor features (controller): + - Add a "GETINFO signal/names" control port command. Implements + ticket 3842. + - Provide default values for all options via "GETINFO config/defaults". + Implements ticket 4971. + + o Minor features (IPv6): + - New config option "AuthDirHasIPv6Connectivity 1" that directory + authorities should set if they have IPv6 connectivity and want to + do reachability tests for IPv6 relays. Implements feature 5974. + - A relay with an IPv6 OR port now sends that address in NETINFO + cells (in addition to its other address). Implements ticket 6364. + + o Minor features (log messages): + - Omit the first heartbeat log message, because it never has anything + useful to say, and it clutters up the bootstrapping messages. + Resolves ticket 6758. + - Don't log about reloading the microdescriptor cache at startup. Our + bootstrap warnings are supposed to tell the user when there's a + problem, and our bootstrap notices say when there isn't. Resolves + ticket 6759; bugfix on 0.2.2.6-alpha. + - Don't log "I learned some more directory information" when we're + reading cached directory information. Reserve it for when new + directory information arrives in response to a fetch. Resolves + ticket 6760. + - Prevent rounding error in path bias counts when scaling + them down, and use the correct scale factor default. Also demote + some path bias related log messages down a level and make others + less scary sounding. Fixes bug 6647. Bugfix against 0.2.3.17-beta. + - We no longer warn so much when generating manpages from their + asciidoc source. + + o Code simplifications and refactoring: + - Enhance our internal sscanf replacement so that we can eliminate + the last remaining uses of the system sscanf. (Though those uses + of sscanf were safe, sscanf itself is generally error prone, so + we want to eliminate when we can.) Fixes ticket 4195 and Coverity + CID 448. + - Move ipv6_preferred from routerinfo_t to node_t. Addresses bug 4620. + - Move last_reachable and testing_since from routerinfo_t to node_t. + Implements ticket 5529. + - Add replaycache_t structure, functions and unit tests, then refactor + rend_service_introduce() to be more clear to read, improve, debug, + and test. Resolves bug 6177. + - Finally remove support for malloc_good_size and malloc_usable_size. + We had hoped that these functions would let us eke a little more + memory out of our malloc implementation. Unfortunately, the only + implementations that provided these functions are also ones that + are already efficient about not overallocation: they never got us + more than 7 or so bytes per allocation. Removing them saves us a + little code complexity and a nontrivial amount of build complexity. + + o New requirements: + - Tor maintainers now require Automake version 1.9 or later to build + Tor from the Git repository. (Automake is not required when building + from a source distribution.) + + +Changes in version 0.2.3.21-rc - 2012-09-05 + Tor 0.2.3.21-rc is the fourth release candidate for the Tor 0.2.3.x + series. It fixes a trio of potential security bugs, fixes a bug where + we were leaving some of the fast relays out of the microdescriptor + consensus, resumes interpreting "ORPort 0" and "DirPort 0" correctly, + and cleans up other smaller issues. + + o Major bugfixes (security): + - Tear down the circuit if we get an unexpected SENDME cell. Clients + could use this trick to make their circuits receive cells faster + than our flow control would have allowed, or to gum up the network, + or possibly to do targeted memory denial-of-service attacks on + entry nodes. Fixes bug 6252. Bugfix on the 54th commit on Tor -- + from July 2002, before the release of Tor 0.0.0. We had committed + this patch previously, but we had to revert it because of bug 6271. + Now that 6271 is fixed, this patch appears to work. + - Reject any attempt to extend to an internal address. Without + this fix, a router could be used to probe addresses on an internal + network to see whether they were accepting connections. Fixes bug + 6710; bugfix on 0.0.8pre1. + - Do not crash when comparing an address with port value 0 to an + address policy. This bug could have been used to cause a remote + assertion failure by or against directory authorities, or to + allow some applications to crash clients. Fixes bug 6690; bugfix + on 0.2.1.10-alpha. + + o Major bugfixes: + - Remove the upper bound on microdescriptor length. We were hitting + the limit for routers with complex exit policies or family + declarations, causing clients to not use them. Fixes the first + piece of bug 6404; fix on 0.2.2.6-alpha. + - Detect "ORPort 0" as meaning, uniformly, that we're not running + as a relay. Previously, some of our code would treat the presence + of any ORPort line as meaning that we should act like a relay, + even though our new listener code would correctly not open any + ORPorts for ORPort 0. Similar bugs in other Port options are also + fixed. Fixes the first half of bug 6507; bugfix on 0.2.3.3-alpha. + + o Minor bugfixes: + - Avoid a pair of double-free and use-after-mark bugs that can + occur with certain timings in canceled and re-received DNS + requests. Fixes bug 6472; bugfix on 0.0.7rc1. + - Fix build and 64-bit compile warnings from --enable-openbsd-malloc. + Fixes bug 6379. Bugfix on 0.2.0.20-rc. + - Allow one-hop directory fetching circuits the full "circuit build + timeout" period, rather than just half of it, before failing them + and marking the relay down. This fix should help reduce cases where + clients declare relays (or worse, bridges) unreachable because + the TLS handshake takes a few seconds to complete. Fixes bug 6743; + bugfix on 0.2.2.2-alpha, where we changed the timeout from a static + 30 seconds. + - Authorities no longer include any router in their microdescriptor + consensuses for which they couldn't generate or agree on a + microdescriptor. Fixes the second piece of bug 6404; fix on + 0.2.2.6-alpha. + - Detect and reject attempts to specify both "FooPort" and + "FooPort 0" in the same configuration domain. (It's still okay + to have a FooPort in your configuration file, and use "FooPort 0" + on the command line to disable it.) Fixes the second half of bug + 6507; bugfix on 0.2.3.3-alpha. + - Make wildcarded addresses (that is, ones beginning with "*.") work + when provided via the controller's MapAddress command. Previously, + they were accepted, but we never actually noticed that they were + wildcards. Fixes bug 6244; bugfix on 0.2.3.9-alpha. + - Avoid crashing on a malformed state file where EntryGuardPathBias + precedes EntryGuard. Fix for bug 6774; bugfix on 0.2.3.17-beta. + - Add a (probably redundant) memory clear between iterations of + the router status voting loop, to prevent future coding errors + where data might leak between iterations of the loop. Resolves + ticket 6514. + + o Minor bugfixes (log messages): + - Downgrade "set buildtimeout to low value" messages to "info" + severity; they were never an actual problem, there was never + anything reasonable to do about them, and they tended to spam logs + from time to time. Fixes bug 6251; bugfix on 0.2.2.2-alpha. + - Downgrade path-bias warning messages to "info". We'll try to get + them working better in 0.2.4. Add internal circuit construction + state to protect against the noisy warn message "Unexpectedly high + circuit_successes". Also add some additional rate-limited notice + messages to help determine the root cause of the warn. Fixes bug + 6475. Bugfix against 0.2.3.17-beta. + - Move log message when unable to find a microdesc in a routerstatus + entry to parse time. Previously we'd spam this warning every time + we tried to figure out which microdescriptors to download. Fixes + the third piece of bug 6404; fix on 0.2.3.18-rc. + + o Minor features: + - Consider new, removed or changed IPv6 OR ports a non-cosmetic + change when the authority is deciding whether to accept a newly + uploaded descriptor. Implements ticket 6423. + - Add missing documentation for consensus and microdesc files. + Resolves ticket 6732. + + +Changes in version 0.2.2.38 - 2012-08-12 + Tor 0.2.2.38 fixes a remotely triggerable crash bug, and fixes a timing + attack that could in theory leak path information. + + o Security fixes: + - Avoid an uninitialized memory read when reading a vote or consensus + document that has an unrecognized flavor name. This read could + lead to a remote crash bug. Fixes bug 6530; bugfix on 0.2.2.6-alpha. + - Try to leak less information about what relays a client is + choosing to a side-channel attacker. Previously, a Tor client would + stop iterating through the list of available relays as soon as it + had chosen one, thus finishing a little earlier when it picked + a router earlier in the list. If an attacker can recover this + timing information (nontrivial but not proven to be impossible), + they could learn some coarse-grained information about which relays + a client was picking (middle nodes in particular are likelier to + be affected than exits). The timing attack might be mitigated by + other factors (see bug 6537 for some discussion), but it's best + not to take chances. Fixes bug 6537; bugfix on 0.0.8rc1. + + +Changes in version 0.2.3.20-rc - 2012-08-05 + Tor 0.2.3.20-rc is the third release candidate for the Tor 0.2.3.x + series. It fixes a pair of code security bugs and a potential anonymity + issue, updates our RPM spec files, and cleans up other smaller issues. + + o Security fixes: + - Avoid read-from-freed-memory and double-free bugs that could occur + when a DNS request fails while launching it. Fixes bug 6480; + bugfix on 0.2.0.1-alpha. + - Avoid an uninitialized memory read when reading a vote or consensus + document that has an unrecognized flavor name. This read could + lead to a remote crash bug. Fixes bug 6530; bugfix on 0.2.2.6-alpha. + - Try to leak less information about what relays a client is + choosing to a side-channel attacker. Previously, a Tor client would + stop iterating through the list of available relays as soon as it + had chosen one, thus finishing a little earlier when it picked + a router earlier in the list. If an attacker can recover this + timing information (nontrivial but not proven to be impossible), + they could learn some coarse-grained information about which relays + a client was picking (middle nodes in particular are likelier to + be affected than exits). The timing attack might be mitigated by + other factors (see bug 6537 for some discussion), but it's best + not to take chances. Fixes bug 6537; bugfix on 0.0.8rc1. + + o Minor features: + - Try to make the warning when giving an obsolete SOCKSListenAddress + a little more useful. + - Terminate active server managed proxies if Tor stops being a + relay. Addresses parts of bug 6274; bugfix on 0.2.3.6-alpha. + - Provide a better error message about possible OSX Asciidoc failure + reasons. Fixes bug 6436. + - Warn when Tor is configured to use accounting in a way that can + link a hidden service to some other hidden service or public + address. Resolves ticket 6490. + + o Minor bugfixes: + - Check return value of fputs() when writing authority certificate + file. Fixes Coverity issue 709056; bugfix on 0.2.0.1-alpha. + - Ignore ServerTransportPlugin lines when Tor is not configured as + a relay. Fixes bug 6274; bugfix on 0.2.3.6-alpha. + - When disabling guards for having too high a proportion of failed + circuits, make sure to look at each guard. Fixes bug 6397; bugfix + on 0.2.3.17-beta. + + o Packaging (RPM): + - Update our default RPM spec files to work with mock and rpmbuild + on RHEL/Fedora. They have an updated set of dependencies and + conflicts, a fix for an ancient typo when creating the "_tor" + user, and better instructions. Thanks to Ondrej Mikle for the + patch series. Fixes bug 6043. + + o Testing: + - Make it possible to set the TestingTorNetwork configuration + option using AlternateDirAuthority and AlternateBridgeAuthority + as an alternative to setting DirServer. Addresses ticket 6377. + + o Documentation: + - Clarify the documentation for the Alternate*Authority options. + Fixes bug 6387. + - Fix some typos in the manpages. Patch from A. Costa. Fixes bug 6500. + + o Code simplification and refactoring: + - Do not use SMARTLIST_FOREACH for any loop whose body exceeds + 10 lines. Also, don't nest them. Doing so in the past has + led to hard-to-debug code. The new style is to use the + SMARTLIST_FOREACH_{BEGIN,END} pair. Addresses issue 6400. + + +Changes in version 0.2.3.19-rc - 2012-07-06 + Tor 0.2.3.19-rc is the second release candidate for the Tor 0.2.3.x + series. It fixes the compile on Windows, reverts to a GeoIP database + that isn't as broken, and fixes a flow control bug that has been around + since the beginning of Tor. + + o Major bugfixes: + - Fix a bug handling SENDME cells on nonexistent streams that could + result in bizarre window values. Report and patch contributed + pseudonymously. Fixes part of bug 6271. This bug was introduced + before the first Tor release, in svn commit r152. + - Revert to the May 1 2012 Maxmind GeoLite Country database. In the + June 2012 database, Maxmind marked many Tor relays as country "A1", + which will cause risky behavior for clients that set EntryNodes + or ExitNodes. Addresses bug 6334; bugfix on 0.2.3.17-beta. + - Instead of ENOBUFS on Windows, say WSAENOBUFS. Fixes compilation + on Windows. Fixes bug 6296; bugfix on 0.2.3.18-rc. + + o Minor bugfixes: + - Fix wrong TCP port range in parse_port_range(). Fixes bug 6218; + bugfix on 0.2.1.10-alpha. + + Changes in version 0.2.3.18-rc - 2012-06-28 Tor 0.2.3.18-rc is the first release candidate for the Tor 0.2.3.x series. It fixes a few smaller bugs, but generally appears stable. @@ -225,7 +640,7 @@ Changes in version 0.2.3.16-alpha - 2012-06-05 indefinitely. Fixes bug 5380; bugfix on 0.2.1.14-rc. - When fetching a bridge descriptor from a bridge authority, always do so anonymously, whether we have been able to open - circuits or not. Partial fix for bug 1938; bugfix on 2.0.7-alpha. + circuits or not. Partial fix for bug 1938; bugfix on 0.2.0.7-alpha. This behavior makes it *safer* to use UpdateBridgesFromAuthority, but we'll need to wait for bug 6010 before it's actually usable. @@ -955,7 +1370,8 @@ Changes in version 0.2.3.11-alpha - 2012-01-22 FastFlagMaxThreshold) to control the range of allowable bandwidths for the Fast directory flag. These allow authorities to run experiments on appropriate requirements for being a "Fast" node. - The AuthDirFastGuarantee config value still applies. + The AuthDirFastGuarantee config value still applies. Implements + ticket 3946. - Document the GiveGuardFlagTo_CVE_2011_2768_VulnerableRelays directory authority option (introduced in Tor 0.2.2.34). @@ -1270,6 +1686,29 @@ Changes in version 0.2.2.35 - 2011-12-16 by removing an absolute path from makensis.exe command. +Changes in version 0.2.1.32 - 2011-12-16 + Tor 0.2.1.32 backports important security and privacy fixes for + oldstable. This release is intended only for package maintainers and + others who cannot use the 0.2.2 stable series. All others should be + using Tor 0.2.2.x or newer. + + The Tor 0.2.1.x series will reach formal end-of-life some time in + early 2012; we will stop releasing patches for it then. + + o Major bugfixes (also included in 0.2.2.x): + - Correctly sanity-check that we don't underflow on a memory + allocation (and then assert) for hidden service introduction + point decryption. Bug discovered by Dan Rosenberg. Fixes bug 4410; + bugfix on 0.2.1.5-alpha. + - Fix a heap overflow bug that could occur when trying to pull + data into the first chunk of a buffer, when that chunk had + already had some data drained from it. Fixes CVE-2011-2778; + bugfix on 0.2.0.16-alpha. Reported by "Vektor". + + o Minor features: + - Update to the December 6 2011 Maxmind GeoLite Country database. + + Changes in version 0.2.3.9-alpha - 2011-12-08 Tor 0.2.3.9-alpha introduces initial IPv6 support for bridges, adds a "DisableNetwork" security feature that bundles can use to avoid diff --git a/Makefile.am b/Makefile.am index 29bba715b..787be4951 100644 --- a/Makefile.am +++ b/Makefile.am @@ -4,22 +4,31 @@ # See LICENSE for licensing information # "foreign" means we don't follow GNU package layout standards -# 1.7 means we require automake vesion 1.7 -AUTOMAKE_OPTIONS = foreign 1.7 +# 1.9 means we require automake vesion 1.9 +AUTOMAKE_OPTIONS = foreign 1.9 subdir-objects -SUBDIRS = src doc contrib +noinst_LIBRARIES= +EXTRA_DIST= +noinst_HEADERS= +bin_PROGRAMS= +CLEANFILES= +TESTS= +noinst_PROGRAMS= +DISTCLEANFILES= +bin_SCRIPTS= +AM_CPPFLAGS= +include src/include.am +include doc/include.am +include contrib/include.am -DIST_SUBDIRS = src doc contrib -EXTRA_DIST = \ +EXTRA_DIST+= \ ChangeLog \ INSTALL \ LICENSE \ Makefile.nmake \ README \ - ReleaseNotes \ - tor.spec \ - tor.spec.in + ReleaseNotes #install-data-local: # $(INSTALL) -m 755 -d $(LOCALSTATEDIR)/lib/tor diff --git a/ReleaseNotes b/ReleaseNotes index 93e38faf3..9e02374ba 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -3,6 +3,42 @@ This document summarizes new features and bugfixes in each stable release of Tor. If you want to see more detailed descriptions of the changes in each development snapshot, see the ChangeLog file. +Changes in version 0.2.2.39 - 2012-09-11 + Tor 0.2.2.39 fixes two more opportunities for remotely triggerable + assertions. + + o Security fixes: + - Fix an assertion failure in tor_timegm() that could be triggered + by a badly formatted directory object. Bug found by fuzzing with + Radamsa. Fixes bug 6811; bugfix on 0.2.0.20-rc. + - Do not crash when comparing an address with port value 0 to an + address policy. This bug could have been used to cause a remote + assertion failure by or against directory authorities, or to + allow some applications to crash clients. Fixes bug 6690; bugfix + on 0.2.1.10-alpha. + + +Changes in version 0.2.2.38 - 2012-08-12 + Tor 0.2.2.38 fixes a remotely triggerable crash bug, and fixes a timing + attack that could in theory leak path information. + + o Security fixes: + - Avoid an uninitialized memory read when reading a vote or consensus + document that has an unrecognized flavor name. This read could + lead to a remote crash bug. Fixes bug 6530; bugfix on 0.2.2.6-alpha. + - Try to leak less information about what relays a client is + choosing to a side-channel attacker. Previously, a Tor client would + stop iterating through the list of available relays as soon as it + had chosen one, thus finishing a little earlier when it picked + a router earlier in the list. If an attacker can recover this + timing information (nontrivial but not proven to be impossible), + they could learn some coarse-grained information about which relays + a client was picking (middle nodes in particular are likelier to + be affected than exits). The timing attack might be mitigated by + other factors (see bug 6537 for some discussion), but it's best + not to take chances. Fixes bug 6537; bugfix on 0.0.8rc1. + + Changes in version 0.2.2.37 - 2012-06-06 Tor 0.2.2.37 introduces a workaround for a critical renegotiation bug in OpenSSL 1.0.1 (where 20% of the Tor network can't talk to itself @@ -294,6 +330,29 @@ Changes in version 0.2.2.35 - 2011-12-16 by removing an absolute path from makensis.exe command. +Changes in version 0.2.1.32 - 2011-12-16 + Tor 0.2.1.32 backports important security and privacy fixes for + oldstable. This release is intended only for package maintainers and + others who cannot use the 0.2.2 stable series. All others should be + using Tor 0.2.2.x or newer. + + The Tor 0.2.1.x series will reach formal end-of-life some time in + early 2012; we will stop releasing patches for it then. + + o Major bugfixes (also included in 0.2.2.x): + - Correctly sanity-check that we don't underflow on a memory + allocation (and then assert) for hidden service introduction + point decryption. Bug discovered by Dan Rosenberg. Fixes bug 4410; + bugfix on 0.2.1.5-alpha. + - Fix a heap overflow bug that could occur when trying to pull + data into the first chunk of a buffer, when that chunk had + already had some data drained from it. Fixes CVE-2011-2778; + bugfix on 0.2.0.16-alpha. Reported by "Vektor". + + o Minor features: + - Update to the December 6 2011 Maxmind GeoLite Country database. + + Changes in version 0.2.2.34 - 2011-10-26 Tor 0.2.2.34 fixes a critical anonymity vulnerability where an attacker can deanonymize Tor users. Everybody should upgrade. diff --git a/acinclude.m4 b/acinclude.m4 index 9ab684877..5ea3097ce 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -1,4 +1,4 @@ -dnl Helper macros for Tor configure.in +dnl Helper macros for Tor configure.ac dnl Copyright (c) 2001-2004, Roger Dingledine dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson dnl Copyright (c) 2007-2008, Roger Dingledine, Nick Mathewson diff --git a/changes/bug6043 b/changes/bug6043 deleted file mode 100644 index b88bafb78..000000000 --- a/changes/bug6043 +++ /dev/null @@ -1,6 +0,0 @@ - o Packaging (RPM): - - Our default RPM spec files have been updated to work with mock - and rpmbuild on RHEL/Fedora. They have an updated set of - dependencies and conflicts, a fix for an ancient typo when creating - the "_tor" user, and better instructions. Thanks to Ondrej - Mikle for the patch series; fix for bug 6043. diff --git a/changes/bug6218 b/changes/bug6218 deleted file mode 100644 index 5d5d108b0..000000000 --- a/changes/bug6218 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - Fix wrong TCP port range in parse_port_range(). Fixes bug 6218; - bugfix on 0.2.1.10-alpha. diff --git a/changes/bug6244_part_c b/changes/bug6244_part_c deleted file mode 100644 index dea6e7b69..000000000 --- a/changes/bug6244_part_c +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (controller): - - Make wildcarded addresses (that is, ones beginning with *.) work when - provided via the controller's MapAddress command. Previously, they - were accepted, but we never actually noticed that they were wildcards. - Fix for bug 6244; bugfix on 0.2.3.9-alpha. - diff --git a/changes/bug6251 b/changes/bug6251 deleted file mode 100644 index c782a93e4..000000000 --- a/changes/bug6251 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - Downgrade "set buildtimeout to low value" messages to INFO - severity; they were never an actual problem, there was never - anything reasonable to do about them, and they tended to spam - logs from time to time. Fix for bug 6251; bugfix on - 0.2.2.2-alpha.
\ No newline at end of file diff --git a/changes/bug6252_again b/changes/bug6252_again deleted file mode 100644 index f7fd00cb3..000000000 --- a/changes/bug6252_again +++ /dev/null @@ -1,11 +0,0 @@ - o Security fixes: - - Tear down the circuit if we get an unexpected SENDME cell. Clients - could use this trick to make their circuits receive cells faster - than our flow control would have allowed, or to gum up the network, - or possibly to do targeted memory denial-of-service attacks on - entry nodes. Fixes bug 6252. Bugfix on the 54th commit on Tor -- - from July 2002, before the release of Tor 0.0.0. We had committed - this patch previously, but we had to revert it because of bug 6271. - Now that 6271 is fixed, this appears to work. - - diff --git a/changes/bug6271 b/changes/bug6271 deleted file mode 100644 index 06b129f73..000000000 --- a/changes/bug6271 +++ /dev/null @@ -1,7 +0,0 @@ - o Major bugfixes - - - Fix a bug handling SENDME cells on nonexistent streams that - could result in bizarre window values. Report and patch - contributed pseudymously. Fixes part of bug 6271. This bug - was introduced before the first Tor release, in svn commit - r152. diff --git a/changes/bug6274 b/changes/bug6274 deleted file mode 100644 index ad1abcde5..000000000 --- a/changes/bug6274 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes: - - Ignore ServerTransportPlugin lines when Tor is not configured as - a relay. Fixes bug 6274; bugfix on 0.2.3.6-alpha. diff --git a/changes/bug6274_2 b/changes/bug6274_2 deleted file mode 100644 index 89576f932..000000000 --- a/changes/bug6274_2 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Terminate active server managed proxies if Tor stops being a - relay. Addresses parts of bug 6274; bugfix on 0.2.3.6-alpha. diff --git a/changes/bug6296 b/changes/bug6296 deleted file mode 100644 index b452b1745..000000000 --- a/changes/bug6296 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - - Instead of ENOBUFS on Windows, say WSAENOBUFS. Fixes - compilation on Windows. Fixes bug 6296; bugfix on 0.2.3.18-rc. diff --git a/changes/bug6377 b/changes/bug6377 deleted file mode 100644 index a3a367278..000000000 --- a/changes/bug6377 +++ /dev/null @@ -1,4 +0,0 @@ - o Testing: - - Make it possible to set the TestingTorNetwork configuration - option using AlternateDirAuthority and AlternateBridgeAuthority - as an alternative to setting DirServer. diff --git a/changes/bug6379 b/changes/bug6379 deleted file mode 100644 index 1f2b6941c..000000000 --- a/changes/bug6379 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - Fix build warnings from --enable-openbsd-malloc with gcc warnings - enabled. Fixes bug 6379. - - Fix 64-bit warnings from --enable-openbsd-malloc. Fixes bug 6379. - Bugfix on 0.2.0.20-rc. - diff --git a/changes/bug6387 b/changes/bug6387 deleted file mode 100644 index 73fc4f7cf..000000000 --- a/changes/bug6387 +++ /dev/null @@ -1,3 +0,0 @@ - o Documentation: - - Clarify the documentation for the Alternate*Authority options. - Fixes bug 6387. diff --git a/changes/bug6397 b/changes/bug6397 deleted file mode 100644 index 23d8359bd..000000000 --- a/changes/bug6397 +++ /dev/null @@ -1,4 +0,0 @@ - o Major bugfixes: - - When disabling guards for having too high a proportion of failed - circuits, make sure to look at each guard. Fix for bug 6397; bugfix - on 0.2.3.17-beta. diff --git a/changes/bug6404 b/changes/bug6404 deleted file mode 100644 index 948f00b92..000000000 --- a/changes/bug6404 +++ /dev/null @@ -1,16 +0,0 @@ - o Minor bugfixes: - - - Remove the maximum length of microdescriptor we are willing to - generate. Occasionally this is needed for routers - with complex policies or family declarations. Partial fix for - bug 6404; fix on 0.2.2.6-alpha. - - - Authorities no longer include any router in their - microdescriptor consensuses for which they couldn't generate or - agree on a microdescriptor. Partial fix for bug 6404; fix on - 0.2.2.6-alpha. - - - Move log message when unable to find a microdesc in a - routerstatus entry to parse time. Previously we'd spam this - warning every time we tried to figure out which microdescriptors - to download. Partial fix for bug 6404; fix on 0.2.3.18-rc. diff --git a/changes/bug6423 b/changes/bug6423 deleted file mode 100644 index 2ea4f1410..000000000 --- a/changes/bug6423 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Consider new, removed or changed IPv6 OR ports a non cosmetic - change. diff --git a/changes/bug6436 b/changes/bug6436 deleted file mode 100644 index 2c163df10..000000000 --- a/changes/bug6436 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features: - - Provide a better error message about possible OSX Asciidoc failure - reasons. Fix for bug 6436. diff --git a/changes/bug6472 b/changes/bug6472 deleted file mode 100644 index dcd42ebe6..000000000 --- a/changes/bug6472 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Avoid a pair of double-free and use-after-mark bugs that can - occur with certain timings in canceled and re-received DNS - requests. Fix for bug 6472; bugfix on 0.0.7rc1. diff --git a/changes/bug6475 b/changes/bug6475 deleted file mode 100644 index 67bab9962..000000000 --- a/changes/bug6475 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes: - - Add internal circuit construction state to protect against - the noisy warn message "Unexpectedly high circuit_successes". - Also add some additional rate-limited notice messages to help - determine the root cause of the warn. Fixes bug 6475. - Bugfix against 0.2.3.17-beta. diff --git a/changes/bug6480 b/changes/bug6480 deleted file mode 100644 index 83ae00b25..000000000 --- a/changes/bug6480 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes: - - Avoid read-from-freed-RAM bug and related double-free bug that - could occur when a DNS request fails while launching it. Fixes - bug 6480; bugfix on 0.2.0.1-alpha. - diff --git a/changes/bug6490 b/changes/bug6490 deleted file mode 100644 index c92daad8f..000000000 --- a/changes/bug6490 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features: - - Warn when Tor is configured to use accounting in a way that will - link a hidden service to some other hidden service or public - address. Fix for bug 6490. diff --git a/changes/bug6500 b/changes/bug6500 deleted file mode 100644 index cac2054a3..000000000 --- a/changes/bug6500 +++ /dev/null @@ -1,2 +0,0 @@ - o Minor bugfixes: - - Fix some typos in the manpages. Patch from A. Costa. Fixes bug 6500. diff --git a/changes/bug6507 b/changes/bug6507 deleted file mode 100644 index 89940cbf7..000000000 --- a/changes/bug6507 +++ /dev/null @@ -1,15 +0,0 @@ - o Major bugfixes: - - Detect 'ORPort 0' as meaning, uniformly, that we're not running - as a server. Previously, some of our code would treat the - presence of any ORPort line as meaning that we should act like a - server, even though our new listener code would correctly not - open any ORPorts for ORPort 0. Similar bugs in other Port - options are also fixed. Fixes bug 6507; bugfix on 0.2.3.3-alpha. - - o Minor features: - - - Detect and reject attempts to specify both 'FooPort' and - 'FooPort 0' in the same configuration domain. (It's still okay - to have a FooPort in your configuration file,and use 'FooPort 0' - on the command line to disable it.) Fixes another case of - bug6507; bugfix on 0.2.3.3-alpha. diff --git a/changes/bug6514 b/changes/bug6514 deleted file mode 100644 index 84633bd27..000000000 --- a/changes/bug6514 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Add a (probably redundant) memory clear between iterations of - the router status voting loop, to prevent future coding errors - where data might leak between iterations of the loop. Resolves - ticket 6514. diff --git a/changes/bug6530 b/changes/bug6530 deleted file mode 100644 index 825bbb752..000000000 --- a/changes/bug6530 +++ /dev/null @@ -1,5 +0,0 @@ - o Major security fixes: - - Avoid a read of uninitializd RAM when reading a vote or consensus - document with an unrecognized flavor name. This could lead to a - remote crash bug. Fixes bug 6530; bugfix on 0.2.2.6-alpha. - diff --git a/changes/bug6538 b/changes/bug6538 new file mode 100644 index 000000000..03c168b60 --- /dev/null +++ b/changes/bug6538 @@ -0,0 +1,16 @@ + o Minor bugfixes: + - Switch weighted node selection rule from using a list of doubles + to using a list of int64_t. This should make the process slightly + easier to debug and maintain. Needed for fix for bug 6538. + + o Security features: + - Switch to a completely time-invariant approach for picking nodes + weighted by bandwidth. Our old approach would run through the + part of the loop after it had made its choice slightly slower + than it ran through the part of the loop before it had made its + choice. Fix for bug 6538. + + o Code simplifications and refactoring: + - Move the core of our "choose a weighted element at random" logic + into its own function, and give it unit tests. Now the logic is + testable, and a little less fragile too. diff --git a/changes/bug6690 b/changes/bug6690 deleted file mode 100644 index 99d42976e..000000000 --- a/changes/bug6690 +++ /dev/null @@ -1,7 +0,0 @@ - o Major bugfixes (security): - - Do not crash when comparing an address with port value 0 to an - address policy. This bug could have been used to cause a remote - assertion failure by or against directory authorities, or to - allow some applications to crash clients. Fixes bug 6690; bugfix - on 0.2.1.10-alpha. - diff --git a/changes/bug6710 b/changes/bug6710 deleted file mode 100644 index 2c8934611..000000000 --- a/changes/bug6710 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (security): - - Reject any attempt to extend to an internal address. Without - this fix, a router could be used to probe addresses on an - internal network to see whether they were accepting - connections. Fix for bug 6710; bugfix on 0.0.8pre1. - diff --git a/changes/bug6732 b/changes/bug6732 deleted file mode 100644 index 7a744e014..000000000 --- a/changes/bug6732 +++ /dev/null @@ -1,3 +0,0 @@ - o Documentation: - - Add missing documentation for consensus and microdesc files. Fix for - bug 6732. diff --git a/changes/bug6743 b/changes/bug6743 deleted file mode 100644 index 6ec78f853..000000000 --- a/changes/bug6743 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes: - - Allow one-hop directory fetching circuits the full "circuit build - timeout" period, rather than just half of it, before failing them - and marking the relay down. This fix should help reduce cases where - clients declare relays (or worse, bridges) unreachable because - the TLS handshake takes a few seconds to complete. Fixes bug 6743; - bugfix on 0.2.2.2-alpha, where we changed the timeout from a static - 30 seconds. - diff --git a/changes/bug6774 b/changes/bug6774 deleted file mode 100644 index 0c137fd67..000000000 --- a/changes/bug6774 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Avoid crashing on a malformed state file where EntryGuardPathBias - precedes EntryGuard. Fix for bug 6774; bugfix on 0.2.3.17-beta. - diff --git a/changes/bug6801 b/changes/bug6801 deleted file mode 100644 index ef21acc98..000000000 --- a/changes/bug6801 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - Avoid segfault when starting up having run with an extremely old - version of Tor and parsing its state file. Fixes bug 6801; bugfix on - 0.2.2.23-alpha. - diff --git a/changes/bug6815 b/changes/bug6815 new file mode 100644 index 000000000..d6a1233a4 --- /dev/null +++ b/changes/bug6815 @@ -0,0 +1,6 @@ + o Major bugfixes: + - Allow routers to correctly detect their own DirPorts as running. + When we removed support for versions_supports_begindir, we also + accidentally removed the mechanism we used to self-test our + DirPort. Diagnosed with help from kargig. Fixes bugs 6814 and + 6815; bugfix on 0.2.4.2-alpha. diff --git a/changes/bug6819 b/changes/bug6819 new file mode 100644 index 000000000..8b5c2243e --- /dev/null +++ b/changes/bug6819 @@ -0,0 +1,3 @@ + o Documentation fixes: + - Resolve a typo in torrc.sample.in. Fixes bug 6819; bugfix on + 0.2.3.14-alpha. diff --git a/changes/bug6831 b/changes/bug6831 new file mode 100644 index 000000000..ac4775ba8 --- /dev/null +++ b/changes/bug6831 @@ -0,0 +1,4 @@ + o Minor bugfixes: + - Fix round_to_power_of_2 so it doesn't invoke undefined behavior + with large values. This was untriggered, but nevertheless incorrect. + Fixes bug 6831; bugfix on 0.2.0.1-alpha. diff --git a/changes/bug6833 b/changes/bug6833 new file mode 100644 index 000000000..4a6a5d3bb --- /dev/null +++ b/changes/bug6833 @@ -0,0 +1,4 @@ + o Minor bugfixes (directory authority): + - Reject consensus votes with more than 64 known-flags. We aren't even + close to that limit yet, and our code doesn't handle it + correctly. Fixes bug 6833; bugfix on 0.2.0.1-alpha. diff --git a/changes/cov709056 b/changes/cov709056 deleted file mode 100644 index 64a75ad8a..000000000 --- a/changes/cov709056 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes: - - Check return value of fputs() when writing authority certificate - file. Fixes Coverity issue 709056; bugfix on 0.2.0.1-alpha. - diff --git a/changes/disable_pathbias_messages b/changes/disable_pathbias_messages deleted file mode 100644 index 3bc996347..000000000 --- a/changes/disable_pathbias_messages +++ /dev/null @@ -1,3 +0,0 @@ - o Disabeled features - - Downgrade path-bias warning messages to INFO. We'll try to get them - working better in 0.2.4. Fixes bug 6475; bugfix on 0.2.3.17-beta. diff --git a/changes/old_gcc_macro_fix b/changes/old_gcc_macro_fix new file mode 100644 index 000000000..5fc0a47c6 --- /dev/null +++ b/changes/old_gcc_macro_fix @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix building with older versions of GCC that don't like preprocessor + directives inside macro arguments. Found by grarpamp. Fixes bug 6842; + bugfix on 0.2.4.2-alpha. + diff --git a/changes/pathsel-BUGGY-a b/changes/pathsel-BUGGY-a deleted file mode 100644 index 2e642c795..000000000 --- a/changes/pathsel-BUGGY-a +++ /dev/null @@ -1,14 +0,0 @@ - o Security fixes: - - - Try to leak less information about what relays a client is - choosing to a side-channel attacker. Previously, a Tor client - would stop iterating through the list of available relays as - soon as it had chosen one, thus finishing a little earlier - when it picked a router earlier in the list. If an attacker - can recover this timing information (nontrivial but not - proven to be impossible), they could learn some coarse- - grained information about which relays a client was picking - (middle nodes in particular are likelier to be affected than - exits). The timing attack might be mitigated by other factors - (see bug #6537 for some discussion), but it's best not to - take chances. Fixes bug 6537; bugfix on 0.0.8rc1. diff --git a/changes/port_doc b/changes/port_doc deleted file mode 100644 index 0e8662f0a..000000000 --- a/changes/port_doc +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features (usability): - - Try to make the warning when giving an obsolete SOCKSListenAddress - a littel more useful. diff --git a/changes/refactor_config b/changes/refactor_config new file mode 100644 index 000000000..8d272e753 --- /dev/null +++ b/changes/refactor_config @@ -0,0 +1,3 @@ + o Code simplification and refactoring: + - Move the generic "config" code into a new file, and have "config.c" + hold only torrc- and state-related code. diff --git a/changes/revert-geoip-may2012 b/changes/revert-geoip-may2012 deleted file mode 100644 index e420947a3..000000000 --- a/changes/revert-geoip-may2012 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes: - - Revert to the May 1 2012 Maxmind GeoLite Country database. In the - June 2012 database, Maxmind marked many Tor relays as country "A1", - which will cause risky behavior for clients that set EntryNodes - or ExitNodes. Addresses bug 6334; bugfix on 0.2.3.17-beta. - diff --git a/changes/smartlist_foreach b/changes/smartlist_foreach deleted file mode 100644 index 2fd3a1a85..000000000 --- a/changes/smartlist_foreach +++ /dev/null @@ -1,8 +0,0 @@ - o Code simplification and refactoring: - - Do not use SMARTLIST_FOREACH for any loop whose body exceeds - 10 lines. Doing so in the past has led to hard-to-debug code. - The new style is to use the SMARTLIST_FOREACH_{BEGIN,END} pair. - Issue 6400. - - Do not nest SMARTLIST_FOREACH blocks within one another. Any - nested block ought to be using SMARTLIST_FOREACH_{BEGIN,END}. - Issue 6400. diff --git a/configure.in b/configure.ac index 8aa90f641..7f3382866 100644 --- a/configure.in +++ b/configure.ac @@ -3,9 +3,11 @@ dnl Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson dnl Copyright (c) 2007-2012, The Tor Project, Inc. dnl See LICENSE for licensing information -AC_INIT -AM_INIT_AUTOMAKE(tor, 0.2.3.18-rc-dev) -AM_CONFIG_HEADER(orconfig.h) +AC_INIT([tor],[0.2.4.2-alpha-dev]) +AC_CONFIG_SRCDIR([src/or/main.c]) +AM_INIT_AUTOMAKE +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) +AC_CONFIG_HEADERS([orconfig.h]) AC_CANONICAL_HOST @@ -301,6 +303,7 @@ AC_CHECK_FUNCS( inet_aton \ ioctl \ issetugid \ + llround \ localtime_r \ lround \ memmem \ @@ -317,18 +320,6 @@ AC_CHECK_FUNCS( vasprintf \ ) -using_custom_malloc=no -if test x$enable_openbsd_malloc = xyes ; then - AC_DEFINE(HAVE_MALLOC_GOOD_SIZE, 1, [Defined if we have the malloc_good_size function]) - using_custom_malloc=yes -fi -if test x$tcmalloc = xyes ; then - using_custom_malloc=yes -fi -if test $using_custom_malloc = no ; then - AC_CHECK_FUNCS(mallinfo malloc_good_size malloc_usable_size) -fi - if test "$enable_threads" = "yes"; then AC_CHECK_HEADERS(pthread.h) AC_CHECK_FUNCS(pthread_create) @@ -703,14 +694,6 @@ AC_CHECK_HEADERS( AC_CHECK_HEADERS(sys/param.h) -TOR_CHECK_PROTOTYPE(malloc_good_size, HAVE_MALLOC_GOOD_SIZE_PROTOTYPE, -[#ifdef HAVE_MALLOC_H -#include <malloc.h> -#endif -#ifdef HAVE_MALLOC_MALLOC_H -#include <malloc/malloc.h> -#endif]) - AC_CHECK_HEADERS(net/if.h, net_if_found=1, net_if_found=0, [#ifdef HAVE_SYS_TYPES_H #include <sys/types.h> @@ -1032,6 +1015,17 @@ if test x$tcmalloc = xyes ; then LDFLAGS="-ltcmalloc $LDFLAGS" fi +using_custom_malloc=no +if test x$enable_openbsd_malloc = xyes ; then + using_custom_malloc=yes +fi +if test x$tcmalloc = xyes ; then + using_custom_malloc=yes +fi +if test $using_custom_malloc = no ; then + AC_CHECK_FUNCS(mallinfo) +fi + # By default, we're going to assume we don't have mlockall() # bionic and other platforms have various broken mlockall subsystems. # Some systems don't have a working mlockall, some aren't linkable, @@ -1317,24 +1311,12 @@ CPPFLAGS="$CPPFLAGS $TOR_CPPFLAGS_libevent $TOR_CPPFLAGS_openssl $TOR_CPPFLAGS_z AC_CONFIG_FILES([ Doxyfile Makefile - contrib/Makefile - contrib/suse/Makefile contrib/suse/tor.sh contrib/tor.logrotate contrib/tor.sh contrib/torctl contrib/torify - doc/Makefile - src/Makefile - src/common/Makefile - src/config/Makefile src/config/torrc.sample - src/or/Makefile - src/test/Makefile - src/tools/Makefile - src/tools/tor-fw-helper/Makefile - src/win32/Makefile - tor.spec ]) AC_OUTPUT @@ -1342,4 +1324,3 @@ AC_OUTPUT if test -x /usr/bin/perl && test -x ./contrib/updateVersions.pl ; then ./contrib/updateVersions.pl fi - diff --git a/contrib/Makefile.am b/contrib/Makefile.am deleted file mode 100644 index 795c351f3..000000000 --- a/contrib/Makefile.am +++ /dev/null @@ -1,23 +0,0 @@ -SUBDIRS = suse -DIST_SUBDIRS = suse - -confdir = $(sysconfdir)/tor - -EXTRA_DIST = \ - cross.sh \ - exitlist \ - linux-tor-prio.sh \ - package_nsis-mingw.sh \ - rc.subr \ - tor-ctrl.sh \ - tor-exit-notice.html \ - tor-mingw.nsi.in \ - tor-tsocks.conf \ - tor.ico \ - tor.nsi.in \ - tor.sh \ - torctl - -conf_DATA = tor-tsocks.conf - -bin_SCRIPTS = torify diff --git a/contrib/findMergedChanges.pl b/contrib/findMergedChanges.pl index 46e070f94..e4ff6163e 100755 --- a/contrib/findMergedChanges.pl +++ b/contrib/findMergedChanges.pl @@ -19,7 +19,7 @@ my $look_for_type = "merged"; if (! @ARGV) { print <<EOF Usage: - findMergedChanges.pl [--merged/--unmerged/--weird/--list] changes/* + findMergedChanges.pl [--merged/--unmerged/--weird/--list] [--branch=<branchname] changes/* A change is "merged" if it has ever been merged to release-0.2.2 and it has had no subsequent changes in master. diff --git a/contrib/include.am b/contrib/include.am new file mode 100644 index 000000000..4a995a37d --- /dev/null +++ b/contrib/include.am @@ -0,0 +1,20 @@ +include contrib/suse/include.am + +EXTRA_DIST+= \ + contrib/cross.sh \ + contrib/exitlist \ + contrib/linux-tor-prio.sh \ + contrib/package_nsis-mingw.sh \ + contrib/rc.subr \ + contrib/tor-ctrl.sh \ + contrib/tor-exit-notice.html \ + contrib/tor-mingw.nsi.in \ + contrib/tor-tsocks.conf \ + contrib/tor.ico \ + contrib/tor.nsi.in \ + contrib/tor.sh \ + contrib/torctl + +conf_DATA+= contrib/tor-tsocks.conf + +bin_SCRIPTS+= contrib/torify diff --git a/contrib/suse/Makefile.am b/contrib/suse/Makefile.am deleted file mode 100644 index 06511c042..000000000 --- a/contrib/suse/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ -confdir = $(sysconfdir)/tor - -EXTRA_DIST = tor.sh diff --git a/contrib/suse/include.am b/contrib/suse/include.am new file mode 100644 index 000000000..4aed0e123 --- /dev/null +++ b/contrib/suse/include.am @@ -0,0 +1 @@ +EXTRA_DIST+= contrib/suse/tor.sh diff --git a/contrib/tor-mingw.nsi.in b/contrib/tor-mingw.nsi.in index 2133a471b..96e27b463 100644 --- a/contrib/tor-mingw.nsi.in +++ b/contrib/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.2.3.18-rc-dev" +!define VERSION "0.2.4.2-alpha-dev" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/contrib/updateVersions.pl b/contrib/updateVersions.pl index 76b6fe567..710d7d935 100755 --- a/contrib/updateVersions.pl +++ b/contrib/updateVersions.pl @@ -1,6 +1,6 @@ #!/usr/bin/perl -w -$CONFIGURE_IN = './configure.in'; +$CONFIGURE_IN = './configure.ac'; $ORCONFIG_H = './src/win32/orconfig.h'; $TOR_NSI = './contrib/tor-mingw.nsi.in'; @@ -13,12 +13,12 @@ demand($CONFIGURE_IN); demand($ORCONFIG_H); demand($TOR_NSI); -# extract version from configure.in +# extract version from configure.ac open(F, $CONFIGURE_IN) or die "$!"; $version = undef; while (<F>) { - if (/AM_INIT_AUTOMAKE\(tor,\s*([^\)]*)\)/) { + if (/AC_INIT\(\[tor\],\s*\[([^\]]*)\]\)/) { $version = $1; last; } diff --git a/doc/HACKING b/doc/HACKING index bc409dc0d..e76b374d3 100644 --- a/doc/HACKING +++ b/doc/HACKING @@ -467,7 +467,7 @@ a stable release, add it to the ReleaseNotes file too. If we're adding to a release-0.2.x branch, manually commit the changelogs to the later git branches too. -4) Bump the version number in configure.in and rebuild. +4) Bump the version number in configure.ac and rebuild. 5) Make dist, put the tarball up somewhere, and tell #tor about it. Wait a while to see if anybody has problems building it. Try to get Sebastian diff --git a/doc/Makefile.am b/doc/Makefile.am deleted file mode 100644 index 6cdd66d51..000000000 --- a/doc/Makefile.am +++ /dev/null @@ -1,89 +0,0 @@ -# We use a two-step process to generate documentation from asciidoc files. -# -# First, we use asciidoc/a2x to process the asciidoc files into .1.in and -# .html.in files (see the asciidoc-helper.sh script). These are the same as -# the regular .1 and .html files, except that they still have some autoconf -# variables set in them. -# -# Second, we use config.status to turn .1.in files into .1 files and -# .html.in files into .html files. -# -# We do the steps in this order so that we can ship the .*.in files as -# part of the source distribution, so that people without asciidoc can -# just use the .1 and .html files. - -regular_mans = tor tor-gencert tor-resolve torify -all_mans = $(regular_mans) tor-fw-helper - -if USE_ASCIIDOC -if USE_FW_HELPER -nodist_man_MANS = $(all_mans:=.1) -doc_DATA = $(all_mans:=.html) -else -nodist_man_MANS = $(regular_mans:=.1) -doc_DATA = $(regular_mans:=.html) -endif -html_in = $(all_mans:=.html.in) -man_in = $(all_mans:=.1.in) -txt_in = $(all_mans:=.1.txt) -else -html_in = -man_in = -txt_in = -nodist_man_MANS = -doc_DATA = -endif - -EXTRA_DIST = HACKING asciidoc-helper.sh \ - $(html_in) $(man_in) $(txt_in) \ - tor-rpm-creation.txt \ - tor-win32-mingw-creation.txt spec/README \ - state-contents.txt - -docdir = @docdir@ - -asciidoc_product = $(nodist_man_MANS) $(doc_DATA) - -# Generate the html documentation from asciidoc, but don't do -# machine-specific replacements yet -$(html_in) : - $(top_srcdir)/doc/asciidoc-helper.sh html @ASCIIDOC@ $(top_srcdir)/doc/$@ - -tor.html.in : tor.1.txt -torify.html.in : torify.1.txt -tor-gencert.html.in : tor-gencert.1.txt -tor-resolve.html.in : tor-resolve.1.txt -tor-fw-helper.html.in : tor-fw-helper.1.txt - -# Generate the manpage from asciidoc, but don't do -# machine-specific replacements yet -$(man_in) : - $(top_srcdir)/doc/asciidoc-helper.sh man @A2X@ $(top_srcdir)/doc/$@ - -tor.1.in : tor.1.txt -torify.1.in : torify.1.txt -tor-gencert.1.in : tor-gencert.1.txt -tor-resolve.1.in : tor-resolve.1.txt -tor-fw-helper.1.in : tor-fw-helper.1.txt - -# use ../config.status to swap all machine-specific magic strings -# in the asciidoc with their replacements. -$(asciidoc_product) : - if test -e $(top_srcdir)/doc/$@.in && ! test -e ./$@.in ; then \ - cp $(top_srcdir)/doc/$@.in .; \ - fi - ../config.status --file=$@; - -tor.1 : tor.1.in -torify.1 : torify.1.in -tor-gencert.1 : tor-gencert.1.in -tor-resolve.1 : tor-resolve.1.in -tor-fw-helper.1 : tor-fw-helper.1.in -tor.html : tor.html.in -torify.html : torify.html.in -tor-gencert.html : tor-gencert.html.in -tor-resolve.html : tor-resolve.html.in -tor-fw-helper.html : tor-fw-helper.html.in - -CLEANFILES = $(asciidoc_product) config.log -DISTCLEANFILES = $(html_in) $(man_in) diff --git a/doc/include.am b/doc/include.am new file mode 100644 index 000000000..bb01038d4 --- /dev/null +++ b/doc/include.am @@ -0,0 +1,69 @@ +# We use a two-step process to generate documentation from asciidoc files. +# +# First, we use asciidoc/a2x to process the asciidoc files into .1.in and +# .html.in files (see the asciidoc-helper.sh script). These are the same as +# the regular .1 and .html files, except that they still have some autoconf +# variables set in them. +# +# Second, we use config.status to turn .1.in files into .1 files and +# .html.in files into .html files. +# +# We do the steps in this order so that we can ship the .*.in files as +# part of the source distribution, so that people without asciidoc can +# just use the .1 and .html files. + +regular_mans = doc/tor doc/tor-gencert doc/tor-resolve doc/torify +all_mans = $(regular_mans) doc/tor-fw-helper + +if USE_ASCIIDOC +if USE_FW_HELPER +nodist_man1_MANS = $(all_mans:=.1) +doc_DATA = $(all_mans:=.html) +else +nodist_man1_MANS = $(regular_mans:=.1) +doc_DATA = $(regular_mans:=.html) +endif +html_in = $(all_mans:=.html.in) +man_in = $(all_mans:=.1.in) +txt_in = $(all_mans:=.1.txt) +else +html_in = +man_in = +txt_in = +nodist_man1_MANS = +doc_DATA = +endif + +EXTRA_DIST+= doc/HACKING doc/asciidoc-helper.sh \ + $(html_in) $(man_in) $(txt_in) \ + doc/tor-rpm-creation.txt \ + doc/tor-win32-mingw-creation.txt doc/spec/README \ + doc/state-contents.txt + +docdir = @docdir@ + +asciidoc_product = $(nodist_man1_MANS) $(doc_DATA) + +# Generate the html documentation from asciidoc, but don't do +# machine-specific replacements yet +$(html_in) : $(txt_in) + $(AM_V_GEN)$(top_srcdir)/doc/asciidoc-helper.sh html @ASCIIDOC@ $(top_srcdir)/$@ + +# Generate the manpage from asciidoc, but don't do +# machine-specific replacements yet +$(man_in) : $(txt_in) + $(AM_V_GEN)$(top_srcdir)/doc/asciidoc-helper.sh man @A2X@ $(top_srcdir)/$@ + +# use ../config.status to swap all machine-specific magic strings +# in the asciidoc with their replacements. +$(asciidoc_product) : $(txt_in) $(man_in) + $(AM_V_GEN)$(MKDIR_P) $(@D) + $(AM_V_at)if test -e $(top_srcdir)/$@.in && ! test -e $@.in ; then \ + cp $(top_srcdir)/$@.in $@; \ + fi + $(AM_V_at)./config.status -q --file=$@; + +$(doc_DATA) : $(html_in) + +CLEANFILES+= $(asciidoc_product) config.log +DISTCLEANFILES+= $(html_in) $(man_in) diff --git a/doc/tor-fw-helper.1.txt b/doc/tor-fw-helper.1.txt index 49b091038..cf769d965 100644 --- a/doc/tor-fw-helper.1.txt +++ b/doc/tor-fw-helper.1.txt @@ -2,6 +2,8 @@ // See LICENSE for licensing information // This is an asciidoc file used to generate the manpage/html reference. // Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +:man source: Tor +:man manual: Tor Manual tor-fw-helper(1) ================ Jacob Appelbaum @@ -39,18 +41,8 @@ OPTIONS **-g** or **--fetch-public-ip**:: Fetch the the public ip address for each supported NAT helper method. -**-i** or **--internal-or-port** __port__:: - Inform **tor-fw-helper** of your internal OR port. This is the only - required argument. - -**-e** or **--external-or-port** __port__:: - Inform **tor-fw-helper** of your external OR port. - -**-d** or **--internal-dir-port** __port__:: - Inform **tor-fw-helper** of your internal Dir port. - -**-p** or **--external-dir-port** __port__:: - Inform **tor-fw-helper** of your external Dir port. +**-p** or **--forward-port** __external_port__:__internal_port__:: + Forward external_port to internal_port. BUGS ---- diff --git a/doc/tor-gencert.1.txt b/doc/tor-gencert.1.txt index 2a2d1179c..aa61ec3ec 100644 --- a/doc/tor-gencert.1.txt +++ b/doc/tor-gencert.1.txt @@ -2,6 +2,8 @@ // See LICENSE for licensing information // This is an asciidoc file used to generate the manpage/html reference. // Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +:man source: Tor +:man manual: Tor Manual tor-gencert(1) ============== Nick Mathewson diff --git a/doc/tor-resolve.1.txt b/doc/tor-resolve.1.txt index bdc636b08..341d30224 100644 --- a/doc/tor-resolve.1.txt +++ b/doc/tor-resolve.1.txt @@ -2,6 +2,8 @@ // See LICENSE for licensing information // This is an asciidoc file used to generate the manpage/html reference. // Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +:man source: Tor +:man manual: Tor Manual tor-resolve(1) ============== Peter Palfrader diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 5aac0542d..03cc50d63 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -2,6 +2,8 @@ // See LICENSE for licensing information // This is an asciidoc file used to generate the manpage/html reference. // Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +:man source: Tor +:man manual: Tor Manual TOR(1) ====== @@ -127,11 +129,11 @@ GENERAL OPTIONS the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. If you want to run a relay in the public network, this needs to be _at the very least_ 30 KB (that is, - 30720 bytes). (Default: 5 MB) + 30720 bytes). (Default: 1 GB) **BandwidthBurst** __N__ **bytes**|**KB**|**MB**|**GB**:: Limit the maximum token bucket size (also known as the burst) to the given - number of bytes in each direction. (Default: 10 MB) + number of bytes in each direction. (Default: 1 GB) **MaxAdvertisedBandwidth** __N__ **bytes**|**KB**|**MB**|**GB**:: If set, we will not advertise more than this amount of bandwidth for our @@ -1169,6 +1171,18 @@ The following options are useful only for clients (that is, if If no defaults are available there, these options default to 20, .70, 0.0, 200, and 4 respectively. +**ClientUseIPv6** **0**|**1**:: + If this option is set to 1, Tor might connect to entry nodes over + IPv6. Note that clients configured with an IPv6 address in a + **Bridge** line will try connecting over IPv6 even if + **ClientUseIPv6** is set to 0. (Default: 0) + +**ClientPreferIPv6ORPort** **0**|**1**:: + If this option is set to 1, Tor prefers an OR port with an IPv6 + address over one with IPv4 if a given entry node has both. Other + things may influence the choice. This option breaks a tie to the + favor of IPv6. (Default: 0) + SERVER OPTIONS -------------- @@ -1733,6 +1747,12 @@ DIRECTORY AUTHORITY SERVER OPTIONS votes on whether to accept relays as hidden service directories. (Default: 1) +**AuthDirHasIPv6Connectivity** **0**|**1**:: + Authoritative directories only. When set to 0, OR ports with an + IPv6 address are being accepted without reachability testing. + When set to 1, IPv6 OR ports are being tested just like IPv4 OR + ports. (Default: 0) + HIDDEN SERVICE OPTIONS ---------------------- diff --git a/doc/torify.1.txt b/doc/torify.1.txt index 4a4be1250..22b3fe537 100644 --- a/doc/torify.1.txt +++ b/doc/torify.1.txt @@ -2,10 +2,10 @@ // See LICENSE for licensing information // This is an asciidoc file used to generate the manpage/html reference. // Learn asciidoc on http://www.methods.co.nz/asciidoc/userguide.html +:man source: Tor +:man manual: Tor Manual torify(1) ========= -Peter Palfrader -Jacob Appelbaum NAME ---- @@ -48,3 +48,7 @@ SEE ALSO -------- **tor**(1), **tor-resolve**(1), **torsocks**(1), **tsocks**(1), **tsocks.conf**(5). + +AUTHORS +------- +Peter Palfrader and Jacob Appelbaum wrote this manual. diff --git a/src/Makefile.am b/src/Makefile.am deleted file mode 100644 index fa2dd560a..000000000 --- a/src/Makefile.am +++ /dev/null @@ -1,5 +0,0 @@ - -# leave in dependency order, since common must be built first -SUBDIRS = common or test tools win32 config -DIST_SUBDIRS = common or test tools win32 config - diff --git a/src/common/Makefile.am b/src/common/Makefile.am deleted file mode 100644 index 5e7684259..000000000 --- a/src/common/Makefile.am +++ /dev/null @@ -1,67 +0,0 @@ - -noinst_LIBRARIES = libor.a libor-crypto.a libor-event.a - -EXTRA_DIST = common_sha1.i sha256.c Makefile.nmake - -#CFLAGS = -Wall -Wpointer-arith -O2 - -if USE_OPENBSD_MALLOC -libor_extra_source=OpenBSD_malloc_Linux.c -else -libor_extra_source= -endif - -libor_a_SOURCES = \ - address.c \ - compat.c \ - container.c \ - di_ops.c \ - log.c \ - memarea.c \ - mempool.c \ - procmon.c \ - util.c \ - util_codedigest.c \ - $(libor_extra_source) - -libor_crypto_a_SOURCES = \ - aes.c \ - crypto.c \ - torgzip.c \ - tortls.c - -libor_event_a_SOURCES = compat_libevent.c - -noinst_HEADERS = \ - address.h \ - aes.h \ - ciphers.inc \ - compat.h \ - compat_libevent.h \ - container.h \ - crypto.h \ - di_ops.h \ - ht.h \ - memarea.h \ - mempool.h \ - procmon.h \ - strlcat.c \ - strlcpy.c \ - torgzip.h \ - torint.h \ - torlog.h \ - tortls.h \ - util.h - -common_sha1.i: $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS) - if test "@SHA1SUM@" != none; then \ - (cd "$(srcdir)" && "@SHA1SUM@" $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS)) | "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > common_sha1.i; \ - elif test "@OPENSSL@" != none; then \ - (cd "$(srcdir)" && "@OPENSSL@" sha1 $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(noinst_HEADERS)) | "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > common_sha1.i; \ - else \ - rm common_sha1.i; \ - touch common_sha1.i; \ - fi - -util_codedigest.o: common_sha1.i -crypto.o: sha256.c diff --git a/src/common/address.c b/src/common/address.c index e88869f1d..dffbcaff4 100644 --- a/src/common/address.c +++ b/src/common/address.c @@ -1697,3 +1697,15 @@ tor_addr_hostname_is_local(const char *name) !strcasecmpend(name, ".local"); } +/** Return a newly allocated tor_addr_port_t with <b>addr</b> and + <b>port</b> filled in. */ +tor_addr_port_t * +tor_addr_port_new(const tor_addr_t *addr, uint16_t port) +{ + tor_addr_port_t *ap = tor_malloc_zero(sizeof(tor_addr_port_t)); + if (addr) + tor_addr_copy(&ap->addr, addr); + ap->port = port; + return ap; +} + diff --git a/src/common/address.h b/src/common/address.h index c6c126862..7a779d888 100644 --- a/src/common/address.h +++ b/src/common/address.h @@ -40,7 +40,7 @@ typedef struct tor_addr_port_t uint16_t port; } tor_addr_port_t; -#define TOR_ADDR_NULL {AF_UNSPEC, {0}}; +#define TOR_ADDR_NULL {AF_UNSPEC, {0}} static INLINE const struct in6_addr *tor_addr_to_in6(const tor_addr_t *a); static INLINE uint32_t tor_addr_to_ipv4n(const tor_addr_t *a); @@ -221,5 +221,7 @@ int tor_inet_ntoa(const struct in_addr *in, char *buf, size_t buf_len); char *tor_dup_ip(uint32_t addr) ATTR_MALLOC; int get_interface_address(int severity, uint32_t *addr); +tor_addr_port_t *tor_addr_port_new(const tor_addr_t *addr, uint16_t port); + #endif diff --git a/src/common/aes.c b/src/common/aes.c index 59d864a3d..c6a997f47 100644 --- a/src/common/aes.c +++ b/src/common/aes.c @@ -134,8 +134,8 @@ int evaluate_evp_for_aes(int force_val) { (void) force_val; - log_notice(LD_CRYPTO, "This version of OpenSSL has a known-good EVP " - "counter-mode implementation. Using it."); + log_info(LD_CRYPTO, "This version of OpenSSL has a known-good EVP " + "counter-mode implementation. Using it."); return 0; } int @@ -212,11 +212,11 @@ evaluate_evp_for_aes(int force_val) e = ENGINE_get_cipher_engine(NID_aes_128_ecb); if (e) { - log_notice(LD_CRYPTO, "AES engine \"%s\" found; using EVP_* functions.", + log_info(LD_CRYPTO, "AES engine \"%s\" found; using EVP_* functions.", ENGINE_get_name(e)); should_use_EVP = 1; } else { - log_notice(LD_CRYPTO, "No AES engine found; using AES_* functions."); + log_info(LD_CRYPTO, "No AES engine found; using AES_* functions."); should_use_EVP = 0; } #endif @@ -263,12 +263,12 @@ evaluate_ctr_for_aes(void) "not using it."); } else { /* Counter mode is okay */ - log_notice(LD_CRYPTO, "This OpenSSL has a good implementation of counter " + log_info(LD_CRYPTO, "This OpenSSL has a good implementation of counter " "mode; using it."); should_use_openssl_CTR = 1; } #else - log_notice(LD_CRYPTO, "This version of OpenSSL has a slow implementation of " + log_info(LD_CRYPTO, "This version of OpenSSL has a slow implementation of " "counter mode; not using it."); #endif return 0; diff --git a/src/common/compat.c b/src/common/compat.c index ca850a303..12025b227 100644 --- a/src/common/compat.c +++ b/src/common/compat.c @@ -18,7 +18,7 @@ /* XXXX024 We should just use AC_USE_SYSTEM_EXTENSIONS in our autoconf, * and get this (and other important stuff!) automatically. Once we do that, * make sure to also change the extern char **environ detection in - * configure.in, because whether that is declared or not depends on whether + * configure.ac, because whether that is declared or not depends on whether * we have _GNU_SOURCE defined! Maybe that means that once we take this out, * we can also take out the configure check. */ #define _GNU_SOURCE diff --git a/src/common/compat_libevent.c b/src/common/compat_libevent.c index 6655ca87d..0d06c49c9 100644 --- a/src/common/compat_libevent.c +++ b/src/common/compat_libevent.c @@ -266,7 +266,7 @@ tor_libevent_initialize(tor_libevent_cfg *torcfg) #if defined(HAVE_EVENT_GET_VERSION) && defined(HAVE_EVENT_GET_METHOD) /* Making this a NOTICE for now so we can link bugs to a libevent versions * or methods better. */ - log(LOG_NOTICE, LD_GENERAL, + log(LOG_INFO, LD_GENERAL, "Initialized libevent version %s using method %s. Good.", event_get_version(), tor_libevent_get_method()); #else diff --git a/src/common/crypto.c b/src/common/crypto.c index a69e6c5cb..7768cc37b 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -221,6 +221,30 @@ try_load_engine(const char *path, const char *engine) } #endif +static char *crypto_openssl_version_str = NULL; +/* Return a human-readable version of the run-time openssl version number. */ +const char * +crypto_openssl_get_version_str(void) +{ + if (crypto_openssl_version_str == NULL) { + const char *raw_version = SSLeay_version(SSLEAY_VERSION); + const char *end_of_version = NULL; + /* The output should be something like "OpenSSL 1.0.0b 10 May 2012. Let's + trim that down. */ + if (!strcmpstart(raw_version, "OpenSSL ")) { + raw_version += strlen("OpenSSL "); + end_of_version = strchr(raw_version, ' '); + } + + if (end_of_version) + crypto_openssl_version_str = tor_strndup(raw_version, + end_of_version-raw_version); + else + crypto_openssl_version_str = tor_strdup(raw_version); + } + return crypto_openssl_version_str; +} + /** Initialize the crypto library. Return 0 on success, -1 on failure. */ int @@ -231,6 +255,19 @@ crypto_global_init(int useAccel, const char *accelName, const char *accelDir) OpenSSL_add_all_algorithms(); _crypto_global_initialized = 1; setup_openssl_threading(); + + if (SSLeay() == OPENSSL_VERSION_NUMBER && + !strcmp(SSLeay_version(SSLEAY_VERSION), OPENSSL_VERSION_TEXT)) { + log_info(LD_CRYPTO, "OpenSSL version matches version from headers " + "(%lx: %s).", SSLeay(), SSLeay_version(SSLEAY_VERSION)); + } else { + log_warn(LD_CRYPTO, "OpenSSL version from headers does not match the " + "version we're running with. If you get weird crashes, that " + "might be why. (Compiled with %lx: %s; running with %lx: %s).", + (unsigned long)OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_TEXT, + SSLeay(), SSLeay_version(SSLEAY_VERSION)); + } + if (useAccel > 0) { #ifdef DISABLE_ENGINES (void)accelName; @@ -3018,6 +3055,7 @@ crypto_global_cleanup(void) tor_free(ms); } #endif + tor_free(crypto_openssl_version_str); return 0; } diff --git a/src/common/crypto.h b/src/common/crypto.h index 76bcbf7d4..456a61173 100644 --- a/src/common/crypto.h +++ b/src/common/crypto.h @@ -111,6 +111,7 @@ typedef struct crypto_digest_t crypto_digest_t; typedef struct crypto_dh_t crypto_dh_t; /* global state */ +const char * crypto_openssl_get_version_str(void); int crypto_global_init(int hardwareAccel, const char *accelName, const char *accelPath); diff --git a/src/common/include.am b/src/common/include.am new file mode 100644 index 000000000..299c92e06 --- /dev/null +++ b/src/common/include.am @@ -0,0 +1,75 @@ + +noinst_LIBRARIES+= src/common/libor.a src/common/libor-crypto.a src/common/libor-event.a + +EXTRA_DIST+= \ + src/common/common_sha1.i \ + src/common/sha256.c \ + src/common/Makefile.nmake + +#CFLAGS = -Wall -Wpointer-arith -O2 +AM_CPPFLAGS += -I$(srcdir)/src/common -Isrc/common + +if USE_OPENBSD_MALLOC +libor_extra_source=src/common/OpenBSD_malloc_Linux.c +else +libor_extra_source= +endif + +src_common_libor_a_SOURCES = \ + src/common/address.c \ + src/common/compat.c \ + src/common/container.c \ + src/common/di_ops.c \ + src/common/log.c \ + src/common/memarea.c \ + src/common/mempool.c \ + src/common/procmon.c \ + src/common/util.c \ + src/common/util_codedigest.c \ + $(libor_extra_source) + +src_common_libor_crypto_a_SOURCES = \ + src/common/aes.c \ + src/common/crypto.c \ + src/common/torgzip.c \ + src/common/tortls.c + +src_common_libor_event_a_SOURCES = src/common/compat_libevent.c + +COMMONHEADERS = \ + src/common/address.h \ + src/common/aes.h \ + src/common/ciphers.inc \ + src/common/compat.h \ + src/common/compat_libevent.h \ + src/common/container.h \ + src/common/crypto.h \ + src/common/di_ops.h \ + src/common/ht.h \ + src/common/memarea.h \ + src/common/mempool.h \ + src/common/procmon.h \ + src/common/strlcat.c \ + src/common/strlcpy.c \ + src/common/torgzip.h \ + src/common/torint.h \ + src/common/torlog.h \ + src/common/tortls.h \ + src/common/util.h + +noinst_HEADERS+= $(COMMONHEADERS) + +DISTCLEANFILES+= src/common/common_sha1.i + +src/common/common_sha1.i: $(libor_SOURCES) $(libor_crypto_a_SOURCES) $(COMMONHEADERS) + $(AM_V_GEN)if test "@SHA1SUM@" != none; then \ + (cd "$(srcdir)" && "@SHA1SUM@" $(src_common_libor_SOURCES) $(src_common_libor_crypto_a_SOURCES) $(COMMONHEADERS)) | "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > $@; \ + elif test "@OPENSSL@" != none; then \ + (cd "$(srcdir)" && "@OPENSSL@" sha1 $(src_common_libor_SOURCES) $(src_Common_libor_crypto_a_SOURCES) $(COMMONHEADERS)) | "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > $@; \ + else \ + rm $@; \ + touch $@; \ + fi + +src/common/util_codedigest.o: src/common/common_sha1.i +src/common/crypto.o: src/common/sha256.c diff --git a/src/common/memarea.c b/src/common/memarea.c index 07bd593cc..e87c0fbed 100644 --- a/src/common/memarea.c +++ b/src/common/memarea.c @@ -118,7 +118,7 @@ alloc_chunk(size_t sz, int freelist_ok) size_t chunk_size = freelist_ok ? CHUNK_SIZE : sz; memarea_chunk_t *res; chunk_size += SENTINEL_LEN; - res = tor_malloc_roundup(&chunk_size); + res = tor_malloc(chunk_size); res->next_chunk = NULL; res->mem_size = chunk_size - CHUNK_HEADER_SIZE - SENTINEL_LEN; res->next_mem = res->u.mem; diff --git a/src/common/mempool.c b/src/common/mempool.c index 2416bce47..b25538670 100644 --- a/src/common/mempool.c +++ b/src/common/mempool.c @@ -70,7 +70,6 @@ #define ASSERT(x) tor_assert(x) #undef ALLOC_CAN_RETURN_NULL #define TOR -//#define ALLOC_ROUNDUP(p) tor_malloc_roundup(p) /* End Tor dependencies */ #else /* If you're not building this as part of Tor, you'll want to define the @@ -165,25 +164,16 @@ static mp_chunk_t * mp_chunk_new(mp_pool_t *pool) { size_t sz = pool->new_chunk_capacity * pool->item_alloc_size; -#ifdef ALLOC_ROUNDUP - size_t alloc_size = CHUNK_OVERHEAD + sz; - mp_chunk_t *chunk = ALLOC_ROUNDUP(&alloc_size); -#else mp_chunk_t *chunk = ALLOC(CHUNK_OVERHEAD + sz); -#endif + #ifdef MEMPOOL_STATS ++pool->total_chunks_allocated; #endif CHECK_ALLOC(chunk); memset(chunk, 0, sizeof(mp_chunk_t)); /* Doesn't clear the whole thing. */ chunk->magic = MP_CHUNK_MAGIC; -#ifdef ALLOC_ROUNDUP - chunk->mem_size = alloc_size - CHUNK_OVERHEAD; - chunk->capacity = chunk->mem_size / pool->item_alloc_size; -#else chunk->capacity = pool->new_chunk_capacity; chunk->mem_size = sz; -#endif chunk->next_mem = chunk->mem; chunk->pool = pool; return chunk; diff --git a/src/common/tortls.c b/src/common/tortls.c index 53bcc9891..a3485c768 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -478,7 +478,7 @@ tor_tls_init(void) * a test of intelligence and determination. */ if (version > OPENSSL_V(0,9,8,'k') && version <= OPENSSL_V(0,9,8,'l')) { - log_notice(LD_GENERAL, "OpenSSL %s looks like version 0.9.8l, but " + log_info(LD_GENERAL, "OpenSSL %s looks like version 0.9.8l, but " "some vendors have backported renegotiation code from " "0.9.8m without updating the version number. " "I will try SSL3_FLAGS and SSL_OP to enable renegotation.", @@ -486,12 +486,12 @@ tor_tls_init(void) use_unsafe_renegotiation_flag = 1; use_unsafe_renegotiation_op = 1; } else if (version > OPENSSL_V(0,9,8,'l')) { - log_notice(LD_GENERAL, "OpenSSL %s looks like version 0.9.8m or later; " + log_info(LD_GENERAL, "OpenSSL %s looks like version 0.9.8m or later; " "I will try SSL_OP to enable renegotiation", SSLeay_version(SSLEAY_VERSION)); use_unsafe_renegotiation_op = 1; } else if (version <= OPENSSL_V(0,9,8,'k')) { - log_notice(LD_GENERAL, "OpenSSL %s [%lx] looks like it's older than " + log_info(LD_GENERAL, "OpenSSL %s [%lx] looks like it's older than " "0.9.8l, but some vendors have backported 0.9.8l's " "renegotiation code to earlier versions, and some have " "backported the code from 0.9.8m or 0.9.8n. I'll set both " diff --git a/src/common/util.c b/src/common/util.c index 6fb597a3a..5329a0504 100644 --- a/src/common/util.c +++ b/src/common/util.c @@ -290,37 +290,6 @@ _tor_free(void *mem) tor_free(mem); } -#if defined(HAVE_MALLOC_GOOD_SIZE) && !defined(HAVE_MALLOC_GOOD_SIZE_PROTOTYPE) -/* Some version of Mac OSX have malloc_good_size in their libc, but not - * actually defined in malloc/malloc.h. We detect this and work around it by - * prototyping. - */ -extern size_t malloc_good_size(size_t size); -#endif - -/** Allocate and return a chunk of memory of size at least *<b>size</b>, using - * the same resources we would use to malloc *<b>sizep</b>. Set *<b>sizep</b> - * to the number of usable bytes in the chunk of memory. */ -void * -_tor_malloc_roundup(size_t *sizep DMALLOC_PARAMS) -{ -#ifdef HAVE_MALLOC_GOOD_SIZE - tor_assert(*sizep < SIZE_T_CEILING); - *sizep = malloc_good_size(*sizep); - return _tor_malloc(*sizep DMALLOC_FN_ARGS); -#elif 0 && defined(HAVE_MALLOC_USABLE_SIZE) && !defined(USE_DMALLOC) - /* Never use malloc_usable_size(); it makes valgrind really unhappy, - * and doesn't win much in terms of usable space where it exists. */ - void *result; - tor_assert(*sizep < SIZE_T_CEILING); - result = _tor_malloc(*sizep DMALLOC_FN_ARGS); - *sizep = malloc_usable_size(result); - return result; -#else - return _tor_malloc(*sizep DMALLOC_FN_ARGS); -#endif -} - /** Call the platform malloc info function, and dump the results to the log at * level <b>severity</b>. If no such function exists, do nothing. */ void @@ -363,9 +332,9 @@ tor_mathlog(double d) return log(d); } -/** Return the long integer closest to d. We define this wrapper here so - * that not all users of math.h need to use the right incancations to get - * the c99 functions. */ +/** Return the long integer closest to <b>d</b>. We define this wrapper + * here so that not all users of math.h need to use the right incantations + * to get the c99 functions. */ long tor_lround(double d) { @@ -378,6 +347,21 @@ tor_lround(double d) #endif } +/** Return the 64-bit integer closest to d. We define this wrapper here so + * that not all users of math.h need to use the right incantations to get the + * c99 functions. */ +int64_t +tor_llround(double d) +{ +#if defined(HAVE_LLROUND) + return (int64_t)llround(d); +#elif defined(HAVE_RINT) + return (int64_t)rint(d); +#else + return (int64_t)(d > 0 ? d + 0.5 : ceil(d - 0.5)); +#endif +} + /** Returns floor(log2(u64)). If u64 is 0, (incorrectly) returns 0. */ int tor_log2(uint64_t u64) @@ -410,12 +394,24 @@ tor_log2(uint64_t u64) return r; } -/** Return the power of 2 closest to <b>u64</b>. */ +/** Return the power of 2 in range [1,UINT64_MAX] closest to <b>u64</b>. If + * there are two powers of 2 equally close, round down. */ uint64_t round_to_power_of_2(uint64_t u64) { - int lg2 = tor_log2(u64); - uint64_t low = U64_LITERAL(1) << lg2, high = U64_LITERAL(1) << (lg2+1); + int lg2; + uint64_t low; + uint64_t high; + if (u64 == 0) + return 1; + + lg2 = tor_log2(u64); + low = U64_LITERAL(1) << lg2; + + if (lg2 == 63) + return low; + + high = U64_LITERAL(1) << (lg2+1); if (high - u64 < u64 - low) return high; else @@ -655,6 +651,16 @@ fast_memcmpstart(const void *mem, size_t memlen, return fast_memcmp(mem, prefix, plen); } +/** Given a nul-terminated string s, set every character before the nul + * to zero. */ +void +tor_strclear(char *s) +{ + while (*s) { + *s++ = '\0'; + } +} + /** Return a pointer to the first char of s that is not whitespace and * not a comment, or to the terminating NUL if no such character exists. */ @@ -2683,9 +2689,9 @@ digit_to_num(char d) * success, store the result in <b>out</b>, advance bufp to the next * character, and return 0. On failure, return -1. */ static int -scan_unsigned(const char **bufp, unsigned *out, int width, int base) +scan_unsigned(const char **bufp, unsigned long *out, int width, int base) { - unsigned result = 0; + unsigned long result = 0; int scanned_so_far = 0; const int hex = base==16; tor_assert(base == 10 || base == 16); @@ -2697,8 +2703,8 @@ scan_unsigned(const char **bufp, unsigned *out, int width, int base) while (**bufp && (hex?TOR_ISXDIGIT(**bufp):TOR_ISDIGIT(**bufp)) && scanned_so_far < width) { int digit = hex?hex_decode_digit(*(*bufp)++):digit_to_num(*(*bufp)++); - unsigned new_result = result * base + digit; - if (new_result > UINT32_MAX || new_result < result) + unsigned long new_result = result * base + digit; + if (new_result < result) return -1; /* over/underflow. */ result = new_result; ++scanned_so_far; @@ -2711,6 +2717,89 @@ scan_unsigned(const char **bufp, unsigned *out, int width, int base) return 0; } +/** Helper: Read an signed int from *<b>bufp</b> of up to <b>width</b> + * characters. (Handle arbitrary width if <b>width</b> is less than 0.) On + * success, store the result in <b>out</b>, advance bufp to the next + * character, and return 0. On failure, return -1. */ +static int +scan_signed(const char **bufp, long *out, int width) +{ + int neg = 0; + unsigned long result = 0; + + if (!bufp || !*bufp || !out) + return -1; + if (width<0) + width=MAX_SCANF_WIDTH; + + if (**bufp == '-') { + neg = 1; + ++*bufp; + --width; + } + + if (scan_unsigned(bufp, &result, width, 10) < 0) + return -1; + + if (neg) { + if (result > ((unsigned long)LONG_MAX) + 1) + return -1; /* Underflow */ + *out = -(long)result; + } else { + if (result > LONG_MAX) + return -1; /* Overflow */ + *out = (long)result; + } + + return 0; +} + +/** Helper: Read a decimal-formatted double from *<b>bufp</b> of up to + * <b>width</b> characters. (Handle arbitrary width if <b>width</b> is less + * than 0.) On success, store the result in <b>out</b>, advance bufp to the + * next character, and return 0. On failure, return -1. */ +static int +scan_double(const char **bufp, double *out, int width) +{ + int neg = 0; + double result = 0; + int scanned_so_far = 0; + + if (!bufp || !*bufp || !out) + return -1; + if (width<0) + width=MAX_SCANF_WIDTH; + + if (**bufp == '-') { + neg = 1; + ++*bufp; + } + + while (**bufp && TOR_ISDIGIT(**bufp) && scanned_so_far < width) { + const int digit = digit_to_num(*(*bufp)++); + result = result * 10 + digit; + ++scanned_so_far; + } + if (**bufp == '.') { + double fracval = 0, denominator = 1; + ++*bufp; + ++scanned_so_far; + while (**bufp && TOR_ISDIGIT(**bufp) && scanned_so_far < width) { + const int digit = digit_to_num(*(*bufp)++); + fracval = fracval * 10 + digit; + denominator *= 10; + ++scanned_so_far; + } + result += fracval / denominator; + } + + if (!scanned_so_far) /* No actual digits scanned */ + return -1; + + *out = neg ? -result : result; + return 0; +} + /** Helper: copy up to <b>width</b> non-space characters from <b>bufp</b> to * <b>out</b>. Make sure <b>out</b> is nul-terminated. Advance <b>bufp</b> * to the next non-space character or the EOS. */ @@ -2747,6 +2836,7 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) } } else { int width = -1; + int longmod = 0; ++pattern; if (TOR_ISDIGIT(*pattern)) { width = digit_to_num(*pattern++); @@ -2759,17 +2849,57 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) if (!width) /* No zero-width things. */ return -1; } + if (*pattern == 'l') { + longmod = 1; + ++pattern; + } if (*pattern == 'u' || *pattern == 'x') { - unsigned *u = va_arg(ap, unsigned *); + unsigned long u; const int base = (*pattern == 'u') ? 10 : 16; if (!*buf) return n_matched; - if (scan_unsigned(&buf, u, width, base)<0) + if (scan_unsigned(&buf, &u, width, base)<0) + return n_matched; + if (longmod) { + unsigned long *out = va_arg(ap, unsigned long *); + *out = u; + } else { + unsigned *out = va_arg(ap, unsigned *); + if (u > UINT_MAX) + return n_matched; + *out = (unsigned) u; + } + ++pattern; + ++n_matched; + } else if (*pattern == 'f') { + double *d = va_arg(ap, double *); + if (!longmod) + return -1; /* float not supported */ + if (!*buf) + return n_matched; + if (scan_double(&buf, d, width)<0) return n_matched; ++pattern; ++n_matched; + } else if (*pattern == 'd') { + long lng=0; + if (scan_signed(&buf, &lng, width)<0) + return n_matched; + if (longmod) { + long *out = va_arg(ap, long *); + *out = lng; + } else { + int *out = va_arg(ap, int *); + if (lng < INT_MIN || lng > INT_MAX) + return n_matched; + *out = (int)lng; + } + ++pattern; + ++n_matched; } else if (*pattern == 's') { char *s = va_arg(ap, char *); + if (longmod) + return -1; if (width < 0) return -1; if (scan_string(&buf, s, width)<0) @@ -2778,6 +2908,8 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) ++n_matched; } else if (*pattern == 'c') { char *ch = va_arg(ap, char *); + if (longmod) + return -1; if (width != -1) return -1; if (!*buf) @@ -2788,6 +2920,8 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) } else if (*pattern == '%') { if (*buf != '%') return n_matched; + if (longmod) + return -1; ++buf; ++pattern; } else { @@ -2801,9 +2935,14 @@ tor_vsscanf(const char *buf, const char *pattern, va_list ap) /** Minimal sscanf replacement: parse <b>buf</b> according to <b>pattern</b> * and store the results in the corresponding argument fields. Differs from - * sscanf in that it: Only handles %u, %x, %c and %Ns. Does not handle - * arbitrarily long widths. %u and %x do not consume any space. Is - * locale-independent. Returns -1 on malformed patterns. + * sscanf in that: + * <ul><li>It only handles %u, %lu, %x, %lx, %<NUM>s, %d, %ld, %lf, and %c. + * <li>It only handles decimal inputs for %lf. (12.3, not 1.23e1) + * <li>It does not handle arbitrarily long widths. + * <li>Numbers do not consume any space characters. + * <li>It is locale-independent. + * <li>%u and %x do not consume any space. + * <li>It returns -1 on malformed patterns.</ul> * * (As with other locale-independent functions, we need this to parse data that * is in ASCII without worrying that the C library's locale-handling will make @@ -3784,10 +3923,17 @@ tor_process_handle_destroy(process_handle_t *process_handle, if (also_terminate_process) { if (tor_terminate_process(process_handle) < 0) { - log_notice(LD_GENERAL, "Failed to terminate process with PID '%d'", - tor_process_get_pid(process_handle)); + const char *errstr = +#ifdef _WIN32 + format_win32_error(GetLastError()); +#else + strerror(errno); +#endif + log_notice(LD_GENERAL, "Failed to terminate process with " + "PID '%d' ('%s').", tor_process_get_pid(process_handle), + errstr); } else { - log_info(LD_GENERAL, "Terminated process with PID '%d'", + log_info(LD_GENERAL, "Terminated process with PID '%d'.", tor_process_get_pid(process_handle)); } } @@ -4256,6 +4402,50 @@ tor_split_lines(smartlist_t *sl, char *buf, int len) } #ifdef _WIN32 + +/** Return a smartlist containing lines outputted from + * <b>handle</b>. Return NULL on error, and set + * <b>stream_status_out</b> appropriately. */ +smartlist_t * +tor_get_lines_from_handle(HANDLE *handle, + enum stream_status *stream_status_out) +{ + int pos; + char stdout_buf[600] = {0}; + smartlist_t *lines = NULL; + + tor_assert(stream_status_out); + + *stream_status_out = IO_STREAM_TERM; + + pos = tor_read_all_handle(handle, stdout_buf, sizeof(stdout_buf) - 1, NULL); + if (pos < 0) { + *stream_status_out = IO_STREAM_TERM; + return NULL; + } + if (pos == 0) { + *stream_status_out = IO_STREAM_EAGAIN; + return NULL; + } + + /* End with a null even if there isn't a \r\n at the end */ + /* TODO: What if this is a partial line? */ + stdout_buf[pos] = '\0'; + + /* Split up the buffer */ + lines = smartlist_new(); + tor_split_lines(lines, stdout_buf, pos); + + /* Currently 'lines' is populated with strings residing on the + stack. Replace them with their exact copies on the heap: */ + SMARTLIST_FOREACH(lines, char *, line, + SMARTLIST_REPLACE_CURRENT(lines, line, tor_strdup(line))); + + *stream_status_out = IO_STREAM_OKAY; + + return lines; +} + /** Read from stream, and send lines to log at the specified log level. * Returns -1 if there is a error reading, and 0 otherwise. * If the generated stream is flushed more often than on new lines, or @@ -4303,6 +4493,33 @@ log_from_handle(HANDLE *pipe, int severity) #else +/** Return a smartlist containing lines outputted from + * <b>handle</b>. Return NULL on error, and set + * <b>stream_status_out</b> appropriately. */ +smartlist_t * +tor_get_lines_from_handle(FILE *handle, enum stream_status *stream_status_out) +{ + enum stream_status stream_status; + char stdout_buf[400]; + smartlist_t *lines = NULL; + + while (1) { + memset(stdout_buf, 0, sizeof(stdout_buf)); + + stream_status = get_string_from_pipe(handle, + stdout_buf, sizeof(stdout_buf) - 1); + if (stream_status != IO_STREAM_OKAY) + goto done; + + if (!lines) lines = smartlist_new(); + smartlist_add(lines, tor_strdup(stdout_buf)); + } + + done: + *stream_status_out = stream_status; + return lines; +} + /** Read from stream, and send lines to log at the specified log level. * Returns 1 if stream is closed normally, -1 if there is a error reading, and * 0 otherwise. Handles lines from tor-fw-helper and @@ -4421,9 +4638,130 @@ get_string_from_pipe(FILE *stream, char *buf_out, size_t count) return IO_STREAM_TERM; } -/* DOCDOC tor_check_port_forwarding */ +/** Parse a <b>line</b> from tor-fw-helper and issue an appropriate + * log message to our user. */ +static void +handle_fw_helper_line(const char *line) +{ + smartlist_t *tokens = smartlist_new(); + char *message = NULL; + char *message_for_log = NULL; + const char *external_port = NULL; + const char *internal_port = NULL; + const char *result = NULL; + int port = 0; + int success = 0; + + smartlist_split_string(tokens, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + + if (smartlist_len(tokens) < 5) + goto err; + + if (strcmp(smartlist_get(tokens, 0), "tor-fw-helper") || + strcmp(smartlist_get(tokens, 1), "tcp-forward")) + goto err; + + external_port = smartlist_get(tokens, 2); + internal_port = smartlist_get(tokens, 3); + result = smartlist_get(tokens, 4); + + if (smartlist_len(tokens) > 5) { + /* If there are more than 5 tokens, they are part of [<message>]. + Let's use a second smartlist to form the whole message; + strncat loops suck. */ + int i; + int message_words_n = smartlist_len(tokens) - 5; + smartlist_t *message_sl = smartlist_new(); + for (i = 0; i < message_words_n; i++) + smartlist_add(message_sl, smartlist_get(tokens, 5+i)); + + tor_assert(smartlist_len(message_sl) > 0); + message = smartlist_join_strings(message_sl, " ", 0, NULL); + + /* wrap the message in log-friendly wrapping */ + tor_asprintf(&message_for_log, " ('%s')", message); + + smartlist_free(message_sl); + } + + port = atoi(external_port); + if (port < 1 || port > 65535) + goto err; + + port = atoi(internal_port); + if (port < 1 || port > 65535) + goto err; + + if (!strcmp(result, "SUCCESS")) + success = 1; + else if (!strcmp(result, "FAIL")) + success = 0; + else + goto err; + + if (!success) { + log_warn(LD_GENERAL, "Tor was unable to forward TCP port '%s' to '%s'%s. " + "Please make sure that your router supports port " + "forwarding protocols (like NAT-PMP). Note that if '%s' is " + "your ORPort, your relay will be unable to receive inbound " + "traffic.", external_port, internal_port, + message_for_log ? message_for_log : "", + internal_port); + } else { + log_info(LD_GENERAL, + "Tor successfully forwarded TCP port '%s' to '%s'%s.", + external_port, internal_port, + message_for_log ? message_for_log : ""); + } + + goto done; + + err: + log_warn(LD_GENERAL, "tor-fw-helper sent us a string we could not " + "parse (%s).", line); + + done: + SMARTLIST_FOREACH(tokens, char *, cp, tor_free(cp)); + smartlist_free(tokens); + tor_free(message); + tor_free(message_for_log); +} + +/** Read what tor-fw-helper has to say in its stdout and handle it + * appropriately */ +static int +handle_fw_helper_output(process_handle_t *process_handle) +{ + smartlist_t *fw_helper_output = NULL; + enum stream_status stream_status = 0; + + fw_helper_output = + tor_get_lines_from_handle(tor_process_get_stdout_pipe(process_handle), + &stream_status); + if (!fw_helper_output) { /* didn't get any output from tor-fw-helper */ + /* if EAGAIN we should retry in the future */ + return (stream_status == IO_STREAM_EAGAIN) ? 0 : -1; + } + + /* Handle the lines we got: */ + SMARTLIST_FOREACH_BEGIN(fw_helper_output, char *, line) { + handle_fw_helper_line(line); + tor_free(line); + } SMARTLIST_FOREACH_END(line); + + smartlist_free(fw_helper_output); + + return 0; +} + +/** Spawn tor-fw-helper and ask it to forward the ports in + * <b>ports_to_forward</b>. <b>ports_to_forward</b> contains strings + * of the form "<external port>:<internal port>", which is the format + * that tor-fw-helper expects. */ void -tor_check_port_forwarding(const char *filename, int dir_port, int or_port, +tor_check_port_forwarding(const char *filename, + smartlist_t *ports_to_forward, time_t now) { /* When fw-helper succeeds, how long do we wait until running it again */ @@ -4437,32 +4775,51 @@ tor_check_port_forwarding(const char *filename, int dir_port, int or_port, static process_handle_t *child_handle=NULL; static time_t time_to_run_helper = 0; - int stdout_status, stderr_status, retval; - const char *argv[10]; - char s_dirport[6], s_orport[6]; + int stderr_status, retval; + int stdout_status = 0; tor_assert(filename); - /* Set up command line for tor-fw-helper */ - snprintf(s_dirport, sizeof s_dirport, "%d", dir_port); - snprintf(s_orport, sizeof s_orport, "%d", or_port); - - /* TODO: Allow different internal and external ports */ - argv[0] = filename; - argv[1] = "--internal-or-port"; - argv[2] = s_orport; - argv[3] = "--external-or-port"; - argv[4] = s_orport; - argv[5] = "--internal-dir-port"; - argv[6] = s_dirport; - argv[7] = "--external-dir-port"; - argv[8] = s_dirport; - argv[9] = NULL; - /* Start the child, if it is not already running */ if ((!child_handle || child_handle->status != PROCESS_STATUS_RUNNING) && time_to_run_helper < now) { - int status; + /*tor-fw-helper cli looks like this: tor_fw_helper -p :5555 -p 4555:1111 */ + const char **argv; /* cli arguments */ + int args_n, status; + int argv_index = 0; /* index inside 'argv' */ + + tor_assert(smartlist_len(ports_to_forward) > 0); + + /* check for overflow during 'argv' allocation: + (len(ports_to_forward)*2 + 2)*sizeof(char*) > SIZE_MAX == + len(ports_to_forward) > (((SIZE_MAX/sizeof(char*)) - 2)/2) */ + if ((size_t) smartlist_len(ports_to_forward) > + (((SIZE_MAX/sizeof(char*)) - 2)/2)) { + log_warn(LD_GENERAL, + "Overflow during argv allocation. This shouldn't happen."); + return; + } + /* check for overflow during 'argv_index' increase: + ((len(ports_to_forward)*2 + 2) > INT_MAX) == + len(ports_to_forward) > (INT_MAX - 2)/2 */ + if (smartlist_len(ports_to_forward) > (INT_MAX - 2)/2) { + log_warn(LD_GENERAL, + "Overflow during argv_index increase. This shouldn't happen."); + return; + } + + /* Calculate number of cli arguments: one for the filename, two + for each smartlist element (one for "-p" and one for the + ports), and one for the final NULL. */ + args_n = 1 + 2*smartlist_len(ports_to_forward) + 1; + argv = tor_malloc_zero(sizeof(char*)*args_n); + + argv[argv_index++] = filename; + SMARTLIST_FOREACH_BEGIN(ports_to_forward, const char *, port) { + argv[argv_index++] = "-p"; + argv[argv_index++] = port; + } SMARTLIST_FOREACH_END(port); + argv[argv_index] = NULL; /* Assume tor-fw-helper will succeed, start it later*/ time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_SUCCESS; @@ -4479,6 +4836,8 @@ tor_check_port_forwarding(const char *filename, int dir_port, int or_port, status = tor_spawn_background(filename, argv, NULL, &child_handle); #endif + tor_free(argv); + if (PROCESS_STATUS_ERROR == status) { log_warn(LD_GENERAL, "Failed to start port forwarding helper %s", filename); @@ -4496,16 +4855,17 @@ tor_check_port_forwarding(const char *filename, int dir_port, int or_port, /* Read from stdout/stderr and log result */ retval = 0; #ifdef _WIN32 - stdout_status = log_from_handle(child_handle->stdout_pipe, LOG_INFO); - stderr_status = log_from_handle(child_handle->stderr_pipe, LOG_WARN); - /* If we got this far (on Windows), the process started */ - retval = 0; + stderr_status = log_from_handle(child_handle->stderr_pipe, LOG_INFO); #else - stdout_status = log_from_pipe(child_handle->stdout_handle, - LOG_INFO, filename, &retval); stderr_status = log_from_pipe(child_handle->stderr_handle, - LOG_WARN, filename, &retval); + LOG_INFO, filename, &retval); #endif + if (handle_fw_helper_output(child_handle) < 0) { + log_warn(LD_GENERAL, "Failed to handle fw helper output."); + stdout_status = -1; + retval = -1; + } + if (retval) { /* There was a problem in the child process */ time_to_run_helper = now + TIME_TO_EXEC_FWHELPER_FAIL; diff --git a/src/common/util.h b/src/common/util.h index 8977d273c..04812df7c 100644 --- a/src/common/util.h +++ b/src/common/util.h @@ -76,7 +76,6 @@ /* Memory management */ void *_tor_malloc(size_t size DMALLOC_PARAMS) ATTR_MALLOC; void *_tor_malloc_zero(size_t size DMALLOC_PARAMS) ATTR_MALLOC; -void *_tor_malloc_roundup(size_t *size DMALLOC_PARAMS) ATTR_MALLOC; void *_tor_calloc(size_t nmemb, size_t size DMALLOC_PARAMS) ATTR_MALLOC; void *_tor_realloc(void *ptr, size_t size DMALLOC_PARAMS); char *_tor_strdup(const char *s DMALLOC_PARAMS) ATTR_MALLOC ATTR_NONNULL((1)); @@ -161,6 +160,7 @@ void tor_log_mallinfo(int severity); /* Math functions */ double tor_mathlog(double d) ATTR_CONST; long tor_lround(double d) ATTR_CONST; +int64_t tor_llround(double d) ATTR_CONST; int tor_log2(uint64_t u64) ATTR_CONST; uint64_t round_to_power_of_2(uint64_t u64); unsigned round_to_next_multiple_of(unsigned number, unsigned divisor); @@ -188,6 +188,7 @@ int strcasecmpstart(const char *s1, const char *s2) ATTR_NONNULL((1,2)); int strcmpend(const char *s1, const char *s2) ATTR_NONNULL((1,2)); int strcasecmpend(const char *s1, const char *s2) ATTR_NONNULL((1,2)); int fast_memcmpstart(const void *mem, size_t memlen, const char *prefix); +void tor_strclear(char *s); void tor_strstrip(char *s, const char *strip) ATTR_NONNULL((1,2)); long tor_parse_long(const char *s, int base, long min, @@ -373,7 +374,8 @@ void write_pidfile(char *filename); /* Port forwarding */ void tor_check_port_forwarding(const char *filename, - int dir_port, int or_port, time_t now); + struct smartlist_t *ports_to_forward, + time_t now); typedef struct process_handle_t process_handle_t; typedef struct process_environment_t process_environment_t; @@ -464,6 +466,16 @@ HANDLE tor_process_get_stdout_pipe(process_handle_t *process_handle); FILE *tor_process_get_stdout_pipe(process_handle_t *process_handle); #endif +#ifdef _WIN32 +struct smartlist_t * +tor_get_lines_from_handle(HANDLE *handle, + enum stream_status *stream_status); +#else +struct smartlist_t * +tor_get_lines_from_handle(FILE *handle, + enum stream_status *stream_status); +#endif + int tor_terminate_process(process_handle_t *process_handle); void tor_process_handle_destroy(process_handle_t *process_handle, int also_terminate_process); diff --git a/src/config/Makefile.am b/src/config/Makefile.am deleted file mode 100644 index 90dd218b4..000000000 --- a/src/config/Makefile.am +++ /dev/null @@ -1,16 +0,0 @@ -confdir = $(sysconfdir)/tor - -tordatadir = $(datadir)/tor - -EXTRA_DIST = geoip -# fallback-consensus - -conf_DATA = torrc.sample - -tordata_DATA = geoip -# fallback_consensus - -# If we don't have it, fake it. -fallback-consensus: - touch fallback-consensus - diff --git a/src/config/include.am b/src/config/include.am new file mode 100644 index 000000000..e6e1fe044 --- /dev/null +++ b/src/config/include.am @@ -0,0 +1,16 @@ +confdir = $(sysconfdir)/tor + +tordatadir = $(datadir)/tor + +EXTRA_DIST+= src/config/geoip +# fallback-consensus + +conf_DATA = src/config/torrc.sample + +tordata_DATA = src/config/geoip +# fallback_consensus + +# If we don't have it, fake it. +src_config_fallback-consensus: + touch src/config/fallback-consensus + diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in index a1a08aa8f..f30fc32fd 100644 --- a/src/config/torrc.sample.in +++ b/src/config/torrc.sample.in @@ -16,7 +16,7 @@ ## configure one below. Set "SocksPort 0" if you plan to run Tor only ## as a relay, and not make any local application connections yourself. #SocksPort 9050 # Default: Bind to localhost:9050 for local connections. -#SocksPort 192.168.0.1:9100 # Bind to this adddress:port too. +#SocksPort 192.168.0.1:9100 # Bind to this address:port too. ## Entry policies to allow/deny SOCKS requests based on IP address. ## First entry that matches wins. If no SocksPolicy is set, we accept diff --git a/src/include.am b/src/include.am new file mode 100644 index 000000000..965a49404 --- /dev/null +++ b/src/include.am @@ -0,0 +1,6 @@ +include src/common/include.am +include src/or/include.am +include src/test/include.am +include src/tools/include.am +include src/win32/include.am +include src/config/include.am
\ No newline at end of file diff --git a/src/or/Makefile.am b/src/or/Makefile.am deleted file mode 100644 index 3cc789a1b..000000000 --- a/src/or/Makefile.am +++ /dev/null @@ -1,158 +0,0 @@ -bin_PROGRAMS = tor -noinst_LIBRARIES = libtor.a - -if BUILD_NT_SERVICES -tor_platform_source=ntmain.c -else -tor_platform_source= -endif - -EXTRA_DIST=ntmain.c or_sha1.i Makefile.nmake - -if USE_EXTERNAL_EVDNS -evdns_source= -else -evdns_source=eventdns.c -endif - -libtor_a_SOURCES = \ - buffers.c \ - circuitbuild.c \ - circuitlist.c \ - circuituse.c \ - command.c \ - config.c \ - connection.c \ - connection_edge.c \ - connection_or.c \ - control.c \ - cpuworker.c \ - directory.c \ - dirserv.c \ - dirvote.c \ - dns.c \ - dnsserv.c \ - geoip.c \ - hibernate.c \ - main.c \ - microdesc.c \ - networkstatus.c \ - nodelist.c \ - onion.c \ - transports.c \ - policies.c \ - reasons.c \ - relay.c \ - rendclient.c \ - rendcommon.c \ - rendmid.c \ - rendservice.c \ - rephist.c \ - router.c \ - routerlist.c \ - routerparse.c \ - status.c \ - $(evdns_source) \ - $(tor_platform_source) \ - config_codedigest.c - -#libtor_a_LIBADD = ../common/libor.a ../common/libor-crypto.a \ -# ../common/libor-event.a - - -tor_SOURCES = tor_main.c - -AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ - -DLOCALSTATEDIR="\"$(localstatedir)\"" \ - -DBINDIR="\"$(bindir)\"" - -# -L flags need to go in LDFLAGS. -l flags need to go in LDADD. -# This seems to matter nowhere but on windows, but I assure you that it -# matters a lot there, and is quite hard to debug if you forget to do it. - - -tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@ -tor_LDADD = ./libtor.a ../common/libor.a ../common/libor-crypto.a \ - ../common/libor-event.a \ - @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ \ - @TOR_LIB_WS32@ @TOR_LIB_GDI@ - -noinst_HEADERS = \ - buffers.h \ - circuitbuild.h \ - circuitlist.h \ - circuituse.h \ - command.h \ - config.h \ - connection.h \ - connection_edge.h \ - connection_or.h \ - control.h \ - cpuworker.h \ - directory.h \ - dirserv.h \ - dirvote.h \ - dns.h \ - dnsserv.h \ - eventdns.h \ - eventdns_tor.h \ - geoip.h \ - hibernate.h \ - main.h \ - microdesc.h \ - networkstatus.h \ - nodelist.h \ - ntmain.h \ - onion.h \ - or.h \ - transports.h \ - policies.h \ - reasons.h \ - relay.h \ - rendclient.h \ - rendcommon.h \ - rendmid.h \ - rendservice.h \ - rephist.h \ - router.h \ - routerlist.h \ - routerparse.h \ - status.h \ - micro-revision.i - -config_codedigest.o: or_sha1.i - -tor_main.o: micro-revision.i - -micro-revision.i: FORCE - @rm -f micro-revision.tmp; \ - if test -d "$(top_srcdir)/.git" && \ - test -x "`which git 2>&1;true`"; then \ - HASH="`cd "$(top_srcdir)" && git rev-parse --short=16 HEAD`"; \ - echo \"$$HASH\" > micro-revision.tmp; \ - fi; \ - if test ! -f micro-revision.tmp ; then \ - if test ! -f micro-revision.i ; then \ - echo '""' > micro-revision.i; \ - fi; \ - elif test ! -f micro-revision.i || \ - test x"`cat micro-revision.tmp`" != x"`cat micro-revision.i`"; then \ - mv micro-revision.tmp micro-revision.i; \ - fi; true - -or_sha1.i: $(tor_SOURCES) $(libtor_a_SOURCES) - if test "@SHA1SUM@" != none; then \ - (cd "$(srcdir)" && "@SHA1SUM@" $(tor_SOURCES) $(libtor_a_SOURCES)) | \ - "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > or_sha1.i; \ - elif test "@OPENSSL@" != none; then \ - (cd "$(srcdir)" && "@OPENSSL@" sha1 $(tor_SOURCES) $(libtor_a_SOURCES)) | \ - "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > or_sha1.i; \ - else \ - rm or_sha1.i; \ - touch or_sha1.i; \ - fi - -CLEANFILES = micro-revision.i - -#Dummy target to ensure that micro-revision.i _always_ gets built. -FORCE: diff --git a/src/or/buffers.c b/src/or/buffers.c index 9acc22971..a8d06cef1 100644 --- a/src/or/buffers.c +++ b/src/or/buffers.c @@ -192,8 +192,6 @@ chunk_new_with_alloc_size(size_t alloc) freelist->lowest_length = freelist->cur_length; ++freelist->n_hit; } else { - /* XXXX take advantage of tor_malloc_roundup, once we know how that - * affects freelists. */ if (freelist) ++freelist->n_alloc; else @@ -216,7 +214,7 @@ static INLINE chunk_t * chunk_new_with_alloc_size(size_t alloc) { chunk_t *ch; - ch = tor_malloc_roundup(&alloc); + ch = tor_malloc(alloc); ch->next = NULL; ch->datalen = 0; ch->memlen = CHUNK_SIZE_WITH_ALLOC(alloc); diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index 9d52b5e02..d4969239c 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -16,6 +16,7 @@ #include "circuitlist.h" #include "circuituse.h" #include "config.h" +#include "confparse.h" #include "connection.h" #include "connection_edge.h" #include "connection_or.h" @@ -32,6 +33,8 @@ #include "router.h" #include "routerlist.h" #include "routerparse.h" +#include "routerset.h" +#include "statefile.h" #include "crypto.h" #undef log #include <math.h> @@ -2470,7 +2473,7 @@ circuit_extend(cell_t *cell, circuit_t *circ) log_debug(LD_CIRC|LD_OR,"Next router (%s:%d): %s", fmt_addr(&n_addr), (int)n_port, msg?msg:"????"); - circ->n_hop = extend_info_alloc(NULL /*nickname*/, + circ->n_hop = extend_info_new(NULL /*nickname*/, id_digest, NULL /*onion_key*/, &n_addr, n_port); @@ -2608,12 +2611,12 @@ pathbias_get_scale_threshold(const or_options_t *options) static int pathbias_get_scale_factor(const or_options_t *options) { -#define DFLT_PATH_BIAS_SCALE_FACTOR 4 +#define DFLT_PATH_BIAS_SCALE_FACTOR 2 if (options->PathBiasScaleFactor >= 1) return options->PathBiasScaleFactor; else return networkstatus_get_param(NULL, "pb_scalefactor", - DFLT_PATH_BIAS_SCALE_THRESHOLD, 1, INT32_MAX); + DFLT_PATH_BIAS_SCALE_FACTOR, 1, INT32_MAX); } static const char * @@ -2645,6 +2648,14 @@ pathbias_count_first_hop(origin_circuit_t *circ) RATELIM_INIT(FIRST_HOP_NOTICE_INTERVAL); char *rate_msg = NULL; + /* We can't do path bias accounting without entry guards. + * Testing and controller circuits also have no guards. */ + if (get_options()->UseEntryGuards == 0 || + circ->_base.purpose == CIRCUIT_PURPOSE_TESTING || + circ->_base.purpose == CIRCUIT_PURPOSE_CONTROLLER) { + return 0; + } + /* Completely ignore one hop circuits */ if (circ->build_state->onehop_tunnel) { tor_assert(circ->build_state->desired_path_len == 1); @@ -2739,6 +2750,14 @@ pathbias_count_success(origin_circuit_t *circ) RATELIM_INIT(SUCCESS_NOTICE_INTERVAL); char *rate_msg = NULL; + /* We can't do path bias accounting without entry guards. + * Testing and controller circuits also have no guards. */ + if (get_options()->UseEntryGuards == 0 || + circ->_base.purpose == CIRCUIT_PURPOSE_TESTING || + circ->_base.purpose == CIRCUIT_PURPOSE_CONTROLLER) { + return; + } + /* Ignore one hop circuits */ if (circ->build_state->onehop_tunnel) { tor_assert(circ->build_state->desired_path_len == 1); @@ -2772,12 +2791,15 @@ pathbias_count_success(origin_circuit_t *circ) } if (guard->first_hops < guard->circuit_successes) { - log_info(LD_BUG, "Unexpectedly high circuit_successes (%u/%u) " + log_notice(LD_BUG, "Unexpectedly high circuit_successes (%u/%u) " "for guard %s=%s", guard->circuit_successes, guard->first_hops, guard->nickname, hex_str(guard->identity, DIGEST_LEN)); } - } else { + /* In rare cases, CIRCUIT_PURPOSE_TESTING can get converted to + * CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT and have no guards here. + * No need to log that case. */ + } else if (circ->_base.purpose != CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) { if ((rate_msg = rate_limit_log(&success_notice_limit, approx_time()))) { log_info(LD_BUG, @@ -2822,9 +2844,11 @@ entry_guard_inc_first_hop_count(entry_guard_t *guard) if (guard->circuit_successes/((double)guard->first_hops) < pathbias_get_disable_rate(options)) { - log_info(LD_PROTOCOL, + /* This message is currently disabled by default. */ + log_warn(LD_PROTOCOL, "Extremely low circuit success rate %u/%u for guard %s=%s. " - "This might indicate an attack, or a bug.", + "This indicates either an overloaded guard, an attack, or " + "a bug.", guard->circuit_successes, guard->first_hops, guard->nickname, hex_str(guard->identity, DIGEST_LEN)); @@ -2835,7 +2859,7 @@ entry_guard_inc_first_hop_count(entry_guard_t *guard) < pathbias_get_notice_rate(options) && !guard->path_bias_notice) { guard->path_bias_notice = 1; - log_info(LD_PROTOCOL, + log_notice(LD_PROTOCOL, "Low circuit success rate %u/%u for guard %s=%s.", guard->circuit_successes, guard->first_hops, guard->nickname, hex_str(guard->identity, DIGEST_LEN)); @@ -2845,8 +2869,18 @@ entry_guard_inc_first_hop_count(entry_guard_t *guard) /* If we get a ton of circuits, just scale everything down */ if (guard->first_hops > (unsigned)pathbias_get_scale_threshold(options)) { const int scale_factor = pathbias_get_scale_factor(options); - guard->first_hops /= scale_factor; - guard->circuit_successes /= scale_factor; + /* For now, only scale if there will be no rounding error... + * XXX024: We want to switch to a real moving average for 0.2.4. */ + if ((guard->first_hops % scale_factor) == 0 && + (guard->circuit_successes % scale_factor) == 0) { + log_info(LD_PROTOCOL, + "Scaling pathbias counts to (%u/%u)/%d for guard %s=%s", + guard->circuit_successes, guard->first_hops, + scale_factor, guard->nickname, hex_str(guard->identity, + DIGEST_LEN)); + guard->first_hops /= scale_factor; + guard->circuit_successes /= scale_factor; + } } guard->first_hops++; log_info(LD_PROTOCOL, "Got success count %u/%u for guard %s=%s", @@ -3761,12 +3795,10 @@ onion_extend_cpath(origin_circuit_t *circ) } else if (cur_len == 0) { /* picking first node */ const node_t *r = choose_good_entry_server(purpose, state); if (r) { - /* If we're extending to a bridge, use the preferred address - rather than the primary, for potentially extending to an IPv6 - bridge. */ - int use_pref_addr = (r->ri != NULL && - r->ri->purpose == ROUTER_PURPOSE_BRIDGE); - info = extend_info_from_node(r, use_pref_addr); + /* If we're a client, use the preferred address rather than the + primary address, for potentially connecting to an IPv6 OR + port. */ + info = extend_info_from_node(r, server_mode(get_options()) == 0); tor_assert(info); } } else { @@ -3817,7 +3849,7 @@ onion_append_hop(crypt_path_t **head_ptr, extend_info_t *choice) /** Allocate a new extend_info object based on the various arguments. */ extend_info_t * -extend_info_alloc(const char *nickname, const char *digest, +extend_info_new(const char *nickname, const char *digest, crypto_pk_t *onion_key, const tor_addr_t *addr, uint16_t port) { @@ -3832,47 +3864,45 @@ extend_info_alloc(const char *nickname, const char *digest, return info; } -/** Allocate and return a new extend_info_t that can be used to build - * a circuit to or through the router <b>r</b>. Use the primary - * address of the router unless <b>for_direct_connect</b> is true, in - * which case the preferred address is used instead. */ +/** Allocate and return a new extend_info that can be used to build a + * circuit to or through the node <b>node</b>. Use the primary address + * of the node (i.e. its IPv4 address) unless + * <b>for_direct_connect</b> is true, in which case the preferred + * address is used instead. May return NULL if there is not enough + * info about <b>node</b> to extend to it--for example, if there is no + * routerinfo_t or microdesc_t. + **/ extend_info_t * -extend_info_from_router(const routerinfo_t *r, int for_direct_connect) +extend_info_from_node(const node_t *node, int for_direct_connect) { tor_addr_port_t ap; - tor_assert(r); + + if (node->ri == NULL && (node->rs == NULL || node->md == NULL)) + return NULL; if (for_direct_connect) - router_get_pref_orport(r, &ap); + node_get_pref_orport(node, &ap); else - router_get_prim_orport(r, &ap); - return extend_info_alloc(r->nickname, r->cache_info.identity_digest, - r->onion_pkey, &ap.addr, ap.port); -} + node_get_prim_orport(node, &ap); -/** Allocate and return a new extend_info that can be used to build a - * circuit to or through the node <b>node</b>. Use the primary address - * of the node unless <b>for_direct_connect</b> is true, in which case - * the preferred address is used instead. May return NULL if there is - * not enough info about <b>node</b> to extend to it--for example, if - * there is no routerinfo_t or microdesc_t. - **/ -extend_info_t * -extend_info_from_node(const node_t *node, int for_direct_connect) -{ - if (node->ri) { - return extend_info_from_router(node->ri, for_direct_connect); - } else if (node->rs && node->md) { - tor_addr_t addr; - tor_addr_from_ipv4h(&addr, node->rs->addr); - return extend_info_alloc(node->rs->nickname, + log_debug(LD_CIRC, "using %s:%d for %s", + fmt_and_decorate_addr(&ap.addr), ap.port, + node->ri ? node->ri->nickname : node->rs->nickname); + + if (node->ri) + return extend_info_new(node->ri->nickname, + node->identity, + node->ri->onion_pkey, + &ap.addr, + ap.port); + else if (node->rs && node->md) + return extend_info_new(node->rs->nickname, node->identity, node->md->onion_pkey, - &addr, - node->rs->or_port); - } else { + &ap.addr, + ap.port); + else return NULL; - } } /** Release storage held by an extend_info_t struct. */ @@ -5150,203 +5180,37 @@ bridge_free(bridge_info_t *bridge) tor_free(bridge); } -/** A list of pluggable transports found in torrc. */ -static smartlist_t *transport_list = NULL; - -/** Mark every entry of the transport list to be removed on our next call to - * sweep_transport_list unless it has first been un-marked. */ -void -mark_transport_list(void) -{ - if (!transport_list) - transport_list = smartlist_new(); - SMARTLIST_FOREACH(transport_list, transport_t *, t, - t->marked_for_removal = 1); -} - -/** Remove every entry of the transport list that was marked with - * mark_transport_list if it has not subsequently been un-marked. */ -void -sweep_transport_list(void) -{ - if (!transport_list) - transport_list = smartlist_new(); - SMARTLIST_FOREACH_BEGIN(transport_list, transport_t *, t) { - if (t->marked_for_removal) { - SMARTLIST_DEL_CURRENT(transport_list, t); - transport_free(t); - } - } SMARTLIST_FOREACH_END(t); -} - -/** Initialize the pluggable transports list to empty, creating it if - * needed. */ -void -clear_transport_list(void) -{ - if (!transport_list) - transport_list = smartlist_new(); - SMARTLIST_FOREACH(transport_list, transport_t *, t, transport_free(t)); - smartlist_clear(transport_list); -} - -/** Free the pluggable transport struct <b>transport</b>. */ -void -transport_free(transport_t *transport) -{ - if (!transport) - return; - - tor_free(transport->name); - tor_free(transport); -} - -/** Returns the transport in our transport list that has the name <b>name</b>. - * Else returns NULL. */ -transport_t * -transport_get_by_name(const char *name) +/** If we have a bridge configured whose digest matches <b>digest</b>, or a + * bridge with no known digest whose address matches any of the + * tor_addr_port_t's in <b>orports</b>, return that bridge. Else return + * NULL. */ +static bridge_info_t * +get_configured_bridge_by_orports_digest(const char *digest, + const smartlist_t *orports) { - tor_assert(name); - - if (!transport_list) + if (!bridge_list) return NULL; - - SMARTLIST_FOREACH_BEGIN(transport_list, transport_t *, transport) { - if (!strcmp(transport->name, name)) - return transport; - } SMARTLIST_FOREACH_END(transport); - - return NULL; -} - -/** Returns a transport_t struct for a transport proxy supporting the - protocol <b>name</b> listening at <b>addr</b>:<b>port</b> using - SOCKS version <b>socks_ver</b>. */ -transport_t * -transport_new(const tor_addr_t *addr, uint16_t port, - const char *name, int socks_ver) -{ - transport_t *t = tor_malloc_zero(sizeof(transport_t)); - - tor_addr_copy(&t->addr, addr); - t->port = port; - t->name = tor_strdup(name); - t->socks_version = socks_ver; - - return t; -} - -/** Resolve any conflicts that the insertion of transport <b>t</b> - * might cause. - * Return 0 if <b>t</b> is OK and should be registered, 1 if there is - * a transport identical to <b>t</b> already registered and -1 if - * <b>t</b> cannot be added due to conflicts. */ -static int -transport_resolve_conflicts(transport_t *t) -{ - /* This is how we resolve transport conflicts: - - If there is already a transport with the same name and addrport, - we either have duplicate torrc lines OR we are here post-HUP and - this transport was here pre-HUP as well. In any case, mark the - old transport so that it doesn't get removed and ignore the new - one. Our caller has to free the new transport so we return '1' to - signify this. - - If there is already a transport with the same name but different - addrport: - * if it's marked for removal, it means that it either has a lower - priority than 't' in torrc (otherwise the mark would have been - cleared by the paragraph above), or it doesn't exist at all in - the post-HUP torrc. We destroy the old transport and register 't'. - * if it's *not* marked for removal, it means that it was newly - added in the post-HUP torrc or that it's of higher priority, in - this case we ignore 't'. */ - transport_t *t_tmp = transport_get_by_name(t->name); - if (t_tmp) { /* same name */ - if (tor_addr_eq(&t->addr, &t_tmp->addr) && (t->port == t_tmp->port)) { - /* same name *and* addrport */ - t_tmp->marked_for_removal = 0; - return 1; - } else { /* same name but different addrport */ - if (t_tmp->marked_for_removal) { /* marked for removal */ - log_notice(LD_GENERAL, "You tried to add transport '%s' at '%s:%u' " - "but there was already a transport marked for deletion at " - "'%s:%u'. We deleted the old transport and registered the " - "new one.", t->name, fmt_addr(&t->addr), t->port, - fmt_addr(&t_tmp->addr), t_tmp->port); - smartlist_remove(transport_list, t_tmp); - transport_free(t_tmp); - } else { /* *not* marked for removal */ - log_notice(LD_GENERAL, "You tried to add transport '%s' at '%s:%u' " - "but the same transport already exists at '%s:%u'. " - "Skipping.", t->name, fmt_addr(&t->addr), t->port, - fmt_addr(&t_tmp->addr), t_tmp->port); - return -1; + SMARTLIST_FOREACH_BEGIN(bridge_list, bridge_info_t *, bridge) + { + if (tor_digest_is_zero(bridge->identity)) { + SMARTLIST_FOREACH_BEGIN(orports, tor_addr_port_t *, ap) + { + if (tor_addr_compare(&bridge->addr, &ap->addr, CMP_EXACT) == 0 && + bridge->port == ap->port) + return bridge; + } + SMARTLIST_FOREACH_END(ap); } + if (digest && tor_memeq(bridge->identity, digest, DIGEST_LEN)) + return bridge; } - } - - return 0; -} - -/** Add transport <b>t</b> to the internal list of pluggable - * transports. - * Returns 0 if the transport was added correctly, 1 if the same - * transport was already registered (in this case the caller must - * free the transport) and -1 if there was an error. */ -int -transport_add(transport_t *t) -{ - int r; - tor_assert(t); - - r = transport_resolve_conflicts(t); - - switch (r) { - case 0: /* should register transport */ - if (!transport_list) - transport_list = smartlist_new(); - smartlist_add(transport_list, t); - return 0; - default: /* let our caller know the return code */ - return r; - } -} - -/** Remember a new pluggable transport proxy at <b>addr</b>:<b>port</b>. - * <b>name</b> is set to the name of the protocol this proxy uses. - * <b>socks_ver</b> is set to the SOCKS version of the proxy. */ -int -transport_add_from_config(const tor_addr_t *addr, uint16_t port, - const char *name, int socks_ver) -{ - transport_t *t = transport_new(addr, port, name, socks_ver); - - int r = transport_add(t); - - switch (r) { - case -1: - default: - log_notice(LD_GENERAL, "Could not add transport %s at %s:%u. Skipping.", - t->name, fmt_addr(&t->addr), t->port); - transport_free(t); - return -1; - case 1: - log_info(LD_GENERAL, "Succesfully registered transport %s at %s:%u.", - t->name, fmt_addr(&t->addr), t->port); - transport_free(t); /* falling */ - return 0; - case 0: - log_info(LD_GENERAL, "Succesfully registered transport %s at %s:%u.", - t->name, fmt_addr(&t->addr), t->port); - return 0; - } + SMARTLIST_FOREACH_END(bridge); + return NULL; } -/** Return a bridge pointer if <b>ri</b> is one of our known bridges - * (either by comparing keys if possible, else by comparing addr/port). - * Else return NULL. */ +/** If we have a bridge configured whose digest matches <b>digest</b>, or a + * bridge with no known digest whose address matches <b>addr</b>:<b>/port</b>, + * return that bridge. Else return NULL. */ static bridge_info_t * get_configured_bridge_by_addr_port_digest(const tor_addr_t *addr, uint16_t port, @@ -5372,11 +5236,13 @@ get_configured_bridge_by_addr_port_digest(const tor_addr_t *addr, static bridge_info_t * get_configured_bridge_by_routerinfo(const routerinfo_t *ri) { - tor_addr_port_t ap; - - router_get_pref_orport(ri, &ap); - return get_configured_bridge_by_addr_port_digest(&ap.addr, ap.port, - ri->cache_info.identity_digest); + bridge_info_t *bi = NULL; + smartlist_t *orports = router_get_all_orports(ri); + bi = get_configured_bridge_by_orports_digest(ri->cache_info.identity_digest, + orports); + SMARTLIST_FOREACH(orports, tor_addr_port_t *, p, tor_free(p)); + smartlist_free(orports); + return bi; } /** Return 1 if <b>ri</b> is one of our known bridges, else 0. */ @@ -5390,30 +5256,12 @@ routerinfo_is_a_configured_bridge(const routerinfo_t *ri) int node_is_a_configured_bridge(const node_t *node) { - int retval = 0; /* Negative. */ - smartlist_t *orports = NULL; - - if (!node) - goto out; - - orports = node_get_all_orports(node); - if (orports == NULL) - goto out; - - SMARTLIST_FOREACH_BEGIN(orports, tor_addr_port_t *, orport) { - if (get_configured_bridge_by_addr_port_digest(&orport->addr, orport->port, - node->identity) != NULL) { - retval = 1; - goto out; - } - } SMARTLIST_FOREACH_END(orport); - - out: - if (orports != NULL) { - SMARTLIST_FOREACH(orports, tor_addr_port_t *, p, tor_free(p)); - smartlist_free(orports); - orports = NULL; - } + int retval = 0; + smartlist_t *orports = node_get_all_orports(node); + retval = get_configured_bridge_by_orports_digest(node->identity, + orports) != NULL; + SMARTLIST_FOREACH(orports, tor_addr_port_t *, p, tor_free(p)); + smartlist_free(orports); return retval; } @@ -5537,7 +5385,7 @@ routerset_contains_bridge(const routerset_t *routerset, if (!routerset) return 0; - extinfo = extend_info_alloc( + extinfo = extend_info_new( NULL, bridge->identity, NULL, &bridge->addr, bridge->port); result = routerset_contains_extendinfo(routerset, extinfo); extend_info_free(extinfo); @@ -5630,12 +5478,11 @@ launch_direct_bridge_descriptor_fetch(bridge_info_t *bridge) address = tor_dup_addr(&bridge->addr); directory_initiate_command(address, &bridge->addr, - bridge->port, 0, - 0, /* does not matter */ - 1, bridge->identity, + bridge->port, 0/*no dirport*/, + bridge->identity, DIR_PURPOSE_FETCH_SERVERDESC, ROUTER_PURPOSE_BRIDGE, - 0, "authority.z", NULL, 0, 0); + DIRIND_ONEHOP, "authority.z", NULL, 0, 0); tor_free(address); } @@ -5775,21 +5622,26 @@ rewrite_node_address_for_bridge(const bridge_info_t *bridge, node_t *node) } } - /* Indicate that we prefer connecting to this bridge over the - protocol that the bridge address indicates. Last bridge - descriptor handled wins. */ - ri->ipv6_preferred = tor_addr_family(&bridge->addr) == AF_INET6; + /* Mark bridge as preferably connected to over IPv6 if its IPv6 + address is in a Bridge line and ClientPreferIPv6ORPort is + set. Unless both is true, a potential IPv6 OR port of this + bridge won't get selected. + + XXX ipv6_preferred is never reset (#6757) */ + if (get_options()->ClientPreferIPv6ORPort == 1 && + tor_addr_family(&bridge->addr) == AF_INET6) + node->ipv6_preferred = 1; /* XXXipv6 we lack support for falling back to another address for the same relay, warn the user */ if (!tor_addr_is_null(&ri->ipv6_addr)) { tor_addr_port_t ap; - router_get_pref_orport(ri, &ap); + node_get_pref_orport(node, &ap); log_notice(LD_CONFIG, "Bridge '%s' has both an IPv4 and an IPv6 address. " "Will prefer using its %s address (%s:%d).", ri->nickname, - ri->ipv6_preferred ? "IPv6" : "IPv4", + node->ipv6_preferred ? "IPv6" : "IPv4", fmt_addr(&ap.addr), ap.port); } } @@ -5979,10 +5831,7 @@ entry_guards_free_all(void) entry_guards = NULL; } clear_bridge_list(); - clear_transport_list(); smartlist_free(bridge_list); - smartlist_free(transport_list); bridge_list = NULL; - transport_list = NULL; } diff --git a/src/or/circuitbuild.h b/src/or/circuitbuild.h index 984d04a99..c3905ca21 100644 --- a/src/or/circuitbuild.h +++ b/src/or/circuitbuild.h @@ -12,21 +12,6 @@ #ifndef _TOR_CIRCUITBUILD_H #define _TOR_CIRCUITBUILD_H -/** Represents a pluggable transport proxy used by a bridge. */ -typedef struct { - /** SOCKS version: One of PROXY_SOCKS4, PROXY_SOCKS5. */ - int socks_version; - /** Name of pluggable transport protocol */ - char *name; - /** Address of proxy */ - tor_addr_t addr; - /** Port of proxy */ - uint16_t port; - /** Boolean: We are re-parsing our transport list, and we are going to remove - * this one if we don't find it in the list of configured transports. */ - unsigned marked_for_removal : 1; -} transport_t; - char *circuit_list_path(origin_circuit_t *circ, int verbose); char *circuit_list_path_for_controller(origin_circuit_t *circ); void circuit_log_path(int severity, unsigned int domain, @@ -56,13 +41,10 @@ int circuit_all_predicted_ports_handled(time_t now, int *need_uptime, int circuit_append_new_exit(origin_circuit_t *circ, extend_info_t *info); int circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *info); void onion_append_to_cpath(crypt_path_t **head_ptr, crypt_path_t *new_hop); -extend_info_t *extend_info_alloc(const char *nickname, const char *digest, +extend_info_t *extend_info_new(const char *nickname, const char *digest, crypto_pk_t *onion_key, const tor_addr_t *addr, uint16_t port); -extend_info_t *extend_info_from_router(const routerinfo_t *r, - int for_direct_connect); -extend_info_t *extend_info_from_node(const node_t *node, - int for_direct_connect); +extend_info_t *extend_info_from_node(const node_t *r, int for_direct_connect); extend_info_t *extend_info_dup(extend_info_t *info); void extend_info_free(extend_info_t *info); const node_t *build_state_get_exit_node(cpath_build_state_t *state); @@ -82,8 +64,6 @@ int getinfo_helper_entry_guards(control_connection_t *conn, void mark_bridge_list(void); void sweep_bridge_list(void); -void mark_transport_list(void); -void sweep_transport_list(void); int routerinfo_is_a_configured_bridge(const routerinfo_t *ri); int node_is_a_configured_bridge(const node_t *node); @@ -151,21 +131,14 @@ void circuit_build_times_network_circ_success(circuit_build_times_t *cbt); /* DOCDOC circuit_build_times_get_bw_scale */ int circuit_build_times_get_bw_scale(networkstatus_t *ns); -void clear_transport_list(void); -int transport_add_from_config(const tor_addr_t *addr, uint16_t port, - const char *name, int socks_ver); -int transport_add(transport_t *t); -void transport_free(transport_t *transport); -transport_t *transport_new(const tor_addr_t *addr, uint16_t port, - const char *name, int socks_ver); - /* DOCDOC find_transport_name_by_bridge_addrport */ const char *find_transport_name_by_bridge_addrport(const tor_addr_t *addr, uint16_t port); - +struct transport_t; int find_transport_by_bridge_addrport(const tor_addr_t *addr, uint16_t port, - const transport_t **transport); -transport_t *transport_get_by_name(const char *name); + const struct transport_t **transport); + +int validate_pluggable_transports_config(void); #endif diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index 7ed942c8f..d9b74bd4c 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -26,6 +26,7 @@ #include "rendcommon.h" #include "rephist.h" #include "routerlist.h" +#include "routerset.h" #include "ht.h" /********* START VARIABLES **********/ diff --git a/src/or/circuituse.c b/src/or/circuituse.c index 20f124eb4..11d581148 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -1570,7 +1570,7 @@ circuit_get_open_circ_or_launch(entry_connection_t *conn, escaped_safe_str_client(conn->socks_request->address)); return -1; } - extend_info = extend_info_alloc(conn->chosen_exit_name+1, + extend_info = extend_info_new(conn->chosen_exit_name+1, digest, NULL, &addr, conn->socks_request->port); } else { diff --git a/src/or/command.c b/src/or/command.c index abf664c1e..88603c924 100644 --- a/src/or/command.c +++ b/src/or/command.c @@ -808,7 +808,7 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) time_t now = time(NULL); long apparent_skew = 0; - uint32_t my_apparent_addr = 0; + tor_addr_t my_apparent_addr = TOR_ADDR_NULL; if (conn->link_proto < 2) { log_fn(LOG_PROTOCOL_WARN, LD_OR, @@ -868,7 +868,9 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) connection_mark_for_close(TO_CONN(conn)); return; } else if (my_addr_type == RESOLVED_TYPE_IPV4 && my_addr_len == 4) { - my_apparent_addr = ntohl(get_uint32(my_addr_ptr)); + tor_addr_from_ipv4n(&my_apparent_addr, get_uint32(my_addr_ptr)); + } else if (my_addr_type == RESOLVED_TYPE_IPV6 && my_addr_len == 16) { + tor_addr_from_ipv6_bytes(&my_apparent_addr, (const char *) my_addr_ptr); } n_other_addrs = (uint8_t) *cp++; @@ -921,7 +923,6 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) /* XXX maybe act on my_apparent_addr, if the source is sufficiently * trustworthy. */ - (void)my_apparent_addr; if (connection_or_set_state_open(conn)<0) { log_fn(LOG_PROTOCOL_WARN, LD_OR, "Got good NETINFO cell from %s:%d; but " @@ -931,10 +932,13 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn) connection_mark_for_close(TO_CONN(conn)); } else { log_info(LD_OR, "Got good NETINFO cell from %s:%d; OR connection is now " - "open, using protocol version %d. Its ID digest is %s", + "open, using protocol version %d. Its ID digest is %s. " + "Our address is apparently %s.", safe_str_client(conn->_base.address), conn->_base.port, (int)conn->link_proto, - hex_str(conn->identity_digest, DIGEST_LEN)); + hex_str(conn->identity_digest, DIGEST_LEN), + tor_addr_is_null(&my_apparent_addr) ? + "<none>" : fmt_and_decorate_addr(&my_apparent_addr)); } assert_connection_ok(TO_CONN(conn),time(NULL)); } diff --git a/src/or/config.c b/src/or/config.c index b9170f420..4557853ce 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -19,6 +19,7 @@ #include "connection_edge.h" #include "connection_or.h" #include "control.h" +#include "confparse.h" #include "cpuworker.h" #include "dirserv.h" #include "dirvote.h" @@ -27,6 +28,7 @@ #include "hibernate.h" #include "main.h" #include "networkstatus.h" +#include "nodelist.h" #include "policies.h" #include "relay.h" #include "rendclient.h" @@ -35,6 +37,8 @@ #include "router.h" #include "util.h" #include "routerlist.h" +#include "routerset.h" +#include "statefile.h" #include "transports.h" #ifdef _WIN32 #include <shlobj.h> @@ -45,48 +49,6 @@ /* From main.c */ extern int quiet_level; -/** Enumeration of types which option values can take */ -typedef enum config_type_t { - CONFIG_TYPE_STRING = 0, /**< An arbitrary string. */ - CONFIG_TYPE_FILENAME, /**< A filename: some prefixes get expanded. */ - CONFIG_TYPE_UINT, /**< A non-negative integer less than MAX_INT */ - CONFIG_TYPE_INT, /**< Any integer. */ - CONFIG_TYPE_PORT, /**< A port from 1...65535, 0 for "not set", or - * "auto". */ - CONFIG_TYPE_INTERVAL, /**< A number of seconds, with optional units*/ - CONFIG_TYPE_MSEC_INTERVAL,/**< A number of milliseconds, with optional - * units */ - CONFIG_TYPE_MEMUNIT, /**< A number of bytes, with optional units*/ - CONFIG_TYPE_DOUBLE, /**< A floating-point value */ - CONFIG_TYPE_BOOL, /**< A boolean value, expressed as 0 or 1. */ - CONFIG_TYPE_AUTOBOOL, /**< A boolean+auto value, expressed 0 for false, - * 1 for true, and -1 for auto */ - CONFIG_TYPE_ISOTIME, /**< An ISO-formatted time relative to GMT. */ - CONFIG_TYPE_CSV, /**< A list of strings, separated by commas and - * optional whitespace. */ - CONFIG_TYPE_LINELIST, /**< Uninterpreted config lines */ - CONFIG_TYPE_LINELIST_S, /**< Uninterpreted, context-sensitive config lines, - * mixed with other keywords. */ - CONFIG_TYPE_LINELIST_V, /**< Catch-all "virtual" option to summarize - * context-sensitive config lines when fetching. - */ - CONFIG_TYPE_ROUTERSET, /**< A list of router names, addrs, and fps, - * parsed into a routerset_t. */ - CONFIG_TYPE_OBSOLETE, /**< Obsolete (ignored) option. */ -} config_type_t; - -/** An abbreviation for a configuration option allowed on the command line. */ -typedef struct config_abbrev_t { - const char *abbreviated; - const char *full; - int commandline_only; - int warn; -} config_abbrev_t; - -/* Handy macro for declaring "In the config file or on the command line, - * you can abbreviate <b>tok</b>s as <b>tok</b>". */ -#define PLURAL(tok) { #tok, #tok "s", 0, 0 } - /** A list of abbreviations and aliases to map command-line options, obsolete * option names, or alternative option names, to their current values. */ static config_abbrev_t _option_abbrevs[] = { @@ -130,31 +92,10 @@ static config_abbrev_t _option_abbrevs[] = { { "HashedControlPassword", "__HashedControlSessionPassword", 1, 0}, { "StrictEntryNodes", "StrictNodes", 0, 1}, { "StrictExitNodes", "StrictNodes", 0, 1}, + { "_UseFilteringSSLBufferevents", "UseFilteringSSLBufferevents", 0, 1}, { NULL, NULL, 0, 0}, }; -/** A list of state-file "abbreviations," for compatibility. */ -static config_abbrev_t _state_abbrevs[] = { - { "AccountingBytesReadInterval", "AccountingBytesReadInInterval", 0, 0 }, - { "HelperNode", "EntryGuard", 0, 0 }, - { "HelperNodeDownSince", "EntryGuardDownSince", 0, 0 }, - { "HelperNodeUnlistedSince", "EntryGuardUnlistedSince", 0, 0 }, - { "EntryNode", "EntryGuard", 0, 0 }, - { "EntryNodeDownSince", "EntryGuardDownSince", 0, 0 }, - { "EntryNodeUnlistedSince", "EntryGuardUnlistedSince", 0, 0 }, - { NULL, NULL, 0, 0}, -}; -#undef PLURAL - -/** A variable allowed in the configuration file or on the command line. */ -typedef struct config_var_t { - const char *name; /**< The full keyword (case insensitive). */ - config_type_t type; /**< How to interpret the type and turn it into a - * value. */ - off_t var_offset; /**< Offset of the corresponding member of or_options_t. */ - const char *initvalue; /**< String (or null) describing initial value. */ -} config_var_t; - /** An entry for config_vars: "The option <b>name</b> has type * CONFIG_TYPE_<b>conftype</b>, and corresponds to * or_options_t.<b>member</b>" @@ -204,12 +145,13 @@ static config_var_t _option_vars[] = { V(AuthDirListBadExits, BOOL, "0"), V(AuthDirMaxServersPerAddr, UINT, "2"), V(AuthDirMaxServersPerAuthAddr,UINT, "5"), + V(AuthDirHasIPv6Connectivity, BOOL, "0"), VAR("AuthoritativeDirectory", BOOL, AuthoritativeDir, "0"), V(AutomapHostsOnResolve, BOOL, "0"), V(AutomapHostsSuffixes, CSV, ".onion,.exit"), V(AvoidDiskWrites, BOOL, "0"), - V(BandwidthBurst, MEMUNIT, "10 MB"), - V(BandwidthRate, MEMUNIT, "5 MB"), + V(BandwidthBurst, MEMUNIT, "1 GB"), + V(BandwidthRate, MEMUNIT, "1 GB"), V(BridgeAuthoritativeDir, BOOL, "0"), VAR("Bridge", LINELIST, Bridges, NULL), V(BridgePassword, STRING, NULL), @@ -223,8 +165,10 @@ static config_var_t _option_vars[] = { V(CircuitPriorityHalflife, DOUBLE, "-100.0"), /*negative:'Use default'*/ V(ClientDNSRejectInternalAddresses, BOOL,"1"), V(ClientOnly, BOOL, "0"), + V(ClientPreferIPv6ORPort, BOOL, "0"), V(ClientRejectInternalAddresses, BOOL, "1"), V(ClientTransportPlugin, LINELIST, NULL), + V(ClientUseIPv6, BOOL, "0"), V(ConsensusParams, STRING, NULL), V(ConnLimit, UINT, "1000"), V(ConnDirectionStatistics, BOOL, "0"), @@ -446,7 +390,7 @@ static config_var_t _option_vars[] = { VAR("VersioningAuthoritativeDirectory",BOOL,VersioningAuthoritativeDir, "0"), V(VirtualAddrNetwork, STRING, "127.192.0.0/10"), V(WarnPlaintextPorts, CSV, "23,109,110,143"), - V(_UseFilteringSSLBufferevents, BOOL, "0"), + V(UseFilteringSSLBufferevents, BOOL, "0"), VAR("__ReloadTorrcOnSIGHUP", BOOL, ReloadTorrcOnSIGHUP, "1"), VAR("__AllDirActionsPrivate", BOOL, AllDirActionsPrivate, "0"), VAR("__DisablePredictedCircuits",BOOL,DisablePredictedCircuits, "0"), @@ -456,7 +400,7 @@ static config_var_t _option_vars[] = { VAR("__OwningControllerProcess",STRING,OwningControllerProcess, NULL), V(MinUptimeHidServDirectoryV2, INTERVAL, "25 hours"), V(VoteOnHidServDirectoriesV2, BOOL, "1"), - V(_UsingTestNetworkDefaults, BOOL, "0"), + VAR("___UsingTestNetworkDefaults", BOOL, _UsingTestNetworkDefaults, "0"), { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; @@ -484,127 +428,18 @@ static const config_var_t testing_tor_network_defaults[] = { V(TestingAuthDirTimeToLearnReachability, INTERVAL, "0 minutes"), V(TestingEstimatedDescriptorPropagationTime, INTERVAL, "0 minutes"), V(MinUptimeHidServDirectoryV2, INTERVAL, "0 minutes"), - V(_UsingTestNetworkDefaults, BOOL, "1"), + VAR("___UsingTestNetworkDefaults", BOOL, _UsingTestNetworkDefaults, "1"), { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } }; -#undef VAR - -#define VAR(name,conftype,member,initvalue) \ - { name, CONFIG_TYPE_ ## conftype, STRUCT_OFFSET(or_state_t, member), \ - initvalue } - -/** Array of "state" variables saved to the ~/.tor/state file. */ -static config_var_t _state_vars[] = { - /* Remember to document these in state-contents.txt ! */ - - V(AccountingBytesReadInInterval, MEMUNIT, NULL), - V(AccountingBytesWrittenInInterval, MEMUNIT, NULL), - V(AccountingExpectedUsage, MEMUNIT, NULL), - V(AccountingIntervalStart, ISOTIME, NULL), - V(AccountingSecondsActive, INTERVAL, NULL), - V(AccountingSecondsToReachSoftLimit,INTERVAL, NULL), - V(AccountingSoftLimitHitAt, ISOTIME, NULL), - V(AccountingBytesAtSoftLimit, MEMUNIT, NULL), - - VAR("EntryGuard", LINELIST_S, EntryGuards, NULL), - VAR("EntryGuardDownSince", LINELIST_S, EntryGuards, NULL), - VAR("EntryGuardUnlistedSince", LINELIST_S, EntryGuards, NULL), - VAR("EntryGuardAddedBy", LINELIST_S, EntryGuards, NULL), - VAR("EntryGuardPathBias", LINELIST_S, EntryGuards, NULL), - V(EntryGuards, LINELIST_V, NULL), - - VAR("TransportProxy", LINELIST_S, TransportProxies, NULL), - V(TransportProxies, LINELIST_V, NULL), - - V(BWHistoryReadEnds, ISOTIME, NULL), - V(BWHistoryReadInterval, UINT, "900"), - V(BWHistoryReadValues, CSV, ""), - V(BWHistoryReadMaxima, CSV, ""), - V(BWHistoryWriteEnds, ISOTIME, NULL), - V(BWHistoryWriteInterval, UINT, "900"), - V(BWHistoryWriteValues, CSV, ""), - V(BWHistoryWriteMaxima, CSV, ""), - V(BWHistoryDirReadEnds, ISOTIME, NULL), - V(BWHistoryDirReadInterval, UINT, "900"), - V(BWHistoryDirReadValues, CSV, ""), - V(BWHistoryDirReadMaxima, CSV, ""), - V(BWHistoryDirWriteEnds, ISOTIME, NULL), - V(BWHistoryDirWriteInterval, UINT, "900"), - V(BWHistoryDirWriteValues, CSV, ""), - V(BWHistoryDirWriteMaxima, CSV, ""), - - V(TorVersion, STRING, NULL), - - V(LastRotatedOnionKey, ISOTIME, NULL), - V(LastWritten, ISOTIME, NULL), - - V(TotalBuildTimes, UINT, NULL), - V(CircuitBuildAbandonedCount, UINT, "0"), - VAR("CircuitBuildTimeBin", LINELIST_S, BuildtimeHistogram, NULL), - VAR("BuildtimeHistogram", LINELIST_V, BuildtimeHistogram, NULL), - { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } -}; #undef VAR #undef V #undef OBSOLETE -/** Represents an English description of a configuration variable; used when - * generating configuration file comments. */ -typedef struct config_var_description_t { - const char *name; - const char *description; -} config_var_description_t; - -/** Type of a callback to validate whether a given configuration is - * well-formed and consistent. See options_trial_assign() for documentation - * of arguments. */ -typedef int (*validate_fn_t)(void*,void*,int,char**); - -/** Information on the keys, value types, key-to-struct-member mappings, - * variable descriptions, validation functions, and abbreviations for a - * configuration or storage format. */ -typedef struct { - size_t size; /**< Size of the struct that everything gets parsed into. */ - uint32_t magic; /**< Required 'magic value' to make sure we have a struct - * of the right type. */ - off_t magic_offset; /**< Offset of the magic value within the struct. */ - config_abbrev_t *abbrevs; /**< List of abbreviations that we expand when - * parsing this format. */ - config_var_t *vars; /**< List of variables we recognize, their default - * values, and where we stick them in the structure. */ - validate_fn_t validate_fn; /**< Function to validate config. */ - /** If present, extra is a LINELIST variable for unrecognized - * lines. Otherwise, unrecognized lines are an error. */ - config_var_t *extra; -} config_format_t; - -/** Macro: assert that <b>cfg</b> has the right magic field for format - * <b>fmt</b>. */ -#define CHECK(fmt, cfg) STMT_BEGIN \ - tor_assert(fmt && cfg); \ - tor_assert((fmt)->magic == \ - *(uint32_t*)STRUCT_VAR_P(cfg,fmt->magic_offset)); \ - STMT_END - #ifdef _WIN32 static char *get_windows_conf_root(void); #endif -static void config_line_append(config_line_t **lst, - const char *key, const char *val); -static void option_clear(const config_format_t *fmt, or_options_t *options, - const config_var_t *var); -static void option_reset(const config_format_t *fmt, or_options_t *options, - const config_var_t *var, int use_defaults); -static void config_free(const config_format_t *fmt, void *options); -static int config_lines_eq(config_line_t *a, config_line_t *b); -static int config_count_key(const config_line_t *a, const char *key); -static int option_is_same(const config_format_t *fmt, - const or_options_t *o1, const or_options_t *o2, - const char *name); -static or_options_t *options_dup(const config_format_t *fmt, - const or_options_t *old); static int options_validate(or_options_t *old_options, or_options_t *options, int from_setconf, char **msg); @@ -635,18 +470,8 @@ static int check_server_ports(const smartlist_t *ports, static int validate_data_directory(or_options_t *options); static int write_configuration_file(const char *fname, const or_options_t *options); -static config_line_t *get_assigned_option(const config_format_t *fmt, - const void *options, const char *key, - int escape_val); -static void config_init(const config_format_t *fmt, void *options); -static int or_state_validate(or_state_t *old_options, or_state_t *options, - int from_setconf, char **msg); -static int or_state_load(void); static int options_init_logs(or_options_t *options, int validate_only); -static uint64_t config_parse_memunit(const char *s, int *ok); -static int config_parse_msec_interval(const char *s, int *ok); -static int config_parse_interval(const char *s, int *ok); static void init_libevent(const or_options_t *options); static int opt_streq(const char *s1, const char *s2); @@ -664,26 +489,6 @@ static config_format_t options_format = { NULL }; -/** Magic value for or_state_t. */ -#define OR_STATE_MAGIC 0x57A73f57 - -/** "Extra" variable in the state that receives lines we can't parse. This - * lets us preserve options from versions of Tor newer than us. */ -static config_var_t state_extra_var = { - "__extra", CONFIG_TYPE_LINELIST, STRUCT_OFFSET(or_state_t, ExtraLines), NULL -}; - -/** Configuration format for or_state_t. */ -static const config_format_t state_format = { - sizeof(or_state_t), - OR_STATE_MAGIC, - STRUCT_OFFSET(or_state_t, _magic), - _state_abbrevs, - _state_vars, - (validate_fn_t)or_state_validate, - &state_extra_var, -}; - /* * Functions to read and write the global options pointer. */ @@ -697,8 +502,6 @@ static or_options_t *global_default_options = NULL; static char *torrc_fname = NULL; /** Name of the most recently read torrc-defaults file.*/ static char *torrc_defaults_fname; -/** Persistent serialized state. */ -static or_state_t *global_state = NULL; /** Configuration Options set by command line. */ static config_line_t *global_cmdline_options = NULL; /** Contents of most recently read DirPortFrontPage file. */ @@ -713,16 +516,6 @@ get_dirportfrontpage(void) return global_dirfrontpagecontents; } -/** Allocate an empty configuration object of a given format type. */ -static void * -config_alloc(const config_format_t *fmt) -{ - void *opts = tor_malloc_zero(fmt->size); - *(uint32_t*)STRUCT_VAR_P(opts, fmt->magic_offset) = fmt->magic; - CHECK(fmt, opts); - return opts; -} - /** Return the currently configured options. */ or_options_t * get_options_mutable(void) @@ -773,8 +566,9 @@ set_options(or_options_t *new_val, char **msg) var->type == CONFIG_TYPE_OBSOLETE) { continue; } - if (!option_is_same(&options_format, new_val, old_options, var_name)) { - line = get_assigned_option(&options_format, new_val, var_name, 1); + if (!config_is_same(&options_format, new_val, old_options, var_name)) { + line = config_get_assigned_option(&options_format, new_val, + var_name, 1); if (line) { for (; line; line = line->next) { @@ -863,9 +657,6 @@ config_free_all(void) or_options_free(global_default_options); global_default_options = NULL; - config_free(&state_format, global_state); - global_state = NULL; - config_free_lines(global_cmdline_options); global_cmdline_options = NULL; @@ -1451,7 +1242,7 @@ options_act(const or_options_t *old_options) } /* Load state */ - if (! global_state && running_tor) { + if (! or_state_loaded() && running_tor) { if (or_state_load()) return -1; rep_hist_load_mtbf_data(time(NULL)); @@ -1554,7 +1345,7 @@ options_act(const or_options_t *old_options) monitor_owning_controller_process(options->OwningControllerProcess); /* reload keys as needed for rendezvous services. */ - if (rend_service_load_keys()<0) { + if (rend_service_load_all_keys()<0) { log_warn(LD_GENERAL,"Error loading rendezvous service keys"); return -1; } @@ -1844,42 +1635,6 @@ options_act(const or_options_t *old_options) return 0; } -/* - * Functions to parse config options - */ - -/** If <b>option</b> is an official abbreviation for a longer option, - * return the longer option. Otherwise return <b>option</b>. - * If <b>command_line</b> is set, apply all abbreviations. Otherwise, only - * apply abbreviations that work for the config file and the command line. - * If <b>warn_obsolete</b> is set, warn about deprecated names. */ -static const char * -expand_abbrev(const config_format_t *fmt, const char *option, int command_line, - int warn_obsolete) -{ - int i; - if (! fmt->abbrevs) - return option; - for (i=0; fmt->abbrevs[i].abbreviated; ++i) { - /* Abbreviations are case insensitive. */ - if (!strcasecmp(option,fmt->abbrevs[i].abbreviated) && - (command_line || !fmt->abbrevs[i].commandline_only)) { - if (warn_obsolete && fmt->abbrevs[i].warn) { - log_warn(LD_CONFIG, - "The configuration option '%s' is deprecated; " - "use '%s' instead.", - fmt->abbrevs[i].abbreviated, - fmt->abbrevs[i].full); - } - /* Keep going through the list in case we want to rewrite it more. - * (We could imagine recursing here, but I don't want to get the - * user into an infinite loop if we craft our list wrong.) */ - option = fmt->abbrevs[i].full; - } - } - return option; -} - /** Helper: Read a list of configuration options from the command line. * If successful, put them in *<b>result</b> and return 0, and return * -1 and leave *<b>result</b> alone. */ @@ -1939,7 +1694,7 @@ config_get_commandlines(int argc, char **argv, config_line_t **result) return -1; } - (*new)->key = tor_strdup(expand_abbrev(&options_format, s, 1, 1)); + (*new)->key = tor_strdup(config_expand_abbrev(&options_format, s, 1, 1)); (*new)->value = want_arg ? tor_strdup(argv[i+1]) : tor_strdup(""); (*new)->command = command; (*new)->next = NULL; @@ -1953,444 +1708,6 @@ config_get_commandlines(int argc, char **argv, config_line_t **result) return 0; } -/** Helper: allocate a new configuration option mapping 'key' to 'val', - * append it to *<b>lst</b>. */ -static void -config_line_append(config_line_t **lst, - const char *key, - const char *val) -{ - config_line_t *newline; - - newline = tor_malloc_zero(sizeof(config_line_t)); - newline->key = tor_strdup(key); - newline->value = tor_strdup(val); - newline->next = NULL; - while (*lst) - lst = &((*lst)->next); - - (*lst) = newline; -} - -/** Helper: parse the config string and strdup into key/value - * strings. Set *result to the list, or NULL if parsing the string - * failed. Return 0 on success, -1 on failure. Warn and ignore any - * misformatted lines. - * - * If <b>extended</b> is set, then treat keys beginning with / and with + as - * indicating "clear" and "append" respectively. */ -int -config_get_lines(const char *string, config_line_t **result, int extended) -{ - config_line_t *list = NULL, **next; - char *k, *v; - - next = &list; - do { - k = v = NULL; - string = parse_config_line_from_str(string, &k, &v); - if (!string) { - config_free_lines(list); - tor_free(k); - tor_free(v); - return -1; - } - if (k && v) { - unsigned command = CONFIG_LINE_NORMAL; - if (extended) { - if (k[0] == '+') { - char *k_new = tor_strdup(k+1); - tor_free(k); - k = k_new; - command = CONFIG_LINE_APPEND; - } else if (k[0] == '/') { - char *k_new = tor_strdup(k+1); - tor_free(k); - k = k_new; - tor_free(v); - v = tor_strdup(""); - command = CONFIG_LINE_CLEAR; - } - } - /* This list can get long, so we keep a pointer to the end of it - * rather than using config_line_append over and over and getting - * n^2 performance. */ - *next = tor_malloc_zero(sizeof(config_line_t)); - (*next)->key = k; - (*next)->value = v; - (*next)->next = NULL; - (*next)->command = command; - next = &((*next)->next); - } else { - tor_free(k); - tor_free(v); - } - } while (*string); - - *result = list; - return 0; -} - -/** - * Free all the configuration lines on the linked list <b>front</b>. - */ -void -config_free_lines(config_line_t *front) -{ - config_line_t *tmp; - - while (front) { - tmp = front; - front = tmp->next; - - tor_free(tmp->key); - tor_free(tmp->value); - tor_free(tmp); - } -} - -/** As config_find_option, but return a non-const pointer. */ -static config_var_t * -config_find_option_mutable(config_format_t *fmt, const char *key) -{ - int i; - size_t keylen = strlen(key); - if (!keylen) - return NULL; /* if they say "--" on the command line, it's not an option */ - /* First, check for an exact (case-insensitive) match */ - for (i=0; fmt->vars[i].name; ++i) { - if (!strcasecmp(key, fmt->vars[i].name)) { - return &fmt->vars[i]; - } - } - /* If none, check for an abbreviated match */ - for (i=0; fmt->vars[i].name; ++i) { - if (!strncasecmp(key, fmt->vars[i].name, keylen)) { - log_warn(LD_CONFIG, "The abbreviation '%s' is deprecated. " - "Please use '%s' instead", - key, fmt->vars[i].name); - return &fmt->vars[i]; - } - } - /* Okay, unrecognized option */ - return NULL; -} - -/** If <b>key</b> is a configuration option, return the corresponding const - * config_var_t. Otherwise, if <b>key</b> is a non-standard abbreviation, - * warn, and return the corresponding const config_var_t. Otherwise return - * NULL. - */ -static const config_var_t * -config_find_option(const config_format_t *fmt, const char *key) -{ - return config_find_option_mutable((config_format_t*)fmt, key); -} - -/** Return the number of option entries in <b>fmt</b>. */ -static int -config_count_options(const config_format_t *fmt) -{ - int i; - for (i=0; fmt->vars[i].name; ++i) - ; - return i; -} - -/* - * Functions to assign config options. - */ - -/** <b>c</b>-\>key is known to be a real key. Update <b>options</b> - * with <b>c</b>-\>value and return 0, or return -1 if bad value. - * - * Called from config_assign_line() and option_reset(). - */ -static int -config_assign_value(const config_format_t *fmt, or_options_t *options, - config_line_t *c, char **msg) -{ - int i, ok; - const config_var_t *var; - void *lvalue; - - CHECK(fmt, options); - - var = config_find_option(fmt, c->key); - tor_assert(var); - - lvalue = STRUCT_VAR_P(options, var->var_offset); - - switch (var->type) { - - case CONFIG_TYPE_PORT: - if (!strcasecmp(c->value, "auto")) { - *(int *)lvalue = CFG_AUTO_PORT; - break; - } - /* fall through */ - case CONFIG_TYPE_INT: - case CONFIG_TYPE_UINT: - i = (int)tor_parse_long(c->value, 10, - var->type==CONFIG_TYPE_INT ? INT_MIN : 0, - var->type==CONFIG_TYPE_PORT ? 65535 : INT_MAX, - &ok, NULL); - if (!ok) { - tor_asprintf(msg, - "Int keyword '%s %s' is malformed or out of bounds.", - c->key, c->value); - return -1; - } - *(int *)lvalue = i; - break; - - case CONFIG_TYPE_INTERVAL: { - i = config_parse_interval(c->value, &ok); - if (!ok) { - tor_asprintf(msg, - "Interval '%s %s' is malformed or out of bounds.", - c->key, c->value); - return -1; - } - *(int *)lvalue = i; - break; - } - - case CONFIG_TYPE_MSEC_INTERVAL: { - i = config_parse_msec_interval(c->value, &ok); - if (!ok) { - tor_asprintf(msg, - "Msec interval '%s %s' is malformed or out of bounds.", - c->key, c->value); - return -1; - } - *(int *)lvalue = i; - break; - } - - case CONFIG_TYPE_MEMUNIT: { - uint64_t u64 = config_parse_memunit(c->value, &ok); - if (!ok) { - tor_asprintf(msg, - "Value '%s %s' is malformed or out of bounds.", - c->key, c->value); - return -1; - } - *(uint64_t *)lvalue = u64; - break; - } - - case CONFIG_TYPE_BOOL: - i = (int)tor_parse_long(c->value, 10, 0, 1, &ok, NULL); - if (!ok) { - tor_asprintf(msg, - "Boolean '%s %s' expects 0 or 1.", - c->key, c->value); - return -1; - } - *(int *)lvalue = i; - break; - - case CONFIG_TYPE_AUTOBOOL: - if (!strcmp(c->value, "auto")) - *(int *)lvalue = -1; - else if (!strcmp(c->value, "0")) - *(int *)lvalue = 0; - else if (!strcmp(c->value, "1")) - *(int *)lvalue = 1; - else { - tor_asprintf(msg, "Boolean '%s %s' expects 0, 1, or 'auto'.", - c->key, c->value); - return -1; - } - break; - - case CONFIG_TYPE_STRING: - case CONFIG_TYPE_FILENAME: - tor_free(*(char **)lvalue); - *(char **)lvalue = tor_strdup(c->value); - break; - - case CONFIG_TYPE_DOUBLE: - *(double *)lvalue = atof(c->value); - break; - - case CONFIG_TYPE_ISOTIME: - if (parse_iso_time(c->value, (time_t *)lvalue)) { - tor_asprintf(msg, - "Invalid time '%s' for keyword '%s'", c->value, c->key); - return -1; - } - break; - - case CONFIG_TYPE_ROUTERSET: - if (*(routerset_t**)lvalue) { - routerset_free(*(routerset_t**)lvalue); - } - *(routerset_t**)lvalue = routerset_new(); - if (routerset_parse(*(routerset_t**)lvalue, c->value, c->key)<0) { - tor_asprintf(msg, "Invalid exit list '%s' for option '%s'", - c->value, c->key); - return -1; - } - break; - - case CONFIG_TYPE_CSV: - if (*(smartlist_t**)lvalue) { - SMARTLIST_FOREACH(*(smartlist_t**)lvalue, char *, cp, tor_free(cp)); - smartlist_clear(*(smartlist_t**)lvalue); - } else { - *(smartlist_t**)lvalue = smartlist_new(); - } - - smartlist_split_string(*(smartlist_t**)lvalue, c->value, ",", - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); - break; - - case CONFIG_TYPE_LINELIST: - case CONFIG_TYPE_LINELIST_S: - { - config_line_t *lastval = *(config_line_t**)lvalue; - if (lastval && lastval->fragile) { - if (c->command != CONFIG_LINE_APPEND) { - config_free_lines(lastval); - *(config_line_t**)lvalue = NULL; - } else { - lastval->fragile = 0; - } - } - - config_line_append((config_line_t**)lvalue, c->key, c->value); - } - break; - case CONFIG_TYPE_OBSOLETE: - log_warn(LD_CONFIG, "Skipping obsolete configuration option '%s'", c->key); - break; - case CONFIG_TYPE_LINELIST_V: - tor_asprintf(msg, - "You may not provide a value for virtual option '%s'", c->key); - return -1; - default: - tor_assert(0); - break; - } - return 0; -} - -/** Mark every linelist in <b>options</b> "fragile", so that fresh assignments - * to it will replace old ones. */ -static void -config_mark_lists_fragile(const config_format_t *fmt, or_options_t *options) -{ - int i; - tor_assert(fmt); - tor_assert(options); - - for (i = 0; fmt->vars[i].name; ++i) { - const config_var_t *var = &fmt->vars[i]; - config_line_t *list; - if (var->type != CONFIG_TYPE_LINELIST && - var->type != CONFIG_TYPE_LINELIST_V) - continue; - - list = *(config_line_t **)STRUCT_VAR_P(options, var->var_offset); - if (list) - list->fragile = 1; - } -} - -/** If <b>c</b> is a syntactically valid configuration line, update - * <b>options</b> with its value and return 0. Otherwise return -1 for bad - * key, -2 for bad value. - * - * If <b>clear_first</b> is set, clear the value first. Then if - * <b>use_defaults</b> is set, set the value to the default. - * - * Called from config_assign(). - */ -static int -config_assign_line(const config_format_t *fmt, or_options_t *options, - config_line_t *c, int use_defaults, - int clear_first, bitarray_t *options_seen, char **msg) -{ - const config_var_t *var; - - CHECK(fmt, options); - - var = config_find_option(fmt, c->key); - if (!var) { - if (fmt->extra) { - void *lvalue = STRUCT_VAR_P(options, fmt->extra->var_offset); - log_info(LD_CONFIG, - "Found unrecognized option '%s'; saving it.", c->key); - config_line_append((config_line_t**)lvalue, c->key, c->value); - return 0; - } else { - tor_asprintf(msg, - "Unknown option '%s'. Failing.", c->key); - return -1; - } - } - - /* Put keyword into canonical case. */ - if (strcmp(var->name, c->key)) { - tor_free(c->key); - c->key = tor_strdup(var->name); - } - - if (!strlen(c->value)) { - /* reset or clear it, then return */ - if (!clear_first) { - if ((var->type == CONFIG_TYPE_LINELIST || - var->type == CONFIG_TYPE_LINELIST_S) && - c->command != CONFIG_LINE_CLEAR) { - /* We got an empty linelist from the torrc or command line. - As a special case, call this an error. Warn and ignore. */ - log_warn(LD_CONFIG, - "Linelist option '%s' has no value. Skipping.", c->key); - } else { /* not already cleared */ - option_reset(fmt, options, var, use_defaults); - } - } - return 0; - } else if (c->command == CONFIG_LINE_CLEAR && !clear_first) { - option_reset(fmt, options, var, use_defaults); - } - - if (options_seen && (var->type != CONFIG_TYPE_LINELIST && - var->type != CONFIG_TYPE_LINELIST_S)) { - /* We're tracking which options we've seen, and this option is not - * supposed to occur more than once. */ - int var_index = (int)(var - fmt->vars); - if (bitarray_is_set(options_seen, var_index)) { - log_warn(LD_CONFIG, "Option '%s' used more than once; all but the last " - "value will be ignored.", var->name); - } - bitarray_set(options_seen, var_index); - } - - if (config_assign_value(fmt, options, c, msg) < 0) - return -2; - return 0; -} - -/** Restore the option named <b>key</b> in options to its default value. - * Called from config_assign(). */ -static void -config_reset_line(const config_format_t *fmt, or_options_t *options, - const char *key, int use_defaults) -{ - const config_var_t *var; - - CHECK(fmt, options); - - var = config_find_option(fmt, key); - if (!var) - return; /* give error on next pass. */ - - option_reset(fmt, options, var, use_defaults); -} - /** Return true iff key is a valid configuration option. */ int option_is_recognized(const char *key) @@ -2413,287 +1730,7 @@ option_get_canonical_name(const char *key) config_line_t * option_get_assignment(const or_options_t *options, const char *key) { - return get_assigned_option(&options_format, options, key, 1); -} - -/** Return true iff value needs to be quoted and escaped to be used in - * a configuration file. */ -static int -config_value_needs_escape(const char *value) -{ - if (*value == '\"') - return 1; - while (*value) { - switch (*value) - { - case '\r': - case '\n': - case '#': - /* Note: quotes and backspaces need special handling when we are using - * quotes, not otherwise, so they don't trigger escaping on their - * own. */ - return 1; - default: - if (!TOR_ISPRINT(*value)) - return 1; - } - ++value; - } - return 0; -} - -/** Return a newly allocated deep copy of the lines in <b>inp</b>. */ -static config_line_t * -config_lines_dup(const config_line_t *inp) -{ - config_line_t *result = NULL; - config_line_t **next_out = &result; - while (inp) { - *next_out = tor_malloc_zero(sizeof(config_line_t)); - (*next_out)->key = tor_strdup(inp->key); - (*next_out)->value = tor_strdup(inp->value); - inp = inp->next; - next_out = &((*next_out)->next); - } - (*next_out) = NULL; - return result; -} - -/** Return newly allocated line or lines corresponding to <b>key</b> in the - * configuration <b>options</b>. If <b>escape_val</b> is true and a - * value needs to be quoted before it's put in a config file, quote and - * escape that value. Return NULL if no such key exists. */ -static config_line_t * -get_assigned_option(const config_format_t *fmt, const void *options, - const char *key, int escape_val) -{ - const config_var_t *var; - const void *value; - config_line_t *result; - tor_assert(options && key); - - CHECK(fmt, options); - - var = config_find_option(fmt, key); - if (!var) { - log_warn(LD_CONFIG, "Unknown option '%s'. Failing.", key); - return NULL; - } - value = STRUCT_VAR_P(options, var->var_offset); - - result = tor_malloc_zero(sizeof(config_line_t)); - result->key = tor_strdup(var->name); - switch (var->type) - { - case CONFIG_TYPE_STRING: - case CONFIG_TYPE_FILENAME: - if (*(char**)value) { - result->value = tor_strdup(*(char**)value); - } else { - tor_free(result->key); - tor_free(result); - return NULL; - } - break; - case CONFIG_TYPE_ISOTIME: - if (*(time_t*)value) { - result->value = tor_malloc(ISO_TIME_LEN+1); - format_iso_time(result->value, *(time_t*)value); - } else { - tor_free(result->key); - tor_free(result); - } - escape_val = 0; /* Can't need escape. */ - break; - case CONFIG_TYPE_PORT: - if (*(int*)value == CFG_AUTO_PORT) { - result->value = tor_strdup("auto"); - escape_val = 0; - break; - } - /* fall through */ - case CONFIG_TYPE_INTERVAL: - case CONFIG_TYPE_MSEC_INTERVAL: - case CONFIG_TYPE_UINT: - case CONFIG_TYPE_INT: - /* This means every or_options_t uint or bool element - * needs to be an int. Not, say, a uint16_t or char. */ - tor_asprintf(&result->value, "%d", *(int*)value); - escape_val = 0; /* Can't need escape. */ - break; - case CONFIG_TYPE_MEMUNIT: - tor_asprintf(&result->value, U64_FORMAT, - U64_PRINTF_ARG(*(uint64_t*)value)); - escape_val = 0; /* Can't need escape. */ - break; - case CONFIG_TYPE_DOUBLE: - tor_asprintf(&result->value, "%f", *(double*)value); - escape_val = 0; /* Can't need escape. */ - break; - - case CONFIG_TYPE_AUTOBOOL: - if (*(int*)value == -1) { - result->value = tor_strdup("auto"); - escape_val = 0; - break; - } - /* fall through */ - case CONFIG_TYPE_BOOL: - result->value = tor_strdup(*(int*)value ? "1" : "0"); - escape_val = 0; /* Can't need escape. */ - break; - case CONFIG_TYPE_ROUTERSET: - result->value = routerset_to_string(*(routerset_t**)value); - break; - case CONFIG_TYPE_CSV: - if (*(smartlist_t**)value) - result->value = - smartlist_join_strings(*(smartlist_t**)value, ",", 0, NULL); - else - result->value = tor_strdup(""); - break; - case CONFIG_TYPE_OBSOLETE: - log_fn(LOG_PROTOCOL_WARN, LD_CONFIG, - "You asked me for the value of an obsolete config option '%s'.", - key); - tor_free(result->key); - tor_free(result); - return NULL; - case CONFIG_TYPE_LINELIST_S: - log_warn(LD_CONFIG, - "Can't return context-sensitive '%s' on its own", key); - tor_free(result->key); - tor_free(result); - return NULL; - case CONFIG_TYPE_LINELIST: - case CONFIG_TYPE_LINELIST_V: - tor_free(result->key); - tor_free(result); - result = config_lines_dup(*(const config_line_t**)value); - break; - default: - tor_free(result->key); - tor_free(result); - log_warn(LD_BUG,"Unknown type %d for known key '%s'", - var->type, key); - return NULL; - } - - if (escape_val) { - config_line_t *line; - for (line = result; line; line = line->next) { - if (line->value && config_value_needs_escape(line->value)) { - char *newval = esc_for_log(line->value); - tor_free(line->value); - line->value = newval; - } - } - } - - return result; -} - -/** Iterate through the linked list of requested options <b>list</b>. - * For each item, convert as appropriate and assign to <b>options</b>. - * If an item is unrecognized, set *msg and return -1 immediately, - * else return 0 for success. - * - * If <b>clear_first</b>, interpret config options as replacing (not - * extending) their previous values. If <b>clear_first</b> is set, - * then <b>use_defaults</b> to decide if you set to defaults after - * clearing, or make the value 0 or NULL. - * - * Here are the use cases: - * 1. A non-empty AllowInvalid line in your torrc. Appends to current - * if linelist, replaces current if csv. - * 2. An empty AllowInvalid line in your torrc. Should clear it. - * 3. "RESETCONF AllowInvalid" sets it to default. - * 4. "SETCONF AllowInvalid" makes it NULL. - * 5. "SETCONF AllowInvalid=foo" clears it and sets it to "foo". - * - * Use_defaults Clear_first - * 0 0 "append" - * 1 0 undefined, don't use - * 0 1 "set to null first" - * 1 1 "set to defaults first" - * Return 0 on success, -1 on bad key, -2 on bad value. - * - * As an additional special case, if a LINELIST config option has - * no value and clear_first is 0, then warn and ignore it. - */ - -/* -There are three call cases for config_assign() currently. - -Case one: Torrc entry -options_init_from_torrc() calls config_assign(0, 0) - calls config_assign_line(0, 0). - if value is empty, calls option_reset(0) and returns. - calls config_assign_value(), appends. - -Case two: setconf -options_trial_assign() calls config_assign(0, 1) - calls config_reset_line(0) - calls option_reset(0) - calls option_clear(). - calls config_assign_line(0, 1). - if value is empty, returns. - calls config_assign_value(), appends. - -Case three: resetconf -options_trial_assign() calls config_assign(1, 1) - calls config_reset_line(1) - calls option_reset(1) - calls option_clear(). - calls config_assign_value(default) - calls config_assign_line(1, 1). - returns. -*/ -static int -config_assign(const config_format_t *fmt, void *options, config_line_t *list, - int use_defaults, int clear_first, char **msg) -{ - config_line_t *p; - bitarray_t *options_seen; - const int n_options = config_count_options(fmt); - - CHECK(fmt, options); - - /* pass 1: normalize keys */ - for (p = list; p; p = p->next) { - const char *full = expand_abbrev(fmt, p->key, 0, 1); - if (strcmp(full,p->key)) { - tor_free(p->key); - p->key = tor_strdup(full); - } - } - - /* pass 2: if we're reading from a resetting source, clear all - * mentioned config options, and maybe set to their defaults. */ - if (clear_first) { - for (p = list; p; p = p->next) - config_reset_line(fmt, options, p->key, use_defaults); - } - - options_seen = bitarray_init_zero(n_options); - /* pass 3: assign. */ - while (list) { - int r; - if ((r=config_assign_line(fmt, options, list, use_defaults, - clear_first, options_seen, msg))) { - bitarray_free(options_seen); - return r; - } - list = list->next; - } - bitarray_free(options_seen); - - /** Now we're done assigning a group of options to the configuration. - * Subsequent group assignments should _replace_ linelists, not extend - * them. */ - config_mark_lists_fragile(fmt, options); - - return 0; + return config_get_assigned_option(&options_format, options, key, 1); } /** Try assigning <b>list</b> to the global options. You do this by duping @@ -2710,7 +1747,7 @@ options_trial_assign(config_line_t *list, int use_defaults, int clear_first, char **msg) { int r; - or_options_t *trial_options = options_dup(&options_format, get_options()); + or_options_t *trial_options = config_dup(&options_format, get_options()); if ((r=config_assign(&options_format, trial_options, list, use_defaults, clear_first, msg)) < 0) { @@ -2737,90 +1774,6 @@ options_trial_assign(config_line_t *list, int use_defaults, return SETOPT_OK; } -/** Reset config option <b>var</b> to 0, 0.0, NULL, or the equivalent. - * Called from option_reset() and config_free(). */ -static void -option_clear(const config_format_t *fmt, or_options_t *options, - const config_var_t *var) -{ - void *lvalue = STRUCT_VAR_P(options, var->var_offset); - (void)fmt; /* unused */ - switch (var->type) { - case CONFIG_TYPE_STRING: - case CONFIG_TYPE_FILENAME: - tor_free(*(char**)lvalue); - break; - case CONFIG_TYPE_DOUBLE: - *(double*)lvalue = 0.0; - break; - case CONFIG_TYPE_ISOTIME: - *(time_t*)lvalue = 0; - break; - case CONFIG_TYPE_INTERVAL: - case CONFIG_TYPE_MSEC_INTERVAL: - case CONFIG_TYPE_UINT: - case CONFIG_TYPE_INT: - case CONFIG_TYPE_PORT: - case CONFIG_TYPE_BOOL: - *(int*)lvalue = 0; - break; - case CONFIG_TYPE_AUTOBOOL: - *(int*)lvalue = -1; - break; - case CONFIG_TYPE_MEMUNIT: - *(uint64_t*)lvalue = 0; - break; - case CONFIG_TYPE_ROUTERSET: - if (*(routerset_t**)lvalue) { - routerset_free(*(routerset_t**)lvalue); - *(routerset_t**)lvalue = NULL; - } - break; - case CONFIG_TYPE_CSV: - if (*(smartlist_t**)lvalue) { - SMARTLIST_FOREACH(*(smartlist_t **)lvalue, char *, cp, tor_free(cp)); - smartlist_free(*(smartlist_t **)lvalue); - *(smartlist_t **)lvalue = NULL; - } - break; - case CONFIG_TYPE_LINELIST: - case CONFIG_TYPE_LINELIST_S: - config_free_lines(*(config_line_t **)lvalue); - *(config_line_t **)lvalue = NULL; - break; - case CONFIG_TYPE_LINELIST_V: - /* handled by linelist_s. */ - break; - case CONFIG_TYPE_OBSOLETE: - break; - } -} - -/** Clear the option indexed by <b>var</b> in <b>options</b>. Then if - * <b>use_defaults</b>, set it to its default value. - * Called by config_init() and option_reset_line() and option_assign_line(). */ -static void -option_reset(const config_format_t *fmt, or_options_t *options, - const config_var_t *var, int use_defaults) -{ - config_line_t *c; - char *msg = NULL; - CHECK(fmt, options); - option_clear(fmt, options, var); /* clear it first */ - if (!use_defaults) - return; /* all done */ - if (var->initvalue) { - c = tor_malloc_zero(sizeof(config_line_t)); - c->key = tor_strdup(var->name); - c->value = tor_strdup(var->initvalue); - if (config_assign_value(fmt, options, c, &msg) < 0) { - log_warn(LD_BUG, "Failed to assign default: %s", msg); - tor_free(msg); /* if this happens it's a bug */ - } - config_free_lines(c); - } -} - /** Print a usage message for tor. */ static void print_usage(void) @@ -3035,112 +1988,11 @@ is_local_addr(const tor_addr_t *addr) return 0; } -/** Release storage held by <b>options</b>. */ -static void -config_free(const config_format_t *fmt, void *options) -{ - int i; - - if (!options) - return; - - tor_assert(fmt); - - for (i=0; fmt->vars[i].name; ++i) - option_clear(fmt, options, &(fmt->vars[i])); - if (fmt->extra) { - config_line_t **linep = STRUCT_VAR_P(options, fmt->extra->var_offset); - config_free_lines(*linep); - *linep = NULL; - } - tor_free(options); -} - -/** Return true iff a and b contain identical keys and values in identical - * order. */ -static int -config_lines_eq(config_line_t *a, config_line_t *b) -{ - while (a && b) { - if (strcasecmp(a->key, b->key) || strcmp(a->value, b->value)) - return 0; - a = a->next; - b = b->next; - } - if (a || b) - return 0; - return 1; -} - -/** Return the number of lines in <b>a</b> whose key is <b>key</b>. */ -static int -config_count_key(const config_line_t *a, const char *key) -{ - int n = 0; - while (a) { - if (!strcasecmp(a->key, key)) { - ++n; - } - a = a->next; - } - return n; -} - -/** Return true iff the option <b>name</b> has the same value in <b>o1</b> - * and <b>o2</b>. Must not be called for LINELIST_S or OBSOLETE options. - */ -static int -option_is_same(const config_format_t *fmt, - const or_options_t *o1, const or_options_t *o2, - const char *name) -{ - config_line_t *c1, *c2; - int r = 1; - CHECK(fmt, o1); - CHECK(fmt, o2); - - c1 = get_assigned_option(fmt, o1, name, 0); - c2 = get_assigned_option(fmt, o2, name, 0); - r = config_lines_eq(c1, c2); - config_free_lines(c1); - config_free_lines(c2); - return r; -} - -/** Copy storage held by <b>old</b> into a new or_options_t and return it. */ -static or_options_t * -options_dup(const config_format_t *fmt, const or_options_t *old) -{ - or_options_t *newopts; - int i; - config_line_t *line; - - newopts = config_alloc(fmt); - for (i=0; fmt->vars[i].name; ++i) { - if (fmt->vars[i].type == CONFIG_TYPE_LINELIST_S) - continue; - if (fmt->vars[i].type == CONFIG_TYPE_OBSOLETE) - continue; - line = get_assigned_option(fmt, old, fmt->vars[i].name, 0); - if (line) { - char *msg = NULL; - if (config_assign(fmt, newopts, line, 0, 0, &msg) < 0) { - log_err(LD_BUG, "Config_get_assigned_option() generated " - "something we couldn't config_assign(): %s", msg); - tor_free(msg); - tor_assert(0); - } - } - config_free_lines(line); - } - return newopts; -} - /** Return a new empty or_options_t. Used for testing. */ or_options_t * options_new(void) { - return config_alloc(&options_format); + return config_new(&options_format); } /** Set <b>options</b> to hold reasonable defaults for most options. @@ -3151,94 +2003,6 @@ options_init(or_options_t *options) config_init(&options_format, options); } -/** Set all vars in the configuration object <b>options</b> to their default - * values. */ -static void -config_init(const config_format_t *fmt, void *options) -{ - int i; - const config_var_t *var; - CHECK(fmt, options); - - for (i=0; fmt->vars[i].name; ++i) { - var = &fmt->vars[i]; - if (!var->initvalue) - continue; /* defaults to NULL or 0 */ - option_reset(fmt, options, var, 1); - } -} - -/** Allocate and return a new string holding the written-out values of the vars - * in 'options'. If 'minimal', do not write out any default-valued vars. - * Else, if comment_defaults, write default values as comments. - */ -static char * -config_dump(const config_format_t *fmt, const void *default_options, - const void *options, int minimal, - int comment_defaults) -{ - smartlist_t *elements; - const or_options_t *defaults = default_options; - void *defaults_tmp = NULL; - config_line_t *line, *assigned; - char *result; - int i; - char *msg = NULL; - - if (defaults == NULL) { - defaults = defaults_tmp = config_alloc(fmt); - config_init(fmt, defaults_tmp); - } - - /* XXX use a 1 here so we don't add a new log line while dumping */ - if (default_options == NULL) { - if (fmt->validate_fn(NULL, defaults_tmp, 1, &msg) < 0) { - log_err(LD_BUG, "Failed to validate default config."); - tor_free(msg); - tor_assert(0); - } - } - - elements = smartlist_new(); - for (i=0; fmt->vars[i].name; ++i) { - int comment_option = 0; - if (fmt->vars[i].type == CONFIG_TYPE_OBSOLETE || - fmt->vars[i].type == CONFIG_TYPE_LINELIST_S) - continue; - /* Don't save 'hidden' control variables. */ - if (!strcmpstart(fmt->vars[i].name, "__")) - continue; - if (minimal && option_is_same(fmt, options, defaults, fmt->vars[i].name)) - continue; - else if (comment_defaults && - option_is_same(fmt, options, defaults, fmt->vars[i].name)) - comment_option = 1; - - line = assigned = get_assigned_option(fmt, options, fmt->vars[i].name, 1); - - for (; line; line = line->next) { - smartlist_add_asprintf(elements, "%s%s %s\n", - comment_option ? "# " : "", - line->key, line->value); - } - config_free_lines(assigned); - } - - if (fmt->extra) { - line = *(config_line_t**)STRUCT_VAR_P(options, fmt->extra->var_offset); - for (; line; line = line->next) { - smartlist_add_asprintf(elements, "%s %s\n", line->key, line->value); - } - } - - result = smartlist_join_strings(elements, "", 0, NULL); - SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); - smartlist_free(elements); - if (defaults_tmp) - config_free(fmt, defaults_tmp); - return result; -} - /** Return a string containing a possible configuration file that would give * the configuration in <b>options</b>. If <b>minimal</b> is true, do not * include options that are the same as Tor's defaults. @@ -4059,7 +2823,7 @@ options_validate(or_options_t *old_options, or_options_t *options, log_notice(LD_GENERAL, "Tor is not configured as a relay but you specified" " a ServerTransportPlugin line (%s). The ServerTransportPlugin " "line will be ignored.", - esc_for_log(options->ServerTransportPlugin->value)); + escaped(options->ServerTransportPlugin->value)); } if (options->ConstrainedSockets) { @@ -4749,7 +3513,7 @@ options_init_from_string(const char *cf_defaults, const char *cf, goto err; } if (i==0) - newdefaultoptions = options_dup(&options_format, newoptions); + newdefaultoptions = config_dup(&options_format, newoptions); } /* Go through command-line variables too */ @@ -4810,7 +3574,7 @@ options_init_from_string(const char *cf_defaults, const char *cf, goto err; } if (i==0) - newdefaultoptions = options_dup(&options_format, newoptions); + newdefaultoptions = config_dup(&options_format, newoptions); } /* Assign command-line variables a second time too */ retval = config_assign(&options_format, newoptions, @@ -5511,8 +4275,8 @@ parse_dir_server_line(const char *line, dirinfo_type_t required_type, fingerprint = smartlist_join_strings(items, "", 0, NULL); if (strlen(fingerprint) != HEX_DIGEST_LEN) { - log_warn(LD_CONFIG, "Key digest for DirServer is wrong length %d.", - (int)strlen(fingerprint)); + log_warn(LD_CONFIG, "Key digest '%s' for DirServer is wrong length %d.", + fingerprint, (int)strlen(fingerprint)); goto err; } if (!strcmp(fingerprint, "E623F7625FBE0C87820F11EC5F6D5377ED816294")) { @@ -6480,180 +5244,6 @@ options_save_current(void) return write_configuration_file(get_torrc_fname(0), get_options()); } -/** Mapping from a unit name to a multiplier for converting that unit into a - * base unit. Used by config_parse_unit. */ -struct unit_table_t { - const char *unit; /**< The name of the unit */ - uint64_t multiplier; /**< How many of the base unit appear in this unit */ -}; - -/** Table to map the names of memory units to the number of bytes they - * contain. */ -static struct unit_table_t memory_units[] = { - { "", 1 }, - { "b", 1<< 0 }, - { "byte", 1<< 0 }, - { "bytes", 1<< 0 }, - { "kb", 1<<10 }, - { "kbyte", 1<<10 }, - { "kbytes", 1<<10 }, - { "kilobyte", 1<<10 }, - { "kilobytes", 1<<10 }, - { "m", 1<<20 }, - { "mb", 1<<20 }, - { "mbyte", 1<<20 }, - { "mbytes", 1<<20 }, - { "megabyte", 1<<20 }, - { "megabytes", 1<<20 }, - { "gb", 1<<30 }, - { "gbyte", 1<<30 }, - { "gbytes", 1<<30 }, - { "gigabyte", 1<<30 }, - { "gigabytes", 1<<30 }, - { "tb", U64_LITERAL(1)<<40 }, - { "terabyte", U64_LITERAL(1)<<40 }, - { "terabytes", U64_LITERAL(1)<<40 }, - { NULL, 0 }, -}; - -/** Table to map the names of time units to the number of seconds they - * contain. */ -static struct unit_table_t time_units[] = { - { "", 1 }, - { "second", 1 }, - { "seconds", 1 }, - { "minute", 60 }, - { "minutes", 60 }, - { "hour", 60*60 }, - { "hours", 60*60 }, - { "day", 24*60*60 }, - { "days", 24*60*60 }, - { "week", 7*24*60*60 }, - { "weeks", 7*24*60*60 }, - { NULL, 0 }, -}; - -/** Table to map the names of time units to the number of milliseconds - * they contain. */ -static struct unit_table_t time_msec_units[] = { - { "", 1 }, - { "msec", 1 }, - { "millisecond", 1 }, - { "milliseconds", 1 }, - { "second", 1000 }, - { "seconds", 1000 }, - { "minute", 60*1000 }, - { "minutes", 60*1000 }, - { "hour", 60*60*1000 }, - { "hours", 60*60*1000 }, - { "day", 24*60*60*1000 }, - { "days", 24*60*60*1000 }, - { "week", 7*24*60*60*1000 }, - { "weeks", 7*24*60*60*1000 }, - { NULL, 0 }, -}; - -/** Parse a string <b>val</b> containing a number, zero or more - * spaces, and an optional unit string. If the unit appears in the - * table <b>u</b>, then multiply the number by the unit multiplier. - * On success, set *<b>ok</b> to 1 and return this product. - * Otherwise, set *<b>ok</b> to 0. - */ -static uint64_t -config_parse_units(const char *val, struct unit_table_t *u, int *ok) -{ - uint64_t v = 0; - double d = 0; - int use_float = 0; - char *cp; - - tor_assert(ok); - - v = tor_parse_uint64(val, 10, 0, UINT64_MAX, ok, &cp); - if (!*ok || (cp && *cp == '.')) { - d = tor_parse_double(val, 0, UINT64_MAX, ok, &cp); - if (!*ok) - goto done; - use_float = 1; - } - - if (!cp) { - *ok = 1; - v = use_float ? DBL_TO_U64(d) : v; - goto done; - } - - cp = (char*) eat_whitespace(cp); - - for ( ;u->unit;++u) { - if (!strcasecmp(u->unit, cp)) { - if (use_float) - v = u->multiplier * d; - else - v *= u->multiplier; - *ok = 1; - goto done; - } - } - log_warn(LD_CONFIG, "Unknown unit '%s'.", cp); - *ok = 0; - done: - - if (*ok) - return v; - else - return 0; -} - -/** Parse a string in the format "number unit", where unit is a unit of - * information (byte, KB, M, etc). On success, set *<b>ok</b> to true - * and return the number of bytes specified. Otherwise, set - * *<b>ok</b> to false and return 0. */ -static uint64_t -config_parse_memunit(const char *s, int *ok) -{ - uint64_t u = config_parse_units(s, memory_units, ok); - return u; -} - -/** Parse a string in the format "number unit", where unit is a unit of - * time in milliseconds. On success, set *<b>ok</b> to true and return - * the number of milliseconds in the provided interval. Otherwise, set - * *<b>ok</b> to 0 and return -1. */ -static int -config_parse_msec_interval(const char *s, int *ok) -{ - uint64_t r; - r = config_parse_units(s, time_msec_units, ok); - if (!ok) - return -1; - if (r > INT_MAX) { - log_warn(LD_CONFIG, "Msec interval '%s' is too long", s); - *ok = 0; - return -1; - } - return (int)r; -} - -/** Parse a string in the format "number unit", where unit is a unit of time. - * On success, set *<b>ok</b> to true and return the number of seconds in - * the provided interval. Otherwise, set *<b>ok</b> to 0 and return -1. - */ -static int -config_parse_interval(const char *s, int *ok) -{ - uint64_t r; - r = config_parse_units(s, time_units, ok); - if (!ok) - return -1; - if (r > INT_MAX) { - log_warn(LD_CONFIG, "Interval '%s' is too long", s); - *ok = 0; - return -1; - } - return (int)r; -} - /** Return the number of cpus configured in <b>options</b>. If we are * told to auto-detect the number of cpus, return the auto-detected number. */ int @@ -6707,14 +5297,6 @@ init_libevent(const or_options_t *options) } } -/** Return the persistent state struct for this Tor. */ -or_state_t * -get_or_state(void) -{ - tor_assert(global_state); - return global_state; -} - /** Return a newly allocated string holding a filename relative to the data * directory. If <b>sub1</b> is present, it is the first path component after * the data directory. If <b>sub2</b> is also present, it is the second path @@ -6765,474 +5347,6 @@ options_get_datadir_fname2_suffix(const or_options_t *options, return fname; } -/** Return true if <b>line</b> is a valid state TransportProxy line. - * Return false otherwise. */ -static int -state_transport_line_is_valid(const char *line) -{ - smartlist_t *items = NULL; - char *addrport=NULL; - tor_addr_t addr; - uint16_t port = 0; - int r; - - items = smartlist_new(); - smartlist_split_string(items, line, NULL, - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); - - if (smartlist_len(items) != 2) { - log_warn(LD_CONFIG, "state: Not enough arguments in TransportProxy line."); - goto err; - } - - addrport = smartlist_get(items, 1); - if (tor_addr_port_lookup(addrport, &addr, &port) < 0) { - log_warn(LD_CONFIG, "state: Could not parse addrport."); - goto err; - } - - if (!port) { - log_warn(LD_CONFIG, "state: Transport line did not contain port."); - goto err; - } - - r = 1; - goto done; - - err: - r = 0; - - done: - SMARTLIST_FOREACH(items, char*, s, tor_free(s)); - smartlist_free(items); - return r; -} - -/** Return 0 if all TransportProxy lines in <b>state</b> are well - * formed. Otherwise, return -1. */ -static int -validate_transports_in_state(or_state_t *state) -{ - int broken = 0; - config_line_t *line; - - for (line = state->TransportProxies ; line ; line = line->next) { - tor_assert(!strcmp(line->key, "TransportProxy")); - if (!state_transport_line_is_valid(line->value)) - broken = 1; - } - - if (broken) - log_warn(LD_CONFIG, "state: State file seems to be broken."); - - return 0; -} - -/** Return 0 if every setting in <b>state</b> is reasonable, and a - * permissible transition from <b>old_state</b>. Else warn and return -1. - * Should have no side effects, except for normalizing the contents of - * <b>state</b>. - */ -/* XXX from_setconf is here because of bug 238 */ -static int -or_state_validate(or_state_t *old_state, or_state_t *state, - int from_setconf, char **msg) -{ - /* We don't use these; only options do. Still, we need to match that - * signature. */ - (void) from_setconf; - (void) old_state; - - if (entry_guards_parse_state(state, 0, msg)<0) - return -1; - - if (validate_transports_in_state(state)<0) - return -1; - - return 0; -} - -/** Replace the current persistent state with <b>new_state</b> */ -static int -or_state_set(or_state_t *new_state) -{ - char *err = NULL; - int ret = 0; - tor_assert(new_state); - config_free(&state_format, global_state); - global_state = new_state; - if (entry_guards_parse_state(global_state, 1, &err)<0) { - log_warn(LD_GENERAL,"%s",err); - tor_free(err); - ret = -1; - } - if (rep_hist_load_state(global_state, &err)<0) { - log_warn(LD_GENERAL,"Unparseable bandwidth history state: %s",err); - tor_free(err); - ret = -1; - } - if (circuit_build_times_parse_state(&circ_times, global_state) < 0) { - ret = -1; - } - return ret; -} - -/** - * Save a broken state file to a backup location. - */ -static void -or_state_save_broken(char *fname) -{ - int i; - file_status_t status; - char *fname2 = NULL; - for (i = 0; i < 100; ++i) { - tor_asprintf(&fname2, "%s.%d", fname, i); - status = file_status(fname2); - if (status == FN_NOENT) - break; - tor_free(fname2); - } - if (i == 100) { - log_warn(LD_BUG, "Unable to parse state in \"%s\"; too many saved bad " - "state files to move aside. Discarding the old state file.", - fname); - unlink(fname); - } else { - log_warn(LD_BUG, "Unable to parse state in \"%s\". Moving it aside " - "to \"%s\". This could be a bug in Tor; please tell " - "the developers.", fname, fname2); - if (rename(fname, fname2) < 0) { - log_warn(LD_BUG, "Weirdly, I couldn't even move the state aside. The " - "OS gave an error of %s", strerror(errno)); - } - } - tor_free(fname2); -} - -/** Reload the persistent state from disk, generating a new state as needed. - * Return 0 on success, less than 0 on failure. - */ -static int -or_state_load(void) -{ - or_state_t *new_state = NULL; - char *contents = NULL, *fname; - char *errmsg = NULL; - int r = -1, badstate = 0; - - fname = get_datadir_fname("state"); - switch (file_status(fname)) { - case FN_FILE: - if (!(contents = read_file_to_str(fname, 0, NULL))) { - log_warn(LD_FS, "Unable to read state file \"%s\"", fname); - goto done; - } - break; - case FN_NOENT: - break; - case FN_ERROR: - case FN_DIR: - default: - log_warn(LD_GENERAL,"State file \"%s\" is not a file? Failing.", fname); - goto done; - } - new_state = tor_malloc_zero(sizeof(or_state_t)); - new_state->_magic = OR_STATE_MAGIC; - config_init(&state_format, new_state); - if (contents) { - config_line_t *lines=NULL; - int assign_retval; - if (config_get_lines(contents, &lines, 0)<0) - goto done; - assign_retval = config_assign(&state_format, new_state, - lines, 0, 0, &errmsg); - config_free_lines(lines); - if (assign_retval<0) - badstate = 1; - if (errmsg) { - log_warn(LD_GENERAL, "%s", errmsg); - tor_free(errmsg); - } - } - - if (!badstate && or_state_validate(NULL, new_state, 1, &errmsg) < 0) - badstate = 1; - - if (errmsg) { - log_warn(LD_GENERAL, "%s", errmsg); - tor_free(errmsg); - } - - if (badstate && !contents) { - log_warn(LD_BUG, "Uh oh. We couldn't even validate our own default state." - " This is a bug in Tor."); - goto done; - } else if (badstate && contents) { - or_state_save_broken(fname); - - tor_free(contents); - config_free(&state_format, new_state); - - new_state = tor_malloc_zero(sizeof(or_state_t)); - new_state->_magic = OR_STATE_MAGIC; - config_init(&state_format, new_state); - } else if (contents) { - log_info(LD_GENERAL, "Loaded state from \"%s\"", fname); - } else { - log_info(LD_GENERAL, "Initialized state"); - } - if (or_state_set(new_state) == -1) { - or_state_save_broken(fname); - } - new_state = NULL; - if (!contents) { - global_state->next_write = 0; - or_state_save(time(NULL)); - } - r = 0; - - done: - tor_free(fname); - tor_free(contents); - if (new_state) - config_free(&state_format, new_state); - - return r; -} - -/** Did the last time we tried to write the state file fail? If so, we - * should consider disabling such features as preemptive circuit generation - * to compute circuit-build-time. */ -static int last_state_file_write_failed = 0; - -/** Return whether the state file failed to write last time we tried. */ -int -did_last_state_file_write_fail(void) -{ - return last_state_file_write_failed; -} - -/** If writing the state to disk fails, try again after this many seconds. */ -#define STATE_WRITE_RETRY_INTERVAL 3600 - -/** If we're a relay, how often should we checkpoint our state file even - * if nothing else dirties it? This will checkpoint ongoing stats like - * bandwidth used, per-country user stats, etc. */ -#define STATE_RELAY_CHECKPOINT_INTERVAL (12*60*60) - -/** Write the persistent state to disk. Return 0 for success, <0 on failure. */ -int -or_state_save(time_t now) -{ - char *state, *contents; - char tbuf[ISO_TIME_LEN+1]; - char *fname; - - tor_assert(global_state); - - if (global_state->next_write > now) - return 0; - - /* Call everything else that might dirty the state even more, in order - * to avoid redundant writes. */ - entry_guards_update_state(global_state); - rep_hist_update_state(global_state); - circuit_build_times_update_state(&circ_times, global_state); - if (accounting_is_enabled(get_options())) - accounting_run_housekeeping(now); - - global_state->LastWritten = now; - - tor_free(global_state->TorVersion); - tor_asprintf(&global_state->TorVersion, "Tor %s", get_version()); - - state = config_dump(&state_format, NULL, global_state, 1, 0); - format_local_iso_time(tbuf, now); - tor_asprintf(&contents, - "# Tor state file last generated on %s local time\n" - "# Other times below are in GMT\n" - "# You *do not* need to edit this file.\n\n%s", - tbuf, state); - tor_free(state); - fname = get_datadir_fname("state"); - if (write_str_to_file(fname, contents, 0)<0) { - log_warn(LD_FS, "Unable to write state to file \"%s\"; " - "will try again later", fname); - last_state_file_write_failed = 1; - tor_free(fname); - tor_free(contents); - /* Try again after STATE_WRITE_RETRY_INTERVAL (or sooner, if the state - * changes sooner). */ - global_state->next_write = now + STATE_WRITE_RETRY_INTERVAL; - return -1; - } - - last_state_file_write_failed = 0; - log_info(LD_GENERAL, "Saved state to \"%s\"", fname); - tor_free(fname); - tor_free(contents); - - if (server_mode(get_options())) - global_state->next_write = now + STATE_RELAY_CHECKPOINT_INTERVAL; - else - global_state->next_write = TIME_MAX; - - return 0; -} - -/** Return the config line for transport <b>transport</b> in the current state. - * Return NULL if there is no config line for <b>transport</b>. */ -static config_line_t * -get_transport_in_state_by_name(const char *transport) -{ - or_state_t *or_state = get_or_state(); - config_line_t *line; - config_line_t *ret = NULL; - smartlist_t *items = NULL; - - for (line = or_state->TransportProxies ; line ; line = line->next) { - tor_assert(!strcmp(line->key, "TransportProxy")); - - items = smartlist_new(); - smartlist_split_string(items, line->value, NULL, - SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); - if (smartlist_len(items) != 2) /* broken state */ - goto done; - - if (!strcmp(smartlist_get(items, 0), transport)) { - ret = line; - goto done; - } - - SMARTLIST_FOREACH(items, char*, s, tor_free(s)); - smartlist_free(items); - items = NULL; - } - - done: - if (items) { - SMARTLIST_FOREACH(items, char*, s, tor_free(s)); - smartlist_free(items); - } - return ret; -} - -/** Return string containing the address:port part of the - * TransportProxy <b>line</b> for transport <b>transport</b>. - * If the line is corrupted, return NULL. */ -static const char * -get_transport_bindaddr(const char *line, const char *transport) -{ - char *line_tmp = NULL; - - if (strlen(line) < strlen(transport) + 2) { - goto broken_state; - } else { - /* line should start with the name of the transport and a space. - (for example, "obfs2 127.0.0.1:47245") */ - tor_asprintf(&line_tmp, "%s ", transport); - if (strcmpstart(line, line_tmp)) - goto broken_state; - - tor_free(line_tmp); - return (line+strlen(transport)+1); - } - - broken_state: - tor_free(line_tmp); - return NULL; -} - -/** Return a string containing the address:port that a proxy transport - * should bind on. The string is stored on the heap and must be freed - * by the caller of this function. */ -char * -get_stored_bindaddr_for_server_transport(const char *transport) -{ - char *default_addrport = NULL; - const char *stored_bindaddr = NULL; - - config_line_t *line = get_transport_in_state_by_name(transport); - if (!line) /* Found no references in state for this transport. */ - goto no_bindaddr_found; - - stored_bindaddr = get_transport_bindaddr(line->value, transport); - if (stored_bindaddr) /* found stored bindaddr in state file. */ - return tor_strdup(stored_bindaddr); - - no_bindaddr_found: - /** If we didn't find references for this pluggable transport in the - state file, we should instruct the pluggable transport proxy to - listen on INADDR_ANY on a random ephemeral port. */ - tor_asprintf(&default_addrport, "%s:%s", fmt_addr32(INADDR_ANY), "0"); - return default_addrport; -} - -/** Save <b>transport</b> listening on <b>addr</b>:<b>port</b> to - state */ -void -save_transport_to_state(const char *transport, - const tor_addr_t *addr, uint16_t port) -{ - or_state_t *state = get_or_state(); - - char *transport_addrport=NULL; - - /** find where to write on the state */ - config_line_t **next, *line; - - /* see if this transport is already stored in state */ - config_line_t *transport_line = - get_transport_in_state_by_name(transport); - - if (transport_line) { /* if transport already exists in state... */ - const char *prev_bindaddr = /* get its addrport... */ - get_transport_bindaddr(transport_line->value, transport); - tor_asprintf(&transport_addrport, "%s:%d", fmt_addr(addr), (int)port); - - /* if transport in state has the same address as this one, life is good */ - if (!strcmp(prev_bindaddr, transport_addrport)) { - log_info(LD_CONFIG, "Transport seems to have spawned on its usual " - "address:port."); - goto done; - } else { /* if addrport in state is different than the one we got */ - log_info(LD_CONFIG, "Transport seems to have spawned on different " - "address:port. Let's update the state file with the new " - "address:port"); - tor_free(transport_line->value); /* free the old line */ - tor_asprintf(&transport_line->value, "%s %s:%d", transport, - fmt_addr(addr), - (int) port); /* replace old addrport line with new line */ - } - } else { /* never seen this one before; save it in state for next time */ - log_info(LD_CONFIG, "It's the first time we see this transport. " - "Let's save its address:port"); - next = &state->TransportProxies; - /* find the last TransportProxy line in the state and point 'next' - right after it */ - line = state->TransportProxies; - while (line) { - next = &(line->next); - line = line->next; - } - - /* allocate space for the new line and fill it in */ - *next = line = tor_malloc_zero(sizeof(config_line_t)); - line->key = tor_strdup("TransportProxy"); - tor_asprintf(&line->value, "%s %s:%d", transport, - fmt_addr(addr), (int) port); - - next = &(line->next); - } - - if (!get_options()->AvoidDiskWrites) - or_state_mark_dirty(state, 0); - - done: - tor_free(transport_addrport); -} - /** Given a file name check to see whether the file exists but has not been * modified for a very long time. If so, remove it. */ void @@ -7250,6 +5364,43 @@ remove_file_if_very_old(const char *fname, time_t now) } } +/** Return a smartlist of ports that must be forwarded by + * tor-fw-helper. The smartlist contains the ports in a string format + * that is understandable by tor-fw-helper. */ +smartlist_t * +get_list_of_ports_to_forward(void) +{ + smartlist_t *ports_to_forward = smartlist_new(); + int port = 0; + + /** XXX TODO tor-fw-helper does not support forwarding ports to + other hosts than the local one. If the user is binding to a + different IP address, tor-fw-helper won't work. */ + port = router_get_advertised_or_port(get_options()); /* Get ORPort */ + if (port) + smartlist_add_asprintf(ports_to_forward, "%d:%d", port, port); + + port = router_get_advertised_dir_port(get_options(), 0); /* Get DirPort */ + if (port) + smartlist_add_asprintf(ports_to_forward, "%d:%d", port, port); + + /* Get ports of transport proxies */ + { + smartlist_t *transport_ports = get_transport_proxy_ports(); + if (transport_ports) { + smartlist_add_all(ports_to_forward, transport_ports); + smartlist_free(transport_ports); + } + } + + if (!smartlist_len(ports_to_forward)) { + smartlist_free(ports_to_forward); + ports_to_forward = NULL; + } + + return ports_to_forward; +} + /** Helper to implement GETINFO functions about configuration variables (not * their values). Given a "config/names" question, set *<b>answer</b> to a * new string describing the supported configuration variables and their @@ -7267,6 +5418,9 @@ getinfo_helper_config(control_connection_t *conn, for (i = 0; _option_vars[i].name; ++i) { const config_var_t *var = &_option_vars[i]; const char *type; + /* don't tell controller about triple-underscore options */ + if (!strncmp(_option_vars[i].name, "___", 3)) + continue; switch (var->type) { case CONFIG_TYPE_STRING: type = "String"; break; case CONFIG_TYPE_FILENAME: type = "Filename"; break; @@ -7296,6 +5450,20 @@ getinfo_helper_config(control_connection_t *conn, *answer = smartlist_join_strings(sl, "", 0, NULL); SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); smartlist_free(sl); + } else if (!strcmp(question, "config/defaults")) { + smartlist_t *sl = smartlist_new(); + int i; + for (i = 0; _option_vars[i].name; ++i) { + const config_var_t *var = &_option_vars[i]; + if (var->initvalue != NULL) { + char *val = esc_for_log(var->initvalue); + smartlist_add_asprintf(sl, "%s %s\n",var->name,val); + tor_free(val); + } + } + *answer = smartlist_join_strings(sl, "", 0, NULL); + SMARTLIST_FOREACH(sl, char *, c, tor_free(c)); + smartlist_free(sl); } return 0; } diff --git a/src/or/config.h b/src/or/config.h index dd76edcf1..9d170b8af 100644 --- a/src/or/config.h +++ b/src/or/config.h @@ -23,11 +23,9 @@ const char *escaped_safe_str_client(const char *address); const char *escaped_safe_str(const char *address); const char *get_version(void); const char *get_short_version(void); - -int config_get_lines(const char *string, config_line_t **result, int extended); -void config_free_lines(config_line_t *front); setopt_err_t options_trial_assign(config_line_t *list, int use_defaults, int clear_first, char **msg); + int resolve_my_address(int warn_severity, const or_options_t *options, uint32_t *addr, char **hostname_out); int is_local_addr(const tor_addr_t *addr); @@ -61,10 +59,6 @@ char *options_get_datadir_fname2_suffix(const or_options_t *options, int get_num_cpus(const or_options_t *options); -or_state_t *get_or_state(void); -int did_last_state_file_write_fail(void); -int or_state_save(time_t now); - const smartlist_t *get_configured_ports(void); int get_first_advertised_port_by_type_af(int listener_type, int address_family); @@ -78,9 +72,7 @@ char *get_first_listener_addrport_string(int listener_type); int options_need_geoip_info(const or_options_t *options, const char **reason_out); -void save_transport_to_state(const char *transport_name, - const tor_addr_t *addr, uint16_t port); -char *get_stored_bindaddr_for_server_transport(const char *transport); +smartlist_t *get_list_of_ports_to_forward(void); int getinfo_helper_config(control_connection_t *conn, const char *question, char **answer, diff --git a/src/or/confparse.c b/src/or/confparse.c new file mode 100644 index 000000000..67cf43fe8 --- /dev/null +++ b/src/or/confparse.c @@ -0,0 +1,1226 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "confparse.h" +#include "routerset.h" + +static uint64_t config_parse_memunit(const char *s, int *ok); +static int config_parse_msec_interval(const char *s, int *ok); +static int config_parse_interval(const char *s, int *ok); +static void config_reset(const config_format_t *fmt, void *options, + const config_var_t *var, int use_defaults); + +/** Allocate an empty configuration object of a given format type. */ +void * +config_new(const config_format_t *fmt) +{ + void *opts = tor_malloc_zero(fmt->size); + *(uint32_t*)STRUCT_VAR_P(opts, fmt->magic_offset) = fmt->magic; + CONFIG_CHECK(fmt, opts); + return opts; +} + +/* + * Functions to parse config options + */ + +/** If <b>option</b> is an official abbreviation for a longer option, + * return the longer option. Otherwise return <b>option</b>. + * If <b>command_line</b> is set, apply all abbreviations. Otherwise, only + * apply abbreviations that work for the config file and the command line. + * If <b>warn_obsolete</b> is set, warn about deprecated names. */ +const char * +config_expand_abbrev(const config_format_t *fmt, const char *option, + int command_line, int warn_obsolete) +{ + int i; + if (! fmt->abbrevs) + return option; + for (i=0; fmt->abbrevs[i].abbreviated; ++i) { + /* Abbreviations are case insensitive. */ + if (!strcasecmp(option,fmt->abbrevs[i].abbreviated) && + (command_line || !fmt->abbrevs[i].commandline_only)) { + if (warn_obsolete && fmt->abbrevs[i].warn) { + log_warn(LD_CONFIG, + "The configuration option '%s' is deprecated; " + "use '%s' instead.", + fmt->abbrevs[i].abbreviated, + fmt->abbrevs[i].full); + } + /* Keep going through the list in case we want to rewrite it more. + * (We could imagine recursing here, but I don't want to get the + * user into an infinite loop if we craft our list wrong.) */ + option = fmt->abbrevs[i].full; + } + } + return option; +} + +/** Helper: allocate a new configuration option mapping 'key' to 'val', + * append it to *<b>lst</b>. */ +void +config_line_append(config_line_t **lst, + const char *key, + const char *val) +{ + config_line_t *newline; + + newline = tor_malloc_zero(sizeof(config_line_t)); + newline->key = tor_strdup(key); + newline->value = tor_strdup(val); + newline->next = NULL; + while (*lst) + lst = &((*lst)->next); + + (*lst) = newline; +} + +/** Helper: parse the config string and strdup into key/value + * strings. Set *result to the list, or NULL if parsing the string + * failed. Return 0 on success, -1 on failure. Warn and ignore any + * misformatted lines. + * + * If <b>extended</b> is set, then treat keys beginning with / and with + as + * indicating "clear" and "append" respectively. */ +int +config_get_lines(const char *string, config_line_t **result, int extended) +{ + config_line_t *list = NULL, **next; + char *k, *v; + + next = &list; + do { + k = v = NULL; + string = parse_config_line_from_str(string, &k, &v); + if (!string) { + config_free_lines(list); + tor_free(k); + tor_free(v); + return -1; + } + if (k && v) { + unsigned command = CONFIG_LINE_NORMAL; + if (extended) { + if (k[0] == '+') { + char *k_new = tor_strdup(k+1); + tor_free(k); + k = k_new; + command = CONFIG_LINE_APPEND; + } else if (k[0] == '/') { + char *k_new = tor_strdup(k+1); + tor_free(k); + k = k_new; + tor_free(v); + v = tor_strdup(""); + command = CONFIG_LINE_CLEAR; + } + } + /* This list can get long, so we keep a pointer to the end of it + * rather than using config_line_append over and over and getting + * n^2 performance. */ + *next = tor_malloc_zero(sizeof(config_line_t)); + (*next)->key = k; + (*next)->value = v; + (*next)->next = NULL; + (*next)->command = command; + next = &((*next)->next); + } else { + tor_free(k); + tor_free(v); + } + } while (*string); + + *result = list; + return 0; +} + +/** + * Free all the configuration lines on the linked list <b>front</b>. + */ +void +config_free_lines(config_line_t *front) +{ + config_line_t *tmp; + + while (front) { + tmp = front; + front = tmp->next; + + tor_free(tmp->key); + tor_free(tmp->value); + tor_free(tmp); + } +} + +/** As config_find_option, but return a non-const pointer. */ +config_var_t * +config_find_option_mutable(config_format_t *fmt, const char *key) +{ + int i; + size_t keylen = strlen(key); + if (!keylen) + return NULL; /* if they say "--" on the command line, it's not an option */ + /* First, check for an exact (case-insensitive) match */ + for (i=0; fmt->vars[i].name; ++i) { + if (!strcasecmp(key, fmt->vars[i].name)) { + return &fmt->vars[i]; + } + } + /* If none, check for an abbreviated match */ + for (i=0; fmt->vars[i].name; ++i) { + if (!strncasecmp(key, fmt->vars[i].name, keylen)) { + log_warn(LD_CONFIG, "The abbreviation '%s' is deprecated. " + "Please use '%s' instead", + key, fmt->vars[i].name); + return &fmt->vars[i]; + } + } + /* Okay, unrecognized option */ + return NULL; +} + +/** If <b>key</b> is a configuration option, return the corresponding const + * config_var_t. Otherwise, if <b>key</b> is a non-standard abbreviation, + * warn, and return the corresponding const config_var_t. Otherwise return + * NULL. + */ +const config_var_t * +config_find_option(const config_format_t *fmt, const char *key) +{ + return config_find_option_mutable((config_format_t*)fmt, key); +} + +/** Return the number of option entries in <b>fmt</b>. */ +static int +config_count_options(const config_format_t *fmt) +{ + int i; + for (i=0; fmt->vars[i].name; ++i) + ; + return i; +} + +/* + * Functions to assign config options. + */ + +/** <b>c</b>-\>key is known to be a real key. Update <b>options</b> + * with <b>c</b>-\>value and return 0, or return -1 if bad value. + * + * Called from config_assign_line() and option_reset(). + */ +static int +config_assign_value(const config_format_t *fmt, void *options, + config_line_t *c, char **msg) +{ + int i, ok; + const config_var_t *var; + void *lvalue; + + CONFIG_CHECK(fmt, options); + + var = config_find_option(fmt, c->key); + tor_assert(var); + + lvalue = STRUCT_VAR_P(options, var->var_offset); + + switch (var->type) { + + case CONFIG_TYPE_PORT: + if (!strcasecmp(c->value, "auto")) { + *(int *)lvalue = CFG_AUTO_PORT; + break; + } + /* fall through */ + case CONFIG_TYPE_INT: + case CONFIG_TYPE_UINT: + i = (int)tor_parse_long(c->value, 10, + var->type==CONFIG_TYPE_INT ? INT_MIN : 0, + var->type==CONFIG_TYPE_PORT ? 65535 : INT_MAX, + &ok, NULL); + if (!ok) { + tor_asprintf(msg, + "Int keyword '%s %s' is malformed or out of bounds.", + c->key, c->value); + return -1; + } + *(int *)lvalue = i; + break; + + case CONFIG_TYPE_INTERVAL: { + i = config_parse_interval(c->value, &ok); + if (!ok) { + tor_asprintf(msg, + "Interval '%s %s' is malformed or out of bounds.", + c->key, c->value); + return -1; + } + *(int *)lvalue = i; + break; + } + + case CONFIG_TYPE_MSEC_INTERVAL: { + i = config_parse_msec_interval(c->value, &ok); + if (!ok) { + tor_asprintf(msg, + "Msec interval '%s %s' is malformed or out of bounds.", + c->key, c->value); + return -1; + } + *(int *)lvalue = i; + break; + } + + case CONFIG_TYPE_MEMUNIT: { + uint64_t u64 = config_parse_memunit(c->value, &ok); + if (!ok) { + tor_asprintf(msg, + "Value '%s %s' is malformed or out of bounds.", + c->key, c->value); + return -1; + } + *(uint64_t *)lvalue = u64; + break; + } + + case CONFIG_TYPE_BOOL: + i = (int)tor_parse_long(c->value, 10, 0, 1, &ok, NULL); + if (!ok) { + tor_asprintf(msg, + "Boolean '%s %s' expects 0 or 1.", + c->key, c->value); + return -1; + } + *(int *)lvalue = i; + break; + + case CONFIG_TYPE_AUTOBOOL: + if (!strcmp(c->value, "auto")) + *(int *)lvalue = -1; + else if (!strcmp(c->value, "0")) + *(int *)lvalue = 0; + else if (!strcmp(c->value, "1")) + *(int *)lvalue = 1; + else { + tor_asprintf(msg, "Boolean '%s %s' expects 0, 1, or 'auto'.", + c->key, c->value); + return -1; + } + break; + + case CONFIG_TYPE_STRING: + case CONFIG_TYPE_FILENAME: + tor_free(*(char **)lvalue); + *(char **)lvalue = tor_strdup(c->value); + break; + + case CONFIG_TYPE_DOUBLE: + *(double *)lvalue = atof(c->value); + break; + + case CONFIG_TYPE_ISOTIME: + if (parse_iso_time(c->value, (time_t *)lvalue)) { + tor_asprintf(msg, + "Invalid time '%s' for keyword '%s'", c->value, c->key); + return -1; + } + break; + + case CONFIG_TYPE_ROUTERSET: + if (*(routerset_t**)lvalue) { + routerset_free(*(routerset_t**)lvalue); + } + *(routerset_t**)lvalue = routerset_new(); + if (routerset_parse(*(routerset_t**)lvalue, c->value, c->key)<0) { + tor_asprintf(msg, "Invalid exit list '%s' for option '%s'", + c->value, c->key); + return -1; + } + break; + + case CONFIG_TYPE_CSV: + if (*(smartlist_t**)lvalue) { + SMARTLIST_FOREACH(*(smartlist_t**)lvalue, char *, cp, tor_free(cp)); + smartlist_clear(*(smartlist_t**)lvalue); + } else { + *(smartlist_t**)lvalue = smartlist_new(); + } + + smartlist_split_string(*(smartlist_t**)lvalue, c->value, ",", + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0); + break; + + case CONFIG_TYPE_LINELIST: + case CONFIG_TYPE_LINELIST_S: + { + config_line_t *lastval = *(config_line_t**)lvalue; + if (lastval && lastval->fragile) { + if (c->command != CONFIG_LINE_APPEND) { + config_free_lines(lastval); + *(config_line_t**)lvalue = NULL; + } else { + lastval->fragile = 0; + } + } + + config_line_append((config_line_t**)lvalue, c->key, c->value); + } + break; + case CONFIG_TYPE_OBSOLETE: + log_warn(LD_CONFIG, "Skipping obsolete configuration option '%s'", c->key); + break; + case CONFIG_TYPE_LINELIST_V: + tor_asprintf(msg, + "You may not provide a value for virtual option '%s'", c->key); + return -1; + default: + tor_assert(0); + break; + } + return 0; +} + +/** Mark every linelist in <b>options</b> "fragile", so that fresh assignments + * to it will replace old ones. */ +static void +config_mark_lists_fragile(const config_format_t *fmt, void *options) +{ + int i; + tor_assert(fmt); + tor_assert(options); + + for (i = 0; fmt->vars[i].name; ++i) { + const config_var_t *var = &fmt->vars[i]; + config_line_t *list; + if (var->type != CONFIG_TYPE_LINELIST && + var->type != CONFIG_TYPE_LINELIST_V) + continue; + + list = *(config_line_t **)STRUCT_VAR_P(options, var->var_offset); + if (list) + list->fragile = 1; + } +} + +/** If <b>c</b> is a syntactically valid configuration line, update + * <b>options</b> with its value and return 0. Otherwise return -1 for bad + * key, -2 for bad value. + * + * If <b>clear_first</b> is set, clear the value first. Then if + * <b>use_defaults</b> is set, set the value to the default. + * + * Called from config_assign(). + */ +static int +config_assign_line(const config_format_t *fmt, void *options, + config_line_t *c, int use_defaults, + int clear_first, bitarray_t *options_seen, char **msg) +{ + const config_var_t *var; + + CONFIG_CHECK(fmt, options); + + var = config_find_option(fmt, c->key); + if (!var) { + if (fmt->extra) { + void *lvalue = STRUCT_VAR_P(options, fmt->extra->var_offset); + log_info(LD_CONFIG, + "Found unrecognized option '%s'; saving it.", c->key); + config_line_append((config_line_t**)lvalue, c->key, c->value); + return 0; + } else { + tor_asprintf(msg, + "Unknown option '%s'. Failing.", c->key); + return -1; + } + } + + /* Put keyword into canonical case. */ + if (strcmp(var->name, c->key)) { + tor_free(c->key); + c->key = tor_strdup(var->name); + } + + if (!strlen(c->value)) { + /* reset or clear it, then return */ + if (!clear_first) { + if ((var->type == CONFIG_TYPE_LINELIST || + var->type == CONFIG_TYPE_LINELIST_S) && + c->command != CONFIG_LINE_CLEAR) { + /* We got an empty linelist from the torrc or command line. + As a special case, call this an error. Warn and ignore. */ + log_warn(LD_CONFIG, + "Linelist option '%s' has no value. Skipping.", c->key); + } else { /* not already cleared */ + config_reset(fmt, options, var, use_defaults); + } + } + return 0; + } else if (c->command == CONFIG_LINE_CLEAR && !clear_first) { + config_reset(fmt, options, var, use_defaults); + } + + if (options_seen && (var->type != CONFIG_TYPE_LINELIST && + var->type != CONFIG_TYPE_LINELIST_S)) { + /* We're tracking which options we've seen, and this option is not + * supposed to occur more than once. */ + int var_index = (int)(var - fmt->vars); + if (bitarray_is_set(options_seen, var_index)) { + log_warn(LD_CONFIG, "Option '%s' used more than once; all but the last " + "value will be ignored.", var->name); + } + bitarray_set(options_seen, var_index); + } + + if (config_assign_value(fmt, options, c, msg) < 0) + return -2; + return 0; +} + +/** Restore the option named <b>key</b> in options to its default value. + * Called from config_assign(). */ +static void +config_reset_line(const config_format_t *fmt, void *options, + const char *key, int use_defaults) +{ + const config_var_t *var; + + CONFIG_CHECK(fmt, options); + + var = config_find_option(fmt, key); + if (!var) + return; /* give error on next pass. */ + + config_reset(fmt, options, var, use_defaults); +} + +/** Return true iff value needs to be quoted and escaped to be used in + * a configuration file. */ +static int +config_value_needs_escape(const char *value) +{ + if (*value == '\"') + return 1; + while (*value) { + switch (*value) + { + case '\r': + case '\n': + case '#': + /* Note: quotes and backspaces need special handling when we are using + * quotes, not otherwise, so they don't trigger escaping on their + * own. */ + return 1; + default: + if (!TOR_ISPRINT(*value)) + return 1; + } + ++value; + } + return 0; +} + +/** Return a newly allocated deep copy of the lines in <b>inp</b>. */ +config_line_t * +config_lines_dup(const config_line_t *inp) +{ + config_line_t *result = NULL; + config_line_t **next_out = &result; + while (inp) { + *next_out = tor_malloc_zero(sizeof(config_line_t)); + (*next_out)->key = tor_strdup(inp->key); + (*next_out)->value = tor_strdup(inp->value); + inp = inp->next; + next_out = &((*next_out)->next); + } + (*next_out) = NULL; + return result; +} + +/** Return newly allocated line or lines corresponding to <b>key</b> in the + * configuration <b>options</b>. If <b>escape_val</b> is true and a + * value needs to be quoted before it's put in a config file, quote and + * escape that value. Return NULL if no such key exists. */ +config_line_t * +config_get_assigned_option(const config_format_t *fmt, const void *options, + const char *key, int escape_val) +{ + const config_var_t *var; + const void *value; + config_line_t *result; + tor_assert(options && key); + + CONFIG_CHECK(fmt, options); + + var = config_find_option(fmt, key); + if (!var) { + log_warn(LD_CONFIG, "Unknown option '%s'. Failing.", key); + return NULL; + } + value = STRUCT_VAR_P(options, var->var_offset); + + result = tor_malloc_zero(sizeof(config_line_t)); + result->key = tor_strdup(var->name); + switch (var->type) + { + case CONFIG_TYPE_STRING: + case CONFIG_TYPE_FILENAME: + if (*(char**)value) { + result->value = tor_strdup(*(char**)value); + } else { + tor_free(result->key); + tor_free(result); + return NULL; + } + break; + case CONFIG_TYPE_ISOTIME: + if (*(time_t*)value) { + result->value = tor_malloc(ISO_TIME_LEN+1); + format_iso_time(result->value, *(time_t*)value); + } else { + tor_free(result->key); + tor_free(result); + } + escape_val = 0; /* Can't need escape. */ + break; + case CONFIG_TYPE_PORT: + if (*(int*)value == CFG_AUTO_PORT) { + result->value = tor_strdup("auto"); + escape_val = 0; + break; + } + /* fall through */ + case CONFIG_TYPE_INTERVAL: + case CONFIG_TYPE_MSEC_INTERVAL: + case CONFIG_TYPE_UINT: + case CONFIG_TYPE_INT: + /* This means every or_options_t uint or bool element + * needs to be an int. Not, say, a uint16_t or char. */ + tor_asprintf(&result->value, "%d", *(int*)value); + escape_val = 0; /* Can't need escape. */ + break; + case CONFIG_TYPE_MEMUNIT: + tor_asprintf(&result->value, U64_FORMAT, + U64_PRINTF_ARG(*(uint64_t*)value)); + escape_val = 0; /* Can't need escape. */ + break; + case CONFIG_TYPE_DOUBLE: + tor_asprintf(&result->value, "%f", *(double*)value); + escape_val = 0; /* Can't need escape. */ + break; + + case CONFIG_TYPE_AUTOBOOL: + if (*(int*)value == -1) { + result->value = tor_strdup("auto"); + escape_val = 0; + break; + } + /* fall through */ + case CONFIG_TYPE_BOOL: + result->value = tor_strdup(*(int*)value ? "1" : "0"); + escape_val = 0; /* Can't need escape. */ + break; + case CONFIG_TYPE_ROUTERSET: + result->value = routerset_to_string(*(routerset_t**)value); + break; + case CONFIG_TYPE_CSV: + if (*(smartlist_t**)value) + result->value = + smartlist_join_strings(*(smartlist_t**)value, ",", 0, NULL); + else + result->value = tor_strdup(""); + break; + case CONFIG_TYPE_OBSOLETE: + log_fn(LOG_INFO, LD_CONFIG, + "You asked me for the value of an obsolete config option '%s'.", + key); + tor_free(result->key); + tor_free(result); + return NULL; + case CONFIG_TYPE_LINELIST_S: + log_warn(LD_CONFIG, + "Can't return context-sensitive '%s' on its own", key); + tor_free(result->key); + tor_free(result); + return NULL; + case CONFIG_TYPE_LINELIST: + case CONFIG_TYPE_LINELIST_V: + tor_free(result->key); + tor_free(result); + result = config_lines_dup(*(const config_line_t**)value); + break; + default: + tor_free(result->key); + tor_free(result); + log_warn(LD_BUG,"Unknown type %d for known key '%s'", + var->type, key); + return NULL; + } + + if (escape_val) { + config_line_t *line; + for (line = result; line; line = line->next) { + if (line->value && config_value_needs_escape(line->value)) { + char *newval = esc_for_log(line->value); + tor_free(line->value); + line->value = newval; + } + } + } + + return result; +} +/** Iterate through the linked list of requested options <b>list</b>. + * For each item, convert as appropriate and assign to <b>options</b>. + * If an item is unrecognized, set *msg and return -1 immediately, + * else return 0 for success. + * + * If <b>clear_first</b>, interpret config options as replacing (not + * extending) their previous values. If <b>clear_first</b> is set, + * then <b>use_defaults</b> to decide if you set to defaults after + * clearing, or make the value 0 or NULL. + * + * Here are the use cases: + * 1. A non-empty AllowInvalid line in your torrc. Appends to current + * if linelist, replaces current if csv. + * 2. An empty AllowInvalid line in your torrc. Should clear it. + * 3. "RESETCONF AllowInvalid" sets it to default. + * 4. "SETCONF AllowInvalid" makes it NULL. + * 5. "SETCONF AllowInvalid=foo" clears it and sets it to "foo". + * + * Use_defaults Clear_first + * 0 0 "append" + * 1 0 undefined, don't use + * 0 1 "set to null first" + * 1 1 "set to defaults first" + * Return 0 on success, -1 on bad key, -2 on bad value. + * + * As an additional special case, if a LINELIST config option has + * no value and clear_first is 0, then warn and ignore it. + */ + +/* +There are three call cases for config_assign() currently. + +Case one: Torrc entry +options_init_from_torrc() calls config_assign(0, 0) + calls config_assign_line(0, 0). + if value is empty, calls config_reset(0) and returns. + calls config_assign_value(), appends. + +Case two: setconf +options_trial_assign() calls config_assign(0, 1) + calls config_reset_line(0) + calls config_reset(0) + calls option_clear(). + calls config_assign_line(0, 1). + if value is empty, returns. + calls config_assign_value(), appends. + +Case three: resetconf +options_trial_assign() calls config_assign(1, 1) + calls config_reset_line(1) + calls config_reset(1) + calls option_clear(). + calls config_assign_value(default) + calls config_assign_line(1, 1). + returns. +*/ +int +config_assign(const config_format_t *fmt, void *options, config_line_t *list, + int use_defaults, int clear_first, char **msg) +{ + config_line_t *p; + bitarray_t *options_seen; + const int n_options = config_count_options(fmt); + + CONFIG_CHECK(fmt, options); + + /* pass 1: normalize keys */ + for (p = list; p; p = p->next) { + const char *full = config_expand_abbrev(fmt, p->key, 0, 1); + if (strcmp(full,p->key)) { + tor_free(p->key); + p->key = tor_strdup(full); + } + } + + /* pass 2: if we're reading from a resetting source, clear all + * mentioned config options, and maybe set to their defaults. */ + if (clear_first) { + for (p = list; p; p = p->next) + config_reset_line(fmt, options, p->key, use_defaults); + } + + options_seen = bitarray_init_zero(n_options); + /* pass 3: assign. */ + while (list) { + int r; + if ((r=config_assign_line(fmt, options, list, use_defaults, + clear_first, options_seen, msg))) { + bitarray_free(options_seen); + return r; + } + list = list->next; + } + bitarray_free(options_seen); + + /** Now we're done assigning a group of options to the configuration. + * Subsequent group assignments should _replace_ linelists, not extend + * them. */ + config_mark_lists_fragile(fmt, options); + + return 0; +} + +/** Reset config option <b>var</b> to 0, 0.0, NULL, or the equivalent. + * Called from config_reset() and config_free(). */ +static void +config_clear(const config_format_t *fmt, void *options, + const config_var_t *var) +{ + void *lvalue = STRUCT_VAR_P(options, var->var_offset); + (void)fmt; /* unused */ + switch (var->type) { + case CONFIG_TYPE_STRING: + case CONFIG_TYPE_FILENAME: + tor_free(*(char**)lvalue); + break; + case CONFIG_TYPE_DOUBLE: + *(double*)lvalue = 0.0; + break; + case CONFIG_TYPE_ISOTIME: + *(time_t*)lvalue = 0; + break; + case CONFIG_TYPE_INTERVAL: + case CONFIG_TYPE_MSEC_INTERVAL: + case CONFIG_TYPE_UINT: + case CONFIG_TYPE_INT: + case CONFIG_TYPE_PORT: + case CONFIG_TYPE_BOOL: + *(int*)lvalue = 0; + break; + case CONFIG_TYPE_AUTOBOOL: + *(int*)lvalue = -1; + break; + case CONFIG_TYPE_MEMUNIT: + *(uint64_t*)lvalue = 0; + break; + case CONFIG_TYPE_ROUTERSET: + if (*(routerset_t**)lvalue) { + routerset_free(*(routerset_t**)lvalue); + *(routerset_t**)lvalue = NULL; + } + break; + case CONFIG_TYPE_CSV: + if (*(smartlist_t**)lvalue) { + SMARTLIST_FOREACH(*(smartlist_t **)lvalue, char *, cp, tor_free(cp)); + smartlist_free(*(smartlist_t **)lvalue); + *(smartlist_t **)lvalue = NULL; + } + break; + case CONFIG_TYPE_LINELIST: + case CONFIG_TYPE_LINELIST_S: + config_free_lines(*(config_line_t **)lvalue); + *(config_line_t **)lvalue = NULL; + break; + case CONFIG_TYPE_LINELIST_V: + /* handled by linelist_s. */ + break; + case CONFIG_TYPE_OBSOLETE: + break; + } +} + +/** Clear the option indexed by <b>var</b> in <b>options</b>. Then if + * <b>use_defaults</b>, set it to its default value. + * Called by config_init() and option_reset_line() and option_assign_line(). */ +static void +config_reset(const config_format_t *fmt, void *options, + const config_var_t *var, int use_defaults) +{ + config_line_t *c; + char *msg = NULL; + CONFIG_CHECK(fmt, options); + config_clear(fmt, options, var); /* clear it first */ + if (!use_defaults) + return; /* all done */ + if (var->initvalue) { + c = tor_malloc_zero(sizeof(config_line_t)); + c->key = tor_strdup(var->name); + c->value = tor_strdup(var->initvalue); + if (config_assign_value(fmt, options, c, &msg) < 0) { + log_warn(LD_BUG, "Failed to assign default: %s", msg); + tor_free(msg); /* if this happens it's a bug */ + } + config_free_lines(c); + } +} + +/** Release storage held by <b>options</b>. */ +void +config_free(const config_format_t *fmt, void *options) +{ + int i; + + if (!options) + return; + + tor_assert(fmt); + + for (i=0; fmt->vars[i].name; ++i) + config_clear(fmt, options, &(fmt->vars[i])); + if (fmt->extra) { + config_line_t **linep = STRUCT_VAR_P(options, fmt->extra->var_offset); + config_free_lines(*linep); + *linep = NULL; + } + tor_free(options); +} + +/** Return true iff a and b contain identical keys and values in identical + * order. */ +int +config_lines_eq(config_line_t *a, config_line_t *b) +{ + while (a && b) { + if (strcasecmp(a->key, b->key) || strcmp(a->value, b->value)) + return 0; + a = a->next; + b = b->next; + } + if (a || b) + return 0; + return 1; +} + +/** Return the number of lines in <b>a</b> whose key is <b>key</b>. */ +int +config_count_key(const config_line_t *a, const char *key) +{ + int n = 0; + while (a) { + if (!strcasecmp(a->key, key)) { + ++n; + } + a = a->next; + } + return n; +} + +/** Return true iff the option <b>name</b> has the same value in <b>o1</b> + * and <b>o2</b>. Must not be called for LINELIST_S or OBSOLETE options. + */ +int +config_is_same(const config_format_t *fmt, + const void *o1, const void *o2, + const char *name) +{ + config_line_t *c1, *c2; + int r = 1; + CONFIG_CHECK(fmt, o1); + CONFIG_CHECK(fmt, o2); + + c1 = config_get_assigned_option(fmt, o1, name, 0); + c2 = config_get_assigned_option(fmt, o2, name, 0); + r = config_lines_eq(c1, c2); + config_free_lines(c1); + config_free_lines(c2); + return r; +} + +/** Copy storage held by <b>old</b> into a new or_options_t and return it. */ +void * +config_dup(const config_format_t *fmt, const void *old) +{ + void *newopts; + int i; + config_line_t *line; + + newopts = config_new(fmt); + for (i=0; fmt->vars[i].name; ++i) { + if (fmt->vars[i].type == CONFIG_TYPE_LINELIST_S) + continue; + if (fmt->vars[i].type == CONFIG_TYPE_OBSOLETE) + continue; + line = config_get_assigned_option(fmt, old, fmt->vars[i].name, 0); + if (line) { + char *msg = NULL; + if (config_assign(fmt, newopts, line, 0, 0, &msg) < 0) { + log_err(LD_BUG, "config_get_assigned_option() generated " + "something we couldn't config_assign(): %s", msg); + tor_free(msg); + tor_assert(0); + } + } + config_free_lines(line); + } + return newopts; +} +/** Set all vars in the configuration object <b>options</b> to their default + * values. */ +void +config_init(const config_format_t *fmt, void *options) +{ + int i; + const config_var_t *var; + CONFIG_CHECK(fmt, options); + + for (i=0; fmt->vars[i].name; ++i) { + var = &fmt->vars[i]; + if (!var->initvalue) + continue; /* defaults to NULL or 0 */ + config_reset(fmt, options, var, 1); + } +} + +/** Allocate and return a new string holding the written-out values of the vars + * in 'options'. If 'minimal', do not write out any default-valued vars. + * Else, if comment_defaults, write default values as comments. + */ +char * +config_dump(const config_format_t *fmt, const void *default_options, + const void *options, int minimal, + int comment_defaults) +{ + smartlist_t *elements; + const void *defaults = default_options; + void *defaults_tmp = NULL; + config_line_t *line, *assigned; + char *result; + int i; + char *msg = NULL; + + if (defaults == NULL) { + defaults = defaults_tmp = config_new(fmt); + config_init(fmt, defaults_tmp); + } + + /* XXX use a 1 here so we don't add a new log line while dumping */ + if (default_options == NULL) { + if (fmt->validate_fn(NULL, defaults_tmp, 1, &msg) < 0) { + log_err(LD_BUG, "Failed to validate default config."); + tor_free(msg); + tor_assert(0); + } + } + + elements = smartlist_new(); + for (i=0; fmt->vars[i].name; ++i) { + int comment_option = 0; + if (fmt->vars[i].type == CONFIG_TYPE_OBSOLETE || + fmt->vars[i].type == CONFIG_TYPE_LINELIST_S) + continue; + /* Don't save 'hidden' control variables. */ + if (!strcmpstart(fmt->vars[i].name, "__")) + continue; + if (minimal && config_is_same(fmt, options, defaults, fmt->vars[i].name)) + continue; + else if (comment_defaults && + config_is_same(fmt, options, defaults, fmt->vars[i].name)) + comment_option = 1; + + line = assigned = + config_get_assigned_option(fmt, options, fmt->vars[i].name, 1); + + for (; line; line = line->next) { + smartlist_add_asprintf(elements, "%s%s %s\n", + comment_option ? "# " : "", + line->key, line->value); + } + config_free_lines(assigned); + } + + if (fmt->extra) { + line = *(config_line_t**)STRUCT_VAR_P(options, fmt->extra->var_offset); + for (; line; line = line->next) { + smartlist_add_asprintf(elements, "%s %s\n", line->key, line->value); + } + } + + result = smartlist_join_strings(elements, "", 0, NULL); + SMARTLIST_FOREACH(elements, char *, cp, tor_free(cp)); + smartlist_free(elements); + if (defaults_tmp) + config_free(fmt, defaults_tmp); + return result; +} + +/** Mapping from a unit name to a multiplier for converting that unit into a + * base unit. Used by config_parse_unit. */ +struct unit_table_t { + const char *unit; /**< The name of the unit */ + uint64_t multiplier; /**< How many of the base unit appear in this unit */ +}; + +/** Table to map the names of memory units to the number of bytes they + * contain. */ +static struct unit_table_t memory_units[] = { + { "", 1 }, + { "b", 1<< 0 }, + { "byte", 1<< 0 }, + { "bytes", 1<< 0 }, + { "kb", 1<<10 }, + { "kbyte", 1<<10 }, + { "kbytes", 1<<10 }, + { "kilobyte", 1<<10 }, + { "kilobytes", 1<<10 }, + { "m", 1<<20 }, + { "mb", 1<<20 }, + { "mbyte", 1<<20 }, + { "mbytes", 1<<20 }, + { "megabyte", 1<<20 }, + { "megabytes", 1<<20 }, + { "gb", 1<<30 }, + { "gbyte", 1<<30 }, + { "gbytes", 1<<30 }, + { "gigabyte", 1<<30 }, + { "gigabytes", 1<<30 }, + { "tb", U64_LITERAL(1)<<40 }, + { "terabyte", U64_LITERAL(1)<<40 }, + { "terabytes", U64_LITERAL(1)<<40 }, + { NULL, 0 }, +}; + +/** Table to map the names of time units to the number of seconds they + * contain. */ +static struct unit_table_t time_units[] = { + { "", 1 }, + { "second", 1 }, + { "seconds", 1 }, + { "minute", 60 }, + { "minutes", 60 }, + { "hour", 60*60 }, + { "hours", 60*60 }, + { "day", 24*60*60 }, + { "days", 24*60*60 }, + { "week", 7*24*60*60 }, + { "weeks", 7*24*60*60 }, + { NULL, 0 }, +}; + +/** Table to map the names of time units to the number of milliseconds + * they contain. */ +static struct unit_table_t time_msec_units[] = { + { "", 1 }, + { "msec", 1 }, + { "millisecond", 1 }, + { "milliseconds", 1 }, + { "second", 1000 }, + { "seconds", 1000 }, + { "minute", 60*1000 }, + { "minutes", 60*1000 }, + { "hour", 60*60*1000 }, + { "hours", 60*60*1000 }, + { "day", 24*60*60*1000 }, + { "days", 24*60*60*1000 }, + { "week", 7*24*60*60*1000 }, + { "weeks", 7*24*60*60*1000 }, + { NULL, 0 }, +}; + +/** Parse a string <b>val</b> containing a number, zero or more + * spaces, and an optional unit string. If the unit appears in the + * table <b>u</b>, then multiply the number by the unit multiplier. + * On success, set *<b>ok</b> to 1 and return this product. + * Otherwise, set *<b>ok</b> to 0. + */ +static uint64_t +config_parse_units(const char *val, struct unit_table_t *u, int *ok) +{ + uint64_t v = 0; + double d = 0; + int use_float = 0; + char *cp; + + tor_assert(ok); + + v = tor_parse_uint64(val, 10, 0, UINT64_MAX, ok, &cp); + if (!*ok || (cp && *cp == '.')) { + d = tor_parse_double(val, 0, UINT64_MAX, ok, &cp); + if (!*ok) + goto done; + use_float = 1; + } + + if (!cp) { + *ok = 1; + v = use_float ? DBL_TO_U64(d) : v; + goto done; + } + + cp = (char*) eat_whitespace(cp); + + for ( ;u->unit;++u) { + if (!strcasecmp(u->unit, cp)) { + if (use_float) + v = u->multiplier * d; + else + v *= u->multiplier; + *ok = 1; + goto done; + } + } + log_warn(LD_CONFIG, "Unknown unit '%s'.", cp); + *ok = 0; + done: + + if (*ok) + return v; + else + return 0; +} + +/** Parse a string in the format "number unit", where unit is a unit of + * information (byte, KB, M, etc). On success, set *<b>ok</b> to true + * and return the number of bytes specified. Otherwise, set + * *<b>ok</b> to false and return 0. */ +static uint64_t +config_parse_memunit(const char *s, int *ok) +{ + uint64_t u = config_parse_units(s, memory_units, ok); + return u; +} + +/** Parse a string in the format "number unit", where unit is a unit of + * time in milliseconds. On success, set *<b>ok</b> to true and return + * the number of milliseconds in the provided interval. Otherwise, set + * *<b>ok</b> to 0 and return -1. */ +static int +config_parse_msec_interval(const char *s, int *ok) +{ + uint64_t r; + r = config_parse_units(s, time_msec_units, ok); + if (!ok) + return -1; + if (r > INT_MAX) { + log_warn(LD_CONFIG, "Msec interval '%s' is too long", s); + *ok = 0; + return -1; + } + return (int)r; +} + +/** Parse a string in the format "number unit", where unit is a unit of time. + * On success, set *<b>ok</b> to true and return the number of seconds in + * the provided interval. Otherwise, set *<b>ok</b> to 0 and return -1. + */ +static int +config_parse_interval(const char *s, int *ok) +{ + uint64_t r; + r = config_parse_units(s, time_units, ok); + if (!ok) + return -1; + if (r > INT_MAX) { + log_warn(LD_CONFIG, "Interval '%s' is too long", s); + *ok = 0; + return -1; + } + return (int)r; +} + diff --git a/src/or/confparse.h b/src/or/confparse.h new file mode 100644 index 000000000..f33208eb5 --- /dev/null +++ b/src/or/confparse.h @@ -0,0 +1,132 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef TOR_CONFPARSE_H +#define TOR_CONFPARSE_H + +/** Enumeration of types which option values can take */ +typedef enum config_type_t { + CONFIG_TYPE_STRING = 0, /**< An arbitrary string. */ + CONFIG_TYPE_FILENAME, /**< A filename: some prefixes get expanded. */ + CONFIG_TYPE_UINT, /**< A non-negative integer less than MAX_INT */ + CONFIG_TYPE_INT, /**< Any integer. */ + CONFIG_TYPE_PORT, /**< A port from 1...65535, 0 for "not set", or + * "auto". */ + CONFIG_TYPE_INTERVAL, /**< A number of seconds, with optional units*/ + CONFIG_TYPE_MSEC_INTERVAL,/**< A number of milliseconds, with optional + * units */ + CONFIG_TYPE_MEMUNIT, /**< A number of bytes, with optional units*/ + CONFIG_TYPE_DOUBLE, /**< A floating-point value */ + CONFIG_TYPE_BOOL, /**< A boolean value, expressed as 0 or 1. */ + CONFIG_TYPE_AUTOBOOL, /**< A boolean+auto value, expressed 0 for false, + * 1 for true, and -1 for auto */ + CONFIG_TYPE_ISOTIME, /**< An ISO-formatted time relative to GMT. */ + CONFIG_TYPE_CSV, /**< A list of strings, separated by commas and + * optional whitespace. */ + CONFIG_TYPE_LINELIST, /**< Uninterpreted config lines */ + CONFIG_TYPE_LINELIST_S, /**< Uninterpreted, context-sensitive config lines, + * mixed with other keywords. */ + CONFIG_TYPE_LINELIST_V, /**< Catch-all "virtual" option to summarize + * context-sensitive config lines when fetching. + */ + CONFIG_TYPE_ROUTERSET, /**< A list of router names, addrs, and fps, + * parsed into a routerset_t. */ + CONFIG_TYPE_OBSOLETE, /**< Obsolete (ignored) option. */ +} config_type_t; + +/** An abbreviation for a configuration option allowed on the command line. */ +typedef struct config_abbrev_t { + const char *abbreviated; + const char *full; + int commandline_only; + int warn; +} config_abbrev_t; + +/* Handy macro for declaring "In the config file or on the command line, + * you can abbreviate <b>tok</b>s as <b>tok</b>". */ +#define PLURAL(tok) { #tok, #tok "s", 0, 0 } + +/** A variable allowed in the configuration file or on the command line. */ +typedef struct config_var_t { + const char *name; /**< The full keyword (case insensitive). */ + config_type_t type; /**< How to interpret the type and turn it into a + * value. */ + off_t var_offset; /**< Offset of the corresponding member of or_options_t. */ + const char *initvalue; /**< String (or null) describing initial value. */ +} config_var_t; + +/** Represents an English description of a configuration variable; used when + * generating configuration file comments. */ +typedef struct config_var_description_t { + const char *name; + const char *description; +} config_var_description_t; + +/** Type of a callback to validate whether a given configuration is + * well-formed and consistent. See options_trial_assign() for documentation + * of arguments. */ +typedef int (*validate_fn_t)(void*,void*,int,char**); + +/** Information on the keys, value types, key-to-struct-member mappings, + * variable descriptions, validation functions, and abbreviations for a + * configuration or storage format. */ +typedef struct { + size_t size; /**< Size of the struct that everything gets parsed into. */ + uint32_t magic; /**< Required 'magic value' to make sure we have a struct + * of the right type. */ + off_t magic_offset; /**< Offset of the magic value within the struct. */ + config_abbrev_t *abbrevs; /**< List of abbreviations that we expand when + * parsing this format. */ + config_var_t *vars; /**< List of variables we recognize, their default + * values, and where we stick them in the structure. */ + validate_fn_t validate_fn; /**< Function to validate config. */ + /** If present, extra is a LINELIST variable for unrecognized + * lines. Otherwise, unrecognized lines are an error. */ + config_var_t *extra; +} config_format_t; + +/** Macro: assert that <b>cfg</b> has the right magic field for format + * <b>fmt</b>. */ +#define CONFIG_CHECK(fmt, cfg) STMT_BEGIN \ + tor_assert(fmt && cfg); \ + tor_assert((fmt)->magic == \ + *(uint32_t*)STRUCT_VAR_P(cfg,fmt->magic_offset)); \ + STMT_END + +void *config_new(const config_format_t *fmt); +void config_line_append(config_line_t **lst, + const char *key, const char *val); +config_line_t *config_lines_dup(const config_line_t *inp); +void config_free(const config_format_t *fmt, void *options); +int config_lines_eq(config_line_t *a, config_line_t *b); +int config_count_key(const config_line_t *a, const char *key); +config_line_t *config_get_assigned_option(const config_format_t *fmt, + const void *options, const char *key, + int escape_val); +int config_is_same(const config_format_t *fmt, + const void *o1, const void *o2, + const char *name); +void config_init(const config_format_t *fmt, void *options); +void *config_dup(const config_format_t *fmt, const void *old); +char *config_dump(const config_format_t *fmt, const void *default_options, + const void *options, int minimal, + int comment_defaults); +int config_assign(const config_format_t *fmt, void *options, + config_line_t *list, + int use_defaults, int clear_first, char **msg); +config_var_t *config_find_option_mutable(config_format_t *fmt, + const char *key); +const config_var_t *config_find_option(const config_format_t *fmt, + const char *key); + +int config_get_lines(const char *string, config_line_t **result, int extended); +void config_free_lines(config_line_t *front); +const char *config_expand_abbrev(const config_format_t *fmt, + const char *option, + int command_line, int warn_obsolete); + +#endif + diff --git a/src/or/connection.c b/src/or/connection.c index 364e4912d..d8f5d875c 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -34,6 +34,7 @@ #include "rendcommon.h" #include "rephist.h" #include "router.h" +#include "transports.h" #include "routerparse.h" #ifdef USE_BUFFEREVENTS @@ -238,7 +239,16 @@ dir_connection_new(int socket_family) } /** Allocate and return a new or_connection_t, initialized as by - * connection_init(). */ + * connection_init(). + * + * Set timestamp_last_added_nonpadding to now. + * + * Assign a pseudorandom next_circ_id between 0 and 2**15. + * + * Initialize active_circuit_pqueue. + * + * Set active_circuit_pqueue_last_recalibrated to current cell_ewma tick. + */ or_connection_t * or_connection_new(int socket_family) { @@ -256,7 +266,10 @@ or_connection_new(int socket_family) } /** Allocate and return a new entry_connection_t, initialized as by - * connection_init(). */ + * connection_init(). + * + * Allocate space to store the socks_request. + */ entry_connection_t * entry_connection_new(int type, int socket_family) { @@ -338,14 +351,11 @@ connection_new(int type, int socket_family) /** Initializes conn. (you must call connection_add() to link it into the main * array). * + * Set conn-\>magic to the correct value. + * * Set conn-\>type to <b>type</b>. Set conn-\>s and conn-\>conn_array_index to * -1 to signify they are not yet assigned. * - * If conn is not a listener type, allocate buffers for it. If it's - * an AP type, allocate space to store the socks_request. - * - * Assign a pseudorandom next_circ_id between 0 and 2**15. - * * Initialize conn's timestamps to now. */ static void @@ -1089,7 +1099,7 @@ connection_listener_new(const struct sockaddr *listensockaddr, * nmap does). We want to detect that, and not go on with the connection. */ static int -check_sockaddr(struct sockaddr *sa, int len, int level) +check_sockaddr(const struct sockaddr *sa, int len, int level) { int ok = 1; @@ -1202,11 +1212,6 @@ connection_handle_listener_read(connection_t *conn, int new_type) return 0; } - if (check_sockaddr_family_match(remote->sa_family, conn) < 0) { - tor_close_socket(news); - return 0; - } - tor_addr_from_sockaddr(&addr, remote, &port); /* process entrance policies here, before we even create the connection */ @@ -2033,9 +2038,9 @@ connection_mark_all_noncontrol_connections(void) /** Return 1 if we should apply rate limiting to <b>conn</b>, and 0 * otherwise. * Right now this just checks if it's an internal IP address or an - * internal connection. We also check if the connection uses pluggable - * transports, since we should then limit it even if it comes from an - * internal IP address. */ + * internal connection. We also should, but don't, check if the connection + * uses pluggable transports, since we should then limit it even if it + * comes from an internal IP address. */ static int connection_is_rate_limited(connection_t *conn) { @@ -3360,13 +3365,6 @@ connection_flush(connection_t *conn) return connection_handle_write(conn, 1); } -/** OpenSSL TLS record size is 16383; this is close. The goal here is to - * push data out as soon as we know there's enough for a TLS record, so - * during periods of high load we won't read entire megabytes from - * input before pushing any data out. It also has the feature of not - * growing huge outbufs unless something is slow. */ -#define MIN_TLS_FLUSHLEN 15872 - /** Append <b>len</b> bytes of <b>string</b> onto <b>conn</b>'s * outbuf, and ask it to start writing. * @@ -3375,10 +3373,9 @@ connection_flush(connection_t *conn) * negative, this is the last data to be compressed, and the connection's zlib * state should be flushed. * - * If it's an OR conn and an entire TLS record is ready, then try to - * flush the record now. Similarly, if it's a local control connection - * and a 64k chunk is ready, try to flush it all, so we don't end up with - * many megabytes of controller info queued at once. + * If it's a local control connection and a 64k chunk is ready, try to flush + * it all, so we don't end up with many megabytes of controller info queued at + * once. */ void _connection_write_to_buf_impl(const char *string, size_t len, @@ -3446,7 +3443,6 @@ _connection_write_to_buf_impl(const char *string, size_t len, if (zlib) { conn->outbuf_flushlen += buf_datalen(conn->outbuf) - old_datalen; } else { - ssize_t extra = 0; conn->outbuf_flushlen += len; /* Should we try flushing the outbuf now? */ @@ -3456,14 +3452,7 @@ _connection_write_to_buf_impl(const char *string, size_t len, return; } - if (conn->type == CONN_TYPE_OR && - conn->outbuf_flushlen-len < MIN_TLS_FLUSHLEN && - conn->outbuf_flushlen >= MIN_TLS_FLUSHLEN) { - /* We just pushed outbuf_flushlen to MIN_TLS_FLUSHLEN or above; - * we can send out a full TLS frame now if we like. */ - extra = conn->outbuf_flushlen - MIN_TLS_FLUSHLEN; - conn->outbuf_flushlen = MIN_TLS_FLUSHLEN; - } else if (conn->type == CONN_TYPE_CONTROL && + if (conn->type == CONN_TYPE_CONTROL && !connection_is_rate_limited(conn) && conn->outbuf_flushlen-len < 1<<16 && conn->outbuf_flushlen >= 1<<16) { @@ -3483,10 +3472,6 @@ _connection_write_to_buf_impl(const char *string, size_t len, } return; } - if (extra) { - conn->outbuf_flushlen += extra; - connection_start_writing(conn); - } } } diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index 1592033c5..ade3b48df 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -33,6 +33,7 @@ #include "rephist.h" #include "router.h" #include "routerlist.h" +#include "routerset.h" #ifdef HAVE_LINUX_TYPES_H #include <linux/types.h> diff --git a/src/or/connection_or.c b/src/or/connection_or.c index d01638793..02345f98a 100644 --- a/src/or/connection_or.c +++ b/src/or/connection_or.c @@ -327,7 +327,7 @@ var_cell_t * var_cell_new(uint16_t payload_len) { size_t size = STRUCT_OFFSET(var_cell_t, payload) + payload_len; - var_cell_t *cell = tor_malloc(size); + var_cell_t *cell = tor_malloc_zero(size); cell->payload_len = payload_len; cell->command = 0; cell->circ_id = 0; @@ -1152,7 +1152,7 @@ connection_tls_start_handshake(or_connection_t *conn, int receiving) #ifdef USE_BUFFEREVENTS if (connection_type_uses_bufferevent(TO_CONN(conn))) { - const int filtering = get_options()->_UseFilteringSSLBufferevents; + const int filtering = get_options()->UseFilteringSSLBufferevents; struct bufferevent *b = tor_tls_init_bufferevent(conn->tls, conn->_base.bufev, conn->_base.s, receiving, filtering); @@ -1540,7 +1540,7 @@ connection_or_client_learned_peer_id(or_connection_t *conn, return -1; } if (authdir_mode_tests_reachability(options)) { - dirserv_orconn_tls_done(conn->_base.address, conn->_base.port, + dirserv_orconn_tls_done(&conn->_base.addr, conn->_base.port, (const char*)peer_id); } @@ -1988,12 +1988,19 @@ connection_or_send_netinfo(or_connection_t *conn) if ((public_server_mode(get_options()) || !conn->is_outgoing) && (me = router_get_my_routerinfo())) { tor_addr_t my_addr; - *out++ = 1; /* only one address is supported. */ + *out++ = 1 + !tor_addr_is_null(&me->ipv6_addr); tor_addr_from_ipv4h(&my_addr, me->addr); len = append_address_to_payload(out, &my_addr); if (len < 0) return -1; + out += len; + + if (!tor_addr_is_null(&me->ipv6_addr)) { + len = append_address_to_payload(out, &me->ipv6_addr); + if (len < 0) + return -1; + } } else { *out = 0; } diff --git a/src/or/control.c b/src/or/control.c index 913d18a7f..74c6acc00 100644 --- a/src/or/control.c +++ b/src/or/control.c @@ -16,6 +16,7 @@ #include "circuitlist.h" #include "circuituse.h" #include "config.h" +#include "confparse.h" #include "connection.h" #include "connection_edge.h" #include "connection_or.h" @@ -1243,6 +1244,27 @@ handle_control_saveconf(control_connection_t *conn, uint32_t len, return 0; } +struct signal_t { + int sig; + const char *signal_name; +}; + +static const struct signal_t signal_table[] = { + { SIGHUP, "RELOAD" }, + { SIGHUP, "HUP" }, + { SIGINT, "SHUTDOWN" }, + { SIGUSR1, "DUMP" }, + { SIGUSR1, "USR1" }, + { SIGUSR2, "DEBUG" }, + { SIGUSR2, "USR2" }, + { SIGTERM, "HALT" }, + { SIGTERM, "TERM" }, + { SIGTERM, "INT" }, + { SIGNEWNYM, "NEWNYM" }, + { SIGCLEARDNSCACHE, "CLEARDNSCACHE"}, + { 0, NULL }, +}; + /** Called when we get a SIGNAL command. React to the provided signal, and * report success or failure. (If the signal results in a shutdown, success * may not be reported.) */ @@ -1250,7 +1272,8 @@ static int handle_control_signal(control_connection_t *conn, uint32_t len, const char *body) { - int sig; + int sig = -1; + int i; int n = 0; char *s; @@ -1259,27 +1282,19 @@ handle_control_signal(control_connection_t *conn, uint32_t len, while (body[n] && ! TOR_ISSPACE(body[n])) ++n; s = tor_strndup(body, n); - if (!strcasecmp(s, "RELOAD") || !strcasecmp(s, "HUP")) - sig = SIGHUP; - else if (!strcasecmp(s, "SHUTDOWN") || !strcasecmp(s, "INT")) - sig = SIGINT; - else if (!strcasecmp(s, "DUMP") || !strcasecmp(s, "USR1")) - sig = SIGUSR1; - else if (!strcasecmp(s, "DEBUG") || !strcasecmp(s, "USR2")) - sig = SIGUSR2; - else if (!strcasecmp(s, "HALT") || !strcasecmp(s, "TERM")) - sig = SIGTERM; - else if (!strcasecmp(s, "NEWNYM")) - sig = SIGNEWNYM; - else if (!strcasecmp(s, "CLEARDNSCACHE")) - sig = SIGCLEARDNSCACHE; - else { + + for (i = 0; signal_table[i].signal_name != NULL; ++i) { + if (!strcasecmp(s, signal_table[i].signal_name)) { + sig = signal_table[i].sig; + break; + } + } + + if (sig < 0) connection_printf_to_buf(conn, "552 Unrecognized signal code \"%s\"\r\n", s); - sig = -1; - } tor_free(s); - if (sig<0) + if (sig < 0) return 0; send_control_done(conn); @@ -1440,6 +1455,16 @@ getinfo_helper_misc(control_connection_t *conn, const char *question, *answer = smartlist_join_strings(event_names, " ", 0, NULL); smartlist_free(event_names); + } else if (!strcmp(question, "signal/names")) { + smartlist_t *signal_names = smartlist_new(); + int j; + for (j = 0; signal_table[j].signal_name != NULL; ++j) { + smartlist_add(signal_names, (char*)signal_table[j].signal_name); + } + + *answer = smartlist_join_strings(signal_names, " ", 0, NULL); + + smartlist_free(signal_names); } else if (!strcmp(question, "features/names")) { *answer = tor_strdup("VERBOSE_NAMES EXTENDED_EVENTS"); } else if (!strcmp(question, "address")) { @@ -1614,10 +1639,13 @@ getinfo_helper_dir(control_connection_t *control_conn, const char *question, char **answer, const char **errmsg) { - const routerinfo_t *ri; + const node_t *node; + const routerinfo_t *ri = NULL; (void) control_conn; if (!strcmpstart(question, "desc/id/")) { - ri = router_get_by_hexdigest(question+strlen("desc/id/")); + node = node_get_by_hex_id(question+strlen("desc/id/")); + if (node) + ri = node->ri; if (ri) { const char *body = signed_descriptor_get_body(&ri->cache_info); if (body) @@ -1626,7 +1654,9 @@ getinfo_helper_dir(control_connection_t *control_conn, } else if (!strcmpstart(question, "desc/name/")) { /* XXX023 Setting 'warn_if_unnamed' here is a bit silly -- the * warning goes to the user, not to the controller. */ - ri = router_get_by_nickname(question+strlen("desc/name/"),1); + node = node_get_by_nickname(question+strlen("desc/name/"), 1); + if (node) + ri = node->ri; if (ri) { const char *body = signed_descriptor_get_body(&ri->cache_info); if (body) @@ -1688,8 +1718,9 @@ getinfo_helper_dir(control_connection_t *control_conn, *answer = tor_strndup(md->body, md->bodylen); } } else if (!strcmpstart(question, "desc-annotations/id/")) { - ri = router_get_by_hexdigest(question+ - strlen("desc-annotations/id/")); + node = node_get_by_hex_id(question+strlen("desc-annotations/id/")); + if (node) + ri = node->ri; if (ri) { const char *annotations = signed_descriptor_get_annotations(&ri->cache_info); @@ -2130,10 +2161,14 @@ static const getinfo_item_t getinfo_items[] = { PREFIX("config/", config, "Current configuration values."), DOC("config/names", "List of configuration options, types, and documentation."), + DOC("config/defaults", + "List of default values for configuration options. " + "See also config/names"), ITEM("info/names", misc, "List of GETINFO options, types, and documentation."), ITEM("events/names", misc, "Events that the controller can ask for with SETEVENTS."), + ITEM("signal/names", misc, "Signal names recognized by the SIGNAL command"), ITEM("features/names", misc, "What arguments can USEFEATURE take?"), PREFIX("desc/id/", dir, "Router descriptors by ID."), PREFIX("desc/name/", dir, "Router descriptors by nickname."), diff --git a/src/or/directory.c b/src/or/directory.c index f1510b970..7df91fb57 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -25,6 +25,7 @@ #include "router.h" #include "routerlist.h" #include "routerparse.h" +#include "routerset.h" #if defined(EXPORTMALLINFO) && defined(HAVE_MALLOC_H) && defined(HAVE_MALLINFO) #ifndef OPENBSD @@ -58,7 +59,6 @@ static void directory_send_command(dir_connection_t *conn, int purpose, int direct, const char *resource, const char *payload, size_t payload_len, - int supports_conditional_consensus, time_t if_modified_since); static int directory_handle_command(dir_connection_t *conn); static int body_is_plausible(const char *body, size_t body_len, int purpose); @@ -89,12 +89,10 @@ static void directory_initiate_command_rend(const char *address, const tor_addr_t *addr, uint16_t or_port, uint16_t dir_port, - int supports_conditional_consensus, - int supports_begindir, const char *digest, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, @@ -227,16 +225,9 @@ router_supports_extrainfo(const char *identity_digest, int is_authority) if (node && node->ri) { if (node->ri->caches_extra_info) return 1; - if (is_authority && node->ri->platform && - tor_version_as_new_as(node->ri->platform, - "Tor 0.2.0.0-alpha-dev (r10070)")) - return 1; } if (is_authority) { - const routerstatus_t *rs = - router_get_consensus_status_by_id(identity_digest); - if (rs && rs->version_supports_extrainfo_upload) - return 1; + return 1; } return 0; } @@ -431,8 +422,6 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, * the behavior supported by our oldest bridge; see for example * any_bridges_dont_support_microdescriptors(). */ - /* XXX024 Not all bridges handle conditional consensus downloading, - * so, for now, never assume the server supports that. -PP */ const node_t *node = choose_random_entry(NULL); if (node && node->ri) { /* every bridge has a routerinfo. */ @@ -440,12 +429,12 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, routerinfo_t *ri = node->ri; node_get_addr(node, &addr); directory_initiate_command(ri->address, &addr, - ri->or_port, 0, - 0, /* don't use conditional consensus url */ - 1, ri->cache_info.identity_digest, + ri->or_port, 0/*no dirport*/, + ri->cache_info.identity_digest, dir_purpose, router_purpose, - 0, resource, NULL, 0, if_modified_since); + DIRIND_ONEHOP, + resource, NULL, 0, if_modified_since); } else log_notice(LD_DIR, "Ignoring directory request, since no bridge " "nodes are available yet."); @@ -506,13 +495,15 @@ directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, } } - if (rs) + if (rs) { + const dir_indirection_t indirection = + get_via_tor ? DIRIND_ANONYMOUS : DIRIND_ONEHOP; directory_initiate_command_routerstatus(rs, dir_purpose, router_purpose, - get_via_tor, + indirection, resource, NULL, 0, if_modified_since); - else { + } else { log_notice(LD_DIR, "While fetching directory info, " "no running dirservers known. Will try again later. " @@ -544,17 +535,25 @@ directory_get_from_all_authorities(uint8_t dir_purpose, continue; rs = &ds->fake_status; directory_initiate_command_routerstatus(rs, dir_purpose, router_purpose, - 0, resource, NULL, 0, 0); + DIRIND_ONEHOP, resource, NULL, + 0, 0); } SMARTLIST_FOREACH_END(ds); } +/** Return true iff <b>ind</b> requires a multihop circuit. */ +static int +dirind_is_anon(dir_indirection_t ind) +{ + return ind == DIRIND_ANON_DIRPORT || ind == DIRIND_ANONYMOUS; +} + /** Same as directory_initiate_command_routerstatus(), but accepts * rendezvous data to fetch a hidden service descriptor. */ void directory_initiate_command_routerstatus_rend(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, @@ -567,6 +566,7 @@ directory_initiate_command_routerstatus_rend(const routerstatus_t *status, struct in_addr in; const char *address; tor_addr_t addr; + const int anonymized_connection = dirind_is_anon(indirection); node = node_get_by_id(status->identity_digest); if (!node && anonymized_connection) { @@ -596,11 +596,9 @@ directory_initiate_command_routerstatus_rend(const routerstatus_t *status, directory_initiate_command_rend(address, &addr, status->or_port, status->dir_port, - status->version_supports_conditional_consensus, - status->version_supports_begindir, status->identity_digest, dir_purpose, router_purpose, - anonymized_connection, resource, + indirection, resource, payload, payload_len, if_modified_since, rend_query); } @@ -623,7 +621,7 @@ void directory_initiate_command_routerstatus(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, @@ -631,7 +629,7 @@ directory_initiate_command_routerstatus(const routerstatus_t *status, { directory_initiate_command_routerstatus_rend(status, dir_purpose, router_purpose, - anonymized_connection, resource, + indirection, resource, payload, payload_len, if_modified_since, NULL); } @@ -833,11 +831,13 @@ static int directory_command_should_use_begindir(const or_options_t *options, const tor_addr_t *addr, int or_port, uint8_t router_purpose, - int anonymized_connection) + dir_indirection_t indirection) { if (!or_port) return 0; /* We don't know an ORPort -- no chance. */ - if (!anonymized_connection) + if (indirection == DIRIND_DIRECT_CONN || indirection == DIRIND_ANON_DIRPORT) + return 0; + if (indirection == DIRIND_ONEHOP) if (!fascist_firewall_allows_address_or(addr, or_port) || directory_fetches_from_authorities(options)) return 0; /* We're firewalled or are acting like a relay -- also no. */ @@ -855,17 +855,15 @@ directory_command_should_use_begindir(const or_options_t *options, void directory_initiate_command(const char *address, const tor_addr_t *_addr, uint16_t or_port, uint16_t dir_port, - int supports_conditional_consensus, - int supports_begindir, const char *digest, + const char *digest, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, const char *resource, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, time_t if_modified_since) { directory_initiate_command_rend(address, _addr, or_port, dir_port, - supports_conditional_consensus, - supports_begindir, digest, dir_purpose, - router_purpose, anonymized_connection, + digest, dir_purpose, + router_purpose, indirection, resource, payload, payload_len, if_modified_since, NULL); } @@ -889,10 +887,9 @@ is_sensitive_dir_purpose(uint8_t dir_purpose) static void directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, uint16_t or_port, uint16_t dir_port, - int supports_conditional_consensus, - int supports_begindir, const char *digest, + const char *digest, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, time_t if_modified_since, @@ -901,9 +898,9 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, dir_connection_t *conn; const or_options_t *options = get_options(); int socket_error = 0; - int use_begindir = supports_begindir && - directory_command_should_use_begindir(options, _addr, - or_port, router_purpose, anonymized_connection); + int use_begindir = directory_command_should_use_begindir(options, _addr, + or_port, router_purpose, indirection); + const int anonymized_connection = dirind_is_anon(indirection); tor_addr_t addr; tor_assert(address); @@ -949,6 +946,7 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, conn->_base.state = DIR_CONN_STATE_CONNECTING; /* decide whether we can learn our IP address from this conn */ + /* XXXX This is a bad name for this field now. */ conn->dirconn_direct = !anonymized_connection; /* copy rendezvous data, if any */ @@ -979,7 +977,6 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, /* queue the command on the outbuf */ directory_send_command(conn, dir_purpose, 1, resource, payload, payload_len, - supports_conditional_consensus, if_modified_since); connection_watch_events(TO_CONN(conn), READ_EVENT | WRITE_EVENT); /* writable indicates finish, readable indicates broken link, @@ -1024,7 +1021,6 @@ directory_initiate_command_rend(const char *address, const tor_addr_t *_addr, /* queue the command on the outbuf */ directory_send_command(conn, dir_purpose, 0, resource, payload, payload_len, - supports_conditional_consensus, if_modified_since); connection_watch_events(TO_CONN(conn), READ_EVENT|WRITE_EVENT); @@ -1074,8 +1070,7 @@ _compare_strs(const void **a, const void **b) * If 'resource' is provided, it is the name of a consensus flavor to request. */ static char * -directory_get_consensus_url(int supports_conditional_consensus, - const char *resource) +directory_get_consensus_url(const char *resource) { char *url = NULL; const char *hyphen, *flavor; @@ -1087,7 +1082,7 @@ directory_get_consensus_url(int supports_conditional_consensus, hyphen = "-"; } - if (supports_conditional_consensus) { + { char *authority_id_list; smartlist_t *authority_digests = smartlist_new(); @@ -1112,9 +1107,6 @@ directory_get_consensus_url(int supports_conditional_consensus, SMARTLIST_FOREACH(authority_digests, char *, cp, tor_free(cp)); smartlist_free(authority_digests); tor_free(authority_id_list); - } else { - tor_asprintf(&url, "/tor/status-vote/current/consensus%s%s.z", - hyphen, flavor); } return url; } @@ -1126,7 +1118,6 @@ static void directory_send_command(dir_connection_t *conn, int purpose, int direct, const char *resource, const char *payload, size_t payload_len, - int supports_conditional_consensus, time_t if_modified_since) { char proxystring[256]; @@ -1189,8 +1180,7 @@ directory_send_command(dir_connection_t *conn, /* resource is optional. If present, it's a flavor name */ tor_assert(!payload); httpcommand = "GET"; - url = directory_get_consensus_url(supports_conditional_consensus, - resource); + url = directory_get_consensus_url(resource); log_info(LD_DIR, "Downloading consensus from %s using %s", hoststring, url); break; diff --git a/src/or/directory.h b/src/or/directory.h index 1ca1c5a6e..fef671a0e 100644 --- a/src/or/directory.h +++ b/src/or/directory.h @@ -22,10 +22,24 @@ void directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose, void directory_get_from_all_authorities(uint8_t dir_purpose, uint8_t router_purpose, const char *resource); + +/** Enumeration of ways to connect to a directory server */ +typedef enum { + /** Default: connect over a one-hop Tor circuit but fall back to direct + * connection */ + DIRIND_ONEHOP=0, + /** Connect over a multi-hop anonymizing Tor circuit */ + DIRIND_ANONYMOUS=1, + /** Conncet to the DirPort directly */ + DIRIND_DIRECT_CONN, + /** Connect over a multi-hop anonymizing Tor circuit to our dirport */ + DIRIND_ANON_DIRPORT, +} dir_indirection_t; + void directory_initiate_command_routerstatus(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, @@ -33,7 +47,7 @@ void directory_initiate_command_routerstatus(const routerstatus_t *status, void directory_initiate_command_routerstatus_rend(const routerstatus_t *status, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, @@ -51,10 +65,9 @@ int connection_dir_finished_connecting(dir_connection_t *conn); void connection_dir_about_to_close(dir_connection_t *dir_conn); void directory_initiate_command(const char *address, const tor_addr_t *addr, uint16_t or_port, uint16_t dir_port, - int supports_conditional_consensus, - int supports_begindir, const char *digest, + const char *digest, uint8_t dir_purpose, uint8_t router_purpose, - int anonymized_connection, + dir_indirection_t indirection, const char *resource, const char *payload, size_t payload_len, time_t if_modified_since); diff --git a/src/or/dirserv.c b/src/or/dirserv.c index f1c9c6232..9ca4c54f7 100644 --- a/src/or/dirserv.c +++ b/src/or/dirserv.c @@ -7,6 +7,7 @@ #include "or.h" #include "buffers.h" #include "config.h" +#include "confparse.h" #include "connection.h" #include "connection_or.h" #include "control.h" @@ -62,6 +63,16 @@ static cached_dir_t *the_directory = NULL; /** For authoritative directories: the current (v1) network status. */ static cached_dir_t the_runningrouters; +/** Array of start and end of consensus methods used for supported + microdescriptor formats. */ +static const struct consensus_method_range_t { + int low; + int high; +} microdesc_consensus_methods[] = { + {MIN_METHOD_FOR_MICRODESC, MIN_METHOD_FOR_A_LINES - 1}, + {MIN_METHOD_FOR_A_LINES, MAX_SUPPORTED_CONSENSUS_METHOD}, + {-1, -1}}; + static void directory_remove_invalid(void); static cached_dir_t *dirserv_regenerate_directory(void); static char *format_versions_list(config_line_t *ln); @@ -388,18 +399,18 @@ dirserv_get_status_impl(const char *id_digest, const char *nickname, strmap_size(fingerprint_list->fp_by_name), digestmap_size(fingerprint_list->status_by_digest)); - /* Versions before Tor 0.2.1.30 have known security issues that + /* Versions before Tor 0.2.2.35 have known security issues that * make them unsuitable for the current network. */ - if (platform && !tor_version_as_new_as(platform,"0.2.1.30")) { + if (platform && !tor_version_as_new_as(platform,"0.2.2.35")) { if (msg) - *msg = "Tor version is insecure. Please upgrade!"; + *msg = "Tor version is insecure or unsupported. Please upgrade!"; return FP_REJECT; - } else if (platform && tor_version_as_new_as(platform,"0.2.2.1-alpha")) { - /* Versions from 0.2.2.1-alpha...0.2.2.20-alpha have known security + } else if (platform && tor_version_as_new_as(platform,"0.2.3.0-alpha")) { + /* Versions from 0.2.3-alpha...0.2.3.9-alpha have known security * issues that make them unusable for the current network */ - if (!tor_version_as_new_as(platform, "0.2.2.21-alpha")) { + if (!tor_version_as_new_as(platform, "0.2.3.10-alpha")) { if (msg) - *msg = "Tor version is insecure. Please upgrade!"; + *msg = "Tor version is insecure or unsupported. Please upgrade!"; return FP_REJECT; } } @@ -717,7 +728,7 @@ dirserv_add_descriptor(routerinfo_t *ri, const char **msg, const char *source) "MAX_DESCRIPTOR_UPLOAD_SIZE (%d) constant is too low.", ri->nickname, source, (int)ri->cache_info.signed_descriptor_len, MAX_DESCRIPTOR_UPLOAD_SIZE); - *msg = "Router descriptor was too large"; + *msg = "Router descriptor was too large."; control_event_or_authdir_new_descriptor("REJECTED", ri->cache_info.signed_descriptor_body, ri->cache_info.signed_descriptor_len, *msg); @@ -980,6 +991,7 @@ dirserv_set_router_is_running(routerinfo_t *router, time_t now) unreachable. */ int answer; + const or_options_t *options = get_options(); node_t *node = node_get_mutable_by_id(router->cache_info.identity_digest); tor_assert(node); @@ -988,17 +1000,27 @@ dirserv_set_router_is_running(routerinfo_t *router, time_t now) answer = ! we_are_hibernating(); } else if (router->is_hibernating && (router->cache_info.published_on + - HIBERNATION_PUBLICATION_SKEW) > router->last_reachable) { + HIBERNATION_PUBLICATION_SKEW) > node->last_reachable) { /* A hibernating router is down unless we (somehow) had contact with it * since it declared itself to be hibernating. */ answer = 0; - } else if (get_options()->AssumeReachable) { + } else if (options->AssumeReachable) { /* If AssumeReachable, everybody is up unless they say they are down! */ answer = 1; } else { - /* Otherwise, a router counts as up if we found it reachable in the last - REACHABLE_TIMEOUT seconds. */ - answer = (now < router->last_reachable + REACHABLE_TIMEOUT); + /* Otherwise, a router counts as up if we found all announced OR + ports reachable in the last REACHABLE_TIMEOUT seconds. + + XXX prop186 For now there's always one IPv4 and at most one + IPv6 OR port. + + If we're not on IPv6, don't consider reachability of potential + IPv6 OR port since that'd kill all dual stack relays until a + majority of the dir auths have IPv6 connectivity. */ + answer = (now < node->last_reachable + REACHABLE_TIMEOUT && + (options->AuthDirHasIPv6Connectivity != 1 || + tor_addr_is_null(&router->ipv6_addr) || + now < node->last_reachable6 + REACHABLE_TIMEOUT)); } if (!answer && running_long_enough_to_decide_unreachable()) { @@ -1008,11 +1030,13 @@ dirserv_set_router_is_running(routerinfo_t *router, time_t now) REACHABILITY_TEST_CYCLE_PERIOD seconds, then the router has probably been down since at least that time after we last successfully reached it. + + XXX ipv6 */ time_t when = now; - if (router->last_reachable && - router->last_reachable + REACHABILITY_TEST_CYCLE_PERIOD < now) - when = router->last_reachable + REACHABILITY_TEST_CYCLE_PERIOD; + if (node->last_reachable && + node->last_reachable + REACHABILITY_TEST_CYCLE_PERIOD < now) + when = node->last_reachable + REACHABILITY_TEST_CYCLE_PERIOD; rep_hist_note_router_unreachable(router->cache_info.identity_digest, when); } @@ -2040,7 +2064,7 @@ version_from_platform(const char *platform) * non-NULL, add a "v" line for the platform. Return 0 on success, -1 on * failure. * - * The format argument has three possible values: + * The format argument has one of the following values: * NS_V2 - Output an entry suitable for a V2 NS opinion document * NS_V3_CONSENSUS - Output the first portion of a V3 NS consensus entry * NS_V3_CONSENSUS_MICRODESC - Output the first portion of a V3 microdesc @@ -2079,15 +2103,34 @@ routerstatus_format_entry(char *buf, size_t buf_len, log_warn(LD_BUG, "Not enough space in buffer."); return -1; } + cp = buf + strlen(buf); /* TODO: Maybe we want to pass in what we need to build the rest of * this here, instead of in the caller. Then we could use the * networkstatus_type_t values, with an additional control port value * added -MP */ - if (format == NS_V3_CONSENSUS || format == NS_V3_CONSENSUS_MICRODESC) + + /* V3 microdesc consensuses don't have "a" lines. */ + if (format == NS_V3_CONSENSUS_MICRODESC) + return 0; + + /* Possible "a" line. At most one for now. */ + if (!tor_addr_is_null(&rs->ipv6_addr)) { + const char *addr_str = fmt_and_decorate_addr(&rs->ipv6_addr); + r = tor_snprintf(cp, buf_len - (cp-buf), + "a %s:%d\n", + addr_str, + (int)rs->ipv6_orport); + if (r<0) { + log_warn(LD_BUG, "Not enough space in buffer."); + return -1; + } + cp += strlen(cp); + } + + if (format == NS_V3_CONSENSUS) return 0; - cp = buf + strlen(buf); /* NOTE: Whenever this list expands, be sure to increase MAX_FLAG_LINE_LEN*/ r = tor_snprintf(cp, buf_len - (cp-buf), "s%s%s%s%s%s%s%s%s%s%s%s%s%s\n", @@ -2114,7 +2157,7 @@ routerstatus_format_entry(char *buf, size_t buf_len, /* length of "opt v \n" */ #define V_LINE_OVERHEAD 7 if (version && strlen(version) < MAX_V_LINE_LEN - V_LINE_OVERHEAD) { - if (tor_snprintf(cp, buf_len - (cp-buf), "opt v %s\n", version)<0) { + if (tor_snprintf(cp, buf_len - (cp-buf), "v %s\n", version)<0) { log_warn(LD_BUG, "Unable to print router version."); return -1; } @@ -2393,8 +2436,6 @@ set_routerstatus_from_routerinfo(routerstatus_t *rs, int listbaddirs, int vote_on_hsdirs) { const or_options_t *options = get_options(); - int unstable_version = - !tor_version_as_new_as(ri->platform,"0.1.1.16-rc-cvs"); uint32_t routerbw = router_get_advertised_bandwidth(ri); memset(rs, 0, sizeof(routerstatus_t)); @@ -2406,8 +2447,7 @@ set_routerstatus_from_routerinfo(routerstatus_t *rs, rs->is_exit = node->is_exit; rs->is_stable = node->is_stable = router_is_active(ri, node, now) && - !dirserv_thinks_router_is_unreliable(now, ri, 1, 0) && - !unstable_version; + !dirserv_thinks_router_is_unreliable(now, ri, 1, 0); rs->is_fast = node->is_fast = router_is_active(ri, node, now) && !dirserv_thinks_router_is_unreliable(now, ri, 0, 1); @@ -2453,6 +2493,14 @@ set_routerstatus_from_routerinfo(routerstatus_t *rs, strlcpy(rs->nickname, ri->nickname, sizeof(rs->nickname)); rs->or_port = ri->or_port; rs->dir_port = ri->dir_port; + if (options->AuthDirHasIPv6Connectivity == 1 && + !tor_addr_is_null(&ri->ipv6_addr) && + node->last_reachable6 >= now - REACHABLE_TIMEOUT) { + /* We're configured as having IPv6 connectivity. There's an IPv6 + OR port and it's reachable so copy it to the routerstatus. */ + tor_addr_copy(&rs->ipv6_addr, &ri->ipv6_addr); + rs->ipv6_orport = ri->ipv6_orport; + } } /** Routerstatus <b>rs</b> is part of a group of routers that are on @@ -2715,6 +2763,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_t *private_key, microdescriptors = smartlist_new(); SMARTLIST_FOREACH_BEGIN(routers, routerinfo_t *, ri) { + const struct consensus_method_range_t *cmr = NULL; if (ri->cache_info.published_on >= cutoff) { routerstatus_t *rs; vote_routerstatus_t *vrs; @@ -2736,17 +2785,22 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_t *private_key, rs->is_flagged_running = 0; vrs->version = version_from_platform(ri->platform); - md = dirvote_create_microdescriptor(ri); - if (md) { - char buf[128]; - vote_microdesc_hash_t *h; - dirvote_format_microdesc_vote_line(buf, sizeof(buf), md); - h = tor_malloc(sizeof(vote_microdesc_hash_t)); - h->microdesc_hash_line = tor_strdup(buf); - h->next = NULL; - vrs->microdesc = h; - md->last_listed = now; - smartlist_add(microdescriptors, md); + for (cmr = microdesc_consensus_methods; + cmr->low != -1 && cmr->high != -1; + cmr++) { + md = dirvote_create_microdescriptor(ri, cmr->low); + if (md) { + char buf[128]; + vote_microdesc_hash_t *h; + dirvote_format_microdesc_vote_line(buf, sizeof(buf), md, + cmr->low, cmr->high); + h = tor_malloc_zero(sizeof(vote_microdesc_hash_t)); + h->microdesc_hash_line = tor_strdup(buf); + h->next = vrs->microdesc; + vrs->microdesc = h; + md->last_listed = now; + smartlist_add(microdescriptors, md); + } } smartlist_add(routerstatuses, vrs); @@ -3273,36 +3327,42 @@ dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, * Inform the reachability checker that we could get to this guy. */ void -dirserv_orconn_tls_done(const char *address, +dirserv_orconn_tls_done(const tor_addr_t *addr, uint16_t or_port, const char *digest_rcvd) { - routerinfo_t *ri; + node_t *node = NULL; + tor_addr_port_t orport; + routerinfo_t *ri = NULL; time_t now = time(NULL); - tor_assert(address); + tor_assert(addr); tor_assert(digest_rcvd); - ri = router_get_mutable_by_digest(digest_rcvd); - - if (ri == NULL) + node = node_get_mutable_by_id(digest_rcvd); + if (node == NULL || node->ri == NULL) return; + ri = node->ri; - if (!strcasecmp(address, ri->address) && or_port == ri->or_port) { + tor_addr_copy(&orport.addr, addr); + orport.port = or_port; + if (router_has_orport(ri, &orport)) { /* Found the right router. */ if (!authdir_mode_bridge(get_options()) || ri->purpose == ROUTER_PURPOSE_BRIDGE) { + char addrstr[TOR_ADDR_BUF_LEN]; /* This is a bridge or we're not a bridge authorititative -- mark it as reachable. */ - tor_addr_t addr, *addrp=NULL; log_info(LD_DIRSERV, "Found router %s to be reachable at %s:%d. Yay.", router_describe(ri), - address, ri->or_port); - if (tor_addr_parse(&addr, ri->address) != -1) - addrp = &addr; - else - log_warn(LD_BUG, "Couldn't parse IP address \"%s\"", ri->address); - rep_hist_note_router_reachable(digest_rcvd, addrp, or_port, now); - ri->last_reachable = now; + tor_addr_to_str(addrstr, addr, sizeof(addrstr), 1), + ri->or_port); + if (tor_addr_family(addr) == AF_INET) { + rep_hist_note_router_reachable(digest_rcvd, addr, or_port, now); + node->last_reachable = now; + } else if (tor_addr_family(addr) == AF_INET6) { + /* No rephist for IPv6. */ + node->last_reachable6 = now; + } } } } @@ -3325,7 +3385,7 @@ dirserv_should_launch_reachability_test(const routerinfo_t *ri, /* It just came out of hibernation; launch a reachability test */ return 1; } - if (! routers_have_same_or_addr(ri, ri_old)) { + if (! routers_have_same_or_addrs(ri, ri_old)) { /* Address or port changed; launch a reachability test */ return 1; } @@ -3338,15 +3398,35 @@ dirserv_should_launch_reachability_test(const routerinfo_t *ri, void dirserv_single_reachability_test(time_t now, routerinfo_t *router) { + node_t *node = NULL; tor_addr_t router_addr; + + tor_assert(router); + node = node_get_mutable_by_id(router->cache_info.identity_digest); + tor_assert(node); + + /* IPv4. */ log_debug(LD_OR,"Testing reachability of %s at %s:%u.", router->nickname, router->address, router->or_port); /* Remember when we started trying to determine reachability */ - if (!router->testing_since) - router->testing_since = now; + if (!node->testing_since) + node->testing_since = now; tor_addr_from_ipv4h(&router_addr, router->addr); connection_or_connect(&router_addr, router->or_port, router->cache_info.identity_digest); + + /* Possible IPv6. */ + if (!tor_addr_is_null(&router->ipv6_addr)) { + char addrstr[TOR_ADDR_BUF_LEN]; + log_debug(LD_OR, "Testing reachability of %s at %s:%u.", + router->nickname, + tor_addr_to_str(addrstr, &router->ipv6_addr, sizeof(addrstr), 1), + router->ipv6_orport); + if (!node->testing_since6) + node->testing_since6 = now; + connection_or_connect(&router->ipv6_addr, router->ipv6_orport, + router->cache_info.identity_digest); + } } /** Auth dir server only: load balance such that we only diff --git a/src/or/dirserv.h b/src/or/dirserv.h index 22269b200..8508c938a 100644 --- a/src/or/dirserv.h +++ b/src/or/dirserv.h @@ -107,7 +107,7 @@ int dirserv_get_routerdesc_fingerprints(smartlist_t *fps_out, const char *key, int is_extrainfo); int dirserv_get_routerdescs(smartlist_t *descs_out, const char *key, const char **msg); -void dirserv_orconn_tls_done(const char *address, +void dirserv_orconn_tls_done(const tor_addr_t *addr, uint16_t or_port, const char *digest_rcvd); int dirserv_should_launch_reachability_test(const routerinfo_t *ri, diff --git a/src/or/dirvote.c b/src/or/dirvote.c index 144859ae0..b1b885cf3 100644 --- a/src/or/dirvote.c +++ b/src/or/dirvote.c @@ -53,29 +53,6 @@ static int dirvote_compute_consensuses(void); static int dirvote_publish_consensus(void); static char *make_consensus_method_list(int low, int high, const char *sep); -/** The highest consensus method that we currently support. */ -#define MAX_SUPPORTED_CONSENSUS_METHOD 13 - -/** Lowest consensus method that contains a 'directory-footer' marker */ -#define MIN_METHOD_FOR_FOOTER 9 - -/** Lowest consensus method that contains bandwidth weights */ -#define MIN_METHOD_FOR_BW_WEIGHTS 9 - -/** Lowest consensus method that contains consensus params */ -#define MIN_METHOD_FOR_PARAMS 7 - -/** Lowest consensus method that generates microdescriptors */ -#define MIN_METHOD_FOR_MICRODESC 8 - -/** Lowest consensus method that ensures a majority of authorities voted - * for a param. */ -#define MIN_METHOD_FOR_MAJORITY_PARAMS 12 - -/** Lowest consensus method where microdesc consensuses omit any entry - * with no microdesc. */ -#define MIN_METHOD_FOR_MANDATORY_MICRODESC 13 - /* ===== * Voting * =====*/ @@ -430,6 +407,21 @@ _compare_vote_rs(const void **_a, const void **_b) return compare_vote_rs(a,b); } +/** Helper for sorting OR ports. */ +static int +_compare_orports(const void **_a, const void **_b) +{ + const tor_addr_port_t *a = *_a, *b = *_b; + int r; + + if ((r = tor_addr_compare(&a->addr, &b->addr, CMP_EXACT))) + return r; + if ((r = (((int) b->port) - ((int) a->port)))) + return r; + + return 0; +} + /** Given a list of vote_routerstatus_t, all for the same router identity, * return whichever is most frequent, breaking ties in favor of more * recently published vote_routerstatus_t and in case of ties there, @@ -437,7 +429,8 @@ _compare_vote_rs(const void **_a, const void **_b) */ static vote_routerstatus_t * compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, - char *microdesc_digest256_out) + char *microdesc_digest256_out, + tor_addr_port_t *best_alt_orport_out) { vote_routerstatus_t *most = NULL, *cur = NULL; int most_n = 0, cur_n = 0; @@ -473,6 +466,39 @@ compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, tor_assert(most); + /* If we're producing "a" lines, vote on potential alternative (sets + * of) OR port(s) in the winning routerstatuses. + * + * XXX prop186 There's at most one alternative OR port (_the_ IPv6 + * port) for now. */ + if (consensus_method >= MIN_METHOD_FOR_A_LINES && best_alt_orport_out) { + smartlist_t *alt_orports = smartlist_new(); + const tor_addr_port_t *most_alt_orport = NULL; + + SMARTLIST_FOREACH_BEGIN(votes, vote_routerstatus_t *, rs) { + if (compare_vote_rs(most, rs) == 0 && + !tor_addr_is_null(&rs->status.ipv6_addr) + && rs->status.ipv6_orport) { + smartlist_add(alt_orports, tor_addr_port_new(&rs->status.ipv6_addr, + rs->status.ipv6_orport)); + } + } SMARTLIST_FOREACH_END(rs); + + smartlist_sort(alt_orports, _compare_orports); + most_alt_orport = smartlist_get_most_frequent(alt_orports, + _compare_orports); + if (most_alt_orport) { + memcpy(best_alt_orport_out, most_alt_orport, sizeof(tor_addr_port_t)); + log_debug(LD_DIR, "\"a\" line winner for %s is %s:%d", + most->status.nickname, + fmt_and_decorate_addr(&most_alt_orport->addr), + most_alt_orport->port); + } + + SMARTLIST_FOREACH(alt_orports, tor_addr_port_t *, ap, tor_free(ap)); + smartlist_free(alt_orports); + } + if (consensus_method >= MIN_METHOD_FOR_MICRODESC && microdesc_digest256_out) { smartlist_t *digests = smartlist_new(); @@ -1685,6 +1711,7 @@ networkstatus_compute_consensus(smartlist_t *votes, int n_listing = 0; int i; char microdesc_digest[DIGEST256_LEN]; + tor_addr_port_t alt_orport = {TOR_ADDR_NULL, 0}; /* Of the next-to-be-considered digest in each voter, which is first? */ SMARTLIST_FOREACH(votes, networkstatus_t *, v, { @@ -1754,7 +1781,7 @@ networkstatus_compute_consensus(smartlist_t *votes, * routerinfo and its contents are. */ memset(microdesc_digest, 0, sizeof(microdesc_digest)); rs = compute_routerstatus_consensus(matching_descs, consensus_method, - microdesc_digest); + microdesc_digest, &alt_orport); /* Copy bits of that into rs_out. */ memset(&rs_out, 0, sizeof(rs_out)); tor_assert(fast_memeq(lowest_id, rs->status.identity_digest,DIGEST_LEN)); @@ -1765,6 +1792,10 @@ networkstatus_compute_consensus(smartlist_t *votes, rs_out.published_on = rs->status.published_on; rs_out.dir_port = rs->status.dir_port; rs_out.or_port = rs->status.or_port; + if (consensus_method >= MIN_METHOD_FOR_A_LINES) { + tor_addr_copy(&rs_out.ipv6_addr, &alt_orport.addr); + rs_out.ipv6_orport = alt_orport.port; + } rs_out.has_bandwidth = 0; rs_out.has_exitsummary = 0; @@ -3512,7 +3543,7 @@ dirvote_get_vote(const char *fp, int flags) * particular method. **/ microdesc_t * -dirvote_create_microdescriptor(const routerinfo_t *ri) +dirvote_create_microdescriptor(const routerinfo_t *ri, int consensus_method) { microdesc_t *result = NULL; char *key = NULL, *summary = NULL, *family = NULL; @@ -3528,6 +3559,12 @@ dirvote_create_microdescriptor(const routerinfo_t *ri) smartlist_add_asprintf(chunks, "onion-key\n%s", key); + if (consensus_method >= MIN_METHOD_FOR_A_LINES && + !tor_addr_is_null(&ri->ipv6_addr) && ri->ipv6_orport) + smartlist_add_asprintf(chunks, "a %s:%d\n", + fmt_and_decorate_addr(&ri->ipv6_addr), + ri->ipv6_orport); + if (family) smartlist_add_asprintf(chunks, "family %s\n", family); @@ -3561,33 +3598,36 @@ dirvote_create_microdescriptor(const routerinfo_t *ri) return result; } -/** Cached space-separated string to hold */ -static char *microdesc_consensus_methods = NULL; - /** Format the appropriate vote line to describe the microdescriptor <b>md</b> * in a consensus vote document. Write it into the <b>out_len</b>-byte buffer * in <b>out</b>. Return -1 on failure and the number of characters written * on success. */ ssize_t -dirvote_format_microdesc_vote_line(char *out, size_t out_len, - const microdesc_t *md) +dirvote_format_microdesc_vote_line(char *out_buf, size_t out_buf_len, + const microdesc_t *md, + int consensus_method_low, + int consensus_method_high) { + ssize_t ret = -1; char d64[BASE64_DIGEST256_LEN+1]; - if (!microdesc_consensus_methods) { - microdesc_consensus_methods = - make_consensus_method_list(MIN_METHOD_FOR_MICRODESC, - MAX_SUPPORTED_CONSENSUS_METHOD, - ","); - tor_assert(microdesc_consensus_methods); - } + char *microdesc_consensus_methods = + make_consensus_method_list(consensus_method_low, + consensus_method_high, + ","); + tor_assert(microdesc_consensus_methods); + if (digest256_to_base64(d64, md->digest)<0) - return -1; + goto out; - if (tor_snprintf(out, out_len, "m %s sha256=%s\n", + if (tor_snprintf(out_buf, out_buf_len, "m %s sha256=%s\n", microdesc_consensus_methods, d64)<0) - return -1; + goto out; - return strlen(out); + ret = strlen(out_buf); + + out: + tor_free(microdesc_consensus_methods); + return ret; } /** If <b>vrs</b> has a hash made for the consensus method <b>method</b> with diff --git a/src/or/dirvote.h b/src/or/dirvote.h index e6f970061..621121864 100644 --- a/src/or/dirvote.h +++ b/src/or/dirvote.h @@ -19,6 +19,32 @@ /** Smallest allowable voting interval. */ #define MIN_VOTE_INTERVAL 300 +/** The highest consensus method that we currently support. */ +#define MAX_SUPPORTED_CONSENSUS_METHOD 14 + +/** Lowest consensus method that contains a 'directory-footer' marker */ +#define MIN_METHOD_FOR_FOOTER 9 + +/** Lowest consensus method that contains bandwidth weights */ +#define MIN_METHOD_FOR_BW_WEIGHTS 9 + +/** Lowest consensus method that contains consensus params */ +#define MIN_METHOD_FOR_PARAMS 7 + +/** Lowest consensus method that generates microdescriptors */ +#define MIN_METHOD_FOR_MICRODESC 8 + +/** Lowest consensus method that ensures a majority of authorities voted + * for a param. */ +#define MIN_METHOD_FOR_MAJORITY_PARAMS 12 + +/** Lowest consensus method where microdesc consensuses omit any entry + * with no microdesc. */ +#define MIN_METHOD_FOR_MANDATORY_MICRODESC 13 + +/** Lowest consensus method that contains "a" lines. */ +#define MIN_METHOD_FOR_A_LINES 14 + void dirvote_free_all(void); /* vote manipulation */ @@ -70,10 +96,12 @@ networkstatus_t * dirserv_generate_networkstatus_vote_obj(crypto_pk_t *private_key, authority_cert_t *cert); -microdesc_t *dirvote_create_microdescriptor(const routerinfo_t *ri); +microdesc_t *dirvote_create_microdescriptor(const routerinfo_t *ri, + int consensus_method); ssize_t dirvote_format_microdesc_vote_line(char *out, size_t out_len, - const microdesc_t *md); - + const microdesc_t *md, + int consensus_method_low, + int consensus_method_high); int vote_routerstatus_find_microdesc_hash(char *digest256_out, const vote_routerstatus_t *vrs, int method, diff --git a/src/or/hibernate.c b/src/or/hibernate.c index 3a9c1e422..b33e5e216 100644 --- a/src/or/hibernate.c +++ b/src/or/hibernate.c @@ -29,6 +29,7 @@ hibernating, phase 2: #include "hibernate.h" #include "main.h" #include "router.h" +#include "statefile.h" extern long stats_n_seconds_working; /* published uptime */ diff --git a/src/or/include.am b/src/or/include.am new file mode 100644 index 000000000..e9811ec96 --- /dev/null +++ b/src/or/include.am @@ -0,0 +1,167 @@ +bin_PROGRAMS+= src/or/tor +noinst_LIBRARIES+= src/or/libtor.a + +if BUILD_NT_SERVICES +tor_platform_source=src/or/ntmain.c +else +tor_platform_source= +endif + +EXTRA_DIST+= src/or/ntmain.c src/or/or_sha1.i src/or/Makefile.nmake + +if USE_EXTERNAL_EVDNS +evdns_source= +else +evdns_source=src/or/eventdns.c +endif + +src_or_libtor_a_SOURCES = \ + src/or/buffers.c \ + src/or/circuitbuild.c \ + src/or/circuitlist.c \ + src/or/circuituse.c \ + src/or/command.c \ + src/or/config.c \ + src/or/confparse.c \ + src/or/connection.c \ + src/or/connection_edge.c \ + src/or/connection_or.c \ + src/or/control.c \ + src/or/cpuworker.c \ + src/or/directory.c \ + src/or/dirserv.c \ + src/or/dirvote.c \ + src/or/dns.c \ + src/or/dnsserv.c \ + src/or/geoip.c \ + src/or/hibernate.c \ + src/or/main.c \ + src/or/microdesc.c \ + src/or/networkstatus.c \ + src/or/nodelist.c \ + src/or/onion.c \ + src/or/transports.c \ + src/or/policies.c \ + src/or/reasons.c \ + src/or/relay.c \ + src/or/rendclient.c \ + src/or/rendcommon.c \ + src/or/rendmid.c \ + src/or/rendservice.c \ + src/or/rephist.c \ + src/or/replaycache.c \ + src/or/router.c \ + src/or/routerlist.c \ + src/or/routerparse.c \ + src/or/routerset.c \ + src/or/statefile.c \ + src/or/status.c \ + $(evdns_source) \ + $(tor_platform_source) \ + src/or/config_codedigest.c + +#libtor_a_LIBADD = ../common/libor.a ../common/libor-crypto.a \ +# ../common/libor-event.a + + +src_or_tor_SOURCES = src/or/tor_main.c +AM_CPPFLAGS += -I$(srcdir)/src/or -Isrc/or + +src/or/tor_main.o: micro-revision.i + +AM_CPPFLAGS += -DSHARE_DATADIR="\"$(datadir)\"" \ + -DLOCALSTATEDIR="\"$(localstatedir)\"" \ + -DBINDIR="\"$(bindir)\"" + +# -L flags need to go in LDFLAGS. -l flags need to go in LDADD. +# This seems to matter nowhere but on windows, but I assure you that it +# matters a lot there, and is quite hard to debug if you forget to do it. + + +src_or_tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@ +src_or_tor_LDADD = src/or/libtor.a src/common/libor.a src/common/libor-crypto.a \ + src/common/libor-event.a \ + @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ @TOR_OPENSSL_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +ORHEADERS = \ + src/or/buffers.h \ + src/or/circuitbuild.h \ + src/or/circuitlist.h \ + src/or/circuituse.h \ + src/or/command.h \ + src/or/config.h \ + src/or/confparse.h \ + src/or/connection.h \ + src/or/connection_edge.h \ + src/or/connection_or.h \ + src/or/control.h \ + src/or/cpuworker.h \ + src/or/directory.h \ + src/or/dirserv.h \ + src/or/dirvote.h \ + src/or/dns.h \ + src/or/dnsserv.h \ + src/or/eventdns.h \ + src/or/eventdns_tor.h \ + src/or/geoip.h \ + src/or/hibernate.h \ + src/or/main.h \ + src/or/microdesc.h \ + src/or/networkstatus.h \ + src/or/nodelist.h \ + src/or/ntmain.h \ + src/or/onion.h \ + src/or/or.h \ + src/or/transports.h \ + src/or/policies.h \ + src/or/reasons.h \ + src/or/relay.h \ + src/or/rendclient.h \ + src/or/rendcommon.h \ + src/or/rendmid.h \ + src/or/rendservice.h \ + src/or/rephist.h \ + src/or/replaycache.h \ + src/or/router.h \ + src/or/routerlist.h \ + src/or/routerset.h \ + src/or/routerparse.h \ + src/or/statefile.h \ + src/or/status.h + +noinst_HEADERS+= $(ORHEADERS) micro-revision.i + +src/or/config_codedigest.o: src/or/or_sha1.i + +micro-revision.i: FORCE + @rm -f micro-revision.tmp; \ + if test -d "$(top_srcdir)/.git" && \ + test -x "`which git 2>&1;true`"; then \ + HASH="`cd "$(top_srcdir)" && git rev-parse --short=16 HEAD`"; \ + echo \"$$HASH\" > micro-revision.tmp; \ + fi; \ + if test ! -f micro-revision.tmp ; then \ + if test ! -f micro-revision.i ; then \ + echo '""' > micro-revision.i; \ + fi; \ + elif test ! -f micro-revision.i || \ + test x"`cat micro-revision.tmp`" != x"`cat micro-revision.i`"; then \ + mv micro-revision.tmp micro-revision.i; \ + fi; true + +src/or/or_sha1.i: $(src_or_tor_SOURCES) $(src_or_libtor_a_SOURCES) $(ORHEADERS) + $(AM_V_GEN)if test "@SHA1SUM@" != none; then \ + (cd "$(srcdir)" && "@SHA1SUM@" $(src_or_tor_SOURCES) $(src_or_libtor_a_SOURCES) $(ORHEADERS) ) | \ + "@SED@" -n 's/^\(.*\)$$/"\1\\n"/p' > src/or/or_sha1.i; \ + elif test "@OPENSSL@" != none; then \ + (cd "$(srcdir)" && "@OPENSSL@" sha1 $(src_or_tor_SOURCES) $(src_or_libtor_a_SOURCES) $(ORHEADERS)) | \ + "@SED@" -n 's/SHA1(\(.*\))= \(.*\)/"\2 \1\\n"/p' > src/or/or_sha1.i; \ + else \ + rm src/or/or_sha1.i; \ + touch src/or/or_sha1.i; \ + fi + +CLEANFILES+= micro-revision.i + +FORCE: diff --git a/src/or/main.c b/src/or/main.c index 20a1e086a..635dcb4bd 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -46,6 +46,7 @@ #include "router.h" #include "routerlist.h" #include "routerparse.h" +#include "statefile.h" #include "status.h" #ifdef USE_DMALLOC #include <dmalloc.h> @@ -953,7 +954,8 @@ directory_info_has_arrived(time_t now, int from_cache) const or_options_t *options = get_options(); if (!router_have_minimum_dir_info()) { - int quiet = directory_too_idle_to_fetch_descriptors(options, now); + int quiet = from_cache || + directory_too_idle_to_fetch_descriptors(options, now); log(quiet ? LOG_INFO : LOG_NOTICE, LD_DIR, "I learned some more directory information, but not enough to " "build a circuit: %s", get_dir_info_status_string()); @@ -1546,11 +1548,15 @@ run_scheduled_events(time_t now) options->PortForwarding && is_server) { #define PORT_FORWARDING_CHECK_INTERVAL 5 - /* XXXXX this should take a list of ports, not just two! */ - tor_check_port_forwarding(options->PortForwardingHelper, - get_primary_dir_port(), - get_primary_or_port(), - now); + smartlist_t *ports_to_forward = get_list_of_ports_to_forward(); + if (ports_to_forward) { + tor_check_port_forwarding(options->PortForwardingHelper, + ports_to_forward, + now); + + SMARTLIST_FOREACH(ports_to_forward, char *, cp, tor_free(cp)); + smartlist_free(ports_to_forward); + } time_to_check_port_forwarding = now+PORT_FORWARDING_CHECK_INTERVAL; } @@ -1561,7 +1567,8 @@ run_scheduled_events(time_t now) /** 12. write the heartbeat message */ if (options->HeartbeatPeriod && time_to_next_heartbeat <= now) { - log_heartbeat(now); + if (time_to_next_heartbeat) /* don't log the first heartbeat */ + log_heartbeat(now); time_to_next_heartbeat = now+options->HeartbeatPeriod; } } @@ -2302,12 +2309,17 @@ tor_init(int argc, char *argv[]) { const char *version = get_version(); + const char *bev_str = #ifdef USE_BUFFEREVENTS - log_notice(LD_GENERAL, "Tor v%s (with bufferevents) running on %s.", - version, get_uname()); + "(with bufferevents) "; #else - log_notice(LD_GENERAL, "Tor v%s running on %s.", version, get_uname()); + ""; #endif + log_notice(LD_GENERAL, "Tor v%s %srunning on %s with Libevent %s " + "and OpenSSL %s.", version, bev_str, + get_uname(), + tor_libevent_get_version_str(), + crypto_openssl_get_version_str()); log_notice(LD_GENERAL, "Tor can't help you if you use it wrong! " "Learn how to be safe at " @@ -2448,6 +2460,7 @@ tor_free_all(int postfork) microdesc_free_all(); if (!postfork) { config_free_all(); + or_state_free_all(); router_free_all(); policies_free_all(); } diff --git a/src/or/microdesc.c b/src/or/microdesc.c index b4d22c1c6..3bda9cbfa 100644 --- a/src/or/microdesc.c +++ b/src/or/microdesc.c @@ -323,8 +323,8 @@ microdesc_cache_reload(microdesc_cache_t *cache) } tor_free(journal_content); } - log_notice(LD_DIR, "Reloaded microdescriptor cache. Found %d descriptors.", - total); + log_info(LD_DIR, "Reloaded microdescriptor cache. Found %d descriptors.", + total); microdesc_cache_rebuild(cache, 0 /* don't force */); diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index fadaf90da..0cc6a2108 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -1168,7 +1168,7 @@ update_v2_networkstatus_cache_downloads(time_t now) directory_initiate_command_routerstatus( &ds->fake_status, DIR_PURPOSE_FETCH_V2_NETWORKSTATUS, ROUTER_PURPOSE_GENERAL, - 0, /* Not private */ + DIRIND_ONEHOP, resource, NULL, 0 /* No payload. */, 0 /* No I-M-S. */); @@ -1530,13 +1530,7 @@ routerstatus_has_changed(const routerstatus_t *a, const routerstatus_t *b) a->is_bad_exit != b->is_bad_exit || a->is_bad_directory != b->is_bad_directory || a->is_hs_dir != b->is_hs_dir || - a->version_known != b->version_known || - a->version_supports_begindir != b->version_supports_begindir || - a->version_supports_extrainfo_upload != - b->version_supports_extrainfo_upload || - a->version_supports_conditional_consensus != - b->version_supports_conditional_consensus || - a->version_supports_v3_dir != b->version_supports_v3_dir; + a->version_known != b->version_known; } /** Notify controllers of any router status entries that changed between @@ -2310,6 +2304,30 @@ networkstatus_parse_flavor_name(const char *flavname) return -1; } +/** Return 0 if this routerstatus is obsolete, too new, isn't + * running, or otherwise not a descriptor that we would make any + * use of even if we had it. Else return 1. */ +int +client_would_use_router(const routerstatus_t *rs, time_t now, + const or_options_t *options) +{ + if (!rs->is_flagged_running && !options->FetchUselessDescriptors) { + /* If we had this router descriptor, we wouldn't even bother using it. + * But, if we want to have a complete list, fetch it anyway. */ + return 0; + } + if (rs->published_on + options->TestingEstimatedDescriptorPropagationTime + > now) { + /* Most caches probably don't have this descriptor yet. */ + return 0; + } + if (rs->published_on + OLD_ROUTER_DESC_MAX_AGE < now) { + /* We'd drop it immediately for being too old. */ + return 0; + } + return 1; +} + /** If <b>question</b> is a string beginning with "ns/" in a format the * control interface expects for a GETINFO question, set *<b>answer</b> to a * newly-allocated string containing networkstatus lines for the appropriate diff --git a/src/or/networkstatus.h b/src/or/networkstatus.h index 0af17512d..dcd58f889 100644 --- a/src/or/networkstatus.h +++ b/src/or/networkstatus.h @@ -71,6 +71,8 @@ int should_delay_dir_fetches(const or_options_t *options); void update_networkstatus_downloads(time_t now); void update_certificate_downloads(time_t now); int consensus_is_waiting_for_certs(void); +int client_would_use_router(const routerstatus_t *rs, time_t now, + const or_options_t *options); networkstatus_v2_t *networkstatus_v2_get_by_digest(const char *digest); networkstatus_t *networkstatus_get_latest_consensus(void); networkstatus_t *networkstatus_get_latest_consensus_by_flavor( diff --git a/src/or/nodelist.c b/src/or/nodelist.c index d17850888..c357bc047 100644 --- a/src/or/nodelist.c +++ b/src/or/nodelist.c @@ -5,19 +5,26 @@ /* See LICENSE for licensing information */ #include "or.h" +#include "address.h" #include "config.h" +#include "control.h" #include "dirserv.h" +#include "geoip.h" +#include "main.h" #include "microdesc.h" #include "networkstatus.h" #include "nodelist.h" #include "policies.h" +#include "rendservice.h" #include "router.h" #include "routerlist.h" +#include "routerset.h" #include <string.h> static void nodelist_drop_node(node_t *node, int remove_from_ht); static void node_free(node_t *node); +static void update_router_have_minimum_dir_info(void); /** A nodelist_t holds a node_t object for every router we're "willing to use * for something". Specifically, it should hold a node_t for every node that @@ -115,19 +122,48 @@ node_get_or_create(const char *identity_digest) return node; } -/** Add <b>ri</b> to the nodelist. */ +/** Called when a node's address changes. */ +static void +node_addrs_changed(node_t *node) +{ + node->last_reachable = node->last_reachable6 = 0; + node->testing_since = node->testing_since6 = 0; + node->country = -1; +} + +/** Add <b>ri</b> to an appropriate node in the nodelist. If we replace an + * old routerinfo, and <b>ri_old_out</b> is not NULL, set *<b>ri_old_out</b> + * to the previous routerinfo. + */ node_t * -nodelist_add_routerinfo(routerinfo_t *ri) +nodelist_set_routerinfo(routerinfo_t *ri, routerinfo_t **ri_old_out) { node_t *node; + const char *id_digest; + int had_router = 0; + tor_assert(ri); + init_nodelist(); - node = node_get_or_create(ri->cache_info.identity_digest); + id_digest = ri->cache_info.identity_digest; + node = node_get_or_create(id_digest); + + if (node->ri) { + if (!routers_have_same_or_addrs(node->ri, ri)) { + node_addrs_changed(node); + } + had_router = 1; + if (ri_old_out) + *ri_old_out = node->ri; + } else { + if (ri_old_out) + *ri_old_out = NULL; + } node->ri = ri; if (node->country == -1) node_set_country(node); - if (authdir_mode(get_options())) { + if (authdir_mode(get_options()) && !had_router) { const char *discard=NULL; uint32_t status = dirserv_router_get_status(ri, &discard); dirserv_set_node_flags_from_authoritative_status(node, status); @@ -167,7 +203,7 @@ nodelist_add_microdesc(microdesc_t *md) return node; } -/** Tell the nodelist that the current usable consensus to <b>ns</b>. +/** Tell the nodelist that the current usable consensus is <b>ns</b>. * This makes the nodelist change all of the routerstatus entries for * the nodes, drop nodes that no longer have enough info to get used, * and grab microdescriptors into nodes as appropriate. @@ -177,6 +213,7 @@ nodelist_set_consensus(networkstatus_t *ns) { const or_options_t *options = get_options(); int authdir = authdir_mode_v2(options) || authdir_mode_v3(options); + int client = !server_mode(options); init_nodelist(); if (ns->flavor == FLAV_MICRODESC) @@ -213,6 +250,11 @@ nodelist_set_consensus(networkstatus_t *ns) node->is_bad_directory = rs->is_bad_directory; node->is_bad_exit = rs->is_bad_exit; node->is_hs_dir = rs->is_hs_dir; + node->ipv6_preferred = 0; + if (client && options->ClientPreferIPv6ORPort == 1 && + (tor_addr_is_null(&rs->ipv6_addr) == 0 || + (node->md && tor_addr_is_null(&node->md->ipv6_addr) == 0))) + node->ipv6_preferred = 1; } } SMARTLIST_FOREACH_END(rs); @@ -231,7 +273,8 @@ nodelist_set_consensus(networkstatus_t *ns) node->is_valid = node->is_running = node->is_hs_dir = node->is_fast = node->is_stable = node->is_possible_guard = node->is_exit = - node->is_bad_exit = node->is_bad_directory = 0; + node->is_bad_exit = node->is_bad_directory = + node->ipv6_preferred = 0; } } } SMARTLIST_FOREACH_END(node); @@ -682,19 +725,6 @@ node_get_all_orports(const node_t *node) return sl; } -/** Copy the primary (IPv4) OR port (IP address and TCP port) for - * <b>node</b> into *<b>ap_out</b>. */ -void -node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out) -{ - if (node->ri) { - router_get_prim_orport(node->ri, ap_out); - } else if (node->rs) { - tor_addr_from_ipv4h(&ap_out->addr, node->rs->addr); - ap_out->port = node->rs->or_port; - } -} - /** Wrapper around node_get_prim_orport for backward compatibility. */ void @@ -718,36 +748,6 @@ node_get_prim_addr_ipv4h(const node_t *node) return 0; } -/** Copy the preferred OR port (IP address and TCP port) for - * <b>node</b> into <b>ap_out</b>. */ -void -node_get_pref_orport(const node_t *node, tor_addr_port_t *ap_out) -{ - if (node->ri) { - router_get_pref_orport(node->ri, ap_out); - } else if (node->rs) { - /* No IPv6 in routerstatus_t yet. XXXprop186 ok for private - bridges but needs fixing */ - tor_addr_from_ipv4h(&ap_out->addr, node->rs->addr); - ap_out->port = node->rs->or_port; - } -} - -/** Copy the preferred IPv6 OR port (address and TCP port) for - * <b>node</b> into *<b>ap_out</b>. */ -void -node_get_pref_ipv6_orport(const node_t *node, tor_addr_port_t *ap_out) -{ - if (node->ri) { - router_get_pref_ipv6_orport(node->ri, ap_out); - } else if (node->rs) { - /* No IPv6 in routerstatus_t yet. XXXprop186 ok for private - bridges but needs fixing */ - tor_addr_make_unspec(&ap_out->addr); - ap_out->port = 0; - } -} - /** Copy a string representation of an IP address for <b>node</b> into * the <b>len</b>-byte buffer at <b>buf</b>. */ void @@ -818,3 +818,590 @@ node_get_declared_family(const node_t *node) return NULL; } +/** Return 1 if we prefer the IPv6 address and OR TCP port of + * <b>node</b>, else 0. + * + * We prefer the IPv6 address if the router has an IPv6 address and + * i) the node_t says that it prefers IPv6 + * or + * ii) the router has no IPv4 address. */ +int +node_ipv6_preferred(const node_t *node) +{ + tor_addr_port_t ipv4_addr; + node_assert_ok(node); + + if (node->ipv6_preferred || node_get_prim_orport(node, &ipv4_addr)) { + if (node->ri) + return !tor_addr_is_null(&node->ri->ipv6_addr); + if (node->md) + return !tor_addr_is_null(&node->md->ipv6_addr); + if (node->rs) + return !tor_addr_is_null(&node->rs->ipv6_addr); + } + return 0; +} + +/** Copy the primary (IPv4) OR port (IP address and TCP port) for + * <b>node</b> into *<b>ap_out</b>. Return 0 if a valid address and + * port was copied, else return non-zero.*/ +int +node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out) +{ + node_assert_ok(node); + tor_assert(ap_out); + + if (node->ri) { + if (node->ri->addr == 0 || node->ri->or_port == 0) + return -1; + tor_addr_from_ipv4h(&ap_out->addr, node->ri->addr); + ap_out->port = node->ri->or_port; + return 0; + } + if (node->rs) { + if (node->rs->addr == 0 || node->rs->or_port == 0) + return -1; + tor_addr_from_ipv4h(&ap_out->addr, node->rs->addr); + ap_out->port = node->rs->or_port; + return 0; + } + return -1; +} + +/** Copy the preferred OR port (IP address and TCP port) for + * <b>node</b> into *<b>ap_out</b>. */ +void +node_get_pref_orport(const node_t *node, tor_addr_port_t *ap_out) +{ + tor_assert(ap_out); + + /* Cheap implementation of config option ClientUseIPv6 -- simply + don't prefer IPv6 when ClientUseIPv6 is not set. (See #4455 for + more on this subject.) Note that this filter is too strict since + we're hindering not only clients! Erring on the safe side + shouldn't be a problem though. XXX move this check to where + outgoing connections are made? -LN */ + if (get_options()->ClientUseIPv6 == 1 && node_ipv6_preferred(node)) + node_get_pref_ipv6_orport(node, ap_out); + else + node_get_prim_orport(node, ap_out); +} + +/** Copy the preferred IPv6 OR port (IP address and TCP port) for + * <b>node</b> into *<b>ap_out</b>. */ +void +node_get_pref_ipv6_orport(const node_t *node, tor_addr_port_t *ap_out) +{ + node_assert_ok(node); + tor_assert(ap_out); + + /* We prefer the microdesc over a potential routerstatus here. They + are not being synchronised atm so there might be a chance that + they differ at some point, f.ex. when flipping + UseMicrodescriptors? -LN */ + + if (node->ri) { + tor_addr_copy(&ap_out->addr, &node->ri->ipv6_addr); + ap_out->port = node->ri->ipv6_orport; + } else if (node->md) { + tor_addr_copy(&ap_out->addr, &node->md->ipv6_addr); + ap_out->port = node->md->ipv6_orport; + } else if (node->rs) { + tor_addr_copy(&ap_out->addr, &node->rs->ipv6_addr); + ap_out->port = node->rs->ipv6_orport; + } +} + +/** Refresh the country code of <b>ri</b>. This function MUST be called on + * each router when the GeoIP database is reloaded, and on all new routers. */ +void +node_set_country(node_t *node) +{ + if (node->rs) + node->country = geoip_get_country_by_ip(node->rs->addr); + else if (node->ri) + node->country = geoip_get_country_by_ip(node->ri->addr); + else + node->country = -1; +} + +/** Set the country code of all routers in the routerlist. */ +void +nodelist_refresh_countries(void) +{ + smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, node_t *, node, + node_set_country(node)); +} + +/** Return true iff router1 and router2 have similar enough network addresses + * that we should treat them as being in the same family */ +static INLINE int +addrs_in_same_network_family(const tor_addr_t *a1, + const tor_addr_t *a2) +{ + return 0 == tor_addr_compare_masked(a1, a2, 16, CMP_SEMANTIC); +} + +/** Return true if <b>node</b>'s nickname matches <b>nickname</b> + * (case-insensitive), or if <b>node's</b> identity key digest + * matches a hexadecimal value stored in <b>nickname</b>. Return + * false otherwise. */ +static int +node_nickname_matches(const node_t *node, const char *nickname) +{ + const char *n = node_get_nickname(node); + if (n && nickname[0]!='$' && !strcasecmp(n, nickname)) + return 1; + return hex_digest_nickname_matches(nickname, + node->identity, + n, + node_is_named(node)); +} + +/** Return true iff <b>node</b> is named by some nickname in <b>lst</b>. */ +static INLINE int +node_in_nickname_smartlist(const smartlist_t *lst, const node_t *node) +{ + if (!lst) return 0; + SMARTLIST_FOREACH(lst, const char *, name, { + if (node_nickname_matches(node, name)) + return 1; + }); + return 0; +} + +/** Return true iff r1 and r2 are in the same family, but not the same + * router. */ +int +nodes_in_same_family(const node_t *node1, const node_t *node2) +{ + const or_options_t *options = get_options(); + + /* Are they in the same family because of their addresses? */ + if (options->EnforceDistinctSubnets) { + tor_addr_t a1, a2; + node_get_addr(node1, &a1); + node_get_addr(node2, &a2); + if (addrs_in_same_network_family(&a1, &a2)) + return 1; + } + + /* Are they in the same family because the agree they are? */ + { + const smartlist_t *f1, *f2; + f1 = node_get_declared_family(node1); + f2 = node_get_declared_family(node2); + if (f1 && f2 && + node_in_nickname_smartlist(f1, node2) && + node_in_nickname_smartlist(f2, node1)) + return 1; + } + + /* Are they in the same option because the user says they are? */ + if (options->NodeFamilySets) { + SMARTLIST_FOREACH(options->NodeFamilySets, const routerset_t *, rs, { + if (routerset_contains_node(rs, node1) && + routerset_contains_node(rs, node2)) + return 1; + }); + } + + return 0; +} + +/** + * Add all the family of <b>node</b>, including <b>node</b> itself, to + * the smartlist <b>sl</b>. + * + * This is used to make sure we don't pick siblings in a single path, or + * pick more than one relay from a family for our entry guard list. + * Note that a node may be added to <b>sl</b> more than once if it is + * part of <b>node</b>'s family for more than one reason. + */ +void +nodelist_add_node_and_family(smartlist_t *sl, const node_t *node) +{ + const smartlist_t *all_nodes = nodelist_get_list(); + const smartlist_t *declared_family; + const or_options_t *options = get_options(); + + tor_assert(node); + + declared_family = node_get_declared_family(node); + + /* Let's make sure that we have the node itself, if it's a real node. */ + { + const node_t *real_node = node_get_by_id(node->identity); + if (real_node) + smartlist_add(sl, (node_t*)real_node); + } + + /* First, add any nodes with similar network addresses. */ + if (options->EnforceDistinctSubnets) { + tor_addr_t node_addr; + node_get_addr(node, &node_addr); + + SMARTLIST_FOREACH_BEGIN(all_nodes, const node_t *, node2) { + tor_addr_t a; + node_get_addr(node2, &a); + if (addrs_in_same_network_family(&a, &node_addr)) + smartlist_add(sl, (void*)node2); + } SMARTLIST_FOREACH_END(node2); + } + + /* Now, add all nodes in the declared_family of this node, if they + * also declare this node to be in their family. */ + if (declared_family) { + /* Add every r such that router declares familyness with node, and node + * declares familyhood with router. */ + SMARTLIST_FOREACH_BEGIN(declared_family, const char *, name) { + const node_t *node2; + const smartlist_t *family2; + if (!(node2 = node_get_by_nickname(name, 0))) + continue; + if (!(family2 = node_get_declared_family(node2))) + continue; + SMARTLIST_FOREACH_BEGIN(family2, const char *, name2) { + if (node_nickname_matches(node, name2)) { + smartlist_add(sl, (void*)node2); + break; + } + } SMARTLIST_FOREACH_END(name2); + } SMARTLIST_FOREACH_END(name); + } + + /* If the user declared any families locally, honor those too. */ + if (options->NodeFamilySets) { + SMARTLIST_FOREACH(options->NodeFamilySets, const routerset_t *, rs, { + if (routerset_contains_node(rs, node)) { + routerset_get_all_nodes(sl, rs, NULL, 0); + } + }); + } +} + +/** Find a router that's up, that has this IP address, and + * that allows exit to this address:port, or return NULL if there + * isn't a good one. + * Don't exit enclave to excluded relays -- it wouldn't actually + * hurt anything, but this way there are fewer confused users. + */ +const node_t * +router_find_exact_exit_enclave(const char *address, uint16_t port) +{/*XXXX MOVE*/ + uint32_t addr; + struct in_addr in; + tor_addr_t a; + const or_options_t *options = get_options(); + + if (!tor_inet_aton(address, &in)) + return NULL; /* it's not an IP already */ + addr = ntohl(in.s_addr); + + tor_addr_from_ipv4h(&a, addr); + + SMARTLIST_FOREACH(nodelist_get_list(), const node_t *, node, { + if (node_get_addr_ipv4h(node) == addr && + node->is_running && + compare_tor_addr_to_node_policy(&a, port, node) == + ADDR_POLICY_ACCEPTED && + !routerset_contains_node(options->_ExcludeExitNodesUnion, node)) + return node; + }); + return NULL; +} + +/** Return 1 if <b>router</b> is not suitable for these parameters, else 0. + * If <b>need_uptime</b> is non-zero, we require a minimum uptime. + * If <b>need_capacity</b> is non-zero, we require a minimum advertised + * bandwidth. + * If <b>need_guard</b>, we require that the router is a possible entry guard. + */ +int +node_is_unreliable(const node_t *node, int need_uptime, + int need_capacity, int need_guard) +{ + if (need_uptime && !node->is_stable) + return 1; + if (need_capacity && !node->is_fast) + return 1; + if (need_guard && !node->is_possible_guard) + return 1; + return 0; +} + +/** Return 1 if all running sufficiently-stable routers we can use will reject + * addr:port, return 0 if any might accept it. */ +int +router_exit_policy_all_nodes_reject(const tor_addr_t *addr, uint16_t port, + int need_uptime) +{ + addr_policy_result_t r; + + SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), const node_t *, node) { + if (node->is_running && + !node_is_unreliable(node, need_uptime, 0, 0)) { + + r = compare_tor_addr_to_node_policy(addr, port, node); + + if (r != ADDR_POLICY_REJECTED && r != ADDR_POLICY_PROBABLY_REJECTED) + return 0; /* this one could be ok. good enough. */ + } + } SMARTLIST_FOREACH_END(node); + return 1; /* all will reject. */ +} + +/** Mark the router with ID <b>digest</b> as running or non-running + * in our routerlist. */ +void +router_set_status(const char *digest, int up) +{ + node_t *node; + tor_assert(digest); + + SMARTLIST_FOREACH(router_get_trusted_dir_servers(), + trusted_dir_server_t *, d, + if (tor_memeq(d->digest, digest, DIGEST_LEN)) + d->is_running = up); + + node = node_get_mutable_by_id(digest); + if (node) { +#if 0 + log_debug(LD_DIR,"Marking router %s as %s.", + node_describe(node), up ? "up" : "down"); +#endif + if (!up && node_is_me(node) && !net_is_disabled()) + log_warn(LD_NET, "We just marked ourself as down. Are your external " + "addresses reachable?"); + node->is_running = up; + } + + router_dir_info_changed(); +} + +/** True iff, the last time we checked whether we had enough directory info + * to build circuits, the answer was "yes". */ +static int have_min_dir_info = 0; +/** True iff enough has changed since the last time we checked whether we had + * enough directory info to build circuits that our old answer can no longer + * be trusted. */ +static int need_to_update_have_min_dir_info = 1; +/** String describing what we're missing before we have enough directory + * info. */ +static char dir_info_status[128] = ""; + +/** Return true iff we have enough networkstatus and router information to + * start building circuits. Right now, this means "more than half the + * networkstatus documents, and at least 1/4 of expected routers." */ +//XXX should consider whether we have enough exiting nodes here. +int +router_have_minimum_dir_info(void) +{ + if (PREDICT_UNLIKELY(need_to_update_have_min_dir_info)) { + update_router_have_minimum_dir_info(); + need_to_update_have_min_dir_info = 0; + } + return have_min_dir_info; +} + +/** Called when our internal view of the directory has changed. This can be + * when the authorities change, networkstatuses change, the list of routerdescs + * changes, or number of running routers changes. + */ +void +router_dir_info_changed(void) +{ + need_to_update_have_min_dir_info = 1; + rend_hsdir_routers_changed(); +} + +/** Return a string describing what we're missing before we have enough + * directory info. */ +const char * +get_dir_info_status_string(void) +{ + return dir_info_status; +} + +/** Iterate over the servers listed in <b>consensus</b>, and count how many of + * them seem like ones we'd use, and how many of <em>those</em> we have + * descriptors for. Store the former in *<b>num_usable</b> and the latter in + * *<b>num_present</b>. If <b>in_set</b> is non-NULL, only consider those + * routers in <b>in_set</b>. If <b>exit_only</b> is true, only consider nodes + * with the Exit flag. + */ +static void +count_usable_descriptors(int *num_present, int *num_usable, + const networkstatus_t *consensus, + const or_options_t *options, time_t now, + routerset_t *in_set, int exit_only) +{ + const int md = (consensus->flavor == FLAV_MICRODESC); + *num_present = 0, *num_usable=0; + + SMARTLIST_FOREACH_BEGIN(consensus->routerstatus_list, routerstatus_t *, rs) + { + if (exit_only && ! rs->is_exit) + continue; + if (in_set && ! routerset_contains_routerstatus(in_set, rs, -1)) + continue; + if (client_would_use_router(rs, now, options)) { + const char * const digest = rs->descriptor_digest; + int present; + ++*num_usable; /* the consensus says we want it. */ + if (md) + present = NULL != microdesc_cache_lookup_by_digest256(NULL, digest); + else + present = NULL != router_get_by_descriptor_digest(digest); + if (present) { + /* we have the descriptor listed in the consensus. */ + ++*num_present; + } + } + } + SMARTLIST_FOREACH_END(rs); + + log_debug(LD_DIR, "%d usable, %d present (%s).", *num_usable, *num_present, + md ? "microdescs" : "descs"); +} + +/** We just fetched a new set of descriptors. Compute how far through + * the "loading descriptors" bootstrapping phase we are, so we can inform + * the controller of our progress. */ +int +count_loading_descriptors_progress(void) +{ + int num_present = 0, num_usable=0; + time_t now = time(NULL); + const networkstatus_t *consensus = + networkstatus_get_reasonably_live_consensus(now,usable_consensus_flavor()); + double fraction; + + if (!consensus) + return 0; /* can't count descriptors if we have no list of them */ + + count_usable_descriptors(&num_present, &num_usable, + consensus, get_options(), now, NULL, 0); + + if (num_usable == 0) + return 0; /* don't div by 0 */ + fraction = num_present / (num_usable/4.); + if (fraction > 1.0) + return 0; /* it's not the number of descriptors holding us back */ + return BOOTSTRAP_STATUS_LOADING_DESCRIPTORS + (int) + (fraction*(BOOTSTRAP_STATUS_CONN_OR-1 - + BOOTSTRAP_STATUS_LOADING_DESCRIPTORS)); +} + +/** Change the value of have_min_dir_info, setting it true iff we have enough + * network and router information to build circuits. Clear the value of + * need_to_update_have_min_dir_info. */ +static void +update_router_have_minimum_dir_info(void) +{ + int num_present = 0, num_usable=0; + int num_exit_present = 0, num_exit_usable = 0; + time_t now = time(NULL); + int res; + const or_options_t *options = get_options(); + const networkstatus_t *consensus = + networkstatus_get_reasonably_live_consensus(now,usable_consensus_flavor()); + int using_md; + + if (!consensus) { + if (!networkstatus_get_latest_consensus()) + strlcpy(dir_info_status, "We have no usable consensus.", + sizeof(dir_info_status)); + else + strlcpy(dir_info_status, "We have no recent usable consensus.", + sizeof(dir_info_status)); + res = 0; + goto done; + } + + if (should_delay_dir_fetches(get_options())) { + log_notice(LD_DIR, "no known bridge descriptors running yet; stalling"); + strlcpy(dir_info_status, "No live bridge descriptors.", + sizeof(dir_info_status)); + res = 0; + goto done; + } + + using_md = consensus->flavor == FLAV_MICRODESC; + + count_usable_descriptors(&num_present, &num_usable, consensus, options, now, + NULL, 0); + count_usable_descriptors(&num_exit_present, &num_exit_usable, + consensus, options, now, options->ExitNodes, 1); + +/* What fraction of desired server descriptors do we need before we will + * build circuits? */ +#define FRAC_USABLE_NEEDED .75 +/* What fraction of desired _exit_ server descriptors do we need before we + * will build circuits? */ +#define FRAC_EXIT_USABLE_NEEDED .5 + + if (num_present < num_usable * FRAC_USABLE_NEEDED) { + tor_snprintf(dir_info_status, sizeof(dir_info_status), + "We have only %d/%d usable %sdescriptors.", + num_present, num_usable, using_md ? "micro" : ""); + res = 0; + control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_DESCRIPTORS, 0); + goto done; + } else if (num_present < 2) { + tor_snprintf(dir_info_status, sizeof(dir_info_status), + "Only %d %sdescriptor%s here and believed reachable!", + num_present, using_md ? "micro" : "", num_present ? "" : "s"); + res = 0; + goto done; + } else if (num_exit_present < num_exit_usable * FRAC_EXIT_USABLE_NEEDED) { + tor_snprintf(dir_info_status, sizeof(dir_info_status), + "We have only %d/%d usable exit node descriptors.", + num_exit_present, num_exit_usable); + res = 0; + control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_DESCRIPTORS, 0); + goto done; + } + + /* Check for entry nodes. */ + if (options->EntryNodes) { + count_usable_descriptors(&num_present, &num_usable, consensus, options, + now, options->EntryNodes, 0); + + if (!num_usable || !num_present) { + tor_snprintf(dir_info_status, sizeof(dir_info_status), + "We have only %d/%d usable entry node %sdescriptors.", + num_present, num_usable, using_md?"micro":""); + res = 0; + goto done; + } + } + + res = 1; + + done: + if (res && !have_min_dir_info) { + log(LOG_NOTICE, LD_DIR, + "We now have enough directory information to build circuits."); + control_event_client_status(LOG_NOTICE, "ENOUGH_DIR_INFO"); + control_event_bootstrap(BOOTSTRAP_STATUS_CONN_OR, 0); + } + if (!res && have_min_dir_info) { + int quiet = directory_too_idle_to_fetch_descriptors(options, now); + log(quiet ? LOG_INFO : LOG_NOTICE, LD_DIR, + "Our directory information is no longer up-to-date " + "enough to build circuits: %s", dir_info_status); + + /* a) make us log when we next complete a circuit, so we know when Tor + * is back up and usable, and b) disable some activities that Tor + * should only do while circuits are working, like reachability tests + * and fetching bridge descriptors only over circuits. */ + can_complete_circuit = 0; + + control_event_client_status(LOG_NOTICE, "NOT_ENOUGH_DIR_INFO"); + } + have_min_dir_info = res; + need_to_update_have_min_dir_info = 0; +} + diff --git a/src/or/nodelist.h b/src/or/nodelist.h index 1e9da88d4..2e978f178 100644 --- a/src/or/nodelist.h +++ b/src/or/nodelist.h @@ -5,17 +5,21 @@ /* See LICENSE for licensing information */ /** - * \file microdesc.h - * \brief Header file for microdesc.c. + * \file nodelist.h + * \brief Header file for nodelist.c. **/ #ifndef _TOR_NODELIST_H #define _TOR_NODELIST_H +#define node_assert_ok(n) STMT_BEGIN { \ + tor_assert((n)->ri || (n)->rs); \ + } STMT_END + node_t *node_get_mutable_by_id(const char *identity_digest); const node_t *node_get_by_id(const char *identity_digest); const node_t *node_get_by_hex_id(const char *identity_digest); -node_t *nodelist_add_routerinfo(routerinfo_t *ri); +node_t *nodelist_set_routerinfo(routerinfo_t *ri, routerinfo_t **ri_old_out); node_t *nodelist_add_microdesc(microdesc_t *md); void nodelist_set_consensus(networkstatus_t *ns); @@ -38,18 +42,18 @@ int node_get_purpose(const node_t *node); int node_is_me(const node_t *node); int node_exit_policy_rejects_all(const node_t *node); smartlist_t *node_get_all_orports(const node_t *node); -void node_get_prim_orport(const node_t *node, tor_addr_port_t *addr_port_out); -void node_get_pref_orport(const node_t *node, tor_addr_port_t *addr_port_out); -void node_get_pref_ipv6_orport(const node_t *node, - tor_addr_port_t *addr_port_out); -uint32_t node_get_prim_addr_ipv4h(const node_t *node); int node_allows_single_hop_exits(const node_t *node); const char *node_get_nickname(const node_t *node); const char *node_get_platform(const node_t *node); +uint32_t node_get_prim_addr_ipv4h(const node_t *node); void node_get_address_string(const node_t *node, char *cp, size_t len); long node_get_declared_uptime(const node_t *node); time_t node_get_published_on(const node_t *node); const smartlist_t *node_get_declared_family(const node_t *node); +int node_ipv6_preferred(const node_t *node); +int node_get_prim_orport(const node_t *node, tor_addr_port_t *ap_out); +void node_get_pref_orport(const node_t *node, tor_addr_port_t *ap_out); +void node_get_pref_ipv6_orport(const node_t *node, tor_addr_port_t *ap_out); smartlist_t *nodelist_get_list(void); @@ -57,11 +61,22 @@ smartlist_t *nodelist_get_list(void); void node_get_addr(const node_t *node, tor_addr_t *addr_out); #define node_get_addr_ipv4h(n) node_get_prim_addr_ipv4h((n)) -/* XXXX These need to move out of routerlist.c */ void nodelist_refresh_countries(void); void node_set_country(node_t *node); void nodelist_add_node_and_family(smartlist_t *nodes, const node_t *node); int nodes_in_same_family(const node_t *node1, const node_t *node2); +const node_t *router_find_exact_exit_enclave(const char *address, + uint16_t port); +int node_is_unreliable(const node_t *router, int need_uptime, + int need_capacity, int need_guard); +int router_exit_policy_all_nodes_reject(const tor_addr_t *addr, uint16_t port, + int need_uptime); +void router_set_status(const char *digest, int up); +int router_have_minimum_dir_info(void); +void router_dir_info_changed(void); +const char *get_dir_info_status_string(void); +int count_loading_descriptors_progress(void); + #endif diff --git a/src/or/or.h b/src/or/or.h index 51c23d305..f7914b830 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -98,6 +98,7 @@ #include "address.h" #include "compat_libevent.h" #include "ht.h" +#include "replaycache.h" /* These signals are defined to help handle_control_signal work. */ @@ -1769,8 +1770,6 @@ typedef struct { /** True if, after we have added this router, we should re-launch * tests for it. */ unsigned int needs_retest_if_added:1; - /** True if ipv6_addr:ipv6_orport is preferred. */ - unsigned int ipv6_preferred:1; /** Tor can use this router for general positions in circuits; we got it * from a directory server as usual, or we're an authority and a server @@ -1793,15 +1792,6 @@ typedef struct { * things; see notes on ROUTER_PURPOSE_* macros above. */ uint8_t purpose; - - /* The below items are used only by authdirservers for - * reachability testing. */ - - /** When was the last time we could reach this OR? */ - time_t last_reachable; - /** When did we start testing reachability for this OR? */ - time_t testing_since; - } routerinfo_t; /** Information needed to keep and cache a signed extra-info document. */ @@ -1833,6 +1823,8 @@ typedef struct routerstatus_t { uint32_t addr; /**< IPv4 address for this router. */ uint16_t or_port; /**< OR port for this router. */ uint16_t dir_port; /**< Directory port for this router. */ + tor_addr_t ipv6_addr; /**< IPv6 address for this router. */ + uint16_t ipv6_orport; /**<IPV6 OR port for this router. */ unsigned int is_authority:1; /**< True iff this router is an authority. */ unsigned int is_exit:1; /**< True iff this router is a good exit. */ unsigned int is_stable:1; /**< True iff this router stays up a long time. */ @@ -1862,16 +1854,7 @@ typedef struct routerstatus_t { * included.) We'll replace all these with a big tor_version_t or a char[] * if the number of traits we care about ever becomes incredibly big. */ unsigned int version_known:1; - /** True iff this router is a version that supports BEGIN_DIR cells. */ - unsigned int version_supports_begindir:1; - /** True iff this router is a version that supports conditional consensus - * downloads (signed by list of authorities). */ - unsigned int version_supports_conditional_consensus:1; - /** True iff this router is a version that we can post extrainfo docs to. */ - unsigned int version_supports_extrainfo_upload:1; - /** True iff this router is a version that, if it caches directory info, - * we can get v3 downloads from. */ - unsigned int version_supports_v3_dir:1; + /** True iff this router is a version that, if it caches directory info, * we can get microdescriptors from. */ unsigned int version_supports_microdesc_cache:1; @@ -1968,6 +1951,10 @@ typedef struct microdesc_t { /** As routerinfo_t.onion_pkey */ crypto_pk_t *onion_pkey; + /** As routerinfo_t.ipv6_add */ + tor_addr_t ipv6_addr; + /** As routerinfo_t.ipv6_orport */ + uint16_t ipv6_orport; /** As routerinfo_t.family */ smartlist_t *family; /** Exit policy summary */ @@ -2006,13 +1993,13 @@ typedef struct node_t { routerstatus_t *rs; /* local info: copied from routerstatus, then possibly frobbed based - * on experience. Authorities set this stuff directly. */ + * on experience. Authorities set this stuff directly. Note that + * these reflect knowledge of the primary (IPv4) OR port only. */ unsigned int is_running:1; /**< As far as we know, is this OR currently * running? */ unsigned int is_valid:1; /**< Has a trusted dirserver validated this OR? - * (For Authdir: Have we validated this OR?) - */ + * (For Authdir: Have we validated this OR?) */ unsigned int is_fast:1; /** Do we think this is a fast OR? */ unsigned int is_stable:1; /** Do we think this is a stable OR? */ unsigned int is_possible_guard:1; /**< Do we think this is an OK guard? */ @@ -2035,8 +2022,23 @@ typedef struct node_t { /* Local info: derived. */ + /** True if the IPv6 OR port is preferred over the IPv4 OR port. */ + unsigned int ipv6_preferred:1; + /** According to the geoip db what country is this router in? */ + /* XXXprop186 what is this suppose to mean with multiple OR ports? */ country_t country; + + /* The below items are used only by authdirservers for + * reachability testing. */ + + /** When was the last time we could reach this OR? */ + time_t last_reachable; /* IPv4. */ + time_t last_reachable6; /* IPv6. */ + + /** When did we start testing reachability for this OR? */ + time_t testing_since; /* IPv4. */ + time_t testing_since6; /* IPv6. */ } node_t; /** How many times will we try to download a router's descriptor before giving @@ -2099,6 +2101,9 @@ typedef struct vote_microdesc_hash_t { typedef struct vote_routerstatus_t { routerstatus_t status; /**< Underlying 'status' object for this router. * Flags are redundant. */ + /** How many known-flags are allowed in a vote? This is the width of + * the flags field of vote_routerstatus_t */ +#define MAX_KNOWN_FLAGS_IN_VOTE 64 uint64_t flags; /**< Bit-field for all recognized flags; index into * networkstatus_t.known_flags. */ char *version; /**< The version that the authority says this router is @@ -3312,6 +3317,7 @@ typedef struct { int AuthDirMaxServersPerAuthAddr; /**< Do not permit more than this * number of servers per IP address shared * with an authority. */ + int AuthDirHasIPv6Connectivity; /**< Boolean: are we on IPv6? */ /** If non-zero, always vote the Fast flag for any relay advertising * this amount of capacity or more. */ @@ -3478,6 +3484,13 @@ typedef struct { * over randomly chosen exits. */ int ClientRejectInternalAddresses; + /** If true, clients may connect over IPv6. XXX we don't really + enforce this -- clients _may_ set up outgoing IPv6 connections + even when this option is not set. */ + int ClientUseIPv6; + /** If true, prefer an IPv6 OR port over an IPv4 one. */ + int ClientPreferIPv6ORPort; + /** The length of time that we think a consensus should be fresh. */ int V3AuthVotingInterval; /** The length of time we think it will take to distribute votes. */ @@ -3559,8 +3572,8 @@ typedef struct { /** If true, do not enable IOCP on windows with bufferevents, even if * we think we could. */ int DisableIOCP; - /** For testing only: will go away in 0.2.3.x. */ - int _UseFilteringSSLBufferevents; + /** For testing only: will go away eventually. */ + int UseFilteringSSLBufferevents; /** Set to true if the TestingTorNetwork configuration option is set. * This is used so that options_validate() has a chance to realize that @@ -4251,12 +4264,15 @@ typedef struct rend_intro_point_t { * intro point. */ unsigned int rend_service_note_removing_intro_point_called : 1; - /** (Service side only) A digestmap recording the INTRODUCE2 cells - * this intro point's circuit has received. Each key is the digest - * of the RSA-encrypted part of a received INTRODUCE2 cell; each - * value is a pointer to the time_t at which the cell was received. - * This digestmap is used to prevent replay attacks. */ - digestmap_t *accepted_intro_rsa_parts; + /** (Service side only) A replay cache recording the RSA-encrypted parts + * of INTRODUCE2 cells this intro point's circuit has received. This is + * used to prevent replay attacks. */ + replaycache_t *accepted_intro_rsa_parts; + + /** (Service side only) Count of INTRODUCE2 cells accepted from this + * intro point. + */ + int accepted_introduce2_count; /** (Service side only) The time at which this intro point was first * published, or -1 if this intro point has not yet been diff --git a/src/or/policies.c b/src/or/policies.c index 6e984211b..568bc88a0 100644 --- a/src/or/policies.c +++ b/src/or/policies.c @@ -1008,8 +1008,7 @@ policy_write_item(char *buf, size_t buflen, addr_policy_t *policy, else addrpart = addrbuf; - result = tor_snprintf(buf, buflen, "%s%s%s %s", - (is_ip6&&format_for_desc)?"opt ":"", + result = tor_snprintf(buf, buflen, "%s%s %s", is_accept ? "accept" : "reject", (is_ip6&&format_for_desc)?"6":"", addrpart); diff --git a/src/or/relay.c b/src/or/relay.c index 5f7fcd8b7..791091569 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1175,7 +1175,7 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, case RELAY_COMMAND_EXTEND: { static uint64_t total_n_extend=0, total_nonearly=0; total_n_extend++; - if (conn) { + if (rh.stream_id) { log_fn(LOG_PROTOCOL_WARN, domain, "'extend' cell received for non-zero stream. Dropping."); return 0; @@ -1267,7 +1267,8 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, if (layer_hint) { if (layer_hint->package_window + CIRCWINDOW_INCREMENT > CIRCWINDOW_START_MAX) { - log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + /*XXXX024: Downgrade this back to LOG_PROTOCOL_WARN after a while*/ + log_fn(LOG_WARN, LD_PROTOCOL, "Bug/attack: unexpected sendme cell from exit relay. " "Closing circ."); return -END_CIRC_REASON_TORPROTOCOL; @@ -1279,7 +1280,8 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, } else { if (circ->package_window + CIRCWINDOW_INCREMENT > CIRCWINDOW_START_MAX) { - log_fn(LOG_PROTOCOL_WARN, LD_PROTOCOL, + /*XXXX024: Downgrade this back to LOG_PROTOCOL_WARN after a while*/ + log_fn(LOG_WARN, LD_PROTOCOL, "Bug/attack: unexpected sendme cell from client. " "Closing circ."); return -END_CIRC_REASON_TORPROTOCOL; @@ -1833,7 +1835,7 @@ packed_cell_free_unchecked(packed_cell_t *cell) /** Allocate and return a new packed_cell_t. */ static INLINE packed_cell_t * -packed_cell_alloc(void) +packed_cell_new(void) { ++total_cells_allocated; return mp_pool_get(cell_pool); @@ -1862,7 +1864,7 @@ dump_cell_pool_usage(int severity) static INLINE packed_cell_t * packed_cell_copy(const cell_t *cell) { - packed_cell_t *c = packed_cell_alloc(); + packed_cell_t *c = packed_cell_new(); cell_pack(c, cell); c->next = NULL; return c; diff --git a/src/or/rendclient.c b/src/or/rendclient.c index 6c751be27..73e1c41d7 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -23,6 +23,7 @@ #include "rephist.h" #include "router.h" #include "routerlist.h" +#include "routerset.h" static extend_info_t *rend_client_get_random_intro_impl( const rend_cache_entry_t *rend_query, @@ -132,6 +133,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc, crypt_path_t *cpath; off_t dh_offset; crypto_pk_t *intro_key = NULL; + int status = 0; tor_assert(introcirc->_base.purpose == CIRCUIT_PURPOSE_C_INTRODUCING); tor_assert(rendcirc->_base.purpose == CIRCUIT_PURPOSE_C_REND_READY); @@ -161,7 +163,8 @@ rend_client_send_introduction(origin_circuit_t *introcirc, } } - return -1; + status = -1; + goto cleanup; } /* first 20 bytes of payload are the hash of Bob's pk */ @@ -184,13 +187,16 @@ rend_client_send_introduction(origin_circuit_t *introcirc, smartlist_len(entry->parsed->intro_nodes)); if (rend_client_reextend_intro_circuit(introcirc)) { + status = -2; goto perm_err; } else { - return -1; + status = -1; + goto cleanup; } } if (crypto_pk_get_digest(intro_key, payload)<0) { log_warn(LD_BUG, "Internal error: couldn't hash public key."); + status = -2; goto perm_err; } @@ -202,10 +208,12 @@ rend_client_send_introduction(origin_circuit_t *introcirc, cpath->magic = CRYPT_PATH_MAGIC; if (!(cpath->dh_handshake_state = crypto_dh_new(DH_TYPE_REND))) { log_warn(LD_BUG, "Internal error: couldn't allocate DH."); + status = -2; goto perm_err; } if (crypto_dh_generate_public(cpath->dh_handshake_state)<0) { log_warn(LD_BUG, "Internal error: couldn't generate g^x."); + status = -2; goto perm_err; } } @@ -256,6 +264,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc, if (crypto_dh_get_public(cpath->dh_handshake_state, tmp+dh_offset, DH_KEY_LEN)<0) { log_warn(LD_BUG, "Internal error: couldn't extract g^x."); + status = -2; goto perm_err; } @@ -269,6 +278,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc, PK_PKCS1_OAEP_PADDING, 0); if (r<0) { log_warn(LD_BUG,"Internal error: hybrid pk encrypt failed."); + status = -2; goto perm_err; } @@ -288,7 +298,8 @@ rend_client_send_introduction(origin_circuit_t *introcirc, introcirc->cpath->prev)<0) { /* introcirc is already marked for close. leave rendcirc alone. */ log_warn(LD_BUG, "Couldn't send INTRODUCE1 cell"); - return -2; + status = -2; + goto cleanup; } /* Now, we wait for an ACK or NAK on this circuit. */ @@ -299,12 +310,17 @@ rend_client_send_introduction(origin_circuit_t *introcirc, * state. */ introcirc->_base.timestamp_dirty = time(NULL); - return 0; + goto cleanup; + perm_err: if (!introcirc->_base.marked_for_close) circuit_mark_for_close(TO_CIRCUIT(introcirc), END_CIRC_REASON_INTERNAL); circuit_mark_for_close(TO_CIRCUIT(rendcirc), END_CIRC_REASON_INTERNAL); - return -2; + cleanup: + memset(payload, 0, sizeof(payload)); + memset(tmp, 0, sizeof(tmp)); + + return status; } /** Called when a rendezvous circuit is open; sends a establish @@ -602,7 +618,8 @@ directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) directory_initiate_command_routerstatus_rend(hs_dir, DIR_PURPOSE_FETCH_RENDDESC_V2, ROUTER_PURPOSE_GENERAL, - !tor2web_mode, desc_id_base32, + tor2web_mode?DIRIND_ONEHOP:DIRIND_ANONYMOUS, + desc_id_base32, NULL, 0, 0, rend_query); log_info(LD_REND, "Sending fetch request for v2 descriptor for " @@ -659,10 +676,17 @@ rend_client_refetch_v2_renddesc(const rend_data_t *rend_query) time(NULL), chosen_replica) < 0) { log_warn(LD_REND, "Internal error: Computing v2 rendezvous " "descriptor ID did not succeed."); - return; + /* + * Hmm, can this write anything to descriptor_id and still fail? + * Let's clear it just to be safe. + * + * From here on, any returns should goto done which clears + * descriptor_id so we don't leave key-derived material on the stack. + */ + goto done; } if (directory_get_from_hs_dir(descriptor_id, rend_query) != 0) - return; /* either success or failure, but we're done */ + goto done; /* either success or failure, but we're done */ } /* If we come here, there are no hidden service directories left. */ log_info(LD_REND, "Could not pick one of the responsible hidden " @@ -670,6 +694,10 @@ rend_client_refetch_v2_renddesc(const rend_data_t *rend_query) "we already tried them all unsuccessfully."); /* Close pending connections. */ rend_client_desc_trynow(rend_query->onion_address); + + done: + memset(descriptor_id, 0, sizeof(descriptor_id)); + return; } @@ -1172,11 +1200,11 @@ rend_parse_service_authorization(const or_options_t *options, strmap_t *parsed = strmap_new(); smartlist_t *sl = smartlist_new(); rend_service_authorization_t *auth = NULL; + char descriptor_cookie_tmp[REND_DESC_COOKIE_LEN+2]; + char descriptor_cookie_base64ext[REND_DESC_COOKIE_LEN_BASE64+2+1]; for (line = options->HidServAuth; line; line = line->next) { char *onion_address, *descriptor_cookie; - char descriptor_cookie_tmp[REND_DESC_COOKIE_LEN+2]; - char descriptor_cookie_base64ext[REND_DESC_COOKIE_LEN_BASE64+2+1]; int auth_type_val = 0; auth = NULL; SMARTLIST_FOREACH(sl, char *, c, tor_free(c);); @@ -1253,6 +1281,8 @@ rend_parse_service_authorization(const or_options_t *options, } else { strmap_free(parsed, rend_service_authorization_strmap_item_free); } + memset(descriptor_cookie_tmp, 0, sizeof(descriptor_cookie_tmp)); + memset(descriptor_cookie_base64ext, 0, sizeof(descriptor_cookie_base64ext)); return res; } diff --git a/src/or/rendcommon.c b/src/or/rendcommon.c index 4722690c1..f6b1bf9f6 100644 --- a/src/or/rendcommon.c +++ b/src/or/rendcommon.c @@ -439,7 +439,7 @@ rend_intro_point_free(rend_intro_point_t *intro) crypto_pk_free(intro->intro_key); if (intro->accepted_intro_rsa_parts != NULL) { - digestmap_free(intro->accepted_intro_rsa_parts, _tor_free); + replaycache_free(intro->accepted_intro_rsa_parts); } tor_free(intro); diff --git a/src/or/rendservice.c b/src/or/rendservice.c index 6af4778df..0bfa17d10 100644 --- a/src/or/rendservice.c +++ b/src/or/rendservice.c @@ -7,6 +7,8 @@ * \brief The hidden-service side of rendezvous functionality. **/ +#define RENDSERVICE_PRIVATE + #include "or.h" #include "circuitbuild.h" #include "circuitlist.h" @@ -21,16 +23,42 @@ #include "router.h" #include "relay.h" #include "rephist.h" +#include "replaycache.h" #include "routerlist.h" #include "routerparse.h" +#include "routerset.h" static origin_circuit_t *find_intro_circuit(rend_intro_point_t *intro, const char *pk_digest); static rend_intro_point_t *find_intro_point(origin_circuit_t *circ); +static extend_info_t *find_rp_for_intro( + const rend_intro_cell_t *intro, + uint8_t *need_free_out, char **err_msg_out); + static int intro_point_accepted_intro_count(rend_intro_point_t *intro); static int intro_point_should_expire_now(rend_intro_point_t *intro, time_t now); +struct rend_service_t; +static int rend_service_load_keys(struct rend_service_t *s); +static int rend_service_load_auth_keys(struct rend_service_t *s, + const char *hfname); + +static ssize_t rend_service_parse_intro_for_v0_or_v1( + rend_intro_cell_t *intro, + const uint8_t *buf, + size_t plaintext_len, + char **err_msg_out); +static ssize_t rend_service_parse_intro_for_v2( + rend_intro_cell_t *intro, + const uint8_t *buf, + size_t plaintext_len, + char **err_msg_out); +static ssize_t rend_service_parse_intro_for_v3( + rend_intro_cell_t *intro, + const uint8_t *buf, + size_t plaintext_len, + char **err_msg_out); /** Represents the mapping from a virtual port of a rendezvous service to * a real port on some IP. @@ -91,16 +119,12 @@ typedef struct rend_service_t { * up-to-date. */ time_t next_upload_time; /**< Scheduled next hidden service descriptor * upload time. */ - /** Map from digests of Diffie-Hellman values INTRODUCE2 to time_t - * of when they were received. Clients may send INTRODUCE1 cells - * for the same rendezvous point through two or more different - * introduction points; when they do, this digestmap keeps us from - * launching multiple simultaneous attempts to connect to the same - * rend point. */ - digestmap_t *accepted_intro_dh_parts; - /** Time at which we last removed expired values from - * accepted_intro_dh_parts. */ - time_t last_cleaned_accepted_intro_dh_parts; + /** Replay cache for Diffie-Hellman values of INTRODUCE2 cells, to + * detect repeats. Clients may send INTRODUCE1 cells for the same + * rendezvous point through two or more different introduction points; + * when they do, this keeps us from launching multiple simultaneous attempts + * to connect to the same rend point. */ + replaycache_t *accepted_intro_dh_parts; } rend_service_t; /** A list of rend_service_t's for services run on this OP. @@ -135,7 +159,9 @@ rend_authorized_client_free(rend_authorized_client_t *client) return; if (client->client_key) crypto_pk_free(client->client_key); + tor_strclear(client->client_name); tor_free(client->client_name); + memset(client->descriptor_cookie, 0, sizeof(client->descriptor_cookie)); tor_free(client); } @@ -171,7 +197,9 @@ rend_service_free(rend_service_t *service) rend_authorized_client_free(c);); smartlist_free(service->clients); } - digestmap_free(service->accepted_intro_dh_parts, _tor_free); + if (service->accepted_intro_dh_parts) { + replaycache_free(service->accepted_intro_dh_parts); + } tor_free(service); } @@ -609,231 +637,274 @@ rend_service_update_descriptor(rend_service_t *service) /** Load and/or generate private keys for all hidden services, possibly * including keys for client authorization. Return 0 on success, -1 on - * failure. - */ + * failure. */ int -rend_service_load_keys(void) +rend_service_load_all_keys(void) { - int r = 0; - char fname[512]; - char buf[1500]; - SMARTLIST_FOREACH_BEGIN(rend_service_list, rend_service_t *, s) { if (s->private_key) continue; log_info(LD_REND, "Loading hidden-service keys from \"%s\"", s->directory); - /* Check/create directory */ - if (check_private_dir(s->directory, CPD_CREATE, get_options()->User) < 0) + if (rend_service_load_keys(s) < 0) return -1; + } SMARTLIST_FOREACH_END(s); + + return 0; +} + +/** Load and/or generate private keys for the hidden service <b>s</b>, + * possibly including keys for client authorization. Return 0 on success, -1 + * on failure. */ +static int +rend_service_load_keys(rend_service_t *s) +{ + char fname[512]; + char buf[128]; + + /* Check/create directory */ + if (check_private_dir(s->directory, CPD_CREATE, get_options()->User) < 0) + return -1; + + /* Load key */ + if (strlcpy(fname,s->directory,sizeof(fname)) >= sizeof(fname) || + strlcat(fname,PATH_SEPARATOR"private_key",sizeof(fname)) + >= sizeof(fname)) { + log_warn(LD_CONFIG, "Directory name too long to store key file: \"%s\".", + s->directory); + return -1; + } + s->private_key = init_key_from_file(fname, 1, LOG_ERR); + if (!s->private_key) + return -1; + + /* Create service file */ + if (rend_get_service_id(s->private_key, s->service_id)<0) { + log_warn(LD_BUG, "Internal error: couldn't encode service ID."); + return -1; + } + if (crypto_pk_get_digest(s->private_key, s->pk_digest)<0) { + log_warn(LD_BUG, "Couldn't compute hash of public key."); + return -1; + } + if (strlcpy(fname,s->directory,sizeof(fname)) >= sizeof(fname) || + strlcat(fname,PATH_SEPARATOR"hostname",sizeof(fname)) + >= sizeof(fname)) { + log_warn(LD_CONFIG, "Directory name too long to store hostname file:" + " \"%s\".", s->directory); + return -1; + } + + tor_snprintf(buf, sizeof(buf),"%s.onion\n", s->service_id); + if (write_str_to_file(fname,buf,0)<0) { + log_warn(LD_CONFIG, "Could not write onion address to hostname file."); + memset(buf, 0, sizeof(buf)); + return -1; + } + memset(buf, 0, sizeof(buf)); - /* Load key */ - if (strlcpy(fname,s->directory,sizeof(fname)) >= sizeof(fname) || - strlcat(fname,PATH_SEPARATOR"private_key",sizeof(fname)) - >= sizeof(fname)) { - log_warn(LD_CONFIG, "Directory name too long to store key file: \"%s\".", - s->directory); + /* If client authorization is configured, load or generate keys. */ + if (s->auth_type != REND_NO_AUTH) { + if (rend_service_load_auth_keys(s, fname) < 0) return -1; + } + + return 0; +} + +/** Load and/or generate client authorization keys for the hidden service + * <b>s</b>, which stores its hostname in <b>hfname</b>. Return 0 on success, + * -1 on failure. */ +static int +rend_service_load_auth_keys(rend_service_t *s, const char *hfname) +{ + int r = 0; + char cfname[512]; + char *client_keys_str = NULL; + strmap_t *parsed_clients = strmap_new(); + FILE *cfile, *hfile; + open_file_t *open_cfile = NULL, *open_hfile = NULL; + char extended_desc_cookie[REND_DESC_COOKIE_LEN+1]; + char desc_cook_out[3*REND_DESC_COOKIE_LEN_BASE64+1]; + char service_id[16+1]; + char buf[1500]; + + /* Load client keys and descriptor cookies, if available. */ + if (tor_snprintf(cfname, sizeof(cfname), "%s"PATH_SEPARATOR"client_keys", + s->directory)<0) { + log_warn(LD_CONFIG, "Directory name too long to store client keys " + "file: \"%s\".", s->directory); + goto err; + } + client_keys_str = read_file_to_str(cfname, RFTS_IGNORE_MISSING, NULL); + if (client_keys_str) { + if (rend_parse_client_keys(parsed_clients, client_keys_str) < 0) { + log_warn(LD_CONFIG, "Previously stored client_keys file could not " + "be parsed."); + goto err; + } else { + log_info(LD_CONFIG, "Parsed %d previously stored client entries.", + strmap_size(parsed_clients)); } - s->private_key = init_key_from_file(fname, 1, LOG_ERR); - if (!s->private_key) - return -1; + } - /* Create service file */ - if (rend_get_service_id(s->private_key, s->service_id)<0) { - log_warn(LD_BUG, "Internal error: couldn't encode service ID."); - return -1; + /* Prepare client_keys and hostname files. */ + if (!(cfile = start_writing_to_stdio_file(cfname, + OPEN_FLAGS_REPLACE | O_TEXT, + 0600, &open_cfile))) { + log_warn(LD_CONFIG, "Could not open client_keys file %s", + escaped(cfname)); + goto err; + } + + if (!(hfile = start_writing_to_stdio_file(hfname, + OPEN_FLAGS_REPLACE | O_TEXT, + 0600, &open_hfile))) { + log_warn(LD_CONFIG, "Could not open hostname file %s", escaped(hfname)); + goto err; + } + + /* Either use loaded keys for configured clients or generate new + * ones if a client is new. */ + SMARTLIST_FOREACH_BEGIN(s->clients, rend_authorized_client_t *, client) { + rend_authorized_client_t *parsed = + strmap_get(parsed_clients, client->client_name); + int written; + size_t len; + /* Copy descriptor cookie from parsed entry or create new one. */ + if (parsed) { + memcpy(client->descriptor_cookie, parsed->descriptor_cookie, + REND_DESC_COOKIE_LEN); + } else { + crypto_rand(client->descriptor_cookie, REND_DESC_COOKIE_LEN); } - if (crypto_pk_get_digest(s->private_key, s->pk_digest)<0) { - log_warn(LD_BUG, "Couldn't compute hash of public key."); - return -1; + if (base64_encode(desc_cook_out, 3*REND_DESC_COOKIE_LEN_BASE64+1, + client->descriptor_cookie, + REND_DESC_COOKIE_LEN) < 0) { + log_warn(LD_BUG, "Could not base64-encode descriptor cookie."); + goto err; } - if (strlcpy(fname,s->directory,sizeof(fname)) >= sizeof(fname) || - strlcat(fname,PATH_SEPARATOR"hostname",sizeof(fname)) - >= sizeof(fname)) { - log_warn(LD_CONFIG, "Directory name too long to store hostname file:" - " \"%s\".", s->directory); - return -1; + /* Copy client key from parsed entry or create new one if required. */ + if (parsed && parsed->client_key) { + client->client_key = crypto_pk_dup_key(parsed->client_key); + } else if (s->auth_type == REND_STEALTH_AUTH) { + /* Create private key for client. */ + crypto_pk_t *prkey = NULL; + if (!(prkey = crypto_pk_new())) { + log_warn(LD_BUG,"Error constructing client key"); + goto err; + } + if (crypto_pk_generate_key(prkey)) { + log_warn(LD_BUG,"Error generating client key"); + crypto_pk_free(prkey); + goto err; + } + if (crypto_pk_check_key(prkey) <= 0) { + log_warn(LD_BUG,"Generated client key seems invalid"); + crypto_pk_free(prkey); + goto err; + } + client->client_key = prkey; } - tor_snprintf(buf, sizeof(buf),"%s.onion\n", s->service_id); - if (write_str_to_file(fname,buf,0)<0) { - log_warn(LD_CONFIG, "Could not write onion address to hostname file."); - return -1; + /* Add entry to client_keys file. */ + desc_cook_out[strlen(desc_cook_out)-1] = '\0'; /* Remove newline. */ + written = tor_snprintf(buf, sizeof(buf), + "client-name %s\ndescriptor-cookie %s\n", + client->client_name, desc_cook_out); + if (written < 0) { + log_warn(LD_BUG, "Could not write client entry."); + goto err; } - - /* If client authorization is configured, load or generate keys. */ - if (s->auth_type != REND_NO_AUTH) { - char *client_keys_str = NULL; - strmap_t *parsed_clients = strmap_new(); - char cfname[512]; - FILE *cfile, *hfile; - open_file_t *open_cfile = NULL, *open_hfile = NULL; - - /* Load client keys and descriptor cookies, if available. */ - if (tor_snprintf(cfname, sizeof(cfname), "%s"PATH_SEPARATOR"client_keys", - s->directory)<0) { - log_warn(LD_CONFIG, "Directory name too long to store client keys " - "file: \"%s\".", s->directory); + if (client->client_key) { + char *client_key_out = NULL; + if (crypto_pk_write_private_key_to_string(client->client_key, + &client_key_out, &len) != 0) { + log_warn(LD_BUG, "Internal error: " + "crypto_pk_write_private_key_to_string() failed."); goto err; } - client_keys_str = read_file_to_str(cfname, RFTS_IGNORE_MISSING, NULL); - if (client_keys_str) { - if (rend_parse_client_keys(parsed_clients, client_keys_str) < 0) { - log_warn(LD_CONFIG, "Previously stored client_keys file could not " - "be parsed."); - goto err; - } else { - log_info(LD_CONFIG, "Parsed %d previously stored client entries.", - strmap_size(parsed_clients)); - tor_free(client_keys_str); - } - } - - /* Prepare client_keys and hostname files. */ - if (!(cfile = start_writing_to_stdio_file(cfname, - OPEN_FLAGS_REPLACE | O_TEXT, - 0600, &open_cfile))) { - log_warn(LD_CONFIG, "Could not open client_keys file %s", - escaped(cfname)); + if (rend_get_service_id(client->client_key, service_id)<0) { + log_warn(LD_BUG, "Internal error: couldn't encode service ID."); + /* + * len is string length, not buffer length, but last byte is NUL + * anyway. + */ + memset(client_key_out, 0, len); + tor_free(client_key_out); goto err; } - if (!(hfile = start_writing_to_stdio_file(fname, - OPEN_FLAGS_REPLACE | O_TEXT, - 0600, &open_hfile))) { - log_warn(LD_CONFIG, "Could not open hostname file %s", escaped(fname)); + written = tor_snprintf(buf + written, sizeof(buf) - written, + "client-key\n%s", client_key_out); + memset(client_key_out, 0, len); + tor_free(client_key_out); + if (written < 0) { + log_warn(LD_BUG, "Could not write client entry."); goto err; } + } - /* Either use loaded keys for configured clients or generate new - * ones if a client is new. */ - SMARTLIST_FOREACH_BEGIN(s->clients, rend_authorized_client_t *, client) - { - char desc_cook_out[3*REND_DESC_COOKIE_LEN_BASE64+1]; - char service_id[16+1]; - rend_authorized_client_t *parsed = - strmap_get(parsed_clients, client->client_name); - int written; - size_t len; - /* Copy descriptor cookie from parsed entry or create new one. */ - if (parsed) { - memcpy(client->descriptor_cookie, parsed->descriptor_cookie, - REND_DESC_COOKIE_LEN); - } else { - crypto_rand(client->descriptor_cookie, REND_DESC_COOKIE_LEN); - } - if (base64_encode(desc_cook_out, 3*REND_DESC_COOKIE_LEN_BASE64+1, - client->descriptor_cookie, - REND_DESC_COOKIE_LEN) < 0) { - log_warn(LD_BUG, "Could not base64-encode descriptor cookie."); - strmap_free(parsed_clients, rend_authorized_client_strmap_item_free); - return -1; - } - /* Copy client key from parsed entry or create new one if required. */ - if (parsed && parsed->client_key) { - client->client_key = crypto_pk_dup_key(parsed->client_key); - } else if (s->auth_type == REND_STEALTH_AUTH) { - /* Create private key for client. */ - crypto_pk_t *prkey = NULL; - if (!(prkey = crypto_pk_new())) { - log_warn(LD_BUG,"Error constructing client key"); - goto err; - } - if (crypto_pk_generate_key(prkey)) { - log_warn(LD_BUG,"Error generating client key"); - crypto_pk_free(prkey); - goto err; - } - if (crypto_pk_check_key(prkey) <= 0) { - log_warn(LD_BUG,"Generated client key seems invalid"); - crypto_pk_free(prkey); - goto err; - } - client->client_key = prkey; - } - /* Add entry to client_keys file. */ - desc_cook_out[strlen(desc_cook_out)-1] = '\0'; /* Remove newline. */ - written = tor_snprintf(buf, sizeof(buf), - "client-name %s\ndescriptor-cookie %s\n", - client->client_name, desc_cook_out); - if (written < 0) { - log_warn(LD_BUG, "Could not write client entry."); - goto err; - } - if (client->client_key) { - char *client_key_out = NULL; - crypto_pk_write_private_key_to_string(client->client_key, - &client_key_out, &len); - if (rend_get_service_id(client->client_key, service_id)<0) { - log_warn(LD_BUG, "Internal error: couldn't encode service ID."); - tor_free(client_key_out); - goto err; - } - written = tor_snprintf(buf + written, sizeof(buf) - written, - "client-key\n%s", client_key_out); - tor_free(client_key_out); - if (written < 0) { - log_warn(LD_BUG, "Could not write client entry."); - goto err; - } - } - - if (fputs(buf, cfile) < 0) { - log_warn(LD_FS, "Could not append client entry to file: %s", - strerror(errno)); - goto err; - } - - /* Add line to hostname file. */ - if (s->auth_type == REND_BASIC_AUTH) { - /* Remove == signs (newline has been removed above). */ - desc_cook_out[strlen(desc_cook_out)-2] = '\0'; - tor_snprintf(buf, sizeof(buf),"%s.onion %s # client: %s\n", - s->service_id, desc_cook_out, client->client_name); - } else { - char extended_desc_cookie[REND_DESC_COOKIE_LEN+1]; - memcpy(extended_desc_cookie, client->descriptor_cookie, - REND_DESC_COOKIE_LEN); - extended_desc_cookie[REND_DESC_COOKIE_LEN] = - ((int)s->auth_type - 1) << 4; - if (base64_encode(desc_cook_out, 3*REND_DESC_COOKIE_LEN_BASE64+1, - extended_desc_cookie, - REND_DESC_COOKIE_LEN+1) < 0) { - log_warn(LD_BUG, "Could not base64-encode descriptor cookie."); - goto err; - } - desc_cook_out[strlen(desc_cook_out)-3] = '\0'; /* Remove A= and - newline. */ - tor_snprintf(buf, sizeof(buf),"%s.onion %s # client: %s\n", - service_id, desc_cook_out, client->client_name); - } + if (fputs(buf, cfile) < 0) { + log_warn(LD_FS, "Could not append client entry to file: %s", + strerror(errno)); + goto err; + } - if (fputs(buf, hfile)<0) { - log_warn(LD_FS, "Could not append host entry to file: %s", - strerror(errno)); - goto err; - } + /* Add line to hostname file. */ + if (s->auth_type == REND_BASIC_AUTH) { + /* Remove == signs (newline has been removed above). */ + desc_cook_out[strlen(desc_cook_out)-2] = '\0'; + tor_snprintf(buf, sizeof(buf),"%s.onion %s # client: %s\n", + s->service_id, desc_cook_out, client->client_name); + } else { + memcpy(extended_desc_cookie, client->descriptor_cookie, + REND_DESC_COOKIE_LEN); + extended_desc_cookie[REND_DESC_COOKIE_LEN] = + ((int)s->auth_type - 1) << 4; + if (base64_encode(desc_cook_out, 3*REND_DESC_COOKIE_LEN_BASE64+1, + extended_desc_cookie, + REND_DESC_COOKIE_LEN+1) < 0) { + log_warn(LD_BUG, "Could not base64-encode descriptor cookie."); + goto err; } - SMARTLIST_FOREACH_END(client); + desc_cook_out[strlen(desc_cook_out)-3] = '\0'; /* Remove A= and + newline. */ + tor_snprintf(buf, sizeof(buf),"%s.onion %s # client: %s\n", + service_id, desc_cook_out, client->client_name); + } - goto done; - err: - r = -1; - done: - tor_free(client_keys_str); - strmap_free(parsed_clients, rend_authorized_client_strmap_item_free); - if (r<0) { - if (open_cfile) - abort_writing_to_file(open_cfile); - if (open_hfile) - abort_writing_to_file(open_hfile); - return r; - } else { - finish_writing_to_file(open_cfile); - finish_writing_to_file(open_hfile); - } + if (fputs(buf, hfile)<0) { + log_warn(LD_FS, "Could not append host entry to file: %s", + strerror(errno)); + goto err; } - } SMARTLIST_FOREACH_END(s); + } SMARTLIST_FOREACH_END(client); + + finish_writing_to_file(open_cfile); + finish_writing_to_file(open_hfile); + + goto done; + err: + r = -1; + if (open_cfile) + abort_writing_to_file(open_cfile); + if (open_hfile) + abort_writing_to_file(open_hfile); + done: + if (client_keys_str) { + tor_strclear(client_keys_str); + tor_free(client_keys_str); + } + strmap_free(parsed_clients, rend_authorized_client_strmap_item_free); + + memset(cfname, 0, sizeof(cfname)); + + /* Clear stack buffers that held key-derived material. */ + memset(buf, 0, sizeof(buf)); + memset(desc_cook_out, 0, sizeof(desc_cook_out)); + memset(service_id, 0, sizeof(service_id)); + memset(extended_desc_cookie, 0, sizeof(extended_desc_cookie)); + return r; } @@ -906,26 +977,6 @@ rend_check_authorization(rend_service_t *service, return 1; } -/** Remove elements from <b>service</b>'s replay cache that are old enough to - * be noticed by timestamp checking. */ -static void -clean_accepted_intro_dh_parts(rend_service_t *service, time_t now) -{ - const time_t cutoff = now - REND_REPLAY_TIME_INTERVAL; - - service->last_cleaned_accepted_intro_dh_parts = now; - if (!service->accepted_intro_dh_parts) - return; - - DIGESTMAP_FOREACH_MODIFY(service->accepted_intro_dh_parts, digest, - time_t *, t) { - if (*t < cutoff) { - tor_free(t); - MAP_DEL_CURRENT(digest); - } - } DIGESTMAP_FOREACH_END; -} - /** Called when <b>intro</b> will soon be removed from * <b>service</b>'s list of intro points. */ static void @@ -1033,42 +1084,55 @@ rend_service_note_removing_intro_point(rend_service_t *service, /** Respond to an INTRODUCE2 cell by launching a circuit to the chosen * rendezvous point. */ - /* XXXX024 this function sure could use some organizing. -RD */ int rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, size_t request_len) { - char *ptr, *r_cookie; - extend_info_t *extend_info = NULL; + /* Global status stuff */ + int status = 0, result; + const or_options_t *options = get_options(); + char *err_msg = NULL; + const char *stage_descr = NULL; + int reason = END_CIRC_REASON_TORPROTOCOL; + /* Service/circuit/key stuff we can learn before parsing */ + char serviceid[REND_SERVICE_ID_LEN_BASE32+1]; + rend_service_t *service = NULL; + rend_intro_point_t *intro_point = NULL; + crypto_pk_t *intro_key = NULL; + /* Parsed cell */ + rend_intro_cell_t *parsed_req = NULL; + /* Rendezvous point */ + extend_info_t *rp = NULL; + /* + * We need to look up and construct the extend_info_t for v0 and v1, + * but all the info is in the cell and it's constructed by the parser + * for v2 and v3, so freeing it would be a double-free. Use this to + * keep track of whether we should free it. + */ + uint8_t need_rp_free = 0; + /* XXX not handled yet */ char buf[RELAY_PAYLOAD_SIZE]; char keys[DIGEST_LEN+CPATH_KEY_MATERIAL_LEN]; /* Holds KH, Df, Db, Kf, Kb */ - rend_service_t *service; - rend_intro_point_t *intro_point; - int r, i, v3_shift = 0; - size_t len, keylen; + int i; crypto_dh_t *dh = NULL; origin_circuit_t *launched = NULL; crypt_path_t *cpath = NULL; - char serviceid[REND_SERVICE_ID_LEN_BASE32+1]; char hexcookie[9]; int circ_needs_uptime; - int reason = END_CIRC_REASON_TORPROTOCOL; - crypto_pk_t *intro_key; char intro_key_digest[DIGEST_LEN]; - int auth_type; size_t auth_len = 0; char auth_data[REND_DESC_COOKIE_LEN]; - crypto_digest_t *digest = NULL; time_t now = time(NULL); char diffie_hellman_hash[DIGEST_LEN]; - time_t *access_time; - const or_options_t *options = get_options(); + time_t elapsed; + int replay; + /* Do some initial validation and logging before we parse the cell */ if (circuit->_base.purpose != CIRCUIT_PURPOSE_S_INTRO) { log_warn(LD_PROTOCOL, "Got an INTRODUCE2 over a non-introduction circuit %d.", circuit->_base.n_circ_id); - return -1; + goto err; } #ifndef NON_ANONYMOUS_MODE_ENABLED @@ -1076,218 +1140,145 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, #endif tor_assert(circuit->rend_data); + /* We'll use this in a bazillion log messages */ base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, circuit->rend_data->rend_pk_digest, REND_SERVICE_ID_LEN); - log_info(LD_REND, "Received INTRODUCE2 cell for service %s on circ %d.", - escaped(serviceid), circuit->_base.n_circ_id); - - /* min key length plus digest length plus nickname length */ - if (request_len < DIGEST_LEN+REND_COOKIE_LEN+(MAX_NICKNAME_LEN+1)+ - DH_KEY_LEN+42) { - log_warn(LD_PROTOCOL, "Got a truncated INTRODUCE2 cell on circ %d.", - circuit->_base.n_circ_id); - return -1; - } /* look up service depending on circuit. */ - service = rend_service_get_by_pk_digest( - circuit->rend_data->rend_pk_digest); + service = + rend_service_get_by_pk_digest(circuit->rend_data->rend_pk_digest); if (!service) { - log_warn(LD_BUG, "Internal error: Got an INTRODUCE2 cell on an intro " + log_warn(LD_BUG, + "Internal error: Got an INTRODUCE2 cell on an intro " "circ for an unrecognized service %s.", escaped(serviceid)); - return -1; - } - - /* use intro key instead of service key. */ - intro_key = circuit->intro_key; - - /* first DIGEST_LEN bytes of request is intro or service pk digest */ - crypto_pk_get_digest(intro_key, intro_key_digest); - if (tor_memneq(intro_key_digest, request, DIGEST_LEN)) { - base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1, - (char*)request, REND_SERVICE_ID_LEN); - log_warn(LD_REND, "Got an INTRODUCE2 cell for the wrong service (%s).", - escaped(serviceid)); - return -1; - } - - keylen = crypto_pk_keysize(intro_key); - if (request_len < keylen+DIGEST_LEN) { - log_warn(LD_PROTOCOL, - "PK-encrypted portion of INTRODUCE2 cell was truncated."); - return -1; + goto err; } intro_point = find_intro_point(circuit); if (intro_point == NULL) { - log_warn(LD_BUG, "Internal error: Got an INTRODUCE2 cell on an intro circ " - "(for service %s) with no corresponding rend_intro_point_t.", + log_warn(LD_BUG, + "Internal error: Got an INTRODUCE2 cell on an " + "intro circ (for service %s) with no corresponding " + "rend_intro_point_t.", escaped(serviceid)); - return -1; + goto err; } - if (!service->accepted_intro_dh_parts) - service->accepted_intro_dh_parts = digestmap_new(); + log_info(LD_REND, "Received INTRODUCE2 cell for service %s on circ %d.", + escaped(serviceid), circuit->_base.n_circ_id); - if (!intro_point->accepted_intro_rsa_parts) - intro_point->accepted_intro_rsa_parts = digestmap_new(); + /* use intro key instead of service key. */ + intro_key = circuit->intro_key; - { - char pkpart_digest[DIGEST_LEN]; - /* Check for replay of PK-encrypted portion. */ - crypto_digest(pkpart_digest, (char*)request+DIGEST_LEN, keylen); - access_time = digestmap_get(intro_point->accepted_intro_rsa_parts, - pkpart_digest); - if (access_time != NULL) { - log_warn(LD_REND, "Possible replay detected! We received an " - "INTRODUCE2 cell with same PK-encrypted part %d seconds ago. " - "Dropping cell.", (int)(now-*access_time)); - return -1; - } - access_time = tor_malloc(sizeof(time_t)); - *access_time = now; - digestmap_set(intro_point->accepted_intro_rsa_parts, - pkpart_digest, access_time); + tor_free(err_msg); + stage_descr = NULL; + + stage_descr = "early parsing"; + /* Early parsing pass (get pk, ciphertext); type 2 is INTRODUCE2 */ + parsed_req = + rend_service_begin_parse_intro(request, request_len, 2, &err_msg); + if (!parsed_req) { + goto log_error; + } else if (err_msg) { + log_info(LD_REND, "%s on circ %d.", err_msg, circuit->_base.n_circ_id); + tor_free(err_msg); + } + + stage_descr = "early validation"; + /* Early validation of pk/ciphertext part */ + result = rend_service_validate_intro_early(parsed_req, &err_msg); + if (result < 0) { + goto log_error; + } else if (err_msg) { + log_info(LD_REND, "%s on circ %d.", err_msg, circuit->_base.n_circ_id); + tor_free(err_msg); + } + + /* make sure service replay caches are present */ + if (!service->accepted_intro_dh_parts) { + service->accepted_intro_dh_parts = + replaycache_new(REND_REPLAY_TIME_INTERVAL, + REND_REPLAY_TIME_INTERVAL); + } + + if (!intro_point->accepted_intro_rsa_parts) { + intro_point->accepted_intro_rsa_parts = replaycache_new(0, 0); + } + + /* check for replay of PK-encrypted portion. */ + replay = replaycache_add_test_and_elapsed( + intro_point->accepted_intro_rsa_parts, + parsed_req->ciphertext, (int)parsed_req->ciphertext_len, + &elapsed); + + if (replay) { + log_warn(LD_REND, + "Possible replay detected! We received an " + "INTRODUCE2 cell with same PK-encrypted part %d " + "seconds ago. Dropping cell.", + (int)elapsed); + goto err; } - /* Next N bytes is encrypted with service key */ - note_crypto_pk_op(REND_SERVER); - r = crypto_pk_private_hybrid_decrypt( - intro_key,buf,sizeof(buf), - (char*)(request+DIGEST_LEN),request_len-DIGEST_LEN, - PK_PKCS1_OAEP_PADDING,1); - if (r<0) { - log_warn(LD_PROTOCOL, "Couldn't decrypt INTRODUCE2 cell."); - return -1; + stage_descr = "decryption"; + /* Now try to decrypt it */ + result = rend_service_decrypt_intro(parsed_req, intro_key, &err_msg); + if (result < 0) { + goto log_error; + } else if (err_msg) { + log_info(LD_REND, "%s on circ %d.", err_msg, circuit->_base.n_circ_id); + tor_free(err_msg); } - len = r; - if (*buf == 3) { - /* Version 3 INTRODUCE2 cell. */ - v3_shift = 1; - auth_type = buf[1]; - switch (auth_type) { - case REND_BASIC_AUTH: - /* fall through */ - case REND_STEALTH_AUTH: - auth_len = ntohs(get_uint16(buf+2)); - if (auth_len != REND_DESC_COOKIE_LEN) { - log_info(LD_REND, "Wrong auth data size %d, should be %d.", - (int)auth_len, REND_DESC_COOKIE_LEN); - return -1; - } - memcpy(auth_data, buf+4, sizeof(auth_data)); - v3_shift += 2+REND_DESC_COOKIE_LEN; - break; - case REND_NO_AUTH: - break; - default: - log_info(LD_REND, "Unknown authorization type '%d'", auth_type); - } - - /* Skip the timestamp field. We no longer use it. */ - v3_shift += 4; - } - if (*buf == 2 || *buf == 3) { - /* Version 2 INTRODUCE2 cell. */ - int klen; - extend_info = tor_malloc_zero(sizeof(extend_info_t)); - tor_addr_from_ipv4n(&extend_info->addr, get_uint32(buf+v3_shift+1)); - extend_info->port = ntohs(get_uint16(buf+v3_shift+5)); - memcpy(extend_info->identity_digest, buf+v3_shift+7, - DIGEST_LEN); - extend_info->nickname[0] = '$'; - base16_encode(extend_info->nickname+1, sizeof(extend_info->nickname)-1, - extend_info->identity_digest, DIGEST_LEN); - - klen = ntohs(get_uint16(buf+v3_shift+7+DIGEST_LEN)); - if ((int)len != v3_shift+7+DIGEST_LEN+2+klen+20+128) { - log_warn(LD_PROTOCOL, "Bad length %u for version %d INTRODUCE2 cell.", - (int)len, *buf); - reason = END_CIRC_REASON_TORPROTOCOL; - goto err; - } - extend_info->onion_key = - crypto_pk_asn1_decode(buf+v3_shift+7+DIGEST_LEN+2, klen); - if (!extend_info->onion_key) { - log_warn(LD_PROTOCOL, "Error decoding onion key in version %d " - "INTRODUCE2 cell.", *buf); - reason = END_CIRC_REASON_TORPROTOCOL; - goto err; - } - ptr = buf+v3_shift+7+DIGEST_LEN+2+klen; - len -= v3_shift+7+DIGEST_LEN+2+klen; - } else { - char *rp_nickname; - size_t nickname_field_len; - const node_t *node; - int version; - if (*buf == 1) { - rp_nickname = buf+1; - nickname_field_len = MAX_HEX_NICKNAME_LEN+1; - version = 1; - } else { - nickname_field_len = MAX_NICKNAME_LEN+1; - rp_nickname = buf; - version = 0; - } - ptr=memchr(rp_nickname,0,nickname_field_len); - if (!ptr || ptr == rp_nickname) { - log_warn(LD_PROTOCOL, - "Couldn't find a nul-padded nickname in INTRODUCE2 cell."); - return -1; - } - if ((version == 0 && !is_legal_nickname(rp_nickname)) || - (version == 1 && !is_legal_nickname_or_hexdigest(rp_nickname))) { - log_warn(LD_PROTOCOL, "Bad nickname in INTRODUCE2 cell."); - return -1; - } - /* Okay, now we know that a nickname is at the start of the buffer. */ - ptr = rp_nickname+nickname_field_len; - len -= nickname_field_len; - len -= rp_nickname - buf; /* also remove header space used by version, if - * any */ - node = node_get_by_nickname(rp_nickname, 0); - if (!node) { - log_info(LD_REND, "Couldn't find router %s named in introduce2 cell.", - escaped_safe_str_client(rp_nickname)); - /* XXXX Add a no-such-router reason? */ - reason = END_CIRC_REASON_TORPROTOCOL; - goto err; - } - extend_info = extend_info_from_node(node, 0); + stage_descr = "late parsing"; + /* Parse the plaintext */ + result = rend_service_parse_intro_plaintext(parsed_req, &err_msg); + if (result < 0) { + goto log_error; + } else if (err_msg) { + log_info(LD_REND, "%s on circ %d.", err_msg, circuit->_base.n_circ_id); + tor_free(err_msg); } - if (len != REND_COOKIE_LEN+DH_KEY_LEN) { - log_warn(LD_PROTOCOL, "Bad length %u for INTRODUCE2 cell.", (int)len); - reason = END_CIRC_REASON_TORPROTOCOL; - goto err; + stage_descr = "late validation"; + /* Validate the parsed plaintext parts */ + result = rend_service_validate_intro_late(parsed_req, &err_msg); + if (result < 0) { + goto log_error; + } else if (err_msg) { + log_info(LD_REND, "%s on circ %d.", err_msg, circuit->_base.n_circ_id); + tor_free(err_msg); } + stage_descr = NULL; + + /* Increment INTRODUCE2 counter */ + ++(intro_point->accepted_introduce2_count); + + /* Find the rendezvous point */ + rp = find_rp_for_intro(parsed_req, &need_rp_free, &err_msg); + if (!rp) + goto log_error; /* Check if we'd refuse to talk to this router */ if (options->StrictNodes && - routerset_contains_extendinfo(options->ExcludeNodes, extend_info)) { + routerset_contains_extendinfo(options->ExcludeNodes, rp)) { log_warn(LD_REND, "Client asked to rendezvous at a relay that we " "exclude, and StrictNodes is set. Refusing service."); reason = END_CIRC_REASON_INTERNAL; /* XXX might leak why we refused */ goto err; } - r_cookie = ptr; - base16_encode(hexcookie,9,r_cookie,4); - - /* Determine hash of Diffie-Hellman, part 1 to detect replays. */ - digest = crypto_digest_new(); - crypto_digest_add_bytes(digest, ptr+REND_COOKIE_LEN, DH_KEY_LEN); - crypto_digest_get_digest(digest, diffie_hellman_hash, DIGEST_LEN); - crypto_digest_free(digest); + base16_encode(hexcookie, 9, (const char *)(parsed_req->rc), 4); /* Check whether there is a past request with the same Diffie-Hellman, * part 1. */ - access_time = digestmap_get(service->accepted_intro_dh_parts, - diffie_hellman_hash); - if (access_time != NULL) { + replay = replaycache_add_test_and_elapsed( + service->accepted_intro_dh_parts, + parsed_req->dh, DH_KEY_LEN, + &elapsed); + + if (replay) { /* A Tor client will send a new INTRODUCE1 cell with the same rend * cookie and DH public key as its previous one if its intro circ * times out while in state CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT . @@ -1299,21 +1290,10 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, "INTRODUCE2 cell with same first part of " "Diffie-Hellman handshake %d seconds ago. Dropping " "cell.", - (int) (now - *access_time)); + (int) elapsed); goto err; } - /* Add request to access history, including time and hash of Diffie-Hellman, - * part 1, and possibly remove requests from the history that are older than - * one hour. */ - access_time = tor_malloc(sizeof(time_t)); - *access_time = now; - digestmap_set(service->accepted_intro_dh_parts, - diffie_hellman_hash, access_time); - if (service->last_cleaned_accepted_intro_dh_parts + REND_REPLAY_TIME_INTERVAL - < now) - clean_accepted_intro_dh_parts(service, now); - /* If the service performs client authorization, check included auth data. */ if (service->clients) { if (auth_len > 0) { @@ -1341,7 +1321,8 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, reason = END_CIRC_REASON_INTERNAL; goto err; } - if (crypto_dh_compute_secret(LOG_PROTOCOL_WARN, dh, ptr+REND_COOKIE_LEN, + if (crypto_dh_compute_secret(LOG_PROTOCOL_WARN, dh, + (char *)(parsed_req->dh), DH_KEY_LEN, keys, DIGEST_LEN+CPATH_KEY_MATERIAL_LEN)<0) { log_warn(LD_BUG, "Internal error: couldn't complete DH handshake"); @@ -1360,7 +1341,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, int flags = CIRCLAUNCH_NEED_CAPACITY | CIRCLAUNCH_IS_INTERNAL; if (circ_needs_uptime) flags |= CIRCLAUNCH_NEED_UPTIME; launched = circuit_launch_by_extend_info( - CIRCUIT_PURPOSE_S_CONNECT_REND, extend_info, flags); + CIRCUIT_PURPOSE_S_CONNECT_REND, rp, flags); if (launched) break; @@ -1368,7 +1349,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, if (!launched) { /* give up */ log_warn(LD_REND, "Giving up launching first hop of circuit to rendezvous " "point %s for service %s.", - safe_str_client(extend_info_describe(extend_info)), + safe_str_client(extend_info_describe(rp)), serviceid); reason = END_CIRC_REASON_CONNECTFAILED; goto err; @@ -1376,7 +1357,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, log_info(LD_REND, "Accepted intro; launching circuit to %s " "(cookie %s) for service %s.", - safe_str_client(extend_info_describe(extend_info)), + safe_str_client(extend_info_describe(rp)), hexcookie, serviceid); tor_assert(launched->build_state); /* Fill in the circuit's state. */ @@ -1384,7 +1365,7 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, memcpy(launched->rend_data->rend_pk_digest, circuit->rend_data->rend_pk_digest, DIGEST_LEN); - memcpy(launched->rend_data->rend_cookie, r_cookie, REND_COOKIE_LEN); + memcpy(launched->rend_data->rend_cookie, parsed_req->rc, REND_COOKIE_LEN); strlcpy(launched->rend_data->onion_address, service->service_id, sizeof(launched->rend_data->onion_address)); @@ -1402,19 +1383,878 @@ rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, if (circuit_init_cpath_crypto(cpath,keys+DIGEST_LEN,1)<0) goto err; memcpy(cpath->handshake_digest, keys, DIGEST_LEN); - if (extend_info) extend_info_free(extend_info); - memset(keys, 0, sizeof(keys)); - return 0; + goto done; + + log_error: + if (!err_msg) { + if (stage_descr) { + tor_asprintf(&err_msg, + "unknown %s error for INTRODUCE2", stage_descr); + } else { + err_msg = tor_strdup("unknown error for INTRODUCE2"); + } + } + + log_warn(LD_REND, "%s on circ %d", err_msg, circuit->_base.n_circ_id); err: - memset(keys, 0, sizeof(keys)); + status = -1; if (dh) crypto_dh_free(dh); - if (launched) + if (launched) { circuit_mark_for_close(TO_CIRCUIT(launched), reason); - if (extend_info) extend_info_free(extend_info); + } + tor_free(err_msg); + + done: + memset(keys, 0, sizeof(keys)); + memset(buf, 0, sizeof(buf)); + memset(serviceid, 0, sizeof(serviceid)); + memset(hexcookie, 0, sizeof(hexcookie)); + memset(intro_key_digest, 0, sizeof(intro_key_digest)); + memset(auth_data, 0, sizeof(auth_data)); + memset(diffie_hellman_hash, 0, sizeof(diffie_hellman_hash)); + + /* Free the parsed cell */ + if (parsed_req) { + rend_service_free_intro(parsed_req); + parsed_req = NULL; + } + + /* Free rp if we must */ + if (need_rp_free) extend_info_free(rp); + + return status; +} + +/** Given a parsed and decrypted INTRODUCE2, find the rendezvous point or + * return NULL and an error string if we can't. + */ + +static extend_info_t * +find_rp_for_intro(const rend_intro_cell_t *intro, + uint8_t *need_free_out, char **err_msg_out) +{ + extend_info_t *rp = NULL; + char *err_msg = NULL; + const char *rp_nickname = NULL; + const node_t *node = NULL; + uint8_t need_free = 0; + + if (!intro || !need_free_out) { + if (err_msg_out) + err_msg = tor_strdup("Bad parameters to find_rp_for_intro()"); + + goto err; + } + + if (intro->version == 0 || intro->version == 1) { + if (intro->version == 1) rp_nickname = (const char *)(intro->u.v1.rp); + else rp_nickname = (const char *)(intro->u.v0.rp); + + node = node_get_by_nickname(rp_nickname, 0); + if (!node) { + if (err_msg_out) { + tor_asprintf(&err_msg, + "Couldn't find router %s named in INTRODUCE2 cell", + escaped_safe_str_client(rp_nickname)); + } + + goto err; + } + + rp = extend_info_from_node(node, 0); + if (!rp) { + if (err_msg_out) { + tor_asprintf(&err_msg, + "Could build extend_info_t for router %s named " + "in INTRODUCE2 cell", + escaped_safe_str_client(rp_nickname)); + } + + goto err; + } else { + need_free = 1; + } + } else if (intro->version == 2) { + rp = intro->u.v2.extend_info; + } else if (intro->version == 3) { + rp = intro->u.v3.extend_info; + } else { + if (err_msg_out) { + tor_asprintf(&err_msg, + "Unknown version %d in INTRODUCE2 cell", + (int)(intro->version)); + } + + goto err; + } + + goto done; + + err: + if (err_msg_out) *err_msg_out = err_msg; + else tor_free(err_msg); + + done: + if (rp && need_free_out) *need_free_out = need_free; + + return rp; +} + +/** Remove unnecessary parts from a rend_intro_cell_t - the ciphertext if + * already decrypted, the plaintext too if already parsed + */ + +void +rend_service_compact_intro(rend_intro_cell_t *request) +{ + if (!request) return; + + if ((request->plaintext && request->plaintext_len > 0) || + request->parsed) { + tor_free(request->ciphertext); + request->ciphertext_len = 0; + } + + if (request->parsed) { + tor_free(request->plaintext); + request->plaintext_len = 0; + } +} + +/** Free a parsed INTRODUCE1 or INTRODUCE2 cell that was allocated by + * rend_service_parse_intro(). + */ +void +rend_service_free_intro(rend_intro_cell_t *request) +{ + if (!request) { + log_info(LD_BUG, "rend_service_free_intro() called with NULL request!"); + return; + } + + /* Free ciphertext */ + tor_free(request->ciphertext); + request->ciphertext_len = 0; + + /* Have plaintext? */ + if (request->plaintext) { + /* Zero it out just to be safe */ + memset(request->plaintext, 0, request->plaintext_len); + tor_free(request->plaintext); + request->plaintext_len = 0; + } + + /* Have parsed plaintext? */ + if (request->parsed) { + switch (request->version) { + case 0: + case 1: + /* + * Nothing more to do; these formats have no further pointers + * in them. + */ + break; + case 2: + extend_info_free(request->u.v2.extend_info); + request->u.v2.extend_info = NULL; + break; + case 3: + if (request->u.v3.auth_data) { + memset(request->u.v3.auth_data, 0, request->u.v3.auth_len); + tor_free(request->u.v3.auth_data); + } + + extend_info_free(request->u.v3.extend_info); + request->u.v3.extend_info = NULL; + break; + default: + log_info(LD_BUG, + "rend_service_free_intro() saw unknown protocol " + "version %d.", + request->version); + } + } + + /* Zero it out to make sure sensitive stuff doesn't hang around in memory */ + memset(request, 0, sizeof(*request)); + + tor_free(request); +} + +/** Parse an INTRODUCE1 or INTRODUCE2 cell into a newly allocated + * rend_intro_cell_t structure. Free it with rend_service_free_intro() + * when finished. The type parameter should be 1 or 2 to indicate whether + * this is INTRODUCE1 or INTRODUCE2. This parses only the non-encrypted + * parts; after this, call rend_service_decrypt_intro() with a key, then + * rend_service_parse_intro_plaintext() to finish parsing. The optional + * err_msg_out parameter is set to a string suitable for log output + * if parsing fails. This function does some validation, but only + * that which depends solely on the contents of the cell and the + * key; it can be unit-tested. Further validation is done in + * rend_service_validate_intro(). + */ + +rend_intro_cell_t * +rend_service_begin_parse_intro(const uint8_t *request, + size_t request_len, + uint8_t type, + char **err_msg_out) +{ + rend_intro_cell_t *rv = NULL; + char *err_msg = NULL; + + if (!request || request_len <= 0) goto err; + if (!(type == 1 || type == 2)) goto err; + + /* First, check that the cell is long enough to be a sensible INTRODUCE */ + + /* min key length plus digest length plus nickname length */ + if (request_len < + (DIGEST_LEN + REND_COOKIE_LEN + (MAX_NICKNAME_LEN + 1) + + DH_KEY_LEN + 42)) { + if (err_msg_out) { + tor_asprintf(&err_msg, + "got a truncated INTRODUCE%d cell", + (int)type); + } + goto err; + } + + /* Allocate a new parsed cell structure */ + rv = tor_malloc_zero(sizeof(*rv)); + + /* Set the type */ + rv->type = type; + + /* Copy in the ID */ + memcpy(rv->pk, request, DIGEST_LEN); + + /* Copy in the ciphertext */ + rv->ciphertext = tor_malloc(request_len - DIGEST_LEN); + memcpy(rv->ciphertext, request + DIGEST_LEN, request_len - DIGEST_LEN); + rv->ciphertext_len = request_len - DIGEST_LEN; + + goto done; + + err: + if (rv) rend_service_free_intro(rv); + rv = NULL; + if (err_msg_out && !err_msg) { + tor_asprintf(&err_msg, + "unknown INTRODUCE%d error", + (int)type); + } + + done: + if (err_msg_out) *err_msg_out = err_msg; + else tor_free(err_msg); + + return rv; +} + +/** Parse the version-specific parts of a v0 or v1 INTRODUCE1 or INTRODUCE2 + * cell + */ + +static ssize_t +rend_service_parse_intro_for_v0_or_v1( + rend_intro_cell_t *intro, + const uint8_t *buf, + size_t plaintext_len, + char **err_msg_out) +{ + const char *rp_nickname, *endptr; + size_t nickname_field_len, ver_specific_len; + + if (intro->version == 1) { + ver_specific_len = MAX_HEX_NICKNAME_LEN + 2; + rp_nickname = ((const char *)buf) + 1; + nickname_field_len = MAX_HEX_NICKNAME_LEN + 1; + } else if (intro->version == 0) { + ver_specific_len = MAX_NICKNAME_LEN + 1; + rp_nickname = (const char *)buf; + nickname_field_len = MAX_NICKNAME_LEN + 1; + } else { + if (err_msg_out) + tor_asprintf(err_msg_out, + "rend_service_parse_intro_for_v0_or_v1() called with " + "bad version %d on INTRODUCE%d cell (this is a bug)", + intro->version, + (int)(intro->type)); + goto err; + } + + if (plaintext_len < ver_specific_len) { + if (err_msg_out) + tor_asprintf(err_msg_out, + "short plaintext of encrypted part in v1 INTRODUCE%d " + "cell (%lu bytes, needed %lu)", + (int)(intro->type), + (unsigned long)plaintext_len, + (unsigned long)ver_specific_len); + goto err; + } + + endptr = memchr(rp_nickname, 0, nickname_field_len); + if (!endptr || endptr == rp_nickname) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "couldn't find a nul-padded nickname in " + "INTRODUCE%d cell", + (int)(intro->type)); + } + goto err; + } + + if ((intro->version == 0 && + !is_legal_nickname(rp_nickname)) || + (intro->version == 1 && + !is_legal_nickname_or_hexdigest(rp_nickname))) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "bad nickname in INTRODUCE%d cell", + (int)(intro->type)); + } + goto err; + } + + if (intro->version == 1) { + memcpy(intro->u.v1.rp, rp_nickname, endptr - rp_nickname + 1); + } else { + memcpy(intro->u.v0.rp, rp_nickname, endptr - rp_nickname + 1); + } + + return ver_specific_len; + + err: + return -1; +} + +/** Parse the version-specific parts of a v2 INTRODUCE1 or INTRODUCE2 cell + */ + +static ssize_t +rend_service_parse_intro_for_v2( + rend_intro_cell_t *intro, + const uint8_t *buf, + size_t plaintext_len, + char **err_msg_out) +{ + unsigned int klen; + extend_info_t *extend_info = NULL; + ssize_t ver_specific_len; + + /* + * We accept version 3 too so that the v3 parser can call this with + * and adjusted buffer for the latter part of a v3 cell, which is + * identical to a v2 cell. + */ + if (!(intro->version == 2 || + intro->version == 3)) { + if (err_msg_out) + tor_asprintf(err_msg_out, + "rend_service_parse_intro_for_v2() called with " + "bad version %d on INTRODUCE%d cell (this is a bug)", + intro->version, + (int)(intro->type)); + goto err; + } + + /* 7 == version, IP and port, DIGEST_LEN == id, 2 == key length */ + if (plaintext_len < 7 + DIGEST_LEN + 2) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "truncated plaintext of encrypted parted of " + "version %d INTRODUCE%d cell", + intro->version, + (int)(intro->type)); + } + + goto err; + } + + extend_info = tor_malloc_zero(sizeof(extend_info_t)); + tor_addr_from_ipv4n(&extend_info->addr, get_uint32(buf + 1)); + extend_info->port = ntohs(get_uint16(buf + 5)); + memcpy(extend_info->identity_digest, buf + 7, DIGEST_LEN); + extend_info->nickname[0] = '$'; + base16_encode(extend_info->nickname + 1, sizeof(extend_info->nickname) - 1, + extend_info->identity_digest, DIGEST_LEN); + klen = ntohs(get_uint16(buf + 7 + DIGEST_LEN)); + + /* 7 == version, IP and port, DIGEST_LEN == id, 2 == key length */ + if (plaintext_len < 7 + DIGEST_LEN + 2 + klen) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "truncated plaintext of encrypted parted of " + "version %d INTRODUCE%d cell", + intro->version, + (int)(intro->type)); + } + + goto err; + } + + extend_info->onion_key = + crypto_pk_asn1_decode((const char *)(buf + 7 + DIGEST_LEN + 2), klen); + if (!extend_info->onion_key) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "error decoding onion key in version %d " + "INTRODUCE%d cell", + intro->version, + (intro->type)); + } + + goto err; + } + + ver_specific_len = 7+DIGEST_LEN+2+klen; + + if (intro->version == 2) intro->u.v2.extend_info = extend_info; + else intro->u.v3.extend_info = extend_info; + + return ver_specific_len; + + err: + extend_info_free(extend_info); + + return -1; +} + +/** Parse the version-specific parts of a v3 INTRODUCE1 or INTRODUCE2 cell + */ + +static ssize_t +rend_service_parse_intro_for_v3( + rend_intro_cell_t *intro, + const uint8_t *buf, + size_t plaintext_len, + char **err_msg_out) +{ + ssize_t adjust, v2_ver_specific_len, ts_offset; + + /* This should only be called on v3 cells */ + if (intro->version != 3) { + if (err_msg_out) + tor_asprintf(err_msg_out, + "rend_service_parse_intro_for_v3() called with " + "bad version %d on INTRODUCE%d cell (this is a bug)", + intro->version, + (int)(intro->type)); + goto err; + } + + /* + * Check that we have at least enough to get auth_len: + * + * 1 octet for version, 1 for auth_type, 2 for auth_len + */ + if (plaintext_len < 4) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "truncated plaintext of encrypted parted of " + "version %d INTRODUCE%d cell", + intro->version, + (int)(intro->type)); + } + + goto err; + } + + /* + * The rend_client_send_introduction() function over in rendclient.c is + * broken (i.e., fails to match the spec) in such a way that we can't + * change it without breaking the protocol. Specifically, it doesn't + * emit auth_len when auth-type is REND_NO_AUTH, so everything is off + * by two bytes after that. Calculate ts_offset and do everything from + * the timestamp on relative to that to handle this dain bramage. + */ + + intro->u.v3.auth_type = buf[1]; + if (intro->u.v3.auth_type != REND_NO_AUTH) { + intro->u.v3.auth_len = ntohs(get_uint16(buf + 2)); + ts_offset = 4 + intro->u.v3.auth_len; + } else { + intro->u.v3.auth_len = 0; + ts_offset = 2; + } + + /* Check that auth len makes sense for this auth type */ + if (intro->u.v3.auth_type == REND_BASIC_AUTH || + intro->u.v3.auth_type == REND_STEALTH_AUTH) { + if (intro->u.v3.auth_len != REND_DESC_COOKIE_LEN) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "wrong auth data size %d for INTRODUCE%d cell, " + "should be %d", + (int)(intro->u.v3.auth_len), + (int)(intro->type), + REND_DESC_COOKIE_LEN); + } + + goto err; + } + } + + /* Check that we actually have everything up to the timestamp */ + if (plaintext_len < (size_t)(ts_offset)) { + if (err_msg_out) { + tor_asprintf(err_msg_out, + "truncated plaintext of encrypted parted of " + "version %d INTRODUCE%d cell", + intro->version, + (int)(intro->type)); + } + + goto err; + } + + if (intro->u.v3.auth_type != REND_NO_AUTH && + intro->u.v3.auth_len > 0) { + /* Okay, we can go ahead and copy auth_data */ + intro->u.v3.auth_data = tor_malloc(intro->u.v3.auth_len); + /* + * We know we had an auth_len field in this case, so 4 is + * always right. + */ + memcpy(intro->u.v3.auth_data, buf + 4, intro->u.v3.auth_len); + } + + /* + * Apparently we don't use the timestamp any more, but might as well copy + * over just in case we ever care about it. + */ + intro->u.v3.timestamp = ntohl(get_uint32(buf + ts_offset)); + + /* + * From here on, the format is as in v2, so we call the v2 parser with + * adjusted buffer and length. We are 4 + ts_offset octets in, but the + * v2 parser expects to skip over a version byte at the start, so we + * adjust by 3 + ts_offset. + */ + adjust = 3 + ts_offset; + + v2_ver_specific_len = + rend_service_parse_intro_for_v2(intro, + buf + adjust, plaintext_len - adjust, + err_msg_out); + + /* Success in v2 parser */ + if (v2_ver_specific_len >= 0) return v2_ver_specific_len + adjust; + /* Failure in v2 parser; it will have provided an err_msg */ + else return v2_ver_specific_len; + + err: return -1; } +/** Table of parser functions for version-specific parts of an INTRODUCE2 + * cell. + */ + +static ssize_t + (*intro_version_handlers[])( + rend_intro_cell_t *, + const uint8_t *, + size_t, + char **) = +{ rend_service_parse_intro_for_v0_or_v1, + rend_service_parse_intro_for_v0_or_v1, + rend_service_parse_intro_for_v2, + rend_service_parse_intro_for_v3 }; + +/** Decrypt the encrypted part of an INTRODUCE1 or INTRODUCE2 cell, + * return 0 if successful, or < 0 and write an error message to + * *err_msg_out if provided. + */ + +int +rend_service_decrypt_intro( + rend_intro_cell_t *intro, + crypto_pk_t *key, + char **err_msg_out) +{ + char *err_msg = NULL; + uint8_t key_digest[DIGEST_LEN]; + char service_id[REND_SERVICE_ID_LEN_BASE32+1]; + ssize_t key_len; + uint8_t buf[RELAY_PAYLOAD_SIZE]; + int result, status = 0; + + if (!intro || !key) { + if (err_msg_out) { + err_msg = + tor_strdup("rend_service_decrypt_intro() called with bad " + "parameters"); + } + + status = -2; + goto err; + } + + /* Make sure we have ciphertext */ + if (!(intro->ciphertext) || intro->ciphertext_len <= 0) { + if (err_msg_out) { + tor_asprintf(&err_msg, + "rend_intro_cell_t was missing ciphertext for " + "INTRODUCE%d cell", + (int)(intro->type)); + } + status = -3; + goto err; + } + + /* Check that this cell actually matches this service key */ + + /* first DIGEST_LEN bytes of request is intro or service pk digest */ + crypto_pk_get_digest(key, (char *)key_digest); + if (tor_memneq(key_digest, intro->pk, DIGEST_LEN)) { + if (err_msg_out) { + base32_encode(service_id, REND_SERVICE_ID_LEN_BASE32 + 1, + (char*)(intro->pk), REND_SERVICE_ID_LEN); + tor_asprintf(&err_msg, + "got an INTRODUCE%d cell for the wrong service (%s)", + (int)(intro->type), + escaped(service_id)); + } + + status = -4; + goto err; + } + + /* Make sure the encrypted part is long enough to decrypt */ + + key_len = crypto_pk_keysize(key); + if (intro->ciphertext_len < key_len) { + if (err_msg_out) { + tor_asprintf(&err_msg, + "got an INTRODUCE%d cell with a truncated PK-encrypted " + "part", + (int)(intro->type)); + } + + status = -5; + goto err; + } + + /* Decrypt the encrypted part */ + + note_crypto_pk_op(REND_SERVER); + result = + crypto_pk_private_hybrid_decrypt( + key, (char *)buf, sizeof(buf), + (const char *)(intro->ciphertext), intro->ciphertext_len, + PK_PKCS1_OAEP_PADDING, 1); + if (result < 0) { + if (err_msg_out) { + tor_asprintf(&err_msg, + "couldn't decrypt INTRODUCE%d cell", + (int)(intro->type)); + } + status = -6; + goto err; + } + intro->plaintext_len = result; + intro->plaintext = tor_malloc(intro->plaintext_len); + memcpy(intro->plaintext, buf, intro->plaintext_len); + + goto done; + + err: + if (err_msg_out && !err_msg) { + tor_asprintf(&err_msg, + "unknown INTRODUCE%d error decrypting encrypted part", + (int)(intro->type)); + } + if (status >= 0) status = -1; + + done: + if (err_msg_out) *err_msg_out = err_msg; + else tor_free(err_msg); + + /* clean up potentially sensitive material */ + memset(buf, 0, sizeof(buf)); + memset(key_digest, 0, sizeof(key_digest)); + memset(service_id, 0, sizeof(service_id)); + + return status; +} + +/** Parse the plaintext of the encrypted part of an INTRODUCE1 or + * INTRODUCE2 cell, return 0 if successful, or < 0 and write an error + * message to *err_msg_out if provided. + */ + +int +rend_service_parse_intro_plaintext( + rend_intro_cell_t *intro, + char **err_msg_out) +{ + char *err_msg = NULL; + ssize_t ver_specific_len, ver_invariant_len; + uint8_t version; + int status = 0; + + if (!intro) { + if (err_msg_out) { + err_msg = + tor_strdup("rend_service_parse_intro_plaintext() called with NULL " + "rend_intro_cell_t"); + } + + status = -2; + goto err; + } + + /* Check that we have plaintext */ + if (!(intro->plaintext) || intro->plaintext_len <= 0) { + if (err_msg_out) { + err_msg = tor_strdup("rend_intro_cell_t was missing plaintext"); + } + status = -3; + goto err; + } + + /* In all formats except v0, the first byte is a version number */ + version = intro->plaintext[0]; + + /* v0 has no version byte (stupid...), so handle it as a fallback */ + if (version > 3) version = 0; + + /* Copy the version into the parsed cell structure */ + intro->version = version; + + /* Call the version-specific parser from the table */ + ver_specific_len = + intro_version_handlers[version](intro, + intro->plaintext, intro->plaintext_len, + &err_msg); + if (ver_specific_len < 0) { + status = -4; + goto err; + } + + /** The rendezvous cookie and Diffie-Hellman stuff are version-invariant + * and at the end of the plaintext of the encrypted part of the cell. + */ + + ver_invariant_len = intro->plaintext_len - ver_specific_len; + if (ver_invariant_len < REND_COOKIE_LEN + DH_KEY_LEN) { + tor_asprintf(&err_msg, + "decrypted plaintext of INTRODUCE%d cell was truncated (%ld bytes)", + (int)(intro->type), + (long)(intro->plaintext_len)); + status = -5; + goto err; + } else if (ver_invariant_len > REND_COOKIE_LEN + DH_KEY_LEN) { + tor_asprintf(&err_msg, + "decrypted plaintext of INTRODUCE%d cell was too long (%ld bytes)", + (int)(intro->type), + (long)(intro->plaintext_len)); + status = -6; + } else { + memcpy(intro->rc, + intro->plaintext + ver_specific_len, + REND_COOKIE_LEN); + memcpy(intro->dh, + intro->plaintext + ver_specific_len + REND_COOKIE_LEN, + DH_KEY_LEN); + } + + /* Flag it as being fully parsed */ + intro->parsed = 1; + + goto done; + + err: + if (err_msg_out && !err_msg) { + tor_asprintf(&err_msg, + "unknown INTRODUCE%d error parsing encrypted part", + (int)(intro->type)); + } + if (status >= 0) status = -1; + + done: + if (err_msg_out) *err_msg_out = err_msg; + else tor_free(err_msg); + + return status; +} + +/** Do validity checks on a parsed intro cell before decryption; some of + * these are not done in rend_service_begin_parse_intro() itself because + * they depend on a lot of other state and would make it hard to unit test. + * Returns >= 0 if successful or < 0 if the intro cell is invalid, and + * optionally writes out an error message for logging. If an err_msg + * pointer is provided, it is the caller's responsibility to free any + * provided message. + */ + +int +rend_service_validate_intro_early(const rend_intro_cell_t *intro, + char **err_msg_out) +{ + int status = 0; + + if (!intro) { + if (err_msg_out) + *err_msg_out = + tor_strdup("NULL intro cell passed to " + "rend_service_validate_intro_early()"); + + status = -1; + goto err; + } + + /* TODO */ + + err: + return status; +} + +/** Do validity checks on a parsed intro cell after decryption; some of + * these are not done in rend_service_parse_intro_plaintext() itself because + * they depend on a lot of other state and would make it hard to unit test. + * Returns >= 0 if successful or < 0 if the intro cell is invalid, and + * optionally writes out an error message for logging. If an err_msg + * pointer is provided, it is the caller's responsibility to free any + * provided message. + */ + +int +rend_service_validate_intro_late(const rend_intro_cell_t *intro, + char **err_msg_out) +{ + int status = 0; + + if (!intro) { + if (err_msg_out) + *err_msg_out = + tor_strdup("NULL intro cell passed to " + "rend_service_validate_intro_late()"); + + status = -1; + goto err; + } + + if (intro->version == 3 && intro->parsed) { + if (!(intro->u.v3.auth_type == REND_NO_AUTH || + intro->u.v3.auth_type == REND_BASIC_AUTH || + intro->u.v3.auth_type == REND_STEALTH_AUTH)) { + /* This is an informative message, not an error, as in the old code */ + if (err_msg_out) + tor_asprintf(err_msg_out, + "unknown authorization type %d", + intro->u.v3.auth_type); + } + } + + err: + return status; +} + /** Called when we fail building a rendezvous circuit at some point other * than the last hop: launches a new circuit to the same rendezvous point. */ @@ -1600,8 +2440,8 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) this case, we might as well close the thing. */ log_info(LD_CIRC|LD_REND, "We have just finished an introduction " "circuit, but we already have enough. Closing it."); - circuit_mark_for_close(TO_CIRCUIT(circuit), END_CIRC_REASON_NONE); - return; + reason = END_CIRC_REASON_NONE; + goto err; } else { tor_assert(circuit->build_state->is_internal); log_info(LD_CIRC|LD_REND, "We have just finished an introduction " @@ -1622,7 +2462,7 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) } circuit_has_opened(circuit); - return; + goto done; } } @@ -1668,9 +2508,16 @@ rend_service_intro_has_opened(origin_circuit_t *circuit) goto err; } - return; + goto done; + err: circuit_mark_for_close(TO_CIRCUIT(circuit), reason); + done: + memset(buf, 0, sizeof(buf)); + memset(auth, 0, sizeof(auth)); + memset(serviceid, 0, sizeof(serviceid)); + + return; } /** Called when we get an INTRO_ESTABLISHED cell; mark the circuit as a @@ -1813,9 +2660,16 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit) /* Change the circuit purpose. */ circuit_change_purpose(TO_CIRCUIT(circuit), CIRCUIT_PURPOSE_S_REND_JOINED); - return; + goto done; + err: circuit_mark_for_close(TO_CIRCUIT(circuit), reason); + done: + memset(buf, 0, sizeof(buf)); + memset(serviceid, 0, sizeof(serviceid)); + memset(hexcookie, 0, sizeof(hexcookie)); + + return; } /* @@ -1929,7 +2783,8 @@ directory_post_to_hs_dir(rend_service_descriptor_t *renddesc, directory_initiate_command_routerstatus(hs_dir, DIR_PURPOSE_UPLOAD_RENDDESC_V2, ROUTER_PURPOSE_GENERAL, - 1, NULL, desc->desc_str, + DIRIND_ANONYMOUS, NULL, + desc->desc_str, strlen(desc->desc_str), 0); base32_encode(desc_id_base32, sizeof(desc_id_base32), desc->desc_id, DIGEST_LEN); @@ -2091,11 +2946,7 @@ upload_service_descriptor(rend_service_t *service) static int intro_point_accepted_intro_count(rend_intro_point_t *intro) { - if (intro->accepted_intro_rsa_parts == NULL) { - return 0; - } else { - return digestmap_size(intro->accepted_intro_rsa_parts); - } + return intro->accepted_introduce2_count; } /** Return non-zero iff <b>intro</b> should 'expire' now (i.e. we diff --git a/src/or/rendservice.h b/src/or/rendservice.h index e5848785a..0d6eddaee 100644 --- a/src/or/rendservice.h +++ b/src/or/rendservice.h @@ -12,9 +12,67 @@ #ifndef _TOR_RENDSERVICE_H #define _TOR_RENDSERVICE_H +#include "or.h" + +typedef struct rend_intro_cell_s rend_intro_cell_t; + +#ifdef RENDSERVICE_PRIVATE + +/* This can be used for both INTRODUCE1 and INTRODUCE2 */ + +struct rend_intro_cell_s { + /* Is this an INTRODUCE1 or INTRODUCE2? (set to 1 or 2) */ + uint8_t type; + /* Public key digest */ + uint8_t pk[DIGEST_LEN]; + /* Optionally, store ciphertext here */ + uint8_t *ciphertext; + ssize_t ciphertext_len; + /* Optionally, store plaintext */ + uint8_t *plaintext; + ssize_t plaintext_len; + /* Have we parsed the plaintext? */ + uint8_t parsed; + /* intro protocol version (0, 1, 2 or 3) */ + uint8_t version; + /* Version-specific parts */ + union { + struct { + /* Rendezvous point nickname */ + uint8_t rp[20]; + } v0; + struct { + /* Rendezvous point nickname or hex-encoded key digest */ + uint8_t rp[42]; + } v1; + struct { + /* The extend_info_t struct has everything v2 uses */ + extend_info_t *extend_info; + } v2; + struct { + /* Auth type used */ + uint8_t auth_type; + /* Length of auth data */ + uint16_t auth_len; + /* Auth data */ + uint8_t *auth_data; + /* timestamp */ + uint32_t timestamp; + /* Rendezvous point's IP address/port, identity digest and onion key */ + extend_info_t *extend_info; + } v3; + } u; + /* Rendezvous cookie */ + uint8_t rc[REND_COOKIE_LEN]; + /* Diffie-Hellman data */ + uint8_t dh[DH_KEY_LEN]; +}; + +#endif + int num_rend_services(void); int rend_config_services(const or_options_t *options, int validate_only); -int rend_service_load_keys(void); +int rend_service_load_all_keys(void); void rend_services_introduce(void); void rend_consider_services_upload(time_t now); void rend_hsdir_routers_changed(void); @@ -27,6 +85,21 @@ int rend_service_intro_established(origin_circuit_t *circuit, void rend_service_rendezvous_has_opened(origin_circuit_t *circuit); int rend_service_introduce(origin_circuit_t *circuit, const uint8_t *request, size_t request_len); +void rend_service_compact_intro(rend_intro_cell_t *request); +int rend_service_decrypt_intro(rend_intro_cell_t *request, + crypto_pk_t *key, + char **err_msg_out); +void rend_service_free_intro(rend_intro_cell_t *request); +rend_intro_cell_t * rend_service_begin_parse_intro(const uint8_t *request, + size_t request_len, + uint8_t type, + char **err_msg_out); +int rend_service_parse_intro_plaintext(rend_intro_cell_t *intro, + char **err_msg_out); +int rend_service_validate_intro_early(const rend_intro_cell_t *intro, + char **err_msg_out); +int rend_service_validate_intro_late(const rend_intro_cell_t *intro, + char **err_msg_out); void rend_service_relaunch_rendezvous(origin_circuit_t *oldcirc); int rend_service_set_connection_addr_port(edge_connection_t *conn, origin_circuit_t *circ); diff --git a/src/or/rephist.c b/src/or/rephist.c index 3b0d9dd35..f9c0b5beb 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -1136,7 +1136,7 @@ rep_hist_load_mtbf_data(time_t now) wfu_timebuf[0] = '\0'; if (format == 1) { - n = sscanf(line, "%40s %ld %lf S=%10s %8s", + n = tor_sscanf(line, "%40s %ld %lf S=%10s %8s", hexbuf, &wrl, &trw, mtbf_timebuf, mtbf_timebuf+11); if (n != 3 && n != 5) { log_warn(LD_HIST, "Couldn't scan line %s", escaped(line)); @@ -1153,7 +1153,7 @@ rep_hist_load_mtbf_data(time_t now) wfu_idx = find_next_with(lines, i+1, "+WFU "); if (mtbf_idx >= 0) { const char *mtbfline = smartlist_get(lines, mtbf_idx); - n = sscanf(mtbfline, "+MTBF %lu %lf S=%10s %8s", + n = tor_sscanf(mtbfline, "+MTBF %lu %lf S=%10s %8s", &wrl, &trw, mtbf_timebuf, mtbf_timebuf+11); if (n == 2 || n == 4) { have_mtbf = 1; @@ -1164,7 +1164,7 @@ rep_hist_load_mtbf_data(time_t now) } if (wfu_idx >= 0) { const char *wfuline = smartlist_get(lines, wfu_idx); - n = sscanf(wfuline, "+WFU %lu %lu S=%10s %8s", + n = tor_sscanf(wfuline, "+WFU %lu %lu S=%10s %8s", &wt_uptime, &total_wt_time, wfu_timebuf, wfu_timebuf+11); if (n == 2 || n == 4) { @@ -1531,7 +1531,7 @@ rep_hist_get_bandwidth_lines(void) const char *desc = NULL; size_t len; - /* opt [dirreq-](read|write)-history yyyy-mm-dd HH:MM:SS (n s) n,n,n... */ + /* [dirreq-](read|write)-history yyyy-mm-dd HH:MM:SS (n s) n,n,n... */ /* The n,n,n part above. Largest representation of a uint64_t is 20 chars * long, plus the comma. */ #define MAX_HIST_VALUE_LEN 21*NUM_TOTALS diff --git a/src/or/replaycache.c b/src/or/replaycache.c new file mode 100644 index 000000000..09104a937 --- /dev/null +++ b/src/or/replaycache.c @@ -0,0 +1,215 @@ + /* Copyright (c) 2012, The Tor Project, Inc. */ + /* See LICENSE for licensing information */ + +/* + * \file replaycache.c + * + * \brief Self-scrubbing replay cache for rendservice.c + */ + +#define REPLAYCACHE_PRIVATE + +#include "or.h" +#include "replaycache.h" + +/** Free the replaycache r and all of its entries. + */ + +void +replaycache_free(replaycache_t *r) +{ + if (!r) { + log_info(LD_BUG, "replaycache_free() called on NULL"); + return; + } + + if (r->digests_seen) digestmap_free(r->digests_seen, _tor_free); + + tor_free(r); +} + +/** Allocate a new, empty replay detection cache, where horizon is the time + * for entries to age out and interval is the time after which the cache + * should be scrubbed for old entries. + */ + +replaycache_t * +replaycache_new(time_t horizon, time_t interval) +{ + replaycache_t *r = NULL; + + if (horizon < 0) { + log_info(LD_BUG, "replaycache_new() called with negative" + " horizon parameter"); + goto err; + } + + if (interval < 0) { + log_info(LD_BUG, "replaycache_new() called with negative interval" + " parameter"); + interval = 0; + } + + r = tor_malloc(sizeof(*r)); + r->scrub_interval = interval; + r->scrubbed = 0; + r->horizon = horizon; + r->digests_seen = digestmap_new(); + + err: + return r; +} + +/** See documentation for replaycache_add_and_test() + */ + +int +replaycache_add_and_test_internal( + time_t present, replaycache_t *r, const void *data, int len, + time_t *elapsed) +{ + int rv = 0; + char digest[DIGEST_LEN]; + time_t *access_time; + + /* sanity check */ + if (present <= 0 || !r || !data || len <= 0) { + log_info(LD_BUG, "replaycache_add_and_test_internal() called with stupid" + " parameters; please fix this."); + goto done; + } + + /* compute digest */ + crypto_digest(digest, (const char *)data, len); + + /* check map */ + access_time = digestmap_get(r->digests_seen, digest); + + /* seen before? */ + if (access_time != NULL) { + /* + * If it's far enough in the past, no hit. If the horizon is zero, we + * never expire. + */ + if (*access_time >= present - r->horizon || r->horizon == 0) { + /* replay cache hit, return 1 */ + rv = 1; + /* If we want to output an elapsed time, do so */ + if (elapsed) { + if (present >= *access_time) { + *elapsed = present - *access_time; + } else { + /* We shouldn't really be seeing hits from the future, but... */ + *elapsed = 0; + } + } + } + /* + * If it's ahead of the cached time, update + */ + if (*access_time < present) { + *access_time = present; + } + } else { + /* No, so no hit and update the digest map with the current time */ + access_time = tor_malloc(sizeof(*access_time)); + *access_time = present; + digestmap_set(r->digests_seen, digest, access_time); + } + + /* now scrub the cache if it's time */ + replaycache_scrub_if_needed_internal(present, r); + + done: + return rv; +} + +/** See documentation for replaycache_scrub_if_needed() + */ + +void +replaycache_scrub_if_needed_internal(time_t present, replaycache_t *r) +{ + digestmap_iter_t *itr = NULL; + const char *digest; + void *valp; + time_t *access_time; + char scrub_this; + + /* sanity check */ + if (!r || !(r->digests_seen)) { + log_info(LD_BUG, "replaycache_scrub_if_needed_internal() called with" + " stupid parameters; please fix this."); + return; + } + + /* scrub time yet? (scrubbed == 0 indicates never scrubbed before) */ + if (present - r->scrubbed < r->scrub_interval && r->scrubbed > 0) return; + + /* if we're never expiring, don't bother scrubbing */ + if (r->horizon == 0) return; + + /* okay, scrub time */ + itr = digestmap_iter_init(r->digests_seen); + while (!digestmap_iter_done(itr)) { + scrub_this = 0; + digestmap_iter_get(itr, &digest, &valp); + access_time = (time_t *)valp; + if (access_time) { + /* aged out yet? */ + if (*access_time < present - r->horizon) scrub_this = 1; + } else { + /* Buh? Get rid of it, anyway */ + log_info(LD_BUG, "replaycache_scrub_if_needed_internal() saw a NULL" + " entry in the digestmap."); + scrub_this = 1; + } + + if (scrub_this) { + /* Advance the iterator and remove this one */ + itr = digestmap_iter_next_rmv(r->digests_seen, itr); + /* Free the value removed */ + tor_free(access_time); + } else { + /* Just advance the iterator */ + itr = digestmap_iter_next(r->digests_seen, itr); + } + } + + /* update scrubbed timestamp */ + if (present > r->scrubbed) r->scrubbed = present; +} + +/** Test the buffer of length len point to by data against the replay cache r; + * the digest of the buffer will be added to the cache at the current time, + * and the function will return 1 if it was already seen within the cache's + * horizon, or 0 otherwise. + */ + +int +replaycache_add_and_test(replaycache_t *r, const void *data, int len) +{ + return replaycache_add_and_test_internal(time(NULL), r, data, len, NULL); +} + +/** Like replaycache_add_and_test(), but if it's a hit also return the time + * elapsed since this digest was last seen. + */ + +int +replaycache_add_test_and_elapsed( + replaycache_t *r, const void *data, int len, time_t *elapsed) +{ + return replaycache_add_and_test_internal(time(NULL), r, data, len, elapsed); +} + +/** Scrub aged entries out of r if sufficiently long has elapsed since r was + * last scrubbed. + */ + +void +replaycache_scrub_if_needed(replaycache_t *r) +{ + replaycache_scrub_if_needed_internal(time(NULL), r); +} + diff --git a/src/or/replaycache.h b/src/or/replaycache.h new file mode 100644 index 000000000..9f3107c51 --- /dev/null +++ b/src/or/replaycache.h @@ -0,0 +1,66 @@ +/* Copyright (c) 2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file replaycache.h + * \brief Header file for replaycache.c. + **/ + +#ifndef _TOR_REPLAYCACHE_H +#define _TOR_REPLAYCACHE_H + +typedef struct replaycache_s replaycache_t; + +#ifdef REPLAYCACHE_PRIVATE + +struct replaycache_s { + /* Scrub interval */ + time_t scrub_interval; + /* Last scrubbed */ + time_t scrubbed; + /* + * Horizon + * (don't return true on digests in the cache but older than this) + */ + time_t horizon; + /* + * Digest map: keys are digests, values are times the digest was last seen + */ + digestmap_t *digests_seen; +}; + +#endif /* REPLAYCACHE_PRIVATE */ + +/* replaycache_t free/new */ + +void replaycache_free(replaycache_t *r); +replaycache_t * replaycache_new(time_t horizon, time_t interval); + +#ifdef REPLAYCACHE_PRIVATE + +/* + * replaycache_t internal functions: + * + * These take the time to treat as the present as an argument for easy unit + * testing. For everything else, use the wrappers below instead. + */ + +int replaycache_add_and_test_internal( + time_t present, replaycache_t *r, const void *data, int len, + time_t *elapsed); +void replaycache_scrub_if_needed_internal( + time_t present, replaycache_t *r); + +#endif /* REPLAYCACHE_PRIVATE */ + +/* + * replaycache_t methods + */ + +int replaycache_add_and_test(replaycache_t *r, const void *data, int len); +int replaycache_add_test_and_elapsed( + replaycache_t *r, const void *data, int len, time_t *elapsed); +void replaycache_scrub_if_needed(replaycache_t *r); + +#endif + diff --git a/src/or/router.c b/src/or/router.c index 38f1cdd49..052ed3807 100644 --- a/src/or/router.c +++ b/src/or/router.c @@ -27,6 +27,9 @@ #include "router.h" #include "routerlist.h" #include "routerparse.h" +#include "statefile.h" +#include "transports.h" +#include "routerset.h" /** * \file router.c @@ -671,7 +674,7 @@ init_keys(void) * we don't really need new keys yet so the descriptor doesn't * change and the old one is still fresh. */ log_info(LD_GENERAL, "Couldn't add own descriptor to directory " - "after key init: %s. This is usually not a problem.", + "after key init: %s This is usually not a problem.", m?m:"<unknown error>"); } } @@ -879,6 +882,21 @@ decide_to_advertise_dirport(const or_options_t *options, uint16_t dir_port) return advertising ? dir_port : 0; } +/** Allocate and return a new extend_info_t that can be used to build + * a circuit to or through the router <b>r</b>. Use the primary + * address of the router unless <b>for_direct_connect</b> is true, in + * which case the preferred address is used instead. */ +static extend_info_t * +extend_info_from_router(const routerinfo_t *r) +{ + tor_addr_port_t ap; + tor_assert(r); + + router_get_prim_orport(r, &ap); + return extend_info_new(r->nickname, r->cache_info.identity_digest, + r->onion_pkey, &ap.addr, ap.port); +} + /** Some time has passed, or we just got new directory information. * See if we currently believe our ORPort or DirPort to be * unreachable. If so, launch a new test for it. @@ -920,12 +938,11 @@ consider_testing_reachability(int test_or, int test_dir) } if (test_or && (!orport_reachable || !circuit_enough_testing_circs())) { - extend_info_t *ei; + extend_info_t *ei = extend_info_from_router(me); + /* XXX IPv6 self testing */ log_info(LD_CIRC, "Testing %s of my ORPort: %s:%d.", !orport_reachable ? "reachability" : "bandwidth", me->address, me->or_port); - /* XXX IPv6 self testing IPv6 orports will need pref_addr */ - ei = extend_info_from_router(me, 0); circuit_launch_by_extend_info(CIRCUIT_PURPOSE_TESTING, ei, CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL); extend_info_free(ei); @@ -939,11 +956,10 @@ consider_testing_reachability(int test_or, int test_dir) /* ask myself, via tor, for my server descriptor. */ directory_initiate_command(me->address, &addr, me->or_port, me->dir_port, - 0, /* does not matter */ - 0, me->cache_info.identity_digest, + me->cache_info.identity_digest, DIR_PURPOSE_FETCH_SERVERDESC, ROUTER_PURPOSE_GENERAL, - 1, "authority.z", NULL, 0, 0); + DIRIND_ANON_DIRPORT, "authority.z", NULL, 0, 0); } } @@ -1539,8 +1555,9 @@ router_rebuild_descriptor(int force) ri->cache_info.published_on = time(NULL); ri->onion_pkey = crypto_pk_dup_key(get_onion_key()); /* must invoke from * main thread */ - if (options->BridgeRelay) { - /* For now, only bridges advertise an ipv6 or-address. And only one. */ + + /* For now, at most one IPv6 or-address is being advertised. */ + { const port_cfg_t *ipv6_orport = NULL; SMARTLIST_FOREACH_BEGIN(get_configured_ports(), const port_cfg_t *, p) { if (p->type == CONN_TYPE_OR_LISTENER && @@ -1565,6 +1582,7 @@ router_rebuild_descriptor(int force) ri->ipv6_orport = ipv6_orport->port; } } + ri->identity_pkey = crypto_pk_dup_key(get_server_identity_key()); if (crypto_pk_get_digest(ri->identity_pkey, ri->cache_info.identity_digest)<0) { @@ -2054,9 +2072,9 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, "router %s %s %d 0 %d\n" "%s" "platform %s\n" - "opt protocols Link 1 2 Circuit 1\n" + "protocols Link 1 2 Circuit 1\n" "published %s\n" - "opt fingerprint %s\n" + "fingerprint %s\n" "uptime %ld\n" "bandwidth %d %d %d\n" "%s%s%s%s" @@ -2075,15 +2093,15 @@ router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, (int) router->bandwidthrate, (int) router->bandwidthburst, (int) router->bandwidthcapacity, - has_extra_info_digest ? "opt extra-info-digest " : "", + has_extra_info_digest ? "extra-info-digest " : "", has_extra_info_digest ? extra_info_digest : "", has_extra_info_digest ? "\n" : "", - options->DownloadExtraInfo ? "opt caches-extra-info\n" : "", + options->DownloadExtraInfo ? "caches-extra-info\n" : "", onion_pkey, identity_pkey, family_line, - we_are_hibernating() ? "opt hibernating 1\n" : "", - options->HidServDirectoryV2 ? "opt hidden-service-dir\n" : "", - options->AllowSingleHopExits ? "opt allow-single-hop-exits\n" : ""); + we_are_hibernating() ? "hibernating 1\n" : "", + options->HidServDirectoryV2 ? "hidden-service-dir\n" : "", + options->AllowSingleHopExits ? "allow-single-hop-exits\n" : ""); tor_free(family_line); tor_free(onion_pkey); @@ -2194,40 +2212,24 @@ router_get_prim_orport(const routerinfo_t *router, tor_addr_port_t *ap_out) ap_out->port = router->or_port; } -/** Return 1 if we prefer the IPv6 address and OR TCP port of - * <b>router</b>, else 0. - * - * We prefer the IPv6 address if the router has one and - * i) the routerinfo_t says so - * or - * ii) the router has no IPv4 address. */ +/** Return 1 if any of <b>router</b>'s addresses are <b>addr</b>. + * Otherwise return 0. */ int -router_ipv6_preferred(const routerinfo_t *router) +router_has_addr(const routerinfo_t *router, const tor_addr_t *addr) { - return (!tor_addr_is_null(&router->ipv6_addr) - && (router->ipv6_preferred || router->addr == 0)); -} - -/** Copy the preferred OR port (IP address and TCP port) for - * <b>router</b> into *<b>addr_out</b>. */ -void -router_get_pref_orport(const routerinfo_t *router, tor_addr_port_t *ap_out) -{ - if (router_ipv6_preferred(router)) - router_get_pref_ipv6_orport(router, ap_out); - else - router_get_prim_orport(router, ap_out); + return + tor_addr_eq_ipv4h(addr, router->addr) || + tor_addr_eq(&router->ipv6_addr, addr); } -/** Copy the preferred IPv6 OR port (IP address and TCP port) for - * <b>router</b> into *<b>ap_out</b>. */ -void -router_get_pref_ipv6_orport(const routerinfo_t *router, - tor_addr_port_t *ap_out) +int +router_has_orport(const routerinfo_t *router, const tor_addr_port_t *orport) { - tor_assert(ap_out != NULL); - tor_addr_copy(&ap_out->addr, &router->ipv6_addr); - ap_out->port = router->ipv6_orport; + return + (tor_addr_eq_ipv4h(&orport->addr, router->addr) && + orport->port == router->or_port) || + (tor_addr_eq(&orport->addr, &router->ipv6_addr) && + orport->port == router->ipv6_orport); } /** Load the contents of <b>filename</b>, find the last line starting with @@ -2345,6 +2347,13 @@ extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, } } + /* Add information about the pluggable transports we support. */ + if (options->ServerTransportPlugin) { + char *pluggable_transports = pt_get_extra_info_descriptor_string(); + if (pluggable_transports) + smartlist_add(chunks, pluggable_transports); + } + if (should_record_bridge_info(options) && write_stats_to_extrainfo) { const char *bridge_stats = geoip_get_bridge_stats_extrainfo(now); if (bridge_stats) { @@ -2754,3 +2763,30 @@ router_free_all(void) } } +/** Return a smartlist of tor_addr_port_t's with all the OR ports of + <b>ri</b>. Note that freeing of the items in the list as well as + the smartlist itself is the callers responsibility. + + XXX duplicating code from node_get_all_orports(). */ +smartlist_t * +router_get_all_orports(const routerinfo_t *ri) +{ + smartlist_t *sl = smartlist_new(); + tor_assert(ri); + + if (ri->addr != 0) { + tor_addr_port_t *ap = tor_malloc(sizeof(tor_addr_port_t)); + tor_addr_from_ipv4h(&ap->addr, ri->addr); + ap->port = ri->or_port; + smartlist_add(sl, ap); + } + if (!tor_addr_is_null(&ri->ipv6_addr)) { + tor_addr_port_t *ap = tor_malloc(sizeof(tor_addr_port_t)); + tor_addr_copy(&ap->addr, &ri->ipv6_addr); + ap->port = ri->or_port; + smartlist_add(sl, ap); + } + + return sl; +} + diff --git a/src/or/router.h b/src/or/router.h index 69805d6f2..c43c30849 100644 --- a/src/or/router.h +++ b/src/or/router.h @@ -86,15 +86,13 @@ int router_pick_published_address(const or_options_t *options, uint32_t *addr); int router_rebuild_descriptor(int force); int router_dump_router_to_string(char *s, size_t maxlen, routerinfo_t *router, crypto_pk_t *ident_key); -void router_get_prim_orport(const routerinfo_t *router, - tor_addr_port_t *addr_port_out); -void router_get_pref_orport(const routerinfo_t *router, - tor_addr_port_t *addr_port_out); -void router_get_pref_ipv6_orport(const routerinfo_t *router, - tor_addr_port_t *addr_port_out); -int router_ipv6_preferred(const routerinfo_t *router); int extrainfo_dump_to_string(char **s, extrainfo_t *extrainfo, crypto_pk_t *ident_key); +void router_get_prim_orport(const routerinfo_t *router, + tor_addr_port_t *ap_out); +int router_has_addr(const routerinfo_t *router, const tor_addr_t *addr); +int router_has_orport(const routerinfo_t *router, + const tor_addr_port_t *orport); int is_legal_nickname(const char *s); int is_legal_nickname_or_hexdigest(const char *s); int is_legal_hexdigest(const char *s); @@ -132,6 +130,8 @@ void router_free_all(void); const char *router_purpose_to_string(uint8_t p); uint8_t router_purpose_from_string(const char *s); +smartlist_t *router_get_all_orports(const routerinfo_t *ri); + #ifdef ROUTER_PRIVATE /* Used only by router.c and test.c */ void get_platform_str(char *platform, size_t len); diff --git a/src/or/routerlist.c b/src/or/routerlist.c index 3c39e362d..98357d6a3 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -11,6 +11,7 @@ * servers. **/ +#define ROUTERLIST_PRIVATE #include "or.h" #include "circuitbuild.h" #include "config.h" @@ -33,6 +34,7 @@ #include "router.h" #include "routerlist.h" #include "routerparse.h" +#include "routerset.h" // #define DEBUG_ROUTERLIST @@ -46,11 +48,8 @@ static const routerstatus_t *router_pick_trusteddirserver_impl( static void mark_all_trusteddirservers_up(void); static int router_nickname_matches(const routerinfo_t *router, const char *nickname); -static int node_nickname_matches(const node_t *router, - const char *nickname); static void trusted_dir_server_free(trusted_dir_server_t *ds); static int signed_desc_digest_is_recognized(signed_descriptor_t *desc); -static void update_router_have_minimum_dir_info(void); static const char *signed_descriptor_get_body_impl( const signed_descriptor_t *desc, int with_annotations); @@ -995,7 +994,7 @@ router_get_my_share_of_directory_requests(double *v2_share_out, } } - if (rs->version_supports_v3_dir) { + { sl_last_total_weighted_bw = 0; router_pick_directory_server(V3_DIRINFO, pds_flags); if (sl_last_total_weighted_bw != 0) { @@ -1127,12 +1126,6 @@ router_pick_directory_server_impl(dirinfo_type_t type, int flags) continue; if (requireother && router_digest_is_me(node->identity)) continue; - if (type & V3_DIRINFO) { - if (!(status->version_supports_v3_dir || - router_digest_is_trusted_dir_type(node->identity, - V3_DIRINFO))) - continue; - } is_trusted = router_digest_is_trusted_dir(node->identity); if ((type & V2_DIRINFO) && !(node->rs->is_v2_dir || is_trusted)) continue; @@ -1155,7 +1148,6 @@ router_pick_directory_server_impl(dirinfo_type_t type, int flags) is_overloaded = status->last_dir_503_at + DIR_503_TIMEOUT > now; if (prefer_tunnel && - status->version_supports_begindir && (!fascistfirewall || fascist_firewall_allows_address_or(&addr, status->or_port))) smartlist_add(is_trusted ? trusted_tunnel : @@ -1343,9 +1335,11 @@ mark_all_trusteddirservers_up(void) /** Return true iff r1 and r2 have the same address and OR port. */ int -routers_have_same_or_addr(const routerinfo_t *r1, const routerinfo_t *r2) +routers_have_same_or_addrs(const routerinfo_t *r1, const routerinfo_t *r2) { - return r1->addr == r2->addr && r1->or_port == r2->or_port; + return r1->addr == r2->addr && r1->or_port == r2->or_port && + tor_addr_eq(&r1->ipv6_addr, &r2->ipv6_addr) && + r1->ipv6_orport == r2->ipv6_orport; } /** Reset all internal variables used to count failed downloads of network @@ -1356,88 +1350,6 @@ router_reset_status_download_failures(void) mark_all_trusteddirservers_up(); } -/** Return true iff router1 and router2 have similar enough network addresses - * that we should treat them as being in the same family */ -static INLINE int -addrs_in_same_network_family(const tor_addr_t *a1, - const tor_addr_t *a2) -{ - /* XXXX MOVE ? */ - return 0 == tor_addr_compare_masked(a1, a2, 16, CMP_SEMANTIC); -} - -/** - * Add all the family of <b>node</b>, including <b>node</b> itself, to - * the smartlist <b>sl</b>. - * - * This is used to make sure we don't pick siblings in a single path, or - * pick more than one relay from a family for our entry guard list. - * Note that a node may be added to <b>sl</b> more than once if it is - * part of <b>node</b>'s family for more than one reason. - */ -void -nodelist_add_node_and_family(smartlist_t *sl, const node_t *node) -{ - /* XXXX MOVE */ - const smartlist_t *all_nodes = nodelist_get_list(); - const smartlist_t *declared_family; - const or_options_t *options = get_options(); - - tor_assert(node); - - declared_family = node_get_declared_family(node); - - /* Let's make sure that we have the node itself, if it's a real node. */ - { - const node_t *real_node = node_get_by_id(node->identity); - if (real_node) - smartlist_add(sl, (node_t*)real_node); - } - - /* First, add any nodes with similar network addresses. */ - if (options->EnforceDistinctSubnets) { - tor_addr_t node_addr; - node_get_addr(node, &node_addr); - - SMARTLIST_FOREACH_BEGIN(all_nodes, const node_t *, node2) { - tor_addr_t a; - node_get_addr(node2, &a); - if (addrs_in_same_network_family(&a, &node_addr)) - smartlist_add(sl, (void*)node2); - } SMARTLIST_FOREACH_END(node2); - } - - /* Now, add all nodes in the declared_family of this node, if they - * also declare this node to be in their family. */ - if (declared_family) { - /* Add every r such that router declares familyness with node, and node - * declares familyhood with router. */ - SMARTLIST_FOREACH_BEGIN(declared_family, const char *, name) { - const node_t *node2; - const smartlist_t *family2; - if (!(node2 = node_get_by_nickname(name, 0))) - continue; - if (!(family2 = node_get_declared_family(node2))) - continue; - SMARTLIST_FOREACH_BEGIN(family2, const char *, name2) { - if (node_nickname_matches(node, name2)) { - smartlist_add(sl, (void*)node2); - break; - } - } SMARTLIST_FOREACH_END(name2); - } SMARTLIST_FOREACH_END(name); - } - - /* If the user declared any families locally, honor those too. */ - if (options->NodeFamilySets) { - SMARTLIST_FOREACH(options->NodeFamilySets, const routerset_t *, rs, { - if (routerset_contains_node(rs, node)) { - routerset_get_all_nodes(sl, rs, NULL, 0); - } - }); - } -} - /** Given a <b>router</b>, add every node_t in its family (including the * node itself!) to <b>sl</b>. * @@ -1459,59 +1371,6 @@ routerlist_add_node_and_family(smartlist_t *sl, const routerinfo_t *router) nodelist_add_node_and_family(sl, node); } -/** Return true iff <b>node</b> is named by some nickname in <b>lst</b>. */ -static INLINE int -node_in_nickname_smartlist(const smartlist_t *lst, const node_t *node) -{ - /* XXXX MOVE */ - if (!lst) return 0; - SMARTLIST_FOREACH(lst, const char *, name, { - if (node_nickname_matches(node, name)) - return 1; - }); - return 0; -} - -/** Return true iff r1 and r2 are in the same family, but not the same - * router. */ -int -nodes_in_same_family(const node_t *node1, const node_t *node2) -{ - /* XXXX MOVE */ - const or_options_t *options = get_options(); - - /* Are they in the same family because of their addresses? */ - if (options->EnforceDistinctSubnets) { - tor_addr_t a1, a2; - node_get_addr(node1, &a1); - node_get_addr(node2, &a2); - if (addrs_in_same_network_family(&a1, &a2)) - return 1; - } - - /* Are they in the same family because the agree they are? */ - { - const smartlist_t *f1, *f2; - f1 = node_get_declared_family(node1); - f2 = node_get_declared_family(node2); - if (f1 && f2 && - node_in_nickname_smartlist(f1, node2) && - node_in_nickname_smartlist(f2, node1)) - return 1; - } - - /* Are they in the same option because the user says they are? */ - if (options->NodeFamilySets) { - SMARTLIST_FOREACH(options->NodeFamilySets, const routerset_t *, rs, { - if (routerset_contains_node(rs, node1) && - routerset_contains_node(rs, node2)) - return 1; - }); - } - - return 0; -} - /** Return 1 iff any member of the (possibly NULL) comma-separated list * <b>list</b> is an acceptable nickname or hexdigest for <b>router</b>. Else * return 0. @@ -1575,56 +1434,6 @@ routerlist_find_my_routerinfo(void) return NULL; } -/** Find a router that's up, that has this IP address, and - * that allows exit to this address:port, or return NULL if there - * isn't a good one. - * Don't exit enclave to excluded relays -- it wouldn't actually - * hurt anything, but this way there are fewer confused users. - */ -const node_t * -router_find_exact_exit_enclave(const char *address, uint16_t port) -{/*XXXX MOVE*/ - uint32_t addr; - struct in_addr in; - tor_addr_t a; - const or_options_t *options = get_options(); - - if (!tor_inet_aton(address, &in)) - return NULL; /* it's not an IP already */ - addr = ntohl(in.s_addr); - - tor_addr_from_ipv4h(&a, addr); - - SMARTLIST_FOREACH(nodelist_get_list(), const node_t *, node, { - if (node_get_addr_ipv4h(node) == addr && - node->is_running && - compare_tor_addr_to_node_policy(&a, port, node) == - ADDR_POLICY_ACCEPTED && - !routerset_contains_node(options->_ExcludeExitNodesUnion, node)) - return node; - }); - return NULL; -} - -/** Return 1 if <b>router</b> is not suitable for these parameters, else 0. - * If <b>need_uptime</b> is non-zero, we require a minimum uptime. - * If <b>need_capacity</b> is non-zero, we require a minimum advertised - * bandwidth. - * If <b>need_guard</b>, we require that the router is a possible entry guard. - */ -int -node_is_unreliable(const node_t *node, int need_uptime, - int need_capacity, int need_guard) -{ - if (need_uptime && !node->is_stable) - return 1; - if (need_capacity && !node->is_fast) - return 1; - if (need_guard && !node->is_possible_guard) - return 1; - return 0; -} - /** Return the smaller of the router's configured BandwidthRate * and its advertised capacity. */ uint32_t @@ -1652,6 +1461,92 @@ router_get_advertised_bandwidth_capped(const routerinfo_t *router) return result; } +/** Given an array of double/uint64_t unions that are currently being used as + * doubles, convert them to uint64_t, and try to scale them linearly so as to + * much of the range of uint64_t. If <b>total_out</b> is provided, set it to + * the sum of all elements in the array _before_ scaling. */ +/* private */ void +scale_array_elements_to_u64(u64_dbl_t *entries, int n_entries, + uint64_t *total_out) +{ + double total = 0.0; + double scale_factor; + int i; + /* big, but far away from overflowing an int64_t */ +#define SCALE_TO_U64_MAX (INT64_MAX / 4) + + for (i = 0; i < n_entries; ++i) + total += entries[i].dbl; + + scale_factor = SCALE_TO_U64_MAX / total; + + for (i = 0; i < n_entries; ++i) + entries[i].u64 = tor_llround(entries[i].dbl * scale_factor); + + if (total_out) + *total_out = (uint64_t) total; + +#undef SCALE_TO_U64_MAX +} + +/** Time-invariant 64-bit greater-than; works on two integers in the range + * (0,INT64_MAX). */ +#if SIZEOF_VOID_P == 8 +#define gt_i64_timei(a,b) ((a) > (b)) +#else +static INLINE int +gt_i64_timei(uint64_t a, uint64_t b) +{ + int64_t diff = (int64_t) (b - a); + int res = diff >> 63; + return res & 1; +} +#endif + +/** Pick a random element of <b>n_entries</b>-element array <b>entries</b>, + * choosing each element with a probability proportional to its (uint64_t) + * value, and return the index of that element. If all elements are 0, choose + * an index at random. Return -1 on error. + */ +/* private */ int +choose_array_element_by_weight(const u64_dbl_t *entries, int n_entries) +{ + int i, i_chosen=-1, n_chosen=0; + uint64_t total_so_far = 0; + uint64_t rand_val; + uint64_t total = 0; + + for (i = 0; i < n_entries; ++i) + total += entries[i].u64; + + if (n_entries < 1) + return -1; + + if (total == 0) + return crypto_rand_int(n_entries); + + tor_assert(total < INT64_MAX); + + rand_val = crypto_rand_uint64(total); + + for (i = 0; i < n_entries; ++i) { + total_so_far += entries[i].u64; + if (gt_i64_timei(total_so_far, rand_val)) { + i_chosen = i; + n_chosen++; + /* Set rand_val to INT64_MAX rather than stopping the loop. This way, + * the time we spend in the loop does not leak which element we chose. */ + rand_val = INT64_MAX; + } + } + tor_assert(total_so_far == total); + tor_assert(n_chosen == 1); + tor_assert(i_chosen >= 0); + tor_assert(i_chosen < n_entries); + + return i_chosen; +} + /** When weighting bridges, enforce these values as lower and upper * bound for believable bandwidth, because there is no way for us * to verify a bridge's bandwidth currently. */ @@ -1702,16 +1597,10 @@ smartlist_choose_node_by_bandwidth_weights(smartlist_t *sl, bandwidth_weight_rule_t rule) { int64_t weight_scale; - int64_t rand_bw; double Wg = -1, Wm = -1, We = -1, Wd = -1; double Wgb = -1, Wmb = -1, Web = -1, Wdb = -1; - double weighted_bw = 0, unweighted_bw = 0; - double *bandwidths; - double tmp = 0; - unsigned int i; - unsigned int i_chosen; - unsigned int i_has_been_chosen; - int have_unknown = 0; /* true iff sl contains element not in consensus. */ + uint64_t weighted_bw = 0; + u64_dbl_t *bandwidths; /* Can't choose exit and guard at same time */ tor_assert(rule == NO_WEIGHTING || @@ -1792,7 +1681,7 @@ smartlist_choose_node_by_bandwidth_weights(smartlist_t *sl, Web /= weight_scale; Wdb /= weight_scale; - bandwidths = tor_malloc_zero(sizeof(double)*smartlist_len(sl)); + bandwidths = tor_malloc_zero(sizeof(u64_dbl_t)*smartlist_len(sl)); // Cycle through smartlist and total the bandwidth. SMARTLIST_FOREACH_BEGIN(sl, const node_t *, node) { @@ -1815,7 +1704,6 @@ smartlist_choose_node_by_bandwidth_weights(smartlist_t *sl, } else if (node->ri) { /* bridge or other descriptor not in our consensus */ this_bw = bridge_get_advertised_bandwidth_bounded(node->ri); - have_unknown = 1; } else { /* We can't use this one. */ continue; @@ -1831,72 +1719,32 @@ smartlist_choose_node_by_bandwidth_weights(smartlist_t *sl, } else { // middle weight = (is_dir ? Wmb*Wm : Wm); } - - bandwidths[node_sl_idx] = weight*this_bw; - weighted_bw += weight*this_bw; - unweighted_bw += this_bw; + /* These should be impossible; but overflows here would be bad, so let's + * make sure. */ + if (this_bw < 0) + this_bw = 0; + if (weight < 0.0) + weight = 0.0; + + bandwidths[node_sl_idx].dbl = weight*this_bw + 0.5; if (is_me) - sl_last_weighted_bw_of_me = weight*this_bw; + sl_last_weighted_bw_of_me = (uint64_t) bandwidths[node_sl_idx].dbl; } SMARTLIST_FOREACH_END(node); - /* XXXX this is a kludge to expose these values. */ - sl_last_total_weighted_bw = weighted_bw; - log_debug(LD_CIRC, "Choosing node for rule %s based on weights " - "Wg=%f Wm=%f We=%f Wd=%f with total bw %f", + "Wg=%f Wm=%f We=%f Wd=%f with total bw "U64_FORMAT, bandwidth_weight_rule_to_string(rule), - Wg, Wm, We, Wd, weighted_bw); - - /* If there is no bandwidth, choose at random */ - if (DBL_TO_U64(weighted_bw) == 0) { - /* Don't warn when using bridges/relays not in the consensus */ - if (!have_unknown) { -#define ZERO_BANDWIDTH_WARNING_INTERVAL (15) - static ratelim_t zero_bandwidth_warning_limit = - RATELIM_INIT(ZERO_BANDWIDTH_WARNING_INTERVAL); - char *msg; - if ((msg = rate_limit_log(&zero_bandwidth_warning_limit, - approx_time()))) { - log_warn(LD_CIRC, - "Weighted bandwidth is %f in node selection for rule %s " - "(unweighted was %f) %s", - weighted_bw, bandwidth_weight_rule_to_string(rule), - unweighted_bw, msg); - } - } - tor_free(bandwidths); - return smartlist_choose(sl); - } + Wg, Wm, We, Wd, U64_PRINTF_ARG(weighted_bw)); - rand_bw = crypto_rand_uint64(DBL_TO_U64(weighted_bw)); - rand_bw++; /* crypto_rand_uint64() counts from 0, and we need to count - * from 1 below. See bug 1203 for details. */ - - /* Last, count through sl until we get to the element we picked */ - i_chosen = (unsigned)smartlist_len(sl); - i_has_been_chosen = 0; - tmp = 0.0; - for (i=0; i < (unsigned)smartlist_len(sl); i++) { - tmp += bandwidths[i]; - if (tmp >= rand_bw && !i_has_been_chosen) { - i_chosen = i; - i_has_been_chosen = 1; - } - } - i = i_chosen; - - if (i == (unsigned)smartlist_len(sl)) { - /* This was once possible due to round-off error, but shouldn't be able - * to occur any longer. */ - tor_fragile_assert(); - --i; - log_warn(LD_BUG, "Round-off error in computing bandwidth had an effect on " - " which router we chose. Please tell the developers. " - "%f " U64_FORMAT " %f", tmp, U64_PRINTF_ARG(rand_bw), - weighted_bw); + scale_array_elements_to_u64(bandwidths, smartlist_len(sl), + &sl_last_total_weighted_bw); + + { + int idx = choose_array_element_by_weight(bandwidths, + smartlist_len(sl)); + tor_free(bandwidths); + return idx < 0 ? NULL : smartlist_get(sl, idx); } - tor_free(bandwidths); - return smartlist_get(sl, i); } /** Helper function: @@ -1917,17 +1765,16 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, bandwidth_weight_rule_t rule) { unsigned int i; - unsigned int i_chosen; - unsigned int i_has_been_chosen; - int32_t *bandwidths; + u64_dbl_t *bandwidths; int is_exit; int is_guard; - uint64_t total_nonexit_bw = 0, total_exit_bw = 0, total_bw = 0; - uint64_t total_nonguard_bw = 0, total_guard_bw = 0; - uint64_t rand_bw, tmp; + int is_fast; + double total_nonexit_bw = 0, total_exit_bw = 0; + double total_nonguard_bw = 0, total_guard_bw = 0; double exit_weight; double guard_weight; int n_unknown = 0; + bitarray_t *fast_bits; bitarray_t *exit_bits; bitarray_t *guard_bits; int me_idx = -1; @@ -1951,10 +1798,9 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, } /* First count the total bandwidth weight, and make a list - * of each value. <0 means "unknown; no routerinfo." We use the - * bits of negative values to remember whether the router was fast (-x)&1 - * and whether it was an exit (-x)&2 or guard (-x)&4. Yes, it's a hack. */ - bandwidths = tor_malloc(sizeof(int32_t)*smartlist_len(sl)); + * of each value. We use UINT64_MAX to indicate "unknown". */ + bandwidths = tor_malloc_zero(sizeof(u64_dbl_t)*smartlist_len(sl)); + fast_bits = bitarray_init_zero(smartlist_len(sl)); exit_bits = bitarray_init_zero(smartlist_len(sl)); guard_bits = bitarray_init_zero(smartlist_len(sl)); @@ -1962,7 +1808,6 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, SMARTLIST_FOREACH_BEGIN(sl, const node_t *, node) { /* first, learn what bandwidth we think i has */ int is_known = 1; - int32_t flags = 0; uint32_t this_bw = 0; i = node_sl_idx; @@ -1975,12 +1820,7 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, if (node->rs->has_bandwidth) { this_bw = kb_to_bytes(node->rs->bandwidth); } else { /* guess */ - /* XXX024 once consensuses always list bandwidths, we can take - * this guessing business out. -RD */ is_known = 0; - flags = node->rs->is_fast ? 1 : 0; - flags |= is_exit ? 2 : 0; - flags |= is_guard ? 4 : 0; } } else if (node->ri) { /* Must be a bridge if we're willing to use it */ @@ -1991,12 +1831,11 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, bitarray_set(exit_bits, i); if (is_guard) bitarray_set(guard_bits, i); + if (node->is_fast) + bitarray_set(fast_bits, i); + if (is_known) { - bandwidths[i] = (int32_t) this_bw; - /* Casting this_bw to int32_t is safe because both kb_to_bytes - and bridge_get_advertised_bandwidth_bounded limit it to below - INT32_MAX. */ - tor_assert(bandwidths[i] >= 0); + bandwidths[i].dbl = this_bw; if (is_guard) total_guard_bw += this_bw; else @@ -2007,14 +1846,16 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, total_nonexit_bw += this_bw; } else { ++n_unknown; - bandwidths[node_sl_idx] = -flags; + bandwidths[i].dbl = -1.0; } } SMARTLIST_FOREACH_END(node); +#define EPSILON .1 + /* Now, fill in the unknown values. */ if (n_unknown) { int32_t avg_fast, avg_slow; - if (total_exit_bw+total_nonexit_bw) { + if (total_exit_bw+total_nonexit_bw < EPSILON) { /* if there's some bandwidth, there's at least one known router, * so no worries about div by 0 here */ int n_known = smartlist_len(sl)-n_unknown; @@ -2025,26 +1866,27 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, avg_slow = 20000; } for (i=0; i<(unsigned)smartlist_len(sl); ++i) { - int32_t bw = bandwidths[i]; - if (bw>=0) + if (bandwidths[i].dbl >= 0.0) continue; - is_exit = ((-bw)&2); - is_guard = ((-bw)&4); - bandwidths[i] = ((-bw)&1) ? avg_fast : avg_slow; + is_fast = bitarray_is_set(fast_bits, i); + is_exit = bitarray_is_set(exit_bits, i); + is_guard = bitarray_is_set(guard_bits, i); + bandwidths[i].dbl = is_fast ? avg_fast : avg_slow; if (is_exit) - total_exit_bw += bandwidths[i]; + total_exit_bw += bandwidths[i].dbl; else - total_nonexit_bw += bandwidths[i]; + total_nonexit_bw += bandwidths[i].dbl; if (is_guard) - total_guard_bw += bandwidths[i]; + total_guard_bw += bandwidths[i].dbl; else - total_nonguard_bw += bandwidths[i]; + total_nonguard_bw += bandwidths[i].dbl; } } /* If there's no bandwidth at all, pick at random. */ - if (!(total_exit_bw+total_nonexit_bw)) { + if (total_exit_bw+total_nonexit_bw < EPSILON) { tor_free(bandwidths); + tor_free(fast_bits); tor_free(exit_bits); tor_free(guard_bits); return smartlist_choose(sl); @@ -2059,12 +1901,12 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, * For detailed derivation of this formula, see * http://archives.seul.org/or/dev/Jul-2007/msg00056.html */ - if (rule == WEIGHT_FOR_EXIT || !total_exit_bw) + if (rule == WEIGHT_FOR_EXIT || total_exit_bw<EPSILON) exit_weight = 1.0; else exit_weight = 1.0 - all_bw/(3.0*exit_bw); - if (rule == WEIGHT_FOR_GUARD || !total_guard_bw) + if (rule == WEIGHT_FOR_GUARD || total_guard_bw<EPSILON) guard_weight = 1.0; else guard_weight = 1.0 - all_bw/(3.0*guard_bw); @@ -2075,29 +1917,25 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, if (guard_weight <= 0.0) guard_weight = 0.0; - total_bw = 0; sl_last_weighted_bw_of_me = 0; for (i=0; i < (unsigned)smartlist_len(sl); i++) { - uint64_t bw; + tor_assert(bandwidths[i].dbl >= 0.0); + is_exit = bitarray_is_set(exit_bits, i); is_guard = bitarray_is_set(guard_bits, i); if (is_exit && is_guard) - bw = ((uint64_t)(bandwidths[i] * exit_weight * guard_weight)); + bandwidths[i].dbl *= exit_weight * guard_weight; else if (is_guard) - bw = ((uint64_t)(bandwidths[i] * guard_weight)); + bandwidths[i].dbl *= guard_weight; else if (is_exit) - bw = ((uint64_t)(bandwidths[i] * exit_weight)); - else - bw = bandwidths[i]; - total_bw += bw; + bandwidths[i].dbl *= exit_weight; + if (i == (unsigned) me_idx) - sl_last_weighted_bw_of_me = bw; + sl_last_weighted_bw_of_me = (uint64_t) bandwidths[i].dbl; } } - /* XXXX this is a kludge to expose these values. */ - sl_last_total_weighted_bw = total_bw; - +#if 0 log_debug(LD_CIRC, "Total weighted bw = "U64_FORMAT ", exit bw = "U64_FORMAT ", nonexit bw = "U64_FORMAT", exit weight = %f " @@ -2110,50 +1948,20 @@ smartlist_choose_node_by_bandwidth(smartlist_t *sl, exit_weight, (int)(rule == WEIGHT_FOR_EXIT), U64_PRINTF_ARG(total_guard_bw), U64_PRINTF_ARG(total_nonguard_bw), guard_weight, (int)(rule == WEIGHT_FOR_GUARD)); +#endif - /* Almost done: choose a random value from the bandwidth weights. */ - rand_bw = crypto_rand_uint64(total_bw); - rand_bw++; /* crypto_rand_uint64() counts from 0, and we need to count - * from 1 below. See bug 1203 for details. */ - - /* Last, count through sl until we get to the element we picked */ - tmp = 0; - i_chosen = (unsigned)smartlist_len(sl); - i_has_been_chosen = 0; - for (i=0; i < (unsigned)smartlist_len(sl); i++) { - is_exit = bitarray_is_set(exit_bits, i); - is_guard = bitarray_is_set(guard_bits, i); - - /* Weights can be 0 if not counting guards/exits */ - if (is_exit && is_guard) - tmp += ((uint64_t)(bandwidths[i] * exit_weight * guard_weight)); - else if (is_guard) - tmp += ((uint64_t)(bandwidths[i] * guard_weight)); - else if (is_exit) - tmp += ((uint64_t)(bandwidths[i] * exit_weight)); - else - tmp += bandwidths[i]; + scale_array_elements_to_u64(bandwidths, smartlist_len(sl), + &sl_last_total_weighted_bw); - if (tmp >= rand_bw && !i_has_been_chosen) { - i_chosen = i; - i_has_been_chosen = 1; - } - } - i = i_chosen; - if (i == (unsigned)smartlist_len(sl)) { - /* This was once possible due to round-off error, but shouldn't be able - * to occur any longer. */ - tor_fragile_assert(); - --i; - log_warn(LD_BUG, "Round-off error in computing bandwidth had an effect on " - " which router we chose. Please tell the developers. " - U64_FORMAT " " U64_FORMAT " " U64_FORMAT, U64_PRINTF_ARG(tmp), - U64_PRINTF_ARG(rand_bw), U64_PRINTF_ARG(total_bw)); + { + int idx = choose_array_element_by_weight(bandwidths, + smartlist_len(sl)); + tor_free(bandwidths); + tor_free(fast_bits); + tor_free(exit_bits); + tor_free(guard_bits); + return idx < 0 ? NULL : smartlist_get(sl, idx); } - tor_free(bandwidths); - tor_free(exit_bits); - tor_free(guard_bits); - return smartlist_get(sl, i); } /** Choose a random element of status list <b>sl</b>, weighted by @@ -2306,7 +2114,7 @@ hex_digest_nickname_decode(const char *hexdigest, * combination of a router, encoded in hexadecimal, matches <b>hexdigest</b> * (which is optionally prefixed with a single dollar sign). Return false if * <b>hexdigest</b> is malformed, or it doesn't match. */ -static int +int hex_digest_nickname_matches(const char *hexdigest, const char *identity_digest, const char *nickname, int is_named) { @@ -2366,129 +2174,6 @@ router_nickname_matches(const routerinfo_t *router, const char *nickname) return router_hex_digest_matches(router, nickname); } -/** Return true if <b>node</b>'s nickname matches <b>nickname</b> - * (case-insensitive), or if <b>node's</b> identity key digest - * matches a hexadecimal value stored in <b>nickname</b>. Return - * false otherwise. */ -static int -node_nickname_matches(const node_t *node, const char *nickname) -{ - const char *n = node_get_nickname(node); - if (n && nickname[0]!='$' && !strcasecmp(n, nickname)) - return 1; - return hex_digest_nickname_matches(nickname, - node->identity, - n, - node_is_named(node)); -} - -/** Return the router in our routerlist whose (case-insensitive) - * nickname or (case-sensitive) hexadecimal key digest is - * <b>nickname</b>. Return NULL if no such router is known. - */ -const routerinfo_t * -router_get_by_nickname(const char *nickname, int warn_if_unnamed) -{ -#if 1 - const node_t *node = node_get_by_nickname(nickname, warn_if_unnamed); - if (node) - return node->ri; - else - return NULL; -#else - int maybedigest; - char digest[DIGEST_LEN]; - routerinfo_t *best_match=NULL; - int n_matches = 0; - const char *named_digest = NULL; - - tor_assert(nickname); - if (!routerlist) - return NULL; - if (nickname[0] == '$') - return router_get_by_hexdigest(nickname); - if (!strcasecmp(nickname, UNNAMED_ROUTER_NICKNAME)) - return NULL; - - maybedigest = (strlen(nickname) >= HEX_DIGEST_LEN) && - (base16_decode(digest,DIGEST_LEN,nickname,HEX_DIGEST_LEN) == 0); - - if ((named_digest = networkstatus_get_router_digest_by_nickname(nickname))) { - return rimap_get(routerlist->identity_map, named_digest); - } - if (networkstatus_nickname_is_unnamed(nickname)) - return NULL; - - /* If we reach this point, there's no canonical value for the nickname. */ - - SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, router, - { - if (!strcasecmp(router->nickname, nickname)) { - ++n_matches; - if (n_matches <= 1 || router->is_running) - best_match = router; - } else if (maybedigest && - tor_memeq(digest, router->cache_info.identity_digest, - DIGEST_LEN)) { - if (router_hex_digest_matches(router, nickname)) - return router; - /* If we reach this point, we have a ID=name syntax that matches the - * identity but not the name. That isn't an acceptable match. */ - } - }); - - if (best_match) { - if (warn_if_unnamed && n_matches > 1) { - smartlist_t *fps = smartlist_new(); - int any_unwarned = 0; - SMARTLIST_FOREACH_BEGIN(routerlist->routers, routerinfo_t *, router) { - routerstatus_t *rs; - char fp[HEX_DIGEST_LEN+1]; - if (strcasecmp(router->nickname, nickname)) - continue; - rs = router_get_mutable_consensus_status_by_id( - router->cache_info.identity_digest); - if (rs && !rs->name_lookup_warned) { - rs->name_lookup_warned = 1; - any_unwarned = 1; - } - base16_encode(fp, sizeof(fp), - router->cache_info.identity_digest, DIGEST_LEN); - smartlist_add_asprintf(fps, "\"$%s\" for the one at %s:%d", - fp, router->address, router->or_port); - } SMARTLIST_FOREACH_END(router); - if (any_unwarned) { - char *alternatives = smartlist_join_strings(fps, "; ",0,NULL); - log_warn(LD_CONFIG, - "There are multiple matches for the nickname \"%s\"," - " but none is listed as named by the directory authorities. " - "Choosing one arbitrarily. If you meant one in particular, " - "you should say %s.", nickname, alternatives); - tor_free(alternatives); - } - SMARTLIST_FOREACH(fps, char *, cp, tor_free(cp)); - smartlist_free(fps); - } else if (warn_if_unnamed) { - routerstatus_t *rs = router_get_mutable_consensus_status_by_id( - best_match->cache_info.identity_digest); - if (rs && !rs->name_lookup_warned) { - char fp[HEX_DIGEST_LEN+1]; - base16_encode(fp, sizeof(fp), - best_match->cache_info.identity_digest, DIGEST_LEN); - log_warn(LD_CONFIG, "You specified a server \"%s\" by name, but this " - "name is not registered, so it could be used by any server, " - "not just the one you meant. " - "To make sure you get the same server in the future, refer to " - "it by key, as \"$%s\".", nickname, fp); - rs->name_lookup_warned = 1; - } - } - return best_match; - } - return NULL; -#endif -} - /** Return true iff <b>digest</b> is the digest of the identity key of a * trusted directory matching at least one bit of <b>type</b>. If <b>type</b> * is zero, any authority is okay. */ @@ -2535,18 +2220,6 @@ hexdigest_to_digest(const char *hexdigest, char *digest) return 0; } -/** Return the router in our routerlist whose hexadecimal key digest - * is <b>hexdigest</b>. Return NULL if no such router is known. */ -const routerinfo_t * -router_get_by_hexdigest(const char *hexdigest) -{ - if (is_legal_nickname(hexdigest)) - return NULL; - - /* It's not a legal nickname, so it must be a hexdigest or nothing. */ - return router_get_by_nickname(hexdigest, 1); -} - /** As router_get_by_id_digest,but return a pointer that you're allowed to * modify */ routerinfo_t * @@ -2890,7 +2563,7 @@ routerlist_insert(routerlist_t *rl, routerinfo_t *ri) &ri->cache_info); smartlist_add(rl->routers, ri); ri->cache_info.routerlist_index = smartlist_len(rl->routers) - 1; - nodelist_add_routerinfo(ri); + nodelist_set_routerinfo(ri, NULL); router_dir_info_changed(); #ifdef DEBUG_ROUTERLIST routerlist_assert_ok(rl); @@ -3119,8 +2792,11 @@ routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old, tor_assert(0 <= idx && idx < smartlist_len(rl->routers)); tor_assert(smartlist_get(rl->routers, idx) == ri_old); - nodelist_remove_routerinfo(ri_old); - nodelist_add_routerinfo(ri_new); + { + routerinfo_t *ri_old_tmp=NULL; + nodelist_set_routerinfo(ri_new, &ri_old_tmp); + tor_assert(ri_old == ri_old_tmp); + } router_dir_info_changed(); if (idx >= 0) { @@ -3263,33 +2939,6 @@ routerlist_reset_warnings(void) networkstatus_reset_warnings(); } -/** Mark the router with ID <b>digest</b> as running or non-running - * in our routerlist. */ -void -router_set_status(const char *digest, int up) -{ - node_t *node; - tor_assert(digest); - - SMARTLIST_FOREACH(trusted_dir_servers, trusted_dir_server_t *, d, - if (tor_memeq(d->digest, digest, DIGEST_LEN)) - d->is_running = up); - - node = node_get_mutable_by_id(digest); - if (node) { -#if 0 - log_debug(LD_DIR,"Marking router %s as %s.", - node_describe(node), up ? "up" : "down"); -#endif - if (!up && node_is_me(node) && !net_is_disabled()) - log_warn(LD_NET, "We just marked ourself as down. Are your external " - "addresses reachable?"); - node->is_running = up; - } - - router_dir_info_changed(); -} - /** Add <b>router</b> to the routerlist, if we don't already have it. Replace * older entries (if any) with the same key. Note: Callers should not hold * their pointers to <b>router</b> if this function fails; <b>router</b> @@ -3457,11 +3106,6 @@ router_add_to_routerlist(routerinfo_t *router, const char **msg, /* Same key, and either new, or listed in the consensus. */ log_debug(LD_DIR, "Replacing entry for router %s", router_describe(router)); - if (routers_have_same_or_addr(router, old_router)) { - /* these carry over when the address and orport are unchanged. */ - router->last_reachable = old_router->last_reachable; - router->testing_since = old_router->testing_since; - } routerlist_replace(routerlist, old_router, router); if (!from_cache) { signed_desc_append_to_journal(&router->cache_info, @@ -4068,27 +3712,6 @@ routerlist_retry_directory_downloads(time_t now) update_all_descriptor_downloads(now); } -/** Return 1 if all running sufficiently-stable routers we can use will reject - * addr:port, return 0 if any might accept it. */ -int -router_exit_policy_all_nodes_reject(const tor_addr_t *addr, uint16_t port, - int need_uptime) -{ /* XXXX MOVE */ - addr_policy_result_t r; - - SMARTLIST_FOREACH_BEGIN(nodelist_get_list(), const node_t *, node) { - if (node->is_running && - !node_is_unreliable(node, need_uptime, 0, 0)) { - - r = compare_tor_addr_to_node_policy(addr, port, node); - - if (r != ADDR_POLICY_REJECTED && r != ADDR_POLICY_PROBABLY_REJECTED) - return 0; /* this one could be ok. good enough. */ - } - } SMARTLIST_FOREACH_END(node); - return 1; /* all will reject. */ -} - /** Return true iff <b>router</b> does not permit exit streams. */ int @@ -4159,11 +3782,6 @@ add_trusted_dir_server(const char *nickname, const char *address, ent->fake_status.dir_port = ent->dir_port; ent->fake_status.or_port = ent->or_port; - if (ent->or_port) - ent->fake_status.version_supports_begindir = 1; - - ent->fake_status.version_supports_conditional_consensus = 1; - smartlist_add(trusted_dir_servers, ent); router_dir_info_changed(); return ent; @@ -4338,7 +3956,7 @@ initiate_descriptor_downloads(const routerstatus_t *source, /* We know which authority we want. */ directory_initiate_command_routerstatus(source, purpose, ROUTER_PURPOSE_GENERAL, - 0, /* not private */ + DIRIND_ONEHOP, resource, NULL, 0, 0); } else { directory_get_from_dirserver(purpose, ROUTER_PURPOSE_GENERAL, resource, @@ -4347,30 +3965,6 @@ initiate_descriptor_downloads(const routerstatus_t *source, tor_free(resource); } -/** Return 0 if this routerstatus is obsolete, too new, isn't - * running, or otherwise not a descriptor that we would make any - * use of even if we had it. Else return 1. */ -static INLINE int -client_would_use_router(const routerstatus_t *rs, time_t now, - const or_options_t *options) -{ - if (!rs->is_flagged_running && !options->FetchUselessDescriptors) { - /* If we had this router descriptor, we wouldn't even bother using it. - * But, if we want to have a complete list, fetch it anyway. */ - return 0; - } - if (rs->published_on + options->TestingEstimatedDescriptorPropagationTime - > now) { - /* Most caches probably don't have this descriptor yet. */ - return 0; - } - if (rs->published_on + OLD_ROUTER_DESC_MAX_AGE < now) { - /* We'd drop it immediately for being too old. */ - return 0; - } - return 1; -} - /** Max amount of hashes to download per request. * Since squid does not like URLs >= 4096 bytes we limit it to 96. * 4096 - strlen(http://255.255.255.255/tor/server/d/.z) == 4058 @@ -4440,11 +4034,6 @@ launch_descriptor_downloads(int purpose, } } } - /* XXX should we consider having even the dir mirrors delay - * a little bit, so we don't load the authorities as much? -RD - * I don't think so. If we do, clients that want those descriptors may - * not actually find them if the caches haven't got them yet. -NM - */ if (! should_delay && n_downloadable) { int i, n_per_request; @@ -4484,9 +4073,9 @@ launch_descriptor_downloads(int purpose, rtr_plural = "s"; log_info(LD_DIR, - "Launching %d request%s for %d router%s, %d at a time", - CEIL_DIV(n_downloadable, n_per_request), - req_plural, n_downloadable, rtr_plural, n_per_request); + "Launching %d request%s for %d %s%s, %d at a time", + CEIL_DIV(n_downloadable, n_per_request), req_plural, + n_downloadable, descname, rtr_plural, n_per_request); smartlist_sort_digests(downloadable); for (i=0; i < n_downloadable; i += n_per_request) { initiate_descriptor_downloads(source, purpose, @@ -4888,230 +4477,6 @@ update_extrainfo_downloads(time_t now) smartlist_free(wanted); } -/** True iff, the last time we checked whether we had enough directory info - * to build circuits, the answer was "yes". */ -static int have_min_dir_info = 0; -/** True iff enough has changed since the last time we checked whether we had - * enough directory info to build circuits that our old answer can no longer - * be trusted. */ -static int need_to_update_have_min_dir_info = 1; -/** String describing what we're missing before we have enough directory - * info. */ -static char dir_info_status[128] = ""; - -/** Return true iff we have enough networkstatus and router information to - * start building circuits. Right now, this means "more than half the - * networkstatus documents, and at least 1/4 of expected routers." */ -//XXX should consider whether we have enough exiting nodes here. -int -router_have_minimum_dir_info(void) -{ - if (PREDICT_UNLIKELY(need_to_update_have_min_dir_info)) { - update_router_have_minimum_dir_info(); - need_to_update_have_min_dir_info = 0; - } - return have_min_dir_info; -} - -/** Called when our internal view of the directory has changed. This can be - * when the authorities change, networkstatuses change, the list of routerdescs - * changes, or number of running routers changes. - */ -void -router_dir_info_changed(void) -{ - need_to_update_have_min_dir_info = 1; - rend_hsdir_routers_changed(); -} - -/** Return a string describing what we're missing before we have enough - * directory info. */ -const char * -get_dir_info_status_string(void) -{ - return dir_info_status; -} - -/** Iterate over the servers listed in <b>consensus</b>, and count how many of - * them seem like ones we'd use, and how many of <em>those</em> we have - * descriptors for. Store the former in *<b>num_usable</b> and the latter in - * *<b>num_present</b>. If <b>in_set</b> is non-NULL, only consider those - * routers in <b>in_set</b>. If <b>exit_only</b> is true, only consider nodes - * with the Exit flag. - */ -static void -count_usable_descriptors(int *num_present, int *num_usable, - const networkstatus_t *consensus, - const or_options_t *options, time_t now, - routerset_t *in_set, int exit_only) -{ - const int md = (consensus->flavor == FLAV_MICRODESC); - *num_present = 0, *num_usable=0; - - SMARTLIST_FOREACH_BEGIN(consensus->routerstatus_list, routerstatus_t *, rs) - { - if (exit_only && ! rs->is_exit) - continue; - if (in_set && ! routerset_contains_routerstatus(in_set, rs, -1)) - continue; - if (client_would_use_router(rs, now, options)) { - const char * const digest = rs->descriptor_digest; - int present; - ++*num_usable; /* the consensus says we want it. */ - if (md) - present = NULL != microdesc_cache_lookup_by_digest256(NULL, digest); - else - present = NULL != router_get_by_descriptor_digest(digest); - if (present) { - /* we have the descriptor listed in the consensus. */ - ++*num_present; - } - } - } - SMARTLIST_FOREACH_END(rs); - - log_debug(LD_DIR, "%d usable, %d present.", *num_usable, *num_present); -} - -/** We just fetched a new set of descriptors. Compute how far through - * the "loading descriptors" bootstrapping phase we are, so we can inform - * the controller of our progress. */ -int -count_loading_descriptors_progress(void) -{ - int num_present = 0, num_usable=0; - time_t now = time(NULL); - const networkstatus_t *consensus = - networkstatus_get_reasonably_live_consensus(now,usable_consensus_flavor()); - double fraction; - - if (!consensus) - return 0; /* can't count descriptors if we have no list of them */ - - count_usable_descriptors(&num_present, &num_usable, - consensus, get_options(), now, NULL, 0); - - if (num_usable == 0) - return 0; /* don't div by 0 */ - fraction = num_present / (num_usable/4.); - if (fraction > 1.0) - return 0; /* it's not the number of descriptors holding us back */ - return BOOTSTRAP_STATUS_LOADING_DESCRIPTORS + (int) - (fraction*(BOOTSTRAP_STATUS_CONN_OR-1 - - BOOTSTRAP_STATUS_LOADING_DESCRIPTORS)); -} - -/** Change the value of have_min_dir_info, setting it true iff we have enough - * network and router information to build circuits. Clear the value of - * need_to_update_have_min_dir_info. */ -static void -update_router_have_minimum_dir_info(void) -{ - int num_present = 0, num_usable=0; - int num_exit_present = 0, num_exit_usable = 0; - time_t now = time(NULL); - int res; - const or_options_t *options = get_options(); - const networkstatus_t *consensus = - networkstatus_get_reasonably_live_consensus(now,usable_consensus_flavor()); - int using_md; - - if (!consensus) { - if (!networkstatus_get_latest_consensus()) - strlcpy(dir_info_status, "We have no usable consensus.", - sizeof(dir_info_status)); - else - strlcpy(dir_info_status, "We have no recent usable consensus.", - sizeof(dir_info_status)); - res = 0; - goto done; - } - - if (should_delay_dir_fetches(get_options())) { - log_notice(LD_DIR, "no known bridge descriptors running yet; stalling"); - strlcpy(dir_info_status, "No live bridge descriptors.", - sizeof(dir_info_status)); - res = 0; - goto done; - } - - using_md = consensus->flavor == FLAV_MICRODESC; - - count_usable_descriptors(&num_present, &num_usable, consensus, options, now, - NULL, 0); - count_usable_descriptors(&num_exit_present, &num_exit_usable, - consensus, options, now, options->ExitNodes, 1); - -/* What fraction of desired server descriptors do we need before we will - * build circuits? */ -#define FRAC_USABLE_NEEDED .75 -/* What fraction of desired _exit_ server descriptors do we need before we - * will build circuits? */ -#define FRAC_EXIT_USABLE_NEEDED .5 - - if (num_present < num_usable * FRAC_USABLE_NEEDED) { - tor_snprintf(dir_info_status, sizeof(dir_info_status), - "We have only %d/%d usable %sdescriptors.", - num_present, num_usable, using_md ? "micro" : ""); - res = 0; - control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_DESCRIPTORS, 0); - goto done; - } else if (num_present < 2) { - tor_snprintf(dir_info_status, sizeof(dir_info_status), - "Only %d %sdescriptor%s here and believed reachable!", - num_present, using_md ? "micro" : "", num_present ? "" : "s"); - res = 0; - goto done; - } else if (num_exit_present < num_exit_usable * FRAC_EXIT_USABLE_NEEDED) { - tor_snprintf(dir_info_status, sizeof(dir_info_status), - "We have only %d/%d usable exit node descriptors.", - num_exit_present, num_exit_usable); - res = 0; - control_event_bootstrap(BOOTSTRAP_STATUS_REQUESTING_DESCRIPTORS, 0); - goto done; - } - - /* Check for entry nodes. */ - if (options->EntryNodes) { - count_usable_descriptors(&num_present, &num_usable, consensus, options, - now, options->EntryNodes, 0); - - if (!num_usable || !num_present) { - tor_snprintf(dir_info_status, sizeof(dir_info_status), - "We have only %d/%d usable entry node %sdescriptors.", - num_present, num_usable, using_md?"micro":""); - res = 0; - goto done; - } - } - - res = 1; - - done: - if (res && !have_min_dir_info) { - log(LOG_NOTICE, LD_DIR, - "We now have enough directory information to build circuits."); - control_event_client_status(LOG_NOTICE, "ENOUGH_DIR_INFO"); - control_event_bootstrap(BOOTSTRAP_STATUS_CONN_OR, 0); - } - if (!res && have_min_dir_info) { - int quiet = directory_too_idle_to_fetch_descriptors(options, now); - log(quiet ? LOG_INFO : LOG_NOTICE, LD_DIR, - "Our directory information is no longer up-to-date " - "enough to build circuits: %s", dir_info_status); - - /* a) make us log when we next complete a circuit, so we know when Tor - * is back up and usable, and b) disable some activities that Tor - * should only do while circuits are working, like reachability tests - * and fetching bridge descriptors only over circuits. */ - can_complete_circuit = 0; - - control_event_client_status(LOG_NOTICE, "NOT_ENOUGH_DIR_INFO"); - } - have_min_dir_info = res; - need_to_update_have_min_dir_info = 0; -} - /** Reset the descriptor download failure count on all routers, so that we * can retry any long-failed routers immediately. */ @@ -5425,149 +4790,6 @@ routers_sort_by_identity(smartlist_t *routers) smartlist_sort(routers, _compare_routerinfo_by_id_digest); } -/** A routerset specifies constraints on a set of possible routerinfos, based - * on their names, identities, or addresses. It is optimized for determining - * whether a router is a member or not, in O(1+P) time, where P is the number - * of address policy constraints. */ -struct routerset_t { - /** A list of strings for the elements of the policy. Each string is either - * a nickname, a hexadecimal identity fingerprint, or an address policy. A - * router belongs to the set if its nickname OR its identity OR its address - * matches an entry here. */ - smartlist_t *list; - /** A map from lowercase nicknames of routers in the set to (void*)1 */ - strmap_t *names; - /** A map from identity digests routers in the set to (void*)1 */ - digestmap_t *digests; - /** An address policy for routers in the set. For implementation reasons, - * a router belongs to the set if it is _rejected_ by this policy. */ - smartlist_t *policies; - - /** A human-readable description of what this routerset is for. Used in - * log messages. */ - char *description; - - /** A list of the country codes in this set. */ - smartlist_t *country_names; - /** Total number of countries we knew about when we built <b>countries</b>.*/ - int n_countries; - /** Bit array mapping the return value of geoip_get_country() to 1 iff the - * country is a member of this routerset. Note that we MUST call - * routerset_refresh_countries() whenever the geoip country list is - * reloaded. */ - bitarray_t *countries; -}; - -/** Return a new empty routerset. */ -routerset_t * -routerset_new(void) -{ - routerset_t *result = tor_malloc_zero(sizeof(routerset_t)); - result->list = smartlist_new(); - result->names = strmap_new(); - result->digests = digestmap_new(); - result->policies = smartlist_new(); - result->country_names = smartlist_new(); - return result; -} - -/** If <b>c</b> is a country code in the form {cc}, return a newly allocated - * string holding the "cc" part. Else, return NULL. */ -static char * -routerset_get_countryname(const char *c) -{ - char *country; - - if (strlen(c) < 4 || c[0] !='{' || c[3] !='}') - return NULL; - - country = tor_strndup(c+1, 2); - tor_strlower(country); - return country; -} - -/** Update the routerset's <b>countries</b> bitarray_t. Called whenever - * the GeoIP database is reloaded. - */ -void -routerset_refresh_countries(routerset_t *target) -{ - int cc; - bitarray_free(target->countries); - - if (!geoip_is_loaded()) { - target->countries = NULL; - target->n_countries = 0; - return; - } - target->n_countries = geoip_get_n_countries(); - target->countries = bitarray_init_zero(target->n_countries); - SMARTLIST_FOREACH_BEGIN(target->country_names, const char *, country) { - cc = geoip_get_country(country); - if (cc >= 0) { - tor_assert(cc < target->n_countries); - bitarray_set(target->countries, cc); - } else { - log(LOG_WARN, LD_CONFIG, "Country code '%s' is not recognized.", - country); - } - } SMARTLIST_FOREACH_END(country); -} - -/** Parse the string <b>s</b> to create a set of routerset entries, and add - * them to <b>target</b>. In log messages, refer to the string as - * <b>description</b>. Return 0 on success, -1 on failure. - * - * Three kinds of elements are allowed in routersets: nicknames, IP address - * patterns, and fingerprints. They may be surrounded by optional space, and - * must be separated by commas. - */ -int -routerset_parse(routerset_t *target, const char *s, const char *description) -{ - int r = 0; - int added_countries = 0; - char *countryname; - smartlist_t *list = smartlist_new(); - smartlist_split_string(list, s, ",", - SPLIT_SKIP_SPACE | SPLIT_IGNORE_BLANK, 0); - SMARTLIST_FOREACH_BEGIN(list, char *, nick) { - addr_policy_t *p; - if (is_legal_hexdigest(nick)) { - char d[DIGEST_LEN]; - if (*nick == '$') - ++nick; - log_debug(LD_CONFIG, "Adding identity %s to %s", nick, description); - base16_decode(d, sizeof(d), nick, HEX_DIGEST_LEN); - digestmap_set(target->digests, d, (void*)1); - } else if (is_legal_nickname(nick)) { - log_debug(LD_CONFIG, "Adding nickname %s to %s", nick, description); - strmap_set_lc(target->names, nick, (void*)1); - } else if ((countryname = routerset_get_countryname(nick)) != NULL) { - log_debug(LD_CONFIG, "Adding country %s to %s", nick, - description); - smartlist_add(target->country_names, countryname); - added_countries = 1; - } else if ((strchr(nick,'.') || strchr(nick, '*')) && - (p = router_parse_addr_policy_item_from_string( - nick, ADDR_POLICY_REJECT))) { - log_debug(LD_CONFIG, "Adding address %s to %s", nick, description); - smartlist_add(target->policies, p); - } else { - log_warn(LD_CONFIG, "Entry '%s' in %s is misformed.", nick, - description); - r = -1; - tor_free(nick); - SMARTLIST_DEL_CURRENT(list, nick); - } - } SMARTLIST_FOREACH_END(nick); - smartlist_add_all(target->list, list); - smartlist_free(list); - if (added_countries) - routerset_refresh_countries(target); - return r; -} - /** Called when we change a node set, or when we reload the geoip list: * recompute all country info in all configuration node sets and in the * routerlist. */ @@ -5590,297 +4812,6 @@ refresh_all_country_info(void) nodelist_refresh_countries(); } -/** Add all members of the set <b>source</b> to <b>target</b>. */ -void -routerset_union(routerset_t *target, const routerset_t *source) -{ - char *s; - tor_assert(target); - if (!source || !source->list) - return; - s = routerset_to_string(source); - routerset_parse(target, s, "other routerset"); - tor_free(s); -} - -/** Return true iff <b>set</b> lists only nicknames and digests, and includes - * no IP ranges or countries. */ -int -routerset_is_list(const routerset_t *set) -{ - return smartlist_len(set->country_names) == 0 && - smartlist_len(set->policies) == 0; -} - -/** Return true iff we need a GeoIP IP-to-country database to make sense of - * <b>set</b>. */ -int -routerset_needs_geoip(const routerset_t *set) -{ - return set && smartlist_len(set->country_names); -} - -/** Return true iff there are no entries in <b>set</b>. */ -int -routerset_is_empty(const routerset_t *set) -{ - return !set || smartlist_len(set->list) == 0; -} - -/** Helper. Return true iff <b>set</b> contains a router based on the other - * provided fields. Return higher values for more specific subentries: a - * single router is more specific than an address range of routers, which is - * more specific in turn than a country code. - * - * (If country is -1, then we take the country - * from addr.) */ -static int -routerset_contains(const routerset_t *set, const tor_addr_t *addr, - uint16_t orport, - const char *nickname, const char *id_digest, - country_t country) -{ - if (!set || !set->list) - return 0; - if (nickname && strmap_get_lc(set->names, nickname)) - return 4; - if (id_digest && digestmap_get(set->digests, id_digest)) - return 4; - if (addr && compare_tor_addr_to_addr_policy(addr, orport, set->policies) - == ADDR_POLICY_REJECTED) - return 3; - if (set->countries) { - if (country < 0 && addr) - country = geoip_get_country_by_ip(tor_addr_to_ipv4h(addr)); - - if (country >= 0 && country < set->n_countries && - bitarray_is_set(set->countries, country)) - return 2; - } - return 0; -} - -/** Return true iff we can tell that <b>ei</b> is a member of <b>set</b>. */ -int -routerset_contains_extendinfo(const routerset_t *set, const extend_info_t *ei) -{ - return routerset_contains(set, - &ei->addr, - ei->port, - ei->nickname, - ei->identity_digest, - -1 /*country*/); -} - -/** Return true iff <b>ri</b> is in <b>set</b>. If country is <b>-1</b>, we - * look up the country. */ -int -routerset_contains_router(const routerset_t *set, const routerinfo_t *ri, - country_t country) -{ - tor_addr_t addr; - tor_addr_from_ipv4h(&addr, ri->addr); - return routerset_contains(set, - &addr, - ri->or_port, - ri->nickname, - ri->cache_info.identity_digest, - country); -} - -/** Return true iff <b>rs</b> is in <b>set</b>. If country is <b>-1</b>, we - * look up the country. */ -int -routerset_contains_routerstatus(const routerset_t *set, - const routerstatus_t *rs, - country_t country) -{ - tor_addr_t addr; - tor_addr_from_ipv4h(&addr, rs->addr); - return routerset_contains(set, - &addr, - rs->or_port, - rs->nickname, - rs->identity_digest, - country); -} - -/** Return true iff <b>node</b> is in <b>set</b>. */ -int -routerset_contains_node(const routerset_t *set, const node_t *node) -{ - if (node->rs) - return routerset_contains_routerstatus(set, node->rs, node->country); - else if (node->ri) - return routerset_contains_router(set, node->ri, node->country); - else - return 0; -} - -/** Add every known node_t that is a member of <b>routerset</b> to - * <b>out</b>, but never add any that are part of <b>excludeset</b>. - * If <b>running_only</b>, only add the running ones. */ -void -routerset_get_all_nodes(smartlist_t *out, const routerset_t *routerset, - const routerset_t *excludeset, int running_only) -{ /* XXXX MOVE */ - tor_assert(out); - if (!routerset || !routerset->list) - return; - - if (routerset_is_list(routerset)) { - /* No routers are specified by type; all are given by name or digest. - * we can do a lookup in O(len(routerset)). */ - SMARTLIST_FOREACH(routerset->list, const char *, name, { - const node_t *node = node_get_by_nickname(name, 1); - if (node) { - if (!running_only || node->is_running) - if (!routerset_contains_node(excludeset, node)) - smartlist_add(out, (void*)node); - } - }); - } else { - /* We need to iterate over the routerlist to get all the ones of the - * right kind. */ - smartlist_t *nodes = nodelist_get_list(); - SMARTLIST_FOREACH(nodes, const node_t *, node, { - if (running_only && !node->is_running) - continue; - if (routerset_contains_node(routerset, node) && - !routerset_contains_node(excludeset, node)) - smartlist_add(out, (void*)node); - }); - } -} - -#if 0 -/** Add to <b>target</b> every node_t from <b>source</b> except: - * - * 1) Don't add it if <b>include</b> is non-empty and the relay isn't in - * <b>include</b>; and - * 2) Don't add it if <b>exclude</b> is non-empty and the relay is - * excluded in a more specific fashion by <b>exclude</b>. - * 3) If <b>running_only</b>, don't add non-running routers. - */ -void -routersets_get_node_disjunction(smartlist_t *target, - const smartlist_t *source, - const routerset_t *include, - const routerset_t *exclude, int running_only) -{ - SMARTLIST_FOREACH(source, const node_t *, node, { - int include_result; - if (running_only && !node->is_running) - continue; - if (!routerset_is_empty(include)) - include_result = routerset_contains_node(include, node); - else - include_result = 1; - - if (include_result) { - int exclude_result = routerset_contains_node(exclude, node); - if (include_result >= exclude_result) - smartlist_add(target, (void*)node); - } - }); -} -#endif - -/** Remove every node_t from <b>lst</b> that is in <b>routerset</b>. */ -void -routerset_subtract_nodes(smartlist_t *lst, const routerset_t *routerset) -{ /*XXXX MOVE ? */ - tor_assert(lst); - if (!routerset) - return; - SMARTLIST_FOREACH(lst, const node_t *, node, { - if (routerset_contains_node(routerset, node)) { - //log_debug(LD_DIR, "Subtracting %s",r->nickname); - SMARTLIST_DEL_CURRENT(lst, node); - } - }); -} - -/** Return a new string that when parsed by routerset_parse_string() will - * yield <b>set</b>. */ -char * -routerset_to_string(const routerset_t *set) -{ - if (!set || !set->list) - return tor_strdup(""); - return smartlist_join_strings(set->list, ",", 0, NULL); -} - -/** Helper: return true iff old and new are both NULL, or both non-NULL - * equal routersets. */ -int -routerset_equal(const routerset_t *old, const routerset_t *new) -{ - if (routerset_is_empty(old) && routerset_is_empty(new)) { - /* Two empty sets are equal */ - return 1; - } else if (routerset_is_empty(old) || routerset_is_empty(new)) { - /* An empty set is equal to nothing else. */ - return 0; - } - tor_assert(old != NULL); - tor_assert(new != NULL); - - if (smartlist_len(old->list) != smartlist_len(new->list)) - return 0; - - SMARTLIST_FOREACH(old->list, const char *, cp1, { - const char *cp2 = smartlist_get(new->list, cp1_sl_idx); - if (strcmp(cp1, cp2)) - return 0; - }); - - return 1; -} - -/** Free all storage held in <b>routerset</b>. */ -void -routerset_free(routerset_t *routerset) -{ - if (!routerset) - return; - - SMARTLIST_FOREACH(routerset->list, char *, cp, tor_free(cp)); - smartlist_free(routerset->list); - SMARTLIST_FOREACH(routerset->policies, addr_policy_t *, p, - addr_policy_free(p)); - smartlist_free(routerset->policies); - SMARTLIST_FOREACH(routerset->country_names, char *, cp, tor_free(cp)); - smartlist_free(routerset->country_names); - - strmap_free(routerset->names, NULL); - digestmap_free(routerset->digests, NULL); - bitarray_free(routerset->countries); - tor_free(routerset); -} - -/** Refresh the country code of <b>ri</b>. This function MUST be called on - * each router when the GeoIP database is reloaded, and on all new routers. */ -void -node_set_country(node_t *node) -{ - if (node->rs) - node->country = geoip_get_country_by_ip(node->rs->addr); - else if (node->ri) - node->country = geoip_get_country_by_ip(node->ri->addr); - else - node->country = -1; -} - -/** Set the country code of all routers in the routerlist. */ -void -nodelist_refresh_countries(void) /* MOVE */ -{ - smartlist_t *nodes = nodelist_get_list(); - SMARTLIST_FOREACH(nodes, node_t *, node, - node_set_country(node)); -} - /** Determine the routers that are responsible for <b>id</b> (binary) and * add pointers to those routers' routerstatus_t to <b>responsible_dirs</b>. * Return -1 if we're returning an empty smartlist, else return 0. diff --git a/src/or/routerlist.h b/src/or/routerlist.h index 8dcc6eb02..58143010b 100644 --- a/src/or/routerlist.h +++ b/src/or/routerlist.h @@ -36,13 +36,9 @@ const routerstatus_t *router_pick_trusteddirserver(dirinfo_type_t type, int router_get_my_share_of_directory_requests(double *v2_share_out, double *v3_share_out); void router_reset_status_download_failures(void); -int routers_have_same_or_addr(const routerinfo_t *r1, const routerinfo_t *r2); +int routers_have_same_or_addrs(const routerinfo_t *r1, const routerinfo_t *r2); int router_nickname_is_in_list(const routerinfo_t *router, const char *list); const routerinfo_t *routerlist_find_my_routerinfo(void); -const node_t *router_find_exact_exit_enclave(const char *address, - uint16_t port); -int node_is_unreliable(const node_t *router, int need_uptime, - int need_capacity, int need_guard); uint32_t router_get_advertised_bandwidth(const routerinfo_t *router); uint32_t router_get_advertised_bandwidth_capped(const routerinfo_t *router); @@ -53,8 +49,6 @@ const node_t *router_choose_random_node(smartlist_t *excludedsmartlist, struct routerset_t *excludedset, router_crn_flags_t flags); -const routerinfo_t *router_get_by_nickname(const char *nickname, - int warn_if_unnamed); int router_is_named(const routerinfo_t *router); int router_digest_is_trusted_dir_type(const char *digest, dirinfo_type_t type); @@ -63,7 +57,6 @@ int router_digest_is_trusted_dir_type(const char *digest, int router_addr_is_trusted_dir(uint32_t addr); int hexdigest_to_digest(const char *hexdigest, char *digest); -const routerinfo_t *router_get_by_hexdigest(const char *hexdigest); const routerinfo_t *router_get_by_id_digest(const char *digest); routerinfo_t *router_get_mutable_by_digest(const char *digest); signed_descriptor_t *router_get_by_descriptor_digest(const char *digest); @@ -80,7 +73,6 @@ void routerlist_remove(routerlist_t *rl, routerinfo_t *ri, int make_old, time_t now); void routerlist_free_all(void); void routerlist_reset_warnings(void); -void router_set_status(const char *digest, int up); static int WRA_WAS_ADDED(was_router_added_t s); static int WRA_WAS_OUTDATED(was_router_added_t s); @@ -133,8 +125,6 @@ void router_load_extrainfo_from_string(const char *s, const char *eos, int descriptor_digests); void routerlist_retry_directory_downloads(time_t now); -int router_exit_policy_all_nodes_reject(const tor_addr_t *addr, uint16_t port, - int need_uptime); int router_exit_policy_rejects_all(const routerinfo_t *router); trusted_dir_server_t *add_trusted_dir_server(const char *nickname, @@ -150,10 +140,6 @@ void update_consensus_router_descriptor_downloads(time_t now, int is_vote, void update_router_descriptor_downloads(time_t now); void update_all_descriptor_downloads(time_t now); void update_extrainfo_downloads(time_t now); -int router_have_minimum_dir_info(void); -void router_dir_info_changed(void); -const char *get_dir_info_status_string(void); -int count_loading_descriptors_progress(void); void router_reset_descriptor_download_failures(void); int router_differences_are_cosmetic(const routerinfo_t *r1, const routerinfo_t *r2); @@ -166,38 +152,6 @@ void routerlist_assert_ok(const routerlist_t *rl); const char *esc_router_info(const routerinfo_t *router); void routers_sort_by_identity(smartlist_t *routers); -routerset_t *routerset_new(void); -void routerset_refresh_countries(routerset_t *rs); -int routerset_parse(routerset_t *target, const char *s, - const char *description); -void routerset_union(routerset_t *target, const routerset_t *source); -int routerset_is_list(const routerset_t *set); -int routerset_needs_geoip(const routerset_t *set); -int routerset_is_empty(const routerset_t *set); -int routerset_contains_router(const routerset_t *set, const routerinfo_t *ri, - country_t country); -int routerset_contains_routerstatus(const routerset_t *set, - const routerstatus_t *rs, - country_t country); -int routerset_contains_extendinfo(const routerset_t *set, - const extend_info_t *ei); - -int routerset_contains_node(const routerset_t *set, const node_t *node); -void routerset_get_all_nodes(smartlist_t *out, const routerset_t *routerset, - const routerset_t *excludeset, - int running_only); -#if 0 -void routersets_get_node_disjunction(smartlist_t *target, - const smartlist_t *source, - const routerset_t *include, - const routerset_t *exclude, int running_only); -#endif -void routerset_subtract_nodes(smartlist_t *out, - const routerset_t *routerset); - -char *routerset_to_string(const routerset_t *routerset); -int routerset_equal(const routerset_t *old, const routerset_t *new); -void routerset_free(routerset_t *routerset); void refresh_all_country_info(void); int hid_serv_get_responsible_directories(smartlist_t *responsible_dirs, @@ -215,6 +169,23 @@ int hex_digest_nickname_decode(const char *hexdigest, char *digest_out, char *nickname_qualifier_out, char *nickname_out); +int hex_digest_nickname_matches(const char *hexdigest, + const char *identity_digest, + const char *nickname, int is_named); + +#ifdef ROUTERLIST_PRIVATE +/** Helper type for choosing routers by bandwidth: contains a union of + * double and uint64_t. Before we call scale_array_elements_to_u64, it holds + * a double; after, it holds a uint64_t. */ +typedef union u64_dbl_t { + uint64_t u64; + double dbl; +} u64_dbl_t; + +int choose_array_element_by_weight(const u64_dbl_t *entries, int n_entries); +void scale_array_elements_to_u64(u64_dbl_t *entries, int n_entries, + uint64_t *total_out); +#endif #endif diff --git a/src/or/routerparse.c b/src/or/routerparse.c index 2bf072b3c..43a95e88c 100644 --- a/src/or/routerparse.c +++ b/src/or/routerparse.c @@ -67,6 +67,7 @@ typedef enum { K_OR_ADDRESS, K_P, K_R, + K_A, K_S, K_V, K_W, @@ -338,6 +339,7 @@ static token_rule_t extrainfo_token_table[] = { static token_rule_t rtrstatus_token_table[] = { T01("p", K_P, CONCAT_ARGS, NO_OBJ ), T1( "r", K_R, GE(7), NO_OBJ ), + T0N("a", K_A, GE(1), NO_OBJ ), T1( "s", K_S, ARGS, NO_OBJ ), T01("v", K_V, CONCAT_ARGS, NO_OBJ ), T01("w", K_W, ARGS, NO_OBJ ), @@ -522,6 +524,7 @@ static token_rule_t networkstatus_detached_signature_token_table[] = { /** List of tokens recognized in microdescriptors */ static token_rule_t microdesc_token_table[] = { T1_START("onion-key", K_ONION_KEY, NO_ARGS, NEED_KEY_1024), + T0N("a", K_A, GE(1), NO_OBJ ), T01("family", K_FAMILY, ARGS, NO_OBJ ), T01("p", K_P, CONCAT_ARGS, NO_OBJ ), A01("@last-listed", A_LAST_LISTED, CONCAT_ARGS, NO_OBJ ), @@ -1257,6 +1260,42 @@ dump_distinct_digest_count(int severity) #endif } +/** Try to find an IPv6 OR port in <b>list</b> of directory_token_t's + * with at least one argument (use GE(1) in setup). If found, store + * address and port number to <b>addr_out</b> and + * <b>port_out</b>. Return number of OR ports found. */ +static int +find_single_ipv6_orport(const smartlist_t *list, + tor_addr_t *addr_out, + uint16_t *port_out) +{ + int ret = 0; + tor_assert(list != NULL); + tor_assert(addr_out != NULL); + tor_assert(port_out != NULL); + + SMARTLIST_FOREACH_BEGIN(list, directory_token_t *, t) { + tor_addr_t a; + maskbits_t bits; + uint16_t port_min, port_max; + tor_assert(t->n_args >= 1); + /* XXXX Prop186 the full spec allows much more than this. */ + if (tor_addr_parse_mask_ports(t->args[0], &a, &bits, &port_min, + &port_max) == AF_INET6 && + bits == 128 && + port_min == port_max) { + /* Okay, this is one we can understand. Use it and ignore + any potential more addresses in list. */ + tor_addr_copy(addr_out, &a); + *port_out = port_min; + ret = 1; + break; + } + } SMARTLIST_FOREACH_END(t); + + return ret; +} + /** Helper function: reads a single router entry from *<b>s</b> ... * *<b>end</b>. Mallocs a new router and returns it if all goes well, else * returns NULL. If <b>cache_copy</b> is true, duplicate the contents of @@ -1513,21 +1552,8 @@ router_parse_entry_from_string(const char *s, const char *end, { smartlist_t *or_addresses = find_all_by_keyword(tokens, K_OR_ADDRESS); if (or_addresses) { - SMARTLIST_FOREACH_BEGIN(or_addresses, directory_token_t *, t) { - tor_addr_t a; - maskbits_t bits; - uint16_t port_min, port_max; - /* XXXX Prop186 the full spec allows much more than this. */ - if (tor_addr_parse_mask_ports(t->args[0], &a, &bits, &port_min, - &port_max) == AF_INET6 && - bits == 128 && - port_min == port_max) { - /* Okay, this is one we can understand. */ - tor_addr_copy(&router->ipv6_addr, &a); - router->ipv6_orport = port_min; - break; - } - } SMARTLIST_FOREACH_END(t); + find_single_ipv6_orport(or_addresses, &router->ipv6_addr, + &router->ipv6_orport); smartlist_free(or_addresses); } } @@ -2060,6 +2086,14 @@ routerstatus_parse_entry_from_string(memarea_t *area, rs->dir_port = (uint16_t) tor_parse_long(tok->args[7+offset], 10,0,65535,NULL,NULL); + { + smartlist_t *a_lines = find_all_by_keyword(tokens, K_A); + if (a_lines) { + find_single_ipv6_orport(a_lines, &rs->ipv6_addr, &rs->ipv6_orport); + smartlist_free(a_lines); + } + } + tok = find_opt_by_keyword(tokens, K_S); if (tok && vote) { int i; @@ -2112,20 +2146,9 @@ routerstatus_parse_entry_from_string(memarea_t *area, tor_assert(tok->n_args == 1); rs->version_known = 1; if (strcmpstart(tok->args[0], "Tor ")) { - rs->version_supports_begindir = 1; - rs->version_supports_extrainfo_upload = 1; - rs->version_supports_conditional_consensus = 1; rs->version_supports_microdesc_cache = 1; rs->version_supports_optimistic_data = 1; } else { - rs->version_supports_begindir = - tor_version_as_new_as(tok->args[0], "0.2.0.1-alpha"); - rs->version_supports_extrainfo_upload = - tor_version_as_new_as(tok->args[0], "0.2.0.0-alpha-dev (r10070)"); - rs->version_supports_v3_dir = - tor_version_as_new_as(tok->args[0], "0.2.0.8-alpha"); - rs->version_supports_conditional_consensus = - tor_version_as_new_as(tok->args[0], "0.2.1.1-alpha"); rs->version_supports_microdesc_cache = tor_version_supports_microdescriptors(tok->args[0]); rs->version_supports_optimistic_data = @@ -2981,6 +3004,16 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, log_warn(LD_DIR, "known-flags not in order"); goto err; } + if (ns->type != NS_TYPE_CONSENSUS && + smartlist_len(ns->known_flags) > MAX_KNOWN_FLAGS_IN_VOTE) { + /* If we allowed more than 64 flags in votes, then parsing them would make + * us invoke undefined behavior whenever we used 1<<flagnum to do a + * bit-shift. This is only for votes and opinions: consensus users don't + * care about flags they don't recognize, and so don't build a bitfield + * for them. */ + log_warn(LD_DIR, "Too many known-flags in consensus vote or opinion"); + goto err; + } tok = find_opt_by_keyword(tokens, K_PARAMS); if (tok) { @@ -4421,6 +4454,14 @@ microdescs_parse_from_string(const char *s, const char *eos, md->onion_pkey = tok->key; tok->key = NULL; + { + smartlist_t *a_lines = find_all_by_keyword(tokens, K_A); + if (a_lines) { + find_single_ipv6_orport(a_lines, &md->ipv6_addr, &md->ipv6_orport); + smartlist_free(a_lines); + } + } + if ((tok = find_opt_by_keyword(tokens, K_FAMILY))) { int i; md->family = smartlist_new(); diff --git a/src/or/routerset.c b/src/or/routerset.c new file mode 100644 index 000000000..263cf79d7 --- /dev/null +++ b/src/or/routerset.c @@ -0,0 +1,426 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "geoip.h" +#include "nodelist.h" +#include "policies.h" +#include "router.h" +#include "routerparse.h" +#include "routerset.h" + +/** A routerset specifies constraints on a set of possible routerinfos, based + * on their names, identities, or addresses. It is optimized for determining + * whether a router is a member or not, in O(1+P) time, where P is the number + * of address policy constraints. */ +struct routerset_t { + /** A list of strings for the elements of the policy. Each string is either + * a nickname, a hexadecimal identity fingerprint, or an address policy. A + * router belongs to the set if its nickname OR its identity OR its address + * matches an entry here. */ + smartlist_t *list; + /** A map from lowercase nicknames of routers in the set to (void*)1 */ + strmap_t *names; + /** A map from identity digests routers in the set to (void*)1 */ + digestmap_t *digests; + /** An address policy for routers in the set. For implementation reasons, + * a router belongs to the set if it is _rejected_ by this policy. */ + smartlist_t *policies; + + /** A human-readable description of what this routerset is for. Used in + * log messages. */ + char *description; + + /** A list of the country codes in this set. */ + smartlist_t *country_names; + /** Total number of countries we knew about when we built <b>countries</b>.*/ + int n_countries; + /** Bit array mapping the return value of geoip_get_country() to 1 iff the + * country is a member of this routerset. Note that we MUST call + * routerset_refresh_countries() whenever the geoip country list is + * reloaded. */ + bitarray_t *countries; +}; + +/** Return a new empty routerset. */ +routerset_t * +routerset_new(void) +{ + routerset_t *result = tor_malloc_zero(sizeof(routerset_t)); + result->list = smartlist_new(); + result->names = strmap_new(); + result->digests = digestmap_new(); + result->policies = smartlist_new(); + result->country_names = smartlist_new(); + return result; +} + +/** If <b>c</b> is a country code in the form {cc}, return a newly allocated + * string holding the "cc" part. Else, return NULL. */ +static char * +routerset_get_countryname(const char *c) +{ + char *country; + + if (strlen(c) < 4 || c[0] !='{' || c[3] !='}') + return NULL; + + country = tor_strndup(c+1, 2); + tor_strlower(country); + return country; +} + +/** Update the routerset's <b>countries</b> bitarray_t. Called whenever + * the GeoIP database is reloaded. + */ +void +routerset_refresh_countries(routerset_t *target) +{ + int cc; + bitarray_free(target->countries); + + if (!geoip_is_loaded()) { + target->countries = NULL; + target->n_countries = 0; + return; + } + target->n_countries = geoip_get_n_countries(); + target->countries = bitarray_init_zero(target->n_countries); + SMARTLIST_FOREACH_BEGIN(target->country_names, const char *, country) { + cc = geoip_get_country(country); + if (cc >= 0) { + tor_assert(cc < target->n_countries); + bitarray_set(target->countries, cc); + } else { + log(LOG_WARN, LD_CONFIG, "Country code '%s' is not recognized.", + country); + } + } SMARTLIST_FOREACH_END(country); +} + +/** Parse the string <b>s</b> to create a set of routerset entries, and add + * them to <b>target</b>. In log messages, refer to the string as + * <b>description</b>. Return 0 on success, -1 on failure. + * + * Three kinds of elements are allowed in routersets: nicknames, IP address + * patterns, and fingerprints. They may be surrounded by optional space, and + * must be separated by commas. + */ +int +routerset_parse(routerset_t *target, const char *s, const char *description) +{ + int r = 0; + int added_countries = 0; + char *countryname; + smartlist_t *list = smartlist_new(); + smartlist_split_string(list, s, ",", + SPLIT_SKIP_SPACE | SPLIT_IGNORE_BLANK, 0); + SMARTLIST_FOREACH_BEGIN(list, char *, nick) { + addr_policy_t *p; + if (is_legal_hexdigest(nick)) { + char d[DIGEST_LEN]; + if (*nick == '$') + ++nick; + log_debug(LD_CONFIG, "Adding identity %s to %s", nick, description); + base16_decode(d, sizeof(d), nick, HEX_DIGEST_LEN); + digestmap_set(target->digests, d, (void*)1); + } else if (is_legal_nickname(nick)) { + log_debug(LD_CONFIG, "Adding nickname %s to %s", nick, description); + strmap_set_lc(target->names, nick, (void*)1); + } else if ((countryname = routerset_get_countryname(nick)) != NULL) { + log_debug(LD_CONFIG, "Adding country %s to %s", nick, + description); + smartlist_add(target->country_names, countryname); + added_countries = 1; + } else if ((strchr(nick,'.') || strchr(nick, '*')) && + (p = router_parse_addr_policy_item_from_string( + nick, ADDR_POLICY_REJECT))) { + log_debug(LD_CONFIG, "Adding address %s to %s", nick, description); + smartlist_add(target->policies, p); + } else { + log_warn(LD_CONFIG, "Entry '%s' in %s is misformed.", nick, + description); + r = -1; + tor_free(nick); + SMARTLIST_DEL_CURRENT(list, nick); + } + } SMARTLIST_FOREACH_END(nick); + smartlist_add_all(target->list, list); + smartlist_free(list); + if (added_countries) + routerset_refresh_countries(target); + return r; +} + +/** Add all members of the set <b>source</b> to <b>target</b>. */ +void +routerset_union(routerset_t *target, const routerset_t *source) +{ + char *s; + tor_assert(target); + if (!source || !source->list) + return; + s = routerset_to_string(source); + routerset_parse(target, s, "other routerset"); + tor_free(s); +} + +/** Return true iff <b>set</b> lists only nicknames and digests, and includes + * no IP ranges or countries. */ +int +routerset_is_list(const routerset_t *set) +{ + return smartlist_len(set->country_names) == 0 && + smartlist_len(set->policies) == 0; +} + +/** Return true iff we need a GeoIP IP-to-country database to make sense of + * <b>set</b>. */ +int +routerset_needs_geoip(const routerset_t *set) +{ + return set && smartlist_len(set->country_names); +} + +/** Return true iff there are no entries in <b>set</b>. */ +int +routerset_is_empty(const routerset_t *set) +{ + return !set || smartlist_len(set->list) == 0; +} + +/** Helper. Return true iff <b>set</b> contains a router based on the other + * provided fields. Return higher values for more specific subentries: a + * single router is more specific than an address range of routers, which is + * more specific in turn than a country code. + * + * (If country is -1, then we take the country + * from addr.) */ +static int +routerset_contains(const routerset_t *set, const tor_addr_t *addr, + uint16_t orport, + const char *nickname, const char *id_digest, + country_t country) +{ + if (!set || !set->list) + return 0; + if (nickname && strmap_get_lc(set->names, nickname)) + return 4; + if (id_digest && digestmap_get(set->digests, id_digest)) + return 4; + if (addr && compare_tor_addr_to_addr_policy(addr, orport, set->policies) + == ADDR_POLICY_REJECTED) + return 3; + if (set->countries) { + if (country < 0 && addr) + country = geoip_get_country_by_ip(tor_addr_to_ipv4h(addr)); + + if (country >= 0 && country < set->n_countries && + bitarray_is_set(set->countries, country)) + return 2; + } + return 0; +} + +/** Return true iff we can tell that <b>ei</b> is a member of <b>set</b>. */ +int +routerset_contains_extendinfo(const routerset_t *set, const extend_info_t *ei) +{ + return routerset_contains(set, + &ei->addr, + ei->port, + ei->nickname, + ei->identity_digest, + -1 /*country*/); +} + +/** Return true iff <b>ri</b> is in <b>set</b>. If country is <b>-1</b>, we + * look up the country. */ +int +routerset_contains_router(const routerset_t *set, const routerinfo_t *ri, + country_t country) +{ + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, ri->addr); + return routerset_contains(set, + &addr, + ri->or_port, + ri->nickname, + ri->cache_info.identity_digest, + country); +} + +/** Return true iff <b>rs</b> is in <b>set</b>. If country is <b>-1</b>, we + * look up the country. */ +int +routerset_contains_routerstatus(const routerset_t *set, + const routerstatus_t *rs, + country_t country) +{ + tor_addr_t addr; + tor_addr_from_ipv4h(&addr, rs->addr); + return routerset_contains(set, + &addr, + rs->or_port, + rs->nickname, + rs->identity_digest, + country); +} + +/** Return true iff <b>node</b> is in <b>set</b>. */ +int +routerset_contains_node(const routerset_t *set, const node_t *node) +{ + if (node->rs) + return routerset_contains_routerstatus(set, node->rs, node->country); + else if (node->ri) + return routerset_contains_router(set, node->ri, node->country); + else + return 0; +} + +/** Add every known node_t that is a member of <b>routerset</b> to + * <b>out</b>, but never add any that are part of <b>excludeset</b>. + * If <b>running_only</b>, only add the running ones. */ +void +routerset_get_all_nodes(smartlist_t *out, const routerset_t *routerset, + const routerset_t *excludeset, int running_only) +{ + tor_assert(out); + if (!routerset || !routerset->list) + return; + + if (routerset_is_list(routerset)) { + /* No routers are specified by type; all are given by name or digest. + * we can do a lookup in O(len(routerset)). */ + SMARTLIST_FOREACH(routerset->list, const char *, name, { + const node_t *node = node_get_by_nickname(name, 1); + if (node) { + if (!running_only || node->is_running) + if (!routerset_contains_node(excludeset, node)) + smartlist_add(out, (void*)node); + } + }); + } else { + /* We need to iterate over the routerlist to get all the ones of the + * right kind. */ + smartlist_t *nodes = nodelist_get_list(); + SMARTLIST_FOREACH(nodes, const node_t *, node, { + if (running_only && !node->is_running) + continue; + if (routerset_contains_node(routerset, node) && + !routerset_contains_node(excludeset, node)) + smartlist_add(out, (void*)node); + }); + } +} + +#if 0 +/** Add to <b>target</b> every node_t from <b>source</b> except: + * + * 1) Don't add it if <b>include</b> is non-empty and the relay isn't in + * <b>include</b>; and + * 2) Don't add it if <b>exclude</b> is non-empty and the relay is + * excluded in a more specific fashion by <b>exclude</b>. + * 3) If <b>running_only</b>, don't add non-running routers. + */ +void +routersets_get_node_disjunction(smartlist_t *target, + const smartlist_t *source, + const routerset_t *include, + const routerset_t *exclude, int running_only) +{ + SMARTLIST_FOREACH(source, const node_t *, node, { + int include_result; + if (running_only && !node->is_running) + continue; + if (!routerset_is_empty(include)) + include_result = routerset_contains_node(include, node); + else + include_result = 1; + + if (include_result) { + int exclude_result = routerset_contains_node(exclude, node); + if (include_result >= exclude_result) + smartlist_add(target, (void*)node); + } + }); +} +#endif + +/** Remove every node_t from <b>lst</b> that is in <b>routerset</b>. */ +void +routerset_subtract_nodes(smartlist_t *lst, const routerset_t *routerset) +{ + tor_assert(lst); + if (!routerset) + return; + SMARTLIST_FOREACH(lst, const node_t *, node, { + if (routerset_contains_node(routerset, node)) { + //log_debug(LD_DIR, "Subtracting %s",r->nickname); + SMARTLIST_DEL_CURRENT(lst, node); + } + }); +} + +/** Return a new string that when parsed by routerset_parse_string() will + * yield <b>set</b>. */ +char * +routerset_to_string(const routerset_t *set) +{ + if (!set || !set->list) + return tor_strdup(""); + return smartlist_join_strings(set->list, ",", 0, NULL); +} + +/** Helper: return true iff old and new are both NULL, or both non-NULL + * equal routersets. */ +int +routerset_equal(const routerset_t *old, const routerset_t *new) +{ + if (routerset_is_empty(old) && routerset_is_empty(new)) { + /* Two empty sets are equal */ + return 1; + } else if (routerset_is_empty(old) || routerset_is_empty(new)) { + /* An empty set is equal to nothing else. */ + return 0; + } + tor_assert(old != NULL); + tor_assert(new != NULL); + + if (smartlist_len(old->list) != smartlist_len(new->list)) + return 0; + + SMARTLIST_FOREACH(old->list, const char *, cp1, { + const char *cp2 = smartlist_get(new->list, cp1_sl_idx); + if (strcmp(cp1, cp2)) + return 0; + }); + + return 1; +} + +/** Free all storage held in <b>routerset</b>. */ +void +routerset_free(routerset_t *routerset) +{ + if (!routerset) + return; + + SMARTLIST_FOREACH(routerset->list, char *, cp, tor_free(cp)); + smartlist_free(routerset->list); + SMARTLIST_FOREACH(routerset->policies, addr_policy_t *, p, + addr_policy_free(p)); + smartlist_free(routerset->policies); + SMARTLIST_FOREACH(routerset->country_names, char *, cp, tor_free(cp)); + smartlist_free(routerset->country_names); + + strmap_free(routerset->names, NULL); + digestmap_free(routerset->digests, NULL); + bitarray_free(routerset->countries); + tor_free(routerset); +} + diff --git a/src/or/routerset.h b/src/or/routerset.h new file mode 100644 index 000000000..ad0832e4d --- /dev/null +++ b/src/or/routerset.h @@ -0,0 +1,48 @@ +/* Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file routerlist.h + * \brief Header file for routerset.c + **/ + +#ifndef TOR_ROUTERSET_H +#define TOR_ROUTERSET_H + +routerset_t *routerset_new(void); +void routerset_refresh_countries(routerset_t *rs); +int routerset_parse(routerset_t *target, const char *s, + const char *description); +void routerset_union(routerset_t *target, const routerset_t *source); +int routerset_is_list(const routerset_t *set); +int routerset_needs_geoip(const routerset_t *set); +int routerset_is_empty(const routerset_t *set); +int routerset_contains_router(const routerset_t *set, const routerinfo_t *ri, + country_t country); +int routerset_contains_routerstatus(const routerset_t *set, + const routerstatus_t *rs, + country_t country); +int routerset_contains_extendinfo(const routerset_t *set, + const extend_info_t *ei); + +int routerset_contains_node(const routerset_t *set, const node_t *node); +void routerset_get_all_nodes(smartlist_t *out, const routerset_t *routerset, + const routerset_t *excludeset, + int running_only); +#if 0 +void routersets_get_node_disjunction(smartlist_t *target, + const smartlist_t *source, + const routerset_t *include, + const routerset_t *exclude, int running_only); +#endif +void routerset_subtract_nodes(smartlist_t *out, + const routerset_t *routerset); + +char *routerset_to_string(const routerset_t *routerset); +int routerset_equal(const routerset_t *old, const routerset_t *new); +void routerset_free(routerset_t *routerset); + +#endif + diff --git a/src/or/statefile.c b/src/or/statefile.c new file mode 100644 index 000000000..499572a07 --- /dev/null +++ b/src/or/statefile.c @@ -0,0 +1,606 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "or.h" +#include "circuitbuild.h" +#include "config.h" +#include "confparse.h" +#include "hibernate.h" +#include "rephist.h" +#include "router.h" +#include "statefile.h" + +/** A list of state-file "abbreviations," for compatibility. */ +static config_abbrev_t _state_abbrevs[] = { + { "AccountingBytesReadInterval", "AccountingBytesReadInInterval", 0, 0 }, + { "HelperNode", "EntryGuard", 0, 0 }, + { "HelperNodeDownSince", "EntryGuardDownSince", 0, 0 }, + { "HelperNodeUnlistedSince", "EntryGuardUnlistedSince", 0, 0 }, + { "EntryNode", "EntryGuard", 0, 0 }, + { "EntryNodeDownSince", "EntryGuardDownSince", 0, 0 }, + { "EntryNodeUnlistedSince", "EntryGuardUnlistedSince", 0, 0 }, + { NULL, NULL, 0, 0}, +}; + +/*XXXX these next two are duplicates or near-duplicates from config.c */ +#define VAR(name,conftype,member,initvalue) \ + { name, CONFIG_TYPE_ ## conftype, STRUCT_OFFSET(or_state_t, member), \ + initvalue } +/** As VAR, but the option name and member name are the same. */ +#define V(member,conftype,initvalue) \ + VAR(#member, conftype, member, initvalue) + +/** Array of "state" variables saved to the ~/.tor/state file. */ +static config_var_t _state_vars[] = { + /* Remember to document these in state-contents.txt ! */ + + V(AccountingBytesReadInInterval, MEMUNIT, NULL), + V(AccountingBytesWrittenInInterval, MEMUNIT, NULL), + V(AccountingExpectedUsage, MEMUNIT, NULL), + V(AccountingIntervalStart, ISOTIME, NULL), + V(AccountingSecondsActive, INTERVAL, NULL), + V(AccountingSecondsToReachSoftLimit,INTERVAL, NULL), + V(AccountingSoftLimitHitAt, ISOTIME, NULL), + V(AccountingBytesAtSoftLimit, MEMUNIT, NULL), + + VAR("EntryGuard", LINELIST_S, EntryGuards, NULL), + VAR("EntryGuardDownSince", LINELIST_S, EntryGuards, NULL), + VAR("EntryGuardUnlistedSince", LINELIST_S, EntryGuards, NULL), + VAR("EntryGuardAddedBy", LINELIST_S, EntryGuards, NULL), + VAR("EntryGuardPathBias", LINELIST_S, EntryGuards, NULL), + V(EntryGuards, LINELIST_V, NULL), + + VAR("TransportProxy", LINELIST_S, TransportProxies, NULL), + V(TransportProxies, LINELIST_V, NULL), + + V(BWHistoryReadEnds, ISOTIME, NULL), + V(BWHistoryReadInterval, UINT, "900"), + V(BWHistoryReadValues, CSV, ""), + V(BWHistoryReadMaxima, CSV, ""), + V(BWHistoryWriteEnds, ISOTIME, NULL), + V(BWHistoryWriteInterval, UINT, "900"), + V(BWHistoryWriteValues, CSV, ""), + V(BWHistoryWriteMaxima, CSV, ""), + V(BWHistoryDirReadEnds, ISOTIME, NULL), + V(BWHistoryDirReadInterval, UINT, "900"), + V(BWHistoryDirReadValues, CSV, ""), + V(BWHistoryDirReadMaxima, CSV, ""), + V(BWHistoryDirWriteEnds, ISOTIME, NULL), + V(BWHistoryDirWriteInterval, UINT, "900"), + V(BWHistoryDirWriteValues, CSV, ""), + V(BWHistoryDirWriteMaxima, CSV, ""), + + V(TorVersion, STRING, NULL), + + V(LastRotatedOnionKey, ISOTIME, NULL), + V(LastWritten, ISOTIME, NULL), + + V(TotalBuildTimes, UINT, NULL), + V(CircuitBuildAbandonedCount, UINT, "0"), + VAR("CircuitBuildTimeBin", LINELIST_S, BuildtimeHistogram, NULL), + VAR("BuildtimeHistogram", LINELIST_V, BuildtimeHistogram, NULL), + { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } +}; + +#undef VAR +#undef V + +static int or_state_validate(or_state_t *old_options, or_state_t *options, + int from_setconf, char **msg); + +/** Magic value for or_state_t. */ +#define OR_STATE_MAGIC 0x57A73f57 + +/** "Extra" variable in the state that receives lines we can't parse. This + * lets us preserve options from versions of Tor newer than us. */ +static config_var_t state_extra_var = { + "__extra", CONFIG_TYPE_LINELIST, STRUCT_OFFSET(or_state_t, ExtraLines), NULL +}; + +/** Configuration format for or_state_t. */ +static const config_format_t state_format = { + sizeof(or_state_t), + OR_STATE_MAGIC, + STRUCT_OFFSET(or_state_t, _magic), + _state_abbrevs, + _state_vars, + (validate_fn_t)or_state_validate, + &state_extra_var, +}; + +/** Persistent serialized state. */ +static or_state_t *global_state = NULL; + +/** Return the persistent state struct for this Tor. */ +or_state_t * +get_or_state(void) +{ + tor_assert(global_state); + return global_state; +} + +/** Return true iff we have loaded the global state for this Tor */ +int +or_state_loaded(void) +{ + return global_state != NULL; +} + +/** Return true if <b>line</b> is a valid state TransportProxy line. + * Return false otherwise. */ +static int +state_transport_line_is_valid(const char *line) +{ + smartlist_t *items = NULL; + char *addrport=NULL; + tor_addr_t addr; + uint16_t port = 0; + int r; + + items = smartlist_new(); + smartlist_split_string(items, line, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + + if (smartlist_len(items) != 2) { + log_warn(LD_CONFIG, "state: Not enough arguments in TransportProxy line."); + goto err; + } + + addrport = smartlist_get(items, 1); + if (tor_addr_port_lookup(addrport, &addr, &port) < 0) { + log_warn(LD_CONFIG, "state: Could not parse addrport."); + goto err; + } + + if (!port) { + log_warn(LD_CONFIG, "state: Transport line did not contain port."); + goto err; + } + + r = 1; + goto done; + + err: + r = 0; + + done: + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + return r; +} + +/** Return 0 if all TransportProxy lines in <b>state</b> are well + * formed. Otherwise, return -1. */ +static int +validate_transports_in_state(or_state_t *state) +{ + int broken = 0; + config_line_t *line; + + for (line = state->TransportProxies ; line ; line = line->next) { + tor_assert(!strcmp(line->key, "TransportProxy")); + if (!state_transport_line_is_valid(line->value)) + broken = 1; + } + + if (broken) + log_warn(LD_CONFIG, "state: State file seems to be broken."); + + return 0; +} + +/** Return 0 if every setting in <b>state</b> is reasonable, and a + * permissible transition from <b>old_state</b>. Else warn and return -1. + * Should have no side effects, except for normalizing the contents of + * <b>state</b>. + */ +/* XXX from_setconf is here because of bug 238 */ +static int +or_state_validate(or_state_t *old_state, or_state_t *state, + int from_setconf, char **msg) +{ + /* We don't use these; only options do. Still, we need to match that + * signature. */ + (void) from_setconf; + (void) old_state; + + if (entry_guards_parse_state(state, 0, msg)<0) + return -1; + + if (validate_transports_in_state(state)<0) + return -1; + + return 0; +} + +/** Replace the current persistent state with <b>new_state</b> */ +static int +or_state_set(or_state_t *new_state) +{ + char *err = NULL; + int ret = 0; + tor_assert(new_state); + config_free(&state_format, global_state); + global_state = new_state; + if (entry_guards_parse_state(global_state, 1, &err)<0) { + log_warn(LD_GENERAL,"%s",err); + tor_free(err); + ret = -1; + } + if (rep_hist_load_state(global_state, &err)<0) { + log_warn(LD_GENERAL,"Unparseable bandwidth history state: %s",err); + tor_free(err); + ret = -1; + } + if (circuit_build_times_parse_state(&circ_times, global_state) < 0) { + ret = -1; + } + return ret; +} + +/** + * Save a broken state file to a backup location. + */ +static void +or_state_save_broken(char *fname) +{ + int i; + file_status_t status; + char *fname2 = NULL; + for (i = 0; i < 100; ++i) { + tor_asprintf(&fname2, "%s.%d", fname, i); + status = file_status(fname2); + if (status == FN_NOENT) + break; + tor_free(fname2); + } + if (i == 100) { + log_warn(LD_BUG, "Unable to parse state in \"%s\"; too many saved bad " + "state files to move aside. Discarding the old state file.", + fname); + unlink(fname); + } else { + log_warn(LD_BUG, "Unable to parse state in \"%s\". Moving it aside " + "to \"%s\". This could be a bug in Tor; please tell " + "the developers.", fname, fname2); + if (rename(fname, fname2) < 0) { + log_warn(LD_BUG, "Weirdly, I couldn't even move the state aside. The " + "OS gave an error of %s", strerror(errno)); + } + } + tor_free(fname2); +} + +/** Reload the persistent state from disk, generating a new state as needed. + * Return 0 on success, less than 0 on failure. + */ +int +or_state_load(void) +{ + or_state_t *new_state = NULL; + char *contents = NULL, *fname; + char *errmsg = NULL; + int r = -1, badstate = 0; + + fname = get_datadir_fname("state"); + switch (file_status(fname)) { + case FN_FILE: + if (!(contents = read_file_to_str(fname, 0, NULL))) { + log_warn(LD_FS, "Unable to read state file \"%s\"", fname); + goto done; + } + break; + case FN_NOENT: + break; + case FN_ERROR: + case FN_DIR: + default: + log_warn(LD_GENERAL,"State file \"%s\" is not a file? Failing.", fname); + goto done; + } + new_state = tor_malloc_zero(sizeof(or_state_t)); + new_state->_magic = OR_STATE_MAGIC; + config_init(&state_format, new_state); + if (contents) { + config_line_t *lines=NULL; + int assign_retval; + if (config_get_lines(contents, &lines, 0)<0) + goto done; + assign_retval = config_assign(&state_format, new_state, + lines, 0, 0, &errmsg); + config_free_lines(lines); + if (assign_retval<0) + badstate = 1; + if (errmsg) { + log_warn(LD_GENERAL, "%s", errmsg); + tor_free(errmsg); + } + } + + if (!badstate && or_state_validate(NULL, new_state, 1, &errmsg) < 0) + badstate = 1; + + if (errmsg) { + log_warn(LD_GENERAL, "%s", errmsg); + tor_free(errmsg); + } + + if (badstate && !contents) { + log_warn(LD_BUG, "Uh oh. We couldn't even validate our own default state." + " This is a bug in Tor."); + goto done; + } else if (badstate && contents) { + or_state_save_broken(fname); + + tor_free(contents); + config_free(&state_format, new_state); + + new_state = tor_malloc_zero(sizeof(or_state_t)); + new_state->_magic = OR_STATE_MAGIC; + config_init(&state_format, new_state); + } else if (contents) { + log_info(LD_GENERAL, "Loaded state from \"%s\"", fname); + } else { + log_info(LD_GENERAL, "Initialized state"); + } + if (or_state_set(new_state) == -1) { + or_state_save_broken(fname); + } + new_state = NULL; + if (!contents) { + global_state->next_write = 0; + or_state_save(time(NULL)); + } + r = 0; + + done: + tor_free(fname); + tor_free(contents); + if (new_state) + config_free(&state_format, new_state); + + return r; +} + +/** Did the last time we tried to write the state file fail? If so, we + * should consider disabling such features as preemptive circuit generation + * to compute circuit-build-time. */ +static int last_state_file_write_failed = 0; + +/** Return whether the state file failed to write last time we tried. */ +int +did_last_state_file_write_fail(void) +{ + return last_state_file_write_failed; +} + +/** If writing the state to disk fails, try again after this many seconds. */ +#define STATE_WRITE_RETRY_INTERVAL 3600 + +/** If we're a relay, how often should we checkpoint our state file even + * if nothing else dirties it? This will checkpoint ongoing stats like + * bandwidth used, per-country user stats, etc. */ +#define STATE_RELAY_CHECKPOINT_INTERVAL (12*60*60) + +/** Write the persistent state to disk. Return 0 for success, <0 on failure. */ +int +or_state_save(time_t now) +{ + char *state, *contents; + char tbuf[ISO_TIME_LEN+1]; + char *fname; + + tor_assert(global_state); + + if (global_state->next_write > now) + return 0; + + /* Call everything else that might dirty the state even more, in order + * to avoid redundant writes. */ + entry_guards_update_state(global_state); + rep_hist_update_state(global_state); + circuit_build_times_update_state(&circ_times, global_state); + if (accounting_is_enabled(get_options())) + accounting_run_housekeeping(now); + + global_state->LastWritten = now; + + tor_free(global_state->TorVersion); + tor_asprintf(&global_state->TorVersion, "Tor %s", get_version()); + + state = config_dump(&state_format, NULL, global_state, 1, 0); + format_local_iso_time(tbuf, now); + tor_asprintf(&contents, + "# Tor state file last generated on %s local time\n" + "# Other times below are in GMT\n" + "# You *do not* need to edit this file.\n\n%s", + tbuf, state); + tor_free(state); + fname = get_datadir_fname("state"); + if (write_str_to_file(fname, contents, 0)<0) { + log_warn(LD_FS, "Unable to write state to file \"%s\"; " + "will try again later", fname); + last_state_file_write_failed = 1; + tor_free(fname); + tor_free(contents); + /* Try again after STATE_WRITE_RETRY_INTERVAL (or sooner, if the state + * changes sooner). */ + global_state->next_write = now + STATE_WRITE_RETRY_INTERVAL; + return -1; + } + + last_state_file_write_failed = 0; + log_info(LD_GENERAL, "Saved state to \"%s\"", fname); + tor_free(fname); + tor_free(contents); + + if (server_mode(get_options())) + global_state->next_write = now + STATE_RELAY_CHECKPOINT_INTERVAL; + else + global_state->next_write = TIME_MAX; + + return 0; +} + +/** Return the config line for transport <b>transport</b> in the current state. + * Return NULL if there is no config line for <b>transport</b>. */ +static config_line_t * +get_transport_in_state_by_name(const char *transport) +{ + or_state_t *or_state = get_or_state(); + config_line_t *line; + config_line_t *ret = NULL; + smartlist_t *items = NULL; + + for (line = or_state->TransportProxies ; line ; line = line->next) { + tor_assert(!strcmp(line->key, "TransportProxy")); + + items = smartlist_new(); + smartlist_split_string(items, line->value, NULL, + SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, -1); + if (smartlist_len(items) != 2) /* broken state */ + goto done; + + if (!strcmp(smartlist_get(items, 0), transport)) { + ret = line; + goto done; + } + + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + items = NULL; + } + + done: + if (items) { + SMARTLIST_FOREACH(items, char*, s, tor_free(s)); + smartlist_free(items); + } + return ret; +} + +/** Return string containing the address:port part of the + * TransportProxy <b>line</b> for transport <b>transport</b>. + * If the line is corrupted, return NULL. */ +static const char * +get_transport_bindaddr(const char *line, const char *transport) +{ + char *line_tmp = NULL; + + if (strlen(line) < strlen(transport) + 2) { + goto broken_state; + } else { + /* line should start with the name of the transport and a space. + (for example, "obfs2 127.0.0.1:47245") */ + tor_asprintf(&line_tmp, "%s ", transport); + if (strcmpstart(line, line_tmp)) + goto broken_state; + + tor_free(line_tmp); + return (line+strlen(transport)+1); + } + + broken_state: + tor_free(line_tmp); + return NULL; +} + +/** Return a string containing the address:port that a proxy transport + * should bind on. The string is stored on the heap and must be freed + * by the caller of this function. */ +char * +get_stored_bindaddr_for_server_transport(const char *transport) +{ + char *default_addrport = NULL; + const char *stored_bindaddr = NULL; + + config_line_t *line = get_transport_in_state_by_name(transport); + if (!line) /* Found no references in state for this transport. */ + goto no_bindaddr_found; + + stored_bindaddr = get_transport_bindaddr(line->value, transport); + if (stored_bindaddr) /* found stored bindaddr in state file. */ + return tor_strdup(stored_bindaddr); + + no_bindaddr_found: + /** If we didn't find references for this pluggable transport in the + state file, we should instruct the pluggable transport proxy to + listen on INADDR_ANY on a random ephemeral port. */ + tor_asprintf(&default_addrport, "%s:%s", fmt_addr32(INADDR_ANY), "0"); + return default_addrport; +} + +/** Save <b>transport</b> listening on <b>addr</b>:<b>port</b> to + state */ +void +save_transport_to_state(const char *transport, + const tor_addr_t *addr, uint16_t port) +{ + or_state_t *state = get_or_state(); + + char *transport_addrport=NULL; + + /** find where to write on the state */ + config_line_t **next, *line; + + /* see if this transport is already stored in state */ + config_line_t *transport_line = + get_transport_in_state_by_name(transport); + + if (transport_line) { /* if transport already exists in state... */ + const char *prev_bindaddr = /* get its addrport... */ + get_transport_bindaddr(transport_line->value, transport); + tor_asprintf(&transport_addrport, "%s:%d", fmt_addr(addr), (int)port); + + /* if transport in state has the same address as this one, life is good */ + if (!strcmp(prev_bindaddr, transport_addrport)) { + log_info(LD_CONFIG, "Transport seems to have spawned on its usual " + "address:port."); + goto done; + } else { /* if addrport in state is different than the one we got */ + log_info(LD_CONFIG, "Transport seems to have spawned on different " + "address:port. Let's update the state file with the new " + "address:port"); + tor_free(transport_line->value); /* free the old line */ + tor_asprintf(&transport_line->value, "%s %s:%d", transport, + fmt_addr(addr), + (int) port); /* replace old addrport line with new line */ + } + } else { /* never seen this one before; save it in state for next time */ + log_info(LD_CONFIG, "It's the first time we see this transport. " + "Let's save its address:port"); + next = &state->TransportProxies; + /* find the last TransportProxy line in the state and point 'next' + right after it */ + line = state->TransportProxies; + while (line) { + next = &(line->next); + line = line->next; + } + + /* allocate space for the new line and fill it in */ + *next = line = tor_malloc_zero(sizeof(config_line_t)); + line->key = tor_strdup("TransportProxy"); + tor_asprintf(&line->value, "%s %s:%d", transport, + fmt_addr(addr), (int) port); + + next = &(line->next); + } + + if (!get_options()->AvoidDiskWrites) + or_state_mark_dirty(state, 0); + + done: + tor_free(transport_addrport); +} + +void +or_state_free_all(void) +{ + config_free(&state_format, global_state); + global_state = NULL; +} + diff --git a/src/or/statefile.h b/src/or/statefile.h new file mode 100644 index 000000000..4770d500d --- /dev/null +++ b/src/or/statefile.h @@ -0,0 +1,22 @@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#ifndef TOR_STATEFILE_H +#define TOR_STATEFILE_H + +or_state_t *get_or_state(void); +int did_last_state_file_write_fail(void); +int or_state_save(time_t now); + +void save_transport_to_state(const char *transport_name, + const tor_addr_t *addr, uint16_t port); +char *get_stored_bindaddr_for_server_transport(const char *transport); +int or_state_load(void); +int or_state_loaded(void); +void or_state_free_all(void); + +#endif + diff --git a/src/or/transports.c b/src/or/transports.c index 4ba239562..34fe679dd 100644 --- a/src/or/transports.c +++ b/src/or/transports.c @@ -39,13 +39,17 @@ * transport_t structs. * * When the managed proxy stops spitting METHOD lines (signified by a - * '{S,C}METHODS DONE' message) we register all the transports - * collected to the circuitbuild.c subsystem. At this point, the - * pointers to transport_t can be transformed into dangling pointers - * at any point by the circuitbuild.c subsystem, and so we replace all - * transport_t pointers with strings describing the transport names. - * We can still go from a transport name to a transport_t using the - * fact that each transport name uniquely identifies a transport_t. + * '{S,C}METHODS DONE' message) we pass copies of its transports to + * the bridge subsystem. We keep copies of the 'transport_t's on the + * managed proxy to be able to associate the proxy with its + * transports, and we pass copies to the bridge subsystem so that + * transports can be associated with bridges. + * [ XXX We should try see whether the two copies are really needed + * and maybe cut it into a single copy of the 'transport_t' shared + * between the managed proxy and the bridge subsystem. Preliminary + * analysis shows that both copies are needed with the current code + * logic, because of race conditions that can cause dangling + * pointers. ] * * <b>In even more detail, this is what happens when a SIGHUP * occurs:</b> @@ -90,6 +94,7 @@ #include "transports.h" #include "util.h" #include "router.h" +#include "statefile.h" static process_environment_t * create_managed_proxy_environment(const managed_proxy_t *mp); @@ -127,6 +132,219 @@ static INLINE void free_execve_args(char **arg); protocol version. */ #define PROTO_VERSION_ONE 1 +/** A list of pluggable transports found in torrc. */ +static smartlist_t *transport_list = NULL; + +/** Returns a transport_t struct for a transport proxy supporting the + protocol <b>name</b> listening at <b>addr</b>:<b>port</b> using + SOCKS version <b>socks_ver</b>. */ +static transport_t * +transport_new(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver) +{ + transport_t *t = tor_malloc_zero(sizeof(transport_t)); + + tor_addr_copy(&t->addr, addr); + t->port = port; + t->name = tor_strdup(name); + t->socks_version = socks_ver; + + return t; +} + +/** Free the pluggable transport struct <b>transport</b>. */ +void +transport_free(transport_t *transport) +{ + if (!transport) + return; + + tor_free(transport->name); + tor_free(transport); +} + +/** Mark every entry of the transport list to be removed on our next call to + * sweep_transport_list unless it has first been un-marked. */ +void +mark_transport_list(void) +{ + if (!transport_list) + transport_list = smartlist_new(); + SMARTLIST_FOREACH(transport_list, transport_t *, t, + t->marked_for_removal = 1); +} + +/** Remove every entry of the transport list that was marked with + * mark_transport_list if it has not subsequently been un-marked. */ +void +sweep_transport_list(void) +{ + if (!transport_list) + transport_list = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(transport_list, transport_t *, t) { + if (t->marked_for_removal) { + SMARTLIST_DEL_CURRENT(transport_list, t); + transport_free(t); + } + } SMARTLIST_FOREACH_END(t); +} + +/** Initialize the pluggable transports list to empty, creating it if + * needed. */ +static void +clear_transport_list(void) +{ + if (!transport_list) + transport_list = smartlist_new(); + SMARTLIST_FOREACH(transport_list, transport_t *, t, transport_free(t)); + smartlist_clear(transport_list); +} + +/** Return a deep copy of <b>transport</b>. */ +static transport_t * +transport_copy(const transport_t *transport) +{ + transport_t *new_transport = NULL; + + tor_assert(transport); + + new_transport = tor_malloc_zero(sizeof(transport_t)); + + new_transport->socks_version = transport->socks_version; + new_transport->name = tor_strdup(transport->name); + tor_addr_copy(&new_transport->addr, &transport->addr); + new_transport->port = transport->port; + new_transport->marked_for_removal = transport->marked_for_removal; + + return new_transport; +} + +/** Returns the transport in our transport list that has the name <b>name</b>. + * Else returns NULL. */ +transport_t * +transport_get_by_name(const char *name) +{ + tor_assert(name); + + if (!transport_list) + return NULL; + + SMARTLIST_FOREACH_BEGIN(transport_list, transport_t *, transport) { + if (!strcmp(transport->name, name)) + return transport; + } SMARTLIST_FOREACH_END(transport); + + return NULL; +} + +/** Resolve any conflicts that the insertion of transport <b>t</b> + * might cause. + * Return 0 if <b>t</b> is OK and should be registered, 1 if there is + * a transport identical to <b>t</b> already registered and -1 if + * <b>t</b> cannot be added due to conflicts. */ +static int +transport_resolve_conflicts(const transport_t *t) +{ + /* This is how we resolve transport conflicts: + + If there is already a transport with the same name and addrport, + we either have duplicate torrc lines OR we are here post-HUP and + this transport was here pre-HUP as well. In any case, mark the + old transport so that it doesn't get removed and ignore the new + one. Our caller has to free the new transport so we return '1' to + signify this. + + If there is already a transport with the same name but different + addrport: + * if it's marked for removal, it means that it either has a lower + priority than 't' in torrc (otherwise the mark would have been + cleared by the paragraph above), or it doesn't exist at all in + the post-HUP torrc. We destroy the old transport and register 't'. + * if it's *not* marked for removal, it means that it was newly + added in the post-HUP torrc or that it's of higher priority, in + this case we ignore 't'. */ + transport_t *t_tmp = transport_get_by_name(t->name); + if (t_tmp) { /* same name */ + if (tor_addr_eq(&t->addr, &t_tmp->addr) && (t->port == t_tmp->port)) { + /* same name *and* addrport */ + t_tmp->marked_for_removal = 0; + return 1; + } else { /* same name but different addrport */ + if (t_tmp->marked_for_removal) { /* marked for removal */ + log_notice(LD_GENERAL, "You tried to add transport '%s' at '%s:%u' " + "but there was already a transport marked for deletion at " + "'%s:%u'. We deleted the old transport and registered the " + "new one.", t->name, fmt_addr(&t->addr), t->port, + fmt_addr(&t_tmp->addr), t_tmp->port); + smartlist_remove(transport_list, t_tmp); + transport_free(t_tmp); + } else { /* *not* marked for removal */ + log_notice(LD_GENERAL, "You tried to add transport '%s' at '%s:%u' " + "but the same transport already exists at '%s:%u'. " + "Skipping.", t->name, fmt_addr(&t->addr), t->port, + fmt_addr(&t_tmp->addr), t_tmp->port); + return -1; + } + } + } + + return 0; +} + +/** Add transport <b>t</b> to the internal list of pluggable + * transports. + * Returns 0 if the transport was added correctly, 1 if the same + * transport was already registered (in this case the caller must + * free the transport) and -1 if there was an error. */ +static int +transport_add(transport_t *t) +{ + int r; + tor_assert(t); + + r = transport_resolve_conflicts(t); + + switch (r) { + case 0: /* should register transport */ + if (!transport_list) + transport_list = smartlist_new(); + smartlist_add(transport_list, t); + return 0; + default: /* let our caller know the return code */ + return r; + } +} + +/** Remember a new pluggable transport proxy at <b>addr</b>:<b>port</b>. + * <b>name</b> is set to the name of the protocol this proxy uses. + * <b>socks_ver</b> is set to the SOCKS version of the proxy. */ +int +transport_add_from_config(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver) +{ + transport_t *t = transport_new(addr, port, name, socks_ver); + + int r = transport_add(t); + + switch (r) { + case -1: + default: + log_notice(LD_GENERAL, "Could not add transport %s at %s:%u. Skipping.", + t->name, fmt_addr(&t->addr), t->port); + transport_free(t); + return -1; + case 1: + log_info(LD_GENERAL, "Succesfully registered transport %s at %s:%u.", + t->name, fmt_addr(&t->addr), t->port); + transport_free(t); /* falling */ + return 0; + case 0: + log_info(LD_GENERAL, "Succesfully registered transport %s at %s:%u.", + t->name, fmt_addr(&t->addr), t->port); + return 0; + } +} + /** List of unconfigured managed proxies. */ static smartlist_t *managed_proxy_list = NULL; /** Number of still unconfigured proxies. */ @@ -217,11 +435,11 @@ proxy_needs_restart(const managed_proxy_t *mp) { /* mp->transport_to_launch is populated with the names of the transports that must be launched *after* the SIGHUP. - mp->transports is populated with the names of the transports that - were launched *before* the SIGHUP. + mp->transports is populated with the transports that were + launched *before* the SIGHUP. - If the two lists contain the same strings, we don't need to - restart the proxy, since it already does what we want. */ + Check if all the transports that need to be launched are already + launched: */ tor_assert(smartlist_len(mp->transports_to_launch) > 0); tor_assert(mp->conf_state == PT_PROTO_COMPLETED); @@ -229,11 +447,11 @@ proxy_needs_restart(const managed_proxy_t *mp) if (smartlist_len(mp->transports_to_launch) != smartlist_len(mp->transports)) goto needs_restart; - SMARTLIST_FOREACH_BEGIN(mp->transports_to_launch, char *, t_t_l) { - if (!smartlist_string_isin(mp->transports, t_t_l)) + SMARTLIST_FOREACH_BEGIN(mp->transports, const transport_t *, t) { + if (!smartlist_string_isin(mp->transports_to_launch, t->name)) goto needs_restart; - } SMARTLIST_FOREACH_END(t_t_l); + } SMARTLIST_FOREACH_END(t); return 0; @@ -245,6 +463,7 @@ proxy_needs_restart(const managed_proxy_t *mp) * preparations and then flag its state so that it will be relaunched * in the next tick. */ static void + proxy_prepare_for_restart(managed_proxy_t *mp) { transport_t *t_tmp = NULL; @@ -255,16 +474,17 @@ proxy_prepare_for_restart(managed_proxy_t *mp) tor_process_handle_destroy(mp->process_handle, 1); mp->process_handle = NULL; - /* destroy all its old transports. we no longer use them. */ - SMARTLIST_FOREACH_BEGIN(mp->transports, const char *, t_name) { - t_tmp = transport_get_by_name(t_name); + /* destroy all its registered transports, since we will no longer + use them. */ + SMARTLIST_FOREACH_BEGIN(mp->transports, const transport_t *, t) { + t_tmp = transport_get_by_name(t->name); if (t_tmp) t_tmp->marked_for_removal = 1; - } SMARTLIST_FOREACH_END(t_name); + } SMARTLIST_FOREACH_END(t); sweep_transport_list(); - /* free the transport names in mp->transports */ - SMARTLIST_FOREACH(mp->transports, char *, t_name, tor_free(t_name)); + /* free the transport in mp->transports */ + SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); smartlist_clear(mp->transports); /* flag it as an infant proxy so that it gets launched on next tick */ @@ -315,6 +535,7 @@ launch_managed_proxy(managed_proxy_t *mp) void pt_configure_remaining_proxies(void) { + int at_least_a_proxy_config_finished = 0; smartlist_t *tmp = smartlist_new(); log_debug(LD_CONFIG, "Configuring remaining managed proxies (%d)!", @@ -352,22 +573,25 @@ pt_configure_remaining_proxies(void) if (!proxy_configuration_finished(mp)) configure_proxy(mp); + if (proxy_configuration_finished(mp)) + at_least_a_proxy_config_finished = 1; + } SMARTLIST_FOREACH_END(mp); smartlist_free(tmp); check_if_restarts_needed = 0; assert_unconfigured_count_ok(); -} -#ifdef _WIN32 + if (at_least_a_proxy_config_finished) + mark_my_descriptor_dirty("configured managed proxies"); +} /** Attempt to continue configuring managed proxy <b>mp</b>. */ static void configure_proxy(managed_proxy_t *mp) { - int pos; - char stdout_buf[200]; - smartlist_t *lines = NULL; + smartlist_t *proxy_output = NULL; + enum stream_status stream_status = 0; /* if we haven't launched the proxy yet, do it now */ if (mp->conf_state == PT_PROTO_INFANT) { @@ -381,28 +605,18 @@ configure_proxy(managed_proxy_t *mp) tor_assert(mp->conf_state != PT_PROTO_INFANT); tor_assert(mp->process_handle); - pos = tor_read_all_handle(tor_process_get_stdout_pipe(mp->process_handle), - stdout_buf, sizeof(stdout_buf) - 1, NULL); - if (pos < 0) { - log_notice(LD_GENERAL, "Failed to read data from managed proxy '%s'.", - mp->argv[0]); - mp->conf_state = PT_PROTO_BROKEN; + proxy_output = + tor_get_lines_from_handle(tor_process_get_stdout_pipe(mp->process_handle), + &stream_status); + if (!proxy_output) { /* failed to get input from proxy */ + if (stream_status != IO_STREAM_EAGAIN) + mp->conf_state = PT_PROTO_BROKEN; + goto done; } - if (pos == 0) /* proxy has nothing interesting to say. */ - return; - - /* End with a null even if there isn't a \r\n at the end */ - /* TODO: What if this is a partial line? */ - stdout_buf[pos] = '\0'; - - /* Split up the buffer */ - lines = smartlist_new(); - tor_split_lines(lines, stdout_buf, pos); - /* Handle lines. */ - SMARTLIST_FOREACH_BEGIN(lines, const char *, line) { + SMARTLIST_FOREACH_BEGIN(proxy_output, const char *, line) { handle_proxy_line(line, mp); if (proxy_configuration_finished(mp)) goto done; @@ -413,123 +627,56 @@ configure_proxy(managed_proxy_t *mp) if (proxy_configuration_finished(mp)) handle_finished_proxy(mp); - if (lines) - smartlist_free(lines); -} - -#else /* _WIN32 */ - -/** Attempt to continue configuring managed proxy <b>mp</b>. */ -static void -configure_proxy(managed_proxy_t *mp) -{ - enum stream_status r; - char stdout_buf[200]; - - /* if we haven't launched the proxy yet, do it now */ - if (mp->conf_state == PT_PROTO_INFANT) { - if (launch_managed_proxy(mp) < 0) { /* launch fail */ - mp->conf_state = PT_PROTO_FAILED_LAUNCH; - handle_finished_proxy(mp); - } - return; - } - - tor_assert(mp->conf_state != PT_PROTO_INFANT); - tor_assert(mp->process_handle); - - while (1) { - r = get_string_from_pipe(tor_process_get_stdout_pipe(mp->process_handle), - stdout_buf, sizeof(stdout_buf) - 1); - - if (r == IO_STREAM_OKAY) { /* got a line; handle it! */ - handle_proxy_line((const char *)stdout_buf, mp); - } else if (r == IO_STREAM_EAGAIN) { /* check back later */ - return; - } else if (r == IO_STREAM_CLOSED || r == IO_STREAM_TERM) { /* snap! */ - log_warn(LD_GENERAL, "Our communication channel with the managed proxy " - "'%s' closed. Most probably application stopped running.", - mp->argv[0]); - mp->conf_state = PT_PROTO_BROKEN; - } else { /* unknown stream status */ - log_warn(LD_BUG, "Unknown stream status '%d' while configuring managed " - "proxy '%s'.", (int)r, mp->argv[0]); - } - - /* if the proxy finished configuring, exit the loop. */ - if (proxy_configuration_finished(mp)) { - handle_finished_proxy(mp); - return; - } + if (proxy_output) { + SMARTLIST_FOREACH(proxy_output, char *, cp, tor_free(cp)); + smartlist_free(proxy_output); } } -#endif /* _WIN32 */ - /** Register server managed proxy <b>mp</b> transports to state */ static void -register_server_proxy(managed_proxy_t *mp) +register_server_proxy(const managed_proxy_t *mp) { - /* After we register this proxy's transports, we switch its - mp->transports to a list containing strings of its transport - names. (See transports.h) */ - smartlist_t *sm_tmp = smartlist_new(); - tor_assert(mp->conf_state != PT_PROTO_COMPLETED); + SMARTLIST_FOREACH_BEGIN(mp->transports, transport_t *, t) { save_transport_to_state(t->name, &t->addr, t->port); log_notice(LD_GENERAL, "Registered server transport '%s' at '%s:%d'", t->name, fmt_addr(&t->addr), (int)t->port); - smartlist_add(sm_tmp, tor_strdup(t->name)); } SMARTLIST_FOREACH_END(t); - - /* Since server proxies don't register their transports in the - circuitbuild.c subsystem, it's our duty to free them when we - switch mp->transports to strings. */ - SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); - smartlist_free(mp->transports); - - mp->transports = sm_tmp; } /** Register all the transports supported by client managed proxy * <b>mp</b> to the bridge subsystem. */ static void -register_client_proxy(managed_proxy_t *mp) +register_client_proxy(const managed_proxy_t *mp) { int r; - /* After we register this proxy's transports, we switch its - mp->transports to a list containing strings of its transport - names. (See transports.h) */ - smartlist_t *sm_tmp = smartlist_new(); tor_assert(mp->conf_state != PT_PROTO_COMPLETED); + SMARTLIST_FOREACH_BEGIN(mp->transports, transport_t *, t) { - r = transport_add(t); + transport_t *transport_tmp = transport_copy(t); + r = transport_add(transport_tmp); switch (r) { case -1: log_notice(LD_GENERAL, "Could not add transport %s. Skipping.", t->name); - transport_free(t); + transport_free(transport_tmp); break; case 0: log_info(LD_GENERAL, "Succesfully registered transport %s", t->name); - smartlist_add(sm_tmp, tor_strdup(t->name)); break; case 1: log_info(LD_GENERAL, "Succesfully registered transport %s", t->name); - smartlist_add(sm_tmp, tor_strdup(t->name)); - transport_free(t); + transport_free(transport_tmp); break; } } SMARTLIST_FOREACH_END(t); - - smartlist_free(mp->transports); - mp->transports = sm_tmp; } /** Register the transports of managed proxy <b>mp</b>. */ static INLINE void -register_proxy(managed_proxy_t *mp) +register_proxy(const managed_proxy_t *mp) { if (mp->is_server) register_server_proxy(mp); @@ -542,10 +689,7 @@ static void managed_proxy_destroy(managed_proxy_t *mp, int also_terminate_process) { - if (mp->conf_state != PT_PROTO_COMPLETED) - SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); - else - SMARTLIST_FOREACH(mp->transports, char *, t_name, tor_free(t_name)); + SMARTLIST_FOREACH(mp->transports, transport_t *, t, transport_free(t)); /* free the transports smartlist */ smartlist_free(mp->transports); @@ -1181,6 +1325,91 @@ pt_prepare_proxy_list_for_config_read(void) tor_assert(unconfigured_proxies_n == 0); } +/** Return a smartlist containing the ports where our pluggable + * transports are listening. */ +smartlist_t * +get_transport_proxy_ports(void) +{ + smartlist_t *sl = NULL; + + if (!managed_proxy_list) + return NULL; + + /** XXX assume that external proxy ports have been forwarded + manually */ + SMARTLIST_FOREACH_BEGIN(managed_proxy_list, const managed_proxy_t *, mp) { + if (!mp->is_server || mp->conf_state != PT_PROTO_COMPLETED) + continue; + + if (!sl) sl = smartlist_new(); + + tor_assert(mp->transports); + SMARTLIST_FOREACH(mp->transports, const transport_t *, t, + smartlist_add_asprintf(sl, "%u:%u", t->port, t->port)); + + } SMARTLIST_FOREACH_END(mp); + + return sl; +} + +/** Return the pluggable transport string that we should display in + * our extra-info descriptor. If we shouldn't display such a string, + * or we have nothing to display, return NULL. The string is + * allocated on the heap and it's the responsibility of the caller to + * free it. */ +char * +pt_get_extra_info_descriptor_string(void) +{ + char *the_string = NULL; + smartlist_t *string_chunks = NULL; + + if (!managed_proxy_list) + return NULL; + + string_chunks = smartlist_new(); + + /* For each managed proxy, add its transports to the chunks list. */ + SMARTLIST_FOREACH_BEGIN(managed_proxy_list, const managed_proxy_t *, mp) { + if ((!mp->is_server) || (mp->conf_state != PT_PROTO_COMPLETED)) + continue; + + tor_assert(mp->transports); + + SMARTLIST_FOREACH_BEGIN(mp->transports, const transport_t *, t) { + /* If the transport proxy returned "0.0.0.0" as its address, and + * we know our external IP address, use it. Otherwise, use the + * returned address. */ + const char *addr_str = fmt_addr(&t->addr); + uint32_t external_ip_address = 0; + if (tor_addr_is_null(&t->addr) && + router_pick_published_address(get_options(), + &external_ip_address) >= 0) { + /* returned addr was 0.0.0.0 and we found our external IP + address: use it. */ + addr_str = fmt_addr32(external_ip_address); + } + + smartlist_add_asprintf(string_chunks, + "transport %s %s:%u", + t->name, addr_str, t->port); + } SMARTLIST_FOREACH_END(t); + + } SMARTLIST_FOREACH_END(mp); + + if (smartlist_len(string_chunks) == 0) { + smartlist_free(string_chunks); + return NULL; + } + + /* Join all the chunks into the final string. */ + the_string = smartlist_join_strings(string_chunks, "\n", 1, NULL); + + SMARTLIST_FOREACH(string_chunks, char *, s, tor_free(s)); + smartlist_free(string_chunks); + + return the_string; +} + /** The tor config was read. * Destroy all managed proxies that were marked by a previous call to * prepare_proxy_list_for_config_read() and are not used by the new @@ -1204,6 +1433,12 @@ sweep_proxy_list(void) void pt_free_all(void) { + if (transport_list) { + clear_transport_list(); + smartlist_free(transport_list); + transport_list = NULL; + } + if (managed_proxy_list) { /* If the proxy is in PT_PROTO_COMPLETED, it has registered its transports and it's the duty of the circuitbuild.c subsystem to diff --git a/src/or/transports.h b/src/or/transports.h index 02f159a5d..86a2530fc 100644 --- a/src/or/transports.h +++ b/src/or/transports.h @@ -11,6 +11,30 @@ #ifndef TOR_TRANSPORTS_H #define TOR_TRANSPORTS_H +/** Represents a pluggable transport used by a bridge. */ +typedef struct transport_t { + /** SOCKS version: One of PROXY_SOCKS4, PROXY_SOCKS5. */ + int socks_version; + /** Name of pluggable transport protocol */ + char *name; + /** The IP address where the transport bound and is waiting for + * connections. */ + tor_addr_t addr; + /** Port of proxy */ + uint16_t port; + /** Boolean: We are re-parsing our transport list, and we are going to remove + * this one if we don't find it in the list of configured transports. */ + unsigned marked_for_removal : 1; +} transport_t; + +void mark_transport_list(void); +void sweep_transport_list(void); +int transport_add_from_config(const tor_addr_t *addr, uint16_t port, + const char *name, int socks_ver); +void transport_free(transport_t *transport); + +transport_t *transport_get_by_name(const char *name); + void pt_kickstart_proxy(const smartlist_t *transport_list, char **proxy_argv, int is_server); @@ -23,11 +47,15 @@ void pt_configure_remaining_proxies(void); int pt_proxies_configuration_pending(void); +char *pt_get_extra_info_descriptor_string(void); + void pt_free_all(void); void pt_prepare_proxy_list_for_config_read(void); void sweep_proxy_list(void); +smartlist_t *get_transport_proxy_ports(void); + #ifdef PT_PRIVATE /** State of the managed proxy configuration protocol. */ enum pt_proto_state { @@ -68,28 +96,7 @@ typedef struct { smartlist_t *transports_to_launch; /* The 'transports' list contains all the transports this proxy has - launched. - - Before a managed_proxy_t reaches the PT_PROTO_COMPLETED phase, - this smartlist contains a 'transport_t' for every transport it - has launched. - - When the managed_proxy_t reaches the PT_PROTO_COMPLETED phase, it - registers all its transports to the circuitbuild.c subsystem. At - that point the 'transport_t's are owned by the circuitbuild.c - subsystem. - - To avoid carrying dangling 'transport_t's in this smartlist, - right before the managed_proxy_t reaches the PT_PROTO_COMPLETED - phase we replace all 'transport_t's with strings of their - transport names. - - So, tl;dr: - When (conf_state != PT_PROTO_COMPLETED) this list carries - (transport_t *). - When (conf_state == PT_PROTO_COMPLETED) this list carries - (char *). - */ + launched. */ smartlist_t *transports; } managed_proxy_t; diff --git a/src/test/Makefile.am b/src/test/Makefile.am deleted file mode 100644 index 31a464ee7..000000000 --- a/src/test/Makefile.am +++ /dev/null @@ -1,49 +0,0 @@ -TESTS = test - -noinst_PROGRAMS = test test-child bench - -AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ - -DLOCALSTATEDIR="\"$(localstatedir)\"" \ - -DBINDIR="\"$(bindir)\"" \ - -I"$(top_srcdir)/src/or" - -# -L flags need to go in LDFLAGS. -l flags need to go in LDADD. -# This seems to matter nowhere but on Windows, but I assure you that it -# matters a lot there, and is quite hard to debug if you forget to do it. - -test_SOURCES = \ - test.c \ - test_addr.c \ - test_containers.c \ - test_crypto.c \ - test_data.c \ - test_dir.c \ - test_microdesc.c \ - test_pt.c \ - test_util.c \ - test_config.c \ - tinytest.c - -bench_SOURCES = \ - bench.c - -test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ - @TOR_LDFLAGS_libevent@ -test_LDADD = ../or/libtor.a ../common/libor.a ../common/libor-crypto.a \ - ../common/libor-event.a \ - @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ - @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ - -bench_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ - @TOR_LDFLAGS_libevent@ -bench_LDADD = ../or/libtor.a ../common/libor.a ../common/libor-crypto.a \ - ../common/libor-event.a \ - @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ - @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ - -noinst_HEADERS = \ - tinytest.h \ - tinytest_macros.h \ - test.h - - diff --git a/src/test/include.am b/src/test/include.am new file mode 100644 index 000000000..03fef2337 --- /dev/null +++ b/src/test/include.am @@ -0,0 +1,55 @@ +TESTS+= src/test/test + +noinst_PROGRAMS+= src/test/test src/test/test-child src/test/bench + +src_test_AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \ + -DLOCALSTATEDIR="\"$(localstatedir)\"" \ + -DBINDIR="\"$(bindir)\"" \ + -I"$(top_srcdir)/src/or" + +# -L flags need to go in LDFLAGS. -l flags need to go in LDADD. +# This seems to matter nowhere but on Windows, but I assure you that it +# matters a lot there, and is quite hard to debug if you forget to do it. + +src_test_test_SOURCES = \ + src/test/test.c \ + src/test/test_addr.c \ + src/test/test_containers.c \ + src/test/test_crypto.c \ + src/test/test_data.c \ + src/test/test_dir.c \ + src/test/test_introduce.c \ + src/test/test_microdesc.c \ + src/test/test_pt.c \ + src/test/test_replay.c \ + src/test/test_util.c \ + src/test/test_config.c \ + src/test/tinytest.c + +src_test_test_CPPFLAGS= $(src_test_AM_CPPFLAGS) + +src_test_bench_SOURCES = \ + src/test/bench.c + +src_test_bench_CPPFLAGS= $(src_test_AM_CPPFLAGS) + +src_test_test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ + @TOR_LDFLAGS_libevent@ +src_test_test_LDADD = src/or/libtor.a src/common/libor.a src/common/libor-crypto.a \ + src/common/libor-event.a \ + @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ + @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +src_test_bench_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ + @TOR_LDFLAGS_libevent@ +src_test_bench_LDADD = src/or/libtor.a src/common/libor.a src/common/libor-crypto.a \ + src/common/libor-event.a \ + @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ + @TOR_OPENSSL_LIBS@ @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +noinst_HEADERS+= \ + src/test/tinytest.h \ + src/test/tinytest_macros.h \ + src/test/test.h + + diff --git a/src/test/test.c b/src/test/test.c index 6bf2d28d9..81172795f 100644 --- a/src/test/test.c +++ b/src/test/test.c @@ -1863,6 +1863,8 @@ extern struct testcase_t dir_tests[]; extern struct testcase_t microdesc_tests[]; extern struct testcase_t pt_tests[]; extern struct testcase_t config_tests[]; +extern struct testcase_t introduce_tests[]; +extern struct testcase_t replaycache_tests[]; static struct testgroup_t testgroups[] = { { "", test_array }, @@ -1875,6 +1877,8 @@ static struct testgroup_t testgroups[] = { { "dir/md/", microdesc_tests }, { "pt/", pt_tests }, { "config/", config_tests }, + { "replaycache/", replaycache_tests }, + { "introduce/", introduce_tests }, END_OF_GROUPS }; diff --git a/src/test/test.h b/src/test/test.h index 0b6e6c60c..6dcb9490b 100644 --- a/src/test/test.h +++ b/src/test/test.h @@ -65,6 +65,10 @@ #define test_memeq_hex(expr1, hex) test_mem_op_hex(expr1, ==, hex) +#define tt_double_op(a,op,b) \ + tt_assert_test_type(a,b,#a" "#op" "#b,double,(val1_ op val2_),"%f", \ + TT_EXIT_TEST_FUNCTION) + const char *get_fname(const char *name); crypto_pk_t *pk_generate(int idx); diff --git a/src/test/test_config.c b/src/test/test_config.c index ff251a24d..d9fcd8b35 100644 --- a/src/test/test_config.c +++ b/src/test/test_config.c @@ -6,6 +6,7 @@ #include "orconfig.h" #include "or.h" #include "config.h" +#include "confparse.h" #include "connection_edge.h" #include "test.h" diff --git a/src/test/test_dir.c b/src/test/test_dir.c index 83c612045..9bf44b116 100644 --- a/src/test/test_dir.c +++ b/src/test/test_dir.c @@ -4,9 +4,12 @@ /* See LICENSE for licensing information */ #include "orconfig.h" +#include <math.h> + #define DIRSERV_PRIVATE #define DIRVOTE_PRIVATE #define ROUTER_PRIVATE +#define ROUTERLIST_PRIVATE #define HIBERNATE_PRIVATE #include "or.h" #include "directory.h" @@ -147,9 +150,9 @@ test_dir_formats(void) "platform Tor "VERSION" on ", sizeof(buf2)); strlcat(buf2, get_uname(), sizeof(buf2)); strlcat(buf2, "\n" - "opt protocols Link 1 2 Circuit 1\n" + "protocols Link 1 2 Circuit 1\n" "published 1970-01-01 00:00:00\n" - "opt fingerprint ", sizeof(buf2)); + "fingerprint ", sizeof(buf2)); test_assert(!crypto_pk_get_fingerprint(pk2, fingerprint, 1)); strlcat(buf2, fingerprint, sizeof(buf2)); strlcat(buf2, "\nuptime 0\n" @@ -161,7 +164,7 @@ test_dir_formats(void) strlcat(buf2, pk1_str, sizeof(buf2)); strlcat(buf2, "signing-key\n", sizeof(buf2)); strlcat(buf2, pk2_str, sizeof(buf2)); - strlcat(buf2, "opt hidden-service-dir\n", sizeof(buf2)); + strlcat(buf2, "hidden-service-dir\n", sizeof(buf2)); strlcat(buf2, "reject *:*\nrouter-signature\n", sizeof(buf2)); buf[strlen(buf2)] = '\0'; /* Don't compare the sig; it's never the same * twice */ @@ -797,6 +800,7 @@ test_dir_v3_networkstatus(void) networkstatus_t *vote=NULL, *v1=NULL, *v2=NULL, *v3=NULL, *con=NULL, *con_md=NULL; vote_routerstatus_t *vrs; + tor_addr_t addr_ipv6; routerstatus_t *rs; char *v1_text=NULL, *v2_text=NULL, *v3_text=NULL, *consensus_text=NULL, *cp; smartlist_t *votes = smartlist_new(); @@ -893,6 +897,9 @@ test_dir_v3_networkstatus(void) rs->addr = 0x99009901; rs->or_port = 443; rs->dir_port = 0; + tor_addr_parse(&addr_ipv6, "[1:2:3::4]"); + tor_addr_copy(&rs->ipv6_addr, &addr_ipv6); + rs->ipv6_orport = 4711; rs->is_exit = rs->is_stable = rs->is_fast = rs->is_flagged_running = rs->is_valid = rs->is_v2_dir = rs->is_possible_guard = 1; smartlist_add(vote->routerstatus_list, vrs); @@ -987,6 +994,8 @@ test_dir_v3_networkstatus(void) test_eq(rs->addr, 0x99009901); test_eq(rs->or_port, 443); test_eq(rs->dir_port, 0); + test_assert(tor_addr_eq(&rs->ipv6_addr, &addr_ipv6)); + test_eq(rs->ipv6_orport, 4711); test_eq(vrs->flags, U64_LITERAL(254)); // all flags except "authority." { @@ -1169,6 +1178,8 @@ test_dir_v3_networkstatus(void) test_eq(rs->addr, 0x99009901); test_eq(rs->or_port, 443); test_eq(rs->dir_port, 0); + test_assert(tor_addr_eq(&rs->ipv6_addr, &addr_ipv6)); + test_eq(rs->ipv6_orport, 4711); test_assert(!rs->is_authority); test_assert(rs->is_exit); test_assert(rs->is_fast); @@ -1381,6 +1392,124 @@ test_dir_v3_networkstatus(void) ns_detached_signatures_free(dsig2); } +static void +test_dir_scale_bw(void *testdata) +{ + double v[8] = { 2.0/3, + 7.0, + 1.0, + 3.0, + 1.0/5, + 1.0/7, + 12.0, + 24.0 }; + u64_dbl_t vals[8]; + uint64_t total; + int i; + + (void) testdata; + + for (i=0; i<8; ++i) + vals[i].dbl = v[i]; + + scale_array_elements_to_u64(vals, 8, &total); + + tt_int_op((int)total, ==, 48); + total = 0; + for (i=0; i<8; ++i) { + total += vals[i].u64; + } + tt_assert(total >= (U64_LITERAL(1)<<60)); + tt_assert(total <= (U64_LITERAL(1)<<62)); + + for (i=0; i<8; ++i) { + double ratio = ((double)vals[i].u64) / vals[2].u64; + tt_double_op(fabs(ratio - v[i]), <, .00001); + } + + done: + ; +} + +static void +test_dir_random_weighted(void *testdata) +{ + int histogram[10]; + uint64_t vals[10] = {3,1,2,4,6,0,7,5,8,9}, total=0; + u64_dbl_t inp[10]; + int i, choice; + const int n = 50000; + double max_sq_error; + (void) testdata; + + /* Try a ten-element array with values from 0 through 10. The values are + * in a scrambled order to make sure we don't depend on order. */ + memset(histogram,0,sizeof(histogram)); + for (i=0; i<10; ++i) { + inp[i].u64 = vals[i]; + total += vals[i]; + } + tt_int_op(total, ==, 45); + for (i=0; i<n; ++i) { + choice = choose_array_element_by_weight(inp, 10); + tt_int_op(choice, >=, 0); + tt_int_op(choice, <, 10); + histogram[choice]++; + } + + /* Now see if we chose things about frequently enough. */ + max_sq_error = 0; + for (i=0; i<10; ++i) { + int expected = (int)(n*vals[i]/total); + double frac_diff = 0, sq; + TT_BLATHER((" %d : %5d vs %5d\n", (int)vals[i], histogram[i], expected)); + if (expected) + frac_diff = (histogram[i] - expected) / ((double)expected); + else + tt_int_op(histogram[i], ==, 0); + + sq = frac_diff * frac_diff; + if (sq > max_sq_error) + max_sq_error = sq; + } + /* It should almost always be much much less than this. If you want to + * figure out the odds, please feel free. */ + tt_double_op(max_sq_error, <, .05); + + /* Now try a singleton; do we choose it? */ + for (i = 0; i < 100; ++i) { + choice = choose_array_element_by_weight(inp, 1); + tt_int_op(choice, ==, 0); + } + + /* Now try an array of zeros. We should choose randomly. */ + memset(histogram,0,sizeof(histogram)); + for (i = 0; i < 5; ++i) + inp[i].u64 = 0; + for (i = 0; i < n; ++i) { + choice = choose_array_element_by_weight(inp, 5); + tt_int_op(choice, >=, 0); + tt_int_op(choice, <, 5); + histogram[choice]++; + } + /* Now see if we chose things about frequently enough. */ + max_sq_error = 0; + for (i=0; i<5; ++i) { + int expected = n/5; + double frac_diff = 0, sq; + TT_BLATHER((" %d : %5d vs %5d\n", (int)vals[i], histogram[i], expected)); + frac_diff = (histogram[i] - expected) / ((double)expected); + sq = frac_diff * frac_diff; + if (sq > max_sq_error) + max_sq_error = sq; + } + /* It should almost always be much much less than this. If you want to + * figure out the odds, please feel free. */ + tt_double_op(max_sq_error, <, .05); + done: + ; +} + #define DIR_LEGACY(name) \ { #name, legacy_test_helper, TT_FORK, &legacy_setup, test_dir_ ## name } @@ -1396,6 +1525,8 @@ struct testcase_t dir_tests[] = { DIR_LEGACY(measured_bw), DIR_LEGACY(param_voting), DIR_LEGACY(v3_networkstatus), + DIR(random_weighted), + DIR(scale_bw), END_OF_TESTCASES }; diff --git a/src/test/test_introduce.c b/src/test/test_introduce.c new file mode 100644 index 000000000..992d9cd50 --- /dev/null +++ b/src/test/test_introduce.c @@ -0,0 +1,528 @@ +/* Copyright (c) 2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#include "orconfig.h" +#include "crypto.h" +#include "or.h" +#include "test.h" + +#define RENDSERVICE_PRIVATE +#include "rendservice.h" + +extern const char AUTHORITY_SIGNKEY_1[]; + +static uint8_t v0_test_plaintext[] = + /* 20 bytes of rendezvous point nickname */ + { 0x4e, 0x69, 0x63, 0x6b, 0x6e, 0x61, 0x6d, 0x65, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + /* 20 bytes dummy rendezvous cookie */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 128 bytes dummy DH handshake data */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 }; + +static uint8_t v1_test_plaintext[] = + /* Version byte */ + { 0x01, + /* 42 bytes of dummy rendezvous point hex digest */ + 0x24, 0x30, 0x30, 0x30, 0x31, 0x30, 0x32, 0x30, + 0x33, 0x30, 0x34, 0x30, 0x35, 0x30, 0x36, 0x30, + 0x37, 0x30, 0x38, 0x30, 0x39, 0x30, 0x41, 0x30, + 0x42, 0x30, 0x43, 0x30, 0x44, 0x30, 0x45, 0x30, + 0x46, 0x31, 0x30, 0x31, 0x31, 0x31, 0x32, 0x31, + 0x33, 0x00, + /* 20 bytes dummy rendezvous cookie */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 128 bytes dummy DH handshake data */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 }; + +static uint8_t v2_test_plaintext[] = + /* Version byte */ + { 0x02, + /* 4 bytes rendezvous point's IP address */ + 0xc0, 0xa8, 0x00, 0x01, + /* 2 bytes rendezvous point's OR port */ + 0x23, 0x5a, + /* 20 bytes dummy rendezvous point's identity digest */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 2 bytes length of onion key */ + 0x00, 0x8c, + /* Onion key (140 bytes taken from live test) */ + 0x30, 0x81, 0x89, 0x02, 0x81, 0x81, 0x00, 0xb1, + 0xcd, 0x46, 0xa9, 0x18, 0xd2, 0x0f, 0x01, 0xf8, + 0xb2, 0xad, 0xa4, 0x79, 0xb4, 0xbb, 0x4b, 0xf4, + 0x54, 0x1e, 0x3f, 0x03, 0x54, 0xcf, 0x7c, 0xb6, + 0xb5, 0xf0, 0xfe, 0xed, 0x4b, 0x7d, 0xd7, 0x61, + 0xdb, 0x6d, 0xd9, 0x19, 0xe2, 0x72, 0x04, 0xaa, + 0x3e, 0x89, 0x26, 0x14, 0x62, 0x9a, 0x6c, 0x11, + 0x0b, 0x35, 0x99, 0x2c, 0x9f, 0x2c, 0x64, 0xa1, + 0xd9, 0xe2, 0x88, 0xce, 0xf6, 0x54, 0xfe, 0x1d, + 0x37, 0x5e, 0x6d, 0x73, 0x95, 0x54, 0x90, 0xf0, + 0x7b, 0xfa, 0xd4, 0x44, 0xac, 0xb2, 0x23, 0x9f, + 0x75, 0x36, 0xe2, 0x78, 0x62, 0x82, 0x80, 0xa4, + 0x23, 0x22, 0xc9, 0xbf, 0xc4, 0x36, 0xd1, 0x31, + 0x33, 0x8e, 0x64, 0xb4, 0xa9, 0x74, 0xa1, 0xcb, + 0x42, 0x8d, 0x60, 0xc7, 0xbb, 0x8e, 0x6e, 0x0f, + 0x36, 0x74, 0x8e, 0xf4, 0x08, 0x99, 0x06, 0x92, + 0xb1, 0x3f, 0xb3, 0xdd, 0xed, 0xf7, 0xc9, 0x02, + 0x03, 0x01, 0x00, 0x01, + /* 20 bytes dummy rendezvous cookie */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 128 bytes dummy DH handshake data */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 }; + +static uint8_t v3_no_auth_test_plaintext[] = + /* Version byte */ + { 0x03, + /* Auth type (0 for no auth len/auth data) */ + 0x00, + /* Timestamp */ + 0x50, 0x0b, 0xb5, 0xaa, + /* 4 bytes rendezvous point's IP address */ + 0xc0, 0xa8, 0x00, 0x01, + /* 2 bytes rendezvous point's OR port */ + 0x23, 0x5a, + /* 20 bytes dummy rendezvous point's identity digest */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 2 bytes length of onion key */ + 0x00, 0x8c, + /* Onion key (140 bytes taken from live test) */ + 0x30, 0x81, 0x89, 0x02, 0x81, 0x81, 0x00, 0xb1, + 0xcd, 0x46, 0xa9, 0x18, 0xd2, 0x0f, 0x01, 0xf8, + 0xb2, 0xad, 0xa4, 0x79, 0xb4, 0xbb, 0x4b, 0xf4, + 0x54, 0x1e, 0x3f, 0x03, 0x54, 0xcf, 0x7c, 0xb6, + 0xb5, 0xf0, 0xfe, 0xed, 0x4b, 0x7d, 0xd7, 0x61, + 0xdb, 0x6d, 0xd9, 0x19, 0xe2, 0x72, 0x04, 0xaa, + 0x3e, 0x89, 0x26, 0x14, 0x62, 0x9a, 0x6c, 0x11, + 0x0b, 0x35, 0x99, 0x2c, 0x9f, 0x2c, 0x64, 0xa1, + 0xd9, 0xe2, 0x88, 0xce, 0xf6, 0x54, 0xfe, 0x1d, + 0x37, 0x5e, 0x6d, 0x73, 0x95, 0x54, 0x90, 0xf0, + 0x7b, 0xfa, 0xd4, 0x44, 0xac, 0xb2, 0x23, 0x9f, + 0x75, 0x36, 0xe2, 0x78, 0x62, 0x82, 0x80, 0xa4, + 0x23, 0x22, 0xc9, 0xbf, 0xc4, 0x36, 0xd1, 0x31, + 0x33, 0x8e, 0x64, 0xb4, 0xa9, 0x74, 0xa1, 0xcb, + 0x42, 0x8d, 0x60, 0xc7, 0xbb, 0x8e, 0x6e, 0x0f, + 0x36, 0x74, 0x8e, 0xf4, 0x08, 0x99, 0x06, 0x92, + 0xb1, 0x3f, 0xb3, 0xdd, 0xed, 0xf7, 0xc9, 0x02, + 0x03, 0x01, 0x00, 0x01, + /* 20 bytes dummy rendezvous cookie */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 128 bytes dummy DH handshake data */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 }; + +static uint8_t v3_basic_auth_test_plaintext[] = + /* Version byte */ + { 0x03, + /* Auth type (1 for REND_BASIC_AUTH) */ + 0x01, + /* Auth len (must be 16 bytes for REND_BASIC_AUTH) */ + 0x00, 0x10, + /* Auth data (a 16-byte dummy descriptor cookie) */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + /* Timestamp */ + 0x50, 0x0b, 0xb5, 0xaa, + /* 4 bytes rendezvous point's IP address */ + 0xc0, 0xa8, 0x00, 0x01, + /* 2 bytes rendezvous point's OR port */ + 0x23, 0x5a, + /* 20 bytes dummy rendezvous point's identity digest */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 2 bytes length of onion key */ + 0x00, 0x8c, + /* Onion key (140 bytes taken from live test) */ + 0x30, 0x81, 0x89, 0x02, 0x81, 0x81, 0x00, 0xb1, + 0xcd, 0x46, 0xa9, 0x18, 0xd2, 0x0f, 0x01, 0xf8, + 0xb2, 0xad, 0xa4, 0x79, 0xb4, 0xbb, 0x4b, 0xf4, + 0x54, 0x1e, 0x3f, 0x03, 0x54, 0xcf, 0x7c, 0xb6, + 0xb5, 0xf0, 0xfe, 0xed, 0x4b, 0x7d, 0xd7, 0x61, + 0xdb, 0x6d, 0xd9, 0x19, 0xe2, 0x72, 0x04, 0xaa, + 0x3e, 0x89, 0x26, 0x14, 0x62, 0x9a, 0x6c, 0x11, + 0x0b, 0x35, 0x99, 0x2c, 0x9f, 0x2c, 0x64, 0xa1, + 0xd9, 0xe2, 0x88, 0xce, 0xf6, 0x54, 0xfe, 0x1d, + 0x37, 0x5e, 0x6d, 0x73, 0x95, 0x54, 0x90, 0xf0, + 0x7b, 0xfa, 0xd4, 0x44, 0xac, 0xb2, 0x23, 0x9f, + 0x75, 0x36, 0xe2, 0x78, 0x62, 0x82, 0x80, 0xa4, + 0x23, 0x22, 0xc9, 0xbf, 0xc4, 0x36, 0xd1, 0x31, + 0x33, 0x8e, 0x64, 0xb4, 0xa9, 0x74, 0xa1, 0xcb, + 0x42, 0x8d, 0x60, 0xc7, 0xbb, 0x8e, 0x6e, 0x0f, + 0x36, 0x74, 0x8e, 0xf4, 0x08, 0x99, 0x06, 0x92, + 0xb1, 0x3f, 0xb3, 0xdd, 0xed, 0xf7, 0xc9, 0x02, + 0x03, 0x01, 0x00, 0x01, + /* 20 bytes dummy rendezvous cookie */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, + /* 128 bytes dummy DH handshake data */ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00, + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, + 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 }; + +static void do_decrypt_test(uint8_t *plaintext, size_t plaintext_len); +static void do_early_parse_test(uint8_t *plaintext, size_t plaintext_len); +static void do_late_parse_test(uint8_t *plaintext, size_t plaintext_len); +static void do_parse_test(uint8_t *plaintext, size_t plaintext_len, int phase); +static ssize_t make_intro_from_plaintext( + void *buf, size_t len, crypto_pk_t *key, void **cell_out); + +#define EARLY_PARSE_ONLY 1 +#define DECRYPT_ONLY 2 +#define ALL_PARSING 3 + +static void +do_early_parse_test(uint8_t *plaintext, size_t plaintext_len) +{ + do_parse_test(plaintext, plaintext_len, EARLY_PARSE_ONLY); +} + +static void +do_decrypt_test(uint8_t *plaintext, size_t plaintext_len) +{ + do_parse_test(plaintext, plaintext_len, DECRYPT_ONLY); +} + +static void +do_late_parse_test(uint8_t *plaintext, size_t plaintext_len) +{ + do_parse_test(plaintext, plaintext_len, ALL_PARSING); +} + +/** Test utility function: checks that the <b>plaintext_len</b>-byte string at + * <b>plaintext</b> is at least superficially parseable. + */ +static void +do_parse_test(uint8_t *plaintext, size_t plaintext_len, int phase) +{ + crypto_pk_t *k = NULL; + ssize_t r; + uint8_t *cell = NULL; + size_t cell_len; + rend_intro_cell_t *parsed_req = NULL; + char *err_msg = NULL; + char digest[DIGEST_LEN]; + + /* Get a key */ + k = crypto_pk_new(); + test_assert(k); + r = crypto_pk_read_private_key_from_string(k, AUTHORITY_SIGNKEY_1, -1); + test_assert(!r); + + /* Get digest for future comparison */ + r = crypto_pk_get_digest(k, digest); + test_assert(r >= 0); + + /* Make a cell out of it */ + r = make_intro_from_plaintext( + plaintext, plaintext_len, + k, (void **)(&cell)); + test_assert(r > 0); + test_assert(cell); + cell_len = r; + + /* Do early parsing */ + parsed_req = rend_service_begin_parse_intro(cell, cell_len, 2, &err_msg); + test_assert(parsed_req); + test_assert(!err_msg); + test_memeq(parsed_req->pk, digest, DIGEST_LEN); + test_assert(parsed_req->ciphertext); + test_assert(parsed_req->ciphertext_len > 0); + + if (phase == EARLY_PARSE_ONLY) + goto done; + + /* Do decryption */ + r = rend_service_decrypt_intro(parsed_req, k, &err_msg); + test_assert(!r); + test_assert(!err_msg); + test_assert(parsed_req->plaintext); + test_assert(parsed_req->plaintext_len > 0); + + if (phase == DECRYPT_ONLY) + goto done; + + /* Do late parsing */ + r = rend_service_parse_intro_plaintext(parsed_req, &err_msg); + test_assert(!r); + test_assert(!err_msg); + test_assert(parsed_req->parsed); + + done: + tor_free(cell); + crypto_pk_free(k); + rend_service_free_intro(parsed_req); + tor_free(err_msg); +} + +/** Given the plaintext of the encrypted part of an INTRODUCE1/2 and a key, + * construct the encrypted cell for testing. + */ + +static ssize_t +make_intro_from_plaintext( + void *buf, size_t len, crypto_pk_t *key, void **cell_out) +{ + char *cell = NULL; + ssize_t cell_len = -1, r; + /* Assemble key digest and ciphertext, then construct the cell */ + ssize_t ciphertext_size; + + if (!(buf && key && len > 0 && cell_out)) goto done; + + /* + * Figure out an upper bound on how big the ciphertext will be + * (see crypto_pk_public_hybrid_encrypt()) + */ + ciphertext_size = PKCS1_OAEP_PADDING_OVERHEAD; + ciphertext_size += crypto_pk_keysize(key); + ciphertext_size += CIPHER_KEY_LEN; + ciphertext_size += len; + + /* + * Allocate space for the cell + */ + cell = tor_malloc(DIGEST_LEN + ciphertext_size); + + /* Compute key digest (will be first DIGEST_LEN octets of cell) */ + r = crypto_pk_get_digest(key, cell); + test_assert(r >= 0); + + /* Do encryption */ + r = crypto_pk_public_hybrid_encrypt( + key, cell + DIGEST_LEN, ciphertext_size, + buf, len, + PK_PKCS1_OAEP_PADDING, 0); + test_assert(r >= 0); + + /* Figure out cell length */ + cell_len = DIGEST_LEN + r; + + /* Output the cell */ + *cell_out = cell; + + done: + return cell_len; +} + +/** Test v0 INTRODUCE2 parsing through decryption only + */ + +static void +test_introduce_decrypt_v0(void) +{ + do_decrypt_test(v0_test_plaintext, sizeof(v0_test_plaintext)); +} + +/** Test v1 INTRODUCE2 parsing through decryption only + */ + +static void +test_introduce_decrypt_v1(void) +{ + do_decrypt_test(v1_test_plaintext, sizeof(v1_test_plaintext)); +} + +/** Test v2 INTRODUCE2 parsing through decryption only + */ + +static void +test_introduce_decrypt_v2(void) +{ + do_decrypt_test(v2_test_plaintext, sizeof(v2_test_plaintext)); +} + +/** Test v3 INTRODUCE2 parsing through decryption only + */ + +static void +test_introduce_decrypt_v3(void) +{ + do_decrypt_test( + v3_no_auth_test_plaintext, sizeof(v3_no_auth_test_plaintext)); + do_decrypt_test( + v3_basic_auth_test_plaintext, sizeof(v3_basic_auth_test_plaintext)); +} + +/** Test v0 INTRODUCE2 parsing through early parsing only + */ + +static void +test_introduce_early_parse_v0(void) +{ + do_early_parse_test(v0_test_plaintext, sizeof(v0_test_plaintext)); +} + +/** Test v1 INTRODUCE2 parsing through early parsing only + */ + +static void +test_introduce_early_parse_v1(void) +{ + do_early_parse_test(v1_test_plaintext, sizeof(v1_test_plaintext)); +} + +/** Test v2 INTRODUCE2 parsing through early parsing only + */ + +static void +test_introduce_early_parse_v2(void) +{ + do_early_parse_test(v2_test_plaintext, sizeof(v2_test_plaintext)); +} + +/** Test v3 INTRODUCE2 parsing through early parsing only + */ + +static void +test_introduce_early_parse_v3(void) +{ + do_early_parse_test( + v3_no_auth_test_plaintext, sizeof(v3_no_auth_test_plaintext)); + do_early_parse_test( + v3_basic_auth_test_plaintext, sizeof(v3_basic_auth_test_plaintext)); +} + +/** Test v0 INTRODUCE2 parsing + */ + +static void +test_introduce_late_parse_v0(void) +{ + do_late_parse_test(v0_test_plaintext, sizeof(v0_test_plaintext)); +} + +/** Test v1 INTRODUCE2 parsing + */ + +static void +test_introduce_late_parse_v1(void) +{ + do_late_parse_test(v1_test_plaintext, sizeof(v1_test_plaintext)); +} + +/** Test v2 INTRODUCE2 parsing + */ + +static void +test_introduce_late_parse_v2(void) +{ + do_late_parse_test(v2_test_plaintext, sizeof(v2_test_plaintext)); +} + +/** Test v3 INTRODUCE2 parsing + */ + +static void +test_introduce_late_parse_v3(void) +{ + do_late_parse_test( + v3_no_auth_test_plaintext, sizeof(v3_no_auth_test_plaintext)); + do_late_parse_test( + v3_basic_auth_test_plaintext, sizeof(v3_basic_auth_test_plaintext)); +} + +#define INTRODUCE_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_introduce_ ## name } + +struct testcase_t introduce_tests[] = { + INTRODUCE_LEGACY(early_parse_v0), + INTRODUCE_LEGACY(early_parse_v1), + INTRODUCE_LEGACY(early_parse_v2), + INTRODUCE_LEGACY(early_parse_v3), + INTRODUCE_LEGACY(decrypt_v0), + INTRODUCE_LEGACY(decrypt_v1), + INTRODUCE_LEGACY(decrypt_v2), + INTRODUCE_LEGACY(decrypt_v3), + INTRODUCE_LEGACY(late_parse_v0), + INTRODUCE_LEGACY(late_parse_v1), + INTRODUCE_LEGACY(late_parse_v2), + INTRODUCE_LEGACY(late_parse_v3), + END_OF_TESTCASES +}; + diff --git a/src/test/test_replay.c b/src/test/test_replay.c new file mode 100644 index 000000000..b08818f06 --- /dev/null +++ b/src/test/test_replay.c @@ -0,0 +1,184 @@ +/* Copyright (c) 2012, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define REPLAYCACHE_PRIVATE + +#include "orconfig.h" +#include "or.h" +#include "replaycache.h" +#include "test.h" + +static const char *test_buffer = + "Lorem ipsum dolor sit amet, consectetur adipisici elit, sed do eiusmod" + " tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim" + " veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea" + " commodo consequat. Duis aute irure dolor in reprehenderit in voluptate" + " velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint" + " occaecat cupidatat non proident, sunt in culpa qui officia deserunt" + " mollit anim id est laborum."; + +static void +test_replaycache_alloc(void) +{ + replaycache_t *r = NULL; + + r = replaycache_new(600, 300); + test_assert(r != NULL); + if (!r) goto done; + + done: + if (r) replaycache_free(r); + + return; +} + +static void +test_replaycache_miss(void) +{ + replaycache_t *r = NULL; + int result; + + r = replaycache_new(600, 300); + test_assert(r != NULL); + if (!r) goto done; + + result = + replaycache_add_and_test_internal(1200, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 0); + + done: + if (r) replaycache_free(r); + + return; +} + +static void +test_replaycache_hit(void) +{ + replaycache_t *r = NULL; + int result; + + r = replaycache_new(600, 300); + test_assert(r != NULL); + if (!r) goto done; + + result = + replaycache_add_and_test_internal(1200, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 0); + + result = + replaycache_add_and_test_internal(1300, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 1); + + done: + if (r) replaycache_free(r); + + return; +} + +static void +test_replaycache_age(void) +{ + replaycache_t *r = NULL; + int result; + + r = replaycache_new(600, 300); + test_assert(r != NULL); + if (!r) goto done; + + result = + replaycache_add_and_test_internal(1200, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 0); + + result = + replaycache_add_and_test_internal(1300, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 1); + + result = + replaycache_add_and_test_internal(3000, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 0); + + done: + if (r) replaycache_free(r); + + return; +} + +static void +test_replaycache_elapsed(void) +{ + replaycache_t *r = NULL; + int result; + time_t elapsed; + + r = replaycache_new(600, 300); + test_assert(r != NULL); + if (!r) goto done; + + result = + replaycache_add_and_test_internal(1200, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 0); + + result = + replaycache_add_and_test_internal(1300, r, test_buffer, + (int)strlen(test_buffer), &elapsed); + test_eq(result, 1); + test_eq(elapsed, 100); + + done: + if (r) replaycache_free(r); + + return; +} + +static void +test_replaycache_noexpire(void) +{ + replaycache_t *r = NULL; + int result; + + r = replaycache_new(0, 0); + test_assert(r != NULL); + if (!r) goto done; + + result = + replaycache_add_and_test_internal(1200, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 0); + + result = + replaycache_add_and_test_internal(1300, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 1); + + result = + replaycache_add_and_test_internal(3000, r, test_buffer, + (int)strlen(test_buffer), NULL); + test_eq(result, 1); + + done: + if (r) replaycache_free(r); + + return; +} + +#define REPLAYCACHE_LEGACY(name) \ + { #name, legacy_test_helper, 0, &legacy_setup, test_replaycache_ ## name } + +struct testcase_t replaycache_tests[] = { + REPLAYCACHE_LEGACY(alloc), + REPLAYCACHE_LEGACY(miss), + REPLAYCACHE_LEGACY(hit), + REPLAYCACHE_LEGACY(age), + REPLAYCACHE_LEGACY(elapsed), + REPLAYCACHE_LEGACY(noexpire), + END_OF_TESTCASES +}; + diff --git a/src/test/test_util.c b/src/test/test_util.c index 4f9eb73e0..2c65903dd 100644 --- a/src/test/test_util.c +++ b/src/test/test_util.c @@ -1109,6 +1109,7 @@ test_util_pow2(void) test_eq(tor_log2(64), 6); test_eq(tor_log2(65), 6); test_eq(tor_log2(63), 5); + test_eq(tor_log2(0), 0); /* incorrect mathematically, but as specified */ test_eq(tor_log2(1), 0); test_eq(tor_log2(2), 1); test_eq(tor_log2(3), 1); @@ -1123,7 +1124,16 @@ test_util_pow2(void) test_eq(round_to_power_of_2(130), 128); test_eq(round_to_power_of_2(U64_LITERAL(40000000000000000)), U64_LITERAL(1)<<55); - test_eq(round_to_power_of_2(0), 2); + test_eq(round_to_power_of_2(U64_LITERAL(0xffffffffffffffff)), + U64_LITERAL(1)<<63); + test_eq(round_to_power_of_2(0), 1); + test_eq(round_to_power_of_2(1), 1); + test_eq(round_to_power_of_2(2), 2); + test_eq(round_to_power_of_2(3), 2); + test_eq(round_to_power_of_2(4), 4); + test_eq(round_to_power_of_2(5), 4); + test_eq(round_to_power_of_2(6), 4); + test_eq(round_to_power_of_2(7), 8); done: ; @@ -1474,12 +1484,28 @@ test_util_control_formats(void) tor_free(out); } +#define test_feq(value1,value2) do { \ + double v1 = (value1), v2=(value2); \ + double tf_diff = v1-v2; \ + double tf_tolerance = ((v1+v2)/2.0)/1e8; \ + if (tf_diff<0) tf_diff=-tf_diff; \ + if (tf_tolerance<0) tf_tolerance=-tf_tolerance; \ + if (tf_diff<tf_tolerance) { \ + TT_BLATHER(("%s ~~ %s: %f ~~ %f",#value1,#value2,v1,v2)); \ + } else { \ + TT_FAIL(("%s ~~ %s: %f != %f",#value1,#value2,v1,v2)); \ + } \ + } while (0) + static void test_util_sscanf(void) { unsigned u1, u2, u3; char s1[20], s2[10], s3[10], ch; int r; + long lng1,lng2; + int int1, int2; + double d1,d2,d3,d4; /* Simple tests (malformed patterns, literal matching, ...) */ test_eq(-1, tor_sscanf("123", "%i", &r)); /* %i is not supported */ @@ -1608,6 +1634,65 @@ test_util_sscanf(void) test_eq(4, tor_sscanf("1.2.3 foobar", "%u.%u.%u%c", &u1, &u2, &u3, &ch)); test_eq(' ', ch); + r = tor_sscanf("12345 -67890 -1", "%d %ld %d", &int1, &lng1, &int2); + test_eq(r,3); + test_eq(int1, 12345); + test_eq(lng1, -67890); + test_eq(int2, -1); + +#if SIZEOF_INT == 4 + r = tor_sscanf("-2147483648. 2147483647.", "%d. %d.", &int1, &int2); + test_eq(r,2); + test_eq(int1, -2147483647-1); + test_eq(int2, 2147483647); + + r = tor_sscanf("-2147483679.", "%d.", &int1); + test_eq(r,0); + + r = tor_sscanf("2147483678.", "%d.", &int1); + test_eq(r,0); +#elif SIZEOF_INT == 8 + r = tor_sscanf("-9223372036854775808. 9223372036854775807.", + "%d. %d.", &int1, &int2); + test_eq(r,2); + test_eq(int1, -9223372036854775807-1); + test_eq(int2, 9223372036854775807); + + r = tor_sscanf("-9223372036854775809.", "%d.", &int1); + test_eq(r,0); + + r = tor_sscanf("9223372036854775808.", "%d.", &int1); + test_eq(r,0); +#endif + +#if SIZEOF_LONG == 4 + r = tor_sscanf("-2147483648. 2147483647.", "%ld. %ld.", &lng1, &lng2); + test_eq(r,2); + test_eq(lng1, -2147483647 - 1); + test_eq(lng2, 2147483647); +#elif SIZEOF_LONG == 8 + r = tor_sscanf("-9223372036854775808. 9223372036854775807.", + "%ld. %ld.", &lng1, &lng2); + test_eq(r,2); + test_eq(lng1, -9223372036854775807L - 1); + test_eq(lng2, 9223372036854775807L); + + r = tor_sscanf("-9223372036854775808. 9223372036854775808.", + "%ld. %ld.", &lng1, &lng2); + test_eq(r,1); + r = tor_sscanf("-9223372036854775809. 9223372036854775808.", + "%ld. %ld.", &lng1, &lng2); + test_eq(r,0); +#endif + + r = tor_sscanf("123.456 .000007 -900123123.2000787 00003.2", + "%lf %lf %lf %lf", &d1,&d2,&d3,&d4); + test_eq(r,4); + test_feq(d1, 123.456); + test_feq(d2, .000007); + test_feq(d3, -900123123.2000787); + test_feq(d4, 3.2); + done: ; } diff --git a/src/tools/Makefile.am b/src/tools/Makefile.am deleted file mode 100644 index 35b0a41f5..000000000 --- a/src/tools/Makefile.am +++ /dev/null @@ -1,22 +0,0 @@ -bin_PROGRAMS = tor-resolve tor-gencert -noinst_PROGRAMS = tor-checkkey - -tor_resolve_SOURCES = tor-resolve.c -tor_resolve_LDFLAGS = -tor_resolve_LDADD = ../common/libor.a @TOR_LIB_MATH@ @TOR_LIB_WS32@ - -tor_gencert_SOURCES = tor-gencert.c -tor_gencert_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ -tor_gencert_LDADD = ../common/libor.a ../common/libor-crypto.a \ - @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ @TOR_OPENSSL_LIBS@ \ - @TOR_LIB_WS32@ @TOR_LIB_GDI@ - -tor_checkkey_SOURCES = tor-checkkey.c -tor_checkkey_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ -tor_checkkey_LDADD = ../common/libor.a ../common/libor-crypto.a \ - @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ @TOR_OPENSSL_LIBS@ \ - @TOR_LIB_WS32@ @TOR_LIB_GDI@ - -SUBDIRS = tor-fw-helper -DIST_SUBDIRS = tor-fw-helper - diff --git a/src/tools/include.am b/src/tools/include.am new file mode 100644 index 000000000..7337eff16 --- /dev/null +++ b/src/tools/include.am @@ -0,0 +1,22 @@ +bin_PROGRAMS+= src/tools/tor-resolve src/tools/tor-gencert +noinst_PROGRAMS+= src/tools/tor-checkkey + +src_tools_tor_resolve_SOURCES = src/tools/tor-resolve.c +src_tools_tor_resolve_LDFLAGS = +src_tools_tor_resolve_LDADD = src/common/libor.a @TOR_LIB_MATH@ @TOR_LIB_WS32@ + +src_tools_tor_gencert_SOURCES = src/tools/tor-gencert.c +src_tools_tor_gencert_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ +src_tools_tor_gencert_LDADD = src/common/libor.a src/common/libor-crypto.a \ + @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ @TOR_OPENSSL_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +src_tools_tor_checkkey_SOURCES = src/tools/tor-checkkey.c +src_tools_tor_checkkey_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ +src_tools_tor_checkkey_LDADD = src/common/libor.a src/common/libor-crypto.a \ + @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ @TOR_OPENSSL_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_GDI@ + +include src/tools/tor-fw-helper/include.am + + diff --git a/src/tools/tor-fw-helper/Makefile.am b/src/tools/tor-fw-helper/Makefile.am deleted file mode 100644 index 393562db0..000000000 --- a/src/tools/tor-fw-helper/Makefile.am +++ /dev/null @@ -1,38 +0,0 @@ -if USE_FW_HELPER -bin_PROGRAMS = tor-fw-helper -else -bin_PROGRAMS = -endif - -tor_fw_helper_SOURCES = \ - tor-fw-helper.c \ - tor-fw-helper-natpmp.c \ - tor-fw-helper-upnp.c -noinst_HEADERS = \ - tor-fw-helper.h \ - tor-fw-helper-natpmp.h \ - tor-fw-helper-upnp.h - -if NAT_PMP -nat_pmp_ldflags = @TOR_LDFLAGS_libnatpmp@ -nat_pmp_ldadd = -lnatpmp @TOR_LIB_IPHLPAPI@ -nat_pmp_cppflags = @TOR_CPPFLAGS_libnatpmp@ -else -nat_pmp_ldflags = -nat_pmp_ldadd = -nat_pmp_cppflags = -endif - -if MINIUPNPC -miniupnpc_ldflags = @TOR_LDFLAGS_libminiupnpc@ -miniupnpc_ldadd = -lminiupnpc -lm @TOR_LIB_IPHLPAPI@ -miniupnpc_cppflags = @TOR_CPPFLAGS_libminiupnpc@ -else -miniupnpc_ldflags = -miniupnpc_ldadd = -miniupnpc_cppflags = -endif - -tor_fw_helper_LDFLAGS = $(nat_pmp_ldflags) $(miniupnpc_ldflags) -tor_fw_helper_LDADD = ../../common/libor.a $(nat_pmp_ldadd) $(miniupnpc_ldadd) @TOR_LIB_WS32@ -tor_fw_helper_CPPFLAGS = $(nat_pmp_cppflags) $(miniupnpc_cppflags) diff --git a/src/tools/tor-fw-helper/include.am b/src/tools/tor-fw-helper/include.am new file mode 100644 index 000000000..cb6c9cd56 --- /dev/null +++ b/src/tools/tor-fw-helper/include.am @@ -0,0 +1,36 @@ +if USE_FW_HELPER +bin_PROGRAMS+= src/tools/tor-fw-helper/tor-fw-helper +endif + +src_tools_tor_fw_helper_tor_fw_helper_SOURCES = \ + src/tools/tor-fw-helper/tor-fw-helper.c \ + src/tools/tor-fw-helper/tor-fw-helper-natpmp.c \ + src/tools/tor-fw-helper/tor-fw-helper-upnp.c +noinst_HEADERS+= \ + src/tools/tor-fw-helper/tor-fw-helper.h \ + src/tools/tor-fw-helper/tor-fw-helper-natpmp.h \ + src/tools/tor-fw-helper/tor-fw-helper-upnp.h + +if NAT_PMP +nat_pmp_ldflags = @TOR_LDFLAGS_libnatpmp@ +nat_pmp_ldadd = -lnatpmp @TOR_LIB_IPHLPAPI@ +nat_pmp_cppflags = @TOR_CPPFLAGS_libnatpmp@ +else +nat_pmp_ldflags = +nat_pmp_ldadd = +nat_pmp_cppflags = +endif + +if MINIUPNPC +miniupnpc_ldflags = @TOR_LDFLAGS_libminiupnpc@ +miniupnpc_ldadd = -lminiupnpc -lm @TOR_LIB_IPHLPAPI@ +miniupnpc_cppflags = @TOR_CPPFLAGS_libminiupnpc@ +else +miniupnpc_ldflags = +miniupnpc_ldadd = +miniupnpc_cppflags = +endif + +src_tools_tor_fw_helper_tor_fw_helper_LDFLAGS = $(nat_pmp_ldflags) $(miniupnpc_ldflags) +src_tools_tor_fw_helper_tor_fw_helper_LDADD = src/common/libor.a $(nat_pmp_ldadd) $(miniupnpc_ldadd) @TOR_LIB_WS32@ +src_tools_tor_fw_helper_tor_fw_helper_CPPFLAGS = $(nat_pmp_cppflags) $(miniupnpc_cppflags) diff --git a/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c index 0e0b385f9..ee6d5f343 100644 --- a/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c +++ b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.c @@ -60,15 +60,15 @@ tor_natpmp_init(tor_fw_options_t *tor_fw_options, void *backend_state) state->lease = NATPMP_DEFAULT_LEASE; if (tor_fw_options->verbose) - fprintf(stdout, "V: natpmp init...\n"); + fprintf(stderr, "V: natpmp init...\n"); r = initnatpmp(&(state->natpmp), 0, 0); if (r == 0) { state->init = 1; - fprintf(stdout, "tor-fw-helper: natpmp initialized...\n"); + fprintf(stderr, "V: natpmp initialized...\n"); return r; } else { - fprintf(stderr, "tor-fw-helper: natpmp failed to initialize...\n"); + fprintf(stderr, "V: natpmp failed to initialize...\n"); return r; } } @@ -80,10 +80,10 @@ tor_natpmp_cleanup(tor_fw_options_t *tor_fw_options, void *backend_state) natpmp_state_t *state = (natpmp_state_t *) backend_state; int r = 0; if (tor_fw_options->verbose) - fprintf(stdout, "V: natpmp cleanup...\n"); + fprintf(stderr, "V: natpmp cleanup...\n"); r = closenatpmp(&(state->natpmp)); if (tor_fw_options->verbose) - fprintf(stdout, "V: closing natpmp socket: %d\n", r); + fprintf(stderr, "V: closing natpmp socket: %d\n", r); return r; } @@ -101,7 +101,7 @@ wait_until_fd_readable(tor_socket_t fd, struct timeval *timeout) FD_SET(fd, &fds); r = select(fd+1, &fds, NULL, NULL, timeout); if (r == -1) { - fprintf(stdout, "V: select failed in wait_until_fd_readable: %s\n", + fprintf(stderr, "V: select failed in wait_until_fd_readable: %s\n", strerror(errno)); return -1; } @@ -110,27 +110,25 @@ wait_until_fd_readable(tor_socket_t fd, struct timeval *timeout) return 0; } -/** Add a TCP port mapping for a single port stored in <b>tor_fw_options</b> - * using the <b>natpmp_t</b> stored in <b>backend_state</b>. */ int -tor_natpmp_add_tcp_mapping(tor_fw_options_t *tor_fw_options, - void *backend_state) +tor_natpmp_add_tcp_mapping(uint16_t internal_port, uint16_t external_port, + int is_verbose, void *backend_state) { - natpmp_state_t *state = (natpmp_state_t *) backend_state; int r = 0; int x = 0; int sav_errno; + natpmp_state_t *state = (natpmp_state_t *) backend_state; struct timeval timeout; - if (tor_fw_options->verbose) - fprintf(stdout, "V: sending natpmp portmapping request...\n"); + if (is_verbose) + fprintf(stderr, "V: sending natpmp portmapping request...\n"); r = sendnewportmappingrequest(&(state->natpmp), state->protocol, - tor_fw_options->internal_port, - tor_fw_options->external_port, + internal_port, + external_port, state->lease); - if (tor_fw_options->verbose) - fprintf(stdout, "tor-fw-helper: NAT-PMP sendnewportmappingrequest " + if (is_verbose) + fprintf(stderr, "tor-fw-helper: NAT-PMP sendnewportmappingrequest " "returned %d (%s)\n", r, r==12?"SUCCESS":"FAILED"); do { @@ -139,8 +137,8 @@ tor_natpmp_add_tcp_mapping(tor_fw_options_t *tor_fw_options, if (x == -1) return -1; - if (tor_fw_options->verbose) - fprintf(stdout, "V: attempting to readnatpmpreponseorretry...\n"); + if (is_verbose) + fprintf(stderr, "V: attempting to readnatpmpreponseorretry...\n"); r = readnatpmpresponseorretry(&(state->natpmp), &(state->response)); sav_errno = errno; @@ -163,16 +161,14 @@ tor_natpmp_add_tcp_mapping(tor_fw_options_t *tor_fw_options, } if (r == NATPMP_SUCCESS) { - fprintf(stdout, "tor-fw-helper: NAT-PMP mapped public port %hu to" + fprintf(stderr, "tor-fw-helper: NAT-PMP mapped public port %hu to" " localport %hu liftime %u\n", (state->response).pnu.newportmapping.mappedpublicport, (state->response).pnu.newportmapping.privateport, (state->response).pnu.newportmapping.lifetime); } - tor_fw_options->nat_pmp_status = 1; - - return r; + return (r == NATPMP_SUCCESS) ? 0 : -1; } /** Fetch our likely public IP from our upstream NAT-PMP enabled NAT device. @@ -189,7 +185,7 @@ tor_natpmp_fetch_public_ip(tor_fw_options_t *tor_fw_options, struct timeval timeout; r = sendpublicaddressrequest(&(state->natpmp)); - fprintf(stdout, "tor-fw-helper: NAT-PMP sendpublicaddressrequest returned" + fprintf(stderr, "tor-fw-helper: NAT-PMP sendpublicaddressrequest returned" " %d (%s)\n", r, r==2?"SUCCESS":"FAILED"); do { @@ -200,12 +196,12 @@ tor_natpmp_fetch_public_ip(tor_fw_options_t *tor_fw_options, return -1; if (tor_fw_options->verbose) - fprintf(stdout, "V: NAT-PMP attempting to read reponse...\n"); + fprintf(stderr, "V: NAT-PMP attempting to read reponse...\n"); r = readnatpmpresponseorretry(&(state->natpmp), &(state->response)); sav_errno = errno; if (tor_fw_options->verbose) - fprintf(stdout, "V: NAT-PMP readnatpmpresponseorretry returned" + fprintf(stderr, "V: NAT-PMP readnatpmpresponseorretry returned" " %d\n", r); if ( r < 0 && r != NATPMP_TRYAGAIN) { @@ -223,15 +219,15 @@ tor_natpmp_fetch_public_ip(tor_fw_options_t *tor_fw_options, return r; } - fprintf(stdout, "tor-fw-helper: ExternalIPAddress = %s\n", + fprintf(stderr, "tor-fw-helper: ExternalIPAddress = %s\n", inet_ntoa((state->response).pnu.publicaddress.addr)); tor_fw_options->public_ip_status = 1; if (tor_fw_options->verbose) { - fprintf(stdout, "V: result = %u\n", r); - fprintf(stdout, "V: type = %u\n", (state->response).type); - fprintf(stdout, "V: resultcode = %u\n", (state->response).resultcode); - fprintf(stdout, "V: epoch = %u\n", (state->response).epoch); + fprintf(stderr, "V: result = %u\n", r); + fprintf(stderr, "V: type = %u\n", (state->response).type); + fprintf(stderr, "V: resultcode = %u\n", (state->response).resultcode); + fprintf(stderr, "V: epoch = %u\n", (state->response).epoch); } return r; diff --git a/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h index 54f541bcf..0f97236af 100644 --- a/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h +++ b/src/tools/tor-fw-helper/tor-fw-helper-natpmp.h @@ -36,8 +36,8 @@ int tor_natpmp_init(tor_fw_options_t *tor_fw_options, void *backend_state); int tor_natpmp_cleanup(tor_fw_options_t *tor_fw_options, void *backend_state); -int tor_natpmp_add_tcp_mapping(tor_fw_options_t *tor_fw_options, - void *backend_state); +int tor_natpmp_add_tcp_mapping(uint16_t internal_port, uint16_t external_port, + int is_verbose, void *backend_state); int tor_natpmp_fetch_public_ip(tor_fw_options_t *tor_fw_options, void *backend_state); diff --git a/src/tools/tor-fw-helper/tor-fw-helper-upnp.c b/src/tools/tor-fw-helper/tor-fw-helper-upnp.c index 7c104f11c..e5c33db0b 100644 --- a/src/tools/tor-fw-helper/tor-fw-helper-upnp.c +++ b/src/tools/tor-fw-helper/tor-fw-helper-upnp.c @@ -91,7 +91,7 @@ tor_upnp_init(tor_fw_options_t *options, void *backend_state) assert(options); r = UPNP_GetValidIGD(devlist, &(state->urls), &(state->data), state->lanaddr, UPNP_LANADDR_SZ); - fprintf(stdout, "tor-fw-helper: UPnP GetValidIGD returned: %d (%s)\n", r, + fprintf(stderr, "tor-fw-helper: UPnP GetValidIGD returned: %d (%s)\n", r, r==UPNP_SUCCESS?"SUCCESS":"FAILED"); freeUPNPDevlist(devlist); @@ -141,7 +141,7 @@ tor_upnp_fetch_public_ip(tor_fw_options_t *options, void *backend_state) goto err; if (externalIPAddress[0]) { - fprintf(stdout, "tor-fw-helper: ExternalIPAddress = %s\n", + fprintf(stderr, "tor-fw-helper: ExternalIPAddress = %s\n", externalIPAddress); tor_upnp_cleanup(options, state); options->public_ip_status = 1; return UPNP_ERR_SUCCESS; @@ -154,44 +154,40 @@ tor_upnp_fetch_public_ip(tor_fw_options_t *options, void *backend_state) return UPNP_ERR_GETEXTERNALIP; } -/** Add a TCP port mapping for a single port stored in <b>tor_fw_options</b> - * and store the results in <b>backend_state</b>. */ int -tor_upnp_add_tcp_mapping(tor_fw_options_t *options, void *backend_state) +tor_upnp_add_tcp_mapping(uint16_t internal_port, uint16_t external_port, + int is_verbose, void *backend_state) { - miniupnpc_state_t *state = (miniupnpc_state_t *) backend_state; - int r; + int retval; char internal_port_str[6]; char external_port_str[6]; + miniupnpc_state_t *state = (miniupnpc_state_t *) backend_state; if (!state->init) { - r = tor_upnp_init(options, state); - if (r != UPNP_ERR_SUCCESS) - return r; + fprintf(stderr, "E: %s but state is not initialized.\n", __func__); + return -1; } - if (options->verbose) - fprintf(stdout, "V: internal port: %d, external port: %d\n", - (int)options->internal_port, (int)options->external_port); + if (is_verbose) + fprintf(stderr, "V: UPnP: internal port: %u, external port: %u\n", + internal_port, external_port); tor_snprintf(internal_port_str, sizeof(internal_port_str), - "%d", (int)options->internal_port); + "%u", internal_port); tor_snprintf(external_port_str, sizeof(external_port_str), - "%d", (int)options->external_port); + "%u", external_port); - r = UPNP_AddPortMapping(state->urls.controlURL, - state->data.first.servicetype, - external_port_str, internal_port_str, + retval = UPNP_AddPortMapping(state->urls.controlURL, + state->data.first.servicetype, + external_port_str, internal_port_str, #ifdef MINIUPNPC15 - state->lanaddr, UPNP_DESC, "TCP", 0); + state->lanaddr, UPNP_DESC, "TCP", 0); #else - state->lanaddr, UPNP_DESC, "TCP", 0, 0); + state->lanaddr, UPNP_DESC, "TCP", 0, 0); #endif - if (r != UPNPCOMMAND_SUCCESS) - return UPNP_ERR_ADDPORTMAPPING; - options->upnp_status = 1; - return UPNP_ERR_SUCCESS; + return (retval == UPNP_ERR_SUCCESS) ? 0 : -1; } + #endif diff --git a/src/tools/tor-fw-helper/tor-fw-helper-upnp.h b/src/tools/tor-fw-helper/tor-fw-helper-upnp.h index f037c75ba..3a061981d 100644 --- a/src/tools/tor-fw-helper/tor-fw-helper-upnp.h +++ b/src/tools/tor-fw-helper/tor-fw-helper-upnp.h @@ -36,7 +36,8 @@ int tor_upnp_cleanup(tor_fw_options_t *options, void *backend_state); int tor_upnp_fetch_public_ip(tor_fw_options_t *options, void *backend_state); -int tor_upnp_add_tcp_mapping(tor_fw_options_t *options, void *backend_state); +int tor_upnp_add_tcp_mapping(uint16_t internal_port, uint16_t external_port, + int is_verbose, void *backend_state); #endif #endif diff --git a/src/tools/tor-fw-helper/tor-fw-helper.c b/src/tools/tor-fw-helper/tor-fw-helper.c index 0510e65d1..4efe515cc 100644 --- a/src/tools/tor-fw-helper/tor-fw-helper.c +++ b/src/tools/tor-fw-helper/tor-fw-helper.c @@ -20,6 +20,9 @@ #include <getopt.h> #include <time.h> #include <string.h> +#include <assert.h> + +#include "container.h" #ifdef _WIN32 #include <winsock2.h> @@ -45,7 +48,7 @@ typedef struct backends_t { void *backend_state[MAX_BACKENDS]; } backends_t; -/** Initalize each backend helper with the user input stored in <b>options</b> +/** Initialize each backend helper with the user input stored in <b>options</b> * and put the results in the <b>backends</b> struct. */ static int init_backends(tor_fw_options_t *options, backends_t *backends) @@ -97,10 +100,7 @@ usage(void) " [-T|--Test]\n" " [-v|--verbose]\n" " [-g|--fetch-public-ip]\n" - " -i|--internal-or-port [TCP port]\n" - " [-e|--external-or-port [TCP port]]\n" - " [-d|--internal-dir-port [TCP port]\n" - " [-p|--external-dir-port [TCP port]]]\n"); + " [-p|--forward-port ([<external port>]:<internal port>])\n"); } /** Log commandline options to a hardcoded file <b>tor-fw-helper.log</b> in the @@ -125,7 +125,7 @@ log_commandline_options(int argc, char **argv) if (retval < 0) goto error; - retval = fprintf(stdout, "ARG: %d: %s\n", i, argv[i]); + retval = fprintf(stderr, "ARG: %d: %s\n", i, argv[i]); if (retval < 0) goto error; } @@ -152,82 +152,141 @@ tor_fw_fetch_public_ip(tor_fw_options_t *tor_fw_options, int r = 0; if (tor_fw_options->verbose) - fprintf(stdout, "V: tor_fw_fetch_public_ip\n"); + fprintf(stderr, "V: tor_fw_fetch_public_ip\n"); for (i=0; i<backends->n_backends; ++i) { if (tor_fw_options->verbose) { - fprintf(stdout, "V: running backend_state now: %i\n", i); - fprintf(stdout, "V: size of backend state: %u\n", + fprintf(stderr, "V: running backend_state now: %i\n", i); + fprintf(stderr, "V: size of backend state: %u\n", (int)(backends->backend_ops)[i].state_len); - fprintf(stdout, "V: backend state name: %s\n", + fprintf(stderr, "V: backend state name: %s\n", (char *)(backends->backend_ops)[i].name); } r = backends->backend_ops[i].fetch_public_ip(tor_fw_options, backends->backend_state[i]); - fprintf(stdout, "tor-fw-helper: tor_fw_fetch_public_ip backend %s " + fprintf(stderr, "tor-fw-helper: tor_fw_fetch_public_ip backend %s " " returned: %i\n", (char *)(backends->backend_ops)[i].name, r); } } -/** Iterate over each of the supported <b>backends</b> and attempt to add a - * port forward for the OR port stored in <b>tor_fw_options</b>. */ +/** Print a spec-conformant string to stdout describing the results of + * the TCP port forwarding operation from <b>external_port</b> to + * <b>internal_port</b>. */ static void -tor_fw_add_or_port(tor_fw_options_t *tor_fw_options, - backends_t *backends) +tor_fw_helper_report_port_fw_results(uint16_t internal_port, + uint16_t external_port, + int succeded, + const char *message) +{ + char *report_string = NULL; + + tor_asprintf(&report_string, "%s %s %u %u %s %s\n", + "tor-fw-helper", + "tcp-forward", + external_port, internal_port, + succeded ? "SUCCESS" : "FAIL", + message); + fprintf(stdout, "%s", report_string); + fflush(stdout); + tor_free(report_string); +} + +#define tor_fw_helper_report_port_fw_fail(i, e, m) \ + tor_fw_helper_report_port_fw_results((i), (e), 0, (m)) + +#define tor_fw_helper_report_port_fw_success(i, e, m) \ + tor_fw_helper_report_port_fw_results((i), (e), 1, (m)) + +/** Return a heap-allocated string containing the list of our + * backends. It can be used in log messages. Be sure to free it + * afterwards! */ +static char * +get_list_of_backends_string(backends_t *backends) { + char *backend_names = NULL; int i; - int r = 0; + smartlist_t *backend_names_sl = smartlist_new(); - if (tor_fw_options->verbose) - fprintf(stdout, "V: tor_fw_add_or_port\n"); + assert(backends->n_backends); - for (i=0; i<backends->n_backends; ++i) { - if (tor_fw_options->verbose) { - fprintf(stdout, "V: running backend_state now: %i\n", i); - fprintf(stdout, "V: size of backend state: %u\n", - (int)(backends->backend_ops)[i].state_len); - fprintf(stdout, "V: backend state name: %s\n", - (const char *) backends->backend_ops[i].name); - } - r = backends->backend_ops[i].add_tcp_mapping(tor_fw_options, - backends->backend_state[i]); - fprintf(stdout, "tor-fw-helper: tor_fw_add_or_port backend %s " - "returned: %i\n", (const char *) backends->backend_ops[i].name, r); - } + for (i=0; i<backends->n_backends; ++i) + smartlist_add(backend_names_sl, (char *) backends->backend_ops[i].name); + + backend_names = smartlist_join_strings(backend_names_sl, ", ", 0, NULL); + smartlist_free(backend_names_sl); + + return backend_names; } /** Iterate over each of the supported <b>backends</b> and attempt to add a - * port forward for the Dir port stored in <b>tor_fw_options</b>. */ + * port forward for the port stored in <b>tor_fw_options</b>. */ static void -tor_fw_add_dir_port(tor_fw_options_t *tor_fw_options, - backends_t *backends) +tor_fw_add_ports(tor_fw_options_t *tor_fw_options, + backends_t *backends) { int i; int r = 0; + int succeeded = 0; if (tor_fw_options->verbose) - fprintf(stdout, "V: tor_fw_add_dir_port\n"); + fprintf(stderr, "V: %s\n", __func__); - for (i=0; i<backends->n_backends; ++i) { - if (tor_fw_options->verbose) { - fprintf(stdout, "V: running backend_state now: %i\n", i); - fprintf(stdout, "V: size of backend state: %u\n", - (int)(backends->backend_ops)[i].state_len); - fprintf(stdout, "V: backend state name: %s\n", - (char *)(backends->backend_ops)[i].name); + /** Loop all ports that need to be forwarded, and try to use our + * backends for each port. If a backend succeeds, break the loop, + * report success and get to the next port. If all backends fail, + * report failure for that port. */ + SMARTLIST_FOREACH_BEGIN(tor_fw_options->ports_to_forward, + port_to_forward_t *, port_to_forward) { + + succeeded = 0; + + for (i=0; i<backends->n_backends; ++i) { + if (tor_fw_options->verbose) { + fprintf(stderr, "V: running backend_state now: %i\n", i); + fprintf(stderr, "V: size of backend state: %u\n", + (int)(backends->backend_ops)[i].state_len); + fprintf(stderr, "V: backend state name: %s\n", + (const char *) backends->backend_ops[i].name); + } + + r = + backends->backend_ops[i].add_tcp_mapping(port_to_forward->internal_port, + port_to_forward->external_port, + tor_fw_options->verbose, + backends->backend_state[i]); + if (r == 0) { /* backend success */ + tor_fw_helper_report_port_fw_success(port_to_forward->internal_port, + port_to_forward->external_port, + backends->backend_ops[i].name); + succeeded = 1; + break; + } + + fprintf(stderr, "tor-fw-helper: tor_fw_add_port backend %s " + "returned: %i\n", + (const char *) backends->backend_ops[i].name, r); } - r = backends->backend_ops[i].add_tcp_mapping(tor_fw_options, - backends->backend_state[i]); - fprintf(stdout, "tor-fw-helper: tor_fw_add_dir_port backend %s " - "returned: %i\n", (const char *)backends->backend_ops[i].name, r); - } + + if (!succeeded) { /* all backends failed */ + char *list_of_backends_str = get_list_of_backends_string(backends); + char *fail_msg = NULL; + tor_asprintf(&fail_msg, "All port forwarding backends (%s) failed.", + list_of_backends_str); + tor_fw_helper_report_port_fw_fail(port_to_forward->internal_port, + port_to_forward->external_port, + fail_msg); + tor_free(list_of_backends_str); + tor_free(fail_msg); + } + + } SMARTLIST_FOREACH_END(port_to_forward); } /** Called before we make any calls to network-related functions. * (Some operating systems require their network libraries to be * initialized.) (from common/compat.c) */ static int -network_init(void) +tor_fw_helper_network_init(void) { #ifdef _WIN32 /* This silly exercise is necessary before windows will allow @@ -247,6 +306,67 @@ network_init(void) return 0; } +/** Parse the '-p' argument of tor-fw-helper. Its format is + * [<external port>]:<internal port>, and <external port> is optional. + * Return NULL if <b>arg</b> was c0rrupted. */ +static port_to_forward_t * +parse_port(const char *arg) +{ + smartlist_t *sl = smartlist_new(); + port_to_forward_t *port_to_forward = NULL; + char *port_str = NULL; + int ok; + int port; + + smartlist_split_string(sl, arg, ":", 0, 0); + if (smartlist_len(sl) != 2) + goto err; + + port_to_forward = tor_malloc(sizeof(port_to_forward_t)); + if (!port_to_forward) + goto err; + + port_str = smartlist_get(sl, 0); /* macroify ? */ + port = (int)tor_parse_long(port_str, 10, 1, 65535, &ok, NULL); + if (!ok && strlen(port_str)) /* ":1555" is valid */ + goto err; + port_to_forward->external_port = port; + + port_str = smartlist_get(sl, 1); + port = (int)tor_parse_long(port_str, 10, 1, 65535, &ok, NULL); + if (!ok) + goto err; + port_to_forward->internal_port = port; + + goto done; + + err: + tor_free(port_to_forward); + + done: + SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp)); + smartlist_free(sl); + + return port_to_forward; +} + +/** Report a failure of epic proportions: We didn't manage to + * initialize any port forwarding backends. */ +static void +report_full_fail(const smartlist_t *ports_to_forward) +{ + if (!ports_to_forward) + return; + + SMARTLIST_FOREACH_BEGIN(ports_to_forward, + const port_to_forward_t *, port_to_forward) { + tor_fw_helper_report_port_fw_fail(port_to_forward->internal_port, + port_to_forward->external_port, + "All backends (NAT-PMP, UPnP) failed " + "to initialize!"); /* XXX hardcoded */ + } SMARTLIST_FOREACH_END(port_to_forward); +} + int main(int argc, char **argv) { @@ -259,22 +379,20 @@ main(int argc, char **argv) memset(&tor_fw_options, 0, sizeof(tor_fw_options)); memset(&backend_state, 0, sizeof(backend_state)); + // Parse CLI arguments. while (1) { int option_index = 0; static struct option long_options[] = { {"verbose", 0, 0, 'v'}, {"help", 0, 0, 'h'}, - {"internal-or-port", 1, 0, 'i'}, - {"external-or-port", 1, 0, 'e'}, - {"internal-dir-port", 1, 0, 'd'}, - {"external-dir-port", 1, 0, 'p'}, + {"port", 1, 0, 'p'}, {"fetch-public-ip", 0, 0, 'g'}, {"test-commandline", 0, 0, 'T'}, {0, 0, 0, 0} }; - c = getopt_long(argc, argv, "vhi:e:d:p:gT", + c = getopt_long(argc, argv, "vhp:gT", long_options, &option_index); if (c == -1) break; @@ -282,14 +400,31 @@ main(int argc, char **argv) switch (c) { case 'v': tor_fw_options.verbose = 1; break; case 'h': tor_fw_options.help = 1; usage(); exit(1); break; - case 'i': sscanf(optarg, "%hu", &tor_fw_options.private_or_port); - break; - case 'e': sscanf(optarg, "%hu", &tor_fw_options.public_or_port); - break; - case 'd': sscanf(optarg, "%hu", &tor_fw_options.private_dir_port); - break; - case 'p': sscanf(optarg, "%hu", &tor_fw_options.public_dir_port); + case 'p': { + port_to_forward_t *port_to_forward = parse_port(optarg); + if (!port_to_forward) { + fprintf(stderr, "E: Failed to parse '%s'.\n", optarg); + usage(); + exit(1); + } + + /* If no external port was given (it's optional), set it to be + * equal with the internal port. */ + if (!port_to_forward->external_port) { + assert(port_to_forward->internal_port); + if (tor_fw_options.verbose) + fprintf(stderr, "V: No external port was given. Setting to %u.\n", + port_to_forward->internal_port); + port_to_forward->external_port = port_to_forward->internal_port; + } + + if (!tor_fw_options.ports_to_forward) + tor_fw_options.ports_to_forward = smartlist_new(); + + smartlist_add(tor_fw_options.ports_to_forward, port_to_forward); + break; + } case 'g': tor_fw_options.fetch_public_ip = 1; break; case 'T': tor_fw_options.test_commandline = 1; break; case '?': break; @@ -297,98 +432,68 @@ main(int argc, char **argv) } } - if (tor_fw_options.verbose) { - fprintf(stderr, "V: tor-fw-helper version %s\n" - "V: We were called with the following arguments:\n" - "V: verbose = %d, help = %d, pub or port = %u, " - "priv or port = %u\n" - "V: pub dir port = %u, priv dir port = %u\n" - "V: fetch_public_ip = %u\n", - tor_fw_version, tor_fw_options.verbose, tor_fw_options.help, - tor_fw_options.private_or_port, tor_fw_options.public_or_port, - tor_fw_options.private_dir_port, tor_fw_options.public_dir_port, - tor_fw_options.fetch_public_ip); + { // Verbose output + + if (tor_fw_options.verbose) + fprintf(stderr, "V: tor-fw-helper version %s\n" + "V: We were called with the following arguments:\n" + "V: verbose = %d, help = %d, fetch_public_ip = %u\n", + tor_fw_version, tor_fw_options.verbose, tor_fw_options.help, + tor_fw_options.fetch_public_ip); + + if (tor_fw_options.verbose && tor_fw_options.ports_to_forward) { + fprintf(stderr, "V: TCP forwarding:\n"); + SMARTLIST_FOREACH(tor_fw_options.ports_to_forward, + const port_to_forward_t *, port_to_forward, + fprintf(stderr, "V: External: %u, Internal: %u\n", + port_to_forward->external_port, + port_to_forward->internal_port)); + } } if (tor_fw_options.test_commandline) { return log_commandline_options(argc, argv); } - /* At the very least, we require an ORPort; - Given a private ORPort, we can ask for a mapping that matches the port - externally. - */ - if (!tor_fw_options.private_or_port && !tor_fw_options.fetch_public_ip) { - fprintf(stderr, "E: We require an ORPort or fetch_public_ip" - " request!\n"); + // See if the user actually wants us to do something. + if (!tor_fw_options.fetch_public_ip && !tor_fw_options.ports_to_forward) { + fprintf(stderr, "E: We require a port to be forwarded or " + "fetch_public_ip request!\n"); usage(); exit(1); - } else { - /* When we only have one ORPort, internal/external are - set to be the same.*/ - if (!tor_fw_options.public_or_port && tor_fw_options.private_or_port) { - if (tor_fw_options.verbose) - fprintf(stdout, "V: We're setting public_or_port = " - "private_or_port.\n"); - tor_fw_options.public_or_port = tor_fw_options.private_or_port; - } - } - if (!tor_fw_options.private_dir_port) { - if (tor_fw_options.verbose) - fprintf(stdout, "V: We have no DirPort; no hole punching for " - "DirPorts\n"); - - } else { - /* When we only have one DirPort, internal/external are - set to be the same.*/ - if (!tor_fw_options.public_dir_port && tor_fw_options.private_dir_port) { - if (tor_fw_options.verbose) - fprintf(stdout, "V: We're setting public_or_port = " - "private_or_port.\n"); - - tor_fw_options.public_dir_port = tor_fw_options.private_dir_port; - } - } - - if (tor_fw_options.verbose) { - fprintf(stdout, "V: pub or port = %u, priv or port = %u\n" - "V: pub dir port = %u, priv dir port = %u\n", - tor_fw_options.private_or_port, tor_fw_options.public_or_port, - tor_fw_options.private_dir_port, - tor_fw_options.public_dir_port); } // Initialize networking - if (network_init()) + if (tor_fw_helper_network_init()) exit(1); // Initalize the various fw-helper backend helpers r = init_backends(&tor_fw_options, &backend_state); - if (r) - printf("tor-fw-helper: %i NAT traversal helper(s) loaded\n", r); - - if (tor_fw_options.fetch_public_ip) { - tor_fw_fetch_public_ip(&tor_fw_options, &backend_state); + if (!r) { // all backends failed: + // report our failure + report_full_fail(tor_fw_options.ports_to_forward); + fprintf(stderr, "tor-fw-helper: All backends failed.\n"); + exit(1); + } else { // some backends succeeded: + fprintf(stderr, "tor-fw-helper: %i NAT traversal helper(s) loaded\n", r); } - if (tor_fw_options.private_or_port) { - tor_fw_options.internal_port = tor_fw_options.private_or_port; - tor_fw_options.external_port = tor_fw_options.private_or_port; - tor_fw_add_or_port(&tor_fw_options, &backend_state); + // Forward TCP ports. + if (tor_fw_options.ports_to_forward) { + tor_fw_add_ports(&tor_fw_options, &backend_state); } - if (tor_fw_options.private_dir_port) { - tor_fw_options.internal_port = tor_fw_options.private_dir_port; - tor_fw_options.external_port = tor_fw_options.private_dir_port; - tor_fw_add_dir_port(&tor_fw_options, &backend_state); + // Fetch our public IP. + if (tor_fw_options.fetch_public_ip) { + tor_fw_fetch_public_ip(&tor_fw_options, &backend_state); } - r = (((tor_fw_options.nat_pmp_status | tor_fw_options.upnp_status) - |tor_fw_options.public_ip_status)); - if (r > 0) { - fprintf(stdout, "tor-fw-helper: SUCCESS\n"); - } else { - fprintf(stderr, "tor-fw-helper: FAILURE\n"); + // Cleanup and exit. + if (tor_fw_options.ports_to_forward) { + SMARTLIST_FOREACH(tor_fw_options.ports_to_forward, + port_to_forward_t *, port, + tor_free(port)); + smartlist_free(tor_fw_options.ports_to_forward); } exit(r); diff --git a/src/tools/tor-fw-helper/tor-fw-helper.h b/src/tools/tor-fw-helper/tor-fw-helper.h index 058afc4e0..08f94d083 100644 --- a/src/tools/tor-fw-helper/tor-fw-helper.h +++ b/src/tools/tor-fw-helper/tor-fw-helper.h @@ -17,24 +17,26 @@ #include <time.h> /** The current version of tor-fw-helper. */ -#define tor_fw_version "0.1" +#define tor_fw_version "0.2" /** This is an arbitrary hard limit - We currently have two (NAT-PMP and UPnP). We're likely going to add the Intel UPnP library but nothing else comes to mind at the moment. */ #define MAX_BACKENDS 23 +/** Forward traffic received in port <b>external_port</b> in the + * external side of our NAT to <b>internal_port</b> in this host. */ +typedef struct { + uint16_t external_port; + uint16_t internal_port; +} port_to_forward_t; + /** This is where we store parsed commandline options. */ typedef struct { int verbose; int help; int test_commandline; - uint16_t private_dir_port; - uint16_t private_or_port; - uint16_t public_dir_port; - uint16_t public_or_port; - uint16_t internal_port; - uint16_t external_port; + struct smartlist_t *ports_to_forward; int fetch_public_ip; int nat_pmp_status; int upnp_status; @@ -50,8 +52,8 @@ typedef struct tor_fw_backend_t { int (*init)(tor_fw_options_t *options, void *backend_state); int (*cleanup)(tor_fw_options_t *options, void *backend_state); int (*fetch_public_ip)(tor_fw_options_t *options, void *backend_state); - int (*add_tcp_mapping)(tor_fw_options_t *options, void *backend_state); + int (*add_tcp_mapping)(uint16_t internal_port, uint16_t external_port, + int is_verbose, void *backend_state); } tor_fw_backend_t; - #endif diff --git a/src/win32/Makefile.am b/src/win32/Makefile.am deleted file mode 100644 index 7f5d74248..000000000 --- a/src/win32/Makefile.am +++ /dev/null @@ -1,3 +0,0 @@ - -EXTRA_DIST = orconfig.h - diff --git a/src/win32/include.am b/src/win32/include.am new file mode 100644 index 000000000..dad59af3a --- /dev/null +++ b/src/win32/include.am @@ -0,0 +1,3 @@ + +EXTRA_DIST+= src/win32/orconfig.h + diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index d780d5d73..beebc6dbd 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -232,7 +232,7 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.2.3.18-rc-dev" +#define VERSION "0.2.4.2-alpha-dev" diff --git a/tor.spec.in b/tor.spec.in deleted file mode 100644 index b452c9bb3..000000000 --- a/tor.spec.in +++ /dev/null @@ -1,360 +0,0 @@ -## NOTE: tor.spec is autogenerated from tor.spec.in . Edit the latter, -## not the former. - -## Things that need to be edited frequently -# -# This should be incremented whenever the spec file changes, but -# can drop back to zero at a new Tor version - -%define specver 1 - -## Things users may want to change -# -# User (and group) name under which the Tor daemon runs. - -%define toruser @TORUSER@ -%define torgroup @TORGROUP@ - -## Version song and dance -# -# This should be the Tor version number, as it appears on the tarball, -# including any "pre<x>" or "rc<y>" suffix. This gets massaged to -# create the RPM version number, in a way that depends on the Tor -# numbering scheme. -%define native_version @VERSION@ - -%define version %(echo %{native_version} | sed -e 's/-/./g') - -## Define output filename -# -# This creates filenames based upon the value of target_cpu defined above - -## Release and OS identification song and dance -# -# This identifies the lineage of the spec file. This file is the -# standard one that comes with Tor; various distributions may -# have their own ideas about the right ways to do things. -%define pkgspec tor - -# This spec is intended to build and install on multiple distributions -# (someday). Detect the distribution we're building on. - -%define is_rh %(test -e /etc/redhat-release && echo 1 || echo 0) -%define is_fc %(test -e /etc/fedora-release && echo 1 || echo 0) -%define is_mdk %(test -e /etc/mandrake-release && echo 1 || echo 0) -%define is_suse %(test -e /etc/SuSE-release && echo 1 || echo 0) -%define is_rfl %(test -e /etc/redflag-release && echo 1 || echo 0) - -%if %{is_fc} -%define ostag %(sed -e 's/^.*release /fc/' -e 's/ .*$//' -e 's/\\./_/g' < /etc/fedora-release) -%endif - -%if %{is_rh} -%define ostag %(sed -e 's/^.*release /rh/' -e 's/ .*$//' -e 's/\\./_/g' < /etc/redhat-release) -%endif - -%if %{is_mdk} -%define ostag mdk -%endif - -%if %{is_suse} -%define ostag suse%(grep openSUSE /etc/SuSE-release | awk '{print $2}' | sed -e 's/\\./_/') -%endif - -%if %{is_rfl} -%define ostag %(sed -e 's/^.*Desktop /redflag/' -e 's/ .*$//' -e 's/\\./_/g' < /etc/redflag-release) -%endif - -# Using the build date ensures that every build really does get -# a different release number. We use this trick for CVS versions. -# For release versions, we don't want or need it. -%define is_dev_version %(echo %{native_version} | grep 'dev' > /dev/null && echo 1 || echo 0) - -%if %{is_dev_version} -%define blddate %(date -u +"%Y%m%d%H%M") -%define release %{pkgspec}.%{specver}.%{ostag}.%{blddate} -%else -%define release %{pkgspec}.%{specver}.%{ostag} -%endif - -## General-purpose macros -# -# Some systems don't have some macros. If a macro doesn't seem -# to exist on your system, add it here... - -%if %{!?__make:1}%{?__make:0} -%define __make make -%endif - -%if %{!?make:1}%{?make:0} -%define make %{__make} -%endif - -%if %{!?_localstatedir:1}%{?_localstatedir:0} -%define _localstatedir @LOCALSTATEDIR@ -%endif - -## Package information -# -Name: tor -Version: %{version} -Release: %{release} - -Summary: Anonymizing overlay network for TCP (The onion router) -URL: https://www.torproject.org/ -Group: System Environment/Daemons - -License: 3-clause BSD -Vendor: The Tor Project (https://torproject.org) -Packager: Erinn Clark <erinn@torproject.org> - -Requires: openssl >= 0.9.7, libevent >= 1.4.13 -BuildRequires: openssl-devel >= 0.9.7, libevent-devel >= 1.4.13, asciidoc - -# Fedora 16 and RHEL 5 have following conflicting packages according to rpm search -%if %{is_rh} -Conflicts: tor-core, tor-lsb, tor-upstart -%endif - -Requires(pre): /usr/bin/id, /bin/date, /bin/sh -Requires(pre): %{_sbindir}/useradd, %{_sbindir}/groupadd - -Source0: https://www.torproject.org/dist/%{name}-%{native_version}.tar.gz - -BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root - -%description -Tor is a connection-based low-latency anonymous communication system. - -This package provides the "tor" program, which serves as both a client and -a relay node. Scripts will automatically create a "%{toruser}" user and -a "%{torgroup}" group, and set tor up to run as a daemon when the system -is rebooted. - -Applications connect to the local Tor proxy using the SOCKS -protocol. The tor client chooses a path through a set of relays, in -which each relay knows its predecessor and successor, but no -others. Traffic flowing down the circuit is unwrapped by a symmetric -key at each relay, which reveals the downstream relay. - -Warnings: Tor does no protocol cleaning. That means there is a danger -that application protocols and associated programs can be induced to -reveal information about the initiator. Tor depends on Privoxy or -similar protocol cleaners to solve this problem. This is alpha code, -and is even more likely than released code to have anonymity-spoiling -bugs. The present network is small -- this further reduces the -strength of the anonymity provided. Tor is not presently suitable -for high-stakes anonymity. - -%prep -%setup -q -n %{name}-%{native_version} - -%build -%if %{is_suse} -%configure --with-tor-user=%{toruser} --with-tor-group=%{torgroup} --docdir=%{_docdir}/%{name} -%else -%configure --with-tor-user=%{toruser} --with-tor-group=%{torgroup} -%endif -%make - -%install -%makeinstall - -# Install init script and control script -%__mkdir_p ${RPM_BUILD_ROOT}%{_initrddir} -%if %{is_suse} -%__install -p -m 755 contrib/suse/tor.sh ${RPM_BUILD_ROOT}%{_initrddir}/%{name} -%else -%__install -p -m 755 contrib/tor.sh ${RPM_BUILD_ROOT}%{_initrddir}/%{name} -%endif -%__install -p -m 755 contrib/torctl ${RPM_BUILD_ROOT}%{_bindir} - -# Set up config file; "sample" file implements a basic user node. -%__install -p -m 644 ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/torrc.sample ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/torrc - -# Install the logrotate control file. -%__mkdir_p -m 755 ${RPM_BUILD_ROOT}%{_sysconfdir}/logrotate.d -%__install -p -m 644 contrib/tor.logrotate ${RPM_BUILD_ROOT}%{_sysconfdir}/logrotate.d/%{name} - -# Directories that don't have any preinstalled files -%__mkdir_p -m 700 ${RPM_BUILD_ROOT}%{_localstatedir}/lib/%{name} -%__mkdir_p -m 755 ${RPM_BUILD_ROOT}%{_localstatedir}/run/%{name} -%__mkdir_p -m 755 ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name} -%__mkdir_p -m 700 ${RPM_BUILD_ROOT}%{_localstatedir}/tmp/%{name} - -%clean -[ "${RPM_BUILD_ROOT}" != "/" ] && rm -rf ${RPM_BUILD_ROOT} - -# These scripts are probably wrong for Mandrake or SuSE. They're certainly -# wrong for Debian, but what are you doing using RPM on Debian? - -%pre - -# If tor is already installed and running (whether installed by RPM -# or not), then kill it, but remember that it was running. -%__rm -f /%{_localstatedir}/tmp/${name}-was-running-%{version}-%{release} -if [ -f %{_initrddir}/%{name} ] && /sbin/service %{name} status ; then - /sbin/service %{name} stop - touch /%{_localstatedir}/tmp/${name}-was-running-%{version}-%{release} -fi - -# -# Create a user and group if need be -# -if [ ! -n "`/usr/bin/id -g %{torgroup} 2>/dev/null`" ]; then - # One would like to default the GID, but doing that properly would - # require thought. - %{_sbindir}/groupadd %{torgroup} 2> /dev/null -fi -if [ ! -n "`/usr/bin/id -u %{toruser} 2>/dev/null`" ]; then - # One would also like to default the UID, but doing that properly would - # also require thought. - if [ -x %{_sbindir}/nologin ]; then - %{_sbindir}/useradd -r -g %{torgroup} -d %{_localstatedir}/lib/%{name} -s %{_sbindir}/nologin %{toruser} 2> /dev/null - else - %{_sbindir}/useradd -r -g %{torgroup} -d %{_localstatedir}/lib/%{name} -s /bin/false %{toruser} 2> /dev/null - fi -fi -exit 0 - -%post - -# If this is a new installation, use chkconfig to put tor in the -# default set of runlevels. If it's an upgrade, leave the existing -# configuration alone. -if [ $1 -eq 1 ]; then - /sbin/chkconfig --add %{name} - /sbin/chkconfig %{name} on -fi - -# Older tor RPMS used a different username for the tor daemon. -# Make sure the runtime data have the right ownership. -%__chown -R %{toruser}.%{torgroup} %{_localstatedir}/{lib,log,run}/%{name} - -if [ -f /%{_localstatedir}/tmp/${name}-was-running-%{version}-%{release} ]; then - /sbin/service %{name} start - %__rm -f /%{_localstatedir}/tmp/${name}-was-running-%{version}-%{release} -fi -exit 0 - -%preun - -# If no instances of tor will be installed when we're done, make -# sure that it gets killed. We *don't* want to kill it or delete -# any of its data on uninstall if it's being upgraded to a new -# version, because the new version will actually already have -# been installed and started before the uninstall script for -# the old version is run, and we'd end up hosing it. -if [ $1 -le 0 ]; then - if [ -f %{_initrddir}/%{name} ] && /sbin/service %{name} status ; then - /sbin/service %{name} stop - fi - %/sbin/chkconfig --del %{name} - %__rm -f ${_localstatedir}/lib/%{name}/cached-directory - %__rm -f ${_localstatedir}/lib/%{name}/bw_accounting - %__rm -f ${_localstatedir}/lib/%{name}/control_auth_cookie - %__rm -f ${_localstatedir}/lib/%{name}/router.desc - %__rm -f ${_localstatedir}/lib/%{name}/fingerprint -fi -exit 0 - -%files -%defattr(-,root,root) -%if %{is_suse} -%doc INSTALL LICENSE README ChangeLog doc/HACKING doc/TODO doc/*html -%endif -%doc INSTALL LICENSE README ChangeLog doc/HACKING doc/TODO -%{_mandir}/man*/* -%{_bindir}/tor -%{_bindir}/torctl -%{_bindir}/torify -%{_bindir}/tor-resolve -%{_bindir}/tor-gencert -%if %{is_suse} -%else -%{_docdir}/* -%endif -%{_datadir}/tor/geoip -%config %{_initrddir}/%{name} -%config(noreplace) %attr(0644,root,root) %{_sysconfdir}/logrotate.d/%{name} -%dir %attr(0755,root,%{torgroup}) %{_sysconfdir}/%{name}/ -%config(noreplace) %attr(0644,root,%{torgroup}) %{_sysconfdir}/%{name}/* -%attr(0700,%{toruser},%{torgroup}) %dir %{_localstatedir}/lib/%{name} -%attr(0750,%{toruser},%{torgroup}) %dir %{_localstatedir}/run/%{name} -%attr(0750,%{toruser},%{torgroup}) %dir %{_localstatedir}/log/%{name} - -%changelog -* Thu Jun 21 2012 Ondrej Mikle <ondrej.mikle@gmail.com> -- fixed to work with both rpmbuild and mock on RHEL/Fedora -- removed unnecessary files from rpm such as .git repo -- fixed build dependencies and package conflicts -- fixed creating _tor user on Fedora 17 (ancient typo) -- added/updated build instructions for RPM creation -- confirmed to build and run on EL5, EL6, Fedora 16/17, OpenSuse 12.1 - -* Fri Aug 20 2010 Erinn Clark <erinn@torproject.org> -- add conflicts for Fedora packages -- add logic for SuSE since it requires special doc handling - -* Mon Feb 22 2010 Erinn Clark <erinn@torproject.org> -- remove AUTHORS from %doc line since it no longer exists upstream -- switch maintainers - -* Fri May 01 2009 Andrew Lewman <andrew@torproject.org> -- clean up distro detection and remove dead comment blocks - -* Sun Feb 22 2009 Andrew Lewman <andrew@torproject.org> -- update the description, vendor, and packager - -* Thu Sep 11 2008 Andrew Lewman <phobos@rootme.org> -- See r16867 -- http://archives.seul.org/or/cvs/Sep-2008/msg00156.html - -* Tue Feb 27 2007 Andrew Lewman <phobos@rootme.org> -- Fix a potential race condition in how we determine the running state of tor. Found by Stefan Nordhausen. -- see OR-CVS for details - -* Fri May 26 2006 Andrew Lewman <phobos@rootme.org> -- Add in a few "SUSEisms" to make dist-rpm actually work on suse -- Turn Tor "on" via chkconfig -- Update -mcpu to -mtune to make GCC happy -- see OR-CVS for details - -* Tue Mar 28 2006 Andrew Lewman <phobos@rootme.org> -- converted to build the specified target cpu and arch -- override related rpm macros to build correctly -- see OR-CVS for details - -* Mon Jan 17 2005 John Bashinski <jbash@velvet.com> -- Take runtime user and group names from configure system. Default - user/group names are now "_tor"; blame Roger... -- Make logrotate control file a separate file in the source distribution, - rather than creating it from the spec file. -- Properly handle the order in which RPM executes scriptlets on upgrade. - The old code would kill the daemon on upgrade. -- Start the tor daemon after installation if and only if it was - running before installation. Preserve runlevel setup on upgrade. -- Package the torctl script; the init script is now a wrapper around it. - -* Tue Nov 5 2004 John Bashinski <jbash@velvet.com> -- Add skeletal support for multiple distributions -- Even more ridiculous level of macro-ization -- Modify version numbers so RPM can determine when it has a newer version -- Return to including distribution name in package release number -- Sharply trim description -- Change user/group name from "tor" to "tordmn"; "tor" is a common - given name (reported by Marius Hjelle) -- Change group to "System Environment/Daemons" (suggested by Marius Hjelle) -- Create logrotate file (suggested by Marius Hjelle) -- Make Tor run as a user proxy by default (suggested by Marius Hjelle) -- Autogenerate spec file from GNU autotools data, substituting version - and whatnot -- Be perhaps excessively paranoid with config file and directory modes -- Remove auto-start and auto-stop at installation time; there's some kind - of weird race going on, and it's arguably a bad thing anyway. - -* Mon Jun 06 2004 Nick Mathewson <nickm@freehaven.net> 0.0.7-0.std.0.1.rc2 -- Make spec file more happy with fc2 packaging - -* Sat Jan 17 2004 John Bashinski <jbash@velvet.com> -- Basic spec file; tested with Red Hat 9. |