Clean the contrib directory with torch and machete.

We've accumulated a lot of cruft in this directory over the years: so
much, that it passed the point of being so disorganized that we no
longer browsed through it to see how bad it had gotten.

This patch (based on changes by rl1987) tries to remove the most
useless items, and split the others into reasonable directories.  It
creates a new scripts/ directory for maint and test scripts.

This patch was generated with the script below.  No other changes are made in
this patch.

#############
# new directories
mkdir -p contrib/test-tools
mkdir -p contrib/or-tools
mkdir -p contrib/dirauth-tools
mkdir -p contrib/operator-tools
mkdir -p contrib/client-tools
mkdir -p contrib/test-tools
mkdir -p contrib/dist
mkdir -p contrib/dist/suse
mkdir -p contrib/win32build

mkdir -p scripts/maint
mkdir -p scripts/test

############
# Deleted -- nobody who wants this is going to be looking for it here any
# longer.  Also, nobody wants it.
git rm contrib/auto-naming/README

# Deleted: We no longer do polipo.
git rm contrib/polipo/Makefile.mingw
git rm contrib/polipo/README
git rm contrib/polipo/polipo-mingw.nsi

# We haven't even tried to run this for ages. It is a relic of a bygone era
git rm contrib/mdd.py

# contrib/dir-tools/directory-archive/
# Tools for running a directory archive. No longer used - deleting them.
git rm contrib/directory-archive/crontab.sample
git rm contrib/directory-archive/fetch-all
git rm contrib/directory-archive/fetch-all-v3
git rm contrib/directory-archive/tar-them-up
git rm contrib/directory-archive/fetch-all-functions
git rm contrib/directory-archive/sort-into-month-folder

# This appears to be related to very old windows packaging stuff.
git rm contrib/bundle.nsi
git rm contrib/package_nsis-weasel.sh
git rm contrib/package_nsis.sh
git rm contrib/netinst.nsi
git rm contrib/torinst32.ico
git rm contrib/xenobite.ico

# This should not be needed for cross-compilation any more, should it?
git rm contrib/cross.sh

# I don't think anyone ever used this.
git rm contrib/make-signature.sh

# These are attempts to send tor controller commands from the command-line.
# They don't support modern authentication.
git rm contrib/tor-ctrl.sh

# this is for fetching about a tor server from a dirauth. But it
# doesn't authenticate the dirauth: yuck.
git rm contrib/sd

# wow, such unused, very perl4.
git rm contrib/tor-stress

####### contrib/dirauth-tools/
# Tools for running a directory authority

git mv contrib/add-tor contrib/dirauth-tools/
git mv contrib/nagios-check-tor-authority-cert contrib/dirauth-tools/

#######
# contrib/or-tools/
# Tools for examining relays
git mv contrib/check-tor contrib/or-tools/check-tor
git mv contrib/checksocks.pl contrib/or-tools/checksocks.pl
git mv contrib/exitlist contrib/or-tools/exitlist

#######
# contrib/operator-tools

# Tools for running a relay.
git mv contrib/linux-tor-prio.sh contrib/operator-tools/linux-tor-prio.sh
git mv contrib/tor-exit-notice.html contrib/operator-tools/tor-exit-notice.html
git mv contrib/tor.logrotate.in contrib/operator-tools/

######
# contrib/dist

git mv contrib/rc.subr contrib/dist/
git mv contrib/tor.sh.in contrib/dist/
git mv contrib/torctl.in contrib/dist/
git mv contrib/suse/* contrib/dist/suse/

######
# client-tools
git mv contrib/torify contrib/client-tools/torify
git mv contrib/tor-resolve.py contrib/client-tools/

######
# win32build

git mv contrib/package_nsis-mingw.sh contrib/win32build/
git mv contrib/tor.nsi.in contrib/win32build/
# Erinn didn't ask for this...
git mv contrib/tor-mingw.nsi.in contrib/win32build/
git mv contrib/tor.ico contrib/win32build/

######
# scripts/test
git mv contrib/cov-blame scripts/test/cov-blame
git mv contrib/cov-diff scripts/test/cov-diff
git mv contrib/coverage scripts/test/coverage
git mv contrib/scan-build.sh scripts/test/

######## scripts/maint
# Maintainance scripts
#
# These are scripts for developers to use when hacking on Tor.  They mostly
# look at the Tor source in one way or another.
git mv contrib/findMergedChanges.pl scripts/maint/findMergedChanges.pl
git mv contrib/checkOptionDocs.pl scripts/maint/checkOptionDocs.pl
git mv contrib/checkSpace.pl scripts/maint/checkSpace.pl
git mv contrib/redox.py scripts/maint/redox.py
git mv contrib/updateVersions.pl scripts/maint/updateVersions.pl
git mv contrib/checkLogs.pl scripts/maint/checkLogs.pl
git mv contrib/format_changelog.py scripts/maint/

3 files changed, 350 insertions, 0 deletions

diff --git a/contrib/operator-tools/linux-tor-prio.sh b/contrib/operator-tools/linux-tor-prio.sh
new file mode 100644
index 000000000..ea9e0ddaa
--- /dev/null
+++ b/contrib/operator-tools/linux-tor-prio.sh
@@ -0,0 +1,192 @@
+#!/bin/bash
+# Written by Marco Bonetti & Mike Perry
+# Based on instructions from Dan Singletary's ADSL BW Management HOWTO:
+# http://www.faqs.org/docs/Linux-HOWTO/ADSL-Bandwidth-Management-HOWTO.html
+# This script is Public Domain.
+
+############################### README #################################
+
+# This script provides prioritization of Tor traffic below other
+# traffic on a Linux server. It has two modes of operation: UID based 
+# and IP based. 
+
+# UID BASED PRIORITIZATION
+#
+# The UID based method requires that Tor be launched from 
+# a specific user ID. The "User" Tor config setting is
+# insufficient, as it sets the UID after the socket is created.
+# Here is a C wrapper you can use to execute Tor and drop privs before 
+# it creates any sockets. 
+#
+# Compile with:
+# gcc -DUID=`id -u tor` -DGID=`id -g tor` tor_wrap.c -o tor_wrap
+#
+# #include <unistd.h>
+# int main(int argc, char **argv) {
+# if(initgroups("tor", GID) == -1) { perror("initgroups"); return 1; }
+# if(setresgid(GID, GID, GID) == -1) { perror("setresgid"); return 1; }
+# if(setresuid(UID, UID, UID) == -1) { perror("setresuid"); return 1; }
+# execl("/bin/tor", "/bin/tor", "-f", "/etc/tor/torrc", NULL);
+# perror("execl"); return 1;
+# }
+
+# IP BASED PRIORITIZATION
+#
+# The IP setting requires that a separate IP address be dedicated to Tor. 
+# Your Torrc should be set to bind to this IP for "OutboundBindAddress", 
+# "ListenAddress", and "Address".
+
+# GENERAL USAGE
+#
+# You should also tune the individual connection rate parameters below
+# to your individual connection. In particular, you should leave *some* 
+# minimum amount of bandwidth for Tor, so that Tor users are not 
+# completely choked out when you use your server's bandwidth. 30% is 
+# probably a reasonable choice. More is better of course.
+#
+# To start the shaping, run it as: 
+#   ./linux-tor-prio.sh 
+#
+# To get status information (useful to verify packets are getting marked
+# and prioritized), run:
+#   ./linux-tor-prio.sh status
+#
+# And to stop prioritization:
+#   ./linux-tor-prio.sh stop
+#
+########################################################################
+
+# BEGIN USER TUNABLE PARAMETERS
+
+DEV=eth0
+
+# NOTE! You must START Tor under this UID. Using the Tor User
+# config setting is NOT sufficient. See above.
+TOR_UID=$(id -u tor)
+
+# If the UID mechanism doesn't work for you, you can set this parameter
+# instead. If set, it will take precedence over the UID setting. Note that
+# you need multiple IPs with one specifically devoted to Tor for this to
+# work.
+#TOR_IP="42.42.42.42"
+
+# Average ping to most places on the net, milliseconds
+RTT_LATENCY=40
+
+# RATE_UP must be less than your connection's upload capacity in
+# kbits/sec. If it is larger, then the bottleneck will be at your
+# router's queue, which you do not control. This will cause congestion
+# and a revert to normal TCP fairness no matter what the queing
+# priority is.
+RATE_UP=5000
+
+# RATE_UP_TOR is the minimum speed your Tor connections will have in
+# kbits/sec.  They will have at least this much bandwidth for upload.
+# In general, you probably shouldn't set this too low, or else Tor
+# users who use your node will be completely choked out whenever your
+# machine does any other network activity. That is not very fun.
+RATE_UP_TOR=1500
+
+# RATE_UP_TOR_CEIL is the maximum rate allowed for all Tor trafic in
+# kbits/sec.
+RATE_UP_TOR_CEIL=5000
+
+CHAIN=OUTPUT
+#CHAIN=PREROUTING
+#CHAIN=POSTROUTING
+
+MTU=1500
+AVG_PKT=900 # should be more like 600 for non-exit nodes
+
+# END USER TUNABLE PARAMETERS
+
+
+
+# The queue size should be no larger than your bandwidth-delay
+# product. This is RT latency*bandwidth/MTU/2
+
+BDP=$(expr $RTT_LATENCY \* $RATE_UP / $AVG_PKT)
+
+# Further research indicates that the BDP calculations should use
+# RTT/sqrt(n) where n is the expected number of active connections..
+
+BDP=$(expr $BDP / 4)
+
+if [ "$1" = "status" ]
+then
+	echo "[qdisc]"
+	tc -s qdisc show dev $DEV
+	tc -s qdisc show dev imq0
+	echo "[class]"
+	tc -s class show dev $DEV
+	tc -s class show dev imq0
+	echo "[filter]"
+	tc -s filter show dev $DEV
+	tc -s filter show dev imq0
+	echo "[iptables]"
+	iptables -t mangle -L TORSHAPER-OUT -v -x 2> /dev/null
+	exit
+fi
+
+
+# Reset everything to a known state (cleared)
+tc qdisc del dev $DEV root 2> /dev/null > /dev/null
+tc qdisc del dev imq0 root 2> /dev/null > /dev/null
+iptables -t mangle -D POSTROUTING -o $DEV -j TORSHAPER-OUT 2> /dev/null > /dev/null
+iptables -t mangle -D PREROUTING -o $DEV -j TORSHAPER-OUT 2> /dev/null > /dev/null
+iptables -t mangle -D OUTPUT -o $DEV -j TORSHAPER-OUT 2> /dev/null > /dev/null
+iptables -t mangle -F TORSHAPER-OUT 2> /dev/null > /dev/null
+iptables -t mangle -X TORSHAPER-OUT 2> /dev/null > /dev/null
+ip link set imq0 down 2> /dev/null > /dev/null
+rmmod imq 2> /dev/null > /dev/null
+
+if [ "$1" = "stop" ]
+then
+	echo "Shaping removed on $DEV."
+	exit
+fi
+
+# Outbound Shaping (limits total bandwidth to RATE_UP)
+
+ip link set dev $DEV qlen $BDP
+
+# Add HTB root qdisc, default is high prio
+tc qdisc add dev $DEV root handle 1: htb default 20
+
+# Add main rate limit class
+tc class add dev $DEV parent 1: classid 1:1 htb rate ${RATE_UP}kbit
+
+# Create the two classes, giving Tor at least RATE_UP_TOR kbit and capping
+# total upstream at RATE_UP so the queue is under our control.
+tc class add dev $DEV parent 1:1 classid 1:20 htb rate $(expr $RATE_UP - $RATE_UP_TOR)kbit ceil ${RATE_UP}kbit prio 0
+tc class add dev $DEV parent 1:1 classid 1:21 htb rate $[$RATE_UP_TOR]kbit ceil ${RATE_UP_TOR_CEIL}kbit prio 10
+
+# Start up pfifo
+tc qdisc add dev $DEV parent 1:20 handle 20: pfifo limit $BDP
+tc qdisc add dev $DEV parent 1:21 handle 21: pfifo limit $BDP
+
+# filter traffic into classes by fwmark
+tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20
+tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 21 fw flowid 1:21
+
+# add TORSHAPER-OUT chain to the mangle table in iptables
+iptables -t mangle -N TORSHAPER-OUT
+iptables -t mangle -I $CHAIN -o $DEV -j TORSHAPER-OUT
+
+
+# Set firewall marks
+# Low priority to Tor
+if [ ""$TOR_IP == "" ]
+then
+	echo "Using UID-based QoS. UID $TOR_UID marked as low priority."
+	iptables -t mangle -A TORSHAPER-OUT -m owner --uid-owner $TOR_UID -j MARK --set-mark 21
+else
+	echo "Using IP-based QoS. $TOR_IP marked as low priority."
+	iptables -t mangle -A TORSHAPER-OUT -s $TOR_IP -j MARK --set-mark 21
+fi
+
+# High prio for everything else
+iptables -t mangle -A TORSHAPER-OUT -m mark --mark 0 -j MARK --set-mark 20
+
+echo "Outbound shaping added to $DEV.  Rate for Tor upload at least: ${RATE_UP_TOR}Kbyte/sec."
+
diff --git a/contrib/operator-tools/tor-exit-notice.html b/contrib/operator-tools/tor-exit-notice.html
new file mode 100644
index 000000000..8cf5c294f
--- /dev/null
+++ b/contrib/operator-tools/tor-exit-notice.html
@@ -0,0 +1,144 @@
+<?xml version="1.0"?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
+    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
+<title>This is a Tor Exit Router</title>
+
+<!--
+
+This notice is intended to be placed on a virtual host for a domain that
+your Tor exit node IP reverse resolves to so that people who may be about
+to file an abuse complaint would check it first before bothering you or
+your ISP. Ex:
+http://tor-exit.yourdomain.org or http://tor-readme.yourdomain.org.
+
+This type of setup has proven very effective at reducing abuse complaints
+for exit node operators.
+
+There are a few places in this document that you may want to customize.
+They are marked with FIXME.
+
+-->
+
+</head>
+<body>
+
+<p style="text-align:center; font-size:xx-large; font-weight:bold">This is a
+Tor Exit Router</p>
+
+<p>
+Most likely you are accessing this website because you had some issue with
+the traffic coming from this IP. This router is part of the <a
+href="https://www.torproject.org/">Tor Anonymity Network</a>, which is
+dedicated to <a href="https://www.torproject.org/about/overview">providing
+privacy</a> to people who need it most: average computer users. This
+router IP should be generating no other traffic, unless it has been
+compromised.</p>
+
+
+<!-- FIXME: you should probably grab your own copy of how_tor_works_thumb.png
+     and serve it locally -->
+
+<p style="text-align:center">
+<a href="https://www.torproject.org/about/overview">
+<img src="https://www.torproject.org/images/how_tor_works_thumb.png" alt="How Tor works" style="border-style:none"/>
+</a></p>
+
+<p>
+Tor sees use by <a href="https://www.torproject.org/about/torusers">many
+important segments of the population</a>, including whistle blowers,
+journalists, Chinese dissidents skirting the Great Firewall and oppressive
+censorship, abuse victims, stalker targets, the US military, and law
+enforcement, just to name a few.  While Tor is not designed for malicious
+computer users, it is true that they can use the network for malicious ends.
+In reality however, the actual amount of <a
+href="https://www.torproject.org/docs/faq-abuse">abuse</a> is quite low. This
+is largely because criminals and hackers have significantly better access to
+privacy and anonymity than do the regular users whom they prey upon. Criminals
+can and do <a
+href="http://voices.washingtonpost.com/securityfix/2008/08/web_fraud_20_tools.html">build,
+sell, and trade</a> far larger and <a
+href="http://voices.washingtonpost.com/securityfix/2008/08/web_fraud_20_distributing_your.html">more
+powerful networks</a> than Tor on a daily basis. Thus, in the mind of this
+operator, the social need for easily accessible censorship-resistant private,
+anonymous communication trumps the risk of unskilled bad actors, who are
+almost always more easily uncovered by traditional police work than by
+extensive monitoring and surveillance anyway.</p>
+
+<p>
+In terms of applicable law, the best way to understand Tor is to consider it a
+network of routers operating as common carriers, much like the Internet
+backbone. However, unlike the Internet backbone routers, Tor routers
+explicitly do not contain identifiable routing information about the source of
+a packet, and no single Tor node can determine both the origin and destination
+of a given transmission.</p>
+
+<p>
+As such, there is little the operator of this router can do to help you track
+the connection further. This router maintains no logs of any of the Tor
+traffic, so there is little that can be done to trace either legitimate or
+illegitimate traffic (or to filter one from the other).  Attempts to
+seize this router will accomplish nothing.</p>
+
+<!-- FIXME: US-Only section. Remove if you are a non-US operator -->
+
+<p>
+Furthermore, this machine also serves as a carrier of email, which means that
+its contents are further protected under the ECPA. <a
+href="http://www.law.cornell.edu/uscode/text/18/2707">18
+USC 2707</a> explicitly allows for civil remedies ($1000/account
+<i><b>plus</b></i>  legal fees)
+in the event of a seizure executed without good faith or probable cause (it
+should be clear at this point that traffic with an originating IP address of
+FIXME_DNS_NAME should not constitute probable cause to seize the
+machine). Similar considerations exist for 1st amendment content on this
+machine.</p>
+
+<!-- FIXME: May or may not be US-only. Some non-US tor nodes have in
+     fact reported DMCA harassment... -->
+
+<p>
+If you are a representative of a company who feels that this router is being
+used to violate the DMCA, please be aware that this machine does not host or
+contain any illegal content. Also be aware that network infrastructure
+maintainers are not liable for the type of content that passes over their
+equipment, in accordance with <a
+href="http://www.law.cornell.edu/uscode/text/17/512">DMCA
+"safe harbor" provisions</a>. In other words, you will have just as much luck
+sending a takedown notice to the Internet backbone providers. Please consult
+<a href="https://www.torproject.org/eff/tor-dmca-response">EFF's prepared
+response</a> for more information on this matter.</p>
+
+<p>For more information, please consult the following documentation:</p>
+
+<ol>
+<li><a href="https://www.torproject.org/about/overview">Tor Overview</a></li>
+<li><a href="https://www.torproject.org/docs/faq-abuse">Tor Abuse FAQ</a></li>
+<li><a href="https://www.torproject.org/eff/tor-legal-faq">Tor Legal FAQ</a></li>
+</ol>
+
+<p>
+That being said, if you still have a complaint about the router,  you may
+email the <a href="mailto:FIXME_YOUR_EMAIL_ADDRESS">maintainer</a>. If
+complaints are related to a particular service that is being abused, I will
+consider removing that service from my exit policy, which would prevent my
+router from allowing that traffic to exit through it. I can only do this on an
+IP+destination port basis, however. Common P2P ports are
+already blocked.</p>
+
+<p>
+You also have the option of blocking this IP address and others on
+the Tor network if you so desire. The Tor project provides a <a
+href="https://check.torproject.org/cgi-bin/TorBulkExitList.py">web service</a>
+to fetch a list of all IP addresses of Tor exit nodes that allow exiting to a
+specified IP:port combination, and an official <a
+href="https://www.torproject.org/tordnsel/dist/">DNSRBL</a> is also available to
+determine if a given IP address is actually a Tor exit server. Please
+be considerate
+when using these options. It would be unfortunate to deny all Tor users access
+to your site indefinitely simply because of a few bad apples.</p>
+
+</body>
+</html>
diff --git a/contrib/operator-tools/tor.logrotate.in b/contrib/operator-tools/tor.logrotate.in
new file mode 100644
index 000000000..6e75f80bf
--- /dev/null
+++ b/contrib/operator-tools/tor.logrotate.in
@@ -0,0 +1,14 @@
+@LOCALSTATEDIR@/log/tor/*log {
+	daily
+	rotate 5
+	compress
+	delaycompress
+	missingok
+	notifempty
+	# you may need to change the username/groupname below
+	create 0640 _tor _tor
+	sharedscripts
+	postrotate
+		/etc/init.d/tor reload > /dev/null
+	endscript
+}