1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
|
Filename: 104-short-descriptors.txt
Title: Long and Short Router Descriptors
Version: $Revision$
Last-Modified: $Date$
Author: Nick Mathewson
Created:
Status: Closed
Implemented-In: 0.2.0.x
Overview:
This document proposes moving unused-by-clients information from regular
router descriptors into a new "extra info" router descriptor.
Proposal:
Some of the costliest fields in the current directory protocol are ones
that no client actually uses. In particular, the "read-history" and
"write-history" fields are used only by the authorities for monitoring the
status of the network. If we took them out, the size of a compressed list
of all the routers would fall by about 60%. (No other disposable field
would save much more than 2%.)
We propose to remove these fields from descriptors, and and have them
uploaded as a part of a separate signed "extra info" to the authorities.
This document will be signed. A hash of this document will be included in
the regular descriptors.
(We considered another design, where routers would generate and upload a
short-form and a long-form descriptor. Only the short-form descriptor would
ever be used by anybody for routing. The long-form descriptor would be
used only for analytics and other tools. We decided against this because
well-behaved tools would need to download short-form descriptors too (as
these would be the only ones indexed), and hence get redundant info. Badly
behaved tools would download only long-form descriptors, and expose
themselves to partitioning attacks.)
Other disposable fields:
Clients don't need these fields, but removing them doesn't help bandwidth
enough to be worthwhile.
contact (save about 1%)
fingerprint (save about 3%)
We could represent these fields more succinctly, but removing them would
only save 1%. (!)
reject
accept
(Apparently, exit polices are highly compressible.)
[Does size-on-disk matter to anybody? Some clients and servers don't
have much disk, or have really slow disk (e.g. USB). And we don't
store caches compressed right now. -RD]
Specification:
1. Extra Info Format.
An "extra info" descriptor contains the following fields:
"extra-info" Nickname Fingerprint
Identifies what router this is an extra info descriptor for.
Fingerprint is encoded in hex (using upper-case letters), with
no spaces.
"published" As currently documented in dir-spec.txt. It MUST match the
"published" field of the descriptor published at the same time.
"read-history"
"write-history"
As currently documented in dir-spec.txt. Optional.
"router-signature" NL Signature NL
A signature of the PKCS1-padded hash of the entire extra info
document, taken from the beginning of the "extra-info" line, through
the newline after the "router-signature" line. An extra info
document is not valid unless the signature is performed with the
identity key whose digest matches FINGERPRINT.
The "extra-info" field is required and MUST appear first. The
router-signature field is required and MUST appear last. All others are
optional. As for other documents, unrecognized fields must be ignored.
2. Existing formats
Implementations that use "read-history" and "write-history" SHOULD
continue accepting router descriptors that contain them. (Prior to
0.2.0.x, this information was encoded in ordinary router descriptors;
in any case they have always been listed as opt, so they should be
accepted anyway.)
Add these fields to router descriptors:
"extra-info-digest" Digest
"Digest" is a hex-encoded digest (using upper-case characters)
of the router's extra-info document, as signed in the router's
extra-info. (If this field is absent, no extra-info-digest
exists.)
"caches-extra-info"
Present if this router is a directory cache that provides
extra-info documents, or an authority that handles extra-info
documents.
(Since implementations before 0.1.2.5-alpha required that the "opt"
keyword precede any unrecognized entry, these keys MUST be preceded
with "opt" until 0.1.2.5-alpha is obsolete.)
3. New communications rules
Servers SHOULD generate and upload one extra-info document after each
descriptor they generate and upload; no more, no less. Servers MUST
upload the new descriptor before they upload the new extra-info.
Authorities receiving an extra-info document SHOULD verify all of the
following:
* They have a router descriptor for some server with a matching
nickname and identity fingerprint.
* That server's identity key has been used to sign the extra-info
document.
* The extra-info-digest field in the router descriptor matches
the digest of the extra-info document.
* The published fields in the two documents match.
Authorities SHOULD drop extra-info documents that do not meet these
criteria.
Extra-info documents MAY be uploaded as part of the same HTTP post as
the router descriptor, or separately. Authorities MUST accept both
methods.
Authorities SHOULD try to fetch extra-info documents from one another if
they do not have one matching the digest declared in a router
descriptor.
Caches that are running locally with a tool that needs to use extra-info
documents MAY download and store extra-info documents. They should do
so when they notice that the recommended descriptor has an
extra-info-digest not matching any extra-info document they currently
have. (Caches not running on a host that needs to use extra-info
documents SHOULD NOT download or cache them.)
4. New URLs
http://<hostname>/tor/extra/d/...
http://<hostname>/tor/extra/fp/...
http://<hostname>/tor/extra/all[.z]
(As for /tor/server/ URLs: supports fetching extra-info documents
by their digest, by the fingerprint of their servers, or all
at once. When serving by fingerprint, we serve the extra-info
that corresponds to the descriptor we would serve by that
fingerprint. Only directory authorities are guaranteed to support
these URLs.)
http://<hostname>/tor/extra/authority[.z]
(The extra-info document for this router.)
Extra-info documents are uploaded to the same URLs as regular
router descriptors.
Migration:
For extra info approach:
* First:
* Authorities should accept extra info, and support serving it.
* Routers should upload extra info once authorities accept it.
* Caches should support an option to download and cache it, once
authorities serve it.
* Tools should be updated to use locally cached information.
These tools include:
lefkada's exit.py script.
tor26's noreply script and general directory cache.
https://nighteffect.us/tns/ for its graphs
and check with or-talk for the rest, once it's time.
* Set a cutoff time for including bandwidth in router descriptors, so
that tools that use bandwidth info know that they will need to fetch
extra info documents.
* Once tools that want bandwidth info support fetching extra info:
* Have routers stop including bandwidth info in their router
descriptors.
|