]>
Commit | Line | Data |
---|---|---|
b373e4d2 SG |
1 | Git Wire Protocol, Version 2 |
2 | ============================ | |
ed10cb95 BW |
3 | |
4 | This document presents a specification for a version 2 of Git's wire | |
5 | protocol. Protocol v2 will improve upon v1 in the following ways: | |
6 | ||
7 | * Instead of multiple service names, multiple commands will be | |
8 | supported by a single service | |
9 | * Easily extendable as capabilities are moved into their own section | |
10 | of the protocol, no longer being hidden behind a NUL byte and | |
11 | limited by the size of a pkt-line | |
12 | * Separate out other information hidden behind NUL bytes (e.g. agent | |
13 | string as a capability and symrefs can be requested using 'ls-refs') | |
14 | * Reference advertisement will be omitted unless explicitly requested | |
15 | * ls-refs command to explicitly request some refs | |
16 | * Designed with http and stateless-rpc in mind. With clear flush | |
17 | semantics the http remote helper can simply act as a proxy | |
18 | ||
19 | In protocol v2 communication is command oriented. When first contacting a | |
20 | server a list of capabilities will advertised. Some of these capabilities | |
21 | will be commands which a client can request be executed. Once a command | |
22 | has completed, a client can reuse the connection and request that other | |
23 | commands be executed. | |
24 | ||
b373e4d2 SG |
25 | Packet-Line Framing |
26 | ------------------- | |
ed10cb95 BW |
27 | |
28 | All communication is done using packet-line framing, just as in v1. See | |
29 | `Documentation/technical/pack-protocol.txt` and | |
30 | `Documentation/technical/protocol-common.txt` for more information. | |
31 | ||
32 | In protocol v2 these special packets will have the following semantics: | |
33 | ||
34 | * '0000' Flush Packet (flush-pkt) - indicates the end of a message | |
35 | * '0001' Delimiter Packet (delim-pkt) - separates sections of a message | |
b0df0c16 DL |
36 | * '0002' Message Packet (response-end-pkt) - indicates the end of a response |
37 | for stateless connections | |
ed10cb95 | 38 | |
b373e4d2 SG |
39 | Initial Client Request |
40 | ---------------------- | |
ed10cb95 BW |
41 | |
42 | In general a client can request to speak protocol v2 by sending | |
43 | `version=2` through the respective side-channel for the transport being | |
44 | used which inevitably sets `GIT_PROTOCOL`. More information can be | |
45 | found in `pack-protocol.txt` and `http-protocol.txt`. In all cases the | |
46 | response from the server is the capability advertisement. | |
47 | ||
b373e4d2 SG |
48 | Git Transport |
49 | ~~~~~~~~~~~~~ | |
ed10cb95 BW |
50 | |
51 | When using the git:// transport, you can request to use protocol v2 by | |
52 | sending "version=2" as an extra parameter: | |
53 | ||
54 | 003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0 | |
55 | ||
b373e4d2 SG |
56 | SSH and File Transport |
57 | ~~~~~~~~~~~~~~~~~~~~~~ | |
ed10cb95 BW |
58 | |
59 | When using either the ssh:// or file:// transport, the GIT_PROTOCOL | |
60 | environment variable must be set explicitly to include "version=2". | |
61 | ||
b373e4d2 SG |
62 | HTTP Transport |
63 | ~~~~~~~~~~~~~~ | |
ed10cb95 BW |
64 | |
65 | When using the http:// or https:// transport a client makes a "smart" | |
66 | info/refs request as described in `http-protocol.txt` and requests that | |
67 | v2 be used by supplying "version=2" in the `Git-Protocol` header. | |
68 | ||
ed10cb95 | 69 | C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 |
bb4d000e | 70 | C: Git-Protocol: version=2 |
ed10cb95 BW |
71 | |
72 | A v2 server would reply: | |
73 | ||
74 | S: 200 OK | |
75 | S: <Some headers> | |
76 | S: ... | |
77 | S: | |
78 | S: 000eversion 2\n | |
79 | S: <capability-advertisement> | |
80 | ||
81 | Subsequent requests are then made directly to the service | |
82 | `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack). | |
83 | ||
b373e4d2 SG |
84 | Capability Advertisement |
85 | ------------------------ | |
ed10cb95 BW |
86 | |
87 | A server which decides to communicate (based on a request from a client) | |
88 | using protocol version 2, notifies the client by sending a version string | |
89 | in its initial response followed by an advertisement of its capabilities. | |
90 | Each capability is a key with an optional value. Clients must ignore all | |
91 | unknown keys. Semantics of unknown values are left to the definition of | |
92 | each key. Some capabilities will describe commands which can be requested | |
93 | to be executed by the client. | |
94 | ||
95 | capability-advertisement = protocol-version | |
96 | capability-list | |
97 | flush-pkt | |
98 | ||
99 | protocol-version = PKT-LINE("version 2" LF) | |
100 | capability-list = *capability | |
101 | capability = PKT-LINE(key[=value] LF) | |
102 | ||
103 | key = 1*(ALPHA | DIGIT | "-_") | |
104 | value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;") | |
105 | ||
b373e4d2 SG |
106 | Command Request |
107 | --------------- | |
ed10cb95 BW |
108 | |
109 | After receiving the capability advertisement, a client can then issue a | |
110 | request to select the command it wants with any particular capabilities | |
111 | or arguments. There is then an optional section where the client can | |
112 | provide any command specific parameters or queries. Only a single | |
113 | command can be requested at a time. | |
114 | ||
115 | request = empty-request | command-request | |
116 | empty-request = flush-pkt | |
117 | command-request = command | |
118 | capability-list | |
119 | [command-args] | |
120 | flush-pkt | |
121 | command = PKT-LINE("command=" key LF) | |
122 | command-args = delim-pkt | |
123 | *command-specific-arg | |
124 | ||
125 | command-specific-args are packet line framed arguments defined by | |
126 | each individual command. | |
127 | ||
128 | The server will then check to ensure that the client's request is | |
129 | comprised of a valid command as well as valid capabilities which were | |
130 | advertised. If the request is valid the server will then execute the | |
131 | command. A server MUST wait till it has received the client's entire | |
132 | request before issuing a response. The format of the response is | |
133 | determined by the command being executed, but in all cases a flush-pkt | |
134 | indicates the end of the response. | |
135 | ||
136 | When a command has finished, and the client has received the entire | |
137 | response from the server, a client can either request that another | |
138 | command be executed or can terminate the connection. A client may | |
139 | optionally send an empty request consisting of just a flush-pkt to | |
140 | indicate that no more requests will be made. | |
141 | ||
b373e4d2 SG |
142 | Capabilities |
143 | ------------ | |
ed10cb95 BW |
144 | |
145 | There are two different types of capabilities: normal capabilities, | |
24966cd9 | 146 | which can be used to convey information or alter the behavior of a |
ed10cb95 BW |
147 | request, and commands, which are the core actions that a client wants to |
148 | perform (fetch, push, etc). | |
149 | ||
150 | Protocol version 2 is stateless by default. This means that all commands | |
151 | must only last a single round and be stateless from the perspective of the | |
152 | server side, unless the client has requested a capability indicating that | |
153 | state should be maintained by the server. Clients MUST NOT require state | |
154 | management on the server side in order to function correctly. This | |
155 | permits simple round-robin load-balancing on the server side, without | |
156 | needing to worry about state management. | |
157 | ||
b373e4d2 SG |
158 | agent |
159 | ~~~~~ | |
ed10cb95 BW |
160 | |
161 | The server can advertise the `agent` capability with a value `X` (in the | |
162 | form `agent=X`) to notify the client that the server is running version | |
163 | `X`. The client may optionally send its own agent string by including | |
164 | the `agent` capability with a value `Y` (in the form `agent=Y`) in its | |
165 | request to the server (but it MUST NOT do so if the server did not | |
166 | advertise the agent capability). The `X` and `Y` strings may contain any | |
167 | printable ASCII characters except space (i.e., the byte range 32 < x < | |
168 | 127), and are typically of the form "package/version" (e.g., | |
169 | "git/1.8.3.1"). The agent strings are purely informative for statistics | |
170 | and debugging purposes, and MUST NOT be used to programmatically assume | |
171 | the presence or absence of particular features. | |
72d0ea00 | 172 | |
b373e4d2 SG |
173 | ls-refs |
174 | ~~~~~~~ | |
72d0ea00 BW |
175 | |
176 | `ls-refs` is the command used to request a reference advertisement in v2. | |
177 | Unlike the current reference advertisement, ls-refs takes in arguments | |
178 | which can be used to limit the refs sent from the server. | |
179 | ||
180 | Additional features not supported in the base command will be advertised | |
181 | as the value of the command in the capability advertisement in the form | |
182 | of a space separated list of features: "<command>=<feature 1> <feature 2>" | |
183 | ||
184 | ls-refs takes in the following arguments: | |
185 | ||
186 | symrefs | |
187 | In addition to the object pointed by it, show the underlying ref | |
188 | pointed by it when showing a symbolic ref. | |
189 | peel | |
190 | Show peeled tags. | |
191 | ref-prefix <prefix> | |
192 | When specified, only references having a prefix matching one of | |
193 | the provided prefixes are displayed. | |
194 | ||
195 | The output of ls-refs is as follows: | |
196 | ||
197 | output = *ref | |
198 | flush-pkt | |
199 | ref = PKT-LINE(obj-id SP refname *(SP ref-attribute) LF) | |
200 | ref-attribute = (symref | peeled) | |
201 | symref = "symref-target:" symref-target | |
202 | peeled = "peeled:" obj-id | |
3145ea95 | 203 | |
b373e4d2 SG |
204 | fetch |
205 | ~~~~~ | |
3145ea95 BW |
206 | |
207 | `fetch` is the command used to fetch a packfile in v2. It can be looked | |
208 | at as a modified version of the v1 fetch where the ref-advertisement is | |
209 | stripped out (since the `ls-refs` command fills that role) and the | |
210 | message format is tweaked to eliminate redundancies and permit easy | |
211 | addition of future extensions. | |
212 | ||
213 | Additional features not supported in the base command will be advertised | |
214 | as the value of the command in the capability advertisement in the form | |
215 | of a space separated list of features: "<command>=<feature 1> <feature 2>" | |
216 | ||
217 | A `fetch` request can take the following arguments: | |
218 | ||
219 | want <oid> | |
220 | Indicates to the server an object which the client wants to | |
221 | retrieve. Wants can be anything and are not limited to | |
222 | advertised objects. | |
223 | ||
224 | have <oid> | |
225 | Indicates to the server an object which the client has locally. | |
226 | This allows the server to make a packfile which only contains | |
227 | the objects that the client needs. Multiple 'have' lines can be | |
228 | supplied. | |
229 | ||
230 | done | |
231 | Indicates to the server that negotiation should terminate (or | |
232 | not even begin if performing a clone) and that the server should | |
233 | use the information supplied in the request to construct the | |
234 | packfile. | |
235 | ||
236 | thin-pack | |
237 | Request that a thin pack be sent, which is a pack with deltas | |
238 | which reference base objects not contained within the pack (but | |
239 | are known to exist at the receiving end). This can reduce the | |
240 | network traffic significantly, but it requires the receiving end | |
241 | to know how to "thicken" these packs by adding the missing bases | |
242 | to the pack. | |
243 | ||
244 | no-progress | |
245 | Request that progress information that would normally be sent on | |
246 | side-band channel 2, during the packfile transfer, should not be | |
247 | sent. However, the side-band channel 3 is still used for error | |
248 | responses. | |
249 | ||
250 | include-tag | |
251 | Request that annotated tags should be sent if the objects they | |
252 | point to are being sent. | |
253 | ||
254 | ofs-delta | |
255 | Indicate that the client understands PACKv2 with delta referring | |
256 | to its base by position in pack rather than by an oid. That is, | |
031fd4b9 | 257 | they can read OBJ_OFS_DELTA (aka type 6) in a packfile. |
3145ea95 | 258 | |
f7e20501 BW |
259 | If the 'shallow' feature is advertised the following arguments can be |
260 | included in the clients request as well as the potential addition of the | |
261 | 'shallow-info' section in the server's response as explained below. | |
262 | ||
685fbd32 BW |
263 | shallow <oid> |
264 | A client must notify the server of all commits for which it only | |
265 | has shallow copies (meaning that it doesn't have the parents of | |
266 | a commit) by supplying a 'shallow <oid>' line for each such | |
267 | object so that the server is aware of the limitations of the | |
268 | client's history. This is so that the server is aware that the | |
269 | client may not have all objects reachable from such commits. | |
270 | ||
271 | deepen <depth> | |
272 | Requests that the fetch/clone should be shallow having a commit | |
273 | depth of <depth> relative to the remote side. | |
274 | ||
275 | deepen-relative | |
276 | Requests that the semantics of the "deepen" command be changed | |
277 | to indicate that the depth requested is relative to the client's | |
278 | current shallow boundary, instead of relative to the requested | |
279 | commits. | |
280 | ||
281 | deepen-since <timestamp> | |
282 | Requests that the shallow clone/fetch should be cut at a | |
283 | specific time, instead of depth. Internally it's equivalent to | |
284 | doing "git rev-list --max-age=<timestamp>". Cannot be used with | |
285 | "deepen". | |
286 | ||
287 | deepen-not <rev> | |
288 | Requests that the shallow clone/fetch should be cut at a | |
289 | specific revision specified by '<rev>', instead of a depth. | |
290 | Internally it's equivalent of doing "git rev-list --not <rev>". | |
291 | Cannot be used with "deepen", but can be used with | |
292 | "deepen-since". | |
293 | ||
ba95710a JT |
294 | If the 'filter' feature is advertised, the following argument can be |
295 | included in the client's request: | |
296 | ||
297 | filter <filter-spec> | |
298 | Request that various objects from the packfile be omitted | |
299 | using one of several filtering techniques. These are intended | |
300 | for use with partial clone and partial fetch operations. See | |
87c2d9d3 JS |
301 | `rev-list` for possible "filter-spec" values. When communicating |
302 | with other processes, senders SHOULD translate scaled integers | |
303 | (e.g. "1k") into a fully-expanded form (e.g. "1024") to aid | |
304 | interoperability with older receivers that may not understand | |
305 | newly-invented scaling suffixes. However, receivers SHOULD | |
306 | accept the following suffixes: 'k', 'm', and 'g' for 1024, | |
307 | 1048576, and 1073741824, respectively. | |
ba95710a | 308 | |
516e2b76 BW |
309 | If the 'ref-in-want' feature is advertised, the following argument can |
310 | be included in the client's request as well as the potential addition of | |
311 | the 'wanted-refs' section in the server's response as explained below. | |
312 | ||
313 | want-ref <ref> | |
314 | Indicates to the server that the client wants to retrieve a | |
315 | particular ref, where <ref> is the full name of a ref on the | |
316 | server. | |
317 | ||
0bbc0bc5 JT |
318 | If the 'sideband-all' feature is advertised, the following argument can be |
319 | included in the client's request: | |
320 | ||
321 | sideband-all | |
322 | Instruct the server to send the whole response multiplexed, not just | |
323 | the packfile section. All non-flush and non-delim PKT-LINE in the | |
324 | response (not only in the packfile section) will then start with a byte | |
325 | indicating its sideband (1, 2, or 3), and the server may send "0005\2" | |
326 | (a PKT-LINE of sideband 2 with no payload) as a keepalive packet. | |
327 | ||
cd8402e0 JT |
328 | If the 'packfile-uris' feature is advertised, the following argument |
329 | can be included in the client's request as well as the potential | |
330 | addition of the 'packfile-uris' section in the server's response as | |
331 | explained below. | |
332 | ||
333 | packfile-uris <comma-separated list of protocols> | |
334 | Indicates to the server that the client is willing to receive | |
335 | URIs of any of the given protocols in place of objects in the | |
336 | sent packfile. Before performing the connectivity check, the | |
337 | client should download from all given URIs. Currently, the | |
338 | protocols supported are "http" and "https". | |
339 | ||
3145ea95 BW |
340 | The response of `fetch` is broken into a number of sections separated by |
341 | delimiter packets (0001), with each section beginning with its section | |
fd194dd5 | 342 | header. Most sections are sent only when the packfile is sent. |
3145ea95 | 343 | |
fd194dd5 JT |
344 | output = acknowledgements flush-pkt | |
345 | [acknowledgments delim-pkt] [shallow-info delim-pkt] | |
cd8402e0 JT |
346 | [wanted-refs delim-pkt] [packfile-uris delim-pkt] |
347 | packfile flush-pkt | |
3145ea95 BW |
348 | |
349 | acknowledgments = PKT-LINE("acknowledgments" LF) | |
350 | (nak | *ack) | |
351 | (ready) | |
352 | ready = PKT-LINE("ready" LF) | |
353 | nak = PKT-LINE("NAK" LF) | |
354 | ack = PKT-LINE("ACK" SP obj-id LF) | |
355 | ||
685fbd32 BW |
356 | shallow-info = PKT-LINE("shallow-info" LF) |
357 | *PKT-LINE((shallow | unshallow) LF) | |
358 | shallow = "shallow" SP obj-id | |
359 | unshallow = "unshallow" SP obj-id | |
360 | ||
516e2b76 BW |
361 | wanted-refs = PKT-LINE("wanted-refs" LF) |
362 | *PKT-LINE(wanted-ref LF) | |
363 | wanted-ref = obj-id SP refname | |
364 | ||
cd8402e0 JT |
365 | packfile-uris = PKT-LINE("packfile-uris" LF) *packfile-uri |
366 | packfile-uri = PKT-LINE(40*(HEXDIGIT) SP *%x20-ff LF) | |
367 | ||
3145ea95 BW |
368 | packfile = PKT-LINE("packfile" LF) |
369 | *PKT-LINE(%x01-03 *%x00-ff) | |
370 | ||
371 | acknowledgments section | |
fd194dd5 JT |
372 | * If the client determines that it is finished with negotiations by |
373 | sending a "done" line (thus requiring the server to send a packfile), | |
374 | the acknowledgments sections MUST be omitted from the server's | |
375 | response. | |
3145ea95 BW |
376 | |
377 | * Always begins with the section header "acknowledgments" | |
378 | ||
379 | * The server will respond with "NAK" if none of the object ids sent | |
380 | as have lines were common. | |
381 | ||
382 | * The server will respond with "ACK obj-id" for all of the | |
383 | object ids sent as have lines which are common. | |
384 | ||
385 | * A response cannot have both "ACK" lines as well as a "NAK" | |
386 | line. | |
387 | ||
388 | * The server will respond with a "ready" line indicating that | |
389 | the server has found an acceptable common base and is ready to | |
390 | make and send a packfile (which will be found in the packfile | |
391 | section of the same response) | |
392 | ||
393 | * If the server has found a suitable cut point and has decided | |
394 | to send a "ready" line, then the server can decide to (as an | |
395 | optimization) omit any "ACK" lines it would have sent during | |
396 | its response. This is because the server will have already | |
397 | determined the objects it plans to send to the client and no | |
398 | further negotiation is needed. | |
399 | ||
685fbd32 | 400 | shallow-info section |
f7e20501 BW |
401 | * If the client has requested a shallow fetch/clone, a shallow |
402 | client requests a fetch or the server is shallow then the | |
403 | server's response may include a shallow-info section. The | |
404 | shallow-info section will be included if (due to one of the | |
405 | above conditions) the server needs to inform the client of any | |
406 | shallow boundaries or adjustments to the clients already | |
407 | existing shallow boundaries. | |
685fbd32 BW |
408 | |
409 | * Always begins with the section header "shallow-info" | |
410 | ||
411 | * If a positive depth is requested, the server will compute the | |
412 | set of commits which are no deeper than the desired depth. | |
413 | ||
414 | * The server sends a "shallow obj-id" line for each commit whose | |
415 | parents will not be sent in the following packfile. | |
416 | ||
417 | * The server sends an "unshallow obj-id" line for each commit | |
418 | which the client has indicated is shallow, but is no longer | |
419 | shallow as a result of the fetch (due to its parents being | |
420 | sent in the following packfile). | |
421 | ||
422 | * The server MUST NOT send any "unshallow" lines for anything | |
423 | which the client has not indicated was shallow as a part of | |
424 | its request. | |
425 | ||
516e2b76 BW |
426 | wanted-refs section |
427 | * This section is only included if the client has requested a | |
428 | ref using a 'want-ref' line and if a packfile section is also | |
429 | included in the response. | |
430 | ||
431 | * Always begins with the section header "wanted-refs". | |
432 | ||
433 | * The server will send a ref listing ("<oid> <refname>") for | |
434 | each reference requested using 'want-ref' lines. | |
435 | ||
436 | * The server MUST NOT send any refs which were not requested | |
437 | using 'want-ref' lines. | |
438 | ||
cd8402e0 JT |
439 | packfile-uris section |
440 | * This section is only included if the client sent | |
441 | 'packfile-uris' and the server has at least one such URI to | |
442 | send. | |
443 | ||
444 | * Always begins with the section header "packfile-uris". | |
445 | ||
446 | * For each URI the server sends, it sends a hash of the pack's | |
447 | contents (as output by git index-pack) followed by the URI. | |
448 | ||
449 | * The hashes are 40 hex characters long. When Git upgrades to a new | |
450 | hash algorithm, this might need to be updated. (It should match | |
451 | whatever index-pack outputs after "pack\t" or "keep\t". | |
452 | ||
3145ea95 BW |
453 | packfile section |
454 | * This section is only included if the client has sent 'want' | |
455 | lines in its request and either requested that no more | |
456 | negotiation be done by sending 'done' or if the server has | |
457 | decided it has found a sufficient cut point to produce a | |
458 | packfile. | |
459 | ||
460 | * Always begins with the section header "packfile" | |
461 | ||
462 | * The transmission of the packfile begins immediately after the | |
463 | section header | |
464 | ||
465 | * The data transfer of the packfile is always multiplexed, using | |
466 | the same semantics of the 'side-band-64k' capability from | |
467 | protocol version 1. This means that each packet, during the | |
468 | packfile data stream, is made up of a leading 4-byte pkt-line | |
469 | length (typical of the pkt-line format), followed by a 1-byte | |
470 | stream code, followed by the actual data. | |
471 | ||
472 | The stream code can be one of: | |
473 | 1 - pack data | |
474 | 2 - progress messages | |
475 | 3 - fatal error message just before stream aborts | |
ecc3e534 | 476 | |
b373e4d2 SG |
477 | server-option |
478 | ~~~~~~~~~~~~~ | |
ecc3e534 BW |
479 | |
480 | If advertised, indicates that any number of server specific options can be | |
481 | included in a request. This is done by sending each option as a | |
482 | "server-option=<option>" capability line in the capability-list section of | |
483 | a request. | |
484 | ||
485 | The provided options must not contain a NUL or LF character. | |
7f46e7ea | 486 | |
487 | object-format | |
488 | ~~~~~~~~~~~~~~~ | |
489 | ||
490 | The server can advertise the `object-format` capability with a value `X` (in the | |
491 | form `object-format=X`) to notify the client that the server is able to deal | |
492 | with objects using hash algorithm X. If not specified, the server is assumed to | |
493 | only handle SHA-1. If the client would like to use a hash algorithm other than | |
494 | SHA-1, it should specify its object-format string. |