]>
Commit | Line | Data |
---|---|---|
5db92105 ÆAB |
1 | gitprotocol-v2(5) |
2 | ================= | |
3 | ||
4 | NAME | |
5 | ---- | |
6 | gitprotocol-v2 - Git Wire Protocol, Version 2 | |
7 | ||
8 | SYNOPSIS | |
9 | -------- | |
10 | [verse] | |
11 | <over-the-wire-protocol> | |
12 | ||
13 | DESCRIPTION | |
14 | ----------- | |
ed10cb95 BW |
15 | |
16 | This document presents a specification for a version 2 of Git's wire | |
17 | protocol. Protocol v2 will improve upon v1 in the following ways: | |
18 | ||
19 | * Instead of multiple service names, multiple commands will be | |
20 | supported by a single service | |
21 | * Easily extendable as capabilities are moved into their own section | |
22 | of the protocol, no longer being hidden behind a NUL byte and | |
23 | limited by the size of a pkt-line | |
24 | * Separate out other information hidden behind NUL bytes (e.g. agent | |
25 | string as a capability and symrefs can be requested using 'ls-refs') | |
26 | * Reference advertisement will be omitted unless explicitly requested | |
27 | * ls-refs command to explicitly request some refs | |
28 | * Designed with http and stateless-rpc in mind. With clear flush | |
29 | semantics the http remote helper can simply act as a proxy | |
30 | ||
31 | In protocol v2 communication is command oriented. When first contacting a | |
89363522 | 32 | server a list of capabilities will be advertised. Some of these capabilities |
ed10cb95 BW |
33 | will be commands which a client can request be executed. Once a command |
34 | has completed, a client can reuse the connection and request that other | |
35 | commands be executed. | |
36 | ||
b373e4d2 SG |
37 | Packet-Line Framing |
38 | ------------------- | |
ed10cb95 BW |
39 | |
40 | All communication is done using packet-line framing, just as in v1. See | |
5db92105 | 41 | linkgit:gitprotocol-pack[5] and linkgit:gitprotocol-common[5] for more information. |
ed10cb95 BW |
42 | |
43 | In protocol v2 these special packets will have the following semantics: | |
44 | ||
45 | * '0000' Flush Packet (flush-pkt) - indicates the end of a message | |
46 | * '0001' Delimiter Packet (delim-pkt) - separates sections of a message | |
9d336655 JS |
47 | * '0002' Response End Packet (response-end-pkt) - indicates the end of a |
48 | response for stateless connections | |
ed10cb95 | 49 | |
b373e4d2 SG |
50 | Initial Client Request |
51 | ---------------------- | |
ed10cb95 BW |
52 | |
53 | In general a client can request to speak protocol v2 by sending | |
54 | `version=2` through the respective side-channel for the transport being | |
55 | used which inevitably sets `GIT_PROTOCOL`. More information can be | |
1e232016 | 56 | found in linkgit:gitprotocol-pack[5] and linkgit:gitprotocol-http[5], as well as the |
1b421e7a | 57 | `GIT_PROTOCOL` definition in `git.txt`. In all cases the |
ed10cb95 BW |
58 | response from the server is the capability advertisement. |
59 | ||
b373e4d2 SG |
60 | Git Transport |
61 | ~~~~~~~~~~~~~ | |
ed10cb95 BW |
62 | |
63 | When using the git:// transport, you can request to use protocol v2 by | |
64 | sending "version=2" as an extra parameter: | |
65 | ||
66 | 003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0 | |
67 | ||
b373e4d2 SG |
68 | SSH and File Transport |
69 | ~~~~~~~~~~~~~~~~~~~~~~ | |
ed10cb95 BW |
70 | |
71 | When using either the ssh:// or file:// transport, the GIT_PROTOCOL | |
72 | environment variable must be set explicitly to include "version=2". | |
1b421e7a JK |
73 | The server may need to be configured to allow this environment variable |
74 | to pass. | |
ed10cb95 | 75 | |
b373e4d2 SG |
76 | HTTP Transport |
77 | ~~~~~~~~~~~~~~ | |
ed10cb95 BW |
78 | |
79 | When using the http:// or https:// transport a client makes a "smart" | |
1e232016 | 80 | info/refs request as described in linkgit:gitprotocol-http[5] and requests that |
ed10cb95 BW |
81 | v2 be used by supplying "version=2" in the `Git-Protocol` header. |
82 | ||
ed10cb95 | 83 | C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 |
bb4d000e | 84 | C: Git-Protocol: version=2 |
ed10cb95 BW |
85 | |
86 | A v2 server would reply: | |
87 | ||
88 | S: 200 OK | |
89 | S: <Some headers> | |
90 | S: ... | |
91 | S: | |
92 | S: 000eversion 2\n | |
93 | S: <capability-advertisement> | |
94 | ||
95 | Subsequent requests are then made directly to the service | |
96 | `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack). | |
97 | ||
98e2d9d6 ÆAB |
98 | Uses the `--http-backend-info-refs` option to |
99 | linkgit:git-upload-pack[1]. | |
100 | ||
1b421e7a JK |
101 | The server may need to be configured to pass this header's contents via |
102 | the `GIT_PROTOCOL` variable. See the discussion in `git-http-backend.txt`. | |
103 | ||
b373e4d2 SG |
104 | Capability Advertisement |
105 | ------------------------ | |
ed10cb95 BW |
106 | |
107 | A server which decides to communicate (based on a request from a client) | |
108 | using protocol version 2, notifies the client by sending a version string | |
109 | in its initial response followed by an advertisement of its capabilities. | |
110 | Each capability is a key with an optional value. Clients must ignore all | |
111 | unknown keys. Semantics of unknown values are left to the definition of | |
112 | each key. Some capabilities will describe commands which can be requested | |
113 | to be executed by the client. | |
114 | ||
115 | capability-advertisement = protocol-version | |
116 | capability-list | |
117 | flush-pkt | |
118 | ||
119 | protocol-version = PKT-LINE("version 2" LF) | |
120 | capability-list = *capability | |
121 | capability = PKT-LINE(key[=value] LF) | |
122 | ||
123 | key = 1*(ALPHA | DIGIT | "-_") | |
124 | value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;") | |
125 | ||
b373e4d2 SG |
126 | Command Request |
127 | --------------- | |
ed10cb95 BW |
128 | |
129 | After receiving the capability advertisement, a client can then issue a | |
130 | request to select the command it wants with any particular capabilities | |
131 | or arguments. There is then an optional section where the client can | |
132 | provide any command specific parameters or queries. Only a single | |
133 | command can be requested at a time. | |
134 | ||
135 | request = empty-request | command-request | |
136 | empty-request = flush-pkt | |
137 | command-request = command | |
138 | capability-list | |
74db416c CW |
139 | delim-pkt |
140 | command-args | |
ed10cb95 BW |
141 | flush-pkt |
142 | command = PKT-LINE("command=" key LF) | |
74db416c | 143 | command-args = *command-specific-arg |
ed10cb95 BW |
144 | |
145 | command-specific-args are packet line framed arguments defined by | |
146 | each individual command. | |
147 | ||
148 | The server will then check to ensure that the client's request is | |
149 | comprised of a valid command as well as valid capabilities which were | |
150 | advertised. If the request is valid the server will then execute the | |
151 | command. A server MUST wait till it has received the client's entire | |
152 | request before issuing a response. The format of the response is | |
153 | determined by the command being executed, but in all cases a flush-pkt | |
154 | indicates the end of the response. | |
155 | ||
156 | When a command has finished, and the client has received the entire | |
157 | response from the server, a client can either request that another | |
158 | command be executed or can terminate the connection. A client may | |
159 | optionally send an empty request consisting of just a flush-pkt to | |
160 | indicate that no more requests will be made. | |
161 | ||
b373e4d2 SG |
162 | Capabilities |
163 | ------------ | |
ed10cb95 BW |
164 | |
165 | There are two different types of capabilities: normal capabilities, | |
24966cd9 | 166 | which can be used to convey information or alter the behavior of a |
ed10cb95 BW |
167 | request, and commands, which are the core actions that a client wants to |
168 | perform (fetch, push, etc). | |
169 | ||
170 | Protocol version 2 is stateless by default. This means that all commands | |
171 | must only last a single round and be stateless from the perspective of the | |
172 | server side, unless the client has requested a capability indicating that | |
173 | state should be maintained by the server. Clients MUST NOT require state | |
174 | management on the server side in order to function correctly. This | |
175 | permits simple round-robin load-balancing on the server side, without | |
176 | needing to worry about state management. | |
177 | ||
b373e4d2 SG |
178 | agent |
179 | ~~~~~ | |
ed10cb95 BW |
180 | |
181 | The server can advertise the `agent` capability with a value `X` (in the | |
182 | form `agent=X`) to notify the client that the server is running version | |
183 | `X`. The client may optionally send its own agent string by including | |
184 | the `agent` capability with a value `Y` (in the form `agent=Y`) in its | |
185 | request to the server (but it MUST NOT do so if the server did not | |
186 | advertise the agent capability). The `X` and `Y` strings may contain any | |
187 | printable ASCII characters except space (i.e., the byte range 32 < x < | |
188 | 127), and are typically of the form "package/version" (e.g., | |
189 | "git/1.8.3.1"). The agent strings are purely informative for statistics | |
190 | and debugging purposes, and MUST NOT be used to programmatically assume | |
191 | the presence or absence of particular features. | |
72d0ea00 | 192 | |
b373e4d2 SG |
193 | ls-refs |
194 | ~~~~~~~ | |
72d0ea00 BW |
195 | |
196 | `ls-refs` is the command used to request a reference advertisement in v2. | |
197 | Unlike the current reference advertisement, ls-refs takes in arguments | |
198 | which can be used to limit the refs sent from the server. | |
199 | ||
200 | Additional features not supported in the base command will be advertised | |
201 | as the value of the command in the capability advertisement in the form | |
2162f9f6 | 202 | of a space separated list of features: "<command>=<feature-1> <feature-2>" |
72d0ea00 BW |
203 | |
204 | ls-refs takes in the following arguments: | |
205 | ||
206 | symrefs | |
207 | In addition to the object pointed by it, show the underlying ref | |
208 | pointed by it when showing a symbolic ref. | |
209 | peel | |
210 | Show peeled tags. | |
211 | ref-prefix <prefix> | |
212 | When specified, only references having a prefix matching one of | |
9db5fb4f JK |
213 | the provided prefixes are displayed. Multiple instances may be |
214 | given, in which case references matching any prefix will be | |
215 | shown. Note that this is purely for optimization; a server MAY | |
216 | show refs not matching the prefix if it chooses, and clients | |
217 | should filter the result themselves. | |
72d0ea00 | 218 | |
59e1205d JT |
219 | If the 'unborn' feature is advertised the following argument can be |
220 | included in the client's request. | |
221 | ||
222 | unborn | |
223 | The server will send information about HEAD even if it is a symref | |
224 | pointing to an unborn branch in the form "unborn HEAD | |
225 | symref-target:<target>". | |
226 | ||
72d0ea00 BW |
227 | The output of ls-refs is as follows: |
228 | ||
229 | output = *ref | |
230 | flush-pkt | |
59e1205d JT |
231 | obj-id-or-unborn = (obj-id | "unborn") |
232 | ref = PKT-LINE(obj-id-or-unborn SP refname *(SP ref-attribute) LF) | |
72d0ea00 BW |
233 | ref-attribute = (symref | peeled) |
234 | symref = "symref-target:" symref-target | |
235 | peeled = "peeled:" obj-id | |
3145ea95 | 236 | |
b373e4d2 SG |
237 | fetch |
238 | ~~~~~ | |
3145ea95 BW |
239 | |
240 | `fetch` is the command used to fetch a packfile in v2. It can be looked | |
241 | at as a modified version of the v1 fetch where the ref-advertisement is | |
242 | stripped out (since the `ls-refs` command fills that role) and the | |
243 | message format is tweaked to eliminate redundancies and permit easy | |
244 | addition of future extensions. | |
245 | ||
246 | Additional features not supported in the base command will be advertised | |
247 | as the value of the command in the capability advertisement in the form | |
2162f9f6 | 248 | of a space separated list of features: "<command>=<feature-1> <feature-2>" |
3145ea95 BW |
249 | |
250 | A `fetch` request can take the following arguments: | |
251 | ||
252 | want <oid> | |
253 | Indicates to the server an object which the client wants to | |
254 | retrieve. Wants can be anything and are not limited to | |
255 | advertised objects. | |
256 | ||
257 | have <oid> | |
258 | Indicates to the server an object which the client has locally. | |
259 | This allows the server to make a packfile which only contains | |
260 | the objects that the client needs. Multiple 'have' lines can be | |
261 | supplied. | |
262 | ||
263 | done | |
264 | Indicates to the server that negotiation should terminate (or | |
265 | not even begin if performing a clone) and that the server should | |
266 | use the information supplied in the request to construct the | |
267 | packfile. | |
268 | ||
269 | thin-pack | |
270 | Request that a thin pack be sent, which is a pack with deltas | |
271 | which reference base objects not contained within the pack (but | |
272 | are known to exist at the receiving end). This can reduce the | |
273 | network traffic significantly, but it requires the receiving end | |
274 | to know how to "thicken" these packs by adding the missing bases | |
275 | to the pack. | |
276 | ||
277 | no-progress | |
278 | Request that progress information that would normally be sent on | |
279 | side-band channel 2, during the packfile transfer, should not be | |
280 | sent. However, the side-band channel 3 is still used for error | |
281 | responses. | |
282 | ||
283 | include-tag | |
284 | Request that annotated tags should be sent if the objects they | |
285 | point to are being sent. | |
286 | ||
287 | ofs-delta | |
288 | Indicate that the client understands PACKv2 with delta referring | |
289 | to its base by position in pack rather than by an oid. That is, | |
031fd4b9 | 290 | they can read OBJ_OFS_DELTA (aka type 6) in a packfile. |
3145ea95 | 291 | |
f7e20501 BW |
292 | If the 'shallow' feature is advertised the following arguments can be |
293 | included in the clients request as well as the potential addition of the | |
294 | 'shallow-info' section in the server's response as explained below. | |
295 | ||
685fbd32 BW |
296 | shallow <oid> |
297 | A client must notify the server of all commits for which it only | |
298 | has shallow copies (meaning that it doesn't have the parents of | |
299 | a commit) by supplying a 'shallow <oid>' line for each such | |
300 | object so that the server is aware of the limitations of the | |
301 | client's history. This is so that the server is aware that the | |
302 | client may not have all objects reachable from such commits. | |
303 | ||
304 | deepen <depth> | |
305 | Requests that the fetch/clone should be shallow having a commit | |
306 | depth of <depth> relative to the remote side. | |
307 | ||
308 | deepen-relative | |
309 | Requests that the semantics of the "deepen" command be changed | |
310 | to indicate that the depth requested is relative to the client's | |
311 | current shallow boundary, instead of relative to the requested | |
312 | commits. | |
313 | ||
314 | deepen-since <timestamp> | |
315 | Requests that the shallow clone/fetch should be cut at a | |
316 | specific time, instead of depth. Internally it's equivalent to | |
317 | doing "git rev-list --max-age=<timestamp>". Cannot be used with | |
318 | "deepen". | |
319 | ||
320 | deepen-not <rev> | |
321 | Requests that the shallow clone/fetch should be cut at a | |
322 | specific revision specified by '<rev>', instead of a depth. | |
323 | Internally it's equivalent of doing "git rev-list --not <rev>". | |
324 | Cannot be used with "deepen", but can be used with | |
325 | "deepen-since". | |
326 | ||
ba95710a JT |
327 | If the 'filter' feature is advertised, the following argument can be |
328 | included in the client's request: | |
329 | ||
330 | filter <filter-spec> | |
331 | Request that various objects from the packfile be omitted | |
332 | using one of several filtering techniques. These are intended | |
333 | for use with partial clone and partial fetch operations. See | |
87c2d9d3 JS |
334 | `rev-list` for possible "filter-spec" values. When communicating |
335 | with other processes, senders SHOULD translate scaled integers | |
336 | (e.g. "1k") into a fully-expanded form (e.g. "1024") to aid | |
337 | interoperability with older receivers that may not understand | |
338 | newly-invented scaling suffixes. However, receivers SHOULD | |
339 | accept the following suffixes: 'k', 'm', and 'g' for 1024, | |
340 | 1048576, and 1073741824, respectively. | |
ba95710a | 341 | |
516e2b76 BW |
342 | If the 'ref-in-want' feature is advertised, the following argument can |
343 | be included in the client's request as well as the potential addition of | |
344 | the 'wanted-refs' section in the server's response as explained below. | |
345 | ||
346 | want-ref <ref> | |
347 | Indicates to the server that the client wants to retrieve a | |
348 | particular ref, where <ref> is the full name of a ref on the | |
b065063c JK |
349 | server. It is a protocol error to send want-ref for the |
350 | same ref more than once. | |
516e2b76 | 351 | |
0bbc0bc5 JT |
352 | If the 'sideband-all' feature is advertised, the following argument can be |
353 | included in the client's request: | |
354 | ||
355 | sideband-all | |
356 | Instruct the server to send the whole response multiplexed, not just | |
357 | the packfile section. All non-flush and non-delim PKT-LINE in the | |
358 | response (not only in the packfile section) will then start with a byte | |
359 | indicating its sideband (1, 2, or 3), and the server may send "0005\2" | |
360 | (a PKT-LINE of sideband 2 with no payload) as a keepalive packet. | |
361 | ||
cd8402e0 JT |
362 | If the 'packfile-uris' feature is advertised, the following argument |
363 | can be included in the client's request as well as the potential | |
364 | addition of the 'packfile-uris' section in the server's response as | |
179776f9 JK |
365 | explained below. Note that at most one `packfile-uris` line can be sent |
366 | to the server. | |
cd8402e0 | 367 | |
2162f9f6 | 368 | packfile-uris <comma-separated-list-of-protocols> |
cd8402e0 JT |
369 | Indicates to the server that the client is willing to receive |
370 | URIs of any of the given protocols in place of objects in the | |
371 | sent packfile. Before performing the connectivity check, the | |
372 | client should download from all given URIs. Currently, the | |
373 | protocols supported are "http" and "https". | |
374 | ||
9c1e657a JT |
375 | If the 'wait-for-done' feature is advertised, the following argument |
376 | can be included in the client's request. | |
377 | ||
378 | wait-for-done | |
379 | Indicates to the server that it should never send "ready", but | |
380 | should wait for the client to say "done" before sending the | |
381 | packfile. | |
382 | ||
3145ea95 BW |
383 | The response of `fetch` is broken into a number of sections separated by |
384 | delimiter packets (0001), with each section beginning with its section | |
fd194dd5 | 385 | header. Most sections are sent only when the packfile is sent. |
3145ea95 | 386 | |
fd194dd5 JT |
387 | output = acknowledgements flush-pkt | |
388 | [acknowledgments delim-pkt] [shallow-info delim-pkt] | |
cd8402e0 JT |
389 | [wanted-refs delim-pkt] [packfile-uris delim-pkt] |
390 | packfile flush-pkt | |
3145ea95 BW |
391 | |
392 | acknowledgments = PKT-LINE("acknowledgments" LF) | |
393 | (nak | *ack) | |
394 | (ready) | |
395 | ready = PKT-LINE("ready" LF) | |
396 | nak = PKT-LINE("NAK" LF) | |
397 | ack = PKT-LINE("ACK" SP obj-id LF) | |
398 | ||
685fbd32 BW |
399 | shallow-info = PKT-LINE("shallow-info" LF) |
400 | *PKT-LINE((shallow | unshallow) LF) | |
401 | shallow = "shallow" SP obj-id | |
402 | unshallow = "unshallow" SP obj-id | |
403 | ||
516e2b76 BW |
404 | wanted-refs = PKT-LINE("wanted-refs" LF) |
405 | *PKT-LINE(wanted-ref LF) | |
406 | wanted-ref = obj-id SP refname | |
407 | ||
cd8402e0 JT |
408 | packfile-uris = PKT-LINE("packfile-uris" LF) *packfile-uri |
409 | packfile-uri = PKT-LINE(40*(HEXDIGIT) SP *%x20-ff LF) | |
410 | ||
3145ea95 BW |
411 | packfile = PKT-LINE("packfile" LF) |
412 | *PKT-LINE(%x01-03 *%x00-ff) | |
413 | ||
414 | acknowledgments section | |
fd194dd5 JT |
415 | * If the client determines that it is finished with negotiations by |
416 | sending a "done" line (thus requiring the server to send a packfile), | |
417 | the acknowledgments sections MUST be omitted from the server's | |
418 | response. | |
3145ea95 BW |
419 | |
420 | * Always begins with the section header "acknowledgments" | |
421 | ||
422 | * The server will respond with "NAK" if none of the object ids sent | |
423 | as have lines were common. | |
424 | ||
425 | * The server will respond with "ACK obj-id" for all of the | |
426 | object ids sent as have lines which are common. | |
427 | ||
428 | * A response cannot have both "ACK" lines as well as a "NAK" | |
429 | line. | |
430 | ||
431 | * The server will respond with a "ready" line indicating that | |
432 | the server has found an acceptable common base and is ready to | |
433 | make and send a packfile (which will be found in the packfile | |
434 | section of the same response) | |
435 | ||
436 | * If the server has found a suitable cut point and has decided | |
437 | to send a "ready" line, then the server can decide to (as an | |
438 | optimization) omit any "ACK" lines it would have sent during | |
439 | its response. This is because the server will have already | |
440 | determined the objects it plans to send to the client and no | |
441 | further negotiation is needed. | |
442 | ||
685fbd32 | 443 | shallow-info section |
f7e20501 BW |
444 | * If the client has requested a shallow fetch/clone, a shallow |
445 | client requests a fetch or the server is shallow then the | |
446 | server's response may include a shallow-info section. The | |
447 | shallow-info section will be included if (due to one of the | |
448 | above conditions) the server needs to inform the client of any | |
449 | shallow boundaries or adjustments to the clients already | |
450 | existing shallow boundaries. | |
685fbd32 BW |
451 | |
452 | * Always begins with the section header "shallow-info" | |
453 | ||
454 | * If a positive depth is requested, the server will compute the | |
455 | set of commits which are no deeper than the desired depth. | |
456 | ||
457 | * The server sends a "shallow obj-id" line for each commit whose | |
458 | parents will not be sent in the following packfile. | |
459 | ||
460 | * The server sends an "unshallow obj-id" line for each commit | |
461 | which the client has indicated is shallow, but is no longer | |
462 | shallow as a result of the fetch (due to its parents being | |
463 | sent in the following packfile). | |
464 | ||
465 | * The server MUST NOT send any "unshallow" lines for anything | |
466 | which the client has not indicated was shallow as a part of | |
467 | its request. | |
468 | ||
516e2b76 BW |
469 | wanted-refs section |
470 | * This section is only included if the client has requested a | |
471 | ref using a 'want-ref' line and if a packfile section is also | |
472 | included in the response. | |
473 | ||
474 | * Always begins with the section header "wanted-refs". | |
475 | ||
476 | * The server will send a ref listing ("<oid> <refname>") for | |
477 | each reference requested using 'want-ref' lines. | |
478 | ||
479 | * The server MUST NOT send any refs which were not requested | |
480 | using 'want-ref' lines. | |
481 | ||
cd8402e0 JT |
482 | packfile-uris section |
483 | * This section is only included if the client sent | |
484 | 'packfile-uris' and the server has at least one such URI to | |
485 | send. | |
486 | ||
487 | * Always begins with the section header "packfile-uris". | |
488 | ||
489 | * For each URI the server sends, it sends a hash of the pack's | |
490 | contents (as output by git index-pack) followed by the URI. | |
491 | ||
492 | * The hashes are 40 hex characters long. When Git upgrades to a new | |
493 | hash algorithm, this might need to be updated. (It should match | |
494 | whatever index-pack outputs after "pack\t" or "keep\t". | |
495 | ||
3145ea95 BW |
496 | packfile section |
497 | * This section is only included if the client has sent 'want' | |
498 | lines in its request and either requested that no more | |
499 | negotiation be done by sending 'done' or if the server has | |
500 | decided it has found a sufficient cut point to produce a | |
501 | packfile. | |
502 | ||
503 | * Always begins with the section header "packfile" | |
504 | ||
505 | * The transmission of the packfile begins immediately after the | |
506 | section header | |
507 | ||
508 | * The data transfer of the packfile is always multiplexed, using | |
509 | the same semantics of the 'side-band-64k' capability from | |
510 | protocol version 1. This means that each packet, during the | |
511 | packfile data stream, is made up of a leading 4-byte pkt-line | |
512 | length (typical of the pkt-line format), followed by a 1-byte | |
513 | stream code, followed by the actual data. | |
514 | ||
515 | The stream code can be one of: | |
516 | 1 - pack data | |
517 | 2 - progress messages | |
518 | 3 - fatal error message just before stream aborts | |
ecc3e534 | 519 | |
b373e4d2 SG |
520 | server-option |
521 | ~~~~~~~~~~~~~ | |
ecc3e534 BW |
522 | |
523 | If advertised, indicates that any number of server specific options can be | |
524 | included in a request. This is done by sending each option as a | |
525 | "server-option=<option>" capability line in the capability-list section of | |
526 | a request. | |
527 | ||
528 | The provided options must not contain a NUL or LF character. | |
7f46e7ea | 529 | |
530 | object-format | |
531 | ~~~~~~~~~~~~~~~ | |
532 | ||
533 | The server can advertise the `object-format` capability with a value `X` (in the | |
534 | form `object-format=X`) to notify the client that the server is able to deal | |
535 | with objects using hash algorithm X. If not specified, the server is assumed to | |
536 | only handle SHA-1. If the client would like to use a hash algorithm other than | |
537 | SHA-1, it should specify its object-format string. | |
f5cdbe48 | 538 | |
2162f9f6 | 539 | session-id=<session-id> |
f5cdbe48 JS |
540 | ~~~~~~~~~~~~~~~~~~~~~~~ |
541 | ||
542 | The server may advertise a session ID that can be used to identify this process | |
543 | across multiple requests. The client may advertise its own session ID back to | |
544 | the server as well. | |
545 | ||
546 | Session IDs should be unique to a given process. They must fit within a | |
547 | packet-line, and must not contain non-printable or whitespace characters. The | |
548 | current implementation uses trace2 session IDs (see | |
086eaab8 TZ |
549 | link:technical/api-trace2.html[api-trace2] for details), but this may change |
550 | and users of the session ID should not rely on this fact. | |
a2ba162c BA |
551 | |
552 | object-info | |
553 | ~~~~~~~~~~~ | |
554 | ||
555 | `object-info` is the command to retrieve information about one or more objects. | |
556 | Its main purpose is to allow a client to make decisions based on this | |
557 | information without having to fully fetch objects. Object size is the only | |
558 | information that is currently supported. | |
559 | ||
560 | An `object-info` request takes the following arguments: | |
561 | ||
562 | size | |
563 | Requests size information to be returned for each listed object id. | |
564 | ||
565 | oid <oid> | |
566 | Indicates to the server an object which the client wants to obtain | |
567 | information for. | |
568 | ||
abcb66c6 | 569 | The response of `object-info` is a list of the requested object ids |
a2ba162c BA |
570 | and associated requested information, each separated by a single space. |
571 | ||
572 | output = info flush-pkt | |
573 | ||
574 | info = PKT-LINE(attrs) LF) | |
575 | *PKT-LINE(obj-info LF) | |
576 | ||
577 | attrs = attr | attrs SP attrs | |
578 | ||
579 | attr = "size" | |
580 | ||
581 | obj-info = obj-id SP obj-size | |
5db92105 | 582 | |
8b8d9a22 ÆAB |
583 | bundle-uri |
584 | ~~~~~~~~~~ | |
585 | ||
586 | If the 'bundle-uri' capability is advertised, the server supports the | |
587 | `bundle-uri' command. | |
588 | ||
589 | The capability is currently advertised with no value (i.e. not | |
590 | "bundle-uri=somevalue"), a value may be added in the future for | |
591 | supporting command-wide extensions. Clients MUST ignore any unknown | |
592 | capability values and proceed with the 'bundle-uri` dialog they | |
593 | support. | |
594 | ||
595 | The 'bundle-uri' command is intended to be issued before `fetch` to | |
596 | get URIs to bundle files (see linkgit:git-bundle[1]) to "seed" and | |
597 | inform the subsequent `fetch` command. | |
598 | ||
599 | The client CAN issue `bundle-uri` before or after any other valid | |
600 | command. To be useful to clients it's expected that it'll be issued | |
601 | after an `ls-refs` and before `fetch`, but CAN be issued at any time | |
602 | in the dialog. | |
603 | ||
604 | DISCUSSION of bundle-uri | |
605 | ^^^^^^^^^^^^^^^^^^^^^^^^ | |
606 | ||
607 | The intent of the feature is optimize for server resource consumption | |
608 | in the common case by changing the common case of fetching a very | |
609 | large PACK during linkgit:git-clone[1] into a smaller incremental | |
610 | fetch. | |
611 | ||
612 | It also allows servers to achieve better caching in combination with | |
613 | an `uploadpack.packObjectsHook` (see linkgit:git-config[1]). | |
614 | ||
615 | By having new clones or fetches be a more predictable and common | |
616 | negotiation against the tips of recently produces *.bundle file(s). | |
617 | Servers might even pre-generate the results of such negotiations for | |
618 | the `uploadpack.packObjectsHook` as new pushes come in. | |
619 | ||
620 | One way that servers could take advantage of these bundles is that the | |
621 | server would anticipate that fresh clones will download a known bundle, | |
622 | followed by catching up to the current state of the repository using ref | |
623 | tips found in that bundle (or bundles). | |
624 | ||
625 | PROTOCOL for bundle-uri | |
626 | ^^^^^^^^^^^^^^^^^^^^^^^ | |
627 | ||
628 | A `bundle-uri` request takes no arguments, and as noted above does not | |
629 | currently advertise a capability value. Both may be added in the | |
630 | future. | |
631 | ||
632 | When the client issues a `command=bundle-uri` request, the response is a | |
633 | list of key-value pairs provided as packet lines with value | |
634 | `<key>=<value>`. Each `<key>` should be interpreted as a config key from | |
635 | the `bundle.*` namespace to construct a list of bundles. These keys are | |
636 | grouped by a `bundle.<id>.` subsection, where each key corresponding to a | |
637 | given `<id>` contributes attributes to the bundle defined by that `<id>`. | |
638 | See linkgit:git-config[1] for the specific details of these keys and how | |
639 | the Git client will interpret their values. | |
640 | ||
641 | Clients MUST parse the line according to the above format, lines that do | |
642 | not conform to the format SHOULD be discarded. The user MAY be warned in | |
643 | such a case. | |
644 | ||
645 | bundle-uri CLIENT AND SERVER EXPECTATIONS | |
646 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
647 | ||
648 | URI CONTENTS:: | |
649 | The content at the advertised URIs MUST be one of two types. | |
650 | + | |
651 | The advertised URI may contain a bundle file that `git bundle verify` | |
652 | would accept. I.e. they MUST contain one or more reference tips for | |
653 | use by the client, MUST indicate prerequisites (in any) with standard | |
654 | "-" prefixes, and MUST indicate their "object-format", if | |
655 | applicable. | |
656 | + | |
657 | The advertised URI may alternatively contain a plaintext file that `git | |
658 | config --list` would accept (with the `--file` option). The key-value | |
659 | pairs in this list are in the `bundle.*` namespace (see | |
660 | linkgit:git-config[1]). | |
661 | ||
662 | bundle-uri CLIENT ERROR RECOVERY:: | |
663 | A client MUST above all gracefully degrade on errors, whether that | |
664 | error is because of bad missing/data in the bundle URI(s), because | |
665 | that client is too dumb to e.g. understand and fully parse out bundle | |
666 | headers and their prerequisite relationships, or something else. | |
667 | + | |
668 | Server operators should feel confident in turning on "bundle-uri" and | |
669 | not worry if e.g. their CDN goes down that clones or fetches will run | |
b39a8418 | 670 | into hard failures. Even if the server bundle(s) are |
8b8d9a22 ÆAB |
671 | incomplete, or bad in some way the client should still end up with a |
672 | functioning repository, just as if it had chosen not to use this | |
673 | protocol extension. | |
674 | + | |
675 | All subsequent discussion on client and server interaction MUST keep | |
676 | this in mind. | |
677 | ||
678 | bundle-uri SERVER TO CLIENT:: | |
679 | The ordering of the returned bundle uris is not significant. Clients | |
680 | MUST parse their headers to discover their contained OIDS and | |
681 | prerequisites. A client MUST consider the content of the bundle(s) | |
682 | themselves and their header as the ultimate source of truth. | |
683 | + | |
684 | A server MAY even return bundle(s) that don't have any direct | |
685 | relationship to the repository being cloned (either through accident, | |
686 | or intentional "clever" configuration), and expect a client to sort | |
687 | out what data they'd like from the bundle(s), if any. | |
688 | ||
689 | bundle-uri CLIENT TO SERVER:: | |
690 | The client SHOULD provide reference tips found in the bundle header(s) | |
691 | as 'have' lines in any subsequent `fetch` request. A client MAY also | |
692 | ignore the bundle(s) entirely if doing so is deemed worse for some | |
693 | reason, e.g. if the bundles can't be downloaded, it doesn't like the | |
694 | tips it finds etc. | |
695 | ||
696 | WHEN ADVERTISED BUNDLE(S) REQUIRE NO FURTHER NEGOTIATION:: | |
697 | If after issuing `bundle-uri` and `ls-refs`, and getting the header(s) | |
698 | of the bundle(s) the client finds that the ref tips it wants can be | |
699 | retrieved entirely from advertised bundle(s), the client MAY disconnect | |
700 | from the Git server. The results of such a 'clone' or 'fetch' should be | |
701 | indistinguishable from the state attained without using bundle-uri. | |
702 | ||
703 | EARLY CLIENT DISCONNECTIONS AND ERROR RECOVERY:: | |
704 | A client MAY perform an early disconnect while still downloading the | |
705 | bundle(s) (having streamed and parsed their headers). In such a case | |
706 | the client MUST gracefully recover from any errors related to | |
707 | finishing the download and validation of the bundle(s). | |
708 | + | |
709 | I.e. a client might need to re-connect and issue a 'fetch' command, | |
710 | and possibly fall back to not making use of 'bundle-uri' at all. | |
711 | + | |
712 | This "MAY" behavior is specified as such (and not a "SHOULD") on the | |
713 | assumption that a server advertising bundle uris is more likely than | |
714 | not to be serving up a relatively large repository, and to be pointing | |
715 | to URIs that have a good chance of being in working order. A client | |
716 | MAY e.g. look at the payload size of the bundles as a heuristic to see | |
717 | if an early disconnect is worth it, should falling back on a full | |
718 | "fetch" dialog be necessary. | |
719 | ||
720 | WHEN ADVERTISED BUNDLE(S) REQUIRE FURTHER NEGOTIATION:: | |
721 | A client SHOULD commence a negotiation of a PACK from the server via | |
722 | the "fetch" command using the OID tips found in advertised bundles, | |
723 | even if's still in the process of downloading those bundle(s). | |
724 | + | |
725 | This allows for aggressive early disconnects from any interactive | |
726 | server dialog. The client blindly trusts that the advertised OID tips | |
727 | are relevant, and issues them as 'have' lines, it then requests any | |
728 | tips it would like (usually from the "ls-refs" advertisement) via | |
729 | 'want' lines. The server will then compute a (hopefully small) PACK | |
730 | with the expected difference between the tips from the bundle(s) and | |
731 | the data requested. | |
732 | + | |
733 | The only connection the client then needs to keep active is to the | |
734 | concurrently downloading static bundle(s), when those and the | |
735 | incremental PACK are retrieved they should be inflated and | |
736 | validated. Any errors at this point should be gracefully recovered | |
737 | from, see above. | |
738 | ||
739 | bundle-uri PROTOCOL FEATURES | |
740 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
741 | ||
742 | The client constructs a bundle list from the `<key>=<value>` pairs | |
743 | provided by the server. These pairs are part of the `bundle.*` namespace | |
744 | as documented in linkgit:git-config[1]. In this section, we discuss some | |
745 | of these keys and describe the actions the client will do in response to | |
746 | this information. | |
747 | ||
748 | In particular, the `bundle.version` key specifies an integer value. The | |
749 | only accepted value at the moment is `1`, but if the client sees an | |
750 | unexpected value here then the client MUST ignore the bundle list. | |
751 | ||
752 | As long as `bundle.version` is understood, all other unknown keys MAY be | |
753 | ignored by the client. The server will guarantee compatibility with older | |
754 | clients, though newer clients may be better able to use the extra keys to | |
755 | minimize downloads. | |
756 | ||
757 | Any backwards-incompatible addition of pre-URI key-value will be | |
758 | guarded by a new `bundle.version` value or values in 'bundle-uri' | |
759 | capability advertisement itself, and/or by new future `bundle-uri` | |
760 | request arguments. | |
761 | ||
762 | Some example key-value pairs that are not currently implemented but could | |
763 | be implemented in the future include: | |
764 | ||
765 | * Add a "hash=<val>" or "size=<bytes>" advertise the expected hash or | |
766 | size of the bundle file. | |
767 | ||
768 | * Advertise that one or more bundle files are the same (to e.g. have | |
769 | clients round-robin or otherwise choose one of N possible files). | |
770 | ||
771 | * A "oid=<OID>" shortcut and "prerequisite=<OID>" shortcut. For | |
772 | expressing the common case of a bundle with one tip and no | |
773 | prerequisites, or one tip and one prerequisite. | |
774 | + | |
775 | This would allow for optimizing the common case of servers who'd like | |
776 | to provide one "big bundle" containing only their "main" branch, | |
777 | and/or incremental updates thereof. | |
778 | + | |
779 | A client receiving such a a response MAY assume that they can skip | |
780 | retrieving the header from a bundle at the indicated URI, and thus | |
781 | save themselves and the server(s) the request(s) needed to inspect the | |
782 | headers of that bundle or bundles. | |
783 | ||
5db92105 ÆAB |
784 | GIT |
785 | --- | |
786 | Part of the linkgit:git[1] suite |