]>
Commit | Line | Data |
---|---|---|
cd8402e0 JT |
1 | Packfile URIs |
2 | ============= | |
3 | ||
4 | This feature allows servers to serve part of their packfile response as URIs. | |
5 | This allows server designs that improve scalability in bandwidth and CPU usage | |
6 | (for example, by serving some data through a CDN), and (in the future) provides | |
7 | some measure of resumability to clients. | |
8 | ||
9 | This feature is available only in protocol version 2. | |
10 | ||
11 | Protocol | |
12 | -------- | |
13 | ||
14 | The server advertises the `packfile-uris` capability. | |
15 | ||
16 | If the client then communicates which protocols (HTTPS, etc.) it supports with | |
17 | a `packfile-uris` argument, the server MAY send a `packfile-uris` section | |
18 | directly before the `packfile` section (right after `wanted-refs` if it is | |
19 | sent) containing URIs of any of the given protocols. The URIs point to | |
20 | packfiles that use only features that the client has declared that it supports | |
21 | (e.g. ofs-delta and thin-pack). See protocol-v2.txt for the documentation of | |
22 | this section. | |
23 | ||
24 | Clients should then download and index all the given URIs (in addition to | |
25 | downloading and indexing the packfile given in the `packfile` section of the | |
26 | response) before performing the connectivity check. | |
27 | ||
28 | Server design | |
29 | ------------- | |
30 | ||
31 | The server can be trivially made compatible with the proposed protocol by | |
32 | having it advertise `packfile-uris`, tolerating the client sending | |
33 | `packfile-uris`, and never sending any `packfile-uris` section. But we should | |
34 | include some sort of non-trivial implementation in the Minimum Viable Product, | |
35 | at least so that we can test the client. | |
36 | ||
37 | This is the implementation: a feature, marked experimental, that allows the | |
38 | server to be configured by one or more `uploadpack.blobPackfileUri=<sha1> | |
39 | <uri>` entries. Whenever the list of objects to be sent is assembled, all such | |
bfc2a36f JT |
40 | blobs are excluded, replaced with URIs. As noted in "Future work" below, the |
41 | server can evolve in the future to support excluding other objects (or other | |
42 | implementations of servers could be made that support excluding other objects) | |
43 | without needing a protocol change, so clients should not expect that packfiles | |
44 | downloaded in this way only contain single blobs. | |
cd8402e0 JT |
45 | |
46 | Client design | |
47 | ------------- | |
48 | ||
49 | The client has a config variable `fetch.uriprotocols` that determines which | |
50 | protocols the end user is willing to use. By default, this is empty. | |
51 | ||
52 | When the client downloads the given URIs, it should store them with "keep" | |
53 | files, just like it does with the packfile in the `packfile` section. These | |
54 | additional "keep" files can only be removed after the refs have been updated - | |
55 | just like the "keep" file for the packfile in the `packfile` section. | |
56 | ||
57 | The division of work (initial fetch + additional URIs) introduces convenient | |
58 | points for resumption of an interrupted clone - such resumption can be done | |
59 | after the Minimum Viable Product (see "Future work"). | |
60 | ||
61 | Future work | |
62 | ----------- | |
63 | ||
64 | The protocol design allows some evolution of the server and client without any | |
65 | need for protocol changes, so only a small-scoped design is included here to | |
66 | form the MVP. For example, the following can be done: | |
67 | ||
68 | * On the server, more sophisticated means of excluding objects (e.g. by | |
69 | specifying a commit to represent that commit and all objects that it | |
70 | references). | |
71 | * On the client, resumption of clone. If a clone is interrupted, information | |
72 | could be recorded in the repository's config and a "clone-resume" command | |
73 | can resume the clone in progress. (Resumption of subsequent fetches is more | |
74 | difficult because that must deal with the user wanting to use the repository | |
75 | even after the fetch was interrupted.) | |
76 | ||
77 | There are some possible features that will require a change in protocol: | |
78 | ||
79 | * Additional HTTP headers (e.g. authentication) | |
80 | * Byte range support | |
81 | * Different file formats referenced by URIs (e.g. raw object) |