Documentation/technical/protocol-v2.txt

   1  Git Wire Protocol, Version 2
   2 ==============================
   3
   4 This document presents a specification for a version 2 of Git's wire
   5 protocol.  Protocol v2 will improve upon v1 in the following ways:
   6
   7   * Instead of multiple service names, multiple commands will be
   8     supported by a single service
   9   * Easily extendable as capabilities are moved into their own section
  10     of the protocol, no longer being hidden behind a NUL byte and
  11     limited by the size of a pkt-line
  12   * Separate out other information hidden behind NUL bytes (e.g. agent
  13     string as a capability and symrefs can be requested using 'ls-refs')
  14   * Reference advertisement will be omitted unless explicitly requested
  15   * ls-refs command to explicitly request some refs
  16   * Designed with http and stateless-rpc in mind.  With clear flush
  17     semantics the http remote helper can simply act as a proxy
  18
  19 In protocol v2 communication is command oriented.  When first contacting a
  20 server a list of capabilities will advertised.  Some of these capabilities
  21 will be commands which a client can request be executed.  Once a command
  22 has completed, a client can reuse the connection and request that other
  23 commands be executed.
  24
  25  Packet-Line Framing
  26 ---------------------
  27
  28 All communication is done using packet-line framing, just as in v1.  See
  29 `Documentation/technical/pack-protocol.txt` and
  30 `Documentation/technical/protocol-common.txt` for more information.
  31
  32 In protocol v2 these special packets will have the following semantics:
  33
  34   * '0000' Flush Packet (flush-pkt) - indicates the end of a message
  35   * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
  36
  37  Initial Client Request
  38 ------------------------
  39
  40 In general a client can request to speak protocol v2 by sending
  41 `version=2` through the respective side-channel for the transport being
  42 used which inevitably sets `GIT_PROTOCOL`.  More information can be
  43 found in `pack-protocol.txt` and `http-protocol.txt`.  In all cases the
  44 response from the server is the capability advertisement.
  45
  46  Git Transport
  47 ~~~~~~~~~~~~~~~
  48
  49 When using the git:// transport, you can request to use protocol v2 by
  50 sending "version=2" as an extra parameter:
  51
  52    003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
  53
  54  SSH and File Transport
  55 ~~~~~~~~~~~~~~~~~~~~~~~~
  56
  57 When using either the ssh:// or file:// transport, the GIT_PROTOCOL
  58 environment variable must be set explicitly to include "version=2".
  59
  60  HTTP Transport
  61 ~~~~~~~~~~~~~~~~
  62
  63 When using the http:// or https:// transport a client makes a "smart"
  64 info/refs request as described in `http-protocol.txt` and requests that
  65 v2 be used by supplying "version=2" in the `Git-Protocol` header.
  66
  67    C: Git-Protocol: version=2
  68    C:
  69    C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
  70
  71 A v2 server would reply:
  72
  73    S: 200 OK
  74    S: <Some headers>
  75    S: ...
  76    S:
  77    S: 000eversion 2\n
  78    S: <capability-advertisement>
  79
  80 Subsequent requests are then made directly to the service
  81 `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
  82
  83  Capability Advertisement
  84 --------------------------
  85
  86 A server which decides to communicate (based on a request from a client)
  87 using protocol version 2, notifies the client by sending a version string
  88 in its initial response followed by an advertisement of its capabilities.
  89 Each capability is a key with an optional value.  Clients must ignore all
  90 unknown keys.  Semantics of unknown values are left to the definition of
  91 each key.  Some capabilities will describe commands which can be requested
  92 to be executed by the client.
  93
  94     capability-advertisement = protocol-version
  95                                capability-list
  96                                flush-pkt
  97
  98     protocol-version = PKT-LINE("version 2" LF)
  99     capability-list = *capability
 100     capability = PKT-LINE(key[=value] LF)
 101
 102     key = 1*(ALPHA | DIGIT | "-_")
 103     value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
 104
 105  Command Request
 106 -----------------
 107
 108 After receiving the capability advertisement, a client can then issue a
 109 request to select the command it wants with any particular capabilities
 110 or arguments.  There is then an optional section where the client can
 111 provide any command specific parameters or queries.  Only a single
 112 command can be requested at a time.
 113
 114     request = empty-request | command-request
 115     empty-request = flush-pkt
 116     command-request = command
 117                       capability-list
 118                       [command-args]
 119                       flush-pkt
 120     command = PKT-LINE("command=" key LF)
 121     command-args = delim-pkt
 122                    *command-specific-arg
 123
 124     command-specific-args are packet line framed arguments defined by
 125     each individual command.
 126
 127 The server will then check to ensure that the client's request is
 128 comprised of a valid command as well as valid capabilities which were
 129 advertised.  If the request is valid the server will then execute the
 130 command.  A server MUST wait till it has received the client's entire
 131 request before issuing a response.  The format of the response is
 132 determined by the command being executed, but in all cases a flush-pkt
 133 indicates the end of the response.
 134
 135 When a command has finished, and the client has received the entire
 136 response from the server, a client can either request that another
 137 command be executed or can terminate the connection.  A client may
 138 optionally send an empty request consisting of just a flush-pkt to
 139 indicate that no more requests will be made.
 140
 141  Capabilities
 142 --------------
 143
 144 There are two different types of capabilities: normal capabilities,
 145 which can be used to to convey information or alter the behavior of a
 146 request, and commands, which are the core actions that a client wants to
 147 perform (fetch, push, etc).
 148
 149 Protocol version 2 is stateless by default.  This means that all commands
 150 must only last a single round and be stateless from the perspective of the
 151 server side, unless the client has requested a capability indicating that
 152 state should be maintained by the server.  Clients MUST NOT require state
 153 management on the server side in order to function correctly.  This
 154 permits simple round-robin load-balancing on the server side, without
 155 needing to worry about state management.
 156
 157  agent
 158 ~~~~~~~
 159
 160 The server can advertise the `agent` capability with a value `X` (in the
 161 form `agent=X`) to notify the client that the server is running version
 162 `X`.  The client may optionally send its own agent string by including
 163 the `agent` capability with a value `Y` (in the form `agent=Y`) in its
 164 request to the server (but it MUST NOT do so if the server did not
 165 advertise the agent capability). The `X` and `Y` strings may contain any
 166 printable ASCII characters except space (i.e., the byte range 32 < x <
 167 127), and are typically of the form "package/version" (e.g.,
 168 "git/1.8.3.1"). The agent strings are purely informative for statistics
 169 and debugging purposes, and MUST NOT be used to programmatically assume
 170 the presence or absence of particular features.
 171
 172  ls-refs
 173 ~~~~~~~~~
 174
 175 `ls-refs` is the command used to request a reference advertisement in v2.
 176 Unlike the current reference advertisement, ls-refs takes in arguments
 177 which can be used to limit the refs sent from the server.
 178
 179 Additional features not supported in the base command will be advertised
 180 as the value of the command in the capability advertisement in the form
 181 of a space separated list of features: "<command>=<feature 1> <feature 2>"
 182
 183 ls-refs takes in the following arguments:
 184
 185     symrefs
 186         In addition to the object pointed by it, show the underlying ref
 187         pointed by it when showing a symbolic ref.
 188     peel
 189         Show peeled tags.
 190     ref-prefix <prefix>
 191         When specified, only references having a prefix matching one of
 192         the provided prefixes are displayed.
 193
 194 The output of ls-refs is as follows:
 195
 196     output = *ref
 197              flush-pkt
 198     ref = PKT-LINE(obj-id SP refname *(SP ref-attribute) LF)
 199     ref-attribute = (symref | peeled)
 200     symref = "symref-target:" symref-target
 201     peeled = "peeled:" obj-id
 202
 203  fetch
 204 ~~~~~~~
 205
 206 `fetch` is the command used to fetch a packfile in v2.  It can be looked
 207 at as a modified version of the v1 fetch where the ref-advertisement is
 208 stripped out (since the `ls-refs` command fills that role) and the
 209 message format is tweaked to eliminate redundancies and permit easy
 210 addition of future extensions.
 211
 212 Additional features not supported in the base command will be advertised
 213 as the value of the command in the capability advertisement in the form
 214 of a space separated list of features: "<command>=<feature 1> <feature 2>"
 215
 216 A `fetch` request can take the following arguments:
 217
 218     want <oid>
 219         Indicates to the server an object which the client wants to
 220         retrieve.  Wants can be anything and are not limited to
 221         advertised objects.
 222
 223     have <oid>
 224         Indicates to the server an object which the client has locally.
 225         This allows the server to make a packfile which only contains
 226         the objects that the client needs. Multiple 'have' lines can be
 227         supplied.
 228
 229     done
 230         Indicates to the server that negotiation should terminate (or
 231         not even begin if performing a clone) and that the server should
 232         use the information supplied in the request to construct the
 233         packfile.
 234
 235     thin-pack
 236         Request that a thin pack be sent, which is a pack with deltas
 237         which reference base objects not contained within the pack (but
 238         are known to exist at the receiving end). This can reduce the
 239         network traffic significantly, but it requires the receiving end
 240         to know how to "thicken" these packs by adding the missing bases
 241         to the pack.
 242
 243     no-progress
 244         Request that progress information that would normally be sent on
 245         side-band channel 2, during the packfile transfer, should not be
 246         sent.  However, the side-band channel 3 is still used for error
 247         responses.
 248
 249     include-tag
 250         Request that annotated tags should be sent if the objects they
 251         point to are being sent.
 252
 253     ofs-delta
 254         Indicate that the client understands PACKv2 with delta referring
 255         to its base by position in pack rather than by an oid.  That is,
 256         they can read OBJ_OFS_DELTA (ake type 6) in a packfile.
 257
 258 If the 'shallow' feature is advertised the following arguments can be
 259 included in the clients request as well as the potential addition of the
 260 'shallow-info' section in the server's response as explained below.
 261
 262     shallow <oid>
 263         A client must notify the server of all commits for which it only
 264         has shallow copies (meaning that it doesn't have the parents of
 265         a commit) by supplying a 'shallow <oid>' line for each such
 266         object so that the server is aware of the limitations of the
 267         client's history.  This is so that the server is aware that the
 268         client may not have all objects reachable from such commits.
 269
 270     deepen <depth>
 271         Requests that the fetch/clone should be shallow having a commit
 272         depth of <depth> relative to the remote side.
 273
 274     deepen-relative
 275         Requests that the semantics of the "deepen" command be changed
 276         to indicate that the depth requested is relative to the client's
 277         current shallow boundary, instead of relative to the requested
 278         commits.
 279
 280     deepen-since <timestamp>
 281         Requests that the shallow clone/fetch should be cut at a
 282         specific time, instead of depth.  Internally it's equivalent to
 283         doing "git rev-list --max-age=<timestamp>". Cannot be used with
 284         "deepen".
 285
 286     deepen-not <rev>
 287         Requests that the shallow clone/fetch should be cut at a
 288         specific revision specified by '<rev>', instead of a depth.
 289         Internally it's equivalent of doing "git rev-list --not <rev>".
 290         Cannot be used with "deepen", but can be used with
 291         "deepen-since".
 292
 293 If the 'filter' feature is advertised, the following argument can be
 294 included in the client's request:
 295
 296     filter <filter-spec>
 297         Request that various objects from the packfile be omitted
 298         using one of several filtering techniques. These are intended
 299         for use with partial clone and partial fetch operations. See
 300         `rev-list` for possible "filter-spec" values.
 301
 302 The response of `fetch` is broken into a number of sections separated by
 303 delimiter packets (0001), with each section beginning with its section
 304 header.
 305
 306     output = *section
 307     section = (acknowledgments | shallow-info | packfile)
 308               (flush-pkt | delim-pkt)
 309
 310     acknowledgments = PKT-LINE("acknowledgments" LF)
 311                       (nak | *ack)
 312                       (ready)
 313     ready = PKT-LINE("ready" LF)
 314     nak = PKT-LINE("NAK" LF)
 315     ack = PKT-LINE("ACK" SP obj-id LF)
 316
 317     shallow-info = PKT-LINE("shallow-info" LF)
 318                    *PKT-LINE((shallow | unshallow) LF)
 319     shallow = "shallow" SP obj-id
 320     unshallow = "unshallow" SP obj-id
 321
 322     packfile = PKT-LINE("packfile" LF)
 323                *PKT-LINE(%x01-03 *%x00-ff)
 324
 325     acknowledgments section
 326         * If the client determines that it is finished with negotiations
 327           by sending a "done" line, the acknowledgments sections MUST be
 328           omitted from the server's response.
 329
 330         * Always begins with the section header "acknowledgments"
 331
 332         * The server will respond with "NAK" if none of the object ids sent
 333           as have lines were common.
 334
 335         * The server will respond with "ACK obj-id" for all of the
 336           object ids sent as have lines which are common.
 337
 338         * A response cannot have both "ACK" lines as well as a "NAK"
 339           line.
 340
 341         * The server will respond with a "ready" line indicating that
 342           the server has found an acceptable common base and is ready to
 343           make and send a packfile (which will be found in the packfile
 344           section of the same response)
 345
 346         * If the server has found a suitable cut point and has decided
 347           to send a "ready" line, then the server can decide to (as an
 348           optimization) omit any "ACK" lines it would have sent during
 349           its response.  This is because the server will have already
 350           determined the objects it plans to send to the client and no
 351           further negotiation is needed.
 352
 353     shallow-info section
 354         * If the client has requested a shallow fetch/clone, a shallow
 355           client requests a fetch or the server is shallow then the
 356           server's response may include a shallow-info section.  The
 357           shallow-info section will be included if (due to one of the
 358           above conditions) the server needs to inform the client of any
 359           shallow boundaries or adjustments to the clients already
 360           existing shallow boundaries.
 361
 362         * Always begins with the section header "shallow-info"
 363
 364         * If a positive depth is requested, the server will compute the
 365           set of commits which are no deeper than the desired depth.
 366
 367         * The server sends a "shallow obj-id" line for each commit whose
 368           parents will not be sent in the following packfile.
 369
 370         * The server sends an "unshallow obj-id" line for each commit
 371           which the client has indicated is shallow, but is no longer
 372           shallow as a result of the fetch (due to its parents being
 373           sent in the following packfile).
 374
 375         * The server MUST NOT send any "unshallow" lines for anything
 376           which the client has not indicated was shallow as a part of
 377           its request.
 378
 379         * This section is only included if a packfile section is also
 380           included in the response.
 381
 382     packfile section
 383         * This section is only included if the client has sent 'want'
 384           lines in its request and either requested that no more
 385           negotiation be done by sending 'done' or if the server has
 386           decided it has found a sufficient cut point to produce a
 387           packfile.
 388
 389         * Always begins with the section header "packfile"
 390
 391         * The transmission of the packfile begins immediately after the
 392           section header
 393
 394         * The data transfer of the packfile is always multiplexed, using
 395           the same semantics of the 'side-band-64k' capability from
 396           protocol version 1.  This means that each packet, during the
 397           packfile data stream, is made up of a leading 4-byte pkt-line
 398           length (typical of the pkt-line format), followed by a 1-byte
 399           stream code, followed by the actual data.
 400
 401           The stream code can be one of:
 402                 1 - pack data
 403                 2 - progress messages
 404                 3 - fatal error message just before stream aborts
 405
 406  server-option
 407 ~~~~~~~~~~~~~~~
 408
 409 If advertised, indicates that any number of server specific options can be
 410 included in a request.  This is done by sending each option as a
 411 "server-option=<option>" capability line in the capability-list section of
 412 a request.
 413
 414 The provided options must not contain a NUL or LF character.