Documentation/technical/protocol-v2.txt

   1 Git Wire Protocol, Version 2
   2 ============================
   3
   4 This document presents a specification for a version 2 of Git's wire
   5 protocol.  Protocol v2 will improve upon v1 in the following ways:
   6
   7   * Instead of multiple service names, multiple commands will be
   8     supported by a single service
   9   * Easily extendable as capabilities are moved into their own section
  10     of the protocol, no longer being hidden behind a NUL byte and
  11     limited by the size of a pkt-line
  12   * Separate out other information hidden behind NUL bytes (e.g. agent
  13     string as a capability and symrefs can be requested using 'ls-refs')
  14   * Reference advertisement will be omitted unless explicitly requested
  15   * ls-refs command to explicitly request some refs
  16   * Designed with http and stateless-rpc in mind.  With clear flush
  17     semantics the http remote helper can simply act as a proxy
  18
  19 In protocol v2 communication is command oriented.  When first contacting a
  20 server a list of capabilities will advertised.  Some of these capabilities
  21 will be commands which a client can request be executed.  Once a command
  22 has completed, a client can reuse the connection and request that other
  23 commands be executed.
  24
  25 Packet-Line Framing
  26 -------------------
  27
  28 All communication is done using packet-line framing, just as in v1.  See
  29 `Documentation/technical/pack-protocol.txt` and
  30 `Documentation/technical/protocol-common.txt` for more information.
  31
  32 In protocol v2 these special packets will have the following semantics:
  33
  34   * '0000' Flush Packet (flush-pkt) - indicates the end of a message
  35   * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
  36
  37 Initial Client Request
  38 ----------------------
  39
  40 In general a client can request to speak protocol v2 by sending
  41 `version=2` through the respective side-channel for the transport being
  42 used which inevitably sets `GIT_PROTOCOL`.  More information can be
  43 found in `pack-protocol.txt` and `http-protocol.txt`.  In all cases the
  44 response from the server is the capability advertisement.
  45
  46 Git Transport
  47 ~~~~~~~~~~~~~
  48
  49 When using the git:// transport, you can request to use protocol v2 by
  50 sending "version=2" as an extra parameter:
  51
  52    003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
  53
  54 SSH and File Transport
  55 ~~~~~~~~~~~~~~~~~~~~~~
  56
  57 When using either the ssh:// or file:// transport, the GIT_PROTOCOL
  58 environment variable must be set explicitly to include "version=2".
  59
  60 HTTP Transport
  61 ~~~~~~~~~~~~~~
  62
  63 When using the http:// or https:// transport a client makes a "smart"
  64 info/refs request as described in `http-protocol.txt` and requests that
  65 v2 be used by supplying "version=2" in the `Git-Protocol` header.
  66
  67    C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
  68    C: Git-Protocol: version=2
  69
  70 A v2 server would reply:
  71
  72    S: 200 OK
  73    S: <Some headers>
  74    S: ...
  75    S:
  76    S: 000eversion 2\n
  77    S: <capability-advertisement>
  78
  79 Subsequent requests are then made directly to the service
  80 `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
  81
  82 Capability Advertisement
  83 ------------------------
  84
  85 A server which decides to communicate (based on a request from a client)
  86 using protocol version 2, notifies the client by sending a version string
  87 in its initial response followed by an advertisement of its capabilities.
  88 Each capability is a key with an optional value.  Clients must ignore all
  89 unknown keys.  Semantics of unknown values are left to the definition of
  90 each key.  Some capabilities will describe commands which can be requested
  91 to be executed by the client.
  92
  93     capability-advertisement = protocol-version
  94                                capability-list
  95                                flush-pkt
  96
  97     protocol-version = PKT-LINE("version 2" LF)
  98     capability-list = *capability
  99     capability = PKT-LINE(key[=value] LF)
 100
 101     key = 1*(ALPHA | DIGIT | "-_")
 102     value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
 103
 104 Command Request
 105 ---------------
 106
 107 After receiving the capability advertisement, a client can then issue a
 108 request to select the command it wants with any particular capabilities
 109 or arguments.  There is then an optional section where the client can
 110 provide any command specific parameters or queries.  Only a single
 111 command can be requested at a time.
 112
 113     request = empty-request | command-request
 114     empty-request = flush-pkt
 115     command-request = command
 116                       capability-list
 117                       [command-args]
 118                       flush-pkt
 119     command = PKT-LINE("command=" key LF)
 120     command-args = delim-pkt
 121                    *command-specific-arg
 122
 123     command-specific-args are packet line framed arguments defined by
 124     each individual command.
 125
 126 The server will then check to ensure that the client's request is
 127 comprised of a valid command as well as valid capabilities which were
 128 advertised.  If the request is valid the server will then execute the
 129 command.  A server MUST wait till it has received the client's entire
 130 request before issuing a response.  The format of the response is
 131 determined by the command being executed, but in all cases a flush-pkt
 132 indicates the end of the response.
 133
 134 When a command has finished, and the client has received the entire
 135 response from the server, a client can either request that another
 136 command be executed or can terminate the connection.  A client may
 137 optionally send an empty request consisting of just a flush-pkt to
 138 indicate that no more requests will be made.
 139
 140 Capabilities
 141 ------------
 142
 143 There are two different types of capabilities: normal capabilities,
 144 which can be used to to convey information or alter the behavior of a
 145 request, and commands, which are the core actions that a client wants to
 146 perform (fetch, push, etc).
 147
 148 Protocol version 2 is stateless by default.  This means that all commands
 149 must only last a single round and be stateless from the perspective of the
 150 server side, unless the client has requested a capability indicating that
 151 state should be maintained by the server.  Clients MUST NOT require state
 152 management on the server side in order to function correctly.  This
 153 permits simple round-robin load-balancing on the server side, without
 154 needing to worry about state management.
 155
 156 agent
 157 ~~~~~
 158
 159 The server can advertise the `agent` capability with a value `X` (in the
 160 form `agent=X`) to notify the client that the server is running version
 161 `X`.  The client may optionally send its own agent string by including
 162 the `agent` capability with a value `Y` (in the form `agent=Y`) in its
 163 request to the server (but it MUST NOT do so if the server did not
 164 advertise the agent capability). The `X` and `Y` strings may contain any
 165 printable ASCII characters except space (i.e., the byte range 32 < x <
 166 127), and are typically of the form "package/version" (e.g.,
 167 "git/1.8.3.1"). The agent strings are purely informative for statistics
 168 and debugging purposes, and MUST NOT be used to programmatically assume
 169 the presence or absence of particular features.
 170
 171 ls-refs
 172 ~~~~~~~
 173
 174 `ls-refs` is the command used to request a reference advertisement in v2.
 175 Unlike the current reference advertisement, ls-refs takes in arguments
 176 which can be used to limit the refs sent from the server.
 177
 178 Additional features not supported in the base command will be advertised
 179 as the value of the command in the capability advertisement in the form
 180 of a space separated list of features: "<command>=<feature 1> <feature 2>"
 181
 182 ls-refs takes in the following arguments:
 183
 184     symrefs
 185         In addition to the object pointed by it, show the underlying ref
 186         pointed by it when showing a symbolic ref.
 187     peel
 188         Show peeled tags.
 189     ref-prefix <prefix>
 190         When specified, only references having a prefix matching one of
 191         the provided prefixes are displayed.
 192
 193 The output of ls-refs is as follows:
 194
 195     output = *ref
 196              flush-pkt
 197     ref = PKT-LINE(obj-id SP refname *(SP ref-attribute) LF)
 198     ref-attribute = (symref | peeled)
 199     symref = "symref-target:" symref-target
 200     peeled = "peeled:" obj-id
 201
 202 fetch
 203 ~~~~~
 204
 205 `fetch` is the command used to fetch a packfile in v2.  It can be looked
 206 at as a modified version of the v1 fetch where the ref-advertisement is
 207 stripped out (since the `ls-refs` command fills that role) and the
 208 message format is tweaked to eliminate redundancies and permit easy
 209 addition of future extensions.
 210
 211 Additional features not supported in the base command will be advertised
 212 as the value of the command in the capability advertisement in the form
 213 of a space separated list of features: "<command>=<feature 1> <feature 2>"
 214
 215 A `fetch` request can take the following arguments:
 216
 217     want <oid>
 218         Indicates to the server an object which the client wants to
 219         retrieve.  Wants can be anything and are not limited to
 220         advertised objects.
 221
 222     have <oid>
 223         Indicates to the server an object which the client has locally.
 224         This allows the server to make a packfile which only contains
 225         the objects that the client needs. Multiple 'have' lines can be
 226         supplied.
 227
 228     done
 229         Indicates to the server that negotiation should terminate (or
 230         not even begin if performing a clone) and that the server should
 231         use the information supplied in the request to construct the
 232         packfile.
 233
 234     thin-pack
 235         Request that a thin pack be sent, which is a pack with deltas
 236         which reference base objects not contained within the pack (but
 237         are known to exist at the receiving end). This can reduce the
 238         network traffic significantly, but it requires the receiving end
 239         to know how to "thicken" these packs by adding the missing bases
 240         to the pack.
 241
 242     no-progress
 243         Request that progress information that would normally be sent on
 244         side-band channel 2, during the packfile transfer, should not be
 245         sent.  However, the side-band channel 3 is still used for error
 246         responses.
 247
 248     include-tag
 249         Request that annotated tags should be sent if the objects they
 250         point to are being sent.
 251
 252     ofs-delta
 253         Indicate that the client understands PACKv2 with delta referring
 254         to its base by position in pack rather than by an oid.  That is,
 255         they can read OBJ_OFS_DELTA (ake type 6) in a packfile.
 256
 257 If the 'shallow' feature is advertised the following arguments can be
 258 included in the clients request as well as the potential addition of the
 259 'shallow-info' section in the server's response as explained below.
 260
 261     shallow <oid>
 262         A client must notify the server of all commits for which it only
 263         has shallow copies (meaning that it doesn't have the parents of
 264         a commit) by supplying a 'shallow <oid>' line for each such
 265         object so that the server is aware of the limitations of the
 266         client's history.  This is so that the server is aware that the
 267         client may not have all objects reachable from such commits.
 268
 269     deepen <depth>
 270         Requests that the fetch/clone should be shallow having a commit
 271         depth of <depth> relative to the remote side.
 272
 273     deepen-relative
 274         Requests that the semantics of the "deepen" command be changed
 275         to indicate that the depth requested is relative to the client's
 276         current shallow boundary, instead of relative to the requested
 277         commits.
 278
 279     deepen-since <timestamp>
 280         Requests that the shallow clone/fetch should be cut at a
 281         specific time, instead of depth.  Internally it's equivalent to
 282         doing "git rev-list --max-age=<timestamp>". Cannot be used with
 283         "deepen".
 284
 285     deepen-not <rev>
 286         Requests that the shallow clone/fetch should be cut at a
 287         specific revision specified by '<rev>', instead of a depth.
 288         Internally it's equivalent of doing "git rev-list --not <rev>".
 289         Cannot be used with "deepen", but can be used with
 290         "deepen-since".
 291
 292 If the 'filter' feature is advertised, the following argument can be
 293 included in the client's request:
 294
 295     filter <filter-spec>
 296         Request that various objects from the packfile be omitted
 297         using one of several filtering techniques. These are intended
 298         for use with partial clone and partial fetch operations. See
 299         `rev-list` for possible "filter-spec" values. When communicating
 300         with other processes, senders SHOULD translate scaled integers
 301         (e.g. "1k") into a fully-expanded form (e.g. "1024") to aid
 302         interoperability with older receivers that may not understand
 303         newly-invented scaling suffixes. However, receivers SHOULD
 304         accept the following suffixes: 'k', 'm', and 'g' for 1024,
 305         1048576, and 1073741824, respectively.
 306
 307 If the 'ref-in-want' feature is advertised, the following argument can
 308 be included in the client's request as well as the potential addition of
 309 the 'wanted-refs' section in the server's response as explained below.
 310
 311     want-ref <ref>
 312         Indicates to the server that the client wants to retrieve a
 313         particular ref, where <ref> is the full name of a ref on the
 314         server.
 315
 316 If the 'sideband-all' feature is advertised, the following argument can be
 317 included in the client's request:
 318
 319     sideband-all
 320         Instruct the server to send the whole response multiplexed, not just
 321         the packfile section. All non-flush and non-delim PKT-LINE in the
 322         response (not only in the packfile section) will then start with a byte
 323         indicating its sideband (1, 2, or 3), and the server may send "0005\2"
 324         (a PKT-LINE of sideband 2 with no payload) as a keepalive packet.
 325
 326 The response of `fetch` is broken into a number of sections separated by
 327 delimiter packets (0001), with each section beginning with its section
 328 header.
 329
 330     output = *section
 331     section = (acknowledgments | shallow-info | wanted-refs | packfile)
 332               (flush-pkt | delim-pkt)
 333
 334     acknowledgments = PKT-LINE("acknowledgments" LF)
 335                       (nak | *ack)
 336                       (ready)
 337     ready = PKT-LINE("ready" LF)
 338     nak = PKT-LINE("NAK" LF)
 339     ack = PKT-LINE("ACK" SP obj-id LF)
 340
 341     shallow-info = PKT-LINE("shallow-info" LF)
 342                    *PKT-LINE((shallow | unshallow) LF)
 343     shallow = "shallow" SP obj-id
 344     unshallow = "unshallow" SP obj-id
 345
 346     wanted-refs = PKT-LINE("wanted-refs" LF)
 347                   *PKT-LINE(wanted-ref LF)
 348     wanted-ref = obj-id SP refname
 349
 350     packfile = PKT-LINE("packfile" LF)
 351                *PKT-LINE(%x01-03 *%x00-ff)
 352
 353     acknowledgments section
 354         * If the client determines that it is finished with negotiations
 355           by sending a "done" line, the acknowledgments sections MUST be
 356           omitted from the server's response.
 357
 358         * Always begins with the section header "acknowledgments"
 359
 360         * The server will respond with "NAK" if none of the object ids sent
 361           as have lines were common.
 362
 363         * The server will respond with "ACK obj-id" for all of the
 364           object ids sent as have lines which are common.
 365
 366         * A response cannot have both "ACK" lines as well as a "NAK"
 367           line.
 368
 369         * The server will respond with a "ready" line indicating that
 370           the server has found an acceptable common base and is ready to
 371           make and send a packfile (which will be found in the packfile
 372           section of the same response)
 373
 374         * If the server has found a suitable cut point and has decided
 375           to send a "ready" line, then the server can decide to (as an
 376           optimization) omit any "ACK" lines it would have sent during
 377           its response.  This is because the server will have already
 378           determined the objects it plans to send to the client and no
 379           further negotiation is needed.
 380
 381     shallow-info section
 382         * If the client has requested a shallow fetch/clone, a shallow
 383           client requests a fetch or the server is shallow then the
 384           server's response may include a shallow-info section.  The
 385           shallow-info section will be included if (due to one of the
 386           above conditions) the server needs to inform the client of any
 387           shallow boundaries or adjustments to the clients already
 388           existing shallow boundaries.
 389
 390         * Always begins with the section header "shallow-info"
 391
 392         * If a positive depth is requested, the server will compute the
 393           set of commits which are no deeper than the desired depth.
 394
 395         * The server sends a "shallow obj-id" line for each commit whose
 396           parents will not be sent in the following packfile.
 397
 398         * The server sends an "unshallow obj-id" line for each commit
 399           which the client has indicated is shallow, but is no longer
 400           shallow as a result of the fetch (due to its parents being
 401           sent in the following packfile).
 402
 403         * The server MUST NOT send any "unshallow" lines for anything
 404           which the client has not indicated was shallow as a part of
 405           its request.
 406
 407         * This section is only included if a packfile section is also
 408           included in the response.
 409
 410     wanted-refs section
 411         * This section is only included if the client has requested a
 412           ref using a 'want-ref' line and if a packfile section is also
 413           included in the response.
 414
 415         * Always begins with the section header "wanted-refs".
 416
 417         * The server will send a ref listing ("<oid> <refname>") for
 418           each reference requested using 'want-ref' lines.
 419
 420         * The server MUST NOT send any refs which were not requested
 421           using 'want-ref' lines.
 422
 423     packfile section
 424         * This section is only included if the client has sent 'want'
 425           lines in its request and either requested that no more
 426           negotiation be done by sending 'done' or if the server has
 427           decided it has found a sufficient cut point to produce a
 428           packfile.
 429
 430         * Always begins with the section header "packfile"
 431
 432         * The transmission of the packfile begins immediately after the
 433           section header
 434
 435         * The data transfer of the packfile is always multiplexed, using
 436           the same semantics of the 'side-band-64k' capability from
 437           protocol version 1.  This means that each packet, during the
 438           packfile data stream, is made up of a leading 4-byte pkt-line
 439           length (typical of the pkt-line format), followed by a 1-byte
 440           stream code, followed by the actual data.
 441
 442           The stream code can be one of:
 443                 1 - pack data
 444                 2 - progress messages
 445                 3 - fatal error message just before stream aborts
 446
 447 server-option
 448 ~~~~~~~~~~~~~
 449
 450 If advertised, indicates that any number of server specific options can be
 451 included in a request.  This is done by sending each option as a
 452 "server-option=<option>" capability line in the capability-list section of
 453 a request.
 454
 455 The provided options must not contain a NUL or LF character.