The Fossil RFCs
The Fossil Delta Format
draft-fossil-delta-format-00 · an illustrated reading

The Fossil Delta Format

Fossil stores many versions of a file as a delta: a short recipe that rebuilds one byte sequence from another it resembles. Here is a real one, from the draft's own worked example. These 79 bytes rebuild a 151-byte file.

The delta, byte by byte. Point at a segment to read it.source 102 B · delta 79 B · target 151 B
Header, four segments, and a trailer. Nothing else.
header and trailer copy from the source literal bytes carried in the delta

§1–2What a delta is

Over a project's life, most versions of a file differ from the previous one by a few lines. Storing every version whole would waste space, so Fossil stores many of them as deltas. A delta encodes one byte sequence, the target, as instructions for rebuilding it from another, the source.

There are two instructions: copy a run of bytes from the source, and insert a run of bytes carried in the delta itself. Where the target shares long runs with the source, the copies make the delta much smaller than the target.

A delta does not name its source. The caller supplies the source, and the storage or transfer layer records which source a delta belongs to. Naming stored content and verifying its integrity belongs to the Fossil Artifact Format. This document covers only the delta's bytes and how to apply them.

TermMeaning
SourceThe bytes a delta is applied against.
TargetThe bytes a delta rebuilds. The delta's header declares its length.
DeltaThe bytes that say how to rebuild the target from the source.
SegmentOne instruction in a delta: a copy or a literal.

The draft uses MUST, SHOULD and MAY in the BCP 14 sense, and only when capitalised.

§3The shape of a delta

A delta is a header, zero or more segments, and a trailer, written back to back with no separators beyond the bytes each part defines. Every number in it uses the same base-64 integer encoding.

header    integer LF                 target length, then 0x0a
copy      count "@" offset ","       copy count source bytes from offset
literal   count ":" payload          insert the next count bytes of the delta
trailer   checksum ";"               32-bit sum of the target

The opcode that follows an integer says what the integer was: @ starts a copy, : a literal, ; the trailer. A reader never needs to look ahead further than one byte.

§3.1Integers in base 64

An integer is an unsigned number written in base 64, most significant digit first, with no leading zeros. Zero is the single character 0. The 64 digits, in value order:

So 0–9 are 0–9, A–Z are 10–35, _ is 36, a–z are 37–62 and ~ is 63. A reader takes digits until the first byte that is not one; that byte is the opcode or separator. Bytes 0x80–0xFF are never digits and always end an integer.

Convert an integerType a decimal number or a base-64 string

Producers MUST use the canonical form, and every value they emit MUST be below 232. Decoders are allowed more slack. Fossil as shipped accepts an empty integer (read as zero) and leading zeros (read as their value). A decoder MAY reject either form, and implementations differ. Values at or above 232 have no interoperable meaning: a decoder MAY reject them, and one that accumulates in 32 bits will wrap where a wider one will not.

§3.2The header

The header is the target length as an integer, followed by the byte 0x0a. The decoder reads it first, so it knows how large the output will be before it reads any segment. A decoder MUST reject a delta whose header integer is not immediately followed by 0x0a. In the example, 2N is 2 × 64 + 23 = 151.

§3.3Copy and literal segments

Copy is count@offset,. It copies count bytes from the source, starting at the zero-based offset, onto the end of the output. The comma is required. A count of zero copies zero bytes, and encoders MUST NOT emit one. Whether the run lies inside the source is checked when the delta is applied.

Literal is count:payload. The next count bytes of the delta go straight to the output. The payload is opaque and may contain any byte, including 0x0a. A reader MUST count exactly count bytes past the colon and MUST NOT scan for a separator. The only structural 0x0a in a delta is the one that ends the header.

§3.4The trailer and its checksum

The delta ends with checksum;. Producers MUST always write the checksum. Decoders MAY verify it, and a decoder that verifies MUST reject a mismatch. A decoder that does not verify still requires the trailer's syntax.

The checksum reads the target as big-endian 32-bit words and adds them, wrapping at 232. A tail of one to three bytes is added as if padded with zeros to a whole word: the first tail byte shifted left 24 bits, the second 16, the third 8.

margin = 0.15in
boxht = 0.5
boxwid = 0.55

# The last whole word, and the tail padded to a word.
W_A: box "20" mono bold
box "74" mono bold
box "61" mono bold
W_D: box "69" mono bold
move 0.8
T_A: box "6C" mono bold fill 0x0C0003
box "2E" mono bold fill 0x0C0003
box "0A" mono bold fill 0x0C0003
PAD: box "00" mono bold dashed

text "word 36" bold "“ tai”" small at ((W_A.w.x + W_D.e.x) / 2, W_A.n.y + 0.27)
text "tail" bold "“l.↵” and one zero" small at ((T_A.w.x + PAD.e.x) / 2, T_A.n.y + 0.27)
text "0x20746169" mono at ((W_A.w.x + W_D.e.x) / 2, W_A.s.y - 0.22)
text "0x6C2E0A00" mono at ((T_A.w.x + PAD.e.x) / 2, T_A.s.y - 0.22)
§3.4 · the short tail counts as a zero-padded wordEdit in pikchr-studio

For the 151-byte example target, the 37 whole words sum to 0x2CE563A1. The tail 6C 2E 0A adds 0x6C2E0A00, giving 0x99136DA1, which is 2568187297 or 2P4rrX in base 64: the trailer.

Checksum any textUTF-8 bytes, as the trailer would carry them

After the trailer the delta is complete. Bytes that follow it in the buffer MAY be ignored.

§4Applying a delta

The reader is given the source bytes and the delta bytes and works through the delta once, front to back:

  1. Reject a delta that is zero bytes long.
  2. Read the header integer as the target length. Reject unless the next byte is 0x0a. Allocate the output and set the position to zero.
  3. While delta bytes remain, read an integer and look at the byte after it.
    • @: read the offset and require a comma. Reject if offset + count exceeds the source, computed without overflow. Reject if the copy would run past the target length. Otherwise copy and advance.
    • :: reject if count exceeds the bytes left in the delta, or if the literal would run past the target length. Otherwise copy the payload and advance past it.
    • ;: reject unless the position equals the target length. If the decoder verifies, check the checksum. The output is the target.
    • Any other byte: reject.
  4. Reject a delta that ends without ;.

B.1Step through the worked example

The figure shows where each part of the target comes from. Two runs are copied from the source; the eight bytes lazy dog are never used. The other two runs arrive as literals inside the delta.

margin = 0.15in
boxht = 0.5
$gap = 0.5
$c1 = 1.15
$c2 = 1.45
$c3 = 1.1
$c4 = 2.15

# Target: one column per segment, in output order.
COPY_35: box "copy" bold "35 bytes" small wid $c1
LIT_12: box "literal" bold "12 bytes" small wid $c2 fill 0x0C0003
COPY_59: box "copy" bold "59 bytes" small wid $c3
LIT_45: box "literal" bold "45 bytes" small wid $c4 fill 0x0C0003

# Source above, each run over the column it fills.
SRC_HEAD: box "0–34" bold "35 bytes" small wid $c1 with .s at COPY_35.n + (0, $gap)
SRC_DOG: box "35–42" bold "unused" small wid $c2 dashed
SRC_TAIL: box "43–101" bold "59 bytes" small wid $c3

# Delta below, each instruction under the column it writes.
HDR: box "2N↵" mono bold "header" small wid 0.55 with .nw at COPY_35.sw + (0, -$gap)
OP_Z: box "Z@0," mono bold "copy" small wid $c1 - 0.55
OP_C: box "C:sleeping fox" mono bold "literal" small wid $c2 fill 0x0C0003
OP_W: box "w@g," mono bold "copy" small wid $c3
OP_I: box "i: An example…" mono bold "literal" small wid $c4 - 0.8 fill 0x0C0003
SUM: box "2P4rrX;" mono bold "trailer" small wid 0.8

text "source" small rjust at SRC_HEAD.w - (0.12, 0)
text "target" small rjust at COPY_35.w - (0.12, 0)
text "delta" small rjust at HDR.w - (0.12, 0)

arrow from SRC_HEAD.s to COPY_35.n
arrow from SRC_TAIL.s to COPY_59.n
arrow from (LIT_12.x, OP_C.n.y) to LIT_12.s
arrow from (LIT_45.x, OP_I.n.y) to LIT_45.s
B.1 · where each run of the target comes fromEdit in pikchr-studio
Decode it one segment at a time
Delta
Source
Output

§4When a decoder must refuse

A decoder MUST report failure for every condition below, beyond the zero-length delta of step 1. Whether it reports one code or one per condition is up to the implementation. Each Try it loads a delta that breaks that rule into the playground below.

#Condition

Empty integers, leading zeros and values at or above 232 are not on this list. A decoder that accepts them still applies every structural and bounds check to the value it uses. A decoder MUST NOT require or reject a particular segmentation, hash window, chain depth or copy strategy: those are encoder choices, not part of the format.

∗Playground

Edit the source and target to see the delta Fossil's encoder would write and the decoder applying it. You can also edit the delta directly. Write a line feed as \n and any other byte as \xNN.

§4.1Asking only for the size

A reader MAY learn the target length by reading just the header integer and its 0x0a, without looking at anything after it. It MUST fail on an empty buffer or a header not followed by 0x0a, and needs no further bytes to succeed. Fossil uses this to allocate the output before applying a delta.

§4.2When the source is missing

The decoder works on bytes already in memory. It cannot ask for a missing source or wait for one, so the caller MUST supply the whole source first and SHOULD check it is available. When a repository receives a delta whose source it does not hold, it records the dependency and rebuilds the target later, once the source arrives from a peer through the Fossil Sync Protocol.

§5How Fossil's encoder finds copies

Informative. The format allows any segmentation that rebuilds the target exactly.

Any sequence of copies and literals that reproduces the target is a valid delta. The draft leaves the search to the encoder. Fossil's own encoder, which the playground runs, works like this:

  1. Cut the source into 16-byte blocks and record each block's rolling hash in a table.
  2. Slide a 16-byte window along the target one byte at a time, updating its hash cheaply, and look the hash up.
  3. On a hit, compare the real bytes, then extend the match backward and forward as far as source and target agree.
  4. Keep the longest match whose copy costs fewer bytes than writing it out. Emit the bytes skipped since the last copy as a literal, then the copy, and continue after it.
margin = 0.15in
boxht = 0.5
boxwid = 1.1

# Source, cut into 16-byte blocks.
BLOCK_0: box "block 0" bold "bytes 0–15" small
BLOCK_1: box "block 1" bold "bytes 16–31" small
BLOCK_2: box "block 2" bold "bytes 32–47" small fill 0x0C0003
BLOCK_3: box "block 3" bold "bytes 48–63" small
BLOCK_4: box "…" bold

# Target on the same axis: the copy is centred under block 2.
COPY: box "" wid 2.3 with .n at BLOCK_2.s + (0, -0.7)
WINDOW: box "window" bold "16 bytes" small wid 1.1 ht 0.36 fill 0x0C0003 dashed at COPY
LITERAL: box "literal" bold "skipped bytes" small wid 1.1 with .e at COPY.w
UNREAD: box "unread" bold "rest of target" small wid 1.1 with .w at COPY.e

text "source" small rjust at BLOCK_0.w - (0.12, 0)
text "target" small rjust at LITERAL.w - (0.12, 0)

arrow from BLOCK_2.s to WINDOW.n
text "same hash, same bytes" small ljust at (BLOCK_2.x + 0.12, (BLOCK_2.s.y + WINDOW.n.y) / 2)
arrow from WINDOW.w to COPY.w
arrow from WINDOW.e to COPY.e
text "copy: the match, extended both ways" small at (COPY.x, COPY.s.y - 0.22)
§5 · one step of Fossil’s encoderEdit in pikchr-studio

A copy is kept only when its length is at least the digits of the pending literal's length, the copy's count and its offset, plus three opcode bytes. Fossil follows at most 250 entries of a hash chain. When the source is 16 bytes or shorter no copy can win, and an encoder MAY write the whole target as one literal. None of these numbers is visible in the delta.

§6Security considerations

  • The checksum is a 32-bit sum. It catches accidental corruption but anyone can forge it. Integrity comes from the artifact's name, a cryptographic hash checked by the layer that stores it.
  • A parser MUST NOT read past the end of the delta, and integer reading MUST stop at the buffer's end. A zero-length delta is rejected before any read.
  • A failed application MUST NOT be reported as success. Callers MUST check the result and discard the output on failure.
  • Decoders SHOULD bound every segment by the declared target length rather than waiting for the trailer. The declared length is itself untrusted: constrained systems SHOULD check it against their own limits before allocating.
  • Bytes 0x80–0xFF MUST end an integer. A decoder that masks the high bit before looking up a digit accepts 64 extra digit bytes and parses crafted input differently from everyone else.

App. AThree corrections to older documentation

Older text saysThe format actually
Integers are written “LSB first”.Writes the most significant digit first. The old phrase describes the order digits are collected before the encoder reverses them.
The checksum is taken “modulo 232−1”.Is an unsigned 32-bit sum that wraps naturally, which is modulo 232.
A copy count of zero means “copy the rest of the input”.Copies zero bytes, and conforming encoders never write one.

App. BTwo small cases

B.4An empty target

The delta 0\n0; rebuilds zero bytes from any source: a header of zero, no segments, and the checksum of nothing, which is zero. Four bytes, 30 0a 30 3b.

B.2Values from the worked example

ValueEncodingDigits
000
12CC = 12
35ZZ = 35
43gg = 43
45ii = 45
59ww = 59
1512N2 × 64 + 23

App. CCollected grammar

ABNF per RFC 5234. Payloads are located by count, which ABNF cannot express, so the count governs. Where the grammar and the prose differ on what a decoder tolerates, the prose wins.

delta     = header *segment trailer
header    = integer LF
segment   = copy / literal
copy      = integer "@" integer ","
literal   = integer ":" payload
            ; <integer> opaque bytes; MAY contain LF;
            ; located by count, not by scanning
trailer   = integer ";"
integer   = 1*DIGIT64
DIGIT64   = %x30-39 / %x41-5A / "_" / %x61-7A / "~"
            ; value 0-9, A-Z=10-35, _=36, a-z=37-62, ~=63
payload   = *OCTET      ; exactly <count> octets
LF        = %x0A

A reading of The Fossil Delta Format, draft-fossil-delta-format-00 (Hipp, Beal and Shearer, July 2026). Section numbers refer to the draft, which remains the normative text.

The playground runs a JavaScript port of Fossil's delta_create. Its output matched fossil test-delta-create (fossil 2.28) byte for byte on 200 random text and binary pairs. The diagrams are Pikchr, drawn in your browser by Pikchr Studio's WebAssembly build.

draft-fossil-artifact-format-00 · an illustrated reading

The Fossil Artifact Format

Fossil keeps a project as a set of artifacts, each named by the hash of its own bytes. Some hold the contents of files. The rest are short text documents built from cards, and they record everything else: check-ins, tags, wiki pages, tickets. Here is a whole one, the root check-in from the draft's Appendix A.1.1.

A check-in manifest, card by card. Point at a token to read it.127 bytes · 6 cards · named by SHA-1
Six cards, one per line, in alphabetical order of their letters. The SHA-1 hash of these 127 bytes is the artifact's name.
card letter token checksum of the bytes above

§1–2What an artifact is

An artifact is an immutable byte sequence named by the cryptographic hash of its content. A repository's project state is the unordered set of artifacts it holds. The order they were added in does not matter, and two repositories holding the same set represent the same project state.

There are two kinds. A content artifact is any byte sequence, such as one version of a file. A structural artifact is a text document built from cards that records relationships among artifacts: a check-in, a wiki page, a ticket change, and five other types. Both kinds are named and stored the same way. A reader tells them apart by attempting a structural parse.

Artifacts are added but never changed or removed in normal operation. A name is a function of content, so an artifact received from any peer is identical to the same artifact held locally, and two repositories converge by exchanging the artifacts each lacks. The draft calls this a grow-only set. User accounts, settings and the map from names to local storage are local state, outside the artifact model. Content may be stored or sent as a delta, which changes neither its bytes nor its name.

TermMeaning
CardOne line of a structural artifact: a letter, then zero or more space-separated tokens.
TokenA maximal run of bytes in a card containing neither space (0x20) nor LF (0x0a).
Decoded formA token's value after its escapes are decoded (§4.3).
Stripped contentAn artifact's bytes once any signature wrapper is removed (§4.1.1).
ManifestThe structural artifact that records a check-in.
Baseline manifestA manifest with no B card. Its F cards list every file.
Delta manifestA manifest with a B card. Its F cards list only differences from the baseline the B card names (§8).

The draft defines these terms for the whole Fossil suite: the Delta Format, Sync Protocol and Repository State drafts reuse them without restating them. It uses MUST, SHOULD and MAY in the BCP 14 sense, and only when capitalised.

§3Naming an artifact

§3.1The hash of the stored bytes

An artifact's name is the hash of its complete raw bytes, taken as one blob. Nothing is added before hashing, and nothing is canonicalised, re-encoded or stripped: the bytes hashed MUST be exactly the stored bytes. A signed artifact's stored bytes include its signature wrapper, so its name covers the wrapper too.

When content arrives together with a name, the receiver's storage step uses that supplied name as the candidate rather than computing a new one. Checking that the name matches the content is a separate job for the caller (§3.4).

§3.2Two algorithms, told apart by length

A name is either 40 hexadecimal digits, a SHA-1 hash, or 64, a SHA3-256 hash. A candidate of any other length MUST be rejected, and the length check comes before any check of the characters. The lengths never overlap, so the length alone says which algorithm made a name. Validation, verification and display all dispatch on it, and there is no other algorithm marker.

SHA3-256 here is the FIPS 202 function, with the 01 domain-separation padding, which differs from the original Keccak submission. Both are written as lowercase hexadecimal. Which of the two names newly stored content is the hash policy's choice (§7).

§3.3Lowercase, compared exactly

Every name an implementation computes or emits MUST be lowercase. A token with an uppercase digit is therefore not a valid name and can never verify. A parser MAY reject it when it validates the token, or MAY accept it there and let it fail verification later. The two choices interoperate because conforming data never contains one. A parser that accepts uppercase MUST NOT lowercase it.

Name some bytesUTF-8 text, hashed both ways

§3.4Verifying a name

Verification picks the algorithm by length, hashes the content, and compares the candidate byte for byte with the digest written in lowercase. A 40-digit name is checked against SHA-1, a 64-digit name against SHA3-256, and any other length never verifies. The caller sees one failure signal: a wrong length, a non-hex character and a digest mismatch look the same. When a receiver is handed a name:

  1. It parses the token as a name and selects it as the candidate storage name.
  2. Normally it rebuilds the full bytes, checks them against the name, and only then stores the artifact. If the check fails, the artifact MUST NOT be treated as verified under that name. The receiver makes this check; the storage step does not.
  3. Some paths store first and check later: a clone, and a delta whose source has not arrived yet. Being stored under a name is never proof that the name was checked.
  4. Storage then uses the name to find duplicates and to decide whether the repository must switch to SHA3 names (§7.4).

§3.5Grammar

lhexdig        = DIGIT / %x61-66             ; 0-9 / a-f
hexdig-ci      = DIGIT / %x41-46 / %x61-66   ; 0-9 / A-F / a-f

artifact-name  = sha1-name / sha3-name        ; what writers MUST produce
sha1-name      = 40lhexdig
sha3-name      = 64lhexdig

accepted-name  = 40hexdig-ci / 64hexdig-ci    ; what a parser may tolerate

§3.6Looking up a prefix

Informative. Prefix lookup is a local convenience, not part of the format.

A lookup takes 4 to 64 hexadecimal characters and lowercases them, unlike the parse path. It then matches the prefix against all names without caring whether they are 40 or 64 digits long. If more than one artifact matches, the name is ambiguous, and a SHA-1 name colliding with a SHA3 name is reported the same way as two names of one kind.

§4Cards

A structural artifact is a sequence of cards. This section covers how a parser frames the bytes, removes a signature, splits lines and tokens, decodes escapes, enforces order and repetition, and checks each card's tokens.

§4.1Four gates before the first card

A parser MUST pass these gates, in this order, before it interprets any card:

  1. The input is not empty and its last byte is LF. This is checked on the raw bytes, so a signed artifact's outer wrapper must end in LF.
  2. Any signature wrapper is removed (§4.1.1). The stripped content also ends in LF, because the cut keeps the LF before the signature armor.
  3. Line 1 meets a stricter rule than other cards: the stripped content is at least 10 bytes long, its first byte is a letter A–Z and its second byte is a space. A first card with no arguments fails here, although argument-less cards are allowed later.
  4. The Z card checksum is verified (§6.2). A Z card that is present but wrong MUST cause rejection. A missing Z card passes this gate and is caught later by §5.2.

When the card loop ends, any bytes left over MUST cause rejection.

§4.1.1Removing a signature wrapper

A reader MUST remove both PGP clearsign wrappers and SSH signed wrappers before parsing:

  1. Stripping happens only when the first 34 bytes are exactly -----BEGIN PGP SIGNED MESSAGE----- or -----BEGIN SSH SIGNED MESSAGE-----.
  2. The body starts after the first blank line, found as LF LF or LF CR LF. That CR is the only CR the whole parse pipeline tolerates. With no blank line, nothing is stripped.
  3. A backward scan cuts the content at the first LF-----BEGIN PGP SIGNATURE----- or LF-----BEGIN SSH SIGNATURE-----, keeping that LF as the last byte. With no match, the content runs to the end.

The armor literals are 30 bytes but only 29 are compared, so an armor line one dash short, followed by any byte, also matches. A receiver that does not know the SSH prefix leaves it in place; the artifact then fails the line-1 gate and is treated as a content artifact.

margin = 0.15in
boxht = 0.5
$gap = 0.22
$c1 = 1.4
$c2 = 1.6
$c3 = 1.0
$c4 = 1.3

# The stored bytes of a signed artifact, left to right.
HEAD: box "wrapper head" bold "armor, blank line" small wid $c1
CARDS: box "cards" bold "every card above Z" small wid $c2
ZCARD: box "Z card" bold "final 35 bytes" small wid $c3
SIG: box "signature" bold "armor block" small wid $c4

# What each operation reads, on the same column grid.
NAME: box "name and signature" bold "every stored byte" small wid $c1 + $c2 + $c3 + $c4 with .nw at HEAD.sw + (0, -$gap)
PARSE: box "card parse" bold "stripped content" small wid $c2 + $c3 with .nw at (CARDS.w.x, NAME.s.y - $gap)
SUM: box "Z checksum" bold "MD5 of these bytes" small wid $c2 fill 0x0C0003 with .nw at (CARDS.w.x, PARSE.s.y - $gap)
§4.1.1, §9 · the bytes a signed artifact's name, card parse and Z checksum each coverEdit in pikchr-studio

The name, the card parser and the Z checksum each read a different stretch of the same bytes (§9). The name covers every stored byte, and so does signature verification when it is done. The parser reads the stripped content. The Z checksum covers the stripped content up to its final 35 bytes. Implementations MUST keep these apart: wrapper bytes never enter the Z checksum and always enter the name. A signed artifact and its unsigned inner content are two artifacts with two names.

§4.1.2Sterilization

Informative.

To make content deliberately unparseable as a structural artifact, a writer adds a first line starting with #, such as # Remove this line to create a well-formed Fossil artifact. It fails the line-1 gate. For signed content the marker goes just before the signature block, so the signature stays at the end. A fast well-formedness probe and a full parser SHOULD give the same verdict on every input.

§4.2Lines and tokens

card        = card-letter ( SP card-args LF / LF )
card-letter = %x41-5A                   ; A-Z; 21 letters defined
card-args   = token *( SP token )
token       = 1*tchar
tchar       = %x00-09 / %x0B-1F / %x21-FF   ; any byte but space and LF

The byte after the card letter MUST be exactly one space, when arguments follow, or LF. Anything else makes the card invalid, a tab included. Tokens split on space and LF only, so a tab is part of a token and a token can never hold a space. A CR is an ordinary token byte. One card is not one line: the W card's body may contain LF bytes (§4.7.18).

§4.2.1Exact arity

Each card's grammar fixes its maximum number of tokens as well as its minimum. A card with more tokens than its grammar allows MUST cause the whole artifact to be rejected: a second token on B, C, G, H, I, K, L, N, R, U, W or Z, a third on J or on the two-token Q, a fourth on A, a fifth on F. The same rule rejects bytes left after the last card.

§4.2.2A closed alphabet

Exactly 21 card letters are defined. A letter outside A–Z MUST be rejected, and so MUST the five unused capitals. There is no path for accepting unknown letters from a future version.

ABCDEFGHIJKLMNOPQRSTUVWXYZ
a card letterrejected

§4.3Escapes

Tokens that carry free text are escaped so they contain no space or LF. The decoder MUST recognise exactly eight pairs:

EscapeByteEscapeByte
\nLF (0x0a)\vVT (0x0b)
\sspace (0x20)\fFF (0x0c)
\ttab (0x09)\0NUL (0x00)
\rCR (0x0d)\\backslash (0x5c)

A backslash before any other character is dropped and the character kept, so \q decodes to q. A backslash that ends a token stays a backslash. Neither raises an error.

§4.3.1Which tokens are decoded

Decoding applies at exactly 12 token positions on 9 card letters. Hashes, checksums, timestamps and the W card are never decoded, so every token of B, D, E, G, I, K, M, P, Q, R, W and Z is read raw.

CardDecodedNever decoded
Afilename, targetsource hash
Ccomment
Ffilename, old namehash, permission
Hthread title
Jvaluefield name
Lwiki title
Nmimetype
Ttag name, valuetarget hash
Ulogin, when not empty

The A card target is the one hash-bearing position decoded before it is validated, because it may hold a wiki name instead (§4.7.2). Validation sees the decoded value, so a token full of escapes MAY decode to pure hex, and is then checked as a hash like any other.

§4.4Card order

Card letters MUST never decrease: each card's letter is greater than or equal to the one before. Equal letters may follow each other, which is how repeatable cards work. One exception is normative. An N card MAY follow a P card when the artifact has already shown a valid E card, which makes it a technote; a parser MUST accept that, and MUST reject every other out-of-order card. Appendix A.1.2 is such a technote. Two consequences:

  • B comes before F. By the time any F card is checked, the parser knows whether a B card exists, so it can decide in one forward pass whether F cards need a hash.
  • Nothing comes after Z. Z is the last letter, and the checksum framing fixes the Z card as exactly the final 35 bytes of the stripped content: Z, a space, 32 hex digits, LF. Being last is enforced by order plus byte position. The Z card's own grammar checks only its token.

§4.5Once, or many times

Twenty letters fall into two classes (§4.5.1). A duplicate of a single-occurrence card MUST be rejected: A, B, C, D, E, G, H, I, K, L, N, R, U, W. A repeatable card adds to a list: F, J, M, T, P, Q. Z is in neither. It has no duplicate check; an earlier Z line is simply content covered by the final checksum, which looks only at the last 35 bytes.

Runs of repeatable cards have their own order (§4.5.2):

CardWithin its run
FStrictly ascending by decoded filename. Equal or descending is rejected.
JStrictly ascending by field name.
MStrictly ascending by hash.
TAscending by name, then by target hash. Equal names need strictly increasing hashes.
PNot sorted or deduplicated. The order means something: primary parent first.
QNo order and no duplicate check.

The F check runs on the decoded name, after decoding. It is what lets a reader binary-search a baseline's file list and merge a delta manifest with its baseline (§8.2).

§4.5.3One comparator

The F, J, M and T checks, the merge walk and single-name lookup all compare the same way: raw bytes as unsigned values, with no locale, no Unicode normalisation and no case folding. A string sorts before any longer string that starts with it, an absent value sorts before any present one, and / is just byte 0x2f. Writers emit F cards in this same order. The R card's “filename order” (§6.3) is this order too.

§4.6Validity predicates

Three predicates check decoded tokens, treated as unsigned bytes.

Pathnames (§4.6.1) apply to the A card filename and the F card filename and old name. A parser MUST reject a name if and only if it is empty, starts with /, is . or .., starts with ./ or ../, contains //, ends with /, or has a path element that is . or ... Nothing else is checked: backslashes, spaces, control bytes and invalid UTF-8 all pass. A writer creating a new name is held to more. It MUST NOT emit a backslash, and MUST emit valid UTF-8 without overlong forms, characters above U+FFFF, surrogates, private-use characters U+E000–U+F8FF, or the noncharacters U+FDD0–U+FDEF and U+FFFE–U+FFFF. A validator of existing artifacts MUST apply only the parser rules.

Wiki names (§4.6.2) apply to the L card title and to an A card target that is not a hash. A parser MUST reject a name if and only if its first byte is 0x20 or below, a later byte is below 0x20, it has two spaces in a row, it ends with a space, or it is not 1 to 100 bytes long. Every byte from 0x21 up is otherwise allowed, with no UTF-8 check.

§4.6.3Reserved filenames

Some names belong to Fossil's own metadata. A decoded F card filename is reserved when:

  1. after removing one trailing -wal, -shm or -journal, compared without case,
  2. the rest ends, without case, in _FOSSIL_ or .fslckout,
  3. and that ending is the whole rest, or follows a /, or a suffix was removed in step 1.

x_FOSSIL_ passes. x_FOSSIL_-wal and dir/.fslckout-journal are reserved. An F card with a reserved name MUST be silently dropped from the file set; the artifact is still accepted. The sort check runs first, so the dropped card must still be in order, and the next F card is compared with the last card kept. In the reference parser the drop happens before the card would make the artifact a manifest. Only the current name is tested, never the old name.

Test a tokenType it as it appears on the wire, escapes and all
Rule§Result

§4.6.4Timestamps

D and E card timestamps are single raw tokens, never decoded. A writer emits YYYY-MM-DDTHH:MM:SS with an optional fraction, and for stored artifacts whose time can order repository state a producer MUST use that form. A reader MAY accept more: a date only, a time only (read as a time on 2000-01-01), the words now, subsec and subsecond, or a bare Julian day number greater than 0 and less than 5373484.5. A date-time may carry a timezone. The separator must be uppercase T; lowercase t is rejected.

Each field has an exact digit count: a 4-digit year, month 01–12, day 01–31 with no calendar check (2024-02-31 passes), hour 00–24, minute and second 00–59. A date MUST fall strictly after -4713-11-24. The relative words are resolved against the clock when parsed, so two ingestions of the same bytes can store different times. They are admitted for compatibility only.

§4.7The 21 cards

Every card's grammar is exact (§4.2.1). The table gives each one's tokens; the cards with more to say follow it.

CardTokensWhat it records§
Afilename target [source]An attachment added to, or removed from, a ticket, technote, forum post or wiki page.4.7.2
BhashThe baseline. Its presence makes a delta manifest, and a writer MUST name a baseline manifest, not another delta.4.7.3
CcommentThe comment. A C card with no token MUST be rejected.4.7.4
DtimestampWhen the artifact was made.4.7.5
Etimestamp idThe technote's date and identifier.4.7.5
Fname [hash [perm [old-name]]]One file.4.7.6
GhashThe forum thread's root post.4.7.7
HtitleA forum thread title, with no content check.4.7.7
IhashThe forum post being replied to.4.7.7
Jfield [value]A ticket field change. The name is stored verbatim, a leading + included; a missing value is the empty string.4.7.8
Kticket-idThe ticket, always 40 hex digits whatever the hash policy.4.7.9
LtitleThe wiki page name, which must pass §4.6.2.4.7.10
MhashOne member of a cluster.4.7.11
NmimetypeThe W body's mimetype on wiki, technote and forum artifacts (default text/x-fossil-wiki); the C comment's on an attachment (default text/plain). On a check-in it is legal and ignored.4.7.12
P*hashParents, in order.4.7.13
Q±hash [hash]A cherry-pick (+) or backout (-), the sign joined to the hash. The optional second hash is the range's baseline, left null when absent.4.7.14
Rmd5A checksum over the check-in's files. Manifests only, never required.4.7.15
T±*name target [value]A tag.4.7.16
U[login]The user. An empty or missing login MUST read as anonymous.4.7.17
Wsize, then bodyA body of exactly size bytes.4.7.18
Zmd5The checksum of everything above it.4.7.19

§4.7.1Hash tokens

Every token that names an artifact MUST be exactly 40 or 64 hexadecimal digits, either case: the A target when it is a hash, the A source, B, the E identifier, the F hash, G, I, M, every P token, both Q tokens once the sign is removed, and the T target unless it is *. The K ticket identifier is the exception at 40 digits only; writers MUST emit it lowercase and parsers MAY accept either case. R and Z hold 32-digit MD5 checksums. None of these is decoded, apart from the A target.

§4.7.2A: attachments

An attachment artifact has exactly one A card. Its filename is decoded and MUST pass the pathname predicate; only the last path component is kept. Its target is decoded and MUST look like a hash or a wiki name, though the parser does not check that it names anything. The target is resolved later, when the artifact is crosslinked, in a fixed order: forum post, ticket, wiki page, technote. A target that names nothing known is still recorded, as a generic artifact reference, and is not an error.

The source decides the operation. With a source, the attachment is added. Without one, the attachment for that (target, filename) pair is deleted. The operation with the latest time wins.

§4.7.6F: files

In a baseline manifest every F card MUST carry a hash. In a delta manifest the hash MAY be left off; the card then marks the file as deleted from the baseline (§8.2).

The third token is always the permission and the fourth, when present, the old name of a renamed file. Producers SHOULD emit only w, x or l, and when they write an old name they MUST write a permission before it, normally w. A reader MUST accept any permission token without checking it: one containing x is executable, otherwise one containing l is a symbolic link, and anything else is a regular file.

§4.7.13P: parents

Zero or more hashes on one line. An empty P card MUST be accepted, because the initial empty check-in has historically had one; Appendix A.1.1 shows it. On a check-in the first hash is the primary parent and the rest are merge parents. On a wiki page they are prior versions; on a technote, one prior version with the same identifier; on a forum post, one prior version of the post. A producer SHOULD emit at most one P card. The ancestry graph and the branch, leaf and trunk views built from it belong to the Repository State draft.

§4.7.16T: tags

The tag name carries its sign: + applies a tag to one artifact, * applies one that propagates to descendants through primary children until blocked, and - cancels or blocks one. The name is decoded, and after the sign it MUST NOT be all hex digits, so it can never be mistaken for a hash prefix. The target is a hash, or * for the artifact carrying the card. A missing value is null, unlike J's empty string.

Parsing only checks syntax and order. The effects arrive when the artifact is crosslinked, and are the Repository State draft's business. Canonical artifacts use T *branch * NAME for a branch, T *sym-NAME * for a symbolic tag, and T +closed * and T -closed * to close and reopen a leaf.

§4.7.18W: a body counted in bytes

The W card is the one card that spans lines. Its token is a decimal size; the body starts right after that line's LF and is exactly size bytes, which may include LF or any other byte. The byte after the body MUST be LF. A writer MUST emit the size as plain decimal digits. A reader takes the leading digits of the token and ignores the rest, so 12x means 12. It MUST detect overflow, MUST check the size against the bytes that remain before reading the body, and MUST NOT scan the body for a card boundary.

§5Eight kinds of structural artifact

§5.1Types

Every structural artifact is exactly one of eight types, each with a fixed code and the name shown on JSON and API surfaces. Code 0 means “not yet known” and exists only during parsing.

CodeTypeExternal nameRecords
1MANIFESTcheckina check-in
2CLUSTERclustera list of artifacts
3CONTROLtagtag changes
4WIKIwikia wiki page version
5TICKETticketa ticket change
6ATTACHMENTattachmentan attachment
7EVENTtechnotea technote
8FORUMforumposta forum post

Existing diagnostic messages use a second vocabulary, in which MANIFEST is manifest and FORUM is forum post. The draft warns against mixing the two.

§5.2Which cards each type allows

An artifact MUST NOT hold a card outside its type's allowed set, and MUST hold every card in its required set. Both checks run once the type is final.

TypeABCDEFGHIJKLMNPQRTUWZ
MANIFEST○○●○○○○○○○●
CLUSTER●●
CONTROL●●●●
WIKI○●●○○●●●
TICKET●●●●●
ATTACHMENT●○●○○●
EVENT○●●○○○○●●
FORUM●○○○○○●●●

● required, ○ allowed. A manifest needs only D and Z, so its comment and user are optional. A wiki page without a U card is rejected. Z is required for every type, even though the checksum gate of §4.1 lets an artifact without one through.

§5.3How the type is decided

Thirteen letters set the type whenever they are read, overwriting any earlier choice:

CardsSet the type to
AATTACHMENT
B F Q RMANIFEST
EEVENT
G H IFORUM
J KTICKET
LWIKI
MCLUSTER

C, D, N, P, T, U, W and Z never set it. Cards come in letter order, so when several type-setting letters appear, the type MUST be the one set by the alphabetically last. If none appeared, the fallback decides: the artifact is a MANIFEST if it had a C card, any F entry or any P parent, and otherwise a CONTROL artifact. A manifest with only C, D, P, U and Z cards never sets a type, so it reaches the fallback; so does one whose only F card had a reserved name and was dropped before it could set the type. The legality checks of §5.2 then reject any artifact whose cards do not fit the type it ended up with.

§5.4Four checks per type

  1. A CONTROL artifact MUST NOT have a T card whose target is *.
  2. In a technote every T card MUST target * and use the + sign.
  3. A forum post MUST NOT have both an H card and an I card.
  4. A forum post's P card MUST NOT list more than one parent, since forum posts are not merged.

A forum post does not have to be consistent with other artifacts: following its I and P chains need not lead to the post its G card names.

§5.5A quick pre-filter

Informative.

A fast check may reject any input whose first card letter is N or later before trying a full parse. That is safe because every type's smallest allowed letter is M or earlier (MANIFEST B, CLUSTER M, CONTROL D, WIKI C, TICKET D, ATTACHMENT A, EVENT C, FORUM D), and order puts the smallest letter first.

∗Inspect an artifact

The inspector applies the rules of §4 to §6 in the order the draft gives them and stops at the first rejection, as a parser does. Edit the bytes to break a rule. Once this draft's repository is open, its own check-in manifests appear as well. The R card is not part of parsing (§6.3), so the inspector recomputes it separately, from the stored files, under the parse checks.

Artifact inspector
CardWhat it says§
Check, in parse order§Result

§6Two checksums

Structural artifacts carry two MD5 checksum cards. The Z card covers the artifact's own text. The R card covers the content of every file in a check-in. Both are 32 hexadecimal digits.

§6.1Token grammar

At parse time both tokens MUST be exactly 32 hex digits, either case. Wherever a value is actually checked, the comparison is byte-exact against a lowercase digest, so in practice only lowercase passes. No artifact bytes are case-folded anywhere in the parse.

§6.2Z: the artifact checks itself

Let n be the length of the stripped content. A Z card fills its last 35 bytes:

OffsetBytesHolds
0 to n−36n−35everything above the Z card: the checksummed bytes
n−35, n−342Z and a space
n−33 to n−232the digest
n−11the terminating byte, which verification never reads

Verification MUST compute MD5 over the first n−35 bytes, write it as 32 lowercase hex digits, and compare it byte for byte with the stored digest. A present but different Z card MUST cause rejection. A missing one is not this step's concern; §5.2 rejects it. Because the comparison is exact, a digest with an uppercase digit MUST fail even when it is numerically right. Producers MUST emit Z, one space, the lowercase digest and LF. Existing implementations always verify; none can turn the check off.

Appendix A.2 works it through for the hero manifest above: the five cards before Z are 27, 22, 2, 35 and 6 bytes, so MD5 runs over bytes 0 to 91 and the Z card sits at bytes 92 to 126.

§6.3R: the check-in's files

A parser checks only that the R token is 32 hex digits. It MUST NOT be expected to recompute the value, and never does. The value is checked at commit time, against the working checkout, by the writer. An R card appears only in a manifest and is never required.

§6.3.2Computing R

One running MD5 is fed every tracked file in filename order, where filename order is the byte order of §4.5.3:

ctx = MD5-Init()
for each tracked file f, in filename order:
    MD5-Update(ctx, name(f))
    MD5-Update(ctx, SP || decimal(size-in-bytes(f)) || LF)
    MD5-Update(ctx, content(f))
r-value = lowercase-hex(MD5-Final(ctx))

The size is plain decimal with no leading zeros. So the checksum covers names and sizes as well as content. For Appendix A.3's three files the input is these 85 bytes, and its MD5 is the R card 95feb799536dc053e7968efb7cdd3ec3:

alpha.txt 12↵hello world↵beta.txt 23↵fossil artifact format↵gamma.txt 12↵R card test↵

An empty file set gives the MD5 of nothing, d41d8cd98f00b204e9800998ecf8427e, which is the R card of the root check-in in the hero.

§6.3.3R on a delta manifest

For a delta manifest, R MUST be computed over the full file set rebuilt from the delta and its baseline (§8.2), skipping delete markers, and not over the delta's own F cards. The writer emits R, a space, 32 lowercase hex digits and LF, nothing else (§6.3.4). An uppercase R card passes parsing but fails the commit-time check.

§7Hash policy

Each repository has a hash policy that decides which algorithm names newly stored artifacts, and whether the other algorithm may be consulted. It is local to the repository and only affects names assigned from then on.

§7.1Five values

Values MUST match exactly, case and all, and an unknown value is a fatal error.

ValueOrdinalNames new content withMay also consult
sha10SHA-1SHA3-256
auto1SHA-1SHA3-256
sha32SHA3-256SHA-1
sha3-only3SHA3-256nothing
shun-sha14SHA3-256nothing

The second algorithm is used to find out whether the same content is already stored under its other name. Under sha3-only and shun-sha1 a request for it MUST fail. Those two policies behave identically for local naming and differ only in what sync admits.

§7.2The default

A repository with no policy set MUST derive one. If it holds any SHA3 name, or no SHA-1 name at all, which includes an empty repository, the policy is sha3. Only when it holds SHA-1 names and no SHA3 names is it auto. The default is never shun-sha1.

§7.3Setting it

The policy is one stored configuration value. Nothing stops an operator setting it back to sha1. Asking for auto in a repository that already holds a SHA3 name silently stores sha3 instead, so once a SHA3 name exists, auto cannot be set again.

§7.4The ratchet

The rule: while the policy is auto, if the name storage is about to assign has 64 digits, the repository MUST switch to sha3 in the same write, before the artifact is looked up, inserted or linked.

Under auto, new content is named with SHA-1, so a 64-digit name arrives in only two ways:

  • incoming content carries a 64-digit name, or
  • local content's SHA3 hash matches an artifact already stored, and that artifact's 64-digit name is reused.

A name the receiver rejects before storage switches nothing. A name stored before it is checked does switch the policy.

§7.5One name per content

When content is stored without a supplied name, storage MUST first hash it with the other algorithm, where the policy allows, and look that name up. Only on a miss does it name the content with the primary algorithm. Under sha1, auto and sha3, local writes never give the same bytes two names. Under sha3-only and shun-sha1 the lookup cannot happen, so new content MUST get a SHA3 name even if the same bytes already have a SHA-1 name. Some content is then stored twice, and delta compression keeps the cost down.

§7.6Names never change

A name is assigned once, when content is first stored. Changing the policy MUST NOT rename or rehash anything already stored.

§7.7During sync

Peers do not negotiate a policy; each applies its own. The one signal that matters is the peer's announced version: if it predates SHA3 support, a sender refuses to send any artifact with a 64-digit name, with an explicit error. It never converts or silently skips it. What shun-sha1 does to incoming SHA-1 artifacts is defined by the Sync Protocol.

§8Rebuilding a delta manifest

A manifest with a B card is a delta manifest; one without is a baseline manifest. Nothing else decides it, not file count, not an R card. A delta manifest lists only what changed since its baseline, and anything that needs the full file set MUST merge the two. A delta manifest is a different thing from the Delta Format, which encodes an artifact's bytes one layer below. This merge works on cards.

Appendix A.4 has a baseline with three files and a delta that modifies one, deletes one and adds one:

margin = 0.15in
boxht = 0.5
boxwid = 1.1
$gap = 0.55
$col = 0.18

# Baseline A.4.1: one column per filename, in sorted order.
BASE_A: box "file-a.txt" mono bold "b3cd28e8…" mono small
BASE_B: box "file-b.txt" mono bold "780ac49d…" mono small with .w at BASE_A.e + ($col, 0)
BASE_C: box "file-c.txt" mono bold "a37be80e…" mono small with .w at BASE_B.e + ($col, 0)

# Delta A.4.2, each entry under the name it changes.
DELTA_A: box "file-a.txt" mono bold "a61d8bf8…" mono small fill 0x0C0003 with .n at BASE_A.s - (0, $gap)
DELTA_C: box "file-c.txt" mono bold "no hash" small dashed with .n at BASE_C.s - (0, $gap)
DELTA_D: box "file-d.txt" mono bold "4ff0cf8d…" mono small fill 0x0C0003 with .w at DELTA_C.e + ($col, 0)

# Reconstructed set.
OUT_A: box "file-a.txt" mono bold "a61d8bf8…" mono small fill 0x0C0003 with .n at DELTA_A.s - (0, $gap)
OUT_B: box "file-b.txt" mono bold "780ac49d…" mono small with .n at (BASE_B.x, DELTA_A.s.y - $gap)
OUT_D: box "file-d.txt" mono bold "4ff0cf8d…" mono small fill 0x0C0003 with .n at DELTA_D.s - (0, $gap)

text "baseline" small rjust at BASE_A.w - (0.12, 0)
text "delta" small rjust at (BASE_A.w.x - 0.12, DELTA_A.y)
text "result" small rjust at OUT_A.w - (0.12, 0)
text "modify" small at (OUT_A.x, OUT_A.s.y - 0.2)
text "carry forward" small at (OUT_B.x, OUT_A.s.y - 0.2)
text "delete" small at (DELTA_C.x, OUT_A.s.y - 0.2)
text "add" small at (OUT_D.x, OUT_A.s.y - 0.2)

arrow from DELTA_A.s to OUT_A.n
arrow from BASE_B.s to OUT_B.n
arrow from DELTA_D.s to OUT_D.n
§8.2, A.4 · rebuilding a delta manifest's file set from its baselineEdit in pikchr-studio

§8.1What the merge needs

Both lists sorted by decoded filename. A conforming parser already guarantees that (§4.5.2). A delete marker is an F card with no hash, allowed only in a delta manifest.

§8.2The merge walk

An implementation MUST walk both lists at once in filename order, comparing decoded names byte by byte. At each step:

  1. Same name, and the delta entry has a hash: emit the delta entry. Both lists advance.
  2. Same name, and the delta entry has no hash: the file is deleted. Both advance and nothing is emitted.
  3. Different names: emit whichever sorts first and advance only its list. A delete marker for a name the baseline lacks is emitted here too, but it adds no file. It is not an error.

The walk ends when both lists are used up. When one ends first, the rest of the other is emitted as it stands, delete markers included. So anything computed from the result MUST skip entries with no hash, as the R card computation does.

§8.3Finding the baseline

The baseline is loaded the first time it is needed. If it is missing or is not a manifest, file enumeration and single-name lookup MUST treat that as fatal. Consumers that can wait, such as a parent-to-child file diff or post-parse relationship processing, MAY instead record the gap, keyed by the dependent artifact and the missing hash, and retry when the baseline arrives. Each step of an enumeration reports no error for a missing baseline; it quietly lists the delta alone. So implementations MUST resolve the baseline before they start.

§8.4Looking up one name

To find one file in a delta manifest, an implementation MUST search the delta's own F cards first, with closest-name matching turned off. A delete marker for the name means not found, without asking the baseline. Only a name the delta does not mention at all sends the lookup to the baseline.

§8.5A view that does not merge

Informative.

The JSON rendering of a manifest lists its own F cards, so for a delta manifest it shows only the changes. This is deliberate, and that view is the one exception.

§9Security considerations

  • A name binds content only as strongly as its hash. SHA-1 has known collision attacks; SHA3-256 does not, and is the default for new artifacts in most repository states. Existing implementations compute SHA-1 with collision detection, and shun-sha1 lets a repository refuse SHA-1 artifacts arriving by sync.
  • MD5 is not collision-resistant. The Z and R cards catch accidental corruption. Protection against a crafted artifact comes from its name.
  • Name, Z checksum and signature each cover a different byte range (§4.1.1), and implementations MUST keep them apart. A wrapper says how the bytes are laid out, nothing more. It is not evidence that a signature was checked or that the signer is trusted; the draft leaves signature checking out of scope.
  • A receiver that admits content as verified under a supplied name without checking it breaks the integrity of everything downstream. Content stored under an unchecked name MUST NOT be treated as verified.
  • Name and digest comparisons are exact lowercase. Accepting a token at validation MUST NOT be taken as evidence of integrity.
  • The reserved-name drop changes the file set without failing the parse. Two readers disagree about such a manifest only if one skips the drop.
  • The W body is found by count. A reader MUST bound the size against what remains, reject overflow, never scan the body, and never read past the end of the buffer.
  • An earlier Z line is just content. Tools that scan text for a Z card can be fooled; verification MUST use the byte position of §6.2.
  • Rebuilding a delta manifest or verifying an artifact needs the artifacts it refers to. Until they arrive the reference stays unresolved, and the Sync Protocol says how to ask for them.

App. AWorked examples

Each example in the appendix is complete: its name, checksums and byte counts recompute from the bytes shown. Long lines are folded with the single-backslash method of RFC 8792, and no real line ends in a backslash, so unfolding is unambiguous. All of them load into the inspector above, which recomputes their names, their Z cards, and their R cards where the draft gives the file contents.

§ArtifactBytesShows
A.1.1Root check-in, SHA-1 name a28c8364…127An empty P card; R as the MD5 of no files.
A.1.2Technote, SHA3-256 name 2fa494a1…655An N card after P, allowed because E came first; a 415-byte W body with blank lines.
A.2The A.1.1 manifest again92 + 35Z as the MD5 of everything before it.
A.3A check-in with three files425R over an 85-byte input of names, sizes and contents.
A.4A baseline and a delta429, 446The merge of §8 and R over the rebuilt set: a 98-byte input.

In A.4.3 the four outcomes of the merge are named add, modify, delete and carry forward. The rebuilt set is file-a.txt with new content, file-b.txt unchanged and the new file-d.txt, and the delta's R card 0d7675864b46dd3f197e988759ac2dfa is computed over those three.

App. BCollected grammar

ABNF per RFC 5234, writer syntax only. Order, repetition, per-type legality, the predicates, the checksums and what readers tolerate are all in the body. The W body is located by count, which ABNF cannot express.

artifact    = 1*card                ; final byte of the artifact is LF
card        = a-card / b-card / c-card / d-card / e-card /
              f-card / g-card / h-card / i-card / j-card /
              k-card / l-card / m-card / n-card / p-card /
              q-card / r-card / t-card / u-card / w-card / z-card

a-card      = %x41 SP filename SP target [ SP hash ] LF
target      = hash / wikiname
b-card      = %x42 SP hash LF
c-card      = %x43 SP comment LF
d-card      = %x44 SP timestamp LF
e-card      = %x45 SP timestamp SP hash LF
f-card      = %x46 SP filename [ SP hash [ SP perm [ SP old-name ] ] ] LF
g-card      = %x47 SP hash LF
h-card      = %x48 SP token LF
i-card      = %x49 SP hash LF
j-card      = %x4A SP token [ SP token ] LF
k-card      = %x4B SP 40hexdig-ci LF
l-card      = %x4C SP wikiname LF
m-card      = %x4D SP hash LF
n-card      = %x4E SP token LF
p-card      = %x50 *( SP hash ) LF
q-card      = %x51 SP q-sign hash [ SP hash ] LF
q-sign      = "+" / "-"
r-card      = %x52 SP md5-token LF
t-card      = %x54 SP tag-sign token SP ( hash / "*" ) [ SP token ] LF
tag-sign    = "+" / "-" / "*"
u-card      = %x55 [ SP [ token ] ] LF
w-card      = %x57 SP size LF body LF
size        = "0" / ( nonzero-digit *DIGIT )
nonzero-digit = %x31-39
body        = *OCTET                ; exactly <size> octets; MAY contain LF
z-card      = %x5A SP md5-token LF

token       = 1*tchar
tchar       = %x00-09 / %x0B-1F / %x21-FF
comment     = token                 ; escape-decoded
filename    = token                 ; escape-decoded; pathname predicate
old-name    = token                 ; escape-decoded; pathname predicate
perm        = token                 ; old-name requires perm; SHOULD be w/x/l
wikiname    = token                 ; escape-decoded; wiki-name predicate

hash        = 40hexdig-ci / 64hexdig-ci
md5-token   = 32hexdig-ci
hexdig-ci   = DIGIT / %x41-46 / %x61-66
lhexdig     = DIGIT / %x61-66

timestamp   = date %x54 time [ frac ]    ; writer form
date        = 4DIGIT "-" 2DIGIT "-" 2DIGIT
time        = 2DIGIT ":" 2DIGIT ":" 2DIGIT
frac        = "." 1*DIGIT

artifact-name = sha1-name / sha3-name    ; writer form
sha1-name     = 40lhexdig
sha3-name     = 64lhexdig
accepted-name = 40hexdig-ci / 64hexdig-ci

SP = %x20    LF = %x0A    DIGIT = %x30-39    OCTET = %x00-FF

App. CWhere older documentation differs

These statements in Fossil's documentation, or in text that comes with existing implementations, do not match the body of the draft. The body is definitive.

Older text saysThe draft says
A manifest requires C, D, U and Z.It requires only D and Z; C and U are optional, for historical artifacts (§5.2).
Every type except a manifest needs a Z card.Every type needs one, manifests included (§5.2).
A wiki artifact with no U card counts as user nobody.It is rejected. A U card with an empty login reads as anonymous (§4.7.17).
A forum post whose G card does not match the root reached through its I and P chains is invalid.No parser enforces that, and the draft does not require it (§5.4).
The timestamp separator is a lowercase t.Only uppercase T is accepted (§4.6.4).
Only PGP clearsign wrappers exist.SSH signed wrappers are removed too (§4.1.1).
The J card's field name and value are both escaped.Only the value is decoded. Implementations differ here (§4.7.8).
(Implementations) One does not decode the N mimetype; one decodes the J field name.The decode map of §4.3.1 is the rule.
(Parsers) An N card is accepted right after P once an E card has made the artifact a technote.That exception is part of the ordering rule. Every other out-of-order card is rejected (§4.4).
(Implementations) One accepts any F permission token, one only w, x or l.Readers accept any token in third place; producers SHOULD write w, x or l (§4.7.6).
(Implementations) One joins repeated P cards, one rejects a second.Producers SHOULD emit one; receivers are not constrained further (§4.7.13).
The F card description does not mention reserved names.They are silently dropped, which changes some historical file sets (§4.6.3).
New repositories default to shun-sha1.The default is never shun-sha1; an empty repository gets sha3 (§7.2).
draft-fossil-repo-state-00 · an illustrated reading

Fossil Repository State

A Fossil repository stores only immutable artifacts. Branches, tags, leaves, closed markers and the current trunk are not written down anywhere: every repository computes them from the artifacts it holds. This reading walks through those rules and shows each derived table beside the section that defines it, read from this draft's own repository.

§1Facts no artifact records

Every check-in, tag and edit is an artifact: a run of bytes named by its own hash, never changed once written. History is the append-only set of those artifacts. That gives strong integrity, but some everyday questions have no artifact that answers them. Which check-in is the latest on trunk? Which branch does a check-in belong to? Which leaves are still open? What does a tag point to now? (§1)

Each answer is implied by the whole set. A repository works it out by reading the parent links and tag records spread across the artifacts. The draft calls the result the repository's state: the check-in ancestry graph, branch and tag membership, open leaves, closed markers, and the check-in a symbolic name selects. The rules are fixed so that two independent implementations reading the same artifacts reach the same answers (§1).

One answer has a local input. The current trunk also depends on the repository's main-branch setting, so two repositories with the same artifacts can name different current-trunk check-ins when that setting differs (§1, §9).

§1.1What this draft covers, and what it borrows

The scope is the meaning computed from parent and tag cards (P and T cards): ancestry, branch membership, named and symbolic references, leaves, closed markers and current-trunk resolution. The companion drafts supply the rest (§1.1):

DraftSupplies
Artifact FormatThe byte grammar of P and T cards, hash formats and lengths, content-type rules, hash-to-check-in resolution, delta-manifest reconstruction.
Delta FormatOnly the content-delta encoding underneath.
Sync ProtocolHow artifacts move between peers. The state rules hold however the artifacts arrived.

Ordering uses the numeric event.mtime and tagxref.mtime values derived from artifact timestamps. Results are reproducible only when the stored D-card timestamp names a fixed instant. The compatibility literals now, subsec and subsecond fall outside the reproducible rules (§1.1).

§2Terminology

The draft uses MUST, SHOULD and MAY in the BCP 14 sense, and only when capitalised. Card, manifest and the card letters come from the Artifact Format unchanged (§2).

TermMeaning
ridThe local integer id of a stored artifact. A hash names an artifact everywhere; its rid names it in one repository only and is never synchronised. The derived views are computed over rids.
Check-in DAGCheck-ins joined by parent links. Every check-in but an initial one names one or more parents.
Primary parentThe first parent listed. It fixes the check-in's place for branch inheritance and ancestry walks.
Merge parentAny parent after the first. It records lines of development joining.
TagA named annotation on a check-in that asserts a property of it and its descendants, or revokes one.
Tag typeCancel (-), singleton (+) or propagating (*), stored as 0, 1 and 2. A type above zero means the tag is active on that check-in.
BranchThe propagating tag 8 and the check-ins it is in effect on. A check-in's branch is the innermost active branch tag reaching it along primary links, never across merge edges.
LeafA check-in with no child on the same branch. It MAY have children on other branches and still be a leaf.
Open leafA leaf without an active closed marker.
Symbolic nameA user-facing name, stored with a sym- prefix. Commands MUST prepend sym- unless a raw-name option is given.
Current-trunkThe check-in the main-branch name (default trunk) resolves to. It need not be an open leaf.

The prefixes sym-, wiki-, tkt- and event- are reserved. Only sym- tags MAY come from ordinary user tagging; the other three MUST NOT be created directly by users (§2).

The tag names live in a tag table indexed by tagid (§2). Here it is for this repository. The draft fixes two ids: tagid 8 is branch and tagid 9 is closed. sym-trunk comes from the initial check-in, which starts the default branch (§6.1).

§3Building the check-in DAG

Each check-in manifest names its parents in its P card. An initial check-in has an empty P card; a merge names several. A reader turns every parent reference into a parent/child linkage record once and walks those records afterwards, instead of re-parsing manifests on every query (§3).

§3.1One row per parent and child

Each linkage row is keyed on the pair (parent-id, child-id) and carries a flag saying whether the parent is primary, the check-in's modification time, and a baseline id used for delta-manifest reconstruction (§3.1). The pair is unique. A manifest MAY name the same parent twice, since the Artifact Format does not require P-card references to be sorted or deduplicated, and the table keeps a single row for the pair (§3.1).

Section 7.1 names this table plink. Its columns map directly: pid and cid are the pair, isprim is the flag, mtime the time, baseid the baseline.

§3.2Primary parent and merge parents

Exactly one linkage row per child has the primary flag set: the row for the first hash on the P card. Every other parent gets a row with the flag unset, marking a merge parent. A sole parent is primary. When a P card repeats a hash, the row keeps the flag chosen by that hash's first appearance (§3.2).

A reader MUST treat the primary row as the mainline ancestor for linear-history operations such as first-parent traversal and the timeline's mainline. Merge rows MUST be included when full ancestry is needed, for a merge base or an ancestor test, and MUST NOT be treated as the mainline predecessor (§3.2).

§3.3Walking both ways

Edges run from parent to child. Manifests name their parents by hash and never change, so an edge's direction is fixed when it is written (§3.3). The same rows serve both walks: toward the origin through an index on (child-id, parent-id), and toward descendants through the (parent-id, child-id) key. There is no separate ancestor or descendant store (§3.3).

§5Tag types and propagation

§5.1Three types

PrefixTypeValueEffect
-cancel0Removes or blocks a tag's effect.
+singleton1Applies to the target only.
*propagating2Applies to the target and carries forward along its primary children.

Any other leading character MUST be a fatal crosslink error (§5.1).

§5.2Where tag state lives

Tag state is the tagxref table, unique per (artifact, tag). Each row holds the tag type; srcid, the artifact whose T card applied the tag, or zero when the value was propagated in from an ancestor; origid, the artifact that originally holds a propagating tag; the value; an mtime; and rid, the artifact the row describes. Because (rid, tag) is unique, an artifact has at most one live entry per tag, and later writes replace earlier ones (§5.2).

When history is a single line on trunk, only the first check-in has direct rows, because it carries the branch and sym-trunk cards (§6.1). Every later row has srcid 0, origid pointing at the first check-in, and the first check-in's mtime, not its own (§5.4 step 4).

§5.3Applying a tag directly

  1. If the (tag, artifact) row already has an mtime at or after the incoming one, the write MUST be skipped. A strictly later time wins. With exactly equal times, the winner depends on write order and is implementation-defined.
  2. Otherwise the row is replaced: the raw type from the card, srcid the artifact carrying the card, origid and rid the target.
  3. A cancel clears the value to null.
  4. Before propagation, a singleton is downgraded to a cancel. It never passes its value on, and it blocks a same-named tag from propagating through it. A propagating tag passes unchanged.

§5.4Propagation

Propagation carries a tag from an artifact to its descendants along primary links only. It MUST NOT follow merge links, so a tag reached only through a merge has no effect past the artifact it names (§5.4). The walk accepts only type 2 or type 0 and MUST reject anything else.

margin = 0.15in
boxht = 0.5
boxwid = 0.8
$gap = 0.3
$drop = 0.45

# Trunk: five check-ins on one primary-parent chain.
C1: box "c1" bold "direct" small fill 0x0C0003
C2: box "c2" bold "inherited" small with .w at C1.e + ($gap, 0)
C3: box "c3" bold "inherited" small with .w at C2.e + ($gap, 0)
C4: box "c4" bold "inherited" small with .w at C3.e + ($gap, 0)
C5: box "c5" bold "inherited" small with .w at C4.e + ($gap, 0)

# Feature: rooted on c2, merged back into c5.
F1: box "f1" bold "direct" small fill 0x0C0003 with .n at C3.s + (0, -$drop)
F2: box "f2" bold "inherited" small with .n at C4.s + (0, -$drop)

text "trunk" small rjust at C1.w - (0.12, 0)
text "feature" small rjust at (C1.w.x - 0.12, F1.y)

arrow from C1.e to C2.w
arrow from C2.e to C3.w
arrow from C3.e to C4.w
arrow from C4.e to C5.w
arrow from F1.e to F2.w
arrow from C2.s down until even with F1 then to F1.w
arrow from F2.e right until even with C5 then to C5.s dashed
text "merge" small ljust at (C5.x + 0.1, (F2.y + C5.s.y) / 2)
§3.2, §5.4 · a branch tag walks primary links only: a direct tag at f1 stops trunk's, and the dashed merge into c5 carries nothingEdit in pikchr-studio

In the figure, c1 and f1 carry direct branch tags. Trunk's tag walks c2 to c5 along primary links. It does not enter f1, whose own row is direct. Feature's tag reaches f2, and the dashed merge into c5 carries nothing. f2 has a child, but on another branch, so it is still a leaf of feature (§2).

  1. Seed a priority queue, ordered by mtime, with the starting artifact.
  2. Take an artifact from the queue and list its primary children.
  3. A child qualifies only if its row has srcid 0 and an mtime strictly earlier than the tag's. A nonzero srcid or an equal or later time keeps the existing row. A child with no row qualifies for type 2 but not for a cancel, so a cancel stops where there is nothing to delete.
  4. Type 2 replaces the child's row with type 2, srcid 0, the origin's origid, value and mtime. Type 0 deletes the child's row.
  5. Each child that changed joins the queue.
  6. The walk ends when the queue is empty.

The stepper runs these rules on a five-check-in primary chain. Every check-in starts with the tag milestone = alpha, applied directly at c1 at time 10 and propagated to the rest. Pick a target and a prefix, then apply; each apply uses the time shown and moves it on by 10. Set the time to 5 to see §5.3 step 1 skip the write, or put a * on c3 and then a later one on c1 to see c3 block it.

Apply a tag to a primary chain§5.3 to §5.5
·
Check-intagtypesrcidorigidvaluemtimeThis step

§5.5What stops a propagating tag

  • A direct same-tag row, srcid nonzero, stops it whatever that row's time. The time comparison applies only to inherited rows.
  • A cancel for the same tag deletes any propagated row there and keeps the value from that artifact's primary children.
  • A singleton never propagates. Downgraded to a cancel, it blocks just as a - card would.

The draft calls this the complete definition of “propagation until blocked” for the whole document set (§5.5).

§6Branches

§6.1The branch of a check-in

Every check-in belongs to exactly one branch. An implementation MUST find it from a tagxref row for the branch tag on that check-in with tagtype above zero; the row's value is the branch name. A check-in with no such row belongs to the default branch (§6.1).

That fallback is narrow. In a conforming repository the first check-in of every branch, the initial check-in included, carries the propagating branch tag and a matching propagating sym-<branch> tag, and later check-ins inherit both. Creating a branch MUST create the sym- marker, without which the name would not resolve. A check-in attributed to the default branch only by the fallback is not thereby its resolvable tip or its open leaf (§6.1).

The default branch name is the main-branch setting, or trunk when it is unset. An implementation MUST read the setting and MUST NOT assume a fixed string when one is set (§6.1).

§6.2Creating a branch

A new branch is two propagating cards on its root manifest: *branch with the new name as value, and *sym-<name>. Both propagate under §5, which is how later check-ins inherit the branch (§6.2). The root MUST also cancel, with - cards, every sym- tag the root would otherwise inherit from its parent, so it does not carry the parent branch's names alongside its own (§6.2). Fossil's two ways of branching differ in the detail: commit --branch cancels the inherited propagating tags and branch new the singletons as well, and the root's active tags come out the same either way (§6.2's informative paragraph).

Creation MUST NOT succeed when the name already names an open branch, and the check MUST happen before the root manifest is written (§6.2).

§6.3Listing branches

The branch list is derived. Group the tagxref rows where tag.tagname = 'branch' and tagtype > 0 by value: each value is a branch, with its greatest event.mtime and the count of its check-ins (§6.3).

The listing's open or closed column checks only the branch's representative check-in, the member with the greatest event.mtime. That can disagree with §9, which calls a branch open when any open leaf remains. If the newest check-in is closed while an older leaf is still open, the listing says closed and §9 says open. The draft says neither is in error: they answer different questions (§6.3). The list can be filtered by state, private status, name glob and merged status, and ordered by name or time (§6.3).

§6.4The current branch of a checkout

Some Fossil code paths read the branch tag's value on a checkout without the tagtype > 0 filter. The draft calls this a textual inconsistency, not a second rule. A conforming implementation MUST apply the filter everywhere, including when it reports a checkout's current branch (§6.4).

§7Leaves

§7.1The leaf table

A leaf is a check-in with no child on the same branch. The set can be computed by joining plink and tagxref, but implementations keep it in a leaf table of one integer column. The table carries no open or closed flag; that is derived separately (§7.1).

§7.2The normative test

A check-in is a leaf if and only if no plink row naming it as parent has a child on the same branch. The test MUST NOT distinguish primary from merge links: a same-branch child reached only by a merge still makes its parent a non-leaf (§7.2). A narrower primary-only child count MAY exist for other uses, and documentation calling it “zero children of any kind” MUST NOT be read as the leaf test (§7.2).

§7.3Keeping the table right

  • Full rebuild. Recompute the table from every check-in, dropping those with a same-branch child. A check-in with no parent at all is included when it has no same-branch child. Fossil's leaf_rebuild() takes candidates from plink child ids, so this is a deliberate normalisation. Reserve it for explicit rebuilds.
  • Incremental check. For one check-in: remove it if it has a same-branch child, otherwise insert it, tolerating a row already present.
  • Deferred scheduling. When a parent link is created or a branch tag applied or propagated, schedule a recheck of the check-in and each of its direct parents.
  • Batch application. Run every scheduled recheck and clear the schedule before the table is trusted. A commit's leaf changes become visible here.

§7.4Open and closed

A leaf is open if and only if it is in the leaf table and has no active closed row, meaning a tagxref row for the closed tag whose type is an assignment. Implementations MUST offer the test as an existence expression, which callers can negate to select open leaves, and as a check on one check-in (§7.4).

§7.5More than one open leaf

The open leaves of a branch are the leaf-table members that are not closed and share the branch. To decide whether a check-in is an ambiguous leaf, an implementation MUST first confirm it is a leaf, and if so test for at least one other open leaf on the same branch. The test reads the leaf table, so a table that has drifted gives wrong answers even when plink is correct (§7.5). A report of ambiguity SHOULD list every open leaf on the branch, newest first, with hash, time and rid, and mark the one the checkout is on (§7.5).

§7.6When to re-evaluate

Leaf status MUST be re-evaluated when a branch tag is applied directly to a check-in, and for every descendant that gains or loses a propagating branch tag. Both go through the deferred schedule (§7.6).

§7.7Leaves are not names

Leaf enumeration asks which check-ins have no same-branch descendant and whether any are open. Name resolution asks what a name refers to. A resolver MAY consult the leaf table, but the leaf table's rules MUST NOT be conflated with a resolver's precedence rules (§7.7).

§8Resolving a name

§8.1One name, one check-in

Name resolution turns a branch name, tag name or reserved keyword into a single check-in: the one carrying the active tag with the greatest event.mtime. It answers even when that check-in is closed, not a leaf, or both (§8.1).

§8.2The rule

For a plain name N, resolution MUST pick the check-in with an active sym-N tag and the greatest event.mtime. This is the last fallback, and it either succeeds or fails outright. Closed status and leaf status MUST NOT affect the choice: the query joins tag, tagxref and event data and nothing else (§8.2).

An implementation MAY first look at only the thirty most recently modified candidates, provided the result is identical to a full scan. When two candidates share the greatest time exactly, the query engine's row order decides and the result is implementation-defined (§8.2).

Type a branch or tag name. The tool runs the §8.2 query and, beside it, the §9 open-leaf query for the same name, so you can see when the two questions part.

Resolve a name

§8.3The order of forms

Before the plain-name rule, resolution tries these, in order (§8.3):

FormResolves to
rid:DIGITSThat rid, optionally checked against its recorded type. Undocumented.
tag:NAMEThe §8.2 rule, skipping any attempt to read NAME as a hash.
tipThe check-in with the greatest event.mtime, regardless of tags.
currentThe open checkout. Only when one is open.
prev, previousThe checkout's primary parent.
nextA child of the checkout, primary links first, then latest plink.mtime.
ckoutA placeholder for the working checkout itself, when the caller accepts one.
root:BRResolve BR, then the parent-branch check-in BR diverged from.
start:BRResolve BR, then the earliest check-in on its branch.
merge-in:BRResolve BR, then the most recent merge-in point on the branch it diverged from.
DatesThe latest event at or before an upper bound, below.
Hex, 4 chars up to a full hashThe artifact with that hash prefix. Two or more matches MUST be rejected as ambiguous (rid −1), never picked silently.
Anything elseThe plain-name rule.

Date forms are date:TIMESTAMP, a bare date-time, the deprecated utc:, and NAME:TIMESTAMP, which limits candidates to check-ins with an active sym-NAME. Each finds the latest event whose mtime is at or below an upper bound built from the input (§8.3):

Input precisionUpper bound
Date onlyYYYY-MM-DD 23:59:59.999
MinuteYYYY-MM-DD HH:MM:59.999
SecondYYYY-MM-DD HH:MM:SS.999
Fraction .1.200
Fraction .12.130
Fraction .123.124
Fraction .999.999

A fraction is padded to three digits with 9s and, unless it is already .999, advanced by one millisecond. The deprecated local: form skips the expansion and hands its text to julianday(). A portable timestamp MUST use YYYY-MM-DDTHH:MM:SS[.fraction]Z. Fossil also takes 8, 12 or 14 digits with an optional Z, and hyphenated forms its SQLite parser accepts; other SQLite syntax MUST NOT be treated as portable. Without a Z, the time is read as UTC or local according to Fossil's display setting (§8.3).

If nothing matches and the name ends in exactly one punctuation character, resolution MUST retry without it, so a name lifted from prose or a URL still resolves. A request for a branch type is rewritten to check-in type, and the result is moved to the branch's first check-in by the branch walk (§8.3).

§8.4The branch walk

root:, start:, merge-in: and branch-type requests share one walk: from the resolved check-in, follow primary parents toward the root, watching whether each ancestor still carries the same active branch tag. “Earliest still on branch” returns the last ancestor that does; “first ancestor off branch” returns the first that does not. It is a parentage walk, not another tag-and-mtime selection (§8.4).

§8.5When the name and the leaves disagree

Take a branch BR with an open, childless C1 at time T1, and a later C2 at T2 that is closed but carries the active sym-BR. Resolving BR MUST return C2. Open-leaf enumeration skips C2 and surfaces C1. Callers MUST NOT substitute one question for the other (§8.5). Section 10.2 shows this happening in a real repository.

§9Current trunk and forks

The current trunk is the main-branch name run through §8, with no special case. The setting is local: not an artifact and never synchronised. Everything after the name is artifact-derived, so current-trunk is a function of the artifact set and the local setting together. Two repositories with identical artifacts can disagree on it (§9).

A branch MAY have more than one leaf, and is then forked. Resolution against a forked branch neither refuses nor raises an ambiguity error; it returns the candidate with the greatest mtime (§9).

Whether a branch is open is a separate question. A check-in is the branch's open leaf, and implementations MUST treat it as such, only when all three hold (§9):

  • it carries an active branch tag whose value is the branch name;
  • it is in the leaf table;
  • it has no active closed tag.

A branch is reported open, with an open-as-of date and a short id, exactly when such a check-in exists. If a report must name one of several open leaves it MUST pick the one with the greatest mtime (§9).

A branch's merge target is read by following at most one plink edge from its representative check-in to a child and taking that child's active branch value. A branch has at most one merge target. When the representative has several children, the draft leaves the choice implementation-defined (§9).

Ties are left open on purpose. When candidates share the greatest mtime to full precision, an implementation MAY return any of them, callers MUST NOT rely on which, and two implementations MAY differ. The draft declines to add a tie-break by hash or id (§9).

§10Worked example: a fork with a closed leaf

Two repository states in which leaves and name resolution see the same graph differently. In both, closed status filters the default leaves view but does not change what whatis and update pick for trunk (§10).

§10.1The closed leaf is the older one

Trunk forks after c4, leaving two trunk leaves and one feature leaf. leaves lists the open trunk leaf 4167aef42f and the open feature leaf 210d9276e4 and omits the closed trunk leaf 7aa867c048. leaves --all adds it back. The closed check-in is still a leaf and still on trunk. whatis trunk and update trunk both give 4167aef42f. Closed status changed the leaf list but not what trunk resolves to (§10.1).

§10.2The closed leaf is the newer one

margin = 0.15in
boxht = 0.5
$gap = 0.3
$drop = 0.4

# Section 10.2: trunk forks; the newer leaf is closed.
FORK: box "fork point" bold "trunk" small wid 1.0
OPEN: box "b05cbea82b" mono bold "open · listed by leaves" small wid 1.95 with .w at FORK.e + ($gap, 0)
CLOSED: box "ee3a7eed71" mono bold "closed · trunk resolves here" small wid 1.95 fill 0x0C0003 with .nw at OPEN.se + ($gap, -$drop)

arrow from FORK.e to OPEN.w
arrow from FORK.s down until even with CLOSED then to CLOSED.w

# Horizontal position is mtime.
AXIS: arrow from (FORK.w.x, CLOSED.s.y - 0.3) to (CLOSED.e.x, CLOSED.s.y - 0.3)
text "later mtime" small rjust at (CLOSED.e.x, AXIS.y - 0.18)
§10.2 · trunk resolves to the newest check-in with sym-trunk, closed or not; leaves lists only the older open oneEdit in pikchr-studio

Now the open leaf b05cbea82b is older than its closed sibling ee3a7eed71. leaves lists only b05cbea82b; leaves --all lists both, newer first. Yet whatis trunk reports ee3a7eed71, and update trunk moves the checkout from b05cbea82b to ee3a7eed71 (§10.2).

§10.3What follows

Tooling that infers the target of update or whatis from leaves output alone will diverge when a closed check-in is newer than the open leaves (§10.3).

§12Security considerations

  • Crosslink state is computed locally and is never transmitted, negotiated or authenticated on its own. Tag and parent claims are whatever the artifact's producer asserted, so the state is only as trustworthy as the artifacts admitted (§12).
  • Implementations MUST treat it as derived data with no trust of its own, and MUST NOT give a tag or parent claim more weight than artifact admission already gave it. Crosslink state from an artifact that admission would reject MUST be rejected or discarded (§12).
  • The trust boundary is artifact admission, so the draft adds no separate mechanism. Weak admission checks weaken the derived view; correct ones bound it (§12).
  • Implementations SHOULD recompute the state from the admitted artifacts instead of caching it across changes, so a rejected or retracted artifact drops out without separate revocation logic (§12).

§11IANA considerations

None (§11).

draft-fossil-sync-protocol-00 · an illustrated reading

The Fossil Sync Protocol

Every Fossil repository is a set of hash-named artifacts that only grows. Sync gives each side what the other lacks, over ordinary HTTP POSTs, until neither has anything left to ask for. Each message is a list of short text lines called cards. This reading follows the draft section by section and replays a real clone of its own repository, one round at a time.

§1What sync does

A repository holds artifacts named by their hash, and the set only ever grows. Two repositories sync by exchanging the difference between their sets, so both end up with the union. It does not matter which side starts or when, because a union of grow-only sets comes out the same in any order (§1, citing the G-Set literature).

A repository can know an artifact's hash without having its content. The draft calls such a hash a phantom. Sync works in rounds: each round announces artifacts, turns unknown hashes into phantoms, asks for phantoms, carries content, and goes again while §3.7 says there is still work to do.

The client drives everything. It sends an HTTP request, reads the reply and decides whether to send another. The server keeps no state between requests (§1).

To see the rounds play out, watch a sync: a real pull or push between two copies of this draft's repository, animated message by message.

The draft covers transport, login, artifact and unversioned-file transfer, clone, configuration framing and termination. It leaves out how rows of the /user, /reportfmt, /shun, /concealed and /subscriber table groups are serialised, and implementations MAY omit those groups. Leaving out the /shun group does not remove the repository's local shun set, which still filters requests and admission (§1, §3.3, §7.7). Sections 3 to 11 are normative.

§2Words the draft uses

TermMeaning
PhantomA known artifact hash whose content is absent. §3.3 governs it.
CardAn LF-terminated operator line, optionally followed by a byte-counted payload.
RoundOne request and its reply. The round ends when the body runs out, with no terminal card.
Servercode, projectcodeThe servercode identifies one repository instance and is informational. The projectcode is shared by every copy of the project. A push or pull with a different projectcode MUST be refused.
igot, gimme“I have this artifact” and “send me this artifact”.
Release integerMAJOR × 10000 + MINOR × 100 + PATCH, so Fossil 2.0 is 20000. It is separate from the clone version.
Clone protocol versionThe VERSION on a clone card, which selects the transfer encoding. Version 3 uses cfile.
Unversioned fileRepository-local state outside the artifact set, synced with its own cards. Unversioned files are not artifacts and ignore the artifact hash policy and tag rules.
FossilizeThe escaping scheme from the Artifact Format, §4: \s for a space, \n for a line feed, and so on.

Branches, tags, leaves and the current trunk are views computed after artifacts are parsed. Sync carries only the artifacts they come from (§2). The draft uses MUST, SHOULD and MAY in the BCP 14 sense, and only when capitalised.

§3The sync loop

The client sends one HTTP request body full of cards. The server handles that body on its own and returns one reply body of cards. The client reads the reply and uses it to build the next request, if there is one. No card says “done”. A round ends when its body is used up, and the session ends when §3.7 finds no reason for another round. The loop has four parts (§3.2, §3.3, §3.7):

  1. Build the request. Start with login, version and the chosen operation, add remembered state such as the cookie, add the cards the last reply called for, and end with random bytes.
  2. Exchange cards. igot announces artifacts, gimme asks for missing ones, file and cfile carry content. The unversioned-file cards run the same announce, ask and send cycle for unversioned files.
  3. Repair phantoms. A side that learns a hash without its content records a phantom, and later rounds ask for eligible phantoms.
  4. Decide whether to continue. After reading the reply, the client counts the round and applies §3.7. New work drives another round. The server never acknowledges anything.

§3.2What a round carries

Every request MUST contain pragma client-version, and it MUST come before any card that asks for content, so the server can apply the SHA3 gate (§7.6) when it handles them. A body login comes before it. So does pragma send-private in the first round of a private sync, though later rounds put it after. The first round of a clone sends no login of either kind.

Push, pull and sync requests repeat push, pull or both in every round. A clone starts with clone 3 1. A request MAY also carry gimme, content, igot, configuration and unversioned cards, and the returned cookie, and it MUST end with a random comment. A normal reply ends with a # timestamp comment. Replies cut short before or during dispatch can leave that comment out.

§3.2How the client reads a reply

After undoing the transport encoding, the client takes the reply's cards in order:

CardWhat the client does
push, pullRecords the peer's servercode and projectcode. Its own operation stays the same.
igotApplies the phantom rules of §3.3.
file, cfileStores content per §7. Sets content-received only if content was stored or a dangling delta created a source phantom.
gimmeNotes that the peer lacks the artifact. If push is still on, the client holds the content and policy allows, it appends the content to the next request.
uvigot, uvfileRuns the unversioned-file state machine (§8.5.1, §7.3). A uvgimme in a reply is the wrong direction (§5.2).
clone_seqno NRecords N if it is decimal and otherwise keeps the old value.
error, cookie, #…Applies §10.1, replaces the stored cookie, or applies the comment rules of §5.3.

When the loop over cards ends, the client counts the round, evaluates §3.7, and only then throws away the state it kept for that round.

§3.1What each side remembers

Table 1 of the draft lists the client's state by how long it lasts. The project code, the phantom set and the cookie persist in the repository. The peer's version and date, the clone sequence number, the round number and the unversioned send table last for the session. Some flags live for one round only: whether an igot created a phantom, whether content arrived, which hashes the peer has or lacks, and the unversioned and private counters. Round state is reset when a round starts and kept until §3.7 has been evaluated. The server keeps its repository plus a few things scoped to one request, such as whether it already sent the unversioned catalog.

Cards for the next request are collected while the reply is still being read. The file and delta counts that §3.7(b) checks are the content cards collected in that window (§3.1).

§3.2.1The order of cards in a message

login USER NONCE SIGNATURE     when named login applies; never in the first clone round
pragma send-private            first private round; later rounds put it after client-version
pragma client-version …        every request, before any card that asks for content
push / pull, or clone 3 NEXT   fixed operation; clone reissue is appended while reading the reply
cookie TEXT                    later rounds, straight after the operation
gimme, file, config, uv cards  as applicable
igot                           as applicable; never during clone
# random                       every request, last

A reply opens with pragma server-version where it applies, then any push, pull, igot, content, gimme and unversioned cards. A clone reply carries its cfile cards, then clone_seqno NEXT, then push. An ordinary reply ends with the # timestamp comment (Table 3).

When cookie login is in use (§6.5), the login travels in the HTTP Cookie: header and the body has no login card. A clone that the server refuses for lack of Clone permission gets a push card and then an error, with no cfile or clone_seqno (Table 3).

∗Replay an exchange

Pick an example from Appendix A to step through it a round at a time. Each card is shown next to the section that defines it. Long hashes are shortened, and payloads are summarised by size. Once this page's repository has loaded, Run a real clone clones it into an empty repository in memory and replays the rounds that clone actually exchanged.

Round by round
Requestclient to server
Card§Meaning
Replyserver to client
Card§Meaning

§3.3How phantoms are born and die

An igot HASH creates a phantom only if the receiver lacks HASH and can get it in the current direction: pull, clone, or receiving a push. A delta whose source the receiver lacks creates a phantom for that source.

In each round the client tracks which hashes the peer has announced or sent and which it has asked for. New phantoms can be requested at once. The scan for phantoms to request skips hashes the peer lacks, hashes in the shun table, and private hashes unless private sync is on. A shun-sha1 policy does not by itself suppress a gimme.

On gimme HASH, a side that holds the content sends it now or in a later message, if the direction and policy allow. If the direction forbids sending, it sends nothing and no error, and does not defer. If it lacks the content, it skips the request silently.

Storing the content deletes the phantom. A delta whose source is missing is kept under its own HASH, its phantom is deleted, a phantom is created for the source, and content-received is set. HASH is not requested again. The target can be rebuilt once its source chain arrives (§3.3, §7.4).

§3.4Clone uses the same loop

A modern clone runs the ordinary loop and §3.7, sending clone 3 SEQNO while it transfers and requests with no operation card after the sequence reaches zero. §8.1 to §8.4 hold the rules.

§3.5Pragmas

Pragmas are ordinary cards that keep their place in the body. Unknown pragma names MUST be ignored. §9 defines the known ones.

§3.6The cookie card

A server MAY send cookie TEXT. The client MUST replace its stored value with it and MAY send it back later. Servers MUST NOT depend on it and MUST tolerate cookies they did not issue and more than one cookie. Clients need not return one. This card is part of the body and is separate from HTTP cookie login (§3.2.1, §6.5).

§3.7When to stop

After reading a reply and counting the round, the client MUST send another round if and only if one of these holds:

LimbAnother round when
(a)A file or cfile stored content (a dangling delta counts), or an igot created a phantom, and the phantom table is still not empty. This applies to every operation.
(b)While reading this reply, the client appended at least one full or delta content card to the next request, or the unversioned upload phase is selected.
(c)This request sent a private igot HASH 1 and the round number is now exactly 1, or the reply's server-version made private inventory eligible and the client appended at least one such igot.
(d)This request sent a uvgimme and either the reply carried a uvfile or the round number is now below 3, or reading this reply appended a uvgimme.
(e)The operation is a modern clone, none of (a) to (d) fired, and this was the first clone round, or content was stored this round, or the sequence is still positive and the count of received artifact cards went up.

A positive clone sequence on its own does not keep a stalled session going. Remaining phantoms on their own MUST NOT cause another round or an error. If nothing holds, the client stops, and the server sends no completion card. If a round is needed but clone reissue is barred, the client sends the cleanup request of §8.2.

§3.8The client as a state machine

The draft restates the loop as five named states, in the style TCP uses. Where the state machine and a controlling section differ, the section wins. The round counter starts at zero just before the first request and goes up only after a reply has been read.

margin = 0.15in
boxht = 0.5
boxwid = 1.6
$col = 2.1
$row = 1.05

# States on a three-column grid, centred on EXCHANGE.
INIT: box "INIT" bold "no round sent" small
CLONE_ROUND0: box "CLONE_ROUND0" bold "first clone round" small at INIT + (-$col, -$row)
EXCHANGE: box "EXCHANGE" bold "repeats while §3.7 holds" small at INIT + (0, -$row)
TERMINATED: box "TERMINATED" bold "no condition holds" small at EXCHANGE + ($col, 0)
ABORTED: box "ABORTED" bold "fatal failure" small at EXCHANGE + (0, -$row)

# Connectors last.
arrow from INIT.w left until even with CLONE_ROUND0 then to CLONE_ROUND0.n
text "clone 3 1" small at ((INIT.w.x + CLONE_ROUND0.x) / 2, INIT.y + 0.15)
arrow from INIT.s to EXCHANGE.n
text "push · pull · sync" small ljust at (INIT.x + 0.12, (INIT.s.y + EXCHANGE.n.y) / 2)
arrow from CLONE_ROUND0.e to EXCHANGE.w
text "reply" small at ((CLONE_ROUND0.e.x + EXCHANGE.w.x) / 2, EXCHANGE.y + 0.15)
arrow from EXCHANGE.e to TERMINATED.w
text "none" small at ((EXCHANGE.e.x + TERMINATED.w.x) / 2, EXCHANGE.y + 0.15)
arrow from EXCHANGE.s to ABORTED.n
text "fatal" small ljust at (EXCHANGE.x + 0.12, (EXCHANGE.s.y + ABORTED.n.y) / 2)
arrow from CLONE_ROUND0.s down until even with ABORTED then to ABORTED.w
text "transport failure" small at ((ABORTED.w.x + CLONE_ROUND0.x) / 2, ABORTED.y - 0.15)
§3.8 · the client's five states; EXCHANGE repeats while any §3.7 limb holdsEdit in pikchr-studio

A first clone round sends no login and no configuration or unversioned hash, and it ignores any error in the reply. It always moves to EXCHANGE, which forces a second clone round. In EXCHANGE, an unknown or malformed reply card logs a local warning and ends only that reply's card loop. What was already parsed stays in effect, and §3.7 is evaluated as usual (§3.8, Table 5).

§3.9A narrow tolerance

Receivers tolerate only the cases the draft names: unknown pragmas, extra tokens on a few cards, one byte after a config payload, other whitespace between tokens, foreign cookies, errors in the first clone reply, and tokens beyond the sixth. The draft deliberately rejects the TCP habit of liberal acceptance, on the grounds that guessing at malformed input lets implementations drift apart. Fossil's tokenizer exposes at most six tokens per line, so senders MUST NOT emit a seventh, and receivers ignore any they find.

§4Transport

Sync is an HTTP POST to the repository URL, unchanged, with an empty path sent as /. Fossil's own client sends HTTP/1.0. Clients MUST NOT append /xfer. A request typed application/x-fossil or application/x-fossil-debug reaches sync whatever its path, and a request that is not a POST is redirected to the home page. Every request MUST carry an exact Content-Length and has a non-empty body. A reply MAY omit it, and Fossil then reads to end of file. Fossil ignores Transfer-Encoding. Over SSH the request adds X-Fossil-Transport: SSH.

  • Redirects. Follow at most 19 redirects with status 301, 302, 307 or 308; the twentieth is fatal. Never go from HTTPS to HTTP, and never redirect SSH or local-file transports. Strip any trailing /xfer from the target and change nothing else in the path. The client resends the same card body with fresh compression, headers and login for the new target. It does not reset peer state or choose cards again.
  • Status. Only HTTP 200 counts as a protocol reply. Any other status aborts before card parsing.
  • Content type of the reply. A missing type means compressed application/x-fossil. A value is matched case-insensitively by prefix: application/x-fossil-debug or -uncompressed selects the uncompressed body, any other application/x-fossil prefix selects compressed, and any other value is rejected. Fossil's own server decompresses a request body only when its type is exactly application/x-fossil.

§4.1Compression and media types

Media typeBodyUsed for
application/x-fossilCompressed containerAll production requests and ordinary replies.
application/x-fossil-debugPlain cardsDiagnostics only. It MUST NOT be used in production.
application/x-fossil-uncompressedPlain cardsClone version 3 replies. The cfile payloads inside are already compressed.

The compressed container is a 4-byte big-endian count of the uncompressed length, followed by a zlib stream: a 2-byte header, a DEFLATE body and a 4-byte Adler-32. It is neither raw DEFLATE nor gzip.

margin = 0.15in
boxht = 0.5

# On the wire: the length prefix, then the zlib stream.
LENGTH: box "length" bold "4 bytes, big-endian" small wid 1.6
HEADER: box "header" bold "2 bytes" small wid 0.9
DEFLATE: box "DEFLATE body" bold "RFC 1951" small wid 1.7 fill 0x0C0003
ADLER: box "Adler-32" bold "4 bytes" small wid 0.9

# The zlib stream, bracketed above its three parts.
$by = HEADER.n.y + 0.18
line from (HEADER.w.x + 0.05, HEADER.n.y + 0.06) up 0.12 then right until even with ADLER.e - (0.05, 0) then down 0.12
text "zlib stream, RFC 1950" small at ((HEADER.w.x + ADLER.e.x) / 2, $by + 0.15)
line from (LENGTH.w.x + 0.05, LENGTH.n.y + 0.06) up 0.12 then right until even with LENGTH.e - (0.05, 0) then down 0.12
text "capacity, not a check" small at (LENGTH.x, $by + 0.15)

# Decoded: the card text, under the body it inflates from.
CARDS: box "card text" bold "pragma client-version …" mono small wid 1.7 with .n at DEFLATE.s + (0, -0.55)
arrow from DEFLATE.s to CARDS.n
text "inflate" small ljust at (DEFLATE.x + 0.12, (DEFLATE.s.y + CARDS.n.y) / 2)
§4.1 · the compressed container: a length prefix, then a zlib stream that inflates to the card textEdit in pikchr-studio

The sender MUST write the real input length, but the receiver treats it as a capacity. After a good decode the receiver uses the bytes it actually got and MUST NOT reject a body for being shorter. Bytes after the zlib stream are ignored. If the zlib data is malformed, Fossil keeps the original bytes and the sync path parses them as cards anyway. Ordinary compression uses zlib's default level, an inline body login uses level 9, and receivers MUST handle any level. Stored blobs use the same container and are retransmitted byte for byte.

§4.2A round is one POST

A round is one POST and its reply, and it ends when the reply body runs out. There is no done or end card, and running out is not an error.

§4.3The cookie card

The cookie TEXT card travels inside the decoded body. HTTP cookie login (§6.5) is a different mechanism.

§5Cards

§5.1Lines and counted payloads

A body is a sequence of LF-terminated lines. Some cards announce a byte count, and that many bytes follow straight after the card's LF. The payload can contain LF, and it ends only at the count. A cfile sender adds one more LF only when the compressed payload does not already end in one, and receivers read that pad as an empty line. A config payload is always followed by one LF (§9.3).

margin = 0.15in
boxht = 0.5
$c1 = 1.55
$c2 = 2.0
$c3 = 0.55
$c4 = 1.45
$gap = 0.35

# A file card: its payload ends at the count, LF and all.
FILE_CARD: box "file H 26↵" mono bold "card line" small wid $c1
FILE_BODY: box "payload" bold "26 bytes, LF allowed inside" small wid $c2 + $c3 fill 0x0C0003
FILE_NEXT: box "gimme H↵" mono bold "next card" small wid $c4

# A cfile card on the same grid, padded when its payload lacks a final LF.
CFILE_CARD: box "cfile H U C↵" mono bold "card line" small wid $c1 with .nw at FILE_CARD.sw + (0, -$gap)
CFILE_BODY: box "payload" bold "C compressed bytes" small wid $c2 fill 0x0C0003
CFILE_PAD: box "↵" mono bold "pad" small wid $c3 dashed
CFILE_NEXT: box "igot H↵" mono bold "next card" small wid $c4
§5.1 · a payload ends at its declared count; cfile adds one LF only when its payload lacks oneEdit in pikchr-studio

§5.2Card syntax

A canonical card has tokens separated by one space, no leading or trailing whitespace, and one LF. Operators are lowercase ASCII letters, except clone_seqno. Fossilized arguments MUST be decoded. Senders MUST emit the canonical form. Receivers MUST accept it, and MAY also split on runs of space, tab, VT, FF or CR. There is no maximum line length (§5.2, §11.3).

What happens to a bad card depends on which side reads it and how far it gets:

CaseServer reading a requestClient reading a reply
Unknown operator, wrong arity, or a card sent in the wrong directionDiscards the reply built so far, emits error bad command: LINE, and goes on with later cards. Earlier changes are not rolled back.Logs a local warning, sends nothing, and stops reading this reply. The cards before it still count, and §3.7 is evaluated.
A file, cfile or uvfile handler rejects its cardDiscards the reply so far, emits the handler's error, counts it, and stops reading the request.Logs a warning, keeps the processed prefix, and continues as usual (§10.1).
Extra tokensIgnored beyond the six-token vector. private ignores any arguments. igot HASH X is private only when X is exactly 1, and public otherwise.

No “unknown message” reply text exists (§5.2, Appendix C).

§5.2.1The nineteen operators

login push pull cookie clone file cfile uvfile private igot uvigot gimme uvgimme reqconfig config clone_seqno pragma error message

§5.3Comment lines

A line starting with # is never dispatched as a card. Servers skip it. Clients skip it too, except for the timestamp line that ends a reply:

# timestamp 2026-07-02T18:31:13 errors 0

The timestamp is exactly 19 bytes of UTC, used to estimate clock skew. The error count has no leading zeros and is informational. Senders MUST use this form. Every request ends with a comment of random bytes, which changes the part of the body the login nonce covers. When a server answers pragma req-clusters, a non-empty run of cluster igot cards MUST sit between # sending-clusters and # end-of-clusters. Receivers ignore both lines.

§6Logging in

§6.1The login card

login USER NONCE SIGNATURE

USER is fossilized. NONCE and SIGNATURE are each 40 lowercase hex digits. A request may carry one login, in the body or in the cookie header. A second is fatal, and the server discards its output and emits an error. The users anonymous and nobody, and SSH connections, need no named login. The first clone round sends none.

§6.2The nonce

NONCE is the SHA-1 of every decompressed body byte after the login line's LF, through the end of the body. With cookie login it covers the whole body and none of the HTTP header.

§6.3The shared secret

shared-secret = SHA1(project-code "/" login-name "/" plaintext-password)

The result is 40 lowercase hex digits, and the login name is exactly USER. The calculation applies once the project code is known. The first clone round sends no login card.

§6.4The signature

SIGNATURE = SHA1(NONCE-hex || shared-secret-hex)

The random comment at the end of each request makes every nonce different. The tool below starts with the values from Appendix A.5.

Compute a login

§6.4How the server checks a login

Before reading any card, the server works out the caller's baseline capabilities from its session, cookie, HTTP Basic or remote-user setup. A successful login can only add capabilities to that baseline. A failed one leaves the baseline alone and ends processing.

  1. nobody and anonymous succeed without a signature, and a configured matching remote user is honoured.
  2. For a named user, the server treats the stored credential bytes as the shared secret and compares SHA1(NONCE || stored) with SIGNATURE, in constant time and case-sensitively.
  3. If that fails and the stored credential is not exactly 40 bytes long, it derives the secret from the stored cleartext, the name and its own project code, and tries again.
  4. Separately, NONCE must equal the SHA-1 of the body tail. A stored credential of exactly 40 bytes never gets the cleartext retry, so deployments that need that retry MUST avoid 40-byte credentials.

Either failure keeps HTTP 200, discards the staged output, emits error login failed and stops reading cards. Fossil keeps no replay cache. A login replay for pull alone is not rejected, while a replayed change can collide with the unique nonce column in the receipt table (§6.4, §11.1).

§6.5Cookie login

Newer servers take the login in an HTTP cookie. The client uses it only when the recorded server version and build time pass this gate:

VERSION >= 22700 AND (DATE > 20250727 OR (DATE == 20250727 AND TIME >= 110500))

Cookie: x-f-l-c=login+USER+NONCE+SIGNATURE

Then the client MUST leave out the body login. Spaces become +, and URL-unsafe bytes of the user name are percent-encoded. If the gate fails or no server-version has arrived yet, the client uses body login. Servers MUST accept body login whatever the gate says. Sending both breaks the one-login rule. Appendix A.3 shows the switch: round 1 logs in in the body, and round 2, after the server reported 22800 from 2026-03-11, logs in by cookie.

§6.6Capabilities

LetterCapabilityNeeded for
oReadPull and discovery
iWritePush
gCloneClone
xPrivatePrivate content
aAdminConfiguration and user management

The server checks each card's capability and emits an error when it is missing, unless the card's own section says to stay silent. The commit-lock pragmas follow only their lock-table rules (§9.4).

§7Moving content

§7.1file

file HASH SIZE
file HASH DELTASRC SIZE

SIZE bytes follow, uncompressed. The short form carries the whole artifact. The long form carries a Fossil delta against DELTASRC. Nothing follows the payload. Push and pull use file.

§7.2cfile

cfile HASH USIZE CSIZE
cfile HASH DELTASRC USIZE CSIZE

CSIZE frames the payload, which is the repository's stored compressed artifact or delta, sent byte for byte. USIZE is the full artifact's size, even in the delta form. It is parsed but not checked by the transfer layer. Clone version 3 always uses cfile, with no size threshold, pragma or capability negotiation.

§7.3uvfile and the unversioned cards

uvfile NAME MTIME HASH SIZE FLAGS
uvigot NAME MTIME HASH SIZE          server to client
uvgimme NAME                         client to server

NAME is fossilized and MTIME is a non-negative Unix second. HASH is the content hash, or - for a deletion. Flag 0x0001 marks deletion and 0x0004 means the content is omitted. Senders MUST clear the other bits, and receivers ignore them, so 6 reads as 4. The canonical deletion is HASH -, SIZE 0, flag 1.

Before applying a received uvfile, the receiver MUST recompute its status from the current local row using §8.5.1 and MUST NOT reuse a status it computed for an earlier uvigot. Sending unversioned changes to a server needs UV Write. When a server gets uvgimme NAME it looks the name up and sends nothing if it is absent, or the SHA3 version error for a SHA3 hash and an old client. If its reply has reached the output cap it sends uvigot instead, which brings another request. Otherwise it sends the full uvfile.

§7.4Deltas whose source is missing

A missing DELTASRC does not cause rejection. The receiver keeps the delta under HASH, deletes HASH's phantom and creates a phantom for DELTASRC, private or public to match the content card. HASH is recorded as dangling and is rebuilt later, once its source is available. The draft is explicit that nothing guarantees a later check of HASH against the rebuilt bytes. Fossil handles the source before it can notice that HASH is already stored, so a source phantom can be left behind even when HASH turns out to be a duplicate.

§7.5Private content

A bare private card marks the next file or cfile as private. Other cards and comments do not use up the mark. In a reply it always sets the mark. In a request it sets the mark only if the sender has the Private capability. Otherwise the server emits its authorization error and the content card that follows is treated as public. A client's mark lasts until used or the session ends, and a server's lasts until the end of its request.

  • A sender MUST NOT send a delta against a private source to a peer without Private permission. It sends full content instead.
  • A missing source for a marked delta becomes a private phantom, and for an unmarked delta a public one.
  • igot HASH 1 from a peer without permission is not an error. It creates no phantom but marks an existing HASH private.
  • Pulling private content needs a successful pragma send-private.

§7.6The SHA3 gate

Never send a file or cfile named by a SHA3-256 hash, or a uvfile with a 64-digit hash, to a peer whose recorded VERSION is below 20000. The server treats an unknown client as version 0 until client-version arrives. The client assumes the server runs its own version until it sees server-version. At each refused artifact the sender emits

error Fossil\sversion\s2.0\sor\slater\srequired.

and carries on with the other cards. It never substitutes a SHA1 name. Outside the first clone reply, a server error ends the client's card loop for that reply and §3.7 is evaluated. In the first clone reply, the error is ignored and the client makes its second clone request. When a sync runs automatically after a commit, an error matching %not authorized to write% marks autopush as failed and parsing continues.

§7.7Hash policy on arrival

The hash policy checks the received HASH, not its DELTASRC. Under shun-sha1, a non-clone file with a SHA1 name is read and thrown away, with no error and no change to the repository. A cfile with a SHA1 name is thrown away even during clone, though it still counts as a received card for §3.7(e). The draft points out one dead end. A delta can arrive against a SHA1 source, the client asks for the source, the source is discarded, and the session ends with that phantom unresolved.

Integrity is checked on the rebuilt artifact bytes, not on the wire payload. A repository's commit hook MUST rehash each newly stored artifact whose full bytes are available and abort the commit on a mismatch. Storing and verifying a SHA3-256 artifact under the auto policy permanently moves the repository to sha3.

§8Clone

A clone begins with pragma client-version, clone 3 1 and the random comment, with pragma send-private first when a private sync is requested. The first request carries no login. Later ones use normal authentication.

§8.1Clone version

clone VERSION SEQNO

In version 3, the profile the draft specifies, the server sends cfile cards in storage order with the type application/x-fossil-uncompressed. Only version 3 is interoperable; the draft leaves other versions out of scope. A SEQNO of 0 gets error invalid clone sequence number, a rollback and nothing else, so clients MUST send a positive decimal SEQNO.

§8.2The sequence cursor

SEQNO is a RID, a row number in the server's blob table. The server serves rows from that RID upward and skips missing numbers silently, so clients MUST NOT guess a count or size from the range. An accepting reply carries cfile cards in RID order, with any per-artifact SHA3 errors in place, then clone_seqno NEXT, then push SERVERCODE PROJECTCODE and the closing comment.

margin = 0.15in
boxht = 0.45
boxwid = 0.7

# The server's blob table by RID. RID 4 is a gap.
RID_1: box "1" bold fill 0x0C0003
RID_2: box "2" bold
RID_3: box "3" bold
RID_4: box "4" bold dashed
RID_5: box "5" bold
RID_6: box "6" bold fill 0x0C0003
RID_7: box "7" bold
RID_8: box "8" bold
text "RID" small rjust at RID_1.w - (0.12, 0)

# Each round's range, bracketed below.
$by = RID_1.s.y - 0.18
line from (RID_1.w.x + 0.04, RID_1.s.y - 0.06) down 0.12 then right until even with RID_5.e - (0.04, 0) then up 0.12
line from (RID_6.w.x + 0.04, RID_6.s.y - 0.06) down 0.12 then right until even with RID_8.e - (0.04, 0) then up 0.12
text "clone 3 1" mono bold "cfile 1–3 and 5, clone_seqno 6" small at ((RID_1.w.x + RID_5.e.x) / 2, $by - 0.24)
text "clone 3 6" mono bold "cfile 6–8, clone_seqno 0" small at ((RID_6.w.x + RID_8.e.x) / 2, $by - 0.24)
§8.2 · the clone cursor walks the blob table by RID, skipping gaps (an example split into two rounds)Edit in pikchr-studio

The client reissues clone 3 NEXT only if §3.7 says to continue, the reply had a well-formed push, and NEXT is positive. NEXT zero means the server has nothing more, and it bars any further clone, pull or push cards in the session. If §3.7 still wants a round, the client sends a cleanup request with no operation card: login if needed, client-version, pragmas, leftover reqconfig, gimme or uvgimme, the cookie and the random comment. With no direction in force, a gimme in cleanup MUST produce neither content nor error, so any phantom left at that point stays unresolved.

SEQNO := 1
first request := [send-private]; client-version; clone 3 1; randomness
loop:
  send; process reply in order; remember whether a well-formed push came
  if first clone round and error: record any later push, ignore the error,
     keep parsing; the second clone round is mandatory
  if clone_seqno NEXT is decimal: SEQNO := NEXT
  evaluate §3.7
  if continue and push seen and SEQNO > 0: next request has clone 3 SEQNO
  else if continue: next request is the operation-less cleanup
  else stop

Clone rounds have no push direction, so a server ignores any igot a cloning client sends, and a client SHOULD NOT send them. A server MUST accept and ignore igot cards that arrive without push.

§8.3The first clone round

The first request has no login. Any error in its reply is ignored as an error, whether it is about private access, Clone permission or the SHA3 gate. The client reads the rest of the reply, records any push, keeps its unversioned send table as it is, and sends the mandatory second request. Appendix A.2 shows this with a client too old for SHA3 artifacts.

§8.4Configuration during clone

The client holds back reqconfig and uv-hash until after round 0. Once Clone permission is confirmed, a server with unversioned sync enabled sends pragma uv-pull-only and its full unversioned catalog straight away in the clone reply, without waiting for the client's hash.

§8.5Push, pull and sync

push SERVERCODE PROJECTCODE, pull …, or both choose the direction. A projectcode mismatch is fatal. Pulling turns the server's igot cards into client phantoms. Pushing turns the client's igot cards into server phantoms. gimme fetches the content.

§8.5.1Unversioned files

At the start of a session the client fills its send table with every local unversioned row that has a hash and is not marked mtime-only. Tombstones are left out. On the first eligible round, never clone round 0, it sends pragma uv-hash HASH once. HASH is the SHA-1 of one line per non-tombstone row, sorted bytewise by name:

NAME SP YYYY-MM-DD SP HH:MM:SS SP HASH LF

If the hashes match, the server sends no catalog. Otherwise, given Read permission, it sends pragma uv-push-ok or uv-pull-only and then one uvigot for every entry, tombstones included. For each uvigot the client compares the remote entry with its own row and picks one of six outcomes:

StatusWhenWhat the client does
0No local rowAsks for the file with uvgimme, unless the remote entry is a deletion
1Hashes differ and the remote is newerThe remote wins: asks for the content, or records the deletion
2Same hash, remote newerUpdates the local mtime only
3Same hash, same mtimeNothing
4Same hash, local newerSends its mtime, if uploads are allowed
5Hashes differ and the local copy is newerThe local copy wins: sends it, if uploads are allowed

Try a comparison:

Compare one unversioned fileLeave the local hash empty for “no local row”; use - for a deletion

When the times are equal and the hashes differ, the bytewise order of the hashes breaks the tie. Revert maps 4 to 2 and 5 to 1, and dry-run changes nothing. Statuses 0 to 3 remove the name from the send table. A pull-only session removes 4 and 5 too. Each qualifying uvigot appends one uvgimme, with no retry table. When remote content wins, the old local row is deleted at once and is not restored if the transfer later fails. During the upload phase the client sends each selected row from its send table and removes it. If the request grows past the usual output threshold, it stops there and §3.7(b) carries the rest into the next round.

Fossil can send a short, malformed uvigot during upload, and the server treats it as a bad command. The draft calls this a source defect, and a cleanroom sender MUST NOT emit it.

§8.6igot and gimme

igot HASH announces a public artifact, igot HASH 1 a private one, and gimme HASH asks for content. They can come in any order. Peers dated before 2020-04-13 get no private igot at all. The client holds its private inventory back until server-version tells it the peer's date, then appends it to the next request, and §3.7(c) sends that request.

§8.6.1Branches, tags and leaves are derived

No card carries a branch, tag, leaf or closed marker. Each peer derives them from stored artifacts per the Artifact Format and Repository State drafts. The current trunk also depends on each peer's own main-branch setting, so two synced peers can disagree about it. While phantoms remain, the views cover only what is present and can change later without any sync error.

§8.7Phantom records

§3.3 is authoritative.

§8.8Which phantoms to ask for

Each round filters gimme candidates by peer-lacks, shun and private as §3.3 says. The filters belong to the round and are never saved.

§8.9Loop termination

§3.7 is authoritative. The server keeps no state and signals no completion.

§9Pragmas

pragma NAME [ARGUMENTS] is an ordinary card. Unknown names MUST be ignored silently. A known name sent in the wrong direction is treated the same way and MUST NOT change state. A bare pragma with no name is an unknown card and gets error bad command. A known pragma whose condition fails does nothing and raises no error.

PragmaDirection§Purpose
client-version V D Tto server9.1Client release, date and time. Required in every request.
server-version V D Tto client9.1Server release, date and time. Sets the SHA3, private-igot and cookie gates.
uv-hash HASHto server9.2Hash of the client's unversioned catalog.
uv-push-ok, uv-pull-onlyto client9.2Whether the client may upload unversioned files. Precedes the catalog.
ci-lock HASH ID, ci-unlock IDto server9.4Take or release a check-in lock.
ci-lock-fail LOGIN MTIMEto client9.4Someone else holds the lock.
send-privateto server9.5Ask for private content. Needs x.
send-catalog, req-clustersto server9.5Ask for a full inventory, or for cluster igot cards.
client-url URL, req-linksto server9.5Tell the server where the client syncs from, and ask for alternative locations.
link URL ARG MTIMEto client9.5An alternative location for this repository, only in reply to req-links.
avoid-delta-manifeststo client9.5Send manifests in full for the rest of the session.

§9.1Version negotiation

A full pragma client-version VERSION DATE TIME sets the server's view of the client, with DATE and TIME as YYYYMMDD and HHMMSS, and makes the server answer with a full server-version. A short form with three or four tokens updates only VERSION and gets no answer. The client applies the same rule to server-version. Until one arrives it assumes the peer runs its own release, dated 00000000 000000. In Appendix A.1, 22800 is Fossil 2.28.00.

§9.2Unversioned-file pragmas

The client sends uv-hash at most once, and never in clone round 0. A matching hash gets no response. On a mismatch, a server with Read sends exactly one uv-push-ok if the caller may write unversioned files, or one uv-pull-only otherwise, and then the full catalog. uv-pull-only drops the client's local-only status 4 and 5 entries. The canonical hash is SHA-1, but the server also accepts an artifact hash such as SHA3-256 and treats a mismatch as usual.

§9.3Configuration

reqconfig /NAME asks for a configuration group. Without Read the server ignores it, and an unknown group gets no cards and no error. Clone holds these requests back. The answer is config GROUPNAME SIZE, exactly SIZE bytes, and one mandatory LF. A Fossil-compatible receiver consumes that byte without checking it. Scalar settings always arrive as config /config SIZE with this payload:

MTIME SP 'name' SP value SP 'value'

Values are wrapped in ' with inner quotes doubled. Table groups use their own group name, and their row format is out of scope. A server MUST require Admin to apply a pushed config card, and otherwise emits error not authorized to push configuration. A config card whose SIZE is not plain decimal takes the unknown-card path. The four transfer-hook settings travel as scalar /config and run only on the server.

§9.4Check-in locks

pragma ci-lock HASH CLIENT-ID asks for a check-in lock on HASH, kept as a ci-lock-HASH row on the server. The server first deletes lock rows that have timed out or whose HASH is no longer a leaf. If a live lock on HASH remains and it belongs to a different client, the server replies pragma ci-lock-fail LOGIN MTIME. If it belongs to the same client, the server says nothing. Otherwise it writes a new lock and sends no acknowledgement. Silence after ci-lock confirms nothing. pragma ci-unlock CLIENT-ID deletes that client's locks and any invalid rows, again with no reply and no Write check. The client sends it whenever it has a stored ID and no current lock, so it may repeat.

§9.5Private content and discovery

  • send-private turns on private sync only for a caller with x. Otherwise it appends error not authorized to sync private content without discarding the reply. Clients repeat it in every private round.
  • send-catalog makes a read-authorized server announce every eligible artifact it holds, after the §3.3 filters.
  • req-clusters asks for public cluster igot cards, wrapped in the §5.3 comments.
  • req-links lets the reply to that request carry pragma link URL ARG MTIME. Without it the server MUST send no links. The client stores ARG verbatim and MUST NOT parse, change or forward it. Links are optional, and they say nothing about artifacts.
  • client-url URL is stored by a server whose caller has Write, as syncfrom: followed by a normalised URL, unless the host contains localhost. The draft spells the normalisation out in detail.
  • avoid-delta-manifests makes the client send full manifests, not deltas, for the rest of the session.

§10Errors and messages

§10.1The error card

error TEXT

TEXT is fossilized and MUST be decoded. HTTP status stays 200 throughout. On the server, each request that reaches card processing runs in one database write transaction, and errors fall into three kinds:

KindExamplesServer outputTransaction
AppendSHA3 gate, unauthorized send-privateError added, parsing continuesCommits at the end of the request
ResetWrong project, authorization, duplicate login, unknown cardPartial output discarded, then the errorEarlier changes still commit
Fail-fastinvalid clone sequence number, invalid config recordError and stopRolled back

A failure before card processing, such as a schema-out-of-date response, returns before any transaction opens. A refused clone is a reset site that still writes push before the error.

On the client, an ordinary server error commits the cards already processed, ends that reply's card loop and sends the client to §3.7. It ends the session only if §3.7 finds nothing to do. Errors in the first clone reply are dropped as errors while the rest of the reply is still read. A content or integrity error found by a handler gives a warning and processing continues as usual. Only an explicit fail-fast path rolls back the round and stops. A private mark already used up by a handler is not restored by a rollback.

§10.2The message card

message TEXT

A message has exactly two tokens, is decoded and is never fatal. A client MAY show it and MUST change no state, with one exception. If push is on and the text matches the glob pull only *, case-sensitively, the client turns push off for the session and hides the message. Cards already queued for the next request stay queued, but later gimme cards in the same reply MUST NOT append content. Later requests carry no push, while unversioned uploads stay allowed.

§11Security considerations

  • Body login sends a user, a nonce and a signature, never the password. HTTP Basic and proxy credentials are outside that property.
  • A sender MUST enforce the 20000 SHA3 gate and MUST NOT truncate, rename or downgrade SHA3 names (§11.4).
  • There is no login replay cache. A byte-for-byte replay with a valid signature is not rejected for that reason. The nonce stored with a change's receipt is unique, but that is a separate matter from replay protection (§11.1).
  • Nothing stops sync over plain HTTP, which exposes logins, private artifacts and cookie logins to anyone reading the traffic. Deployments SHOULD use HTTPS, SSH or another confidential, integrity-protected, server-authenticated transport (§11.2).
  • There is no per-line limit. Payload sizes come from declared counts and storage policy (§11.3).

§12IANA considerations

The draft registers three media types and needs no port or service name.

Media typeEncodingIntended usage
application/x-fossilzlib-compressed syncCOMMON
application/x-fossil-debugUncompressed sync, for debugging and testsLIMITED USE
application/x-fossil-uncompressedUncompressed clone reply with compressed cfile payloads; replies onlyCOMMON

App. AExamples

The examples show decompressed bodies. The replay tool in §3 steps through A.1, A.2 and the last scenario of A.3.

A.1Push and pull in two rounds

The client announces three artifacts. The server asks for two and announces two of its own, so the client makes two phantoms. In round 2 each side sends the other what it asked for, and no round 3 is needed. Anonymous push and pull are allowed, so no login appears. The first file payload in round 2 is a 272-byte manifest of seven lines, each ending in LF, and the second is the 26-byte line alpha content from client with its LF. The reply's payloads are exactly 270 and 31 bytes. The counts frame them, and the lines inside are payload, not cards.

A.2The SHA3 gate in a clone

A client reporting release 19999 asks to clone. The server refuses a SHA3 artifact with the version error, then sends clone_seqno 0 and a push. No SHA1 fallback happens, and the client still makes the required second request.

A.3Private content

An anonymous pull never sees private artifacts. A crafted gimme for a private hash gets an igot … 1 announcement. An unauthorized send-private gets an error and the public inventory. An authorized user logs in, pulls, and receives two private files, the second as a delta against a public artifact.

A.4Comment lines

Random comments are ignored as cards. The timestamp estimates clock skew, and errors 0 is informational.

A.5Authentication

The user, password, project code and covered body behind the three SHA-1 values loaded into the §6 tool.

App. BCollected grammar

An excerpt of the draft's ABNF (RFC 5234). Payload lengths come from the counts in the prose, which ABNF cannot express.

sync-message     = 1*( sync-element )
sync-element     = bare-card / file-transfer / cfile-transfer /
                   uvfile-transfer / config-transfer / comment-line
file-transfer    = file-card payload-octets
cfile-transfer   = cfile-card payload-octets [ LF ]
uvfile-transfer  = uvfile-card [ payload-octets ]
config-transfer  = config-card payload-octets LF

push-card        = "push" SP servercode SP projectcode LF
pull-card        = "pull" SP servercode SP projectcode LF
clone-card       = "clone" SP decimal SP decimal LF
clone-seqno-card = "clone_seqno" SP decimal LF
igot-card        = "igot" SP artifact-hash LF /
                   "igot" SP artifact-hash SP "1" LF
gimme-card       = "gimme" SP artifact-hash LF
file-card        = "file" SP artifact-hash SP decimal LF /
                   "file" SP artifact-hash SP artifact-hash SP decimal LF
cfile-card       = "cfile" SP artifact-hash SP decimal SP decimal LF /
                   "cfile" SP artifact-hash SP artifact-hash SP decimal
                   SP decimal LF
login-card       = "login" SP user SP nonce SP signature LF
error-card       = "error" SP fossilized LF
artifact-hash    = sha1-hash / sha3-hash   ; 40 or 64 lowercase hex
timestamp-comment = "# timestamp" SP timestamp SP "errors" SP canonical-decimal LF

App. CTwo corrections to older descriptions

Older text saysThe protocol actually
An unknown card gets an “unknown message” reply.Has no such text. A server discards its output, emits error bad command: LINE and carries on. A client logs a warning, stops reading that reply and evaluates §3.7.
Several logins in one request add up their permissions.Treats a second login as fatal. One successful login adds to the request's baseline capabilities only.
Version Show

Read from the repository once Fossil loads.

The draft's text, unchanged. Headings marked Illustrated open the explained version of that section.

delta-format.fossil · running in your browser

The draft, as a Fossil repository

The Draft text view reads its text from a real Fossil repository, opened here with go-libfossil, compiled to WebAssembly.

Loading Fossil (6 MB)
Saved as a .zip holding the repository. Unzip it and run fossil ui on the file inside.

Timeline

Every check-in that touched the draft, newest first. Each is a manifest: a small text artifact whose cards name the files, the parent and the author, and whose hash is the check-in's name. Choose one to read its manifest.

  1. Loading…

What changed between the drafts

Fossil's own diff between the two check-ins. In the draft view, "Mark what this version changed" uses the same history line by line.

How the repository stores the draft

Every artifact is stored zlib-compressed. Fossil also keeps older file versions as deltas against newer ones, so the newest text reads fastest. Older drafts here are not stored in full: each is a delta against a newer one, in exactly the format the Delta Format RFC specifies. Decode one below with that reading's own decoder.

ArtifactWhat it isSizeStoredStored as
Loading…