What we set out to test
The previous post ended with pijul pulling over the mount fixed and one question left open: what happens when both peers edit? One-directional sync is the easy half. The owner writes, the reader follows. Real use is two people with the same folder mounted, both saving, sometimes into the same file. So we built two integration suites. One drives the write patterns programs use at the syscall level: rename-overwrite, truncate and rewrite, in-place writes, rename bursts. The other drives the real tools: vim, sqlite3, LibreOffice headless, ffmpeg, GIMP, pijul. Every scenario runs in both directions, and convergence means the md5 read through each peer's own mount matches, checked live on both sides.
How programs actually save
The traces settle it quickly. vim with default settings saves through a temp-and-rename dance. The same vim with -u NONE on Linux truncates the file to zero and rewrites it in place, both inside the same mtime tick. GIMP truncates and rewrites in place, no temp file anywhere, after a burst of probe opens that write nothing. LibreOffice writes a fresh zip container every save. Sublime Text builds its working copy on a different filesystem entirely and moves it back over the target; rename(2) cannot cross filesystems, so that move degrades to a copy into the target, which from underneath is truncate-and-rewrite again.
Almost nothing edits bytes inside a file. Editors either replace the file's contents wholesale under the same name, or build a new file and rename it over the old one. The exception is databases: sqlite rewrites B-tree pages in place with a journal beside them. Editors replace; databases edit. The previous post was about why the database class is hard. This one is about everything else.
The rename case is the hard one for a sync layer. A rename onto an existing name replaces that file's entire contents without a single write landing on it. Treat it as a delete plus a new file and you forget everything you knew about the target: which chunks the other peer already holds, and which version the replacement was based on. The cost is re-transferring whole files for one-line saves, and misjudging which of two concurrent saves was built on stale data.
Three bugs the reverse direction forced out
Running every scenario backwards, the reader editing the owner's files, surfaced problems the forward direction cannot hit.
First, the writer's own view can lie. vim's in-place save truncates to zero, which caches a size of zero, then writes the new content within the same mtime tick. Our attribute cache refreshed from disk only on a strictly newer timestamp, so it served size zero indefinitely. The reader converged on the new content while the writer's own mount showed an empty file. The fix is to state the obvious in code: the write that just happened is the newest truth about the file. The write path now raises the cached size and stamps the time itself. No extra syscall, nothing added to the hot path.
Second, announcement rules left over from the one-way era. "Never announce the peer's own files back at them" is correct until the reader legitimately renames a file the owner created; then the owner keeps the old name forever. Every rename arriving through the kernel is a local user action and is announced now. Renames driven by the peer enter through the sync service, not the kernel, so nothing echoes.
Third, accepting an update has to cancel your own pending announce for that path. Outbound announcements debounce for 200 ms. When the peer's version was accepted inside that window, the queued announce still flushed afterwards and refreshed its metadata from disk, and the disk now held the peer's content, so it announced their own bytes back at them with a fresh local timestamp. Both sides then believed they held a conflicting newer version. The Linux runs exposed this; the macOS kernel cache had been masking it.
Concurrent edits without trusting clocks
When both peers save the same file at the same time, one version becomes the canonical file, decided by last-writer-wins on announced time. But announced times order the announcements; they say nothing about what each writer had seen. "Edited after receiving my version" and "edited blind on a machine whose clock runs ahead" look identical in metadata. Deciding whether the losing version deserves to survive requires lineage.
Every announcement now carries a base: the newest version of that file the announcing peer had shown the world when its edit session began. A fresh create carries an explicit "no prior version". On acceptance, the receiver compares the winner's base against the version it has already made authoritative locally. A base older than that version means both sides edited concurrently, and the local bytes are preserved before the winner lands: renamed to name.conflict-<timestamp>, one metadata operation, no copy. The conflict file then syncs like any other file, so both peers end holding both versions.
The base a peer announces is a timestamp from its own clock, and it is only ever compared against other timestamps from that same clock, after a round trip. The two machines' clocks are never compared with each other, so clock skew between peers can neither forge a conflict nor hide one. And with exactly two peers, this single comparison carries the information a vector clock would in the general case: per file there are only two writers.
The suites prove the negative case too: no turn-taking flow, however fast the handoff, produces a conflict copy. Only writes landing inside the same debounce window do. The first run of the concurrency test found one more bug on top: both peers preserved within the same UTC second, minted the same conflict filename, and the two conflict files started overwriting each other, until one read back as 24 NUL bytes. Conflict names carry nanoseconds now.
Swap saves keep their transfer state
With lineage in place, the rename case gets the full treatment. The rename announcement declares which version the swapped-in file was based on, the target keeps its chunk tracking across the replacement instead of resetting, and the chunk-hash reconcile path transfers only what differs. Measured on a 24 MiB file where a save changed about 1 MiB: the other peer re-downloaded 128 KiB on Linux and 2 MiB on macOS, not 24 MiB.
Propagation for the rest: a first cold read of a fresh 512 KiB file lands in 30 ms. A save-swap reaches the other peer in 20 to 550 ms. In-place writes ride the 200 ms debounce and converge in 0.5 to 1.1 s. The reverse direction matches the forward numbers.
Where it stands
All of this is on the 0.4.0 branch, verified under the race detector on macOS and in repeated loops of the full suites on Linux. Open and known: two writers inside one mmap'd database stay unsupported over any network filesystem, ours included, the same stance as the previous post. If both peers announce before either accepts, the side that ends up rejecting cannot preserve bytes it never fetched; closing that window needs fetch-by-version. Two edits with identical nanosecond timestamps would each reject the other and never converge; we recorded that case for the model checker and expect never to see it. The GUI editor tier and the audio and video tools are next, on the same harness.