LogMerge

A distributed job leaves its story scattered across a dozen files: driver logs, executor logs, sidecars, rotated files and their .gz archives. LogMerge reads them all at once and writes every line in timestamp order, still traceable to its file, with stack traces kept under the line that raised them — at about 1 GB/s from disk on an M1 Max.

The whole trick is a min-heap with one entry per open file, keyed by its next line's timestamp: pop the earliest, stream it until its next line would overtake the heap, push back. A goroutine per file parses the first timestamp so the merge starts with a full heap; archives are read in place, an entry inside one addressable as archive.tar.gz!/name.log. Timestamps of many shapes — ISO 8601, ctime, date-time-millis — come out one width, and nothing allocates per line.

It is written in Go with no runtime dependencies, released as binaries for Linux, macOS and Windows on both architectures, built with profile-guided optimisation.