logfold

Collapse repeated log lines

logfold is a filter for the noisy part of a log: it prints each run of identical lines once and tells you how many there were. One file, no dependencies, MIT licensed.

Before

10:00:01 worker-2 connection refused
10:00:02 worker-2 connection refused
10:00:03 worker-2 connection refused
10:00:09 worker-2 connected

After logfold -n

10:00:01 worker-2 connection refused  (x3)
10:00:09 worker-2 connected

With -n timestamps, hex values and numbers are masked before lines are compared, so lines that differ only in those parts fold together. The first line of each run is kept as it was.

Quick start

$ curl -fsSO https://s3.f0ura.win/downloads/logfold-0.9.4.py
$ python3 logfold-0.9.4.py -n /var/log/app.log | less

It reads standard input too, so it fits at the end of a pipe: journalctl -u app | python3 logfold-0.9.4.py -n. See Install for checksums and for putting it on your PATH.