logfold

FAQ

How is this different from uniq -c?

uniq compares whole lines, so a log where every line starts with a timestamp never folds. logfold can mask the timestamp and the numbers first and still prints an original line for each run. It also leaves short runs alone with -m.

Does it keep the whole file in memory?

No. It holds one line at a time, so it works on files larger than memory and on endless streams such as tail -f. Output for a run appears when the run ends.

Why did my lines not fold?

Without -n, lines must match exactly. Try -n. If the lines still differ, something else in them changes, such as a request id made of letters and digits; masking covers numbers and hex with a 0x prefix only.

Does it change the order of lines?

Never. Only consecutive lines fold, and everything else is printed in the original order.

Which Python versions are supported?

3.8 and newer. The script uses the standard library only.

Where do I report a bug?

Write to bugs@s3.f0ura.win with the version (logfold -V) and a few input lines that show the problem.

What is the license?

MIT. You may use, copy and modify it, including in commercial work, as long as the copyright notice stays.