dedup
The Unix utility uniq takes a collection of lines and removes its adjacent
duplicates. This repository contains a program, dedup, that acts like uniq,
only it removes the adjacency condition; that is, it removes all duplicates
from such a collection. The program preserves the order of the input.
The implementation is naive and simplistic but, for my current purposes, it is good enough.
Installing
go install github.com/adamroyjones/[email protected]
Usage
# Deduplicate STDIN and print the result to STDOUT.
cat file | dedup
# Deduplicate the contents of `file` and print the result to STDOUT.
dedup file
# Deduplicate the contents of `file` and write the results to `file`.
dedup -w file
