Duplicate Line Remover
Paste a list — duplicates disappear as you type. Your list never leaves your device.
Clean lists without a spreadsheet detour
Duplicate lines creep into every merged list: two exports of the same mailing list, tags collected from multiple pages, keywords pasted from three research sessions. The usual fix — paste into a spreadsheet, apply a dedupe function, copy back out — is a lot of ceremony for a one-column problem. This tool does the same job in the textarea itself, live on every keystroke.
The comparison is deliberately conservative by default: lines must match exactly, character for character. The checkboxes loosen that one axis at a time — fold case, ignore surrounding whitespace, drop blank lines — and each surviving line is always returned in its original spelling, first occurrence first.
Worked example
Paste this six-line shopping list: oat milk, bananas, coffee, oat milk, rye bread, coffee (one item per line). With default settings the result is four lines — oat milk, bananas, coffee, rye bread — and the counters read 6 lines in, 4 unique lines out, 2 duplicates removed. Order is preserved: each survivor sits where it first appeared.
Frequently asked questions
Does it keep the first or the last duplicate?
The first. The tool scans top to bottom and keeps a line the first time it appears, dropping every later repeat. That means the surviving line keeps its original position and its original spelling — even with case-insensitive matching on, you get back the form that appeared first.
Can I keep my list in its original order?
Yes — that's the default. Lines come back in first-appearance order with repeats removed. Only if you tick 'Sort output A→Z' does the tool reorder the result alphabetically.
Why aren't two 'identical' lines being removed?
Usually invisible whitespace: a trailing space or tab makes 'apple ' a different line from 'apple'. With the trim option off, that difference is respected on purpose. Tick 'Trim whitespace before comparing' to ignore leading and trailing spaces, and 'Case-insensitive' to treat Apple and apple as the same line.
How large a list can it handle?
The only limit is your browser's memory, because nothing is uploaded. Tens of thousands of lines dedupe in well under a second on an ordinary laptop; even hundreds of thousands are fine, though the textarea itself gets sluggish to scroll long before the deduping does.