About
Use this text tool to locate unique lines within a plain text document and eliminate duplicate entries, enhancing text clarity. This tool is especially useful for avoiding repetitive content in a list.
Case Sensitivity
When the case-sensitive option is enabled, duplicates will only be removed if they match the letter case of the original line. Disable this option for case-insensitive matching.
Related Tools
- Remove Spaces - Eliminate leading and trailing white spaces (spaces and tabs), and convert multiple spaces into a single space.
- Remove Empty Lines - Remove all lines in a text that consist solely of white space (spaces or tabs).
Remove repeated lines while keeping the first occurrence. Clean a list of labels, references, or values before further editing.
How to use
Paste one entry per line. Choose case sensitivity and trimming, then remove duplicates. Review the retained spelling and the number removed.
Example
Apple, Pear, and Apple on separate lines become Apple and Pear. Case settings determine how apple compares with Apple.
FAQ
Can spaces make two lines different?
Yes. Enable trimming to compare lines without surrounding spaces. Leave it off when indentation is meaningful.
A practical task
Choose the comparison rule before cleaning. Product codes can be case sensitive, while a list of ordinary labels might intentionally ignore case. Record which rule you used if the cleaned list will be compared with another dataset.
Another useful question
Which duplicate survives?
The first matching occurrence is retained. If different rows carry useful context, merge that context before removing duplicates rather than assuming later copies are disposable.
Details for interpreting the result
| Detail | Explanation |
|---|---|
| Order and sorting | Reversing changes the existing order from end to start. Sorting calculates a new order from values. A reversed list is only sorted if its previous arrangement and the chosen ordering rule happen to make that true. |
| Natural sorting | Natural sorting compares numeric parts as numbers. Character sorting compares their symbols. Labels containing 2 and 10 can therefore switch places between the two methods even when their text is unchanged. |
| Case sensitivity | Case sensitive matching distinguishes capital and lowercase letters. Ignoring case merges some of those distinctions, but it does not turn different spellings or word forms into the same term. |
| Trimming and content | Trimming removes characters from an edge rather than the middle. Whole text trimming differs from trimming every line. Preserve indentation in code or structured text when it carries meaning. |
| Whitespace characters | An ordinary space, a tab, a nonbreaking space, and a zero width space are different characters. They can look similar or invisible. Choose the exact kind of cleanup instead of treating all blank looking input as equivalent. |
