Sort and dedupe: why "10" comes before "9".

Every sorted list you've ever seen — file managers, spreadsheets, sort on the command line — is built on one primitive: compare two strings, decide which comes first. The default way to do that is brutally simple, and its simplicity is where all the surprises come from.

How text sorting actually works.

The default comparison is lexicographic: walk both strings character by character, and at the first position where they differ, the character with the smaller code number wins. That's the whole algorithm. It never asks what the characters mean — 9 is just code 57, A is just code 65, and the sort is a numeric comparison over those code numbers.

For plain lowercase words this coincides with dictionary order, which is why the default feels correct right up until it doesn't.

Trap one: "10" before "9".

What you wanted        What you got
file-9                 file-10
file-10                file-2
file-2  → sorted →     file-9

Compare "10" and "9" character by character: first characters are 1 (code 49) and 9 (code 57). 49 < 57, comparison over — "10" wins. The string sort never notices that "10" is a bigger number; it only sees that 1 is a smaller character.

Two fixes exist. The robust one is zero-padding at creation time: file-002, file-009, file-010 sort correctly under any dumb string sort forever, which is why logs and exports do it. The retroactive one is natural sort (a.k.a. numeric-aware sort): the comparator spots digit runs and compares them as numbers. File managers on every major OS do this, JavaScript offers it via localeCompare(..., { numeric: true }), and GNU sort spells it sort -V (version sort). Natural sort is a different order than plain sort — pick one and be consistent, or the same list sorts two ways in two tools.

Trap two: Z before a.

Banana
Zebra
apple      ← all lowercase lines sort after ALL capitalized ones

In ASCII, the entire uppercase alphabet (codes 65–90) sits before the entire lowercase alphabet (97–122) — a deliberate bit of 1960s layout (the case bit, covered in the ASCII guide linked above), with the two cases exactly one bit apart. So a case-sensitive sort doesn't interleave apple among the As; it exiles every lowercase line below every capitalized one. Tools differ silently on the default: JavaScript's Array.sort() and Python's sorted() are case-sensitive, while GNU sort's behavior depends on the locale environment — the same command gives different orders on different machines, a classic "works on my machine" in miniature. If you want dictionary-style ordering, you want an explicit case-insensitive mode (compare case-folded, display original).

Trap three: accents and locales.

Past ASCII, "correct order" stops being a fact and becomes a language opinion. By raw code numbers, every accented letter sorts after z — étude lands below zebra, which no French dictionary would accept. Proper collation (locale-aware comparison, localeCompare in JavaScript) fixes that, but the fix is locale-specific: German sorts ö with o, Swedish sorts it near the end of the alphabet — both correct, for their language. The practical guidance: for human-facing lists, sort with the user's locale; for machine-facing output that must be reproducible across systems (test fixtures, lockfiles, diffs), sort by plain code points on purpose and say so. Reproducible-but-ugly beats pretty-but-unstable when a diff depends on it.

Dedupe and the invisible characters.

Deduplication looks even simpler than sorting — keep the first copy of each line, drop the rest — and it inherits the same literal-mindedness: two lines are duplicates only if they're byte-for-byte identical. The ways "identical-looking" lines differ, in descending order of frequency:

Trailing whitespace. "apple" and "apple " are different lines. Invisible in every editor that doesn't render whitespace, and the number-one reason a dedupe "didn't work."

Case. Apple ≠ apple. Whether that's a duplicate is a decision, not a fact — which is why dedupe tools offer a case-insensitive toggle.

Line endings. A file assembled from Windows and Unix sources can hold apple\r and apple — identical on screen, different bytes. (The full story of \r\n lives in the ASCII guide, and a line-ending converter fixes it in one pass.)

Lookalike characters. A non-breaking space instead of a space, a zero-width space from a webpage copy-paste, a Cyrillic а standing in for a Latin a. And subtler still: Unicode can spell é as one code point or as e plus a combining accent — rendered identically, compared unequal, reconciled only by Unicode normalization (NFC/NFD).

Takeaways.

None of these are bugs — every tool is faithfully executing the literal comparison you asked for. The fix is always the same: decide which notion of "same" and "before" you actually mean, and make it explicit.