Cleaning methodology

Every recorded change begins with a published rule.

CopyPrune is a deterministic text transformation pipeline. This page documents what it examines, what each mode permits, and where human review is still required.

Rules version: 2026.07 · Published: July 22, 2026 · Last substantive update: July 23, 2026

The pipeline

  1. Normalize Unicode artifacts. Line endings are standardized. Optional rules remove soft hyphens and selected invisible spacing characters, convert nonbreaking spaces, and expand common PDF ligatures.
  2. Classify lines. The cleaner identifies blank rows, likely list items, short headings, blockquotes, indented code, and table-like rows.
  3. Estimate visual wrapping. Typical line width and nearby punctuation help identify line endings likely caused by page layout rather than meaning.
  4. Protect structure. Blank-line paragraphs and enabled structural categories are kept before any prose joins are attempted.
  5. Apply permitted repairs. The selected mode and rule toggles determine which boundaries can change.
  6. Build the proof ledger. Each transformation records its type, rule, before/after specimen, confidence label, and approximate source line.

Careful mode

Careful mode joins a boundary only when multiple signals point toward a visual wrap—for example, a wrap-sized line without sentence-ending punctuation followed by lowercase continuation text. Likely end-of-line split words may be repaired. Uncertain boundaries remain in the result and appear under “Needs review.”

Standard and Flatten

Standard mode accepts more probable wrap boundaries while still protecting enabled structure. Its wrap operations are labeled medium confidence. Flatten mode first applies enabled text-normalization rules, including conservative split-word repair, and then replaces the remaining line and paragraph breaks with spaces. It should be used only when document structure is unwanted.

What the confidence labels mean

High, medium, and review-needed are qualitative rule tiers, not calibrated probabilities or accuracy percentages. They describe which deterministic conditions fired and how much human review the result deserves. They do not certify that a repair matches the author’s intent.

Safety invariants

Why copied PDF text can lose structure

Adobe’s accessibility documentation notes that character rendering order is not necessarily logical reading order, and its WordFinder documentation warns that location order is not generally reading order for multi-column text. CopyPrune works only after a PDF reader has exposed selectable text, so it does not claim to reconstruct page geometry or reading order.

Unicode and browser behavior

Unicode’s line-breaking specification defines the soft hyphen U+00AD as an invisible character that marks a possible intraword break. CopyPrune can surface and remove that character inside a reviewed word. The cleaner also standardizes CR, LF, and CRLF line endings before classification. Copy actions use the browser clipboard API only after the user requests them.

Known limitations

Copied text alone does not expose all PDF page geometry. The cleaner cannot reliably recover multi-column reading order, distinguish every legitimate compound from a split word, reconstruct tables, or remove repeated page headers when page boundaries are absent. Poetry, code, citations, legal clauses, and unusually formatted documents deserve particularly careful review.

Primary technical references

Use the documented workflows

Use the PDF Text Cleaner for mixed formatting artifacts, Remove Line Breaks Online for newline replacement and paragraph preservation, or Dehyphenate PDF Text for a review-first split-word workflow.

Change record

July 23, 2026: Clarified the qualitative confidence tiers, added primary technical references, and documented the current rules version. No cleaning rules changed in this documentation update.