Tesseract.js reads table borders as letters. What fixed our OCR
Out of the box, tesseract.js reads a clean paragraph well and a real-world screenshot badly. Table borders come back as [ and |, a spreadsheet collapses into one column, and a photo with no text at all returns hundreds of confident-looking “words”. None of this needs a bigger model. It needs work before and after the recognition step. These are the fixes that moved Image to Text OCR from “copies garbage” to usable.
Remove grid lines before recognition
Tesseract treats a ruled table as text with strange characters between the words. Those characters also fill the gaps between columns, so a column detector sees one wide column. The fix is to erase the lines in the image first: find thin, long, nearly continuous ridges, paint them with the background next to them, and only then run OCR.
Two details matter. Crossbars and serifs of letters also line up along a text line, but with holes between the letters, so a line must be dense, not just long. And the stems of l, h, d and 1 in a large font are thin and tall too; a vertical segment counts as a border only if both ends reach a horizontal rule. Without that check, “Male” became “Mae”.
Use confidence in context, not as a threshold
Tesseract always returns something. Icons, frames and photo texture come back as words with low confidence. A hard threshold removes them, and also removes real words: a correctly read word can carry a confidence of zero. What works is the line. If most of a line is read with confidence, keep its uncertain words too; if most of it is not, keep only the very confident ones.
The same rule gives the honest answer. When nothing on the image is read with confidence, the result is “No readable text found”, not a paragraph of noise.
Scale by letter height, not by image size
Tesseract is most accurate when a word is about 30 pixels tall. Scaling “small images ×2” misses the common case: a large screenshot with small text. Measure the letters instead. The connected dark blobs of an image have heights; their upper quartile is close to the capital height. Scale the image so that this lands near 30 pixels and read it once. That replaced two passes with one and made a ruled table read in 0.4 seconds.
Invert dark panels separately
A blue table header with white text disappears under a single global threshold. Find connected dark regions that are large and solid, invert them, and stretch their background to white, so the page becomes dark text on light everywhere.
Let the grid lines define the columns
The erased vertical lines are still useful: a line that runs through most of the table and lies between words is a column border, even when the neighbouring cells almost touch. A sparsely filled column, like deposits in a bank statement, is a second trap: it does not have enough ink to be seen by a projection, and merges with its neighbour unless it is handled explicitly.
Take a second look at uncertain words
A short isolated value is where tesseract fails most. The number 11 in a table cell came back as “TI”, with low confidence. Cropping the uncertain word and reading it alone as a single line fixes most of these; if its column neighbours are numbers, allow digits only. Retrying at another scale matters: two thin vertical strokes are taken for lines at one size and read correctly at the next.
What still does not work
Photos at a steep angle, handwriting and text over a busy picture. For those the right behaviour is to say so. The engine runs in the browser, in the extension and on the site, and is covered by tests on rendered samples and on raw OCR output saved as fixtures, so each of these rules stays fixed.