feat(search): index PDF text, OCR scanned PDFs on import
All checks were successful
CI / Windows build (push) Successful in 15m50s
All checks were successful
CI / Windows build (push) Successful in 15m50s
Search now covers handwriting, the PDF text layer, AND scanned (rasterized) PDFs. - PdfTextIndexer runs at import: sums the embedded text layer across pages; if present it stores that as the document body, otherwise the PDF is rasterized and its rendered pages are OCR'd in the background. The result lands in the sidecar `pageText` field (distinct from `ocrText`, the handwriting OCR). Idempotent (skips a sidecar that already has pageText); degrades gracefully with no OCR engine. - pdfrx_page_text_source abstracts text/render so it's testable. - VaultSearchIndex now harvests title + typed text + handwriting OCR + PDF pageText, so search finds notes, typed PDFs and scanned PDFs. analyze clean, 409 tests green.
This commit is contained in:
@@ -10,9 +10,12 @@
|
||||
// pages,
|
||||
// * the handwriting OCR text persisted in the sidecar's `ocrText` field.
|
||||
//
|
||||
// It does NOT (yet) index a PDF's embedded text layer — that is a known gap (see
|
||||
// the REPORT in the task / the class doc below). Matching uses the existing pure
|
||||
// search primitives (normalize / rank / snippet), so CJK substring search works.
|
||||
// It ALSO indexes a file-backed PDF's document body text, captured once at
|
||||
// import into the sidecar's `pageText` field by [PdfTextIndexer]: the embedded
|
||||
// (printed) text layer for a normal PDF, or background OCR of the rendered pages
|
||||
// for a RASTERIZED / scanned PDF that has no text layer. Matching uses the
|
||||
// existing pure search primitives (normalize / rank / snippet), so CJK substring
|
||||
// search works.
|
||||
|
||||
import 'dart:io';
|
||||
|
||||
@@ -71,13 +74,14 @@ class VaultSearchHit {
|
||||
/// source of truth).
|
||||
///
|
||||
/// HONEST SCOPE (what search covers / does NOT):
|
||||
/// * COVERS: note/doc titles, typed text boxes, and handwriting OCR text that
|
||||
/// has been persisted into a sidecar's `ocrText` field.
|
||||
/// * DOES NOT cover: a PDF's embedded (printed) text layer — only the user's
|
||||
/// annotations are indexed, not the underlying document body. Indexing the
|
||||
/// PDF text layer would require rendering each page through pdfrx at scan
|
||||
/// time; deferred. OCR is only present where it has already been run and
|
||||
/// written back to the sidecar.
|
||||
/// * COVERS: note/doc titles, typed text boxes, handwriting OCR text persisted
|
||||
/// into a sidecar's `ocrText` field, AND a PDF's document body text persisted
|
||||
/// into `pageText` at import — the embedded text layer, or background OCR of
|
||||
/// a rasterized/scanned PDF (see [PdfTextIndexer]).
|
||||
/// * CAVEAT: `pageText` is only present once import-time indexing has run and
|
||||
/// written it back to the sidecar. A PDF imported before this feature (or
|
||||
/// whose OCR backend was unavailable) has no `pageText`, so only its
|
||||
/// annotations are searchable until it is re-indexed.
|
||||
class VaultSearchIndex {
|
||||
VaultSearchIndex(this._vault);
|
||||
|
||||
@@ -174,6 +178,11 @@ class VaultSearchIndex {
|
||||
}
|
||||
final ocr = sidecar.ocrText;
|
||||
if (ocr != null && ocr.trim().isNotEmpty) parts.add(ocr.trim());
|
||||
// Document body text captured at import: the PDF's embedded text layer,
|
||||
// or background OCR of a rasterized/scanned PDF. Covers the underlying
|
||||
// document, not just the user's annotations.
|
||||
final body = sidecar.pageText;
|
||||
if (body != null && body.trim().isNotEmpty) parts.add(body.trim());
|
||||
}
|
||||
return parts.join('\n');
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user