rag.core.loaders¶

Framework-agnostic PDF page loading for ingest backends.

Classes¶

PageDocument

One extractable PDF page.

Functions¶

list_pdf_paths(→ list[pathlib.Path])

List PDF files under papers_dir (non-recursive).

load_pdf_pages(→ list[PageDocument])

Extract per-page prose from PDFs with pypdf.

Module Contents¶

class rag.core.loaders.PageDocument[source]¶

One extractable PDF page.

Variables:

  • text (str) – Prose text from the page.

  • metadata (dict) – File name, path, and 1-based page fields shared by all backends.

text: str[source]¶
metadata: dict[str, Any][source]¶
rag.core.loaders.list_pdf_paths(*, papers_dir: pathlib.Path) → list[pathlib.Path][source]¶

List PDF files under papers_dir (non-recursive).

Parameters:

papers_dir (Path) – Directory containing PDFs.

Returns:

Sorted PDF paths.

Return type:

list of Path

Raises:

FileNotFoundError – If the directory does not exist or contains no PDFs.

rag.core.loaders.load_pdf_pages(*, pdf_paths: collections.abc.Sequence[pathlib.Path]) → list[PageDocument][source]¶

Extract per-page prose from PDFs with pypdf.

Pages that fail is_prose_text (figures, dedications, glyph salad) are skipped.

Parameters:

pdf_paths (sequence of Path) – PDF files to load.

Returns:

One document per extractable page.

Return type:

list of PageDocument

Raises:

FileNotFoundError – If no page yields extractable text.