rag.core.ingest

PDF ingest pipeline: load → chunk → embed → Chroma (LlamaIndex path).

Classes

IngestResult

Summary of an ingest run.

Functions

build_embed_model(...)	Construct the local HuggingFace embedding model.
load_pdf_documents(→ list[llama_index.core.Document])	Extract per-page LlamaIndex documents from PDFs.
ingest_llamaindex(→ IngestResult)	Ingest PDFs with LlamaIndex into persistent Chroma.
ingest_papers(→ IngestResult)	Ingest PDFs using config.strategy.

Module Contents

class rag.core.ingest.IngestResult

Summary of an ingest run.

Variables:

• documents (int) – Number of page-level documents written (PDFs with extractable text).

• nodes (int) – Number of chunks written to Chroma.

• chroma_dir (Path) – Persistence directory used.

• collection_name (str) – Chroma collection name.

documents: int

nodes: int

chroma_dir: pathlib.Path

collection_name: str

rag.core.ingest.build_embed_model(*, config: rag.core.config.RagConfig) → llama_index.embeddings.huggingface.HuggingFaceEmbedding

Construct the local HuggingFace embedding model.

Parameters:

config (RagConfig) – Provides embed_model_name.

Returns:

LlamaIndex embedding wrapper.

Return type:

HuggingFaceEmbedding

rag.core.ingest.load_pdf_documents(*, pdf_paths: collections.abc.Sequence[pathlib.Path]) → list[llama_index.core.Document]

Extract per-page LlamaIndex documents from PDFs.

Parameters:

pdf_paths (sequence of Path) – PDF files to load.

Returns:

One LlamaIndex document per extractable page.

Return type:

list of Document

Raises:

FileNotFoundError – If no page yields extractable text.

rag.core.ingest.ingest_llamaindex(*, config: rag.core.config.RagConfig, embed_model: llama_index.core.embeddings.BaseEmbedding | None = None, rebuild: bool = True) → IngestResult

Ingest PDFs with LlamaIndex into persistent Chroma.

Parameters:

• config (RagConfig) – Paths, chunking, and model settings.

• embed_model (BaseEmbedding or None) – Optional pre-built embedder (tests can inject a stub).

• rebuild (bool) – When True, delete the existing collection before writing.

Returns:

Counts and paths for the run.

Return type:

IngestResult

rag.core.ingest.ingest_papers(*, config: rag.core.config.RagConfig, embed_model: llama_index.core.embeddings.BaseEmbedding | None = None, rebuild: bool = True) → IngestResult

Ingest PDFs using config.strategy.

Parameters:

• config (RagConfig) – Paths, chunking, model settings, and orchestration strategy.

• embed_model (BaseEmbedding or None) – Optional LlamaIndex embedder (ignored by other backends).

• rebuild (bool) – When True, delete the existing collection before writing.

Returns:

Counts and paths for the run.

Return type:

IngestResult