rag.core.query ============== .. py:module:: rag.core.query .. autoapi-nested-parse:: Query / chat engine over the papers Chroma index. Attributes ---------- .. autoapisummary:: rag.core.query.RETRIEVAL_OVERFETCH rag.core.query.PAPERS_TEXT_QA_TEMPLATE rag.core.query.PAPERS_REFINE_TEMPLATE rag.core.query.CORPUS_SYNTHESIS_PROMPT rag.core.query.AUTHOR_SYNTHESIS_PROMPT Classes ------- .. autoapisummary:: rag.core.query.ProseNodePostprocessor rag.core.query.QueryResult Functions --------- .. autoapisummary:: rag.core.query.require_openai_api_key rag.core.query.build_llm rag.core.query.build_query_engine rag.core.query.ask_llamaindex rag.core.query.ask Module Contents --------------- .. py:data:: RETRIEVAL_OVERFETCH :type: Final :value: 3 .. py:data:: PAPERS_TEXT_QA_TEMPLATE .. py:data:: PAPERS_REFINE_TEMPLATE .. py:class:: ProseNodePostprocessor(/, **data: Any) Bases: :py:obj:`llama_index.core.postprocessor.types.BaseNodePostprocessor` Keep retrieved chunks that look like readable paper prose. Author questions drop acknowledgement / thanks chunks and prefer earlier pages (title-page author lists over end-matter). :ivar keep: Maximum number of prose nodes to retain after filtering. :vartype keep: int .. py:attribute:: keep :type: int :value: None .. py:class:: QueryResult Answer plus grounding citations. :ivar answer: Model response text. :vartype answer: str :ivar citations: Retrieved sources. :vartype citations: list of Citation :ivar citations_markdown: Pre-formatted markdown for UIs. :vartype citations_markdown: str .. py:attribute:: answer :type: str .. py:attribute:: citations :type: list[rag.core.citations.Citation] .. py:attribute:: citations_markdown :type: str .. py:function:: require_openai_api_key() -> str Load ``.env`` if present and return ``OPENAI_API_KEY``. :returns: API key value. :rtype: str :raises EnvironmentError: If the key is missing or empty. .. py:function:: build_llm(*, config: rag.core.config.RagConfig, api_key: str | None = None) -> llama_index.llms.openai.OpenAI Construct the OpenAI chat LLM. :param config: Provides ``llm_model_name``. :type config: RagConfig :param api_key: Explicit key; otherwise read from the environment. :type api_key: str or None :returns: LlamaIndex OpenAI LLM wrapper. :rtype: OpenAI .. py:function:: build_query_engine(*, config: rag.core.config.RagConfig, embed_model: llama_index.core.embeddings.BaseEmbedding | None = None, llm: Any | None = None, index: llama_index.core.VectorStoreIndex | None = None) -> llama_index.core.query_engine.BaseQueryEngine Build a retrieval-augmented query engine. :param config: Retrieval and model settings. :type config: RagConfig :param embed_model: Defaults to the HuggingFace model from config. :type embed_model: BaseEmbedding or None :param llm: Defaults to OpenAI from config + env key. :type llm: Any or None :param index: Defaults to loading the persistent Chroma index. :type index: VectorStoreIndex or None :returns: Configured LlamaIndex query engine. :rtype: BaseQueryEngine .. py:data:: CORPUS_SYNTHESIS_PROMPT :type: Final :value: Multiline-String .. raw:: html
Show Value .. code-block:: python """You are summarizing a researcher's full PDF library. The catalog below is the COMPLETE set of papers ({paper_count} files), not a retrieval sample. --------------------- {catalog_markdown} --------------------- If the files span multiple application domains (medical imaging, networks, environment, finance, news, engineering, …) but share supervised learning, deep learning, graph ML, or similar methods, the unifying theme is applied machine learning. State that umbrella first, then name several distinct domains from the filenames/titles. Do not treat one paper as the whole corpus. Question: {query_str} Answer: """ .. raw:: html
.. py:data:: AUTHOR_SYNTHESIS_PROMPT :type: Final :value: Multiline-String .. raw:: html
Show Value .. code-block:: python """You are identifying authors from a researcher's PDF library. The catalog below is the COMPLETE set of papers ({paper_count} files). Each line is a filename plus opening-page text (title and usually authors). --------------------- {catalog_markdown} --------------------- SPIE footers like 'edited by' name volume editors, not the paper's authors. People thanked for revising a manuscript are not authors. If one person is first author or appears on most papers, they are the main author of this library — say so, then name frequent co-authors. If the question names a specific paper or topic, answer for that paper only. Do not invent names that are not in the catalog. Question: {query_str} Answer: """ .. raw:: html
.. py:function:: ask_llamaindex(*, question: str, config: rag.core.config.RagConfig, query_engine: llama_index.core.query_engine.BaseQueryEngine | None = None) -> QueryResult Ask a paper-level question using the LlamaIndex query engine. :param question: User question. :type question: str :param config: Used when constructing a default engine. :type config: RagConfig :param query_engine: Injected engine (tests / Streamlit cache). :type query_engine: BaseQueryEngine or None :returns: Answer and citations. :rtype: QueryResult .. py:function:: ask(*, question: str, config: rag.core.config.RagConfig, query_engine: llama_index.core.query_engine.BaseQueryEngine | None = None, catalog: rag.core.catalog.PaperCatalog | None = None, llm: Any | None = None) -> QueryResult Ask a question against the papers index or the full corpus catalog. Corpus-level questions (for example "common topic among all papers") skip vector retrieval and synthesize from the paper catalog. Author questions use the same catalog (opening-page title/author snippets) instead of similarity hits, which often match SPIE volume editors. :param question: User question. :type question: str :param config: Used when constructing a default engine or loading the catalog. :type config: RagConfig :param query_engine: Injected LlamaIndex engine (tests / Streamlit cache). :type query_engine: BaseQueryEngine or None :param catalog: Injected catalog; otherwise loaded from disk or ``papers_dir``. :type catalog: PaperCatalog or None :param llm: Injected chat LLM for corpus synthesis. :type llm: Any or None :returns: Answer and citations. :rtype: QueryResult