Retrieval-Augmented Generation with LangChain
Heads up... You’re accessing parts of this content for free, with some sections shown as text.
Heads up... You’re accessing parts of this content for free, with some sections shown as text.
Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.
Unlock now
Query analysis is a query-optimization technique that refines queries to improve retrieval search. Install the following modules to begin:
pip install -qU langchain langchain-community langchain-openai
langchain-chroma wikipedia
for doc in docs:
doc.metadata["words"] = round_to_nearest_thousand(len(
doc.page_content.split(" ")))
print_summary(doc)
print()
Title: 2022 Ballon d'Or - Wikipedia
Approximate Word Count: 4000
Title: 2023 Ballon d'Or - Wikipedia
Approximate Word Count: 2000
Title: 2022–23 NBA season - Wikipedia
Approximate Word Count: 10000
Title: 2021–22 NBA season - Wikipedia
Approximate Word Count: 14000
Title: 2022–23 Premier League - Wikipedia
Approximate Word Count: 8000
Title: 2021–22 Premier League - Wikipedia
Approximate Word Count: 7000
Title: 2021–22 UEFA Champions League - Wikipedia
Approximate Word Count: 6000
Title: 2022–23 UEFA Champions League - Wikipedia
Approximate Word Count: 4000
Title: 2023 Cricket World Cup - Wikipedia
Approximate Word Count: 6000
search_results = database.similarity_search("Who won the 2022 ballon d'or?")
print_summary(search_results[0])
Title: 2022 Ballon d'Or - Wikipedia
Approximate Word Count: 4000
search_results = database.similarity_search("Suggest a sports article with
approximately 14000 words")
print_summary(search_results[0])
Title: 2023 Cricket World Cup - Wikipedia
Approximate Word Count: 6000
from typing import Optional
from pydantic import BaseModel, Field
class SportsSearch(BaseModel):
"""Search over a database of sports articles."""
query: str = Field(
...,
description="Similarity search query applied to sports articles.",
)
words: Optional[int] = Field(None, description="Number of words in article")
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI
system = """You are an expert at converting user questions into
database queries. \
You have access to a database of sports articles. \
Given a question, return a list of database queries optimized to
retrieve the most relevant results.
If there are acronyms or words you are not familiar with, do
not try to rephrase them."""
prompt = ChatPromptTemplate.from_messages(
[
("system", system),
("human", "{question}"),
]
)
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
structured_llm = llm.with_structured_output(SportsSearch)
query_analyzer = {"question": RunnablePassthrough()} | prompt | structured_llm
from typing import List
from langchain_core.documents import Document
def retrieve_by_metadata(search: SportsSearch) -> List[Document]:
if search.words not None:
_filter = {"words": {"$eq": search.words}}
else:
_filter = None
return database.similarity_search(search.query, filter=_filter)
query_analyzer.invoke("Who won the 2022 ballon d'or?")
SportsSearch(query="2022 Ballon d'Or winner", words=None)
search_results = database.similarity_search("2022 Ballon d'Or winner")
print_summary(search_results[0])
Title: 2022 Ballon d'Or - Wikipedia
Approximate Word Count: 4000
query_analyzer.invoke("Suggest a sports article with approximately
14000 words")
SportsSearch(query='sports article', words=14000)
retrieval_chain = query_analyzer | retrieve_by_metadata
search_results = retrieval_chain.invoke("Suggest a sports article with
approximately 14000 words")
print_summary(search_results[0])
Title: 2021–22 NBA season - Wikipedia
Approximate Word Count: 14000
search_results = retrieval_chain.invoke("Suggest a football article
with approximately 6000 words")
print_summary(search_results[0])