Monday, June 24, 2024

Bounce-start Your RAG Pipelines with Superior Retrieval LlamaPacks and Benchmark with Lighthouz AI | by Wenqi Glantz | Jan, 2024

Must read

Exploring sturdy RAG improvement with LlamaPacks, Lighthouz AI, and Llama Guard

Towards Data Science
Picture generated by DALL-E 3 by the writer

For the reason that launch in late November 2023, LlamaPacks has curated over 50 packs to assist jump-start your RAG pipeline improvement. Amongst these, many superior retrieval packs emerged. On this article, let’s dive into seven superior retrieval packs; see the diagram beneath.

Picture supply: LlamaIndex X submit on seven superior retrieval LlamaPacks

We’ll carry out two steps:

  • Given a use case, we’ll generate the benchmarks utilizing Lighthouz AutoBench and consider the packs with Lighthouz Eval Studio to find out which most accurately fits our use case.
  • As soon as the profitable pack is recognized, we’ll add Llama Guard to the RAG pipeline, tweak its customized taxonomy, re-evaluate it with Eval Studio, and observe how the analysis rating for classes corresponding to immediate injection adjustments.

First, let’s have a look at these seven superior retrieval LlamaPacks to see how they work underneath the hood.

Hybrid Fusion

This pack ensembles the vector retrievers and BM25 (Greatest Match 25) retrievers utilizing fusion. BM25 estimates the relevance of paperwork to a given search question, serving to rank paperwork so as of almost definitely relevance to the consumer’s wants.

Hybrid Fusion fuses outcomes from the vector retriever and BM25 retriever out of the field; you’ll be able to present different retriever templates you need by customizing this pack.

paperwork = SimpleDirectoryReader(RAG_DIRECTORY).load_data()
node_parser = SimpleNodeParser.from_defaults()
nodes = node_parser.get_nodes_from_documents(paperwork)

# obtain and set up dependencies
HybridFusionRetrieverPack = download_llama_pack(
"HybridFusionRetrieverPack", "./hybrid_fusion_pack"

# create the pack
hybrid_fusion_pack = HybridFusionRetrieverPack(

Supply hyperlink

More articles


Please enter your comment!
Please enter your name here

Latest article