Discussion about this post

User's avatar
Dr Srinivas Padmanabhuni's avatar

pip install agentest-ai (https://pypi.org/project/agentest-ai/): using a cheap probability-scoring step to gate retrieval and sufficiency decisions makes sense to me, and those scores are easy to test against the trace. In a multi-hop RAG agent I tested, the search retrieved the documents holding the answer and the agent still replied "Not found". A check comparing what was retrieved with what the final answer claims catches that. Code is at https://github.com/srpadmanabhuni/agentest and developers are welcome to enhance and contribute.

Rahul Saini's avatar

Hi Aurimas Griciūnas, Do these decision models RAGs also apply to Local AI ? Say a use case is I select 20 documents and give it to AI, then decision RAG sees the documents size, pages and decides it go to Small LLM, but if I select 100 or 200 documents, it should decide to use a Large LLM. How is this Model decision routing achieved locally (assuming both small LLM and Large LLM GGUF files are placed locally)

9 more comments...

Ready for more?