Ah, I see - so your probabilistic routers are similar to Anthropic's routing pattern where the LLM does classification? Since LLM classification naturally gives probability distributions for each route. Are you using those confidence scores/probabilities in any interesting ways when deciding the routing?
Yes, exactly that. I haven't used any advanced combinations of probs yet as we were doing it prior to the article release, but I will sure look into the suggestions for the future work/improvements. It was similar with contextual embeddings - when Anthropic released the idea it sounded obvious, but it didn't come to us before :)
I was wondering if there are some ways in which we can also ask the agent to plan the sequence of tool calls.
Sometimes, there might be cases when multiple tools can be called parallely but sometimes, there might be cases when input of 1 tool requires output of the other tool. Can it be handled in the same agent, or we need to keep writing agents sequentially in such cases?
question on your definition of tools and response format - you are not using `tools` parameter you can fill when invoke open.ai's API, and effectively ignore their output structure as well.
is that you do for educational purposes only? I can imagine that open.ai (or Llama) models are trained on specific prompt format for tools use, so for maximum performance, we can use their formats.
then, if you want to be model agnostic, you need to come up with clients for every provider / model family
Interesting, I am working on a vscode copilot called Codebuddy. This definitely is a good material to get me started.
Glad you found it useful and hope to see you in the following parts :)
Interesting article, aurimas
Thank you for writing this
I was wondering your thoughts abt Anthropic piece on Dec 19 about agents and workflows
You agree with their definition of agents and that most times you don’t really need agents?
Yes, I am in that exact camp. I am building agentic flows myself as day-to-day and in most cases it's a complex chain of probabilistic routers :)
Ah, I see - so your probabilistic routers are similar to Anthropic's routing pattern where the LLM does classification? Since LLM classification naturally gives probability distributions for each route. Are you using those confidence scores/probabilities in any interesting ways when deciding the routing?
Previously I forgot to add the link to the Anthropic piece https://www.anthropic.com/research/building-effective-agents
Yes, exactly that. I haven't used any advanced combinations of probs yet as we were doing it prior to the article release, but I will sure look into the suggestions for the future work/improvements. It was similar with contextual embeddings - when Anthropic released the idea it sounded obvious, but it didn't come to us before :)
Getting started on this now, can’t be left out of the party and kindly include something using neuxs cloud as I still have mine 25$ in there
I will have more Nebius related content in the following month.
Great article to go to the basics and step by step guide and explanation. I was quite stuck with some other tools and frameworks. Thanks!
For other rookies like myself, probably will be good to include initially the imports:
from dataclasses import dataclass
from typing import Callable, Dict, Any, _GenericAlias, get_type_hints, List
import inspect
import urllib.request
import json
import openai
import os
from dotenv import load_dotenv
load_dotenv()
Sorry for that, the code is also available in the linked GitHub repository with all the imports needed
Glad you found it useful, hope to see you in the next episode :)
Wondering why Serbia into Japan?
Random number generator :)
Cool. I am from Serbia 🇷🇸, that is why I am asking.
Hey, thanks for sharing this crisp tutorial!
I was wondering if there are some ways in which we can also ask the agent to plan the sequence of tool calls.
Sometimes, there might be cases when multiple tools can be called parallely but sometimes, there might be cases when input of 1 tool requires output of the other tool. Can it be handled in the same agent, or we need to keep writing agents sequentially in such cases?
This is really interesting; I am learning...
thanks for the share...
hey Aurimas, thank you for posting the article
question on your definition of tools and response format - you are not using `tools` parameter you can fill when invoke open.ai's API, and effectively ignore their output structure as well.
is that you do for educational purposes only? I can imagine that open.ai (or Llama) models are trained on specific prompt format for tools use, so for maximum performance, we can use their formats.
then, if you want to be model agnostic, you need to come up with clients for every provider / model family
I did a deep dive in this here on a bunch of articles on medium and I am a proponent of not using frameworks that obscure what's going on also. I'm currently playing with pushing a lot of this into the data tier which I'm quite excited about. You can see the first of my articles from last year here... https://medium.com/@mrsirsh/7-days-of-agent-framework-anatomy-from-first-principles-day-1-d54d5fb6d0a3
I'm currently playing