Groq has introduced an open source finetune of Llama 3 for tool use. It performs highly on the Berkeley Function Calling leaderboard, making it a top contender for tool use LLMs. It outperforms proprietary models like Claude Sonnet 3.5, GPT-4, and Gemini 1.5 Pro. Notably, it was trained solely on synthetic data.
The model processes over 1k tokens per second for the 8B version and 330 tokens per second for the 70B version. It was developed with Glaive AI, a service for dataset creation and model training.
Due to a drop in general purpose performance, a hybrid approach is recommended. This involves using query analysis to route queries to either the specialized tool use model or a general purpose model, based on the query’s nature and requirements.
The model has a permissive license, identical to the original Llama 3 model. It’s sensitive to temperature and top_p sampling settings; starting values of 0.5 and 0.65 respectively are recommended.
Official announcement: wow.groq.com/introducing-llama-3-groq-tool-use-models/