Mixture of Agents MoA

The CLI will prompt you to input instructions interactively:

  1. Start by entering your instruction at the ">>>" prompt.
  2. The system will process your input using the predefined reference models.
  3. It will generate a response based on the aggregated outputs from these models.
  4. You can continue the conversation by inputting more instructions, with the system maintaining the context of the multi-turn interaction.


You can configure the demo by specifying the following parameters:

  • --aggregator: The primary model used for final response generation.
  • --reference_models: List of models used as references.
  • --temperature: Controls the randomness of the response generation.
  • --max_tokens: Maximum number of tokens in the response.
  • --rounds: Number of rounds to process the input for refinement. (num rounds == num of MoA layers - 1)
  • --num_proc: Number of processes to run in parallel for faster execution.
  • --multi_turn: Boolean to toggle multi-turn interaction capability.

FLASK offers fine-grained evaluation of models across multiple dimensions. Our MoA method significantly outperforms the original Qwen1.5-110B-Chat on harmlessness, robustness, correctness, efficiency, factuality, commonsense, insightfulness, completeness. Additionally, MoA also outperforms GPT-4 Omni in terms of correctness, factuality, insightfulness, completeness, and metacognition.

Please feel free to contact us if you have difficulties in reproducing the results.


Different open source model ?

You can run and configure MoA with GROQ. You can use the 4 available open source models as : llama3-70b , llama3-8b, gemma-7b-it, mixtral-8x7b .

Mixture of Agents (MoA) is a novel approach that leverages the collective strengths of multiple LLMs to enhance performance, achieving state-of-the-art results. By employing a layered architecture where each layer comprises several LLM agents, MoA significantly outperforms GPT-4 Omni’s 57.5% on AlpacaEval 2.0 with a score of 65.1%, using only open-source models!

Subscribe to groq.com and get the api_key . Clone the repo and cd into it. Here you need to make some changes to files bot.py and utils.py .

Replace all existing models , with the groq models to look like this :

Screenshot from 2024-07-02 21-40-18

Change the temperature value if needed, default is 0.7

Change the max_tokens value to 2048

On the line 81 you need to make also some changes, replace the model name with the model name from groq, we use llama3-70b for this example.

Screenshot from 2024-07-02 21-42-14

On line 113 you need to proceed and make the same changes , overwrite the existing model with the model from groq.

Screenshot from 2024-07-02 21-43-44

save the bot.py file and open the utils.py .

on the line 31 replace the endpoint value to look like this :

endpoint = "https://api.groq.com/openai/v1/chat/completions"

on line 47 replace the API Key provider to GROQ

"Authorization": f"Bearer {os.environ.get('GROQ_API_KEY')}",

on line 86 replace the endpoint to :

endpoint = "https://api.groq.com/openai/v1"

on the line 90 replace the endpoint to :

endpoint = "https://api.groq.com/openai/v1/chat/completions"

- Create a new .env file inside of working directory

- add to .env file :

GROQ_API_KEY = " your api key here"


save the .env file

# you need to install the dotenv library with following command :

pip install python-dotenv

load the env file inside of utils.py script :

from dotenv import load_dotenv

save the the utils.py file


02. Usage

MoA - Mixture of Agent
  • Category : LLM
  • Time Read:10 Min
  • Source: GitHub
  • Author: Partener Link
  • Date: July 2, 2024, 6:57 p.m.
Providing assistance

The web assistant should be able to provide quick and effective solutions to the user's queries, and help them navigate the website with ease.


The Web assistant is more then able to personalize the user's experience by understanding their preferences and behavior on the website.


The Web assistant can help users troubleshoot technical issues, such as broken links, page errors, and other technical glitches.


Please log in to gain access on MoA - Mixture of Agent file .