Building a Multi-Agent AI for Company LinkedIn Pages - Part 5: Building the Critic Agent

Our system can now classify the topic, do the research and back it with examples. Before handing everything over to the writing agents, we first need to identify weak assumptions, unsupported claims, and irrelevant examples.

That's why we built the Critic Agent.

One important thing to note is that the Critic Agent doesn't rewrite any part of the research chunks or examples. Nor does it invent new data points, it just points out what isn't right.

Research Agent
↓

Finds information

Examples Agent
↓

Finds supporting evidence

Critic
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/lr2n5pih6uygoyyqiqep.png) Agent
↓

Finds weaknesses

Before we start writing code, let's write the system prompt. I want the agent to be harsh. Even if the research appears strong, I still want the agent to think like a skeptical reviewer and identify potential weaknesses. The mistakes shouldn't be more about writing style or grammar, it should focus on questioning accuracy of facts stated, relevance of examples, checking if the topic, research and examples are co-related to each other. Also reinforce the fact that it shouldn't make any changes to the research chunks or examples, but flag mistakes in them.

Unlike the Examples Agent, the Critic Agent takes topic, research chunks and examples as in the user message.

user_message = f"TOPIC: {cleaned_topic}\n\nRESEARCH_CHUNKS:\n" + "\n---\n".join(research_chunks)
user_message2 = f"{user_message}\n\nExamples:\n" + "\n---\n".join(example) 

Sometimes LLMs don't return valid JSON. I've wrapped the call in a retry loop so the agent gets up to three attempts before failing.

Transient formatting issues are surprisingly common with LLMs, so retrying is often simpler than failing the entire pipeline.

for attempt in range(max_retries):

I wanted to make sure empty lists are rejected as the Critic Agent should atleast return one critique point. So, returning valid but empty critiques isn't useful.

if len(parsed) > 0:
   return parsed

Before writing the final code, let's breakdown what the function does. It takes topic, research chunks, examples and max retries ie 3 retries as input. The topic is cleaned followed by the user message which consists of topic, research chunks and examples.

Then we write rest of the code within the loop where we call the LLM to generate response and strip fences to make the JSON valid.

As always, we parse the JSON and make sure that it atleast returns 1 critique point in try and except blocks to handle exceptions.

def be_critique(topic:str, research_chunks: list[str], example: list[str], max_retries = 3) -> list[str]:
    cleaned_topic = clean_topic(topic)
    user_message = f"TOPIC: {cleaned_topic}\n\nRESEARCH_CHUNKS:\n" + "\n---\n".join(research_chunks)
    user_message2 = f"{user_message}\n\nExamples:\n" + "\n---\n".join(example)
    for attempt in range(max_retries):
        raw_response = call_llm(SYSTEM_PROMPT, user_message2)
        refined = strip_json_fences(raw_response)
        try:
            parsed = json.loads(refined)
            if len(parsed) > 0:
                return parsed
        except json.JSONDecodeError:
            print(f"Critic attempt {attempt + 1} failed, retrying...")
    return []

At this point, our system can understand a topic, gather supporting research, find real-world examples, and critically evaluate everything before any writing begins.

In the next article we'll build the Brief Agent to structure all the research chunks, examples and critique points in a proper way before handing it over to the writing agents.

Github Repo: https://github.com/Manav-N4/linkedin-agent#linkedin-agent