Building Conversational Memory in AI Agents with Solon's ChatSession API

If you've built any LLM-powered application, you've hit this wall: large language models are stateless by design. Every API call is a fresh conversation. The model doesn't remember what you said five messages ago — unless you tell it.

The standard approach is "multi-message prompting": stuff the entire conversation history into each request. But managing that history — trimming old messages, persisting across sessions, handling concurrent users — gets messy fast.

I've been working with Solon's AI module (solon-ai) for a few months, and its ChatSession abstraction solves this cleanly. Let me walk through how it works.

The Problem: Stateless LLMs

When you call chatModel.prompt("Hello").call(), the model has no context. If you ask "What did I just say?", it won't know. The model only sees the current message.

To maintain context, you need to pass previous messages as part of the prompt. This is where ChatSession comes in.

The Solution: ChatSession

Solon's ChatSession interface manages a sequence of messages for a conversation. It records every input and output, and automatically feeds the history back into subsequent prompts.

import org.noear.solon.ai.chat.ChatSession;
import org.noear.solon.ai.chat.session.InMemoryChatSession;

ChatSession chatSession = InMemoryChatSession.builder()
        .maxMessages(10)
        .sessionId("session-1")
        .build();

Usage with synchronous calls

chatModel.prompt("Hello, I'm looking for a laptop recommendation.")
         .session(chatSession)
         .options(o -> {
            o.systemPrompt("You are a helpful tech support agent.");
         })
         .call();

Usage with streaming

chatModel.prompt("What did I ask you about?")
         .session(chatSession)
         .stream()
         .filter(resp -> resp.hasContent())
         .forEach(resp -> System.out.print(resp.getContent()));

The .session(chatSession) call is the key. It tells the model: "Use this session's history as context, and record this interaction too."

Three Built-in Implementations

Solon ships with three ChatSession implementations, each suited for different scenarios:

Web Integration: Full Example with SSE

Here's a complete web endpoint that maintains per-user conversation memory with Server-Sent Events streaming:

import org.noear.solon.ai.chat.ChatModel;
import org.noear.solon.ai.chat.ChatSession;
import org.noear.solon.ai.chat.session.InMemoryChatSession;
import org.noear.solon.annotation.Controller;
import org.noear.solon.annotation.Header;
import org.noear.solon.annotation.Inject;
import org.noear.solon.annotation.Mapping;
import org.noear.solon.web.sse.SseEvent;
import reactor.core.publisher.Flux;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

@Controller
public class ChatController {
    @Inject
    ChatModel chatModel;

    final Map<String, ChatSession> sessionMap = new ConcurrentHashMap<>();

    @Mapping("chat")
    public Flux<SseEvent> chat(@Header("sessionId") String sessionId, String prompt) {
        ChatSession chatSession = sessionMap.computeIfAbsent(sessionId,
                k -> InMemoryChatSession.builder()
                        .sessionId(k)
                        .maxMessages(20)
                        .build());

        return chatModel.prompt(prompt)
                    .session(chatSession)
                    .options(o -> {
                        o.systemPrompt("You are a helpful assistant.");
                     })
                    .stream()
                    .filter(resp -> resp.hasContent())
                    .map(resp -> new SseEvent().data(resp.getContent()));
    }
}

Serialization for Custom Persistence

If you need to build your own persistence layer (database, S3, etc.), Solon provides serialization utilities for ChatMessage:

import org.noear.solon.ai.chat.message.ChatMessage;

// Single message: to/from JSON
String json = ChatMessage.toJson(message);
ChatMessage msg = ChatMessage.fromJson(json);

// Batch: to/from NDJSON
String ndjson = ChatMessage.toNdjson(messages);
List<ChatMessage> batch = ChatMessage.fromNdjson(ndjson);

These methods are available since v3.8.0 and make it straightforward to store conversation history in any database.

The ChatSession Interface

For reference, here's the full interface (available in org.noear.solon.ai.chat):

public interface ChatSession {
    String getSessionId();
    List<ChatMessage> getMessages();
    List<ChatMessage> getLatestMessages(int windowSize);
    void removeLatestMessage(int windowSize);
    void addMessage(String userMessage);
    void addMessage(ChatMessage... messages);
    void addMessage(Prompt prompt);
    void addMessage(Collection<? extends ChatMessage> messages);
    boolean isEmpty();
    void clear();
    Map<String, Object> attrs();
}

Practical Tips

1. Set maxMessages wisely — too few breaks context, too many blows up token usage. Start with 10-20.
2. Use sessionId for multi-tenancy — map one session per user.
3. Combine with system prompts — .options(o -> o.systemPrompt(...)) sets persona, .session() handles history.
4. Production deployments — swap InMemoryChatSession for RedisChatSession when scaling.

Summary

Solon's ChatSession API solves the "stateless LLM" problem with a clean, minimal abstraction. You get three implementations out of the box (in-memory, file, Redis), a simple builder pattern, and seamless integration with both synchronous and streaming calls.

The web integration example above is production-ready — just add Redis and you have a horizontally scalable chat service with full conversation memory.

All code examples are based on Solon v4.0.3. Official documentation: https://solon.noear.org/article/925