Imagine you are preparing for your Kenya Certificate of Secondary Education (KCSE) examinations. You have three months before your final examinations, and there is quite a lot of material to cover.
You start reading during the first month, covering different subjects and topics. However, after reading a particular topic, you never revisit it until the examination period approaches. By then, there is a good chance that you will struggle to recall some of the things you studied.
Now, consider another student who follows a different approach. Instead of reading everything once and leaving it, they revisit previously covered topics every month. They go through what they learned, test themselves and try to recall important concepts without necessarily referring to their notes.
There is also a third student who does this more frequently, perhaps every fortnight. By the time the examinations arrive, this student has had more opportunities to revisit the material and reinforce what they learned.
Naturally, all three students may have different levels of understanding. However, the students who consistently revisit and actively recall previously learned material generally have better opportunities to retain it than those who study everything once and never look at it again.
Now, let's consider something different.
Have you ever wondered why it becomes difficult to remember certain things you learned many years ago?
For instance, you might have studied geography in high school, covering topics such as landforms, weathering and erosion. Perhaps you performed exceptionally well in those examinations. However, years later, you might struggle to explain some of those concepts.
On the other hand, mathematical concepts you learned in school may still come naturally. You encounter numbers, perform calculations and apply mathematical reasoning regularly. Because you continue using these concepts, you have more opportunities to reinforce what you learned.
Of course, continued use is not the only reason. How well you initially understood something, the amount of practice you received and other information learned over the years can also influence what you remember.
The difference is that some information continues to be revisited and used, while other information gradually becomes harder to recall because it is no longer reinforced.
Interestingly, even when something becomes difficult to remember, it does not necessarily mean that it has been completely erased. Sometimes, revisiting the material or encountering the right clues can make it easier to recall.
These everyday experiences demonstrate something interesting about how we retain information. What we repeatedly revisit and use tends to remain more accessible, while information that we rarely encounter can become increasingly difficult to recall.
Interestingly, artificial intelligence systems face a related challenge.
A memory bank is a mechanism that allows a Large Language Model (LLM) to preserve and retrieve relevant information from previous interactions using an external storage system.
To understand why this is necessary, we first need to understand how an ordinary LLM handles conversations.
An LLM generates responses using the information supplied in its current input, together with knowledge encoded in its learned parameters. However, it does not automatically retain every interaction as a persistent personal memory that it can access during future, independent conversations.
For instance, imagine having a conversation with an AI assistant today. You explain your business, discuss your preferred technologies and provide details about an ongoing project.
Tomorrow, you start another conversation and ask the assistant to help you develop a feature for that project.
Would it automatically know everything you discussed yesterday?
Not necessarily. Unless the application provides the previous conversation history or retrieves previously stored information, the model will not automatically have access to those interactions.
This is where a memory bank becomes useful.
Instead of relying exclusively on the information supplied during the current conversation, the application maintains a separate storage mechanism where relevant information from previous interactions can be preserved.
When a new conversation begins, the application can retrieve information relevant to the current discussion and provide it to the LLM as additional context.
This allows the assistant to maintain continuity across conversations without requiring the user to explain everything repeatedly.
It is important to understand that the memory bank belongs to the surrounding application architecture. The LLM can assist in identifying, summarising and updating information, but the application and its memory-management mechanisms are responsible for maintaining persistent storage.
A memory bank involves more than simply saving conversations in a database. It requires mechanisms for identifying useful information, storing it, retrieving it when needed and maintaining it over time.
Let's break this down.
1. Capturing Relevant Information
Imagine discussing a software project with an AI assistant.
During the conversation, you mention that you use Python and Flask, prefer PostgreSQL and are currently developing an application for a particular client.
The conversation may contain hundreds or even thousands of words. However, not every sentence needs to become a permanent memory.
The application needs to identify information that could be useful in future interactions.
For example:
- The user prefers Python and Flask for backend development.
- The user uses PostgreSQL.
- The user is developing a particular application.
- The application has specific requirements.
These are potential memory items that can be extracted from the conversation and preserved separately.
An LLM can help identify and summarise these details, while the memory-management mechanism determines what should be stored and how it should be represented.
The objective is not to remember every word exchanged but to preserve information that could improve future interactions.
2. Storing and Organising Memories
Once relevant information has been identified, it needs to be stored somewhere that the application can access later.
This is where external databases and storage systems come into play.
A memory bank might use a traditional database, a vector database or a combination of different storage mechanisms.
For example, a memory item could be stored as a record containing the information itself, its creation date, its importance, its last retrieval date and other relevant metadata.
Depending on the architecture, memories can also be organised into different categories.
These could include conversation records, summaries of previous interactions and information about the user.
The choice of storage mechanism depends on the nature of the information and how the application intends to retrieve it.
3. Retrieving Relevant Memories
Storing information is only half the equation. The application must also be able to retrieve it when necessary.
Let's return to our software development example.
Suppose you start a new conversation and ask the assistant to implement a feature in your existing Flask application.
Rather than starting from scratch, the application can search its memory bank for relevant information.
It retrieves your preferred programming language, framework and database, then supplies these details to the LLM.
The model can now generate a response that takes your previous interactions into account.
Importantly, the application does not need to retrieve every memory it has stored. It only needs information relevant to the current conversation.
This is what makes memory retrieval particularly useful. The application can maintain a substantial collection of memories without including all of them in every prompt.
The retrieval process may use several techniques, including keyword matching, semantic similarity search and metadata filtering.
Semantic similarity search is particularly useful because it allows the application to retrieve information based on meaning rather than relying exclusively on exact words.
4. Updating and Forgetting Memories
Now, let's return to our earlier examples.
Remember the student who revisits previously learned material and the student who rarely revisits what they studied?
The same underlying idea can inspire how a memory bank manages its stored information.
A memory bank should not simply accumulate information indefinitely. As conversations continue, some information becomes more relevant, some gets updated, while other information may no longer be useful.
For instance, suppose a user initially tells an assistant that they are using Flask for a project. Several months later, they mention that the project has migrated to Django.
The system should not continue treating Flask as the current framework for that project.
Instead, it should update the existing memory to reflect the new information.
Similarly, consider information that was useful during a particular conversation but has not been relevant for a long time. Depending on its importance and the system's retention policy, its retrieval priority may decrease or the memory may eventually be removed.
This is where things become interesting.
A memory bank can incorporate mechanisms that reinforce frequently useful information while gradually reducing the priority of information that is no longer relevant.
However, there is an important distinction here.
When a human recalls something they previously learned, the retrieval process itself can strengthen their ability to remember that information.
In an AI memory bank, retrieving an item does not automatically strengthen the stored information. The application must explicitly implement a mechanism to update its priority or retention score.
For example, whenever a memory item is retrieved, the application could update its retrieval count, last-accessed timestamp or usefulness score. These signals can then influence how the system prioritises that memory in the future.
However, retrieval frequency should not be the only consideration. Some information remains important even when it is rarely accessed.
For example, a critical project decision might only be needed once every few months, but that does not mean it should be discarded.
Consequently, a well-designed system considers several factors, including relevance, importance, recency, retrieval frequency and whether the information has been updated.
There is also a distinction between making information harder to retrieve and actually deleting it.
A memory can still exist in the database but fail to appear in a retrieval because it has a low priority or the search mechanism does not identify it as relevant.
Actual deletion, on the other hand, means removing the stored memory according to the application's deletion policy.
These are different operations, and a properly designed memory system needs to handle both.
The idea of information becoming harder to recall over time is closely associated with the Ebbinghaus forgetting curve.
German psychologist Hermann Ebbinghaus studied how people retain information and demonstrated that forgetting generally happens more rapidly shortly after learning, then slows as time passes.
The important observation is that information does not remain equally accessible forever without reinforcement.
The forgetting curve is commonly represented using an exponential decay function. However, it is important to note that this does not mean everyone forgets exactly half of what they learn every day. The rate varies depending on the information, how well it was initially learned and other factors.
One way to improve retention is through repeated learning and retrieval.
For example, when preparing for examinations, revisiting previously learned material at intervals and actively testing yourself can improve your ability to recall it later.
This is the principle behind what we call spaced repetition and retrieval practice.
Research on learning has consistently demonstrated the benefits of spacing learning sessions and actively recalling information rather than relying exclusively on repeated reading.
However, more frequent revision is not automatically better in every situation. The effectiveness of spacing depends on the intervals between reviews and how long the information needs to be retained.
For instance, reviewing material every few minutes is not necessarily better than allowing some time to pass before revisiting it.
The objective is to revisit information at appropriately spaced intervals, giving yourself opportunities to recall what you learned before it becomes too difficult to retrieve.
Now, how does this relate to a memory bank?
A memory bank can borrow from this principle by incorporating an updating mechanism that considers how frequently particular memories are accessed and how useful they have been.
For example, suppose an AI assistant frequently retrieves information about your preferred programming language whenever you discuss software development.
The application may use these repeated interactions as evidence that the information remains relevant and should continue to be prioritised.
Conversely, information that has not been accessed for an extended period may receive a lower priority, depending on its importance and other retention criteria.
Some implementations can use time-based decay functions inspired by the forgetting curve to influence how memories are prioritised.
It is important to understand that an AI memory bank does not naturally forget information in exactly the same way a human brain does. Its forgetting mechanism is deliberately implemented through algorithms and retention policies.
The Ebbinghaus forgetting curve provides a useful conceptual foundation, but it is not a mandatory algorithm for every memory bank.
There are different ways an AI application can preserve information from previous interactions. The choice depends on what the application needs to remember and how that information will be used.
Let's explore three common forms.
1. Conversation History
This involves storing the actual conversations exchanged between the user and the AI assistant.
For example, if you have a ten-message conversation discussing a particular software project, the application can preserve all ten messages.
When the user returns, the system can retrieve previous messages to maintain conversational continuity.
The advantage is that the original information remains available rather than being reduced to a short summary.
However, storing and retrieving entire conversations can become expensive and inefficient as the conversation history grows.
2. Summaries of Previous Conversations
Instead of preserving every interaction as the primary memory, an application can generate summaries containing the most important information from previous conversations.
For instance, after discussing a software project, the assistant might generate a summary containing:
- The project's objective.
- The technologies being used.
- Important decisions made.
- Outstanding requirements.
- Pending tasks.
This gives the system a condensed representation of previous interactions without requiring it to retrieve the entire conversation.
Summaries are particularly useful when dealing with long-running projects and interactions involving multiple conversations.
3. User Profiles
Another approach involves maintaining a profile containing relatively stable information about the user.
For instance, the application might preserve information about the user's preferred programming languages, communication preferences, professional interests and working preferences.
Unlike conversation history, which records what was said, a user profile focuses on information that can remain useful across different conversations.
These three approaches are not mutually exclusive. An application can combine conversation history, summaries and user profiles to achieve a more comprehensive memory system.
At this point, an important question arises.
How different is a memory bank from Retrieval-Augmented Generation (RAG)?
Both approaches involve retrieving information from external sources and supplying that information to an LLM to improve its responses.
However, their primary objectives are different.
RAG focuses primarily on retrieving relevant information from an external knowledge source to answer a particular query.
For example, suppose you have a database containing technical documentation, articles and company policies.
When someone asks a question, the RAG pipeline searches the available knowledge sources, retrieves relevant information and supplies it to the LLM to generate an answer.
A memory bank, on the other hand, focuses on preserving information acquired through previous interactions and managing that information over time.
For example, it might remember your preferred programming language, previous project decisions or information about an ongoing task.
A useful way to distinguish the two is to consider their primary responsibilities.
RAG is primarily retrieval-centred. Its main concern is finding relevant information from an available knowledge source.
Memory management is lifecycle-centred. It concerns what information to preserve, how to represent it, when to retrieve it, how to update it and when to forget it.
Another common distinction is how information enters the system.
A typical RAG pipeline indexes existing knowledge sources, such as documents and databases, before retrieving relevant information when a query arrives.
A memory bank commonly creates new memory items from ongoing interactions, then maintains and updates them as conversations continue.
However, this distinction is not absolute. RAG systems can continuously ingest new information, and memory banks can also incorporate existing documents.
The difference is largely in their intended responsibilities rather than the underlying storage technology.
Both systems can use vector databases, embeddings and similarity search to retrieve relevant information.
In fact, a memory bank can incorporate RAG-like retrieval mechanisms as part of its implementation.
Large Language Models are powerful, but maintaining continuity across interactions requires more than simply generating responses from the current prompt.
Memory banks provide one approach to addressing this limitation by allowing AI applications to preserve, organise and retrieve relevant information from previous interactions.
The interesting part is that memory management goes beyond storing information. It also involves deciding what is worth retaining, what needs updating, what should be prioritised and what can eventually be forgotten.
Just as repeatedly revisiting previously learned material makes it easier to recall, an AI memory system can use information about previous retrievals and relevance to manage what remains useful over time.
However, unlike human memory, these processes do not happen naturally in an AI application. They must be deliberately implemented through storage, retrieval and memory-management mechanisms.
Ultimately, the objective is not to remember every interaction. It is to preserve the right information, maintain it appropriately and make it available when it matters.
No comments yet
Be the first to share your thoughts on this article!