With River and other AI agents, memory has always been an interesting problem on my mind as it feels like a completely separate field from AI that complements it so well. Specifically, I'm talking about information storage and retrieval. Having worked on similar problems at Gradient Ascent, I've gone through the weeds in deciding how to store information, what metadata is necessary, how to chunk, scoring metrics for retrieving relevant docs, and more.

My honest opinion is that no one truly knows what works best and we throw solutions until one sticks. Having benchmarks helps us fine-tune our efforts to work towards a solution, but there's infinite possibilities on how we store it for the most efficient retrieval. And once retrieval is complete, how do we decide what manner of presenting the information to an LLM is best for its output? Not enough context, and it won't see the connection between the retrieved documents that it needs to. Too much, and it may miss important information.

That said, I worked on this problem 2 years ago, and perhaps the LLMs of today are good enough that these considerations aren't needed anymore.