How do you actually make a personal agent useful when half the value depends on memory?

I’ve been using a personal agent called Macaron a lot lately, and it got me thinking about something. The easy parts are straightforward to evaluate: does it respond correctly, does the workflow run, does the mini app do what it’s supposed to do. But once the value starts depending on memory, how do you actually measure it? Is it just whether it remembered the right things? Whether the outputs feel more personalized? Or do you simply use it for a few weeks and see if it becomes noticeably less generic over time? I originally thought the most valuable part was that Macaron can turn repeated needs into mini apps without any coding. But over time, it feels like the real value comes from whether it remembers the right preferences, constraints, and patterns—and actually uses them well. Is evaluating memory-based agents just inherently fuzzier than evaluating normal software, or is there a framework people use for this?