Cloe

I've built the super assistant that I've always wanted. Cloe is a voice/WhatsApp assistant that has long-term memory and can do a bunch of things like managing my wishlist, grocery list, product recommendations, and meal/exercise tracking.

Watch Demo

As an important fundamental, I gave her long-term vector memory through Memkit. She can store and recall memories like my (and my family's) favorites, things we own, facts about our life and experiences, and our preferences. I even use it for journal entries now. Memkit is portable as an MCP server, so the memories created in Cloe can be recalled in ChatGPT and Claude or vice versa.

Then the ability to search the web and a bunch of MCP tools I've built make Cloe capable.

Wishlist

Normally I have a wishlist on Amazon that I add books to, but it's not getting much use other than that. Cloe's wishlist allows me to take photos of things I see out in the wild and send them as a WhatsApp message to Cloe so she can add them to my wishlist. Later on I can tell Cloe I've bought the thing and she removes it from the wishlist.

Example: I needed an extension cord for charging our EV. I called Cloe on my Apple Watch and told her to do a web search for the top-reviewed EV car extension cords and add them to my wishlist along with Amazon links. She did it. Then I just bought the top one she added without having to do additional research on my own. Felt pretty good.

Grocery List

Normally we use the Apple Reminders app for our family grocery list, which is fine. But while grocery shopping it's hard for me to know which thing is in which aisle and then check it off. I tried to solve this by giving Cloe household shared grocery items capability. So my wife and I can add items to our grocery list separately.

Then the fun part is that I had Cloe learn what's in which aisle at what store. To do this I took photos of each aisle, then told Cloe which aisle it was. She stored them in venues and venue_sections. So when we're at the supermarket we can just tell her which aisle we're at, then she matches what she knows about what's in that aisle to what we have in our grocery list and tells us exactly what to buy in that aisle. Then we can tell her after we bought it and she marks it as purchased.

Product Recommendations

Because Cloe has access to our family's favorite things, I can have her recommend products. In the demo you'll see how I took a photo of my wife's closet, sent it to Cloe, and she figured out the colors, patterns, and styles in the photos. Then she recorded it in Memkit as my wife's style taste. Then I called Cloe while driving and asked her about handbag recommendations based on my wife's style tastes and she gave me ideas. Then I sent her a photo of some options I was considering and she told me which one was most suitable. Same thing when trying to buy toys my son might like.

Meal/Exercise Tracking

I had built this before and now gave it to Cloe as well. Take a photo of what you ate and send it to Cloe as a WhatsApp message. She'll estimate the calories and log it. Tell her about your exercise and weight, and she'll log those too. Easy tracking.

What I really want Cloe to do is go all the way to purchase. I'm going to be digging into the Agentic Commerce Protocol to see if we can have Cloe complete a purchase using it.

You can go to cloe.fyi/join to try it out. DM me on LinkedIn to let me know what you think and if you have any ideas for capabilities that Cloe should have.

Bora