Refectly

The Problem

Most chatbots are amnesiac. Start a new session, and the model has no idea who you are, what you told it last time, or what you were wrestling with a week ago. For a casual Q&A bot, that's fine. For a reflective journaling tool — the kind you come back to when you're trying to untangle a thought, work through a feeling, or track a goal — losing context every turn defeats the whole purpose.

I wanted something that would remember. Not just the last few messages, but the shape of who you are and the threads you've been pulling on. A CLI I could run on my own machine, talk to honestly, and have it actually get to know me over time.

What I Built

Refectly is a Python command-line journaling companion. You type, it responds with empathetic reflection, and crucially — it remembers. It runs on gpt-3-turbo-32k for the conversational layer and ChromaDB as a long-term vector store for everything it's learned about you.

The whole thing is two scripts:

• chat.py — the main client. Captures input, talks to OpenAI, manages memory.
• chromadb_peek.py — a small inspector so you can look inside the knowledge base and see what it's stored about you.

There's also a user_profile.txt you seed with your name, profession, interests, beliefs, plans, and communication preferences — and a key_openai.txt for your API key. No databases to spin up, no cloud services. Just Python.

How It Works

The chat loop is a while True: with a small but deliberate pipeline on every turn:

User input
    │
    ├── log to disk
    ├── search KB (ChromaDB) for relevant context
    │       │
    │       └── rewrite system message with retrieved context
    ├── generate response (OpenAI)
    ├── update user_profile.txt from recent messages
    └── update KB: add new entry OR update + split if too long

The interesting part is that the system message isn't static. Before each generation, the bot queries ChromaDB with the current conversation, pulls back the most semantically relevant memories, and rewrites the system prompt to include them. The model only ever sees a small, curated slice of your history — not the entire dump.

After the response, the same conversation turn is summarized and either appended as a new KB entry or merged into an existing one. If an existing entry grows past a threshold, it gets split in two, so no single embedding ever gets unwieldy.

Key Design Decisions

A knowledge base the bot manages itself

I stole this idea from a repo by David Sappiro and it changed how I thought about conversational memory. Instead of stuffing the context window with everything, the bot itself decides what to remember, where to put it, and when to split it. The result feels less like a search engine and more like a journal that writes back.

A living user profile

The user_profile.txt file isn't just seed data — it gets rewritten after every turn based on the most recent exchange. The model effectively curates its own understanding of you, and that file is what makes responses feel personalized even on the first message of a new session.

Multi-threading for perceived speed

The pipeline has several independent steps: KB search, response generation, profile update, KB write. Running them sequentially added up. I introduced multi-threading for the I/O-bound and parallelizable steps, and the difference in felt responsiveness was night and day — even though the model call is still the bottleneck, killing the dead time between steps matters when you're in the middle of a thought.

gpt-3-turbo-32k on purpose

The 32k context window was a deliberate pick. Reflective journaling runs long — you might pour out a paragraph and then ask the bot to engage with all of it. A standard 4k window would have forced aggressive truncation, and the whole point is to not truncate the user.

What's Included

Future Enhancements

There are two directions I've been itching to take this:

• A GUI — a terminal UI is fine for writing, but a richer interface (think Tauri or Electron) would make long sessions much more pleasant. Image support, better rendering, maybe a sidebar of past entries.
• A LangChain rewrite — the current code is intentionally direct (raw OpenAI calls, hand-rolled KB logic), and it works. But LangChain's chains, memory abstractions, and document loaders would let me focus on the reflective parts of the experience instead of plumbing.

Links

• GitHub

Timeline

A short experiment to see if a chatbot with its own self-managing knowledge base could feel like a real journaling companion. Single commit, but the ideas stuck around.

• Jun 2023 — Built as an experiment: GPT-powered reflective journaling CLI with ChromaDB vector memory, self-managing knowledge base, and living user profile