Jump to content

Righting Sentences - Release Notes - 2.5

From Saxton Publishing Technical Documentation

Righting Sentences is a web-based, editor-developed text generation application that provides a chat UI for interacting with multiple LLM providers, with authentication and state persistence through Google Cloud services. This release is only running locally and will not be deployed to Google Cloud Run.

Highlights

Bi-directional chat UI with streaming responses Persistent conversations and the user's message history are stored in Google Firestore A "Honeypot” system designed to capture examples of good and bad text generation for fine-tuning a foundational model in the future API routing to OpenAI, Anthropic, and Google Gemini

What’s New

Chat conversations (two-way UI)

    • The UI now supports interactive, back-and-forth chat rather than one-way prompting.
    • Assistant responses stream progressively to the UI to improve perceived latency and usability during longer generations.

Firestore-backed chat history

    • Conversations and messages are stored in Google Firestore (users, conversations, messages).
    • Chat histories can be listed, searched, and loaded from within the UI.

Honeypot data capture for editorial training workflows

    • A honeypot mechanism was introduced to log editorial “events” associated with generated output (for downstream scoring and model fine-tuning experiments).
    • When enabled, honeypot events are written to Google Cloud Storage (GCS) as JSONL, including prompt, model metadata, output text, and a pass/fail-style score field.
    • Current triggering behavior is based on detecting banned substrings in generated output (editorial guardrail prototype).

Technical Notes

Backend

    • FastAPI application serving both API routes and the static UI.
    • Streaming responses are provided via text/event-stream.

Authentication

    • Google OAuth (OpenID Connect) is used for sign-in.
    • Auth state is stored as a signed JWT in an HTTP-only cookie.

Storage

    • Firestore stores users, conversations, and messages.
    • GCS stores honeypot JSONL logs when honeypot logging is enabled.

Model routing

    • Requests are routed by model name to OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, or Google Gemini streaming endpoints.
    • The OpenAI base URL can be pointed at a proxy (e.g., LiteLLM) via configuration.

Known Limitations

Text-only workflow (no multimodal input/output). No file upload support. Chats must be cleared manually in the UI. Model switching workflow is clumsy. Parts of the UI are not fully developed (e.g., the chat copy button has poor contrast in darker themes and can be hard to see).

Known Bugs

Chat history must be cleared before selecting a new model. Model selection is allowed within the same chat window; this can cause errors and incorrect API calls (workaround: clear the chat before switching models). Conversation titles in the chat history do not reliably update when starting a new chat. Chat histories are not always saved reliably. User chat bubble margins are too large.

Retrieved from "https://docs.saxtonpublishing.com/index.php?title=Righting_Sentences_-_Release_Notes_-_2.5&oldid=94"