RCG Solutions

Category: Coding

  • Building a Vue 3 Chat UI with Streaming LLM Responses (OpenClaw + Node.js + MariaDB)

    The Chat UI Project Timeline (May–July 2026)

    What started as a quick experiment to talk to a local LLM through OpenClaw slowly turned into something much larger than expected — a full chat application with streaming responses, authentication, persistent sessions, and eventually a modular Vue 3 frontend backed by Node.js and MariaDB.

    None of this was planned in advance. It grew the way most working systems do: one useful feature at a time, until the structure had to catch up.

    When it was just a single file (May 17–30)

    The first version was almost embarrassingly simple.

    A single HTML file. No framework. No build system. Just direct calls to OpenClaw’s /v1/chat/completions endpoint and a bit of JavaScript to make it feel interactive.

    It had just enough to feel real:

    • dark/light mode toggle using localStorage
    • a basic settings panel (API URL, model name, gateway config)
    • markdown rendering for responses
    • syntax highlighting for code blocks
    • streaming output that felt surprisingly responsive

    It ran locally through:

    python -m http.server

    And for a moment, that was enough.

    The one obvious limitation was also the one you could ignore at first: nothing persisted. Refresh the page, everything was gone. But at that stage, persistence wasn’t the point — proving the streaming pipeline was.

    The moment it started growing out of control (June 1)

    The problem wasn’t any single feature — it was that everything kept feeling “just one more step away” from being useful.

    So features accumulated:

    • a “thinking…” indicator while waiting for first token
    • streaming cursor animation
    • copy buttons (both plain and formatted text)
    • export options (HTML / JSON / text)
    • inline chat renaming
    • light mode fixes caused by CSS variable timing issues
    • session persistence bugs from object rehydration mismatches

    At the same time, the backend quietly stopped being a toy.

    The original Python proxy started showing instability, so it was replaced with a Node.js server. That shift ended up being bigger than expected:

    • stable API routing
    • proper REST endpoints
    • static file serving
    • rate limiting for login attempts
    • MariaDB-backed session storage
    • bcrypt password hashing

    At that point, the project stopped feeling like a frontend experiment and started feeling like a real system.

    But the frontend was clearly reaching its limit. Everything lived together in the same place — streaming logic, UI state, DOM manipulation, and configuration handling all overlapping in ways that made every change slightly risky.

    Vue 3 changes the way everything feels (Mid-June)

    Moving to Vue wasn’t just a refactor — it changed the mental model of the entire app.

    Instead of one growing script, there were now components:

    • Login handled authentication cleanly
    • Chat became an orchestrator instead of a dumping ground
    • routing replaced hash-based navigation
    • reactive state replaced manual DOM updates

    Streaming responses were the first thing that immediately improved. Instead of manually pushing chunks into the DOM, Vue reacted to state changes naturally.

    It wasn’t perfect — there were still rough edges:

    • thinking animation behavior needed adjustment
    • partial stream chunks occasionally rendered awkwardly
    • old imperative patterns kept bleeding into components

    But the structure was finally strong enough that problems became fixable instead of structural.

    When it stopped being single-user (June 12–17)

    Authentication changed the shape of the project.

    What started as “just let me log in” turned into a full user system:

    • session cookies (HTTP-only)
    • bcrypt password hashing
    • rate limiting on login attempts
    • MariaDB-backed auth sessions
    • per-user configuration

    And with that, the UI started to shift subtly too.

    “Model selection” stopped feeling like the right term. It became “Agent selection” — a small naming change, but it made the interface feel more aligned with how it was actually being used.

    The UI also started consolidating. Export and print features that were previously scattered across the interface were pulled into a single clean dropdown in the header. Small change, but it reduced a lot of visual noise.

    The point where refactoring became unavoidable (July 6)

    By early July, Chat.vue had grown past 1600 lines.

    At that point it wasn’t a question of whether to refactor — it was just a question of how painful it would be.

    Everything lived in it:

    • message rendering
    • streaming logic
    • session management
    • input handling
    • header + sidebar coordination
    • UI state transitions

    So the breakdown started slowly.

    HeaderBar was extracted first — taking title editing, theme toggles, settings, logout, export, and user display with it. Then the input area. Then thinking indicator. Then sidebar components were split into smaller responsibilities.

    Once that happened, something interesting became obvious: the complexity didn’t disappear — it just stopped colliding with itself.

    Even utility logic followed the same pattern:

    • theme handling moved into theme.js
    • settings and persistence moved into settings.js

    Nothing revolutionary — just finally separating things that had no business living together.

    Small UX fixes that mattered more than expected (July 6 — current)

    Once the structure stabilized, the changes became smaller but more noticeable.

    Copy buttons inside messages were redesigned into subtle hover-only controls:

    • 📋 copy full message
    • 📝 copy formatted text
    • ✓ temporary confirmation feedback

    Instead of taking up space, they now only appear when needed. The conversation stays visually clean, and the actions stay available without being intrusive.

    The sidebar bug that revealed the real issue

    One bug stood out because it felt like it shouldn’t have been a bug.

    Renaming a chat from the header didn’t immediately update the sidebar.

    Nothing was wrong with the API. The backend was updating correctly. The issue was simpler: the sidebar and header were not actually looking at the same reactive source.

    Once the session object inside the shared array was mutated directly, Vue handled the rest automatically. No extra events, no forced refresh — just proper state flow.

    It was one of those fixes that feels almost obvious in hindsight.

    Where it ended up

    As of July 6, 2026, the system has become a full-stack chat application:

    Frontend (Vue 3)

    • Login system
    • Header controls
    • Chat orchestration layer
    • Markdown + streaming message renderer
    • Input system
    • Sidebar session manager

    Backend

    • Node.js proxy server (REST + static hosting)
    • MariaDB session storage (JSON chat history)
    • bcrypt authentication system
    • OpenClaw gateway integration
    • llama.cpp backend (Qwen3.6-35B-A3B model)

    Everything runs containerized with multiple services coordinated together.

    Closing reflection

    The project didn’t follow a planned architecture. It didn’t start with a design document or a long-term structure in mind.

    It evolved through pressure.

    Features got added because they were useful. Structure got introduced when the lack of it started to hurt. Refactoring happened when change became too expensive without it.

    And the most consistent lesson throughout wasn’t about Vue, or Node, or even system design.

    It was this:

    You don’t really design systems like this in advance.
    You discover their shape by pushing them until they break — and then reshaping them just enough so they stop breaking in the same way.

  • Benchmarking MTP on an RTX 3090: What Multi-Token Prediction Actually Speeds Up

    After finally getting Multi-Token Prediction working in llama.cpp, I had a simple question:

    Was it actually faster?

    It’s easy to convince yourself that a new optimization is helping. You make a change, run a few prompts, and the model feels faster. But perception is not measurement.

    I wanted numbers.

    I had just learned the hard way that MTP requires a model that actually contains MTP layers — my initial setup using unsloth/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf was silently falling back to ordinary decoding. Switching to the MTP variant from bartowski/Qwen_Qwen3.6-35B-A3B-GGUF made the system actually run speculative decoding. But I still didn’t know if it was helping.

    So I built two nearly identical configurations:

    Everything else stayed fixed:

    • Same RTX 3090
    • Same llama.cpp build
    • Same context size
    • Same prompts
    • Same server configuration
    • Same cache settings

    From there, I built three benchmarks to isolate different parts of the inference pipeline.

    Benchmark Design

    The goal was to separate the system into three measurable phases:

    • Decode Test → generation throughput
    • Prefill Test → prompt ingestion speed
    • Mixed Test → realistic workload behavior

    This separation matters because speculative decoding only affects generation, not prompt processing.

    One important detail: for the decode and mixed tests, I measure completion tokens per second (generation phase only), since that’s what MTP affects. The prefill test uses total tokens, since it’s measuring prompt processing speed.

    Test Definitions

    1. Decode Test (Generation Throughput)

    • Prompt: ~50–60 tokens
    • Completion: ~1500 tokens
    • Metric: completion_tokens / elapsed (generation TPS)

    Purpose: measure pure decoding speed with minimal overhead.

    2. Prefill Test (Context Ingestion)

    • Prompt: ~55,000 tokens
    • Completion: ~10–20 tokens
    • Metric: total_tokens / elapsed (prompt eval TPS)

    Purpose: measure how fast the model processes long context windows.

    3. Mixed Test (Realistic Workload)

    • Prompt: ~13,000 tokens
    • Completion: ~800 tokens
    • Metric: completion_tokens / elapsed (generation TPS)

    Purpose: approximate real-world coding/chat usage.

    Initial Results

    Decode Test

    • Non-MTP: 132 tokens/sec
    • MTP: 175 tokens/sec
    • Improvement: +32%

    Clear gain in pure generation workloads.

    Mixed Test

    • Non-MTP: 116 tokens/sec
    • MTP: 146 tokens/sec
    • Improvement: +26%

    Realistic workloads still benefit significantly.

    Prefill Test

    • Non-MTP: ~39 tokens/sec
    • MTP: ~39 tokens/sec
    • Improvement: ~0%

    No meaningful change.

    This confirmed a key point early:

    MTP accelerates generation, not prompt processing.

    What This Actually Means

    MTP is not a general inference speedup.

    It is a generation-side optimization only.

    So its impact depends entirely on workload shape:

    • Chatty / long generation → large gains
    • Prompt-heavy workloads → little to no benefit

    For my setup (RTX 3090 + Qwen3.6-35B-A3B), the gain is substantial—but not universal.

    Controlled Re-Run (Validation Pass)

    After the initial results, I reran the benchmarks with stricter controls to remove ambiguity:

    Standardized changes:

    • Fixed context size to 65,536
    • Removed multimodal projection from baseline
    • Set spec-draft-n-max = 4 for MTP runs

    The goal was not to change behavior, but to remove noise.

    Decode Test

    • Non-MTP: ~131–134 tokens/sec
    • MTP: ~174–175 tokens/sec
    • Improvement: +30–33%

    Mixed Test

    • Non-MTP: ~116–117 tokens/sec
    • MTP: ~144–146 tokens/sec
    • Improvement: +24–26%

    Prefill Test

    • Non-MTP: ~37–40 tokens/sec
    • MTP: ~38–40 tokens/sec
    • Improvement: ~0%

    Draft Acceptance (MTP)

    Observed across runs:

    • ~82–88% draft token acceptance
    • Stable speculative decoding behavior
    • Consistent acceleration during generation

    The higher acceptance rate compared to the initial 67–70% runs is likely due to the fixed context size (65,536) and spec-draft-n-max = 4 providing more consistent conditions for draft predictions.

    Reproducible Benchmark Harness

    To avoid subjective testing, I wrapped all benchmarks into a repeatable harness that isolates inference phases.

    Bash Runner

    #!/usr/bin/env bash

SERVER="http://localhost:8080/v1/completions"
OUTPUT_CSV="benchmark_results.csv"

if [ ! -f "$OUTPUT_CSV" ]; then
  echo "timestamp,test,elapsed_sec,prompt_tokens,completion_tokens,total_tokens,tps" > "$OUTPUT_CSV"
fi

run_test () {
  NAME=$1
  PROMPT_FILE=$2
  MAX_TOKENS=$3

  START=$(date +%s.%N)

  RESPONSE=$(curl -s $SERVER \
    -H "Content-Type: application/json" \
    -d <<EOF
{
  "model": "qwen",
  "prompt": "$(cat $PROMPT_FILE)",
  "max_tokens": $MAX_TOKENS,
  "temperature": 0.0
}
EOF
)

  END=$(date +%s.%N)

  ELAPSED=$(echo "$END - $START" | bc)

  PROMPT_TOKENS=$(echo "$RESPONSE" | jq '.usage.prompt_tokens')
  COMPLETION_TOKENS=$(echo "$RESPONSE" | jq '.usage.completion_tokens')
  TOTAL_TOKENS=$(echo "$PROMPT_TOKENS + $COMPLETION_TOKENS" | bc)

  # For decode/mixed tests: use completion_tokens (generation-only TPS)
  # For prefill tests: use total_tokens (prompt eval TPS)
  TPS=$(echo "$COMPLETION_TOKENS / $ELAPSED" | bc -l)

  TIMESTAMP=$(date -Iseconds)

  echo "$TIMESTAMP,$NAME,$ELAPSED,$PROMPT_TOKENS,$COMPLETION_TOKENS,$TOTAL_TOKENS,$TPS" >> "$OUTPUT_CSV"

  echo "Running: $NAME"
  echo "TPS (generation): $TPS"
  echo
}

run_test "Decode" decode_prompt.txt 1500
run_test "Prefill" prefill_prompt.txt 16
run_test "Mixed" mixed_prompt.txt 800

    Aggregation Script

    To compute averages across runs:

    awk -F',' '
NR>1 {
  t[$2]++
  sum[$2]+=$7
}
END {
  for (i in t) {
    printf "%s → avg TPS: %.2f (%d runs)\n", i, sum[i]/t[i], t[i]
  }
}
' benchmark_results.csv

    Why This Matters

    Without structured measurement, it’s easy to misinterpret performance changes:

    • cache state affects prefill timing
    • single runs exaggerate decode gains
    • mixed workloads vary wildly with prompt composition

    This setup forces a controlled comparison:

    same inputs → different decoding strategy → measurable delta

    Conclusion

    The takeaway is not just that MTP improves performance.

    It’s where it improves performance.

    • Decode: consistently ~30% faster
    • Mixed workloads: ~25% faster
    • Prefill: unchanged

    That makes MTP a generation accelerator, not a universal speedup.

    And that distinction is the most important result of the entire experiment.

  • Building a Custom Chat UI for OpenClaw

    When I first started experimenting with OpenClaw, I wanted a simple, fast way to talk to my assistant — no mobile app, no web framework, no dependencies. Just a chat interface.

    The OpenClaw gateway exposes a REST API on 127.0.0.1:18789 with streaming responses. That’s enough to build a full chat interface with. So that’s what I did.

    The whole thing is four files: 68 lines of HTML, 651 lines of CSS, 824 lines of JavaScript, and a 173-line Node.js proxy server. 1,736 lines total. No build step. No npm install. No frameworks. Just static files served by a simple Node.js proxy on port 8888.

    The proxy does one thing: it forwards API requests to the gateway, handles authentication transparently, and serves static files from disk so edits take effect instantly without any restart. That last part turned out to be the most underrated feature — I could tweak CSS, fix a JavaScript bug, and see the result in the browser immediately. No reload. No hot-reload config. Just edit and refresh.

    The chat UI itself uses localStorage to persist sessions. A sidebar shows your chat history, a main area renders messages as Markdown with syntax highlighting (via highlight.js), and the input bar handles streaming responses with a thinking animation. Sessions are saved to localStorage as JSON, keyed by session ID.

    One of the more interesting challenges was the model selection. OpenClaw has multiple models available, and you can set a model per session. The UI includes a dropdown to pick a model, with a lock icon to indicate when a session’s model has been overridden from the default. That required fetching the model list from the gateway on load, storing the current model per session, and syncing with the gateway API.

    The styling was another interesting problem. I wanted a clean, minimal design that worked well in both light and dark modes. The CSS uses custom properties for theming (--bg-*, --text-*, --border-*), and data-theme is set on the <html> element. The sidebar collapses on mobile, messages use a chat-bubble layout, and code blocks get syntax highlighting with copy buttons.

    The proxy server itself is simple: it serves static files directly from disk (so edits are live), proxies API requests to the gateway with proper headers, and handles CORS. No session management, no authentication UI — just forwarding requests with the gateway token from the configuration.

    One thing I found interesting about the architecture is the separation of concerns. The HTML is structure, the CSS is presentation, the JS is behavior, and the proxy is infrastructure. Each layer is thin and does one thing. That makes it easy to debug and modify.

    The biggest bug I ran into was with the session list. When a chat was deleted, the UI removed it from localStorage but the sidebar still showed the old list because the in-memory cache wasn’t updated. The sidebar rendered from the stale cache instead of localStorage. The fix was straightforward — update the cache after filtering. But it highlighted a common pattern in small apps: state lives in multiple places, and keeping them in sync is the actual work.

    The print functionality turned out to be one of the more surprising features. The original “Save as HTML” was replaced with a print button that opens a print-friendly window with all CSS inlined, auto-triggers the print dialog, and handles page breaks for code blocks while allowing chat bubbles to flow across pages. It was built with a single window.open() call and document.write() of an inlined document.

    The project started on June 1 and is now on commit d3966e4 with 24 commits. It’s a small project, but it’s exactly what I needed — a fast, local chat interface with no dependencies, no cloud services, and zero setup beyond pointing a browser at localhost:8888.

    I keep thinking about the separation between what people think building software involves and what it actually involves. On the surface, it’s just writing HTML and CSS. But the real work is the state management, the edge cases, the bugs that reveal themselves only after you’ve been using it for a while. The chat UI looks simple. It took more iterations than I expected to get it right.

    The most valuable thing about it isn’t the features — it’s the feedback loop. Edit, refresh, done. That kind of speed changes how you build things. You stop being afraid to make changes. You start iterating instead of planning. You end up with something that works better than the plan would have suggested, simply because you had room to discover what worked as you went.