docs
Search docsCmd K
Docs/operations

Operations Docs

Chat API System Design

/operations/chat-api-system-design

Chat API System Design

Date: 2026-05-06

Purpose

Membloc currently has two chat backends:

  • Firebase/Firestore is the default production chat path.
  • App Engine already has a Stage 14G HTTP chat bridge for channels, messages, pinned messages, attachments, read cursors, and push fanout.

The next API cycle should make the engine chat domain safe to run under MEMBLOC_BACKEND_MODE=engine without losing the newer Firebase chat features: requested messages, due/completion state, vote/proposal cards, thread state, reactions, edit/delete, search, and live update semantics.

Current Facts

Engine routes already exist under:

  • GET /api/families/:familyId/channels
  • POST /api/families/:familyId/channels
  • GET /api/families/:familyId/channels/:channelId
  • GET /api/families/:familyId/channels/:channelId/messages
  • POST /api/families/:familyId/channels/:channelId/messages
  • GET /api/families/:familyId/channels/:channelId/pinned-messages
  • POST /api/families/:familyId/channels/:channelId/messages/:messageId/pin
  • DELETE /api/families/:familyId/channels/:channelId/messages/:messageId/pin
  • POST /api/families/:familyId/channels/:channelId/read
  • GET /api/families/:familyId/channel-reads

Engine chat persistence currently has:

  • channels
  • messages
  • channel_reads

Engine message type support currently allows only:

  • text
  • image
  • video
  • audio
  • file
  • context
  • system

Flutter ChannelRepository expects a wider surface:

  • requested-message stream and mutation
  • requested-message due date and completion
  • vote/proposal creation, response, close, promote, retry promote
  • reaction stream and toggle
  • edit and soft-delete
  • channel state mute/draft
  • thread state read/mute/draft
  • search
  • typing/presence

EngineChannelRepository currently polls every 3 seconds and throws UnimplementedError for requested messages, votes, proposals, reactions, edit/delete, and several state APIs.

Decision

Use Postgres/app-engine as the source of truth for engine mode. Do not mirror engine-mode chat writes back to Firestore.

Keep Firestore as the default path until the engine reaches feature parity for the user-visible chat flows. Firebase Cloud Functions can remain for legacy/default mode while engine mode gets equivalent HTTP endpoints.

Use HTTP polling for the next cycle, but design every endpoint as if it will later emit typed SSE/WebSocket events. This keeps the implementation small now without boxing the system into polling.

Architecture

flowchart LR
  A["Flutter UI"] --> B["ChannelRepository"]
  B --> C{"Backend mode"}
  C -->|"firebase"| D["Firestore + Firebase Functions"]
  C -->|"engine"| E["EngineChannelRepository"]
  E --> F["/api/families/:familyId/channels/..."]
  F --> G["ChannelHandler"]
  G --> H["ChannelService"]
  H --> I["ChannelRepository (Postgres)"]
  H --> J["CalendarService"]
  H --> K["PushService"]
  I --> L["Postgres chat tables"]

Data Model Additions

1. Extend messages

Add typed JSON columns for card messages and summaries:

ALTER TABLE messages
  ADD COLUMN vote JSONB,
  ADD COLUMN proposal JSONB,
  ADD COLUMN reaction_summary JSONB NOT NULL DEFAULT '{}'::jsonb,
  ADD COLUMN search_prefixes TEXT[] NOT NULL DEFAULT '{}';

ALTER TABLE messages
  DROP CONSTRAINT IF EXISTS messages_type_check,
  ADD CONSTRAINT messages_type_check CHECK (
    type IN (
      'text', 'image', 'video', 'audio', 'file', 'context', 'system',
      'vote', 'proposal'
    )
  );

CREATE INDEX idx_messages_search_prefixes
  ON messages USING GIN (search_prefixes);

Store vote and proposal in JSONB for the next cycle because the Flutter payload shape already exists and can evolve without adding many narrow tables. If vote/proposal analytics become important, split them into relational tables later.

2. Add requested-message projection

CREATE TABLE requested_messages (
  message_id TEXT PRIMARY KEY REFERENCES messages(id) ON DELETE CASCADE,
  family_id TEXT NOT NULL REFERENCES families(id) ON DELETE CASCADE,
  channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
  requested_by TEXT NOT NULL REFERENCES users(uid),
  requested_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  due_at TIMESTAMPTZ,
  completed_at TIMESTAMPTZ,
  completed_by TEXT REFERENCES users(uid),
  assignee_uid TEXT REFERENCES users(uid),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_requested_messages_channel_open
  ON requested_messages(channel_id, requested_at DESC);

CREATE INDEX idx_requested_messages_family_due
  ON requested_messages(family_id, due_at)
  WHERE completed_at IS NULL;

This mirrors the Firestore sibling projection. It avoids scanning messages for requested items and gives the daily panel an efficient family-wide aggregate path.

3. Add reactions

CREATE TABLE message_reactions (
  channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
  message_id TEXT NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
  uid TEXT NOT NULL REFERENCES users(uid),
  emoji TEXT NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (message_id, uid, emoji)
);

CREATE INDEX idx_message_reactions_message
  ON message_reactions(message_id);

messages.reaction_summary is maintained transactionally when reactions change, so list responses do not need to aggregate per row.

4. Add user channel/thread state

CREATE TABLE channel_user_states (
  family_id TEXT NOT NULL REFERENCES families(id) ON DELETE CASCADE,
  channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
  uid TEXT NOT NULL REFERENCES users(uid),
  muted BOOLEAN NOT NULL DEFAULT false,
  pinned BOOLEAN NOT NULL DEFAULT false,
  draft_text TEXT NOT NULL DEFAULT '',
  last_read_message_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
  last_read_at TIMESTAMPTZ,
  last_seen_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (channel_id, uid)
);

CREATE TABLE thread_user_states (
  family_id TEXT NOT NULL REFERENCES families(id) ON DELETE CASCADE,
  channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
  thread_root_message_id TEXT NOT NULL REFERENCES messages(id)
    ON DELETE CASCADE,
  uid TEXT NOT NULL REFERENCES users(uid),
  muted BOOLEAN NOT NULL DEFAULT false,
  draft_text TEXT NOT NULL DEFAULT '',
  last_read_message_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
  last_read_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (channel_id, thread_root_message_id, uid)
);

The existing channel_reads table can be folded into channel_user_states with a compatibility view or left in place for the first migration. The cleaner target is one state table per scope.

5. Add presence/typing as ephemeral state

For the next API cycle, use in-memory TTL for typing status if only one engine instance is running in dev/closed beta. For multi-instance production, use Redis or Postgres with short TTL cleanup. Do not encode typing state into messages.

API Additions

Message parity

PATCH  /api/families/:familyId/channels/:channelId/messages/:messageId
DELETE /api/families/:familyId/channels/:channelId/messages/:messageId
GET    /api/families/:familyId/channels/:channelId/messages/search

Patch body:

{
  "content": "updated text",
  "mentionUserIds": [],
  "mentionDisplayNames": []
}

Delete is soft-delete only. Keep row identity so source links, calendar events, memories, and archives do not dangle.

Reactions

POST /api/families/:familyId/channels/:channelId/messages/:messageId/reactions/:emoji/toggle

Response for message list includes reactionSummary.

Requested messages

GET    /api/families/:familyId/channels/:channelId/requested-messages
GET    /api/families/:familyId/requested-messages
PUT    /api/families/:familyId/channels/:channelId/messages/:messageId/requested
DELETE /api/families/:familyId/channels/:channelId/messages/:messageId/requested
PATCH  /api/families/:familyId/channels/:channelId/messages/:messageId/requested

Patch body:

{
  "dueAt": "2026-05-07T00:00:00Z",
  "completed": true,
  "assigneeUid": "uid-or-null"
}

Family-wide GET /requested-messages is required for the daily panel and cross-channel aggregator.

Chat summary

GET /api/families/:familyId/chat/summary

Response includes channel-card summaries, recent cross-channel pinned notices, and open requested messages:

{
  "channels": [
    {
      "id": "channel-id",
      "name": "전체 가족",
      "lastMessagePreview": "토요일 병원 잊지 말자",
      "lastMessageSenderName": "민지",
      "lastMessageAt": "2026-05-03T01:00:00Z",
      "unreadCount": 2,
      "openRequestedCount": 1
    }
  ],
  "pinnedNotices": [],
  "requestedMessages": [],
  "generatedAt": "2026-05-06T00:00:00Z"
}

Vote/proposal cards

POST /api/families/:familyId/channels/:channelId/messages/vote
POST /api/families/:familyId/channels/:channelId/messages/:messageId/vote
POST /api/families/:familyId/channels/:channelId/messages/:messageId/vote/close
POST /api/families/:familyId/channels/:channelId/messages/:messageId/vote/promote

POST /api/families/:familyId/channels/:channelId/messages/proposal
POST /api/families/:familyId/channels/:channelId/messages/:messageId/proposal
POST /api/families/:familyId/channels/:channelId/messages/:messageId/proposal/close
POST /api/families/:familyId/channels/:channelId/messages/:messageId/proposal/promote

Promotion calls should run inside a database transaction and call the calendar service in the same unit of work where possible. If a single transaction cannot span both service operations cleanly, use an idempotent outbox row:

CREATE TABLE chat_promotions (
  id TEXT PRIMARY KEY,
  family_id TEXT NOT NULL,
  channel_id TEXT NOT NULL,
  message_id TEXT NOT NULL,
  kind TEXT NOT NULL CHECK (kind IN ('vote', 'proposal')),
  status TEXT NOT NULL CHECK (status IN ('pending', 'completed', 'failed')),
  calendar_event_id TEXT,
  error TEXT,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  UNIQUE (channel_id, message_id, kind)
);

State APIs

GET   /api/families/:familyId/channel-states
PATCH /api/families/:familyId/channels/:channelId/state

GET   /api/families/:familyId/channels/:channelId/thread-states
PATCH /api/families/:familyId/channels/:channelId/threads/:threadRootMessageId/state

Patch body:

{
  "muted": true,
  "draftText": "...",
  "lastReadMessageId": "..."
}

The existing channel-reads endpoint can remain as a compatibility alias until Flutter switches to the state endpoints.

Live transport

Keep polling in this cycle. Add a versioned endpoint stub only after the DB events are shaped:

GET /api/families/:familyId/chat/events?cursor=...

Event shape:

{
  "id": "monotonic-event-id",
  "familyId": "...",
  "channelId": "...",
  "type": "message.created",
  "entityId": "...",
  "createdAt": "..."
}

This can be backed by a chat_events outbox table later. Do not start with WebSocket state until message/state parity is done.

Backend Configuration

Required local/prod settings:

MEMBLOC_BACKEND_MODE=engine
MEMBLOC_ENGINE_URL=http://localhost:8080
FIREBASE_PROJECT_ID=homblabs-a1e67
FIREBASE_CREDENTIALS_PATH=/secure/path/service-account.json
DATABASE_URL=postgres://membloc:.../membloc
TOKEN_SECRET=...

Recommended new settings:

CHAT_POLL_MIN_INTERVAL_SECONDS=3
CHAT_MESSAGE_PAGE_LIMIT_DEFAULT=50
CHAT_MESSAGE_PAGE_LIMIT_MAX=200
CHAT_ATTACHMENT_READ_TTL_SECONDS=600
CHAT_TYPING_TTL_SECONDS=8
CHAT_FEATURE_REQUESTED_MESSAGES=true
CHAT_FEATURE_VOTE_PROPOSAL=true
CHAT_FEATURE_REACTIONS=true
CHAT_FEATURE_EDIT_DELETE=true

Feature flags are server-side guardrails. The Flutter app may still hide UI with build flags, but the API must reject unsupported feature calls deterministically instead of exposing partial behavior.

Permission Model

Baseline:

  • Family member can read channels/messages/pinned/requested.
  • Family member can send messages.
  • Sender can edit or soft-delete own message.
  • Family member can pin/unpin in MVP.
  • Family member can mark requested messages.
  • Requested-message completion can be changed by any family member for MVP; assignee-only can be a later policy.
  • Vote/proposal response is self-only.
  • Vote/proposal close/promote is family member in MVP, but the service must enforce idempotency.
  • User state writes are self-only.

All endpoints must validate both family membership and channel/message ownership within that family. Never trust familyId alone.

Implementation Phases

Phase 1: Safe parity foundation

  1. Add migrations for message type expansion, reactions, requested messages, channel/thread user state.
  2. Add repository methods with service-level membership validation.
  3. Add handlers and route registration for requested-message and state APIs.
  4. Wire EngineChannelRepository requested-message/state methods.
  5. Add handler/service tests for 401, 403, 404, idempotent toggle, and requested due/completion.

Exit criteria:

  • Long-press requested flow works in engine mode.
  • Daily panel can read family-wide requested messages.
  • No UnimplementedError for requested-message methods.

Phase 2: Message actions

  1. Add edit/soft-delete endpoints.
  2. Add reaction endpoints and transactionally maintain summary.
  3. Add search prefixes to message writes and search endpoint.
  4. Wire Flutter edit/delete/reaction/search methods.

Exit criteria:

  • Engine mode supports the same basic chat room actions as Firebase for text/image/context messages.

Phase 3: Vote/proposal cards

  1. Extend messages.type to vote and proposal.
  2. Persist vote and proposal JSONB payloads.
  3. Add submit/close/promote/retry-promote service methods.
  4. Integrate promotion with CalendarService.CreateFromChat or a chat_promotions outbox.
  5. Wire Flutter EngineChannelRepository vote/proposal methods.

Exit criteria:

  • Vote/proposal creation, response, close, and calendar promotion work in engine mode.
  • Promotion is idempotent under repeated taps or network retry.

Phase 4: Transport hardening

  1. Add chat_events outbox when parity is stable.
  2. Replace or augment polling with SSE.
  3. Keep ChannelRepository stream interface unchanged.
  4. Add observability for active subscribers, event lag, DB query time, and push result counts.

Exit criteria:

  • Polling can be reduced or disabled for active chat rooms.
  • New message latency is below 1 second in normal network conditions.

Verification Plan

Engine:

  • go test ./...
  • Handler tests for every new route.
  • Service tests with Postgres for transaction and idempotency.
  • Migration up/down test on an empty DB and on a DB seeded with Stage 14G messages.

Flutter:

  • flutter analyze
  • flutter test test/repositories/engine_channel_repository_test.dart
  • Chat-room controller tests covering requested, reaction, vote, proposal, edit/delete under engine repository fakes.

Manual:

  • Run app with:
flutter run \
  --dart-define=MEMBLOC_BACKEND_MODE=engine \
  --dart-define=MEMBLOC_ENGINE_URL=http://localhost:8080
  • Verify:
    • send text/image/context
    • long-press pin/request
    • requested due/completion
    • reaction toggle
    • edit/delete
    • vote/proposal promote to calendar
    • notification deep link opens channel/thread

Risks

  • Dual-source drift: Firebase and engine schemas are already different. Mitigation: only flip MEMBLOC_BACKEND_MODE=engine after parity exit criteria are met.
  • Polling load: current 3-second polling is acceptable for dev/closed beta, not public scale. Mitigation: add chat event outbox before scale-up.
  • Promotion consistency: vote/proposal promotion touches chat and calendar. Mitigation: use transaction when possible; otherwise use idempotent outbox with unique (channel_id, message_id, kind).
  • Attachment URLs: keep the Stage 14G-2 rule. Persist file identity, derive read URLs per response.
  • Push credentials: engine logs show push sender disabled without Firebase admin credentials. Production engine mode needs FIREBASE_CREDENTIALS_PATH.

Implement Phase 1 first. It directly unlocks the latest user-facing work: requested-message badges, due dates, completion, and daily panel aggregation under engine mode. Defer vote/proposal parity until the requested-message API is complete and tested.

SDK assets

Static SDK files

JavaScript runtime and TypeScript definitions are served from this docs app.
membloc-sdk.jsmembloc-sdk.d.ts