text-os architecture

The host, the macOS app, and how the renderer puts text on screen. Written for someone who has built TUIs, CLIs and web UIs but has not worked inside a renderer.

1. The big picture

text-os is two processes. A host written in TypeScript (run by Bun) owns all state and logic and decides what every cell of a 120-column text grid shows. A client written in Swift paints that grid in a macOS window and sends input back.

If you know terminals, it is a terminal emulator and a TUI program. The difference is the protocol between them: JSON messages over a WebSocket rather than escape codes over a pty.

• The app launches the host. launcher.swift spawns bun src/main.ts --client-owned and then connects to it. The host watches its parent process id and exits if the window dies.
• ⌘R reloads the host, not the app. The host re-executes itself in place (same process id, same port). The window stays open and reconnects.
• The host also runs on Linux. The client is the only macOS-specific part. A remote host is reached through an ssh tunnel, so the client always connects to 127.0.0.1.

2. How the host makes a frame

The host's loop is Elm-style: state → event → update → render. One root (src/ui/runtime/runtime.ts) holds a single JSON-serialisable state tree. Apps, the tab bar and dialogs are all views with the same shape: { init, update, render }.

• Drawing is writing into a 2D array. Views call grid.put(row, col, text, style) (src/ui/draw/grid.ts). There is no retained tree and no diffing: every frame is a full redraw.
• Colours stay as palette names on the host and become #rrggbb only while the frame is turned into JSON. A theme switch is therefore just the next frame.
• Two cheap savings. Redraws are capped at one per 16ms (src/host/run.ts), and a frame whose JSON is identical to the previous one is not sent (src/host/transport.ts).
• Dialogs travel as layers. Each open dialog is rendered separately and attached as a layer with an x/y origin; the base frame is flagged dim.

3. The wire

Every message is one complete JSON object with a type. A frame looks like this (field names are real, values are illustrative):

{
  "type": "frame",
  "background": "#161616",
  "foreground": "#e6e6e6",
  "rows": [
    [ { "t": " Files ", "bg": "#2a2a2a", "bold": true }, { "t": " main.ts" } ],
    [ { "t": "╭──────╮", "fg": "#808080" } ]
  ],
  "cursor": { "row": 3, "col": 14 },
  "layers": [ { "x": 30, "y": 10, "w": 60, "h": 12, "rows": [ ] } ],
  "dim": true
}

• Frames are deflate-compressed. The docs measure a 9,200-byte frame at 534 bytes on the wire.
• The client sends its version in an x-text-os-version header on the WebSocket upgrade. A mismatch is a hard stop. The header also keeps web pages out, because a browser cannot set a custom header on a WebSocket.
• The field shapes are mirrored by hand in TypeScript (src/ui/draw/paint.ts, src/event-types.ts) and Swift (grid.swift, transport.swift). They must change in the same commit.

4. The macOS app

The client is about 7,300 lines of Swift in src/client/, compiled with plain swiftc (no Xcode project). About 1,600 of those lines are audio and speech, which this report skips.

GridView is an MTKView: an ordinary NSView whose backing store is a GPU texture. It is paused, meaning it has no render loop. It draws only when a frame arrives, the cursor blinks, or the window resizes.

The window stays hidden until the first valid frame arrives. The client owns no colours of its own, so there is nothing sensible to show before then.

What is not drawn by Metal

Four things are ordinary AppKit views or layers stacked on top of the grid:

• The drag ghost, which follows the pointer between frames.
• The media portal, a WKWebView placed over a rectangle of cells to show images, video and PDFs.
• The latency meter (⌥⌘F).
• The connection border, white while reconnecting and red after three seconds.

5. Renderer primer: three ideas

The renderer is a "GPU glyph atlas" design. The client README says this is the model Ghostty, Alacritty and WezTerm use. Three ideas carry it, and each has a web analogy.

Texture atlas

A sprite sheet. Each glyph is rasterised once into a tile of one big texture, then copied from there every time it is needed.

Instancing

One draw call that says "draw this rectangle N times", plus an array of N small structs. The GPU runs the same code for every copy and hands each copy its own struct. Think of array.map(drawCell) where the loop runs on the GPU.

Shaders

Two small functions that run on the GPU. The vertex shader runs once per rectangle corner and answers "where on screen is this corner?". The fragment shader runs once per pixel inside the rectangle and answers "what colour is this pixel?".

The older design drew each cell with CoreText on the CPU and cost about 40ms per frame in the worst case. The README puts the current design at under 2ms, which is why the client repaints the whole grid every time with no dirty-rectangle tracking.

6. A frame's journey through the client

• Pending slot. Decoded frames wait in a single slot, not a queue. If a newer frame arrives before the main thread has drawn the older one, the older one is dropped. A fast host cannot back up the main thread.
• Synchronous draw. setFrame calls draw() directly instead of asking AppKit to schedule a redraw later.
• Ring of three buffers. The CPU writes per-cell data into a buffer the GPU then reads. Three buffers rotate, and a semaphore stops the CPU from overwriting one the GPU has not finished with.
• No display sync. displaySyncEnabled is off because, with it on, the window server holds a finished image for one extra refresh.

7. Layout: fitting the grid to the window

Layout.fit (layout.swift) picks the font size, and everything else follows from it:

• Width and height both constrain the font. It is the largest size at which 120 columns fit across and 40 rows fit down, capped at 20pt and snapped down to 0.25pt steps. The bundled font is JetBrains Mono.
• Taller windows add rows, not bigger text. When the row count changes the client sends resize and the host re-renders at the new height.
• Cells are whole physical pixels wide. Cell width is rounded up to a backing pixel and the grid origin is pixel-aligned. This is the single most important rule for sharp text: an atlas tile is exactly one cell in size, so its pixels land 1:1 on screen pixels.
• Hit-testing uses the same numbers. A mouse point becomes a cell by dividing by the same cell width and height the renderer draws with.

8. Building the instances

Every frame, on the CPU, buildInstances (renderer.swift) walks the segments and writes one small struct per cell. This is the "array of N structs" from the primer.

• Order of filling. Blank every cell with the frame background, then the base rows, then each dialog layer over the top. An owners array records which layer last wrote each cell.
• Dimming happens here. When a dialog is open, base colours are replaced by dark greys on the CPU before they reach the GPU.
• behind and below exist for rounded shapes. A cell with rounded corners has to paint the corner pixels in the colour of whatever is around it, and that can differ between top and bottom.
• meta.kind tells the fragment shader which of seven ways to draw the cell (section 11).

9. The glyph atlas

• Lazy. The first time a (character, bold, italic) combination appears, CoreText draws it in white into a cell-sized greyscale bitmap, which is uploaded into the next free tile (atlas.swift).
• Keyed by glyph, not colour. Themes and syntax colours never grow the cache.
• Two atlases. Text goes in the 8-bit coverage atlas. Emoji carry their own colours, so they go in a second, full-colour atlas.
• Growth. When full, the texture doubles in height. A CPU-side copy is kept so existing tiles are re-uploaded rather than re-drawn, and no tile moves.
• Resize. Any layout change throws both atlases away. They refill as the next frames draw.
• Wide glyphs are scaled down to the cell width when rasterised. This is where "one scalar, one cell" is paid for.

10. One draw call

enc.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 4, instanceCount: maxCells)

• There is no geometry buffer. The vertex shader works out each corner from two integers: which corner (0 to 3) and which instance. The cell's column and row are just the instance index divided by 120.
• Every cell is drawn every frame, including blank ones. Each quad is opaque and covers exactly its cell, so nothing needs sorting or blending in this pass.
• A handful of numbers are shared by all cells ("uniforms"): viewport size, grid origin, cell size in pixels, tile size, and the two atlas sizes.

11. The fragment shader

For each pixel of each cell, grid_fragment (shaders.swift) looks at meta.kind and takes one of seven paths.

Why draw box lines with maths instead of the font?

Cell width is rounded up to whole pixels, so the font's own line and block glyphs stop slightly short of the cell edge. A run of them shows hairline seams. Computing them guarantees that a line leaving one cell meets the line entering the next at the same pixel. Rounded backgrounds (pills, buttons) are also computed here, by rounding the corners of any cell flagged as the end of a rounded shape.

12. Colour

• The render target is sRGB (bgra8Unorm_srgb). The shader outputs linear-light values and the GPU encodes them on write. Blending in linear light avoids dark fringes where two saturated colours meet.
• Text weight is corrected. Pure linear blending makes light-on-dark text look heavier than people expect. coverageMix adjusts the coverage value first, so the result's brightness matches what a gamma-space blend would give, and then mixes the colours in linear space. This is my reading of that function, not something a comment states.
• The client holds no palette. Every frame carries its own background and foreground and every segment's colours as hex.

13. Overlays and dialogs

• Overlay pipeline. A second, alpha-blended pipeline draws plain rectangles over the cells. It handles the cursor (a 2-pixel bar that takes the ink colour of the cell beneath it), underlines and strikethroughs, and the bar-coloured padding at the window edges. Dotted, dashed and wavy lines are computed per pixel from the absolute x position, so the pattern stays continuous across cells.
• Why dialogs need an extra pass. A dialog's rounded border leaves small wedges outside the curve where the dimmed screen behind should show. Each quad is opaque, so the shader needs the real pixels from underneath. Pass 1 renders the base alone into an offscreen texture, and pass 2 samples it for exactly those wedges.
• The cursor blink is the client's only animation. The host sends a cursor position; a local 0.53s timer toggles it. This is what lets the host stay event-driven with no timer of its own.

14. Input and connection

Input

• GridView turns each AppKit event into a JSON event with cell coordinates and sends it. Selection, focus, hit-testing and drag logic all live in the host.
• Mouse moves are sent only when the pointer enters a different cell. Trackpad scrolling is converted from points to whole cells, carrying the fraction forward.
• click is sent only if the button is released on the cell it was pressed on. That lets a drag across a button select text without pressing it.
• Every key is swallowed after it is sent. The cost, listed as accepted in the client README, is no IME or dead-key composition.
• Row 0 drags the window, unless the host has marked the hovered spot as clickable.

Connection

• Reconnect. On a dropped socket the client retries five times 250ms apart, then once a second. The window stays open with a border.
• Liveness. Sending any event also sends a probe and arms a 300ms deadline. Any inbound byte disarms it. If it fires, the path is treated as dead and rebuilt. Nothing is measured while idle.
• Goodbyes are messages. A deliberate close arrives as a farewell message, because the WebSocket close code does not survive compression in Apple's client.

15. Where to read next

1. src/client/README.md, the "Rendering" section.
2. src/client/renderer.swift: render(), then buildInstances, fillRows, cellInstance.
3. src/client/shaders.swift: grid_vertex, then grid_fragment.
4. src/client/tests/rendering-main.swift: pixel-readback tests that show what each piece is meant to guarantee.
5. docs/platform-spec.md for the reasons behind the host/client split and the wire.

Not covered here: ssh tunnelling to remote hosts (tunnel.swift, src/remote-setup/), the audio and speech engine, file drop uploads, and the app-authoring API (docs/api-one-pager.md).

16. Stale references found along the way

• docs/platform-spec.md:439 and src/client/grid.swift:1235 point to src/host/runtime.ts. The file is src/ui/runtime/runtime.ts.
• docs/platform-spec.md says there are "exactly two" client capabilities. CAPABILITY_NAMES has four; ssh.forward and host.switch were added.
• src/client/README.md says "6000 cells". The count is 120 × rows, which is 4,920 at the 41-row floor.
• src/os/ and src/tui/ contain no files.