smocketRecorded study · assertions passed

Application evidence report

Chat-room application case study

Abstract

For one moderated two-room workflow, observable behavior matched across Real Socket.IO 4.8.3, published Smocket 0.4.2, and a handwritten mock, while the owned test-support surface differed. This report compares that surface and the pinned implementation evidence behind the result.

Recorded
2026-08-12
Environment
darwin arm64 · Node v22.16.0
Targets
Socket.IO 4.8.3 · Smocket 0.4.2 · handwritten

The static Markdown document is the authoritative interpretation. This page is an interactive form of the same pinned observation data.

Read authoritative Markdown ↗

These observations apply only to the recorded chat-room workflow and are not a claim of overall Socket.IO compatibility.

01

Research question & method

What does each approach own to support the same selected application test?

All three targets run the same application, scenario, and assertions. Only dependency wiring and bootstrap change; the handwritten fixture additionally owns the mock being compared. No shared file contains a target branch or workaround.

Listeners register before their actions. Acknowledgements and later per-socket markers—not delays or timeouts—establish completion and non-receipt. The same assertions execute twice per target in one process.

Shared application and test code, unchanged across all three targets
FileResponsibilityObserved surfaceEvidence
app.jsChat-room application handlers86 physical source linesPinned source ↗362afdbb7d9c
scenario.jsShared workflow orchestration183 physical source linesPinned source ↗1ebe7269a18b
assertions.jsExpected observation and deep equality98 physical source linesPinned source ↗1636bb1a4995

02

Authored support surface

Exact target-owned JavaScript measured for this recorded workflow.

Real Socket.IO61 lines
61bootstrap
Exact published Smocket28 lines
28bootstrap
Handwritten mock28 + 212 lines
28bootstrap212owned mock
Physical source lines, including blank and comment lines. This is an observed authored surface, not a productivity score; generated lockfiles are excluded and the counts do not generalize beyond this workflow.
Target-owned setup, code, failure paths, and change locations
QuestionReal Socket.IOExact published SmocketHandwritten mock
Exact dependencies / clean installsocket.io@4.8.3, socket.io-client@4.8.3smocket@0.4.2None
Bootstrap / runtime setupLocal HTTP server, real Socket.IO server/client, and an ephemeral loopback TCP port.Published smocket package with an in-memory Server/connect bootstrap.No package dependency; an in-memory fixture imports its own Socket.IO-shaped mock.
HTTP server / port ownershipFixture owns HTTP server creation, listen(0), address lookup, and the port.No HTTP server or port setup in the fixture.No HTTP server or port setup in the fixture.
Client activation / shutdownFixture disables auto-connect/reconnection, activates each client, and closes the server.Connection is available in memory; activate is empty and the fixture closes Smocket.Connection is available in memory; activate is empty and the fixture closes its server.
Authored fixture filesbootstrap.js (61)bootstrap.js (28)bootstrap.js (28) + handwritten-socket-io.js (212)
Application-owned mockNo application-owned mock implementation.No application-owned mock implementation; the published package is the dependency.Application fixture owns the 212-line handwritten-socket-io.js implementation.
Shared branches / workaroundsNone in shared app, scenario, or assertionsNone in shared app, scenario, or assertionsNone in shared app, scenario, or assertions
Explicit failure / debugging surfaceObserved source paths: listen errors, missing TCP address, and close callback errors are explicit.Observed source path: this bootstrap adds no target-specific error branch.Observed source paths: missing registered server and disconnected emitWithAck reject explicitly.
Locations changed with wiring / semanticsDependency manifest/lock and fixture bootstrap when target wiring changes.Dependency manifest/lock and fixture bootstrap when target wiring changes.Fixture bootstrap plus handwritten-socket-io.js when exercised mock semantics change.
Directly observed simpler aspectReference behavior without application-owned mock logic.Avoids HTTP server, ephemeral port, and explicit client activation setup.Simplest dependency installation and port setup of the three recorded fixtures.

Inference Because the handwritten fixture owns the observed 212-line implementation, changes to exercised room or event semantics may require changes there. Future maintenance effort was not measured.

03

Pinned implementation evidence

Selecting an approach changes its setup, owned files, failure paths, and exact source.

Real Socket.IO

socket.io@4.8.3, socket.io-client@4.8.3

Local HTTP server, real Socket.IO server/client, and an ephemeral loopback TCP port.

Owned files
  • bootstrap.js · bootstrap · 61 lines
Explicit source paths

Observed source paths: listen errors, missing TCP address, and close callback errors are explicit.

Real Socket.IO bootstrapcase-studies/chat-room/fixtures/socket-io/bootstrap.js:161

HTTP server, ephemeral port, client activation, and shutdown.

Open pinned source ↗
import { createServer } from 'node:http';
import { Server } from 'socket.io';
import { io as connect } from 'socket.io-client';
import { createChatApplication } from './app.js';
import { runChatRoomScenario } from './scenario.js';

function listen(httpServer) {
  return new Promise((resolve, reject) => {
    httpServer.once('error', reject);
    httpServer.listen(0, '127.0.0.1', () => {
      httpServer.off('error', reject);
      resolve();
    });
  });
}

function close(io) {
  return new Promise((resolve, reject) => {
    io.close((error) => {
      if (error) reject(error);
      else resolve();
    });
  });
}

function createClient(url, options) {
  const client = connect(url, {
    ...options,
    autoConnect: false,
    forceNew: true,
    reconnection: false,
  });

  return {
    client,
    activate: () => client.connect(),
  };
}

export function runScenario() {
  return runChatRoomScenario({
    createClient,
    async startApplication() {
      const httpServer = createServer();
      const io = new Server(httpServer);
      await listen(httpServer);

      const address = httpServer.address();
      if (!address || typeof address === 'string') {
        throw new Error('Socket.IO fixture did not receive a TCP address');
      }

      const url = `http://127.0.0.1:${address.port}`;
      return createChatApplication({
        io,
        url,
        close: () => close(io),
      });
    },
  });
}

04

Workflow behavior matrix

The same structured observation passed the same assertion for every target.

Passed means equal only within this selected workflow and shared assertion.
Selected behaviorReal Socket.IOExact published SmocketHandwritten mock
Passed · same observation Passed · same observation Passed · same observation
Passed · same observation Passed · same observation Passed · same observation
Passed · same observation Passed · same observation Passed · same observation
Passed · same observation Passed · same observation Passed · same observation
Passed · same observation Passed · same observation Passed · same observation
Passed · same observation Passed · same observation Passed · same observation

Selected structured observation · Acknowledged joins

[
  {
    "participantId": "alice",
    "channel": "general",
    "acknowledgement": {
      "accepted": true,
      "channel": "general"
    }
  },
  {
    "participantId": "alice",
    "channel": "support",
    "acknowledgement": {
      "accepted": true,
      "channel": "support"
    }
  },
  {
    "participantId": "bob",
    "channel": "general",
    "acknowledgement": {
      "accepted": true,
      "channel": "general"
    }
  },
  {
    "participantId": "carol",
    "channel": "support",
    "acknowledgement": {
      "accepted": true,
      "channel": "support"
    }
  }
]
Acknowledged joins: expected observationexamples/chat-room/assertions.js:931

Shared expected value used by all three targets.

Open pinned source ↗
export const expectedObservation = {
  joins: [
    {
      participantId: 'alice',
      channel: 'general',
      acknowledgement: { accepted: true, channel: 'general' },
    },
    {
      participantId: 'alice',
      channel: 'support',
      acknowledgement: { accepted: true, channel: 'support' },
    },
    {
      participantId: 'bob',
      channel: 'general',
      acknowledgement: { accepted: true, channel: 'general' },
    },
    {
      participantId: 'carol',
      channel: 'support',
      acknowledgement: { accepted: true, channel: 'support' },
    },
  ],
Acknowledged joins: application handlerexamples/chat-room/app.js:2032

Shared application code used without a target branch.

Open pinned source ↗
    socket.on('join-channel', async (channel, acknowledge) => {
      if (!channels.has(channel)) {
        acknowledge({ accepted: false, reason: 'unknown-channel' });
        return;
      }

      await socket.join(channel);
      io.to(socket.id).emit('welcome', {
        channel,
        text: `Welcome to #${channel}.`,
      });
      acknowledge({ accepted: true, channel });
    });
Room routing and sender exclusioncase-studies/chat-room/fixtures/handwritten/handwritten-socket-io.js:3382

Room union routing, deduplication, joins, and sender exclusion.

Open pinned source ↗
class BroadcastOperator {
  constructor(server, rooms, excludedSocketIds = new Set()) {
    this.server = server;
    this.rooms = new Set(rooms);
    this.excludedSocketIds = excludedSocketIds;
  }

  emit(event, ...args) {
    const recipients = this.rooms.size === 0 ? new Set(this.server.sockets.keys()) : new Set();

    for (const room of this.rooms) {
      for (const socketId of this.server.rooms.get(room) ?? []) {
        recipients.add(socketId);
      }
    }

    for (const socketId of recipients) {
      if (this.excludedSocketIds.has(socketId)) continue;
      this.server.sockets.get(socketId)?.deliver(event, cloneArguments(args));
    }

    return true;
  }
}

class ServerSocket extends Emitter {
  constructor(id, server, client, auth) {
    super();
    this.id = id;
    this.server = server;
    this.client = client;
    this.handshake = { auth: auth ?? {} };
    this.rooms = new Set();
    this.connected = true;
  }

  join(roomOrRooms) {
    if (!this.connected) return;

    for (const room of asArray(roomOrRooms)) {
      this.rooms.add(room);
      const members = this.server.rooms.get(room) ?? new Set();
      members.add(this.id);
      this.server.rooms.set(room, members);
    }
  }

  to(roomOrRooms) {
    return new BroadcastOperator(this.server, asArray(roomOrRooms), new Set([this.id]));
  }

05

Supporting observation evidence

The shared transcript is secondary because target selection cannot change it.

Supporting evidence: shared transcript 10 canonical lines

Filter transcript by participant

Filter transcript by event

Showing 10 of 10 shared lines.

  1. 01[alice] Welcome to #general.
  2. 02[alice] Welcome to #support.
  3. 03[bob] Welcome to #general.
  4. 04[carol] Welcome to #support.
  5. 05[alice] Bob in #general: Hello, everyone!
  6. 06[bob] Announcement rejected: moderator-only
  7. 07[alice] Alice to #general, #support: Maintenance starts at 18:00.
  8. 08[bob] Alice to #general, #support: Maintenance starts at 18:00.
  9. 09[carol] Alice to #general, #support: Maintenance starts at 18:00.
  10. 10[alice] Bob left #general.

06

Interpretation & evidence boundaries

Three questions require three different limits on what this record can support.

Fidelity

Published Smocket did not change the selected application's observable result relative to real Socket.IO. This says nothing about behavior outside the shared assertions; the conformance report remains authoritative for declared compatibility.

Reliability

The runner is repeatable and each recorded target passed twice in one process. This is one snapshot, not evidence of continued success over time. Recurring published-package validation is separate integration evidence.

Productivity

Physical source lines, including blank and comment lines, describe concrete code surfaces. They are not a productivity score, and generated lockfiles are excluded from that comparison.

Neutral and unfavorable findings

  • The handwritten target is simpler in dependency installation and port setup.
  • The real target supplies reference behavior without application-owned mock logic.
  • Published Smocket keeps the shared application and assertions unchanged with an in-memory bootstrap.

Inferencechanges to the exercised event or room semantics may require maintaining the handwritten mock's additional implementation surface.

07

Reproduction & provenance

Run these commands in the pinned Smocket source. Recording replaces the canonical snapshot, so use it only intentionally.

Commands

  • pnpm case-study:chat-room
  • pnpm case-study:chat-room:check
  • pnpm case-study:chat-room:record
  • node scripts/run-chat-room-case-study.mjs --target socket-io
  • node scripts/run-chat-room-case-study.mjs --target published-smocket
  • node scripts/run-chat-room-case-study.mjs --target handwritten

Shared application source

examples/chat-room

  • app.jsapplication · 86 physical source lines
  • scenario.jsapplication · 183 physical source lines
  • assertions.jsassertions · 98 physical source lines

Recorded provenance

Recorded at
2026-08-12T08:51:16.147Z
Environment
darwin arm64; Node v22.16.0; npm 11.17.0
Pinned source commit
fa90e07e272c7fd0db64ebfd73cbb104664ddb81
Authoritative publication commit
6a17477beef33fb014ab629b914d80a6f144b31b
Observation SHA-256
414b07fb27b70cc836d8b71d78d63a0f530d2cae28dbd32b60e77462a64f4bad
Application source SHA-256
e3884c42af5987b4db154c7f13538054e405e12b496803b8d321ac9a409b62d5