Pastes

Experiment ledger for EE Holdings

Whitepaper v0.1 -- Sep 9, 2026 Product: Pastes (CodeShare) -- pastes.reversal.org Owner: EE Holdings / Reversal Status of this document: strategy + current inventory. Unbuilt work is labeled.

Labels used throughout:

• CURRENT -- shipped in app/pastesv2 as of Sep 9, 2026
• PLANNED -- designed, not implemented
• FUTURE -- depends on earlier phases
• RESEARCH -- open design questions

Companion chapters: Architecture, API, Data model, Evaluation, EE VIZN, Agent Lenses, Model Router, Security, Roadmap, Commercialization.

---

1. Cover

Pastes is an API-first store for text and structured outputs. Today it is a paste bin with named keys so agents can write notes. The strategic product is an experiment ledger: every generation can be stored, identified, versioned, scored, compared, promoted, and reused.

This paper specifies how Pastes, EE VIZN, and a future EE Model Router turn disposable model output into compounding infrastructure.

2. Executive summary

Thesis: Pastes transforms disposable AI outputs into a persistent experimentation dataset that allows EE Holdings systems to measure, compare, learn from, and continuously improve the models and agents operating across its software ecosystem.

Long-term vision: Every EE Holdings agent should eventually be able to remember what worked, compare alternatives, see the applications it modifies through EE VIZN, and improve its decisions from accumulated experiment history.

CURRENT (Phase 1 start). PHP/MySQL CodeShare on pastes_db. Public/unlisted/private pastes. Session and Google auth. pvsk_ keys. POST/GET /api/v1/pastes/ create and fetch (list GET requires a key; human docs /docs/). POST /api/v1/notes/ with external_id idempotency. Project Command binds a key per project and forwards notes. Staff /admin/ lists users, keys, pastes, and notes. No paste version graph, no experiment objects, no scores, no screenshots, no vision models.

PLANNED. Immutable versions, experiments, runs, model/prompt metadata, human ratings, weighted automated evaluation, object storage for assets, workers, Playwright capture, EE VIZN scoring, Command dashboards, then a model router trained on Pastes history.

Git versions source. Pastes versions artifacts and judgments. Together they are the memory of how EE software is generated, not only how it is compiled.

3. Problem

EE Holdings already generates sites (Compacter), jobs and changelogs (Project Command), dashboards (EE Financial), checkout surfaces (Backout), and agent traces (Cursor notes into CodeShare). Almost every generation is treated as a one-off blob.

When a layout scores poorly, the next agent does not see the losing prompt, the winning screenshot, or the model that produced the better card. The organization re-learns in chat instead of in data.

The failure is not "we need another paste bin." The failure is that evaluations die with the session.

4. Why current AI workflows lose experiment information

Typical loop: prompt in a chat tool, glance at output, copy into a repo or CMS, forget the losers.

Lost fields: model id and version, temperature and tools, system prompt, candidate set, judge prompt, human rationale, cost, latency, screenshot, production outcome.

Without those fields, EE cannot compute P(best model | task, app). Routing stays folklore ("use model B for UI"). Folklore does not compound.

CURRENT notes capture title, body, source (web|api|command), api_key_id, and external_id. That is a trace, not an experiment. It is the right ingest spine to hang versions and scores on later.

5. Pastes overview

Pastes should become reusable infrastructure for:

• AI agents and traces
• ML experiments and benchmarks
• prompts and model outputs
• generated website components and UI
• screenshots and visual QA
• structured agent JSON
• code generations
• human feedback and automated scoring
• historical model comparisons

Every output should be able to be: stored, uniquely identified, versioned, linked to model and prompt and parameters, evaluated, scored, compared, accepted or rejected, promoted as current best, and reused through APIs.

CURRENT identity. public_id (16 char) plus optional access_key for private links. PLANNED identity. Opaque experiment/run/version ids with immutable bytes and a content hash.

6. Versioning architecture

Git versions source files. Pastes versions generated artifacts.

CURRENT. One row in pastesv2_pastes. Edits are not a version chain. PLANNED. Append-only paste_versions. Never update a scored blob in place.

7. Experiment architecture

Workflow (PLANNED):

Input -> Baseline generation -> Store baseline -> Evaluate -> Record score -> Generate alternatives -> Evaluate each -> Compare -> Select highest score -> Store winning configuration -> Promote winner -> Feed results into future model selection -> Repeat

Example: a website component scores 71/100. Pastes records model, model version, prompt, system prompt, parameters, page, component, screenshot, score, evaluator, timestamp. Five challengers score 74, 81, 78, 89, 83. Version D (89) is preferred. The ledger keeps losers. Future agents can query "what won for card components on Backout."

8. Evaluation system

No single metric picks a winner. PLANNED weighted score, configurable per project:

Overall = Visual 30% + Functionality 25% + Brand 15% + Accessibility 10% + Performance 10% + Human 10%

Human: thumbs, 1-10, A/B, production-ready, reject with reason, defect tags. Store as structured ratings rows, not chat asides.

Automated: LLM-as-judge, vision models, lint, a11y, Lighthouse, screenshot diff, schema validation, unit/browser tests, latency, cost, conversion. See Evaluation.

CURRENT. No ratings table. Admin can read paste bodies. That is inspection, not scoring.

9. Multi-model benchmarking

Task: "Generate the pricing section for Backout." Models A-D receive the same structured requirements. Each run is stored. Scores: visual quality, usability, brand, technical correctness, accessibility, responsiveness, conversion potential, latency, cost, human preference.

Over time EE maintains a routing matrix, for example:

• UI generation -> Model B
• PHP debugging -> Model C
• SQL generation -> Model A
• Brand design -> Model D
• Data analysis -> Model C
• Vision inspection -> EE VIZN model (FUTURE)
• Documentation -> Model A

The matrix is not a slide. It is GET /api/v1/models/recommend backed by Pastes history (FUTURE).

10. EE VIZN

EE VIZN (FUTURE) is a visual intelligence and evaluation layer. It is not "image generation." It is the eyes of autonomous agents: websites, dashboards, mobile layouts, screenshots, branding, spacing, type, a11y, broken layouts, forms, buttons, charts, nav, app states, visual regressions.

Agents today see HTML, APIs, and logs. Users see pixels. VIZN is the bridge. See EE VIZN.

11. Agent Lenses

Traditional agents observe APIs, databases, files, text, DOM, logs. EE VIZN adds visual state.

Think -> Act -> Observe API -> Observe database -> Observe visual result -> Evaluate -> Correct -> Repeat

Example: HTTP 200, DOM contains the button, CSS overlap hides it. A text-only agent reports success. A lensed agent stores a failing screenshot, a defect tag, and a Pastes evaluation. That distinction is the product. See Agent Lenses.

12. Website visual optimization

Website -> Automated browser -> Screenshot -> EE VIZN analysis -> Pastes experiment -> Score -> Generate alternatives -> Screenshot alternatives -> Score -> Select winner -> Human or auto promote -> Deploy -> Recheck -> Record outcome

Winning patterns become shared design intelligence: nav, cards, dashboards, forms, pricing, onboarding, checkout, buttons, type, tables, charts, landing sections, mobile layouts.

Example: Card v1 = 71, v2 = 76, v3 = 88. v3 is the recommended template. Pastes is the historical memory behind the EE visual language. FUTURE. Compacter can consume winners; it does not today.

13. Machine learning feedback loop

Each experiment yields: input, candidates, scores, winner, rejected outputs, human edits, production outcome.

Learnable targets (RESEARCH):

• P(best model | task type, application, history)
• P(best prompt | objective)
• P(human approval | candidate attributes)
• P(conversion improvement | visual configuration)
• P(defect | screenshot characteristics)

Phase 7 trains rankers on this ledger. Do not train on paste bodies that contain secrets. Redact first (see Security).

14. Proprietary dataset opportunity

The moat is not storage. The moat is the joined history of task, model, prompt, config, visual output, machine rating, human rating, and production outcome. That dataset is expensive to rebuild and gets more valuable with every EE app that writes experiments.

Network effect: more apps -> more experiments -> better routing -> better generations -> more apps on the ledger.

15. Model Router

EE Model Router (FUTURE): task + project + requirements in. Query Pastes. Choose model, prompt, temperature, tools, evaluation policy, candidate count. Generate. Evaluate. Store. Update stats.

Closed loop. See Model Router.

16. API architecture

CURRENT (CodeShare, pastes.reversal.org):

• GET /api/v1/pastes/ pvsk_ list (yours only; 401 without a key)
• POST /api/v1/pastes/ session create
• POST /api/v1/notes/ pvsk_ create (unlisted default, external_id)
• GET /api/v1/notes/ last 50 for that key
• GET/POST/PATCH /api/v1/keys/ session mint/revoke

CURRENT (Command, projectcommand.reversal.org): POST /api/v1/pastes/ bind or ping with cpcl_.

PLANNED experiment API (same envelope, new resources): versions, experiments, runs, evaluations, assets, ratings, leaderboard, promote, models/performance, models/recommend. REST first. Webhooks and SDKs later. See API.

17. Data model

CURRENT MySQL pastes_db: pastesv2_users, pastesv2_pastes (+ api_key_id, source, external_id), pastesv2_api_keys, pastesv2_api_rate. Staff flag is_admin.

PLANNED PostgreSQL (or MySQL if ops cost wins) plus object storage: organizations, projects, paste_versions, experiments, experiment_runs, models, prompts, evaluations, ratings, assets, deployments, datasets, webhooks, audit_logs.

Large screenshots do not live as LONGBLOB in the primary DB. See Data model.

18. Security architecture

CURRENT. Hashed pvsk_ (SHA-256), shown once. CSRF on session key mint. Rate 120/min per key, 20 keys/user. Unlisted API notes. Command encrypts bound keys (AES-256-GCM). Admin hides raw secrets and password hashes. TLS at the edge.

PLANNED. RBAC, org/project isolation, signed asset URLs, secret redaction, retention/deletion, audit logs, webhook HMAC. Never store credentials inside paste content. See Security.

19. EE Holdings integrations

20. Project Command integration

CURRENT UI: CodeShare notes bind + ping. Snap-in pastes.

PLANNED dashboard: Models, Experiments, Evaluations, Deployments, Visual QA, Best Configurations. Cards: production model, challengers, win rate, average score, cost, latency, human approval, running vs completed experiments, regressions, promoted versions.

21. Architecture diagram

flowchart LR
  apps[EE Applications]
  api[Pastes API]
  db[(MySQL now / PostgreSQL planned)]
  obj[Object storage FUTURE]
  eng[Experiment engine PLANNED]
  q[Queue PLANNED]
  models[Model providers]
  eval[Evaluators PLANNED]
  vizn[EE VIZN FUTURE]
  cmd[Project Command]
  apps --> api
  api --> db
  api --> obj
  api --> eng
  eng --> q
  q --> models
  q --> eval
  q --> vizn
  api --> cmd

CURRENT path: Application or Cursor -> Pastes notes API or Command forwarder -> pastesv2_pastes row. No queue, no VIZN, no object store.

22. Experiment workflow diagram

flowchart TD
  b[Baseline] --> s[Score]
  s --> c[Generate challengers]
  c --> e[Evaluate]
  e --> r[Rank]
  r --> w[Select winner]
  w --> p[Promote]
  p --> l[Learn]
  l --> b

Status: PLANNED (Phase 2-5).

23. Vision QA workflow diagram

flowchart TD
  a[Agent] --> act[Application action]
  act --> apiObs[API observation]
  act --> br[Browser observation]
  br --> vis[Vision analysis]
  apiObs --> ev[Pastes evaluation]
  vis --> ev
  ev --> fix[Correction]
  fix --> re[Re-test]

Status: FUTURE (Phase 3 and 6). Playwright does not run in Pastes today.

24. Model selection workflow

flowchart LR
  t[Task] --> hist[Pastes historical data]
  hist --> r[Model Router]
  r --> cfg[Best model plus configuration]
  cfg --> g[Generate]
  g --> ev[Evaluate]
  ev --> st[Store outcome]
  st --> r

Status: FUTURE (Phase 5-7).

25. Development roadmap

Phase 1 CURRENT/next: paste API, keys, notes, admin, then version history, projects, basic ratings. Phase 2: experiments, candidates, model/prompt metadata, leaderboards, human winner. Phase 3: eval workers, Playwright, vision scoring, regression. Phase 4: Command analytics. Phase 5: router + automatic challengers. Phase 6: EE VIZN lenses. Phase 7: ranker from history. Phase 8: external developer platform.

Detail: Roadmap.

26. Commercialization strategy

Internal ledger first. Later products: Developer, Teams, Experiments, Vision, Router, Enterprise. Revenue: seats, API usage, eval/browser/vision runs, retention tiers, private cloud. Do not sell customer paste bodies. Sell the evaluation and routing control plane. Commercialization.

27. Competitive moat

Storage is not a moat. Joined experiment history is. Competitors can clone a paste API in a weekend. They cannot clone years of EE-specific (task, model, screenshot, human, production) tuples. Switching cost is the dataset and the router calibrated on it.

28. Risks and limitations

• Evaluating UI with LLMs is noisy; always keep human override.
• Auto-promote can ship a high-scoring inaccessible layout; keep payment and auth paths on deterministic tests.
• Screenshot storage and browser minutes are expensive; cap per project.
• Training on pastes can leak secrets; redaction is mandatory before RESEARCH training.
• PostgreSQL/S3/K8s are not required on day one; premature platform rewrite delays the ledger.
• Public CodeShare content is not an experiment corpus; only keyed, project-scoped writes should enter routing stats.

29. Long-term vision

Pastes is the memory. EE VIZN is the eyes. The Model Router is the habit. Together they let EE Holdings software improve because the company measures generations the way it already measures deploys.

30. Conclusion

Ship Phase 1 honesty: CodeShare plus agent notes is a working ingest API, not a lab. The next durable step is immutable versions + ratings on that ingest, not a greenfield Next.js rewrite.

When versions and scores exist, VIZN and the router have something to learn from. Until then, every "best model for UI" claim remains an anecdote.

Pastes exists so EE does not throw away the experiment.