Aug 27, 2026

Engineering

Attachments Engineering

Summary

The team debated architecture for comments and recursive capabilities, deciding to defer generic CRDT system development.

Managing Comments and Capabilities
Discussions centered on modeling comments as nodes and managing recursive capabilities for documents. The group clarified that capabilities remain recursive within the current product structure.

Plugin and Architecture Trade-offs
Debates addressed using external processes for plugins to ensure daemon stability and data safety. Participants evaluated the trade-offs of using snapshots versus full history for verifiable attribution.

Deferring Generic System Design
The team decided to avoid designing a generic JSON conflict-free replicated data type system, prioritizing stable node identifiers and existing data structures instead. This approach simplifies implementation while allowing for future architectural expansion.

Decisions

Aligned

    'No recursive' capability deprecation The 'no recursive' capability feature is deprecated and will not be supported in the new system model.

    User invitation scope limitation User invitations are restricted exclusively to the homepage, disabling the ability to invite users to specific sub-documents.

    Nodes architecture and data adoption The team will proceed with the existing nodes architecture and current data while deferring the design of a new, custom JSON-like CRDT system.

    Custom CRDT system development The team will develop a custom CRDT system instead of adopting existing third-party libraries to ensure necessary support for decentralized attribution and history management.

    Support for mime types and schemas The system will support both mime types and schemas to allow for robust data description and interpretation.

    Capabilities defined as special resources Capabilities will be designated as a special, distinct category of resources within the permission and node hierarchy system.

    Mutable entity naming convention The mutable entities in the system will be officially named 'resources'.

    Home document structural model Home documents and profiles will be modeled as nodes anchored to an empty parent with a fixed, hardcoded node ID.

    Comment and edit mechanism Comments will be created as new nodes without IDs, while edits to existing comments will be implemented by referencing the original node ID.

    Data migration strategy Data migration will result in comments, contacts, and profiles being converted to a snapshot format, while documents will be converted into a tree of changes.

    We've updated the Decisions section using your feedback.

    Let us know what you think: Helpful or Not Helpful

    Next steps

      [ ] [The group] Stop Legacy Capabilities: Stop respecting legacy path-based and non-recursive capabilities in the system after the app update.

      [ ] [The group] Restrict Capability Creation: Refuse or hide new capabilities that are not associated with the home document.

      [ ] [The group] Deprecate Recursion Logic: Deprecate and remove the no recursive concept from the system architecture.

      [ ] [The group] Document Responsibilities: Define the division of responsibilities for the daemon, specifically regarding data syncing, storage, and permissions.

      [ ] [The group] Document Questions: Compile a list of unanswered questions concerning the generic resource system architecture.

      [ ] [The group] Decide Naming Conventions: Agree on final names for mutable and immutable system objects to ensure consistent internal communication.

      [ ] [The group] Research CRDT Systems: Explore existing JSON-like CRDT libraries to identify potential solutions or development patterns.

      [ ] [Eric Vicenti] Outline naming document: Draft an outline in a blank document covering concept naming and migration strategy to prepare for the final document.

      [ ] [Alexandr Burdiyan] Write summary document: Draft a summary document declaring precise names for system elements and migration procedures based on the outline provided.

      [ ] [The group] Review naming document: Convene to collaborate on the naming and migration document to ensure progress and alignment.

    Details

    Did the screenshots in this section make your notes better or worse?

      Comment Modeling and Space Permissions: Eric Vicenti proposed modeling comments as normal nodes using invented identifiers and transaction state IDs under the root, while Alexandr Burdiyan raised a concern that space owners granting access to their entire space would allow writers to edit comments, blurring the line between writer and agent capabilities. Eric Vicenti argued that this is a sound policy logically since space writers have access to everything in the space, and they noted that Gabo H Beaumont's home document members do not have active writers anyway.

      Contacts, Capabilities, and Legacy Node IDs: Gabo H Beaumont mentioned that if users need to add permissions again it is fine as long as links do not break and content is not deleted. Alexandr Burdiyan questioned whether contacts and capabilities should be separate things, with Eric Vicenti agreeing that capabilities should be separate resources. Alexandr Burdiyan noted that treating comments as nodes allows space writers to write comments as those spaces, though overlapping reference and comment names could cause references to supersede comments. Alexandr Burdiyan suggested using a special legacy node ID type for comments requiring space ownership or agent status to operate on them.

      Evaluation of Recursive Capabilities: Eric Vicenti raised the topic of recursive capabilities and whether to revisit having documents that restrict writing access despite homepage invitations, while Alexandr Burdiyan and Eric Vicenti noted that default capabilities are recursive and a "no recursive" true flag was previously used. Alexandr Burdiyan stated they should keep default recursion and not support non-recursive capabilities yet, agreeing with Eric Vicenti to deprecate and delete the "no recursive" concept entirely to simplify the system. Alexandr Burdiyan and Eric Vicenti planned to stop respecting capabilities using paths or the "no recursive" journey after the upcoming application update.

      App Restrictions on Sub-Document Sharing: Alexandr Burdiyan and Eric Vicenti discussed whether the application currently allows creating non-recursive or path-scoped capabilities, noting that capabilities are always recursive in the product currently. Eric Vicenti pointed out that grandchildren documents from the home will fail because node IDs are derived by hashing. Horacio Herrera committed to removing the ability to share documents in a non-recursive way immediately, ensuring users can only invite people to their homepage.

      Managing Special Blob Types and Resources: Eric Vicenti and Alexandr Burdiyan debated how to handle comments and whether to hardcode comment types or use a generic approach. Gabo H Beaumont urged the group to consider performance and immediacy when discussing comments. Alexandr Burdiyan argued against requiring users to create extra wrapper blobs for raw data types like portable document format files or emails, wanting custom indexer code and plugins to handle blob type introspection. Eric Vicenti questioned how the network would know and respect permissions if raw data is placed in InterPlanetary File System without a referencing node.

      Indexing and Custom Plugin Architecture: Alexandr Burdiyan asserted that schemas, contacts, and comments should exist without requiring node blobs describing them, whereas Eric Vicenti maintained that a single system should manage permissions and mutations. Alexandr Burdiyan advocated for loading runtime code and plugins to interpret various data types, while Eric Vicenti argued that the daemon does not need custom code and can utilize a generic indexer based on conventions. Alexandr Burdiyan questioned how external data formats like tweets or emails would be indexed without custom plugins to parse target fields and citations.

      Generic Change Formats and CRDT Limitations: Eric Vicenti described an approach for importing tweets by downloading source formats like portable document format, hypertext markup language, and JavaScript Object Notation into InterPlanetary File System, converting them into hypermedia standard blocks, and referencing them in a resource. This led to a discussion about needing a generic change format operating as a conflict-free replicated data type engine for arbitrary data not strictly tied to document structure. Alexandr Burdiyan noted that documents currently allow free-form data in changes as long as lists and blocks are avoided, while Eric Vicenti argued document semantics are too restrictive.

      Improving CRDT Operations and List Item Identifiers: Eric Vicenti highlighted limitations in the current document conflict-free replicated data type set of operations regarding metadata and move operations, suggesting small changes to assumptions and rules can create a generic conflict-free replicated data type. Eric Vicenti proposed that every list item must be an object with a unique identifier to allow reliable addressing and secure move operations. Alexandr Burdiyan noted this approach imposes limitations similar to blocks, requiring preserved history, while Eric Vicenti explained that changes and snapshots can be interspersed, allowing previous versions to point to snapshots.

      Snapshot Verification and Full History Requirements: Alexandr Burdiyan argued that loading snapshots without downloading the full history loses verifiable attribution, as users cannot verify which parts were authored by whom. Eric Vicenti countered that snapshots can be cryptographically signed, though Alexandr Burdiyan emphasized that trustless systems require downloading the full history to understand git-style blame and authorship. Eric Vicenti acknowledged that full history is needed to understand history, but snapshots help reach the current state by walking back to the previous snapshot. Alexandr Burdiyan questioned how comments fit into this, asking if they should use blob nodes and changes or snapshots.

      Plugin Execution and Externalized Architecture: Eric Vicenti suggested introducing a snapshot format for node blobs where snapshots can point to raw data or contain actual data, similar to how comments function. Alexandr Burdiyan raised concerns about full-text search and how the system generically knows which parts of a comment are searchable without custom plugin execution or domain-specific languages. Eric Vicenti shared a shift in thinking, proposing that instead of running untrusted plugin code inside the daemon, full-text search engines and plugins should run as sibling processes external to the daemon so that a crash does not take down the daemon. Alexandr Burdiyan questioned how developers would share plugins if they could not use direct text files or WebAssembly code loaded by the daemon.


      Sandboxing and Division of Responsibilities: Alexandr Burdiyan argued that plugins can be sandboxed securely, citing examples like Caddy, Traefik reverse proxies, and HashiCorp Terraform plugins that run safely as separate processes with RPC communication, similar to C Lightning. Eric Vicenti maintained that the daemon's primary responsibility must remain strictly focused on data syncing, storage, and permissions, keeping other systems interfacing through existing methods to protect data safety. Eric Vicenti emphasized that unifying around the resource model provides explicit, baked-in permissions, making it clear who is allowed to read random linked data.


      Wrapping Raw Data and Custom Indexing Trade-offs: Alexandr Burdiyan returned to the email scenario, suggesting that raw data like emails could be wrapped in a node resource with a node ID derived from hashing the subject, while the content remains interpretable via custom plugins rather than strict document shapes. Eric Vicenti outlined the user choices: either restructure data to fit the daemon's indexer to utilize built-in querying features, or build a third-party indexer via an interface where the daemon remains unaffected if the third-party indexer crashes. Alexandr Burdiyan agreed that this trade-off is fair.

      Node IDs, Paths, and Trustless Systems: Eric Vicenti pointed out the irony that node IDs are arbitrary strings and user-defined, making them functionally similar to deprecated paths in reference version 1, except for legacy refs that use path hashes. Alexandr Burdiyan explained that full space writers can use arbitrary strings for node blobs, whereas users invited to only a subset of a site must use non-forgeable node IDs. Eric Vicenti noted that non-forgeable node IDs are critical for preserving parent-child relationships deterministically within existing sites. Alexandr Burdiyan discussed potential edge cases with maintaining pretty human-readable paths republishing stable opaque IDs, but noted permission and efficiency issues.

      Challenges of Generic JSON CRDT Systems: Alexandr Burdiyan stated that developing an efficient generic JavaScript Object Notation conflict-free replicated data type system capable of expressing links and resource IDs requires substantial additional thinking before it can be declared usable. Eric Vicenti suggested documenting all questions and sketching solutions in the meantime, inquiring about conflict resolution during concurrent attribute renames. Alexandr Burdiyan noted that object identifiers are required to handle moves and concurrent updates safely.

      Object Identifiers, Move Operations, and Future Plug-ins: Eric Vicenti affirmed that move operations require target objects to have unique identifiers within the document, and any sane implementer would reject operations with duplicate identifiers. Alexandr Burdiyan acknowledged that link indexing requires data to be formatted into standard conflict-free replicated data type forms or interpreted via custom plugins. Eric Vicenti concluded that future plug-in systems could handle custom operations by matching operation names to specific plugins, while current investments in resources and node blobs do not close off future architectural paths.


      JSON-Like Conflict-Free Replicated Data Type System Design: Alexandr Burdiyan proposed proceeding with the nodes idea and current data instead of immediately designing a generic JSON conflict-free replicated data type system, expressing that *they* hoped to avoid designing such a system immediately. Eric Vicenti agreed, noting that arbitrary data shapes can use snapshots and document operations for now, while *they* can mix generic operations and snapshots in the future. Eric mentioned having an agent crawl the web to find existing JSON-like conflict-free replicated data type systems.


      Evaluation of Existing Conflict-Free Replicated Data Type Libraries: Alexandr Burdiyan reviewed existing conflict-free replicated data type libraries like Laurel, noting that while these libraries embody recent learnings, they lack garbage collection of history and do not support decentralized attribution and signatures. Consequently, Alexandr argued that adopting these external libraries is unfeasible, and *they* will likely need to develop a custom solution. Eric Vicenti suggested that parts of external systems could be adapted similarly to how IPFS was integrated.

      Comment Versioning and Node Block Blobs: Eric Vicenti explained that previous comments can be transparently converted into nodes, allowing new comments to reference previous versions either as snapshots or changes. Eric noted that node block blobs could potentially contain both operations and snapshots, which would require conflict handling and user warnings if discrepancies indicate buggy or malicious software. Alexandr Burdiyan raised concerns about inlining content directly on node declarations to avoid data repetition and size limitations when moving content around.

      Node Schemas and Mime Types: Alexandr Burdiyan suggested defining mime types on nodes to describe payload types, such as hypermedia super JSON conflict-free replicated data types, raw PNG bytes, or Unix file system files. Eric Vicenti proposed that nodes reference schemas addressable via strings, allowing hierarchical extension from base types like strings to markdown and GitHub markdown. Eric argued that schemas enable third-party applications to interpret unfamiliar data types, such as SQLite database schemas. Alexandr maintained that standard mime types managed by the Internet Assigned Numbers Authority should also be supported alongside schemas since these dictionaries are well-defined, a point Eric acknowledged.

      Nomenclature of Resources and Nodes: Eric Vicenti and Alexandr Burdiyan discussed terminology, with Eric proposing that mutable items be called "resources" while immutable data is called "nodes" or blobs. Alexandr argued that nodes are mutable because node content and pointers can change while maintaining a stable identity over their lifetime. Horacio Herrera noted agreement that the stable identity changes rather than the node itself being immutable in terms of content.


      Urgency of Naming Consensus: Eric Vicenti expressed frustration that a lack of consensus on naming conventions slows down progress and communication, urging the team to make definitive decisions. Alexandr Burdiyan requested time to digest the extensive discussions before *they* commit to specific names.


      Home Documents, Profiles, and Comments Structure: Eric Vicenti brought up undefined concepts regarding home documents and profiles. Alexandr Burdiyan and Eric discussed modeling home documents and profiles using fixed node IDs restricted to space owners, mirroring permission models for space editing. In contrast, comments require creating new nodes without prior IDs that reference the user's space. Alexandr agreed to write a comprehensive summary document covering these details.

      Migration Strategy and Legacy Data Conversion: Eric Vicenti outlined a migration strategy utilizing the indexer, where old data passes through a "magic box" function to be transparently converted into new data blobs, while signatures must be validated ahead of time. Eric, Horacio Herrera, and Gabo H Beaumont began sketching core concepts like capabilities, comments, and contacts.

      Permission Inheritance and Scope Management: Alexandr Burdiyan highlighted complexities around permission inheritance, noting that moving nodes can cause nodes to lose inherited permissions and invalidate certain node declarations. Gabo H Beaumont expressed distress over the broad scope of the discussion, worrying that *they* exceeded permissions. Alexandr clarified that *their* current focus is specifically on repurposing legacy data and producing new data for immediate application needs, rather than implementing the generic JSON conflict-free replicated data type system yet.

    You should review Gemini's notes to make sure they're accurate. Get tips and learn how Gemini takes notes

    How is the quality of these specific notes? Take a short survey to let us know your feedback, including how helpful the notes were for your needs.

Do you like what you are reading? Subscribe to receive updates.

Unsubscribe anytime