Lukas Murdock
Subscribe for updates and more.

CMS

Planted 02022-03-25

The future of document editing.

The web is mere documents—yet, it sucks at document editing, publishing, and subscribing. Social networks provide better ways to edit, publish, and subscribe to documents. So let’s reinvent how we edit, publish, and subscribe to documents.

Documents should:

  • Be local-first
  • Be editable across devices
  • Support real-time collaboration
  • Have an easily transformable format (ASTs)
  • Be publishable anywhere (default to versioning since last publish)
    • Backup anywhere
      • Google Drive
      • Dropbox
      • iCloud
      • Github
      • S3
      • etc.
    • to a webhook
    • to an email list
    • as a Twitter post/thread
    • as an Instagram post
    • etc.
  • Have useful document history
  • Collectible by default (e.g., collections/content types/document types: Posts, Notes, Images, Books)
  • Be block-based
  • Provide a graphical interface for block editing
  • Provide default block types that have customizable fallbacks in text and image formats (e.g., image has text fallback “alt text”, text has image fallback, and posts and recipes have image fallbacks—enabling Instagram posts):
    • Post → Posts (WordPress)
    • Note → Notes (Twitter)
    • Image → Images (Instagram/VSCO)
    • Photo → Photos (Glass)
    • Book → Bookshelf (Goodreads)
    • Bookmark → Bookmarks (Pocket/Instapaper)
    • Recipe → Recipes (All Recipes?)
    • Task/Habit → Tasks/Habits (Todoist/Trello/Habitica)
    • TimelineEvent → Timeline (iffy, but LinkedIn? Facebook?)
    • Countdown → Countdowns (why do calendars not give time-until???)
    • Image these blocks:
  • Provide a graphical interface to optionally build custom blocks (collection types) with text and image fallback types
  • Provide a graphical interface to optionally build custom “page” schemas with blocks
  • Provide a default optional website builder that uses the layout schema to build pages and provides web feeds for all collections
  • Provide a plug-and-play optional website hosting solution
  • Provide optional hosted endpoints for API-based edits (but still local-first)

Our lives revolve around editing documents, so this is more of an “operating system for life”, as AnyType puts it.

The documentation for AnyType looks promising and the vision aligns. It’s a version of what I’ve been envisioning. Wow, seriously this is amazing.

The vision of Anytype premise is: “operating system for digital objects”. Applications work with digital objects, so let’s remove the boundaries between applications and focus on making interoperable digital objects.

Structured data formats (relevant xkcd):

Subject to change, it seems the web pretty much standardized JavaScript Object Notation (JSON) as the data-interchange format.

Twitter links software Similar to Anytype:

Problems with future document authoring on the web

Thinking in layers,

  • Layer 1: Documents need to be local-first (and support real-time collaboration) with a document level block-editor GUI, optional layout GUI to design the view layer, and powered by AST documents and support for Layer 2.3
    • Layer 1.1: A document needs document creation tools, imagine custom blocks for:
      • Excalidraw (Basically built for this)
      • Figma image design
      • Canva image design
      • Photoshop
      • Note: why do CMS not have such document creation built-in?
  • Layer 2: Provide transforms for AST documents to publish anywhere
    • Layer 2.1: Publish data to a private storage platform
      • Layer 2.1.1: Generate a static site and push to storage platform
    • Layer 2.2: Publish data to a social network
    • Layer 2.3: Publish data to a hosted server to generate and host a website

Everything below this point is thought vomit. The above is my attempt at organizing it.

Simplified, there are only two things most websites should have:

  1. Collection editor GUI with customizable collection schema
  2. Layout editor GUI with block editing for collection items

But, layout building should also be optional—it starts with editing collections and collection items—a headless CMS. Document editing. And if documents are made of blocks, block editing.

Simplified, collection types (blocks) are a group of fields that define the makeup of the content—a form. E.g., a book collection with a book collection type might provide title, author, isbn, numberOfPages, and datePublished fields. But a recipe collection type might provide cookTime, recipeCategory, recipeCuisine, recipeInstructions, recipeYield, recipeIngredient, and step fields.

Social networks have become the primary document editors and publishers. Social network platforms took over the web in part due to providing better default solutions to creating collection types, publication, and subscription than the web. It’s time to bring it back.

Social networks have proven what collection types people enjoy creating. A CMS should have default collection types for all the content big social networks and more:

  • Writing, blogging (WordPress)
  • Notes, microblogging (Twitter)
  • Photos (Instagram/VSCO)
  • Bookshelf (Goodreads)
  • Bookmarks (Pocket/Instapaper)
  • Recipes (All Recipes?)
  • Timeline (iffy, but LinkedIn? Facebook?)
  • Now, Calendar, Countdown (platform bios? But also why do calendars not give me time-until???)
  • Task, Habits, Kanban (Todoist/Trello/Habitica)

A CMS should also be extensible to create custom collection and block types—like Canva templates.

A CMS should have the ability to publish anywhere. This means the data must be separate from the view layer—and editing will probably default to some form-styled block editing (which is how current CMSs mostly handle block editing). This means storing collection items as data (e.g., JSON structures) instead of HTML or markdown to better control the view.

There are three steps in a document lifecycle:

  1. Document editing/authoring
  2. Publication
  3. Subscription

In a sentence: I envision software to collaboratively edit custom documents offline and in real-time, publish anywhere (easily moveable and ownable), and subscribe.

The closest thing I’m aware of to a collection editor GUI with customizable schema would be the Strapi content types builder, with configurable fields

Field type resources:

  • Strapi fields
    • Text
    • Rich text
    • Number
    • Date
    • Boolean
    • Relation/Link (Link to a collection)
    • Email
    • Media
    • JSON
    • UID
    • Collection (group of fields)
  • Sanity schema types
    • Array
    • Block (rich-text editor for block content)
    • Boolean
    • Date
    • Datetime
    • Document
    • File
    • Geopoint
    • Image
    • Number
    • Object
    • Reference
    • Slug
    • String
    • Text
    • URL
  • Netlify CMS built in entry fields
    • Boolean
    • Code
    • Color
    • DateTime
    • File
    • Hidden
    • Image
    • List
    • Map (GeoJSON)
    • Markdown
    • Number
    • Object
    • Relation
    • Select
    • String
    • Text
Strapi Sanity NetlifyCMS
Text String (Text type for multiline) String (Text type for multiline)
Rich text Block Markdown
Number Number Number
Date (Date/DateTime/Time) Date (also DateTime) DateTime
Boolean Boolean Boolean
Relation Reference Relation
Email N/A N/A
Media File (Image type for images) File (Image type for images)
JSON N/A? N/A
UID N/A N/A
Collection (group of fields) Array List
Enumeration String Select
Component Object Object
N/A N/A Color
  Geopoint Map (GeoJSON)
  URL  
  Slug  

The closest thing I’m aware of to a layout editor GUI with block editing for collection items is Universe website builder—it really is a magical new way to build websites.

NetlifyCMS treats calls custom blocks blocks “widgets” and treats them as forms with fields.

The Language and Protocols of the Web:

  • Document language: HTML
  • Publication protocol: Atom, WebSub
  • Subscription protocol: Atom, WebSub

Feeds in the form of Atom are wonderful. However, companies don‘t want you to use them, instead they want you to:

  1. CONSUME INCESSANTLY
  2. Give them more data

I’m not very technically literate in this area, but I primarily use RSS 2.0 and Atom feeds as a subscription method. I’m not sure there’s a default way to handle federated access control.

But, the default document editing experience (HTML) is garbage compared to the interfaces provided by social networks—most everyone will never pick HTML as a language to edit documents.

Markdown took off as the primary document experience by providing an “easy-to-read, easy-to-write plain text format” that would be converted to structurally valid HTML. Markdown is remarkably pervasive in tech-oriented places.

Historically, “plain text” was optimal for a few reasons:

  1. Interoperability
    • It’s simple to process and a timeless format—guaranteeing readability ten years from now
  2. Quick, easy, distraction-less authoring
  3. Distributed access (access across applications or systems)
    • Easy distribution means easy ownership

The Rich-text format is a data format for saving and sharing documents—now supported by most word processing software.

The vision

The Stages of document authoring:

  1. Past: Local-first and offline Word document editing, send document back and forth with comments for changes.
  2. Present: Online-only real-time document collaboration with Google Docs and Word for the web. Local-first documents with bad syncing issues. And the document is separate from any form of publishing.
  3. Future: Offline Local-first document collaboration with online real-time capabilities, custom block editing, and the ability to publish anywhere.

Editing

Documents are a block-editing experience. With the ability for custom blocks to be made and shared by others.

Publishing

Ready to publish a new document version?

  • Choose to save it to (Layer 2.1):
    • Google Drive
    • Dropbox
    • iCloud
    • Github
    • S3
    • etc.
  • Choose to push it out (Layer 2.2):
    • to your email list
    • to a webhook
    • as a Twitter post/thread
    • as an Instagram post
    • etc.
  • Choose to push it to your website with layouts also designed with the CMS (Layer 2.3)

As this CMS is local by default, there is no always-on “server” by default without self-hosting; however, you can’t expect most people to self-host. Thus, there should be strong defaults to sync to hosted “publishing servers”.

The base is a local CMS that’s publishable and movable anywhere. The power comes in building features for “online-nodes” than can be either your local instance or a connection to an always-on publishing server—akin to WordPress, and just like WordPress, publishing web-servers should provide defaults Atom feeds for every collection and support layouts built in the CMS to generate a website. “Always-on” publishing servers seem required to build in default protocols like Atom and enable websites. But it’s important to note that the base is always moveable data and online connectivity or publishing is not required and other people can build publishing servers that handle the data differently.

Aside: If there are publishing servers, could you try to provide free content hosting by inserting advertisements into the feed? And then “pay for no ads” on both sides of the market (either publishers or subscribers can pay). Or maybe sponsorships?

To be able to publish anywhere, blocks would require fallback formats:

  1. Nothing, hide
  2. Text, markdown (rich-text)
  3. Image (Canva and Excalidraw are web image editors, let’s build one into the CMS)

And to publish to the hosted website, a similar layout editor can be used to simply build block layouts.

Documents should be stored as an Abstract-Syntax-Tree (ASTs) (see: unified) so transformers/compilers can convert document to different output formats.

The future is powered by Conflict-free Replicated Data Types (CRDTs).

Conflict-free Replicated Data Types (CRDTs)

TL;DR two people editing on the same document means conflicting edits. Conflict-free Replicated Data Types (CRDTs) provide an easy way to handle said conflicts.

CRDTs are the future.

CRDT Resources:

Data type for real-time rich-text editing

Preface: I know very little about data structures and only have experience primarily writing TypeScript over the past two years.

CRDT rich-text editing resources

Operations: insert, remove, addMark, removeMark.

  1. Create event sourcing format
  2. Turn events into a proper document display
[
  { char: "T", opId: "9@B", deleted: false, markOpsBefore: [
    {
      action: "addMark",
      opId: "19@A",
      start: { type: "before", opId: "9@B" },
      end:   { type: "before", opId: "10@B" },
      markType: "bold"
    }
  ] },
  { char: "t", opId: "1@A", deleted: true },
  { char: "h", opId: "2@A", deleted: false },
  { char: "e", opId: "3@A", deleted: false },
  { char: " ", opId: "4@A", deleted: false },
  { char: "f", opId: "5@A", deleted: false },
  { char: "o", opId: "6@A", deleted: false },
  { char: "x", opId: "7@A", deleted: false },
  { char: " ", opId: "10@B", deleted: false, markOpsBefore: [] },
  …
]

Iterate over character list and convert each set of mark operations to the corresponding formatted span.

[
  { text: "The ",     format: { bold: true } },
  { text: "fox",      format: { bold: true, italic: true } },
  { text: " jumped.", format: { italic: true } }
]

When an operation is applied:

  1. Updating internal document state
  2. Compute a patch describing how the operation affects the text editor document state
// a patch to insert the letter “x” at index 6 with bold formatting might look like this
{
  type: "insert",
  char: "x",
  index: 6,
  format: { bold: true }
}

Data type for real-time block editing

Again, I don’t even know what I don‘t know about data structures.

However, my thinking is to lean into ASTs in the unified Universal Syntax Tree format.

interface Node {
  type: string
  data: Data?
  position: Position?
}

interface Position {
  start: Point
  end: Point
  indent: [number >= 1]?
}

interface Point {
  line: number >= 1
  column: number >= 1
  offset: number >= 0?
}

Inspiration

Documents should:

Block editing resources

Open standards