Production Outage/Issue
CPU was slammed in prod
On-Call rotation for production issues
Monitoring systems
Escalation procedures
Procedures for recovery (that anyone in the OC rotation can follow)
CPU lowered for some daemon requests (ListDocs)
SSR reduced
Mitigated/blocked some crawler traffic
Peak traffic was ~12/req/sec
Follow-through!
General Process
Commit msgs should explain "why", not "what"!
Timeouts in the desktop app
Sometimes, occasionally, hard to reproduce
Eric speculates: ListDocuments
Burdi: probably related to non-paginated requests
InteractionSummary expensive in backend
Uses ListCitations internally. Should be indexed material.
Should we build an automated stress test?
Maybe it relates to writing data? Not just "clicking around" the app
Gabo: Sees ListDocuments timeout immediately after starting the app!!
No user-facing bug noticed
Publish document is VERY slow
We have a 5-sec timeout for PushDocuments
@Untitled Contact owning this project for now <3
Private Documents functionality
Phase 1: Caps on private docs - CONSENSUS!
Write+READ caps
Phase 2: Children of private docs?
Somebody to propose a formal def for permission inheritance
Maybe "inherit permissions" is a flag
Anybody with a link can read/write
Revocations?
Related to invite workflow- grant a cap to somebody who doesn't yet have an account (or a known account ID)
Burdi suggesting having a metadata-based inheritance workflow, rather than using path inheritance.
Eric: We could use flat, non-human readable path names for docs. Use redirects for pretty paths?
Is inheritance implemented through path names or relationships between canonicalized document ID. @Untitled Contact thought experimenting. Along with Revocations
Users assume private by default?
"Public by default" was our mindset as a publishing system, but we are trending towards a knowledge system
AI Threats
"Private documents are private until some advanced AI comes along and hacks your data"
What is the budget in engineering hours? (And in tokens?)
Maybe: 2-eng team works on this every Friday. We will seek help from advisors/friendlies
We need to audit+redteam our deps like libp2p+ipfs.
This is Exercise that we need to build strength
Our Frontier AI wranglers, every Friday: @Untitled Contact + @Eric Vicenti
We are not using production infra! Running in VMs only
Starting Stack: Kimi Sub (because its willing to work) and Pi (good harness that is model-agnostic)
More Agenda?
Review where we have broken (or missing) pagination!
Where do we do frontend sorting?
How to learn about indexed DB schema?
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime