Skip to content

Overview of a mycroblog server

The first page to read before running a server. mycroblog is a microblog that federates with the rest of the fediverse over ActivityPub; my is its server software, one binary that holds the server, the job worker, the web client, and the commands that look after them. This page shows what a server is made of, what you choose by configuration, how a small server grows into a large one, and where to read next.

The architecture of a mycroblog server: clients reach the my processes over HTTPS; the processes use the parts chosen by configuration and the services outside the server that you turn on. The sections below describe each part.

From the top of the diagram down:

  • Who connects. People use the web client in their browser (it can be installed as an app), apps and bots use the client API, and other fediverse servers deliver and fetch posts over ActivityPub. All of them reach the server over HTTPS.
  • In front of the server. By default my speaks plain HTTP, so a reverse proxy or load balancer terminates HTTPS in front of it. On a single machine, my can serve HTTPS itself instead (serving HTTPS from the server). A CDN in front of the media files is optional.
  • The my processes. Each process runs my serve, and its role says what it runs: web the HTTP server, worker the job worker, which delivers posts, fetches from other servers, sends email, processes media, and runs the periodic jobs, and all, the default, both in one process.
  • The parts you choose. Each part in the middle of the diagram has one setting that picks its adapter, listed in the next section.
  • Services outside the server. CAPTCHA, voice chat, payments, plugins, and the import of cloud costs each stay off until you set them, and the server then calls the service you name.

Every part is a setting of the configuration reference, set in a TOML file or as an environment variable (MY_ and the key in capitals, such as MY_DATABASE). The first option of each list is the default. Where a part keeps data, choose it before the first start: the data does not move by itself when the setting changes.

Part Setting Options Read next
Database database sqlite (a file), postgres (PostgreSQL and compatible services) Database, backup
Job queue queue db (a table in the database), inline (runs each job at once in the same process, for tests and development), cloudtasks (Google Cloud Tasks wakes the server for each job; needs PostgreSQL) The job queue
Key-value store, for caches, rate limits, and the answers kept for a day to writes sent again kv db (a table in the database), memory (each process on its own), valkey (Valkey or another Redis-protocol server, shared by every process) kv
Media storage blob fs (a directory), s3 (Amazon S3, Cloudflare R2, MinIO, Garage, and other S3-compatible stores), gcs (Google Cloud Storage) Docker Compose files, backing up media
Media processing media_image, media_video, media_audio, one for each kind of file native (in the server), plugin (a plugin of yours converts the files of that kind) Converting media
Real-time stream between processes stream hub (inside one process), postgres (LISTEN / NOTIFY), valkey, sns (Amazon SNS and SQS), pubsub (Google Cloud Pub/Sub) stream
Key manager, which encrypts the accounts’ private keys key_manager local (a key-encryption key you keep), gcpkms (Google Cloud KMS), awskms (AWS KMS), plugin Managing the key
Email mail none, smtp, ses (Amazon SES), plugin Sending email
CAPTCHA captcha none, turnstile (Cloudflare Turnstile), hcaptcha, recaptcha (Google reCAPTCHA v2) CAPTCHA
Voice chat voice off, p2p (browser to browser, with TURN from voice_turn: none, coturn, or cloudflare), livekit, sora Voice chat
Supporter payments payments none, stripe, payjp (PAY.JP) Supporters, Stripe, PAY.JP
Plugins plugin_<kind>_address none by default; one address for each kind: translator, search, billing, mailer, media_image, media_video, media_audio, key_manager, and the decision hooks inbound.status, local.status, inbound.follow, signup.check, media.check, inbound.actor, local.profile Plugins, decision hooks
Translation of posts translator_<provider>_*, plugin_translator_address none (only the browser’s own translation); libretranslate, deepl, google (Google Cloud Translation), aws (Amazon Translate), or a plugin of yours in front of the service you choose Translating posts, a translator plugin
Full-text search of posts search none; or elasticsearch, opensearch, or meilisearch (search engines); or plugin, a plugin of yours in front of another engine Full-text search, a search plugin
CDN for media media_origin unset (the server serves media from its own origin), or the origin of your CDN Serving media through a CDN
Cloud costs billing_sources none, or any of aws (AWS Cost Explorer), gcp (Google Cloud’s billing export or budget), cloudflare (Cloudflare’s usage charges), plugin Importing cloud costs
Metrics metrics_exporter none, prometheus, otlp (OpenTelemetry), emf (CloudWatch) Metrics, alerts
Traces traces_exporter none, otlp Traces
Error reports errors_exporter none (the log line and the error groups only), sentry (Sentry, GlitchTip, Datadog), otlp Error reporting

The released binary and container image hold every adapter of the table.

my config check reports every problem of the configuration at once, before you start the server (production checklist).

Three steps from a small server to a large one: one process on SQLite and the file system, then PostgreSQL and object storage behind a CDN, then separate web and worker processes with a stream broker. The list below describes each step.

  1. Start small. One process with role = all, on SQLite, with media on the file system and the defaults for the rest: the queue and the key-value store in the database, and the real-time stream inside the process. The Docker Compose files in deploy/docker start this setup on one machine (deploying with Docker Compose). It suits a personal or small server. stream = hub reaches only the process it runs in, so this setup stays at one process.
  2. Move the data out. PostgreSQL for the database, an S3-compatible bucket or Google Cloud Storage for the media, and a CDN in front of the media. The server is still one process, but it no longer depends on one disk, and the processes of the next step can share the data. Decide on this before the first start if you can: moving the data from SQLite and the file system later is a job of its own (what to back up lists what has to move).
  3. Scale out. Run processes with role = web for the requests and role = worker for the jobs, as many as each needs, and split the workers by job type (splitting workers). With more than one process, the real-time stream needs a broker that reaches every process (postgres, valkey, sns, or pubsub), and the key-value store should be shared (db or valkey), since memory keeps the rate limits and the answers to writes sent again of each process apart. The job queue stays in PostgreSQL, or moves to Google Cloud Tasks (Cloud Tasks).

The Cloud Run deployment is an example of a setup between the second and the third step: role = all on Cloud Run, PostgreSQL on Neon, media in Cloud Storage, Cloud Tasks for the jobs, and Pub/Sub for the stream, with no instance running while nobody uses the server.

Read the operator guides in this order:

  1. Get started: run the server or deploy it on Cloud Run, then go through the production checklist before you open it to users.
  2. Configure the instance: its settings, registration, external login providers, email, CAPTCHA, the CDN, link previews, and search engines and AI crawlers.
  3. Turn on features: moderation, relays, webhooks, plugins and decision hooks, voice chat, full-text search, supporters and their payments, and importing cloud costs.
  4. Run it day to day: monitoring, alerts, error reporting, maintenance, backup, the database, the key-encryption key, and administrators and roles.
  5. Legal and closing: the privacy policy and record of processing templates, and closing the service.

The configuration reference lists every setting, and the jobs reference every job the workers run.

For LLMs and coding agents:this page as Markdown, llms.txt,llms-full.txt, llms-small.txt

Analytics on this site