# Overview of a mycroblog server

The first page to read before running a server. mycroblog is a microblog that federates
with the rest of the fediverse over ActivityPub; `my` is its server software, one binary that holds the server, the job
worker, the web client, and the commands that look after them. This page shows what a server is made of, what you
choose by configuration, how a small server grows into a large one, and where to read next.

## What a server is made of

![The architecture of a mycroblog server: clients reach the my processes over HTTPS; the processes use the parts chosen by configuration and the services outside the server that you turn on. The sections below describe each part.](https://github.com/mycroblog/my/blob/main/docs/operations/images/architecture.svg?raw=true)

From the top of the diagram down:

- **Who connects.** People use the web client in their browser (it can be installed as an app), apps and bots use the
  client API, and other fediverse servers deliver and fetch posts over ActivityPub. All of them reach the server over
  HTTPS.
- **In front of the server.** By default `my` speaks plain HTTP, so a reverse proxy or load balancer terminates HTTPS in front of it.
  On a single machine, `my` can serve HTTPS itself instead ([serving HTTPS from the server](https://mycroblog.dev/next/operations/running/#serving-https-from-the-server)).
  A CDN in front of the media files is optional.
- **The `my` processes.** Each process runs `my serve`, and its [`role`](https://mycroblog.dev/next/reference/configuration/#role) says what it
  runs: `web` the HTTP server, `worker` the job worker, which delivers posts, fetches from other servers, sends email,
  processes media, and runs the periodic jobs, and `all`, the default, both in one process.
- **The parts you choose.** Each part in the middle of the diagram has one setting that picks its adapter, listed in
  [the next section](#parts-you-choose-by-configuration).
- **Services outside the server.** CAPTCHA, voice chat, payments, plugins, and the import of cloud costs each stay off
  until you set them, and the server then calls the service you name.

## Parts you choose by configuration

Every part is a setting of the [configuration reference](https://mycroblog.dev/next/reference/configuration/), set in a TOML file or as an
environment variable (`MY_` and the key in capitals, such as `MY_DATABASE`). The first option of each list is the default.
Where a part keeps data, choose it before the first start: the data does not move by itself when the setting changes.

| Part | Setting | Options | Read next |
| --- | --- | --- | --- |
| Database | [`database`](https://mycroblog.dev/next/reference/configuration/#database) | `sqlite` (a file), `postgres` (PostgreSQL and compatible services) | [Database](https://mycroblog.dev/next/operations/database/), [backup](https://mycroblog.dev/next/operations/backup/) |
| Job queue | [`queue`](https://mycroblog.dev/next/reference/configuration/#queue) | `db` (a table in the database), `inline` (runs each job at once in the same process, for tests and development), `cloudtasks` (Google Cloud Tasks wakes the server for each job; needs PostgreSQL) | [The job queue](https://mycroblog.dev/next/operations/running/#the-job-queue-and-failed-dead-jobs) |
| Key-value store, for caches, rate limits, and the answers kept for a day to writes sent again | [`kv`](https://mycroblog.dev/next/reference/configuration/#kv) | `db` (a table in the database), `memory` (each process on its own), `valkey` (Valkey or another Redis-protocol server, shared by every process) | [`kv`](https://mycroblog.dev/next/reference/configuration/#kv) |
| Media storage | [`blob`](https://mycroblog.dev/next/reference/configuration/#blob) | `fs` (a directory), `s3` (Amazon S3, Cloudflare R2, MinIO, Garage, and other S3-compatible stores), `gcs` (Google Cloud Storage) | [Docker Compose files](https://mycroblog.dev/next/operations/running/#docker-compose-files), [backing up media](https://mycroblog.dev/next/operations/backup/#media-in-a-bucket) |
| Media processing | [`media_image`](https://mycroblog.dev/next/reference/configuration/#media_image), [`media_video`](https://mycroblog.dev/next/reference/configuration/#media_video), [`media_audio`](https://mycroblog.dev/next/reference/configuration/#media_audio), one for each kind of file | `native` (in the server), `plugin` (a plugin of yours converts the files of that kind) | [Converting media](https://mycroblog.dev/next/operations/plugins/#converting-media) |
| Real-time stream between processes | [`stream`](https://mycroblog.dev/next/reference/configuration/#stream) | `hub` (inside one process), `postgres` (`LISTEN` / `NOTIFY`), `valkey`, `sns` (Amazon SNS and SQS), `pubsub` (Google Cloud Pub/Sub) | [`stream`](https://mycroblog.dev/next/reference/configuration/#stream) |
| Key manager, which encrypts the accounts' private keys | [`key_manager`](https://mycroblog.dev/next/reference/configuration/#key_manager) | `local` (a key-encryption key you keep), `gcpkms` (Google Cloud KMS), `awskms` (AWS KMS), `plugin` | [Managing the key](https://mycroblog.dev/next/operations/key-management/) |
| Email | [`mail`](https://mycroblog.dev/next/reference/configuration/#mail) | `none`, `smtp`, `ses` (Amazon SES), `plugin` | [Sending email](https://mycroblog.dev/next/operations/email/) |
| CAPTCHA | [`captcha`](https://mycroblog.dev/next/reference/configuration/#captcha) | `none`, `turnstile` (Cloudflare Turnstile), `hcaptcha`, `recaptcha` (Google reCAPTCHA v2) | [CAPTCHA](https://mycroblog.dev/next/operations/captcha/) |
| Voice chat | [`voice`](https://mycroblog.dev/next/reference/configuration/#voice) | `off`, `p2p` (browser to browser, with TURN from `voice_turn`: `none`, `coturn`, or `cloudflare`), `livekit`, `sora` | [Voice chat](https://mycroblog.dev/next/operations/voice/) |
| Supporter payments | [`payments`](https://mycroblog.dev/next/reference/configuration/#payments) | `none`, `stripe`, `payjp` (PAY.JP) | [Supporters](https://mycroblog.dev/next/operations/supporters/), [Stripe](https://mycroblog.dev/next/operations/payments-stripe/), [PAY.JP](https://mycroblog.dev/next/operations/payments-payjp/) |
| Plugins | `plugin_<kind>_address` | none by default; one address for each kind: `translator`, `search`, `billing`, `mailer`, `media_image`, `media_video`, `media_audio`, `key_manager`, and the decision hooks `inbound.status`, `local.status`, `inbound.follow`, `signup.check`, `media.check`, `inbound.actor`, `local.profile` | [Plugins](https://mycroblog.dev/next/operations/plugins/), [decision hooks](https://mycroblog.dev/next/operations/decision-hooks/) |
| Translation of posts | `translator_<provider>_*`, [`plugin_translator_address`](https://mycroblog.dev/next/reference/configuration/#plugin_translator_address) | none (only the browser's own translation); `libretranslate`, `deepl`, `google` (Google Cloud Translation), `aws` (Amazon Translate), or a plugin of yours in front of the service you choose | [Translating posts](https://mycroblog.dev/next/operations/translation/), [a translator plugin](https://mycroblog.dev/next/operations/plugins/#translating-posts) |
| Full-text search of posts | [`search`](https://mycroblog.dev/next/reference/configuration/#search) | `none`; or `elasticsearch`, `opensearch`, or `meilisearch` ([search engines](https://mycroblog.dev/next/operations/search-engines/)); or `plugin`, a plugin of yours in front of another engine | [Full-text search](https://mycroblog.dev/next/operations/search/), [a search plugin](https://mycroblog.dev/next/operations/plugins/#searching-posts) |
| CDN for media | [`media_origin`](https://mycroblog.dev/next/reference/configuration/#media_origin) | unset (the server serves media from its own origin), or the origin of your CDN | [Serving media through a CDN](https://mycroblog.dev/next/operations/cdn/) |
| Cloud costs | [`billing_sources`](https://mycroblog.dev/next/reference/configuration/#billing_sources) | none, or any of `aws` (AWS Cost Explorer), `gcp` (Google Cloud's billing export or budget), `cloudflare` (Cloudflare's usage charges), `plugin` | [Importing cloud costs](https://mycroblog.dev/next/operations/billing-import/) |
| Metrics | [`metrics_exporter`](https://mycroblog.dev/next/reference/configuration/#metrics_exporter) | `none`, `prometheus`, `otlp` (OpenTelemetry), `emf` (CloudWatch) | [Metrics](https://mycroblog.dev/next/operations/running/#metrics), [alerts](https://mycroblog.dev/next/operations/alerts/) |
| Traces | [`traces_exporter`](https://mycroblog.dev/next/reference/configuration/#traces_exporter) | `none`, `otlp` | [Traces](https://mycroblog.dev/next/operations/running/#traces) |
| Error reports | [`errors_exporter`](https://mycroblog.dev/next/reference/configuration/#errors_exporter) | `none` (the log line and the error groups only), `sentry` (Sentry, GlitchTip, Datadog), `otlp` | [Error reporting](https://mycroblog.dev/next/operations/error-reporting/) |

The released binary and container image hold every adapter of the table.

`my config check` reports every problem of the configuration at once, before you start the server
([production checklist](https://mycroblog.dev/next/operations/production-checklist/)).

## From a small server to a large one

![Three steps from a small server to a large one: one process on SQLite and the file system, then PostgreSQL and object storage behind a CDN, then separate web and worker processes with a stream broker. The list below describes each step.](https://github.com/mycroblog/my/blob/main/docs/operations/images/growth.svg?raw=true)

1. **Start small.** One process with `role = all`, on SQLite, with media on the file system and the defaults for the
   rest: the queue and the key-value store in the database, and the real-time stream inside the process. The Docker
   Compose files in `deploy/docker` start this setup on one machine ([deploying with Docker Compose](https://mycroblog.dev/next/operations/deploy-docker-compose/)).
   It suits a personal or small server. `stream = hub` reaches only the process it runs in, so this setup stays at one
   process.
2. **Move the data out.** PostgreSQL for the database, an S3-compatible bucket or Google Cloud Storage for the media,
   and a CDN in front of the media. The server is still one process, but it no longer depends on one disk, and the
   processes of the next step can share the data. Decide on this before the first start if you can: moving the data
   from SQLite and the file system later is a job of its own ([what to back up](https://mycroblog.dev/next/operations/backup/#what-to-back-up) lists
   what has to move).
3. **Scale out.** Run processes with `role = web` for the requests and `role = worker` for the jobs, as many as each
   needs, and split the workers by job type ([splitting workers](https://mycroblog.dev/next/operations/running/#splitting-workers-by-job-type)).
   With more than one process, the real-time stream needs a broker that reaches every process (`postgres`, `valkey`,
   `sns`, or `pubsub`), and the key-value store should be shared (`db` or `valkey`), since `memory` keeps the rate
   limits and the answers to writes sent again of each process apart. The job queue stays in
   PostgreSQL, or moves to Google Cloud Tasks ([Cloud Tasks](https://mycroblog.dev/next/operations/running/#cloud-tasks)).

The [Cloud Run deployment](https://mycroblog.dev/next/operations/deploy-cloud-run/) is an example of a setup between the second and the third step:
`role = all` on Cloud Run, PostgreSQL on Neon, media in Cloud Storage, Cloud Tasks for the jobs, and Pub/Sub for the
stream, with no instance running while nobody uses the server.

## Where to go next

Read the operator guides in this order:

1. **Get started:** [run the server](https://mycroblog.dev/next/operations/running/) or [deploy it on Cloud Run](https://mycroblog.dev/next/operations/deploy-cloud-run/), then go through the
   [production checklist](https://mycroblog.dev/next/operations/production-checklist/) before you open it to users.
2. **Configure the instance:** its [settings](https://mycroblog.dev/next/operations/instance-settings/), [registration](https://mycroblog.dev/next/operations/registration/),
   [external login providers](https://mycroblog.dev/next/operations/login-providers/), [email](https://mycroblog.dev/next/operations/email/), [CAPTCHA](https://mycroblog.dev/next/operations/captcha/), [the CDN](https://mycroblog.dev/next/operations/cdn/),
   [link previews](https://mycroblog.dev/next/operations/link-previews/), and [search engines and AI crawlers](https://mycroblog.dev/next/operations/crawlers/).
3. **Turn on features:** [moderation](https://mycroblog.dev/next/operations/moderation/), [relays](https://mycroblog.dev/next/operations/relays/), [webhooks](https://mycroblog.dev/next/operations/webhooks/),
   [plugins](https://mycroblog.dev/next/operations/plugins/) and [decision hooks](https://mycroblog.dev/next/operations/decision-hooks/), [voice chat](https://mycroblog.dev/next/operations/voice/), [full-text search](https://mycroblog.dev/next/operations/search/),
   [supporters](https://mycroblog.dev/next/operations/supporters/) and their payments, and [importing cloud costs](https://mycroblog.dev/next/operations/billing-import/).
4. **Run it day to day:** [monitoring](https://mycroblog.dev/next/operations/monitoring/), [alerts](https://mycroblog.dev/next/operations/alerts/), [error reporting](https://mycroblog.dev/next/operations/error-reporting/),
   [maintenance](https://mycroblog.dev/next/operations/maintenance/), [backup](https://mycroblog.dev/next/operations/backup/), [the database](https://mycroblog.dev/next/operations/database/),
   [the key-encryption key](https://mycroblog.dev/next/operations/key-management/), and [administrators and roles](https://mycroblog.dev/next/operations/admin-cli/).
5. **Legal and closing:** the [privacy policy](https://mycroblog.dev/next/operations/privacy-policy-template/) and
   [record of processing](https://mycroblog.dev/next/operations/processing-record-template/) templates, and [closing the service](https://mycroblog.dev/next/operations/service-closure/).

The [configuration reference](https://mycroblog.dev/next/reference/configuration/) lists every setting, and the [jobs reference](https://mycroblog.dev/next/reference/jobs/)
every job the workers run.
