content/posts/matrix-delegation/index.md

   1 ---
   2 title: "Matrix delegation and how it may bite you"
   3 date: 2024-01-14T11:19:48+01:00
   4 summary: One of the ways small details can cause big issues.
   5 ---
   6 For those who don't know, delegation in Matrix is used in server-to-server
   7 communication to figure out which server serves a given domain. As an example,
   8 if my own Matrix homeserver was running on `matrix.dujemihanovic.xyz` instead of
   9 `dujemihanovic.xyz`, I could delegate the latter to the former to save anyone
  10 wanting to contact me from having to type out the `matrix.`.
  11
  12 Besides domain name, delegation can also be used to specify which port to use
  13 for server-to-server communication. The default is `8448`, and if it's blocked
  14 you can use delegation to use `443` for server-to-server as client-to-server
  15 does by default. However, if you can, **I'd strongly suggest using `8448`!** I had
  16 been delegating S2S to `443` almost the whole time I have had this server for no
  17 reason and it seems that this caused an extremely weird issue with a certain
  18 room:
  19
  20 ## What happened?
  21
  22 Message fetching kept breaking **constantly**. What I mean by that is that when
  23 I joined the room everything would work fine the first few messages, but at some
  24 point I would start getting notifications without any new message being present
  25 in that room. I have noticed that logging out and back in would get the missing
  26 messages in my client, but then the forementioned cycle would repeat again no
  27 matter how many times I logged out and back in *(this also happened on other
  28 clients besides Element desktop)*. To confirm my homeserver was the issue, I
  29 joined the room with my old matrix.org account and sure enough that worked just
  30 fine.
  31
  32 I tried the usual things such as restarting Dendrite and the whole VPS, but to
  33 no avail. I was pretty insistent that the issue was not with my homeserver but
  34 the main server hosting the room *(which, unsurprisingly, turned out to be
  35 false)* and so I gave up on that. The eyeopening moment was me reading the
  36 [conduit
  37 documentation](https://gitlab.com/famedly/conduit/-/blob/next/DEPLOY.md) *(I had
  38 considered migrating to it)*, specifically this:
  39
  40 > If Conduit runs behind Cloudflare reverse proxy, which doesn't support port
  41 > 8448 on free plans,
  42
  43 This implies that routing server-to-server traffic to `443` should only be done
  44 if it's **absolutely impossible** to use `8448` for this, and the [Synapse
  45 documentation](https://matrix-org.github.io/synapse/latest/delegate.html#when-do-i-need-delegation)
  46 said something similar:
  47
  48 > **However**, if your homeserver's APIs aren't accessible on port 8448 and on
  49 > the domain server_name points to, you will need to let other servers know how
  50 > to find it using delegation.
  51
  52 ## Fixing the issue
  53
  54 Encouraged by this, I fixed up my server:
  55
  56 * allow port `8448` in `ufw`
  57 * add something like this to `Caddyfile`:
  58 ```
  59 dujemihanovic.xyz:8448 {
  60         reverse_proxy /_matrix/* localhost:8008
  61 }
  62 ```
  63 * change `/.well-known/matrix/server` to point to `dujemihanovic.xyz:8448` *(in
  64   theory, I could have gotten rid of that `return` directive altogether as
  65   `8448` is default anyway, but I still chose to specify it just to be safe)*
  66 * reload `caddy` and restart `dendrite` *(the latter is, again, just to be
  67   safe)*
  68
  69 Once all this was done, the room finally started acting normally.
  70
  71 ## Small sidenote
  72
  73 I must note that delegating federation to `443` **should not cause breakage like
  74 this**. Despite this, it still did so in my case and for that reason I wrote
  75 about it anyway. It's very unlikely that you will be affected by this issue, but
  76 I still believe it should be pointed out in the event that it does.