matrix-riot.md 31.2 KB
Newer Older
pedro-nonfree's avatar
pedro-nonfree committed
1 2 3 4 5 6 7
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**

- [Presentation](#presentation)
- [Install on Debian 9 stretch 2017-10-06](#install-on-debian-9-stretch-2017-10-06)
  - [First steps](#first-steps)
pedrolab's avatar
pedrolab committed
8
  - [TLS renewal](#tls-renewal)
pedro-nonfree's avatar
pedro-nonfree committed
9 10 11 12 13
  - [DNS configuration](#dns-configuration)
  - [Accessibility to the server](#accessibility-to-the-server)
    - [reverse proxy server with nginx](#reverse-proxy-server-with-nginx)
    - [web static client](#web-static-client)
      - [script to upgrade static riot](#script-to-upgrade-static-riot)
14 15
  - [Data](#data)
  - [Test federation](#test-federation)
pedrolab's avatar
pedrolab committed
16
- [other installation guides](#other-installation-guides)
pedro-nonfree's avatar
pedro-nonfree committed
17 18 19
- [known problems](#known-problems)
  - [notifications](#notifications)
- [todo / extra](#todo--extra)
pedrolab's avatar
pedrolab committed
20 21 22 23 24 25
  - [irc bridge to freenode](#irc-bridge-to-freenode)
    - [previous steps](#previous-steps)
    - [do the bridge!](#do-the-bridge)
  - [spam](#spam)
    - [configure IRC room to avoid spam](#configure-irc-room-to-avoid-spam)
    - [utility to kill spam](#utility-to-kill-spam)
pedrolab's avatar
pedrolab committed
26
  - [save disk space: purge history](#save-disk-space-purge-history)
pedro-nonfree's avatar
pedro-nonfree committed
27
  - [HTML formatted messages](#html-formatted-messages)
pedro-nonfree's avatar
pedro-nonfree committed
28
  - [telegram bridge](#telegram-bridge)
pedro-nonfree's avatar
pedro-nonfree committed
29
  - ["telegram channel" (lista de difusión) in riot/matrix ?](#telegram-channel-lista-de-difusi%C3%B3n-in-riotmatrix-)
pedro-nonfree's avatar
pedro-nonfree committed
30
  - [captcha](#captcha)
pedro-nonfree's avatar
pedro-nonfree committed
31
  - [specification for script that migrates user from one server to another](#specification-for-script-that-migrates-user-from-one-server-to-another)
pedro-nonfree's avatar
pedro-nonfree committed
32
    - [script to blank account in server A](#script-to-blank-account-in-server-a)
pedro-nonfree's avatar
pedro-nonfree committed
33
  - [use our own integration server](#use-our-own-integration-server)
pedro-nonfree's avatar
pedro-nonfree committed
34
    - [particular interest: monitoring with prometheus and grafana](#particular-interest-monitoring-with-prometheus-and-grafana)
pedrolab's avatar
pedrolab committed
35 36 37
  - [bridges](#bridges)
  - [bots](#bots)
  - [do your own piwik analitics on matrix](#do-your-own-piwik-analitics-on-matrix)
38
- [dark things about matrix and riot](#dark-things-about-matrix-and-riot)
pedrolab's avatar
pedrolab committed
39
  - [extra notes 2019-2-22](#extra-notes-2019-2-22)
pedro-nonfree's avatar
pedro-nonfree committed
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

# Presentation

- [Matrix](https://matrix.org/) server: now [synapse](https://github.com/matrix-org/synapse), comming soon [dendrite](https://github.com/matrix-org/dendrite)
- [How it works?](https://matrix.org/#about)
- [Status of the project](https://matrix.org/blog/2017/07/07/a-call-to-arms-supporting-matrix/). [Extra](https://matrix.org/blog/2017/07/19/status-update/)
- Based on #channels, and @people attached to them. Its access: Guest (Read-only, Read + Write), Login (Local OR LDAP), Encrypted channels through Double [Ratchet algorithm](https://en.wikipedia.org/wiki/Double_Ratchet_Algorithm) - device based encryption (encrypted-devices.png). History control (no, public, semipublic, private)
    - `@user:<server>, #channel:<server>`
    - public channels
        - Who can access this room?
        - Anyone who knows the room's link, including guests
        - List this room <domain>'s room directory?

- Clients:
    - [There are a lot](https://matrix.org/docs/projects/try-matrix-now.html)
pedro-nonfree's avatar
pedro-nonfree committed
57
    - Recommended: riot.im/app HTML5 App, Desktop (Electron), AppStore, Google Play, F-Droid
pedro-nonfree's avatar
pedro-nonfree committed
58 59 60
        - Custom server config (login-riot-guifi.png)
        - Room directory (public channels)
        - Start chat
pedro-nonfree's avatar
pedro-nonfree committed
61 62
- Bridge with other networks. [Thought](https://xkcd.com/1810/), [response from Matrix](https://twitter.com/matrixdotorg/status/841424770025545730)
- Riot Integrations: IRC networks (TODO: upload integrations-irc.png), jitsi (TODO: upload jitsi.png), [more](https://medium.com/@RiotChat/riot-im-web-0-12-7c4ea84b180a)
pedro-nonfree's avatar
pedro-nonfree committed
63 64 65 66
- Locations: Spanish 79%, Catalan 0%

# Install on Debian 9 stretch 2017-10-06

pedro-nonfree's avatar
pedro-nonfree committed
67 68
This guide helps you to install a matrix server using authentication of a particular LDAP (guifi.net) with a postgresql database. Hope it helps you to be inspired on your particular needs.

pedro-nonfree's avatar
pedro-nonfree committed
69 70 71 72 73
matrix-riot homeservers up & running that used this howto:

- https://riot.guifi.net
- https://riot.musaik.net

pedro-nonfree's avatar
pedro-nonfree committed
74 75
## First steps

pedrolab's avatar
pedrolab committed
76 77 78 79
If you want to automate this steps (terraform, ansible, docker) you would like to use advice that to debian with the following variable

    export DEBIAN_FRONTEND="noninteractive"

pedro-nonfree's avatar
pedro-nonfree committed
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96
Run all following commands as root user

Add repository

```
cat <<EOF > /etc/apt/sources.list.d/synapse.list
deb http://matrix.org/packages/debian/ stretch main
deb-src http://matrix.org/packages/debian/ stretch main
EOF
```

Add repo key

    curl -s https://matrix.org/packages/debian/repo-key.asc | apt-key add -

Install synapse matrix server

pedrolab's avatar
pedrolab committed
97
    apt-get install matrix-synapse-py3
pedro-nonfree's avatar
pedro-nonfree committed
98 99 100 101 102 103 104 105 106 107

The two asked options are stored here:

- /etc/matrix-synapse/conf.d/report_stats.yaml
- /etc/matrix-synapse/conf.d/server_name.yaml

Config at `/etc/matrix-synapse/homeserver.yaml` is overridden by config in `/etc/matrix-synapse/conf.d`. Let's add all stuff at `/etc/matrix-synapse/conf.d/guifi.yaml`:

```
cat <<EOF > /etc/matrix-synapse/conf.d/guifi.yaml
108 109 110 111 112

# from synapse version 1 the TLS must be certified (letsencrypt is enough)
tls_certificate_path: "/etc/letsencrypt/live/matrix.example.com/fullchain.pem"
tls_private_key_path: "/etc/letsencrypt/live/matrix.example.com/privkey.pem"

pedro-nonfree's avatar
pedro-nonfree committed
113 114 115 116 117 118 119 120 121 122 123 124
# overridden: default is sqlite
database:
  name: psycopg2
  args:
    user: synapse_user
    password: synapse_user
    database: synapse
    host: localhost
    cp_min: 5
    cp_max: 10

# LDAP from guifi
pedro-nonfree's avatar
pedro-nonfree committed
125 126 127 128 129 130 131 132 133 134 135 136

password_providers:
 - module: "ldap_auth_provider.LdapAuthProvider"
   config:
     enabled: true
     uri: "ldaps://ldap.guifi.net"
     start_tls: true
     base: "o=webusers,dc=guifi,dc=net"
     attributes:
        uid: "uid"
        mail: "mail"
        name: "uid"
pedro-nonfree's avatar
pedro-nonfree committed
137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154

# overridden: default is false
allow_guest_access: True

# reverse proxy -> https://github.com/matrix-org/synapse#using-a-reverse-proxy-with-synapse
# just adds on port 8008:
#  + bind_addresses: ['127.0.0.1']
#  + x_forwarded: true

listeners:
  # Main HTTPS listener
  # For when matrix traffic is sent directly to synapse.
  -
    # The port to listen for HTTPS requests on.
    port: 8448

    # Local interface to listen on.
    # The empty string will cause synapse to listen on all interfaces.
pedro-nonfree's avatar
pedro-nonfree committed
155 156 157
    #bind_address: ''
    # includes IPv6 -> src https://github.com/matrix-org/synapse/issues/1886
    bind_address: '::'
pedro-nonfree's avatar
pedro-nonfree committed
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198

    # This is a 'http' listener, allows us to specify 'resources'.
    type: http

    tls: true

    # Use the X-Forwarded-For (XFF) header as the client IP and not the
    # actual client IP.
    x_forwarded: false

    # List of HTTP resources to serve on this listener.
    resources:
      -
        # List of resources to host on this listener.
        names:
          - client     # The client-server APIs, both v1 and v2
          - webclient  # The bundled webclient.

        # Should synapse compress HTTP responses to clients that support it?
        # This should be disabled if running synapse behind a load balancer
        # that can do automatic compression.
        compress: true

      - names: [federation]  # Federation APIs
        compress: false

  # Unsecure HTTP listener,
  # For when matrix traffic passes through loadbalancer that unwraps TLS.
  - port: 8008
    tls: false
    bind_address: '127.0.0.1'
    type: http

    x_forwarded: true

    resources:
      - names: [client, webclient]
        compress: true
      - names: [federation]
        compress: false

pedro-nonfree's avatar
pedro-nonfree committed
199 200 201
# enable communities feature
enable_group_creation: True

pedro-nonfree's avatar
pedro-nonfree committed
202 203 204
EOF
```

pedrolab's avatar
pedrolab committed
205 206 207 208 209 210 211 212 213 214 215 216
`/etc/matrix-synapse/conf.d/extra.yaml`:

```
# The public-facing base URL for the client API (not including _matrix/...)
#  $ curl -k https://matrix.example.com:8448/_matrix/client/r0/
# {
#     "errcode": "M_UNRECOGNIZED",
#     "error": "Unrecognized request"
# }
public_baseurl: https://matrix.example.com:8448/
```

pedro-nonfree's avatar
pedro-nonfree committed
217 218
[Set up requirements for guifi LDAP](https://github.com/matrix-org/matrix-synapse-ldap3#installation)

pedro-nonfree's avatar
pedro-nonfree committed
219
    apt-get install python-matrix-synapse-ldap3
pedro-nonfree's avatar
pedro-nonfree committed
220 221 222

[Set up requirements](https://wiki.debian.org/PostgreSql#Installation)

pedro-nonfree's avatar
pedro-nonfree committed
223
    apt-get install postgresql
pedro-nonfree's avatar
pedro-nonfree committed
224 225 226 227 228

create user

    su -s /bin/bash postgres -c "createuser synapse_user"

pedro-nonfree's avatar
pedro-nonfree committed
229
[enter postresql CLI](https://wiki.debian.org/PostgreSql#User_access):
pedro-nonfree's avatar
pedro-nonfree committed
230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246

    su -s /bin/bash postgres -c psql

[put password to user](https://stackoverflow.com/questions/12720967/how-to-change-postgresql-user-password)

    ALTER USER "synapse_user" WITH PASSWORD 'synapse_user';

and [set up database](https://github.com/matrix-org/synapse/blob/master/docs/postgres.rst#set-up-database)

    CREATE DATABASE synapse
     ENCODING 'UTF8'
     LC_COLLATE='C'
     LC_CTYPE='C'
     template=template0
     OWNER synapse_user;


pedro-nonfree's avatar
pedro-nonfree committed
247
[Set up client in Debian/Ubuntu](https://github.com/matrix-org/synapse/blob/master/docs/postgres.rst#set-up-client-in-debianubuntu)
pedro-nonfree's avatar
pedro-nonfree committed
248

249
    apt-get install libpq-dev python-pip python-psycopg2
pedro-nonfree's avatar
pedro-nonfree committed
250 251 252 253 254 255 256 257 258 259 260

note: [synapse currently assumes python 2.7 by default](https://github.com/matrix-org/synapse#archlinux)

Start or restart matrix service

    service matrix-synapse restart

To check if is running:

    service matrix-synapse status

261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283
## TLS renewal

In guifi.yaml we already added the tls files, extra work is required to access letsencrypt files and to do renewall succesful.

Considering you created a user for the letsencrypt renewal process with letsencrypt user. Add matrix-synapse in the group of letsencrypt

    gpassswd -a matrix-synapse letsencrypt

And change permissions of /etc/letsencrypt directory to be able to access through matrix [reference](https://community.letsencrypt.org/t/how-to-use-certs-in-non-root-services/2690/4)

    chgrp -R letsencrypt /etc/letsencrypt
    chmod -R g=rX /etc/letsencrypt

One time each month a renewal is performed:

```
cat <<EOF > /etc/cron.monthly/matrix-synapse
#!/bin/bash
service matrix-synapse restart
EOF
chmod +x /etc/cron.monthly/matrix-synapse
```

pedro-nonfree's avatar
pedro-nonfree committed
284 285 286 287
## DNS configuration

This DNS configuration is required to see federation working in your matrix server

pedrolab's avatar
pedrolab committed
288
[More info setting up federation](https://github.com/matrix-org/synapse#setting-up-federation)
pedro-nonfree's avatar
pedro-nonfree committed
289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323

    matrix.example.com IN A <IP>
    riot.example.com IN A <IP>
    _matrix._tcp.example.com. 3600 IN SRV 10 0 8448 matrix.example.com.

## Accessibility to the server

Requirements:

    apt-get install certbot nginx-full

### reverse proxy server with nginx

```
matrix_domain="matrix.example.com"
cat <<EOF > /etc/nginx/sites-available/${matrix_domain}
server {
    listen 80;
    listen [::]:80;
    server_name ${matrix_domain};

    location /.well-known {
            default_type "text/plain";
            allow all;
            root /var/www/html;
    }

    return 301 https://\$host\$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name ${matrix_domain};

pedro-nonfree's avatar
pedro-nonfree committed
324 325 326 327
    # good security tips ON - check yours: https://www.ssllabs.com/ssltest/analyze.html?d=matrix.example.com&latest
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    # Ciphers based on the Mozilla SSL Configuration Generator
    ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
pedro-nonfree's avatar
pedro-nonfree committed
328
    # run: `openssl dhparam -out /etc/ssl/dhparams2048.pem 2048` src https://weakdh.org/sysadmin.html extra src https://gist.github.com/plentz/6737338
pedro-nonfree's avatar
pedro-nonfree committed
329 330 331 332 333 334
    ssl_dhparam /etc/ssl/dhparams2048.pem;

    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

    ssl_stapling on;
    ssl_stapling_verify on;
pedrolab's avatar
pedrolab committed
335

pedro-nonfree's avatar
pedro-nonfree committed
336
    ssl_prefer_server_ciphers on;
pedro-nonfree's avatar
pedro-nonfree committed
337 338 339 340 341 342 343 344

    # enable session resumption to improve https performance
    # http://vincent.bernat.im/en/blog/2011-ssl-session-reuse-rfc5077.html
    # found here: https://gist.github.com/plentz/6737338
    ssl_session_cache shared:SSL:50m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;

pedro-nonfree's avatar
pedro-nonfree committed
345 346
    # good security tips OFF

pedro-nonfree's avatar
pedro-nonfree committed
347 348 349
    ssl_certificate /etc/letsencrypt/live/${matrix_domain}/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/${matrix_domain}/privkey.pem;

pedro-nonfree's avatar
pedro-nonfree committed
350
    # static front page to anounce how works the service
pedro-nonfree's avatar
pedro-nonfree committed
351
    # example: https://github.com/guifi-exo/public/tree/master/web/matrix.guifi.net
pedro-nonfree's avatar
pedro-nonfree committed
352
    location / {
pedro-nonfree's avatar
pedro-nonfree committed
353
        root /var/www/html;
pedro-nonfree's avatar
pedro-nonfree committed
354
        try_files /matrix.html /matrix.html;
pedro-nonfree's avatar
pedro-nonfree committed
355 356
    }

pedro-nonfree's avatar
pedro-nonfree committed
357
    location /_matrix {
358
        proxy_pass http://127.0.0.1:8008;
pedro-nonfree's avatar
pedro-nonfree committed
359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396
        proxy_set_header X-Forwarded-For $remote_addr;
    }
}
EOF

ln -s /etc/nginx/sites-available/${matrix_domain}.conf /etc/nginx/sites-enabled/${matrix_domain}.conf
certbot certonly -n --keep --agree-tos --email ${matrix_email} --webroot -w /var/www/html/ -d ${matrix_domain}
service nginx reload
```

### web static client

```
riot_domain="riot.example.com"
riot_email="info@example.com"
cat <<EOF > /etc/nginx/sites-available/${riot_domain}.conf
server {
    listen 80;
    listen [::]:80;
    server_name ${riot_domain};

    location /.well-known {
            default_type "text/plain";
            allow all;
            root /var/www/html;
    }

    return 301 https://\$host\$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name ${riot_domain};

    ssl_certificate /etc/letsencrypt/live/${riot_domain}/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/${riot_domain}/privkey.pem;

397
    # static front page to anounce how works the service (inside riot)
pedro-nonfree's avatar
pedro-nonfree committed
398
    # example: https://github.com/guifi-exo/public/tree/master/web/riot.guifi.net
pedrolab's avatar
pedrolab committed
399 400
    # WARNING: the welcome page with new riot 1.0 looks is no longer possible -> https://github.com/vector-im/riot-web/issues/8676
    location /welcome_custom {
401 402 403 404 405 406 407
        alias /var/www/html/riot.guifi.net;
        index /matrix.html;
    }
    location /fp-img {
        alias /var/www/html/riot.guifi.net/fp-img;
    }

pedro-nonfree's avatar
pedro-nonfree committed
408
    root /var/www/html/riot-web;
pedro-nonfree's avatar
pedro-nonfree committed
409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426
}
EOF

ln -s /etc/nginx/sites-available/${riot_domain}.conf /etc/nginx/sites-enabled/${riot_domain}.conf

certbot certonly -n --keep --agree-tos --email ${riot_email} --webroot -w /var/www/html/ -d ${riot_domain}

service nginx reload
```

#### script to upgrade static riot

Requirements:

    apt-get install jq

edit file `vi /usr/local/bin/update-riot.sh`, add following content:

427
https://github.com/grigruss/Riot-web-server-update/blob/master/riot-update.sh my commit reference is [f3a3990](https://github.com/grigruss/Riot-web-server-update/commit/f3a3990c9813a9e01ffa4d1f845fd3c22aa88bbf)
pedrolab's avatar
pedrolab committed
428 429 430 431 432 433

changing this

```diff
-WWW="/www/html/"
+WWW="/var/www/html/riot-web"
pedro-nonfree's avatar
pedro-nonfree committed
434 435
```

pedrolab's avatar
pedrolab committed
436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472
add a config file for your site like `config.riot.example.com.json` inside riot-web directory `/var/www/html/riot-web`

```
{
  "default_hs_url": "https://matrix.example.com",
  "default_is_url": "https://vector.im",
  "disable_custom_urls": false,
  "disable_guests": false,
  "disable_login_language_selector": false,
  "disable_3pid_login": false,
  "brand": "Riot",
  "integrations_ui_url": "https://scalar.vector.im/",
  "integrations_rest_url": "https://scalar.vector.im/api",
  "integrations_jitsi_widget_url": "https://scalar.vector.im/api/widgets/jitsi.html",
  "bug_report_endpoint_url": "https://riot.im/bugreports/submit",
  "features": {
    "feature_groups": "labs",
    "feature_pinning": "labs"
  },
  "default_federate": true,
  "default_theme": "light",
  "roomDirectory": {
    "servers": [
      "matrix.org"
    ]
  },
  "welcomeUserId": "@riot-bot:matrix.org",
  "enable_presence_by_hs_url": {
    "https://matrix.org": false
  }
}
```

if you want a custom welcome page add to the previous file:

```diff
+  "welcomePageUrl": "https://riot.example.com/welcome/matrix.html",
pedro-nonfree's avatar
pedro-nonfree committed
473 474 475 476 477 478 479 480 481 482
```

edit file `vi /etc/cron.d/updateriot`, add following content:

```
SHELL=/bin/bash

40 6 * * * root /usr/local/bin/update-riot.sh
```

483

pedro-nonfree's avatar
pedro-nonfree committed
484 485 486 487
## Data

Data grows here:

pedro-nonfree's avatar
pedro-nonfree committed
488 489 490
- `/var/lib/postgresql`
- `/var/lib/matrix-synapse/media`
- `/var/log/matrix-synapse/` - warning, uses up to 1 GB, change behavior in `/etc/matrix-synapse/log.yaml`
pedro-nonfree's avatar
pedro-nonfree committed
491 492 493

I symlink this directories to specific volume

pedro-nonfree's avatar
pedro-nonfree committed
494 495
learn more about how data grows https://matrix.org/docs/projects/other/hdd-space-calc-for-synapse.html

496 497 498 499
## Test federation

To test federation you can use this service: https://matrix.org/federationtester/api/report?server_name=matrix.example.com

500 501 502
with more visuals: https://matrix.org/federationtester/ or https://arewereadyyet.com/

source: https://github.com/matrix-org/matrix-federation-tester
503

pedrolab's avatar
pedrolab committed
504 505 506 507 508 509 510 511
# other installation guides

https://matrix.org/docs/guides/installing-synapse

https://matrix.org/docs/guides/#installing-synapse

src https://matrix.org/blog/2019/01/17/bens-favourite-projects-2018/

pedro-nonfree's avatar
pedro-nonfree committed
512 513 514 515 516 517 518 519
# known problems

## notifications

some people do not receive notifications in its smartphone

# todo / extra

pedrolab's avatar
pedrolab committed
520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582
## irc bridge to freenode

This describes how to use official matrix bridge. The matrix discussion room is #irc:matrix.org

note: I hope for other IRC networks should be very similar

### previous steps

To do an IRC bridge you have to be operator (op)

Check the general state of the chat room with the following command

    /msg ChanServ info #channel

If you are alone in the room and seems that is not configured you can register it

    /msg ChanServ register #channel

If you loose the operator of your channel (you logged out), ask it to ChanServ

    /msg ChanServ op #channel yourUser

To register a room you require to register your current nickname

    /msg NickServ REGISTER password youremail@example.com

Change nickname

    /nick newUsername

To identify your existing user

    /msg NickServ IDENTIFY yourUser password

note: if the room is not configured but the room is full of people, nobody can claim to be operator. A solution is try to convince everyone to quit the room (difficult), or get help from #frenode channel/freenode staff (not tested)

/msg ChanServ info #coopdevs

References:

- Nickname Registration -> src https://freenode.net/kb/answer/registration

### do the bridge!

From Riot client in a matrix channel select *Manage Integrations*, select IRC, select the network Freenode, select operator (you need your IRC user logged and being operator of that channel), link

After some seconds you will receive a message like

```
<MatrixBridge> @youruser:matrix.example.com has requested to bridge 'My room in Matrix' (https://matrix.to/#/#myroominmatrix:matrix.example.com) with #my-room-in-IRC on this IRC network.
               Respond with 'yes' or 'y' to allow, or simply ignore this message to disallow. You have 300 seconds from when this message was sent.
<yourUser> y
<MatrixBridge> Thanks for your response, bridge request authorised.
```

## spam

### configure IRC room to avoid spam

With the following configuration everyone can read the channel but only registered IRC users can write to it. With the exception of matrix users (who are not registered accounts in IRC!):

    /mode #channel +qe $~a *!*@gateway/shell/matrix.org/*

pedrolab's avatar
pedrolab committed
583
if you are fed up of lots of login/logout traffic from spammers, you can ban unidentified users from IRC with exception of matrix bridge:
584 585 586

    /mode #channel +be $~a *!*@gateway/shell/matrix.org/*

pedrolab's avatar
pedrolab committed
587 588 589 590 591 592 593 594
check all/configuration of a channel with:

    /mode #channel bqeI

References

- info about channel modes -> src https://freenode.net/kb/answer/channelmodes
- bans and exceptions to channel modes -> src https://freenode.net/kb/answer/extbans
pedrolab's avatar
pedrolab committed
595 596
- extra info about appservice-irc from matrix.org -> https://github.com/matrix-org/matrix-appservice-irc/wiki/End-user-FAQ

pedrolab's avatar
pedrolab committed
597
### utility to kill spam
pedrolab's avatar
pedrolab committed
598 599 600 601 602

https://gitlab.com/guifi-exo/matrix-killspam

## save disk space: purge history

pedrolab's avatar
pedrolab committed
603 604
the most relevant script (untested): https://git.fosscommunity.in/disroot/matrix_scripts/synapse_scripts/blob/master/purge_matrix_rooms.sh

pedrolab's avatar
pedrolab committed
605 606 607 608 609 610 611 612 613 614 615 616
Interesting discussions:

- https://github.com/matrix-org/synapse/issues/1730
- https://github.com/matrix-org/synapse/pull/911
- https://github.com/matrix-org/synapse/issues/1621
- https://github.com/matrix-org/synapse/pull/2540/files

Relevant documentation:

- https://github.com/matrix-org/synapse/tree/master/docs/admin_api
- https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.rst

pedro-nonfree's avatar
pedro-nonfree committed
617
## HTML formatted messages
pedro-nonfree's avatar
pedro-nonfree committed
618 619 620 621 622 623 624 625 626 627 628 629

You can send HTML formatted messages (for example for bot/alert notification) with `curl`, at the moment this is [officially not documented](https://github.com/matrix-org/matrix-doc/issues/917)

```
token="check your token access in riot general settings"
room_id="vfFxDRtZSSdspfTSEr" #test:matrix.org
room_server="matrix.org"
homeserver="matrix.org"

curl -XPOST -k -d '{"msgtype":"m.text", "body": "", "format": "org.matrix.custom.html", "formatted_body":"<b>test</b> test <font color =\"red\">red test</font> https://docs.google.com/document/d/1QPncBmMkKOo6\_B2jyBuy5FFSZJrRsq7WU5wgRSzOMho/edit#heading=h.arjuwv7itr4h <table style=\"width:100%\"><tr><th>Firstname</th><th>Lastname</th><th>Age</th></tr><tr><td>Jill</td><td>Smith</td><td>50</td></tr><tr><td>Eve</td><td>Jackson</td><td>94</td></tr></table> https://www.w3schools.com/html/html\_tables.asp"}' "https://matrix.guifi.net:8448/\_matrix/client/r0/rooms/%21$room\_id:$room\_server/send/m.room.message?access\_token=$token"
```

pedro-nonfree's avatar
pedro-nonfree committed
630 631 632 633
## telegram bridge

https://github.com/SijmenSchoon/telematrix

pedro-nonfree's avatar
pedro-nonfree committed
634 635 636 637
https://t2bot.io/telegram/

https://medium.com/@mujeebcpy/bridging-of-riot-and-telegram-cccb16a955f1

638 639 640
Comment about telegram bridge: A matrix els usuaris de telegram es veuen com un usuari de matrix, però els missatges escrits a matrix a telegram els diu un bot que diu "tal persona a dit"

## "telegram channel" (lista de difusión) in riot/matrix ?
pedro-nonfree's avatar
pedro-nonfree committed
641

pedro-nonfree's avatar
pedro-nonfree committed
642 643
Only some people can send messages, and lots of listeners

pedro-nonfree's avatar
pedro-nonfree committed
644 645
https://telegram.org/faq_channels#q-what-39s-a-channel

pedro-nonfree's avatar
pedro-nonfree committed
646 647
Solution? You can restrict users from sending messages to the room and only accept Moderator, Admin, etc. The rule has the name "To send messages, you must be a" 

pedro-nonfree's avatar
pedro-nonfree committed
648 649 650
## captcha

https://github.com/matrix-org/synapse/blob/master/docs/CAPTCHA_SETUP.rst
651

pedro-nonfree's avatar
pedro-nonfree committed
652
## specification for script that migrates user from one server to another
653

pedro-nonfree's avatar
pedro-nonfree committed
654
implementation: work in progress
655 656 657 658 659 660 661 662 663 664 665

ingredients:

- matrix API
- userA_A: user A in server matrix.a.com
- userA_B: user A in server matrix.b.com

objective: perform migration from userA_A to userA_B

procedure:

pedro-nonfree's avatar
pedro-nonfree committed
666
- userA_A massively invites userA_B to all its rooms
667 668
- userA_B massively accepts invitations of userA_A
- userA_A puts the same permissions it has on userA_B
pedro-nonfree's avatar
pedro-nonfree committed
669 670
- export riot encryption keys from userA_A to file
- import riot encryption keys from file to userA_B
pedro-nonfree's avatar
pedro-nonfree committed
671
- [optional: blank account in server A] userA_A leaves all rooms she is in
pedro-nonfree's avatar
pedro-nonfree committed
672

pedro-nonfree's avatar
pedro-nonfree committed
673
### script to blank account in server A
pedro-nonfree's avatar
pedro-nonfree committed
674

pedro-nonfree's avatar
pedro-nonfree committed
675
```python
pedro-nonfree's avatar
pedro-nonfree committed
676
#!/usr/bin/env python3
pedro-nonfree's avatar
pedro-nonfree committed
677

678 679 680 681 682 683 684
# motivation: 
# - there are problems to deactivate account and rooms in matrix
#     - https://github.com/matrix-org/synapse/issues/2017
#     - https://github.com/matrix-org/synapse/issues/1853
# - this will be part of the migration script for users between one server and another

# this script login to a specific matrix account and LEAVES ALL ROOMS
pedro-nonfree's avatar
pedro-nonfree committed
685 686 687 688 689

# requirements
# - apt install python-pip3
# - pip3 install wheel matrix_client

pedro-nonfree's avatar
pedro-nonfree committed
690 691
# based on https://github.com/matrix-org/matrix-python-sdk/blob/master/samples/ChangeDisplayName.py
# useful resource https://github.com/matrix-org/matrix-python-sdk/tree/master/matrix_client
pedro-nonfree's avatar
pedro-nonfree committed
692 693 694 695 696 697 698 699 700

import sys

from matrix_client.client import MatrixClient
from matrix_client.api import MatrixRequestError
from requests.exceptions import MissingSchema

# access data

pedro-nonfree's avatar
pedro-nonfree committed
701
host="https://matrix.example.com"
pedro-nonfree's avatar
pedro-nonfree committed
702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728
username="myuser"
password="mypassword"

# login

client = MatrixClient(host)

try:
    client.login_with_password(username, password)
except MatrixRequestError as e:
    print(e)
    if e.code == 403:
        print("Bad username or password.")
        sys.exit(4)
    else:
        print("Check your server details are correct.")
        sys.exit(2)
except MissingSchema as e:
    print("Bad URL format.")
    print(e)
    sys.exit(3)

# actions

roomlist = client.get_rooms()

print("leaving rooms:")
pedro-nonfree's avatar
pedro-nonfree committed
729
# list() does a copy of dictionary to avoid error: "RuntimeError: dictionary changed size during iteration" -> src https://stackoverflow.com/questions/11941817/how-to-avoid-runtimeerror-dictionary-changed-size-during-iteration-error
pedro-nonfree's avatar
pedro-nonfree committed
730 731
for room in list(roomlist.values()):
    room.leave()
pedro-nonfree's avatar
pedro-nonfree committed
732
    print("  " + room.room_id + " left")
pedro-nonfree's avatar
pedro-nonfree committed
733 734
```

pedro-nonfree's avatar
pedro-nonfree committed
735 736
## use our own integration server

pedro-nonfree's avatar
pedro-nonfree committed
737 738
- https://dimension.t2bot.io
- https://github.com/turt2live/matrix-dimension
pedro-nonfree's avatar
pedro-nonfree committed
739

740 741 742 743 744
### particular interest: monitoring with prometheus and grafana

- add grafana widget https://github.com/turt2live/matrix-dimension/issues/86
- prometheus alertmanager bot https://github.com/turt2live/matrix-wishlist/issues/28

pedrolab's avatar
pedrolab committed
745
## bridges
pedro-nonfree's avatar
pedro-nonfree committed
746 747 748 749

https://github.com/turt2live/t2bot.io

https://t2bot.io
750

pedrolab's avatar
pedrolab committed
751 752
with tchncs.de: https://ibcomputing.com/bridging-matrix-telegram-rooms/

pedrolab's avatar
pedrolab committed
753 754 755 756
## bots

tomeshnet bot remindering meetings: https://github.com/tomeshnet/meshbot

pedrolab's avatar
pedrolab committed
757 758
bot to interact with gitlab stuff https://github.com/maubot/gitlab https://matrix.org/docs/projects/bot/maulabbot.html

pedrolab's avatar
pedrolab committed
759 760 761
## do your own piwik analitics on matrix

something useful
762

pedrolab's avatar
pedrolab committed
763 764 765 766
# matrix email bridge idea

requires this particular [Federated Matrix Identity Server - mxisd](https://github.com/kamax-matrix/mxisd)

767
https://github.com/kamax-matrix/matrix-appservice-email - you can use this provider to test https://t2bot.io/email/
pedrolab's avatar
pedrolab committed
768 769 770

then you can probably talk with https://delta.chat/en/ users

771 772 773 774
# whatsapp and telegram bridge (take 2) idea

https://matrix.org/blog/2019/02/26/bridging-matrix-with-whatsapp-running-on-a-vm/#respond

775
https://github.com/tulir/mautrix-telegram - you can use this provider to test https://t2bot.io/telegram/
776 777 778

https://github.com/tulir/mautrix-whatsapp

779 780
# dark things about matrix and riot

pedrolab's avatar
pedrolab committed
781
Political concern: [Main funder is?was? AMDOC](https://matrix.org/blog/who-is-matrix-org/), search amodoc-mossad on google and you can discover something interesting. The funding situation is strongly changing (see relevant notes)
782 783 784 785 786 787 788

Tecnological concern: matrix generate ALOT ot metadata on the central server, even if the communication is encrypted, the metadata are not and most of the software analysis to generate network maps use just metadata, more you have more the map is accurate

Riot client is not free. Such a hype for a server-client non-free [0] software which tries to act as a Person In The Middle interconnecting any other network...

[0] https://directory.fsf.org/wiki/Talk:Riot-im

789 790
relevant notes:

791 792
- matrix foundation 1/2 https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/
- towards open governance https://matrix.org/blog/2018/06/20/towards-open-governance-for-matrix-org/
793 794 795
- https://matrix.org/blog/2018/04/26/matrix-and-riot-confirmed-as-the-basis-for-frances-secure-instant-messenger-app/
- https://matrix.org/blog/2018/01/29/status-partners-up-with-new-vector-fueling-decentralised-comms-and-the-matrix-ecosystem/
- https://matrix.org/blog/2017/07/07/a-call-to-arms-supporting-matrix/
pedrolab's avatar
pedrolab committed
796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824

## extra notes 2019-2-22

Here follows a backup of the insights from [Maximus](https://gitlab.com/maxidorius) and [this is the link to the original conversation](https://matrix.to/#/!RlmAHyIuKhyVybkriS:kamax.io/$1550849934243IAKmm:matrix.guifi.net?via=kamax.io&via=matrix.org&via=matrix.ordoevangelistarum.com)

Given the inability to implement and document a functional Matrix protocol by New Vector and resolve security vulnerabilities that are public for years, mxhsd will no longer try to implement the Matrix protocol. Instead it will implement what we (Kamax) think makes sense and document as we go. -> https://github.com/kamax-matrix/mxhsd

In no specific order:

- State resets which can be used to abuse permissions or create states that never existed (it was used last year to take over Matrix HQ per ex)
- The value of depth is not checked/validated properly, leading to possible room takeover attacks
- Until very recently, malicious/fake evens could be injected
- There are amplification/proxying attacks via the /send_join and /media endpoints

and that's just what's on top of my head - I didn't bother to keep scores in the end

there are also a whole set of (D)DoS attacks on a server by abusing server ACLs, max values of various fields and the likes

I've spent several years in the Matrix ecosystem and built a bunch of projects on it, and at least one for each spec (Client, Server, Application, Push). There is much more to it than what I just listed, but it would be a waste of time to discuss them further. I've already written things like a [Server ACL feature review](https://gist.github.com/maxidorius/b25769f1a89c8860b928babe795bbaaf) or just explained the various issues to length in various rooms and on my [18 months as a Matrix dev manifesto](https://gist.github.com/maxidorius/2f245351cf65534df730ccf262ebc6f4).

In terms of how to solve the issues, I know several ways to solve them personally, but they tend to be all related so it's faster to just take a different approach, which I am now doing using The Grid project (fork of Matrix) and this rewrite of mxhsd now called Gridepo.

I have tried my best to also report those issues and talk to the MAtrix dev about them before going down this road but to no avail. Matrix is deeply insecure and not privacy friendly at all. The only reason it has been getting away with it so far is because there is no alternative implementation that truely challenges the protocol and the implementations. If you look in the client world per example, many are just copying what Riot is doing instead of following the spec. In terms of servers, there hasn't been any successfully federated server ever until mxhsd & construct came along.

Given that we have to reverse engineer everything all the time, we just know were all the issues are, and we are now taking action to solve them the best way we can: forking.

As for promoting Riot, my personal opinion is that it just promotes a closed protocol/ecosystem as Riot is actually not Matrix compatible and very bad in terms of privacy. One of the worse horror story is that everytime you switch to a room, a HTTP call is made to vector.im with a token directly used to retrieve your Matrix ID. They can know exactly when you do something, your Matrix ID, etc. This can also be hijacked by a third-party potentially as there is no safeguard in place to prompt the user about sharing their personal info (All this done via the /openid endpoints).

This is the kind of thing that shows me there is no path forward in Matrix as the ecosystem is simply dominated by Riot and Synapse which do not follow the spec themselves and block any implementation that stick to the spec to actually be able to do anything. So you are force to implement the custom ways of Riot and synapse (whatever you build) which prevents from solving the security issues even if you wanted to.
pedrolab's avatar
pedrolab committed
825 826

3pid problem https://github.com/matrix-org/synapse/issues/4540
827 828

In the case of federation/state resolution/internal room stuff, I know that there is no homeserver to this day, synapse included, that actually implement the [S2S API]() spec. Even just trying to [send data](https://matrix.org/docs/spec/server_server/r0.1.1.html#put-matrix-federation-v1-send-txnid) fails with a 404 with synapse, because it's actually using another URL. (this is just the tip of the iceberg). So rather than spend more time reverse-engineering a moving target, I prefer invest my own time somewhere else.