July 14, 2017

Matrix community roundup

08:34 -0400

Last week, the Matrix team put out a call to arms for the community to support Matrix financially, and the community response has been great. Response in online forums such as reddit and Hacker News has been extremely positive, and in the first 24 hours, the community pledged well over $1000/month through Patreon and Liberapay. Although it has slowed down since then, it is now over $1700/month, getting close to the first goal of supporting one developer working half time on Matrix.

While last week was the first time that the Matrix community was able to contribute hard dollars towards the development of Matrix, the community has been supporting Matrix in other ways for quite a while. Since I started following Matrix a little less than a year ago, I've seen that the Matrix community has been quite active testing, filing and triaging bugs, contributing code to core projects, writing bots and bridges, providing support in the Matrix rooms, and more.

Here's a quick roundup of some of the new things that the community has come up with since the Matrix Holiday Special. I have undoubtedly missed some projects, so apologies in advance for all those that I've missed.

Voyager

TravisR has been experimenting a lot with Matrix, and one of the unique things he has come up with is a bot that maps out how Matrix rooms are related to each other by noting when one room is mentioned in another room. The results are then mapped in a massive graph.

Linux distributions

Gentoo: PureTryOut has created an overlay for Gentoo for installing some Matrix-related software.

Debian: Synapse and the Matrix plugin for Pidgin have been packaged for Debian and are included in Debian unstable, and other Matrix-related software has been packaged and submitted for inclusion. Myself and others have also been working on forming a Matrix packaging team.

Integrations

One of Matrix's main features is the ability to bridge with different networks, and while the core Matrix team has had their hands full maintaining the IRC, Gitter, and Slack bridges, the community has been writing bridges to other networks.

Puppeting bridges: The Matrix Hacks group has added quite a few new bridges this year so far. Looking at their GitHub account, they have bridges for Hangouts, Slack, Skype, Signal, and GroupMe, in addition to the previously-announced iMessage bridge. Discussion and support for these bridges take place in #matrix-puppet-bridge:matrix.org.

email: Two email integrations have been written, which work in different ways. Max's bridge allows email users to participate in Matrix rooms while TravisR's bot sends messages to rooms when it receives an email.

Discord: Half-Shot has also written a Discord bridge.

Clients

Riot: While the core Riot team was busy working on creating an improved experience for new users, the community implemented some of Riot Web/Desktop's most requested features, resulting in the release of Riot Web/Desktop 0.10 in which all of the major new features were initiated by the community. I think that's an achievement that the community can be very proud of, as well as the core Riot team, for being able to foster such an active community.

Nheko: As good as Riot is, it isn't for everyone. Development on other clients such as Quaternion has continued, but a new one, Nheko, has been started recently which already seems quite promising.

Matrix Recorder: Although Matrix keeps all history on the server, some people want to keep their own copy of history. Alex created Matrix Recorder, which saves history to a local SQLite database. Matrix Recorder even supports saving history from end-to-end encrypted rooms.

e2e crypto: While end-to-end cryptography is still in beta in Riot, some brave souls have been experimenting with it in other clients. In addition to Matrix Recorder's support of e2e rooms as mentioned above, penguin42 has done work on adding e2e to the Matrix Pidgin plugin and davidar has added e2e support to his Hubot adapter.

SHA2017 badge: One of the most intriguing projects is the badge for the SHA2017 camp, which reportedly contains a Matrix client. I don't know what they're using Matrix for, so I hope they do a write up at some point.

Documentation/Talks

The Matrix community has been busy writing documentation and blog posts, and doing talks about Matrix. Coffee has been collecting helpful information about Matrix into a machine-readable knowledge base. Some guides for Riot have been written, including usage basics by muppeth, maxigaz's guide, and an introduction from an IRC perspective. CryptoAUSTRALIA recently had a workshop for setting up Synapse and Riot, and published a tutorial online. And PureTryOut did a Matrix talk at the Dutch Linux User's Group a few months ago. There have certainly been other people from the community doing talks about Matrix that I am not aware of.

Server list

A federated communications protocol is less valuable if users can't find servers to join. Since there is no official list yet, Alex set up a list of Matrix servers. Though to call it just a list of servers is an understatement. It includes statistics on each server such as uptime, response times from various locations, and SSL test scores, so that users can make a more informed choice of servers. If you are running a Matrix server, whether public or private, please consider submitting your server to the list to improve visibility for your server and to strengthen the federation.

GSoC

Matrix was again accepted as a mentoring organization for Google Summer of Code and has three students. Two of them are working on iOS-related projects, and since I don't have an iOS device, I haven't been following their progress. However, Michael (a.k.a t3chguy), in addition to improving Riot, has been working on creating a search engine-friendly view of public rooms, which will be helpful for Matrix rooms that are used as support forums.


Matrix would not be what it is today without the support of the community, and I'm looking forward to seeing what the community will develop in the future. Last week, the community was invited to contribute financially towards Matrix's development. But for those who are unable to contribute in this way but still want to support Matrix, or for those who have pledged money but still want to do more, hopefully this list gives some ideas for how you can help out, either by supporting an existing project or starting your own.

Addendum (July 19, 2017)

Some projects that I missed:

Max has written an alternative Identity Server implementation called mxisd. Identity Servers haven't been getting as much attention as homeservers, application services, or clients, so it's great that someone has been working on an alternative implementation.

TravisR has also been working on an Dimension, alternative implementation of Riot's integration manager.

0 Comments
July 2, 2017
23:12 -0400
Hubert Chathi: Celebrated Canada Day in two different provinces, and glad to finally be home after a long night and day of driving through two more provinces.
0 Comments
July 1, 2017
07:52 -0400
Hubert Chathi: Happy 150th birthday!
0 Comments
June 30, 2017
16:39 -0400
Hubert Chathi: My # talk for @debconf17.debconf.org has been accepted. # #
0 Comments
May 28, 2017
11:34 -0400
Hubert Chathi: After about 2.5 years, I'm finally back in the @debian.org keyring
0 Comments
April 1, 2017

An alternate transport for the Matrix Client-Server API

00:00 -0400

Matrix is an open communications protocol that has many great features. However, one flaw that it has is that the baseline specification is based on long-polling HTTP requests, which is not very efficient. In order to address this deficiency, I've created a spec that presents an alternative transport for the Matrix Client-Server API that uses a protocol that was designed for real-time communications instead of using HTTP.

0 Comments
March 13, 2017

The latest additions to my init.el

11:00 -0400

Inspired by xkcd (but using Alt-mousewheel):

(global-set-key (kbd "<M-mouse-5>") 'undo)
(global-set-key (kbd "<M-mouse-4>") 'redo)

And, since I sometimes need to paste from an HTTP request into a buffer:

(defun insert-from-url (url)
  (interactive "MURL: ")
  (let ((url-request-method "GET")
        (dest (current-buffer))
        (src (url-retrieve-synchronously url)))
    (set-buffer src)
    (goto-char (point-min))
    (search-forward "\n\n")
    (set-buffer dest)
    (insert-buffer-substring src (match-end 0))))
0 Comments
March 9, 2017
23:00 -0500
Hubert Chathi: First, @google.com combines Google Talk with Hangouts, and now they are separating the conferencing and chat functionality again
0 Comments
March 3, 2017
15:19 -0500
Hubert Chathi: so true
0 Comments
February 27, 2017
20:11 -0500
Hubert Chathi: Congratulations to all the organizations accepted to GSoC
0 Comments
February 24, 2017
09:56 -0500
Hubert Chathi: Well, thats just embarassing.
0 Comments
February 23, 2017
22:12 -0500
Hubert Chathi: Anyone proxied by @cloudflare.com or using sites proxied by them: your private data may have been leaked #
0 Comments
11:18 -0500
Hubert Chathi: SHA-1 is officially broken #
0 Comments
February 15, 2017
19:02 -0500
Hubert Chathi: RIP Stuart McLean
0 Comments
February 11, 2017
10:00 -0500
Hubert Chathi: Sign the petition to ask the government to honour their promise to fix our electoral system #
0 Comments
January 31, 2017
08:54 -0500
Hubert Chathi: Got our free Parks Canada Discovery Pass yesterday. Get yours at http://www.parksorders.ca/
0 Comments
January 25, 2017

On transparency

21:01 -0500

I've written briefly before about the value of companies being open and transparent. Back then, I wrote that the way that companies react when things go wrong is a good way to differentiate between them. No matter what company you deal with, things will go wrong at one point or another. Some companies try to avoid responsibility, or only tell you that something has happened if you ask them. Others companies are much more open about what happened.

Matrix.org (and the associated Riot.im) is an example of a team that falls into the latter category. And last night's incident is a good example. Their post-mortem blog post is a great example for others to follow. It gives a detailed timeline of what happened and why the outage occurred. And it finishes off with steps that they will take to prevent future incidents.

Kudos to the Matrix.org team for their transparency.

0 Comments
December 25, 2016
09:30 -0500
Hubert Chathi: Merry Christmas
0 Comments
December 1, 2016

Let's Encrypt for Kubernetes

21:08 -0500

A while ago, I blogged about automatic Let's Encrypt certificate renewal with nginx. Since then, I've also set up renewal in our Kubernetes cluster.

Like with nginx, I'm using acme-tiny to do the renewals. For Kubernetes, I created a Docker image. It reads the Let's Encrypt secret key from /etc/acme-tiny/secrets/account.key, and CSR files from /etc/acme-tiny/csrs/{name}.csr. In Kubernetes, these can be set up by mounting a Secrets store and a ConfigMap, respectively. It also reads the current certificates from /etc/acme-tiny/certs/{name}, which should also be set up by mounting a ConfigMap (called certificates), since that is where the container will put the new certificates.

Starting an acme-tiny pod will start an nginx server to store the .well-known directory for the Acme challenge. Running /opt/acme-tiny-utils/renew in the pod will renew the certificate if it will expire within 20 days (running it with the -f option will disable the check). Of course, we want the renewal to be automated, so we want to set up a sort of cron task. Kubernetes has cron jobs since 1.4, but at the time I was setting this up, we were still on 1.3. Kubernetes also does cron jobs by creating a new pod, whereas the way I want this to work is to run a program in an existing pod (though it could be set up to work the other way too). So I have another cron Docker image, which I have set up to run

kubectl exec `kubectl get pods --namespace=lb -l role=acme -o name | cut -d / -f 2` --namespace=lb ./renew sbscalculus.com

every day. That command finds the acme-tiny pod and runs the renew command, telling it to renew the sbscalculus.com certificate.

Now in order for the Acme challenge to work, HTTP requests to /.well-known/acme-challenge/ get redirected to acme-tiny rather than to the regular pods serving those services. Our services are behind our HAProxy image. So I have a 0acmetiny entry (the 0 causes it to be sorted before all other entries) in the services ConfigMap for HAProxy that reads:

    {
      "namespace": "lb",
      "selector": {
        "role": "acme"
      },
      "hostnames": ["^.*"],
      "path": "^/\\.well-known/acme-challenge/.*$",
      "ports": [80]
    }

This causes HAProxy to all the Acme challeges to the acme-tiny pod, while leaving all the other requests alone.

And that's how we have our certificates automatically renewed from Let's Encrypt.

0 Comments
November 21, 2016
21:31 -0500
Hubert Chathi: Congratulations to @riot.im and @matrix.org for the beta release of cross-platform end-to-end encryption
0 Comments