Setup a dedicated drupal.org dedicated matrix server

Can I propose that we do not use matrix.drupal.org and instead use a more generic url that is not locked into a protocol.

• chat.drupal.org
• com.drupal.org
• vox.drupal.org

Above are some suggestions of dubious quality. I am not trying to pave the way for easy migration later but subdomains like matrix.* irc.* and slack.* are only helpful for users of those services. It is very possible that someone doesn't know what a matrix, irc, or slack is.

Two different URLs might be needed: 1 for Matrix/Synapse, and a different one for Riot. Due to a possible XSS attack (which I don't really understand) it's recommended to run Riot on a different domain than Synapse.

The Synapse URL is what is associated with identity -- both of users, and room aliases. So I would suggest simply "drupal.org" for this, if that's acceptable. For this to be an option, we would need to have "https://drupal.org/_matrix/" proxy to the actual Synapse server, and we would need a SRV record set up for federation. That's really all that's necessary to get started -- then users that are attached to this homeserver would have a matrix ID of "@username:drupal.org", and we could set up room aliases like "#support:drupal.org", "#contribute:drupal.org" (these aliases can easily get added to the existing bridged rooms). If we use a subdomain for Matrix, that subdomain would be present in user ids, room aliases, and new room ids -- which might lead to more confusion.

Running our own Riot is a different thing. People can use riot.im/app and log into Drupal's homeserver directly, although this might not work with non-standard authentication/provisioning. Riot is entirely static HTML/JS/CSS, so there's no server requirements other than a basic webserver. The URL for where we set up Riot is where people go to use the service. For this, I would suggest https://chat.drupal.org.

Cheers,
John

My comment was for the user-facing front end. On the backend it makes sense to use the protocol for the sub-domain.

Hey folks, I would like to point everyone at the latest comment here:
#2490332-267: Evaluate whether to replace drupal IRC channels with another communication medium

We understand everyone's really passionate about this, but this seems to be still a very unsettled issue. In many ways the community at large has voted with their feet in terms of the majority of activity now taking place on Slack.

In the thread linked above two options are being explored 1) the status quo of fragmented channels and 2) simply making Slack the official channel.

In terms of using a client of a choice, right now we encourage you to use things like the IRC client for slack, but for the moment we will not be implementing a bridge.

Furthermore, while we appreciate that the existing unofficial bridge was set up for testing this notion, it is going to be shut down by IRC admins in 24 hours.

As I just stated in that other issue, we solve those problems by running our own Matrix server as well as running a bridge. This way, we won't have to deal with any of the problems that come with using matrix.org.

Neither of those options make sense. This issue should be postponed until #2490332: Evaluate whether to replace drupal IRC channels with another communication medium is sorted.

@hestenet, Why is the IRC bridge being shut down?

@frob Several reasons:

1. It was initially authorized as a way to experiment with potential solutions for the community - not to bridge all channels for ever.
2. In it's current form (without federated login) there's no protections from impersonation as it stand right now - which is not just a hypothetical concern, it's happened to several users (not necessarily just through the bridge of course)
3. The community has not yet agreed that "It's okay to remove individual contributors agency with respect to which platforms their communications are sent to, in return for the convenience of those communications being available on all platforms" <-- that's a major philosophical question with positives and negatives on both sides

@hestenet

About the imminent shutdown of the Matrix - IRC bridge.

I understand your distrust / worries, given that most of the people involved in these bridges and tests are not even part of the Infra team. Perhaps that can change if you allow us to prove you can trust us.

However, when was it decided to stop these bridges / tests?
Who decided? Who was consulted? By who? Was there a vote by the Infra team, IRC staff, etc?
Did you consult further with the Freenode and Matrix team?
Are you available to investigate with us the possible use case of a Drupal.org Matrix/RocketChat/Whatever implementation, or even other solutions?

We need a bit of transparency.

Thanks

@hestenet Then we need to shut down the slack channel as well. Impersonation has happened there too. There is no federation of login on Slack too.

edit 0

We also need to shut down the bots that exist in both IRC and Slack as they log the conversations in each of those mediums and copy all conversations out of the medium that they where first created. Same with all the IRC bouncers, they need to be blocked.

edit 1

In fact, my IRC client also needs to be blocked, it is logging everything in every room I am in right now.

@lpalgarvio

However, when was it decided to stop these bridges / tests?

• This was decided yesterday.
• However, let me note it was never decided that the tests would include bridging as many channels as were bridged. It was only supposed to be a *few* channels where the people in them/owners of the channels new about it and agreed.

Who decided? Who was consulted? By who? Was there a vote by the Infra team, IRC staff, etc?

Did you consult further with the Freenode and Matrix team?

Note: the decision right now was not to change any existing policy, it was only to put the brakes on the bridging experiment which has been running ahead of community acceptance.

• The slack admin channel met and agreed.
• The IRC admins met and agreed (except for Ricardo, who was not online at the time, but who I emailed)
• Several core and community folks were consulted
• One of the other IRC admins (not me) consulted with the Freenode folks

Are you available to investigate with us the possible use case of a Drupal.org Matrix/RocketChat/Whatever implementation, or even other solutions?

• From a purely personal point of view(speaking only as myself) I actually like the idea of a bridging solution, and I would not mind exploring it as an individual community member.
• From my role within the DA - lots of people are coming to me uncomfortable with how fast this is happening, and with the notion that channels they were in were being bridged without them knowing it. They feel this is different from knowing that someone could screenshot/copy-paste/set up a bouncer. (Maybe that's just because it's so new, but they were very concerned)
• Also from my role within the DA - it's looking like some (but not all) of the issues could be addressed by an official server, but I don't have that in my budget, and it would require me to go to our ED/Board and set up fundraising, which is not something that can be done at the drop of a hat, and may not be approved - so just want to put a big caution there (As much as I personally may be interested in it)

@frob/@ricardoamaro

In terms of why is the bridge being shut down but not IRC or Slack (even though you are 100% correct that there are similar impersonation concerns on both platforms).... The main difference between this point and the other platforms is the lack of opt-in/opt-out nature to using those platforms in the first place. If you use just Slack or just IRC you can choose to accept the risks and known concerns of those individual platforms. If they're bridged you can only use all of then or none of them.

The community has not yet agreed that "It's okay to remove individual contributors agency with respect to which platforms their communications are sent to, in return for the convenience of those communications being available on all platforms" <-- that's a major philosophical question with positives and negatives on both sides

In a general sense, we may ultimately want some kind of better authentication controls to verify identity and prevent impersonation *regardless of platform* but that doesn't solve the above question about the ability to selectively choose platforms. And unfortunately that idea is fundamentally at odds with the primary value of matrix - everything being everywhere.

At the moment I feel the need to err on the side of conservatism about people's privacy, even though it isn't my personal opinion.

@hestenet, That statement is far better than:

Furthermore, while we appreciate that the existing unofficial bridge was set up for testing this notion, it is going to be shut down by IRC admins in 24 hours.

I would have appreciated this statement first.

Communication has been lacking #2490332: Evaluate whether to replace drupal IRC channels with another communication medium has close to 300 comments and has been running for nearly 2 years. The plan to use matrix was not a quick decision. Momentum has grown lately, but this has not been moving fast. What has been happening is we now have a plan, things have already been discussed. We began evaluating riot.im and matrix as a platform. The serious work has begun and now is a bad time to block the bridges.

We have also been attempting to create discussion to work through the issues in other places. Here, and #2885060: Drupal Chat Bridges IRC <-> Matrix <-> Slack, and #2864161: Change Drupal Slack allowed/installed apps. Frankly the communication from DA and stake holders has been poor. Many time my own comments have been completely ignored. This poor communication leads to two things, frustration, and a feeling as though nothing is getting done so we might as well just do it ourselves.

@hestenet the privacy/transparency concerns you bring up about Matrix and IRC are pretty much the same ones Freenode had, and made a condition to Matrix bridging Freenode in the first place.

Specifically, channel presence. The one way channel presence works, is that when you're in a bridged IRC room, all Matrix users in the room are visible in the user list. This was an absolute condition Freenode had from the start. So if you're in an IRC room, you can trust that you see all the Matrix users in the room, always (with a couple of temporary glitches now and then). In this respect, the Matrix bridge is exactly like a bouncer.

The reverse is generally true as well -- joins/parts and membership lists are now sync'd from IRC into Matrix, though at times this functionality has been suspended due to server load.

I've been using the Matrix bridge as a bouncer for 2 years now, in a bunch of different #drupal rooms. If you sever the bridge, you're essentially saying I can't use my IRC client in IRC. The unofficial bridges I've been using are "portal" rooms that use strict room settings -- the key ones being that history is only visible to users who were present at the time, and no guest access -- you can only see the room with a registered account. This is more strict than Freenode itself... The new "plumbed" rooms can get set up with different history settings, as well as "guest" access.

Why would you intentionally cripple something that's been working for many people for a long time already? The recent effort has simply been to go above board and get more community support for something we're gong to do anyway -- there's a large number of us are here because we refuse to use proprietary platforms -- and that is a big part of why we use Drupal. Are you really trying to alienate contributors, and large part of the community? For the sake of a proprietary platform?

Was Slack ever subjected to this kind of scrutiny before it became "the defacto place"? I suspect you'll find a growing number of us making Matrix the "defacto" place, at least for those of us who think open source software should use open source infrastructure. Why stand in the way?

To be clear - I don't personally care whether the chat solution that people end up on is multiple solutions, one solution, proprietary, open-source, whatever - so long as the people who are using the platform agree to be using it. (And ideally the community *all* agrees to use *one* platform, but that's not been the case so far).

To be clear, the difference between Slack + IRC + whatever else (fragmentation) --- and a Matrix bridge is this:

Scenario: If I'm user A who doesn't believe in proprietary solutions or has privacy concerns with any one platform (e.g: Slack)

• Option A: The status quo, several semi-official communication channels. User A can participate in the one they believe in, and not have their words put in one they don't believe in.
• Option B: The community 'votes with their feet' and moves to Slack as an official channel.... this sucks for User A, as they'll have to choose to opt out of that channel, but at least they can do so. And they can also continue to use unofficial channels if its their choice to do so.
• Option C: The DA officially bridges all communications channels to each other... User A has no choice but to put their words into the proprietary channel they don't want to support (e.g: slack) or into the channel they're concerned about privacy in or whatever. The only way to opt out is not to participate in *any* communication channel.

Right now - I don't think that issue displays enough buy in to go forward with Option C... (Edited to add: Even if the other options also kind of suck right now).

Option C in no way precludes people setting up their own smaller channels on whatever platform they like.

What is the goal here? To create a bunch of small cliques of in-circles? Where is a new user supposed to find a person to answer a question they need answered?

"Oh, well, try Slack. If you don't find someone there, maybe IRC? Oh, and there's a growing number of people on Matrix." New user runs away screaming.

Sheesh. This is totally absurd. If you want your own private conversation, go have one wherever you want. If you want to participate in a public conversation, you should be able to do it on an acceptable platform. And if somebody wants to, uh, actually get some help -- they shouldn't have to go hunting around from silo to silo to find somebody who can help.

By refusing to bridge and supporting both Slack and IRC, you're saying that chat is not for users looking for help -- it's a political oligarchical way of identifying who matters in a community -- which is plainly not people trying to get anything done with Drupal. Good luck with that...

The ideal outcome for me:

• Everyone communicates in the same place.
• No one was non-consensually added to a communication medium.

Unfortunately I'm not sure these two things are compatible- and the community is *not* even close to agreement on this either way. In the meantime I feel morally obligated to be conservative with people's privacy.

I'm totally confused why "privacy" is an issue on a "public" channel.

@freelock, to be honest, I'm kind of with you on that front - but there lots of people expressing that concern to me.

Well, it seems easy enough to meet:

No one was non-consensually added to a communication medium.

... just add it to the topic in each bridged room, on each platform, something along the lines of "This room is bridged, all messages are visible in Slack, IRC, and Matrix." We're not bridging all of Slack, or all of Freenode -- just the public channels people may want to find and use.

The bridge can't relay messages from before it exists, so setting the topic is essentially notice. If people have a problem with that, they can certainly go create a private room -- the goal I'm trying to address is fragmentation of support resources, and this is a clear way to solve that problem.

Hello dear all IRC admins and Infra team,

Our #drupal-pt has been affected by this change and it can not be.
You see, it is oversee, ran and managed solely by the independent Drupal Portugal Community, and the ADP - Associação Drupal Portugal, non-profit organization registered and running in Portugal, which also founded all the Drupal Portugal channels across different platforms - channels which you can consult here and here.

This has happened because in the past we have decided together with the Drupal Association many years ago, in 2010 or 2011, to allow in the #drupal-pt channel the drupalircowner user, and set it as the owner, in order to Protect our IRC channel from misuse and abandonment.

You have however decided to also block (ban) the bridge in our #drupal-pt channel, as well as in other community channels, such as #drupal-be, #drupal-in, #drupal-nl and others.

It is causing us harm and grief.
We kindly ask you to unblock the bridges in such community channels, as they are run independently.

Thanks

I have now 3 servers available at Linode for usage on Matrix, RocketChat, Mattermost, bridges and related tests.
3x 1 CPU / 2 GB RAM / 30 GB Storage
https://www.linode.com/pricing

Debian 8, with Docker and Rancher setup.
Ping me if you need resources.

Moving this issue to the matrix project.

At now Matrix.org release a new 0.2 version of Slack bridge that implement new way of bridging via native Slack app, that much easier to setup, and have limited list of permissions: https://slack.com/apps/A1BKR8Y8J-riot-bridge - so no private info and private rooms will be shared.

All Drupal IRC rooms on #freenode are already bridged to Matrix.org server.

Rocket.Chat rooms can be bridged via https://github.com/matrix-org/matrix-appservice-rocketchat or https://github.com/42wim/matterbridge and in future - via this: https://github.com/matrix-org/matrix-appservice-rocketchat

SSO can be implemented via https://github.com/kamax-io/mxisd and LDAP or any other modern auth provider.

Also Matrix.org implemented GDPR policy.

Private messages and rooms can be E2E encrypted for increase privacy.

Matrix.org is a decentralized network, so we can start implementing bridging on one server, and in any time move this to any other server.

So, at now we already can implement bridge of all Drupal chat communities in one Drupal Supercommunity, without dedicated Drupal Matrix server, and when server will be created - do live migration (mirroring) of all rooms to Drupal dedicated server without needs to re-register users or any other changes. Maybe there are time to start bridging now?

Also want to tell, that KDE community totally moved from IRC to Matrix (instead of Slack, Rocket.Chat, Mattermost, etc) - https://dot.kde.org/2019/02/20/kde-adding-matrix-its-im-framework

The issue was postponed in #2905534-8: Setup a dedicated drupal.org dedicated matrix server:

Neither of those options make sense. This issue should be postponed until #2490332: Evaluate whether to replace drupal IRC channels with another communication medium is sorted.

I am un-postponing the current issue, since the referenced issue is closed for over an year now.

Matrix.org have released stable 1.0 version of his protocol and Synapse server, and decrease memory and resource requirements. So, maybe already there are time to launch dedicated Matrix server for Drupal community and configure bridges to all Slack, IRC and RocketChat channels?

#35 +1

Re: the Synapse server - could someone closer to the project than I help me discover what authentication methods are properly supported?

https://github.com/kamax-matrix/mxisd seems to have a few features, but a few still incomplete.

For example, according to this thread there is not OpenID connect support: https://www.reddit.com/r/selfhosted/comments/bz4u25/matrix_synapse_with_... - however there is nearly solid SAML setup, although the pull request has yet to be merged: https://github.com/matrix-org/synapse/pull/5422

I'm having a little trouble finding documentation on all the Matrix Federation and Identity options supported by the 1.0 release in one place...

.. It would be unfortunate for us to implement a federated identity method that's not supported.

Matrix Synapse server authentication methods is not so rich at now, because the main priorities was be stabilization of protocol core, and new features implementation was freezed. So now, after stable version released, new features will continue developing.

Basic auth can be implemented using LDAP protocol via https://github.com/matrix-org/matrix-synapse-ldap3 module, that used by kde.org community. I think that modular.im service (from Matrix.org core team developers) will be happy to help with configuring auth and other integration features for Drupal community, like for KDE community before, and maybe can provide free hosting.

Also there are exists https://github.com/kamax-matrix/matrix-synapse-rest-password-provider and https://github.com/kamax-matrix/mxisd contrib modules, that can be integrated with any other auth method using REST interface - they are archived (temporary, I hope), but works very stable for all my Matrix servers setups.

So, can anybody from official Drupal team members talk directly with Modular.im specialists about integration Drupal community to Matrix in #modular:matrix.org room?

Hi, Matrix project lead here. As Murz says, we haven't been focusing on authentication methods on the core project whilst working on 1.0, but we're playing catch-up now. LDAP exists via https://github.com/matrix-org/matrix-synapse-ldap3; CAS is supported too out of the box, and SAML will be supported very shortly too (Mozilla are using it for their test currently). We don't recommend mxisd currently, not least because it's currently between maintainers. The biggest omissions on SSO are OAuth and OpenID connect flows, but if there's a requirement for these we could see about making them happen.

If you did want to get up and running quickly, https://modular.im is the best way to do it.

Matrix.org have released new version of LDAP integration module https://github.com/matrix-org/matrix-synapse-ldap3/releases/tag/v0.1.4 so now we can easily integrate Matrix server with drupal.org accounts via LDAP!