As issue #2490332: Evaluate whether to replace drupal IRC channels with another communication medium has shown, a lot of people have their preference as to what a chat service/protocol should be.
We think we can find a middle ground and just make bridges between all these chat services and protocols in use.
The people behind the Matrix protocol were so kind to free up the #drupal:matrix.org room.

The idea is that we run bridges between those chat protocols. This would mean everyone is on all networks, and crosstalk is possible.

So, instead of bikeshedding endlessly about what service should be used, let's bridge em together. But for creating a bridge, we need a person with an op on each channel.
Matrix has infrastructure in place to provide such bridges, but someone with an op on IRC is needed to confirm the bridge request. This is the list who received ops over the #drupal irc room on freenode over the years: https://www.pastery.net/sjqwvn

That's for the IRC part. Now the Slack part. As creating a bridge would need for someone managing the slack channels to set an incoming and outgoing webhook.

What is needed now? We need people with the +o flag on IRC channels to allow a bridge to start.
On https://www.pastery.net/sjqwvn are users with op flags for the #drupal room.
Speak up to those persons about this issue and let's get this thing going!

IRC server #room SLACK server #room MATRIX #room:server
freenode #drupal (328) drupal #general (3834) #drupal:matrix.org
freenode #drupal-contribute (204) drupal #contrib (346) #drupal-contribute:matrix.org
freenode #drupal-support (195) drupal #support (3452) #drupal-support:matrix.org
freenode #drupal-accessibility (4) drupal #accessibility (195) #drupal-accessibility:matrix.org
freenode #drupal-commerce (61) drupal #commerce (326) #drupal-commerce:matrix.org
freenode #drupal-be (3) drupalbe #general (32) #drupal-belgium:matrix.org
freenode #drupal-console (2) drupal #drupal-console (149) #drupal-console:matrix.org
freenode #drupalcon (21) drupal #drupalcon (253) #drupal-drupalcon:matrix.org
freenode #drupal-twig (19) drupaltwig #twig (912) #drupal-twig:matrix.org
freenode #drush (42) - #drupal-drush:matrix.org
freenode #aegir (33) drupal #aegir (16) #drupal-aegir:matrix.org
freenode #drupal-media (54) drupal #media (88) #drupal-media:matrix.org
freenode #drupal-search-api (16) drupal #search (60) #drupal-search:matrix.org
freenode #drupalorg (25) drupal #drupalorg (5) #drupal-drupalorg:matrix.org
freenode #drupal-vm (33) - #drupal-vm:matrix.org
CommentFileSizeAuthor
#26 Selection_361.png31.62 KBfreelock
#26 Selection_359.png50.54 KBfreelock
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

betz created an issue. See original summary.

betz’s picture

Matrix channels link to the riot web client showing that specific room on the corresponding (matrix.org) server.
You can use whatever matrix client to access the room. See https://matrix.org/docs/projects/try-matrix-now.html for more matrix clients.

betz’s picture

Above are all Drupal rooms on their networks, mapped.
The amount of members at the time of visiting is always listed.

freelock’s picture

Matthew @ matrix.org did mention he had offered/planned to provide a private homeserver for Drupal and other groups... spinning up a dedicated homeserver could allow us to have a more permanent home that could be easily moved to DA control if they decide they want to manage this...

This would allow the Matrix channels to have handles like:

#general:drupal.org
#contrib:drupal.org
#support:drupal.org

... etc.

This would involve setting up a SRV record for drupal.org pointing to the dedicated homeserver.

Of course, these could be simply added as aliases later -- the beauty of Matrix is that any room can have multiple aliases, and is equally shared across all the homeservers that have users in the room.

Otherwise, a big +1...

colan’s picture

What's the IRC ops command to run to allow the bridge? I should be able to do this for Aegir.

ricardoamaro’s picture

Hello there,

IRC channels are managed by us on the infra team via the drupalircowner.
Probably giving op to a unattended bot at someone's home server is not very secure and could lead to either unreliable service or hijack.
As the bridge is something that I personally would like to see happen in a more managed way i suggest engaging the infra team to setup a dedicated place for the bridge.
On the other end we already have a bot managed by us, the Drupalbot which is a good candidate to perform this service.

freelock’s picture

@ricardoamaro Matrix.org already runs a public bridge infrastructure to all of Freenode... we're already using Matrix on Freenode, so the main reason to do more explicit plumbing would be to allow accounts to have Matrix power granted from their Freenode ops status...

To me this is a lower priority than bridging Slack. While Slack is dividing the community into fragments and excluding all the IRC users, Matrix can bring them together again. I'm not sure whether we would need an op on the Freenode/Matrix side to bridge into Slack -- I do know we need authorization to add two integrations to each room on the Slack side to get this hooked up. (Otherwise we probably would have done it by now!)

betz’s picture

Issue summary: View changes
betz’s picture

colan, i'm online on matrix, ping me, i start the op process.
What will happen is you will get a message on IRC which you need to reply a 'yes' within 15 minutes.
https://riot.im/app/#/room/#drupal-aegir:matrix.org

betz’s picture

about slack bridging, i got an answer from davidhernandez:
"Hi. The short answer is no. We discussed that a bit previously amongst admins, and it is not something we are going to pursue just yet. See this issue for background https://www.drupal.org/node/2864161"

I propose to focus on IRC for now and get all rooms bridged already with respective matrix channels. This can hopefully show the potential and bring (admin) people on slack in motion.

betz’s picture

ricardoamaro, about the irc bridge, the irc bot does not need op rights.
A person with op right just needs to confirm 'yes' when messaged with question to start the bridge.
This is just to prevent misuse of the matrix.org bridge infra. Again, the bridge can work perfect without op rights.

ricardoamaro’s picture

Ok, i seem to understand a bit better now what the intent is here.
I agree, that having the community divided between IRC and SLACk is really alarming and should be attenuated.
I'm open for any initiatives that are being taken to get the all community communicating again.
About what @davidhernandez says... we do control all the #drupal-* IRC channels and we should do whatever is in our reach to not let the gap grow bigger.
IRC is historically the open way of growing an opensource project and we don't mind that people use whatever they like when they send a message to it as long as the message gets there and they get their responses.

davidhernandez’s picture

In the other issue (#2864161: Change Drupal Slack allowed/installed apps) there were a number of points being made with regard to privacy concerns, control of software, archiving, user notification, policies and procedures, etc. None of the Slack admins were outright saying this was a bad idea, but many of those were not fully addressed/thought out and it seems some people are ignoring them for the sake of "we can technically make it work, so just do it already". No, I do not agree it is a "no brainer".

And the fact that someone will privately messages me in Slack and then copy and paste my response in public tells me a great deal about how much consideration is going into any of those concerns.

I agree, that having the community divided between IRC and SLACk is really alarming and should be attenuated.

I don't agree. Let's not pretend anything but a certain percentage of the community was ever on IRC. We saw large numbers of people jump on Slack precisely because it wasn't IRC. That supports the point of bridging the apps, but we haven't created a new problem. The community was never converged on a single communications platform, so, no, the sky isn't falling. (For one thing, there are three different Drupal community Slacks that I'm logged into. I don't imagine we'll be bridging them all.)

mlhess’s picture

I have a tech question about bridges.

On IRC, if a user from matrix is misbehaving, can we use the standard IRC tools to kick the user, or does this require someone to go to matrix to kick the user?

I guess the reverse question is also valid, can you kick an IRC user from matrix?

betz’s picture

On IRC, if a user from matrix is misbehaving, can we use the standard IRC tools to kick the user, or does this require someone to go to matrix to kick the user?

I guess the reverse question is also valid, can you kick an IRC user from matrix?

All users are regular users. So matrix users on irc are regular irc users and can be kicked.
Same for irc users on matrix.

lpalgarvio’s picture

@betz can you add the table to the issue description?

Drupal Portugal is synchronized as well.

IRC server #room: freenode #drupal-pt
SLACK server #room: not possible without hooks
MATRIX #room:server: #drupal-pt:matrix.org / #drupal-portugal:matrix.org / #drupalportugal:matrix.org
Telegram group: drupalportugal
Gitter room: drupalportugal/community

more info - https://www.meetup.com/drupalportugal/pages/22738118/Chat_platforms

ekes’s picture

None of the Slack admins were outright saying this was a bad idea, but many of those were not fully addressed/thought out

I'm not sure what the specific requirements are but the potential is:

  • All history visible to all;
  • All history visible to all who join the channel;
  • History visible to those who those who join only from the point they were invited;
  • History visible to those who join from the point they join.

The last is the enforced default for automatically bridged Freenode channels. So this is something we already have for IRC as all Freenode channels can automatically bridge into those sort of matrix channels. It means that you can only see the history of the channel when you are visible in it to other users in the bridged IRC channel.

I'd love it if someone could explain how much history is maintained by Matrix. I think the original was all the things that a user has access to forever; but there is now a history Purge API or something? As I understand it, and I would be pleased if someone running a server can confirm or correct, a channel federated to another matrix server synchronizes as much history as is required for the users of that server. Being involved in another community that was dubious about bridging to Matrix I also know it is possible to have channels that do not federate, so the history remains only on servers that you choose.

In addition, within Matrix, you have the option to have encrypted only channels. These are such that the history is encrypted. You can't even decrypt the history yourself if you no longer have the device key that it was encrypted too. The app automatically generates a key, you can share this, but if you delete it you won't have the history yourself even.

All these options can be configured per-channel.

In conclusion as far as I can see, it should be possible to have sensible defaults if they are decided; and if there are specific requirements each channel should specify them, and they should be implemented.

freelock’s picture

I'd love it if someone could explain how much history is maintained by Matrix. I think the original was all the things that a user has access to forever; but there is now a history Purge API or something? As I understand it, and I would be pleased if someone running a server can confirm or correct, a channel federated to another matrix server synchronizes as much history as is required for the users of that server.

Hi, @ekes,

We've been running our own Matrix homeserver for nearly 2 years. Rooms in Matrix are not "owned" by any server whatsoever -- they are shared by all the homeservers that have members in the room. So room history is stored on all homeservers that have a user in the room.

If the room permits viewing history before a user has joined, the homeserver will request history from other homeservers as the user scrolls back in the room, a page at a time. It won't ever just download the full history -- just what users request. So yes, you're correct, the Homeserver only downloads the history required for the users of that server (or requested, if the room settings permit that).

The "Purge API" is something a homeserver administrator can execute manually to remove history from a room. This is mainly intended to reclaim disk space, and has no effect on what other homeservers may have or store -- and if a user in a room with purged history scrolls back to missing posts, the homeserver will request the messages from other homeservers in the room.

So history, for all intents and purposes, is permanent -- it is only lost if ALL homeserver admins choose to purge the history of a room they share.

The best analogy is a mail server. Anyone may run a mail server, and have their own message retention policies, but they have no control over the retention of messages on other users' mail servers.

Setting a room to "unfederated" would make it so that only user accounts on that homeserver can join the room. This might be useful for very sensitive topics internal to a company, but otherwise not generally useful here -- nobody could join the room without creating and using an account on that server.

End-to-End encryption is far more useful for sensitive topics -- this would be really great for a room devoted to security issues, as server admins cannot decrypt the messages. (For plain text rooms, it is possible for a homeserver admin to read messages by poking around in the Postgres database, at least as far back as their users have retrieved history). But the bridge is not currently capable of handling E2E, and it would be pointless on bridged rooms as the bridge would have to drop chats unencrypted into IRC.

betz’s picture

The #general slack room on the drupalbe slack server just was bridged with matrix!

betz’s picture

#drupal:matrix.org, #drupal-support:matrix.org, #drupal-contribute:matrix.org, #drupal-media:matrix.org, #drupal-drupalorg:matrix.org, #drupalcon:matrix.org and #drupaldevdays:matrix.org are bridged with their IRC channels now.

lpalgarvio’s picture

Congratulations.

But I feel left out of this, having pioneered some of these initiatives and experimentations with the Drupal Portugal communication channels and having already inputted my views and opinions both here, on other threads, and via email to some Infrastructure members.

Taking input, sometimes ignoring or neglecting it, then refusing to let people in to be part of experimentation and internal debates, and then deciding and posting these decisions and changes, without giving any credit or heads up, or even stating you are at the moment debating things, is not the FOSS way of doing this.

Now I pose questions,
What are the policies for these integrations? What can users and admins of channels across services expect?
Why again aren't we part of this debate? Why wasn't the knowledge and experience of some people who contributed in some way used on any tests? Were there any tests at all? Who gets to decide which channels have integrations? And which integrations?

I'm sorry but it just doesn't feel right. This is not the path to openness nor growth nor increasing people's contributions or communities participation. It's exactly the reverse.

All it tells me is to quit helping out.

colan’s picture

@betz: As requested in #16, would you kindly move the table to the issue description so that it can be edited by others? Thanks.

davidhernandez’s picture

@lpalgarvio because apparently some people are more interested in progress how they see fit than cooperation or collaboration.

What are the policies for these integrations?

There are none.

What can users and admins of channels across services expect?

Your guess is as good as anyone's.

betz’s picture

Issue summary: View changes
betz’s picture

@colan: done.

lpalgarvio & davidhernandez: Sorry. I stepped on toes apparently.
I was just focusing for the last month on getting bridges working and ask around.
Someone from drupal infra told he wanted to start the IRC bridges.
And for the drupalbe slack server i just asked the admins of that server.

Don't see what I should have done differently. Start another 200 comment issue?
Just want to get those rooms bridged, no personal feelings. :)

freelock’s picture

FileSize
50.54 KB
31.62 KB

timplunkett had some concerns in #drupal about being able to kick abusive users who might join the Matrix side. I confirmed in #matrix-irc that the bridge propagates IRC ops and kicks/bans through the bridge... so we ran a test.

Tim kicked me out of #drupal from the IRC side, and I got kicked out of the Matrix room. I was actually in two different bridged rooms into #drupal -- #drupal:matrix.org and #freenode_#drupal:matrix.org -- and got kicked out of both at the same time.

So adding these bridges to IRC does not change the abuse control in any way -- it's not necessarily any better (though Matrix has abuse prevention/reputation systems on their radar) but it's not any worse.

Attached are screenshots showing what it looks like when you're kicked, along with the bridge granting moderator status to timplunkett when he kicked me...

drumm’s picture

How exactly are these bridges being set up? https://matrix.org/docs/guides/types-of-bridging.html looks like quite a lot of options and I haven’t been able to match up any of them to what is mentioned here.

The https://www.pastery.net/sjqwvn link in the issue summary is not good anymore.

freelock’s picture

@drumm The bridges we're trying to get set up are "Plumbed" bridges on that guide. This allows the same room to bridge to other networks, such as Slack and/or Gitter.

The IRC bridges are "double-puppeted," but the Slack bridge is a "Bot-API" bridge and not quite as transparent. The Gitter bridge I think is a "bridgebot" style, but they're working on making that a "simple puppeted" bridge.

Of course, there have been Portal bridges to various IRC rooms for quite some time -- I used a Portal-bridged room to be on #drupal for the past 18 months, before the Plumbed room was set up.

Cheers,
John

ricardoamaro’s picture

I think that for this initial phase things went pretty smooth.
What are we missing to bridge the Slack channels?

betz’s picture

No idea what is still missing or unclear.
I contacted timplunkett about one month ago, for starting slack bridges and this was his response:
"I’m not bridging anything until a decision is made on the D.o issue".

So, what we need to sort out?
Few questions come to mind:
- Privacy issue, make it clear the chat is cross-platform.
- Administration. If an op kicks someone from IRC, is this person also kicked from matrix/slack?
- ?

For the privacy issue, this seems manageable by a disclaimer linked from the topic.
Administration, we had a test already. Someone on the matrix protocol can be kicked from IRC, this works.
For slack, we can't do this. To kick someone on slack, you need to kick em from slack. The webhooks don't allow this.

lpalgarvio’s picture

@administration
Eventually the bridge can be improved (or not).
But as long as there are enough active admins on slack, this doesn't seem an issue to me. If there are not enough, find more.
Even if admins are away, the bridge can be disabled or silenced (kick/ban the bot/bridge) temporarily in case of a catastrophe.

@privacy
If it's stated, then it's obvious. So let's state it.
if it's not, people should know that this a possible behaviour on Slack, just as it is possible on IRC, Matrix, Telegram and other protocols. We live in the age of the Cloud, like it or not (i don't).

Further more, if nothing changed, the Slack Team is on a free plan with no costs, which allows for storing only up to 10k messages at a time, which was already noted somewhere. I'm guessing this is easily exhausted every day if there are 20 channels bridged with IRC and Matrix. IMO, the potential to leak something out for an extended amount of time to Slack is smaller than the other way around. Messages that are deemed sensitive (like passwords) can be deleted in Matrix, Telegram and Slack.

I'm not affiliated in any way with the current bridges, nor have insights, but I base my advices on experience elsewhere.
Nothing else to add.

tim.plunkett’s picture

@betz, you quote me directly from a private message without asking me first whether I was comfortable with that being shared publicly.
This further reinforces your lack of concern for the privacy of users.

Harassing Slack admins will not further your cause.

betz’s picture

I guess you could see this as interaction with public figures. This was not a personal chat, but a reach to a infrastructure maintainer.
A right to record the cops / politicians? :) I would never post private communication.
Anyhow, the matrix server and the bridge infastructure is not mine, so my poor privacy respect is not related. I am not an admin.

Could you elaborate on what you mean with 'until a decision is made on that d.o issue'?
What are the exact items that need clarification before a decision 'can' be made?

betz’s picture

OK sorry, I spoke too fast there.
I currently am an admin, an admin of the matrix channel.
Just because I created the matrix channels.
But I can contact Matthew, the main matrix developer, to remove myself and put a infrastructure user as admin.

davidhernandez’s picture

I would never post private communication.

You literally copied and pasted a private communication with me in an earlier comment.

Your excuse that we are admins indicates we have no privacy rights anywhere on Slack for anything we say. This only increases the lack of trust for your decision making around how people's data is handled. We've clearly indicated that isn't taken lightly.

betz’s picture

I am sorry to have doing this.
I felt this was ok as this was a request about this issue, not general private chat.
But I made the wrong judgement there. I am sorry Tim.

Still, can we make a list of what are the open issues left?

betz’s picture

BTW, we will have a bof on drupalcon vienna about matrix
https://events.drupal.org/vienna2017/bofs/irc-and-bridging-drupal-community

Thinking of making it a sprint. Continue work on the drupal matrix api module.

pwolanin’s picture

I'm strongly opposed to having this bridge set up. I want to be able to control the distribution of my communication and not have it automatically copied into other chats.

In addition, I think this would be a very poor user experience in terms of the volume of messages and lack of context.

betz’s picture

I'm strongly opposed to having this bridge set up. I want to be able to control the distribution of my communication and not have it automatically copied into other chats.

But these are no 'other' chats, just another protocol to the same community.

In addition, I think this would be a very poor user experience in terms of the volume of messages and lack of context.

We have a few channels already bridged, most IRC channels and a few country slack channels.
From the client you are using there really is no difference. We want to bridge the corresponding channels with each other.
For example: #drupal-nl on IRC, #drupal-nl on matrix, and the #general room on drupalnl on slack.
So same context, no confusion. We wouldn't want to bridge #drupal-nl with the main #drupal room, as an example.

gdd’s picture

I would like to strongly express my opposition to this move. One of the things that has been happening in recent months is the appearance of sockpuppet or spoof users in Slack. There have been two incidents (that I know of) wherein a user impersonated a community member in Slack, complete with their user profile picture and details, and began impersonating that user in different channels. One of the reasons we were able to identify these people swiftly is because they are forced to register in Slack, and one of the admins was able to check their email addresses and discovered that in both cases they were throwaways. This is only possible if we actually have that registration information to begin with.

There are other moderation difficulties around this as well. For instance, would the moderation teams on both sides be the same group? If not what happens when there are disagreements over implementation of policy? (Personally I would rather see that group be centralized.) Right now when you join Slack you have to agree to the Drupal CoC and it applies to all channels within that slack, but joining IRC requires no such global agreement. There may be per-channel agreements or reminders, although I don't remember this from back when I was IRC'ing.

These issues are very real and we are dealing with them every day on Slack. While Slack isn't perfect itself from the moderation perspective (in particular the ability to preview channels without joining them is a real problem) it offers a level of accountability that helps a lot, and I would hate to see that degraded in any way.

eatings’s picture

I also question this move, and find it fundamentally flawed. This issue proposes to add an additional layer of systems complexity in order to bridge a negatively-perceived gap, while not addressing any of the underlying issues that led to the gap in the first place, while also introducing significant additional communication and community management complexity for what gains?

As davidhernandez put it in #13,

Let's not pretend anything but a certain percentage of the community was ever on IRC. We saw large numbers of people jump on Slack precisely because it wasn't IRC.

Slack itself is not without its flaws. IRC is certainly not without its flaws, or else folks wouldn't have moved away from it in as large numbers as we have. The answer is not to now engineer an ugly hybrid that doesn't actually solve actual problems beyond papering over a personal distaste for one communication tool over another.

betz’s picture

We did some tests today about impersonating behaviour, please read from this comment on.

Greg Boggs’s picture

After reading this thread and the other very long thread here's what I have gleaned.

Pros:
1. The community gets access to free software chat applications for Slack.
2. The community can run 1 program to chat all things Drupal.
3. The 15+ Drupal servers can consolidate into a single server.
4. We get logging without paying a million dollars a year.
5. Folks who use Slack and stay on Slack won't experience the difference.
6. Folks uncomfortable promoting closed-software should be more likely to join the chat.

Cons:
1. Spoofing Slack users is a problem we'd like to not make worse. Lets follow up.
2. People want to understand more about how privacy works in Matrix.
3. Adding tech adds complexity. So, it's only worth it if there is significant gain.
4. The setup is labor intensive.
5. Channel names get confusing. We have 15 or so #general channels in Slack.

davidhernandez’s picture

1. The community gets access to free software chat applications for Slack.

Not sure I understand this one. Use of Slack is free. Or are you referring to using open source? You can use an IRC client to connect to Slack. You just have to have a registered account to authenticate with. The bridge bypasses the account part.

3. The 15+ Drupal servers can consolidate into a single server.

Are you referring to the different Slack teams that exist? Like regional ones, Drupal Twig, etc? Those won't go away. They're still all managed independently from each other.

6. Folks uncomfortable promoting closed-software should be more likely to join the chat.

I don't think everyone would necessarily agree with this. Part of the workings is that your communications will still get passed through systems you don't want to promote or use. You couldn't actually boycott Slack. It's arguably worse. You won't have an account, or agree to their terms of service, but your persona and words are transmitted through them and archived by them. And, to my knowledge, you can't opt out of this. You'd have to stop using IRC or at least the channels which are bridged. (Actually, does anyone know if that is possible? To choose just your comms not to go through the bridge, even if you are in a bridged channel.)

freelock’s picture

Ha, great summary @Greg Boggs! You nailed the pros...

Addressing the cons from your list:

1. Matrix/IRC users on the Slack side of the bridge are clearly marked as as "Bot user", but show the Matrix display name/avatar of the Matrix user. Unsure about the IRC users. So spoofing will need to get handled on whichever side of the bridge the users are coming from.

2. In short, all posts are (potentially) forever, and user accountability/controls are similar to IRC -- you can kick or ban, but that's all the controls there are. See below for potential improvements coming...

3. True -- but the Pros you listed all sound very worthwhile to me and many others.

4. Not true -- the setup is quite easy and can be done in minutes for each room to be bridged. It just requires admin approval to create the incoming and outgoing webhooks in Slack (I think once per Slack organization) and then a Matrix room admin to set up the hooks in each room and complete the integration. 5 - 6 clicks with a couple copy/pastes, and the bridge is in place.

5. Not being a Slack user, can't comment on this.

So, for #2, there are 3 new points to make:

a - If abuse proves to be a problem, the integration can easily be severed on either side, by simply removing the webhooks.

b - Matrix/Riot is on the verge of providing "Groups" functionality -- an entirely new set of functionality that supports adding rooms and users to groups, with per-group badges, powerlevels, etc. This functionality is what the Matrix team is working on right now, with release expected within weeks. This might be a great solution for handling abuse -- allowing known users to have more power than anonymous, and setting a minimum power level required for posting. (probably wouldn't be that hard to leverage Drupalbot karma to give people certain badges...)

c - Other reputation systems -- Abuse has been a concern of the Matrix core team since the beginning, though it has not been enough of a problem to warrant developing a solution just yet. It is possible now for users to entirely ignore other users (although only certain clients expose the control for this -- it's on Riot Android, for example, but not on Riot Web). I think groups is the main mechanism the core team is developing for this now, but they are open to other concepts as well.

Cheers,
John

freelock’s picture

@davidhernandez Slack is not free like Drupal is. It's free like Squarespace, Weebly, or other Saas products. Should we quit developing Drupal because those platforms are "free" of cost, at least for a crippled starting user experience? For many of us, using a proprietary software as a service when there's a free/open source alternative is important -- and part of why we develop with Drupal in the first place.

But aside from ideological issues, there are practical issues with Slack -- particularly with scaling, and handling more than 5,000 users -- several other open source projects have left Slack because it became unworkable at scale, and that's a threshold it sounds like we're fast approaching.

(Actually, does anyone know if that is possible? To choose just your comms not to go through the bridge, even if you are in a bridged channel.)

Of course not. Once you write something online, it's entirely out of your control, and your hands. The bridge relays everything it can see in the room. Somebody can take a screenshot if nothing else. Many IRC users use bouncers, and you have no indication that they do. Slack keeps all your messages, but you only have the benefit of seeing the past 10K messages in your room -- why are you letting them have an archive when you get no benefit from it?

Matrix puts it all out on the table -- it's permanently stored on all participating homeservers, and you should assume that it won't go anywhere. Messages may be "redacted", but you cannot be certain that other homeservers didn't keep the previous text. That's the way it is on the Internet, get used to it.

However, in this respect, Matrix has the capability of being far better than Slack or IRC -- if you really care about privacy, you can create an E2E encrypted room, and verify each room participants' devices -- and you can set your client to not encrypt to devices/users you have not verified. If you really want to have a private group conversation with no chance of eavesdropping, Matrix might be the best option out there, beating Signal and Whatsapp in its group functionality. Then not even any server admin can decrypt the messages traversing the servers.

Greg Boggs’s picture

Hi David. Great feedback.

1. By free software I mean: "Free software is software that can be freely used, modified, and redistributed with only one restriction: any redistributed version of the software must be distributed with the original terms of free use, modification, and distribution"

3. By consolidating into a single server, I mean the matrix server where all the Drupal Slack servers and Freenode can be bridged into a single unified server with channels. I've only spent an hour testing things, so I'm still fuzzy on the details.

6. I was referring to the Drupal project being used to promote Slack.

If we did bridge through Matrix, I'd be in favor of not sharing channels.

frob’s picture

What about not bridging. Just straight up moving. Shutting down IRC, shutting down slack, and hosting our own chat.

davidhernandez’s picture

That was being discussed in the "Evaluate whether to replace drupal IRC..." issue. The main issue is whether a self-hosted option meets people's needs and the DA staff can handle the burden of another service, cost- and time-wise.

frob’s picture

@davidhernandez, If we are going off the recommendations from #2490332: Evaluate whether to replace drupal IRC channels with another communication medium then the next course of action is to evaluate (in both performance and security) matrix.org and riot.im as possible partner and then from that evaluation; if they pass and we can use them then use them as a hosted chat service and if not then setup our own matrix infrastructure.

That is if we are going of the resolution that #2490332: Evaluate whether to replace drupal IRC channels with another communication medium came to.

webchick’s picture

I don't think that's so. There's still the flat-out no to a third-party service due to security/privacy concerns. And the self-hosted option has also been eliminated, due to the DA having other priorities and not having budget for it. That's not even touching the usability, data control, and other challenges of bridging.

I think this issue is actually probably won't fix, at least for now.

fuzzy76’s picture

There's still the flat-out no to a third-party service due to security/privacy concerns.

But IRC *is* a third-party service. One which spreads any sensitive info across a plethora of servers within the network, run by a lot of different people.

lpalgarvio’s picture

My request for the Incoming and Outgoing webhooks in Slack to be used to bridge Matrix to Slack on the Drupal Portugal channel has been rejected.

@mlhess denied your request to install this app.
lpalgarvio
for drupal portugal

lpalgarvio’s picture

We (Drupal Portugal) have quit trying to work with Slack.

Things we can't do at the moment:
- Bridge our Slack channel to other services like Matrix, Telegram and IRC
- Add, change or delete app integrations in our Slack channel. Yeah, that's right
- Have administrator or moderator rights in our Slack channel. Yes, it's ridiculous
- Keep or force topic or other settings in our Slack channel. It's not designed for this
- Kick or ban people who abuse us in our Slack channel. Never had we thought we would be this limited
- Archive or delete the channel, because we would need to be server admins and we are not

Very very open, free, transparent, supportive, inclusive, manageable, independent. /irony
Slack is not for a big and complex community as Drupal is.
We are abandoning support for it.

Elijah Lynn’s picture

Just wanted to pop into this thread and say thanks for the initiative getting this project this far. I fully support this effort. Slack has no plans to support open source communities and their closed nature is not a good fit for Drupal. I would like to see our community embrace Matrix, even if the bridges aren't made.

Murz’s picture

Seems the integration process is stopped. Can anybody describe why? We need only get Slack Webhook URL for https://drupal.slack.com/messages/C06GX3K08/ channel, and fill it in Matrix.org room https://matrix.to/#/#drupal:matrix.org via admin of this room.

After doing this - we can gradually move our Drupal community from commercial Slack to opensource Matrix with minimal problems and without lost messages (and merge IRC messages to Matrix+Slack too).

Also Matrix implement communities feature https://medium.com/@RiotChat/communities-aka-groups-are-here-announcing-... so we can create Drupal community, that will merge all Matrix Drupal chats (merged with IRC & Slack) in one place.

If Slack don't want to provide webhooks for Drupal community channels, we can use https://github.com/42wim/matterbridge for link needed rooms (channels).

hestenet’s picture

I know many folks involved in this effort are also working to organize Drupal Dev Days in Lisbon this year, which has taken up much of their time.

Also, from our side over here at the DA, we're behind on getting an Oauth endpoint set up for resolving some of the identity side of things, however we have that planned in upcoming sprints.

e0ipso’s picture

In case anyone missed it Slack is shutting down the IRC/XMPP bridges: https://news.ycombinator.com/item?id=16539857

frob’s picture

I did miss that, thank you e0ipso. All the more reason to keep this going. Slack could change their API and shutdown the hacky invite loophole that allows public rooms to work at any time.

e0ipso’s picture

This also happened recently, which makes me hesitant to bridge data to Slack: https://www.fastcompany.com/40547684/slack-picked-a-weird-time-to-make-i...

We should take this into consideration because users of Matrix/Riot may not be aware that their private messages may be compromised via the brigded Slack integration.

Murz’s picture

Matrix.org have released new version of Slack integration module: https://github.com/matrix-org/matrix-appservice-slack/releases/tag/0.2.0

This version is much more easy to setup: admins of Slack Drupal community only need to install Riot Bridge app into his community, after this any Matrix user will can configure bridge between needed Slack channels and Matrix rooms.

Can any of Slack Drupal community admins do this easy thing?

rodrigoaguilera’s picture

There is a long discussion on this thread on why is not so straightforward to bridge channels and there is other issues apart from the technical ones.

#2490332: Evaluate whether to replace drupal IRC channels with another communication medium

Murz’s picture

@rodrigoaguilera, can you list current active issues, that stops bridging process?
As I understand, main of those are two:

  1. data privacy - most of Slack channels are publicly visible, so privacy are already equal zero, and bridge to Matrix will not change anything. Here https://murznn.slack.com/apps/A1BKR8Y8J-riot-bridge?next_id=0 you can lookup Matrix bridge app permission list, so this app will have access only to public channels and will not have access to private channels, workspace user emails and any other private info, that will not available publicly. Private channels will not be bridged automatically without confirmation from this private channel members, so users will bridge only those channels, that ready to decrease privacy, so here I don't see global privacy decreasing too. New private chats and channels will can be created on Matrix side with enabled E2E encryption, that will increase those rooms privacy to maximum level (greater that in Slack).
  2. banning users - this is harder thing, but can be implemented via special commands to Slack bot, I post a feature request about this here: https://github.com/matrix-org/matrix-appservice-slack/issues/116

If this two issues will be solved - does any other issues exists, that stops Matrix integration process?

rodrigoaguilera’s picture

I can't find a summary of all the issues that stop the Drupal community from bridging. I think is buried among the related issues and endless conversations.

AFAIK the path to using matrix is currently at
#2905534: Setup a dedicated drupal.org dedicated matrix server

The situation today is that you have part of the community on slack, part on https://drupalchat.me/ and I guess some are still on IRC.

There is also a channel there to talk about bridging the different chats if you want to push the issue forward.
https://drupalchat.me/channel/drupal-bridges

Murz’s picture

The situation today is that you have part of the community on slack, part on https://drupalchat.me/ and I guess some are still on IRC.

Today Matrix already can aggregate all those communities into one Drupal supercommunity!

  1. Bridge of all IRC rooms to Matrix on freenode is already done and works well.
  2. Bridge any Slack public room via Slack bridge can be implemented on demand via https://slack.com/apps/A1BKR8Y8J-riot-bridge
  3. Bridge any drupalchat.me (Rocket.Chat) room can be done via https://github.com/matrix-org/matrix-appservice-rocketchat or https://github.com/42wim/matterbridge and in future - via this: https://github.com/matrix-org/matrix-appservice-rocketchat
  4. Russian Drupal community have active Telegram groups, that already successfully bridged to Matrix: Drupal на русском, Drupal для новичков, Drupal русская локализация

As result, we will have one place, that aggregate all Drupal community rooms and full message history (instead of trimmed to 10000 Slack history) in one place, all old users will can continue use his favorite app to chat with community, new members will can join to whole community via any preferred protocol (IRC, Slack, Rocket.Chat, etc).

AFAIK the path to using matrix is currently at #2905534: Setup a dedicated drupal.org dedicated matrix server

Matrix is decentralized network, so we can start bridging to Matrix.org (or any other) server, and when Drupal Matrix server will be ready - do a live migration (mirroring) all rooms to Drupal separate server.

Murz’s picture

Slack developers fixed bug when Riot Bridge Slack app asks private info (email addresses) from workspace users (this was interface bug, not security): https://github.com/matrix-org/matrix-appservice-slack/issues/117#issueco... - really Slack provide only domain part of users email.

So now Drupal Slack workspace admins can not fear that Matrix users will see his private info, and allow Riot Bridge app in Drupal workspace.

Also want to tell, that KDE community totally moved from IRC to Matrix (instead of Slack, Rocket.Chat, Mattermost, etc) - https://dot.kde.org/2019/02/20/kde-adding-matrix-its-im-framework

Murz’s picture

Via configuring Matrix <-> Slack bridge we can also join drupalchat.me Rocket.Chat community to one chat: #2954073: Promote Rocket Chat instance as an alternative to Slack!

Can anybody describe, which active problems now we have, that stops from adding Riot Bridge Slack app app to current Slack instance?

ricardoamaro’s picture

Hello @murz
unfortunately slack admins are not willing to open that to us, thus it's another limitation for Slack.

freelock’s picture

Yes, it's purely a political blocker, not a technical one. We did have some Slack channels bridged at one point, but due to an unfounded fear of impersonation, those got shut down.

I say unfounded because the measures to block these types of users are equivalent or better on Matrix than what's available on Slack...

I think the other concern was they wanted everybody to agree to the contributors' code of conduct before gaining any access...

So as a result, we're stuck on an inferior proprietary platform that only has history visible to people who are not part of our community (e.g. Slack employees) beyond the past week... instead of a free, open platform with unlimited history, the ability to self-host, end-to-end encryption available for those who want to ensure privacy for things like embargoed security releases, etc.

hchonov’s picture

Why don't we start a vote for a Drupal chat requirements including full history access to everyone? From whom do we need approval for a vote in the community?

If we accept such a requirement and Slack admins does not want to accept it, then our community isn't welcome there and we would have to shut down the channels.

Murz’s picture

Want inform, that Matrix have first stable release of Slack bridge: https://matrix.org/blog/2019/10/03/matrix-appservice-slack-bridge-1-0-is... and, I think, Matrix.org Foundation can host the bridge for popular opensource projects (like Drupal) for free on his infrastructure.

frob’s picture

If we accept such a requirement and Slack admins does not want to accept it, then our community isn't welcome there and we would have to shut down the channels.

This is wishful thinking and more than a little hostile.

The current vote is taking place by people using the slack channel. You will *almost* never find me there because I don't support it. But most people do use it -- for most people it is good enough. If the Matrix stuff takes off and Slack becomes deserted (like IRC) then eventually we wont have to worry about Slack at all anyway.

freelock’s picture

I set up my own Matrix <-> Slack bridge over the weekend, and connected most of the Slack workspaces I interact with into Matrix. What a breath of fresh air!

But I still have to use Slack for Drupal... :-( It's particularly offensive when really cool answers come up -- you have to save them out quickly or you'll never see them again...

And recently I was sent to Rocketchat (Drupalchat.me) for #search api questions.

Fragmentation sucks. Slack is alienating a large chunk of our community, and allowing a Matrix bridge would do a lot to mitigate that.

Matrix has been adopted by both Gnome and KDE, and is a leading contender for Mozilla (who is still evaluating alternatives for another week or two). Maybe it's time to re-open this discussion?

Murz’s picture

The current vote is taking place by people using the slack channel. You will *almost* never find me there because I don't support it. But most people do use it -- for most people it is good enough. If the Matrix stuff takes off and Slack becomes deserted (like IRC) then eventually we wont have to worry about Slack at all anyway.

Matrix will not replace current Slack channels, Matrix will bridge them! So Slack users will can continue talking in Slack channels for unlimited time, but his messages will be visible in Matrix channels too, and Slack users will see Matrix user messages too!

So bridging with Matrix will not break any thing for current Slack community! This will only extend Slack community via Matrix users!

For implement this, only one thing, that current Slack admins must do, is to enable Riot bridge app in Slack Drupal workspace!

hestenet’s picture

Thanks for the heads up of the stable release. We're so caught up in getting ready for D9 release that even though we've got some proposals above that might help us solve those identity and private message access concerns, we haven't made much forward momentum. 😓

That said, I'll certainly look at the stable release. May be able to use some time in Amsterdam to crawl back through these comments and update the issue summary with the blockers to a path forward.

andypost’s picture

@hestenet as the bridge is really stable it makes sense to give a try before Amsterdam to make sure that it will not block messaging at the event time

IS is not helpful for explaining current state

Murz’s picture

Even nearest competitor WordPress moving to Matrix! Here is more info about this: https://matrix.org/blog/2020/05/21/welcoming-automattic-to-matrix
Does Drupal have any progress with this task? We will even can merge Rocket.Chat community to Drupal+Slack via https://github.com/exul/matrix-rocketchat bridge.

hestenet’s picture

It seems like there's been a significant increase in adoption among other open source solutions. That gives me much more confidence and makes me want to revisit all of our initial evaluation criteria.

Sadly we haven't made any more progress recently, and with the frantic-pace of last minute support for D9 release I'm not sure we will very quickly - but do want to pick this evaluation up again.

Murz’s picture

Recently even Gitter moves to Matrix backend: https://blog.gitter.im/2020/09/30/gitter-element-acquisition/ also TeamSpeek starts same thing: https://community.teamspeak.com/t/teamspeak-development-status-update/11...

And Matrix developers released new version of Matrix<->Slack bridge, that makes integration seamless! So Slack users will can continue use Slack client to talk with Drupal Matrix community, like before!

So what the reasons for Drupal community to still stay on commercial closed-sourced Slack as main chat app?

Murz’s picture

As first easy step, will be good to simply add Element Bridge (Matrix bridge) bot into Drupal Slack workspace, that will allow bridge needed rooms with Matrix (and via Matrix bridges to Telegram, Skype, WhatsApp, etc) for Drupal community. I fill a new separate issue about this step: #3174730: Allow adding Element Bridge (Matrix bridge) bot to Drupal Slack workspace

Pasqualle’s picture

Do we have an open issue with clear list of chat client requirements, chat client evaluation criteria?
I have listed some in #3017642: Evaluate GitLab + Gitter issue comments, but I would like to close that issue, as the reason does not stand any more..

calbasi’s picture

What I don't understand is WHY people stay at Slack instead of move to Matrix... There is free beer there??

calbasi’s picture

I mean, people belonging to a FLOSS community using a private tool when there are some good open/free alternatives... Just trying to understand ;-)

Murz’s picture

Also want to add, that Slack is a paid service, and Drupal seems now in a Pro plan: https://drupal.slack.com/plans/pro?entry_point=team_menu_plan_info

€6.25 per person, per month, when billed yearly

How much it costs totally for Drupal community?

Alongside the Matrix is a free and opensource project, so you must pay only for hardware, and even this is not required, because rooms are decentralized and you can start Drupal rooms on any free server, and connect other servers to it, and even if starter server will shut down, rooms will continue to work on other servers!

ricardoamaro’s picture

Just a friendly reminder that the Drupal community group in matrix can be subscribed in https://matrix.to/#/+drupal:matrix.org

Murz’s picture

Also Matrix protocol allow to host virtual events like DrupalCon, instead of splitting community between Slack, Hopin and drupalcontributions.opensocial.site (using Big Blue Button)!

And it's very convenient to have a separate room (channel) for each presentation, as long-term continuation of discussion about it.

Here are some of events, hosted via Matrix.org protocol:

jurgenhaas’s picture

@ricardoamaro any chance of turning the community into a space?

Murz’s picture

@betz We already have a decentralized Drupal Matrix Space: https://matrix.to/#/#drupal-space:matrix.org that can aggregate all Drupal related rooms in Matrix, and even bridge (mirror) communities rooms from other messengers like Telegram, Discord, Slack.

But for Drupal Slack workspace the Matrix Bridge (Element bridge) bot is still stays as restricted: https://drupal.slack.com/apps/A1BKR8Y8J-element-bridge ;-(

irinaz’s picture

Status: Active » Postponed

Waiting for SSO with Gitlab

Murz’s picture

Meanwhile Rocket.Chat has implemented native bridging with Matrix servers: https://matrix.org/blog/2022/05/30/welcoming-rocket-chat-to-matrix so now we can merge https://drupalchat.me/ with Matrix too!

Murz’s picture

Waiting for SSO with Gitlab

Matrix Synapse server (and Element matrix client) already support Gitlab SSO integration - here https://app.element.io/#/login you can sign in using Gitlab account.

Also it support Active Directory integration and many other SSO auth ways. If it is not enough - it's not so hard to implement even our own integration with Drupal.org auth method.

Murz’s picture

Status: Postponed » Active

@irinaz SSO with GitLab is implemented! You can see and try it at https://app.element.io/#/login

irinaz’s picture

@murz, that is great news, thanks! We are still in the process of finalizing SSO with Gitlab on the Drupal side, let's follow up in early 2023, it is only few months from now!