strugee.net

Recently some denial-of-service vulnerabilities were discovered in various modules that pump.io indirectly depends on. I've bumped Express and send to pull in patched versions, and I've updated our fork of connect-auth to require a patched version of Connect, too. The remaining vulnerabilities I've confirmed don't affect us.

Because of these version bumps, I've just put out security releases which all administrators are encouraged to upgrade to as soon as possible. A semver-major release (5.0.0) was released within the past 6 months so per our security support policy this means there are three new releases:

1. pump.io 5.0.2 replaces 5.0.0 and is available now on npm
2. pump.io 4.1.3 replaces 4.1.2 and is available now on npm
3. pump.io 4.0.2 will replace 4.0.1 and is currently undergoing automated testing (it'll be on npm shortly) Update: pump.io 4.0.2 is now on npm

As these are security releases we encourage admins to upgrade as soon as possible. If you're on 5.0.0 installed via npm - our recommended configuration - you can upgrade by issuing:

$ npm install -g pump.io@5

If you're on 4.1.3, you can upgrade by issuing:

$ npm install -g pump.io@4

And when 4.0.2 is out, if you're on 4.0.1 you can upgrade by issuing:

$ npm install -g pump.io@4.0

Note though that 4.1.3 is a drop-in replacement for 4.0.2, so you should consider just upgrading to that instead. Or even better, upgrade to 5.x!

If you don't have an npm-based install, you'll have to upgrade however you normally do. How to do this will depend on your particular setup.

As always, if you need help, you should get in touch with the community. I'd also like to specifically thank Jason Self, who generously deployed a 24-hour private beta of these fixes on Datamost. One of the version bumps was ever-so-slightly risky, and being able to test things in production before rolling out patches for the entire network was invaluable. I wouldn't be as confident as I am in these releases without his help. So thanks, Jason - I really appreciate it!

As some of my readers (particularly Recursers) know, a couple of weeks ago I became an Invited Expert at the Social Working Group at the W3C (World Wide Web Consortium). The W3C is a standards body. That means it's responsible for defining things like how things work on the web, such as how web pages are styled using CSS and how web developers can protect their apps from security vulnerabilities using Content Security Policy.

My first thought when I got the email that my application had been accepted was, "WHOOOOOOOOO!" It was probably one of the most thrilling moments of my whole life. My second thought was, "how in the world did I get here!?" The truth is, it was almost an accident.

It started when I got involved in the pump.io project. pump.io, for those who haven't heard me talk about this endlessly (e.g. at RC), is a decentralized social network. That means that there can be multiple servers run by different people that are part of the social network, but the users on those servers can interact with each other in just the same way they could if it was just one big centralized server[1]. I first got involved in the pump.io project in August 2015. I was experimenting with different social networking software and decided to deploy pump.io on my server. When I did I realized that pump... well, it didn't work very well. The web UI was kinda basic[2], everything was pretty buggy, and there were a lot of problems with the overall user experience. In fact, I know the exact day I set up pump.io (August 12th) because all throughout the experience I was filing bugs on things needing improvement. It was a shame, I thought, because this software seemed really neat. I thought it had a lot of potential.

After about two weeks it became clear that there was no activity in the upstream pump.io project. So after some deliberation, I ended up forking it (briefly). You can watch this talk around 16:00 to hear me talk about this a bit, though to be honest it's kind of just a footnote in the project's history. In the end Evan Prodromou, pump.io's author, ended up handing off some commit rights to community members.

Well, I thought, that was the end of that. Everything's smooth sailing from here on out! There were some big problems, though: the people who now had commit rights all were involved in other things and, more importantly, none of them knew JavaScript or Node.js! This makes me chuckle to this day, honestly.

So I started triaging issues. When people sent Pull Requests, I'd review them since it seemed like no one else was going to do it. #1114 was, as far as I can tell (or remember), the very first of these "unofficial" PR reviews. I kept going; I even reviewed Menno Vossen's epic PR which fixed all the tests (fixing the tests being a feat which, having tried to start that work myself, I am to this day in awe of and incredibly thankful for). For that last one in particular, you'll note that I merged it, not Chris Webber. At some point in January(?), he asked me in #pump.io on IRC if I'd like write access to the repository, to which I said (paraphrased) "heck yes!" So he made it happen.

I never really intended for that to happen. However, I was the one doing almost all of the work. After a while it just made sense. This is what, among other things, I find so incredible about freedom-respecting software: you can just do things. I didn't ask anyone for permission to do those reviews. I just saw the need for a reviewer, and decided I'd help out.

Fast-forward to today, and I'm now an owner of the pump.io organization on GitHub. I make technical decisions about what to prioritize and what should go into pump.io core. I do a lot of the day-to-day work running the project, too, and setting up technical and policy infrastructure (with a lot of help from the community, of course, plus input from Evan). That, too, just made sense, as did my becoming an Invited Expert - I was pretty deeply engaged with the SocialWG's ActivityPub specification already since it's based on the pump.io protocol, and I was really excited about said protocol being standardized. So I was participating pretty heavily and I think it just made sense to people in the Working Group for me to join. In fact, that also kinda happened by accident. I couldn't get edit access to the W3C wiki so we were speculating in #social on the W3C IRC server that it might be because I wasn't a "W3C member" or something. So some people at W3C were pinging the sysops team, etc., trying to mark me as a "trusted" user when someone - Sandro Hawke, I believe - said, "the other option is for you to just join the Working Group." To which I said, "well, but I'd have to join as an Invited Expert, and I don't think I qualify as an expert." Chris Webber's response? "You're just as much of an expert as me when I joined!"

tl;dr how in the world did I get here? I tried some software and got annoyed at it, so I just kind of "did some stuff" that led to me doing code reviews. That led to me getting involved in the decentralized social web which led to me "doing some more stuff" that got me involved in standards. Then because of that, I tried to edit a wiki and ended up being invited to apply as a W3C Invited Expert.

I mean, what the hell? Honestly. I can't emphasize enough that I didn't plan ANY of this. It just sort of... happened. And that, I think, is what's so cool about the free software community. It isn't about who you are, where you come from, or what your goals are. It's only about, do you show up? Do you show up and do awesome stuff?

I showed up, kind of by accident, and I now run a decentralized social network with thousands of users called pump.io.

What will happen if you show up?

Thanks so much to Anja and Julia for providing feedback on a draft version of this post.

[1]: I really hope this explanation makes sense and if it doesn't, I apologize - I use diagrams to explain this in real life.

[2]: Still is, but that should improve now that the technical debt work I've been focusing on for the past year is now basically done!

pump.io 4.0.0 is officially in beta! Whooo!

Highlights

This is a positively huge release, and I'm so excited to share it with the community. Some highlights:

• Express 4.x - I wrote about the significance of this change here, but suffice to say that this significantly improves security, performance, and future maintainability
• Performance and correctness improvements to the web UI's JavaScript
• Better administrative experience, including the ability to specify configuration via environment variables
• Better interoperability with the IndieWeb

Upgrading

The upgrade to Express 4.x and the improvements to configuration loading have the potential to break some existing pump.io installations, although 95% of installs should be completely unaffected. If you want to help test this beta, please set aside extra time as necessary to perform this upgrade - full documentation can be found on ReadTheDocs.

As always, this release will follow our normal release cycle, which means that the stable 4.0.0 release will go out in about a month.

Test days

Due to the complexity of this upgrade, we've decided to have some test days during the beta where we upgrade prominent nodes for a day, then downgrade them again. This will help expose problems earlier and make the upgrade smoother for everyone. So far Jason Self, who runs Datamost, has volunteered for this - if you're interested in joining him, please get in touch!

Happy hacking!

For a long while I've really disliked configuration management. This mostly stemmed from my experience managing Apache via Puppet, which I found indirect and unnecessary - the only reason I did this was basically to get version control. In fact, I even started a project called bindslash which I literally described as "not configuration management".

However, last Thursday, steevie (my primary server) crashed again. So I went into a fallback DigitalOcean VM I'd set up the last time this had happened and updated stuff. I presented my LibrePlanet slides from that. And eventually I bit the bullet and set up a secondary email server which, to my great surprise, has not received a flood of spam yet (though I'm sure it will at some point).

The whole ordeal really made me understand the benefit of configuration management. I would've spent less time and been less stressed if I could just plug in a config management system to get a useful failover system. So as of today, I'm on board with configuration management, and bindslash is dead.

I still kinda hate Puppet, so I think I'll try out Ansible and maybe Chef. Ansible's agentless model in particular probably makes a lot of sense for my needs. It also makes me sad to kill bindslash, since I still think it would be a useful project and there's definitely a place for it in the world. But I no longer have any reason to work on it, so I'm just going to stop pretending I'll ever finish it. If anyone is interested in that approach, talk to me and I'll happily give you the name, the repo, my thoughts on its design, etc.

Anyway. Now to set up outbound mail on the failover VM.

*big sigh*

So I thought I'd take a moment to announce that the upgrade from Express 2.x to Express 4.x is finally complete! I fixed up the last couple test failures last Wednesday, and the branch got merged on Thursday.

A long time coming

Believe it or not, the work to do this upgrade started almost an entire year ago. Express 2.x has been outdated and unmaintained for a long time now, so upgrading has been a high priority. However, it wasn't as simple as adjusting a version number - there were a staggering number of changes that needed to be made due to Express deprecating, removing, and changing things around. One of the most significant problems was the fact that the old template system that we used, utml, was not compatible with Express 3.x and above. That meant that we had to rewrite every single template into a modern language - an effort that resulted in over a thousand lines changed!

However, the time for Express 4.x has finally arrived. With that and some other trivial version bumps, I'm proud to announce that pump.io is fully up-to-date in terms of dependencies with only three, non-critical exceptions. Whooooo!

Immediate benefits

There are a lot of reasons this is immediately awesome:

1. Express 4.x fixes significant performance problems that existed in Express 3.x
2. Relatedly, Express 4.x fixes some security problems present in 3.x
3. The fact that our dependencies are finally up-to-date means that we can (and do!) now make use of Greenkeeper and the Node Security Platform to automatically track dependencies to make sure they're up-to-date and not introducing security vulnerabilities

That last one is particularly significant. Greenkeeper and NSP will continuously monitor the project's dependencies and automate away a lot of the pain that's associated with keeping pump.io up-to-date. Everyone will get a more secure and stable codebase because of this setup.

Looking forward

The Express 4.x upgrade is a big change, and it's definitely possible that stuff has broken. We want to make sure that breakage doesn't make it into production. This change went into pump.io 4.0, which will go through our normal release cycle. That means it'll be in beta for a month before being released. As a part of that, Jason Self - who's kind enough to administer Datamost - has agreed to have a test day where Datamost upgrades to the beta for a day, then downgrade it again. This test day will give us much wider exposure than we would've gotten otherwise, which will be incredibly valuable feedback in the effort to identify and fix regressions. We haven't set a date yet, but if you'd like to join Jason in helping us find bugs, please get in touch with the community. We'd love your help.

Beyond the immediate release, though, there's still things to look forward to. Express 4.x gives us a better way to structure routing code, and a refactor to use this structure is planned. There's a lot of room for improvement. But really, the most important benefit is this: technical debt is a far less pressing issue than before. That means that we can shift focus and spend more time fixing user-facing bugs, adding useful features, and generally improving the experience for our users. I couldn't be more excited.