strugee.net

I'm pleased to announce that filter-other-days 1.1.0 and 2.0.0 are now available. In fact, they were both released simultaneously over three weeks ago while I was at SeaGL, but things at college have been so hectic I'm only just finding time to write about them! If you're not already familiar with filter-other-days - which provides reliable, Artificial Ignorance-compatible logfile date filtering even in the face of unknown formats - I would encourage you to read my blog post introducing the tool for the first time. Or, if you read this post late enough, you could even watch video of the talk I gave at SeaGL, which talks about filter-other-days before pivoting into a broader discussion of the kind of runaway complexity filter-other-days is designed to address.

1.1.0 and 2.0.0 are both feature releases. Due to filter-other-days 2.0.0 breaking compatibility with OpenBSD, I'm providing 1.1.0 which contains everything that filter-other-days 2.0.0 does except for the feature that breaks OpenBSD support - localization in the logfile filters. (Localization does not and to my knowledge cannot work on OpenBSD because OpenBSD does not support the POSIX features that filter-other-days' localization relies on.)

Here are the highlights of the engineering that both 1.1.0 and 2.0.0 share:

• filter-other-days -d operates on any day instead of the current date on supported systems
• filter-other-days is portable to OpenBSD, NetBSD, OpenIndiana and OmniOS (i.e. illumos), and Cygwin
• GNU seq is no longer required; the only requirement for core functionality is now POSIX
• Several bugs have been fixed
• Release artifacts are built reproducibly
• Automated testing has been improved

Note that filter-other-days -d does require more than POSIX - it needs a system with either GNU date -d semantics or BSD date -r semantics. This is because POSIX does not provide enough support to implement this feature otherwise. If your system does not support either of these, filter-other-days will simply turn the feature off. You can check if -d is available by looking for it in the help output - it will show up only if the system supports it.

In addition to the above, filter-other-days 2.0.0 also includes support for filtering logfiles in different locales. This means that if your system logs things like month names in languages other than English, filter-other-days will now be able to process these logs! filter-other-days will automatically use the C locale (which is mandated to be available by POSIX) and will additionally use the locale defined by the $LANG environment variable, if set. You can also specify more locales to be loaded by specifiying the -l command line flag. filter-other-days extracts the information it needs using specific keywords in the system locales, which means that if you want filter-other-days to load a particular locale to filter with, you need to have that locale installed.

Unfortunately, there's one more complication: some systems are buggy and do not have keywords that properly conform to POSIX. FreeBSD 12.0 and below as well as NetBSD are known to have these bugs. Since these systems are relatively popular, I am providing patched versions of filter-other-days that will work around these bugs. You can recognize these versions because they have freebsd in the tarball filename. They will also tell you they've been patched for FreeBSD (and NetBSD) in all relevant places, like the version output and the manpage.

So, to summarize what version to use:

• If you're on OpenBSD, use 1.1.0
• If you're on FreeBSD 12.0 or below, or on NetBSD, use 2.0.0 with FreeBSD patches
• Otherwise, use the unpatched 2.0.0 tarballs

I hope these releases of filter-other-days are useful to people! I'm super proud of them and I couldn't be more excited for people to try them out. And as always, feel free to report any bugs you find!

Recently the cross-site-scripting sanitization library that pump.io uses, DOMPurify, published several security advisories for mXSS vulnerabilities affecting browsers based on the Blink rendering engine - you can find the latest one, for example, here. As we've done in the past, the pump.io project is publishing security releases to ensure that everyone is using the latest version of DOMPurify. Per our security support policy, we are providing patches for the current stable release and the previous stable release:

1. pump.io 5.1.2 has been updated to pump.io 5.1.3
2. pump.io 5.0.2 has been updated to pump.io 5.0.3

As these are security releases we encourage administrators to upgrade as soon as possible. Both 5.1.3 and 5.0.3 are drop-in replacements for their predecessors. If you have pump.io 5.1 installed via npm - our recommended configuration - you can upgrade with:

$ npm install -g pump.io@5

If you're on pump.io 5.0, we recommend that you also run the above command to upgrade to 5.1 - it's a drop-in replacement for 5.0. However, if you want to stick with 5.0 for the time being, you can install a patched version with:

$ npm install -g pump.io@5.0

Note that if you have a source-based install, the above commands won't work and you will need to upgrade however you usually do - this will depend on how exactly you have pump.io set up.

If you need help, or if you have questions about these security releases, get in touch with the community.

It's been a (relatively) long time since we've put anything on this blog, and I think it's high time for an update - especially since there are so many exciting things afoot! Not only is pump.io 5.1.1 now on npm, but we have new experimental Docker images! With upstream having already dropped security support, we're also planning to drop support for Node 4 soon.

Let's take these one at a time.

pump.io 5.1.1

Several months ago I landed a patch from contributor Camilo QS fixing a bug in pump.io's session handling in a route serving uploads. This bug made it so that non-public uploads would always return HTTP 403 Unauthorized, even if the user actually was authorized. Clearly, this makes uploads unusable for people who don't like to post everything publicly. Evan suggested that we should backport this bugfix since it's so high-impact, and I agree. So that's what pump.io 5.1.1 contains: a bugfix for uploads. Since it's a patch release 5.1.1 is a drop-in replacement for any 5.x pump.io release, so I'd highly encourage administrators to upgrade as soon as it's convenient. We'd also love it if you file any bugs you find, and feel free to get in touch with the community if you need help or have questions. As a reminder, you can subscribe to our low-volume announce mailing list to get email when we put out new releases like this. Also, I would be remiss if I didn't mention that my signing key setup has changed temporarily - see here if you want to cryptographically verify the 5.1.1 release.

If you're on an npm-based install, you can upgrade with npm install -g pump.io@5.1.1. If you're on a source-based install, you can upgrade by integrating the latest commits in the 5.1.x branch. See here for the changelog.

But that's not all. pump.io 5.1.1 also includes another exciting change: with this release, we've integrated automation to relase pump.io Docker images too.

Docker images

We've wanted to release pump.io Docker images for a long time. But Docker has a well-known problem: security vulnerabilities in Docker Hub images are rampant. Even though we've had a Dockerfile in the repository for a while thanks to contributor thunfisch, we didn't want to release official Docker images if we weren't sure we could always provide security support for them.

Unfortunately, Docker the company has done very little to address this problem. Most of their solutions are aimed at image consumers, not authors. Docker Hub has some capacity for automatically rebuilding images, but unfortunately, it's not enough and you end up having to roll everything yourself anwyay. Pretty disappointing - so we had to get creative.

Our solution to this problem is to utilize Travis CI's cron functionality. Every day, Travis will automatically trigger jobs that do nothing but build pump.io Docker images. These images are then pushed to Docker Hub. If nothing has changed, Docker Hub recognizes that the "new" images are actually identical with what's already there, and nothing happens. But if there has been a change, like a native dependency receiving a security update, then the image ID will change and Docker Hub will accept the updated image. This cronjob is enabled for the 5.1.x branch and master (which as a side effect, means that alpha Docker images are published within 24 hours of a git push), and in the future it will be enabled on all branches that we actively support. Thus, Docker users can easily set up automation to ensure that they're running insecure images for, at most, 24 hours.

If you're interested in trying out the Docker images, we'd love to know how it goes. They should still be treated as experimental at the moment, and early feedback would be super useful. You can read more details in our ReadTheDocs documentation.

Note that there are still more changes that we'd like to make to the Docker images. These changes didn't make it into the 5.1.1 packaging since they felt too invasive for a patch release. Instead we plan to make them in the next release, which is planned to be semver-major. Which brings me neatly to the last topic...

Sunsetting Node 4, 5, and 7 support

We had a good run, but it's time to say goodbye: Node.js upstream has marked Node 4.x as end-of-life, and in accordance with our version policy, we're doing the same. Since this is a semver-major change, we're also taking the opportunity to drop support for Node 5.x and Node 7.x. These changes have been made as of commit 32ad78, and soon we'll be ripping out old code used to support these versions, as well as upgrading dependencies that have recently started requiring newer Nodes.

Anyone still on these versions is encouraged to upgrade as soon as possible, as Node.js upstream is no longer providing security support for them. Administrators can use the NodeSource packages, or they can try out our new Docker images, which use a modern Node version internally.

Please reach out to the community if you need any help making the transition. And good luck!

Last night I officially published pump.io 5.1 to npm as a stable release!

As I wrote in the beta announcement, this release contains a variety of improvements:

• Zero-downtime restarts, which allows administrators to seamlessly roll over to new configurations and codebases
• The daemon now generates startup log warnings on bad configurations, including insecure secret values and internal parameters
• An official Dockerfile is now included with the release
• The logged-out mobile homepage's menu icon is no longer incorrectly styled as black
• An authorization problem with SockJS connections has been fixed

5.1 stable does include one change the beta didn't: a bump to the version of the gm npm package which we depend on. This bump was done as a precautionary measure, as previous versions of gm depended on a version of the debug module which was vulnerable to denial-of-service security bugs.

As a project, we addressed these bugs back in October when we issued security releases for all supported release branches, and at the time we confirmed that the vulnerable function wasn't used by gm. Today's gm bump does not constitute a security release; instead, we're just bumping the version as a precautionary measure in case we missed something in October's assessment of the situation.

Aside from the gm bump, there are (as usual) miscellaneous version bumps included in this release. We've also started tracking test suite coverage information as well as overhauled our documentation on ReadTheDocs, moving most of the in-repository documentation there.

If you want even more details of this release, you can also check out the changelog.

pump 5.1 is a drop-in replacement for 5.0. That means if you're using our recommended installation method and installing from npm, you can upgrade with npm install -g pump.io@5.1. If you have a source-based install, you should merge and/or switch to the v5.1.0 tag. And as always, if you encounter any problems, please feel free to reach out to the community or file bugs you find.

Finally, I would be remiss if I didn't point out that pump.io has a brand-new announcement mailing list! While the blog is great for announcing new releases, not everyone finds it convenient to check. Also, if we issue new betas in the middle of a release cycle, these aren't typically announced on the blog. Therefore in the future all new releases will be announced on the mailing list, not just initial betas. If you want to subscribe to the mailing list, you may do so here - you'll get announcements of new features only, not e.g. feature announcements as seen on this blog. I hope people find this service useful!

This post also appeared on IndieNews.

Last night I hit publish on version 1.0.0 of a new project, lazymention! Whoohoo!

tl;dr: lazymention exists to add Webmention support to static sites!

To elaborate a little bit, I developed lazymention because I had a problem with this site: I wanted to send outbound Webmentions when I link to things, but my website is completely static. (Webmention, in case you didn't know, is a way to notify another website that you linked to them, so the other website can display some UI about your reply or whatever.) The page builds happen on my local machine, not on the server. One option would be to just send Webmentions from my local machine too, but this isn't really a good solution for a couple reasons. First, I couldn't do it automatically at build-time because the built pages wouldn't have been deployed to the server yet, so receivers of my Webmentions would reject the mentions due to the source being nonexistant. That meant that I would have to have a separate step, which wouldn't really be that big of a deal (lazymention requires pinging a server too) except for the second reason: I would need some way to keep track of where I'd already sent Webmentions to, and that would require synchronizing across computers. Probably the only decent way to do that would be to check it into Git, but having a program's data store checked in right next to the source code just feels kinda ugly. Plus, then it can't be shared with other people as a service.

So instead of doing it locally, I elected to build a server instead. Here's how it works: you mark up your stuff with h-feed and h-entry, and whenever anything happens (e.g. you publish a new blog post or whatever), you ping lazymention with the URL (either the feed or the post itself). lazymention will use your microformats2 markup to find the canonical location for a given post, then it will find all the links in the post and send Webmentions for them. And presto! You've just sent Webmentions for your blog. lazymention also records when it's sent mentions, so if you ping it again, nothing will happen unless you've updated your content. I'm also planning to add WebSub support to lazymention, too, and that'll work in the exact same way.

lazymention is super easy to get started with, especially because I've provided thorough documentation in the README. If you find anything that's confusing or missing, please let me know by filing an issue! I'd love to get it fixed. In fact, I'd be thrilled to hear about both positive and negative installation experiences.

Oh, and one more thing - lazymention is reusable in other applications. If you're writing a Node.js app and want to reuse its HTTP API, you can use its embedding API to get at the Express application and Router used internally. I'm not sure if people will actually find this useful, but I wrote it just for kicks anyway. See the embedding documentation for more!

Cheers, and happy mentioning! Elegant outbound Webmention for static sites is here.