strugee.net

Last night I officially published pump.io 5.1 to npm as a stable release!

As I wrote in the beta announcement, this release contains a variety of improvements:

• Zero-downtime restarts, which allows administrators to seamlessly roll over to new configurations and codebases
• The daemon now generates startup log warnings on bad configurations, including insecure secret values and internal parameters
• An official Dockerfile is now included with the release
• The logged-out mobile homepage's menu icon is no longer incorrectly styled as black
• An authorization problem with SockJS connections has been fixed

5.1 stable does include one change the beta didn't: a bump to the version of the gm npm package which we depend on. This bump was done as a precautionary measure, as previous versions of gm depended on a version of the debug module which was vulnerable to denial-of-service security bugs.

As a project, we addressed these bugs back in October when we issued security releases for all supported release branches, and at the time we confirmed that the vulnerable function wasn't used by gm. Today's gm bump does not constitute a security release; instead, we're just bumping the version as a precautionary measure in case we missed something in October's assessment of the situation.

Aside from the gm bump, there are (as usual) miscellaneous version bumps included in this release. We've also started tracking test suite coverage information as well as overhauled our documentation on ReadTheDocs, moving most of the in-repository documentation there.

If you want even more details of this release, you can also check out the changelog.

pump 5.1 is a drop-in replacement for 5.0. That means if you're using our recommended installation method and installing from npm, you can upgrade with npm install -g pump.io@5.1. If you have a source-based install, you should merge and/or switch to the v5.1.0 tag. And as always, if you encounter any problems, please feel free to reach out to the community or file bugs you find.

Finally, I would be remiss if I didn't point out that pump.io has a brand-new announcement mailing list! While the blog is great for announcing new releases, not everyone finds it convenient to check. Also, if we issue new betas in the middle of a release cycle, these aren't typically announced on the blog. Therefore in the future all new releases will be announced on the mailing list, not just initial betas. If you want to subscribe to the mailing list, you may do so here - you'll get announcements of new features only, not e.g. feature announcements as seen on this blog. I hope people find this service useful!

This post also appeared on IndieNews.

Last night I hit publish on version 1.0.0 of a new project, lazymention! Whoohoo!

tl;dr: lazymention exists to add Webmention support to static sites!

To elaborate a little bit, I developed lazymention because I had a problem with this site: I wanted to send outbound Webmentions when I link to things, but my website is completely static. (Webmention, in case you didn't know, is a way to notify another website that you linked to them, so the other website can display some UI about your reply or whatever.) The page builds happen on my local machine, not on the server. One option would be to just send Webmentions from my local machine too, but this isn't really a good solution for a couple reasons. First, I couldn't do it automatically at build-time because the built pages wouldn't have been deployed to the server yet, so receivers of my Webmentions would reject the mentions due to the source being nonexistant. That meant that I would have to have a separate step, which wouldn't really be that big of a deal (lazymention requires pinging a server too) except for the second reason: I would need some way to keep track of where I'd already sent Webmentions to, and that would require synchronizing across computers. Probably the only decent way to do that would be to check it into Git, but having a program's data store checked in right next to the source code just feels kinda ugly. Plus, then it can't be shared with other people as a service.

So instead of doing it locally, I elected to build a server instead. Here's how it works: you mark up your stuff with h-feed and h-entry, and whenever anything happens (e.g. you publish a new blog post or whatever), you ping lazymention with the URL (either the feed or the post itself). lazymention will use your microformats2 markup to find the canonical location for a given post, then it will find all the links in the post and send Webmentions for them. And presto! You've just sent Webmentions for your blog. lazymention also records when it's sent mentions, so if you ping it again, nothing will happen unless you've updated your content. I'm also planning to add WebSub support to lazymention, too, and that'll work in the exact same way.

lazymention is super easy to get started with, especially because I've provided thorough documentation in the README. If you find anything that's confusing or missing, please let me know by filing an issue! I'd love to get it fixed. In fact, I'd be thrilled to hear about both positive and negative installation experiences.

Oh, and one more thing - lazymention is reusable in other applications. If you're writing a Node.js app and want to reuse its HTTP API, you can use its embedding API to get at the Express application and Router used internally. I'm not sure if people will actually find this useful, but I wrote it just for kicks anyway. See the embedding documentation for more!

Cheers, and happy mentioning! Elegant outbound Webmention for static sites is here.

Once again I need to test whether lazymention - and more importantly, the underlying libraries (node-send-webmention and node-get-webmention-url) - complies with the W3C spec. I'll be using webmention.rocks, so, here's a large list of links:

Test 1, Test 2, Test 3, Test 4, Test 5, Test 6, Test 7, Test 8, Test 9, Test 10, Test 11, Test 12, Test 13, Test 14, Test 15, Test 16, Test 17, Test 18, Test 19, Test 20, Test 21, Test 22, Test 23

Hopefully it works \o/

I'm pleased to announce filter-other-days 1.0.1. This is a bugfix release primarily improving portability to other Unix-like operating systems; in particular, the test suite now fully passes under FreeBSD. Specifically:

• Various portability bugs in the test suite itself were fixed - the test suite no longer relies on a GNU date (with GNU date -d semantics) or a fully-functional /dev/fd (the fallback option is named pipes), and it doesn't hardcode bash's install path as /bin/bash
• Some non-portable uses of echo "\n" which break under BSD systems were replaced with printf invocations
• Travis CI now checks filter-other-days with Debian's checkbashisms script, which is run in strict mode
• Non-portable uses of test's -o option were caught by checkbashisms and replaced with ||

With these changes I expect that filter-other-days will probably run on all major BSD distributions. I intend to confirm this hypothesis soon and have filed bugs for OpenBSD and NetBSD, plus illumos just for kicks.

As with 1.0.0, you can clone filter-other-days from GitHub or you can download a (signed) tarball. Please do report any bugs you find in the release.

Enjoy!

I've just published version 1.0 of my latest project, filter-other-days - a shell script to filter logfiles for today's date in an Artificial Ignorance-compatible way.

If you haven't heard of Artificial Ignorance, it's something you should look into cause it's pretty awesome. Here's the tl;dr: it doesn't make sense to look for all the "interesting" things in logfiles, because it's not actually possible to enumerate all the failure conditions of a system. So instead what we do is throw away entries that we're sure are just routine. Since we've gotten rid of all the uninteresting entries, whatever is left has to be interesting.

I find this pretty compelling, and decided to start implementing it on my Tor relay. I quickly realized that my ideal workflow would be to configure cron to send me email with a daily report of interesting log entries. However, this presented a problem: how to get just today's log entries? I wanted to be able to handle all logfiles at once instead of receiving different reports for different logs, so I had to be able to parse all logfiles in the same way. My relay runs on FreeBSD, so the logs are unstructured text files, and even worse, several daemons (like Tor itself) write timestamps in a different format - this makes parsing all logfiles at once super difficult because I couldn't just trivially grep for today's date since that would end up dropping legitimate entries from logfiles that formatted their timestamps differently.

I briefly considered trying to write a regex to match all sorts of different timestamp formats, but quickly rejected this idea as too fragile. There are a lot of moving parts in a modern operating system - what if e.g. a daemon changed its defaults about how to format timestamps? Or, more likely, what if I simply missed a particular format present in my logs? Then I'd be accidentally throwing away an entire logfile. To solve this problem, I decided to apply the same idea behind Artificial Ignorance - if I couldn't reliably, 100% match log entries from today's date, I could do the next best thing and attempt to discard all entries from other dates. In this case the worst that could happen is me recieving irrelevant information, and I'd be basically guaranteed to never miss an legitimate entry from today.

filter-other-days is a working implementation of this design. Originally I put it with the other random scripts I keep with my dotfiles, but it quickly became obvious that it was useful as a standalone project. So I extracted it into its own repository, which now lives on GitHub. From there I continued to improve the script while adding a test suite and writing extensive documentation (including a Unix manpage - I always feel like a wizardly hacker when writing those things). This took, by my estimation, somewhere between 10 and 15 hours because this is actually a shockingly non-trivial problem, but mostly because regexes are hard.

But today I finally finished! So I'm super excited to announce that version 1.0 of filter-other-days is now available. You can either clone it from GitHub or download a tarball (and the accompanying signature, if you want). It works pretty well already, but I have some ideas for future directions the project could go:

1. Logic allowing you to actually specify the date you want to filter for, instead of assuming it's today (though you actually can already get this behavior using faketime; that's what the test suite does)
2. Removal of the dependency on GNU seq - this is, to my knowledge, the only non-POSIX requirement of filter-other-days
3. Debian package, maybe?
4. More log formats (please report bugs if you have formats filter-other-days doesn't recognize - which you probably do!)

If you find this project useful, let me know! I'd love to hear about how people are using it. Or if it breaks (or doesn't fill your usecases), please report bugs or send patches - I love those, too! Either way, may the logs be with you!