strugee.net

I'm pleased to announce that filter-other-days 1.1.0 and 2.0.0 are now available. In fact, they were both released simultaneously over three weeks ago while I was at SeaGL, but things at college have been so hectic I'm only just finding time to write about them! If you're not already familiar with filter-other-days - which provides reliable, Artificial Ignorance-compatible logfile date filtering even in the face of unknown formats - I would encourage you to read my blog post introducing the tool for the first time. Or, if you read this post late enough, you could even watch video of the talk I gave at SeaGL, which talks about filter-other-days before pivoting into a broader discussion of the kind of runaway complexity filter-other-days is designed to address.

1.1.0 and 2.0.0 are both feature releases. Due to filter-other-days 2.0.0 breaking compatibility with OpenBSD, I'm providing 1.1.0 which contains everything that filter-other-days 2.0.0 does except for the feature that breaks OpenBSD support - localization in the logfile filters. (Localization does not and to my knowledge cannot work on OpenBSD because OpenBSD does not support the POSIX features that filter-other-days' localization relies on.)

Here are the highlights of the engineering that both 1.1.0 and 2.0.0 share:

• filter-other-days -d operates on any day instead of the current date on supported systems
• filter-other-days is portable to OpenBSD, NetBSD, OpenIndiana and OmniOS (i.e. illumos), and Cygwin
• GNU seq is no longer required; the only requirement for core functionality is now POSIX
• Several bugs have been fixed
• Release artifacts are built reproducibly
• Automated testing has been improved

Note that filter-other-days -d does require more than POSIX - it needs a system with either GNU date -d semantics or BSD date -r semantics. This is because POSIX does not provide enough support to implement this feature otherwise. If your system does not support either of these, filter-other-days will simply turn the feature off. You can check if -d is available by looking for it in the help output - it will show up only if the system supports it.

In addition to the above, filter-other-days 2.0.0 also includes support for filtering logfiles in different locales. This means that if your system logs things like month names in languages other than English, filter-other-days will now be able to process these logs! filter-other-days will automatically use the C locale (which is mandated to be available by POSIX) and will additionally use the locale defined by the $LANG environment variable, if set. You can also specify more locales to be loaded by specifiying the -l command line flag. filter-other-days extracts the information it needs using specific keywords in the system locales, which means that if you want filter-other-days to load a particular locale to filter with, you need to have that locale installed.

Unfortunately, there's one more complication: some systems are buggy and do not have keywords that properly conform to POSIX. FreeBSD 12.0 and below as well as NetBSD are known to have these bugs. Since these systems are relatively popular, I am providing patched versions of filter-other-days that will work around these bugs. You can recognize these versions because they have freebsd in the tarball filename. They will also tell you they've been patched for FreeBSD (and NetBSD) in all relevant places, like the version output and the manpage.

So, to summarize what version to use:

• If you're on OpenBSD, use 1.1.0
• If you're on FreeBSD 12.0 or below, or on NetBSD, use 2.0.0 with FreeBSD patches
• Otherwise, use the unpatched 2.0.0 tarballs

I hope these releases of filter-other-days are useful to people! I'm super proud of them and I couldn't be more excited for people to try them out. And as always, feel free to report any bugs you find!

Recently the cross-site-scripting sanitization library that pump.io uses, DOMPurify, published several security advisories for mXSS vulnerabilities affecting browsers based on the Blink rendering engine - you can find the latest one, for example, here. As we've done in the past, the pump.io project is publishing security releases to ensure that everyone is using the latest version of DOMPurify. Per our security support policy, we are providing patches for the current stable release and the previous stable release:

1. pump.io 5.1.2 has been updated to pump.io 5.1.3
2. pump.io 5.0.2 has been updated to pump.io 5.0.3

As these are security releases we encourage administrators to upgrade as soon as possible. Both 5.1.3 and 5.0.3 are drop-in replacements for their predecessors. If you have pump.io 5.1 installed via npm - our recommended configuration - you can upgrade with:

$ npm install -g pump.io@5

If you're on pump.io 5.0, we recommend that you also run the above command to upgrade to 5.1 - it's a drop-in replacement for 5.0. However, if you want to stick with 5.0 for the time being, you can install a patched version with:

$ npm install -g pump.io@5.0

Note that if you have a source-based install, the above commands won't work and you will need to upgrade however you usually do - this will depend on how exactly you have pump.io set up.

If you need help, or if you have questions about these security releases, get in touch with the community.

A month or two ago, my friend Evan tweeted:

Fuck reCaptcha.

I am sick of doing unpaid labour classifying images for Google.

We need a captcha widget that contributes to the global commons instead of siphoning value into yet another proprietary lockbox.

Frankly I agree. Not only am I being forced to do Google's dirty work, but ReCaptcha is known to make life extremely difficult for Tor users. I've literally spent 15 minutes before trying to solve a stupid captcha and eventually I gave up because Google just wouldn't let me past. ReCaptcha profits off of innocent users who are just trying to go about their business on the web, and Google is exploiting people in order to build a locked-up, proprietary image recognition system. Why are we, the users, not allowed to have access to the fruits of the labor that we are forced to provide for free? Because of this, today I am announcing an advanced, next-generation, cutting-edge platform that is poised to revolutionize this problem space.

Nah, just kidding (mostly). Inspired by another tweet from Evan, I threw together a browser extension in like 30 minutes that changes "I'm not a robot" to "I want to do unpaid image classification". After a long exchange with the fine folks behind addons.mozilla.org (because my account is so old that when I logged in I hit this bug and got 500 Internal Server Error), I've finally sorted out my logins, and today I've uploaded the latest version of this extension to both addons.mozilla.org and the Chrome Web Store. So at least if you're getting screwed by Google, you'll be able to make them be honest about how they're screwing you. Here's a screenshot:

Also, for some extra irony, I took the icon from Google's Apache 2.0-licensed Material Design icon set. If someone feels like contributing a real icon, I would probably replace the current icon with it. (I would also be happy to take translations into different languages!)

In the words of the README:

This was Evan Prodromou's idea unless you like it, in which case the idea to follow through and make an actual extension was totally all mine.

I hope people enjoy this extension! Make ReCaptcha's "I'm not a robot" accurate is available as a Firefox extension and as a Chrome extension, and the source is published on GitHub.

It's been a (relatively) long time since we've put anything on this blog, and I think it's high time for an update - especially since there are so many exciting things afoot! Not only is pump.io 5.1.1 now on npm, but we have new experimental Docker images! With upstream having already dropped security support, we're also planning to drop support for Node 4 soon.

Let's take these one at a time.

pump.io 5.1.1

Several months ago I landed a patch from contributor Camilo QS fixing a bug in pump.io's session handling in a route serving uploads. This bug made it so that non-public uploads would always return HTTP 403 Unauthorized, even if the user actually was authorized. Clearly, this makes uploads unusable for people who don't like to post everything publicly. Evan suggested that we should backport this bugfix since it's so high-impact, and I agree. So that's what pump.io 5.1.1 contains: a bugfix for uploads. Since it's a patch release 5.1.1 is a drop-in replacement for any 5.x pump.io release, so I'd highly encourage administrators to upgrade as soon as it's convenient. We'd also love it if you file any bugs you find, and feel free to get in touch with the community if you need help or have questions. As a reminder, you can subscribe to our low-volume announce mailing list to get email when we put out new releases like this. Also, I would be remiss if I didn't mention that my signing key setup has changed temporarily - see here if you want to cryptographically verify the 5.1.1 release.

If you're on an npm-based install, you can upgrade with npm install -g pump.io@5.1.1. If you're on a source-based install, you can upgrade by integrating the latest commits in the 5.1.x branch. See here for the changelog.

But that's not all. pump.io 5.1.1 also includes another exciting change: with this release, we've integrated automation to relase pump.io Docker images too.

Docker images

We've wanted to release pump.io Docker images for a long time. But Docker has a well-known problem: security vulnerabilities in Docker Hub images are rampant. Even though we've had a Dockerfile in the repository for a while thanks to contributor thunfisch, we didn't want to release official Docker images if we weren't sure we could always provide security support for them.

Unfortunately, Docker the company has done very little to address this problem. Most of their solutions are aimed at image consumers, not authors. Docker Hub has some capacity for automatically rebuilding images, but unfortunately, it's not enough and you end up having to roll everything yourself anwyay. Pretty disappointing - so we had to get creative.

Our solution to this problem is to utilize Travis CI's cron functionality. Every day, Travis will automatically trigger jobs that do nothing but build pump.io Docker images. These images are then pushed to Docker Hub. If nothing has changed, Docker Hub recognizes that the "new" images are actually identical with what's already there, and nothing happens. But if there has been a change, like a native dependency receiving a security update, then the image ID will change and Docker Hub will accept the updated image. This cronjob is enabled for the 5.1.x branch and master (which as a side effect, means that alpha Docker images are published within 24 hours of a git push), and in the future it will be enabled on all branches that we actively support. Thus, Docker users can easily set up automation to ensure that they're running insecure images for, at most, 24 hours.

If you're interested in trying out the Docker images, we'd love to know how it goes. They should still be treated as experimental at the moment, and early feedback would be super useful. You can read more details in our ReadTheDocs documentation.

Note that there are still more changes that we'd like to make to the Docker images. These changes didn't make it into the 5.1.1 packaging since they felt too invasive for a patch release. Instead we plan to make them in the next release, which is planned to be semver-major. Which brings me neatly to the last topic...

Sunsetting Node 4, 5, and 7 support

We had a good run, but it's time to say goodbye: Node.js upstream has marked Node 4.x as end-of-life, and in accordance with our version policy, we're doing the same. Since this is a semver-major change, we're also taking the opportunity to drop support for Node 5.x and Node 7.x. These changes have been made as of commit 32ad78, and soon we'll be ripping out old code used to support these versions, as well as upgrading dependencies that have recently started requiring newer Nodes.

Anyone still on these versions is encouraged to upgrade as soon as possible, as Node.js upstream is no longer providing security support for them. Administrators can use the NodeSource packages, or they can try out our new Docker images, which use a modern Node version internally.

Please reach out to the community if you need any help making the transition. And good luck!

Tonight I'll have been back at college for a full week, and I wanted to write up a little retrospective of winter break to see what I accomplished (and in particular, which goals I completed or missed).

You may wish to skip directly to the executive summary.

Resolved goal: Node.js manpage PR

I didn't complete this goal per se, but I did at least resolve it by closing the Pull Request. I felt pretty bad about it (especially because I kept saying I intended to finish it) but honestly, it became clear to me that I'd just lost the motivation to keep going with it. I would love it if this was included in Node.js core, but I just consistently have higher priorities. So rather than leave it hanging and cluttering up the Pull Requests view, I just closed it to reflect reality. I made sure not to delete the branch though, in case someone (distant future me?) wants to pick it up again.

Failed goal: deal with GPG keysigning

I did nothing to push this goal forward. While I made numerous improvements to my GPG setup, I did not actually sign anyone's key, which was what this goal was about. This feels unfortunate. (I also do not have access to the private key material in college, and am certainly not about to ask that it be shipped to me.)

Partially completed goal: push Debian packaging work forward

There were two components to this: Profanity packaging upgrades and getting the new filter-other-days packaging into Debian. I made no progress on the Profanity packaging. However, I did fix a misconfiguration in my .reportbugrc (which annoyingly had previously sent my incredibly detailed email about Profanity packaging to /dev/null) and then submitted an ITP (Intent To Package bug, which is a Debian thing) for filter-other-days. I then used that ITP bug number to fix the last .deb Lintian warning (although see below). Then I paired with Anja who is, as always, an angel, and we figured out the weirdness that is dput and mentors.debian.net. Finally I was able to upload filter-other-days(!) BUT I was in for a rude awakening - apparently Lintian checks for .debs and .dscs are different. So while my binary package was Lintian-clean, my source package unfortunately wasn't. This is something I will need to work on in the weeks to come. That being said, I'm still pretty proud of what I've accomplished here! I've made significant progress on this front.

Completed goal: lazymention v1

One of the first things I did was ship lazymention 1.0.0 - and I wrote an announcement blog post to accompany it! (In fact, I syndicated that blog post to IndieNews using lazymention, which felt pretty damn awesome.) I got some great feedback on it from the IndieWeb community, and my lobste.rs submission - which also got some great engagement - even made the front page, which was pretty unreal! I still have a lot more work to do with lazymention - in particular, it doesn't actually respect edits (so it'll resend Webmentions with every job submission) - but for now it works well. I'm super pleased with it, and have integrated it into my site deployment workflow. I even wrote a module so other people can do this, too!

Failed goals: ActivityPub in pump.io core, pump.io PR review

No progress on this front. I did start hacking on a telemetry server which will eventually be helpful for our ActivityPub rollout, but that did not in any way directly help fulfill these goals. I also released 5.1 stable, but that's pretty routine by this point.

Partially completed goal: two blog posts per week

I stuck with this goal all the way up until the final week, where I didn't write any. (Although I wrote about my GPG keys around the time I actually flew back to college.) The first week, I wrote about my thoughts on shell script and about lazymention; the second, I wrote about the pump.io 5.1 stable release and about talking to Pull Request reviewers if you think they're wrong.

Failed stretch goal: paper editing

I did absolutely no editing on the paper I intend to get published (which I originally wrote for a writing class). This was a stretch goal though, so that's totally fine.

Additional activity: steevie maintenance

After I finally found the cable I needed, I swapped out the cable that connects steevie's motherboard with the drives' SATA ports. This seemed to significantly improve disk performance, although there are still noticeable performance problems. I'm very, very happy to have finally gotten this done.

Additional activity: Tor relay migration from Amazon EC2 to DigitalOcean

After getting some advice on tor-relays, I finally sat down and looked into moving my relay away from Amazon Web Services. This is because AWS bills by usage, which for a Tor relay ends up being incredibly inefficient. It turned out to actually be way easier than I thought, which only served to make me mad that I hadn't done it sooner. In any case, I now save approximately $240/year AND I can push 1000GB/month as opposed to the 10GB/month I pushed before. In the words of the commit where I made this change: "this change made possible by the fact that I'm no longer getting billed up the wazoo by Amazon Web Services." Here's a of a Tor Metrics graph (captured today) that shows the jump:

Anyway, I'm super happy I can contribute more to the Tor network and save lots of money in the process. That being said I am pretty damn salty I didn't realize this in the four years I've been running a Tor relay.

Additional activity: offandonagain.org maintenance

After turning on NodeSecurity monitoring for offandonagain.org, I found out that the module that underlies it, node-serve-random, had some vulnerable dependencies. Not only did I fix this, I wrote a large test suite for the module and found (and fixed!) several bugs in the process. Writing a test suite also allowed me to turn on Greenkeeper for the module, which will be a huge help to me in keeping the module up-to-date.

Additional activity: Stratic work

First off, I released beta 7 of generator-stratic! Nothing major, just some polishing stuff. Stratic is getting very close to the point where I might want to start promoting it more aggressively, or declaring it stable, and (with a lot of super-helpful feedback from my family) I worked on something that's super important before we get there: a logo!

Here are two of my favorites so far:

These are based off the JS logo, in case you hadn't seen it before:

Anyway, I have to post another iteration in the GitHub issue based on some feedback from Saul (who I had a lovely lunch with) - he thinks I should reverse it so the pipe is on the left, so it looks like the file is coming out of the pipe. But anyway you should comment there if you have feedback!

Additional activity: IndieWeb stuff

I attended Homebrew Website Club in San Fransisco, which was incredible. I got to meet a bunch of new people, as well as say hi to Ryan and Tantek again, which was so nice - it's always just better to talk in real life. Tantek said (at least if I recall correctly) that my laptop was one of the best-stickered laptops he'd ever seen, which made me feel just unbelievably special. He also gave me a microformats sticker (and helped me figure out where to put it), which I had on my old laptop and had been really missing, as well as a Let's Encrypt sticker. The latter was so special I elected to put it on the inside of my laptop, which I reserve only for really special things (basically a Recurse Center refucktoring sticker and a sticker of Noah in this video, which he handed to me like a business card the first time we met). Anyway, every time I look at the Let's Encrypt sticker I just feel so happy. I love Let's Encrypt so damn much.

Homebrew Website Club was super inspiring, so when I got back to where I was staying (at my mom's boyfriend's house) I started implementing an IndieWeb-style social stream for strugee.net. It still needs some polishing but is actually pretty close to being done. Who knows when I'll have time to finish it, but it's getting there! I'm so freaking excited, honestly. Also, I added proper timestamp mf2 metadata to my site, as well as a visual display for post edits, and I expanded what type of Webmentions the display code recognizes too!

Executive summary

I resolved or completed 2 goals, partially completed 2 goals, failed 3 goals, and failed 1 stretch goal. Additionally I did significant work in 5 other areas. Out of the goals I set for myself, I completed 51% (Debian packaging work is ~2/5 complete; blog posts were written 2/3 of the time); not counting the stretch goal, I completed 61.2%. I'm pretty happy with what I got done during this period; however, while I was productive, the numbers show that I did a mediocre job sticking to my goals. In the future I should focus on making more realistic goals and then sticking to them (though not too much - it is a break, after all!).

Speaking of which, partway through break I felt like I was on the edge of burnout, which to me was a very clear sign that I was pushing myself way too hard during a time I should have been unwinding. Because of that I cut what I was doing a lot, which helped pretty dramatically. In fact, I think without that I wouldn't have been able to do some of the later stuff, like all the IndieWeb work. So that's another reason I have to find a way to balance sticking to goals and just relaxing (which doesn't necessarily mean not coding, just doing whatever coding I feel like in the moment) - I feel like I was pushing myself too hard to meet my goals (and then getting distracted and not meeting them) and that's what led to the feeling. Obviously there are different constraints for e.g. schoolwork; here I'm referring only to free time like breaks.

Spring semester goals

With that in mind, I want to set some very broad goals for spring semester. I may update this list as time goes on, but for now I have four overarching goals I want to accomplish (besides the usual day-to-day code maintenance stuff):

• Finish editing the paper I wrote last semester on freedom-respecting software and intersectionality, and get it published
• Make some measurable progress on my Push-based Two-factor Authentication IETF draft
• Get access to the University of Rochester darkroom and start developing/printing photos again
• Start pushing the University of Rochester library (and maybe the journalism department?) to start adopting Tor technologies

I'm excited to see how it goes!