strugee.net

Blog

Show only posts from 2014 2015 2016 2017 2018 2019 or categorized in personal, development, politics and explanations

filter-other-days 1.1.0 and 2.0.0 are now available

I'm pleased to announce that filter-other-days 1.1.0 and 2.0.0 are now available. In fact, they were both released simultaneously over three weeks ago while I was at SeaGL, but things at college have been so hectic I'm only just finding time to write about them! If you're not already familiar with filter-other-days - which provides reliable, Artificial Ignorance-compatible logfile date filtering even in the face of unknown formats - I would encourage you to read my blog post introducing the tool for the first time. Or, if you read this post late enough, you could even watch video of the talk I gave at SeaGL, which talks about filter-other-days before pivoting into a broader discussion of the kind of runaway complexity filter-other-days is designed to address.

1.1.0 and 2.0.0 are both feature releases. Due to filter-other-days 2.0.0 breaking compatibility with OpenBSD, I'm providing 1.1.0 which contains everything that filter-other-days 2.0.0 does except for the feature that breaks OpenBSD support - localization in the logfile filters. (Localization does not and to my knowledge cannot work on OpenBSD because OpenBSD does not support the POSIX features that filter-other-days' localization relies on.)

Here are the highlights of the engineering that both 1.1.0 and 2.0.0 share:

  • filter-other-days -d operates on any day instead of the current date on supported systems
  • filter-other-days is portable to OpenBSD, NetBSD, OpenIndiana and OmniOS (i.e. illumos), and Cygwin
  • GNU seq is no longer required; the only requirement for core functionality is now POSIX
  • Several bugs have been fixed
  • Release artifacts are built reproducibly
  • Automated testing has been improved

Note that filter-other-days -d does require more than POSIX - it needs a system with either GNU date -d semantics or BSD date -r semantics. This is because POSIX does not provide enough support to implement this feature otherwise. If your system does not support either of these, filter-other-days will simply turn the feature off. You can check if -d is available by looking for it in the help output - it will show up only if the system supports it.

In addition to the above, filter-other-days 2.0.0 also includes support for filtering logfiles in different locales. This means that if your system logs things like month names in languages other than English, filter-other-days will now be able to process these logs! filter-other-days will automatically use the C locale (which is mandated to be available by POSIX) and will additionally use the locale defined by the $LANG environment variable, if set. You can also specify more locales to be loaded by specifiying the -l command line flag. filter-other-days extracts the information it needs using specific keywords in the system locales, which means that if you want filter-other-days to load a particular locale to filter with, you need to have that locale installed.

Unfortunately, there's one more complication: some systems are buggy and do not have keywords that properly conform to POSIX. FreeBSD 12.0 and below as well as NetBSD are known to have these bugs. Since these systems are relatively popular, I am providing patched versions of filter-other-days that will work around these bugs. You can recognize these versions because they have freebsd in the tarball filename. They will also tell you they've been patched for FreeBSD (and NetBSD) in all relevant places, like the version output and the manpage.

So, to summarize what version to use:

  • If you're on OpenBSD, use 1.1.0
  • If you're on FreeBSD 12.0 or below, or on NetBSD, use 2.0.0 with FreeBSD patches
  • Otherwise, use the unpatched 2.0.0 tarballs

I hope these releases of filter-other-days are useful to people! I'm super proud of them and I couldn't be more excited for people to try them out. And as always, feel free to report any bugs you find!


pump.io DOMPurify security fixes available

Recently the cross-site-scripting sanitization library that pump.io uses, DOMPurify, published several security advisories for mXSS vulnerabilities affecting browsers based on the Blink rendering engine - you can find the latest one, for example, here. As we've done in the past, the pump.io project is publishing security releases to ensure that everyone is using the latest version of DOMPurify. Per our security support policy, we are providing patches for the current stable release and the previous stable release:

  1. pump.io 5.1.2 has been updated to pump.io 5.1.3
  2. pump.io 5.0.2 has been updated to pump.io 5.0.3

As these are security releases we encourage administrators to upgrade as soon as possible. Both 5.1.3 and 5.0.3 are drop-in replacements for their predecessors. If you have pump.io 5.1 installed via npm - our recommended configuration - you can upgrade with:

$ npm install -g pump.io@5

If you're on pump.io 5.0, we recommend that you also run the above command to upgrade to 5.1 - it's a drop-in replacement for 5.0. However, if you want to stick with 5.0 for the time being, you can install a patched version with:

$ npm install -g pump.io@5.0

Note that if you have a source-based install, the above commands won't work and you will need to upgrade however you usually do - this will depend on how exactly you have pump.io set up.

If you need help, or if you have questions about these security releases, get in touch with the community.


Make ReCaptcha's "I'm not a robot" accurate

A month or two ago, my friend Evan tweeted:

Fuck reCaptcha.

I am sick of doing unpaid labour classifying images for Google.

We need a captcha widget that contributes to the global commons instead of siphoning value into yet another proprietary lockbox.

Frankly I agree. Not only am I being forced to do Google's dirty work, but ReCaptcha is known to make life extremely difficult for Tor users. I've literally spent 15 minutes before trying to solve a stupid captcha and eventually I gave up because Google just wouldn't let me past. ReCaptcha profits off of innocent users who are just trying to go about their business on the web, and Google is exploiting people in order to build a locked-up, proprietary image recognition system. Why are we, the users, not allowed to have access to the fruits of the labor that we are forced to provide for free? Because of this, today I am announcing an advanced, next-generation, cutting-edge platform that is poised to revolutionize this problem space.

Nah, just kidding (mostly). Inspired by another tweet from Evan, I threw together a browser extension in like 30 minutes that changes "I'm not a robot" to "I want to do unpaid image classification". After a long exchange with the fine folks behind addons.mozilla.org (because my account is so old that when I logged in I hit this bug and got 500 Internal Server Error), I've finally sorted out my logins, and today I've uploaded the latest version of this extension to both addons.mozilla.org and the Chrome Web Store. So at least if you're getting screwed by Google, you'll be able to make them be honest about how they're screwing you. Here's a screenshot:

Screenshot of a test page with Google ReCaptcha on it; "I'm not a robot" has been replaced with "I would like to do unpaid image classification"

Also, for some extra irony, I took the icon from Google's Apache 2.0-licensed Material Design icon set. If someone feels like contributing a real icon, I would probably replace the current icon with it. (I would also be happy to take translations into different languages!)

In the words of the README:

This was Evan Prodromou's idea unless you like it, in which case the idea to follow through and make an actual extension was totally all mine.

I hope people enjoy this extension! Make ReCaptcha's "I'm not a robot" accurate is available as a Firefox extension and as a Chrome extension, and the source is published on GitHub.


Legacy

Recently I turned 20 years old. As the days continue to go by, I have been struggling with the concept of time.

Probably this feeling started with me seeing photos of this year's SAAS robotics teams. I was on the robotics team for four years (I've written about it a little before) but recently I found that all of a sudden, I know barely anyone in the photos. Fifteen people in robotics graduated[1] high school last year, people I've largely known since they were freshmen (or in the case of Jesse and Matt, even longer). Some of them are very close friends. And I have found that the way I think about Redshift Robotics has changed: I don't quite think of it as something I used to be a part of, at least not that simply. Instead I think of it as my legacy.

legacy: n. something handed down from an ancestor or a predecessor or from the past

- from The American Heritage® Dictionary of the English Language, 4th Edition, via Wordnik

I don't belong there anymore. FIRST Tech Challenge has changed since I've left. They play a different game than I did, they run their GitHub differently than I envisioned when I set it up, they have different mentors and leadership than we did, and I'm sure they practice and compete differently than we did. That's okay. But it's weird. I know exactly what they're thinking because I thought it too, years and years ago - to them, I am a vague echo in someone else's past; an accumulation of events that has shaped what the club is today but that they would never be able to name. I'm someone they've vaguely heard of, maybe, but never really known. What happens next is all in their hands and there's nothing I can or want to do about it; I witnessed a former mentor of SAAS robotics turn the entire club around and that's history they'll never have seen, and maybe will never even know about. But whatever is happening there now is something I'll never see, either.

Pretty soon there will be no students left at SAAS I still know. Two more very close friends graduate this year. It'll only take a year or two more after that before the last student I know personally is gone.

There are places I remember all my life
Though some have changed
Some forever, not for better
Some have gone and some remain.

- The Beatles, In My Life

I think maybe as time goes on I'm getting better at watching people grow and change. I'm on year two of college and already I see it happening - last year a close group of friends and I were just freshmen, bright-eyed and bushy-tailed. This year one's an RA, two are First-year Fellows[2] and I'm a D'Lion[3]. The rest are living their lives in upper-class housing. As I was sitting in the dining hall earlier tonight I thought, how lucky we are that we even met last year. We're all scattered around now, generally in wildly different departments, and we're all focusing on different next steps. Most of us live in different buildings from all the others, and I can't help but wonder if we would all have met later. I think the answer is probably no. Causality is a strange thing to me; the circumstances that brought us together are dead and gone and yet the friendship lives on. Legacy.

I see the world changing too. There was a point in my life where it seemed like nothing big and cosmic would ever change; it seemed like the world would adjust itself only in small details. But at some point that stopped being the case, when I wasn't watching. James Porter, who's part of the faculty at the Recurse Center, once told me that whenever new batches came in he was always super sad to see so many people he'd made friends with leave to be replaced by strangers he didn't know yet. But, he said, he never quite noticed the point where people in-batch switched from strangers to friends. That feeling of not noticing is how I feel about the world right now. And it just boggles the mind that I'm at an age now where I can actually say yeah, I've seen those things change. I see people even just a year or two younger than me so much more attached to their phones; I say "back in the day" unironically when I talk about tech things.

The traffic in Seattle has gotten worse, almost unbearable compared to what it once was. We didn't used to hear about wildfires like we do now; the country and the planet are burning. And above all else, the country is angrier. So much angrier.

I turned 20 today. A question for people older and wiser than I: I feel incredibly weird to not be a "teenager" anymore, what the hell do I do???

- me, a couple days ago, on pump.io and on Twitter

Footnotes:

[1]: and we all know how I feel about graduations.

[2]: upper-class student at University of Rochester who lives with first-years and helps them with academic stuff (especially course registration)

[3]: like a First-year Fellow except instead of academic stuff, they help bring their hall together and make sure everyone's having a good time and making friends, etc.



~