strugee.net

I'm back! Sort of. Very sort of.

I've known for a while that I'm going to ditch Blogger. That's a large part of why I delayed posting stuff to my blog for so long: I didn't have blog software that I really wanted to use, but I didn't want to just put more data into Blogger. Eventually, though, I realized that it would take me a while to write software that a) built an actual blog from Markdown and b) worked the way I wanted it to, so I decided that I would just start cranking out posts and write the software to build them later. And then of course there was the week or three that I spent procrastinating on writing... oh well. I'm here now.

Anyway, it's two three AM as I write this, so I should go to sleep soon. I have a lot of stuff I want to write about, so I'll be brief.

Summer

I haven't written anything publicly since Reset the Net, way back in June. So I should probably cover some of the things I've been doing.

First, I ran a CryptoParty! It was hosted at Black Coffee's old location on Pine, and it was absolutely fantastic. We had a small group of people but it was really fun, anyway. The slides are on GitHub - and speaking of which, I've switched to Bespoke.js for all my presentations. Hell yeah HTML5!

Right after the CryptoParty was over, I actually had the opportunity to drive down to Portland for DebConf '14, which was one of the most fantastic experiences of my entire life. I met a lot of really cool people there, I got my new key (also generated over the summer) signed by a lot of Debian folks, and I played a lot of evening games (not Werewolf - the other one). One of the coolest parts was the fact that I actually got to meet two of my heros - Linus Torvalds came and did a Q&A with us (video of it is available here) and John Sullivan, Executive Director of the FSF, did a BoF-style talk on how we can get to a point where Debian is on the FSF free distribution list. I got a chance to talk to both of them afterwards, which was undoubtedly one of the coolest things that has ever happened to me thus far.

Robotics

Yes, the school year started and I'm a junior now. It's rough.

I'm on 5619 this year at robotics (Gabe, our main mentor, actually called me at DebConf to talk about it). I was a little annoyed to not be on 2856 at first, but it's worked out for the best. We're actually doing really well this year, and I'm so proud of the work we've been doing on the robot. It's difficult - it's much more complicated this year than last year, and there's basically no room on the thing. We had to move wires out of the way to make room for our scissor lift to come down properly. Speaking of which, we have a working scissor lift, which has never been accomplished at SAAS before, at least not during a season! So that's awesome.

We had a rough time our second competition, due to a lot of things - the Field Control System lag was bothering our driver, Wilson, so I wrote up something quick to fix it. I went to test it, and as soon as I ran our scissor lift up, one of the bars - which (of course) we made out of wood - snapped. So we had to rush to fix it before we had to go on the field in three matches or something. And because the bar broke, I didn't get to test the new teleop, and when Wilson tried to drive with it on the field, it broke horribly. Luckily we did pretty well in the first competition, and that gave us a nice buffer to make up for our losses the second time. I can't claim that we're doing well, but we're not doing horribly, either. Here's a video showcasing the first competition, and here's a second one of the work the club did beforehand to prepare. Please excuse the weird camera angle of me intensely working on the code because of the time crunch of FTC competition.

Patching Firefox

Over Thanksgiving break, I wrote my first Firefox patch! There was a bit of the DevTools that was bugging me, so I fixed it, in true free and open source software fasion. Unfortunately I'm having some trouble writing tests for it, and I haven't had time to track down the information that I need, so the patch hasn't made it into the tree yet. Soon, though! It's on my list of things to do during break. You can see everything over in bug 1106353.

Mail

I've spent hours of work, spread out over a number of months, working on steevie's mail subsystem. And I'm proud to say that as of a couple weeks ago, I'm finally done. I had to buy a block of static IPs for it, which I felt really cool for doing. There's still a lot of work to do - SPF, outbound DKIM signing, better TLS, Roundcube, ManageSieve, antispam, moving to LDAP from MySQL... the list goes on and on and on, but the system works. And I'm really proud of it. Anyway, I have a new email now: alex@strugee.net. I've even set up the customary names to forward to me: you can email postmaster for email trouble, webmaster for problems with the web server, and even hostmaster for general stuff. Or root, if you're APT or cron or somesuch. It'll all reach me.

Christmas

It was Christmas yesterday! Merry Christmas, Internet! I got a bunch of books, including The C Programming Language, Second Edition (yes, this is the book that K&R C is sort of named after), which was very exciting for me. I also got a budget for steevie approved, so now I can buy a bunch of hardware that I need. Hello, RAID 10 array!

Stratic

So, finally onto the juicy development part. Stratic is the name of my new pet project. Stratic is the STReaming stATIC site builder. It's like Wintersmith, except that it runs on Gulp, which I've fallen in love with over the summer. Because of that, it's a little weird - there'll be some custom components used to support it, but the main body of code is actually... a Yeoman generator.

There hasn't been much activity in the repo because I'm using the strugee.github.com repository, which still runs strugee.net even though I'm not on GitHub Pages anymore, as a testbed for Stratic. Once I've ironed out all the kinks, then I'll land all my work in the generator-stratic repository.

I'm very tired, and I want to go to bed, but before I do I figure I should explain the format of this post. Yes, this is Stratic format. It's pure Markdown, but with some additional semantics that Stratic will use to build out the blog. I figure that not many people will use <h1>s in their posts, so the Markdown equivalent (#) is used to distinguish the actual post text from what is essentially a header. I did something unusual, though, because even though the header is essentially for Stratic - who authored the post and when, what it's called, etc. - I wanted the Markdown to be at least somewhat readable in source form. Therefore, you're actually allowed to put anything you want in the header section. The values are distinguished by double quotation marks. Stratic will figure out what they mean based on their position. The first set of quotation marks contain the title, the second set contains the date, the third set contains the author, and the fourth set contains a comma-separated list of categories that the blog post should go in. The date looks a little weird - it's seconds since the epoch plus an optional UTC offset. It could be made more human-readable, but then you've got to parse stuff and it just turns into a nightmare. So I opted to sacrifice readability for elegance.

So! This is a blog post without a blog. Soon, I'll finish up Stratic and this will no longer remain solely in source form. I'm excited! I've already rewritten strugee.net in Jade, and used the opportunity to refresh the services page. Time to get crackin'.

Hey everyone, welcome to the inaugural Unix & Linux community blog post. Let’s get started.

Introduction

If you’re like me, you work on many different kinds of computers. I have two Darwin machines that I regularly work on; I have about four different installs of Arch GNU/Linux, and I am planning to add even more esoteric distributions to one of my boxes. I work in Cygwin. And I have a Debian Sid box, which needs to be converted to something else.

My point is not that I like distribution-hopping. My point is that many of us use very diverse environments, and even if we don’t use very diverse environments, we all have a lot of environments that we work in. I’m also guessing that most of us here have at least some basic configurations accumulated, and many of us have extremely tricked-out shells. Wouldn’t it be nice if there was a coherent way to manage all of your configurations?

Enter version control. Not only can you use version control to synchronize your configurations between computers, but you’ll also inherit all the other benefits of (hosted) version control:

• A nice history of everything you’ve ever done to your configurations – and if you write good, proper commit messages, a nice history of why you made those changes!
• The ability to link to your configuration when you’re asking for help, so that people can see the overall context of the snippets that you post
• Nice ways to track subprojects (e.g. frameworks like oh-my-zsh)
• Show off your stuff way more easily

In this post, I’ll show you how to recreate the exact setup that I use to control my configurations. I prefer to use git, but you should be able to adapt most (if not all) of this guide to your preferred VCS. The only reason you might run into trouble is if your VCS has no equivalent of git submodules.

The initial import

First, create your repository online. The details will, of course, vary depending on what VCS host you use. Personally, I use GitHub, but again, anything will work.

Next, we need to import all our stuff to the repository. First create a local clone.

git clone git@github.com:strugee/dots.git

If you don’t like whatever name you chose for the repository, you should rename it now, before you do anything else.

mv dots configs

Now, copy all the configs that you like into the repository.

cp .z* configs/ # copies all the zsh stuff

Take care to ensure that you create the directory structure properly and safely. For example, the following is a terrible idea:

cp -r .gnupg configs/

If you do this, you may accidentally commit .gnupg/secring.gpg into git! This is obviously something that you don’t want to do. Instead, do this:

mkdir configs/.gnupg
cp .gnupg/gpg.conf configs/.gnupg/

Another thing that you should definitely watch out for is the .ssh directory, but you may have other tools that store secret data next to config files. Just be smart.

That’s it! We’re done importing. Now let’s commit.

cd configs
git status # double-check what you're commiting
git commit -m "Initial commit"
git push

We’re done. Your configuration is live on the internet. There’s just one problem…

Hooking up your configurations

…and that is that all of your real configurations are separate from your version controlled configurations! If you make changes in one, it won’t do anything to the other.

When I was originally setting all this up for myself, I put my configurations straight into ~. This is a horrible idea for two reasons:

• Every directory that’s a child of your home, indirect or direct, will be considered a part of a git repository, because a parent of that directory (said parent being ~) has a .git directory.
• You either risk accidentally introducing tons of crap from your home directory into your repository, or you have to maintain a large and ugly .gitignore.

The solution that I now use is simply symlinking all my configurations into ~ (or wherever). If you find that this approach incurs too much overhead, you may want to look into a framework to do this for you, as I mention in the “taking it further” section. That being said, I’d encourage you to try simply symlinking, at least at first. It’s surprisingly robust.

Anyway, creating your symlinks is dead-simple. Watch out for things that fail (you’ll want to remember them; if you’re feeling especially forgetful, redirect stderr to a file or something).

cd ~
ln -s configs/.* configs/* .

This will symlink everything that’s in configs/ into your home directory. This is probably not exactly what you want, so let’s do some cleanup:

# pwd is still ~
rm .git # if you don't remove this, git will think that ~ is a repository!
rm COPYING # license information that doesn't need to be in your home directory

Notice that we haven’t removed .ssh and .gnupg. Why? These (probably) didn’t get symlinked when you ran ln because they’re (probably) already there. You’ll have to fix these manually, along with anything else that failed when you ran ln.

ln -s configs/.gnupg/gpg.conf .gnupg/
ln -s configs/.ssh/config .ssh/
# you'll have a fair amount of stuff in .config; anything that respects XDG_CONFIG_HOME puts stuff there
ln -s configs/.config/awesome/ .config/awesome

All done! You’re now managing your configurations with git.

Adding a framework

If you’re like me, you use a framework, like the excellent oh-my-zsh, or something like liquidprompt, or whatever other whizzbang what-have-you project you have a preference for. What you’ve probably done right now is simply copied the directory of your framework straight into your repository. This means that whenever your framework updates, you have to commit all the new stuff into your repo. Not only that, but when you clone to a new computer, you have to do some black magic to restore the “repository state” on the new computer, because git automatically ignores any .git directory that isn’t a direct child of the root of your current repository. (It all gets very confusing to talk about at this point.)

The solution is to use git submodules. If you’ve never used git submodules before, I’d encourage you to read up on them, either by invoking man git-submodule or by reading the relevant page of Pro Git. I won’t go into details here, since it would just be duplicating information, but the tl;dr is that git submodules let you embed another git repository into your main repository, but in the Right Way(tm).

Let’s take my old .oh-my-zsh setup and fix it to use git submodules.

First, we simply get rid of the old stuff.

# pwd is ~
cd configs/
rm -r .oh-my-zsh

Then we add it back as a git submodule:

git submodule add git://github.com/robbyrussell/oh-my-zsh.git .oh-my-zsh

And then we commit the result into the repository:

git add .gitmodules .oh-my-zsh
git commit

And you’re done! Just push.

Oh your other computers, you’ll have to initialize the submodule after you pull. This is also super easy. Just run:

git submodule init # only needs to be run the first time
git submodule update

Taking it further

There are a number of things that you can do (one of which you should do) to take your version-controlled configurations to the next level.

License your stuff (this is the thing that you should do)

Your configurations are now public and usable by other people. That means that you should add a license, otherwise people aren’t legally allowed ot reuse your stuff. At risk of starting a flame war, I’ll state that my personal preference for small projects like this is the WTFPL.

Please, please do this. The public thanks you.

Getting something to manage it for you

There’s a bunch of stuff in our new setup that you need to run manually. You need to pull. If there are new files, you need to link them. If there are submodule updates, you need to run git submodule update. Wouldn’t it be nice if something could do all this for you?

There are a bunch of solutions to this problem. Some people write makefiles. Some people write shell scripts. Some people use a real framework. It’s all up to you.

Personally, I used this problem as an excuse to learn to use make. With the Makefile that I’m in the process of writing, I can pull, update my submodules, and reinstall all my symlinks, all with a simple invocation of

cd configs/
make

However, like I said, it’s all up to you. If you prefer to use a framework, you should check out Zach Holman’s dotfiles (he has a Rakefile that helps you manage this stuff), Homesick (which is an even more organized framework), Ben Alman’s dotfiles (which has a more casual, loose organization), Eduardo Lundgren’s dotfiles (which has a Grunt-powered configuration tool), ghar, and vcsh (this may be especially interesting if you just hate the fact that you have to have a separate directory for your config repo). fresh is also pretty neat – it’s like a package manager for dotfiles.

While I haven’t used any of these myself, they’re all excellent projects.

Further reading

dotfiles.github.io is an excellent resource for all this stuff (and, in fact, it’s where I pulled the above list of frameworks from). You may also be interested in a list of all dotfiles repositories on GitHub, sorted by popularity. dotfiles.org is also a place where people share dotfiles, albeit in a much less organized fashion. I’ve never used it myself, but I’d encourage you to try it out.

Conclusion

git is awesome. Configurations are awesome. Sketchy rsync scripts aren’t the best when you could have organization (and history!) instead.

Good luck and have fun! Let me know in the comments if you run into problems, find new resources, or have anything else to say. (Flames to /dev/null, but constructive criticism welcomed.)

Thanks to @terdon for editing this post prior to publication.