Review of Secret Cinema, Spoilers Included

Update: After a few pieces of earnest feedback that I’m actually taking away from people’s experiences, I decided to alter the details of this post to exclude direct mention of the movie. Given the strong negatives of the overall experience, I feel like what movie it is doesn’t really matter. Still, doesn’t do to diminish someone else’s time. Fair warning: This post still contains details from the event that are pertinent for giving a coherent review. Some people may feel these details are spoilers. I’m including them not to be a jerk, but to be illustrative.

Wanted to offer some feedback about Secret Cinema from Friday, March 21st. This is my second Secret Cinema experience and I was really looking forward to it. My first was the Casablanca showing about a year ago.


Aspects of this Secret Cinema event were very good. I’ll just list them for brevity’s sake:

  1. Exploring the space was amazing. Every room was exquisitely detailed. The notion of serendipitous interaction with other people and actors is a brilliant idea that in some cases played out very well.
  2. Some of the actors were amazingly good. One actor in particular was a genuine pleasure to be in the presence of. My date and I lingered in his room for quite some time just watching him interact with people. So good.
  3. The court room scene was an unexpected and mostly fun experience.


That said, the were some extremely negative aspects of the experience. To be honest, the negatives of the experience so far outweighed the positives, I don’t think I’m likely to participate in Secret Cinema again. Especially considering that some of the negatives were consistent from my first time at Secret Cinema last year.

Mean “Actors”

When we arrived, we were greeted by someone who told us to go “over there” where a group of people were gathered, so that we could get our hands stamped. We were immediately asked to show our tickets by a woman patrolling the line. I had the email on my phone, so I started loading it. The person asking me was very abrupt with me. She then moved on to the next people in line. As the line moved towards the front, I was asked my last name, and had my hand stamped without ever seeing the tickets. Which begs the question — why was I questioned about my tickets if all that was needed was my last name? Also, while waiting in this line, a man dressed as a police officer swore at the people in line in front of us and told them to speak when spoken to. Okay, if this were a prison movie or if we were in character as suspects for something (or kidnapped people perhaps) that makes sense. However, if his character is an officer on the street and we’re upstanding citizens, what sort of behaviour is that? Just felt oddly mean and degrading for no clear reason.

When we entered the Chinese Laundry, we were greeted by a character who was to shepherd us into the underworld. This character was openly antagonistic. Beyond being a bad actor whose awkward Boston-meets-Chicago-meets-London gangster accent was painful, he seemed to be just indulging in the opportunity to swear at us without making it feel like a legitimate experience at all. The mismatch between what would have made sense and what he was doing was so conspicuous that it took away from the experience. If he had chosen to say nothing at all, the effect would have been more resounding and interesting.

Once we were inside, I was greeted by a tall guy in a dark suit and hat. He bumped into me from behind. I turned to face him and apologised, unclear at that point that this was his act. Face to face, he leaned in close and blew e-cigarette smoke in my face. I could smell his not-clean breathe and feel the moistness of it on my lips. Gross. He then said, “You want to make a hundred bucks?” Let’s take a step back and think about real life. Let’s imagine that we’re in the real world. I’m in a club. Some guy walks up to me, runs into me, blows smoke in my face, then asks me if I want to do work for him. How would I feel? I would feel like punching the guy. I would settle for getting away from him as quickly as possible because he’s a really unpleasant guy to be around. So, given that this is ostensibly play (though there was nothing playful about him blowing smoke in my face), I just said “No. Not remotely.” He looked at me disgusted, told me “You’re not going to have any fun around here,” and walked away.

Ironically, he was pretty much right. What he didn’t understand is that I already wasn’t having any fun. No aspect of the experience so far put me in a state of freedom to play. Nothing so far felt like there were safe bounds between reality and play and that we were all in this together. No, in fact the experience felt much more power imbalanced in favour of those who knew what was going on than those who didn’t.

Secret Cinema feels more like the Stanford Prison Experiment than a playful immersion in the world of the movie.

Sadly, the list of bad interactions keeps going.

At one point, we were all ushered into a real court room to witness a fake trial. I was grabbed aside and drawn into a room with a guy who told us to render “Not Guilty” no matter what in exchange for $500 bucks. This seemed like the moment to be playful for me. Getting to participate in an intense jury debate sounds fun. Sadly that wasn’t the way this worked. I was to do as I was told here. Being told what to do and when to do it is not fun. Then, in the court room one actor posing as a bailiff told us to stand in one place. Then another came along, swore at us, and told us to go stand somewhere else, asking who told us to stand there in the first place. Hilariously, it was his fellow actor. Being treated like dogs expected to roll over again and again is not fun.

At another point, we tried to go into a club but we were stopped at the door by an actress. She told us it was members only and that we couldn’t just walk in “unless we had something for her.” So, playing along, we offered her as much money as we had. Yet, it wasn’t good enough. She kept up the act of surly door minder to the point that my date and I just felt truly unwelcome and walked away. It wasn’t mean-with-a-wink, it was just mean.

Later, I was standing in one of the rooms and a guy walked in, pointed harshly at me, said, “You — come with me,” and stormed out. Not a chance in hell. By this point, I was so fed up with the experience that I would rather stand off with the guy right then and there than engage in any more “play.”

I can keep going, but you get the point. I’ll leave out the other court room bailiff who stopped us to extract a “jury fee” from us and all the other small interactions that were antagonistic by default. Why couldn’t actors just have interesting personalities? Why was aggressive, exploitive behaviour the default interaction with every person? Sure, the movie they showed was a gritty movie, but that doesn’t mean that every person you meet should treat you badly as the first note of the interaction. Also, the fact that everyone in the movie is a grifter is ostensibly a secret, so we should have had no idea why the hell everyone was being so abusive. The whole thing feels like an indulgent experience for the “actors” and an abusive experience for the attendees.

The most absurd reality of this is that I’m an extremely playful person. I love fancy dress. I love acting a fool and not taking myself seriously. I do a lot of public speaking and feel very comfortable in a semi-structured environment that is semi-contrived and semi-real. I was once kicked out of a public park with some friends for acting out Hamlet on the amphitheatre stage after the park had closed. For god’s sake, I went to Burning Man last year and had an amazing time. I also got really into this night. I rented an authentic 20s suit and followed all the procedures ahead of time. So did my date. I wanted to come and have an exciting and playful night. If someone like me can’t get into Secret Cinema, who the hell is it for? Who would have a good time?

Cellphone Confiscation

Ruined screen protector

Back at the start of the night, after getting our hands stamped, we walked to where we had to give up our phones for the evening. Part of my work is building systems that operate at all hours of the day on complex infrastructure. Sometimes systems break and I have to act quickly to figure out what’s going wrong. It makes me very uncomfortable giving up my phone because it’s my notification mechanism when things go wrong. Also, I use my phone as part of two-factor authentication to lots of systems. If they lose my phone, it would be more than a day of work to get things back in order, starting with buying a new phone and a trip to my service provider to get a replacement SIM. Not to mention the trouble it would be trying to get them to cough up the money for the phone later. Not remotely my notion of a good time.

So, I questioned their procedure, asking what assurance I have that they were going to do this correctly and not cause me tons of grief. The response I got was a compassion-less “We haven’t lost a phone yet. This is company policy.”

Interestingly, none of the information I was given ahead of time included an explanation that this would be required. I went back through every email and every web page I was directed to. No mention at all. If they had told me ahead of time, maybe I would have changed my mind about going. Maybe that’s their reasoning behind surprising you with it when you get there.

The guy taking my phone explained that I would be given a ticket that would allow me access to my phone at any point if I needed it urgently. Unhappy but not willing to end the evening five minutes into it,  I gave up my phone, took my packet and went inside.

At the end of the night, my date and I went to retrieve our phones. There was already a group of people starting to accumulate, but we were first in line. A guy still acting half-way in character took our ticket, radioed for our phones to be brought up then walked away. Minutes into this, with no more info about what was happening, one of the other people waiting had someone return with a satchel with his ticket’s number on it. The woman returning his phone peeked inside the bag and then asked him to describe his phone to her. This is their system. They decided that a number and a satchel was the correct way to do this. Interrogating people afterward is insane. If you take my phone from me and give me a reclamation means for it, I do not expect to also have to prove my knowledge of my own property. It also rang closely of being asked for my tickets at the entrance when my name was all that was required.

Being honest, I lost my temper at this. I spoke harshly to this lady. Something along the lines of “Just give him his fucking phone back.” Reasonably, he looked surprised. She looked antagonistic right back at me and told me this was necessary because they wanted to make sure people got their correct phone. I would argue that it’s complete bullshit and that they should come up with a more reliable system if they wanted to take ownership of tens of thousand of pounds of nearly identical devices. If I found any ticket lying on the ground, went to them, answered their question with “an iPhone,” I would have a super high likelihood of walking away with a phone that wasn’t mine. It’s a bad system at best. Also, when we got our phones later, they didn’t ask for a description. Arbitrary treatment where someone else has power over you and you just have to do whatever they cook up for you to do is not fun.

Around that minute, chaos ensued. A group of people down the hall had been setting up a table for returning phones to people. It was unclear who we should be talking to. It was also completely obvious that the organisers had done nothing ahead of time. They were sorting the phone satchels numerically while fifty-ish people stood and waited. This was around 5 hours into the evening. It didn’t occur to them to have this done ahead of time? That at the end of the movie people would want to leave? Or to make sure the phones stayed in order as they were collected? I cannot understand why this was an issue at all. Every coat check I’ve ever been to just gets this right by default.

The group of us waiting for our phone grew from the original ten, to fifty, to probably half of the people at the event. This is an interesting data point, because we were leaving early. As we walked out of the movie hall, multiple people told us to stay because there was a dance afterward. We didn’t exactly feel like dancing. When the credits rolled, I felt like I had put up with the fullness of the experience and, relieved, could now leave. Perhaps some of the more than a hundred people gathered in the hallway felt the same way.

So at this point we were waiting somewhere between the desk and the table (theoretically, our phones would come to the desk where the guy had radioed for them), we heard someone shouting “Over 200 on the right, under 200 on the left”. They were trying to partition a mob of more than a hundred of people *after* it had become a mob. Also at this point, despite having left early to get out before the crowd, my date and I were now swimming in a mob of people with no idea where our phones my show up — the desk where the guy had radioed or the table where the mob was looking.

We walked to the tables (where they were still shuffling phones while people waited) and asked. We were told to please wait in line. There were no lines, just two mobs of people running the length of the hallway now. I was feeling sort of apoplectic at this point so I just walked away. I would rather sit quietly somewhere than wait in an infinitely long line while they sort out something that should never have been a problem. My date was more persistent with the person holding back the mob, got a bit aggressive, and had our phones returned immediately. I feel bad that we jumped the mob. I feel like we got special privilege because we were rude, which is not something I ever want to be. This feeling is counterbalanced by the fact that we took special efforts to get ahead of exactly that situation and the incompetent handling of the situation ruined that and left us standing around with no notion of what to do until the situation was going to be a mess no matter what.

When we got back home, my date noticed that they had managed to scratch up her screen protector in the process of confiscating and returning our phones. No damage to the phone, fortunately, but it indicates that they weren’t going out of their way to take care of our phones.

In sum, Secret Cinema made for a bad evening. If I had it to do all over again, I would skip the event without batting an eyelash. I would have had more fun watching a lot of other movies alone.

Lessons: A Year and A Decade

Original post:

This is my answer to The Birthday Question for the 30th year of my existence. I was asked the question by a number of people this year, which I think is awesome. For those of you who haven’t been exposed, the question is this — what is the most important lesson you learned this year? Because this is a special year, I was also asked to answer the same question, but scoped over the last decade. Serious business.

To answer the original Birthday Question, I wrote a massive list of things I learned and then sorted them according to impact to my life. What follows below are the things from that list in order decreasing order of importance. I’ve omitted a few things that I don’t think make sense to share this way. If you’re itching to know, I’ll probably answer privately.

So, the most important lesson I learned this year is that it’s okay to fix me and also to love me while broken. Or, to put it less harshly, to accept myself as good, flaws and all. I would guess for a lot of people that’s a strange thing to hear me say. I probably come across as confident and happy (or at least upbeat even if I’m raging) most of the time. In reality, I have very high expectations of myself that sometimes I don’t stack up against. I don’t think anyone who knows me would blink if I said it’s my nature to put a lot of pressure on people to achieve. The part that not a lot of people know is that I have always put much, much more pressure on myself than I put on other people. Part of my internal ruleset is to avoid letting other people feel the pressure I apply to myself.

This year, I learned (in some senses the hard way) that I have to be able to accept myself even in the face of failing some of my internal checklists, especially if I want to love other people. I learned that letting the cracks show and not being so perfect means I can grow and heal, better and faster. I learned to be compassionate beyond what I knew was possible. It also means I can be an open heart for others because I’m not afraid of being faulty myself. Obviously, I’m not just a wide open heart all the time, but it’s something I can be. This year I learned that people respect you for owning your flaws.
This year I learned that what I think is most important might be something that not everyone else agrees with or sees value in. I guess I’ve known this for a long time, but not in general form. This year I learned that this can be true at work as well as family as well as friends, and so on. Sometimes that means doing things other people disagree with and being willing to deal with the consequences of the difference.

I learned that everything most likely will just be okay. That doesn’t mean I learned to trust that knowledge 100% of the time, but I try to and fundamentally I know it’s true. I also learned this year that trusting people is easier than I thought it was.

This year I learned that choosing to work on what I *need* to work on more than I what I *should* work on is a dangerous proposition. Sometimes serving one need means neglecting something else. This year I put less of myself in my work than I have in the past 5 or 6 years. I needed to. I had some healing to do and some growing and I wasn’t willing to let work hold me back on that front. It worked out poorly for me in my review, but I learned a lot in the process. Sometimes accepting the cost of “failing” in some way affords breathing room that is more valuable anyway.

This year I learned that finding a role model and letting that role model know she or he is a role model changes the relationship. Not necessarily in a good or a bad way. It’s just different than being quiet about it.

This year I learned that I’m on a mission to realize a world that I wish I had. I think I wasn’t aware of this before, but I was just as ardent about achieving it. Knowing makes a huge difference. I get to decide how much energy I want to devote to it and decide if the world I’m driving towards is a good one (and so I can change course if I’ve gone wonky).

In a broad sense, this year I learned to be less afraid. Also that being less afraid means other people get to be less afraid, too.

Because this is my 30th year, the wise and giving Chaitanya Mishra asked me to answer this question for the last decade. Same process, different answers.

This decade I learned that where I come from means only as much as I’ll have it mean. I learned that I want it to mean more than I thought I did. I learned that I respect and value my humble beginnings and never want to lose sight of what that feels like. Importantly though, who and what I want to be matters more than where I come from in the end.

This decade I learned that I don’t need much love and affection, but I do need some.

This decade I learned that I want important people in my life who stay in it.

This decade I learned that I’m capable of more than I had any working models for when I was young.

This decade I learned to code.

Sassy !== Assertive and Female

Original post:

At a dinner some weeks back, I was having a conversation with some male engineer friends of mine from a bunch of companies in the London area. The challenge of finding engineers who are women came up. It’s a topic that comes up a lot when I’m at the table because I think it’s really important for male engineers to understand the experience women have as the gender minority in technical roles. Usually I find these conversations enlightening. Especially if there’s a female engineer at the table, I often find bugs in my perspective as a guy. I’m far from bias-free as a man in tech (though I could probably just delete the “in tech” part).


At this particular dinner, I found the opposite to be true. As we talked through the factors that make hiring women in technical roles difficult, we started thinking through the characteristics women we work with have in common. There do seem to be some common things. Really strong technical skills was (and usually is) at the top of the list. We also talked about women who code not being afraid to be the only girl in the room a lot of the time. This is where something unexpected popped out of the conversation. It took me a while to grok why I was so bothered by it, but I think I understand it now. The flow of the conversation went something like this:


* Started with a trailing conversation about women in tech who are awesome, focusing on particular women we know. *


Male engineer #1 –

Yes, _redacted_ is really good. She just has a sassy personality that prevents her from getting pushed around by men.


Male engineer #2 –

I think a lot of women who do well in tech are sassy that way.


Me –

*semi-automatic facepalm, vague confusion*

So, by sassy — you mean, like, assertive and female?


Male engineers #1 and #2 –

*General agreement*


Me –

You know those aren’t synonymous — right?


Male engineers #1 and #2 –

*Lots of counter arguments*


All of us –

*Lots of terrible validation of gender stereotypes, completeley assinine judgments, and typically male presumed-top-of-the food-chain awfulness.*



It’s not all the time that I’m aware of subtly sexist things I do or say. If I’m being completely honest with myself, I’m pretty sexist in a bunch of ways that fall out in a lot of places. It’s embarassing to admit. It’s more embarassing to try to hide it. My goal here is not to flame some people for saying a sexist thing. I’m completely guilty of the occasional TWSS joke and definitely guilty of letting my biased thoughts roll out unfiltered into conversations with peers and coworkers irresponsibly. I have absolutely been that guy who has to stop laughing when a woman walks in the room because I’m too embarassed to try to answer her if she asks what’s funny. Knowing this about myself, a goal here is to point out a class of sexism that happens all over the place and to flesh out the underlying perspective that enables it. Also, to hopefully own my role in it in a way that will help me change.


Back to dinner. In the immediate conversation that followed, I called the guys out about it as much as I could then. I argued that sassy is not gender neutral and that it carries a bunch of bad connotations. Surprisingly, I got skepticism and push back. I don’t know why I’m suprised really. Smart engineers don’t fix problems that don’t exist. They want proof and want to do their own thinking about the proof they’re given. Why wouldn’t they push back against an accusation that, if valid, would require them to rethink a lot of how they think and behave? Why wouldn’t they push back against someone telling them that they need to be more self-critical about everything they say about women? Fair arguments. My response is: because it really, really matters.


Now let’s dig in to the tricky stuff. What’s actually the problem with sassy? Let’s start with why this is a bad way to describe someone, starting from least offensive to most.


For starters, let’s be clear about the meaning. Sassy means self-assured and bold. Good things. It also means impudent, saucy, cheeky, conceited, and pretentious (all from the OED, btw). Very bad things. Someone who is sassy has an agenda other than what is important in the moment. They might react overly strongly to something in a way that doesn’t fit the situation and probably draws attention to them.


I can see why that would be annoying if you’re trying to get work done. I can also see two massive reasons why being annoying is good and being labeled ‘sassy’ sucks in this context. Imagine the gentle discrimination felt by someone on the receiving end — someone assertive enough to address it — by people who don’t notice what they’re doing. That feeling of “wow, this sucks” can easily lead to a kind of “impertinence.” She’s pushing back on the loaded situation she’s in. Meanwhile, the guys are ignoring it because “hey, it’s not a problem I feel any evidence of.” In this situation she’s behaving reasonably, I think. I have done the same when someone is being dismissive. Labeling this as “sass” is sort of accurate (by definition) and also completely terrible in basically every way. It means short changing the person in the moment, frustrating her resolve to fix things, and simultaneously maintaining the hegemony. Triple facepalm.


Okay, so, that’s obviously really bad, but it’s not the worst of it.


Worse than that, sassy is a gender identifying term in this context. In my adult life, I have never heard a guy called sassy. To fully own my assertion here I will add that all of us at the dinner party actually discussed this very point in the same conversation. We agreed, because we’re terrible people, that sassy is a term only fit to describe women and gay men. My internal “I’m telling everyone what a massive asshole I am” alarm is going off like crazy as I write that, but I’m committed to being honest here so I’m ignoring that alarm. I’ll indict myself fully in the next paragraphs.


Most tragically, sassy is really, really belittling. After giving a lot of thought, I pretty sure it’s only appropriate to refer to children as sassy. Why? Because my right to label someone “sassy” implies that I know what’s appropriate in a given context and they don’t. It implies that I’m qualified to judge them. That they’re somehow under me in a way that makes it okay for me to assess and label their assertiveness one way or another. To put this more painfully, it shows that we men felt entitled to judge her and all women (and apparently even gay men, to my surprise) on their ability to participate at “our level” in things. That’s phenomenally bad.


The sum of all of this is pretty obvious but I’m going to spell it out anyway. We, men, these men, myself included, are still a bunch of male-supremacist assholes. When another man asserts himself or has a generally boisterous personality we probably call him things like boisterous or assertive. Maybe in the extreme case, aggressive or domineering, or an asshole. We probably also compliment him for it because he’s an engine — a steamroller. What’s interesting is that we describe him relative to how he makes us feel for the most part. With this woman, we described her in a way that reiterates her status relative to us. We put her beneath our implicit superiority — because she was trying to reach across and behave as a peer. We don’t give her the option of being able to make us feel any particular way other than sassed by an inferior.


For the haters:

‘Dude, you’re reading a lot into this one conversation?’

Yes. Yes, I am.

‘Man, is the problem really *this* bad? Aren’t you just making it worse drawing so much attention to it?’

Depends on how much you care about giving everyone who would love to be in technical roles the ability to be in them without having to look up a steep hill of tiny bias-pebbles every single day. I would say that’s pretty important.


‘Doesn’t it seem hopeless making this much noise when there are countless similar events happening in others contexts without anyone batting an eyelash?’

Not on your life. You have to start somewhere. This is my first attempt at changing things by being willing to be the guy who owns what we as guys think and to hold us accountable for it.


‘Oh, this again.’

Haters gonna hate.


Main point, again:

My goal in writing this isn’t to flame a handful of dudes about an offhand comment made at a dinner. It’s to point out, own, and debug a perspective problem that manifests in offhnad comments at dinners everywhere all the time. I want to be a guy that women want to work with. Someone a woman will trust to give her credit where it’s due — to take her just as seriously as anyone else with the same abilities and different anatomy.


Aside from the fact that it’s just extremely obviously wrong to feel any differently, I feel this way because my experiences working on teams with both women and men is that women make kick ass engineers and kick ass teammates. If you disagree fundamentally, you should be very skeptical of your perspective. You might be suffering from the same subtle biases I’m trying to get past.

This Web Page is a Dick: T-Mobile

Screen shot of a browser inspector showing the HTML that governs the behavior of a webpage

T-Mobile, why do you hate your users?

It boggles me endlessly when sites have draconian requirements for passwords. I use a password generator/store application called 1Password. This application lets me create crazy strong passwords for every site I interact with. This means I can ratchet up the special characters and length without any worry that I won’t remember the password. I know I won’t remember, I use a computer to do the remembering. Anyway, this works great unless I’m up against sites like T-Mobile that offer ridiculous rules:

  • Must be between 8 and 15 characters.
  • Must include letters and numbers.
  • Must contain uppercase and lowercase letters.
  • Cannot contain spaces or special characters (!, @, $, %, ‘).

So the first rule makes some sense. Short passwords are less secure. However, length of password should have no restriction because more bits means longer crack times in the case someone does get a hold of the hash. Senseless.

Requiring letters and numbers is good as is requiring upper and lower case letters. Adding more variety to the symbol set means someone trying to brute force the password will be at it for a lot longer.

No spaces or special characters… wtf? There are dozens of special characters available on a standard keyboard. Thousands if you count multi-key sequences that produce special characters. Allowing these characters would make it extremely burdensome to try to haxX0r people’s passwords.

As if that isn’t bad enough T-Mobile goes even further to make life miserable for its users: complete restriction on the input methods the user can use to set their password. In the screen shot above, you can see that the UI engineer who built this page has disabled pasting into the fields. They’ve also made the refuse to accept drag and drop text. I can’t fathom the product decision making that would require this sort of input. For me, this means that if I want to use a highly varied, randomly generated password that I would have to manually enter all 15 characters of it exactly correctly… twice. Fucking insanity. Fortunately for me at least, the HTML DOM is a dynamic place so I can just remove these dubious “features” via the web inspector and get back to what I was trying to do.  This web page is a dick.

Generating a Color Picker Style Color Wheel in Python

color gamut generated from Python

This color wheel was generated with Python. Check out the code on GitHub.

For the record, I am very bad at Python. Yet, it has become more and more my go-to language for doing little things that aren’t web. For instance, I recently undertook to create a color picker/color wheel in Python.

For the uninitiated, a color wheel or color spectrum is a series of hues that run from red to indigo (roughly purple to our eyes). It’s the rainbow. The color wheel is this same thing but wrapped around in a way that shows all the hues the human eye can see. In reality, indigo doesn’t converge on red as you go up in light frequency, but the hues that you would see if it did are colors we know well — magenta/pink.

So, generating a color wheel is an interesting challenge. The primary colors can be created from the hex colors 0xff0000 (red), 0x00ff00 (green), and 0x0000ff (blue). You probably see the pattern easily. A color breaks down into three channels, each of which has up to 0xff or 256 values from 0, completely absent, to 0xff, completely displayed. Blending from blue to green implies moving from 0x00ff00 to 0x0000ff by decrementing the green channel monotonically as you monotonically increment the blue channel. So, you can probably start to see how the logic for generating the color spectrum graphic will have to work. Of course I’m ignoring a few colors that are important. Red, green, and blue are only half the colors that are necessary for the color wheel to look complete. The others are 0x00ffff (cyan), magenta (0xff00ff), and yellow (0xffff00). These colors appear in color spectrum interwoven between the primary colors. If you’re wondering why red and green make yellow, you’ll probably want to read about Color by Addition. Though, from a numbers perspective it’s easy to see a relationship between 0xff0000 -> 0xffff00 -> 0x00ff00.

Creating the gamut requires calculating how much of each color to use in the blending. One important aspect of a color wheel is that any color you can select is a blend of at most two colors. So, one way to generate a color wheel is to go pixel-by-pixel through the image you want to create figuring out what sextant you’re in, deciding which color you’re closest to, then figuring out how much of the other color to blend into it. That’s the approach I took. It’s true that you can do this other ways. In fact, I’m pretty sure there is a slick algorithm for this that I didn’t realize. I was on a plane when I wrote this and consider it more about learning Python than about making the most compact implementation of color spectrum generation.

Anyway, here’s the code (using the Python Image Library) on GitHub.

Hey Android, Where’s the USB Data?

Does your ping look like this when using Android’s sweet USB tethering feature? I tried lots of things:

  • Disabling/Enabling USB Tethering
  • Disconnecting/Reconnecting the USB cable
  • Restarting the phone
  • Disabling/Enabling the network interface on the computer
  • Restarting the computer

All of this got me exactly no where. The phone is still connected to the interwho, but it’s just not sharing it with my laptop. So sad. So, thinking a little bit more laterally (read as, poking around looking for a sign from the gods), I tried toggling the data access of the phone. This means going to Menu > Wireless & networks > Mobile network settings > Data enabled and unchecking the box.

Why does it make sense to turn off data when what you want is data? Well… it doesn’t. But, if you then turn the data back on, the phone may well realize that it has been fraudulently denying your poor computer all the data goodness the marvelous (if mystifying) Android platform promises.  I say “may well” because with the fragmentation of the Android market and the amount of irresponsible hackery the carriers subject the OS to, this may not actually fix your problem though it fixes mine. I’m using a vanilla-tainted-by-T-Mobile-and-LG version of Android 2.3 running on the T-Mobile G2X. This phone also goes by the name of Optimus 2X. Soon enough I’ll ditch it for the Galaxy Nexus, but I digress.

This what I get trying to ping after turning back on the “Enable Data” option. Hooray. All it took was doing an action equivalently counterintuitive to the famous “Oh you want to shut down? First, go to start…” paradigm. At least this one is unintentional.

Also, you may be wondering why on earth I would be using USB Tethering when my phone provides WiFi Hotspot capability. The answer is that I’ve found the latency to be lower and the signal to be more consistent using USB Tethering. The bandwidth is also higher, which I suppose makes sense because one phone using the same wireless antenna for sending and receiving data probably isn’t sending as fast as it could be. I should be careful to specify that I don’t actually know that is what’s happening. Anyway, USB Tethering gives me a faster connection. The other perk is that my laptop, which I’m going to plug in when I get to work, can power the phone while it’s on USB Tethering. With WiFi Hotspotting, I’m pretty guaranteed to drain my phone to the point where I will have to plug it in to make it through the day. There you have it.

Baby Steps: Vim Macros

I spend probably a third of my maker time writing code (another third doing root cause analysis on bugs and another third reviewing code, more or less). In my coding time, I do almost all of my work in Vim and Bash, mostly Vim. You IDE buffs out there can scoff if you will. Vim is a tool that has proven its worth over two decades. Two decades. That’s like two milennia in computer years. I’m an admittedly bad Vim user, but I’m making strides to recover. One stride I’ve just taken is using Vim Macros to automate stuff, like converting the format of text files.

If you’ve never used macros in any context, you may not have a sense of how powerful a tool they really are. Macros give you the ability to take a series of actions that achieve some goal and put them into a recording. You can then replay those actions again and again. This means that complex transformations on some text only have to be done correctly one time. The instructions can be captured and reused. We humans are pretty good at getting something hard done correctly once. Our error rate goes up the more times we do the thing. So, macros are helpful because when you get the steps right once, you get to use your own actions again and again. The replay is exact. The computer doesn’t accidentaly skip step three like you or I might the fifteenth time we walk through the process.

The other benefit is that the steps get executed at human speed (i.e., slow and clumsy speed) only the first time. Each time the actions are replayed, all the time you spent head scratching and double checking that you’re doing the right thing disappears and the actions get executed as fast as the computer can do it. Vim has macros built-in as do a lot of applications. Oh how I long for real life macrobot. Imagine being able to program a robot, one time, to fold laundry. I would never match a pair of socks again. Baby steps.

The background for all this macro talk is a task I took on recently. I needed to get the data out of a bunch of hand-rolled, inconsistently formatted PDFs and into some reasonable format. The goal was to have a structured data representation of the data that could be easily maintained over time. I guess it’s not strictly insane to try to keep a bunch of willy-nilly PDF files full of timetables up to date, but it’s far from efficient. To achieve this textual cleaning of the stables, I wanted to use Vim Macros because I knew they were a good tool for the job and I didn’t already know how to use them. Before this little jaunt, my exposure to Vim’s macros was pretty much just accidentally starting the recording process and trying every combination of colon, escape, and q to get out of it. Like I said, I aspire to be a competent Vim user. Baby steps.

Anyway, my tactic was to copy and paste the tables from the PDF files into a Vim document and determine an automated way to turn the data into a CSV format. Okay, actually I’m lying. I started by pasting that data into TextMate and trying to use find and replace hacks in addition to my amazing repetitive typing abilities to get the data into a CSV format. As I started to get into the rhythm of it, I could feel my inner smarty pants engineer sighing at me. Yes, I know there is no good reason to be using my pitiful human brain for simple, repeated tasks, but I don’t already know how to use them.  Yes, I should use the generic tools available for automating the process so I can get it right and do it fast, but I just want to get it done. This is a classic dilemma. Option one: Do it the slow, hard way that is guaranteed to get finished in a fixed amount of time but may introduce errors at worst and at best is a pain in the ass. Option two: Maybe do it the super fast way, assuming you can actually figure out how to use the scary new tools properly. XKCD 974 comes to mind.

After a few moments of complete philosophical system lock, I decided it was time to brain-up and make the computer do my bidding. Since I knew I needed to learn macros, I decided that would be the weapon with which I would dispatch these gobs of poorly structured nonsense.

Copying and pasting from the PDF, I get pretty garbled data, like this:

9:00 AM Location 1 6:30 PM Location 5
9:05 AM Location 2 6:35 PM Location 4
9:20 AM Location 3 7:45 PM Location 6
10:25 AM Location 4 7:55 PM Location 2
10:30 AM Location 5 8:00 PM Location 1
9:30 AM Location 1 7:30 PM Location 5
9:35 AM Location 2 7:35 PM Location 4
9:50 AM Location 3 8:45 PM Location 6
10:55 AM Location 4 8:55 PM Location 2
11:00 AM Location 5 9:00 PM Location 1
10:15 AM Location 1 8:10 PM Location 5
10:20 AM Location 2 8:15 PM Location 4
10:35 AM Location 3 9:25 PM Location 6
11:35 AM Location 4 9:35 PM Location 2
11:40 AM Location 5 9:40 PM Location 1
...and so on for about a hundred lines.

Though it’s bad, it’s a bearable starting place because it’s consistent in at least one way. For every time, the location that follows it is correct. The ideal CSV output would have the locations as the first row and the times for each location in the correct CSV column on the following rows.  Of course there are some hurdles here. In the PDF paste, there are two columns because the PDF was in a two column layout and whoever created it decided to just stretch one table across the page rather than created two separate tables. So, step one is to get the columns cleaned up. Easy enough, right? After all, I know VIM MACROS!!!

It’s macro time.

I actually didn’t know much about macros in Vim, but I’ve used them in other contexts. The basic idea is that a macro is a recording of some actions that can be replayed. There are some basic rules one has to know to get full value out of macros. As far as I can tell, there are two:

Remember the computer has no intuition so it must be told exactly what to do

Maybe this is obvious, but it’s worth reiterating that when you, as a human, say “select this text and do this with it” in your head, you’re glossing over tons and tons of steps that a computer will have to be told. To write a macro, you have to break down complex actions into each of the sub-actions until you arrive at steps simple enough for the computer to do it. Baby steps.

Use the most general commands possible to get the effect you want

Once you have simple steps the computer can handle, you have to figure out how to make those action work on all the text you want to modify. For instance, I could read the first line of my data and say, “There are 19 characters in the first column. Computer, go forward 19 characters to the start of the second column.” The problem is that this is a specific rather than a general command. It’s specific to the first line only. If you skip ahead to the fourth line, you can see it wouldn’t work correctly. Instead, I have to come up with a general way of getting to the position I want. This will require finding something that is true of every line of the data and using that consistency to my advantage. In this dataset, I know that every line has exactly two colons in it. I can also see that the second column always starts at the first space before the second colon. I can use this consistency to write general commands that always put me in the column space. As you can probably tell, this part of macros requires some cleverness.

Keeping these two rules in mind, here’s exactly how I think my macro should work:

  1. Move to the beginning of the line
  2. Set a marker at the current line
  3. Jump to the second column
  4. Delete from the start of the second column to the end of the line
  5. Jump to the bottom of the file
  6. Create a new line and paste the column data
  7. Jump back up to my marker
  8. Go down to the next line

These are pretty simple steps, but the real test will be converting them to Vim commands. As Vim commands, this is all pretty straight-forward except for the “jump to the second column” part, but I’ll explain it in detail below.

Here’s what the guts of my macro are going to look like, starting from visual mode (i.e., not Insert or Ex mode) on the first line of the data.

  1. 0 Jump to the beginning of the line
  2. mm Create a marker labelled “m”
  3. f:f: Jump forward to the second colon; my cursor is now in the middle of the time string in the second column
  4. bb Go back to the beginning of the word, then go back to the word before that, this puts my cursor on the number character at the end of the first column
  5. f<space> Forward to the space; my cursor is now right where I want it, in between the two columns
  6. d$ Delete from the cursor position to the end of the line; the text is in-memory and I can paste it somewhere else
  7. G Jump to the last line of the file
  8. o Create a new empty line, which puts me in Insert mode
  9. <escape> Go back to Edit mode
  10. p Paste the data into the line
  11. 0x Jump to the beginning of the line and delete the space that I don’t need
  12. 'm Jump back to the line I started the process with
  13. j Go down to the next line to start the process over

Cool, I have steps that can be repeated line after line. Now it’s time to hit the docs. According to the Macros page on the Vim Tips Wiki, to record a macro, hit “q” and then a label key, just like with markers. So, I choose “s” as my label. It’s a good, memorable letter. I now type “qs”, which gives me the familiar “recording” text at the bottom left of my editor. Only this time, I actually intend for it to show up. Huzza. I’m a pro.

If I’ve designed my macro correctly, I should be able to follow it exactly, then hit the “q” key to stop recording. Of course, as a frail human, I botched the steps up the first time and had to start over. Sad panda. Life goes on. On the second try, I got the commands right. Hitting the “q” key finishes the macro.

It’s the moment of truth. I’m on the second line of my data set. The first line has already been converted. According to the macro documentation, if I want to replay my macro, I hit “@” and the macro label. Drumroll. I hit “@s” and instantly my cursor is on the third line and the second line has been converted in exactly the way I want. At the bottom of the file, the second column of the second line has been appended. It’s magic!

I know I started the process willing to do all that work myself, but now I’m spoiled. The computer is my bitch and I want it to get me some cookies and milk while I lay on the couch basking in my genius. Translation: I realize that I’m still going to have to type “@s” a bunch of times and I’m sad. There has to be a better way! And, reading the documentation further, it turns out that you can replay a macro as many times in a row as you want by declaring the number of times to repeat the “@” then the label key. So, doing some quick math, I see I have 116 more lines to convert. I type “116@qs” and BAM! The file is now one column of data. Macros are awesome.

When you think about it, if you’re doing work in Vim already, macros are pretty much just a recording of the steps you would take anyway. From now on, I’m going to think about every editing procedure I do in terms of simple, sequential actions and general commands.

The time it took me to devise the macro plus the time it took me to get it recorded was about 15 minutes, including the time it took to read the docs. Sure, that’s longer than it would’ve taken to do the conversion by hand. But, I got smarter in that time. Manual work would’ve made me more inclined to take the route of drudgery, which I’ll submit is the equivalent of getting dumber. Fifteen minutes spent learning and applying a new skill is definitely a more leveraged use of the time than reapplying a well-worn skill. If you factor in the damage done by the error I probably would’ve made doing it all manually as well as the time it would take to find and correct the error, learning to use automation is easily the cheaper way to go. The next time around it’ll be an even better option. Speaking of the next time around, now I need to create a macro to convert this one-column state I’m in into a multicolumn file using CSV format. This is going to be be fun.