Friday, 11 July 2014

Indications of vindication

It's always cool to have a big name scientist agree with you, especially if that big name scientist is Scott Aaronson. Here's Scott's thoughts on funding for quantum computing:

What happens when it turns out that some of the most-hyped applications of quantum computers (e.g., optimization, machine learning, and Big Data) were based on wildly inflated hopes—that there simply isn’t much quantum speedup to be had for typical problems of that kind, that yes, quantum algorithms exist, but they aren’t much faster than the best classical randomized algorithms?  What happens when it turns out that the real applications of quantum computing—like breaking RSA and simulating quantum systems—are nice, but not important enough by themselves to justify the cost?  (E.g., when the imminent risk of a quantum computer simply causes people to switch from RSA to other cryptographic codes?  Or when the large polynomial overheads of quantum simulation algorithms limit their usefulness?)  Finally, what happens when it turns out that the promises of useful quantum computers in 5-10 years were wildly unrealistic?
I’ll tell you: when this happens, the spigots of funding that once flowed freely will dry up, and the techno-journalists and pointy-haired bosses who once sang our praises will turn to the next craze.  And they’re unlikely to be impressed when we protest, “no, look, the reasons we told you before for why you should support quantum computing were never the real reasons!  and the real reasons remain as valid as ever!”
In my view, we as a community have failed to make the honest case for quantum computing—the case based on basic science—because we’ve underestimated the public.  We’ve falsely believed that people would never support us if we told them the truth: that while the potential applications are wonderful cherries on the sundae, they’re not and have never been the main reason to build a quantum computer.  The main reason is that we want to make absolutely manifest what quantum mechanics says about the nature of reality.  We want to lift the enormity of Hilbert space out of the textbooks, and rub its full, linear, unmodified truth in the face of anyone who denies it.  Or if it isn’t the truth, then we want to discover what is the truth.

Sound familiar? A while ago I posted the following:

With this in mind isn't it a bit...I don't know, shady of us, to be collecting money under the pretenses of maybe-eventually-possibly producing some kind of new technology that'll likely never arrive? No, I'd rather make the case for funding fundamental physics research simply as it is, without having to dangle a carrot in front of the public's nose. For one, it's more honest, which I tend to be in favour of. For another, it avoids backlash - after all, if we promise miracles, we had better darn well deliver them (I doubt the people will be so quick to grant us a 2000 year grace period). And for yet another (if ethical qualms don't move you) it allows physicists much more freedom in their research: freedom to explore, to investigate, to follow the winds of evidence wherever they lead, even if it's away from application.

I'm not sure if I totally agree with everything I wrote in that post anymore, but that part I definitely still think is true (and so does Scott, apparently). We need to be honest when we ask the public for science funding, for reasons both ethical and pragmatic. We really don't know when basic science research will pan out in terms of practical applications, and pretending otherwise will only come back to bite us in the ass.

[epistemic status: gloating]

I have a lot of doubt too! I'm sure of it!

This is a very good article. You should read it.


...have you read it yet? I'll wait.

Okay, good. So the article (which I'll now summarize, assuming you didn't read it) asserts that one of the major factors limiting women's success in the workplace has been a simple lack of confidence. The basic problem seems to be that people in general are extremely susceptible to displays of confidence. If someone claims to know what they're doing, and they say it confidently enough, it seems to be hardwired into our brains to just believe them by default. The person doing the claiming does not in any way have to actually know what they're doing, of course - but they do have to think they know what they're doing, in order to be able to say it confidently enough. And so people who are overconfident (or just regularly confident) tend be more successful in the getting promotions or raises or whatever, and (surprise surprise) it turns out that men are way better at being overconfident. Hence, the glass ceiling.

Like I said, I enjoyed the article and thought it was very good. It's both well-researched and surprisingly in-depth. And it's also a very hopeful article - if it's only underconfidence holding women back, then maybe the gender imbalance won't turn out to be as hard to fix as we thought. Which would be pretty great! But that being said, I did have a few issues with it.

First, the boring stuff. It really bugs me when people stretch the truth to make their points, especially when they have a good point to begin with. It jumps out at me immediately and tends to make me less inclined to listen to them than if they had just given the boring old unembellished facts. The "stretching the truth" metaphor is surprisingly apt, actually - to me it really does feel as if something physical were being stretched; being forced into a state it shouldn't be in. It's become almost an aesthetic thing for me at this point - it gives me a sort of vague, unpleasant twinge, like I imagine a car person would feel if I were driving in too low a gear. I wish people wouldn't do it.

Anyway, this article isn't particularly bad or anything. But a few things stuck out. For example:
The shortage of female confidence is increasingly well quantified and well documented. In 2011, the Institute of Leadership and Management, in the United Kingdom, surveyed British managers about how confident they feel in their professions. Half the female respondents reported self-doubt about their job performance and careers, compared with fewer than a third of male respondents.
My god! Half of female respondents, you say!? And fewer than a third of male respondents? Why that's...

...50% vs. something like 30%. Huh. When you say it like that, it doesn't sound quite so impressive.

I see this fairly often in articles, where statistics pretty-much-but-don't-totally support the author's point, and they can't not include statistics because otherwise annoying scientists would nitpick about little things like "lack of evidence" or "having no factual basis in reality". So they put the statistics in the article but do it drive-by style, sandwiching them in between anecdotes, and dressing them up with phrases like "more than [impressive-sounding fraction]" and "less than [small-sounding fraction]" instead of using plain old numbers. [I should point out that this approach is exactly backwards, of course - statistics should be the focus of an article like this, rather than an afterthought. In an ideal world an opinion piece would live or die based on the extent to which it was supported by data, and pieces written without substantial empirical backing would be looked at, not with contempt, but with confusion. But this is not that world, and that's not really my main point anyway.] The issue I have here isn't so much that statistics are marginalized, it's that the statistics they do include are in tension with the main point of the article, and that isn't acknowledged.

I mean, don't get me wrong, the data obviously show a difference between men and women. I'm sure it's significant in both the statistical and ordinary senses of the word. But to me it almost looks as if there's...plenty of guys who experience self-doubt, and plenty of girls who don't? Which is of course what you would expect, but the article goes to great lengths to gloss over this fact, with quotes like:
Currie rolled her eyes when we asked whether her wellspring of confidence was as deep as that of a male athlete. “For guys,” she said, in a slightly mystified, irritated tone, “I think they have maybe 13- or 15-player rosters, but all the way down to the last player on the bench, who doesn’t get to play a single minute, I feel like his confidence is just as big as the superstar of the team.” She smiled and shook her head. “For women, it’s not like that.”
 “I think that’s really interesting,” Brescoll said with a laugh, “because the men go into everything just assuming that they’re awesome and thinking, Who wouldn’t want me?”
And then paying lip service to the idea that men have doubts, but seemingly dismissing them as different not just in severity, but in kind:
Do men doubt themselves sometimes? Of course. But not with such exacting and repetitive zeal, and they don’t let their doubts stop them as often as women do.
My point is: the trend is enough! If underconfidence hurts career outcomes and women tend to be less confident, then that's a super important fact all by itself. There's no need to pretend that 100% of women doubt and 0% of men do. And in fact, because you so helpfully included the data, you can't pretend - 30% of men experiencing self-doubt about work is a sizable (and last I checked, non-zero) fraction of the population. Less than women, sure, but still sizable. There's no need to exaggerate - you have a good case!

Truth. Stretch. Twinge.

I'll come back to this idea, because I think it's important. But first, another boring observation. Here's a very confident-sounding sentence that appears - in bold and large font - as one of those floating quotes in between paragraphs:
In studies, men overestimate their abilities and performance, and women underestimate both. Their performances do not differ in quality.
This is followed up with another very confident quote:
“It is one of the most consistent findings you can have,” Major says of the experiment. Today, when she wants to give her students an example of a study whose results are utterly predictable, she points to this one.
Now, seeing as this is a highly consistent finding, I would naturally assume that it (that is, men being overconfident) holds true for pretty much any study you can find. At the very least, I would assume that studies included in this article would follow the trend. So I can't resist pointing out that these quotes come only a few paragraphs after the following:
The women rated themselves more negatively than the men did on scientific ability: on a scale of 1 to 10, the women gave themselves a 6.5 on average, and the men gave themselves a 7.6. When it came to assessing how well they answered the questions, the women thought they got 5.8 out of 10 questions right; men, 7.1. And how did they actually perform? Their average was almost the same—women got 7.5 out of 10 right and men 7.9.
...which is an example of men, in a study, underestimating themselves! Sure, they underestimated themselves less than women, but it was still an underestimation. Even if this study is a total anomaly, and all other studies shows overconfidence by men...well then, for one, why did you pick this study, and for another, why didn't you acknowledge the contradiction that this creates? Again: piece of the article, in tension with another piece, being ignored. It bugs me.

Anyway, so that's the sort of mundane stuff. That's not the real reason I'm commenting on this article. The real reason I wanted to comment on this article is because I identified with it so much. As I was reading it, over and over I would think: that's me! I do that! I feel that! I tend to have a lot of self-doubt, I'm very sensitive to criticism, if I weren't in the bubble that is grad school I would totally be all passive about asking for raises and promotions, and hesitant to apply for jobs I didn't have 100% of the qualifications for. Point is, I felt like the article was describing me. And, uh (last I checked, anyway) I am not, in fact, female. No matter how much Matt calls me his bitch.

I'll be honest, this made the article very frustrating to read. Not just frustrating - kind of hurtful, actually. I felt like the article was saying that, as a man, I couldn't feel the way I do. Either my feelings were simply nonexistent, or they were invalid. I felt marginalized, if I can say that without sounding too melodramatic or MRA-ish.

Now, I get it. I really do. I'm obviously not a typical guy, and women really do have a lot of disadvantages in the workplace. And if an article comes across as wishy-washy and stops to include disclaimers every half-paragraph, people won't take that as evidence that a complex and nuanced issue is being discussed - they'll just mentally chalk it up as a draw, and probably ignore the article altogether. So it's likely in an author's best interest, generally speaking, to make it seem like things are more one-sided than they really are, in order to get their (very legitimate) point across.

Still, though. Still I'm annoyed. I mean, let's go back to those numbers for a second. Earlier, we had the lovely statistic that half of all surveyed women reported experiencing self-doubt about their job, and fewer than a third of men did. Fine. Clearly, the numbers already "favour" women in this case (in the sense of them being more underprivileged), but let's say even that is an underestimate. In fact, let's pretend that those numbers are total bullshit. Instead, we'll charitably assume, for whatever reason, that women drastically under-reported how much they experience self-doubt, and men drastically over-reported (I don't think this makes any sense really, given that men experience a general cultural pressure to not display weakness, but who cares). Let's say that the real numbers are that 90% of women have self-doubt about their job, and only 10% (say) of men do. Surely, in this hypothetical case, it would make sense for society to focus on women, right? I mean, 90% versus 10%? Come on. Given the vast disparity between the genders, it would behoove us to simply forget men for a second, and just try to encourage girls to be more confident, yes?

No. No no no.

I can imagine different worlds. I can imagine a world in which we simply couldn't tell if a given person was overconfident or underconfident. A world in which we weren't able to discern who had a growth mindset and who had a fixed mindset. A world in which personality tests didn't exist, in which we were helpless to distinguish between different people, in which we were forced to reason based on correlates.

This is not that world.

We do have ways of distinguishing between people. We can administer questionnaires, we can conduct surveys. Heck, we can just look at people's behavior. We do not just have to go by the second-best method of assuming everyone who has the same gender has the same level of self-confidence. If you assume that all girls are under-confident and all boys are over-confident, you will automatically be wrong by a factor of (at the very least) 10%, guaranteed. You will encourage the 10% of girls who are overconfident to be even more confident, and the 10% of boys who are underconfident to be even less confident.

This, while perhaps better than nothing, is far from ideal (and remember, I'm being generous with the percentage estimates). The problem starts the second we try to frame the issue in terms of men versus women. It may seem to make sense, given the strong correlation between gender and confidence level in this case - but remember, the correlation is not perfect. Grouping by gender is quite simply not a natural category when it comes to self-confidence levels. Instead, we should be grouping by...well, by those who are underconfident and those who are overconfident! Remember, we have science! We can measure things! A much, much, better approach would be to devise a test that measures levels of self-confidence (I'm assuming this shouldn't be that hard, given what we know so far about the subject), administer it to second-graders or whatever, and then do [whatever you were going to do to encourage girls to be more confident] to [all people who displayed signs of low self-confidence on the test, regardless of gender]. Most of those people would be girls, sure - but not all of them! Some of them would be boys (like me!) who would no doubt greatly benefit from the intervention. God knows I would have done well to read Carol Dweck at an earlier age.

This seems to me to be an example of one of my least favourite fallacies, the ecological fallacy (if there were one piece of understanding I could force into everyone's brain in the world, it might be this - that, or the obvious superiority of candlepin bowling). The ecological fallacy is when one assumes that because on average a group has a certain trait, a given individual in that group must have the trait as well. Kind of like how no woman has ever been taller than a man, because girls are on average shorter than boys. Or how most politicians tend to be men, so Angela Merkel doesn't exist. Or how I'm forced to believe that every single person has one breast and one testicle, because, hey, that's the average.

Point is, it's a really stupid fallacy. The real world deals with distributions, not averages. Two groups can have overlapping distributions while still having different means. It's not that difficult really. I can't tell you how many times I've seen a study claiming something innocuous like "Left-handed people are more likely to use fabric softener" and the first comment is someone saying "This study is obviously bullshit my dad is left-handed and he's never used fabric softener in his life, in fact he actually uses fabric hardener why are scientists so dumb" and then acting as if the study has been completely invalidated. I...find it hard to grok what's going on in people's minds when they say things like this. I hate to be uncharitable, they literally not know what scientists mean when they say "tendency"? There certainly seems to be some kind of understanding gap, because I can't imagine ever trying to refute a study that claimed a between-group difference by pointing out a single counterexample that I had at hand. You would expect counterexamples! It would be surprising if there weren't any counterexamples! Maybe in rare cases, when the study makes a very strong claim, like that 99.9% of all left-handed people use fabric softener, and you personally know that all five of the lefties you've met have never touched the stuff - then yeah, maybe be a bit suspicious. But when the study is only claiming small effect sizes, like less than half a standard deviation (which, let's face it, most studies are), then just bite the bullet and accept that, due to your own very small and unrepresentative sample of the population, you are incapable of discerning the truth of the matter. Just trust the goddamn study, even if it goes against what you've observed in the world so far. That's what studies are for.

Alright, I've ranted enough. What are the takeaways here? Well, underconfidence is a significant problem in women (and a much less significant problem in men - or rather, a just-as significant problem in men, but for a much smaller fraction of the population). People (in general) who are underconfident should work to combat that for the sake of their future success, either by consciously correcting for the bias, working to expand their comfort zone, or reading Carol Dweck. Journalists should try to pay more attention to statistics and try extra hard to stop exaggerating things, mostly for my sake. People (in general) should read the wikipedia article on the ecological fallacy every tuesday morning, mostly for my sake.

And everyone should switch to candlepin bowling. Mostly for my sake.

Seriously, it's so much better.

[A blog I read has the habit of tagging all posts with an epistemic status. This seems like a really good idea. So, epistemic status for this post: reasonably confident, but much less so than the very strong tone throughout would imply. In fact, that's probably a safe assumption for almost all of my posts. I'm...not very confident, you see. Even if I sound like it.]