Ve Half Spies Efreyvare, Und You Vil Lak It

The problem with quants, well among many problems, is that they tend to be disassociated from the real world. What's worse, so are their managers. The Great Recession happened just because neither the quants nor their managers paid any attention to the basic metric: the ratio of median house price to median income ranges over a very small interval in a stable economy. Deviation from this interval is prima facie evidence for either bubble or bursting. There is no middle ground.

We saw this play out, as it still does. Today's NY Times Magazine carries a long form article of another aspect of quants living in a bubble (Maher variety; if you don't watch Maher, you're missing the best 60 minutes of the week). This time, it's the quant's (the story is about a single individual) isolated self-interest and that of his bosses' which lead to problems. Not so bad as The Great Recession, although possibly worse. Let me explain.

In writing "1984", Orwell told a tale of gummint intruding on the lives of citizens. "RoboCop", on the other hand, tells the tale of corporations establishing de facto control of society. The Right Wingnuts' boogeyman is Washington, DC, while the Left Wing points (when it bothers to respond) to corporations. The Right Wingnuts complain about activist, unelected judges, but install those which promote the corporation at every opportunity. There is no constitutional advantage to corporations in either the original text or amendments. The "rights" of corporations to act as persons were not enacted, but granted by fiat by various Supreme Courts. It's no secret that Mussolini enshrined this process in the word fascism.

What we have with Target is an example of corporate data gathering and computing power turned against citizens. Is that too strong? I think not. Manipulation of consumers is not something envisioned by the apotheosis of Right Wing theology, Adam Smith (the real one). They lay claim to Smith, but ignore his notion of smallness. His notion was that each player in the economic game could only affect a limited number of other players, and that concentration of power wouldn't, and couldn't, happen due to capital and labor shifting to revealed profitable venues. Smith would today be considered a micro-economist; where the unit of interest is the individual, company or landlord or worker. The notion that there is such a thing as macro-economics came later with Fisher (mostly of the crackpot strain) and Keynes. When I was a grad student, the "macro is just the sum of the micro actors" notion had infiltrated, largely as a result of mathification of the study. It's much easier to build math models when the object of interest is some notion of individual; aggregation thus becomes macro-economics. That's poppycock, and the main reason I ran for the exits without the Ph.D.

If one reads the business literature (I use the term broadly), the most important advantage to be sought isn't a better mousetrap, but rather a deeper and wider moat around one's existing business. The intellectual property and patent regimes are clearly out of control when Apple can be granted a patent for a rectangle; yes, that is true.

I abide a deep ambivalence for the hero of the Times' article. On the one hand, using data to solve problems is a really neat job. When I was a Fed in the 1970's, working for what is now the Office of Personnel Management, we were tasked to devise a quantitative measure of discrimination in Federal civilian employment. McElhone, who ran the group and had the Ph.D. in math stats, told the Suits that we would need some number from them to use as the benchmark for discrimination. They didn't like the question, and my recollection is that it ended the project. Similarly with Target; what is the acceptable intrusion into the lives of "customers"? To what extent do such quant programs aid the customer? Or do they just reduce customers to lab rats, changing the reward to induce them to tap the lever ever more frequently? So, on the other hand, does solving such problems only aid those who seek to tie our fetters ever tighter?

The Tea Baggers rail against the gummint, but blithely ignore where intrusion really comes from. Psychologists call such behaviour cognitive dissonance.

Target erred in not even considering that what they were doing was an implicit invasion of privacy. McNealy and Zuckerberg, of two different generations, have disparaged privacy. When will the backlash arrive? Probably never. Once corporations gain control, they won't be forced to relinquish for the simple reason that they will have been aided and abetted by the very institution, gummint, which has the power to rein them in. It is a brave new world.

A Plan for All Seasons

Well, here we go again. The NY Fed's "Empire Manufacturing" survey results were released today, and were good. These are, of course, the seasonally adjusted numbers. Sound familiar? Well, I emailed the NY Fed (yes, one can do that) to ask how it was they do seasonal adjustment, specifically whether the weights are re-calculated each period based on the seasonal factors experienced during the period: weather, events, period length, etc. The answer I got back was to review the methodology, which they publish in some level of detail. I had planned to do that anyway, but a simple "yes" or "no" would have sufficed to answer my question.

Although I didn't get a simple answer, I did get a couple of links to Financial Times discussions of why seasonal adjustment may be a bit wacky these days. For those that don't follow links, the conclusion is that the Fed has adjusted the adjustments (that's a job I'd like: data chiropractor). For those who might not know: seasonal adjustment isn't, so far as I can find, done by measuring data in the period for some set of seasonal factors (weather, holidays, business days in period, etc.), then calculating the period's seasonal weights. Rather, standard practice, since I was in school and long before, is to wave a statistical wand over recent past periods, and coax out seasonality. These links deal with this, possible, bias: the Great Recession happened during autumn and winter (Northern Hemisphere) and the standard algorithms may have bled some part of the Great Recession into seasonality. Using such weights now will boost unadjusted data above where a longer term seasonal adjustment would (or one not using Great Recession data at all; aside, I just got a bit further into the second link, and find this "The second approach is to excise the financial crisis period (specifically, omitting one year of data beginning just before the Lehman bankruptcy) and estimate seasonal factors using this series." Great minds run in the same gutter.). The point being: seasonal adjustment is still fixed weights from past data.

For those interested in what BLS is up to, here's the document.

Or, to quote from the second link, below:
"These biases exist because the computational techniques used to seasonally adjust economic data inappropriately interpreted some of the downturn in the fourth quarter of 2008 as a new seasonal trend."

So, we have two possible upward biases.

Here:
http://ftalphaville.ft.com/blog/2011/12/20/806221/tis-the-seasonality-hold-the-jolly/
http://ftalphaville.ft.com/blog/2012/01/04/817881/

Even if you're not an economist or other data life form, it makes for interesting reading. Interesting in the Chinese curse sense, that is.

Nuke 'em, Danno

This endeavor has been constructed upon a foundation of a few themes. Chief among these is the stupidity and self-importance of poor white folks. Such behavior is explained in magnificent detail by Robert Frank (whom I've recently seen on BookTV, and he's not Sy Hersh's age, after all; kind of a young guy) in "What's the Matter with Kansas?".

Today these hypocrites get totally nuked by a three page (!) front page article in the Times. I generally just post a link, and shut up, but the money quote is way too tempting to let go:

"It's hard to beat up on the government when they've been so good to you," he finally said. "I've never really thought about it, I guess."

That's said by a guy who basically lives off the gummint, but says he's a conservative. These Red State Washington haters are the most benefited by Washington (just look at those beautiful maps). Always have been, going back to at least TVA under FDR. Hypocrites.

Lies, Damn Lies, and the BLS [updated]

That's Bureau of Labor Statistics. The headline over the column in my dead trees version of the NY Times reads: "Stagnant Job Growth is Expected in Report". Yet, the number this morning was glorious. How does that happen? Why did the numbers come in so much better than expected? Were the numbers figured? Let's see whether we can find out.

First: these numbers are estimates from sample surveys. The only population number related to employment is the weekly UI filing number. Everything else is an estimate from some kind of survey.

Second: as every fourth grader knows, a percent is just a decimal fraction with the point shoved over two places to the right. And every fourth grader knows that a percent goes up whenever the numerator goes up or the denominator goes down. Or both.

This is real time, in the sense that I haven't looked at the numbers. I'm going to go out on a limb and say that, at least, the denominator went down. That number is the estimate of the total labor force. That number has been declining through 2011 - 2012 (here is the table and this is the almost raw data), the difference being seasonal adjustment. Seasonal adjustment of economic data is still controversial.

Now, have a look at the next table. What we see is that the number not in the labor force is up, as is the number working multiple jobs.

Table A-15 is where the doubters gather. U-6 is the number cited by both Left Wing and Right Wing sympathizers as justification for either blaming all those lazy poor people who can't seem to stay put in Mitt's safety net, or blaming Mitt and Friends for slicing away at the safety net. The not seasonally adjusted is the number which matters. And it's up a tad from December to January.

I rest my case.

[update]
Today's Times has a longer piece, complete with interviews, which I don't get to do from my drafty Frost Belt garret. As well, I see that I wasn't sufficiently explicit about seasonal adjustment. What I expect happened is that the adjusted numbers (the ones nearly always quoted) overstate the level of employment. One of the reasons to adjust the numbers is truly seasonal: weather is lousy here in the Frost Belt in January, and the purpose of adjustment is to make a level playing field, month to month. Since the Frost Belt last month was more like October (the January "season" was largely absent), the quoted number got a double dose. I don't expect to see it continue, alas.