academia | advice | alcohol | American Indians | architecture | art | artificial intelligence | Barnard | best | biography | bitcoin | blogging | broken umbrellas | candide | censorship | children's books | Columbia | comics | consciousness | cooking | crime | criticism | dance | data analysis | design | dishonesty | economics | education | energy | epistemology | error correction | essays | family | fashion | finance | food | foreign policy | futurism | games | gender | Georgia | health | history | inspiration | intellectual property | Israel | journalism | Judaism | labor | language | law | leadership | letters | literature | management | marketing | memoir | movies | music | mystery | mythology | New Mexico | New York | parenting | philosophy | photography | podcast | poetry | politics | prediction | product | productivity | programming | psychology | public transportation | publishing | puzzles | race | reading | recommendation | religion | reputation | review | RSI | Russia | sci-fi | science | sex | short stories | social justice | social media | sports | startups | statistics | teaching | technology | Texas | theater | translation | travel | trivia | tv | typography | unreliable narrators | video | video games | violence | war | weather | wordplay | writing

Wednesday, May 30, 2018

"I wasn't mad at anybody"

Most of the US veterans in my family tree fought for the south in the Civil War. That's a hard legacy to know how to understand and relate to.

One man in my family tree back then was, as I recall being told, a doctor. He never enlisted in the Confederate army, which was something of a source of public shame. I don't know his full reasons, but when asked why he hadn't fought (or served as a field doctor), he apparently would answer, "Well, I wasn't mad at anybody."

This was met with scoffs, scorn, and eyerolls. Of course, an even better reason would be to recognize that the cause was white supremacy. But when deeply limited people are called to war, the world would be a better place if more people would refuse, who at least don't actively believe in the war cause.

"I'm not mad at anybody" is a perfectly good reason not to wage war, and I wish it would were felt, and cited, more often.

Labels: ,

Tuesday, May 22, 2018

Why airplanes stay in the sky: an epistemological window

Growing up, I was told by several science teachers that airplanes obtained lift via the "equal transit-time" phenomenon (from Wikipedia's "Airplane" entry):
...the "equal transit-time"... states that the parcels of air which are divided by an airfoil must rejoin again; because of the greater curvature (and hence longer path) of the upper surface of an aerofoil, the air going over the top must go faster in order to "catch up" with the air flowing around the bottom. Therefore, because of its higher speed the pressure of the air above the airfoil must be lower.
Turns out this is basically false. See these pictures for evidence that air does not need to rejoin at the same place at all, though of course that doesn't mean that no air acceleration occurs at all.

As Wikipedia points out in its article on lift:
Such an explanation would predict that an aircraft could not fly inverted, which is demonstrably not the case. When an aircraft is flying inverted, the air moving over the bottom (in the aircraft reference frame) surface of the airfoil is moving faster. The explanation also fails to account for airfoils which are fully symmetrical yet still develop significant lift.
It is unclear why this explanation has gained such currency, except by repetition by authors of populist (rather than rigorously scientific) books, and perhaps the fact that the explanation is easiest to grasp intuitively without mathematics.

For my money, the best explanation is from "See How it Flies", an online book about airplane physics. Here's the crucial part, which gets into circulation and vortexes (OK, "vortices") as not just byproducts of airplane lift, but an indispensable aspect of it. I highly recommend following the author's suggestion to explore how a curveball (or a tennis ball hit with backspin) uses circulation to move laterally through the air; you can demonstrate this using any ordinary business card:

Drop the card from shoulder height, with its long axis horizontal. As you release it, give it a little bit of backspin around the long axis. It will fly surprisingly well...
Jef Raskin, who started the Macintosh project within Apple, has a great page about what makes airplanes fly, which gets into curveball aerodynamics, upside-down flight, and the wing Albert Einstein designed for the Luftwaffe in WWI.
...the most common explanation of lift seen in elementary texts and popular articles today... is based on the Bernoulli effect, which correctly correlates the increased speed with which air moves over a surface and the lowered air pressure measured at that surface.

In fact, most airplane wings do have considerably more curvature on the top than the bottom, lending credence to this explanation. But, even as a child, I found that it presented me with a puzzle: how can a plane fly inverted (upside down). When I pressed my 6th grade science teacher on this question, he just got mad, denied that planes could fly inverted and tried to continue his lecture. I was very frustrated and argued until he said, "Shut up, Raskin!" I will relate what happened next later in this essay. A few years later I carried out a calculation according to a naive interpretation of the common explanation of how a wing works. Using data from a model airplane I found that the calculated lift was only 2% of that needed to fly the model.

Jef, you had me at pissing off your 6th grade science teacher!

So, why does the equal transit-time myth persist, to the detriment of the much simpler explanation of angle of attack? I think part of the explanation must be that it makes a good story; another part is that a full understanding of all the dynamics at play is quite out of reach of the typical grade school science teacher, at least in the know-nothing United States. Incidentally, the question of airplane lift makes for a good shibboleth for distinguishing between those who think through scientific questions from first principles, and those who just go by What they assume to be conventional wisdom. I'm not claiming I understood this before I encountered the dispute recently, but as soon as someone pointed out the problem of an airplane flying upside down, I realized my understanding was at least incomplete.

Labels: , ,

Toyota acceleration: harbinger of the future

Yesterday a Toyota Prius accelerated wildly on a highway before police managed to coach the driver into stopping. What was their solution? To slow down by using the emergency brake and regular brake, and then put the car into neutral.

I'm more disturbed that the driver didn't think to put the current into neutral then I am by the acceleration problem itself. How many drivers of automatic cars don't really know what the neutral gear does? I don't recall reading whether any Toyotas with manual transmissions have had acceleration problems, but I can't imagine many such drivers calling 911 from their accelerating car instead of engaging the clutch and shifting out of fifth.

As our cars, in our lives, become run increasingly by software and digital logic, we are losing the ability to cope when things break down. Steve Wozniak, one of the founders of Apple, complained publicly earlier this year -- before the Toyota recall -- that his Prius was prone to accelerating out of control. Woz comes from a different technological era; he built much of the first Apple computers by hand while driving a VW that he could repair himself. What does it mean that even Woz is so distanced from the workings of his vehicle that he doesn't understand why it does what it does, and has no reasonable possibility of ever understanding?

Labels: ,

Collecting organizational knowledge: a hybrid approach

Stack Overflow recently announced Stack Overflow for Teams. I'm hopeful that it will be what you wish your company wiki (that nobody uses) was.

I'm a big believer in starting with tools that people actually use, and it just makes so much more sense for a project manager to post a question on Stack Overflow than to conventionally ask the team, "Can someone write up how the new AWS deployment works?" Somehow, answering the query through Stack Overflow's interface and conventions makes it seem so much easier to me as an engineer; maybe it's that the format puts direct information ahead of fancy or comprehensive documentation?

This ongoing question of how to collect an organization's knowledge is endlessly fascinating to me. I think most organizations should employ a librarian specifically to do this, by interviewing people verbally (it so much easier to answer verbal questions than written ones), following discussions, and creating an internal newsletter. (Compass, the tech-boosted real estate brokerage, has a full-time internal journalist who does this.)

Discussing S.O.F.T. with friends, we were wondering how close this sort of information collection is to being able to be done implicitly, and on the fly, rather than explicitly and deliberately, in advance of the time that knowledge is requested.

For instance, Slack famously claims in their promotional interviews and writeups that explicit documentation should be a thing of the past, because Slack can use search to source relevant comments and explanations. In practice, I think it's widely agreed that this doesn't work well.

One friend wondered if machine learning was close to being able to collect documentation, either just in time or by preemptively collecting it into reports. I think there's too much complex context to do that well using a pure machine learning approach, at least for the near future.

But what I do think is possible, if not now than at least very soon, is a hybrid approach where machine learning augments deliberate human work, by sourcing suggestions that prompt humans to pick up the ball and run with it. An AI-curated list of likely useful links and excerpts wouldn't replace human editing and summarization, because it wouldn't try to; it would just make that process shorter, most of the time.

For instance, take a slightly different domain--AI-assisted scheduling, which is far behind where glitzy startups like x.ai would like you to believe. While I don't think machine learning is good enough yet to handle scheduling communications by itself, I do think it would be reasonable to train a model to look at your email history and calendars for events that appear to be calls or in-person meetings, and to make a Gmail widget that suggests times accordingly, but also lets you just pick them explicitly. That approach would make the system's worst errors inconsequential, while letting it save you time in the typical case. It's the sort of thing that I think people might actually use, without feeling like they are struggling against the machine.

Getting more ambitious, what about a librarian service that has plugins for Slack, Gmail, Asana, etc., and steadily brings notable items to your attention? First and foremost, it could make it very easy to flag snippets of conversation or changes of product requirements as having long-term informational value. Second, it could respond to your choices by helping you navigate among possible related items, by shifting the thresholds for showing items as you accept and reject suggestions. Someone in a project management role who doesn't have a deep technical background might still be able to produce helpful collections of info, even if they require someone with deeper technical knowledge to apply context and tie the information together.

Then, when, say, someone on a Slack conversation asks if we are still using X, that could get flagged by the system and reasonably turn into a note on the document that details the active stack for that project. There would still be false positives and false negatives, but the incidence of both would be reduced by bringing in humans in the middle.

At least for the immediate future, this sort of thing is much more about sophisticated human UX design then it is about advanced ML; and I suspect that will be true in many domains for a long time.

Labels: , , ,

Monday, May 21, 2018

AI is far behind where corporate visionaries tell you it is

An internal Google video about machine learning from 2016 recently leaked. It's a compelling watch.

There's nothing in this video's theory that I disagree with, but I think it overstates how good recommendation analysis is currently.

The University of Minnesota runs an experimental movie recommendation site, Movielens, which represents a decent approximation of state of the art. I've been using it for years. It's better than just using imdb, which is biased towards what immature internet users like. And, it's still nearly useless. I often find the movies it recommends unwatchable. If I highly rate a slow movie like The Killing of a Sacred Deer, it immediately thinks I like anything unwatchably boring like I Don't Feel at Home in This World Anymore.

At the other extreme, I think it's clear that someone who deeply understands art and culture and has wide-ranging appreciation for various tastes would be able to interview another person about their tastes and recommend movies or books or whatever that they would like. So, like, if you had my wife rate all the movies she's ever seen, and then she watches several new ones that I watch separately, I think I could outperform any existing recommendation algorithm on mean squared error. (it might pick up on subtle systematic trends like "she tends to rate horror 1.3 stars less than their average" that I don't have the bandwidth to grok, but I think I would do a much better job avoiding gross misjudgments of cultural context. I know she'll like Girls' Trip way better than Pitch Perfect, partially because I'm able to bring in tons of other contextual information about how the movie's qualities jibe with or conflict with her interest that the recommender will only have indirect access to.)

So let's agree that this sort of "ambient, ongoing, intrepid recommendation" logic exists somewhere between the capability of machine learning today (let's call that t=0), and the capability of an artificial superintelligence ("ASI") with all the capabilities of a brilliant human plus fast execution and the ability to process jobs in parallel (let's call that t=1).

The question is, what is the value of t necessary to make the sort of vision in "The Selfish Ledger" practically helpful?

I think the popular consensus is something like t=0.2 . But I suspect it's more like t=0.7 .

I do appreciate the concept of a profile which has proved itself so valuable that you prefer to add to it rather than go without a profile (or start over). But the machine learning aspects of this seem wildly overblown. Google has astonishing capabilities to perform machine learning using my information now. How is it being used to help me? By recognizing spam, certainly, but I don't think my own spam categorizations make much of a difference. By recognizing the people and things in my photos and making it easier to search for them--cool, but hardly life-changing, and still very primitive. By recognizing my speech on my android phone when I'm transcribing--but it seems that there is very little learning my specific voice, or (I suspect) tracking my corrections after the fact; there's no capability to specifically train it on my voice, for one thing.

The quality of the user experience across Google products has everything to do with product design and engineering to support that design, and next to nothing to do with machine learning.

Labels: , , , , ,

Wednesday, May 16, 2018

Why I support the Palestinian cause

Hamas and Gaza:

Noura Erakat, a professor at George Mason, gave an extraordinary interview on CBS News about the currently ongoing (May 2018) Gaza border massacre.

It is rare on mainstream news in the US to hear such a cogent presentation of the Palestinian view. At the same time, I think she seriously weakens her argument by being in denial about the extent of Hamas's support, and its willingness to accept a two state solution. Though they dropped their demand for the destruction of Israel in 2017, their official policy is still "There shall be no recognition of the legitimacy of the Zionist entity".

I'm not saying they necessarily should accept an Israel that permanently makes Palestinians second class. But the terms of the divide are wider than Erakat is making them out to be.

Does Israel offer peace?

Still, there was never a choice for peace available to Hamas. Peace means that if I walk to my family's home--say, the home my family lived in until we were told at gunpoint, in 1948 were after, that we were forbidden to return--I don't get arrested or shot.

That seems like a ludicrously low bar for those claiming to want peace, but in reality it is such an unacceptable idea in Israeli politics that it has been refused by every major party for 70 years.

In other words, there has been a choice available to Israel in those seven decades, between permanent conflict (on the one hand) and risking Jews becoming a minority (on the other), and a huge majority have consistently chosen to guarantee ethnic rule even if it means permanent conflict.

This is often framed, especially in the US, as an issue purely of security. But any understanding of the history of the region shows how inadequate an explanation this is.

Before the advent of Palestinian terrorism, there were essentially no efforts in mainstream Israeli politics to accept the return of completely peaceful Palestinian families to their homes. And since, though there are thousands of families who have done absolutely everything that scolds tell them to--keep their heads down, focus on work and education, don't engage in violence--those families, some which still carry the keys to their old homes within Israel, still have absolutely no audience for their grievances.

I have Jewish ancestry, so I could move to Israel today and become a citizen, and live in their old home. They cannot, and there is no sign that they ever will be, no matter how compliant or peaceful they are. The only real reason is that I am one ethnicity, and they are another.

1948: counterfactuals

I do think both sides (and Europe and the US) made decisions in and around 1948 that really undermined the security and prosperity of their own people. Ever since, both sides have been backing further into corners because of never owning their own role in the conflict.

What seems clear to me is that if the Arab countries had won in 1948, Jews would have been largely exiled and oppressed. And unarmed Jewish people moving towards their families' homes being shot by Arab soldiers would be rightly condemned as a horrific injustice.

Seriously though, think for a moment about this scenario. If Jews had been mass evicted at gunpoint, rather than Palestinians, and continued to have their homes bulldozed and bombed, their land seized, and Palestinian settlements and exclusive roads carving up their meager remaining territory--would anyone in the US who vehemently defends Israel today criticize them? or would they see their cause as unquestionably righteous? When there would be atrocities committed by zealots on the Jewish side, would these be universally condemned, or subject to excuses?

A standard refrain is: "If instead the Palestinians hadn't demonstrated their murderous antisemitism by aligning themselves with the Arab powers that invaded without any provocation in 1948..."

As if unilaterally declaring an ethnostate that permanently disenfranchises people of a particular ethnicity, with the help of imperial powers, isn't provocation! As if a similar declaration by Palestinians now, which includes thousands of Israeli homes and farms, wouldn't be met with violence!

Regarding the partition proposal that Jews supported but Palestinians rejected:

One part of the story that seldom gets talked about is that many Europeans considered the partition plan to have failed and to be null since both sides didn't agree to it. A Swedish diplomat was sent, who came up with a plan that more Palestinians agreed with. But instead of Jewish groups engaging about it and trying to find a peaceful resolution, the Jewish terrorist group Irgun shot him to death in his car.

"A land without a people,for a people without a land"

I was raised around a lot of progressive Jewish hippies, and even they told me that there was basically no one else in Palestine when Jews colonized it.

If you read Theodore Hertzl's speculative fiction novel about the Jewish state he envisioned, Old-New Land, what's most remarkable is that there are basically just no Palestinians anywhere. (IIRC, there is a single Palestinian who's happily assimilated.) The colonial and imperialistic frame of reference was so strong that it doesn't seem to have occurred to Hertzl to even wonder about the repercussions of forcibly evicting an ethnic group from their land and seizing their property.

Ethnic rule vs. democracy

It's important to note that in the US, it's controversial to suggest that Israel uses ethnicity as a central reason for including and excluding people from its land.

But in Israel, it's not only not controversial, it's unquestionable policy. Every major party in Israel for 70 years has held as a core principle that Israel-controlled land must be kept clear from too many ethnic Palestinians--including Palestinians who built their homes and farms themselves and lived there for centuries.

When Turkey lets foreign-born ethnic Turks become Turkish citizens but won't let ethnic Armenians whose families are from inside Turkey's borders return to their hometowns, we have no problem understanding that that's ethnic cleansing. "The people in this land should be Turks, not Armenians, and we'll use guns to keep them clear of it" is ethnic cleansing, even if there is some wiggle room in the form of thousands of ethnic Armenians in Turkey who are allowed to vote.

But with Israel, people have this idea that it can't be ethnic cleansing. Why? Because they support it!

if you're paying attention, I think you must acknowledge that this idea is fundamentally incompatible with democracy. That's why it's so maddening to supporters of the Palestinian cause when we hear Israel called the only democracy in the Middle East.

Certainly, Israel is much more democratic for its citizens than any other country in the Middle East. And that is significant!

But you can't be a democracy if that citizenship itself is primarily determined by ethnicity. Israel is more properly called an ethnocracy, or maybe an ethnocratic democracy.

Proposals in the Knesset to make it a true democracy--to drop all legal preference based on ethnicity, and to restore property and residence that was seized or forbidden on the basis of ethnicity--are literally shouted down, and have been for decades. Israel is proudly, vehemently, emphatically not a democracy, to the point that those who want it to be one are treated as a laughingstock, or as enemies of the state.

But aren't there equal rights in Israel?

Some argue that this is not an ethnic conflict, but a nation-vs.-nation one.

But calling this a "national" divide, rather than an ethnic one, seems deeply misinformed. Ethnicity was used, from the start, to determine nationality. A car of people is stopped at a military checkpoint in 1948--the operative question is, what is their ethnicity? One ethnicity gets included as residents, as citizens, as recipients of state-controlled property, and if they have the deed to a home, it's respected. Another ethnicity gets told at gunpoint that they are forbidden to return; they are forbidden to be citizens; their deed is worthless; not only can't they receive state-controlled property, their property is likely to be seized by the state. Yes, there are Arab Israelis--and their roughly equal rights are great! (Though not without exceptions.) But I could become a citizen of Israel, and own and live in a particular home, before the family who built that home could--for one reason, and one reason only. It's not nationality, and it's not religion, for that matter.

But aren't equal rights impractical?

One cogent objection is that if Palestinian intergenerational grievances must be satisfied, shouldn't those of Native Americans?

I agree that if Palestinians have the right to repatriation and restoration of property, others--including Native Americans and black people--do to, in America. Yes, the amount of time passing matters; but I think it's disingenuous to act as though this is an insurmountable complexity. There are black people alive today in America who were subject to widespread theft and violence on the basis of race, especially in the 30s-60s; that's certainly close enough to me that I would feel deeply immoral if I just declared restitution impractical.

Likewise, there are Palestinians alive today--thousands--who grew up in what is now Israel. I think nothing short of cruelty or dishonesty is required if you're going to tell them you don't intend to respect their right to live and own their own homes.

As for the practicality of my proposals, I can't pretend to have wisdom into what is politically passable in the Knesset. I'm not trying to say my ideas are politically palatable. But I don't think that should get in the way of being clear-eyed about what's wrong, and I don't think holding what's right hostage to politics is morally defensible. And I don't think there's anything maximalist about recognizing the direction that seems most just, and

The bottom line

There's really only one question here: do you believe that countries' policies should discriminate on the basis of ethnicity?

Israeli soldiers are killing unarmed Palestinian children today, by the government's own admission, because Israel's leaders answer YES to this. There have been dozens of laws--from property seizure to land use rights to citizenship rights--that explicitly discriminate on the basis of ethnicity, and hundreds more that do so with the full understanding that they will be enforced only against one group, and not another.

If your answer is no, you disagree with every ruling party in the history of Israel. Welcome to the movement for Palestinian rights.

If your answer is yes, I hope you see the pain you're helping to cause in every injured or killed Palestinian child. Their suffering is the consequence of the explicit discrimination you support.

A grain of salt

When I get deep into a productive discussion with Israeli friends, it seems that in many ways we are on the same page about the status quo.

We agree that Israel's internal political reality is that systematic ethnic discrimination is not just seen as permissible but absolutely necessary; that if civilians won't go along with exile and denial of their property due to their ethnicity, at some point Israel simply has to arrest or kill them; and that Palestinians are correct in understanding that nothing short of violent force can change that. (Let's call this reality "A".)

Of course, that's not exactly how they'd put it! They talk instead about Israel's existential concerns, and the threat of demise of Israel as a Jewish state. But there's no actual daylight between their position, and mine; just a choice of focus, and, I think, a difference in how honestly they're willing to describe the situation.

I'll certainly agree that instilling equal rights without regard to ethnicity opens the door to the possibility of the Jewish population becoming a minority, and to the possibility of even more conflict and social dissolution within Israel. (Let's call this possibility "B".)

But I think there are reasonable steps towards equal rights that can be taken that are not apocalyptically dangerous, and which I posit would be more likely to increase justice and peace than to decrease them. Start a process by which a stream of Palestinian families can repatriate and have their property restored, that considers their property claims and history of violence. Announce that Jerusalem can be the capital of Palestine. Reverse settlement expansion. Stop demolishing homes of Palestinians, inside Israel and outside. (Let's call these steps "C".)

Now, there's lots about Israel and Palestine that I don't know. I don't pretend to be an expert, or to be particularly good at foreseeing events. So why do I feel skeptical that B merits so much fear?

In part, because those who publicly warn loudly about B seldom seem to be honest about A, and attentive to the practical details of C.

There are plenty of people in Israeli politics and commentary who are honest about what A and C are--and they openly celebrate A, and reject C not because it's dangerous to Israel, but because they have no intention of increasing justice for Palestinians. From what I've read and seen, they frame the matter as a simple question of power, not of security.

That is, they want to more or less clear the land of the unwanted ethnicity if it's docile; and if it's not, all the better to sell the case. They want to win, and think sharing is for faithless losers and cowards--which is what they call the Israeli peace movement. It's the mirror image of what conservative supporters of Israel accuse Fatah and Hamas of.

Could B be so dangerous that C is a bad idea, for the survival and security of Israel? Maybe. But look at the voices who warn about any weakening of A leading to B. How much should we believe and trust them? Do they have our true understanding of the situation at heart?

Show me someone who is honest about A, articulates what C would look like, and rejects C  because of the danger of B. Seriously, show me someone who does that! I suspect that they don't exist. I think C is eminently reasonable in any framework with a modicum of morality. To reject it, I think you need to either abandon morality and operate only from power principles--or lie about A, B, and/or C.

A glimmer of hope

Why does the lying about A matter?

Friends of mine who support Israel's status quo think that "People of another ethnicity: abandon your homes and go far away, or we'll have to kill you" is a horrendous thing for Palestinians to say, and an acceptable thing for Israel to say.

I think both are horrendous.

What's interesting is that given how universally that quote is agreed with in Israel, among liberals I know, there seems to be quite a lot of evasiveness and shame about saying it.

There's an opening there--a glimmer of hope.advocating for it.

I say all this because I believe in Israel

I know this stuff is complicated, but there's a narrative of the Israeli side choosing peace that you've heard so many times because there are many people lying to you.

For the record, I am a Zionist, a proud Jew, and a supporter of Israel. I wish for Israel to be a second home for me and my people, but I feel as though it has been hijacked from the start by its most violent and shrill voices. I'm part of the tiny minority who has always believed that a Jewish state must be a democratic one, and that if you are telling children at gunpoint that they can't live in their family's own home, you're being contrary to everything we believe in as Jews.

Here's to the Israel we long for, the Israel we think still can be.

Labels: , ,

Monday, May 14, 2018

Drowning doesn't look like drowning

Via Ben Evans, "Drowning Doesn't Look like Drowning" is a classic viral explainer article by Mario Vittone.

Vittone tells you how to recognize the signals that someone is drowning, certainly. But the broader frame of the article is how dangerous it is that we assume, wrongly, that we know what drowning looks like--to the point that dozens of parents in the US every year watch as their child drowns to death, having no idea they're dying.

It reminds me of Erving Goffman’s classic 1952 essay “On Cooling the Mark Out", in that it's revelatory to think about this as a pattern and metaphor. What else doesn't look like what it is, but has massive consequences?

A few possibilities:

* wasting your money doesn't look like "poverty"
* oppression usually doesn't look overt, and can look quite genteel
* exposing children to lead-contaminated water doesn't look like "poisoning"
* crossing streets in a way that'll eventually get you hit looks perfectly safe
* speeding and driving while texting don't look like murder
* poor epistemology and unreflective patriotism don't come with warning label swastikas
* con artistry doesn't look distrustful. Quite the opposite! As David Mamet wrote in the excellent film House of Games, "Why is it called a confidence game? Because you, the mark, give me your confidence? No, because I give you mine."

Labels:

Wednesday, May 02, 2018

Software that does it all: a bug, not a feature

10 years ago, Kevin Kelly posted a request for software functionality much like the type I frequently post:
I know what I want. It’s pretty ambitious:

I want a database-centric personal publishing platform.

The front end is sort of like a word processor. But it is really a database.

...Then I can extract this data in many forms. My first extract would be to send it to a blog for posting. Perhaps I highlight the group of text and images i want to send. The blog has a style sheet which styles the output. I check the result and look and I get to manually override styles before I publish.

The same information is also sent out in a an email with a different set of styles and parameters. Or it may have some extra bits of info taken from the database.

I may then take a whole bunch of postings and want to reformat it for a more permanent web page, with a different look, structure, function.

And I will definitely want to extract the same data into a page-makeup layout program and make a long-form “book” out of the same stuff with different formatting, style, etc and make a PDF or print it.

Today, I wrote to him about why his call for software is misguided. It's advice I would do well to take myself!

Here's what I wrote:

Kevin, I think the trend has been away from all-encompassing software systems towards an ever-changing constellation of tools--for better or worse.

E.g., consider if someone had built a solution approximating your description at the time that you wrote this. Would the email component be as good as MailChimp? Would the blog component be as good as Medium?

And then there is version control. Think how often you might make a change in one context--say, more frequent paragraph breaks on your blog--that you don't want to percolate to the rest, say, the book version. Then, you rearrange the wording of a sentence that now starts a paragraph. Should that change percolate? Should it prompt you to percolate the paragraph break after all, in case they are contextually linked?

One problem with using our imaginations to fantasize about functionality is that we tend to see the intuitive ease-of-use and not the pain that would come from avoiding hard decisions in the design. When we look at the limitations around us, they seem hopelessly small minded and unimaginative; and often they are! But a lot of the time those limitations come in part because they allow us to avoid the misery of recursive unintended consequences of overly ambitious software.

In other words, the fact that you couldn't do all this stuff in 2008, and still can't today, may well be a feature of successful software, not a bug!

Labels: ,

Friday, April 27, 2018

The coming age of AI-assisted design

SketchCode is an experimental program that uses AI to turn hand-drawn sketches into simple HTML/CSS websites.

Even just interpreting a drawing and creating matching HTML is impressive, if indeed that works well. I'm skeptical, but even if SketchCode is more of a concept than an actual application, I think it points to a direction that will be hugely fruitful over the next decade.

Almost every domain where AI is making advances is one in which the space of possible interpretations is narrow enough, or basic enough, that it's only exceeding human performance by degree (in either speed or quality or both).

  1. Chess: the space of possible chess moves is generally learned by humans in a single sitting; AI is just good at considering a shitload of them
  2. High frequency trading: algorithms generally are deciding among a very small number of discreet choices of action that could be matched by a day trader with a ton of spreadsheets, if it weren't for the market's speed
  3. Self-driving cars have several dimensions of action happening at once, but that's still a narrow domain of outputs
  4. IBM's Jeopardy bot is able to avoid some complexity of its intermediary information structures thanks to the very simple form of the output, and the constraints humans are under in the game semicolon you or me with access to Google and more time and a fair shot at the buzzer beat Watson handily.
Meanwhile, many popular AI projects that have output in a much less constrained domain turn out to actually be massively human guided. For instance, the pop song writing machine learning programs. Since we don't have good generalizable tools for composing the thesis structures of pop songs, those still need to be provided by human composers.

Only at the bleeding edge of AI is it able to leapfrog what a smart adult with a little training can do slowly. Language translation qualifies, because it's gotten good enough that an adult trying to translate into a language they don't know well, even armed with several dictionaries and grammar books, gets stymied by idioms and phrases.

Coming back to HTML interface design, isn't it crazy that there aren't any super popular tools for fully designing arbitrary HTML interfaces, besides just text editors (with live preview)? Wix, Squarespace, Dreamweaver, FrontPage... They either massively limit your control over aspects of layout, or they make you quickly dive into the HTML code. I mean, when Wix added support for columns, they announced that as if it were a new feature!

In other words, HTML layout is fucking complicated and there are too many subtle aspects to be handled fully by WYSIWYG interfaces. XCode's interface builder is a complete nightmare. So if indeed this AI project is succeeding at interpreting aspects of that layout from a hand drawing, that is leapfrogging what an adult human with a little training can do without AI.

That's what's ambitious about this.

Labels:

Monday, April 23, 2018

There are dozens of reasons to fear AI, and only one mistaken reason not to

James Vincent just wrote a review in The Verge of Chris Paine's new documentary Do You Trust This Computer?, which features noted AI alarmist Elon Musk (and is being distributed for free online thanks to a donation by Musk).

The review leaves a lot to be desired. Yes algorithmic policing deserves criticism and attention... but that's a totally different film. This one seems focused on the existential risk. There's a very good chance animal life on earth will end in the next 100 years because of AI. Vincent has objections like (paraphrasing) "But a lot of people are publishing papers about possible ways to rein in AI!" I think that deeply misses the point.

There are thousands of different, independently developed AI programs running right now that play MMORPGs and are programmed to mine resources, forge weapons, seek the inhabitants of their world and kill them. Lots of these were made by teenagers. No one had to program them to be evil--they're not evil!

How far is Boston Dynamics from being able to perform all of those steps in real life? How far is the US military? There will probably be an arms race with killer robots, there will be tons of private sector research, there will be script kiddies, we already have homebrewed armed drones... and there's no real way to reliably keep software from taking actions in the real world that they would take in a virtual world.

The rhetorical task in front of the alarmist camp is merely to point out that there are many reasonable scenarios in which we inadvertently destroy the human race (and perhaps most animal life on earth) in the course of improving AI.

The rhetorical task in front of the optimists is to prove that all such scenarios that anyone could ever conceive are all unreasonable and all unlikely. I've read a lot of optimists, and I haven't encountered one that seems to be thinking about the problem thoroughly and rigorously.

They mistakenly operate from the presumption that only one body of researchers needs to come to agreement about restrictions on AI research. Or they mistakenly assume that evil is necessary for a machine to kill. Or the act is that we are still in the mainframe or personal computer era, and AI agents will be physically unpluggable. Or they misunderstand how easy it would be to kill hundreds of millions of people (just get a few dozen sheets of uranium together in various places and clap them together, or set up a network of autonomous factories that disproportionately use up the atmosphere's gases, or make tiny drones that seek and burrow into necks...) Or they are missing the aggressive direction of major power militaries towards creating software and hardware for self-perpetuating, autonomous killing machines. Or they misunderstand how opaque the inner workings of trained machine learning algorithms are.

When I hear someone say that they are certain there's no risk that this ends with the destruction of humanity (and probably most animal life on Earth), I think they are experiencing a profound failure of imagination.

Labels: , , ,

Thursday, April 19, 2018

Comey is dead wrong: mass incarceration is real

Reviewing James Comey's new memoir in The Intercept, Peter Maass focuses on an episode in which Comey complained to Barack Obama that the presidents use of the term mass incarceration was offensive:
Comey did not hide these views while at the FBI, and after making a speech in Chicago in 2015 that was not well received by the civil rights community, he was summoned to the Oval Office by then-President Barack Obama. Comey describes that session in his book, and he seemed to double down, telling the country’s first black president that the law enforcement community was upset at the way Obama had used the phrase “mass incarceration.” It was offensive, Comey told the president.

“I thought the term was both inaccurate and insulting to a lot of good people in law enforcement who cared deeply about helping people trapped in dangerous neighborhoods,” Comey writes. “It was inaccurate in the sense that there was nothing ‘mass’ about the incarceration: every defendant was charged individually, represented individually by counsel, convicted by a court individually, sentenced individually, reviewed on appeal individually, and incarcerated. That added up to a lot of people in jail, but there was nothing ‘mass’ about it.”

Maass makes a number of good points in his counterargument, including the point that more than 90% of cases on the state and federal level are settled with a plea bargain and not given their day in court.

A few more points I would add:

  1. There are hundreds of thousands of people sitting in jail, some for months or even years, who are there not because they were convicted but because they have a pending case and can't afford bail.
  2. Police departments routinely sweep up and jail groups of people with little knowledge of whether or not they have individually committed any crime, and release them without charge. (This is most visible at political protests when it happens to middle-class professionals, such as the dentist on his way to work who was arrested in New York City when police just grabbed everyone in the vicinity of a protest against the 2004 Republican convention. But it happens far more often than that.)
  3. Numerous police departments have faced revelations in recent years that their police and prosecutors routinely incarcerated people on charges they knew were false.
  4. Prison management and construction firms have lobbied, both legally and illegally, for increased imprisonment in multiple states. In Pennsylvania, a judge who secretly received a $2 million "finder's fee" from a private prison jailed kids for *years* for such offenses as having been given a bike by his parents that they bought on Craigslist that turned out to have been stolen.
  5. In numerous police departments, whistleblowers have revealed the persistence of quotas for arrests and citations.

In all of these aspects, people who did not commit crimes or who have not been convicted of crimes are incarcerated for large amounts of time due to political and economic dynamics far beyond any formal demonstration of their guilt.

Mass incarceration of the guilty is also an issue; but at the very least, any decent human being with a modicum of honesty must concede that the widespread, systematic incarceration of those whose guilt has not been determined and demonstrated by the system constitutes mass incarceration. In this country, some people get individual consideration and are presumed innocent; others are not, but rather treated a significant amount of the time as an undifferentiated mass of poor people and people of color.

Maass mentions a separate part of the book where Comey is surprisingly frank about this distinction. He quotes Comey as writing, of former general David Petraeus's slap on the wrist for lying to the FBI and mishandling classified information:

I believed, and still believe, that Petraeus was treated under a double standard based on class... A poor person, an unknown person — say a young black Baptist minister from Richmond — would be charged with a felony and sent to jail.

Comey is right about that, but of course there's even more than those two levels of standard. For many, arrested and jailed en masse, nothing close to the level of offense and amount of evidence in Petraeus's case is necessary to jail you for months or years.

Labels: , ,

Wednesday, April 11, 2018

The real problem with scientific papers

An article by Richard Goerg in the Atlantic arguing that the scientific paper is obsolete has been making the rounds.

There are too many lazy assumptions in this (commonly repeated) line of analysis to count. I do appreciate the skepticism of Wolfram that the author briefly discusses, but I wish he thought to apply it more broadly. Yes, it's definitely really cool that you can use Mathematica to do all kinds of math. But from my experience playing with Wolfram Alpha, the whole "analyze the rainfall during the battle of the Somme" stuff is way overblown. When you get away from cherry picked Wolfram-provided examples, the data sources are thin and spotty and most importantly, they presume a single universal source of truth as if we haven't learned from the past 25 years that any data or facts you assume to be true without looking at them carefully turn out to be at least somewhat false.

Bret Victor is a genius, but like Wolfram he suffers from a smugness so deep that it gets in the way of his brilliance. This research paper rewrite is a perfect example. It's great, of course--but it's great on its own and after a tremendous amount of work by an unusually brilliant programmer.

Compare the technologies the original authors used--a word processor plus some kind of spreadsheet with graphing capabilities--to the technologies Victor used--custom javascript, css, and html. How many researchers have access and capabilities to producing rapidly with the former versus the latter? How many people in the world could quickly and reliably produce the latter? How likely are the products of each to be obsolete in 5 years, 10 years, 20 years? (When was the last time you ran software from more than 10 years ago on anything but its original dedicated hardware?) How replicable/applicable is Victor's work here to other papers in a domain even a few degrees away from this particular type of graph analysis?

Yes, Mathematica points to a direction where non-programmers could produce complex interactive widgets and such, but given that centuries of person-hours by brilliant people has gone into it and it still isn't being used for that, how broadly achievable is that vision, really?

I think there's a much more fruitful direction for explanatory publishing: the clarity, and focus on what matters, that a good explainer brings when a companion asks them to explain something on a napkin at a bar. (I call this a "Gawker explanation", because Gawker was founded with the intention of telling you the New York media-related news that an informed insider would share with you off the record at a bar.) There are ideas too complex to explain this way, but most published papers aren't that complex. This can be done visually with text and images--think how easy would be to convert Victor's fun interactive sliders into static images and text that would be nearly as good (much like he already did in the bulk of that redesign)--and it can be done in the great explanatory medium of our era, video! That Victor, and this author, don't seem to seriously consider video as an explanatory medium tells you, I think, that they still have a lot of work to do in their work to improve the ways we inform others.

The replication failure crisis is a result of the medium of publishing, yes, but that medium is mostly social, economic, and institutional, not textual. These forces award surprising new findings and reinforce academic power structures, and discourage work to double check, declare null findings, and explore alternative hypotheses, and tell the truth about research labor and credit. Similarly, video explainers such as TED talks tend to be mediocre because their medium, viral social media, awards fame to speakers who can line up emotional zingers and simplistic narratives that reassure their audience by obscuring complexity and providing easy answers.

People don't tell the truth about what they do and don't understand. They create easy answers and positive narratives. They worship power and obey authority. They go along to get along. They respect confidence more than skepticism and see honest self-doubt as weak. They construct pyramid schemes of authority wherever they go. That's the problem with academic papers, not "print".

Labels: , , , , , , ,

Thursday, March 29, 2018

The Cosa Nostra of cryptocurrency?

This profile of the founder of the cryptocurrency exchange Binance is jaw-dropping. Binance is a totally new kind of startup: according to the article, it isn't officially based in any country, and it doesn't even have a bank account!

I'm skeptical. How do they pay for server time? Does this only mean that there's no publicly announced bank account?

At any rate, if this claim isn't entirely true yet, it will be possible soon. When you exist purely in the cryptocurrency realm, the barriers to entry are way lower. I think that might be one of the more profound long term effects of cryptocurrencies. It might even mean that governments lower regulatory barriers to forming businesses, to hold onto tax profits from complying businesses that might otherwise choose to form entirely out of encrypted bytes in the cloud.

Which raises the question: who will be the Stripe/Braintree of cryptocurrencies? It's theoretically feasible today to set up a company in a single day that offers some SaaS, charges using cryptocurrencies, and pays for its server time using traditional currency. But those last two are still somewhat headaches, and ground the company in some jurisdiction, given the need to convert.

And what does this mean for money-laundering? At some point only foolish criminals will demand their palms be greased with legal tender. The smartest criminals will do their racketeering in cryptocurrency land itself--removing any need to launder it.

Labels: , , ,

Tuesday, February 27, 2018

A progressive national agenda

An online discussion has emerged about what the Democrats agenda should be, for when they next enter power.

Here is my wishlist:

  1. Politics:
    1. Make Washington DC a state

    2. Aggressive federal involvement in any state that Gerrymanders to prevent statewide representation reflecting the population. Require maps to fall within formulas for compactness and representation matching.

    3. Abolish the electoral college.

    4. Replace the party balloting system nationally with one where all candidates with signatures from at least some percentage of voters (say, 1%) from some percentage of states (say, 25%) are listed in all states

    5. Replace the single choice presidential vote system with approval voting, allowing for each single voter to choose as many candidates as they approve of, so as not to act like a spoiler.

  2. Health:
    1. Single-payer healthcare system built on expansion of Medicare for all

    2. Still allow supplemental insurance, to cover doctors not part of Medicare

    3. Outlaw drug, gun, lawyer and military advertising except for 2-color, non-pictorial text ads

    4. Make drug approval process for drugs already tested extensively in other countries extremely expedited

    5. Expand FDA testing and regulation of health supplements, medicines. Create grade system reflecting company's history of their products proven to contain what they say they contain; new entrants can buy into this by putting up their own capital in a bet that their products will prove pure

    6. Federal rule to suspend drivers' licenses in all cases where injury occurs and the driver is partially to blame, on increasing scale

  3. Poverty:
    1. UBI, including accounts that are inheritable, earn interest

    2. Attack poverty with the Harlem Children's Zone model: increased federal funds in poor areas for schools, hospitals, health clinics, job training, home visits by nurses

  4. Social justice:
    1. Make May 31st (date of 1921 Greenwood Race riot) a national day of remembrance for the victims of slavery and Jim Crow, with ceremony in Washington reading the names and brief description of all known victims

    2. $1T of reparations for slavery and Jim Crow to all black descendants of US slaves, paid over 25 years into UBI accounts (about $1k/person/year).

  5. Violence:
    1. Federal arms restrictions based on testing for rate of lethality in different scenarios (like auto safety)

    2. Fund and conduct firearms and other public health research. May require permanently budgeted, independent public health research office

  6. Environment:
    1. Emissions and pollution cap & trade system

    2. Aggressively pursue international pollution/carbon pacts

    3. Federal gasoline tax of $1/gal to force Saudi Arabia to effectively subsidize our shift to renewable energy

    4. Tax environmental damage at the source. Whatever you produce or import, you need to pay or account for the cost to the environment. This should make single use plastic bags unaffordably expensive.

  7. Taxes:
    1. Wealth tax of 0.5%/year

    2. Increase the earned income tax credit, the standard deduction, and the per child tax credit by 50%

    3. Taxes can be paid by estimate by each taxpayer. If IRS challenges, can pay $500 to make taxpayer do taxes. Taxpayer owes 5x any discrepancy.

  8. Foreign policy:
    1. Open all foreign policy and domestic surveillance archives pre-1990, except for info that could be used to identify agents still in the field

    2. Recall most overseas troops

    3. Multiply foreign aid budget by 10

    4. Withhold military aid and trade to all countries that officially discriminate on the basis of sex, religion, ethnicity, or sexual orientation, or who use their military to colonize lands outside their country

    5. Demand that all countries that officially discriminate on the basis of sex, religion, ethnicity, or sexual orientation, or who use their military to colonize lands outside their country, and specifically call out nations that violate this

    6. US pays reparations to all countries with democratic gov'ts overthrown or civilians murdered through US funds

  9. Immigration:
    1. Abolish ICE's immigration lawbreaking division

    2. Greatly increase rate of admitting refugees

  10. Education:
    1. Federal rule, with funding, for minimum of 1 full time school psychologist per 200 kids, 1 full time teacher per 20 kids

  11. Economics:

      End the mortgage interest deduction, the health insurance and benefits deduction, and most other specific deductions

    1. End most farm subsidies

    2. National and 50-state quasi-governmental funding body to make 7-year, $20k/yr grants to journalists

    3. Deregulate predictive betting--let's let smart people make money by sharing what they're right about with the world, and punish consistently wrong pundits

    4. Rescind DMCA and other overreaching big gov't intrusions into the internet, and restore net neutrality

    5. Break up Google (YouTube, Waymo, Maps), Facebook (Insta, Messenger, WhatsApp); regulate Apple (allow competitors to App Store)

    6. Do not allow social networks to acquire each other

  12. Criminal justice:
    1. Outlaw for-profit jails

    2. Require prisons to allow unannounced spot inspections by federally approved NGOs

    3. Outlaw solitary confinement for more than a small percentage of prisoner-hours

    4. Direct federal prosecutors to focus on crimes that erode the public trust, such as fraud, embezzlement, false advertising, false representation, and violent crime

    5. Require all firearms to carry a serial number and to be registered in a federal database; failure to report a firearms transaction and removal of serial number are federal crimes

    6. All firearms transactions require a background check and a waiting period of at least 7 days

    7. Treat issuing guns like issuing drugs or poisons: partial criminal liability for all past gun owners including importers and manufacturers when guns are used in a crime, suicide or accidental death; liability is small per incident, but accumulates over time

    8. Make perversion of justice punishable by up to life imprisonment and full asset seizure; prosecutors, and judges and police given public trust who abuse it should go to jail

    9. Clarify/revise federal perjury rules: stating false information, even if it cannot be demonstrated that you knew it was false, is criminal.

    10. Prosecute all perjurers aggressively, even if they claim a good reason for perjury.

    11. Prosecute for perjury and contempt of Congress those who refused to answer questions, like Steve Bannon and Hope Hicks

    12. Prosecutors, police and judges who knowingly participated in prosecuting people they had reason to believe were innocent should be prosecuted for being accessories to crime after the fact, since they helped the truly guilty avoid justice.

    13. DOJ white collar crime commission to figure out why Manafort/Gates crimes were not investigated until Mueller; prosecute it with max sentences

    14. Make false advertising a federal crime punished by both fines and imprisonment

    15. Make false representation a federal crime punished by both fines and imprisonment; bring charges for false representation against banks and their employees such as Wells Fargo and Goldman Sachs

    16. Federal legislation to forbid judicial settlements for criminal activity that do not contain criminal punishments and admissions of guilt

    17. Criminally prosecute those who enforce discrimination by gender, ethnicity, or sexual orientation in public facilities such as buses in Williamsburg

    18. Impeach Mitch McConnell for refusing to fulfill Senate duty to advise President Obama's Supreme Court nomination

    19. Impeach and criminally prosecute Clarence Thomas for perjury, fraud and/or obstruction of justice

    20. Impeach and criminally prosecute Cyrus Vance for obstruction of justice in dropping solid cases against Harvey Weinstein and Trump children

    21. Prosecute many members of Bush administration for roles in torture, Iraq War

Labels: , , , , , , ,

Wednesday, February 14, 2018

Last night, yesterday morning: a Wheeler family story

My father's patrilineal family line, the Wheelers, have a particularly great family story that's been passed down from generation to generation.

The story takes place in the south, probably in Kentucky, sometime before the Civil War; my great-aunt, who knew her grandparents' generation well, placed it around 1950.

County fairs were the big entertainment in those days, huge week-long extravaganzas full of pumpkin growing contests, horse pulls (seeing whose horse can pull the greatest weight), bake-offs, and various carnival games. In the days before movies, let alone television, if you weren't in a city and near a vaudeville theatre, this was just the best it could get.

My great great grandfather, Charles James Merriweather ("C.M.") Wheeler, born around 1830, loved the carnival games as a boy, and as he got older he was especially fascinated by the unconvetional games--where instead of throwing hoops around a nail, you were trying to outwit a carnival veteran.

His favorite booth each year was the memory expert: a man who swore he could remember any story under a hundred words, which had to be grammatically correct of course, after only one hearing. To challenge him, you had to obtain or come up with the story, and read it aloud off a piece of paper; the crowd behind you would follow the words, so there would be a record to compare the carny's response to.

The fireworks came after you'd finish reading; there would be a pause, when it would seem to the crowd perfectly impossible to remember every word; and then the memory expert would rattle it off, the whole thing, in one or two breaths, slap his hand on the money you'd wagered, and add it to the bank, while the crowd gasped and clapped and shook their heads.

Oh, and that wager was a whole quarter, a good day's wage in that time. The Wheelers were schoolteachers who had recently immigrated from Scotland, and a quarter was no small amount to part with on a lark.

If by some miracle you won--something C. M. had never seen, in several seasons of hanging around the booth--you would win a "whole ham". In other words, the cured hindquarter of a hog, a massive two hundred pound piece of meat that would feed a family through the winter--an enormous payoff.

We all know this story in my family because C. M. saved up, wrote the most nonsensical story he could think of, and one summer day plunked down his quarter, waited for the crowd to stop shouting, pulled out his paper and read aloud the following:

Last night yesterday morning, about one o'clock in the afternoon, a hungry boy about 40 years old bought a custard for a levy and threw it through a brick wall nine feet thick, and jumping over it, broke his right ankle above his left knee and fell into a dry mill pond and was drowned. Ten years later on the same day, a cat hatched nine turkey gobblers and a high wind blew Yankee Doodle on a frying pan in Boston, where a deaf and dumb man was talking to his Aunt Peter.

The memory expert got half way through the mill pond, and no farther. C. M. won the ham, became a local hero, and (with plenty of help) brought home a huge surprise for his parents and brothers and sisters.

I asked my father to comment on this, and he said: "Of course he won the prize! This ditty has been handed down from father or mother to son and daughter across five enerations now, in the 150+ years since."

Labels: ,