Monday, December 5, 2011

Climate Change Skepticism

Some climate change skeptics who are very dear to me keep expressing puzzlement over why I'm not outraged by the series of leaked emails that constitute "Climategate". Specifically why do I not conclude from them that the best information that we have at present should not make us confident that anthropogenic climate change is real.

I thought I'd explain and give climate change skeptics something easy to attack. A positive argument.

  1. A conspiracy is only damaging if it is effective. A bunch of ineffectual twits sitting around a table plotting is of no danger to anyone. Leaked emails tell us where to look for evidence of damage
    but are not, by themselves, evidence of damage.
  2. If there were an effective conspiracy to doctor data to create a false consensus then it could only have been successful in creating a false consensus if there are papers that depend on doctored data.
    1. If so, it should be easy to find examples, via the paper citation graph, and present them.
    2. According to the skeptics, 1000's of scientists are outraged (outraged I tell you) that "the whole thing is a fraud" which permeates the IPCC and the climatological profession. Being scientists, they should be able to follow citations to find those papers. That they haven't over a year after the first "climategate" is reason enough to conclude tentatively that
      they can't.
  3. If there were an effective conspiracy to claim a consensus where none exists, then there should be plenty of paleo-climatologists who assume the contrary position in the peer-reviewed literature. Oreskes et al. is a positive example of a survey that argues they assume climate change is real.
    1. If there were an effective conspiracy to suppress the vast majority of research papers then there would be a large number of quality research papers that have been rejected and a large number of disaffected climatologists who would be clamoring to create their own journals, conferences, and review boards. This has not happened in the decades during which climate change has been discussed. Why not?
    2. The unsanitized rejected papers would have data that is substantially different from the sanitized mainstream literature or draw starkly different conclusions from the same data. The BEST study from Berkeley is positive evidence against the first. The second requires a large effective conspiracy as in (4).
  4. If there is a large, effective conspiracy that includes the majority of climatologists, then again, their data would have to be faked or their conclusions would not follow from this data.
    1. If their data is cooked, it would show. Again, the BEST study from Berkeley is positive evidence against this claim.
    2. If they make up false conclusions, then the evidence is in the literature, and there would have to be some mechanism by which large numbers of scientists coordinate to avoid contradictions that are obvious to lay-people. What is this mechanism?
  5. Conclusion: If the data is not cooked (not 2), paleo-climatologists genuinely try to draw reliable conclusions from their data (not 4), and paleo-climatologists genuinely agree that anthropogenic climate-change is real (not 3) after decades of research, then the best available information says that anthropogenic climate change is real.

If you are a climate change skeptic who wants to convince me of your position, argue against these propositions or present positive evidence. I would love to see any of the following:

  1. Papers that have been infected with cooked data, ideally including a proportion of the citation graph that depends on these papers.
  2. A survey of the recent existing literature that shows that many paleo-climatologists make assumptions inconsistent with anthropogenic climate change. Ideally, this study would compare the proportion of historians of 20th century Europe who believe the holocaust did not happen with the proportion of paleo-climatologists who believe anthropogenic climate change did not happen.
  3. A comparison of the data in papers that were rejected due to the conspiracy and those that were rejected because they were genuinely crap.
  4. A coordinating organization that helps large numbers of conspiratorial scientists to come up with non-obviously-contradictory non-sequiturs based on real data.

I am not going to waste my time looking at:

  1. Damning emails. These are at best pointers where to look for evidence of damage caused by a conspiracy. I've worked too long with databases to be impressed by hysterical emails claiming "the data is all corrupt". When you dig into the problem, usually there isn't one, and most of the rest of the time some forensic work with logs and backups will let me recover the ground truth. Emails are the least reliable signal. Demonstrated damage in dependent applications is the best, and logs that allow forensic analysis are a distant second.
  2. Lists of dissenters. I've looked at a few of these lists and the ones I've checked are full of non-climatologists, economists, and weather forecasters. I'm sure they're lovely people and most of them are probably competent in their own domains, but they are not competent on the question of "is anthropogenic climate change real?" "A Scientific Dissent from Darwinism is another scientist list in a different domain that I find equally unconvincing. If you understand why I find Project Steve so funny, you'll understand why I find your lists unimpressive.

Thursday, July 21, 2011

Ramifications of my personal ineptitude 1: Absolute belief

I have been accused of wishy-washiness for not having conviction, for not having the strength of absolute belief in things like apple pie or America (Fuck, yeah!). This has always struck me as horribly unfair so allow me to set the record straight.

I believe absolutely and without reservation that (drumroll, please) I have at least one belief that is not correct. Obviously, the idea that I could be wrong about a belief I have because I have no beliefs about which I could be incorrect, or because I am incorrect about whether I hold this belief, beggars belief (pun intended). To not be correct about this particular belief because I am correct in all my beliefs is absurd.

Those who know me probably now think I am both wishy-washy and a pedant, but I think this belief is a better starting place for argument than Descarte's famous "Cogito ergo sum", "I think therefore I am." Descarte fails to rule out solipsism because there is no object to which he relates himself; no attribute that distinguishes Descarte from anything else in the universe. Presumably Descarte thought thoughts but his statement specifies no relationship between the thinker and the thought, so without saying something about what "I am," the statement "I exist" is meaningless

My absolute belief defines me as a believer in relationship to my beliefs any of which may not be correct according to a standard independent of me, and it further establishes that I and my beliefs are proper objects of belief. Not much, but it's a start.

¿Credo ergo erro?

Monday, April 18, 2011

Making JQuery templates safe against HTML injection

JQuery templates make it very easy for web app developers to compose HTML to keep their user interface up-to-date, but they make it very hard to do so in a way that is safe against XSS.

E.g. the JQuery template
<a href="{$url}">{$text}</a>
means create a link whose target is the result of url and whose body contains the result of text. JQuery does a lot better than most templating languages; message is assumed to be plain text and automatically encoded so that the page can't be XSSed that way. But it does nothing about url. If an attacker can make url be javascript:alert(1337) they can execute code with the full privileges of the containing page. This is a serious hole as URLs abound in web applications.

So right now, JQuery templates are easy to learn, but they have to be used cautiously. They would be even easier to use if template authors (and the poor maintainers) didn't have to worry about the provenance of data values, and could promiscuously inject into URLs, CSS, and inline JavaScript.

I've been cooking up a way to fix templating languages like JQuery. You can play around with Secure JQuery templates in the playground, or read up on the design which also includes an analysis of the performance of this system as applied to another templating language.

Sunday, January 9, 2011

Domestic bliss and the keys thereto

In the short time I've been married, I've come to realize that the key to domestic bliss is having a theoretical framework before engaging in delicate negotiations. (Before you snigger, no, "negotiation" isn't meant euphamistically like routes to other kinds of bliss: "commerce", "congress", "intercourse", "relations", "union"; though you might find the rest of this post much more entertaining if you add a dash of your own innuendo.)

The wife and I are thinking that we might not want to spend the rest of our lives together in a 1200 ft2 apartment. Before plonking down hard cash on something with a volatile price, we thought it might be good to figure out what we (collectively) want, so while the wife looked at house listings, I came up with a theoretical framework to save our marriage from the stress that hasty decisions would lead to decades from now.

Observation 1: When buying a house, you usually have to buy the whole house. You can't buy the kitchen and bedroom and not pay for the formal dining room. Which parts are valuable to me and which aren't?

Observation 2: When selling a house, people expect certain things. In addition to the obvious walls, they expect a foyer, and many houses devote more space to baths than to showers. Since Winston Churchhill, has there been a gainfully employed adult with time to take baths?

Observation 3: After living in small apartments in San Francisco for a decade and change, I struggle to imagine what we could do with 7000 ft2. What do people do with 13000 ft2 houses? Sharks with laser beams? I need a way to separate the definitely useless from the merely probably excessive.

So I want a way to think about the following questions:

  • How much is a given existing house worth to me?
  • Of two houses with rooms of the same dimensions, and the same fixtures, but different layouts, which should I prefer?
  • Given time and a plot of land, what should I build on it, and what should I build now?

Being a reductionist, I will try to answer these questions by dividing each concrete house into smaller abstract houses.

Each quality of the real house (the existence of a room, a fixture, a nice material or finishing, and qualities like the view, location, neighbors, and availability of jobs) goes into at most one of these houses and I should be able to easily classify things as I walk around a house or look at a plan.

A practical house
The portion of the house that its inhabitants use on a regular basis. At a minimum, this includes the master-bedroom, a front-door, and a bathroom. Unless the nature of your work makes it impossible to bring it home, it will almost certainly include some kind of quiet study space.
Note: Bachelors, this may not include the kitchen.
An entertaining house
The portion of the house (not in the practical house above) that encourages wanted guests to visit. This probably includes guest rooms, extra pantry space, wet bar, extra parking, and any dedicated dining room.
Note: See "safe house" re unwanted guests.
A safe house / a contingent house
That portion of the house that you hope you will never use, but might. This includes your alarms, guns, flashlight batteries, canned food, fire-extinguisher, and other things that you want to be able to lay your hands on quickly; but that you'd lose if mixed in with your everyday things. Unless you're older or disabled, this probably includes any elevators.
Note: Survivalists, this includes your moat, bomb shelter, life boat, escape pods, and extra ammo.
Note: Yuppies, this includes your portable wealth, "the hounds" and their release switch, and that earthquake preparedness kit you've never opened.
Note: Hipsters, this includes your zombie preparedness kit, and the sealed cache of LPs that you're preserving for future generations.
Note: Zombies, no, brains go under (1).
An ornamental house
That portion of the house that exists solely to impress non-inhabitants to allow resale or to meet legal requirements. This probably includes any formal dining room, and most if not all baths.
Note: New-Agers, this does not include your chakra attuned crystal chandelier.

Hopefully as the examples make clear, this is a subjective assessment. A particular feature, like a pool, might be practical to some people who swim often, entertaining to people who want to entice their grandkids to visit, may be an essential safety feature to someone who is afraid of catching on fire, or might be an ornamental feature when every other house on the block has a pool. Finally it might not be in any house, an expensive hazard to busy parents with a toddler.

Is there anything I'm likely to want in a house that doesn't fit in these four houses?

If the residents don't want to use it (1), guests don't want to use it (2), it's not useful just in case (3), and it's not required for resale or for appearances, then why should it be part of the house?

This will not satisfy a pack-rat, to which I respond, "Don't be a pack-rat!" I have no better argument for its completeness than that I haven't seen a counter-example ; I've tried applying this taxonomy as I've looked at houses and have yet to find something that I could not easily classify based on my likes/dislikes and expected usage patterns.

How does this answer "How much is a given existing house worth to me?"

Once the inhabitants agree what is in which house, then the problem reduces to a simpler set of questions:

  • What kind of creature comforts are important to us? Value of (1) to me
  • What would our guests enjoy? Value of (2) to me
  • How likely are various disasters? Value of (3) to me
  • How soon will I resell the house, and how much will an impressive house benefit my career or community life? Value of (4) to me
  • How many fewer creature comforts are we willing to put up with to get our friends to visit? (1) vs (2)
  • How many comforts are we willing to forego for emergencies? (1) and (2) vs (3)
  • How many comforts are we willing to forego for increased status? (1) and (2) vs (4)
  • etc.

How does this answer "Of two houses with rooms of the same dimensions, and the same fixtures, but different layouts, which should I prefer?"

If we had the luxury of embedding a house with N rooms in a space with N+2 spacelike dimension, then we could easily have every room adjacent to every other and still be 3 dimensional. Alternatively, we could wedge shaped rooms radiating out from one central corridor that is an N-gon.

But such houses are either physically impossible or hard to resell, so we need to know what makes a conventional 2½ dimensional house livable.

If the practical house is dispersed, then we're going to spend an inordinate amount of time walking through rooms that we don't use. So we should prefer that the practical house be concentrated and that it be easy to get from one part to another.

11         11

When entertaining, we're almost certainly going to be using some parts of (1) like bathrooms and kitchen space, but we may want to keep guests away from other parts of (1). So a good arrangement has (1) adjacent to the entertaining house with the common parts at the joining edge.

11         22

In an emergency, we want to be able to get to the safe house from (1) and less often from (2), but we don't want to lose our emergency stuff by mixing it with a bunch of everyday stuff, so (3) is ideally located at the periphery of (1) and (2) ideally adjacent to both.

2211?????????33   ZOMB
2211         33   IES!

Finally, (4) should almost be entirely at the periphery where possible. Why would we want to waste time walking through parts that nobody uses?

114444222444433   ZOMB
11    222    33   IES!

So if you consider an adjacency graph of rooms, then it seems that the best layout has the practical house (1) as a densely connected subgraph (and probably adjacent to the outside), with (2) next to it with special attention paid to common areas, and (3) distinct but reachable from (1) and with (4) out of the way.


These diagrams should be taken with a grain of salt. Each real room doesn't necessarily fall within one abstract house. E.g. most of a stove might be practical, used daily, but maybe you only bake when entertaining, so a nice convection oven would be in the entertaining house.

How does this answer "Given time and a plot of land, what should I build on it, and what should I build now?"

The second part of this question is easily answered. You need most of the practical house early on. You can delay expensive parts of the other houses, and there's no real need to build the parts of the ornamental house that are for resale until you plan to resell. Building (1) then (2) then (3) then (4) may be ideal if you arrange it so the adjacency graph is in shells to get the layout described above.

The first part of the question can be answered by coming up with a wish list for each house and budgeting for it. The relative size of the practical and entertaining houses depends in part on how often you think you'll be able to entertain. The size of the ornamental house depends on how well funded your neighbors are, and the size of the safe house depends on how risk-averse you are and how well funded your enemies are.

Then you need to pare things back. Are all the things in the practical house things you will actually use on a regular basis? If not, maybe kick it into the entertaining house. Are all the things in the entertaining house things you or your guests will use? If not, get rid of it or move it into the ornamental house. The value of something in the safe house is the portion of the cost of the disaster mitigated by the probability of the disaster minus the cost. Consider insurance instead. For the ornamental house, ask whether it can be delayed or faked.

If you've pared it back to the bachelor minimum (bedroom, bathroom, front door), congratulations, your wife is going to leave you so you're done.

Does this framework make delicate negotiations easier, more tractable, more likely to come to correct conclusions?

I don't know. The post-reduction questions above are by no means easy, but for a relationship to survive delicate negotiations, each needs to learn what the other values. They need to get things out in the open so that decisions are made with eyes open to minimize the chances of hidden resentment down the road, and maximize the chances of both enjoying their decisions. I hope these questions get to the heart of that.

† : I use the term "use" broadly; one who appreciates art uses it, and that a house is in a good school district is a practical concern if a child uses the school afforded by the house's location on a regular basis.