Skip to main content

What "safety" means to a PL designer

I just finished reading Justus Robertson & R. Michael Young's INT 2013 paper about their work using the fact that players have incomplete knowledge of a simulation to accommodate incongruous player choices -- those that don't meet the story's goals -- by selectively rewriting the past. I made some sketchnotes summarizing the paper.

Of course, if all one wanted to do were to ensure an authorial goal (e.g. that Batman goes to Harvey's location), one could just simply make all seemingly-significant "choices" lead to the same scene. What I find interesting about this work is that it ensures that any railroading of this form is done in a perceptually consistent way, one that won't leave the player wondering how it was possible for the outcome to occur, because what they saw was consistent with their knowledge. There's a world model that led to Harvey being in one place and Rachel being in another, and that world model can change as long as it goes unobserved. The planner will only create changes that maintain this property.

This guarantee -- that the world model's rearrangements don't disturb perceptual consistency -- is, in a PL designer's terms, a form of safety. It's a property of a planning system that can be modeled in a logical formalism and checked with a computer.

The majority of the time, if you learn about type systems and type safety, you learn that it means preventing program crashes and mundane bugs (like applying a non-function). In sophisticated type systems, it might mean ensuring that you never dereference a null pointer or that your concurrent program is deadlock-free.

Because we talk about safety properties like these the vast majority of the time, I think people understandably get an impression of type- and logic-based programming language design as needlessly uptight for creative, freewheeling endeavors like game creation. Jonathan Blow certainly thinks so. It's as though we're suggesting wearing a lifejacket every time you go swimming, when all you wanna do is splash around in the shallow end.

I think the word "safety" is kind of a PR disaster for PL folks. Yes, sometimes we really do have safety-critical systems that we want to ensure have no bugs whatsoever, because actual human lives are at stake. Sometimes we care about the intersection with security & information flow, which affects people's personal information. And of course no one likes it when things crash or we get our code subtly wrong. But must we sell the Big PL Ideas as being fundamentally about never making mistakes, or preventing as many mistakes as possible, as opposed to actively enabling creative work, empowering people to express their design intent in a way that can, for instance, create more compelling game experiences?

Comments

  1. I agree. I routinely advise the writers of the kinds of application which concern me (grant applications) that "correctness by construction", as a slogan, doesn't sound all that relevant to the writers of the kinds of application that most people use on their computers. I prefer to send a message about types, used with as little theorem-proving as I can get away with, improving the basic hygiene of our work and enabling more of the executable code to be inferred from the stated design.

    The culture of "type inference" has done a lot of damage: the mistake is to imagine that what we write is what gets run, with types policing us on the basis of minimal annotation. I'd much prefer to write design statements and high-level programming strategies which don't get run at run-time, but allow the executable code to be generated from minimal source in a type-directed way.

    Suit-of-armour safety is the wrong message. Type synthesis is the wrong tool. It's time we left last century behind.

    ReplyDelete
  2. There seems to be one way how a suit-of-armor argument *can* sell types to the industry: test-driven-design (TDD) is really big now, that is, writing lots of unit tests. It's necessary too since otherwise big projects can't keep their quality as they scale up. Now the amount of necessary unit tests demonstrably sinks with increasing strength of the type system. So a strong type system increases productivity!

    Also, types allow for better tooling like improved code completion and semi-automatic refactoring tactics. Many programmers I know love that, because it boosts their productivity.So again, typing increases productivity.

    ReplyDelete
  3. I think part of the problem is that a lot of us (explicitly or not) believe that our concerns as PL researchers are universal. If a language purports to ensure correctness, it should really do that; if it purports to exclude certain kinds of bugs, it should really do that. And if it purports to (quoting the comment above) allow code to be generated from minimal source, the generation had better not introduce fresh bugs itself. So the kind of work we need to do as PL researchers tends to fall around the "no bugs whatsoever" end of the spectrum, which biases us into thinking this is typical of programming in general.

    ReplyDelete
  4. Hi Chris,

    I used to agree with your point very strongly -- I used to say that I wanted to make programming fun, not safe --- but over the last year or two I have become much more conventional in my views. Basically the recent NSA spying revelations have convinced me that the combination of spying, computer science and advertising have turned our society into a total-surveillance police state, and so safety (and I mean actual safety for actual humans) is a much more pressing problem than I thought it was.

    Obviously, formal verification has some chance to hinder the NSA's hack-all-the-things policy, but more surprisingly, it's Twitter that convinced me that CS techniques have some hope for improving the real safety of real people. Basically, techniques like differential privacy have the "burn the database" problem -- once you've performed a certain number of queries then you can't access the database without breaking security properties. However, Twitter is a *streaming* service, so stream queries can constantly renew themselves, without breaking either security or hitting a burn-the-database state.

    ReplyDelete
    Replies
    1. Ok, you have a point. I guess I should be careful not to suggest that the common meaning of safety *shouldn't* be at all related to programming language safety, as the former is definitely an important application of the latter! It still, however, seems important to *distinguish* the ideas, so that "safety" in an expressiveness sense isn't mistaken for the much more critical notion of "safety" in a harm-to-humans sense.

      Delete
  5. Yes! In fact, PL safety can actually be opposed to real safety: for example, if we made all tablet/phone OS's memory and capability safe, then this would be more PL-safe. But it would be less safe for the people using it, since there would now be no way to escape the jailed garden.

    ReplyDelete
  6. Did you know that you can generate money by locking premium pages of your blog or website?
    To begin just join AdscendMedia and run their content locking tool.

    ReplyDelete

Post a Comment

Popular posts from this blog

Using Twine for Games Research (Part II)

This preliminary discussion introduced my thoughts on using Twine as a tool for creating prototypes for games research. I'll start with documenting my first case study: a hack-and-slash RPG-like setting where the player character has a record of attributes ("stats") that evolve through actions that turn certain resources (money, health, items) into others. I've selected this hack-and-slash example because it falls outside the canonical "branching story" domain thought to be Twine's primary use case, but it is not too much trickier to implement. It relies crucially on the management of state in ways that simple branching stories would not, but it does so in a fairly straightforward way.

If all goes well, this post may also serve as a tutorial on the "basics" of Twine (links + variables + expressions). In particular, I'll be using Twine 2/Harlowe, and I haven't seen many tutorials for this new version published yet.

To me, the main "…

Why I don't like the term "AI"

Content note: I replicate some ableist language in this post for the sake of calling it out as ableist.

In games research, some people take pains to distinguish artificial intelligence from computational intelligence (Wikipedia summary), with the primary issue being that AI cares more about replicating human behavior, while CI is "human-behavior-inspired" approaches to solving concrete problems. I don't strongly identify with one of these sub-areas more than the other; the extent to which I hold an opinion is mainly that I find the distinction a bit silly, given that the practical effects seem mainly to be that there are two conferences (CIG and AIIDE) that attract the same people, and a journal (TCIAIG - Transactions on Computational Intelligence and Artificial Intelligence in Games) that seems to resolve the problem by replacing instances of "AI" with "CI/AI."

I have a vague, un-citeable memory of hearing another argument from people who dislike the…

Using Twine for Games Research (Part III)

Where we last left off, I described Twine's basic capabilities and illustrated how to use them in Twine 2 by way of a tiny hack-and-slash RPG mechanic. You can play the result, and you should also be able to download that HTML file and use Twine 2's "import file" mechanism to load the editable source code/passage layout.

Notice that, in terms of game design, it's not much more sophisticated than a slot machine: the only interesting decision we've incorporated is for the player to determine when to stop pushing her luck with repeated adventures and go home with the current spoils.

What makes this type of RPG strategy more interesting to me is the sorts of decisions that can have longer-term effects, the ones where you spend an accumulation of resources on one of several things that might have a substantial payoff down the road. In a more character-based setting, this could be something like increasing skill levels or adding personality traits.

Often, the game-…