In the concept video I recently did for laying out the interface paradigms for Firefox Mobile, I listed five guiding principals.

1. Touch it with your finger
2. Large targets are good
3. Visual Momentum and Physics are compelling
4. Typing is difficult
5. Content is king

It’s these principals that inform the design of new features long after the original design as been coded, released, and iterated on. In discussions with the perspicacious Mike Beltzner, another design principal emerged.

 6. Use modal overlays sparingly, if at all.

To be sure we are on the same page—I’m may be partaking in the dangerous hobby of coining new terminology—an overlay is simply a content area that sits in front of the content beneath it. The aspect that makes a modal overlay modal is that when it is up, the content “beneath” the overlay cannot be acted upon until the overlay is dismissed. Although a modal overlay may be visually transparent, it is never interaction transparent: you must always take action, like clicking “okay”, before continuing with your workflow. While I’m living dangerously, I’ll toss one more phrase into the mix: a state-forgetting modal overlay is an overlay whose state is reset every time it is summoned. That is, any work you do in the overlay is lost when you dismiss it.

Some examples of modal overlays are dialog/monologue boxes, ever-so-Web-2.0 Lightboxes, and the bookmarks interface for Mobile Safari. Some examples of overlays that aren’t modal are transparent messages and the OS X’s on-screen display for volume. The former have a number of interaction pitfalls that the latter do not share.

What’s wrong with modal overlays? In a word, they are modal: You are either interacting with the content or the overlay. Modal overlays don’t allow you to refer back and forth between two sources of information, or move fluidly between two actions. The second problem with modal overlays are that they are disconnected and disjoint from other overlays—knowing how to access one doesn’t yield a physical sense of how to access another one; they do not scale to give a unified, cohesive interface.

Let’s take a look at a plausible interface to illustrate the point; an implementation of search-and-replace using a Lightbox.

At first glance, this seems like a friendly, Web 2.0 way of doing things: it affords the opportunity for large typography and an uncluttered screen. While using a Lightbox is Web 2.0, it isn’t necessarily friendly. In fact, it has a fairly clunky workflow, for anything but the most basic case. Imagine you want to replace all instances of the text “insightfully thought” with “perspicaciously reckoned”, both of which exist in the text, although separated by a couple pages. You copy the first term, summon the Lightbox-based modal overlay, and paste it into the “replace” field. The next logical step is to scroll through the content to find “perspicaciously reckoned”, copy it, and put it into the “with” field. Unfortunately, because the search-and-replace form is in a modal overlay, you first have to dismiss the overlay before interacting with your content, then call it up again when you are done. It’s a slow, unwieldy feedback loop. On top of that, there isn’t a great way to indicate the changes in the text without somehow hiding the overlay.

Let’s take another example, this time from the real-world. At Humanized, we wrote a review of Mint.com’s thoughtful interface. One of the thing’s I liked about Mint’s interface—that I called “stunning”—was the unafraid use of text to create a huge, easily scanned list of possible categories that enabled filing a particular expense quickly. The problem, which I didn’t talk about then, was that they used a Lightbox. It feels heavy and slow at a visceral level. For example, when you are categorizing the expense type of a purchase, Lightbox’s modal nature keeps your from examining surrounding expenses that might help you to contextualize and categorize the purchase in question. Further, because it is a state-forgetting overlay, anything you enter into the create-your-own-category input is lost as soon as you consult another expense. Not good.

Lightboxes are tempting to use as a turn-key solution—we unfortunately use them in Songza—but there are better solutions, which I’ll come to in the next section.

Let’s go closer to home for the next example. Bookmarking. Firefox UX designer Madhava Enros did excellent work on many of the early UI prototypes for Firefox Mobile. In Proposal 8, which experimented with Songza-esque pie menus, you can see a lot of the thinking that directly informed the current design of the location bar. It also used a modal, state-forgetting overlay for bookmarking. This meant that you couldn’t both be bookmarking a page and interacting with the page at the same time—and worse, if you started to delve into your hierarchy of bookmark folders to file the page away, and wanted to refer back to the page before saving the bookmark, you’d have to start all over again.

Solutions

Now that we’ve explored the problems with modal overlays, it’s time to look at solutions. In particular, on the solution meant to replace modal overlays that might require complex interactions—which means that transparent messages don’t cut it. I’ll talk about two solutions in this post, but I am sure that there are other solutions out there. I’d love to hear ideas in the comments.

The Tray

The first solution is arrived at by simply not having an overlay be modal, with a bit of animation for polish. The overlay appears as a tray anchored to the edge of the content area. The tray must not force interaction, meaning that the tray can be ignored, and the content perused as if the tray didn’t exist. The only down side to not interacting with/dismissing a tray is that it eats up some screen real estate.

In the browsing world, Firefox uses the tray method to defeat the annoying-as-a-blackberry-pip-stuck-in-your-molars dialog box that asked if you wanted to save your password (and worse, it asked before you knew if you had typed it right!).

Because a tray overlay isn’t modal, you can answer the question in your own time. The tray is a great way to make user-dependent decision asynchronous. In fact, almost all user-dependent questions that are in situ with your browsing experience are done via the tray-style prompts. Take, for example, the prospective geolocation prompt. Coincidentally, the conviction of never using a modal overlay/dialog box in Firefox meant changing the W3C geolocation specification. Ironically, if you drag a tray is extended to take up the entire screen it becomes a modal overlay again. Further examples of trays in action can be found in Algorithm Ink’s browsing and editing functions.

As a final example, there is extensive use of tray-style interaction in Adobe Lightroom, generally docked on the left and right side of the screen. It’s a well-placed implementation, allowing quick access to a range of manipulations that don’t get in the way of moving around the image.

The Slide

The second solution isn’t an overlay at all, instead it uses scrolling or sliding. By placing the new area next to the original content it’s easy to understand how to move between the two areas. It’s also easy to extend the metaphor across numerous additions — new features get a new physical location. This is the technique demonstrated in the concept video, where the browser controls are located to the left of the web page content.

Using sliding as an overlay substitute mechanism (when we aren’t using slow-to-use scrollbars) is a fast way of moving between content in a non-modal way that also takes advantage of visual momentum and spatial memory.

Let’s take a look at how the slide mechanism can extend the browsing controls for Fennec. Here’s a schematic view of accessing the add-on manager and preferences:

Notice that once we’ve introduce the slide, we’ve opened up the possibility for a scalable way to expand ad infinitum. It’s a good way of coping with the limited screen size of mobile.

End Game

I’ve only started to think about solutions to the modal overlay problem. The tray and the slide are two passable solutions—I’m sure there are even better solutions waiting to be discovered. Quasimodal overlays, for instance. Let me know if you find/think of any.

RT @aza Mobile Firefox and Designing Without Modal Overlays | Follow @aza on Twitter | All blog posts