Posts Tagged ‘ubiquity’

TaskFox Mouse: Iteration 0

Monday, April 13th, 2009

Two of TaskFox’s goals are to: Work with existing workflows, not against them, and to help people complete tasks rather than be forced to laboriously jump through hoops.

We’ve been looking at what TaskFox looks like in the URL bar. It’s an interface that excels when you want to start a new navigation task or do something somewhat unrelated to what you are currently looking at. If you see an street address on a page, an artist you are interested in, or text in a undecipherable language, you should be able to start using TaskFox right there.


Rapid Prototyping with Greasemonkey

Monday, March 9th, 2009

I recently was admiring the slick keyboard interface for Dropular, a sort of communal blog for sharing interesting finds on the net. It has that Swiss-minimalism-meets-interaction-design feel, which is enhanced by a modernist legend explaining how to operate the thing (although I wish the legend faded away after I got the hang of it). The interaction is simple: you use the arrow keys to automatically animate to the next/previous item.


A New Way of Writing Ubiquity Commands: Feedback Needed

Saturday, February 21st, 2009

In Mozilla Labs, one of our mantras when creating news was to write browser functionality is the “five minute rule”. If we can’t write a “hello world” example in five minutes — to get to that gratification of seeing the results of your work — then we’re probably on the wrong path.

Ubiquity passes with flying colors. Writing a hello world command can be done in under a minute. But only for a certain types of commands — as soon as they start requiring a lot of presentation, they become a bit unwieldy. In fact, we haven’t really updated the “map” command because its complexity sort of scares us.

We’ve been working on a new way of writing Ubiquity features which has three major advantages:

(1) It’s much more “webby”.
(2) It’s much easier to debug.
(3) It’s secure.

The goal of this blog post isn’t to announce what we’ve got, but rather to get feedback. We need your feedback before we make it a standard part of Ubiquity.

How it looks

By the end of this “tutorial” we’ll recreate most of the map command. The hard work for the map command is in creating the preview, so we’ll leave that for last.

Here’s the code the defines the command “new-map”. It looks remarkably like the old way of defining a command, but now it’s done inside a something that looks like a webpage. This lets you include complex HTML without the gross anti-pattern of building HTML with Javascript string concatenation.

<script src="ubiquity.js"></script>
  name: "new-map",
  takes: "address",
  previewUrl: "map-preview.html",
  execute: function(address) {"" + address.text);

There are two fancy things going on here.

Ubiquity In the Firefox: Round 2

Thursday, February 19th, 2009

We’ve been iterating hard on ideas to bring the power of Ubiquity to Firefox main. The two places it makes sense to surface Ubiquity-like power are (a) in situ with content when we are trying to manipulate, and (b) in the location bar, where we already type to perform navigation tasks. This post focuses on the second use case.

The three design goals, in shorten form, from round 1 were:

(1) Don’t force new work flows.
(2) It must be localizable.
(3) It should feel like Firefox.

We’ve added a new design goal, as a subset of not forcing new work flows: discoverability. The interfaces we design should be self-learnable. In this case, that doesn’t mean ever piece of functionality is immediately obvious, but that over time the system can teach you — step by step — how to use more and more of itself.

Note that all of these mockups are sketches. They don’t imply anything about the final visual style. From an interaction standpoint, they focus on tight feedback loops, as well as putting contextual autocomplete as close to the text being entered as possible.

Mockup 1

The Ubiquity-esque actions appear in the Awesome Bar results, and are subject to the same ranking algorithms as everything else.

The inset image on the right is an alternative way of accessing verbs: instead of having them appear in the awesome bar results, they appear as autocorrect-style text above what you’ve typed. The benefit is that you can always hit tab to quickly get to the action you want (as opposed to using the arrow keys for navigating the awesome bar results). It can also be unified with methods of structured modifiers (see later mockups). The detriment is that it is yet another mechanism and is visually noisy.

Other thoughts: The background of the url bar can change colors to add a visual key that an action is taking place. We can also unify the keyword mechanism, so that if you type “g ” it automatically gets expanded to a “Google” action.

Mockup 2


Scaling Ubiquity to 60+ Languages: We Need Your Help

Tuesday, February 17th, 2009

The year is 1953. Robert Floyd, the man most cited in Donald Knuth’s The Art of Computer Programing, takes the stage. His topic is on using English as a programming language:

“Each word must be implemented by a procedure which somehow contains its meaning and consistently interlocks with other procedures for other words. I can’t think of any task, intellectual or not, which has ever been carried out which approaches this magnitude.”

Over half a century later, those words ring prophetically true. Instructing a computer to do what you want with natural language is astronomically hard. That’s why Ubiquity cheats left and right to do it. It embraces ambiguity, uses tight feedback loops, and a restricted vocabulary plus grammar to give the appearance of something human. And it can do a lot better.

As we think about Ubiquity uplift into Firefox, we aren’t just bound to English. We need to have an interface that can potentially work in the 60+ languages that Firefox has been localized too. For that we need your help.

I’m pleased to announce that Mitcho Erlewine, a linguist-coder, will be leading the charge in helping us understand how to bring conversational computing to the Firefox scale. His first blog post is on How natural should Ubiquity really be? Although he speaks four languages fluently (English, Japanese, French, Chinese) and is a gifted linguist, he can’t do it alone. Especially if your native tongue is not English we need you to get involved in blogging, thinking, and mocking up ideas for how Ubiquity in Firefox’s Awesome Bar can work in your language. Put your mockups on Flickr and tag them with ubiquity and mozconcept.

Question: What are the greatest difficulties in bringing Ubiquity to your language?

Solving the “It” Problem

Tuesday, February 10th, 2009

One of the neater features of Ubiquity is it’s ability to understand pronouns in a natural and contextual way. For example you can write the command

email That's brilliant! to him

and if you’ve got the name “Jono” selected, Ubiquity understand that “him” means Jono, and thus to look up his email address. Thus we get “email That’s brilliant! to”. Although this makes for a great bit of magic, and is wonderfully useful, Ubiquity isn’t perfect in guessing what to interpolate. Say you’ve got “Lolz” selected and then use the command:

twitter I've seen links to this a lot

The question is do you mean to twitter “I’ve seen links to this a lot” or “I’ve seen links to Lolz a lot?”. Both make sense, and which one is better is unclear. The method that Jono DiCarlo pioneered was to embrace the ambiguity: we can make a smart guess about what the user means, but always give them the final choice in disambiguation.

The problem we’ve seen is that as we add more pronouns, the number of possibilities for interpolation goes up exponentially. For two pronouns, there are 4 disambiguation choices; for three pronouns, there are 8 disambiguation choices; etc. And for every extra command that matches, you double the number of disambiguation choices. It’s not a scalable solution.

Also bad is that often what Ubiquity thinks it should be interpolating is not your locus of attention, so as we use Ubiquity we often make mode errors. And we see this happening fairly often in the wild, where tweets have a random words from an errant selection where the word “it” should have been.

Bring the Fight To Them

There are a couple of potential solutions to the pronoun substation problem, but many of them don’t fly. For instance, we don’t want any special markup or computery quotation marks for indicating the to-be-interpolated pronouns. It should feel natural.

The solution I like best so far flattens the decision tree by having you choose as you type. It takes inspiration from cell-phone autocomplete/auto-correcter interfaces.

As you type a magic word, you can click or up-arrow to select the substitution. If you ignore it and keep typing, you get the plain text you typed. Think of it as on-the-fly text augmentation and structuring: we can greatly reduce the complexity of natural language processing by stealing a cycle from the user in a humane way. This scales nicely to other magic words. For instance, “url” can be replaced with the current pages url.

As focus in on Ubiquity uplift into Firefox, smart pronoun substitution may be largely important.

Question: What other solutions are there? We’re interested in radically different ideas to explore the solutions-space before we commit to one.

Weekly Ubiquity Focus: Email Command

Thursday, February 5th, 2009

It’s time to start improving the built-in Ubiquity commands! Since the launch of Ubiquity last August, huge improvements have been made. But most of these new changes haven’t improved the commands in any way. Aside from a few bug fixes, the “wikipedia” command, the “twitter” command and others are still the same as they were back in August.

We are starting a weekly focuses so that everyone can concentrate their efforts on a single piece of funcationality for a week or two to make it really sing. The first one is email.

For details of how to participate, head over to Abi’s blog, who is leading the charge.

Ubiquity Photo Editor: A Sketch

Tuesday, February 3rd, 2009

The open Web has no good way to edit images. There are tools like Picnic, Sumo, and Aviary, but they all revolve around proprietary tools and destination sites. Making graphical edits is a fundamental action that should be available anywhere you see an image on the web.

Using Ubiquity as a platform for innovation, here is what an browser-based image editor might look like:

How You Use It

(1) Place your mouse over an image, or select it.

(2) Bring up Ubiquity and use the “edit” command. Alternatively, right-click to get a context menu or use the to-be-implemented mouse-based version of Ubiquity. This brings up the full-screen editor.

(3) The full-screen editor has four major areas arranged along the top: reshape, develop, effects, and done. Reshape gives you the basics of resizing, cropping, and rotating the image.

Develop let’s you fiddle with the contrast, hue, saturation, color curves, etc. Effects are for turning a photo into a reprise of the end of 2001: A Space Odyssey. The sub-actions take place down the right-side of the screen in a non-modal way, and are dependent on which of the four top actions has been selected. The interface takes some inspiration from Adobe Lightroom.

(4) When you are done editing a photo the tricky part begins. Most pages aren’t read-write (although, in the original web browsers of yesteryear they were). To “save” we are going to have to do something clever.

We can put the image back into the page, by replacing the image with the new one as encoded by a data URL. From there, you can right-click to save it or, if it is in an email, just send it off. The browser can even annotate the page locally so that when you browse back to the same page, you see your modifications, just like the “edit-page” command does in Ubiquity now. Of course, those changes won’t be shown to the rest of the world.

The other options, besides simply downloading it to your computer, is to save it by sharing it via social networking sites like Facebook, FriendFeed, or Flickr. Flickr currently accepts POST requests for uploading modified images. Hopefully, as time goes on and image manipulations becomes a more integrated part of the web, more and more sites will think of their content as user-mutable.

Using Open Web Technology

As the prolific Jacob Seidelin has shown, canvas is able to support a full range of photo-editing capabilities. You can see some of those capabilities in action with his canvas-based library Pixastic.

Jacob will begin leading work on a Ubiquity-based editor in around a month. I’m super excited to see where it goes. We’ll keep you posted.

If anyone else wants to get involved, join us.

Open Design: Five Lessons

Tuesday, February 3rd, 2009

The virtual Ubiquity team recently did an open redesign of the Ubiquity logo. The penultimate result (one more round of polish needed) was spectacular, as was the journey. The process even got some press.

Open design is hard. There are many opportunities for arguments to become entrenched and for a stop-energy to become insurmountable. For the Ubiquity logo redesign, none of these things happened. That’s mostly due to the passion and creativity of everyone that participated.

Along the way, five observations/lessons emerged:

(1) The process takes time. The freedom to do a couple iterations is key — giving enough to to absorb feedback and channel it into a better result. If there was one thing we could change about our redesign process, it would have been to budget more time for another iteration or two.

(2) Open-ended questions are answered in open-ended ways. If you want concretes answers, give concrete questions. The further in the design process you get, the more focused your questions should get.

(3) When making decisions, it helps to create a highly visible framework/rubric by which those decisions are being made/entries judged. This shapes and focuses the discussion and staves-off bike shedding.

(4) Have a focused point-of-contact to take feedback, incorporate it, and iterate on it. Although the Ubiquity logo design process was open, we had a single designer incorporating the feedback of many. This yielded a funneling process with a consensus conclusion, rather than a diverging process without a conclusion (a major danger with open design). In our case Sebastiaan de With was our gifted designer and point-of-contact.

(5) Constant communication is key. Use Twitter, blogs, news groups, etc. to keep the dialogue going among everyone in the community. Open design isn’t a one-way street, but it feels that way if there’s a dearth of communication. And when that happens it becomes a monologue, not a dialouge. (Thanks to John Slater for this observation).

Did anyone else have observations or lessons they learned from the Ubiquity logo redesign? Does anyone have wisdom gained from other open design projects?

How to write a Ubiquity Command

Friday, January 30th, 2009

Want to know how to write a Ubiquity command, but don’t know how yet? Jono DiCarlo has a wonderful and in-depth tutorial on exactly how to do it. If you know just a little bit of Javascript, this will bring you from knowing nothing at all about Ubiquity development, to being able to create useful commands. The example you’ll learn here is a command that lets you bug your local congress member!

Ubiquity Command Development Tutorial