Ubiquity in Firefox: Round 1

Advertisement:

POWERED by FUSION

Mozilla Lab’s Ubiquity experiment continues to be a big success, with over two-hundred thousand active users, and millions of downloads. It’s time to start seriously thinking about how to bring Ubiquity-like features to all Firefox users, not just the early adopters.

Design Goals

Before we get into specific designs, let’s start with some overarching design goals to frame potential solutions.

(1) Don’t force new work flows. A large part of perceived software bloat comes from piling on new features without a unified vision — each disjoint, with their own way of doing things, and own way to invoke them. The more we can place time-saving designs in the path of current work flows, the smarter, quicker, and lighter the software feels. Done well, the feature doesn’t even really feel like a feature, just a lending hand: “of course it was supposed to work that way”, says the user.

A corollary, however, is that we can’t block the old work flow by trying to “help”. That would be rude.

(2) It must be localizable. Firefox currently ships in over sixty languages. Porting even the restricted natural-language parser in Ubiquity to all of those 60+ languages is nigh impossible (we’ll be getting some linguist help soon). We’ll need an interface informed by Ubiquity for Firefox, one that’s easier to bring to more languages.

(3) Should feel like Firefox. This is a bit more touchy-feely. The features uplifted from Ubiquity should feel at home in Firefox.

Designs

Design 1: Awesome Bar

Basic concepts:
* Ubiquity actions live in the Awesome Bar and are identified in the identity button
* Commands get a frecency score, just like web pages (already a feature in Ubiquity 0.1.5)
* Commands look like chrome, same color and texture
* Self describing text in the field persists when it has the focus, can be adapted to explain what input a command takes.
* After selecting a command and entering additional text, the autocomplete results area can be used for feedback (as opposed to the current two-pane Ubiquity interface)

In this mockup, the “identity” button, is co-opted to identify the selected action/verb. Visually, this is the clean but there is a problem: How do you get back to the normal URL-going state of the Awesome bar? Presumably you could hit “escape” or “backspace” to go back, or there can be the standard “x” in the URL bar — but is that discoverable enough?

Design 2: Awesome Bar

This is similar to the first design, but instead of the verbs moving to the identity button, they live in tokens which get added to the URL bar. This is a wire-frame/schematic illustration by mAlex Faaborg has a lot of details that need to be fill in, but does address the backspace problem. To me, it doesn’t feel quiet right, but I can’t put my finger on why.

An unanswered question in all both of these mockups is how actions/verbs like translate, that take multiple arguments, work.

Design 3: Awesome Bar

Alex Faaborg demonstrates how we can, in addition, include commonly used actions/verbs to the right of the Awesome Bar to help with recognition over recall.

Basic concept:
* Reminds you of what commands are available (”Oh yeah, I could use the map command more because I like to map stuff.”)
* Allows you to enter input and then select the command, like type “french revolution” and then click Wikipedia.
* Eventually we can phase out the search bar in favor of this type of interface, since selecting the search engine first with a drop down is clunky.

Design 4: Mouse

An illustration of how we can intercept the selection gesture, enabling a mouse-based Ubiquity.

Your Turn

This is just the first round of brainstorming. The sky is still the limit. We’re really interested in seeing more mockups, and having more people participate in the brainstorming process.

Challange: Create your own mockup of how Ubiquity could look in Firefox and post them here in the comments, on your blogs, or put them on Flickr tagged them with “mozconcept”.

About this entry

You’re currently reading “Ubiquity in Firefox: Round 1,” an entry on Aza’s Thoughts

Published:: 1.20.09 / 2pm

Category:: WEBLOG

tylerstyle 1.20.09 / 2pm

I can see the pain and limitations on all the designs.
But sometimes a risky and obvious path has to be taken, when facing a road blockage.
I think design 1 with an added really bright red x button would work more then well, as long as it doesn’t brake the other workings of the awesomebar.
That said I come to my point.
If technically possible, I think an official open release of Ubiquity to the general public, that integrates into the awesomebar. Not as functionality built into firefox, but as a recommended addon would be the next step to take in ubiquity’s life.

That way, you don’t generally force it to everyone and you would get feedback on it from people, that aren’t as technically inclined, as we early adopters are.

If that feature gets as much attention, as I think it will, then the time will come to think about integrating it.
Something in the field of Firefox 5 could be a goal.

Thoughts?

Zack 1.20.09 / 3pm

Any design that involves switching from keyboard to mouse is a nonstarter IMO…

Evan Sharp 1.20.09 / 3pm

Wouldn’t the Search Bar make more sense?

It already has much of Ubiquity’s working functionality; it’s a Verb Bar to Awesome Bar’s (Proper) Noun Bar. Maybe you could play with giving the Search Bar a little more width and seeing how it looks.

Also, thinking about it: isn’t Ubiquity essentially a new, unified interface for Firefox extensions? Why have developers rewrite code to duplicate functionality that the browser already can do? For example, there are already extensions to map what you’re doing in a page, or look for flights, or send highlighted text to Twitter, etc.

Maybe the goal of Ubiquity is to unify action-oriented Firefox extensions and give them a unified location and simplified interface. Just another way of looking at the concept.

Endolith 1.20.09 / 3pm

I have never understood why Firefox has a separate address bar and search bar in the first place.

Can’t you cleanly combine all three (#3 = Ubiquity) into a single universal input area?

Also, don’t be afraid to use keyboard shortcuts for speed in addition to typing out commands. I’ve always thought that using Ctrl+Enter for .com, Shift+Enter for .net, and Ctrl+Shift+Enter for .org was a waste of perfectly good keyboard shortcuts.

Donny 1.20.09 / 3pm

I think the concept of merging Ubiquity in with the Awesome Bar is a good idea. I find myself on occasion using CTL+L to jump to the Awesome Bar out of habit when I intend on bringing up Ubiquity so that would be a plus to just have everything in one spot.

Just a suggestion here. On bookmarks you can specify a keyword to execute that bookmark with a shortcut from the Awesome Bar. For example I would have g as a shortcut to do a Google search or a to perform an Amazon search and so on. Would it be possible to use this for Ubiquity. If I want to use Ubiquity in the Awesome Bar then just type u then the command and then the parameters like u map chattanooga, tn.

Then as mentioned in Design 1 to get back to the normal state of the URL bar all you would have to do is back out the text entered.

Zach Lym 1.20.09 / 3pm

I think the value Faaborgs design adds is that it tries to incorporate Ubiquity even after the user has typed in a space or the full URL.

This could be integrated into the first design by keeping suggestions present even after punching in the full URL.

As well, his visual reminder is a bit jarring. Perhaps include a second line of that present multiple alternate commands horizontally.

Chris Buchholz 1.20.09 / 4pm

Hi,

I just want to notify you guys, and especially you, Aza, about this thread I have create at the Mozilla Labs Forum about Ubiquity, and how I think it’s potential and greatness is wasted by being enclosed in Firefox.

https://labs.mozilla.com/forum/comments.php?DiscussionID=4989&page=1

Patrick Dubroy 1.20.09 / 5pm

I think #2 is the way to go, although I don’t personally like the visual treatment in #2 as much as #1.

But why does the command need to be a token? Couldn’t it act just like the default Ubiquity overlay already does, and just treat the command name as text?

Bryan 1.20.09 / 6pm

We have been looking at some very similar issues. Email is such a structured data set that we wanted to help people complete on objects/ideas in a similar manner to verbs and nouns.

Here are some examples:
http://clarkbw.net/designs/toolbar/thunderbar.png
http://clarkbw.net/designs/toolbar/toolbar-search-examples.png

I think we came to similar conclusions wrt to design attempts, however I’m interested in continuing with the current ubiquity format. Perhaps it’s just a matter of adding in a [ History > search element that is the default with awesomebar completions to the right. Design 3 seems almost full circle, if you only just declared that the awesome bar results are a history search.

Always looking forward to more

Gordon P. Hemsley 1.20.09 / 7pm

For the record, I’d like to say that I like Design 1 the best, out of the 4.

Miron 1.20.09 / 7pm

Would be cool if you took the first design, but instead make the favicon/chrome part clickable.

That way you could click on it and have a list of actions to choose from in the form of a drop down.

reds 1.20.09 / 8pm

yup, i’m going with design 1 too

Gray Norton 1.20.09 / 8pm

I’d like to do some deeper thinking about this, but one quick reaction:

The Awesome Bar (or more generally, the big field at the top of any browser window) is so strongly associated with the location of the current page that it seems odd to contemplate using the same field to invoke commands that perform non-navigational functions (e.g., commands that, instead of “going” anywhere, modify the content of the current page, perform an action without changing pages, or do all of their work in the preview pane).

Perhaps it won’t be an issue — people might acclimate easily to the change — but my instinct is that there will be some tricky details to work out in terms of setting expectations and providing feedback when the action to be taken is not fundamentally navigational.

JonStats 1.20.09 / 9pm

We have started a new project http://www.searchon.me to help understand trends in web searching. It is a project that needs a critical mass of users who basically install a simple firefox add-on that aggregates search data. We would welcome your help.

Ryan Scott Scheel 1.20.09 / 9pm

Designs 2&3 use both a keyboard and a mouse from what I can tell. This is very bad in terms of usability. I personally want to keep the locationbar a locationbar, but hey, taking the search bar and changing it into a “tools” bar would do better.

Gray 1.21.09 / 1am

i’m going right for #1 here.
but for arguments:
it should be that whatever we type in is merely a search string by default. then below it, the awesomebar shows the possible commands, just like #1/#2 but it only takes a command when we “shift + enter”
example:
w i k (ubiquity guesses wikipedia) t i o n a r y (ubiquity guesses wiktionary) f r e n c h (ubiquity guesses “french version of wiktionary”) [shift + enter]
then only will it take that as a command, and “bubble” it (but i prefer it to be an icon, like in #1)

Endolith 1.21.09 / 1am

Before you go live with this in Firefox, you definitely need to have a command-writing contest. I’d love to see the whiz-bang killer appz a little competition would produce.

Maybe during a release candidate, so it has more publicity, but the commands are finished in time to roll them out with the final release.

@Chris Buchholz:

You’re looking for Ubiquity’s predecessor: http://humanized.com/enso/

Jigar Shah 1.21.09 / 1am

Its really great concept and Ubiquity definately has ultimate destination on Awesomebar only. But i would say make it more extensible. Don’t include everything in Firefox. Keep hooks. So if some one wants to change, say, engine for selecting action, he can do that.

I am in favor of #1 as it less intrusive. Issue related to reverting back to regular addressbar, backspace or escape should work. We can have small close button in command itself to close it.

Regarding awsomebar itself, I think we should wait for enough usability tests of awsomebar and Ubiquity. (How many people use it and how many really like it.) Cos if i search ‘awesomebar’ i get lots of result which says ‘why they abandoned firefox because of awsomebar’. And regarding Ubiquity, Lets make it big public release addon. Let people use it. If it makes sense for them to use it like that. Not just geeks like me :)

Owen 1.21.09 / 5am

The first time I read this, I thought the awesome bar was a great place, but the more I think about this, the more I think this isn’t so good, I think instead of ubiquity coming to the awesome bar, the awesome bar should come to ubiquity. My point is, if you build an interface much more like the one you have right now, it would allow the browser to dedicate 100% screen real estate to the content. Also, by building it out in a more quicksilver style, this would allow for the eventual expansion out of the browser itself and onto the desktop, without having to totally transfer from the awesome bar back to a more similar state to how it is now.

Just something worth considering, also, on a slightly unrelated topic and as a slight gripe, why do we still have the search bar when the awesome bar has that function built in?

Alex Faaborg 1.21.09 / 5am

>I have never understood why Firefox has a separate address
>bar and search bar in the first place.
>Can’t you cleanly combine all three (#3 = Ubiquity) into a >single universal input area?

I think if we can solve some of the common problems that we’ve previously encountered when trying to do this, this would ultimately be the ideal solution. I’ve found that some people refuse to enter non-url data into the awesome bar, because habit appears to be more powerful than knowledge of a feature. Also, the forced choice between the two fields really is a significant enough cognitive task that it momentarily interrupt’s the user’s flow.

>his visual reminder is a bit jarring. Perhaps include a >second line of that present multiple alternate commands >horizontally.

Yeah, I think placing them at the bottom, and considerably smaller might work considerably better than that mockup.

>Don’t include everything in Firefox. Keep hooks. So if >some one wants to change, say, engine for selecting >action, he can do that.

We would probably only ship with a very small number of commands, for only core tasks that are universally useful. Like all of the current search engines and mapping, and that’s it. Then the set of commands would of course be open to much further additional and customization for user’s particular needs.

>if you build an interface much more like the one you have >right now, it would allow the browser to dedicate 100% >screen real estate to the content. Also, by building it out in >a more quicksilver style

Yeah, I totally agree. I’ve been thinking about a design kind of like this. Another great attribute of something that only appears when you start typing is that you can leverage the up and down arrows, so potentially do something kind of like this, but more ubiquity-ish: http://people.mozilla.com/~faaborg/files/20070705-kui/i1kuiWebSearch.png_large.png

Funtomas 1.21.09 / 5am

Exactly. Isn’t searching one of the Ubiquity applications? No need for dedicated UI real estate. Move it under Ubiquity. Same applies to address bar as typing in the URL can be addressed in Ubiquity too. Suddenly, we get significant space freed up.
Next step is incorporating mouse gestures to get rid of the key-hole buttons.
Go on Aza, you know the way now.

Alex Faaborg 1.21.09 / 6am

>Next step is incorporating mouse gestures to get rid of the >key-hole buttons.

Indeed

http://people.mozilla.com/~faaborg/files/labs/clover.png

get both of those and we are well on our way to a chromeless browser UI :)

Tristan 1.21.09 / 1pm

Hey Aza,

As always, I’m super excited by Ubiquity, and I’m relieved to see localization mentioned in your post!

This said, let me bitch about your blog template design:

It would be great to have the date of the post somewhere on the post page. Many people don’t think about this when they design templates, but it really makes sense to know when something was published. In this particular case, I could more or less guess the publishing date thanks to the comments, but it’s just guessing. Since I was sent the URL by a friend, I don’t see the date that is mentioned on the home page of the blog.

Other than that, keep up the good^W awesome work!

Daniel Stern 1.21.09 / 4pm

I definitely like the mouse based idea the most… it reminds me of hyperwords, another mozilla add on that is pretty amazing. With hyperwords, you simply highlight text, then right click it, and a menu of options appears, allowing the user to do a wikipedia search, email the blurb, etc… ubiquity would be a nice compliment to this especially if it includes a lot of mapping and social features…

The power of the right click has been completely ignored until now!

Varun Ravishankar 1.22.09 / 12am

I feel that Ubiquity belongs where the search bar belongs right now, replacing the Search Bar’s verb commands with something more powerful and extensible. The address bar has been marked by using noun searches and typing web addresses in; at this point, trying to add functionality such as Ubiquity to the Awesomebar would only confuse users and serve to hide features. I know that I only rarely type in search commands into the location bar. The idea of typing only web addresses into the location bar is so ingrained in my mind that adding Ubiquity’s search features would only serve to confuse or annoy me, and possibly others. I can certainly imagine mixing up web URLs and Ubiquity search results if Ubiquity was integrated into the Awesomebar; the results look far too similar.

Simply put, use those same ideas of integrating Ubiquity with the Search bar, not the Awesomebar. That reduces redundancy and improves workflow.

Chris Buchholz 1.22.09 / 4am

@Endolith
Not quite, I’m afraid … Ubiquity contains a lot of stuff that Enso doesn’t. Ubiquity’s more grown, if understand.

Keisuke Omi 1.22.09 / 5am

I think this feature needs its own workflow so that it can evolve without having to worry about conflicting with already complex features like the awesome bar.

Some notes and rough wireframes:
http://keisukeomi.com/ubiquity-in-firefox/round-1/

Amad 1.22.09 / 12pm

Doesn’t safari have a similar thing, a ’snap back’ button to go back? I wonder how discoverable it is to safari users? Hmm. Or just a simple red ‘x’ in place of favicon will do?

I’m for no.1 (combined with 3) then. The only problem is mixing up of commands and urls.

I don’t like the current mouse one, I found that the scrolling is rather annoying, a popup like the existing ui would be better. Keisuke Omi above seems to have some good ideas

On a side note, if we are going to replace the search bar, we’d have to better integrate mycroft plugins into ubiquity.
I think its also important to have a configuration to revert back to the old way for those who don’t want this, especially if we are going to put it in the awesome bar.

Jorge Martins Rosa 1.23.09 / 6am

Rather than a full integration into Firefox, and to keep it lightweight, there should be 2 «levels» of Ubiquity. Most common commands could be integrated (for «common users»), others kept in a «Ubiquity +» add-on (for «poer users»).

For common users:
Something between designs 1 and 3 (1 is more esthetically pleasing, 3 has the advantage of «recognition», but it’s more cumbersome). Perhaps a unified bar, a la Chrome — low-end people search instead of directly inserting URLs –, and some hotkey:
? — do a Google search (with Amazon, etc. as alternatives)
! — invoke most used commands in the Awesome Bar (map, translate, etc.). Example: !map streetname… ; !translate word from lang to lang
I confess I use (while I’m learning other commands) Ubiquity mostly for these tasks, so perhaps I’m just projecting my own experience. But nevertheless this could keep the browser lightweight.
No hotkey: Awesome Bar behavior remains the same as now.
I still don’t know about the Mouse option (4): common users just don’t use the keyboard for actions (not even in forms), but isn’t this defeating the original concept of Ubiquity?

Power users:
Ubiquity+ add-on
Either keep Ctrl-Space (or Cmd-Space) (thus keeping also the distinction between the browser and the add-on) or integrate commands in the former suggestion (!command)

As to localization, ALWAYS keep the original English version at hand. I’m Portuguese, but I think I’ll never get used to weird things like changing Ctrl-B in MSWord for Ctrl-N (just because bold is «Negrito» in Portuguese); same goes for commands (which, by the way, become weird and unnatural: «map something» is OK, «mapear something» isn’t).

Sebastian L Lewis 1.23.09 / 10am

Firefox already has support for keywords, I have a set of bookmarks for commonly used search engines where I replace the query with %s and assign a keyword like “gg” for Google, granted this sort of thing isn’t setup by default, the great thing about this is that I can replace the keyword with anything that I want later on if I decide to adopt a different keyword, and I can add support for any number of search engines provided I can figure out where the query is in the URL.

Assuming Ubiquity were to intrude on the URL bar, as I view it since this approach already works well for me, I’d want the ability to add any search engine in a similar way, as well as change the keyword if I find that “gg” has become too much of a habit and some better search engine comes out in the future.

Of course this keyword function could be extended to any Ubiquity command, for example in the second mock-up you have buttons appear in the address bar for the command, so if I were using a twitter command to post to twitter I could do something like “search” tab and my default search engine would appear there, or I could type “gg” tab and Google (my default search engine) would appear, but if I did “search” I could tab twice perhaps for a different search engine, perhaps my second favorite engine or a vertical of my favorite (Google Maps for example), or I could type “map” and cycle through first my default map provider and then a list of other map providers, so in this way I’d have two different keywords, I’d have a keyword for the category that I want, and I’d have a keyword for a specific search engine, so in this way I could have a number of maps providers or search engines available to me without having to assign a specific keyword to them, say maybe I don’t use them enough to bother remembering what that keyword would be, but I might occasionally want to make use of it for some reason or another, but the most important thing is that I would have complete control over my preferences of what those keywords are and how I wish to use them.

As far as a mouse-based UI goes, as long as there was a way to turn it off, it’s not a bad idea for those that prefer that.

Sebastian

For some reason I didn’t even think to mention this in the last post, but the ability to assign any ubiquity command in such a system to any other category would also be nice, I love it when software developers can predict what I wish to do and how I wish to do things, but when they’re wrong and there’s no user override it’s rather annoying.

Thanks,
Sebastian

hello 1.23.09 / 10pm

The mouse-over thingy should be in the right-click menu

egroeg 1.24.09 / 7am

i think you should pulse the address bar when text is selected and turn the address bar into a button you could click that would pull up a grey translucent like terminal window (tilda?) from the bottom that gives suggestions on what you could do w/ it.

also an icon by the back/forward buttons could work.

or web2.0y have a fold of the page peal back on a corner?

Sander D 1.26.09 / 12pm

Of these designs, I think #1 looks best as doesn’t add extra buttons and has a colour that looks ‘dynamic’.

But it doesn’t look like you can use natural language there: the Map command is heavily separated from the address argument. What was wrong with your original mockup?
http://www.azarask.in/gfx/future_ubiquity.png

Also, is it really neccessary to introduce a new mode to the AwesomeBar? As you noted, this design requires a way to get back to the URL-going state. Most screens are large enough to put the preview box inside the list with other (URL) suggestions.

Sean S 1.26.09 / 5pm

I like the design for # 1 best, but I think that the points made about combining the various bars are valid and need to be addressed.

Since the Awesome Bar + Ubiquity replicates the functionality of the search bar, I suggest that the search bar be replaces with a Ubiquity Commands dropdown.

I think it is important to be able to use the commands with only the keyboard, but that no user should be forced to give up the mouse to use these features. I like how # 1 places the command as another option in the Awesome Bar, and I think that backspace is appropriate for elilminating it, at least for our keyboard-only types.

For those that prefer the mouse, the search bar can be replaced with commands instead - perhaps featuring only the top 10 or 5, or open a pane like History for searching for some command you have that isn’t displayed. For users that have a lot of commands, having one place to find all of them would be useful. Selecting one would populate the Awesome Bar with that command and make it look like # 1. If the wrong one is selected, they can clear it from that dropdown or choose another.

I think the average user doesn’t want to lose their search bar, so having the same dropdown functionality as before would make it easy for the average user to mentally adjust from two bars to one. Naturally there would have to be some tutorial about “commands” being introduced, but the average user should be able to make the connection since the basics don’t change much.

ChrisJF 1.26.09 / 9pm

I’m actually a little surprised that this hasn’t been brought up already but I think #1 needs to be tweaked a bit.

Design #1 mixes metaphors too much with the identity button. The identity button is a place for *website* identity and *security* information. Ubiquity does not have anything to do with website identity and security. Obviously, #1 looks better than the rest because it is more polished. However, I think #2 (with more polish, and maybe #1’s drop-down menu) would be the best choice.

Also, just a thought, you might want to consider renaming the Location Bar to the Action Bar if the Location bar and Ubiquity are combined.

Keep up the good work, guys!

Ian 1.28.09 / 6am

Mouse and Keyboard interface should ideally be the same. In your words “Fewer choices means fewer worries”

Henrik 1.31.09 / 11am

Aza, as what you always say: Let’s reduce the UI. Personally I like design #1. No additional elements or pop-ups which will irritate users. Only the wanted output is visible. All together a good reuse of the awesome bar.

david 2.3.09 / 7am

Slightly off-topic I think if ubiquity moves to awesomebar/searchbar the adding of the selected text preview needs to come with.
So the new design would be the command with the selected text, followed by the command without the selected text.

I would also prefer to have a unified location bar instead of locationbar-searchbar duo.

Michael 3.30.09 / 3pm

Well, since it seems quite natural to use / to get to a seach maybe the text interface should start #!

Maxx69 10.22.09 / 3pm

Briefly, they are find, identify, select, and obtain. ,

Matěj Cepl 11.27.09 / 5pm

Search bar must die!!! I have already switched it of and use Search keywords only (search keywords are kind of forgotten and yet they are doing essentially the same what Ubiquity does now).

Ubiquity in Firefox: Round 1

Design Goals

Designs

Your Turn

About this entry

43 Comments

Have your say

Previous

About me...

Projects