Can Ubiquity be Used Only with the Mouse?

1. Dan 12.23.08 / 10pm

   For your open questions I have a couple ideas. :)

   Invoking the pane by itself would probably be easiest when done with a simple toolbar or statusbar icon, as many current extensions do (you could say… it’s ubiquitous!). If users don’t have need for that functionality they can remove the icon from the toolbar.

   You could also show a slim black vertical bar on the right side of the browser window, past the scrollbar… however that might not be the best solution since users trying to resize the window or scroll the window are bound to hit it by accident (or hit one of the others while trying to open ubiquity.

   And of course a keyboard shortcut would be a fun way to do it too, although not a mousey way. :)

   I think I would like mouse Ubiquity docked to the top or bottom of the window a la Firebug. This would give it more of a similar layout to the current Ubiquity.

   I’m not sure what happens to the content being scrolled off the window when you activate mouse ubiquity in the video (is there a horizontal scrollbar + a fixed width website or is it unreachable?), but having iubiquity’s pane docked to the top or bottom would allow a wider area for content in the main window and allow all content to be scrolled and reached while the pane is open.

   Of course you would have to have previews that are formatted for horizontal viewing… which may require ubiquity command script changes, unless you can do something fancy involving -moz-column-width or something.

   Another thing I noticed is this is a bit like the existing sidebars we see in Firefox. You could even just make this another sidebar which could help users who have used sidebars use it… although I don’t know how many users that is. :) As well it would greatly lessen the coolness factor with no sliding and the chunky gray border.

   As for the actual interface… I rather liked some of the aspects of the failed mouse interface you blogged about earlier, although I am thinking now a docked pane would allow for you to show more data without having cascading menus covering the browser window.

   I’m thinking (in a pane docked on the top/bottom) the bottom half has previews running horizontally across, while the top half has input forms on the left and command selection on the right.

   Looking forward to seeing how you choose to develop this. :)
2. Yatrik 12.23.08 / 10pm

   Hey Aza,

   Really nice idea! The scroll is great, maybe implement that into the actual Ubiquity interface load, in that it doesn’t cover the page up (albeit the scroll does hide a part).

   Where are the command CSS files stored? I’d like to play with the pane styling some. I have some ideas.

   Really nice again!
   Yatrik.
3. Yatrik 12.23.08 / 10pm

   @Dan: Some interesting ideas.
   The sidebar kind of changes my perception of Ubiquity, which is a little more “command-interface” like, to put it in the simplest words (can’t think of better ones).
   The top/bottom idea did occur to me once, the only problems I had is that it can be somewhat strange with selecting the actual command.

   I’ll think about that one again.
4. Stephen 12.23.08 / 10pm

   I am a part time user of ubiquity, I have not fully integrated into my everyday, but I do like this mouse-based option. I have seen the Fennec and thought the sliding window is a great idea, so naturally when you all did this I thougtht, sweet! Now this may only be me, but after I installed it I do not see the little icon, instead my cursor change to the pointy-finger, when I click it the screen slides, but it does not show anything, just blank grey, so I am not sure what’s up with that. As for the side to slide from, if that could be a user defined thing it’d be great, but right side works for me.
5. P. Dant 12.23.08 / 10pm

   Aza –

   I’d like this opportunity to discuss the all too common user interface of English mistake of misusing the word, “only.”

   You said, “Can Ubiquity be Used Only with the Mouse?” This is suggesting that Ubiquity’s interface is so mouse-centric that it raises the question of whether the keyboard could be used as well.

   What you meant was, “Can Ubiquity be Used with Only the Mouse?” (or “… Mouse Only?”) This suggests that one normally has to use mouse and keyboard, and raises the question of whether it is possible to use Ubiquity with the mouse only.

   Here’s to hoping you receive many boxes this holiday season, giving you greater opportunity to think outside them.
6. Jimmy 12.23.08 / 11pm

   I have the same problem as Stephen. I’m on a an intel imac. It occurs both on firefox 3.0.5 and 3.1b2. Hope that helps.
7. Yatrik 12.23.08 / 11pm

   I confirm the same bug as Jimmy and Stephen on Firefox 3.0.5 Windows XP Pro.
8. Amad 12.23.08 / 11pm

   Mouse gestures could be another way to invoke ubiquity..
9. JR 12.24.08 / 12am

   Hi Aza,

   The mouse interface is looking much better than last time! I really like the slide in from the right. As far as menus go, there are definitely a lot of options, but I’d like to vote against menus like the fisheye, carousel, and accordion - I feel that, like you mentioned with the fading in and out of the “ubiquity button” during testing, the extra animation just attracts too much attention and gets in the way of what the user’s trying to do. It could just be me, of course, but I’ve always found fisheye menus unsettling. The tabbed menu idea shows promise, though, especially if it were combined with the tagging idea you mentioned in the video - if (for example) you gave the slide-bar a Search tab, a Language tab, a Math tab, and an algorithm smart enough to auto-select the most appropriate one, you’d be well on your way to perfection.
10. JR 12.24.08 / 12am

    @Amad - mouse gestures are certainly powerful, and could definitely be considered as an additional option, but I think the power of the transparent ubiquity-button shown in the video is that anyone could use it. My mother could use it, and that’s saying something. Select text, click button, click “wikipedia” - that’s real power right there. Speaking as a web developer: it’s hard enough to get the general public to use Firefox in the first place, much less teach them what mouse gestures are and get them to remember the specific one necessary to invoke Ubiquity.
11. Nathan 12.24.08 / 12am

    Nice work! Regarding the slide/scrolling, I think it’s important (at least in some cases) to be able to see the selection in context when the Ubiquity area appears. Perhaps the Ubiquity area should just appear as an overlay (without the slide) if the selection is near the left edge of the browser?
12. Edwin Khodabakchian 12.24.08 / 1am

    Very interesting. There are similarities between what you are doing here and what feedly has been doing with our sidebar concept. If you have the time, I would love to connect and exchange notes and see if there are ways feedly could better integrate with ubiquity.
13. Mark 12.24.08 / 3am

    I vote to try out the Fisheye and the Hierarchical approach. Because the list of services may expand a great deal, perhaps try combining the two. Sorting through long lists of services can be a problem, fisheye may help solve that. The hierarchical approach reduces data clutter which, again, is very necessary so as not to overwhelm the user.
    Perhaps try a fisheyed hierarchical with a warped grid behind it. We’d have to try it out, but there would be a need to make the look and animation feel cool.
14. Morton 12.24.08 / 4am

    May you can avoid selection by letting Ubiquity guest what to do. For example, translation could be guess by comparing the browser’s language to site-language (extracted out of the page’s code or by index of coincidence).
    I also think the position of the pane is a question of task. A translation could rests on one or two lines on top or bottom (on top: How to interact with the yellow notifyarea?) but a search may be better on the right (or left). Short: I think on times of widescreens left/right should hold the long passages and top/bottom short information. Right and top seems more natural to me than left and bottom but may it is a question of personal view.
15. Phil 12.24.08 / 10am

    I have the same bug as Stephen and all the other, and I found another one. If you search something in the box, than you click on the page and you select some text, the badge doesn’t “appear” (the cursor doesn’t turn into the hand).
16. david 12.24.08 / 12pm

    Like Phil the badge doesn’t appear but i wonder how you could add a file on my computer by subscribing to a feed?

    If it’s possible wouldn’t that be dangerous for the feed subscribers?

    I did see my cursor change in a hand, after i executed the command setup, but it is based on guessing where the badge could be. Sometimes the selected text retreats to the first full word.
17. Amad 12.25.08 / 4am

    The reason everyone is not seeing the badge is because there is none..
    @JR: I was referring to the case where you don’t select anything. Mouse gestures could be used, for a single command, or you can just right click and select ‘ubiquity’.. ;)
    Opera has a pretty good implentation of this sort of thing.
    @david: embedding the image in the feed s one way, and no, there is no file as far as can tell
18. david 12.25.08 / 9am

    @Amad: if you check the code you see
    url(resource://ubiquity-scripts/selection_mark.png)
    That is the badge image i assume?

    Lets say you embed the image in the feed how does it know where it should be stored on the users computer?
19. Jimmy 12.25.08 / 12pm

    This will really make me use ubiquity so much more. I have to say it is a great idea. So sorry we all are reporting bugs, but another one is when you click on a picture from flickr you can’t click on any other commands after that.

    Like I said love this. And i confirmed the only problem is the missing picture. Made a crappy one myself and it put it in that location. It works.
20. Aza Raskin 12.25.08 / 4pm

    Hey guys, sorry about failing to put up graphic. This is what happens when I release and then go to the holidays :)

    It should be fixed now.
21. Pascal 12.28.08 / 12pm

    Could it be possible to add some voice recognition ? This should be great for ubiquity command input; I mean as a complementary input canal to mouse and keyboard. It could be very efficient. Don’t you think so ?
22. Aza Raskin 12.28.08 / 1pm

    @Pascal: Absolutely. It’s something we hope to experiment with soon.
23. zmarties 12.29.08 / 5am

    No one seems to be mentioning the big problem with the sidebar based interface: there is no locality of interaction - you select some text in the original content, click on a nice local action button, and then suddenly have to move the mouse and your visual focus to some totally unrelated part of the screen where the sidebar has appeared. One you get there, you have to start your visual scan again, trying to find the correct command to click upon to invoke the action you really want.

    Since the common screen format is now a widescreen format, the distances involved between the original content and any sidebar are large, and dont lead to a fast workflow.

    I know that you had problems scaling the context menu approach, but that is generally a much better workflow - select, choose from actions directly, get result - all achieved with only relatively small mouse movements, and no shifting of visual focus required.

    One way to help with the scaling of the context menu approach may be to use the mouse scroll wheel to scroll through menu views, which is generally much better than needing additional clicks or hovers to bring up additional sub menus. (I realize Mac users who dont use a PC wheel mouse may have a harder job using such an interface).
24. Tarwin Stroh-Spijer 1.1.09 / 2am

    Interesting ideas.

    The position of the panel should, by default, depend on the language setting I think. On the right for left to right read languages and on the left for right to left read languages. This means you continue in the direction you are moving your mouse after you have selected text.

    The layout of the panel seems a little strange at the moment. I think it should be more towards:

    —————————————
    [ search box etc ] [ action button ]
    [ favourite actions ]
    [ context sensitive actions ]
    [
    Results
    Results
    Results
    ]
    [ other actions and menus ]
    —————————————

    Having too many options to choose from will both scare people away and make too much clutter. Giving only a top four or eight actions for an item, that a user can change if they want, means they know what they can use this service for ‘usually’. If they really want to delve into what it can do more powerfully they can go down the bottom and play with the menus there.

    Having a place for context sensitive (ie translate, add to address book, send email etc) actions would be good to keep them separate from default commands.

    In response to ‘zmarties’: I think more and more people with large wide screens don’t set their browsers to the full window anymore (on Windows) but use their browser more like a Mac user anyway, floating around being ‘as big as it needs to be’. This said I do think it would be good to have the option of a context menu, maybe if you hold down on the badge, of your top commands. This could be a simple list or box type pie menu (circular menus are just annoying for this kind of thing, think more what programs like Maya do!)

    Hope I made some sense there - my post NYs head ain’t so bright today!
25. Tarwin Stroh-Spijer 1.1.09 / 2am

    A quick note on what I’d said earlier. Having a small set number of default options I’ve found to be a big plus when using a utility program. The best example I can think of right now is a program called “Pennies” for the iPhone, which lets you track expenses. It doesn’t have a lot of functionality and you can only put expenses in a preset of 8 different categories. To start with this is annoying because you’re always “why isn’t there a list for me to put all the money I spend on beer” but it’s this simplicity which makes you use it all the time: it’s never more than 3 clicks and you’ve added something. No scrolling around thinking where it should go.

    I know this isn’t exactly the same here, but I think it would make a big difference having your “most used” commands available.

    How about the idea of adding icons to commands? Like favicons?

    Some comments on my use so far.

    * Great so far. I never used Ubiquity because I really don’t have the time to “learn” to use something like this. The fact that it’s just there is very useful.

    * When you select text on the left of a page, the panel then comes in and pushes that off the screen and I can’t see what my selection was any more.

    * If you double click to select a word (probably bad practice anyway) when the panel is open, the panel slides away, the badge comes up at the end of the selection, but the selection is not shown.
26. Stephen 1.4.09 / 10am

    I am still having the problem of not seeing anything in the slide screen? Is it possible that Adblocker plus blocks whatever is loaded in the slide screen?
27. Emil Ivanov 1.4.09 / 11am

    Aza - I just wanted to point you to another project that is very similar to ubiquity, but for the desktop - Gnome Do: http://do.davebsd.com/
28. Jim Jeffers 1.5.09 / 2am

    Hey Aza,

    I really liked this idea and wrote a jQuery script to produce something similar for blog commenting:

    http://donttrustthisguy.com/2009/01/04/encouraged-commentary/

    I gave you credit for the idea. I really like this design pattern. I’m not sure how successful it will be unless more people implement it for other reasons with selected text. It will be interesting how it all plays out. I could see it becoming troublesome if browser plugins and sites both implement similar functionality with highlighted text.

    Either way thanks for sharing this. I found it extremely insightful and useful.
29. John 1.8.09 / 2pm

    Potential solution to invoking Ubiquity without a selection:
    Left click and hold, right click once or twice, release left click.

    I know there are a lot of people who rely on rocker gestures but it’s just an idea.

    Plus, there’s always the push and hold right click.
30. Blair McBride 1.14.09 / 2am

    A bit late in getting to this, but better late than never… Anyway, here’s my list of thoughts I had while watching the video and experimenting:

    * drag and drop selection into side panel to act on new selection, when panel is already open
    * panel should be on left side, where the normal ubiquity popup is? user expectation
    * badge can be dual purpose - can also act as a target to right-click for a gesture based menu (pie menu)
    * command suggestions could be in a frequency graph
    * command suggestions in categories + best matches category

    - Blair
31. Francisco 1.16.09 / 6pm

    Hi Aza,

    i think that the fact of exist 3 diferent boxes in the same interface of the browser is very confused for the user. I believe that the three boxes should be only one like as in the navigation bar of the Google Chrome, for simplicity and for improve the usability.

    One post about Google Chrome and Ubiquity (sorry, only in spanish but you can make click in the link of the automatic translations to english):

    http://tentandole.blogsome.com/2009/01/16/google-chrome-20-ubiquity-navegadores-sistemas-operativos-y-el-nuevo-modelo-de-interaccion/

    From Madrid, Spain, cordial greetings
32. Philip Renich 1.17.09 / 11pm

    I haven’t been able to read over all the comments, but I figure if one more voice speaks, at least you can see where people are leaning.

    With that said, I love what you have done here. It’s very smooth and seamless. I have a couple thoughts in relation to your questions.

    1 - “We tried having the badge fade in after a short time”
    There is no reason to make users wait. If they have Ubiquity running, they want this kind of interaction and expect it. Rather than fading in, consider a fade out after something like three seconds. This means that if they are just highlighting to read, as they continue toward the end of the highlighted area where the badge rests, it will disappear out of their way.

    2 - Where should the pane rest, how should it deal with scrolling?

    I like right. For cultures that read left->right it makes sense. It’s like the page turning, moving forward, rather than backwards. The only argument I see for the left-hand side is often in textbooks, the left margin of a left page will contain snippets and asides to the reader.

    As for scrolling, would it make sense to hold the Ubiquity pane fixed while the actual website scrolls by itself?
33. stylized.fact 1.20.09 / 9am

    Where did the “Types of (Context) Menus” graphic come from?
34. Pete 1.20.09 / 7pm

    Maybe the badge only should be viable when you hold the pointer over the selected text?
35. Sam Hasler 1.21.09 / 5pm

    The popup might be better positioned below the selection as with a multi-line selection that ends mid-line the popup appears over the selection.

    There’s also an occasional positioning bug if you select up to the end of a line the popup can appear over the start of the next line. Also, try selecting text by dragging from right-to-left on the about:ubiquity page for some strange results.
36. cabotine 1.30.09 / 2pm

    I am impressed… I think this extremly usefull…
    options to select commands

    view selection source would be a useful one for example
    Great stuff
37. Tom 2.5.09 / 11pm

    For opening the interface without a selection - what about using a mouse click held down for some duration? Or multiple mouse buttons? (old school Mac users - get with the times, and get a multi-button mouse!)
38. Juan Perez 3.12.09 / 11pm

    Like this?
    http://www.microsoft.com/windows/internet-explorer/beta/videos.aspx?vindex=2

    :(
39. Arno Nyhm 3.25.09 / 10am

    i would like to enable it on every page (not only after typing setup on every page)

    like

    setup mouseubi always on
    setup mouseubi always off

    or

    setup mouseubi enable
    setup mouseubi disable
40. شات 11.27.09 / 4pm

    VVere Nice>
41. دردشه صوتيه 12.4.09 / 8am

    VVere Nice>
42. شات 2.17.10 / 5am

    i would like to enable it on every page (not only after typing setup on every page) [شات كتابي] [شات صوتي][كوت الكويت]

All material copyright Aza Raskin. But you can have the color magenta.

Entries RSS Comments RSS