Ambient Information in the Browser
My browser knows a lot about me. It knows what I’m typing, what I buy, what websites I go to and how often, my calendar, and with whom I communicate. In fact, my computer probably knows more about me than I do myself. Yet, despite the wealth of information about my interests and habits, my browser rarely uses that knowledge to make my life easier.
In Mozilla Labs, we’ve been starting to think about how to harness the data the browser has to make your browsing experience better—while also maintaining privacy and security. We are particularly interested in “ambient” use of this information. That is, we don’t want to add extra features to the browser that require interaction. The data should be able to gracefully enhance our browsing experience without getting in our way.
Take, for example, RSS feeds. Despite being widely evangelized, RSS is still a technology mainly for the technorati. Why is that? Because it is a technology that is meant for consumption by computers instead of humans, and (more importantly) there is a high time-cost of setting up an RSS reader and populating it with feeds.
My browser knows which sites I visit frequently and which sites I check compulsively. Instead of manually having to subscribe to an RSS feed, the browser should augment my experience with the information it knows I want. By doing so, it takes the power of RSS and bite-sized updates and pushes it mainstream. It gracefully enhances my online experience — helping to reduce information overload, without bothering me.
Where should we display this information? Well, there’s that tempting giant white area displayed every time a new tab is opened…
There are other ways that the browser can use ambient information to silently enhance my browsing experience. The Awesome Bar is currently the best example of using information in an ambient way to enhance the browsing experience by tailoring its behavior, in a more or less tacit way, to my own. There are others. My browser knows where I read my web-mail; why not bubble the number of unread messages up into the interface (a sort of mail-pressure indicator)? My browser knows which tabs I’m switching between frequently; why not use that information to make my workflow more efficient? Why not pre-fetch the pages I visit often, and pre-render them so that my standard browsing actions become instant-fast? The browser knows I visit certain sites in the morning, and different sites in the evening. There’s something that the browser should do with that information.
One of my favorite uses of tacit use of such information is in Bret Victor’s Bart Widget, which learns where you are going from and to every day. It does this by silently watching your behavior; if you are traveling to East Bay in the morning and back to San Francisco in the evening, when you launch the widget it will default to showing the train information you most likely want.
We should be doing this kind of no-cost action more often in the browser. In Labs, we’ve got some specific thoughts of how to apply this in Firefox, but before we taint the thinking with our thoughts, I’d love to open the discussion.
How should ambient information, and all of the meta-data available to the browser, be used to enhance our everyday browsing experience?
UPDATE: Atul Varma has released Ambient News, which is a add-on that begins to explore the RSS idea from this post. In fact, this post was entirely inspired by Atul (which he says was inspired by some Humanized work we did for a company called OpenEnd—inspiration is hard to track…).
RT @azaaza Ambient Information in the Browser | Follow @azaaza on Twitter | All blog posts
Anon
Interesting posts. But some of your suggestions are in my eyes not a good idea: I’m already drowning in information, so please don’t add a mail notifier and please don’t fill that peaceful white space in a new tab with attention-grabbing noise :-)
David Naylor
The place to start would be to improve the form-filling experience. Make Firefox better at guessing what I want to fill into a form. Perhaps give the user a right-click option to “Let Firefox fill out the forms on this page”.
iacchi
If I can add my 2 cents to the discussion, I’d like to discuss about the idea in general rather than how to apply it.
When I read this topic I first thought it was an amazing thing to have everything I need (more or less :P) given by the browser automatically, without loosing time. Then, I started to think about it, and I thought that no, it isn’t such a good idea.
Let me explain: as we said at the summit, Internet (the whole IT) is divided into two categories: mainstream users and core users. Let’s try to see the point of views of both parts.
The majority of IT mainstream users don’t know anything about the things they interact with: they just find and use them. If they will find these new things you’re proposing in their browser, the first times they’ll find these things fantastic, something new. Then, they will completely change their mind. Souspiciousness and fear aren’t always rational, and a browser that now a lot about you, and it shows it to you can be regarded as a privacy-breaking threat, even if it isn’t.
Moving your train ticket example a step forward; people don’t feel what’s given, but complain when something is worse than the usual. Let me explain: to buy a train ticket I have to choose the itinerary, and buy the ticket; it’s a normal thing and I know I have to do it if I want the ticket. Now: If I buy 100 times the same ticket and 5 times a different ticket and the browser know it and it show me a precompiled itinerary, when I need that ticket I’ll need a step less than the usual to buy the ticket, but I won’t understand (maybe better say think about) it, because it’s a given thing; but if I have to buy a different train ticket, I’ll have to cancel the given itinerary, choose the right one, and buy the ticket, which is one step more than the usual, and now I’ll be aware of the fact and complain about it. Actually, sometimes I’ve found myself in a similar situation and sometimes I’ve complained, too!
Now the core users point of view: as I said, I first thought that this was an amazing thing, but later I’ve changed my mind. The fact is that in this way you save time, but loose control. Let’s take the rss feed example: on one hand it would be time-saving to have the browser knowing your habits and showing you what you need, but what about if what you need isn’t a mirror of your habits? I’d fear everytime the browser doesn’t show me something I want and I’d look everytime for what’s missing, loosing a lot of time. You may say that you can configure it for some special cases, but this way I can simply set up a feed reader. Moreover, even if you really save time, you loose control in all cases. Actually, I prefer to loose some time and keep control of what I’m doing, understand it, rather than save time and loose consciousness. It’s just a mental habit, and I think most core users have it.
If I can make a simile, it’s like WYSIWYG editors against hand-writing code (let’s say HTML): maybe a WYSIWYG editor is faster, but if you use just the WYSIWYG features you loose control on the code, and I’m sure almost all the peole that you can call core users write their code by hand, because they want this control.
So, what I want to say is: don’t throw this idea in the bin, but think very carefully on what you do.
Now, to finish this long comment: I’m writing a summry of what we said at the summit for my team, and reading my notes I saw a sentence said by one of the Mozilla labs members (I think) during the browser history and future talk: “creating an interface that appeals mainstream users alienate core users”. Well: this is bad.
While it’s with the mainstream users that a browser (or, actually, anything in general) gets popularity, are the core users that build the browser the mainstream users will use. Core users are the developers, the localizators, the bug filers and the evangelizers. If core users loose interest on a project (and this is more true with FLOSS software) because it doesn’t reflects any more their needs, you’ll loose everything. Why should I spend my time on a project I don’t need and I don’t believe in? It’s plenty of projects that need my help outside there.
So: ok to think about mainstream users, but keep core users quiet.
I hope my English is understandable, ask if it is not.
iacchi
I’m reading now the previous comments: the auto-filling thing (only when called) would be a great idea!
Patrick Dubroy
@iacchi: I don’t think it’s accurate or productive to say that all Firefox users are either “mainstream” users or “hardcore” users. I think that open source usability has often suffered from this kind of attitude.
Anyhow…
I do think it would be great if Firefox could use some of this contextual data to improve the interface. However, this kind of automation is difficult to get right.
The first thing I’d like to see is existing UI elements enhanced by some of this contextual information. For example, you could use subtle effects on a tab to indicate whether the tab has been read or not, how long it’s been since you were on that tab, or how much time you have spent on that tab. In the awesomebar and/or the history, you could give more weight to pages that you visit more frequently, or spend more time on.
iacchi
Ok, maybe I’ve been a little bit too drastic, but I’m not so far from the truth. About the features: you’re talking about little enhancements in your comments, and they’re good. Aza, in his post, is talking about big things.
Winz
I don’t have a proposition to give for a use of ambient information in information, but at least i can give you a quite good example of it.
Some people may have think about it too, it’s about the “News Feed” of your facebook profile.
It doesn’t exactly use YOUR ambient information, but without any interaction or settings asked from the users, it brings together in the “top stories” events, applications or photos that have a high probability to be related to you (even if sometimes it’s just a kind of snowball effect: your friends are using this social app, so you will too).
And one of its caracteristics that makes this news feed easier to work is that it is a feed : it has a temporal aspect. It is a flow, even in the settings it requires less interaction from you to just work and be useful.
Moreover the news feed doesn’t need to be used itself, using facebook is enough.
Just have friends.
Aza Raskin
@Anon: Information overload is a huge problem on the web, one that I fight constantly. There are lots of ways to make ambient information exacerbate the problem and not help it. We should be avoiding those :)
The browser contains a lot of data. The crux of the difficulty in making good AI isn’t in accumulating data, it’s in filtering and reducing large amounts of data into useful inferences. Figuring out how to throw away the right data is difficult, but it’s worthwhile to think about.
@David Naylor: Good point. Are there any extensions that you think do it right?
@iacchi: Lot’s of good points in there. You are exactly right that trying to make interfaces too clever yields something that is annoying. C.f., Microsoft’s Clippy. When contextually enhancing an interface we need to make sure that even when we guess wrong, we still don’t encumber the user.
People often find change scary. You have to be different to be better, but there are costs for being different. It’s a balance. If we can change the browser to go from a quarter of a billion people being helped, to 2 billion people we — together — have made the right choices.
@ Patrick: Agreed. It’s really hard. That’s why I think we haven’t seen much innovation in this space — it takes a lot of thinking to not move backwards. I’d love to see some mockups for those subtle enhancements. Let’s see if we can get them into the browser.
Aza Raskin
@Winz: Great example. It would be interesting to combine the Coop with news feeds somehow. Something else to look at is Myk Melez’s Snowl, which is a prototype backend for supporting messages in the browser.
Mike Krieger
Great post — I think there are a ton of great possibilities in thinking about ambient tech in the browser, especially since we spend so much time in our browsers, and our browsers “know” (or could infer) a ton about us.
One thing I’d love to see is some application of Will Hill’s “read wear” idea on the sites we visit — a ‘calm technology’/ambient demonstration of how other visitors arrived, read, interacted, and otherwise left their mark on a site could be very interesting. Browsing the Web is often a stream-of-consciousness experience, through the pages we visit — seeing or even following other’s Web/brain-streams might be fun.
Jimmy
I was just wondering why there is no link to Atul’s
Ambient News? (unless of course I’m done and over looked it) Especially considering both articles came out on the same day. So far (very little testing) I like it. This is a wonderful idea and can’t wait to see what it produces.
Thomas Holloway
I believe one option that may be worth exploring is aggregating recommended content – be it RSS Feeds or just Websites. It would be nice to be able to explore the web in this “ambient” manner.
Thomas Holloway
I think that one route of content filtering may not necessarily be looking at the large spectrum of “relations” but recommending those little links that we have never visited despite their direct connection to sites we have visited.
Thomas Holloway
I have a feeling like I’m stating the obvious, but would it be brash to say that this kind of “ambient” tech is already done with the Google Toolbar and Google Trends?
Aza Raskin
@Jimmy: This post drew a lot of inspiration for conversations with Atul. Now that Ambient News is posted, I’ve linked to it. Thanks!
Aza Raskin
@Thomas: I’m not quiet sure what you mean by “ambient” tech in the Google Toolbar — not that I’ve played with it much. Want to fill me in.
@Thomas, Mike: The constant fight for bubbling up more information is information overload. Showing the “flows” of the web are interesting, but doubly so if it can help reduce the information overload problem. Any thoughts on how to do that? My hunch is to use that data to augment what I’m already seeing&mash;like making links that my friends have clicked on look more bold, or brighter, etc.
Petr Stedry
Like the BART Widget, Firefox could really present in the new tab:
* 3 – 5 page thumbnails the user most frequently visited around the current time
* additional 3 – 5 pages (although not so pronounced, not those displayed above) the user also visited during this time period
This would be like Speed Dial minus the tedious manual configuration. And you could even start to pre-load the most frequently visited pages with a clear progress indicator (if the users’ line to the internet is not used (closest approximation is if the browser itself is not downloading anything).
Other use for this would be to pre-fill those annoying ‘certify your age’ inputs. But they’re usually found inside Flash video players anyway, so though luck with this one.
Another thing, that would be great to see is contextual history searches. Imagine this scenario:
* (optional step) the user (me) has the web history set to 365 days ;)
* the user visits a page and asks the browser if he doesn’t know about any already visited pages, that match information about the topic the currently displayed page presents …
This could be done by indexing some content on the page and storing that in the browser history along with the URL.
You could take your imagination for a spin and project this concept one step further and you get:
– the user visits a page and asks the browser if some of his favorite (=frequently visited or bookmarked or starred or whatever) sites contain some information the currently displayed site does.
David Naylor
“@David Naylor: Good point. Are there any extensions that you think do it right?”
I have been trying a few just now, and in a word: no. They aren’t smart enough and,mainly, they require a lot of pre-configuration.
Instead, form filling should take a few lessons from the awesomebar:
1. Use data from forms on all visited websites to give suggestions. So if I have ever entered “David Naylor” into a box it will be suggested when I want to enter it (and start typing “D…”) on a website I never visited before.
Using all the saved entries as suggestions will give us a problem of very/too many suggestions. This can be countered by the following improvements:
2. a) Count form entries per site: When visiting Gmail, keep track of the fact that I’m entering my own user name almost every time and put it at the top of the suggested list (even before I type anything). Then stick my girlfriend as number two, etc. (Currently entries are listed in the order they were first entered, right?)
2. b) Count form entries across sites: When I visit a new webshop, understand that I’m most likely to type “David Naylor” when I start typing “D…” into a form.
3. I guess, ideally, Firefox should understand whether I’m about to type into a “name” box or an “e-mail” box and use that knowledge to bring up the relevant suggestions quicker/further up. This function would have to be implemented with non-english websites in mind though.
4. When asking whether to save a password, give the user the option to “log in automatically” (fill login + password and submitting) in the future.
Anup
Have you seen IE8 Beta 2’s new “blank” tab? Looks like similar (and useful) ideas, perhaps? Here is one screenshot: http://news.cnet.com/2300-1032_3-6246111-9.html
I think more can be done because of all the browsing history etc, letting people refine what they are looking for when they open a new tab on the blank page itself, based on the places they have visited etc. I guess the additional challenge is how to do it in a slick way so that it is not seen as ugly and intrusive.
Zayıflama Lida Fx15 Ve Biber Hapı Zlfvbh
Interesting posts. But some of your suggestions are in my eyes not a good idea: I’m already drowning in information, so please don’t add a mail notifier and please don’t fill that peaceful white space in a new tab with attention-grabbing noise :-)