Archive for the ‘Software Development’ Category

Ubiquity Update: 0.1.2

Sunday, October 26th, 2008

Ubiquity Update

We’re pleased to say that the distributed team at Mozilla Labs working on Ubiquity has recently released an update to Ubiquity. If you already have Ubiquity, it will auto-update itself. Otherwise, you can download it here.

What’s In It?

We’ve got an amazing amount of feedback about Ubiquity since it’s launch. One of the main points has been wanting features and bug fixes for the commands included in Ubiquity, as well as more commands. The web moves quickly and Ubiquity has the ability to stream commands: it’s silly that we aren’t using that ability to let development of those built-in commands rage on, without being impeded by the slower Ubiquity release cycle. The largest update in “Raging Stream” is the ability for all of the built-in commands to be streamed in from the HTTPS-secured ubiquity.mozilla.com. Along with that is an experimental way of having all preview  and command code combined into one large file.

Also, for those of you who are writing commands, we’ve greatly improved the Ubiquity command editor. For a full list of changes and notes, check out the release notes on the wiki.

Welcome!

Between Ubiquity 0.1.1 and 0.1.2, we are proud to have been joined by many new faces. In particular, two new members now have commit access to Ubiquity. Please welcome Fernando Takai, and Davanum Srinivas! By the quality of code, and quality of their participation in the community they have shown themselves to be stewards and champions for the rest of our contributors.

If you have patches or questions, please don’t hesitate to ping us on the #ubiquity channel on IRC at irc.mozilla.org. The handles of the folk who can review code are now fern, dims, jono, atul, abi, unfocused, and aza.

ContextFree.js & Algorithm Ink: Making Art with Javascript

Monday, June 30th, 2008

In honor of the the release of Firefox 3, I’m releasing ContextFree.js today, along with the demo site Algorithm Ink. ContextFree.js is about drawing striking images—and making art—with minimal amounts of code.


An introduction to ContextFree.js & Algorithm Ink

Computers programs lost something important when displaying a splash of color stopped being one line of code. As a kid, I remember being able to type “plot x,y” on the Apple II to throw up a phosphorescent splotch. When the simplicity of the one-line plotter went away, so did the delight at being so effortlessly generative—in a visual way—on the computer. ContextFree.js is a stab at making it easy again. It’s like a grown up version of Logo (or at least the Turtle Graphics part of Logo).

Go ahead and play with it on Algorithm Ink. It has goodies like a gallery to see other people’s art, the ability to view the source-code of any piece of art, and the ability to save the art to your desktop with a single right-click (that comes along for free, more on this in a second). It works best in Firefox 3, but should also work in Opera and Safari.

The inspiration and kick-in-the-pants motivation for this project came from the always-excellent John Resig (also of Mozilla) finally releasing Processing.js.

It’s All About the Javascripts

ContextFree.js is a port of Context Free Art by Chris Coyne. Algorithm Ink uses open-web, standards-based tools: no plugins required. It uses Canvas to to the drawing, and Javascript to compile and interpret the Context Free grammar. The code that does the compiling and render is open-source and available here. Handling thumbnails and other manipulations of images is just a couple lines of Javascript. No need to muck around with server-side black magic. With the addition of arbitrary transforms to Safari 3.1, all modern browsers, except IE, support Canvas fully enough to run ContextFree.js.

One of the great things about using open-web standards is that it plays nicely with user expectations. If you like a particular piece of art, you can just right click to save it as an image. It would also be possible to use an image as the background for a blog—imagine a unique piece of art adorning your site, one that’s different for every visitor. The bleeding edge versions of Webkit have, in fact, implemented the ability to use Canvas with CSS. Open-web technology helps to break down the silos of inaccessibility: backgrounds images done with Flash have never been adopted because of the heavy-weight feel, as well as the lack of interoperability with the rest of the DOM. As Canvas takes becomes more wide spread, I’m looking forward to an explosion of infographics and other data visualizations that weren’t possible before without a heavy server-side setup, or without a compile step that breaks the ability to view-source-and-learn cycle of the web (Flash, Java, &c, I’m looking at you).

A Peak at the Syntax

The grammar is pretty simple. Let’s start by making a circle:


startshape shape

rule shape{
 CIRCLE{}
}

This says start by drawing the rule named “shape”. The rule shape says to draw a circle.

Now let’s make things more interesting:


startshape shape

rule shape{
 CIRCLE{}
 shape{ s .5 b .2 x -.25 }
}

We’ve added a single line which says that part of the rule for “shape” is to draw itself, scaled down by half, with a slightly increased brightness, drawn on the left side of its parent circle. This makes an off-centered bulls-eye pattern.

Now let’s go to the right, as well as the left:


startshape shape

rule shape{
 CIRCLE{}
 shape{ s .5 b .2 x -.25 }
 shape{ s .5 b .2 x .25 }
}

For comparison, this is what the equivalent code in Processing.js looks like:


// Processing.js code to make the same circle pattern.
void setup()
{
  size(200, 200);
  noStroke();
  smooth();
  noLoop();
}

void draw()
{
  drawCircle(126, 170, 6);
}

void drawCircle(int x, int radius, int level)
{
  float tt = 126 * level/4.0;
  fill(tt);
  ellipse(x, 100, radius*2, radius*2);
  if(level > 1) {
    level = level - 1;
    drawCircle(x - radius/2, radius/2, level);
    drawCircle(x + radius/2, radius/2, level);
  }
}

It’s much longer, involves more setup, and more fiddly calculations. In fact, the entire ContextFree.js source is as long as the setup function. Of course, by simplifying so much in ContextFree.js, we’ve also removed the flexibility Processing.js affords.

Adding one more line of code gives an unexpected result: The Sierpinski Triangle. I’ll leave it as an exercise for the reader to figure out what that line of code is, but I doubt that it will be much of a puzzle.

Follow the mouse

The equivalent of “Hello, World!” for a graphical environment is a dot that follows the cursor, replete with a fading trail. This is what it looks like in ContextFree.js:


startshape DOT

rule DOT{
  SQUARE{ s 4 b 1 a -.9 }
  SQUARE{ s .3 }
}

rule MOUSEMOVE{
  DOT{ }
}

We start by defining the rule “DOT” which draws two things: A large square (scaled up 4x), which is white (brightness of 1), and mostly transparent (alpha of -.9), and then a smaller black square. The large transparent white square acts to fade out more and more of the previous DOTs that have been drawn (it’s a standard trick for doing motion blur effects). The rule MOUSEMOVE simply draws a dot at the current mouse position, whenever the mouse moves.

Here’s the result.

There’s one more example that demonstrates the rest of the features that makes ContextFree.js powerful at the end of this post.

The Big Picture

Besides being pretty, why is ContextFree.js interesting? Because it shows the power of Open web technologies for making graphically-enabled, compelling interaction. The true power of the web revolves around anyone being able to dive in, see what someone else has done, and expand upon it. Canvas lowers the cost of entry to creating graphical mashups and other dynamic, graphical content. It also shows the progress the web has made: a year ago, this demo would not have been possible. Canvas wasn’t ready, and Javascript interpreters weren’t fast enough. Looking at the qualitative difference in speed from Firefox 2 to Firefox 3 indicates the amazing and substantial progress made towards speeding up Javascript since the last major browser release cycle.

Implementation

There are three major components to ContextFree.js: a tokenizer, a compiler, and a renderer. Each can be instantiated and used separately. The compiler returns a JSON object of the parsed code, which makes it easy to write a new front-end. I’d be interested to see if a Flash implementation of ContextFree.js would be faster than the pure Javascript, and if so, how much faster. As a side note, I’d also like to see an implementation of the Canvas API done in Flash, as a better replacement for excanvas for IE.

Because the computation and drawing of fractals can be intensive, the rendering occurs in it’s own threaded queue. How quickly it iterates over the shapes is dependent on how long the last drawing operation happened. This helps to keep ContextFree.js from freezing the browser.

There were a couple problems with browser inconsistencies and Canvas deficiencies that make the renderer interesting.

The first problem I ran into was that whilst the Canvas specification does support setting the transformation matrix, it does not support getting the transformation matrix. This means that if you want to know how large a shape will be when you draw it (for instance, to know when an element in a fractal is too small to be seen and is therefore safe to not draw), or want to draw in a threaded environment where you can’t guarantee a global transformation state, you need to re-implement the entire transformation infrastructure. This is a bad breakage of DRY, and slow from a development standpoint. According to Ian Hickson, the reason for the lack of a getter seems to stem from a possible implementation detail, although I don’t understand the argument. In personal communications with irrepressible Mozillian Vlad Vukićević, it appears that there is an possibility of adding the getter to the Firefox Canvas API.

The second problem I ran into was that versions of Safari older than 3.1 do not support setting the transformation matrix. The work around isn’t pretty. It involves taking the desired transformation matrix, de-convolving it into an equivalent set of rotations, scales, and translations, and then applying those base transforms to the Canvas. That’s a lot of math and eigen values. Luckily, Dojo already did the heavy lifting and I was able to stand on their shoulders for ContextFree.js.

The last problem I ran into is that even the latest editions of Safari do not support .setTransformation, which means that there is no call to return to reset the current transformation matrix to the identity. I used lead Webkit-developer Maciej Stachowiak’s solution, which is to save the pristine transformation state (using .save), perform the desired transformation and draw step, and then restore the pristine transformation state (using .restore).

The end result is that while there are inconsistencies, JS libraries can tide us over until us user-agents get fully in sync. That said, I recommend the addition of .getTransformation to the Canvas specification: it will save a lot of unnecessary code rewriting, most of which is matrix multiplication best done in a low-level language.

Improvments

There are a number of improvements to be made to both ContextFree.js and Algorithm Ink. With the former, it isn’t a full port (it’s missing things like comments, z-index, and loop notation). With the later, there is still a strong disconnect between a text-based input and a graphical output. I would love the ability for authors to indicate which variables are interesting in their code, and have the UI expose that variability through sliders and other graphical means. I’d also like to graphically teach the system rules: for example draw a couple shapes, group them, copy and scale them, and have Algorithm Ink generalize that as a rule, translate that rule to code, and draw the full fractal.

The ever-inspiring Bret Victor had some excellent suggestions that I hope someone takes up:

* Highlight a section of code (or “mark” a rule somehow), and the parts of the picture that were generated by by that code/rule are marked somehow—by applying a tint or a glow, perhaps. Because things are so recursive, you should be able to tell when a part of the picture has been marked multiple times—darker tint, eg. The idea is to be able to quickly explore a program and see what’s doing what.

* The opposite: put the mouse over a pixel, and see what code is responsible for that pixel. Ideally, you could see the entire call stack. Perhaps with annotated pictures so you can see what was drawn at each level of the call stack. Let the artist answer the question, “Why did this [part of the picture] happen?”

* Scrub through the construction of the pic. At each timestep, both the picture being added and the code responsible for it are highlighed. The current transform matrix is shown. I can step through time and understand what is happening and why.

Given that ContextFree.js’s “bytecode” is simply a JSON object, as I mentioned above, it wouldn’t be hard to write a new front-end. For example, here’s the “bytecode” for the “follow the mouse” example:


{
  startshape:"DOT",
  DOT:[
  {
   weight:1,
   draw:[
    {shape:"CIRCLE", s:4, b:1, a:-0.9},
    {shape:"CIRCLE", s:0.33, b:1, a:-0.5},
    {shape:"CIRCLE", s:0.3, b:0.5, sat:0}
   ]
  }],
  MOUSEMOVE:[{ weight:1, draw:[{shape:"DOT"}] }]
}

Get Involved

ContextFree.js is open source and hosted on Google Code. Feel free to jump in. Feel free to email me at my first name at mozilla.com.

Appendix A: Full Example

The last example adds demonstrates a couple more features that make ContextFree.js powerful:


startshape scale

rule scale{
  spiral{s .1 y -.5 x 1}
}

rule spiral{
 SQUARE{ a -.5 }
 spiral{ y 1.5 r 10 s .99}
}

rule spiral .01 {
 TRIANGLE{ }
 spiral{ y 1.5 r 10 s .95}
 spiral{ y 1.5 r 10 s .95 flip 90}
}

Let’s step through the code. The “scale” rule simply makes everything smaller and translates it down and to the right. Let’s look at the first rule “spiral”. It creates a half-transparent square, then makes a slightly smaller square that’s transposed and rotated ten degrees to the left. Repeating this gets a stepping-stone spiral. There are, however, two rules for “spiral”. This is where randomness comes into the design: whenever the rule “spiral” is encountered, ContextFree.js randomly chooses one definition among all definitions of that rule. The number after the rule name indicates the weight of the choice, where the default weight is 1. Thus, this rule draws something like this:

Modify the code slightly we can get an almost hand-drawn look that makes for a compelling background image.


startshape scale

rule scale{
  shape{s .1 y -.5 x 1}
}

rule shape{
 SQUARE{ b 1 s 1.1 }
 SQUARE{ a -.5 }
 shape{ y 1.5 r 10 s .99}
}

rule shape{
 SQUARE{ b 1 s 1.1 }
 SQUARE{ }
 shape{ y 1.5 r 5 s .99}
}

rule shape .03 {
 TRIANGLE{ }
 shape{ y 1.5 r 10 s .95 b .1}
 shape{ y 1.5 r 10 s .95 flip 90 b .1}
}

A large version of this image is also available. You can also play with this pattern live on Algorithm Ink. And for those who missed it, here’s the ContextFree.js code. Email me, or comment below to get involved.

Enso 2.0 Design Thoughts

Thursday, January 31st, 2008

As part of the our move to Mozilla and thinking about a free-as-in-speech Enso, I want to be more transparent with our design directions and goals. Our designs can only benefit by incorporating the criticism and suggestions of the community we have here. Open-source design is a balancing act between making final decisions and finding consensus. We hope to take the lessons that Jono spelled out in his excellent article on successfully humane open-source projects and use them in our own projects.

This post is about the new directions we are taking Enso. If you haven’t done so yet, start by reading about some of the motivations for doing some Enso redesign. In short:

  • Enso shouldn’t make you type all of “open” every time
  • Enso should be able to open paths and urls
  • Enso should support international character input
  • Enso should gracefully handle the case where there’s no convenient place to enter text
  • Enso shouldn’t require you to type out text, select it, and then run a command when you’d rather run the command and then enter the text (think calculate)
  • Enso shouldn’t make you hold down a key while typing lots of characters

We think we’ve solved these problems with our Enso 2.0 redesign. In this post, and possible follow on posts, I’ll walk through the new stuff. I should note that our upcoming prototype will not yet have have all of the features mentioned here. (more…)

Enso Developer Prototype

Wednesday, October 24th, 2007

Ever since we launched Enso, we’ve been getting feedback like this:

— When are you going to come out with the Python API for commands?
— I’m looking for a way to extend Enso.
— Is it is possible to enrich Enso with custom commands?
— I heard there will be support to create your own commands. How far off is this?
— An SDK would be wonderful!
— Being a Python developer, I am very interested in the ability to expand Enso.
— Would you have any available API’s?

The entire Humanized team is extremely proud to announce that the answer to whether is “Yes“, and the answer to when is “Now“! Today marks the release of Enso Developer Prototype, which lets anyone develop commands for Enso. It’s free and it’s available for download. (more…)

Undo Made Easy with Ajax (Part 2): Time-Sensitive Actions

Monday, October 22nd, 2007

This is the second part of the Undo Made Easy with Ajax series. If you are jumping in late, make sure to check out part 1.

Last time we covered the event queue method, an entirely client-side way of implementing a light-weight, multi-level undo. I mentioned a couple of caveats, including that it does not work in multi-user collaborations, and that it doesn’t work for time-sensitive actions like sending emails. I missed a significant caveat that one of my readers, Alexander Botero-Lowry, pointed out: That two tabs, pointed to the same page, could get out of sync. I wrote about how to solve that issue with cookies.

This time, let’s take a look at solving Undo for time-sensitive actions. In the event queue method, we could wait until the “onunload” event to sync the user’s actions from client-side to server-side. For time-sensitive actions like sending emails, we don’t have that luxury. Worse, email is a push technology. Once an email has been released into the wild tubes of the Internet, it cannot be unsent.* For the unfortunate who hits “send” only to realize that they’ve CC’ed their boss on a steamy love letter, the only things left to hope for are power outages and spam filters. Given how often spam slips past my filters, the outlook is not so good. (more…)

Undo Made Easy with Ajax (Part 1.5)

Friday, September 21st, 2007

This is the second half of the first part of the Undo Made Easy with Ajax series. If you are jumping in late, make sure to check out Part 1.

One of my readers, Alex Botero-Lowry, pointed out a big caveat to the entirely client-side event queue method of implementing Undo: If a user opens a second tab to a page they were viewing, those pages might be out of sync. In the to-do list example, if you delete three to-do items and then open a new tab/window to the same page, those to-do items would still be there. Why? Because the first page knows about the deletes (even though they are uncommitted) and the second does not.

Let’s take the client-side only approach and run with it. There is something nice about not having to mess with the back-end. We can fix the multiple-tab problem by syncing the event queue across all open pages with a cookie. (more…)

Undo Made Easy with Ajax (Part 1)

Friday, September 14th, 2007

As users, we make mistakes. As designers, we need to design with mistakes in mind, as I argued in my recent article, Never Use a Warning When You Mean Undo. Undo is the ultimate safety net, lending an incredible sense of solidity to an interface. That’s why every desktop application from Word to Photoshop provides multiple-level Undo.

So, then, why are Web apps that provide any sort of Undo so few and far between? The answer I often get is that Undo is hard to implement. I’m here to tell you that it is not.

In this series of blog posts, my goal is to explain just how easy it is to provide Undo functionality. Recently, I gave a preliminary version of this post in a workshop. After giving the front-facing demo of how Undo could work, the audience moved slightly towards the edge of their seats (it’s all you can hope for in the post-lunch session). When I opened the source code and started showing how I implemented undo, the universal response was, “Why are you bothering to explain this implementation? It’s barely anything at all. We’re software engineers. This is easy.”

That’s my point!

Adding Undo to your interfaces profoundly and positively affects the usability of your site. It reduces user frustration, and increases user trust. Both of those outcomes mean that more users continue coming back, which helps your bottom line. Remember: To the user, the interface is the product. (more…)