Fixing IE by porting Canvas to Flash

This is a guest blog from Grant Jones.

In the Algorithm Ink blog post, I mentioned that I’d be interested to see if a Flash implementation of ContextFree.js would be faster than the pure Javascript, and if so, how much faster. Would an implementation of the Canvas API done in Flash make a better solution for Canvas in IE than Google’s excanvas? Within days, Grant had jumped in, wrote some slick code, and answered those questions!

My hat is off to Grant for the awesome work he’s done, as well as for writing this post.

Intro to Internet Explorer Canvas-Emulation

Before we go into the current state of Internet Explorer canvas-emulation, here’s a brief look back at the history - or the history as far as I could tell from blog posts:

1. Manish Jethani mentions something about the canvas tag being used for a 3D game, John Dowdell from adobe responds in Nov. 2005
2. Manish Jethani suggests a way to do a SWF-based Canvas on Dec 1st, 2005
3. iecanvas from Emil A Eklund released December, 2005 (uses VML)
4. excanvas from Google released March, 2006 (uses VML)
5. AFLAX (uses ExternalInterface) adds canvas emulation in March 2006

Overall, interest in this area seems to die off after that. Only recently with things like Processing.js and ContextFree.js has there been a renewed interested in backwards compatibility for the canvas tag.

What is FlashCanvas?

FlashCanvas is a canvas tag to flash bridge for browsers which do not support the canvas tag like Internet Explorer. FlashCanvas is an experiment to see if any performance improvement could be realized by using flash technology. The javascript code is a fork of the ExplorerCanvas project:

Firefox, Safari and Opera 9 support the canvas tag to allow 2D command-based drawing. ExplorerCanvas brings the same functionality to Internet Explorer. To use, web developers only need to include a single script tag in their existing web pages.

Back in July I was using the canvas element to create simple ‘platformer’ style games. The original idea was to create one basic game every Monday, Wednesday, and Friday. The first few were Flash-based but the rest were done in javascript and HTML. The problem is that Internet Explorer still has a major percentage of the market share and couldn’t play my games. I tried excanvas from Google, found it was too slow, and in the grand tradition of rolling-my-own created FlashCanvas.

How does FlashCanvas work?

FlashCanvas is modeled after ExplorerCanvas which means it is a turn-key solution for adding Canvas support to IE. You can code away, happily using open standards and then use FlashCanvas to forcefully and silently upgrade IE to also being standards compliant.

There are two main components in FlashCanvas: the base FlashCanvas.swf flash file (a mere 688 bytes), and the FlashCanvas.js wrapper. I’ve used the excellent swfobject.js to embed the Flash into the page.

The FlashCanvas.js file implements a fake-canvas object and converts all existing canvas element into a flash object. The javascript intercepts canvas commands and forwards them to the FlashCanvas.swf movie file using the ExternalInterface provided by the flash player. The flash movie clip then interprets the command and draws accordingly.

Download FlashCanvas

Download it here: FlashCanvas 0.2.

Version 0.2 includes a batched mode. Look for the “example1_stress.htm” inside of the examples folder for a demo of it in use. It’s Released under the same Apache License as the explorer canvas project.

FlashCanvas is a one-day project, meaning that this release only implements lines and fills, which is only enought to support the first two examples from ExplorerCanvas.

Unfortunately, there exists a flash player bug which does not allow local connections to communication with actionscript from javascript using ExternalInterface even if permission is granted. A python script for running a SimpleHTTPServer instance is provided for testing purposes.

Does FlashCanvas provide better performance?

The short version: not really.

The longer version: All drawing commands are being forwarded from Javascript -> ExternalInterface (down into the browser) -> ActionScript -> Flash Drawing commands. This is a lot of overhead for drawing code. Doing something simple like a moveTo and lineTo would result in at a very minimum 8 separate levels of indirection.

Performance

FlashCanvas did not yield as much of a speed increase as I was expected. The question is, where is the slowdown occuring.

Flash performance is not the limiting factor as it is roughly the same across all platforms, and is quiet speedy with graphics. The speed of javascript execution is also not the limiting factor. Although, from the limited testing I did it is clear that javascript executes slower on Internet Explorer than in either Firefox or Safari — it is not enough to explain the difference in render times.

The slowdown is occurring somewhere between the javascript call and the actual rendering to the Flash MovieClip. To investiage further, I wrote a dummy ping-pong page/flash file which simply calls a “ping” method from javascript into actionscript to test timing.

The results revealed where the problem was:

JS2AS = JavaScript calling ActionScript; JS2AS2JS = Javascript calling ActionScript and ActionScript calling Javascript; OS X and Windows machines were physically different

Each call via the ExternalInterface is taking approximately 0.5 ms. Since we know the time it takes to call an actionscript method from javascript, we can deduce the amount of time it takes for Flash to render. For the example1.htm page it takes ~24 ms to render 20 anti-aliased lines. In Safari a call into actionscript takes roughly 0.4 ms — this is only the call time, it doesn’t include the time to render anything. For example1.htm a single “particle” calls actionscript 3 times with the commands: [lineStyle,moveTo,lineTo]; each particle takes 1.2 ms in JS-to-AS calls; to take 1 second to render the frame ~833 particles need to be rendered. Changing the example code to render that many particles resulted in:

(where time is in milliseconds)

The conclusion is: the time it taken for the javascript canvas wrapper, the javascript particle simulation and the actionscript code/line rendering is insignificant.

For comparison this is safari doing the same exact thing using it’s native canvas element:

(where time is in milliseconds)

The next logical step is to batching draw commands into an array to be passed only once into the flash renderer. Immediately this raises an issue about how the canvas API has been typically used: there is no explicit end of rendering or flush equivalent so the canvas-wrapper would never really know when drawing is finished. The most fine-grained control would be at the individual path level, which would result in nearly the same performance for something like the example1.htm file. Most graphics API have some sort of buffer flush either explicitly, like glFlush in opengl, or implicitly during a buffer swap.
Still, it was worth testing to see if this could solve the performance issues. A flush was added to the canvas-wrapper but batching commands into an array does not eliminate the performance problem in ExternalInterface. FlashCanvas was changed to batch commands and then send them to flash with only one call via ExternalInterface. The result is only slightly faster (800 ms instead of 1 sec and a really slow 20 secs /frame in Internet Explorer 7). Since that part of flash is all closed this was basically as far as I could go — my guess is that some sort of serialization/translation is taking all the time.

What other approaches could be taken?

Here is where we look to the recent news that Mozilla’s Vladimir Vukićević has a working prototype Active-X plug-in which emulates the canvas element. This is ultimately the best bet as far as performance is concerned. There may still be potential performance issues due to the fact that all the javascript code is still being interpreted by Internet Explorer which seems to have some undesirable characteristics especially in memory management.