Coding In Paradise

Just in time for Thanksgiving is another SVG Web release. Our tradition is to name SVG Web releases after monsters from D&D just to increase the geek factor, so in that spirit our release name this time is "Gelatinous Cube":

The Gelatinous Cube is a truly horrifying creature:

A gelatinous cube looks like a transparent ooze of mindless, gelatinous matter in the shape of a cube. It slides through dungeon corridors, absorbing everything in its path, digesting everything organic and secreting non-digestible matter in its wake. Contact with its exterior can result in a paralyzing electric shock, after which the cube will proceed to slowly digest its stunned and helpless prey.

Fun times.

Highlights of this release, thanks to many many people helping with patches, bug testing, and more:

• Loads of important bugs fixed
• Performance improvements
• You can now dynamically create new SVG root tags
• All the namespace aware functions now implemented: setAttributeNS, getAttributeNS, etc.
• You can now clone SVG nodes (cloneNode)
• You can now right-click on the SVG when using Flash to view the dynamic updated SVG source
• Running getElementsByTagNameNS scoped to a particular node now works, such as myGroup.getElementsByTagNameNS(svgns, 'text')
• and much much more

The full list of issues fixed:

• Issue 358: Opera throws exception on patch to currentTranslate
• Issue 413: Implement node.getElementsByTagNameNS scoped by container node
• Issue 401: currentTranslate.setXY does translate the svg, but doesn't affect currentTranslate.getX or getY
• Issue 201: Support cloneNode
• Issue 385: Implement getAttributeNS
• Issue 384: Make sure ownerDocument defaults to 'document'
• Issue 386: Implement hasAttributeNS and hasAttribute
• Issue 387: Implement removeAttributeNS and removeAttribute
• Issue 202: Programmatically Creating the SVG node
• Issue 383: QA on Firefox 3.6 Beta
• Issue 335: Specific SVG file crashes browser when used with SVG Web
• Issue 364: onload event does not fire when image url is a security error
• Issue 362: dynamically applied color should cascade
• Issue 349: gradients with bounding box cooordinates are positioned wrongly on circles
• Issue 371: Linear gradients incorrectly start at left of screen rather than each circle
• Issue 367: Flash blend mode used for groups is a performance problem and can be avoided generally
• Issue 331: Animations based on events not implemented; problems with transform animations
• Issue 275: Add View Dynamic Source to context menu
• Issue 297: Aspect resolution of viewBox not honored on resize of browser
• Issue 238: SVGImageNode.as should absorb exceptions due to invalid image URL
• Issue 337: https generates insecure warning with IE6 (FIXED for IE7 and IE8)
• Issue 388: Cannot Add USE elements dynamically
• Issue 361: Text label is stealing mouse click

Our featured SVG Web project this release is the work Michael Neutze has been doing which he shipped last week:

Today the Federal Statistical Office of Germany (think census bureau
or national statistics) published their latest population projection
for Germany until the year 2060 and updated their animated population
pyramid. It had been ASV3 only when it was first introduced in 2003,
then in 2006 it was ASV3 plus native browsers (Firefox 1.5+, Safari
3+, Opera 9+) and now it uses SVG Web.

Apart from supporting Internet Explorer SVG Web was a great choice for
mixing SVG and HTML. Population projections only make sense when you
know the assumptions and those can now be displayed side by side with
the population pyramid using standard HTML and CSS. New in this
release is also the ability to switch between four different
scenarios.

SVG Web's small size (only three files), minimal requirements on the
server-side and the ability to use standard markup were the main
selling points.

Check it out for yourself, it is available in English, French, German
and Russian at
http://www.destatis.de/bevoelkerungspyramide/

Watch a short screencast in english to see all it can do
http://vimeo.com/7687560

Learn more about SVG Web:

• Latest Download
• SVG Web Home Page
• 1 Minute Introduction to SVG Web
• Quick Start Guide

About SVG Web and SVG:

SVG Web is a JavaScript library which provides SVG support on many browsers, including Internet Explorer, Firefox, and Safari. Using the library plus native SVG support you can instantly target close to 100% of the existing installed web base. SVG itself stands for Scalable Vector Graphics, an open standard that is part of the HTML 5 family of technologies for interactive, search-engine friendly web vector graphics.

Labels: announcement, open source, release, svg, svgweb

I saw Ralph Nader speak at Google yesterday, and he had the best definition of freedom I've ever heard. An excerpt from Nader's 2004 commencement address:

"Cicero, while we are quoting the ancients, once defined freedom as participation in power [bold added]. I think it is the best definition of freedom that I have heard. Freedom is participating in power such as a deliberative democracy, such as government, such as corporations, such as trade unions. People have to participate in those areas of power that affect their lives so intimately."

It sets a high-bar and makes me ask how the open systems I'm engaged in live up to it. I'm also not sure yet how to integrate meritocracy into this, which I think is important, especially for open source. Any ideas?

Labels: freedom, open source, politics

This qualifies as an Evil Genius Project of the week. If it works its amazing:

"NestedVM provides binary translation for Java Bytecode. This is done by having GCC compile to a MIPS binary which is then translated to a Java class file. Hence any application written in C, C++, Fortran, or any other language supported by GCC can be run in 100% pure Java with no source changes."

Anyone play with this? How well does it work?

Labels: java, open source, web extension mechanisms

Folks toss the term "Open Web" around a bunch, but what exactly is it? Is the Open Web HTTP, HTML, JavaScript, etc., or is it something deeper? Rather than a laundry list of technologies the Open Web is a set of philosophies. These philosophies include:

• Decentralization - Rather than controlled by one entity or centralized, the web is decentralized -- anyone can create a web site or web service. Browsers can work with millions of entities, rather than tying into one location. It's not the Google or Microsoft Web, but rather simply the web, an open system that anyone can plug into and create information at the end-points.
• Transparency - An Open Web should have transparency at all levels. This includes being able to view the source of web pages; having human-readable network identifiers, such as URLs; and having clear network entry points, such as HTTP and REST exposes.
• Hackability - It should be easy to lash together and script the different portions of this web. MySpace, for example, allows users to embed components from all over the web; Google's AdSense, another example, allows ads to be integrated onto arbitrary web pages. What would you like to hack together, using the web as a base?
• Openness - Whether the protocols used are de facto or de-jure, they should either be documented with open specifications or open code. Any entity should be able to implement these standards or use this code to hook into the system, without penalty of patents, copyright of standards, etc.
• From Gift Economies to Free Markets - The Open Web should support extreme gift economies, such as open source and Wikis, all the way to traditional free market entities, such as Amazon.com and Google. I call this Freedom of Social Forms; the tent is big enough to support many forms of social and economic organization, including ones we haven't imagined yet.
• Third-Party Integration - At all layers of the system third-parties should be able to hook into the system, whether creating web browsers, web servers, web services, etc.
• Third-Party Innovation - Parties should be able to innovate and create without asking the powers-that-be for permission.
• Civil Society and Discourse - An open web promotes both many-to-many and one-to-many communication, allowing for millions of conversations by millions of people, across a range of conversation modalities.
• Two-Way Communication - An Open Web should allow anyone to assume three different roles: Readers, Writers, and Code Hackers. Readers read content, Writers write content, and Code Hackers hack new network services that empower the first two roles.
• End-User Usability and Integration - One of the original insights of the web was to bind all of this together with an easy to use web browser that was integrated for ease of use, despite the highly decentralized nature of the web. The Open Web should continue to empower the mainstream rather than the tech elite with easy to use next generation browsers that are highly usable and integrated despite having an open infrastructure. Open should not mean hard to use. Why can't we have the design brilliance of Steve Jobs coupled with the geek openness of Steve Wozniak? Making them an either/or is a false dichotomy.

Notice something very important; at no time above did I bind the Open Web to a particular set of technologies. Today the above philosophy is instantiated using a particular set of technologies, including URLs, HTTP, HTML, CSS, JavaScript, etc. However, if we define the Open Web in terms of these technologies, then we risk losing sight of what makes the web special and being able to have the intellectual nimbleness to evolve the infrastructure of the web. For example, we can and should evolve better layout languages than CSS, better document formats than HTML, etc., especially if we want the web to survive as a long-term social institution and public good, similar to the electricity grid, public-water systems, etc. We will be fighting yesterdays battle while allowing new, proprietary technologies to take over if we focus on technologies rather than philosophy. If we take the long term view, how can we give the web an open enough infrastructure to evolve over time and meet each generations needs, while maintaining its structure enough to actually mean something and stay true to its promise, similar to the U.S. Constitution?

The Open Web is like something from an Arthur C. Clarke science fiction story: its a globe spanning, hypertext network containing billions of documents, conversations, and applications, used by a huge cross section of society. Who would have thought it ever would have been successful or stayed as open as it has? It's not controlled by any one government or company. Our historical closeness to the web creates a kind of myopia, where we can't see how amazing it is. It's a billion Library of Alexandria's dropped into our laps.

Douglas Engelbart, father of much of what we have in the computer industry, including the mouse, hypertext, and groupware, thought that computers would become as fundamental to humanity as the development of writing and language have been. Language is probably only about forty-thousand years old, while writing is only about eight-thousand years old. If you pull back and take the larger view, the web and computers are part of a grand development playing out over decades around new tools for communication. Writing and language have fundamentally changed our sense of self, with positive and negative ramifications; computers and the web hold the same promise, though it will take decades for this to play out.

If the web and computers hold this promise, its important to keep the resulting system as open and accessible as possible. Do we want a system that devolves into something like Ancient Egypt, with an authoritarian force controlling and centralizing the water supply? Many archaeologists believe that deep control of access to water, literally something required for life, lead to the longest known authoritarian civilization in history, lasting for thousands of un-broken years. For example, what if the pencil and paper had never escaped the grasp of the Church? If we can keep the open web nimble and open, it can set the stage to fuel further innovations and inventions, just as writing and language gave rise to books, social polities, etc.

If we agree that the Open Web is important, how do we create a way to update the web and keep it relevant? The U.S. Constitution, for example, includes special provisions to evolve itself and stay relevant. Even with its warts, the U.S. is now the world's oldest and continuous republic.

The web's existing update mechanisms just don't work. It takes years for new features to go from proposal to show up across enough browsers to be used consistently; this is a recipe for fail if we want the web to exist as a long-term entity, rather than a one-hit wonder.

I joined Google to help with a project known as Gears. Gears is an open source plug-in that teaches current web browsers new tricks. Gears is a clever way to raise the bar cross-browser and cross-platform, today, running inside of Firefox and Internet Explorer on Windows, Mac OS X, and Linux. No more waiting years for features to show up across all browsers and platforms.

I want to look back five years from now and say that I worked with the community to build an open source update mechanism for the web. Why can't we rev the base infrastructure of the web much quicker, plus create more robust, open extension points along the entire web stack, ala Greasemonkey? Is Gears the answer to this? I'm not sure, but its the best answer we have today. Gears is a great way to get the conversation started, plus get HTML 5 out to today's browsers.

On a more prosaic level, Gears gets the following features into today's browsers without waiting years, ready to use by web devs right now:

• A real embedded relational database (SQLite) for web sites
• Client-side full text search
• Thread-like execution for JavaScript
• Offline web applications
• Secure and fast cross-domain mashups
• Desktop shortcuts
• Better web APIs for mobile devices
• and more.

[Disclosure: I work with the Gears team]

Labels: gears, open source, open web

I've created some simple utilities to make it much easier to use Google Gears without having to delve into JavaScript. This new open source library, named Gears PubTools, is a simple collection of JavaScript files that make it easy for content authors to work with Google Gears with just a little bit of HTML. With PubTools, you can easily offline-enable your content within minutes:

- Point to a Gears manifest file (a special file that lists all the pages to take offline) with a simple HTML attribute:

<html gears-manifest="manifest.js">

- Create a desktop shortcut icon that points right to your app and
use a custom image:

<html shortcut="true" gears-manifest="manifest.js">
<head>
 <link rel="shortcut.icon" title="32x32" href="icon32x32.png"></link>

- You don't even have to write the manifest file yourself! PubTools includes a simple bookmarklet that developers can drag to their browser's toolbar. Just navigate to the web pages you want offline, press the "Generate Manifest" bookmarklet, and a dialog will appear with the full manifest file generated for you ready to cut and paste into a file.

Quick Start

Want to get started using PubTools? First check out the simple demo page, then get started:

1) Use the PubTools bookmarklet to create a Gears manifest file. Go to src/manifest-bookmarklet.html to get the bookmarklet and follow the directions. Save this to a file, such as manifest.js.

2) In the web page you want to offline-enable, pull in the PubTools JavaScript file:

<script type="text/javascript" src="pubtools-gears.js"></script>

3) Point to your Gears manifest file:

<html gears-manifest="manifest.js">

4) If you want a desktop shortcut icon, add shortcut="true":

<html gears-manifest="manifest.js" shortcut="true">

You must provide at least one icon in PNG format; possible sizes are 16x16, 32x32, 48x48, 128x128. Providing all of them is best to allow the OS to choose the best one. Specify them using a LINK tag:

<link rel="shortcut.icon" title="16x16" href="icon16x16.png"></link>
<link rel="shortcut.icon" title="32x32" href="icon32x32.png"></link>
<link rel="shortcut.icon" title="48x48" href="icon48x48.png"></link>
<link rel="shortcut.icon" title="128x128" href="icon128x128.png"></link>

By default, PubTools chooses intelligent defaults for the shortcut icon. You can override these if you wish using META tags:

<meta name="shortcut.name" content="Test Application"></meta>
<meta name="shortcut.description"
content="An application at http://www.test.com/index.html"></meta>
<meta name="shortcut.url" content="http://www.test.com/index.html"></meta>

Most of these have sensible defaults and can be left out:
'shortcut.name' defaults to the HTML TITLE element if not present.
'shortcut.url' defaults to the page's URL if not present.
'shortcut.description' is optional and defaults to the string
"Offline Web Application"

Note that shortcut icons are only in the development version of Gears; the currently deployed version of Gears does not support shortcuts yet. PubTools will correctly continue to function if the user's installed version of Gears doesn't support shortcuts.

5) PubTools provides a simple default, optional UI for the user. To get this UI, add an element with the magic ID "gears-pubtools-offline-widget" into your HTML:

<div id="gears-pubtools-offline-widget"></div>

When the page loads, the user will see "Download Offline." When pressed, everything in the manifest is made offline and the shortcut icon is created if desired, with the user given feedback on the process. Now, the user will see "Delete Offline." By pressing this a user can delete the offline resources and start over at a later date.

If you don't want the default UI, your custom UI can call PubTools.downloadOffline() to initiate the download of offline resources, and can call PubTools.deleteOffline() to delete the offlined content. These can only be called after the page is loaded.

6) That's it! You're done. See the demos/ directory for a demo and boilerplate you can copy.

Quick note: Make sure not to list the pubtools-gears.js file in your manifest file. The manifest generator takes this into account to leave this out, but if you create it by hand leave it out. We do a trick where when you are offline we simply don't have the file, which doesn't display the default UI to the user since it makes no sense to show that when offline.

I've opened a code.google.com open source project for PubTools. I've done a bunch of QA testing on it, but make sure to file any bugs you might find. If you have patches send them in to me.

Labels: announcement, gears, open source, release

I've put up a new release of Purple Include, iteration 3. Purple Include is a client-side JavaScript library that allows you to do client-side transclusions.

What the heck does that mean?

It means that you can include and display fragments of one HTML page in another without copying and pasting any content.

To use Purple Include, just add the following SCRIPT tag to the following of your HTML page:


The big new feature of this release is a new addressing scheme that is much easier to use, named 'quote'. Here's an example:


Notice the 'href' attribute; this will cause Purple Include to fetch the given page and inline it into that DIV (you can add the href attribute to DIVs, SPANs, BLOCKQUOTEs, Ps, and more). Also notice the #quote() at the end. This is the new addressing scheme in this release; you can now quote and grab a range of any remote page by just having a few words from the page, followed by three dots, followed by the end of the text to grab. In the example above we have:


This goes to the Dojo Offline tutorial, starts grabbing HTML and text from the beginning of 'In the background Dojo Offline is checking' and continues doing so until it encounters 'when they are on- or off-line so you don't have to'. Purple Include has proper HTML range support, which means that nested tags are correctly setup when they are returned and inlined into your page.

Here is the example actually running; after a few seconds you should see the content in-line (if you are reading this in an RSS feed click here to see the example):



The older XPath addressing scheme is still there for advanced usage; the quote syntax is just much simpler and has true range support. Details on the XPath scheme here.

The other big change in this release is that I have refactored the server-side addressing system to be truly generic, so that it is much easier to drop in new addressing schemes. For example, we could create a line() scheme to grab ranges of lines from source code files.

Please note that this software is still in development. In particular, the quote() scheme needs to be tested more, and I haven't done performance work on large documents yet. I wanted to 'release early release often' on this before getting it perfect.

You don't need to host anything to use this. If you want to run your own server, however, here is the ZIP file for you to download. In your HTML you will need to point Purple Include at your custom server using a META tag. See tests/example.html for details on how to do this at the top. The client-side is BSD licensed, the server-side is GPL.

Labels: announcement, engelbart, hypertext, hypertext geekery, open source, purple include, release

Thanks to Alex's blog I just saw this cool site called Ohloh:

"Ohloh is an open source network that connects people through the software they create and use."

Ohloh seems to automatically determine the top commiters for different open source projects, and then link them across different projects! Some interesting pages:

Here are the top 10 commiters on Dojo and their length of time.

Here's my individual Ohloh page with some statistics on my commits.

Here's the Ohloh Google Gears and GWT pages.

Are some of your projects or stuff on Ohloh?

Labels: ohloh, open source

Brian Dillard releases Really Simply History 0.6! Congratulations Brian!

(BTW, I'm on the Google Bus for my first day of work; wish me luck)

Labels: open source, projects, really simple history, rsh

I'm really proud of the great work Brian Dillard has been doing on the Really Simple History library. Check out the new beta release he just put out.

Labels: ajax, dhtml, libraries, open source, really simple history, rsh

Hi everyone! I'm excited to report that Brian Dillard is going to be the new maintainer of the Really Simple History library. Brian's a really sharp guy with a great background; RSH is one of my software babies and I wanted to make sure that it found a great new coder to help move it forward. The last year I've had to focus too much on my business and other open source responsibilities to give RSH the attention it deserves, especially since many people now depend on it, and Brian has a great list of ideas to bring it up to date to the latest round of browsers. Thanks Brian! I can't wait to see what you do with the library.

Labels: ajax, brian dillard, history, open source, rsh

Hi everyone; I'm back from my sabbatical and rearing to go. The first thing I wanted to hack on was to take the cool Purple Include work that Jonathan, Eugene, and I hacked together last month to a new version. I'm proud to announce another release of Purple Include, version 1.9 -- as you will see below, this is a major refactoring that simplifies working with the library and now works across all major browsers.

Purple Include is a client-side JavaScript library that allows you to do client-side transclusions.

What the heck does that mean?

It means that you can include and display fragments of one HTML page in another without copying and pasting any content. For example, you could quote the second paragraph from another person's blog entry by embedding something like:

<div href="http://foo.com/bar.html#xpath(/p[2])"></div>

in your blog page. The expression following the explanation point in the URL is an XPath expression.

If the page you want to transclude has a fragment identifier or a purple number, you can transclude that directly:

<div href="http://foo.com/purple.html#nid32"></div>

In fact, all you have to do is add an 'href' attribute to any of the following types of HTML tags in order to have that URL transcluded right into the page when the page loads:

<p href="http://foobar.com#nid32"></p>

<blockquote href="http://foobar.com#xpath(/p[2])"></blockquote>

<div href="includeme.html#foobar"></div>

<span href="../../relativefile.html#foobar"></span>

<q href="http://foobar.com#foobar"></q>

Here's the great thing about this new release -- there's nothing to install! We do some magic (see the Release Notes below to see how) to make it so that you have absolutely no server to install. I host everything on my webserver now, at codinginparadise.org, even the inclusion service and JavaScript, so all you have to do is add the following JavaScript to the top of your page:

<script src="http://codinginparadise.org/projects/purple-include/purple-include.js"></script>

Plus, Purple Include now works across Safari, Internet Explorer, and Firefox (Opera probably too -- I just haven't tested). The client-side JavaScript is now just about 9K.

See the example page for examples and usage. Also see the README file. You can view the JavaScript file as well if you want.

This is all beta stuff, so as usual, if you see bugs, tell me -- or even better, fix it ;)

RELEASE NOTES for August 15th, 2007 Purple Include 1.9

This is a pretty radical refactoring of the Purple Include code. The big highlights in this version:

* Now works cross-browser: Internet Explorer, Firefox, and Safari (Opera should work but has not been tested)

* There are no server-side requirements anymore. Instead, the inclusion service is hosted on my web site at codinginparadise.org and we do a trick in the JavaScript (the JSONP/Script tag trick) in order to do transclusions through a third-party web site. All you have to do is drop the JavaScript into your page and start using it.

* We no longer have an <hx:include> tag; instead, you can simply add an 'href' attribute to many different HTML types and have that type transclude it's contents:

<p href="http://codinginparadise.org/paperairplane#xpath(//[@id='table_of_contents']")</p>

This works for the following tags: P, BLOCKQUOTE, Q, PRE, DIV, SPAN

* The client-side script has gotten vastly smaller and simpler -- the script is now only about 9K.

* We now use the notation #xpath(//p) around an XPath expression rather than using an exclamation point, such as #xpath!//p. This is in keeping with the pseudo-standard that has developed around this practice, such as #xpointer() -- it also opens up the possibility of chaining together expressions in the future, such as #xpath(//expression1)xpath(//expression2), which would return the results of both expressions.

* The little roller image, roller.gif, used to be a pain in the butt to configure because it was always relative to the page you are using transclusions on, which therefore required you to specify a relative path or something using a META tag. To cut down on configuration, I now just host this image on my web server -- you can set it to your own path using the META tag 'purple.include.rollerURL' but we default it inside the code to my webserver. This means that all you have to do now to use Purple Include is have the JavaScript file -- that's it.

Labels: announcement, engelbart, hypertext, hypertext geekery, open source, purple include, release

My buddies Jay Dedman and Ryanne Hodson, video bloggers extraordinaire, did a video interview of me about Dojo Offline:

Labels: dojo, dojo offline toolkit, dot, open source, video, videoblog

Today we released the Dojo Offline Toolkit. Dojo Offline is a free, open source toolkit that makes it easy for web applications to work offline. It consists of two pieces: a JavaScript library bundled with your web page and a small (~300K) cross-platform, cross-browser download that helps to cache your web application's user-interface for use offline.

This was such a crazy application for me to work on. I've been crunching away on the offline problem for years, throwing different things against the wall to see what would stick: AMASS, Dojo Storage, early offline work using Dojo Storage and browser caching tricks, and now Dojo Offline. This project was a serious coding stretch for me; it involved using 6 different languages, 3 of which I don't even know (Perl, NSIS, Bash Shell Scripting) and one in which I'm frankly not very good at (C). Here's the languages it took for me to get this out the door:

• JavaScript - for Dojo Offline JavaScript library bundled with web page
• C - for local proxy
• Java - for server-side portion of Moxie demo app, using servlets and MySQL to create a full syncing example
• Perl - for Mac OS X installer
• Bash Shell Scripting - for Mac OS X installer
• Nullsoft Scriptable Install Script - for Windows installer

It also involved some relatively serious interactions with the Win32 API, Cocoa, and cross-compilation environments. I've gotten lazy doing web programming -- I forgot how damn hard client-side development is, and how slow it can be to develop. It's definently reinforced my belief that moving more of this to the web is the correct thing, just for the ease of QA and distribution.

This project almost didn't happen multiple times -- there were many times in which I thought it would be impossible and got pretty close to giving up, but the thought of not delivering kept me going. What we have up is a beta -- there are bugs and glitches for some edge cases, but the functionality is there and the core use cases work well. We now just have to drive the beta forward and keep iterating to press the bugs out across more and more tested PCs. Very special thanks to SitePen (Dylan Schiemann and Carrie Sackett in particular) -- without their support the Dojo Offline Toolkit would never have happened.

I'm really going to need the help of the wider community to pitch this tent. Dojo Offline is really just a scrappy open source project, and like any low-to-the-ground open source project it's resource constrained. What we really need is one or two more really good C-programmers. With that the sky would be the limit.

Ok, now I'm going to go sleep for a very long time :)

Labels: ajax, dojo offline toolkit, open source