This is my personal blog. The views expressed on these pages are mine alone and not those of my employer.

Monday, October 17, 2005

AMASS: Ajax MAssive Storage System

On Sunday I finished the README and instructions for working with AMASS, the AJAX MAssive Storage System that makes it possible to permanently store large amounts of data on the client-side of browsers, opening the door to new kinds of AJAX applications. Download the latest alpha release of AMASS, or view the README file.

The README file:

AJAX MAssive Storage System (AMASS)


Brad Neuberg

bkn3@columbia.edu

What is This?

The AJAX MAssive Storage System (AMASS) uses a hidden flash applet to allow
JavaScript AJAX applications to store an arbitrary amount of sophisticated information
on the client side. This information is permanent and persistent; if a
user closes their browser or navigates away from the web site, the information
is still present and can be retrieved later by the web page. Information stored
by web pages is private and locked to a single domain, so other web sites
can not access this information.

AMASS makes it possible to store an arbitrary amount of sophisticated
data, way pass the 4K limit of cookies or the 64K limit of Internet Explorer's
proprietary client-side storage system. An AMASS-enabled web site can store up to 100K without user permission. After 100K,
users are prompted on whether the web site can store the requested amount of
information. Users can approve or deny the storage request. The AMASS system
informs the client-side application on whether the storage request was allowed
or denied. In my own testing I have been able to store up to ten megabytes
with good performance; I'm sure even more information can be stored, I just
have never tried beyond this amount.

AMASS works on Internet Explorer 6+ and Gecko-based browsers, like
Firefox. Users must have the Flash plugin version 6+ installed to use
AMASS; Flash 6+ is installed in 95% of machines, however.

News

Versions

The most recent version of AMASS is version 0.02.

Contributors

The primary developer on AMASS is Brad Neuberg.

License

AMASS is available under a BSD license.

How Do I Use This?

Working with AMASS is simple. The AMASS framework creates the abstraction of
a permanent hash table that persists even after the user has left the page
or closed their browser.

The first step in working with AMASS is to load the AMASS script:

<!-- Load the Permanent Storage framework -->
<script src="storage.js"></script>

AMASS must wait for its internal machinery to finish loading. In order
to use AMASS, you must wait until it is finished loading by adding
a listener:

storage.onLoad(initialize);

function initialize() {
}

Once AMASS is loaded, you can begin to work with it by using it's hash table
methods, such as put, get, and hasKey:

var keyName = "message";
var keyValue = new Object();
keyValue.message = "hello world";
keyValue.testArray = ["test1", "test2", "test3"];
keyValue.testObject = {someProperty: "someValue"};

if (storage.hasKey(keyName) == false) {
storage.put(keyName, keyValue, statusHandler);
}
else {
var results = storage.get(keyName);
}

The AMASS framework makes it possible for you to serialize entire JavaScript
objects into the storage system, such as the keyValue object we
serialize above. Note that DOM nodes and browser objects, such as the
XMLHttpRequest object, will not be serialized.

Applications can store up to 100K without user permission. After this,
a popup generated by the underlying Flash system is created that prompts the
user for permission. The AMASS framework knows when the popup appears, generating
a DIV and bringing the Flash file to the forefront of the application, centering
it on the screen:


Users can either APPROVE or DENY a storage request. You must create your

application ready to have its storage request denied. The put()
method takes a status handler as its third argument that informs your code on whether the storage
request was successful or not. In the code above, statusHandler
is a callback function that will receive whether the request was successful
or failed:

function statusHandler(status) {
 if (status == Storage.SUCCESS) {
   var results = storage.get(keyName);
   alert("Results from statusHandler="+results);
 }
 else if (status == Storage.PENDING) {
   alert("Results pending approval of storage space from user");
 }
 else if (status == Storage.FAILED) {
   alert("Storage request denied");
 }
};

Status can be one of three values: Storage.SUCCESS,
Storage.PENDING, or Storage.FAILED. If the popup
appears, you will get a callback of Storage.PENDING. Later, if
the user approves the request, you will receive a Storage.SUCCESS;
if the request was denied, you will receive Storage.FAILED. When
the user approves the request, they can also indicate whether they give
permission to future requests to automatically store information without
having to popup the permission dialog again.

When working with giant strings or XML, you should not use the default
put() or get() methods. Instead,
you should use the putString() and getString()
methods. The put and get methods attempt to
unserialize a stringified JavaScript object back into a real one, using
eval(). If you are just storing a dumb string or XML than this
will major impact your performance, especially for large datasets. For
very large datasets you should definently storage your information as XML
strings or just plain strings using the putString and
getString methods.

How Does It Internally Work?

Internally, we use a hidden Flash file and Flash's SharedObject
functionality to permanently store the information. We script the Flash
file using it's ActiveX methods on IE and it's LiveConnect methods on
Firefox. We use Flash's SharedObject's callbacks to detect
when the request storage dialog is on the screen, and pass these back to
the JavaScript application. We also center these values on screen.

Articles



The O'Reilly Network will be publishing
a forthcomming article, titled "Permanent Storage for AJAX Applications."
This article will go into much more detail, with full examples, than this README
can accomodate.


Demos and Examples

The O'Reilly Network article will have full examples. Until then, see the
testing class for the AMASS framework for sample code:

Support

My primary means of support is through open source consulting using the
kinds of frameworks I create and give away. For this reason
I generally charge for support. I can answer simple questions, but beyond
that we should structure a consulting arrangement to solve your problems
using the AMASS framework. For details, email at
bkn3@columbia.edu
or call me at 510-938-3263.

Download

Download the latest release of the AMASS framework.

Known Issues


// posted by Brad GNUberg @ Monday, October 17, 2005 2 Comments Links to this post

Really Simple History

Yesterday I finished the README file detailing how to work with the Really Simple History framework; download the full Really Simple History framework.

The README file:

Really Simple History


Brad Neuberg

bkn3@columbia.edu



What is This?

The Really Simple History (RSH) framework makes it easy for AJAX applications
to incorporate bookmarking and back and button support. By default, AJAX systems
are not bookmarkable, nor can they recover from the user pressing the browser's
back and forward buttons. The RSH library makes it possible to handle both cases.

In addition, RSH provides a framework to cache transient session information that
persists after a user leaves the web page. This cache is used by the RSH framework
to help with history issues, but can also be used by your own applications to
improve application performance. The cache is linked to a single instance of the
web page, and will disappear when the user closes their browser or clear their
browser's cache.

RSH works on Internet Explorer 6+ and Gecko-based browsers, like
Firefox. Safari is not supported.

News

Versions

The most recent version of RSH is version 0.04.

Contributors

The primary developer on RSH is Brad Neuberg. Special thanks to Erik Arvidsson
for several important suggestions concerning the framework, as well as Alex
Russell and the Dojo Toolkit project for pioneering history support in AJAX
applications.

License

RSH is available under a BSD license.

How Do I Use This?

The RSH framework exposes two primary objects you will use to add history
support to your application: dhtmlHistory and
historyStorage.

dhtmlHistory is the primary entry point for adding bookmark
and back button support to your application. The primary flow when working
with RSH is as follows:

  1. Initialize dhtmlHistory
  2. Register your application as interested in being notified whenever the user
    presses the back or forward buttons.
  3. Determine the initial location of your application and initialize your
    state accordingly.
  4. As a user interacts with your AJAX application add history entries
    to the dhtmlHistory object. The history entry specifies a new location,
    such as edit:somePage, as well as some optional contextual data that
    is associated with the new location. When you add a new history entry,
    the RSH framework updates the browser's location bar with the new location,
    added after a hash, such as http://somesite.com/myapp#edit:somePage.
  5. If a user presses the back or forward buttons, the RSH framework will call
    the history change callback you registered earlier, passing in the new
    location as well as any history data that might have been associated with
    this location by you. The browser's URL field will also jump between any
    previous or next hash history entries.

The chief idea behind RSH is that your application adds custom history
events as a user interacts with the application; the framework itself uses the
history entry to update the browser's location bar. The only way to update a browser
location without reloading the entire page is by using a hash fragment, so the
new location is added after a hash fragment, such as #edit:somePage.
When you add a history entry, you can also pass in contextual data that is
associated with this location; this is optional information and can be useful
for sophisticated uses, such as saving the state of an edit form.

If a user presses the back or forward buttons, the RSH framework "jumps"
through any history entries that were added by the programmer before. The
RSH framework then calls your history change listener, passing in the new
location as well as any history data that was associated with this location.
Your application now has the responsibility of taking this location and updating
it's UI accordingly; RSH doesn't update your UI at all, since this is very
application specific.

You should choose location URLs that contain enough state for you to
update your application or initialize yourself. For example, if the user
jumps to the location #view:somePage, your application should
be able to use this to know that you must remotely fetch somePage
and view it. Choose location URLs that make sense to your application, recording
enough state to re-initialize themselves.

Finally, you must include a file named blank.html in the same
directory as your application. This file is included with the RSH
download and is needed by Internet Explorer.

Here's some simple pseudo-code on how to use RSH:

/** RSH must be initialized after the
   page is finished loading. */
window.onload = initialize;

function initialize() {
 // initialize RSH
 dhtmlHistory.initialize();

 // add ourselves as a listener for history
 // change events
 dhtmlHistory.addListener(handleHistoryChange);

 // determine our current location so we can
 // initialize ourselves at startup
 var initialLocation =
               dhtmlHistory.getCurrentLocation();

 // if no location specified, use the default
 if (initialLocation == null)
   initialLocation = "location1";

 // now initialize our starting UI
 updateUI(initialLocation, null);
}

/** A function that is called whenever the user
   presses the back or forward buttons. This
   function will be passed the newLocation,
   as well as any history data we associated
   with the location. */
function handleHistoryChange(newLocation,
                            historyData) {
 // use the history data to update our UI
 updateUI(newLocation, historyData);                          
}

/** A simple method that updates our user
   interface using the new location. */
function updateUI(newLocation,
                 historyData) {
 var output = document.getElementById("output");

 // simply display the location and the
 // data
 var historyMessage;
 if (historyData != null)
   historyMessage = historyData.message;
  
 var message = "New location: "
               + newLocation
               + ", history data="
               + historyMessage;
              
 output.innerHTML = message;
}

Our HTML is straightforward; it simply consists of a serious of links
that users can click on, which we use to update the history:

<body>
 <h1>Output</h1>
 <div id="output"></div>

 <div onclick="dhtmlHistory.add(
                    'location1',
                    {message: 'hello world 1'})">
   Change to Location 1                  
 </div>       

 <div onclick="dhtmlHistory.add(
                    'location2',
                    {message: 'hello world 2'})">
   Change to Location 2                  
 </div>             

 <div onclick="dhtmlHistory.add(
                    'location3',
                    {message: 'hello world 3'})">
   Change to Location 3                  
 </div>  
</body>

Notice that we use dhtmlHistory.add() in the links above
to update the history with a new entry.

RSH also provides an object that can be used to store transient session
information for the page, named historyStorage.
historyStorage simulates a hash table, making
it possible to put and get name/value pairs that persist
even after the user has left the web page. Note that
these values are linked just to the single instance of
the web page they were stored on. If the user opens a new browser
window and navigates to your site then the values will not be
visible. For permanent, long-term storage of large amounts
of information you should use the AJAX MAssive Storage System (AMASS) instead.

Sample code for working with historyStorage:

/** RSH must be initialized after the
   page is finished loading. */
window.onload = initialize;

function initialize() {
 // initialize RSH
 dhtmlHistory.initialize();

 // storage a value into the historyStorage
 // if it doesn't exist yet; otherwise, grab
 // the pre-existing value from the history
 // storage
 var savedObject;
 if (historyStorage.hasKey("message"))
   savedObject = historyStorage.get("message");
 else {
   savedObject = new Object();
   savedObject.id = "someId";
   savedObject.message = "Hello Watason";
   savedObject.testArray = new Array();
   savedObject.testArray[0] = "Hello";
   savedObject.testArray[1] = "World";
   savedObject.nestedObject = {someProp: "bye"};
  
   historyStorage.put("message", savedObject);
 }

 // now work with savedObject
 alert("message="+savedObject.message);
 alert("nestedObject="+savedObject.nestedObject);

The key for historyStorage must be a string, while the
value can be an arbitrary JavaScript object. The RSH framework will fully
serialize your JavaScript object and pull it back out as an object.
Note that DOM nodes and native browser objects, such as XMLHttpRequest,
will not be saved or persisted.

Both historyStorage and dhtmlHistory
have a few more methods that are useful in some cases; see the
source code for details.

How Does It Internally Work?

RSH works differently internally for different browsers, but in general
we use a combination of hidden iframes, timers, and hidden form fields
to detect various history changes and to persist history and location data
in a way that will still be around if the user leaves and then returns
to the page. The hidden form field is used to persist information
between page loads using the auto save
capability of web forms; see my blog
post
"AJAX Tutorial: Saving Session Across Page Loads Without Cookies,
On The Client Side" for implementation details. historyStorage
wraps the auto save trick with an easy hash table API for developers, and
the main RSH framework then uses the historyStorage class to implement
stateless tracking of history; variables that allow the detection of fake
versus real page load events; and more.

Articles

The O'Reilly Network will be publishing
a forthcomming article, titled "AJAX: How to Handle Bookmarks and Back Buttons."
This article goes into much more detail, with full examples, than this README
can accomodate.

Demos and Examples

The O'Reilly Network article will have full examples. Until then, see the
two testing classes for the RSH framework for sample code:

Support

My primary means of support is through open source consulting using the
kinds of frameworks I create and give away. For this reason
I generally charge for support. I can answer simple questions, but beyond
that we should structure a consulting arrangement to solve your problems
using the RSH framework. For details, email at
bkn3@columbia.edu or call me at 510-938-3263.

Download

Download the latest release of the
Really Simple History framework.

Known Issues


// posted by Brad GNUberg @ Monday, October 17, 2005 4 Comments Links to this post

First Post! W00t!

I just found my old, very first blog post floating around in cyberspace, from my first weblog named Tinker, hosted at EditThisPage; this is from May 2000:

"Welcome to Tinker! This is where I play around, "tinker", and explore new ideas. Some will pan out and lead to cool things, others will drop like a thud. I'm a lover of the new, so I'll be posting junk-loads of links to interesting technologies, ideas, and net-politics to this 'blog.

Let me introduce myself. My name is Brad Neuberg. I live in New York City, and recently graduated from Columbia University. I work at a great company named Random Walk, who told me when they hired me that it would be okay to tinker after work! For an inveterate open-sourcer (and a crabby Socratic who finds faults to fix in everything), this was a dream, since I can't stop my mind from thinking of new stuff. I also learn a great deal at Random Walk, since they're on the bleeding-edge of Java and CORBA.

Since I'm no supporter of vapor-ware, I've started this 'blog out by posting three projects that I am currently either working on or recently worked on. The first one is FlashJava. This is an open-source project that is attempting to integrate Flash and Java together; check out the source-code and the presentations on the FlashJava site. Another project is Fusion, which attempts to integrate previously disparate communication paradigms like email, instant messaging, newsgroups, and source-control systems. There is a prototype based on Mozilla, screen shots, a presentation, and a whitepaper on the Fusion site. The third project is OpenPortal, which is an older, discontinued project but which still has alot to offer in terms of ideas for EditThisPage. This was a project to create open, growable, transportable, changeable, and interoperable websites. Check out the OpenPortal website for the full whitepaper, design document, and final code drop."


// posted by Brad GNUberg @ Monday, October 17, 2005 0 Comments Links to this post

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]