Coding In Paradise

One of the things I was doing on my recent sabbatical was asking myself "what the heck am I working on, and why?" I was taking a bit of a breather to more clearly figure out what I want to focus on, and what direction I want to take my life and career.

Today I want to blog about having your own Personal Research Agenda. A Research Agenda is a list of questions to focus on -- they are the destinations you are tacking towards, the organizing principle around which you work. You might not be focusing on all of them all the time, but just formulating a list and putting it down will cause your mind to roll over them in the background like a kind of background thread.

Usually only research institutions and folks in academia have a research agenda, but I believe every computer scientist should as well. Research and invention are too important to leave to academia and corporate R&D labs -- academia doesn't do computer science research very well anymore ("how many angels fit on the head of a pin?"), and corporate R&D labs don't pump out the future of the future like they used to before the 90s. Plus, especially in computer science where you don't need an expensive laboratory or equipment like astronomy or experimental physics does, everyone is empowered to take their own stab at interesting questions.

There is a paper I love called "You and Your Research" that talks about having a list of questions that you should be pursuing, and having it clearly in your mind:

Here's my own list of 'important problems' -- some of them are old and I have been hanging out with them for awhile, some are very new. I've listed the question along with a bit of info on what I mean by each one, and some of the potential progress or software for each one that I might have thrown against the wall to see if it will stick. Just because I have a list of questions doesn't mean I have any answers -- these are more inquiries I'm holding on to to see if anything bubbles up. Who knows if I'll get any answers; at the end of the day I'm just a schlub asking deep questions :)

I'd love to see others create similar lists on their own blogs. One of the reasons for posting this list is I want to attract others who are interested in similar things, so we can brainstorm about them over a strong cup of coffee (or a strong Guinness).

Personal Research Questions

1. What does a true, next-generation Internet word processor look like, one that deeply embeds the browser and community?

   The word processors we have today are like a bad haircut from the 1990s. Microsoft Word, for example, is focused on printing pretty documents on paper, rather than focusing on such activities as lightweight collaboration, browser-based documents, advanced hypertext, and more. This is probably due to a breakdown in the market and a lack of imagination. We don't need new clones of Microsoft Office on the web -- what we need is deep, sustained innovation that merges Wikis into our documents and browser, and solves the conundrum of why groupware sucks.
   

   I have been obsessed with this research question for a long time. I've taken several stabs at it, turning the wheel a few times.
   

   The first one was OpenPortal, a project of mine from about 1999. In OpenPortal I combined DHTML like easy editing with the Wiki philosophy. From the OpenPortal docs:

   The second big project along these lines was Paper Airplane, which I touch on in the next research question.
   

   An interesting side effect of this research question is that almost everything I have done the last two years doing strange stuff with Ajax, such as the history support stuff with RSH; the client-side storage stuff with Dojo Storage; and the offline browser support with Dojo Offline was all done with an eye towards this goal: a few years ago I started another next-generation Wiki embedding some of the ideas above, named Kiwi, and wanted it to work offline, have good history support, etc. I then ended up working on those side issues instead of the primary question and Kiwi, which is not necessarily a bad thing.
   

2. Is it possible to continue creating radically new versions of the web and browsers that introduce fundamental new advances?

   The web is more than 15 years old now. While there have been some great breakthroughs in terms of the technical standards of the web the last 8 years, such as CSS, XML, DHTML, etc., our actual browsers have changed very little. Firefox cleaned up the browser and simplified things a bit, and there have been some nice features here and there, but it seems like we have created very few user-facing new breakthroughs in the browser-itself since the late 90s. Perhaps this is just the result of a mature technology, but my intuition tells me its more that we haven't focused on it and have ingested some Sacred Cows around the browser that we could challenge and bring some fresh thinking to.

   This has long been a goal I am passionate about. I mentioned Paper Airplane in the last research question; I poured everything I had into Paper Airplane a few years ago, from about 2001 to 2004. Paper Airplane was done in the great Dark Ages of the Browser, when the .com world had crashed, DHTML was "a dead technology", and Microsoft sat on the browser like a fat dragon roosting on a pile of gold. I was frustrated with things, sold off most of what I owned in New York City, and moved to an island in Thailand where I could live for about 200 bucks a month. There were no jobs to be had anyway, since the tech economy had collapsed, and living in New York City was bankrupting me, so I outsourced myself.

   While there, living on the edge of the jungle, I played around with some ideas for web browsers that deeply embed collaboration and editing. Working with Hoang Dinh, we created a full mockup of what this web would look like. Here's a brief excerpt from the Intro in the final Paper Airplane Research Paper:

   Paper Airplane tried to answer this question with a series of interesting new pieces of functionality; another excerpt:

   The full vision of Paper Airplane was too big for one person to create, so I prototyped out two major pieces of it that actually worked and shipped to prove some of the research study: a portion of the seamless editing/saving as a Mozilla plugin/toolbar called the Paper Airplane Plugin, and the P2P network itself called P2P Sockets. The Paper Airplane Plugin:

   P2P Sockets was an implementation of a virtual overlay network, so that any node could talk to any other node even if they were blocked by a NAT device, etc. This allowed any person to publish a Paper Airplane website, without having to provision a web server. Further, P2P Sockets implemented a P2P Domain Name System, making it easy for users to create entirely new endings, such as www.nike.laborpolicy. P2P Sockets worked hand in hand with the Paper Airplane Plugin to enable the actual network functionality. All of this worked and was running, in about 2004. View a screencast of all this working. Unfortunately both the JXTA network and the Mozilla plugin API have changed significantly since 2004, so neither component now works with present software.
   

3. The flip side of the previous question is: How far can we push the present generation of web browsers from the inside through clever tricks?

   Even as we are trying to push browsers from the outside in new directions, we can also see how far browsers can be expanded from the inside. This research question has been my main focus the last two years, with the work done on the Really Simple History library for bookmark and back button support for Ajax applications; AMASS and Dojo Storage for large client-side storage; Dojo Offline for offline web apps; and HyperScope and Purple Include for expanded hypertext abilities on the contemporary web. To be honest, at this point I'm not quite sure how to personally push it forward more, except for the advanced hypertext stuff (see below).
   

4. How can we create communication technologies that provide ever greater levels of interpersonal connection, collapsing the geographical distance between loved ones and family? Related subgoal: how can we create mediated communication technologies that feel "unmediated"?

   I have friends and family all across the country since I've moved around alot. However, I feel like I can't really maintain a strong, real ongoing friendship with them when they live far away -- it's more like our connection is on 'life support', and we touch base through the phone every now and then. The real connection happens when they visit. Why can't we create communication technologies that collapse geographical space and make it possible to really feel like they are next door? The ones we have so far don't fill this need yet for me; connecting through email, for example, is like talking through a straw. I haven't made any progress on this goal or had any big realizations, but it is always in the back of my mind and one of the research questions that arises out of a real need of mine.

5. How can we create email systems that almost completely eliminate email from your life, allowing you to focus on the emails that you need to with as little time invested as possible? A related subgoal is: how can we create information technologies of focus and minimal distraction versus the technologies of information and collaboration overload that we have today?

   The last big breakthrough we had with email was GMail. However, all of our email systems assume that we love email, enjoy spending time checking email, and that more is better. With the arrival of all of our networking technologies, I believe that we don't just have Information Overload but Collaboration Overload as well now, where everyone and the world is trying to contact us in some way, whether it is personal, professional, or about Viagra ads. Not all emails are equal, and current systems force us to treat them like they are. Some email systems allow you to create custom filters. Last week, however, as I was spending hours creating and tending my Gmail filters I realized that the computer should be doing this for me. Custom filters aren't enough -- they are just another distraction and thing that has to be attended to.

   I think this area is ripe for some big innovations. This is a new research goal for me, borne out of my own frustration currently with email, but I have a few ideas to throw at it. Here are some ideas around GMail in particular.

   The big idea is that more is not better -- GMail should not be dumping everything into the inbox. The ideal situation is an empty inbox, where you see nothing. GMail should strive to move as much out of the inbox as possible, into side containers that you can attend to when your time and energy permits, rather than as a constant tap tap tap on your shoulder -- not all taps are equal. Gmail should:
   

   • be autotagging and figuring out my email lists, tagging them, and moving them out of my attention view. For example, I am subscribed to a Coworking email list, a Hyperscope email list, and a Dojo email list -- these should be autotagged as coworking, hyperscope, and dojo and removed from my inbox.
   
   
   • figure out what are social networking emails (MySpace, LinkedIn, Tagged, etc) and get these the heck out of my main view, autotagging them as "social_network". These are the worst offenders -- I'll get to them (twice a month) when I care and have time.
   
   
   • be figuring out who my friends are and tagging these as personal, also removing them from the main inbox and tagging them as 'personal'. It can either figure these out over time, or perhaps just import my MySpace social network in and figure it out from there -- I spent all this time building up a personal social network on MySpace, just import it into my email contacts so it can autotag these folks as 'personal' and get them out of the inbox so they don't get lost in the general email clutter, which is what happens for me now.
     
   
   
   • be figuring out my "professional" or business contacts and autotagging these as well, perhaps through my LinkedIn social network. Get these out of the inbox as well. If these social networks overlap (MySpace and LinkedIn), then it will just tag them as both.
   
   
   • have an automatic personal assistant that goes to bat for me to deny others from grabbing my attention in a way that doesn't work for me. I should be able to register some keywords, such as Really Simple History for my open source library, and bounce back a reply (an FAQ) that satisfies the most common questions. Perhaps I run several different businesses and could bounce back different FAQs for each.
     
   
   
   • autodiscover those crappy once a month "newletters" that everyone seems to send out (my Yoga studio, dance place, etc.) and again tag it as newsletters and get it out of the inbox

   In general, I've been thinking in terms of Attention Vampires -- things that steal my attention, and take away my creative energy versus making me feel better, more powerful, and more productive, a kind of Law of Conservation of Attention. Is there software that promotes more of the former rather than of the latter?

6. How can we create search systems that make answering our questions even more intuitive and integrated then they are now?

   Google is a fantastic invention. Like any successful invention, ever since coming into contact with search that actually works I want even more of it :) This research question ranges from being able to just talk into my phone and ask arbitrary questions (why isn't 411 truly about any kind of information you need?), to having search engines be able to "parse out" what they are dealing with a bit more, such as knowing what are 'How To' articles, what are 'Recipes', what are 'Articles', etc. so that when I do a Google search saying "how to change car oil" Google can respond with a bit more formatting of a list of howto articles at the top of the search results.

7. Can we create hypertext technologies that transform how we link and talk about information and documents? Further, how can we port these hypertext technologies to the contemporary web so they are actually adopted?

   I love that the World Wide Web massively simplified hypertext to get it actually deployed and adopted by much of the world -- I think that was a brilliant approach by Tim Berners Lee. However, it is now 2007, and computers are now smarter (and people are too when it comes to the conventions of hypertext) -- it's time to evolve our 8-bit hypertext and introduce new capabilities. Admittedly, most of the academic hypertext research has a morass of usability issues, however there are some great ideas in there (most from Douglas Engelbart and Ted Nelson) that could be cleaned up, ported to the contemporary web using clever Ajax tricks and usability studies, and integrated into our writing and publishing tools. This is the focus of the HyperScope and Purple Include work I have been a member of. There is just so much more cool and interesting ground we can cover in terms of (usable) hypertext than we have today.
   

8. Handling events still sucks -- how can we make scheduling and discovering events that are important and relevant to you and your groups suck much less than it does now?

   I've long wanted a bumper sticker that says "Evite Must Die." I can't believe that it is 2007 and we are still using Evite -- using Evite gives me the same trashy feeling I get when I watch E Entertainment. Event systems don't communicate with each other, they are a pain in the butt to sync, their UIs suck, and more. Why do I have to manually enter social events that I am interested in into my calendar? Why can't I just say that I am interested in yoga and meditation in my zip code, for example, and I could always just look at my calendar and find good and interesting yoga classes automatically there and recommended to me? I made this great giant poster during my sabbatical of all the free and interesting events happening in the city, yoga classes, etc. so that I could always have something (cheap/interesting) to do. After making this, I was like, shouldn't the computer be doing this for me?

9. How can we be more effective at creating ideas?

   I love creating new ideas and thinking outside the box -- I'm always looking for new ways to facilitate brainstorming. I once heard a great quote about relationships that I try to apply to my business relationships -- "Two people should stay together if together they are better people than they would be individually." 98% of the brainstorming I have encountered in my life feels like it makes the group less effective than they would be individually. Perhaps group brainstorming is the problem, but I think it's because the process by which we do group brainstorming is broken. I'm constantly looking for better ways to approach this; I talk about three ways here: The Cranky Inventor, Sacred Cows, and I Against I.

10. Can we create technologies that are as fundamental and important as writing has been?

    I don't claim I'll ever be able to do this -- this research question is more of a Zen Koan than anything else. In the long term perspective writing is relatively new, and is perhaps one of human kinds greatest inventions so far. What I love about this question is it generates so many more great questions:

    • Why is writing fundamental?
    • What does it mean for a technology to be important and fundamental?
    • How can the computer be harnessed in such a way that it becomes as important to human evolution as language and writing have been?
    • Writing requires training -- how else can we use training to augment human capability in relation to computers?

Labels: aboutme, brainstorming, invention, personal research agenda, research

A few months ago Yahoo asked me to give the keynote at their internal Front-End Engineering Summit conference. This was a collection of all their front-end engineers from different sites, such as Flickr, the new Ajaxy Yahoo Mail, and more.

Yahoo asked that I come in and talk about the next 5 years of the web. Prediction is hard, especially 5 years in the computer industry, so instead I talked about inventing the future rather than predicting it.

This was a really deep speech for me; I struggled with it for 2 weeks, unable to work on anything else; it was a difficult creative process. Here's a 'markety' synopsis of the speech:

Brad Neuberg was asked to give the keynote at the Yahoo 2007 Internal Engineering Summit on the next five years of the web. Instead of talking about the future of the web, he came in and gave a heartfelt speech on invention, inventing the future, and knowing what you stand for. This speech challenges Yahoo to change, goes into the core of invention, and provides techniques for shaking the world up with powerful, new ideas. Become a better software engineer by being an inventive leader.

Much thanks to Eric Miraglia of the YUI team for setting the keynote up, helping with feedback, manning the cameras, and helping to mix the digital video. It was a great experience getting to know him; he's a super classy guy.

View the presentation:

Labels: future studies, invention, keynote, presentation, video, yahoo