Thursday, December 16, 2004
Why BitTorrent is Important and SwarmStream is Doomed
Overview:
- How Useful is Swarming and the Onion Networks SwarmStreaming Product?
- What are the Potential Drawbacks of the SwarmStream Product?
- Why BitTorrent is Important and SwarmStream is Doomed
How Useful is Swarming and the Onion Networks SwarmStreaming Product?
There are two seperate things to consider: the importance of the swarming concept and the SwarmStream product itself.Swarming is an extremely powerful idea that is here to stay. In the short term it is redefining the speed and economy of sending large files around; it extends the benefits of an Akamai-like network to everyone, not just the Fortune 500. It is a robust solution for both flash crowds and serving content to many people over long periods of time.
In terms of the SwarmStream product, Justin Chapweske is to be credited with creating the swarming concept and introducing technologies and algorithms to achieve it. The SwarmStream product has an excellent Java API that makes it easy to integrate into existing applications as well as for use in enterprise settings. It also has extremely good stream control, offering progressive rendering of content as it is swarmed as well as pause and resume control. I believe it will have uptake by two primary crowds: forward-thinking enterprise developers who will use it as an efficient way to move large amounts of data either behind the firewall or over global WANs, as well as well-heeled large application companies like Real Networks that will integrate swarming into their client and server products using SwarmStream.
What are the Potential Drawbacks of the SwarmStream Product?
SwarmStream has two major drawbacks that I believe will spell its doom. The first major issue is its price and proprietary nature. The SwarmStream Development Kit is twenty-five thousand dollars to use in applications. While this is pennies for a large enterprise or application company, small startups, open source developers, and tinkerers can simply not use the SwarmStream API. This would not matter if BitTorrent did not exist as BitTorrent is the achilles heel of SwarmStream.Why BitTorrent is Important and SwarmStream is Doomed
I predict that in several years BitTorrent will be a core Internet protocol. It is following exactly the same evolution as the RSS blogging protocols, which are themselves being integrated into the Internet's infrastructure and evolving beyond their humble roots. BitTorrent is experiencing the same kind of grassroots excitement and evolution that occured around the blogging standards, which produced RSS, OPML, XML-RPC, the blogging XML-RPC APIs, and more. It is the thin end of the wedge that will introduce P2P content distribution into mainstream Internet practices. Bram Cohen created an excellent protocol without biting off too much. Now, other like minded individuals are evolving this "good enough" protocol and extending its functionality and solving its issues incrementally. Some are attempting to distribute trackers, for example, using a variety of approaches such as distributed hash tables. Others are integrating BitTorrent with other existing protocols like RSS, audio streaming, and digital video. While the machinery of BitTorrent is only currently viable and realistic for large files above 3 megabytes, I do believe it will be slowly optimized to make it possible to swarm and retrieve content that is in the one hundred kilobytes to 1 meg file range, opening its use in distributing popular websites. Also in the long term I believe we will see BitTorrent transparently integrated with Mozilla's download manager, making BitTorrent connections through the browser as transparent as FTP ones are now, as well as innovations in making it extremely easy for anyone to host content that can be swarmed through BitTorrent. Even better, this evolved BitTorrent will be open, non-proprietary, and available in a variety of open source SDKs that SwarmStream will simply not be able to compete with. If SwarmStream were an order of magnitude better than BitTorrent than I might reconsider, but it is only incrementally better in some areas.Update: Matthew Gertner posts a well-written, balanced response to this blog entry on his blog.
Comments:
Links to this post:
<< Home
Define "doomed"...
All they have to do is sell 100 copies and they make $2,500,000 ...
Thats hardly doomed.
What I would do if I were them is to use a GPL license like MySQL and then sell into this market. Its essentially the same revenue model. MySQL seems to be doing JUST fine.
Proprietary software is hard enough as it is... Right now if you're a proprietary company its the worst of both models. I have to PAY and I don't get to SEE the source.
No thanks. I'd rather give my money to companies like MySQL that respect my rights to see the source and fix my own bugs.
I'd consider licensing something like Swarmstream but only if I could submit patches and only if it were OSS.
And for this record comes not from some random OSS developer. If I were to recommend a technology like this Onion Networks would get cut a check... ;)...
All they have to do is sell 100 copies and they make $2,500,000 ...
Thats hardly doomed.
What I would do if I were them is to use a GPL license like MySQL and then sell into this market. Its essentially the same revenue model. MySQL seems to be doing JUST fine.
Proprietary software is hard enough as it is... Right now if you're a proprietary company its the worst of both models. I have to PAY and I don't get to SEE the source.
No thanks. I'd rather give my money to companies like MySQL that respect my rights to see the source and fix my own bugs.
I'd consider licensing something like Swarmstream but only if I could submit patches and only if it were OSS.
And for this record comes not from some random OSS developer. If I were to recommend a technology like this Onion Networks would get cut a check... ;)...
Hey Brad, Justin here. Thats a heck of a gripping title :) Almost as good's as Xeni's headline "Hollywood Wants Bittorrent Dead".
Except for the enlightened technologists and bloggers such as yourself, Bittorrent is firmly perceived as a tool for piracy. In the world of piracy, I hope and agree that SwarmStream is doomed.
Is $25k too expensive for an individual developer working on an RSS aggregator or podcasting tool? Of course, but $100 is probably too expensive as well. Is it worth $25k to be trained in on a product, massively increase the reliability and performance of an enterprise application using commodity hardware and saving tons of money on infrastructure to boot? You bet.
The reason that individual developers get so passionate about this issue that they are incredibly hungry for the technology.
Witness how much blogwidth is spent trying to solve the RSS bandwidth problem or how to scale podcasting, or even to reliabily cache content across a wireless network.
Swarmstreaming is better able to solve these problems than Bittorrent will ever be able to. They key difference is that swarmstreaming is a web native technology that lives and breaths HTTP, not some proprietary protocol and no silly .torrent files. If you want to add swarmstreaming to your application, you simply need to proxy your traffic through and EVERYTHING is transparently accelerated. Thats it, no extra work required on the part of developers or users.
I sincerely want to help these guys out, because swarming HTTP *is* going to be the foundation of the next generation web.
It has always been my intent to allow everyone access to this technology, so keep your eyes peeled to my blog and we'll be making an announcement in the weeks ahead that will answer all of your concerns.
-Justin
Except for the enlightened technologists and bloggers such as yourself, Bittorrent is firmly perceived as a tool for piracy. In the world of piracy, I hope and agree that SwarmStream is doomed.
Is $25k too expensive for an individual developer working on an RSS aggregator or podcasting tool? Of course, but $100 is probably too expensive as well. Is it worth $25k to be trained in on a product, massively increase the reliability and performance of an enterprise application using commodity hardware and saving tons of money on infrastructure to boot? You bet.
The reason that individual developers get so passionate about this issue that they are incredibly hungry for the technology.
Witness how much blogwidth is spent trying to solve the RSS bandwidth problem or how to scale podcasting, or even to reliabily cache content across a wireless network.
Swarmstreaming is better able to solve these problems than Bittorrent will ever be able to. They key difference is that swarmstreaming is a web native technology that lives and breaths HTTP, not some proprietary protocol and no silly .torrent files. If you want to add swarmstreaming to your application, you simply need to proxy your traffic through and EVERYTHING is transparently accelerated. Thats it, no extra work required on the part of developers or users.
I sincerely want to help these guys out, because swarming HTTP *is* going to be the foundation of the next generation web.
It has always been my intent to allow everyone access to this technology, so keep your eyes peeled to my blog and we'll be making an announcement in the weeks ahead that will answer all of your concerns.
-Justin
Hey Brad, Justin here. Thats a heck of a gripping title :) Almost as good's as Xeni's headline "Hollywood Wants Bittorrent Dead".
Except for the enlightened technologists and bloggers such as yourself, Bittorrent is firmly perceived as a tool for piracy. In the world of piracy, I hope and agree that SwarmStream is doomed.
Is $25k too expensive for an individual developer working on an RSS aggregator or podcasting tool? Of course, but $100 is probably too expensive as well. Is it worth $25k to be trained in on a product, massively increase the reliability and performance of an enterprise application using commodity hardware and saving tons of money on infrastructure to boot? You bet.
The reason that individual developers get so passionate about this issue that they are incredibly hungry for the technology.
Witness how much blogwidth is spent trying to solve the RSS bandwidth problem or how to scale podcasting, or even to reliabily cache content across a wireless network.
Swarmstreaming is better able to solve these problems than Bittorrent will ever be able to. They key difference is that swarmstreaming is a web native technology that lives and breaths HTTP, not some proprietary protocol and no silly .torrent files. If you want to add swarmstreaming to your application, you simply need to proxy your traffic through and EVERYTHING is transparently accelerated. Thats it, no extra work required on the part of developers or users.
I sincerely want to help these guys out, because swarming HTTP *is* going to be the foundation of the next generation web.
It has always been my intent to allow everyone access to this technology, so keep your eyes peeled to my blog and we'll be making an announcement in the weeks ahead that will answer all of your concerns.
-Justin
Except for the enlightened technologists and bloggers such as yourself, Bittorrent is firmly perceived as a tool for piracy. In the world of piracy, I hope and agree that SwarmStream is doomed.
Is $25k too expensive for an individual developer working on an RSS aggregator or podcasting tool? Of course, but $100 is probably too expensive as well. Is it worth $25k to be trained in on a product, massively increase the reliability and performance of an enterprise application using commodity hardware and saving tons of money on infrastructure to boot? You bet.
The reason that individual developers get so passionate about this issue that they are incredibly hungry for the technology.
Witness how much blogwidth is spent trying to solve the RSS bandwidth problem or how to scale podcasting, or even to reliabily cache content across a wireless network.
Swarmstreaming is better able to solve these problems than Bittorrent will ever be able to. They key difference is that swarmstreaming is a web native technology that lives and breaths HTTP, not some proprietary protocol and no silly .torrent files. If you want to add swarmstreaming to your application, you simply need to proxy your traffic through and EVERYTHING is transparently accelerated. Thats it, no extra work required on the part of developers or users.
I sincerely want to help these guys out, because swarming HTTP *is* going to be the foundation of the next generation web.
It has always been my intent to allow everyone access to this technology, so keep your eyes peeled to my blog and we'll be making an announcement in the weeks ahead that will answer all of your concerns.
-Justin
Hi Justin; good to hear from you. I actually really respect your work and the engineering in the SwarmStream product. I have read over the documentation and watched the presentation you posted on JavaLobby and have been following your work for several years. However, I wanted to give an honest appraisal of what I see as the strengths and weaknesses of SwarmStream, and unfortunately my end opinion is that it is not competitive enough with BitTorrent in the long term.
I agree that BitTorrent will not be used in enterprises, right now. SwarmStream is an excellent product for an enterprise, especially with the bundled support. However, I do believe that BitTorrent will continue to improve to the point that enterprises, probably beginning with academians who need to move large scientific files around, will begin to use it both within the LAN and over their WANs. One issue affecting both SwarmStream and BitTorrent is that the concept of swarming is still very new to enterprises, and only adventerous enterprises would possibly implement it using either one. This means that an enterprise that is adventerous enough to use an unconventional solution like swarming might be adventerous enough to use BitTorrent, especially if an engineer introduces it in a "stealth" way to get a problem solved quickly, just as ssh, rsync, etc. are gaining traction in the enterprise from the rank and file sys admins.
You make a good point that SwarmStreaming works over the HTTP protocol versus rolling its own like BitTorrent; I personnally like protocols that run over HTTP, but in the long run I don't think it matters. BitTorrent has generated quite a large amount of traction with its protocol, and it is simple enough that others have been able to implement it in their own programs quickly. It is the Worse is Better argument (http://www.jwz.org/doc/worse-is-better.html), as well as BitTorrent being a good example of an evolvable system (http://www.shirky.com/writings/evolve.html).
I believe you deserve to make a profit as the originator of both the swarming idea as well as realistic ways to achieve swarming. However, I believe you probably need to change your approach. The BitTorrent vs. SwarmStream argument is looking to me like the RSS vs. the Vignette-backed ICE standard (whats that? exactly). Vignette created an enterprise syndication technology called ICE that went up against RSS, and over the long term it failed even though they signed up many enterprise customers. Selling to the enterprise reminds me of the hill tribes in Thailand; companies that sell into the enterprise and don't embrace bottom up technologies are forced to go to higher and higher niches to support themselves, just as the hill tribes were forced onto less and less arable land up mountains as those from the bottom encroached on their land, until they had no more distance to go up.
Instead of fighting BitTorrent, what if you got out in front of it and became a technology leader in the BitTorrent community? Smooth out its edges and offer an enterprise BitTorrent product. Then, as BitTorrent gains horizontal traction on the Internet and is embedded and used in a variety of applications you can market to vertical enterprises with your product, secure in the knowledge that the bottom up technology won't make your technological base disappear. I think the first company that realizes that BitTorrent is _much_ more than pirating will gain significant first mover advantage; the web, RSS, and more were also damned with being too simple, not good enough, used for piracy or only by amateurs, etc.
I agree that BitTorrent will not be used in enterprises, right now. SwarmStream is an excellent product for an enterprise, especially with the bundled support. However, I do believe that BitTorrent will continue to improve to the point that enterprises, probably beginning with academians who need to move large scientific files around, will begin to use it both within the LAN and over their WANs. One issue affecting both SwarmStream and BitTorrent is that the concept of swarming is still very new to enterprises, and only adventerous enterprises would possibly implement it using either one. This means that an enterprise that is adventerous enough to use an unconventional solution like swarming might be adventerous enough to use BitTorrent, especially if an engineer introduces it in a "stealth" way to get a problem solved quickly, just as ssh, rsync, etc. are gaining traction in the enterprise from the rank and file sys admins.
You make a good point that SwarmStreaming works over the HTTP protocol versus rolling its own like BitTorrent; I personnally like protocols that run over HTTP, but in the long run I don't think it matters. BitTorrent has generated quite a large amount of traction with its protocol, and it is simple enough that others have been able to implement it in their own programs quickly. It is the Worse is Better argument (http://www.jwz.org/doc/worse-is-better.html), as well as BitTorrent being a good example of an evolvable system (http://www.shirky.com/writings/evolve.html).
I believe you deserve to make a profit as the originator of both the swarming idea as well as realistic ways to achieve swarming. However, I believe you probably need to change your approach. The BitTorrent vs. SwarmStream argument is looking to me like the RSS vs. the Vignette-backed ICE standard (whats that? exactly). Vignette created an enterprise syndication technology called ICE that went up against RSS, and over the long term it failed even though they signed up many enterprise customers. Selling to the enterprise reminds me of the hill tribes in Thailand; companies that sell into the enterprise and don't embrace bottom up technologies are forced to go to higher and higher niches to support themselves, just as the hill tribes were forced onto less and less arable land up mountains as those from the bottom encroached on their land, until they had no more distance to go up.
Instead of fighting BitTorrent, what if you got out in front of it and became a technology leader in the BitTorrent community? Smooth out its edges and offer an enterprise BitTorrent product. Then, as BitTorrent gains horizontal traction on the Internet and is embedded and used in a variety of applications you can market to vertical enterprises with your product, secure in the knowledge that the bottom up technology won't make your technological base disappear. I think the first company that realizes that BitTorrent is _much_ more than pirating will gain significant first mover advantage; the web, RSS, and more were also damned with being too simple, not good enough, used for piracy or only by amateurs, etc.
Except for the enlightened technologists and bloggers such as yourself, Bittorrent is firmly perceived as a tool for piracy.except for people who have purchased world of warcraft (240,000 on the first day), and gotten the first big patch to the system delivered via a fairly seamless bittorrent download.
Is Justin really the originator of swarming? eDonkey was out and in use before any Swarmcast code was released. Somehow I doubt that even Jeb McCaleb invented it. The idea was probably floating around before then.
"Harness the power of 2000 electronic mules!"
Post a Comment
"Harness the power of 2000 electronic mules!"
Links to this post:
<< Home
Subscribe to Posts [Atom]
