Sunday, September 14, 2003
Things I need to read:
P2P MetaData Search Layers
The abstract:
"Distributed Hashtables (DHTs) provide a scalable method of
associating file-hashes with a particular location in a distributed network
environment. Modifying DHTs directly to support meta-data is difficult, and
meta-data search systems such as flooding tend to scale poorly. However, a
number of more scalable distributed meta-data search systems have recently
been developed that could be deployed in tandem with DHTs, and several are
discussed here along with some novel simulation results that concern the
scalability and resource limitations of a meta-data search layer that employs
semantic routing. Semantic routing is a method of pruning a flooding search
such that queries are preferentially forwarded to nodes that can answer those
queries."
P2P MetaData Search Layers
The abstract:
"Distributed Hashtables (DHTs) provide a scalable method of
associating file-hashes with a particular location in a distributed network
environment. Modifying DHTs directly to support meta-data is difficult, and
meta-data search systems such as flooding tend to scale poorly. However, a
number of more scalable distributed meta-data search systems have recently
been developed that could be deployed in tandem with DHTs, and several are
discussed here along with some novel simulation results that concern the
scalability and resource limitations of a meta-data search layer that employs
semantic routing. Semantic routing is a method of pruning a flooding search
such that queries are preferentially forwarded to nodes that can answer those
queries."
Naanou, a P2P file-sharing application built on top of a distributed hashtable, has been GPLed. It's written in C#.
The MIT OpenCourseWare site has some good lecture notes on Distributed Hashtables from their Distributed Computer Systems class. Here's the first lecture, and a discussion on Frangipani, an example distributed filesystem using DHTs. Here's the second lecture, and a discussion on Scribe, a layer implementing scalable multicast using DHTs.
Went to the P2P Dim Sum lunch today; it came out to about $30 bucks a person! Ouch!
Chatting with some folks there, we came up with some crazy conspiracy theories around spam. What if spam is actually sent by intelligence agencies, but a message is hidden in either the individual message or over several messages using steganography? This reminds me of those strange beasts of the cold war called Number Stations, which were channels on the shortwave part of the specturm where mechanical voices intoned a seemingly random series of numbers. No one still understands what these are, but they were probably spies in other countries using shortwave to transmit encoded information.
Chatting with some folks there, we came up with some crazy conspiracy theories around spam. What if spam is actually sent by intelligence agencies, but a message is hidden in either the individual message or over several messages using steganography? This reminds me of those strange beasts of the cold war called Number Stations, which were channels on the shortwave part of the specturm where mechanical voices intoned a seemingly random series of numbers. No one still understands what these are, but they were probably spies in other countries using shortwave to transmit encoded information.
In my continuing quest to best Zooko's Triangle, here is some info on Threshold Cryptography, another beast that I think may hold some clues to solving the Triangle (yesterday I talked about Identity Based Encryption, another piece of the puzzle). Here's some good info about this subject from a recent paper I've been reading. I've added bold to a section that surprised me on some recent developments in this field:
"Threshold cryptography addresses [the] secret
key storage problem by breaking a master key into numerous
secret shares and storing them on multiple machines.
For example, key K can be broken into k1, k2,
and k3, each of which is stored on a different machine.
All components of the broken key must be obtained
to reconstruct the entire key. Operations using the key
never allow it to be reconstructed in a single location.
Master key operations occur by submitting a request to
each machine holding a share. The requestor receives
all the computed responses and mathematically reconstructs
them into the correct message without revealing
the master key. For the key to be compromised, a person
would have to compromise each node containing
the shares of the key. With a large number of key components
distributed across the network, this becomes
extremely difficult.
Until recently, threshold cryptography suffered
from the requirement that the master key be generated
by a trusted party. That party would then separate
the master key and distribute the components.
This would render threshold cryptography useless in
a self-managed, distributed system. However, it has
been shown that a master key can be generated in a
distributed environment without constructing the key
at any single node [full details]. As long as less than half of the
nodes chosen to generate the key do not collude, the
key is not compromised during generation."
Basicly, we can create systems that depend on having a master private key for certain operations. Instead of needing to isolate this private key on a single trusted server, we can break it into pieces and distribute each of the pieces to a set of relatively untrusted peers. Then, Threshold Cryptography makes it possible to recombine these pieces in such a way that no one has full knowledge of the entire private key while still being able to use it for certain operations. I'm excited about the Boneh paper because it seems to show a more efficient form of threshold cryptography that doesn't require a master source to generate the private key; instead, the peers co-generate it. This is the same Boneh by the way who is behind the recent breakthrough in Identity Based Encryption.
"Threshold cryptography addresses [the] secret
key storage problem by breaking a master key into numerous
secret shares and storing them on multiple machines.
For example, key K can be broken into k1, k2,
and k3, each of which is stored on a different machine.
All components of the broken key must be obtained
to reconstruct the entire key. Operations using the key
never allow it to be reconstructed in a single location.
Master key operations occur by submitting a request to
each machine holding a share. The requestor receives
all the computed responses and mathematically reconstructs
them into the correct message without revealing
the master key. For the key to be compromised, a person
would have to compromise each node containing
the shares of the key. With a large number of key components
distributed across the network, this becomes
extremely difficult.
Until recently, threshold cryptography suffered
from the requirement that the master key be generated
by a trusted party. That party would then separate
the master key and distribute the components.
This would render threshold cryptography useless in
a self-managed, distributed system. However, it has
been shown that a master key can be generated in a
distributed environment without constructing the key
at any single node [full details]. As long as less than half of the
nodes chosen to generate the key do not collude, the
key is not compromised during generation."
Basicly, we can create systems that depend on having a master private key for certain operations. Instead of needing to isolate this private key on a single trusted server, we can break it into pieces and distribute each of the pieces to a set of relatively untrusted peers. Then, Threshold Cryptography makes it possible to recombine these pieces in such a way that no one has full knowledge of the entire private key while still being able to use it for certain operations. I'm excited about the Boneh paper because it seems to show a more efficient form of threshold cryptography that doesn't require a master source to generate the private key; instead, the peers co-generate it. This is the same Boneh by the way who is behind the recent breakthrough in Identity Based Encryption.
Kevin Burton has fixed the Java XPI downloads. Go get 'em!
The java.mozdev.org site is a developer oriented site, but I think ordinary users are starting to go there and download the XPIs to get their Java 'on in FireBird and Mozilla. It's just so much easier to click an XPI download link and have the JVM silently install than having to deal with the standard JVM installation. Mozilla should do all its plugins this way; just click a silent download link for Flash and boom, you're ready to go.
Wanna learn more about what XPI and XPInstall are? Check out this tutorial on how to use the Mozilla XPInstall technology in your applications.
The java.mozdev.org site is a developer oriented site, but I think ordinary users are starting to go there and download the XPIs to get their Java 'on in FireBird and Mozilla. It's just so much easier to click an XPI download link and have the JVM silently install than having to deal with the standard JVM installation. Mozilla should do all its plugins this way; just click a silent download link for Flash and boom, you're ready to go.
Wanna learn more about what XPI and XPInstall are? Check out this tutorial on how to use the Mozilla XPInstall technology in your applications.
Man it was a fricking hot day here in San Francisco. Did all I could to avoid the heat. Had a going-away party for one of my best friends Michael. He is planning on jumping into his car tomorrow and driving down to central and south america, trying to hit every country. At his party someone asked me, "Don't you think Michael is crazy for trying to drive down into south america?" "Michael is the only sane person here," I responded.
Subscribe to Posts [Atom]
