Peer-to-peer: comment ça marche ?
Napster, Gnutella, Kazaa, BitTorrent, we have been sharing files with tools on the net for a while... Did you know that they rely on peer-to-peer computing technology?
In a network organized in a peer-to-peer infrastructure, users share resources through direct exchange between computers, which are called “nodes”. The data is distributed among the nodes instead of being sent to servers for processing. Unlike in client-server computing technology each node plays a symmetric and autonomous role to provide to the end user the expected solution.
Peer-to-peer systems have a number of characteristics:
- Symmetric, distributed & decentralized: all the nodes play a similar role, acting as both client and server. They fetch, distribute and process content.
- Dynamic participants: peer-to-peer systems must be resilient to nodes joining and leaving, whereas a centralized system expects its servers to remain up at all times.
- Resource localization: one of the key challenges in peer-to-peer filesystems is to find the peer hosting the requested data. A well-known technique is to use a Distributed Hash Table (DHT), such as "Kademlia", which is itself distributed among the peers. These algorithms are very efficient and scalable even with a large number of nodes and resources.
- Rebalancing and replication: as nodes become overloaded, or leave the network the peer- to-peer system must ensure that services remain accessible, available, performant and data remains persistent.
- Scalability and security: in a peer-to-peer network there could be millions of nodes. Skype at its peak had over 300M users. Such networks must have irreproachable security tools which don't deteriorate as the size of the network grows.
Hive's peer-to-peer storage relies on these core principles. It is based on the open source IPFS protocol for the core filesystem layer, on top of which we have built additional services and features to provide:
- End-to-end encryption: no private data leaves the end users' device in clear form. The encryption model enables the sharing of data across multiple participants without replicating the content, nor sharing keys.
- Proof of storage: as participants in the Hive network store other's data, they are incentivized to do so as long as they continue providing proof of storage.
- Location awareness: Hive's peer-to-peer placement algorithms take into account the user's privacy requirements and preferred locations for both data storage and processing.
- Error correction: when nodes suddenly go offline, the data they hold is no longer available. When such an event occurs Hive will recreate the lost data and distribute it to other nodes to ensure durability of the stored files.