Request for P2P knowledge in green text

Peer-to-peer architecture research

I am working on a private sharing app. I need recommendations for a peer-to-peer architecture. Ideally, I don’t have to build the peer-to-peer infrastructure. I really just want to build a great app on top of one.

Use case

  • A person’s data is stored primarily on their phone.
    • A person’s data can be backed up on their computer or perhaps cloud storage of their choice.
  • A person will not have more than 250 friend peer connections.
    • It’s ok for people to invite each other to connect using an out-of-bands method (email, text message, in person, et al). An ugly identifier, like a public key string, is ok.
  • A person’s data is shared selectively with another person (a friend). For example, the person might have some posts that are meant for all friends, some friends, or just themself.
    • There is no public sharing. Data does not have a public URL because it’s only ever sent from its creator to the creator’s friends.
  • A person’s data can be selectively requested by their friends. For example, a friend should not have to store everything a person has sent to them forever. The friend can cache the data, purge the cache, and re-request the data in the future from the person.
  • A person can delete their data and request their friends’ devices also delete the data.
  • Ideally, a person’s data can only be obtained from the person (or a system the person has authorized to act on their behalf), not from friends the person has sent their data to. No relaying a person’s data by a mutual friend or friend-of-a-friend.
    • This is not a hard requirement. Perhaps peers could help with the offline problem?
    • It’s ok for data to not be available immediately, but it’s not ideal. If a person’s friend is offline, perhaps requests are queued until the person and friend are both online. Perhaps both a person’s phone and their backup computer could respond to requests from friends.
  • The role of a server not operated by the person should be minimal.
    • It could provide an assist for connecting people.
    • It could provide temporary caching or relaying of data if done in a way the server operator could not inspect the contents of the data
    • Ideally whatever the server must do is able to be done by other independently run servers as well.
  • The app is a progressive web app. The web app only contains the business logic. User data is never stored by the server hosting the web app. Any webserver could host the web app and people could connect their locally stored data to it. Preferably, the web app does not need to be wrapped in a native OS app wrapper to function.

Architecture thoughts

  • No immutable data structures. When a person deletes their data, it should be as if the person never stored the data on their device.
    • A person’s control of their data is more important than censorship resilience. Some protocols optimize for replication durability. I believe people are more likely to regret sharing something more than they are to be subjects of state-sponsored censorship.
      • This is one reason why I don’t think IPFS is helpful for this use case. If you think it could be, please educate me.
  • I don’t think blockchains or cryptocurrencies are helpful to this use case. If you think they could be, please educate me.
  • I have a strong bias against GraphQL, but will tolerate it if necessary.
  • I wish Solid were better and usable by normal people. I can imagine people using my app with their data being stored in their pod and their friends having per-user access permissions to access certain files. If a person had an always-online device within their control like this, that would simplify things.
    • I can imagine something like a server that runs as a proxy on top of people’s Google Drive, Dropbox, Fastmail files, Proton cloud storage as an interim solution. It would require high trust of the proxy server provider.
    • Asking people to sign up for a new cloud storage provider is a high barrier to entry and not one that I think is acceptable for my app.
  • I can imagine something like Signal’s server architecture working as an offline intermediary. When both parties are online, they communicate directly. When one party is offline, requests for the other party’s data are sent to an intermediary. The Signal protocol allows for anonymity of the sender, receiver, and message contents. When the offline party comes back online, they check for new requests and respond with the data. It’d be like HTTP request and responses being facilitated by an intermediary for when either the requester or responder are offline.
    • I think the Matrix protocol could also be implemented for use in this way. The one aspect I am uncertain about is clearing the messages from the server once retrieved. Signal’s server only ever temporarily stores messages.
    • A goal is to avoid a centralized, controlling server. Multiple servers could be operated for this purpose. I assume this would mean the app would have to check a list of servers for requests/responses and the user could manage which ones they wanted to use.
  • I can imagine each person being their own Tor Onion Service. This would only allow data exchange when both parties are online. I am not sure if an Onion Service can be run from the browser, but I do know running the Tor Snowflake proxy works in the browser.

Things I’ve heard about but not fully investigated

If you have any knowledge or experience with these, please let me know how they may or may not be useful for the use case described above.

Have ideas?

Please send me your thoughts, knowledge, things you’ve heard about, or anything else you think might be helpful.

You can respond to this post on the social web or email me.

I will update this post as I learn!


15 responses to “Peer-to-peer architecture research”

  1. ^ Any ideas @NGIZero or @dweb? Boosts appreciated.

    1. @Jeremiah @dweb Someone here may have an answer. I'll ask. (May take a while).
      Edit: 'the' > 'an'

  2. I'm mostly just responding because I also want to know, but wanted to ask if you've had a look at

    1. @gendor Hello! Heard via NLnet about your search for a p2p/f2f system. I’m one of the co-authors of aforementioned Willow, which is very much designed with the use-case you describe (and, importantly, now has implementations). Happy to talk if that sounds interesting.

  3. I'm currently writing a p2p app for #KaiOS,!/start you can find the code here: maybe it will help you

  4. Hey restoration, please come over to to find p2p-minded individuals.

    The community explicitly aims at bridging these gaps.

    There's also a lot of discussion ongoing after the recent #p4p camp at @offline a few weeks ago.

    There was traction around @pzp or Braid Antimatter, but also seems a good starting point (for research).

    Maybe @zelf or @adz know more.

    1. @lutindiscret Given the outlined constraints I think @VeilidNetwork would be a great foundation to build on. Combined with a local database and a protocol to send messages between specific peers. Do you care about people being able to load a persons data when the person is offline?

  5. Your use case seems eerily similar to my own. I'm also frustrated at the amount of infra I've had to build to make it work, but it's starting to come together and you may be interested it.

    I've been working on a portable identity system called Burger and a peer-to-peer network for the web called the SmolWeb protocol. The next step is a kind of personal data server that can be used on the network. Everything is end-to-end encrypted. And once all of that is working, then I can finally build the damn app.

    Your app may be able to use this infra too. Let me know if that sounds interesting.


    Maybe: Hyphanet, GNUnet, SSB

  7. May I suggest Veilid? It is a P2P platform which focuses on privacy using end to end encryption. The flagship program is VeilidChat.


    I have a bad feeling about a web app, hosted on some server, using data stored locally. How can I know, that tomorrow the new version of the web app won’t send all my data to the server?

    1. The same can be said of any app. It’s good to be skeptical about security claims. I aspire to earn people’s trust by having the code be open source with verifiable builds. It also will be self-hostable and probably will work just by opening the HTML from the filesystem. (I will need to check which PWA features work when loaded via file://.)

  9. Hello Jeremiah,

    I am Niko and I am working on NextGraph, a P2P ecosystem that matches your requirements. It is a project funded by NLnet/NGI, that takes its inspiration from P2PCollab/ (mentioned by Jon in the comments of your article).

    You can find more details about it on our website

    Documentation is going to be published in few weeks from now (when the whole framework will be usable, which it isn't right now)

  10. What I want when evaluating a peer-to-peer framework:
    • explanation of how the protocol works
    • design tradeoff decisions
    • detailed use case examples
    • comparison to other technologies

    What I get from peer-to-peer frameworks:
    • indictments of the Web architecture
    • misunderstanding of Web 2.0
    • manifestos for solving all of capitalism’s problems by eliminating servers somehow
    • aspirations to be everything to everyone all at once
    • 17 links to similar sounding components they depend on