178 lines
10 KiB
Markdown
178 lines
10 KiB
Markdown
# Overview
|
||
|
||
## Scope
|
||
This document aims to provide background on:
|
||
- Why/how The Grid project came to be
|
||
- Its intended goals and values
|
||
- Differences with the Matrix protocol
|
||
- The high-level methodology/processes we want to put in place to ensure this protocol and project are successful
|
||
- The set of first priorities that will be worked on
|
||
|
||
## Origin
|
||
As independent developers on the Matrix protocol, we decided to pursue an alternative approach and set of goals and
|
||
provide a fresh set of eyes and solutions to the remaining challenges the protocol is trying to solve.
|
||
|
||
We have spent to this day over 18 months learning, analyzing and reverse-engineering the Matrix protocol, implemented
|
||
nearly all components with various use cases, granting us extensive knowledge about its:
|
||
- Strengths and weaknesses
|
||
- Current limitations
|
||
- Security vulnerabilities, both used and hypothetical
|
||
|
||
While we will like to take an alternate road from the Matrix protocol and become a stand-alone protocol, we believe in
|
||
the fundamental concepts and the building blocks and will keep The Grid protocol in line with those for the time being.
|
||
|
||
## Goals
|
||
The protocol will aim for the following goals:
|
||
- Solve the problem originally identified by the Matrix protocol, fragmented communications, engaging it from a social
|
||
point of view.
|
||
- Simple client implementations, relying on the server for heavy lifting
|
||
- Federated communication between servers
|
||
- Decentralized data storage, spread across participating servers
|
||
- Designed for a full self-hostable stack to avoid centralization and/or monopoly
|
||
|
||
## Values
|
||
### Protocol
|
||
- Libre (Free) and Open protocol where its contributors and users are empowered
|
||
- Privacy of data and metadata
|
||
- Security of data and of the network
|
||
- Data exchange relying on a federated network, allowing everyone to spin-up a server and provide the service to a set of users
|
||
- Decentralized data storage, ensuring no single point of failure
|
||
|
||
### Management
|
||
- Transparency and openness around any talk, design session, workshop, meetings and any other related activity for the
|
||
protocol, without secretive or confidential sessions
|
||
- Clear, understandable specification documents are the top priority and the main deliverable. A protocol is nothing
|
||
without a specification.
|
||
- Taken actions and their consequences are used in evaluation, instead of intent.
|
||
- Decisions are backed with documented facts/feedback/issues
|
||
|
||
### Documentation
|
||
Documentation will be considered a generic term. Complexity, size, format and shape will be scoped to what is deemed
|
||
appropriate by the Governing Body for the various use cases.
|
||
|
||
As a protocol only exists in its specification and the only seeked resource, documentation will hold a special place for
|
||
this project and acts as the currency in spirit of the project, with various value depending on its complexity, size,
|
||
format, shape, use case and scope at hand.
|
||
|
||
It will also be used to enforce various good practices and force facts checking.
|
||
More details will be given in a separate, dedicated document to Documentation.
|
||
|
||
## The Grid vs Matrix
|
||
### Differences
|
||
#### Core values
|
||
Matrix’s ideals are Openness, Decentralization and Encryption [1](#links-and-sources) while The Grid’s ideals are
|
||
Freedom, Privacy and Security.
|
||
|
||
We see Matrix ideals as a subset of The Grid: While Matrix attempts to only solve the technical problem of fragmented
|
||
communications, The Grid wants to solve the problem at a social level, a higher level which includes the technical one.
|
||
|
||
While Matrix values and licenses align with the ideals of Open Source, The Grid would follow those of Libre (Free) software.
|
||
This would ensure that users and their rights are the focus and put at the center of each decision, instead of limiting
|
||
the discussion to technical matters only.
|
||
|
||
We believe in a social gain with The Grid protocol rather than technical or monetary gain, which should be reflected in
|
||
its own ecosystem.
|
||
|
||
#### Actors
|
||
Given the social purpose of the protocol, we believe involvement, participations and decision making should reflect the
|
||
social nature of the protocol.
|
||
|
||
Therefore, we want to put in place an ecosystem where anyone has a mean to express their opinion, provide feedback, or
|
||
contribute to the project itself. Making sure that no single entity is in a position of power alone. Instead, several
|
||
entities are in place to keep each other in check.
|
||
|
||
The creators of this protocol will initially stand alone in a position of power to bootstrap the ecosystem. Afterwards,
|
||
the newly-created non-profit will provide within its articles the requirements to join the various boards and decision
|
||
groups. See the Governing body section for more details.
|
||
|
||
#### Documentation
|
||
Matrix relies on reverse-engineering reference implementations since its creation 4 years ago, and has yet to produce a
|
||
version of its documentation that can be used to implement a full set of usable and interportable implementations.
|
||
A new spec proposal process was put in place but we believe it does not address the existing issues that led to the
|
||
current situation.
|
||
|
||
The Grid will therefore take a different approach and attempt to turn documentation into something positive and rewarding
|
||
for contributors and implementers, while putting processes and limits to avoid stagnation and status-quo.
|
||
It will also take the approach that the spec (the end-result) is to be implemented by at least two projects, independent
|
||
of each other, to ensure that the documentation and the specification are not academic exercises, but real-world and
|
||
problem-solving.
|
||
|
||
This practical approach where contributors/implementers can see their work be used and recognized, is an example of a
|
||
rewarding experience for those who wish to contribute to the project.
|
||
|
||
### Which is better?
|
||
We don’t believe one is better than the other - The Grid and Matrix will each follow different values and a different
|
||
type of management.
|
||
|
||
While they will most likely solve technical problems in different ways, The Grid will initially build from what Matrix
|
||
is today and re-use existing and validated documentation. APIs like Client, Application services and Identity service
|
||
will remain compatible in the short term as the focus will be on producing a usable document that describes the server
|
||
protocol, which Matrix currently doesn’t have.
|
||
|
||
As The Grid and Matrix will both have concepts of Application Services and therefore Bridges, linking both protocols and
|
||
network will be a property that will be heavily leveraged on The Grid side to smooth out any transition or API change
|
||
that would make the two protocols otherwise incompatible and unable to exchange data.
|
||
|
||
If you believe in the same values and ideas as we do, we’ll be happy to have you help us with The Grid.
|
||
|
||
## Next steps
|
||
The high level steps are:
|
||
- Creation of a non-profit to kick-start, grow, foster and maintain The Grid protocol.
|
||
- Document all the concepts and network exchanges.
|
||
- Ensuring that at least two independent implementations validate the documentation.
|
||
|
||
### Governing body
|
||
To ensure the protocol is free of control from a single entity and that the users are empowered to control it, a non-profit
|
||
organisation will be created to represent the protocol lead and will be funded on donations.
|
||
|
||
This body will be created as soon as possible as we believe it is key to ensure neutrality in a communication protocol.
|
||
|
||
This governing body will be created using public and community-reviewed articles and will put in place technical board(s)
|
||
composed of key entities from the community to fulfill its mission.
|
||
|
||
This governing body will have for mission to ensure the protocol fostering and that its management values are respected
|
||
and followed.
|
||
|
||
Details about this controlling body, its missions and its proposed articles for creation are available in [a separate
|
||
document](governing-body.md).
|
||
|
||
### Initial Documentation
|
||
We recognize the hard work, effort and engineering level of the current Matrix protocol and wish to build on that,
|
||
instead of starting from scratch. We also believe that having a documentation of the as-is is key for people to contribute
|
||
constructively and avoid an unfair amount of work spent on reverse-engineering implementations to reach a level of
|
||
understanding sufficient to even start meaningful discussions/contributions to improve the protocol.
|
||
|
||
The biggest lack of documentation is currently the as-is for the server protocol, which deals with internal structures
|
||
of a Homeserver, the federation exchanges, the DAG model and its underlying algorithms. It is therefore not possible
|
||
to build on an existing documentation for those core concepts.
|
||
|
||
As the current federation protocol has several critical security vulnerabilities like state resets which have been used
|
||
in attacks [2](#links-and-sources) [3](#links-and-sources) on the live network, and very few underlying algorithms and
|
||
endpoints have been documented, we will take this opportunity to attempt creating internal structures and federation
|
||
exchanges that are immune to those known vulnerabilities, while holding back on any kind of optimization that would
|
||
require re-engineering on working, secure and tested concepts.
|
||
|
||
As all this will be done in the open, as per our core values, current issues to be addressed will be documented and
|
||
labeled appropriately so anyone in the community can offer a documented technical solution for them. We believe that
|
||
fresh sets of eyes and brains are required to solve those vulnerabilities.
|
||
|
||
Work has already been started on this and is currently being committed to this Github repository, and a generated online
|
||
version is also available. This work will be relocated under the newly The Grid organisation repositories very soon.
|
||
|
||
### Independent implementations
|
||
Meanwhile, we will encourage, foster and proactively seek that independent entities implement the protocol using its
|
||
latest documentation.
|
||
|
||
We believe that a communication protocol that does not have at least two independent implementations is fundamentally
|
||
dysfunctional, as its contributors and governing body have failed to communicate to or be understood by other parties
|
||
about its worth, merit and the related technical knowledge.
|
||
|
||
In the near and middle term, emphasis will be put on the various protocol efforts to not diverge too much too fast from
|
||
the Matrix protocol. Independent implementers will be able to easily and painlessly implement The Grid as well, ideally
|
||
in an invisible manner for the end user.
|
||
|
||
## Links and Sources
|
||
[1]: https://matrix.org/blog/home/ - “Support Matrix Today” section
|
||
[2]: https://matrix.org/blog/2018/05/01/security-update-synapse-0-28-1/
|
||
[3]: https://matrix.org/blog/2018/06/14/security-update-synapse-0-31-2/
|