diff --git a/content/intro/arch.md b/content/intro/arch.md deleted file mode 100644 index dd26bf702..000000000 --- a/content/intro/arch.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: "Architecture Diagrams" -audit: 1 -dashboardWeight: 0.2 -dashboardState: reliable -dashboardAudit: n/a ---- - -# Architecture Diagrams - -## Overview Diagram -{{< details title="TODO" >}} -- cleanup / reorganize - - this diagram is accurate, and helps lots to navigate, but it's still a bit confusing - - the arrows and lines make it a bit hard to follow. We should have a much cleaner version (maybe based on [C4](https://c4model.com)) -- reflect addition of Token system - - move data_transfers into Token -{{< /details >}} - -![Protocol Overview Diagram](diagrams/overview1/overview.dot) - -## Protocol Flow Diagram - -![Deals on Chain](diagrams/sequence/full-deals-on-chain.mmd) - -## Parameter Calculation Dependency Graph - -This is a diagram of the model for parameter calculation. This is made with [orient](https://github.com/filecoin-project/orient), our tool for modeling and solving for constraints. - -![Protocol Overview Diagram](diagrams/orient/filecoin.dot) \ No newline at end of file diff --git a/content/intro/concepts.md b/content/intro/concepts.md index 0f8fe0a21..599b06c8f 100644 --- a/content/intro/concepts.md +++ b/content/intro/concepts.md @@ -1,6 +1,6 @@ --- -title: "Key Concepts" -audit: 2 +title: Key Concepts +weight: 3 dashboardWeight: 0.2 dashboardState: reliable dashboardAudit: n/a @@ -8,38 +8,63 @@ dashboardAudit: n/a # Key Concepts -For clarity, we refer the following types of entities to describe implementations of the Filecoin protocol: +The Filecoin Protocol is a complex system that includes many different and novel concepts. It is highly recommended that users unfamiliar with the protocol go through the [Glossary](glossary) to familiarize themselves with terms that are unique to Filecoin. Here we provide a selected list of the most central concepts needed in order to proceed to the rest of the specification. + +## FIL Network & Protocol + +- **Node Types:** Nodes in the Filecoin network are defined in an abstract manner and are primarily identified in terms of the services they provide. The type of node, therefore, depends on which services the node provides. A basic set of services in the Filecoin network include: chain verification, storage market client, storage market provider, retrieval market client, retrieval market provider and storage mining. Nodes that extend the blockchain need to implement the chain verification and storage mining services. See [Node Types section](filecoin_nodes) for more details. + +- **Files, Messages and Data:** + - _File:_ a file represents the data that a user wants to store on the Filecoin network. Each node runs a `FileStore` which is simply an abstraction for any system or device where a user stores the data it has submitted to Filecoin. See the [File section](file) for more details. + + - _Sector:_ the default unit of storage that miners put in the network. The default value is set to 32GB, but sectors of 64GB are supported too. Sectors can contain data from multiple deals and multiple clients. Sectors are also split in "Regular Sectors", i.e., those that contain deals and "Committed Capacity" (CC), i.e., the sectors/storage that have been made available to the system, but for which a deal has not been agreed yet. See the [Sector section](sector) for more details. + + - _Piece:_ The [Piece](piece) is the main unit of account and negotiation for the data that a user wants to store on Filecoin. A piece is not a unit of storage and therefore, it can be of any size up to the size of a [sector](sector). If a piece is larger than a sector (currently set to 32GB or 64GB and chosen by the miner), then it has to be split in two (or more) pieces. A Piece has got its own _Piece CID_. + + - _Message:_ A message that includes a storage deal between a miner and a client. It is similar to what is referred to as a _transaction message_ in other blockchains. Messages are submitted to the network by clients and propagate to the network of miners. + + - _Block message:_ The Block is the main unit which helps advance the Filecoin Blockchain. Blocks pack transaction messages together. New blocks are produced upon every round completion, or _"Epoch"_, whose duration is 30sec. + + - _Tipset:_ A set of up to 5 blocks. The _Tipset_ is the unit of increment of the Filecoin blockchain. This means that in every round or epoch the Filecoin blockchain is extended by one Tipset (as opposed to one block as is common in other blockchains). A Tipset is a set of blocks that have the same parent set and have been mined in the same epoch. + + - _Message Pool:_ The message pool, `mpool` or `mempool` is a collection of messages that miners have submitted to be included in the blockchain. Every miner node maintains a collection of messages out of which it picks the ones it wants to include in the next block it intends to mine. See the [Message Pool section](message_pool) for more details. + +- _Deal:_ The agreement between the client and a miner for the storage of data. A deal has its own *deal CID* that includes the identifiers of the client, the miner and details about the deal itself. See the [Storage Market section](storage_market) for more details on storage deals. + +- _Sealing:_ the process that takes as input the data that a user submits to the Filecoin network and produces an encoded output that links the data with the specific node. This binding of a sealed sector to a node and the recording of this on the blockchain is what keeps the system secure, i.e., avoids colluding nodes from sharing storage space and reporting to the system more storage than they actually commit to the system. The sealing process enables much of the operation of the Proof of Replication and Proof of SpaceTime and is effectively what keeps the system safe. See the [Sector Sealing section](sealing) for more details. + +- **Actors:** Actors are the Ethereum equivalent of smart contracts in the Filecoin blockchain. Actors carry the logic needed in order to submit transactions, proofs and blocks, among other things, to the Filecoin blockchain. Every actor is identified by a unique address. See the [System Actors section](sysactors) for more details. + +- **Virtual Machine:** The Filecoin Virtual Machine executes actors code and maintains the _state tree_, which is the latest source of truth in the Filecoin blockchain. See the [Virtual Machine section](systems/filecoin_vm) for more details. + +- **Power Table:** The table where the power (in terms of storage capacity) of each miner is kept. + +- **Token:** The Filecoin token, also referred to as FIL. It is the unit of payment in the Filecoin network. + +- **Storage Mining:** The system that guarantees that a storage miner can participate in: i) the Storage Market by agreeing deals with clients to store their data, and ii) the Storage Power Consensus to verify blocks, reach consensus and grow the blockchain. See the [Storage Mining System section](filecoin_mining) for more details. + +- **Algorithms/Proofs:** the proofs that can verify that a specific storage miner has sealed and is actually storing the data as they promised according to the storage deal. + - **Proof of Replication:** The algorithm that verifies that a miner has created a unique copy of some data, according to a storage deal agreed previously. + - **Proof of SpaceTime:** The algorithm that verifies that the miner has continuously stored a unique copy of some data. + +- **Faults:** A miner might fail to generate a proof out of a sector, which triggers a fault. This is a very important part of the system. A fault might be due to a technical failure (hardware or network failure), or due to malicious intentions (?), e.g., a miner not storing the sector anymore in order to allocate storage to a different sector and attempt to double its reward. There are different types of Faults defined based on whether and when a miner has reported the fault to the system. Depending on these parameters the corresponding fee/penalty is applied to the miner. + +- **Markets:** There are two types of Markets in the Filecoin Network: the Storage Market and the Retrieval Market. Peers negotiate storage and retrieval deals, which include all the details on the price of the service and other important details. See the [Filecoin Markets section](filecoin_markets) for more details. + +## FIL Implementation + +For clarity, we refer the following types of entities to describe implementations of the Filecoin protocol: - **_Data structures_** are collections of semantically-tagged data members (e.g., structs, interfaces, or enums). -- **_Functions_** are computational procedures that do not depend on external state (i.e., mathematical functions, - or programming language functions that do not refer to global variables). - -- **_Components_** are sets of functionality that are intended to be represented as single software units - in the implementation structure. - Depending on the choice of language and the particular component, this might - correspond to a single software module, - a thread or process running some main loop, a disk-backed database, or a variety of other design choices. - For example, the [ChainSync](chainsync) is a component: it could be implemented - as a process or thread running a single specified main loop, which waits for network messages - and responds accordingly by recording and/or forwarding block data. - -- **_APIs_** are the interfaces for delivering messages to components. - A client's view of a given sub-protocol, such as a request to a miner node's - Storage Provider component to store files in the storage market, - may require the execution of a series of API requests. - -- **_Nodes_** are complete software and hardware systems that interact with the protocol. - A node might be constantly running several of the above _components_, participating in several _subsystems_, - and exposing _APIs_ locally and/or over the network, - depending on the node configuration. - The term _full node_ refers to a system that runs all of the above components and supports all of the APIs detailed in the spec. - -- **_Subsystems_** are conceptual divisions of the entire Filecoin protocol, either in terms of complete protocols - (such as the [Storage Market](storage_market) or [Retrieval Market](retrieval_market)), or in terms of functionality - (such as the [VM - Virtual Machine](intro/filecoin_vm)). They do not necessarily correspond to any particular node or software component. - -- **_Actors_** are virtual entities embodied in the state of the Filecoin VM. - Protocol actors are analogous to participants in smart contracts; - an actor carries a FIL currency balance and can interact with other actors - via the operations of the VM, but does not necessarily correspond to any particular node or software component. +- **_Functions_** are computational procedures that do not depend on external state (i.e., mathematical functions, or programming language functions that do not refer to global variables). + +- **_Components/Services_** are sets of functionality that are intended to be represented as single software units in the implementation structure. Depending on the choice of language and the particular component, this might correspond to a single software module, a thread or process running some main loop, a disk-backed database, or a variety of other design choices. For example, the [ChainSync](chainsync) is a component: it could be implemented as a process or thread running a single specified main loop, which waits for network messages and responds accordingly by recording and/or forwarding block data. + +- **_APIs_** are the interfaces for delivering messages to components. A client's view of a given sub-protocol, such as a request to a miner node's Storage Provider component to store files in the storage market, may require the execution of a series of API requests. + +- **_Nodes_** are complete software and hardware systems that interact with the protocol. A node might be constantly running several of the above _components or services_, participating in several _subsystems_, and exposing _APIs_ locally and/or over the network, depending on the node configuration. The term _full node_ refers to a system that runs all of the above components and supports all of the APIs detailed in the spec. + +- **_Subsystems_** are conceptual divisions of the entire Filecoin protocol, either in terms of complete protocols (such as the [Storage Market](storage_market) or [Retrieval Market](retrieval_market)), or in terms of functionality (such as the [VM - Virtual Machine](intro/filecoin_vm)). They do not necessarily correspond to any particular node or software component. + +- **_Actors_** are virtual entities embodied in the state of the Filecoin VM. Protocol actors are analogous to participants in smart contracts; an actor carries a FIL currency balance and can interact with other actors via the operations of the VM, but does not necessarily correspond to any particular node or software component. diff --git a/content/intro/economy/_index.md b/content/intro/economy/_index.md new file mode 100644 index 000000000..4fd192d79 --- /dev/null +++ b/content/intro/economy/_index.md @@ -0,0 +1,84 @@ +--- +title: Filecoin Economy +weight: 6 +dashboardWeight: 0.2 +dashboardState: reliable +dashboardAudit: n/a +--- + +# Filecoin Economy + +## A Market for Data + +Filecoin is more than a network; the protocol lays the groundwork for a market economy built around a marketplace for the storage and retrieval of data. + +Markets possess the required mechanisms to balance supply and demand, determining the prices at which goods or services trade. In the case of Filecoin, when a node has extra storage capacity, which they want to contribute to the Filecoin network, they do so for a fee, while those seeking data storage may be willing to pay for storage services. Aligning these two prices would result in a deal (i.e., a market clearing transaction). Creating a marketplace allows consumers and providers to come together and express how much they value commodities. In thriving economies, there are so-called "gains from trade," where trades increase the net utility (or happiness) of both participants. + +The concept of Storage and Retrieval Markets are central to the existence and operation of Filecoin. + +As a global digital marketplace, it is necessary to choose at least one valid currency for transactions. Because Filecoin is a permissionless market with cryptographically verifiable goods, design constraints could only be satisfied with a native token, the filecoin (FIL). This token acts as a medium of exchange, facilitating transactions and production activities, somewhat similar to the in-game currencies of virtual economies in multiplayer online games. However, as a currency, the token also acts as a store of value; its minting must be tied to adding utility to the network. The high-level interactions underpinning this digital storage market are illustrated in the following Figure. + +![Basic client-miner interaction sequence: clients propose deals to miners, miners accept deals, data transfer occurs, miners seal deal data into sectors, miners prove their sectors and start earning block reward, clients retrieve files from miners.](client-miner-interaction.jpg) + +Given the nature of Filecoinʼs Economy, the protocol must generate its own currency. The creation of new tokens can facilitate more trades and stimulate the economy, but if supply grows too quickly, it may harm the economy. Ideally, the rate of token minting should approximately match the rate of value creation, which provides a clear mandate for the protocolʼs token generation rates. Value is created as goods are traded (i.e. supply clears with demand, creating gains from trade). Filecoinʼs minting function tries to approximate this value, in the absence of a reliable and incentive-compatible measure of economic surplus. + +The overarching goal for the Filecoin economy is to make the production as efficient as possible and make the network as attractive as it can be to the outside world. An economy producing more valuable goods more efficiently will lead to more demand for the goods and more demand for the networkʼs token. Malicious participants in the Filecoin Economy, who attempt to exploit the protocol, will drive real demand away from the economy and destroy value, harming all members of the economy. As a result, the goal of all participants should be to get real paying demand for the goods that the network produces - key to a thriving economy that can operate with little or no subsidies. In the long term, the Filecoin Network must be a protocol where demand from clients is so strong that mining is still profitable with little or no block reward. + +## Participants in the Filecoin Economy + +Within the Filecoin Economy, there are two distinct markets, and participants in the network exchange different goods or services. In the **storage market**, storage miners offer to rent out digital storage that will be verified by the Filecoin Network. Conversely, storage clients offer filecoin to have their data stored. This storage is priced based on the amount of storage space used and the duration of the contract in time. There is a **retrieval market**, where clients pay filecoin to retrieval miners to provide them with a copy of the data. Lastly, token exchanges could enable participants to trade to put filecoin in the hands of clients, miners, and other token holders. The five primary stakeholder groups, as well as the common interactions between them, are illustrated in the following Figure. + +![Illustration of different network participants and common sample interactions. Note that an individual or entity can take on multiple roles. For instance, anyone in possession of filecoin would be considered a token holder.](participants.png) + +Collaboration across participants is required to make Filecoin a robust and efficient economy in producing valuable storage-related goods and services. Some participants may join temporarily to exploit subsidies from the protocol. However, the protocol is designed so that rational decisions are aligned with the creation of value for the network; therefore their profits are increased if they join the network and participate on long time horizons. + +Given the permissionless nature of the network, the rules and parameters must incentivize participant alignment with the long-term goals and mission of the network. It is possible to rely on trust in a small community. But in order to support permissionless, pseudonymous interactions, rules must be enforceable with cryptography and aligned with incentives. As a distributed storage network, the Filecoin Networkʼs value is fundamentally determined by its ability to deliver long-term, reliable, and useful storage. As a result, a community of rational actors will lead to a network where real value is created, as clients are drawn to the functionality of the network. + +## The Storage Network + +Storage clients play an important role in the ecosystem, not just by using the storage services, but also by shaping the storage network offering: because storage clients bring demand to an export-oriented economy, anyone with filecoin tokens to spend on storage can push the network to better meet consumer demand. + +### Client Lifecycle + +Equipped with data and filecoin tokens, likely obtained at an exchange, a storage client is ready to store data on the Filecoin Network. Before engaging miners, a storage client might choose to become verified, which will be elaborated on later in this section. The client begins participation in the storage market by preparing data into a piece, finding a miner, and proposing a deal to the miner. Once terms like price and duration are agreed upon, the miner and client enter into a deal and the client can later pay for the miner to retrieve it. The Figure below presents an overview of clients lifecycle. + +![Lifecycle of clients, acquire tokens, optionally become a verified client, browse miner asks and make specific requests, make deals with miners once agreements are reached, chain verifies storage for clients and pays miners for storage, retrieves files from miners.](client-lifecycle.jpg) + +### Deal Proposals + +Deals are communicated using a gossip network, where proposals are offered by clients and miners. This free exchange of prices acts as a decentralized storage market. Many third party entities operating on this platform can enhance the deal proposal and matching process, including participation from exchanges or the creation of novel aggregators of deal bids and asks. These communications are not even limited to one protocol or platform; storage deals could be proposed over digital spaces like Twitter or email (especially for storing large amounts of data), for example. + +### Deals + +The deals themselves are formed when storage demand and supply are met on Filecoin. In addition to stating the storage amount, price, and duration, deals can also specify the amount of collateral that both storage miners and clients pledge during deal creation. A storage miner would offer a provider deal collateral above the minimum amount as a differentiating factor, allowing them to signal their commitment and level of service. Additionally, a miner could select for deals where clients also provide collateral. This mechanism exists but is left optional for market participants. + +However, there is a minimum provider deal collateral set by the protocol, needed because the chain state and bandwidth are limited resources and providers should guarantee a minimum level of service to clients. Minimum deal collateral requirement per unit quality adjusted power is set such that if the network baseline is filled with deals, the sum of deal collateral targets 5% of the circulating supply of Filecoin. + +In addition to collateral commitments from both parties, thereʼs also a deal payment from client to miner. This payment is initially locked by the client when the deal is incorporated into the blockchain; as a result, the clientʼs exposure to Filecoin price volatility ends the moment they enter into a storage deal. Payment is released to the miner as some fraction of the total deal fee per payment period. Deal payments begin to pay out immediately to provide a greater incentive for miners to seek paid client deals. If a miner terminates before a deal expires, the miner will pay a termination fee and the provider deal collateral will be burned. Additionally, the remaining deal payments will be returned to the client. + + +There is clear utility to the network from developing some additional features. To this end, the following improvements are prioritized for future releases: +- **Deal Extension:** A client and miner could easily agree to renew a deal. +- **Deal Migration:** Instead of implicitly bounding the maximum deal duration by the sector lifetime, a deal could be capable of outliving the sectors that contains it, permitting it to be migrated to a new sector. +- **Deal Substitution:** A client could replace one of their deals in a sector with another file of the same size. +- **Deal Transfer:** A miner could pass on deals to other miners with client consent. + +### Verified Clients + +Verified clients make the Filecoin Economy more robust and valuable. While a storage miner may choose to forgo deal payments and self-deal to fill their storage and earn block rewards, this is not as valuable to the economy and should not be heavily subsidized. However, it is impossible in practice to tell useful data apart from encrypted zeros. Introducing verified clients pragmatically solves this problem through social trust and validation. There will be a simple and open verification process to become a verified client; this process should attract clients who will bring real storage demand to the Filecoin Economy. + +Verifiers should eventually form a decentralized, globally distributed network of entities that confirms the useful storage demand of verified clients. If a verifier evaluates and affirms a clientʼs demand to have real data stored, that client will be able to add up to a certain amount of data to the network as verified client deals; this limit is called a DataCap allocation. Verified clients can request an increased DataCap once they have used their full allocation and Verifiers should perform some due diligence to ensure that the clients are not maliciously exploiting verification. The verification process will evolve over time to become more efficient, decentralized, and robust. + +Storage demand on the network will shape the storage offering provided by miners. With the ability to deploy data with a greater sector quality multiplier, verified clients play an even more important role in shaping the quality of service, geographic distribution, degree of decentralization, and consensus security of the network. Verifiers and verified clients must be cognizant of the value and responsibility that come with their role. Additionally, it is conceivable for miners to have a business development team to source valuable and useful dataset in the world, growing demand for the storage they provide. Teams would be incentivized to help their clients through the verification process and start storing data on the Filecoin Network, in addition to providing their clients with strong SLAs. + +## The Delivery Network + +The storage market alone is not sufficient for a functioning decentralized storage network that can serve as a foundation for humanityʼs information. The network must additionally provide a mechanism by which content can be efficiently delivered to locations where it is requested. + +At present, the internet achieves this distribution using a quasi-hierarchical architecture with efficient communication along the so-called backbone. Latency generally increases as the content and the client move farther from these high-speed connections and each other. To facilitate efficient distribution of data, content delivery networks (CDNs) use a geographically distributed network of servers and data centers to route requests toward the nearest copy of the data. Moving physical copies of data is increasingly cheaper and faster than increasing bandwidth and connectivity between servers and clients, as exemplified by [Netflixʼs Open Connect program](https://openconnect.netflix.com/en/), wherein content is physically delivered to ISPs as appliances, or [AWSʼs Snow Family](https://aws.amazon.com/snow/), which addresses the inverse problem of getting large datasets from individuals to datacenters. + +Filecoin aims to provide infrastructure for a permissionless data-retrieval market. This will allow retrieval miners to bring data to the clients. Marketplaces can form in different regions with local retrieval miners serving content to local clients as illustrated in the Figure below. Market forces inform not only how capabilities should be allocated to meet demand, but also what characteristics and features are best-suited to the users' needs and interests. As a result, the role this retrieval market plays is clearly distinct from the storage market, where miners provide long term durable storage. It is also likely for storage miners to provide direct retrieval services in the early days of the network. + +![Illustration of retrieval market on Filecoin.](filecoin-retrieval.png) + +A transaction on the retrieval market proceeds as follows: the client finds a provider of a given piece and queries the provider to see if it meets its retrieval criteria. The client schedules a data transfer passing the RetrievalDealProposal as a voucher. Clients exchange payment for data with the chosen retrieval miner using a payment channel. To reduce the risk of malicious behavior, small amounts of data and FIL are exchanged incrementally. The profitability of retrieval will depend on pricing, popularity, locality and latency. \ No newline at end of file diff --git a/content/intro/economy/client-lifecycle.jpg b/content/intro/economy/client-lifecycle.jpg new file mode 100644 index 000000000..426a6e767 Binary files /dev/null and b/content/intro/economy/client-lifecycle.jpg differ diff --git a/content/intro/economy/client-miner-interaction.jpg b/content/intro/economy/client-miner-interaction.jpg new file mode 100644 index 000000000..9e17d4304 Binary files /dev/null and b/content/intro/economy/client-miner-interaction.jpg differ diff --git a/content/intro/economy/filecoin-retrieval.png b/content/intro/economy/filecoin-retrieval.png new file mode 100644 index 000000000..7282a5f81 Binary files /dev/null and b/content/intro/economy/filecoin-retrieval.png differ diff --git a/content/intro/economy/participants.png b/content/intro/economy/participants.png new file mode 100644 index 000000000..2d7260ea4 Binary files /dev/null and b/content/intro/economy/participants.png differ diff --git a/content/intro/filecoin_vm.md b/content/intro/filecoin_vm.md index c85ac3994..ac5dd2e1d 100644 --- a/content/intro/filecoin_vm.md +++ b/content/intro/filecoin_vm.md @@ -1,6 +1,6 @@ --- title: "Filecoin VM" -weight: 3 +weight: 5 dashboardWeight: 0.2 dashboardState: reliable dashboardAudit: n/a diff --git a/content/intro/lifecycle.md b/content/intro/lifecycle.md new file mode 100644 index 000000000..79f066270 --- /dev/null +++ b/content/intro/lifecycle.md @@ -0,0 +1,30 @@ +--- +title: Lifecycle of Data +weight: 4 +dashboardWeight: 0.2 +dashboardState: reliable +dashboardAudit: n/a +--- + +# Lifecycle of Data in Filecoin + +## Preparing for storing data on Filecoin + +1. Data stored on the Filecoin network is split in chunks, similarly to what is happening when adding files to IPFS. Once a file is split into chunks, the IPLD DAG is created, which has its own root CID, called *payload CID* in Filecoin. +2. After some conversions (described in more detail in the corresponding [section](piece)) the DAG is converted to what constitutes the *Filecoin Piece*, which is the main unit of negotiation from the client's point of view. In other words, the *piece* is what the client is negotiating a deal for later on in the process. Every piece has its own CID, called the *piece CID*, or *CommP* (Piece Commitment). + +## Finding a deal and publishing the deal on the Filecoin blockchain. + +1. A user that wants to store data in the Filecoin network needs to agree a deal with the network/a miner. Finding and agreeing on a deal is done out of band, i.e., without involving the blockchain. + +2. The deal is negotiated and agreed between the user and a miner (out-of-band). The *piece CID* is augmented with deal details to create the *Deal Proposal* that has its own *deal CID*. Among other things, the *deal CID* contains the identities of both the client and the miner, as well as transaction details. Data is sent from the client to the miner. + +3. Once the data transfer is complete (and the miner verifies that the data is the same as the ones negotiated), the deal is published on the blockchain and the system enters the phase of Storage Mining. Together with publishing the deal on the blockchain, the miner is submitting Initial Pledge collateral, so that it can be held accountable if it fails to prove storage later in the process. + +4. The miner places incoming data in a Sector and starts the Sealing process. Once the sealing process is complete the deal is announced/recorded to the Storage Market Actor (through the `ProveCommitSector` function). At this point the miner also puts pledge collateral proportional to the amount of storage they have committed on chain. This triggers the Proof of Replication algorithm. Note that both a sealed copy of the data is _unique to all of the following at the same time_: i) the specific data, ii) the miner storing the data, but also iii) the time at which the miner has sealed the data and produced the _payload CIDs_. In other words, if a miner attempts to re-seal the same data that it has sealed before it will produce a different _payload CID_. + +5. **Proof-of-Replication** is the process by which the miner proves that they store a unique copy of the data. This proof is taking place only once when the miner first receives and stores the data. The proof consists of the "Sealed Sector CID" (or CommR, Commitment of Replication?), which is unique both to the miner and the sealing process itself. That is, if the same miner seals the same data again, they will produce a different "Sealed Sector CID". + +6. From that point on, the miner is expected to keep the data stored for as long as the deal specifies. The system checks that the miner is indeed storing the sealed data through **Proofs of SpaceTime**. The system is continuously (i.e., not only once as in PoRep) asking the miner to solve puzzles based on some random part of the data. This process proves that the miner indeed stores a unique copy of the data. If the miner fails to provide the Proof-of-SpaceTime, the miner loses their collateral, which they have submitted to the system earlier. + +7. Throughout the process, miners might run into Faults, which keeps them back from providing the data to the client, but also from proving that they store the data (provifing Proofs of SpaceTime). Depending on when in the process a miner reports a fault, they receive the corresponding penalty: a miner that reports a fault immediately is penalized less than a miner who fails to provide a proof, when asked. diff --git a/content/intro/motivation/_index.md b/content/intro/motivation/_index.md new file mode 100644 index 000000000..c028334e0 --- /dev/null +++ b/content/intro/motivation/_index.md @@ -0,0 +1,28 @@ +--- +title: Motivation +weight: 2 +dashboardWeight: 0.2 +dashboardState: reliable +dashboardAudit: n/a +--- + +# Motivation + +Filecoin is a decentralized data storage and distribution network; it is therefore necessary to place such a network in today's networking and storage market landscape. Cloud storage has evolved into an industry valued at [over $46B](https://www.alliedmarketresearch.com/cloud-storage-market) in 2019, where data is stored in enormous data centers far from end users, where space and utility costs are low. Content Delivery/Distribution Networks (CDNs) then acquire space and pack servers in data centers to cache content nearer to users. The CDN industry is currently valued at [over $12B](https://www.marketsandmarkets.com/PressReleases/cdn.asp). The rate at which we generate data is growing as video and images supplement text, resolution grows, and the number of internet-accessible devices increases. The growing amount of data stored globally is depicted in the Figure below. And the top five storage providers control [77 percent](https://www.gartner.com/en/newsroom/press-releases/2019-07-29-gartner-says-worldwide-iaas-public-cloud-services-market-grew-31point3-percent-in-2018) of the global Infrastructure as a Service (IaaS) market, making it difficult for new market entrants to compete. Any new entrant to the market must compete against the reputation, infrastructure, and potentially network effects possessed by existing providers. Without a single platform for coordination and collaboration, small entities cannot effectively compete with the capacity, scale, and reputation of existing providers. + +![Size of the global datasphere by year](data-market.png) +[Figure source](https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf) + +At the same time, the profit of CDN companies from storage and distribution is falling, due to: i) increased competition that drives prices down, ii) non-decreasing bandwidth cost, and iii) exploding demand for bandwidth from modern applications. This has led traditional CDN companies to turn to other markets and services (e.g., providing security solutions and IT services) in order to keep up with their profit expectations. + +This is where Filecoin comes in to fill the gap in the market and serve as a universal platform for low-cost and decentralised storage. The Filecoin network collapses market entry costs and allows anyone to become a storage provider, while at the same time fuelling the Filecoin Economy. + +The data itself sits on a spectrum of access. Most data is never read; users simply want to know that they can access that document, photograph, record, or footage if the need arises. Alternatively, some data is accessed very frequently, or goes viral, and must be replicated across the internet to be readily available to requesters with low latency. Just as Airbnb allows homeowners to compete with hotels, the Filecoin Protocol will allow any willing and able storage provider to join a storage network and offer their storage for sale, while offloading auxiliary tasks like tooling, documentation, and branding to the Filecoin Network itself. At first, the network will be best adapted to infrequently read files, optionally supporting fast retrieval at a clientʼs request; the number of use cases will grow as the network grows, as the protocol evolves, and as tools are built. The Filecoin Network has a unique value proposition: + +- **Verifiable storage:** Rather than needing to trust a cloud storage provider or rely on legal recourse, the Filecoin Protocol cryptographically verifies whether or not clients' data are being stored. +- **Open participation:** Anyone with sufficient hardware and an internet connection can participate in the Filecoin Network. +- **Distribution of storage empowers local optimization:** Driven by open participation, market forces in the Filecoin Economy should communicate information more efficiently and the network will be more responsive than centralized storage platforms. The website or video a client requested could be hosted as close as the Filecoin Mining node next door. This will increase the utility of the network by ensuring that data can be stored near the network edge, close to the clients, in a cost-effective way. +- **Flexible storage options:** Because the Filecoin Network will be an open platform, the network will support many additional tools and auxiliary services provided by the community of developers improving or building on top of the protocol. +- **The network has been built by a community:** Filecoin will be stronger because it provides participants the opportunity to obtain stake in the networkʼs success. Filecoin participants will benefit by working together to improve the Filecoin Network. + +Given the growth of both demand and supply, a decentralized storage networkʼs differentiated approach to cloud storage could unlock significant efficiency improvements and growth. diff --git a/content/intro/motivation/data-market.png b/content/intro/motivation/data-market.png new file mode 100644 index 000000000..5f3b1b2cb Binary files /dev/null and b/content/intro/motivation/data-market.png differ