Merge pull request #2075 from hyperledger/develop

Iroha v1.5 release
hyperledger-iroha · Apr 8, 2022 · 0d22d11 · 0d22d11
2 parents 7060aa8 + 15f54d9
commit 0d22d11
Show file tree

Hide file tree

Showing 172 changed files with 3,354 additions and 5,410 deletions.
diff --git a/.github/TESTS_ALLOWED_TO_FAIL b/.github/TESTS_ALLOWED_TO_FAIL
@@ -1 +1,4 @@
 system_irohad_test
+integration_add_peer_test
+regression_regression_test
+integration_remove_peer_test
diff --git a/README.md b/README.md
@@ -57,6 +57,7 @@ It is based on JS Wallet but is up-to-date.
 
 [Iroha Explorer](https://codeberg.org/diva.exchange/iroha-explorer)
 [Iroha Docker container with Postgres 10](https://hub.docker.com/r/divax/iroha) and it's [source code](https://codeberg.org/diva.exchange/iroha)
+[Tool to deploy Iroha instances](https://github.com/kuvaldini/iroha-swarm)
 
 ### Want to help us develop Iroha?
 

diff --git a/docs/source/build/index.rst b/docs/source/build/index.rst
@@ -263,13 +263,13 @@ The cmake parameters such as ``-DUSE_BURROW=ON`` are exactly the parameters you
 
 .. code-block:: shell
 
-  cmake --build . --target irohad
+  cmake --build ./build --target irohad
 
 .. warning:: If you want to use tests later, instead of building `irohad` target, you need to use this:
 
 .. code-block:: shell
 
-  cmake --build . --target all 
+  cmake --build ./build --target all 
 
 
 3. Check the result by running the help: 

diff --git a/docs/source/configure/db.rst b/docs/source/configure/db.rst
@@ -0,0 +1,25 @@
+======================
+PostgreSQL vs. RocksDB
+======================
+
+When you use Iroha, you have a choice of building and using it with either PostgreSQL (relational database) or RocksDB (key-value).
+
+Both options are reliable and can be used in production environment but there are some key differences we would like to tell you about that might help you make your choice. 
+
+Specific features of PostgreSQL:
+********************************
+
+- Classic database option -- which means that there are many tools to work with it;
+- When Iroha is working in Docker, PostgreSQL runs in a separate container;
+- With a lot of data PostgreSQL might become slower
+
+.. tip:: You can learn more about this database in its documentation: https://www.postgresql.org/docs/
+
+Specific features of RocksDB:
+*****************************
+
+- Fast (see `performance testing results <https://wiki.hyperledger.org/display/iroha/Release+1.3.0>`_);
+- RocksDB is embedded -- both WSV and blockstore are in the same database which means more consistency, but there is a possibility of manually adding a hash with access to the database which might cause some security-related concerns;
+- Takes less space on the disk and there is no information that it could grow too big
+
+.. tip:: You can learn more about this database in its documentation: https://rocksdb.org/docs/getting-started.html
diff --git a/docs/source/configure/index.rst b/docs/source/configure/index.rst
@@ -8,6 +8,7 @@ Configure
       :maxdepth: 1
 
       torii-tls.rst
+      db.rst
 
 In this section we will understand how to configure Iroha.
 Some configuration parameters must be the same in all the nodes (they are marked with \*) and some can differ.
@@ -43,10 +44,11 @@ Deployment-specific parameters
 - ``utility_service`` (optional) endpoint for maintenance tasks.
   If present, must include ``ip`` address and ``port`` to bind to.
   See `shepherd docs <../maintenance/shepherd.html>`_ for an example usage of maintenance endpoint.
-- ``metrics`` (optional) endpoint to monitor iroha's metrics. Prometheus HTTP server listens on this endpoint.
+- ``metrics`` (optional) endpoint to monitor Iroha's metrics. Prometheus HTTP server listens on this endpoint.
   If present, must correspond format "[addr]:<port>" and could be for example "127.0.0.1:8080", "9090", or ":1234".
   Wrong values implicitly disables Prometheus metrics server. There are also cmdline options ```--metrics_port`` and
   ``--metrics_addr`` to override this parameter.
+- ``healthcheck_port`` (optional) endpoint for Iroha healthcheck. Sending a request to this endpoint in the form of ``http://<host>:<healthcheck_port>/healthcheck`` will return you information about the status of the node: current memory consumption (``memory_consumption``), current number of blocks (``last_block_round``), current count of reject rounds (``last_reject_round``), if the node is syncing information with a remote node at the moment (``is_syncing``), if the node is currently up (``status``). 
 
 There is also an optional ``torii_tls_params`` parameter, which could be included
 in the config to enable TLS support for client communication.
@@ -85,13 +87,7 @@ Environment-specific parameters
   ``10``. However, we recommend to increase this number if you have a lot of
   transactions per second.
 
-    **This parameter affects performance.** Increase this parameter, if your network has a big number of transactions going. If you increase ``max_proposal_size`` due to an inreased throughput, you can increase it independently. But if the speed stays approximately the same, you need to also increase ``proposal_delay`` to allow all these transactions to get into this one big proposal. By increasing this parameter you can improve the performance but note that at some point increasing this value can lead to degradation of the performance.
-
-
-- ``proposal_delay`` is a timeout in milliseconds that a peer waits a response
-  from the orderding service with a proposal. **Important: proposal_delay must be bigger than proposal_creation_timeout. Not following this rule will lead to unstable system.**
-
-    **This parameter affects performance.** If you want bigger proposal size, you will need to give the system time to collect this increased number of transactions into one proposal.
+    **This parameter affects performance.** Increase this parameter, if your network has a big number of transactions going. If you increase ``max_proposal_size`` due to an inreased throughput, you can increase it independently. By increasing this parameter you can improve the performance but note that at some point increasing this value can lead to degradation of the performance.
 
 - ``vote_delay`` \* is a waiting time in milliseconds before sending vote to the
   next peer. Optimal value depends heavily on the amount of Iroha peers in the
@@ -121,7 +117,6 @@ Environment-specific parameters
   long idle time.
   This parameter allows users to find an optimal value in a tradeoff between
   resource consumption and the delay of getting back to work after an idle
-  period. **Important: proposal_delay must be bigger than proposal_creation_timeout. Not following this rule will lead to unstable system.**
 
     **This parameter affects resource consumption.** When you can expect Iroha to stay idle for longer periods of time and would like to save some resources, increase this value - it will make Iroha check for new transactions more rarely. NB: the first transaction after idle period might be a little delayed due to that. Second and further blocks will be processed quicker.
 
@@ -163,7 +158,6 @@ Here is the configuration we used:
 .. code-block:: javascript
 
   "max_proposal_size" : 10000,
-  "proposal_delay" : 1000,
   "vote_delay" : 1000,
   "mst_enable" : true,
   "mst_expiration_time": 1440,
@@ -190,7 +184,6 @@ Unix
   export IROHA_INTERNAL_PORT=10001
   export IROHA_PG_OPT="host=172.19.0.2 port=5432 user=iroha password=helloworld"
   export IROHA_MAX_PROPOSAL_SIZE=10
-  export IROHA_PROPOSAL_DELAY=5000
   export IROHA_VOTE_DELAY=5000
   export IROHA_MST_ENABLE=false
   export IROHA_MST_EXPIRATION_TIME=1440

diff --git a/docs/source/deploy/multiple.rst b/docs/source/deploy/multiple.rst
@@ -16,4 +16,7 @@ In order to form a block, which includes more than a single peer, or requires cu
 Automated
 ---------
 
-Follow `this guide <https://github.com/hyperledger/iroha-deploy/blob/master/ansible/roles/iroha-docker/README.md>`__
+Here is a `community provided tool <https://github.com/kuvaldini/iroha-swarm>`_ to easily deploy Iroha instances.
+
+
+Here is also `a guide <https://github.com/hyperledger/iroha-deploy/blob/master/ansible/roles/iroha-docker/README.md>`__ that might be outdated but could provide some helpful information.
diff --git a/docs/source/maintenance/index.rst b/docs/source/maintenance/index.rst
@@ -15,4 +15,5 @@ Hardware requirements, deployment process in details, aspects related to securit
     troubleshooting.rst
     metrics.rst
     migration-rocksdb.rst
-
+    migration-practice.rst
+
diff --git a/docs/source/maintenance/migration-practice.rst b/docs/source/maintenance/migration-practice.rst
@@ -0,0 +1,21 @@
+========================
+Good Migration Practices
+========================
+
+Iroha maintainers often receive questions about the best ways to migrate projects to new versions of Iroha, so we decided to share our practices that worked the best for us and our projects.
+
+**On an example of a 4 peer network we will go through a migration procedure (also moving from PostgeSQL database to RocksDB) that turned out to be the most stable and reliable in our projects.**
+
+Here are the steps:
+*******************
+
+1. You have the 4 nodes running Iroha with the old version on Postgres
+2. Follow the instructions on `Iroha Database migration <migration-rocksdb.html>`_. Copy the RocksDB folder. *Skip this step if you do not need to switch between Postgres and RocksDB*
+3. Add a new peer running the new version of Iroha using the `Add Peer command <add_peer.html>`_ and with the RocksDB folder on it.
+4. Add similar nodes 2 more times
+5. Now you have 7 nodes -- 4 running the old version and 3 running the new one
+6. Switch off the first node with the old version and update this node with the same key pair to the new version (with RocksDB, if that is your goal) 
+7. Repeat this for every node with the old Iroha version
+8. Now you can remove the "new" nodes and continue the work on your project!
+
+
diff --git a/docs/source/requirements.txt b/docs/source/requirements.txt
@@ -8,7 +8,7 @@ docopt==0.6.2
 docutils==0.14
 idna==2.6
 imagesize==0.7.1
-Jinja2==2.11.3
+Jinja2
 livereload==2.5.1
 MarkupSafe
 pathtools==0.1.2

diff --git a/example/config-win.sample b/example/config-win.sample
@@ -4,7 +4,6 @@
   "internal_port" : 10001,
   "pg_opt" : "host=localhost port=5432 user=postgres password=mysecretpassword",
   "max_proposal_size" : 10,
-  "proposal_delay" : 5000,
   "vote_delay" : 5000,
   "mst_enable" : false,
   "mst_expiration_time" : 1440

diff --git a/example/config.docker b/example/config.docker
@@ -12,7 +12,6 @@
     "maintenance database": "postgres"
   },
   "max_proposal_size" : 10,
-  "proposal_delay" : 5000,
   "vote_delay" : 5000,
   "mst_enable" : false,
   "mst_expiration_time" : 1440,

diff --git a/example/config.postgres.sample b/example/config.postgres.sample
@@ -12,7 +12,6 @@
      "maintenance database": "postgres"
   },
   "max_proposal_size" : 10,
-  "proposal_delay" : 5000,
   "vote_delay" : 5000,
   "mst_enable" : false,
   "mst_expiration_time" : 1440,

diff --git a/example/config.sample b/example/config.sample
@@ -6,7 +6,6 @@
      "path": "/path/to/wsv/folder"
   },
   "max_proposal_size": 10,
-  "proposal_delay": 5000,
   "vote_delay": 5000,
   "mst_enable": false,
   "mst_expiration_time": 1440,

diff --git a/irohad/CMakeLists.txt b/irohad/CMakeLists.txt
@@ -13,7 +13,6 @@ add_subdirectory(network)
 add_subdirectory(model)
 add_subdirectory(simulator)
 add_subdirectory(synchronizer)
-add_subdirectory(multi_sig_transactions)
 add_subdirectory(pending_txs_storage)
 add_subdirectory(util)
 add_subdirectory(maintenance)

diff --git a/irohad/ametsuchi/CMakeLists.txt b/irohad/ametsuchi/CMakeLists.txt
@@ -185,15 +185,31 @@ target_link_libraries(rocksdb_indexer
     RocksDB::rocksdb
     )
 
+add_library(common_burrow_storage
+    impl/burrow_storage.cpp
+    )
+target_link_libraries(common_burrow_storage 
+    common
+    )
+
+add_library(rocksdb_burrow_storage
+    impl/rocksdb_burrow_storage.cpp
+    )
+target_link_libraries(rocksdb_burrow_storage
+    common_burrow_storage
+    RocksDB::rocksdb
+    )
+
 target_compile_definitions(postgres_indexer
     PRIVATE SOCI_USE_BOOST HAVE_BOOST
     )
 
 add_library(postgres_burrow_storage
-    impl/burrow_storage.cpp
     impl/postgres_burrow_storage.cpp
     )
-target_link_libraries(postgres_burrow_storage common)
+target_link_libraries(postgres_burrow_storage 
+    common_burrow_storage
+    )
 
 add_library(default_vm_call INTERFACE)
 if(USE_BURROW)
@@ -222,6 +238,7 @@ target_link_libraries(ametsuchi
     libs_files
     common
     postgres_burrow_storage
+    rocksdb_burrow_storage
     postgres_options
     shared_model_interfaces
     shared_model_plain_backend

diff --git a/irohad/ametsuchi/impl/postgres_specific_query_executor.cpp b/irohad/ametsuchi/impl/postgres_specific_query_executor.cpp
@@ -1498,7 +1498,7 @@ namespace iroha {
             for (const auto &row : range) {
               iroha::ametsuchi::apply(
                   row,
-                  [this, &peers](
+                  [&peers](
                       auto &peer_key, auto &address, auto &tls_certificate) {
                     if (peer_key and address) {
                       peers.push_back(