Merge #2078: [Backport] 5.0 backports

9390f8e [Refactor] Pass caught logic_error by reference in CreateSig (random-zebra)
7f37d7e Only use one set of parentheses for optional arguments (Fuzzbawls)
19f5eae [RPC] Use "shield" nomenclature instead of "shielded" (Fuzzbawls)
0cb01c9 GUI: settings information, fix missing initial masternodes count value. (furszy)
20bd97a GUI: ccontrol: remove column checkbox extra whitespace in tree mode (random-zebra)
1555406 warnings: Compiler warning on memset usage for non-trivial type (Lenny Maiorani)
359f0c6 Drop defunct IS_TRIVIALLY_CONSTRUCTIBLE handling from prevector.h (Ben Woosley)
9675199 Reduce redundant code of prevector and speed it up (random-zebra)
e6ffe2a Add new prevector benchmarks. (random-zebra)
04756ee Fixup various formatting inconsistencies with RPC help output (Fuzzbawls)
41fc192 Don't bother registering the `multisend` command (Fuzzbawls)
bbbc7c9 Fix typo in `getsaplingnotescount` help text (Fuzzbawls)
87c4d21 [Build] Unify header file extensions (Fuzzbawls)
411edee [BUG] Miner not paying valid finalized budget if <20 active masternodes (random-zebra)
dabe995 Tests: minor fixes for mining_pos_reorg_py (random-zebra)
03ec6bb Wallet: return unconfirmed balance for all spendable coins by default (random-zebra)
e9a9d54 Consensus: Allow setting old spork key before its expiration (random-zebra)
c5abbda Consensus: update spork keys for v5 (random-zebra)
60f1535 GUI: Dashboard, fix type filtering disappearance. (furszy)
1f28551 Params: clear testnet checkpoints (random-zebra)
5b809b2 Params: Fix PoS block value for testnet to 5 (random-zebra)
4c847eb Params: Update testnet params - remove unused ones (random-zebra)
f2ea00d Params: change testnet datadir to testnet5 (random-zebra)
2444e54 BUG: Miner don't check for PoS activation at negative height (random-zebra)
f3b01ba Consensus: Budget: Fix instant killing of proposals (random-zebra)
0d0fe24 [BUG][GUI] Receive widget: check typeRole before refreshing view (random-zebra)
40e3203 Refactor: initialize dummy Spend/Output descriptions only once (random-zebra)
6ea78ad Sapling: Skip proofs and signatures during fee calculation loop (random-zebra)
0aa9439 Sapling: add option to skip ProveAndSing in TransactionBuilder::Build (random-zebra)
ff2dec2 Sapling: Decouple ProveAndSign from Build (random-zebra)
5b7e86d [Wallet] Do not perform ECDSA signing in the fee calculation inner loop (random-zebra)
7c87241 [Consensus] Bump TxVersion::SAPLING to 3 (random-zebra)
05fe877 Cleanup: remove unneeded check (time-based) for version 4 header (random-zebra)
a29f1e3 Miner: set proper block version based on upgrade enforcements (random-zebra)
8e0a9d9 Consensus: reject blocks with outdated version (<8) after v5 enforcement (random-zebra)
25e2816 BUG: Fix TestBlockValidity for PoW blocks (random-zebra)
1466f0b [GUI] Connect alert signal (furszy)
6850224 Move GetWarnings() into its own file. (furszy)
edf51b3 util GetWarnings, adding "gui" parameter type. (furszy)
60cc6f2 Eliminate data races for strMiscWarning and fLargeWork*Found. (furszy)
3fc1597 Make QT runawayException call GetWarnings instead of directly access strMiscWarning. (furszy)
0ece82d Separate CTranslationInterface from CClientUIInterface (furszy)
a46f00d [Cleanup] Remove unused global fields from util.h/cpp (furszy)
ee5306f Tests: Update sapling_malleable-sigs test, now failing (random-zebra)
8780161 Tests: update sighash json test data for saplingVersion txes (random-zebra)
0a82c10 Script: Properly sign/verify shielded tx with sigversion sapling (random-zebra)
6b5d96d Tests: Add trolling exploit demo (random-zebra)
460a875 Tests: fix order of sapData serialization (random-zebra)
43f7ca7 BUG: Guard access to tx.sapData in connectBlock (random-zebra)
c12ff2b BUG: SSPKM, guard GetDebit/GetCredit for non-shield v2 txes (random-zebra)
5797e2b [GUI][Trivial] Adjust CoinControl column widths for readability (Ambassador)
8c500df Tests: Add sapling_fillblock functional test (random-zebra)
ce7f265 Consensus: check total size of all shielded txes inside a block (random-zebra)
7b8425a Miner: do not include more than 750 kB of shielded txes inside a block (random-zebra)
c314d7b Trivial: Add Zcash copyright (random-zebra)
99fa644 Refactor: Decouple ShieldedSendManyTo from sendtoaddress/sendmany (random-zebra)
2b7b466 [Test] Add case for (shielded) sendtoaddress (random-zebra)
25b2513 [RPC] Redirect sendtoaddress to shieldedsendmany when shielded recipient (random-zebra)
8c2c328 Refactor: always check mnp blockhash in CMasternodePing::CheckAndUpdate (random-zebra)
aca3764 MN: pass sigTime to CMasternodePing constructor (random-zebra)
1ca9476 MN: pass lastPing to CMasternodeBroadcast constructor (random-zebra)
974b965 [Validation] Remove unused DisconnectBlock function definition. (furszy)
5a33bb1 [Validation] Remove unused DisconnectBlocks function. (furszy)
76ae626 [Validation] Remove unused ReprocessBlocks function. (furszy)
626170e [Masternde] Refactor activeMasternode.GetKeys callers to not expect a bool. (furszy)
545ca77 [Masternode] Clean now unused global strMasterNodePrivKey. (furszy)
8a88cd9 [Masternode] Stop parsing the MN private key string on every finalized budget vote. (furszy)
83080ea [Masternode] Stop parsing the masternode private key string on every ping. (furszy)
77a3959 [Masternode] Cache decoded Masternode private key (furszy)
d4b8a32 [net] Avoid initialization to a value that is never read (furszy)
bbc3a7e Revert "Use async name resolving to improve net thread responsiveness" (furszy)
e9469b1 Use Socks5ErrorString() to decode error responses from socks proxy. (furszy)
2e23740 Make Socks5() InterruptibleRecv() timeout/failures informative. (furszy)
7eefaba SOCKS5 connecting and connected messages with -debug=net. (furszy)
6baf377 Make failures to connect via Socks5() more informative and less unnecessarily scary. (furszy)
aff68db [GUI] Transaction detail, scroll memo fix, starting from the top. (furszy)
38895a9 Make tests pass after 2020 (Bernhard M. Wiedemann)
ac8d818 [Wallet] BugFix: automatic upgrade for unlocked wallet isn't upgrading the wallet feature version. (furszy)
b1133c4 [Tests] Check spent nullifier in sapling_wallet_listreceived (random-zebra)
9576897 [RPC] Add nullifiers to listshieldedunspent output (random-zebra)
f99a176 [BUG][RPC] Fix check in wallet upgrade RPC (random-zebra)
fb58f5f [Miner] Update block header's current version to v8 (Fuzzbawls)
99c2d5e [TierTwo] Remove unneeded extra logging. (furszy)
3066165 [Sapling] Transparent inputs selection, fixing inverted order. Sort decrementally instead of incrementally. (furszy)
0eaf735 [Test] Adding sapling transaction validation unit test coverage. (furszy)
f695d12 CMakeList, pivxd and test_pivx fix for missing MacOS framework. (furszy)

Pull request description:

  backports the following PRs from master to the 5.0 branch:

  #2041
  #2042
  #2046
  #2045
  #2052
  #2047
  #2049
  #2048
  #2059
  #2055
  #2054
  #2056
  #2058
  #2060
  #2050
  #2061
  #2062
  #2033
  #2064
  #2063
  #2067
  #2051
  #2065
  #2066
  #2068
  #2069
  #2073
  #2071
  #2080
  #2081
  #2084
  #2087
  #2088
  #2083
  #2070
  #2075
  #2086
  #2091

ACKs for top commit:
  random-zebra:
    utACK 9390f8e
  furszy:
    utACK 9390f8e

Tree-SHA512: aca2dc8fd3b8d234a27ba7b128168cc3e48b0b775b5699f8fe53e84ac548eb99121b3f0304939896c6e876ffd252f2023958ccb38a344425093948f206238892

CMakeLists.txt

@@ -400,6 +400,7 @@ set(COMMON_SOURCES
         ./src/script/script_error.cpp
         ./src/spork.cpp
         ./src/sporkdb.cpp
+        ./src/warnings.cpp
         )
 add_library(COMMON_A STATIC ${BitcoinHeaders} ${COMMON_SOURCES})
 target_include_directories(COMMON_A PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/src
@@ -570,6 +571,10 @@ if(MINIUPNP_FOUND)
     target_include_directories(pivxd PUBLIC ${MINIUPNP_INCLUDE_DIR})
 endif()
 
+if(${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
+    target_link_libraries(pivxd "-framework Cocoa")
+endif()
+
 target_link_libraries(pivxd ${sodium_LIBRARY_RELEASE} -ldl -lpthread)
 
 add_subdirectory(src/qt)

COPYING

@@ -3,6 +3,7 @@ Copyright (c) 2014-2015 Dash Developers
 Copyright (c) 2011-2013 The PPCoin developers
 Copyright (c) 2013-2014 The NovaCoin Developers
 Copyright (c) 2014-2018 The BlackCoin Developers
+Copyright (c) 2016-2020 The Zcash developers
 Copyright (c) 2015-2020 PIVX Developers
 
 Permission is hereby granted, free of charge, to any person obtaining a copy

src/Makefile.am

@@ -131,13 +131,13 @@ LIBSAPLING_H = \
   sapling/sapling_core_write.h \
   sapling/sapling_util.h \
   sapling/prf.h \
-  sapling/noteencryption.hpp \
-  sapling/address.hpp \
-  sapling/note.hpp \
+  sapling/noteencryption.h \
+  sapling/address.h \
+  sapling/note.h \
   sapling/zip32.h \
   sapling/saplingscriptpubkeyman.h \
-  sapling/incrementalmerkletree.hpp \
-  sapling/proof.hpp \
+  sapling/incrementalmerkletree.h \
+  sapling/proof.h \
   sapling/sapling_transaction.h \
   sapling/transaction_builder.h \
   sapling/sapling_operation.h
@@ -289,6 +289,7 @@ BITCOIN_CORE_H = \
   destination_io.h \
   wallet/wallet.h \
   wallet/walletdb.h \
+  warnings.h \
   zpivchain.h \
   zpiv/deterministicmint.h \
   zpiv/mintpool.h \
@@ -511,6 +512,7 @@ libbitcoin_common_a_SOURCES = \
   script/script.cpp \
   script/sign.cpp \
   script/standard.cpp \
+  warnings.cpp \
   script/script_error.cpp \
   spork.cpp \
   sporkdb.cpp \

src/Makefile.bench.include

@@ -13,7 +13,7 @@ bench_bench_pivx_SOURCES = \
   bench/crypto_hash.cpp \
   bench/perf.cpp \
   bench/perf.h \
-  bench/prevector_destructor.cpp
+  bench/prevector.cpp
 
 bench_bench_pivx_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) $(EVENT_CFLAGS) $(EVENT_PTHREADS_CFLAGS) -I$(builddir)/bench/
 bench_bench_pivx_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)

src/activemasternode.cpp

@@ -33,7 +33,6 @@ OperationResult initMasternode(const std::string& _strMasterNodePrivKey, const s
 
     // Global params set
     strMasterNodeAddr = _strMasterNodeAddr;
-    strMasterNodePrivKey = _strMasterNodePrivKey;
 
     // Address parsing.
     const CChainParams& params = Params();
@@ -56,11 +55,12 @@ OperationResult initMasternode(const std::string& _strMasterNodePrivKey, const s
 
     CKey key;
     CPubKey pubkey;
-
-    if (!CMessageSigner::GetKeysFromSecret(strMasterNodePrivKey, key, pubkey)) {
+    if (!CMessageSigner::GetKeysFromSecret(_strMasterNodePrivKey, key, pubkey)) {
         return errorOut(_("Invalid masternodeprivkey. Please see the documentation."));
     }
+
     activeMasternode.pubKeyMasternode = pubkey;
+    activeMasternode.privKeyMasternode = key;
     fMasterNode = true;
     return OperationResult(true);
 }
@@ -169,19 +169,16 @@ bool CActiveMasternode::SendMasternodePing(std::string& errorMessage)
         return false;
     }
 
-    CPubKey pubKeyMasternode;
-    CKey keyMasternode;
-
-    if (!CMessageSigner::GetKeysFromSecret(strMasterNodePrivKey, keyMasternode, pubKeyMasternode)) {
-        errorMessage = "Error upon calling GetKeysFromSecret.\n";
+    if (!privKeyMasternode.IsValid() || !pubKeyMasternode.IsValid()) {
+        errorMessage = "Error upon masternode key.\n";
         return false;
     }
 
     LogPrintf("CActiveMasternode::SendMasternodePing() - Relay Masternode Ping vin = %s\n", vin->ToString());
 
     const uint256& nBlockHash = mnodeman.GetBlockHashToPing();
-    CMasternodePing mnp(*vin, nBlockHash);
-    if (!mnp.Sign(keyMasternode, pubKeyMasternode)) {
+    CMasternodePing mnp(*vin, nBlockHash, GetAdjustedTime());
+    if (!mnp.Sign(privKeyMasternode, pubKeyMasternode)) {
         errorMessage = "Couldn't sign Masternode Ping";
         return false;
     }
@@ -234,3 +231,12 @@ bool CActiveMasternode::EnableHotColdMasterNode(CTxIn& newVin, CService& newServ
 
     return true;
 }
+
+void CActiveMasternode::GetKeys(CKey& _privKeyMasternode, CPubKey& _pubKeyMasternode)
+{
+    if (!privKeyMasternode.IsValid() || !pubKeyMasternode.IsValid()) {
+        throw std::runtime_error("Error trying to get masternode keys");
+    }
+    _privKeyMasternode = privKeyMasternode;
+    _pubKeyMasternode = pubKeyMasternode;
+}

src/activemasternode.h

@@ -40,6 +40,7 @@ class CActiveMasternode
     // Initialized by init.cpp
     // Keys for the main Masternode
     CPubKey pubKeyMasternode;
+    CKey privKeyMasternode;
 
     // Initialized while registering Masternode
     Optional<CTxIn> vin;
@@ -55,6 +56,8 @@ class CActiveMasternode
     bool SendMasternodePing(std::string& errorMessage);
     /// Enable cold wallet mode (run a Masternode with no funds)
     bool EnableHotColdMasterNode(CTxIn& vin, CService& addr);
+
+    void GetKeys(CKey& privKeyMasternode, CPubKey& pubKeyMasternode);
 };
 
 #endif

src/bench/prevector.cpp

@@ -0,0 +1,89 @@
+// Copyright (c) 2015-2017 The Bitcoin Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#include "bench/bench.h"
+
+#include "compat.h"
+#include "prevector.h"
+#include "serialize.h"
+#include "streams.h"
+
+#include <type_traits>
+
+// GCC 4.8 is missing some C++11 type_traits,
+// https://www.gnu.org/software/gcc/gcc-5/changes.html
+#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ < 5
+#define IS_TRIVIALLY_CONSTRUCTIBLE std::has_trivial_default_constructor
+#else
+#define IS_TRIVIALLY_CONSTRUCTIBLE std::is_trivially_default_constructible
+#endif
+
+struct nontrivial_t {
+    int x;
+    nontrivial_t() :x(-1) {}
+};
+static_assert(!IS_TRIVIALLY_CONSTRUCTIBLE<nontrivial_t>::value,
+              "expected nontrivial_t to not be trivially constructible");
+
+typedef unsigned char trivial_t;
+static_assert(IS_TRIVIALLY_CONSTRUCTIBLE<trivial_t>::value,
+              "expected trivial_t to be trivially constructible");
+
+template <typename T>
+static void PrevectorDestructor(benchmark::State& state)
+{
+    while (state.KeepRunning()) {
+        for (auto x = 0; x < 1000; ++x) {
+            prevector<28, T> t0;
+            prevector<28, T> t1;
+            t0.resize(28);
+            t1.resize(29);
+        }
+    }
+}
+
+template <typename T>
+static void PrevectorClear(benchmark::State& state)
+{
+
+    while (state.KeepRunning()) {
+        for (auto x = 0; x < 1000; ++x) {
+            prevector<28, T> t0;
+            prevector<28, T> t1;
+            t0.resize(28);
+            t0.clear();
+            t1.resize(29);
+            t0.clear();
+        }
+    }
+}
+
+template <typename T>
+void PrevectorResize(benchmark::State& state)
+{
+    while (state.KeepRunning()) {
+        prevector<28, T> t0;
+        prevector<28, T> t1;
+        for (auto x = 0; x < 1000; ++x) {
+            t0.resize(28);
+            t0.resize(0);
+            t1.resize(29);
+            t1.resize(0);
+        }
+    }
+}
+
+#define PREVECTOR_TEST(name)                                               \
+    static void Prevector ## name ## Nontrivial(benchmark::State& state) { \
+        PrevectorResize<nontrivial_t>(state);                              \
+    }                                                                      \
+    BENCHMARK(Prevector ## name ## Nontrivial);                            \
+    static void Prevector ## name ## Trivial(benchmark::State& state) {    \
+        PrevectorResize<trivial_t>(state);                                 \
+    }                                                                      \
+    BENCHMARK(Prevector ## name ## Trivial);
+
+PREVECTOR_TEST(Clear)
+PREVECTOR_TEST(Destructor)
+PREVECTOR_TEST(Resize)

src/budget/budgetmanager.cpp

@@ -401,17 +401,12 @@ int CBudgetManager::GetHighestVoteCount(int chainHeight) const
 
 bool CBudgetManager::GetPayeeAndAmount(int chainHeight, CScript& payeeRet, CAmount& nAmountRet) const
 {
-    const CFinalizedBudget* pfb = GetBudgetWithHighestVoteCount(chainHeight);
-    if (!pfb) return false;
-
-    // Check that there are enough votes
-    int mnCount = mnodeman.CountEnabled(ActiveProtocol());
-    int nFivePercent = mnCount / 20;
-    if ((nFivePercent == 0 && !(Params().IsRegTestNet() && mnCount > 0) ) ||
-        pfb->GetVoteCount() < nFivePercent)
+    int nCountThreshold;
+    if (!IsBudgetPaymentBlock(chainHeight, nCountThreshold))
         return false;
 
-    return pfb->GetPayeeAndAmount(chainHeight, payeeRet, nAmountRet);
+    const CFinalizedBudget* pfb = GetBudgetWithHighestVoteCount(chainHeight);
+    return pfb && pfb->GetPayeeAndAmount(chainHeight, payeeRet, nAmountRet) && pfb->GetVoteCount() > nCountThreshold;
 }
 
 bool CBudgetManager::FillBlockPayee(CMutableTransaction& txNew, const int nHeight, bool fProofOfStake) const
@@ -500,10 +495,8 @@ void CBudgetManager::VoteOnFinalizedBudgets()
     // Get masternode keys
     CPubKey pubKeyMasternode;
     CKey keyMasternode;
-    if (!CMessageSigner::GetKeysFromSecret(strMasterNodePrivKey, keyMasternode, pubKeyMasternode)) {
-        LogPrintf("%s: Unable to get masternode keys\n", __func__);
-        return;
-    }
+    activeMasternode.GetKeys(keyMasternode, pubKeyMasternode);
+
     // Sign finalized budgets
     for (const uint256& budgetHash: vBudgetHashes) {
         CFinalizedBudgetVote vote(*(activeMasternode.vin), budgetHash);

src/budget/budgetproposal.cpp

@@ -79,10 +79,10 @@ void CBudgetProposal::SyncVotes(CNode* pfrom, bool fPartial, int& nInvCount) con
     }
 }
 
-bool CBudgetProposal::IsHeavilyDownvoted()
+bool CBudgetProposal::IsHeavilyDownvoted(bool fNewRules)
 {
-    if (GetNays() - GetYeas() > mnodeman.CountEnabled(ActiveProtocol()) / 10) {
-        strInvalid = "Active removal";
+    if (GetNays() - GetYeas() > (fNewRules ? 3 : 1) * mnodeman.CountEnabled(ActiveProtocol()) / 10) {
+        strInvalid = "Heavily Downvoted";
         return true;
     }
     return false;
@@ -162,8 +162,12 @@ bool CBudgetProposal::UpdateValid(int nCurrentHeight)
 {
     fValid = false;
 
-    if (IsHeavilyDownvoted()) {
-        return false;
+    // !TODO: remove after v5 enforcement and use fixed multiplier (3)
+    bool fNewRules = Params().GetConsensus().NetworkUpgradeActive(nCurrentHeight, Consensus::UPGRADE_V5_0);
+
+    // Never kill a proposal before the first superblock
+    if (!fNewRules || nCurrentHeight > nBlockStart) {
+        if (IsHeavilyDownvoted(fNewRules)) return false;
     }
 
     if (IsExpired(nCurrentHeight)) {

src/budget/budgetproposal.h

@@ -27,7 +27,7 @@ class CBudgetProposal
     std::string strInvalid;
 
     // Functions used inside UpdateValid()/IsWellFormed - setting strInvalid
-    bool IsHeavilyDownvoted();
+    bool IsHeavilyDownvoted(bool fNewRules);
     bool IsExpired(int nCurrentHeight);
     bool CheckStartEnd();
     bool CheckAmount(const CAmount& nTotalBudget);