Store txs in block (write part) #4023

BinLiu97 · 2023-11-06T10:29:38Z

Add new tables to store txs:
SYS_NUMBER_2_BLOCK_TXS block number -> txs data (for storing txs in block)
SYS_TXHASH_2_NUMBER tx hash -> block number + index (for getTxByTxHash)

into feature-3.6.0-store_txs_in_block_write

bcos-txpool/bcos-txpool/TxPool.cpp

into feature-3.6.0-store_txs_in_block_write

codecov · 2023-11-09T07:33:56Z

Codecov Report

Attention: 60 lines in your changes are missing coverage. Please review.

Comparison is base (ec31a2c) 0.00% compared to head (6890913) 63.05%.
Report is 19 commits behind head on feature-3.6.0.

❗ Current head 6890913 differs from pull request most recent head b484afa. Consider uploading reports for the commit b484afa to get more accurate results

Files	Patch %	Lines
bcos-ledger/src/libledger/Ledger.cpp	38.29%	56 Missing and 2 partials ⚠️
bcos-ledger/src/libledger/LedgerMethods.cpp	0.00%	1 Missing ⚠️
bcos-scheduler/src/BlockExecutive.cpp	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##           feature-3.6.0    #4023       +/-   ##
==================================================
+ Coverage               0   63.05%   +63.05%     
==================================================
  Files                  0     1040     +1040     
  Lines                  0    88037    +88037     
  Branches               0    28378    +28378     
==================================================
+ Hits                   0    55516    +55516     
- Misses                 0    31455    +31455     
- Partials               0     1066     +1066

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

into feature-3.6.0-store_txs_in_block_write

bcos-ledger/src/libledger/Ledger.cpp

into feature-3.6.0-store_txs_in_block_write

bcos-ledger/src/libledger/Ledger.cpp

bxq2011hust · 2023-11-14T11:26:40Z

If this modification decreases the performance, we need a way to rollback fast, so add a switch of the feature

bxq2011hust · 2023-11-14T11:35:50Z

bcos-ledger/src/libledger/Ledger.cpp

+    // if blockVersion >= 3.6, txs will be stored when commit block, not here
+    if (blockVersion >= uint32_t(bcos::protocol::BlockVersion::V3_6_VERSION))
+    {
+        --TOTAL_CALLBACK;


Suggested change

--TOTAL_CALLBACK;

TOTAL_CALLBACK = 7;

why TOTAL_CALLBACK = 7, i think should be TOTAL_CALLBACK = 8

bxq2011hust · 2023-11-14T11:47:39Z

bcos-txpool/bcos-txpool/TxPool.cpp

-                            [txpool, block]() { txpool->storeVerifiedBlock(block); });
-                    }
+
+                    // Note: To avoid incomplete transactions in the block, the block is only stored once during commit


this should not be comment, is necessary for consensus

bxq2011hust · 2023-11-14T11:57:04Z

bcos-ledger/src/libledger/Ledger.cpp

+            txHashes.push_back(concepts::bytebuffer::toView((*txsToStoreHash)[unstoredTxsCount]));
+
+            numberAndTxid[unstoredTxsCount] =
+                blockNumberStr + "+" + boost::lexical_cast<std::string>(unstoredTxsCount);


Suggested change

blockNumberStr + "+" + boost::lexical_cast<std::string>(unstoredTxsCount);

blockNumberStr + "_" + boost::lexical_cast<std::string>(unstoredTxsCount);

bcos-ledger/src/libledger/Ledger.cpp

bxq2011hust · 2023-11-14T12:01:14Z

bcos-ledger/src/libledger/Ledger.cpp

+            bcos::protocol::Transaction::Ptr txData;
+            if (blockNumber == SYS_CONTRACT_DEPLOY_NUMBER)
+            {
+                txData = m_blockFactory->transactionFactory()->createTransaction(


the left doesn't have this logic? can you explain why we need this?

bcos-ledger/src/libledger/Ledger.cpp

bxq2011hust · 2023-11-14T12:38:11Z

bcos-ledger/src/libledger/Ledger.cpp

+            else
+            {
+                txData =
+                    m_blockFactory->transactionFactory()->createTransaction(bcos::ref(encodeData));


tx is a transaction pointer, why encode and then decode?

into feature-3.6.0-store_txs_in_block_write

binliu21 added 2 commits November 6, 2023 15:14

store transactions in a block (write part)

f8cc38f

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

3ab0e53

into feature-3.6.0-store_txs_in_block_write

JimmyShi22 reviewed Nov 7, 2023

View reviewed changes

bcos-txpool/bcos-txpool/TxPool.cpp Show resolved Hide resolved

binliu21 added 4 commits November 7, 2023 14:22

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

73bbc8e

into feature-3.6.0-store_txs_in_block_write

store transactions in a block (write part), modify bcos default version

ccb12f5

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

b7989d8

into feature-3.6.0-store_txs_in_block_write

store transactions in a block (write part), modify version to 3.6.0

3bbb960

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch from 61357e6 to 3bbb960 Compare November 8, 2023 07:18

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

a2dfae7

into feature-3.6.0-store_txs_in_block_write

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch from 36c64af to a2dfae7 Compare November 9, 2023 07:52

JimmyShi22 reviewed Nov 9, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 9, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 9, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

store transactions in a block (write part), modify asyncPrewriteBlock()

8352943

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch 2 times, most recently from 01a1477 to f31fa45 Compare November 10, 2023 02:30

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

2d05832

into feature-3.6.0-store_txs_in_block_write

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch from f31fa45 to 2d05832 Compare November 10, 2023 03:08

binliu21 added 2 commits November 10, 2023 16:34

store transactions in a block (write part), modify asyncPrewriteBlock()

cbdcc66

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

9cec0ac

into feature-3.6.0-store_txs_in_block_write

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch from 09420d8 to 9cec0ac Compare November 10, 2023 08:35

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

5a4fc58

into feature-3.6.0-store_txs_in_block_write

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch from 44ce74d to 5a4fc58 Compare November 10, 2023 13:19

JimmyShi22 reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

JimmyShi22 reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch 3 times, most recently from 0f815f3 to 19675ad Compare November 14, 2023 03:10

bxq2011hust reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

bxq2011hust reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

bxq2011hust reviewed Nov 14, 2023

View reviewed changes

bcos-ledger/src/libledger/Ledger.cpp Outdated Show resolved Hide resolved

bxq2011hust reviewed Nov 14, 2023

View reviewed changes

Merge branch 'feature-3.6.0' of https://github.com/FISCO-BCOS/FISCO-BCOS

b484afa

into feature-3.6.0-store_txs_in_block_write

BinLiu97 force-pushed the feature-3.6.0-store_txs_in_block_write branch from 1dd84db to b484afa Compare January 3, 2024 02:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Store txs in block (write part) #4023

Store txs in block (write part) #4023

BinLiu97 commented Nov 6, 2023

codecov bot commented Nov 9, 2023 •

edited

bxq2011hust commented Nov 14, 2023

bxq2011hust Nov 14, 2023

bxq2011hust Nov 14, 2023

bxq2011hust Nov 14, 2023

bxq2011hust Nov 14, 2023

bxq2011hust Nov 14, 2023

bxq2011hust Nov 14, 2023

	blockNumberStr + "+" + boost::lexical_cast<std::string>(unstoredTxsCount);
	blockNumberStr + "_" + boost::lexical_cast<std::string>(unstoredTxsCount);

Store txs in block (write part) #4023

Are you sure you want to change the base?

Store txs in block (write part) #4023

Conversation

BinLiu97 commented Nov 6, 2023

codecov bot commented Nov 9, 2023 • edited

Codecov Report

bxq2011hust commented Nov 14, 2023

bxq2011hust Nov 14, 2023

Choose a reason for hiding this comment

bxq2011hust Nov 14, 2023

Choose a reason for hiding this comment

bxq2011hust Nov 14, 2023

Choose a reason for hiding this comment

bxq2011hust Nov 14, 2023

Choose a reason for hiding this comment

bxq2011hust Nov 14, 2023

Choose a reason for hiding this comment

bxq2011hust Nov 14, 2023

Choose a reason for hiding this comment

codecov bot commented Nov 9, 2023 •

edited