{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":6934395,"defaultBranch":"main","name":"rocksdb","ownerLogin":"facebook","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2012-11-30T06:16:18.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/69631?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1714595653.0","currentOid":""},"activityList":{"items":[{"before":"db0960800a2971e94c73d7e7eb1bd2e9b8d3c54c","after":"c72ee4531b288bf08b9414155fafb86cc4378fb4","ref":"refs/heads/main","pushedAt":"2024-05-22T22:39:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix recycled WAL detection when wal_compression is enabled (#12643)\n\nSummary:\nI think the point of the `if (end_of_buffer_offset_ - buffer_.size() == 0)` was to only set `recycled_` when the first record was read. However, the condition was false when reading the first record when the WAL began with a  `kSetCompressionType` record because we had already dropped the `kSetCompressionType` record from `buffer_`. To fix this, I used `first_record_read_` instead.\n\nAlso, it was pretty confusing to treat the WAL as non-recycled when a recyclable record first appeared in a non-first record. I changed it to return an error if that happens.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12643\n\nReviewed By: hx235\n\nDifferential Revision: D57238099\n\nPulled By: ajkr\n\nfbshipit-source-id: e20a2a0c9cf0c9510a7b6af463650a05d559239e","shortMessageHtmlLink":"Fix recycled WAL detection when wal_compression is enabled (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2290547598\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12643\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12643/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12643\">#12643</a>)"}},{"before":"733150f6aaf52c8dcedb843fc329b472bba4ffd7","after":"db0960800a2971e94c73d7e7eb1bd2e9b8d3c54c","ref":"refs/heads/main","pushedAt":"2024-05-22T18:34:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add Transaction::PutEntity to the stress tests (#12688)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12688\n\nAs a first step of covering the wide-column transaction APIs, the patch adds `PutEntity` to the optimistic and pessimistic transaction stress tests (for the latter, only when the WriteCommitted policy is utilized). Other APIs and the multi-operation transaction test will be covered by subsequent PRs.\n\nReviewed By: jaykorean\n\nDifferential Revision: D57675781\n\nfbshipit-source-id: bfe062ec5f6ab48641cd99a70f239ce4aa39299c","shortMessageHtmlLink":"Add Transaction::PutEntity to the stress tests (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2310919265\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12688\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12688/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12688\">#12688</a>)"}},{"before":"014368f62c4654c64f1b199faf322b17c6fad56a","after":"733150f6aaf52c8dcedb843fc329b472bba4ffd7","ref":"refs/heads/main","pushedAt":"2024-05-22T18:16:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Flush WAL upon DB close (#12684)\n\nSummary:\n**Context/Summary:** https://github.com/facebook/rocksdb/pull/12556 `avoid_sync_during_shutdown=false` missed an edge case where `manual_wal_flush == true` so WAL sync will still miss unflushed WAL. This PR fixes it.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12684\n\nTest Plan: modified UT to include this case `manual_wal_flush==true`\n\nReviewed By: cbi42\n\nDifferential Revision: D57655861\n\nPulled By: hx235\n\nfbshipit-source-id: c9f49fe260e8b38b3ea387558432dcd9a3dbec19","shortMessageHtmlLink":"Flush WAL upon DB close (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2309511630\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12684\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12684/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12684\">#12684</a>)"}},{"before":"1827f3f98374a065cef08faf82dbda006beb2830","after":"014368f62c4654c64f1b199faf322b17c6fad56a","ref":"refs/heads/main","pushedAt":"2024-05-22T18:10:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix the names of function objects added in PR 12681 (#12689)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12689\n\nThese should be in `snake_case` (not `camelCase`) per our style guide.\n\nReviewed By: jowlyzhang\n\nDifferential Revision: D57676418\n\nfbshipit-source-id: 82ad6a87d1540f0b29c2f864ca0128287fe95a9e","shortMessageHtmlLink":"Fix the names of function objects added in PR 12681 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2310959400\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12689\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12689/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12689\">#12689</a>)"}},{"before":"ad6f6e24c80c8aa14ab107711ecf2de482ad89cf","after":"1827f3f98374a065cef08faf82dbda006beb2830","ref":"refs/heads/main","pushedAt":"2024-05-22T14:18:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Remove extra semi colon from internal_repo_rocksdb/repo/table/sst_file_reader.cc\n\nSummary:\n`-Wextra-semi` or `-Wextra-semi-stmt`\n\nIf the code compiles, this is safe to land.\n\nReviewed By: palmje\n\nDifferential Revision: D57632757\n\nfbshipit-source-id: 1dbad2a2e185381e225df8b9027033e06aeaf01b","shortMessageHtmlLink":"Remove extra semi colon from internal_repo_rocksdb/repo/table/sst_fil…"}},{"before":"62600cb2d4bf6392d871103cad5a31102c669bea","after":"ad6f6e24c80c8aa14ab107711ecf2de482ad89cf","ref":"refs/heads/main","pushedAt":"2024-05-22T07:52:30.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix txn_write_policy check in crash test script (#12683)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12683\n\nWith optimistic transactions, the stress test parameter `txn_write_policy` is not applicable and is thus not set. When the parameter is subsequently checked, Python's dictionary `get` method returns `None`, which is not equal to zero. The net result of this is that currently, `sync_fault_injection` and `manual_wal_flush_one_in` are always disabled in optimistic transaction mode (most likely unintentionally).\n\nReviewed By: cbi42\n\nDifferential Revision: D57655339\n\nfbshipit-source-id: 8b93a788f9b02307b6ea7b2129dc012271130334","shortMessageHtmlLink":"Fix txn_write_policy check in crash test script (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2309478459\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12683\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12683/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12683\">#12683</a>)"}},{"before":"cee32c5ccecc6beaf72ad6c405875792252594b9","after":"62600cb2d4bf6392d871103cad5a31102c669bea","ref":"refs/heads/main","pushedAt":"2024-05-22T00:25:22.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix rebuilding transactions containing PutEntity (#12681)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12681\n\nWhen rebuilding transactions during recovery, `MemtableInserter::PutCFImpl` currently calls `WriteBatchInternal::Put` regardless of value type, which is incorrect for `PutEntity` entries, as well as `TimedPut`s and the blob indexes used by the old BlobDB implementation. The patch fixes the handling of `PutEntity` and returns `NotSupported` for `TimedPut`s and blob indices.\n\nReviewed By: jaykorean, jowlyzhang\n\nDifferential Revision: D57636355\n\nfbshipit-source-id: 833de4e4aa0b42ff6638b72c4181f981d12d0f15","shortMessageHtmlLink":"Fix rebuilding transactions containing PutEntity (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2309040228\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12681\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12681/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12681\">#12681</a>)"}},{"before":"d89ab23bec4c4d539f6c9377ef0eaa2163a849b3","after":"cee32c5ccecc6beaf72ad6c405875792252594b9","ref":"refs/heads/main","pushedAt":"2024-05-21T19:59:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"use nullptr instead of NULL / 0 in rocksdbjni (#12575)\n\nSummary:\nWhile I was trying to understand issue https://github.com/facebook/rocksdb/issues/12503, I found this minor problem. Please have a look adamretter rhubner\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12575\n\nReviewed By: ajkr\n\nDifferential Revision: D57596055\n\nPulled By: cbi42\n\nfbshipit-source-id: ee0860bdfbee9364cd30c23957b72a04da6acd45","shortMessageHtmlLink":"use nullptr instead of NULL / 0 in rocksdbjni (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2258389255\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12575\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12575/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12575\">#12575</a>)"}},{"before":"d7b938882e4e81892721d5787c446348ffb782c8","after":"d89ab23bec4c4d539f6c9377ef0eaa2163a849b3","ref":"refs/heads/main","pushedAt":"2024-05-21T17:21:38.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Disallow memtable flush and sst ingest while WAL is locked (#12652)\n\nSummary:\nWe recently noticed that some memtable flushed and file\ningestions could proceed during LockWAL, in violation of its stated\ncontract. (Note: we aren't 100% sure its actually needed by MySQL, but\nwe want it to be in a clean state nonetheless.)\n\nDespite earlier skepticism that this could be done safely (https://github.com/facebook/rocksdb/issues/12666), I\nfound a place to wait to wait for LockWAL to be cleared before allowing\nthese operations to proceed: WaitForPendingWrites()\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12652\n\nTest Plan:\nAdded to unit tests. Extended how db_stress validates LockWAL\nand re-enabled combination of ingestion and LockWAL in crash test, in\nfollow-up to https://github.com/facebook/rocksdb/issues/12642\n\nRan blackbox_crash_test for a long while with relevant features\namplified.\n\nSuggested follow-up: fix FaultInjectionTestFS to report file sizes\nconsistent with what the user has requested to be flushed.\n\nReviewed By: jowlyzhang\n\nDifferential Revision: D57622142\n\nPulled By: pdillinger\n\nfbshipit-source-id: aef265fce69465618974b4ec47f4636257c676ce","shortMessageHtmlLink":"Disallow memtable flush and sst ingest while WAL is locked (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2293662772\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12652\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12652/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12652\">#12652</a>)"}},{"before":"ef1d4955ba703ebec80877a83b88a6dc47cdb398","after":"d7b938882e4e81892721d5787c446348ffb782c8","ref":"refs/heads/main","pushedAt":"2024-05-21T00:37:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Sync WAL during db Close() (#12556)\n\nSummary:\n**Context/Summary:**\nBelow crash test found out we don't sync WAL upon DB close, which can lead to unsynced data loss. This PR syncs it.\n```\n./db_stress --threads=1 --disable_auto_compactions=1 --WAL_size_limit_MB=0 --WAL_ttl_seconds=0 --acquire_snapshot_one_in=0 --adaptive_readahead=0 --adm_policy=1 --advise_random_on_open=1 --allow_concurrent_memtable_write=1 --allow_data_in_errors=True --allow_fallocate=0 --async_io=0 --auto_readahead_size=0 --avoid_flush_during_recovery=1 --avoid_flush_during_shutdown=0 --avoid_unnecessary_blocking_io=1 --backup_max_size=104857600 --backup_one_in=0 --batch_protection_bytes_per_key=0 --bgerror_resume_retry_interval=1000000 --block_align=0 --block_protection_bytes_per_key=2 --block_size=16384 --bloom_before_level=1 --bloom_bits=29.895303579352174 --bottommost_compression_type=disable --bottommost_file_compaction_delay=0 --bytes_per_sync=0 --cache_index_and_filter_blocks=0 --cache_index_and_filter_blocks_with_high_priority=1 --cache_size=33554432 --cache_type=lru_cache --charge_compression_dictionary_building_buffer=1 --charge_file_metadata=0 --charge_filter_construction=1 --charge_table_reader=1 --checkpoint_one_in=0 --checksum_type=kxxHash64 --clear_column_family_one_in=0 --column_families=1 --compact_files_one_in=0 --compact_range_one_in=0 --compaction_pri=0 --compaction_readahead_size=0 --compaction_style=0 --compaction_ttl=0 --compress_format_version=2 --compressed_secondary_cache_ratio=0 --compressed_secondary_cache_size=0 --compression_checksum=1 --compression_max_dict_buffer_bytes=0 --compression_max_dict_bytes=0 --compression_parallel_threads=4 --compression_type=zstd --compression_use_zstd_dict_trainer=1 --compression_zstd_max_train_bytes=0 --continuous_verification_interval=0 --data_block_index_type=0 --db=/dev/shm/rocksdb_test/rocksdb_crashtest_whitebox --db_write_buffer_size=0 --default_temperature=kUnknown --default_write_temperature=kUnknown --delete_obsolete_files_period_micros=0 --delpercent=0 --delrangepercent=0 --destroy_db_initially=1 --detect_filter_construct_corruption=1 --disable_wal=0 --dump_malloc_stats=0 --enable_checksum_handoff=0 --enable_compaction_filter=0 --enable_custom_split_merge=0 --enable_do_not_compress_roles=1 --enable_index_compression=1 --enable_memtable_insert_with_hint_prefix_extractor=0 --enable_pipelined_write=0 --enable_sst_partitioner_factory=0 --enable_thread_tracking=1 --enable_write_thread_adaptive_yield=0 --expected_values_dir=/dev/shm/rocksdb_test/rocksdb_crashtest_expected --fail_if_options_file_error=0 --fifo_allow_compaction=1 --file_checksum_impl=none --fill_cache=0 --flush_one_in=1000 --format_version=5 --get_current_wal_file_one_in=0 --get_live_files_one_in=0 --get_property_one_in=0 --get_sorted_wal_files_one_in=0 --hard_pending_compaction_bytes_limit=274877906944 --high_pri_pool_ratio=0 --index_block_restart_interval=6 --index_shortening=0 --index_type=0 --ingest_external_file_one_in=0 --initial_auto_readahead_size=16384 --iterpercent=0 --key_len_percent_dist=1,30,69 --last_level_temperature=kUnknown --level_compaction_dynamic_level_bytes=1 --lock_wal_one_in=0 --log2_keys_per_lock=10 --log_file_time_to_roll=0 --log_readahead_size=16777216 --long_running_snapshots=0 --low_pri_pool_ratio=0 --lowest_used_cache_tier=0 --manifest_preallocation_size=5120 --manual_wal_flush_one_in=0 --mark_for_compaction_one_file_in=0 --max_auto_readahead_size=0 --max_background_compactions=1 --max_bytes_for_level_base=67108864 --max_key=2500000 --max_key_len=3 --max_log_file_size=0 --max_manifest_file_size=1073741824 --max_sequential_skip_in_iterations=8 --max_total_wal_size=0 --max_write_batch_group_size_bytes=64 --max_write_buffer_number=10 --max_write_buffer_size_to_maintain=0 --memtable_insert_hint_per_batch=0 --memtable_max_range_deletions=0 --memtable_prefix_bloom_size_ratio=0.5 --memtable_protection_bytes_per_key=1 --memtable_whole_key_filtering=1 --memtablerep=skip_list --metadata_charge_policy=0 --min_write_buffer_number_to_merge=1 --mmap_read=0 --mock_direct_io=True --nooverwritepercent=1 --num_file_reads_for_auto_readahead=0 --num_levels=1 --open_files=-1 --open_metadata_write_fault_one_in=0 --open_read_fault_one_in=0 --open_write_fault_one_in=0 --ops_per_thread=3 --optimize_filters_for_hits=1 --optimize_filters_for_memory=1 --optimize_multiget_for_io=0 --paranoid_file_checks=0 --partition_filters=0 --partition_pinning=1 --pause_background_one_in=0 --periodic_compaction_seconds=0 --prefix_size=1 --prefixpercent=0 --prepopulate_block_cache=0 --preserve_internal_time_seconds=3600 --progress_reports=0 --read_amp_bytes_per_bit=0 --read_fault_one_in=0 --readahead_size=16384 --readpercent=0 --recycle_log_file_num=0 --reopen=2 --report_bg_io_stats=1 --sample_for_compression=5 --secondary_cache_fault_one_in=0 --secondary_cache_uri= --skip_stats_update_on_db_open=1 --snapshot_hold_ops=0 --soft_pending_compaction_bytes_limit=68719476736 --sst_file_manager_bytes_per_sec=0 --sst_file_manager_bytes_per_truncate=0 --stats_dump_period_sec=10 --stats_history_buffer_size=1048576 --strict_bytes_per_sync=0 --subcompactions=3 --sync=0 --sync_fault_injection=1 --table_cache_numshardbits=6 --target_file_size_base=16777216 --target_file_size_multiplier=1 --test_batches_snapshots=0 --top_level_index_pinning=0 --unpartitioned_pinning=3 --use_adaptive_mutex=1 --use_adaptive_mutex_lru=0 --use_delta_encoding=1 --use_direct_io_for_flush_and_compaction=0 --use_direct_reads=0 --use_full_merge_v1=0 --use_get_entity=0 --use_merge=0 --use_multi_get_entity=0 --use_multiget=1 --use_put_entity_one_in=0 --use_write_buffer_manager=0 --user_timestamp_size=0 --value_size_mult=32 --verification_only=0 --verify_checksum=1 --verify_checksum_one_in=1000 --verify_compression=0 --verify_db_one_in=100000 --verify_file_checksums_one_in=0 --verify_iterator_with_expected_state_one_in=5 --verify_sst_unique_id_in_manifest=1 --wal_bytes_per_sync=0 --wal_compression=zstd --write_buffer_size=33554432 --write_dbid_to_manifest=0 --write_fault_one_in=0 --writepercent=100\n\n Verification failed for column family 0 key 000000000000B9D1000000000000012B000000000000017D (4756691): value_from_db: , value_from_expected: 010000000504070609080B0A0D0C0F0E111013121514171619181B1A1D1C1F1E212023222524272629282B2A2D2C2F2E313033323534373639383B3A3D3C3F3E, msg: Iterator verification: Value not found: NotFound:\nVerification failed :(\n```\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12556\n\nTest Plan:\n- New UT\n- Same stress test command failed before this fix but pass after\n- CI\n\nReviewed By: ajkr\n\nDifferential Revision: D56267964\n\nPulled By: hx235\n\nfbshipit-source-id: af1b7e8769c129f64ba1c7f1ff17102f1239b929","shortMessageHtmlLink":"Sync WAL during db Close() (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2249263790\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12556\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12556/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12556\">#12556</a>)"}},{"before":"b7520f4815bdbc5eb0d160df2820bdbe15b55947","after":"ef1d4955ba703ebec80877a83b88a6dc47cdb398","ref":"refs/heads/main","pushedAt":"2024-05-21T00:09:04.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix the output of `ldb dump_wal` for PutEntity records (#12677)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12677\n\nThe patch contains two fixes related to printing `PutEntity` records with `ldb dump_wal`:\n1) It adds the key to the printout (it was missing earlier).\n2) It restores the formatting flags of the output stream after dumping the wide-column structure so that any `hex` flag that might have been set does not affect subsequent printing of e.g. sequence numbers.\n\nReviewed By: jaykorean, jowlyzhang\n\nDifferential Revision: D57591295\n\nfbshipit-source-id: af4e3e219f0082ad39bbdfd26f8c5a57ebb898be","shortMessageHtmlLink":"Fix the output of <code>ldb dump_wal</code> for PutEntity records (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2306903783\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12677\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12677/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12677\">#12677</a>)"}},{"before":"35985a988c9fac5fd1f480af0aefd4738af994d4","after":"b7520f4815bdbc5eb0d160df2820bdbe15b55947","ref":"refs/heads/main","pushedAt":"2024-05-20T23:13:42.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Support building ldb with buck (#12676)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12676\n\nThe patch extends the RocksDB buckifier script so it also creates a `buck` target for the `ldb` tool and updates the `TARGETS` file with the results of the new version of the script.\n\nReviewed By: cbi42\n\nDifferential Revision: D57588789\n\nfbshipit-source-id: 2ed58b405b3f216e802cf6bcbdbf9809e7386c8b","shortMessageHtmlLink":"Support building ldb with buck (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2306851078\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12676\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12676/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12676\">#12676</a>)"}},{"before":"c87f5cf91c17407fb4a67ebe09213bb26eaaa572","after":"35985a988c9fac5fd1f480af0aefd4738af994d4","ref":"refs/heads/main","pushedAt":"2024-05-20T20:26:17.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix value of `inplace_update_support` across stress test runs (#12675)\n\nSummary:\nthe value of `inplace_update_support` option need to be fixed across runs of db_stress on the same DB (https://github.com/facebook/rocksdb/issues/12577). My recent fix (https://github.com/facebook/rocksdb/issues/12673) regressed this behavior. Also fix some existing places where this does not hold.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12675\n\nTest Plan: monitor crash tests related to `inplace_update_support`.\n\nReviewed By: hx235\n\nDifferential Revision: D57576375\n\nPulled By: cbi42\n\nfbshipit-source-id: 75b1bd233f03e5657984f5d5234dbbb1ffc35c27","shortMessageHtmlLink":"Fix value of <code>inplace_update_support</code> across stress test runs (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2306480649\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12675\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12675/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12675\">#12675</a>)"}},{"before":"f910a0c025bf9c415c6906aede786697d1a987da","after":"c87f5cf91c17407fb4a67ebe09213bb26eaaa572","ref":"refs/heads/main","pushedAt":"2024-05-20T17:46:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add GetEntityForUpdate to optimistic and WriteCommitted pessimistic transactions (#12668)\n\nSummary:\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12668\n\nThe patch adds a new `GetEntityForUpdate` API to optimistic and WriteCommitted pessimistic transactions, which provides transactional wide-column point lookup functionality with concurrency control. For WriteCommitted transactions, user-defined timestamps are also supported similarly to the `GetForUpdate` API.\n\nReviewed By: jaykorean\n\nDifferential Revision: D57458304\n\nfbshipit-source-id: 7eadbac531ca5446353e494abbd0635d63f62d24","shortMessageHtmlLink":"Add GetEntityForUpdate to optimistic and WriteCommitted pessimistic t…"}},{"before":"4dd084f66dcfc747d6a42ebc8cfc576328078f45","after":"f910a0c025bf9c415c6906aede786697d1a987da","ref":"refs/heads/main","pushedAt":"2024-05-20T16:45:04.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix unreleased bug fix .md name (#12672)\n\nSummary:\nContext/Summary: as above\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12672\n\nTest Plan: no code change\n\nReviewed By: ajkr\n\nDifferential Revision: D57505136\n\nPulled By: hx235\n\nfbshipit-source-id: 0e216dc5974e9be10027b444eb6b4034f679dfd8","shortMessageHtmlLink":"Fix unreleased bug fix .md name (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2303466273\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12672\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12672/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12672\">#12672</a>)"}},{"before":"c4782bde41fa368de3b9c63570357c8063f6e88e","after":"4dd084f66dcfc747d6a42ebc8cfc576328078f45","ref":"refs/heads/main","pushedAt":"2024-05-19T01:05:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"fix gcc warning about dangling-reference in backup_engine_test (#12637)\n\nSummary:\ngcc 14.1 reports some warnings about dangling-reference occured in backup_engine_test.\n```c++\n/data/rocksdb/utilities/backup/backup_engine_test.cc: In member function 'virtual void rocksdb::{anonymous}::BackupEngineTest_ExcludeFiles_Test::TestBody()':\n/data/rocksdb/utilities/backup/backup_engine_test.cc:4411:64: error: possibly dangling reference to a temporary [-Werror=dangling-reference]\n 4411 |         std::make_pair(alt_backup_engine, backup_engine_.get())}) {\n      |                                                                ^\n/data/rocksdb/utilities/backup/backup_engine_test.cc:4410:23: note: the temporary was destroyed at the end of the full expression 'std::make_pair<rocksdb::BackupEngine*, rocksdb::BackupEngine*&>(((rocksdb::{anonymous}::BackupEngineTest_ExcludeFiles_Test*)this)->rocksdb::{anonymous}::BackupEngineTest_ExcludeFiles_Test::rocksdb::{anonymous}::BackupEngineTest.rocksdb::{anonymous}::BackupEngineTest::backup_engine_.std::unique_ptr<rocksdb::BackupEngine>::get(), alt_backup_engine)'\n 4410 |        {std::make_pair(backup_engine_.get(), alt_backup_engine),\n      |         ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n/data/rocksdb/utilities/backup/backup_engine_test.cc:4411:64: error: possibly dangling reference to a temporary [-Werror=dangling-reference]\n 4411 |         std::make_pair(alt_backup_engine, backup_engine_.get())}) {\n      |                                                                ^\n/data/rocksdb/utilities/backup/backup_engine_test.cc:4411:23: note: the temporary was destroyed at the end of the full expression 'std::make_pair<rocksdb::BackupEngine*&, rocksdb::BackupEngine*>(alt_backup_engine, ((rocksdb::{anonymous}::BackupEngineTest_ExcludeFiles_Test*)this)->rocksdb::{anonymous}::BackupEngineTest_ExcludeFiles_Test::rocksdb::{anonymous}::BackupEngineTest.rocksdb::{anonymous}::BackupEngineTest::backup_engine_.std::unique_ptr<rocksdb::BackupEngine>::get())'\n 4411 |         std::make_pair(alt_backup_engine, backup_engine_.get())}) {\n      |         ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n```\nIt seems to be related to this update in gcc:\nhttps://gcc.gnu.org/gcc-14/changes.html#:~:text=%2DWdangling%2Dreference%20false%20positives%20have%20been%20reduced.%20The%20warning%20does%20not%20warn%20about%20std%3A%3Aspan%2Dlike%20classes%3B%20there%20is%20also%20a%20new%20attribute%20gnu%3A%3Ano_dangling%20to%20suppress%20the%20warning.%20See%20the%20manual%20for%20more%20info.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12637\n\nReviewed By: cbi42\n\nDifferential Revision: D57263996\n\nPulled By: ajkr\n\nfbshipit-source-id: 1e416c38240d3d1adda787fc484c0392e28bb7f1","shortMessageHtmlLink":"fix gcc warning about dangling-reference in backup_engine_test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2289799709\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12637\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12637/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12637\">#12637</a>)"}},{"before":"0ed93552f4cd6004e966815e1c18347e01628830","after":"c4782bde41fa368de3b9c63570357c8063f6e88e","ref":"refs/heads/main","pushedAt":"2024-05-18T23:51:46.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Disable `inplace_update_support` in crash test with unsynced data loss (#12673)\n\nSummary:\nWith unsynced data loss, we replay traces to recover expected state to DB's latest sequence number. With `inplace_update_support`, the largest sequence number of memtable may not reflect the latest update. This is because inplace updates in memtable do not update sequence number. So we disable `inplace_update_support` where traces need to be replayed.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12673\n\nReviewed By: ltamasi\n\nDifferential Revision: D57512548\n\nPulled By: cbi42\n\nfbshipit-source-id: 69278fe2e935874faf744d0ac4fd85263773c3ec","shortMessageHtmlLink":"Disable <code>inplace_update_support</code> in crash test with unsynced data loss ("}},{"before":"ffd7930312dffc6fe7a29e05c4f5870db23869b8","after":"0ed93552f4cd6004e966815e1c18347e01628830","ref":"refs/heads/main","pushedAt":"2024-05-18T02:17:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Implement obsolete file deletion (GC) in follower (#12657)\n\nSummary:\nThis PR implements deletion of obsolete files in a follower RocksDB instance. The follower tails the leader's MANIFEST and creates links to newly added SST files. These links need to be deleted once those files become obsolete in order to reclaim space. There are three cases to be considered -\n1. New files added and links created, but the Version could not be installed due to some missing files. Those links need to be preserved so a subsequent catch up attempt can succeed. We insert the next file number in the `VersionSet` to `pending_outputs_` to prevent their deletion.\n2. Files deleted from the previous successfully installed `Version`. These are deleted as usual in `PurgeObsoleteFiles`.\n3. New files added by a `VersionEdit` and deleted by a subsequent `VersionEdit`, both processed in the same catchup attempt. Links will be created for the new files when verifying a candidate `Version`. Those need to be deleted explicitly as they're never added to `VersionStorageInfo`, and thus not deleted by `PurgeObsoleteFiles`.\n\nTest plan -\nNew unit tests in `db_follower_test`.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12657\n\nReviewed By: jowlyzhang\n\nDifferential Revision: D57462697\n\nPulled By: anand1976\n\nfbshipit-source-id: 898f15570638dd4930f839ffd31c560f9cb73916","shortMessageHtmlLink":"Implement obsolete file deletion (GC) in follower (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2296567894\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12657\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12657/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12657\">#12657</a>)"}},{"before":"131c8ccfcd48c3a869782e29665a51ae5253e255","after":"ffd7930312dffc6fe7a29e05c4f5870db23869b8","ref":"refs/heads/main","pushedAt":"2024-05-17T00:25:43.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add more debug print to `DBTestWithParam.ThreadStatusSingleCompaction` (#12661)\n\nSummary:\nThis test is flaky and a recent failure prints the following:\n```\n[ RUN      ] DBTestWithParam/DBTestWithParam.ThreadStatusSingleCompaction/0\nthread id: 1842811, thread status:\nthread id: 1842803, thread status:\ndb/db_test.cc:4697: Failure\nExpected equality of these values:\n  op_count\n    Which is: 0\n  expected_count\n    Which is: 1\n[  FAILED  ] DBTestWithParam/DBTestWithParam.ThreadStatusSingleCompaction/0, where GetParam() = (1, false) (307 ms)\n```\nEmpty thread status implies that operation_type of the threads are all OP_UNKNOWN. From https://github.com/facebook/rocksdb/blob/3ed46e0668f840bea490e29beeac7777c50ae8fb/monitoring/thread_status_updater.cc#L197, this can be due to thread_data->operation_type being OP_UNKNOWN or that thread_data->cf_key it not in `cf_info_map_`, potentially due to how cf_key_ is accessed with relaxed memory order. This PR adds some debug print to print the cf_name to check this.\n\nThis PR also prints num_running_compaction and lsm state to check if a compaction is indeed running, and removes some not needed options and ensures that exactly 4 L0 files are created.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12661\n\nTest Plan:\n- Cannot repro the failure locally: `gtest-parallel --repeat=10000 --workers=200 ./db_test --gtest_filter=\"*ThreadStatusSingleCompaction*\"`\n- New failure message will look like:\n```\n[ RUN      ] DBTestWithParam/DBTestWithParam.ThreadStatusSingleCompaction/0\nop_count: 1, expected_count 2\nthread id: 6104100864, thread status: , cf_name\nthread id: 6103527424, thread status: Compaction, cf_name default\nrunning compaction: 1 lsm state: 4\ndb/db_test.cc:4885: Failure\nValue of: match\n  Actual: false\nExpected: true\n[  FAILED  ] DBTestWithParam/DBTestWithParam.ThreadStatusSingleCompaction/0, where GetParam() = (1, false) (115 ms)\n```\n\nReviewed By: hx235\n\nDifferential Revision: D57422755\n\nPulled By: cbi42\n\nfbshipit-source-id: 635663f26052b20e485dfa06a7c0f1f318ac1099","shortMessageHtmlLink":"Add more debug print to <code>DBTestWithParam.ThreadStatusSingleCompaction</code> ("}},{"before":"2eb404de13499e2c7611ef2cc82e2feca742debf","after":"131c8ccfcd48c3a869782e29665a51ae5253e255","ref":"refs/heads/main","pushedAt":"2024-05-16T22:22:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add EntryType for TimedPut (#12669)\n\nSummary:\nRepresent internal kTypeValuePreferredSeqno in the public API as kEntryTimedPut (because it is created by TimedPut, until the entry can be safely converted to a regular value entry in compaction)\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12669\n\nTest Plan: for follow-up work actually using it. But putting this in place in the public API gives us more flexibility in rolling out that follow-up work (e.g. as a user extension or patch if needed).\n\nReviewed By: jowlyzhang\n\nDifferential Revision: D57459637\n\nPulled By: pdillinger\n\nfbshipit-source-id: 160ccf7c4e524ee479558846b2a207d51b8b3d9c","shortMessageHtmlLink":"Add EntryType for TimedPut (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2301306730\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12669\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12669/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12669\">#12669</a>)"}},{"before":"4eaf6281202c87fa0c09c07c0ca197a70004fd66","after":"2eb404de13499e2c7611ef2cc82e2feca742debf","ref":"refs/heads/main","pushedAt":"2024-05-16T04:17:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Print non-ok status for multi_ops_txns_stress test (#12660)\n\nSummary:\nCurrently `assert(s.ok())` does not offer much information to debug.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12660\n\nTest Plan:\nrevert https://github.com/facebook/rocksdb/issues/12639 and run `python3 ./tools/db_crashtest.py blackbox --write_policy write_prepared --recycle_log_file_num=1 --threads=1 --test_multiops_txn`\n\nbefore this PR:\n```\nExit Before Killing\nstdout:\n\nstderr:\n db_stress: db_stress_tool/multi_ops_txns_stress.cc:1529: void rocksdb::MultiOpsTxnsStressTest::PreloadDb(rocksdb::SharedState*, int, uint32_t, uint32_t, uint32_t, uint32_t): Assertion `s.ok()' failed.\n```\n\nafter this PR:\n```\nExit Before Killing\nstdout:\n\nstderr:\n Verification failed: PreloadDB failed: Invalid argument: WriteOptions::disableWAL option is not supported if DBOptions::recycle_log_file_num > 0\ndb_stress: db_stress_tool/db_stress_test_base.cc:517: void rocksdb::StressTest::ProcessStatus(rocksdb::SharedState*, std::string, rocksdb::Status, bool) const: Assertion `false' failed.\nReceived signal 6 (Aborted)\n```\n\nReviewed By: ajkr\n\nDifferential Revision: D57410819\n\nPulled By: cbi42\n\nfbshipit-source-id: a03c2202c3fd666eb2f58bae24e0c9e3e6ed4265","shortMessageHtmlLink":"Print non-ok status for multi_ops_txns_stress test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2298862563\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12660\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12660/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12660\">#12660</a>)"}},{"before":"3ed46e0668f840bea490e29beeac7777c50ae8fb","after":"4eaf6281202c87fa0c09c07c0ca197a70004fd66","ref":"refs/heads/main","pushedAt":"2024-05-16T02:16:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add `Iterator` property \"rocksdb.iterator.is-value-pinned\" (#12659)\n\nSummary:\n`ReadOptions::pin_data` already has the effect of pinning the `Slice` returned by `Iterator::value()` when the value is stored inline (e.g., `kTypeValue`). This PR adds a bit of visibility into that via a new `Iterator` property, \"rocksdb.iterator.is-value-pinned\", as well as some documentation and tests.\n\nSee also: https://github.com/facebook/rocksdb/issues/12658\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12659\n\nReviewed By: cbi42\n\nDifferential Revision: D57391200\n\nPulled By: ajkr\n\nfbshipit-source-id: 0caa8db27ca1aba86ee2addc3dfd6f0e003d32e2","shortMessageHtmlLink":"Add <code>Iterator</code> property \"rocksdb.iterator.is-value-pinned\" (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2298466539\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12659\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12659/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12659\">#12659</a>)"}},{"before":"b4c6956a5933792267fa2b9ac7ba7c95117abdfc","after":"3ed46e0668f840bea490e29beeac7777c50ae8fb","ref":"refs/heads/main","pushedAt":"2024-05-14T23:48:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Handle early exit in DBErrorHandlingFSTests (#12655)\n\nSummary:\nTo avoid use-after-free on custom env on ASSERT_WHATEVER failure.\n\nThis is motivated by a rare crash seen in DBErrorHandlingFSTest.WALWriteError (VersionSet::GetObsoleteFiles in a SstFileManagerImpl::ClearError thread) and wanting to rule out this being related to that.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12655\n\nTest Plan: manually seeing ASSERT_WHATEVER failures, especially under ASAN\n\nReviewed By: cbi42\n\nDifferential Revision: D57358202\n\nPulled By: pdillinger\n\nfbshipit-source-id: 4da2a0d73a54380b257e5cc1ab6c666e26b83973","shortMessageHtmlLink":"Handle early exit in DBErrorHandlingFSTests (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2296499043\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12655\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12655/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12655\">#12655</a>)"}},{"before":"b9fc13db69ebddc746a0a0ebad7f6cf37144a0cb","after":"b4c6956a5933792267fa2b9ac7ba7c95117abdfc","ref":"refs/heads/main","pushedAt":"2024-05-14T23:37:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add MultiGetEntity AttributeGroup API to stress test (#12640)\n\nSummary:\nContinuing from https://github.com/facebook/rocksdb/pull/12605, adding AttributeGroup `MultiGetEntity` API to stress tests.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12640\n\nTest Plan:\n**AttributeGroup Tests**\n\nNonBatchOps\n```\npython3 tools/db_crashtest.py blackbox --simple --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nBatchOps\n```\npython3 tools/db_crashtest.py blackbox  --test_batches_snapshots=1 --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nCfConsistency Test\n```\npython3 tools/db_crashtest.py blackbox --cf_consistency --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=1 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\n**Non-AttributeGroup Tests**\n\nNonBatchOps\n```\npython3 tools/db_crashtest.py blackbox --simple --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=0 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nBatchOps\n```\npython3 tools/db_crashtest.py blackbox  --test_batches_snapshots=1 --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=0 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nCfConsistency Test\n```\npython3 tools/db_crashtest.py blackbox --cf_consistency --max_key=25000000 --write_buffer_size=4194304 --use_attribute_group=0 --use_put_entity_one_in=1 --use_multi_get=1\n```\n\nReviewed By: ltamasi\n\nDifferential Revision: D57233931\n\nPulled By: jaykorean\n\nfbshipit-source-id: 8cea771ac2e5749050bf5319360c6c5aa476d7d5","shortMessageHtmlLink":"Add MultiGetEntity AttributeGroup API to stress test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2290331088\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12640\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12640/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12640\">#12640</a>)"}},{"before":"1567d50a2754753cd80b1c2e56b862673ad09bbd","after":"b9fc13db69ebddc746a0a0ebad7f6cf37144a0cb","ref":"refs/heads/main","pushedAt":"2024-05-14T22:57:15.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Add padding before timestamp size record if it doesn't fit into a WAL block. (#12614)\n\nSummary:\nIf timestamp size record doesn't fit into a block, without padding `Writer::EmitPhysicalRecord` fails on assert (either `assert(block_offset_ + kHeaderSize + n <= kBlockSize);` or `assert(block_offset_ + kRecyclableHeaderSize + n <= kBlockSize)`, depending on whether recycling log files is enabled)  in debug build. In release, current block grows beyond 32K, `block_offset_` gets reset on next `AddRecord` and all the subsequent blocks are no longer aligned by block size.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12614\n\nReviewed By: ltamasi\n\nDifferential Revision: D57302140\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: cacb5cefb7586885e52a8137ae23a95e1aefca2d","shortMessageHtmlLink":"Add padding before timestamp size record if it doesn't fit into a WAL…"}},{"before":"c110091d368b8a01b5be36a14198769e60786c05","after":"1567d50a2754753cd80b1c2e56b862673ad09bbd","ref":"refs/heads/main","pushedAt":"2024-05-14T16:32:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Temporarily disable file ingestion if LockWAL is tested (#12642)\n\nSummary:\nAs titled. A proper fix should probably be failing file ingestion if the DB is in a lock wal state as it promises to \"Freezes the logical state of the DB\".\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12642\n\nReviewed By: pdillinger\n\nDifferential Revision: D57235869\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: c70031463842220f865621eb6f53424df27d29e9","shortMessageHtmlLink":"Temporarily disable file ingestion if LockWAL is tested (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2290515809\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12642\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12642/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12642\">#12642</a>)"}},{"before":"20213d01a333aca20182e5aab6e4513537d9758c","after":"c110091d368b8a01b5be36a14198769e60786c05","ref":"refs/heads/main","pushedAt":"2024-05-13T22:46:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Support read timestamp in ldb (#12641)\n\nSummary:\nAs titled. Also updated sst_dump to print out user-defined timestamp separately and in human readable format.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12641\n\nTest Plan:\nmanually tested:\nExample success run:\n./ldb --db=$TEST_DB multi_get_entity 0x00000000000000950000000000000120 --key_hex --column_family=7  --read_timestamp=1115613683797942 --value_hex\n0x00000000000000950000000000000120 ==> :0x0E0000000A0B080906070405020300011E1F1C1D1A1B181916171415121310112E2F2C2D2A2B282926272425222320213E3F3C3D3A3B383936373435323330314E4F4C4D4A4B484946474445424340415E5F5C5D5A5B58595657545552535051\nExample failed run:\nFailed: GetEntity failed: Invalid argument: column family enables user-defined timestamp while --read_timestamp is not a valid uint64 value.\n\nsst_dump print out:\n'000000000000015D000000000000012B000000000000013B|timestamp:1113554493256671' seq:2330405, type:1 => 010000000504070609080B0A0D0C0F0E111013121514171619181B1A1D1C1F1E212023222524272629282B2A2D2C2F2E313033323534373639383B3A3D3C3F3E\n\nReviewed By: ltamasi\n\nDifferential Revision: D57297006\n\nPulled By: jowlyzhang\n\nfbshipit-source-id: 8486d91468e4f6c0d42dca3c9629f1c45a92bf5a","shortMessageHtmlLink":"Support read timestamp in ldb (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2290486271\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12641\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12641/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12641\">#12641</a>)"}},{"before":"7747abdc151be4227a3885379403bfd7a268208b","after":"20213d01a333aca20182e5aab6e4513537d9758c","ref":"refs/heads/main","pushedAt":"2024-05-13T20:16:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Fix crash in CompactFiles() of conflict range under `preclude_last_level_data_seconds > 0` (#12628)\n\nSummary:\n**Context/Summary:**\n\nPreviously `CompactFiles()` used `RangeOverlapWithCompaction()` to check for conflict when sanitizing input files while later used `FilesRangeOverlapWithCompaction()` to assert for no conflict. The latter function checks for more conflict scenarios than the former does, particularly the ones arising from `preclude_last_level_data_seconds > 0` (i.e, compaction can output to second-to-the-last level). So we ran into assertion violation in `CompactFiles()` like below\n```\n Assertion `output_level == 0 || !FilesRangeOverlapWithCompaction( input_files, output_level, Compaction::EvaluatePenultimateLevel(vstorage, ioptions_, start_level, output_level))' failed.\n```\n\nThis PR make `CompactFiles()` used `FilesRangeOverlapWithCompaction()` and return Aborted status upon range conflict instead of crashing (during debug build) or proceed incorrectly (during non-debug build). To do so cleanly, I included a refactoring to make `FilesRangeOverlapWithCompaction()` part of `SanitizeAndConvertCompactionInputFiles()`, replacing `RangeOverlapWithCompaction()`.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12628\n\nTest Plan: New UT crashed before the fix and return correct status after the fix.\n\nReviewed By: cbi42\n\nDifferential Revision: D57123536\n\nPulled By: hx235\n\nfbshipit-source-id: f963a2c9e7ba1a9927a67fcc87f0dce126d3a430","shortMessageHtmlLink":"Fix crash in CompactFiles() of conflict range under `preclude_last_le…"}},{"before":"b75438f9860e3cff5e713917ed22e0ac394a758c","after":"7747abdc151be4227a3885379403bfd7a268208b","ref":"refs/heads/main","pushedAt":"2024-05-13T19:46:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Disable PromoteL0 in crash test (#12651)\n\nSummary:\nSeeing way too many errors likely related to PromoteL0 from https://github.com/facebook/rocksdb/issues/12617, containing\n```\nCannot delete table file #N from level 0 since it is on level X\n```\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12651\n\nTest Plan: watch crash test results\n\nReviewed By: hx235\n\nDifferential Revision: D57286208\n\nPulled By: pdillinger\n\nfbshipit-source-id: f7f0560cc0804ca297373c8d20ebc34986cc19d0","shortMessageHtmlLink":"Disable PromoteL0 in crash test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2293473262\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12651\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12651/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12651\">#12651</a>)"}},{"before":"7d9642d8761725cbc02a6bd6e8011a2865ec4d00","after":"b75438f9860e3cff5e713917ed22e0ac394a758c","ref":"refs/heads/main","pushedAt":"2024-05-11T00:59:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"facebook-github-bot","name":"Facebook Community Bot","path":"/facebook-github-bot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6422482?s=80&v=4"},"commit":{"message":"Allow disableWAL+recycle with WritePreparedTxnDB internals (#12639)\n\nSummary:\nFollow-up from https://github.com/facebook/rocksdb/issues/12403\n\nThe crash test was periodically failing with the\n\"disableWAL option is not supported if recycle_log_file_num > 0\" failure, despite not setting the disableWAL from the user side.\n\nPull Request resolved: https://github.com/facebook/rocksdb/pull/12639\n\nTest Plan: db_stress reproducer now passes. Added WAL recycling to txn DB unit tests, which is generally more difficult for correctness. Many tests now cover this change and pass.\n\nReviewed By: anand1976\n\nDifferential Revision: D57227617\n\nPulled By: pdillinger\n\nfbshipit-source-id: db9abefeb505bce624b45bc64009694d2a5baed9","shortMessageHtmlLink":"Allow disableWAL+recycle with WritePreparedTxnDB internals (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2290310813\" data-permission-text=\"Title is private\" data-url=\"https://github.com/facebook/rocksdb/issues/12639\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/facebook/rocksdb/pull/12639/hovercard\" href=\"https://github.com/facebook/rocksdb/pull/12639\">#12639</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEUW59owA","startCursor":null,"endCursor":null}},"title":"Activity · facebook/rocksdb"}