Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scylla crash when reading from mutation fragments with token() filtering #18637

Closed
xemul opened this issue May 13, 2024 · 1 comment
Closed
Labels
Backport candidate backport/5.4 Issues that should be backported to 5.4 branch once they'll be fixed

Comments

@xemul
Copy link
Contributor

xemul commented May 13, 2024

c8cc47d

cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 'replication_factor': 1};
cqlsh> CREATE TABLE test.test (pk int PRIMARY KEY, c int);
cqlsh> insert into test.test (pk, c) values (0, 0);
cqlsh> select * from mutation_fragments(test.test) where token(pk) <= -1;
OperationTimedOut: errors={'127.0.0.1:9042': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042
Segmentation fault on shard 0.
Backtrace:
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x3f040c
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x41c705
  /lib64/libc.so.6+0x3dbaf
  0x1b09502
  0x1b0945d
  0x1aefba8
  0x1ae69ca
  0x1ad5168
  0x1ad9d83
  0x1ad4773
  0x1ad754e
  0x1ad0723
  0x1ee7917
  0x1bf4614
  0x1bc9a65
  0x1bc7be7
  0x1b5b6db
  0x1ead1c9
  0x1eacaf2
  0x19ba2f1
  0x19d30d0
  0x19b9ecf
  0x19b53d9
  0x19e6862
  0x19e870b
  0x19e859a
  0x19e82a5
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x37bce2
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x3ff22f
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x4003b7
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x3ff888
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x36deb8
  /home/xemul/src/scylla/build/dev/seastar/libseastar.so+0x36d0f2
  0x101677a
  0x1053b20
  0x1015667
  /lib64/libc.so.6+0x27b89
  /lib64/libc.so.6+0x27c4a
  0x1013fe4
Segmentation fault (core dumped)

decoded

[Backtrace #0]
/home/xemul/src/scylla/build/dev/seastar/libseastar.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=78a46b5f2304bb78779b580d270bbc1a66cc09fc, not stripped

seastar::print_with_backtrace(seastar::backtrace_buffer&, bool) at reactor.cc:?
seastar::install_oneshot_signal_handler<11, (void (*)())(&seastar::sigsegv_action)>()::{lambda(int, siginfo_t*, void*)#1}::__invoke(int, siginfo_t*, void*) at reactor.cc:?
/lib64/libc.so.6: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=9148cab1b932d44ef70e306e9c02ee38d06cad51, for GNU/Linux 3.2.0, not stripped

__GI___sigaction at :?
cql3::functions::token_fct::token_fct(seastar::lw_shared_ptr<schema const>) at functions.cc:?
seastar::shared_ptr_count_for<cql3::functions::token_fct>::shared_ptr_count_for<seastar::lw_shared_ptr<schema const>&>(seastar::lw_shared_ptr<schema const>&) at functions.cc:?
cql3::functions::functions::get(data_dictionary::database, seastar::basic_sstring<char, unsigned int, 15u, true> const&, db::functions::function_name const&, std::vector<seastar::shared_ptr<cql3::assignment_testable>, std::allocator<seastar::shared_ptr<cql3::assignment_testable> > > const&, seastar::basic_sstring<char, unsigned int, 15u, true> const&, std::optional<std::basic_string_view<char, std::char_traits<char> > const>, cql3::column_specification const*) at functions.cc:?
seastar::shared_ptr<db::functions::function> cql3::functions::functions::get<std::vector<seastar::shared_ptr<cql3::assignment_testable>, std::allocator<seastar::shared_ptr<cql3::assignment_testable> > >&>(data_dictionary::database, seastar::basic_sstring<char, unsigned int, 15u, true> const&, db::functions::function_name const&, std::vector<seastar::shared_ptr<cql3::assignment_testable>, std::allocator<seastar::shared_ptr<cql3::assignment_testable> > >&, seastar::basic_sstring<char, unsigned int, 15u, true> const&, std::optional<std::basic_string_view<char, std::char_traits<char> > const>, cql3::column_specification const*) at prepare_expr.cc:?
cql3::expr::prepare_function_call(cql3::expr::function_call const&, data_dictionary::database, seastar::basic_sstring<char, unsigned int, 15u, true> const&, schema const*, seastar::lw_shared_ptr<cql3::column_specification>) at prepare_expr.cc:?
_ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultISt8optionalIN4cql34expr10expressionEEEEO18overloaded_functorIJZNS6_22try_prepare_expressionERKS7_N15data_dictionary8databaseERKN7seastar13basic_sstringIcjLj15ELb1EEEPK6schemaNSF_13lw_shared_ptrINS5_20column_specificationEEEE3$_0ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_1ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_2ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_3ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_4ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_5ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_6ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_7ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_8ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E3$_9ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E4$_10ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E4$_11ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E4$_12ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E4$_13ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E4$_14ZNS6_22try_prepare_expressionESC_SE_SJ_SM_SP_E4$_15EERSt7variantIJNS6_11conjunctionENS6_15binary_operatorENS6_12column_valueENS6_21unresolved_identifierENS6_25column_mutation_attributeENS6_13function_callENS6_4castENS6_15field_selectionENS6_13bind_variableENS6_16untyped_constantENS6_8constantENS6_17tuple_constructorENS6_22collection_constructorENS6_20usertype_constructorENS6_9subscriptENS6_9temporaryEEEEJEEESt16integer_sequenceImJLm5EEEE14__visit_invokeES17_S1Q_ at prepare_expr.cc:?
cql3::expr::try_prepare_expression(cql3::expr::expression const&, data_dictionary::database, seastar::basic_sstring<char, unsigned int, 15u, true> const&, schema const*, seastar::lw_shared_ptr<cql3::column_specification>) at prepare_expr.cc:?
cql3::expr::prepare_binary_operator(cql3::expr::binary_operator, data_dictionary::database, schema const&) at prepare_expr.cc:?
cql3::expr::validate_and_prepare_new_restriction(cql3::expr::binary_operator const&, data_dictionary::database, seastar::lw_shared_ptr<schema const>, cql3::prepare_context&) at restrictions.cc:?
cql3::restrictions::statement_restrictions::statement_restrictions(data_dictionary::database, seastar::lw_shared_ptr<schema const>, cql3::statements::statement_type, cql3::expr::expression const&, cql3::prepare_context&, bool, bool, bool, seastar::bool_class<cql3::restrictions::check_indexes_tag>) at statement_restrictions.cc:?
seastar::shared_ptr_count_for<cql3::restrictions::statement_restrictions>::shared_ptr_count_for<data_dictionary::database&, seastar::lw_shared_ptr<schema const>&, cql3::statements::statement_type const&, cql3::expr::expression&, cql3::prepare_context&, bool, bool&, bool&, seastar::bool_class<cql3::restrictions::check_indexes_tag>&>(data_dictionary::database&, seastar::lw_shared_ptr<schema const>&, cql3::statements::statement_type const&, cql3::expr::expression&, cql3::prepare_context&, bool&&, bool&, bool&, seastar::bool_class<cql3::restrictions::check_indexes_tag>&) at select_statement.cc:?
cql3::statements::raw::select_statement::prepare_restrictions(data_dictionary::database, seastar::lw_shared_ptr<schema const>, cql3::prepare_context&, seastar::shared_ptr<cql3::selection::selection>, bool, bool, seastar::bool_class<cql3::restrictions::check_indexes_tag>) at select_statement.cc:?
cql3::statements::raw::select_statement::prepare(data_dictionary::database, cql3::cql_stats&, bool) at select_statement.cc:?
cql3::statements::raw::select_statement::prepare(data_dictionary::database, cql3::cql_stats&) at create_view_statement.cc:?
cql3::query_processor::get_statement(std::basic_string_view<char, std::char_traits<char> > const&, service::client_state const&) at query_processor.cc:?
cql3::query_processor::execute_direct_without_checking_exception_message(std::basic_string_view<char, std::char_traits<char> > const&, service::query_state&, cql3::query_options&) at query_processor.cc:?
cql_transport::process_query_internal(service::client_state&, seastar::sharded<cql3::query_processor>&, cql_transport::request_reader, unsigned short, unsigned char, service_permit, tracing::trace_state_ptr, bool, std::unordered_map<unsigned char, std::optional<seastar::basic_sstring<signed char, unsigned int, 31u, false> >, std::hash<unsigned char>, std::equal_to<unsigned char>, std::allocator<std::pair<unsigned char const, std::optional<seastar::basic_sstring<signed char, unsigned int, 31u, false> > > > >) at server.cc:?
seastar::future<boost::outcome_v2::basic_result<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > >, utils::exception_container<exceptions::mutation_write_timeout_exception, exceptions::read_timeout_exception, exceptions::read_failure_exception, exceptions::rate_limit_exception>, utils::exception_container_throw_policy> > cql_transport::cql_server::connection::process<seastar::future<std::variant<boost::outcome_v2::basic_result<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > >, utils::exception_container<exceptions::mutation_write_timeout_exception, exceptions::read_timeout_exception, exceptions::read_failure_exception, exceptions::rate_limit_exception>, utils::exception_container_throw_policy>, seastar::shared_ptr<cql_transport::messages::result_message::bounce_to_shard> > > (*)(service::client_state&, seastar::sharded<cql3::query_processor>&, cql_transport::request_reader, unsigned short, unsigned char, service_permit, tracing::trace_state_ptr, bool, std::unordered_map<unsigned char, std::optional<seastar::basic_sstring<signed char, unsigned int, 31u, false> >, std::hash<unsigned char>, std::equal_to<unsigned char>, std::allocator<std::pair<unsigned char const, std::optional<seastar::basic_sstring<signed char, unsigned int, 31u, false> > > > >)>(unsigned short, cql_transport::request_reader, service::client_state&, service_permit, tracing::trace_state_ptr, seastar::future<std::variant<boost::outcome_v2::basic_result<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > >, utils::exception_container<exceptions::mutation_write_timeout_exception, exceptions::read_timeout_exception, exceptions::read_failure_exception, exceptions::rate_limit_exception>, utils::exception_container_throw_policy>, seastar::shared_ptr<cql_transport::messages::result_message::bounce_to_shard> > > (*)(service::client_state&, seastar::sharded<cql3::query_processor>&, cql_transport::request_reader, unsigned short, unsigned char, service_permit, tracing::trace_state_ptr, bool, std::unordered_map<unsigned char, std::optional<seastar::basic_sstring<signed char, unsigned int, 31u, false> >, std::hash<unsigned char>, std::equal_to<unsigned char>, std::allocator<std::pair<unsigned char const, std::optional<seastar::basic_sstring<signed char, unsigned int, 31u, false> > > > >)) at server.cc:?
cql_transport::cql_server::connection::process_query(unsigned short, cql_transport::request_reader, service::client_state&, service_permit, tracing::trace_state_ptr) at server.cc:?
cql_transport::cql_server::connection::process_request_one(fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit) at server.cc:?
seastar::noncopyable_function<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)>::direct_vtable_for<std::_Mem_fn<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection::*)(fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)> >::call(seastar::noncopyable_function<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)> const*, cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit) at server.cc:?
seastar::noncopyable_function<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)>::direct_vtable_for<seastar::inheriting_concrete_execution_stage<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > >, cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit>::make_stage_for_group(seastar::scheduling_group)::{lambda(cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)#1}>::call(seastar::noncopyable_function<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)> const*, cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit) at server.cc:?
seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > std::__invoke_impl<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > >, seastar::noncopyable_function<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)>&, cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit>(std::__invoke_other, seastar::noncopyable_function<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > > (cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit)>&, cql_transport::cql_server::connection*&&, fragmented_temporary_buffer::istream&&, unsigned char&&, unsigned short&&, service::client_state&, cql_transport::cql_server::connection::tracing_request_type&&, service_permit&&) at server.cc:?
seastar::concrete_execution_stage<seastar::future<seastar::foreign_ptr<std::unique_ptr<cql_transport::response, std::default_delete<cql_transport::response> > > >, cql_transport::cql_server::connection*, fragmented_temporary_buffer::istream, unsigned char, unsigned short, service::client_state&, cql_transport::cql_server::connection::tracing_request_type, service_permit>::do_flush() at server.cc:?
seastar::lambda_task<seastar::execution_stage::flush()::$_0>::run_and_dispose() at execution_stage.cc:?
seastar::reactor::run_some_tasks() at ??:?
seastar::reactor::do_run() at ??:?
seastar::reactor::run() at ??:?
seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at ??:?
seastar::app_template::run(int, char**, std::function<seastar::future<int> ()>&&) at ??:?
scylla_main(int, char**) at main.cc:?
std::function<int (int, char**)>::operator()(int, char**) const at ??:?
main at main.cc:?
__libc_start_call_main at ??:?
__libc_start_main_alias_2 at :?
_start at ??:?
@xemul
Copy link
Contributor Author

xemul commented May 13, 2024

    auto schema = std::invoke([&] () -> schema_ptr {
        if (receiver_cf.has_value() && db.has_schema(receiver_ks, *receiver_cf)) {
            return db.find_schema(receiver_ks, *receiver_cf);
        } else {
            return nullptr;
        }
    });

checks has_schema(test, test_$mutation_fragments) and doesn't find it. The nullptr schema crashes later

xemul added a commit to xemul/scylla that referenced this issue May 13, 2024
Getting token() function first tries to find a schema for underlying
table and continues with nullptr if there's no one. Later, when creating
token_fct, the schema is passed as is and referenced. If it's null crash
happens.

It used to throw before 5983e9e (cql3: test_assignment: pass optional
schema everywhere) on missing schema, but this commit changed the way
schema is looked up, so nullptr is now possible.

fixes: scylladb#18637

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
@xemul xemul added the backport/5.4 Issues that should be backported to 5.4 branch once they'll be fixed label May 13, 2024
mergify bot pushed a commit that referenced this issue May 15, 2024
Getting token() function first tries to find a schema for underlying
table and continues with nullptr if there's no one. Later, when creating
token_fct, the schema is passed as is and referenced. If it's null crash
happens.

It used to throw before 5983e9e (cql3: test_assignment: pass optional
schema everywhere) on missing schema, but this commit changed the way
schema is looked up, so nullptr is now possible.

fixes: #18637

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
(cherry picked from commit df8a446)
denesb pushed a commit that referenced this issue May 16, 2024
Getting token() function first tries to find a schema for underlying
table and continues with nullptr if there's no one. Later, when creating
token_fct, the schema is passed as is and referenced. If it's null crash
happens.

It used to throw before 5983e9e (cql3: test_assignment: pass optional
schema everywhere) on missing schema, but this commit changed the way
schema is looked up, so nullptr is now possible.

fixes: #18637

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
(cherry picked from commit df8a446)

Closes #18698
xemul added a commit to xemul/scylla that referenced this issue May 20, 2024
…ts()

When selecting from mutation_fragments(table) one may want to apply
token() filtering againts partition key. This doesn't work currently,
but used to crash. This patch adds a regression test for that

refs: scylladb#18637

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
xemul added a commit to xemul/scylla that referenced this issue May 20, 2024
…ts()

When selecting from mutation_fragments(table) one may want to apply
token() filtering againts partition key. This doesn't work currently,
but used to crash. This patch adds a regression test for that

refs: scylladb#18637

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
nyh pushed a commit that referenced this issue May 26, 2024
…ts()

When selecting from mutation_fragments(table) one may want to apply
token() filtering againts partition key. This doesn't work currently,
but used to crash. This patch adds a regression test for that

refs: #18637
refs: #18768

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>

Closes #18759
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backport candidate backport/5.4 Issues that should be backported to 5.4 branch once they'll be fixed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants