Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rd_kafka_produceva: double free headers on message sending error #4627

Open
6 tasks done
blindspotbounty opened this issue Feb 28, 2024 · 2 comments
Open
6 tasks done
Labels

Comments

@blindspotbounty
Copy link

blindspotbounty commented Feb 28, 2024

Description

Accidentally, we found a problem related to headers double free in rd_kafka_produceva method:

=================================================================
==49557==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001178e9dd8 at pc 0x00010f6813b4 bp 0x00016b719000 sp 0x00016b718ff8
READ of size 8 at 0x0001178e9dd8 thread T18
#0 0x10f6813b0 in rd_list_destroy_elems rdlist.c:276
    #1 0x10f681828 in rd_list_destroy rdlist.c:300
    #2 0x10f2fb2d0 in rd_kafka_headers_destroy rdkafka_header.c:37
    #3 0x10f465414 in rd_kafka_produceva rdkafka_msg.c:521
    #4 0x10fbcc5dc in RDKafkaClient._produceVariadic(topicHandle:partition:messageFlags:key:value:opaque:cHeaders:) RDKafkaClient.swift:218
    #5 0x10fbc8910 in closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) RDKafkaClient.swift:137
    #6 0x10fbe5188 in partial apply for closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) <compiler-generated>
    ...
    
0x0001178e9dd8 is located 8 bytes inside of 48-byte region [0x0001178e9dd0,0x0001178e9e00)
freed by thread T18 here:
#0 0x105133380 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53380)
    #1 0x10f2fb2fc in rd_free rd.h:151
    #2 0x10f2fb2d8 in rd_kafka_headers_destroy rdkafka_header.c:38
    #3 0x10f460c04 in rd_kafka_msg_destroy rdkafka_msg.c:118
    #4 0x10f4652ec in rd_kafka_produceva rdkafka_msg.c:505
...

previously allocated by thread T18 here:
#0 0x105133244 in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53244)
    #1 0x10f2fb3b4 in rd_malloc rd.h:139
    #2 0x10f2fb330 in rd_kafka_headers_new rdkafka_header.c:44
    #3 0x10f4646f4 in rd_kafka_produceva rdkafka_msg.c:431

How to reproduce

I was reproducing it with swift-kafka-client wrapping client within issue swift-server/swift-kafka-client#150

git clone --recursive https://github.com/swift-server/swift-kafka-client
git apply diff // (from issue)
swift test --sanitize=address --filter testProduceAndConsumeWithMessageHeaders

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • librdkafka version (release number or git tag): v2.3.0
  • Apache Kafka version: N/A
  • librdkafka client configuration: any invalid broker address
  • Operating system: ubuntu 22.04 lts
  • Provide logs (with debug=.. as necessary) from librdkafka: ASAN logs instead
  • Provide broker log excerpts: N/A
  • Not a critical issue
@emasab
Copy link
Collaborator

emasab commented Mar 27, 2024

Yeah, the hdrs variable should be set to NULL before rd_kafka_msg_destroy
if it corresponds to the message headers, otherwise it's freed a second time here

@emasab emasab added the bug label Mar 27, 2024
@blindspotbounty
Copy link
Author

Yes, exactly.
Btw: fixed comment as requested in PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants