Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zfree_with_size #453

Open
wants to merge 6 commits into
base: unstable
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 9 additions & 4 deletions src/sds.c
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ sds sdsdup(const sds s) {
/* Free an sds string. No operation is performed if 's' is NULL. */
void sdsfree(sds s) {
if (s == NULL) return;
s_free((char *)s - sdsHdrSize(s[-1]));
s_free_with_size(sdsAllocPtr(s), sdsAllocSize(s));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this is ugly AND I feel this is not your fault.

The mix between requested length and allocated size is non intuitive and error prone.

I suggest:
1/ sdsalloc() will explicitly refuse to work for SDS_TYPE_5 by asserting
case SDS_TYPE_5: assert(0, "Type 5 does not have alloc")
which is fair as Type 5 does not have alloc value and any answer is wrong

2/ similarly sdsAllocSize function is simply and will fail spectacularly on Type 5

size_t sdsAllocSize(sds s) {
    return sdsHdrSize(s[-1] & SDS_TYPE_MASK) + sdsalloc(s) + 1;
}
 // how come we dont have sdsTYPE(s) macro?
// Why sdsHdrSize does not take SDS but take a type?
// Why dont we have sdsHdrSizeByType
// So many questions...

And thus, should never be called for TYPE_5

3/ If its interesting anywhere to anyone... add a function which explicitly fuse the meaning and with a very descriptive comment as to what it actually returns

size_t sdsAllocSizeOrLen(sds s) {
   if (s[-1] & SDS_TYPE_MASK == SDS_TYPE_5) { 
       return sdsLen(s) + 1;
   }
   return sdsAllocSize(s);
}

I suspect this function is not really needed at all

4/ And just here in sdsFree do the if else...

void sdsfree(sds s) {
    if (s == NULL) return; // By the way this is lazy we should not allow sdsfree-ing a null pointer... but thats a point for future PR as there is probably code that relay on this thing 
   if (flags & SDS_TYPE_MASK == SDS_TYPE_5) { 
      s_free((char *)s - 1);
   } else {
     s_free_with_size(sdsAllocPtr(s), sdsAllocSize(s[-1]));
  }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just peeked and following my suggestion will actually unnecessarily bloat this PR.

So I am OK with the PR as is... and its worth considering a future cleanup in a separate PR

}

/* Set the sds string length to the length as obtained with strlen(), so
Expand Down Expand Up @@ -369,7 +369,7 @@ sds sdsResize(sds s, size_t size, int would_regrow) {
* We aim to avoid calling realloc() when using Jemalloc if there is no
* change in the allocation size, as it incurs a cost even if the
* allocation size stays the same. */
bufsize = zmalloc_size(sh);
bufsize = sdsAllocSize(s);
alloc_already_optimal = (je_nallocx(newlen, 0) == bufsize);
#endif
if (!alloc_already_optimal) {
Expand Down Expand Up @@ -412,8 +412,13 @@ sds sdsResize(sds s, size_t size, int would_regrow) {
* 4) The implicit null term.
*/
size_t sdsAllocSize(sds s) {
size_t alloc = sdsalloc(s);
return sdsHdrSize(s[-1]) + alloc + 1;
char type = s[-1] & SDS_TYPE_MASK;
/* SDS_TYPE_5 header doesn't contain the size of the allocation */
if (type == SDS_TYPE_5) {
return s_malloc_size(sdsAllocPtr(s));
Copy link
Contributor

@ranshid ranshid Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is currently my only concern with this PR. IIUC before this PR this would have been a very slim and trivial operation to call sdsAllocSize for SDS_TYPE_5, but now this will become more expensive. I guess in your tests this is being compensated by the fact that we eliminated it in free, but in some cases (like evictions) we might experience overhead?
Can we try and test CPU when heavy evictions are in place?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I see it is mainly used for AOF buf size. So I guess the concern is minor

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess my main point here is that we do not optimize SDS_TYPE_5 in this pr but we do impact the way we calculate it's allocation size. This makes sense from cleaner code maybe, but I wonder if changing it is something we want.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyway, since it DOES make the sdsResize and sdsFree code much cleaner I am in favor of approving this PR. (maybe @madolson will think differently?)

Copy link
Contributor Author

@poiuj poiuj Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also note that SDS_TYPE_5 will not be used for AOF. AOF buffer is initialized with an empty SDS to be expanded later. In this case we use SDS_TYPE_8.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I thought more about the future uses of sdsAllocSize

} else {
return sdsHdrSize(type) + sdsalloc(s) + 1;
}
}

/* Return the pointer of the actual SDS allocation (normally SDS strings
Expand Down
1 change: 1 addition & 0 deletions src/sdsalloc.h
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
#define s_trymalloc ztrymalloc
#define s_tryrealloc ztryrealloc
#define s_free zfree
#define s_free_with_size zfree_with_size
#define s_malloc_usable zmalloc_usable
#define s_realloc_usable zrealloc_usable
#define s_trymalloc_usable ztrymalloc_usable
Expand Down
36 changes: 26 additions & 10 deletions src/zmalloc.c
Original file line number Diff line number Diff line change
Expand Up @@ -361,22 +361,38 @@ size_t zmalloc_usable_size(void *ptr) {
}
#endif

void zfree(void *ptr) {
#ifndef HAVE_MALLOC_SIZE
void *realptr;
size_t oldsize;
/* Frees the memory buffer pointed by ptr and updates statistics. When using
* jemalloc this function uses the fast track by specifying the buffer size.
*
* ptr must point to the start of the buffer. On systems where we have
* the additional header with size, the caller must do the necessary adjustments
* to ptr. ptr must not be NULL.
* The caller is responsible to provide the real allocaction size.
*
* If the caller can't satisfy any of the conditions above, it should call
* zfree() instead. */
void zfree_with_size(void *ptr, size_t size) {
update_zmalloc_stat_free(size);

#ifdef USE_JEMALLOC
je_sdallocx(ptr, size, 0);
#else
free(ptr);
#endif
}

void zfree(void *ptr) {
if (ptr == NULL) return;

#ifdef HAVE_MALLOC_SIZE
update_zmalloc_stat_free(zmalloc_size(ptr));
free(ptr);
size_t size = zmalloc_size(ptr);
#else
realptr = (char *)ptr - PREFIX_SIZE;
oldsize = *((size_t *)realptr);
update_zmalloc_stat_free(oldsize + PREFIX_SIZE);
free(realptr);
ptr = (char *)ptr - PREFIX_SIZE;
size_t data_size = *((size_t *)ptr);
size_t size = data_size + PREFIX_SIZE;
#endif

zfree_with_size(ptr, size);
}

char *zstrdup(const char *s) {
Expand Down
1 change: 1 addition & 0 deletions src/zmalloc.h
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ __attribute__((malloc, alloc_size(1), noinline)) void *ztrymalloc(size_t size);
__attribute__((malloc, alloc_size(1), noinline)) void *ztrycalloc(size_t size);
__attribute__((alloc_size(2), noinline)) void *ztryrealloc(void *ptr, size_t size);
void zfree(void *ptr);
void zfree_with_size(void *ptr, size_t size);
void *zmalloc_usable(size_t size, size_t *usable);
void *zcalloc_usable(size_t size, size_t *usable);
void *zrealloc_usable(void *ptr, size_t size, size_t *usable);
Expand Down