Use mmap for better file writing performance

[michaeleisel]

New Issue Checklist

• I have read and understood the CONTRIBUTING guide
• I have read the Documentation or CocoaDocs
• I have searched for a similar issue in the project and found none

Issue Info

Info	Value
Platform Name	e.g. ios

Issue Description and Steps

I find that calls to -[NSFileHandle writeData:] are one of the biggest bottlenecks for the logger. This makes sense, as it is generally a costly function. It would be great if the logger used a memory-mapped file to do writes. It would have the same durability AFAIK and the performance would be much, much better.

[lolgear]

@michaeleisel
Could you try latest release?

Thanks!

[michaeleisel]

If you're referring to #1001, a buffer may solve it for some, but it creates a trade-off between efficiency and durability. mmap does not. Although I do see a lot of other bottlenecks as well upon further review, so perhaps there is lower hanging fruit to pick.

[michaeleisel]

To be honest, I wonder if #1001 should just be reverted. It is more complicated, both for the consumers as well as library developers, and less durable, than mmap. What is the performance data backing the change? Has it been compared to mmap?

[michaeleisel]

As in, #1001 reverted alongside the addition of mmap

[shayanbo]

Hi, @michaeleisel , do you have more detailed mmap solution?

[michaeleisel]

The top answer of this question shows how to both and read and write using mmap (although we only need to write). Once you’ve set everything up, you can use the mmap’d pointer just like any C pointer, like how they pass it to memset to write to it. But, also be sure to verify that the file writing code (without using any intermediate memory buffers) is a bottleneck in the current implementation. There may be even larger bottlenecks that are simpler to solve, but I wanted to call attention to this one since work was being put into it anyways.

[michaeleisel]

Link: https://stackoverflow.com/questions/26259421/use-mmap-in-c-to-write-into-memory

[jcbertin]

@michaeleisel Not sure that is the right thing to do when writing in a log file:
https://stackoverflow.com/questions/35891525/mmap-for-writing-sequential-log-file-for-speed

[michaeleisel]

Using a memory buffer incurs both the complexity of implementing the buffer as well as the fragility of only persisting data in memory. IMO it’s a next step after mmap

[jcbertin]

But appending data to a mapped file will force you to unmap/resize/remap the file which incurs a performance penalty.

[michaeleisel]

CocoaLumberjack uses fixed-size files, correct? So you can "pre-allocate" by making the file as large as it would potentially grow to, e.g. by seeking to the end and writing a single byte.