Replies: 3 comments 3 replies
-
Hey @perpi! |
Beta Was this translation helpful? Give feedback.
-
Store hashes instead of the raw data. For example: // This code runs on a server
import * as crypto from 'crypto'
const components = request.body.fingerprintComponents
const hash = value => crypto.createHash('sha512').update(value).digest('base64')
// Deep-clone `components` if you don't want to mutate it
if (components.canvas?.value?.geometry) {
components.canvas.value.geometry = hash(components.canvas.value.geometry)
}
if (components.canvas?.value?.text) {
components.canvas.value.text = hash(components.canvas.value.text)
}
if (components.math?.value) {
components.math.value = hash(JSON.stringify(components.math.value))
}
// `components` is now compressed |
Beta Was this translation helpful? Give feedback.
-
@mohaa7 many databases can compress data very efficiently, like @makma mentioned. E.g. ClickHouse will compress and store the deltas of data, I think the simple math of item_size x number_of_items shouldn't be used. |
Beta Was this translation helpful? Give feedback.
-
Each record is approximately 25k in size, which is 230 terabytes in our annual estimate for 10 billion visiting records. It is very large, and the cost of infrastructure would be very expensive! What to do?
To whom are currently using it in the operational environment, what solution do you have for data storage?
Beta Was this translation helpful? Give feedback.
All reactions