You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A common workflow for me with joblib.Memory.cache is that I have a function which I investigate:
@mem.cache
def myfunc(a, b):
return sum([costly_function(b, 42, callback=otherfunction) for i in range(a)])
for b in range(10):
print(myfunc(1000, 10))
Then some time later, I realise that I want to also test variations of the other hard-coded variables. I change to:
@mem.cache
def myfunc(a, b, c=42):
return sum([costly_function(b, c, callback=otherfunction) for i in range(a)])
or
@mem.cache
def myfunc(a, b, callback=otherfunction):
return sum([costly_function(b, 42, callback=callback) for i in range(a)])
for example.
Now joblib.Memory realises that the function is different, discards the cache.
I was wondering whether for simple situations, where a single value (constant, function, class) that is used directly once, is promoted to a parameter, could be identified as the same identical function.
This would require either
a) a hash(function, parameters) that gives the same value when the argument to the function is not given but hardcoded in the function, or,
b) upon encountering such a rewrite to see whether the old cache can be explained with a simple function generalisation (the new function where newly added parameters with their defaults is equivalent to the old function). Then the old cache can be migrated for the new function code with updated arguments.
b) seems easier.
This is probably wishful thinking and difficult to implement, so feel free to close.
The text was updated successfully, but these errors were encountered:
A common workflow for me with joblib.Memory.cache is that I have a function which I investigate:
Then some time later, I realise that I want to also test variations of the other hard-coded variables. I change to:
or
for example.
Now joblib.Memory realises that the function is different, discards the cache.
I was wondering whether for simple situations, where a single value (constant, function, class) that is used directly once, is promoted to a parameter, could be identified as the same identical function.
This would require either
a) a hash(function, parameters) that gives the same value when the argument to the function is not given but hardcoded in the function, or,
b) upon encountering such a rewrite to see whether the old cache can be explained with a simple function generalisation (the new function where newly added parameters with their defaults is equivalent to the old function). Then the old cache can be migrated for the new function code with updated arguments.
b) seems easier.
This is probably wishful thinking and difficult to implement, so feel free to close.
The text was updated successfully, but these errors were encountered: