Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use local OS' path separator #304

Open
csm10495 opened this issue Feb 4, 2024 · 3 comments
Open

Use local OS' path separator #304

csm10495 opened this issue Feb 4, 2024 · 3 comments

Comments

@csm10495
Copy link

csm10495 commented Feb 4, 2024

I use diskcache and save the cache directory to a share drive. Later on, I use diskcache again to pull / update the data.

The problem I recently hit was when I added a Linux host to my previously-all-Windows pool:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/diskcache/core.py", line 1173, in get
    value = self._disk.fetch(mode, filename, db_value, read)
  File "/usr/local/lib/python3.10/site-packages/diskcache/core.py", line 281, in fetch
    with open(op.join(self._directory, filename), 'rb') as reader:
OSError: [Errno 22] Invalid argument: '/mnt/z-drive/Temp/shared/csmutil.cache/32\\25\\8a75134ac891498afaeb4efa3ec7.val'

It seems like diskcache saves a path inside the db based off the OS' path separator., which leads to this error ^. Can the path be normalized to the local-OS' path seperater?

@csm10495
Copy link
Author

csm10495 commented Feb 4, 2024

(One other side note: it was a bit of a pain to find this error, since the default value was getting used since it caught the OSError). Maybe the caught error could be more granular in the future :)

@csm10495
Copy link
Author

csm10495 commented Feb 4, 2024

Lots of possible ways to allow this:

  • Maybe an option to chose the path separator, with a default of today's behavior.
  • Maybe just .replace('\\', '/') on the paths in fetch() and i think remove()
  • Maybe just always swap to using forward slashes regardless of OS. I can't think of an OS that at user-level doesn't allow forward slashes.

@csm10495
Copy link
Author

csm10495 commented Feb 5, 2024

For anyone out there who hits this, you can resolve via:

<diskcache_obj>._sql('UPDATE Cache SET filename = REPLACE(filename, "\\", "/") WHERE filename is not NULL').fetchall()

I run this every time i get a cache object just in case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant