Nextcloud Integration

[EagleStar11]

I love the features of paperless-ngx! But if I work with it, I'm forced use the UI and need to log in.
I put all my docs in a nextcloud instance.

Thats why I think about a nextcloud integration.
It would be cool to use the nextcloud UI inclusive the tags and correspondent etc.

Is there a plan to realize a nextcloud integration?

[almereyda]

Someone has documented a similar case in German at:

• https://apfelcast.com/paperless-ngx-nextcloud-scanner-automatischer-workflow/
• https://www.youtube.com/watch?v=DQIbbwHPAqo

[simowNgithub]

But here the issue is, that if you use Docker and Standalone instances, it is really complex to connect the systems.

Two possible solutions for doing a better automation microservice:

1. Do a direct integration for moving files with automation options to nextcloud (would be veeeery nice, and there is no simular thing on nextcloud side)
2. Better web api support for connection web hooks and more if event gets fired on the paperless ngx instance

[bhugh]

I just did this using the Apfelcast instructions and it actually seems to work quite well. I'm running NextCloud AIO.

• I tried connecting the two via local filesystem (which seems like the simple & logical approach) but it was not so easy - permission problems at every step. AIO doesn't seem to play very nice with External Storage in local directories.

• The FTP solution was, by contrast, quite easy. Also it opens the opportunity to run the two programs on two separate computers if necessary.

• I just made a special FTP-paperless account, restricted it to the home directory of the account, and then mounted the Paperless consume and documents directories within that home directory.

• Then made two External Storage directories (under Administration) - one is Paperless-consume to the consume folder and the other is Paperless-documents to the documents folder.

Paperless-documents is set "read only" as you don't want to mess with moving/editing/changing those docs.

The other part of the puzzle is setting up somewhat reasonable Storage Paths & filenames within Paperless so that you can reasonably find things within the directory structure.

For example, our default storage path is:

{created_year}/{correspondent}/{created_year}-{created_month}-{title}

For some special things where it is useful to have similar things grouped into a single folder, you can switch it up like:

/Medical/{created_year}/{created_year}-{created_month}-{correspondent}-{title}
/Income Tax/{created_year}/{created_year}-{created_month}-{correspondent}-{title}
/Freds-Journal/{created_year}/{created_year}-{created_month}-{correspondent}-{title}

[ptpu]

The Youtube video from Apfelcast definitely gives the insight into the possibility to include the input and output folder as external FTP storage in Nextcloud.

Another possibility would be to include a local path. This also requires the Nextcloud addon for external storage. You can then mount a local path as Nextcloud folder. I use Paperless and Nextcloud as Docker container. In the Nextcloud container I mount the consume path from Paperless and specify it in Nextcloud as input folder. The same goes for the output folder of course. In the default configuration, the filenames in the output folder are not necessarily meaningful. But this could also be solved: https://docs.paperless-ngx.com/advanced_usage/#file_name_handling

[flemmingss]

I have mounted my consume folder as external storage, it works for consuming, but somehow the files are not removed from the NC UI.

[almereyda]

Is there a simple procedure that others could replicate, how a minimal setup for

1. Consuming files from Nextcloud into Paperless-ngx
2. Displaying Paperless-ngx files in Nextcloud

can be achieved with using an external storage configuration?

[smehrens]

I have done the nextcloud integration with external storage in nextcloud. Works fine.

And now i miss some additional feature like

• sync tags, correspodents, type as tags to nextcloud.
• content menue items
• • add to paperless
• • delete in paperless
• • jump/link to paperless
• link from paperless to nextcloud (set a external url for storagepaths in paperless)

Thanks for your great work.

Best regards

[Bugsbane]

I'm looking into using paperless, but haven't yet, so forgive me if I'm asking something obvious.

Anyway, it seems to me that the main benefit of Paperless is it's OCR and automatic tagging... So... if mounting Paperless' output folder as an external storage in Nextcloud doesn't give it access to the documents tags or OCR, what's the benefit? Why not just upload the documents directly to Nextcloud? 🤔

[smehrens]

because paperless do a better job then nextcloud in many ways

• asn handling (barcode/qrcode)
• • split documents by asn qrcodes
• • reference the document (online to paper version)
• • link paperversion to online document (is you use a prefix like "http://shortdomain/" then "asn" and let apache rewrite it from"http://shorturl/10010" to "https://mydomain/documents?archive_serial_number=10010"
• storagepath handling
• • change the path for multiple documents by changing storagepath
• • change file location based on metadata
• metadata export/import (dont know how to save tags in nextcloud)
• better filtering
• adding metadata based on rules
• better frontend
• many more .....

but nextcloud has a better accesibility from client devices through the nextcloud client.

this is why i choose to use both.

and i am happy with it, but it could be always better, thats why i mentioned my ideas.

[platini76]

how can I test your integration?

[almereyda]

In the original message it says this feature was used:

• Configuring External Storage (GUI) — Nextcloud latest Administration Manual latest documentation

[noseshimself]

And now i miss some additional feature like

• sync tags, correspodents, type as tags to nextcloud.

For me document type and correspondent are just additional namespaces for tags but the feature that is really missing in Nextcloud are non-global tags that aren't workflow processing markers and can be managed by non-administrators. Ideally these tags would carry ACLs, too. Without that Nextcloud is not really usable as DMS and there is no viable method of mapping the metadata/semantic content of the DMS onto Nextcloud features. I opened issues with feature requests but did not get any answers.

• content menue items

No. There are entirely too many already.

[almereyda]

• Can we instead store a file alongside folders which carries an Linked Data (LD) RDF markup, eventually human- and machine-readable, and exposes a common vocabulary like dcterms or Schema.org for a start?
  • Which reminds me of the way how a semantic layer is put on top of Mediawiki with an addition such as Wikibase.
  • And also of earlier experiments to expose a Solid Pod from Nextcloud. Solid by default uses LD RDF. It should be possible to hook into that for an "RDF store", if it works at all.
• Is there a generic file name convention that can inform a decision on where to look for such file or folder related metadata?
  • Is there a generic location for file system metadata?
    Early experiments with the 'Semantic Desktop' ceased to exist, where indexed and queryable RDF metadata on files was tried.
• Instead of desireable semantic metadata, can we fall back to a Nextcloud-native key-value store, such as the unmaintained SteKoe/nextcloud-customproperties? A fork could be a solid basis and answer to existing demand, How to set Custom Properties on multiple files at once? - 🍱 Features & apps - Nextcloud community, providing a suitable foundation for tighter integration of metadata from and about other platforms. Using CURIE syntax for key descriptors is highly encouraged, to provide a lean migration path towards handling the metadata as RDF and as such linked data.

I'd like to know if you could also share with us which feature requests you had once filed/ignored?

I'm seeing a point here where Nextcloud metadata handling seems improvable in general, before a Paperless integration seems approachable. When that is solved, search integration of (1) metadata and (2) OCR full text (which might be available out of the box with using the Full text search - Nextcloud App) come to mind, next to support in Workflows in the long run, too.

When ACLs from Paperless groups can be mapped to Nextcloud groups, they can also be mapped to OIDC groups et al. Reusing groups from OIDC claims would allow to easily sync them between Nextcloud and Paperless, making a strong case for approaching its support again as well:

• Users and Groups - Basic Usage - Paperless-ngx
• [Other] Support some form of central user management #625
• Feature: Add OpenID Connect SSO support via django-allauth #1746
• Integration of openidconnect #3773
• [Feature Request] OAuth2/OIDC support #4006

Supporting some proxy authentication scheme, as with Authalia or Træfik ForwardAuth, could be a little more simple for a start, but the Django-AllAuth implementation path of the previous attempts seems very promising, too.

SAML and SCIM would only make sense in highly regulated environments, where central removal and logout are eased, and seem out of scope for a start. But supporting this would greatly enhance Paperless' standing in the self-hoster world, as it is integrated well with tools of similar scope and reach. Nextcloud for a start, and Keycloak, Authentik and alike for more advanced use cases.

In another place it is said that Webhooks could be sent from a post-consume script.

• Post-consume script - Advanced Topics - Paperless-ngx
• Pass OCR content to post-consumption script hook or provide webhook for post-consumption #205
• Is there a "webhook" feature where paperless can notifiy other services of new documents? #1002
• Coupling loose ends: tasks and document id‘s #4015

Further references to DMS use cases with Nextcloud:

Apparently custom metadata, if not shown directly in the UI, but stored, can already be saved via the WebDAV interface:

• How to add custom meta data to your documents on Nextcloud | by Mathias Conradt | Medium

MayanEDMS exemplifies a DMS that syncs to Nextcloud on its own, without a Nextcloud App. Such an integration could request an App Token from the instance.

• Smarte Dokumenten-Management - uniXa-Blog

Seems we're not there yet completely, but people are good in identifying requirements and integration vectors:

• Use Case: Custom Properties - 🍱 Features & apps - Nextcloud community
• How to set Custom Properties on multiple files at once? - 🍱 Features & apps - Nextcloud community
• Where are DAV properties persisted and how to add new ones? - 💻 Development - Nextcloud community
• Files extended metadata - 🍱 Features & apps / App ideas - Nextcloud community

[almereyda]

Update: Custom Properties is abandoned.

[almereyda]

https://nextcloud.com/blog/nextcloud-hub8/ says there is a new Paperless integration.

It lives in https://github.com/nextcloud/integration_paperless and reads:

It adds a file action menu item that can be used to upload a file from your Nextcloud Files to Paperless.

Published at https://apps.nextcloud.com/apps/integration_paperless

[Bugsbane]

What I'm most interested in, is if it integrates things like tags and OCR'd text, so they're searchable from inside Nextcloud.

[almereyda]

Apparently that's something the OpenProject integration knows to do. Forwarding search indexes to a central frontend is a great use case for such a link between applications.

[shamoon]

Im inclined to mark the above comment from almereyda (links to https://apps.nextcloud.com/apps/integration_paperless ) the answer here. I dont think theres any intent to work on this on our side anyway so would probably make sense for the folks interested to checkout or even contribute to the repository linked above

[piratPavel]

It's a cool feature, but I haven't managed to get it working yet.

Nextcloud 29 (HUB 8), I haven't found anywhere how to set up a connection with Paperless-ngx.
Downloaded from the catalog, should be active and I'm not shopen to find anything anywhere.

Too bad there isn't at least a little bit of documentation on this

[JokerGermany]

/settings/user/connected-accounts

[piratPavel]

Thank you very much - it works

[github-actions[bot]]

This discussion has been automatically closed because it was marked as answered. Please see our contributing guidelines for more details.