Problems when using container with read-only root FS

[SimonWoidig]

I am trying to deploy KAUI on kubernetes and also fulfill some security requirements. One common thing is a read-only root filesystem. The app then can write only to mounted volumes under some path (if volume is not set as read-only) - for example: mount an empty dir to /tmp for temporary data.
Well I tried kpm pull_defaultbundles but it failed on a tempdir creation.

Read-only file system @ dir_s_mkdir - /var/lib/tomcat/killbill20231218-96-1531coz
/opt/kpm-0.11.1-linux-x86_64/lib/ruby/lib/ruby/2.6.0/tmpdir.rb:89:in `mkdir'
/opt/kpm-0.11.1-linux-x86_64/lib/ruby/lib/ruby/2.6.0/tmpdir.rb:89:in `block in mktmpdir'
/opt/kpm-0.11.1-linux-x86_64/lib/ruby/lib/ruby/2.6.0/tmpdir.rb:135:in `create'
/opt/kpm-0.11.1-linux-x86_64/lib/ruby/lib/ruby/2.6.0/tmpdir.rb:87:in `mktmpdir'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tenant_config.rb:23:in `<class:TenantConfig>'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tenant_config.rb:8:in `<module:KPM>'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tenant_config.rb:7:in `<top (required)>'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tasks.rb:471:in `require'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tasks.rb:471:in `block in included'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tasks.rb:14:in `class_eval'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tasks.rb:14:in `included'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/cli.rb:7:in `include'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/cli.rb:7:in `<class:Cli>'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/cli.rb:6:in `<module:KPM>'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/cli.rb:5:in `<top (required)>'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/bin/kpm:8:in `require'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/bin/kpm:8:in `<top (required)>'
/opt/kpm-latest/lib/vendor/ruby/2.6.0/bin/kpm:23:in `load'
/opt/kpm-latest/lib/vendor/ruby/2.6.0/bin/kpm:23:in `<main>'

I am using the image you supply (docker.io/killbill/kaui:2.0.11) so it runs under tomcat. I am not sure, how to specify a temp directory (via an env. variable for example). I tried TMPDIR, TMP and TEMP, but it still tries to write at /var/lib/tomcat/.... I also looked through the source code in ruby, but I think, it should work. On the other hand, I am not sure, how the default behaviour is altered by running the app under jRuby (creating a "sandbox" in the app's folder or something?).
Is there a way to specify the path? Am I missing something?

[SimonWoidig]

Edit 2: I didn't specify the correct bundle destination, my mistake!

Ok, so I set the root FS to the default behaviour (read and write). I still am having problems.

tomcat@kaui-fb654f9f-cn2jp:~$ kpm pull_defaultbundles
Permission denied @ rb_sysopen - /sha1.yml
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/sha1_checker.rb:104:in `initialize'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/sha1_checker.rb:104:in `open'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/sha1_checker.rb:104:in `init!'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/sha1_checker.rb:15:in `initialize'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/sha1_checker.rb:10:in `new'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/sha1_checker.rb:10:in `from_file'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/killbill_server_artifact.rb:23:in `info'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/base_installer.rb:213:in `install_default_bundles'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/lib/kpm/tasks.rb:215:in `pull_defaultbundles'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
/opt/kpm-0.11.1-linux-x86_64/lib/vendor/ruby/2.6.0/gems/kpm-0.11.1/bin/kpm:8:in `<top (required)>'
/opt/kpm-latest/lib/vendor/ruby/2.6.0/bin/kpm:23:in `load'
/opt/kpm-latest/lib/vendor/ruby/2.6.0/bin/kpm:23:in `<main>'

tomcat@kaui-fb654f9f-cn2jp:~$ cd ../kaui/bundles/

tomcat@kaui-fb654f9f-cn2jp:/var/lib/kaui/bundles$ ls -lah
total 4.0K
drwxr-xr-x. 1 tomcat tomcat  22 Aug 25 11:53 .
drwxr-xr-x. 1 tomcat tomcat  21 Aug 25 11:53 ..
-rw-r--r--. 1 tomcat tomcat 366 Aug 25 11:53 sha1.yml

The permissions look correct. The file is present. KPM still fails.

Edit: I have a security context config in place, so that the container runs as the tomcat user and with some other security measures.
Pod security context:

      securityContext:
        fsGroup: 1000
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000

Container security context:

          securityContext:
            capabilities:
              drop:
                - ALL
            allowPrivilegeEscalation: false
            privileged: false
            runAsNonRoot: true
            # TODO: stupid ruby writing everywhere and no way to change it
            # readOnlyRootFilesystem: true
            runAsUser: 1000
            runAsGroup: 1000