[Bug]: Caching of source code / no loading of changed code from git repository

[RoseDeSable]

Nuclio Version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of Nuclio.

Issue Description

Hi,
I have stored the sourcecode of the main program of my function in our git repository. The yaml-file of the function is so constructed, that the source code is loaded from the git repository at time of developing. First developing after the importing of the yaml is successful. But if I change the source in the git repository, the changing shows no effort after the developing. I examined this circumstance and come to the conclusion, that the old source is obviously cached in Nuclio and is not replaced by the new.
An experiment confirms this: I deleted the source from the git repository and started the developing. This was successful ! First, if I deleted the function and newly loaded the yaml-file, then I received the message of the absence of the code file. The developing failed.

Best Regards
Rose

Expected Behavior

Deployment Method

Docker

Nuclio Version

1.11.24

Additional Information

No response

[TomerShor]

Hi @RoseDeSable ,

Nuclio skips building a new image in some cases, such as when a specific image is set on the function spec.
However, in your case and if not done explicitly, it sounds like it shouldn't skip building on redeployment.

Some questions to better understand the issue:

1. What code entry type do you set? git or github?
2. Can you share your function configuration yaml (redact any sensitive info). I'm mainly interested in the spec.build section.

[RoseDeSable]

Good Morning,
my code entry type is "git"

Here is the build section - the sensitive parts are modified

---

build:
path: "https://scm.int.my-enterprise/my-account/Nuclio.git"
baseImageRegistry: "registry.int.my-enterprise:7008"
image: cvat.our-project-modell_0_0_11-standard-ubuntu-python
noCache: true
baseImage: "registry.int.my-enterprise:7003/standard-ubuntu-python"
commands:
- 'apt-get update '
- 'apt-get install -y libglib2.0-0 libgl1 libsm6 libxrender1 libfontconfig1 libxext6 libzbar-dev libdmtx-dev libpq-dev build-essential cmake libleptonica-dev '
- 'apt-get install -y git'
- 'cd /root;git -c http.sslVerify=false clone https://my-git-api-key@scm.int.my-enterprise/AI/our-project.git'
- 'pip install --no-cache-dir -r /root/our-project/requirements.txt'
- 'pip install PyYAML'
- 'mkdir /root/our-project/spur/model'
- 'cd /root/our-project/spur/model;curl -sLO http://fx.int.my-enterprise/models/our-project/model_0_0_14/frozen_inference_graph.pb'
- 'cd /root/our-project/spur/model;curl -sLO http://fx.int.my-enterprise/models/our-project/model_0_0_14/pipeline.config'
offline: true
runtimeAttributes:
repositories: []
codeEntryType: git
codeEntryAttributes:
branch: master
headers: ""
password: my-nuclio-password
reference: ""
tag: ""
username: my-nuclio
workDir: our-project-modell-v1.0.2/main.py
timestamp: 1705559455
dependencies: []
loggerSinks:
- level: debug
platform: {}
securityContext: {}
eventTimeout: 30s

---

[TomerShor]

@RoseDeSable Since you're working in a Docker environment, I suspect that the first created image is still cached by docker, and thus when it's building the function during redeployment, it essentially builds on top of the cached image, instead of building a fresh image.

If possible, I would appreciate seeing the relevant logs from the nuclio-dashboard container.
You can do that by the following:

1. Run docker logs <nuclio-dashboard-container-name>.
2. Search for the log Building processor image, and locate the requestID relevant for this function's creation.
3. grep on the logs for all logs with the same requestID.

This will help us better understand this and find the root cause. Thanks§1

[RoseDeSable]

Ok,
I speak with my collegues. It could take a long time, until I will inform you.

[RoseDeSable]

Hello,
here are the log data, that you wish. I hope, it is correct so.
log_nuclio.txt