You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CronJob resources deployed to Kubernetes environments by Spring Cloud Dataflow have a successfulJobsHistoryLimit value of 3 by default.
By this setting, the maximum number of Task Pods created from a CronJob is three (Completed).
However, Sub Task Pods created from Composed Task may have problems with IP assignment, Max Pod count limit, etc. because they are in a situation where more than 3 Completed pods are maintained.
Solution description:
This is caused by the lack of an Ownerreferences setting.
Task Pods that are created from the flow CronJob -> Job -> Pod have Ownerreferences as normal. As a result, when the Job is deleted, the Task Pods are also cleaned up by the GC.
However, Sub task pods that are additionally created by Composed Tasks are not automatically cleaned up because they do not have Ownerrefences.
This causes various issues and should be fixed so that Sub task pods are given the same Ownerreferences.
Additional context:
kubernetes: 1.28.6-eks
batch/v1 CronJob
spring-cloud-dataflow: 2.11.2
composed-task-runner: 2.11.2
The text was updated successfully, but these errors were encountered:
Until this issue is resolved, we want to manually clean up the completed sub pods, but we are struggling because we can't find any association between the composed task pod and the sub task pods that are created by the composed task pod.
Is there any way to distinguish which composed task pod a particular sub task pod is created from?
(Note that we have created two schedule resources from one task, running daily and monthly.
So there are two cronjobs, but the name format of the subtask pods that are created is the same.)
Problem description:
CronJob resources deployed to Kubernetes environments by Spring Cloud Dataflow have a
successfulJobsHistoryLimit
value of 3 by default.By this setting, the maximum number of Task Pods created from a CronJob is three (Completed).
However, Sub Task Pods created from
Composed Task
may have problems with IP assignment, Max Pod count limit, etc. because they are in a situation where more than 3 Completed pods are maintained.Solution description:
This is caused by the lack of an
Ownerreferences
setting.Task Pods that are created from the flow CronJob -> Job -> Pod have
Ownerreferences
as normal. As a result, when the Job is deleted, the Task Pods are also cleaned up by the GC.However, Sub task pods that are additionally created by Composed Tasks are not automatically cleaned up because they do not have
Ownerrefences
.This causes various issues and should be fixed so that Sub task pods are given the same
Ownerreferences
.Additional context:
The text was updated successfully, but these errors were encountered: