[YSQL] pg_cron: Use stateful service to assign cron leader #22360
Assignees
Labels
jira-originated
kind/enhancement
This is an enhancement of an existing feature
priority/low
Low priority
Comments
yugabyte-ci
added
jira-originated
kind/enhancement
hari90
added a commit
that referenced
this issue
May 15, 2024
Summary: Use shared memory `is_cron_leader` field to indicate which node will be the cron leader. Only the leader will schedule and run cron jobs. This ensures that cron jobs are run at most once. Distributed job scheduling and execution will happen as part of #22336. If we had two or more leaders in the same minute we would not run the task multiple times since cron does not run tasks for the first minute of leadership. Interval jobs do not have this guarantee since their timer starts when we become the leader. Non Yugabyte pg_cron has the same behavior when pg restarts. Check comments in `YbCheckLeadership` and `StartAllPendingRuns` for more details. For testing purposes we set `is_cron_leader` via `FLAG_TEST_is_ysql_cron_leader`. `TabletServer` uses a flag callback to set the shared memory to the value of the flag. #22360 will add a StatefulService that will ensure one and only one tserver is always picked as the pg_cron leader. - Moved `YBCPgResetCatalogReadTime` inside `RefreshTaskHash` as we do not need to do this every time - Added `YbUpdateCatalogCacheVersion` since without it we hit `The catalog snapshot used for this transaction has been invalidated`. - Replaced `YBCDeleteSysCatalogTuple` with `CatalogTupleDelete` so that indexes entries are also deleted. Fixes #22260 Jira: DB-11179 Test Plan: PgCronTest.AtMostOnceTest PgCronTest.PerMinuteTask yb_pg_cron-test Reviewers: tnayak, fizaa Reviewed By: tnayak Subscribers: yql, ybase Differential Revision: https://phorge.dev.yugabyte.com/D34926
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-11263
The text was updated successfully, but these errors were encountered: