fix(caldav): automatically delete outdated scheduling objects

[miaulalala]

• Resolves: Remove old scheduling objects from INBOX and oc_schedulingobjects via cron #43621

Summary

oc_schedulingobjects currently grows without ever deleting outdated objects. This PR a repair step that is declared as expensive so admins can decide to run them at their convenience for the initial delete.

The delete is chunked to 50k rows on each transaction so the database isn't locked for a long time (especially in clustered setups this could cause issues). MySQL needs special treatment as it doesn't support LIMITs on DELETE queries, so it does a SELECT on the ids to delete, and then runs the delete on those.

After the repair step has run, a regular cron job is added to the Jobs List that runs every hour to get rid of scheduling objects that are older than an hour. We don't really need them and could theoretically delete them as soon as they're processed by the ITip\Broker but as rooms and resources are also run in a cron job, keeping them until the principal room and resources are added is probably a good idea as I can't exclude unwanted side effects. I also updated the runtime for rooms and resources to run every half hour for that reason.

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[miaulalala]

Do a slow repair job for this so we don't kill the instance.

1. Loop over all data DELETEs and do 50000 each loop - MySQL needs to be handled differently as it doesn't do LIMITs on its queries 💢
2. Do repair job that also does that
3. BG job is TIME_SENSITIVE and should run every hour as it has a longblob so it's better to delete often in smaller batches
4. wrap the DELETE in a transaction so no index updates are done during the run.
   See: More than 1000 expressions in a list are not allowed on Oracle activity#1384 for how we handled it for activity

[miaulalala]

server/lib/private/Repair.php

Lines 223 to 228 in b0bfe3e

	public static function getExpensiveRepairSteps() {
	return [
	new OldGroupMembershipShares(\OC::$server->getDatabaseConnection(), \OC::$server->getGroupManager()),
	\OC::$server->get(ValidatePhoneNumber::class),
	];
	}

for the expensive repair job

[ChristophWurst]

Let's get rid of the hourly full table scan

👍 otherwise

[ChristophWurst]

explicit transaction is superfluous for a single query

[miaulalala]

yeah I need to move this further up so each 1k delete doesn't do the full table scan. Will an index solve this? I don't think it will, seeing as there is a blob associated with this query. When I discussed it with Joas, this was the intention behind the transaction (even though I applied it at the wrong code point 😉 )

[ChristophWurst]

I don't see how an explicit transaction is different to our AUTOCOMMIT=1 sessions

[ChristophWurst]

Suggested change

	$queryResult = 0;
	$numDeleted = 0;

naming

[ChristophWurst]

if you order by last modified, the values of the chunks will be closer together and make it more efficient for the database to traverse the index.

[ChristophWurst]

there is no index for this query. it will perform a full table scan:

> EXPLAIN SELECT id FROM oc_schedulingobjects WHERE lastmodified <= 1700000000;
+------+-------------+----------------------+------+---------------+------+---------+------+------+-------------+
| id   | select_type | table                | type | possible_keys | key  | key_len | ref  | rows | Extra       |
+------+-------------+----------------------+------+---------------+------+---------+------+------+-------------+
|    1 | SIMPLE      | oc_schedulingobjects | ALL  | NULL          | NULL | NULL    | NULL | 104  | Using where |
+------+-------------+----------------------+------+---------------+------+---------+------+------+-------------+

[miaulalala]

Add an index on the lastmodified?

[ChristophWurst]

Yes

[ChristophWurst]

why is this time sensitive? wouldn't it be sufficient to clean up once a day in the maintenance window?

[miaulalala]

Well it entirely depends on the size of the instance. For 100 users, no problem. For 1k or 10k, a lot of data could be produced in a short amount of time, especially if there's a lot of usage on the calendar. That's why I'd rather clean up once an hour during business hours than have this run outside of it with lots of rows.

[ChristophWurst]

Right now we have no cleanup at all and most instances are fine. So I think 99.99% of instances will also survive if data is only cleaned up daily :)

A large range DELETE query can cause table locks that block other operations. We have seen this with #27695.

[ChristophWurst]

Relevant too: #43605

[ChristophWurst]

caveat with timing this once per hour while the resources job runs twice: it assumes that background jobs run reliably. this might not be the case for ajax cron.

[miaulalala]

The resources job running every 30 minutes is just a precaution, I don't remember my exact thought pattern as to why, but I can revert. It was something along the lines of not deleting scheduling objects that might not have been processed yet but the more I think about it the less it makes sense, as I also delete scheduling objects 1 hour in the past only.