[BUGFIX] Add check to ensure that setTimeout is not called with a value greater than MAX_SIGNED_INTEGER

[pdfowler]

Summary

In my local dev environment I've occasionally run into a scenario where my console gets flooded with TimeoutOverflowWarnings. I eventually traced the problem back to agenda's processJobs logic which would set the timeout for re-processing a future job without checking the bounds of the timeout value.

Investigation

Unfortunately for us, the setTimeout method interprets values greather than 2e31-1 as a negative value, which is in turn interpreted as 1ms, causing a tight loop which will continue until the differen between "now" and nextRunAt is less than MAX_SIGNED_INTEGER.

Changes

• Added a upper limit on the runIn interval calculation by adding a Math.min computation against MAX_SIGNED_INTEGER (2e31-1)
• Opted to still set a timeout for MAX_SIGNED_INTEGER to ensure no weirdness in long-running servers

Keywords: TimeoutOverflowWarning

[simllll]

This is already fixed in @hokify/agenda already since 2020 :-) see https://github.com/hokify/agenda/blob/1d75092b99c6128c2439921e95fe3171b0e5dec1/src/JobProcessor.ts#L457
you could try this fork instead

[pdfowler]

Thanks @simllll, I've been looking at it recently and am debating my next move

[GorillaCoder]

Problem with this fix is that it still calls setTimeout, for a time far outside the current processing period. If you have a whole bunch of jobs in this state, you will overload Node with work that will not run for at least 25 days. Imagine 10K or 100K+ jobs doing this. That is not an extreme load: 1 job/second * 3600 seconds/hour * 24 hours/day is 86400 jobs/day.

A better fix is to not process the job at all. Wait until the correct processing period to deal with it. This fix is simple, see #1453. Essentially,

diff --git a/lib/agenda/find-and-lock-next-job.ts b/lib/agenda/find-and-lock-next-job.ts
index 04c3ddc..98e85de 100644
--- a/lib/agenda/find-and-lock-next-job.ts
+++ b/lib/agenda/find-and-lock-next-job.ts
@@ -37,6 +37,7 @@ export const findAndLockNextJob = async function (
           },
           {
             lockedAt: { $lte: lockDeadline },
+            nextRunAt: { $lte: this._nextScanAt },
           },
         ],
       },