Key value checking when saving job data

[1YalcinSahin1]

If the payload value of a data set previously added to the queue does not appear to be the same, I want to update the content.

I tried to create a similar structure with the code below.

    protected function pushToDatabase($queue, $payload, $delay = 0, $attempts = 0, $key = null)
    {
        if (!empty($key)) {
            if (Cache::has($key)) {
                $jobID = Cache::get($key);
            } else {
                $database_control_key = $this->database->table($this->table)->where('key', $key)->get(['id', 'attempts']);

                if ($database_control_key->isEmpty()) {
                    $jobID = null;
                } else {
                    $jobID = $database_control_key[0]->id;
                }
            }

            if (!empty($jobID)) {
                $this->database->table($this->table)->where('id', $jobID)
                    ->update($this->buildDatabaseRecord(
                        $this->getQueue($queue), $payload, $this->availableAt($delay), $attempts, $key
                    ));
            } else {
                $jobID = $this->database->table($this->table)->insertGetId($this->buildDatabaseRecord(
                    $this->getQueue($queue), $payload, $this->availableAt($delay), $attempts, $key
                ));

                Cache::put('job-' . $key, $jobID, Carbon::now()->addHours(3));
            }
            return $jobID;
        } else {
            return $this->database->table($this->table)->insertGetId($this->buildDatabaseRecord(
                $this->getQueue($queue), $payload, $this->availableAt($delay), $attempts
            ));
        }
    }

Job:

   public function handle(): void
    {
        Cache::forget('job-' . $this->LockKey);
    }

Class:

                    $lock = Cache::lock(
                        $this->LockKey,
                        $this->optionsExtra[OptionsExtra::cacheLockTime]
                    );
                  
                    if ($lock->get()) {
                        $curl_list_product->result[$i]->Owner = $lock->owner();

                        $job = (new TestJob($curl_list_product->result[$i], $this->options))
                            ->onQueue($this->options[Options::onQueueName])
                            ->setKey($this->LockKey);
                  
                        $jobID = app(Dispatcher::class)->dispatch($job);
                    }

If there is a job waiting in the queue for a long time, isn't there an easy way to update it with new data?

[henzeb]

you should not store data subject to change in a job. That is my idea of it. Although I sometimes do, but it's more a matter of putting the job back on the queue, for continuing processing it at a later time.

[1YalcinSahin1]

@henzeb I can give you an example like this.You have a specific list of products.You are checking for price or stock changes on these products.You cannot employ more than one worker on jobs because of a deadlock. With FIFO logic, only one worker works.After a certain period of time, the job count in the queue becomes overloaded.I do not want it to be used with old data in a job that has been waiting for a long time.

I would be very happy if you suggest a different solution to this issue :)

[henzeb]

as I said, don't store data subject to change in a job. You should have ID's and then use those to retrieve the data you want to work on in that job....

[1YalcinSahin1]

Since I got the data set from another source, I cannot do what you said. What I actually want to do with the lock key logic is the same. If the content of the job for which I know the lock key is still in the queue, I want to update the data.What I want to do here is to process the data I received from another source over a longer period of time.What I'm trying to imagine right now makes sense to me. I don't fully understand your hesitation here.

[1YalcinSahin1]

I would also like to point out this additionally.I have a wide variety of clients. I have to constantly check their product lists at 10-minute intervals.Each job has its own special queue and table.In order not to put too much load on the RAM, I queue them and check them one by one.