Seem reindexing is Flusing magento page cache

[ia-gaurav]

Preconditions and environment

We are using Redis for cache and page cache , after reindexing redis seems flushing page cache in redis

Steps to reproduce

1: wait for redis page cache size to grow
2: Key in page_cahe of redis is around 20 k
3: we did reindexing on magento php bin/magento index:reindex
4: Keys in page_cache went down from 20 k to around 1.5k

Expected result

Reindexing should not flush page_cache in redis

Actual result

Page_cache in redis is flushed with number of keys in page_cache database of redis goes down

Additional information

No response

Release note

No response

Triage and priority

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant]

Hi @ia-gaurav. Thank you for your report.
To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

• @magento give me 2.4-develop instance - upcoming 2.4.x release
• For more details, review the Magento Contributor Assistant documentation.
• Add a comment to assign the issue: @magento I am working on this
• To learn more about issue processing workflow, refer to the Code Contributions.

---

Join Magento Community Engineering Slack and ask your questions in #github channel.
⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.
🕙 You can find the schedule on the Magento Community Calendar page.
📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

[m2-assistant]

Hi @engcom-Delta. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue.
• 3. Add Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
• 4. Verify that the issue is reproducible on 2.4-develop branch
  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
• 5. Add label Issue: Confirmed once verification is complete.
• 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Delta]

Hi @ia-gaurav ,

Verified the issue on 2.4-develop and we see there is a decrement in the cache keys but we think it is as intended, as it is starting afresh.

Steps to reproduce:-

1: wait for redis page cache size to grow
2: Verify Key in page_cache of redis
3: Perform reindex on magento php bin/magento index:reindex
4: Keys in page_cache went down

We want know which keys shouldn't be deleted or cleared as particular, so that it will be easy to debug.

Please refer the attached screenshots

redis cache before reindexing

perform reindexing

redis cache after reindexing

[engcom-Delta]

Hi @ia-gaurav ,

Waiting for your reply on this issue to move further.

Thanks,

[ia-gaurav]

yes - we should look for way to clear specific categroy or detail page cache instead of full cache flush,

Also for product details page is there way to ignore utm and show cached data

[hostep]

yes - we should look for way to clear specific categroy or detail page cache instead of full cache flush,

If you do a full reindex, it will go over every single product & category, so it makes sense that almost the entire cache will be emptied out. But a full reindex is something that almost never should happen on a production environment.

During normal operation, where you don't run a full reindex all the time, only products/categories that are changed will then cause a partial reindex and only a small amount of cache should get flushed.

Also for product details page is there way to ignore utm and show cached data

No idea if this is possible with Redis page caching, probably not, but switching to Varnish for FPC and using the default VCL Magento ships with, should already ignore the utm param from the query string and don't cause double unnecessary cache entries:

magento2/app/code/Magento/PageCache/etc/varnish6.vcl

Lines 84 to 88 in c893121

	# Remove all marketing get parameters to minimize the cache objects
	if (req.url ~ "(\?|&)(gclid|cx|ie|cof|siteurl|zanpid|origin|fbclid|mc_[a-z]+|utm_[a-z]+|_bta_[a-z]+)=") {
	set req.url = regsuball(req.url, "(gclid|cx|ie|cof|siteurl|zanpid|origin|fbclid|mc_[a-z]+|utm_[a-z]+|_bta_[a-z]+)=[-_A-z0-9+()%.]+&?", "");
	set req.url = regsub(req.url, "[?|&]+$", "");
	}

[ia-gaurav]

@hostep thank for you valuable information ,

1: is there way i can ignore marking query string using redis FPC
2: is there way to do reindeing for specific category ids only

[hostep]

1. probably not, unless you write some custom code yourself (or find a module that has already done that)
2. yes, don't run a full reindex manually and let the system handle it for you (when indexers are set to "update by schedule" mode, and you've setup the crontab correctly to run once per minute, than the indexers will only index entities (products/categories) that were recently changed)

[ia-gaurav]

@engcom-Delta @engcom-Hotel - can you please guide how to ignore marketing query string for Redis FPC also can this be added in future releases as it cause issues with high server response time