De-duping shards in ShardsBatchGatewayAllocator based on ShardId inst…

[rajiv-kv]

Description

ShardsBatchGatewayAllocator de-dupes the unassigned Shards based on ShardRouting. If index is configured with muliple replicas, there will be multiple ShardRouting entries for same replica shard. The equals comparison of ShardRouting includes comparison based on <ShardMetada, UnassignedInfo>. This can cause incorrect de-duplication if UnassignedInfo is different for Replica copies. Hence changing the implementation to de-dupe based on ShardId, so that only one entry is created in cache and there are no duplicate entries across batches.

Related Issues

Resolves #[13705]

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• API changes companion pull request created.
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

✅ Gradle check result for 85a748d: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.57%. Comparing base (b15cb0c) to head (8ee9ae4).
Report is 292 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #13710      +/-   ##
============================================
+ Coverage     71.42%   71.57%   +0.15%     
- Complexity    59978    61274    +1296     
============================================
  Files          4985     5060      +75     
  Lines        282275   287856    +5581     
  Branches      40946    41689     +743     
============================================
+ Hits         201603   206023    +4420     
- Misses        63999    64817     +818     
- Partials      16673    17016     +343     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[shiv0408]

Changes look good in perspective of fixing the bug @rajiv-kv. Can we have a follow up issue where we can look into the optimization where if there multiple replicas of a shard, we don't need n round of reroutes/fetches to get them assigned? It would be good to see that optimization in future.

[github-actions]

❌ Gradle check result for e4c0cc6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 8ee9ae4: SUCCESS