[FEATURE] Dynamic pool resizing in FusionCacheMemoryLocker

[engineering87]

@jodydonetti we are in contact on Linkedin, greetings and congratulations for the excellent work on FusionCache.
I just started analyzing the source and my attention fell on the classes FusionCacheMemoryLocker, and ProbabilisticMemoryLocker, perhaps I have an improvement in mind.

Problem

Those classes currently uses a static-size semaphore pool. This could causes overutilization or underutilization of resources in cases of bursty or unpredictable workloads-for instance, the pool is too small-where there is either excessive or minus resource use for fixed-size allocation.

Solution

The solution is to implement dynamic pool management:

• Monitor active lock count and then change pool size based on some usage patterns.
• Pool size is to increase if active lock count exceeds the upper threshold set.
• Pool size is to decrease if active lock falls below a lower threshold set.
• The resizing of the pool should be introduced in a thread-safe way.

Additional context

This could enhance the locking mechanism's overall efficiency in higher concurrency environments.
The pool size could also be modified dynamically based on the observed rate of use so that system performance could be maintained at optimal resource utilization.

[jodydonetti]

Hi @engineering87

greetings and congratulations for the excellent work on FusionCache.

Thanks, appreciate that!

Those classes currently uses a static-size semaphore pool. This could causes overutilization or underutilization of resources in cases of bursty or unpredictable workloads-for instance, the pool is too small-where there is either excessive or minus resource use for fixed-size allocation.

Not really: in the StandardMemoryLocker the actual semaphores to prevent cache stampede are currently handled with an inner cache, and the pool is used just to protect access to that underlying memory cache since it does not have cache stampede protection built in already. The use is minimal.

In the ProbabilisticMemoryLocker instead the pool is fixed, true, but that's the whole point: it's based on probabilistic theory, particularly on the fact that actual contention would probably happen not when 2 cache keys collide on the same semaphore, but when that happen at the same time, and so the probability of actual contention is dramatically reduced.
Btw the probabilistic one is currently just an experiment, and is internal only.

Thoughts?