All geek questions in one place

How to save dynamic data changes in server cache?

EDIT:The purpose of this site is to call it Utopiapimp.com. This is a third-party game utility called utopia-game.com. At the moment, the site has more than 12 thousand users, and I am launching the site. The game is fully textual and will always remain so. Users copy and paste full pages of text from the game and paste the copied information into my site. I run a series of regular expressions against the inserted data and break them. Then I insert from 5 values ​​to more than 30 values ​​in the database based on one paste. Then I take these values ​​and run queries against them in order to display the information in a VERY simple and understandable form. The game is team based and each team has 25 users. Thus, each team is a group, and each line is ONE user information.Users can update all 25 lines or only one line at a time. I need to store things in the cache, because the site very slowly makes more than 1000 requests almost every minute.

So here is the deal. Imagine I have excel EDIT(Excel is just an example of how to present this, I don't actually use Excel) with 100 columns and 5000 rows. Each line has two unique identifiers. One for a row he himself and one to put together 25 rows a piece. There are about 10 columns in a row that will almost never change, and the remaining 90 columns will always change. We can say that some of them will even change in a matter of seconds depending on how quickly the row is updated. Rows can also be added and removed from a group, but not from a database. Rows are taken from approximately 4 queries from the database to display the latest and updated data from the database. Therefore, every time something in the database is updated, I would also like the row to be updated.If the line or group was not updated after 12 or about hours, it will be deleted from the cache. After the user calls the group again through database queries. They will be cached.

Above was what I would like. This is a desire.

In Reality, I still have all the lines, but the way to store them in the cache currently does not work. I store every line in the class, and the class is stored in the server cache through a huge list. When I go to update / delete / insert items in a list or lines, most of the time it works, but sometimes it throws errors because the cache has changed. I want to be able to lock the cache, because the database locks the lock in the row more or less. I have DateTime stamps to delete things after 12 hours, but it almost always breaks because other users are updating the same 25 lines in the group or just changed the cache.

, Cache, , 10 , . , 12 :

DateTime dt = DateTime.UtcNow;
    if (HttpContext.Current.Cache["GetRows"] != null)
    {
        List<RowIdentifiers> pis = (List<RowIdentifiers>)HttpContext.Current.Cache["GetRows"];
        var ch = (from xx in pis
                  where xx.groupID == groupID 
                  where xx.rowID== rowID
                  select xx).ToList();
        if (ch.Count() == 0)
        {
            var ck = GetInGroupNotCached(rowID, groupID, dt); //Pulling the group from the DB
            for (int i = 0; i < ck.Count(); i++)
                pis.Add(ck[i]);
            pis.RemoveAll((x) => x.updateDateTime < dt.AddHours(-12));
            HttpContext.Current.Cache["GetRows"] = pis;
            return ck;
        }
        else
            return ch;
    }
    else
    {
        var pis = GetInGroupNotCached(rowID, groupID, dt);//Pulling the group from the DB
        HttpContext.Current.Cache["GetRows"] = pis;
        return pis;
    }

, .

, Whats ? , ? ? , .

EDIT: SQLCacheDependency LINQ, Remus. , . Entire Rows, Remus Idea.

.

var ck = (from xx in db.GetInGroupNotCached
              where xx.rowID== rowID
              select new {                 
                  xx.Item,
                  xx.AnotherItem,
                  xx.AnotherItem,
                  }).CacheSql(db, "Item:" + rowID.ToString()).ToList();


var ck = (from xx in db.GetInGroupNotCached
              where xx.rowID== rowID
              select new ClassExample {              
                Item=  xx.Item,
                 AnotherItem= xx.AnotherItem,
                 AnotherItemm = xx.AnotherItemm,
                  }).CacheSql(db, "Item:" + rowID.ToString()).ToList();
+3
4

, . List<T> , O (n).

, , , :

, O (log (n)). , . , - . SQL Server . SQL Server Profiler , . (, GroupId O (n) O (n/25), 25 ).

SQL ( , N + 1, ). .

, , . - , . - , .

+4

, , , ( ) , . , , - -. Cache Hit, Cache, , , . , Cache Hit, , .

, . . 1:1, . , 10 , 90, . , 10, 90, .

, , . , . ------ ( , ). , , CacheDependency , , .

, - , - CacheDependency . , , , . , .

, CacheDependencies, ICacheItemExpiration CachingBlock Enterprise Library. , CachingBlock , ASP.NET . DatabaseExpirationManager, , . , , DatabaseExpirationManager. DatabaseExpirationManager , , . , , , . DatabaseExpirationManager , . , , , .

Ok. -, , . -, , .

backback, 2005 / 2006 , .NET 2.0, , ( ). 2005 ./ 2005 ./ 2006 . 2006 CodePlex.

, , - . , CacheManager. ( Microsoft.Practices.EnterpriseLibrary.Caching): Cache BackgroundScheduler ExpirationPollTimer

Cache - EntLib. BackgroundScheduler . ExpirationPollTimer Timer.

, -, , Cache . . EntLib ASP.NET , , , . , , . -, , .

BackgroundScheduler : DatabaseExpirationWorker DatabaseExpirationManager. DatabaseExpirationManager , :

private object _syncRoot = new object();
private List<Guid>  _objectChanges = new List<Guid>();
public event EventHandler<DatabaseExpirationEventArgs> ExpirationFired;
...
public void UpdateExpirations()
{
    lock ( _syncRoot )
    {
        DataTable dt = GetExpirationsFromDb();
        List<Guid> keys = new List<Guid>();
        foreach ( DataRow dr in dt.Rows )
        {
            Guid key = (Guid)dr[0];
            keys.Add(key);
            _objectChanges.Add(key);
        }

        if ( ExpirationFired != null )
            ExpirationFired(this, new DatabaseExpirationEventArgs(keys));
    }
}

DatabaseExpirationEventArgs :

public class DatabaseExpirationEventArgs : System.EventArgs
{
    public DatabaseExpirationEventArgs( List<Guid> expiredKeys )
    {
        _expiredKeys = expiredKeys;
    }

    private List<Guid> _expiredKeys;
    public List<Guid> ExpiredKeys
    {
        get  {  return _expiredKeys;  }
    }
}

. . PK . , , ( , ), , GetExpirationsFromDb , . , . : - ( datetime IIRC). , . , Guid .

DatabaseExpirationWorker BackgroundScheduler, , DoExpirationTimeoutExpired DatabaseExpirationManager UpdateExpirations. BackgroundScheduler virtual, BackgroundScheduler .

, , EntLib CacheManager, DatabaseExpirationWorker BackgroundScheduler, :

private List<Guid> _objectExpirations;
private void OnExpirationFired( object sender, DatabaseExpirationEventArgs e )
{
    _objectExpirations = e.ExpiredKeys;
    lock(_objectExpirations)
    {
        foreach( Guid key in _objectExpirations)
            this.RealCache.Remove(key);
    }
}

private Microsoft.Practices.EnterpriseLibrary.Caching.CacheManager _realCache;
private Microsoft.Practices.EnterpriseLibrary.Caching.CacheManager RealCache
{
    get
    {
        lock(_syncRoot)    
        {       
            if ( _realCache == null )
                _realCache = Microsoft.Practices.EnterpriseLibrary.Caching.CacheManager.CacheFactory.GetCacheManager();

            return _realCache;
        }
    }
}


public object this[string key]
{
    get
    {
        lock(_objectExpirations)
        {
            if (_objectExpirations.Contains(key))
                return null;
            return this.RealCache.GetData(key);
        }
    }
}

, , , . , , . EntLib, , . , , , , . , , . , .

+4

, . , , "" , SQL-. ...

, , - . rowID groupID, . , , . The Mysterious Notification, , . ASP.Net , , SqlCacheDependency.

+2

, , , , , .

, .
, ...

, - . .
, .
, set, , .
. using() Dispose(), .

Here is a cache class (group) and a row class.
Add database entry after comment// Add code to read from database...

public class GroupCache : SimpleCache<RowObject, int>
{
    private static readonly object GroupCacheObjectLock = new object();

    public GroupCache(int groupId)
    {
        GroupId = groupId;
    }
    public int GroupId { get; private set; }

    public static GroupCache GetGroupCache(int groupId)
    {
        lock (GroupCacheObjectLock)
        {
            if (HttpContext.Current.Cache["Group-" + groupId] == null)
            {
                HttpContext.Current.Cache["Group-" + groupId] 
                    = new GroupCache(groupId);
            }
        }
        return HttpContext.Current.Cache["Group-" + groupId];
    }

    public override RowObject CreateItem(int id, 
            SimpleCache<RowObject, int> cache)
    {
        return new RowObject(id, GroupId, this);
    }

}

public class RowObject : SimpleCacheItem<RowObject, int>
{
    private string _property1;

    public RowObject(int rowId, int groupId, SimpleCache<RowObject, int> cache)
        : base(rowId, cache)
    {
        // Add code to read from database...
    }

    public string Property1
    {
        get { return _property1; }
        set
        {
            if (!AcquireLock(-1)) return;
            _property1 = value;
#if DEBUG
            Trace.WriteLine(string.Format("Thread id: {0}, value = {1}", 
                Thread.CurrentThread.ManagedThreadId, value));
#endif
        }
    }
}

This unit test is mainly to show how to use classes.

[TestFixture]
public class GroupCacheTest
{
    private int _threadFinishedCount;
    private void MultiThreadTestWorker(object obj)
    {
        for (int n = 0; n < 10; n++)
        {
            for (int m = 0; m < 25; m++)
            {
                using (RowObject row 
                    = GroupCache.GetGroupCache(n).GetCachedItem(m))
                {
                    row.Property1 = string.Format("{0} {1} {2}", obj, n, m);
                    Thread.Sleep(3);
                }
            }
        }
        Interlocked.Increment(ref _threadFinishedCount);
    }
    [Test]
    public void MultiThreadTest()
    {
        _threadFinishedCount = 1;
        for (int i = 0; i < 20; i++)
        {
            ThreadPool.QueueUserWorkItem(MultiThreadTestWorker, "Test-" + i);
        }
        while (_threadFinishedCount < 10)
            Thread.Sleep(100);
    }
}

Here are the base classes.

public abstract class SimpleCacheItem<T, TKey> : IDisposable where T : class
{
    private readonly SimpleCache<T, TKey> _cache;

    protected SimpleCacheItem(TKey id, SimpleCache<T, TKey> cache)
    {
        Id = id;
        _cache = cache;
    }

    protected TKey Id { get; private set; }

    #region IDisposable Members

    public virtual void Dispose()
    {
        if (_cache == null) return;
        _cache.ReleaseLock(Id);
    }

    #endregion

    protected bool AcquireLock(int timeout)
    {
        return _cache.AcquireLock(Id, -1);
    }
}

public abstract class SimpleCache<T, TKey> where T : class
{
    private static readonly object CacheItemLockSyncLock = new object();
    private static readonly object CacheItemStoreSyncLock = new object();
    private readonly Dictionary<TKey, int> _cacheItemLock;
    private readonly Dictionary<TKey, T> _cacheItemStore;

    public abstract T CreateItem(TKey id, SimpleCache<T, TKey> cache);

    public T GetCachedItem(TKey id)
    {
        T item;
        lock (CacheItemStoreSyncLock)
        {
            if (!_cacheItemStore.TryGetValue(id, out item))
            {
                item = CreateItem(id, this);
                _cacheItemStore.Add(id, item);
            }
        }
        return item;
    }

    public void ReleaseLock(TKey id)
    {
        lock (CacheItemLockSyncLock)
        {
            if (_cacheItemLock.ContainsKey(id))
            {
                _cacheItemLock.Remove(id);
            }
        }
#if DEBUG
        Trace.WriteLine(string.Format("Thread id: {0} lock released", 
        Thread.CurrentThread.ManagedThreadId));
#endif
    }

    public bool AcquireLock(TKey id, int timeOut)
    {
        var timer = new Stopwatch();
        timer.Start();
        while (timeOut < 0 || timeOut < timer.ElapsedMilliseconds)
        {
            lock (CacheItemLockSyncLock)
            {
                int threadId;
                if (!_cacheItemLock.TryGetValue(id, out threadId))
                {
                    _cacheItemLock.Add(id, 
                        Thread.CurrentThread.ManagedThreadId);
#if DEBUG
                    Trace.WriteLine(string.Format(
                        "Thread id: {0}, lock acquired after {1} ms", 
                        Thread.CurrentThread.ManagedThreadId, 
                        timer.ElapsedMilliseconds));
#endif
                    return true;
                }
                if (threadId == Thread.CurrentThread.ManagedThreadId) 
                    return true;
            }
            Thread.Sleep(15);
        }
        return false;
    }
}
+1
source

Source: https://habr.com/ru/post/1740196/

More articles:


All Articles