Regular expressions slowing down a program

Question

I am trying to create a program that parses data from a game chat log. So far, I managed to get the program to work and analyze the data I want, but my problem is that the program is getting slower.

It currently takes 5 seconds to parse a 10 MB text file, and I noticed that it drops to 3 seconds if I add RegexOptions.Compiled to my regex.

I believe that I pointed out the problem to my regular expressions. One line is currently being read 5 times due to 5 regular expressions, so the program will become even slower when I add a later one.

What should I do so that my program does not slow down with a few regular expressions? All suggestions for improving the code are welcome!

if (sender.Equals(ButtonParse)) { var totalShots = 0f; var totalHits = 0f; var misses = 0; var crits = 0; var regDmg = new Regex(@"(?<=\bSystem\b.* You inflicted )\d+.\d", RegexOptions.Compiled); var regMiss = new Regex(@"(?<=\bSystem\b.* Target evaded attack)", RegexOptions.Compiled); var regCrit = new Regex(@"(?<=\bSystem\b.* Critical hit - additional damage)", RegexOptions.Compiled); var regHeal = new Regex(@"(?<=\bSystem\b.* You healed yourself )\d+.\d", RegexOptions.Compiled); var regDmgrec = new Regex(@"(?<=\bSystem\b.* You take )\d+.\d", RegexOptions.Compiled); var dmgList = new List<float>(); //New list for damage values var healList = new List<float>(); //New list for heal values var dmgRecList = new List<float>(); //New list for damage received values using (var sr = new StreamReader(TextBox1.Text)) { while (!sr.EndOfStream) { var line = sr.ReadLine(); var match = regDmg.Match(line); var match2 = regMiss.Match(line); var match3 = regCrit.Match(line); var match4 = regHeal.Match(line); var match5 = regDmgrec.Match(line); if (match.Success) { dmgList.Add(float.Parse(match.Value, CultureInfo.InvariantCulture)); totalShots++; totalHits++; } if (match2.Success) { misses++; totalShots++; } if (match3.Success) { crits++; } if (match4.Success) { healList.Add(float.Parse(match4.Value, CultureInfo.InvariantCulture)); } if (match5.Success) { dmgRecList.Add(float.Parse(match5.Value, CultureInfo.InvariantCulture)); } } TextBlockTotalShots.Text = totalShots.ToString(); //Show total shots TextBlockTotalDmg.Text = dmgList.Sum().ToString("0.##"); //Show total damage inflicted TextBlockTotalHits.Text = totalHits.ToString(); //Show total hits var hitChance = totalHits / totalShots; //Calculate hit chance TextBlockHitChance.Text = hitChance.ToString("P"); //Show hit chance TextBlockTotalMiss.Text = misses.ToString(); //Show total misses var missChance = misses / totalShots; //Calculate miss chance TextBlockMissChance.Text = missChance.ToString("P"); //Show miss chance TextBlockTotalCrits.Text = crits.ToString(); //Show total crits var critChance = crits / totalShots; //Calculate crit chance TextBlockCritChance.Text = critChance.ToString("P"); //Show crit chance TextBlockDmgHealed.Text = healList.Sum().ToString("F1"); //Show damage healed TextBlockDmgReceived.Text = dmgRecList.Sum().ToString("F1"); //Show damage received var pedSpent = dmgList.Sum() / (float.Parse(TextBoxEco.Text, CultureInfo.InvariantCulture) * 100); //Calculate ped spent TextBlockPedSpent.Text = pedSpent.ToString("0.##") + " PED"; //Estimated ped spent } } 

And here is a sample text:

 2014-09-02 23:07:22 [System] [] You inflicted 45.2 points of damage. 2014-09-02 23:07:23 [System] [] You inflicted 45.4 points of damage. 2014-09-02 23:07:24 [System] [] Target evaded attack. 2014-09-02 23:07:25 [System] [] You inflicted 48.4 points of damage. 2014-09-02 23:07:26 [System] [] You inflicted 48.6 points of damage. 2014-10-15 12:39:55 [System] [] Target evaded attack. 2014-10-15 12:39:58 [System] [] You inflicted 56.0 points of damage. 2014-10-15 12:39:59 [System] [] You inflicted 74.6 points of damage. 2014-10-15 12:40:02 [System] [] You inflicted 78.6 points of damage. 2014-10-15 12:40:04 [System] [] Target evaded attack. 2014-10-15 12:40:06 [System] [] You inflicted 66.9 points of damage. 2014-10-15 12:40:08 [System] [] You inflicted 76.2 points of damage. 2014-10-15 12:40:12 [System] [] You take 18.4 points of damage. 2014-10-15 12:40:14 [System] [] You inflicted 76.1 points of damage. 2014-10-15 12:40:17 [System] [] You inflicted 88.5 points of damage. 2014-10-15 12:40:19 [System] [] You inflicted 69.0 points of damage. 2014-10-19 05:56:30 [System] [] Critical hit - additional damage! You inflict 275.4 points of damage. 2014-10-19 05:59:29 [System] [] You inflicted 92.8 points of damage. 2014-10-19 05:59:31 [System] [] Critical hit - additional damage! You inflict 251.5 points of damage. 2014-10-19 05:59:35 [System] [] You take 59.4 points of damage. 2014-10-19 05:59:39 [System] [] You healed yourself 84.0 points. 

Answer 1

These are the problems that I see.

• As suggested in the comments, there is too much work with the regular expression parser for basic template situations.
• Why process data several times in one text? Create one regex pattern to do all the work with one scan per line.
• In WPF, do not delay the GUI thread to do the work, do the work in the background job and update the viewmodel (are you using MVVM correctly?), Which will distribute information to the screen using INotifyPropertyChanged events.

The following is one regex pattern solution that works on a per-line basis. His first task is to verify that [System] contained in the string. If this is not the case, there will be no matching on this line. If he has a system, he searches for specific keywords and possible values ​​and puts them in regex named match captures in a key / value pair situation.

As soon as this is done using linq, it will summarize the values ​​found. Note that I commented on the pattern and the regex engine ignored it.

 string pattern = @"^ # Beginning of line to anchor it. (?=.+\[System\]) # Within the line a literal '[System]' has to occur (?=.+ # Somewhere within that line search for these keywords: (?<Action> # Named Match Capture Group 'Action' will hold a keyword. inflicte?d? # if the line has inflict or inflicted put it into 'Action' | # or evaded # evaded | take # or take | yourself # or yourself (heal) ) (\s(?<Value>[\d.]+))?) # if a value of points exist place into 'Value' .+ # match one or more to complete it. $ #end of line to stop on"; // IgnorePatternWhiteSpace only allows us to comment the pattern. Does not affect processing. var tokens = Regex.Matches(data, pattern, RegexOptions.IgnorePatternWhitespace | RegexOptions.Multiline) .OfType<Match>() .Select( mt => new { Action = mt.Groups["Action"].Value, Value = mt.Groups["Value"].Success ? double.Parse(mt.Groups["Value"].Value) : 0, Count = 1, }) .GroupBy ( itm => itm.Action, // Each action will be grouped into its name for summing itm => itm, // This is value to summed amongst the individual items of the group. (action, values) => new { Action = action, Count = values.Sum (itm => itm.Count), Total = values.Sum(itm => itm.Value) } ); 

Result

Result linq returns each token as an object that sums all the values ​​for the actions, but also counts the number of times these actions occurred.

DATA

 string data=@ "2014-09-02 23:07:22 [System] [] You inflicted 45.2 points of damage. 2014-09-02 23:07:23 [System] [] You inflicted 45.4 points of damage. 2014-09-02 23:07:24 [System] [] Target evaded attack. 2014-09-02 23:07:25 [System] [] You inflicted 48.4 points of damage. 2014-09-02 23:07:26 [System] [] You inflicted 48.6 points of damage. 2014-10-15 12:39:55 [System] [] Target evaded attack. 2014-10-15 12:39:58 [System] [] You inflicted 56.0 points of damage. 2014-10-15 12:39:59 [System] [] You inflicted 74.6 points of damage. 2014-10-15 12:40:02 [System] [] You inflicted 78.6 points of damage. 2014-10-15 12:40:04 [System] [] Target evaded attack. 2014-10-15 12:40:06 [System] [] You inflicted 66.9 points of damage. 2014-10-15 12:40:08 [System] [] You inflicted 76.2 points of damage. 2014-10-15 12:40:12 [System] [] You take 18.4 points of damage. 2014-10-15 12:40:14 [System] [] You inflicted 76.1 points of damage. 2014-10-15 12:40:17 [System] [] You inflicted 88.5 points of damage. 2014-10-15 12:40:19 [System] [] You inflicted 69.0 points of damage. 2014-10-19 05:56:30 [System] [] Critical hit - additional damage! You inflict 275.4 points of damage. 2014-10-19 05:59:29 [System] [] You inflicted 92.8 points of damage. 2014-10-19 05:59:31 [System] [] Critical hit - additional damage! You inflict 251.5 points of damage. 2014-10-19 05:59:35 [System] [] You take 59.4 points of damage. 2014-10-19 05:59:39 [System] [] You healed yourself 84.0 points.";