You will need the ability to create scripts because there is no way to remove a duplicate row without pushing the matching position to that row.
Therefore, you will have to sit in a loop, rebooting from the beginning until all duplicates are deleted.
Perl while ( str ~= s/regex/$1/g ) {} example while ( str ~= s/regex/$1/g ) {}
It can be done. It may take a little extra time, but it is doable.
Anyway, this is a regular expression that you will need to do.
Globally:
Find (?m)((^[^\S\r\n]*?(?=\S)(?:(?!Niveau|stime).)+$)[\S\s]*?)^\2$(?:\r?\n)?
Replace $1
Do this until there are no more globally matches (i.e. replacements)
Clarification:
(?m) # Multi-line mode ( # (1 start), To be written back ( # (2 start), The line to test ^ # BOL begin of line [^\S\r\n]*? # Spurious horizontal whitespace (?= \S ) # Must be a non-whitespace ahead (?: # Skip lines containing these (?! Niveau | stime ) . )+ $ # EOL end of line ) # (2 end) [\S\s]*? # Anything up to the duplicate ) # (1 end) ^ \2 $ # The actual duplicate line (?: \r? \n )? # Optional linebreak (if last line, then ok)
Note that there is now a regular expression, no cropping of horizontal spaces
on BOL and EOL, so the text must be accurate.
It is easy, however, to add additional trim if necessary.
update
A faster version of the above regular expression uses the \K construct to simplify the replacement.
Globally:
Find (?m)(^[^\S\r\n]*?(?=\S)(?:(?!Niveau|stime).)+$)[\S\s]*?\K^\1$(?:\r?\n)?
Replace '' (nothing)
Explanation
(?m) # Multi-line mode ( # (1 start), The line to test ^ # BOL begin of line [^\S\r\n]*? # Spurious horizontal whitespace (?= \S ) # Must be a non-whitespace ahead (?: # Skip lines containing these (?! Niveau | stime ) . )+ $ # EOL end of line ) # (1 end) [\S\s]*? # Anything up to the duplicate \K # Disregard the match up to here ^ \1 $ # The actual duplicate line to be deleted (?: \r? \n )? # Optional linebreak (if last line, then ok)