From WikiChip
Editing mirc/regex

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.

This page supports semantic in-text annotations (e.g. "[[Is specified as::World Heritage Site]]") to build structured and queryable content provided by Semantic MediaWiki. For a comprehensive description on how to use annotations or the #ask parser function, please have a look at the getting started, in-text annotation, or inline queries help pages.

Latest revision Your text
Line 1: Line 1:
 
{{mirc title|RegEx}}
 
{{mirc title|RegEx}}
 
Regular expressions, from here on referred to as '''regex''', can be used to perform complicated pattern matching operations. Users should already be familiar with, and comfortable using, Regular expressions at this point. The [[regular expressions|Regular Expressions]] page contains more detailed information for users who are new to regex but a reminder of the syntax will be provided at the end.
 
Regular expressions, from here on referred to as '''regex''', can be used to perform complicated pattern matching operations. Users should already be familiar with, and comfortable using, Regular expressions at this point. The [[regular expressions|Regular Expressions]] page contains more detailed information for users who are new to regex but a reminder of the syntax will be provided at the end.
 
'''Note''': a back reference is the same as a capturing group.
 
  
 
== General Information ==
 
== General Information ==
mIRC uses the 8bits version of [[PCRE library]] to implement regex with the following options enabled:
+
mIRC uses the [[PCRE library]] to implement regex with the following options enabled:
  
 
* --enable-utf8
 
* --enable-utf8
Line 11: Line 9:
 
* --with-match-limit - around 1,000,000
 
* --with-match-limit - around 1,000,000
 
* --with-match-limit-recursion - 999
 
* --with-match-limit-recursion - 999
 
The newline sequence by default is $lf.
 
  
 
mIRC has four custom modifiers for regex:
 
mIRC has four custom modifiers for regex:
Line 20: Line 16:
 
* u - Enable the (*UCP) option (makes \b and \w works for unicode letters) and the (*UTF8) option (forces the string and pattern to be interpreted as utf8).  
 
* u - Enable the (*UCP) option (makes \b and \w works for unicode letters) and the (*UTF8) option (forces the string and pattern to be interpreted as utf8).  
  
mIRC does not allow you to retrieve the full match. You can access simple captures but you cannot access named capture. Here is an alias which allow you to get the full match (made by jaytea):
+
mIRC also supports 3 custom markers inside a pattern:
 
 
<source lang="mIRC">
 
;$regexm(input,regex,Nth).pos returns the Nth full match or its position
 
alias regexm {
 
  noop $regex(full, $2, /^(?|m(.)?|(/)|^)(.*?)(?:\1(?!.*\1)(.*)$)?$/usD)
 
  noop $regex(pcre, $regmlex(full, 1, 2), /^((?:(?!\((?:(?:MARK|PRUNE|SKIP|THEN|\*(?=:))(?::[^()]*)?|ACCEPT|COMMIT)\))\(\*.*?\))*)(.*)/us)
 
  if (!$regex( , $+(/, $regmlex(pcre, 1, 1), |, $regmlex(pcre, 1, 2), /))) {
 
    echo -eagc i * $!regexm: Invalid expression ( $+ $regerrstr $+ )
 
    return
 
  }
 
  var %char, %exp
 
  while ($chr($r(2048, 55295)) isin $1) /
 
  %char = $v1
 
  var %exp = $+(m, $regmlex(full, 1, 1), $regmlex(pcre, 1, 1), (?: $+ $regmlex(pcre, 1, 2) $+ \E)(?(R)|\K), $regmlex(full, 1, 1), $regmlex(full, 1, 3))
 
  var %str = $regsubex($1, %exp, %char) .
 
  if (!$pos(%str, %char, $regex($1, $2))) {
 
    noop $regex(check, $regsubex($1, $2, %char), / %char ( %char ?)/gxu)
 
    %str = $regsubex(fix, $left(%str, -2), / %char \K/gxu, $regml(check, \n)) .
 
  }
 
  noop $regex(final, $left(%str, -2), $+(/\Q, $replacecs($regsubex($1, $2, %char), \E, \E\\E\Q, %char, \E(.*?)\Q $+ %char), \E/u))
 
  if ($prop == pos) && ($3. isnum 1- $regml(final, 0)) return $calc(1 + $regml(final, $3).pos - $3)
 
  returnex $regml(final, $3 1)
 
}</source>
 
  
 +
* \cc - will match $chr(3) (control code for the color sequence)
 +
* \co - will match $chr(15) (control code for ending a color sequence)
 +
* \cb - will match $chr(2) (control code for the bold sequence)
  
mIRC remembers up to 50 regex matches. After 50 matches, the first match is overwritten and the backreferences for that match are lost.
+
'''Note:''' mIRC remembers up to 50 regex matches. After 50 matches, the first match is overwritten and the backreferences for that match are lost.
  
 
Example:  
 
Example:  
Line 52: Line 28:
 
/(This is) (a ) pattern/
 
/(This is) (a ) pattern/
  
This represent one regular expression (or one pattern) with 2 captured backreferences.
+
This represent one regular expression (or one match) with 2 captured backreferences.
 
 
=== Sanitizing the input ===
 
 
 
If you pass value dynamically in a regex pattern, you have to escape any characters in such value that could otherwise be interpreted by the PCRE regex engine.
 
 
 
Instead of escaping a bunch of characters with \, which is supported, PCRE allows for the \Q \E construct to not interpret anything between that construct:
 
 
 
Looking for a match on a nickname such as nick[name], you could use /\b $+ \Q $+ $nick $+ \E\b/
 
 
 
This is good but still has one flaw, if $nick contains \E, it will terminate the escaping sequence and subsequent character would be interpreted by PCRE.
 
 
 
To solve the issue, you have to "escape" all the \E inside the value you're using: /b\Q $+ $replacecs($nick,\E,\E\\E\Q) $+ \E\b/
 
 
 
There's no way to escape a character inside the escaping sequence \Q\E, you have to first terminate the sequence yourself with \E, then you have to match the actual \E from the input with \\E, and then start a new escape sequence with \Q.
 
  
 
== Regex Identifiers ==
 
== Regex Identifiers ==
Line 77: Line 39:
  
 
If the /g modifier is used, that number can be greater than 1.
 
If the /g modifier is used, that number can be greater than 1.
You may see a negative value being returned if an error occured, the list below is not exhaustive, but the error ommited are the one that can only happen because of a bug in mIRC in the way it uses pcre, which should never happen in practice.
+
You may see a negative value being returned if an error occured:
 
 
* -5 - rare, the compiled pattern has an error in it, could be due to a bug in pcre or because the pattern has been overwritten
 
* -6 - memory could not be allocated
 
* -7 - memory could not be allocated (specific to retrieving capturing groups)
 
 
* -8 - you reached the maximum number of backtracks allowed, for example: $regex($str(a,4000),(a+a+)*b)
 
* -8 - you reached the maximum number of backtracks allowed, for example: $regex($str(a,4000),(a+a+)*b)
* -14 - mIRC uses this value when compiling the pattern fails but it's a valid value meaning an internal error occured as well. If this error is returned because compiling the pattern failed, {{mIRC|$regerrstr}} is set with an error string, otherwise -14 is the pcre meaning, which is more or less the same as -5 but happens elsewhere in the code.
 
 
* -21 - you reached an internal recursion limit, for example: $regex($str(a, 1000), (a)+)
 
* -21 - you reached an internal recursion limit, for example: $regex($str(a, 1000), (a)+)
* -26 - this error is returned when pcre detects a recursion loop within the pattern. Specifically, it means that either the whole pattern or a subpattern has been called recursively for the second time at the same position in the input string. Some simple patterns that might do this are detected and faulted at compile time, but more  complicated cases, in particular mutual recursions between two different subpatterns, cannot be detected until run time. ex: $regex(,((?2))((?3))((?1)))
 
  
 
=== $regml ===
 
=== $regml ===
Line 97: Line 53:
  
 
'''Note''': $regml is a list of all captures accross all the matches made (/g modifier), which is often enough, but can be a problem in some cases.
 
'''Note''': $regml is a list of all captures accross all the matches made (/g modifier), which is often enough, but can be a problem in some cases.
<source lang="mIRC">//noop $regex(name, teasat, /([es])(a)/g) | echo -a $regml(name, 0) : $regml(name, 1) -- $regml(name, 2) -- $regml(name,3) -- $regml(name,4)</source>would display "4 : e -- a -- s -- a "
+
<source lang="mIRC">//noop $regex(name, teasat, /([es])(a)/g) | echo -a $regml(name, 0) : $regml(name, 1) -- $regml(name, 2) -- $regml(name,3) -- $regml(name,4)</source>would display "2 : e -- a -- s -- a "
You can now access the Nth captured group for a given match number with {{mIRC|$regmlex}}.
+
You can now access the Nth captured group for a given match number with $regmlex.
  
 
=== $regmlex ===
 
=== $regmlex ===
Line 119: Line 75:
 
This time, <subtext> is evaluated during substitution so you can use %variables and $identifiers there, they will be evaluated.
 
This time, <subtext> is evaluated during substitution so you can use %variables and $identifiers there, they will be evaluated.
  
'''Note''': You can now use $regsubex the same way as $regsub to get it to return the number of match and filling a %variable with the result, the name is required, both also supports output to a binvar: $regsubex(name,<input>,<regex>,<subtext>,%var|&binvar)
+
'''Note''': You can now use $regsubex the same way as $regsub to get it to return the number of match and filling a %variable with the result, the name is required: $regsubex(name,<input>,<regex>,<subtext>,%var)
  
 
==== Markers, $1- and Nested $regsubex calls ====
 
==== Markers, $1- and Nested $regsubex calls ====
Line 241: Line 197:
 
So how do you use the value of the marker of the outer $regsubex inside the subtext of the inner $regsubex?
 
So how do you use the value of the marker of the outer $regsubex inside the subtext of the inner $regsubex?
  
The solution is to use '''<nowiki>[[ \t ]]</nowiki>''':
+
The solution is to use <nowiki>[[ \t ]]</nowiki>:
  
 
<source lang="mIRC">$regsubex(name,abcdefcdab,/(cd)/g,$regsubex(\t,/(.)/g,$upper( [[ \t ]] )))</source>
 
<source lang="mIRC">$regsubex(name,abcdefcdab,/(cd)/g,$regsubex(\t,/(.)/g,$upper( [[ \t ]] )))</source>
Line 283: Line 239:
  
 
The metacharacters are as follows:
 
The metacharacters are as follows:
 
+
* '''\''' -- general escape character with several uses
\   general escape character with several uses
+
* '''^''' -- assert start of string (or line, in multiline mode)
^   assert start of string (or line, in multiline mode)
+
* '''$''' -- assert end of string (or line, in multiline mode)
$   assert end of string (or line, in multiline mode)
+
* '''.''' -- match any character except newline (by default)
.   match any character except newline (by default)
+
* '''[''' -- start character class definition
[   start character class definition
+
* '''|''' -- start of alternative branch
|   start of alternative branch
+
* '''(''' -- start subpattern
(   start subpattern
+
* ''')''' -- end subpattern
)   end subpattern
+
* '''?''' -- extends the meaning of '(', also 0 or 1 quantifier, also "quantifier minimizer"
?   extends the meaning of '(', also 0 or 1 quantifier, also "quantifier minimizer"
+
* '''*''' -- 0 or more quantifier
*   0 or more quantifier
+
* '''+''' -- 1 or more quantifier, also "possessive quantifier"
+   1 or more quantifier, also "possessive quantifier"
+
* '''{''' -- start min/max quantifier
{   start min/max quantifier
 
  
 
=== Inside a character class ===
 
=== Inside a character class ===
Line 302: Line 257:
  
 
In a character class the only metacharacters are:
 
In a character class the only metacharacters are:
 +
* '''\''' -- general escape character
 +
* '''^''' -- negate the class, but only if the first character
 +
* '''-''' -- indicates character range
 +
* '''[''' -- POSIX character class (only if followed by POSIX syntax)
 +
* ''']''' -- terminates the character class
  
\  general escape character
 
^  negate the class, but only if the first character
 
-  indicates character range
 
[  POSIX character class (only if followed by POSIX syntax)
 
]  terminates the character class
 
  
 
=== The Backslash \ ===
 
=== The Backslash \ ===
Line 315: Line 270:
 
This use of backslash as an escape character applies both inside and outside character classes.
 
This use of backslash as an escape character applies both inside and outside character classes.
  
Another usage of the backslash is to represent non printable character, they can all be used inside a character class:
+
Another usage of the backslash is to represent non printable character:
 
 
\a        Bel character, ascii 07
 
\cx        where x is any ascii character. The  precise effect of \cx on ASCII characters is as follows: if x is a lower case letter, it is converted to upper case. Then bit 6  of  the character (hex 40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and  \c;  becomes hex  7B (; is 3B). If the data item (byte or 16-bit value) following \c has a value greater than 127, a compile-time error occurs.  This locks out non-ASCII characters.
 
\e escape character, ascii 27
 
\f        form feed character, ascii 12
 
\n        linefeed character, ascii 10
 
\r        carriage return character, ascii 13
 
\t        tab character, ascii 09
 
\0dd      character with octal code 0dd. After \0, up to two further octal digits are read. If there are fewer than two digits, just those that are present are used. Thus the sequence \0\x\015 specifies two binary zeros followed by a CR character. Make sure you supply two digits after the initial zero if the pattern character that follows is itself an octal digit.
 
\ddd      character with octal code ddd, or back reference
 
\o{ddd..} character with octal code ddd..  The escape \o must be followed by a sequence of octal digits, enclosed in braces. An error occurs if this is not the case. This escape is an addition to Perl; it provides way of specifying character code points as octal numbers greater than 0777, and it also allows octal numbers and back references to be unambiguously specified.
 
\xhh      character with hex code hh
 
\x{hhh..} character with hex code hhh..
 
 
 
For greater clarity and unambiguity, it is best to avoid following \ by a digit greater than zero. Instead, use \o{} or \x{} to specify character numbers, and \g{} to specify back references. The  following  paragraphs describe the old, ambiguous syntax.
 
 
 
The handling of a backslash followed by a digit other than 0 is complicated, and Perl has changed in recent releases, causing PCRE also to change.
 
 
 
Outside a character class, PCRE reads the digit and any following digits as a decimal number.
 
If the number is less than 8, or if there  have been at least that many previous capturing left parentheses in the expression, the entire sequence is taken as a back reference.
 
 
 
Inside a character class, or if the decimal number following \ is greater than 7 and there have not been that many capturing subpatterns, PCRE handles \8 and \9 as the literal characters "8" and "9", and otherwise re-reads up to three octal digits following the backslash, using them to generate a data character. Any subsequent digits  stand for themselves.
 
 
 
For example:
 
 
 
        \040  is another way of writing an ASCII space
 
        \40    is the same, provided there are fewer than 40 previous capturing subpatterns
 
        \7    is always a back reference
 
        \11    might be a back reference, or another way of writing a tab
 
        \011  is always a tab
 
        \0113  is a tab followed by the character "3"
 
        \113  might be a back reference, otherwise the character with octal code 113
 
        \377  might be a back reference, otherwise the value 255 (decimal)
 
        \81    is either a back reference, or the two characters "8" and "1"
 
 
 
Note that octal values of 100 or greater that are specified using this syntax must not be introduced by a leading zero, because no more than three octal digits are ever read.
 
 
 
By  default, after \x that is not followed by {, from zero to two hexadecimal digits are read (letters can be in upper or lower  case).
 
Any number of hexadecimal digits may appear between \x{ and }. If a character other than a hexadecimal digit appears between \x{  and  }, or if there is no terminating }, an error occurs.
 
 
 
Characters that are specified using octal or hexadecimal numbers are limited to certain values, less than 0x10ffff and a valid codepoint.
 
     
 
Invalid Unicode codepoints are the range  0xd800  to  0xdfff  (the  so called "surrogate" codepoints), and 0xffef.
 
 
 
All the sequences that define a single character value can be used both inside and outside character classes. In addition, inside a character class, \b is interpreted as the backspace character (ascii 08).
 
 
 
\N matches a non newline character, (same as the dot '.' without single line mode (/s modifier), it is not allowed inside a character class.
 
 
 
\g+N or \g-N is a relative back reference, it matches the value of the capturing group that can be found by counting as many opening parentheses of named or numbered capturing groups as specified by the number from right to left starting at the backreference. (a)(b)(c)(d)\g<-3> matches abcdb.
 
 
 
Another use of backslash is for specifying generic character types:
 
  
        \d    any decimal digit
+
* \a        Bel character, ascii 07
        \D    any character that is not a decimal digit
+
* \cx        where x is any ascii character. The  precise effect of \cx on ASCII characters is as follows: if x is a lower case letter, it is converted to upper case. Then bit 6  of  the character (hex 40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and  \c;  becomes hex  7B (; is 3B). If the data item (byte or 16-bit value) following \c has a value greater than 127, a compile-time error occurs.  This locks out non-ASCII characters.
        \h    any horizontal white space character
+
* \e escape character, ascii 27
        \H    any character that is not a horizontal white space character
+
* \f        form feed character (hex 0C)
        \s    any white space character
+
* \n        linefeed (hex 0A)
        \S    any character that is not a white space character
+
* \r        carriage return (hex 0D)
        \v    any vertical white space character
+
* \t        tab (hex 09)
         \V    any character that is not a vertical white space character
+
* \0dd      character with octal code 0dd
         \w    any "word" character
+
* \ddd      character with octal code ddd, or back reference
         \W    any "non-word" character
+
* \o{ddd..} character with octal code ddd..
[[Category:mIRC|regex]]
+
         \xhh      character with hex code hh
 +
         \x{hhh..} character with hex code hhh.. (non-JavaScript mode)
 +
         \uhhhh    character with hex code hhhh (JavaScript mode only)

Please note that all contributions to WikiChip may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see WikiChip:Copyrights for details). Do not submit copyrighted work without permission!

Cancel | Editing help (opens in new window)