Description
Use this to allow to index URLs that matches/doesn't match the
given argument. First three optional parameters describe the type of
comparison. Default values are Match, NoCase, String.
Use "NoCase" or "Case" values to choose case insensitive or case sensitive
comparison respectively.
Use "Regex" to choose regular expression comparison.
Use "String" to choose string with wildcards comparison.
Widlcards are "*" for any number of characters and "?" for one character.
Note that "?" and "*" have special meanings in "String" match type. Please use
"Regex" to describe documents with "?" and "*" signs in URL.
"String" match is much faster than "Regex". Use "String" where it
is possible.
You may use several arguments for one "Allow"' command.
You may use this command whenever you want.
This command takes global effect for whole configuration file.
Note that mnoGoSearch automatically adds one "Allow regex .*"
command after reading configuration file. It means it allows everything
that is not disallowed.