Description
URL inserts the given URL into the database.
This is useful when you need to add multiple start points (entry points)
to the same server, for example, to index some separate area of your server
which is not referenced from the other documents.
When inserting, indexer does not check the URL against the rules described by
Allow / Disallow filters
and Server / Realm commands.
If the URL is disallowed by filters, or does not have a corresponding
Server or Realm command,
it will be deleted when indexer pulls this URL from the
crawler queue.
The URL command has no effect
if the given URL is already in the database.