.\" Automatically generated by Pod::Man 4.11 (Pod::Simple 3.35)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "LWP::Parallel::UserAgent 3"
.TH LWP::Parallel::UserAgent 3 "2016-05-29" "perl v5.26.3" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
LWP::Parallel::UserAgent \- A class for parallel User Agents
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
.Vb 3
\&  require LWP::Parallel::UserAgent;
\&  $ua = LWP::Parallel::UserAgent\->new();
\&  ...
\&
\&  $ua\->redirect (0); # prevents automatic following of redirects
\&  $ua\->max_hosts(5); # sets maximum number of locations accessed in parallel
\&  $ua\->max_req  (5); # sets maximum number of parallel requests per host
\&  ...
\&  $ua\->register ($request); # or
\&  $ua\->register ($request, \*(Aq/tmp/sss\*(Aq); # or
\&  $ua\->register ($request, \e&callback, 4096);
\&  ...
\&  $ua\->wait ( $timeout ); 
\&  ...
\&  sub callback { my($data, $response, $protocol) = @_; .... }
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
This class implements a user agent that access web sources in parallel.
.PP
Using a \fILWP::Parallel::UserAgent\fR as your user agent, you typically start by
registering your requests, along with how you want the Agent to process 
the incoming results (see \f(CW$ua\fR\->register).
.PP
Then you wait for the results by calling \f(CW$ua\fR\->wait.  This method only
returns, if all requests have returned an answer, or the Agent timed
out.  Also, individual callback functions might indicate that the
Agent should stop waiting for requests and return. (see \f(CW$ua\fR\->register)
.PP
See the file LWP::Parallel for a set of simple examples.
.SH "METHODS"
.IX Header "METHODS"
The LWP::Parallel::UserAgent is a sub-class of LWP::UserAgent, but not all
of its methods are available here. However, you can use its main
methods, \f(CW$ua\fR\->simple_request and \f(CW$ua\fR\->request, in order to simulate 
singular access with this package. Of course, if a single request is all
you need, then you should probably use LWP::UserAgent in the first place,
since it will be faster than our emulation here.
.PP
For parallel access, you will need to use the new methods that come with
LWP::Parallel::UserAgent, called \f(CW$pua\fR\->register and \f(CW$pua\fR\->wait. See below
for more information on each method.
.ie n .IP "$ua = LWP::Parallel::UserAgent\->\fBnew()\fR;" 4
.el .IP "\f(CW$ua\fR = LWP::Parallel::UserAgent\->\fBnew()\fR;" 4
.IX Item "$ua = LWP::Parallel::UserAgent->new();"
Constructor for the parallel UserAgent.  Returns a reference to a
LWP::Parallel::UserAgent object.
.Sp
Optionally, you can give it an existing LWP::Parallel::UserAgent (or 
even an LWP::UserAgent) as a first argument, and it will \*(L"clone\*(R" a
new one from this (This just copies the behavior of LWP::UserAgent.
I have never actually tried this, so let me know if this does not do
what you want).
.ie n .IP "$ua\->initialize;" 4
.el .IP "\f(CW$ua\fR\->initialize;" 4
.IX Item "$ua->initialize;"
Takes no arguments and initializes the UserAgent. It is automatically
called in LWP::Parallel::UserAgent::new, so usually there is no need to
call this explicitly.
.Sp
However, if you want to re-use the same UserAgent object for a number
of \*(L"runs\*(R", you should call \f(CW$ua\fR\->initialize after you have processed the
results of the previous call to \f(CW$ua\fR\->wait, but before registering any
new requests.
.ie n .IP "$ua\->redirect ( $ok )" 4
.el .IP "\f(CW$ua\fR\->redirect ( \f(CW$ok\fR )" 4
.IX Item "$ua->redirect ( $ok )"
Changes the default value for permitting Parallel::UserAgent to follow
redirects and authentication-requests.  The standard value is 'true'.
.Sp
See \f(CW\*(C`$ua\-\*(C'\fRregister> for how to change the behaviour for particular
requests only.
.ie n .IP "$ua\->nonblock ( $ok )" 4
.el .IP "\f(CW$ua\fR\->nonblock ( \f(CW$ok\fR )" 4
.IX Item "$ua->nonblock ( $ok )"
Per default, LWP::Parallel will connect to a site using a blocking call. If
you want to speed this step up, you can try the new non-blocking version of 
the connect call by setting \f(CW$ua\fR\->nonblock to 'true'. 
The standard value is 'false' (although this might change in the future if
nonblocking connects turn out to be stable enough.)
.ie n .IP "$ua\->duplicates ( $ok )" 4
.el .IP "\f(CW$ua\fR\->duplicates ( \f(CW$ok\fR )" 4
.IX Item "$ua->duplicates ( $ok )"
Changes the default value for permitting Parallel::UserAgent to ignore
duplicate requests.  The standard value is 'false'.
.ie n .IP "$ua\->in_order ( $ok )" 4
.el .IP "\f(CW$ua\fR\->in_order ( \f(CW$ok\fR )" 4
.IX Item "$ua->in_order ( $ok )"
Changes the default value to restricting Parallel::UserAgent to
connect to the registered sites in the order they were registered. The
default value \s-1FALSE\s0 allows Parallel::UserAgent to make the connections
in an apparently random order.
.ie n .IP "$ua\->remember_failures ( $yes )" 4
.el .IP "\f(CW$ua\fR\->remember_failures ( \f(CW$yes\fR )" 4
.IX Item "$ua->remember_failures ( $yes )"
If set to one, enables ParalleUA to ignore requests or connections to
sites that it failed to connect to before during this \*(L"run\*(R". If set to
zero (the dafault) Parallel::UserAgent will try to connect to every
single \s-1URL\s0 you registered, even if it constantly fails to connect to a
particular site.
.ie n .IP "$ua\->max_hosts ( $max )" 4
.el .IP "\f(CW$ua\fR\->max_hosts ( \f(CW$max\fR )" 4
.IX Item "$ua->max_hosts ( $max )"
Changes the maximum number of locations accessed in parallel. The
default value is 7.
.Sp
Note: Although it says 'host', it really means 'netloc/server'! That
is, multiple server on the same host (i.e. one server running on port
80, the other one on port 6060) will count as two 'hosts'.
.ie n .IP "$ua\->max_req ( $max )" 4
.el .IP "\f(CW$ua\fR\->max_req ( \f(CW$max\fR )" 4
.IX Item "$ua->max_req ( $max )"
Changes the maximum number of requests issued per host in
parallel. The default value is 5.
.ie n .IP "$ua\->register ( $request [, $arg [, $size [, $redirect_ok]]] )" 4
.el .IP "\f(CW$ua\fR\->register ( \f(CW$request\fR [, \f(CW$arg\fR [, \f(CW$size\fR [, \f(CW$redirect_ok\fR]]] )" 4
.IX Item "$ua->register ( $request [, $arg [, $size [, $redirect_ok]]] )"
Registers the given request with the User Agent.  In case of an error,
a \f(CW\*(C`HTTP::Request\*(C'\fR object containing the HTML-Error message is
returned.  Otherwise (that is, in case of a success) it will return
undef.
.Sp
The \f(CW$request\fR should be a reference to a \f(CW\*(C`HTTP::Request\*(C'\fR object
with values defined for at least the \fBmethod()\fR and \fBurl()\fR attributes.
.Sp
\&\f(CW$size\fR specifies the number of bytes Parallel::UserAgent should try
to read each time some new data arrives.  Setting it to '0' or 'undef'
will make Parallel::UserAgent use the default. (8k)
.Sp
Specifying \f(CW$redirect_ok\fR will alter the redirection behaviour for
this particular request only. '1' or any other true value will force
Parallel::UserAgent to follow redirects, even if the default is set to
\&'no_redirect'. (see \f(CW\*(C`$ua\-\*(C'\fRredirect>) '0' or any other false value
should do the reverse. See LWP::UserAgent for using an object's
\&\f(CW\*(C`requests_redirectable\*(C'\fR list for fine-tuning this behavior.
.Sp
If \f(CW$arg\fR is a scalar it is taken as a filename where the content of
the response is stored.
.Sp
If \f(CW$arg\fR is a reference to a subroutine, then this routine is called
as chunks of the content is received.  An optional \f(CW$size\fR argument
is taken as a hint for an appropriate chunk size. The callback
function is called with 3 arguments: the data received this time, a
reference to the response object and a reference to the protocol
object. The callback can use the predefined constants C_ENDCON,
C_LASTCON and C_ENDALL as a return value in order to influence pending
and active connections. C_ENDCON will end this connection immediately,
whereas C_LASTCON will inidicate that no further connections should be
made. C_ENDALL will immediately end all requests and let the
Parallel::UserAgent return from \f(CW$pua\fR\->\fBwait()\fR.
.Sp
If \f(CW$arg\fR is omitted, then the content is stored in the response
object itself.
.Sp
If \f(CW$arg\fR is a \f(CW\*(C`LWP::Parallel::UserAgent::Entry\*(C'\fR object, then this
request will be registered as a follow-up request to this particular
entry. This will not create a new entry, but instead link the current
response (i.e. the reason for re-registering) as \f(CW$response\fR\->previous
to the new response of this request.  All other fields are either
re-initialized ($request, \f(CW$fullpath\fR, \f(CW$proxy\fR) or left untouched ($arg,
\&\f(CW$size\fR). (This should only be use internally)
.Sp
LWP::Parallel::UserAgent\->request also allows the registration of
follow-up requests to existing requests, that required redirection or
authentication. In order to do this, an Parallel::UserAgent::Entry
object will be passed as the second argument to the call. Usually,
this should not be used directly, but left to the internal
\&\f(CW$ua\fR\->handle_response method!
.ie n .IP "$ua\->on_connect ( $request, $response, $entry )" 4
.el .IP "\f(CW$ua\fR\->on_connect ( \f(CW$request\fR, \f(CW$response\fR, \f(CW$entry\fR )" 4
.IX Item "$ua->on_connect ( $request, $response, $entry )"
This method should be overridden in an (otherwise empty) subclass in
order to present customized messages for each connection attempted by
the User Agent.
.ie n .IP "$ua\->on_failure ( $request, $response, $entry )" 4
.el .IP "\f(CW$ua\fR\->on_failure ( \f(CW$request\fR, \f(CW$response\fR, \f(CW$entry\fR )" 4
.IX Item "$ua->on_failure ( $request, $response, $entry )"
This method should be overridden in an (otherwise empty) subclass in
order to present customized messages for each connection or
registration that failed.
.ie n .IP "$ua\->on_return ( $request, $response, $entry )" 4
.el .IP "\f(CW$ua\fR\->on_return ( \f(CW$request\fR, \f(CW$response\fR, \f(CW$entry\fR )" 4
.IX Item "$ua->on_return ( $request, $response, $entry )"
This method should be overridden in an (otherwise empty) subclass in
order to present customized messages for each request returned. If a
callback function was registered with this request, this callback
function is called before \f(CW$pua\fR\->on_return.
.Sp
Please note that while \f(CW$pua\fR\->on_return is a method (which should be
overridden in a subclass), a callback function is \s-1NOT\s0 a method, and
does not have \f(CW$self\fR as its first parameter. (See more on callbacks
below)
.Sp
The purpose of \f(CW$pua\fR\->on_return is mainly to provide messages when a
request returns. However, you can also re-register follow-up requests
in case you need them.
.Sp
If you need specialized follow-up requests depending on the request
that just returend, use a callback function instead (which can be
different for each request registered). Otherwise you might end up
writing a \s-1HUGE\s0 if..elsif..else.. branch in this global method.
.ie n .IP "$us\->discard_entry ( $entry )" 4
.el .IP "\f(CW$us\fR\->discard_entry ( \f(CW$entry\fR )" 4
.IX Item "$us->discard_entry ( $entry )"
Completely removes an entry from memory, in case its output is not
needed. Use this in callbacks such as \f(CW\*(C`on_return\*(C'\fR or <on_failure> if
you want to make sure an entry that you do not need does not occupy
valuable main memory.
.ie n .IP "$ua\->wait ( $timeout )" 4
.el .IP "\f(CW$ua\fR\->wait ( \f(CW$timeout\fR )" 4
.IX Item "$ua->wait ( $timeout )"
Waits for available sockets to write to or read from.  Will timeout
after \f(CW$timeout\fR seconds. Will block if \f(CW$timeout\fR = 0 specified. If
\&\f(CW$timeout\fR is omitted, it will use the Agent default timeout value.
.ie n .IP "$ua\->handle_response($request, $arg [, $size])" 4
.el .IP "\f(CW$ua\fR\->handle_response($request, \f(CW$arg\fR [, \f(CW$size\fR])" 4
.IX Item "$ua->handle_response($request, $arg [, $size])"
Analyses results, handling redirects and security.  This method may
actually register several different, additional requests.
.Sp
This method should not be called directly. Instead, indicate for each
individual request registered with \f(CW\*(C`$ua\-\*(C'\fR\fBregister()\fR> whether or not
you want Parallel::UserAgent to handle redirects and security, or
specify a default value for all requests in Parallel::UserAgent by
using \f(CW\*(C`$ua\-\*(C'\fR\fBredirect()\fR>.
.ie n .IP "\s-1DEPRECATED\s0 $ua\->deprecated_simple_request($request, [$arg [, $size]])" 4
.el .IP "\s-1DEPRECATED\s0 \f(CW$ua\fR\->deprecated_simple_request($request, [$arg [, \f(CW$size\fR]])" 4
.IX Item "DEPRECATED $ua->deprecated_simple_request($request, [$arg [, $size]])"
This method simulated the behavior of LWP::UserAgent\->simple_request.
It was actually kinda overkill to use this method in
Parallel::UserAgent, and it was mainly here for testing backward
compatibility with the original LWP::UserAgent.
.Sp
The name has been changed to deprecated_simple_request in case you 
need it, but because it it no longer compatible with the most recent
version of libwww, it will no longer run by default.
.Sp
The following 
description is taken directly from the corresponding libwww pod:
.Sp
\&\f(CW$ua\fR\->simple_request dispatches a single \s-1WWW\s0 request on behalf of a
user, and returns the response received.  The \f(CW$request\fR should be a
reference to a \f(CW\*(C`HTTP::Request\*(C'\fR object with values defined for at
least the \fBmethod()\fR and \fBurl()\fR attributes.
.Sp
If \f(CW$arg\fR is a scalar it is taken as a filename where the content of
the response is stored.
.Sp
If \f(CW$arg\fR is a reference to a subroutine, then this routine is called
as chunks of the content is received.  An optional \f(CW$size\fR argument
is taken as a hint for an appropriate chunk size.
.Sp
If \f(CW$arg\fR is omitted, then the content is stored in the response
object itself.
.ie n .IP "\s-1DEPRECATED\s0 $ua\->deprecated_request($request, $arg [, $size])" 4
.el .IP "\s-1DEPRECATED\s0 \f(CW$ua\fR\->deprecated_request($request, \f(CW$arg\fR [, \f(CW$size\fR])" 4
.IX Item "DEPRECATED $ua->deprecated_request($request, $arg [, $size])"
Previously called 'request' and included for compatibility testing with 
LWP::UserAgent. Every day usage was deprecated, and now you have to call it
with the deprecated_request name if you want to use it (because an incompatibility
was introduced with the newer versions of libwww).
.Sp
Here is what LWP::UserAgent has to say about it:
.Sp
Process a request, including redirects and security.  This method may
actually send several different simple reqeusts.
.Sp
The arguments are the same as for \f(CW\*(C`simple_request()\*(C'\fR.
.ie n .IP "$ua\->as_string" 4
.el .IP "\f(CW$ua\fR\->as_string" 4
.IX Item "$ua->as_string"
Returns a text that describe the state of the \s-1UA.\s0  Should be useful
for debugging, if it would print out anything important. But it does
not (at least not yet). Try using LWP::Debug...
.SH "ADDITIONAL METHODS"
.IX Header "ADDITIONAL METHODS"
.ie n .IP "$ua\->use_alarm([$boolean])" 4
.el .IP "\f(CW$ua\fR\->use_alarm([$boolean])" 4
.IX Item "$ua->use_alarm([$boolean])"
This function is not in use anymore and will display a warning when 
called and warnings are enabled.
.SH "Callback functions"
.IX Header "Callback functions"
You can register a callback function. See LWP::UserAgent for details.
.SH "BUGS"
.IX Header "BUGS"
Probably lots! This was meant only as an interim release until this
functionality is incorporated into LWPng, the next generation libwww
module (though it has been this way for over 2 years now!)
.PP
Needs a lot more documentation on how callbacks work!
.SH "SEE ALSO"
.IX Header "SEE ALSO"
LWP::UserAgent
.SH "COPYRIGHT"
.IX Header "COPYRIGHT"
Copyright 1997\-2004 Marc Langheinrich <marclang@cpan.org>
.PP
This library is free software; you can redistribute it and/or
modify it under the same terms as Perl itself.
.SH "POD ERRORS"
.IX Header "POD ERRORS"
Hey! \fBThe above document had some coding errors, which are explained below:\fR
.IP "Around line 1533:" 4
.IX Item "Around line 1533:"
You forgot a '=back' before '=head1'
.IP "Around line 1535:" 4
.IX Item "Around line 1535:"
\&'=item' outside of any '=over'
.IP "Around line 1546:" 4
.IX Item "Around line 1546:"
You forgot a '=back' before '=head1'