urllib2 Namespace Reference

Data Structures
class	URLError
class	HTTPError
class	GopherError
class	Request
class	OpenerDirector
class	BaseHandler
class	HTTPDefaultErrorHandler
class	HTTPRedirectHandler
class	ProxyHandler
class	CustomProxy
class	CustomProxyHandler
class	HTTPPasswordMgr
class	HTTPPasswordMgrWithDefaultRealm
class	AbstractBasicAuthHandler
class	HTTPBasicAuthHandler
class	ProxyBasicAuthHandler
class	AbstractDigestAuthHandler
class	HTTPDigestAuthHandler
class	ProxyDigestAuthHandler
class	AbstractHTTPHandler
class	HTTPHandler
class	HTTPSHandler
class	UnknownHandler
class	FileHandler
class	FTPHandler
class	CacheFTPHandler
	fw.ftp.set_debuglevel(1) More...
class	GopherHandler
class	OpenerFactory

Functions
def	urlopen
def	install_opener
def	build_opener
def	encode_digest
def	parse_keqv_list
def	parse_http_list

Variables
	sha = None
string	__version__ = "2.0a1"
	_opener = None
string	localhost = 'bitdiddle.cnri.reston.va.us'
list	urls
tuple	cfh = CacheFTPHandler()
	if localhost is not None: urls = urls + [ 'file://%s/etc/passwd' % localhost, 'http://%s/simple/' % localhost, 'http://%s/digest/' % localhost, 'http://%s/not/found.h' % localhost, ] More...
	req = None
	XXX try out some custom proxy objects too! def at_cnri(req): host = req.get_host() print host if host[-18:] == '.cnri.reston.va.us': return 1 p = CustomProxy('http', at_cnri, 'proxy.cnri.reston.va.us') ph = CustomProxyHandler(p) More...
tuple	f = urlopen(url, req)
tuple	buf = f.read()

Detailed Description

An extensible library for opening URLs using a variety of protocols

The simplest way to use this module is to call the urlopen function,
which accepts a string containing a URL or a Request object (described
below).  It opens the URL and returns the results as file-like
object; the returned object has some extra methods described below.

The OpenerDirectory manages a collection of Handler objects that do
all the actual work.  Each Handler implements a particular protocol or
option.  The OpenerDirector is a composite object that invokes the
Handlers needed to open the requested URL.  For example, the
HTTPHandler performs HTTP GET and POST requests and deals with
non-error returns.  The HTTPRedirectHandler automatically deals with
HTTP 301 & 302 redirect errors, and the HTTPDigestAuthHandler deals
with digest authentication.

urlopen(url, data=None) -- basic usage is that same as original
urllib.  pass the url and optionally data to post to an HTTP URL, and
get a file-like object back.  One difference is that you can also pass
a Request instance instead of URL.  Raises a URLError (subclass of
IOError); for HTTP errors, raises an HTTPError, which can also be
treated as a valid response.

build_opener -- function that creates a new OpenerDirector instance.
will install the default handlers.  accepts one or more Handlers as
arguments, either instances or Handler classes that it will
instantiate.  if one of the argument is a subclass of the default
handler, the argument will be installed instead of the default.

install_opener -- installs a new opener as the default opener.

objects of interest:
OpenerDirector --

Request -- an object that encapsulates the state of a request.  the
state can be a simple as the URL.  it can also include extra HTTP
headers, e.g. a User-Agent.

BaseHandler --

exceptions:
URLError-- a subclass of IOError, individual protocols have their own
specific subclass

HTTPError-- also a valid HTTP response, so you can treat an HTTP error
as an exceptional event or valid response

internals:
BaseHandler and parent
_call_chain conventions

Example usage:

import urllib2

# set up authentication info
authinfo = urllib2.HTTPBasicAuthHandler()
authinfo.add_password('realm', 'host', 'username', 'password')

proxy_support = urllib2.ProxyHandler({"http" : "http://ahad-haam:3128"})

# build a new opener that adds authentication and caching FTP handlers
opener = urllib2.build_opener(proxy_support, authinfo, urllib2.CacheFTPHandler)

# install it
urllib2.install_opener(opener)

f = urllib2.urlopen('http://www.python.org/')

Function Documentation

def urllib2.build_opener

(

handlers)

Create an opener object from a list of handlers.

The opener will use several default handlers, including support
for HTTP and FTP.  If there is a ProxyHandler, it must be at the
front of the list of handlers.  (Yuck.)

If any of the handlers passed as arguments are subclasses of the
default handlers, the default handlers will not be used.

Definition at line 355 of file urllib2.py.

References inspect.isclass().

def build_opener(*handlers):
    """Create an opener object from a list of handlers.

    The opener will use several default handlers, including support
    for HTTP and FTP.  If there is a ProxyHandler, it must be at the
    front of the list of handlers.  (Yuck.)

    If any of the handlers passed as arguments are subclasses of the
    default handlers, the default handlers will not be used.
    """

    opener = OpenerDirector()
    default_classes = [ProxyHandler, UnknownHandler, HTTPHandler,
                       HTTPDefaultErrorHandler, HTTPRedirectHandler,
                       FTPHandler, FileHandler]
    if hasattr(httplib, 'HTTPS'):
        default_classes.append(HTTPSHandler)
    skip = []
    for klass in default_classes:
        for check in handlers:
            if inspect.isclass(check):
                if issubclass(check, klass):
                    skip.append(klass)
            elif isinstance(check, klass):
                skip.append(klass)
    for klass in skip:
        default_classes.remove(klass)

    for klass in default_classes:
        opener.add_handler(klass())

    for h in handlers:
        if inspect.isclass(h):
            h = h()
        opener.add_handler(h)
    return opener