Thursday, June 3, 2010

Use script to fetch URL protected by NTLM authentication.

Windows IIS server use NTLM authentication, the following show three methods -wget, curl, and Perl- to download URL protected by NTLM.


wget is able to negotiate auth method automatically. So the following command works for Basic or NTLM auth method.
wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS
Instead of exposing credentials in command line, putting credentials in a file $HOME/.wgetrc.


The following simplified command will read above file.



curl can’t negotiate auth method automatically, it has to told to use ntlm

curl -u 'DOMAINNAME\USERNAME':pass  --ntlm

curl also supports reading credentials from file $HOME/.netrc

machine  login DOMAINNAME\USERNAME  password PASS
The following simplified command will read above file.
curl  -n    --ntlm


HTML::TreeBuilder and HTML::FormatText are used strip out htlm tags to print pretty output. Perl also need Authen::NTLM and LWP::Authen::Ntlm modules installed.

#!/usr/bin/perl -w
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
use HTML::TreeBuilder;
use HTML::FormatText;

my $url = ''; #port is mandatory
my $username="DOMAINNAME\\USERNAME";
my $password='PASS' ;
my ($host) = $url =~ mhttp://([^/]*)/i;
my $ua = LWP::UserAgent->new( keep_alive => 1 );

$ua->credentials( $host, '', $username, $password);
my $req = GET $url;

my $response = $ua->request($req);
if ( $response->is_error() ) {
printf " %s\n", $response->status_line;
else {
my $document = $response->content();

$html = HTML::TreeBuilder->new();
$formatter = HTML::FormatText->new( leftmargin => 0, rightmargin => 50 );
$ascii = $formatter->format($html);
print "$ascii";



  1. Hi, this is by saving credentials some place, is there a way to pass the domain\username:password directly ?

    I am trying to get past a proxy authentication, which combines domain name along with the username and password. Not too comfortable saving the credentials to a file. I read someplace that using only ':' in cURL passes the NT login credentials but this only passes the username & the password and not the domain name.
    Below is the code, any suggestions ?

    curl --proxy-ntlm --proxy-user : --proxy %PROXY_URL% -f -O --url "%SITE_URL%"


  3. good article i like your post keep this and i will do what you write exactly thank you l

    Proy Sites