Thursday, June 3, 2010

Use script to fetch URL protected by NTLM authentication.

Windows IIS server use NTLM authentication, the following show three methods -wget, curl, and Perl- to download URL protected by NTLM.

wget

wget is able to negotiate auth method automatically. So the following command works for Basic or NTLM auth method.
wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS http://www.example.com/info.asp
Instead of exposing credentials in command line, putting credentials in a file $HOME/.wgetrc.

http-user=DOMAINNAME\USERNAME
http-password=PASS

The following simplified command will read above file.

wget  http://www.example.com/info.asp

curl

curl can’t negotiate auth method automatically, it has to told to use ntlm

curl -u 'DOMAINNAME\USERNAME':pass  --ntlm  http://www.example.com/info.asp

curl also supports reading credentials from file $HOME/.netrc

machine www.example.com  login DOMAINNAME\USERNAME  password PASS
The following simplified command will read above file.
curl  -n    --ntlm   http://www.example.com/info.asp

Perl



HTML::TreeBuilder and HTML::FormatText are used strip out htlm tags to print pretty output. Perl also need Authen::NTLM and LWP::Authen::Ntlm modules installed.

#!/usr/bin/perl -w
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
use HTML::TreeBuilder;
use HTML::FormatText;

my $url = 'http://www.example.com:80/info.asp'; #port is mandatory
my $username="DOMAINNAME\\USERNAME";
my $password='PASS' ;
my ($host) = $url =~ mhttp://([^/]*)/i;
my $ua = LWP::UserAgent->new( keep_alive => 1 );

#$ua->agent("");
$ua->credentials( $host, '', $username, $password);
my $req = GET $url;

#$req->referer("");
my $response = $ua->request($req);
if ( $response->is_error() ) {
printf " %s\n", $response->status_line;
}
else {
my $document = $response->content();

$html = HTML::TreeBuilder->new();
$html->parse($document);
$formatter = HTML::FormatText->new( leftmargin => 0, rightmargin => 50 );
$ascii = $formatter->format($html);
print "$ascii";

}

4 comments:

  1. Hi, this is by saving credentials some place, is there a way to pass the domain\username:password directly ?

    I am trying to get past a proxy authentication, which combines domain name along with the username and password. Not too comfortable saving the credentials to a file. I read someplace that using only ':' in cURL passes the NT login credentials but this only passes the username & the password and not the domain name.
    Below is the code, any suggestions ?

    curl --proxy-ntlm --proxy-user : --proxy %PROXY_URL% -f -O --url "%SITE_URL%"

    ReplyDelete
  2. http://www.youtube.com/watch?v=w3fOLVpw2qs

    ReplyDelete
  3. good article i like your post keep this and i will do what you write exactly thank you l


    Proy Sites

    ReplyDelete