Thursday, June 3, 2010

Use script to fetch URL protected by NTLM authentication.

Windows IIS server use NTLM authentication, the following show three methods -wget, curl, and Perl- to download URL protected by NTLM.

wget

wget is able to negotiate auth method automatically. So the following command works for Basic or NTLM auth method.
wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS http://www.example.com/info.asp
Instead of exposing credentials in command line, putting credentials in a file $HOME/.wgetrc.

http-user=DOMAINNAME\USERNAME
http-password=PASS

The following simplified command will read above file.

wget  http://www.example.com/info.asp

curl

curl can’t negotiate auth method automatically, it has to told to use ntlm

curl -u 'DOMAINNAME\USERNAME':pass  --ntlm  http://www.example.com/info.asp

curl also supports reading credentials from file $HOME/.netrc

machine www.example.com  login DOMAINNAME\USERNAME  password PASS
The following simplified command will read above file.
curl  -n    --ntlm   http://www.example.com/info.asp

Perl



HTML::TreeBuilder and HTML::FormatText are used strip out htlm tags to print pretty output. Perl also need Authen::NTLM and LWP::Authen::Ntlm modules installed.

#!/usr/bin/perl -w
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
use HTML::TreeBuilder;
use HTML::FormatText;

my $url = 'http://www.example.com:80/info.asp'; #port is mandatory
my $username="DOMAINNAME\\USERNAME";
my $password='PASS' ;
my ($host) = $url =~ mhttp://([^/]*)/i;
my $ua = LWP::UserAgent->new( keep_alive => 1 );

#$ua->agent("");
$ua->credentials( $host, '', $username, $password);
my $req = GET $url;

#$req->referer("");
my $response = $ua->request($req);
if ( $response->is_error() ) {
printf " %s\n", $response->status_line;
}
else {
my $document = $response->content();

$html = HTML::TreeBuilder->new();
$html->parse($document);
$formatter = HTML::FormatText->new( leftmargin => 0, rightmargin => 50 );
$ascii = $formatter->format($html);
print "$ascii";

}

6 comments:

  1. Hi, this is by saving credentials some place, is there a way to pass the domain\username:password directly ?

    I am trying to get past a proxy authentication, which combines domain name along with the username and password. Not too comfortable saving the credentials to a file. I read someplace that using only ':' in cURL passes the NT login credentials but this only passes the username & the password and not the domain name.
    Below is the code, any suggestions ?

    curl --proxy-ntlm --proxy-user : --proxy %PROXY_URL% -f -O --url "%SITE_URL%"

    ReplyDelete
  2. http://www.youtube.com/watch?v=w3fOLVpw2qs

    ReplyDelete
  3. good article i like your post keep this and i will do what you write exactly thank you l


    Proy Sites

    ReplyDelete
  4. Hello sir,
    Thank you very much for sharing the explosion with us. It is extremely useful and informative.
    Keep blogging for new updates.

    DedicatedHosting4u.com

    ReplyDelete
  5. I'm a long-serving digital marketing professional and full-service as a social media marketing manager. I'm offering services at a competitively low cost. I have experience in keyword research, Article writing or Rewriting, Guest posting, B2B Lead Generation , Data Entry ,link building, web 2.0 backlink ,
    . I have 5 years of experience in the field and are assured of delivering High Quality and manual work. I have my own site name as AbidhTech. My Blog site also here. This is a Bangla deshi Science club site .

    ReplyDelete