Thursday, June 3, 2010

Use script to fetch URL protected by NTLM authentication.

Windows IIS server use NTLM authentication, the following show three methods -wget, curl, and Perl- to download URL protected by NTLM.


wget is able to negotiate auth method automatically. So the following command works for Basic or NTLM auth method.
wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS
Instead of exposing credentials in command line, putting credentials in a file $HOME/.wgetrc.


The following simplified command will read above file.



curl can’t negotiate auth method automatically, it has to told to use ntlm

curl -u 'DOMAINNAME\USERNAME':pass  --ntlm

curl also supports reading credentials from file $HOME/.netrc

machine  login DOMAINNAME\USERNAME  password PASS
The following simplified command will read above file.
curl  -n    --ntlm


HTML::TreeBuilder and HTML::FormatText are used strip out htlm tags to print pretty output. Perl also need Authen::NTLM and LWP::Authen::Ntlm modules installed.

#!/usr/bin/perl -w
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
use HTML::TreeBuilder;
use HTML::FormatText;

my $url = ''; #port is mandatory
my $username="DOMAINNAME\\USERNAME";
my $password='PASS' ;
my ($host) = $url =~ mhttp://([^/]*)/i;
my $ua = LWP::UserAgent->new( keep_alive => 1 );

$ua->credentials( $host, '', $username, $password);
my $req = GET $url;

my $response = $ua->request($req);
if ( $response->is_error() ) {
printf " %s\n", $response->status_line;
else {
my $document = $response->content();

$html = HTML::TreeBuilder->new();
$formatter = HTML::FormatText->new( leftmargin => 0, rightmargin => 50 );
$ascii = $formatter->format($html);
print "$ascii";



  1. Hi, this is by saving credentials some place, is there a way to pass the domain\username:password directly ?

    I am trying to get past a proxy authentication, which combines domain name along with the username and password. Not too comfortable saving the credentials to a file. I read someplace that using only ':' in cURL passes the NT login credentials but this only passes the username & the password and not the domain name.
    Below is the code, any suggestions ?

    curl --proxy-ntlm --proxy-user : --proxy %PROXY_URL% -f -O --url "%SITE_URL%"


  3. good article i like your post keep this and i will do what you write exactly thank you l

    Proy Sites

  4. Hello sir,
    Thank you very much for sharing the explosion with us. It is extremely useful and informative.
    Keep blogging for new updates.

  5. I'm a long-serving digital marketing professional and full-service as a social media marketing manager. I'm offering services at a competitively low cost. I have experience in keyword research, Article writing or Rewriting, Guest posting, B2B Lead Generation , Data Entry ,link building, web 2.0 backlink ,
    . I have 5 years of experience in the field and are assured of delivering High Quality and manual work. I have my own site name as AbidhTech. My Blog site also here. This is a Bangla deshi Science club site .

  6. The Original Forex Trading System: roboforex login Is The Original Forex Trading System. It Is 100% Automated And Provides An Easy-to-follow Trading System. You Get Access To Real-time Signals, Proven Methods, And A Money-back Guarantee.

  7. Get the latest Mmatf Stock quotes, news and trends at your fingertips across all of your devices, available 247. Our stock quote system helps financial professionals keep up on the latest data and locate charts to help in their own business.

  8. At Tradingzy, we’re digital marketing specialists. We focus on search engine optimization for the forex market. We’ve got the best ranking on Google for keywords like “ What Is Liquidity In Cryptocurrency .”