Thursday, June 10, 2010

How to generate core dump for an application in Linux

In order to troubleshoot application issue, a process content in memory can be dumped to a file and then the file can be analysed with debug tool, such as gdb in Linux. 
Another way of doing this is to send QUIT signal to the PID (kill –3  PID), but the thread dump will be directed to stdout, which can be viewed with “cat /proc/PID/fd/1 |tee /tmp/dump.log”, messages will be constantly directed to  /proc/PID/fd/1 until the process is stopped. So it is useful for real time debugging.
The following java application example use gcore command in gdb. (gcore or kill –3 will not stop the process )
Linux default core file size is 0, which means core dump is disabled, It needs to be changed to unlimited
#ulimit -a | grep core
core file size          (blocks, -c) 0
#ulimit –c unlimited
#ulimit -a | grep core
core file size          (blocks, -c) unlimited

Firstly, find the virtual memory size of the process, the PID is 10008 in following example.
# ps aux | egrep 'VSZ| 10008'
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root       10008  0.4  8.3 2231312 660100 ?      Sl   Jun03  43:57 /opt/sunjdk6/bin/java

The process VSZ=2.2GB, it will be the size of the core file. Go to a dump dir which has more than 2.2GB free space
$cd /var/tmp

Attach to running process PID.
$gdb --pid=10008

At the gdb prompt, enter gcore command
gdb>gcore

wait for few minutes for core file to be generated. type in quit to exit gdb, answer yes to detach the process.
gdb>quit

core file is generated
$ ls -lh /var/tmp/core.10008
-rw-r--r-- 1 root root 2.2G Jun 10 11:59 /var/tmp/core.10008
file command reveals the source program name
$ file /var/tmp/core.10008
/var/tmp/core.10008: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), SVR4-style, from 'java'

Thursday, June 3, 2010

Use script to fetch URL protected by NTLM authentication.

Windows IIS server use NTLM authentication, the following show three methods -wget, curl, and Perl- to download URL protected by NTLM.

wget

wget is able to negotiate auth method automatically. So the following command works for Basic or NTLM auth method.
wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS http://www.example.com/info.asp
Instead of exposing credentials in command line, putting credentials in a file $HOME/.wgetrc.

http-user=DOMAINNAME\USERNAME
http-password=PASS

The following simplified command will read above file.

wget  http://www.example.com/info.asp

curl

curl can’t negotiate auth method automatically, it has to told to use ntlm

curl -u 'DOMAINNAME\USERNAME':pass  --ntlm  http://www.example.com/info.asp

curl also supports reading credentials from file $HOME/.netrc

machine www.example.com  login DOMAINNAME\USERNAME  password PASS
The following simplified command will read above file.
curl  -n    --ntlm   http://www.example.com/info.asp

Perl



HTML::TreeBuilder and HTML::FormatText are used strip out htlm tags to print pretty output. Perl also need Authen::NTLM and LWP::Authen::Ntlm modules installed.

#!/usr/bin/perl -w
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
use HTML::TreeBuilder;
use HTML::FormatText;

my $url = 'http://www.example.com:80/info.asp'; #port is mandatory
my $username="DOMAINNAME\\USERNAME";
my $password='PASS' ;
my ($host) = $url =~ mhttp://([^/]*)/i;
my $ua = LWP::UserAgent->new( keep_alive => 1 );

#$ua->agent("");
$ua->credentials( $host, '', $username, $password);
my $req = GET $url;

#$req->referer("");
my $response = $ua->request($req);
if ( $response->is_error() ) {
printf " %s\n", $response->status_line;
}
else {
my $document = $response->content();

$html = HTML::TreeBuilder->new();
$html->parse($document);
$formatter = HTML::FormatText->new( leftmargin => 0, rightmargin => 50 );
$ascii = $formatter->format($html);
print "$ascii";

}