Thursday, June 10, 2010

How to generate core dump for an application in Linux

In order to troubleshoot application issue, a process content in memory can be dumped to a file and then the file can be analysed with debug tool, such as gdb in Linux. 
Another way of doing this is to send QUIT signal to the PID (kill –3  PID), but the thread dump will be directed to stdout, which can be viewed with “cat /proc/PID/fd/1 |tee /tmp/dump.log”, messages will be constantly directed to  /proc/PID/fd/1 until the process is stopped. So it is useful for real time debugging.
The following java application example use gcore command in gdb. (gcore or kill –3 will not stop the process )
Linux default core file size is 0, which means core dump is disabled, It needs to be changed to unlimited
#ulimit -a | grep core
core file size          (blocks, -c) 0
#ulimit –c unlimited
#ulimit -a | grep core
core file size          (blocks, -c) unlimited

Firstly, find the virtual memory size of the process, the PID is 10008 in following example.
# ps aux | egrep 'VSZ| 10008'
root       10008  0.4  8.3 2231312 660100 ?      Sl   Jun03  43:57 /opt/sunjdk6/bin/java

The process VSZ=2.2GB, it will be the size of the core file. Go to a dump dir which has more than 2.2GB free space
$cd /var/tmp

Attach to running process PID.
$gdb --pid=10008

At the gdb prompt, enter gcore command

wait for few minutes for core file to be generated. type in quit to exit gdb, answer yes to detach the process.

core file is generated
$ ls -lh /var/tmp/core.10008
-rw-r--r-- 1 root root 2.2G Jun 10 11:59 /var/tmp/core.10008
file command reveals the source program name
$ file /var/tmp/core.10008
/var/tmp/core.10008: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), SVR4-style, from 'java'

Thursday, June 3, 2010

Use script to fetch URL protected by NTLM authentication.

Windows IIS server use NTLM authentication, the following show three methods -wget, curl, and Perl- to download URL protected by NTLM.


wget is able to negotiate auth method automatically. So the following command works for Basic or NTLM auth method.
wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS
Instead of exposing credentials in command line, putting credentials in a file $HOME/.wgetrc.


The following simplified command will read above file.



curl can’t negotiate auth method automatically, it has to told to use ntlm

curl -u 'DOMAINNAME\USERNAME':pass  --ntlm

curl also supports reading credentials from file $HOME/.netrc

machine  login DOMAINNAME\USERNAME  password PASS
The following simplified command will read above file.
curl  -n    --ntlm


HTML::TreeBuilder and HTML::FormatText are used strip out htlm tags to print pretty output. Perl also need Authen::NTLM and LWP::Authen::Ntlm modules installed.

#!/usr/bin/perl -w
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
use HTML::TreeBuilder;
use HTML::FormatText;

my $url = ''; #port is mandatory
my $username="DOMAINNAME\\USERNAME";
my $password='PASS' ;
my ($host) = $url =~ mhttp://([^/]*)/i;
my $ua = LWP::UserAgent->new( keep_alive => 1 );

$ua->credentials( $host, '', $username, $password);
my $req = GET $url;

my $response = $ua->request($req);
if ( $response->is_error() ) {
printf " %s\n", $response->status_line;
else {
my $document = $response->content();

$html = HTML::TreeBuilder->new();
$formatter = HTML::FormatText->new( leftmargin => 0, rightmargin => 50 );
$ascii = $formatter->format($html);
print "$ascii";