Monday, August 12, 2013

Bytebuffer class in C++

I was wondering if a class similar to Java ByteBuffer is available in C++ is as well. Strinstream works, but it doesn't have the control what Java ByteBuffer can offer. It does a very simple job. But if a class is already available, then it becomes convenient for the developers. There are many libraries that can generate the data encoding code for us. Good examples are Apache Thrift and Google protobuf.  But there are many softwares requiring messages to be encoded in some other formats hoping they won't be accessible only by the languages supported by Thrift, Protobuf etc.  

My implementation for ByteBuffer in C++ is accessible at this github link.
There is a test program (test.cpp) which demonstrates the operation.
 This can be compiled as shown below:
$g++ -I .  test.cpp bytebuffer.cpp bytebuffer_exception.cpp

Then run it as shown below:
$./a.out

The source for the test program is pasted below:

#include <string.h>
#include <stdlib.h>
#include <iostream>
#include <bytebuffer.hpp>
#include <stdio.h>

using namespace GeetPutula;
using namespace std;

int main()
{
    ByteBuffer buffer(256, ByteBuffer::LITTLE);
    buffer.putInt32(102);
    buffer.rewind();
    int32_t val = buffer.getInt32();
    cout << val << endl;
    val = buffer.getInt32();
    cout << val << endl;
    size_t position = buffer.currentPosition();
    cout << buffer.position(1024) << endl;
    cout << buffer.position(1024) << endl;
    if (buffer.putInt32(100) == false) {
        cout << "Worked as expected \n";
    }
    buffer.position(position);
    buffer.putBytes((void *)"this is my string", strlen("this is my string") + 1);
    void *buf = malloc(strlen("this is my string") + 1);
    buffer.position(position);
    buffer.readBytes(buf, strlen("this is my string") + 1);
    cout << (char *) buf << endl;
    free(buf);
    position = buffer.currentPosition();
    buffer.putInt16(45);
    buffer.position(position);
    cout << buffer.getInt16() << endl;
    cout << "Current position " << buffer.currentPosition() << endl;
    position = buffer.currentPosition();
    buffer.putInt64(-123456789123400);
    buffer.position(position);
    cout << buffer.getInt64() << endl;
    position = buffer.currentPosition() ;
    cout << "Current position " << position << endl;
    buffer.putFloat(-1223.3233);
    buffer.position(position);
    float f  = buffer.getFloat() ;
    printf("%f floattttt\n", f);
    
    position = buffer.currentPosition() ;
    cout << "Current position " << position << endl;
    buffer.putDouble(-1223879967.3233129);
    buffer.position(position);
    cout << buffer.getDouble() << endl;
    buffer.position(position);
    double x = buffer.getDouble();
    printf("%lf \n", x);
    return 0;
}

Monday, July 15, 2013

Daemontools compilation problem on Linux 64 bit system

Recently I was working on evaluating daemontools for some components which need to be run as Linux services and which are to be restarted if they goes down for some reason. I downloaded daemontools and followed the instruction to build it. But it failed with the below error:

./makelib unix.a alloc.o alloc_re.o buffer.o buffer_0.o buffer_1.o \
buffer_2.o buffer_get.o buffer_put.o buffer_read.o buffer_write.o \
coe.o env.o error.o error_str.o fd_copy.o fd_move.o fifo.o lock_ex.o \
lock_exnb.o ndelay_off.o ndelay_on.o open_append.o open_read.o \
open_trunc.o open_write.o openreadclose.o pathexec_env.o \
pathexec_run.o prot.o readclose.o seek_set.o sgetopt.o sig.o \
sig_block.o sig_catch.o sig_pause.o stralloc_cat.o stralloc_catb.o \
stralloc_cats.o stralloc_eady.o stralloc_opyb.o stralloc_opys.o \
stralloc_pend.o strerr_die.o strerr_sys.o subgetopt.o wait_nohang.o \
wait_pid.o
./load envdir unix.a byte.a
/usr/bin/ld: errno: TLS definition in /lib64/libc.so.6 section .tbss mismatches non-TLS reference in envdir.o
/lib64/libc.so.6: could not read symbols: Bad value
collect2: ld returned 1 exit status
make: *** [envdir] Error 1

The problem was solved by including the standard errno.h  header file in error.h (extracted to admin/daemontools-0.76/src/error.h from daemontools-0.76.tar.gz) in the source. And we can use the nice tool without any issue :):)

Friday, April 5, 2013

Executing commands on multiple Linux machines


Here is a small and convenient tool to execute same command over a set of Linux machines.
To use the tool we need to configure the machines for password-less SSH, also need Perl module Net::OpenSSH. We have to create a file $HOME/.allhosts and put the IP addresses or host names in this file in separate lines. 

Configuring password-less access to all the machines you have

generate punblic and private keys
$ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa

copy the public and private keys to all the machines you want to passwordless access:
scp ~/.ssh/id_rsa.pub userid@machine-ip:~/.ssh/id_rsa.pub
scp ~/.ssh/id_rsa userid@machine-ip:~/.ssh/id_rsa

Add the public key to the authorized keys files on every machines:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Now you will be able to ssh from any of these machines to any of these machines without a password

Install perl Net::OpenSSH module as shown below:
$sudo yum install perl-CPAN
$sudo perl -MCPAN -e "install Net::OpenSSH" 

Examples
to kill all the java processes running in the machines listed in yout $HOME/.allhosts, issue the below command:
$ perl multihostcmd.pl  "ps -ef  | grep java  | grep -v grep | awk ' { print \$2 }'  | xargs kill -9"
Of course you will need permission to kill those processs

To check all the users logged to these machines, issue the below command
$ perl multihostcmd.pl  who

Below is the script (you may also access it in github ):

#!/usr/bin/perl                                
############################################################################
# Save this in a a file, say multihostcmd.pl                                            
############################################################################   

use threads (
    'yield', 
    'stack_size' => 64 * 4096,
    'exit'       => 'threads_only',
    'stringify'                    
);                                 
use Cwd qw(abs_path);              
use Net::OpenSSH;                  

#
# executeCmdRemote
# executes the command on a remote host 
# command execution happens over ssh    
#                                       

sub executeCmdRemote {
    my ( $host, $command, @others ) = @_;
    my $ssh =  Net::OpenSSH->new($host,  
         master_opts => [-o => "ConnectionAttempts=2", -o => "ConnectTimeout=5"] );
    if ($ssh->error) {                                                             
        return;                                                                    
    }                                                                              
    ($out, $err, @others) =  $ssh->capture2({timeout => 20}, $command);            
    $outdata = "FROM $host ******************************************\n";          
    if ($out ne "") {                                                              
        $outdata .= $out;                                                          
    }                                                                              
    if ($err ne "") {                                                              
        $outdata .= $err;                                                          
    }                                                                              
    $outdata .= "\nEND OF DATA FROM $host ************************************* \n\n";
    print $outdata;                                                                   
}                                                                                     

#
# readHostFile
# reads from the file $HOME/.allhosts where this file contain a list of hosts
# each line on this file denotes a host name or host ip                      
# The given command is executed on this host                                 
#                                                                            
sub readHostFile {                                                           
    my ($array, @others) = @_;                                               
    $hostfile = $ENV{'HOME'} . '/.allhosts';                                 
    if (! -e  $hostfile) {                                                   
        print $hostfile . ' doesn\'t exist';                                 
        return;                                                              
    }                                                                        
    if (! -f  $hostfile) {                                                   
        print $hostfile . ' is not a regular file';                          
        return;                                                              
    }                                                                        
    open FH, "< $hostfile"  || return;                                       
    @$array = <FH>;                                                          
    close FH;
}

sub main {
    if ( $#ARGV < 0 ) {
        $executing_script = abs_path($0);
        print "Usage: $executing_script command-to-execute\n";
        exit(1);
    }
    $cmdline = join( ' ', @ARGV );
    my @hostarry = ();
    readHostFile(\@hostarry);
    if ($#hostarry < 0) {
        print "Empty list for hosts";
        exit(1);
    }

    my $thr = undef;
    my @allthreads = ();
    foreach my $host (@hostarry) {
       $host =~ s/^\s+//;
       $host =~ s/\s+$//;
       if  ($host eq "") {
           next;
       }
       # Create a thread to execute the command on the remote host
       $thr = threads->create('executeCmdRemote', $host, $cmdline);
       push @allthreads, $thr;
    }
    foreach $thr (@allthreads){
        $thr->join();
    }
}

# Call main
main();