script to replace a pattern interactively in a directory of files

by rk on 11/23/2014

script to replace pattern in a directory of files interactively from bash shell
This is a file written to replace a pattern with another pattern from bash shell. This script will repeatedly invoke vim editor and wait for your interaction to replace the pattern. use it and buy me a beer if it ever saved your job.

#!/usr/bin/env bash
#
# Author: RavikumarNaidu.T
# ————————

from=$1
to=$2
find . -name “*.[c,h]” | xargs grep -l “$from” > files
echo “replacing $from to $to “
        for i in `cat files`
          do
            echo $i
            vim $i -c “%s/$from/$to/c”
            echo -n “continue with next file ? y/n[y]:”
            read choice
            if [[ $choice == “n” ]]
            then
               exit
            fi
        done

No Comments

using boost asio with message queues on Linux

by rk on 11/23/2014

In this post i will talk about integrating POSIX message queues with boost asio. You might ask why would any body need that ? We will need this if we are integrating boost asio with existing software or are migrating legacy code to boost asio.

I will show you how can you integrate boost asio with your existing message queue code with minimal disruption. The code just involves opening the message queue, attaching it to the io service and then installing the callback handler to be called when there is data on the message queue.

Boost provides 2 primitives for users who want to "read" and "process" the data on the descriptor themselves. Null buffers tell boost asio that the user wants to read data from the channel himself and boost stream descriptors provide a way to wrap a bare unix descriptor in to a asio object.

The code will not be a full fledged example which you can download and compile , its just a snippet which guides you how to do things.

In my next post i will explain how to integrate linux inotify descriptor with the boost asio.

This piece of code is taken from a live project and might include some undefined variables and might not compile as is. do let me know if you need help on compiling this. 

static boost::asio::posix::stream_descriptor *gwMqFd = nullptr;
static boost::asio::io_service gIoSvc;


static void
handleMqRead(boost::system::error_code ec)
{
     char data[256];
    _info<<"data available on message queue.";
     //read the data and process it 
    int rc = mq_receive(gwMqFd->native_handle(),
                       data,
                       sizeof(data),
                       nullptr);
     //reissue the async call again.  
    gwMqFd->async_read_some(boost::asio::null_buffers(),
                            boost::bind(&handleMqRead,
                            boost::asio::placeholders::error));
    return;
}



std::string mqName = "/mymessagequeue";
struct mq_attr mattr = {0, 10, 256, 0};
//open the message queue and add it to the boost ioservice.
int mqfd = mq_open(mqName.c_str(), O_RDWR | O_CREAT | O_NONBLOCK, 0660, NULL);
gwMqFd = new boost::asio::posix::stream_descriptor(gIoSvc);
gwMqFd->assign(mqfd);
//hook up the read call to the message queue.
gwMqFd->async_read_some(boost::asio::null_buffers(),
                        boost::bind(&handleMqRead,
                        boost::asio::placeholders::error));


 
No Comments

bash script to calculate the code coverage of a directory after running gcov

by rk on 11/23/2014

bash script to calculate the code coverage of a directory after running gcov
A bash script to calculate the code coverage of a directory after running gcov in the directory. leave me a comment if you have any suggestions or you find a bug in the script. ofcourse buy me a beer if it saved your job.

#Shell script to calculate code coverage report for
#a given directory.
#Author .Ravikumarnaidu.T
#————————————————–

#usage :
#——-
# compile the component source code for generating the coverage
# with the command “make G=1″.
# run your test cases
# if there are .da files generated in your component directory
# execute this script with the command “. gcov.sh ”

{
( find . -name “*.c” | cut -f1 -d” ” ) > tmp ;
for i in `cat tmp`
do
gcov $i >> gcov_summary.txt
cat gcov_summary.txt | grep ‘%’ | cut -f2,4 -d” ” | sed -e s/\%\/” “/g;
done
} > values;

cat values | awk ‘BEGIN{} sum+=$1*$2{} total+=$2{} END{print “coverage=” sum/total}’

#clean the shit left behind
#————————–
rm tmp;
rm values;

No Comments

how to rsync files along with their extended attributes ?

by rk on 11/23/2014
rsync files along with their extended attributes

 rsync -razvXe “ssh” –delete –progress –stats local_directory root@192.168.1.100:remote_directory

No Comments

what are extended attributes and how are they useful to linux programmers ?

by rk on 11/23/2014

The fields are all self explanatory and you can set and get them with their respective system calls ( man 2 stat ). For an ordinary user this information is more than enough, a typical user will be interested only in the last access fields and the size of the file. 
But for system builders, there often arises a need to store some extra “user defined” information along with the file in the file system. Suppose if the user is building a document management system and he wants to provide a field “state” for the document. The state field can be “open” , “locked” and “review” etc. As we see the above stat structure there is no field to store this piece of information. 
Now he has only one choice to store this information, he has to store this field along with the file name in a database, the database can be indexed with the file name and the record can be retrieved. But there is a problem with this solution, every time the file system is altered the database needs to be kept in sync with the file system. 
Enter extended attributes, extended attributes provide a facility to store a “blob” of information along with the file in the file system itself. i will now show the power of extended attributes with an example. 
In the below example i will store a user defined attribute of the file.
enabling the extended attributes on the system: 
By default some systems do not enable the extended attributes. You can do so by following command.
sudo apt-get install attrs. 
Apis and code: 
We will accept 2 arguments the first one being file name and the second argument is the state of the file name. 
save the code in a file and name it as xattr.c 

//file name attr.c
//—————————————-
#include <string.h>
#include <stdio.h>
#include <sys/types.h>
#include <attr/xattr.h>
int
main(int ac, char **av)
{
    if(ac < 3)
    {
        fprintf(stderr,\nNeed 3 arguments to run: usage ./attr <file_name> <state=’open’,’closed’,’locked’>);
        return -1;
    }
    int rc  = setxattr(av[1], user.file.state, av[2], strlen(av[2]), 0);
    if(rc < 0)
    {
        perror(setxattr failed:); 
        return -1; 
    }
    char buf[1024] = {‘\0′};
    rc = getxattr(av[1], user.file.state, buf, sizeof(buf));
    if(rc < 0)
    {
        perror(getxattr failed:); 
        return -1; 
    }
    fprintf(stderr,state: %s, buf);
    return 0;
}
build it with 
make xattr 
run it with 
./xattr file_name “locked”
note: 
The name of the extended attributes on linux should always start with “user.” prefix, otherwise the file system will keep on throwing error. Always ensure your attribute names are named as “user.file.xxxx”.
No Comments

making sudo command work for ldap users on ubuntu

by rk on 11/23/2014
This is quite easy but a bit less documented,

we will start on the server side whats required, i have a fedora directory server running on centos, this server comes with all the required schema definitions for the creating the sudoers container and the necessary attributes.
on the ldap server create a new “organizational unit” and name it “sudoers”.  Name the attributes like below.

dn: ou=sudoers,dc=example,dc=com
ou: sudoers
description:: c3Vkb2VycyBjb250YWluZXIgZm9yIHN1ZG9lcnMg
objectClass: top
objectClass: organizationalunit
objectClass: sudorole
cn: %eng   <———————— This should be the name of the group all ldap users belong to
sudoUser: %eng  <——————- same as the “cn” attribute above.
sudoOption: env_keep+=SSH_AUTH_SOCK
sudoHost: ALL
sudoCommand: ALL

we are done with the server side.

on the client side,
we have to set a root password for root first do this with command “sudo passwd” a password prompt will appear , type in the root password and then confirm it.

There is a separate package for sudo with ldap support on ubuntu, that is “sudo-ldap”, install this package
with
#sudo apt-get install sudo-ldap

configure the sudo-ldap by editing file /etc/sudo-ldap.conf



base dc=example,dc=com
uri ldap://example.com:6513


ssl=no


ldap_version 3
sudoers_base   ou=sudoers,dc=example,dc=com
sudoers_search_filter (&(attribute=sudorole))

in the file “/etc/nsswitch.conf” add a line with 
sudoers: ldap files

while the ldap user logged in go to “users accounts” from the administration menu  menu.
change the “account type” of the ldap user logged in to “Administrator”.
note:
while doing this the system will prompt you to enter the admin password of the current admin user.

reboot the client (not sure whether a “service nscd restart” is enough). 
login as the ldap user, issue a command as “sudo ls” , it should be working , if not let me know we will  debug together ;-) 
No Comments

setting up home directories on nfs server for ldap users

by rk on 11/23/2014
In my previous blog i discussed how to create ldap users on ubuntu. Creating just a user in ldap is not enough we need to create home directory for him on nfs server, other wise the user will have home on the client desktop and all of his data and configs will be stored on that computer, if he uses another client desktop then he wont see the old files and configs on the new computer which he is using. This is not the right way to setup the ldap users, the home directory should be available from all the client desktops and there should be only one home directory created per user and that too on the nfs server.
so lets see how a home directory is created on the nfs server for each user

 

 

NOTE:
Currently nfs4 does not work as documented on ubuntu 11.10, i ran in to very very deep shit trying and wasted many days. The idmapd daemon will just not work and there is very little documentation available for idmapd. my advise is just to stay off from the nfs4.

lets say all of our ldap users are in a group called "eng" with gid : 9999

on the server do the following
become root with "su" command.
#su
Create a directory called "/homes" on the nfs server with the command
#mkdir /homes

lets say we have a user "dick" with uid 1010
create a home directory for him in the "/homes/" folder and then using "chown" command make "dick" owner of the directory and its contents.

#mkdir /homes/dick
#chown -R 1010:9999 /homes/dick

mark the folder "/homes" as an exported one by putting it in the /etc/exports file like below

[root@helen /]# cat /etc/exports
/homes 192.168.0.2/16(fsid=0,rw,insecure,no_subtree_check,sync)

fsid=0 is required because the folder being exported is in "/" directory.

run the "exportfs" command with "-r" option
#exportfs -r

now on the client side (ubuntu 11.10)

set the nfs folder on the nfs-server to be automounted on to client during bootup, this can be done by putting details in the  /etc/fstab file.  put a line like below in /etc/fstab file.
192.168.0.2:/homes    /homes nfs    nfsvers=3,sync 0 0

before putting the line in the /etc/fstab file just try to mount the nfs directory by hand with command
mount  192.168.0.2:/homes -t nfs -o nfsvers=3  /mnt/nfs
/mnt/nfs folder should be created on the client machine if not present already.
try to see if there are any errors.

now issue a reboot of the server and  client, after the user is logged in check the present dir of the user
with "pwd" command. try to create some files in the folder and write to them.
 

No Comments

setting up ldap client authentication on ubuntu 11.10 and 12.04

by rk on 11/22/2014
Recently I have to setup a linux corporate network and had a hard time to get things work. The setup consists of postfix, dovecot, svn and all were using LDAP for central authentication and authorization. I will try to describe the steps required in a series of blogs as a first step i will try to describe how to setup ldap client authentication on ubuntu 11.10 (12.04) machines.
 
By the end of this document you should be able to authenticate ldap users on the ubuntu client. Ldap users donot have local unix accounts on the client pc The client contacts the ldap server during the login to authenticate and authorize the access.
 
These steps are tested with a ubuntu client pc trying to authenticate against a fedora directory server running on Centos6. Both the client and the server are in intranet.
 
requirements :
ubuntu PC which acts as a client.
Centos6 running fedora directory server.
 
NOTE: 
Installing fedora directory server on a centos6 server is not less than any nightmare if you are able to get this you are super lucky.
You should have root password or admin priviliges on both the client and server 
machines.
 
Steps:
i) Issue the below command [ with out quotes ]
   "sudo apt-get install ldap-utils libpam-ldap libnss-ldap nslcd"
   NOTE: 
   During the installation of the above packages a dialog will pop up and ask some ldap configuration, you should give right  parameters      there    for things to work. 
 
ii) open /etc/nsswitch.conf [ remember you need to be sudo to edit this file ]
    append "ldap" word to the following lines
 
    #Original file looks like this
    passwd: compat
    group : compat
    shadow: compat
 
    #After appending "ldap" lines look like these
    passwd: compat ldap
    group : compat ldap
    shadow: compat ldap
 

iii) Comment out the line "rootbinddn" [ not sure why we need to do that ]if you don't want to create home directories on the work station but want to create them on the nfsserver, you can directly go to step vi).

iv) open the file /etc/pam.d/login and paste the below line
session required pam_mkhomedir.so skel=/etc/skel umask=0022 
v) open the file /etc/pam.d/lightdm paste the below line
session required pam_mkhomedir.so skel=/etc/skel umask=0022
vi) issue command [with out quotes] "sudo update-rc.d nslcd enable"
 
reboot the ubuntu client and your ldap user should be able to login.
 
Problems likely to encounter and solutions : 
i) Ldap user takes extremely long time, in the order of mins before he can see the desktop.
This is a very serious problem and you are likely to get in to this, actually the problem is not on the client but on the server side. This happens because the nss-ldap module is looking very hard to get the group information but its not able to find the group of the user whether in ldap or on local system.
check whether you have created a POSIX group on the ldap and associated the users with this group.
 
ii) checking the errors in /var/log/auth.log can be helpful. All the ldap errors will be logged there.
 
Debugging "unable to contact ldap server" 
Check whether the ldap is reachable and the port is opened.
Try to ping the ldap server whether its reachable and the name of the ldap machine is resolved properly.
Try to check whether the ldap port is opened or not (ports can be 6513 or 389).
you can find out by the telnet command just issue telnet 6513 or 389
if you see any characters on the console then the port is opened. if not then the port is closed and you need to open them in firewall.Remove the 'i' from the 'ldapi' where you specify the address of the machine running ldap server, the 'i' is required only when the ldap server is running on the local machine you are authenticating , its pretty obvious that the ldap server will always run on a different machine, so use "ldap" instead of "ldapi" in the address of ldap server.
 
Debugging "no such object" 
It means you are trying to access an object which does not exist, try to verify the "basedn" and see whether there is any object with that basedn.

The login prompt accepts the user password and name but the user is not able to login, the system does not present the ldap user with the desktop. This happens because there is no home directory for the user. you have to create them on the nfs server and mount them on to the work station, creating nfs home directories for ldap users is discussed in my other post, this problem will go away soon after the nfs home directories are created for ldap users.And finally drop me a comment if you are stuck with some thing, i might be able to help you out.

 
No Comments