Linux : renaming files with rename in a loop

Following command (a small shell script) changes file extensions of all files from .TXT to .txt in the working directory.

for file in `find . -type f -name "*.TXT"`; do rename .TXT .txt $file; done

The above works in bash. For use in other shells, you'll have to change the looping sytax.

Perltidy

Perltidy is a great utility for formatting your perl codes. It splits long lines into short lines, does formatting with blocks thus increasing the readability of your codes. You can save your own settings related to formatting in a file called .perltidyrc. This file should saved in $HOME directory.

For more information and download of Perltidy, visit
http://perltidy.sourceforge.net/

Perltidy is coool !

Linux : Killing processes

Sometimes you have to kill several processes generated by a service/server (the main process) and killing one process is not enough. Recently, I had trouble with 'apachectl stop'. This would not stop all the processes of httpd for some strange reason. Killing all processes by entering the process id's would take quite some time. I figured a way out -

Here's the time-saver

ps -ef | grep httpd | awk '//{print $2}' | xargs kill

You can use the same logic for other processes. You need to replace httpd with ..... you know. As you can see, awk separates just the process ids and it becomes easier for us to kill the processes !

Perl : Is this module Installed ?

How do you check if a certain perl module is installed and is in @INC ?

On the command-line, issue this command

perl -e 'use X::Y'

X::Y is the module
With -e, you can executes the perl code in quotes on command line.

If the module is found, there won't be any errors (no output).

Perl : Generating password with a subroutine

A subroutine to generate a password :-) When you find it hard to create one.

sub generatePassword
{
    my $passwd_size = shift;
    my @alphanum = ('a'..'z', 'A'..'Z', 0..9,'!','#','$','%','&','*','@','`');
    my $password = join '', map $alphanum[rand @alphanum], 0..$passwd_size;

    return $password;
}

Usage

&generatePassword(10); ### returns password of 10 chars

Perl : Installing CPAN modules

This post if for those who find it hard or don't know how to install CPAN modules.

1 Become root

2. Run

perl -MCPAN -e shell

First time you run this command, you'll be asked some questions regarding where to download/extract files etc. Just select the defaults.

3. This opens the cpan prompt. Now, run

install Module::Name

That's all you have to do. Isn't this really simple ?

osix.net : A good website for programmers

I would like everybody to know I am a registered user of www.osix.net. There, I have cleared Geek Level 6 from the "Geek Challenge". These levels are programming puzzles on solving which you earn some reward points. You can exchange these reward points for hints from other geeks (experts), creating polls,  email address or pop access to email. There are some other redemption possibilities.

A great site for geeks !

My profile there
http://www.osix.net/modules/viewprofile/?name=ketan404

Perl : use Tie::File

With Tie::File module you can read all lines of a file in an array. Updating array elements updates the corresponding lines in the file. This is more efficient than reading the whole file, doing replacements and writing it back.

Following code snippet is an example

use Tie::File;

tie @line_array, 'Tie::File', 'file' or die "Can not tie file:$!";
for (@line_array) {
        s/pattern/replacement/g;
        ### or do anything that you wish to
        ### updating elements of array will update corresponding lines in the tied file
        ### pushing an element will add a line
}
untie @line_array;

Tie::File is very efficient for updating large files.

Linux : Virtual Memory Statistics

vmstat command shows Virtual Memory Statistics.

Sample outputs of the command are shown below.

$ vmstat

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd    free    buff    cache   si   so    bi   bo     in  cs        us  sy  id   wa
 0  0 507912  6016  5216  61768   0     0     1    5      3    3         1   0   98    0

With -s option (this prints the vm table)

$ vmstat -s

       515420  total memory
       510060  used memory
       340984  active memory
        18404  inactive memory
         5360  free memory
         5580  buffer memory
        62024  swap cache
      1024088  total swap
       507896  used swap
       516192  free swap
      5159612 non-nice user cpu ticks
       284811 nice user cpu ticks
      1343974 system cpu ticks
    441240715 idle cpu ticks
      1061329 IO-wait cpu ticks
       266848 IRQ cpu ticks
       130451 softirq cpu ticks
    135296968 pages paged in
     21489954 pages paged out
       402670 pages swapped in
       414068 pages swapped out
    402061903 interrupts
    401831289 CPU context switches
   1157798489 boot time
       185206 forks


To get just one number say that of free memory, you have to pipe this command to a awk (or perl or some other) that does filtering.

$ vmstat -s | awk '/free memory/{print $1}'
5360

awk is sometimes handier than grep :-)

PHP : variable names by form fields

Here's a quick way to make form fields accessible just by appending a $ sign as in old php3.
Notice the two $ signs.

foreach ($_POST as $key=>$val)
{
 $$key=$val;
}

Linux : Finding unique types of files

Here's a quick way to figure out what different file-types (based on extension) you have. The following example checks the working directory. A hash of extensions is used in the perl code that appears in single quotes.

find . -name "*.*" | perl -e 'while(<>){$ext{$1}++ if(/\.([\w]+)$/);} $,="\n"; print sort keys %ext;'

Do you find this useful ?

Linux : Changing ownership with find command

Good example of find with -exec...

Following command finds all files under /var/www/html owned by user ketan and changes the owner and group to webuser and web respectively. Only root can do this.

find /var/www/html -user ketan -exec chown webuser:web {} \;

Documentation with Pod

Pod is a simple-to-use markup language used for writing documentation for Perl, Perl programs, and Perl modules. POD stands for Plain Old Documentation . Pod documentation can be included in perl scripts or modules.

Translators are available for converting Pod to various formats like plain text, HTML, man pages, and more.

Following commands are supported

    =head1 Heading Text
    =head2 Heading Text
    =head3 Heading Text
    =head4 Heading Text
    =over indentlevel
    =item stuff
    =back
    =cut
    =pod
    =begin format
    =end format
    =for format text

Following is an example of a pod document. To test, copy the following part into a file and run perldoc filename.

#######################################
=head1 NAME

abc.pl

=head1 DESCRIPTION

This line shows up as description. This line shows up as description. This line shows up as description. This line shows up as description.

=head1 USAGE

I< perl abc.pl >

 The script asks for 3 inputs. They are -

 * x
 * y
 * z

=head2 heading2

Try this out ! You may want to include documentation of your scripts in Pod.

=head1 AUTHOR

ketan404

=cut
###############

Linux : find and replace with grep and sed

Following command finds PATTERN and replaces it with REPLACEMENT in all files under directory/path/.

grep -rl PATTERN directory/path/ | xargs sed -i 's/PATTERN/REPLACEMENT/g'

Notes :

-r option of grep does recursive search
-i option of sed edits files in place
g does substitutions in all occurences

Bash : Saving time with Environment Variables

You can create an environment variable $dr (for document_root) to save typing the whole path /var/www/html.

In your .bashrc file add this line

export dr=/var/www/html

Then source .bashrc. After this you'll be able to use shortcuts like

vi $dr/index.htm

With export, we create a new environment variable. In c shell, the syntax is

setenv dr /var/www/html

You can see all the environment variables with env.

Perl : Regular Expression to match an email address

Following regular expression matches a valid email address

/^(\w|\-|\_|\.)+\@((\w|\-|\_)+\.)+[a-zA-Z]{2,}$/

To learn more about regular expressions in Perl, read

man perlre

Linux : find files, find patterns and replace in one go

Following command finds all text file in current working directory and replaces XYZ with ABC in all found files.

find . -name "*.txt" -exec perl -pe 's/XYZ/ABC/g' -i {} \;

Notes

• You need to have perl installed on your system
• with -exec you can execute a command on the found files
  
• -i option of perl is used to update files
• -p option of perl command assumes a loop around the code just like the -n switch. Only difference is that it prints the lines.
• XYZ is a regular expression to be searched and ABC is the replacement text.
  

Fwd: Back after many days !

Hi !

I am back after many days of inactivity.

PHP : Getting image size

PHP has a very handy function getimagesize() which returns an array of values like width, height, type etc.

Example

$image_size = getimagesize("path/to/the/image.jpg"); ## URLs can also be passed

$image_size[0] is the width and $image_size[1] is the height of the image.

You can also use the following line (picked from php.net) so as to get the values in four different variables directly with the use of list() function.

list($width, $height, $type, $attr) = getimagesize( "img/flag.jpg");

Linux : Appending a tar file with xargs

Whereas -c switch of tar command creates an archive, -r is used to append files to a tar files. In the following example, xargs keeps on sending htm files as input to the tar command. These files are appended to htm_files.tar.

find . -name "*.htm" | xargs tar rvf htm_files.tar

Usual way of tar -cvf does not work for the above purpose.

Linux : Find and Rename files with -exec

Following command finds .html files in the current working directory and changes the extension to .htm. The command illustrates the use of rename with find's exec switch. A quick way of finding and renaming files.

find . -name "*.html" -exec rename .html .htm {} \;

Find is regarded as a power command by many Linux users. The above example supports the claim ! There are many seemingly complicated or time consuming tasks which can be done quickly with commands.

Linux : Counting files per directory with Bash command

With commands, you can do many intricate things. Following command which is a small script, counts the number of .htm files in each subdirectory upto depth 2 of the current working directory.

for x in `find . -type d -maxdepth 2`; do y=`find $x -name "*.htm" | wc -l`; if [ $y != 0 ]; then echo $x - $y; fi done

This command is useful for webmasters who spend some time once in a while to figure out number of files in each of the website's subdirectories.

Notes

1. The value of -maxdepth may be changed as needed
2. The above command uses current directory (find . -type ...). The dot may be replaced by path of a directory of which the stats is needed.
3. "*.htm" part of the filename may also be replaced with whatever you want

Perl : Obfuscation of ARGV

Did you know ?

$a=eval shift @{$x.=chr for qw * 65 82 71 86 * ;$x};

is just another way of saying

$a = shift @ARGV;

Same thing can be done in many different ways. And some ways are intended at making it hard for humans to grasp the code ! Read more about Code obfuscation.

Perl : Formatting code

Following one liner attempts to format your codes. Works on programs of perl, c, php etc where curly braces - '{ and }', are used for blocks, loops, conditions etc.

perl -pe '$tabs = substr($tabs,1) if (/^\s*}\s*$/);s/^\s*/$tabs/;$tabs.="\t" if(/{[^}]*$/);' -i.bk filename.c

Files are backed up with .bk extension.

Wildcards are allowed so *.c can be used in place of filename.c.

Perl : Editing files on the command line

With -i switch of perl, files can be edited on command line. You can optionally take a back-up of the old files by providing an extension.

perl -pe 's/\r\n/\n/;' -i file1

Replaces \r\n (dos newlines) to \n from file1.

perl -pe 's/\<\?php/<?/;' -i.bk *.php

Replaces <?php with <? in php files and creates back-up files with .bk extension.

vi/vim : Changing the tabsize

Default tabstop is 8. To set the tabsize to 4 spaces, just issue this command

:set ts=4

ts stands for tabstop.

To make this a default setting, write this line without the colon ':' in .exrc (.vimrc if you use vim) file.

Vim : Dictionary completion facility

Dictionary completion facility is useful when you don't remember the spelling of a word. To make use of this, we must first set the dictionary option so that vim knows where the dictionary is. Following command does that.

:set dictionary-=/usr/share/dict/words dictionary+=/usr/share/dict/words

Now, start typing a word in insert mode.

astro<ctrl+x><ctrl+k>

This shows the first match in the dictionary. You can loop through the matches by using <ctrl+n> (for next) and <ctrl+p> (for previous).

Notes

To understand why the -= and +=, use this command
:help :set+=

Regular Expressions : Dealing with special characters like asterisk

This post is for beginners of perl/regular expressions who find it hard to deal with special characters like asterisks if they appear in patterns. For instance, here is a pattern that contains an asterisk
4*3
If you want to replace or remove the asterisk, you will have to escape it with a back-slash ('\') in a regular expression. Following is a sample code.

$x = "4*3";
print "$x\n";

## following line removes the special character asterisk from the scalar $x
## note that the asterisk is 'escaped' by using a back-slash
$x =~ s/\*//;

print "$x\n";

Output

4*3
43

Use of a back-slash ('\') for escaping special characters is common where regular expressions are used. It's also used for escaping spaces in directory-names on shell prompt.

Some examples

character : escaping it
\ : \\
* : \*
+ : \+
/ : \/

Perl : Removing blank lines from a file

You can do intricate things even on command line. The following command prints filename on the command standard output after removing blank lines

cat filename | perl -ne 's/^\s*\n$//; print;'

Notes

• Replace filename with the path of the file from which you want to remove blank lines.
• You can optionally redirect the output by using '>' operator to create a new file.
  

Linux : Installing Packages .... common procedure

This post is for novices who are scared of building/installing new packages on linux :-) Here are the steps.

1. Download the package. It's normally a .tar.gz file. It could be in some other format like bz2 as well.
2. Extract it with tar -zxvf package.tar.gz (for .tar.gz files)
   
3. Change to the newly created directory with this extraction
4. Read README files (good practice)
5. Run on prompt
   ./confgure
6. then
   make
7. Become root if you have not already. Default location of installation of almost all new packages is /usr/local so you need write permission to create any new files there.
8. issue
   make install

You can install packages in other locations by using --prefix=/path/where/you/want/to/install option to ./configure. Learn more about options by using --help (normally) switch of ./configure scripts. All configure scripts may not present you with nice help.

Linux : Synchronizing directories with rdist

rdist is a linux command for keeping identical copies of directories/files over two hosts. File permissions and modification time are preserved if possible. It can use various protocols like ssh, rsh (default). You can have all the synchronization details stored in a file and pass the filename as an argument with option -f to the command as illustrated below.

rdist -f rdist_file

To specify the protocol, use -P option

rdist -P /usr/bin/ssh -f rdist_file

rdist_file is of the following format.

## comments start with '#' sign
/path/on/master/server    ->   remote_server_name  install  /path/on/slave/server ;

-o option is used for specifying an action to be taken on those files on the slave (remote server) which are missing from the master. For truly synchromising two directlries, you can use -oremove to remove the files. The directive then takes the following form.

/path/on/master/server    ->   remote_server_name  install -oremove /path/on/slave/server ;

For more about actions and about the command, see man pages.

Linux : crontab does not run perl scripts ?

Recently, I had hard time making crontab run my perl script. The problem was with the environment variable PATH. When given correct path, it started executing the scripts. Following could be the problems

• Wrong shebang path (the first line that starts with #!). This line tells which command/program to use for execution of the script.
• Incorrect permissions
  

If you want to execute a perl script, make sure the output of 'which perl' is used as the path in your shebang.

Away for quite some time

I'm perturbed by Governments resolution to increase the quot for OBC by 27 %. My own view is that reservations, if any, should be based on economic status and not on caste or religion. Does the Indian Government want to divide the society ?

I hope to be back for posting relevant content soon.

PHP : Website Security

PHP files that do database inserts or some other server-side tasks,
should not be accessible via a web browser. All files under Document
Root of a website are accessible by a web-browser. Document Root is
the directory on the server's filesystem. This directory contains the
files/web pages of the website.

To avoid direct access of your php files, you can keep them in a
directory that is not inside Document Root. These files can then be
included into your templates. For web applications, a configuration
file can be included in all files using a relative path. This
configuration file may contain variables like database connection
details and other configuration variables. Include for database
connection too, should reside outside the Document Root.

Variables of help

$_SERVER['DOCUMENT_ROOT']
This followed by relative path of your conf or include files can be
used on most servers.

On shared hosting server, Document Root is not defined for each
website (virtual host), so this variable shows the default value. In
such cases, you can use

dirname(__FILE__)

This gives you the value of the directory containing the file.

Regular Expressions : Removing blank lines of a file in vi

In this post, I'm going to show you how blank lines of a file can be removed in vi editor with Regular Expressions.

Following is an ex editor command.

:1,$g/^$/d

Notes :

• ex editor commands start with a colon.
• 1,$ stands for first line to last line of the file
• g for global (look for the pattern following everywhere; do not stop after the first match)
• pattern to look for is placed inside two slashes (//)
• ^ is for start of line and $ for end of line. Here, there's nothing between the two so the pattern is used for an empty line
• d in the end is the command to delete matching lines.

This regular expression does not match a line if it contains spaces. If you want to remove those lines as well, here's the modified command.

:1,$g/^ *$/d

Note that there's an asterisk (*) after a space. An asterisk matches the character preceeding it zero or more times.
e.g.

/a*/ matches a,aa,aaa,any number of times.
/\d*/ mathes an integer. \d is for a digit.

Linux : music compositions with sox

Recently, I created a small web application in php and javascript for Tabla compositions. For those who don't know ... Tabla is a percussion instrument. Currently, it is in its primitive stage. By the way, here is the link to it.

I'd like to take it further and add functionality which will allow merging of wav files with sox command that comes with linux distributions. This will allow composer to download his composition in wave format and hear it ! This will require reading the text file and mergin the sound files in the same order. This should be possible with a script (perl/php ?).

I'll soon be collecting .wav recordings of all the basic tabla bols (aksharas) and give it a try. Your comments or ideas are welcome.

Linux : date command

Following are some illustrations of the date command.

date
# This prints the current date. Something of this form - "Tue Jan 31 12:16:53 IST 2006"

date +%d
# prints current day of the month

Following examples illustrate getting time related information of a day/date other than the current date

date --date="3 days ago"
# prints date that was 3 days ago

Try these too !
date --date="last Sunday"
date --date="next Sunday"
date --date="Jan 26, 2004" +%a
# prints Day of the week on Jan 26, 2004

Setting the system date. You need to have admin privileges to use --set option.

date --set='+10 minutes'
# Takes system clock forwad by 10 minutes.

Perl : What are Regular Expressions, by the way ?

A regular expression describes a set of strings, according to certain syntax rules.

In this post, I just want to give beginners an idea. The topic 'Regular Expressions' is quite big. Unix commands - grep and sed were the first to popularize the concept. Many programming language support regular expressions for string manipulation. Perl has a powerful regex engine. The support/use of regular expressions gives Perl very powerful text-processing capabilities.

man perlre gives you access to a man page dedicated to regular expressions and their use in Perl. Beginners, mastering regular expressions is the key to excelling in Perl programming !

Enjoy regular expressions :-)

Perl : Executing one line codes on the command line

Small codes of perl can be executed on command line with -e switch. The code is contained in single quotes. Following is an
example.

Command :

perl -e ' $\=" "; for (1 .. 9) { print; } '

Output :

1 2 3 4 5 6 7 8 9

Another frequently used switch is -n. This switch assumes 'while(<>){...}' around your code. Useful for processing all lines of the input coming through pipes. Example follows.

Command :

ls -l | perl -ne ' print if(/^d/); '

Here, output of ls -l (long listing) is piped to the perl command. -n makes the code loop over all the lines of the input that come from the previous command. The regular expression checks if the lines start with the letter d. Thus the code prints details of only the directories contained in the working directory. The command is equivalent to

ls -l | perl -e ' while(<>){ print if(/^d/); } '
(without -n switch)

-a is autosplit mode with -n or -p (splits $_ into @F ). See how it works

ls -l | perl -ane ' print $F[0]."\t".$F[7]."\n"; '

This prints the 0th and the 7th fields of the long directory listing (output of ls -l).

You can learn more about switches with this command
perl -h

To see version, built and other license related info,
perl -v

Perl : Sorting a list/array

Sorting of a list/array can be done with sort function. Reversing the order is easy; done with the function reverse. See following code and the output.

Code :

@fruit = ("banana","mango","apple");
@sorted_fruit = sort @fruit;

$,=" "; # $, is the output field separator (a predefined variable). This is used here to put spaces after printing every element of the array.
$\="\n"; # $\ is the output record separator. Used to add a newline to print in this example.

print @sorted_fruit;
print reverse @sorted_fruit; # reverses the order in the list

Output :

apple banana mango
mango banana apple

This Blog : Conventions I use

Hi !

This post does not have any code :-) Relief.

Wanted to make you aware of the conventions I have been following.
You'll observe that green color is for the code. Comments are preceded
by a '#' sign. This is because codes of perl, shell scripts and php
use # for commenting. You'll find commented code in brown (well
mostly; I may miss out on a few occasions). I'll also use the same
brown color for [key]words worth noting. Your suggestions in this
regard are welcome.

Feel free to share this blog with your friends. I believe this blog is
shaping into a good learning resource. If you wish to have posts on a
particular subject, let me know. I'll be more than happy to help you
as much as I can.

You'll see some more Perl code snippets in the coming posts.
Thanks for your patience.

Perl : 'Hello World !' and some more

This post is for those who want to start learning Perl. Save the following line in a file called 'hello.pl'. You can start learning Perl from the following example codes

print "hello world !";

and run it with

perl hello.pl

That's it ! It shows the output on the console.

Some more code with comments follows. You may save this in another file and run it with perl command.
Comments begin with a #.

Code :

$name = "ketan404"; ## a scalar. Note the $ sign. It's good to use my while variable declarations but we shall cover that la ter.

@fruits = ("apple","banana","mango"); ## this is an array. On the right side of the '=' sign is a list. Note the @ sign.

%fruit_prices = ("apple"=>4,"banana"=>1,"mango"=>10); ## this is called a hash. Names of the fruits are its keys and thier p rices the values.

## printing these

print $name."\n"; ## print the scalar and concatenate a newline to it. A dot (.) is used for concatenation.

print @fruits; ## works ! print can also take an array as an argument.

print @fruits."\n"; ## Oops ! You appended a newline to it ? Using @fruits in scalar context gives the number of elements in the array and thus prints 3 with a newline. You'll understand these things as you make progress :-)

## printing keys and values of a hash

foreach $k (keys %fruit_prices){
        print $k."=>".$fruit_prices{$k}."\n";
} ## works as expected
## keys function returns an array of keys of a hash.
## foreach or for are used for traversing an array

PHP : Object Oriented Programming in PHP

Object oriented programming is the use of objects to represent functional parts of an application. With php too, you can do object oriented programming to reduce and simplify the code. Following code is an example of the use of a php class.

Code :

<?php
class web_developer{

        var $skills; # class variables
        var $sal;

        function web_developer($sal){
        $this->sal=$sal;
        }

        function getSal(){
        return $this->sal;
        }

        function addSkill($technology,$yrs){
        $this->skills[$technology]=$yrs;
        }

        function getSkills(){
        return $this->skills;
        }

} # class ends here

$x=new web_developer(2000); # creating a web_developer (an object) with associated salary !

echo $x->getSal()."<br />";# retrieving a property (sal) of the object

$x->addSkill('php',5); # adding his skill-set and no. of years of experience
$x->addSkill('perl',4);

$x_skills=$x->getSkills();# retrieving the property (skills) of the object

foreach ($x_skills as $k => $v){
echo "$k - $v<br />";
}
?>

Output in a browser:

2000
php - 5
perl - 4

New on this Blog : Subscription by Email

Hello readers !

You might have noticed that I have added a form on the right for email subscriptions. Don't be afraid. I'm not going to fill your inboxes with emails :-) If you find the content on this blog useful and would like to be notified about the new posts, consider filling out the form. I shall send you email notification manually from my gmail account since I don't have an account with any mass-emailing website.

Enjoy Perl for some time now !

Perl : map() with your subroutines

You can use map function with your subroutines as well. Have a look at the following example.

Code:

## subroutine that returns double of a number (or whatever passed to it as an argument :-)
sub double_it{
        2*$_;
}

@nums = (1 .. 10); # array containing numbers 1 to 10
@doubles = map(&double_it, @nums); ## using map to create @doubles with subroutine double_it

$\=" "; # $\ is the built-in variable to specify the output record separator.

        for(@doubles){
        print;
        }

Output:

2 4 6 8 10 12 14 16 18 20

Perl : map function

See what the following code does.

Code:

@numarray = (97 .. 122);
@charray = map(chr, @numarray);
print @charray;

Output:

abcdefghijklmnopqrstuvwxyz

Explanation:

• .. is a range operator. @numarray contains numbers from 97 to 122
• map function applies function char on each element of @numarray. The output is stored in @charray.
• chr function returns the character represented by that NUMBER in the character set.  For example, "chr(65)" is "A".
  
• For more information on built-in perl functions, do 'man perlfunc'.

PHP : Showing an image randomly from a collection of images

Following code shows one .jpg image randomly from all images in a directory. The code needs to be saved in a file in the directory containing the images. This code can be used for creating random image gallery.

<?php
############### settings #############
$path = "/image_gallery/";
#######################################
$dir = $_SERVER['DOCUMENT_ROOT'].$path;
$ig = Array();
if (is_dir($dir)) {
    if ($dh = opendir($dir)) {
        while (($file = readdir($dh)) !== false) {
                if(ereg("\.jpg$",$file))
                array_push($ig,$file);
        }
        closedir($dh);
    }
}
$num = mt_rand(0,count($ig)-1);
echo "<img src=\"".$ig[$num]."\">";
?>

Notes :

1. The code opens a directory specified by $dir and puts all jpg filenames in an array $ig.
   
2. mt_rand function returns an integer between 0 and count of $ig i.e. number of jpg images in the directory.

CSS : Chaging cursor styles

By changing the value of cursor, you can change the appearance of the cursor. Cursor may have one of the following values.

• auto
• crosshair
• default
• help
  
• move
• pointer
• progress
• text
• wait
  

Move mouse over the lines below to see how the cursors look !

Cursor auto

When the value of cursor is set to auto, browser determines what style to show depending upon the context. i.e. if it sees
a link, it shows a pointer (a hand), default otherwise.

Cursor crosshair

Cursor help

Cursor default

Cursor move

Cursor pointer

Cursor progress

Cursor text

Cursor wait

Changing cursor styles with CSS adds great functionality to web pages. This can also be done in JavaScript. For Rich Web Applications (that use XMLHttpRequest for speed and better interaction), it's a good idea to change the cursor style to 'wait' after sending request to the server so that user expects some response from the server to be seen somewhere in the web page.

Following line sets cursor style to wait.

document.body.style.cursor="wait";

After loading completely the response from the server, the style may again be set to default.

Ajax - The key to Rich Web Applications

Asynchronous JavaScript And XML is a web development technique for developing rich web applications. AJAX is aimed at increasing speed, interactivity and usability of web applications. More info from Wikipedia.

Typically (in most cases), getting a response from server involves refreshing/reloading the page you are viewing or going to a new page. With ajax, you can do it without refreshing. This is ajax in short :-) The natural question that should arise is 'HOW?'. Modern browsers come with support for an object called XMLHttpRequest. This object is used in ajax applications that require making an http request to the server and loading the response without refreshing the page.

While creation of an online account you have to provide a username. And if somebody has already taken it, you come to know about it only after submission of the form. This can now be avoided ! You can send an http request to the server with the username filled in by the new user as soon as the user leaves the text-field (onBlur event of javascript). The response can be shown next to the textfield so that user notices it immediately (as soon as it appears there).

So why should you use ajax ?

1. With ajax, you don't have to refresh your page for server responses.
2. You can get only the required server reponse and show in the desired place on your web-page. This reduces the byte-transfer considerably. There's no need to transfer all the html that show your headers, footers, images and navigation.

The methods and properties of this object are explained below.

var request = new XMLHttpRequest(); // creates an object of XMLHttpRequest

open() - new request to server.
send() - Sends a request to the server.
abort() - aborts current request
readyState - provides the current HTML ready state.
responseText - the text that the server sends back to respond to a request.

More information on the subject can be found on this link.

Some examples -
1. www.gmail.com - most of us know what it is :-)
2. http://gollum.easycp.de/en/ - a wikipedia browser
3. http://a-i-studio.com/cmd/ - a WebShell !

Bash : sample loops (shell scripting)

Here are some sample loops in bash scripting. Feel free to leave a comment if something is not clear to you. I recommend using bash for learing shell scripting.
--------------------
#!/bin/bash
for i in `seq 1 10`
do
let i=i*5
echo $i
done

--------------------
# seq outputs a sequence of numbers (1 to 10)
# let is a bash built-in command that allows arithmetic operations to be performed and assigned to a variable as done above
---------------
for i in $(ls)
do
echo $i
done

---------------
j=0
while [ $j -lt 10 ]
do
echo $j
let j=j+1
done

----------------
# same example with until
# note the -eq switch in until loop
--------------------
j=0
until [ $j -eq 10 ]
do
echo $j
let j=j+1
done

---------------------

Shell scripting

What is shell scripting ?

A shell script is a file containing unix/linux commands. Shell scripts are written to avoid repetitive work of issuing the same set of commands over and again. Shell grammar allows us to put conditional and looping constructs (as commands) making it easy to write scripts to do complex tasks.

Following is a simple shell script that shows date, operating system info and list of files in a working directory.

----------------------------
## simple.sh (filename)
echo "a simple shell script"
date
uname -a
ls
----------------------------
Lines starting with a # are comments.

To execute this script,

bash simple.sh
or
sh simple.sh

Shell scripts can be used like any other operating system commands. For this they need to be placed in a directory which the shell searches for commands. You can see these directories by

echo $PATH

You can update the envronment variable PATH to add your own directory to it. Say you want to add ~/bin (directory bin in your home), issue the following commands (bash specific).

PATH=$PATH:$HOME/bin
export PATH

The scripts/programs in this directory need to have execute permission if you want to use them as commands. Execute permission can be added by using chmod command

chmod +x scriptfile

I'll post some examples of small shell scripts in next few posts.

Linux/Unix : File permissions, groups and access control

There are three types of permissions namely r(read), w(write), x(execute).

Read permission - in context of a file, this means you can read (and thus copy) the file. If you have read permission on a directory, you can see the contents of the directory (usually with ls command).

Write permission - on file means you can change the contents of the file. Write permission on a directory means you can create or delete files/directories in that directory.

Execute permission - If a file has execute permission you can run it just like a command. Usually, shell scripts, if they are used like a command, need x permission for the user. Execute permission on a directory means you can change to that directory using cd command.

On Linux (and unix flavors) users can be put in groups. A given set of users works on the files owned by a certain group. One user can be a member of many groups. Access control becomes effective with proper use of groups and permissions.

Long listing format (ls -l) shows the permissions on a file/directory.

The output is of the following form

-rw-rw-r--    1 abcd  web     24593 Mar  4  2006 test.txt
drwxrwxr-x    2 abcd  web      4096 Oct  7 16:06 temp_files

The first field shows the permissions on the files. Second field shows number of files in the corresponding directory. Third
 field (abcd) is the owner of the file. Next is the name of the group. Other fields are file-size, modification time and fil
ename in the same order.

The permissions field consists of 10 characters. First denotes the file-type. '-' for a plain file, 'd' for a directory. Following three characters show permissions of the owner of the file. In our case, the owner is 'abcd' and his permissions are 'rw-' (read, write but no execute). Following three characters are for the group (web). In our case, all members of this group have rw- permissions i.e. the same as the owner. Next three charaters are for others. Others have just 'read' permission.

Types of files
----------------
d - directory
l - link
p - pipe
b - block special device
c - character special device

Changing permissions
--------------------------
Only owner or root (administrator) can change permissions on a file. Following are some illustrations.

chgrp web filename
changes group of the file to web.

chmod g+w somefile
gives write permission on somefile to group.

chmod +x some_script
gives execute permission to all on some_script. Typically, shell scripts or some other executable files are given execute permissions.

chmod o-x some_script
removes execute permission for others (but retains for the owner and the group)

If in chmod command permissions start with a letter, following are the meanings of these letters

u - user or owner of the file
g - group members
o - others (rest of the world)
a - all

What does the following command mean then ?
chmod 664 some_filename

Permissions can also be set using octal value for the three bit pattern. Using this method, permissions on a file can be set in one go. r, w and x have corresponding values.

r = 4
w = 2
x = 1

Therefore, the above command assigns 6=4+2 i.e. 'read' and 'write' to the owner of the file, same for group whereas others have just 4 i.e. 'read' permissions on the file. To get more information on changing permissions refer the man page of chmod.

Perl : Arrays [2]

In this chapter, I'll discuss some frequently performed functions on Perl arrays.

Let's consider this array
@fruit = ("apple","banana","pineapple","mango","guava");

push @fruit,"plum";
Adds an element to the end of the array.

my $new_fruit = shift  @fruit;
The above statement removes the first element of @fruits and assigns it to $new_fruit.

my $popped_fruit = pop @fruit;
The above line pops (removes last element from the list) and assigns it to $popped_fruit.

Use of @_ in a subroutine

@_ is the default array that is passed to a subroutine.
See the following two subroutines and what they do

sub test{
($name,$surname,$age)=@_;
}

We can assign three scalars of a subroutine with the values of the arguments to the subroutines. @_ is the array of argument
s passed to the subroutine.

sub test2{
$name = shift;
$surname=shift;
$age=shift;
}

If these arguments are not going to be required for further processing, the default array can be shift'ed and the assignment
 be made.

Linux : Use of Pipes in commands

A pipe ( | ) is used to send output of one command to some other command. Many a time, use of pipe simplifies our task to a great extent. I'll discuss a few examples

Count number of files in a directory

ls | wc -l

List all users on a system who are running some processes (process owners)

ps -ef | cut -d " " -f1 | sort | uniq

List all process-listings that contain the word "mozilla"

ps -ef | grep mozilla

Find the word "services" in all .htm files

find . -name "*.htm" | xargs grep "services"

Sort lines in a file and print on the console

cat filename | sort

Killing mozilla

ps -ef | grep "mozilla" | cut -f1 | xargs kill

Print last 10 lines of a file

tail filename | lpr

Here's a good page on Linux Tips.

This wikipedia page has more information on unix pipes.

Perl : Arrays in perl

Creation of arrays

Following statements create perl arrays.

my @fruits = ("apples","bananas","mangoes","grapes");
my @numbers = (23, 42, 69);
my @mixed   = ("bananas", 4.2, 25);

Creating arrays from other arrays.

my @new_fruits = @fruits[1..3]

This cerates a new array which has "bananas","mangoes" and "grapes" of @fruits. This is called an array slice.
Some more ways ...

my @new_fruits = @fruits[1..$#fruits]

$# is a special variable which gives the last index of an array.

my @new_fruits = @fruits[1,2]
creates array of 2nd and 3rd element of @fruits.

Looping through arrays

$\="\n";
for (1..12){
print;
}

Probably the simplest form to understand. You can set a range with '..'
No argument is provided to 'print' so it takes the default input to the loop.
$\ is the output field separator. In our case, it's a newline. Alternatively, (without the first line) we could have written the print statement as follows.

print "$_\n";

$_ is the default input. As in the first case, perl assumes the default input '$_' as an argument to print (and other) statements if not mentioned explicitly.

The following code prints the locations of the perl modules on your system. @INC is the predefined perl array.

for (@INC){
print "$_\n";
}

If used in scalar context, @fruits gives the number of elements in @fruits (array).
e.g.

$x = @fruits;
print $x;

This prints 4 (number of elements in the array).

Some more on perl arrays will be posted here soon. Stay tuned !

Perl : Sorting a hash

Consider this hash.
%my_hash = ("apple"=>20,"banana"=>5,"chiku"=>10,"pineapple"=>30,"mango"=>50);

Keys of this hash are names of fruit and the values are numbers.
With 'sort' and 'keys' functions we get a hash sorted by its keys.

foreach $k(sort keys %my_hash){
        print "$k => $my_hash{$k}\n";
}

This prints
----------------------
apple => 20
banana => 5
chiku => 10
mango => 50
pineapple => 30
----------------------

Following code reverse-sorts the same hash by its values and prints it.

foreach $k(sort { $my_hash{$b} <=> $my_hash{$a} } keys %my_hash){
        print "$k => $my_hash{$k}\n";
}

Output
---------------------
mango => 50
pineapple => 30
apple => 20
chiku => 10
banana => 5
---------------------

The above can be done with this code.

foreach $k(reverse sort { $my_hash{$a} <=> $my_hash{$b} } keys %my_hash){
        print "$k => $my_hash{$k}\n";
}

Notes :

• 'reverse' has been added in the above code so that sorting takes place in revese way. However, the positions of $a and $b have changed too.
• <=> is used for numerical comparison. This is the comparison operator.
• 'cmp' is used for comparison of strings.
• The operators '<=>' and 'cmp' return either of 1,0,-1. These when used with 'sort' do the sorting !
  

Knowledge management with Perlfect (a search engine coded in Perl)

We tend to forget things that we knew once and time has to be spent either in digging the old archives/notes or learning the same thing afresh. Yes, many people have to deal with such a range of subjects that they can not recall everything when they need it. For example, a LAMP freelancer will find it hard to remember all Apache (web-server) directives and the new directives that came with the latest version. Besides, this programmer needs to have quick access to Linux, MySQL, PHP, Perl related documents. Efficient knowledge management will certainly boost his/her performance. This becomes more beneficial if knowledge can be shared in a group or organization.

This is typically done by running an intranet website on the Local Area Network and running a search engine to index the documents on it.

How can I set up an intranet website ?

You need to run a web-server. Apache which is a free software is the web-server of our choice. If you have a LAN, you should be able to access your intranet from any computer on the Local Area Network.

Knowledge management with Perlfect

Perfect is a search engine developed in Perl and distributed under GNU GPL (General Public License). You can download and use this for free. Once installed, you need to run the indexer to index the files on your intranet. You are all set to use the results !

Create a directory on your intranet to save your knowledge-documents and get the files indexed. A LAMP programmer may want to download php documentation, to be indexed so that he does not have to search the internet often. Text and pdf documents are also indexed so you can save your own tips/tricks in text files for future reference.

Contributing to the growth of your Knowledge base

If you are working in a group, encourage everybody to save their tips, findings in text files or any other format that Perlfect understands. While writing your knowledge-document, make sure you include all the possible search phrases that a user is likely to use. Assign the responsibility to run the indexer once in a week or more frequently depending upon the size of your group. Perlfect search form can be set to search within a particular directory so you can have different forms for different subdirectories of your intranet if the search starts yielding too many results.

Linux/Unix: vi/vim editing tips

How do I see line numbers ?

:set nu
:set nonu
does the opposite.

Indentation for formatting your programs/scripts

:set autoindent

Saving vi/vim preferences

You can create a file (a hidden file) by name .exrc and write all your ex commands one per line. You can copy-paste the following two lines as .exrc. These options/preferences will work next time you open vi.

set autoindent
set nu

Moving around quickly

* Last line - shift+g
* First line - gg
* One screen forward - ctrl+f
* One screen backward - ctrl+b
* Half screen forward (down) - ctrl+d
* Half screen backward (up) - ctrl+u
* 50th line - 50,shift+g
* Move forward a word - w
* Move back a word - b
* To start of line - 0
* To end of line - $
* Down a line - j
* Up a line - k
* One char left - h
* One char right - l

Deleting content

* Delete a character - x
* Delete a word - dw
* Delete a line - dd

You can multiply these commands with numbers ! e.g. ' 5dw' deletes 5 words.

Chaging content

* Change word - cw
* Change line - cc

Inserting content

You can go into insert mode by following ways

* Insert where cursor is - i
* From start of line - I (shift+i)
* Next line - o
* Previous line - O (shift+o)
* Next character - a
* End of line - A (shift+a)

Copying and pasting

The command to copy content is 'y' (yank).

* Copy a line - yy
* Copy a word - yw

Copy 5 lines - 5yy as explained above. Multiplication applies here as well.
Paste - p
Paste before the cursor position/previous line - P (shift+p)

Content from the last delete (cut) command is stored in a buffer by default. dd followed by p cuts current line and pastes it after the next line. Similarly, x followed by p swaps the chars of a word.

Saving and reading files

* :r filename - read file named filename and insert after current line (the line with cursor)
* :w - write current contents to file named in original vi call
* :w newfile - write current contents to a new file named newfile
* :12,35w smallfile - write the contents of the lines numbered 12 through 35 to a new file named smallfile
* :w! prevfile - write current contents over a pre-existing file named prevfile

Find and replace (substitutions)

This is normally done with ex editor command. Following is the syntax.
For replacing everywhere in the file,
:1,$s/pattern/replacement/g

Explanation

: indicates the start of an ex editor command
1,$ - from the first line to the last line. Alternatively, you can use % (for all lines of a file)
/ used to separate pattern and replacement. Here, you can use regular expressions.
g stands for all occurences of the pattern on a line

Linux/Unix: Environment variables

What are they ?

Environment variables are a set of dynamic values that can affect the way running processes will behave. All unix like operating systems as well as windows have their set of enviraonment variables. In unix like systems change in environment variable will affect only the child process run by that particular program or shell script which changes the environment.

Examples -

$PATH - shows the directories where shell looks for commands issued by a user
$HOME - shows where user's home directory is located on the file-system

env command shows all environment variables specific to your login.

Setting an environment variable

In bash, following two commands set up an environment variable called 'name'

> name=Ketan
> export name

To verify this, you can

> echo $name
or
> env | grep name

If you want a certain environment variable to be set when you login, keep the two commands in your .bashrc file.

Misc: why the name "Ketan's weekly"?

I intend to post something here on weekends. Rarely on weekdays :-)
Hope you find this blog useful.