ERROR: This RRD was created on another architecture

Recently I have to migrate our cacti installation to new machine during which i encounter few issues.

After copying the RRD files to new machines and accessing cacti web http://myserver/cacti gives empty graphs.  (wired)

After digging the logs (in apache error log) found out that RRD files is not portable between different architechture.

ERROR: This RRD was created on another architecture

In my case the old machine was 32bit arch with 32bit OS, and the new machine is 64bit with 64bit OS.

Quick googling tells that using rrddump and rrdrestore one can transfer RRD between architectures with ease.


EXAMPLES

To transfer an RRD between architectures, follow these steps:

1.  On the same system where the RRD was created, use rrdtool dump to export the data to XML format.

# for i in ./*.rrd;do rrdtool dump $i ../rrd/$i.xml;done
(to converted all the files in rrd directory to XML format)

2.  Transfer the XML dump to the target system.

3.  Run rrdtool restore to create a new RRD from the XML dump.

# for i in ./*.xml; do rrdtool restore "$i" "../rrd/${i%.xml}"; done

After this cacti was up and running with all the old graphs migrated successfully.

Installing Nagios-plugins and NRPE on Centos 6

To monitor remote hosts CPU load, disk partitions, processes etc with Nagios, requires to install NRPE and nagios-plugins on the remote host.

Nagios-plugins and NRPE is not available from Centos official repositories, so first of all we need to configure RPMforge> repo from where to install the required packages.

Read my previous article before configuring and installing anything from third party repo.

Installing RPMforge on Centos 6

Download and install the rpmforge-release package. Choose one of the two links below, selecting to match your host's architecture. If you are unsure of which one to use you can check your architecture with the command uname -i

x386
# rpm -Uvh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.2-1.el6.rf.i686.rpm

x86_64
# rpm -Uvh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.2-1.el6.rf.x86_64.rpm

Do not forget to set the priority,

vi /etc/yum.repos.d/rpmforge.repo

[rpmforge]
name = RHEL $releasever - RPMforge.net - dag
baseurl = http://apt.sw.be/redhat/el6/en/$basearch/rpmforge
mirrorlist = http://apt.sw.be/redhat/el6/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1
priority=10

Now everything is setup now you can install nagios-plugins and nrpe packages from RPMforge repo

# yum install nagios-plugins nagios-nrpe

Running NRPE under xinetd

Edit /etc/xinetd.d/nrpe for

disable         = no
only_from       = 127.0.0.1 Nagios ServerIP

Restart xinetd service
# /etc/init.d/xinetd restart

My KVM howto

Enabling KVM support on your hardware

The host machines need to be running either Intel VT or AMD-V chipsets that support hardware-assisted
virtualization.

# grep -E 'vmx|svm' /proc/cpuinfo

If this command returns output, then your system supports KVM. The vmx CPU feature flag represents
Intel VT chipset while the svm flag represents AMD-V. Note which KVM flag was returned as it will be
useful for loading the right module later.

Next you need to ensure the KVM-related feature is enabled in the BIOS

1. In the BIOS menu, select Advanced Step → CPU Options.
2. Make sure the Intel Virtualization Technology option is Enabled.

Installing and configuring KVM-related package

Install the KVM software using yum:
# yum install kvm

Install additional virtualization management packages:

# yum install libvirt python-virtinst libvirt-python libvirt-client bridge-utils


Configuring KVM after installing the packages

After you have installed the KVM-related packages, load the right KVM modules by following these
steps:

Procedure
1. Insert KVM module by running the following command:
# modprobe kvm

2. Insert the chip-specific KVM module by running one of these commands:
For the AMD chip (svm flag)
# modprobe kvm-amd

For Intel chip (vmx flag)
# modprobe kvm-intel

You can verify that the modules are inserted and running.
# lsmod|grep kvm
kvm-intel    86248 3
kvm     223264 1 kvm_intel

3. Start the libvirtd daemon service:
# /etc/init.d/libvirtd start

Starting libvirtd daemon:
# /etc/init.d/libvirtd status
libvirtd (pid 6584) is running.

4. Set up libvirtd to start on every reboot:
# chkconfig libvirtd on

5. Better to reboot the host machine.


Creating LVM for Guest OS (Run lvdisplay before creating new LVM)

# lvcreate -L 4G -n kvm5 vg_kvm

-L 10G = replace with filesystem size
-n kvm1 = replace with kvm#

Creating KVM guest using virt-install

virt-install --name kvm1 \
--ram 512 \
--os-type='linux' \
--disk path=/dev/mapper/vg_kvm-kvm1 \
--network network:default \
--accelerate \
--vnc \
--cdrom /iso/CentOS-5.5-x86_64-bin-DVD-1of2.iso

For network installation replace --cdrom with --location with URL to OS installation link.

Eg, Centos 5.6 64bit
--location http://mirror.its.sfu.ca/mirror/CentOS/5.6/os/x86_64/

Creating Scientific Linux 6 64bit vm with bridge networking and two virtual CPUs

Note: br0 bridge interface should exist before using it for KVM

virt-install --name kvm3 \
--ram 1024 \
--vcpus 2 \
--os-type='linux' \
--disk path=/dev/mapper/vg_kvm-kvm3 \
--network bridge:br0 \
--accelerate \
--vnc \
--location http://ftp1.scientificlinux.org/linux/scientific/6/x86_64/os/


*** Installing Slackware off NFS (NFS running on xxx.xxx.xxx.60 machine)

virt-install --name slack64 \
--ram 512 \
--vcpus 1 \
--os-type='linux' \
--disk path=/dev/mapper/vg_kvm-kvm8 \
--network bridge:br0 \
--accelerate --vnc \
--cdrom /iso/slackware_x86_64-13.37-mini-install.iso

Cloning KVM VM
1. LVM based
Cloning KVM (Create LVM partition for cloned first ie 'lvcreate -L 4G -n kvmX vg_kvm' run lvdisplay to check existing LVMs)
virt-clone --original kvm1  \
             --name kvmX \
              --file /dev/mapper/vg_kvm-kvmX --prompt


2. File based
virt-clone --original kvm1  \
             --name kvmX \
              --file /var/lib/libvirt/images/centos6-2.img

Resizing KVM VM disk storage
1. install libguestfs-tools package.
2. Best to have separate parititons for /, /boot, and swap, then following http://askaralikhan.blogspot.com/2011/07/expanding-kvm-guest-disk-image-using.html


Troubleshooting For network interface

1. Open /etc/sysconfig/network-scripts/ifcfg-eth0 and delete the following line and save the file
HWADDR=52:54:00:8f:62:01
2. Remove the following filen and reboot the virtual machine
/etc/udev/rules.d/*persistent-net*'
After reboot eth0 network interface will be available.

Removing LVM partition: Remove LVM partition which is no longer need.


# lvchange -an /dev/mapper/vg_kvm-kvm8 (change LVM status to inactive)
# lvremove /dev/mapper/vg_kvm-kvm8 (remove the LVM paritition)

Expanding a KVM guest disk image using virt-resize

Decide on how you wish to expand the guest. Run virt-df -h and virt-list-partitions -lh on the guest disk, as shown in the following output:

# virt-df -h /dev/vg_kvm/kvm7
Filesystem                                Size       Used  Available  Use%
/dev/vg_kvm/kvm7:/dev/sda1              484.2M      48.8M     410.4M   11%
/dev/vg_kvm/kvm7:/dev/sda3               2.5G       1.3G      25.4G    5%

# virt-list-partitions -lh  /dev/vg_kvm/kvm7
/dev/sda1 ext4 500.0M
/dev/sda2 swap 1.0G
/dev/sda3 ext4 2.5G

This example we will increase the size of /dev/sda3 to 30G

1. Shutdown guest

2. Rename the original disk for the backup purpose, for lvm use lvrename command, for file use 'mv' command

# lvrename /dev/vg_kvm/kvm7 /dev/vg_kvm/kvm7.backup
  Renamed "kvm7" to "kvm7.backup" in volume group "vg_kvm"

3. Create new disk
# lvcreate -L 30G -n kvm7 vg_kvm
Logical volume "kvm7" created

4. Now resize the /dev/sda3, refer to virt-resize man for detail.

# virt-resize \

       /dev/vg_kvm/kvm7.backup /dev/vg_kvm/kvm7 \

       --expand /dev/sda3

Summary of changes:
/dev/sda1: partition will be left alone
/dev/sda2: partition will be left alone
/dev/sda3: partition will be resized from 2.5G to 28.5G
Copying /dev/sda1 ... done
Copying /dev/sda2 ... done
Copying /dev/sda3 ... done

Parition /dev/sda3 contains a filesystem or LVM PV, then this content is not automatically resized.  You can resize it afterwards either using guestfish(1) (offline) or using commands inside the guest (online resizing).

Here we resize the filesystem using guestfish

[root@kvm ~]# guestfish -a /dev/vg_kvm/kvm7

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

> run
> resize2fs /dev/sda3
libguestfs: error: resize2fs: resize2fs 1.41.12 (17-May-2010)
Please run 'e2fsck -f /dev/vda3' first.
> e2fsck-f /dev/sda3
> resize2fs /dev/sda3
> exit

[root@kvm ~]# virt-df -h sl6-5
Filesystem                                Size       Used  Available  Use%
sl6-5:/dev/sda1                         484.2M      48.8M     410.4M   11%
sl6-5:/dev/sda3                          28.1G       1.3G      25.4G    5%

Try to boot the virtual machine. If it works you can delete the backup disk.

# lvchange -an vg_kvm/kvm7.backup

# lvremove vg_kvm/kvm7.backup
  Logical volume "kvm7.backup" successfully removed

Clone KVM with virt-clone

virt-clone is a command line tool for cloning existing virtual machine images using the "libvirt" hypervisor management library. It will copy the disk images of any existing virtual machine, and define a new guest with an identical virtual hardware configuration. Elements which require uniqueness will be updated to avoid a clash between old and new guests

virt-clone of python-virtinst package is a great tool for creating cloned KVM virtual machine quickly. 

Example:  LVM based kvm

# virt-clone --original sl6-3  \
             --name sl6-4 \
              --file /dev/mapper/vg_kvm-kvm6 --prompt

In the above example /dev/mapper/vg_kvm-kvm6 LVM should be created before executing the virt-clone command.

Creating LVM
# lvcreate -L 4G -n kvm6 vg_kvm

Example file based storage
virt-clone \
              --original ubuntu11 \
              --name ubuntu-2 \
              --file /var/lib/libvirt/images/ubuntu-2.img

--original sl6-3 = name of the existing KVM (make sure the original KVM is shutdown before creating clone out of it)

--name se6-4 = Name of the New KVM (cloned)
--file = In LVM case path to the existing storage, in file based storage the file will be automatically created by virt-clone

Troubleshooting Network issue
Upon booting the new cloned KVM VM I encounter with problem of not showing any network interface. After digging more it appears that udev on virtual machine changed the eth0 interface with eth1 due to change in MAC of the interface. Quick search on Google suggested the following solution to get back the 'eth0' interface on cloned vm.

1. Open /etc/sysconfig/network-scripts/ifcfg-eth0 and delete the following line and save the file

HWADDR=52:54:00:8f:62:01

2. Remove the following filen and reboot the virtual machine

/etc/udev/rules.d/*persistent-net*'

After reboot eth0 network interface will be available.

Scientific Linux 6

Recently I start using Scientific Linux on KVM virtual hosts. For anyone new to Scientific its a Enterprise Linux based on RHEL just like Centos maintained by Fermi National Accelerator Laboratory and the European Organization for Nuclear Research 'CERN' - and RHEL, of course :)

Scientific Linux folks is closely following the upstream srpm provider (RHEL) and its already shipped with version 6 just like RHEL 6.

The other things which inspired me to give a try to Scientific Linux is the latest version of Linux Kernel with latest hardware support, the ext4 filesystem is the default, KVM enhancement, latest versions of software eg PHP, Mysql which means I don't have to mess around with test repos or yum priorities

For detail What's new in RHEL/Scientific Linux please refer to RHEL.

I encourage anyone interested in RHEL and/or building your own test server, to experiment with this distro.

Medialess installation of Redhat Enterprise 6/Centos/Scientic Linux 6

This procedure assumes you are already using Red Hat Enterprise Linux or another relatively modern Linux distribution, and the GRUB boot loader. It also assumes you are a somewhat experienced Linux user.

To perform medialess installation of Redhat Enterprise 6/Centos/Scientific Linux 6 without media or a PXE server, the system must have two files stored locally, a kernel and an initial RAM disk

Copy the vmlinuz and initrd.img files from a Red Hat Enterprise Linux DVD (or DVD image) to the /boot/ directory, renaming them to vmlinuz-install and initrd.img-install.

Editing the GRUB Configuration

The GRUB boot loader uses the configuration file /boot/grub/grub.conf. To configure GRUB to boot from the new files, add a boot stanza to /boot/grub/grub.conf that refers to them


title Installation
    root (hd0,0)
    kernel /boot/vmlinuz-install ip=xxx.xxx.xx.x netmask=255.255.255.0 gateway=xxx.xx.xx.x dns=xxx.xx.xx.x vnc vncpassword=qwerty repo=http://ftp1.scientificlinux.org/linux/scientific/6/x86_64/os/ lang=en_US keymap=us
    initrd /boot/initrd.img-install

If using network interface other than eth0 then you must have to specify it with ksdevice=ethX, then the kernel line will read something like this for eth1

kernel /boot/vmlinuz-install ip=xxx.xxx.xx.x netmask=255.255.255.0 gateway=xxx.xx.xx.x dns=xxx.xx.xx.x ksdevice=eth1 vnc vncpassword=qwerty repo=http://ftp1.scientificlinux.org/linux/scientific/6/x86_64/os/ lang=en_US keymap=us

The following options are generally useful for medialess installations:

IP = yourIP
netmas = Netmask
gateway = your Gateway
dns = your dns
vnc (for remote graphical installation)
vncpassword = Yourpassword
repo = Redhat/Centos/SL repo (In the sample I use the Scientific Linux 6 repo)

Also change the entry "default 0" to boot from new kernel entries, when done reboot the system.

Completing the installation:
From your local system use vncviewer to connect to the remote server and complete the installation process

$ vncviewer RemoteHostIP

KVM virt-install: Install Scientific Linux 6 (RHEL6) as KVM Guest

Network installation of SL6 64bit as KVM guest.

virt-install --name se6 \
--ram 884 \
--os-type='linux' \
--disk path=/var/lib/libvirt/images/se6.img,size=10 \
--network network:default \
--accelerate \
--vnc \
--location http://ftp1.scientificlinux.org/linux/scientific/6/x86_64/os/

Connect from your local system to continue the installation

virt-viewer -c qemu+ssh://root@IPAdress/system se6

Installing skype on Fedora 14 64bit

Skype only available for 32bit version of Fedora while most of the folks already get into wagon of 64bit OS, here is short howto installing the skype on 64bit Fedora. The hack is to install 32bit version of the following packages on 64bit Fedora.

yum install glibc.i686 alsa-lib.i686 libXv.i686 libXScrnSaver.i686 qt.i686 qt-x11.i686

Download Skype Fedora 13+ package from Skype official site http://www.skype.com/intl/en-us/get-skype/on-your-computer/linux/

And then install it.

yum install --nogpgcheck skype-2.1.0.81-fc10.i586.rpm

Hoping this is helpful

Upgrade to Fedora 14 using Preupgrade

Few days back Fedora folks released Fedora 15, however I was still running Fedora 13 on my work laptop. So it was good time to upgrade before Fedora 13 become unsupported version.

Backup Backup Backup
Its recommended to backup your important stuff before going for online upgrade using preupgrade or yum to avoid any data loose in case of system goes to its knees :P

The Upgrade
Preupgrade is a nice tool for upgrading Fedora from one version to another, it can be run from GUI and also from CLI.

# preupgrade-cli "Fedora 14 (Laughlin)"

Depends on your Internet pipe it will takes quite some time to download all the required Fedora 14 packages, once packages successfully downloaded reboot is required to boot the system into Fedora 14 and complete the installation process of the packages.

Go get cup of tea while the installation process running. Once installation finished a final reboot and you will find your box booted into shiny Fedora 14.

Its was a clean upgrade, everything working fine after upgrade. Thumbs up for Fedora folks.

For detail please refer to Preupgrade wiki.

Note: I prefer not to go for Fedora 15 at this time due to the fuse around the Gnome 3, but will wait until the community find some solid solutions. I do not want to loose my desktop icons and application launchers ;)

Inserting colon after every 2nd digit using sed

Sometime while working with the MAC address from BRAS I need to convert the MAC address into colon delimited. Here is how to insert colon after every second digit

Contents of mac.txt

00e04dc540fe
00e04dc541f6
00e04dc54366
00e04dc5452e
00e04dc543f6
00e04dc5473a
00e04dc54852
00e04dc54a9e
c8d5fe1fd02c
c8d5fe1fa69c

The command:
$ sed 's/../&:/g; s/:$//' < mac.txt

Result:
00:e0:4d:c5:40:fe
00:e0:4d:c5:41:f6
00:e0:4d:c5:43:66
00:e0:4d:c5:45:2e
00:e0:4d:c5:43:f6
00:e0:4d:c5:47:3a
00:e0:4d:c5:48:52
00:e0:4d:c5:4a:9e
c8:d5:fe:1f:d0:2c
c8:d5:fe:1f:a6:9c

Upgrade to CentOS 5.6 and KVM failure

Recently I have upgraded my KVM server from Centos 5.5 to 5.6  I can no longer boot any of virtual machines. While trying to boot VM its gives error...  

Booting from Hard Disk...
Boot failed: not a bootable disk
FATAL: No bootable device

My all VMs are qcow2 format, quick Googling reports its issue with the 'type=raw' in .xml file for the VM.

<driver name='qemu' type='raw'/>

Fixed:
Change the type='qcow2' in xml file, so that its read something like this

<driver name='qemu' type='qcow2'/>

And restart the libvirtd service.

This fixed all the KVM VMs for me.

Linux Tips and Tricks - Display the IPs on interfaces

Using ifconfig

FIBER_ADDR=$(ifconfig $FIBER | grep 'inet addr' | sed 's/^[^:]\+://;s/\([^ \t]\+\).\+$/\1/')
echo $FIBER_ADDR

Same using IP command

ip addr show | awk '/inet[^6]/ { print $2 }' | sed -e 's/\(.\+\)\/.\+/\1/'

Find the disk space used by a user

To find the disk space being used by a single user, you need to combine du with the find command to only report disk usage for a specific user:

$ find . -user username -type f -exec du -k {} \;

To get total disk used

$ find . -user username -type f -exec du -k {} \;|awk '{ s = s+$1 } END { print "Total used: ",s }'

You use the same principle with groups using the -group option to find

find . -group mcslp -type f -exec du -k {} \;|awk '{ s = s+$1 } END { print "Total used: ",s }'

Creating thumbnails on a fly using ImageMagick

To create thumbnails for all the JPG files in the current directory

for img in *.jpg; do convert -resize 200 "$img" thumb_"$img";done

Linux Tips & Tricks

Removing commented lines '#' from a file
grep -v '^#' squid.conf > squid.conf.new

to remove empty lines from a text file. I've been using sed with
sed '/^ *$/d' squid.conf.new > squid.conf.nospace

Do the same thing with sed in one line
sed '/^#/d' squid.conf.default | sed /^$/d > squid.conf.new

Or
sed -e '/^\(#\|$\)/ d' squid.conf.default > squid.conf.new

Remove comments and blank lines from a file
egrep -v '^(#|$)' squid.conf.default > squid.conf.new

Or
grep '^[^#]' squid.conf > squid.conf.new

2011 World Cup cricket ended in disappointment for Pakistan

Long story short.......In electrifying semi final at MOHALI India beat Pakistan by 29 runs on Wednesday to set-up a World Cup final against Sri Lanka in Mumbai on Saturday.

The 100 Oldest Currently Registered .COM Domains

Here is the list of 100 oldest .com domains if anyone wondering to buy any of the these domain eg sun.com (the 12th oldest domain) which Oracle will soon spare after $7 billion purchase of Sun Microsystems. then remember you need a bottom less pocket :).
Domain brokers speculate Oracle could sell it for $1 million or more, if it chose to do so.

My Baby Girl Fatime

If anyone wondering about my baby girl Fatime here she :). Fatima is now almost two months old.

Its more fun with Ayesha and Fatima I would say its the greatest time of our lives.

Installing oracle-instantclient on Centos 5.5

1. Download oracle instantclient rpm from Oracle for your platform

Need to download these two rpms.

oracle-instantclient-basic-*
oracle-instantclient-devel-*

2. Install
# rpm -ivh oracle-instantclient-*

Getting and compiling oci8 extension for PHP

# pear download pecl/oci8
# tar zxvf oci8-1.4.5.tgz
# cd oci8-1.4.5
# ./configure --with-oci8=instantclient
# make
# make install

Create a file called /etc/php.d/oracle_oci.ini and place the following line
extension=oci8.so

Restart Apache

Create a .php file with call to phpinfo(); and confirm that oci8 is loaded in apache, you can also check if module is loaded with php -m.

HP EliteBook 8540p Hibernate/Suspend Fixed in Fedora 13

Today got time to play with the long standing problem of hibernate/suspend problem on my work notebook HP EliteBook 8540p running Fedora 13 (upgrading to Fedora 14 still pending :( )

Quick Google search reports an article on The Linux Laptop Wiki and astonishingly within 3 minutes the problem fixed and now the thing goes on suspend/hibernate (as per power management settings) if the  laptop lid is closed while on battery power or when when the battery power is critically low.

As per the article all have to do is to create a script,

1. Create a script file.
$ sudo vi /etc/pm/sleep.d/01fix_usb3

#!/bin/sh
# Put the laptop to suspend/hibernate
MOD=xhci-hcd

case $1 in
suspend | hibernate) modprobe -r $MOD ;;
resume | thaw) modprobe $MOD ;;
*) echo "USB3 fix script: wrong argument $1" ;;
esac

2. Make it executable
sudo chmod +x /etc/pm/sleep.d/01fix_usb3

Thanks to nice folks at The Linux Laptop Wiki.

Update: This hack is no longer required in Fedora 14,  Hibernate/Suspend working out of the box in Fedora 14

Bulk data uploading into the Oracle database using SQL*Loader

SQL*Loader is a bulk loader utility used for moving data from external files into the Oracle database.

 SQL*Loader (sqlldr) is the utility to use for high performance data loads. The data can be loaded from any text file and inserted into the database.

Recently I got the chance to use SQL*Loader to automate data uploading from a text file in which each field delimited with the pipe "|" into the Oracle.

1. Create a control file, call it anything in my case its add.ctl

load data  infile '-'
APPEND
into table records
fields terminated by '|'
(
field1,
field2,
.
.
fieldN
)

load data infile '-' : In my case there are multiple text files which to be process by sqlldr so instead of using filename I used the '-'. And used APPEND to append the new data into the table.

2. Script to run to call sqlldr utility for uploading the data from text files.

#!/bin/sh
# Author: Askar Ali Khan
# Purpose: To upload data from text files into Oracle Table - RECORDS
# records_upload

# Define Oracle variables
ORACLE_BASE=/oracle
ORACLE_HOME=$ORACLE_BASE/102
ORACLE_SID=SID
LD_LIBRARY_PATH=$ORACLE_HOME/lib
PATH=$PATH:$ORACLE_HOME/bin
export ORACLE_BASE ORACLE_HOME ORACLE_SID LD_LIBRARY_PATH PATH

SRCDIR="/data"

# Must be run as oracle user
if [[ $UID -ne 501 ]]; then
    echo "$0 must be run as oracle"
    exit 1
fi

cd "$SRCDIR"

# If directory contains *.txt files then call sqlldr otherwise do nothing

for file in *.txt; do
 test -f "$file" || continue
 cat $file | sqlldr oraclUser/Password add.ctl
 rm -f $file
done

The script process files in /data directory one by one by piping into sqlldr and delete it. Deletion is must coz in the next run there will be new files fetched into /data directory by another FTP script.

3. Define a cron job under oracle user to run script every 15 minutes

$ crontab -e
*/15 * * * * /usr/local/bin/records_upload > /dev/null 2>&1

Backup using Rsync

Rsync is a great tool for backup your servers, home directories or anything else, I thought of go for a little howto but there is already plenty of material available online therefore no need to reinvent the wheel :)

Anyone interested in quick howto for Rsync I would suggest to look into very nice article on LinuxPlanet.

Screencasting in Linux

To create screencast in Linux there lots of Open Source applications but I found RecordMyDesktop the easiest one to use.

Installing of recordmydeskop is straight forward under most of Linux distributions and there is a chance that its available in your Linux distro repository.

Installing on Fedora 13

sudo yum install gtk-recordmydesktop

Or prefer to install QT based which is the fork of gtk based then,

sudo yum install qt-recordmydesktop

Using recordmydesktop is very easy and its do not require any explanation, but still interested in learning more about then refer to recordmydesktop documentions.

Installing Oracle 11gR2 on Centos 5.5 64bit

1. The /etc/redhat-release should be faked to show Red Hat Enterprise Linux Server release 5 (Tikanga). same goes for /etc/issues file.
But to be sure apply this settings (backup /etc/redhat-release and /etc/issue before) change existing value to
Red Hat Enterprise Linux Server release 5 (Tikanga)

2. Install fake redhat-release RPM file from http://ivan.kartik.sk/

rpm -i centos-fake.noarch.rpm

3. Create Required UNIX Groups and User (For detail check http://download.oracle.com/docs/html/B15521_01/toc.htm)

# /usr/sbin/groupadd oinstall
# /usr/sbin/groupadd dba

# /usr/sbin/useradd -g oinstall -G dba oracle

Enter the following command to set the password of the oracle user:

# passwd oracle

4. Create Required Directories
Create directories with names similar to the following and specify the correct owner, group, and permissions for them:

    * /u01/app/oracle (the Oracle base directory)
    * /u02/oradata (an optional Oracle datafile directory)

# mkdir -p /u01/app/oracle
# mkdir -p /u02/oradata

#

Change the permissions on the directories that you created to 775:

# chmod -R 775 /u01/app/oracle
# chmod -R 775 /u02/oradata


Depending on the oracle user's default shell, make the following changes to the default shell start-up file:

    * For the Bourne, Bash, or Korn shell, add the following lines to the /etc/profile file (or the /etc/profile.local file on SUSE systems):

      if [ $USER = "oracle" ]; then
              if [ $SHELL = "/bin/ksh" ]; then
                    ulimit -p 16384
                    ulimit -n 65536
              else
                    ulimit -u 16384 -n 65536
              fi
      fi

Fulfilling package requirements:

       kernel-headers-2.6.18-194.el5.x86_64.rpm \
           glibc-headers-2.5-49.x86_64.rpm \
           glibc-devel-2.5-49.x86_64.rpm \
           libstdc++-devel-4.1.2-48.el5.x86_64.rpm \
           compat-db-4.2.52-5.1.x86_64.rpm \
           libgomp-4.4.0-6.el5.x86_64.rpm \
           gcc-4.1.2-48.el5.x86_64.rpm \
           gcc-c++-4.1.2-48.el5.x86_64.rpm \
           sysstat-7.0.2-3.el5.x86_64.rpm \
           unixODBC-2.2.11-7.1.x86_64.rpm \
           unixODBC-devel-2.2.11-7.1.x86_64.rpm \
           libXp-1.0.0-8.1.el5.i386.rpm \
           libstdc++-devel-4.1.2-48.el5.i386.rpm \
           unixODBC-2.2.11-7.1.i386.rpm \
           unixODBC-devel-2.2.11-7.1.i386.rpm \
           libaio-devel-0.3.106-5.x86_64.rpm \
           elfutils-libelf-devel-static-0.137-3.el5.x86_64.rpm \
           elfutils-libelf-devel-0.137-3.el5.x86_64.rpm \
           compat-gcc-34-3.4.6-4.x86_64.rpm \
           compat-gcc-34-c++-3.4.6-4.x86_64.rpm \
           gdb-7.0.1-23.el5.x86_64.rpm \
           libtermcap-devel-2.0.8-46.1.x86_64.rpm \
           readline-devel-5.1-3.el5.x86_64.rpm \
           glibc-devel-2.5-49.i386.rpm \
           libaio-devel-0.3.106-5.i386.rpm


Install Oracle Database 11gR2

1. Download the linux.x64_11gR2_database_1of2.zip and linux.x64_11gR2_database_2of2.zip from www.oracle.com
2 unzip both archive this will give one database directory
3. cd database directory and run ./runInstaller (the installer must be run as oracle user)

Installing KVM guest using virt-install tool

These has been tested on Centos 5.5 64bit host.

1. KVM Guest installation with virt-install using default networking and storage image created automatically.

virt-install \
     --name ubuntu_server \
     --ram 256 \
     --disk path=/var/lib/libvirt/images/ubuntu_server.img,size=5 \
     --network network:default \
     --accelerate \
     --vnc \
     --cdrom FullPathTo.iso

2. KVM Guest installation with virt-install using bridging interface, and storage image file created with qemu-img

For this its assume that bridge 'br0' interface is already configured on host, refer to how to create bridge interface on Redhat/Centos.

a) To create storage image
# qemu-img create -f qcow2 /var/lib/libvirt/images/ubuntu_server.img 10G

b)
virt-install \
     --name ubuntu_server \
     --ram 256 \
     --os-type='linux' \
     --disk path=/var/lib/libvirt/images/ubuntu_server.img \
     --network bridge:br0 \
     --accelerate \
     --vnc \
     --cdrom FullPathTo.iso

3. Install a KVM in text mode from remote http server, storage image created with qemu-img. Check (a) for creating storage image

virt-install \
              --name centos2 \
              --ram 256 \
              --os-type='linux' \
              --disk path=/var/lib/libvirt/images/centos2.img \
              --network network:default \
              --nographics \
              --location http://mirror.switch.ch/ftp/mirror/centos/5.5/os/x86_64/ \
              --hvm \
              --extra-args console=ttyS0

Note: --extra-args will only work when performing a guest install from "--location", when doing install from local ISO image it should be mounted somewhere

Tips for completing the installing of guest operating system

1) To Connect from remote PC/laptop

1. install virt-viewer on your local pc

2. Upload your ssh public key to KVM host /root/.ssh/authorized_keys (For this you should allow root login in /etc/ssh/sshd_config)

3. Run the following by replace KVMHostIP with KVM host IP address and vmName with the KVM virtual machine name.

$ virt-viewer -c qemu+ssh://root@KVMHostIP/system vmName

To connect KVMs from the KVM Host

To connect using the virt-viewer tool, run the following command:
# virt-viewer vmName

To connect using the virt-manager tool, run the following command:
# virt-manager vmName

Auto start KVM VMs on Host system boot

The following command will mark the VM for auto start on system reboot

# virsh autostart vmName

Creating KVM Guests using virsh tool

An example XML file to install an operating system from an ISO image The following example XML file can be modified to create a KVM and boot to any operating system from an ISO image or a CD-ROM drive.

<domain type='kvm'>
   <name>kvm1</name>
   <uuid>a1e40383-2dfb-47f7-9cf6-589fc9679aa7</uuid>
   <memory>524288</memory>
   <currentMemory>524288</currentMemory>
   <vcpu>1</vcpu>
   <os>
     <type arch='x86_64' machine='pc'>hvm</type>
     <boot dev='cdrom'/>
 </os>
 <features>
    <acpi/>
    <apic/>
     <pae/>
 </features>
 <clock offset='localtime'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
     <emulator>/usr/libexec/qemu-kvm</emulator>
     <disk type='file' device='disk'>
       <source file='/var/lib/libvirt/images/kvm1.img'/>
       <target dev='hda' bus='ide'/>
     </disk>
    <disk type='file' device='cdrom'>
     <source file='/home/askarali/.bb/Fedora-14-x86_64-Live/Fedora-14-x86_64-Live-Desktop.iso'/>
     <target dev='hdc' bus='ide'/>
     <readonly/>
     </disk>
     <interface type='network'>
      <mac address='54:52:00:2a:58:0d'/>
      <source network='default'/>
    </interface>
     <input type='mouse' bus='ps2'/>
     <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
 </devices>
 </domain>

Edit the following before proceeding
a) Each guest needs a universal unique identifier (uuid), You must generate one for your guest by running the following command, and then copy and paste the identifier into the XML file <uuid>HERE</uuid>

# uuidgen

b) Replace <memory>524288</memory> and <currentMemory>524288</currentMemory> tag to memory you want to allocate to your guest.

c) Replace <mac address='54:52:00:2a:58:0d'/> with the mac address, To generate unique mac address, get the python script from http://www.centos.org/docs/5/html/5.2/Virtualization/sect-Virtualization-Tips_and_tricks-Generating_a_new_unique_MAC_address.html

Creating Storage Image file
# qemu-img create -f qcow2 /var/lib/libvirt/images/kvm1.img 10G

Using virsh to create a KVM
1. Define your KVM by running:
# virsh define kvm1.xml

2. Start the KVM so that the installation of its operating system can begin:
# virsh start kvm1

If your KVM does not start, complete one of the following to redefine the KVM:

# virsh edit <Name of KVM>

After editing the definition file, try to start guest vm again

# virsh start vmName

Tips for installing your guest operating system (This should be run on mother host

To Connect from your PC/laptop

1. install virt-viewer
2. Upload your ssh public key to KVM host /root/.ssh/authorized_keys (For this you should allow root login in /etc/ssh/sshd_config)

From your laptop, run

$ virt-viewer -c qemu+ssh://root@KVMHostIP/system vmName

To connect KVMs from the Host

To connect using the virt-viewer tool, run the following command:
# virt-viewer vmName

To connect using the virt-manager tool, run the following command:
# virt-manager vmName

Editing the KVM definition file after operating system installation

To boot to the installed guest after the guest operating system installation is complete, revise the guest
definition so that it will boot from its hard disk drive.

# virsh edit vmName

1.  Replace <boot dev='cdrom'/> line with <boot dev='hd'/> (To boot from hard disk)
2.  and remove the ISO disk definitions are removed because they are no longer needed.

Propagating your KVMs
For example, if your task is to create 10 identical KVMs that have simple configurations, you can propagate existing KVMs rather than manually installing 10 identical KVMs.

1. Stop the template KVM with the virsh tool:

# virsh shutdown templateKVM

2. Make copies of the KVM disk image by using the qemu-image command with the convert option:

# qemu-img convert templateKVM.img -O qcow2 NewKVM1.img

3. Make copies of the templateKVM XML definition file. You can find it in the /etc/libvirt/qemu/directory or you can run the following command to see the definition file.

# virsh dumpxml templateKVM

4. Edit the definition files for name, memory, mac address, source file etc to define the new KVMs. It is important that a unique MAC address is used in the definition file to make sure that the network functions without a problem.

Networking in Bridge Mode

1. Configure the bridge interface on host operating system
2. replace <interface type='network'> section in kvm.xml with the following

<interface type='bridge'>
      <mac address='00:16:3e:56:40:ad'/>
      <source bridge='br0'/>
</interface>

3. Install the guest OS, after installation configure the IP on guest.

Auto start KVM VMs on Host system boot

The following command will mark the VM for auto start on system reboot

# virsh autostart vmName

VirtualBox 4.0: Creating a Virtual Machine on a headless server

The following instructions may give you an idea how to create a virtual machine on a headless server over a network connection. We will create a virtual machine, establish an RDP connection and install a guest operating system -- all without having to touch the headless server. All you need is the following:

1. VirtualBox on a server machine with a supported host operating system. The VirtualBox extension pack for the VRDP server must be installed For the following example, we will assume a Linux server.

2. An ISO file accessible from the server, containing the installation data for the guest operating system to install (we will assume Windows XP in the following example).

3. A terminal connection to that host through which you can access a command line (e.g. via ssh).

4. An RDP viewer on the remote client rdesktop, rdesktop-vrdp, Microsoft mstsc

Note again that on the server machine, since we will only use the headless server, neither Qt nor SDL nor the X Window system will be needed.

1. On the headless server, create a new virtual machine:

VBoxManage createvm --name "Windows XP" --ostype WindowsXP --register

Note that if you do not specify --register, you will have to manually use the registervm command later.

Note further that you do not need to specify --ostype, but doing so selects some sane default values for certain VM parameters, for example the RAM size and the type of the virtual network device. To get a complete list of supported operating systems you can use

VBoxManage list ostypes

2. Make sure the settings for this VM are appropriate for the guest operating system that we will install. For example:

VBoxManage modifyvm "Windows XP" --memory 256 --acpi on --boot1 dvd --nic1 nat

3. Create a virtual hard disk for the VM (in this case, 10GB in size):

VBoxManage createhd --filename "WinXP.vdi" --size 10000

4. Add an IDE Controller to the new VM:

VBoxManage storagectl "Windows XP" --name "IDE Controller" --add ide --controller PIIX4

5. Set the VDI file created above as the first virtual hard disk of the new VM:

VBoxManage storageattach "Windows XP" --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium "WinXP.vdi"

6. Attach the ISO file that contains the operating system installation that you want to install later to the virtual machine, so the machine can boot from it:

VBoxManage storageattach "Windows XP" --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium /full/path/to/iso.iso

7. Start the virtual machine using VBoxHeadless:

VBoxHeadless --startvm "Windows XP"

If everything worked, you should see a copyright notice. If, instead, you are returned to the command line, then something went wrong.
  
8. On the client machine, fire up the RDP viewer and try to connect to the server (see the section called “Common third-party RDP viewers” above for how to use various common RDP viewers).

You should now be seeing the installation routine of your guest operating system remotely in the RDP viewer.

To stop a VM
VBoxManage controlvm "Windows XP" poweroff

 To pause a VM
 VBoxManage controlvm "Windows XP" pause

Allow multi connections:
VBoxManage modifyvm "Windows XP" --vrdemulticon on

Listen on different port
VBoxHeadless --startvm "Windows XP" --vrdeproperty "TCP/Ports"=5001


Installing Guest Addition on Headless VM

1. First attach the Guest Addition .iso to VM (vm should be poweroff)

VBoxManage storageattach "Windows XP" --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium /usr/share/virtualbox/VBoxGuestAdditions.iso

2. Start VM and now you can explorer and install the guest addition.

VirtualBox VBoxHeadless, the remote desktop server

Which produces no visible output on the host at all, but instead only delivers VRDP data. This front-end has no dependencies on the X Window system on Linux and Solaris hosts, this is usefull on servers where the only purpose is to run VMs and access it remotely.

Prerequisite:
On Virtualbox 4.0 you must have extension pack installed.

To start a virtual machine with VBoxHeadless, you have two options:

VBoxManage startvm "VM name" --type headless

The extra --type option causes VirtualBox to use VBoxHeadless as the front-end to the internal virtualization engine instead of the Qt front-end.

The alternative is to use VBoxHeadless directly, as follows:

VBoxHeadless --startvm

This way of starting the VM is preferred because you can see more detailed error messages, especially for early failures before the VM execution is started. If you have trouble with VBoxManage startvm, it can help greatly to start VBoxHeadless directly to diagnose the problem cause.

Note that when you use VBoxHeadless to start a VM, since the headless server has no other means of output, the VRDP (VirtualBox Remote Desktop Protocol) server will always be enabled, regardless of whether you had enabled the VRDP server in the VM's settings

Accessing the VM remotely:

On Linux host you can test the Remote display of VM using rdesktop, or rdesktop-vrdp which is installed with VirtualBox

rdesktop -a 16 -N 1.2.3.4:3339

As said for the Microsoft viewer above, replace "1.2.3.4" with the host IP address, and 3389 with a different port if necessary. The -a 16 option requests a color depth of 16 bits per pixel, which we recommend. (For best performance, after installation of the guest operating system, you should set its display color depth to the same value). The -N option enables use of the NumPad keys.

rdesktop-vrdp 127.0.0.2:3339

When connecting to localhost in order to test the connection, the addresses localhost and 127.0.0.1 might not work. Instead, the address 127.0.0.2[:3389] has to be used.

On Windows, you can use the Microsoft Terminal Services Connector (mstsc.exe) that ships with Windows. You can start it by bringing up the "Run" dialog (press the Windows key and "R") and typing "mstsc". You can also find it under "Start" -> "All Programs" -> "Accessories" -> "Remote Desktop Connection". If you use the "Run" dialog, you can type in options directly:

mstsc 1.2.3.4[:3389]

VirtualBox Remote display (VRDP support)

VirtualBox can display virtual machines remotely, meaning that a virtual machine can execute on one machine even though the machine will be displayed on a second computer, and the machine will be controlled from there as well, as if the virtual machine was running on that second computer.

Prerequisites:
VirtualBox Extentions Pack

You can also check my eariier post for how to install VirtualBox and extention pack.

Enabling VRDP

Even when the extension is installed, the VRDP server is disabled by default. It can easily be enabled on a per-VM basis either in the VirtualBox Manager in the "Display" settings or with VBoxManage:

 VBoxManage modifyvm "VM name" --vrde on

By default, VRDP uses TCP port 3389. You will need to change the default port if you run more than one VRDP server, since the port can only be used by one server at a time; you might also need to change it on Windows hosts since the default port might already be used by the RDP server that is built into Windows itself. Ports 5000 through 5050 are typically not used and might be a good choice.

Accessing the VM remotely:

On Linux host you can test the Remote display of VM using rdesktop, or rdesktop-vrdp which is installed with VirtualBox

rdesktop -a 16 -N 1.2.3.4:3339

As said for the Microsoft viewer above, replace "1.2.3.4" with the host IP address, and 3389 with a different port if necessary. The -a 16 option requests a color depth of 16 bits per pixel, which we recommend. (For best performance, after installation of the guest operating system, you should set its display color depth to the same value). The -N option enables use of the NumPad keys.

rdesktop-vrdp 127.0.0.2:3339

When connecting to localhost in order to test the connection, the addresses localhost and 127.0.0.1 might not work using mstsc.exe. Instead, the address 127.0.0.2[:3389] has to be used.


On Windows, you can use the Microsoft Terminal Services Connector (mstsc.exe) that ships with Windows. You can start it by bringing up the "Run" dialog (press the Windows key and "R") and typing "mstsc". You can also find it under "Start" -> "All Programs" -> "Accessories" -> "Remote Desktop Connection". If you use the "Run" dialog, you can type in options directly:

mstsc 1.2.3.4[:3389]

Installing VirtualBox 4.0 on Fedora 13

Oracle recently released VirtualBox 4.0, a new major release. It now has an open architecture using extension packs and the base package is under GPLv2 and lots of new cool features. So its worth to install the VB 4.0

This article assume that you have already older version of VirtualBox installed along with all the dependencies.

1.       First install the DKMS package

$ sudo yum install dkms

2.       Download the VB 4.0 for Fedora 13 64bit

$ wget http://download.virtualbox.org/virtualbox/4.0.0/VirtualBox-4.0-4.0.0_69151_fedora13-1.x86_64.rpm

3.       Uninstall the older version of VB (make sure no virtual machine is running)

$ rpm -qa | grep VirtualBox <- this will give you the package name of installed VB

$ sudo rpm –e PackageName (replace package name with the name got from earlier command)

4.       Install VirtualBox 4.0

sudo rpm –Uvh VirtualBox-4.0-4.0.0_69151_fedora13-1.x86_64.rpm

Download and install the extension pack as follows:

$ wget http://download.virtualbox.org/virtualbox/4.0.0/Oracle_VM_VirtualBox_Extension_Pack-4.0.0-69151.vbox-extpack

$ sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.0.0-69151.vbox-extpack