View conf file without comments

6 01 2010

Short and sweet entry to start 2010.  Some linux configuration files are pretty long and really include very few directives, a prime example is squid.conf (mine is currently an almighty 4898 lines!).  Most of these lines are comments and if we were so inclined we could probably copy the file without comments and sort the wheat from the chaff (of the 4898 lines, my squid.conf actually only uses 68 lines!)  I use the following to display or pipe the file without any comments:

grep ^[^#] /usr/local/squid/etc/squid.conf


grep ^[^#] /usr/local/squid/etc/squid.conf > tmp

This shows only the directives.

Alternatively we could redo the grep only this time show 2 of lines of comments before the directive using:

grep -B 2 ^[^#] /usr/local/squid/etc/squid.conf

Using USB Storage in Kickstart

21 12 2009

Recently I’ve been going through several kickstart build scripts – mainly to setup the correct packages and settings.  I thought it would be convenient if I was able to use the same kickstart DVD for all machines but read in the appropriate settings from a USB stick.  To do this, I put the following in my kickstart postscript…

# Check for USB storage and profile definition
if [ -d /proc/scsi/usb-storage ]; then
DEVICENUM=$(ls /proc/scsi/usb-storage/)
DEVICELETTER=$(<<<$DEVICENUM tr ‘[1-9]’ ‘[a-i]’)
echoOut “USB storage found at sd”$DEVICELETTER
mkdir -p /mnt/usb
mount /dev/sd${DEVICELETTER}1 /mnt/usb
if [ -f /mnt/usb/network-settings-*.bash ]; then
source /mnt/usb/network-settings-*.bash
cp /mnt/usb/network-settings-*.bash /root/
echoOut “Loaded profile found on USB storage”
echoOut “Unable to find profile settings on USB storage”
echoOut “No USB storage present…”

The echoOut function is just an echo function I wrote to do an ordinary echo, log to a file and tee it to tty3 (for use the kickstart installation).  I could go further with this script and perhaps use a menu in the case of several available files, but a single file on the USB is fine for me.

Custom Installation CentOS/RHEL DVD

30 11 2009

I recently had to create a custom CentOS 5.4 DVD that included a set list of installed packages, some files copied from the DVD and a comprehensive kickstart script.  After trawling the web for a couple of days I did manage to successfully get my image operational – there are quite a few quirks and the required information is on the web but it’s not exactly in one place… So here’s my effort:

– Copy the CentOS DVD to a local image folder (you can use the actual physical CentOS DVD or just the ISO file).  At this stage we have to ensure that all files from the disk are copied including .discinfo and .treeinfo.  If we don’t we’ll get the ‘CentOS cd was not found error…:

#mkdir -p /mnt/centos (we’ll only use this mount the iso/dvd)

#mount -o loop /home/mark/CentOS-5.4-i386-bin-DVD.iso /mnt/centos (if using the iso file)

#mount /dev/cdrom /mnt/centos (if using the physical DVD)

#cp -av /mnt/centos /home/mark/  (the options -av ensure all files and links are copied)

#mv /home/mark/centos /home/mark/customdvd

– Remove the repodata information from our customdvd image folder, we’re going to regenerate this data later.  (NB: It’s important not to remove repodata/comp.xml as this is a full package list we can edit):

#cd /home/mark/customdvd/

#rm ./repodata/*.gz ./repomd.xml

– Modify or add packages to our customdvd image folder.  In this example I’ll add the lshw rpm package to keep it straightforward.  I downloaded this package from  The DVD has a CentOS folder which contains all the rpm packages, we’ll keep ours in there too. (NB: you’ll need to add any dependencies into this folder too, if they’re not already there):

#cp /home/mark/lshw-2.14-1.el5.rf.i386.rpm /home/mark/customdvd/CentOS/

– At this stage we need to add our lshw package to the comps.xml file.  The comps.xml file is a xml database file that lists all the packages in the familiar installation groups along with whether or not they’re installed as part of that group by default or not.  The repodata we create later reads the comps.xml file and outputs the required file information for our installation.  With only one package it’s probably easiest to just add the package name to the ‘core’ package group, but just in case you’ve a bunch more packages to install I’m going to create a new group called ‘custom’ and add lshw to the new group.  For comps.xml it’s important you add the rpm package name, this is not necessarily the same name as the file/package, so query the file first to find the name with the command:

#rpm -q –qf ‘%{NAME}\n’ -p /home/mark/lshw-2.14-1.el5.rf.i386.rpm

In our case the package name actually is ‘lshw’ so we edit the file comps.xml and enter something like the following (hopefully this is self-explanatory):

<name xml:lang=”uk”>custom</name>
<description>Custom Tools</description>
<description xml:lang=”uk”>Custom Tools</description>
<packagereq type=”mandatory”>lshw</packagereq>

– Now we need to recreate the repodata for our installation.  The repodata is the link between comps.xml and the rpm packages in the DVD’s CentOS folder.  Using the createrepo utility we can iterate through all the rpm’s and create the appropriate files to help with our installation (we deleted they earlier).  The files that createrepo generates need to be relative to the DVD’s path so we have to declare the path before doing the regeneration:

#cd /home/mark/customdvd/

#declare -x discinfo=`head -1 .discinfo`

#createrepo -u “media://$discinfo” -g repodata/comps.xml .

Hopefully at this stage you’ll see the rpms being iterated over and you’ll end up with additional files in customdvd/repodata.  If you don’t have the createrepo utility installed use ‘yum install createrepo’ to get it.

At this stage our custom installation DVD could be made into an iso and burnt, however if you want to add more files to the DVD (not rpm packages) copy them into the customdvd image folder. Before we do that however we may want to add a kickstart file to the DVD.

– (Optional) Copy the file kickstart file to the DVD:

#cp /home/mark/ks.cfg /home/mark/customdvd/

At this point we could create the DVD iso and just boot and type ‘linux ks=cdrom:/ks.cfg’ to run through our kickstart installation, however to have it automatically run our kickstart installation, edit the file isolinux/isolinux.cfg file so that our kickstart runs automatically, for example:

label linux
kernel vmlinuz
append initrd=initrd.img ks=cdrom:/ks.cfg

To use files from the DVD during kickstart installation, for example predefined config files, you’ll need to copy the files using a %post –nochroot script.  Don’t panic though, after copying the files you can simply using a standard %post for example:

%post –nochroot
mkdir /mnt/sysimage/tempdir
mkdir -p /mnt/cdrom
mount -t iso9660 /tmp/cdrom /mnt/cdrom
cp /mnt/cdrom/CentOS/filex /mnt/sysimage/tempdir/

After this you can use filex from within your %post, for example:

cp /tempdir/filex /opt

Ok – so now we have our customdvd image the way we need it, we can create our iso using mkisofs (you probably won’t need all the options so just strip out the ones you don’t need):

mkisofs  -o /home/mark/customcentos.iso -r -J -N -d -hide-rr-moved \
-sysid `hostname`  -V ‘Custom CENTOS’  \
-no-emul-boot -boot-info-table -boot-load-size 4 \
-b isolinux/isolinux.bin -c isolinux/ \

After this we should have a shiny new custom CentOS/RHEL DVD.  Ordinarily I check the output ISO file using a virtual machine.  Best of luck.

Using sed to find and replace

20 10 2009

I’ve already looked at find and replace in vi, but this essentially means you have to open the file before editing.  An alternative way of going about this, is to use sed.  The basic command format is:

sed -i “s/wordA/wordB/” /etc/smb/smb.conf

The -i means do an in place change, in other words don’t change the name of this file.  The next phrase within the “” is the command to run in place on the file, it means substitute all instances of wordA with wordB.  The / markers in this command act as delimiters.  I normally tend to use this command over a set of files, for example:


fileArray=`find /home/user/logs/ -name *.log`
for i in $fileArray
sed -i “s/localhost/hostname/” $i

This will go through the /home/user/logs directory changing all instances of localhost to hostname in all files with the .log extension.  The IFS is the internal field separator, we’re setting this to a newline so we put each find result into our fileArray.

Expand an existing Linux partition

13 10 2009

I’ve to do this from time to time – but never regularly enough to remember all the commands and the order in which to go about it.  This is just one way of doing it (and by the way this does not cover logical volume groups, just plain vanilla EXT2/3/4 partitions).  In this example I’ll extend the root partition / which just happens to be at /dev/sda2 on my box.

Whilst in the OS check free space on partition.

#df -h /

Now check the partition table using fdisk (this should let you know which disk device / is on, as well as the partition number it mounted at eg; /dev/sda2 or /dev/hdb5)

#fdisk -l

To change the partition size the disk needs to be unmounted, I think the easiest way of doing this is just booting to a Linux live CD/USB.  I keep a small non-GUI one handy.

Check the disk hasn’t moved ID by running fdisk again.  At this stage it is necessary to note the starting cylinder of the partition you want to expand – this is in the start column.

#fdisk -l

Now use fdisk to delete the relevant partition (yes I did say delete, don’t panic if we start the new partition on the same cylinder and resize the partition all data will be intact provided our new partition is larger than the original).

#fdisk /dev/sda

>d (ENTER)

>2 (ENTER)  …delete the second partition, ie; /dev/sda2

>n (ENTER) …create a new partition

>p (ENTER) …create a new primary partition

>2 (ENTER)  …the number of the new partition

>(ENTER) …the start cylinder of the new partition, it should default to the same cylinder id recorded earlier

><last_cylinder>(ENTER)  …the last cylinder of the new partition, the default is the end of the disk but you can specify a size, for example +40G for a 40GB partition

>w(ENTER)  …write the changes to the disk and exit

Now we’ve deleted and re-created our partition (only this time larger) all we have to do is run a file check on it and tell our partition it’s been resized.

Check and if necessary fix the new partition.

#e2fsck -f /dev/sda2

Resize the partition.

#resize2fs /dev/sda2

Hopefully if all has gone well up to this point we can restart the original OS and check disk free using the command.

#df -h /

Replace all occurances of a word using vi

9 10 2009

Posted this as I have to do this all the time, so hopeful this passes as something useful for others…

Open the file using vi:

$vi <filename>

Then in command mode type (if in doubt, hit ESC):

:%s/<original-word>/<new-word>/g (ENTER)

To save and close:



2 10 2009

After 10 Years using the Internet for IT tips/problem-solving/training/etc, thought it was about time I rolled up my sleeves and gave something back.  Don’t promise I’ll always get it right, post anything useful or even post with any regularity, but hey I’d rather try…