Grep funny issue

grep: invalid option -- t


This happens when there is a file in the local directory that has a name that starts with a “-”, for exampe “-t ” 

remove the file with: rm -i ./-t  and then grep will work  

Frontend Apache with Tomcat?

Many people has  asked me the same question again and  again  so its better to document it.

Took this from apache wiki http://wiki.apache.org/tomcat/FAQ/Connectors#Q3

Why should I integrate Apache with Tomcat? (or not)

There are many reasons to integrate Tomcat with Apache. And there are reasons why it should not be done too. Needless to say, everyone will disagree with the opinions here. With the performance of Tomcat 5 and 6, performance reasons become harder to justify. So here are the issues to discuss in integrating vs not.

• Clustering. By using Apache as a front end you can let Apache act as a front door to your content to multiple Tomcat instances. If one of your Tomcats fails, Apache ignores it and your Sysadmin can sleep through the night. This point could be ignored if you use a hardware loadbalancer and Tomcat's clustering capabilities.
• Clustering/Security. You can also use Apache as a front door to different Tomcats for different URL namespaces (/app1/, /app2/, /app3/, or virtual hosts). The Tomcats can then be each in a protected area and from a security point of view, you only need to worry about the Apache server. Essentially, Apache becomes a smart proxy server.
• Security. This topic can sway one either way. Java has the security manager while Apache has a larger mindshare and more tricks with respect to security. I won't go into this in more detail, but let Google be your friend. Depending on your scenario, one might be better than the other. But also keep in mind, if you run Apache with Tomcat - you have two systems to defend, not one.
• Add-ons. Adding on CGI, perl, PHP is very natural to Apache. Its slower and more of a kludge for Tomcat. Apache also has hundreds of modules that can be plugged in at will. Tomcat can have this ability, but the code hasn't been written yet.
• Decorators! With Apache in front of Tomcat, you can perform any number of decorators that Tomcat doesn't support or doesn't have the immediate code support. For example, mod_headers, mod_rewrite, and mod_alias could be written for Tomcat, but why reinvent the wheel when Apache has done it so well?
• Speed. Apache is faster at serving static content than Tomcat. But unless you have a high traffic site, this point is useless. But in some scenarios, tomcat can be faster than Apache httpd. So benchmark YOUR site. Tomcat can perform at httpd speeds when using the proper connector (APR with sendFile enabled). Speed should not be considered a factor when choosing between Apache httpd and Tomcat
• Socket handling/system stability. Apache has better socket handling with respect to error conditions than Tomcat. The main reason is Tomcat must perform all its socket handling via the JVM which needs to be cross platform. The problem is socket optimization is a platform specific ordeal. Most of the time the java code is fine, but when you are also bombarded with dropped connections, invalid packets, invalid requests from invalid IP's, Apache does a better job at dropping these error conditions than JVM based program. (YMMV)

Network Tunning in Linux

# Kernel tuning settings for CentOS5

net.core.netdev_max_backlog=30000
net.ipv4.tcp_max_tw_buckets=2000000
# Bump up send/receive buffer sizes
net.core.rmem_default=262141
net.core.wmem_default=262141
net.core.rmem_max=262141
net.core.wmem_max=262141
net.ipv4.tcp_sack=0

net.ipv4.tcp_dsack=0
net.ipv4.tcp_retries2=5
net.ipv4.tcp_fin_timeout=60
net.ipv4.tcp_keepalive_time=120
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_keepalive_probes=3
net.core.somaxconn=256

net.core.optmem_max=20480
use netstat -s to determine the best usuage of your network

Hotplug CPU in Linux

Today playing with a VM machine i tried to hotplug a CPU in linux . Awesome feature handy when troubleshooting cpu load or process cpu scheduling .
First List the cpu Status
# grep "processor" /proc/cpuinfo

To disable cpu1 do the following
# echo 0 > /sys/devices/system/cpu/cpu1/online

To enable cpu1 do the following
# echo 1 > /sys/devices/system/cpu/cpu1/online

Best Way to Kill Java Program

ps -ef | grep java | grep -v grep | awk '{print $2}' | xargs kill -9

Performance Tunning Your Application Server

Instruction to performance tune OS for JBOSS/Tomcat/WebSphere/Oracle AS

First, you need to set the kernel parameter for shared memory to be at least as big as you need for the amount of memory you want to set aside for the JVM to use as large page memory. Personally, I like to just set it to the maximum amount of memory in the server, so I can play with different heap sizes without having to adjust this every time. You set this by putting the following entry into /etc/sysctl.conf:

Set the kernel Parameter for Shared memory in /etc/sysctl.conf to maximum amount of memory in the system so u don’t have to worry about the heap sizes adjustment .


kernel.shmmax = n for 4 GB server use 4294967296

where n is the number of bytes.

Set a virtual memory kernel parameter of how many large memory pages you want


vm.nr_hugepages = n

where n is the number of pages, based on the page size listed in /proc/meminfo for
Hugepagesize: 2048 kB

So, I wanted to set this to 3GB. I set the parameter to 1536, which is (1024*1024*1024*3)/(1024*1024*2). Which is 3GB divided by 2MB, since 2048 KB is 2MB.

Set another virtual memory parameter, to give permission for your process to access the shared memory segment. In /etc/group, created a new group, called hugetablespace let suppose guid is 1501

Put that group id in /etc/sysctl.conf as follows:
vm.hugetlb_shm_group = 1501

This GUID should be attached to the same user as that with which JBOSS is running .

/etc/security/limits.conf as follows:
jboss soft memlock n
jboss hard memlock n

where n is equal to the number of huge pages, set in vm.nr_hugepages, times the page size from /proc/meminfo, , 1536*2048 = 3145728. This concludes the OS setup, and now we can actually configure the JVM.

The JVM parameter for the Sun JVM is -XX:+UseLargePages use the same large pages /proc/meminfo

Clearing Deffer-ed Emails from mailqueues

# Clearing Deffer-ed Emails from mailqueues
#!/bin/bash
cd /var/spool/mqueue
for del in `mailq | grep MAILER-DAEMON | cut -d" " -f1`;
do rm df$del;
rm qf$del;
done

Linux Hugemem Kernels

Linux Hugemem Kernels

For using 4 GB or more RAM with Linux 32 bit in RHEl5,Linux kernel should be compiled with PAE capable machines.The kernel provide support for upto 64GB of high memory.

Install PAE kernel
kernel-PAE-2.6.18-8.el5.i686.rpm

Verify the initial RAM disk image
To verify that an initial RAM disk already exists,
use the command ls -l /boot to make sure the initrd-.img file was created

Verify the boot loader
#cat /boot/grub/grub.conf | grep PAE

Edit /etc/grub.conf to make sure to new kernel as default.

VMware Tools installation fails on 64-bit Linux

VMware tools may fail to start on 64 bit system if you have glibc-common 32 bit rpm installed mainly if you have Oracle Apps, Oracle Database , or any other ERP installed.

Follow these steps :

1. Untar the VMware tools.
2. Compile the VMware tools using install script.
3. Link the correct libraries

#ln –s /usr/lib/vmware-tools/lib64/libdnet.so.1/libdnet.so.1 /lib64/libdnet.so.1
#ln –s /usr/lib/vmware-tools/lib64/libproc-3.2.7.so/libproc-3.2.7.so /lib64/libproc-3.2.7.so

4. Run the vmware-config-tools.pl
5. You should be all set.

LVM - Regular Task

LVM - Regular Task

I was searching from my old docs and found a interesting article i saved for LVM.

I) How to create a Volume Group (VG).

Note The following example is using the disk c1t6d0, the volume group vg01 and the logical volume lvhome

1) Prepare the disk
pvcreate /dev/rdsk/c1t6d0

Note if the disk was previously used in another VG use the following command instead:

pvcreate -f /dev/rdsk/c1t6d0

2) Create the Volume Group (VG):
a) mknod /dev/vg01/group c 64 0x010000

Note: the group number (last parameter) is in hexadecimal and should be different for each volume group. For vg02, that number would be 0x020000. The default limit is 10 volume groups as set by the kernel parameter maxvgs.

b) vgcreate /dev/vg01 /dev/dsk/c1d0s2

Note: When a volume group is created the maximum physical
extents per volume (max_pe parameter) will be set to the max_pe
of the largest physical volume (PV) or 1016, which ever is
greater, if no max_pe is specified. The effect of not setting
the max_pe parameter would be that any PV added to the volume
group in the future regardless of there size will be limited
to the volume groug creation value of max_pe. Therefore,
consider increasing the max_pe to accommodate PV’s that may
likely be larger than the largest PV used to create the Volume
Group. The formula to use to determine the value is:

physical_extent_size * max_pe = size_of_the_disk.

The default value for physical_extent_size is 4M and the maximum
value for max_pe is 65535 (example for 18 gig disk use a value
4608 for max_pe: 4M * 4608 = 18 gig).
There is also a default value of a maximum of 16 disks per volume
group. The following is an example of the creation of a volume
group modifying these two parameters (max_pe = 4608, maximum
number of disk = 24):

vgcreate -e 4608 -p 24 /dev/vg01 /dev/dsk/c1d0s2

II) How to create a Logical Volume (LV) and mount the filesystem.
1) Create the Logical Volume (LV)
lvcreate -L 120 -n lvhome /dev/vg01
Note: this will create a logical volume of 120 meg.
2) Create the filesystem
newfs -F vxfs /dev/vg01/rlvhome
Note: to create an hfs filesystem change vxfs to
hfs in the previous command.
3) Mount the Logical Volume:
a) mkdir /home
b) mount /dev/vg01/lvhome /home

III) How to add a disk to a Volume Group
Note The following examples is using the disk c1t6d0 and the volume
group vg01

1) Prepare the disk
pvcreate /dev/rdsk/c1t6d0

Note if the disk was previously used in another VG use the
following command instead:

pvcreate -f /dev/rdsk/c1t6d0

Note: Use caution when using pvcreate -f as this will
overwrite the existing volume group information on the disk.

2) Add the disk to the Volume Group
vgextend /dev/vg01 /dev/dsk/c1t6d0

IV) How to increase the size of a logical volume without online JFS
(advanced JFS).
Note: the following example is using the volume group vg01 and the
logical volume lvhome

Note: Increasing the root filesystem (/) is not feasible

1) lvextend -L 240 /dev/vg01/lvhome
The new total size will be 240M.
2) umount /home
Note: If the filesystem is in use, it is impossible to unmount it.
Therefore stop all the processes (applications) that use the
filesystem then unmount it.
Processes that use /usr and /var cannot be all stopped,
the only solution is to reboot in single user mode.
3) extendfs -F vxfs /dev/vg01/rlvhome
4) mount /dev/vg01/lvhome /home

V) How to remove a Logical Volume
Note: the following example is using the volume group vg01 and the
logical volume lvhome
1) Backup all user data
2) Umount the filesystem
umount /home
3) remove the Logical volume
lvremove /dev/vg01/lvhome

VI) How to reduce the size of a logical volume without online JFS (advanced JFS)
Note: the following example is using the volume group vg01 and the
logical volume lvhome
1) Backup all user data
2) Umount the filesystem
umount /home
3) Reduce the size
lvreduce -L 60 /dev/vg01/lvhome
Note: the new total size will be 60M.
4) Re-create the filesystem
newfs -F vxfs /dev/vg01/rlvhome
Note: to create an hfs filesystem change vxfs by
hfs in the previous command.
5) Mount the Logical Volume:
mount /dev/vg01/lvhome /home
6) Restore the user data

VII) How to remove a disk from a volume group
Note: the following example is using the disk c1t6d0 and the
volume group vg01

1) Make sure that the disk is not in use:
pvdisplay /dev/dsk/c1t6d0
Look at line starting with Allocated PE the number at the end
of the line should be 0. If it is not the disk is still in use.
2) Remove the disk
vgreduce /dev/vg01 /dev/dsk/c1t6d0

VIII) How to remove a volume group
Note: the following example is using the volume group vg01 and the
logical volume lvhome

1) Backup all user data
2) Find the name of all logical volume in this volume group
vgdisplay -v /dev/vg01
3) unmount all logical volumes
Note: repeat the following command for each logical command
umount /dev/vg01/lvhome
4) Remove the volume group:
vgexport /dev/vg01
Note: using vgexport to remove a volume group is easier
and faster than using the vgreduce on each physical volume
except the last one, followed by a vgremove. The other
advantage is that the /dev/vg01 directory is also removed.

IX) How to increase the primary swap

Note: Because of the contiguous allocation policy, create a bigger
logical volume and modify the Boot Data Reserved Area (BDRA) to make it
primary.

1) lvcreate -C y -L 240 /dev/vg00
The name of this new logical volume will be displayed on the
screen, note it, it will be needed later. (let say it
is /dev/vg00/lvol8)
Note: This new logical volume has to be in vg00
2) lvlnboot -v /dev/vg00
This will display the current root and swap volumes
Note: lvol2 is the default primary swap.
3) lvlnboot -s /dev/vg00/lvol8 /dev/vg00
Note: use the logical volume name from step 1
4) lvlnboot -R /dev/vg00
Recover any missing links to all of the logical volumes specified
in the BDRA and update the BDRA of each bootable physical volume in
the volume group
5) reboot the system

X) How to create a secondary boot disk
Note: This will create an identical copy of the current vg00. The
new volume group needs to as big as vg00. This will also be a static
version of the primary boot disk which could be use in case of
problem.

Note: The following example is using the disk c1t6d0 and the
volume group vg01

1) Initialize the disk and make it bootable
pvcreate -B /dev/rdsk/c1t6d0
Note: the -B parameter tells pvcreate that this will be a bootable
disk.
mkboot /dev/dsk/c1t6d0
mkboot -a "hpux" /dev/rdsk/c1t6d0
2) Create the volume group
mkdir /dev/vg01
mknod /dev/vg01/group c 64 0x010000
vgcreate /dev/vg01 /dev/dsk/c1t6d0
3) Find the size of each logical volume in vg00
vgdisplay -v /dev/vg00 | more
look at LV Size (Mbytes) for each logical volume and note it.
Note: this example will use these value:
lvol1 84M
lvol2 256M
lvol3 140M
lvol4 500M
lvol5 64M
lvol6 20M
lvol7 500M
lvol8 500M

Note: The size of the new logical volumes needs to be exactly the
same as the size of the logical volumes on the primary root disk.
4) Create the first 3 logical volumes contiguous (needed by the system)
lvol1:
lvcreate -L 84 -C y -r n /dev/vg01
lvol2:
lvcreate -L 256 -C y -r n /dev/vg01
lvol3:
lvcreate -L 140 -C y -r n /dev/vg01
5) Now create the other logical volumes
lvol4:
lvcreate -L 500 /dev/vg01
lvol5:
lvcreate -L 64 /dev/vg01
lvol6:
lvcreate -L 20 /dev/vg01
lvol7:
lvcreate -L 500 /dev/vg01
lvol8:
lvcreate -L 500 /dev/vg01
6) Copy each logical volume except the swap which is usually lvol2.
dd if=/dev/vg00/rlvol1 of=/dev/vg01/rlvol1 bs=1024k
dd if=/dev/vg00/rlvol3 of=/dev/vg01/rlvol3 bs=1024k
dd if=/dev/vg00/rlvol4 of=/dev/vg01/rlvol4 bs=1024k
dd if=/dev/vg00/rlvol5 of=/dev/vg01/rlvol5 bs=1024k
dd if=/dev/vg00/rlvol6 of=/dev/vg01/rlvol6 bs=1024k
dd if=/dev/vg00/rlvol7 of=/dev/vg01/rlvol7 bs=1024k
dd if=/dev/vg00/rlvol8 of=/dev/vg01/rlvol8 bs=1024k
7) Verify the integrity of all the new volume except swap.
Note: The following lines are base on a system with vxfs
filesystems except for /stand (lvol1) which needs to be hfs.
fsck -F hfs /dev/vg01/rlvol1
fsck -F vxfs /dev/vg01/rlvol3
fsck -F vxfs /dev/vg01/rlvol4
fsck -F vxfs /dev/vg01/rlvol5
fsck -F vxfs /dev/vg01/rlvol6
fsck -F vxfs /dev/vg01/rlvol7
fsck -F vxfs /dev/vg01/rlvol8
8) Now configure the Boot Data Reserved Area (BDRA)
Note: The following commands assume that /stand is lvol1,
swap is lvol2 and / is lvol3
lvlnboot -b /dev/vg01/lvol1 /dev/vg01
lvlnboot -r /dev/vg01/lvol3 /dev/vg01
lvlnboot -s /dev/vg01/lvol2 /dev/vg01
lvlnboot -d /dev/vg01/lvol2 /dev/vg01
9) Modify the fstab file on the new disk.
a) If /tmp_mnt doesn't exist create it
mkdir /tmp_mnt
b) Mount the new root filesystem on /tmp_mnt
mount /dev/vg01/lvol3 /tmp_mnt
c) change to etc directory on the new disk.
cd /tmp/etc
d) Modify all occurence of vg00 in the fstab for vg01
sed "s/vg00/vg01/" fstab > fstab.out
mv fstab fstab.BAK
mv fstab.out fstab
e) Unmount the new root filesystem
cd /
umount /tmp_mnt

XI) How to mirror a logical volume
Note: Data mirroring is provided by an additionnal purchasable
software product called MirrorDisk/UX.

Note: the following example is using the volume group vg01 and the
logical volume lvhome

To add a mirror to an existing logical volume:
lvextend -m 1 /dev/vg01/lvhome

This will add 1 mirror (2 copies of the filesystem).
To add 2 mirrors (3 copies of the filesystem) use -m 2 instead.

To create a new logical volume of 200M with 1 mirror:
lvcreate -m 1 -L 200 /dev/vg01

XII) How to unmirror a logical volume
Note: the following example is using the volume group vg01 and the
logical volume lvhome

lvreduce -m 0 /dev/vg01/lvhome

XIII) How to create a mirrored boot disk

Note the following example is using the disk c1t6d0 as the
mirrored boot disk and c0t6d0 as the boot disk.

1) Initialize the disk and make it bootable
pvcreate -B /dev/rdsk/c1t6d0
Note: the -B parameter tell pvcreate that this will be a
bootable disk.
2) Add the physical volume to the volume group
vgextend /dev/vg00 /dev/dsk/c1t6d0
3) Use mkboot to place the boot utilities in the boot area and add
the AUTO file.
mkboot /dev/dsk/c1t6d0
mkboot -a "hpux -lq" /dev/rdsk/c1t6d0
4) Use mkboot to update the AUTO file on the primary boot
disk.
mkboot -a "hpux -lq" /dev/rdsk/c0t6d0
5) Mirror the stand, root and swap logical volumes
lvextend -m 1 /dev/vg00/lvol1
lvextend -m 1 /dev/vg00/lvol2
lvextend -m 1 /dev/vg00/lvol3
Note: LVM will resynchronize the new mirror copies. This step will
takes several minutes

Repeat the lvextend for all other logical volumes on the boot
mirror.
6) Modify your alternate boot path to point to the mirror copy of the
boot disk.
setboot -a 8/8.6.0 # Use the Hardware path for your new
boot disk.

XIV) How to mirror a logical volume on a specific physical volume
Note: the following example, is using the disk c1t6d0 for the
primary copy, c2t6d0 for the mirror copy, the volume group vg01,
the logical volume lvhome and the size will be 200M

lvcreate -n lvhome /dev/vg01
lvextend -L 200 /dev/vg01/lvhome /dev/dsk/c1t6d0
lvextend -m 1 /dev/vg01/lvhome /dev/dsk/c2t6d0

XV) How to create a Physical Volume Group (PVG)
create a file named /etc/lvmpvg with the following syntax:
VG vg_name
PVG pvg_name
pv_path
...
PVG pvg_name
pv_path
...
VG vg_name
PVG pvg_name
pv_path
...

For example, to use two PVGs in vg01 with c1t6d0 and c2t6d0
in one PVG (PVG0), c3t6d0 and c4t6d0 in the other PVG (PVG1) the
contents of the file (/etc/lvmpvg) should be:

VG /dev/vg01
PVG PVG0
/dev/dsk/c1t6d0
/dev/dsk/c2t6d0
PVG PVG1
/dev/dsk/c3t6d0
/dev/dsk/c4t6d0

XVI) How to use PVG to mirror logical volumes on specific physical volumes.

Note: in the following text, the volume group will be vg01 and the
logical volume will be name lvhome

After creating the /etc/lvmpvg file as describe above, each copy
of your mirror could be force on different PVG. To achieve this if
the logical volume is already created but not mirrored yet, use the
following command:
lvchange -s g /dev/vg01/lvhome
lvextend -m 1 /dev/vg01/lvhome
If the logical volume is not created yet:
lvcreate -s g -m 1 -n lvhome -L 200 /dev/vg01

###############################################################################################################################################
3200070937
SYS ADM: How to create Swap or Dump logical volumes
Document Information Table

PROBLEM

Please describe how to create swap or dump logical volumes
on an HP-UX 11.0 computer.


CONFIGURATION

Operating System - HP-UX
Version - 11.0
Subsystem - SYS ADM

RESOLUTION

To create a dump logical volume, do the following:

Note: The dump devices have to be part of vg00.

1. Create a contiguous logical volume with bad block reallocation
turned off.

NOTE: If the system is 10.20, the dump partition must be part
of vg00.

If the system is 11.0 or higher, the dump partition
can be created outside vg00 provided that it is included
in the /etc/fstab and the /etc/rc.config.d/crashconf is
configured to scan the /etc/fstab. See document_id
S3100006107 for more details.

The following command creates a logical volume inside vg00 with
bad block reallocation turned off:

lvcreate -L size -C y -r n /dev/vg00

2. Turn on the dump device for the logical volume just created:

lvlnboot -d /dev/vg00/lvolX

To create a swap logical volume, do the following:
Note: The swap and dump can be created on the same logical
volumes. The swap devices do not need to be part of
vg00, but they can be.

1. Create a contiguous logical volume:

lvcreate -L size -C y /dev/vgX

2. Turn swap on in the logical volume:

swapon /dev/vgX/lvolY

IMPORTANT: The default priority for swap is "1". It is
recommended to configure swap areas at the same
priority level when they are on separate disks,
or descending priority levels when they are on
the same disk.

3. Edit the /etc/fstab so that swap will be turned on at
boot-up; add the following line:

/dev/vgX/lvolY ... swap defaults 0 2

NOTE: If you want to remove swap or change the priority of
swap, then it is necessary to edit the /etc/fstab and
then reboot the system.

The difference between file system swap and device swap is as
follows:

File system swap:

o can be configured dynamically.

o allows a process to use an existing file system if it
needs more than the designated device swap space.

o is used only when device swap space is insufficient to
meet demand-paging needs.

o consumes a variable amount of space because it only uses
that portion of a file system that it needs.

Once a file system has been enabled for file system swap, it is
not possible to unmount that file system until the swap is
disabled at the next reboot. It is possible to limit file system
swap to a fixed size to prevent it from consuming too much space.