Linux, PHP, LAMP, The Web in a blog

Rolling back yum packages:
http://dailypackage.fedorabook.com/index.php?/archives/17-Wednesday-Why-Repackaging-and-Rollbacks.html
http://www.vincentverhagen.nl/2007/12/10/how-to-roll-back-packages-on-centos-5-rhel-5/

When you’ve installed/updates packages with yum or rpm, you can quite easily roll back the updates/installations using rpm.
For this, yum and rpm need to save roll back information, which they do not do by default.
To enable the roll back feature, do the following:

Add tsflags=repackage to /etc/yum.conf.
Add %_repackage_all_erasures 1 to /etc/rpm/macros. If /etc/rpm/macros does not exist, just create it.

You can now install, erase and update packages with yum and/or rpm, and they will save roll back information.

When you want to roll back, use rpm to do so.
You do this by specifying the –rollback switch and a date/time, like the examples below:

rpm -Uhv –rollback ‘19:00′
rpm -Uhv –rollback ‘8 hours ago’
rpm -Uhv –rollback ‘december 31′
rpm -Uhv –rollback ‘yesterday’

Optimize tables overnight

Graphs with free memory everyday (with gnuplot and/or Ganglia)

clever report of the most frequent Apache errors

Cronjob to check Apache and MySQL are up, otherwise start them

Cronjob to check the space left

Make sure you don’t use personal email addresses for the monitoring/alerts (ie. mike, dan) but use aliases like: admin, developers

http://www.pingdom.com

CPU

• top -> for diagnosys - must be less than the number of processors
• profiling -> xDebug
  Can’t involve just one page (unless you use a very simple one, not xDebug for sure) but the all site. Maybe you don’t want to do it on the production server (because of the loss in performance). Then you have to replicate the live environment on a profiling machine and generate some synthetic traffic
• opcode caching -> APC
• smarty -> for even better performance, compile_check off on production because we are not supposed to modify files on production apart from scheduled deploys

Disk

• most limiting factor: speed of the hard disk
• iostat for diagnostic

Network

• It’s rarely a problem in a local cluster of server because they have gigabits interfaces
• diagnostic: netstat

Memory

diagnosis: top, vmstat, ps aux, pstree

Database

• Supposing we have a common method to send execute queries in the database, we can always include a part of the debug backtrace in the query as comment. That way, it is easier to identify the location in the code when you read the processlist or a log
• Query profiling - in MySQL it’s possible to log all the executed queries. You can enable it temporary to spot out some performance issue. Unfortunately it requires the server to be restarted
• To improve performance:
  • indexes
  • caching
    • memcached is a great example
    • very good results if the application performes a lot of reads and few writes
    • very important to remember to invalidate the cache when data is updated
  • denormalization
    • to avoid joins
    • at a certain scale, it becomes more and more essential
    • you can use some tools to check data is in sync (even overnight)

DNS Diagnostic

• whois (simply displays the NS servers for the domain)
• host -v domain.com
• To query NS servers:
  • dig plancake.com displays info about A records
  • dig plancake.com MX displays info about MX records
  • dig @ns.123-reg.co.uk plancake.com MX to query a particular server
• nslookup can be used in an interactive mode, too.
  Here is an example:

  # nslookup
  > server ns2.libero.it [I’m connected to Libero NS server]
  > set q=ns [setting which kind of query to do. ‘ns’=nameserver OR ‘any’]
  > italandia.it [displays the records stored on the server ns2.libero.it concerning the domain italandia.it]

Network Diagnostic

/etc/services: list of conventional ports
netstat: says the active connections
nmap server_ip_address: says the open ports (launch it from outside the server)
Once you know an open port, in order to know the process is using it:
lsof -i | grep port_number
traceroute

Database maintenance

OPTIMIZE TABLE tablename
It performs a sort of defragmentation for the files containing the table’s data.

Examine the The Slow Query Log

Login control

* last - shows the file /var/log/wtmp
* grep -i “accepted” /var/log/auth

If you find some strange accesses:
less /root/.bash_history
[I’m not sure this file is updating very often]
If this file is empty, points to /dev/null or contains some strange commands, it means the cracker was able to do whatever he wanted.

Misc

* Checks the logs, in particular cron’s one (/var/log/cron.log)

* Keep the system update and patched

#!/bin/bash
#
# my_backup.sh V1.0
#
# An useful script that performs a variety of operations in order to backup different data.
# It’s a good starting point for your bespoke one

# Creating all the definitions
datestamp=$(date +’%Y%m%d%H%M%S’)
tarname=my_backup-$datestamp”.tgz”
tempdir_base=tmpbckdir$datestamp
tempdir=*/var/tmp_backups/*$tempdir_base
tempdir_db=$tempdir”/db”

# Creating the directories for the operations
mkdir $tempdir
mkdir $tempdir_db

# Backuping databases
mysqldump -u *superuser* -p*superuser_psw* *database1* | gzip –best > $tempdir_db/*database1*.sql.gz
mysqldump -u *superuser* -p*superuser_psw* *database2* | gzip –best > $tempdir_db/*database2*.sql.gz
mysqldump -u *superuser* -p*superuser_psw* *database3* | gzip –best > $tempdir_db/*database3*.sql.gz

# Backuping databases - There are a lot of databases called like this: user_user1, user_user2,…
cd /var/lib/mysql/
for i in $( echo user_* ); do
mysqldump -u *superuser* -p*superuser_psw* $i > $tempdir_db/$i.sql
done

# Backuping files
cp -ra *dir1* $tempdir/*dir1*
cp -ra *dir2* $tempdir/*dir2*

# Sending a mail with the content of a log file
mutt -s “log file” -a *file_path* *mail address* < *file_path*
# truncating the file
echo “” > *file_path*

# Storing the compressed backup in a local disk
mount /dev/hdc1 /mnt
cd /etc/sm_it
tar czf /mnt/$tarname $tempdir_base
umount /mnt

# Deleting the temp files
rm -r $tempdir

LAMP Backup Script

• Copy the file iptables.sh in the directory /etc/init.d with these rules (but obviously you need to customize them).
• Make the file executable.
• Assuming the current run level is the second one:ln -s /etc/init.d/iptables.sh /etc/rc2.d/S97iptables
  Actually you should link the script in any runlevel to cover the case you change runlevel, even temporary.

******************************************************************

iptables script

******************************************************************

Useful Commands:
List of active rules:
iptables -L
Removes all the rules
iptables -F

The backup file will be stored in the directory /var/dobackup-pool.
The comment will be stored in the same directory; the filename will be the one of the backuped file plus the postfix ‘comment’.
Its filename (once stored) will the the absolute path (in which the character / is replaced by two dashes) + the actual filename + a timestamp

dobackup

MRTG