Linux, PHP, LAMP, The Web in a blog

Command Line with PHP

_ Introduced with PHP 4.3 (2002-12-27)
_ PHP packed with very useful functions: exec, shell_exec, passthru
_ PEAR package Console_CommandLine to easily handle arguments and options
_ You can do system administration and release management with a language you already know
_ Bad: if you run very long and complex processes (i.e. Parsing XML files into database), you can experience memory leaks
_ Bad: PHP is mainly for running short scripts to serve web pages so core developers focus time on that
_ Those issues are going to be addressed soon (something already done as of 5.3)

MultiProcess programming with PHP
_ ps auxf to see the parent/child process tree
_ There are basically 2 methods:
__ pcntl_fork()
__ exec

• http://www.electrictoolbox.com/article/php/process-forking/
• http://www.rooftopsolutions.nl/article/213
• http://dev.mysql.com/doc/refman/5.1/en/gone-away.html (search for fork)
• See the PHP ProcessControl class

This is to see whether you server is CPU-bound or RAM-bound.

We have to stress it and see which one of those components gets saturated first.

If that is the RAM, we can add easily more RAM. The amount of RAM to add depends of the share of the CPU that can still be used before the website becomes slow.

ab -n 300 -c 1 http://www.mywebsite.com

This are the command to run during the stress test

_ uptime

_ free

_ ps -ylC httpd –sort:rss > number_of_apache_processes.txt

_ netstat -nt | grep :80 | wc -l

And obviously check how quick the website is.

Rolling back yum packages:
http://dailypackage.fedorabook.com/index.php?/archives/17-Wednesday-Why-Repackaging-and-Rollbacks.html
http://www.vincentverhagen.nl/2007/12/10/how-to-roll-back-packages-on-centos-5-rhel-5/

When you’ve installed/updates packages with yum or rpm, you can quite easily roll back the updates/installations using rpm.
For this, yum and rpm need to save roll back information, which they do not do by default.
To enable the roll back feature, do the following:

Add tsflags=repackage to /etc/yum.conf.
Add %_repackage_all_erasures 1 to /etc/rpm/macros. If /etc/rpm/macros does not exist, just create it.

You can now install, erase and update packages with yum and/or rpm, and they will save roll back information.

When you want to roll back, use rpm to do so.
You do this by specifying the –rollback switch and a date/time, like the examples below:

rpm -Uhv –rollback ‘19:00′
rpm -Uhv –rollback ‘8 hours ago’
rpm -Uhv –rollback ‘december 31′
rpm -Uhv –rollback ‘yesterday’

Optimize tables overnight

Graphs with free memory everyday (with gnuplot and/or Ganglia)

clever report of the most frequent Apache errors

Cronjob to check Apache and MySQL are up, otherwise start them

Cronjob to check the space left

Make sure you don’t use personal email addresses for the monitoring/alerts (ie. mike, dan) but use aliases like: admin, developers

Use Public/Private Keys for Authentication

Using encrypted keys for authentication offers two main benefits. Firstly, it is convenient as you no longer need to enter a password (unless you encrypt your keys with password protection) if you use public/private keys. Secondly, once public/private key pair authentication has been set up on the server, you can disable password authentication completely meaning that without an authorized key you can’t gain access - so no more password cracking attempts.

It’s a relatively simple process to create a public/private key pair and install them for use on your ssh server.

First, create a public/private key pair on the client that you will use to connect to the server (you will need to do this from each client machine from which you connect):

$ ssh-keygen -t rsa
This will create two files in your (hidden) ~/.ssh directory called id_rsa and id_rsa.pub. id_rsa is your private key and id_rsa.pub is your public key.

If you don’t want to still be asked for a password each time you connect, just press enter when asked for a password when creating the key pair. It is up to you to decide whether or not you should password encrypt your key when you create it. If you don’t password encrypt your key, then anyone gaining access to your local machine will automatically have ssh access to the remote server. Also, root on the local machine has access to your keys although one assumes that if you can’t trust root (or root is compromised) then you’re in real trouble. Encrypting the key adds additional security at the expense of eliminating the need for entering a password for the ssh server only to be replaced with entering a password for the use of the key.

Now set permissions on your private key:

$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/id_rsa
Copy the public key (id_rsa.pub) to the server and install it to the authorized_keys list:

$ cat id_rsa.pub >> ~/.ssh/authorized_keys
Note: once you’ve imported the public key, you can delete it from the server.

and finally set file permissions on the server:

$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/authorized_keys
The above permissions are required if StrictModes is set to yes in /etc/ssh/sshd_config (the default).

Now when you login to the server you won’t be prompted for a password (unless you entered a password when you created your key pair). By default, ssh will first try to authenticate using keys. If no keys are found or authentication fails, then ssh will fall back to conventional password authentication.

Once you’ve checked you can successfully login to the server using your public/private key pair, you can disable password authentication completely by adding the following setting to your /etc/ssh/sshd_config file:

# Disable password authentication forcing use of keys
PasswordAuthentication no

This is the most reliable method because is actually scans the ports:

nmap -sT -O localhost

Other methods based on internal checks that give you also the information about the program using the ports:

netstat -anp
lsof -i

yum install php-pear
yum install php-devel
yum install httpd-devel

Edit the file /usr/share/pear/pearcmd.php and add the following at the beginning:
@ini_set(’memory_limit’, ‘16M’);
otherwise you’ll probably get a fatal error whilst building the extension:
pecl install apc

Now configure PHP to use the new extension.
Create the file /etc/php.d/apc.ini and in that file put:
extension=apc.so

Now restart apache
sudo /etc/init.d/httpd greaceful

In the future, if new versions of APC are released, you can easily upgrade them using:
sudo pecl upgrade apc

Now, to configure APC there are two options: either edit php.ini, or add apc.ini to the php.d folder. Either way, I use the following:

extension = apc.so
apc.enabled = 1
apc.shm_size = 48
apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX

It’s very important to choose the size of the cache (apc.shm_size) because when it gets full, APC empties it.

Finally I restart apache again (service httpd restart) and verify that php -v returns the same output. If I want to verify that apc is working, I locate apc.php on the server and then copy it to a web-accessible directory. Then I load that script in my browser, and if I can see pretty graphs, then it’s working.

APC works thanks to share memory (RAM).

apc.shm_size aside, it’s also important to set correctly the number of sectors and apc.stat that it’s about whether to check the timestamp of the files (before serving) to re-cache them (see documentation).

IMPORTANT! I had this problem: the cache was flushed completely after getting full. That was because I need to set:
apc.ttl and apc.user_ttl to a non-zero value (for example 7200). In your vim /etc/php.d/apc.ini:
apc.ttl=7200
apc.user_ttl=7200

http://www.pingdom.com

CPU

• top -> for diagnosys - must be less than the number of processors
• profiling -> xDebug
  Can’t involve just one page (unless you use a very simple one, not xDebug for sure) but the all site. Maybe you don’t want to do it on the production server (because of the loss in performance). Then you have to replicate the live environment on a profiling machine and generate some synthetic traffic
• opcode caching -> APC
• smarty -> for even better performance, compile_check off on production because we are not supposed to modify files on production apart from scheduled deploys

Disk

• most limiting factor: speed of the hard disk
• iostat for diagnostic

Network

• It’s rarely a problem in a local cluster of server because they have gigabits interfaces
• diagnostic: netstat

Memory

diagnosis: top, vmstat, ps aux, pstree

Database

• Supposing we have a common method to send execute queries in the database, we can always include a part of the debug backtrace in the query as comment. That way, it is easier to identify the location in the code when you read the processlist or a log
• Query profiling - in MySQL it’s possible to log all the executed queries. You can enable it temporary to spot out some performance issue. Unfortunately it requires the server to be restarted
• To improve performance:
  • indexes
  • caching
    • memcached is a great example
    • very good results if the application performes a lot of reads and few writes
    • very important to remember to invalidate the cache when data is updated
  • denormalization
    • to avoid joins
    • at a certain scale, it becomes more and more essential
    • you can use some tools to check data is in sync (even overnight)

It’s good for deploy as well.
It can be used locally (like this: rsync -aHvzO –progress staging/ www/).
Very useful to move big amount of data over the Internet because of its incremental feature.

By default it uses SSH for transfering data.
Probably you would like to perform an automatic SSH login (without entering the password). For doing that, read the related article.

Installation and creation of a dedicated account
yum install rsync
/usr/sbin/adduser backup_user
/usr/bin/passwd backup_user

Uses
You can use rsync as a server/client application or ‘on-demand’ basis.
In the first use, there will be a server machine in which there is rsync running as a server. You can run it as a stand-alone server (using rsync –daemon) or using xinetd. See the attachment at the end of this article or visit this URL: http://transamrit.net/docs/rsync/ to see how to set up the server. In this case, the server will have some shares (something like repositories) and a client can use them.
In the second use, on the destination machine there won’t be a rsync running. You need just to install rsync to use this mode. This second one is the more likely method you will use unless you want to create some public repositories.

The command on the source system is pretty much the same in both the modes, but the way to define the destination URL is slightly different.

Some examples of ‘on-demand’ use - Good for backup
This will back up some local directories on the server vps.

rsync -rlDzC --progress --delete -e 'ssh -p 233 -l backup_user' /mnt/data/penny/ vps:/home/autobck/total_penny/
rsync -rlDzC --progress --delete -e 'ssh -p 233 -l backup_user' /mnt/data/foto/ vps:/home/autobck/photos/

• The option -O prevents an error (probably creating a new directory remotely) from happening.
• As SSH is already used for transport by default, you don’t need the -e option. You do need it if the port of the SSH server (running on the destination machine) is not the standard one or if the user you want to use for the SSH session is not the one you are logged in locally
• vps is an entry in your /etc/hosts file associated with the ip of the remote system

Some example scripts:
Backing up from local machine to remote backup server (with the same user):

#!/bin/bash
export RSYNC_RSH=/usr/bin/ssh
dest=backup1
user=$(whoami)
cd || exit 1
rsync -aHvz --progress --delete . "${user}@${dest}:."

[-z -> to compress, not necessary in LAN]
[-b -> backup of the already exisitng destination files]
[–delete keeps the two copies perfectly in sync, by deleting deleted files]
SSH is useful just if you send data over the Internet.

Listing the files on the backup server:
#!/bin/bash
dest=server1
user=$(whoami)
cd || exit 1
rsync “${user}@${dest}:.” | more

Restoring (the script runs locally):
#!/bin/bash
dest=server1
user=$(whoami)
cd || exit 1
for file in “$@” ; do
rsync -aHPvz “${user}@${dest}:./${file}” “./${file}”
done

For restoring, you simply run the script, passing the names of the files to be restored as arguments on the command line.
We can also restore all the files at once by using a dot as the filename

How to setup a rsync server