Posts Tagged ‘utf8’

Web Development with UTF8

Sunday, November 16th, 2008

There are two mappings: the numerical value corresponding to a character (charset) and the binary representation of that value (encoding). UTF is a charset that has a number of encodings; utf8 is an encoding.
Thanks to utf, we can have different languages on the same webpage.

First of all, make sure the encoding of your text editor is utf8.

PHP

header("Content-Type: text/html; charset=utf-8");

You don’t need that if you managed to edit your httpd.conf properly.
As long as you don’t want to mess with the content of a string, you can pass it around blindly. But as soon as you want to use any function that relies on substring operations (such as substring, wordwrap, chunk_split) you need to use the equivalent multibyte functions, obtaining prepending mb_ (you need to install the extension on PHP)

MySQL

Similar to PHP (careful with substring and fulltext indexing) but no problem if the charset of the db is UFT8

Email

Similar to HTML, just use:
<meta http-equiv=”content-type” content=”text/html; charset=utf-8″ />
The problem is that that meta applies only to the body but it’s quite likely you need UTF8 also for the to/from header. In that case you need a trick you can find on Google

HTML

  • Don’t forget the following in your head section:
    <meta http-equiv=”content-type” content=”text/html; charset=utf-8″ />
  • Javascript

    It fully supports UTF8 apart from the escape() function. Google the utf8 version if you need it.

    User Input

    No problem with the user input because the browser sends data back to the server using the same encoding the server is using. But there are some exceptions (very old browsers, hacking attempts,…)

    Server Setup and LAMP Setup From Scratch with full UTF support (utf8)

    Sunday, February 3rd, 2008

    Set the clock
    date MMDDhhmmYYYY
    hwclock –systohc &

    Where: M=month, D=day, h=hour, m=minute, Y=year
    The latter command is essential: it copies the date into the BIOS and makes the change definitive

    First steps
    * echo “hostname_you_chose” > /etc/hostname
    /bin/hostname -F /etc/hostname

    Changing the hostname could be very different distro by distro.

    * /etc/host.conf :
    order hosts,bind
    multi on
    * /etc/hosts:
    127.0.0.1 localhost.localdomain localhost
    192.168.0.10 hostname_you_chose.localdomain hostname_you_chose
    * Add these lines at the bottom of the file /etc/profile (or of the file .bashrc of an user - even root - whether you want these options applied just on a specific user):
    alias ls=’ls –color’
    alias rm=”rm -i”
    alias halt=”echo command disabled by alias”
    alias ifdown=”echo command disabled by alias”
    alias iptables=”echo you’re working on the server!”
    alias exit=”/etc/init.d/general_check.sh; exit”

    where /etc/init.d/general_check.sh is a script that performs some important check (permission, file existance, …) to make sure you’re not ruining anything in your last session.

    N.B.: in the above commands, it’s very important not to use extra spaces between words.

    Cleaning up the system

    * nmap server_ip_address [to find out the open ports]
    * Disable portmap (if active). It should be binded to the port 111 and refers to the service rpcbind.
    * Disable fingerd (if active)
    * IMPORTANT: if you want to shutdown a service, it’s not sufficient you stop it but you must be sure there isn’t a symbolic link in /etc/rcX.d
    * IMPORTANT: don’t uninstall at all the default MTA (for example exim4) because it could be essential for the local operations.

    SSH e SCP

    /etc/ssh/sshd_config: [both for improving security]
    Port a_different_port_rather_than_the_default_one
    PermitRootLogin no

    Before restarting sshd, MAKE SURE there is a non-root user in your system who can connect via SSH (and test it in another terminal), otherwise you’re bust!!!!!!!!!!!!!!!!!!!!!!!!!!

    useradd -g users -s /bin/bash daniele
    passwd daniele

    Apache

    Change the default user and group owner of the apache processes. Let’s say, user: pippo and group: pippo.
    groupadd pippo
    useradd -g pippo -s /dev/null pippo

    So in the configuration file:
    User pippo
    Group pippo
    and through the command line:

    In this way, the user pippo won’t be granted to login to the system and launch commands. These should increase the security.

    Then, create a new user that will be the owner of all the web documents. It must belong to the group pippo so Apache can access the web documents as well.

    useradd -g pippo -d /var/www/html/ -s /bin/bash dev

    chown -R dev:pippo /var/www/html

    chmod -R 770 /var/www/html

    chown -R dev:pippo directory_that_contains_sessions_see_the_php_ini

    Always in the configuration file, remember to disable potential directives for the generation of the log file for the rewrite module (unless you temporary need it for debugging) as it’s very computation intensive.

    Always in the configuration file, check that DirectoryIndex directive contains:

    index.html index.htm index.php

    Remember to customize the error pages, for example:
    ErrorDocument 404 /misc/error_pages/error_404.php
    You can put it in the VirtualHost Section

    MySQL

    Set a (very hard to guess!) password for root
    mysqladmin -u root password root_password_you_like
    IMPORTANT: don’t use the same password as for the root user in Linux
    You could need this for the changes to take effect:
    mysqladmin -u root -p flush-privileges

    MySQL allows anonymous login by default. To disable it, just Google on the MySQL documentation.

    To access the server:
    mysql -u root -p

    IMPORTANT: take a look at the user table. Make sure any user has got a password and they can access the server only locally that is they have ‘localhost’ as host. You can modify this table through the GRANT command.

    IMPORTANT: Set utf8_general_ci as the server default collation

    IMPORTANT: The databases are stored in /var/lib/mysql

    touch /var/log/mysqld_slow_queries.log
    chown mysql:mysql /var/log/mysqld_slow_queries.log
    chmod 640 /var/log/mysqld_slow_queries.log
    In the my.cnf
    log-slow-queries = /var/log/mysqld_slow_queries.log
    long_query_time = 1
    The above will log queries taking longer than one second to the specified log file.

    PHP

    If you’re confident no malicious scripts will be running, increase
    max_execution_time
    memory_limit

    Make the services start at the boot

    This is achieved by symbolic links in the directory related to the runlevels.

    Make everything UTF8

    • httpd.conf:
      AddCharset UTF-8 .utf8
      AddDefaultCharset UTF-8
    • php.ini
      default_charset = “utf-8″
    • my.cnf
      character-set-server=utf8
      default-collation=utf8_unicode_ci

      Then, if you’re writing PHP scripts, soon after opening your connection to mysql, issue one of the following:
      SET NAMES ‘utf8′;
      OR, if you are running the mysqli extension:
      mysqli_set_charset(’utf8′);

    • Last suggestions

      * Make sure you gave the minimal permission to any file added or modified in the server
      * Set a cron job for backups
      * Make sure the log for cron is active. There should be a line to uncomment in the file /etc/syslog.conf or something similar. The log file should be /var/log/cron.log. Then restart the cron service.
      * Keep your system updated with security patches and new versions of the installed software.