PHP libxml issue while compiling

Today I had some trouble getting libxml to work in PHP. I was compiling PHP 5.3.10 for a client's Joomla site and it needs libxml. The issue seemed to be commonly happening to many people and I came across a number of forum threads on the topic.

The suggestions mostly hover around installing libxml2 development package. This can be done in Debian or Ubuntu by:
apt-get install libxml2 libxml2-dev

On other Linux distributions you may try libxml2-devel

But I had already done that and still libxml was not showing up in PHP. I had the --with-libxml-dir setting in ./configure with no luck.

Then I came across the --enable-libxml setting, which although is not mentioned directly in the `./configure --help`, it should have come to my mind. This is how you enable (or disable with --disable) any module from the ext/ directory. Anyway, I found it mentioned here and that worked! The reason for this is because I had --disable-all option set. So all default modules were off, and I had to enable libxml with its enable option.

Hope this helps someone else...

Building a Yii app: The Data Model

A friend of mine needs a web application to be revamped. I had originally created it about 2 years back using a custom PHP mini-framework that I had built for many projects at that time. The development had stopped for different reasons and parts of his web application were incomplete. For example images were not upload to Amazon S3, which was originally planned. Some Model edits were not working in many parts and there were some data validation issue. He has been doing the groundwork for his business and now has decided to finally complete the application.

The application is related to Medical needs, information about patients, doctors, hospitals etc. I have decided to make the new version using Yii, since some other developer will take over if the project is successful and Yii (or other popular frameworks) is a very well documented framework for anyone to use. The choice of Yii against other PHP frameworks is rather just an impulsive one. I have read comparisons of the good PHP frameworks and Yii is among the top few. Anyway moving on...

While I am building this application, which is a moderately feature rich one, I intend to write about my experiences. Hope this helps anyone looking for a quick introduction and example for Yii. I am trying to make this a tutorial for Yii, I will try my best here. The project needs data for multiple types:

• General User Profile (could be a patient)
• Doctor
• Hospital
• Nursing Home
• Other Medical companies like: Medical Shop, Diagnostic Center, Fitness Center, Ambulance Provider, Nurse, etc.

Other information include:

• Specialization: this is related to Doctors, explained below
• Address: City, State, Country, etc. Any entity can have multiple addresses
• Phone: can be either a fixed or a mobile (cellular) phone. Any entity can have multiple phones, also address can have phones associated with them
• Department: Hospitals or Nursing Homes can specify many departments
• Branch: Hospitals, Nursing Homes of Other companies can have multiple locations/branches.
• Image: multiple images for any entity.
• User: this is used for authentication, simple email/password for now.

In order to manage the mappings of many entity types to Address, Phone, Branch, Image etc. I have used a central Entity table. Every type of physical entity has an entry in the Entity table (including each branch). Then Address, Phone, Branch, Image, Department are mapped to Entity table.

In Yii terms the relations look like this:

• Doctors can have one or more Specialization (HAS_MANY in Yii Model)
• Doctor, Hospital, Nursing Home, Other, Profile have a one-to-one mapping to Entity (BELONGS_TO in Yii)
• Entity can have multiple Addresses, Phones, Images (HAS_MANY in Yii)
• Entity can have many Departments (HAS_MANY). This is limited to only Hospital, Nursing Home or Other types though at the application level.
• Entity can have another Entity as Branch (HS_MANY). This is also limited to Hospital, Nursing Home or Other types at the application level.

I will write about setting up Yii Models in the next blog entry...

Templates with Mustache in JavaScript and PHP

For a long time I have used PHP as the template language with nothing else on top of it. No template engines ever. This was all fine till the point that we wanted a full AJAX UI in forums.com and also support basic view in non JavaScript mode. We also wanted to also change to a template structure which is more graphic designer or HTML developer friendly where only knowledge of HTML/CSS is needed. Thus we needed a logic-less no-frills template engine which has support in JavaScript and PHP at the minimum.

After searching a little we came across Mustache, a logic-less template engine with support in many more languages than we needed. But we were certainly happy to see support in Python and Scala, just in case... Well a brief look at the readme in JavaScript version and PHP version will tell you the story: it is damn simple.

Mustache supports Number, String, Array or Object like data structures as its data source (view). It can also check for Boolean and function returns. Well that is more than sufficient for our needs. The major benefit was not that though. If you continue using any PHP based template system which you may end up having some PHP logic stuff somewhere in your templates after a while. This happened to us and we hated it. Our design team obviously wants to keep templates to themselves and thus the need to move to very clean templates.

Now how do we deal with PHP and JavaScript versions? We make one file per module in our system. So say discuss.html for templates related to discuss (Gab) system. Similarly user.html and usermanage.html and so on. Each file contains multiple individual templates which correspond to the sub-feature being requested. These are separated by HTML comments. The files are read in by a PHP code and provided as PHP array of strings to the PHP rendering engine or as JSON array of strings to the JavaScript rendering engine. This rendering engine is not Mustache, its a layer on top of it, which collects templates and data (view in Mustache language) and then send them for rendering.

The major difference between the PHP and JavaScript version of rendering is the way the layout is built. In the case of PHP, every sub-template is parsed, every inch of data prepared and then actual Mustache rendering starts. All this is synchronous. Smaller parts are fed to larger ones as strings, and so on. So we use View Partials in Mustache. But in the case of JavaScript rendering it is a different world. We only render outer portions first. For partials we render them later and modify DOM to insert them back to the page once the partials are rendered. So this allows us to have partials request their own data in totally asynchronous manner which is what we wanted.

We are still in process of developing this whole system. Of course we threw away hundreds of lines of old code to put this new system in place. And much of utility has shifted to JavaScript mode only. Also the back-end now behaves more like an API which just server JSON/PHP arrays.

Odd error with Redis, PHPRedis, Gearman, PHP

We have a setup for background jobs for all the websites we manage like forums.com, files.com, payments.com, etc. The background jobs play around with lists of data which are stored in Redis. We use Tokyo Cabinet for our main data store though.
The setup is somewhat like this:

• PHP with gearmand extension
• Gearmand (the daemon)
• PHP with PHPRedis extension
• Redis daemon

Now our background workers were typical Gearman workers written in PHP. All was fine till the time we shifted list manipulation codes to background. Usually in background workers, we connect to all needed daemons (Tokyo Tyrant, Redis, etc.) and go into the Gearman wait loop.
We never had any errors here until we brought in Redis. Earlier we used to manage lists of data in Tokyo itself with PHP based arrays. Now we were shifting to Redis lists. But we had this error coming to us (in the following form):
protocol error, got '%c' as reply type byte

We spent days on this and could not figure out why. We changed from PHPRedis to native PHP extensions, we had similar errors. Finally we had a hunch that the Redis connection was getting an idle timeout. The reason for such an assumption was that we had the errors every once a while and then we had all things fine again. Again after a while a few errors. But when we had errors we would lose data.

We checked the configuration for Redis and found the fault. The connection timeout was set to 300, which we then changed to 0 (no timeout). Doing this fixed the errors. And all stuff work great now.

Only if the errors from PHPRedis was a bit clearer, it would have been easier for us to catch the error.

Senior developer for software.com (PHP)

We are looking for a senior developer for software.com. Here are the details:

• At least 4 years experience in developing with PHP
• At least 1 year in a large E-commerce website
• Must have dealt with large website projects (1 Million PV/month at least)
• Must have experience in any MVC framework
• In depth knowledge of HTML/CSS/JavaScript or other UI technologies and browser issues

Skills needed (must):

• PHP version 4/5
• HTML/CSS/JavaScript
• Linux (Fedora/Red Hat/Debian/CentoOS)
• E-commerce system (any large E-commerce platform in PHP)

Extra skills (bonus):

• Amazon AWS (EC2, S3)
• MySQL optimization and tuning for high volume websites
• Python, C, C++, Java experience

The person must be able to manage her/his daily task, and communicate the same to the Australian team.
We do not have managers in the team so proper work management for oneself is a must.

Location: AD-38, Sector 1, Salt Lake City, Kolkata, India

Company Portfolio: The position is for software.com, running under Startive Networks Australia (investor into software.com, freelancer.com, forums.com etc.)
The person will be on payroll of Pixlie Web Solutions Pvt. Ltd. which already manages forums.com.
software.com will have a fresh team to build an e-commerce application on the software.com domain.
It will be similar to buy.com but for software business.

Compensation: For the right candidate we are willing to provide up to Rs. 80,000/- (about USD 2K) a month.

Other details: We are a very young and cool team working on some of the greatest domains available out there.
Breakfast, lunch, evening snacks are available at office free every day.
Also dedicated gaming machine, movies every once a while, and other indoor games available.
If you are willing to shift from outside Kolkata, we will arrange for your accommodation for first 3 months.

Experienced PHP Developer Needed

We need a very experienced PHP developer willing to work full time on a contract basis. This is telecommute work so you can work from anywhere. You will be working on new products mostly on Amazon Web Services platform. Required:

Linux, PHP, MySQL
Amazon Web Services: EC2, S3, SDB, SQS
Using Tarzan AWS PHP library
Experience in JS is a plus (AJAX will be used)

We are looking for experienced developer who can work in a distributed team environment.
Pay is very good to say the least.
We will prefer someone from Kolkata so (s)he can work from my place if needed.

Import MySQL dump with PHP on webhosts

Webhosts like GoDaddy and other similar often limit how much you can import with the online control panel (phpMyAdmin in most cases). So here is a simple PHP script that does this for you. You have to take a mysql dump from your database. Upload it to you host.

Make sure you change the mysql_connect parameters to reflect your database server settings. The following line numbers need change:

1. line 2: mysql_connect( ) parameters: Database host, username, password.
2. line 3: mysql_select_db( ) parameters: Database name.
3. line 4: fopen( ) parameters: path and name of you SQL dump file.

<?php
mysql_connect('host', 'user', 'password');
mysql_select_db('database');
$file = fopen('dump.sql', 'r');
print '<pre>';
print mysql_error();
$temp = '';
$count = 0;

while($line = fgets($file)) {
  if ((substr($line, 0, 2) != '--') && (substr($line, 0, 2) != '/*') && strlen($line) > 1) {
    $last = trim(substr($line, -2, 1));
    $temp .= trim(substr($line, 0, -1));
    if ($last == ';') {
      mysql_query($temp);
      $count++;
      $temp = '';
    }
  }
}
print mysql_error();
print "Total {$count} queries done\n";
print '</pre>';

You may use the code for any purposes

What I have done is simply read the dump line by line. Lines starting with -- or /* are considered comments and left out. Now we need to build full SQL commands which span across many lines in the dump. So we check for the end semicolon (;) and until we find one we just join the lines together to get one SQL statement. Once done we execute it and move on to the next.

If you find any errors please comment. Also if you port this to use PostgreSQL or use other programming languages please share your code's link on this post's comment.

My laptop: my local web server

Well two months back I bought my first laptop. A Compaq V3155AU with following configuration:
AMD Turion64 (single core) 2.0Ghz
Hyper Threading support
512MB DDR2 system memory (I upgraded that to 1GB)
nVidia 6150 graphics chip-set with shared memory
14.1" display
60GB SATA hard disk drive
DVD/CD R+RW
Bluetooth
Ethernet
Wireless Lan
At sub Rs. 40,000 (sub US$ 900) this was a great deal. I was using Fedora Core 6 (64bit edition) on it but was not using it much since I still love my desktop and do not have enough work to justify simultaneous usage of two computers. Recently my friend planned to join me on web development work and I thought this was a good time to use it regularly. So I had to first make sure the Wireless Lan worked since I Fedora had not done that automatically. I read:
"Compaq Presario V3000 with Ubuntu 6.06" and it helped a lot. Actually I got the Wireless Lan running in no time. It was very easy. If after a shutdown or restart you need to connect just use the usual commands (iwconfig, iwlist, dhclient to scan for network, connect, etc.).
I setup up my usual structure for web projects. PHP was already there, so was Apache. A few needed PHP modules were added (APC, MagickWand for ImageMagick) and I had a working local web server !!!
Things are really that easy if you are using the right software :) and there is always Google to help you when need it.

Upgrading forum with phpBB2 and restoring from database backup

Well I had to do some phpBB (phpBB2) issues for a client who runs hispanito.com. He had a forum which ran on probably phpBB2 version 2.0.7 and with MySQL 4.x. Now I had to bring those data to a different server with MySQL 5.x I tried a fresh install with latest phpBB2 version 2.0.21, but it failed as soon as I tried to bring in the old data. I tried an Upgrade too, but that failed too. Then I tried many other ways, none of which worked so at last I simply understood that table structures must have changed other than an very visible change in the number of tables ("session_keys" was a new table). Now I had to edit the tables that had changed almost by hand, so if you must have to do the same here are the tables which I changed: "users": two columns added: "user_login_tries" and "user_last_login_try" "sessions": one column added: "session_admin" "search_results": one column added: "search_time" You have to do the above changes after you make a normal install of new phpBB2 and then restore old database backup, which will change the table structures. Hope that helps.