Including benchmarks

As part of the namespace discussion, the performance difference between including multiple files and concatenating those files into a single file, mainly because of the aforementioned limitation.

In response to an argument for concatenation, Rasmus Lerdorf (original author of PHP) said:

Note that what you are saving by combining the files into one is just a single stat syscall per file. And that can be alleviated by setting apc.stat=0 in your config. That of course means you need to restart your server or flush your cache whenever you want to update the files. In apc.stat=0 mode you gain nothing by merging all the files.

Now, I’d always theorized that concatenation could a present measurable performance increase, so I was a little muffed by this statement. Consequently, I ventured forth to prove this out.

Setup

First, I generated 100 files each containing a single class. The files and classes were named consecutively, 1-100. I then created one script that dynamically included each file, another that statically included each file, and one that contained all of the files concatenated together. I used Apache Bench and had APC enabled. I ran one set of tests with apc.stat=0 and one with apc.stat=1. The tests were run on a AMD Sempron 2600+ with 512MB of RAM, running Apache 2.0.58 with mpm-prefork and PHP 5.1.6.

Results

I was a little surprised by the results. Because they were better than I’d ever theorized: even with apc.stat=0, the concatenated file request times were less than half those of the static list.

Dynamic

Requests per second:    247.86 [#/sec] (mean)
Time per request:       40.346 [ms] (mean)

Static

Requests per second:    279.86 [#/sec] (mean)
Time per request:       35.732 [ms] (mean)

Concatenated

Requests per second:    605.07 [#/sec] (mean)
Time per request:       16.527 [ms] (mean)

Also of note: the dynamic includes didn’t really lose by much to the static includes. That should make people using __autoload (like me, at work) happy (it does).

Apparently, there’s still some performance to be gained by concatenating your libraries into a single file, even with a byte-code cache. Now, granted, you may not always include 100 files, or your entire library. However, by analyzing your usage patterns, you could always create a single file containing the classes you use the most, and leave those only occasionally used to your autoload implementation.

3 Comments so far

  1. n1zyy on July 25th, 2007

    Huh, I have to applaud you for taking the time to test it.

    How would these numbers change if no caching was being used? (Alarmingly, that’s currently the case here, although I hope to remedy that situation eventually.)

  2. andrew on July 25th, 2007

    Turning the cache off seemed to close the gap between the dynamic and static tests: the dynamic averaged about 150 requests/second, static just under 160. The concatenated file rang in at about 450 requests/second, again over twice as fast.

  3. Matteo on August 19th, 2007

    hi nice post, i enjoyed it

Leave a Reply