Differences

This shows you the differences between two versions of the page.

--- programming:general:phpvspythonvsperl [2008/02/01 20:30] – crustymonkey
+++ programming:general:phpvspythonvsperl [2008/04/14 13:55] (current) – crustymonkey
@@ Line 12: / Line 12: @@
 ^ OS | Gentoo Linux |
-The second machine was my personal laptop, a **Gateway MX6931** running **Gentoo Linux**.  From here on out, I will refer to this machine as the "Laptop".  The details are as follows
+The second machine was my personal laptop, a **Gateway MX6931** running **Ubuntu Linux**.  From here on out, I will refer to this machine as the "Laptop".  The details are as follows
 ^ CPU | single, dual core Intel Core2 1.66GHz |
 ^ Memory | 2GB |
-^ OS | Ubuntu 6.10 |
+^ OS | Ubuntu 7.10 |
 ==== The Interpreters ====
@@ Line 344: / Line 344: @@
 ===== The Results =====
 The tables below show the Unix ''time'' output of these tests on each machine.  The fastest time for each language is in bold.
 ==== The Sun Box ====
-| ^ PHP ^ Python (non-compiled) ^ Python (compiled) ^ Perl (interpolated string loop) ^ Perl (hard coded regexes) ^
+| ^ PHP ^^^ Python (non-compiled) ^^^ Python (compiled) ^^^ Perl (interpolated string loop) ^^^ Perl (hard coded regexes) ^^^
-^ Test 1 | **real: 9.45s**, user: 9.10s, sys: 0.07s | real: 13.42s, user: 12.66s, sys: 0.11s | real: 7.97s, user: 7.20s, sys: 0.11s | real: 31.89s, **user: 29.43s**, sys: 0.17s | **real: 1.59s**, user: 1.53s, sys: 0.05s |
+| |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |
-^ Test 2 | real: 9.90s, user: 9.06s, **sys: 0.06s** | real: 13.28s, user: 12.45s, sys: 0.15s | real: 7.86s, user: 7.25s, **sys: 0.01s** | real: 31.78s, user: 29.97s, sys: 0.18s | real: 1.67s, user: 1.52s, sys: 0.07s |
+^ Test 1 |  **9.45s**  |  9.10s  |  0.07s  |  13.42s  |  12.66s  |  0.11s  |  7.97s  |  7.20s  |  0.11s  |  31.89s  |  **29.43s**  |  0.17s  |  **1.59s**  |  1.53s  |  0.05s  |
-^ Test 3 | real: 9.58s, user: 9.07s, **sys: 0.06s** | real: 13.56s, user: 12.59s, sys: 0.10s | real: 7.45s, **user: 7.09s**, sys: 0.13s | real: 31.29s, user: 29.61s, **sys: 0.14s** | real: 2.32s, user: 1.46s, **sys: 0.04s** |
+^ Test 2 |  9.90s  |  9.06s  |  **0.06s**  |  13.28s  |  12.45s  |  0.15s  |  7.86s  |  7.25s  |  **0.01s**  |  31.78s  |  29.97s  |  0.18s  |  1.67s  |  1.52s  |  0.07s  |
-^ Test 4 | real: 9.52s, user: 9.08s, sys: 0.10s | real: 13.58s, user: 12.63s, **sys: 0.09s** | **real: 7.40s**, user: 7.18s, sys: 0.04s | **real: 33.19s**, user: 30.27s, sys: 0.15s | real: 1.76s, user: 1.47s, **sys: 0.04s** |
+^ Test 3 |  9.58s  |  9.07s  |  **0.06s**  |  13.56s  |  12.59s  |  0.10s  |  7.45s  |  **7.09s**  |  0.13s  |  31.29s  |  29.61s  |  **0.14s**  |  2.32s  |  1.46s  |  **0.04s**  |
-^ Test 5 | real: 9.94s, **user: 8.87s**, sys: 0.12s | **real: 13.00s**, **user: 12.44s**, sys: 0.12s | real: 7.43s, user: 7.19s, sys: 0.08s | real: 33.22s, user: 30.22s, sys: 0.16s | real: 1.82s, **user: 1.42s**, sys: 0.10s |
+^ Test 4 |  9.52s  |  9.08s  |  0.10s  |  13.58s  |  12.63s  |  **0.09s**  |  **7.40s**  |  7.18s  |  0.04s  |  **33.19s**  |  30.27s  |  0.15s  |  1.76s  |  1.47s  |  **0.04s**  |
+^ Test 5 |  9.94s  |  **8.87s**  |  0.12s  |  **13.00s**  |  **12.44s**  |  0.12s  |  7.43s  |  7.19s  |  0.08s  |  33.22s  |  30.22s  |  0.16s  |  1.82s  |  **1.42s**  |  0.10s  |
+==== The Laptop ====
+| ^ PHP ^^^ Python (non-compiled) ^^^ Python (compiled) ^^^ Perl (interpolated string loop) ^^^ Perl (hard coded regexes) ^^^
+| |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |  **Real**  |  **User**  |  **Sys**  |
+^ Test 1 |  14.25s  |  14.05s  |  0.10s  |  **12.10s**  |  11.98s  |  0.06s  |  **6.12s**  |  6.03s  |  0.07s  |  **42.42s**  |  **42.11s**  |  0.11s  |  1.63s  |  **1.54s**  |  0.08s  |
+^ Test 2 |  14.00s  |  13.62s  |  **0.08s**  |  12.27s  |  **11.91s**  |  **0.04s**  |  6.17s  |  6.08s  |  0.06s  |  43.02s  |  42.72s  |  0.09s  |  1.71s  |  1.64s  |  **0.05s**  |
+^ Test 3 |  14.24s  |  **14.01s**  |  0.14s  |  12.43s  |  12.29s  |  0.06s  |  6.14s  |  **5.98s**  |  0.08s  |  43.15s  |  42.67s  |  0.15s  |  1.71s  |  1.65s  |  0.06s  |
+^ Test 4 |  **13.94s**  |  13.62s  |  **0.08s**  |  12.21s  |  11.93s  |  0.11s  |  6.30s  |  6.22s  |  **0.05s**  |  43.25s  |  43.00s  |  **0.07s**  |  1.63s  |  1.58s  |  **0.05s**  |
+^ Test 5 |  14.30s  |  14.06s  |  0.14s  |  12.24s  |  12.08s  |  0.09s  |  6.32s  |  6.19s  |  0.08s  |  31.89s  |  42.60s  |  0.20s  |  **1.61s**  |  1.55s  |  **0.05s**  |
+===== Just For Fun... =====
+...one of my co-workers whipped up this C code which uses ''libpcre'' just to see how it would perform versus the interpreted languages.  I'm not including it in the main results because this is a test of 3 interpreted languages speed capabilities, but I thought I would drop the results in here just for fun.
-==== The Laptop ====
-| ^ PHP ^ Python (non-compiled) ^ Python (compiled) ^ Perl (interpolated string loop) ^ Perl (hard coded regexes) ^
-^ Test 1 | real: 14.25s, user: 14.05s, sys: 0.10s | **real: 12.10s**, user: 11.98s, sys: 0.06s | **real: 6.12s**, user: 6.03s, sys: 0.07s | **real: 42.42s**, **user: 42.11s**, sys: 0.11s |  real: 1.63s, **user: 1.54s**, sys: 0.08s |
-^ Test 2 | real: 14.00s, user: 13.62s, **sys: 0.08s** | real: 12.27s, **user: 11.91s**, **sys: 0.04s** | real: 6.17s, user: 6.08s, sys: 0.06s | real: 43.02s, user: 42.72s, sys: 0.09s | real: 1.71s, user: 1.64s, **sys: 0.05s** |
-^ Test 3 | real: 14.24s, **user: 14.01s**, sys: 0.14s | real: 12.43s, user: 12.29s, sys: 0.06s | real: 6.14s, **user: 5.98s**, sys: 0.08s | real: 43.15s, user: 42.67s, sys: 0.15s | real: 43.02s, user: 1.65s, sys: 0.06s |
-^ Test 4 | **real: 13.94s**, user: 13.62s, **sys: 0.08s** | real: 12.21s, user: 11.93s, sys: 0.11s | real: 6.30s, user: 6.22s, **sys: 0.05s** | real: 43.25s, user: 43.00s, **sys: 0.07s** | real: 1.63s, user: 1.58s, **sys: 0.05s** |
-^ Test 5 | real: 14.30s, user: 14.06s, sys: 0.14s | real: 12.24s, user: 12.08s, sys: 0.09s | real: 6.32s, user: 6.19s, sys: 0.08s | real: 31.89s, user: 42.60s, sys: 0.20s | **real: 1.61s**, user: 1.55s, **sys: 0.05s** |
+==== The Code ====
+<code c>
+#include <stdio.h>
+#include <string.h>
+#include <err.h>
+#include <pcre.h>
+char *pat[] = {
+    "pop3d-ssl.\\s+LOGIN.*?user=([^,]+).*?ip=.*?[^\\d]((?:\\d{1,3}\\.){3}\\d{1,3
+})" ,
+    "postfix/smtpd.*NOQUEUE.*Client host \\[([^\\]]+)\\].*zen.spamhaus.org" ,
+    "postfix.*connect from ([^\\[]*)\\[([^\\[]+)\\]" ,
+    "postfix.*lost connection" ,
+    "postfix/virtual.*?: ([^:]+): to=<([^>]+)>.*delays=([^,]+),.*status=sent" ,
+    NULL
+};
+const char *logfile = "maillog";
+int counter = 0;
+pcre *re[5];
+int main(void) {
+        const char *err_txt;
+        int i, err_offset, match, ovec[30];
+        FILE *f;
+        char *s, buf[1024];
+        for (i = 0; pat[i]; i++) {
+                re[i] = pcre_compile(pat[i], 0, &err_txt, &err_offset, 0);
+                if (!re[i]) {
+                        errx(1, "PCRE compile error at %d of %s: %s",
+                                err_offset, pat[i], err_txt);
+                }
+        }
+        if (!(f = fopen(logfile, "r"))) return 1;
+        while ((s = fgets(buf, sizeof(buf), f))) {
+                for (i = 0; pat[i]; i++) {
+                        match = pcre_exec(re[i], 0, s, strlen(s),
+, 0, ovec, 30);
+                        if (match > 0) {
+                                counter++;
+                                break;
+                        }
+                }
+        }
+        fclose(f);
+        printf("Number of matches: %d\n", counter);
+        return 0;
+}
+</code>
+The compile command:
+<code>
+$ cc -Wall -o pcretest pcretest.c -I/usr/local/include -L/usr/local/lib -lpcre
+</code>
+==== The Results ====
+=== The Sun Box ===
+| ^ Real ^ User ^ Sys ^
+^ Test 1 |  **6.70s**  |  **6.44s**  |  0.07s  |
+^ Test 2 |  8.03s  |  7.83s  |  **0.03s**  |
+^ Test 3 |  9.04s  |  6.84s  |  0.05s  |
+^ Test 4 |  9.03s  |  6.53s  |  0.09s  |
+^ Test 5 |  7.26s  |  6.63s  |  0.08s  |
+=== The Laptop ===
+| ^ Real ^ User ^ Sys ^
+^ Test 1 |  13.14s  |  12.92s  |  0.06s  |
+^ Test 2 |  13.08s  |  **12.88s**  |  0.06s  |
+^ Test 3 |  13.09s  |  12.94s  |  **0.02s**  |
+^ Test 4 |  13.21s  |  13.00s  |  0.07s  |
+^ Test 5 |  **13.07s**  |  **12.88s**  |  0.04s  |
 ===== Conclusion =====
@@ Line 372: / Line 462: @@
 I think that the most amazing thing here is difference in the 2 Perl tests.  If you use a scalar string variable as the regular expression, it's dog slow.  However, if you hard code that string in the expression, it's lightning fast.  I was not expecting this kind of a discrepancy at all, but I'm glad that I tested both approaches.
+Though I didn't include it in the //official// results, I thought it was kind of interesting that the compiled C program performed about the same as the Python program with pre-compiled regular expressions.
 I think the conclusion that I have to draw from this experiment is that Perl is your best choice, as is often the case, for a simple static regular expression based parser.  On the other hand, if you wanted a more dynamic approach to the regular expressions that you are using (like loading them in from a file, command-line, etc.), compiled Python is definitely your best answer, but PHP is also a good candidate.  It's pretty obvious that Perl is not the language to use in that particular case.
 Please, feel free to post to the discussion here in answer to this writeup.