Archive for the ‘hack’ Category

JavaScript Task Queue – Hack it #9

Sunday, July 17th, 2011

Suppose you have a lot of tasks with completion callbacks to complete in the browser, but you want to do them sequentially. For instance, you might have a list of AJAX requests to make to your server.

This JavaScript demonstrates a simple solution:

If you can’t see the source code, look here.

Ultra Simple PHP Profiling – Hack it #8

Tuesday, November 9th, 2010

We wanted to profile a few PHP scripts, so we tried xdebug, but there were a few issues:

  • Not suitable for a production server
  • Difficult to turn on/off for specific scripts and requests
  • Difficult to profile MySQL queries, without also attracting a mass of other data

xdebug is better for offline application profiling.

If you just want to track down a bottleneck, or see what your script is spending all its time doing, here’s a simple alternative.

To profile your SQL, replace mysql_query with mysql_queryx.

This generates a CSV file at /tmp/profile.1 for easy analysis.

Now create the file profile.php:

Too easy!

To stop profiling a page, set $profile = false.

If you’re concerned about performance, do something like $profile = rand(0, 100) > 99.

py2exe, python 2.6 and wxPython – hack it #7

Tuesday, July 6th, 2010

Python
If you’ve tried to build a client application with Python 2.6 and wxPython, you might hit a problem.

Python 2.6 no longer includes the Visual Studio dependencies required by wxPython. These dependencies may not be present, either on your build machine, or the target user’s machine.

When building with py2exe, you may encounter an error similar to:

error: MSVCP90.dll: No such file or directory

The solution!

To successfully build your executable, you need the Visual Studio 2008 redistributable package, which can be obtained from Microsoft. Run this on your build machine to install the appropriate DLLs.

If your build still fails, you may need to copy the DLLs into your build directory. Locate msvcp90.dll, msvcm90.dll and msvcr90.dll on your machine and copy them to the build directory.

Once you have a successful Python build, you also want your executable to run on the target machine. To do this, your install script needs to install the vcredist package as part of the installation.

The command you need to run as part of your client installation is:

vcredist_x86.exe /q:a

If you are using NSIS as your installer, the appropriate code is:

File /r vcredist_x86.exe ;to copy the file
ExecWait ‘”$INSTDIR\vcredist_x86.exe” /q:a”‘ ;to install

With the Visual Studio redistributable installed, it will now execute successfully on the target machine. Hurrah!

Remember me with PHP – hack it #6

Monday, March 15th, 2010

PHP persistent sessions
“Remember me” functionality was recently implemented on the Supernifty login screens. The PHP documentation is surprisingly non-obvious so here’s how to do it.

If you’re already using the $_SESSION variable, the default is for this variable to live only for as long as the browser. Once the user closes the browser, the cookie will be deleted, and they’ll be logged out.

To remember a user across browser sessions, here’s what needs to be done.

Tell the browser to remember the session

You do this with the line:

setcookie( session_name(), session_id(), time() + 86400*30 );

This overwrites the default session cookie, with a cookie that will persist on the user’s browser for one month.

Note that since cookies are set in the headers sent to the browser, this needs to be done before writing any content to the browser.

With HTML like

<input type="checkbox" name="remember" value="remember"/>

You could have a handler like:

$remember = $_POST['remember'];
if ( isset( $remember ) && $remember == 'remember' ) {
setcookie( session_name(), session_id(), time() + 86400*30 );
}

Tell the server to remember the session

Bizarrely, PHP’s default session time is something ridiculous like 24 minutes.

To tell the server to remember the session for longer, you need to set some PHP environment variables, like so:

ini_set("session.gc_maxlifetime","2592000"); // 24*30 hours
session_set_cookie_params( 86400*30 );
session_start();

More details about PHP’s session lifetime

With these two pieces in place, you can now implement a “remember me” checkbox on your login screen with PHP, while continuing to use the $_SESSION variable.

Paralympics 2008 – where do our paralympians come from?

Saturday, September 13th, 2008

paralympic medal map

How great are our paralympic athletes? Their efforts at the Beijing Paralympics have been incredible. Their stories, hardships and sacrifices have been at times heartbreaking, but more often inspirational.

In short, it’s been very good.

Further to Supernifty’s map of Olympics medallists, Supernifty is proud to announce a similar deal for the Paralympics medallists.

Using this map, you can see where all our local heroes hail from. If you’re considering setting up a tickertape parade, rock concert or air-force flyover in their honour, this is a chance to check that you are in the right town.

OK, so that last paragraph was sheer silliness.

But you should check out where our Beijing Paralympics medallists come from.

The map will be updated daily as more medals are won, until the conclusion of the games, so feel free to check it frequently.

Backup multiple MySQL databases with Python – hack it #5

Wednesday, May 28th, 2008

Keep copies, or risk the great sadness

Backing up is important.

If you host a website which has a database, and you’re not backing it up, you should.

If your database was wiped, and you lost your entire blog, wiki, customer details, and the rest, if you’re like me, you’d be deeply unhappy.

Most webhosting companies do backups, but are somewhat vague on guarantees and process. They often charge to do restores as well.

Best bet is to do it yourself.

This set of instructions applies to MySQL, and can be applied to one database or many.

Step 1. Backup your databases on the server
Below is a python script that will backup multiple MySQL databases. Put this on your web server, preferably in a directory called backup, with the name backup.py.

import os
 
databases = dict()
databases['*** db1 ***'] = '*** password1 ***'
databases['*** db2 ***'] = '*** password2 ***'
databases['*** db3 ***'] = '*** password3 ***'
 
def set_pass( name ):
 file = open( '/*** your home directory ***/.my.cnf', 'w' )
 file.writelines( ( '[client]\n', 'password=%s\n' % databases[name] ) )
 file.flush()
 file.close
 
for key in databases.keys():
 set_pass( key )
 os.system( "mysqldump --user %s --opt %s > %s.sql" % ( key, key, key ) )
 
os.system( "zip backup.zip *.sql" )
os.system( "rm *.sql" )
os.system( "chmod 600 backup.zip" )

About this script:

  • A problem with backing up multiple databases without intervention is the entering of passwords in a secure way. This script uses .my.cnf to set the password securely.
  • Fill in the database usernames and passwords. This script assumes the database has the same name as the username.
  • Fill in your home directory e.g. /home/john
  • The result of this script is backup.zip, in the current directory.

Step 2. Copy backup to your machine
This script connects to your web server, runs the above backup script, then copies the result back to your local machine. Put this on your local machine.

ssh -i "your_key" you@yourhost.com "cd backup; python backup.py"
rsync -avz --rsh="ssh -i your_key" you@yourhost.com:~/backup/ backup

About this script:

  • This script assumes the backup script is in the “backup” directory on your web server.
  • You need ssh and rsync to use this script. On Windows, PuTTY provides a great ssh client, but I don’t know of a good rsync implementation, other than cygwin. Instead of rsync, you could use scp, which comes with PuTTY.
  • If you want to automate this script (see later), you’ll want passwordless ssh, which means setting up “your_key”. Google on how to set this up.

Step 3. Repeat periodically
You want to automate this process so you can forget all about it.

  • Unix: crontab. If you’re unfamiliar with crontab, try “man crontab”. The main commands to use are “crontab -e” and “crontab -l”.
  • Windows: scheduler. Check out Control Panel->Scheduled Tasks, and add your client script.
  • As mentioned above, you need passwordless ssh so your scheduled task can run unattended.

There’s your nutshell guide to backing up multiple MySQL databases with Python.

Backups – Hack It #4

Wednesday, May 21st, 2008

Keep copies
Backups are an essential feature of any IT installation.

You’ve got to have backups. Not backing up is a ridiculous, unnecessary risk.

The average hard drive has a 20% chance of failure each year. How would you fare if your computer was stolen, or melted in a fire? If you don’t have a plan for data recovery, you are asking for trouble.

It shouldn't happen to you!

Try to put a value on losing everything on your hard drive, and remember that fires, robberies and hardware failure are all out of your control.

For instance, a friend recently lost all his photos. Over 10 years worth of memories, gone forever! Man, he was sad.

These days there are great, inexpensive options for backups.

Option 1: buy an external hard drive.
This is a big improvement on not backing up at all, however, there’s a problem. If you keep the external drive with your computer, you’re not covering all the bases.

Any localized event, such as a natural disaster or robbery, will render your backup solution useless.

If you copy your data off your computer, then take your external hard drive away to another location, then this is a fine solution. Otherwise, it’s not recommended.

Option 2: online backup
In recent times, some very competitive options have sprung up. This stores your data safely away from your computer, so this is a winner – you can’t lose both in the one event. The main factor in my opinion is security.

  • Mozy – unlimited storage for $4.95/month.
  • Carbonite – unlimited storage for $49.95/year.
  • Amazon S3 – incremental storage of 15c/GB/month. This is not on its own a backup solution. You need a user-friendly client to connect you to their backup service. For instance, JungleDisk.

I chose S3 and JungleDisk, primarily for security reasons. Most solutions tell you that your data is safe and encrypted, but that they can recover your data if you lose your password.

This means that if the company is hacked, or if you don’t trust company employees, your data is vulnerable. JungleDisk has the option to encrypt your data before it leaves your computer, and cannot be decrypted without your password.

It’s worth pointing out, that with Amazon’s incremental pricing, if you only need to backup a few gigs, then you’ll be paying less than $1/month to keep your data safe.

It’s a tiny price to pay for the guarantee that your data will not be lost.

PHP sessions on shared hosting – Hack It! #3

Saturday, April 19th, 2008

PHP

If you use sessions with PHP on shared hosting, there are a couple of potential issues:

  • You might be vulnerable to session hijacking. PHP stores sessions as files. Where are they being stored? If they are stored in a publicly readable directory such as /tmp (the default), then other users of your shared host can hijack any session from your website. The session ID is part of the filename, so an attacker can build his own cookie that will be authenticated by your website.
  • You don’t have control over session timeouts. The default session timeout is 24 minutes. From the php manual: “If different scripts … share the same place for storing the session data then the script with the minimum value will [determine the session timeout]”.

So… by default, sessions timeout after 24 minutes, which I think many users would find pretty irritating, and you are vulnerable to session hijacking from anyone having access to your shared server.

Not good!

The solution. Here is one way to solve both of these potential issues. Whenever your code starts handling a session, include the following PHP:

ini_set("session.gc_maxlifetime","21600"); // 6 hours
ini_set("session.save_path", "/your_home/your_sessions/");
session_start();

The first setting is the timeout in seconds, so for each hour of session survival, add 3600.
The second setting is the path to save session files to. Change this to a real directory in your home directory. Note that the first setting will not work unless you set this. Also note that the process running PHP needs to be able to write to this directory.

With this hack you can increase the security of your PHP sessions, and have them timeout over a period appropriate to your website. Hurrah!

Hack It #2 – Convert any Windows application to a service

Monday, November 5th, 2007

I recently had to convert a C++ application to run as a service, so that it could always run in the background. This application was supposed to already be capable of doing this, but instead would crash instantly.

Rather than going into the code and figuring out why that would be, I decided that someone would have written a wrapper to convert any application into a service. It turns out there is such a wrapper, and it’s called srvany.exe.

Here is how to convert any Windows application to run as a service.

By the way, if your application is Java based, there is a specific Java Service Wrapper so you probably should go there instead of reading these instructions. This is for generic executable or batch files.

Download and install the NT Resource Kit

The files you are going to need from this:

  • srvany.exe
  • instsrv.exe

Install the srvany service

  • Run the command: instsrv service_name [path]\srvany.exe, where service_name is the name of your service

Configure the new service

  • Run regedit
  • Browse to HKEY_LOCAL_MACHINE\ SYSTEM\ CurrentControlSet\ Services\ service_name
  • Add key “Parameters”
  • Right click on “Parameters”
  • New string value, “Application”, with the value as the executable you want to run, including any parameters.
  • If your application is not an executable, bat a batch file, set the value to “C:\Windows\system32\cmd.exe /k full_path_to_batch_file.bat”
  • Of course the batch file or executable can’t exit, as this will end the service.

And some more configuration

  • Control Panel, Administration, Services, find your service.
  • The default is “Automatic”, which means as long as nothing goes wrong, your service will always be running. You might want to change this.
  • You might want to set the service to run as something other than “Local Service”, e.g. have it run under your name.

Test It

  • Start the service
  • Your application should appear in task manager

Celebrate

Now you can have something run all the time, behind the scenes, as a service. Hurrah!

Hack It! #1 – Change the TortoiseSVN SSH port

Thursday, September 27th, 2007

I used to have a boss who was particularly fond of the ‘quick hack’. So much so that many of us used to cough “Hack It!” whenever he was around. We found this endlessly amusing. This segment is dedicated to him.

TortoiseSVN is really great. Best source control client ever. However, the svn+ssh protocol doesn’t support a non-standard SSH port. e.g. svn+ssh://me@myserver.com:9022/. That’s a bummer, since my employer just changed the SSH port.

In order to Hack It! (TM), just go to TortoiseSVN->Settings->Network and set the SSH client to

C:\Program Files\TortoiseSVN\bin\TortoisePlink.exe -P 9022

Winner.

This is a hack because if you have other repositories they will also start connecting to this funky port. Non-hack suggestions welcome.