Performance improvement projects

Start smart: measure before you run

According to the pyramid of S.P. Anderson, which is inspired on Maslows hierarchy of needs, a bad performance is a dissatisfier for usability. The rest of the pyramid doesn’t make sense without a good performance (next to functional correctness). And although often underestimated, in every large or small organization at a certain point somebody decides it is time to spent some energy, resources and money to optimize the site for speed. They estimated the ROI of a speed optimization project and realized the investment will pay off. Good! A speed project team is put together to improve the loading time.

Now how to approach such a speed project? What are the requirements to pick up? Where to start? What is actually considered to be the performance of a web site? There is only one group who can define this for a certain website: the visitors of that website. Therefore I like to work with the ‘Perceived performance from a user perspective’ as the base for the performance tweaking projects to work on.

Perceived performance is the time it takes on a page in order to start the action the user wants to do. This means that if a user wants to search for a product, the search box and submit buttons are driving the measurements for the performance (instead of the marketing campaign banner that may also be located on the page). This definition helps in starting the performance improvement project. Measurements on performance should be focussed on the user experience for performing actual tasks and may go along with user satisfaction research in order to check if the perceived performance improved during the project.

Creating a web site is a multi discipline activity involving marketing, user experience, design, technical architecture, development and infrastructure. Performance improvement projects are slightly different to the creation of the web site. As with the creation of the web site, you need all disciplines to be involved. Performance improvement projects can focus just on the improvement of the perceived performance. The performance improvement team defines the goals of the project by defining detailed user experience scenarios and the performance goals per step.
You also need to make sure everyone on the team agrees on the way of measuring the performance from a user perspective in an end-to-end scenario. Each step in the scenario should be logged with performance information. I like to work with automated tools like Selenium, but you could use a manual testing approach if automation is too hard.

The results of the measurements should be continuously visible to the team. Try to sit in the same room and post the results on the wall by a beamer, a TV set or print them. If this isn’t possible, setup a small web site with the results and mail the results to the team on a regular base.

As soon as you start analyzing the results of the tests with the team, you will likely find one or more steps that need further investigation since they take too much time. Those steps need more detailed measurements for further investigation/ analysis.

We started with the end-user performance measurements. Next we can interface directly to the web server (on the web server system) using the same scenario as we did before. If the performance is similar to the end-to-end measurements, we re-use the same approach on the application server, the database, etc. Make sure you have the (technical) domain experts available when you focus on their domain. They should be able to indicate whether a performance is ‘as expected’ or can be improved drastically. If one of the tests on the server itself is drastically faster, there is something wrong with the infrastructure configuration (switches, load balancers, ssl offloaders, etc) and you need your infrastructure experts to join as well.

I use a general rule in working with teams. If the project fails, the team failed. If the project is a success, it is the team who created the success. Make sure you don’t start a blame game as soon as you find a cause of bad performance, but report the team success instead.

About the author:
Richard den Adel has worked on many web site improvement projects as a technical facilitator. Richard is employed by QNH in the Netherlands. QNH is specialised in designing and developing web sites using agile development methodology with a strong focus on performance. To support this way of working, it developed its own open source tool to test these web sites on functionality and performance.



No comments

Leave a comment

optional