KitGuru Photoshop Benchmark – V1(4)

1. KitGuru Photoshop Benchmark2. Photoshop CS5 - whats new?3. Benchmarking Preparations4. Benchmark Results5. View All Pages

For as long as I can remember enthusiasts around the world have been keen to see how their systems perform with many commercially available benchmarks from such companies as Futuremark. While these applications are extremely useful to a wide audience sometimes it can be helpful to focus on a ‘real world' application which many of us use on an almost daily basis.

When I was in charge of DriverHeaven I wrote a scripted benchmark which was not only used to help users ascertain system specific performance levels but was useful in reviews to accurately measure performance in key areas. Leading tech sites such as Madshrimps, Benchmark Reviews and Hardocp used my benchmark in their reviews also.

Today I am releasing the newest version of my benchmark which is entitled V1(4), because it is the first version for Kitguru.net but actually the fourth iteration I have written for the public domain.

This script is fully compatible with both Macintosh and Windows platforms and is stored in a user friendly zip format. Before you jump into the benchmarking however i would like to detail some information on the following pages which is important to read, and I will also give additional information on what is going on behind the scenes.

This version of the script has been thoroughly tested on Adobe Photoshop CS5 and CS4 and is compatible with Photoshop 7. It is important to document however that different versions of Photoshop give different end results. Adobe have been fine tuning the program over the years and enhancing specific filters and algorithms for better performance (some filters were rewritten during the CS2 time period). This means ultimately that comparing times with users running other versions of Photoshop is not going to give consistent results.

To remain fully ‘in the now' most of the testing and screenshots used in this article for comparison purposes were taken from CS5 on both Macintosh Snow Leopard and Windows 7 operating systems.

Before a new version of Photoshop is released there are normally months of mass hysteria as to what cool new features have made the grade, and while there have been many additions to the application the main focus has been on the Content Awareness system – I won't be delving into that today but if you are curious check out this video which gives a good overview of its capabilities.

The big news for me and many other industry users is the fact that Photoshop CS5 on the Macintosh is finally a true 64 bit application. It was a bugbear that regardless of how much ram was installed in your Macintosh that the application could only use the first 3GB. This is rather hard to believe as Windows has had a fully 64bit compliant version for years but Adobe have only recently adopted Cocoa APIs. Not only is more ram accessible by the program, but most modern processors will be fully utilised – I recorded 10-15% speed increases when comparing the 64bit CS4 exe against the 32bit version, this is nothing to sniff at, especially when working with huge multilayered files.

What many people do not know however is that Apple have enforced by default a 32bit set of instructions within OSX for ‘compatibility' reasons. This has a slightly detrimental effect on Photoshop performance and until recently, changing this to 64bit was a laborious process. Thankfully a nice bloke spent some time coding a little application which switches back and forward – snag it over here.

I won't bore you by detailing EFI and Kernel information, if you follow that link I just posted all the information is over there. Please be aware that if some low priority driver stops working due to forcing on 64bit mode then it is easy to switch back to 32 bit. In all my testing I never had an issue running in 64bit mode and I have a wide variety of devices attached to my Macintosh systems – so I have left it on.

If you want to check which mode you are running in. go to the Apple Menu via the Finder, select About This Mac then More Info, and scroll to the software header (see image above).

Windows users basically end up with a faster and more powerful version of CS4 with all the new treats – there is little to report here but if you are interested in learning about the new features in CS5, head to the Adobe site over here and download the free trial. Their interface will detect the platform you are using and offer the correct application suites.

Get the Benchmark for yourself over here

First point to make (and its an important one) is that the Kitguru Photoshop Benchmark V1(4) is going to need at least 1GB of ram allocated to Photoshop for the results to be indicative of CPU and system performance – the image file used throughout testing is 10,000 pixels wide and I would recommend 2GB of ram or more to basically eliminate ‘scratch' disk accessing (hard drive ‘virtual memory'). If you are running your stratch disk from an SSD this will not be such an issue, however memory is still considerably faster than any drive mechanism available. My rule of thumb is to open Photoshop fresh with every run to counter any caching algorithms which will slowly over time improve the results – in the real world this is a good thing, but you will find it hard to reproduce figures if you keep the program open and re-run the script repeatedly.

Accessing the performance settings to change the memory accessible by Photoshop is achieved via the PHOTOSHOP menu within CS5. Older versions of Photoshop sometimes have this setting within the EDIT menu. Poke around if you can't find it, its pretty easy to spot. Once you are in this far then scroll down to the performance submenu and select it.

Within this performance panel there is a slider which details memory usage. This shows total ram available to Photoshop and you can adjust the settings to suit your desires. Photoshop normally offers a reasonable ‘best guess' which you can leave alone, however if you are frequently working with large files (like the one on test in our benchmark) then you might want to adjust it to access more of your available memory. 2GB is a sweet spot for Photoshop, however if all you do is work with smallish 72dpi web graphics then you most likely wont need to increase it over the default. For those interested it is worth pointing out that Photoshop doesn't immediately suck all the memory you have requested into the application – it only takes the memory it needs to the maximum draw point you (or the program) selected.

As many of you will know there has been much talk of the graphics acceleration adopted within CS4 and CS5 however for the purpose of our system benchmarking today it makes no difference to the results. The Graphics settings are more based on interface enhancements with the graphics card handling redraw via OpenGL Acceleration. Photoshop CS4 and CS5 leverages the graphics display card's GPU, instead of the computers main processor to speed its screen redraw. For Photoshop to access the GPU, your display card much contain a GPU which supports OpenGL and has enough RAM to support Photoshop functions – at least 128MB of RAM and a display driver that supports OpenGL 2.0 and Shader Model 3.0. For the purposes of our performance benchmark it makes no difference if this is turned on or off, so whichever setting you already have (or prefer) there is really no need to adopt a change.

Once these settings are changed (if they are), then accept and quit the application and reload for the new settings to take hold. With every script run it is worth reinforcing the rule to quit and reload the program to ensure no caching is taking place from prior runs.

The next stage is to download this file, extract it and either directly double click on the actions file or load the actions palette within Photoshop. If you have two versions of Photoshop installed and you double click the actions file, it may very well open and load into an older version of Photoshop if it takes precedence in your operating system. Once the actions palette is opened via the WINDOW menu then the action needs to be loaded.

Once you have the script loaded, then you need to ensure that the actions palette is forced into ‘button mode'.

It is important that you toggle on ‘Button Mode' (indicated by the check tick to the left) otherwise you will not be able to run the script.

Open the image in the folder entitled ‘draconians' – this is an image I rendered years ago for a game company as a pre-production guideline, its 10,000 pixels wide and expands to 228.9MB when opened. This is a very intensive test for even modern day high specification enthusiast systems – older or weaker systems will likely choke on this, so if you are running it on an ATOM powered netbook, then be prepared for a considerable wait to get a final benchmark figure!

Finally, once the script is set up in button mode the last step is to set Photoshop to use “Timing Mode”, this is accessible via a navigation button on the bottom left of an open image. Once this is set, it remains this way until changed again, even on application reloads. You will have noticed by opening the image that the timing already shows a figure, this is basically the time it took to open the image. This counter resets after every action, so there is no need to adjust anything for the benchmark.

Get the Benchmark for yourself over here

The KitGuru Photoshop Benchmark V1 (4) comprises the following tests:

1. Texturiser Test (1)
A canvas filter with scaling set to 112 and relief 10. Light direction, bottom without invert texture.

2. CMYK Colour Conversion
Important for all the professionals as this 4 plate (cyan, magenta, yellow and black (k-key colour)) colour mode is used in graphics bureaus and printing presses for colour reproduction to newsprint or magazine. Digital photographs will be converted to this mode for final output.

3. RGB Colour Conversion
Important test for web designers as this colour mode is used for webpages and will be the mode of choice for digital images raw from the camera.

4. Ink Outlines
A relatively demanding test which applies a brush stroke filter to the image. Stroke Length: 6. Dark Intensity: 20. Light Intensity: 11.

5. Dust and Stratches
A filter which is used quite widely by professionals scanning less than perfect original flat copy. Radius is set to 3 and Threshold is set to 36.

6. Watercolor
An artistic filter to create a realistic watercolor image. Brush Detail 11. Shadow Intensity: 3. Texture: 1

7. Texturiser Test (2)
A canvas filter follow up to the first with a variation of settings. Scaling: 67. Relief 9. Light Direction, top right with invert texture applied.

8. Stained Glass
A very intensive filter which seen recent optimisations in CS2/3 and again in CS5. Cell size: 6. Border Thickness: 3 and Light Intensity: 4.

9. Mosiac Tiles
Makes use of memory bandwidth. Tile size 5. Grout Width: 2 and lighten Grout 9

10. Extrude
One of the most intensive filters and a test of CPU core efficiency and memory bandwidth – the latest Core I7 processors put in a great set of results with this filter algorithm. Size: 12. Depth: 6. Without solid front faces. Without mask incomplete blocks. Type: Blocks

11. Rough Pastels
A test which gives the image a canvas texture like appearance and tests CPU. Effect: Rough Pastels. Stroke Length: 12. Stroke Detail: 7. Texture Type: Canvas. Scaling: 129. Relief: 29. Light Direction: Bottom. With Invert Texture.

12. Smart Blur
A very intensive filter which puts an emphasis on overall system FSB, Cpu and Memory performance. If you are lucky enough to own a Core i7 CPU, this annihilates all contenders. Radius: 20.7. Threshold: 28.7. Quality high and mode normal.

13. Underpainting
Brush size 2, Texture coverage 23. Texture type: Canvas. Scaling 100, Relief 6. Light Direction top with invert texture.

14. Mosiac Tiles
Makes use of memory bandwidth. Tile size 5. Grouth Width: 2. and Lighten Grout 9

15. Spherize
A useful filter which is used to create many interest effects on objects. Amount 100, Mode: Normal

16. Palette Knife
Stroke size 23, Stroke Detail 3 and softness 0

17. Sponge
Brush size 2. Definition 12 and Smoothness 5

18. Smudge Stick
Stroke Length 4. Highlight Area 7. Intensity 9.

As seen from the test suite, there is a good cross section of intensive, real world filters which stress any modern system. Unfortunately CS5 filters have not been coded to take full advantage of 4 or 8 core systems. Therefore much of the performance benefits will relate to core efficiency and clock speed. In fact many of the results can be improved by increasing system FSB, ram speeds or tightening the timings. I had hoped that Adobe might adopt a new way of thinking with CS5, however they have basically just repeated the CPU structure they used in CS4. If you have a sexy Nehalem 16 core system then most of your cores will sit idle throughout, which is quite disappointing. Adobe need to allow users to toggle how many cores they want Photoshop to use, but I feel my cries are falling on deaf ears.

Here are some results from testing on several systems:

Macintosh MacBook Pro 17 inch Rev 5.2. 2.93ghz Core 2 Duo CPU, 9600GT M, 8 GB of DDR3 memory and SSD Drive. (lower scores are obviously better). Snow Leopard 10.6.3 (64 bit Kernel enabled).

1: Texturiser (1) – CS5 2.7 seconds / CS4 3.1 seconds
2: CMYK – CS5 10.3 seconds / CS4 10.4 seconds
3: RGB  – CS5 10.4 seconds / CS4 10.9 seconds
4: Ink Outlines – CS5 45.8 seconds / CS4 53.4 seconds
5: Dust and Stratches CS5 7.3 seconds / CS4 7.9 seconds
6: Watercolor CS5 28.7 seconds / CS4 33.6 seconds
7: Texturiser CS5 2.2 seconds / CS4 3.8 seconds
8: Stained Glass CS5 30.9 seconds / CS4 39.5 seconds
9: Mosiac Tiles CS5 18.8 seconds / CS4 32.6 seconds
10: Extrude CS5 319.3 seconds / CS4 348.2 seconds
11: Rough Pastels CS5 16.3 seconds / CS4 32.1 seconds
12: Smart Blur CS5 163.4 seconds / CS4 252.0 seconds
13: Underpainting CS5 40.6 seconds / CS4 72.6 seconds
14: Mosiac Tiles CS5 23.2 seconds / CS4 31.1 seconds
15: Spherize CS5 5.3seconds / CS4 5.7 seconds
16: Palette Knife CS5 52.7 seconds / CS4 53.5 seconds
17: Sponge CS5 46.2 seconds / CS4 47.8 seconds
18: Smudge Stick CS5 12.4 seconds / CS4 15.3 seconds
Total CS5 836.65 seconds / CS4 1053.1 seconds

Intel Core 920 watercooled @ 4ghz. 6GB 1600mhz Corsair DDR3 1T, 1TB 500GBx2 HD's. Nvidia 285 GTX. Windows 7 64 bit – 64 bit EXE's.

1: Texturiser (1) – CS5 1.8 seconds / CS4 1.9 seconds
2: CMYK – CS5 1.7 seconds / CS4 1.9 seconds
3: RGB  – CS5 1.6 seconds / CS4 1.6 seconds
4: Ink Outlines – CS5 29.9 seconds / CS4 31.9 seconds
5: Dust and Stratches CS5 2.8 seconds / CS4 4.1 seconds
6: Watercolor CS5 26.7 seconds / CS4 28.8 seconds
7: Texturiser CS5 1.8 seconds / CS4 2.1 seconds
8: Stained Glass CS5 23.3 seconds / CS4 25.7 seconds
9: Mosiac Tiles CS5 13.5 seconds / CS4 15.1 seconds
10: Extrude CS5 151.9 seconds / CS4 153.3 seconds
11: Rough Pastels CS5 12.2 seconds / CS4 14.1 seconds
12: Smart Blur CS5 95.6 seconds / CS4 102.2 seconds
13: Underpainting CS5 35.4 seconds / CS4 41.3 seconds
14: Mosiac Tiles CS5 13.7 seconds / CS4 15.8 seconds
15: Spherize CS5 2.6 seconds / CS4 3.3 seconds
16: Palette Knife CS5 22.7 seconds / CS4 22.9 seconds
17: Sponge CS5 40.2 seconds / CS4 43.8 seconds
18: Smudge Stick CS5 12.4 seconds / CS4 15.3 seconds

Total CS5 486.9 seconds / CS4 525.1 seconds

The results indicate that the move to 64 bit Cocoa on Macintosh OSx reaps huge rewards as would be expected and that the move to CS5 on Windows gives increases, but with figures more in line with an updated version rather than anything revolutionary. The Core i7 in the Windows 7 system certainly shows the older range of processors a clean pair of heels and I hope to rerun these tests shortly on a new Core i7 Macintosh system.

Please feel free to discuss this and share your findings on our forums, your results and interest in this will help us to document and improve the script in the future.

UPDATE: We are working on a database system so people can enter and compare results, check back later.

Become a Patron!