WordPress Frameworks Compared on Performance

50 Comments

The Goal

There are some great posts out there that compare WordPress frameworks. My recent favorites have been by Clifford Paulick. His part A and part B are the hallmark of robust comparisons. In a conversation with him a few months ago, I wondered aloud about a comparison of their performance. I’ve seen WordPress Frameworks compared by tons of factors including features and price like Clifford did, but I’ve never seen them looked at from a performance perspective.

And you can see why. You’d first have to buy them all, which most folks wouldn’t do. Then you have to design a single layout that all could do – which means you’d have to know how they all work. And then you’d run your performance tests.

But if you did such a thing, you could let people know that – all things being equal – which theme frameworks performed better than others. And that might, just maybe, be interesting to someone who was looking to purchase just one and get to know it well.

The Competition

Before I get into methodology, let me give you some context on which theme frameworks were compared.

  1. Numer of theme frameworks: 7
  2. All personally purchased – no freebies to influence my testing.
  3. One xml export from ChrisLema.com – 157 posts, 583 comments, 33 pages, 6 categories.
  4. One standard blog layout with a custom menu, and three elements in the sidebar (About me, Twitter feed, Tag list)
  5. 10 posts on home page
The theme frameworks compared were:

Comparison Methodology

What I did was jump over to my playground (wpadvisor.com) where I do all my testing and I created seven sites on that multisite site. So each site was stand alone. And in each site, I imported the same file. I even imported images so they’d be local. After the import, I deleted the initial post and page, and configured the menu to show the core set of categories. From there I configured the sidebar to have the widgets that I mentioned above. Each theme had different requirements to get all that configured, but none of them took longer then 20-30 minutes to get everything done.

Then I hit each site multiple times. Because I wanted to make sure none of these sites would do their first load during the test.

At this point I should mention that no caching plugins were installed because I host my playground on WP Engine where things move fast without them. I should also note that the page speeds here may not match your own experiences if you’re hosting your site on crap solutions by companies rhyming with MoBaddy.

At that point, I took the home page urls and feed them to the Tools page at Pingdom.

I captured the data and pulled it into spreadsheets for comparison. So here’s the info you’ve been waiting for.

Comparing Request Counts

Framework
# of Requests
Headway12
iThemes Builder15
Genesis19
Pagelines21
Catalyst / Dynamik22
WooThemes (Canvas)25
Ultimatum28

As you can see, the spread was wide. I would have guessed that it was much tighter together, but that wasn’t the case. That said, it only matters in my mind if there’s a relationship between the count and the load time. Which is why you’ll see how some of the larger request counts still did well in terms of speed to load.

Comparing Load Times

Framework
Load Time
Headway226 ms
Genesis964 ms
WooThemes (Canvas)992 ms
Catalyst (w/Dynamik)1.2 s
Ultimatum1.22 s
iThemes (Builder)1.26 s
Pagelines5.18 s

Headway blew the others away – which may have been helped by how few requests it made. But WooThemes loaded up quickly despite it’s request count. Falling to the bottom was Pagelines, but that wasn’t surprising when you see the next table (looking at page sizes).

Comparing Page Sizes

Framework
Page Size
iThemes (Builder)299 kb
Headway1.1 mb
Genesis1.1 mb
Catalyst (w/Dynamik)1.6 mb
WooThemes (Canvas)1.7 mb
Ultimatum1.8 mb
Pagelines5.2 mb

iThemes and Headway jumped in front, but many of the others were right behind them. Except Pagelines, which was a bummer.

Looking at Pingdom’s Performance Grade

Framework
Pingdom Perf Grade
iThemes (Builder)100
Headway98
Pagelines96
WooThemes (Canvas)94
Ultimatum91
Catalyst (w/Dynamik)90
Genesis84

Any time you get a combined score from anyone, like Pingdom, it’s worth digging into how they’re creating their algorithm. I confess, I didn’t do the digging. I may later. But that’s mostly because all of the themes had a nice high score. I’ve seen sites that have far lower than 84 in the total Performance grade.

One other thing to note is that many of the factors that affected the performance grade could have been solved by minifying static files, combining them, and using a CDN. But I wanted a straight comparison, so I left things native.

Conclusion

As always, your selection of a “winner” is based on what you value most. Not every theme here was a drag and drop framework – and maybe you care about that more than anything. Some are easier to learn than others.

In terms of pure performance, Headway seemed to run away with things with iThemes in second place. Genesis & WooThemes were right behind, when I created an amalgamated score.

What does that tell us? Well one interpretation would go like this:

The players that have been in the game longest have had the time to really optimize their frameworks for performance.

You might have a different take. That’s on you.

About This Site

This site is hosted by Pagely - running on Amazon's hardware, making it lightning fast. They're a managed WordPress host which means they take care of my site so I don't have to. Starting at $24/month, this may be the perfect solution for you too.
  • http://twitter.com/raisononline Raison (@raisononline)

    Fantastic article. 5.2mb’s for Paglines – wow, what a whopper. Always a danger that sites become bloated, so good to start small. Great to see the speed differences too – thanks for doing the leg work for us. Going to stick with Canvas for the foreseeable future.

  • http://ka2id.com Kekeli

    I’m an Builder gal myself, started using it a year ago with very little experience with WordPress and php and it has really helped cut down the time spent learning how to design WP sites. I just found your blog through twitter and I really love the clean simplicity of it, I’m curious, what do you use for your site?

    • http://chrislema.com chrislema

      I switch between multiple frameworks but with roughly the same design. Right now I’m using Catalyst.

  • http://Surfinwithstan.com Stan Bush

    Great testing methods Chris. I’ve got a couple of WP installs. If they decide to go deeper, I’ll let you know.

  • http://pross.org.uk Simon

    I find it hard to believe PageLines served up 5+ Mb of data when the entire framework package is less than 1.5M, where is the test site so we can see it?

  • http://Apple-Blog.dk Aleksander Hansson

    I think the test is bs. PageLines Framework is only around 1.2mb so how can you ever get 5.2mb without uploading extremely large graphics?

    Give us the Pingdom Tools links so that we can verify the results you got.

    Also, why would you want to setup multisite where you have all the frameworks running instead of replacing a framework on a singlesite?

  • http://nickhaskins.com Nick Haskins

    Not quite sure how you’re getting 5s wait and 5mb pageload for PageLines? Where is the consistency in your evaluation? Where are the results? Where are the benchmarks?
    I just loaded up my personal site, and am getting a 2s load with a weight of 564 kb on PageLines.
    http://tools.pingdom.com/fpt/#!/pnbEAZihN/http://nickhaskins.com

  • http://twitter.com/CoreyFreeman Corey Freeman (@CoreyFreeman)

    I’m excited to see that my favorite theme has pulled ahead! I love working with Headway, and ever since the 3.0.5 update it’s become blazing fast and hyper optimized. Awesome comparison :)

  • http://twitter.com/MZAWeb Daniel Dvorkin (@MZAWeb)

    Really interesting. Props to you to take the time (and budget) to do it.

    I’m wondering how much real-world results may vary. For instance, most of this frameworks come with a lot of custom widgets. I’ve experienced myself how some of those widgets are total performance killers, but I guess people use most of the functionalities the frameworks come with.

    So, when you deploy all the particular features for each framework (widget, filters, settings, etc), instead of using a simple template like yours, how do they compare?

  • chrislema

    To the folks who can’t believe my results, who want more proof, who disagree with my approach, or who missed the fact that on chrislema.com I use a 600×400 image for every post, I don’t mind approving your respectful comments and disagreements.

    • http://pross.org.uk Simon

      The 5 images add up to 916kb, so where the other 4.5Mb coming from? Are all the themes/frameworks set to show the image/full post on the blog page?

      • http://chrislema.com chrislema

        It was 10 posts. Not 5. Only titles, excerpts, and images were on the page. I’m positive that some images were larger than others, but all were resized to the 600 width I specified.

        • http://pross.org.uk Simon

          Odd, your main blog only shows 5. So did you set all the other themes to show the images too?

          • http://chrislema.com chrislema

            Yes. All themes were set up identically – with 10 posts. See the beginning of the article.

    • http://nickhaskins.com Nick Haskins

      So all the themes have the same images ya? With that being said, they should all weigh the same (the images). Obviously the themes in your test aren’t setup the same way, otherwise, all frameworks would be consistent in the weight of the images.

      Your ranges are completely out of the realm of possible. 5+mb for PageLines, compared to all the other themes that only show 1mb. It’s obvious that those other themes didn’t load the images, or we’d see the weight of those images in your results.

      • http://chrislema.com chrislema

        They were all set up the same guys. Try your own comparisons and write your own review and I’ll link it from here if you like. I get that you’re incredulous, but I didn’t have it in for Pagelines. I like the theme and was surprised. I’d guess it was related to image processing but it’s just a guess.

        • http://pross.org.uk Simon

          Well as there are no links to the pingdom pages I guess we are all just left guessing.

      • http://www.realestateconnected.com.au Merv

        I disagree, the child theme Chris used May have not been serving the right size image eg thumbnail or medium size. I’ve made the mistake myself with a theme and even though I wanted a 200px image it was resizing to that from the 1200px image.

        At the end of the day if your starting child theme is crap,you get crap.

        Personally I use Ithemes builder and after seeing your review would never change as it it now super fast. Genesis on the other hand to me is pretty average (I’ve been asked to compare them) and I created a similar test site. The biggest difference is dev time.

        You can’t beat a consistent framework like ithemes builder having a clear and consistent naming convention.

        Also the layout engine is incredible.

        Just looking at genesis makes me cry in the inefficiency within the child theme

  • Linda Sherman

    This is great information. Thank you Chris! We are currently using Genesis and have been sitting on a developer license for Headway. Now we are going to try it out.

  • http://austinpassy.com Austin “TheFrosty” Passy

    I am a Hybrid Core guy myself.

  • http://gravatar.com/pixelyzed pixelyzed

    Awesome post Chris. I’m not surprised to see Builder and Canvas at the top but happy to see them there as they are my go to themes.

    Btw, how would you change your scoring of the different frameworks in your other comparison post now that Builder 4.0.x is out with responsive support? Personally, I love how flexible it is and how you can set your own cutoff sizes in a simple function in functions.php. I love Chris Jean’s vision of evolving the framework…

    Thanks again for your very informative posts!

  • http://headwaythemes.com Grant Griffiths

    Hey Chris. What a great comparison review. Thanks for including Headway in your test subjects too.

    The true winners in any review is the WordPress community. Isn’t it great the community has so many choices of themes and frameworks to pick from. No one theme or framework is the answer for everyone. And having all of these choices for the community is the win-win here.

    Thanks again Chris for putting in the time and effort to do this review.

  • http://gravatar.com/cortjohnson Cort

    Thanks so much for doing this..It was one thing that was missing in the Paulick reviews…:)

  • http://notagrouch.com Oscar Gonzalez aka @notagrouch

    Awesome breakdown man. Bookmarking & saving for reference.

  • Pingback: WordPress Frameworks Compared on Performance | My Collection of Stuff Online

  • http://www.facebook.com/danieljsj Daniel Schulz-Jackson

    I’m confused; you say in your conclusion that Headway comes out on top, but numerically Builder is actually on top both times… what’s the rationale for that? Thanks for this great review,

    • http://chrislema.com chrislema

      A weighted ranking based on position put it in second. But it’s still great.

  • Robert

    Could you look at http://swiftthemes.com/ and test that one as well? They claim to be the fastest. Even faster than Headway! Is this true?

    • http://chrislema.com chrislema

      I may check them out. I’ll let you know.

  • http://www.webmatros.com Oliver Nielsen

    When I first checkout Headway 3 I was hesitant, as I feared the amazing flexibility and drag & drop editor would hamper the performance.

    But to my suprise, Headway is super optimized, fast and lean. Since then, I’ve been a huge Headway fanboy =)

    Good to see this test Chris, as it will prove to many potential Headway users that Headway indeed is among the top frameworks in the WordPress community.

  • mikezielonka

    @chris I run PageLines on dozens of sites and my customers who upload huge graphics dont see load times like that on WP Engine.

    I’d like to run the same test on a single instance and not multisite on WP Engine. Would you mind sending me your xml file?

    • http://chrislema.com chrislema

      I think we’re finding the potential issue. I think it’s related to how Pagelines deals with images. I not only have large images I use. Sometimes I upload a really large image but use one of the pre-generated sizes. I’m checking to see if one of those images was loaded by Pagelines and sized on the page rather than before loading. That would explain it.

      • http://www.webmatros.com Oliver Nielsen

        Sounds feasible. Still, that would be a very ineffective use of images. Potentially expensive, hosting-wise as well.

        • http://chrislema.com chrislema

          No doubt. That said, my hosting provider is awesome, sometimes I’m careless on the run, and the other frameworks seemed to handle it, suggesting at least one place to improve or one warning to end users.

          • http://www.webmatros.com Oliver Nielsen

            Just wanna clarify:
            I meant ineffective use of images *by the framework* not by you;-) I myself upload many large images to my sites. That way there’s always an original available, for batch resizing etc.

            So it is actually PageLines doing something wrong here.

            Why did I come back here, after a few months? Because PageLines DMS does the same stunt. Load the original images for thumbnails. Trying to figure out why, and how to solve it.

  • http://pross.org.uk Simon

    Chris, PageLines by default shows the featured image thumbnail, like any other theme. Also only shows excerpts on the main blog page by default.
    Unless you changed the settings to show the full image..?

    • http://www.webmatros.com Oliver Nielsen

      But PageLines, DMS included, has a tendency to not dynamically create resized versions of the thumbnail images original file. If one uploads big images, expecting the theme to create proper, resized thumbs; one will be surprised that it isn’t the case with PageLines.

      It’s also an issue in DMS, so I and a few others have pointed it out to the PageLines team, and now it seems like it may be rectified in a future version.

      That’s probably why Chris’ measurements turned out as they did. He didn’t do anything wrong. PageLines did;)

  • http://usmanjilani.com Usman

    Hey Chrislema sorry for the previous message. I just found your no hold barred review on Thesis 2, and you’ve made your point pretty clear! Loved the comments as well! I personally don’t mind Thesis 2 as I can code css to make further customizations. But will be giving Ultimatum a try as well! As far as calling Thesis 2 a drag and drop framework, I don’t think Thesis 2 qualifies for that!

    Thanks for your work man!

  • Pingback: Headway rocks for WordPress! Here's why - WebMatros.com

  • Pingback: WordPress Frameworks Compared on Performance | ...

  • http://organizedassistant.com Janet Barclay (@JanetBarclay)

    I’ve been using Builder since it came out, and love it. In fact, I now use it exclusively for new sites, but a few of my sites are very slow and I was afraid it might have to do with the theme. Thanks for doing this study – now I’m not worried about having to learn a new framework!

  • Pingback: PageLines DMS is not the only DMS in town! So is Headway...

  • http://anseltaft.com/ Ansel Taft

    Chris, the tables in this post are broken.

  • https://www.facebook.com/fiona.howarth.3 Fiona Howarth

    I love your articles Chris – no BS and great information. I am using Woothemes Canvas but I don’t really love it. Will have a look at Headway which I’m not familiar with and see how that works for me.

    • http://www.webmatros.com Oliver Nielsen

      Hi Fiona

      I know Canvas, have worked with it and reviewed it, and doesn’t love it either;-) I use Headway, so let me know if you have any questions, comparing the two.

      =)
      Oliver

  • Pingback: iThemes Builder - Getting Past Shiny Stuff Syndrome

  • Pingback: Want a Premium Slider?

  • Pingback: EP10 - "Theme Frameworks" - WPwatercooler -- November 26 2012 | Appthemes Showcase

  • Andrew J Klimek

    Hey Chris, the date at the top of this page says 2014, but on the archives page it said 2012. Anyway, just curious if this was Genesis 1.x and if so, did you have any feelings about improved performance in 2.0?

    • Chris Lema

      This is an older article. I haven’t done a refresh lately.