YesNoOk
  1. Welcome!
    EosAlpha BBS is a fork of the popular Simple Machines Forum software. We aim at creating a new forum software, adding several new features and a modern and fresh design on top of the existing SMF code base.

    This software is currently in an early stage of development and this forum acts primarily as a testing platform for the ongoing development.

    Feel free to look around to get an idea about how it feels and looks.
avatar

To twig or not to twig (Read 7523 times)

Started by Nightwish, June 22, 2012, 23:05:24

0 Members and 1 Guest are viewing this.
Share this topic:
#1
To twig or not to twig |
June 22, 2012, 23:05:24
A couple of days ago, I posted about the idea of using a template engine for EoS and the first one I tried was Twig, a very mature and widely used product.

Initially, I thought there was no impact on performance, but this was due to a nasty bug in the timer code (it didn't even record template output at all, so the execution times I got were totally wrong). After fixing that, I was shocked to see the real impact, which is quite huge and - in my opinion - not acceptable. Here are a few numbers I recorded while testing the Twig engine on various pages:

Board index - average "per request" execution time for 10 requests.
Default templates: 14ms
Twig: 38ms

Topic page (25 posts per page)
Default templates: 26ms
Twig: 105ms

Looking at the generated PHP code (like most template engines, Twig compiles templates into PHP code), I quickly realized why it is so slow. The code is pretty complex, but this is necessary, owing to the broad feature set and design of Twig.
Now, Twig isn't bad, it's actually very good and a joy to work with, but it's probably not designed for the heavy duty usage pattern of a forum - lots of code to execute, long loops (especially on topic display) and lots of data to output. Also, SMF still has a bit too much logic in the template code - there is lots of stuff that could be moved from the templates to the backend which would result in leaner and therefore more lightweight templates.

So, basically the experiment is over before it really started. I tried a few things, read a lot of docs, but even with all possible optimizations, the impact on performance was simply too big and unacceptable, especially for the topic display part, the factor 3-4 in execution time makes a huge difference.

Solutions?
Well, yes. Good old Smarty - a lot simpler than Twig, but also a lot faster. Smarty's syntax is a lot less abstract and much more similar to raw PHP, but it still shares most of the advantages of using a template engine with Twig. Performance tests gave promising results.

Here are the Smarty times for the two examples above:
Board index: 18ms
Topic page: 33ms

That's pretty good compared to the default templates which are basically hand-written PHP code and will always outperform any template engine, simply because SMF templates are raw PHP code with no additional overhead.

     Posted: June 25, 2012, 04:11:36
A bit more about benchmarking template engines.

First, I've changed the timer code so that it gives a bit more detailed output.

An example:
0.019s CPU (0.007s template), 13 queries. URL rewrites: 4 (0.001s CPU, 1 queries.

The red number is the overall PHP execution time including template rendering but without output buffer rewriting. The red number does include the blue number, which indicates how much time was spent for template rendering. Note that for topic display pages, the blue number is not really accurate, because it includes the time for prepareDisplayContext() (one callback from the template rendering per message) which prepares a single message for rendering. So for topic display pages, the time spent in templates is much higher, because it contains a fair share of backend code execution, but for all other pages, the numbers should be fairly accurate.

The last part (teal) is the time spent for buffer rewriting and it is NOT included in the red number. The main consumer here is the URL rewriting and this has absolutely nothing to do with the templates.

Smarty templates are, of course, a bit slower than normal SMF-style PHP templates, but the impact isn't really big, just a few milliseconds and this is on a fairly slow processor (the machine running this forum isn't the fastest one - a many years old dual Xeon CPU).

Twig, on the other hand, was about factor 3-5 slower, resulting in unacceptable high template execution times (blue numbers), so the decision to use Smarty was an easy one. Smarty's feature set is more than good enough and I already have some ideas how to use it to implement interesting things for supporting a plugin architecture. Template hooks are very easy to do for example.
__
Every program has at least one bug and can be shortened by at least one instruction -- from which, by induction, one can deduce that every program can be reduced to a single instruction that doesn't work.
My SMF-based forum fork
1 Useful  | 1 Like It 
Last Edit: February 24, 2013, 14:32:14 by Nightwish
#2
Re: To twig or not to twig |
June 27, 2012, 20:26:01
Hey there.

I was recently searching for sphinxsearch smf on google which led me to your fork. What caught my attention on your page were actually the privacy-enabled Like-thingies on the right, and that you were the author of tabsrmm and clist nicer, both of which I used for a while back in the day. Most importantly, you develop using git (and on github!), which is always a huge plus. Anyways I looked around for a bit and I must say I love what I'm seeing, concepts and implementation. Great job overall, thank you for your work! Hope to see a release of this some day!

Ok, on topic now.

I read your arguments, but I am not convinced using Smarty is a good idea: The performance cost is not that insignificant, it might be "only a few milliseconds", but those milliseconds are actually a 25% increase. More importantly though, I was always sceptical about the practical use of smarty. It's additional syntax one has to learn so it's not really easier to get into. Security doesn't really matter because this isn't a scenario where template code comes from an untrusted source. And the code is really not that much cleaner than just writing plain php - although that's obviously an opinion. Still, to me the little benefits seem far from worth the cost.

To actually benefit from a layer between php and html, I would expect something that has syntax which is actually easier to write than plain html, like haml. Smarty just seems like a half measure to avoid using the echo statement in comparison.

You mentioned plugin architecture there, maybe that's the bit I'm not seeing. Could you elaborate on why you think a template engine (or smarty in particular) could be worth the cost here?

 - Valodim
2 Like It 
#3
Re: To twig or not to twig |
June 27, 2012, 21:32:11
Ok, on topic now.

I read your arguments, but I am not convinced using Smarty is a good idea: The performance cost is not that insignificant, it might be "only a few milliseconds", but those milliseconds are actually a 25% increase. More importantly though, I was always sceptical about the practical use of smarty. It's additional syntax one has to learn so it's not really easier to get into.
Well, if you have to learn from scratch, smarty *should* be easier. Writing templates for SMF requires fairly good PHP skills and this is probably the reason why 90% of all SMF themes are simply "restylings" of the default theme (i.e. CSS and image edits). Writing complete themes with a new set of templates is a daunting task with all these endlessly long echo statements with complex punctuation (getting one '.' or ',' wrong can lead to bugs that are hard to find and fix for example).

Also, I *do* plan to move some logic from the template(s) back to the code where it actually belongs to. A lot of template code (especially URL generation and things like that) has imho no business there and slows down template rendering - templates should only output data which is already there.
Quote
To actually benefit from a layer between php and html, I would expect something that has syntax which is actually easier to write than plain html, like haml. Smarty just seems like a half measure to avoid using the echo statement in comparison.
Yes, smarty provides only a very low level of abstraction - but that's probably the reason why it can be so fast. In the end, a template engine *must* compile its code into PHP, because everything else (i.e. parsing at run time) would be way too slow for the heavy duty usage of a forum.
Quote
You mentioned plugin architecture there, maybe that's the bit I'm not seeing. Could you elaborate on why you think a template engine (or smarty in particular) could be worth the cost here?
Well, template hooks come to mind. Plugins should have the ability to register template fragments in various hook positions. These hooks are "chainable", so multiple plugins could populate the same hook.

Of course, such a system could be done using standard SMF templates as well, but I still think that smarty templates would make things a bit easier for plugin authors

There are also some other benefits, like template inheritance, a much easier way to modularize templates and more. I'm not saying that these features can only be implemented with smarty, in fact, everything *could* be done without it, just a bit harder and more confusing for plugin- and theme authors.

That said, I think using a template engine would make sense. I've often read that one of SMF's biggest disadvantages is its theme system, because it's complex and lacks a couple of features available in the competition and that's why I think a minimal impact on performance is acceptable - even with smarty templates, EoS isn't much slower than a default SMF.
__
Every program has at least one bug and can be shortened by at least one instruction -- from which, by induction, one can deduce that every program can be reduced to a single instruction that doesn't work.
My SMF-based forum fork
#4
Re: To twig or not to twig |
June 27, 2012, 21:58:18
Hm. I suppose mods work better with alternate themes if there is a template system inbetween.

Well, you seem to know your salt, so if you think it's worth it, go for it.
#5
Re: To twig or not to twig |
February 22, 2013, 13:52:48
oh! I wish I had read this before :P I also take a look at twig and had the same results, I jut checked it against an experimental blog code, using a framework contributed to the slow times as well.

As much as I personally like the default SMF template system (not because is easy to use but because is pretty much the only template engine I ever used) I also can see the benefit (for themers) of using an "inverse" method, aka, wordpress template where PHP is inside HTML. But thats somehow ugly and I def wouldn't want thousands of <?php ?> tags just filling up space.

As for template engines like smarty or twig, I have mixed feelings, when I see a twig template, my mind immediately goes "oh, yet another pseudo code and syntax  I need to learn", perhaps it is just me being reticent, I still need to find a cute HTML template and port it over to twig code to see if I'll ever be interested on it.

And I agree, the templates def needs to lose some weight if they want to fit in a pretty and fashion twig engine :P  that not only applies for normal templates but for mobile versions as well, there is just too much displayed data that makes it terrible difficult to implement a really nice, clean and objective mobile version.
__
Sin talento no busques grandeza, porque nunca la vas a obtener.
#6
Re: To twig or not to twig |
February 23, 2013, 14:48:32
oh! I wish I had read this before :P I also take a look at twig and had the same results, I jut checked it against an experimental blog code, using a framework contributed to the slow times as well.
Twig is actually pretty cool, it's just not well-suited (and probably not designed) for the heavy duty usage in a forum, especially SMF where templates are heavy, with lots of if/else and loop constructs.

The reason why I have chosen to use a template engine is because it's easier to write and debug templates. With smarty, it's just HTML code (with some special tags embedded) instead of endless echo() constructs which are sometimes hard to read. Also, smarty is easily extendable with custom functions and tags and the modular concept of blocks makes it easy to assemble larger pages made of many smaller sub-templates.

For example, there are template bits for topic rows which are re-used in many places - the same template block that displays a topic row in messageindex is also being used in search results, unread topics and elsewhere. Chainable template hooks (important for addons) are also easy to implement and the goal is to have a plugin system that do not require any changes to existing templates.
__
Every program has at least one bug and can be shortened by at least one instruction -- from which, by induction, one can deduce that every program can be reduced to a single instruction that doesn't work.
My SMF-based forum fork
#7
Re: To twig or not to twig |
February 24, 2013, 01:35:40
Well, in my case I also discovered that twig doesn't play nice with code that doesn't follow proper MVC patterns :P   my blog pages had too much code that should be in a controller but I was lazy and I only wanted to try out silex so I didn't care that much about that.

I don't know if forum templates can be light enough for a template system to really works without issues, a forum by definition holds lots and lots of data, as much as you cleverly show stuff or try to keep as less logic as possible, there will still be lots of data to display and lots of logic that just can't be moved out of the view, unless you have control of absolutely everything which somehow reduces the possibility of what can be done with the software itself (extensibility, scalability, plugins, themes, all that stuff.)

I agree with re-usage, I always wondered why SMF have separate templates for private messages and normal messages since they are the same, I also wondered why the user info in messages was printed with every message, that is static data that is been fetched and displayed multiple times, you can have a separate template for that and just call the template one time and print it as many tmes as you want/need.
__
Sin talento no busques grandeza, porque nunca la vas a obtener.
1 Like It 
#8
Re: To twig or not to twig |
February 24, 2013, 06:21:14
I don't know if forum templates can be light enough for a template system to really works without issues
Well, it can. Look at XenForo :) It pretty much uses MVC design patterns for everything and its templates are lightweight. If you want this in SMF, you basically have only one option: Rewrite the entire software from scratch - probably a task that would result in enough work to keep a couple of good fulltime developers busy for a year or more :) It's not something I plan to do though.
Quote
I agree with re-usage, I always wondered why SMF have separate templates for private messages and normal messages since they are the same, I also wondered why the user info in messages was printed with every message, that is static data that is been fetched and displayed multiple times, you can have a separate template for that and just call the template one time and print it as many tmes as you want/need.
Most likely performance reasons and limitations by technology. About 10 years ago, PHP was a very different thing and lacked a lot of today's features. Since SMF's templates are pure hand-written PHP code, they come with practically zero overhead. Template engines like Twig or smarty are also compiling their code into pure PHP, but the resulting code is less efficient. The performance impact is acceptable with today's hardware resources, but 10 years ago things were different.

Having the full power of PHP available in your templates is a tempting thing and results in templates doing lots of things that should better be done by the backend code (something that happens quite often in SMF). It's not so much a question of using MVC - just a very simple thing of separating the display process from the program logic. You can do this without ever spending a single thought on MVC or any other design approach, such patterns will only help you to make things easier and more organized but they are not a requirement for a strict separation of logic and presentation. In fact, SMF's way of populating the huge $context[] array and then using templates to output what's inside $context is a very simple (and efficient) method of separation. In SMF, it's just not enforced strictly enough.

__
Every program has at least one bug and can be shortened by at least one instruction -- from which, by induction, one can deduce that every program can be reduced to a single instruction that doesn't work.
My SMF-based forum fork