As we all know, Qt5 is approaching. The “big new thing” with Qt5 is the introduction of QtQuick/QML for writing dynamic user interfaces. If you are (somehow :P) unfamiliar with QtQuick and QML, you can read up about it here. The rest of this post assumes that you know something about Qt Quick.
Qt5 is imminent, with a release set for sometime within the next few months. But as of currently, there seems to be several questions that remain in regards to QML. I have encountered several questions, some directed at me, such as “Why would a desktop application want to use QML?”, “How will traditional desktop applications be able to use QML?”, “How will QML applications integrate with KDE and its existing applications?”, and, more specifically, “How will Muon Discover integrate with KDE since it is using QML?”. In this post I will attempt to answer these questions, with my opinions and thoughts on the matters.
Why would a desktop application want to use QML?
Traditionally, user interfaces for applications have been written to be composed of discreet “forms” or “pages”, with each form containing a (mostly) static collection of controls called “widgets”. To present new forms to the user, an application would either present the new form as a separate window/dialog, or replace the form in the main window with a new one entirely, using a widget stack to contain the forms. These interfaces were coded either in straight C++ (or with language bindings), or by compiling XML files created with a WYSIWIG-esque interface designer. A common set of widgets is shared among applications of a particular toolkit. (Be it Qt or Gtk, etc) This, along with theming, allows for very tight graphical consistency between applications written in the same toolkit.
Now, there is nothing particularly “wrong” with the way things currently work, but that does not mean things couldn’t be better. Animations are a useful tool with regards to usability. Aside from looking slick, user interfaces that morph gradually are easier for the human mind to “track”, as opposed to instantaneous change. But until the advent of QML, providing animations (and fluid interfaces in general) with the traditional static widget-driven user interface, you had to roll your own. Qt did provide several APIs within the last few releases to provide animations/effects within QWidget-based interfaces, but even then doing animations and graphical effects was not exactly trivial to “get right”. And even when using these APIs, the effects’ drawing performance is tied to the painting backend in use with Qt, which can be slow (e.g. the old X11 Qt painting plugin), or at the very least can’t take advantage of the dedicated graphics hardware of the modern computer.
In short, QML makes it very easy to write fluid user interfaces, while retaining speed. The advantages expand well beyond the mobile application sphere, and represent a new way to do GUIs.(tm) [/marketing mode]
How would a desktop application use QML?
But what, specifically, would you use QML for within a desktop application? Aside from generally animating the interface, how would QML be used to provide advantages of the current QWidget paradigm?
For one, it makes it much easier to present non-control media such as pictures in an appealing fashion. Take for example, the slideshow for featured items in the home page of Muon Discover. It is written in a mere 126 lines of QML, and provides navigation between featured items, animated item switching, fading in new items, and it automatically pans around screenshots that are too big to fit in the banner. Doing the equivalent with QWidgets or even QGraphicsItem and friends would be a lot more involved, requiring lots of custom painting an animation code, with potentially worse performance.
Aside from the presentation of things that are strictly media (though to be fair, clicking on the aforementioned slideshow will take you to the app’s page), QML can also aide in the creation of user controls that themselves can incorporate media and animations.
Take delegates for list views as an example. If you want to go beyond the simple icon-and-text that the default Qt list view delegate offers, you must subclass your own delegate and re-implement the paint function. Re-implementing the paint function requires you to do a lot of pixel math with a lot of code that, frankly, I never mastered. Whenever I’ve needed a custom delegate, such as for the Muon Software Center and Muon Package Manager, I’ve resorted to searching lxr.kde.org for delegate subclasses, searching for a delegate that kind of does what I want, and tweaking that until I get something useful. At the end of the day, even creating a delegate that paints an icon, two lines of text, and a rating widget takes around 300 lines of code. You get a whole lot of control, at the expense of a whole lot of hassle.
300 lines of code for this itemview delegate
With QML, a similar listview delegate takes only a bit more than 100 lines of QML, without forcing the UI designer to do tons of pixel-based positional calculations within the QML script. With it being so easy to make itemview delegates in QML, Aleix was even able to create a second “icon grid view” delegate that incorporates animation within the delegate to show different information when moused over, as seen in his video here.
On the issue of integration and consistency
One of the strengths of KDE is how well its applications fit together. Most every applications share exactly the same set of controls. (Buttons, list views, line edits, scroll bars, etc, etc) Similarly, these common controls all follow a common theme. The combination of these two provide a consistent experience common to all KDE applications, which makes it easier for somebody new to KDE to pick it up. If you learn how a combobox or list view works in one application, generally, you’ve learned how they work in all KDE/Qt applications.
This level of integration is not yet available in QML, which is why we’re holding off from replacing the Muon Software Center with Muon Discover until things change. As of now, Qt does not offer a set of common components like we see with the current set of QWidgets. There is the set of common QML Elements that provide a rudimentary set of common components, but it’s nothing as extensive as what we have currently. If you need a UI control in QML, currently you must roll your own. Doing this results in un-integrated controls reimplemented by each application that potentially do not share a common theme. Muon Discover is currently using Plasma’s QML components, which while providing graphical and widget-level integration with Plasma and making Muon Discover fit in on a Plasma Active system, does not provide integration with most of the other QWidget applications on “the desktop”.
The solution to this does appear to be coming, however, in the form of the Qt Desktop Components for QML. The QML Desktop Components provide a set of UI controls analagous to the ones available with QWidgets. Additionally, they use QWidget theming, allowing graphical integration as well as behavioral integration with QWidgets. You can see them in action in KAlgebra Mobile here. Through the use of desktop components, we can make applications that utilize all the benefits of QML while retaining integration with the rest of the desktop. The only caveat is that the desktop components haven’t actually been released yet. Previews can be found at their Gitorious site, and we even have a git branch for Muon Discover that (partially) uses the desktop components. I’ve not heard a concrete release date for the QML desktop components, but seems like they will be The True Way Forward ™ for QML on the desktop.
Conclusion
I feel very strongly about consistency, so to re-iterate Muon Discover will not displace the Muon Software Center until we have a way to integrate with the rest of KDE. QML does, however, provide a very powerful way to create attractive, easy-to-use interfaces without too much effort. While we wait for a solution for the desktop to become available, we can continue to hack on Muon Discover and allow it to mature. And hey, it already integrates well with Plasma via the use of the Plasma QML components, so perhaps in the nearish future (with a bit of work to make Muon Discover touch-suitable) you could find Muon Discover on a Kubuntu Active spin near you. 😉
Jonathan's Blog 
[…] source: Go to Source Feed source: http://planet.debian.org/deriv/rss20.xml License: The original licenses are retained […]
Small correction: QML was already in Qt 4.7 and is used in many places already in the Plasma Workspaces.
The reason for the version bump to 5.0 is a binary compatibility break to make room for lighter applications (no Qt3support) and more platform compatibility (former “Lighthouse/QPA” project). For example, pure QML apps no longer to link to the widgets library, and the QML engine is rewritten to directly use OpenGL scene graphs, instead of (resource intensive) rendering calls.
Other than that, users probably won’t notice the difference when KDE switches to Qt 5 (regressions aside). Application developers will need many years to rewrite their applications in QML, if they want to do it at all. For new code, however, QML is a nice toy, which developers should try and get familar with. And, as you said, once we have themable QML components, it can also be used for serious work.
Of course; you’re right. I knew that QML was already around, too, so I feel kind of silly now…
it’s 10 times more annoying to debug javascript/ecma than c++, where even simple typos cause you to waste time running iterations of your application just to find out you made a simple typo, because nothing is checked until runtime.
now we get to debug entire UI animations, transitions, and interfaces like this. Yipee. if there’s good solution to this i want to hear it.
secondly it’s already going to be Qt 5 and there’s still no complete widget/themed Qt Components type system? this is like the KDE3->KDE4 transition disaster, too early, not well thought out enough yet. it’s just not ready for real UI’s. only mobile toys and little fun animations whose novelty will wear off soon enough (sort of like 3d composited toy effects), especially since the ‘transition’ to these are going to in such small increments rather than large scale at least in major part by the incompleteness of the current spec (in terms of desktop utility)
finally, as cute as these little UI’s are, i question whether qt had to introduce an entirely new (rather syntactically finicky) state language based on ecmascript rather than the c++ family like they did with moc. they just added another barrier for programmers to overcome in order to hop on the apple insipired UI bandwagon which hopefully in 5 years will have worn out its welcome by consumers (after windows 8 fails to meet expectations) and be primarily relegated back to the touch-based mobile space where it probably belongs. make no mistake, qt quick was created for touch mobile and only incidentally is now finding itself on the desktop. it was an afterthought -if even that- and that’s why we now have an incomplete runtime language half-assedly fumbling its way around in the desktop arena.
it’s 10 times more annoying to debug javascript/ecma than c++, where even simple typos cause you to waste time running iterations of your application just to find out you made a simple typo, because nothing is checked until runtime.
now we get to debug entire UI animations, transitions, and interfaces like this. Yipee. if there’s good solution to this i want to hear it.
secondly it’s already going to be Qt 5 and there’s still no complete widget/themed Qt Components type system? this is like the KDE3->KDE4 transition disaster, too early, not well thought out enough yet. it’s just not ready for real UI’s. only mobile toys and little fun animations whose novelty will wear off soon enough (sort of like 3d composited toy effects), especially since the ‘transition’ to these are going to in such small increments rather than large scale at least in major part by the incompleteness of the current spec (in terms of desktop utility)
finally, as cute as these little UI’s are, i question whether qt had to introduce an entirely new (rather syntactically finicky) state language based on ecmascript rather than the c++ family like they did with moc. they just added another barrier for programmers to overcome in order to hop on the apple insipired UI bandwagon which hopefully in 5 years will have worn out its welcome by consumers (after windows 8 fails to meet expectations) and be primarily relegated back to the touch-based mobile space where it probably belongs. make no mistake, qt quick was created for touch mobile and only incidentally is now finding itself on the desktop. it was an afterthought -if even that- and that’s why we now have an incomplete runtime language half-assedly fumbling its way around in the desktop arena.
I like the idea of using QML for all different platforms and formfactors. The possibility for sharing over 90% of code base between very different kinds of devices while at the same time being optimized for each and everyone sounds nice. I’m personally very found of clean animations and the unfortunate fact nowadays is that they aren’t all that smooth. Thanks to Qt Quick and Scene graph these problems should be history. Then there’s the new possibility for lowering the bar for desinger to enter to the developement and faster UI desinging. I can’t wait for the day that entire KDE UI developement can be done with just QML and SVGs but I guess that’s going to take years.
it’s ten times more annoying to recompile your c++ application every time you make a small change on the ui.
now we finally got something were we can view changes instantly.
desktop components will come with Qt 5.1 or 5.2 .
It’s absolutely no disaster as for existing QtWidget absolutely nothing will change. just recompile your stuff against Qt5.
KDE3 to KDE4 was a huge change under the hood and a lot of applications had to be rewritten partially or as a whole. this will not happen with Qt5
“cute as this little UI’s are” – pile of shit.
you can do nearly every type of ui a lot faster than in c++. especially you can do very fast prototyping.
and doing this stuff with moc..
qml is about property binding, a superb feature. but i guess it would be a lot easier to call a macro like QBind(stuff) for every property. would look ugly, but might not be that hard to implement using c++11 lamdas. but as of now a lot of compilers don’t support this standard, so qt will not make use of it.
qml WILL be used on the desktop – successfully.
it’s just some c++ fanatics being to lazy to learn how to use qml and (super easy) to integrate it with c++.
Animations are just an annoyance which distracts from the actual application and negatively impacts productivity. I disable all the animations that can be disabled and curse at the others. Spamming all applications with more and more useless animations is a huge step in the wrong direction.
As for “retaining speed”, QML, being interpreted, is necessarily slower than the compiled C++.
And finally, people who cannot do simple mathematics should not be programmers. Pixel positioning requires only elementary-school-level mathematics.
There’s a difference between being able to do pixel mathematics and wanting to. Programmers have limited time, and a great many better things to spend it on than pixel maths.
and thst the reson we must find designers thst need to care about such litle minute details..
“QML, being interpreted, is necessarily slower than the compiled C++.”
This is a misconception.
QML is _not_ an interpreted programming language.
It is a data definition language, which is used for defining the object tree (visual + behavioral objects) which the QtQuick/SceneGraph engine (which is written in C++) should operate on.
The QML file is translated into an in-memory QObject tree _once_ while the application starts up.
After that all you have is a C++ engine operating on an in-memory binary data structure at native speed, i.e. the fact that the binary QObject tree was automatically instantiated from a QML file rather than from custom C++ code makes no difference whatsoever.
This declarative approach allows graphics-intensive applications with Qt Quick 2 to be _faster_, not slower, than their old-fashioned imperative Qt/C++ counterparts, because this way the engine can manage the whole scene together and perform the drawing and animating in a very optimized fashion that gets the most out of modern graphics hardware (i.e. delegating most graphics operations to the GPU, not CPU).
Also note that developers of Qt have already mentioned the possibility for a future Qt 5.x release to support translating QML into the binary data structure at compile time rather than during application start-up, so you will be able to even save those few milliseconds during application start-up if that’s what bothers you… 🙂
—-
What _does_ get interpreted though (kind-of) is JavaScript snippets embedded as text nodes in the aforementioned object tree, used e.g. for dynamic property bindings or event handlers.
I say “kind-of”, because I’m pretty sure that the V8 JavaScript engine used in Qt Quick 2 compiles each JavaScript function into byte-code _once_, and then runs that byte-code every time the function is called – which is still slower than true native code, but a lot faster that “interpreted”.
Anyways, Qt Quick developers are free to embed as little (or even none) JavaScript snippets in their QML object tree as they want.
For simple bindings and onClick handlers etc. it is extremely convenient though, and if used sparingly the overhead is negligible, especially compared to the large performance gains caused by the declarative architecture as a whole (see above).
Any more complex logic should be implemented in C++ anyways, and made available in the form of objects/components/data-models which can then be instantiated through QML.
(Also Note that QML allows even pretty complex animations, transitions, states, etc. to be defined as declarative behavioral objects with no JavaScript at all, so it’s really not comparable to HTML+JS where even the most trivial behavioral features require at least some JavaScript code).
In other words: Use C++ for the application’s backend functionality, and QML for defining the look and behavior of the UI, with just as much convenience JavaScript thrown in as you’re comfortable with – and altogether you’ll most likely get much better performance than you would get with a QWidget/QGraphicsView based application.
ever done delegates containing widgets using old model/view framework?
it’s a mess and includes a lot of hacks to actually work
just look at the code of KWidgetItemDelegate.
and finally, offending Jonathans mathematic and programming skills is very hilarious. as you can see by browsing the code of the Muon Suite you can clearly see that this guy knows how to design software.
QML has a concept of “platform” animations, i.e. the same application may automatically use different transitions depending on the platform it is executed on.
For the desktop, those platform animations will probably be defined by the style. So you should be able to pick your style of choice and configure it to use no transitions.
Regarding “speed”: it’s not like the entire application is implemented in Ecmascript, just the description of the UI. We also have XMLGUI since KDE2, which doesn’t appear to be too slow.
But is it possible to build QML UIs with a WYSIWYG editor?
I don’t want to code my UI. I want to do it like I’ve done it with QWidgets.
Qt Creator has support for this: http://doc.qt.nokia.com/qtcreator-2.5/creator-qml-application.html
Also note that QML files are _much_ nicer to work with, compared to writing imperative C++ QWidget code or manually editing Qt Designer’s XML-based .ui files (which are also QWidget-based).
Rather than basing it on an existing machine-friendly but human-unfriendly format like XML, the Qt team has designed QML from scratch with a human-friendly, JSON-like syntax – see http://qt-project.org/doc/qt-4.8/qdeclarativeintroduction.html
So even if manual source code editing wasn’t for you with those older technologies, try it again with QML, in my opinion it’s more productive than using point-and-click editing.
Could you be more specific as toe what you are missing in the Plasma Components for desktop apps? They are supposed to be suitable replacements, so we need to know about issues in the real world in order to improve them.
The main thing that makes Plasma Components unsuitable for the desktop is the lack of integration with the desktop itself. The Qt theme isn’t used, and so while it does make sense to use the Plasma Components when making applications targeting mobile platforms such as Plasma Active, I don’t believe it makes sense to sacrifice the existing integration with the user’s existing Qt style.
Plasma has always been very insisting in that the shell styling should differ from apps. Generally applications (aka the windows) are supposed to use Oxygen, I understand.
If you use Plasma Components, you use a wrong styling :/.
If you think this could and should be fixed, please get in touch with me, I’d really like to work on it.
IMHO, the distinction between Plasma styling and application styling has been broken from day 1. It’s just a lame excuse to explain away the glaring design inconsistency which sticks out like a sore thumb.
I actually don’t feel like relying on QStyle is going to be a solution in the long run, actually with Qt5 this will be considered done.
Anyhow, the Qt Components concept is very powerful and we should find a good way to use them for powerful styling (aka hopefully having oxygen components).
In any case, it’s going to be interesting to face this issue and something that will definitely be beneficial, if done properly.
Guys in case you are going to do U’s with Pure QML plese find a designer (a good one) thst can help you with that… It is far from a trivial design task to make anything that looks consystent with defoult style with QML.
QML is great and fantastic and it my facvorite tool this days, but unless you consider yoursell a very good designer I would avoid creting custom desktop UI’s with it.
Even smaller correction:
> ‘composed of discreet “forms” or “pages”’
You mean “discrete”.
The original QML components for desktop project seems to be dead (or frozen, or internal). Take a look at http://codecereal.blogspot.de/2012/04/qml-themingstyling.html which introduces a very interesting project. It will hopefully contribute to bringing QML to the desktop in the right way.
I’ve been using Qt 4.x for years and love it. It takes me minutes to create simple applications. I’m excited for QML, I think the minds behind Qt have got the right approach.
Nevertheless, until such a time that I can create basic desktop apps in Qt with close to the same ease as the current paradigm, I won’t make the jump. Having to come up with code for basic widgets every time is just not a reasonable investiture of my time.
Every jump in paradigm – from specifying pixel positions to using a layout manager, from programmatically defining UIs to using a graphical editor that generates an XML document – are meant to bring the time required to define UIs in line with the greater complexities demanded of them.
At some point, it’ll be easier in QML to make complex applications. It already is where animations are concerned. When it gets almost as easy for defining forms, I’ll be there.