Blue Systems, the Muon Suite, and Kubuntu

Blue Systems

Like several of my Kubuntu comrades, I too will be joining Blue Systems. I will be starting in September, working half-time as to accommodate my academic journey in college. The scope of my work includes the development the Muon Suite, its supporting libraries, and Kubuntu in general. I would like to thank Blue Systems for giving me this opportunity to focus my non-academic attention to FOSS, and Kubuntu in specific. I would also like to thank my previous employers for giving me valuable experience in the field of software engineering, albeit not in the arena of FOSS.

I would like to take this opportunity to share with the wider Kubuntu and KDE community what my plans are going forward.

My Plans

I will continue my work developing and maintaining the Muon Suite, as well as its supporting LibQApt library that facilitates interaction with the APT package system. I also plan on continuing in my role as a Kubuntu Developer, contributing to the various blueprints brainstormed at Ubuntu Developer Summits, as well as the packaging of new KDE releases and KDE software. I do have a list of some specific things I would like to accomplish, at least to start with.

LibQApt 2

LibQApt is the library used “behind the scenes” by Muon for interaction with Debian’s APT package manager. It’s main purpose is to provide an easy way for Qt applications to get information about packages, using Qt data types and providing a sane Qt-style API for interacting with packages and the package system.

Following the release of LibQApt 1.4.0 this last Thursday, LibQApt is now 2 years old! Over these past two years, LibQApt has proven to be a solid library and APT worker implementation, with defects decreasing release-by-release. It’s also proven itself to be fairly feature-complete; as a testament to that, the Muon team didn’t have to add very many new features at all during the 1.4 release cycle to accomplish progress with Muon, and there were no requests for additional API by third-parties using QApt across that time period.

That being said, LibQApt is by no means perfect. There are several flaws within QApt that come from foundational design decisions I made when I first started working on LibQApt in late 2009, two and a half years ago. At the time, LibQApt was the most ambitious and technically challenging project I had taken on, so I’ll try to excuse to 2009-me. But there are some times when I look back at the stuff I did back then and say “why on Earth would you do that, 2009-me!?”. Now that LibQApt has a robust feature set and slowed need for feature development, coupled with the approach of Qt5 and KDE Frameworks 5, I believe now is the time to break API stability to fix the structural problems within LibQApt and release LibQApt 2.0. With ABI (and in some cases API) incompatible releases of core libraries such as Qt and KDE on the horizon, the near future seems to be the prime time to introduce API changes to LibQApt itself.

The Issues

The design issues I mentioned earlier mainly have to do with LibQApt’s runtime component, the QApt Worker, and the API within LibQApt used to communicate with it. The QApt Worker is a small daemon running as root that gets invoked whenever an application using LibQApt wants to initiate a transaction with the APT package system. (Say, to install or remove a package, update the package cache, or any other task that requires administrative privileges) Communication between the worker and the QApt library happens via a D-Bus interface, where an instance of the “Backend” class in LibQApt listens for signals on the D-Bus interface. The only problems is that with the current API, any instance of QApt::Backend can receive these signals at any time. This means that any frontend application listening to LibQApt can receive events from the worker even when they’ve not entered into a state where they would be expecting them, and if they’re not careful crash trying to handle the events.

This can be rectified by having any function that initiates a worker transaction to return a D-Bus object that the worker and the client that specifically requested the transaction can use to communicate with each other, as opposed to every client instance talking over the same interface and stepping over each others’ toes. This has been bandaged up with some funky workarounds in the Muon Suite to ensure we only respond to events we expect to get, but it’s a “gotcha” that really shouldn’t exist. Changes to the DBus API should be able to rectify this.

Additionally, changes need to be made to how LibQApt communicates with the worker in regards to what changes the worker needs to make to packages. Currently, when a user hits the “apply changes” button to initiate a commit, LibQApt takes a look at changes the user has made to the internal dependency cache, and creates a dict of packages and the state that the user wishes to apply to the package. The worker will then go through this list an manually mark each package in to the proper state in its own dependency cache, before acting on the requested changes. This, as it turns out, can take a long time with large sets of packages and introduces a long waiting period when starting a commit. It also makes it so the worker cannot tell that a package being installed is being installed as the dependency of another package, so the “auto-installed” flag is set to false. When a user later removes the program, the dependencies will not be marked as auto-removable since the QApt Worker didn’t know any better and installed them with the auto-installed flag as false.

To solve this issue, in LibQApt 2 I plan on communicating transactions to the QApt Worker as lists of actions taken by the user, which the worker can apply to its own cache before running the commit. This will not only decrease the amount of time spent marking individual packages by the worker (the APT dependency cache is much faster at figuring those sorts of things out),  but will also ensure that the auto-installed flag is properly set for dependencies of manually installed packages.

There are also a few other things here and there that require ABI/API changes to fix (like renaming functions, changing return types, changing internal class layouts) that have little impact in terms of runtime quality, but will make the library easier to use by developers. A few deprecated functions, signals and slots will also be removed. Depending on how things go, I may also port LibQApt to Qt5, though I’d like to see how that all turns out and don’t want to commit to anything regarding to Qt5 just yet.

Muon Suite Plans

Of course, if API changes are made in LibQApt, Muon will have to be modified to adapt to these changes. This alone would make the next Muon release Muon 2.0. There are some nice new features that have already been committed to Git master for the next Muon Suite release, but these are more incidental and are part of the existing feature development cadence of Muon. What I guess I’m trying to say is that the Muon Suite 2.0 will *not* be a large rewrite of the project, but more of an evolution of the existing codebase adapting to changes in the software stack underneath Muon. I’d also assume that version 2.0 of Muon Suite would use KDE Frameworks 5 and Qt5. This would obviously depend on the time frame within which KF5 would materialize. But hey, maybe part of my work at Blue Systems could include helping with the Frameworks effort!

LibQApt2 or Qt5 porting aside, the current feature cadence will still be maintained within the Muon Suite. Aleix did a good write up of what we have done for 1.5 (or possibly 2.0) here. I’ve adapted the Muon Software Center to the backend changes required for the backend abstraction work that Aleix did, so you’ll be able to play around with the new KNewStuff3 backend in either the Muon Software Center or Muon Discover next feature release. The backend abstraction stuff itself is pretty exciting, and opens up a lot of possibilities for delivering all types of software in a central application.

Kubuntu Plans

It’s no secret that most of my contributions to Kubuntu today come in the form of the development of Muon. (And there’s nothing wrong with that) I still do like to contribute to Kubuntu in other forms, though, like helping the rest of the team package KDE releases, as well as implement blueprint specifications. Sponsorship by Blue Systems will help ensure that I will be able to still contribute in this way, since combining work and FOSS contributions means I will have one less thing to juggle in my life. 🙂


All in all, I am very excited about this opportunity and look forward to the coming months. Of course, nothing here is set in stone. I may come up with more things to do, and priorities may change. One thing is for certain; for me, the future is looking very blue. 😛

29 Responses to Blue Systems, the Muon Suite, and Kubuntu

  1. Congrats Jonathan for your new work. 🙂

  2. I recommend you use apt-fast for the backend, axel is the daddy

  3. John says:

    I still have no idea who blue systems is or what they do or how they stay afloat… :S

    • Markus S. says:

      Blue Systems is basically a guy with lots of cash and no business plan. See [German]

      I find it sad to see that he’s continuing Canonical’s policy to divide the community. Instead of treating Debian as proper Kubuntu upstream and maintain KDE packages there, Kubuntu still hosts its completely independent KDE infrastructure on Launchpad instead of treating KDE packages like all the other packages that simply get imported every 6 months.

      • I think it’s good to make sure KDE has a good presence on Ubuntu. Like it’s good to have KDE on Microsoft Windows – they might be ‘hostile platforms’ (neither company is too fond of KDE and open communities) but they have lots of users and we need to be where the users are.

        Although from a Freedom perspective I’d say it would be bad if the best KDE experience would be on a less-than-fully-open platform. If KDE apps would work better on Windows or Ubuntu than on Debian, Fedora, Arch or openSUSE that would entice users to stay on such platforms. And that is not good for Free Software. But for now that’s not the case, and at least most of what Blue Systems supports gets back to KDE (benefiting ALL platforms) so all power to them.

        I especially appreciate the attention to detail I hope/expect to see from people paid to work on these things. It is something long missing in KDE – attention to the last mile, so to say.

  4. Eduard Gotwig says:

    Why does the Muon suite use Python?

    • It’s used to integrate with Kubuntu’s release upgrade system. (E.g. to jump from 12.04 -> 12.10) which uses python. Also, the K/Ubuntu update notifier uses python.

      • Eduard Gotwig says:

        But I have to say that its quite fast for a python app.

        You made a good job on that.

        I didnt’t knew that Python can be that fast.

        (And I am a Pyhton 3 hobby dev)

      • Very little of it’s actually python. Less than 1 percent of the code in Muon is python, there are just a few bits here and there to interact with the release upgrade code and update notification code from Kubuntu.

  5. Eduard Gotwig says:

    I think they can’t do it becouse of extra traffic for the repos for ubuntu/kubuntu, and the extra data connections that get opened

  6. Markus S. says:

    Why are Apper and Muon two projects? What’s wrong with PackageKit? Why not at least share the GUI between both?

  7. Pj says:

    Wonderful! I have to say that muon is really working quite well. Except maybe one thing… I think the design from muon software manager is a bit odd. Is it planed to maybe redo a custom interface in QML that resembles ubuntu software center (which i think is the best “user friendly” software installer available) more closely?

  8. Nice, just make sure that the wrappers do not slow things down, as is the case with packagekit, which spawns 10 windows and takes a couple minutes instead of 5 seconds in the shell. If I were you, I’d make being as fast as the cli apt my first and foremost goal 🙂

    Congratulations on blue systems as well. Those of you who keep asking what the heck BS are, well, have you ever been an IT student yourself? These guys allow us students to work on what we love at home instead of sitting there in an open-space 9 to 5 (well even half-time) and doing boring proprietary coding. Ain’t that cool?

    • Don’t worry, I made QApt exactly because the abstraction layers around at the time made things way too slow when compared to something as fast as APT. 🙂 Performance has been kept forefront in our minds, and the introduction of the KNS3 Backend hasn’t slowed anything down noticeably. (Obviously any abstraction will cause some amount of indirection, but as long as its done properly)

  9. Kria says:

    what we really need für apt is something like delta-rpm
    But thanks für your work and its good to get payed for it now (:

  10. ronnoc says:

    Thanks for your hard work, and congrats in joining JR and the others at Blue Systems. I must say that the future of Kubuntu is brighter than ever. Combined with the (hopefully) newfound stability with Qt, I’m a very happy Kubuntu user.

  11. Mark says:

    Will there be a function for autoupdates in future? That is something I really miss. I have installed KDE on several machines for friends. But nobody ever really cares for the updates. Which is something I can understand since it is done in Windows automatically. The problem I guess is that APT sometimes wants to remove important packages, so a manual check is required. If somehow the updates would only work if the updates do not require to remove (important) packages that would be a huge step forward.

    • JanC says:

      Install the ‘unattended-upgrades’ package and see ‘/usr/share/doc/unattended-upgrades/README’ for how to configure it.

      If it isn’t there already, it would probably be good if this could be configured using Muon and/or some configuration tool in Kubuntu. (I’m not a Kubuntu user.)

  12. Yuriy says:

    Congratulations, Jonathan! Glad you’ll have funding for all the great work you’re doing on Kubuntu and have been doing for many years now. Also I’m glad you’ve stepped up to the task of a good package management library and UI for KDE — this has been a constant problem with many half-baked solutions for a decade.

  13. Mitch says:

    Congratulations. I am a big fan of Muon. Keep up the good work.

  14. ikeahloe says:

    Will there ever be an ability to have hooks into the Ubuntu Software Center so that Muon can be used to download paid applications like the humble indie bundle or some of the business software available. Maybe the paid applications/USC part could be a plug-in so that the application can still be used on non ubuntu based distros

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: