#But...why?
As a bit of a backstory, I have been writing more C++ recently in the form of the new implementation of the budgie-daemon. The focus has been on output management under Wayland (so leveraging Wayland protocols) and all fancy DBus bits with sdbus-cpp, for the purpose of providing the plumbing for display configuration in Budgie Control Center as part of Buddies of Budgie's move to Wayland for Budgie 10 (and beyond). Meanwhile, Campbell (serebit) on the team has been writing Budgie Desktop's Wayland compositor (leveraging wlroots) in C++ and I would like actually be useful in that area as well at some point.
Putting the two together and you end up kinda living C++ a fair bit, so that played a part in the decision-making process.
As I have been working on that daemon and alongside the rest of the team engaging with more folks from more projects such as KDE, Cinnamon, and XFCE -- you can't help but get inspired by all the mutual goals on Waylandifying the ecosystem and fostering cross-project collaboration. Koto plays into that a bit with some discussions I had with Clément on XApps before it became a thing (it being similar to the "Modern Desktop Initiative" I had basically only announced on my Patreon back in 2021 with a similar goal), as many of us in the community would like to see more applications that are not adversarial to each other's desktop environment, but rather strive to make those applications feel as native to that environment as possible.
Expanding on that, at least a personal vision of mine is building a Budgie Desktop platform that promotes a composable experience (not just for us, but third-parties not even involved in Budgie Desktop). Composable desktop experience + composable application / ala carte experience? Sounds like a win to me.
While Koto may not necessarily be an XApp, as at least at the moment there is no real definition on what that even is, in spirit the goal is for it to be.
Yet all of the above was still not enough to actually make me say "alright, I think it is time to resurrect Koto and build something cool". I had half-heartedly mentioned in the team standups that I might do it at some point, but in the same vein one would say "yea I should really go dive into that icy lake in the middle of a cold Finnish winter". Never done that despite living here for over 10 years, so I think the comparison is pretty apt.
What finally got the wheel turning was Jolla's announcement on the Jolla C2 Community Phone, running Sailfish OS 5.0.
#If you wish bake an apple pie from scratch, you must first invent the universe.
Fortunately I did not have to invent the universe however I did have to figure out what ingredients should go into my apple pie and how to bake it. All while livestreaming like my old Koto streams! The old Koto was built with:- GTK as the toolkit (emphasis on the kit part, foreshadowing here)
- GLib, lots of stuff but most notably GDBus for media keys and MPRIS support
- gstreamer-1.0 and gstreamer-player-1.0 for the playback
- sqlite3 (C API) for the database
- taglib for id3v2 metadata parsing
- tomlc99 for the configuration (TOML)
- Sorting out CMake to be used as the build system. As much as I love Meson, the Qt6 module simply is not at feature parity and that is okay. I took the opportunity to look at KDE's Extra CMake Modules (ECM) which provided a ton of useful functions related to QML.
- Figuring out at a high level what libraries, APIs, and UI library / libraries I would use. Which, if any, of the libraries I had used were directly translatable over from the C world. What would I need to do from scratch and what work could I leverage from others.
- Stretch goal: Some initial QML UI bits. Nothing fancy.
- Qt6 as the toolbelt
- Qt6 DBus (including qdbusxml2cpp) or alternatively using sdbus-cpp (including sdbus-c++-xml2cpp) like I do in Budgie Daemon v2 for generating C++ code from the XML interface definitions, as well as the DBus client / server bits
- Media playback: No need to interface with gstreamer-1.0 / gstreamer-player-1.0 C APIs, there is QtMultimedia and QMediaPlayer
- sqlite3? Yea I was about to do that by hand but to nobody's surprise, there is an entire the Qt SQL module and its APIs are great. You are not necessarily limited to sqlite3 either as you can just as easily use the MariaDB, PostgresSQL, or other database drivers
- In the monorepo, have a Sailfish OS build variant that is in a completely different toplevel sub-directory to the desktop variant (which I put under
./desktop/even though yes it could be used for Plasma Mobile) that is written in Qt5. - Leverage some very recent work by Sailfish OS community members piggz and rinigus on Qt6 packaging, which conveniently enough was done so piggz could also implement a music player (though for Subsonic) using Kirigami! Doesn't magically fix the Silica part though.
#You smell what I am cooking?
Okay, one can only do so much research and planning. No plan survives contact with the ~~enemy~~ IDE. I will skip over all the literal hours of me fighting with different editors (Zed, Visual Studio Code, CLion, Qt Creator, etc.) and just say that the most consistent and enjoyable editor experience for working on Koto has been CLion. Just remember to use the "Reload CMake Project" option if it ever hurts itself in confusion.
Great, now we have (checks notes) almost 21 hours of content to get through, so I will try to condense this down a fair bit.
#Day 1
The first proper day of development was spent on a mix of things. I wanted to get some initial UI bootstrapped just to get the gears turning on that before switching gears to the configuration and data lake (build or extract from one or more data sources, transform it into a format for Koto, and load it into the application). For the user interface, I got acquainted with some useful applications suggested by the livestream: Kirigami Gallery (gallery of Kirigami widget examples) and Icon Explorer (self-explanatory). I built some initial primary navigation, mirroring the primary navigation of old Koto ("if it ain't broke, don't fix it"). Before I could start work on building the file indexer, I first needed to know what content I was going to index to begin with. Aside from "sane defaults" like XDG_MUSIC_DIR, that meant implementing the configuration parsing and defining the structure for the config itself. I settled on the below config format, which is essentially identical to old Koto.["preferences.ui"]
album_info_show_description = true
album_info_show_genre = true
album_info_show_narrator = true
album_info_show_year = true
last_used_volume = 0.5
[[libraries]]
name = "Music"
path = "/home/joshua/Music"
type = "music"
#Day 2
The second day of Koto was spent building out the file indexer and let me tell you, this was considerably easier thanks to QDirIterator compared to what I had to do in GLib land. Of course, it helped that I had done all this before, I used KFileMetaData instead of taglib directly, and I am yet to implement all the cool tricks for manual filename parsing that I still want to do for handling audiobooks and tracks without proper ID3v2 metadata. All that said, things still things went much more smoothly than I was expecting. Within just that day's work, I was able to implement a functional file indexer that showed my tracks, in the correct albums, for the correct artists. I had even built out a similar "Cartographer" class to my old Koto that held all the hash tables, using QHash with QUuids to ease lookups during indexing. 🎉 That day, we cooked.#Day 3
On the third day, the focus was on taking our indexed files, storing them in a sqlite3 database, and reading them back from the database as part of the application loading. As I mentioned earlier in the blog, my original line of thinking was that I was probably going to use sqlite3 directly. After all, why would I expect to be blown away yet again by Qt having something very usable out-of-the-box? Of course, as you have already learned, I fortunately discovered Qt SQL and they have some really good guides to get started with the basics of connecting to a database, executing queries, and reading with records. With QSqlQuery, I was able to trivially writecommit functions for all my relevant classes (KotoArtist, KotoAlbum, and KotoTrack) with binding values. You still have to write the queries of course, as it is not an ORM, but it absolutely beats out the madness I was doing with g_strdup_printf with a function that basically called sqlite3_exec.
Was there a better way of doing it back with old Koto? Probably. But I am just comparing old Koto with the new. The fact is that the binding values method of accomplishing it with QSqlQuery is well documented and not hidden away, so the "nice" (in my opinion) way of doing it was the most obvious one.
Code comparison of KotoTrack on old versus new:
gchar * commit_msg = "INSERT INTO tracks(id, artist_id, album_id, name, disc, position, duration, genres)" \
"VALUES('%s', '%s', '%s', quote(\"%s\"), %d, %d, %d, '%s')" \
"ON CONFLICT(id) DO UPDATE SET album_id=excluded.album_id, artist_id=excluded.artist_id, name=excluded.name, disc=excluded.disc, position=excluded.position, duration=excluded.duration, genres=excluded.genres;";
// Combine our list items into a semi-colon separated string
gchar * genres = koto_utils_join_string_list(self->genres, ";"); // Join our GList of strings into a single
gchar * commit_op = g_strdup_printf(
commit_msg,
self->uuid,
self->artist_uuid,
koto_utils_string_get_valid(self->album_uuid),
g_strescape(self->parsed_name, NULL),
(int) self->cd,
(int) self->position,
(int) self->duration,
genres
);
if (new_transaction(commit_op, "Failed to write our file to the database", FALSE) != SQLITE_OK) {
return;
}
QSqlQuery query(KotoDatabase::instance().getDatabase());
query.prepare(
"INSERT INTO tracks(id, artist_id, album_id, name, disc, position, duration, genres) "
"VALUES (:id, :artist_id, :album_id, :name, :disc, :position, :duration, :genres) "
"ON CONFLICT(id) DO UPDATE SET artist_id = :artist_id, album_id = :album_id, name = :name, disc = :disc, position = :position, duration = :duration, "
"genres = :genres");
query.bindValue(":id", this->uuid.toString());
query.bindValue(":artist_id", !this->artist_uuid.isNull() ? this->artist_uuid.toString() : NULL);
query.bindValue(":album_id", this->album_uuid.has_value() ? this->album_uuid.value().toString() : NULL);
query.bindValue(":name", this->title);
query.bindValue(":disc", this->disc_number);
query.bindValue(":position", this->track_number);
query.bindValue(":duration", this->duration);
query.bindValue(":genres", this->genres.join(", "));
query.exec();
id and you use QSqlRecord functions for casting it directly to the type you need like QString or int. No sorcery needed.
KotoTrack* track = new KotoTrack();
track->uuid = QUuid {query.value(record.indexOf("id")).toString()};
auto artist_id = query.value(record.indexOf("artist_id"));
if (!artist_id.isNull()) { track->artist_uuid = QUuid {artist_id.toString()}; }
auto album_id = query.value(record.indexOf("album_id"));
if (!album_id.isNull()) { track->album_uuid = QUuid {album_id.toString()}; }
track->title = QString {query.value(record.indexOf("name")).toString()};
track->disc_number = query.value(record.indexOf("disc")).toInt();
track->track_number = query.value(record.indexOf("position")).toInt();
track->duration = query.value(record.indexOf("duration")).toInt();
track->genres = QList {query.value(record.indexOf("genres")).toString().split(", ")};
#Day 4
Congratulations Josh, you have files in a database, can we show them in the GUI now?
No apparently that was the less straight-forward part and probably a part of the Qt6 documentation that needs some work. To facilitate this, you need to ensure your C++ classes inherit from QObject, use various macros like: Q_OBJECT, QML_ELEMENT and optionally QML_SINGLETON; be careful to not accidentally instantiate more than one instance of what you were expecting to be a singleton class, etc. I took the opportunity to change how we store references to KotoArtists from a QHash to a class (KotoArtistModel) inheriting from QAbstractListModel. The benefit of this is direct integration with QML ListView models.
That all sounds pretty simple when it is all put that way but the journey extended beyond the stream and fortunately George (the same one mentioned earlier!) was kind enough to file an issue explaining why my ported Cartographer class was not able to show artists in the QML ListView I made...well to list artists. Thank you very much, the screenshot for the featured image would be less exciting otherwise.
Some guides if you are interested:
#Summary
To summarize the work itself, I would say I have been very pleased with what has been accomplished over the course of those few days working on Koto and a couple evenings of tinkering. I was able to get to the point of actually listing artists, albeit after the latest session. There is still a mountain of work ahead such as: the artist view (displaying albums, tracks within those albums), playlists, and of course the actual playback of content; however I think thanks to my past experiences with old Koto, Qt being a more feature-rich tool~~kit~~belt, KDE Frameworks being excellent, and some very patient people -- I will be able to get back to feature parity with old Koto sooner than even I probably imagine. Maybe it is a honeymoon phase. Maybe the grass really is greener on the other side. No matter, I am very happy I resurrected Koto and I look forward to all of the new and interesting problems I encounter. Maybe one day you will even be able to get it on the device now resting on my monitor shelf.
So here's to more Dev Diaries! 🍻