GNOME

It’s been around two months when I officially became a GSOC student Developer at Pitivi and now, the 1st coding month has completed. Although we had a structured proposal to follow during GSOC, we adapted as per what looked much more suitable and ... [More] made more sense. If you have been using Pitivi, you are in for a surprise. Old Render Dialog-1 The first task we were looking forward to was replacing the Render Settings Dialog with a Render Perspective. However, after trying different designs, we thought of keeping the original design and include something new to it. To make the settings much more intuitive, we moved forward with working on the presets section of the render settings. Old Render Dialog-2 Previously, the preset selection consists of a combo box, to choose the preset profile to apply. We were thinking of changing it to something similar to what DaVinci Video Editor tool presents, that is, a bar with all preset profiles available. But there was limited space in this design which created problems with providing more info to user and making most out of the preset selection menu, therefore, we decide to innovate a little. Da Vinci Preset bar My mentor, Alexandru suggested me this great design with popular Gtk Popover. As compared to the previous design, this made us present much more information to the users about these profiles in lesser space and made the design easier to interact with for users. We replaced the combo box with a button which highlights the selected profile along with the corresponding icon. When this button is clicked, a list of preset profiles is presented using the GTK Popover. For every preset profile, we have included an icon along with a brief description of the profile and path to where the profile file is saved. Updated Render Dialog-1 We asked for the help of GNOME’s Design Team and Pitivi community members for their feedback on the new design. The icon is an important part of each profile as it’s making the profile easier to identify visually. For example, for ‘YouTube’ profile, we have used the YouTube branding logo, thanks to YouTube for allowing us to use the logo. While for other profiles we have used the GTK-icons. Updated Render Dialog with popover To be able to allow the user to remove an existing preset, we also added another property path to the GST-plugins-base code in file encoding-target.c. You can check the MR here. So, if you have been working with this project, feel free to use it as per your need. I have learned a lot during this period and hope to learn more over the upcoming months. It feels great to know that my work would help thousands of Pitivi’s users. Looking forward to you trying these features out and give me feedback! Thanks! [Less]

I began work on GNOME Gingerblue on July 4th, 2018, two years ago and I am going to spend the next four years to complete it for GNOME 4. GNOME Gingerblue will be a Free Software program for musicians who would compose, record and share original ... [More] music to the Internet from the GNOME Desktop. The project isn’t yet ready for distribution with GNOME 3 and the GUI and features such as sound recording must be implemented. At the moment I have only released GNOME Gingerblue version 0.1.4 with very few features: Song Files in $HOME/Music/ Setup Wizard XML Parsing The GNOME release team complained at the early release cycle in July and call the project empty, but I estimate it will take at least 4 years to complete 1.0.0 in reasonable time for GNOME 4 to be released between 2020 and 2026. The Internet community can’t have Free Music without Free Recording Software for GNOME, but GNOME 4 isn’t built in 1 day. I am trying to get gtk_record_button_new() into GTK+ 4.0. I hope to work more on the next minor release of GNOME Gingerblue during Christmas 2020 and perhaps get recording working as a new feature in 0.2.0. Meanwhile you can visit the GNOME Gingerblue project domain www.gingerblue.org with the GNOME wiki page, test the initial GNOME Gingerblue 0.1.4 release that writes Song files in $HOME/Music/ with Wizard GUI and XML parsing from August 2018, or spend money on physical goods such as the Norsk Kombucha GingerBlue soda or the Ngs Ginger Blue 15.6″ laptop bag. [Less]

GNOME Internet Radio Locator 3.0.1 features updated language translations, new, improved map marker palette and now also includes radio from Washington, United States of America; WAMU/NPR, London, United Kingdom; BBC World Service, Berlin, Germany; ... [More] Radio Eins, and Paris, France; France Inter/Info/Culture, as well as 118 other radio stations from around the world with audio streaming implemented through GStreamer.  The project lives on www.gnomeradio.org and Fedora 32 RPM packages for version 3.0.1 of GNOME Internet Radio Locator are now also available: gnome-internet-radio-locator.spec gnome-internet-radio-locator-3.0.1-1.fc32.src.rpm gnome-internet-radio-locator-3.0.1-1.fc32.x86_64.rpm [Less]

Hello! My name is Günther Wagner and i try to give some insights in the current development of GNOME Builder. All these changes can already been tested with GNOME Builder Nightly so go ahead and give us feedback! This newsletter is called “This week ... [More] in …” but probably we won’t post every week. So the interval will be a little bit arbitrary. Let’s start! New Features We included a new code spellchecker plugin leverage the fantastic codespell-project. [Less]

During the past month I have been hacking on Epiphany’s Preferences dialog. The first piece of submitted work was splitting the dialog source code files into smaller ones. The split didn’t reflect any visual changes on Epiphany’s user interface so I ... [More] decided to postpone writing this blog post. Personally I prefer to have some form of visual content in my blog posts That leads me to the second piece of submitted work which does include modifications in Epiphany’s interface. If a picture is worth a thousand words, then a gif is worth a million so I’ll use them to illustrate the changes This is how the Passwords dialog was invoked before the latest commits: The main disadvantage with this method was that it would spawn a dialog from within another dialog which should be avoided, as explained in the original Gitlab issue which I used as a reference. Passwords is now a view nested inside the Preferences dialog and is presented like this: This approach also has the benefit of being intuitive on mobile and touch devices. When inside the Passwords view the user can swipe back to return to the main Preferences view. Lastly, instead of clicking on the small gear button, the user can now click/tap anywhere on the whole Passwords row inside the Privacy page in order to invoke the view. A more subtle change is the red Clear all button which has been moved inside a GtkActionBar at the bottom of the view. The reasoning behind this was also concerning touch devices as in the previous layout it could have been very easy to tap the Back button instead of the Clear all button or vice versa. Another benefit is that the Clear all label is a bit more explicit than the trash icon. Cookies were merged into Personal Data Epiphany used to have a separate view for clearing Cookies but now they have been moved into the Clear Personal Data view because of the following reasons: The previous Cookies view was slow as it contained a large GtkListBox Cookies are actually a category of stored data Next up These have been the most substantial recent changes worth mentioning in this post. Stay tuned for upcoming news regarding the History dialog ! Lastly, a thanks message to this year’s project mentors, Michael Catanzaro and Jan-Michael Brummer, and also thanks to Alexander Mikhaylenko for helping out with guidance in how to use libhandy ! [Less]

In my last blog post, I explained how selection mode was implemented in Games. That was one of the first steps to support Collections in Games, as an efficient way to select games to add/remove from collection is crucial for managing collections. In ... [More] this post I’ll be talking about how “Favorites Collection” will be implemented in GNOME Games. Collections The first thing to do was to introduce a Collection interface to define a behavior that all types of collections must follow. All collections must have an ID and a title. Apart from that, all collections must provide a way to add and remove games from it. And on adding or removing a game from the collection, it should emit a “game added” or “game removed” signal respectively. A collection must also implement a load(), which when called, should load the games belonging to a collection from the database. Since there’s going to be different types of collections, how a collection has to be loaded might differ from each other. Every collection has its own GameModel and must implement a get_game_model(). A GameModel is a ListModel which stores the list of games in a collection, and get_game_model() returns its GameModel which can be bound to the flowbox of a GamesPage (a widget where games can be displayed with thumbnail and title). Other than these, all collections must also implement on_game_added(), on_game_removed() and on_game_replaced(). These are unrelated to games being added or removed to or from a collection. These has to do with games being discovered, and when some games are no longer available to the app. When a game is discovered by tracker or a cached game is loaded, it is added to a games hash table. This emits a game_added signal (unrelated to a collection’s game_added), which every collection listens to. If the added game belongs to the collection, it adds this game to the collection. Similarly on_game_removed() and on_game_replaced() handles stuff related to when a game which was cached but is no longer found by the app, and when a game has been renamed, moved to a different directory, or when it’s still the same cached game but with different UID etc. With the general behavior of a collection defined, it was time to introduce a FavoritesCollection which implements Collection. Favorites Collection As obvious from the name, a “Favorites collection” is a collection that stores games that a user marks as favorite. Favorite games are marked with a star icon on the thumbnail. A game can be added to favorites from the main Games Page or from the Platforms Page. A game can removed from favorites from the above said pages as well as from the Favorites Collection Page. Games are added or removed from favorites “automagically” depending on the list of currently selected games. If all of the selected games are favorite then clicking on the favorite action in the action bar will remove them from favorites. If none of the selected games are favorite, then they are added to favorite. If selected games are a mix of favorite and non favorite games, then all the non favorite games are added to favorites. The icon in the favorite action button in the action bar, dynamically changes from starred to semi-starred to non-starred depending on the selected games. This along with tool tips help users know what the button will do. Collections: Behind The Scenes Database So once a FavoritesCollection was ready to go, I needed to work on how it will be stored in the database and how it should load those collections. Favorite games are stored in the database by having a new is_favorite column in the games table. games table stores all games, including “manually added” games, and games found by tracker. So adding or removing any type of game from Favorites is a matter of updating the is_favorite column in games. However, I can’t just simply add a new column in the games table creation query. In order to migrate from the old database with no is_favorite column to the new one with the column, I’ll have to give some extra commands to the database. This shouldn’t be done always, but should depend on the version of the database. As of now, there isn’t a database versioning system in Games, so a simple database migration support was quickly implemented. This migration leverages a .version file in the data directory. This was used to migrate from old data directory structure to a newer one. However the file is empty and data directory migration (not database migration) only checks if the .version file exists or not. This is fine for a one time migration support. But having versions in the .version file might come in handy later on, so I now configured the database migration to write the version number into the .version so it can be used later on. No .version file is treated as version 0, empty .version is treated as version 1, and from version 2+, the .version file will have the version number in it. So in this use case, the database migration to support favorite games should be applied for all versions less than 2. Luckily, migration in this case only required to drop the games table and create it again with the new is_favorite column :). And after applying this migration, bump the database version and write it into the .version file. Now any future migrations may make use of this .version file too. The database allows to add, remove, fetch complete list of favorite games, and check if a game is favorite. That’s about it for Favorites. For other types of collections, which will be introduced in the upcoming days, they would mostly be stored in different tables. Collection Manager As of now, the only available collection is Favorites Collection. But as said, soon there will be “Recently Played” and custom collections that users can create and manage. So when a CollectionManager is introduced, it has to be designed keeping in mind that it has to handle all types of collections. But most of the collection specific behavior can be neatly abstracted. CollectionManager handles the creation of all types of collections. It consists of a hash table to store all collections. And as of now, it has a FavoritesCollection object. Since any actions related to a collection is better to be handled by CollectionManager it also provides a favorite_action() which accepts a list of games and depending on it adds or removes them to favorites. Apart from that, on creation of CollectionManager, it calls load() on every collection in the hash table, which loads all the games that belongs to that particular collection from the database. Collection Model It is a ListModel that contains all the available collections, which can be bound to the CollectionsPage’s flowbox. CollectionsPage (similar to GamesPage) is a widget which displays available collection to the user with thumbnails and title. Collections: What You See With all these put together we get a functional Favorites Collection. Here are some pictures: Games Page. With a star in the thumbnails of favorite games Collections Page. Might look a bit lonely but will soon be accompanied by other collections : ) Favorites Page. This is more or less how any other collection would look like too. As you can see, the notable visual changes here are: A new Collections Page which can be navigated to, from the view switcher A star on thumbnails of games marked as favorite, in Games Page and Platforms Page A collection thumbnail which is generated from the covers of games that belong to that collection A Collection (Sub)Page which when clicked opens a page with games that belong to that collection By the way, you can also see the new game covers with the blurred background that I was experimenting with in the last blog post. So this is where my progress is at currently. You can see the relevant MR here. Whats Next? In the upcoming weeks I’ll be implementing the rest of the collections. Currently my plan is to implement “Recently Played” collection next as it would the simpler one to implement next. There are also some smaller stuff which needs some work such as search and selection support for Collections. But that is planned to be at the end, after introducing all types of collection. Conclusion The work is going great and all the challenges have been fun to solve so far. Many thanks to my mentor, Alexander for being very helpful as always. Thank you all for reading my blog posts. See you next time :) [Less]

Coding period for GSoC 2020 has started and I have begun my work on my summer project. As said in my introductory post, I will be working on adding functionality to create and manage game collections in GNOME Games with help from Alexander (@alexm). ... [More] After the project is complete, it will provide users with a shiny new ability to add any games to their own custom collections. And some additional feature to provide users with a quickly accessible, automatically generated collections such as recently played, favorites and hidden games. I started out by separating the work into independently manageable chunks so that I can open several smaller merge requests, rather than a single large one, which I can imagine would be horrible to manage, and even worse for Alexander to review. And my code, however small it is, usually needs a lot of fixing. So the first chunk I decided to work on is… Selection Mode! I decided selection mode would be the best part to start with so that when I get to modifying the database part to store all the collections and the games in it, I will have all the necessary functionality to test it with actual real world data rather than some made up data using temporary spaghetti code. Selection Mode Selection mode will help the user select games to be added to their collections. It may also be useful later on for anything that requires a user to choose a list of games (maybe even a list of collections), since its not tightly integrated with collections. Selection mode in Games is just like in most other apps. There will be a button in the header bar which can be clicked to enable selection mode. It changes the header bar into a blue “selection-mode” styled header bar with options to search, change selection modifier and cancel the selection mode. On to implementation details, I use GtkButtons for the selection mode and cancel buttons. A GtkToggleButton for the search which already works as is. A GtkMenuButton for the selection modifier with select-all and select-none actions. A simple is-selection-mode prop which is bound, propagated and listened to, for it all to come together. And when selection mode is enabled the header bar changes, and the GtkCheckButtons in an overlay inside the game thumbnail reveals itself smoothly. Each game has it’s thumnail in a GameIconView, and several of them are children to a GtkFlowBox. And GameIconView has a checked property which when set automatically handles all the stuff of adding/removing games to a GenericSet named selected_games. Since Vala arrays do not have an inbuilt remove element function, Alexander directed me to GenericSet which works nicely here. The select all action can be used to select all games in the current page. Which means that apart from selecting all games in the main games page, it can also select all games that only belong to currently selected platform, or select all games that only match the user’s search query. Since Games uses Libhandy (A very cool library for adaptive and sleek widgets and UI), its a very adaptive app, and the selection mechanism requires some slight tweaks. At first I had some trouble with all the edge cases when the HdyLeaflet folds from desktop view to a more compact UI. Thanks to Alexander, I realized I shouldn’t be manually changing a lot of unrelated props in unrelated places, and that made the code much much better with a lot fewer bugs. And now, I present to you the current state of selection mode: Some things to note: While most of the selection code is ready to go, it still isn’t really complete without a bottom bar which presents the user options to add games to a collection, or mark it as favorite/hidden. This will be done in the up coming weeks with work on the database and CollectionsPage For those nice blue checkboxes to work without any workarounds, it needs a little Adwaita theme fix. Alexander helped me open my first MR to GTK which will remove the need for any css style workarounds to get that blue checkboxes for use cases such as in this case. There is a slight hiccup with selecting multiple games quickly. It happens even when input is “slow enough” for some real world usage, so this might annoy some users who are a bit quick with their inputs. But we couldn’t yet pinpoint what causes it, but it happens with other apps that use GtkFlowBox too, so probably not something that Games is doing. The next part to work on as said above would be modifying the database to support collections. It will need new queries to add games to collections, get games in a certail collection, delete, rename etc. And then I will work on a CollectionPage where users can see Recently Played, Favorites, and then their custom collections and manage it. I think I’m moving in the right pace, maybe even a bit fast, but thats alright I guess, becuase my exams can come up at any day in the upoming weeks, and no one really knows exactly when, due to recent events. You can view the selection mode MR here. Other stuffs I’ve been doing If you are still here, I can share some other stuff I’ve been poking around with :) Finally scratched my itch to checkout OpenGL Some games/emulators produce a rotated display output which makes the game unplayable. So I thought I’d give a shot at trying to implement display rotation for Games. Games uses retro-gtk which is a GTK+ Libretro frontend framework. It helps Games work with libretro games/emulators. So that’s the place where I should poke around with to implement display rotation. I honestly don’t know what I expected, but I played with the first thing that I suspected and voila, it actually worked, the output was rotated. It turned out that I was playing with texture coordinates and simply “cycled” it and the display was rotated. The End…except no, it isn’t that simple, because that’s a bit of a hack™. What I did, could technically be used, but it’ll not flexible later on. One particular use case suggested by Alexander was adding animations on rotating, so that its more elegant. So I should look for a more flexible solution. And so began my journey in lands of OpenGL. I never knew a thing about OpenGL, but I was always interested in how graphics/video cards worked. And I have all the time I need, since its a lock down due to COVID-19. And the solution was to use GLSL shaders and framebuffer objects or shortly fbo. So shaders are little programs that run on the graphic cards which is incredibly parallel. So to rotate a texture (think of texture as a game’s output in this case), I need to write a simple vertex shader which rotates every vertices (here 4 since its a rectangle) of the texture, and a basic pass-through fragment shader (which is a copy pasta since I don’t need to modify the pixels in the texture). So I do that. But I cant just use these shaders yet, because retro-gtk use another set of shaders to support video filters such as smooth, CRT effects which can be applied on top of the game’s video output for a retro feel. So I will need to first render the output with video filter effects and then rotate it using my brand new shaders. And for that I need to use an offscreen framebuffer. Think of framebuffer as something that holds information about color of pixels, and how “deep” a pixel is from the screen etc. For this use case, really think of it as a temporary place to store a texture. Because what I did was to draw the texture with video filter onto the offscreen framebuffer, then use our shiny new rotate shader to rotate the output in the offscreen framebuffer and “draw” it onto the default framebuffer which is what you see on the screen. This method of rendering to an offscreen framebuffer for sampling it in the next stage of the pipeline is often called “render to texture”. I was stuck (blank output) at rendering to the default framebuffer until Alexander helped me understand that at the final stage I should bind to GtkGLArea’s framebuffer and not the default framebuffer, since retro-gtk uses a GtkGLArea for rendering with OpenGL. Other issue I was stuck with for a month was that rotating corrupted the window’s headerbar in client side rendering. I was very frustrated with this bug, because it seems to be working but something that is unfixable by me made it unusable. Or so I thought. Again thanks to Alexander, I realized I shouldn’t call OpenGL functions outside functions provided by GtkGLArea. Finally with it all working here’s a demo: Hopefully it will be available soon. Regardless, this was an awesome experience. I worked in C, a short introduction to OpenGL, and learned basic dynamic memory management. You can see this work here. A Cairo Adventure This is one of my recent projects. I felt Game’s thumbnail covers could have a bit more eye candy instead of the letter-boxed covers which you can see in my first selection mode picture. So I thought of blurred enlarged background of the cover behind the cover for the 1:1 cover aspect ratio instead of letter-boxing it. Alexander gave me the green light and I began search for how to do it. I assumed someone on the internet already had this done blur with cairo. So I created a simple app to test out different blur implementations. But most of them did not produce the right effect we were looking for and I was too lazy to tweak it for the required effect. And after trying about 4 different implementations I almost gave up. But then Alexander directed me at GTK’s own blur used for drawing shadows. So I tried it out and it kinda works, but only that the colors go away and picture becomes wavy with higher blur radius. That’s because I had commented out an assert that made sure the image I give it is in A8 format (only alpha), so that it doesn’t crash when I give it RGB24 images just for an experiment :p. But I had an idea. In the hopes that GTK’s cairo blur was initially implemented in RGB24 I checked out the gtkcairoblur.c’s history, and… it was! It supported ARGB32, RGB24, and A8. After some quick ctrl+c ctrl+v which is what I’m good at, It worked! And now I need to port it to Vala so that it can be used in Games (without extern funcs). You can see my work up until now here. I added flatpak and CI support cause why not :). And finally here’s a sample of what the covers would look like with blurred background: Conclusion I’m really having a great time contributing to GNOME. It’s exciting and the community is very nice and helping. And especially thanks to my mentor, Alexander, for all the help. I’ll try blogging at least once or twice a month :). But until then goodbye! [Less]

Readers be advised, this is somewhat of a deep dive into the guts of Mutter. With that out in the open, lets start! Not too long ago mutter saw a merge request land, that has one major aim: split up the frame clock so that when using the Wayland ... [More] session, each monitor is driven by its own frame clock. In effect the goal here is that e.g. a 144 Hz monitor and a 60 Hz monitor being active in the same session will not have to wait for each other to update, and that the space they occupy on the screen will draw at their own pace. A window on the 144 Hz monitor will paint at 144 Hz, and mutter will composite to the monitor at 144 Hz, while a window on the 60 Hz monitor will paint at 60 Hz and Mutter will composite to the monitor at 60 Hz. glxgears on a 75 Hz monitor next to weston-simple-egl on a 60 Hz monitor. All of this is roughly achieved by the changes summarized below. Preface In the beginning of times, Clutter was an application toolkit. As such, it assumed (1) the existence of a window compositor, and (2) the compositor is a different process. Back then, Wayland was still in its early infancy, and those assumptions wouldn’t conflict with writing a X11 window manager. After all, a X11 window manager is pretty much just another client application. Over time, however, Clutter started to grow Wayland integration in itself. Deeper and deeper surgeries were made to it to accomodate it being used by a Wayland compositor. In 2016, the Cogl and Clutter codebases were merged with Mutter codebase, and they all live in the same repository now. However, to this day, relics from the time when Clutter was an application toolkit are still present in Mutter’s Clutter. One such relic is ClutterMasterClock . ClutterMasterClock ClutterMasterClock was the main frame clock that drove Clutter painting. As an application toolkit, only a single, global frame clock was necessary; but as a compositor toolkit, this design doesn’t fit the requirements for multi-monitor setups. Over the last cycles, there has been some attempts to make it handle multiple monitors slightly better by juggling multiple monitors with their own refresh rates and clocks using various tricks, but the fundamental design was standing in the way for making substantial progress, so it has been completely decommissioned. Enters ClutterFrameClock. ClutterFrameClock is the new frame clock object that aims to drive a single “output”. Right now, it has a fixed refresh rate, and a single “frame listener” and “presenter” notifying about frames being presented. It is also possible to have multiple frame clocks running in parallel. However, ClutterFrameClock alone isn’t enough to achieve independence of monitor redraws. Stage Views Mutter has a single stage that covers the union of all monitor rectangles. But how does it render different contents to each one of them? That’s one of the main responsibilities of ClutterStageView. ClutterStageView was the answer to the need of drawing the stage at different framebuffers. ClutterStageView corresponds roughly to one monitor. Each ClutterStageView holds the on-screen framebuffer that the monitor displays; if using shadow framebuffers, ClutterStageView also handles them; and finally, it also handles the monitor rotation. Now, ClutterStageView also handles the monitor’s frame clock. By handling the frame clock, each view is also responsible of notifying about frames being presented, and handling the frame clock dispatching The frame scheduling related logic (including flip counting, schedule time calculation, etc) was spread out in ClutterMasterClockDefault, ClutterStage, ClutterStageCogl, MetaRendererNative, MetaStageNative, and MetaStageX11, but has now now been concentrated to ClutterFrameClock and ClutterStageView alone. Actors, Actors Everywhere When animating interface elements, the core object that does that is ClutterTimeline and its subclass, ClutterTransition . Timelines and transitions saw frames whenever the master clock ticked. With the master now clock gone, they need to find an appropriate frame clock to drive them. In most (and after this change effectively all) cases a timeline was used to directly drive an animation related to an actor. This indirect relationship is now made explicit, and the timeline uses the actor to find what stage view it is being displayed on, and with that information, picks an appropriate frame clock to attach to. For transitions, used extensively by GNOME Shell to implement animations, this is handled by making a ClutterAnimatable provide the actor, and for stand-alone timelines, it’s a property set directly on the timeline before it’s started. This means that when an actor moves across the stage and enters a different stage view, the timeline will be notified about this and will decide whether to migrate to a different frame clock. What About X11? In the X11 session, we composite the whole X11 screen at once, without any separation between monitors. This remains unchanged, with the difference being where scheduling takes place (as mentioned in an earlier point). The improvements described here are thus limited to using the Wayland session. Be aware of API changes This is quite a substantial change in how painting works in mutter, API changes could not be avoided. With that in mind, the changes needed are small, and mostly handled transparently by GNOME Shell itself. In fact, in all of GNOME Shell’s Javascript code, only two places needed change. To be specific, for extension developers, there are two things to keep in mind: If you use a St.Adjustment. You must now pass an actor when constructing it. This actor will determine what frame clock will drive the adjustment. Some signals saw their type signatures change, namely ClutterStage::presented, ClutterStage::after-paint. Final Thoughts This is a big achievement to Mutter, GNOME Shell, its users, and especially to the contributors that were part of this. The road to reach this point was long and tortuous, and required coordinated efforts of dozens of contributors over the course of at least 5 years. We’d like to take a moment to appreciate this milestone and congratulate each and every single contributor that was part of this. Thank you so much! [Less]

The volunteers and contributors working on Mutter and GNOME Shell have been busy in the past couple of months — so much so that we didn’t have bandwidth to write the May development report! As a consequence, this development summary will have an ... [More] above average number of changes to highlight. GNOME Shell Preparations for Customizable App Grid As part of the preparations for a customizable application grid, a new layout manager was written and replaced the current icon grid code. This new layout manager is in many ways more suitable for current and future changes: It follows the delegation pattern that is common to Clutter. As such, it is a layout manager, and not an UI element itself. It allows more precise control over the how the grid is displayed. It uses modern JavaScript practices and is, in general, a more maintainable and comprehensive code. The most visible impact is that it now selects a row x column configuration that is closest to the aspect ratio of the display: New layout manager on portrait mode There are still improvements to make, especially with ultra-wide displays, but the foundation work is already there, and it will be vastly easier to fine-tune the behavior of the app grid on different scenarios. Also as part of the preparations for a customizable application grid, the Frequent tab was removed. You can read more about the reasons for this removal in the corresponding issue. Actor Tree Inspector GNOME Shell’s development tool, the Looking Glass, received a handy new tab to inspect the actor tree: The new actor tree tab in the Looking Glass This new inspector has been useful for developing GNOME Shell, and hopefully it’ll help extension developers too. App Folder Dialog Updates App folder dialogs received a bunch of visual and behavioral improvements, such as covering the entire monitor, and not changing the size of the app grid itself. Take a look: https://gitlab.gnome.org/GNOME/gnome-shell/uploads/691c12b639ab727c9a4c14f359bd6158/dialog_cleanups.webm These dialogs are now paginated, and fixed to 9 app icons per page: Paginated folder dialogs with 9 items Like the app grid, folder dialogs now also have better support for touchpad gestures and Drag n’ Drop. Updates to the Message List Popup This year, GNOME Shell has a Google Summer of Code intern working on the messages dialog. As a preparation for this project, some cleanups and reorganizations of the message list popup landed. More work in this front is happening, and an influx of improvements is expected to come soon, stay tuned! Other Changes GNOME Shell now supports the PrefersNonDefaultGPU key of the Desktop File specification, and will set the appropriate environment variables to launch applications using a dedicated GPU when available. An unfortunate oversight was causing the Do Not Disturb setting to be reset on startup. This bug was fixed. A potential D-Bus race condition when creating MPRIS media players was corrected. App icons do not vertically stretch in the top bar anymore. These bugfixes were backported to GNOME 3.36. The rendered contents of labels are now cached in the GPU. The code that deals with workspaces in GNOME Shell is old, but a large number of cleanups to it has landed (!1119, !1251, !1294, !1297, !1298, !1307, !1310, !1313, !1320, !1333), and even more is under review. These cleanups were much needed in order to improve the overall quality and maintainability of the codebase. The Extensions app saw some improvements too. The Logout button now works correctly. When the host system changes timezones, GNOME Shell now properly updates the timezone offsets of the “World Clocks” section of the messages popover. Finally, the Wacom buttom mapping on-screen display received various quality-of-life improvements. Mutter Layout Machinery Optimizations A few exciting optimizations and improvements to Clutter’s layout machinery landed, and they bring groundwork for future improvements as well. The removal of allocation flags allowed skipping the allocation phase of actors whose absolute position (that is, the on-screen position after performing the linear transformation of the actor vertices) didn’t change. While routinely profiling Mutter, it was noticed that an abnormally high number of safety type checks were happening in a rendering hot path, during the redraw cycle. Those checks were removed. Combined, these changes are of notable significance due to how expensive it is to recalculate the layout of actors. Some of them are also required for per-CRTC frame clocks. Rendering Pipeline Improvements Cogl now supports setting a maximum mipmap level, in addition to the minimum one, and background set a maximum mipmap level. This avoids creating mipmaps  that won’t be used. Last year, MetaShapedTexture was made into a ClutterContent implementation. This change was important for a multitude of reasons, and will play a special role in the future with upcoming cleanups. However, it also introduced an unforeseen regression: Clutter paints ClutterContents before running the main painting routines, and this broke the existing culling mechanism of Mutter. After some investigation, culling was fixed again. At last, MetaShapedTexture now uses a lighter, more appropriate function to combine opaque areas of windows. Other Changes Mutter saw a very, very, very, very large number of code cleanups. In fact, these cleanups combined got rid of almost the entirety of deprecated code! Mutter also received a series of improvements to its test suit. These improvements range from fixing broken tests, make CI more reliable, add more tests, reorganize the entire test suit, among other changes. Damage tracking, especially when combined with shadow framebuffers, is now working reliably and correctly. Importing DMA buffers is more careful about failures when importing scanout buffers. Finally, a couple of small memory leaks were plugged. [Less]

Graphics overlays are everywhere nowadays in the live video broadcasting industry. In this post I introduce a new demo relying on GStreamer and WPEWebKit to deliver low-latency web-augmented video broadcasts. Readers of this blog might remember a few ... [More] posts about WPEWebKit and a GStreamer element we at Igalia worked on. In december 2018 I introduced GstWPE and a few months later blogged about a proof-of-concept application I wrote for it. So, learning from this first iteration, I wrote another demo! The first demo was already quite cool, but had a few down-sides: It works only on desktop (running in a Wayland compositor). The Wayland compositor dependency can be a burden in some cases. Ideally we could imaginge GstWPE applications running “in the cloud”, on machines without GPU, bare metal. While it was cool to stream to Twitch, Youtube and the like, these platforms currently can ingest only RTMP streams. That means the latency introduced can be quite significant, depending on the network conditions of course, but even in ideal conditions the latency was between one and 2 seconds. This is not great, in the world we live in. To address the first point, WPE founding engineer, Žan Doberšek enabled software rasterizing support in WPE and its FDO backend. This is great because it allows WPE to run on machines without GPU (like continuous integration builders, test bots) but also “in the cloud” where machines with GPU are less affordable than bare metal! Following up, I enabled this feature in GstWPE. The source element caps template now has video/x-raw, in addition to video/x-raw(memory:GLMemory). To force swrast, you need to set the LIBGL_ALWAYS_SOFTWARE=true environment variable. The downside of swrast is that you need a good CPU. Of course it depends on the video resolution and framerate you want to target. On the latency front, I decided to switch from RTMP to WebRTC! This W3C spec isn’t only about video chat! With WebRTC, sub-second live one-to-many broadcasting can be achieved, without much efforts, given you have a good SFU. For this demo I chose Janus, because its APIs are well documented, and it’s a cool project! I’m not sure it would scale very well in large deployments, but for my modest use-case, it fits very well. Janus has a plugin called video-room which allows multiple participants to chat. But then imagine a participant only publishing its video stream and multiple “clients” connecting to that room, without sharing any video or audio stream, one-to-many broadcasting. As it turns out, GStreamer applications can already connect to this video-room plugin using GstWebRTC! A demo was developed by tobiasfriden and saket424 in Python, it recently moved to the gst-examples repository. As I kind of prefer to use Rust nowadays (whenever I can anyway) I ported this demo to Rust, it was upstreamed in gst-examples as well. This specific demo streams the video test pattern to a Janus instance. Adapting this Janus demo was then quite trivial. By relying on a similar video mixer approach I used for the first GstWPE demo, I had a GstWPE-powered WebView streaming to Janus. The next step was the actual graphics overlays infrastructure. In the first GstWPE demo I had a basic GTK UI allowing to edit the overlays on-the-fly. This can’t be used for this new demo, because I wanted to use it headless. After doing some research I found a really nice NodeJS app on Github, it was developed by Luke Moscrop, who’s actually one of the main developers of the Brave BBC project. The Roses CasparCG Graphics was developed in the context of the Lancaster University Students’ Union TV Station, this app starts a web-server on port 3000 with two main entry points: An admin web-UI (in /admin/ allowing to create and manage overlays, like sports score boards, info banners, and so on. The target overlay page (in the root location of the server), which is a web-page without predetermined background, displaying the overlays with HTML, CSS and JS. This web-page is meant to be fed to CasparCG (or GstWPE :)) After making a few tweaks in this NodeJS app, I can now: Start the NodeJS app, load the admin UI in a browser and enable some overlays Start my native Rust GStreamer/WPE application, which: connects to the overlay web-server mixes a live video source (webcam for instances) with the WPE-powered overlay encodes the video stream to H.264, VP8 or VP9 sends the encoded RTP stream using WebRTC to a Janus server Let “consumer” clients connect to Janus with their browser, in order to see the resulting live broadcast. (If the video doesn’t display, here is the Youtube link.) This is pretty cool and fun, as my colleague Brian Kardell mentions in the video. Working on this new version gave me more ideas for the next one. And very recently the audio rendering protocol was merged in WPEBackend-FDO! That means even more use-cases are now unlocked for GstWPE. This demo’s source code is hosted on Github. Feel free to open issues there, I am always interested in getting feedback, good or bad! GstWPE is maintained upstream in GStreamer and relies heavily on WPEWebKit and its FDO backend. Don’t hesitate to contact us if you have specific requirements or issues with these projects :) [Less]