Mozilla Firefox

Work Basically, I spent most of the week dealing with coding the new workflow for anonymous bug reporting. And we (Guillaume, Ksenia, Mike and myself) made good progress. I believe we will be ready for Berlin All Hands. Also GitHub decided to revive ... [More] our anonymous bugs, around 39,000 bugs are back. We haven't yet reactivated our anonymous reporting. Next week I will be working on the last bit that will allow us to restart the normal process. Oh! Before I forget shout out to Ksenia. When I was reviewing one of her pull requests, I discovered and explored a couple of things I didn't know in the unittest.mock python module. That is one thing we forget too often, reviewing code is a way to learn cool stuff. Friday, I let the code on the side and did a bit of diagnosis, so the pile of bugs doesn't grow insanely. Specifically with Berlin events coming. Murphy Also as Murphy's law put it, last week bad omens kept creeping into this week. My personal VMs just decided to not reboot, and basically I lost everything. This blog (Pelican driven with markdown files), personal blog and a couple of other sites, and my personal email server. So next week, I'll have to rebuild things during night. I have backups of blog content, and emails are under imap on my local machine. The joy of having everything as static content is making things a lot simpler. So it's why this post is published later than sooner. Mozilla lay-offs Let's address the elephant in the room. By now, everyone knows the sad news. 70 persons that you met, crossed path, worked with. 70 persons who shared a desire to make a better place for the Web. My social upbringing (fight for your workers rights) is making things difficult. I find the outpouring of help wonderful, but at the same time, I can't help myself and think that there would not be at #WalmartLifeBoat type of support for people who need to find jobs to not be on the street the day after. The value of Syndicats de salariés (loosely translated by Trade unions in USA, but really with a complete different culture) and the laws protectings one's job in France are essential. Every day working in a North American job culture makes you realize this. All the best to my coworkers who were forced to leave. It's always a sad moment. Every individual circumstance is unique, emotionally, economically, etc. and if I can do anything, tell me. Otsukare! [Less]

What a week! webcompat.com: Project belt-on. So last week, on Friday (Japanese time), I woke up with a website being half disabled and then completely disabled. We had been banned by GitHub because of illegal content we failed to flag early enough. ... [More] And GitHub did what they should do. Oh… and last but not least… mike asked me what Belt-on meant. I guess so let's make it more explicit. Now we need to fix our own mess, that will take a series of steps. Some of them are already taken. Right now, anonymous reporting is disabled. And it will not come back as it was. Done The only way to allow anonymous reporting again would be to switch to a moderation first model, then make it public once it has been made properly reviewed. ToDo We also probably need to remove inline screenshot images. Currently the images are shown directly to users. On Webcompat.com, our nsfw tag blurs images, but this has no effect on github obviously as we do not control the CSS there. ToDo Remove illegal content. Done Discussions have started on how we need to handle the future workflow for anonymous bug report. Created a diagram for a solution which doesn't involve URL redirections in case of moderation. Berlin All Hands Berlin All Hands will be a very interesting work week for the Webcompat team with probably a lot of new things to think and probably for the best of the webcompat.com experience. Incidents can become the source of improvements. Misc The list of 2020Q1 OKRs has been started. Publish some notes about reporting on webcompat.com to help me figuring out a plan on the new anonymous report workflow, aka stating the obvious. A bit of diagnosis on Thursday and Friday. I might have discovered a bug in the Responsive Design Mode related to HTTP redirection and User Agent String. This will be published later, because Murphy hits again and my own server is also down. Reading Interesting read about making the meta viewport the default value. I wonder which breakage impact it could have on Web compatibility. Some websites dynamically test if a meta viewport is already available in the markup and insert it or not dynamically based on user agent strings or screen sizes. Thomas reminded me of the discussion going on about @viewport Release Notes for Safari Technology Preview 98 Otsukare! [Less]

As discussed previously, I planned a few changes to my blog. Well, this Sunday, it all happened. I moved my blog back to a self-hosted WordPress, but am powering it with Jetpack to offer many of the same features as during the seven months it ... [More] ran on WordPress.com. I am also using the same theme, just have rearranged a few things. The privacy policy was updated to reflect the new status. The site is hosted at a Germany-based provider that is very open-source friendly. The main server tasks happen on the command line. The most accessible: Text in, text out. I also managed to transfer the followers acquired with the wordpress.com-hosted blog. I also hope to merge the stats with the ones I already gathered with this blog, and which Jetpack remembered because I am using the same domain again. What did not transfer over were e-mail subscribers that do not use a WordPress.com account. Sorry about that, you’ll have to resubscribe if you want to. There is technically no way to transfer these. Hoping to get you all on board again with the blog that now moved back to self-hosting. Buy me a coffee Donate to the maintenance of this blog or show your appreciation for my content by buying me a virtual cup of coffee. €4.50 [Less]

Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git. Get it on github. These release notes are also available on the git-cinnabar wiki. What’s ... [More] new since 0.5.2? Updated git to 2.25.0 for the helper. Fixed small memory leaks. Combinations of remote ref styles are now allowed. Added a git cinnabar unbundle command that allows to import a mercurial bundle. Experimental support for python >= 3.5. Fixed erroneous behavior of git cinnabar {hg2git,git2gh} with some forms of abbreviated SHA1s. Fixed handling of the GIT_SSH environment variable. Don’t eliminate closed tips when they are the only head of a branch. Better handle manifests with double slashes created by hg convert from Mercurial < 2.0.1, and the following updates to those paths with normal Mercurial operations. Fix compatibility with Mercurial libraries >= 3.0, < 3.4. Windows helper is now statically linked against libcurl. [Less]

This post focuses on the work I accomplished as part of the Treeherder team during the last half of last year.Some work I already blogged about: Reduced time-to-deploy Database spikes affecting UI responsiveness Heroku Review apps for Treeherder ... [More] Deprecated the taskcluster-treeherder serviceThe Taskcluster team requested that we stop ingesting tasks from the taskcluster-treeherder service and instead use the official Taskcluster Pulse exchanges (see work in bug 1395254). This required rewriting their code from Javascript to Python and integrate it into the Treeherder project. This needed to be accomplished by July since the Taskcluster migration would leave the service in an old set up without much support. Integrating this service into Treeherder gives us more control over all Pulse messages getting into our ingestion pipeline without an intermediary service. The project was accomplished ahead of the timeline. The impact is that the Taskcluster team had one less system to worry about ahead of their GCP migration.Create performance dashboard for AWSYI September I created a performance dashboard for the Are We Slim Yet project (see work in bug 1550235). This was not complicated because all I had to do was to branch off the existing Firefox performance dashboard I wrote last year and deploy it on a different Netlify app.One thing I did not do at the time is to create different configs between the two branches. This would make it easier to merge changes from the master branch to the awsy branch without conflicts.On the bright side, Divyanshu came along and fixed the issue! We can now use an env variable to start AWFY & another for AWSY. No need for two different branches!MiscellaneousOther work I did was to create a compare_pushes.py script that allows you to compare two different instances of Treeherder to determine if the ingestion of jobs for a push is different.I added a management command to ingest all Taskcluster tasks from a push, an individual task/push or all tasks associated to a Github PR. Up until this point, the only way to ingest this data would be by having the data ingestion pipeline set up locally before those tasks started showing up in the Pulse exchanges. This script is extremely useful for local development since you can test ingesting data with precise control and havingI added Javascript code coverage for Treeherder through codecov.io and we have managed not to regress farther since then. This was in response that most code backouts for Treeherder were due to frontend regressions. Being able to be notified if we are regressing is useful to adjust tests appropriately.I maintain a regular schedule (twice a week) to release Treeherder production deployments. This guarantees that Treeherder would not have too many changes being deployed to production all at once (I remember the odd day when we had gone 3 weeks without any code being promoted).I helped with planning the work for Treeherder for H2 of this year, H1 next year and I helped the CI-A’s planning for 2020.I automated merging Javascript dependencies updates for the Treeherder.I created a process to share secrets safely within the team instead of using a Google Doc. [Less]

Years ago, Andrew Kennedy published a foundational paper about a type checker for units of measure, and later implemented it for F#. To this day, F# is the only mainstream programming language which provides first class support to make sure that you ... [More] will not accidentally confuse meters and feet, euros and dollars, but that you can still convert between watts·hours and joules. I decided to see whether this could be implemented in and for Rust. The answer is not only yes, but it was fun :) [Less]

It’s been almost 9 years since we released the first Firefox for Android. Hundreds of millions of users have tried it and over time provided us with valuable feedback that allowed us to continuously improve the app, bringing more features to our ... [More] users that increase their privacy and make their mobile lives easier. Now we’re starting a new chapter of the Firefox experience on Android devices. Testing to meet the users’ needs Back in school, most of us weren’t into tests. They were stressful and we’d rather be playing or hanging out with friends. As adults, however, we see the value of testing — especially when it comes to software: testing ensures that we roll out well-designed products to a wide audience that deliver on their intended purposes. At Firefox, we have our users at heart, and the value our products provide to them is at the center of everything we do. That’s why we test a lot. It’s why we make our products available as Nightly (an early version for developers) and Beta versions (a more stable preview of a new piece of software), put the Test Pilot program in place and sometimes, when we enter entirely new territory, we add yet another layer of user testing. It’s exactly that spirit that motivated us to launch Firefox Preview Beta in June 2019. Now we’re ready for the next step. A new Firefox for Android: the making-of When we started working on this project, we wanted to create a better Firefox for Android that would be faster, more reliable, and able to address today’s user problems. Plus, we wanted it to be based on our own mobile browser engine GeckoView in order to offer the highest level of privacy and security available on the Android platform. In short: we wanted to make sure that our users would never have to choose between privacy and a great browsing experience. We had an initial idea of what that new Android product would look like, backed up by previous user research. And we were eager to test it, see how users feel about it, and find out what changes we needed to make and adjust accordingly. To minimize user disruption, early versions of this next generation browser were offered to early adopters as a separate application called Firefox Preview. In order to ensure a fast, efficient and streamlined user experience, we spent the last couple of months narrowing down on what problems our users wanted us to solve, iterating on how we built and surfaced features to them. We looked closely at usage behaviour and user feedback to determine whether our previous assumptions had been correct and where changes would be necessary. The feedback from our early adopters was overwhelmingly positive: the Firefox Preview Beta users loved the app’s fresh modern looks and the noticeably faster browsing experience due to GeckoView as well as new UI elements, such as the bottom navigation bar. When it came to tracking protection, we learned that Android users prefer a browsing experience with a more strict protection and less distractions — that’s why we made Strict Mode the default in Firefox Preview Beta, while Firefox for Desktop comes with Standard Mode. Firefox Preview Beta goes Nightly Based on the previous 6 months of user testing and the positive feedback we have received, we’re confident that Android users will appreciate this new browsing experience and we’re very happy to announce that, as of Tuesday (January 21, 2020), we’re starting to roll it out to our existing Firefox for Android audience in the Nightly app. For current Nightly users, it’ll feel like a big exciting upgrade of their browsing experience once they update the app, either manually or automatically, depending on their preset update routine. New users can easily download Firefox Preview here. As for next milestones, the brand new Firefox for Android will go into Beta in Spring 2020 and land in the main release later in the first half of this year. In the meantime, we’re looking forward to learning more about the wider user group’s perception of the new Firefox for Android as well as to more direct feedback, allowing us to deliver the best-in-class mobile experience that our users deserve. The post A brand new browsing experience arrives in Firefox for Android Nightly appeared first on Future Releases. [Less]

The annual curl developers conference, curl up, is being held in Berlin this year; May 9-10 2020. Starting now, you can register to the event to be sure that you have a seat. The number of available seats is limited. Register here curl ... [More] up is the main (and only?) event of the year where curl developers and enthusiasts get together physically in a room for a full weekend of presentations and discussions on topics that are centered around curl and its related technologies. We move the event around to different countries every year to accommodate different crowds better and worse every year – and this time we’re back again in Germany – where we once started the curl up series back in 2017. The events are typically small with a very friendly spirit. 20-30 persons Sign up We will only be able to let you in if you have registered – and received a confirmation. There’s no fee – but if you register and don’t show, you lose karma. The curl project can even help fund your travel and accommodation expenses (if you qualify). We really want curl developers to come! Register here Date May 9-10 2020. We’ll run the event both days of that weekend. Agenda The program is not done yet and will not be so until just a week or two before the event, and then it will be made available => here. We want as many voices as possible to be heard at the event. If you have done something with curl, want to do something with curl, have a suggestion etc – even just short talk will be much appreciated. Or if you have a certain topic or subject you really want someone else to speak about, let us know as well! Expect topics to be about curl, curl internals, Internet protocols, how to improve curl, what to not do in curl and similar. Location We’ll be in the co.up facilities in central Berlin. On Adalbertstraße 8. Planning and discussions on curl-meet Everything about the event, planning for it, volunteering, setting it up, agenda bashing and more will be done on the curl-meet mailing list, dedicated for this purpose. Join in! The Stockholm edition. Anti Google? If you have a problem with filling in the Google-hosted registration form, please email us at [email protected] instead and we’ll ask you for the information over email instead. Credits The images were taken by me, Daniel. The top one at the Nuremburg curl up in 2017, the in-article photo in Stockholm 2018. [Less]

Today, on behalf of the Rust Fuzzing Authority, I’d like to announce new releases of the arbitrary, libfuzzer-sys, and cargo fuzz crates. Collectively, these releases better support writing fuzz targets that take well-formed instances of custom input ... [More] types. This enables us to combine powerful, coverage-guided fuzzers with smart test case generation. Install or upgrade cargo fuzz with: cargo install --force cargo-fuzz To upgrade your fuzz targets, bump your libfuzzer-sys dependency to 0.2.0 on crates.io. That should be all that’s needed for most cases. However, if you were already using Arbitrary inputs for your fuzz target, some changes will be required. See the upgrading fuzz targets section below for more details. Fuzzing with Well-Formed, Custom Inputs Imagine we are testing an ELF object file parser. In this scenario, it makes sense to fuzz the parser by feeding it raw byte buffers as input. Parsers for binary formats take raw byte buffers as input; there’s no impedance mismatch here. Additionally, coverage-guided fuzzers like libFuzzer naturally generate byte buffers as inputs for fuzz targets, so getting fuzzing up and running is easy. Now instead imagine we are testing a color conversion library: converting between RGB colors to HSL colors and back. Our color conversion functions don’t take raw byte buffers as inputs, they take Rgb or Hsl structures. And these structures are defined locally by our library; libFuzzer doesn’t have any knowledge of them and can’t generate instances of them on its own. We could write a color string parser, parse colors from the fuzzer-provided, raw input bytes, and then pass the parsed results into our color conversion functions. But now the fuzzer is going to spend a bunch of time exploring and testing our color string parser, before it can get “deeper” into the fuzz target, where our color conversions are. Small changes to the input can result in invalid color strings, that bounce off the parser. Ultimately, our testing is less efficient than we want, because our real goal is to exercise the conversions but we’re doing all this other stuff. On top of that, there’s this hump we have to get over to start fuzzing, since we have to write a bunch of parsing code if we didn’t already have it. This is where the Arbitrary trait comes in. Arbitrary lets us create structured inputs from raw byte buffers with as thin a veneer as possible. As best it can, it preserves the property that small changes to the input bytes lead to small changes in the corresponding Arbitrary instance constructed from those input bytes. This helps coverage-guided fuzzers like libFuzzer efficiently explore the input space. The new 0.3.0 release of arbitrary contains an overhaul of the trait’s design to further these goals. Implementing Arbitrary for custom types is easy: 99% of the time, all we need to do is automatically derive it. So let’s do that for Rgb in our color conversion library: use arbitrary::Arbitrary; #[derive(Arbitrary, Debug, PartialEq, Eq)] pub struct Rgb { r: u8, g: u8, b: u8, } The libfuzzer-sys crate lets us define fuzz targets where the input type is anything that implements Arbitrary: // The fuzz target takes a well-formed `Rgb` as input! libfuzzer_sys::fuzz_target!(|rgb: Rgb| { let hsl = rgb.to_hsl(); let rgb2 = hsl.to_rgb(); // RGB -> HSL -> RGB is the identity function. This // property should hold true for all RGB colors! assert_eq!(rgb, rgb2); }); Now we have a fuzz target that works with our custom input type directly, is the thinnest abstraction possible over the raw, fuzzer-provided input bytes, and we didn’t need to write any custom parsing logic ourselves! This isn’t limited to simple structures like Rgb. The arbitrary crate provides Arbitrary implementations for everything in std that you’d expect it to: bool, u32, f64, String, Vec, HashMap, PathBuf, etc… Additionally, the custom derive works with all kinds of structs and enums, as long as each sub-field implements Arbitrary. For more details check out the new Structure-Aware Fuzzing section of The Rust Fuzzing Book and the arbitrary crate. The book has another neat example, where we’re testing a custom allocator implementation and the fuzz target takes a sequence of malloc, realloc, and free commands. Bonus: Improved UX in cargo fuzz When cargo fuzz finds a failing input, it will display the Debug formatting of the failing input (particularly nice with custom Arbitrary inputs) and suggest common next tasks, like reproducing the failure or running test case minimization. For example, if we write a fuzz target that panics when r < g < b for a given Rgb instance, libFuzzer will very quickly find an input that triggers the failure, and then cargo fuzz will give us this friendly output: Failing input: fuzz/artifacts/my_fuzz_target/crash-7bb2b62488fd8fc49937ebeed3016987d6e4a554 Output of `std::fmt::Debug`: Rgb { r: 30, g: 40, b: 110, } Reproduce with: cargo fuzz run my_fuzz_target fuzz/artifacts/my_fuzz_target/crash-7bb2b62488fd8fc49937ebeed3016987d6e4a554 Minimize test case with: cargo fuzz tmin my_fuzz_target fuzz/artifacts/my_fuzz_target/crash-7bb2b62488fd8fc49937ebeed3016987d6e4a554 Upgrading Fuzz Targets First, make sure you’ve upgraded cargo fuzz: cargo install --force cargo-fuzz Next, upgrade your libfuzzer-sys from a git dependency to the 0.2.0 version on crates.io in your Cargo.toml: [dependencies] -libfuzzer-sys = { git = "https://github.com/rust-fuzz/libfuzzer-sys.git" } +libfuzzer-sys = "0.2.0" If your existing fuzz targets were not using custom Arbitrary inputs, and were taking &[u8] slices of raw bytes, then you’re done! If you’re implementing Arbitrary for your own custom input types, you’ll need to bump your dependency on Arbitrary to version 0.3. We recommend that, unless you have any specialized logic in your Arbitrary implementation, that you use the custom derive. Enable the custom derive by requiring the "derive" feature: [dependencies] arbitrary = { version = "0.3", features = ["derive"] } And then derive Arbitrary automatically: use arbitrary::Arbitrary; #[derive(Arbitrary, Debug)] struct MyStruct { // ... } Finally, if you do you specialized logic in your Arbitrary implementation, and can’t use the custom derive, your implementations will change something like this: use arbitrary::{Arbitrary, Unstructured}; impl Arbitrary for MyType { - fn arbitrary(u: &mut U) -> Result - where - U: Unstructured + ?Sized, - { + fn arbitrary( + u: &mut Unstructured + ) -> arbitrary::Result { // ... } } The trait has been simplified a little bit, and Unstructured is a concrete type now, rather than a trait you need to parameterize over. Check out the CHANGELOG for more details. CHANGELOGs arbitrary @ 0.3.0 libfuzzer-sys @ 0.2.0 cargo fuzz @ 0.7.0 Thank You! 💖 Thanks to everyone who contributed to these releases! Alex Rebert koushiro Manish Goregaokar Nick Fitzgerald Simonas Kazlauskas And a special shout out to Manish for fielding so many pull request reviews! FAQ What is fuzzing? Fuzzing is a software testing technique used to find security, stability, and correctness issues by feeding pseudo-random data as input to the software. Learn more about fuzzing Rust code in The Rust Fuzzing Book! How is all this different from quickcheck and proptest? If you’re familiar with quickcheck or proptest and their own versions of the Arbitrary trait, you might be wondering what the difference is between what’s presented here and those tools. The primary goal of what’s been presented here is to have a super-thin, efficient layer on top of coverage-guided, mutation-based fuzzers like libFuzzer. That means the paradigm is a little different from quickcheck and proptest. For example, arbitrary::Arbitrary doesn’t take a random number generator like quickcheck::Arbitrary does. Instead it takes an Unstructured, which is a helpful wrapper around a raw byte buffer provided by the fuzzer. It’s similar to libFuzzer’s FuzzedDataProvider. The goal is to preserve, as much as possible, the actual input given to us by the fuzzer, and make sure that small changes in the raw input lead to small changes in the value constructed via arbitrary::Arbitrary. Similarly, we don’t want different, customizable test case generation strategies like proptest supports, because we leverage the fuzzer’s insight into code coverage to efficiently explore the input space. We don’t want to get in the way of that, and step on the fuzzer’s toes. This isn’t to say that quickcheck and proptest are without value! On the contrary, if you are not using a coverage-guided fuzzer like libFuzzer or AFL — perhaps because you’re working on an unsupported platform — and are instead using a purely-random, generation-based fuzzing setup, then both quickcheck and proptest are fantastic options to consider. [Less]

If you’re reading these lines, you may have recently been laid off from your job. Or maybe, depending on your country and its laws, you’re waiting to know if you’re being laid off. Well, I’ve been there and I’ve survived it, so, based on my experience, here are a few suggestions: