I Use This!
Activity Not Available


Analyzed 7 months ago. based on code collected 7 months ago.
Posted 4 days ago
If you use GNOME or Ubuntu, then GNOME Disks is probably what you rely on if you ever need to do any disk management operations, so it’s a relatively important piece of software for GNOME and Ubuntu users. Now if you’re a command line geek, you ... [More] might handle disk management via command line, and that’s fine, but most users don’t know how to do that. Or if you’re living in the past like Ubuntu and not yet using Wayland, you might prefer GParted (which does not work under Wayland because it requires root permissions, while we intentionally will not allow applications to run as root in Wayland). But for anyone else, you’re probably using GNOME Disks. So it would be good for it to work reliably, and for it to be relatively free of bugs. I regularly receive new bug reports against GNOME Disks. Sometimes they’re not very well-constructed or based on some misunderstanding of how partitioning works, in which case I’ll close them, but most of them are good and valid. So who fixes bug reports against GNOME Disks? The answer is: nobody! Unless it’s really, really easy — in which case I might allocate five minutes for it — nobody is going to fix the bug that you reported. What a shame! Who is the maintainer? In this case, it’s me, but I don’t actually know much anything about the application and certainly don’t have time to fix things; I just check Bugzilla to see if anybody has posted a patch, so that contributors’ patches (which are rare) don’t get totally neglected, and make new releases every once in a while, and only because I didn’t want to see such a critical piece of software go completely unmaintained. If you’re a software developer with an interest in both GNOME and disk management, GNOME Disks would be a great place to help out. A great place to start would be to search through GNOME Bugzilla for issues to work on, and submit patches for them. Of course, Disks is far from the only unmaintained or undermaintained software in GNOME. Last year, Sébastien set up a wiki page to track unmaintained and undermaintained apps. It has had some success: in that time, GNOME Calculator, Shotwell, Gtranslator, and Geary have all found maintainers and been removed from the list of unmaintained modules. (Geary is still listed as undermaintained, and no doubt it would be nice to have more Geary maintainers, but the current maintainer seems to be quite active, so I would hesitate to list it as undermaintained. Epiphany would love to have a second maintainer as well. No doubt most GNOME apps would.) But we still have a few apps that are listed as unmaintained: Bijiben (GNOME Notes) Empathy GNOME Disks No doubt there are more GNOME modules that should be listed. If you know of some, please add them or leave a comment here. Help would be very much welcome with any of these. In particular, Empathy and Bijiben are both slated to be removed from Fedora beginning with Fedora 27 due to their unacceptable dependencies on an old, insecure version of WebKitGTK+ that is about to be removed from the distribution. Most of the work to port these applications to modern WebKitGTK+ is already done (and, in the case of Empathy, I’ve already committed the port to git), but an active maintainer is required to finish the job and get things to a releasable state. Last I checked, Bijiben also still needed to be ported to GTK+ 3.20. If nobody is interested in helping out, these apps are going to disappear sooner rather than later. Disks, fortunately, is not going to disappear anytime soon. But the bugs aren’t going to fix themselves. P.S. This blog is not the right place to complain about no longer allowing applications to run as root. Such applications can and should use Polkit to move privileged operations out of the GUI and into a helper process. This should have been done roughly a decade ago. Such applications might themselves be unmaintained or undermaintained; can you help them out? [Less]
Posted 5 days ago
Thanks to Georg Vienna, Builder can now manage your Rust installations using RustUp!
Posted 6 days ago by nore...@blogger.com (Jim Hall)
In a comment on my other article about how web pages are becoming hard to read, Shaun referenced the W3C Web Content Accessibility Guidelines. They provide an algorithm to determine if your text meets minimum accessibility guidelines.The W3C ... [More] definition of the contrast ratio requires several calculations: given two colors, you first compute the relative luminance of each (L1 and L2) then calculate the contrast ratio. The ratio will fall in the range 1 to 21 (typically written 1:1 or 21:1). The higher the contrast ratio, the more the text will stand out against the background. For example, black text on a white background has a contrast ratio of 21:1.The W3C says body text should have a contrast ratio of at least 4.5:1, with headings at least 3:1. But that seems to be the bare minimum. The W3C also recommends at least 7:1 for body text and at least 4.5:1 for headings.Calculating this can be a chore, so it's best to automate it. Shaum implemented the algorithm in XSLT so he could test the various colors in websites. I created a similar implementation using Bash. It's a little ugly, but I thought I'd share it here:First, you need a way to input colors. I wanted something that could interpret different representations of colors: in html and css, black is the same as #000 or #000000 or rgb(0,0,0). When evaluating the readability of my text, I might want to use any of these.Fortunately, there's a neat tool in GNOME to provide that input. GNOME Zenity is a scripting tool to display GTK+ dialogs. It supports many modes to read input and display results. One of the input modes is a color selector, which you use this way: zenity --color-selectionYou can give it other options to set the window title and provide a default color. Zenity returns the selected color on standard output. You can even set a default color. So to present two dialogs, one to read the text color and another to read the background color, you simply do this:color=$( zenity --title 'Set text color' --color-selection --color='black' )background=$( zenity --title 'Set background color' --color-selection --color='white' )Zenity returns values like rgb(255,140,0) and rgb(255,255,255), which is fortunate because the W3C calculation for luminance requires values in the range 0 to 255. I wrote a simple function to pull apart the RGB values. There are probably simpler ways to parse RGB, but a quick and easy way is to let awk split the values at the comma. That means a value like rgb(255,140,0) gets split into rgb(255 and 140 and 0) so the R value is a substring starting at the fifth character, G is the second element, and B is a substring up to the last parenthesis.Once I have the RGB values, then I calculate the luminance using bc. The funky math with e() and l() are to get around a limitation in bc. Specifically, the formula requires a fractional power, and bc can only do integer powers. But if you follow the math, you can get there using e() and l():function luminance(){        R=$( echo $1 | awk -F, '{print substr($1,5)}' )        G=$( echo $1 | awk -F, '{print $2}' )        B=$( echo $1 | awk -F, '{n=length($3); print substr($3,1,n-1)}' )        echo "scale=4rsrgb=$R/255gsrgb=$G/255bsrgb=$B/255if ( rsrgb <= 0.03928 ) r = rsrgb/12.92 else r = e( 2.4 * l((rsrgb+0.055)/1.055) )if ( gsrgb <= 0.03928 ) g = gsrgb/12.92 else g = e( 2.4 * l((gsrgb+0.055)/1.055) )if ( bsrgb <= 0.03928 ) b = bsrgb/12.92 else b = e( 2.4 * l((bsrgb+0.055)/1.055) )0.2126 * r + 0.7152 * g + 0.0722 * b" | bc -l}Once you have the luminance value of the text color and background color, you can compute the contrast ratio. The W3C formula to do this is quite simple, but requires knowing which is the lighter and darker colors. This is an extra step in bc. I wrote this Bash function to calculate the ratio based on two colors:function contrast(){        echo "scale=2if ( $1 > $2 ) { l1=$1; l2=$2 } else { l1=$2; l2=$1 }(l1 + 0.05) / (l2 + 0.05)" | bc}With those functions, it's fairly straightforward to write a Bash script that reads two colors, then computes the contrast ratio. My script also uses Zenity to output the data: #!/bin/sh# read color and background color:color=$( zenity --title 'Set text color' --color-selection --color='black' )if [ $? -ne 0 ] ; then        echo '** color canceled - assume black'        color='rgb(0,0,0)'fibackground=$( zenity --title 'Set background color' --color-selection --color='white' )if [ $? -ne 0 ] ; then        echo '** background canceled - assume white'        background='rgb(255,255,255)'fi# compute luminance:function luminance(){…}lum1=$( luminance $color )lum2=$( luminance $background )# compute contrastfunction contrast(){…}rel=$( contrast $lum1 $lum2 )# print results( echo "Color is $color on $background"echo "Contrast ratio is $rel"if [ ${rel%.*} -ge 4 ] ; then        echo "Ok for body text"else        echo "Not good for body text"fiif [ ${rel%.*} -ge 3 ] ; then        echo "Ok for title text"else        echo "Not good for title text"fi) | zenity --text-info --title='Contrast Ratio'With this script, I have a handy way to calculate the contrast ratio of two colors: text color vs background color. For black text on a white background, the contrast ratio is 21.00, the most visible. The #333 dark gray on white has a contrast ratio of 12.66, which is fine. And the lighter #808080 gray on white has a contrast ratio of 3.95, too low for normal text but acceptable for large text like headings. Very light #ccc gray on white has a contrast ratio of 1.60, which is way too low. [Less]
Posted 6 days ago
How librsvg exports reference-counted objects from Rust to C Librsvg maintains a tree of RsvgNode objects; each of these corresponds to an SVG element. An RsvgNode is a node in an n-ary ... [More] tree; for example, a node for an SVG "group" can have any number of children that represent various shapes. The toplevel element is the root of the tree, and it is the "svg" XML element at the beginning of an SVG file. Last December I started to sketch out the Rust code to replace the C implementation of RsvgNode. Today I was able to push a working version to the librsvg repository. This is a major milestone for myself, and this post is a description of that journey. Nodes in librsvg in C Librsvg used to have a very simple scheme for memory management and its representation of SVG objects. There was a basic RsvgNode structure: typedef enum { RSVG_NODE_TYPE_INVALID, RSVG_NODE_TYPE_CHARS, RSVG_NODE_TYPE_CIRCLE, RSVG_NODE_TYPE_CLIP_PATH, /* ... a bunch of other node types */ } RsvgNodeType; typedef struct { RsvgState *state; RsvgNode *parent; GPtrArray *children; RsvgNodeType type; void (*free) (RsvgNode *self); void (*draw) (RsvgNode *self, RsvgDrawingCtx *ctx, int dominate); void (*set_atts) (RsvgNode *self, RsvgHandle *handle, RsvgPropertyBag *pbag); } RsvgNode; This is a no-frills base struct for SVG objects; it just has the node's parent, its children, its type, the CSS state for the node, and a virtual function table with just three methods. In typical C fashion for derived objects, each concrete object type is declared similar to the following one: typedef struct { RsvgNode super; RsvgLength cx, cy, r; } RsvgNodeCircle; The user-facing object in librsvg is an RsvgHandle: that is what you get out of the API when you load an SVG file. Internally, the RsvgHandle has a tree of RsvgNode objects — actually, a tree of concrete implementations like the RsvgNodeCircle above or others like RsvgNodeGroup (for groups of objects) or RsvgNodePath (for Bézier paths). Also, the RsvgHandle has an all_nodes array, which is a big list of all the RsvgNode objects that it is handling, regardless of their position in the tree. It also has a hash table that maps string IDs to nodes, for when the XML elements in the SVG have an "id" attribute to name them. At various times, the RsvgHandle or the drawing-time machinery may have extra references to nodes within the tree. Memory management is simple. Nodes get allocated at loading time, and they never get freed or moved around until the RsvgHandle is destroyed. To free the nodes, the RsvgHandle code just goes through its all_nodes array and calls the node->free() method on each of them. Any references to the nodes that remain in other places will dangle, but since everything is being freed anyway, things are fine. Before the RsvgHandle is freed, the code can copy pointers around with impunity, as it knows that the all_nodes array basically stores the "master" pointers that will need to be freed in the end. But Rust doesn't work that way Not so, indeed! C lets you copy pointers around all you wish; it lets you modify all the data at any time; and forces you to do all the memory-management bookkeeping yourself. Rust has simple and strict rules for data access, with profound implications. You may want to read up on ownership (where variable bindings own the value they refer to, there is one and only one binding to a value at any time, and values are deleted when their variable bindings go out of scope), and on references and borrowing (references can't outlive their parent scope; and you can either have one or more immutable references to a resource, or exactly one mutable reference, but not both at the same time). Together, these rules avoid dangling pointers, data races, and other common problems from C. So while the C version of librsvg had a sea of carefully-managed shared pointers, the Rust version needs something different. And it all started with how to represent the tree of nodes. How to represent a tree Let's narrow down our view of RsvgNode from C above: typedef struct { ... RsvgNode *parent; GPtrArray *children; ... } RsvgNode; A node has pointers to all its children, and each child has a pointer back to the parent node. This creates a bunch of circular references! We would need a real garbage collector to deal with this, or an ad-hoc manual memory management scheme like librsvg's and its all_nodes array. Rust is not garbage-collected and it doesn't let you have shared pointers easily or without unsafe code. Instead, we'll use reference counting. To avoid circular references, which a reference-counting scheme cannot handle, we use strong references from parents to children, and weak references from the children to point back to parent nodes. In Rust, you can add reference-counting to your type Foo by using Rc (if you need atomic reference-counting for multi-threading, you can use an Arc). An Rc represents a strong reference count; conversely, a Weak is a weak reference. So, the Rust Node looks like this: pub struct Node { ... parent: Option>, children: RefCell>>, ... } Let's unpack that bit by bit. "parent: Option>". The Weak is a weak reference to the parent Node, since a strong reference would create a circular refcount, which is wrong. Also, not all nodes have a parent (i.e. the root node doesn't have a parent), so put the Weak reference inside an Option. In C you would put a NULL pointer in the parent field; Rust doesn't have null references, and instead represents lack-of-something via an Option set to None. "children: RefCell>>". The Vec>> is an array (vector) of strong references to child nodes. Since we want to be able to add children to that array while the rest of the Node structure remains immutable, we wrap the array in a RefCell. This is an object that can hand out a mutable reference to the vector, but only if there is no other mutable reference at the same time (so two places of the code don't have different views of what the vector contains). You may want to read up on interior mutability. Strong Rc references and Weak refs behave as expected. If you have an Rc, you can ask it to downgrade() to a Weak reference. And if you have a Weak, you can ask it to upgrade() to a strong Rc, but since this may fail if the Foo has already been freed, that upgrade() returns an Option> — if it is None, then the Foo was freed and you don't get a strong Rc; if it is Some(x), then x is an Rc, which is your new strong reference. Handing out Rust reference-counted objects to C In the post about Rust constructors exposed to C, we talked about how a Box is Rust's primitive to put objects in the heap. You can then ask the Box for the pointer to its heap object, and hand out that pointer to C. If we want to hand out an Rc to the C code, we therefore need to put our Rc in the heap by boxing it. And going back, we can unbox an Rc and let it fall out of scope in order to free the memory from that box and decrease the reference count on the underlying object. First we will define a type alias, so we can write RsvgNode instead of Rc and make function prototypes closer to the ones in the C code: pub type RsvgNode = Rc; Then, a convenience function to box a refcounted Node and extract a pointer to the Rc, which is now in the heap: pub fn box_node (node: RsvgNode) -> *mut RsvgNode { Box::into_raw (Box::new (node)) } Now we can use that function to implement ref(): #[no_mangle] pub extern fn rsvg_node_ref (raw_node: *mut RsvgNode) -> *mut RsvgNode { assert! (!raw_node.is_null ()); let node: &RsvgNode = unsafe { & *raw_node }; box_node (node.clone ()) } Here, the node.clone () is what increases the reference count. Since that gives us a new Rc, we want to box it again and hand out a new pointer to the C code. You may want to read that twice: when we increment the refcount, the C code gets a new pointer! This is like creating a hardlink to a Unix file — it has two different names that point to the same inode. Similarly, our boxed, cloned Rc will have a different heap address than the original one, but both will refer to the same Node in the end. This is the implementation for unref(): #[no_mangle] pub extern fn rsvg_node_unref (raw_node: *mut RsvgNode) -> *mut RsvgNode { if !raw_node.is_null () { let _ = unsafe { Box::from_raw (raw_node) }; } ptr::null_mut () // so the caller can do "node = rsvg_node_unref (node);" and lose access to the node } This is very similar to the destructor from a few blog posts ago. Since the Box owns the Rc it contains, letting the Box go out of scope frees it, which in turn decreases the refcount in the Rc. However, note that this rsvg_node_unref(), intended to be called from C, always returns a NULL pointer. Together, both functions are to be used like this: RsvgNode *node = ...; /* acquire a node from Rust */ RsvgNode *extra_ref = rsvg_node_ref (node); /* ... operate on the extra ref; now discard it ... */ extra_ref = rsvg_node_unref (extra_ref); /* Here extra_ref == NULL and therefore you can't use it anymore! */ This is a bit different from g_object_ref(), which returns the same pointer value as what you feed it. Also, the pointer that you would pass to g_object_unref() remains usable if you didn't take away the last reference... although of course, using it directly after unreffing it is perilous as hell and probably a bug. In these functions that you call from C but are implemented in Rust, ref() gives you a different pointer than what you feed it, and unref() gives you back NULL, so you can't use that pointer anymore. To ensure that I actually used the values as intended and didn't fuck up the remaining C code, I marked the function prototypes with the G_GNUC_WARN_UNUSED_RESULT attribute. This way gcc will complain if I just call rsvg_node_ref() or rsvg_node_unref() without actually using the return value: RsvgNode *rsvg_node_ref (RsvgNode *node) G_GNUC_WARN_UNUSED_RESULT; RsvgNode *rsvg_node_unref (RsvgNode *node) G_GNUC_WARN_UNUSED_RESULT; And this actually saved my butt in three places in the code when I was converting it to reference counting. Twice when I forgot to just use the return values as intended; once when the old code was such that trivially adding refcounting made it use a pointer after unreffing it. Make the compiler watch your back, kids! Testing One of the things that makes me giddy with joy is how easy it is to write unit tests in Rust. I can write a test for the refcounting machinery above directly in my node.rs file, without needing to use C. This is the test for ref and unref: #[test] fn node_refs_and_unrefs () { let node = Rc::new (Node::new (...)); let mut ref1 = box_node (node); // "hand out a pointer to C" let new_node: &mut RsvgNode = unsafe { &mut *ref1 }; // "bring back a pointer from C" let weak = Rc::downgrade (new_node); // take a weak ref so we can know when the node is freed let mut ref2 = unsafe { rsvg_node_ref (new_node) }; // first extra reference assert! (weak.upgrade ().is_some ()); // "you still there?" ref2 = unsafe { rsvg_node_unref (ref2) }; // drop the extra reference assert! (weak.upgrade ().is_some ()); // "you still have life left in you, right?" ref1 = unsafe { rsvg_node_unref (ref1) }; // drop the last reference assert! (weak.upgrade ().is_none ()); // "you are dead, aren't you? } And this is the test for two refcounts indeed pointing to the same Node: #[test] fn reffed_node_is_same_as_original_node () { let node = Rc::new (Node::new (...)); let mut ref1 = box_node (node); // "hand out a pointer to C" let mut ref2 = unsafe { rsvg_node_ref (ref1) }; // "C takes an extra reference and gets a new pointer" unsafe { assert! (rsvg_node_is_same (ref1, ref2)); } // but they refer to the same thing, correct? ref1 = rsvg_node_unref (ref1); ref2 = rsvg_node_unref (ref2); } Hold on! Where did that rsvg_node_is_same() come from? Since calling rsvg_node_ref() now gives a different pointer to the original ref, we can no longer just do "some_node == tree_root" to check for equality and implement a special case. We need to do "rsvg_node_is_same (some_node, tree_root)" instead. I'll just point you to the source for this function. [Less]
Posted 6 days ago
As #LinuxPlaya draws near, we’ve been preparing things to the event. We first did a workshop to help others to finish the GTK+Python tutorial for developers. While some other students from different universities in Lima did some posts to prove that ... [More] they use Linux (FEDORA+GNOME). You can see in the following list, the various areas where they had worked: design, robotics, education, by using tech as Docker and a Snake GTK game. Thanks to GNOME and FEDORA we are going to go to the beach Santa Maria on March 4th, and we are going to do more than social networking.  In the morning we are going to present our projects and then we are going to encourage them to apply to the GSoC program. Lunch time and afternoon games are also planned for this occasion. These are  the summer merchandising we are going to offer to our guests. It’s a pleasure to have Damian Nohales from GNOME Argentina as our international guest Most of the participants are also leaders in their universities and they are going to replicate the meetings in their places.  This is the case of Leyla Marcelo who is an entrepreneur leader in her university UPN and our designer in the last Linux events I organised in Lima, Peru.  Special thanks to Softbutterfly for the Internet support that day! Filed under: FEDORA, GNOME Tagged: #LinuxPlaya, evento Linux, fedora, GNOME, Julita, Julita Inca, Lima, linux, Linux Playa, Perú, Softbuttterfly [Less]
Posted 7 days ago
One of the core features of Builder is, well, building. So we need to provide the best experience we can. That means we need wide support for build systems and languages. How we get from source code to a product can vary in a multitude of ways. For ... [More] example, you might need to alter how you launch programs to run inside a container. You might need access to headers that only exist in the SDK and not the application runtime. You might even need to download or build dependencies for your project at a given point. Furthermore, this could all change based on what plugins you’ve chosen to enable in your Builder environment. When I set out to create Builder, I knew that I wouldn’t be able to see all the possibilities that Builder needs to handle. So I explicitly avoided designing an abstract solution until I understood the problem better. But I think we understand the problem well enough now. So over the winter holiday I started putting together the design of IdeBuildPipeline. The pipeline is a one-directional series of build phases. Each phase can have stages attached to it. Builds progress in the order of their attached stages, each of which can be an asynchronous operation. We only need to advance the build up to the requested phase. If the IDE detects something has changed, it can invalidate various phases of the pipeline causing them to be executed again at the next build request. Plugins should implement the IdeBuildPipelineAddin interface to register any necessary pipeline stages. Generally, build stages are an IdeSubprocessLauncher, but you can implement custom IdeBuildStage subclasses to do more fancy things. For example, if you need to download something, you might add an IdeBuildStageTransfer and you’ll see the transfer show up in the transfers window. Of course, some problems are complex enough where we don’t know if they are invalidated upfront and the build stage may query some external resource. For that, the IdeBuildStage class can implement the query signal. Of course we should strive to perform blocking operations asynchronously so consider pausing the stage until your asynchronous request has completed. We know how difficult it is for newcomers to get started so we’ve taught Builder how to download Flatpak runtimes and SDKs based on your project configuration. That means that you can omit those flatpak install commands from your “project setup” guide. Just clone the project URL and click Build, we’ll take care of the rest. Thanks to this new design, implementing a build system is much easier. To prove this, I finally implemented a basic cmake+ninja plugin in a matter of an hour. Check it out here. If you’d like to write CFLAG extraction for cmake, we could use that still! We now have support for autotools, cargo, cmake, and meson build systems. [Less]
Posted 8 days ago
I assume that you already familiar with using IPython.parallel, otherwise have a look at the documentation. Just one point of caution, if you move code that you want to run in parallel to a package or module, you might stumble upon the problem that ... [More] Python cannot find the function you imported. The problem is that IPython.parallel only looks in the global namespace. The solution is using @interactive from IPython.parallel as described in this Stack Overflow post. Although, IPython.parallel comes with built-in support for distributing work using MPI, it is going to create MPI tasks itself. However, usually compute clusters come with a job scheduler like SLURM that manages all resources. Consequently, the scheduler determines how many resources you have access to. In the case of SLURM, you have to define the number of tasks you want to process in parallel and the maximum time our job will require when adding your job to the queue. #!/bin/bash #SBATCH -J ipython-parallel-test #SBATCH --ntasks=112 #SBATCH --time=00:10:00 The above script gives the job a name, requests resources to 112 tasks and sets the maximum required time to 10 minutes. Normally, you would use ipcluster start -n 112, but since we are not allowed to create MPI tasks ourselves, we have to start the individual pieces manually via ipcontroller and ipengine. The controller provides a single point of contact the engines connect to and the engines take commands and execute them. profile=job_${SLURM_JOB_ID}_$(hostname)   echo "Creating profile ${profile}" ipython profile create ${profile}   echo "Launching controller" ipcontroller --ip="*" --profile=${profile} --log-to-file & sleep 10   echo "Launching engines" srun ipengine --profile=${profile} --location=$(hostname) --log-to-file & sleep 45 First of all, we create a new IPython profile, which will contain log files and temporary files that are necessary to establish the communication between controller and engines. To avoid clashes with other jobs, the name of the profile contains the job's ID and the hostname of the machine it is executed on. This will create a folder named profile_job_XYZ_hostname in the ~/.ipython folder. Next, we start the controller and instruct it to listen on all available interfaces, use the newly create profile and write output to log files residing in the profile's directory. Note that this command is executed only on a single node, thus we only have a single controller per job. Finally, we can create the engines, one one for each task, and instruct them to connect to the correct controller. Explicitly specifying the location of the controller is necessary if engines are spread across multiple physical machines and machines have multiple Ethernet interfaces. Removing this option, engines running on other machines are likely to fail connecting to the controller because they might look for the controller at the wrong location (usually localhost). You can easily find out whether this is the case by looking at the ipengine log files in the ~/.ipython/profile_job_XYZ_hostname/log directory. Finally, we can start our Python script that uses IPython.parallel to distribute work across multiple nodes. echo "Launching job" python ipython-parallel-test.py --profile ${profile} To make things more interesting, I created an actual script that approximates the number Pi in parallel. import argparse from IPython.parallel import Client import numpy as np import os     def compute_pi(n_samples):         s = 0         for i in range(n_samples):                 x = random()                 y = random()                 if x * x + y * y <= 1:                         s += 1         return 4. * s / n_samples     def main(profile):         rc = Client(profile=profile)         views = rc[:]         with views.sync_imports():                 from random import random                 results = views.apply_sync(compute_pi, int(1e9))         my_pi = np.sum(results) / len(results)         filename = "result-job-{0}.txt".format(os.environ["SLURM_JOB_ID"])         with open(filename, "w") as fp:                 fp.write("%.20f\n" % my_pi)     if __name__ == '__main__':         parser = argparse.ArgumentParser()         parser.add_argument("-p", "--profile", required=True,                 help="Name of IPython profile to use")           args = parser.parse_args()           main(args.profile) I ran this on the Linux cluster of the Leibniz Supercomputing Centre. Using 112 tasks, it took a little more than 8 minutes, and the result I got was 3.14159637160714266813. You can download the SLURM script and Python code from above here. [Less]
Posted 8 days ago
I've meant to do this release for quite a while now and last week I finally had some time to package everything and update the dependencies. scikit-survival contains the majority of code I developed during my Ph.D. About Survival Analysis Survival ... [More] analysis – also referred to as reliability analysis in engineering – refers to type of problem in statistics where the objective is to establish a connections between a set of measurements (often called features or covariates) and the time to an event. The name survival analysis originates from clinical research: in many clinical studies, one is interested in predicting the time to death, i.e., survival. Broadly speaking, survival analysis is a type of regression problem (one wants to predict a continuous value), but with a twist. Consider a clinical study, which investigates coronary heart disease and has been carried out over a 1 year period as in the figure below. Patient A was lost to follow-up after three months with no recorded cardiovascular event, patient B experienced an event four and a half months after enrollment, patient D withdrew from the study two months after enrollment, and patient E did not experience any event before the study ended. Consequently, the exact time of a cardiovascular event could only be recorded for patients B and C; their records are uncensored. For the remaining patients it is unknown whether they did or did not experience an event after termination of the study. The only valid information that is available for patients A, D, and E is that they were event-free up to their last follow-up. Therefore, their records are censored. Formally, each patient record consists of a set of covariates $x \in \mathbb{R}^d$ , and the time $t > 0$ when an event occurred or the time $c > 0$ of censoring. Since censoring and experiencing and event are mutually exclusive, it is common to define an event indicator $\delta \in \{0; 1\}$ and the observable survival time $y > 0$. The observable time $y$ of a right censored sample is defined as \[ y = \min(t, c) = \begin{cases} t & \text{if } \delta = 1 , \\ c & \text{if } \delta = 0 , \end{cases} \] What is scikit-survival? Recently, many methods from machine learning have been adapted for these kind of problems: random forest, gradient boosting, and support vector machine, many of which are only available for R, but not Python. Some of the traditional models are part of lifelines or statsmodels, but none of those libraries plays nice with scikit-learn, which is the quasi-standard machine learning framework for Python. This is exactly where scikit-survival comes in. Models implemented in scikit-survival follow the scikit-learn interfaces. Thus, it is possible to use PCA from scikit-learn for dimensionality reduction and feed the low-dimensional representation to a survival model from scikit-survival, or cross-validate a survival model using the classes from scikit-learn. You can see an example of the latter in this notebook. Download and Install The source code is available at GitHub and can be installed via Anaconda (currently only for Linux) or pip. conda install -c sebp scikit-survival pip install scikit-survival The API documentation is available here and scikit-survival ships with a couple of sample datasets from the medical domain to get you started. [Less]
Posted 8 days ago by nore...@blogger.com (Jim Hall)
An article at Backchannel discusses an interesting trend in website design, and how the web became unreadable. It's a good read, but I'll summarize briefly:Web pages are becoming too hard to read.Put another way, a popular trend in web design is to ... [More] use low-contrast text. Maybe that looks really cool, but it is also really hard to read. From the article: "I thought my eyesight was beginning to go. It turns out, I’m suffering from design."I've noticed this trend too, and I do find it hard to read certain websites. Even this blog used to use a #333 dark grey text on white, just because I thought it looked better that way. And to be honest, others were doing it, so I did it too. But when I changed the text to black on white, I find my blog easier to read. I hope you do too.The colors you choose for your text can affect the readability of your site. This is directly connected to usability.Don't believe me? Here is a sample paragraph, repeated using different colors.White on black:Space: the final frontier. These are the voyages of the starship Enterprise. Its continuing mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no one has gone before.Black on white:Space: the final frontier. These are the voyages of the starship Enterprise. Its continuing mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no one has gone before.White on dark gray:Space: the final frontier. These are the voyages of the starship Enterprise. Its continuing mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no one has gone before.Dark grey on white:Space: the final frontier. These are the voyages of the starship Enterprise. Its continuing mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no one has gone before.White on gray:Space: the final frontier. These are the voyages of the starship Enterprise. Its continuing mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no one has gone before.Gray on white:Space: the final frontier. These are the voyages of the starship Enterprise. Its continuing mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no one has gone before.Which one do you find easiest to read? [Less]
Posted 8 days ago by nore...@blogger.com (Marcus Lundblad)
So, at FOSDEM a bit over a week ago, me, Jonas Danielsson, Mattias Bengtsson, and Andreas Nilsson talked about plans for landing the transit routing feature and we started doing some patch reviewing over some beers on Saturday evening.Thanks to a lot ... [More] of awesome reviewing work done by Jonas, this work has finally landed in master!As always when merging such a long-living branch where I have been dwelling for about a year or so almost feels like saying farewell to an old friend.Though, until we have sorted out some infrastructure for running OpenTripPlanner, you would have to use the debug environment variable OTP_BASE_URL (or modify the service file) as I have described in earlier blog posts.Another feature that I intend to squeeze in before we release 3.24 is the ability to reverse routes. This is something that came up during Andreas' user interviews, one user had a use-case of going into town to meet up with friends and finding a suitable route home later in the evening. I realized our current UI is a bit awkward for this (even though you can drag and drop the entered places/addresses).So with this new feature you get a handy button to reverse what you just searched for:Searching for a route. After clicking the reverse button (the up/down arrows) Looking at one of tripsSo, until next time, map along! :-) [Less]