Parrot

FINALLY! The big day has come. I’ve just merged whiteknight/io_cleanup1 to master. Let us rejoice! When I started the project, months ago, I had intended to work on the branch for maybe a week or two at the most. Get in, clean what I could, get out. ... [More] Wash, rinse, repeat. That’s exactly why I named the branch “io_cleanup1”, because I intended it to just be the first of what would be a large series of small branches. Unfortunately as I started cleaning I was lead to other things that needed to go. And those things lead elsewhere. Before I new it I had deleted just about all the code in all the files in src/io/* and started rewriting from the ground up. Sometimes sticking with a plan and breaking up projects into small milestones is a good thing. Othertimes when you know what the final goal is and you’re willing to put in the effort, it’s good to just go there directly. That’s what I ended up doing. To give you an idea of what my schedule was originally, I had intended to get this first branch wrapped up and merged before GSOC started, so that I could keep my promise of implementing 6model concurrently with that program. With GSOC over last week (I’ll write a post-mortem blog entry about it soon), I’ve clearly failed at that. I’m extremely happy with the results so far and given the choice I would not go back and do things any differently. The IO system was in terrible condition and it desperately needed this overhaul. I wish it hadn’t taken me so long, but with a system that’s so central and important, it was worthwhile taking the extra time to make sure things were correct. Where to go from here? My TODO list for the near future is very short: Threads 6model More IO work The Threads branch, the magnum opus of Parrot hacker nine is 99.9% of the way there. If we can just push it up over the cliff, we should be able to merge soon and open up a whole new world of functionality and cool features for Parrot. I’m already planning out all the cool additions to Rosella I’m going to make once threads are merged: Parallel test harness. Asynchronous network requests, an IRC client library. The addition of a real, sane threading system opens up so many avenues to us that really haven’t been available before. Sure there are going to be plenty of hiccups and speedbumps to deal with as we really get down and start to use this system for real things, but the merge of the threads branch represents a huge step forward and a great foundation to build upon. I’m going to be putting forward as much effort as I can to getting this branch wrapped up and merged. Some of the remaining problems only manifest on hard-to-test platforms, which is where things start to get tricky. As I mentioned in an email to parrot-dev a while ago, test reports on rare platforms are great, but if we can’t take action on the reported failures we can get ourselves into something of a bind. The capability to find problems on those platforms and the capability to fix problems on those platforms are two very different capabilities. But, most of the time that’s a small issue and we’re going to just have to find a way to muscle through and get this branch merged one way or the other. If we can merge it without purposefully excluding any platforms, that would be great. Before anybody thinks that I’m done with IO and that system is now complete, think again. There is still plenty of work to be done on the IO subsystem, and all sorts of cool new features that become possible with the new architecture and unified type semantics. I want to separate out Pipe logic from FileHandle into a new dedicated PMC type. Opening FileHandles in “p” mode for pipes is clumsy at best, and I want a more sane system. And while I’m at it, 2-way and 3-way pipes would make for a great feature addition (we can’t currently do these in any reliable way). The one thing that has changed most dramatically in the new IO system is buffers. The buffering subsystem has not only been rewritten but completely redesigned. Instead of being type-specific they are now unified and type independent. Buffers are their own struct with their own API. Instead of having a single buffer that is used for both read and write, handles now have separate read and write buffers that can be created and managed independently. I want to create a new PMC type to wrap these buffers and give the necessary management interface so they can be used effectively from the PIR level and above. Finally, the whiteknight/io_cleanup1 branch tried to stay as backwards compatible as possible, so many breaking changes I wanted to make had to wait until later. In the future expect to see many smaller branches to remove old broken features, old crufty interfaces, and old bad semantics. We’ll make these kinds of disruptive changes in much smaller batches, with more space between them. [Less]

On behalf of the Parrot team, I'm proud to announce Parrot 4.7.0, also known as "Hispaniolan". Parrot is a virtual machine aimed at running all dynamic languages. read more

On behalf of the Parrot team, I’m proud to announce Parrot 4.7.0, also known as “Hispaniolan”. Parrot is a virtual machine aimed at running all dynamic languages. Parrot 4.7.0 is available on Parrot’s FTP site, or by following the download ... [More] instructions at http://parrot.org/download. For those who would like to develop on Parrot, or help develop Parrot itself, we recommend using Git to retrieve the source code to get the latest and best Parrot code. Parrot 4.7.0 News: - Core + Added .all_tags() and .all_tagged_pmcs() methods to PackfileView PMC + Several build and coding standards fixes The SHA256 message digests for the downloadable tarballs are: 4360ac3dffafffaa00bce561c1329df8ad134019f76930cf24e7a875a4422a90 parrot-4.7.0.tar.bz2 c0bffd371dea653b9881ab2cc9ae5a57dc9f531dfcda0a604ea693c9d2165619 parrot-4.7.0.tar.gz Many thanks to all our contributors for making this possible, and our sponsors for supporting this project. Our next scheduled release is 18 September 2012. The release is indeed out a day late. It’s not that I forgot about it, it’s just that I can’t read a calendar and HOLY CRAP, IT’S WEDNESDAY ALREADY? When did that happen? So, and I can’t stress this enough, Mea Culpa. [Less]

Tomorrow is google's appointed 'firm pencils down' date. That seems like a good time to discuss the results of my work on mod_parrot so far. mod_parrot is, as I have mentioned before, a two-layered system, with one half interfacing with apache (the ... [More] module) and the other half with the interpreter and the compilers, the 'loader'. There is also a vital third component, the test system called pudding. read more [Less]

This week I finally got arround to giving a new, fresh structure to the mod_parrot module code. I have complained, perhaps not loudly enough, about the various inadequacies of the old codebase, mostly with regards to extensibility. A cleanup was needed. As such, here is a walkthrough of the new structure, also serving as documentation. read more

I for one am totally for whimsical blog titles, wouldn't you agree? In other news, after a lull of two weeks (codewise at least) I've finally started to work on mod_parrot again. The big (dis)?advantage from not working on code is that you start to ... [More] think more of what you could do (and should have done), rather than what you have done. As it turns out, I handle interpreters in a rather confused manner. My goal for the next two weeks is to fix that. First, let me describe what should be done to correctly run a script on an interpreter using mod_parrot: read more [Less]

I appear to be continuing my weekly blogging every 14 days. Ah, well. My progress has been fairly intermittent as I work out this whole "getting sleep with a newborn around", but I'm starting to make real progress again. Today's blog will discuss ... [More] what I've done in the last couple weeks and an updated schedule for the next month. My progress can be split into a few topics: syntax highlighting, style changes, bug fixes, test helpers, and tests themselves. read more [Less]

This morning I made a few last commits on my whiteknight/io_cleanup1 branch, and I’m cautiously optimistic that the branch is now ready to merge. The last remaining issue, which has taken the last few days to resolve, has been fixing readine ... [More] semantics to match some old behavior. A few days ago I wrote a post about how complicated readline is. At the time, I thought I had the whole issue under control. But then Moritz pointed out a problem with a particular feature unique to Socket that was missing in the new branch. In master, you could pass in a custom delimiter sequence as a string to the .readline() method. Rakudo was using this feature like this: str = s.readline("\r\n") Of course, as I’ve pointed out in the post about readline and elsewhere, there was no consistency between the three major builtin types: FileHandle, Socket and StringHandle. The closest thing we could do with FileHandle is this: f.record_separator("\n"); str = f.readline(); Notice two big differences between FileHandle and Socket here: First, FileHandle has a separate record_separator method that must be called separately, and the record separator is stored as state on the FileHandle between .readline() calls. Second, FileHandle’s record separator sequence may only be a single character. Internally, it’s stored as an INTVAL for a single codepoint instead of as a STRING*, even though the .record_separator() method takes a STRING* argument (and extracts the first codepoint from it). Initially in the io_cleanup1 branch I used the FileHandle semantics to unify the code because I wasn’t aware that Socket didn’t have the same restrictions that FileHandle did, even if the interface was a little bit different. I also didn’t think that the Socket version would be so much more flexible despite the much smaller size of the code to implement it. In short, I really just didn’t look at it closely enough and assumed the two were more similar than they actually were. Why would I ever assume that this subsystem ever had “consistency” as a driving design motivation? So I rewrote readline. From scratch. The new system follows the more flexible Socket semantics for all types. Now you can use almost any arbitrary string as the record separator for .readline() on FileHandle, StringHandle and Socket. In the whiteknight/io_cleanup1 branch, as of this morning, you can now do this: var f = new 'FileHandle'; f.open('foo.txt', 'r'); f.record_separator("TEST"); string s = f.readline(); …And you can also do this, which is functionally equivalent: var f = new 'FileHandle'; f.open('foo.txt', 'r'); string s = f.readline("TEST"); The same two code snippets should work the same for all built-in handle types. For all types, if you don’t specify a record separator by either method, it defaults to “\n”. Above I mentioned that almost any arbitrary string should work. I use the word “almost” because there are some restrictions. First and foremost, the delimiter string cannot be larger than half the size of the buffer. Since buffers are sized in bytes, this is a byte-length restriction, not a character-length restriction. In practice we know that delimiters are typically things like “\n”, “\r\n”, ”,”, etc. So if the buffer is a few kilobytes this isn’t a meaningful limitation. Also, the delimiter must be the same encoding as the handle uses, or it must be able to convert to that encoding. So if your handle uses ascii, but you pass in a delimiter which is utf16, you may see some exceptions raised. I think that the work on this branch, save for a few small tweaks, is done. I’ve done some testing myself and have asked for help to get it tested by a wider audience. Hopefully we can get this branch merged this month, if no other problems are found. [Less]

"Wild Parrots of Telegraph Hill" read more

So, this was rather an unproductive week, unfortunately. I'm completely busy with moving right now (and will be coming week). What did happen is that I poked a hole into parrot, and the community (nine) fixed it. The story: I started my 'loader' ... [More] script by directly invoking a subroutine. That by-passed the starting of a green thread on the interpreter, which caused a crash when I tried to do something with that thread, such as sleeping. Nine fixed this issue by starting a green thread upon invocation using the api, which causes my tests to crash no more. read more [Less]