You know that a conference is good when you go home with a list of stuff that you never heard of but now need to go follow up on. The JVM Language Summit was exceptional in this regard. Sun provided a location and a few of the speakers, but most of the speakers at the Summit were not Sun employees, although there were a few Sun alumni amongst the speaker ranks. The topics that were discusssed went all the way from type theory (including the usual greekified type proofs), typical language design stuff, VM design, all the way down to discussions of how high allocation rates can cause hot data to get flushed out of caches on the bare metal. Slides for all the talks are available on the wiki for the conference, and some of the talks will have video at either InfoQ or YouTube. Here are some of my favorites from the three days.

Clojure

I’d been aware of Clojure prior to the summit and had looked at the page on Clojure’s use of persistent data structures, so I thought that I had some idea of what was interesting about Clojure. I was wrong. Rich Hickey’s 30 minute presentation on Clojure had a really large amount of information per unit time. By the end of the time I was really interested in Clojure, and I was able to find out a bit more about it by going to an open space session and by being at the same table as Rich during dinner one night. As and old Lisp guy, my usual reaction to Lisps on the JVM or CLR is why? They don’t typically fit in that well with the host VM, and there are great implementations of Common Lisp that can compile to very efficient machine code. I was looking forward to Arc, but that has turned out to be very disappointing. Clojure has taken a very practical approach to the Lisp parts of the language. It fits in very nicely with the JVM, is able to call Java code easily and has the potential to achieve very good performance on the JVM. Also, Rich has made a number of design decisions which improve the syntax (he showed a short program in both Python and Clojure, and they occupy the same amount of vertical space and have roughly the same visual density) of Clojure. He’s also generalized many operations that would have worked on lists to work on sequences, which really means any Java sequence type. Like many Lisps, you can supply data type hints, and the compiler will use those to make the program more efficient. There is a nice library of collection operations, which look very comparable (or better) than Python or Ruby’s facilities for collection types. There are some other really interesting data structures in the libraries, like bit partitioned hash tries and zippers.

Beyond the Lispish stuff in Clojure, there are a several interesting features in Clojure related to the problem of concurrency. In Clojure things are immutable by default, which is a huge benefit – a benefit shared by functional languages, and quasi functional languages like Erlang. Beyond that, Clojure supports persistent data structures as a way of managing state in a concurrency friendly manner. The idea is that “updating” a persistent data structure yields both the version before and after the update. This means that updates don’t impede readers of the old version, and are not blocked by readers of the old version.

Lastly, Clojure provides an interesting mechanism for utilizing Software Transactional Memory. Normally STM systems make all accesses to memory transactional. This makes the STM a bottleneck, and makes it much more likely that the performance of the STM system will be the limiting factor in a concurrent system. Clojure requires you to make uses of the STM explicit via its Ref structure. This yields a potentially much more controlled usage of the STM, which could help preventing the STM from being a bottleneck.

My original impression of Clojure was that it was still in the very early stages, but it seems to be bit further along that that. I was surprised by the size of the community, and by other parts of the ecosystem, like the tool support. There are several Emacs modes, integration with SLIME, and even a Netbeans plugin for Clojure.

I will definitely be giving Clojure a closer look, and I am not alone. There was a lot of energy in the room during and after Rich’s talk, and there was a burst of Twitter traffic during the talk. It’s pretty interesting to see the number of language geeks on Twitter.

Davinci Machine

If you’ve been following John Rose’s blog and Charlie Nutter’s recent writings on invokedynamic, you wouldn’t be very surprised by the content of John’s presentation on the DaVinci Machine Project. This is a highly important piece of work for non-Java languages on the JVM, so it was good to hear John tell a more complete version of what he’s been up to. It was also my chance to meet John in person. We somehow missed each other at JavaOne, so it was good to put a face to the name, and have some in person contact. John and Brian Goetz did a great job of organizing the summit, and John was always trying to find out what kind of features would be useful to JVM implementors. JSR-292 can’t happen soon enough.

Fortress

David Chase talked about the work that folks at Sun labs have been doing on Fortress. I never really paid much attention to Fortress, since they are really aiming at the scientific, high performance computing space, and that’s kind of outside of my interests. The Fortress guys are doing some interesting explorations as far as concurrency is concerned. In fact, David referred to Fortress as “infested with parallelism”. My todo item from the Fortress talk has to do with the work-stealing model that they have for concurrency. Apparentl this work is based on a data structure known as an ABP queue, so I’ll be tracking down the paper on that one.

JRuby

I’ve heard Charlie Nutter talk about JRuby several times, and have talked with him a little about JRuby. Even so, his talk on JRuby was really interesting, because he was able to go full out for an hour to a very sophisticated audience. I know from talking to some of the Jython guys, that there were a few aha moments for them, even though they’ve been to the same talks that I’ve been to.

Dynalang MOP

Attila Szegedi described his proposal for a MOP for dynamic languages. Once you start hosting a bunch of languages on the JVM (or any VM), then people start to ask if they can call code written in language A (say Clojure) from language B (say Python). The tough part is that the code in language A may have compiled to Java bytecodes in a way that doesn’t really resemble Java code, and you can end up in a situation where B can call A but does it by grabbing things which are really artifacts of A’s implementation. Of course, A’s implementors will continue to improve A, and in the process of doing so, might change the details. You can see what the problem is going to be. Attilla’s MOP would go a long way towards helping here. I hope that people will give it a serious look.

Gradual Typing

Jython committer Jim Baker has been after me about the work that Jeremy Siek (UC Boulder) has been doing on adding types to Python. His system is called gradual typing and allows a programmer to selectively add type annotations to a program. It’s a cool idea in principle, and I hope that it will end up being cool when it finally gets implemented all the way. I have to admit that the first time that I saw an annotated program, I had a violent reaction. There were a ton of angle brackets due to the type annotations. Jeremy and his students are working on ways the reduced the amount of notation that is needed. I hope that they’ll be successful — in Python at least, it’s going to be key to whether people will adopt it or not.

Fundamentalist FP

I’ve been an admirer of Erik Meijer‘s work for some time, so I was glad to be able to hear him speak in person. There was another talk on LINQ, so Erik didn’t talk about that. Instead he talked about what he called “Fundamentalist Functional Programming”, which is really just the functional programming that the old school functional programming people have always talked about. I think that Erik is concerned about the amount of lifting of functional programming ideas and idioms, without a full understanding of the essence of functional programming. His presentation style is very entertaining. The major thrust of his argument was that for the past 50 years of computing, we have been abstracting, but abstracting over the wrong things. He asserted that the thing that we really need to do is to abstract over evaluation order. Given the coming many/multicore world, this is understandable, but don’t think that I agree that all the other lessons that we’ve learned about abstraction are invalid. He provided the simplest explanation of monads that I have ever heard or read, as well as showing how to handle things like object creation and process creation monadically. In the end, though, his talk reduced to the essentials of lazy pure functional programming.

Bytecodes for fast JVM’s

Cliff Click asked that JVM language implementors send him an implementation of a particular program written in their language. Cliff then ran those programs (in their respective languages) on Hotspot and on the Azul JVM. His talk was a report of his findings as to what was keeping various languages from getting the best results on the JVM. He said that he wasn’t trying to compare the merits of one language versus another, but more to give the implementors insight into what was up with their code. I found this talk to be tremendously interesting because Cliff really knows the guts of HotSpot and because he was able to be very specific about what was causing problems for the various languages.

Parrot

I’ve known Allison Randal for several years now, mostly via her organizing of the FLOSSFoundations meetings that happen every year at OSCON. In all that time, we’ve never really sat down and talked about her work on Parrot, and it’s been several years since I heard a talk on Parrot. I give John and Brian a lot of credit for inviting Allison to come and talk about Parrot. The architecture of Parrot is very unlike either the JVM or the CLR. They started with very different assumptions and goals, which unsurprisingly lead to a different design. As far as I can tell, Parrot is looking reasonable on the performance front, will be able to use the C libraries of Python, Ruby, PHP, etc, without much hassle, and will have a good story for interoperability between hosted languages. Control flow is modeled using continuations which means that continuations are really cheap to create. Allison also talked about what a different method of doing call site caching – Parrot does the caching in the Parrot class object, not in little caches strewn all over the call sites. This makes it easy to invalidations the cache when the class hierarchy changes, for example. I’m still trying to digest all of what I heard, as well as the conversation that several of us had with Allison after her talk.

The Parrot team has been lying low and working away on Parrot, and they are definitely making progress. Allison showed some very preliminary benchmarks of the incomplete Python, Ruby, and PHP implementations on top of Parrot versus the C based versions. She told me afterwards that the project has reached the point where they are working to time based milestones, and that they are hoping to do a 1.0 release early in 2009. Chalk up another to-do.

Random Thoughts

There aren’t any pictures to go with this report because I was not motivated to take any. There were several people snapping away quite frequently during the conference, and I didn’t want to add the slaps from the D3 to the cacophany and the flash light show.

It seems clear to me that many folks share some of the same problems, and I hope that on result of the summit will be that people will start to work together when it makes sense. I know that the Jython and JRuby folks are working in that direction, and it seems likely to me that there will be some collaboration around the dynalang MOP as well. There was a lot of good energy in the room: people were very respectful and curious about other people’s work.

I think that the only regret that I had was that this was the first annual JVM Language Summit. Imagine where we’d be if this had been the fifth…

Update: finished the sentences about persistent data structures in Clojure