Archive for the ‘programming languages’ Category

The First Annual JVM Language Summit

Saturday, September 27th, 2008

You know that a conference is good when you go home with a list of stuff that you never heard of but now need to go follow up on. The JVM Language Summit was exceptional in this regard. Sun provided a location and a few of the speakers, but most of the speakers at the Summit were not Sun employees, although there were a few Sun alumni amongst the speaker ranks. The topics that were discusssed went all the way from type theory (including the usual greekified type proofs), typical language design stuff, VM design, all the way down to discussions of how high allocation rates can cause hot data to get flushed out of caches on the bare metal. Slides for all the talks are available on the wiki for the conference, and some of the talks will have video at either InfoQ or YouTube. Here are some of my favorites from the three days.


I’d been aware of Clojure prior to the summit and had looked at the page on Clojure’s use of persistent data structures, so I thought that I had some idea of what was interesting about Clojure. I was wrong. Rich Hickey’s 30 minute presentation on Clojure had a really large amount of information per unit time. By the end of the time I was really interested in Clojure, and I was able to find out a bit more about it by going to an open space session and by being at the same table as Rich during dinner one night. As and old Lisp guy, my usual reaction to Lisps on the JVM or CLR is why? They don’t typically fit in that well with the host VM, and there are great implementations of Common Lisp that can compile to very efficient machine code. I was looking forward to Arc, but that has turned out to be very disappointing. Clojure has taken a very practical approach to the Lisp parts of the language. It fits in very nicely with the JVM, is able to call Java code easily and has the potential to achieve very good performance on the JVM. Also, Rich has made a number of design decisions which improve the syntax (he showed a short program in both Python and Clojure, and they occupy the same amount of vertical space and have roughly the same visual density) of Clojure. He’s also generalized many operations that would have worked on lists to work on sequences, which really means any Java sequence type. Like many Lisps, you can supply data type hints, and the compiler will use those to make the program more efficient. There is a nice library of collection operations, which look very comparable (or better) than Python or Ruby’s facilities for collection types. There are some other really interesting data structures in the libraries, like bit partitioned hash tries and zippers.

Beyond the Lispish stuff in Clojure, there are a several interesting features in Clojure related to the problem of concurrency. In Clojure things are immutable by default, which is a huge benefit – a benefit shared by functional languages, and quasi functional languages like Erlang. Beyond that, Clojure supports persistent data structures as a way of managing state in a concurrency friendly manner. The idea is that “updating” a persistent data structure yields both the version before and after the update. This means that updates don’t impede readers of the old version, and are not blocked by readers of the old version.

Lastly, Clojure provides an interesting mechanism for utilizing Software Transactional Memory. Normally STM systems make all accesses to memory transactional. This makes the STM a bottleneck, and makes it much more likely that the performance of the STM system will be the limiting factor in a concurrent system. Clojure requires you to make uses of the STM explicit via its Ref structure. This yields a potentially much more controlled usage of the STM, which could help preventing the STM from being a bottleneck.

My original impression of Clojure was that it was still in the very early stages, but it seems to be bit further along that that. I was surprised by the size of the community, and by other parts of the ecosystem, like the tool support. There are several Emacs modes, integration with SLIME, and even a Netbeans plugin for Clojure.

I will definitely be giving Clojure a closer look, and I am not alone. There was a lot of energy in the room during and after Rich’s talk, and there was a burst of Twitter traffic during the talk. It’s pretty interesting to see the number of language geeks on Twitter.

Davinci Machine

If you’ve been following John Rose’s blog and Charlie Nutter’s recent writings on invokedynamic, you wouldn’t be very surprised by the content of John’s presentation on the DaVinci Machine Project. This is a highly important piece of work for non-Java languages on the JVM, so it was good to hear John tell a more complete version of what he’s been up to. It was also my chance to meet John in person. We somehow missed each other at JavaOne, so it was good to put a face to the name, and have some in person contact. John and Brian Goetz did a great job of organizing the summit, and John was always trying to find out what kind of features would be useful to JVM implementors. JSR-292 can’t happen soon enough.


David Chase talked about the work that folks at Sun labs have been doing on Fortress. I never really paid much attention to Fortress, since they are really aiming at the scientific, high performance computing space, and that’s kind of outside of my interests. The Fortress guys are doing some interesting explorations as far as concurrency is concerned. In fact, David referred to Fortress as “infested with parallelism”. My todo item from the Fortress talk has to do with the work-stealing model that they have for concurrency. Apparentl this work is based on a data structure known as an ABP queue, so I’ll be tracking down the paper on that one.


I’ve heard Charlie Nutter talk about JRuby several times, and have talked with him a little about JRuby. Even so, his talk on JRuby was really interesting, because he was able to go full out for an hour to a very sophisticated audience. I know from talking to some of the Jython guys, that there were a few aha moments for them, even though they’ve been to the same talks that I’ve been to.

Dynalang MOP

Attila Szegedi described his proposal for a MOP for dynamic languages. Once you start hosting a bunch of languages on the JVM (or any VM), then people start to ask if they can call code written in language A (say Clojure) from language B (say Python). The tough part is that the code in language A may have compiled to Java bytecodes in a way that doesn’t really resemble Java code, and you can end up in a situation where B can call A but does it by grabbing things which are really artifacts of A’s implementation. Of course, A’s implementors will continue to improve A, and in the process of doing so, might change the details. You can see what the problem is going to be. Attilla’s MOP would go a long way towards helping here. I hope that people will give it a serious look.

Gradual Typing

Jython committer Jim Baker has been after me about the work that Jeremy Siek (UC Boulder) has been doing on adding types to Python. His system is called gradual typing and allows a programmer to selectively add type annotations to a program. It’s a cool idea in principle, and I hope that it will end up being cool when it finally gets implemented all the way. I have to admit that the first time that I saw an annotated program, I had a violent reaction. There were a ton of angle brackets due to the type annotations. Jeremy and his students are working on ways the reduced the amount of notation that is needed. I hope that they’ll be successful — in Python at least, it’s going to be key to whether people will adopt it or not.

Fundamentalist FP

I’ve been an admirer of Erik Meijer‘s work for some time, so I was glad to be able to hear him speak in person. There was another talk on LINQ, so Erik didn’t talk about that. Instead he talked about what he called “Fundamentalist Functional Programming”, which is really just the functional programming that the old school functional programming people have always talked about. I think that Erik is concerned about the amount of lifting of functional programming ideas and idioms, without a full understanding of the essence of functional programming. His presentation style is very entertaining. The major thrust of his argument was that for the past 50 years of computing, we have been abstracting, but abstracting over the wrong things. He asserted that the thing that we really need to do is to abstract over evaluation order. Given the coming many/multicore world, this is understandable, but don’t think that I agree that all the other lessons that we’ve learned about abstraction are invalid. He provided the simplest explanation of monads that I have ever heard or read, as well as showing how to handle things like object creation and process creation monadically. In the end, though, his talk reduced to the essentials of lazy pure functional programming.

Bytecodes for fast JVM’s

Cliff Click asked that JVM language implementors send him an implementation of a particular program written in their language. Cliff then ran those programs (in their respective languages) on Hotspot and on the Azul JVM. His talk was a report of his findings as to what was keeping various languages from getting the best results on the JVM. He said that he wasn’t trying to compare the merits of one language versus another, but more to give the implementors insight into what was up with their code. I found this talk to be tremendously interesting because Cliff really knows the guts of HotSpot and because he was able to be very specific about what was causing problems for the various languages.


I’ve known Allison Randal for several years now, mostly via her organizing of the FLOSSFoundations meetings that happen every year at OSCON. In all that time, we’ve never really sat down and talked about her work on Parrot, and it’s been several years since I heard a talk on Parrot. I give John and Brian a lot of credit for inviting Allison to come and talk about Parrot. The architecture of Parrot is very unlike either the JVM or the CLR. They started with very different assumptions and goals, which unsurprisingly lead to a different design. As far as I can tell, Parrot is looking reasonable on the performance front, will be able to use the C libraries of Python, Ruby, PHP, etc, without much hassle, and will have a good story for interoperability between hosted languages. Control flow is modeled using continuations which means that continuations are really cheap to create. Allison also talked about what a different method of doing call site caching – Parrot does the caching in the Parrot class object, not in little caches strewn all over the call sites. This makes it easy to invalidations the cache when the class hierarchy changes, for example. I’m still trying to digest all of what I heard, as well as the conversation that several of us had with Allison after her talk.

The Parrot team has been lying low and working away on Parrot, and they are definitely making progress. Allison showed some very preliminary benchmarks of the incomplete Python, Ruby, and PHP implementations on top of Parrot versus the C based versions. She told me afterwards that the project has reached the point where they are working to time based milestones, and that they are hoping to do a 1.0 release early in 2009. Chalk up another to-do.

Random Thoughts

There aren’t any pictures to go with this report because I was not motivated to take any. There were several people snapping away quite frequently during the conference, and I didn’t want to add the slaps from the D3 to the cacophany and the flash light show.

It seems clear to me that many folks share some of the same problems, and I hope that on result of the summit will be that people will start to work together when it makes sense. I know that the Jython and JRuby folks are working in that direction, and it seems likely to me that there will be some collaboration around the dynalang MOP as well. There was a lot of good energy in the room: people were very respectful and curious about other people’s work.

I think that the only regret that I had was that this was the first annual JVM Language Summit. Imagine where we’d be if this had been the fifth…

Update: finished the sentences about persistent data structures in Clojure

Book Review: Learning Website Development with Django

Friday, September 19th, 2008

The folks at Packt Publishing sent me a copy of Learning Website Development with Django by Ayman Hourieh. It’s a fairly thin book, weighing in at just 243 pages, but it it is a very good introduction to Django. Knowledge of Python is assumed, so if you need that information, you’ll need another book. Hourieh uses a running example, a social bookmarking application, throughout the book. This means that he is adding features to the application in each chapter, which is a style that I like because it shows how to use Django to build up a decently featured application piece by piece. There is lots of code, which I found easy to read, except for some places where the bold facing of new code was somehow omitted.

The only problem with the book was that it was finished before Django went 1.0, so there are undoubtedly some places where things are not quite in sync with Django 1.0. Hourieh did make an effort to deal with some of the largest 1.0 changes, like newforms, but omitted other like Unicode support. This is inevitable when the book and software are not in sync, but since Django only went 1.0 a few weeks ago, the only documentation that is completely up to date is the official Django documentation itself.

If you know Python and for some reason have been holding off on learning Django, this is a good book to help you get started. There is also an e-book version available via Packt’s website.

PyConUK 2008

Thursday, September 18th, 2008

I am slowly recovering from the transatlantic flights for PyCon UK, so I think that I’m ready to give a report.

The conference

PyCon UK 2008

In Europe, at least, the Python community seems to be sprouting more regional conferences as opposed to having the single EuroPython conference. This probably makes it easier for people to get to a conference, but it also means that people like me have to travel more in order to attend all these meetings. PyCon UK, also takes place over a weekend, again to facilitate people being able to attend without the hassle of getting time off from work and so forth. The conference was roughly the size of EuroPython, with attendance in the 230-250 range.

As usual, I did some of the regular program and some hallway track. The top three talks that I went to were Jacob Kaplan-Moss‘s update on the new stuff in Django 1.0 (that’s Jacob in the picture above). Chris Withers did a good talk on the problems with Python package management. The basic answer there is that a lot of stuff layered on top of a shaky foundation, in this case, distutils. Chris discussed some (partial, in my opinion) solutions to the problems, the virtualenv and buildout packages. Perhaps one day there will be a really good solution. Micheal Sparks from BBC research described the Kamaelia project, which includes a Software Transactional Memory (STM) module. I am neutral on the benefits of transactional memory, so I am glad to see that someone in the Python community is exploring how STM might fit with Python. Slides for some of the talks are being posted on the conference wiki. In the hallway track, I attended a BOF for Python Bloggers, and a PGP keysigning. I also had some excellent extended conversations with Holger Krekel and Maciej Fijalkowski of PyPy, Adewale Oshineye from Google, and Michael Foord from Resolver Systems.

I was invited to give one of the two keynotes for the conference, and my topic was “Challenges for Dynamic Languages”. I tried to step back from the specifics of Python and look at some of the challenges that face all of the dynamic language communities if we want to gain broader adoption of dynamic languages. I was somewhat concerned when Mark Shuttleworth covered some of the same topics during his keynote the day before, but maybe it’s a good sign when people from two different organizations and backgrounds can come to similar conclusions about the future of Python and related languages. My presentation is very picture oriented, without a lot of text or bullet points, so I am not sure whether posting the slides will actually help anyone or not. This was my first time giving a keynote, and it seems like it was well received, despite some laptop/projector interaction problems. Thanks to Jacob Kaplan-Moss for saving the day by lending me his MacBook Pro, remote, and memory stick.


PyCon UK 2008

This was my second transatlantic flight since June, and it seems that I really can’t sleep on long airplane flights. This turns the 10 hour flight from Seattle to most Star Alliance hubs into a pretty long ordeal. I did manage to finish a book and read a few papers, but I spent way too much time using the in flight entertainment system (I prefer Lufthansa’s to SAS’s). I also experienced my longest wait (1 hour) in passport control in the Birmingham airport. During that wait, I listened to the two American women right in front of me discovering that each of them was moving to the UK to marry an Englishman. Ten people or so before I reached the inspectors, a passenger collapsed in an epileptic fit (no previous history apparently), just to make sure things were memorable.

I was only in Birmingham for two and a half days, and spent almost two days inside of airplanes. Several people commented on the jet lag, which didn’t seem that bad until it caught up with me when I finally arrived home.


Photographically, perhaps the bigger story is the pictures that I didn’t get. The D3 does not fit in my camera/computer bag with a lens mounted on it, which means that it is hard to get shots where you just need to pull the camera out, point and shoot. I had a flight leg from Seattle to Frankfurt, and on the Frankfurt end we took busses from the airplane to the terminal. While we were waiting for all the passengers to board the bus, I realized that I was staring at a great view of the rear left quarter of an Airbus A330-300. Perfect for a wide angle shot, but there wasn’t enough time to get the shot.

When I arrived on Thursday, I went over to the Pycon UK social, had some dinner and hung out. I left a little early when the noise level got to be more than I could handle. As I was walking back to the hotel, I saw some beautiful artwork that was being projected on the side of a building in Paradise Circle as part of Birmingham’s Light Night and Artsfest. I had left the camera at the hotel because I didn’t want to have to haul it around, but I was so taken with the display that I went and picked it up. Here’s some of what I saw:

PyCon UK 2008

PyCon UK 2008

PyCon UK 2008

This is a shot as I walked back to my hotel via one of Birmingham’s canals.

PyCon UK 2008

Here‘s the rest of the set. There are not many pictures of the conference proper — I go to a lot of conferences now, and I’ve shot enough pictures of people sitting in rows or giving a talk. There’s an entire group on Flickr for PyCon UK if you want better coverage of the conference.

OSCON 2008

Tuesday, July 29th, 2008

Another OSCON has come and gone, and as usual, I am exhausted in the aftermath. I’ve developed a love-hate relationship with OSCON over the years. The diversity of the OSCON community is one of the huge pluses of the conference. I got involved in open source via Apache, and OSCON was where I really started to get more of a sense of the open source community as a whole. That’s led to friendships with people doing all sorts of open source stuff, which makes the conference a natural place to reconnect with many of those folks. Which leads to the primary downside of OSCON, which is that there is just no way to keep up with, never mind see all the people that you’d like to see. Combine that with the sheer scale of the event, and you have recipe for burnout. This year is no exception, which is why this post is delayed by a few days.


It’s fitting to start a review of OSCON with programming languages, since OSCON began as a Perl conference. There are still lots of Perl hackers running around, and by the distribution of the program (the Python track was 1 day shorter than the Perl, PHP, and Ruby tracks), it seems that Perl is not going anywhere anytime soon. I think that we are going to need to drum up some more Python talks for OSCON next year. Then again, with PyCon topping 1000 people this year, maybe all the Python folks are going there. It certainly is cheaper than going to OSCON. Despite all of this, I saw lots of people that I knew from the Python community, as well as plenty of people who had affixed a yellow Python ribbon to their badge. The ribbons are a nice way of helping people find their tribe at a big show like OSCON – a lower tech version of what the Pathable folks are doing.

I spent a lot of time nosing around various concurrency oriented sessions. I attended Steven Parkes’ tutorial on Actors, which was pretty well attended. Steven has implemented a version of Actors as a set of Ruby and Python libraries. During the tutorial I was able to meet Debasish Ghosh, who has a great blog and Twitterstream on high-level languages, and concurrency topics in general. I also took in a BOF on Actors, which had some really interesting conversation. There were a lot of Erlang folks in the room for that one, which made the discussion pretty interesting.


OSCON 2008

There was lots of non-traditional database stuff happening at OSCON this year. I am one of the mentors for the CouchDB project at Apache, and I was finally able to meet my first CouchDB commiter, Jan Lehnardt, at the show. Jan gave a nice high level overview talk on CouchDB, which was well attended, and I was interested to see Brian Aker of MySQL/Drizzle in the audience and among the throng of questioners after the talk.

OSCON 2008

I also went to a talk on Prophet, which is a peer to peer database that is being done by some of the folks that brought us SVK. I’m not sure that I quite recovered from my initial reaction to that revelation, but Jan was sitting next to me during the entire talk, and was saying something about stealing some ideas from the Prophet guys. In open source we call that standing on the shoulders of giants, or something like that.


The XMPP folks had a three day summit during the conference, which I gather was well attended. There was a decent amount of XMPP buzz floating around in the hallways, so I expect the blogosphere to be full of XMPPness during the next week or so. I’ve done a bunch of blogging on XMPP in the past, and while things have improved, they haven’t improved to the point where XMPP is taking over the world. Things like Twitter are definitely helping, but there is still a long way to go before XMPP achieves world domination. But we can hope. And at least XMPP makes a great advertisement for Erlang.

Along with XMPP, we had the microblogging meme. I made heavy use of Twitter throughout the week, and it definitely played a useful part in making connections with people. Well, except for the times when it was down. I was able to spend a little time with Leah Culver, the founder of Pownce, which has the virtue of being written in Python, and of having a very nice API for dealing with the service. It’s interesting to get additional perspectives on a problem, and since I had already talked some with the Twitter guys, it definitely helped to hear Leah describe Powce’s take on the problem(s) and solutions. O’Reilly was not to be outdone, and did some very active boostering for I’ve got very mixed feelings on One the one hand, I should love, because it’s open source. On the other hand, it’s written in PHP, which means I won’t be touching the code, and more importantly, my network is not there. Actually, it was kind of annoying to have to explain to lots of zealots that it’s the network that’s the value, not the software, or ironically, the quality of the service. Still if another microblogging service can convince my network to move, and remain up, and even deliver some new functionality, I would definitely switch. I think I could probably write another post about “microblogging”, but I’ll refrain for now.

Theo Schlossnagle gave an amazing presentation called “Full-stack introspection crash course”, which is code for “let me show you some amazing stuff that you’ll only be able to do with DTrace”. This was a brilliant choice of title on Theo’s part, because it didn’t scare away all of those people whose preconceptions about DTrace or Sun would prevent them from coming to such a talk. Instead, Theo played to a very full room, and I would say that about one-third of the audience actually uttered the phrase “Oh My God” out loud at some point during the presentation. This was certainly true for thetwo gentlemen sitting directly to my right and directly behind me. I later heard from people at the Sun booth, that a bunch of people came to the booth having heard about DTrace (I assume at Theo’s talk), asking for whatever CD’s they needed in order to be able to use it. Theo clearly understands how to communicate about DTrace. We at Sun need to learn that lesson.

Open Source

Of course, you can’t have a conference on open source without meta stuff about open source itself. I was fortunate to attend the morning session of Microsoft’s Participate08 event, which was an interesting case study led by Karim Lakhani from the Harvard Business School. The case was on and involved a lot of issues which are very relevant to injecting corporate involvement into an existing community based organization. I’ve been following Karim’s work over the years (he studied under Eric von Hippel, whose work I am also fond of), so I was happy for the chance to meet him and participate in an activity with him. I also met Siobhan O’Mahony, who is also doing great work studying open source communities. I’m not sure what direct value Microsoft got out of sponsoring Participate, other than being able to say that they did an event around OSCON, but I know that I definitely appreciated the chance to interact with a bunch of people.

OSCON 2008

Microsoft was all over the news by the end of OSCON, having announced that they would become a Platinum sponsor of the Apache Software Foundation. This was not a complete surprise to me: Justin Erenkrantz, the current ASF president told me what was happening the night before at a party. I think that this is an interesting step for Microsoft, an it’s definitely a step in the right direction. However, as one questioner pointed out, Microsoft has a long history of incendiary rhetoric towards the open source community, and that’s going to mean that just about everything happens in steps. I do find it interesting that one of the reasons that the ASF has taken donations is to build up a legal defense fund against what we regarded as inevitable legal attacks. It’s somehow ironic to think of Microsoft’s $100,000 going into that pool. I think that the next interesting milestone in Microsoft’s relationship with the ASF will be when the first Microsoft sponsored project shows up at the front door of the Apache Incubator.

I also contributed to the metaness with a talk titled “Open Source Community Antipatterns” (slides are now available on the O’Reilly slide page). The talk was decently attended, but I suspect that the all-star antipatterns panel immediately following my talk drew off some of the audience that might have come to my talk. The people track expanded a great deal this year, which I think is a good thing.


I always have photographic memories associated with OSCON. I got my first digital SLR, right before OSCON 2005, and I’ve shot a bit a each OSCON, and even won the OSCON photo contest one year. This year I found myself shooting less. There were too many other things that I needed to do, and between knowing that Duncan is making is covering stuff and some artistic blockage, I lacked both time and motivation to crank out the shots.

Duncan has been a great friend and photographic mentor, and I always look forward to catching up with him during OSCON. This time was no exception. We did a bunch of stuff together, ranging from hanging out, having a wide angle shootout (well he was wide) to Duncan putting one of his cards into my D3 and giving the pixels a once over. Probably the most fun thing that we did was an impromptu photoshoot. Duncan was shooting headshots of the OSCON staff for a thank you slide for the closing keynotes. Only problem was that he needed one of himself, so he drafted me. With the safe shot in hand, we spent a few more minutes doing something a little edgier and fun.

OSCON 2008


That’s it for another OSCON. I hope we’ll be back in Portland again next year.

IDE’s and Dynamic Languages

Sunday, July 20th, 2008

Just about two weeks ago, Stephen O’Grady was wondering about the market for IDE’s for dynamic languages. His post followed on Sun’s announcement that we’re going to be doing some work on IDE support for Python. Here’s his closing pondering:

But I do wonder how much success IDEs will have in markets currently dominated by text editors, and what the metrics for success will look like for IDE advocates and purveyors.

Back to the Future

One of the points that I’ve been trying to make since I’ve gotten back in to the languages space is that a lot of what is happening in languages now is unpausing the nuclear winter that Java imposed on the programming language space. If you haven’t been following this space for a while, you’d believe that all this dynamic language stuff was invented in the last 5 or 10 years or so. But it wasn’t. Today when you say dynamic languages, people assume that you are talking about Ruby, Python, Groovy, Javascript, PHP, or Perl, with a number of other languages also entering the mix. The truth is that the intellectual forbears of these languages, Lisp and Smalltalk, were invented about 30 years ago or so. At their height, both of them were running on hardware that was specifically designed to execute programs written in these languages. The ideas for GUI’s which we now take for granted in our modern desktop work environments were pioneered on these systems. These systems included the predecessors of today’s IDE’s. They were written in dynamic languages, for use on dynamic languages. They did not include some of the most maligned features of “modern” IDE’s, such as bloat or wizards that spew out pages of autogenerated boilerplate code. People that actually have used these systems were highly productive on them.

I don’t think that word means what you think it means…

Stephen asserts that he’s talked to lots of dynamic language developers and that they don’t use IDE’s. He cites the Rails committers usage of Textmate as proof that IDE’s are not being used. I guess it depends what an IDE is. Textmate includes automatic syntax verification and code completion, which are not really features that have anything to do with edit text, and both of which require some understanding of what a program means in order to do their job. I’d call those IDE features. The ability to perform semantic manipulations on programs is the essence of what I think an IDE is. Maybe we should just say “good tools” instead of IDE, since a number of people that I’ve talked to seem to think that in order to be an IDE, the tool has to be huge, written in Java, and include feature for generating reams of boiler plate code – none of which fit my criteria for what makes a good programming tool. As best I can tell, Textmate (and other text editors) do not provide features like “find all uses of an identifier” or the ability to refactor ones code. Given the high Test Driven nature of the Ruby community, and to a lesser extent the Python community, I’d be surprised if a tool that could correctly and reliably refactor programs would be uninteresting.

The flight to dynamic languages is a flight away from boilerplate code, a flight to semantically richer control abstractions via mechanism like closures, a flight to appropriate use of metalinguistic programming via meta-level functionality (also known as Doman Specific Languages), and generally a flight to more productivity. If developers are willing to change languages to get more productivity, I believe that sufficiently powerful tools will also be adopted if they can consistently deliver higher degrees of productivity.

Chasm crossing

Stephen talked to existing dynamic language developers. While early adopters are highly valuable audience, they are not the only audience. The lack (or perceived lack, depending on your point of view) of good tools is a barrier to the adoption of dynamic languages. If dynamic languages are to attract a much larger audience, then I think that the development of good tools is an important step towards making that happen. If you think that that tool is named an “IDE”, then we need that.

My experience working at OSAF was that we hired a number of people who were skilled developers but had no previous Python experience. These folks were able to learn the language without any problems, but many of them wanted richer tools. Most of the available tools for Python had problems of one kind or another, including slowness, instability, unreliability and lack of refactoring tools. These were highly skilled people who had worked on and shipped big systems before, who both appreciated the benefits of Python, and the benefits of high quality tools. There are more of these people who are the target audience to jump to using dynamic languages, and I think that having a good tool story for these folks is important.

What is success?

Stephen wants to know what success looks like. Here’s my personal take:

Deliver tools that understand programs at a semantic level and which use that knowledge to reliably refactor dynamic languages using a substantial number of the refactorings in the book “Refactoring: Improving the Design of Existing Code” (Martin Fowler, Kent Beck, John Brant, William Opdyke, Don Roberts). These tools should have outstanding user experiences, including performance and footprint characteristics. People coming from Eclipse/Netbeans/IntelliJ/Visual Studio/XCode should not be wishing for their old tools when using these tools.

Once such tools are actually shipping, there are several possible definitions of success:

In the Java world (not so much the C# world) you have your IDE people, and you have the people using vi and emacs. If the percentage breakdown of IDEs vs text editors for dynamic languages, is the same as that for Java, then that’s success.

Real success would be when lots of people willingly give up their text editors with IDE features for these tools – I was an Emacs guy until Eclipse started refactoring programs.

Jython 2.5 Alpha 1

Tuesday, July 15th, 2008

The Jython development team has released the first alpha of Jython 2.5. The guys were hoping to have this done in time for EuroPython, but it wasn’t to be. Still, they are ahead of where they planned to be, so this is good news. If you are interested in an up to date version of Python on the JVM, go kick the tires and report bugs. Getting more testing done should help speed the process of shaking out all the bugs.

My first EuroPython

Friday, July 11th, 2008

I’ve been doing Python stuff since 2003, and my first PyCon was back in 2004, but this was the first time that I’ve been able to attend EuroPython. The conference reminded me very much of the first PyCon that I went to back in 2004. It was around 250 people, and it had a much smaller and more intimate feeling to it. That made it much easier to find and sit down with people and really dig in to what they are doing.

EuroPython 2008

There weren’t really any tracks per se, but I spent almost the entire time in a single room which was occupied either by a Jython talk or a PyPy talk. I somehow missed the PyPy talks at PyCon this year, so it was good to see the people and reconnect with what they are doing. We had a fun discussion about ways that the two projects could collaborate in the future. I really hope that we’ll be able to establish a good collaboration there.

EuroPython 2008

During one of the meals I got to spend some time with Raymond Hettinger, one of the Python core developers, who is also a professional photographer. It is always good to find people in the same tribe, even better to find multiple tribal overlaps. Thank you also to several of you who read this blog and stopped by to say hi. I completely appreciate the encouragement.

EuroPython 2008

The most riveting presentation of the conference was Hans Rosling’s presentation on the data visualization tools at GapMinder, which was really an excuse for him to present all sorts of interesting slicings of international health and economic development data. Despite being very sleepy and hungry, this talk had my undivided attention – I didn’t even really notice the passage of time. Professor Rosling gave a talk at TED earlier this year, and I expect that this is part of the content which we saw in the keynote.

EuroPython 2008

Python conferences seem to be taking off. In addition to EuroPython, there was PyCon Italia earlier this year, which was well attended, from what I was told. Also, in September, I will be speaking at PyCon UK in Birmingham, and I was able to meet John Pinner, one of the organizers, and nail down a bit more of that.

Thus far, I haven’t had any major travel hiccups. I made all my flights (2 from Seattle to Vilnius, and another 2 from Vilnius to Prague) and no bags were lost or anything else. I am writing this from a flexible office at Sun’s Prague development office, which is home to much of the NetBeans engineering team. I’ll save Prague for a posting when I make my way home.

EuroPython 2008

Python in NetBeans: NBPython

Tuesday, July 8th, 2008

One of the obvious things that Sun could do in the Python world is to make Python a supported language in the NetBeans IDE. Netbeans has really nice support for Ruby and Javascript, so why should Python be left out? So today Sun is announcing that a future version of NetBeans will provide support for Python. We are not announcing which release of NetBeans that will be because we are taking an unconventional (at least for NetBeans) path towards providing that support.

Before Frank Wierzbicki and I were even hired by Sun to work on Python and Jython, Allan Davis, a member of the NetBeans community, decided to start implementing support for Python in NetBeans in a project called NBPython. What we’ve decided to do is to work together with Allan and the rest of the NBPython community to produce a high quality Python plugin for NetBeans. The plugin is an early stage of development, but if you are brave, you can get Milestone 4, from the NBPython page and try it for yourself. If you decide to do that, you might find this blog post helpful with the installation – you will need a nightly build of NetBeans – NetBeans 6.1 will not do). Geertjan Wielenga from Sun did an e-mail interview with Allan Davis that has a few more technical details in it. There is plenty of work to do, so if you are interested in helping, drop into the development mailing list.

A month or two ago, the Sun Developer Network (SDN) started up a Ruby developer center. When that happened, I twittered to see whether there was any interest in a similar site for Python, and a number of folks expressed interest. So I am happy to announce that the Python developer center is now up and running. This is just a beginning for this site, and we are definitely looking for feedback and suggestions on this.

EuroPython 2008

Tuesday, July 1st, 2008

Next week I’ll be at EuroPython 2008, in Vilnius, Lithuania, where I’ll be moderating a panel on Jython. If you have questions that you’d like to put to the Jython committers, leave them as comments on this post. I’ll make sure that I post answers for any of those questions.

After the conference ends, I’ll be spending a few days at Sun’s office in Prague, which is where much of the NetBeans team is located. I’ve never been to Vilnius or Prague, so suggestions for things to see, etc., would also be great comments.

DTrace on Linux?

Monday, June 30th, 2008

I’ve been meaning to write a post about DTrace, and Tim Bray’s tweet finally got me moving. It looks like some people are trying to make DTrace a topic for this year’s Linux Kernel Summit. I hope they succeed. I also hope that those folks pushing for user level tracing have their voices heard. I was amused to read one of the messages which claimed that DTrace is:

DTrace is more a piece of sun marketing coolaid which they use to beat us up at every opportunity.

My experience at Sun thus far is that people generally don’t really appreciate the benefits of DTrace. It stems from a view that I also saw in the LKS threads, which is that DTrace (and tools like Systemtap) is a tool for system administrators, because it reports on activity on the kernel. That’s not how I look at it. DTrace is a tool for dealing with full system stack problems, which initially manifest themselves as operating system level problems. The fact that DTrace can trace user land code as well as kernel code is what makes it so important, especially to people building and running web applications. Because of all the moving parts in a complicated web application (think relational database, memcached or other caching layers, programming language runtime, etc), it can be hard to debug a web application that has gone awry in production. Worse, sometimes the problems only appear in production. Tools which cut across several layers of the system are very important, and DTrace provides this capability, if all the layers have probes installed. When a web application goes wrong in production, you see it at the operating system level – high usage of various system resources. That’s where you start looking, but you will probably end up somewhere else (unless you are ace at exercising kernel bugs). Perhaps a bad SQL query or perhaps a bad piece of code in part of the application. A tool that can help connect the dots between operating system level resource problems and application level code is a vital tool. That’s where the value is.

One of the cooler features of DTrace is that you can register a user level stack helper (a ustack helper), which can translate the stack in a provider specific manner. One cool example of this is the ustack helper that John Levon wrote for Python, which annotates the stack with source level information about the Python file(s) being traced. On an appropriately probed system, this would mean that you could trace the Python code of a Django application, memcached, and your relational database (PostgreSQL and soon MySQL). That would be very handy.

I’d love to see DTrace on Linux, because I have it on OS X and it’s in OpenSolaris and FreeBSD, but I’d also be happy to see SystemTap get to the point where it could do the same job.