Category Archives: python

dtrace -G on Mac OS 10.5 or dtrace for Python

Let’s suppose that you were trying to port dtrace probes from (Open)Solaris to Mac OS X, and the makefile for the probes on (Open)Solaris calls for the -G option, which isn’t recognized by dtrace in 10.5. You might want to check out this mail thread to find out what to do.   

The driver for this is an effort to port OpenSolaris’ dtrace probes for Python to run on OS X. One benefit of this will be that we’ll have access to John Levon’s ustack provider for Python on the Mac. If someone wants to tackle a port for FreeBSD 7.1, it would be great to get this support into all dtrace enabled platforms.

Python in NetBeans

Along with today’s launch of NetBeans 6.5, Sun, in cooperation with the NBPython community, are releasing an early access version of Python support for NetBeans. This is a result of the collaboration between Sun people and the NBPython project that I wrote about back in July. This release has been tested by folks in the NetBeans community and some folks from Sun’s NetBeans QA team, and it’s in pretty good shape for an early access release. We’re interested in getting people’s feedback. We would also love to see more people get involved with NBPython.

How to get it?

You can get NetBeans Python from the NetBeans download page.

What’s in it?

The basic feature set for the early access release consists of an editor for Python, the ability to execute Python programs (using CPython or Jython), and a debugger.

There’s a tutorial up on the NetBeans wiki.

Tor Norbye, who did most of the work on the editor, has written a series of blog posts detailing various features of the Python editor.

Who did it

Allan Davis – project and platform management, interactive console.

Jean-Yves Mengant – Jean-Yves is the author of the jpydbg debugger, which he’s merged into NBPython.

Amit Saha – documentation and help sets – Amit works for Sun, but he’s doing Python on his own time.

Tor Norbye (Sun) – editing.

Tomas Zezula (Sun) – project and platform management.

Ted Leung (me) (Sun) – various behind the scenes stuff.

Frank Wierzbicki (Sun) – NBPython is using Jython’s parser and Frank worked with Tor to add support for positions and better error reporting.

Peter Lam (Sun) – Sun QA

Tony Beckham (Sun) – Sun QA

The NetBeans CAT community as well as those folks who drove by and reported bugs.

How to get involved

NBPython has become a full fledged NetBeans project, so the main project page is now on NetBeans.org, as are the issue tracker and mailing lists:

nbpython-dev@netbeans.org
nbpython-issues@netbeans.org
nbpython-commits@netbeans.org
nbpython@netbeans.org

Book Review: Practical Django Projects

James Bennett, the release manager for Django has written a book called Practical Django Projects. At 237 pages, this book is even shorter than Learning Website Development with Django. Miraculously, Bennett manages to pack even more content into his book. He uses three different projects to illustrate the basic concepts of using Django. Early on in the book you are introduced to Django’s generic views mechanism, which makes it very easy to do the kinds of displays that are common in many web applications. Along with generic views, there’s a good treatment of how to keep functionality separated and reusable by good design of URLs. There’s also a much stronger treatment of the templating aspects of Django. He covers template inheritance, but also covers the creation of custom template tags.

Like “Learning Website Development with Django”, “Practical Django Projects” was completed before Django 1.0 shipped, so there are going to be some differences between what is described in the book and what you’ll encounter with Django 1.0. I like both books, and I’m probably biased by the order in which I read them. Either book would be fine for someone learning Django from scratch, but I think that “Practical Django Projects” is the book that I would turn to first when I couldn’t recall how to do something. In addition to the extra topics covered (and that’s not entirely fair because “Learning Website Development with Django” also has a few areas that it covers better), there’s more of an application building focus in Bennett’s book. That emphasis appeals to me, and Bennett has lots of application architecture hints sprinkled throughout the book.

Django is a great piece of work and deserves very serious consideration by people looking to build web applications. I’m glad to see that publishers are getting Django books out into the world.

Book Review: Learning Website Development with Django

The folks at Packt Publishing sent me a copy of Learning Website Development with Django by Ayman Hourieh. It’s a fairly thin book, weighing in at just 243 pages, but it it is a very good introduction to Django. Knowledge of Python is assumed, so if you need that information, you’ll need another book. Hourieh uses a running example, a social bookmarking application, throughout the book. This means that he is adding features to the application in each chapter, which is a style that I like because it shows how to use Django to build up a decently featured application piece by piece. There is lots of code, which I found easy to read, except for some places where the bold facing of new code was somehow omitted.

The only problem with the book was that it was finished before Django went 1.0, so there are undoubtedly some places where things are not quite in sync with Django 1.0. Hourieh did make an effort to deal with some of the largest 1.0 changes, like newforms, but omitted other like Unicode support. This is inevitable when the book and software are not in sync, but since Django only went 1.0 a few weeks ago, the only documentation that is completely up to date is the official Django documentation itself.

If you know Python and for some reason have been holding off on learning Django, this is a good book to help you get started. There is also an e-book version available via Packt’s website.

PyConUK 2008

I am slowly recovering from the transatlantic flights for PyCon UK, so I think that I’m ready to give a report.

The conference

PyCon UK 2008

In Europe, at least, the Python community seems to be sprouting more regional conferences as opposed to having the single EuroPython conference. This probably makes it easier for people to get to a conference, but it also means that people like me have to travel more in order to attend all these meetings. PyCon UK, also takes place over a weekend, again to facilitate people being able to attend without the hassle of getting time off from work and so forth. The conference was roughly the size of EuroPython, with attendance in the 230-250 range.

As usual, I did some of the regular program and some hallway track. The top three talks that I went to were Jacob Kaplan-Moss‘s update on the new stuff in Django 1.0 (that’s Jacob in the picture above). Chris Withers did a good talk on the problems with Python package management. The basic answer there is that a lot of stuff layered on top of a shaky foundation, in this case, distutils. Chris discussed some (partial, in my opinion) solutions to the problems, the virtualenv and buildout packages. Perhaps one day there will be a really good solution. Micheal Sparks from BBC research described the Kamaelia project, which includes a Software Transactional Memory (STM) module. I am neutral on the benefits of transactional memory, so I am glad to see that someone in the Python community is exploring how STM might fit with Python. Slides for some of the talks are being posted on the conference wiki. In the hallway track, I attended a BOF for Python Bloggers, and a PGP keysigning. I also had some excellent extended conversations with Holger Krekel and Maciej Fijalkowski of PyPy, Adewale Oshineye from Google, and Michael Foord from Resolver Systems.

I was invited to give one of the two keynotes for the conference, and my topic was “Challenges for Dynamic Languages”. I tried to step back from the specifics of Python and look at some of the challenges that face all of the dynamic language communities if we want to gain broader adoption of dynamic languages. I was somewhat concerned when Mark Shuttleworth covered some of the same topics during his keynote the day before, but maybe it’s a good sign when people from two different organizations and backgrounds can come to similar conclusions about the future of Python and related languages. My presentation is very picture oriented, without a lot of text or bullet points, so I am not sure whether posting the slides will actually help anyone or not. This was my first time giving a keynote, and it seems like it was well received, despite some laptop/projector interaction problems. Thanks to Jacob Kaplan-Moss for saving the day by lending me his MacBook Pro, remote, and memory stick.

Travel

PyCon UK 2008

This was my second transatlantic flight since June, and it seems that I really can’t sleep on long airplane flights. This turns the 10 hour flight from Seattle to most Star Alliance hubs into a pretty long ordeal. I did manage to finish a book and read a few papers, but I spent way too much time using the in flight entertainment system (I prefer Lufthansa’s to SAS’s). I also experienced my longest wait (1 hour) in passport control in the Birmingham airport. During that wait, I listened to the two American women right in front of me discovering that each of them was moving to the UK to marry an Englishman. Ten people or so before I reached the inspectors, a passenger collapsed in an epileptic fit (no previous history apparently), just to make sure things were memorable.

I was only in Birmingham for two and a half days, and spent almost two days inside of airplanes. Several people commented on the jet lag, which didn’t seem that bad until it caught up with me when I finally arrived home.

Photography

Photographically, perhaps the bigger story is the pictures that I didn’t get. The D3 does not fit in my camera/computer bag with a lens mounted on it, which means that it is hard to get shots where you just need to pull the camera out, point and shoot. I had a flight leg from Seattle to Frankfurt, and on the Frankfurt end we took busses from the airplane to the terminal. While we were waiting for all the passengers to board the bus, I realized that I was staring at a great view of the rear left quarter of an Airbus A330-300. Perfect for a wide angle shot, but there wasn’t enough time to get the shot.

When I arrived on Thursday, I went over to the Pycon UK social, had some dinner and hung out. I left a little early when the noise level got to be more than I could handle. As I was walking back to the hotel, I saw some beautiful artwork that was being projected on the side of a building in Paradise Circle as part of Birmingham’s Light Night and Artsfest. I had left the camera at the hotel because I didn’t want to have to haul it around, but I was so taken with the display that I went and picked it up. Here’s some of what I saw:

PyCon UK 2008

PyCon UK 2008

PyCon UK 2008

This is a shot as I walked back to my hotel via one of Birmingham’s canals.

PyCon UK 2008

Here‘s the rest of the set. There are not many pictures of the conference proper — I go to a lot of conferences now, and I’ve shot enough pictures of people sitting in rows or giving a talk. There’s an entire group on Flickr for PyCon UK if you want better coverage of the conference.

OSCON 2008

Another OSCON has come and gone, and as usual, I am exhausted in the aftermath. I’ve developed a love-hate relationship with OSCON over the years. The diversity of the OSCON community is one of the huge pluses of the conference. I got involved in open source via Apache, and OSCON was where I really started to get more of a sense of the open source community as a whole. That’s led to friendships with people doing all sorts of open source stuff, which makes the conference a natural place to reconnect with many of those folks. Which leads to the primary downside of OSCON, which is that there is just no way to keep up with, never mind see all the people that you’d like to see. Combine that with the sheer scale of the event, and you have recipe for burnout. This year is no exception, which is why this post is delayed by a few days.

Languages

It’s fitting to start a review of OSCON with programming languages, since OSCON began as a Perl conference. There are still lots of Perl hackers running around, and by the distribution of the program (the Python track was 1 day shorter than the Perl, PHP, and Ruby tracks), it seems that Perl is not going anywhere anytime soon. I think that we are going to need to drum up some more Python talks for OSCON next year. Then again, with PyCon topping 1000 people this year, maybe all the Python folks are going there. It certainly is cheaper than going to OSCON. Despite all of this, I saw lots of people that I knew from the Python community, as well as plenty of people who had affixed a yellow Python ribbon to their badge. The ribbons are a nice way of helping people find their tribe at a big show like OSCON – a lower tech version of what the Pathable folks are doing.

I spent a lot of time nosing around various concurrency oriented sessions. I attended Steven Parkes’ tutorial on Actors, which was pretty well attended. Steven has implemented a version of Actors as a set of Ruby and Python libraries. During the tutorial I was able to meet Debasish Ghosh, who has a great blog and Twitterstream on high-level languages, and concurrency topics in general. I also took in a BOF on Actors, which had some really interesting conversation. There were a lot of Erlang folks in the room for that one, which made the discussion pretty interesting.

Databases

OSCON 2008

There was lots of non-traditional database stuff happening at OSCON this year. I am one of the mentors for the CouchDB project at Apache, and I was finally able to meet my first CouchDB commiter, Jan Lehnardt, at the show. Jan gave a nice high level overview talk on CouchDB, which was well attended, and I was interested to see Brian Aker of MySQL/Drizzle in the audience and among the throng of questioners after the talk.

OSCON 2008

I also went to a talk on Prophet, which is a peer to peer database that is being done by some of the folks that brought us SVK. I’m not sure that I quite recovered from my initial reaction to that revelation, but Jan was sitting next to me during the entire talk, and was saying something about stealing some ideas from the Prophet guys. In open source we call that standing on the shoulders of giants, or something like that.

“Memes”

The XMPP folks had a three day summit during the conference, which I gather was well attended. There was a decent amount of XMPP buzz floating around in the hallways, so I expect the blogosphere to be full of XMPPness during the next week or so. I’ve done a bunch of blogging on XMPP in the past, and while things have improved, they haven’t improved to the point where XMPP is taking over the world. Things like Twitter are definitely helping, but there is still a long way to go before XMPP achieves world domination. But we can hope. And at least XMPP makes a great advertisement for Erlang.

Along with XMPP, we had the microblogging meme. I made heavy use of Twitter throughout the week, and it definitely played a useful part in making connections with people. Well, except for the times when it was down. I was able to spend a little time with Leah Culver, the founder of Pownce, which has the virtue of being written in Python, and of having a very nice API for dealing with the service. It’s interesting to get additional perspectives on a problem, and since I had already talked some with the Twitter guys, it definitely helped to hear Leah describe Powce’s take on the problem(s) and solutions. O’Reilly was not to be outdone, and did some very active boostering for identi.ca. I’ve got very mixed feelings on identi.ca. One the one hand, I should love identi.ca, because it’s open source. On the other hand, it’s written in PHP, which means I won’t be touching the code, and more importantly, my network is not there. Actually, it was kind of annoying to have to explain to lots of identi.ca zealots that it’s the network that’s the value, not the software, or ironically, the quality of the service. Still if another microblogging service can convince my network to move, and remain up, and even deliver some new functionality, I would definitely switch. I think I could probably write another post about “microblogging”, but I’ll refrain for now.

Theo Schlossnagle gave an amazing presentation called “Full-stack introspection crash course”, which is code for “let me show you some amazing stuff that you’ll only be able to do with DTrace”. This was a brilliant choice of title on Theo’s part, because it didn’t scare away all of those people whose preconceptions about DTrace or Sun would prevent them from coming to such a talk. Instead, Theo played to a very full room, and I would say that about one-third of the audience actually uttered the phrase “Oh My God” out loud at some point during the presentation. This was certainly true for thetwo gentlemen sitting directly to my right and directly behind me. I later heard from people at the Sun booth, that a bunch of people came to the booth having heard about DTrace (I assume at Theo’s talk), asking for whatever CD’s they needed in order to be able to use it. Theo clearly understands how to communicate about DTrace. We at Sun need to learn that lesson.

Open Source

Of course, you can’t have a conference on open source without meta stuff about open source itself. I was fortunate to attend the morning session of Microsoft’s Participate08 event, which was an interesting case study led by Karim Lakhani from the Harvard Business School. The case was on threadless.com and involved a lot of issues which are very relevant to injecting corporate involvement into an existing community based organization. I’ve been following Karim’s work over the years (he studied under Eric von Hippel, whose work I am also fond of), so I was happy for the chance to meet him and participate in an activity with him. I also met Siobhan O’Mahony, who is also doing great work studying open source communities. I’m not sure what direct value Microsoft got out of sponsoring Participate, other than being able to say that they did an event around OSCON, but I know that I definitely appreciated the chance to interact with a bunch of people.

OSCON 2008

Microsoft was all over the news by the end of OSCON, having announced that they would become a Platinum sponsor of the Apache Software Foundation. This was not a complete surprise to me: Justin Erenkrantz, the current ASF president told me what was happening the night before at a party. I think that this is an interesting step for Microsoft, an it’s definitely a step in the right direction. However, as one questioner pointed out, Microsoft has a long history of incendiary rhetoric towards the open source community, and that’s going to mean that just about everything happens in steps. I do find it interesting that one of the reasons that the ASF has taken donations is to build up a legal defense fund against what we regarded as inevitable legal attacks. It’s somehow ironic to think of Microsoft’s $100,000 going into that pool. I think that the next interesting milestone in Microsoft’s relationship with the ASF will be when the first Microsoft sponsored project shows up at the front door of the Apache Incubator.

I also contributed to the metaness with a talk titled “Open Source Community Antipatterns” (slides are now available on the O’Reilly slide page). The talk was decently attended, but I suspect that the all-star antipatterns panel immediately following my talk drew off some of the audience that might have come to my talk. The people track expanded a great deal this year, which I think is a good thing.

Photography

I always have photographic memories associated with OSCON. I got my first digital SLR, right before OSCON 2005, and I’ve shot a bit a each OSCON, and even won the OSCON photo contest one year. This year I found myself shooting less. There were too many other things that I needed to do, and between knowing that Duncan is making is covering stuff and some artistic blockage, I lacked both time and motivation to crank out the shots.

Duncan has been a great friend and photographic mentor, and I always look forward to catching up with him during OSCON. This time was no exception. We did a bunch of stuff together, ranging from hanging out, having a wide angle shootout (well he was wide) to Duncan putting one of his cards into my D3 and giving the pixels a once over. Probably the most fun thing that we did was an impromptu photoshoot. Duncan was shooting headshots of the OSCON staff for a thank you slide for the closing keynotes. Only problem was that he needed one of himself, so he drafted me. With the safe shot in hand, we spent a few more minutes doing something a little edgier and fun.

OSCON 2008


Fin

That’s it for another OSCON. I hope we’ll be back in Portland again next year.

Jython 2.5 Alpha 1

The Jython development team has released the first alpha of Jython 2.5. The guys were hoping to have this done in time for EuroPython, but it wasn’t to be. Still, they are ahead of where they planned to be, so this is good news. If you are interested in an up to date version of Python on the JVM, go kick the tires and report bugs. Getting more testing done should help speed the process of shaking out all the bugs.

My first EuroPython

I’ve been doing Python stuff since 2003, and my first PyCon was back in 2004, but this was the first time that I’ve been able to attend EuroPython. The conference reminded me very much of the first PyCon that I went to back in 2004. It was around 250 people, and it had a much smaller and more intimate feeling to it. That made it much easier to find and sit down with people and really dig in to what they are doing.

EuroPython 2008

There weren’t really any tracks per se, but I spent almost the entire time in a single room which was occupied either by a Jython talk or a PyPy talk. I somehow missed the PyPy talks at PyCon this year, so it was good to see the people and reconnect with what they are doing. We had a fun discussion about ways that the two projects could collaborate in the future. I really hope that we’ll be able to establish a good collaboration there.

EuroPython 2008

During one of the meals I got to spend some time with Raymond Hettinger, one of the Python core developers, who is also a professional photographer. It is always good to find people in the same tribe, even better to find multiple tribal overlaps. Thank you also to several of you who read this blog and stopped by to say hi. I completely appreciate the encouragement.

EuroPython 2008

The most riveting presentation of the conference was Hans Rosling’s presentation on the data visualization tools at GapMinder, which was really an excuse for him to present all sorts of interesting slicings of international health and economic development data. Despite being very sleepy and hungry, this talk had my undivided attention – I didn’t even really notice the passage of time. Professor Rosling gave a talk at TED earlier this year, and I expect that this is part of the content which we saw in the keynote.

EuroPython 2008

Python conferences seem to be taking off. In addition to EuroPython, there was PyCon Italia earlier this year, which was well attended, from what I was told. Also, in September, I will be speaking at PyCon UK in Birmingham, and I was able to meet John Pinner, one of the organizers, and nail down a bit more of that.

Thus far, I haven’t had any major travel hiccups. I made all my flights (2 from Seattle to Vilnius, and another 2 from Vilnius to Prague) and no bags were lost or anything else. I am writing this from a flexible office at Sun’s Prague development office, which is home to much of the NetBeans engineering team. I’ll save Prague for a posting when I make my way home.

EuroPython 2008

Python in NetBeans: NBPython

One of the obvious things that Sun could do in the Python world is to make Python a supported language in the NetBeans IDE. Netbeans has really nice support for Ruby and Javascript, so why should Python be left out? So today Sun is announcing that a future version of NetBeans will provide support for Python. We are not announcing which release of NetBeans that will be because we are taking an unconventional (at least for NetBeans) path towards providing that support.

Before Frank Wierzbicki and I were even hired by Sun to work on Python and Jython, Allan Davis, a member of the NetBeans community, decided to start implementing support for Python in NetBeans in a project called NBPython. What we’ve decided to do is to work together with Allan and the rest of the NBPython community to produce a high quality Python plugin for NetBeans. The plugin is an early stage of development, but if you are brave, you can get Milestone 4, from the NBPython page and try it for yourself. If you decide to do that, you might find this blog post helpful with the installation – you will need a nightly build of NetBeans – NetBeans 6.1 will not do). Geertjan Wielenga from Sun did an e-mail interview with Allan Davis that has a few more technical details in it. There is plenty of work to do, so if you are interested in helping, drop into the development mailing list.

A month or two ago, the Sun Developer Network (SDN) started up a Ruby developer center. When that happened, I twittered to see whether there was any interest in a similar site for Python, and a number of folks expressed interest. So I am happy to announce that the Python developer center is now up and running. This is just a beginning for this site, and we are definitely looking for feedback and suggestions on this.