iPhone 4
I love the phone. I do see the much written about antenna attenuation problem, but day to day it doesn’t affect me as much as AT&T’s network does. One of the prime times for me to use my phone is while standing in line waiting for the ferry. The worst time is during the afternoon, because there are several hundred people all packed into the ferry terminal, all trying to pull data on their iPhones. The antenna has nothing to do with this.

In every other way, the phone is fantastic. My iPhone 3G would frequently hit the red line on the battery indicator by the time I hit the afternoon ferry, and that was after I had carefully managed my use of the device during the day. With the iPhone 4, I don’t have to worry about managing the battery. That alone has made the upgrade worth it for me.

The upgraded camera has been a huge success for me. I attribute this to a single factor – startup time. I was always reluctant to pull out my iPhone 3G for use as a camera, because quite frequently I would miss the moment by the time the camera came up. I’ve been using Tap Tap’s excellent Camera+ and I like it quite a bit. Unfortunately, you can’t get it on the app store right now, because the developer inserted an easter egg that would allow you to use one of the volume buttons to trigger the shutter. Apple then pulled the app from the store. This is the first time that App Store policy has affected an app that I care about, and I’m obviously not happy about it. It seems to me that Camera+ could have a preference that controlled this feature, and that users would have to turn it on. Since the user would have turned on that feature, they would’t be confused about the takeover of the volume button. It seems simple to me. I really like Camera+’s light table feature, but I really hate the way that it starts up trying to imitate the look of a DSLR rangefinder. The other area where Camera+ could use improvement is in the processing / filters area. It has lots of options, but most of them don’t work for me. I have better luck with Chase Jarvis’ Best Camera on this front. In any case, I’m very happy with the camera as ” the camera that is always always with me”. The resolution is also very good, and I’ve been using it to photograph whiteboards into Evernote quite successfully.

iPad

I’ve been carrying my iPad on a daily basis. I’m using it enough that when I forgot it one day, it made a difference. One thing that I’ve learned is that the iPad really needs a case. I got much more relaxed about carrying mine once it was inside a case. Originally, I thought that I would wait for one of the third party cases, but all of the ones that looked like a fit for me were out of stock, so I broke down and ordered the Apple case. It does the job, but I am not crazy about the material, and I wish that it had one or two small pockets for a pen, a little bit of paper, and perhaps some business cards.

I am pretty much using the iPad as my “away from my desk device” when I am in the office. Our office spans 5 floors in a skyscraper, and I have meetings on several floors during the course of a day. The iPad’s form factor and long battery life, make it well suited as a meeting device. I have access to my e-mail and calendar, and I’m using the iPad version of OmniFocus to keep my tasks and projects in sync with my laptop. I’ve written some py-appscript code that looks at the day’s calendar in Entourage and then kicks out a series of preformatted Evernote notes so that I can pull those notes on my iPad and have notes for the various events of the day. This kind of Mac GUI to UNIX to Mac GUI scripting is something that I’ve commented on before. Thanks to multi-device application families like Evernote, I expect to be doing some more of this hacking to extend my workflow onto the iOS devices. I don’t have a huge need for sharing files between the iPad and the laptop, but Dropbox has done a great job of filling in the gap when I’ve needed to share files.

Several people have asked me about OmniFocus on the iPad, and whether or not it is worth it. I have a large number of both work and personal projects, so being able to use the extra screen real estate on the iPad definitely does help. I have come to rely on several features in OmniFocus for iPad which are not in the desktop version. There is a great UI for bumping the dates for actions by 1 day or 1 week, which I use a lot. I am also very fond of the forecast view, which lets you look at the actions for a give day, with a very quick glance at the number of actions for each day of a week. Both of these features are smart adaptations to the iPad touch interface, and are examples of iPad apps coming into a class of their own.
Another application that I’ve been enjoying is Flipboard. Flipboard got a bunch of hype when it launched back in July, and things have died down because they couldn’t keep up with the demand. Conceptually, Flipboard is very appealing, but the actual implementation still has some problems as far as I am concerned. I can use Flipboard to read my Facebook feed, because Facebook’s timeline is just highly variable in terms of including stuff from my friends. I don’t feel that I can read Twitter via Flipboard, because it can’t keep up with the volume, so I end up missing stuff, and I hate that. Some of the provided curated content is reasonable, but not quite up to what I’d like. Flipboard is falling down because there’s not a good way for me to get the content that I want. I want Flipboard to be my daily newspaper or magazine app. But I can’t get the right content feed(s) to put into it.   

As far as the iOS goes, my usage of the iPad is making me horribly impatient for iOS 4. I would use task switching all the time. Of course, then I would be unhappy because the iPad doesn’t have enough RAM to keep my working set of applications resident. Text editing on iOS is very painful on the iPad. I’m not sure what a good solution would be here, but it definitely is a problem that I am running into on a daily basis – perhaps I need to work on my typing. There is also the issue of better syncing/sharing. My phone and iPad are personal devices, so they sync to my iTunes at home. I use both devices at work, where I have a different computer. This is definitely an area that Apple needs to improve significantly. At the moment, though, the fact that I am using my iPad hard enough to really be running into the problem means that the iPad has succeeded in legitimizing the tablet category – at least for me.

Talks

As always, there are a huge number of talks at OSCON, and this year I found it particularly hard to choose between sessions, though in several cases, hallway track conversations ended up making those choices for me. There wasn’t a theme to my talk attending this year, because a lot of topics are relevant to the work that I am doing now.

I attended a talk about Face Recognition on the iPhone. Sadly this turned out to be very focused on setting up for and calling the OpenCV library and less about face recognition, UI, or integrating face recognition into applications. As I’ve written previously, I think that new interface modalities may be arriving along with new devices, so I was hoping for a bit more than I got.

Big Data is also a topic of interest for me, so I went to Hadoop, Pig, and Twitter, and Mahout: Mammoth Scale Machine Learning. Hadoop, Pig, and Mahout are all projects at Apache, and each of them have an important part to play in the emerging Big Data story. The sort of analytics that people will be using these technologies for are part of the reason that data is now the big area of concern when discussing lock in.

The open source guy in me likes the idea of WebM, but it looks to me like there’s quite a way to go before it will be replacing H.264. I was surprised that the speaker didn’t have a better answer than “our lawyers must have checked this when we acquired On2″. More than anything else, getting clarity on the patent provenance for VP8 is what would make me feel good about WebM.

Robert Lefkowitz (the r0ml), is always an entertaining and thought provoking speaker. His OSCON presentations are not to be missed. This year he gave two talks, and you can read some of my commentary in my twitter stream. Unfortunately, r0ml picked licensing as the topic of his second presentation, and his talk was interrupted by an ill-tempered and miffed free software enthusiast, thus proving r0ml’s earlier solution that open source conferences are really legal conferences.

I’ve been following / predicting the server side javascript space for several years now. One of the issues with that space is the whole event based programming model, which caused mortal Python programmers headaches when dealing with the Twisted Python framework. Erik Meijer’s group at Microsoft has been grabbing techniques from functional program to try to make the programming model a bit more sane. I had heard most of the content in his Reactive Extensions For JavaScript talk before, and I’m generally enthusiastic about the technology. The biggest problem that I have is that RxJS is not licensed under an open source license. At JSConf I was told that this is being worked on, so I dropped in for the second half of Erik’s talk hoping to hear an announcement about the licensing. It was OSCON after all, and the perfect place to make such an announcement, but no announcement was made. I hope that Microsoft won’t wait until next year’s OSCON to get this done.

This year there were two of the keynote presentations that made enough of an impression to write about. The first was Rob Pike’s keynote on Go, where he eloquently noted some of the problems with mainstream programming languages. There was no new information here, but I liked the approach that he took in his analysis. The second was Simon Wardley’s Situation Normal, Everything Must Change . Simon is an excellent presenter and full of insight. While his talk was ostensibly about cloud computing, I think it was a little deeper than that. His story about the cloud is the story about commoditization of technologies, and since one of the roles of open source is commoditization of technology, I felt that there was some nice insight there for those of us working in open source. Simon also discussed the mismatch between innovation and mature organizations, another issue that people in open source often run into. The video is already up on YouTube, so you can make your own assessment of the talk.

The size and breadth of OSCON gives it one of the richest hallway tracks of any conference. This year was no exception – the hallway track started on the train from Seattle to Portland, and extended all the way through the return train trip as well. I always look forward to these discussions and to connecting with old friends. One friend that I caught up with was Cliff Schmidt, now executive director of Literacy Bridge, which is working to make knowledge accessible to people in poor rural communities throughout the world via their Talking Book device. Cliff had one with him, and this was the first time I had seen one. Just in case you haven’t, here’s what they look like:

Languages

OSCON began as a language conference, and this year, there were two special events in that space, the Scala Summit and the Emerging Languages Camp.

I have followed the Scala community pretty closely, because they are quickly accumulating real-world experience with functional programming. There are lots of cool tricks that remind me of stuff that I played with when I was in graduate school, and there are lots of bright people. But there are some things that I find worrying. For example, one of the speakers was touting the fact that Scala’s type system is now Turing complete. If I’m using Scala, one reason is that I want my programs to type check at compile time. Having the type checker go off and fail to halt is not what I had in mind. I recognize that you’d have to write some gnarly type declarations for this to happen, but still.

This was the first year for Emerging Languages Camp, and from what I can tell it was a roaring success. I didn’t attend as many sessions as I would have liked. This was due to a combination of factors – other talks that I wanted to see, being the biggest. The other factor was that the first talk I attended was Rob Pike’s talk on Go, and the room was very full, which made it hard for me to concentrate (probably had more to do with me than the room). When I saw that all the talks were being recorded and that the video folks promised to have them up in 2-4 weeks, it made it seem less urgent to try to pop in and out and fight the crowd. Still, this is a sign of success, and I hope that the minimum, the Emerging Language Camp will be given a larger room next year. Part of me would like to see it be a completely separate event from OSCON, but that’s probably not realistic.

Of the talks that I was able to attend, I found the Caja and BitC talks to be the most relevant. Fixing Javascript to have security is important for both client and the burgeoning server applications of Javascript. I wish that I had seen the talk on Stratified Javascript, since concurrency is ever the hot topic these days. As far as BitC goes, we are well beyond the time when we should have had a safe systems programming language. As much as C has contributed to the world, we really need to move on.

What is OSCON for?

I had a few discussions along the lines of “What is OSCON for?”, and Tim Bray shared some thoughts in his OSCON wrap up. As I have written before, I think that open source has “won”, in the sense that we no longer need to prove that open source software is useful, or that the open source development process is viable. There are still questions about open source business models, but that’s a topic that I’m not as interested in these days. Open source having “won” doesn’t mean that our ideas have permeated the entire world of computing yet, so there is still a need for a venue to discuss these kinds of topics. OSCON is more than that, though. It’s also a place where hackers (in the good sense) have a chance to showcase their work, and to exchange ideas. In that sense, part of OSCON is like a computing focused eTech. Apparently O’Reilly is no longer running eTech, which is fine – the one time that I attended, I was underwhelmed. I think that perhaps what is happening in the Emerging Languages Camp might be an example of how things might move in the future.

Of course, there’s a larger question, which is why do we have conferences at all anymore? Many conferences now produce video content of the sessions. I don’t really think there’s a lot of value in having an event to do product launches or announcements. The big thing is the hallway track, which allows for realtime interchange of ideas and opinions, and in the case of open source, provides a dosage of high bandwidth, high touch interaction that helps keep the communities running smoothly. We’re in the 21st century now. Is there something better that we can do?

iPhone

I am using a 3G iPhone now, and I’m not having the best user experience at the moment. There are lots of lags and stutters at inopportune moments, both in the user interface and in the performance of AT&T’s 3G network. I’ve grown used to the briskness of the iPad, and I expect that on my phone now. Apple has set their own bar here. So getting me to iPad level responsiveness is job one. Job two is to get me decent battery life. My iPad lasts way longer than my iPhone. I understand why, but I don’t like it. I really, really want to be able to use my phone without redlining it every day. The last of the big items has to do with AT&T or a rumored second carrier. I want to be able to rely on the phone for accessing data. Right now it’s not as reliable as it needs to be, and I think that everyone knows that. It’s not at all clear to me that a second carrier will do any better, because I doubt that they are prepared for the level of traffic that is coming their way once they get the iPhone. Sprint and Verizon crumbled at Google IO, so let’s not kid ourselves that the other carriers are going to magically fix things. But maybe if a bunch of people jump ship to another carrier, things will get better on AT&T.

There are some secondary issues:    16GB has turned out to be less space than I anticipated, but since the 3GS already comes in a 32GB size, I expect the next generation to come in at 64GB, although I won’t be disappointed if it doesn’t. I expect there to be camera upgrades, and I am pretty sure that I’ll be happy with what happens there. The real trick in cameras is the lenses not the megapixels, and all camera phones are on the same footing there.

This time around, there’s a “but”.

Android

After the Android 2.2 (Froyo) announcements, I am considering an Android phone as my next phone. There’s no question that today, the iPhone user interface is more highly developed, polished and intuitive than Android. At the moment fragmentation of the Android platform is a reality, despite Google’s assurances that this will get cleaned up in the future. There are numerous good apps in the Android Marketplace, but some of the applications that I use the most are not there, because they are the iPhone counterparts of Mac desktop applications. That’s a fairly large problem. These are all good reasons to stick with the iPhone.

There are two big reasons that I am looking more closely at Android. The first is that Android has much much better “integration with the cloud”. One of the biggest annoyances that I have with my iPad is the hassle of moving PDF ebook files from my Macintosh to the iPad. I shouldn’t have to use a cord, and I shouldn’t have to use iTunes. 1Password can implement wireless syncing to the iPad and iPhone, why won’t Apple? The second and more important reason is that I like what I see in some of the directions that Google is taking the user interface. Specifically, I’m talking about use use of voice and (possibly) the use of computer vision as demonstrated in Google Goggles. The iPhone, and more recently, the iPad have done something very interesting with multitouch/gestural interfaces. If you subscribe to the theory that science fiction influence science fact, then we could look at Iron Man 2 for some examples of future interfaces. Tony Stark interacts with his computer via a combination of gestures and voice commands, and from the content of the voice commands, it is clear that the computer is employing something like vision in order to resolve references in Stark’s words. As great as Apple’s advances in multitouch have been, they have done very little in terms of voice. Perhaps their acquisition of Siri is a step in that direction, but Apple’s famed secrecy makes it hard to know. The same is true in vision, except that Apple has made no such acquisition. There’s quite some distance to go before Android’s speech and vision could bring about a multimodal interface like the one in Iron Man, but at least I can see the signs that Google is going that direction. Of course, I could just wait it out on a few more generations of iPhone while Google engineers work all these issues out, but I see signs of Google acting like a leader instead of a catch up player, and I like that.

What about Apple’s recent behavior with regard to languages other than Objective-C? Yes, I am bothered by it, but it’s not as big an issue to me as working well in an internet centric world, or working towards a much more multimodal user interface. Nobody is leaving the web platform because they are unable to write in-browser applications in their favorite language, and lots of people are delivering all kinds of interesting stuff in that space. More choice would definitely be nice, but if choice or freedom are your high order bit, that’s what Android is for.

If nothing else, I think it’s a good sign that there are two mobile platforms good enough to put me in this conundrum.

Last week there were some articles, blog posts and tweets about the relationship between Platform as a Service (PaaS) offerings and open source. The initial framing of the conversation was around PaaS and the LAMP (Linux/Apache/MySQL/{PHP/Perl/Python/Ruby}) stack. An article on InfoQ gives the jumping off points to posts by Geva Perry and James Uruqhardt. There’s a lot of discussion which I’m not going to recapitulate, but Uruqhardt’s post ends with the question

I’d love to hear your thoughts on the subject. Has cloud computing reduced the relevance of the LAMP stack, and is this indicative of what cloud computing will do to open-source platform projects in general?

Many PaaS offerings are based on open source software. Heroku is based on Ruby and is now doing a beta of Node.js. Google’s App Engine was originally based on Python, and later on Java (the open sourceness of Java can be debated). Joyent’s Smart Platform is based on Javascript and is open source. Of the major PaaS offerings, only Force.net and Azure are based on proprietary software. I don’t have hard statistics on market share or number of applications, but from where I sit, open source software still looks pretty relevant.

Also I think it’s instructive to look at how cloud computing providers are investing in open source software. Rackspace is a big sponsor of the Drizzle project, and of Cassandra, both directly and indirectly through its investment in Riptano. EngineYard hired key JRuby committers away from Sun. Joyent has hired the lead developer of node.js, and VMWare bought SpringSource and incorporated it into VMForce. That doesn’t sound to me like open source software is less relevant.

Cloud computing is destined to become a commodity

The end game for cloud computing is to attain commodity status. I expect to see markets in the spirit of CloudExchange, but instead of trading in EC2 spot instances, you will trade in the ability to run an application with specific resource requirements. In order for this to happen, there needs to be interoperability. In the limit, that is going to make it hard for PaaS vendors to build substantial platform lockin, because businesses will want the ability to bid out their application execution needs. Besides, as Tim O’Reilly has been pointing out for years, there’s a much more substantial lock in to be had by holding a business’s data as opposed to a platform lock. This is all business model stuff, and the vendors need to work this out prior to large scale adoption of PaaS.

Next Generation Infrastructure Software

The more interesting question for developers has to do with infrastructure software. In my mind LAMP is really a proxy for “infrastructure software” If you’ve been paying any attention at all to the development of web application software, you know that there is a lot happening with various kinds of infrastructure software. Kiril Shenynkman, one of the commenters on Geva Perry’s post wrote:

Yes, yes, yes. PHP is huge. Yes, yes, yes. MySQL has millions of users. But, the “MP” part of LAMP came into being when we were hosting, not cloud computing. There are alternative application service platforms to PHP and alternatives to MySQL (and SQL in general) that are exciting, vibrant, and seem to have the new developer community’s ear. Whether it’s Ruby, Groovy, Scala, or Python as a development language or Mongo, Couch, Cassandra as a persistence layer, there are alternatives. MySQL’s ownership by Oracle is a minus, not a plus. I feel times are changing and companies looking to put their applications in the cloud have MANY attractive alternatives, both as stacks or as turnkey services s.a. Azure and App Engine.

How many of the technologies that Shenykman lists are open source? All of them.   

Look at Twitter and Facebook, companies whose application architecture is very different from traditional web applications. They’ve developed a variety of new pieces of infrastructure. Interestingly enough, many of these technology pieces are now open source (Twitter, Facebook). Open source is being used in two ways in these situations. It is being used as a distribution mechanism, to propagate these new infrastructure pieces throughout the industry. But more importantly (and for those observing more closely, quite imperfectly), open source is being used as a development methodology. The use of open source as a development methodology (also known as commons-based peer production) is definitely contributing to these innovative technologies. Open source projects are driving innovation (this also happened in the Java space. Witness the disasters of EJB 1.0 and 2.0 which lead to the development of EJB 3.0 using open source technologies like Hibernate, and which provided the impetus for the development of Spring). Infrastructure software is a commons, and should be developed as a commons. The cloud platform vendors can (and are) harvesting these innovations into their platforms, and then finding other axes on which to compete. I want this to continue. As I mentioned in my DjangoCon keynote last year, I also want open source projects to spend more time thinking about how to be relevant in a cloud world.   

My question on PaaS is this: Who will build a PaaS that consolidates innovations from the open source community, and will remain flexible enough to continue to integrate those innovations as they continue to happen?

iPad

Conference like settings are one of the situations where I felt that I could make the best of the iPad. Apparently, I was not alone, because there were probably somewhere between 5 and 10 iPads at the event.

My flights from Seattle to JSConf included 6 hours of flying time, and hour and a half of layovers, plus the usual waiting around time in airports. During that time I read some e-mail, watched about 90 minutes of video, and read several PDF books / documents. By the time I finally ended up in my hotel room, I still had around 80% of the battery charge remaining. I used the iPad as much as possible during the first day of JSConf, and the battery finished at 49% at the end of the first day. Thus far, the battery life is beyond my expectations.

During the conference, the primary activities that I was doing were e-mail reading, web browsing, twittering, and taking notes. For the first two activities, I used the built in Mail and Safari. For Twitter, I switched back and forth between Twitterific and TweetDeck. I used Evernote as my primary note taking tool.

I started out using Twitterific, but at some point it stopped working and was giving a message about an nvalid server certificate error. Echofon on the Mac was having a similar problem. I had TweetDeck installed on the iPad as a leftover from trying it on the iPhone, so I gave it a try and it worked. On the desktop I am not a fan of Tweetdeck’s AIR based user interface, which outweighs it’s advantage of having columns. When I use Syrinx on the desktop, I just open a stack of windows and that works fine. But on the iPad, Tweetdeck’s column based model makes a lot of sense, especially if you hold the iPad in landscape mode. I was mostly happy with the experience, although Tweetdeck has some weird UI in places:

• It’s hard to get a sense of when the various columns refresh, and there doesn’t appear to be a way to get individual columns to refresh. I’d love to be able to use Tweetie 2’s pull down to refresh gesture to do this.
• Favoriting tweets (which is how I keep track of interesting information on a mobile device) takes over the whole screen for a moment, causing an annoying flash/blink effect.
• In Landscape mode you can’t click links or view profiles (the latest update to TweetDeck has added support for link clicking)
• If you select a tweet and then discover that you need the additional menus popup, then you need to select another tweet and then reselect the tweet you want to act on

I love Evernote, and I’ve written about that before. The iPad version of Evernote is fantastic, with perhaps one exception. If you try to edit a rich text note, you are put into a weird append only kind of mode. I have some Python scripts that create rich text notes from items on my calendar, so it’s annoying to go back to Evernote on the iPad and then be put into append mode. I would love to see a full rich text editing capability come to a future version of Evernote for iPad (and sure, iPhone). Other than that, it was a workhorse at JSConf.

At many conferences, there are multiple WiFi networks, and you have to switch among them as you go from room to room. This was the case at JSConf. On the iPad, this meant a trip to the Settings app in order to select a new network. It would be great if the iPad would switch among multiple known networks based on signal strength. I can think of some reasons why you might not want to do this, but in my situation, it would have been really convenient.

All in all I had a pretty good experience with the iPad as my primary device. I can definitely see it as my primary conference machine, as well as my “in a meeting” machine. iPhone OS 4.0’s “multitasking” will reduce the annoyance associated with waiting for apps to restart on switching.

MacBook Pro

At work they issued me a unibody MacBook Pro 15″. These are supposed to have much better battery life than their pre-unibody forbears. As far as I can see this is true. I imagine that the recently refreshed models are even better on this count. The only other thing that I noticed was that the power adapter gets pretty hot while recharging the machine.   

GF1

Like many photographers, I’ve been looking for a small, high quality, camera that I could carry with me almost all the time. I have my cell phone at all times, and in a pinch, a cell phone picture is better than nothing. But a cell phone camera, regardless of megapixels lacks the controls that I’ve grown used to when making pictures. I’ve started carrying a Panasonic GF1 with the 20mm lens. The wide aperture prime suits the style that I like to shoot in, and the Micro 4/3 sensor gives pretty decent looking pictures. The GF1 produces 12 megapixel RAW files, which in principle is the same as my D3. Of course, there’s a vast difference in quality of those pixels, but thus far I am pretty happy. It has all the controls that I was looking for, as well as a hot shoe for Strobist shenanigans. It’s going to take me a while to master the controls, but I’m in no hurry. It did seem odd to be setting around with the tiny GF1 while the DSLR toting strobists were doing the photos of JSConf. I’ll be doing most of my Dailyshoot assignments with the GF1 — I’m looking forward to drawing material from downtown Seattle. Here are a few of the shots so far:

Bose QuietComfort 15

I am pretty sensitive to noise. Between commuting on the ferry every day, working in a building with thin walls, and spending time on airplanes, I decided that I needed help in coping with all the noise. Ever since the Bose noise canceling headsets came out, I’ve been interested in them for cutting the noise and helping me concentrate. I’ve started carrying a set of the Bose QuietComfort 15 headphones. These do a great job of cutting out noise. Most kinds of background noise gets cut out, but you can still hear human voices, albeit at a reduced volume. A little bit of music takes care of that quite easily. Like many people who reviewed these headphones, I do experience the sensation of pressure while wearing them, but these headphones are much more wearable than the earplug style Etymotic headphones that they are replacing. The only other drawback that I’ve found is that they don’t appear to built super well, so I am taking care to carry them in the semi hard case that they came in, which makes them a little less convenient.   

I think that I am well equipped to survive commuting and office life.

The program

The JSConf program was very high quality. Here are some of the talks that I found most interesting.

Yahoo’s Douglas Crockford was up first and describe Javascript as a “a functional language with dynamic objects and a familiar syntax”. He took a some time to discuss some of the features being considered for the next version of Javascript. Most of his talk was focused on the cross site scripting (XSS) problem. He believes the solving the XSS problem should be the top priority of the next version of Javascript, and he feels that this is so urgent that we ought to do a reset of HTML5 in order to focus on this problem. Crockford thinks that HTML5 is only going to make things worse, because it adds new features / complexity. He called out local storage as one feature that would introduce lots of opportunity for XSS exploits. I was very surprised to hear him advocating a security approach based on capabilities. He mentioned the Caja project and his own proposal at www.adsafe.org. He stated that “ECMAScript is being transformed into an Object Capability Language; the Browser must be transformed into an Object Capability system”. This was a very good talk, and it caused a swirl of conversation during the rest of the conference.

Jeremy Ashkenas talked about Coffeescript, which is a language that compiles into Javascript. It has a very functional flavor to it, which was interesting in light of Crockford’s description of Javascript. It also seemed to be influenced by some ideas from Python, at least syntactically. I really liked what I saw, but I’m wary of the fact that it compiles to Javascript. I am not bothered by languages that compile to JVM bytecode, but somehow that feels different to me than compiling to Javascript. I’m going to spend some time playing with it – maybe I’ll get over the compilation thing.

Gordon is a Flash runtime that is implemented in Javascript.   Tobias Schneider caused quite a stir with his talk. He showed several interesting demos of Gordon playing Flash files that were directly generated by tools in the Adobe toolset. Tobias was careful to say that he doesn’t yet implement all of flash, although he definitely wants to get full support for Flash 7 level features. It’s not clear how Gordon would handle newer versions of Flash, because of the differences beween Javascript and Actionscript. Bridging that gap is probably a whole lot of work.

Since 2008 I’ve had several opportunities to hear Erik Meijer talk about his work on Reactive Programming at Microsoft. He’s talked about this work in the context of AJAX, and a common example that he uses is autocompletion in the browser. Jeffrey Van Gogh came to JSConf to talk about RxJS , a library for Javascript which implements these ideas and provides a better experience for doing asynchronous programming, both on the client and server side. In his talk Jeffrey described RxJS bindings for Node.js.  I also met Matt Podwysocki, who I’ve been following on Twitter for some time. Matt has been writing a series of blog posts examining the Reactive Extensions. One hitch in all of this is that the licensing of RxJS is unclear. You can use RxJS in your programs and extend it but it’s not open source, and you can’t distribute RxJS code as part of an open source project. I’m interested in the ideas here, but I haven’t decided whether I am going to actually click on the license.

I dont’ remember the first time that I heard about SproutCore, but I really started paying attention to it when I saw Erich Ocean’s presentation at DjangoCon last year. The original speaker for SproutCore couldn’t make it, but Mike Ball and Evin Grano, two local members of the SproutCore community stepped in to give the talk. Their talk was heavy on demonstrations along with updates on various parts of SproutCore. They showed some very interesting UI’s that were built using SproutCore. The demo that really got my attention was related to the work on touch/multiouch interfaces. NPR had their iPad applications in the App Store on the iPad launch day. Mike and Evin showed a copy of the NPR application that had been built in 2 weeks using SproutCore. The SproutCore version can take advantage of hardware acceleration, and seemed both polished and responsive. Dion Almaer has a screenshot of the NPR app up at Ajaxian.

Raphaël is a Javascript toolkit for doing vector based drawing. It sits on top of either SVG or VML depending on what browser is being used. In the midst of all the hubub about Flash on Apple devices, Dmitry Baranovskiy, the author of Raphaël pointed out that Android devices don’t include SVG, and thus cannot run Raphaël. Apparently people think of Raphaël as something to be used for charts but Baranoskiy showed a number of more general usages of vector drawing that would be applicable to every day web applications.

Steve Souders works on web client performance at Google and has written several books about this topic. His presentation was a conglomeration of material from other talks that he has done. There were plenty of useful tidbits for those looking to improve the performance of their Javascript applications.

Billy Hoffman’s talk on security was very sobering. While Crockford was warning about the dangers of XSS in the abstract, Hoffman presented us with many concrete examples of the ways that Javascript can be exploited to circumvent security measures. A simple example of this was a simple encoding of javascript code as whitespace, so that inspection of a page’s source code would show nothing out of the ordinary to either an uninformed human or to a security scanner.

In the past, Brendan Eich and I have had some conversations in the comments of my blog, but I don’t recall meeting him in person until this weekend. Chris Williams snuck Brendan into JSConf as a surprise for the attendees, and many people were excited to have him there. Brendan covered a number of the features being worked on for the ECMAScript Harmony project, and he feels that the outlook for Javascript as a language is improving. Someone did ask him about Crockford’s call to fix security, and Brendan replied that you can’t just stop and fix security once for all time, but that you need to fix things at various levels all the time. His position was that we need more automation that helps with security, and that the highest leverage places were in the compiler and VM.

I’ve been keeping an eye on the server-side Javascript space. Ever since the competition between Javascript engines heated up two years ago, I’ve been convinced that Javascript on the server could leverage these new Javascript engines and disrupt the PHP/Ruby/Python world. If you subscribe to that line of thinking, then Ryan Dahl’s Node.js is worth noting. Node uses V8 to provide a system to build asynchronous servers. It arrived in the scene last year, and has built up a sizable community despite the fact that It is changing extremely rapidly – Ryan said he would like to “stop breaking the API every day”. In his presentation Ryan showed some benchmarks of Node versus Tornado and nginx, and Node compared pretty favorably. It’s not as fast as nginx, but it’s not that much slower, and it was handily beating Tornado. He showed a case where Node was much slower because V8’s generational garbage collector moves objects in memory. In the example, node was being asked to serve up large files, but because of the issue with V8, it could only write to the result socket indirectly. Ryan added a non-moving Buffer type to Node, which then brought it back to being a close second behind nginx. I was pleased to see that Ryan is very realistic about where Node is at the moment. At one point he said that noone has really built anything on Node that isn’t a toy. If he gets his wish to stabilize the API for Node 0.2, I suspect that we’ll see that change.

Jed Schmidt is a human language translator for his day job. In his off hours he’s created fab.js a DSL for creating asynchronous web applications in Node. Fab is pretty interesting. It has a functional programming flavor to it. I’m interested in comparing it with the RxJS bindings for Node. It’s interesting to see ideas from functional programming (particularly functional reactive programming) percolating into the Javascript server side space. In some ways it’s not surprising, since the event driven style of Node (and Twisted and Tornado) basically forces programmers to write their programs in continuation passing style.

I didn’t get to see Jan Lehnardt’s talk on evently, which is another  interesting application of Javascript (via JQuery) on the server side. I need to make some time to go back and watch Chris Anderson’s screencast on it.

The conference

As far as the conference itself goes, JSConf was well organized, and attendees were well taken care of. The conference reminds me of PyCon in its early days, and that’s my favorite kind of conference to go to. There was very little marketing, lots of technical content, presented by the people that are actually doing the work. I heard lots of cross pollination of ideas in the conversations I participated in, and in conversations that I heard as I walked the halls. I especially liked the idea of “Track B” which was a track that got assembled just in time. It’s not quite the same thing as PyCon’s open spaces, but it was still quite good. Chris and Laura Williams deserve a big hat tip for doing this with a 10 person staff, while closing on a house and getting ready for their first child to arrive.

Last thoughts

The last two years have been very exciting in the Javascript space, and I expect to see things heating up quite a bit more in the next few years. In his closing remarks, Chris Williams noted that last year, there was a single server side Javascript presentation, and this year the content was split 50/50. This is an area that you ignore at your own risk.

The User Experience

After playing with my iPad for a bit, I feel that it has captured some of the things that I envisioned in an ideal Newton experience. The form factor is right – we had had slate sized Newton prototypes that were never produced. The MessagePad 2000/2100, which you can see next to my iPad, was both too small and too large. The split between the iPhone and iPad form factors is closer to the right set of tradeoffs, at least for me. The achievements in hardware are impressive. The A4 powering the iPad can trace its lineage to the StrongARM powering the MessagePad 2xxx’s, and the ARM 6xx’s that powered the original Newton. The iPad is very responsive, much more so than my iPhone 3G or the MessagePad. That makes a huge contribution to the overall experience when you use the device. Performance is part of the user experience. Going back to the iPhone after using the iPad is a very frustrating experience. I hope that Apple will be announcing an A4 powered iPhone on Thursday. A4 and the rest of the hardware design have pushed iPad’s battery life over a key threshold. The 10-12 hour lifetimes being reported mean that the iPad should easily be able to run all day on a single battery charge. It also means that I can use the device all day without worrying about whether the battery is going to die on me. In contrast, if I am using wireless data on my iPhone, human power management is part of the user experience. Internet access is also part of the user experience. The iPad is significantly less valuable without a network connection – the Newton barely had any connectivity.

The Hardware

As happy as I am with the performance and the battery life, there are some aspects of the hardware that could be improved. The iPad screen has a glossy finish, a featured shared by my new work MacBook Pro and LED Cinema Display. Much as I love the way that photographs and colors render on these displays, the reflections and glare are problems that I haven’t been able to get over. I would have preferred a matte screen. The iPad casing is a machined single block of aluminum, again, like the MacBook Pro. I have no problems walking around carrying the MacBook Pro (at last with the display closed). When carrying the iPad in the halls in the office, I have this feeling that it might just slip out of my hand. The MessagePad 2000 series had a special rubberized paint (which was expensive) which made it easy to grip. It also had a fold over plastic cover for the screen. This version of the iPad really needs some kind of case to overcome these two issues.

The iPad has an issue when charging from non “high-power” USB ports. When attached to one of these ports, the iPad will only charge when it is asleep. If you charge your iPad overnight, this shouldn’t be a big issue, but it would have been nice to find this out from the Apple documentation rather than one of the Mac news sites.

The Software/Apps

The iPad software is largely like the iPhone software with some additional interface elements to deal with the larger screen. On the surface this just doesn’t seem like a big deal, but it is. The combination of the large screen and the performance, along with those new elements yields a much better experience. This is obvious if you run the iPhone only version of an application and then try the iPad version. In every case where I did this, I much preferred the iPad version. It is true that iPhone applications run just fine on the iPad, and that you can use pixel doubling to make them fill the full screen. But compared to a native iPad version, apps running in compatibility mode are a joke. This puts the truth to the idea that there is a new form factor in between the smart phone and the desktop/laptop. I know that in any place where I have WiFi, I will reach for the iPad instead of my iPhone. Going back to doing things on the iPhone after using the iPad seems like a kind of torture.

I wish that there were more iPad applications out there. Many of the ones that I use regularly have not been updated yet. Some of the applications that I like at the moment:

• Evernote – this is my go-to note taker on the Mac, mostly because of the syncing to iPhone. The iPad version really takes advantage of the new form factor, and I’m looking forward to being able to use the iPad as a real replacement for a paper notebook.
• Instapaper – I love Instapaper, and I’d definitely prefer to read my Instapaper articles on the iPad’s larger screen. My need for it has gone down a little bit because I signed up for Boingo in order to use WiFi on the ferry to work, so I have connectivity in many more of the situations when I would have used Instapaper
• Goodreader – This is a big one. The e-reader that I want can show me my Manning MEAP editions, the research papers from the ACM Digital Library, and MIT PhD dissertations from 1978. That means it has to do PDF. Unfortunately, the iPad doesn’t come with a PDF reader built in, which seems nuts to me. Goodreader was only a dollar and seems to have more features that an iPad of Preview might, but still.
• AccuWeather Cirrus – This is a flashy weather display program. It looks cool. And I love the little clock based UI for the hourly forecast. Yes, it’s eye candy.
• MindNode – MindNode Pro on the Mac is my program of choice for Mind Mapping, and the iPad is great form factor for mindmapping, especially that stage where you are trying to organize jumbled up thoughts
• Adobe Ideas – This is a cool little visual sketchbook application – I’m sure it will be good for doodling and quick napkin type sketches. For the heavy duty diagramming, I’m probably going to end up at OminGraffle.
• The Elements – This is an “interactive” book rendition of the paper book “The Elements” which is about the periodic table. Thus far, this is the best example of what books could become on a device like the iPad. That said, I think that we are just at the beginning of what will be possible – we’re going to see a lot of exploration and experimentation in this area over the next several years, I am sure.

In my original post on the iPad, I was inspired by the UI interactions that I saw in iWork. Of the three programs in the suite, I’ve only downloaded Keynote. I am still impressed by the UI, but I am not impressed by the compatibility restrictions. When I imported my presentations from 2009, Keynote reported a number of problems. Some of the fonts that I used were not present on the iPad, but more importantly, Keynote stripped out all my speaker notes. I hope that Apple will be adding speaker note support in a future update. On the font side, it seems like it ought to be possible to package the needed fonts as part of the Keynote presentation itself. I’m less hopeful that this will happen since there is probably some legal restriction on the ability to “distribute” fonts in this way. Keynote and iWork also showcase an area which I am unhappy about, which is integration with the filesystem on the Mac (or PC, if you must). It is very annoying to have to use iTunes to manage the files that are going in and out of iWork. It’s even more annoying when you consider something like Dropbox. I’d really like to see Apple improve this part of the experience. At the moment it feels like a copy of the Newton Connection Kit, and unforunately, that’s not a compliment.

Many applications developers still haven’t finished their iPad versions. Here’s are some of the applications that I am still waiting for:

• Either Tweetie or Echofon. I am using Twitterific at the moment, and it’s good, but on the iPhone, both Tweetie and Echofon are better. As in worth paying for better.
• Dropbox
• Facebook, Foursquare, and Yelp
• Tripit
• Meebo
• Airsharing
• Darkslide
• Google Earth
• Almost the entire Omni Group’s product line. Ok well really OmniOutliner and OmniFocus

The Omni apps are particularly important to me because they will be ports / companions of their desktop versions, which should make the iPad more usable for me in a work setting.

Open Issues

There are some other issues with the iPad which are getting a lot of discussion.

First there is the issue of freedom or openness, depending on where you come from. This has been beaten to death already. I would certainly prefer a more open ecosystem on the iPad, and I don’t think that there is an enormous amount that would need to change in order to satisfy me. After a few days of playing with a production iPad, I am convinced that this is an important device, and that the iPad is the first entrant in a mass market tablet space. I also believe that it is likely to be the most innovative because of Apple’s ability to integrate the hardware and software. There is plenty of room in the space for other players, and I believe that in the end Apple will need to make some concessions if they want to be the high volume player in the space.

The next issue is the “multitasking” issue. I remember the MacOS when there was no multitasking, then cooperative multitasking, and finally in OS X, true preemptive multitasking. At the end of the day, I want to be able to switch between multiple applications without them losing their context. I do use a few applications that could benefit from running in the background all the time, but that’s not a huge number. I would happily trade a hour of the iPad’s 10-12 hour battery life to get this capability. I am sure that the Apple team knows how to implement both the low level functionality needed as well as a good end user interface for this functionality. Multitasking is just a matter of time. It’s inevitable. Maybe it’s even tomorrow.

For those in the ebook side of the world, there’s a different sort of issue. I’ve heard several people pontificating about the difficulty and cost of creating / producing interactive books. As far as I can tell, the toolchain for this is non-existent. It looks to me that iLife includes many of the applications that someone might need in order to produce an interactive or multimedia book. Conventional wisdom used to be that it took a big movie studio to produce a decent movie. The advent of consumer HD cameras and the broadening availability of powerful computers and production is changing that. Expect the same thing to happen to interactive books.

It’s the beginning

I look at the iPad and I see the beginning of something. Even though it appears polished, I think that we have a lot more to learn about the form factor, size appropriate UI’s, the more intimate experience that tablets create, and other attributes of the platform. I for one, am looking forward to learning the lessons.

The Disney Interactive Media Group is located in downtown Seattle, a few blocks from the ferry terminal. So after nine years of working at home, I’ll be going to work in a “normal” office setting. I had several work at home offers, but I have been feeling restless about working at home, so I’ve decided to shake things up a bit on that front. Seattle locals, I’d love to catch lunch or coffee with you.

Companies

“Services” are more interesting than “pure software”. Many of the companies that I found most interesting were not creating software for distribution, but were creating or modifying software in the course of providing some other service. This is a trend that has been going on for some time, arguably since the arrival of the web, but for some reason, this stood out to me in a way that it hadn’t before.

Open Source

Open source has won, at least for the companies that I’ve talked to. Most of them were using infrastructure mostly based on open source software. Many had people contributing changes back to various open source projects. A few were looking to open source their internal software as a way of defraying development costs, increasing adoption, and/or many of the other known benefits of open source software.

There’s still some ways left to in terms of people understanding the world of open source software. Several interviewers thought that I worked for (as in got paid by) Apache. That’s probably partially LinkedIn’s fault, but it also shows that while people are eager to use open source software, they do so without an understanding of the nature and role of open source foundations.   

Nonetheless, I’m happy to see evidence that open source software is coming closer to being standard operating procedure.

Technologies

Of course it is always interesting to hear about the technologies that people are working with, especially if they have put them into production. Here are some technologies or areas which appeared often enough to be notable: Cassandra, Redis, Hadoop, mobile devices, good analytics, machine learning / prediction, and “cloud computing”.

Peopleware

I was definitely surprised by the number of companies, particularly startup companies, that were willing to take on a remote employee, especially given the state of the economy.

Process

I’ve accepted an offer for a job, and I’ll be writing about that tomorrow. For now, I’d like to say thank you to everyone who contacted me with a job opportunity. It’s nice to know that there are jobs out there, since much of what we hear about the economy is quite negative. Even more than that, I am grateful that people extended themselves to help someone (in this case me) in need.   

Yesterday MacHeist started offering pre public beta access to Tweetie 2 for Mac.   That caught my eye because Syrinx, my primary Twitter client has been a little slow at keeping up with Twitter features.   I didn’t really want to get the MacHeist bundle (don’t want to hassle with packages that I don’t want) just to get the private beta, but I mentioned on Twitter that I was thinking about it.   Several folks suggested that I try Echofon.   I gave it a whirl, found some things that I like and other that I didn’t.   I started keeping notes about Syrinx vs Echofon, and now it’s turned into a blog post.

My usage style / requirements

I follow a bunch of people, including many people who live in Europe who tweet while I am asleep.   I need a client that can remember unread tweets from overnight.    I’ve found very few clients that are able to do this.     My reading style tends to be bursty as well, so I want the client to do a good job of keeping track of what I’ve read and what I have not.    These two requirements are what has kept me on Syrinx – it can hold days worth of tweets without a problem.   Syrinx’s bookmark also gives me definite way of marking what has been read and what has not, and puts control of that mark directly in my hands.

The other major requirement is that I spend some time (probably too much) on airplanes, without net access.   I want a client (mostly on my iPhone) that can go back in fill in the gaps left by being in the air.   Tweetie 2 for the iPhone can do this, but the experience of switch back and forth between reading the stream on desktop Syrinx and iPhone Tweetie 2 is annoying.

A minor requirement is to be able to monitor a number of Twitter searches at once – that means opening a window for each search, something that Syrinx also does.

Now, let’s have a look at how Syrinx and Echofon stack up for me.

Syrinx

The obvious things that I like about Syrinx are that it can hold as many tweets as I want, as well as the bookmark.    I’ve also grown accustomed to the way that it displays time in absolute format, something which Tweetie 2 / iPhone also does.   One other nicety in Syrinx is that it can display real names in addition to Twitter handles, because sometimes handles and people are hard to match up.   When you have tons of tweets lying around in the? client, sometimes you want to go back to one, and Syrinx obliges with the ability to search all the tweets that it currently has in memory.

So what are the problems with Syrinx? It’s been occasionally unstable, but not in a show stopping fashion. It doesn’t have good support for lists, but I still haven’t made much use of lists. Syrinx does great on opening windows for searches, but it doesn’t remember what searches you have open, so you have to keep track of that yourself. Probably the biggest drawback of Syrinx is that its development is going slowly because its author has a day job.

Echofon

When I compare Echofon and Syrinx, I realize that a lot of the things that I prefer in Echofon are niceties. I like that it can open browser links in the background. I like the way that the drawer is used for dealing with Twitter users and profiles and for displaying conversations.   I just wish it could display more than one conversation at once – but that’s hard in the drawer model. The ability to colorize tweets matching keywords makes it easier to pick out tweets on high priority topics.    As a photographer, I appreciate the ability to display pictures without going all the way to the browser.    I do wish there was a way to get some kind of preview of those pictures right in the tweet stream.   Echofon does this clever thing where it combines “rapid-fire” tweets from the same person.   This seems to work really well, and the visual cue is definitely helpful.  

Looking at the tweet authoring side,  I love the “retweet with comment” option.   One reason that I stopped commenting on retweets was that it was annoying to do it.  No more.   Echofon can tab complete Twitter id’s when @replying or direct messaging.    I still wish for a direct message “rolodex” – there are some people who have hard to remember Twitter id’s.   bit.ly is my preferred URL shortener because of the analytics, but you have to be logged in to bit.ly in order for that to work well.   Fortunately Echofon is able to log into bit.ly accounts so that your analytics work.

In theory, I like the idea of an Echofon ecosystem that syncs the desktop and mobile clients.   I haven’t tried this yet because I have iPhone Twitter client fatigue, and because as much as I like Echofon, there are some issues that make it hard for me to switch over.

The first of these issues is that Echofon won’t hold all of the tweets that happen overnight.  It looks like Echofon will hold about 5 hours of tweets before it starts to drop them on the floor.  There go some of those European tweets.

The next big issue is that marking read/unread doesn’t work for me.  If I am scrolling up through my home tweets and I hit the top, everything gets marked read.   It’s easy to do that by accident.   Switching to the @, DM, or search tabs also marks my home tweets as all read, and that doesn’t work for me at all.

Compared to those two issues, everything else is just nits, but here goes, just to be complete.   Echofon doesn’t display absolute time or real names.    Also, Echofon doesn’t let you search your home tweets.

Wild and crazy wishes

Certain URL shortening services (su.pr and ow.ly come to mind) wrap the page in a header bar, which is annoying.  I’d love if my client would route through those services so that the URL that I got in the browser was the actual content.

Sometimes there are links that are retweeted a bunch.   I would love it if a client could compress all those retweets into a single entry which showed how many / which people I follow retweeted a link, along with an indication of whether or not I had already “read” an earlier retweeter (which would mean I had already read the link).

I guess I’ll have to do another version of this post when Tweetie 2 for Mac finally ships.   Or maybe it’s still early enough for some of these ideas to make the cut.

Pythonistas (and others) may be wondering what this means for dynamic languages at Oracle. I wish I knew. I don’t have any direct knowledge of this, since I’ve never actually spoken to anyone at Oracle about the topic.   

I am definitely looking for another opportunity. During my time at Sun I’ve worked on a bunch of Python related stuff, as well as a few things related to cloud computing. Other skills in my repertoire include server side development (Java and Python), open source community work, and engineering management. I’m definitely open to different possibilities. The about page of this blog has my contact information, and my LinkedIn profile is a pretty good summary of my credentials.

User Interface

One the one hand, the iPad is the same iPhone OS that is familiar to 70 million iPhone users. On the other hand, some of the keynote demos show that the larger form factor is going to have some interesting UI potential.

I usually try to pay careful attention to presentations by game developers. It’s not because I am a big gamer myself, but it’s because the people doing games are usually doing some of the most insane, crazy, and interesting things in the business, and it’s worth paying attention to the things that they say are important. Both of the game demos for the iPad had some pretty interesting UI and commentary on the experience of the machine as a whole.

The other really interesting part of the keynote was the iWork demo. I am very impressed with the way that iWork has been adapted to the touch screen. There are a number of really cool multitouch gestures that were demonstrated. This is going to be the beginning of some very interesting user interface stuff.

Integration

I spent most of yesterday watching the Oracle/Sun strategy webcast, and a major theme was the way that Oracle plans to tightly integrate Sun’s hardware, and to optimize the entire hardware and software stack. The Oracle Exadata database machine was repeatedly touted as an example of this kind of integration. If the benchmarks and early customer experiences are indicative, this integration has paid off handsomely, as it has also with the Sun Storage 7000.

The new A4 processor powering the iPad received only brief mention during the keynote, but here too is the same kind of integration. Details on the A4 are very scarce, but speculation is that it was done by the team that Apple acquired from PA semiconductor. It appears to be an ARM compatible (iPhone apps do run) system on a chip design, and I would bet that it is contributing to the (relatively) low price, long battery life, and high performance (according to Gruber) of the device.

I think that it’s worth noting that companies like Google are also doing this kind of vertical integration, building their own custom PC designs, having custom Linux kernels and other software. Many of us in the “open” world decry vertical integration because it is almost inevitably closed, but the kind of engineering virtuosity that is on display does impress.

Wireless

Apple appears to have gotten iPad users a deal on 3G pricing from AT&T. I am not really sure that this is a step in the right direction. If Apple is to believed, we are entering a world where a person could have no less that 3 devices (phone, pad, laptop) in need of wireless data (and voice) connectivity. A contract/plan for each device might be great for the carriers, but it is horrible for the users. Since even Apple has backed down in the face of the carriers, it doesn’t look like this is going to change much, but it ought to.

Me

Will I buy one? I’ve been toying with the idea of buying Kindle for some time now. I wanted the size of the Kindle DX, since I wanted to read PDFs of books and research papers, but I felt that $499 for the DX was too much to pay for a book reader. The iPad is obviously a much more capable device than a Kindle, and I’d expect Amazon to upgrade their Kindle iPhone app to run on the iPad.   

I think that the iPad would be vastly superior to my iPhone as a means of showing my photographic portfolio. I can also imagine using an iPad as a tethered shooting target, which would definitely be interesting. The tablet form factor could lead to some pretty interesting photography applications, and the iPad CPU appears to be reasonably capable.

I’ll say this much – I definitely want to play with one.

January

This year I did a lot more work with local and regional dancers. Here’s a danceseattle rehearsal shot.

February

I continued my role as the official photographer for Bainbridge Island Chinese Connection’s Chinese New Year Celebration.

March

I caught this shot of Guido van Rossum at PyCon by being in the right place at the right time. My camera was lying on the table next to me when Guido suddenly grabbed the Django Pony and started running down the aisle. He was moving fast enough that I had to snap off a bunch of frames to catch him in focus.

April

April was busy dance month. The Olympic Performance Group put on “The Toymaker’s Doll” (also known as Coppelia).

danceseattle had their first ever performance.

For the first time in a long time, I actually was able to show up to a Seattle Flickr Garage shoot.

May

May is when the weather in the Seattle area starts to get decent, so I was able to get some nature subjects in front of the lens.

June

I headed to San Francisco for JavaOne in early June.

I finished out June with Bainbridge Ballet’s end of year recital.

July

The Bainbridge Island Fourth of July Parade is always a family and photographic staple.

Also in July, we had the first guinea pig born in our house.

August

I made it to a second Seattle Flickr garage shoot.

September

Senior pictures for Bainbridge High School are due at the end of September, and I did 4 sessions in the space of 12 days or so.

October

School was in full swing in October, and one of the science lessons that Julie did with the girls involved extracting DNA using Bacardi 151 rum.

November

This year was the 10th Anniversary of the Apache Software Foundation (and my involvement with it). I did take a few shots while I was at ApacheCon.

I was also fortunate enough to get a slot to J. Mark Wallace’s US Meetup Tour when it hit Seattle.

December

Photographically, December is dominated by the Olympic Performance Group’s production of the Nutcracker.

Talks

This year I was unable to attend all of the conference due to some scheduling problems, so I can’t give an in depth report on talks. I used some of the time that I might normally have spent in talks to catch up with people that I haven’t seen in a while. I was able to attend a good number of the talks in the Hadoop track. The track was larger than last year’s track (due in part to a larger room), but I felt that last year’s track was stronger. It might also be that I’ve become a bit more familiar with Hadoop, making it harder to make a bing impression. It’s definitely the case that there was a lot of interest in Hadoop, and I expect that to continue.

Unfortunately, I missed the NoSQL meetup during the Apache BarCamp. I think that there could/should have been an entire NoSQL track, especially given the fact that Cassandra and CouchDB are both frequently mentioned NoSQL technologies, and both are housed at Apache.

One talk that surprised me was Ross Gardler’s talk Teaching and Learning about Open Development, originally I didn’t think that I would have time to stay for that talk slot, but a rearrangement of my return flight loosened my schedule so that I could stick around. Ross is the chairman of the newly created Community Development PMC at Apache. This is a new effort aimed at improving the experience of contributors and new committers. Some of the people on the PMC have been heavily involved in the ASF’s Google Summer of Code outreach, and will be bringing their experiences over with them. It seems like this PMC will also be a good place for people concerned about diversity issues to dig in and help in a concrete fashion.

Celebration

This year’s ApacheCon’s have been a celebration of the 10 year anniversary of the founding of the Apache Software Foundation. At Oakland, there was a cake, a proclamation from the Mayor of Oakland, and (I didn’t get to see this) a letter of congratulations from the Governor of California. Rather than try and describe the festivities in prose, I’ll leave you with some photos:

The entire set of photos is up on Flickr.

November is just around the corner, which means that once again it’s time for ApacheCon US. This year is a special year for the Apache Software Foundation – its 10 year anniversary. Since I got involved with Apache just a few months after the foundation was created, it is also my 10 year anniversary of being involved in open source software.

This year I am going to be speaking twice. On Wednesday I’ll be speaking on the Apache Pioneers Panel, and on Thursday I’ll be giving a talk titled How 10 years of Apache has changed my life. I owe a huge professional debt to the ASF and its members and committers, so in my talk I’ll be interweaving important events in the life of the foundation with my own personal experiences and lessons learned.

Unfortunately, I’m not going to be there for all of the conference this year – I’ll be arriving Tuesday afternoon and flying out on Thursday evening. If you want to meet up, I’m in the ApacheCon Crowdvine, and I’ll be around with camera in hand (and on the LumaLoop).

The following week at DjangoCon, I lost the strap portion of my Upstrap quick release strap. I liked the Upstrap, but it wasn’t ideal. The Upstrap was great because of the non stick rubber pad that they use – it really won’t move. But like most other camera straps, I found that I was constantly getting it fouled in my arms or something, especially between landscape and portrait modes.

Duncan had promised me one of the early prototypes of the LumaLoop, so I put the official black and neon yellow strap on the D3 and waited patiently. Yesterday, my LumaLoop arrived, and I quickly installed it in place of the Nikon strap. The LumaLoop is a “sling strap” similar to the Black Rapid R-Straps that have become popular recently. The Black Rapid straps screw into the tripod socket on your camera, which is a problem if you have any kind of heavy duty tripod plate mounted on your camera, or if you shoot vertically a lot (this is even more of a problem if you have small hands and a camera with a battery grip). The LumaLoop attaches to one of the regular strap mounts on your camera, and once attached, you can slide the camera up and down the strap. The mounting loop is attached with a quick release clip, so swapping cameras/straps is easy as well. Duncan has a series of blog posts that detail the reasoning behind the design:

• Hacking the R-Strap
• Camera Strap Update 1
• Camera Strap Update 2

Here’s a quick snapshot of mine:

You can see the loop part that goes on the camera, as well as the quick release between the loop and the rest of the strap. It’s a bit harder see the padded non-slip shoulder pad.

The LumaLoop is going to be available from Luma Labs sometime very soon (Duncan gave me perimission to talk about the LumaLoop in advance of its general availability). You can follow Luma Labs on Twitter to keep up with all of the news and the official announcement. I’m excited to have a strap that both holds my camera securely and stays out of my way when the action gets going.

I made the statement

From where I sit, this is all about exploiting multicore hardware

because I’ve seen a pile of actor and other concurrency libraries which have not taken parallel execution of the concurrent program seriously. If I am going to go to the trouble of writing a concurrent program, then I want that execution to be parallel, especially in a multicore world.

Simon Marlow from the GHC team said that if programming multicore machines is the only goal we ought to be looking at parallelism first and concurrency only as a last resort. Haskell has some nice features for taking advantage of parallelism. However, I explicitly stated that I was not as interested in highly regular or data parallel computations, which is what Haskell’s parallelism tools are aimed at. These are fine ways to get parallelism, but I am interested in problems which are genuinely concurrent, not just parallel. In a Van Roy hierarchy, these are the problems with observable nondeterminism. I also specifically called out reduction of latency as one of my goals, something which Marlow says is a possible benefit of concurrency. The GHC team is interested in a different mix of problems than I am.

Van Roy in short

I also forgot to mention Peter Van Roy’s paper Programming Paradigms for Dummies: What Every Programmer Should Know, which includes an overview of his stratification of concurrency and parallelism (and other stuff). If you don’t have time to read his book, the paper is shorter and more digestible.

The Cambrian

The Cambrian period was marked by a rapid diversification of lifeforms. I think that we are in a similar situation with concurrency today. Although many of the ideas that are being tossed around for concurrency have been around for some time, I don’t think that we really have a broad body of experience with any of them. So I’m less optimistic than Tim and Bruce Tate, at least on time frame. I think that we have a lot of interesting languages, embodying a number of interesting ideas for working with concurrency. I think that some of those languages have gained enough interest/adoption that we are now in a position to get a credible amount of experience so that we can start evaluating these ideas on their merits. But I think that the window for doing that is pretty large, on the order of 5 to 10 years.   

What kinds of problems

The kinds of problems I am interested in are general purpose programming problems. I’m specifically not interested in scientific, numeric, highly regular kinds of computations or data parallel computations. Unlike Tim, I do think that web systems are a valid problem domain. I see this being driven by the need to drive down latency to provide good user response time, not to provide additional scalability (although it probably will).

It’s not like Java

Erik Engbrecht, one of Tim’s commenters said:

To get Java, you basically take Smalltalk and remove all of the powerful concepts from it while leaving in the benign ones that everyday developers use.

I think there’s something to be learned from that.

This presupposes that you know what all the good concepts are and what the benign ones are. It doesn’t seem like we are at that point. When Java was created, both Lisp and Smalltalk had existed for quite sometime and it was possible to do this kind of surgery. I don’t have a clear sense of what actually works well, much less what is powerful or benign.

The hardware made me do it

From where I sit, this is all about exploiting multicore hardware, and when I say this I mean machines with more than 4 or 8 hardware threads (I say threads, not cores – actual parallelism is what is important). The Sun T5440 is a 256 thread box. Intel’s Nehalem EX will let you build a 128 thread box later this year. Those are multicore boxes. If you look at experiments, you see that systems that seem to work well at 2 or 4 threads don’t’ work well at 16 or 64 threads. Since there’s not a huge amount of that kind of hardware around yet, it’s hard for people to run experiments at larger sizes. Experiments being run on 2 thread MacBook Pro’s are probably not good indicators of what happens at even 8 threads.. This is partially because dealing with more hardware threads requires more administrative overhead, and as the functional programming people found out, that overhead is very non-trivial. The point is, you have to run on actual hardware to have believable numbers.   This makes it hard for me to take certain kinds of systems seriously, like concurrency solutions running on language implementations with Global Interpreter Locks. See David Beazley’s presentation on Python’s Global Interpreter Lock, for an example.

Comments on specific languages

At this point I am more interested in paradigms and constructs as opposed to particular languages. However, the only way to get real data on those is for them to be realized in language designs and implementations.

• Haskell – Functional Laziness aside, the big concurrency thing in Haskell is Software Transactional Memory (STM). There are other features in Haskell, but STM is the big one. STM is an active research field in computer science, and I’ve read a decent number of papers trying to make heads from tails. Among the stack that I have read, it seems to be running about even between the papers touting the benefits of STM and the the papers saying that STM cannot scale and will not work in practice. The jury is very much out on this one, at least in my mind.
• Erlang – I like Erlang. It’s been in production use for a long time, and real systems have been built using it. In addition to writing some small programs and reviewing some papers by Erlang’s designers, I spent a few days at the Erlang Factory earlier this year trying to get a better sense of what was really happening in the Erlang community. While there’s lots of cool stuff happening in Erlang, I observed two things. First, the biggest Erlang systems I heard described (outside of Facebook’s) are pretty small compared to a big system today. Second, and more importantly, SMP support in Erlang is still relatively new. Ulf Wiger’s DAMP09 presentation has a lot of useful information in it. On the other hand, BEAM, the Erlang VM is architected specifically for the Erlang process/actor model. This feels important to me, but we need some experimental evidence.
• Clojure – Clojure as a ton of interesting ideas in it. Rich Hickey has really done his homework, and I have a lot of respect for the work that he is doing. Still it’s the early days for Clojure, and I want to see more data. I know Rich has run some stuff on one of those multiple hundred core Azul boxes, but as far as I know, there’s not a lot of other data.
• Scala – The big thing in Scala for concurrency is Actors, but if you compare to Erlang, Actors are the equivalent of Erlang processes. A lot of the leverage that you get in Erlang comes from OTP, and to get that in Scala, you need to look at Jonas Boner’s highly interesting Akka Actor Kernel project. Akka also includes an implementation of dataflow variables, so Akka would give you a system with Actors, supervision, STM, and Dataflow (when it’s done).   
• libdispatch/Grand Central Dispatch – Several of Tim’s commenters brought up Apple’s Grand Central Dispatch, now open sourced as libdispatch. This is a key technology for taking advantage of multicore in Snow Leopard. GCD relies on programmers to create dispatch queues which are then managed by the operating system. Programmers can send computations to these queues via blocks (closures), which are a new extension to Objective-C. When I look at Apple’s guide to migrating to GCD from threads, I do see a model that I prefer to threads, but it is not as high level as some of the others. Also, the design seems oriented towards very loosely coupled computations.   It will be several years before we can really know how well GCD is working. I am typing this post on a 16 thread Nehalem Mac Pro, and I rarely see even half of the CPU meters really light up, even when I am running multiple compute intensive tasks. Clearly more software needs to take advantage of this technology before we have verdict on its effectiveness in production.
• .Net stuff like F#/Axum, etc – There is some concurrency work happening over on the CLR, most notably in F# and Axum. I spent some time at Lang.NET earlier this year, and got a chance to learn a bit about these two technologies. If you look at paradigms, the concurrency stuff looks very much like Erlang or Scala, with the notable exception of join patterns, which are on Martin Odersky’s list for Scala. I will admit to not being very up to speed on these, mostly for lack of Windows and the appropriate tools.

Other thoughts

Jacob’s take away from my talk at OSCON was “we’re screwed”. That’s not what I wanted to convey. I don’t see a clear winner at the moment, and we have a lot of careful experimentation and measuring to do. We are quite firmly in the Cambrian, and I’m not in a hurry to get out – these things need to bake a bit longer, as well as having some more experimentation.

In addition to my talk, and Tim’s wiki page, if you are really interested in this space, I think that you should read Concepts, Techniques, and Models of Computer Programming by Peter van Roy and Seif Haridi. No book can be up to date with the absolute latest developments, but this book has the best treatment that I’ve seen in terms of trying to stratify the expressiveness of sequential and concurrent programming models.

Talk Highlights

Here are some of the talks that I found notable:

Shawn Rider and Nowell Strite of PBS gave a talk titled: Pluggable, Reusable Django Apps: A Use Case and Proposed Solution. I think that very few people in the Django community had any idea that Django was being used extensively in PBS. That definitely falls into the category of pleasant surprises.    One of the strengths of Django is the focus on building small single purpose applications which can be used to build up larger applications. Doing this is harder than it sounds, and Shawn and Nowell described some of the problems that they ran into, as well as showing some ways of dealing with those issues. There was some crosstalk between the PBS guys and the Pinax developers, who are also doing a lot with reusable apps. I hope that these folks will work to share and combine their knowledge and then disseminate that to the broader Django community.

There were a number of talks which followed the theme of “how not to use parts of Django”. It’s interesting because people like Django, and even if they don’t like some parts, they want to use the rest, and are willing to work to make that possible. You would expect people to just walk away from the framework in cases like these.

Eric Ocean, one of the developers of SproutCore gave what I considered to be a pretty interesting talk. Unfortunately, his talk didn’t have much of a connection to Django, other than to suggest some things that Django could do to support SproutCore better. I know from watching the IRC and Twitter back talk, that people were put off by the style of presentation (a little too like a commercial), and the weak connection to Django. SproutCore is interesting to me because it’s at a different level than most of the Javascript frameworks. It’s at a higher level, which I think will be necessary as browser based applications become more sophisticated. I know that I am going to be taking a closer look at SproutCore, and I hope that a useful Django/Sproutcore collaboration will emerge from the sprints.

Simon Willison gave a keynote about Cowboy programming. The big piece for me was his description of how the Guardian built an application to help the public scrutinize the expenses of British MP’s. There’s something about these situations that appeals to me, against my knowledge and better judgment of “sound software engineering practices”. I guess it’s a guilty pleasure of sorts.

Ian Bicking gave a keynote which might be described as “a free software programmer’s midlife crisis”. Ian was very philosophical and reminded us that free software (as opposed to open source software) was rooted in a set of moral (not economic or process) imperatives. It was a very thoughtful speech, and I think that its worth several reads of his text (something which is hard to do on a train ride with an iPhone) and some additional ponderings.

Avi Bryant’s keynote took its root in his experiences building Trendly. As one might expect, Avi started building Trendly using Seaside. But by the time he finished, he noticed that very little of Seaside was actually being used. He attributed this to the fact that Trendly’s architecture involves loading a single HTML, with a ton of Javascript. That Javascript then manages all of the interaction with the server, which consists of snippets of JSON data. This range true to me because we used a similar architecture for Chandler Hub, the web based version of Chandler (our interaction with the server was based on atom and atompub, not JSON), and it’s the kind of architecture that GMail is based on. Avi also treated us to a demonstration of Clamato, his Smalltalk dialect that compiles to Javascript. Again, another attempt to deal with the challenges of engineering large Javascript applications in a web browser.

There were plenty of other good talks, and many of the slides are already available.

My keynote

I’m afraid that I am not equal to the task of writing out my presentation text in full as Ian and Jacob have done, so you will have to settle for the highlights and wait until the video appears.

My keynote was organized around two major sections.

The first section was a look at what I see in the Django community at present. This includes a look at some pseudo statistics around job postings and a poll of web frameworks being used by startups in an effort to get some view into whether and how much adoption of Django is happening. The short answer is that things look promising, but there is still plenty of room to grow. On the technology side, I pointed out the emphasis on combining applications and the work of the Pinax and PBS folks. The other major technology thing that I called out was GeoDjango, which is undoubtedly the most sophisticated GIS functionality in any web framework in any language or platform. This is going to be very attractive to people building location aware mobile apps, and I showed two examples of augmented reality applications as illustrations. This section ends with some observations about the Django community, using the PyCon sprints as an example. Ok, there are also some lighthearted slides about Django’s mascot, the djangopony.

The remainder of the talk was about the ways that web applications are changing and how Django might adapt to them. There are (at least) three groups of people that will be impacted by these changes. From the view point of users, the two big things are richer, more interactive applications, and access from location enable devices. Developers are going to need help in dealing with these new requirements, and the people who operate web applications need much more support than they currently have.   

I see several technologies that will be important in facilitating these changes. The first of these is some Rich Internet Application technology. The second is API’s to web applications. A digression on this point. When the iPhone was introduced, the only way to develop applications was using web technologies. This made a lot of people very angry, and Apple followed up with the ability to build native platform applications. It should be possible to build rich web interfaces on the iPhone. My observation is that given the choice beween a rich web interface and a native iPhone application, users pick the native application. Look no further than the furor over the native Google Voice application. The native applications are talking to the servers using API’s. Those API’s are not just cool Web 2.0 frosting. The last technology is cloud computing, which started out as a deployment/operations technology and is now moving up to impact application development at many levels.

In light of this, what are framework developers to do? I did a quick survey of several web frameworks which have interesting ideas or approaches in them, so that the Django folks could see what their “competitors” are up to. The frameworks that I included are Rails, Lift (Scala), Webmachine (Erlang), Nitrogen (Erlang), CouchApps (CouchDB + JavaScript – this isn’t quite a framework in the traditional sense, but it met the spirit of my criteria), and Javascript. In the case of Javascript, the observation is that the rapid increase in Javascript performance coupled with a good Javascript framework leads to something which is economically attractive (same technology in the server and client).

The talk finishes with a set of proposals for “science projects” that might be attempted in the context of Django. Some of what I outlined is emerging, and in some cases speculative. Django doesn’t need to blow itself up and start over. Instead, what’s needed is for people with Django sensibilities to look some of these problems and see if a Django flavored solution can be found. Here’s the list of projects:

1. Asynchronous Messaging – if there’s any use of messaging, it’s typically to do jobs in the background. What would happen if we made the use of messaging pervasive throughout the framework?
2. Comet – I think that the Django+orbited approach to Comet is limited in comparison to what you see in Lift or Nitrogen. Can Django do Comet support at the same level (or better) than these frameworks? What would happen if the Comet stuff were hooked up directly to the messaging stuff I just described? Imagine the equivalent of urls.py that routed Comet requests to messaging.
3. REST – There are several good packages for dealing with REST in Django. It would be nice to have this all packaged up neatly and made available for people.
4. Deployment – This is really a mess. Are there changes that could be made to Django to make it easier to deploy, or to work better with tools like Puppet, Chef, Fabric, etc?
5. Monitoring – Typically frameworks provide very little monitoring information. It seems like there is a lot that could be done here.
6. Analytics – Once you have raw monitoring information, the next step is to do some analytics on it. Django is famous for creating admin UI’s with a very small amount of effort. What if we applied that same thinking to analytics?
7. Cloud – If you add up the first 6 items, you are well on your way to what might be a cloud friendly framework. There is still some work to do in terms of making applications on the framework adapt to elastic deployment scenarios, but it would be a good step.
8. Stacks – A very basic step towards cloud stuff would be to build a preconfigured stack of software to run/develop a Django app. This is a controversial idea, because everyone has their own idea of what software should be in such a stack, and how all the configuration switches should be set. I still that having one (or more) such stack would help more than it would hurt. In my ideal world this stack would be delivered as a virtual machine image that could also be uploaded to cloud providers.

All of the talks were videotaped so those of you who were unable to attend will be able to catch up soon.
I had a great time hanging out with the Djangonauts. My thanks to Leah Culver and Robert Lofthouse for inviting me to speak.