Monthly Archive for November, 2008

Lifestreaming clients

I have usernames on most of the major lifestreaming services (Twitter, FriendFeed, identi.ca, and so on). For a variety of reasons, I really only use Twitter, and the only way that Twitter is useful / manageable for me is the existence of rich client side applications.

Mac OS X

For some time, I’ve been using Craig Hockenberry’s excellent Twitterific. I liked the UI, and the feature set was good. From time to time, I would try the Adobe AIR based twhirl, which had the virtue of also being a FriendFeed and identi.ca client. Unfortunately, I could never keep twhirl because of a bug in AIR 1.1 that caused clicked URLs to open in a new Firefox window instead of a new tab. That bug was fixed in this weeks AIR 1.5 release, so I gave twhirl another try earlier this week. I liked having FriendFeed and identi.ca up (having identi.ca up meant that I saw Allison Randall’s messages about the Parrot Developer summit and their new release schedule). I didn’t like having a window for each service — I don’t care about keeping it separate, and I’m still having some trouble finding a theme that works for my aging eyes. Twhirl also doesn’t seem to remember window positions between runs, which makes the multiple windows even more of a pain. I also miss seeing people’s “real names” and the Growl notifications that I was getting from Twitterific. I put twhirl back on the shelf, but will probably come back to it again.

A week or two ago, I discovered Syrinx, which is a Twitter only client. There were a few things that persuaded me to try it out. The ability to set a bookmark at some point in the message stream and then go back to it. This seems to work better for my style of reading than individual read/unread markers on each Tweet. The keyboard shortcut means that I can jump right to where I left off, which is nice. Syrinx also lets you search the stream, which is useful. I follow enough people that searching is useful. I was also (incorrectly) under the impression that Syrinx would save a slice of the message stream locally, which would be a nicety. I can page backwards on the Twitter site, but that way lies pain. Syrinx has a way of tracking twitter “conversations” and finding the supposedly relevant tweets and presenting them. I like this idea, I just wish it wouldn’t take over the main message stream window in order to show it. The biggest problem with Syrinx is that there something awfully bloated in there, which means that after some time, the app is eating memory and slowing down. Which means you have to restart it, which means you have to catch up first. MRR, the author of Syrinx, knows that this is a problem and is working on a solution. I hope that won’t take a long time.

Because of the AIR 1.5 release I also tried TweetDeck this week. I tried it, and there were some interesting features. I liked the ability to make my own groups of people – but Twitter should be supporting that. I also liked the way that replies and direct messages could be in their own column – I really liked that, actually. I liked the idea of TwitScoop, but what I’d really like would be a TwitScoop of my Twitter network – that would be cool. TweetDeck was great when I put it on my 30″ main display. You can see lots of stuff and quickly see if there is anything useful. Unfortunately, I’m not willing to dedicate that much screen real estate — whatever client I use has to live (and share) on the “outboard” main LCD of the MacBook Pro.

iPhone

When I got my iPhone, I started using Twinkle. There pretty much wasn’t anything else, and I sort of liked the idea of having some kind of location awareness of people using the service. Turns out that very few people that I know use the Twinkle location stuff, and I’ve pretty much switched to using Brightkite for that kind of thing, and even there, the jury is out. User interface wise, I like the fact that it colors replies and direct messages differently — it makes them much easier to pick out. I don’t like that I have to tap on a tweet containing a link in order to open the link.

I’ve since switched to using the iPhone version of Twitterific. I don’t have to tap on tweets to follow links, and Twitterific is pretty good about storing a decent number of tweets on the phone. I can usually take a 2 hour plane flight and not have missed much when I land on the other side. I’d love to not miss anything at all. One annoyance is that Twitterific for iPhone doesn’t remember the last tweet that I was looking at very well, so I end up doing a lot more scrolling than I should have to.

Wish List

Here’s a consolidation of the some of the things that I think are important in rich clients for Twitter and services like it.

  • Good management of windows – I don’t want a window for each service – I want one big stream.
  • Good visual design that easily lets you differentiate between different kinds of messages (tweets/replies/direct messages). Make links easy to see and follow.
  • Keep a local, searchable, history of messages.
  • Provide a good, low maintenance way for me to keep my place in a busy stream.
  • Give me a way to follow conversations (chains of replies). I would be happy to have a menu for this.
  • Integrate some of the third party services that are springing up, like TwitScoop.

On the mobile side, there is one feature that I would consider killer.

I want a “direct message” rolodex. There are people who I want to direct message on a frequent basis. I don’t remember everybody’s twitter user name – that’s what computers are for. I want a “picker” that contains a “speed direct message” list. That would be awesome.

This is one space where rich/desktop applications are by no means dead.

Evernote and other applications that are getting a workout

It’s been a while since I reported on the state of my Macintosh. Here are a few apps that I’ve been using a lot recently.

Evernote

I’ve had Evernote installed for quite some time, but I didn’t really start using it until after I got my iPhone. So I was interested to read Ars Technica’s report that 57% of Evernote’s users are using the iPhone client. Evernote is a great example of the “rich application architecture of the future”. Evernote’s family of applications include desktop clients for Mac OS X and Windows, a web applications, and mobile clients, most notably the iPhone. All of these pieces work together to make a great integrated solution. This is the kind of ecosystem that we were building around Chandler, although we never got to the mobile part, and as the Evernote data suggests, we would have been fine just creating an iPhone client. Of course, hindsight is 20/20.

Apple helped Evernote tremendously by providing a barely functional notes application on the iPhone, and then providing no way to sync notes back to a Mac. So the iPhone Evernote client fills a great hole in the iPhone application suite. That got me started using Evernote for information that might need to move back and forth between desktop and device. The next step up for me was that I started using Evernote to take notes for conferences. I used to use Ecto for that, and I would then rewrite my notes into a blog post. But I missed having the raw notes, so I decided that instead of creating a billion drafts in Ecto to hold the raw notes, I would just take all the notes in Evernote, and then write the posts in Ecto. This of course had the added benefit of me being able to use other features of Evernote. I definitely think that the Evernote team is doing something that desktop and mobile software developers ought to be paying attention to.

1Passwd

Another good example of this desktop/web/mobile trend is the fantastic 1Passwd password manager for Mac OS X and iPhone. I got 1Passwd as part of a MacUpdate software bundle some time back. It took me quite some time to start using it, because I was happily using Firefox’s built in password manager. 1Passwd has the advantage of working with Firefox, Safari, and NetNewsWire on my desktop. It does a much better job of dealing with odd web site logins. It does a great job of managing my ridiculous number of passwords. Actually it has a great password generator built in, which makes it easy to stop the common practice of having a few relatively easy to remember passwords that you use everywhere. Which is just plain bad security. 1Passwd also has an iPhone version, which means that accessing sites from my iPhone is no problem at all either. Great piece of software.

PathFinder

The last piece of software is PathFinder, which is PODS (plain old desktop software). PathFinder is a great replacement for the Finder, and the latest version, 5.0, adds a dual plan feature that makes file management tasks much easier. You can also manage sets of tabs. I use this feature to manage projects, by creating a set of tabs for each project. I can then flip a PathFinder pane into exactly the configuration that I want for working on that project. It’s a shame that Apple has been so lackadasical about improving the Finder. Maybe this will improve with the rewrite of the Finder for Snow Leopard. In the meantime, PathFinder is a good solution for those of us that need a little more than what the Finder provides.

CouchDB graduates

Congratulations to the Apache CouchDB project, which has graduated from the Apache Incubator and become a top level project of the Apache Software Foundation.

Python in NetBeans

Along with today’s launch of NetBeans 6.5, Sun, in cooperation with the NBPython community, are releasing an early access version of Python support for NetBeans. This is a result of the collaboration between Sun people and the NBPython project that I wrote about back in July. This release has been tested by folks in the NetBeans community and some folks from Sun’s NetBeans QA team, and it’s in pretty good shape for an early access release. We’re interested in getting people’s feedback. We would also love to see more people get involved with NBPython.

How to get it?

You can get NetBeans Python from the NetBeans download page.

What’s in it?

The basic feature set for the early access release consists of an editor for Python, the ability to execute Python programs (using CPython or Jython), and a debugger.

There’s a tutorial up on the NetBeans wiki.

Tor Norbye, who did most of the work on the editor, has written a series of blog posts detailing various features of the Python editor.

Who did it

Allan Davis – project and platform management, interactive console.

Jean-Yves Mengant – Jean-Yves is the author of the jpydbg debugger, which he’s merged into NBPython.

Amit Saha – documentation and help sets – Amit works for Sun, but he’s doing Python on his own time.

Tor Norbye (Sun) – editing.

Tomas Zezula (Sun) – project and platform management.

Ted Leung (me) (Sun) – various behind the scenes stuff.

Frank Wierzbicki (Sun) – NBPython is using Jython’s parser and Frank worked with Tor to add support for positions and better error reporting.

Peter Lam (Sun) – Sun QA

Tony Beckham (Sun) – Sun QA

The NetBeans CAT community as well as those folks who drove by and reported bugs.

How to get involved

NBPython has become a full fledged NetBeans project, so the main project page is now on NetBeans.org, as are the issue tracker and mailing lists:

nbpython-dev@netbeans.org
nbpython-issues@netbeans.org
nbpython-commits@netbeans.org
nbpython@netbeans.org

My Nikon D3 Report

I’ve been dithering back and forth about writing this, but Duncan’s recent posts about his new D700, as well as several camera discussions that I had at ApacheCon have pushed me over the edge.

Back in April I bought a new camera. When I got my first digital SLR back in 2005, I was just getting (back) into photography, and I had no idea if was going to really take to it or whether I would be any good. As a result, I went for the best cheapest camera that you could get at the time, which was Canon’s Digital Rebel XT. That camera served me well, but thanks to the digital format, I’ve been getting better at a pretty decent rate, and I was starting to run into areas where the camera was interfering with my ability to get the shots that I wanted. I knew that a new camera was not going to bump my work up a huge amount, but I was starting to get frustrated with it. It also wasn’t a smart idea for me to play with a Nikon D300 at one of the Seattle Flickr Meetups.

If I was going to upgrade cameras, I was also probably going to go full frame, because I like very shallow depth of field shots, and the possibilities for thin depth of field are better on full frame. This presented a problem. I only had one really good lens in my Canon set, the 17-55mm EF-S lens. The Canon EF-S lenses are unusable on the Canon full frame cameras, which basically meant that I would have to start over in terms of good lenses. Since I was going to have to start over, it only made sense to look at all the cameras in the marketplace.

When I did that, I was really impressed with the ergonomics of the Nikon cameras, so I started really looking at them much more seriously. Nikon has been been very aggressive about improving their cameras. This is in contrast to Canon, which had not dramatically improved the 20D/30D/40D series, had yet to announce the 5DMk2 and which has had very public problems with the top of the line 1DMk3. Back in April, there was only a single full frame Nikon camera, the D3. So after a bunch of deep breaths, that’s what I decided to buy.

Likes

So far, I am really happy with the camera. It is sensitive to light in a way that goes even beyond what my eyes normally see (unless I am really careful). As an example, at a recent Flickr get together, I took a shot of some chairs up against a red wall. When I looked at the picture on the back of the camera, there was a definite gradient in the lighting, but neither I nor several people that I asked saw the gradient without the benefit of the D3 picture.

Brews and Views at Crimson C

When I was in Prague back in July, I was able to take a number of night shots completely hand held. Ordinarily, I would have had to use a tripod for many of these, but these shots are more than passable for handheld.

Prague, The Czech Republic

As far as image quality goes, I am very happy. I am quite satisfied with the sharpness and color rendition of D3 images. The Nikon white balance does a pretty good job, better than the XT’s auto white balance, but of course, that’s not a very fair comparison at all. The biggest thing that I’ve run into is that the exposure really needs to be spot on, because the camera is so sensitive to light, that it is easy to blow out highlights. High ISO performance is really good, and when the noise starts to creep in, it looks much less objectionable than the noise that I’m used to on the Canon sensors. I shoot entirely in RAW, and I’ve brought back both badly over and underexposed shots in postproduction (I’m pushing myself to shoot fully manual as much as possible, and sometimes I forget to adjust). One drawback is that the 12MP RAW files take up around 12-13MB. The D3 eats CF cards very quickly, and I rarely shoot in one of the burst modes. This translates into demands for more hard disk space and bandwidth, and ultimately ends up pushing the computer harder, as Lightroom and Photoshop have to work harder to get all that data into memory and then do all the image processing operations. A Mac Pro is definitely in my future for these reasons, and I don’t even want to think about what this means for people shooting the Canon 5dMk2, 1DsMK3, or the Sony A900, at 20+MP resolutions.

The build quality is fantastic. Everything is solid and well crafted. Even though my hands probably on the smaller side, I find that the camera fits my hands well, and that the camera is well balanced, even with a 70-200 zoom lens mounted on the front. My hands fall naturally on the command dials and the autofocus point selector, whether I am using the main controls or the vertical controls on the grip. All the build quality means extra weight, which is taking a little getting used to, but it is good exercise.

It’s taken me a little while to get used to all the controls, but I much prefer Nikon’s system of using buttons in combination with dials as opposed to forcing me to take the camera from my eye in order to change things in a menu. I’m finding that I’m very quick to make adjustments, with one exception. I still haven’t quite gotten the hang of switching metering modes without taking the camera down from my eye. The control for this is up on the prism housing, and requires a decent amount of force to switch. In some low light situations, I tend to switch back and forth between evaluative (matrix) and spot metering modes, so this is an inconvenience. The alternative is for me to spend more time shooting in manual mode and learning to compensate for how the meter behaves, which is probably a good skill to be developing anyhow.

The autofocus system has performed really well. I’ve been able to accurately track fast moving action, and even in fully automatic mode, the camera finds the correct focus point a high percentage of the time. For portrait work, I still switch to a mode where I can select the focus point, because I just want the extra level of control. The only thing that I have noticed is that under some very low light situations, the camera can take a little while to lock. It’s still not entirely predictable to me when this is likely to happen. I love the huge viewfinder on the D3. I frequently had shots where I framed carefully in the camera, only to have extra stuff creep into the picture due to incomplete coverage in the viewfinder. I like this more than I expected to.

It seems to be conventional wisdom that Nikon’s TTL flash system is better than Canon’s, and based on my experience I’d have to agree. For the most part, I am an off camera lighting guy, but there are some situations, like parties and wedding receptions, where you just don’t have the time to make the adjustments for manual lighting. So far, I’ve found that the iTTL system works better than the Canon system. The D3′s high ISO performance adds to this by allowing you to shoot bounce flash pictures in rooms with much higher ceilings than previously possible.

There are lots of smaller things to like. The battery for the D3 lasts forever. I shot three ballet performances in one weekend, using a big image stabilized zoom in continuous focus mode, on a single battery charge. There was plenty of charge remaining. The other thing that I like is the dual Compact Flash card slots. I like the flexibility of using two smaller cards and overflowing from one slot to another. It means less worrying about managing cards in high volume shooting situations.

Dislikes

There’s one major thing that I have found that I dislike. On Canon’s you can switch between Aperture Priority and Manual modes, and have two different aperture settings. This makes it easy to have a set of manuals setting for strobes, and then flip to Aperture Priority for natural light. On the Nikon, whatever Aperture you set, will be the same for both modes. So if you are at f/8, 1/250th in Manual, but need to shoot at f/2.8 in Aperture Priority, when you switch from Manual to Aperture Priority, you’ll need to also switch the Aperture to f/2.8. And when you go back to manual, you need to go back to f/8. I can understand why it’s designed this way, but for the way that I use the camera, it’s something that I miss from the Canon.

Another dislike (well, I don’t mind it that much) is that the shutter snap on the D3 is pretty loud. Back in May I spent a day shooting with some wedding photographers. Almost all of them were shooting Canon 5D’s, and a few people exclaimed over how loud the shutter was. I guess that Kevlar shutter is going to last.

The D3/D700 start at ISO200. You can get down to an ISO 100 equivalent, but its an extension. If you are outside trying to knock down the Sun with strobes, ISO200 is one stop higher than you want to be. This is one area where the 5D/5Dmk2 have a nice advantage — you can get down to ISO50, 2 stops better. I guess you can always bring more watt/seconds, but it’s kind of a pain. Or you could use something like RadioPoppers to get your sync speed up several stops.

It’s annoying that Nikon didn’t put the dust shaker from the D700 into the D3. I’ve learned to clean the sensor myself, but it is annoying. On the other hand, even people with dust shakers need to have their cameras cleaned periodically, so maybe it’s just not that big a deal.

Some shots

This is an awesome camera – so if you see bad shots from me, you know it can’t possibly be the equipment.

JavaOne 2008

Fort Worden OSP Trash The Dress Shoot

Bainbridge Ballet Recital 2008

Bainbridge Ballet Recital 2008

EuroPython 2008
OSCON 2008
PyCon UK 2008
ApacheCon US 2008

Olio – a web 2.0 benchmark?

So there’s a new project in the Apache Incubator called Olio. It’s a “toolkit that can be used to evaluate the suitability, functionality and performance of web technologies”. There are already implementations in Java, PHP, and Ruby on Rails. Here are some additional versions that I would like to see:

  • Django (Python)
  • Turbogears (Python)
  • Lift (Scala)
  • Seaside (Smalltalk)
  • Erlang
  • Clojure
  • Haskell
  • Common Lisp
  • ASP.NET

If I’ve left out your favorite language, framework, feel free to add it in the comments. Or better yet, show up to the project with an implementation. Wide Finder this ain’t, but the results could still be pretty interesting.

My personal story on the Sun Storage 7000 series

Today Sun is announcing a new line of storage appliances. I haven’t been involved with this product at all, but I do have a personal angle on them. Shortly after I joined Sun this year, I took a trip to California to meet various people in person. Amongst the people that I met up with were Bryan Cantrill, Mike Shapiro, and Adam Leventhal, the inventors of DTrace. I was a graduate student at Brown when Bryan and Mike were undergraduates. I was mostly interested in talking to them about DTrace, because DTrace is an important part of your toolkit if you are building web applications using dynamic (and other languages).

During a break, Mike took me aside and asked if anybody had shown me what they were working on. I said that they hadn’t, so he took me back into a server room and showed me a prototype of the Sun Storage 7000 product. There’s lots to write about regarding this project, and there will be a veritable storm of blog posts about it today. The two things that stood out to me when I saw the prototype were:

a) the innovative use of flash memory as part of the storage hierarchy, and the work that has been done to ZFS in order to manage flash in an intelligent way. If you are interested in the science/engeineering behind this, you should look at Adam Leventhal’s article in the first issue of the revamped CACM on this topic. The impact on both cost and performance is very impressive.

b) the AJAX based UI that the Fishworks team has created for interacting with monitoring tools like DTrace.

You can read Mike and Bryan‘s blogs for the full story of how the products came to be, and you can go to the launch and product pages for the details the specific product offerings.

Well done Mike, Bryan, and team!

ApacheCon US 2008

ApacheCon US has come and gone for 2008, and here’s the roundup.

Talks

I’ve known Simon Phipps since we worked together at IBM in the late 1990′s. He’s always been great at finding and articulating trends in the computer industry, and I had read his post on the Adoption Led market. I expected him to talk about just that one issue, but it turned out that he had five points to make. Besides the adoption led characterization of open source, he talked about substitutability as preferable to interoperability, with a corollary of standards body reform. He also made the point that effective marketing messages for open source will be either first or second derivatives of freedom. I’m still doing some pondering on some of these.

Kevin Crowston from Syracuse University has been coming to ApacheCons for five years now, and I was very interested in his talk summarizing some of the things that he and his students have learned. I had a very extended in person conversation with Kevin, and it’s hard for me to remember what content came from where. Kevin and his students have really become a part of the ApacheCon community – I always look forward to talking to them and seeing what they are finding interesting. As I’ve written in other posts, the social/cultural/organizational lessons of open source development will likely turn out to be more enduring than much of the actual software.

ApacheCon US 2008

It’s fashionable for some company to be controlling the ASF. Every year it is a different company. Over the years this honor has been held by IBM, Google, and Joost. This year, I would say that the honor belongs to Yahoo. There was an entire track dedicated to Hadoop, which originated with Yahoo. Many of the Hadoop sessions were jam packed, so Hadoop contributed substantially to the attendance at ApacheCon this year. I predict that nextt year’s Hadoop Camp will be double or even triple the size of this years’ track. The talk that I got the most out of was Dhruba Borthakur’s description of the way that they use Hadoop at Facebook.

The conference

I think that I’ve only missed a single US ApacheCon (I’ve never made it to an EU ApacheCon) since 2000. This year, there were some changes to the conference structure. In addition to the Hadoop Camp track, there was also a track for OFBiz. I didn’t attend any of the talks, but from talking to people, it seems that the OFBiz track also contributed heavily to the attendance. ApacheCon’s program is determined pretty much by who submits and that can lead to a very haphazard program. I think that the success of two well planned coherent tracks is something that should influence the direction of the conference. The conference needs whole or half day tracks which are technically coherent and which accommodate the skill levels of the various Apache user bases.

ApacheCon US 2008

Another experiment was the decision to run a BarCamp during the second day of the Hackathon. I think that this was very successful. We had a very good discussion on git versus subversion, a topic that is generating a lot of interested within the foundation. There was a session on dynamic languages, which wasn’t very successful because the organizer ran through a long slide deck rather than engaging the participants. I ran a session on digital photography, and the people who participated seemed to get some value out of it. I would love to see a BarCamp, Open Space, or similar track be included in the main conference, rather than being limited to just the Hackathon.

ApacheCon US 2008

The lightning talk session is always well attended. The ApacheCon lightning talks are decidedly biased towards the light hearted (like Aaron Farr’s hilarious portrayal of Apache world domination), in contrast to the PyCon lightning talks, which tend to be more serious. I think that the Python community is gettting more bang for the buck when it comes to the lightning talks. There’s much more dissemination of information that might be hard to get accepted as a conference talk, either because it’s a tidbit, or it’s very new work, for example. The demand is so great that there are multiple days of lightning talks – a sign that something is working really well.

There were two other unusual features of ApacheCon this year. The first was that it took place during this year’s historic U.S. election. I’ve never watched election returns much, and never with a group. The whole experience of sitting in the Sheraton bar/lobby and watching the events on a huge TV screen is one of those memories of a lifetime.

ApacheCon US 2008

The other unusual happening took place at the end of the lightning talks, where a few of the sponsors got together and organized a “death of proprietary software parade”, complete with brass band and a police motorcycle escort as we marched through a few New Orleans streets. The parade’s final destination was a local club for the ensuing “celebration”.

I really love ApacheCon, but it was impossible for me to ignore the difference between this year’s ApacheCon and PyCon. At over 1000 people, PyCon was easily twice the size of ApacheCon, and the level of energy just seemed higher at PyCon. I also continue to be impressed by the PyCon sprints, which are just blowing things off the map in terms of attendance, productivity, and ability to engage completely new people in the various Python projects. From talking to a number of people on the ASF ConCom, I know that some changes are in the works, and I am hopeful that next year’s ApacheCon will be revitalized and energized.

Photography

I talked with people more about photography this time than any other conference that I’ve been too. Several people told me that my previous photo coverage had made it possible for them to “attend” ApacheCon even though they didn’t physically attend. It’s gratifying to feel that the picture making is doing some good beyond indulging my need for an artistic release. Beyond the aforementioned BarCamp, I had a number of gear related discussions with people, and on the basis of those, you’ll probably be seeing some gear postings from me in the next little while.

ApacheCon US 2008

Shane Curcuru, the conference lead made it a point to give me a spare key to very nice suite that the hotel gave him. It was up on the 48th floor, and you could get a pretty decent view of New Orleans, which made it pretty hard to turn that offer down. I took two trips up there and shot a bunch of frames to turn into panoramas. That was a relatively new experience, and I didn’t pack a tripod, so the result is somewhat less than optimal, but it was fun nonetheless.

As usual there is a set of photos for the conference on Flickr.

Travel

ApacheCon US 2008

I had a pretty smooth travel experience – the “worst” part was missing a ferry because of a delayed leg. This was my first time to New Orleans, so I’m not sure how to judge it. All the meals that I had were fantastic, right down the the Jambalaya that I got from one of the airport restaurants. I’ve been messing around with BrightKite on my last few trips, wondering if it would really amount to much. Shortly after touching down in Denver on the way home, I got a phone call, and several directed Tweets. That seems like some thing useful, so I’m probably going to keep playing with it. I don’t have any more trips scheduled for the rest of 2008, so that experiment will have to pick up sometime next year.

Next year ApacheCon is in Oakland, where we’ll be celebrating the 10th birthday of the ASF. That’s one birthday party that I am really looking forward to.