state_machine 1.0: Jump for joy!

by aaron

Today I’m proud to announce that state_machine 1.0 has been officially released. This release marks the end of a long journey to build a library that brings the simplicity and power of state machines to Ruby and its major ORM frameworks. In case you missed it, we’re celebrating the release tonight at the Asgard Irish Pub & Restaurant in Cambridge, MA.

Individually, we are one drop. Together, we are an ocean.

In past announcements, I’ve always ended with a special thank you to everyone who contributed to the release. Today, however, I feel it’s important to highlight this project’s group of contributors first, not by name but as a whole. An enormous number of individuals (over 100!) have, directly or indirectly, contributed toward this effort. Moreover, many open-source projects have inspired several of the features now found in state_machine.

I can’t possibly express the gratitude we all have for everyone who has been involved.

A retrospective: Taking a look back

While much of what I’ve contributed to the community has come in the form of code, I thought it might be useful to take a few moments to reflect on what I’ve learned, as the core contributor of this project and a member of the Ruby community. If any of the following seems like a regurgitation of the same things other folks have spoken about before me, consider this at least a testimony to those experiences.

Fragmentation can lead to unity

Like most open-source projects, state_machine was initially built out of a need for specific functionality that was not offered at the time nor could be easily extended onto existing libraries. When it became a part of the PluginAWeek project, it was just a budding seed of the library it is today. At that time, there was a significant fragmentation of features both between the various Ruby state machine projects and within their own forks. They all had similar goals, but marched to a different beat.

To better understand the fragmentation of these libraries meant reading through Rubyforge / Github projects, project forks, and community forums to find out what features were important and how they were used. This research proved invaluable as it provided the information needed to unite of all of these fragmented ideas and assemble them under a single vision of how they could work together.

Success means sharing a single vision with an open mind

As I began to realize the potential for the project, I had a single, simple vision: to build a library that brought an array of complex state machine features into a simple, united interface and that could be officially endorsed by the major Ruby ORMs.

The former part of that vision is something that every project with even a little popularity will encounter with difficulty. As state_machine began to grow, so did the number of use cases and features that its users wanted to have. However, many of these suggestions were made with only a partial view of the world. My job was to filter through those and find out how they fit into the single vision I had for state_machine.

I discovered during this process that it was critical to address every problem or feature that a user reported. If I could not suggest the right way to address the issue, then there was a flaw in the library. That could sometimes mean spending hours, days, or weeks thinking about an issue before coming up with the right solution. Sometimes the solution was simply an explanation of why things work the way they do… other times it meant building a new feature that directly or indirectly solved the issue.

While I understand that being the single gatekeeper to this project slowed its progress significantly, I also attribute much of its success to the fact that every single piece of functionality that was added was rigorously scrutinized and made sense with the whole vision in mind.

Setting standards sets expectations

I mentioned earlier that one of the goals for this project was to build it in such a way that it could be endorsed by the major Ruby ORMs. The primary reason for this was so that the community could be provided with a standard interface for building state machines which was consistent across ORMs, similar to how the i18n or activemodel libraries have been embraced.

To that end, I set four critical standards for this project:

  1. Every feature must be implemented to the highest degree of quality, both from a functional and design perspective
  2. Every change must be backed by well-written unit tests to ensure functional quality
  3. A significant amount of documentation must be provided for every feature, whether part of the public or private API
  4. Support must be provided, at the very least, in the earlier stages of the project

Everyone’s encountered their fair share of libraries that have failed to provide at least one of the above standards. It can be a frustrating, and unnecessary, experience. You can choose to control the expectations for your project if you set simple standards like this.

Not all ORMs are created equal

It was about two and a half years ago when I made the decision to rewrite state_machine with the ability to be used within a simple Ruby class or within an ORM library like ActiveRecord or DataMapper. Since then, integration support has been added for ActiveModel, Mongoid, MongoMapper, and Sequel. The upshot of having all of these integrations is twofold: new integrations can reveal flaws in the design of your project (such as a lack of interface hooks) and they can open up a bigger audience of users. The downside is that not only are these projects moving at a rapid rate of development (which means constantly keeping up with all of them), but they vary significantly in ease of integration.

Reading through the state_machine integration for each of these ORMs can reveal a great deal about their design for extensibility. First, I have to commend the Rails folks for building ActiveModel and helping the community move to a standardized interface. This has been a huge success (as I’m sure it has with others) in helping to simplify integrations. Second, John Nunemaker and the MongoMapper developers deserve an extra mention for building the easiest library to extend. ActiveRecord and Mongoid were not much harder, but they have room to improve. (addendum: I accidentally confused the Mongoid and MongoMapper integrations the first time. I’ve since updated to indicate that MongoMapper has been the easiest)

With praise, however comes criticism (in the constructive way, of course). Of all the integrations, I found Sequel to be the most frustrating library to integrate with. It lacks many of the hooks found in other libraries and will likely be the most difficult to maintain moving forward. (addendum: Please don’t take this criticism the wrong way… I purposefully didn’t go into details here. We’ll have to work together to improve things… that’s how these things go :))

Moving targets need to be managed

Every project moves at its own pace and, at least in the post-GitHub world, that pace can be at a breakneck speed. As ORMs make significant changes to their public APIs, updates often need to be made to the associated state_machine integration as a result. Supporting multiple versions of a library can be an minor inconvenience; supporting multiple versions of multiple libraries can be a major problem.

The only way to keep up with moving targets like this is to provide a framework for managing it. In addition, it should ideally be done in a way that keeps it simple for developers and allows them to continue to get new features even for older ORM integrations. By designing each version to act like an integration itself, this allowed for new features to be added and version-specific issues to be fixed with ease even as the ORMs continued rapid development.

Open-source projects require good management… and lots of time

Open-sourcing a small library you whipped up over the weekend or while building a feature at work is easy. Managing that project as it begins to get followers and takes on a grander vision can be difficult. More than anything, I’ve discovered that the hardest part is finding the time to work on it. Granted this is the beauty of open-source, and GitHub in particular, where the masses can help build new features and fix bugs… and all I have to hit is the one-click merge button. However, it’s not always that simple.

As I mentioned earlier, a project like state_machine can require a much more hands-on management style where every change is assessed for consistency, compatibility, quality, and more. After you take into account acting as the support contact, being a core developer, and more, the time adds up.

Despite the difficulties associated with managing a project like this, it’s been completely worth it. When developing software is a hobby, it should be fun… and it is.

Thank you

I’m sure there are other things we could talk about, but I feel like I’ve yapped for long enough. As a Ruby / Rails engineer, I’ve been fortunate enough to have been able to be in a job where so much of the software we take advantage of every day was built and open-sourced by our fellow engineers. I can think of no better way to pay those gestures forward than by making my own contribution back to the community.

Oh, and before I forget… hitting 1.0 wasn’t the only milestone we hit this week. As of Monday, state_machine crossed the 1,000 followers mark over at GitHub thanks to Patrick Klingemann and the 999 people before him. Awesome.

As always, be sure to follow the project on GitHub! Happy state machining!

Comments

9 Responses to “state_machine 1.0: Jump for joy!”

  1. cremes on May 12th, 2011 10:13 am

    Congratulations on hitting this milestone. I have used this project off and on for a few years. I haven’t had a need for it lately (last 15 months or so) but a rewrite of some code looks like it could be improved via a FSM so I’ll be looking at this project again.

    Thanks for sticking with it and incrementally improving it for *years*. It’s rare to see that kind of perseverance in a Ruby opensource project from a single developer.

  2. good open source takes time | Bibliographic Wilderness on May 12th, 2011 3:15 pm

    [...] The developer of the ruby state_machine library (whose name I think might be Aaron Pfeiffer) talks about some of the challenges and reasons for success he and the other developers had in making… [...]

  3. Philippe Creux on May 12th, 2011 5:10 pm

    Congratulations and thanks for this awesome project.

  4. John W Higgins on May 13th, 2011 11:02 am

    Re Sequel support

    While I’m not certain they meet your full needs there certainly are plenty of hooks available within Sequel

    A cursory 2 second glance at the docs shows the following list of hooks available

    :before_create, :before_update, :before_save, :before_destroy, :before_validation
    :after_initialize, :after_create, :after_update, :after_save, :after_destroy, :after_validation

    That certainly looks like at least 2 of the hooks you needed (save/validation) are fully there.

    I guess I also missed your post to the sequel mailing list looking for assistance with making it work prior to stating that something wasn’t there. I also guess Jeremy failed to respond to that request which would be totally outside the normal cycle of some of the quickest and most helpful assistance any project owner shows within the ruby world.

    Please at least try to reach out prior to trashing a project against its peers. I’m certain you would appreciate the opportunity to explain your project should a similar issue arise within it and someone was having difficult using it compared to your peers.

  5. aaron on May 13th, 2011 12:21 pm

    John -

    I’m not sure what to say. Rather than take my feedback and respond with a wild overstatement that I’m “trashing” Sequel, why not reach out to me like Jeremy did and ask for more information? In an open-source world, you have to take criticisms and appreciate them as constructive. I *explicitly* stated this in my post. I knew that someone might misinterpret my criticism so I tried to avoid that by making it clear in this post. I’m sorry that I didn’t try harder to make it clear enough.

    What I stated was my experience. Nothing more. Nothing less. I never indicated that Sequel was a terrible library. I never told folks to not use it. If I thought that, I would have never built an integration for it. I realize that Sequel is a useful library that has many followers and I chose Sequel for that very reason. As I also indicated in this post, every time someone expressed that there was an issue in state_machine, that they were having problems, or they were frustrated with it, I always took this as an indication that there was something missing, be it docs, features, or otherwise. I reached out to those individuals to find out how to improve the library and never took feedback as offensive.

    If you’re offended by my frustration, then I apologize. I’m happy to discuss what was frustrating out-of-band, but I have no interest working with individuals (or projects for that matter) who won’t take constructive criticisms.

    P.S. I just want to be clear that I purposefully did not go into the details of why I found it easier to integrate certain ORMs than others because my point was to keep things at a high-level and not get off track from the goal of this post.

  6. John W Higgins on May 13th, 2011 1:12 pm

    My apologies for the use of the word “trashed” it was harsh and a very poor choice of words on my part.

    I guess I just don’t understand why you wouldn’t have asked first. I’ve got no problem with criticism (constructive or otherwise), I just don’t think it’s fair to have not asked for assistance first. What’s the point of having a support mechanism if no one will use it?

    If a project met all 4 of your basic requirements for a project and someone says “project X doesn’t do this or that”. Yet there are instructions all over the docs and message after message in the project mailing list on how to do it, and there are specs directly addressing the issue, does that still qualify as constructive criticism of a project? I’m not saying any of the above exist for your issue but there still is a point where the criticism fails to be valid because of a lack of some effort by the end user to work through the problem via support mechanisms.

  7. aaron on May 13th, 2011 1:42 pm

    John -

    I didn’t go into the details because that was not the point of this post. I’ve been through the internals of Sequel inside and out. I understand what callbacks are there. The specifics are that there are not proper around_save and around_validation callbacks. I need those to integrate. Sequel doesn’t have them. I can’t use before_* and after_* callbacks. They don’t provide the means to easily integrate with state_machine. The lack of around_* callbacks was an issue prior to Rails 3 and made it difficult to integrate ActiveRecord / MongoMapper / etc. I know I could have said this in the first place. However, again – I didn’t want to make code specifics a part of this post.

    Your assumption that my criticism isn’t valid is because we haven’t spoken out of band about the specifics. I know developers appreciate when a user reaches out. I’ve done that in the past for DataMapper and others have done it in the past for state_machine. You’re right that I haven’t asked for around callbacks in Sequel yet. That doesn’t take away from the fact that the criticism can still be valid.

    However, isn’t blogging just another way of reaching out and getting the right folks on the case? That was never my intention, but if that’s what the result is, then I think that’s awesome of the Sequel community. Yourself and Jeremy have been proactive about finding out why it was difficult to integrate and that, to me, is the best type of support. I’m always watching blogs, forums, and websites for any problems people have with state_machine in the hopes that I can reach out, find out what the problem is, and fix it.

    Let’s put aside our differences, embrace the feedback, and now work together to improve things.

  8. Mike Mondragon on May 13th, 2011 7:26 pm

    Nice work, I’m using state_machine ona mongomapper based project and will be glad to change this bundler line from

    gem ‘state_machine’, :git => “git://github.com/pluginaweek/state_machine.git”

    to

    gem ‘state_machine’, ‘=1.0

  9. Henning Koch on May 15th, 2011 3:26 pm

    Congratulations to making 1.0. I’m working with state_machine every day. Thank you for an awesome library.