I’ve found that Scrum, and Agile methods in general, are incredibly valuable in single-man “teams” if done properly.  In my experience though, there are some sticking points that need to be worked through with how you work with your clients and manage communications.  These will have a significant impact on your projects and how successful Scrum can be for you.

Before I get too far, let me give a quick overview of Scrum:

Scrum development is a business process that seeks to speed up development, decrease bugs, and increase the quality of products that are delivered.  It doesn’t have to be focused on software development, but that’s where it has the greatest pull right now, and the use that I am going to focus on.

Scrum, at least how we use it (mixed with Extreme Programming), uses several key components:

• Iterative and Incremental Development – A product is released after each “Sprint” (typically 1-4 weeks).
• Test Driven Development – Automated unit tests are created to test the code, databases, and user interface.  For PHP, we use PHPUnit and some in-house database testing tools.  For Java and integration endpoint API tests (regardless of language), JUnit would be a good start.
• One-Touch Build, Document & Deploy – Automated build and deployment sequences are developed to reduce variables, problems and time.  Ant is a great tool for this.
• Continual Integration – The software is compiled and thoroughly tested nightly (and as needed) throughout the sprint.  CruiseControl is useful for this, although you can also do something as easy as schedule a nightly cron job to execute an ant build file.
• Source Version Control – We use a central version control system (Subversion), to manage our code.  This allows me to work from multiple computers on the same code (such as when I’m traveling), and it also allows me to work with other developers when I’m partnering or outsourcing work.
• Product Backlog – The product backlog is a list of stories and feature sets that our customers want in their software that haven’t been completed yet.  Each story or feature is then prioritized by business necessity and value to the business.  I’ve heard good things about XPlanner and Pivotal Tracker, and IDEs such as Visual Studio, Eclipse and Netbeans support backlogs.  We prefer to use Kanban boards, which are adapted from the Lean Manufacturing method.
• Sprint Backlog – The sprint backlog is the list of stories and feature sets that we will complete during a single sprint.  The items that are put in this backlog are negotiated between us and our client.  Once the items are chosen for the Sprint, they are then fed into the “Work In Progress.”  The stories and features are then fed into our web-based project management tool.  We currently use dotProject for this, but a wiki, Google Wave, or a project management application like BaseCamp or MS Project would work just as well.  The important thing is that your client must be able to read each item and make notes on them.
• Work In Progress – A series of stages are used to track the progress of work that is in progress during a sprint:
  1. Awaiting Tests
  2. Test Writing in Progress – We develop the tests before we write the code.
  3. Awaiting Code
  4. Coding in Progress – In this stage, the application is actually being built
  5. Awaiting Integration
  6. Integration in Progress – The code changes are integrated into the main development code base for this sprint, and all tests must pass before continuing to the next phase.
  7. Awaiting Documentation
  8. Documentation in Progress – The documentation for the application is updated and rebuilt.  PHPDoc is a great help for us in our PHP applications, and can extract documentation directly from our in-line documentation in the code.  Similar applications exist for many other languages.
  9. Awaiting Acceptance
  10. Done – The story or feature has proper test code coverage, all tests pass, the code has been properly integrated, documented, and signed-off by person in charge of Quality Assurance.  In our case, this is done through a demo with the client.
• Sprint Review – During the review, we reflect on and catalog all our lessons learned during the sprint.  By looking at went well and what didn’t go well, we can make changes in our process for our next sprint.  Also, the client’s likes and dislikes of the last product release are included.  We use a wiki for this, such as MediaWiki or dokuWiki.

A typical Scrum team consists of:

• Product owner – The person who is responsible for the delivery of the project to the client.  This is usually a Business Analyst or Project Manager.
• Users - The end-users, commonly the client’s staff.
• Stakeholders - Typically, these people are the client-side Project Managers and are the ones that are paying for the product.
• Scrum Master – The member of the consulting or development team that is responsible for leading the Team Members, and reports to the Product Owner.
• Team Members – The actual development team

When you are running a Micro-Scrum (such as if you are a lone consultant), the scrum master, team members and product owner are consolidated to a single person (you).  When your clients are like ours, and are almost all small businesses, your client contact ends up filling multiple roles.  The actual Scrum roles end up looking more like this:

• Scrum Master/Team/Product Owner (Consultant)
• Stakeholder/User (President, CEO, or General Manager, etc.)
• User (Client’s Staff)

By consolidating the Scrum roles, I, as the Product Owner, now don’t just need to make sure that the product is developed and delivered.  I also need to fill the roll of a Business Analyst, so I need to understand the business case, consolidate all the user’s needs and negotiate with the client to prioritize the importance of each user story.  I also have to avoid focusing on my own “pet features”, which developers can sometimes do, and instead have to focus on what will bring my client the greatest value in the shortest amount of time.

Standing in the client’s shoes, the client needs to be both the stakeholder and the end user.  As the stakeholder, the client needs to watch the business’s bottom line and help prioritize the user stories for the development team.  The stakeholder’s goal is to get the best product that meets the most critical needs within both schedule and budget.  As the end user, the client also needs to look critically at their business processes and workflows.  He or she needs to be able to understand and explain the strengths and weaknesses of their current software to the rest of the team.  The client also needs to provide his/her time and in-depth knowledge to the team, enabling them to architect and develop the software that best meets the client’s business needs.

Consolidating those roles makes things both easier and harder at the same time.  It’s a bit easier because there are fewer people that you need to communicate with.  However, it also makes things more difficult because there is more responsibility that is asked of each person, and, by its’ nature, each person has a greater impact on the success or failure of the project.

The ultimate success of a Micro-Scrum development project hinges on the ability of both the consultant and the client to be proactive, efficient and thorough communicators.  In my experience, the quality of this communication, especially early in a project, will make all the difference.

There needs to be a very clear understanding from both the client and the consultant on what is going to happen throughout the life of the project and how the product is going to be developed and delivered.  Both the consultant and client need to be transparent and honest with one-another.  Micro-Scrum works especially well if both parties work hard at building a good rapport with one another.

When both the client and consultant are on the same page and can effectively communicate, the project works very well.  When our client is not a willing communicator, delivering a product that meets their needs is a lot more difficult.

5 Hard-Learned Lessons in Effective Micro-Scrum Projects

1. In Micro-Scrum, the focus needs to be on quality, NOT on cost.

All too often, small Agile studios like mine are compared to freelancers or overseas developers on sites like Elance and Guru.  The bottom line is that we cannot complete with their hourly cost, so we don’t even try.  What we focus on is Quality, above all else.

We deliver unit-tested code, usability and performance tests, code reviews, security audits, and are actively involved in communication with our clients throughout the entire process.

It has worked well for us, and we’ve kept our current clients for nearly 2 years at full workload.  The past year has been our best year ever, despite the recession.

2. Micro-Scrum development usually costs more, initially, than methodology-less development (aka cowboy coding).

This goes along with the previous lesson.  The up-front costs are usually higher, however the bug rate is much lower and the documentation is better.  Also, the risk and time involved when refactoring and modifying the code is MUCH lower.  Software changes are anticipated, so we develop with that in mind.

3. When working with a new client, deliver early and often.

When working with non-tech-savvy businesses, keep the sprints short.  We like to do 2-3 day sprints at the beginning to get them accustomed to the process.  Even if that means just delivering them a framework, some raw code, a library or sketches, it pays off dividends over the course of the project.  It opens up communication and makes sure that our direction matches their vision.

In addition, you get to run through a deployment dry run each time.  If you notice problems in the process, you can fix them early on rather than waiting until the end and being surprised that their server doesn’t support a dependency that you built into your software.  (It may sound silly, but it happens.)

The risk with rapid iteration is that if you don’t explain why you’re doing it, your client may feel like you just don’t understand what they want.  They’re right!

The entire purpose of these short sprints at the beginning is to teach your client how to communicate with you, and to learn what your client wants by giving them something tangible that they can critique and use as a reference.  That tangible prototype can help you understand what they really want, which often is different that what their Request for Proposal or Product Specification actually says.

4. Keep the up-front requirements gathering as short as possible.

Feel free to post flame-comments below. 

I’ve run into many clients that don’t always know how to explain their product vision and be clear with what they ultimately want.  For about 1 full month back in 2008, I beat my head against a wall because one of my clients seemed to contradict their needs on a weekly basis with an accounting system I was developing for them.

I decided, out of desperation, to cut the requirements gathering short and push forward with what I had at that point.  Amazingly, I was pretty much right.  After Sprint 1, there were some modifications and changes, of course, but I was able to hit the target with only about 30% of the requirements being known up-front.  In addition, it forced us to build a very flexible architecture so that we could easily adapt to the unknowns and changes when we hit them.

What we do now is start with a very quick Sprint 0, maybe a week at most.  During Sprint 0, we build the user stories, initial specifications and product backlog.  There will be lots of unknowns and changes later on, so knowing every specific detail is neither important nor possible.  Managing and working around those unknowns is just part of software development, especially when you’re working with companies that don’t understand software development.

5. Failure is GOOD!

This may seem counter-intuitive, but failure is a good thing.  It opens the lines of communication for constructive feedback.

A goal we have for all our projects is to “fail fast.”  What that means is that we deliver a product in an early iteration (ideally Sprint 1) that our client will see lots of problems with.  Maybe the interface doesn’t meet their needs, or the work flow isn’t intuitive.  Whatever it is, I actually want my client to come back to me and say “I don’t like this.”  I then ask them “What specifically don’t you like?”  Or I’ll ask my favorite question: “Can I set up a screencast for you and you can show me what’s happening?”

We take each failure as an opportunity to open the lines of communication even further.  You can learn exactly what your client wants by watching him or her use your application.  Developing a Micro-Scrum team takes time, and building the relationship and trust through open communication is essential.

I’m sure you’ve noticed that communication is a common theme in Micro-Scrum.  Actually, it’s really the primary driver behind Scrum and any other Project Management methodology.  In order to deliver what a client wants, you need to have clear communication to truly understand their needs.  In my eyes, there is no other way to deliver a successful product.

Scrum formalizes the communications and works to get everyone, from the users and stakeholders to the developers, engaged in the success of the product.  When you are Micro-Scrumming, this communication becomes even more vital since there are fewer people involved and more responsibility on each person.  Harnessed wisely, it can lead to very successful projects that have good short-term value and outstanding long-term value.

If you’re in the Columbus, OH area and want to learn more about Agile Software Development, I highly suggest going to the Columbus Ohio Agile Alliance (COHAA) meetings and getting involved in TechLife Columbus meetup group.

Technorati Tags: agile, junit, microscrum, phpunit, scrum, submerged solutions