Java has been the tool of choice for enterprise application development for many organizations for over 10 years. We are seeing more and more applications that are aging and suffering for it. We have worked with several recent clients to breathe new life in to these legacy applications so they can evolve and grow with the business they support.
This is the first of a series of articles about improving the quality of existing software and the processes that produce it.
Why Not Rewrite
We’ve all encountered software that we would rather just rewrite than try to fix. The problem may be poor design, poor implementation, lack of tests, bugs or any of a host of other software ills. We see ahead of us the pain of working with a poor quality system and the risks to schedule and quality of end product it creates.
At Carbon Five, we have participated in many successful projects to rewrite applications for our clients. Sometimes starting with a clean slate and all the lessons learned from a previous effort is exactly the right approach. Often it is not.
In our experience the greatest obstacle to a rewrite effort is that the existing system is the only accurate record of the requirements for the system. Applications evolve over years to meet the changing needs of a organization and its users. Many small decisions are made and captured in the functioning application only. Rewriting to reproduce the features of an existing system can be a very difficult effort to define and scope.
Usually the troubled application will continue to be use while a rewrite effort is underway. Often support, maintenance and feature development will need to continue on that application even while a rewrite is under way. A rewrite will compete for development resources and will be chasing new feature development on the existing application.
Not everything in the troubled application is rotten. There are often good pieces, or at least components that are reliable and well understood. It would be great to not have to rewrite them.
How can you be sure that a rewritten application will be so much better than the one it is replacing? All too often, the source of poor software quality is poor process and practices. Unless you fix those problems it is not worth embarking on a new software development effort.
How to Rescue
Sometimes a decision to rewrite is made because rescuing an application seems too daunting. Where do you start? How long will it take? Here are some high level thoughts from our experiences with our customers. We will get in to more detail on specifics in later articles.
Set Short Term Goals
Be sure to set demonstrable short term goals with meaningful names. When your job is to make the system “better” your job is never done. You need discrete goals with names that you can finish, demonstrate, release and feel good about before moving on. Easily communicated goals help to show management that the effort is progressing successfully and help your team build confidence in an often daunting task.
On one of our projects we set a goal of having a unit test in place for a component in our service layer. “Service Test” would be a fine name for that goal. In a system with no testing support, this was a big task that once completed opened the door for much easier testing.
Don’t Let “Refactor” Become a Dirty Word
A lot of the work you will do when rescuing an application can be characterized as refactoring:
Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.
Martin Fowler, http://www.refactoring.com
Don’t let “refactor” become a dirty word. It’s too easy to say, “Yesterday I worked on refactoring our persistence layer. Today I’m going to keep working on that.” Again, come up with a name or description for the work you are doing that enables others to understand what you are doing and for you to describe progress through that task. Refactoring is one activity of many that you will engage in during this process. Avoid using “refactor” in the names of your short term goals.
Introduce Testing
A common attribute of many struggling applications is conspicuous lack of automated testing and the design and infrastructure to support it. We have found that the many changes required to support and write automated unit and integration tests are exactly the changes an application needs to get healthy. They include:
- Automating and streamlining build systems
- Decoupling application components
- Minimizing dependencies on application server features
- Building test data sets
- Continuous integration
- Building a team interest in quality
Each of these items is challenging to implement. You can’t do them all at once yet they are dependent on each other. Fortunately there are good open tools and practices available that provide a helpful bootstrap including Maven for convention-based builds, EasyMock to test code with lots of dependencies, Spring for component management outside of an application server, and CruiseControl for continuous integration (not to ignore the excellent almost-free options like TeamCity and Bamboo).
Introducing testing to a legacy application with no support for it is a daunting challenge and slow going at first. We’ve had good success in the past by first cleaning up the build and adding the ability to run tests, then finding a place in the application to break apart components and wedge a first test in. With a running test, you have something to automate so you can get continuous integration running. You also have a lot more insight into the architectural changes that will both improve the health of the application and make it easier to add more tests.
We have a lot more to say about this topic and specific experiences to share in coming articles.
A Culture of Quality
Another common attribute of a struggling application is that the team maintaining and developing the application is in continuous firefighting mode. They’ve lost hope of making fundamental improvements to their system and on a daily basis are fixing emergency bugs and hacking in features with their fingers crossed that they are not breaking something else.
Teams working in this mode collaborate poorly with product management. Product management feels that the developers are resistant to new features. Getting releases out is painful and unpredictable because defect rates are high.
These teams also collaborate poorly with each other. Developers tend to become isolated from each other as sole maintainers of one application component or another. They do not talk about how to make things better and become resigned to living with the status quo.
To be successful in a rescue mission or even a rewrite, you have to turn the firefighting culture into one where developers value quality and work for it daily. They should be excited to make things better and be engaging each other with ideas and practices to get there.
Again, this can be a very difficult effort. Sometimes it requires dramatic measures. In our experience this includes:
- Changing a workspace to remove barriers to casual conversation
- Relocating developers to the same physical location
- Hiring new blood and firing those resistant to change
- Pair programming
- Book clubs and study groups
These changes are often the most painful to make. As a consulting company Carbon Five can advocate for these changes, teach pair programming and run a study group, but the real changes have to come from within. We have seen the most success in making these changes when there is an commitment to improve from the business and champions of this effort in management.
nice article, Alon. The issues you raise are especially interesting to me at the moment because I just finished a re-write of some code written by non-professional developers. The code was written by smart people but without a lot of good design or development practices (e.g., minimal modularization and very little systematic testing).
I would add the following based on this experience:
it can be good to identify a core piece of the existing system, break it out as a module or subsystem, and re-implement it first.
Sometimes attempting to re-write an entire system is too overwhelming given the people and time available. Once a core part is rewritten, spec’d out, nicely documented in psuedo-code or whatever, and solid testing is in place, it becomes much easier to expand out and rework the remaining parts.
Being able to test the behavior of the new implementation and compare it with the existing system is really critical. The tests help people become confident in the new implementation and also expose differences between the new and old systems.
The system I worked on was somewhat gnarly and had evolved over time and there was no complete specification. It turned out to be easiest to write a compact description of the desired behavior as a side-effect of the re-implementation effort. This work also exposed a significant bug in the old system, where its actual behavior failed to satisfy one of the key high level behavioral goals.
Another point: sometimes it helps to pretty much ignore the existing implementation (especially at the code level). In this case, re-implementing from the desired behavior without looking at the existing code seemed to go much faster than attempting to rework the original system. This worked in combination with the first point: trying to rework the entire system and trying to understand the whole thing all at once was like walking thru a bog.
By the way, I’d like to put in a plug for test-driven development. Even in a situation like this where you’re re-implementing something, it really helps to first think of examples of the desired behavior, write tests, and then implement to pass the test.
Alon,
You might also want look at our Parabuild for Continuous Integration http://www.viewtier.com/products/parabuild/index.htm
Good article. Even though I rarely admit it to my development teams, there are certainly times where it makes sense to rewrite an application. The problem though is that developers are too eager to rewrite software, partly because it’s easier to write new stuff than it is to read somebody else’s code (especially somebody else’s crappy code), and partly because it’s just more fun to write new stuff than maintain old stuff. I always become super skeptical when somebody claims that we need to rewrite an app. Could be, but I really doubt it.
There is a classic article on this subject (and the thoughts above mostly parrot things I’ve read in the following):
http://www.joelonsoftware.com/articles/fog0000000069.html
Alon,
Great article. I’ve been involved in a number of these types of situations, and I have my $.02.
Pulling this up one level, one of the key factors in this decision is to convince everyone in the company that this is key strategic business decision that needs to be analyzed from many angles. Key business problems that drive this kind of evaluation often include customer satisfaction problems, customer delivery problems (especially in the case of software+service type solutions), the inability to innovate and falling behind competitors, cost of development. Talking to folks in all parts of the business (executives, sales, marketing, support and services) is critical to truly understand the key business problems that may help drive the rewrite or rescue problem. It also help ensure that its not just that the development team wants to rewrite….
I’ve found if you can demonstrate how a certain direction will alleviate these issues, you can get strong decisions made from the top that help drive the decision through the organization.
Another key factor, is that a rewrite decision is a huge risk from a business perspective as it takes a significant investment and based on the “average” success rate of software projects. That risk goes up exponentially if there is not solid buy in and commitment from all key people in the organization.
Finally, for us, in all these cases where the analysis was done, the case for rewrite made in solid business terms, getting buy in from all parties involved, and then doing all the right things in terms of doing the rewrite (good architecture, good team, good process) it was a big success and the best decision for the business.
-Kamal