<Software Cleanup and Review FAQ

Redforest Enterprises
Technical Services
Software Cleanup and Review - FAQ

Q: What is software cleanup and review?

It is the software equivalent of copy editing, applying the final polish on the work. We go through your software, and do all of the things to the code base that you mean to do, but never get around to. We make sure that the formatting is consistent to the specified style. We create manpage-style headers for all of the functions. We make sure that the code is adequately commented with correct and up-to-date information. While we are at it, we review the code checking for bugs, and may even suggest architectural changes. We also check, or create the external documentation of the software such as design documents. Or we can do a subset, or superset, of these functions.

Q: What are the benefits?

A thorough code inspection can find a significant percentage of bugs. The process of making sure that the comments and the code match will find errors in both the comments and the code.

Making sure that the code is well, and correctly, documented makes it a lot easier, and quicker for new employees to learn the code base, making them productive sooner. It also reduces the potential for errors during code maintanance, bug fixes and enhancements, after either the original authors have forgotten details of the code, or when it is being modified by someone new to the code.

Q: Will you make me look bad?

Many people worry that hiring someone to clean up their code will make them look bad. If they had done the job right in the first place then it wouldn't need to be fixed. However, instead of looking bad, hiring someone to perform this service is a sign of professionalism.

Just like many busy professionals hire housekeepers to keep their home tidy because they are too busy doing tasks which they can't delegate, we can come in and make your codebase tidy, while you concentrate on the creative and strategic aspects of the job.

Engineering is about compromise and optimization. Sometimes it is more important to have something that works, today, rather than something that works perfectly a week, month, or a year from now. There is a lot of work that goes into making sure that the code base is consistent and well documented, things will come up that keep you from having the time to do all of that work, rather than adding the new feature that the customer needs to have by tomorrow.

Schedule pressures never seem to allow the time to go back and clean up all of these quick fixes, or additions. Besides, if the code seems to work, cleaning it up doesn't seem nearly as important, until someone comes in and tries to learn it, or fix that one little bug, without causing four other large bugs.

We will do the drudge work for you, we'll go through the code, and clean up all of the little details, and track down the inconsistancies, give you the time to concentrate on the strategic issues that only you can.

Just as tools like lint, and purify have their place, there are some tasks that take human intelligence, and for those, we are the best tool for the job.

By helping you get the code clean, consistent and maintainable, we'll improve your projects reliability and maintainability, while reducing the length of time that it takes new programmers to become productive, while minimizing the effect on your schedule for developing new features and new products.

Q: What is the actual process?

When Redforest Enterprises does a cleanup and review of your codebase, we come in and clean up all of the details that your permanent staff is too busy to handle. We will read any external documentation available, depending upon the availability of the engineers, we may meet with them to be taught directly, or we may learn the software in the same manner of all new hires, when we go through the code line by line.

The first documents that we will usually prepare, or repair, are the specifications and the design documents. Once we have these, will will have your staff review them for accuracy. It is expected that this process will stimulate the memory of your staff, helping them to remember many previously forgotten details, leading to another revision or two of the specification and design documents.

The next step is the actual copyediting of the source code. Once a company standard style is defined, we can use a pretty print program to make a first pass at reformatting the software to the standard. The next step is to insert, or rationalize a header for each function. These headers will be delimited in a manner such that they can easily be extracted from the source code, and will typically have all of the information that would be found in a man page on that function, (parameters, output, side effects) as well as a brief description of how the function works.

The next phase is cleaning up the formatting of the source code, as no automated pretty-print software does a perfect job. At the same time, we will make sure that the comments reflect the code, and as well as we can, we will try to make sure that the code does what it is supposed to. At this point, we may be able to propose ways of rewriting the code so that it is either more efficient or robust.

If desired, we can propose, or add, compiler switched code, that is not in the existing code base, that would aid in the testing and debugging of the software.

Finally, the finished product will be reviewed by the permanent staff, to confirm the accuracy of our observations and to evaluate the proposed changes.

In reality, these steps may happen in different order, or even, effectively all at once, depending upon the project being copy edited, and the availability of the permanent staff.

Q: What other services can be provided?

We can train your employees in the principles of a workable software quality process, providing longterm benefits after the initial cleanup and review has been completed. As experienced programmers, we can provide traditional consulting services, especially after we're familiar with your code base.

Being experienced engineers, we can also help with other temporary staffing needs. Once we've done a cleanup and review of your software, we'll be quite familiar with it and won't have nearly the learning curve that someone off the street would. What we do will definitely help someone new learn the code, but that's never as fast as for someone who already knows it.

Q: Can't it be done with automated tools?

Pretty printers exist, but the output still needs to be massaged. The formatting from them is never perfect, and there are always lines of code that are long enough that splitting them up so that readability is enhanced rather than hampered takes a human touch.

We do a lot of the bulk work with a pretty printer, and finish the reformatting by hand. Checking, writing, or fixing the documentation is something that needs to be done by hand. It is also something that needs to be done by an experienced programmer. Likewise a code review is something that needs to be done by a human being.

There are many other automated tools that have been, or could be written to assist in these tasks and we will use them as appropriate, and write them when practical.

Q: Isn't it just tech writing? Can't we just use our in-house technical writers?

Technical writing is only part of the job. Just as some programmers can write clear documentation, there are many reasons why you have that job done by technical writers. Most technical writers don't have the programming skills to read, reformant and review the source code.

Q: Are you fluent in our programming language?

We have experience in several languages. Fortunately reading software doesn't require the in-depth knowledge that writing it does. If your software is in a language that we have less experience in, we will be a bit slower initially, but we come up to speed fairly quickly.

Q: Why not do it in house?

I've got a bunch of those projects around my house too. The ones that I really ought to do, but I always find something else to do instead. Depending upon your current workload, a lot of the actual work can end up being done by your own engineering staff, work that would be put off indefinitely without the impetus of someone else looking at the source. Part of it is the same effect that makes it a lot easier to get motivated to clean your house when you are expecting company. Part of it is the impetus to get things done that is provided by a hard deadline.

There is also the advantage that by having us do this work, it frees up your staff to do the creative work such as adding new features, minimizing the impact that this work has on the overall schedule.

Q: What about intellectual property? Can we trust an outside consultant to look at all of our sourcecode?

Any employee, permanent, or contractor, would have to go through most of this process anyways, and would learn substantially the same things. The primary difference is that after we're done, you have the necessary documentation to make it quicker and easier for anyone who comes along.

Q: Is this a one-time thing, or is it ongoing?

You can think of it as like hiring someone to clean up a very messy yard. Once everything has been cleaned up, it's a lot easeier to just keep it clean. Even so, some people prefer to hire gardeners rather than doing the unpleasant yardwork themselves. So the answer is that it depends upon you and how you want to deal with it.

Q: If it ain't broke, why fix it?

Are you sure it isn't broken? How much time does it take a new hire to learn the code base? How much time is being spent writing code that duplicates existing code because programmers can't find an existing function that does what they need, or if they do find it, they don't use it because they don't know what side effects there are? How many bugs are lurking in the code that would be found in a code review?

Q: Why bother reformatting the code?

The first reason is that software that changes formatting styles from one file to the other is very hard to read. You get used to looking for things in certain places. If the formatting changes between files, that therefore makes it easier to make stupid mistakes, because you are looking for things in the wrong places.

Q: Won't this introduce a bunch of errors?

Since the compiler does not "see" comments, nor the formatting, most of our work won't actually make any changes. When we find what we think is a bug, rather than correcting it, we will mark it with an easily findable tag so that the programmer responsible for the code can make the appropriate fixes.

Q: Who defines the formatting style of the software?

It generally works best if a few of the senior programmers sit down, and hammer it out amongst themselves. Once they have a document, they can propose that to the rest of the programmers and take feedback. No matter what decisions are made, there will inevitably be people unhappy with the standard. The important thing is almost never what the standard style is defined as being, but that everybody uses the same style.

Q: What if some of my programmers don't like the standardized style?

It is an unfortunate fact that almost everyone will be unhappy with some portion of it. The "One True Style" is a religious issue which can almost never be resolved. If you have a programmer who adamantly refuses to change to the standardized style, that just makes a bit more work for whoever does the cleanup and review.

Q: Will all this revising put a big load on our QA people to test your changes?

Very few of the changes will be functional changes, and so if any bugs are introduced they should be caught in regression testing.

Q: After you've cleaned up the source code, will I need to make a new release of the cleaned-up version to get the benefit of your cleanup, even if the existing version seems to be working in the field?

In order to reap the benefits of the bug fixes, it will be necessary to do a new release, but realistically, they can be one as part of the normal release cycle. Most of the benefits of this service will be from increasing the efficiency of your programmers, especially the new hires.

Q: How does this relate to Common Criteria?

It is roughly analogous to EAL 5. Since each job is tailored to the customer's requirements, other levels could be targeted.

Redforest Enterprises * Larry Colen's Resume
Software Cleanup And Review - Description * Software Cleanup And Review - FAQ
Writing debuggable code

Larry Colen
lrc@red4est.com
Copyright (C) 2005 Larry Colen
Most recently modified by lrc at Tue Jul 19 12:31:20 PDT 2005