The art of refactoring in large systems

Refactoring is a crucial part of the development process, it gives us an opportunity to make the code easier to read, reuse, maintain and extend the system with the minimum possible expenses in the future. It is not enough to write code that simply compiles. During my practice in commercial software development of large products, I have accumulated a number of practices that I would like to share with you.

Preparation

Try to select the period when the activity of adding new features to the subsystem will be minimal. For example, it is convenient to concentrate on the backend, while the team focuses on the frontend. Read the code well and see why you need it at all. What architecture is the basis of the subsystem? What standards or best practices were used? Clearly define for yourself what the new concept is and what exactly you want to change, because there must be a measurable final goal.

Determine the area of code where most changes will be made. If possible, isolate it into a separate module/directory. This will be very useful in the future.Refactoring without tests is very dangerous. You should have at least unit tests. Run them with the coverage calculation, it will give you a lot of information to think about. Fix the broken tests that relate to the subsystem you are looking for.

Analyzing information about method coverage, you can find and delete unused code. Strange as it may seem, this often happens up to 10-15%. The more code you manage to remove, the less refactoring it is. By coverage, determine which parts of the code are not covered. It is very important to add the missing tests. If it takes a lot of time and effort to write tests, write at least high-level smoke tests. Try to bring the coverage up to 80-90% of the meaningful code. Do not try to cover everything, it is not worth the time spent.

Let us start

Wrap your subsystem around the interface and translate all external code to this interface. This not only forces good programming practices but also simplifies refactoring. Make sure that your tests work with the interface, not the implementation. Make it possible to specify which implementation of this interface to use when starting. This feature should be supported in both tests and production.

Write down the revision of the version control at the very beginning of refactoring. From now on, every commit to your old subsystem is your enemy. Now you can write a new implementation of your subsystem interface. Sometimes you can start from scratch, sometimes you can use the favoured method of copy & paste. Periodically run tests on the new implementation. Your goal is to make all the tests run successfully on the new and old implementation.

Do not postpone commit to the repository to the very end. Push the code into the repository as often as possible, leaving the old implementation enabled. If you keep the code in local branches for a long time, it will inevitably lead to merge conflicts, because other people will refactor the modules you use. Believe me: simply renaming a method can cause you a lot of problems.

Remember to write tests specific to the new implementation, if any. After that, all the tests should be done with both the old and the new subsystem. As soon as you are convinced of the stability of the new version of your module - block commits into the old subsystem. Try to minimize the time of existence of two subsystems simultaneously, as this is not only an additional cost but also a lot of additional risks.

Additional advice

All new features created in parallel with refactoring should be 100% covered by tests. Any bug fixing should be done according to the following principle: first, we write a test that reproduces the problem and then fix it. Very soon everyone will get used to this technique :)

If you have additional time and resources, make a separate build for the time of refactoring, where all tests on the new subsystem will be run. The automatic build makes your new, unused code "official". The same policies and rules should apply as to everything else.

It often happens that you do not know if you have fixed everything you wanted in the old code for the new architecture. For example, you don't know if your code uses a direct JDBC connection somewhere instead of Hibernate. Or suddenly a message slipped through REST instead of gRPC call. To detect such places, we need to think of a way to make the old business logic inefficient - to break it in tests. For example, to break the message delivery system or to slip the system a non-functioning JDBC driver. Practice shows that in this case, you can find a lot of forgotten and unrepaired places.

Do talk to other programmers, keep them informed of your progress. If they know that you have a week left, they can sometimes move their tasks to the release time of a new version of your subsystem, because merge conflict is the worst thing that can happen during refactoring.

Experience suggests that even scary and large subsystems can be brought to the proper form if you act step by step and strictly according to the instructions.