Legacy Systems Are People, Too
The original Doom came out in the early 90s and it had a multiplayer component. My friend and I both had the game and we both had modems (he had a blazing fast 56k modem, I only had a 14.4k modem).
I bet we can play together.
That was my first computer project. A number of failures, boot disks, and Hey try this, this, and this/hang up the phone/dial up the modem/try again cycles and we finally had it going.
And it was awesome.
Ever since then, I was hooked. That experience led to building computers, programming, and into the career I have now.
And while the 90s might seem like a long time ago -- especially for technology -- a lot of companies are still using computer systems that are much older. Some of these systems have been relegated to support, marginalized by newer systems or by business processes that have evolved over time; but, a good number of them are still absolutely mission critical to the fundamental process of running a business.
My professors in college regaled our class with stories of mainframes and AS/400s that "are older, but will still be around for a long time, which is why we're teaching you COBOL".
Phooey, I said.
But, as a good student, I took the 2 required COBOL courses and started purging it from my mental archives as soon as I accepted my diploma and got my first job.
I settled in to my cubicle, ready to take on the technical world by storm.
I'm ready for this.
Then I overheard a conversation a couple of cubicles down.
So-and-so is out for a few days and we need to understand what this does. We have a review of the requirements tomorrow.
Only so-and-so knows this, so we'll have to push back the meeting.
Curious, I popped up out of my cubicle and walked over. What's the problem? Can I help?
Thanks, Mike, but I don't think so. We need so-and-so to tell us what this module does, and they're not here today.
Programming is my thing. I just graduated from college. I know EVERYTHING.
Show me. I might be able to help.
Crap.
This is... COBOL?, I asked, genuinely flabbergasted.
Yeah, there are a bunch of COBOL programs that use this module, but only so-and-so knows how it all works.
I made a mental note to send my professor a mea culpa.
If you give me access, I can figure it out. I know COBOL.
The flabbergastedness flipped around. You're too young to know COBOL!
To wrap up the story, I mapped out all of the programs that used the module, documented the inputs and outputs, and spent a considerable amount of time deciphering the usage of a mysterious variable t.
The meeting was successful. It's subject?
Mainframe retirement.
Before you ask, no, the mainframe didn't completely retire during my tenure with the company.
Why Are These Old Systems Still Around, Anyway?
These systems have survived decades of bug fixes, refinement, and plain weird things that have been asked of them for that one large client that will make or break the company.
Over that time, large sums of money, developer time, and aspirin have been invested in an asset that was responsible for keeping the core of the business running.
In addition, these systems were designed specifically for its company's business and, therefore, were flexible enough (with enough business savvy and developer moxie) to handle anything that was thrown at them. It was easy to write up a job, bring in another file, and modify a program (or set of programs) to process a new account and get it on the books.
This approach, however, added nuanced complexity to the code base. In order to make changes, you'd have to be pretty familiar with the way the code was written and, more importantly, what business problem the code was meant to solve. The system now requires unique, intimate knowledge in order to understand what it's doing, but also why it's doing it.
Furthermore, documentation was typically left up to the development teams. Sometimes there'd be meticulous documentation that was 100% accurate at the time it was written and other times, there was Frank who'd been with the company for a while and had written all of the core modules from scratch.
Tried and true, those systems continued to do what all systems do: exactly what they're told, every single time.
But years go by and people get older. Slowly, fewer and fewer people are around to make the kind of modifications that the company was used to making.
Going back to Frank: he was smart with his finances and moved his family to Hawaii after he retired. He passed down his knowledge to his fellow team members and then said Aloha after the first official gathering of the mainframe tribal council of knowledge.
Workarounds start cropping up because the system suddenly became less flexible.
More years go by. The tribal council still exists, but its members are vastly different. The once vivid stories are now mixed truths, legends even.
No one knows what that does, so no one ever touches it. We don't want to upset the system.
The company's leaders are told of the system and its old, cranky ways.
We understand the concern. Let's see what it will take to replace it.
A moment of relief? A brief respite? A seemingly wonderful reprieve? (I really just wanted to use 3 R's there.)
A team with a customizable, off the shelf system walks through the company doors. There are meetings, discussions, whiteboard sessions, and the like. In order to get a good idea of whether the new system can live up to the existing system's reverence, the new team needs to know the requirements it needs to fulfill.
What better place than to start with what you know?
Well, what does the system do today for
The tribal council is questioned and brought in for a discussion.
Our system does what your system can do... and it's already in place!
Bias aside, requirements are exchanged and a dollar amount given, with a rough timeline that may or may not be met (depending on your definition of success). The number isn't exactly small.
A decision is made to see what can be done that's less than that number in order to make some "quality of life" improvements.
The improvements are put in place around the now rigid, staunchly steadfast system. Team members come in to enable the system to work with newer technologies (e.g. hide that green screen and slap a good-looking UI on top of it). It slowly becomes a black box that "just works". Those that can keep it running are hard to come by. Fewer and fewer people belong to the tribal council.
However, the system endures. Continuing to chug along and provide the same value it's provided all this time.
Does This Mean They Will Be Around Forever and There's Nothing That Can Be Done?
No, they won't be around forever (and neither will the systems you're working with now).
And yes, something can be done, but it's not always as straightforward as a keep it around or kill it with fire.
The real question is should anything be done?
As has been previously mentioned, these systems have been hardened by years of bug fixes, hot fixes, and testing. They even survived the mini-apocalypse that was Y2K.
In essence, the systems adapted to the business until the business was forced to adapt to the system. Business processes were built around what the system enabled them to do, as well as what the system wasn't capable of doing.
In the end, the business became a reflection of the system. Those quirky things you may have heard from customer service when you called to have an issue resolved...
Didn't you just update my information? Why do I have to wait overnight before I can get confirmation that it was corrected?
...probably has something to do with the way their old system works (or their new system modeled after the old system; or their business process that didn't change even though the old/new system can provide an immediate answer, etc.).
At any rate, if the way business is conducted is unable or unwilling to change, what business reason is there to change systems? Companies with a business model that hasn't changed in years have little reason to change, besides what they spend to maintain their aging system and the risks associated with it (e.g. keeping resources around at a premium, security risks due to platform incompatibility, etc.).
Ironically enough, technology has made it even easier to keep these old systems around. You can lift and shift some of these capabilities to a virtualized environment or offload the hardware to a specialized data center. New technology is looking out for its older technology relatives.
On the other hand, if your business is evolving to meet different demands or needs to pivot to stay competitive, an older system may begin to hamper what you're able to do and the speed with which you need to get it done. In addition, administrative costs and technology risks are constraints that need to be addressed.
The primary purpose of software systems is to enable a business to perform the functions they need in order to be effective. When a system starts to legitimately "get in the way", the problem needs to be addressed in a timely manner to avoid becoming a bigger problem further down the road.
And don't forget your users: they're trying to do a job and when they encounter a roadblock, they'll get creative if they have to so that they can be successful.
So if you have an aging system, ask yourself a few questions:
- How much of what you do as a business is dictated by the way a system works (or doesn't work)? Do you need/want to change how you conduct business?
- Is a system legitimately hindering a business function without considerable overhead (e.g. more staff, considerable workarounds, etc.)?
- Is the level of risk you have with the system acceptable? Is your platform supported? Security standards met?
- Do you have reliable staff to support it if one or more persons win the lottery or retire to live in Hawaii?
- Is the ROI from your technology investments being realized?
If any of those questions give you a sinking feeling in the pit of your stomach, it may be time to have an honest, real discussion about your technology going forward.
Software systems are a tool just like anything else: they are there to enable and fulfill a function. A new system isn't a silver bullet that's going to solve all of the problems a business may have; conversely, keeping an aging system around because it has "just worked" for so many years may not be the answer, either, if the costs and risks outweigh the perceived benefits of maintaining the status quo.