Tuesday, December 20, 2011

The architectural success of any solution is directly related to the capabilities of the people who deliver upon it

One thing that an Architect must never forget is the capabilities of those around them. Over-complicate your solution to adhere ridiculously high standards or lengthy time-consuming review and approval processes and it will more than likely fail. Your vision must be able to be translated into modular, workable components that can be designed to integrate seamlessly into a larger picture as highly robust, reusable and deliverable, software components. In other words you must ensure your designers and developers are able to design, develop, test and deploy their work in as simple and efficient a manner as possible. Too much fluff leads to too much uncertainty and guesswork, too much complexity and the project timeline will be half-over before you’ve got any code running on a build server

Being a good architect requires (amongst many other skills) the overall ability to choreograph a high-wire balancing act that revolves around producing a solution that solves the business problem, meets the business functional requirements, meets the quality standards of their IT departments (and yours), delivers on-time but most importantly of all: can be delivered with the people you have at your disposal. To blindly assume every developer is as highly competent or better than you are is to fail before you’ve even started.

A good test for assessing how easy or complex your delivery environment has become is by noticing the ramp-up time for a new starter. If a new person needs to spend a considerable amount of time building their workstation, asking numerous questions of co-workers, has to perform a number of operating system tweaks or custom software installs to just get to the starting line then something is already going wrong.

Anyway rather than waffle any further let me present a story from the trenches to illustrate the point.

Early in my career I cut my teeth working on a highly complex, bespoke developed CRM application that went through a number of versions and iterations over its lifetime. This application was massive (and I do mean massive), it had over 100 developers alone at its peak, over a 1000 CR’s, and featured hundreds of Use Cases. As it grew bigger with every release more architects got involved to assist with the delivery of the enhancements. This injection of architects (all with their own opinions and persuasions) caused complexity to go through the roof as they deemed that new architecture frameworks were necessary to meet the increasing functionality demands of the business. Wild ideas ran rampant, various splinter groups went off on tangents creating their own processes and standards to deliver upon them as they all strived to create perfect architecture frameworks. A fall-out of this process also meant highly complex co-existence measures were required to ensure the new frameworks still worked with the previously established ones too – primarily because re-platforming was not an option. Knowing that delivery complexity was increasing exponentially as a result of adopting these new frameworks (and fearing that the technical designers, developers and testers were struggling to understand their vision), the architects started micro-managing every minute detail of the solutions being delivered to ensure they conformed to the numerous standards and processes they had created. They established numerous review and QA stage-gates to measure and ensure project delivery remained compliant with their architectural vision and established lengthy review and approval processes to govern the production of key deliverables.

Starting to see the tidal wave building now? The guiding mantra of simplicity and efficiency effectively went out the window at this point.

Approval processes to make for small changes took weeks to get finalised and developers who could not grasp the complex processes started to under-perform and produce poor code. This then in turn caused numerous defects, blew out testing times and projects started running well over budget causing late nights for everyone concerned (the record holder was set by a deployment architect – 40 hours straight, I only managed 24 as it was around that time that the code started to dance on the screen and I developed muscle spasms around my eyes). In short the desire to gold-plate and genericise every aspect of the delivery put such high demands on developers that they began to crumble under the strain. Thus the delivery of new business initiatives on the new architecture frameworks became impossible using existing time-frames established in the past for similar sized business initiatives. The large numbers of developers just could not grasp the complex coding structures, frameworks and processes set by the architects.

Delivery costs are now increasing, try explaining to the Business why you need 30% more time to deliver a CR now, than you did in the past, for an equivalent sized piece of functional change. Architectural purity? Good luck!

Compensation for the increased delivery times, and exponentially increasing complexity, meant the estimates produced for new projects blew out in ever increasing margins because no-one really knew how it all hung together. Tech Leads built in large amounts of contingency as they were afraid of not making the deadlines with the teams they had at their disposal. Delivery resources that had been on the project for a number of years were fought over at the resourcing table by PM’s as no-one wanted to take on a “new guy” because they knew they would not cope. And by and large they didn’t so those that were competent were kept in their roles because they could be relied upon – to the detriment of their own promotional aspirations.

Now a “hero” culture has been created.

So as the estimates kept growing fights broke out between the PM’s, technical designers and the architects as those at the coal-face could see the processes was not working. The architects struggled to translate their vision and ideas, a lot did not really know the intricacies of the frameworks, and others did not see how it had become so difficult. This malaise in turn caused other groups involved in the delivery of the application (PM’s, Testers, DBA’s etc..) to grow frustrated at the late deliveries which also were causing them late nights to test/deploy changes to get the projects over the line. The finger pointing began in earnest as all grew disenchanted with the entire delivery process and hence lost faith in the architecture team because they could not see the value in what they doing.

Once people lose faith in the architecture you’re on a very slippery slope

The architects however, despite the numerous issues could see the value in what they doing. Their frameworks were creating a much more modular, generic, extendible and reusable enterprise architecture platform. It was highly configurable, well-documented and the code (at least at face value) looked very well-written as they had condensed so much of the common functions down that developers only needed to (in theory) fill in the code logic gaps. Unfortunately to everyone else all the new architecture frameworks did for them was to make everything take longer to deliver, become more complex to test and deploy than it did developing on the older architecture frameworks. In other words it’s not that it was fundamentally flawed and would never work, it’s just that it was too clever for its own good and the capabilities and abilities of the people tasked to deliver on it.

So what did I learn from being involved in all of this?

Always consider the simplicity and efficiency of the architectural decisions you make as they are interpreted and communicated down the chain to the delivery, testing and deployment teams. Always keep in mind the capabilities of those that will implement it and the SDLC process they must follow to deliver it. Compensations must always be made in order to have a successful delivery of IT projects, knowing where and how best to do this unfortunately only comes with experience – you can’t teach it.

And finally always remember that when choreographing your high-wire act you don’t set the rope too high to cause a high degree of danger to your team or too low to provide a low-degree of value for your audience.  Better make sure you have some safety nets in place to catch those that are likely to fall too.