Enterprise Architecture is failing. Is it because there are too many consultants?

An interesting discussion on Linkedin entitled Enterprise Architecture is failing. Is it because there are too many consultants? was posted. What follows is my take on the subject. One of the interesting posted comments was:

"Successful EA requires a disciplined commitment at all levels of organization. While that is moderately challenging to attain at the upper levels, propagating that discipline through the management ranks, both business and technology, is THE familiar change management challenge. That is trouble for any organization that attempts the "give me a little EA first" approach - and doesn't apply it to a complete division or subsidiary. And since we're so good at financialdiscipline, that is usually the approach that executives gravitate toward." - Aleks Buterman

I would like to add to Alesks’ first. I find that many enterprise architecture groups base their visions and roadmaps on best practices without taking into account the human dimension impact of their choices… without taking in consideration the culture of the organization. For example, if a company has a culture based on competition and silos it will be extremely hard (not to say impossible) to implement an enterprise data quality management vision.

With regards to consultants, it is very difficult for “outsiders” to truly understand the culture of an organization from being immersed in it during a short period. Hence, going back to my first comments, an “outsider” enterprise architect will NOT be able to truly understand the human impact of his recommendations. What follows is that his recommendations are either not followed, not implemented correctly or their implementation is not sustained in the long run because people do not feel it is “their” solution… this is typical resistance to change. A good way to avoid this is for consultants to act as a process consultant instead of a content expect, hence the consultant HELPS/FACILITATES the hiring company to define an enterprise architecture which aligns with it culture or helps the company to discover the discrepancies between what the company wants and what its culture can sustain. For more info on this see Peter Block. Another interesting post was :

"It is a cycle, while I think there are a lot of EA consultants our there that are a problem in their own right I think from the bigger picture point of view they are a symptom. A lot enterprises think that they can buy EA like fruit at the market a good EA program is more about growing the fruit. EA is something that a successful organization struggles over, strives towards, and is commited to. Bringing in a prepackaged EA is like renting a wife and expecting a marrige. It has to be your EA, not 's EA" - Art Freas

So in resume, I don’t think that the number of EA consultants is the true problem. As Art said, I believe that organizations believe that “off-the-shelf” solutions exist for EA hence they believe that a consultant can help them. I believe that this problem is itself a symptom of organizations which do not take into account the human dimension of EA. I personally believe that Organizational Development should be incorporate into the EA function, this would help greatly with this disconnect.

Is Enterprise Architecture sociotechnical systems (STS) design ?

Sociotechnical systems (STS) theory warns about optimizing the technical dimension of systems without considering the social dimension. Moreover, optimizing both dimensions of a system independently of each other often causes the system to be sub-optimal as a whole. Personally, I see the traditional approach to enterprise architecture as only addressing the technical (technical in this context must be understood from the perspective of STS and not IT) dimension of systems (organizations). I believe that enterprise architecture should be considered equivalent to STS design, hence we must incorporate the human dimension if we want enterprise architecture to be effective.

If my reasoning is correct, the next questions I would ask are :

• Could STS theory be a good framework for evaluating the potential effectiveness of EA frameworks/methodologies?
• How effective would current EA frameworks/methodologies be with regards to an STS perspective?

I find that many enterprise architecture groups base their visions and roadmaps on “technical” best practices without taking into account the human dimension impact of their choices. For me, this is an example of "optimizing the technical dimension of systems without considering the social dimension".

I think that many people underestimate the impact that an organization’s culture will have on the implementation of “technical” best practices. I believe that this is symptomatic of that fact that many IT people use an empirical-rational strategy when managing the impact that enterprise architecture has on organizations. This strategy is based on the assumptions that : people are rational beings and will follow their self-interest – once it is revealed to them – and that successful change is based on the communication of information and the proffering of incentives. Often, these assumptions are not totally accurate; hence compromising the “technical” dimension of a vision (not using best practices) in favor of the “social” dimension could lead to a better overall solution.

My feelings on Business Enterprise Architecture Modeling (BEAM)

As I have mentioned in earlier posts, I have become increasingly interested in organizational development as a tool for addressing the human dimension of enterprise architecture and its challenges. My readings on OD have brought me to believe that:

• the empirical-rational strategies to change/learning that most EA groups use which are based on the assumptions that “people are rational” and “will follow what is in their self-interest if it is revealed to them” are not necessarily effective strategies.
• an EA group that functions in a elitist and “top-down” manner will create more resistance then is necessary and will also have a lot of difficulty in getting people “engaged” in realizing the proposed visions.
• most IT organizations under-estimate and under-invest in on-going learning and the strategic management of skills.

My mid-term objective is to define an EA framework which incorporates values and techniques from the discipline of organizational development in order to achieve the following:

• an EA approach which truly recognized the human component of an organization as the most important one and fosters the definition of visions and solutions which take into account the organization’s culture as well as the learning needs of people required by these visions and solutions;
• an EA approach which is inclusive (cross-department and cross-level), based on teamwork, fosters double-loop learning and system thinking;
• an EA approach which privileges the definition of visions which are “bottom-up” and based on shared visioning;

I believe that tools such as appreciative inquiry, shared visioning, open system thinking, group process facilitation and culture transformation will be key elements of a “human-oriented" framework for EA.

I have discovered a Enterprise Architecture approach called Business Enterprise Architecture Modeling (BEAM) promoted by Ken Orr. My interest in BEAM is that I feel that the RAD sessions and the alternative scenarios which the approach includes incorporates, at a certain level, elements of the values and techniques I mentioned above. For example, I perceive the RAD sessions as fostering an approach which is inclusive, teamwork-oriented and based on shared-visioning. I perceive the alternative scenarios elements as similar to the scenario planning work done at Dutch/Shell which is base on system thinking and shared visioning.

Challenges when combining SOA and Agile

For me, both SOA and Agile require deep cultural changes in organizations in order to be effectively implemented.

Usually, many assumptions about using Agile rest on having self-organizing, self-managing and cross-functional teams. This is typically a step away from traditional “top-down” monitor and control approaches to management and “over the fence” requirement/delivery management. Organizations are open systems that require a holistic approach when changing; hence trying to implement Agile locally without taking into account the non-local forces that will impact the agile delivery group could result in frustrations and poor performance. To illustrate this, one only has to think about the manager that wishes to foster agile delivery but must supply the “traditional” project management indicators/elements to a VP (long term estimates/planning, weekly progression, change requests, etc.) for a dashboard.

SOA also requires deep culture transformation. For me, SOA is not about technology (Web Services, REST, etc), its about having a clear understanding about operational processes as well as their segmentation, governance and data requirements. In my mind, in order to achieve a truly effective SOA implement most of the values of Quality Management (or Total Quality Management) must be implemented such as “global interests above local interests”. SOA requires a holistic, cross-functional approach to its implementation and management.

An example of a possible challenging situation that an Agile-SOA team could face is one which requires the team to delivery a list of services that surpasses its knowledge base. In this context, chances are that “over the fence” attitudes will show its face; instead of using a self-contained cross-functional delivery team, requirements gathering will probably be thrown over the fence (because it is usually the hardest) and the SOA team will only manage delivery. This situation creates a potential gap between what is required and what is delivered as well as an “us vs them” dynamic which does not follow the values of Agile. Now, we can also imagine that the team must delivery data-oriented services in which the consuming departments and the producing departments of the data are not the same. In this situation, if the organization is not supporting holistically the values of SOA, the team will struggle deeply with the prioritization of the services when “local interests” pressures build around the task without the capability to manage “global interests”. In this context, SOA adds an extra layer of complexity because even if the team manages to holistically prioritize the services to deliver (holistic planning) they must also manage to get people with potentially no interest in certain data to produce it for another group (holistic delivery). The latter combined with the former requirements-delivery gap and an “us vs them” dynamic can make for a very explosive experience and frustrating.

So in a nutshell implementing “SOA-Agile” = implementing “holistic planning and delivery” which requires a deep change in culture in most organizations… its no just about technology.