Topic 3: Enterprise Data Architecture

Reflection 1: The Six Principles of Modern Data Architecture

The first article I want to reflect on lists the six principles of a modern data architecture.  I almost dismissed this article as a list of banal bullet points which we have seen many times over. However, it was the last bullet point (#6 Eliminate Data Copies and Movement) that grabbed me.  

I’ve always focused on establishing single points-of-entry (into the enterprise) for critical data entities.  Once these single points-of-entry are established, a robust SOA can allow the rest of the enterprise to request the data, rather than creating redundant data capture capabilities which always lead to data inconsistencies throughout the enterprise.  

My thinking also assumes that systems requesting data from the point-of-entry system(s) will store the data locally for performance purposes.  Even though the data will be replicated, at least it will be single-sourced and consistent.  It is that very assumption of mine that made principle #6 so compelling.  It’s a great vision, eliminating data redundancies and movements, but has technology progressed enough to make it a data architecture principle that all enterprises can adopt?  

The same observation was noted in Andrew Richard’s blog posting from September 11, 2016, where he notes the usual and customary approach of replicating data for performance purposes, even when a robust SOA is available to share data.  Mr. Richard goes on to ask if there are any good reference architectures that address the replication problem.  The short write-up for principle #6 mentions a product called Hadoop which I had never heard of before and which is where my second reflection is headed toward.

Reflection 2: Running Operational Applications (OLTP) on Hadoop...

After Googling “Hadoop” and reading various articles, I quickly learned that it is an open sourced, big data platform.  I will admit that I am not well versed with the big data trend.  My high-level understanding is that it is largely pertinent to data warehousing and analytics.  However, for the question raised in Reflection 1, what I’m really interested in are traditional relational databases used by OLTP applications.  Can Hadoop (along with a compatible RDBMS) and big data reach beyond the data warehousing and analytics world and solve the replication problem in the OLTP world?  Can a business actually have a single “big data” OLTP database serving much, or maybe even all, of the enterprise?

Those questions brought me to the second article I want to reflect on.  This article asks and answers four different questions about Hadoop.  The first question sets the stage quite well...

Hadoop is primarily known for running batch based, analytic workloads. Is it ready to support real-time, operational and transactional applications?

This is exactly what I want to understand, so let’s look at the second question which asks how enterprises can take advantage of Hadoop.

How can enterprises, specifically in the Retail industry, take advantage of a Hadoop RDBMS?

The author responds to this question with a scalability and performance answer, stating that enterprises can leverage Hadoop and Splice Machine (Hadoop-compatible RDBMS) to provide scalability and performance for extremely large RDBMSs.  So, scalability and performance is examined while eliminating data replication and movement is not.  My hope that this article was going to address principle #6 from my Reflection 1 article seems very much in doubt now (even though principle #6 specifically mentions Hadoop).  Let’s take a look at the third question.

Can we run mixed workloads – transactional (OLTP) and analytical (OLAP) – on the same Hadoop cluster?

Now I see how this article is addressing principle #6 and why Hadoop was mentioned in the first place.  Remember principle #6 is titled “Eliminate Data Copies and Movement”.  I immediately interpreted that as a call to eliminate redundancies between OLTP databases and the movement of redundant data between OLTP databases.  However, this third question reminds me of the more likely intent, which is the elimination of data redundancies and movement between OLTP and OLAP environments.  I don’t know if that’s what the first article had in mind with principle #6, but I think it is.

The fourth question simply explores the return on investment for implementing a Hadoop/Split Machine solution for massive RDBMs that mix OLTP and OLAP capabilities.

Even though article 1 was not specific in what it meant by principle #6, and even though article 2 did not actually explore the possibility of some enterprise one day building a single enterprise-wide OLTP database, I can’t help but think the enabling technology exists with Hadoop and Split Machine - the biggest challenge with such a crazy concept will likely be organizational.

Reflection 3: Do You Know What Your Company’s Data Is Worth?

The third article I want to reflect on shifts gears to a different topic, which is treating data as an asset.  Among the Lesson 3: Enterprise Data Architecture reading material for EA-874 is an article from Gartner titled “Managing Information as an Asset: Enterprise Architects, Beware!” which describes this “information as an asset” concept as treating your data with care, ensuring consistency, accuracy, accessibility, utility, safety, and transparency. Upon reading the title, I felt I wanted to include the article among my three reflections.  Upon reading the article itself, I no longer felt it was worth reflecting on - businesses needing to treat their data with care like it’s an asset seemed too obvious and not very interesting.

However, just a few days later I came across the Reflection 3 article from above.  This article actually does talk about assigning a dollar value to a company’s data, which is a new and interesting concept, and one I want to reflect on here.  

My initial position is that data valuations don’t belong on the balance sheet as its value is already reflected in intangible asset categories such as goodwill, brand equity, and intellectual capital.  However, what I learned in reading the article is that this idea of valuing data is not about adding new assets to a company’s balance sheet, but is instead about taking the intangible assets valuation and breaking it down further to assign a chunk of that valuation to the company’s data.

The important realization for me is that the total value of intangible assets will still be calculated in their traditional ways.  Here is a very simple approach:

Intangible assets = market capitalization - (year-end sales + tangible assets)    

The article isn’t saying businesses are now expected to mess with their balance sheets and inflate them with valuations of the data they possess.  It’s simply saying that businesses need to take their intangible assets valuation and figure out how much of that valuation is attributable to their data.

So why is this important?  The answer is risk management.  The higher the data valuation, the greater the exposure to such things as cyber attacks and system outages (such as Delta Airlines recent outage).  Once investors, business leaders, and IT leaders have a clearer picture of risk exposure, then, it stands to reason, better decisions will be made to mitigate the exposure associated with safeguarding a company’s data.

Topic 2: Enterprise Application Architecture

Since the topic for this blog posting is Enterprise Application Architecture and since the title of this blog is Enterprise Architecture for SMBs, it seems fitting to analyze one of the critical elements of an Enterprise Application Architecture, that being SOA, for small and medium sized businesses.

Reflection 1: Small and medium businesses: We simply don't need SOA

The first article which I want to reflect on explores the position that SOA isn’t necessarily important for SMBs.  The reason offered is that SMBs do not have the application complexity and diversity that large enterprises have.  I currently work at a $60 million business which has two primary lines of business.  Each line of business has a primary back-office system (one is custom-built, the other is commercial) along with shared commercial finance and CRM systems.  That’s about it - definitely not complex, so the article has my attention.

The article references an informal survey of mid-market CIOs which reveals that two-thirds feel there is no business need for SOA at their enterprise.  This opinion may be due to an overall simpler IT environment that “works” or it may be due to limited resources (people and budget).  Whatever the reasoning is among these CIOs, I think the informal survey provides an accurate representation.  The article is from six years ago, but having worked at SMBs for the past nine years, I think the opinions expressed in this 2010 article are still holding true in 2016.  

Further into the article, however, the author chooses to pivot from the opinion of the mid-market CIOs and offers some of his own reasoning on why SMBs should care about SOA.  He says:

Of course, the mid-market is the prime candidate for cloud computing, and this is where the value of service orientation will ultimately be realized, on an enterprise-to-enterprise scale. But for applications within the infrastructures of smaller enterprises, SOA is still beyond reach.  

Still, the more a business of any size can service orient, the better. SOA is an extension of enterprise architecture, and EA is important at any level.

I’m in agreement with the author and I think the first paragraph above is very telling.  Even six years after this article was published, I can attest that for many SMBs, running a solid, well-managed data center is beyond their capabilities.  With the data center yoke around our necks, the notion of implementing a service-oriented architecture is accurately stated as beyond our reach.  We need to get out of the data center business, move to the cloud, and then start innovating with our Enterprise Application Architecture.

Reflection 2: How Can Small to Medium Sized Enterprises Benefit from SOA?

For my second reflection, I want to remain on topic with SOA at SMBs and get some additional perspectives on why SMBs can, and should, move to a SOA.  The article being examined is actually a series of responses to the question “How Can Small to Medium Sized Enterprises Benefit from SOA?”.  Most of the answers lack substance and depth, but there were a couple of comments that I found very relatable.  In the posting from John Power he states the following:

However, from my perspective the real benefit to SMBs is the possibilities it offers them to integrate their systems with large customers or suppliers.

This struck me in two ways.  First, it struck me for its accuracy.  It has been my experience that invoking a few web services and standing up a few of our own in order to integrate with our suppliers and partners is exactly what many SMBs do for their SOA initiatives (without going much further).  

The second manner in which it struck me was how it reinforces the confusion and hype around SOA.  Integrating with a handful of external businesses through web services in no way constitutes the deployment of a broad service-oriented architecture.  However, I will state that those simple external integrations are extremely valuable to SMBs and they work well, leading many CIOs, I reckon, to conclude “that’s good enough for us”.

The other posting that caught my attention was from Kelly Emo.  She states the following:

Secondly, SMBs will find themselves brought into the SOA world whether they explicitly plan for it or not. Packaged apps and SaaS solutions are rapidly evolving to be architected based on the concept of loosely-coupled shared services...

This claim is also very accurate in how it matches my SMB experience.  It is the packaged applications and SaaS solutions which SMBs utilize, that essentially forces them to eventually learn and leverage the service orientation built into those commercial offerings.  

It seems the ultimate takeaway for me is that SMBs will adopt SOA gradually (even glacially) and very deliberately, avoiding big-bang transformations.  If resistance persists, then, to the point made by Kelly Emo, SOA will likely start deploying itself in the enterprise.

Reflection 3: It's Time to Retire the Concept of the Three-Tier Application

The third article I want to reflect on comes from the Lesson 2 reading material for EA-874.  This article seemed like it would be a fitting cap to my SOA-themed blog post.  I expected the article to largely espouse the virtues of service-orientation for building applications. In other words, the author just wants us to replace the traditional locked-in business logic of the middle tier with services that can be invoked throughout the enterprise.

The author quickly reminded me, however, that replacing the locked-in middle-tier with services still leaves us with a three-tiered application.  So, my assumption about the article was wrong and I was suddenly wondering what new trend all the SMBs will be lagging behind on now.

It turns out the author is not actually recommending that we retire the three-tiered separation of concerns between user interfaces, business logic, and data access.  He, in fact, states that we need to continue reinforcing the three-tiered separation of concerns.  All the author is saying, as a natural evolution after implementing a SOA, is that the user interfaces of applications should no longer be homogenous and monolithic - they should be a collection of components supporting many devices.   These user interface components will invoke many business services, and these business services will invoke many data services.  

Keep in mind that all these pieces are still to be architected with the separation of concerns being resolutely applied.  We just need to stop thinking about single applications as a homogenous three-tiered stack and instead start thinking about them as a collection of user components and business services that cannot be easy abstracted as a nice, neat three-tier stack.

The bottom line is that the separation of concerns (user interface, business logic, data access) is not going away, but simple, self-contained applications are. All in all, not too big of a paradigm shift for SMBs to work toward...once we start making more progress on our SOAs.