Thursday, February 24, 2011

Meta data versus folders in MS SharePoint

Somewhere in the 90's when we were building our Intranet, one of my colleagues pushed for the concept of removing folders for the management of documents and to replace this with meta-data. The idea was that folders are one dimensional and have a strict hierarchy, but with meta-data you can create a multi-dimensional structure.

For example you have a folder structure as displayed to the right. With a folder structure you cannot list all documents in all "Analysis" folders. You can do this with only one folder at the time. Using meta-data, you would be able to do this without specifying a value for a project name.

Microsoft implemented this concept in SharePoint in the form of "columns". SharePoint consultants are adamant that folders should not be used anymore (or much less) and should be replaced with meta-data. The concept sounds very appealing but has some limitations (you might want to translate some of them into strengths, but I look at this from the perspective of the lazy end-user):

  • The concept of multiple dimensions eludes many people. It might be that Gen-Y people over time pick this concept up easier since they have not worked in the strict folder world for that long. However I like to warn against comparing this with the easy uptake of social networking by Gen-Y and the fact that they have replaced email with Facebook, Twitter and SMS. The use there is free and for the moment only. In a business context you need to have more structure and need to keep data for a longer period of time. Note that folders are a concept that we understand from the real life. We have folders in boxes on shelves. When we lose this concept, people are a bit lost in where the document really is. We do not always need to know this. My blog is somewhere in the cloud and I am not too worried about where it physically resides. But I think SharePoint should come with a better user interface for navigation and management to give the users a better sense of control.

  • In SharePoint users cannot freely create new columns. Where a folder impacts only the folder in which it is created as a sub-folder, a column impacts the whole site. 

  • It requires to have the right permissions. If people can't make their own columns (folders) they will feel too much locked in and will resort to using alternative places to store their documents.

  • With columns you need to think ahead and define values and the required use beforehand. While many people have a document or a few documents and just want to place it somewhere that makes sense for that moment in time. If it becomes too much work, they will place it on the local C drive. If you could create columns on the fly and as easy as creating a folder and add values to columns on the fly, it might make more sense.

  • When specifying in which folder and where in the hierarchy you want to create a document, you can do that using mouse-clicks. It works via a drill down mechanism and each time the number of options are limited. With columns, the selection of a value for the first column does not reduce the number of options for the second column.

  • The way SharePoint works now, it is not easy to navigate using the multi-dimensional features. You need to define views for that.
Though more and more organisations are implementing SharePoint, Microsoft should improve the ease of use in order to improve the uptake. It is disappointing that after more than 10 years, I haven't seen a good easy to use commercially available implementation of the concept.

Tuesday, February 15, 2011

IT does not always need to be a partner in strategic change

We have read and wrote enough about the IT-Business divide and that IT should take more initiative in business improvement. I find it difficult to actually drive business improvement pure through IT initiatives, but in cooperation with business teams over the years I have been able to contribute to business improvements.
I don't say that there aren't options to do so. But in the end you need support from upper hand and need to get buy in from your business stakeholders. If you don't get this, you will find resistance and will be told to focus on the tasks given to you.

In my previous role as consultant I was able to contribute to strategic initiatives and coming up with ideas to improve the organisation structure and business processes that were implemented successfully.

In the more recent period I have been able to work with our Finance and Procurement teams where we as IT had a direct input in ideas for improvement. As a combined business and IT team we implemented a series of systems to improve operational efficiency and corporate governance. For example, years ago I identified potential improvements in the way we processed our invoices. Once we had the context in terms of systems and processes right, we came to a full automated solution for Invoice Processing. We selected the Basware technology and Basware was effective in implementing their solution. As a result Finance has earlier insight in outstanding payments and improved its cash management.

In another example we were able to transform a set of disparate online learning solutions for our customers into a single global web application. The result so far is that this website won an award for the best online training program by the German trade magazine ‘touristik aktuell’.

And that is where I think the role of IT should be: participating in operational and strategic initiatives and driving as such business improvements. I don’t think, bar some exceptions, that IT really will ever in the position to take the lead.

IT will always respond to possibilities. What we see now for example with strong growth in consumer technology that is being brought into the organisation, that it still is a response to new technology in the market. The business is strongly driving the demand for iPads, iPhones or the use of Twitter or Facebook, simply because they already know it from private use. Even if IT is first on the ball, it is still bringing it as an idea to the business and together with the business teams to define the business case or just to try things out. Even if we talk about all the possibilities in the cloud, we might talk about strategic choices for IT but from business perspective it is much operational efficiency. Only in exceptional cases IT might be able to drive strategic change and we should not want to try too much more.

When I look where our organisation was 5 years ago and where it is now in relation to finance and procurement, I am proud with this silent revolution to which we had a major contribution, not only for the implementation but also for idea development.

This can happen anywhere in the business as long as both parties are open to work together and see each other as equal partners.

Louie Ehrlich, President, Chevron Information Technology Company, and CIO, Chevron Corp might explain it a bit better here. But it comes down to the simple statement that if the business is not ready for IT being a strategic partner, don't try to push it.

The IT Evengalist addresses it from the angle of innovation and states that every innovation within IT (within that organisation) automatically is an innovation for the business. His complaint is the same as mine: " I have lost count of the number of the experts and pundits who are forecasting the demise of every CIO who doesn't "step up" and contribute to if not drive business innovation."

He also postulates that we as IT might be wanting too much to be the strategic partner: "So while IT is waiting for all of those business leaders to read the articles on the "IT organizations of the future," IT can prepare itself for the day the business actually looks to IT to help drive business innovation."

As an enabler for te business, you automatically assist with business innovation and support strategic develoment. In order to be a good enabler, you first of all need to have your house in order and not spend all your time fire fighting. Often the business will come with a new idea that needs to be done yesterday. To avoid that this results in a fire fighting situation, you need to understand where the business is heading to and be ready for the next challenge. It does not mean you will live up to the expectation of having the work done by yesterday. You simply need to accept that there will always be a conflict between expectations and the reality of getting things done.

But that is different than driving strategy or driving innovation.

Who is steering the car? The person behind the wheel or the person on the backseat telling the driver where to go? Does it really matter? As long as you get where you need to be in an efficient way.

The problem is in my opion more that due to the lack of partnering, opportunities are lost and efficiency is compromised. The driver and the passenger need to work together to make sure you get to the right address via the quickest route (and that is definetely true when using a taxi in Sydney!).

Monday, February 7, 2011

Jazz Fusion and complexity of software architectures

I love Jazz Fusion: Weather Report, Brand X, Return to Forever, Pat Metheney, etc. I know a lot of people think the music is too complex, it makes them nervous or it does not resonate with them. Yes, Jazz Fusion has that extra bit of complexity but if you hear how it all comes together it is not that complex anymore. I cannot sit down and listen to most other music. I need something that keeps the brain stimulated, otherwise it becomes background music. Jazz Fusion with its complexity relaxes me.



Within Jazz Fusion, the bass and drum usually play more an equal role in the whole composition and are not just there to set a rhythm. I simply love how Percy Jones plays his bass. I know that I myself said in the past it is as if all musicians play a solo at the same time.

I have managed over the years various systems of which people at first glance said they were too complex and that simpler alternatives should be sought.

In general I too prefer to look for the Justin Bieber equivalent for a software application. Recently I have been looking for a very simple Business Intelligence reporting tool. The first idea was a simple desktop solution connected directly to the operational database where we expose some data via SQL views. The users should then be able to create their own reports.

However it turned out that this type of solution hardly exists and if so, not easy enough for the user to work with. So we resorted to a server based solution, but still a relatively simple solution. In this case proCube from the Satori Group.

But many of my systems are or were indeed complex. But they need(ed) to be complex because that is what the business needs. Complexity is there for various reasons. One of them is to let the data flow between systems and another is to allow for flexibility and the variable needs over time. For those of us who understand how it all works together, the complexity is part of the beauty of the solution and actually makes life simpler. We know that if you start oversimplifying, it will lead to increased dissatisfaction and more work for IT to facilitate the changing requirements and to facilitate data integration through manual processes. Though the total solution might be constructed through Oracle, Microsoft, Java solutions and what else, it is still an orchestrated symphony; there is harmony.

If an IT infrastructure is well composed and technology is used correctly, it is never complex to manage and to support; even though others might consider it a disharmony (I like the Dutch word Kakofonie much better).

Tuesday, February 1, 2011

Plato’s Revenge – Part 5: The grand design of the multi-universe - by Hawking and Mlodinow

In a previous post I wrote about the book “The Grand Design” by Stephen Hawking and Leonard Mlodinow. I saw the book in the bookshop and thought that since I said something about it, I better read it as well.

It is a relatively easy read and contains no maths. One of the key arguments is that there are most likely other universes that are created and that disappear over time. Since many universes can be created at any moment, in time the lucky situation occurred that a universe was created with exactly the laws and parameters that allows for the creation of solar systems and planets and therefore life as we know it. One of the other arguments is that mathematically there is no need for a God for this creation and they also explain that intelligent life and specifically human life with the freedom of will do not need God for this creation. Due to the complexity and the immense large number of parameters and dependencies involved, we experience freedom of will but it does not necessarily mean that this is the case.

Universes can be created spontaneously and this randomness relies, as far as I understood, on Heisenberg’s uncertainty principle.

The mathematics and the theories we have defined so far are not always that elegant yet and we only have models that seem to work relatively well but part is this is because we have our own limitations with respect to discovering the laws of nature.

A nice explanation is given about the goldfish in the bowl as an alternative to Plato’s Allegory of the Cave (see my first post). The goldfish inside the bowl has a distorted view on the world and therefore would come up with rather complex models of the laws of nature for whatever happens outside the bowl.

I think the book is a good read and challenges us to think further. However I see two weaknesses in their arguments:

1) As far as I understood, the spontaneous creation of universes relies on Heisenberg’s uncertainty principle. Since this is one of the great miracles in physics, would it not be very well possible that due to our limitations and our distorted view on reality (being a fish in a bowl), that we see this uncertainty but that in reality something completely different appears behind the scenes?

One of the key things I belief that we humans do not really understand is time. Physicists might be pretty happy with the way they calculate with it, but do we really understand it? We can’t grab time, we can’t go back and forward in it as we can in space. I like their explanation of time to see it as the vertical axis on a circle. When you are on top of the circle, you can move left and right, but there is no downward movement in time. So at the top of the circle time stands still but when you move in space (horizontally) time starts and the further you progress along the circle, time is moving until you come to the bottom of the circle.

2) At one moment time started and at the moment of the big bang, the universe was created out of the singularity. As they explain, we probably can’t apply our normal laws of physics to those very early stages of the creation of the universe. But my question is, where does the singularity come from? Why should there be a creation at all? Why should there be anything at all? Why should Heisenberg’s uncertainty principle apply? The questions that arise are not much different than asking who created God.

The conclusion that this is all explained by the laws of physics is not very well grounded and they skim over this issue too quickly. They very nicely start explaining that we have a distorted view and therefore our theories and models are more or less approximations. But then they don't really apply it to their own reasoning. Then they should not bring their conclusions with this certainty. They should bring it more as if it is just one potential explanation.

Given all this and with all its weaknesses in the arguments, I do not have any problems accepting that different universes can be created or that there are other parallel universes.

In my day to day life, I see that different people live in different universes where different laws apply. Gravity does not seem to apply for babies. At one moment in time they have something in their hands and at another moment it disappears. Then suddenly it appears in their hands again. Over time, the concept of gravity grows into their world. I still see it with my daughter that gravity is not always existent in her world. She can put a glass of water near or halfway the edge of the table. Sometimes the glass suddenly disappears and then there is much emotion. But later the glass will magically appear clean and well again in the cupboard. In my universe, the glass falls on the ground and I can clean up the mess.

In the work context you see that people live in different universes as well. Though there are two major sets of laws of nature that govern the work context (economics and human behaviour) plus some industry specific laws (e.g. biology, physics, etc.), different people live in universes with a different mix of those laws.

The laws of economics are to certain extend straightforward. You produce something for the costs of $5 and sell it for $7 and you have $2 profit. However you do need the laws of human behaviour in order to sell your product and in order to make the people in your organisation achieve the goal of making profit. Human behavior is much more a mystery. (OK, you might want to argue that economics also covers human behavior, but that's not the point here).

In some universes the laws of economics are loosely applied while the laws of human behaviour seem the focus of attention. In most cases cost efficiencies and profitability are then a coincidental side effect. Money seems to appear out of nothing, coming in through a wormhole from another parallel universe.

In other universes, everything is well defined in financial structures but where there are some strange laws about the rules to create certain products or where there are mysterious rules about organisational structures equivalent to Heisenberg’s uncertainty principle. The reason for doing things seem to be rather random.

In effective organisations that sustain growth and profitability over a longer period of time, both sets of laws are well formulated into a single model (M-theory).

Take for example Apple. They created the first i-Phones with the rounded edges. Engineers probably have complained about it, and I may assume that the marketeers demanded this. They understood human behaviour and created something that looked nice and something that people wanted regardless whether this contributes to a technically good product or not. Apple was very successful in creating a hype and this way revolutionised our world even though we might say that there are many practical limitations to their product.

Organisations where these laws are not brought into a unified model can still be created and still can have strong growth and can temporarily experience strong profitability, but if there is no balance over time the organisation will disappear; just as an instable universe will disappear.

The same logic applies between departments within an organisation. The IT department usually is governed by the laws of economics and is strong in thinking in terms of efficiency. In most cases that is their remit, while other departments are more focussed on the laws of human behaviour such as the HR and Marketing departments. And those laws are not always easily reconciled. In order to exist, we need to allow for a certain amount of uncertainty and randomness.

For example, we need to accept that if we commence a project, there is only a certain probability that the project will ever be finished and the resulting system will be used. The better the unified model, the higher the probability. The challenge of coming to a good model is that you sit within your glass bowl and have a distorted view on your own universe and that makes change from within the organisation difficult. We call it the culture of the organisation. Revolutionary insights (e.g. Einstein’s theories) or long term growth towards maturity (people slowly becoming aware of gravity) are required to achieve change.


I also like to make some concluding remarks about the argument relating to the existence of God. Of course that creates quite a hullabaloo around the world and everyone jumping on the subject to say something about it. (Somebody in favour of the argument:  http://motls.blogspot.com/2010/09/stephen-hawking-grand-design.html; Wikipedia also gives an overview of the criticism) At least it helps sell the book. The issue of free will is also heavily discussed.

But I did not see anything in the book that really excludes the existence of God. I don't think that the book really needed this argument at all.

I don't like these type arguments in relation to the existence of God in either direction. 

I believe in God and therefore Hawking and Mlodinow are wrong.
I do not believe in God and therefore they have a valid argument.

Years ago I watched a show on TV in the Netherlands discussing which city was better. The one camp argued that because they were born in Rotterdam that each time when they arrived back in the city, they felt at home. Therefore Rotterdam was the best city and Amsterdam was nothing. The other camp argued the other way around. What a nonsense!