Searching information about products and vendors is part of my job profile, mostly to know what will be best for my clients and the projects I am responsible for. Therefore reading analyst reports about products and how they group them into quadrants is important feedback from another perspective than mine. Recently my opinion as to the usefulness of analyst vendor assessments has changed. I obviously knew that analysts do no more than voice an opinion, but I assumed that they are spending a lot of their time to come to that opinion that they then sell to their clients as expensive reports. Well, I was wrong.

My first change of mind occurred when I realized how much IT managers and consultants of large corporations rely on analyst reports for vendor selection. A vendor that I felt was very suitable in terms of verified product functionality as well as size and similar successful projects, was rejected because the vendor was not in the top analyst quadrant. The responsible manager was sacrificing a faster project and much lower TCO because he was not willing to take the risk. This is not good!

I decided to find out for myself and visited this medium size software vendor who has been in the market for a long time with a you-know-them-all banking and insurance reference list from Los Angeles to London and Tokyo. I found a company with a unique product that was amazingly innovative, years ahead of what the analysts saw and more than anything offering a true customer focus and partnership. Both product and market position made this vendor clearly a leader on a worldwide basis. But it did not pay the right people …

This vendor happened to be in the middle of assessing analyst relationships, mostly because it had been excluded by a major analyst company from a report on its oldest and most successful market segment. Totally unknown and irrelevant new vendors were in the report as well as all the large players. It came all down to the fact that this vendor was not a client of this analyst company (which costs €23.000 a year) and therefore no analyst interviews had been conducted. Thus: no inclusion in the report. This vendors assumes that he will be losing a substantial amount of revenue because too many IT buyers will only send RFPs to the vendors in the report and select only one of the ones in the top quadrant. Why does no one understand that this hurts the IT industry as a whole and it kills the innovation we are all looking for!

All that would be fine if the analyst reports were reasonably accurate. But they are not! I suggest that analyst vendor ratings are no more than self-fulfilling prophecies. Analysts opinions are mostly related to market share, revenue and the amount of money and time the vendor spends with the analyst. It is the same conflict of interest that the large five consultancy companies had a few years ago between consulting and accounting services. You can’t serve two masters.

While the analysts reports are nowhere near as relevant in Europe and Asia, it is hard for an IT company to gain a worldwide reputation without marketshare in the US. It seems appealing to ignore this problem and simply fight it out. While are large IT vendor can make it past vendor assessments with mere marketing might, a smaller business can not do the same. IBM created its WebSphere brand without a product with billions spent on advertizing. This vendor will therefore most probably have to spend the money and pay for educating the analysts to the fact that their assessments are wrong. Analysts do typically not visit vendors or users, they search the web, copy marketing materials, and conduct a few phone interviews with vendor selected users.

In defense of the Analyst the reasoning behind inclusion (and to some degree exclusion) is, as a particular snapshot of the market it is no better or worse than you’ll find elsewhere. Clearly, consultants and buyers should be cautious in using this as a tool to pull short or long lists together, let alone actually decide on a provider, since it only tells a partial story.

Remember: Analysts are just people. They mostly live in their little ivory towers and are far away from the reality of our day to day IT drama. They do not have to implement the solutions they propose, they just create fragmented markets with new three letter acronyms. And then they get paid for explaining to IT buyers the confusion they created. Buyers of technology on the other hand might be better advised to spend their time and money fully researching the market and available options and ask for Proof of Concept and Technology instead of believing the marketing.

Can anyone tell me what is going on?

PS: I would seriously love to hear from the analysts.

Do IT vendors have to buy analyst ratings? I suggest they do – if they want to survive