When “Mad Men” and Mathematics Collide

Posted on February 3, 2016 by Paul Hofmann

Insights on the Benefit from Text Analytics

  • Understanding humans by understanding text. Data volume on the Internet and elsewhere is growing exponentially, and most of it is human-generated text. This is why text analytics is necessary. If you want to understand people, especially your customers and how they use your product and engage with your organization, then you have to be able to possess a strong capability to analyze text.
  • Context matters. Marketers ( notionally the ” Mad Men of Madison Avenue”) spend a lot of time and money understanding the touch points customers reach on their pre purchase journey. Attribution, in that sense, equals context, and text analytics is a key ingredient in establishing that context. The concept is important not only in marketing but in government, research and development, finance, and elsewhere. Attribution is in effect a form of predictive modeling. It is a way to predict, based on past behaviors, what a customer likely will do. For that we have to combine human sentiment with the trace customers leave on the Internet for example. To understand sentiment we need quite complex text analytics.
  • Live not by text alone. If not for combining text and statistical data, it is unlikely he would have made the breakthroughs he did for an aeronautics client. You will see, at best, half of the picture. After all, combining the two, he states, is essentially what the human brain does.  You have to combine semantics with statistics, I am very deeply convinced of that.

Let’s considers the television show “Mad Men” a snapshot of marketing in transition, one that underscores his point. By the end of the show’s run, its cynical ad men were conceding, if grudgingly, that there was business value in the primitive computer that had been installed in their agency. Of course, the insights gleaned from a computer in 1970 would have been limited, but today, he says, those limits have all but disappeared. In fact nowadays compute power and the plethora of available and capturable data provide tremendous insights. In today’s world, and tomorrow’s too, it is an absolute necessity to take advantage of the ability of machines to predict a customer’s behavior, he remarks.

We have been transitioning from these guys who are a little bit chauvinistic, who infer or assume behavior while they smoke the cigars and drink the booze—and then make a commercial. Gut feeling is being replaced by actual insights derived by analyzing big data.

The exploding field of marketing attributions is at a crossroads where text data and statistics merge to generate deep insights that Madison Avenue could scarcely have dreamed about five decades ago. Customers create a recording and story by leaving digital markers along their unique  engagement with vendors and products. Today, we have fine-tuned tools to read  and analyze those markers. The ad man’s intuition is supplanted by machines that learn.

Before those guys on Madison Avenue had it all in their head and they had to figure it out by inference and lengthy validation cycles. Today, we can really analyze and predict people to understand the feelings and intentions they have about a certain product to enable the marketer to speak directly buyers and consumers.

I had a big revelation some years back while doing a job for a major aerospace defense contractor when I combined statistics along with text analytics and achieved an otherwise unobtainable outcome.

The client wanted to institute a system of preventive aircraft maintenance. The idea was to predict the failure parts before they would actually break. The traditional method involved replacing parts on a timetable—often well before they were worn out. When we do that, we leave money on the table. Applying natural language processing and machine learning, I discovered several critical, undiagnosed problems. Some sensors, for example, consistently burned out early because undetected engines failures generated more electricity than the sensors could handle, but the newly replaced sensors showed no record of this problem. Text in the written maintenance logs were the crucial pointer to the undetected engine failures; this is how combining text analysis and statistics results in insights and value.

It requires no great logical leap to transfer the lesson of that story into the marketing realm. When Mad Men Collide with Mathematics is the holy grail of marketing. We all want to predict why and when a customer will buy, and when the part will fail before it actually breaks, and to do that, text analytics is a critical necessity.

Posted in Technology | Tagged , | Comments Off on When “Mad Men” and Mathematics Collide

Will The Sun Rise Tomorrow?

Posted on April 27, 2014 by Paul Hofmann

Is there a difference between deterministic differential equations and a probabilistic model?

Reading Max Tegmark’s terrific book, “The Mathematical Universe – My Quest for the Ultimate Nature of Reality”, inspired me to state the following fact, which I use to puzzle my friends at parties. Even in classical dynamics we cannot so easily say from observation if a system is deterministic or random. This has deep implications for the answer to the question, will the sun rise tomorrow?

Let’s look at so-called hyperbolic systems. In such a system uncertainties double at a stable rate until the cumulative unknowns destroy all specific information about the system. For such a system the so-called shadowing lemma of mathematics apply. This lemma says that for such systems one cannot tell whether a given system is deterministic or randomly perturbed (like the Saturn rings for example) at a particular scale, if you can observe it only at that scale. As you may know, the KAM Theorem explains the Saturn rings perfectly. Kolmogorov, Arnold and Moser were able to derive the stable tori in the phase space for the Saturn rings.

Hubbard gave once an intuitive example of such systems – angle doubling. “…start with a circle with its center at 0, and take a point on that circle measuring the angle that the radius to that point forms with the x-axis. Now double the angle, and double it again and again. You know the starting point exactly and you know the rules, so of course you can predict where you will be after 50 doublings, say. BUT now imagine that each time you double the angle you jiggle it by “some unmeasurably small amount”, say at the 10 to 15 decimal digit (assuming your computer cuts off numbers after the 15 digit). Now the system has become completely random in the sense that there is nothing you could measure at time zero that still gives you any information whatsoever after 50 doublings…”.

Does this disturb you?

It means that if you were presented with an arbitrarily long list of figures (each 15 decimals) produced by such an angle doubling system you could not tell whether you were looking at a randomly perturbed system or an unperturbed (deterministic) one. The data would be consistent with both explanations. The shadowing lemma tells us that (assuming we can measure angles only with 10 to 15 digits accuracy) there will always be a starting point that, under an unperturbed system will produce the same data as that produced by a randomly perturbed system. Actually, it will not be the same starting point, but you do not know the starting point, therefore you will not be able to choose between the starting point A and the deterministic system, and some unknowable starting point B and the random system.

Does this disturb you now?

There is no way of telling deterministic from random for a wide class of dynamical systems. You can choose between deterministic differential equations and a probabilistic model. The “facts” are consistent with both approaches. This is quite similar to the difference between how my compatriot Schroedinger viewed the universe in quantum mechanics and Heisenberg did. Heisenberg says that the universe is given for all time; we are just discovering more and more about it as times goes on. Schroedinger says that the world is really evolving and we are part of the changing world. Both are right – matrix mechanics and wave equation are mathematically speaking equivalent. We have known the Heisenberg-Schroedinger dichotomy for a while. It is a quite surprising fact that even in classical dynamics we cannot so easily say from observation if a system is deterministic or random.

In other words, we cannot yet answer with certainty if the sun will rise tomorrow. We could use Bayesian reasoning and use the prior observations that the sun came up every morning the last 4.5 billion years. But priors don’t mean much for  non-linear systems. A small perturbation could have huge effects; our planetary system could be a chaotic systems that has been quasi-regular for many cycles and could still fall apart after a small perturbation from outer space.

Posted in Technology | Comments Off on Will The Sun Rise Tomorrow?

Rough Waters for Facebook in the Era of Mobile

Posted on August 14, 2012 by Paul Hofmann

Facebook’s faltering stock price since the IPO shows clearly its has not yet the right business model. The social networks staff selling their stocks this week is the next great threat to its stock price. The decline in market value shows a fundamental strategic problem for the social networkers that want to make the world a better place. The smartphone revolution threatens Facebook’s advertising business.

Facebook is on its way down. The value of its stock has decreased by around 45 percent since it has gone public 90 days ago; it could still fall further. Thursday the holding period for 271 million employee shares will expire. Selling part of the employee stocks will depress the price further.

It’s possible that Facebook gets away unscathed because most employees who want to sell, may have done it in advance by warrant. In this case the end of the holding period would be already priced in. Nevertheless, the stock is likely to remain under pressure because other holding periods are running out in the coming months.

That fact that Facebook has to deal with such mundane things as expiration of the holding period is characteristic for the shift Facebook faces. As recent as May 18th, the date of their IPO, FB has been hailed as the new star. The stock has been so heavily overvalued that the company’s value even after the crash is still more than 48-times its annual income of 2011. FB’s market cap is about the same as behemoths like Colgate Palmolive, Monsanto, Sumitomo, DuPont, or Nike. A market cap of 48 billion dollars seems super optimistic for a company with such an uncertain outlook even when considering that we talk about the touted digital industry.

The share price reflects the hope that Facebook could somehow grow exponentially in the coming years. But the business model of the social network has been shaken to the bones. The boom of Internet-enabled mobile phones is the responsible game changer. End of 2011 a bout 6 Billion people have had a mobile phone. There are more people with a mobile phone than are connected to the electrical grid. Not only Facebook is in trouble but  Google and other web giants too who earn their money mainly with online advertising.

The Dawn of a New Web Era

Gartner estimates that in the coming year the worldwide Internet access via mobile phone and tablet will be for the first time larger than the traffic from the PC at home or the office. Facebook experiences the mobile revolution already; more than half of its roughly 955 million users access FB from their phone. Many of us use mobile devices for our daily FB access. A recent survey of Google users shows similar results for web search.

This leads to the following issues. The page views on smartphones bring only a fraction of the always-connected ad revenue. New business models that compensate for these losses are still under development. I’m not sure that the usual suspects will be able transfer their always-connected Internet market leadership to the mobile web.

We are at the beginning of a new era, the third Internet era of mobile Web and interest networks. The first era were Web portals like Yahoo and Google search engine. Facebook, LinkedIn and Zynga have dominated the second era, the so-called social Web. Each Web era had its special needs. Usually, the rulers of one age had difficulties adjusting to the dawn of a new era. Jay Jamison calls the mobile era the Web 3.0 in his IT blog at TechCrunch.

The Web 1.0 pioneer AOL plays no role any more. Yahoo is a company that needs severe surgery. Google is still brilliant, but it earns its billions almost exclusively with its Web 1.0 business model of Internet search. Its ad sense churn rate is about 70% and its social network Google+ is a far cry from Facebook’s success.

Facebook has announced during its IPO to expand its mobile activities significantly. My friends at FB say that CEO Mark Zuckerberg tests all new features first on his phone. Implementing its mobile strategy Facebook faces similar problems as Google with its social network. The change requires nothing less than a cultural change.

Context on the Mobile Induces A Culture Change for App Design

When surfing the Internet on the phone we expect a very different app behavior than on the laptop though we use the same information from the Web. Mobile services have to be context rich. This means they have to include, location (from the GPS of the phone), user preferences, friends nearby and integrated payment via e-wallet (in the very near future). Mobile apps have to display less information on a small display and have to be usable with fewer clicks. This means they must be very well tailored to the task at hand – be it finding a nearby taxi, or a nearby fish restaurant recommended by friends.

Facebook’s approach seems very different. We are presented with more and more information that pushes us to ever-greater screens with higher resolution in order to display status messages, invitations to parties, games, advertising, chat and much more. Facebook has just transferred this app with its wealth of information from the static web to our mobile. The FB app seems overloaded, while the value of its location and context-aware services is very limited.

Companies that have designed their apps from the beginning for the mobile web deliver much better user experience closer to the customer. A mobile food service is just about spotting personal restaurant recommendations and about showing its users how it is cooked for example. One app per task is the credo of mobile app leaders like Apple, Amazon (Kindle) and Twitter.

Take the mobile photo community Instagram for example. Yahoo’s photo service Flickr and Facebook’s picture upload look shabby compared to Instagram. Instagram has gathered more than 80 million users in roughly two years. Early April Facebook bought the service for a billion dollars.

The New World of Advertising and Interest Networks

The mobile web changes the business models too. Companies who finance themselves mainly through advertising face a problem. User attention on the phone is different from the PC at home. Banners have not been made for the small mobile phone screens.

Advertising has to change from passive consumption to active participation. Take for instance the Groupon++ startup LOCALsense and mutu, a music video aggregator. Both startups integrate the user. LOCALSense rewards the user when he or she passes coupons on to friends; a cool way to learn about a person’s interest network. Who of my friends likes coffee, sailing, or go to the movies? All this information about the interests of my friends is readily available for LOCALsense but very hard to get for FB. How does FB even know what types of friends I have?  How do they know who of my roughly 1000 friends is family, close friend, co-worker or business friend? They ask me to categorize my friends. Well, this is the reason why Google+ has failed with circles. Who on earth is going to keep all these relationships up to date manually? Interest networks go much further by understanding that relationships are multifaceted with regards to how people interact. For example, friends may sail together and be close friends, or friends share a passion for movies and are co-workers. LOCALSense analyzes the connections, interactions and buying behavior of its members with the advanced analytics platform Saffron Technology in real time and automatically. Its easy to monetize this information by feeding it back to sellers.

Mutu aggregates music videos and predicts customer taste in order to offer a Pandora like service to its listeners. Mutu starts out with an innovative business model. Besides ad revenue it earns money by selling back to the producers of music videos what videos people like, how long they watch them and what they share with their friends.

I think there will be two big revenue streams on mobile, gaming and interest networks. End of last year I have been on a panel with Bill Chang, Chief Scientist of Baidu in Shanghai, organized by Erik Brynjolfsson from MIT Sloan School. Bill predicted that gaming would be the most popular mobile apps in China. Gamming apps need context, interest networks and location too.

The successful companies of the information age will generate value by creating information from the ever growing sea of Big Data. Think about GE opening a center for Big Data analytics in the Silicon Valley for 1 billion dollars. GE intends to sell services and the value add it will generate from operations data – railroad traffic and congestions, or usage data of its medical equipment in hospitals.

Marketers know all that intuitively therefore they pay less for mobile ads. 1000 clicks on a mobile ad banner cost 35-40 cents according to the venture capital company Kleiner Perkins. Ads on Web pages sell up to 20 times more expensive.

The lower ad prices are reflected in the balance sheets of web companies like Google and Facebook. Google whose revenue depends more than 90 percent on advertising has shown steadily declining per click sales in the past three quarters; in the last quarter sales have dropped by as much as 16 percent. Google’s countermeasure is it’s free mobile operating system Android. This ensures that the Google has at least access to the handset market, and thus potentially to millions of advertisers. Where is the business model here? So far I see only Samsung making money with Android smart phones.

Facebook’s revenue depends to 85 percent on advertising too. Revenue has slowed down because ad sales and user numbers grow less than some month ago. The increase in ad revenue in 2011 was still 88 percent while it slowed to 45 percent in the first quarter and 32 percent in the second quarter of 2012.

This ailing growth is incompatible with the high expectations for the Facebook stocks. They need a new growth strategy for the mobile web. Transferring overloaded web apps on the mobile Web has shown very modest returns. Recently, Facebook has started to mingle ads with status messages when the user or their friends are befriended with the respective company. This seems the right direction, user interaction using context and the wealth of data FB has gathered about us. According to FB, its making half a million dollars in sales per day with these mingled adds.

Customers rather than clicks

Other companies have been off to a better start when monetizing mobile ads. Those Web 3.0 companies understand marketers asking, why pay for clicks when one can steer customers via mobile ads to a shop, restaurant or service?

Another location-based social app I would like to mention is Waze, an app that combines GPS with a social network. Motorists transfer their speed while commuting to work. Waze calculates road conditions from the speed taking into account reports from fellow drivers and delivers turn by turn navigation that automatically re-routes you when road conditions change. This is a great example of how dynamic interest networks are unlike the static social networks. I don’t even know the people who are in my interest network when driving to work. Still the commuters’ community helps each other to get faster to work.

Actually, Facebook, Google and the other incumbents are in the best staring position to deliver mobile services around interest networks. They have more users than any new Web 3.0 mobile community and therefore command the reach needed to get such services off the ground. However, the incumbents are not the ones driving the mobile revolution. I’m having a déjà vu from the Web portals era when the Web 2.0 newcomers ate the lunch of the Web 1.0 crowed.

Posted in Cloud Computing, Technology | Comments Off on Rough Waters for Facebook in the Era of Mobile

Why Did Google Pay $12 B for Motorola Mobility which has Traded only at $7.7 B?

Posted on August 15, 2011 by Paul Hofmann

The sensational news of the day “Google acquired Motorola Mobility” makes one thing clear; the power between hardware and software manufacturers in this still relatively young market has shifted dramatically. Only recently has Nokia -once the leading mobile phone manufacturer in the world- announced that all their smart phones will run Windows7 while HP has announced to give up its PC and tablet/webOS business.

This entire shift of power has been caused by the upstart Apple. With the iPhone Steve Jobs has taken the software to the surface of the phone and its touch-sensitive screen. Google has taken up this idea with its own mobile OS and made it accessible to a wider market. With manufacturers like HTC and Samsung, Google found partners that can build attractive smartphones, some of which are significantly cheaper than Apple’s flagship device.

Motorola Mobility has switched entirely to Android. After they have built the Droid -one of the first really popular Android devices- the company put all their eggs in the Android basket. Motorola’s runs an Android version on its first tablet computer Xoom too.

Now the search engine giant acquired the manufacturer of the first commercially available mobile phone for $ 12.5 billion in cash. This came as a surprise to most. The shareholders have not foreseen this move for sure; as late as last Friday Motorola was valued at “only” $ 7.7 billion.

Why does Google want Motorola so badly?

I see three reasons:

1)      Control the entire digital distribution chain.

Google wants to be the better Apple in terms of business model. All IT consumer giants aspire to this strategy. No matter whether Google, Amazon, Microsoft, Apple or Sony (and possibly HP), all have the same wish list:

  • Access to individual users via hardware or software, preferably both.
  • An online shop for digital media content and games.
  • A separate digital payment system.
  • Comprehensive access to usage data for optimal advertising and marketing.
  • Preferably a search engine, of course! and
  • A social network, to map the structure of the customers and to refine marketing and advertising accordingly.

It’s all about controlling the entire digital distribution chain to serve the market for digital information and entertainment.

Now, Google completes this wish list through the acquisition of Motorola. With Android, Motorola, Google Books, Google Music Beta, Google TV/Google @ Home (and Motorola’s consumer expertise), YouTube and Android Market for online media libraries Google has all the key elements to control digital distribution. Google Checkout, Google +, Gmail and Google Search complete this offer proliferating their enormously extensive customer data and insights.

Competitors like Apple are missing the search engine, Microsoft has no hardware platform, and Amazon has so far only the Kindle (but that will change soon). All three lack a functioning social network.

Google has now occupied all the strategic positions to dominate the digital future sustainably.

2)      Patents.

Once the deal will be approved Google will have more than 17,000 mobile patents from Motorola Mobility; thousand more applications are still pending. With these patents, Google intends to defend itself against lawsuits from competitors.

Originally US patents were intended to protect the little guy who comes up with a brilliant invention in his garage against robber knights that won’t give the inventor his fair share. Now it seems to me that patents provide government sanctioned monopolies and market protection. Companies that owe IP use it to extort money from companies that make products or offer open SW.

A bitter dispute has broken out over the appearance and functionality of smartphones. Even basic design features like touch screens or hardware buttons can be patented despite the fact that these ideas are obvious to the skilled in the field. Numerous legal battles are fought because mobile devices look sometimes confusingly similar to each other.

For manufacturers of phones and accessories the escalating patent war is an enormous financial risk. For example, last year Apple has sued the Android company HTC, because their smartphones are too similar to the iPhone. In October Motorola took Apple to court for violating 18 patents with its iPhone. After years of dispute (six courts in four countries) Nokia and Apple have settled this year for an undisclosed sum to be paid from Apple to Nokia. Apple has recently banned Samsung from importing its tablet computer into the EU. Microsoft is said to earn $ 5 royalties for every smartphone sold by HTC. Early August Google accused the competition publicly of patent-bullying. Apple, Microsoft and Oracle have teamed up to buy 6,000 patents from bankrupt Nortel Networks. Google fears licensing fees of $ 15 per unit for Android producers. Just after the Nortel deal ($4.5 billion) Google acquired some 1000 patents from IBM.

With the Motorola acquisition the one-time search engine and advertising giant Google has not only acquired a hardware manufacturer but new ammunition for the patent war. Being a young and naive company focused on innovation instead of filing patents Google has found itself in the past on the short end of the patent stick. The Motorola Mobility acquisition gives Google a much bigger stick to hit its attackers.

3)      Enterprise Features.

Google has been behind in security and other features required by enterprise customers. Bare Android doesn’t support many frequently used VPN protocols while Motorola, Samsung, etc. have differentiated themselves by adding the most needed enterprise features on top of Google Android. Google will probably to the smart move and take this features to all Android licensees for free.

What is the snag?

What will Samsung, HTC, Sun and the other Android partners say?

In the past, Google has developed each new version of Android with a specific but changing hardware partner. For example, Android 3.0 (Honeycomb) with Motorola for Xoom. Before, HTC and Samsung had the first mover advantage of building certain devices in cooperation with Google. All other hardware vendors fell behind because the privileged hardware partner could offer the most current software update exclusively for several months. For example, Motorola Droid was the first phone with Android’s built-in street navigation – a not to be underestimated marketing advantage.

In the future this preferred hardware partner will be solely Motorola. The result will be cell phones the hardware of which is not constructed for the requirements of Google’s SW but hardware that is being developed jointly with the SW.

The upstart Apple has perfected this principle with the iPhone. It’s so sexy and successful because hardware and software development are in one hand.

Google claims that Motorola’s competitors won’t be hurt. I wonder how this complex balancing act will play out.

 

Posted in Technology | Tagged , , , | Comments Off on Why Did Google Pay $12 B for Motorola Mobility which has Traded only at $7.7 B?

The Future Of Analytics

Posted on January 23, 2011 by Paul Hofmann

Last Thursday I have been on a panel with Admiral Poindexter on the Future of Analytics by invitation of TiE in North Carolina. TiE is the world’s largest non-profit technology entrepreneurship association.

Manny Aparicio, a good friend of mine, founder and president of Saffron Technologies opened the panel as moderator stating that data analytics is undergoing a huge transformation. Manny said, not only is there a growing demand for analytics to extract more value from data, but analytics must also be easier, faster and more accurate than ever before.

In his opening statement Dr. Poindexter pointed out that the Government misses (much like industry) the capability to bring data from different data bases together and make quantitative statements on future risks. Machine learning technologies like Associative Memory Systems could offer a unified view on disparate data from different data bases and text sources.

So far Analytics have been dominated for rear view analysis of corporate or intelligence data. Sense making and predictive capabilities have been very rarely deployed yet.

As Manny has asked me to bring in a Silicon Valley point of view I have decided to make my case for consumerization of machine learning talking about two of the companies which have received the most press coverage in recent weeks in the Silicon Valley, Groupon and Facebook.

Groupon is said by Fortune to be the company that has reached $ 1 B revenue the fastest ever. Some two years old their market cap was $ 1.35 B in April 2010. They have refused an offer by Google for some $ 6 B. Today their market cap is estimated to be $ 15 B. Why? Groupon bridges the gap between seller and buyer. The seller knows all about his product, e.g. when can it be produced at the lowest price, what are the detailed specifications, etc. The buyer on the other hand has little knowledge about the product.

Understanding the buyers needs and desires Groupon offers coupons that have to be purchased upfront and finds a good local or global producer that meets the customers demand. Groupons ads -witty and capturing- save buyers time to read through product specifications; they are fun to read too. If there are enough coupons bought for the product of the day the seller has to produce it and deliver it at the promised discounted prize; a classical win-win.

Groupon claims to have saved their online customers $ 1 B. In my opinion this is the first real biz model of the internet economy. Groupon offers global and local producers upfront buyer buy-in; the traditional coupon model lacks this risk mitigation for producers. Analytics with strong predictive capabilities are core to Groupon to preselect the right stores and producers and match them to the buyers demand. Of course, human beings take the final decision which product to promote.

The consumerization of AI techniques like machine learning offer analytics at our fingertips supporting the information worker from the sales person to the executive being deployed as associative memory systems, as in-memory data bases or on mobile devices.

Predictive analytics are essential for the productivity of enterprises. Making Sense by connecting seemingly unconnected information like

  • the failed investment by Warren Buffet in Lehman with
  • the reluctance of some European banks to trade Lehman credit swaps with
  • the near collapse of Bear Stearns which caused all the short sellers to turn their attention to Lehman and cause a run on Lehman’s stock with
  • CEO Fuld trying to put a deal together to merge Lehman and Barclays, etc

could have predict the bankruptcy of Lehmann ahead of time. A certain European bank needed 3 weeks to formulate the right query to answer the question, “how many and which of their customers had an exposure bigger than X hundred thousand dollars to Lehman”. This European bank has realized the imminent Lehman bankruptcy only on September 13, 2008 when Timothy F. Geithner, then president of the Federal Reserve Bank of New York called a meeting on the future of Lehman; two days before the actual bankruptcy filing. If this European bank had predictive analytics they would have been able to know about their customer’s exposure before this famous Friday to take appropriate action.

SAP and Saffron Technologies have shown a prototype at SAPs SAPPHIRE 2010 that could have predicted the imminent bankruptcy of Lehman well ahead of time just using public documents. These documents were ingested by SAPs ThingFinder and analyzed and triaged by Saffron’s memory base.

An associative memory base can be used for sense making because it predicts data trends, connects entities and ranks them not just at the document level (like Google) but at sentence level. The Saffron Memory base is like an RDF triple store but its schema less and much faster.

I feel honored to have been on this great panel with the admirable Admiral and fellow physicist.

Posted in Technology | Tagged , | Comments Off on The Future Of Analytics

Will There Be No More CIOs?

Posted on January 23, 2011 by Paul Hofmann

 The Limits of Public Clouds for Business Applications

An overly simplistic reliance on the utility model risks blinding us to the real opportunities and challenges of cloud computing.

Abstract
Descriptions of cloud computing often emphasize the silver lining more than the chances of getting wet. Utility computing offers many benefits, but will the cloud — especially the public cloud — lead to the extinction of CIOs because IT will be consumed as simply as electricity? No doubt, cloud computing is a breakthrough technology that will continue to unleash new innovations and bring new efficiencies and advantages to business. It removes infrastructure and capital expense as a barrier to entry and allows startups to scale up cheaply and rapidly.
On the other hand, enterprises face limitations in using the cloud for high-performance and mission-critical applications such as ERP. Unfortunately, the cloud’s limits are often obscured by all the hype. It’s time to stop looking at the cloud as a panacea. This blog seeks to clear up some misperceptions and help people make better choices.

1. The Sunny Side of the Cloud
Certainly, cloud computing offers many attractive benefits to enterprises. The cloud model moves IT infrastructure from an upfront capital expense to an operational one. Companies can use the cloud for large batch-oriented tasks — those involving large spikes in requirements for processing power — that otherwise would be out of reach or require huge investment. Many enterprises provision computing resources for peak loads, which often exceed average use by a factor of 2 to 10. Consequently, server utilization in datacenters is often as low as 5 to 20 percent. One key benefit of cloud computing is that it spares companies from having to pay for these underutilized resources. Cloud computing shifts the IT burden and associated risks to the vendor, who can spread variations over many customers. Organizations can use the cloud to rapidly scale up or down; they can also buy or release IT resources as needed on a pay-as-you-go model. As one group of researchers from the University of California, Berkeley noted, “This elasticity of resources, without paying a premium for large scale, is unprecedented in the history of IT” (www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf ).

2. Plug and Play?
Cloud proponents often compare utility computing to electrical utilities. One of the most prominent voices behind this argument is Nicholas Carr, author of The Big Switch: Rewiring the World, from Edison to Google (Norton, 2008). Carr hails utility computing as a historic shift similar to the advent of electrical utilities. This utility analogy has taken hold in the public imagination. Although useful, this analogy isn’t entirely accurate because it blinds us to the cloud’s limitations for enterprises. The reality is that cloud computing simply can’t achieve the same plug-and-play simplicity as electricity.

3. The Trade-Offs of the Cloud
Enterprises can expect to face many trade-offs when they move IT into the cloud.

Security
Security is one of the biggest challenges to the cloud model, and it’s often an emotional one as well. Behind the firewall, enterprises have control of their data. In the cloud, they must trust the provider. For organizations whose existence depends upon safeguarding customer data, trade secrets, classified information, or proprietary information, public cloud providers don’t offer sufficient protection. Most providers find it hard, if not impossible, to meet standards for auditablity and comply with legislation such as Sarbanes- Oxley and the Health and Human Services Health Insurance Portability and Accountability Act (HIPAA).

Interoperability and Lock-In
As cloud offerings proliferate, there will be ongoing challenges with interoperability, portability, and migration. In an on premise model, enterprises control their infrastructure and platforms at any time. In the cloud, they’re locked in to a provider and no longer offer unique, and often proprietary, data storage (for example, Google’s BigTable, Amazon’s Dynamo, and Facebook’s Cassandra). Scalable data storage isn’t yet a commodity and is unlikely to be so for a long time due to the fact that there is no simple generic solution for distributed data storage.

Absence of Service-Level Agreements
Another problem is the lack of well defined (SLAs) by cloud providers. What’s the guaranteed uptime? What are the repercussions if the provider fails to meet these standards? What happens to customer data if the company moves to a different provider? Cloud providers offer precious few protections to enterprises that trust all their IT to the cloud.

Performance Instability
The cloud is often touted as a solution for organizations with large variations in computing demands. Less well known is the performance variability in the clouds themselves. Researchers in Australia conducted stress tests to demonstrate that Amazon, Google, and Microsoft suffered from variations in performance and availability due to loads. (www.itnews.com.au/News/153451,stress-tests-rain-on-amazons-cloud.aspx). Another example for the limitations of performance predictability is research by Kossmann et al. (D. Kossmann, T. Kraska, and S. Loesing, “An Evaluation of Alternative Architectures for Transaction Processing in the Cloud,” Proc. SIGMOD Conf., 2010, pp. 579–590). They showed that cloud providers don’t yet deliver electricity-like performance.

Latency and Network Limits
As long as we rely on fiber-optic cables, we’re limited by network speed (unfortunately, the speed of light isn’t amenable to the kind of speed improvements associated with Moore’s law). As applications make ever-more intense use of large volumes of data, data transfer poses an increasing bottleneck. For example, University of California, Berkeley, computer scientists calculated the costs of shipping 10 Tbytes of data from the Bay Area to Amazon in Seattle. Given the average bandwidth, sending this data would take 45 days and cost US$1,000 in network transfer fees. In contrast, shipping 10 1-Tbyte disks overnight would cost only $400.

No Scalable Storage
Cloud computing isn’t simply a matter of adding an infinite number of servers. Some problems and processes can’t be solved simply by adding more nodes — they require different architectures of processing, memory, and storage. Most business applications today rely on consistent transactions supported by RDBMSs, which unfortunately do not scale. The cloud lacks scalable storage with an API as rich as SQL.

Stifling Innovation?
Perhaps the cloud’s biggest limitation is that it might impair innovation. Implemented properly, ERP represents a significant source of competitive advantage, but if ERP becomes a commodity — the cloud model’s central premise — it limits a company’s ability to innovate. IT represents a source of competitive advantage for many organizations. In a 2008 Harvard Business Review article, Andrew McAfee and Erik Brynjolfsson found that competition within the US economy had accelerated to unprecedented levels in the wake of the mainstream adoption of the Internet and commercial enterprise software. The main catalyst was the massive increase in IT power. As the authors write, “a company’s unique business processes can now be propagated with much higher fidelity across the organization by embedding it in enterprise information technology. As a result, an innovator with a better way of doing things can scale up with unprecedented speed to dominate an industry.” Real IT innovation comes from tailoring ERP systems to the unique needs of every company.
Cloud providers offer the smallest common denominator limiting the ability of customers to tailor their software and wringing real competitive advantage from their IT systems. Consider Apple. Its shift from a perpetual license model to the iTunes store’s pay-per-use option allowed it to quadruple revenues in four years. The Apple model depends on tight integration between Apple’s ERP system and the billing engine, which handles 10 million sales per day. It would be difficult, if not impossible, to set up such a tight integration between the cloud’s ERP and Apple’s highly proprietary billing software.

Posted in Cloud Computing, Technology | Tagged , | Comments Off on Will There Be No More CIOs?