Real Time Text Analytics

I’ve recently noticed a small buzz building about the notion of “real time” text analytics.  In fact, I’ve come across several vendors talking about it in relation to customer experience and financial trading.  The idea is that these companies analyze a lot of  unstructured data quickly and provide real time  information to the people who need it.  Of course, real time can mean something different depending on the context.  It might mean continuously monitoring customer feedback from multiple sources to improve customer retention.  This could mean analyzing information on an hourly basis.  Or it might mean millisecond response time analysis in the case of monitoring current events to use for trading purposes.   In the first example, millisecond response time may not be necessary.  In the case of financial trading and other activities, it can make a difference.


One vendor that offers this kind of analytical power in a SaaS model is Psydex.  Robin Bloor and I recently had the opportunity to speak to Rob Usey and Don Simpson about Psydex.    Robin has also written about this company in his blog.


Psydex analyzes huge amounts of unstructured feeds to assess the impact of news events.  The company takes in and analyzes feeds from various news sources like Thomson Reuters, Dow Jones, Associated Press, Business Wire as well as social networking sites like Twitter to extract useful information.   It can even pull in TV news feeds and text messaging.  Latency is less than 20 milliseconds to query decades of content.    The secret sauce is the company’s ability to organize streaming content in-memory and around time.  The goal is when an event hits, rather than taking hours or minutes to get information to the person who needs to know, it takes seconds. 


How it works


Psydex organizes information around semantic topics.  These topics are built using rules that represent events, people, themes, places, and so on.  For example oil might be a topic.  The topic oil might include oil, crude oil and the price of oil. Another topic, such as Oil Problem, might incorporate this topic as well as any information relating to spills, explosions, etc. 


The company uses a proprietary grid-based indexing scheme for organizing content in memory with topic models stored separately. These topics are then analyzed for trends and patterns.   Specifically, Psydex uses all of its information to establish a baseline for normal topic noise levels.  The company tracks these topics and can detect when a statistically significant deviation occurs.  The screen shot below illustrates this idea.  This shot shows what a Psydex user might see when a plane crash hits the news wires.  In this case, it is about the plane crash in Japan earlier this week.


In this example, one semantic topic is plane crash.  The topic was built using a rule that includes phrases such as plane is down,plane crashed,plane crash,jet has crashed,helicopter crashed,helicopter crash,plane down,jet crashed,jet crash,plane went down, just crashed,plane has crashed,airplane has crashed,helicopter has crashed,jet has crashed,airliner has crashed,plane just crashed,plan, crashes,flight+crashes,flight+crashed – you get the idea.   There is also another topic called Japan.




The view is a five day hourly view.  Immediately you see a spike for Japan and then two other spikes beginning at 1800 hours.   You can also see the associated content stating that a plane went down in flames.  This content is coming from multiple sources.  You can also see that the noise level is up significantly for the Plane Crash topic.   The user interface also allows you to see potential related topics. In this case, Japan is a related topic.  These are topics that were found in the same articles and/or time slots.  The spikes for the two topics together are shown in blue. 



While this plot shows hourly spikes, the company can take this time interval down to the second.   Psydex sent over a log from yesterday that showed the story about the fedex crash hit the wire at 18:21:47.  Psydex signals were generated at 18:21:48. 


Real time BI and Real Time Text Analytics


There has been a lot of hype about real time BI (dealing with structured data) over the past few years and its use in operational systems.  And, then came complex event processing (again, structured data).   And, now real time text analytics.  You can imagine some good use cases for analyzing news-related text in real time.  Trading is obviously a use case that might require some really fast response time.  Government applications might be another.  Psydex would argue that another use case might be brand management because companies might want to use some piece of news or chatter to update their online advertising campaign.  Or, be the first to know if something negative is being said about your company.    Of course, there are other scenarios in which continuously analyzing text other than news feeds as part of an operational process might be useful, as well.








Five things Verizon Wireless Should Know About Service Management

I was at a Verizon Wireless store late last month.  My mission was to enable my blackberry phone to receive emails while I was in the Bahamas.  I had already spent about a half hour on the phone with a Verizon wireless customer service representative who told me that:


  1. My Subscriber Identity Module (SIM) card wasn’t activated 
  2. I needed to change my billing plan in order to cost-effectively receive email while traveling. 
  3. I would have to go to the Verizon Wireless store to do this.


So, on a Sunday a few weeks back, I trudged down to the store.  As I waited for my name to come to the top of the digital queue, I started thinking about how Verizon Wireless is dealing with service management.  As many of you know, Hurwitz & Associates is in the process of writing our latest book, Service Management for Dummies.  I figured that while I was waiting, at least I could be thinking about something useful.   Actually, that isn’t entirely true.  I was actually too aggravated while I was waiting there to think clearly about it.  I thought more about service management once the whole incident was over.


What is Service Management?


First, a very simple primer on service management.  We define a service as


“A purposeful activity carried out for the benefit of a known target.” 


In this case, I was the target and the activity was to get my emails coming onto my phone when I was overseas.   Each service has inputs and outputs.  The internal activity of the service (i.e. what happens when I asked my question) transforms the inputs into the outcome and it typically involves some skilled participants as well as assets and tools that are used to execute the service.   My request triggered a series of activities which theoretically should have resulted in a positive outcome. 


We define service management as:


“The management of a service to ensure that it meets the critical outcomes the customer values and the stakeholders want to provide.”


In other words, meeting customer expectations is key to a successful service management strategy.  The delivery of a service that meets these expectations can be complex as it often involves people, process, and various layers of technology.   The management of a single service such as the one I described, can also involve multiple component layers.  However, if the goal is to optimize the service this means that every link in the chain needs to work.


The technology to support service management ranges from service desks to monitoring systems to data center optimization.  Here, I am just focusing primarily on the service desk technology and the people and processes that go along with it.  The service desk provides a single point of contact for customers and internal users to report any issues they may have with the IT service.


What Verizon Wireless needs to Improve


As it turned out, it took me over an hour to get my SIM card activated and a new billing plan put on my account.  And, as it turned out, I couldn’t receive my email in the Bahamas anyway because of some new frequency in the Bahamas that didn’t work with my model phone.  That took another 30 minutes to figure out once I was in the Bahamas.  And, customer service never changed my billing plan back to my original plan to reflect this fact, even though I had asked them to.  That took another 20 minutes to resolve once I got back. Whew! 


So, what should this company do better in terms of service desk management and customer support? 


  1. Service Desk Queue.   When I came into the store, I entered my name and information into Verizon’s service desk system at an in-store kiosk.  I put my name on the technical queue and subsequently found out that my problem was really a customer service related problem.  Then, I had to put my name on the customer service queue.  While all of the people on the technical support and sales queue were serviced, I sat there waiting (about 45 minutes) for a customer support person.  There were two customer service reps in the store but the other one was taking a break.  The queue needs to be monitored.  If someone is in the queue for too long (like more than 10 minutes) then either the person “on break” needs to come off of break or a technical service person who was free should have helped – at least to install the SIM card. This monitoring is directly related to KPIs and SLAs described in bullet 4.  
  2. Training.  Once I finally got to speak to a customer service representative she couldn’t figure out how to make the billing change for me.  She had to call over her supervisor and they had a long discussion about it.  What happened to all of that training that Verizon advertises?
  3. Service Desk Notification.  Two things here. 
    • As I mentioned, I first called a customer service representative who directed me to the store.  When my service didn’t work, I called a customer service rep (the international one) again.  When I had a billing question I called yet another customer service representative.  Yet, none of these people seemed to know about any of my calls or visits.  The service desk should be able to track this. It would have moved things along much faster and improved those metrics.
    • While I was waiting for the customer service representative to help me, another man was talking to a rep and said that he wanted to get a new LG phone.  The rep said that he could sell the man one, but wouldn’t activate it because there was a problem with the phone.  When the man questioned this, the rep confessed that he didn’t know what the problem was but said it would be fixed within 48-72 hours and the man should contact the store for updates.  Wrong on several counts.  First, the rep should have known what the problem was. The service desk had notified him about the expected time to repair, but didn’t say much about the problem.   The man already owned the phone – perhaps he was having the same problem.  Second, the rep should have said that he would notify the guy.  This would be easy enough if the guy had a text message feature on his phone.  At least he could have offered.
  4. Key Performance Indicators.  Verizon needs to put some business focused key performance indicators in place that would help them understand whether they are meeting customer expectations.  This shouldn’t be too difficult since it appears to have a service desk.  Obvious metrics include time to service, time spent servicing (by type of issue).  Others might include average time to resolve an incident,  Customer support turnover and I could go on and on.  It would also be helpful if the customer could answer a few customer satisfaction questions at the kiosk on the way out.   That way they could correlate dropped service with incident management.
  5. The Knowledge base.  Two different customer service representatives at Verizon didn’t know that I couldn’t access my email using my model Blackberry in the Bahamas.  Verizon needs to deploy a knowledge base that stores information like this.  If it has a knowledge base, it needs to update it and train its people how to use it.


Clearly, my expectations for service were not met.


Get every new post delivered to your Inbox.

Join 1,710 other followers