“Social Media Analytics Expert Interview Series: Part 2” is conducted by the Chief Editor of Text Analytics news, Ezra Steinberg. The interview panel includes: Meta Brown, General Manager of Analytics at LinguaSys; Christine Campbell, Director of Marketing at Socialware; and Pirouz Nilforoush, President & Co-Founder of NetShelter Technology Media. All three interviewees will be speaking at the Social Media Analytics Summit in San Francisco in April. The interview sheds some light on customer interaction; some helpful questions and responses from the interview follow:

 “USM:  What do you believe the average consumer thinks about companies’ social media listening initiatives?

Nilforoush (Netshelter): I think the average consumer is confused as to why different brands are initiating conversations with them online that can resemble advertising or spam. Brands need to focus their efforts around engaging their top influencers, rather than trying to engage with every single person that has something to say about their brand. It is not a scalable model for the brands and can be annoying for the end user. Instead, brands should focus their efforts on the people that have the biggest impact on their brand. These influencers will do the work for brands on their own and impact the masses.

USM:  What would you tell someone who is thinking about employing social media analytics for their company?

Brown (LinguaSys): Start with just one narrow project tied to a specific business problem. Choose something where you feel confident that quick improvement is possible. Plan carefully – what’s the path from data collection to analytics to action to returns? Give yourself the best opportunity to succeed – don’t begin until you have made a plan that gives you a way to demonstrate measurable value for your investment in social analytics!

The interview focuses on planning for implementation of social media analytics and consumer’s thoughts on the topic. Many organizations would benefit by considering the opinions and thoughts provided by these leaders in social media. The full interview can be found here and can give insight on building relationships via social media and what to anticipate during the process.

Andrea Hayden, February 02, 2012

PolySpot is a provider of open search solutions. The company offers a robust and innovative architecture which supports search-centric applications accessible from any device connected to a client’s network.

I was interested in Mr. Andre’s view of PolySpot. The search and content processing sector is in transition, and the role of open source solutions continues to gain traction. He told me:

PolySpot’s agile framework, its use of open source technology like Lucene, and a focus on putting information in the business work flow. Olivier Lefassy, David Fischer – our CTO – and I had designed some interesting ideas, and I was eager to fine tune these elements into a business model that would propel PolySpot over the hurdles which cause many enterprise information solutions to fail.

With open source making in roads at IBM and other major technology providers, I asked about Mr. Andre’s involvement in the “communities” which play an important role in the sector. He told me:

When I was board member at Talend, a very successful French initiative in the ETL [extract, transform, load] segment from inception in 2006 to December 2010, I came to understand the potential of open source software. PolySpot gives me a chance to leverage my knowledge about fast growth, high potential companies, open source software, and the “big data” opportunity around us. I think you can say that data management and information are woven throughout my business fabric.

The PolySpot approach boasts a robust framework. I asked what PolySpot has constructed around Lucene, the open source search system:

We build the connectors I mentioned before and a connector software development kit. We engineered out proprietary transformation and enrichment platform (that’s the Sense Builder components) which adds intelligence to raw information. We also developed a very innovative end to end administration console enabling to design and maintain search applications with no particular technical skill, this eases Lucene and Solr configuration but also amplifies the search functionalities provided by Solr. Last, we have added display modules, information views, and graphical user interfaces. These can easily be customized. To make it brief, PolySpot delivers the first end-to-end packaged search infrastructure over Lucene and SOLR core technologies.

After seeing several demonstrations of client deployments, I was impressed with the PolySpot technology. To learn more about PolySpot’s solutions and technical approach, navigate to www.polyspot.com. The full text of the interview with Mr. Andre is located in the ArnoldIT’s series Search Wizards Speak at this link.

Stephen E Arnold, December 13, 2011

Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search

You can find the interview at this link. The interview carries the date of September 27, 2011. Our error. At age 67, I lose my pen several times a day.

Stephen E Arnold, December 11, 2011

In October 2011, I had a chance to talk informally with Dr. Jerry Lucas, an expert in telecommunications and the owner of the Telestrategies conference series. I was quite interested in his views about content processing. His interest spans text and the large volumes of information that accrue in modern telecommunications systems. One theme which threaded thought his observations was the large volume of data that is now available in digital form. I don’t want to denigrate the commercial services who chit chat about “big data” for figuring out which soap detergent is perceived as having a “smiley face” on the brand. I do want to point out that the Telestrategies’ conferences are designed for law enforcement agencies, intelligence professionals, and practitioners who either work as advisors to agencies or as product developers. Put that your AtomicPR water balloon, insert XML tags, and spam the connected world.

The full text of the interview with him appears as part of the Search Wizards Speak series, which is the largest, free collection of first person narratives about information retrieval. The full text of my conversation with him is at this link. The master index for the series is available on the Beyond Search Web site at Wizards Index.

I wanted to highlight two points Dr. Lucas made in our discussion.

First, I asked him, “What is your view of the challenges flows of digital information pose to government professionals working in law enforcement or the intelligence community?” He told me:

First and foremost are the lack of updated laws creating new lawful interception mandates. In the US the last technical mandate law passed by Congress was the Communications Assistance for Law Enforcement Act. CALEA was passed in 1994 and enacted in 1995. The key players providing today’s communication services used by bad guys—specifically, Apple, Facebook, Google, Second Life, Skype, etc.–are not covered by CALEA mandates nor any other interception assistance laws. These companies have to respond to court orders but these companies don’t have to deploy any infrastructure features to assist law enforcement. I think this is a challenge which must be resolved. A second big challenge law enforcement and intelligence professionals is the lack of educational and budget support by their senior management. As you know, today’s senior management developed professionally in their careers depending on voice calls and e-mail messaging as their prime electronic communications tools. Today many senior managers still make phone calls and send e-mails during working hours and likely watch TV during off hours. So here is my point. To understand what’s needed to police a community you have to live in that community. I call this Policing 101. But Today’s senior managers usually don’t live in the Facebook, Twitter, Skype, Second Life and other cyber space environments as part of their every day activities.

Second, I asked Dr. Lucas, “You have a unique vantage point on some quite interesting technologies. If you were to advise a developer at a large firm specializing in digital information analysis, what would be the three most important features the company should include in their next product release?”

I know you are aware of the phenomenal requirements regarding data privacy or who can access data in a law enforcement agency. Privacy policies and safeguards for open source search in an enterprise can be very lax with regard to a law enforcement agency. Data gathered on a bad guy from a communications service provides under a court order is not fair game for searching over time by law enforcement professional. Those data may have to be erased over time and more. So compliance is an essential characteristic of many products and service. Second, product feature to consider is interoperability with legacy lawful intercept and intelligence gathering products. Interoperability is very important. So called “fork lifts” are rare events in this space and no one in this space wants to see an additional screen introduced in the central monitoring center. I want to emphasize that the user interface must be simple and shouldn’t require the user to be highly computer literate.

I found that the Apple influence is extending beyond the iPhone and iPad crowd. For more of Dr. Lucas’ insights and views, point your browser to the Dr. Jerry Lucas Interview. Information about Telestrategies is located at www.ISSWorldTraining.com.

Stephen E Arnold, November 14, 2011

Sponsored by Pandia.com

OpenSearchServer project started in a French B2B media group in 2007. The company was looking for a search solution. The project emerged when a group of engineers became frustrated with the commercial search solutions.

Raphael Perez said:

Because no available solution was available at a decent price or offering all wished features, decision were made to create an in-house solution as an open source project based on Lucene. Emmanuel Keller, then the chief information officer, led the projects, and after two years of work more than 12 applications were installed and providing high value results. In December 2009, Emmanuel purchased the rights of the solution and formed a company to develop the community and offer them high level professional services, support, community management. It was the start of the story.

Today OpenSearchServer is one of a number of firms using Lucene as a component of a commercial enterprise search solution. One of the value adds his engineering team has crafted is a Classifier. He told me:

One a major module our Enterprise customers appreciate is called Classifier. It brings a very innovative set of features for applications with automatic classifications, matching and that are very appreciated in many businesses. Offering this module helps us to bring a nice differentiation for customers. Also we offer log reporting tools and a SOAP Web service.

The firm has a number of clients, including a rich media firm, an investment firm, and the vehicle information vendor ETAI, an Infopro group company.

You can read the full text of the exclusive interview in the ArnoldIT.com enterprise search service, Search Wizards Speak. Search Wizards Speak is the largest collection of first person narratives about search, content processing, and analytics available without charge. There are more than 50 interviews available in the series.

For more information about OpenSearchServer, navigate to the company’s Web site.

Stephen E Arnold, October 31, 2011

Sponsored by Pandia.com

Dr. Ric Upton will be leading Digital Reasoning’s Washington, DC area office and team in this exciting time for the company. Their product Synthesys is exactly what analysts require in this era of ever-amassing data.

While many other firms offering intelligence software focus on an aspect of entity extraction, Synthesys provides analysts with a comprehensive package for automating the interpretation of big data when the work of search and content processing systems has been undone.

In an exclusive  Arnoldit.com interview, Upton revealed how Digital Reasoning deals with such high volumes of real time information. He said:

[O]ur processing and analytics often have to complement these high volume data flows. We do this in part through judicious use of cloud-based processing augmented by intelligent methods of processing and storing data as it becomes available so that we can avoid the need to perform batch processing or redundant processing of previously-captured data.

The real value is their focus on content centric analytics instead of using statistical algorithms to analyze structured data. Essentially, they decipher the subtext and implicit meanings of content that doesn’t have to be well-structured. The real feat in this is that Digital Reasoning can automate this analysis without any data preparation.

Without Digital Reasoning’s systematic interpretation of data, analysts and clients would actually have to spend hours upon hours of time reading and comprehending content.

Upton shared the reasons why clients have typically used their software:

Our ability to automate understanding is critical to customers with concerns about time, accuracy, completeness, or even the ability to leverage the massive amount of data they have generated.

Serving as an intermediary between the raw data and analysts in the business process, this software has the capabilities to understand the subtleties of the human language. Synthesys can understand the underlying messages in the context of the content’s medium—whether it is a blog, a tweet, or an SMS.

In the interview, Upton sheds insight into how this rich entity extraction manifests itself:

We don’t just extract a name, we can develop and create a persona – the sum of what a person is called, where they have been and when, their relationships with other persona, their behaviors over time, etc.

Digital Reasoning is already looking towards the future, which forecasts that other media such as video and audio sources hold clout as data. As they work on developing methods to analyze these structures, competitors’ opportunities to dominate this field dwindle away.

Megan Feil, October 5, 2011

Sponsored by Pandia.com

John Steinhauer, vice president of technology, Search Technologies

John Steinhauer

On August 8, I spoke with John Steinhauer,  vice president of technology of Search Technologies. Before joining Search Technologies, Mr. Steinhauer was the director of product management at Convera. He attended Boston University and the University of Chicago. At Search Technologies, Mr. Steinhauer is Responsible for the day-to-day direction of all technical and customer delivery operations. He manages a growing team of more than 75 engineers and project managers. Mr. Steinhauer is one of the most experienced project directors in the enterprise search space, having been involved with hundreds of sophisticated search implementations for commercial and government clients. The full text of the interview appears below.

What’s your role at Search Technologies?

Search Technologies is an IT services provider focused on search engines. Working with search engines is essentially all we do. We’re technology independent and work with most of the leading vendors, and with open source. The things we do with search engines covers a broad spectrum – from helping companies in need of some expert resources to deliver a project on time, to fully inclusive development projects where we analyze, architect, develop and implement a new search-based solution for a customer, and then provide a fully managed service to administer and maintain the application. If required, we can also host it for the customer, at one of our hosting facilities or in the cloud.

My title is VP, Technology and I am one of the three original founders of the company and have been in the search engine business full-time since 1997. I am responsible for the technical organization, comprised of 70+ people, including Professional Services, Engineering, and Technical Support.

From your point of view, what do customers value most about your services?

We bring hard-won experience to customer projects and a deep knowledge of what works and where the difficult issues lie. Our partners, the major search vendors, sometimes find it difficult to be pragmatic, even where they have their own implementation departments, because their primary focus is their software licensing business. That’s not a criticism. As with most enterprise software sectors, license fees pay for all of the valuable research & development that the vendors put in to keep the industry moving forward. But it does mean that in a typical services engagement, less emphasis is put on the need for implementation planning, and ongoing processes to maintain and fine-tune the search application. We focus only on those elements, and this benefits both customers, who get more from their investment, and search engine partners who end up with happier customers.

In your role as VP of Technology, what achievements are you most proud of?

I’m proud that we have built a company with happy customers, happy employees, and good profits. I’m also proud that we’ve delivered some massively complex projects on time and on budget, even after others have tried and failed. It is gratifying that we have ongoing, multi-year relationships with household names such as the US Government Printing Office, Library of Congress, Comcast, the BBC, and Yellowpages.com.

But our primary achievement is probably the level of expertise of our personnel, along with the methodologies and best practices they use that are now embedded into our company culture. When we engage with customers, we bring experience and proven methodologies with us. That mitigates risks and saves money for customers.

Do you recommend search engines to customers?

Occasionally, but only after conducting what we call an “Assessment”. We start from first principles and understand the customer’s circumstances; business needs, data sets, user requirements, infrastructure, existing licensing arrangements, etc. Based on a full knowledge of those issues, we offer independent advice and product recommendations including, where appropriate, open source alternatives.

So you also work with customers who have already chosen a search engine?

This is our primary business. Often, our initial engagement with a customer is to solve a problem; they’ve acquired a software license, spent significant time and money on implementation and are having technical problems and/or trouble meeting their deadlines and budgets. Problems include poor relevancy, performance and scaling issues, security issues, data complexity issues, etc. Probably 70% of our customers first engaged with us by asking us to look at a narrow problem and solve it. Once they discover what we can do and how cost effective we are, they typically expand the scope into implementation of the full solution. We help people to implement best practices to reduce complexity and ownership cost, while dramatically improving the quality of the search service.

So, what’s your secret sauce?

With search projects, usually the secret sauce is that there is no secret sauce. Success is down to hard work and execution at the detail level.

What makes Search Technologies unique?

Sure. If there is any secret to building great search applications, it is usually in showing greater respect for the data and how best to process and enhance it to enable sophisticated search features to work effectively through the front end. That and just experience from hundreds of search application development projects. When a customer hires a Search Technologies Engineer to participate in their project, they are not just getting a well-trained, hard working and hugely experienced individual who writes good code, they are getting access to 80+ technical colleagues in the background with more than 40,000 person-days experience on search projects. We’re great at sharing experiences and best practices – we’ve worked hard at that since the beginning. Also, our staff turnover is really low. People who like working with search engines like it here, and they tend to stick around. That huge body of experience is our differentiation.

So you’re pure services, no software of your own?

In customer engagements we’re pure services. That’s our business. But as a company of largely technical people, of course we’ve developed software along the way. But we do so for the purposes of making our implementation services more efficient, and our support and maintenance services more reliable and sustainable.

Where is the search engine industry heading?

There are now two 800 pound Gorillas in the market, called Microsoft and Google. That’s a big difference from the somewhat fractious market that existed for 10 years ago. That will certainly make it harder for smaller vendors to find oxygen. But at the same time, these very large companies have their own agendas for what features and platforms matter for them and their customers. They will not attempt to be all things to all prospective customers in the same way that smaller hungrier vendors have. In theory this should leave gaps for either products or services companies to fill where specific and relatively sophisticated capabilities are required. We see those requirements all over the place.

Open source (primarily SOLR/Lucene) is making major inroads too. We are seeing a lot of large companies move in this direction.

So is innovation dead?

Not at all. Actually we see lots of companies doing really cool and innovative things with search. Many people have been operating on the assumption that search software would reach a sort of commodity state. Analysts have predicted this for years, that once all the hard problems had been solved, then all search engines would have equivalent capabilities and compete on price. What we’re seeing is very different from that. People are realizing that these problems can’t just be solved and then packaged into an off the shelf solution.

Instead the software vendors are putting a ring fence around the core search functionality and then letting integrators and smart customers go from there. With search, there are now some firmly established basics: Platforms need good indexing pipelines, relevancy algorithms that can be tweaked to suit the audience, navigation options based on metadata, readable, insightful results summaries. But that’s just the starting point for great search.

Here’s an example we’ve been involved with recently. Auto-completion functions have been around for years. You start the search clue, the system suggests what you’re looking for, to help you complete it more quickly. We’ve recently implemented some innovative new ways of doing this, working with a customer who has a specific business need. This includes relevancy ranking and tweaking of auto-completions suggestions, and the inclusion of industry jargon. Influencing search behavior in this way not only helps the customer to provide a very efficient search service, it also supports business goals by promoting particular products and services in context. Think of it as a form or relevancy tuning, but applicable to the search clue and not just the results. These are small tweaks that can have a big impact on the customer’s bottom line.

Another big innovation is SaaS models for search applications. This has also been talked about for years, but is really just now coming into focus in practical ways that customers can leverage.

I understand that your business is growing. Where are you heading and what might Search Technologies look like in a couple of years?

Perhaps the most pleasing thing of all for me personally, is that a lot of our growth, which is averaging 20%+ year on year, comes from perpetuating existing relationships with customers. This speaks well for customer satisfaction levels. We’ve just renewed our Microsoft GOLD partner status, and as a part of that, we conduct a customer satisfaction survey and share the results with Microsoft. The returns this year have been really great. So one of the places we are heading is to build ever longer, deeper relationships with companies for who search is a critical application. We initially engaged with all of our largest customers by providing a few consultant-days of search expertise and implementation services. Today, we provide these same customers with turnkey design and implementation, hosting services, and “hands-off” managed services where all the customer does is use the search application and focus on their core business. This model works really well. Through our experience and focus on search we can run search systems very efficiently and provide a consistently excellent search experience to the customer’s user community. In the future we’ll do a lot more of this.

Finally, tell me something about yourself

I grew up in Michigan, have lived in Chicago, Boston, DC, London and now in San Diego. The best thing about that is I can ride my bike to work most mornings year round. I have two boys (4 years old and 6 months old), neither of whom have the slightest clue what a Michigan winter entails. I expect that will continue for the foreseeable future.

Don C Anderson, August 29, 2011

Sponsored by Search Technologies

The company has offices throughout Europe, the Middle East, and in the United States. The firm offers solutions in market sentiment, reputation management, risk assessment, crisis management, and competitive intelligence.

In the wide ranging interview, Ms. Athayde mentioned that she had been recognized as an exceptional manager, but she was quick to give credit to her staff and her chief technical officer, who was involved in the forward looking Datops SA content analytics service, now absorbed into the LexisNexis organization.

I asked her what pulled her into the vortex of content processing and analytics. She told me:

My background is business and marketing management in the sports field. In my first professional experience, I had to face major challenges in communication and marketing working for the International Olympic Committee. The amount of information published on those subjects was so huge that the first challenge was to solve the infoglut: not only to search for relevant information and build a list, but to understand opinions and assess reputation at an international level….I decided to fund a company to deliver a solution that could make use of information in textual form, what most people call unstructured data. But I knew that the information had to be presented in a way that a decision maker could actually use. Data dumps and row after row of numbers usually mean no one can tell what’s important without spending minutes, maybe hours deciphering the outputs.

I asked her about the firm’s technical plumbing. She replied:

The architecture of our own crawling system is based on proprietary methods to define and tune search scenarios. The “plumbing” is a fully scalable architecture which distributes tasks to schedulers. The content is processed, and we syndicate results. We use what we call “a source monitoring approach” which makes use of standard Web scraping methods. However, we have developed our own methods to adjust the scraping technology to each source in order to search all available documents. We extract metadata and relevant content from each page or content object.  Only documents which have been assessed as fresh are processed and provided to users. This assessment is done by a proprietary algorithm based on rules involving such factors as the publication date. This means that each document collected by Spotter’s tracking and monitoring system is stamped with a publication date. This date is extracted by the Web scraping technology, from the document content. The type of behavior of the source; that is, the source has a known update cycle. We analyze the text content of the document. And we use the date and time stamp on the document itself.

Anyone who has tried to use the dates provided in some commercial systems realizes that without accurate time context, much information is essentially useless without additional research and analysis.

To read the complete interview with Ms. Athayde, point your browser to the full text of our discussion. More information about Spotter SA is available at the firm’s Web site www.spotter.com.

Stephen E Arnold, August 16, 2011

Freebie but you may support our efforts by buying a copy of The New Landscape of Enterprise Search

Ms Hlava’s 33 years in the search industry informed her observations on current trends, three of which she sees as significant: Cloud and Software as a Service (SaaS) computing, term mining, and the demand for metadata.

The move to the Cloud and SaaS computing demands more of our hardware, not less, Hlava insists. In particular, broadband networks are struggling to keep up. One advantage of the shift is a declining need to navigate labyrinths of hardware, software, and even internal politics on the client side. Other pluses are the motion toward increased data sharing and service enhancement. Also, more ways to maintain security and intellectual property rights are on the horizon.

She says that term mining is “a process involving conceptual extraction using thesaurus terms and their synonyms with a rule-base, then looking for occurrences to create more detailed data maps,” according to Hlava. Her company leverages this concept to make the most of clients’ large data sets. She is interested in new angles like mashups, data fusion, visualization, linked data, and personalization, but with a caveat: success in all these depends on the quality of the data itself. “Rotten data gives rotten results.”

Ms. Hlava regards taxonomies and other metadata enrichment as the way to bring efficiency to our searches. In that realm, the benefits have only begun:

“In terms of taxonomies and search, ‘I think we have just scratched the surface. With good data, our clients are in a good position to do an incredible array of new and interesting things. Good taxonomies take everything to the next level, forming the basis of not only mashups, but also author networks, project collaborations, deeper and better information retrieval,’ she concluded.”

Wise words from a wise woman. We look forward to observing these predictions take shape as the search industry moves forward. The interview with Margie Hlava, can be read in full here.

Access Innovations offers a wide range of content management services. The company has been building its semantic-based solutions for over thirty years and prides itself on its unique tool set and experienced personnel.

Stephen E Arnold, August 4, 2011

Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search

SearchLion reflects a different approach from the keyword method that is quite different from the brute force approach used by the early Web search systems. In fact, the tagline for the service is “The New Way to Search.” To make certain a user understands the new direction the company is taking, the splash page offers the greeting, “Welcome to 21st century Web search.”

I ran queries on the system, which offers relevance ranked search results from Google and Yahoo. I found the output useful. When I clicked on the Open button next to an entry in the results list, the system displayed in the browser a preview of the Web page. IN addition, other hits are listed in the right hand column of the display which are related to the result I “opened”.

Source: www.searchlion.com

When I spoke with Mr. Arnstein, I was curious about the inspiration for the interface, which puts the focus on content, not ads. The idea for the content centric interface was, according to Mr. Arnstein, a result of his work in the financial services sector. Screens for traders, for example, are filled with information important to the task at hand. He said:

My first professional background was as a Technical Futures trader. I spent several years making a living day trading equity futures from my own private office. When you trade equities, you use software which makes use of every inch of screen space. So, for example, you can have a screen which is evenly split into four equity charts. The concept is simple: the more data you can access on the screen, the more productive you will be. I was accustomed to the efficiency of trading software. I realized that when searching and browsing the web, there were big parts of the screen going to waste. So I sought to find ways to use the available screen space to give the user more data.

He noted:

We think this fosters switching back and forth which is time consuming and can be confusing to many users. If you can have results and the source both on the same screen, our research suggests that users can find what they looking for much more quickly. In addition to opening the live sites, you can also save your searches together with the live sites. When you then load a search from your saved list, the live sites open automatically. We’ve used the same concepts without our MultiView features. Instead of the live Web site, MultiView uses the blank areas of the page to show you a different type of search result; for example, images, news, videos, etc.

The technical challenges were “interesting”, according to Mr. Arnstein. He added:

When showing more information, your browser will be using more resources. It took a lot of work and innovation to make sure the user gets his additional information, whether the live sites or the various types of results and still be extremely fast.

You can read the full interview with Mr. Arnstein on the ArnoldIT.com subsite, Search Wizards Speak. The Search Lion site is at http://www.searchlion.com.

Stephen E Arnold, August 2, 2011

Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search