Detailed Advice on Enterprise Scale Semantic Systems
February 1, 2013
In his AI3 blog, semantic technology pro Mike Bergman presents a new series of articles on “Enterprise-Scale Semantic Systems.” Bergman and his business partner recently gained some eye-opening experience while deploying enterprise-wide semantic systems, and wanted to share their new insights with the world.
The article starts with some history on semantic technology, beginning with the World Wide Web Consortium’s (W3C) adoption of the Resource Description Framework as a standard in 1999. Bergman describes early developments and high hopes for the technology, as well as certain disappointments that followed. Once one turns away from the unrealized grand visions, however, the actual possibilities are quite promising, if not so dramatic. Though information management within the enterprise remains problematic, Bergman sees reason for hope:
“Daily we see success of semantic technologies in multiple locations, and the market is coming to understand the uses and potential benefits. The benefits of graph-based knowledge structures in search and recommendation systems are becoming accepted. We see how basic search is being enhanced with entity recognition and characterization, as well as richer links between entities. The ability of the RDF data model and ontologies to act as integration frameworks is no longer an assertion, but a fact. Despite the despair of some semantic advocates, the market place is increasingly understanding the concepts and potential of semantic technologies.”
While insisting that there is good money to be made in this field, Bergman lists some factors which continue to make implementing enterprise-wide semantics a challenge. For example, many businesses still doubt that it is a wise investment, though he says the demonstration of improved search functionality often seals the deal. Not surprisingly, budget constraints are another prominent hurdle.
If this first article is any indication, expect the series to include a lot of detailed explanations and tips for approaching enterprise-scale semantic systems. A worthwhile read for anyone in that field.
Cynthia Murrell, February 01, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Exclusive Interview: Miles Kehoe, LucidWorks
January 30, 2013
Miles Kehoe, formerly a senior manager at Verity and then the founder of New Idea Engineering, joined LucidWorks in late 2012. I worked with Miles on a project and found him a top notch resource for search and the tough technical area which was our concern.
I was able to interview Miles Kehoe on January 25, 2013. He was forthcoming and offered me insights which I found fresh and practical. For example, he told me:
You know I come from a ‘platform neutral’ background, and I know many of the folks involved with ElasticSearch. Their product addresses many of the shortcomings in Solr 3.x, and a year or two ago that would have been a coup. But now, Solr 4 completely addresses those shortcomings, and then some, with SolrCloud and Zoo Keeper. ES says it doesn’t require a pesky ‘schema’ to define fields; and when you’re playing with a product for the first time, that is kind of nice. On the other hand, folks I know who have attempted production projects with ES tell me there’s no way you want to go into production without a schema. Apache Lucene and Solr enjoy a much larger community of developers. If you check the Wikipedia page, you’ll see that Lucene and Solr both list the Apache Software Foundation as the developer; Elastic Search lists a single developer, who it turns out, has made the vast majority of updates to date. While it is based on Apache Lucene, Elastic Search is not an Apache project. Both products support RESTful API usage, but Elastic requires all transactions to use JSON. Solr supports JSON as well, but goes beyond to support transactions in many formats including XML, Java, PHP, CSV and Python. This lets you write applications to interact with Solr in any language and with any protocol you want to use. But the most noticeable difference is that Solr has an awesome Web Based Admin UI, ES doesn’t. If you’re only writing code, you might not care, but the second a project is handed over to an Admin group they’re bound to notice! It makes me smile every time somebody says ES and “ease of use” in the same sentence – you remember the MS DOS prompt back in 1990? Although early adopters enjoyed that “simplicity”, business people preferred mouse-based systems like the Mac and Windows. We’re seeing this play out all over again – busy IT people want an admin UI – they don’t want to spend all day at what amounts to a “web command line”, stitching together URLs and JSON commands.
I found this comment prescient. I learned about a possible issue triggered by ElasticSearch in “Github Search Exposes Passwords Then Crashes.”
I pressed Mr. Kehoe for key points of differentiation in open source search. I pointed out that every vendor is rushing to embrace open source search. Some do it with lights flashing like IBM and others operate in a lower profile manner like Attivio. He told me:
Just as we have different products and services for our customers, we can customize our engagements to meet our customers’ needs. Some of our customers want to have deep product expertise in-house, and with training, best practice and advisory consulting, and operations/production consulting, we help them come up to speed. We also provide ongoing technical and production support for mission critical applications – just last month an eCommerce site ran into production problems on the Friday afternoon before Christmas. We were able to help them out and have them at full capacity before dinner. Not to dwell on it, but what sets LucidWorks apart is the people. We employ a large number of the team that created and enhances Lucene and Solr including Grant Ingersoll, Steve Rowe and Yonik Seeley. We also have significant expertise on the business side as well. At the top, Paul Doscher grew Exalead from an unknown firm into a major enterprise search player over just a few years; my former business partner Mark Bennett and I have built up deep understanding of search since our Verity days in the early 1990s.
Important information for those analyzing search systems I believe.
You can read the full text of the interview on the ArnoldIT Search Wizards Speak series at http://goo.gl/31682. Search Wizards Speak is the largest, no cost, freely available collection of interviews with experts in search and content processing. There are more than 60 interviews available. You can find the full series listing at http://www.arnoldit.com/search-wizards-speak/ and http://arnoldit.com/wordpress/wizards-index/.
Stephen E Arnold, January 30, 2013
Sponsored by Dumante.com
Liferay is Not Dead Yet
January 29, 2013
Gartner recognizes Liferay as a leader in portal technology. And yet, the technology is offers some would say is doomed to fail and on its way out. However, Liferay keeps carrying on. See what tricks Liferay has up its proverbial sleeve in The Register article, “Liferay’s Not Dead Yet – But What’s Keeping it Alive?”
The author begins:
“The enterprise portal market should have died years ago . . . the market was being written off almost as soon as it began. More bluntly, in a market that was limping to single-digit growth in 2008, with growth stalling since then, how could an open-source player like Liferay hope to survive, particularly given its penchant for using the company as a vehicle for doing as much social good as company profit? And yet Liferay is steering toward $100m in revenues, with financials that look dramatically better than competitors like Jive Software.”
The article goes on to talk about the humanitarian work that Liferay employees are engaged in, with the support of the company. Liferay appears to be here to stay, but so does many other open source based enterprise solutions. LucidWorks is another enterprise option that is beating the odds; a thriving company built on open source. Explore their LucidWorks Big Data and LucidWorks Search for solutions that could affordably and seamlessly bolster your enterprise.
Emily Rae Aldridge, January 29, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
List of Significant Open Source Programs Neglects Search Engines
January 28, 2013
Zdnet’s recent article focusing on listing, “The 10 Oldest Significant Open Source Programs,” still in popular usage today becomes redundant and neglects to mention other, more relevant projects. Open source software and freeware projects have been influencing software development since the early days of computers.
According to the article:
“Both concepts were actually used long before proprietary software showed up. As Richard M. Stallman, (rms) free software’s founder noted, ‘When I started working at the MIT Artificial Intelligence Lab in 1971, I became part of a software-sharing community that had existed for many years. Sharing of software was not limited to our particular community; it is as old as computers […]'”
Linux has certainly had incredible success as the foundation for the internet and the most ported operating system in the world, running on everything from Android devices to super computers. Python has also proven its impact by becoming the fastest growing open source programming language.
While the article goes on to list several other programming languages and another operating system, I cannot help but notice the lack of open source search engine and indexing software. Lucene and Solr have been around since 1999 and 2004, respectively. These projects merged in March 2010, and have just received a robust update. Not only are these programs currently still in use, but they are making strides towards solving the search problems that plague big enterprise.
Michael Cole, January 28, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Enterprise Organizations Still Have Petabytes Left to Crunch
January 24, 2013
Enterprise organizations are quickly losing time for which they can gain a competitive advantage by using big data solutions to churn insights out of raw and unenriched petabytes. Crowdshifter discusses the amount left unmined and available for the crunching in “Behold the Untapped Big Data Gap.”
An IDC study referenced in the article reported that 23% of data within the digital universe of 2012 could be useful for big data collection and analysis purposes if tagged. However, there is a huge gap in the amount that has been tagged versus the amount that remains without semantic enrichment. Only 3% has been tagged and only .5% has been analyzed.
The article discusses a potential reliance on good old intuition to drive decisions within our data-driven age. Christa Carone, the CMO of Xerox states:
‘I wouldn’t want to give up the data that helps us make fact-based decisions quickly. But I fear that marketers’ access to and obsession with measuring everything takes away from the business of real marketing. It’s impossible to measure ‘squishier’ meaningful intangibles, such as human emotion, personal connection and the occasional ‘ahhhh’ moment. Those things often come with a marketer’s intuition, and they deliver big-time. To me, this means trust your gut even while as you’re trying to embrace Big Data.’
Trusting intuition works after a certain point, but to reach the point to make an informed decision requires looking as a massive amount of data points in our ever-connected world. Information delivery technologies like PolySpot helps decisions makers get to that point.
Megan Feil, January 24, 2012
Sponsored by ArnoldIT.com, developer of Beyond Search
PolySpot Connects Data from Multiple Apps Across the Enterprise
January 17, 2013
Now that more companies understand that big data is the ticket to big opportunity and big insights, the job descriptions of data scientists are becoming more than fleshed out on paper. People are being hired in this position and companies are hoping that articles like “Data Scientists Will Unlock Big Data’s Promise” from The Wall Street Journal’s CIO Report are correct.
After the requisite reflective move to take a step back and look at what data means today, as opposed to their article on big data last week, the article moves in to a discussion on a holistic approach that data scientists take because of the inherent interdisciplinary nature of their work.
The article then takes a holistic look at our usage of data analysis:
Over the past few centuries, we have significantly increased our understanding of the natural world around us by learning how to collect large amounts of data and by developing disciplined ways to study, analyze, model and make sense of all that data. We have similarly applied our scientific methods in the social sciences to enhance our understanding of societies and human behavior.
Where there is a will, there is a way. This is definitely the case with both the jobs of data scientists and also big data solutions that enables the real-time connectivity of information sourced in various apps across the enterprise. We saw PolySpot conquer the latter and we will see data scientists help further the efforts of these technologies.
Megan Feil, January 17, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
PolySpot Covers Enterprise Big Data Needs with Real Time Insights
January 14, 2013
When trying to identify a definition for big data many people turn to Gartner’s popular 3 V’s. However, we heard word from Information Week on something that goes above and beyond a simple definition in their article “Big Data 101: New Vendor-Neutral Guide addressing a new handbook for enterprises on big data from the Open Data Center Alliance (ODCA).
Despite the fact that many technologies are poised to address big data and are currently functioning with success, their are still organizations that express a greater desire for information on the basics of big data. The ODCA has answered their pleas for more information with their new “Big Data Consumer Guide.”
The article informed us:
The consumer guide summarizes how big data platforms can help a variety of industries. Banks, for instance, can correlate data from multiple, unrelated sources to potentially spot credit card fraud. In addition, the guide provides common definitions and lingo that organizations can use when working with big data providers.
This endeavor will help enterprises get a clear picture on the landscape at the time it is published, but new technological solutions in the big data arena pop up all the time. Innovative information delivery in the enterprise will continue to start and end with PolySpot, however.
Megan Feil, January 14, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Nexans Licenses PolySpot Search Engine
January 10, 2013
We learned that the Nexans Group, a global player in the infrastructure, industry, building, and local area network data markets, uses the PolySpot search engine to cross-reference its internal sources of information and all of its information portals.
According to information provided to Beyond Search:
The goal of the roll out was to streamline collaborative processes while ensuring the integrity of security regulations. The strong point of the solution lies in the simplification of access to information regardless of the source. Using the PolySpot engine, users, customers, partners, and employees of Nexans can now easily access the product catalog and at a level of information consistent with their right of access.
Nexans tested various competing solutions on the market. PolySpot told Beyond Search:
PolySpot stood out because of the performance and strength of its Microsoft SharePoint connector. The PolySpot connector for Microsoft SharePoint is characterized by a very large tolerance of isolated errors (corrupt data, crashes, micro network outages, etc).
PolySpot supports the indexing of an entire SharePoint farm and provides fine-grained filtering. The PolySpot approach limits the the maximum load on the application and network.
As a result, Nexans Group users have access to faceted navigation within the relevant Group directory. Information access is pivotal to Nexans Group’s business which offers an extensive range of cables and cabling systems to raise industrial productivity, improve business performance, enhance security, enrich the quality of life, and assure long-term network reliability.
For more information about PolySpot’s technology and services, visit www.polyspot.com.
Stephen E Arnold, January 10, 2013
Focus Within Big Data is Steered Towards Enterprise Information Delivery
January 10, 2013
The New Year brings about resolutions of all shapes and sizes. We spotted an interesting one from Tech Crunch author Leena Rao in regards to big data. Unlike many CEOs and IT professionals seeking to utilize this cutting edge technology, she wants to abolish the term ‘big data.’ In the article, “Why We Need To Kill ‘Big Data‘” she chalks up the concept to a mere buzz word.
The author states the following in the referenced article:
Why have I grown to hate the words “big data”? Because I think the term itself is outdated, and consists of an overly general set of words that don’t reflect what is actually happening now with data. It’s no longer about big data, it’s about what you can do with the data. It’s about the apps that layer on top of data stored, and insights these apps can provide.
Rao signals an important change in enterprise readiness for big data; companies are prepared to take action by deploying and utilizing big data solutions. We will see many companies use technologies focused on enterprise information delivery across the entire organization. The key is having the right knowledge in the right place and at the right time.
Megan Feil, January 10, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
Google Pushes into Enterprise Market
January 9, 2013
Google is about to tackle the enterprise market head on, particularly targeting the areas of the cloud, social media, and mobile software. Could this have anything to do with softening ad revenues? Not sure; that is one topic not mentioned in the informative interview from Computer Business Review, “Q&A with Thomas Davies, Head of Google Enterprise UK and Ireland.”
Journalist Steve Evans spoke to Davies about Google’s enterprise push. Davies names three trends, cloud, social, and mobile, that his division focuses on, and specifies mobile as the most influential. Because smartphones and tablets are becoming important tools to many businesses, Google plans to seize the day with Android for the enterprise.
There are those who question whether the security and management capabilities of Android are up to the task. Davies assures us, though, that his team has been communicating with business leaders and is tailoring the OS to meet their specifications. Furthermore, he says, Android is now pushing out updates the way Chrome does, so companies won’t have to contend with different stages. The chaotic way Android versions have historically been distributed, though, suggests that every worker would have to have the same iteration of the same device for that to work; not ideal.
As for social media, Google anticipates that it will soon melt into collaboration tools, rather than remain a standalone destination application. Oh, but make no mistake– Google+ will remain a destination app. They have a lot invested in that project, Davies says. Evans presses that point, asking why Google + is different from failed attempts like Wave and Buzz. Because Google learns from mistakes, of course. Well, that’s good.
Regarding the cloud, Evans notes that it’s a big step for companies. Davies replies that it’s all about the money. Businesses have now been hard pressed for years- years!- to reduce costs, and there’s only so much you can cut. According to him, porting to the cloud can save on operational costs by 30 to 50 percent—a reduction many companies find well worth the bother.
So, it seems that now is the time for Google to aggressively push into the enterprise market. Financial pressure and technical advancement have come together to create the perfect opportunity, and they are not about to let the moment escape them. Davies concludes:
“People want to change. I think the time for personal productivity – going to the office, filling in your spreadsheets and sending them to someone else – is going. There was a standard, monolithic build; SAP in the background, Office and IE on the desktop and BlackBerry. That’s changing and I think the speed of that change has caught IT departments by surprise.
“That plays nicely into our hands. I think where we will win, and where we are winning, is when it comes to the three main benefits: business benefits, technical benefits and cultural transformation. That’s how you change an entire company.”
Cocky, isn’t he? But the man has a point. I predict the company will succeed spectacularly in this venture.
Cynthia Murrell, January 09, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
-
- Subscribe to Beyond Search
Feature archive
News archive
-
