Relevance: Once Ignored, Now a Core Issue
September 23, 2011
The Google recipe for its Web site placement order for searches is closely guarded despite the company’s open-source policy. The article, Google Discusses Their Algorithm Change Process, on Search Engine Journal, explains the lengthy and arduous process Googlers must go through in the quest for search engine impact.
Google explains that they must guard the algorithms to keep the manipulation of its numerical recipes which contain mathematical formulas and secret sauce, within Google defined boundaries. In fact, an entire industry has grown up around trying to crack Google’s search algorithms in an effort to bolster one’s relevance in Google’s recipe. Google isn’t just sitting around; rather, the company is constantly updating and tweaking their algorithms. We learned:
Each idea is based on improving the user experience, but not every idea actually shows a positive user impact; while over 500 changes were made last year, over 20,000 experiments were conducted in that same time period. The key takeaway is that, while it’s a good idea to pay attention to experiments, only a small cut will ever become a part of the standard – and, with 500 changes a year, even those alterations are subject to reversal.
With many changes occurring behind the curtain, how are Web masters who want users to find their content to achieve findability? Although 500 changes may be made in a year, not all of them (hardly any at all) have an impact of the majority of site rankings. The sites which may be affected are, we have heard, on their own. Google does not offer support, but it does provide a useful list of guidelines.
The few changes that do impact some sites can pack a wallop. The search engine optimization industry typically responds with suggestions, ideas, and work arounds. Google then changes its algorithm and the process begins again.
What’s the fix? Our conclusion is that one may have to “go Google”. Embrace all things Google, and buy Adwords. Trying to outwit Google may be a time consuming and unrewarding activity.
Catherine Lamsfuss, September 23, 2011
Sponsored by Pandia.com
Search Is a Major Project: Does Doom Await?
September 22, 2011
Datamation ran a long article called “Why Major IT Projects are More Likely to Fail Than Any Others” informs us of a study published by Oxford University which found that major IT projects are twenty times more likely to fail than other IT projects. On average, these larger projects ran 27 percent over-budget and took 55 percent longer to complete than originally planned. One in six eventually spiral out of control.
According to Silicon.com’s article, “Five ways to stop your IT projects spiralling out of control and overbudget” describes why this is the case and details tips for controlling projects. The article states:
The risk of tech projects going rogue is down to IT being harder to manage than other business disciplines, according to Bent Flyvbjerg, BT professor and founding chair of major program management at Oxford University. ‘Our theory is this is because IT projects are less tangible, they are also more complex,’ he told silicon.com.
Is it possible that a main culprit behind this phenomenal statistic is complexity? Are information technology companies attempting to develop elaborate plans for the newest and the best and aiming too high? I think it’s very likely. Perhaps if developers could simplify their ideas and end the game of out-performing each other, we could easily have more IT projects completed.
Search deployments often become expensive headaches, but it may not just be the peculiarities of search or search vendors, integrators, or staff. The problem may reside in the fact that the complexity of the undertaking is overlooked, ignored, or not understood. Too bad. Some search vendors take the blame for a problem created by a business process, not technology.
When I spoke with Stephen E Arnold, publisher of Beyond Search, he told me:
Software and systems are complex. The environments into which engineers insert these things is complex. Complexity adds to complexity. Costs sky rocket and systems don’t work particularly well. Vendors often take the blame for problems caused by casual, uninformed, or inappropriate business processes used to scope a project and spell out requirements. Search falls victim to these issues just like enterprise resource planning, accounting, and document management systems.
Quite a challenge for those responsible for a large scale project awaits it seems.
Andrea Hayden, September 22, 2011
Sponsored by Pandia.com
Protected: SharePoint and Product Lifecycle Management
September 21, 2011
Protected: An X Ray of the SharePoint User Subsystem
September 16, 2011
Kroll in the UK and Its Content Technology
September 14, 2011
The recent disturbances in London have lead UK Prime Minister David Cameron to reach across the pond to consult Kroll Chairman and former American police chief, William Bratton on preventing gang related violence and building safer communities. There’s nothing like an outside US expert to come to the aid of our British cousins.
Altergrity, a specialized law enforcement training company and owner of Kroll, quoted Mr. Bratton in an Aug 12, Media Release:
I would certainly be in a position to discuss the contemporary American experience and my work in these areas – in particular the successes that created real reductions in gang-related crime in Boston, New York and most recently in Los Angeles, where we also saw significant improvements in the relations between the police and the city’s diverse communities. There are many lessons from these experiences that I believe are relevant to the current situation in England.
Based on this release, Mr. Bratton appears confident in his abilities to solve the world’s security concerns. We hope that UK police and civilians are equally secure in the role that his company takes in dispelling the violence affecting their country. If you want some basic information about the types of search and content processing tools that Mr. Bratton brings to his engagements, navigate to the interview with former Kroll wizard David Chaplin here. This is quite impressive technology.
Jasmine Ashton, September 14, 2011
Sponsored by Pandia.com
Hlava on Machine Assisted Indexing
September 8, 2011
On September 7, 2011, I interviewed Margie Hlava, president and co-founder of Access Innovations. Access Innovations has been delivering professional taxonomy, indexing, and consulting services to organizations worldwide for more than 30 years. In our first interview, Ms. Hlava discussed the needs for standards and the costs associated with flawed controlled term lists and some loosely-formed indexing methods.
In this podcast, I spoke with her about her MAI or machine assisted indexing technology. The idea is that automated systems can tag in a consistent manner high volume flows of data. The “big data” challenge often creates significant performance problems for some content processing systems. MAI balances high speed processing with the ability to accommodate the inevitable “language drift” that is a natural part of human content generation.
In this interview, Ms. Hlava discusses:
- The value of a neutral format so that content and tags can be easily repurposed
- The importance of metadata enrichment which allows an indexing process to capture the nuances of meaning as well as the tagging required to allow a user to “zoom” to a septic location in a document, pinpoint the entities in a document, and automated summarization of documents
- The role of an inverted index versus the tagging of records with a controlled vocabulary.
One of the key points is that flawed indexing contributes to user dissatisfaction with some search and retrieval systems. She said, “Search is like standing in line for a cold drink on a hot day. No matter how good the drink, there will be some dissatisfaction with the wait, the length of the line, and the process itself.”
You can listen to the second podcast, recorded on August 31, 2011, by pointing your browser to http://arnoldit.com/podcasts/. You can get additional information about Access Innovations at For more information about Access Innovations at this link. The company publishes Taxodiary, a highly regarded Web log about indexing and taxonomy related topics.
Stephen E Arnold, September 8, 2011
Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search
Possible Changes Ahead for eDiscovery Rules
September 8, 2011
E-discovery 2.0 asks, “New eDiscovery Rules on the Horizon?” Potential amendments to the Federal Rules of Civil Procedure are to be discussed at a mini-conference scheduled for September 9, 2011 by the Advisory Committee on Civil Rules. Writer Matthew Nelson explains the significance of this meeting:
The mini-conference is important because it is part of a seven step process that could ultimately lead to new rule amendments affecting all litigators and the organizations they represent. Any new rule proposals developed by the subcommittee at the September mini-conference will be considered by the Advisory Committee this November in Washington D.C. The proposals, in one form or another, could ultimately become law. Both Supreme Court and Congressional approval are ultimately required.
One area that cries to be addressed is the controversial question, at what point does the duty to preserve evidence kick in? If the answer is when a complaint is served, that may leave too much leeway for evidence destruction at the first sign of a potential complaint.
Many feel that the current rules are too murky, making companies anxious about what they must do to avoid future sanctions. Further complicating the picture are questions about the impact of cloud computing on civil litigation.
We’re just at the beginning of the long process of amending these rules. If your business is concerned with eDiscovery, though, you’ll want to keep up on the progress.
Cynthia Murrell, September 8, 2011
Sponsored by Pandia.com
IBM Acquires i2 Ltd.
September 1, 2011
IBM purchased i2 Group. Founded in 1990 by Mike Hunter, i2 is a widely used content processing and case management system for law enforcement and intelligence professionals. The company received the EuroIntel Golden Candle Award, for its contribution to the global intelligence community. On several occasions, the ArnoldIT team worked on some i2 products several years. The company has moved outside the somewhat narrow market for sophisticated intelligence analysis systems.
“IBM Acquiring I2 for Criminal Mastermind Software” reported:
IBM plans to fuse i2’s products with its own data collection, analysis and warehousing software. It will then offer packages based on this combinations to organizations looking to spot suspicious behavior within vast collections of data.
Not surprisingly, there has been considerable confusion about the company. Part of the reason is that the name “i2” was used by a back office and supply chain company. The firm benefited from its acquisition from the low profile Silver Lake Sununu. Silver Lake purchased i2 from Choicepoint in 2008 for about $185 million. “IBM Bolsters Big Data Security Credentials with i2 Buy” opines that the deal was worth more than $500 million, a fraction of what UK vendor Autonomy commanded from Hewlett Packard in August 2011.
i2’s technology is not well understood by those without direct experience using the firm’s pace setting products. One example in the Analyst’s Notebook, a system which allows multiple case details to be processed, analyzed, and displayed in a manner immediately familiar to law enforcement and intelligence professionals. i2 acquired Coplink, developed at an academic institution in Arizona.
The core technology continues to be enhanced. i2 now provides its system to organizations with an interest in analyzing data across time, via relationships, and with specialized numerical recipes.
My position is that I am not going to dive into the specific features and functions of the i2 system. If you want to know more about i2’s technology, you can visit the firm’s Web site at http://www.i2group.com/us. The Wikipedia page and many of the news and trade write ups about i2 are either incorrect or off by 20 degrees or more.
What will IBM “do” with the i2 technology? My hunch is that IBM will maintain the present market trajectory of i2 and expose the firm’s technology to IBM clients and prospects with specific security needs. Please, appreciate that the nature of the i2 technology is essentially the opposite of software available for more general purpose applications. My view is that IBM will probably continue to support the integration of i2 Clairty component with the Microsoft SharePoint platform. Like the descriptions of Autonomy’s technology, some of the write ups about i2 may require further verfication.
We have reported on the legal dust up about the i2 ANB file format and some friction between Palantir and i2 in Inteltrax. Most of the legal hassles appear to be worked out, but contention is certainly possible going forward.
I have been a fan of i2’s technology for many years. However, some firms have moved into different analytical approaches. In most cases, these new developments enhance the functionality of an i2 system. Today we are featuring an editorial by Tim Estes, founder of Digital Reasoning, a company that has moved “beyond i2.” You can read his views about the Autonomy deal in “Summer of Big Deals”. More information about Digital Reasoning is available at www.digitalreasoning.com. Digital Reasoning is a client of ArnoldIT, the publisher of this information service.
Stephen E Arnold, September 1, 2011
Sponsored by Pandia.com
EasyAsk Sweetens Sugar CRM
August 31, 2011
The world of customer relationship management (CRM) just got a lot sweeter a few months ago with the announcement that Sugar CRM is partnering with EasyAsk. SugarCRM is a leader in the world of customer relationship management. SugarCRM said:
SugarCRM helps your business communicate with prospects, share sales information, close deals and keep customers happy. SugarCRM is an affordable web-based CRM solution for small- and medium-sized businesses. Offered in the Cloud or on-site, it is easy to customize and adapt to the way you do business.
“EasyAsk and SugarCRM to Provide Natural Language Search and Analysis,” covers the news of this exciting joint venture. We learned:
EasyAsk and SugarCRM announced . . . at SugarCon 2011 that they will team up to offer EasyAsk for SugarCRM, a new version of EasyAsk with natural search and analysis software integrated with SugarCRM. The integrated product will deliver SugarCRM information through EasyAsk’s language interface and tools. With EasyAsk for SugarCRM, users can ask questions in English and get immediate answers FROM THEIR SugarCRM system.
The natural language processing (NLP) offered by EasyAsk allows users to query and communicate in English, smoothing the language barrier between man and machine. EasyAsk’s NLP technology and engineering really move the SugarCRM cloud offerings to the next level.
We are glad to see such a natural partnership taking place between two innovators. Other businesses, especially those in eCommerce and mobile apps, would do well to incorporate the EasyAsk language interface and tools into their offerings. Doing so would most certainly increase user satisfaction and reduce their own engineering and design stress.
Other independent software vendors have embedded the EasyAsk natural language interface into their offerings with considerable success. Among the companies using EasyAsk’s NLP technology are Siemens, Personnel Data Systems, Ceridian, and Gensource.
Although IBM has been intent on wowing the consumer with the Jeopardy game show demonstration, EasyAsk has been building a market for real-world natural language solutions. In addition, EasyAsk has also delivered a version of its NLP tools on NetSuite, one of the leaders in software as a service enterprise resource planning solution providers.
The EasyAsk-SugarCRM partnership came together smoothly, with both firms able to sweeten their product offerings while solving problems for licensees. We will continue to cover EasyAsk. We think the search firm will continue to prove itself a market and technology leader.
Stephen E Arnold, August 31, 2011
Sponsored by Pandia.com
Social Media: Making the Personal Impersonal
August 25, 2011
Search engines are now using social media data to rank query results. As crazy as it sounds, your Tweets could now alter the way Google gives you information on such benign things as “George Washington” or “grilled cheese sandwiches.” eSchool News takes a look at how “New Web-Search Formulas Have Huge Implications for Students and Society.”
Search results now differ from person to person based on algorithms have been altered to include social media data. Knowing that most people don’t go past the second page of results, they have tailored their ranking system to consider links you have clicked on and create a filter system based on those previous links. This isn’t something ground breaking since Amazon and Netflix have been using it for years to recommend books and movies, but is new to the major search engines.
At the 2011 Technology, Entertainment, and Design talk, Eli Pariser, the author of The Filter Bubble, shared his reservations with the “invisible algorithmic editing of the web.” He believes it only shows us what it thinks we want and not what we need to see.
[I]t was believed that the web would widen our connections with the world and expose us to new perspectives, Pariser said: Instead of being limited to the newspapers, books, and other writings available in our local communities, we would have access to information from all over the globe. But thanks to these new search-engine formulas, he said, the internet instead is coming to represent ‘a passing of the torch from human gatekeepers [of information] to algorithmic ones.’ Yet, algorithms don’t have the kind of embedded ethics that human editors have, he noted. If algorithms are going to curate the world for us, then ‘we need to make sure they’re not just keyed to [personal] relevance—they also should show us things that are important, or challenging, or uncomfortable.’
It seems that search engines may be focusing on personal factors, but are not personalizing the process. The user has no control over results. That customization is left to a rigid algorithm. If a restaurant says that they make burgers “made-to-order,” then I expect to be able to pick mustard and onions on one visit, and pick cheese and ketchup on the next visit. The server should not just look at my past orders and make an educated guess. There is nothing “personal” about that.
Could this lead some well meaning people down an unintended and risky path to censorship-by-computer. Users must gain more control over these search formulas. There are certainly times when social media parameters are acceptable, but sometimes you want and need to see the other side. It depends if you are settling an argument between your friends over song lyrics or writing a thesis on communism. Until users are offered more liberal control, I think this “personal” ranking system will actually suppress and limit a lot of important information that users are seeking. The social impact on a search comes at a pretty high price.
Jennifer Wensink, August 25, 2011
Sponsored by Pandia.com
-
- Subscribe to Beyond Search
Feature archive
News archive
-
