Specifications: Do We Know What We Want?
August 13, 2013
It is tough to make sales. It is tough to fill time at work when the company wakes up each morning asking, “What buzzword will we try to sell today?” Remember to distant days of mythical cities like Smallville. Stores sold shoes, fruit, and dry goods. Not today, companies pitch “services” like cloud data, analytics, enterprise search, and content marketing.
Little surprise: Computerworld ran a story with the Google-ized title “A Software Project with 6,000 Pages of Specs Ends Badly.” The application was a tax collection system. Not search obviously, but the key passage in the story could have been applied to a number of search, content processing, and analytics systems with which I am familiar; to wit:
In 2007, Orange County hired Tata America International Corp., a subsidiary of the largest offshore outsourcing firm in the world — Mumbai, India-based TCS — to develop custom software to handle most of the county’s tax functions. The county collects some $4.5 billion in taxes a year. Before hiring Tata, the county had used another contractor to develop the business documentation for the projects.That documentation ran more than 6,000 pages and outlined every aspect of the project, from legal requirements to operating systems.
Observations? As Groucho Marx said, “You bet your life.”
First, notice the prominent role contractors played into this alleged misfire. Contractors wrote the specifications. Contractors built the system. Who was responsible for the alleged floparoo? Contractors. In my view, perhaps the cause of the problem rests with the governmental organization and its employees. Without knowledge about the specific problem to be solved, the specifications embrace everything and anything those contributing to the specification heard at conferences, read in blog posts, and picked up from marketers’ sales presentations. Converting gibberish into a specification is similar to running a document from one language through a translation system a couple of times. Mix and match the languages and interest outputs result. Are the outputs usable? Nope. Amusing? Yep.
Second, notice that the matter is in the hands of attorneys. The progression strikes me as:
- No clear idea of the problem to be solved
- No inputs anchored in facts
- Contractors who converted inputs into a draft specification
- Rewrites fattened up the specification
- The specification was converted into a request for proposal or similar document
- Consultants responded to the proposal
- Individuals with “qualifications” to decide which vendor had the oars in the water reviewed the specifications
- The bidders revised the proposals
- Individuals with “qualifications” narrowed the field to a couple of bidders
- Presentations ensued
- Individuals with”qualifications” picked the winning bidder
- The bidder implemented the specification
- The “system” did not match the needs which were encapsulated in 6,000 pages of word brambles
- Enter the lawyers
- Read story in Computerworld
My question, “When will those buying software do the pre-award work necessary to get the best from a contractor?” Better yet, “Why not hire professionals who can do their job in a manner that produces software that solves a problem?”
The story reminded me of the messages I hear when I am boarding an airplane. Meaningless actions from meaningless thinking.
Stephen E Arnold, August 13, 2013
Sponsored by Xenky
Study Reveals Social Data to be Isolated in Silos
August 13, 2013
When enterprise organizations understand the value of unstructured data, and especially the value of it when it is integrated with structured data, what kind of solutions do they utilize? According to a recent study by Altimeter Group reported in “Enterprise Social Data Isolated in Departmental Silos,” 42 percent of the 35 large organizations surveyed were using business intelligence tools. Other areas where data sets converged were market research at 35 percent, CRM at 27 percent, email marketing at 27 percent and sensor data at four percent.
The main argument presented by the article is that as long as people are working in departmental silos, information and data will be first and foremost stored in a way that parallels how people are organized.
We learned more about why some organizations face challenges when integrating data:
The report also revealed it’s not always easy to integrate this data, attributing the issue to the fact that so many organizational departments touch the data, ‘all with varying perspectives on the information,’ the article states, adding: ‘The report also notes the numerous nuances within social data make it problematic to apply general metrics across the board and, in many organizations, social data doesn’t carry the same credibility as its enterprise counterpart.’
We know that one company, Expert System, would have quite the rebuttal to this argument that unstructured data may not be worthy across the board for all departments. Their solution Cogito Intelligence API yields insights and actionable information after parsing both structured and structured data while using sentiment analysis and natural language processing technologies.
Megan Feil, August 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Useful Suggestion For The Magic Bullet Hadoop
August 13, 2013
Downloading Hadoop and expecting it to solve all your problems is dumb way to use the software. Silicon Angle has some suggestions on how to use Hadoop in, “To Succeed With Hadoop: Find Specific Problem Areas And Solve Them.” The advice comes from Datameer CEO/Founder Stefan Groschupf at the recent Hadoop Summit 2013.
Groschupf acknowledged that Hadoop is another tool in the big data toolbox and the real power of a company does not come from just its tools, build its customer base, quality products with an edge that no one else has, and to stay in the black. Most importantly is to find a problem no one else has resolved and do it yourself.
That seems to be the only advice the article offers. The rest is an advertisement for Datameer 3.0, which is the newest tool for big data analytics:
“Datameer 3.0 adds new Smart Analytic functions. With a single click, it automatically identifies patterns, relationships, and recommendations based on data stored in Hadoop. For the first time, four advanced machine learning techniques become self-service and accessible for data-driven business users: Clustering, Decision Trees, Column Dependencies and Recommendations. Until now, these advanced analytics required highly specialized data scientists to build custom functionality, which was a costly and time-consuming process.”
So get a gimmick kids! Once you have that you will succeed. It worked for Bill Gates, Walt Disney, Steve Jobs, and Lady Gaga.
Whitney Grace, August 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
eDiscovery Amends The Federal Rules Of Civil Procedure
August 13, 2013
When eDiscovery came onto the scene it changed the way information was processed, which in turn changed laws and litigation. Clearwell Systems’s eDiscovery Blog takes “A Comprehensive Look At The Newly Proposed eDiscovery Amendments To The Federal Rules Of Civil Procedure.” The Standing Committee on Rules of Practice and Procedure approved for public comment amendments that are supposed to streamline discovery, encourage cooperative advocacy, eliminate gamesmanship, and most importantly provide solutions for the endless eDiscovery–related problems. The blog plans to cover all the proposed amendments with insights on how they will affect judicial case management. It will pay special attention to Federal Rule 37(e) that will crate a uniform national standard for discovery sanctions stemming from failures to preserve evidence.
What exactly will the amendments change?
“Drafted by the Civil Rules Advisory Committee, the proposed amendments are generally designed to facilitate the tripartite aims of Federal Rule 1 in the discovery process. To carry out Rule 1’s lofty yet important mandate of securing “the just, speedy, and inexpensive determination” of litigation, the Committee has proposed several modifications to advance the notions of cooperation and proportionality. Other changes focus on improving “early and effective judicial case management.” Judicial Conference of the United States, Report of the Advisory Committee on Civil Rules 4 (May 8, 2013) (Report).”
So far Rule 1 is trying to make lawyers and associated parties play together nicely. No doubt parameters will be set, but would it not be easier to have everyone watch a few episodes of Sesame Street?
Whitney Grace, August 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
News Channels Are Not Dying They Are Expanding
August 13, 2013
If you do not read enough about customer relationship management (also known as search) at Arnold IT, you might find Paper.li interesting. Paper.li is an aggregation service that focuses on the above topic, plus many more that makes the subject matter wander. Take a look and not only will you see stories about IT, business, and entertainment, but there are some items that look like they have been pulled from Reddit.
How does Paper.li work? Its approach is similar to an old-fashioned newspaper with customable options on your topic of choice:
“The key to a great newspaper is a great newsroom. The Paper.li platform gives you access to an ever-expanding universe of articles, blog posts, and rich media content. Paper.li automatically processes more than 250 million social media posts per day, extracting & analyzing over 25 million articles. Only Paper.li lets you tap into this powerful media flow to find exactly what you need, and publish it easily on your own online newspaper.”
Aggregation services are gaining more prominence, especially since the Google Reader went kaput. Also another factor is that people do not enjoy having to sift through search results. There is definitely a market, so why not find a sponsor. Perhaps Attentisity? Let us know if you know the funding agent.
Whitney Grace, August 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
LucidWorks to Appear at DataWeek
August 13, 2013
LucidWorks has already had an exciting 2013 of conference appearances, special event hosting and sponsorship, and trainings. They continue their momentum by joining DataWeek 2013 in San Francisco in late September. Read more in the LucidWorks announcement, “DataWeek, SF.”
The announcement says:
“DataWeek is an industry event with the goal of placing the most innovative data technologies in front of Bay-area and nation-wide technology executives, nation-wide developers and data scientists, as well as potential customers for those technologies. We aim to routinely maintain our forward-thinking edge on conference topics such as the API economy, the Hadoop ecosystem, data science and machine learning tools, and data hacker tools.”
LucidWorks definitely fits into the theme of innovative data technologies. And in terms of partnership with Hadoop, their LucidWorks Big Data solution pairs the power of Apache Solr with the strength of Hadoop to bring leading edge Big Data technology to the enterprise.
Emily Rae Aldridge, August 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Wolters Kluwer Selects Innovative Cogito Semantic Platform
August 12, 2013
The market for semantic search seems to only grow larger as time moves forward and there is a greater understanding of the functionalities and capabilities for semantic search in many fields. We spotted a press release, “Expert System Develops Semantic Search Engine for Wolters Kluwer Italy,” on a company that is making waves with their natural language interface.
We learned from their press release that Expert System’s Cogito was chosen by Wolters Kluwer Italy as the semantic platform to improve access to information on its online portal for legal and public sector professionals.
Luca Scagliarini, VP of Strategy & Business Development for Expert System was quoted as stating the following:
“Search applications that are able to understand the nuances of everyday language and ensure the most relevant results, are advantages that only a semantic approach can guarantee because it has a contextual understanding of the meaning of content. By choosing our technology, Wolters Kluwer Italy offers customers a better user experience and improved efficiency through the convenience of always having what they need.”
Essentially, Cogito improves the effectiveness of search by allowing end users to search using natural language terms; their semantic technology understands the contextualized meaning of words. Expert System not only offers cutting edge solutions like Cogito, but they also know make it easy for business executives to understand how Cogito works by featuring a demo.
Megan Feil, August 12, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Google Search Explained and Explainedg
August 12, 2013
One of my two or three readers sent me a link to a series of comments on the social networking information aggregation site Reddit. Navigate to http://goo.gl/EJLarJ. The entry is “How does Google search the whole Internet for something in a matter of seconds or even less?”
I found the explanations fascinating.
One person said, ”
Google spends all day every day searching the Web with bots. Web sites and their data are catalogued in a database and when you search, it is the database that is being looked through. It’s also not the whole Internet. Lots of sites have code that prevents them from showing up in the search engine.
Other comments of interest are:
“Amazon has over 2500 sub site maps. ”
“I like that the best way to find out things Google doesn’t know is by using Google.”
“I remember reading somewhere that Google estimated that only 0.02% of the internet is cataloged in the Google database.”
Google’s search index is over 100 million gigabytes big
“I heard an analogy once, that searching the internet with google is like dragging a net through a pond. You’ll get stuff from the surface but there’s a lot of material deeper down you don’t get.”
“Google has only indexed 0.004% of the entire Internet.”
“Imagine there are spiders(web crawlers) going around the web and gathering all the insects(web pages) in stuck there. Then they pile the different insects into cocoons and label them (hash code). Now you can find your favorite insect from the labeled cocoons by keyword and they are brought to you in an order of popularity.”
And my favorite:
“Think of Google like the Index Cards they [librarians] had at the library before computers. The index card system is just an organized collection of where the books (Web sites) exist in the library. All of the actual information is held in the books. A librarian (Web crawler) has to keep the index card system up to date but they [sic] don’t need to do it in realtime every time a book is requested. They keep a database of where everything is instead.”
Yep, librarians with advertising. I am delighted with the explanations of the Google. Delighted, I say.
Stephen E Arnold, August 12, 2013
Sponsored by Xenky
Our Nanotech Future
August 12, 2013
A must-read article at the Public Intelligence Blog examines the state of future tech in, “Stephen E. Arnold: Google Glass: Are Weaponized Nanodevices and Ultra-Miniaturization Coming?” Yes, the author is our own beloved Stephen E. Arnold, but I assure you—the article is truly interesting. It is a good example of Arnold’s talent for pulling together his keen observations in order to anticipate trends.
The article begins with an examination of Google‘s ad-revenue stream, which currently seems to be losing momentum. Arnold suggests this could be one reason Google has been pushing its Glass project. He wraps in some observations from a nanotechnology textbook, and projects a future where our devices literally become part of us. No, this is not a new idea, but Arnold backs up his conclusions with real-world logic, not science fiction. (Though he refers to that, too.) I cannot give his reasoning justice here—you’ll just have to check out the write-up.
Arnold concludes:
“The goal is not glasses. The goal may be to put a computer in an eyeball. The military and intelligence applications are easy to identify: covert surveillance, weaponized medical devices which can render a person of interest harmless, and elimination of bulky battery packs for operational personnel, among others.
“I must admit that these implementations are likely to be years, maybe decades in the future. I think I will skip the smartphone watch and smartphone contact lens. A nano-device sounds just right to me. The interesting thought is that for many young people, a nano-device will be the obvious choice. Only old people carry a smartphone, wear a watch, or insert a contact lens.
“Nano-devices are likely to be hotter than the latest Nike sneaker worn by a special ops team.”
Given the trajectory of our technology and our collective imagination, these predictions seem right on point. I look forward to sporting my nanotech while visiting great-grandkids on the moon.
Cynthia Murrell, August 12, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Another Content Management Company Another Day
August 12, 2013
Content management companies are springing up and gaining attention due to the Big Data boom. One of the companies that our content wranglers pulled out of an Internet Search is Applied Relevance. They specialize in several aspects of the content management spectrum, but the company’s Web site prominently promotes its taxonomy services. Applied Relevance offers the AR-Classifer tagging engine that can run on a variety of platforms. Its AR-Semantics is the flagship organization and categorization software, while the AR-Taxonomy is the tool needed to edit and manage taxonomies and if you want to search your taxonomies the AR-Navigator is available.
All this talk about Applied Relevance’s taxonomy software is informative, but what is interesting is the company’s description on the main page:
“Applied Relevance produces software and services to help enterprise users find the information they need. Our solutions augment traditional search engines by providing context for the search results. The AR toolset and our partners provide cost effective technology for the full spectrum of enterprise content management and search applications. With our tools, a search term and a few clicks, users can zero-in past ambiguities and come up with the right answer in the right context. Applied Relevance is located on the west coast of the east coast of North America.”
Descriptive, but not a word on taxonomy or what exactly the company specifically does. The tagline at the end about Applied Relevance’s location is even more ambiguous.
Whitney Grace, August 12, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
