Behind the Scenes at DuckDuckGo
February 14, 2013
High Scalability gives us an in-depth look at the burgeoning DuckDuckGo derived from an interview with the site’s founder, Gabriel Weinberg, in “DuckDuckGo Architecture—1 Million Deep Searches a Day and Growing.” Writer Todd Hoff notes that the Duck is proudly famous for (or famously proud of) refusing to collect data on their users. Though it is understandable that Weinberg emphasizes that popular stance, Hoff is more interested in the mechanics behind the service. He writes:
“What I found most compelling is DDG’s strong vision of a crowdsourced network of plugins giving broader search coverage by tying an army of vertical data suppliers into their search framework. For example, there’s a specialized Lego plugin for searching against a complete Lego database. Use the name of a spice in your search query, for example, and DDG will recognize it and may trigger a deeper search against a highly tuned recipe database. Many different plugins can be triggered on each search and it’s all handled in real-time.
“Can’t searching the Open Web provide all this data? No really. This is structured data with semantics. Not an HTML page. You need a search engine that’s capable of categorizing, mapping, merging, filtering, prioritizing, searching, formatting, and disambiguating richer data sets and you can’t do that with a keyword search.”
He’s right. I do turn to DuckDuckGo for such deep searches, but I often go back to Google if I need a broader one. It is good to have a variety of tools. All else being equal, I do prefer the Duck’s privacy policy.
That bragging point, however, comes at a cost. Like other Web search engines, DuckDuckGo is ad-supported, but their key policy makes it impossible to take advantage of the most lucrative source of revenue—the targeted ad. Our view is that 2013 is about revenue, not about bits and bytes, or about popularity. We hope our fellow water-fowl makes it through okay.
Do check out Hoff’s article if you are interested in the mechanics behind DuckDuckGo. It is chock-full of detailed information.
Cynthia Murrell, February 14, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Google Plus Nips at Facebooks Heels
February 14, 2013
It looks like victory is approaching for Google. The International Business Times reports, “Google Plus Becomes No. 2 Social Network After Facebook, Knocking Off Twitter.” I guess that’s what mandatory registration will do for a social network.
The article cites a recent report from Global Web Index, which found that the number of Google+‘s active users grew by 27 percent in the last quarter of 2012. On top of that, Google’s property YouTube came in at number three. This was the first time that site was included in this social-network tracking study. Facebook maintains its healthy lead, though, with nearly 693 million active users to Google+’s 343 million and YouTube’s 300 million.
Writer Dave Smith points out that Google+ and YouTube both benefit from their close user-base integration. He notes some other smart moves on Google’s part:
“Not too long ago, Google Plus was against the ropes, struggling to maintain traffic and momentum after its public debut. The site, at the time, looked very plain and lacked any real key differentiator from Facebook, besides its video chat offering, Hangouts.
“In about a year and a half, Google has done many things to beef up its social offering, giving it a new design, new technology and a really sleek mobile application. But the best thing Google did in the last 16 months — something its competitors should learn from — is learning how to seamlessly integrate its services.”
Smith believes that capturing the second and third spots stands as a declaration of Google’s Internet dominance. He also praises Google+ as a (now) well-designed app. See the article for more of his observations on the resurgent social network.
Cynthia Murrell, February 14, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Yahoo and Walking a Tightrope: Ms Mayer as Elvira Madigan
February 13, 2013
Elvira Madigan was a 19th century tightrope walker, immortalized in a film I saw when in graduate school. Is Yahoo’s Marissa Mayer’s the 21st equivalent. Instead of doing a freestyle slacklining, the chief Yahooligan may be attempting to work with both Facebook and Google.
Tricky stuff.
To get the basic idea of the new Yahoo tactics, navigate to “Yahoo CEO Says Microsoft Search Deal Underperforms.” With Yandex gaining momentum and the Google getting its mojo back, Yahoo should be concerned about the Bing search deal. The Thomson Reuters’ report included this useful sequence:
“We need to see monetization working better because we know that it can and we’ve seen other competitors in the space illustrate how well it can work,” Mayer said of the search deal with Microsoft. Yahoo and Microsoft entered into a 10-year search partnership in 2010, hoping their combined efforts could mount a more competitive challenge to Google Inc, the world’s No.1 search engine. But the partnership has not lived up to expectations. Google remains the dominant search engine, with a 66.7 percent share of the U.S. market in December, almost unchanged from its 66.6 percent share two years earlier, according to online analytics firm comScore.
Underperform, therefore, seems to be money, not precision, recall, or whizzy Bing features.
I also noted the Bloomberg story “Yahoo CEO Mayer to Cement Facebook Ties While Pushing Mobile.” The story said:
Enhancing social features is crucial to Yahoo’s success, Mayer said, as she reinforced her preference to partner with companies like Google, Apple Inc. and Facebook rather than build expensive new products. “One of the things that people really want to do is share their interests with their friends,” she said. “We need to have sharing built as a fundamental component.”
Makes sense. Most of what Ms. Mayer does makes sense, including the decision to bail out of the “controlled chaos” of her former employer.
My view is that balancing can be tricky when some of the folks holding the tightrope may let go.
Stephen E Arnold, February 13, 2013
Enterprise Information Delivery Strengthened with Solid Library of Connectors
February 13, 2013
Now that many organizations have begun to employ big data initiatives we are seeing many more surveys that are examining what stage of the process these companies are at. An article from ReadWrite discusses the results of a survey conducted by Infochimps and SSWUG.org, a community site for IT professionals. “Big Data: Many CIOs Completely Clueless” offers solid insight into what kind of technologies these companies need in order to succeed with their initiatives.
The article states:
The greatest challenge with Big Data is getting at the data trapped in various business applications across an organization, the survey found. Pooling this huge amount of information is necessary in order to run the necessary analytics to find ways to cut costs and run a more efficient business. But before that can happen, all the data has to be converted into a usable format.
Understanding a more complex picture, at a bird’s eye view above the individual enterprise applications, is likely to be much easier when the proper tools are employed. We have had our eyes on one such possibility, PolySpot. Enterprises benefit from the heightened information access and enriched information that comes with a product featuring such a strong library of connectors like this technology offers.
Megan Feil, February 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
DataStax Attempts Security with NoSQL
February 13, 2013
DataStax, a company based on the Cassandra NoSQL database, has announced the release of DataStax Enterprise 3.0. The new platform is not just an upgrade; it is really an overhaul. Kristin Bent covers the release for CRN in the story, “DataStax Merges Enterprise Security, NoSQL In Big Data Platform.”
The article states:
“Big data applications vendor DataStax said this week it will start shipping its next-generation data management platform on Feb. 25, a release the company says melds the flexibility of NoSQL databases with enterprise-level security. The new platform, dubbed DataStax Enterprise (DSE) 3.0, is targeted at organizations looking to adopt NoSQL databases — a type of next-generation, non-relational database optimized for big data — without sacrificing the robust security features native to more traditional SQL databases, explained Robin Schumacher, vice president of products at DataStax.”
Most NoSQL solutions do not have built in security. But enterprises have grown used to advanced security features. DataStax hopes to bridge the gap by bringing enterprise security solutions to the NoSQL base. However, some may not trust the first version of such a blended solution. Many will still trust traditional enterprise search solutions built on trusted names, for instance, LucidWorks and its usage of Apache Lucene and Solr technologies.
Emily Rae Aldridge, February 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Microsoft Gets Closer to Open Source Goodness
February 13, 2013
Is that a pig I see flying by the window? Or, as Simon Phipps at InfoWorld puts it, “Has Microsoft Finally Embraced Open Source?” The company recently added support for the open-source version-control system Git to its developer tool Visual Studio. It looks to us like the Microsoft leopard may be changing its spots in order to follow in IBM‘s footprints.
Phipps notes that he has observed other positive signs, like changes to the developer terms for the Windows Phone that favor open-source licenses. At the same time, though, the company is still showing signs of hostility. It has accused the city of Munich, Germany, of under-reporting the costs of its Linux-based system, going so far as to commission a “secret report” to bolster its charges. The apparent contradiction, the article notes, stems from the fact that Microsoft (like any corporation) is actually a collection of divisions, teams, and individuals, some of which have different perspectives from others. Phipps writes:
“Microsoft is on a long march toward accepting the market inevitability of open source, but the right foot doesn’t always know what the left is doing. The company is still fighting open source on the desktop, while staying mostly silent about its taxation of open source usage (in the form of ‘royalties’ for supposed software patent infringements, in return for promising not to litigate). Other teams see the wisdom of nonconfrontation, while some —such as the developer tools team —seems to want to engage in a positive way.
“When corporations embark on such a journey, it remains smart and reasonable for communities to assume that previous behavior will continue until a clear pattern of experience shows otherwise.”
Good point; the open source community would be wise to keep its guard up until a company-wide shift in its favor has been unequivocally declared. However, there is nothing wrong with indulging in a little hope.
Cynthia Murrell, February 13, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Healthy Competition?
February 13, 2013
I read an interesting article the other day, “Google, the EU and Competition: Speaking Different Languages” by David Vranicar. It was a good beginner’s read from Tech News World about the intricacies of competition within the tech world via the US and EU.
“So this term “competition keeps popping up, but I wonder if maybe the term means something different to Europeans, or at least to European regulators, than it does in the U.S. I feel like in the U.S., “competition” refers to the process by which consumers pick out their favorite products and in Europe it might be more of the process by which companies are allowed and encouraged to enter the market.”
Vranicar talks about the idea of monopoly in terms of legality within the US and EU. This is where Google has violated the trust of Americans and Europeans. The European idea of monopoly centers around the term “harm.” If one entity has a monopoly on a service or product then others who seek to enter are harmed as are consumers who aren’t given a choice.
“Whereas in Europe, the point is made that Google, which has a much bigger share here —Google here is sort of 95 percent of searches, compared to the U.S., where it’s about 65 percent— so their point is Google basically covers the whole landscape.”
Now, not to throw a monkey wrench into things but I’m not sure that the author is correct in his assumption. I think that the “competition” we see — in smart-phone technology for instance — has in large part been taken out of the consumers hands, in the United States AND abroad thanks in large part to the patent wars, like the one Google is currently waging with Apple and Microsoft.
It keeps others from entering the market so is it really just a case of “lost in translation” between the bureaucracy and the techies?
Leslie Radcliff, February 13, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Pack Your Bags Watson We Are Off
February 13, 2013
Forbes’s Jan. 30 article on “IBM’s Watson Heads To University,” was a great pick-me-up for one of the oldest technological universities in he United States.
Watson is the computing system created by IBM that was a repeat winner on Jeopardy. The Watson system utilized cognitive computing in order to prevail over its human counterparts in the game.
And now, after hanging out in the health care field for a while, analyzing data and creating detailed diagnosis for physicians, Watson is headed to the Rensselaer Polytechnic Institute (RPI).
“Other sophisticated computing systems have tackled the difficult problems of understanding voice and text…but with their backing by defense and intelligence agencies one suspects some of their most advanced accomplishments remain out of sight…”
Well, that’s a duh statement. IBM doesn’t have a stake in the defense department so they’re free to prostitute Watson out as they choose. Jeopardy or medical school, it’s all the same. But why take it out of the medical industry if it’s doing so well with diagnosis and predictive answering?
In all fairness, it’s only fitting that RPI get a chance to collaborate with this newest Watson software, it was one of the eight universities that worked with IBM during the production of the program. So it’s more of a “homecoming” and less of a “beauty school drop-out” for Watson.
Leslie Radcliff, February 13, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Connectors in Big Data Solutions Link Data Necessary for Comprehensive Picture
February 12, 2013
A recent article from GigaOM furthers the conversation in the midst of what some describe as a data backlash in “Why Big Data Matters and Data-ism Doesn’t.” New York Times columnist David Brooks is credited with coining the term data-ism to characterize the common phenomenon where people reduce everything in the world to statistics and this GigaOM writer agrees that data-ism is something to stay far away from.
While many data enthusiasts are simply content with lists of data and statistics for the sake of the data, it is important to see beyond the mere data points. Big data and the technological tools available are helping to further the possibilities and opportunities that data offers every field from research to business.
The author of this article states:
If there’s one thing I’ve learned, it’s that the real value of data isn’t just in uncovering statistical realities, but in finding methods for doing so where it was hitherto impossible and in creating entirely new products that change the way we interact with our world. Big data is a technological revolution centered around collecting, storing and processing more data of more types than ever before.
One way we are seeing the larger connections hidden within the various and high volumes of data points that make big data arise to the surface is through solutions like PolySpot. This technology scores big in the realm of connectors with over one hundred different types.
Megan Feil, February 12, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
ApacheCon Keynotes Announced
February 12, 2013
ApacheCon is the premier open source software conference, highlighting the Apache Software Foundation. This year’s conference will be held from February 24th to March 2nd in Portland, Oregon. Exciting keynote speakers have just been announced for this year’s upcoming conference. Read more in the MarketWatch article, “ApacheCon North America 2013 — Conference Keynotes Announced.”
The article begins:
“The Apache Software Foundation announces ApacheCon Keynotes (http://na.apachecon.com/keynotes/) by thought leaders Theo Shlossnagle, OmniTI; Steve Holden, The Open Bastion and Luke Kanies, Puppet Labs. The Apache Software Foundation (ASF), established in 1999, oversees 150+ Apache projects led by a volunteer community of over 350 individual Members and 3,000 Committers across six continents. Apache powers half the Internet, terabytes of data, teraflops of operations and billions of objects. Its aim is to shepherd, develop, and incubate Open Source innovations.”
Sponsors of this year’s conference include: Citrix, Hortonworks, Red Hat, IBM, and LucidWorks among others. LucidWorks is a consistent investor in the Apache Software Foundation, offering support, training, and value-added software for Apache Lucene and Solr. LucidWorks also employs a record number of Apache Lucene/Solr committers. Regardless of your needs, LucidWorks has a solution that will benefit your enterprise.
Emily Rae Aldridge, February 12, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
