Elastic Teams With Startup Insight.io for Semantic Search
August 10, 2018
We’ve learned that a Search company we’ve been following with some interest, Elastic, is pairing with a Palo Alto-based startup to develop and integrate semantic search tools. Computer Weekly shares some details in, “Elastic Puts ‘Semantic Code Search’ Into Stack With Insight.io.” Writer Adrian Bridgwater tells us:
“Known for its Elasticsearch and Elastic Stack products, Elastic insists that Insight.io’s technology is ‘highly complementary’ to other Elastic use cases and solutions—indeed, Insight.io is built on the Elastic Stack. Insight.io provides an interface to search and navigate the source code that is said to ‘go beyond’ simple free text search. Current programming language support includes C/C++, Java, Scala, Ruby, Python, and PHP. This ‘beyond text search’ function gives developers the ability to search for code pertaining to specific application functionality and dependencies. Essentially it provides IDE-like code intelligence features such as cross-reference, class hierarchy and semantic understanding. The impact of such functionality should stretch beyond exploratory question-and-answer utility, for example, enabling more efficient onboarding for new team members and reducing duplication of work for existing teams as they scale.”
According to Elastic’s CEO, integration of the technology will be familiar to anyone who observed how they did it with past acquisitions, like Opbeat and Prelert. We’re also assured that all of Insight.io’s workers are being welcomed into Elastic’s development fold. Bridgwater notes that, with the startup’s Beiging-based engineering team, Elastic now has its first “formal” dev team located in China. Founded in 2012, Elastic is now based in Mountain View, California.
Cynthia Murrell, August 10, 2018
Qwant Search Now Integrated into Vivaldi Browser
July 27, 2018
We notice that French search system Qwant has been working to expand its reach. Earlier this year, the French delegation to China suggested that country consider implementing Qwant. We observed at the time that this was an interesting direction for the privacy-centered platform. Now, we learn the search engine has made its way into a rising browser from the post, “Vivaldi Update Integrates Qwant Search Engine” at gHacks. Writer Martin Brinkmann reports:
“Qwant promises that it ‘does not collect data about its users when they search,’ and that it does not use ‘any cookie nor any tracking device’ to track the browsing habits of users or create tracking profiles. The search engine does not put searchers into filter bubbles either as users from the same region will get the same set of results when they search for the same terms. You can select Qwant with a click on the small down arrow icon next to the search symbol in the search bar, or by opening the Search preferences vivaldi://settints/search/. There you can make Qwant the default search engine if you want and enable use as a private search engine. Last but not least, you may also use the nickname q to run searches on Qwant from Vivaldi’s address bar. Just type q searchterm to do so.”
Billed as a browser for “power users,” Vivaldi tends to put out a new release about 4 times a year. Its version 1.15 was released in April, and the inclusion of Qwant takes place in the most recent of three updates pushed out since then. See the write-up for a list of the other improvements. Vivaldi still captures just a small portion internet search traffic around the world, but is clearly working to grow those numbers. Founded in 2014, Vivaldi is based in Oslo, Norway. The Paris-based Qwant was founded in 2011 and launched its browser in 2013. Qwant incorporates some of the spirit of the Pertimm search and retrieval system. Pertimm was, shall we say, quirky.
Cynthia Murrell, July 27, 2018
Insight into Google Image Search
July 22, 2018
I read “This Is What Happens When You Google the Word “Idiot.” The insight pivots on a query sent to Google Images for the word “idiot.” The results presented images of the US president. The same query fed to Bing generates a set of results without the image of Donald Trump. Here’s the explanation about the “why” of these results:
Google states that “[image search] analyzes the text on the page adjacent to the image, the image caption and dozens of other factors to determine the image content.” Added to that, Google uses sophisticated algorithms to remove duplicate images and ensure that the best quality images are presented first in your results. What this means is that whoever writes an article determines (mostly, there are other factors too) whether an image appears in Google Image Search results or not. This partly depends on the keywords they use adjacent or in the caption of the image, not necessarily the “content” of the image. Also, Google indexes the images on a website the same way it indexes web pages, by crawling across the Internet periodically. A quick investigation of the pages in the search results for the word “idiot” proves this to be true. In each of the links where Donald Trump’s image appears, the word “idiot” appears as a keyword and in most cases close to his image or sometimes in the caption.
Seems simple enough. Word plus image equals relevance.
Stephen E Arnold, July 21, 2018
Are Some Google Docs Exposed to Web Indexing Systems?
July 21, 2018
Recently, Russian search giant Yandex reported seeing Google Docs turn up in search results. Previously, this was thought to be impossible. However, this brings up a lot of questions that others have taken for granted: namely, how secure are documents on the cloud? This was looked at more closely in the Media Post story, “Private Google Docs Serve Up In Yandex Search Engine Results.”
According to the story:
“[O]ther search engines can only serve up Google documents that had either been deliberately made public by its authors or when a user publishes a link to a document and makes it available for public access and search… Saving and protecting users’ personal data is our main priority for search engines. A Yandex spokesperson said the search only yields files that don’t require logins or passwords.”
For its part, Google appears to deflect the Yandex observation. Regardless, the Yandex assert arrives near the muddy heels of other security woes like the idea that our Gmail messages and their content could be used by developers. With the Android matter behind it, the EU may look at access to certain Google content.
Patrick Roland, July 21, 2018
Why So Few Search Vendors Index the Web?
July 5, 2018
How many companies are indexing the Surface Web, the Dark Web, and the other bits and pieces which comprise the accessible Internet?
The answer is, “Not many most people can name.”
Another question, “Why don’t more companies just index the Internet?
The answer is, “Money, resources, time, expertise, and generating revenue.”
The write up from 2012 “How t Crawl a Quarter Billion Webpages in 40 Hours” surfaced again after an absence of six years. The article remains valid even thought the principal change in the last 72 months is the increased concentration of Google’s index. Microsoft, a company which insists that its Bing system, provides an alternative to Google has not significantly stopped Google’s market magnetism. Many of the systems which are marketed as Web indexes like Duckduckgo.com and Startpage.com are metasearch engines; that is, the users’ queries are passed to other services and may be supplemented with some original crawling. A bit of fiddling ensures that the results lists seem to be different. But there is a sameness to the result sets, particularly on popular queries. Yandex, the Russian Web search system, does a good job of handling certain sets of domains, but the overall coverage is not that different from what one can find in Google or its country centric indexes.
What’s interesting about “How to Crawl” from 2012 is the use of the Amazon system. This is important because the plumbing required to index the Internet can be large, complicated, and expensive.
Does Amazon still operate its A9 Web index? We have heard yes and no as an answer to this question. With a significant number of queries seeking product information, it makes sense to consider Amazon as a potential competitor to Bing, Google, and Yandex.
After rereading the “How to Crawl” paper, one thing jumps out. The notion that a quarter of a billion pages is a non trivial chunk of the Internet is interesting but a bit misleading. There may be upwards of more than 30 billion indexable Web pages. A large number of these content objects exist in mobile forms; thus, deduplication becomes an interesting issue. That’s why the Google has multiple indexes.
The big question becomes, “Is there another company able to compete with Google?”
After reading “How to Crawl” after a lapse of six years, the answer may be,
“Very, very few companies. And some of the outfits indexing the Surface and Hidden Internet may not make their activities public.”
Monocultures are okay but these can be vulnerable to something the monoculture cannot resist. Is Google like today’s banana? What happens if a blight attacks? One can shift to durian I suppose.
Stephen E Arnold, July 5, 2018
Calendars Are Now Search… If One Is Busy and Eschews Print Schedulers
July 3, 2018
You might not think it, but your doctor’s appointments and dinner parties are a big deal to search companies. With the rise of digital assistants like Siri and Alexa, your datebook is the next big horizon to conquer. The ways in which this will unfold might surprise you, according to a recent Japan Today story, “Google’s ‘Reserve’ Tool Winning Converts and Taking Search to the Next Level.”
According to the story:
“[S]even software firms that supply schedule data to Google described the volume as significant, with as much as 75 percent of bookings representing new customers. Consumers like the convenience. Business owners say the tool is putting their names in front of more potential clients.”
It is no coincidence that several experts are touting the ability of digital assistants to help with travel planning. In a weird way, voice search can now do a lot of the work of a travel agent, in terms of eyeing your schedule, finding deals, and even purchasing flight tickets. From getting reservations to booking flights to making sure someone is picking up junior from soccer practice, there is a revolution happening in search and how it relates to daily life. Search and scheduling: A wonderful way to fill one’s day with useful activities.
Patrick Roland, July 3, 2018
Search History? No Big Deal Maybe
June 29, 2018
What you search for leaves a digital footprint, or more accurately, a fingerprint. So much identifying data is left behind in your search history. However, there are some angles to this predicament many people are overlooking. We realized just how much bad information people are getting after reading a recent Pagal Parrot article, “Searching These Five Things Can Make Trouble For You.”
This odd little story seems to really give some elementary advice on what not to search for, like:
“#2 Your Name- It’s not a big secret that in this era of the internet our privacy questioned. If you try to Google most probably you will get stumble upon some unpleasant results, bad photos of you, outdated information, irrelevant content. we take such things way too seriously. If you find something like this, you want to delete it.”
This is a little obscure, considering there are such worse implications of your search history. For one, it informs all the bots what is sent through your social media feed. So, for example, a simple search about fake news might just land you with a glut of bogus stories. Thankfully, there is better advice out there than not searching your name, like how to wipe your Facebook and Google search history so that you aren’t fed to the algorithm monsters. Much more practical, in our book!
Patrick Roland, June 29, 2018
Search Now Maps Physical Products
June 21, 2018
Search has slowly been creeping into the real world, but rarely have we seen it making a positive impact on our lives when it does. Until now! A new search engine we discovered bridges the gap between the digital world and the physical world with impressively helpful results. We learned more from a recent LifeHacker story, “See What’s Actually In You Skincare Products With this Search Engine.”
The site is called Incidecoder, and this is what the article had to say:
“You can search for individual ingredients and popular products by name on INCIDecoder, and it will list out all of the ingredients as well as descriptions of what they actually are and what they do. Because while I know what Aqua is, I’m less familiar with PPG-26-Buteth-26 and Ethylhexylglycerin.”
Another way search is sneaking into the real world is in the fashion industry, where AI and predictive analytics can tell designers what look is hot now, but also what trends will pop up in the future. Expect to see more of this trend beyond fashion and beauty aids. This seems like it will be a huge market for blending search and AI into our daily lives.
Patrick Roland, June 21, 2018
Google Search Evaluator Handbook
June 12, 2018
How does Google shape search results? The pay to play search giant allegedly has a guide for individuals who interact with the automated search system. The information appears at this link. The information dates to 2017. There may be a revision or additional instructional material online. If we come across that information, we will post the link in Beyond Search.
The information is described as “Search Quality Rating System.” A sample from the table of contents for the documentation appears below:
An example of the information provided to the human making quality decisions appears below:
Here’s the guidance for queries about kittens:
In my first Google monograph (The Google Legacy, 2004), I gathered about 100 factors allegedly used to determine “quality” of Google search results. What I found interesting is that Google’s listing has many more entries than I identified 14 years ago.
Quality, it seems, is more difficult to pinpoint today. The rules for relevance, however, seem to have been marginalized.
I do know that in order to obtain useful results from Google, I have to craft my queries carefully. In fact, creating a query for an old school Boolean system is easier to do. Google has added on to what was essentially a key word system by wrapping layers of software around an ageing core.
Worth spending a few minutes with the document in my opinion.
Stephen E Arnold, June 12, 2018
Want Info about a Small Town? Hit the Library
June 6, 2018
For many, libraries are obsolete, deader than a Peruvian mummy. This is true for some, but if you live in a small town then libraries are far from dead. Big news outlets cover global issues, so they skip over small town stories. Small towns, however, still have news and the residents want to read it. Where do they go to get information when local newspapers dried up? They go to the local library. The Atlantic shares the story, “The Libraries Bringing Small-Town News Back To Life” and how the US’s smaller cities still rely on libraries as information centers.
Libraries have seen their budgets slashed, branches closed down, and the librarian profession has been traded for para-professionals. Yet people still go to libraries and even trust librarians over journalists and other news sources. Why? Librarians also understand the importance of accurate information and their sources.
Librarians have picked up the slack where local news sources fail or disappeared. In some towns, being a news source has increased participation at libraries. The write up stated:
“Various types of community building are happening across the nation. In some cities, libraries are partnering with established news sources, teaming up in Dallas to train high schoolers in news gathering or hosting a satellite studio in Boston for the public radio station WGBH. In San Antonio, the main library offers space to an independent video news site that trains students and runs a C-SPAN-style operation in America’s seventh-biggest city. (That site was the only video outlet covering a mayoral debate last year in which the incumbent mayor’s comments on poverty became a national story—and may have contributed to her electoral defeat.)”
Where once libraries use to store information, they are turning into the information source. They are also reinforcing important information literacy skills, which are in desperate need as fake news and instant search weakens people’s judgment skills.
Whitney Grace, June 6, 2018
-
- Subscribe to Beyond Search
Feature archive
News archive
-
