Fess Up: Elasticsearch Is a Threat to Proprietary Search and Retrieval
December 1, 2020
We have been poking around the world of Elasticsearch-based information retrieval systems. There are some interesting plays; that is, entrepreneurs use Elasticsearch (Shay Banon’s open source system) as a platform.
Fess provides Elasticsearch for personal use, although one can employ the system for an organization. The system is:
Fess is Elasticsearch-based search server, but knowledge/experience about Elasticsearch is NOT needed because of All-in-One Enterprise Search Server. Fess provides Administration GUI to configure the system on your browser. Fess also contains a crawler, which can crawl documents on Web/File System/DB and support many file formats, such as MS Office, pdf and zip.
Fess became available in 2019. The CEO of the N2SM, Inc. company is Masaharu Manabe. Demonstrations and links to the code are available at this link. A fee-based version of the software is provided under the name N2 Search. More information about the for fee version is here. A discussion forum is available at this link.
Observation: The Elasticsearch ecosystem is providing alternatives to the proprietary search systems. Beyond Search thinks that some vendors of proprietary search software are likely to be see Elasticsearch as digital kudzu. Good news or bad news for the Coveos, Fabasofts, and Microsoft Fast type folks? That’s a question some of these types of vendors stakeholders may be asking as they beat the bushes for deals in customer service, chatbots, business intelligence, and smart software services.
Stephen E Arnold, December 1, 2020
Elastic: The Add Value to Open Source Outfit Bounces Along
November 25, 2020
Elastic Adds New Features to Enterprise Search, Observability, and Security Solutions
Search and data-management firm Elastic has some new features to crow about. BusinessWire posts “Elastic Announces Innovations Across its Solutions to Optimize Search and Enhance Performance and Monitoring Capabilities.” One new tool is Kibana Lens, a visual data analysis tool with a drag-and-drop interface described as intuitive. There is also a beta launch of the searchable snapshots, an efficient way to manage data storage tiers with searchable snapshots. The press release tells us:
“New expanded Elastic Observability features, including user experience monitoring and synthetics, give developers new tools to test, measure, and optimize end-user website experiences. The launch of a new dedicated User Experience app in Kibana provides Elastic customers with an enhanced view and understanding of how end users experience their websites. In addition, Elastic customers can use the new user experience monitoring feature to review Core Web Vitals, helping website developers interpret digital experience signals. Elastic users can also leverage a dev preview release of synthetic monitoring in Elastic Uptime to simulate complex user flows, measure performance, and optimize new interaction paths without impact to a website’s end users. The combination of these two new observability features gives Elastic customers a deeper view of their customers’ digital experience before and after a site update is deployed.”
See the write-up for its list of specific updates and features to Elastic’s Enterprise Search, Observability, Security, Stack, and Cloud products. Built around open source software, the company prides itself on its user-friendly products that have been adopted by major organizations around the world, from Cisco to Verizon. Elastic began as Elasticsearch Inc. in 2012, simplified its name in 2015, and went public in 2018. The company is based in Mountain View, California, and maintains offices around the world.
Cynthia Murrell, November 25, 2020
Court Case Hunger? Judyrecords Is Available
November 24, 2020
Unable to pay the fee for LexisNexis-type commercial search systems? You are not alone. If you want information from court records, navigate to Judyrecords. Within the last couple of months, the system has added more than 35 million cases. Aren’t these data available for free elsewhere? Sure, if you like going through hoops like verification procedures. Judyrecords lets a user plug in the names of entities and view results. I ran one of my go to queries: “Palantir IBM.” Here are the results:
This may not be important to you, but for those who have to wade through for fee legal search systems, Judyrecords is helpful. But for how long? Yes, that is a good question. For now, however, give it a whirl. Keep in mind that US court systems without online technology or special arrangements for document access prevent the system from being comprehensive. Lawyers enjoy results which must be checked by billable professionals, however.
Stephen E Arnold, November 23, 2020
Open Source Kumbaya in 2020: Pay Me for Support
November 16, 2020
I read “No, Open Source Does Not Mean Includes Free Support.” The write up illustrates one small change in the open source community in the last five or six years. With more and more organizations using open source software as the engine for their “platform” or “system”, individuals who create open source software are shifting. For example, when we worked on the Lucene Revolution conference years ago, there was a lot of talk about the community, the FOSS spirit, the desire to break free of the chains proprietary software vendors locked to licensees, etc.
Compare that kumbaya approach to this statement in the write up:
Don’t get me wrong. I’m happy to help. Selling support is what keeps the lights on here (did I mention the cost of running a web server?). But coming to me under false pretense and/or expecting that I must provide free service on top of a software I gave away without charge is not going to win you any favors. It stops being free, when it starts costing me! My time is valuable. If you want a piece of it, I want money in return. Period.
This is an excellent point. In my own experience, we know that some high profile products would not exist without open source software. In fact, some vendors do not reveal the extent of their dependence on software which can be downloaded and used without providing so much as an email address, let alone a credit card.
Net net: 2020 may become the year in which open source kumbaya is replaced with a different ethos. Come by here but bring a way to pay.
Stephen E Arnold, November 16, 2020
Microsoft: The Joy of Figuring Out What Code Can Do
October 26, 2020
DarkCyber finds Microsoft in an interesting spot. On one hand, Microsoft wants to be open sourcey. The idea of community created and community supported software provides a useful source of ready-to-microwave code nuggets, hints about whom to hire, and an opportunity to reduce the maintenance cost of certain components.
On the other hand, monitoring what’s on GitHub and, more importantly, how code can be used is a sticky wicket.
“RIAA Blitz Takes Down 18 GitHub Projects Used for Downloading YouTube Videos” explains:
Microsoft-owned GitHub has removed today 18 projects from its code-hosting portal following a legal request filed by the Recording Industry Association of America (RIAA)….In a letter sent to GitHub, RIAA argued that the “clear purpose of this source code [the youtube-dl library]” was to “circumvent the technological protection measures used by authorized streaming services such as YouTube” and to allow users to “reproduce and distribute music videos and sound recordings […] without authorization.”
The issue is likely to be a thorny one. Code can be used for many things:
- To perform a function
- A way to learn how to do a task
- Create software unrelated to the GitHub offering.
Microsoft has removed the “offending” software. But the problem could become the seed of a giant junk maple in the main Redmond campus green space. The article makes this point, and it is an important one:
RIAA isn’t alleging the library infringed on its rights, but that the library is illegal in itself.
Just as Microsoft wants to get open sourcey and more social, it finds itself in an interesting spot. Who or what will fertilize and water this tiny take down seed? Exactly what can code do? Exactly to what purposes can code be put? What about software which includes code which can do something a third-party defines as illegal? So many questions for the JEDI knights.
Stephen E Arnold, October 26, 2020
Open Source: A New Slogan Emerges. No Poster Art Yet
October 23, 2020
I read “Huawei’s Open Source Innovation Inspired by Of All, By All, for All.” Interesting. Microsoft is interested in open source. Amazon is semi interested in open source. Google is probably still interested in open source unless the team working on open source lost interest. But Huawei? Huawei is interested in open source. The write up reports:
Huawei has acknowledged the importance of open source and the role it plays in accelerating innovation within the software industry, stating that ecosystems such as openEuler, openGauss, openLooKeng, and MindSpore have created an ecosystem of open source basic software projects….The openEuler, openGauss, openLooKeng and MindSpore open source communities are all ‘led’ by Huawei as the company seeks to lay the groundwork for full-stack hardware and software collaboration.
Does Huawei’s support of open source fit into the strategic plan for Chinese technology?
The article provides a partial answer:
Huawei Cloud & AI Open Source business general manager Du Junping says that open source enables organizations to create innovation and value in an environment that is ‘open, fair, transparent, and secure’. Huawei says it is inspired by the mindset of fostering a sustainable, open source basic software ecosystem ‘Of All, By All, For All’.
Catchy: Of all, by all, for all. Very egalitarian and kumbaya-ish. Is it similar to “Smash the gang of four” or “Have fewer children, raise more pigs”? No, of course not.
Stephen E Arnold, October 23, 2020
Comparison of Elasticsearch, Solr, and Sphinx
October 8, 2020
Search and retrieval underpins most policeware and intelware systems. Open source search software has made life more challenging for vendors of proprietary enterprise search solutions. There are versions of an “in depth” enterprise search analysis like this available for thousands of dollars from marketers like https://www.adroitmarketresearch.com sporting this title:
Enterprise Search Market Demand Analysis and Projected huge Growth by 2025| IBM Corp, Coveo Corp., Polyspot & Sinequa Inc., Expert System Inc., HP Autonomy, Lucidworks, Esker Software Corp., Dassault Systemes Inc., Perceptive Software Inc., and Marklogic Inc.
Notice that none of the search vendors in “Elasticsearch vs. Solr vs. Sphinx: Best Open Source Search Platform Comparison” appears in the Adroit Market Research report. That’s important for one reason: Open source search has driven vendors of proprietary systems into a corner. What’s even more intriguing is that some vendors of enterprise search like Attivio and IBM Corp. use open source search technology but take pains to avoid revealing the plumbing under the house trailer.
The comparison is, for now, available without charge online, courtesy of Greenice. This firm, based in Ukraine, is what I would describe as a DevOps consulting and services company. It’s a mash up of advisory, coding, and technical deliverables.
The comparison contains some useful information; for example:
- Inclusion of examples of the search systems’ visualization capabilities
- Examples of organizations using each of the three systems compared
- Presentation of the analyst’s perception of strengths and weaknesses of each system
- References to machine learning in the context of the three systems.
What caught my attention is the disconnect between the expensive and somewhat over enthusiastic for fee study about search and this free analysis.
Many of the problems in search are a result of what may be described as “over enthusiastic marketing.” This approach to jazzing up what can be accomplished by information retrieval technology has resulted in at least one jail sentence for an enterprise search entrepreneur and may be followed by jail time for other companies’ executives who practice razzmataz sales techniques.
The principal value of the free comparison is that it does a good job of walking through basic information without the Madison Avenue hucksterism. Net net: A free write up with some helpful information.
Stephen E Arnold, October 8, 2020
Google and Its User Privacy: Happy Hunting
October 4, 2020
DarkCyber spotted an open source intelligence tool called GHunt. By the time an open source software becomes publicly available, DarkCyber believes that hardened systems and methods are integrated into specialized policeware and intelware systems. If you want to try to learn more about a particular Google email user, for instance, you may want to take a look at GHunt. There are screenshots and basic information available on Github. Google appears to be taking steps to address some of the “features” which the GHunt software taps. Some interesting open source software becomes available and then disappears; for example, DARPA Memex tools have evidenced this type of behavior. If you want this tool, DarkCyber suggests you move along in a sprightly manner.
Stephen E Arnold, October 4, 2020
Elastic: Making Improvements
August 27, 2020
Elasticsearch is one of the most popular open-source enterprise search platforms. While Elasticsearch is free for developers to download, Elastic offers subscriptions for customer support and enhanced software. Now the company offers some new capabilities and features, HostReview reveals in, “Elastic Announces a Single, Unified Agent and New Integrations to Bring Speed, Scale, and Simplicity to Users Everywhere.” The press release tells us:
“With this launch, portions of Elastic Workplace Search, part of the Elastic Enterprise Search solution, have been made available as part of the free Basic distribution tier, enabling organizations to build an intuitive internal search experience without impacting their bottom line. Customers can access additional enterprise features, such as single sign-on capabilities and enhanced support, through a paid subscription tier, or can deploy as a managed service on Elastic Cloud. This launch also marks the first major beta milestone for Elastic in delivering comprehensive endpoint security fully integrated into the Elastic Stack, under a unified agent. This includes malware prevention that is provided under the free distribution tier. Elastic users gain third-party validated malware prevention on-premises or in the cloud, on Windows and macOS systems, centrally managed and enabled with one click.”
The upgrades are available across the company’s enterprise search, observability, and security solutions as well as Elastic Stack and Elastic Cloud. (We noted Elastic’s welcome new emphasis on security last year.) See the write-up for the specific updates and features in each area. Elasticsearch underpins operations in thousands of organizations around the world, including the likes of Microsoft, the Mayo Clinic, NASA, and Wikipedia. Founded in 2012, Elastic is based in Silicon Valley. They also happen to be hiring for many locations as of this writing, with quite a few remote (“distributed”) positions available.
Cynthia Murrell, August 27, 2020
Twitch: Semantic Search Stream to Lure Gamers, Trolls, and Gals?
July 31, 2020
Amazon Twitch may be more versatile than providing the young at heart with hours of sophisticated content. There are electronic games, trolls (lots of trolls armed with weird icons), and what appear to be females.
Now Twitch will be moving along the content spectrum with the addition of a stream about semgrep. If you are not on a first name basis, semgrep is a semantic search thing. You can join in for free, no waiting rooms, and no big technical hurdles. I suppose one could create a lecture about semantic methods in TikTok 30-second videos which might be a first for the non-invasive, controversial app. Nah, go for Twitch. Skip YouTube and Facebook. Go Bezos bulldozer.
Navigate to https://twitch.tv and go to the jeanqasaur stream. The time on July 31, 2020? The show begins at 4 pm US Eastern time.
The program is definitely perceived by some as super important. A motivated semantic wizard posted a message on the TweetedTimes.com semantic page. Here’s what the message looks like:
DarkCyber’s suggestions:
- Do not become distracted by Raj recruiting, Bad Bunny, or Celestial Fitness. Keep your eye on the grep as it were.
- Sign up because Amazon wants you to be part of the family. Prime members may receive extra Bezos bucks somewhere down the line
- Exercise good grammar, be respectful, and keep your clothes on. Twitch banned SweetSaltyPeach who reinvented herself as RachelKay, Web developer, fashion model, and gamer icon. You may have to reincarnate yourself too.
- Avoid the lure of Animal Crossing Arabia II.
Stephen E Arnold, July 31, 2020
-
- Subscribe to Beyond Search
Feature archive
News archive
-
