DarkCyber noted “Algolia Finds $110M from Accel and Salesforce for Its Search-As-a-Service, Used by Slack, Twitch and 8K Others.” The write up reports that the company has “closed a Series C of $110 million, money that it plans to invest in R&D around its search technology, including doubling down on voice, and further global expansion in Europe, North America and Asia Pacific.”
Amazon Loses JEDI: Now What?
October 26, 2019
Friday (October 25, 2019) Amazon and the Bezos bulldozer drove into a granite erratic. The Department of Defense awarded the multi-year, multi-billion dollar contract for cloud services to Microsoft. “Microsoft Snags Hotly Contested $10 Billion Defense Contract, Beating Out Amazon” reported the collision between PowerPoint’s owner and the killing machine which has devastated retail.
CNBC reports:
If the Joint Enterprise Defense Infrastructure deal, known by the acronym JEDI, ends up being worth $10 billion, it would likely be a bigger deal to Microsoft than it would have been to Amazon. Microsoft does not disclose Azure revenue in dollar figures but it’s widely believed to have a smaller share of the market than Amazon, which received $9 billion in revenue from AWS in the third quarter.
The write up pointed out:
While Trump didn’t cite Amazon CEO Jeff Bezos by name at the time, the billionaire executive has been a constant source of frustration for the president. Bezos owns The Washington Post, which Trump regularly criticizes for its coverage of his administration. Trump also has gone after Amazon repeatedly on other fronts, such as claiming it does not pay its fair share of taxes and rips off the U.S. Post Office.
There are other twists and turns to the JEDI story, but I will leave it to you, gentle reader, to determine if the Oracle anti-Amazon campaign played a role.
There are some questions which I discussed with my DarkCyber team when we heard the news as a rather uneventful week in the technology world wound down. Let’s look at four of these and the “answers” my team floated as possibilities.
Question 1: Will this defeat alter Amazon’s strategy for policeware and intelware business?
Answer 1: No. Since 2007, Amazon has been grinding forward in the manner of the Bezos bulldozer with its flywheel spinning and its electricity sparking. As big as $10 billion is, Amazon has invested significant time and resources in policeware and intelware inventions like DeepLens, software like SageMaker, and infrastructure designed to deliver information that many US government agencies will want and for which many of the more than 60 badge-and-gun entities in the US government will pay. The existing sales team may be juggled as former Microsoft government sales professional Teresa Carlson wrestles with the question, “What next?” Failure turns on a bright spotlight. The DoD is just one, albeit deep pocket entity, of many US government agencies needing cloud services. And there is always next year which begins October 1, 2020.
Question 2: Has Amazon tuned its cloud services and functions to the needs of the Department of Defense?
Answer 2: No. Amazon offers services which meet the needs of numerous government agencies at the federal as well as local jurisdictional levels. In fact, there is one US government agency deals with more money than the DoD that is a potential ATM for Amazon. The Bezos bulldozer drivers may be uniquely positioned to deliver cloud services and investigative tools with the potential payout to Amazon larger than the JEDI deal.
Amazon AWS Revenue
October 25, 2019
Amazon’s third quarter 2019 results revealed that net sales went up. The number of interest in Harrod’s Creek is AWS. The company’s data report:
- AWS revenue hit $9 billion, up from $6.7 billion in the third quarter of 2018
- Amazon rolled out a fully managed service for business forecasting
- The Quantum Ledger Database is now available as a fully managed service
- AWS cut prices of storage for several classes of service.
Net net: Plenty of cash but Microsoft’s cloud service may be nibbling at some service areas in which Amazon had minimal competition for a number of years.
Stephen E Arnold, October 25, 2019
Security Industry Blind Spot: Homogeneity
October 24, 2019
Push aside the mewlings about Facebook. Ignore Google’s efforts to quash employee meetings about unionization. Sidestep the phrase “intelligent cloud revenue.”
An possibly more significant item appeared in “Information Security Industry at Risk from Lack of Diversity.” The write up states:
The Chartered Institute of Information Security (CIISec) finds that 89 percent of respondents to its survey are male, and 89 percent over 35, suggesting the profession is still very much in the hands of older men.
Furthermore, the security industry is wallowing in venture funding. That easy money has translated into a welter of security solutions. At cyber security conferences, one can license smart monitoring, intelligent and proactive systems, and automated responses.
The problem is that this security country club may be fooling itself and its customers.
The write up quotes from the CIISec report, presenting this segment:
“If the industry starts to attract a more diverse range of people whilst spreading awareness of the opportunity available, we could be well on the way to truly modernizing the industry,” adds Finch. “Key to all this will be both organizations and individuals having a framework that can show exactly what skills are necessary to fulfill what roles. This will not only help hire the right people. It will also mean that it the routes to progress through an individual’s career are clearly marked, ensuring that individuals who enthusiastically join the industry don’t over time become jaded or burn out due to a lack of opportunity.”
Partially correct opines DarkCyber. The security offered is a me-too approach. Companies find themselves struggling to implement and make use of today’s solutions. The result? Less security and vendors who talk security but deliver confusion.
Meanwhile those bad actors continue to diversify, gain state support, and exploit what are at the end of a long day, vulnerable organizational systems.
Stephen E Arnold, October 24, 2019
Automating Machine Learning: Works Every Time
October 24, 2019
Automated machine learning, or AutoML, is the natural next step in the machine learning field. The technique automates the process of creating machine learning models, saving data scientists a lot of time and frustration. Now, InfoWorld reports, “A2ML Project Automates AutoML.” Automation upon automation, if you will.
An API and command-line tools make up the beta-stage open source project from Auger.AI. The company hopes the project will lead to a common API for cloud-based AutoML services. The API naturally works with Auger.AI’s own API, but also with Google Cloud AutoML and Azure AutoML. Writer Paul Krill tells us:
“Auger.AI said that the cloud AutoML vendors all have their own API to manage data sets and create predictive models. Although the cloud AutoML APIs are similar—involving common stages including importing data, training models, and reviewing performance—they are not identical. A2ML provides Python classes to implement this pipeline for various cloud AutoML providers and a CLI to invoke stages of the pipeline. The A2ML CLI provides a convenient way to start a new A2ML project, the company said. However, prior to using the Python API or the CLI for pipeline steps, projects must be configured, which involves storing general and vendor-specific options in YAML files. After a new A2ML application is created, the application configuration for all providers is stored in a single YAML file.”
Krill concludes his write-up by supplying this link for interested readers to download A2ML from GitHub for themselves.
Cynthia Murrell, October 24, 2019
Attorneys Are Getting Better at Tech But There Are Still Some Challenges
October 24, 2019
The best attorneys put bad actors in prison, but in order to do that they need to gather evidence to support their cases in court. With the plethora of data types and sources, attorneys must organize it for quick recall, but data also comes with its own mistakes. JD Supra reveals the, “Top Five Data Collection Mistakes” and ways to avoid them in the litigation process.
There are two main data types: traditional and nontraditional. Users create traditional data, organize and place it in workflows. Nontraditional workflows comes from sources there have few or no collection or processing procedures. These usually come from social media, chat applications, cloud platforms, and text messages. Attorneys need to determine what data types they are handling in litigation, but be aware of potential mistakes.
The easiest mistake to make is not realize that different data types require different collection methods. Extracting information from a computer requires knowledge about its operating system and manufacturer. Cell phone data has its own complications, such as if the data is backed up on a cloud or if the vendor must be contacted to retrieve metadata. Discovering who owns data is another issue. Data is stored on personal devices, the cloud, third party systems, and more. Ownership becomes questionable as well as if data must be shared if not physically owned. Governance policies, customer workflows, and data maps are necessary in order to address data ownership.
Proportionality cannot be ignored. A court could rule that retrieving data outweighs its usefulness. Any data, however, could change a case:
“As always, the success of this argument will depend on the specific facts of a case. For example, one federal court held that a request for text messages was disproportional to the burden of collecting and producing them even though they had been produced in a pre-litigation investigation because the text messages only added minimal evidentiary value to the case. Litigators must be able to clearly articulate a proportionality argument in order to successfully avoid the production of minimally relevant/useful data.”
Misunderstanding proportionality is understandable, but not recognizing data structure and storage is a beginner’s mistake. In order for eDiscovery algorithms to work, they need to be programmed to scan data from different database structures and storage devices. Programming the algorithm wrong is the same as expecting a US electric appliance to work in another country. Data structure and storage is not universal. Attorneys need to remember to cover all data points, search everything. Another amateur mistake is forgetting to collect data that does not provide context for raw data, it is like trying to decipher a secret code without the cipher key.
These are simple mistakes to make, but with new technology and data types new mistakes will develop. Keeping abreast of new trends, technology, communication methods, and data laws will prevent them from appearing.
Whitney Grace, October 24, 2019
Tracking Trends in News Homepage Links with Google BigQuery
October 17, 2019
Some readers may be familiar with the term “culturomics,” a particular application of n-gram-based linguistic analysis to text. The practice arose after a 2010 project that applied such analysis to five million historical books across seven languages. The technique creates n-gram word frequency histograms from the source text. Now the technique has been applied to links found on news organizations’ home pages using Google’s BigQuery platform. Forbes reports, “Using the Cloud to Explore the Linguistic Patterns of Half a Trillion Words of News Homepage Hyperlinks.” Writer Kalev Leetaru explains:
“News media represents a real-time reflection of localized events, narratives, beliefs and emotions across the world, offering an unprecedented look into the lens through which we see the world around us. The open data GDELT Project has monitored the homepages of more than 50,000 news outlets worldwide every hour since March 2018 through its Global Frontpage Graph (GFG), cataloging their links in an effort to understand global journalistic editorial decision-making. In contrast to traditional print and broadcast mediums, online outlets have theoretically unlimited space, allowing them to publish a story without displacing another. Their homepages, however, remain precious fixed real estate, carefully curated by editors that must decide which stories are the most important at any moment. Analyzing these decisions can help researchers better understand which stories each news outlet believed to be the most important to its readership at any given moment in time and how those decisions changed hour by hour.”
The project has now collected more than 134 billion such links. The article describes how researchers have used BigQuery to analyze this dataset with a single SQL query, so navigate there for the technical details. Interestingly, one thing they are looking at is trends across the 110 languages represented by the samples. Leetaru emphasizes this endeavor demonstrates how much faster these computations can be achieved compared to the 2010 project. He concludes:
“Even large-scale analyses are moving so close to real-time that we are fast approaching the ability of almost any analysis to transition from ‘what if’ and ‘I wonder’ to final analysis in just minutes with a single query.”
Will faster analysis lead to wiser decisions? We shall see.
Cynthia Murrell, October 17, 2019
Algolia: Cash Funding Hits $184 Million
October 15, 2019
Exalead was sucked into Dassault Systèmes. Then former Exaleaders abandoned ship. Algolia benefited from some Exalead experience. But unlike Exalead, Algolia embraced venture funding with cash provided by Accel, Point Nine Capital, Storm Ventures, and Y Combinator, among others.
The write up adds:
Having Salesforce as a strategic backer in this round is notable: the CRM giant currently does not have a native search product in its wide range of cloud-based services for enterprises, instead opting for endorsed integrations with third parties, such as Algolia competitor Coveo. The plan will be to further integrate with Salesforce although no products to speak of as of yet.
The challenge will be to go where few search and retrieval systems have gone before.
Some people have forgotten the disappointments and questionable financial tricks promising search vendors delivered to stakeholders and customers.
With venture firms looking for winners, returns of 20 percent will not deliver what the sources of the funds expect. The good old days of a 17X return may have cooled, but generating an 8X or 12X return may be a challenge.
Why?
In the course of our researching and writing the enterprise search report in 2003 to 2006 and out and our subsequent work, several “themes” or “learnings” surfaced:
- Good enough search is now the order of the day; that is, an organization-wide search system does not meet the needs of many operating units. Examples range from the legal department to research and development to engineering and the drawings plus data embedded in product manufacturing systems to information under security umbrellas with real time data and video content objects. Therefore, the “one solution” approach dissipates like morning fog.
- Utility search from outfits like Amazon are “good enough.” This means that a developer using Amazon blockchain services and workflow tools may use the search functions available from Amazon. Maybe Amazon will buy Algolia, but for the foreseeable future, search is a tag-along function, not a driver of the big money apps which Amazon is aiming toward.
- Search, regardless of vendor, must spend significant sums to enrich the functions of the system. Natural language processing, predictive analytics, entity extraction, and other desired functions are moving targets. Adding and tuning these capabilities becomes expensive. And it the experiences of Autonomy and Fast Search & Transfer are representative, the costs become difficult to control.
DarkCyber hopes that Algolia can adapt to these research factoids. If not, search and retrieval may be rushing toward a disconnect between revenues, sustainable profits, and investor expectations.
The wheel of fortune is spinning. Where will it stop? On a winner or a loser? This is a difficult question to answer, and one which Attivio, BA-Insight, Coveo, Elastic, IBM Watson, Lucidworks, Microsoft, Sinequa, Voyager Search, and others have been trying to answer with millions of dollars, thousands of engineering hours, and massive investments in marketing. I am not including the search vendors positioned as policeware and intelware; for example, BAE NetReveal, Diffeo, LookingGlass, Palantir Technologies, and Shadowdragon, among others.
Worth monitoring the trajectory of Algolia.
Stephen E Arnold, October 15, 2019
Real Life Q and A for Information Access Allegedly Arrives
October 14, 2019
DarkCyber noted “Promethium Tool Taps Natural Language Processing for Analytics.” The write up, which may be marketing oriented, asserts:
software, called Data Navigation System, was designed to enable non-technical users to make complex SQL requests using plain human language and ease the delivery of data.
The company developing the system is Promethium, founded in 2018, may have delivered what users have long wanted: Ask the computer a question and get a usable, actionable answer. If the write up is accurate, Promethium has achieved with $2.5 million in funding a function that many firms have pursued.
The article reports:
After users ask a question, Promethium locates the data, demonstrates how it should be assembled, automatically generates the SQL statement to get the correct data and executes the query. The queries run across all databases, data lakes and warehouses to draw actionable knowledge from multiple data sources. Simultaneously, Promethium ensures that data is complete while identifying duplications and providing lineage to confirm insights. Data Navigation System is offered as SaaS in the public cloud, in the customer’s virtual private cloud or as an on-premises option.
More information is available at the firm’s Web site.
Stephen E Arnold, October 14, 2019
A List of Enterprise Search Vendors
October 7, 2019
DarkCyber does not follow the enterprise search sector. In fact, two of the flagships from the 2000s found themselves caught in embarrassing financial missteps. Why? It certainly suggests that making big bucks from a search and retrieval service is difficult.
We came across a Web site called Trust Radius. This site has a section devoted to enterprise search. What we found interesting is that the site lists what seem to be the key players in the sector today. With most LE and intel policeware platforms relying on open source search like Lucene, DarkCyber was quite surprised with the line up of systems and the information provided by Trust Radius.
Here’s the list of vendors in alphabetical order, a method of presenting information which is not in favor with some whiz kids:
3RDi Search
Aderant Handshake (knowledge management for law firms)
Agree Ya Site Administrator
Algolia
Amazon Cloud Search (Lucene)
Apache Lucene
Apache Solr
Expert Systems Cogito Discover
Constructor.io Search
Coveo
Customer Matrix (customer support)
Dassault Systems Exalead (Exalead)
Dieselpoint
Elasticsearch (Elastic)
Fabasoft Mindbreeze
Fabasoft Mindbreeze Inspire
Google Search Appliance (discontinued)
IBM Watson (once Omnifind)
IBM Watson Discovery for Salesforce
IBM Watson Explorer
IManage Insight (Interwoven, Autonomy, HP, now a standalone)
Inbenta Enterprise Search
Lookeen Desktop Search (listed as Enterprise Search however)
Lucidworks Fusion ($100 million in funding)
Maana
Microfocus IDOL (Autonomy to HP to HPE to Microfocus)
Microsoft Azure (Fast Search & Transfer)
Microsoft Bing Search
Perceptive Search (ISYS Search Software to Lexmark to Highland)
Rocket NXT Enterprise Search (Aerotext)
Rockset
Searchify
Search Spring (product search)
Search Tap
Search Unify
Sinequa
SLI Systems (e commerce)
Swiftype
Synacor Video Search & Discovery
TeraText Searchable Archive for Files and Email (SAIC)
Zakta
What DarkCyber finds interesting is the omission of outfits like Oracle Endeca, Antidot, and Blossom. Also, of this listing of 41 “search systems” there are multiple enterprise search products from single companies like IBM and Microsoft. There are also e-commerce search systems and systems which do not handle enterprise content because the service supports desktops. There are two “no longer around” products and a weird blend of search utilities with text processing features. In short, this list is illustrative of the chaos, confusion, and craziness that makes some information technology professionals to buy a solution that just delivers key word and some option features.
DarkCyber believes that Amazon’s approach is likely to gain traction. That’s bad news for most of the companies on this list, particularly search vendors who manage to confuse individuals or the smart software used to create this list at Trust Radius.
It seems that the message from this list is that search is a bit of a dog’s breakfast—just as it has been for decades.
Stephen E Arnold, October 7, 2019
Open Source: Everything New Is Old Again
October 7, 2019
The Andreessen Horowitz open source info blitz contains some good stuff. You will want to read the essay “Open Source: From Community to Commercialization” and, if you qualify, download the pdf of lecture notes. We noted this statement from the essay about the SaaS open source business model:
In a SaaS model, you provide a complete hosted offering of the software. If your value and competitive edge is in the operational excellence of the software, then SaaS is a good choice. However, since SaaS is usually based around cloud hosting, there is the potential risk that public clouds will choose to take your open source code and compete.
Accurate.
We noted this statement at the end of the article:
I [Peter Levine / Jennifer Li?] believe Open Source 3.0 will expand how we think of and define open source businesses. Open source will no longer be RedHat, Elastic, Databricks, and Cloudera; it will be – at least in part – Facebook, Airbnb, Google, and any other business that has open source as a key part of its stack. When we look at open source this way, then the renaissance underway may only be in its infancy. The market and possibilities for open source software are far greater than we have yet realized.
Correct.
Years ago, the DarkCyber team undertook a study of a dozen open source software vendors specializing in search and retrieval. Today, most of those vendors have embraced “artificial intelligence”, “predictive analytics”, and “natural language processing”. That’s because search is a utility and the developers and vendors of general purpose open source software have to differentiate themselves. In the course of that research, DarkCyber noted several things.
- Big companies in 2008 were among the most enthusiastic testers and eventually users of open source software. Why? Our data suggested that open source allowed users of commercial proprietary software more freedom to make changes. Bug fixes would often arrive in a more timely way. Plus, the IBM- and Oracle-style license fees did not come along for the ride. That is probably true in some cases today.
- Open source was a free lunch. The developers often contributed for the common good; others created and made available open source software as a way to demonstrate and prove their capabilities. Translation, as one person told one of my researchers, “A job, man. Big bucks.”
- Monetization was mostly “little plays”; that is use our free stuff and then pay for support or proprietary extensions.
Flash forward to today. Some of these three decade old findings may still be in play, but the context is now very different.
What’s changed?
For the first time, meta plays are possible. Forget the investment, merger, and acquisition angles that motivate venture capital firms. Think in terms of just using Amazon and paying for what you need.
Start ups no longer just use Microsoft because it is available and works. Start ups use Amazon because it appears to be open source, cheap or subsidized, and available globally.
The challenge this presents to open source is significant. DarkCyber is not convinced that open source developers, users of open source software, analysts, and other professionals recognize what Amazon’s meta play and strategy is doing; that is, creating a new context of open source.
Want to learn more about Amazon’s meta play for open source? Write seaky2000 at yahoo dot com and inquire about our Amazon strategy webinar. Note: It’s not a freebie.
Everthing new is old again, including vendor lock in.
Stephen E Arnold, October 7, 2019
-
- Subscribe to Beyond Search
Feature archive
News archive
-
