Omnity Search: Adjusting Fast and Slow
October 14, 2018
Beyond Search maintains a file about the Omnity search system. We noted that a new white paper became available in April 2018. If you want a copy of the 42 page document, you can download a free copy at this url.
The white paper is interesting because it suggests that the current methods of finding information are “inherently biased.” Omnity’s indexing is different; for example:
Omnity has developed a semantic signature technology that impartially and mathematically articulates the deep structure of a document, and self-assembles by inter-connecting to other documents with similar structure.
Omnity may be the first search and retrieval syst4em to embrace blockchain technology, but we are not 100 percent certain. Frankly we don’t pay much attention to distributed databases because the technology is another spin down database lane and the next big thing mall.
The document contains some interesting diagrams. Some of these remind us of sense making systems for law enforcement and intelligence professionals. The company positions itself against Palantir and Quid as well as Bloomberg and Lexis Nexis. Surprisingly Linguamatics is a “leader” like Omnity.
What is fascinating is that Omnity seems to be embracing the digital currency approach to raising funds. One of the firm’s advisors is the really famous Danny Kahneman.
My recollection is that Omnity was going to knock Google search off its mountain top. Then Omnity shifted to a commercial model like the old Dialog Information Services. Now it is blending findability with blockchain and crypto currency.
More information about the company is at www.omnity.io. Get the white papers. Check out the diagrams. One question is, “Should Palantir and Quid be looking over their individual and quite broad shoulders?”
Omnity’s approach is a good example of search vendors repositioning fast and slow.
Stephen E Arnold, October 15, 2018
DarkCyber for October 9, 2018, Now Available
October 9, 2018
DarkCyber for October 9, 2018, is now available at www.arnoldit.com/wordpress and on Vimeo at https://vimeo.com/293949062
Stephen E Arnold’s DarkCyber is a weekly video news and analysis program about the Dark Web and lesser known Internet services. This week’s program covers two stories related to Israel’s thriving intelligence technology capabilities.
The first story discusses the allegation that NSO, based in Israel, has licensed its Pegasus system to the United Arab Emirates. A number of news services have suggested that NSO has the capability to turn a mobile phone into a remote surveillance device. Another allegation explored in this week’s DarkCyber is that NSO can access an Apple iPhone when only the mobile telephone number is provided to the company. If these assertions are accurate, NSO has leapfrogged other forensic and intelligence related firms’ capabilities.
The second story explores the startup Cobwebs Technologies. The company, founded in 2015, has implemented a wide range of capabilities into one easy to use system. Unlike IBM i2 Analyst Notebook and Palantir Technologies Gotham, Cobwebs Technologies’ approach reduces user training to a few days. Most advanced functions such as generating a relationship map or analyzing a stream of social media content require no programming. Stephen E Arnold, producer of Dark Cyber and author of CyberOSINT: Next Generation Information Access, said: “Cobwebs Technologies simplification of the content acquisition and analytics process makes advanced technology within the reach of most law enforcement and intelligence personnel. Until now, most users of advanced intelligence systems needed some programming knowledge and specialized training in the software system. Cobwebs changes the game in a significant way.”
DarkCyber appears each Tuesday. A special four part series about Amazon’s policeware capabilities begins on October 30, 2018. The program will be available on the Beyond Search blog, YouTube, and Vimeo.
Remember our special four part series about Amazon policeware begins on October 30, 2018.
Kenny Toth, October 9, 2018
HSSCM: Updates from Facebook and Snapchat
October 5, 2018
High school science club management methods are flourishing.
I wanted to highlight two examples of interesting ways to operate publicly traded companies in the spotlight.
The first example of HSSCM comes from Facebook, truly a gold mine of examples. I learned that a Facebook executive sat in a photo op location during the Brett Kavanaugh hearing. My source was “Facebook’s Head of Public Policy Is Supporting the Kavanaugh Nomination, and Some Employees Are Livid.” A Facebook top dog named Joel Kaplan appeared, at least to the Verge, to endorse “his close friend.”
The result, according to the write up, “roiled the social network.” I like the word roiled. The real journalists at the Verge reported:
For Facebook, the controversy over Kaplan represents a new point of division at a company that is still grappling with the Instagram founders’ unexpected departure and the largest data breach in its history. Only when it comes to the Kaplan controversy, it’s not clear to me what the company’s next move should be. The C suite seems to have been annoyed by Kaplan’s attendance, but was initially dismissive of employees’ concerns. (How concerned are employees? My favorite detail in Isaac’s story is that they went into Kaplan’s calendar and learned that he had not in fact taken the 27th as a personal day, as Kaplan initially stated. The calendar was later updated to reflect that it was, indeed, a personal day.)
It appears in the canon of the HSSCM method that it is not necessary to know what senior executives are doing. Furthermore, revisionism in the form of modifying a digital calendar, is supported. I also enjoyed learning that in this particular HSSCM example, the concerns of the employees were not at the top of the to do list.
Implications? I suppose there is a possibility that some MBA might interpret the method as exemplary decision making regarding a chain of command, time allocation, and awareness of the senior staff activities.
The second example concerns Snapchat, a service which I admit I never understood and still do not.
I captured this HSSCM method in the write up “9 Highlights from Snapchat CEO’s 6000-Word Leaked Memo on Survival.” Like leaks at Google and Palantir, the fact that someone released an internal memo suggests that shared values about how to handle confidential information may be lacking. Science club members have to be loyal, right?
My first reaction to the write up was, “6,000 words is a lot of words for a memo.” If I worked at Snapchat, I am not sure I would read that document. Some employees obviously concluded that reading and leaking were proof of the HSSCM approach.
I found a couple of the management principles embedded in the memo semi-interesting.
Item One, I noted the admission that the company moved too quickly. Well, it did break things, including usage of the software. An added plus was social media visibility. Annoyed users expressed their displeasure with a Snapchat design tweak. HSSCM knows how to get publicity. That’s a plus.
Item Two, a service called Discover is good. But the service is a “mess.” Hey, that’s part of the break things. The fix? Make Discover a “lean back” experience. I love the word experience. HSSCM methods deliver experiences, just like Google ads’ redesign which features the word “experience” instead of “increasing revenues”. See the email sent by Google on October 4, 2018, with the friendly, warm experience of the mail address: ads-noreply@google.com.
To sum up, here are the HSSCM methods I extracted from these two case examples:
- It is not necessary to know what senior managers are doing during the day on prime time television for six or seven hours
- It is okay to make it easy to modify digital calendars which is a trendy approach to revisionism
- It is okay to make decision in a hurry, ignore feedback, and then apologize. That is a way to move fast and put broken things back together.
- It is okay to create services which appear to be a mess. Creative destruction, right?
No wonder sign ups for MBA degrees are sinking. Who needs to study management when HSSCM methods are within reach of anyone working at a tech centric company?
Stephen E Arnold, October 5, 2018
Amazon and Its Star Approach to Popularity
September 27, 2018
I suggest you read “Introducing Amazon 4-Star.” Ignore the explanation of using purchaser data to determine what to put in a retail store. Read the article in this context:
What advantage will Amazon have in its policeware business if it offers lower cost, easier to use versions of policeware and intelligence centric software, solutions, and services?
Here in Harrod’s Creek, we have been monitoring Amazon’s move into the law enforcement and intelligence support markets. The rumors of the new Amazon’s headquarters in the DC area are interesting.
Net net?
We think that Amazon’s hoisting of systems like Palantir Gotham and many other US and non US policeware systems will provide data which gives Amazon an opportunity.
We explore some of the larger issues in our four part series about Amazon policeware which will be in the form of DarkCyber videos. These programs become available at the end of October 2018. Watch for details.
In the meantime, think about four star popularity applied to government procurement. Nifty, right?
Stephen E Arnold, September 27, 2018
Can IBM Watermark Neural Networks?
August 8, 2018
Leave it to IBM to figure out how to put their stamp on their AI models. Of course, as with other intellectual property, AI code can be stolen, so this is a welcome development for the field. In the article, “IBM Patenting Watermark Technology to Protect Ownership of AI Models at Neowin, we learn the technology is still in development, and the company hasn’t even implemented it in-house yet. However, if all goes well, the technology may find its way into customer products someday. Writer Usama Jawad reports:
“IBM says that it showcased its research regarding watermarking models developed by deep neural networks (DNNs) at the AsiaCCS ’18 conference, where it was proven to be highly robust. As a result, it is now patenting the concept, which details a remote verification mechanism to determine the ownership of DNN models using simple API calls. The company explains that it has developed three watermark generation algorithms…
These use different methods; specifically:
- Embedding meaningful content together with the original training data as watermarks into the protected DNNs,
- Embedding irrelevant data samples as watermarks into the protected DNNs
- Embedding noise as watermarks into the protected DNNs.
We learned:
“IBM says that in its internal testing using several datasets such as MNIST, a watermarked DNN model triggers an ‘unexpected but controlled response’.”
Jawad notes one drawback as of yet—though the software works well online, it still fails to detect ownership when a model is deployed internally. From another article, “IBM Came Up With a Watermark for Neural Networks” at TheNextWeb, we spotted an interesting tidbit—Writer Tristan Greene points out a distinct lack of code bloat from the watermark. This is an important factor in neural networks, which can be real resource hogs.
For more information, you may want to see IBM’s blog post on the subject or check out the associated research paper. Beyond Search wonders what smart software developers will use these techniques. Amazon, Facebook, Google, Oracle, Palantir Technologies? Universities with IBM research support may be more likely candidates, but that is, of course, speculation from rural Kentucky.
Cynthia Murrell, August 8, 2018
The Fortune Magazine Channels the Onion. Is SNL in the Future of Fortune?
July 31, 2018
Perhaps ola is a good way to approach this Fortune Magazine story. Now the story is satire. Satire triggers in my mind thoughts of the Onion online service, now owned by Univision. I want to mention that Univision is trying to sell the Onion. Maybe the phrase I want is Hasta la vista?
I read “Man Terrified of Palantir, More Terrified to Explain What Palantir Is,” a write up labeled in small light blue type as “Humor. Satire from Fortune.” Yep, a million laughs.
The problem is that the write up is not particularly humorous. If the reader does not pay attention to Palantir Technologies and how it uses the Amazon Web Services platform, the article makes no sense. Therefore, the laughter quotient is small. In the calculus of humor, I would suggest that there is an infinitesimal chuckle in the story.
If one is familiar with Gotham, Metropolitan, the use cases for the software, the write up may be interpreted as an accurate “real” news story.
For me, I think Fortune has another agenda. My hunch is that Fortune wants to do a Buzzfeed dive bombing of the company. Perhaps the story ended up on a dull spike in the humming editorial hive of the magazine.
Here’s a passage I noted:
As Watts described his style of news consumption, it became clear how one man can be so fearful of a company without understanding its purpose.
With Fortune’s “real” story on ICE, I think this statement applies to much of the coverage of Palantir Technologies.
Yep, real news is satire, not fake news. Why not chase down a non fake, non humorless story about the company? Why not explain what the Gotham wheel menu provides a user? I like the wheel menu. Real news about a “wheel” subject.
Stephen E Arnold, July 31, 2018
Amazon and Its Unrest Prediction
July 24, 2018
The Guardian, a “real” newspaper, published “Why the Amazon Boss’s Warning of No-Deal Brexit Unrest Rings Hollow.” The write up is a response to an Amazon executive’s prediction that muffing the Brexit trade deal bunny will lead to “civil unrest.” My interest is not the fate of the UK. I am, however, intrigued by an Amazon executive making a statement similar to those offered by individuals with access to intelligence centric next generation information access systems. (If you want additional information about NGIAs, check out CyberOSINT.)
The question which crossed my mind when I heard about the prediction was:
Does Amazon have access to an NGIA system similar to an IBM Analyst Notebook or a Palantir Gotham?
If the answer is, “No,” then I asked myself:
Does Amazon have its own intelligence analysis system?
From my vantage point in rural Kentucky, I have zero first hand information about Amazon and its possible intelligence capabilities.
It is indeed interesting to have this prediction emitted from what is usually a quite secretive outfit. My instincts suggest that Amazon does have an active intelligence system. This prediction may be a planned or unanticipated factoid by an Amazon executive.
Amazon does have some interesting capabilities, new functions which have been patented, and a number of vendors of policeware and intelware who use the Amazon AWS plumbing.
My takeaway from the prediction and the Guardian type coverage suggests that a closer, more informed look at what Amazon does to inform its executives of possible developments is warranted.
Stephen E Arnold, July 24, 2018
Amazon: Information and Its Pharma Play
June 29, 2018
Amazon sells quite a few health related products. One I found interesting is Amazon’s hydrogen peroxide. That’s an interesting chemical, and I wondered who has order a higher concentration version. I assumed that Amazon could answer that question, among others; for example, who bought certain books describing the use of the compound.
I thought about Amazon’s health products when I read “Amazon to Buy Online Pharmacy PillPack, Jumping Into the Drug Business.” I think the deal is an interesting one, but my view is different. Hey, I live in rural Kentucky, one of the states associated with opioid abuse.
The news about the deal had an immediate impact on the outfits dedicated to putting store fronts selling soft drinks, batteries, and snacks on every corner. Oh, I almost forgot to mention that these Walgreen-type outfits are distribution points for medications prescribed by doctors. Amazon seems to be serious about disrupting how many people pick up their medicines, fill out forms for insurance, and provide endlessly repetitive details like their date of birth, insurance data, and home address.
Amazon assumes that it can make this traditional doctor-pharmacy-consumer business more efficient and make a buck along the way.
Several thoughts crossed my mind as I read the NYT story. Not surprisingly, these have not be mentioned in the NYT story or the other coverage I have scanned.
Let me share a handful of questions which struck me after reading the NYT write up.
- Will Amazon be able to determine what individuals are acquired specific medicines and place these data on a timeline?
- Will Amazon be able to determine what medications are flowing to specific geographic regions; for example, specific zip codes within the Commonwealth of Kentucky or any other geocoded area?
- Will Amazon be able to pinpoint the physicians, dentists, etc. who are prescribing specific medicines and array those data on a timeline or a compound output like those available from Palantir Gotham or IBM Analysts Notebook?
- Can Amazon cross correlate these medicine related data with other specific Amazon customer behavior?
- Can Amazon provide insights about possible improper script issuance, medical fraud, or other similar activity?
I assume that Amazon may not have these questions. Amazon sells books.
I want to raise a final question:
What if Amazon can process these drug related data in a way that reveals patterns, identifies abusers, or provides data to flag medical fraud?
- If the answer is no, that’s okay with most Amazon customers.
- If the answer is yes, that suggests a number of other questions.
Amazon is an interesting company indeed.
Stephen E Arnold, June 29, 2018
What Has Happened to Enterprise Search?
June 28, 2018
I read “Enterprise Search in 2018: What a Long Strange Trip It’s Been.” I found the information presented interesting. The thesis is that enterprise search has been on a journey almost like a “Wizard of Oz” experience.
The idea of consolidation, from my point of view, boils down to executives who want to cash in, get out, and move on. The reasons are not far to seek: Over promising and under delivering, financial manipulations, and positioning a nuts and bolts utility as something that solves information problems.
Some, maybe many, licensees of proprietary enterprise search systems may have viewed their investment as an opportunity that delivered an unexpected but inevitable outcome. Where is that lush scenery? Where’s the beach?
The reality is that enterprise search vendors were aced by Shay Banon. His Act II of Compass: A Finding Story was Elasticsearch and the company Elastic. Why not use free and open source software. At least the code gets some bugs fixed unlike old school proprietary enterprise search systems. Bug fixes? Yep, good luck with your Fast Search & Retrieval ESP platform idiosyncrasies.
The landscape today is a bit like the volcanic transformation of Hawaii’s Vacationland. Real estate agents will be explaining that the lava flows have created new beach views, promising that eruptions are a low probability event.
The write up does not highlight one simple fact: Enterprise search has given way to “roll up” services or what I call “meta-plays.” The idea is that search is tucked inside systems like Diffeo, Palantir Gotham, and other “intelligence” platforms.
Why aren’t these enterprise grade solutions sold as “enterprise search” or “enterprise business intelligence and discovery solutions”?
The answer is that the information retrieval nest has been marginalized by the actions of vendors stretching back to the Smart system and to the present with “proprietary” solutions which actually include open source technology. These systems are anchored in the past.
Consider Diffeo?
Why offer enterprise search when one can provide a solution that delivers information in context, provides collaboration tools, and displays information in different ways with a single mouse click?
Amazon: A Corporate Monster?
June 25, 2018
I am not sure about companies becoming republics or democracies. When I went to work at Halliburton Nuclear, I knew the business and understood that I would get paid to do what my boss told me to do.
I am oriented the same way today in 2018 as I was in 1973. Call me old fashioned.
I read “Amazon Staffers Protest Giant’s ‘Support of the Surveillance State‘”. Okay, employees do not want Amazon to work on certain government projects. Why not quit?
I assume that’s not an option.
I did notice some interesting word choices in the write up; for example:
- “cops and spies”
- “moral objections”
- “separating children”
- “asylum seeking parents”
- “chain link cages”
- “shudder Amazonians”
- “refuse to build the platform”
- “ethically concerned Amazonians”
- “demand a choice”
- “data slurping outfit Palantir”
- “linked to the Cambridge Aanlytica data harvesting saga?
- “immoral US policy”
- “inhumane treatment:
- “enable ICE and DHS”
- “Privacy International”
- “disgraced US agency”
- “condemn the policy”
Quite a series of phrases.
Yep, real “news”. Perfect for a Facebook, Google Plus, or Twitter post or two. Wait, wait. That’s how exploiting algorithmic functions gain their momentum. Words work wonders.
Stephen E Arnold, June 25, 2018
-
- Subscribe to Beyond Search
Feature archive
News archive
-
