Real Time Remains Complicated and Costly
July 14, 2016
I love it when search and content processing vendors yammer about their “real time systems.” Years ago I did a report for a client in Europe about the costs of real time. Summarizing the six month research effort is easy: Real time is tough in computer systems. Latency exists.
To get a sense of how tough it is to accelerate certain online actions, navigate to “Legendary Hedge Fund Wants to Use Atomic Clocks to Beat High-Speed Traders.” Despite the wonky “legendary,” I noted this comment:
Patent application no. 14/451,356 [to find this puppy use US2016/0035027] has one goal: to outrun the speed demons of Wall Street. The 16-page document was quietly published by the U.S. Patent and Trademark Office in February. Replete with schematic drawings, the filing describes a novel way for “executing synchronized trades in multiple exchanges.” The invention consists of not only sophisticated algorithms and a host of computer servers, but atomic clocks — precisely calibrated to vibrations of irradiated cesium atoms — to sync orders to within a few billionths of a second.
I also highlighted:
Its invention, developed by the firm’s co-chief executive officers, Robert Mercer and Peter Brown, first sends an order to a central server, which breaks it up into multiple smaller orders. Those are then routed to venues that offer the best prices and most liquidity, much the same as brokers do now. But before that happens, the smaller orders are sent to servers located as close to the exchanges as possible, along with instructions on the precise times they should be executed. The co-located servers sync their transactions so HFT firms won’t have enough time to identify an order on one exchange and then race to another to trade against it. A crucial part of the system is the optical, atomic or GPS clocks that will be used synchronize those orders. Renaissance says in its application that GPS clocks are accurate to within nanoseconds and any time differences between them are “too small to be perceived” by HFT firms.
How much is an atomic clock centric trading system? A well heeled Wall Street firm can afford these systems. The reason is that the infrastructure to pull off this near real time approach is out of reach for many outfits.
How close to real time is a search and content processing system? You can believe the index is up to date, but I would suggest that you are looking at last week’s leftover barbeque chicken in a Tupperware box. Saying “real time” is less difficult and expensive than shaving milliseconds off an online action. Enjoy that chicken?
Stephen E Arnold, July 14, 2016
Thomson Reuters: Selling at Peak Value?
July 12, 2016
I read “Thomson Reuters Announces Definitive Agreement to Sell its Intellectual Property & Science Business to Onex and Baring Asia for $3.55 billion.” Thomson Reuters has been working hard to pump up revenue and generate a juicy profit for its stakeholders. Like IBM, it seems that the best way to get a large, established company in gear is to sell assets. According to the write up, Thomson Reuters’ management thinks:
“With the completion of this divestiture, Thomson Reuters will be even more focused on operating at the intersection of global commerce and regulation.”
What’s next for Thomson Reuters? More video? More Palantir repackaging? Higher fees for its professional information services?
Thomson Reuters has tried many things in the last two decades. The result is suggested in this chart:
The top line has been drifting down. The profit margin (the all important red line) has been a roller coaster. The net income has been a result of management moves and cost controls.
The question is: Is this collection of patent and IP related properties at peak value?
My hunch is that Thomson Reuters found the deal palatable. What will the new owners do with the properties. Both are investment outfits. The trajectory of these “services” like Compumark will be interesting to follow.
For Thomson Reuters, the hurdle remains growth. Isn’t that a problem with which IBM is struggling? Running specialist businesses with those who are not experts in each niche has been a challenge for many firms. Now the new owners Onex Partners and Baring Private Equity Asia have an opportunity to display their management expertise.
Selling is easier than innovating. Managing a bundle of businesses may be even more difficult.
Stephen E Arnold, July 12, 2016
Bad Actor Investments in Research
July 8, 2016
Want to chop your overhead? One easy way is to eliminate research and development. Most US companies are thinking long and hard about buying start ups to get innovative opportunities. Bad actors like hackers and online thieves are taking a slightly different approach. Navigate to “Hackers Investing 40% of Crime Proceeds in New Criminal Technologies.” Now this 40 percent number may be fluff, but the idea is an interesting one. Instead of recycling old exploits which smart networking monitoring services are able to thwart, the bad actors are doing the Thomas Edison thing: Some innovative, some borrowing, and some rethinking old methods (hybridization).
The write up highlights the innovation angle, stating:
Nikolay Nikiforov, an official spokesperson at Russia’s Ministry of Communications, told SC that investment of crime proceeds in new attack methods is mainly due to a change of priorities by hackers seen in recent years whereby they are no longer solely interested in attacks on the private bank accounts of individuals, but mainly targeting the breach of correspondent accounts of banks.
I find the moving upstream angle an intriguing one. What happens when US companies chop off spending for research and development? Do these outfits become more vulnerable? One of my former clients commented that his firm would worry about hacking when something happened. Yep, that’s enlightened thinking.
Stephen E Arnold, July 8, 2016
Digital Censorship: The Dartmouth Solution
June 21, 2016
I love the Ivy League. Yale, Princeton, Dartmouth, et al. I read “New Tool to Take Down Terrorism Images Online Spurs Debate on What Constitutes Extremist Content.” The source is the Bezos owned newspaper, which may or may not make a difference when applying the filters.
Here’s the method I circled in Hawthorne A red:
It [censoring mechanism] works by creating a distinct digital signature or “hash” for each image, video or audio track. The idea is to create a database of hashed content that Internet firms can use in automated fashion to vet images uploaded to their platforms. If there’s a match, the company can determine whether it violates its terms of service and should be taken down.
The write up brings up the challenge of defining what should and should not be filtered. If you are interested in automated filtering, check out the write up and the Dartmouth wizard who has created what he thinks may be a “game changer.”
What could go wrong with an Ivy League created system? If one cannot find information online, it does not exist. Is non existence a net good?
Stephen E Arnold, June 21, 2016
Digital Currencies: More Excitement
June 21, 2016
An “attacker” explains the legal perception he has. You can read this argument at this link. I do not have a horse in this race. In my recent lecture at a security conference in Myrtle Beach, SC, I pointed out that digital currencies work reasonably well for what I call small scale transactions. Putting one’s life savings into a digital currency is a step some bad actors are reluctant to take. Traditional non digital money laundering and tax evasion methods will slowly yield to Fancy Dan types of “money.” But if you are adventurous, have a go.
Stephen E Arnold, June 21, 2106
Murdoch Wall Street Journal Factiva: Known Unknowns
June 10, 2016
That Donald Rumsfeld statement about known knowns, known unknowns, etc. Is back. The Wall Street Journal ran an ad for Factiva. You remember Factiva. It is the Dow Jones Information Service repositioned and renamed a number of times over the last 15 or 20 years.
If you are into for fee search, you will know about Factiva and its kissing cousins: LexisNexis (bring your legal client’s purchase order), CSA ProQuest Dialog (bring your library acquisition budget), and Ebsco (bring your credit card). For fee information services serve the professional searcher market. Most people — including Gen X and Millennials researchers — are happy with Google. Objective results every time.
The for-fee services are still around. Public library and university fund raising programs help pay for access. Some queries returning zero useful results can cost $100 or more. Hey, you didn’t know, right?
If you navigate to the June 2, 2016, Wall Street Journal, page A7 in my dead tree edition ran a full page ad for Factiva. The ad highlights a couple of pie charts. Here they are in a tough to read gray and blue motif. Users of commercial database services have really sharp eyes and don’t need high contrast text, right?
The first pie chart shows your life consumed with research. Notice how little time one has to eat lunch. Note what a tiny portion of one’s day is available for email, Facebook, talking with colleagues, making sales calls, printing, the youth soccer telephone tree.
Now look at the second chart.
Look at the many different tasks one can undertake in a single work day. One can, of course, “take lunch.” I eat lunch, but that’s because here in rural Kentucky, we “eat” a meal. We make decisions. Apparently in Factiva land one takes a meal and probably takes decisions.
Other tasks one can pursue when one has Factiva are:
- Collaborating across departments
- Advise colleagues
- Stay on top of the news (Hey, it is part of that real journalism outfit owned by Mr. Murdoch. No bugging telephones, please.)
- Create a company newsletter. (I assume this word is “blog”, a Snapchat, or a tweet, but I could be off base.)
- Build powerful infographics. (Hmmm. I thought art types created infographics based on the data generated by a business intelligence system.)
- Research. Yes via Factiva.
Now I know that I am really out of the flow. The diagram showing the different between Baby Boomers and Millennials created by ace research analyst Mary Meeker reminded me of the gulf between my demographic and the zippy millennials.
Slide 51 from the Meeker, State of the Internet report.
The main point for me is that I possess zero of the attributes of millennials. I don’t earn to spend. I am retired. I conserve to pay for the old age home which I believe millennials call “opportunities for bingo.”
But the best part of the Factiva ad is the copy. I know words. Those nifty pie charts were the cat’s pajamas, weren’t they?
Here’s the guts of the message:
Spend your day working, not searching. Factiva’s reputable sources, flexible search and powerful insights provide access to thousands of quality, licensed, news and information sources in 28 languages. Know unknowns. [Emphasis added]
If Ms. Meeker is correct in her research and the supporting information from Hillhouse Capital and dozens of what appear to be primary sources and many hours of online searching commercial and Web resources — messaging apps are where the future is. Oh, there are videos too, but the takeaway is that traditional methods of getting digital information are in the same spot newspapers were yesterday.
The ad warrants several questions:
- Why does it have to be so darned big? Maybe small ads in the Wall Street Journal are ignored?
- How many of the Wall Street Journal’s readers are information specialists trained in the use of commercial online services? Judging from the Special Library Association’s challenges, I would suggest that the ad would have made sense to the corporate information specialist working in 1986, not 2016.
- What’s with the wonky pie charts? When I worked at a commercial database company, I don’t recall meeting any online users who spent the bulk of every day online. There were reference interviews (remember them, millennials?), culling the outputs from dot matrix printers, and planning search strategies before going online and whacking away.
Mr. Rumsfeld’s statement about knowns and unknowns emerged from his brush with the murky world of government related information. If he were to use Factiva today, would he have modified this famous statement:
There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.
Perhaps Factiva, like IBM Watson, is easier to describe than turn an information search system into a lean, mean, money making machine? I would suggest that the answer for decades has been an unknown unknown.
Stephen E Arnold, June 10, 2016
eBay and Facebook: Different Spins in Online Sales
April 14, 2016
I noted two seemingly unrelated items about two different companies. Here are the two items:
- Russian Diplomat: ISIS Making $200 Million Selling Stolen Artifacts on eBay
- Weapons for Sale on Facebook in Libya
In our work on the “Dark Web Notebook,” we have examined a number of sites which purport to offer contraband or prohibited products. These sites have been accessible using special software.
What is interesting is that the difference between the Dark Web and the “regular” Web seem to be blurring.
If these two stories are accurate, questions about governance by the owners of the Web sites may be raised. Since we began working on this new study of online content, we have noted that the boundary separating the Web which billions use from the Web tailored to a smaller set of online users is growing more difficult to discern.
In itself, the boundary’s change is interesting.
Stephen E Arnold, April 14, 2016
US Control of Internet Over
March 20, 2016
Short honk: I read “Quietly, Symbolically, US Control of the Internet Was Just Ended.” The write up explains that at a meeting in Morocco, people who run the “Internet’s naming and numbering system” have a plan
to end direct US government oversight control of administering the internet and commit permanently to a slightly mysterious model of global “multi-stakeholderism”.
What’s multi stakeholderism? I noted the reference to Snowden but multi stakeholderism?
Stephen E Arnold, March 20, 2016
Anonymous Hacks Turkish Cops
February 17, 2016
No Dark Web needed.
Anonymous has struck again, this time hacking the Turkish General Directorate of Security (EGM) in its crusade against corruption. The International Business Times reports, “Anonymous: Hacker Unleashes 17.8 GB Trove of Data from a Turkish National Police Server.” It is believed that the hacker responsible is ROR[RG], who was also deemed responsible for last year’s Adult Friend Finder breach. The MySQL-friendly files are now available for download at TheCthulhu website, which seems to be making a habit of posting hacked police data.
Why has Anonymous targeted Turkey? Reporter Jason Murdock writes:
“Anonymous has an established history with carrying out cyberattacks against Turkey. In 2015 the group, which is made up of a loose collection of hackers and hacktivists from across the globe, officially ‘declared war’ on the country. In a video statement, the collective accused Turkish President Recep Tayyip Erdo?an’s government of supporting the Islamic State (Isis), also known as Daesh. Turkey is supporting Daesh by buying oil from them, and hospitalising their fighters,’ said a masked spokesperson at the time. ‘We won’t accept that Erdogan, the leader of Turkey, will help Isis any longer. If you don’t stop supporting Isis, we will continue attacking your internet […] stop this insanity now Turkey. Your fate is in your own hands.’”
We wonder how Turkey will respond to this breach, and what nuggets of troublesome information will be revealed. We are also curious to see what Anonymous does next; stay tuned.
Cynthia Murrell, February 16, 2016
Stealing Data on the Dark Web Just Became Easier
February 1, 2016
“Underground Black Market: Thriving Trade in Stolen Data, Malware, and Attack Services” assumes that the reader knows the basics of the Dark Web. Let’s stake a step back.
Before we talk about stealing data on the Dark Web we must first define what we mean by the Dark Web. Most internet uses never go beyond the surface web, that part of the Web that consists of static Web sites such as Google, Facebook, and YouTube. What makes the Dark Web so interesting is that is it not entirely dark.
In fact, many Dark Web sites and their content are visible to the public. What is not visible is the server addresses which block most people from seeing who is running the sites.
In the article, Candid Wueest talks about a new paradigm for stealing and moving stolen data on the Dark Web. I noted that crimeware-as-a-service lets:
Attackers can easily rent the entire infrastructure needed to run a botnet or any other online scams. This makes cybercrime easily accessible for budding criminals who do not have the technical skills to run an attack campaign on their own. A drive-by download web toolkit, which includes updates and 24/7 support, can be rented for between $100 and $700 per week.
That means that it is becoming increasing easier for criminals to find, access, and sell data. Now you know. Now, anyone, including your local bad actor or your 11 year old, can access and steal data.
Here’s a troubling factoid from “The Tangled World of Stolen Data,” which we assume is spot on: It takes about 205 days for a company detect a data breach, more than enough time for a cybercriminal to sell the data and get it distributed on the Dark Web.
So what can law enforcement agencies do? New advances in Dark Web access, such as I2P, are making it more difficult for these agencies to identify and react to data crimes. What this means is that the law security companies and law enforcement agencies will need to be creative. The FBI ran an offensive image site to get a grip on alleged wrong doers.
Perhaps the Dark Web is not as dark as many assume.
Martin A. Matisoff, MSc, February 1, 2016
-
- Subscribe to Beyond Search
Feature archive
News archive
-
