Change Is Difficult: Especially So for the Big Search Folks
September 25, 2018
There is a pretty good reminder that plumbing is an issue. Most users of smart search just assume that systems will continue to work. Hey, it is 2018. This Moore’s Law stuff, free services, and nifty new “old” phones are slam dunks.
Not quite.
I found “The Woes of Incremental Resource Drains in Big Systems” useful because it offers practical information. Most of the content snagged by my monitoring systems return what I consider marketing craziness.
The write up explains that an attempt to implement a necessary change can go off the rails. It’s like magic. One day Gmail or Amazon doesn’t work. Hand waving. Tweets. Then the problem is solved and forgotten.
The write up explains:
Let’s say you have a tier of 100,000 web servers. Every one of them opens one connection to your database. That connection is shared with all of the concurrent hits/code executing on those web servers. It just gets multiplexed down the pipe and off it goes. Then, one fine day, someone decides to write a change that makes the web servers open four connections to the database. They’ve invented this new “pooling” strategy, such that requests grab the least-busy connection instead of always sharing the single one. It’s supposed to help latency by n% (and get them a promotion, but let’s not get into that now).
Do the math. Dead system.
Worth remembering because as certain companies become like the timesharing systems of the past, excitement will be inevitable. Some can be hidden. Some will surface in unexpected ways.
Bing, Facebook, Google, and maybe Amazon may face this type of challenge more often than one believes.
Stephen E Arnold, September 25, 2018
Factualities for September 18, 2018
September 19, 2018
Believe ‘em or not:
- Zero. The change in the average age of the IBM workforce after reductions in workforce. Source: Poughkeepsie Journal
- $115. The cost of a holiday pine tree from Amazon. Source: APNews
- $100. The expected cost of Sony’s forthcoming PlayStation Classic game device. Source: TechSpot
- 63 million. The number of Apple OSX mobile phones sold. Source: Bloomberg
- 621 miles. The distance a Google Loon balloon can beam an Internet connection. Source: Fortune
- 40 percent. The percentage of economic experiments and studies which cannot be replicated. Source: Science Magazine
- 14 percent. How much larger the font Times Newer Roman is than Times New Roman. Source: The Verge
- 15. The number of employers who advertised on Facebook for job candidates of a specific sex. Source: ProPublica
- $9 million US. Amount of new capital injected into the vegan food delivery service Allplants. Source: TechCrunch
Stephen E Arnold, September 19, 2018
Technical Debt: A Bit of a Misunderstanding of the Iron Maiden Effect
September 18, 2018
Let’s go back in time. It is 1979 and Lockheed Martin has nosed into the commercial database business. The system was designed around IBM mainframe technology and due to costs and other factors, the Dialog Information Services outfit embraced Hitachi plug compatible mainframes. Now it is 1982, what’s the technical situation?
The answer is, “Nowhere.”
In fact, one can data the slow degradation and eventual marginalizing of the Lockheed Martin operation and the original commercial online business from the early 1980s. The challenges boiled down to:
An inability to perform tasks customers wanted because the technical architecture made the “changes” to deliver what the customer wanted too expensive, too complex, too time consuming, and too different from what mainframe architectures could deliver. And what did customers want? Reports. Yep, a report that showed who used what database. The trick mainframe architecture managers use to discourage a customer from getting a report was to charge an outrageous fee. In fact, a signal that a technical architecture cannot be bent to the will of the customer is a wild and crazy charge for what seems a simple request.
This is an iron maiden. It doesn’t look like much. Put an MBA, accountant, or lawyer running a high technology company inside and suddenly the technology makes its point or points as it were.
A lack of cash and managerial willingness to recreate a business on a more modern computing architecture. For the mid 1980s, the switch would have been to slightly less restrictive computers from non mainframe manufacturers like Digital Equipment Corporation. The money part is easy to understand. Investing in the future would require abandoning a good but slowly growing line of business to confront the risky world of the start up. That’s because a technical shift is just a start up when stripped of the fancy PowerPoints.
A failure to listen to those who explained that a change was indeed necessary. I worked for a woman who carried this message of change to luminaries in the commercial online timesharing world. Although tolerated, the managers smiled and went about their day to day mainframe approach to commercial online services. There is no fix for a person’s lack of desire and inability to listen and comprehend the message.
Okay, so that’s my view of the built in problem with most online services oriented co9mpanies. Against this set of personal ideas, I read “Too Big to Survive: There Is No Bailout for Technical Debt.” The write up makes a good point; specifically:
The only difference between technical debt and financial debt is that costs are more often known in advance when taking on financial debt. Both types of debt are a tool when used intelligently with purpose and a plan to manage it, and both can take a devastating toll when used recklessly or imposed through misdirection or miscommunication.
However, the idea that a manager can avoid the problem I described with the commercial online services business in the mid 1980s strikes me as falsely optimistic. The recommendation that a person should go through the MBA hand waving when the problem is identified within an organization is not particularly useful. The problem resides within the usually small group of executives who have the most to lose when a major shift is required to survive.
To sum up, in my experience, technology based companies are not trapped in an innovator’s dilemma? High technology companies are victims of the technology used to build the business. Many business school students learn about the problem of the buggy whip manufacturer. I have a different view of a high technology company’s getting kicked to the side of the road only to die an agonizing death while waiting for a self driving Tesla to stop and offer the near death outfit a ride to a crowd funding meet up.
The high technology company does not want to get in the Tesla. The outfit’s management prefers to wait for a better offer. Maybe the better offer arrives. Maybe not.
The result is the same. High technology companies become disconnected from what’s happening around them. Examples range from the tone deaf actions of Facebook to the oddly skewed failure of Microsoft in its mobile device business to Google’s social failures like leaking and protesting employees. Amazon is allegedly mounting PR campaigns to explain how well employees fare in a giant warehouse where robots seem to have more fun at work.
My point is that when one creates a business based on a technology and that technology becomes increasingly complicated, the technology will resist being changed. The humans are essentially along for the ride, modifying their world view about change when the implications of failing to make a change are sensed.
In short, we are not dealing with technological debt. We are dealing with an inherent characteristic of technology which seems liberating at first and then becomes a digital iron maiden. When the door begins to close, the pain begins.
Those moving to the side of a road don’t need a lift from a Tesla driver. An emergency vehicle is a better bet, but the likelihood of survival decreases with time. That’s not debt, that’s a far more grim outcome. When one is inside a technology, the door closes. Point made, right?
Stephen E Arnold, September 18, 2018
Machine Learning Frameworks: Why Not Just Use Amazon?
September 16, 2018
A colleague sent me a link to “The 10 Most Popular Machine Learning Frameworks Used by Data Scientists.” I found the write up interesting despite the author’s failure to define the word popular and the bound phrase data scientists. But few folks in an era of “real” journalism fool around with my quaint notions.
According to the write up, the data come from an outfit called Figure Eight. I don’t know the company, but I assume their professionals adhere to the basics of Statistics 101. You know the boring stuff like sample size, objectivity of the sample, sample selection, data validity, etc. Like information in our time of “real” news and “real” journalists, some of these annoying aspects of churning out data in which an old geezer like me can have some confidence. You know like the 70 percent accuracy of some US facial recognition systems. Close enough for horseshoes, I suppose.
Here’s the list. My comments about each “learning framework” appear in italics after each “learning framework’s” name:
- Pandas — an open source, BSD-licensed library
- Numpy — a package for scientific computing with Python
- Scikit-learn — another BSD licensed collection of tools for data mining and data analysis
- Matplotlib — a Python 2D plotting library for graphics
- TensorFlow — an open source machine learning framework
- Keras — a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano
- Seaborn — a Python data visualization library based on matplotlib
- Pytorch & Torch
- AWS Deep Learning AMI — infrastructure and tools to accelerate deep learning in the cloud. Not to be annoying but defining AMI as Amazon Machine Learning Interface might be useful to some
- Google Cloud ML Engine — neural-net-based ML service with a typically Googley line up of Googley services.
Stepping back, I noticed a handful of what I am sure are irrelevant points which are of little interest to a “real” journalists creating “real” news.
First, notice that the list is self referential with python love. Frameworks depend on other python loving frameworks. There’s nothing inherently bad about this self referential approach to shipping up a list, and it makes it a heck of a lot easier to create the list in the first place.
Second, the information about Amazon is slightly misleading. In my lecture in Washington, DC on September 7, I mentioned that Amazon’s approach to machine learning supports Apache MXNet and Gluon, TensorFlow, Microsoft Cognitive Toolkit, Caffe, Caffe2, Theano, Torch, PyTorch, Chainer, and Keras. I found this approach interesting, but of little interest to those creating a survey or developing an informed list about machine learning frameworks; for example, Amazon is executing a quite clever play. In bridge, I think the phrase “trump card” suggests what the Bezos momentum machine has cooked up. Notice the past tense because this Amazon stuff has been chugging along in at least one US government agency for about four, four and one half years.
Third, Google brings up dead last. What about IBM? What about Microsoft and its CNTK. Ah, another acronym, but I as a non real journalist will reveal that this acronym means Microsoft Cognitive Toolkit. More information is available in Microsoft’s wonderful prose at this link. By the way, the Amazon machine learning spinning momentum thing supports the CNTK. Imagine that? Right, I didn’t think so.
Net net: The machine learning framework list may benefit from a bit of refinement. On the other hand, just use Amazon and move down the road to a new type of smart software lock in. Want to know more? Write benkent2020 @ yahoo dot com and inquire about our for fee Amazon briefing about machine learning, real time data marketplaces, and a couple of other most off the radar activities. Have you seen Amazon’s facial recognition camera? It’s part of the Amazon machine learning imitative, and it has some interesting capabilities.
Stephen E Arnold, September 16, 2018
Factualities for September 12, 2018
September 12, 2018
Believe ‘em or not. The “facts”, that is.
- $7,800 per month and up for the cloud version of IBM Information Server. Source: Datamation
- 280,000. The number of routers infected with crypto jacking. Source: Next Web
- 250 million. The number of Pinterest users. Source: Engadget
- $131 trillion. The amount of money artificial intelligence will add to global production. Source: Axios
- 44 percent. The number of people who have deleted the Facebook app this year. Source: Reddit
- Less than one percent of start ups fail due to competition. Source: Techcrunch
- Five times an hour. How often Americans check their mobile phones. Source: San Francisco Chronicle
- Zero percent. The use of open source in 1993. Source: Silicon Angle
Stephen E Arnold, September 12, 2018
New Technology: A Problem Solver or Problem Generator?
August 23, 2018
Noodling about the k-epsilon model is probably not popular at most cocktail parties. In brief when flows occur, chaos usually turns up at the party.
Consider a large company anchored in technology from IBM, SAP, and some home brew applications. Toss in a mainframe, an AS/400 legacy system, and run-of-the-mill desktops.
Technological change is difficult, and when a switch is needed from Windows 3.11 to Windows 95, the shift may take years. The mainframe keeps on chugging along with CICS, MVS TSO, and green screens. The SAP system gets updated, but after a three year install process, who wants too make changes.
Today, the world of enterprise computing is different. Even the US government wants to move to the cloud. Virtualization is the big thing, not hardware down the hall behind a keycarded door.
When I read “Fragmenting Budgets and Rapid Pace of Change Creates Perfect Storm for IT Decision Makers.” The write up explains a situation which I thought most computer centric folks knew and understood.
The write up explains:
IT decision-makers are increasingly tasked with the difficult decision of choosing technology within business operations and finding the correct IT solutions for business needs. This extra link in the chain combined with the ever-accelerating pace of technological development is creating a perfect storm. In fact, a recent survey of IT decision-makers found that more than half are struggling to keep up with the pace of new technology. Most (84 per cent), acknowledge that they are not currently running the most optimum IT systems and significantly, 28 per cent admit that their organization has actually fallen behind the rate of technological change.
Nothing is as compelling as fear in an organization.
What’s happening is that the friction brakes of old school systems and methods are being replaced with the equivalent of dragging a sneaker on the pavement to slow down a bicycle. For some young at heart managers, the sneaker brake is great fun.
The downside is what I call a chaos problem. Semi stable flows become chaotic or, in more colloquial language, pretty darned crazy.
IT managers now find themselves in a technology environment less stable than those that existed a scant 10 years ago. The decision to embrace fast changing innovations can be a significant. Not only will the competitiveness of the organization be affected but the work environment may no longer match what must be accomplished to remain a viable entity.
Examples include the well publicized engineer revolts at Facebook and Amazon. The technical waffling from chip vendors when flaws are discovered. The presence of point of sale units at fast good chains and grocery stores which employees cannot operate.
The write up documents an accelerating opportunity for consultants. For those crushed with the fragments from technical chaos, the future may require rehab.
Stephen E Arnold, August 23, 2018
Technology and Government: A Management Challenge for the 21st Century
August 15, 2018
Throughout history, government funding has led to some of the greatest technological advances known to man. Thank NASA next time you strap on your Velcro shoes or sip some Tang. Recently, some voices in Silicon Valley spoke out to try and repair the rift among tech and government. We learned more from a recent Washington Post Op-Ed, “Silicon Valley Should Stop Ostracizing the Military.”
According to the story:
“The world is safer and more peaceful with strong U.S. leadership. That requires the U.S. government to maintain its advantage in critical technologies such as AI. But doing so will be difficult if Silicon Valley’s rising hostility toward working with Washington continues. In June, Google…announced that it would not renew a Pentagon contract for an AI program called Project Maven when it expires next year.”
The biggest concern is that Russia and China are rapidly advancing their AI weaponry and leaving behind the US. This, they argue, weakens the freedom-loving world, so it is time for these often diametrically opposed organizations to make up for the good of the planet.
With the Department of Defense moving toward a decision about the $10 billion cloud procurement, Beyond Search anticipates more employee-management tension at the high technology giants jockeying for US government contracts.
Should employees expect a company’s Board of Directors and senior management to go in the direction employees want?
MBAs and high school math club thinking may create administrative friction. Whom does a tech slow down benefit? Electric scooter riders?
Patrick Roland, August 15, 2018
Google Contributes to the History of Kubernetes
August 15, 2018
It is time for a history lesson; the Google Cloud Platform Blog proffers, “From Google to the World: The Kubernetes Origin Story.” Anyone curious about the origins of the open source management system may want to check it out. The post begins with a description of the 2013 meeting at which the Kubernetes co-founders pitched their idea to executive Urs Holzle, which only happened because one of those founders (and author of the post) Craig McLuckie found himself on a shuttle with the company’s then-VP of Cloud Eric Brewer. To conclude the post, McLuckie notes Kubernetes is now deployed in thousands of organizations and has benefitted from some 237 person-years’ worth of coding put in by some 830 contributors. In between we find a little Star Trek-related trivia; McLuckie writes:
“In keeping with the Borg theme, we named it Project Seven of Nine. (Side note: in an homage to the original name, this is also why the Kubernetes logo has seven sides.) We wanted to build something that incorporated everything we had learned about container management at Google through the design and deployment of Borg and its successor, Omega — all combined with an elegant, simple and easy-to-use UI. In three months, we had a prototype that was ready to share.
We also noted this statement:
“We always believed that open-sourcing Kubernetes was the right way to go, bringing many benefits to the project. For one, feedback loops were essentially instantaneous — if there was a problem or something didn’t work quite right, we knew about it immediately. But most importantly, we were able to work with lots of great engineers, many of whom really understood the needs of businesses who would benefit from deploying containers (have a look at the Kubernetes blog for perspectives from some of the early contributors).”
McLuckie includes links for potential users to explore the Kubernetes Engine and, perhaps, begin a two-month free trial. Finally, he suggests we navigate to his Kubernetes Origins podcast hosted by Software Engineering Daily for more information.
History is good.
Cynthia Murrell, August 15, 2018
Applique Logic: Alex Jones and Turbo Charging Magnetism
August 9, 2018
I am not sure I have read an Alex Jones’ essay or watched an Alex Jones’ video. In fact, he was one of the individuals of whom I was aware, but he was not on my knowledge radar. Now he is difficult to ignore.
Today’s New York Times corrected my knowledge gap. I noted in my dead tree edition today (August 9, 2018) these stories:
- Facebook’s Worst Demons Have Come Home to Roost, page B1
- Infowars App Is Trending As Platforms Ban Content, B6
- The Internet Trolls Have Won. Get Used to It, B7
I want to mention “Rules Won’t Save Twitter. Values Will” at this online location.
From my vantage point in rural Kentucky, each of the writes up contributes to the logic quilt for censoring the real Alex Jones.
Taken together, the information in the write ups provide a helpful example of what I call “appliqué logic.”
Applique means, according to Google which helpfully points to Wikipedia, another information source which may be questionable to some, is:
Appliqué is ornamental needlework in which pieces of fabric in different shapes and patterns are sewn or stuck onto a larger piece to form a picture or pattern. It it commonly used as decoration, especially on garments. The technique is accomplished either by hand or machine. Appliqué is commonly practiced with textiles, but the term may be applied to similar techniques used on different materials.
Applique logic is reasoning stuck on to something else. In this case, the “something else” are the online monopolies which control access to certain types of information.
The logic is that the monopolies are technology, which is assumed to be neutral. I won’t drag you through my Eagleton Award lecture from a quarter century ago to remind you that the assumption may not be correct.
The way to fix challenges like “Alex Jones” is to stick a solution on the monopoly. This is similar to customizing a vehicle like this one:
Notice how the school bus (a mundane vehicle) has been enhanced with what are appliqués. The result does not change the functioning of the school bus, but it now has some sizzle. I suppose the appliqué logician could write a paper and submit the essay to an open access publisher to explain the needed improvements the horns add.
With the oddly synchronized actions against the Alex Jones content, we have the equivalent of a group of automobile customizers finding ways to “enhance” their system.
The result is to convert what no one notices into something that would make a Silicon Valley PR person delighted to promote. I assume that a presentation at a zippy new conference would be easy for the appliqué team to book.
The apparent censorship of Alex Jones is now drawing a crowd. Here I am in Harrods Creek writing about a person to whom I previously directed zero attention. The New York Times coverage is doing a better job than I could with a single write up in a personal blog. In the land of “free speech” the Alex Jones affair may become an Amazon Prime or Netflix original program. Maybe a movie is in the works?
Back to appliqué logic. When it comes to digital content, sticking on a solution may not have the desired outcome. The sticker wants one thing. The stickee is motivated to solve the problem; for example, the earthquake watcher Dutch Sinse has jumped from YouTube to Twitch to avoid censorship. He offered an explanation about this action and referenced the Washington Post. I don’t follow Dutch Sinse so I don’t know what he is referencing, and I don’t care to be honest.
But the more interesting outcome of these Alex Jones related actions is that the appliqué logic has to embrace the “stickoids.” These are the people who now have a rallying point. My hunch is that whatever information Alex Jones provides, he is in a position to ride a pretty frisky pony at least for a a moment in Internet time.
Why won’t appliqué logic work when trying to address the challenges companies like Facebook, Google, et al face?
- Stick ons increase complexity. Complexity creates security issues which, until it is too late, remain unknown
- Alex Jones type actions rally the troops. I am not a troop, but here I am writing about this individual. Imagine the motivation for those who care about Mr. Jones’ messages
- Opportunities for misinformation, disinformation, and reformation multiply. In short, the filtering and other appliqué solutions will increase computational cost, legal costs, and administrative costs. Facebook and Google type companies are not keen on increased costs in my opinion.
- Alex Jones type actions attack legal eagles.
What’s the fix? There is a spectrum of options available. On one end, believe that the experts running the monopolies will do the right thing. Hope is useful, maybe even in this case. At the other end, the Putin approach may be needed. Censorship, fines, jail time, and more extreme measures if the online systems don’t snap a crisp salute.
Applique solutions are what’s available. I await the final creation. I assume there will be something more eye catching than green paint, white flame decoration, and (I don’t want to forget) the big green horns.
For Alex Jones, censorship may have turbocharged his messaging capability. What can one stick on him now? What will the stickoids do? Protest marches, Dark Web collections of his content, encrypted chat among fans?
I know one thing: Pundits and real journalists will come up with more appliqué fixes. Easy, fast, and cheap. Reasoning from the aisles of Hobby Lobby or Michael’s is better than other types of analytic thought.
Stephen E Arnold, August 9, 2018
Factualities for Wednesday, August 8, 2018
August 8, 2018
Beyond Search noted these factualities in the last week. Believe ‘em or not:
- TGI Fridays, The home of the loaded baked potato, allegedly doubled business and grew
“engagement” by 500 percent with… artificial intelligence. Source: Venture Beat - Machine learning is like medieval alchemy. Source: Guardian
- According to Internet Live Stats, Google conducts 40,000 searches per second. No data about relevance was provided. Source: CBS
- There will be 90% fewer attorneys in the next 5 to 10 years. No word on what happens to these proud professionals. Source: Egypt4U
- Google and Facebook together controlled about 61 per cent of all online advertising revenues in 2017 and cornered a 25 per cent share of all media advertising revenues…Google earns around 85 per cent of its revenues through ads, for Facebook that figure is close to 98 per cent. Source: Rediff
- 50 percent: The number of people who purchased electronic gadgets for their pets. Source: Shinyshiny
Stephen E Arnold, August 8, 2018
-
- Subscribe to Beyond Search
Feature archive
News archive
-
