The Flaws in Smart Software Methods
March 15, 2018
I read “Machine Learning Models Keep Getting Spoofed by Adversarial Attacks and It’s Not Clear If This Can Ever Be Fixed.” About four years ago I gave a series of lectures about the most commonly used mathematical procedures used in smart software. The lecture included equations, which I learned, are not high on the list of popular ways law enforcement and intelligence professionals favorite types of information.
Despite the inclusion of this lecture in some of my conference talks, only since the allegations, assertions, and counter assertions about interference via social media has the topic of flawed methods become popular.
The write up “Machine Learning Models…” is okay. The write up covers the basics, but specific information about why clustering can be disrupted or why anomaly detection numerical recipes can go off the rails is not included.
My point is that models can be enhanced and improved. However, in order to make even incremental progress, the companies, universities, and individuals involved in cooking up warmed over mathematical procedures have to take the initiative; for example:
- Question the use of textbook methods. Does Google’s struggle to identify faces in images reflect a dependence on Dr. Norvig’s recipe book?
- Become more demanding when threshold settings are implemented by an intern or an engineer who thinks the defaults are just dandy.
- Examine outputs in the context of a user who has subject matter expertise in the content and can identify wonky outputs
- Encourage developers to move beyond copying and pasting routines from college courses or methods invoked from a library someone said was pretty good.
- Evaluate the work flow sequence for its impact on system outputs
- Verify that “more data” works around flaws in the data by magic.
Until these types of shifts take place, smart software — whether for machine learning or making sense of real time flows of data — will remain less than perfect for many use cases.
Stephen E Arnold, March 15, 2018
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
