How-To Overview of Building a Data Platform to Handle Real-Time Datasets
March 11, 2016
The article on Insight Data Engineering titled Building a Streaming Search Platform offers a glimpse into the Fellows Program wherein grad students and software engineers alike build data platforms and learn cutting-edge open source technologies. The article delves into the components of the platform, which enables close to real-time search of a streaming text data source, with Twitter as an example. It also explores the usefulness of such a platform,
On average, Twitter users worldwide generate about 6,000 tweets per second. Obviously, there is much interest in extracting real-time signal from this rich but noisy stream of data. More generally, there are many open and interesting problems in using high-velocity streaming text sources to track real-time events. … Such a platform can have many applications far beyond monitoring Twitter…All code for the platform I describe here can be found on my github repository Straw.”
Ryan Walker, a Casetext Data Engineer, describes how these products might deliver major results in the hands of a skilled developer. He uses the example of a speech to text monitor being able to transcribe radio or TV feeds and send the transcriptions to the platform. The platform would then seek key phrases and even be set up to respond with real-time event management. There are many industries that will find this capability very intriguing due to their dependence on real-time information processing, including finance and marketing.
Chelsea Kerwin, March 11, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Organized Cybercrime Continues to Evolves
March 10, 2016
In any kind of organized crime, operations take place on multiple levels and cybercrime is no different. A recent article from Security Intelligence, Dark Web Suppliers and Organized Cybercrime Gigs, describes the hierarchy and how the visibility of top-level Cybercrime-as-a-Service (CaaS) has evolved with heightened scrutiny from law enforcement. As recently as a decade ago, expert CaaS vendors were visible on forums and underground boards; however, now they only show up to forums and community sites typically closed to newcomers and their role encompasses more expertise and less information sharing and accomplice-gathering. The article describes their niche,
“Some of the most popular CaaS commodities in the exclusive parts of the Dark Web are the services of expert webinjection writers who supply their skills to banking Trojan operators.
Webinjections are code snippets that financial malware can force into otherwise legitimate Web pages by hooking the Internet browser. Once a browser has been compromised by the malware, attackers can use these injections to modify what infected users see on their bank’s pages or insert additional data input fields into legitimate login pages in order to steal information or mislead unsuspecting users.”
The cybercrime arena shows one set of organized crime professionals, preying on individuals and organizations while simultaneously being sought out by organized cyber security professionals and law enforcement. It will be most interesting to see how collisions and interactions between these two groups will play out — and how that shapes the organization of their rings.
Megan Feil, March 10, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Germany Launches Malware to Spy on Suspicious Citizens
March 10, 2016
The article titled German Government to Use Trojan Spyware to Monitor Citizens on DW explains the recent steps taken in Germany to utilize Trojans, or software programs, created to sneak into someone else’s computer. Typically they are used by hackers to gain access to someone’s data and steal valuable information. The article states,
“The approval will help officials get access to the suspect’s personal computer, laptop and smartphone. Once the spyware installs itself on the suspect’s device, it can skim data on the computer’s hard drive and monitor ongoing chats and conversations. Members of the Green party protested the launching of the Trojan, with the party’s deputy head Konstantin von Notz saying, “We do understand the needs of security officials, but still, in a country under the rule of law, the means don’t justify the end.”
Exactly whom the German government wants to monitor is not discussed in the article, but obviously there is growing animosity towards not only the Syrian refugees but also all people of Middle Eastern descent. Some of this hostility is based in facts and targeted, but the growing prejudice towards innocent people who share nothing but history with terrorists is obviously cause for concern in Germany, Europe, and the United States as well. One can only imagine how President Trump might cavalierly employ malware to spy on an entire population that he has already stated his distrust of in the most general terms.
Chelsea Kerwin, March 10, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
A Place to Express Yourself on the Dark Web
March 7, 2016
For evidence that the dark web is not all about drugs and cybercrime, check out this article at Motherboard: “The Dark Web Now Has a Literary Journal.” As it turns out, anonymity is also good for people who wish to freely explore their creativity and private thoughts.
The new journal, the Torist, was just launched by a professor at the University of Utah, Robert W. Ghel, and a person known simply as GMH. Inspired by the free discussions on their dark-web-based social network, Galaxy, they have seized their chance to create something unexpected. The journal’s preface asks:
“If a magazine publishes itself via a Tor hidden service, what does the creative output look like? How might it contrast itself with its clearweb counterparts? Who indeed will gravitate towards a dark web literary magazine?”
So, why is one of the Torist’s creators anonymous while the other is putting himself out there? Writer Joseph Cox tells us:
Gehl, after being pitched the idea of The Torist by GMH, decided to strip away his pseudonym, and work on the project under his own name. “I thought about that for a while,” Gehl said. “I thought that because GMH is anonymous/pseudonymous, and he’s running the servers, I could be a sort of ‘clear’ liason.”
So while Gehl used his name, and added legitimacy to the project in that way, GMH could continue to work with the freedom the anonymity awards. “I guess it’s easier to explore ideas and not worry as much how it turns out,” said GMH, who described himself as someone with a past studying the humanities, and playing with technology in his spare time.
Gehl and GMH say part of their reasoning behind the journal is to show people that anonymity and encryption can be forces for good. Privacy furthers discussion of controversial, personal, and difficult topics and, according to GMH, should be the default setting for all communications, especially online.
Submissions are currently being accepted, so go ahead and submit that poem or essay if you have something to get off your chest, anonymously. If you dare to venture into the dark web, that is.
Cynthia Murrell, March 7, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
If You See Something, Say Something Adopts New Cybersecurity Meaning
March 4, 2016
A post-9/11 campaign for increasing security awareness will inform a similar public service announcement campaign to bring cybersecurity top of mind. See something suspicious online? Homeland Security wants to know about it published by NextGov reports on this 2016 Department of Homeland Security initiative. The decision to launch this campaign comes from an IDC recommendation; the US lacks a culture of cybersecurity concern, unlike Israel, according to the article. While $1 million is allotted for this campaign, the article describes bigger future plans,
“Last week, the Obama administration rolled out a new Cybersecurity National Action Plan, which establishes a new public commission on cybersecurity and proposes billions in new funding to upgrade hard-to-secure legacy IT systems in use at federal agencies, among several other steps.”
This year’s cybersecurity public and private sector awareness campaign was modeled after the “If You See Something, Say Something” campaign rolled out after September 11. However, this is not Homeland Security’s first attempt at educating the public about cybersecurity. The department has sponsored October as National Cybersecurity Awareness Month since 2004. As the article mentions, previous educational efforts have not appeared to influence culture. It would be interesting to know what metrics they are using to make that claim.
Megan Feil, March 4, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Artificial Intelligence Competition Reveals Need for More Learning
March 3, 2016
The capabilities of robots are growing but, on the whole, have not surpassed a middle school education quite yet. The article Why AI can still hardly pass an eighth grade science test from Motherboard shares insights into the current state of artificial intelligence as revealed in a recent artificial intelligence competition. Chaim Linhart, a researcher from an Israel startup, TaKaDu, received the first place prize of $50,000. However, the winner only scored a 59.3 percent on this series of tasks tougher than the conventionally used Turing Test. The article describes how the winners utilized machine learning models,
“Tafjord explained that all three top teams relied on search-style machine learning models: they essentially found ways to search massive test corpora for the answers. Popular text sources included dumps of Wikipedia, open-source textbooks, and online flashcards intended for studying purposes. These models have anywhere between 50 to 1,000 different “features” to help solve the problem—a simple feature could look at something like how often a question and answer appear together in the text corpus, or how close words from the question and answer appear.”
The second and third place winners scored just around one percent behind Linhart’s robot. This may suggest a competitive market when the time comes. Or, perhaps, as the article suggests, nothing very groundbreaking has been developed quite yet. Will search-based machine learning models continue to be expanded and built upon or will another paradigm be necessary for AI to get grade A?
Megan Feil, March 3, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Delve Is No Jarvis
March 3, 2016
A podcast at SearchContentManagement, “Is Microsoft Delve Iron Man’s Edwin Jarvis? No Way,” examines the ways Delve has yet to live up to its hype. Microsoft extolled the product when it was released as part of the Office 365 suite last year. As any developer can tell you, though, it is far easier to market than deliver polished software. Editor Lauren Horwitz explains:
“While it was designed to be a business intelligence (BI), enterprise search and collaboration tool wrapped into one, it has yet to make good on that vision. Delve was intended to be able to search users’ documents, email messages, meetings and more, then serve up relevant content and messages to them based on their content and activities. At one level, Delve has failed because it hasn’t been as comprehensive a search tool as it was billed. At another level, users have significant concerns about their privacy, given the scope of documents and activities Delve is designed to scour. As BI and SharePoint expert Scott Robinson notes in this podcast, Delve was intended to be much like Edwin Jarvis, butler and human search tool for Iron Man’s Tony Stark. But Delve ain’t no Jarvis, Robinson said.”
So, Delve was intended to learn enough about a user to offer them just what they need when they need it, but the tool did not tap deeply enough into the user’s files to effectively anticipate their needs. On top of that, it’s process is so opaque that most users don’t appreciate what it is doing, Robinson indicated. For more on Delve’s underwhelming debut, check out the ten-minute podcast.
Cynthia Murrell, March 3, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Drone2Map: Smart Software
March 2, 2016
If you are interested in mapping and geospatial analyses, you will want to read “ESRI Introduces Drone2Map to Process Aerial Images.” The write up reports:
Drone2Map incorporates Pix4D’s powerful image-processing engine to analyze images taken from drones and convert them into a variety of 2-D and 3-D maps.
What’s interesting to me is that the software is available for public download. You will need to know about ArcGIS and some other tools.
You can find the software at this link. You will have to jump through a couple of hoops. Don’t forget to register your drone.
Stephen E Arnold, March 2, 2016
Stolen Online Account Info Now More Valuable than Stolen Credit Card Details
March 2, 2016
You should be aware that criminals are now less interested in your credit cards and other “personally identifiable information” and more keen on exploiting your online accounts. As security firm Tripwire informs us in their State of Security blog, “Stolen Uber, PayPal Accounts More Coveted than Credit Cards on the Dark Web.” Writer Maritza Santillan explains:
“The price of these stolen identifiers on the underground marketplace, or ‘the Dark Web,’ shows the value of credit cards has declined in the last year, according to security firm Trend Micro. Last week, stolen Uber account information could be found on underground marketplaces for an average of $3.78 per account, while personally identifiable information, such as Social Security Numbers or dates of birth, ranged from $1 to $3.30 on average – down from $4 per record in 2014, reported CNBC. Furthermore, PayPal accounts – with a guaranteed balance of $500 –were found to have an average selling price of $6.43. Facebook logins sold for an average of $3.02, while Netflix credentials sold for about 76 cents. By contrast, U.S.-issued credit card information, which is sold in bundles, was listed for no more than 22 cents each, said CNBC.”
The article goes on to describe a few ways criminals can leverage these accounts, like booking Uber “ghost rides,” or assembling personal details for a very thorough identity theft. Pros say the trend means service providers to pay closer attention to usage patterns, and to beef up their authentication processes. Specifically, says Forrester’s Andras Cser, it is time to move beyond passwords; instead, he proposes, companies should look for changes in biometric data, like phone position and finger pressure, which would be communicated back to them by our mobile devices. So we’re about to be even more closely monitored by the companies we give our money to. All for our own good, of course.
Cynthia Murrell, March 2, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
No Search Just Browse Images on FindA.Photo
March 2, 2016
The search engine FindA.Photo proves itself to be a useful resource for browsing images based on any number of markers. The site offers a general search by terms, or the option of browsing images by color, collection (for example, “wild animals,” or “reflections”) or source. The developer of the site, David Barker, described his goals for the services on Product Hunt,
“I wanted to make a search for all of the CC0 image sites that are available. I know there are already a few search sites out there, but I specifically wanted to create one that was: simple and fast (and I’m working on making it faster), powerful (you can add options to your search for things like predominant colors and image size with just text), and something that could have contributions from anyone (via GitHub pull requests).”
My first click on a swatch of royal blue delivered 651 images of oceans, skies, panoramas of oceans and skies, jellyfish ballooning underwater, seagulls soaring etc. That may be my own fault for choosing such a clichéd color, but you get the idea. I had better (more various) results through the collections search, which includes “action,” “long-exposure,” “technology,” “light rays,” and “landmarks,” the last of which I immediately clicked for a collage of photos of the Eiffel Tower, Louvre, Big Ben, and the Great Wall of China.
Chelsea Kerwin, March 2, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
