Prepare To Update Your Cassandra
June 2, 2015
It is time for an update to Apache’s headlining, open source, enterprise search software! The San Diego Times let us know that “DataStax Enterprise 4.7 Released” and it has a slew of updates set to make open source search enthusiasts drool. DataStax is a company that built itself around the open source Apache Cassandra software. The company specializes in enterprise applications for search and analytics.
The newest release of DataStax Enterprise 4.7 includes several updates to improve a user’s enterprise experience:
“…includes a production-certified version of Cassandra 2.1, and it adds enhanced enterprise search, analytics, security, in-memory, and database monitoring capabilities. These include a new certified version of Apache Solr and Live Indexing, a new DSE feature that makes data immediately available for search by leveraging Cassandra’s native ability to run across multiple data centers.”
The update also includes DataStax’s OpCenter 5.2 for enhanced security and encryption. It can be used to store encryption keys on servers and to manage admin security.
The enhanced search capabilities are the real bragging points: fault-tolerant search operations-used to customize failed search responses, intelligent search query routing-queries are routed to the fastest machines in a cluster for the quickest response times, and extended search analytics-using Solr search syntax and Apache Spark research and analytics tasks can run simultaneously.
DataStax Enterprise 4.7 improves enterprise search applications. It will probably pull in users trying to improve their big data plans. Has DataStax considered how its enterprise platform could be used for the cloud or on mobile computing?
Whitney Grace, June 2, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Russian High Tech Propaganda
June 1, 2015
The Soviet Union was known for its propaganda, and Russia under Vladimir Putin seems to have brought the art into the digital age. The Guardian gives us the inside scoop in, “Salutin’ Putin: Inside a Russian Troll House.” Journalists spoke to two writers who were formerly among the hundreds working at the nondescript headquarters of Russia’s “troll army” in St Petersburg. There, writers are tasked with lauding Putin and lambasting the evils of the West in posts and comment sections on a wide variety of websites. Though the organization cannot be directly tied to the Kremlin, it’s reported the entity does not pay any taxes and does not register its employees. It does, however, seem to have grown heartily in the two years since Russia went (back) into the Ukraine.
It is said that working conditions at the “troll house” involve 12-hour shifts, a dreary environment, strict rules, and low pay, though that sounds no different from conditions in many jobs around the world. Workers describe writing a certain number of “ordinary posts” about things like music, travel, or dating advice; writers are responsible for coming up with those topics themselves. Interspersed with such bland content, however, they write pieces asserting political perspectives assigned to them each morning. Editors check carefully to make sure the stories are on point.
I’d recommend reading through the whole article, but this is the section that struck me most:
“‘I would go home at the end of the day and see all the same news items on the television news. It was obvious that the decisions were coming from somewhere,’ said Marat. Many people have accused Russian television of ramping up propaganda over the past 18 months in its coverage of Ukraine, so much so that the EU even put Dmitry Kiselev, an opinionated television host and director of a major news agency, on its sanctions list.
“After two months of working in the troll agency, Marat began to feel he was losing his sanity, and decided he had to leave. From the snatched conversations over coffee, he noted that the office was split roughly 50/50 between people who genuinely believed in what they were doing, and those who thought it was stupid but wanted the money. Occasionally, he would notice people changing on the job.
“‘Of course, if every day you are feeding on hate, it eats away at your soul. You start really believing in it. You have to be strong to stay clean when you spend your whole day submerged in dirt,’ he said.”
Sounds like some people I know who always have a certain U.S. news channel blasting away in the background. Writer Shaun Walker is unsure whether the site they found in St Petersburg is the only location for this activity, or whether there are other hubs throughout Russia. The effectiveness of such propaganda on Russian citizens, however, seems clear to Russian journalist Andrei Soshnikov (quoted in the article), especially with the older, less tech-savvy set. As disheartening as these revelations are, they should not be surprising.
Cynthia Murrell, June 1, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Sinequa and Systran Partner on Cyber Defense
May 20, 2015
Enterprise search firm Sinequa and translation tech outfit Systran are teaming up on security software. “Systran and Sinequa Combine in the Field of Cyber Defense,” announces ITRmanager.com. (The article is in French, but Google Translate is our friend.) The write-up explains:
“Sinequa and Systran have indeed decided to cooperate to develop a solution for detecting and processing of critical information in multiple languages ??and able to provide investigators with a panoramic view of a given subject. On one side Systran provides safe instant translation in over 45 languages, and the other Sinequa provides big data processing platform to analyze, categorize and retrieve relevant information in real time. The integration of the two solutions should thus facilitate the timely processing of structured and unstructured data from heterogeneous sources, internal and external (websites, audio transcripts, social media, etc.) and provide a clear and comprehensive view of a subject for investigators.”
Launched in 2002, Sinequa is a leader in the Enterprise Search field; the company boasts strong business analytics, but also emphasizes user-friendliness. Based in Paris, the firm maintains offices in Frankfurt, London, and New York City. Systran has a long history of providing innovative translation services to defense and security organizations around the world. The company’s headquarters are in Seoul, with other offices located in Daejeon, South Korea; Paris; and San Diego.
Cynthia Murrell, May 20, 2015
Stephen E Arnold, Publisher of CyberOSINT at www.xenky.com
Searching Bureaucracy
May 19, 2015
The rise of automatic document conversion could render vast amounts of data collected by government agencies useful. In their article, “Solving the Search Problem for Large-Scale Repositories,” GCN explains why this technology is a game-changer, and offers tips for a smooth conversion. Writer Mike Gross tells us:
“Traditional conversion methods require significant manual effort and are economically unfeasible, especially when agencies are often precluded from using offshore labor. Additionally, government conversion efforts can be restricted by document security and the number of people that require access. However, there have been recent advances in the technology that allow for fully automated, secure and scalable document conversion processes that make economically feasible what was considered impractical just a few years ago. In one particular case the cost of the automated process was less than one-tenth of the traditional process. Making content searchable, allowing for content to be reformatted and reorganized as needed, gives agencies tremendous opportunities to automate and improve processes, while at the same time improving workflow and providing previously unavailable metrics.”
The write-up describes several factors that could foil an attempt to implement such a system, and I suggest interested parties check out the whole article. Some examples include security and scalability, of course, as well as specialized format and delivery requirements, and non-textual elements. Gross also lists criteria to look for in a vendor; for instance, assess how well their products play with related software, like scanning and optical character recognition tools, and whether they will be able to keep up with the volumes of data at hand. If government agencies approach these automation advances with care and wisdom, instead of reflexively choosing the lowest bidder, our bureaucracies’ data systems may actually become efficient. (Hey, one can dream.)
Cynthia Murrell, May 19, 2015
Stephen E Arnold, Publisher of CyberOSINT at www.xenky.com
Open Source Conquers Proprietary Software, Really?
May 19, 2015
Open source is an attractive option for organizations wanting to design their own software as well as saving money of proprietary licenses. ZDNet reports that “It’s An Open Source World-78 Percent of Companies Run Open Source Software”, but the adopters do not manage their open source systems very well. Every year Black Duck Software, an open source software logistics and legal solutions provider, and North Bridge, a seed to growth venture capital firm, run the Future of Open Source Survey. Organizations love open source, but
“Lou Shipley, Black Duck’s CEO, said in a statement, ‘In the results this year, it has become more evident that companies need their management and governance of open source to catch up to their usage. This is critical to reducing potential security, legal, and operational risks while allowing companies to reap the full benefits OSS provides.’”
The widespread adoption is due to people thinking that open source software is easier to scale, has fewer security problems, and much faster to deploy. Organizations, however, do not have a plan to manage open source, an automated code approval process, or have an inventory of open source components. Even worse is that they are unaware of the security vulnerabilities.
It is great that open source is being recognized as a more viable enterprise solution, but nobody knows how to use it.
Whitney Grace, April 19, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Hacking a Newspaper: Distancing and Finger Pointing
May 15, 2015
I read “This Is How the Syrian Electronic Army Hacked the Washington Post.” Hacking into a company’s computer system is not something I condone. The target of the hacking is not too keen on the practice either I assume.
One of our Twitter accounts was compromised. We contacted Twitter. Even though we knew the CTO, it took a couple of days to sort out the problem. Apparently Miley Cyrus became a fan of Beyond Search and wanted to share her photo graphs via the blog’s newsfeed. One reader, an Exalead professional, was quite incensed that I was pumping out Miley snaps. I assume he found a better source of search and content processing news or left the field entirely due to the shock I imparted to him. I did not objectify the hacking incident. I don’t think I mentioned it until this moment. A script from somewhere in the datasphere got lucky.
In the aforementioned write up, I noted this passage:
Th3 Pr0, one of the members of the group, confirmed to Motherboard that they were indeed the group behind the attack, which appeared to last for around 30 minutes. Th3 Pr0 said that they were able to insert the alerts by hacking into Instart Logic, a content delivery network (CDN) used by the Washington Post. “We hacked InStart CDN service, and we were working on hacking the main site of Washington Post, but they took down the control panel,” Th3 Pr0 told Motherboard in an email. “We just wanted to deliver a message on several media sites like Washington Post, US News and others, but we didn’t have time :P.” The group often defaces media sites by hacking into other third parties, such as ad networks, that serve content on the sites.
The Washington Post, it seems, was not the problem. A content delivery network was the problem.
The article then reminded me:
This is the second time the hackers get to the Washington Post. The group briefly disrupted the site in 2013 with a phishing attack.
But the kicker for me is this statement:
This hack shows, once again, that a site is only as secure as its third-party resources,including ads, are.
Well, these problems are short lived. The problems are not the problems of the Washington Post. Bueno indeed. Perhaps Amazon’s Jeff Bezos will provide some security inputs to the Washington Post folks. Fool me once, shame on me. Fool me twice, well, blame the third party.
Works in Washington I assume.
Stephen E Arnold, May 15, 2015
SAP and Business Intelligence: Simple Stuff, Really Simple
May 14, 2015
I came across an interesting summary of SAP’s business intelligence approach. Navigate to “SAP BI Suite Roadmap Strategy Update from ASUG SapphireNow.” ASUG, in case you are not into the SAP world, means America’s SAP User Group. Doesn’t everyone know that acronym? I did not.
The article begins with a legal disclaimer, always a strange attractor to me. I find content on the Web which includes unreadable legal lingo sort of exciting.
It is almost as thrilling as some of the security methods which SAP employs across its systems and software. I learned from a former SAP advisor that SAP was, as I recall the comment, “Security has never been a priority at SAP.”
The other interesting thing about the article is that it appears to be composed of images captured either from a low resolution screen capture program or a digital camera without a massive megapixel capability.
I worked through the slides and comments as best as I could. I noted several points in addition to the aforementioned lacunae regarding security; to wit:
- SAP wants to simplify the analytics landscape. This is a noble goal, but my experience has been that SAP is a pretty complex beastie. That may be my own ignorance coloring what is just an intuitive, tightly integrated example of enterprise software.
- SAP likes dedicating servers or clusters of servers to tasks. There is a server for the in memory database. There is a server for what I think used to be Business Objects. There is the SAP desktop. There are edge servers in case your SAP installation is not for a single user. There is the SAP cloud which, I assume, is an all purpose solution to computational and storage bottlenecks. Lots of servers.
- Business Objects is the business intelligence engine. I am not confident in my assessment of complexity, but, as I recall, Business Objects can be a challenge.
My reaction to the presentation is that for the faithful who owe their job and their consulting revenue to SAP’s simplified business intelligence solutions and servers, joy suffuses their happy selves.
For me, I keep wondering about security. And whatever happened to TREX? What happened to Inxight’s Thingfinder and related server technologies?
How simple can an enterprise solution be? Obviously really simple. Did I mention security?
Stephen E Arnold, May 14, 2015
Elasticsearch Transparent about Failed Jepsen Tests
May 11, 2015
The article on Aphyr titled Call Me Maybe: Elasticsearch 1.5.0 demonstrates the ongoing tendency for Elasticsearch to lose data during network partitions. The author goes through several scenarios and found that users can lose documents if nodes crash, a primary pauses, a network partitions into two intersecting components or into two discrete components. The article explains,
“My recommendations for Elasticsearch users are unchanged: store your data in a database with better safety guarantees, and continuously upsert every document from that database into Elasticsearch. If your search engine is missing a few documents for a day, it’s not a big deal; they’ll be reinserted on the next run and appear in subsequent searches. Not using Elasticsearch as a system of record also insulates you from having to worry about ES downtime during elections.”
The article praises Elasticsearch for their internal approach to documenting the problems, and especially the page they opened in September going into detail on resiliency. The page clarifies the question among users as to what it meant that the ticket closed. The page states pretty clearly that ES failed their Jepsen tests. The article exhorts other vendors to follow a similar regimen of supplying such information to users.
Chelsea Kerwin, May 11, 2014
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Defense Contractor Makes Leap Investment Into Cybersecurity
April 30, 2015
The expression goes “you should look before you leap,” meaning you should make plans and wise choices before you barrel headfirst into what might be a brick wall. Some might say Raytheon could be heading that way with their recent investment, but The Wall Street Journal says they could be making a wise choice in the article, “Raytheon To Plow $1.7 Billion Into New Cyber Venture.”
Raytheon recently purchased Websense Inc., a cybersecurity company with over 21,000 clients. Websense will form the basis of a new cyber joint venture and it is projected to make $500 million in sales for 2015. Over the next few years, Raytheon predicts the revenue will surge:
“Raytheon, which is based in Waltham, Mass., predicted the joint venture would deliver high-single-digit revenue growth next year and mid-double-digit growth in 2017, and would be profitable from day one. Raytheon will have an 80% stake in the new cyber venture, with Vista Partners LLC holding 20%.”
While Raytheon is a respected name in the defense contracting field, their biggest clients have been with the US military and intelligence agencies. The article mentions how it might be difficult for Raytheon’s sales team and employees to switch to working with non-governmental clients. Raytheon, however, is positioned to use Websense’s experience with commercial clients and its own dealings within the security industry to be successful.
Raytheon definitely has looked before its leapt into this joint venture. Where Raytheon has shortcomings, Websense will be able to compensate and vice versa.
Whitney Grace, April 30, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
IBM Provides Simple How-To Guide for Cloudant
April 24, 2015
The article titled Integrate Data with Cloudant and CouchDB NoSQL Database Using IBM InfoSphere Information Server on IBM offers a breakdown of the steps necessary to load JSON documents and attachments to Cloudant. In order to follow the steps, the article notes that you will need Cloudant, CouchDB, and IBM InfoSphere DataStage. The article concludes,
“This article provided detailed steps for loading JSON documents and attachments to Cloudant. You learned about the job design to retrieve JSON documents and attachments from Cloudant. You can modify the sample jobs to perform the same integration operations on a CouchDB database. We also covered the main features of the new REST step in InfoSphere DataStage V11.3, including reusable connection, parameterized URLs, security configuration, and request and response configurations. The JSON parser step was used in examples to parse JSON documents.”
Detailed examples with helpful images guide you through each part of the process, and it is possible to modify the examples for CouchDB. Although it may seem like a statement of the obvious the many loyal IBM users out there, perhaps there are people who still need to be told. If you are interested in learning the federation of information with a logical and simple process, use IBM.
Chelsea Kerwin, April 24, 2014
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
