Open Source is a Mindset
June 5, 2013
Open source is almost single-handedly powering the Big Data movement. And to that end, open source is getting a lot of credit and a lot of coverage. Many open source solutions are popping up in the trends’ wake. Among them is Appcelerator. The Appcelerator CEO, Jeff Haynie, weighs in on the power of open source in the current market in the Silicon Angle article, “Open Source is a Mindset, says Appcelerator CEO.”
The article states:
“Haynie also talked about how Appcelerator leverages with open-source while also giving back to its investors. For him, open-source does not necessarily mean free. It’s more like a mindset that enables a company to build technology through the benefit of a community that help each other innovate by expanding the innovation curve to prevent them from getting locked in to one proprietary system. ‘If you’re a platform company, you have to be open-source at some degree,’ Haynies added.”
Innovation and community are definitely two linchpins of open source. LucidWorks knows this and has done its best to capitalize on both. Their LucidWorks Search and LucidWorks Big Data platforms are built on the power of Apache Lucene and Solr, capitalizing on one of the strongest and longstanding open source communities.
Emily Rae Aldridge, June 5, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Advances in Search Drive Big Data Future
June 4, 2013
Big Data is getting the lion’s share of attention in the enterprise search market. As the cost of data storage continues to plummet, even smaller organizations are interested in capturing the meaning that may exist in massive amounts of routine data, or Big Data. So the Big Data market is exploding. Smart Data Collective is tracking the trends in their article, “7 Big Data Trends That Will Impact Your Business.”
The article highlights trends that will impact the average business. The third trend noted is advances in search and LucidWorks is a standout contender in this category:
“Sifting through massive amounts of data to find that preverbal needle in the haystack is no simple task. Over time we will likely see more big data solutions injecting search support into their solutions. Leading the way in this endeavor are LucidWorks, IBM, Oracle through the acquisition of Endeca (full disclosure, I’m a former Endeca employee), Autonomy and MarkLogic. LucidWorks combines an open source stack of Lucene/Solr, Hadoop, Mahout and NLP.”
So for those who are interested in capturing and using Big Data, but want an interface they can negotiate, LucidWorks is a great choice. Their LucidWorks Big Data is an application development platform that incorporates the power of Lucene, Solr, Hadoop, Hive, and other major open source powerhouses into a usable, efficient platform. Try it and see.
Emily Rae Aldridge, June 4, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Cloudera Falls Short in Big Data Security
May 31, 2013
Hadoop and its surrounding landscape are a big discussion point among not just the open source crowd, but also the entire discussion of enterprise data management. Cloudera is a leading contender for this interested in value-added solutions based on Hadoop. However, recent findings show that Cloudera is troubling some experts when it comes to security. Read more in the Cloud Tweaks article, “Cloudera Not Cutting It With Big Data Security.”
The article begins:
“Cloudera is, for the moment, a dominating presence in the open source Hadoop landscape; but does it have staying power? While Cloudera’s Big Data platform is the darling of the Hadoop space, they and their open source distribution competitors have so far failed to adequately address the elephant in the room: enterprise data security.”
When it comes to the enterprise, security failings cannot be easily overlooked. However, for those who are interested in harnessing the power of Hadoop, but maintaining the highest security standards, MapR recently launched a partnership with LucidWorks that does just that. MapR’s interest in LucidWorks was to bring powerful and secure analytics to Big Data through Hadoop. LucidWorks has a long record of industry trust and success, so enterprises can feel better about entrusting their data to a tested name in enterprise.
Emily Rae Aldridge, May 31, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Arnold in London: Big Data and Search
May 30, 2013
Stephen E Arnold appears in a two minute video. This program provides a summary of his main point in his lecture at the Enterprise Search Summit, May 15-16, 2013. What about Big Data magic? Watch the video here and find out. Navigate to http://youtu.be/CXap21nNjm8.
Don Anderson, May 30, 2013
FOCUS Issue 29 on Big Data Available Now
May 27, 2013
Datacenter Dynamics’ publication, FOCUS, just released issue 29 with an emphasis on Big Data. Penny Jones provides a look into the latest edition in her story, “FOCUS 29 – the Big Data Issue. Read it Online Now!” which appears on the Datacenter Dynamics blog.
To introduce the topic of Big Data, the author discusses the concept with the head of CERN’s open lab project, Bob Jones. She quotes him here:
“Jones likens CERN to a big industrial plant, which is quite fitting for GE’s purpose. GE pushes big data solutions focused on the Industrial Internet (see Pages 30-33).”
She then continues by introducing another leading mind in the world of Big Data:
“Another vendor representative I spoke with, LucidWork’s CTO Grant Ingersoll, told me machines and the Internet – funnily enough – are the reason we have big data in the first place. ‘[Big data] is this unique combination of man and machine that work together on data that brings new business insights and solves complex problems that, until now, have not been tackled. In many ways, business intelligence (BI), big data and analysis tools are closely related.’”
LucidWorks definitely has a voice in the Big Data market, as they seek to aid businesses with efficiency and accuracy in their enterprise search and Big Data needs. Built on Apache Lucene and Solr, the LucidWorks offerings do deliver on the promise that Ingersoll makes above, ensuring that business intelligence is coupled with analysis and Big Data in a way that ensures a competitive advantage.
Emily Rae Aldridge, May 27, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Big Data: O(log n) Again to Calculating Bad Presentations and Lousy Management
May 26, 2013
I participated in a quite interesting Big Data “event” recently. (Sorry, no link. I want to leave my post adrift in the sea of saucisson which makes up the Internet today.)
Big Data as a concept has been with us as long as there are people and storage. If a stack of clay tablets would not fit in a cabinet in ancient Babylon, the hapless analyst had a Big Data problem. When I had a Wang mini in my closet, Big Data was anything larger than 80 megabytes.
In my view, Big Data is a bit of a marketing confection. When companies cannot sell their core product like enterprise search, these outfits just tell the sales and marketing consultants to whip something up. Big Data is in my view one popular junk food when processed by some folks.
Let me set the stage.
A self appointed “expert” tried to organize a two-day lecture series to explain the basics of Big Data to those seeking information about this hot concept. The line up of teachers included marketers who took an interesting intellectual approach and me, the addled goose.
Now shift gears and think in terms of airline food. You are hungry on a flight from New York to Paris and the airline serves up a stale cracker and a substance which, from a distance, looks like cheese. Once up close, the combination did not deliver haute cuisine. Heck, the Big Data event was not the equivalent of meals ready to eat or MREs left in the equatorial sun for a couple of weeks in an undisclosed location.
The aftermath of Chernobyl event captured my impression of how misinformation about Big Data can set the stage for flawed decisions and catastrophic financial issues. Those emergency systems sure worked well in the engineering models, didn’t they? A happy quack to http://goo.gl/5u2E3.
Three impressions struck me as I reflected on the two-day event I attended with two of my colleagues. (More about my two colleagues in a good news moment.)
First, the self-aggrandizing poobah who was the “maestro of Big Data” left about mid way through Day Two. I am familiar with “experts” and azure chip consultants who have more pressing business than sticking with something to it completion. I was reminded of the behavior of the Costa Concordia captain. Was this disappearing act an indication of disorganization or craw fishing from failure? The steady attrition of paying attendees was evident by the third talk on the first day of the seminar. In fact, on Day One, Hour One, I counted 58 people in a room which was set to handle about 120. When the “maestro of Big Data” flew the coop, there were 15 people in the room. My talk, the end note for the event, pulled 30 people, up from the low of 15 at 2 pm on Day Two. Who introduced me? No one. Who stepped in to handle the last two hours of the event? My team, thank you.
Second, in my opinion, the majority of the speakers’ presentations were like most of the content on Slideshare, a business marketing service owned by LinkedIn, a job hunting service. (I think some of the speakers are denizens of LinkedIn, which I find quite amusing.) In my opinion, the Slideshare approach business information for many “members” is to take familiar, well-worn buzzwords. Then add a couple of trendy Hacker News references. Transfer recycled information to PowerPoint slides. And then serve up cold. I learned a great deal about SAP and how wonderful the company is for just about anything Big Data. I tuned out after the sixth or seventh worshipful reference. Although addled, I pick up on stuff once I hear the same old refrain three or four times. If you are interested in what was not covered in the seminar lectures, navigate to “What does O(log n) mean exactly?” I included one diagram with information about Big O, but marketers in my lecture lapsed into a coma when I mentioned the concept.
Third, I learned from several of the attendees that the Big Data sessions did not meet their expectations. I did deliver a lecture, and I had 30 people in the audience, not counting my two colleagues. I am not sure where these folks came from. We let them in the lecture hall because the organizer’s staff had wandered off to do more important tasks I assume. And, then — surprise, surprise — after my talk, five individuals clustered around me. Two of my colleagues witnessed the clump of groupies. I was hoping for major press coverage and maybe a Project Runway designer as fans. Instead, I got five — I am still in shock — mathematicians. The key comment witnessed by two experienced special librarians was, “You were the only speaker who told me how to think about Big Data problems. Very good.” I am not much of a thinker. I was not sure whether the adoring PhD from Rutgers was pulling my leg or speaking about the dismal quality of the other folks who were doing the Slideshare marketing thing. I was pleased with the feedback.
And, most importantly, I want to thank my two colleagues, Constance Ard, an honest to goodness, straight shooting law librarian, and Delores Meglio, once a New York Times’ executive who also worked with me at Ziff and who was was part of the senior management team at Elsevier Knovel, for:
- Stepping in when the “maestro” of the event disappeared because he had more pressing business than witnessing the sinking of his Titanic event. The event organizer’s staff apparently had to beat the commute rush home
- Facilitating the question and answer period which lasted a full 20 minutes after my lecture ended at 5 pm Eastern
- Chasing down an audio visual person to turn on the microphone and turn on the projection device. Apparently the show organizer’s team had better things to do that watch one speaker after another drive people from the room. I do not know if any paying customers were crying, but I would not rule anything out.
Will I reveal who organized this event? Nope.
Will I write a memo to the organizer, offering helpful suggestions to the organizer? Nope.
Will I point out which speakers scored a perfect 10 on the Slideshare airline food quality scale? Nope.
Why not?
I was thrilled to experience once again how people on my team deliver even when it is not their job.
Believe me when I say I was proud of Constance Ard’s and Delores Meglio’s spontaneous action. They made the last two hours of the Big Data event a success for the remaining attendees.
Did I tell Constance and Delores to step in? Nope.
Like others on my team of 30 people, Ms. Ard and Ms. Meglio are old-school professionals. Both believed that those in the Big Data lecture hall deserved 100 percent attention and effort. I was able to focus on making my talk the best it could be.
Could I have done a better job? Sure. Did I try my to do my best? Yes. I delivered despite the unprofessional setting in which I was placed.
Perhaps conference organizers and Big Data maestros could learn something about commitment and initiative by talking with people like Constance Ard and Delores Meglio, and ignoring the marketers who promise and, in my opinion, frequently fail to deliver?
Shift to another event commitment I had that same day, right after the Big Data lecture.l
In sharp contrast was the Startupalooza sponsored by iBreakfast event at John Jay College of Criminal Justice. I was asked to evaluate 15 start up ideas over a period of three hours. I want to point out that the Startupalooza event was organized, dynamic, professional, and exciting.
The reason?
The iBreakfast team and the 43 participants and five evaluators, were engaged. The innovators pitching start up ideas responded to the professionalism of the event and stepped up their game.
For me, the difference between the two events was as clear as choking down an MRE and chowing down at Le Bernadin.
Oh, what about my presentation and work at Startupalooze?
Those groupie mathematicians thought my talk was pretty good. What do math people know anyway? But I had three entrepreneurs clump around me after Startupalooza. Constance Ard extracted me because I continued to deliver at the 100 percent even though I was burning will power to keep going at 9 pm after my Big Data lecture.
My hope is that those younger than me try hard, do a professional job, and stick with commitments. My team performs in this manner. Will others follow Startupalooza’s, Ms. Ard’s, and Ms. Meglio’s example?
I sure hope so. Events succeed for many reasons. Professionalism is just one element and may, in some situations, be the catalyst for rising above mediocrity.
Stephen E Arnold, May 26, 2013
Sponsored by Augmentext
Positive and Negative Future Implications of Crowdoptic
May 24, 2013
The attempt to mine for insights in big data is not a new concept. The Huffington Post confirms this as they describe one of the more interesting pushes in this area. We learned more about Crowdoptic in “Visual Data Mining from Crowdsourcing: From Augmented Reality to Augmented Security?”
On the ever-continuous hunt for elusive metadata-laden images and other files, Crowdoptic focuses on the majority rules idea. This technology filters through files to find ones where people are/were “crowding” to click photos. Additionally, they can pinpoint hotspots within that given location where people are physically focusing their cameras. Of course this can be done in real time.
The article discusses this technology’s potential for augmented marketing and advertising:
“The potential uses for this kind of technology in business and marketing are still to be explored fully. The technology basically identifies what is holding the attention of people at a place at a given time. It is basically like Twitter trending, but with images posted online. And if the company’s claims are anything to go by, if they have a target location and time, the technology is capable of mining online visual data and pinpointing events or places that many people focus on with their smartphone cameras (basically, what people are looking at) in a matter of seconds.”
Not only marketing is discussed, but also security — for purposes of justice but also excessive surveillance such as in Orwell’s “1984.” Keep these new controversial technologies coming in; this is better than Hollywood gossip.
Megan Feil, May 24, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Government IT Professionals Not Ready for Big Data
May 18, 2013
It is not a surprise that 97 percent of state and local IT professional expect their data to grow by more than 50 percent over the next two years. However, more than 75 percent of them are only somewhat or not very familiar with the term big data. These findings are found in a recent report by MeriTalk and GCN did a nice write up on the implications of the study in, “Is Big Data Big Trouble for State, Local Governments?”
A survey of 150 state and local government CIOs and IT managers taken in November and December 2012 comprise the respondents in “The State and Local Big Data Gap.”
The article lists more of the statistics gleaned from the study:
“Seventy-nine percent of responding agencies said it will be at least three years before they are able to take full advantage of big data, even though they see it improving overall efficiency (57 percent); increasing the speed and accuracy of the decision-making process (54 percent); and providing a greater understanding of citizens’ needs (37 percent). And although 79 percent said they were just somewhat or not very familiar with the term, they do report having the kind of problems that big data techniques are intended to solve.”
Are state and local governments able to tap the alleged power of big data? Maybe not yet? That is certainly the conclusions that the numbers speak to.
Megan Feil, May 18, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Users Benefit from MapR Partnership with LucidWorks
May 17, 2013
MapR Technologies, specializing in Hadoop for Big Data, announced a new partnership with LucidWorks to bring full-text search and discovery to the platform. Read all the benefits to customers in the KM World article, “MapR taps LucidWorks for Hadoop.”
The announcement begins:
“MapR Technologies has announced distribution of LucidWorks Search with the MapR Platform for Apache Hadoop. MapR says the move allows customers to perform predictive analytics, full search and discovery, as well as conduct advanced database operations, on a single platform. The MapR/LucidWorks enterprise-class search capability works directly on Hadoop data but can also index and search standard files without having to perform any conversion or transformation.”
LucidWorks is a known leader in full-text search for the enterprise. Their LucidWorks Search and LucidWorks Big Data solutions are built on the sturdy open source infrastructure of Apache Lucene/Solr. Partnering with LucidWorks adds functionality to MapR solutions as well as shows a willingness to do what is best for the customer. LucidWorks’ strong track record only adds to MapR’s reputation and legitimacy.
Emily Rae Aldridge, May 17, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
LucidWorks Stories the Evolution of Big Data Search
May 16, 2013
Dan Kuznetsky is a trusted authority in enterprise search. He brings his expertise to the topic of search in Big Data in his latest article for ZDNet, “Evolution of Search in Big Data as Told by LucidWorks.”
After a discussion of how LucidWorks is contributing to the open source community through its participation in the Apache Software Foundation, Kuznetsky goes on to explore this interesting development model:
“LucidWorks is one of a growing number of technology companies that are building products based upon open-source software that was created in products hosted by the Apache Software Foundation. It is fascinating how they are cooperating to build the basic technology and then focusing on different competitive market niches. Each time I mentioned what I thought was a competitor, the folks from LucidWorks pointed out that those companies are partners that are trying to use their individual strengths together to serve the market. This is an area that is worth watching.”
Kuznetsky hit on the strength of LucidWorks and the rest of the value-added open source market. Innovation is encouraged, many benefit, but each company finds a niche that makes it profitable, but also useful. In this way, innovation is encouraged, open source development is encouraged, and users benefit from continuous improvement and support for the solutions in which they invest. Sounds like a win-win.
Emily Rae Aldridge, May 16, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
