Addressing the Limitations of Traditional Databases
October 20, 2009
I saw a news release (“SAS and Teradata Extend In Database Analytics Leadership, Capabilities”) that will be good news for SAS and Teradata (once part of NCR, I believe) customers struggling with very large datasets. The idea is that the two firms will join forces to put more crunching within the data warehouse data table environment. The idea is that processing will be faster and some of the tedious export and crunch steps will be reduced and in some cases eliminated.
When I read the story, however, I saw the tie up as yet another work around for the limitations of traditional data management systems. With large data sets, it takes too much time and effort to extract and process with numerical recipes subsets produced by an old-style query. Speed has to be a key feature. Even the opaque Google Squared and Wolfram Alpha services work quickly and allow a user to interact with data. Obviously for users accustomed to the cache based systems offered by Web search companies the hurry up and wait approach that traditional databases impose on users is a problem.
My view is that a leapfrog technology will create a lot of pain for vendors anchored in the traditional RDBMS approach to data. I don’t see leapfrog innovation from any of the database vendors. I see more innovation from companies like Aster Data and Infobright. But even these firms’ solutions may not be able to handle the petascale data flows for real time information streams and the growing piles of legacy data that must be crunched for a thorough analysis.
In short, a disruptive data management technology could disrupt the traditional RDBMS market. And quickly. Characteristics of a leapfrog solution could include:
- On premises and cloud hybrid approach
- Support for new types of queries such as lineage which makes it possible to “see” where data originate
- Massively parallel computation without a data cube
- User friendly syntax.
Any candidates to suggest?
Stephen Arnold, October 20, 2009
I got paid a dollar by SAS a couple of years ago but nothing for pointing out that this solution is promising but not a leapfrog. Ah, that’s life.
Comments
2 Responses to “Addressing the Limitations of Traditional Databases”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
Hi,
We have a well tried and tested database toolkit and method that is non traditional and may well be tomorrow’s technology today. It has been developed using private funding in UK over last 7 years, has been used to develop all of our own applications proving to be robust and dependable. Version 2 works fine, but would not cope with very high data volumes, version 3 is in Beta test and can cope with very high data volumes. We believe it has particularly good attributes for the warehouse market.
Contact me if this is of any interest to you.
Is this Redenet’s Germ?