The Future of Distributed Computing Rests with Hadoop
Moore's law has finally hit the wall and CPU speeds have decreased in the last few years; the industry is reacting with hardware with more cores and software that can leverage 'grids' of distributed computing resources. Further, the assimilation of computing into our daily lives is enabling the generation of data at unprecedented rates. The amount of digital information churned out in 2011 is estimated to be 10 times that of what was produced in 2006; that is, 1800 exabytes. The rising number of web applications serving millions of Internet users and dealing with petabytes of data, the advent of cheap storage capacity resulting in a tremendous growth in data retention, and the availability of cheap resources to process that data have all reinstated the need for large-scale data processing.