An open source distributed big data system for expressive, declarative, and efficient batch & streaming data processing and analysis
The Apache Software Foundation (ASF) announced on the 12th of January 2015 that Apache FlinkTM (an open source distributed data analysis engine for batch & streaming data) has graduated from the Apache Incubator to become a Top-Level Project (TLP). The origins of Apache Flink can be traced back to the 2009 DFG (Deutsche Forschungsgemeinschaft) funded, Stratosphere Research Project, also supported by the EIT ICT Labs.
The Stratosphere Project conducted R&D to explore the power of massively parallel computing for complex information management applications. The project was spearheaded by Prof. Volker Markl, Head of the Database Systems & Information Management (DIMA) Group at TU Berlin and director of the research group “intelligent analysis for massive data” at the German Research Center for Artificial Intelligence (DFKI), in conjunction with TU Berlin, Humboldt University of Berlin, and Hasso Plattner Institute (HPI) Principal Investigators/Researchers. Their joint collaboration enabled the development of a novel, database-inspired, massively parallel data processing system. Their success lead several DIMA researchers to launch, dataArtisans, a Berlin-based startup committed to growing the Flink code base and community, all the while ensuring that Flink is entirely (100%) freely available as open source. This success story serves as a testament to the industriousness, passion, and a spirit for technological innovation that is fostered among all DIMA Group members. Apache Flink is also the technological foundation of the Berlin Big Data Center, which has the goal to perform groundbreaking research and development, to train the "data scientists" of tomorrow and to create solutions that facilitate the deep analysis of massive amounts of heterogeneous data sets and streams at high velocity.
1. For the official ASF Press Release visit s.apache.org/YrZ
2. For more information about ApacheTM FlinkTM visit flink.apache.org
3. For more information about the dataArtisans visit data-artisans.com
4. For an interview with two former DIMA members and dataArtisans founders, visit http://sdtimes.com/inside-apache-software-foundations-newest-top-level-project-apache-flink/
5. For more information about DIMA visit www.dima.tu-berlin.de
6. For more information about DFKI visit www.dfki.de
7. For more information about the BBDC visit bbdc.berlin
