EIT Digital Hadoop Innovation Activity wins 2017 IEEE Scalable Computing Challenge

Hadoop Open Platform-as-a-Service (Hops) provides a WebUI to manage projects, search and visualize your DataSets and interactively analyze your data.

Hadoop Open Platform-as-a-Service (Hops) provides a WebUI to manage projects, search and visualize your DataSets and interactively analyze your data.


The EIT Digital Hadoop Innovation Activity HopsWorks is even more confident about the prospects for dissemination of its "Hops" platform and gaining Venture Capital now that it has won the IEEE International Scalable Computing Challenge (SCALE 2017), organised by the world's largest technical professional society IEEE*.

Hops is the brand name for a Hadoop** based next generation open-source software framework. A startup, Logical Clocks AB, has been established to distribute Hops and focuses on the Internet of Things (IoT) and telecoms markets, as well as, owners of sensitive Big Data. Both are outcomes of the HopsWorks Innovation Activity launched in January 2017. Dr. Jim Dowling, Senior Researcher at the RISE SICS and the leader of the Innovation Activity, said:

"Winning the IEEE SCALE Challenge is a validation of the Hops platform by our peers in academia and helps wider dissemination of the platform within the research community. Further, this will also strengthen our attractiveness in attracting venture capital, as it provides further validation of the platform as the best-in-class Hadoop distribution."

The objective of the 2017 IEEE SCALE Challenge was to highlight and showcase real-world problem solving using computing that scales. Effective solutions to many scientific and engineering problems require applications that can scale. The SCALE Challenge is concerned with advances in application development and their supporting infrastructure to enable scaling. Hops framework, selected as one of the four finalists, was chosen as the winner. Dr. Dowling, continued:

"Hops introduces a new scale-out metadata architecture that enables our filesystem, HopsFS, to massively outperform the existing Hadoop Distributed Filesystem - FS. Another differentiating feature of Hops is its production-readiness. The jury was quite impressed by the fact that HopsFS is a complete system which is used by real customers."

Unlike existing offerings, Hops promises to allow organisations to host multiple sensitive datasets on the same cluster, providing dynamic role-based access control for both the Hadoop Distributed File System and Kafka. The platform is unified in an intuitive user interface and provides first-class support for Spark and Flink streaming. Essential parts of this has already been redeemed. Jim Dowling explained:

"Since January 2017, Hops has been extended to support graphics processing unit - GPU - resources in YARN cluster management technology and is the only Hadoop platform to have native support for Distributed Tensorflow, the world's most popular deep learning framework."

In addition to the Swedish RISE SICS, the other EIT Digital partners participating in the HopsWorks Innovation Activity are the Hungarian Academy of Sciences Institute for Computer Science and Control MTA Sztaki, and Atos, a global leader in digital transformation.


*The IEEE (Institute of Electrical and Electronics Engineers) is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. IEEE and its members inspire a global community through its highly cited publications, conferences, technology standards, and professional and educational activities.

**Hadoop is an Apache open-source software framework that provides massive storage for all kinds of data, enormous processing power and the ability to handle a vast number of concurrent tasks or jobs. It also enables the storage of files that are bigger than what can be stored on one particular node or server.

© 2010-2020 EIT Digital IVZW. All rights reserved. Legal notice. Privacy Policy.