Clouds, data centres, green computing and virtualisation
Our work seeks to guarantee the performance of virtualised systems and solve a wide range of problems that are related to cloud computing (and data centre) technologies.
Our research
Due to the massive success of virtualisation, more and more services will be run inside virtualised data centres to reduce both operational (OPEX) and capital expenditures (CAPEX). Aligned with the general trend of migrating traditional IT architectures to clouds (public or private), the next generation of telecommunication (Telecom) networks, such as 5G, are also expected to run on virtualised environments. In these environments network functions are deployed through virtual machines instead of current proprietary boxes. Guaranteeing the performance of such virtualised systems is challenging, however, because it requires accurate modelling and efficient optimisation of such services in their new environments.
This theme also deals with solving a wide range of problems that are related to cloud computing (and data centres) technologies. These problems include:
- resource allocation and scheduling
- self-optimisation techniques
- quality of service and data management
- accountability and provenance
- secure application isolation
- energy-aware algorithms
- autonomic management protocols
- service-level agreements.
The efficient solution of these problems should lead to environmentally friendly green cloud systems.
The addition of energy or power as an added constraint makes the above problems more complex. Moreover, the solution of these problems should lead to delivery of the required performance levels and minimises energy usage between all the components of a cloud environment simultaneously. However, any solution needs to be holistic in nature, taking into account all of the above issues along with the appropriate gluing tools that can lead to efficient, yet transparent solutions that don’t require continuous intervention from human operators.
Key projects
Our experts: Professor Albert Zomaya
Our partner: Associate Professor Javid Taheri (University Karlstad, Sweden)
Industry partner: Ericsson AB, Sweden
This project proposes tailor-made mathematical models for the virtual network function (VNF) / virtual machine (VM) placement / migration problem as a robust optimisation problem and solve it using classic/exhaustive (for example, linear programming) search algorithms. Solutions found in this mode will be accurate, yet very time consuming to find; they are however valuable to evaluate the suitability of several heuristics to be designed for faster convergence. The next step is to design heuristics (for example, genetic algorithm, particle swarm optimisation, ant colonies) to solve the aforementioned optimisation problem for real deployments.
Outcomes of this project can also be used as Network functions virtualization (NFV) specific optimisation problems for other researchers in this field. Other academic, research and industry groups can directly benefit from well-established results by either extending them to their own specific cases or using them as showcases, to aid learning about modelling their specific problems.
Our experts: Professor Albert Zomaya, Dr Mohammadreza Hoseinyfarahabady, Dr Wei Li
Our partners: Professor Paulo Pires and Professor Flavia Delicato (Federal University of Rio de Janeiro)
Industry Partner: ATMC, Sydney
The goal of this project is to specify an adaptive multicloud-based platform for processing data-intensive applications. The platform will enable applications to use resources provided by different underlying cloud computing platforms and dynamically adapt the infrastructure to deal with performance variations, while achieving energy efficiency and meeting quality of service (QoS) requirements.
Our experts: Professor Albert Zomaya, Dr Mohammadreza Hoseinyfarahabady
Industry Partner: ATMC, Sydney
Today more and more companies are faced with a huge amount of streaming data that needs to be quickly processed in real time to extract meaningful information. As a concrete example, big enterprises apply sophisticated machine learning algorithms to extract deeper insights from the raw data that continuously enter their systems. Two of the most popular platforms that can be used for real-time streaming data processing are Apache Storm and Apache Spark. In both systems a huge amount of data must be analysed/transformed continuously in the main memory units before it is stored on the hard drive. One of the major issues posed by such platforms is keeping the promised QoS level under fluctuations of request rates. Past research showed that the presence of high-arrival rate of streaming data within short periods can cause serious degradation to the overall performance of the underlying system.
In this project we are developing advanced controller techniques for famous streaming data processing engines such as Apache Storm, Apache Spark and Dask platform to best allocate computing resources. Our main goal is to preserve the QoS enforced by end users while keep the resources throughput at an optimal level.
Our experts: Professor Albert Zomaya, Dr Mohammadreza Hoseinyfarahabady
Our partner: Dr Young Choon Lee (Macq)
Industry partner: Precitech, Melbourne
Workload consolidation (either using virtualisation techniques or container-based methods) has attracted much attention in big cloud data centres in recent times. In this project, we try to find the best trade-off to reaching efficient energy consumption among shared resources. We are also seeking an optimised performance level of running collocated applications in a modern data centre that normally contains thousands of multi/many core boxes within a future cloud computing infrastructure. Past research has shown that performance interference due to shared resources across co-located virtual machines makes today's cloud computing paradigm inadequate for performance-sensitive applications, and more expensive than necessary for the others. To satisfy service quality demanded by customers, cloud providers have to deploy virtual machines in more physical machines (than actually is needed) to achieve a better resource isolation, which in turn adversely consumes higher levels of energy. Hence, a challenging problem for providers is identifying (and managing) performance interference between the VMs that are co-located at any given PM.
In this project we aim to propose consolidation algorithms to reduce energy consumption of a given data centre and avoid system performance degradation, with a focus on the impact of shared resources such as last level cash (LLC), and so on.
Our experts: Dr Dong Yuan, Associate Professor Bing Zhou
Today we are facing the challenge of storing and processing ever-increasing amounts of data produced by large-scale applications such as social network data sharing and worldwide collaborative scientific projects. In order to effectively and efficiently process and store geo-distributed big data, reliability must be assured. Replication technologies have been widely used in cloud systems; these not only guarantee data reliability but also improve the Quality of Service (QoS) for large-scale applications. Managing massive geo-distributed data replicas has large overheads, including bandwidth cost for transferring replicas among and within datacentres, and duplicated information in replicas are stored in the cloud.
This project will investigate data placement and deduplication techniques to improve performance and reduce storage needs in cloud systems.
Our experts: Dr Dong Yuan, Associate Professor Bing Zhou
Our industry partners: PolarSeven, Sydney
Public cloud service providers operate in a competitive environment that caters to users’ requirements for computing resources. Virtualisation technologies enable dynamic provisioning of cloud resources as virtual machines to users. However, the diverse applications pose a series of challenges to optimise resource utilisation, including the performance degradation caused by application interference. Container technologies such as Docker Swarm, Google Kubernetes and Apache Mesos provide the separation of different applications while allowing more flexible scheduling to optimise the resource utilisation.
This project will investigate practical co-scheduling strategies of virtual machines and containers for users of public cloud services.
Our experts: Professor Albert Zomaya, Dr Mohammadreza Hoseinyfarahabady
Industry Partner: Precitech, Melbourne
The aim of this project is to investigate the optimal scheduling of bag-of-task applications in hybrid cloud environments (comprising both private and public resources) in the presence of heavy-tailed and bursty traffic. This traffic model is considerably different from traditional arrival traffic patterns,which can be modelled using Poisson or Markov-modulated processes. However, empirical evidence supported by deep analysis on workload of today’s applications in several grid and cloud environments showed that the traffic of tasks' arrival is intrinsically bursty and exhibits correlations over longer time scales than the traditional models. In such scenarios that involve applications with heavy-tailed running time tasks, there are relatively few studies on the problem of task scheduling on hybrid cloud resources. One of the key considerations in the design of such a scheduling policy is the concept of Pareto-optimality of multiple objective functions, such as total incurring cost and/or makespan.
The output of this project will include an implementation of a suggested scheduling algorithm in the real cloud environment, with deep analysis of several well-known workload patterns.