|
|
Real Time Data Processing with Strimzi Project
Orsák, Maroš ; Malík, Viktor (oponent) ; Rogalewicz, Adam (vedoucí práce)
Container technologies become broadly used in modern times. In prevailing, applications made on the micro-service architecture are rising. This thesis analyzes the design of an application that will process data in real-time. Further, the application will be built using state-of-the-art technologies used by world companies like Netflix, Uber. They are using the systems for real-time data processing such as Apache Kafka, and in recent times they raised it on a higher level by encapsulating this system in the container environment, which guaranteeing effortless scalability. Additionally, using the latest native Kubernetes technologies for processing dozens of data with Quarkus and Strimzi. The problem, which arises, is that these types of real-time data processing systems encapsulated in the containers are especially challenging to test. The main goal of this thesis is a proof-of-concept application based on Strimzi project and also show the designed long term test of the application also known as Marathon, which is the ideal demonstration of user conditions.
|
|
|
Configurable Parallel Execution of System Tests within the Strimzi Project
Orsák, Maroš ; Rogalewicz, Adam (oponent) ; Češka, Milan (vedoucí práce)
In recent years, many companies have adopted Kubernetes and the microservices architec- ture it enables. This technology was opened up many new possibilities not just for large companies, but also for small software developers. Kubernetes is a container-orchestration system and recently a new concept has emerged arround how to orchestrate the containers more efficiently - the Operator pattern. One such operator is developed and maintained under an open-source project called Strimzi. The Strimzi project gathers together several tools, which take care of the deployment of Apache Kafka on Kubernetes. Since Kafka is a complex, horizontally scalable, distributed system, you can imagine that its installation is a relatively complex action. Therefore, one of the biggest challenges of using Kubernetes is how to effectively and quickly test projects such as Kafka and Strimzi and at the same time verify integration with other similar products. The resources needed by Kubernetes are much more demanding compared to the deployment of Kafka on virtual machines or typi- cal container instances. To tackle this problem, we adopt the principles of parallel execution and created a mechanism within Strimzi system tests, which runs tests in parallel against only a single Kubernetes cluster. Furthermore, we proposed a brand new architecture for the end-to-end tests. The improvements aim at scalability and reduction of execution time . Through several experiments, this paper shows that proposed mechanism with different configurations of the Kubernetes cluster (including number of Kubernetes nodes , number of tests and suites executed in parallel ) significantly accelerated execution of the tests.
|
|
|
Configurable Parallel Execution of System Tests within the Strimzi Project
Orsák, Maroš ; Rogalewicz, Adam (oponent) ; Češka, Milan (vedoucí práce)
In recent years, many companies have adopted Kubernetes and the microservices architec- ture it enables. This technology was opened up many new possibilities not just for large companies, but also for small software developers. Kubernetes is a container-orchestration system and recently a new concept has emerged arround how to orchestrate the containers more efficiently - the Operator pattern. One such operator is developed and maintained under an open-source project called Strimzi. The Strimzi project gathers together several tools, which take care of the deployment of Apache Kafka on Kubernetes. Since Kafka is a complex, horizontally scalable, distributed system, you can imagine that its installation is a relatively complex action. Therefore, one of the biggest challenges of using Kubernetes is how to effectively and quickly test projects such as Kafka and Strimzi and at the same time verify integration with other similar products. The resources needed by Kubernetes are much more demanding compared to the deployment of Kafka on virtual machines or typi- cal container instances. To tackle this problem, we adopt the principles of parallel execution and created a mechanism within Strimzi system tests, which runs tests in parallel against only a single Kubernetes cluster. Furthermore, we proposed a brand new architecture for the end-to-end tests. The improvements aim at scalability and reduction of execution time . Through several experiments, this paper shows that proposed mechanism with different configurations of the Kubernetes cluster (including number of Kubernetes nodes , number of tests and suites executed in parallel ) significantly accelerated execution of the tests.
|
|
|
Real Time Data Processing with Strimzi Project
Orsák, Maroš ; Malík, Viktor (oponent) ; Rogalewicz, Adam (vedoucí práce)
Container technologies become broadly used in modern times. In prevailing, applications made on the micro-service architecture are rising. This thesis analyzes the design of an application that will process data in real-time. Further, the application will be built using state-of-the-art technologies used by world companies like Netflix, Uber. They are using the systems for real-time data processing such as Apache Kafka, and in recent times they raised it on a higher level by encapsulating this system in the container environment, which guaranteeing effortless scalability. Additionally, using the latest native Kubernetes technologies for processing dozens of data with Quarkus and Strimzi. The problem, which arises, is that these types of real-time data processing systems encapsulated in the containers are especially challenging to test. The main goal of this thesis is a proof-of-concept application based on Strimzi project and also show the designed long term test of the application also known as Marathon, which is the ideal demonstration of user conditions.
|
host ::
RSS.