|
|
Computational tasks for Parallel data processing course
Horečný, Peter ; Rajnoha, Martin (referee) ; Mašek, Jan (advisor)
The goal of this thesis was to create laboratory excercises for subject „Parallel data processing“, which will introduce options and capabilities of Apache Spark technology to the students. The excercises focus on work with basic operations and data preprocessing, work with concepts and algorithms of machine learning. By following the instructions, the students will solve real world situations problems by using algorithms for linear regression, classification, clustering and frequent patterns. This will show them the real usage and advantages of Spark. As an input data, there will be databases of czech and slovak companies with a lot of information provided, which need to be prepared, filtered and sorted for next processing in the first excercise. The students will also get known with functional programming, because the are not whole programs in excercises, but just the pieces of instructions, which are not repeated in the following excercises. They will get a comprehensive overview about possibilities of Spark by getting over all the excercices.
|
|
| |
|
| |
|
|
Scalable preprocessing of data using Hadoop tool
Marinič, Michal ; Šmirg, Ondřej (referee) ; Burget, Radim (advisor)
The thesis is concerned with scalable pre-processing of data using Hadoop tool which is used for processing of large volumes of data. In the first theoretical part it focuses on explaining of functioning and structure of the basic elements of Hadoop distributed file system and MapReduce methods for parallel processing. The latter practical part of the thesis describes the implementation of basic Hadoop cluster in pseudo-distributed mode for easy program-debugging, and also describes an implementation of Hadoop cluster in fully-distributed mode for simulation in practice.
|
|
|
Application for Parallel Data Processing Control
Grepl, Filip ; Smrž, Pavel (referee) ; Dytrych, Jaroslav (advisor)
This work deals with the design and implementation of a system for parallel execution of tasks in the Knowledge Technology Research Group. The goal is to create a web application that allows to control their processing and monitor runs of these tasks including the use of system resources. The work first analyzes the current method of parallel data processing and the shortcomings of this solution. Then the work describes the existing tools including the problems that their test deployment revealed. Based on this knowledge, the requirements for a new application are defined and the design of the entire system is created. After that the selected parts of implementation and the way of the whole system testing is described together with the comparison of the efficiency with the original system.
|
|
|
Application for Parallel Data Processing Control
Grepl, Filip ; Smrž, Pavel (referee) ; Dytrych, Jaroslav (advisor)
This work deals with the design and implementation of a system for parallel execution of tasks in the Knowledge Technology Research Group. The goal is to create a web application that allows to control their processing and monitor runs of these tasks including the use of system resources. The work first analyzes the current method of parallel data processing and the shortcomings of this solution. Then the work describes the existing tools including the problems that their test deployment revealed. Based on this knowledge, the requirements for a new application are defined and the design of the entire system is created. After that the selected parts of implementation and the way of the whole system testing is described together with the comparison of the efficiency with the original system.
|
|
|
Computational tasks for Parallel data processing course
Horečný, Peter ; Rajnoha, Martin (referee) ; Mašek, Jan (advisor)
The goal of this thesis was to create laboratory excercises for subject „Parallel data processing“, which will introduce options and capabilities of Apache Spark technology to the students. The excercises focus on work with basic operations and data preprocessing, work with concepts and algorithms of machine learning. By following the instructions, the students will solve real world situations problems by using algorithms for linear regression, classification, clustering and frequent patterns. This will show them the real usage and advantages of Spark. As an input data, there will be databases of czech and slovak companies with a lot of information provided, which need to be prepared, filtered and sorted for next processing in the first excercise. The students will also get known with functional programming, because the are not whole programs in excercises, but just the pieces of instructions, which are not repeated in the following excercises. They will get a comprehensive overview about possibilities of Spark by getting over all the excercices.
|
|
|
Vyhodnocování relačních dotazů v proudově orientovaném prostředí
Kikta, Marcel ; Bednárek, David (advisor) ; Černý, Tomáš (referee)
This thesis deals with the design and implementation of an optimizer and a transformer of relational queries. Firstly, the thesis describes the theory of the relational query compilers. Secondly, we present the data structures and algorithms used in the implemented tool. Finally, the important implementation details of the developed tool are discussed. Part of the thesis is the selection of used relational algebra operators and design of an appropriate input. Input of the implemented software is a query written in a XML file in the form of relational algebra. Query is optimized and transformed into physical plan which will be executed in the parallelization framework Bobox. Developed compiler outputs physical plan written in the Bobolang language, which serves as an input for the Bobox. Powered by TCPDF (www.tcpdf.org)
|
|
| |
|
|
Scalable preprocessing of data using Hadoop tool
Marinič, Michal ; Šmirg, Ondřej (referee) ; Burget, Radim (advisor)
The thesis is concerned with scalable pre-processing of data using Hadoop tool which is used for processing of large volumes of data. In the first theoretical part it focuses on explaining of functioning and structure of the basic elements of Hadoop distributed file system and MapReduce methods for parallel processing. The latter practical part of the thesis describes the implementation of basic Hadoop cluster in pseudo-distributed mode for easy program-debugging, and also describes an implementation of Hadoop cluster in fully-distributed mode for simulation in practice.
|
guest ::
