Machine Learning with Distributed Data Management and Process Architecture

Engin Baysal; Cuneyt Bayilmis

doi:10.1109/UBMK.2019.8907073

Machine Learning with Distributed Data Management and Process Architecture

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

With the development of technology that takes place more and more every day in our lives, it becomes almost impossible to manage and process the data produced and thus brought about the necessity of storage and analysis. Both the data size and the increase in the variety of data have necessitated the development of new methods in this context. In this study, distributed data management and analysis tools which are developed for data that cannot be processed in traditional regulations have been used. The machine learning application has been developed by using Logistic Regression classification algorithm. The application was implemented with the data set obtained from the sensors using pyspark libraries on the Spark cluster created using the Google Cloud service. And the working environment managed by YARN, has been observed during the implementation of the application.

Original language	English
Title of host publication	UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	53-57
Number of pages	5
ISBN (Electronic)	9781728139647
DOIs	https://doi.org/10.1109/UBMK.2019.8907073
Publication status	Published - Sept 2019
Externally published	Yes
Event	4th International Conference on Computer Science and Engineering, UBMK 2019 - Samsun, Turkey Duration: 11 Sept 2019 → 15 Sept 2019

Publication series

Name	UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering

Conference

Conference	4th International Conference on Computer Science and Engineering, UBMK 2019
Country/Territory	Turkey
City	Samsun
Period	11/09/19 → 15/09/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

apache spark
big data
big data analytics
logistic regression
machine learning
pyspark
yarn

Access to Document

10.1109/UBMK.2019.8907073

Cite this

Baysal, E., & Bayilmis, C. (2019). Machine Learning with Distributed Data Management and Process Architecture. In UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering (pp. 53-57). Article 8907073 (UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/UBMK.2019.8907073

@inproceedings{5f418b46f1ce4966861abdade2829163,

title = "Machine Learning with Distributed Data Management and Process Architecture",

abstract = "With the development of technology that takes place more and more every day in our lives, it becomes almost impossible to manage and process the data produced and thus brought about the necessity of storage and analysis. Both the data size and the increase in the variety of data have necessitated the development of new methods in this context. In this study, distributed data management and analysis tools which are developed for data that cannot be processed in traditional regulations have been used. The machine learning application has been developed by using Logistic Regression classification algorithm. The application was implemented with the data set obtained from the sensors using pyspark libraries on the Spark cluster created using the Google Cloud service. And the working environment managed by YARN, has been observed during the implementation of the application.",

keywords = "apache spark, big data, big data analytics, logistic regression, machine learning, pyspark, yarn",

author = "Engin Baysal and Cuneyt Bayilmis",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 4th International Conference on Computer Science and Engineering, UBMK 2019 ; Conference date: 11-09-2019 Through 15-09-2019",

year = "2019",

month = sep,

doi = "10.1109/UBMK.2019.8907073",

language = "English",

series = "UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "53--57",

booktitle = "UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering",

address = "United States",

}

Baysal, E & Bayilmis, C 2019, Machine Learning with Distributed Data Management and Process Architecture. in UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering., 8907073, UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering, Institute of Electrical and Electronics Engineers Inc., pp. 53-57, 4th International Conference on Computer Science and Engineering, UBMK 2019, Samsun, Turkey, 11/09/19. https://doi.org/10.1109/UBMK.2019.8907073

Machine Learning with Distributed Data Management and Process Architecture. / Baysal, Engin; Bayilmis, Cuneyt.
UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering. Institute of Electrical and Electronics Engineers Inc., 2019. p. 53-57 8907073 (UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Machine Learning with Distributed Data Management and Process Architecture

AU - Baysal, Engin

AU - Bayilmis, Cuneyt

PY - 2019/9

Y1 - 2019/9

N2 - With the development of technology that takes place more and more every day in our lives, it becomes almost impossible to manage and process the data produced and thus brought about the necessity of storage and analysis. Both the data size and the increase in the variety of data have necessitated the development of new methods in this context. In this study, distributed data management and analysis tools which are developed for data that cannot be processed in traditional regulations have been used. The machine learning application has been developed by using Logistic Regression classification algorithm. The application was implemented with the data set obtained from the sensors using pyspark libraries on the Spark cluster created using the Google Cloud service. And the working environment managed by YARN, has been observed during the implementation of the application.

AB - With the development of technology that takes place more and more every day in our lives, it becomes almost impossible to manage and process the data produced and thus brought about the necessity of storage and analysis. Both the data size and the increase in the variety of data have necessitated the development of new methods in this context. In this study, distributed data management and analysis tools which are developed for data that cannot be processed in traditional regulations have been used. The machine learning application has been developed by using Logistic Regression classification algorithm. The application was implemented with the data set obtained from the sensors using pyspark libraries on the Spark cluster created using the Google Cloud service. And the working environment managed by YARN, has been observed during the implementation of the application.

KW - apache spark

KW - big data

KW - big data analytics

KW - logistic regression

KW - machine learning

KW - pyspark

KW - yarn

UR - http://www.scopus.com/inward/record.url?scp=85076221467&partnerID=8YFLogxK

U2 - 10.1109/UBMK.2019.8907073

DO - 10.1109/UBMK.2019.8907073

M3 - Conference contribution

AN - SCOPUS:85076221467

T3 - UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering

SP - 53

EP - 57

BT - UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th International Conference on Computer Science and Engineering, UBMK 2019

Y2 - 11 September 2019 through 15 September 2019

ER -

Baysal E, Bayilmis C. Machine Learning with Distributed Data Management and Process Architecture. In UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering. Institute of Electrical and Electronics Engineers Inc. 2019. p. 53-57. 8907073. (UBMK 2019 - Proceedings, 4th International Conference on Computer Science and Engineering). doi: 10.1109/UBMK.2019.8907073

Machine Learning with Distributed Data Management and Process Architecture

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this