AI SearchAutomation of dataflow for geospatial data using APACHE NiFi : a case study of mobile data
Call Number: AIT RSPR no.RS-20-03 Material type:
SeriesSeries: Asian Institute of Technology. Research studies project report ; no. RS-20-03 ; Publication details: Pathum Thani, Thailand : Asian Institute of Technology, 2020Description: 50 leaves : ill. (some col.)Subject(s): Dissertation note: Research Studies Project Report (M. Eng.) - Asian Institute of Technology. 2020 Summary: Today Information Technology has accomplished new climb by utilizing the innovation on Big Data particularly geospatial information. Dissecting information in tera bytes and past that has forced more prominent difficulties on information investigators. Maintenance and investigation of colossal information is a major test before corporate organizations which calls for utilization of Apache NIFI and utilizing different parts ofNIFI. In this paper we discussed the usage of NIFI for geospatial information. Important processors that can help in controlling the spatial data are discussed along with an example dataflow. The case study conducted with mobile data information was processed by creating three custom processors which prove that Nifi is highly scalable. Big data is of no use if it cannot be Visualized. So, all the processed data was Visualized on a web browser using Plotly and leaflet java libraries. The execution times for different sized files were recorded. The execution times to process 100MB ,200MB, 400MB, 800MB, 1200MB, 2000MB, 3000MB sized files were observed as 3.2, 3.8, 4.7, 6.9, 9.4, 15.7, 20.9 minutes. The speed of processing in Nifi was observed as linearly increasing with increase in size.
| Cover image | Item type | Current library | Home library | Collection | Shelving location | Call number | Materials specified | Vol info | URL | Copy number | Status | Notes | Date due | Barcode | Item holds | Item hold queue priority | Course reserves | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
40-Archives
|
Asian Institute of Technology Library Archives | AIT RSPR no.RS-20-03 (Browse shelf(Opens below)) | 1 | Available | 30050120885370 | |||||||||||||
20-AIT Publication
|
Asian Institute of Technology Library AIT Publications | AIT RSPR no.RS-20-03 (Browse shelf(Opens below)) | 1 | Available | 30050211021950 | |||||||||||||
|
20-AIT Publication
|
Asian Institute of Technology Library AIT Publications | AIT RSPR no.RS-20-03 (Browse shelf(Opens below)) | 2 | Available | 30050211021943 |
A research study submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Remote Sensing and Geographic Information Systems
Research Studies Project Report (M. Eng.) - Asian Institute of Technology. 2020
Today Information Technology has accomplished new climb by utilizing the innovation on Big Data particularly geospatial information. Dissecting information in tera bytes and past that has forced more prominent difficulties on information investigators. Maintenance and investigation of colossal information is a major test before corporate organizations which calls for utilization of Apache NIFI and utilizing different parts ofNIFI. In this paper we discussed the usage of NIFI for geospatial information. Important processors that can help in controlling the spatial data are discussed along with an example dataflow. The case study conducted with mobile data information was processed by creating three custom processors which prove that Nifi is highly scalable. Big data is of no use if it cannot be Visualized. So, all the processed data was Visualized on a web browser using Plotly and leaflet java libraries. The execution times for different sized files were recorded. The execution times to process 100MB ,200MB, 400MB, 800MB, 1200MB, 2000MB, 3000MB sized files were observed as 3.2, 3.8, 4.7, 6.9, 9.4, 15.7, 20.9 minutes. The speed of processing in Nifi was observed as linearly increasing with increase in size.
There are no comments on this title.