Architectural improvements for the optimization of a time series application - Hussain, Mufaddal Enayath; Chutiporn Anutariya; Chaklam Silpasuwanchai; Rajapaksha, Praboda; AIT Scholarships (Pathum Thani, Thailand : Asian Institute of Technology, 2023)
This 6-month internship report highlights the journey of enhancing the architecture of the HOTS application. Commencing in February and concluding in August, the intern ship focused on a primary objective, addressing real-time data handling and optimiz ing data management for larger datasets. To achieve these goals, a streaming service with Apache Kafka was carefully designed and integrated into the HOTS application. Rigorous testing, important feature implementation like the schema registry and conflu ent control center and SSL encryption were integrated, and deployment on a dedicated server resulted in a robust Kafka architecture. Code optimization techniques, includ ing batch processing and improved memory management, were employed to overcome bottlenecks in HOTS. The outcomes showed promising results with significant improve ments in running time, RAM usage, and dataframe sizes. Participation in Smile{u2019}s LLM Hackathon provided valuable insights into real-world LLM applications. The internship fostered valuable experiences in Python development, streaming platform design, and the importance of code optimization. Overall, this report showcases the successful im plementation of architectural improvements for the HOTS application, contributing to a deeper understanding of software engineering and the importance of optimizing code for optimal performance.
An Internship Study submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Computer Science
Internship Report (M. Eng.) - Asian Institute of Technology, 2023
Keywords: Application software -- Development