Arrow Electronics
Kalyubia
Position: Data Automation Engineer Job Description: As a Data Automation Engineer, you will play a crucial role in developing and maintaining automated data processing systems. You will work closely with data analysts, and other stakeholders to design, implement, and optimize data pipelines, ensuring the efficient and accurate flow of data throughout the organization. What You'll Do: Develop and implement automated data pipelines for ingesting, processing, and storing structured and unstructured data. Integrate data from various sources in both real-time and batch mechanisms and perform transformations to ensure consistency, accuracy, and reliability. Implement data quality checks and monitoring mechanisms to identify and address issues proactively. Write code and scripts for automation of data processes using languages such as Python, Java, or Scala. Leverage ETL (Extract, Transform, Load) tools and frameworks [Apache spark, AWS Glue] for efficient data processing....