Senior Data Engineer

We are looking for an experienced data engineer responsible for expanding and optimizing our big data pipelines. You will be working alongside other engineers and developers working on different layers of the infrastructure. Therefore, a commitment to collaborative problem solving, sophisticated design, and the creation of quality products is essential. 

Responsibilities:

• Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems.
• Build & maintain ETL/ELT Ingestion pipelines and analytics solutions using cloud technologies.
• Ensure the design, code and procedural aspects of solution are production ready, in terms of operational, security and compliance standards.
• Participate in day-to-day project and production delivery status meetings, and provide technical support for faster resolution of issues.
• Work with Stakeholders and Understand their Requirements

Qualifications

• Bachelor's degree from an accredited college/university or equivalent work experience.
• Strong knowledge and understanding of SQL & Spark
• Proficient in one of the object-oriented and functional languages: Python, Java, Scala, etc.
• Strong problems solving and critical thinking skills
• Experience in creating data pipelines using ETL tools like Airflow, ADF
• 4 years of overall experience in data engineering & data warehousing domain.
• 2+ years' experience in Big Data Distributed systems such as Databricks, AWS EMR, AWS Glue
• Experience of writing effective and maintainable unit and integration tests for ingestion pipelines.
• Working knowledge of CI/CD pipelines for deployments.
• Working knowledge of using code version control repositories like Git.
• Knowledge of security best practices including encryption of sensitive data, encryption at rest and encryption in flight.
• Understanding with Data Warehouse, Data Lakehouse, Delta Lake, Apache Hudi.

We have an amazing team of 700+ individuals working on highly innovative enterprise projects & products. Our customer base includes fortune 5 retail and CPG companies, leading store chains, fast growth fintech and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 certified. We have a vibrant culture of learning via collaboration and making workplace fun.

People who work with us work with the cutting-edge technologies while contributing success to the company as well as to themselves. 

To know more about Confiz Limited, visit: https://web.facebook.com/lifeatconfiz