Senior Data Engineer

Job Description

Join a leading engineering network as a Senior Data Engineer, where you will design, construct, and oversee scalable data pipelines (ETL/ELT) to bolster analytics and AI initiatives. You'll refine data architectures, data lakes, and data warehouses, collaborating closely with stakeholders to convert business needs into impactful data solutions and insights. Responsibilities include crafting and maintaining data processing scripts and automation tools, enhancing AI/analytics platforms, and ensuring pipeline performance through monitoring, testing, and tuning.

Qualifications

1. Requires over 5 years of data engineering or software engineering experience. 2. Strong programming skills in Python and SQL are essential. 3. Experience with ETL/ELT pipelines and data modeling is necessary. 4. Must have hands-on experience with data warehousing tools like Snowflake, Redshift, and BigQuery. 5. Familiarity with big data tools such as Spark, Kafka, and Hadoop is required. 6. Experience with cloud platforms like AWS, Azure, or GCP is necessary. 7. Knowledge of relational and NoSQL databases is essential. 8. Familiarity with CI/CD and DevOps practices is needed. 9. Experience with real-time/streaming data pipelines is required. 10. Exposure to AI/ML data pipelines or feature engineering is a plus. 11. Experience with containerization technologies like Docker and Kubernetes is beneficial. 12. Knowledge of data observability and monitoring tools like ELK and Splunk is preferred. 13. Experience in large-scale distributed systems is required.

Benefits

- Opportunity for repeat engagements based on performance. - Clear scope with no ambiguity over deliverables.