Senior Data Engineer
CloudSmiths
📍 South Africa 💼 full_time 💰 competitive, industry benchmark compensation
Job Description
Join Our Team as a Senior Data Engineer: Architecting the Future of Data
Are you a passionate and experienced Senior Data Engineer eager to make a significant impact? We are seeking a dynamic and driven individual to lead the charge in designing, developing, and maintaining our innovative data solutions within a cutting-edge, cloud-first environment. If you have a proven track record of technical leadership, a deep understanding of data engineering principles, and a passion for building scalable and reliable systems, we invite you to apply!
Your Mission: Drive Innovation and Deliver High-Impact Data Solutions
As a Senior Data Engineer, you’ll be at the forefront of our data strategy, responsible for:
- Building and Optimizing Data Pipelines: Craft and maintain efficient ETL/ELT pipelines utilizing tools such as Apache Airflow and PySpark.
- Data Modeling and Design: Develop robust database schemas, implement dimensional models (Kimball/Inmon), and ensure data normalization for both relational and NoSQL databases.
- Data Warehouse and Lake Management: Contribute to the evolution and maintenance of our data warehouses, data lakes, and data lakehouses.
- Database Diversity: Work with a diverse ecosystem of databases, including Azure SQL, PostgreSQL, Google BigQuery, MongoDB, and Google Firestore.
- Big Data Handling: Efficiently manage structured, semi-structured, and big data file formats such as Avro, CSV, Parquet, ORC, and Delta.
- API Development and Integration: Design and maintain APIs for seamless data integration and workflows, leveraging your understanding of REST and microservices architectures.
- Codebase Excellence: Oversee codebase maintenance and optimization, utilizing Git for version control.
- Quality Assurance: Implement thorough integration testing and ensure the delivery of high-quality solutions for all new data processing scenarios.
- Technical Leadership: Provide technical design and coding assistance to team members, ensuring the successful completion of project milestones.
- Data Source Integration: Assess and integrate new data sources to meet the evolving needs of our business.
What You Bring to the Table: Expertise and Passion
We’re looking for a Senior Data Engineer with:
- Strong proficiency in Python and SQL (PostgreSQL or SQL Server preferred).
- Proven experience with Apache Airflow and PySpark.
- Familiarity with Databricks.
- Solid understanding of cloud platforms such as Azure, GCP, or AWS.
- In-depth knowledge of data warehousing concepts, dimensional modelling, and database normalization.
- Experience with big data file formats like Avro, Parquet, ORC, and Delta.
- Proficiency in working with APIs, REST, and microservices architectures.
Education and Experience: A Foundation for Success
The ideal candidate will possess:
- A Bachelor’s degree in Computer Science, Data Science, or a related field.
- 5+ years of progressive experience in data engineering, cloud computing, and technology implementation.
- Experience managing multi-shore projects and working within cloud ecosystems (SaaS/PaaS).
- Demonstrated experience in leading technical teams and mentoring team members.
What We Offer: A Rewarding Career and Exceptional Benefits
Join our team and enjoy:
- Opportunities to expand your skill set and advance your career.
- A competitive, industry-leading compensation package.
- Flexible working hours and a remote-friendly work environment.
- The chance to be part of a rapidly growing and innovative business.
- Collaboration with industry experts and a supportive team culture.
- Fantastic perks, including great coffee, team-building activities (like Samoosa Fridays!), and company-sponsored learning and certifications.
- A commitment to work-life balance and a “no work on your birthday” policy.
“