Job Details

Azure Data Engineering Lead / Architect

The Azure Data Engineering Lead / Architect will be responsible for designing, implementing, and managing scalable and secure data architectures using Azure Data Lake and Databricks. This role involves strategic planning, team leadership, and close collaboration with cross-functional teams to ensure data systems align with business objectives and industry best practices.

Responsibilities

• Architectural Design and Implementation:
• Design and implement scalable and efficient data architectures leveraging Azure Data Lake and Databricks.
• Define and enforce best practices for data ingestion, storage, processing, and retrieval.
• Optimize data workflows to ensure high performance and cost efficiency.
• Data Governance and Security:
• Develop and implement robust security measures for Azure Data Lake in compliance with organizational and regulatory standards.
• Manage RBAC, encryption, and other security protocols within Azure Subscriptions.
• Collaborate with security teams to conduct audits and vulnerability assessments.
• Team Leadership and Collaboration:
• Lead and mentor a team of data engineers, providing technical direction and professional growth opportunities.
• Collaborate with data scientists, business analysts, and IT teams to deliver high-quality data solutions.
• Drive Agile methodologies and ensure timely project delivery.
• Platform Optimization and Monitoring:
• Oversee deployment and management of Azure Data Lake and Databricks environments.
• Implement monitoring, alerting, and optimization strategies for performance and reliability.
• Evaluate and integrate new Azure services and technologies to enhance the data platform.
• Strategic Planning and Roadmap Development:
• Develop and execute a data engineering roadmap aligned with business and technical goals.
• Stay current with industry trends and Azure advancements to guide data strategy.
• Provide long-term recommendations for data lakehouse adoption and cloud optimization.

Education and Experience Requirements

• Bachelors or Masters degree in Computer Science, Information Technology, or a related field.
• 8+ years of experience in data engineering, including at least 3 years in a leadership role.
• Proven expertise in architecting and implementing large-scale data solutions.
• Hands-on experience with Azure Data Lake, Databricks, and other Azure data services.

Required Knowledge and Skills

Technical Skills:

• Strong proficiency in SQL.
• Deep understanding of Azure security concepts, subscription management, RBAC, and data encryption.
• Experience in data modeling, ETL pipelines, and big data technologies.
• Familiarity with CI/CD pipelines and DevOps practices in data engineering.

Soft Skills:

• Strong leadership and team management capabilities.
• Excellent communication and stakeholder management skills.
• Analytical, proactive, and solution-oriented mindset.

Preferred Qualifications

• Azure certifications such as Azure Data Engineer Associate or Azure Solutions Architect Expert.
• Experience implementing data lakehouse architectures.
• Familiarity with data governance frameworks (GDPR, CCPA, HIPAA).