Data Platform Engineering Lead

Randstad Digital is specialized in the recruitment of middle and senior positions for the Digital Market.

Our client is the leading company in the Aerospace, defence, and security sectors on a global scale and he is looking for a Data Platform Engineering Lead for Lisbon or Oporto.

The Data Platform Engineering Lead will be responsible for overseeing the design, implementation and maintenance of our Data & Analytics platform, pipelines and infrastructure. This role requires a combination of strong technical expertise, leadership abilities and a strategic mindset. The ideal candidate will have a proven track record in data architecture, data engineering, cloud and on-premise technologies, with experience in managing a deeply technical team and driving complex projects to completion. This role involves collaborating with cross-functional teams to ensure our data platform is robust, scalable, secure and aligned with the company's strategic objectives.

responsabilidades chave

Leadership: Lead and mentor a team of data architects and data engineers, fostering a collaborative and high-performance culture.

Platform Development: Oversee the design, development and maintenance of scalable and robust data architectures and pipelines, ensuring efficient data processing, storage and retrieval, including data ingestion, storage, processing, management and analytics components.
Architecture Design: Develop and maintain the architecture of the data platform, ensuring it meets the needs of various business units and supports future growth.
Technology Strategy: Define the technology stack and architecture standards for the data platform, ensuring alignment with industry best practices and emerging trends.
Data Integration: Develop strategies for integrating diverse data sources, ensuring seamless data flow and high data quality.
Scalability and Performance: Design solutions that can scale with growing data volumes and ensure optimal performance across the data platform.
Performance Optimization: Continuously monitor and optimize the performance of the data platform to handle large-scale data efficiently.
Security and Compliance: Implement robust data security measures and ensure compliance with relevant regulations and industry standards.
Collaboration: Work closely with data scientists, analysts, product managers and other stakeholders to understand data requirements and deliver effective solutions.
Innovation: Stay up-to-date with the latest advancements in data technologies and drive continuous innovation within the data platform.
Documentation: Maintain comprehensive documentation of data pipelines, ETL processes and architectural decisions.

competências

Education: Bachelors or Masters degree in Computer Science, Engineering, Information Systems, or a related field.
Experience: 7+ years of experience in data engineering or a similar role, with at least 3 years in a leadership position.
Technical Expertise:

Strong experience with Data Platform reference architectures (e.g. Lambda architecture, Data Mesh).
Deep knowledge of big data technologies (e.g., Hadoop, Spark, Kafka) and data warehousing solutions (e.g., Redshift, Snowflake).
Extensive experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services, with a focus on Google Cloud. Google Cloud certification is preferred.
Experience with migration from on-premise to cloud and vice versa.
Good knowledge of relevant security frameworks & standards.
Proficiency in programming languages such as Python, Java, or Scala.
Strong understanding of database management systems (e.g., SQL, NoSQL, NewSQL). Experience with SQL and database management systems (e.g., MySQL, PostgreSQL, SQL Server).
Knowledge of data integration tools and frameworks (e.g., Apache Nifi, Talend, Informatica).
Familiarity with data modeling, data warehousing and data governance practices.
Experience with Iaac (e.g. Ansible, Terraform), data pipeline orchestration (e.g. Airflow), log exploration tools (e.g. Streamlit, Dash), data extraction (e.g. PostGIS, Kafka, Airflow, FastAPI), pandas, scikit-learn, Docker.
Solid knowledge of DevOps best practices and tools: GIT, CI/CD, telemetry and monitoring, etc.
Analytical Skills: Strong analytical and problem-solving skills with a focus on delivering scalable and efficient data solutions.
Leadership: Proven leadership skills with experience in building and leading high-performing teams.
Communication: Excellent verbal and written communication skills, with the ability to effectively collaborate with technical and non-technical stakeholders.
Project Management: Strong project management skills with the ability to manage multiple projects and priorities simultaneously.
Attention to Detail: High attention to detail and a commitment to ensuring data quality and accuracy.
Adaptability: Ability to work in a fast-paced, dynamic environment and manage multiple priorities simultaneously.

principais benefícios

An excellent work environment and an opportunity to create a real impact in the world;
A truly high-tech, state-of-the-art engineering company with flat structure and no politics;
Professional development opportunities;
Collaborative and inclusive work environment;
Salary compatible with the level of proven experience.