Senior Data Engineer

Market insights

Stack

Free. Sign in with email.

Apply

Upload your CV — we will connect you with the employer directly through our pool.

Description

Accountabilities: - Design, develop, and optimize scalable ETL/ELT data pipelines for batch and real-time processing using Python, SQL, PySpark, Spark Structured Streaming, and Delta Live Tables. - Implement and maintain Medallion Architecture, organizing data into Bronze, Silver, and Gold layers to support data quality, performance, and analytical requirements. - Process terabyte-scale datasets using distributed computing techniques, ensuring pipeline reliability, scalability, and efficient resource utilization. - Manage Databricks environments, including workflows, clusters, Delta Lake, and related lakehouse components. - Implement Unity Catalog for metadata management, granular access controls, OAuth-based authentication, and enterprise-level data security and governance. - Develop and execute cloud migration strategies for legacy databases, including Oracle, Teradata, and SQL Server, to cloud-based data platforms and storage solutions such as Google Cloud Storage, Amazon S3, Azure Data Lake Storage, Snowflake, and Amazon Redshift. - Build and maintain CI/CD pipelines using Terraform and GitHub Actions to automate infrastructure provisioning, data pipeline deployments, testing, and monitoring. - Develop custom data processing scripts and data-driven APIs using Python and web frameworks such as Flask. - Use Docker to containerize data processing workloads and support portable, consistent deployment environments. - Evaluate emerging technologies and approaches for data-inten

Full description, company and the link to apply — after you sign in. Sign in

Employer contacts (email/phone/telegram) are hidden from the public preview — send your CV, and we will connect you directly.