Senior Data Engineer

Аналитика рынка

Стек

Бесплатно. Вход по почте.

Откликнуться

Загрузите резюме — мы свяжем вас с работодателем напрямую через нашу базу.

Описание

Accountabilities: - Design, develop, and optimize scalable ETL/ELT data pipelines for batch and real-time processing using Python, SQL, PySpark, Spark Structured Streaming, and Delta Live Tables. - Implement and maintain Medallion Architecture, organizing data into Bronze, Silver, and Gold layers to support data quality, performance, and analytical requirements. - Process terabyte-scale datasets using distributed computing techniques, ensuring pipeline reliability, scalability, and efficient resource utilization. - Manage Databricks environments, including workflows, clusters, Delta Lake, and related lakehouse components. - Implement Unity Catalog for metadata management, granular access controls, OAuth-based authentication, and enterprise-level data security and governance. - Develop and execute cloud migration strategies for legacy databases, including Oracle, Teradata, and SQL Server, to cloud-based data platforms and storage solutions such as Google Cloud Storage, Amazon S3, Azure Data Lake Storage, Snowflake, and Amazon Redshift. - Build and maintain CI/CD pipelines using Terraform and GitHub Actions to automate infrastructure provisioning, data pipeline deployments, testing, and monitoring. - Develop custom data processing scripts and data-driven APIs using Python and web frameworks such as Flask. - Use Docker to containerize data processing workloads and support portable, consistent deployment environments. - Evaluate emerging technologies and approaches for data-inten

Полное описание, компания и ссылка на вакансию — после входа. Войти

Контакты работодателя (email/phone/telegram) скрыты из публичного превью — отправьте резюме, чтобы мы связали вас напрямую.