Big Data Engineer (Python, Azure) in Constanţa

GitFLow (advanced) Git (advanced) Airflow (advanced) Pandas (advanced) PySpark (advanced) Python (advanced) WE ARESoftServe is a leading advisory and IT development service company focused on helping clients to build high-end innovative software solutions.We are a big distributed team working with a fantastic customer, a giant in the mining industry, located in Canada. Our customer is a diversified resource company committed to responsible mining and mineral development with business units focused on steelmaking coal, copper, zinc, and energy. Headquartered in Vancouver they own or have interests in 13 operating mines, a large metallurgical complex, and several major development projects in the Americas. The client has expertise across a wide range of activities related to exploration, development, mining, and minerals processing, including smelting and refining, health and safety, environmental protection, materials stewardship, recycling, and research.Our customer’s corporate strategy is focused on exploring for, developing, acquiring, and operating world-class, long-life assets in stable jurisdictions that operate through multiple price cycles.YOU AREExcited about the following technologiesAzure Data FactoryDatabricks or HDInsight or Synapse AnalyticsAzure SQL DW (a.k.a. Synapse SQL Dedicated Pool)Azure Blob Storage and Azure Data LakeAzure SQL DB/Azure SQL MI/SQL ServerAzure FunctionsApache SparkEvent Hubs or similar (Kafka)Azure Stream Analytics or similar (Spark, Flink, Storm)Azure Cosmos DB (nice to have)In addition, you are proficient inComplex cloud services management conceptsNetworking (virtual networks, firewalls, communication between services)Authentication and security (IAM, service principals and managed identities), logging and monitoring, event-driven architecture (Event Grid, Azure Functions, Logic Apps)Hadoop ecosystem (e.g. Hive, Impala, Kafka, Spark, HBase, Presto), streaming platforms and stream ingestion services (Event Hubs, Kafka), stream analytics concepts and technologies (Flink, Spark, Storm, Azure Stream Analytics)Database modeling, DWH design, and implementationETL/ELT process design and implementationMassive-parallel-processing (MPP) systems: Azure SQL DW (Synapse), GCP BigQuery, Impala, AWS RedshiftNoSQL databasesOptimization of code, ETL processes, and SQL queriesSQL language and database management techniques (row-level security, built-in database roles, partitioning, index maintenance)OLAP systems and reporting tools (Azure Analysis Service, Power BI)Python, Pyspark, Pandas, Airflow, Git and Gitflow, CI/CD process building tools (Azure DevOps)Project management methodologies (Agile, Waterfall), the end-to-end architecture of Big Data solutions design and implementationCommunication with stakeholders in written and verbal form, suggesting the best solution among different options, and conducting trade-off analysisProblem-solving and team leadershipUpper-Intermediate English levelYOU WANT TO WORK WITHModern technologies popular on the marketInteresting customers and challenging projectsEither alone on a subproject or in a teamBoth developing new functionality and improving the existing onesRemote teams located in Ukraine, Poland & CanadaTOGETHER WE WILLAccomplish great thingsGet a great deal of learning and development opportunities along with our structured career pathProcess dynamic projects at a stable place of workTake part in internal and external events where you can build and promote your personal brandGive you access to experienced specialists willing to share their knowledgeCare about your individual initiatives: we are open to them, just come and share your ideas