Codersbrain technology pvt ltd
This listing was posted on hirist.
ETL Data Engineer - Azure Databricks/PySpark (6-12 yrs) Bangalore (Analytics &
Location:
Bangalore
Description:
Job Description: ETL - Data EngineerResponsibilities:- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system performance issue related to data processing and implement solution to address them.- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.- Maintain integrity and quality across all pipelines and environments.- Understand and follow secure coding practice to make sure code is not vulnerableSkills :- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with strong python scripting exposure like list comprehensions, Dictionary variables etc.- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.- Proficient in SQl and database Design concepts.- Hands on experience with good proficiency in Delta table and delta file operations like merge, Insert override, Partition overrides etc.- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.- Knowledge on GitHub and build management.- Passion for data and experience working within a data driven organization- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: ETL Data Engineer - Azure Databricks/PySpark (6-12 yrs) Bangalore (Analytics &
ETL Data Engineer - Azure Databricks/PySpark (6-12 yrs) Bangalore (Analytics & is a Engineering Engineer Job at Codersbrain Technology located in India. Find other listings like ETL Data Engineer - Azure Databricks/PySpark (6-12 yrs) Bangalore (Analytics & by searching Oodle for Engineering Engineer Jobs.
ETL Data Engineer - Azure Databricks/PySpark (6-12 yrs) Bangalore (Analytics & is a Engineering Engineer Job at Codersbrain Technology located in India. Find other listings like ETL Data Engineer - Azure Databricks/PySpark (6-12 yrs) Bangalore (Analytics & by searching Oodle for Engineering Engineer Jobs.