Peepal Technology and Management
This listing was posted on hirist.
Data Engineer - Synapse Analytics (4-6 yrs) Bangalore (Analytics & Data Science)
Location:
Bangalore
Description:
Key Responsibilities :- Design, develop, and maintain data pipelines using Spark, Python, Scala, and related technologies on Hadoop/Cloudera Platform- Work with high volume data and ensure data quality and accuracy.- Implement data security and privacy best practices to protect sensitive data.- Develop and maintain documentation on data pipeline architecture, data models, and data workflows.- Monitor and troubleshoot data pipelines to ensure they are performing optimally.- Stay up to date with the latest developments in Azure, AWS, Spark, Python, Scala, and related technologies and apply them to solve business problems.- Optimize data pipelines for cost and performance.- Automate data processing tasks and workflows to reduce manual intervention.- Ability to work in Agile Feature teams.- Provide training and educate other team members around core capabilities and helps them deliver high quality solutions and deliverables/documentation.- Self-Motivator to perform Design / Develop user requirements, test and deploy the changes into production.Who we're looking for- 4-8 Years of experience on Spark Core, Spark SQL, SQL/Hive/Impala - Exposure on Hadoop Ecosystem(HDP/Cloudera/MapR/EMR etc)- - Experience of working on File formats (Parquet/ORC/AVRO/Delta/Hudi etc.)- - Experience with high volume data processing and data streaming technologies.- Experience and knowledgeable on the following:- Azure data offerings - ADF, ADLS2, Azure Databricks, Azure Synapse, Eventhubs, CosmosDB etc, Presto/Athena- - Experience of using Orchestration tools like Control-m - Strong experience in data modelling, schema design, and ETL development using SQL and related technologies. - Familiarity with data security and privacy best practices - Good exposure on TDD - Exposure on using CI tools like Git, Bitbucket, Github, Gitlab, Azure DevOps - Exposure on using CD tools like Jenkins, Bamboo, Azure DevOps - Cloud exposure (Hadoop)- - Exposure of working on Power BI- - Prior experience in building or working in team building reusable frameworks - Good understanding of Data Architecture and design principles. (Delta/Kappa/Lambda architecture)- - Exposure to Code Quality - Static and Dynamic code scans- - Good knowledge of NoSQL Databases/ HBase/ MongoDB / Cassandra / Cosmos DB - Good knowledge of GraphDB(Neo4J) - Experience with enterprise data management, Datawarehouse, data modelling, Business Intelligence, data integration. - Expertise in SQL and stored procedure. - Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge on NoSQL platforms. - Experience in working on Azure SQL Data warehouse (Synapse) and Azure analysis service or Redshift or Synapse - Propose best practices/standards - Translate, load and present disparate datasets in multiple formats/sources including JSON, XML etc. - Should be able to provided scalable and robust solution architecture depending on the business needs. - Should be able to compare tools and technologies and recommend a tool or technology - Should be well versed with overall IT landscape, technologies and should be able to analyse how different technologies integrates with each other (ref:hirist.tech)
Education/experience:
2 To 5 Years
Company:
Peepal Technology And Management Consultant
Posted:
May 25 on hirist
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Data Engineer - Synapse Analytics (4-6 yrs) Bangalore (Analytics & Data Science)
Data Engineer - Synapse Analytics (4-6 yrs) Bangalore (Analytics & Data Science) is a Engineering Engineer Job at Peepal Technology and Management Consultant located in India. Find other listings like Data Engineer - Synapse Analytics (4-6 yrs) Bangalore (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.
Data Engineer - Synapse Analytics (4-6 yrs) Bangalore (Analytics & Data Science) is a Engineering Engineer Job at Peepal Technology and Management Consultant located in India. Find other listings like Data Engineer - Synapse Analytics (4-6 yrs) Bangalore (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.