CloudQ IT Services
This listing was posted on hirist.
Lead Data Engineer - ETL/Python (8-11 yrs) Trivandrum/Thiruvananthapuram
Location:
Trivandrum/Thiruvananthapuram
Description:
Role Overview :The Lead Data Engineer will design, implement, and maintain data solutions on Microsoft Azure and SQL Server platforms. You will lead a team of data engineers, collaborate with stakeholders, and ensure efficient data processing, storage, and retrieval.Technical Responsibilities :- Design and maintain scalable data pipelines.- Create solutions using Azure Data Factory and Databricks for ETL processes.- Develop solutions in SSIS.- Analyze and optimize the existing data landscape for efficiency and scalability.- Collaborate with onshore counterparts to address technical gaps and complex scenarios.- Monitor and troubleshoot data systems to ensure performance and reliability.- Apply database management principles to ensure data quality.- Optimize data processes for speed and efficiency.- Ensure data architecture supports business needs and governance policies.- Define and execute data engineering strategies aligned with company goals.- Integrate data from various sources, ensuring quality and consistency.- Stay updated with emerging technologies and industry trends.- Translate banking industry business processes into data requirements.- Manage data migrations across different databases and servers.- Validate data transformations and support machine learning models.- Define data requirements and mappings in collaboration with business and IT teams.- Maintain and test data pipelines for performance validation.- Assemble large data sets that meet business requirements.- Identify data gaps and align with business and IT needs.- Understand the impact of data conversions on operations.- Manage complex data cases with accuracy and efficiency.Leadership Responsibilities : - Provide technical direction to the development team.- Review and approve code to ensure quality standards.- Troubleshoot and resolve technical issues.- Foster a culture of collaboration, innovation, and continuous improvement.- Optimize data flow for cross-functional teams.- Work closely with onshore data counterparts, product owners, and stakeholders to understand data needs.- Collaborate with IT and DevOps teams to align data infrastructure with IT architecture.- Implement best practices for data security and privacy. - Drive continuous improvement in data engineering.Candidate Expectations :- Design and develop warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services.- Proficient in SSIS, SQL, and query optimization.- Experience as a technical lead mentoring data engineers and managing data projects.- Experience in an onshore-offshore model handling challenging scenarios.- Lead data architecture and implementation to optimize data warehouse accuracy and usage.- Expertise in handling large data sets, building ETL pipelines, and generating insights using Data Science and Analytics.- Proficiency in Azure, AWS cloud services, and DevOps/CI/CD frameworks.- Ability to transform ambiguous requirements into deliverables.- Strong technical and interpersonal skills with excellent communication.- Drive automation using Infrastructure as Code (IaC), Terraform, Jenkins, and other CI/CD tools.- Help define architecture frameworks, best practices, and processes.- Collaborate on data warehouse architecture and technical design.- Expertise in Azure Data Factory for building ETL pipelines.- Proficient in SQL and relational databases.- Proficient in Python for ETL projects.- Experience with Databricks is an advantage.- Expertise in data lifecycle management, data ingestion, transformation, loading, validation, and performance tuning. (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Lead Data Engineer - ETL/Python (8-11 yrs) Trivandrum/Thiruvananthapuram
Lead Data Engineer - ETL/Python (8-11 yrs) Trivandrum/Thiruvananthapuram is a Engineering Engineer Job at Cloudq IT Services located in India. Find other listings like Lead Data Engineer - ETL/Python (8-11 yrs) Trivandrum/Thiruvananthapuram by searching Oodle for Engineering Engineer Jobs.
Lead Data Engineer - ETL/Python (8-11 yrs) Trivandrum/Thiruvananthapuram is a Engineering Engineer Job at Cloudq IT Services located in India. Find other listings like Lead Data Engineer - ETL/Python (8-11 yrs) Trivandrum/Thiruvananthapuram by searching Oodle for Engineering Engineer Jobs.