Zyoin
This listing was posted on hirist.
Senior Data Engineer - Spark/Python (4-10 yrs) Bangalore (Analytics & Data
Location:
Bangalore
Description:
Mode : WFO Role Purpose :As a Senior Data Engineer, you will play a crucial role in our data infrastructure, ensuring the availability and reliability of data for our organization. You will work closely with cross functional teams to design, develop, and maintain data pipelines, optimize data storage and retrieval processes, and contribute to the overall data architecture. Your expertise in Python, Spark, SQL, Airflow, and AWS will be essential in building scalable and efficient data solutions.Key Accountabilities and Decision Ownership :- Data Pipeline Development : Develop and maintain robust data pipelines to ingest, transform, and deliver data from various sources to downstream systems, data lakes, and data warehouses.- Data Transformation : Perform data cleansing, enrichment, and transformation using Python, Spark, and SQL to ensure data quality, consistency and distributed data processing.- Data Modeling : Design and implement data models and schemas to support analytical and reporting requirements.- Performance Optimization : Optimize data processing and storage solutions for efficiency and scalability, working with large volumes of data.- Data Integration: Collaborate with data scientists, analysts, and other teams to integrate data into analytics and machine learning workflows.- Monitoring and Maintenance : Implement monitoring and alerting systems to ensure data pipeline reliability, and perform routine maintenance tasks.- Documentation : Maintain documentation for data pipelines, schemas, and processes to ensure knowledge sharing and best practices.Core Competencies, Knowledge and Experience :- Bachelor's degree in Computer Science, Information Technology, or a related field (Master's preferred).- At least 6 years of experience as a Senior Data Engineer or similar role.- At least 2 years of experience as team lead or team manager or similar role.- Familiarity with data warehousing concepts and technologies (e.g., Delta Lake).- Experience in building Big Data Architectures leveraging Spark, Delta Lake, Hadoop or similar- Strong programming skills in Python for data manipulation and transformation.- Proficiency in Apache Spark for distributed data processing - Batch & Real Time.- Advanced SQL skills for data querying and optimization.- Experience with workflow management tools like Apache Airflow.- Understanding of data security and privacy principles.- Excellent problem-solving and analytical abilities.- Strong communication and collaboration skills to work effectively in a cross-functional team.- Ability to work in a fast-paced environment and manage multiple projects simultaneously.- Continuous learning mindset to stay updated with the latest industry trends and technologies.Preferred Skills :- Proficiency in AWS services such as AWS Glue, AWS EMR, AWS Redshift, AWS S3, AWS Athena, and AWS Lambda.- Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).- Experience with data streaming technologies (e.g., Kafka, Kinesis).- Data pipeline orchestration and automation expertise.- Experience with version control systems (e.g., Git). (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Senior Data Engineer - Spark/Python (4-10 yrs) Bangalore (Analytics & Data
Senior Data Engineer - Spark/Python (4-10 yrs) Bangalore (Analytics & Data is a Engineering Engineer Job at Zyoin located in India. Find other listings like Senior Data Engineer - Spark/Python (4-10 yrs) Bangalore (Analytics & Data by searching Oodle for Engineering Engineer Jobs.
Senior Data Engineer - Spark/Python (4-10 yrs) Bangalore (Analytics & Data is a Engineering Engineer Job at Zyoin located in India. Find other listings like Senior Data Engineer - Spark/Python (4-10 yrs) Bangalore (Analytics & Data by searching Oodle for Engineering Engineer Jobs.