Data Engineer - Scala/PySpark (6-10 yrs) Noida (Analytics & Data Science)
Location:
Noida
Description:
Role : Data EngineerExp : 6+ YearsLocation : Noida Contract Duration : 3 monthsJob Description :Skills : - PySpark or Scala with Spark, Spark Architecture, Hadoop, SQL - Streaming Technologies like Kafka etc.- Proficiency in Advanced SQL (Window functions)- Airflow, S3, and Stream Sets or similar ETL tools.- Basic Knowledge on AWS IAM, AWS EMR and Snowflake.Responsibilities :- Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines to collect, process, and store large volumes of structured and unstructured data.- Data Modeling: Develop and maintain data models, schemas, and metadata to support the organization's data initiatives. Ensure data integrity and optimize data storage and retrieval processes.- Data Integration: Integrate data from various sources, including databases, data warehouses, APIs, and streaming platforms, ensuring compatibility, consistency, and quality.- Performance Optimization: Optimize data pipelines and processing systems for performance, scalability, and reliability. Identify and resolve bottlenecks and inefficiencies in data workflows.- Data Quality Assurance: Implement data quality checks, validation rules, and monitoring mechanisms to ensure the accuracy, completeness, and consistency of data across different systems.- Infrastructure Management: Collaborate with DevOps and IT teams to provision, configure, and manage infrastructure components such as databases, clusters, and cloud services required for data processing and storage.- Security and Compliance: Implement data security best practices and compliance standards to protect sensitive information and ensure regulatory compliance (e.g., GDPR, HIPAA). Monitor and audit data access and usage to prevent unauthorized activities.- Documentation and Communication: Document data pipelines, systems architecture, and technical processes. Communicate effectively with cross-functional teams to gather requirements, provide updates, and address issues.- Continuous Learning: Stay updated on emerging technologies, tools, and techniques in data engineering and related fields. Evaluate and recommend new technologies and approaches to improve data infrastructure and workflows. (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Data Engineer - Scala/PySpark (6-10 yrs) Noida (Analytics & Data Science)
Data Engineer - Scala/PySpark (6-10 yrs) Noida (Analytics & Data Science) is a Engineering Engineer Job at Forward Eye Technologies located in India. Find other listings like Data Engineer - Scala/PySpark (6-10 yrs) Noida (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.
Data Engineer - Scala/PySpark (6-10 yrs) Noida (Analytics & Data Science) is a Engineering Engineer Job at Forward Eye Technologies located in India. Find other listings like Data Engineer - Scala/PySpark (6-10 yrs) Noida (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.