Back
India   India   Engineer   Circuit Compilers -

Data Engineer - Python/PySpark (5-8 yrs) Anywhere in India/Multiple Locations | Engineer in Engine1

Circuit Compilers

This listing was posted on hirist.

Data Engineer - Python/PySpark (5-8 yrs) Anywhere in India/Multiple Locations

Share
Tools
Location:
Anywhere in India/Multiple Locations
Description:

Job Description :- 5 to 8 years in the build release management, software configuration, design, development, and cloud implementation. - Data Engineer with expertise in Python, Scala, PySpark, AWS S3, Glue, Redshift, Lambda, DBT, Serverless, Snowflake, SQL, Datalake, and Data Pipeline. - Leveraged AWS Glue for data cataloging, ETL job orchestration, and schema evolution. - Expert knowledge in AWS Data Lake implementation and support (S3, Glue,DMS Athena, Lambda, API Gateway, Redshift) - Handling of data related activities such as data parsing, cleansing quality definition data pipelines, storage and ETL scripts - Experiences in programming language Python/Pyspark/SQL- Experience with data migration with hands-on experience - Experiences in consuming rest API using various authentication options with in AWS Lambda architecture orchestrate triggers, debug and schedule batch job using a AWS Glue, Lambda and step functions- Understanding of AWS security features such as IAM roles and policies- Exposure to Devops tools- AWS certification in AWS is highly preferred- Should develop and maintained scalable data pipelines using Python, Scala and PySpark to process and transform large volumes of structured and unstructured data. - Should develop and maintained scalable and distributed data processing pipelines using Scala and Apache Spark, handling large volumes of data efficiently. - Should Implement complex data transformations, aggregations, and analytics using Scala and Spark - Collaborated with data scientists to integrate machine learning algorithms into Spark workflows, enabling advanced data analysis and predictive modeling. - Optimized Spark jobs by tuning configuration parameters, optimizing data partitioning, and leveraging in-memory caching techniques, resulting in significant performance improvements. (ref:hirist.tech)
Education/experience:
2 To 5 Years
Company:
Circuit Compilers
Posted:
June 13 on hirist
Share
Tools
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
  • Always meet the employer in person.
  • Avoid sharing sensitive personal and financial information.
  • Avoid employment offers that require a deposit or investment.

To learn more, visit the Safety Center or click here to report this listing.

More About this Listing: Data Engineer - Python/PySpark (5-8 yrs) Anywhere in India/Multiple Locations
Data Engineer - Python/PySpark (5-8 yrs) Anywhere in India/Multiple Locations is a Engineering Engineer Job at Circuit Compilers located in India. Find other listings like Data Engineer - Python/PySpark (5-8 yrs) Anywhere in India/Multiple Locations by searching Oodle for Engineering Engineer Jobs.