Techno Wise
This listing was posted on hirist.
Big Data Engineer - PySpark (4-8 yrs) Mumbai/Bangalore/Pune (Analytics & Data
Location:
Mumbai/Bangalore/Pune
Description:
Job Role : Big Data EngineerRelevant Experience : 3+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a pivotal role in designing, implementing, and maintaining scalable data solutions leveraging the Big Data technologies like PySpark. This role requires a strong understanding of big data technologies, data engineering best practices, and the ability to collaborate effectively with cross-functional teams.Responsibilities :1. Design, develop, and implement end-to-end data pipelines using PySpark to process, cleanse, and transform large-scale datasets.2. Collaborate with data scientists, analysts, and stakeholders to understand data requirements and translate them into technical solutions.3. Optimize and tune Spark jobs for performance and scalability, ensuring efficient resource utilization and processing speed.4. Develop and maintain data ingestion processes from various sources such as databases, streams, and file systems into the Databricks environment.5. Implement data quality checks and validation processes to ensure the accuracy, completeness, and reliability of the data.6. Have worked on Data Orchestration Tools like Airflow.7. Working on storytask using Agile Methodology.8. Must have hand-on on GitHub and GitLab9. Work closely with DevOps teams to automate deployment processes and ensure the reliability and availability of data pipelines.10. Monitor, troubleshoot, and resolve issues related to data pipeline failures, performance degradation, and system errors.11. Stay updated on emerging technologies and industry trends in big data, distributed computing, and cloud platforms to drive continuous improvement and innovation.12. Implementing Data Solutions by using Azure Services like Azure Data Factory.13. Creating Data Pipelines using Notebooks using different Clusters in Databricks.Qualifications :1. Bachelor's or Master's degree in Computer Science, Engineering, or a related field.2. Proven experience as a Big Data Engineer with expertise in building and optimizing data pipelines using PySpark.3. Strong programming skills in Python and proficiency with SQL for data manipulation and analysis.4. Familiarity with distributed computing frameworks like Hadoop, Scala.5. Hands-on experience with cloud platforms such as Azure or AWS.6. Working experience on Databricks7. Excellent problem-solving skills and the ability to work independently as well as collaboratively in a fast-paced environment.8. Effective communication skills with the ability to explain complex technical concepts to non-technical stakeholders.Benefits You'll Get :- Unlimited opportunities to learn on our multiple Training Platforms- Certifications Reimbursement- Flexibility- Opportunity to work on multiple technologies- Medical Coverage &Life Insurance- Company Events and Outings- Tech Thursdays and Fun Fridays- 5 days working- Work-Fun Environment (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Big Data Engineer - PySpark (4-8 yrs) Mumbai/Bangalore/Pune (Analytics & Data
Big Data Engineer - PySpark (4-8 yrs) Mumbai/Bangalore/Pune (Analytics & Data is a Engineering Engineer Job at Techno Wise located in India. Find other listings like Big Data Engineer - PySpark (4-8 yrs) Mumbai/Bangalore/Pune (Analytics & Data by searching Oodle for Engineering Engineer Jobs.
Big Data Engineer - PySpark (4-8 yrs) Mumbai/Bangalore/Pune (Analytics & Data is a Engineering Engineer Job at Techno Wise located in India. Find other listings like Big Data Engineer - PySpark (4-8 yrs) Mumbai/Bangalore/Pune (Analytics & Data by searching Oodle for Engineering Engineer Jobs.