RapidBraiins
This listing was posted on hirist.
PySpark Architect - Solution Design & Development (12-15 yrs)
Location:
Chennai/Kolkata/Pune
Description:
Job Description- As a PySpark Architect, you will be responsible for architecting, designing, and implementing large-scale data processing systems using PySpark.- You will work closely with cross-functional teams to develop robust, scalable, and efficient data solutions.- Your deep experience in big data technologies and hands-on expertise in PySpark will be critical in driving our data initiatives forward.Key Responsibilities :Architectural Leadership :- Lead the architecture, design, and implementation of large-scale data processing applications using PySpark.- Provide technical guidance and mentorship to development teams on best practices for data processing and PySpark development.Solution Design and Development :- Design and implement end-to-end data pipelines to ingest, process, and analyze large volumes of data.- Develop scalable and efficient PySpark applications to handle complex data transformations and aggregations.- Ensure the solutions are designed for high availability, scalability, and performance.Optimization and Performance Tuning :- Optimize PySpark applications for performance and efficiency.- Perform code reviews and provide recommendations for improvements.- Identify and resolve performance bottlenecks in data processing workflows.Collaboration and Stakeholder Management :- Work closely with data engineers, data scientists, and other stakeholders to understand their requirements and translate them into technical solutions.- Collaborate with infrastructure teams to ensure the deployment and scaling of data applications.Documentation and Standards :- Maintain comprehensive documentation of the architecture, design, and implementation details.- Establish and enforce coding standards, best practices, and development methodologies.Continuous Improvement :- Stay updated with the latest trends and advancements in big data technologies.- Identify opportunities for continuous improvement in data processing and analytics solutions.Required Qualifications:- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.- 12 to 15 years of experience in software development with at least 5 years of experience in big data technologies.- Extensive hands-on experience with PySpark and the Spark ecosystem.- Strong expertise in designing and implementing large-scale data processing systems.- Proficiency in Python programming and related libraries for data processing.- Solid understanding of distributed computing principles and big data architectures.- Experience with cloud platforms such as AWS, Azure, or GCP.- Strong knowledge of data warehousing and ETL processes.- Excellent problem-solving and analytical skills.- Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.Preferred Qualifications:- Experience with additional big data technologies such as Hadoop, Hive, or Kafka.- Knowledge of containerization and orchestration tools such as Docker and Kubernetes.- Familiarity with data visualization tools and techniques.- Certification in cloud platforms or big data technologies.Job Location: Chennai/Pune/Kolkata (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: PySpark Architect - Solution Design & Development (12-15 yrs)
PySpark Architect - Solution Design & Development (12-15 yrs) is a Architecture & Design Architect Job at Rapidbraiins located in India. Find other listings like PySpark Architect - Solution Design & Development (12-15 yrs) by searching Oodle for Architecture & Design Architect Jobs.
PySpark Architect - Solution Design & Development (12-15 yrs) is a Architecture & Design Architect Job at Rapidbraiins located in India. Find other listings like PySpark Architect - Solution Design & Development (12-15 yrs) by searching Oodle for Architecture & Design Architect Jobs.