Robosoft Technologies
This listing was posted on hirist.
Robosoft Technologies - Data Engineer - Scala/Spark (10-12 yrs) Bangalore
Location:
Bangalore
Description:
Roles and Responsibilities :- Develop and automate large scale, high-performance data processing systems (batch and/or streaming) to drive Ola group business growth and improve the product experience.- Evangelize high quality software engineering practices towards building data infrastructure and pipelines at scale.- Lead data engineering projects to ensure pipelines are reliable, efficient, testable, & maintainable- Design our data models for optimal storage and retrieval and to meet critical product and business requirements.- Understand and influence logging to support our data flow, architecting logging best practices where needed- Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across the company- Partner with leadership, engineers, program managers and data scientists to understand data needs.- Educate your partners: Use your data and analytics experience to 'see what's missing', identifying and addressing gaps in their existing logging and processes.- Work with stakeholders to build data lineage, data governance and data cataloging.- Leading projects using agile methodologies.- Communicate effectively with people of all levels in the organization.- Recruit, retain and develop people skills to take bigger responsibilities and challenges.Experience & Skills :- 10+ years of relevant industry experience.- 6+ years of experience in managing a 10 plus members team.- 7+ years experience in custom ETL design, implementation and maintenance.- 5+ years of experience with workflow management engines like Airflow, Dagster etc.- Working knowledge of relational databases and query authoring (SQL).- Experience with Java / Scala / Spark is preferred- Working experience with data at the petabyte scale.- Experience designing, building and operating robust distributed systems.- Experience designing and deploying high performance systems with reliable monitoring and logging practices.- Effectively work across team boundaries to establish overarching data architecture, and provide guidance to individual teams.- Expertise of Amazon Web Services (AWS) and/or other relevant Cloud Infrastructure solutions like Microsoft Azure or Google Cloud.- Experience in managing and deploying containerized environments using Docker, Mesos/Kubernetes is a plus.- Experience in managing projects using scrum methodology.- TechStack : CloudEra Stack, Oozie, Hive, spark, Flink, Spark, K8s, EMR, Presto, Pinot, Trino, IceBerg, FileFormat: parquet, Avro ORC format, DeltaLake, Parquet, Airflow, Druid, Nifi, Hive/HiveQL, Clickhouse, Debezium (CDC) / DBT, Lakehouse, Spark Structured/Streaming, Flink, Apache Beam, Kafka, Hadoop/HbAse/HDFS, Hudi, File Formats (ORC, Parquet, Iceberg), Query engines (Presto, Hue, Trino), Trino or Presto, Stream Processing: Hudi vs iceberg vs delta-lake, Druid vs apache pinot vs trino, Apache Nifi, Apache Flume,Fabric and Secor, Maxwell, K8S, LakeHouse,Educational Qualifications : - Bachelor's or Master's degree in Engineering or related technical discipline (from premier institutes preferred) (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Robosoft Technologies - Data Engineer - Scala/Spark (10-12 yrs) Bangalore
Robosoft Technologies - Data Engineer - Scala/Spark (10-12 yrs) Bangalore is a Engineering Engineer Job at Robosoft Technologies located in India. Find other listings like Robosoft Technologies - Data Engineer - Scala/Spark (10-12 yrs) Bangalore by searching Oodle for Engineering Engineer Jobs.
Robosoft Technologies - Data Engineer - Scala/Spark (10-12 yrs) Bangalore is a Engineering Engineer Job at Robosoft Technologies located in India. Find other listings like Robosoft Technologies - Data Engineer - Scala/Spark (10-12 yrs) Bangalore by searching Oodle for Engineering Engineer Jobs.