Back
India   India   Development Engineer   Marketscope -

Lead/Software Development Engineer - Data Engineering (4-10 yrs) Hyderabad | Development Engineer,1

MarketScope

This listing was posted on hirist.

Lead/Software Development Engineer - Data Engineering (4-10 yrs) Hyderabad

Share
Tools
Location:
Hyderabad
Description:

Key Skills : - Spark Core/Streaming with Any programming language Scala/ Java/ Python- Good Understanding of Data Structures, Algorithms, Logical and problem solving skills- Data Transformation, Data Ingestion, Optimization mechanism/techniques- Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies, AWS, Postgresql.Education : Masters/Bachelors degree in Computer Science, Statistics, Engineering or a related technical discipline will be preferredJob Description :We deal with huge amounts of data at a massive scale, so we are looking for engineers who love solving challenging problems through conducting independent research and collaborating with teams across our product teams to help improve the overall product experience.Key Qualifications :- Working experience in Big Data platforms and building large-scale distributed systems with high availability.- Experience in developing Spark Applications using Spark RDD API, Spark-SQL, Spark GraphX API, Spark Streaming API, Spark Mlib API, and Data frames APIs- Should have broad knowledge of Spark Advantages, Spark Workflows, How to write Spark Jobs, Spark Workflows, Spark query tuning and performance optimization.- Solid in Data Structures, Algorithms basics and Should have good hands-on experience with any - one programming language (Scala/ Java8 - 1st Preference OR Python - 2nd Preference OR Java - 3rd preference) and Strong investigative and problem-solving skills- Data Ingestion, Optimization Techniques, Data Transformation and aggregation pipeline design/development knowledge is required.- Experience working on cutting-edge Big Data storage systems and technologies like Hadoop, HDFS, AWS S3, AWS Lambda, Storm/Heron, Cassandra, Apache Kafka, Solr/ElasticSearch, MongoDB, DynamoDB, Postgres, and/or MySQL etc.Roles & Responsibilities :- Create new, and maintain existing Scala/Spark jobs for data transformation and aggregation from simple to Complex Data transformations involving structured & unstructured data.- Produce unit tests for Spark transformations and helper, performance optimization methods.- Develop data processing pipelines, data storage, and management architecture.- Define scalable calculation logic for interactive and batch use cases- Interact with infrastructure and data teams to produce complex analysis across data- You'll be working on a unique and challenging big data ecosystem with focuses on storage efficiency, data security, and privacy, scalable and performant queries, expandability and flexibility, etc, and with the goal to help better measure the quality of map data.- You will work with engineers to build a big data platform that processes and manages Exabytes of data and enables efficient access to those data. (ref:hirist.tech)
Education/experience:
2 To 5 Years
Company:
Marketscope
Posted:
May 4 on hirist
Share
Tools
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
  • Always meet the employer in person.
  • Avoid sharing sensitive personal and financial information.
  • Avoid employment offers that require a deposit or investment.

To learn more, visit the Safety Center or click here to report this listing.

More About this Listing: Lead/Software Development Engineer - Data Engineering (4-10 yrs) Hyderabad
Lead/Software Development Engineer - Data Engineering (4-10 yrs) Hyderabad is a Computer & Software Jobs Development Engineer, Software Developer Job at Marketscope located in India. Find other listings like Lead/Software Development Engineer - Data Engineering (4-10 yrs) Hyderabad by searching Oodle for Computer & Software Jobs Development Engineer, Software Developer Jobs.