People Gamut HR Solutions
This listing was posted on hirist.
Lead Data Engineer - Python/SQL (7-11 yrs) Bangalore (Analytics & Data Science)
Location:
Bangalore
Description:
Responsibilities :- Responsible for Ingesting data from a variety of data sources like CSVs/PDFs/docs/storage containers, streams, and databases, processing it to get meaningful insights.- Develop programs in Python notebooks/.py files & SQL as part of data extraction, cleaning, transformation, and processing.- Rest API's development for sharing data.Primary Responsibilities :- Over 7+ years of experience in Data Engineering- Strong understanding & familiarity with all Azure / Bigdata Ecosystem components.- In-depth knowledge and understanding of Apache Kafka to handle streaming data.- In-depth knowledge and hands-on experience of Azure Data bricks/spark, Spark streaming, and its various core components like GraphX, and MLLib.- Design and develop data ingestion programs to process large data sets in Batch mode using Python, Databricks, and Spark.- Strong understanding of underlying Azure Architectural concepts and distributed computing paradigms.- Hands-on programming experience in Python & OOPS concepts.- Hands-on experience with major components like Airflow / Nifi / ADF.- Experience working with SQL (MySQL/Postgres/SQL Server), NoSQL (Cassandra/Mongo/Cosmos DB) & Graph databases (Neo4j).- Advanced working SQL knowledge to create complex queries.- Hands-on experience with visualization tools like Grafana / kibana / Power BI.- Good understanding of Linux commands, shell scripting, and monitoring Linux-based production machines. Experience working on Azure Cloud services IaaS, PaaS.- Hands-on Experience in working on Microsoft Azure Services like ADLS, Event Hubs, scale sets, Load Balancers, Azure Functions, Logic Apps, Azure Data Factory, etc.- Should have worked on various big data file formats like Avro, ORC, Parquet, and Nested JSON.- Experience in data migration and deployment from On-Prem to Cloud environment and vice-versa.- Individual Contributor. Problem-solving skills.Primary Responsibilities :- Responsible for Ingesting data from a variety of data sources like CSVs/PDFs/docs/storage containers, streams, and databases, processing it to get meaningful insights.- Develop programs in Python notebooks/.py files & SQL as part of data extraction, cleaning, transformation, and processing.- Rest API's development for sharing data.- In-depth knowledge and hands-on experience of Azure Data bricks/spark, Spark streaming, and its various core components like GraphX, and MLLib.- Experience working with SQL (MySQL/Postgres/SQL Server), NoSQL (Cassandra /Mongo/ Cosmos DB) & Graph databases (Neo4j). (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Lead Data Engineer - Python/SQL (7-11 yrs) Bangalore (Analytics & Data Science)
Lead Data Engineer - Python/SQL (7-11 yrs) Bangalore (Analytics & Data Science) is a Engineering Engineer Job at People Gamut HR Solutions located in India. Find other listings like Lead Data Engineer - Python/SQL (7-11 yrs) Bangalore (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.
Lead Data Engineer - Python/SQL (7-11 yrs) Bangalore (Analytics & Data Science) is a Engineering Engineer Job at People Gamut HR Solutions located in India. Find other listings like Lead Data Engineer - Python/SQL (7-11 yrs) Bangalore (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.