KARYA CONSULTANTS PRIVATE LIMITED
This listing was posted on hirist.
ML Data Engineer - Pandas/Numpy (4-6 yrs) Bangalore (Analytics & Data Science)
Location:
Bangalore
Description:
Responsibilities : - Designing, developing, and executing data pipelines to ingest, preprocess, and transform data for Generative AI model training and inference.- Proficiency in data manipulation and preprocessing using tools like NumPy, Pandas, or SQL.- Familiarity with big data technologies such as Hadoop and Spark for processing and analyzing large-scale datasets.- Designing and implementing data pipelines for Generative AI projects by utilizing various technologies including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, LangChain, LlamaIndex, Open AI functions, AWS Functions, Redshift, and SSIS.- This involves integrating these tools logically and efficiently to create seamless, high-performance data flows supporting the data requirements of our AI initiatives.- Collaborating with data scientists, AI researchers, and other stakeholders to understand data requirements and translate them into effective data engineering solutions.- Demonstrating familiarity with data integration services like AWS Glue and Azure Data Factory, effectively utilizing these platforms for seamless data ingestion, transformation, and orchestration across various sources and destinations.- Proficiency in constructing data warehouses and data lakes, organizing and consolidating large volumes of structured and unstructured data for efficient storage, retrieval, and analysis.- Implementing data security and governance policies to ensure the privacy and integrity of sensitive data used in Generative AI projects.- Monitoring and optimizing data pipelines for performance, scalability, and cost-effectiveness.- Staying updated on the latest advancements in data engineering tools and technologies (e.g. Apache Spark, Airflow, Snowflake, Data Bricks) and applying them to our Generative AI platform.- Effectively communicating with technical and non-technical stakeholders about data quality and availability for Generative AI projects.Qualifications : Minimum Qualifications : - Bachelor's degree in computer science, Data Science, Statistics, or a related field, or equivalent experience.- Experience in data engineering or related roles such as data pipeline development, data storage, or ETL/ELT processes.- Proven experience in building and maintaining data pipelines for machine learning projects.- Strong understanding of data modeling principles, data quality measures, and data security best practices.- Proficiency in programming languages like Python, SQL, and scripting languages (e.g. Bash, Shell).- Familiarity with cloud platforms (e.g. AWS, Azure) for data storage and processing.- Excellent communication, collaboration, and problem-solving skills.- Ability to work independently and as part of a team (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: ML Data Engineer - Pandas/Numpy (4-6 yrs) Bangalore (Analytics & Data Science)
ML Data Engineer - Pandas/Numpy (4-6 yrs) Bangalore (Analytics & Data Science) is a Engineering Engineer Job at Karya Consultants located in India. Find other listings like ML Data Engineer - Pandas/Numpy (4-6 yrs) Bangalore (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.
ML Data Engineer - Pandas/Numpy (4-6 yrs) Bangalore (Analytics & Data Science) is a Engineering Engineer Job at Karya Consultants located in India. Find other listings like ML Data Engineer - Pandas/Numpy (4-6 yrs) Bangalore (Analytics & Data Science) by searching Oodle for Engineering Engineer Jobs.