AI Palette
This listing was posted on hirist.
AI Palette - Data Engineering Manager - Web Scraping & Crawling (7-10 yrs)
Location:
Bangalore
Description:
We are Ai Palette : We think the world would be a nicer place to be if Food Companies could create products that the consumers really want. So we are making it happen. We want to be the most preferred Food AI company in the world. We're making it possible by building an AI-powered SaaS platform based on our founders' experience in the Food Industry & AI.We are a Series A round funded company with experienced investors like pi Ventures, Exfinity ventures and Anthill ventures. We are building a global company and are already working with customers across the globe. Our customers include Fortune 500 companies and FMCG brands.We have won several accolades & awards in our short period of existence such as:Top 10 Global Food & Retail Tech startup at Kickstart Innovation'20 at SwitzerlandTop 15 Global Food tech Startup at Slingshot'19 at SingaporeTop 200 Global startups at HKSTP Epic'20 at HongKongTop 500 Global Food Tech startups by Forward FoodingWHAT'S IT LIKE TO WORK AT AI PALETTE? We're a growing technology startup headquartered in Singapore with our Engineering base in Bangalore and a team in US, We PaletteeRs are a highly passionate and motivated bunch of people that help each other do remarkable things and achieve extraordinary results every single day. We are active learners, have a positive impact on consumers' lives and settle for nothing short of excellence. We face challenges together and we win together. In our vision to build the World's First AI Platform for New Product Innovation , there isn't a day that goes by where we don't have "Aha" moments. We strive together to deliver world-class solutions that transform the way consumer products of today and tomorrow will be created.Responsibilities:- Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and develop scalable data crawling solutions.- Design, develop, and maintain data crawling pipelines, ensuring efficient and timely acquisition of data from various sources.- Evaluate and implement appropriate data crawling technologies and tools to optimize the crawling process and ensure data quality and integrity.- Develop and enforce data engineering best practices, standards, and processes related to data crawling.- Identify and resolve issues related to data crawling, such as handling complex data structures, mitigating crawling bottlenecks, and addressing website-specific challenges.- Collaborate with stakeholders to define data engineering project requirements, timelines, and deliverables related to data crawling.- Perform data extraction, transformation, and loading (ETL) tasks to convert crawled data into usable formats for downstream analysis and processing.- Monitor data crawling performance and implement mechanisms to ensure the reliability and scalability of crawling pipelines.- Stay up to date with the latest trends and advancements in data crawling techniques, web scraping frameworks, and related technologies. - Experience in Data ModellingRequirements:- A minimum of 2-3 years of experience in leading and managing teams of data engineers, Lead and manage a team of 4-6 engineers- Bachelor's or master's degree in computer science, data engineering, or a related field.- Proven experience (7+ years) working as a data engineer, with a specialisation in data crawling and web scraping.- Strong programming skills in languages such as Python, Java, or Scala, with expertise in web scraping frameworks like Scrapy, Beautiful Soup, or Selenium.- Solid understanding of web protocols (HTTP, HTTPS), HTML, CSS, and JavaScript to effectively crawl and extract data from websites.- Experience with distributed crawling frameworks such as Apache Nutch or Apache Storm is a plus.- Proficiency in SQL and database technologies (e.g., PostgreSQL, MySQL, or Oracle) for data storage and retrieval.- Familiarity with cloud platforms (e.g., AWS, Azure, or Google Cloud) and related data services for scalable and reliable data crawling.- Knowledge of data modeling, data warehousing, and ETL processes.- Strong analytical and problem-solving skills, with a focus on data quality and accuracy.- Excellent leadership and team management abilities, with a proven track record of leading data engineering teams.- Effective communication and collaboration skills, with the ability to explain complex technical concepts to non-technical stakeholders.Benefits:- Great progression opportunities - we want you to grow with us.- Look after yourself with health insurance including Hospital/Surgical.- Learn new skills with sponsored training on MOOCs such as Coursera, Udemy. EQUAL OPPORTUNITY: We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: AI Palette - Data Engineering Manager - Web Scraping & Crawling (7-10 yrs)
AI Palette - Data Engineering Manager - Web Scraping & Crawling (7-10 yrs) is a Engineering Engineering Manager Job at AI Palette located in India. Find other listings like AI Palette - Data Engineering Manager - Web Scraping & Crawling (7-10 yrs) by searching Oodle for Engineering Engineering Manager Jobs.
AI Palette - Data Engineering Manager - Web Scraping & Crawling (7-10 yrs) is a Engineering Engineering Manager Job at AI Palette located in India. Find other listings like AI Palette - Data Engineering Manager - Web Scraping & Crawling (7-10 yrs) by searching Oodle for Engineering Engineering Manager Jobs.