Whizz HR
This listing was posted on hirist.
Python Web Scraping Engineer (4-6 yrs) Bangalore/Hyderabad (Analytics & Data
Location:
Bangalore/Hyderabad
Description:
We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts, extracting data from various sources, and delivering high-quality datasets for analysis and decision-making.Responsibilities :Web Scraping Development : - Design, develop, and maintain web scraping scripts and automation tools using Python.- Write efficient and scalable code to extract data from websites, APIs, databases, and other online sources.Data Extraction : - Implement data extraction techniques to retrieve structured and unstructured data from HTML pages, JSON responses, XML documents, and other formats.- Handle pagination, AJAX requests, and dynamic content rendering.Data Transformation : - Clean, preprocess, and transform raw data into usable formats for analysis and visualization.- Perform data validation, deduplication, normalization, and enrichment as needed.Proxy Rotation and CAPTCHA Handling : - Implement strategies for proxy rotation and CAPTCHA handling to bypass rate limits, IP blocks, and other anti-scraping measures.- Use proxy servers, user-agent rotation, and CAPTCHA solvers to ensure uninterrupted scraping.Scalability and Performance Optimization : - Optimize web scraping scripts for performance, scalability, and efficiency.- Implement asynchronous and parallel processing techniques to improve speed and throughput.Error Handling and Logging : - Handle errors, exceptions, and edge cases gracefully in web scraping scripts.- Implement logging and monitoring solutions to track scraping activities, detect anomalies, and troubleshoot issues.Compliance and Legal Considerations : - Ensure compliance with website terms of service, copyright laws, and data privacy regulations while scraping data.- Respect robots txt rules and obtain necessary permissions for scraping restricted or sensitive content.Documentation and Collaboration : - Document technical specifications, data schemas, and scraping workflows.- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders.Qualifications :- Educational Qualifications: Bachelor's/Master's- 4 to 6 years of experience in web scraping and data extraction using Python.- Proficiency in web scraping libraries and frameworks such as BeautifulSoup, Scrapy, Selenium, or Requests.- Strong understanding of HTML, CSS, XPath, JSON, and regular expressions for data parsing and extraction.- Experience with database systems such as MySQL, PostgreSQL, MongoDB, or SQLite for storing and managing scraped data.- Familiarity with proxy servers, CAPTCHA solving services, and IP rotation techniques for bypassing anti-scraping measures.- Excellent problem-solving, analytical, and communication skills. (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: Python Web Scraping Engineer (4-6 yrs) Bangalore/Hyderabad (Analytics & Data
Python Web Scraping Engineer (4-6 yrs) Bangalore/Hyderabad (Analytics & Data is a Engineering Engineer Job at Whizz HR located in India. Find other listings like Python Web Scraping Engineer (4-6 yrs) Bangalore/Hyderabad (Analytics & Data by searching Oodle for Engineering Engineer Jobs.
Python Web Scraping Engineer (4-6 yrs) Bangalore/Hyderabad (Analytics & Data is a Engineering Engineer Job at Whizz HR located in India. Find other listings like Python Web Scraping Engineer (4-6 yrs) Bangalore/Hyderabad (Analytics & Data by searching Oodle for Engineering Engineer Jobs.