![](https://i.oodleimg.com/a/unverified_userphoto.png)
Tech Recruitz Solutions
This listing was posted on hirist.
DevOps Engineer - AWS Cloud Services (3-5 yrs) Anywhere in India/Multiple
Location:
Anywhere in India/Multiple Locations
Description:
RESPONSIBILITIES :- Design and maintain robust web scrapers using Python and Scrapy to handle concurrent scraping tasks on Kubernetes clusters, ensuring the collection of tens of gigabytes of data daily.- Manage and scale Kubernetes environments (K8S/K3S) to support extensive data scraping and processing operations.- Organize, process, and deliver large datasets (in terabytes) using tools like MongoDB, Airflow, and Jupyter Notebook.- Optimize data for AI training by processing video content to remove non-key frames and capturing time-based captions in multiple languages.- Work with AWS cloud services to deploy, monitor, and scale applications.- Ensure the integrity and availability of data, implementing best practices for data security and storage.- Collaborate with cross-functional teams to meet project deadlines and deliverables.REQUIREMENTS :- Proven experience in backend development with a strong command of Python and Node.js.- Expertise in setting up and managing web scrapers on a large scale, particularly with Scrapy and Airflow.- Strong experience with Kubernetes and managing applications in a clustered environment.- Familiarity with MongoDB and other NoSQL databases for handling and organizing large data sets.- Proficient with AWS services, capable of efficiently utilizing cloud resources.- Experience with media scraping and processing, especially with video data for AI applications.- Ability to work collaboratively in a fast-paced and evolving environment.- Excellent problem-solving skills and attention to detail.- Available full-time during US work hours. (ref:hirist.tech)
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to hirist
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.
More About this Listing: DevOps Engineer - AWS Cloud Services (3-5 yrs) Anywhere in India/Multiple
DevOps Engineer - AWS Cloud Services (3-5 yrs) Anywhere in India/Multiple is a Engineering Engineer Job at Tech Recruitz Solutions located in India. Find other listings like DevOps Engineer - AWS Cloud Services (3-5 yrs) Anywhere in India/Multiple by searching Oodle for Engineering Engineer Jobs.
DevOps Engineer - AWS Cloud Services (3-5 yrs) Anywhere in India/Multiple is a Engineering Engineer Job at Tech Recruitz Solutions located in India. Find other listings like DevOps Engineer - AWS Cloud Services (3-5 yrs) Anywhere in India/Multiple by searching Oodle for Engineering Engineer Jobs.