Senior Software Engineer - Crawler
Fuld tid
ZoomInfo
Responsibilities
- Design and implement scalable, fault-tolerant web crawling and extraction pipeline components.
- Write clean, production-grade Java and Python code.
- Build and operate large-scale ETL/ELT pipelines for data extraction and transformation.
- Work with GCP and AWS cloud infrastructure, primarily on GKE.
- Improve observability, reliability, and operational excellence across contributed systems.
- Partner with product and data science teams to deliver solutions.
- Contribute to code reviews, documentation, and knowledge sharing.
- Keep current with web technologies, anti-crawling mechanisms, and AI-powered extraction approaches.
Requirements
- At least 5 years of professional software engineering experience building production systems.
- Strong knowledge of algorithms, data structures, concurrency, and distributed systems.
- Proficiency in Java and/or Python.
- Experience owning features end-to-end from design through deployment and operation.
- Experience making component-level architectural decisions.
- Hands-on experience with cloud data warehouses such as BigQuery or Snowflake.
- Experience designing and operating large-scale ETL/ELT pipelines.
- Experience with orchestration tools such as Apache Airflow.
- Experience with streaming or event-driven systems such as Apache Kafka.
- Production experience on GCP or AWS, with GCP preferred.
- Hands-on experience with Kubernetes, including GKE or EKS, for distributed workloads.
- Familiarity with infrastructure-as-code tools such as Terraform.
- Strong communication, pragmatic problem-solving, adaptability, and self-direction.
- Web crawling at scale, Scrapy, proxy infrastructure, anti-bot evasion, SERP extraction, AI/LLM-based extraction, or B2B data-company experience is preferred.
Benefits
- Remote work arrangement is indicated by the LI-REMOTE designation.
- Comprehensive benefits are offered, along with holistic mind, body, and lifestyle well-being programs.
Ledig stilling opslået Indrykket for 9 timer siden
Tilsvarende jobs, som kan have interesseBaseret på Senior Software Engineer - Crawler vedrørende den ledige stilling i Fjernarbejde
- ...India) creates quality solutions for its customers by combining software, back office, and knowledge processing operations to fulfill... ...Summary We are seeking a highly motivated and skilled Software Engineer with 5–8 years of professional experience in designing,...
500 $ om året
...About the team: Builder.io's Publish engineering team builds the platform that empowers marketing and content teams to launch pages... ...and AI-driven automation, this is the team for you. As a Senior Software Engineer at Builder.io, you will… Design and build the APIs...- ...cover and their potential bottlenecks Work with deployment engineers to ensure that what you build goes as quickly as possible to the... ...science or related field ~3+ years of experience as a Software Engineer (Backend) ~2+ years of experience using TypeScript...
- ...excellence matter every day. We're looking for a hands-on Senior Corporate IT Engineer to own and scale the technology that supports our workforce... ...About the Role House Rx is hiring a Senior Security Software Engineer to help us build, review, and harden security-sensitive...
- ..., and PCI 4.0—that protects the infrastructure and customer data behind our industry-leading Voice AI API. We're hiring a Senior Software Engineer in the security space to join our IT & Security team as the company's first security engineering role. This role sits at...
- ...Summary The Wikimedia Foundation is seeking a Senior Software Engineer to join the team supporting the Wikidata Platform — the structured data backbone of Wikimedia projects and a key part of the global open knowledge ecosystem. You’ll help scale and sustain the...
- ...subscriptions at scale, combining the agility of a high-growth business with the backing of a global organization. As the Senior Software Engineer, Vendor Integrations, you will be responsible for building and evolving the CloudBlue platform, a microservices-based SaaS...
- ...At Cloudbeds, we're not just building software, we’re transforming hospitality. Our intelligently... ...How You'll Make an Impact: As a Senior Software Engineer on the POS team, you'll work directly... ...-first applications, synchronization engines, or conflict-resolution workflows ~...
- ...EVs, heat pumps, and home batteries. The missing piece is software that can connect and orchestrate these devices at scale.... ...energy transition. What we're looking for: We're hiring a Senior Software Engineer to help build the software layer that makes home energy...
- ...EVs, heat pumps, and home batteries. The missing piece is software that can connect and orchestrate these devices at scale.... ...energy transition. What we're looking for: We’re hiring a Senior Backend Engineer to make Enode’s EV platform more reliable, observable, and...
- ...Senior Software Engineer, MediaWiki Summary The Wikimedia Foundation is looking for a Senior Software Engineer to join the MediaWiki Platform team. MediaWiki is the open-source platform that powers Wikipedia and its sister projects - impacting hundreds of millions...
800 $
...PublicInput PublicInput develops and supports Software-as-a-Service (Saas) solutions designed... ...a frontend-leaning Full Stack Software Engineer to help build and improve the... ...feedback and insights with support from senior engineers. Develop an understanding of how...- ...won’t just be writing code; you’ll be shaping the future. As a Software Engineer at Xapo, you will design, build, and deploy high-impact... ...design to production operations, all while being mentored by senior engineers. You could join one of our dynamic, business domains...
- ...30%+ at ~$2B ARR is that we need great engineers everywhere, and we have the room to shape... ...NYSE: IOT) sits at the center of hardware, software, AI, and the physical world. The... ...is broad and flexible: we're hiring from Senior through Principal, across platform and product...
- ...Responsibilities Own delivery of high-quality software projects and coordinate with teammates and stakeholders to meet agreed timelines... ...ambiguity, and drive projects forward. Participate in engineering recruiting through interviewing, problem specification, and role...
2000 $ om året
...across PC and IoT technologies. Our teams partner with specialist engineers from major silicon companies to integrate next-generation... ...integration for the latest IoT and server-class hardware platforms and software stacks Work with partners to deliver a delightful,...- ...and working with brilliant people. You are an accomplished senior Python programmer that enjoys challenging projects in mission-critical... ...one of the industry's pioneers in open source with intelligent engineers at every level from engineer to CTO and CEO level. At Canonical...
- ...with customers and subject-matter experts, and take ownership of product outcomes. Requirements Four or more years of software engineering experience, most of it in frontend work, at companies with strong engineering practices. Strong frontend capability in either...
- ...Responsibilities Own delivery of high-quality software projects and coordinate with teammates and stakeholders on timelines. Contribute... ...ambiguity, and drive projects forward. Participate in engineering recruiting through interviewing, problem specification, and...
210000 $ - 260000 $ om året
...That's why you can do here what you can never do at a legacy software company. move fast and prove it, not theorize it. We think from... ...operates. What You'll Be Doing : Lead a small team of engineers, providing technical mentorship, code review, and...- ...our platform scales globally, we're looking for people who can turn complex systems into practical, working solutions. As a Software Engineer in Partner Engineering, you’ll play a pivotal role in shaping technical solutions that directly impact our top-tier partners....
44000 € - 60000 € om året
...We’re looking for a Software Engineer who would contribute to PVcase software and would not be afraid of heights (at least career ones)! You will play an important role in our exciting journey while accelerating the global transition to clean energy. Participate and...- ...for task authoring, expert annotation, review, quality control, and structured data collection. Collaborate with researchers and engineers to ensure evaluation data and published outputs are accurate, consistent, and integrated. Requirements ~4+ years of...
2000 $ om året
...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading...2000 $ om året
...This is the general track for Engineering Director at Canonical, apply here if you are confident... ...staff engineers. Canonical’s largest software products are built by multiple teams... ...of the result. Even though this is a senior management position, managing managers,...- ...Makes Cloudbeds Unique At Cloudbeds, we're not just building software, we’re transforming hospitality. Our intelligently designed... ...(Europe) How You'll Make an Impact: As a Staff Software Engineer in Revenue Intelligence, you will shape the technical direction...
2000 $ om året
...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading...- ...distinguish model limitations from infrastructure, harness, or environment failures. Requirements ~3–5+ years of professional software engineering experience, particularly in backend, infrastructure, platform, SRE, or distributed systems engineering. ~ Strong...
2000 $ om året
...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading...2000 $ om året
...Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform... ...initiatives such as public cloud, data science, AI, engineering innovation and IoT. Our customers include the world's leading...
Vil du modtage flere ledige stillinger?
Abonnér og modtag tilsvarende ledige stillinger for Senior Software Engineer - Crawler. Bliv den første ansøger!
