Big Data Engineer

Serving the marketing and advertising industry. We bring together real time survey data and insights on millions of consumers. We provide this insightful data about online behavior all in one platform.

As an engineering team member for the big data team, you will be responsible for the technical direction and development of critical data ingestion pipelines that power the business. You will be able to provide support and technical direction to a team of highly skilled data engineers and facilitate interaction with the technical product management team members. You will be a member of a high-quality, dedicated and friendly team passionate about delivering quality products, solving hard problems, and transforming the data and insights industry.
Responsibilities

• Collaborate in the design, planning and development of products and initiatives to deliver on product requests for the data engineering squad
• Oversee production, quality assurance, quality control, testing and maintenance as required.
• Bachelor’s degree in Computer Science, Computer Engineering, or equivalent
• 7+ years experience professional experience in software engineering, data engineering, or a related field
• 3+ years of experience in Spark/Scala DataFrame/Dataset APIs
• Prior experience with debugging and problem solving in Big data ecosystem
• Proven experience in tuning multi terra byte / Peta byte scale Spark Applications
• Prior relational database experience is required
• Deep knowledge of software architecture and engineering standard methodologies, especially modern cloud computing stacks for processing Big Data.
• Experience with AWS as a cloud provider, Spark, Kafka, Hadoop, Elastic Stack, Java, Docker for containers, etc.
• Strong understanding of the different parts of the software development lifecycle, from exploration and design to delivery to production
• Good understanding of principles of solution architecture, technical design, data structures, and data modeling
• Prior experience with Non-functional requirement is required
• Excellent communication skills
• Positive attitude and excellent teamwork

Preferred Qualifications

• Demonstrated success in implementing scaled high cardinality data system
• Experience in using probabilistic data structures
• Love for open-source technologies