The mission of our data team at Red Ventures to make data easy-to-use for everyone. We are hiring Senior Spark Engineers who are excited to overcome challenges, work with the latest AWS technologies, and continue making our data better.
Our Spark Engineers are a key part of building our data infrastructure; this includes using Spark for streaming applications and/or as an ETL tool, Data Aggregation, Column Operations, User Defined Functions, Caching and using the Spark UI to analyze behavior and performance.
You will join a team of highly skilled engineers who design, develop and automate high-quality, scalable solutions across the entire data lifecycle, from raw data to powerful insights and analytics. Basically, we are using Spark as our universal program on how we are transferring data across multiple platforms.
If you want to be a part of a dynamic team solving business problems using data, this is the role for you.
- Spark, working in RDDs and DataFrames/Datasets API (with emphasis on DataFrames) to query and perform data manipulation
- Spark Structured Streaming (We process a ton of data in real time)
- Experience building large scale Spark applications, ideally with either Batch processing and/or Streaming processing
- Scala would be ideal but a solid knowledge of Java is also acceptable
- Experience in SparkSQL (Broadcast Joins)
- Experience with cloud computing platforms, we use AWS (Kinesis, S3, Lambda or DynamoDB would also work)
- Has experience with ANSI SQL relational database (Oracle, SQL, Postgres, MySQL)
Nice to have:
- Linux common working knowledge, including navigating through the file system and simple bash scripting
- General knowledge of distributed systems and distributed data processing frameworks
- Experience with Storm, Kafka, or Cassandra is a plus
- Knowledge about agile software processes
About Red Ventures:
Red Ventures is a multi-billion-dollar portfolio of digital companies that specialize in bringing consumers and brands together. Through bespoke technology, integrated digital commerce and sales, distinguished partnerships, data science, and original content from the company’s proprietary brands and marketplaces, Red Ventures provides better end-to-end consumer experiences throughout the buying cycle. Headquartered in the Charlotte metro area, Red Ventures has more than 3,600 employees globally in offices across the US, UK, and Brazil.