Data Engineer - Spark

Data Engineer - Spark
Charlotte, NC

The mission of our data team at Red Ventures to make data easy-to-use for everyone. We are hiring Spark Engineers who are excited to overcome challenges, work with the latest AWS technologies, and continue making our data better.

Our Spark Engineer will be a key part of building our data infrastructure; this includes using Spark for streaming applications and/or as an ETL tool, Data Aggregation, Column Operations, User Defined Functions, Caching and using the Spark UI to analyze behavior and performance.

You will join a team of highly skilled engineers who design, develop and automate high-quality, scalable solutions across the entire data lifecycle, from raw data to powerful insights and analytics. Basically, we are using Spark as our universal program on how we are transferring data across multiple platforms.

If you want to be a part of a dynamic team solving business problems using data, this is the role for you.

Skills:

  • Spark, working in RDDs and DataFrames/Datasets API (with emphasis on DataFrames) to query and perform data manipulation
  • Spark Structured Streaming
  • Experience building large scale Spark applications, ideally with either Batch processing and/or Streaming processing
  • Scala would be ideal but a solid knowledge of Java is also acceptable
  • Experience in SparkSQL (Broadcast Joins)
  • Experience with cloud computing platforms, we use AWS (Kinesis, S3, Lambda, DynamoDB)
  • Has experience with ANSI SQL relational database (Oracle, SQL, Postgres, MySQL)

Nice to haves:

  • Linux common working knowledge, including navigating through the file system and simple bash scripting
  • General knowledge of distributed systems and distributed data processing frameworks
  • Experience with Storm or Cassandra is a plus
  • Knowledge about agile software processes

 

About Red Ventures:

Red Ventures is a multi-billion-dollar portfolio of digital companies that specializes in bringing consumers and brands together. Through bespoke technology, integrated digital commerce and sales, distinguished partnerships, data science, and original content from the company’s proprietary brands and marketplaces, Red Ventures provides better end-to-end consumer experiences throughout the buying cycle. Headquartered in the Charlotte metro area, Red Ventures has more than 3,600 employees globally in offices across the US, UK, and Brazil.

 

Expand Description
Resume/CV
Cover Letter

Education History

Please tell us more about your educational history.

Add Education History

HEADQUARTERS
1101 Red Ventures Drive
Fort Mill, SC 29707
704.971.2300