Big Data Software Engineer
Gracenote, a Nielsen Company is the leading provider of entertainment metadata and media recognition technology that powers discovery features for top TV, music, sports and automotive platforms.
We are presently looking for a Junior or Senior Software Engineer in our Big Data team to help ingest and process and consolidate video metadata from all around the world.
Tech Stack: Scala, Apache Spark, Hadoop, Pig or Hive, Apache Ignite, AWS, MapReduce, Java
If you love Scala and/or Spark or want to learn it, this job is for you. Whether you’ve been using it for years or you are looking to learn the language/framework combination that is taking the big data world by storm, we want to hear from you. We encourage using the latest technologies to solve problems and aren’t afraid of failure when things don’t work as expected.
We are building the next generation metadata processing framework that collects video metadata from around the world and consolidates into one universal datastore that is queried by the largest video companies in the world.
This team is comprised of Engineers with experience building profiles, analytics, and big data platforms for leading web consumer and open source companies in the Bay Area.
Top reasons to work for us:
Significant opportunity to work with all pieces of big data technology stack
Weekly coding challenges to help hone your skills
Real influence on entertainment experience of hundreds of millions of people all around the world
Open culture where fun, collaboration and happiness are key
Profitable company, plus stability and security of a larger parent company
Fun activities – we’re in media and entertainment business
Work environment that promotes work/life balance
What You Will Be Doing:
Developing an ingestion and processing platform for terabytes of video metadata using a variety of tools such as Spark, Spark Streaming, Kafka, Cassandra, and Apache Ignite
Figuring out ways to speed up our processing pipeline using in memory RDD databases like Apache Ignite
Brainstorming ideas to potentially store all the world’s video, sports, and music data in an entertainment graph
Operating and scaling our infrastructure in AWS
3+ years experience in Java or Scala development
3+ years development with MapReduce, Apache Hadoop, and Apache Spark
Understanding of data flows, data architecture, ETL and processing of structured and unstructured data
Desire to continually work to improve as a software engineer and architect
Ability to work as a part of a team and self-motivated
BS in computer science, mathematics, or equivalent work experience, Master’s preferred
Nice to have skills:
Apache Ignite, Spark GraphX, Spark MLLib, Apache Storm
Familiar with data mining concepts, machine learning algorithms and basic statistics