Senior Data Engineer, Top Location Intelligence Company

Location: New York, NY
Date Posted: 09-11-2017
One of the top location intelligence companies - a leading force in changing how location information enriches our real-world and digital lives - is looking for a talented Senior Data Engineer to join their renowned Enterprise Engineering team. You will be responsible for creating and maintaining the offline pipeline that wrangles their raw data into SaaS products.
Ready to bring new ideas to the table? Love to dig into the data to find the source of a problem or validate an assumption? Are you someone who can quickly understand, discuss, and optimize the performance characteristics of a complex offline data pipeline? They want someone who can: 
  • Maintain and improve their data pipelines using Hadoop, Scalding, Luigi, Spark, Mongo and more
  • Partner with the Data Science team to investigate and implement advanced statistical models and machine learning pipelines
  • Identify and implement performance improvements across all pipelines
  • Data investigations to validate assumptions or find the source of a problem
  • 3+ years of proven experience working with Hadoop MapReduce and/or other big data technologies and pipelines
  • You consider yourself both a Data Scientist and a Senior Developer, and are just as happy working on challenging data problems as you are tinkering with the cluster
  • You have a solid foundation in computer science fundamentals with particular expertise in data structures, algorithms, and design
  • You obsess over data:  everything needs to be accounted for and be thoroughly tested
  • You are constantly thinking of ways to squeeze better performance out of the pipelines
  • Strong Java or other object-oriented programming experience or, even better, experience and/or interest in functional languages (we use Scala!)
  • Experience with Scala, Scalding, Luigi, Hive, machine learning pipelines and model training is a plus
  • Bachelors Degree or higher in Computer Science, Electrical Engineering or related field
this job portal is powered by CATS