View all jobs

Lead Software Engineer (Big Data)

Boston, MA
Our client is seeking a resourceful Software Engineer (Big Data Analytics) to help build the next-generation analytics platform. You'll create our cloud (AWS) based data processing pipeline, data processing infrastructure, and internal APIs. Our platform needs to support multiple types of scalability challenges - from handling terabytes of data to providing analytics in real-time. This is a critical role that will have a significant impact on the  product direction, technology and business.

  • Build data processing pipeline in that collects, connects, centralizes, and curates data from various internal and external data sources
  • Architect scalable and reliable data engineering solutions for moving data efficiently across systems at near real-time
  • Build a distributed data store that will be central source of truth
  • Design, implement, test and deploy data processing infrastructure
  • Research and assess the viability of new processing and data storage technologies
  • Ability to take ownership and get things done in an Agile team setting
  • Help break down, estimate, and provide just-in-time design for small increments of work
  • Big data technologies - Spark, Kinesis, Kafka, Storm or Hadoop
  • Software experience with Java
  • Source control (preferably Git) and bug tracking systems
  • Ability to reason about performance tradeoffs
  • Preferably have experience with SQL and AWS
  • Scala programming a plus
  • Must be comfortable working in an open, highly collaborative team environment
  • Experience with Big Data technologies
  • Software Development experience, including having worked with Java and exposure to big data technologies
  • Bachelors in Computer Science / related field or equivalent work experience
Powered by