TL;DR

Software Engineer, Search Data Infrastructure (AI): Designing and developing scalable offline indexing pipelines to process and transform data for Slack's search infrastructure with an accent on high availability and reliability of a distributed search system. Focus on planning resources and continuously optimizing the infrastructure and configuration of Solr clusters to maintain cluster health and search performance.

Location: Must be based in the US

Company

Slack is looking for experienced engineers to join its Core Infrastructure organization to help design, develop, and maintain the information retrieval infrastructure that supports Slack Search and many other product experiences.

What you will do

  • Design and develop scalable, resilient offline indexing pipelines that process and transform data for Slack's search infrastructure.
  • Collaborate with product engineering to conceptualize and build new features for our large user base.
  • Collaborate with team members to ensure high availability and reliability of a distributed search system.
  • Actively own team features or systems and define their long-term health, while also improving the health of surrounding systems.
  • Plan resources and continuously optimize the infrastructure and configuration of our Solr clusters to maintain cluster health and search performance.
  • Improve engineering standards, tooling, and processes.

Requirements

  • 3+ years of experience
  • Experience with distributed data processing frameworks such as Spark, Hadoop, Map/Reduce, or EMR.
  • Experience with Solr, ElasticSearch, Lucene, or other search technology.
  • Exposure to infrastructure/ops tools and frameworks such as Terraform, Chef, and Kubernetes
  • Proficient with functional or imperative programming languages — e.g., PHP, Python, Ruby, Go, C, or Java.
  • Bachelor's degree in Computer Science, Engineering or related field, or equivalent training, fellowship, or work experience required.

Nice to have

  • Experience around text mining and parsing.
  • You’ve built large-scale data pipelines professionally and can craft clean and beautiful code in Java, Scala, and/or SQL, and you possess exposure to Hadoop/Spark as well.

Culture & Benefits

  • Slack has a positive, diverse, and supportive culture.
  • Smart, humble, hardworking and, above all, collaborative team.