This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial services, autonomous cars, Government, academia, research and manufacturing.

"DDN's A3I solutions are transforming the landscape of AI infrastructure." – IDC 

“The real differentiator is DDN. I never hesitate to recommend DDN. DDN is the de facto name for AI Storage in high performance environments” - Marc Hamilton, VP, Solutions Architecture & Engineering | NVIDIA 

DDN is the global leader in AI and multi-cloud data management at scale. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. With a proven track record of performance, reliability, and scalability, DDN empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence. 

Our success is driven by our unwavering commitment to innovation, customer-centricity, and a team of passionate professionals who bring their expertise and dedication to every project. This is a chance to make a significant impact at a company that is shaping the future of AI and data management. 

Our commitment to innovation, customer success, and market leadership makes this an exciting and rewarding role for a driven professional looking to make a lasting impact in the world of AI and data storage. 

We are seeking a talented and experienced Senior ML Engineer to help us deploy AI/ML training and advanced Retrieval-Augmented Generation (RAG) pipelines for high-performance AI applications. You will be responsible for designing, deploying, and optimizing large-scale AI training and inference pipelines. You will work closely with data scientists and software developers to operationalize models using open-source tools like Apache Spark, Airflow, and MLflow. You will collaborate in our efforts to scale Retrieval-Augmented Generation (RAG) pipelines for AI applications, ensuring robust and efficient deployment.

Key Responsibilities:

• Design and deploy large-scale AI/ML training pipelines using open-source tools such as Apache Spark and Apache Airflow.
• Integrate MLflow with DDN’s Infinia product for tracking and managing machine learning experiments, model versioning, and deployment.
• Implement and scale Retrieval-Augmented Generation (RAG) pipelines to enable efficient retrieval of knowledge for generative models.
• Automate, monitor, and optimize the end-to-end ML workflows and pipelines for production-grade applications.
• Work collaboratively with cross-functional teams including data science, engineering, and product to operationalize AI/ML models.
• Maintain and improve CI/CD pipelines for ML models, ensuring smooth transitions from research to production environments.
• Utilize cloud platforms (AWS, GCP, or Azure) for scalable infrastructure management.
• Monitor and troubleshoot pipeline performance issues, implementing solutions to optimize runtime and resource usage.
• Ensure best practices in version control, containerization (Docker, Kubernetes), and infrastructure as code (Terraform, Ansible).
• Keep up-to-date with the latest developments in MLOps, AI/ML frameworks, and tooling.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Data Science, Machine Learning, or related fields.
• 8+ years of experience in machine learning operations (MLOps) or related roles.
• Extensive experience with Apache Spark, Apache Airflow, and MLflow or equivalent.
• Proven expertise in building and scaling AI/ML pipelines.
• Strong understanding of machine learning frameworks and libraries (TensorFlow, PyTorch, NVIDIA NeMo).
• Experience in deploying open-source vector databases at scale.
• Solid understanding of cloud infrastructure (AWS, GCP, Azure) and distributed computing.
• Proficiency with containerization tools (Docker, Kubernetes) and infrastructure as code.
• Excellent problem-solving and troubleshooting skills, with attention to detail and performance optimization.
• Strong communication and collaboration skills.

Preferred Qualifications:

• Experience with large-scale data processing and storage solutions (Hadoop, Hive, HDFS).
• Knowledge of NLP techniques and tools for model deployment.