**Company Description** Vichara is a Financial Services focused products and services firm headquartered in NY and building systems for some of the largest i\-banks and hedge funds in the world. **Job Description** **Key Responsibilities** **Architecture \& System Design** * Architect, design, and lead **multi\-agent LLM systems** using **LangGraph, LangChain, and Promptfoo** for prompt lifecycle management and benchmarking. * Build **Retrieval\-Augmented Generation (RAG)** pipelines leveraging **hybrid vector search** (dense \+ keyword) using **LanceDB, Pinecone, or Elasticsearch**. * Define system workflows for summarization, query routing, retrieval, and response generation, ensuring minimal latency and high precision. * Develop **RAG evaluation frameworks** combining retrieval precision/recall, hallucination detection, and latency metrics — aligned with analyst and business use cases. **AI Model Integration \& Fine\-Tuning** * Integrate **GPT\-4o, PaLM 2, and open\-weight models (LLaMA, Mistral)** for task\-specific contextual Q\&A. * Fine\-tune transformer models (BERT, SentenceTransformers) for document classification, summarization, and sentiment analysis. * Manage prompt routing and variant testing using **Promptfoo** or equivalent tools. **Agentic AI \& Orchestration** * Implement **multi\-agent architectures** with modular flows — enabling task\-specific agents for summarization, retrieval, classification, and reasoning. * Design **fallback and recovery behaviors** to ensure robustness in production. * Employ **LangGraph** for parallel and stateful agent orchestration, error recovery, and deterministic flow control. **Data Engineering \& RAG Infrastructure** * Architect ingestion pipelines for structured and unstructured data — including financial statements, filings, and PDF documents. * Leverage **MongoDB** for metadata storage and **Redis Streams** for async task execution and caching. * Implement vector\-based search and retrieval layers for high\-throughput and low\-latency AI systems. **Observability \& Production Deployment** * Deploy end\-to\-end AI systems on **AWS EKS / Azure Kubernetes Service**, integrated with **CI/CD pipelines (Azure DevOps)**. * Build comprehensive **monitoring dashboards** using **OpenTelemetry** and **Signoz**, tracking latency, retrieval precision, and application health. * Enforce testing and regression validation using golden datasets and structured assertion checks for all LLM responses. **Cross\-functional Collaboration** * Collaborate with DevOps, MLOps, and application development teams to integrate AI APIs with **React / FastAPI**\-based user interfaces. * Work with business analysts to translate credit, compliance, and customer\-support requirements into actionable AI agent workflows. * Mentor a small team of GenAI developers and data engineers in RAG, embeddings, and orchestration techniques. **Qualifications** * Experience: + 5\+ years as an AI or ML Engineer * **Required Skills \& Experience** * **LLMs \& GenAI:** GPT\-4o, PaLM 2, LangGraph, LangChain, Promptfoo, SentenceTransformers * **RAG Frameworks:** LanceDB, Pinecone, ElasticSearch, FAISS, MongoDB * **Agentic AI:** LangGraph multi\-agent orchestration, routing logic, task decomposition * **Fine\-Tuning:** BERT / domain\-specific transformer tuning, evaluation framework design * **Infra \& MLOps:** FastAPI, Docker, Kubernetes (EKS/AKS), Redis Streams, Azure DevOps CI/CD * **Monitoring:** OpenTelemetry, Signoz, Prometheus * **Languages \& Tools:** Python, SQL, REST APIs, Git, Pandas, NumPy * **Nice\-to\-Have Skills** * Knowledge of **Reranker\-based retrieval** (MiniLM / CrossEncoder) * Familiarity with **Prompt evaluation and scoring** (BLEU, ROUGE, Faithfulness) * Domain exposure to **Credit Risk, Banking, and Investment Analytics** * Experience with **RAG benchmark automation** and **model evaluation dashboards** **Additional Information**