Senior AI Engineer - MCP Servers, Agentic Systems, & Digital Workers

OtterSoft is a software development company focused on building, scaling, and modernizing digital solutions, with a strong emphasis on AI-driven technologies. We specialize in delivering innovative, data-driven applications—particularly within the healthcare industry—helping organizations improve outcomes, enhance efficiency, and navigate complex systems. Our teams bring expertise in full-stack development, cloud-based solutions, and artificial intelligence to accelerate innovation and solve real-world challenges. At OtterSoft, we are committed to delivering high-quality results with speed, precision, and a customer-first mindset.

Role Summary

We are a Microsoft-centric engineering organization building the next generation of AI-powered product experiences. We are looking for an expert-level AI Engineer who specialises in the design and delivery of Model Context Protocol (MCP) servers, conversational AI agents, and autonomous digital workers that operate at scale within our platform.

In this role you will own the full lifecycle — from API analysis and MCP server design through to agent orchestration, deployment, and observability — working exclusively within our .NET / C# and Azure AI Foundry ecosystem.

Why This Role Matters

Our platform exposes a rich set of proprietary APIs. This engineer will make those APIs intelligent and conversational by wrapping them with production-grade MCP servers and then orchestrating fleets of AI agents and digital workers on top of them. The result: users interact with the platform through natural language chat; automated digital workers perform complex multi-step actions without human intervention.

Key Responsibilities
*MCP Server Development*

• Analyze existing platform REST APIs and design idiomatic MCP tool schemas for each capability
• Build, test, and publish MCP servers in C# / .NET 8+ using the official MCP .NET SDK
• Implement authentication, rate-limiting, input validation, and structured error handling inside each server
• Version and maintain MCP server contracts; manage schema evolution without breaking downstream agents
• Write comprehensive integration tests and document each server with usage examples

*Conversational AI Agents*

• Design multi-turn chat agents using Azure AI Foundry Agent Service and Semantic Kernel
• Integrate agents with MCP servers as tool providers; define tool-call policies and fallback strategies
• Implement memory patterns (short-term conversation context, long-term vector memory via Azure AI Search)
• Build retrieval-augmented generation (RAG) pipelines to ground agent responses in platform data
• Tune system prompts, tool descriptions, and sampling parameters for reliability and latency targets
• Expose agents through REST endpoints and WebSocket channels consumed by product UIs

*Digital Workers (Autonomous Agents)*

• Architect autonomous, goal-directed digital workers capable of planning and executing multi-step platform workflows without human intervention
• Implement agent loop patterns (ReAct, Plan-and-Execute, Reflection) using Semantic Kernel Process Framework or Azure Durable Functions for stateful orchestration
• Design human-in-the-loop checkpoints, approval gates, and escalation paths for high-risk actions
• Instrument workers with structured telemetry (Azure Monitor, App Insights) to track task completion, error rates, and token consumption
• Build retry, compensation, and idempotency logic so workers recover gracefully from partial failures

*Platform & Infrastructure*

• Deploy all components to Azure (AKS, Azure Container Apps, or Azure Functions)
• Manage secrets and credentials with Azure Key Vault; apply least-privilege RBAC throughout
• Integrate with Azure API Management as the gateway for all MCP and agent endpoints
• Participate in architecture reviews, security threat-modelling, and cost-optimization exercises

Required Skills & Experience
*LISTED BELOW: Skill Area; Required Level; Key Technologies/Context*

• Model Context Protocol (MCP); Expert Level; MCP .NET SDK, tool schema design, server lifecycle management
• C# / .NET; Expert; .NET 8+, async/await, dependency injection, minimal APIs, xUnit
• Azure AI Foundry; Expert; Agent Service, Prompt Flow, model deployments, evaluations
• Semantic Kernel; Expert; Kernel plugins, planners, memory, Process Framework, SK Agents
• LLM Integration; Expert; Azure OpenAI (GPT-4o, o-series), function calling, structured output
• AI Agent Patterns; Expert; ReAct, Plan-and-Execute, multi-agent orchestration, tool selection
• Azure Platform; Strong; AKS, Container Apps, Durable Functions, API Management, Key Vault
• API Design; Strong; REST, OpenAPI / Swagger, GraphQL, gRPC — as both consumer and author
• Observability; Strong; Azure Monitor, App Insights, OpenTelemetry, structured logging
• Security; Solid; OAuth 2.0, Managed Identity, RBAC, secrets management, OWASP top 10

Qualifications

*Required*

• 7+ years of professional software engineering experience, with at least 3 years focused on AI / ML systems
• Demonstrable production experience building and shipping LLM-powered applications
• Hands-on MCP server development experience (shipped at least one MCP server consumed by a real agent)
• Deep expertise in C# / .NET and the Microsoft Azure ecosystem
• Proven ability to design reliable, observable, and secure agentic workflows

*Preferred*

• Experience with multi-agent frameworks (AutoGen, CrewAI, or equivalent)
• Familiarity with enterprise governance requirements for AI systems (content filtering, audit logging, PII handling)
• Background in a regulated industry (finance, healthcare, legal) where responsible AI practices are mandatory

*Nice to Have*

• Experience with the Microsoft 365 Agents SDK or Copilot Studio for surfacing agents within Microsoft Teams
• Knowledge of Azure AI Content Safety and Responsible AI tooling
• Familiarity with A/B testing and online evaluation of LLM outputs in production
• Prior experience mentoring engineers on agentic patterns and prompt engineering best practices

Pay: $80.00 - $100.00 per hour

Application Question(s):

• This is a contracted position (1099). Do you have any concern/trouble working in that capacity?
• Do you require Visa Sponsorship?
• Are you working through an employer that is providing Visa sponsorship?

Experience:

• C# / .NET: 6 years (Required)
• professional work: 7 years (Required)
• AI: 1 year (Required)
• Azure: 6 years (Required)

Work Location: Remote