MIPSEval: Automated Multi-Turn Injection Planning for LLM Security

by ddos · February 3, 2026

Multi-turn Injection Planning System for LLM Evaluation

MIPSEval is a modular framework for simulating and evaluating the behavior of Large Language Models (LLMs) in adversarial or structured multi-turn conversational scenarios. It supports both OpenAI-hosted models and locally hosted models.

MIPSEval uses LLMs to design a conversation strategy as well as execute it, making it fully automated. The strategy can further be adapted by the LLM, based on the ongoing conversation. The successful strategies are saved so that they can be automatically run multiple times to check if they are common pitfalls for the LLM being tested.

Features

Modular structure with planner, executor, and judge components
Multi-turn conversation handling
Configurable attack logic via YAML
Supports both OpenAI and local LLMs
JSONL logging of interaction history
Fully automated evaluation
Strategy and execution are performed by LLMs
3 prompt types: Benign, Probing, and Malicious
Strategies are updated based on the ongoing conversation
LLM is used to judge success
High variety of malicious tasks and jailbreaks/prompt injections
Working in explore or exploit mode
Evolving of successful strategies
Any LLM can be tested with MIPSEval
An extensible framework that allows evaluation of other aspects of LLMs

Install

[pastacode lang=”markup” manual=”git%20clone%20https%3A%2F%2Fgithub.com%2Fstratosphereips%2Fmipseval.git%0Acd%20mipseval%0Apip%20install%20-r%20requirements.txt%0Acd%20src” message=”” highlight=”” provider=”manual”/]

Before running the tool, RAG of prompt injections and jailbreaks needs to be set up. It can be done with the following command in src folder:

python add_json_to_rag.py

The definitions and examples of jailbreaks and prompt injection that are used for RAG are provided in the prompt_injections_and_jailbreaks.json file.

You must also create a .env file with your API key (if using OpenAI):