# Developer Feedback Loop

## Summary

The **Developer Feedback Loop** analyzes findings that developers have previously dismissed (as false positives or risk-accepted) and uses an LLM to generate prevention rules that improve the accuracy of future Arnica AI SAST scans.

The generated rules are saved as **additional-prompt text** that Arnica appends to the prompt used by AI SAST. They can be saved at one of two scopes:

* **Global** — appended to the **Additional Prompt** field on **Admin → AI Configurations → Model Configuration**. Applied to AI SAST scans across the tenant.
* **Per product** — appended to the **Additional Prompt** field on the **AI SAST** tab of a product (**Inventory → Products → *****product***** → AI SAST**). Applied only to scans of that product.

The feature is currently labeled **Preview**.

{% hint style="info" %}
The **Auto Generate** badge next to either Additional Prompt field is the entry point to the Developer Feedback Loop for that scope.
{% endhint %}

{% hint style="warning" %}
The Developer Feedback Loop is a different feature from [Developer Feedback on Push](/arnica-documentation/code-risks/code-risk-policy-settings/developer-feedback-on-push.md). Developer Feedback on Push is a code-risk policy that surfaces findings to developers at push time. The Developer Feedback Loop only edits AI SAST additional prompts — it does not dismiss findings, change policies, or affect non-AI SAST scans.
{% endhint %}

## How it works

1. Arnica queries dismissed SAST findings from the time window you configure (default: last 90 days).
2. The selected LLM analyzes the dismissals to identify recurring patterns — for example, framework-specific sanitizers, internal helper functions, or testing utilities — that consistently produce false positives.
3. Arnica produces a list of suggested rules, each with a confidence score and a scope (**global** rules apply across the tenant, while **product** rules apply only to a single product).
4. You review and select the rules you want to keep, and optionally preview the resulting prompt before saving.
5. When you save, the selected rules are appended as natural-language text to the corresponding **Additional Prompt** field — globally, per product, or both — and take effect on subsequent AI SAST scans.

## Access

Navigate to **Admin** → **AI Configurations** → **Developer Feedback Loop**.

<figure><img src="/files/t9n2AeLnMYTSy0bh2hXG" alt="Developer Feedback Loop tab in Admin → AI Configurations"><figcaption><p>The Developer Feedback Loop tab inside AI Configurations.</p></figcaption></figure>

## Run an analysis

1. Use the **Products** selector to scope the analysis to one or more products, or leave it as **All Products** to analyze across the tenant.
2. Expand **Advanced Settings** to fine-tune the analysis (see [Advanced settings](#advanced-settings) below).
3. Click **Analyze Findings**. The analysis is queued and runs against the configured LLM.
4. Track progress in the **Recent Analyses (Last 7 Days)** table. The status moves from **Enqueued** to **In progress** to **Completed**, and each row can be expanded to see step-by-step logs.

<figure><img src="/files/cJSKo6rSP2QlSycS6Q8A" alt="Recent Analyses table showing an analysis in progress"><figcaption><p>An analysis in progress in the Recent Analyses table.</p></figcaption></figure>

5. When the analysis completes, click **See results** in the **Actions** column to load the suggested rules.

{% hint style="info" %}
Recent analyses are retained for 7 days so you can re-load their results without re-running the analysis (and re-spending AI tokens).
{% endhint %}

## Advanced settings

<figure><img src="/files/LfAoFePXlkYOOJnEqjQj" alt="Advanced Settings expanded with Days Range, LLM Integration, and inclusion checkboxes"><figcaption><p>Advanced Settings control the scope of the analysis.</p></figcaption></figure>

### Days Range

Number of days of dismissal history to include in the analysis. Defaults to **90**.

### LLM Integration

The AI provider used to analyze dismissals and generate rules. The default is the tenant's primary OpenAI integration. Any provider configured under [Artificial Intelligence](/arnica-documentation/getting-started/artificial-intelligence.md) (Azure OpenAI, OpenAI ChatGPT, Anthropic, or Amazon Bedrock) can be selected.

The specific model used for each run is recorded in the **Model** column of the Recent Analyses table.

### Include Risk Accepted

When checked, findings dismissed as **Risk Accepted** are included along with **Dismissed – Not Accurate** findings. Disable this to analyze only false positives.

<figure><img src="/files/bjMKTS7v9cioiKS5vd7A" alt="Tooltip text for Include Risk Accepted"><figcaption><p>Tooltip for the Include Risk Accepted toggle.</p></figcaption></figure>

### Include Regular SAST

When checked, both regular (non-AI) SAST findings and AI SAST findings are analyzed. Disable this to focus the analysis exclusively on AI SAST findings.

<figure><img src="/files/76c5PzJKEsfA86Xq1Q7j" alt="Tooltip text for Include Regular SAST"><figcaption><p>Tooltip for the Include Regular SAST toggle.</p></figcaption></figure>

## Review and save suggested rules

After loading results, the page displays the suggested rules along with the actions you can take on them.

The header summarizes the analysis (for example, *Analyzed 117 findings across 1 product. Showing 53 global and 16 product rules.*) and offers the following controls:

| Control              | Purpose                                                                                                                      |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Start Over**       | Discard the current results and return to the configuration step.                                                            |
| **Select All**       | Select every rule currently visible after filtering.                                                                         |
| **Clear Selection**  | Deselect all rules.                                                                                                          |
| **Preview**          | Show the resulting AI SAST prompt(s) with the selected rules applied, without saving anything.                               |
| **Save Rules (N/M)** | Save the **N** selected rules out of the **M** suggested rules. Each rule is appended to its target Additional Prompt field. |

### Minimum Confidence Filter

Each suggested rule receives a confidence score from the LLM. Use the **Minimum Confidence Filter** slider to hide rules below a chosen threshold (default: **70%**). Lowering the threshold reveals more — but lower-quality — suggestions; raising it narrows the list to the strongest candidates.

### Selecting and previewing rules

Tick the checkbox next to each rule you want to keep. Each suggested rule is tagged either **global** or with a specific product, indicating which Additional Prompt field it will be appended to when saved.

Click **Preview** at any time to see the resulting AI SAST prompt with the selected rules applied. Use the preview to confirm the rules behave as expected before persisting them.

### Saving rules

Click **Save Rules** to commit the selected suggestions. Each rule is appended to its target Additional Prompt field and becomes active for the next AI SAST scan in scope:

* **Global rules** are appended to **Admin → AI Configurations → Model Configuration → Additional Prompt**.

  <figure><img src="/files/XOSuDMW2ODHcEVAmjy7I" alt="Model Configuration tab showing the global Additional Prompt field with the Auto Generate badge"><figcaption><p>Global destination — Admin → AI Configurations → Model Configuration → Additional Prompt.</p></figcaption></figure>
* **Product rules** are appended to the **Additional Prompt** field on the **AI SAST** tab of the corresponding product page (**Inventory → Products →&#x20;*****product*****&#x20;→ AI SAST**).

  <figure><img src="/files/KUN4GGoZDfnSOqgwTT0W" alt="Per-product AI SAST tab showing the Additional Prompt field with the Auto Generate badge"><figcaption><p>Per-product destination — Inventory → Products → <em>product</em> → AI SAST → Additional Prompt.</p></figcaption></figure>

Both fields display an **Auto Generate** badge that opens the Developer Feedback Loop pre-filtered for that scope, so you can re-run the analysis from the destination at any time.

You can edit, trim, or remove saved rules at any time directly in the corresponding Additional Prompt field — saved rules are plain text and are not stored as separate entities.

{% hint style="warning" %}
Developer Feedback Loop rules are not the same as **Vibe Coding Rules**. Vibe Coding Rules are stored as standalone AI Coding Assistant Rules under **Admin → AI Configurations → Vibe Coding Rules** and are used by AI coding assistants. Developer Feedback Loop rules are stored as additional-prompt text and only affect AI SAST scans.
{% endhint %}

## Recent Analyses

The **Recent Analyses (Last 7 Days)** table records every run, including:

* **Status** — Enqueued, In progress, Completed, or Failed
* **Started** / **Completed** — relative timestamps for each phase
* **Summary** — for example, *117 findings from 1 product / 69 rules generated*
* **Model** — the AI model that performed the run (for example, `gpt-5-mini`)
* **Actions** — **See results** loads a completed run back into the review pane without re-running the analysis

Click any row to expand a detailed log of the run's stages (search, analysis, rule generation).

## Empty result

If no dismissed findings match the configured filters, Arnica displays the notification *"No dismissed findings found matching the criteria."* Adjust the **Days Range**, the **Include Risk Accepted** / **Include Regular SAST** toggles, or the **Products** selector and re-run.

## Related

* [Static Application Security Testing (SAST)](/arnica-documentation/code-risks/static-application-security-testing-sast.md)
* [Custom SAST Rules](/arnica-documentation/code-risks/static-application-security-testing-sast/custom-sast-rules.md)
* [Artificial Intelligence Integrations](/arnica-documentation/getting-started/artificial-intelligence.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.arnica.io/arnica-documentation/code-risks/static-application-security-testing-sast/developer-feedback-loop.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.