# error-handling

Adopt this skill: https://proskills.md/skills/other/microsoft-data-formulator-cursor-skills-error-handling

Page: https://proskills.md/skills/other/microsoft-data-formulator-cursor-skills-error-handling
This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required.

## File: .cursor/skills/error-handling/SKILL.md
size: 8230
upstream: https://github.com/microsoft/data-formulator/blob/main/.cursor/skills/error-handling/SKILL.md

````text
---
name: error-handling
description: 统一错误处理系统。在添加 API 端点、修改错误处理、添加前端 API 调用、编写错误相关测试时使用。
---

# Error Handling Skill

Unified error handling system for DF. Use when adding API endpoints, modifying error handling, or adding frontend API calls.

> **Prerequisites**: Read `docs/dev-guides/7-unified-error-handling.md` before changing API error behavior.
> Read `docs/dev-guides/2-log-sanitization.md` when the work involves logging, credentials, external services, or DataLoaders.
> If your work introduces new error handling patterns or conventions, update this file and related dev-guides accordingly.

## Architecture Overview

```
Frontend                              Backend
────────                              ───────
apiClient.ts                          errors.py
├── apiRequest()    ←── JSON ────     ├── ErrorCode (enum)
├── streamRequest() ←── NDJSON ──     └── AppError (exception)
└── parseStreamLine()
                                      error_handler.py
errorCodes.ts                         ├── register_error_handlers(app)
└── getErrorMessage()                 ├── classify_and_wrap_llm_error()
                                      └── stream_error_event()
errorHandler.ts
└── handleApiError()                  security/sanitize.py
                                      └── classify_llm_error() (internal)
MessageSnackbar ← dfSlice.messages
```

## Protocol Snapshot

Use this contract for all new or reworked DF APIs:

| Scenario | HTTP | Shape |
|----------|------|-------|
| Non-streaming success | `200` | `{"status": "success", "data": ...}` |
| Non-streaming business/validation error | `200` | `{"status": "error", "error": {"code", "message", "retry", "request_id"}}` |
| Non-streaming auth/authorization error | `401` / `403` | same structured error body |
| Streaming preflight error | `200` | `application/json` + `{"status": "error", "error": ...}` |
| Streaming in-flight fatal error | `200` | NDJSON line: `{"type": "error", "error": ...}` |
| No Flask route / too large / unhandled crash | `404` / `413` / `500` | transport-level error |

Do not use HTTP `400`/`422` for application validation errors in new code.
Do not convert in-flight NDJSON errors to `status: "error"`; once the stream has
started, event `type` is the protocol discriminator.

## Backend: Adding a New API Endpoint

### HTTP Status Code Policy

**Application-controlled business and validation errors return HTTP 200** with
`status: "error"` in the body. Only these use non-200:
- `401`/`403` — auth errors (`AUTH_REQUIRED`, `AUTH_EXPIRED`, `ACCESS_DENIED`)
- `404` — no matching Flask route
- `413` — WSGI body limit exceeded
- `500` — unhandled exception (program bug)

### Non-streaming endpoint

```python
from data_formulator.errors import AppError, ErrorCode
from data_formulator.error_handler import json_ok

@bp.route('/my-endpoint', methods=['POST'])
def my_endpoint():
    content = request.get_json()
    if not content.get('required_field'):
        raise AppError(ErrorCode.INVALID_REQUEST, "Missing required_field")

    try:
        result = do_work(content)
    except SomeBusinessError as e:
        raise AppError(ErrorCode.DATA_LOAD_ERROR, "Failed to load data") from e
    except Exception as e:
        from data_formulator.error_handler import classify_and_wrap_llm_error
        raise classify_and_wrap_llm_error(e) from e

    return json_ok(result)
# Global handler returns: HTTP 200 + {"status": "error", "error": {code, message, retry}}
# Auth errors (AUTH_REQUIRED/AUTH_EXPIRED/ACCESS_DENIED) return 401/403
```

Legacy `{"status": "error", "message": "..."}`, `error_message`, bare `{error}`,
and `status: "ok"` responses are historical formats. Do not add new compatibility
branches for them; migrate the route to `json_ok()` / `AppError` before using
`apiRequest()`.

### Streaming endpoint

Validation MUST be outside the generator. Failures return 200 JSON (not NDJSON).

```python
from data_formulator.errors import AppError, ErrorCode
from data_formulator.error_handler import (
    classify_and_wrap_llm_error,
    stream_error_event,
    stream_preflight_error,
)

@bp.route('/my-stream', methods=['POST'])
def my_stream():
    if not request.is_json:
        return stream_preflight_error(
            AppError(ErrorCode.INVALID_REQUEST, "Invalid request")
        )

    content = request.get_json()
    client = get_client(content['model'])

    def generate():
        try:
            for event in agent.run(...):
                yield json.dumps(event, ensure_ascii=False) + "\n"
        except Exception as e:
            yield stream_error_event(classify_and_wrap_llm_error(e))

    return Response(stream_with_context(generate()), mimetype='application/x-ndjson')
```

Streaming runtime errors intentionally use `{"type": "error", "error": ...}`.
They cannot use a top-level `status` envelope because the HTTP response and NDJSON
event stream have already started.

## Frontend: Consuming an API

### Non-streaming

```typescript
import { apiRequest } from '../app/apiClient';
import { handleApiError } from '../app/errorHandler';

try {
    const { data } = await apiRequest<ResponseType>(getUrls().MY_ENDPOINT, {
        method: 'POST',
        body: JSON.stringify(payload),
        headers: { 'Content-Type': 'application/json' },
    });
} catch (e) {
    handleApiError(e, 'MyComponent');
}
```

For UI loading state, model the request lifecycle explicitly with
`LoadableState` from `src/app/loadableState.ts`. Do not infer loading from
missing data (`!data`), because failed requests may legitimately leave data
empty while loading has ended.

### Streaming

```typescript
import { streamRequest } from '../app/apiClient';
import { handleApiError } from '../app/errorHandler';

try {
    for await (const event of streamRequest(url, options, abortController.signal)) {
        switch (event.type) {
            case 'text_delta':
                break;
            case 'error':
                // Error arrived mid-stream — show inline in component.
                break;
            case 'done':
                break;
        }
    }
} catch (e) {
    handleApiError(e, 'MyComponent');
}
```

### With callbacks

```typescript
handleApiError(e, 'MyComponent', {
    onAuth: () => redirectToLogin(),        // AUTH_REQUIRED / AUTH_EXPIRED
    onRetryable: () => retryOperation(),    // LLM_RATE_LIMIT / LLM_TIMEOUT
    silent: true,                           // don't show Snackbar (component handles display)
});
```

### Migration and special cases

DF API consumers should use `apiRequest()` / `streamRequest()` and
`handleApiError()`. Direct `fetchWithIdentity()` is for lower-level client helpers
and explicit protocol exceptions such as file downloads, blob/CSV responses,
OIDC redirects, SPA fallback, or third-party URLs.

Do not apply the normal JSON API protocol mechanically to file downloads / CSV
streaming, SPA fallback, OIDC redirect flows, frontend fetches to third-party
URLs, or errors after a streaming response has already started. Check the route's
protocol first, then preserve safe error bodies and avoid `str(exc)` exposure.

## Adding a New Error Code

1. **Backend** — Add to `py-src/data_formulator/errors.py` `ErrorCode`:
   ```python
   MY_NEW_ERROR = "MY_NEW_ERROR"
   ```
   No HTTP mapping needed — defaults to HTTP 200. Only add to `ERROR_CODE_HTTP_STATUS` if it's an auth code.

2. **Frontend mapping** — Add to `src/app/errorCodes.ts` `ERROR_CODE_I18N_MAP`:
   ```typescript
   MY_NEW_ERROR: 'errors.myNewError',
   ```

3. **Translations** — Add to both locale files:
   - `src/i18n/locales/en/errors.json`: `"myNewError": "English message"`
   - `src/i18n/locales/zh/errors.json`: `"myNewError": "中文消息"`

## Migrated Endpoints Reference

All streaming endpoints are now on the unified protocol:

| Endpoint | Format | Notes |
|----------|--------|-------|
| `/data-agent-streaming` | NDJSON + `stream_error_event()` | Emits top-level `type` events; errors use `{type:"error", error:{...}}` |
| `/ge
````

## File: README.md
upstream: https://github.com/microsoft/data-formulator/blob/main/README.md

```text
<h1 align="center">
  <img src="./public/favicon.ico" alt="Data Formulator icon" width="28">&nbsp;
  Data Formulator: AI-powered Data Visualization
</h1>

<p align="center">
  🪄 Explore data with visualizations, powered by AI agents.
</p>

<p align="center">
  <a href="https://data-formulator.ai"><img src="https://img.shields.io/badge/🚀_Try_Online_Demo-data--formulator.ai-F59E0B?style=for-the-badge" alt="Try Online Demo"></a>
  &nbsp;
  <a href="#get-started"><img src="https://img.shields.io/badge/💻_Install_Locally-uvx_|_pip-3776AB?style=for-the-badge" alt="Install Locally"></a>
</p>

<p align="center">
  <a href="https://pypi.org/project/data_formulator/"><img src="https://img.shields.io/pypi/v/data_formulator.svg?label=pypi%3A%20data_formulator" alt="PyPI"></a>&ensp;
  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>&ensp;
  <a href="https://www.youtube.com/watch?v=GfTE2FLyMrs"><img src="https://img.shields.io/badge/YouTube-white?logo=youtube&logoColor=%23FF0000" alt="YouTube"></a>&ensp;
  <a href="https://github.com/microsoft/data-formulator/actions/workflows/python-build.yml"><img src="https://github.com/microsoft/data-formulator/actions/workflows/python-build.yml/badge.svg" alt="build"></a>&ensp;
  <a href="https://discord.gg/mYCZMQKYZb"><img src="https://img.shields.io/badge/discord-chat-green?logo=discord" alt="Discord"></a>
</p>

<!-- [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/data-formulator?quickstart=1) -->
<!-- 
https://github.com/user-attachments/assets/8ca57b68-4d7a-42cb-bcce-43f8b1681ce2 -->

## Why Data Formulator?

Working with data is hard for two simple reasons:

1. **Data lives everywhere.** Connecting agents to files, databases,
   warehouses, and BI tools takes time. It is even harder when agents start
   answering before the relationships between data sources are clear.
2. **Questions evolve as you explore.** Each answer can lead to follow-up
   questions, comparisons, and new directions. A long chat history makes it
   hard to see where you are and how you got there.

Data Formulator provides one visual workspace for exploring and analyzing data:

1. **Data connectors** give agents a common way to connect to different data
   sources and maintains a data memory to remember the relationships between them.
2. **Data Threads** let you branch into different questions, compare paths,
   and use visualizations to discover deeper insights without losing context.

https://github.com/user-attachments/assets/8e4f8a08-6423-4227-a1f7-559e0126ce31

> [!TIP]
> **Love the charts?** They're built on [**Flint**](https://microsoft.github.io/flint-chart/). It's an open-source visualization language that compiles compact chart specs into polished visualizations.

## News 🔥🔥🔥

[08-15-2026] **Data Formulator 0.8 beta 1** (`0.8.0b1`) introduces:

- **One unified flow:** load data, ask questions, review results, and branch in the Data Thread.
- **More data sources:** use files, local folders, databases, and platforms such as Databricks.
- **Better charts:** explore more Flint-powered charts, recommendations, themes, and styling tools.

> Preview with `pip install --pre data_formulator==0.8.0b1` or `uvx data_formulator@0.8.0b1`.
> Install the latest stable release (0.7) with `pip install data_formulator` or run instantly with `uvx data_formulator`.

See the [changelog](CHANGELOG.md) for release details.

## Previous Updates

Here are milestones that lead to the current design:
- **v0.7** (05-28-2026): Turn ANY data into insights in five steps — connect governed data sources, load via agents, explore with the unified `DataAgent` + Data Thread, refine 30+ chart types (semantic chart engine powered by [Flint](https://github.com/microsoft/flint-chart)) with a style-refinement agent, and share as reports. Plus persistent sessions & workspaces and a multilingual (English/Chinese) UI.
- **v0.6** ([Demo](https://github.com/microsoft/data-formulator/releases/tag/0.6)): Real-time insights from live data — connect to URLs and databases with automatic refresh
- **uv support**: Faster installation with [uv](https://docs.astral.sh/uv/) — `uvx data_formulator` or `uv pip install data_formulator`
- **v0.5.1** ([Demo](https://github.com/microsoft/data-formulator/pull/200#issue-3635408217)): Community data loaders, US Map & Pie Chart, editable reports, snappier UI
- **v0.5**: Vibe with your data, in control — agent mode, data extraction, reports
- **v0.2.2** ([Demo](https://github.com/microsoft/data-formulator/pull/176)): Goal-driven exploration with agent recommendations and performance improvements
- **v0.2.1.3/4** ([Readme](https://github.com/microsoft/data-formulator/tree/main/py-src/data_formulator/data_loader) | [Demo](https://github.com/microsoft/data-formulator/pull/155)): External data loaders (MySQL, PostgreSQL, MSSQL, Azure Data Explorer, S3, Azure Blob)
- **v0.2** ([Demos](https://github.com/microsoft/data-formulator/releases/tag/0.2)): Large data support with DuckDB integration
- **v0.1.7** ([Demos](https://github.com/microsoft/data-formulator/releases/tag/0.1.7)): Dataset anchoring for cleaner workflows
- **v0.1.6** ([Demo](https://github.com/microsoft/data-formulator/releases/tag/0.1.6)): Multi-table support with automatic joins
- **Model Support**: OpenAI, Azure, Ollama, Anthropic via [LiteLLM](https://github.com/BerriAI/litellm) ([feedback](https://github.com/microsoft/data-formulator/issues/49))
- **Python Package**: Easy local installation ([try it](#get-started))
- **Visualization Challenges**: Test your skills ([challenges](https://github.com/microsoft/data-formulator/issues/53))
- **Data Extraction**: Parse data from images and text ([demo](https://github.com/microsoft/data-formulator/pull/31#issuecomment-2403652717))
- **Initial Release**: [Blog](https://www.microsoft.com/en-us/research/blog/data-formulator-exploring-how-ai-can-help-analysts-create-rich-data-visualizations/) | [Video](https://youtu.be/3ndlwt0Wi3c)

## Overview

**Data Formulator** is a Microsoft Research project for data exploration with visualizations powered by AI agents. It combines *UI interactions* with *natural language* so analysts can communicate intent, branch into alternative analyses, and share results — starting from any data format (screenshot, text, CSV, or database).

## Get Started

Play with Data Formulator with one of the following options. 

### Desktop downloads

CI builds self-contained Windows and macOS applications for pull requests and
every update to `main`. Download the latest archives from the **Artifacts**
section of the most recent
[desktop builds workflow](https://github.com/microsoft/data-formulator/actions/workflows/desktop-build.yml).
Workflow artifacts are retained for 30 days. Tagged builds are also attached
as permanent downloads to the corresponding
[GitHub Release](https://github.com/microsoft/data-formulator/releases).

Extract the archive, then launch Data Formulator using the instructions for
your operating system:

- **Windows:** Run `Data Formulator.exe`. If Microsoft Defender SmartScreen
  appears, select **More info**, verify that you downloaded the archive from
  this repository, and then select **Run anyway**.
- **macOS:** Move `Data Formulator.app` to **Applications**. The first time you
  open it, macOS may report that Apple could not verify the app. Open
  **System Settings → Privacy & Security**, scroll to **Security**, and select
  **Open Anyway** for Data Formulator. Confirm by selecting **Open** when
  prompted.

> [!WARNING]
> These are automated preview builds and are not currently code-signed or
> notarized. Only bypass the operating-system warning when the archive was
> downloaded directly from this repository's workflow artifacts or releases.

- **Option 1: Install via uv (recommended)**
  
  [uv](https://docs.astral.sh/uv/) is an extremely fast Python package manager. If yo
```
