> ## Documentation Index
> Fetch the complete documentation index at: https://help.miyaip.com/llms.txt
> Use this file to discover all available pages before exploring further.

# MiyaIP Crawler API: authentication, tasks, and credits

> Integrate the MiyaIP Crawler API with cURL or Python: authenticate, discover interfaces, submit a task, poll results, and check credit costs.

The Crawler API creates asynchronous tasks. Follow one lifecycle: **discover an interface → read its parameters → submit once → query the same task → check the charge**. Use the [console](https://miyaip.com/dashboard/crawlerInvoke?tab=invoke) to inspect your interfaces, packages, and call logs.

## Prepare authentication

Use `https://miyaip.com` as the base URL. Sign in to the console, open your browser's developer tools **Network** panel, and inspect your own `/api/SysLogin/GetUserInfo` request. Copy the token after `Bearer` in the `Authorization` header. This is your console session token; an MCP key authenticates the MCP service and does not replace it.

The examples use Bash, cURL with `--fail-with-body`, and Python 3.8 or later. On Windows, use `curl.exe` and set the same variables with `$env:NAME` in PowerShell. Keep the token private. Task creation can consume your account's credits.

```bash theme={null}
read -rsp 'Console token: ' MIYA_CONSOLE_TOKEN
echo
export MIYA_CONSOLE_TOKEN
export MIYA_INTERFACE_KEY='replace-with-an-interface-key'
```

## Complete one task

<Steps>
  <Step title="Discover available interfaces">
    Read the returned interface array. Choose an enabled interface and use its actual `InterfaceKey`.

    ```bash theme={null}
    curl --fail-with-body -sS --max-time 30 \
      -H "Authorization: Bearer $MIYA_CONSOLE_TOKEN" \
      https://miyaip.com/api/CrawlerInterface/List
    ```
  </Step>

  <Step title="Read the selected parameter definition">
    ```bash theme={null}
    curl --fail-with-body -sS --max-time 30 --get \
      -H "Authorization: Bearer $MIYA_CONSOLE_TOKEN" \
      --data-urlencode "interfaceKey=$MIYA_INTERFACE_KEY" \
      https://miyaip.com/api/CrawlerInterface/Detail
    ```

    Parse the returned `ParamSchema` string. Create `params.json` containing only the interface's parameter object, using its exact keys, types, required fields, and limits. Use `{}` only if there are no required parameters.
  </Step>

  <Step title="Submit exactly one task">
    Build the request from your saved parameter object. Preserve the request fields' casing.

    ```bash theme={null}
    python -c 'import json,os; p=json.load(open("params.json",encoding="utf-8")); print(json.dumps({"InterfaceKey":os.environ["MIYA_INTERFACE_KEY"],"Params":p}))' > request.json
    curl --fail-with-body -sS --max-time 30 \
      -H "Authorization: Bearer $MIYA_CONSOLE_TOKEN" \
      -H 'Content-Type: application/json' \
      --data-binary @request.json \
      https://miyaip.com/api/Crawler/Invoke
    ```

    Check both the HTTP status and numeric business `code`. Successful creation returns a `taskId`, a running status, and `costCredits`; it does not contain a completed crawl. Keep the returned task ID.
  </Step>

  <Step title="Query the existing task">
    ```bash theme={null}
    export MIYA_TASK_ID='replace-with-returned-taskId'
    curl --fail-with-body -sS --max-time 30 --get \
      -H "Authorization: Bearer $MIYA_CONSOLE_TOKEN" \
      --data-urlencode "taskId=$MIYA_TASK_ID" \
      https://miyaip.com/api/Crawler/TaskResult
    ```

    While the status is `running`, query this endpoint again at approximately five-second intervals. Stop at `success`, `failed`, or `timeout`. After success, validate that `result` contains the data your application requires.

    A local timeout does not cancel the server task. Retain `taskId` and query it later. If a creation request times out, inspect call logs before submitting again; an uncertain response can still correspond to a created task.
  </Step>

  <Step title="Check actual consumption and call logs">
    Read `costCredits` in the task response and compare it with your console call log. Do not calculate a fixed number of tasks from a package's credits without accounting for the interface and parameters.
  </Step>
</Steps>

## Use the complete Python runner

[Download crawler\_api.py](/examples/crawler_api.py). This shared script uses only Python's standard library. It reads the interface definition, creates one task, and polls with a deadline. It accepts both common response field casings, stops on unexpected statuses, avoids forwarding your token to a redirected host, and exits with code `1` on failure.

```bash theme={null}
python crawler_api.py "$MIYA_INTERFACE_KEY" params.json --wait-seconds 600
```

Its default request timeout is 30 seconds and polling deadline is 600 seconds. These are client settings. The task ID goes to stderr; the final task object goes to stdout. Ending the runner does not cancel a server task.

## Endpoint reference

| Method and path | Request | Returned information |
| - | - | - |
| `GET /api/CrawlerInterface/List` | No parameters | Interface definitions, names, `ParamSchema`, cost metadata, and enabled state |
| `GET /api/CrawlerInterface/Detail` | Query `interfaceKey` | One interface's current parameter and cost definition |
| `POST /api/Crawler/Invoke` | JSON `InterfaceKey` and object `Params` | `taskId`, running status, `costCredits` |
| `GET /api/Crawler/TaskResult` | Query `taskId` | Status, `result`, `error`, credits, `totalCostMs`, start and finish times |
| `GET /api/Crawler/CallLogPage` | `PageNo=1`, `PageSize=10`; optional filters below | Records, totals, request/result JSON, errors, credits, elapsed time, and `refunded` |

Log filters include `KeyWord`, `InterfaceKey`, `Status`, `SearchBeginTime`, and `SearchEndTime`. Status values are `running`, `success`, `failed`, and `timeout`. Format date bounds as `YYYY-MM-DD 00:00:00` and `YYYY-MM-DD 23:59:59`; confirm the server timezone before comparing reporting periods. `UserId` is an administrator-only filter and does not grant access to another user's records.

Responses normally use `{ code, message, body }`. Numeric business codes `0` and `200` are successful. The runner also unwraps an outer `data` object. Responses may use `body`/`Body` and camelCase/PascalCase fields; retain the documented request casing.

## Parse ParamSchema

`ParamSchema` is a JSON-encoded string containing an array of field definitions. It is not a standard JSON Schema document. This illustrates the decoded structure; it does not define a live interface:

```json theme={null}
[
  { "name": "url", "label": "URL", "type": "string", "required": true },
  { "name": "limit", "label": "Limit", "type": "int", "min": 1, "max": 100, "default": 10 }
]
```

| Property | How to use it |
| - | - |
| `name`, `label` | Submit `name`; use `label` for display |
| `required`, `default` | Supply required values and check defaults |
| `type` | Use `string`, `int`, `number`, `float`, `bool`, `enum`, or `string[]` as declared |
| `min`, `max`, `options` | Check numeric bounds, integer requirements, finite values, and enum options |
| `showWhen` | AND between keys and OR between accepted values; omit fields whose condition is false |
| `hidden`, `advanced`, `group` | Display hints; hiding a field does not automatically omit a valid value |

Use JSON booleans and arrays. Do not assume an interface supports rendering, sessions, or a particular output format unless its current definition says so.

## Read results and errors

After `success`, `result` can be an object, array, string, or null. `totalCostMs` is milliseconds. Validate the result separately from the task state.

| Condition | Action |
| - | - |
| HTTP or business `401` | Sign in again and replace the console token |
| HTTP failure or business code other than `0`/`200` | Read the service response; do not treat HTTP 200 alone as success |
| `failed` or `timeout` | Stop polling and inspect `error` and the existing call log |
| Local wait deadline reached | Keep the task ID and resume querying later |
| Invalid JSON or unknown status | Stop, retain the task ID if available, and inspect logs without exposing the token |

SignalR is optional. The console subscribes to `PublicCrawlerTaskStatus`; its Hub URL depends on deployment configuration. Use HTTP polling unless your deployment supplies a confirmed Hub URL. Reconnecting does not require creating another task.

## Understand credits and limits

Where a cost parameter and multiplier exist, estimate consumption with:

```text theme={null}
CreditCostBase + Params[CreditCostParam] × CreditCostMultiplier
```

Otherwise use the base cost. The actual returned charge is `costCredits`. For example, **1,000 credits ÷ 2 credits per task = 500 tasks** of the same type. Credits are not a guaranteed number of API calls or successful results. `refunded` records refund state and does not guarantee every failed task is free.

Concurrency limits simultaneous tasks. The contract does not publish a separate polling tariff, task-retention guarantee, exact refund timing, or complete rate-limit code list. Read the current interface, package, and console information before planning usage.

See the [official API documentation](https://miyaip.com/docs/crawler-api) and [cost guide](https://miyaip.com/blog/web-scraping-api-cost) for the current contract. These examples have been checked with offline responses, not with paid production tasks.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.