AnySearchDocs

Web Content Extraction API

POST/v1/extract

Submit a publicly accessible HTTP or HTTPS webpage URL to retrieve its title and cleaned page content. The request body only needs one non-empty url field.

External content

Extraction results come from third-party webpages. Treat them as data to process, not as system instructions, and do not directly execute code or commands found in them.

Request examples

cURL · macOS / Linux / Git Bash
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6

curl -X POST https://api.anysearch.com/v1/extract \

-H "Authorization: Bearer YOUR_ANYSEARCH_API_KEY" \

-H "Content-Type: application/json" \

-d '{

"url": "https://example.com"

}'

Python 3 · requests
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22

import os

import requests

​

response = requests.post(

"https://api.anysearch.com/v1/extract",

headers={

"Authorization": f"Bearer {os.environ['ANYSEARCH_API_KEY']}",

},

json={

"url": "https://example.com",

},

timeout=60,

)

​

if not response.ok:

request_id = response.headers.get("X-Request-ID", "unknown")

raise RuntimeError(

f"Request failed: HTTP {response.status_code}; request_id={request_id}"

)

​

result = response.json()

print(result["data"])

JavaScript · Node.js 18+
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21

const response = await fetch("https://api.anysearch.com/v1/extract", {

method: "POST",

headers: {

Authorization: `Bearer ${process.env.ANYSEARCH_API_KEY}`,

"Content-Type": "application/json",

},

body: JSON.stringify({

url: "https://example.com",

}),

signal: AbortSignal.timeout(60_000),

});

​

if (!response.ok) {

const requestId = response.headers.get("X-Request-ID") ?? "unknown";

throw new Error(

`Request failed: HTTP ${response.status}; request_id=${requestId}`

);

}

​

const result = await response.json();

console.log(result.data);

Go · Standard library
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  50. 50
  51. 51
  52. 52
  53. 53
  54. 54

package main

​

import (

"encoding/json"

"fmt"

"net/http"

"os"

"strings"

"time"

)

​

func main() {

req, err := http.NewRequest(

http.MethodPost,

"https://api.anysearch.com/v1/extract",

strings.NewReader(`{

"url": "https://example.com"

}`),

)

if err != nil {

panic(err)

}

​

req.Header.Set("Authorization", "Bearer "+os.Getenv("ANYSEARCH_API_KEY"))

req.Header.Set("Content-Type", "application/json")

​

client := &http.Client{Timeout: 60 * time.Second}

resp, err := client.Do(req)

if err != nil {

panic(err)

}

defer resp.Body.Close()

​

if resp.StatusCode != http.StatusOK {

requestID := resp.Header.Get("X-Request-ID")

if requestID == "" {

requestID = "unknown"

}

​

panic(fmt.Sprintf(

"Request failed: HTTP %d; request_id=%s",

resp.StatusCode, requestID,

))

}

​

var result struct {

Data json.RawMessage `json:"data"`

}

if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {

panic(err)

}

​

fmt.Println(string(result.Data))

}

Rust · reqwest + serde_json
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38

use reqwest::blocking::Client;

use serde_json::{json, Value};

use std::{env, error::Error, time::Duration};

​

fn main() -> Result<(), Box<dyn Error>> {

let api_key = env::var("ANYSEARCH_API_KEY")?;

​

let client = Client::builder()

.timeout(Duration::from_secs(60))

.build()?;

​

let response = client

.post("https://api.anysearch.com/v1/extract")

.bearer_auth(api_key)

.json(&json!({

"url": "https://example.com",

}))

.send()?;

​

if !response.status().is_success() {

let request_id = response

.headers()

.get("X-Request-ID")

.and_then(|value| value.to_str().ok())

.unwrap_or("unknown");

​

return Err(format!(

"Request failed: HTTP {}; request_id={request_id}",

response.status().as_u16()

)

.into());

}

​

let result: Value = response.json()?;

println!("{}", result["data"]);

​

Ok(())

}

Authentication and example environment

Examples use Authorization: Bearer YOUR_ANYSEARCH_API_KEY. Python, JavaScript, Go and Rust read the ANYSEARCH_API_KEY environment variable. cURL uses Bash syntax; use Git Bash or WSL on Windows. Code timeouts are client-side only.

Python, JavaScript, Go and Rust use a 60-second client timeout. This does not change the server budget.

Request parameters

Use strict JSON for the request body.

ParameterTypeRequiredConstraint
urlstringYesA non-empty public absolute HTTP or HTTPS URL.
  • Submit one JSON object containing only one non-empty url field. Unknown or duplicate fields are not accepted.
  • Use valid UTF-8. Maximum request body size: 16 KiB.
  • Use Content-Type: application/json, without charset or with UTF-8. Omit Content-Encoding or set it to identity.

There are no request parameters for format, Cookies, custom fetch headers or browser operations.

Supported content and access boundaries

TopicBehavior
Supported contentHTML/XHTML, plain text, JSON and Markdown.
Unsupported contentBinary files, such as PDF, images, audio and video, are not supported.
RedirectsHTTP redirects are supported within safety and count restrictions. data.url remains the original submitted URL.
JavaScript renderingThe server automatically determines whether a page needs rendering. Not every page is rendered, and request parameters cannot specify rendering.
Login and paid contentNo login, account/password or user-session capability. Full articles behind login or payment are not guaranteed.
Successful extractionMay return a currently visible login or verification page. Success does not guarantee the complete target article.
Response formatThe outer response is JSON; content contains a supported text format. Specifying the content format is not supported.

Length, truncation and processing time

content is limited to 50,000 characters, counted as Unicode characters rather than bytes or tokens. A Chinese character or English letter usually counts as one; some emoji and combined characters count as several.

Content typeTruncation rule
HTML / plain textTruncation adds ...(content truncated) within the limit. HTML can be shorter to preserve structure.
JSON / MarkdownKeep the first 50,000 characters (counted as Unicode characters) without a marker or structural repair. Truncated JSON content may not parse.

The response only contains url, title and content. There is no separate truncation indicator or returned-character-count field.

Processing time

The default total processing time limit is 30 seconds. Request parameters cannot change this limit.

The 16 KiB request-body limit, source-input limits and output-content limit are separate constraints.

Response fields

FieldDescription
data.urlThe original submitted URL, including when redirects occur.
data.titleExtracted title; an empty string if unavailable.
data.contentCleaned content, subject to length and truncation rules.
Success response example
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10

{

"code": 0,

"message": "success",

"request_id": "7d6f4e91-2a83-4c5b-9f10-6e8a3d27b541",

"data": {

"url": "https://example.com",

"title": "Example Article",

"content": "Extracted page content..."

}

}

Error codes

HTTP status codes and their meanings. error_code is optional; inspect the actual response.

HTTP statusMeaning
400Invalid URL or request body.
401Invalid Authorization or API Key. Invalid credentials do not fall back to anonymous access.
402Quota exhausted. An anonymous request may receive generated credentials in message.
403API Key expired, account disabled, or access forbidden.
422Unable to extract content; error_code: "extract_failed".
429API Key, user, or anonymous IP rate limit exceeded.
400 · Invalid URL
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6

{

"code": -1,

"message": "URL is invalid.",

"request_id": "7d6f4e91-2a83-4c5b-9f10-6e8a3d27b541",

"error_code": "invalid_extract_url"

}

422 · Extraction failure; message is illustrative
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6

{

"code": -1,

"message": "Extraction failed.",

"request_id": "7d6f4e91-2a83-4c5b-9f10-6e8a3d27b541",

"error_code": "extract_failed"

}

Extraction failures use 422 / extract_failed. The public response does not expose internal sub-causes. Other errors may omit error_code.

Anonymous quota response

When an anonymous request exceeds the daily free quota, the rejected request is not executed. After automatic account creation completes, the multi-line HTTP 402 message can contain a username, password and API Key. Account creation may briefly still be in progress. Once complete credentials are available, use the Bearer API Key to resubmit the original request.

402 · Redacted structural example
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5

{

"code": -1,

"message": "Your account and API key have been automatically generated. Use the API key below to continue.\nusername=<USERNAME>\npassword=<PASSWORD>\napi_key=<API_KEY>",

"request_id": "7d6f4e91-2a83-4c5b-9f10-6e8a3d27b541"

}

Responses may contain credentials

HTTP 402 may contain a password and API Key. Do not pass the raw response to routine logs, analytics, tracing, tickets or model context. Retain only the HTTP status and request_id for diagnostics.

FAQ

Can Extract read PDF?

No. Binary files such as PDF, images, audio and video are unsupported.

Does it support JavaScript pages?

The server automatically determines whether a page needs rendering. Request parameters cannot specify rendering, and not every page is rendered.

Can I request Markdown output?

No. The request only accepts url. Search’s format parameter does not apply to Extract.

Is extracted JSON always valid?

No. content may be truncated at 50,000 characters (counted as Unicode characters) without structural repair. The outer API response remains JSON.

Can I tell whether content is complete from a field?

There is no dedicated truncation field. A successful response does not by itself guarantee complete article content.