Skip to content

Latest commit

 

History

History
906 lines (630 loc) · 32.1 KB

API_REFERENCE.md

File metadata and controls

906 lines (630 loc) · 32.1 KB

API Reference

Every HTTP operation Headwater exposes: 70 operations on 60 paths, 54 of them under /api/v1. This file is generated from the application's OpenAPI schema, so it lists what the code actually serves. To refresh it after changing an endpoint, run python scripts/generate_api_reference.py.

Basics

  • Base path: the data APIs live under /api/v1 (for example /api/v1/google-news/search/). The health and configuration endpoints sit at the root.
  • Authentication: send your key in the X-API-Key request header. Operations marked Auth: none below accept requests without it.
  • Interactive docs: outside production the service serves Swagger UI at /api/docs, ReDoc at /api/redoc and the raw OpenAPI schema at /openapi.json. In production none of these are registered, so fetch the schema from a non-production instance or run this script.
  • Schemas: request and response types named below are the component schemas in the OpenAPI document (components.schemas), where their fields are listed in full.

Contents

Google Autocomplete API

GET /api/v1/google-autocomplete/autocomplete

Get Google Autocomplete suggestions

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
aqs query string | null no Assisted query stats
authuser query integer | null no Google account index (0, 1, 2...)
callback query string | null no JSONP callback function name
client query ClientType | null no Client type (chrome, firefox, safari, opera)
complete query integer | null no Completion type
cp query integer | null no Cursor position in query
cr query string | null no Country restrict (e.g., countryUS)
ds query DataSource | null no Data source: yt=YouTube, i=Images, n=News, s=Shopping, b=Books, fin=Finance
filter query integer | null no Filter duplicates (0=off, 1=on)
gl query string no "US" Country code (ISO)
gs_id query string | null no Session ID
gs_l query string | null no Google location codes
gs_rn query integer | null no Request sequence number
hl query string no "en" Language code (ISO)
ie query string | null no "UTF-8" Input encoding
jsonp query string | null no JSONP wrapper (alt to callback)
lr query string | null no Language restrict (e.g., lang_en)
nfpr query integer | null no Disable auto-correct (0=on, 1=off)
oe query string | null no "UTF-8" Output encoding
oq query string | null no Original typed query
output query OutputFormat no "toolbar" Response format (chrome/firefox=JSON with metadata, toolbar/xml=basic XML)
pq query string | null no Previous query for refinement
psi query integer | null no Personalized search (0=off, 1=on)
pws query integer | null no Personalized web search (0=off, 1=on)
q query string yes Search query
safe query SafeSearch | null no SafeSearch filtering (active=filter, off=show all)
sclient query SearchClient | null no Search client ID
spell query integer | null no Spell correction (0=off, 1=on)
suggid query string | null no Suggestion tracking ID
variations query boolean | null no false Return keyword variations instead of raw suggestions
xssi query string | null no XSSI protection (t=on, f=off)

Google News API

GET /api/v1/google-news/article-details/

Article Details

  • Auth: X-API-Key header
  • Response 200: object
Name In Type Required Default Description
url query string yes Article URL to analyze

GET /api/v1/google-news/articles/

Bulk Articles

  • Auth: X-API-Key header
  • Response 200: NewsResponse
Name In Type Required Default Description
country query string no "US" Country code
language query string no "en" Language code
max_results query integer no 5 Max results (1-100)
period query string no "1d" Period: 7d, 1w, 1m, 1y
query query string no "news" Search query

GET /api/v1/google-news/available-countries/

Available Countries

  • Auth: X-API-Key header
  • Response 200: object

GET /api/v1/google-news/available-languages/

Available Languages

  • Auth: X-API-Key header
  • Response 200: object

GET /api/v1/google-news/location/

News by Location

  • Auth: X-API-Key header
  • Response 200: NewsResponse
Name In Type Required Default Description
country query string no "US" Country code
end_date query string | null no End date (YYYY-MM-DD)
exclude_duplicates query boolean no false Exclude duplicates
language query string no "en" Language code
location query string yes Location name
max_results query integer no 5 Max results (1-100)
start_date query string | null no Start date (YYYY-MM-DD)

GET /api/v1/google-news/search/

Search News

  • Auth: X-API-Key header
  • Response 200: NewsResponse
Name In Type Required Default Description
country query string no "US" Country code
end_date query string | null no End date (YYYY-MM-DD)
exact_match query boolean no false Exact match only
exclude_duplicates query boolean no false Exclude duplicates
language query string no "en" Language code
max_results query integer no 5 Max results (1-100)
query query string yes Search query
sort_by query string no "relevance" Sort by: relevance, date
start_date query string | null no Start date (YYYY-MM-DD)

GET /api/v1/google-news/source/

News by Source

  • Auth: X-API-Key header
  • Response 200: NewsResponse
Name In Type Required Default Description
country query string no "US" Country code
end_date query string | null no End date (YYYY-MM-DD)
exclude_duplicates query boolean no false Exclude duplicates
language query string no "en" Language code
max_results query integer no 5 Max results (1-100)
source query string yes Source domain or URL
start_date query string | null no Start date (YYYY-MM-DD)

GET /api/v1/google-news/top/

Top News

  • Auth: X-API-Key header
  • Response 200: NewsResponse
Name In Type Required Default Description
country query string no "US" Country code
language query string no "en" Language code
max_results query integer no 10 Max results (1-100)

GET /api/v1/google-news/topic/

News by Topic

  • Auth: X-API-Key header
  • Response 200: NewsResponse
Name In Type Required Default Description
country query string no "US" Country code
exclude_duplicates query boolean no false Exclude duplicates
language query string no "en" Language code
max_results query integer no 5 Max results (1-100)
topic query string yes Topic name (WORLD, TECHNOLOGY, SPORTS, etc.)

Google Trends API

GET /api/v1/google-trends/categories

Categories

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
find query string | null no Search category names
root query string | null no Root category ID for subcategories

GET /api/v1/google-trends/geo

Geolocations

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
find query string | null no Search location names

GET /api/v1/google-trends/interest-by-region

Interest By Region

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
cat query string | null no Category ID
geo query string | null no Location code (US, GB)
keyword query string yes Single keyword
resolution query string no "COUNTRY" Detail level: COUNTRY, REGION, CITY, DMA
timeframe query string no "today 12-m" Time range

GET /api/v1/google-trends/interest-over-time

Interest Over Time

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
cat query string | null no Category ID (e.g., 13=Computers)
geo query string | null no Location code (US, US-NY, GB)
gprop query string | null no Property: images, youtube, news, froogle
keywords query string yes Comma-separated keywords
timeframe query string no "today 12-m" Time range: now 1-H, now 4-H, today 1-m, today 3-m, today 12-m

GET /api/v1/google-trends/related-queries

Related Queries

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
cat query string | null no Category ID
geo query string | null no Location code (US, GB)
gprop query string | null no Property: images, youtube, news, froogle
keyword query string yes Single keyword
timeframe query string no "today 12-m" Time range

GET /api/v1/google-trends/related-topics

Related Topics

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
cat query string | null no Category ID
geo query string | null no Location code (US, GB)
gprop query string | null no Property: images, youtube, news, froogle
keyword query string yes Single keyword
timeframe query string no "today 12-m" Time range

GET /api/v1/google-trends/trending-now

Trending Now

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
geo query string | null no "US" Location code (US, GB)

GET /api/v1/google-trends/trending-now-by-rss

Trending Now (RSS)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
geo query string | null no "US" Location code (US, GB)

GET /api/v1/google-trends/trending-now-news-by-ids

News by IDs

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
geo query string no "US" Location of the /trending-now call the IDs came from; applies to bare IDs only
max_news query integer no 3 Max articles to retrieve (1-50)
news_tokens query string yes News tokens from /trending-now: comma-separated numeric IDs (4830466997,4830466998), or its news_tokens JSON array ([[4830466997,"en","US"]]). At most 50.

GET /api/v1/google-trends/trending-now-showcase-timeline

Trending Timeline

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
keywords query string yes Comma-separated keywords
timeframe query HumanFriendlyBatchPeriod yes Time range: past_4h, past_24h, past_48h, past_7d

YouTube Transcripts API

POST /api/v1/youtube-transcripts/batch-get-transcripts

Batch Get Transcripts

  • Auth: X-API-Key header
  • Request body (optional): BatchTranscriptRequest | null
  • Response 200: array[TranscriptResponse]
Name In Type Required Default Description
languages query array[string] | null no Language codes by priority
preserve_formatting query boolean | null no Preserve HTML formatting
video_ids query array[string] | null no List of video IDs (query form)

GET /api/v1/youtube-transcripts/format-transcript

Format Transcript

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
format_type query string no "json" Output format: json, txt, vtt, srt, csv
languages query array[string] | null no ["en"] Language codes by priority
video_id query string yes YouTube video ID

GET /api/v1/youtube-transcripts/get-transcript

Get Transcript

  • Auth: X-API-Key header
  • Response 200: TranscriptResponse
Name In Type Required Default Description
languages query array[string] | null no ["en"] Language codes by priority
preserve_formatting query boolean no false Preserve HTML formatting
video_id query string yes YouTube video ID

GET /api/v1/youtube-transcripts/list-transcripts

List Transcripts

  • Auth: X-API-Key header
  • Response 200: TranscriptListResponse
Name In Type Required Default Description
video_id query string yes YouTube video ID

GET /api/v1/youtube-transcripts/translate-transcript

Translate Transcript

  • Auth: X-API-Key header
  • Response 200: TranscriptResponse
Name In Type Required Default Description
source_languages query array[string] | null no ["en"] Source language codes
target_language query string yes Target language code
video_id query string yes YouTube video ID

Google Maps API

GET /api/v1/google-maps/autocomplete

Place autocomplete

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
input query string yes Search input
language query string no "en" Language code
latitude query number | null no Bias latitude
longitude query number | null no Bias longitude
radius_meters query integer | null no Bias radius
types query string | null no Place types filter

POST /api/v1/google-maps/bounding-box-search

Search within a bounding box

  • Auth: X-API-Key header
  • Request body (required): BoundingBoxRequest
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
timeout query integer no 600 Timeout in seconds

POST /api/v1/google-maps/bulk-search

Bulk search places

  • Auth: X-API-Key header
  • Request body (required): BulkSearchRequest
  • Response 200: JSON (no response model declared)

POST /api/v1/google-maps/competitors

Analyze competitors

  • Auth: X-API-Key header
  • Request body (required): CompetitorRequest
  • Response 200: JSON (no response model declared)

GET /api/v1/google-maps/directions

Get directions (GET)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
alternatives query boolean no false Return alternatives
destination_lat query number yes Destination latitude
destination_lng query number yes Destination longitude
mode query string no "driving" Travel mode
origin_lat query number yes Origin latitude
origin_lng query number yes Origin longitude

POST /api/v1/google-maps/directions

Get directions

  • Auth: X-API-Key header
  • Request body (required): DirectionsRequest
  • Response 200: JSON (no response model declared)

GET /api/v1/google-maps/geocode

Geocode address (GET)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
address query string yes Address to geocode

POST /api/v1/google-maps/geocode

Batch geocode addresses

  • Auth: X-API-Key header
  • Request body (required): GeocodeRequest
  • Response 200: JSON (no response model declared)

GET /api/v1/google-maps/grid-search

Grid-based search (GET)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
center_lat query number yes Center latitude
center_lng query number yes Center longitude
grid_size query integer no 5 Grid dimension
language query string no "en" Language code
max_results_per_point query integer no 10 Results per point
query query string yes Search query
radius_km query number no 5.0 Radius in km
timeout query integer no 600 Timeout

POST /api/v1/google-maps/grid-search

Grid-based search for comprehensive area coverage

  • Auth: X-API-Key header
  • Request body (required): GridSearchRequest
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
timeout query integer no 600 Timeout in seconds

GET /api/v1/google-maps/health

Check Google Maps scraper health

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)

GET /api/v1/google-maps/jobs

List all scraping jobs

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
limit query integer no 50 Maximum jobs to return
offset query integer no 0 Pagination offset
status query string | null no Filter by status (pending, running, completed, failed)

GET /api/v1/google-maps/jobs/{job_id}

Get job status

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
job_id path string yes Job ID to check

DELETE /api/v1/google-maps/jobs/{job_id}

Delete a job

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
job_id path string yes Job ID to delete

GET /api/v1/google-maps/jobs/{job_id}/export

Export job results

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
job_id path string yes Job ID
format query ExportFormat no "json" Export format

GET /api/v1/google-maps/jobs/{job_id}/results

Get job results

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
job_id path string yes Job ID to get results for
format query string no "json" Output format (json, csv)

GET /api/v1/google-maps/location-search

Search by location name (GET)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
grid_size query integer no 5 Grid dimension
language query string no "en" Language code
location query string yes Location name (city, ZIP, address)
max_results_per_point query integer no 10 Results per point
query query string yes Search query
radius_km query number no 5.0 Radius in km
timeout query integer no 600 Timeout

POST /api/v1/google-maps/location-search

Search by location name

  • Auth: X-API-Key header
  • Request body (required): LocationSearchRequest
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
timeout query integer no 600 Timeout in seconds

GET /api/v1/google-maps/monitors

List monitors

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
limit query integer no 50 Maximum monitors
offset query integer no 0 Pagination offset
status query string | null no Filter by status

POST /api/v1/google-maps/monitors

Create place monitor

  • Auth: X-API-Key header
  • Request body (required): MonitorRequest
  • Response 200: JSON (no response model declared)

GET /api/v1/google-maps/monitors/{monitor_id}

Get monitor status

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
monitor_id path string yes Monitor ID
include_history query boolean no true Include change history

DELETE /api/v1/google-maps/monitors/{monitor_id}

Delete monitor

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
monitor_id path string yes Monitor ID

GET /api/v1/google-maps/nearby

Search nearby places (GET)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
language query string no "en" Language code
latitude query number yes Center latitude
longitude query number yes Center longitude
max_results query integer no 20 Maximum results
query query string | null no Filter query
radius_meters query integer no 1000 Search radius

POST /api/v1/google-maps/nearby

Search nearby places

  • Auth: X-API-Key header
  • Request body (required): NearbySearchRequest
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
timeout query integer no 300 Timeout in seconds
wait_for_results query boolean no true Wait for results

POST /api/v1/google-maps/place/lookup

Lookup place by URL or ID

  • Auth: X-API-Key header
  • Request body (required): PlaceLookupRequest
  • Response 200: JSON (no response model declared)

GET /api/v1/google-maps/place/{place_id}

Get place details by ID

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Google Place ID (CID) or URL-encoded place identifier

GET /api/v1/google-maps/place/{place_id}/analytics

Get review analytics

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
include_keywords query boolean no true Include keyword extraction
include_sentiment query boolean no true Include sentiment analysis
include_trends query boolean no true Include rating trends
time_period query string no "all" Time period

GET /api/v1/google-maps/place/{place_id}/attributes

Get place attributes

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID

GET /api/v1/google-maps/place/{place_id}/availability

Check reservation availability

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
date query string yes Date to check (YYYY-MM-DD)
party_size query integer no 2 Number of guests

GET /api/v1/google-maps/place/{place_id}/history

Get place history

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
end_date query string | null no End date (YYYY-MM-DD)
field query string | null no Specific field to get history for
start_date query string | null no Start date (YYYY-MM-DD)

GET /api/v1/google-maps/place/{place_id}/menu

Extract menu

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
categorize query boolean no true Categorize items
include_descriptions query boolean no true Include descriptions
include_prices query boolean no true Include prices

GET /api/v1/google-maps/place/{place_id}/photos

Get place photos

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
category query string | null no Photo category filter
max_photos query integer no 20 Maximum photos
size query string no "large" Photo size (thumbnail, medium, large, original)

GET /api/v1/google-maps/place/{place_id}/qa

Get place Q&A

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
include_answers query boolean no true Include answers
limit query integer no 20 Maximum Q&A pairs

GET /api/v1/google-maps/place/{place_id}/reviews

Get place reviews

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
place_id path string yes Place ID
include_owner_responses query boolean no true Include owner responses
limit query integer no 50 Number of reviews
min_rating query integer | null no Minimum rating filter
offset query integer no 0 Pagination offset
sort_by query string no "most_relevant" Sort order

GET /api/v1/google-maps/search

Search Google Maps places (GET)

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
depth query integer no 1 Crawl depth
email_extraction query boolean no false Extract emails
geo_coordinates query string | null no Search center (lat,lng)
language query string no "en" Language code
max_results query integer no 20 Maximum results (~12s each)
query query string yes Search query
timeout query integer no 300 Timeout in seconds
wait_for_results query boolean no true Wait for results
zoom query integer no 15 Map zoom level

POST /api/v1/google-maps/search

Search Google Maps places

  • Auth: X-API-Key header
  • Request body (required): SearchRequest
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
timeout query integer no 300 Maximum seconds to wait for results (if wait_for_results=True)
wait_for_results query boolean no true Wait for results (True) or return job ID immediately (False)

GET /api/v1/google-maps/webhooks

List webhooks

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)

POST /api/v1/google-maps/webhooks

Register webhook

  • Auth: X-API-Key header
  • Request body (required): WebhookRequest
  • Response 200: JSON (no response model declared)

DELETE /api/v1/google-maps/webhooks/{webhook_id}

Delete webhook

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)
Name In Type Required Default Description
webhook_id path string yes Webhook ID

Health

GET /health

Basic health check

  • Auth: none
  • Response 200: JSON (no response model declared)

GET /health/detailed

Detailed health check

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)

GET /ping

Simple ping endpoint

  • Auth: none
  • Response 200: JSON (no response model declared)

GET /status

Application status

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)

Configuration

GET /api-config

API configuration

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)

GET /config-sources

Configuration sources

  • Auth: X-API-Key header
  • Response 200: JSON (no response model declared)