ATTEXDocsattexdata.ai ↗
Contents

Load the export

Change feed

Every pull returns everything that is new since your last pull: impressions that were added, impressions that changed, and impressions that were removed. With it comes a cursor. Pass the cursor with your next pull and you get whatever happened after it; upsert the rows on impression_id, and your table holds the whole export.

The change feed covers mobile rows ("version": "v2"). Desktop rows come through the day export: Day export →

It does not matter how often you fetch. Pull every hour or once a week and the result is the same; a pull that failed, or a week you skipped, is covered by the next one. There are no dates to keep track of and no days to retry. Data that arrives late, and updates to earlier days, come with the next pull like everything else.

1. Authenticate

The steps show each raw call in curl; the Python tab builds them into one loader.py with requests and duckdb.

Every request carries your key in Authorization, without a scheme prefix. Requests without a key get 401; a key outside its entitlements gets 403.

export ATTEX_API="https://global-api.attentionpanel.com"
export ATTEX_API_KEY="…"

2. Ask for what is new

A pull is one request. The first time, send "feed": true: you get everything your key may see, from its first allowed day. After that, send "since" with the cursor from your last pull: you get everything added, changed or removed after it. One feed covers one market.

All fields and errors: Create an export →

# The first pull
curl -sS -X POST "$ATTEX_API/impressions/export" \
  -H "Authorization: $ATTEX_API_KEY" -H "Content-Type: application/json" \
  -d '{"device":"mobile","market":"semarket","version":"v2","feed":true}'

# Every pull after that
curl -sS -X POST "$ATTEX_API/impressions/export" \
  -H "Authorization: $ATTEX_API_KEY" -H "Content-Type: application/json" \
  -d "{\"device\":\"mobile\",\"market\":\"semarket\",\"version\":\"v2\",\"since\":\"$CURSOR\"}"
Response
{ "export_id": "acme_3f6c1d2e-8b4a-4c55-9a77-2d1f0e9b6a10", "status": "RUNNING", "version": "v2" }

3. Wait for it to be ready

A pull takes a few minutes, also when nothing is new. Check its status now and then, waiting a little longer each time, until it is no longer RUNNING. When it says SUCCEEDED, it carries the download links and next_cursor: the cursor for your next pull.

States, failure reasons and errors: Get an export →

curl -sS "$ATTEX_API/exports/$EXPORT_ID" -H "Authorization: $ATTEX_API_KEY"
Response
{
  "status": "SUCCEEDED",
  "version": "v2",
  "download_urls": [
    "https://…/acme_3f6c1d2e-8b4a-4c55-9a77-2d1f0e9b6a10_2026_10.tar.gz?X-Amz-Signature=…"
  ],
  "next_cursor": "eyJ2IjogMSwgImltcHJlc3Npb25zIjogMTQ2LCAiZW5yaWNobWVudCI6IDI5OX0"
}

4. Download the files

One link per calendar month with rows in the pull: <export_id>_<year>_<month>.tar.gz, holding a day=DD/ folder per day with gzipped JSON Lines files. A pull with nothing new has no links. URLs are presigned and valid for 24 hours after completion; do not send your API key to them, the signature is in the query string. Unpack each archive under year=<year>/month=<month>/ so the day folders line up as year=/month=/day=, and clear the folder first so the load sees exactly this pull.

Download links and their expiry: Get an export →

rm -rf feed/semarket
curl -sS "$ATTEX_API/exports/$EXPORT_ID" -H "Authorization: $ATTEX_API_KEY" \
  | jq -r '.download_urls[]' \
  | while read -r url; do
      name=$(basename "${url%%\?*}")                 # <export_id>_<year>_<month>.tar.gz
      ym=${name%.tar.gz}; month=${ym##*_}; year=${ym%_*}; year=${year##*_}
      dir="feed/semarket/year=$year/month=$month"
      mkdir -p "$dir" && curl -sS "$url" | tar -xz -C "$dir" --strip-components=1
    done

5. Upsert, and keep the cursor

A pull holds each impression it sends once: as it is now, or, when it is gone, as {"impression_id": "…", "date": "2026-10-21", "removed": true} under its day. Upsert on impression_id: delete every id in the pull from your table, then insert the rows that are not removed. A removed id you never had is a no-op. A first pull replaces the table. Read the files as JSON Lines with schema union, so a missing key is a null and a field that appears later becomes a new column.

DuckDB below; the same delete and insert works in BigQuery, Snowflake or Postgres. Read the pull into batch and gone first, then apply it and save the cursor in one transaction, so a run that fails part-way changes nothing and the next run repeats the same pull safely.

CREATE OR REPLACE TEMP TABLE batch AS
  SELECT * FROM read_json_auto('feed/semarket/**/*.json.gz', union_by_name = true, hive_partitioning = true);
CREATE OR REPLACE TEMP TABLE gone AS
  SELECT json ->> 'impression_id' AS impression_id
  FROM read_ndjson_objects('feed/semarket/**/*.json.gz') WHERE json ->> 'removed' = 'true';
CREATE TABLE IF NOT EXISTS impressions_mobile AS
  SELECT COLUMNS(lambda c: c NOT IN ('removed', 'date')) FROM batch LIMIT 0;

BEGIN;
DELETE FROM impressions_mobile WHERE impression_id IN (SELECT impression_id FROM batch);
INSERT INTO impressions_mobile BY NAME
  SELECT COLUMNS(lambda c: c NOT IN ('removed', 'date')) FROM batch
  WHERE impression_id NOT IN (SELECT impression_id FROM gone);
INSERT OR REPLACE INTO feed_cursor VALUES ('semarket', '<next_cursor>');
COMMIT;
db = duckdb.connect("warehouse.db")
db.execute("CREATE TABLE IF NOT EXISTS feed_cursor (market VARCHAR PRIMARY KEY, cursor VARCHAR)")

def read_pull(folder) -> None:
    files = f"{folder}/**/*.json.gz"
    db.execute(f"""CREATE OR REPLACE TEMP TABLE batch AS
        SELECT * FROM read_json_auto('{files}', union_by_name = true, hive_partitioning = true)""")
    db.execute(f"""CREATE OR REPLACE TEMP TABLE gone AS
        SELECT json ->> 'impression_id' AS impression_id
        FROM read_ndjson_objects('{files}') WHERE json ->> 'removed' = 'true'""")
    db.execute("""CREATE TABLE IF NOT EXISTS impressions_mobile AS
        SELECT COLUMNS(lambda c: c NOT IN ('removed', 'date')) FROM batch LIMIT 0""")
    have = {c for (c,) in db.execute(
        "SELECT column_name FROM information_schema.columns WHERE table_name = 'impressions_mobile'").fetchall()}
    for name, kind in db.execute(
            "SELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'batch'").fetchall():
        if name not in have and name not in ("removed", "date"):
            db.execute(f'ALTER TABLE impressions_mobile ADD COLUMN "{name}" {kind}')

def apply_pull(market, job, first) -> None:
    db.begin()
    if first:
        db.execute("DELETE FROM impressions_mobile")
    if job.get("download_urls"):
        db.execute("DELETE FROM impressions_mobile WHERE impression_id IN (SELECT impression_id FROM batch)")
        db.execute("""INSERT INTO impressions_mobile BY NAME
            SELECT COLUMNS(lambda c: c NOT IN ('removed', 'date')) FROM batch
            WHERE impression_id NOT IN (SELECT impression_id FROM gone)""")
    db.execute("INSERT OR REPLACE INTO feed_cursor VALUES (?, ?)", [market, job["next_cursor"]])
    db.commit()

6. Repeat whenever you like

Run the same steps again on any schedule; each pull starts from the cursor of the one before. We recommend a daily pull. Any schedule gives the same table, and a cursor does not expire, so after a failed run or a long pause the next pull simply catches up. Fetching every few minutes gains nothing, since a pull takes a few minutes itself.

# 06:00 UTC daily; any schedule works
0 6 * * *  python loader.py

Starting over

Occasionally a pull answers 400 CURSOR_GAP, with an action: changes after your cursor can no longer be sent, so start again with a first pull, which replaces your table (the loader above does this). Do the same when your access changes, for example a new market or an earlier first day, since a feed only sends changes to rows. A 400 CURSOR_INVALID means the cursor is not one the API issued: pass next_cursor back exactly as you received it.

Every error code: API reference →