Data Feeds

A data feed is a dataset backed by a data source. The public feed embeds the data source under a source object (the internal datasource_id is not exposed). In column formulas, a datasource-qualified reference is surfaced as the placeholder __source__@... and translated back to the real datasource id on write.

Generating a feed to build a metric on? See Creating a metric from a service, from a REST endpoint, or from uploaded data.

Full field reference (types, enums, required/read-only flags) is generated from components.schemas.DataFeed in the API Reference tab, and tabulated field by field — with what each one's read-only status means for a write — in Field Reference. Or fetch the live OpenAPI document directly, see Getting Started for the URL.

GET /data_feeds · GET /data_feeds/:feedId

{
  "id": "2feb372516d445bf95069cd09c974b25",
  "name": "Sales export",
  "description": "Nightly CSV of orders",
  "ownerUserId": "7c2f599e6dfb402996b2194ab9b11164",
  "is_locked": false,
  "last_refresh_outcome": "success",
  "last_successful_refresh_timestamp": "2026-05-28T06:00:00.000Z",
  "columns": [
    { "id": "e190a3817d86438da7e45315321b8572", "name": "Order ID", "type": "TEXT", "formula": "__source__@order_id" },
    { "id": "2e06aa1c3fed4935ad1c8e26a1d6beb3", "name": "Amount", "type": "NUMERIC", "aggregation": "sum", "formula": "__source__@amount" }
  ],
  "source": {
    "connector": "local",
    "format": "csv",
    "refresh_interval": 14400,
    "is_dynamic": false,
    "disabled": false
  },
  "version": "1.0"
}

last_refresh_outcome is lowercase — one of success or error. refresh_interval is a plain number of seconds (e.g. 14400 = every 4 hours, 0 = never automatically refreshed). A column's fmtArgs, when present, is an object whose shape varies by the column's format.

source reports read-side connection metadata, plus — for a feed generated from a service connection — the source.query the data source pulls, in the same shape POST /data_feeds/generate?type=queryBuilder takes it, each field named by name. Read it before replacing it: a query is replaced wholesale, so a caller that has not read the current field list cannot send a complete one.

An entry marked "hidden": true is a parameter the service needs in order to answer the query at all, rather than a column of the feed. Send it back unchanged, or leave it out and it is carried over for you.

source.query is writable — see Changing the query a feed pulls. The rest of the data source's connector configuration — request headers, credentials and connection details, i.e. the properties you set on generate/update — is not returned.

POST /data_feeds/generate

Generate a new feed. Selects a strategy via the type query parameter.

• type=queryBuilder — body requires a connectionId, viewName and query. The gateway resolves the connection and delegates feed generation.
• type=dataUpload — the body is the data (a CSV, spreadsheet, JSON or XML payload), described by the Content-Type header. See Uploading data below.
• type=rest — body requires a url, method and contentType. The gateway creates a data source that fetches the endpoint. See Reading a REST endpoint below.

A service only your user can connect can't be generated at all — ask for a link instead, with GET /data_feeds/creation_url below.

POST /data_feeds/generate?type=queryBuilder

{
  "connectionId": "cd882b55c245465e93857cd331cb3ffa",
  "viewName": "orders",
  "query": {
    "fields": [
      { "name": "NewUsers", "filters": [], "filterOption": "all" },
      { "name": "Date", "filters": [], "filterOption": "all" }
    ]
  }
}

Returns 201 with the meta/data envelope described in Getting Started (public shape, with embedded datasource). An unsupported type returns 400.

The query field

query is a JSON object describing which fields to pull and how to filter them. Use GET /query_builder/services/:serviceName/views/:viewName/fields to discover the available field names for a view.

• fields — the fields to include, in order. Each entry has:
  • name — the field's name as returned by that endpoint (e.g. NewUsers, Date, City).
  • filters — zero or more filters applied to that field. Each filter has:
    • values — the values to match (an array, even for a single value).
    • op — the comparison operator (e.g. eq for equals).
    • inverted — true to negate the filter (i.e. "not").
  • filterOption — how a field's own filters combine: all (every filter must match) or any (at least one must match).

Every field must be one the view has, and may be named at most once. A name the view does not have returns 400 listing the names it does — it is checked here because the connector does not refuse one: it reads an unrecognised field as empty, which would generate a feed with a column that holds nothing and never will. Naming a field twice would ask for the same data in two columns, and returns 400 naming the field. A field with an empty filters array is returned unfiltered; filterOption is then irrelevant. The example below returns NewUsers and Date unfiltered, and City restricted to rows matching either Washington or Boston:

{
  "fields": [
    { "name": "NewUsers", "filters": [], "filterOption": "all" },
    { "name": "Date", "filters": [], "filterOption": "all" },
    {
      "name": "City",
      "filters": [
        { "values": ["Washington"], "op": "eq", "inverted": false },
        { "values": ["Boston"], "op": "eq", "inverted": false }
      ],
      "filterOption": "any"
    }
  ]
}

Uploading data (type=dataUpload)

Send the data as the request body — there is no JSON envelope — and describe it with the Content-Type header. The gateway creates a data source to hold the uploaded data, builds a feed on it, and models its columns automatically.

curl -X POST \
  -H "Authorization: Bearer kf_live_<keyid>.<secret>.<checksum>" \
  -H "Content-Type: text/csv" \
  --data-binary @orders.csv \
  "https://api-public.klipfolio.com/v1.0-beta/data_feeds/generate?type=dataUpload&name=Orders"

Because the body is the data, the feed's own details travel in the query string:

An empty body returns 400.

For csv and xls the returned feed is fully modelled — its columns are detected from the data and typed. json and xml uploads are stored and the feed is created, but its columns have to be modelled separately.

The first row is taken as the header row. If any step fails, whatever was created along the way is removed, so a failed call leaves no partial feed behind.

Returns 201 with the meta/data envelope described in Getting Started, data in the same public shape as GET /data_feeds/:feedId. Replace the data later with POST /data_feeds/:feedId/raw_data.

Reading a REST endpoint (type=rest)

Describe an HTTP endpoint and the gateway creates a simple_rest data source that fetches it, builds a feed on that source, and models its columns automatically — the same shape as an upload, with the data pulled instead of pushed.

POST /data_feeds/generate?type=rest

{
  "url": "https://api.example.com/v1/orders?since=2026-01-01",
  "method": "GET",
  "contentType": "csv",
  "name": "Orders",
  "description": "Nightly orders",
  "properties": {
    "parameters": "[{\"name\":\"Authorization\",\"value\":\"Bearer abc\",\"type\":\"header\"}]"
  }
}

Any further field is passed straight through to the data source, and overrides what the gateway would otherwise set — so connector settings it does not name itself (is_dynamic, disabled, …) can be set on the same call.

The gateway sets connector to simple_rest, format from contentType, and properties.endpoint_url / properties.method from the url and method.

refresh_interval defaults to 14400 — four hours, in the seconds the field is counted in. Pass your own to override it (3600 hourly, 86400 daily).

Which endpoints can be fetched

The endpoint is fetched by Klipfolio, not by your client, so it has to be one reachable from the public internet. A url naming an address inside our own network is rejected with 400:

• localhost, and the .localhost, .local, .internal and .home.arpa names
• loopback and "this host" (127.0.0.0/8, 0.0.0.0/8)
• private ranges (10/8, 172.16/12, 192.168/16) and carrier-grade NAT (100.64/10)
• link-local (169.254/16), which is where cloud metadata endpoints live
• the IPv6 equivalents (::1, fc00::/7, fe80::/10), including an IPv4 address embedded in one

Alternative spellings of those addresses are rejected too — http://2130706433/ and http://0177.0.0.1/ are both 127.0.0.1, and a user@host prefix does not change which host is fetched. Only http and https are fetched at all.

As with dataUpload, csv and xls feeds come back fully modelled, json and xml have to be modelled separately, and a failure part-way through removes whatever was created along the way.

Returns 201 with the generated feed in the same public shape as GET /data_feeds/:feedId.

GET /data_feeds/creation_url

Some services can't be connected on a user's behalf — authorizing the service, choosing which account or property to read, and so on has to be done by the person who owns the data. There is no API call that can create those feeds, so ask for a link and hand it to your user instead:

GET /data_feeds/creation_url?service=google_adwords

{
  "url": "https://app.klipfolio.com/trends/data-feed/create-data-feed?service=google_adwords"
}

Send your user to that url and Klipfolio opens data feed creation with the service already selected, ready for them to connect it.

Nothing is created by this call — the feed appears under GET /data_feeds only once your user finishes the flow, so poll that if you need to know when it does.

A missing service, or one that matches no service provider, returns 400.

PUT /data_feeds/:feedId

Update the feed and its data source together. Send the feed spec with a nested source object — the same field name the data source is returned under. The gateway updates the referenced data source in place and then the feed, and returns the updated feed in its public shape.

A source.query replacing the feed's own is checked the same way POST /data_feeds/generate checks one, with one difference: only the fields the replacement adds are checked. Names the query already had are left alone whatever the view reports now — a service renames a metric, a connection's scope changes, and a feed built when a field existed should not become uneditable when it stops existing. So a field you are adding that the view does not have returns 400 and nothing is written, while one that was already there is carried over as it is (and can still be removed).

Modeling columns on a json/xml feed

json and xml feeds (from either generate?type=dataUpload or generate?type=rest) come back with no columns — unlike csv/xls, nothing detects a schema for them automatically. Model them with a follow-up PUT /data_feeds/:feedId that sends a hand-written columns array (the same shape GET /data_feeds/:feedId returns them in), omitting source:

{
  "columns": [
    { "name": "Order ID", "type": "TEXT", "formula": "__source__@order_id" },
    { "name": "Amount", "type": "NUMERIC", "aggregation": "sum", "formula": "__source__@amount" }
  ]
}

Each column's formula is a __source__@<field> reference to a field in the raw data (the same placeholder columns are returned with — see the note at the top of this guide). type is TEXT, NUMERIC or DATE. The order of the array is the feed's column order, and a write replaces the whole array — a column left out of it is removed.

This names the field, which is how an uploaded or REST-backed feed's raw data is addressed. A feed generated from a service connection is not addressed that way: its references are positional (__source__@C:C;), and what sits at a position is fixed by its query — see below.

Changing the query a feed pulls

A feed generated from a service connection pulls a query: the list of fields it asks the service for. Send it back under source.query, in the same shape generate takes it, and the data source is written with the new query and the feed with the columns you sent alongside it:

PUT /data_feeds/2feb372516d445bf95069cd09c974b25

{
  "source": {
    "query": {
      "fields": [
        { "name": "Date", "filters": [], "filterOption": "all" },
        { "name": "Sessions", "filters": [], "filterOption": "all" },
        { "name": "Conversions", "filters": [], "filterOption": "all" }
      ]
    }
  },
  "columns": [
    {
      "id": "55a4cfcade631f67e0dd278dcf6617df",
      "name": "Date",
      "type": "DATE",
      "formula": "__source__@A:A;",
      "fmtArgs": { "dateInputFormat": "custom", "dateInputFormatCustom": "yyyy-MM-dd'T'HH:mm:ss", "inputTimezone": "GMT" },
      "aggregation": "auto"
    },
    { "id": "57a9f37724266142bee07ebcaeb564d8", "name": "Sessions", "type": "NUMERIC", "aggregation": "sum", "formula": "__source__@B:B;" },
    { "name": "Conversions", "type": "NUMERIC", "aggregation": "sum", "formula": "__source__@C:C;" }
  ]
}

The first two columns are the feed's own, sent back with the id and everything else GET /data_feeds/:feedId reported for them; the third is new, so it has no id yet.

Returns 200 with the updated feed, in the same shape GET /data_feeds/:feedId returns. The connection and the view are the feed's own and are not restated.

Send the columns with the query. A query-builder column's formula addresses its source field by position, not by name: __source__@A:A; is the first field in fields, B:B the second, and so on — hidden fields excluded, since they produce no column. So a query that gains, loses or reorders a field moves every column after it, and the two only agree if you write them from one another:

Send the query without matching columns and the feed keeps reporting the old columns over the new positions — the same data under the wrong names. Columns you added on top of the generated ones (a calculated column, say) are yours to keep or drop: send back the full columns array you want the feed to have, in the order you want them — the order of the array is the feed's column order.

A column you are adding needs a name, a type and a formula; keep the id on each column you are keeping, so its format and anything else you did not send survives. A DATE column should also carry the fmtArgs that say how its values are formatted — copy them from the feed's existing date column, as the example above does; without them nothing tells the feed how to read the dates the service sends.

The query is replaced, not merged. Send the full field list, read from GET /data_feeds/:feedId immediately before writing — a replacement built from a stale read loses whatever changed in between. A field marked "hidden": true is a parameter the service needs in order to answer the query rather than a column of the feed: send it back unchanged, or leave it out and it is carried over for you.

The data source still holds the rows it fetched under the old query, so a new query also queues a refresh. It is queued, not awaited: the response comes back before the data does, so GET /data_feeds/:feedId/data can briefly report the previous rows — and nothing at all for a field the query just gained. A query that comes back unchanged is neither rewritten nor refreshed, so a PUT that only renames a feed costs it nothing.

A feed whose data source has no query of its own (an upload, or a REST endpoint) has none to replace, and source.query on one returns 400.

DELETE /data_feeds/:feedId

Removes the referenced datasource, then the feed. 204 No Content.

POST /data_feeds/:feedId/refresh

Trigger a refresh. Returns the refresh result from the upstream service.

GET /data_feeds/:feedId/data

Download the feed's processed data as CSV. Returns text/csv with a Content-Disposition: attachment; filename="<name>.csv" header.

GET /data_feeds/:feedId/raw_data

Download the feed's raw uploaded data. Returns the stored content with its original Content-Type and Content-Disposition.

POST /data_feeds/:feedId/raw_data

Replace (or append to) the feed's raw data. The request body is the raw content; set Content-Type to match the format (e.g. text/csv) — see Uploaded content types. Pass ?append=true to append instead of replace.

The feed's data source already fixes the format, so unlike generate?type=dataUpload the Content-Type need not name one: generic application/octet-stream bytes are fine and no format parameter is read.

POST /data_feeds/feed_42/raw_data?append=true
Content-Type: text/csv

date,amount
2026-05-01,100

Returns 200:

{ "location": "/data_feeds/feed_42/raw_data" }

Uploaded content types

The two endpoints that take data in the request body — POST /data_feeds/generate?type=dataUpload and POST /data_feeds/:feedId/raw_data — accept the same content types:

Anything else returns 415. Parameters on the header (text/csv; charset=utf-8) are ignored.

Generating a feed creates the data source, so it has to know the format: an application/octet-stream upload must name it with ?format=csv|xls|json|xml, and that parameter overrides the Content-Type for any upload. Replacing an existing feed's data needs no format — the data source has one already.

generate?type=rest uploads nothing, so it has no Content-Type header to read: its contentType field names the format from the same table, either as a format (csv) or as a media type that names one (text/csv). Generic application/octet-stream names no format, so it is not accepted there.

Uploads are limited to 10 MB; your account's own data source size limit applies as well.