# Add user to workspace
Source: https://help.datafetcher.com/account/add-user-to-account
You can add team members to your workspace, allowing them to use Data Fetcher too.
Your Data Fetcher workspace must be on the Pro, Business or Business + plan.
## How to add a user to your workspace
1. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click the **Users** section on the left-hand menu.
4. Click **Add User**.
5. Enter the user's **Email** in the input field.
6. Select a role for the user:
1. **Owner** - Can add/remove users, manage billing, delete the workspace, monitor usage, and work with requests.
2. **Creator** - Can create, update, delete, duplicate, and run requests and sequences.
7. Click **Add user**.
The user will be added to your workspace. They'll receive an invitation email and can access your workspace whenever they sign in to the Data Fetcher extension.
# Authorize Data Fetcher and Airtable
Source: https://help.datafetcher.com/account/authorize-data-fetcher-airtable
Data Fetcher needs to access your Airtable data when you're not using the extension to schedule runs, use triggers, or webhook URLs. You'll need to authorize Data Fetcher in the extension using either OAuth or a personal access token (PAT).
You only need to do this once, as Data Fetcher will securely store your authentication details.
Your Data Fetcher workspace uses a single authorization (whether through OAuth
or PAT) to access all your Airtable bases. So you need to give it access to
every base you want to run scheduled / webhook URL / triggers in.
OAuth connections can occasionally be flaky due to token expiration or Airtable API instability, while personal access tokens tend to be more reliable for consistent long-term access to your Airtable data.
## Authorize with OAuth
1. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click the **Airtable access** section on the left-hand menu.
4. Under **OAuth Connection**, click **+ Authorize**.
5. On the modal that opens, click **I understand, let's authorize**.
6. A new tab/window will open - select **+ Add all resources** to avoid access issues in the future.
7. Click **Grant access**.
8. Back in the Data Fetcher extension, you should see **Connected** under **Airtable OAuth Connection**.
## Authorize with a personal access token (PAT)
1. Go to [Airtable's token creation page](https://airtable.com/create/tokens/new).
2. Create a new token with access to all bases and these scopes:
* `data.records:read`
* `data.records:write`
* `schema.bases:read`
* `webhook:manage`
3. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
4. Make sure the correct workspace is selected in the top-left dropdown.
5. Click the **Airtable access** section on the left-hand menu.
6. Under **Personal Access Token (PAT)**, click **+ Add**.
7. Paste in your Airtable personal access token.
8. Click **Save**.
Your Airtable PAT will be saved in your Data Fetcher workspace.
### Important: PAT takes precedence
If both OAuth and a PAT are configured for a workspace, Data Fetcher will always use the PAT for authentication. This means:
* If you want to use OAuth, make sure no PAT is configured
* If you want to switch from PAT to OAuth, you'll need to remove the PAT first
To remove a PAT, go to workspace settings and click the delete icon (🗑️) next to it.
# Update name or email address
Source: https://help.datafetcher.com/account/change-personal-details
You can update your personal information in Data Fetcher through either the extension or on [datafetcher.com](https://datafetcher.com). Follow the instructions below to make these changes.
If you need to update your billing information instead (such as the billing name, address, or email that appears on invoices), please refer to our [Update Billing Information](/account/update-billing-information) article.
## Update in the Data Fetcher extension
### Update your name
1. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
2. Click your profile icon in the top right.
3. Click **Profile**.
4. Under **Your Details**, click **Edit name**.
5. Enter your updated name in the field provided.
6. Click **Save** to apply your changes.
### Update your email address
1. Hover over the Data Fetcher extension.
2. Click your profile icon in the top right.
3. Click **Profile**.
4. Under **Your Details**, click **Edit email**.
5. Enter your updated email address in the field provided.
6. Click **Save** to apply your changes.
## Update on datafetcher.com
### Update your name
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in)
2. Click your profile icon in the top right.
3. Click **Profile**.
4. Under **Your Details**, click **Edit name**.
5. Enter your updated name in the field provided.
6. Click **Save** to apply your changes.
### Update your email address
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in)
2. Click your profile icon in the top right.
3. Click **Profile**.
4. Under **Your Details**, click **Edit email**.
5. Enter your updated email address in the field provided.
6. Click **Save** to apply your changes.
# Roles and permissions
Source: https://help.datafetcher.com/account/roles-and-permissions
## Workspace roles
Data Fetcher has two distinct user roles with different permission levels:
* **Owner** - full administrative control to add/remove users, manage billing, delete workspaces, and monitor usage. Owners can create, update, delete, duplicate, and run all requests and sequences. A workspace can have multiple owners, and must always have at least one.
* **Creator** - operational permissions focused on working with data. Creators can create, update, delete, duplicate, and run requests and sequences, but cannot perform administrative functions.
## Change a user's role
As an Owner, you can modify the roles of other users in your workspace:
1. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click the **Users** section on the left-hand menu.
4. Click **Add User**.
5. Click the **Edit** icon (✏️) next to the user's role.
6. Select the desired role from the dropdown.
7. Click **Save** to update that user's role.
# Switch workspace
Source: https://help.datafetcher.com/account/switch-workspace
## What is a Data Fetcher workspace?
Workspaces in Data Fetcher are shared collaborative environments where team members can access, edit, and run saved requests together. Think of them like Airtable workspaces—they provide a central location where everyone on your team sees the same content.
When you sign up for Data Fetcher, we automatically create a default workspace for you. You may also be invited to join existing workspaces created by your teammates.
### Key things to know about workspaces
* All billing information and usage limits are tied to the workspace, not individual users
* Any changes you make to requests are visible to all workspace members
* Switching between workspaces is easy within the Data Fetcher extension and website
## How to switch between Data Fetcher workspaces
Follow these simple steps to change which workspace you're currently using:
1. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click the **Back** button in the bottom-left.
After switching, all saved requests and settings will reflect those of your newly selected workspace. Your team members using the same workspace will see identical content.
# Update billing information
Source: https://help.datafetcher.com/account/update-billing-information
You can update the billing name, address and email address that appear on your Data Fetcher invoices by following these steps:
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click **Settings** in the top menu.
4. Click **Billing** in the left-hand sidebar.
5. Under **Payment information**, click **Update billing info**.
6. This will redirect you to Stripe (Data Fetcher's payment processor).
7. Under the **BILLING INFORMATION** section, click **Update information**.
8. Make your changes to the billing name, address and/or email address.
9. Click **Save** to apply your changes.
Note that these changes will only apply to future invoices. It's not possible to update information on invoices that have already been issued.
# Update payment card
Source: https://help.datafetcher.com/account/update-payment-card
You can update the payment card used for your Data Fetcher workspace by following these steps:
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click **Settings** in the top menu.
4. Click **Billing** in the left-hand sidebar.
5. Under **Payment information**, click **Update payment card**.
6. This will redirect you to Stripe (Data Fetcher's payment processor).
7. Click **+ Add payment method**.
8. Enter your card details.
9. Ensure **Use as default payment method** is selected.
10. Click **Add** to save your new payment card.
# Upgrade workspace
Source: https://help.datafetcher.com/account/upgrade-account
Upgrading your Data Fetcher workspace unlocks advanced features and higher usage limits for your entire team. A workspace upgrade provides:
* Increased [usage limits](/account/view-monthly-usage)
* Ability to [schedule automated requests](/create-requests/schedule-a-request)
* Ability to [trigger requests](/create-requests/add-a-request-trigger) when a record is created / updated / deleted
* Ability to [run requests using a webhook URL](/create-requests/run-request-using-webhook)
* Additional [team member seats](/account/add-user-to-account) (Pro, Business & Business + plans).
All billing occurs at the workspace level, meaning your entire team benefits from a single subscription.
## How to upgrade your Data Fetcher workspace
Follow these steps to upgrade your workspace to a paid plan:
1. On the Data Fetcher extension home screen, click **Upgrade**.
2. This will open a new window on [datafetcher.com](https://datafetcher.com), where you can view available plans.
3. Select your preferred pricing plan from the options.
4. Complete the checkout process with your payment information.
5. Return to the Data Fetcher extension in Airtable to start using your upgraded features.
All payments are processed securely through Stripe, Data Fetcher's payment processor. You can change your plan or cancel your subscription at any time through your workspace billing settings at [datafetcher.com](https://datafetcher.com/).
# View invoices
Source: https://help.datafetcher.com/account/view-invoices
To view your invoices for Data Fetcher, you'll need to follow these steps:
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in)
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click **Settings** in the top menu.
4. Click **Billing** in the left-hand sidebar.
5. Under **Invoices**, click **View invoices**.
6. This will redirect you to Stripe (Data Fetcher's payment processor).
7. Once in Stripe, you can find your complete invoice history under the **INVOICE HISTORY** section.
8. From there, you can view and download any invoices you need.
# View monthly usage
Source: https://help.datafetcher.com/account/view-monthly-usage
Your Data Fetcher plan includes a monthly allowance of runs and response records. Usage resets on the same day each month that you originally upgraded your workspace - not at the start of each calendar month.
## What counts as a run?
A run is a single API request. Each time you click the **Run** button, you'll use at least one run from your monthly allowance. Here's when you might use multiple runs:
1. [Run for each record](https://help.datafetcher.com/create-requests/use-table-values-in-requests#run-for-each-record): When you run a request for each record in a view, each record processed counts as one separate run. For example, fetching stock prices for 20 records would use 20 runs.
2. [Pagination](https://help.datafetcher.com/advanced-options/pagination): When your request retrieves data across multiple pages, each page fetched counts as one run. For instance, if results span 3 pages, that's 3 runs.
Failed API requests don't count against your usage, and creating or refreshing your response field mapping is also free.
## What counts as a response record?
When you run a request, Data Fetcher transforms the API response into Airtable records using your configured [response data transformation](https://help.datafetcher.com/advanced-options/response-data-transformations) and [field mapping](https://help.datafetcher.com/create-requests/response-field-mapping). The number of records counted against your usage is based on how many transformed items result from this process, not how many values actually change in Airtable.
For example:
* A response containing a list of 50 items would count as 50 records (even if sent in a single run).
* A response containing a single object would count as just 1 record.
## How do different run types affect my usage?
All types of runs in Data Fetcher consume your monthly allowance in the same way, regardless of how they're triggered:
* **Manual runs**: When you click the **Run** button yourself
* **Scheduled runs**: When Data Fetcher automatically runs requests on your defined schedule
* **Webhook URL runs**: When your request is triggered by calling a request webhook URL
* **Trigger runs**: When a request is triggered by a record being created, updated or deleted
Each of these methods counts as one or more runs against your monthly allowance, following the same rules. For example, if a scheduled request retrieves paginated data across 3 pages, it will use 3 runs—exactly the same as if you had clicked **Run** manually and retrieved the same data.
The same applies to response records - the number of records used depends on the data returned, not how the request was triggered.
## How to check your current usage
To view your run and response record usage for the current billing period:
1. Sign in to your account at [datafetcher.com/sign-in](https://datafetcher.com/sign-in).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. On the workspace **Home** screen, you will see a chart of your usage this billing cycle, and how close to your plan limits you are.
## How to check your usage reset date
To find out exactly when your monthly usage allowance will reset:
1. Sign in to your account at [datafetcher.com/sign-in](https://datafetcher.com/sign-in).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Under **Limits**, you will see a message explaining when your usage cycle resets.
# Add fixed values to output
Source: https://help.datafetcher.com/advanced-options/add-fixed-value-to-output
You can add fixed (constant) values to every record written to the destination table when a request runs.
This feature can help you track which request created or updated the record, making it easy to identify the data source for each entry. Combined with Airtable formula fields, it can also be used to add a timestamp to the output data.
## When to use fixed values
Fixed values are particularly useful when you're importing similar data from multiple sources into the same table. For example, if you have an "Orders" table that collects orders from both Shopify and Etsy, you can use fixed values to automatically tag each record with its source platform.
Here's how this works in practice:
1. Create an Airtable field in your destination table to store the source information (e.g. a "Single select" field called "Platform" with "Shopify" and "Etsy" as options).
2. Create a separate request for each data source.
3. Configure each request to automatically add the appropriate fixed value ("Shopify" for the Shopify request and "Etsy" for the Etsy request)
When the requests run, every imported record will be automatically tagged with its source, so you'll always know whether an order came from Shopify or Etsy.
## How to add fixed values
1. Navigate to the request screen in Data Fetcher.
2. Scroll down to the bottom and click to open the **Advanced settings**.
3. Under **Add Fixed Values**, click **+ Add fixed value**.
4. Select a field from the destination table using the left-hand dropdown.
5. Enter the value you want to add to each record in the right-hand input.
You can add more fixed values using the same steps.
The fixed values will be added to all records created / updated by this request.
## Add fixed values for empty results
By default, fixed values are only added when your request returns data. Enable **"Add fixed field even when no records returned"** to create a record with your fixed value even when the API returns no results.
This is useful for tracking failed requests, maintaining audit trails, or logging when scheduled requests run but find no new data.
## Use Airtable values as fixed values
Instead of using static values, you can reference values from your Airtable tables as fixed values. This is particularly useful for adding dynamic information like timestamps or configuration data that changes over time.
**Common use cases:**
* Add a timestamp showing when Data Fetcher last ran
* Linking to a source record when running for each record
**How to set this up:**
1. In the fixed value input, click the **+** button to open the table reference modal.
2. Select the **Table** and **Field** containing the value you want to use.
3. Choose a specific record with the cell value you need. This is optional but strongly-recommended. You can read about the [two ways of using table values here](/create-requests/use-table-values-in-requests).
4. Click **Confirm**.
For example, to automatically track when your data was last updated, you could create a formula field in an Airtable "Config" table with `DATETIME_FORMAT(NOW(), 'YYYY-MM-DD HH:mm:ss')` and reference that cell. Every time your request runs, it will add the current timestamp to your imported records.
# CSV and XML response handling
Source: https://help.datafetcher.com/advanced-options/csv-and-xml-response-handling
Data Fetcher supports API responses in multiple formats, including **CSV** and **XML**. These options in **Advanced settings** control how Data Fetcher interprets those responses before converting them into Airtable records.
## CSV: includes header row
The **CSV: includes header row** option controls how Data Fetcher interprets CSV responses.
When enabled, Data Fetcher treats the **first row of the CSV file as column headers** and uses them to determine the field names written to Airtable.
### When to enable this option
Enable **CSV: includes header row** if the CSV file includes column names in the first row.
Example CSV:
```
name,email,company
Jane Doe,jane@example.com,Acme
John Smith,john@example.com,Globex
```
In this case, the first row (`name,email,company`) will be used as the field names.
### When to disable this option
Disable this option if the CSV file **does not include header rows**. Otherwise, the first row of data may be interpreted as field names.
## XML array handling
When Data Fetcher fetches XML responses, it converts the XML into tabular data (fields and records) in two steps:
1. The [xml2js](https://www.npmjs.com/package/xml2js) package converts the XML into a JavaScript object.
2. Data Fetcher then transforms this object into tabular data using the standard [response data transformation](/advanced-options/response-data-transformations).
By default, XML child nodes are only treated as arrays when there are multiple nodes. If there is only one child node, it is treated as an individual object.
To treat **all child nodes as arrays**, even when there is only one:
1. Navigate to the request screen in Data Fetcher.
2. Open **Advanced settings**.
3. Scroll down and toggle **Always put child nodes in array**.
# Denormalize response
Source: https://help.datafetcher.com/advanced-options/denormalize
Some API endpoints return JSON data where each value is nested under different keys that change between responses. This creates problems when mapping response fields to a destination table.
## The problem
For example, if calling a currency conversion API gives us this JSON:
```
{
"base": "EUR",
"start_date": "2020-01-01",
"end_date": "2020-01-07",
"rates": {
"2020-01-01": {
"USD": 1.1221,
"GBP": 0.8462
},
"2020-01-02": {
"USD": 1.1221,
"GBP": 0.8466
},
"2020-01-03": {
"USD": 1.1171,
"GBP": 0.8495
}
}
}
```
When mapping this [response using field mapping](/create-requests/response-field-mapping), we get a different field name for each date key under `rates`. For example, `rates.2020-01-01.USD` and `rates.2020-01-02.USD` would be treated as separate fields.
If we change the request parameters to use different dates (like `2021-01-01` to `2021-01-03`), we get completely new keys under `rates` that won't map to any existing fields:
```
{
"base":"EUR",
"start_date":"2021-01-01",
"end_date":"2021-01-03",
"rates":{
"2021-01-01":{
"USD":1.217582
},
"2021-01-02":{
"USD":1.2135
},
"2021-01-03":{
"USD":1.223409
}
}
}
```
## The solution
What we really want is just two consistent fields: a `Date` field and a `USD` field. We want different dates to appear as separate records with these same two fields.
We can do this by denormalizing the response. We set:
1. **Denormalize Path** to `$.rates` (the [JSONPath](https://github.com/json-path/JsonPath) expression for the top-level `rates` key)
2. **Denormalize Field Name** to `Date`
After denormalization, the response becomes:
```
{
"base": "EUR",
"start_date": "2020-01-01",
"end_date": "2020-01-07",
"rates": [
{
"Date": "2020-01-01",
"USD": 1.1221,
"GBP": 0.8462
},
{
"Date": "2020-01-02",
"USD": 1.1221,
"GBP": 0.8466
},
{
"Date": "2020-01-03",
"USD": 1.1171,
"GBP": 0.8495
}
]
}
```
Now all dates appear under a consistent `rates.Date` field and all USD exchange rates appear under a consistent `rates.USD` field. This gives us stable field names that we can reliably map to our destination table.
## How to denormalize the response
1. On the create screen, scroll to the bottom and click to open the **Advanced Settings**.
2. Under **Denormalize Path**, enter the JSONPath to where you want to denormalize the data (for the example above, use `$.rates`).
3. Enter a **Denormalize Field Name** for the field that will be added to each record (for the example above, use `Date`).
# Error handling
Source: https://help.datafetcher.com/advanced-options/error-handling
Data Fetcher’s error handling lets you control how consecutive errors are handled for manual and automated runs. It helps you catch systematic failures while tolerating occasional issues.
## How to configure error handling
1. On the request or sequence screen, click to open the **Advanced settings**.
2. Scroll down to find the **Error Handling** section.
3. Choose how errors should be handled:
* **Stop on first error**
* Or **Continue through errors**
4. Configure the specific thresholds:
* **Stop after X errors**
* **Pause automated runs after Y errors**
5. (Optional) Enable **Treat empty results as an error**
### Notify and stop after consecutive errors
This setting controls when Data Fetcher will notify you of errors and stop the current execution.
**For manual runs:** You'll see the error message and execution will stop.
**For automated runs:** You'll receive an email notification and the current run will stop, but the automation remains active for future runs.
**Example scenarios:**
* Set to **1** for critical data where any error needs immediate attention
* Set to **5** for resilient imports that should tolerate a few sporadic failures
### Pause automated runs after consecutive errors
This setting determines when Data Fetcher pauses your automation (schedules, webhook URLs, and triggers) to prevent runaway failures.
**Important:** This setting only applies to automated runs. Manual runs are not affected.
When this threshold is reached, your automation will be paused and you'll receive an email notification. You'll need to manually re-enable the automation after fixing the underlying issue.
**Example scenarios:**
* Set to **5** for immediate protection against systematic failures
* Set to **50** for maximum resilience before pausing
### Treat empty results as an error
When enabled, a run that returns zero records after transformation and mapping is treated as an error — even if the API call itself succeeded. This is useful when you expect data to always be present (e.g. daily sales, inventory updates) and want to be alerted when it's missing.
Leave this off if data volume naturally varies or empty periods are normal for your use case.
Empty results count toward your consecutive error thresholds.
## Understanding consecutive errors
Data Fetcher counts **consecutive** errors, meaning successful runs reset the error counter to zero.
**Example with "Notify after 3 errors" setting:**
* Run 1: Error (count: 1)
* Run 2: Error (count: 2)
* Run 3: Success (count: reset to 0)
* Run 4: Error (count: 1)
* Run 5: Error (count: 2)
* Run 6: Error (count: 3) → **Notification sent**
This prevents sporadic, unrelated errors from triggering notifications while still catching systematic problems.
## Why this matters: A practical example
Let's say you have a scheduled request that imports customer data from your CRM every hour.
### Scenario 1: API key expires
Your CRM API key expires, causing authentication errors.
**With settings 1 / 5:**
* First error → You get notified immediately via email
* If you don't fix it, after 5 consecutive failures the schedule pauses
**With settings 10 / 50:**
* After 10 consecutive errors → You get notified via email
* If you don't fix it, after 50 consecutive failures the schedule pauses
### Scenario 2: Running for each record with some 404s
You're running a request for each of 100 customer IDs from your Airtable, calling an API to get order details for each customer.
**The situation:** Some customers may not have any orders, causing the API to return 404 errors for those IDs.
**With settings 1 / 5:**
* First 404 error → Request stops, only processes 1 customer
**With settings 10 / 50:**
* Individual 404 errors are tolerated, request continues processing all customers
* Only systematic issues (like API authentication failing) trigger notifications
### Scenario 3: Intermittent rate limit errors
Your API occasionally returns rate limit errors during peak hours.
**With settings 1 / 5:**
* Every rate limit → Immediate notification
* Frequent pausing of your schedule
**With settings 10 / 50:**
* Occasional rate limits don't trigger notifications
* Only systematic issues (like 10 consecutive rate limits) trigger alerts
## Sequences vs individual requests
**For sequences:** Error handling settings on the sequence take precedence over individual request settings when the sequence runs.
**For individual requests:** Each request uses its own error handling settings when run individually.
This gives you centralized control over sequence behavior while maintaining flexibility for standalone request execution.
## Best practices
**For critical data:** Use lower thresholds (1-3) for quick notification and close monitoring.
**For resilient syncing:** Use higher thresholds (10-20) to handle API variability while preventing runaway failures.
**For development:** Start with conservative settings (1 / 5) while building requests, then adjust based on observed API behavior.
# Fixed IP Address
Source: https://help.datafetcher.com/advanced-options/fixed-ip-address
Some APIs require you to add specific IP addresses to an allowlist before accepting connections. Data Fetcher's **Fixed IP Address** option ensures your requests always come from the same IP address, making it easy to meet this security requirement.
This feature requires a paid Data Fetcher plan.
## How to enable a fixed IP address
1. Navigate to the request screen in Data Fetcher.
2. Open **Advanced settings**.
3. Scroll down and toggle on **Use fixed IP address**.
4. Once enabled, your requests will run from the following IP address:
```
54.173.229.200
```
When configuring your API service, add this IP address to its allowlist so it can accept requests from Data Fetcher.
## When to use a fixed IP address
Enable this option when the API you’re connecting to requires incoming requests to come from a known or allowlisted IP address. This is common with internal APIs, private services, or systems with strict firewall rules.
If the API does not require IP allowlisting, you can leave this setting turned off.
# Max response records
Source: https://help.datafetcher.com/advanced-options/max-response-records
When Data Fetcher processes an API response, it can sometimes return more records than you need. Use the **Max response records** setting to limit the number of records in your output.
For example, if an API returns 100 records, but you only want the first 10, set **Max response records** to 10.
## Set the max response records
1. On the request screen, click to open the **Advanced settings**.
2. Scroll to **Max response records**, and enter the maximum number of records you want to receive.
3. When you run the request, the [Response field mapping](/create-requests/response-field-mapping) will refresh with the Max response records setting applied.
These settings are also available on the [Response Field
Mapping](/create-requests/response-field-mapping#access-advanced-settings).
# Pagination
Source: https://help.datafetcher.com/advanced-options/pagination
Many API endpoints that return a list of data split that data up into separate "pages", where each page is a subset of the data. This is called pagination. It is easy to handle all types of pagination in Data Fetcher.
## How to identify your API's pagination type
Most APIs document their pagination method in a dedicated "Pagination" section or within specific endpoint documentation.
Check your API documentation for these patterns:
* **URL parameters:** "page", "offset", "limit", "cursor", or "after"
* **Request body fields:** Pagination parameters sent in the body instead of URL
* **Response fields:** "next\_url", "next\_cursor", "has\_more", or "total\_pages"
* **Headers:** Custom pagination headers like "page" or "x-page"
* **GraphQL:** Cursor-based with "first", "after", or "before" arguments
## Data Fetcher pagination types
Data Fetcher handles the most common types of pagination. Click any type below to jump to its setup instructions:
1. Page
Uses a page number parameter in the URL. For example:
* Page 1: `https://www.example.com/?page=1`
* Page 2: `https://www.example.com/?page=2`
2. Offset
Uses offset and limit parameters in the URL to specify data range. For example:
* Page 1: `https://www.example.com/?offset=0&limit=50`
* Page 2: `https://www.example.com/?offset=50&limit=100`
3. Cursor
Uses an ID from the previous response as a starting point parameter. For example:
* Page 1: `https://www.example.com`
* Page 2: `https://www.example.com/?starting_after=cus_IOCwDqeBZGGsrF`
4. Next URL
The API response includes the complete URL for the next page. Sometimes provided in response headers as a Link header.
5. Offset body
Uses offset and limit parameters in the request body. For example:
* Page 1 body: `{ "startRow": 0, "rowLimit": 1000 }`
* Page 2 body: `{ "startRow": 1000, "rowLimit": 1000 }`
6. Cursor body
Uses a cursor value from the previous response in the request body. For example:
* Page 1 body: `{ "pageToken": "" }`
* Page 2 body: `{ "pageToken": "cus_IOCwDqeBZGGsrF" }`
7. Page body
Use a page number field in the body. For example:
* Page 1 body: `{ "page": 1 }`
* Page 2 body: `{ "page": 2 }`
8. Header
Uses a header value that increments with each request. For example:
* Page 1 headers: `page: 1`
* Page 2 headers: `page: 2`
9. Path
Uses a path parameter in the URL that increments with each request. For example:
* Page 1: `https://api.example.com/stocks/1`
* Page 2: `https://api.example.com/stocks/2`
10. GraphQL Cursor
The GraphQL query takes a cursor argument, set to an ID from the previous response, to tell the API where to start the next page. For example, for this GraphQL query:
```
query($after: String) {
pullRequests(first: 100, after: $after) {
nodes {
id
createdAt
number
title
}
pageInfo {
endCursor
hasNextPage
}
}
}
```
* Page 1: `after` would be set to null
* Page 2: `after` would be set to the `endCursor` value (or the final `ID` value for some APIs).
## Select pagination type
1. On the request screen, click to open the **Advanced settings**.
2. Scroll to **Max response records**, and enter the maximum number of records you want to receive.
3. Under **Pagination**, select the pagination type using the dropdown.
## Set the number of pages to fetch
Each page that is fetched will use 1 out of your monthly workspace runs.
After selecting a pagination type, choose one of these options:
* **Fetch all pages:** Automatically paginate through every page until no more records are returned. Test this option first, as it can sometimes cause infinite runs if the API always returns data.
* Set a maximum **number of pages:** Enter a specific number of pages to fetch. Data Fetcher will stop when it reaches this limit, when no items are returned, or when a page has fewer records than the last one.
* If you want to force pagination up to this number of pages even when fewer items are returned, click the settings icon (⚙️) next to the number of page input and enable **Keep paginating when no/fewer items returned**.
## Set up pagination
### Set up page pagination
1. Set the pagination type to "Page".
2. Set the **number of pages** to fetch or select **Fetch all pages.**
3. Enter the name of the **page parameter**. This is usually "page" but some APIs may use a different name.
4. If the API supports it, you can set a page size parameter and size, so that Data Fetcher knows when to stop paginating more reliably.
1. Set the name of the **page size parameter.**
2. Set the **page size.**
### Set up offset pagination
1. Set the pagination type to "Offset".
2. Set the **number of pages** to fetch or select **Fetch all pages.**
3. Enter the name of the **Offset parameter**. This is often called "offset" but some endpoints may use a different name.
4. Enter the name of the **Limit parameter**. This is often called "limit" but some endpoints may use a different name.
5. Enter the numerical **Limit value**. This is the number of items that are fetched in each page. The offset value will be increased by this amount for each request.
6. Optionally, enter a **Starting offset**. If you leave this blank, `0` will be used.
### Set up cursor pagination
1. Make sure you import the cursor field (usually an ID) on the [response field mapping](https://help.datafetcher.com/create-requests/response-field-mapping) and run the request once to create the cursor field in the destination table.
2. Set the pagination type to "Cursor".
3. Set the **number of pages** to fetch or select **Fetch all pages.**
4. Enter the name of the **cursor parameter**.
5. Select the **Cursor field** from the destination table fields using the dropdown.
6. If the API supports it, you can set a page size parameter to help Data Fetcher know when to stop paginating.
1. Set the name of the **page size parameter**.
2. Set the **page size**.
### Set up next URL pagination
1. Make sure you import the cursor field (usually an id) on the [response field mapping](https://help.datafetcher.com/create-requests/response-field-mapping) and run the request once to create the next URL field in the destination table.
2. Set the pagination type to "Next URL".
3. Set the **number of pages** to fetch or select **Fetch all pages.**
4. Select the **Next URL field** from the destination table fields using the dropdown:
5. If the API supports it, you can set a page size parameter to help Data Fetcher know when to stop paginating.
1. Set the name of the page size parameter.
2. Set the page size.
### Set up next URL pagination using Link header
1. Set the pagination type to "Next URL".
2. Set the **number of pages** to fetch or select **Fetch all pages.**
3. Turn on **Or, use link in response header**.
### Set up offset body pagination
1. Set the pagination type to "Offset Body".
2. Set the **number of pages** to fetch or select **Fetch all pages.**
3. Enter the **Offset path** for the request body. This uses JSONPath syntax (e.g., `$.pagination.offset`).
4. Enter the **Limit path** for the request body. This uses JSONPath syntax (e.g., `$.pagination.limit`).
5. Enter the **Limit value.**
6. Optionally, enter a **Starting offset**. If you leave this blank, 0 will be used.
### Set up cursor body pagination
1. Make sure you select the cursor field (usually an ID) on the response field mapping and run the request once to create and populate the cursor field in the destination table.
2. Set the pagination type to "Cursor Body".
3. Set the **number of pages** to fetch or select **Fetch all pages.**
4. Enter the **Cursor path** for the request body. This uses JSONPath syntax (e.g., `$.pagination.cursor`).
5. Select the **Cursor field** from the destination table fields in the dropdown.
### Set up page body pagination
1. Set the pagination type to "Page Body".
2. Enter the **Page path** for the request body. This uses JSONPath syntax (e.g., `$.page`).
3. If the API supports it, you can set a **page size path** (e.g. `$.pageSize`) and **page size** (e.g. `100`), so that Data Fetcher knows when to stop paginating more reliably.
4. Enter the **Starting page**, or leave it blank to use `1` as the starting page.
### Set up header pagination
1. Set the pagination type to "Header".
2. Set the **number of pages** to fetch or select **Fetch all pages.**
3. Enter the name of **Page header**, e.g. "page".
4. Optionally, enter a **Page size header**.
5. Optionally, enter a **Page size**.
6. Optionally, enter a **Starting page**. If you leave this blank, 1 will be used.
### Set up path pagination
1. In the request screen's main **URL** field, enter the URL with `{}` around the path parameter, e.g. `https://api.example.com/stocks/{page}` .
2. Back in the pagination settings, set the pagination type to "Path".
3. Set the **number of pages** to fetch or select **Fetch all pages.**
4. Enter the **Path parameter** (e.g. "page" in this example).
5. Optionally, enter a **Page size parameter**.
6. Optionally, enter a **Page size**.
7. Optionally, enter a **Starting page**. If you leave this blank, 1 will be used.
### Set up GraphQL Cursor Pagination
1. Make sure you import the cursor field (usually an ID) on the response field mapping and run the request once to create the cursor field in the destination table.
2. In the request screen's main **Body → GraphQL → Query** section, enter the GraphQL query, including any pagination arguments.
1. There is no need to add them in the **Variables** section, as Data Fetcher will handle this automatically.
3. Set the pagination type to "GraphQL Cursor".
4. Set the **number of pages** to fetch or select **Fetch all pages.**
5. Enter the **Cursor variable** name.
6. Select the **Cursor field** from the dropdown. Typically, this is an ID field or a pagination cursor field in the response.
7. You can optionally enter a **Page size variable** or just a **Page size** value. This will help Data Fetcher know when to stop paginating if you have selected the **Fetch all pages** option.
# Rate limit
Source: https://help.datafetcher.com/advanced-options/rate-limit
Many APIs enforce rate limits to prevent overuse - typically expressed as a maximum number of requests per minute or second. When you exceed these limits, APIs may reject your requests or temporarily block access.
Data Fetcher helps you stay within these boundaries by automatically adding delays between requests when you configure a rate limit.
## How to set a rate limit
1. Navigate to the request screen in Data Fetcher.
2. Scroll down to the bottom and click to open the **Advanced settings**.
3. In the **Rate Limit** field, enter your desired limit.
4. **Save** your request.
Data Fetcher will automatically pace your requests to stay within the specified limit, preventing API errors.
# Data transformation
Source: https://help.datafetcher.com/advanced-options/response-data-transformations
When you run a request, Data Fetcher converts the API response into a table of fields and records that can be written to your destination Airtable table. You can control how this data is structured using **response field mapping** and **data transformation** settings.
## Understanding the two transformation types
Data Fetcher offers two ways to handle nested arrays in your API responses: **Single record per item** and **Expand nested items**. The key difference is whether nested arrays stay **combined in one record** or are **expanded into multiple records**.
Let's use this sample JSON response to illustrate both approaches:
```json theme={null}
[
{
"id": "company1",
"positions": [{ "id": "position1" }, { "id": "position2" }]
},
{
"id": "company2",
"positions": [{ "id": "position3" }, { "id": "position4" }]
}
]
```
### Single record per item
**Single record per item** keeps each parent entity in one row by combining nested values with commas. Each company stays in its own record:
| **Id** | **Positions id** |
| -------- | ------------------- |
| company1 | position1,position2 |
| company2 | position3,position4 |
**Single record per item** doesn't mean the entire response becomes one record.
Data Fetcher finds the **first array in the response** and creates **one record for each item in that array**.
### Expand nested items
**Expand nested items** creates a new record for each item in a nested array. Parent fields are duplicated so each nested item has its own row.
| **Id** | **Positions id** |
| -------- | ---------------- |
| company1 | position1 |
| company1 | position2 |
| company2 | position3 |
| company2 | position4 |
## How to set the transformation mode
1. On the create screen, open **Advanced settings**.
2. Under **Data Transformation**, select either **Single record per item** or **Expand nested items**.
## Advanced Single record per item options
### Combine nested object keys
By default, nested object values are combined into a **comma-separated field**. Using our example above, all `positions.id` values become a single-line text field called **Positions id**.
You can disable this to create separate numbered fields instead:
| **Id** | **Positions 1 id** | **Positions 2 id** |
| -------- | ------------------ | ------------------ |
| company1 | position1 | position2 |
| company2 | position3 | position4 |
**To disable combining nested object keys:**
1. In advanced settings, ensure transformation is set to **Single record per item.**
2. Turn off **Combine nested object keys**.
### Combine arrays of text/numbers
Similarly, arrays of text or numbers are combined into a **comma-separated value** by default. For this response:
```json theme={null}
[
{
"id": "company1",
"categories": ["Fintech", "Startup"]
},
{
"id": "company2",
"categories": ["Healthtech", "Public", "Unicorn"]
}
]
```
The default output combines categories:
| **Id** | **Categories** |
| -------- | ------------------------- |
| company1 | Fintech,Startup |
| company2 | Healthtech,Public,Unicorn |
You can disable this to create separate numbered fields:
**To disable combining arrays:**
1. In advanced settings, ensure transformation is set to **Single record per item.**
2. Turn off **Combine arrays of text/numbers.**
### Custom separators
Instead of commas, you can specify a custom separator for combined values. This is useful if your data **already contains commas** and you need a different delimiter such as the pipe symbol (`|`).
Both **Combine nested object keys** and **Combine arrays of text/numbers** use the separator you specify in the **Separator** input field.
**Need More Control?**
For more advanced control over how your response is structured, you can use a [response JMESPath](/advanced-options/response-jmespath) to customize exactly how the data is extracted and transformed.
# Response JMESPath
Source: https://help.datafetcher.com/advanced-options/response-jmespath
Transform your API responses with JMESPath, a powerful query language that helps you extract and reshape JSON data before it reaches Airtable. Use JMESPath to keep only the fields you need, rename confusing field names, filter data based on conditions, and work with nested structures.
## How to set up Response JMESPath
1. On the request screen, click the **Advanced Settings** button to expand additional options.
2. Scroll down to the **Response JMESPath** field and enter your expression.
3. Run your request again to see the transformed data, then reconfigure your response field mappings to match the new JSON structure.
You can [copy the response
JSON](/create-requests/response-field-mapping#working-with-response-data) and
test your JMESPath expressions on the [official JMESPath
website](https://jmespath.org/) before adding them to Data Fetcher.
## Common use cases and examples
The examples below show how JMESPath can solve common data transformation challenges. In each case, the result is JSON that Data Fetcher can easily map to Airtable fields.
### Keep only specific fields
**Problem:** Your API returns too much data, and you only need a few fields.
**Sample JSON:**
```json theme={null}
{
"coins": [
{
"id": "bitcoin",
"symbol": "btc",
"name": "Bitcoin",
"current_price": 27151
},
{
"id": "ethereum",
"symbol": "eth",
"name": "Ethereum",
"current_price": 1892
}
]
}
```
**JMESPath Expression:** `coins[].{id:id, name:name}`
**Result:** Only the `id` and `name` fields are kept, reducing clutter in your Airtable base.
```json theme={null}
[
{
"id": "bitcoin",
"name": "Bitcoin"
},
{
"id": "ethereum",
"name": "Ethereum"
}
]
```
**How it works:** In JMESPath, use `new_name:old_name` syntax inside curly braces to rename fields.
### Filter data by conditions
**Problem:** You only want records that meet specific criteria.
Use conditions with a question mark `?` followed by operators to filter your data:
| **Operator** | **Example JMESPath** | **Example in words** |
| ------------ | ------------------------------------- | -------------------------------------------------------- |
| `==` | `countries[?code == 'GB']` | Keep objects where code equals GB |
| `>` | `coins[?price > 100]` | Keep objects where price is greater than 100 |
| `>=` | `coins[?price >= 100]` | Keep objects where price is greater than or equal to 100 |
| `<` | `coins[?price < 100]` | Keep objects where price is less than 100 |
| `<=` | `coins[?price <= 100]` | Keep objects where price is less than or equal to 100 |
| `&&` | `coins[?price > 100 && price < 1000]` | Keep objects where price is between 100 and 1000 |
**Example:** To keep only coins worth more than \$100, use: `coins[?current_price > 100]`
### Work with nested data
**Problem:** The data you need is buried inside nested objects or arrays.
**Sample JSON with nested structure:**
```json theme={null}
{
"coins": [
{
"id": "bitcoin",
"stats": {
"current_price": 27105,
"market_cap": 525583712973
}
}
]
}
```
**JMESPath Expression:** `coins[].{id:id, price:stats.current_price}`
**Result:** The nested `current_price` becomes a top-level `price` field.
```json theme={null}
[
{
"id": "bitcoin",
"price": 27105
}
]
```
**How it works:** Use dot notation (`stats.current_price`) to access nested object properties.
### Choose the right array
**Problem:** Your JSON has multiple arrays, but Data Fetcher is using the wrong one.
**Sample JSON:**
```json theme={null}
{
"coins": [{ "id": "bitcoin", "current_price": 27105 }],
"exchanges": [
{ "id": "ace", "name": "ACE" },
{ "id": "nominex", "name": "Nominex" }
]
}
```
**Solution:** Use `exchanges[]` to tell Data Fetcher to create records from the exchanges array instead of the coins array.
**Result:** The `exchanges` array is kept and becomes a top-level array.
```json theme={null}
[
{
"id": "ace",
"name": "ACE"
},
{
"id": "nominex",
"name": "Nominex"
}
]
```
### JMESPath syntax quick reference
* **Arrays:** Access with `array_name[]`
* **Objects:** Access with `object_name.field_name`
* **Conditions:** Use `[?condition]` to filter arrays
* **Field selection:** Use `{new_name:old_name}` to pick and rename fields
* **Combine techniques:** Chain operations like `coins[?price > \`10`].{id:id, price:current_price}`
After applying a JMESPath expression, the Response field mapping will automatically refresh next time you run the request.
# Run request from button field
Source: https://help.datafetcher.com/advanced-options/run-request-from-button-field
Button fields in Airtable let you run Data Fetcher requests directly from your table (or interface) without opening the extension dashboard. This saves you time and allows you to run requests using values from specific records.
## How to set up button fields
Follow these steps to create your first button field request:
1. [**Create and save a request**](/create-requests/create-a-request) in Data Fetcher. You can close the extension dashboard afterward, but remain logged into Data Fetcher.
2. [Create a button field](https://support.airtable.com/docs/button-field) in Airtable that opens Data Fetcher.
3. Click any button in your new button field column.
4. A modal will appear asking you to select which request / sequence to run. Use the **Request** dropdown menu to make your selection.
5. Click **Save** to save your selection, and run the request immediately.
After this initial setup, clicking any button in that field will run your selected request immediately without requiring you to set it up again.
## Configure multiple button setups
You can assign different requests (and [sequences](/create-sequences/create-a-sequence)) to button fields across different tables or views within the same table.
1. Open the Buttons modal by clicking **Buttons** in the top menu on Data Fetcher's home screen.
2. Add a new configuration by clicking **+ Add** and selecting your table, view, and request / sequence.
3. Set view specificity by either selecting a specific view or leaving it blank to apply to all views in the table.
4. Click **Save** to confirm your configuration.
**Note:** Each table and view combination can only have one assigned request / sequence. If you configure both a general table setting (blank view) and a specific view setting, the specific view setting takes priority.
## Choose your execution scope
When working with requests that [run for each record](/create-requests/use-table-values-in-requests#run-for-each-record), button fields offer two execution options:
* **Run once mode (default):** The request runs only for the record containing the clicked button.
* **Run for each record mode:** The request runs for every record in your request's designated **input view**.
To use Run for each record mode, enable **Run for each record, not just the record clicked** when configuring your button setup.
# Run without writing to Airtable
Source: https://help.datafetcher.com/advanced-options/run-without-writing-to-airtable
The **Run without writing to Airtable** option lets you run an API request without creating or updating any records in your Airtable base.
When enabled, Data Fetcher will send the API request normally, but the response will not be written to your destination table.
## When to use this
This setting is useful when the API request performs an action rather than returning data.
Common examples include:
* **POST requests** that send data from Airtable to another system
* **PATCH or PUT requests** that update records in an external API
* API endpoints that **return no meaningful response data**
In these cases, you may want Data Fetcher to execute the request but ignore the response.
## How to enable it
1. Open your request in Data Fetcher.
2. Open **Advanced settings**.
3. Scroll down and enable **Run without writing to Airtable**.
4. Confirm the warning dialog.
The request will still run and send the API request, but **no records will be created or updated in Airtable**.
# Status code overrides
Source: https://help.datafetcher.com/advanced-options/status-code-overrides
By default, Data Fetcher treats HTTP error responses from an API (such as **404** or **500**) as errors. These errors count toward your error handling thresholds and may stop a run or pause automated runs.
**Status code overrides** let you specific HTTP status codes so they won't cause a run to fail or pause your automations.
Errors will still appear in the [run history](/organise-requests/run-history), but they won't count toward your consecutive error limits.
## How to configure status code overrides
1. On the request or sequence screen, click to open **Advanced settings**.
2. Scroll to the **Status Code Overrides** section.
3. Enter the HTTP status code you want to ignore (e.g. **404**).
4. Press **Enter** to add it.
You can add multiple status codes if needed.
## When to use status code overrides
Status code overrides are useful when an API returns certain error responses that are expected in your workflow.
### Example: Ignoring 404 responses
Suppose you're running a request **for each record** in your Airtable table to fetch order details for customers.
Some customers may not have any orders yet. In this case, the API may return:
```
404 Not Found
```
Without a status code override, each 404 counts as an error and may stop your request or pause your automation.
If you add **404** as a status code override:
* The request continues running
* Your automation will not pause
* The 404 responses still appear in run history for reference
### Example: APIs that return 409 for duplicates
Some APIs return:
```
409 Conflict
```
when you attempt to create a record that already exists.
If this is expected behavior in your workflow, adding **409** as an override allows the request to continue without triggering your error handling thresholds.
## How this interacts with error handling
Status code overrides affect **what counts as an error**.
If a status code is overridden:
* It will **not count toward your consecutive error thresholds**
* It will **not stop runs**
* It will **not pause automations**
However, the response will still appear in **run history** so you can review it if needed.
To configure how errors are handled once they occur, see [Error handling](/advanced-options/error-handling).
# Verify SSL certificate
Source: https://help.datafetcher.com/advanced-options/verify-ssl-certificate
The **Verify SSL certificate** option ensures that Data Fetcher verifies the API's SSL certificate when making a request. This helps confirm that the connection is secure and that the server you are connecting to is trusted.
In most cases, this setting should remain enabled.
## When to disable SSL verification
Disable **Verify SSL certificate** only if the API you are connecting to uses a **self-signed SSL certificate**. Some internal or private APIs use self-signed certificates, which can cause SSL verification to fail.
If SSL verification fails, you may see errors such as:
* `self signed certificate`
* `unable to verify the first certificate`
Turning this option off allows the request to proceed without validating the certificate.
Disabling SSL verification reduces connection security. Only disable this option if you trust the API you are connecting to.
## How to enable or disable it
1. Navigate to the request screen in Data Fetcher.
2. Open **Advanced settings**.
3. Scroll down and toggle **Verify SSL certificate** on or off.
# Add a request trigger
Source: https://help.datafetcher.com/create-requests/add-a-request-trigger
Request triggers automatically run your Data Fetcher requests whenever specific changes occur in your Airtable base. Your requests will execute automatically when records are created, updated, or deleted - even when you're not logged into Airtable or don't have Data Fetcher open.
Unlike scheduled requests that run at fixed intervals, triggers respond to events in near real-time, making them ideal for workflows that need immediate data synchronization or notifications.
## Set up a request trigger
### Authorize Data Fetcher
Before creating triggers, you need to authorize Data Fetcher to access your Airtable account using OAuth. You only need to complete this authorization once.
[For detailed authorization instructions, see our guide to authorization](/account/authorize-data-fetcher-airtable).
### Configure the trigger
1. On the custom request screen, click the **Trigger** tab.
2. Select your trigger type from the available options:
* **Record created** - Runs when new records are added
* **Record updated** - Runs when existing records are modified
* **Record deleted** - Runs when records are removed
You can select multiple trigger types if needed.
3. Choose the **Table** you want to monitor for changes.
4. Optionally, select a specific **View** to limit the trigger scope. If no view is selected, all record changes in the table will trigger the request.
5. If you selected **Record updated**, choose which specific fields to monitor for changes.
6. Click **Save** to activate your trigger.
Once your trigger is active, you can verify it's working by making a test change to your table and checking the [Run History](/organise-requests/run-history) to see if the request executed automatically.
## Trigger limitations
You can add a maximum of two triggers per base (across all Data Fetcher installations in that base and workspace) due to Airtable's webhook API limitations.
## Advanced trigger options
### Run when a record enters a view
You can trigger requests when records enter a specific view by using the **Record created** trigger type and selecting both a **Table** and **View**. This is useful for workflows where records moving into a view indicates they're ready for processing.
### Include Airtable API changes
By default, triggers ignore changes made through the Airtable REST API to prevent conflicts. You can enable the **Run when the base is changed via the Airtable API** setting to include these changes.
Enabling API change detection can cause infinite loops. This happens when a
trigger-created or trigger-updated record causes the same trigger to run
again, potentially consuming your entire monthly usage limit. Only enable this
setting if you're certain your workflow won't create feedback loops.
# List of Airtable integrations
Source: https://help.datafetcher.com/create-requests/airtable-integrations
Data Fetcher has [Airtable integrations](https://datafetcher.com/airtable-integrations) with many of the most popular apps. Click below to see the full tutorials on each.
Import database collection records and users
Import records from other bases
Import stock prices and financial data
Generate text using Claude AI
Create screenshots from URLs
Scrape data from any website
Import data from S3 or any AWS service
Import cryptocurrency market data
Enrich URLs and emails with company info
Import current and historical exchange rates
Import Facebook and Instagram Ads metrics
Import Facebook Lead Ads form submissions and lead data
Import Facebook Page and posts insights
Import Google Ads metrics
Import Google Analytics 4 (GA4) metrics
Search for books and import book and author data
Import calendar and event information
Notify Google when pages are added or removed
Look up distance, time, lat/long and business info
Monitor your Google Search performance
Import cell data from Google Sheets
Import data from any GraphQL API
Get any website's favicon
Import Instagram Page and posts insights
Connect Airtable to JSON APIs
Import LinkedIn profile and company data
Create screenshots and PDFs from URLs
Extract text from images and documents
Generate text and images using ChatGPT
Look up NFT collection data and statistics
Look up the weather forecast for a location
Scrape any website's data into Airtable
Sync your deals, leads and contacts
Import data from any REST API
Import your customers, subscriptions and more
Import time entries and other data
Create screenshots from URLs
Import posts, pages, comments and more
Import invoices, purchases orders and more
Get analytics from your YouTube channel
Look up public data about any channel
Get deals, leads, contacts and more
# Create a request
Source: https://help.datafetcher.com/create-requests/create-a-request
Requests are the core of Data Fetcher - they're saved instructions that tell Data Fetcher how to connect your Airtable base with external applications and APIs. Every time you run a request, it imports fresh data into your base or exports data to external systems.
## Choose your integration approach
Data Fetcher offers two ways to connect with external services:
**Pre-built integrations** - Ready-to-use connections for popular platforms like Facebook Ads, OpenAI, Google Analytics, Stripe, and many others. Our [Airtable integrations](https://datafetcher.com/airtable-integrations) handle the technical setup for you, so you can focus on configuring what data to import.
**Custom requests** - Connect to any REST or GraphQL API, or import from remote JSON, CSV, or XML files. Perfect when you need to integrate with specialized APIs or services that don't have pre-built integrations.
## Set up a request
### Choose your application
1. On the create request screen, select the **Application** you want to connect to:
* Choose **Custom** to create a custom request for any API
* Select a pre-built integration from the list of available applications
### Set up authentication
2. Complete the **Authorization** step by connecting your account or entering required credentials:
* **Pre-built integrations** often use OAuth to connect your existing accounts
* **Custom requests** typically require API keys or other authentication methods
* Some public APIs don't require any authorization
**Pro tip:** You can store API keys and other reusable values as variables in your workspace to use across multiple requests.
### Complete the setup
3. If applicable, select an **Endpoint** from the available options. This step varies by application - some have multiple endpoints while others have just one.
4. Enter a descriptive name for your request in the input field at the top of the screen.
5. Click **Save & Continue** in the bottom right corner.
6. Fill in any required (\*) configuration options for your chosen application and endpoint. These options control what specific data gets imported. **Advanced option:** You can use values from your Airtable tables in requests, including running separate requests for each record in a table or view.
7. Select your **Destination Table and View** where the imported data should go. By default, this uses your current table and view.
8. Click **Save & Run** in the bottom right corner.
## Configure field mapping and run
The Response Field Mapping window opens, where you control how data from the external application maps to your Airtable fields. You only need to configure this once unless you change your data transformation settings.
**For each field:**
* **Select or deselect** by clicking the field name - only selected fields will be imported
* **Map to existing fields** by choosing an existing column in your table
* **Create new fields** by specifying field names and types
9. Once you're satisfied with the response field mapping, click **Save & Run** in the bottom right corner.
Your request will execute and import data into your Airtable base according to your configuration. You can click **Run** anytime to import the latest data.
## Next steps
Now that you have created a request, explore these key features:
Run automatically at intervals, even when you're not using Airtable
Run requests instantly when your base data changes
Control how records are created and updated in your table
# Troubleshoot errors
Source: https://help.datafetcher.com/create-requests/errors
Here are some common errors that might occur during Data Fetcher runs, with steps describing how to fix them.
## Duplicate records being created
The most common cause of duplicate records is Airtable filters applied to your destination view.
Data Fetcher only searches for existing records in your selected destination view, not the entire table. If filters hide existing records, Data Fetcher can't find them and creates duplicates.
**Solution:** Create a dedicated view for Data Fetcher with no filters. Also verify your [**Match records using**](/create-requests/write-and-merge-data#choose-a-field-to-match-on) setting is using a unique identifier field.
## Authorization Errors
Authorization errors usually happen when connecting third-party accounts (for example, Google Ads) to Data Fetcher. This connection uses OAuth, and it can fail or become invalid if the wrong account is used, permissions are missing, or the connection has expired.
Common symptoms include 401 / 403 errors, missing accounts, or requests that previously worked but now fail.
**Solution:** Recreate the authorization connection.
1. Open your request.
2. Go to the **Authorization** section.
3. Click **+ New** to create a new connection.
4. When prompted:
* Make sure you're signed into the **correct account**.
* Approve **all permissions** that Data Fetcher requests during the authorization flow.
5. Save the request and run it again.
Re-authorizing ensures Data Fetcher has valid access tokens and the correct permissions for the account you're trying to use.
## Request works in Postman but not in Data Fetcher
This is usually caused by different headers being sent, particularly the User-Agent header.
**Solution:** Copy all headers from Postman to Data Fetcher:
1. In Postman, go to the **Headers** tab of your working request
2. Note all headers being sent (including hidden/auto-generated ones - click **Show auto-generated headers**)
3. In Data Fetcher, click the **Headers** tab
4. Add each header from Postman, especially:
* User-Agent
* Accept
* Accept-Encoding
The User-Agent header is often the critical one - some APIs block requests without a browser-like User-Agent (Data Fetcher sends `axios/0.22.0` as the User-Agent).
## Request works in browser but not in Data Fetcher
This typically indicates the website has bot protection (like Cloudflare) that blocks programmatic requests.
**Solution:** First verify if the API URL accepts any programmatic requests:
1. Test the same URL in Postman
2. Or test using curl in your command line: `curl "YOUR_URL"`
3. If these also fail with 403 errors, the URL likely blocks all automated requests
If the site has anti-bot protection, you generally cannot access it via Data Fetcher. Consider:
* Looking for an official API
* Checking if the site offers developer access or API keys
* Contacting the site owner for programmatic access
Note: Some sites only work in browsers due to JavaScript rendering or cookie requirements that Data Fetcher cannot replicate.
## Cannot create records in linked table X for response field Y
You have a linked field in the response field mapping and Data Fetcher needs to create records in the linked table (in order to link to them) but is not able to. This typically happens for two different reasons:
1. The linked table is a synced table. Records cannot be created in synced tables.
To fix this, you can turn off creating records in the linked table:
* Open the [response field mapping](/create-requests/response-field-mapping) and find the response field.
* Click the field settings icon .
* Disable the [Create missing records](/create-requests/response-field-mapping#create-missing-linked-records) option.
* Click "Save" on the field settings.
* Click "Save" on the response field mapping modal.
Alternatively, you may want to investigate why Data Fetcher has to create new records in the synced table in the first place - perhaps the Airtable table sync is not working as expected.
1. The linked field in the linked table is a computed field, e.g. a formula. Records cannot be created with values in computed fields.
To fix this, you can either follow the steps above to turn off creating missing linked records or consider changing the field type of the linked field to a non-computed type, e.g. Number.
## X variable is missing in workspace
Your request uses a variable but this variable no longer exists in your Data Fetcher workspace. You need to [create the variable](/create-requests/use-variables-in-requests#how-to-create-and-use-a-variable) again in your workspace or update the request to not use this variable.
## Destination table X not found in base
The destination table you selected has been deleted. You need to select a different destination table for this request.
* Open the request.
* Under **Destination Table & View**, select a table.
* Click "Save".
## Destination view X not found in base
The destination view you selected has been deleted. You need to select a different destination view for this request.
* Open the request.
* Under **Destination Table & View**, select a view.
* Click "Save".
## Field X cannot accept the provided value
There is a mismatch between the field's type in the response field mapping and the Airtable field's type. For example, a number field is mapped to a "Single line text" field in Airtable. This can happen if you change the Airtable field's type after setting up the response field mapping.
* [Open the response field mapping](/create-requests/response-field-mapping).
* Find the relevant field.
* Use the field type dropdown to select a type which matches the field's type in the destination table.
## Too many records in table X
The table is at the record limit for your Airtable plan.
You need to delete some records or upgrade to a larger Airtable (not Data Fetcher) plan with higher base/table record limits.
You can also turn off record creation in Data Fetcher:
* Open the request.
* Ensure **Merge with existing records** is turned on.
* Turn off **Create new records if no match**.
## Data Fetcher does not have access to base X. Please re-authenticate via the extension and select "All current and future bases in all current and future workspaces"
Your Data Fetcher workspace needs to be authorized to access all the bases where you want to use scheduled/ trigger/ webhook URLs runs. You either need to turn off the failing request's scheduled/trigger/webhook URL or [re-authenticate Data Fetcher and Airtable](/account/authorize-data-fetcher-airtable).
## Your X references table Y but it is missing in base. Has it been renamed?
Your request references a table by name, but that table cannot be found in the base. This most commonly happens after the table is renamed.
* Open the request.
* Find the outdated table reference.
* Clear the reference to the old table name (use the x button on the right-hand side).
* Create a new reference (use the + button on the right-hand side).
## Your Airtable OAuth connection has expired or been revoked
Your Airtable OAuth token has become invalid. This can happen due to Airtable API issues or if the token was revoked. You have two options to fix this:
**Option 1: Re-authorize with OAuth**
1. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click the **Airtable access** section on the left-hand menu.
4. Under **OAuth Connection**, click **+ Re-authorize**.
5. On the modal that opens, click **I understand, let's authorize**.
6. A new tab/window will open - select **+ Add all resources** to avoid access issues in the future.
7. Click **Grant access**.
8. Back in the Data Fetcher extension, you should see **Connected** under **Airtable OAuth Connection**.
**Option 2: Switch to Personal Access Token (PAT) - Recommended**
Personal Access Tokens are more reliable and don't expire unexpectedly. To switch:
1. Go to [Airtable's token creation page](https://airtable.com/create/tokens/new).
2. Create a new token with access to all bases and these scopes:
* `data.records:read`
* `data.records:write`
* `schema.bases:read`
* `webhook:manage`
3. Hover over the Data Fetcher extension, then click the **settings icon** (⚙️).
4. Make sure the correct workspace is selected in the top-left dropdown.
5. Click the **Airtable access** section on the left-hand menu.
6. Under **Personal Access Token (PAT)**, click **+ Add**.
7. Paste in your Airtable personal access token.
8. Click **Save**.
# Response field mapping
Source: https://help.datafetcher.com/create-requests/response-field-mapping
Response field mapping is where you control which data from your API response gets imported into Airtable and how it maps to your table's fields.
## Set up response field mapping
### Initial setup
When you create a request for the first time:
1. Click **Save & Run** on your request configuration.
2. The Response Field Mapping modal will open, showing all available fields from the API response along with a preview of the records.
### Map your fields
For each response field, you can:
* **Select or deselect fields** - Only data from selected fields will be imported when the request runs.
* **Map to existing fields** - Select **Existing field** and choose a field from the dropdown to use an existing column in your table.
* **Map to new fields** - Select **New field**, enter a field name, and choose a field type. Data Fetcher automatically suggests appropriate field types based on the response data.
* **Search for fields** - Use the **Find field** input to quickly locate specific fields in large responses.
Once you've configured your mappings, click **Save & Run**. Data Fetcher will create any new fields and import your data.
## Updating field mappings
### Reopen field mapping
After your initial setup, you can modify field mappings anytime by clicking **Open field mapping** on your request.
### Refresh fields
When your API response changes (for example, due to parameter updates), you can refresh available fields without using your monthly usage limits:
1. Click **Open field mapping**.
2. Click **Refresh fields** in the top-right corner.
The field mappings also refresh automatically when you change [Response data transformations](/advanced-options/response-data-transformations), [Response JMESPath](/advanced-options/response-jmespath), or [Max response records](/advanced-options/max-response-records).
## Additional field options
### Formula
Apply transformations to field values before they reach your destination table using formulas. Reference the original value as `value` in your formula (e.g., `upper(value)`).
1. Click the settings icon next to any field to open field settings.
2. Enter your formula in the **Formula** input.
3. Click **Save**.
You can find a full list of [available formulas here](https://andycloke.github.io/formula/) and test them in our [JavaScript Formula Playground](https://datafetcher.com/javascript-formula-playground).
### Create linked records
Data Fetcher can automatically create and update linked records during imports. This is useful when you have related data, such as tasks linked to users.
**Use an existing linked field:**
1. Select **Existing field** and choose a linked field from your destination table.
2. By default, Data Fetcher uses the primary field in the linked table to find records. To change this, click the field settings icon and select a different field under **Linked table field**.
**Create a new linked field:**
1. Select **New field**, enter a field name, and set the field type to **Linked record.**
2. Click the field settings icon and select a table to link to under **Linked table**.
When the request runs, Data Fetcher will create links by looking for response values in the linked table. If no matching record is found, it will create a new record automatically.
#### Link to multiple records
Enable comma-separated values to create multiple linked records:
1. Click the field settings icon next to the field name.
2. Under **Allow multiple links**, select **Allow**.
#### Create missing linked records
By default, Data Fetcher creates missing records in linked tables. You can disable this for read-only tables:
1. Click the settings icon next to the field name.
2. Under **Create missing records**, enable or disable **Create**.
### Handle attachment fields
Map remote file URLs (PNG, SVG, PDF, etc.) directly to Airtable attachment fields by selecting **Attachment** as the field type.
**Add to existing attachments** instead of overwriting:
1. Click the settings icon next to the attachment field.
2. Under **Attachment write mode**, enable **Add to existing**.
3. Click **Save**.
**Only write if cell is empty:**
1. Click the settings icon next to the attachment field.
2. Under **Write the attachment only if the cell is empty**, enable **Only if empty**.
3. Click **Save**.
## Access raw response data
### View raw response
Data Fetcher converts API responses into tabular format, but you can view the original JSON or XML:
1. Click the settings sidebar icon (➡️) on the left side.
2. Click **Response preview** to display the raw response.
**For JSON responses:** Click any value to toggle whether it's imported. Click the arrow next to field names to navigate directly to that field in the response.
**Note:** For paginated requests and Run for each record requests, the response field mapping shows only the first request's response data.
### Copy or download response
You can copy the JSON response to your clipboard or download it, which is helpful when working with JMESPath expressions:
1. Click the settings sidebar icon (➡️) on the left side.
2. Click **Response preview**.
3. Click **Copy** or **Download**.
## Access advanced settings
You can access the Advanced settings in the Response Field Mapping settings sidebar:
1. Click the settings sidebar icon (➡️) on the left side.
2. Click **Advanced settings**.
3. Make changes to [Response data transformations](/advanced-options/response-data-transformations), [Response JMESPath](/advanced-options/response-jmespath), or [Max response records](/advanced-options/max-response-records) and see the effects straight away.
## Troubleshooting tips
* Data Fetcher suggests field types based on response data, but you can modify these suggestions (for example, changing from Number to Currency)
* Be cautious when changing suggested data types, as incompatible values can cause import errors (for example, mapping text like "sdlf3421asd" to a Number field will fail)
# Run request using webhook URL
Source: https://help.datafetcher.com/create-requests/run-request-using-webhook
Every Data Fetcher request can be triggered automatically using its unique webhook URL. This allows external systems to run your requests by making HTTP calls to the webhook URL. For example, you can trigger data imports when a form is submitted on your website, when an e-commerce order is placed, or when events occur in other business tools.
You can also use webhook URLs within Airtable Automations to run requests based on any [automation trigger](https://support.airtable.com/docs/airtable-triggers).
This feature requires a paid Data Fetcher plan.
## Set up request webhook URL
### Authorize Data Fetcher
Before using webhooks, you need to authorize Data Fetcher to access your Airtable account. You only need to complete this once.
[For detailed authorization instructions, see our guide](/account/authorize-data-fetcher-airtable).
### Turn on the webhook URL
1. On the request screen, click the **Webhook URL** tab.
2. Toggle on **Turn on webhook for request**.
3. Copy the webhook URL by clicking the copy icon (📋) next to it. Your webhook URL will look similar to this:
```
https://hook.datafetcher.com/h/rj4X86TTJ747CAPj9gVU9mrKnN33wKqT46CbaaTxnaXqsr
```
4. Test the webhook by visiting the URL in your browser. You should see a success message, and you can verify the request ran by checking the Run History.
## Using webhook URLs
#### Run for all records
Once your webhook is active, calling the webhook URL will execute your request with its current configuration. This is useful for triggering data imports or updates from external systems.
#### Run for a specific record
You can target a specific record by adding a `record_id` parameter to the webhook URL. First, make sure you've selected an **input table** and **input view** in your request settings.
With the record ID parameter, your webhook URL becomes:
```
https://hook.datafetcher.com/h/rr4X86TTJ747CAPj9gVU9mrKnN33wKqT46CbaaTxnaXqsr?record_id=reca19WnKiU2L46cj
```
## Integrate with Airtable Automations
You can trigger Data Fetcher requests directly from Airtable Automations, allowing you to run requests when records are updated, forms are submitted, conditions are met, or any other automation trigger fires.
### Basic automation setup
1. Create an Airtable automation with your desired [trigger](https://support.airtable.com/airtable-triggers) (e.g., **When a record updated**).
2. For the automation action, select **Run a script**.
3. Add this script, replacing `YOUR_WEBHOOK_URL` with your actual webhook URL:
```
await fetch(`YOUR_WEBHOOK_URL`);
```
![Webhook URL script automation]()
### Target a specific record from automation
To run the request for only the record that triggered the automation:
1. In your Airtable automation, add an input variable called `record_id` with the record ID from the trigger.
2. Use this script instead, replacing `YOUR_WEBHOOK_URL` with your webhook URL:
```
await fetch(`YOUR_WEBHOOK_URL?record_id=${input.config().record_id}`)
```
This approach ensures your Data Fetcher request processes only the specific record that triggered the automation, making your workflows more efficient and targeted.
![Webhook URL script automation with record ID]()
# Schedule a request
Source: https://help.datafetcher.com/create-requests/schedule-a-request
Scheduled requests run automatically at regular intervals, keeping your data fresh without manual intervention. Your requests will execute even when you're not logged into Airtable or don't have the Data Fetcher extension open.
This feature requires a paid Data Fetcher plan.
## Set up scheduled requests
### Authorize Data Fetcher
Before scheduling requests, you need to authorize Data Fetcher to access your Airtable account using OAuth. You only need to complete this authorization once.
For detailed authorization instructions, see our [guide to authorization](/account/authorize-data-fetcher-airtable).
### Configure the schedule
1. Make sure you've completed the authorization steps above.
2. On the custom request screen, click the **Schedule** tab.
3. Toggle on **Schedule this request**.
4. Select your **Interval** from the dropdown (minutes, hours, days, etc.).
5. Choose the frequency - for example, every 1 hour or every 30 minutes.
6. If using minutes or hours intervals, select which days of the week the request should run.
7. Select a different timezone from the dropdown if needed (for intervals other than hours).
8. Click **Save** to activate your schedule.
## Monitor scheduled requests
Once your schedule is active, your request will run automatically according to the timing you configured. You can verify that scheduled runs are working by checking the [Run History](/organise-requests/run-history), which shows all automatic executions along with their timestamps and results.
Scheduled requests are perfect for regular data imports, daily reports, or any workflow that benefits from consistent, time-based updates.
# Use table values in requests
Source: https://help.datafetcher.com/create-requests/use-table-values-in-requests
You can use values from your Airtable tables directly in your Data Fetcher requests. This makes your requests dynamic, automatically pulling data from your base when they run.
Data Fetcher offers two ways to use table values, depending on whether you want to run one request per record or use a single value from your base.
**When should I use each option?**
Use **Run for each record** when you have a list of records and want one API call per record — for example, fetching stock prices for every ticker in your table.
Use **Single cell** when you need one dynamic value, like today's date, an API token, or a configuration setting.
## Run for each record
This mode runs one request per record in a view, making it ideal for bulk operations.
### Set up "Run for each record"
1. Click the **+** button in any compatible input, e.g. **URL** to open the **Add table field or variable** window.
2. Select the **Table** and **Field** you want to reference from the dropdowns.
3. Under **Use value from**, select **Each record (in a view)**.
4. Select the **Input view** containing the records you want to process. If one is already set for this request, it will be selected automatically.
5. Click **Confirm**.
Once set up, Data Fetcher will run one request for each record in the selected view, using that record’s field value in the request.
Each record processed counts as one run toward your monthly usage.
### Input table and input view
When using **Run for each record**, Data Fetcher needs to know which records to loop through. This is defined by the **Input table** and **Input view**.
* **Input table** determines which table contains the records to process.
* **Input view** determines which records in that table will run.
When you select **Each record (in a view)** in the **Add table field or variable** window, these settings are automatically configured for you based on your selection. You can adjust them directly in the main request settings.
The input table must match the table used in your reference. If they don’t match, Data Fetcher won’t know which records to iterate over.
When using Run for each record, only use **[Match records
using](/create-requests/write-and-merge-data#choose-a-field-to-match-on)** to
match records. Do not match by position, as each record's response is written
to the destination table individually.
### Continue through errors
When using Run for each record, you may want the request to continue running through errors. You can enable this by selecting **Continue through errors** under [Error Handling](/advanced-options/error-handling).
## Single cell
This method pulls a value from one specific cell, perfect for dynamic values or configuration settings.
### Set up a Single cell reference
1. Click the **+** button in any compatible input field to open the **Add table field or variable** window.
2. Select the **Table** and **Field** you want to reference.
3. Under **Use value from**, select **Single cell**, then select the specific record containing the cell value you need.
4. Click **Confirm**.
When the request runs, Data Fetcher uses the value from that selected cell.
You can edit any table reference by clicking on it to reopen the **Add table field or variable** window.
## Use dynamic values with formula fields
For values that change over time (like current dates), create Airtable formula fields and reference them using **Single cell**.
**Example: Using the current date and time**
Let's create a dynamic timestamp in the format `2023-03-22 12:00:00.000`:
1. Create a new table in Airtable called "Config" to keep things organized.
2. Delete all records except one.
3. Create a formula field called "Date today" with this formula:
```
DATETIME_FORMAT(NOW(), 'YYYY-MM-DD HH:mm:ss.sss')
```
4. Create a **Single cell** reference to this formula field in your request.
**Important:** Use a **Single cell** reference here, not **Each record**. Since this field contains only one value, Data Fetcher should read it once. Selecting **Each record** would cause errors because there’s no list of records to iterate through.
When your request runs (manually or automatically), Data Fetcher will use the current date and time value from the formula field.
## URL encoding for table references
When using table values inside URLs (for example, search queries or query parameters), special characters like spaces or `&` must be URL encoded. Data Fetcher can handle this automatically:
In the **Add table field or variable** window, use the **Encode URL table references** option at the bottom to enable or disable automatic URL encoding for your reference.
This ensures spaces and special characters are properly formatted in your request URL.
# Use variables in requests
Source: https://help.datafetcher.com/create-requests/use-variables-in-requests
Variables are fixed values that you save in your Data Fetcher workspace and can reuse across multiple requests. You can use variables in request URLs, headers, parameters, and other input fields throughout your requests.
Variables are especially useful for API keys, base URLs, or other values that you use frequently. If a value changes, you only need to update the variable in one place, instead of editing every request that uses it.
## How to create and use a variable
You can create a variable directly from any compatible input field:
1. Look for any input field that displays a **+** button on the right-hand side.
2. Click the **+** button to open the table field / variables modal.
3. In the modal that opens, click **Variable** to switch to the variables tab.
4. Enter a **Variable name**, and **Variable value**.
5. Click **+ Add**. Your variable is now available to use in any request within this workspace.
6. Click **Confirm**.
7. The input field will now display a variable reference that looks like: `-*-currency-*-`
To re-use an existing variable, follow steps 1-3 above, then select the variable you want to use from the list and click **Confirm**.
When your request runs, Data Fetcher automatically replaces the variable reference with the actual variable value.
## How to edit and delete variables
You can edit or delete existing variables through the same modal:
1. Follow steps 1-3 above to open the variables tab.
2. Find the variable you want to manage.
3. Click the three horizontal dots (⋯) on the right-hand side of the variable.
4. Select your desired action from the dropdown menu (edit or delete).
Changes to variables will affect all requests that use them within your
workspace.
## Common use cases for variables
* **API keys and tokens** - Store sensitive authentication information once and use it across multiple requests
* **Base URLs** - Define common API endpoints that multiple requests share
* **Configuration values** - Store settings like currency codes, region identifiers, or format preferences
Variables help keep your requests organized, secure, and easy to maintain as your API integrations grow.
# Write and merge data
Source: https://help.datafetcher.com/create-requests/write-and-merge-data
When Data Fetcher runs your request, it writes the API response data to your destination table & view. By default, **Merge with existing records** is turned on. This means Data Fetcher updates matching records in your destination view instead of always creating new ones.
## Merge with existing records
With **Merge with existing records** enabled, Data Fetcher updates records already in your destination view instead of creating new ones.
If no match field is selected, records are updated by position — the first record in the API response updates the first record in your destination view, and so on.
**Example:** Your current destination view contains:
| ID | Name | **Units sold** |
| --- | ------------ | -------------- |
| 229 | Orange Hat | 5 |
| 107 | Purple Skirt | 10 |
The API response:
| ID | Name | **Units sold** |
| :-- | :----------- | :------------- |
| 692 | Blue T-Shirt | 7 |
| 229 | Orange Hat | 10 |
| 107 | Purple Skirt | 17 |
After running with merge enabled but no match field, records update by position:
| ID | Name | **Units sold** |
| :-- | :----------- | :------------- |
| 692 | Blue T-Shirt | 7 |
| 229 | Orange Hat | 10 |
| 107 | Purple Skirt | 17 |
This behavior can lead to records being overwritten incorrectly. The first record that was originally "Orange Hat" is now "Blue T-Shirt." Any existing links, comments, or custom field values tied to that record are now associated with the wrong item.
To avoid this, select a match field.
### Choose a field to match on
Select a field under **Match records using** to match incoming records with existing records by value instead of position.
1. Turn on **Merge with existing records** (enabled by default).
2. Under **Match records using**, choose a field with unique values (e.g. ID or email).
3. You can choose more than one field if needed. Data Fetcher will use the unique combination of values across the selected fields to match records.
With "ID" selected as the match field, the same API response now updates correctly:
| ID | Name | **Units sold** |
| :-- | :----------- | :------------- |
| 229 | Orange Hat | 10 |
| 107 | Purple Skirt | 17 |
| 692 | Blue T-Shirt | 7 |
Existing records are updated in place using the ID to match, preserving their identity and relationships. The new "Blue T-Shirt" record is added separately.
For best results, use a field with unique values. If no single field is
unique, select multiple fields whose combination is unique.
### Create new records if no match
When enabled (the default), any records in the API response that don't match an existing record in your destination view are created as new records.
Turn this off if you only want to update existing records and ignore new ones.
### Delete records not in response
When enabled, Data Fetcher removes records from your destination view that no longer appear in the API response. This is useful when the source has deleted data and you want your table to stay in sync.
This only affects records visible in your destination view.
## Important: destination view matters
Data Fetcher only looks at records in your selected destination view, not the entire destination table.
If your destination view has Airtable filters, Data Fetcher may not find existing records to match against. This can cause duplicate records to be created or incorrect deletions.
**Recommendation:** Create a dedicated, locked view with no filters to use as your destination view.
## Append only (create new records)
Turn off **Merge with existing records** to switch to append behavior. Each run creates new records without modifying existing records.
This is useful for tracking data over time, like daily sales snapshots.
Using the same example data, after running with merge turned off:
| ID | Name | **Units sold** |
| :-- | :----------- | :------------- |
| 229 | Orange Hat | 5 |
| 107 | Purple Skirt | 10 |
| 692 | Blue T-Shirt | 7 |
| 229 | Orange Hat | 10 |
| 107 | Purple Skirt | 17 |
With a "Created time" field in Airtable, you can use this to track how values change over time.
# Create a sequence
Source: https://help.datafetcher.com/create-sequences/create-a-sequence
## What is a sequence?
A sequence is a powerful workflow automation feature that allows you to run multiple API requests in a specific order. When you create a sequence, Data Fetcher executes each request one after the other. You can use Airtable as a bridge between requests - writing data with one request and referencing it in subsequent ones, creating connected workflows. Sequences can be triggered manually, scheduled to run automatically, via triggers or webhook URLs—just like individual requests.
## Create and run a sequence
1. First, create all the individual requests you want to include in your sequence.
2. From the Data Fetcher home screen, click the small arrow to the right of the **Create request** button.
3. Select **Create sequence** from the dropdown menu.
4. On the sequence screen, provide a descriptive name for your sequence.
5. Under the **Requests** section, select all the requests you want to include in your workflow. You can add more requests with the **+ Add request** button.
6. Click **Save** to store your sequence.
7. To run your sequence, click the **Run** button on the sequence screen.
8. Data Fetcher will save your sequence, navigate to each request in order, run it, and then return to the sequence screen when complete.
## Handling request failures
By default, if any request in your sequence fails, Data Fetcher will stop the entire sequence. However, you can customize this behavior:
1. On the sequence screen, locate the **Failure Handling** section.
2. Enable the option **If one of these requests fails, run the next request anyway**.
3. With this setting enabled, Data Fetcher will continue executing the remaining requests in your sequence even if one fails.
## Connecting requests through Airtable
Since Data Fetcher doesn't directly pass data between sequence steps, you can use Airtable as an intermediary:
1. In your first request, write data to Airtable (like API responses, tokens, or IDs).
* Example: GET a list of customer data and write it to an Airtable base.
2. In your subsequent requests, read from that same Airtable base.
* Example: For each customer record, fetch the full customer object from the API.
* Example: Enrich customers based on their email address.
This approach creates a workflow where each step builds upon the results of previous steps.
# Run sequence from button field
Source: https://help.datafetcher.com/create-sequences/run-sequence-from-button-field
You can run a sequence by clicking a button field in your table. This makes it quicker to execute sequences without having to manually navigate through the extension dashboard.
Button fields work similarly for both requests and sequences. This guide focuses on sequences specifically.
## Run a sequence from a button field
1. Create and save your sequence first. You can close the extension dashboard afterward, but remain logged into Data Fetcher.
2. [Create a button field](https://support.airtable.com/docs/button-field) in Airtable that opens Data Fetcher.
3. Click any button in this new button field.
4. A modal will appear asking you to select which sequence to run. Use the **Sequence** dropdown menu to make your selection.
5. Click **Save** to run the sequence immediately.
6. For future uses, simply click the button field and the sequence will run automatically without prompting for selection.
## Run different sequences from different buttons
This functionality works identically to request buttons. See [Run different requests from different buttons](/advanced-options/run-request-from-button-field#run-different-requests-from-different-buttons) for complete instructions.
## Running a sequence once versus for each record
This functionality works identically to request buttons. See [Choose your execution scope](/advanced-options/run-request-from-button-field#choose-your-execution-scope) for detailed guidance.
# Run sequence using webhook URL
Source: https://help.datafetcher.com/create-sequences/run-sequence-using-webhook
A webhook URL allows you to run a sequence automatically when specific events occur, such as when an Airtable record updates or when someone submits a form.
This feature requires a paid Data Fetcher plan.
## Enable the Webhook URL for a Sequence
To make your sequence runnable via webhook URL:
1. On the sequence screen, click the **Webhook URL** tab.
2. Toggle on **Turn on webhook for sequence.**
3. Click the clipboard icon (📋) to copy your unique webhook URL, which looks like:
```
https://hook.datafetcher.com/h/sr4X86TTJ747CAPj9gVU9mrKnN33wKqT46CbaaTxnaXqsr
```
4. Test your webhook by visiting the URL in your browser - you should see a success message and the sequence will run immediately.
## Run sequence for a specific record
To run your sequence for just one particular record:
1. Make sure all requests in your sequence use the same **input table**.
2. Add the `record_id` parameter to your webhook URL like this:
```
https://hook.datafetcher.com/h/sr4X86TTJ747CAPj9gVU9mrKnN33wKqT46CbaaTxnaXqsr?record_id=reca19WnKiU2L46cj
```
3. The record ID used must belong to a record in your specified table.
## Set up an Airtable automation
You can trigger your sequence from Airtable automations when specific events occur:
1. In Airtable, create a new automation with your desired trigger (e.g., **When a record is updated**).
2. Add a **Run a script** action.
3. Paste the following code into the script, replacing `YOUR_WEBHOOK_URL` with your sequence's webhook URL:
```
await fetch(`YOUR_WEBHOOK_URL`);
```
![Webhook URL script automation]()
# Schedule a sequence
Source: https://help.datafetcher.com/create-sequences/schedule-a-sequence
Data Fetcher allows you to schedule sequences to run automatically on a regular basis, even when you're not logged into Airtable or don't have the Data Fetcher extension open.
This feature requires a paid Data Fetcher plan.
To schedule sequences, you will first need to [authorize Data Fetcher and Airtable](/account/authorize-data-fetcher-airtable).
## Schedule a sequence
After authorizing Data Fetcher, you can create your schedule:
1. On the sequence screen, find the **Schedule** section.
2. Toggle on **Schedule this sequence.**
3. Select an **Interval** from the dropdown menu (Minutes, Hours, Days, Weeks, Months).
4. Specify the frequency (e.g., every 1 hour, every 3 days).
5. If applicable, select which days of the week the sequence should run.
6. Optional: Change the timezone if needed
* This option is available for all intervals except **Hours**
7. Click **Save** to activate your schedule.
Your sequence will now run automatically according to your specified schedule, without requiring you to manually trigger it.
# Add request body
Source: https://help.datafetcher.com/custom-requests/add-request-body
Adding a request body allows you to send data from your Airtable base to other platforms. You can include table values, variables, and even attachment files in your request body.
Data Fetcher supports seven different body types: JSON, x-www-form-urlencoded, form-data, XML, GraphQL, Binary, and Plain Text. You'll typically want to add a request body when using POST, PATCH, or PUT request methods. Check your target API's documentation to determine which body type to use.
## How to add a request body
Follow these steps to add any type of request body to your Data Fetcher request:
1. On the create request screen, click the **Body** tab.
2. Select the tab for your desired body type (JSON, x-www-form-urlencoded, form-data, XML, GraphQL, Binary, or Plain Text).
3. Follow the specific instructions below for your chosen body type.
### JSON request body
JSON is commonly used for REST APIs and web services.
1. Make sure the **JSON** tab is active.
2. Enter your JSON data into the input field. You can validate your JSON using [this tool](https://jsonlint.com).
3. Use the **Beautify** button to format your JSON for better readability.
### x-www-form-urlencoded request body
This format sends data as key-value pairs, similar to HTML form submissions.
1. Click the **x-www-form-urlencoded** tab.
2. Click **+ Add** to create a new key-value pair.
3. Enter a key and its corresponding value in the input fields.
4. Repeat steps 2-3 to add additional key-value pairs as needed.
### form-data request body
Form-data is useful for sending files or when the API specifically requires this format.
1. Click the **form-data** tab.
2. Click **+ Add** to create a new key-value pair.
3. Enter a key and its corresponding value in the input fields.
4. Repeat steps 2-3 to add additional key-value pairs as needed.
### XML request body
XML format is often used by older APIs or enterprise systems.
1. Click the **XML** tab.
2. Enter your XML content in the input field. You can validate your XML using [this tool](https://www.xmlvalidation.com).
### GraphQL request body
GraphQL allows you to query specific data fields and is automatically converted to JSON when sent.
1. Click the **GraphQL** tab.
2. Add your GraphQL query in the **Query** input field.
3. Add any variables as JSON in the **Variables** input field.
4. When the request runs, Data Fetcher automatically converts your GraphQL query and variables into a JSON body. You can preview this conversion using our GraphQL to JSON Body Converter.
### Binary request body
Binary format sends raw file data, perfect for uploading attachments from Airtable.
1. Click the **Binary** tab.
2. Use the **+** button on the right side of the input field to reference an attachment field from your Airtable base.
3. When the request runs, the raw attachment data will be sent directly in the request body.
### Plain Text request body
Plain text format sends simple text content without any special formatting or encoding.
1. Click the **Plain Text** tab.
2. Enter your text content directly in the input field.
3. The text will be sent exactly as entered in the request body.
# Connect to a preset OAuth API
Source: https://help.datafetcher.com/custom-requests/connect-to-a-preset-oauth-api
## What is OAuth?
OAuth is an authorization standard that allows you to use your existing accounts on platforms like Google and Facebook to access third-party websites and apps. When you connect via OAuth, you're authorizing these services to perform actions on your behalf.
Under the hood, OAuth works by obtaining and refreshing access tokens. These tokens are obtained when you first log in to the platform and are automatically refreshed as needed. They're then sent along with API requests to authorize each request to the third-party platform.
## Create an OAuth connection
When you create OAuth connections in Data Fetcher, they're saved in your workspace. This means you only need to create them once and can then use them for multiple requests.
1. On the create request screen, click **Authorization**.
2. Use the **Type** dropdown to select **OAuth**.
3. Next to **Connection**, click **+ New**.
4. Click the button for the application you want to connect to (for example, **Google Ads**). A new window will open asking for permission to connect your account to Data Fetcher.
5. Configure the rest of your API details like URL as you normally would and run the request.
Authorization headers and parameters will be automatically sent when you run the request.
## Rename an OAuth Connection
1. Hover over the extension, then click the Settings icon (⚙️) to open the settings screen.
2. Under **Workspaces**, click **Manage** next to the workspace you want.
3. Under **OAuth Connections**, click the pencil icon next to the connection you want to rename.
4. In the modal, enter the new name in the input field.
5. Click **Save**.
## Delete an OAuth Connection
1. Hover over the extension, then click the Settings icon (⚙️) to open the settings screen.
2. Under **Workspaces**, click **Manage** next to the workspace you want.
3. Under **OAuth Connections**, click the trash icon next to the connection you want to delete.
4. Click **Delete** in the modal that opens.
Some platforms like Google will revoke all your OAuth connections when you revoke one connection. For example, deleting Google Analytics would also revoke Google Ads.
## Create Custom OAuth Connection
If you don't see the OAuth API you want to connect to in the list of preset options, you can [create a custom OAuth connection](/custom-requests/create-a-custom-oauth-connection).
# Create a custom OAuth connection
Source: https://help.datafetcher.com/custom-requests/create-a-custom-oauth-connection
## When do I need to create a custom OAuth connection?
You'll need to create a custom OAuth connection when:
* The API you want to connect to only provides access via OAuth (not API keys or other authorization methods).
* Data Fetcher does not have a [preset OAuth connection](/custom-requests/connect-to-a-preset-oauth-api) for that specific API.
## Step 1. Set up an app with the API provider
Most APIs require you to create an app in their developer portal before you can use OAuth. Check the OAuth or Authorization section of the API documentation for specific steps, which typically involve:
1. Navigating to the developer section of your account
2. Creating a new app or application
3. Configuring the app settings
**Important:** When asked for a redirect URL during app setup, use this Data Fetcher redirect URL:
Copy
```
https://oauth.datafetcher.com/auth/callback
```
After completing the setup, you'll receive a **client ID** and **client secret**. Save these securely - you'll need them in the next step.
## Step 2: Create the custom OAuth connection in Data Fetcher
1. [Create a custom request](/custom-requests/create-a-custom-request) in Data Fetcher.
2. Click on the **Authorization** tab.
3. For **Type**, select **OAuth**.
4. Click the **+ New** button.
5. At the bottom of the modal that opens, click **create custom OAuth connection**.
This will open the custom OAuth connection screen, where you'll enter the following details:
### Required fields
* **Name**: a descriptive name to help you identify this OAuth connection later.
* **Auth type**: choose OAuth 1 or OAuth 2 based on what the API documentation specifies (most modern APIs use OAuth 2)
* **Authorization URL**: the URL used for the authorization request\*\*, \*\*when Data Fetcher opens a new tab and asks for permission to access the API (typically looks like `https://example.com/oauth/v2/auth`).
* **Token URL**: the URL used after the authorization request to generate access tokens from the authorization code. This is typically something like `https://example.com/oauth/v2/token`.
* **Client ID**: the app client ID from step 1.
* **Client secret**: the app client secret from step 1.
* **Base URL**: the root URL that all requests to this API use. e.g. `https://example.com/api/`.
* **Scopes (comma separated)**: a comma separated list of the scopes that you want to access for this API. You can find these in the API documentation.
### Optional fields
* **Authorization Parameters**: the parameters that will be added to the Authorization URL when making the authorization request. The API documentation will have a list of these parameters. Data Fetcher will automatically include the standard OAuth parameters `(response_type,client_id,redirect_uri,state,client_secret)`.
* **Token Parameters**. The parameters that will be added to the Token URL when making the token request. Data Fetcher will automatically add the `grant_type=authorization_code` parameter. If you need to use `grant_type=client_credentials` instead, simply add this as a token parameter.
* **Request Headers**: any extra headers to be supplied with every request when using this custom OAuth connection.
#### Complete the setup
After entering all details, click **Create connection**. Data Fetcher will open a new tab to begin the authorization process. If your details are correct, you'll be able to authorize Data Fetcher and start using this custom OAuth connection in your requests.
## Example: Connecting to Zoho People API
Here's how to set up a custom OAuth connection for the Zoho People API:
1. **Register with Zoho:** Go to the Zoho API Console and register a new client to get your client ID and client secret
![Zoho OAuth setup]()
2. **Configure in Data Fetcher:** Using the Zoho People API OAuth documentation, we enter these values:
| Setting | Value |
| ------------------------ | ------------------------------------------------------------------------------------ |
| Auth type | OAuth 2 |
| Authorization URL | [https://accounts.zoho.com/oauth/v2/auth](https://accounts.zoho.com/oauth/v2/auth) |
| Authorization Parameters | Client secret from step 1. |
| Token URL | [https://accounts.zoho.com/oauth/v2/token](https://accounts.zoho.com/oauth/v2/token) |
| Client ID | Client ID from step 1. |
| Client secret | Client secret from step 1. |
| Scopes | ZOHOPEOPLE.forms.ALL |
| Request headers | Authorization=Bearer \$auth.accessToken |
| Base URL | [https://people.zoho.com/people/api/](https://people.zoho.com/people/api/) |
# Create a custom request
Source: https://help.datafetcher.com/custom-requests/create-a-custom-request
Custom requests allow you to connect to any REST or GraphQL API directly in Airtable without writing code. You can configure the request URL, method, headers, body, authorization, pagination, and more using an interface similar to popular API clients like Postman or Insomnia.
## How to create a custom request
1. On the create request screen, under **Application**, select **Custom**.
2. Enter the complete request **URL** in the URL input field. You can find this in your target API's documentation - it consists of a base URL plus the specific endpoint path.
For example, use this URL to fetch CoinGecko crypto prices:
```
https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd
```
You can paste a cURL command directly into the URL input field, and Data Fetcher will automatically import all the request settings.
### Configure request options
Set up any additional options your API requires:
* [**Method**](/custom-requests/set-request-method) - Choose GET, POST, PUT, PATCH, DELETE, or HEAD
* [**Parameters**](/custom-requests/set-request-parameters) - Add query parameters to modify the API response
* [**Authorization**](/custom-requests/request-authorization) - Configure authentication (Bearer token, Basic auth, OAuth, etc.)
* [**Headers**](/custom-requests/set-request-headers) - Add custom headers required by your API
* [**Body**](/custom-requests/add-request-body) - Include request body data for POST, PUT, or PATCH requests
* [**Pagination**](/advanced-options/pagination) - Set up automatic handling of paginated responses
### Complete the setup
1. Enter a descriptive name for your request in the input field at the top of the screen.
2. Select your **Destination Table and View** where the imported data should go. By default, this uses your current table and view.
3. Click **Save & Run** in the bottom right corner.
## Configure field mapping and run
When you first run a request, the Response Field Mapping window opens. This is where you tell Data Fetcher how to match API response fields with your Airtable fields.
**Field selection:**
* Click field names to select or deselect them for import
* Only selected fields will be imported when the request runs
**Field mapping:**
* Map each selected field to either an **Existing field** in your table or create a **New field**
* Choose the appropriate field types for new fields
Once you're satisfied with the mapping, click **Save & Run** in the bottom right corner.
Your request will execute and import data into your Airtable base according to your configuration. You can click **Run** anytime to import the latest data.
## Next steps
Now that you have created a request, explore these key features:
Run automatically at intervals, even when you're not using Airtable
Run requests instantly when your base data changes
Control how records are created and updated in your table
# Request authorization
Source: https://help.datafetcher.com/custom-requests/request-authorization
Many APIs require authentication to verify your identity and authorize access to their services. Data Fetcher makes this easy by automatically handling the necessary headers and parameters when you configure authorization on the **Authorization** tab.
Data Fetcher supports six different authentication methods:
* [Bearer token](/custom-requests/request-authorization#bearer-token-authentication)
* [Basic authentication](/custom-requests/request-authorization#basic-authentication)
* [OAuth](/custom-requests/request-authorization#oauth-authentication)
* [Header](/custom-requests/request-authorization#header-authentication)
* [Parameter](/custom-requests/request-authorization#parameter-authentication)
* [AWS signature V4](/custom-requests/request-authorization#aws-signature-v4-authentication)
Check your API's documentation for an "Authentication" section that explains which method to use.
## How to set up authorization
All authorization methods follow the same initial steps:
1. On the custom request screen, click the **Authorization** tab.
2. Use the **Type** dropdown to select your authentication method.
3. Fill in the required fields for your chosen method (detailed below).
4. When your request runs, Data Fetcher automatically adds the necessary authentication information.
### Bearer token authentication
Bearer tokens are commonly used by modern APIs and services.
1. Select **Bearer Token** from the Type dropdown.
2. Enter your bearer token in the **Token** input field.
3. When the request runs, Data Fetcher sends an `Authorization` header with the value `Bearer YOUR_BEARER_TOKEN`.
### Basic authentication
Basic authentication uses a username and password combination.
1. Select **Basic** from the Type dropdown.
2. Enter your username in the **Username** input field.
3. Optionally, enter your password in the **Password** input field.
4. When the request runs, Data Fetcher automatically sends an `Authorization` header with the value `Basic username:password` (base64 encoded).
### OAuth authentication
OAuth provides secure, token-based authentication for third-party services. Data Fetcher offers two OAuth options:
**Preset OAuth connections** - For popular APIs like Google, Facebook, and other major platforms that Data Fetcher supports out of the box. [See our guide on how to connect to a preset OAuth API](/custom-requests/connect-to-a-preset-oauth-api).
**Custom OAuth connections** - For APIs that require OAuth but aren't available as preset options. [See our guide on how to create a custom OAuth connection](/custom-requests/create-a-custom-oauth-connection).
### Header authentication
Some APIs require custom authentication headers with specific names and values.
1. Select **Header** from the Type dropdown.
2. Enter the required header name in the **Name** input field.
3. Enter the corresponding header value in the **Value** input field.
4. When the request runs, Data Fetcher adds a header with the exact name and value you specified.
### Parameter authentication
Parameter authentication adds authentication information directly to the request URL.
1. Select **Parameter** from the Type dropdown.
2. Enter the parameter name in the **Name** input field.
3. Enter the parameter value in the **Value** input field.
4. When the request runs, Data Fetcher adds the parameter to the URL with the name and value you specified.
### AWS Signature V4 authentication
AWS Signature V4 is required for authenticating requests to Amazon Web Services.
1. Select **AWS Signature V4** from the Type dropdown.
2. Enter your AWS Access Key in the **Access key** input field.
3. Enter your AWS Secret Key in the **Secret key** input field.
4. Optionally, specify the AWS **Region**. If left blank, Data Fetcher attempts to infer it from the URL.
5. Optionally, specify the AWS **Service**. If left blank, Data Fetcher attempts to infer it from the URL.
6. If using temporary security credentials, enter your AWS **Session token**.
7. When the request runs, Data Fetcher automatically calculates and adds the `X-Amz-Content-Sha256`, `X-Amz-Date`, and `Authorization` headers required for AWS authentication.
# Set request headers
Source: https://help.datafetcher.com/custom-requests/set-request-headers
HTTP headers allow you to send additional information with your API requests, such as authentication tokens, API version specifications, or content type declarations. Many APIs require specific headers to function properly.
Check your target API's documentation to see which headers you need to include with your requests. You can also [learn more about HTTP headers in general here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers).
## How to manage request headers
### Add a request header
1. On the custom request screen, click the **Headers** tab.
2. Click **+ Add** to create a new header.
3. Enter the header name in the left input field and its corresponding value in the right input field.
4. Use the **+** icon next to the value field to [insert table values from your Airtable base](/create-requests/use-table-values-in-requests) into the header.
5. Repeat steps 2-4 to add additional headers as needed.
### Remove a request header
1. On the create request screen, click the **Headers** tab.
2. Find the header you want to remove.
3. Click the **-** button next to that header to delete it.
## Common header examples
Here are some frequently used headers you might need:
* **Authorization**: Used for API authentication (e.g., `Bearer your-token-here`)
* **Content-Type**: Specifies the format of your request body (e.g., `application/json`)
* **Accept**: Tells the API what response format you prefer (e.g., `application/json`)
* **User-Agent**: Identifies your application to the API server
# Set request method
Source: https://help.datafetcher.com/custom-requests/set-request-method
The HTTP request method determines what action your API request will perform. Different methods serve different purposes: `GET` requests retrieve data from an API to import into Airtable, while `POST` requests send data from Airtable to an external API.
Your API's documentation will specify which method to use for each endpoint. You can also learn more about HTTP request methods in general [here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods).
## How to set the request method
1. On the custom request screen, locate the **Method** dropdown (to the left of the **URL** input field).
2. Click the dropdown to see available methods.
3. Select the method you need for your specific API endpoint.
## Common request methods
Here are the most frequently used HTTP methods and their typical purposes:
* **GET** - Retrieve data from the API (most common for importing data into Airtable)
* **POST** - Send new data to the API (creating new records or resources)
* **PUT** - Update or replace existing data on the API
* **PATCH** - Partially update existing data on the API
* **DELETE** - Remove data from the API
* **HEAD** - Check if a resource exists or get metadata without downloading the full response
The method you choose should match what your API endpoint expects and what action you want to perform.
# Set request parameters
Source: https://help.datafetcher.com/custom-requests/set-request-parameters
Query parameters are key-value pairs added to URLs that help modify what data an API returns. They appear after a question mark in the URL, like `?ticker=IBM&includePrices=true`. Many APIs use parameters to filter results, specify data formats, or control response details.
You can add parameters in two ways: manually type them directly into the **URL** input field, or use the **Parameters** tab for easier management and automatic URL formatting.
### Before you begin
Make sure you've entered the complete API base URL and endpoint path in the **URL** input field. For example:
```
https://api.coingecko.com/api/v3/coins/markets
```
## How to manage query parameters
### Add a query parameter
1. On the custom request screen, click the **Parameters** tab.
2. Click **+ Add** to create a new parameter.
3. Enter the parameter name in the left input field and its corresponding value in the right input field.
4. The parameter will automatically be added to your **URL** with proper formatting.
5. Use the **+** icon next to the value field to insert table values from your Airtable base into the parameter.
6. Repeat steps 2-5 to add additional parameters as needed.
### Remove a query parameter
1. On the custom request screen, click the **Parameters** tab.
2. Find the parameter you want to remove.
3. Click the **-** button next to that parameter to delete it.
4. The parameter will automatically be removed from your **URL**.
Reference Airtable values in parameters by clicking the **+** icon located on the
right-hand side of an input.
# Get a Google Maps API key
Source: https://help.datafetcher.com/get-a-google-maps-api-key
When using the Google Maps no-code integration in Data Fetcher, you will need to add an API key. You create this API key in Google Cloud Console. The steps to do this are different for different endpoints. Use the instructions for the endpoint you are using:
| **Endpoints** | **Step 1 (Billing & Enable API) Instructions** | **Step 2 (Create API Key) Instructions** | **Pricing Details** |
| ------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Find a place from name/address/phone number
Find details for a place id | [https://developers.google.com/maps/documentation/places/web-service/get-api-key](https://developers.google.com/maps/documentation/places/web-service/get-api-key) | [https://developers.google.com/maps/documentation/places/web-service/get-api-key#creating-api-keys](https://developers.google.com/maps/documentation/places/web-service/get-api-key#creating-api-keys) | [https://developers.google.com/maps/documentation/places/web-service/usage-and-billing](https://developers.google.com/maps/documentation/places/web-service/usage-and-billing) |
| Calculate distance and time between start and end locations | [https://developers.google.com/maps/documentation/distance-matrix/cloud-setup](https://developers.google.com/maps/documentation/distance-matrix/cloud-setup) | [https://developers.google.com/maps/documentation/distance-matrix/get-api-key#creating-api-keys](https://developers.google.com/maps/documentation/distance-matrix/get-api-key#creating-api-keys) | [https://developers.google.com/maps/documentation/distance-matrix/usage-and-billing](https://developers.google.com/maps/documentation/distance-matrix/usage-and-billing) |
| Geocode - find latitude/ longitude from address
Reverse geocode - find address from latitude/ longitude | [https://developers.google.com/maps/documentation/geocoding/get-api-key](https://developers.google.com/maps/documentation/geocoding/get-api-key) | [https://developers.google.com/maps/documentation/geocoding/get-api-key#creating-api-keys](https://developers.google.com/maps/documentation/geocoding/get-api-key#creating-api-keys) | [https://developers.google.com/maps/documentation/geocoding/usage-and-billing](https://developers.google.com/maps/documentation/geocoding/usage-and-billing) |
# Welcome
Source: https://help.datafetcher.com/index
Welcome to Data Fetcher, the no-code API connector for Airtable.
Create a request and import data in minutes
## Common next steps
Automate data syncs into Airtable
Reference Airtable values dynamically
Chain multiple requests together
Look up solutions to common issues
## Support
We're here to help you succeed with Data Fetcher. The fastest way to get help is to email [support@datafetcher.com](mailto:support@datafetcher.com).
We usually respond within a couple of hours (Monday to Friday, 9am to 6pm UK time).
## Learning resources
* [📺 Watch tutorials on YouTube](https://www.youtube.com/channel/UCnED2FBe_jzntrCiB759y9w/videos)
* [⚙️ Read guides to each integration](https://datafetcher.com/airtable-integrations)
* [✍️ Read the latest tutorials on our blog](https://datafetcher.com/blog)
# Add request description
Source: https://help.datafetcher.com/organise-requests/add-request-description
You can add a description to your request to include helpful details or notes that make it easier to remember its purpose. Descriptions appear when hovering over the info icon (ℹ️) on both the request and home screens.
## How to add a description to a request
Follow these steps to add a description to your Data Fetcher request:
1. On the request screen, click the info icon (ℹ️) located next to the request name at the top of the screen.
2. Enter your description text in the input field that appears.
3. Click **Confirm** to save your description.
4. Click **Save** to finalize your request with the new description.
5. Once saved, you can view the description by hovering over the info icon on either the request screen or the home screen.
# Delete a request
Source: https://help.datafetcher.com/organise-requests/delete-a-request
You can remove saved requests you no longer need from your Data Fetcher workspace using either the extension or the website.
## How to delete a request from the extension
1. Open the Data Fetcher extension.
2. On the home screen, locate the request you want to delete.
3. Hover your mouse over the request and click the **...** (three dots) that appears on the right-hand side.
4. Click **Delete** from the dropdown menu.
5. Click **Confirm** to permanently remove the request.
## Delete a request when the Airtable base has been deleted
If you've already deleted the Airtable base containing your request, you'll need to use the Data Fetcher website instead:
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click **Requests** in the top menu.
4. Find the request you want to delete and click the three dots (**...**) next to its name.
5. Click the **Delete** button (🗑️) that appears in the popup menu.
6. Click **Delete** on the confirmation modal to permanently remove the request.
Note that deleted requests cannot be recovered, so make sure you no longer need the request before deleting it.
# Duplicate a request
Source: https://help.datafetcher.com/organise-requests/duplicate-a-request
Duplicating requests saves you time by creating an identical copy of a configured request instead of starting from scratch.
## How to duplicate a request
You can duplicate a request in two ways:
### From the home screen:
1. Navigate to the home screen in Data Fetcher.
2. Locate the request you want to duplicate.
3. Hover your mouse over the request, and click the three dots (...) on the right-hand side.
4. Select **Duplicate** from the menu.
A new request will immediately appear in your list, containing all the same settings and parameters as the original.
### From the request screen:
1. Open the request you want to duplicate.
2. Click the **Duplicate** button in the top menu.
A new request will be created, and you'll be automatically taken to the duplicated request screen, where you can begin making changes.
# Download requests as JSON file
Source: https://help.datafetcher.com/organise-requests/export-requests-as-json-file
Downloading requests creates JSON files you can re-use in other bases. Only request configurations and response JSON are exported, not workspace data, base information, or personal settings.
## How to download requests as JSON files
1. Sign in to your account and navigate to the Data Fetcher extension home screen.
2. Hover your mouse over the requests you want to download and click the empty checkbox (☐) next to each one to select them.
3. Once you've selected all desired requests, click the ⬇️ button (**Download requests)** that appears in the menu bar.
4. In the download requests modal, click **Download** to save the JSON file to your computer.
The file will be downloaded to your default downloads location with a name based on the current time.
## Security considerations when exporting requests
**Important:** Be mindful about sensitive information when exporting requests. Your export may contain:
* API keys and authentication tokens
* Private endpoint URLs
* Custom headers with sensitive information
* Personally identifiable information in request parameters or response JSON
For better security, consider these best practices:
1. [Replace sensitive values with variables](/create-requests/use-variables-in-requests) before exporting, so only variable names (not actual values) are included in the export file.
2. Review the JSON file before sharing it with others to ensure no confidential information is accidentally disclosed.
3. If sharing exported requests with team members, use secure channels for file transfer.
Your exported requests can be imported into any Data Fetcher installation using the [Import from JSON feature](/organise-requests/import-requests-from-json-file).
# Import request from cURL command
Source: https://help.datafetcher.com/organise-requests/import-request-from-curl-command
Save time by importing cURL commands directly into Data Fetcher. This feature allows you to instantly set up requests without manually configuring headers, parameters, or authentication details.
## What is cURL?
cURL (Client URL) is an industry-standard command-line tool for transferring data and making API requests. You'll commonly encounter cURL commands:
* Throughout API documentation
* In code samples shared by developers
* When [inspecting network requests in your browser's developer tools](https://www.scrapingbee.com/tutorials/how-to-extract-curl-requests-from-chrome/)
For example, you might want to import the following cURL command from [Stripe's API documentation](https://stripe.com/docs/api/customers/list):
```bash theme={null}
curl -G https://api.stripe.com/v1/customers \
-u 2430578aslkdfjh43254332: \
-d limit=3
```
## How to import cURL commands into Data Fetcher
### Method 1: Direct paste into URL field
1. [Create a new custom request](/custom-requests/create-a-custom-request) or open an existing one
2. Find the **URL** input field at the top of the request screen
3. Paste your entire cURL command into this field
4. Data Fetcher will instantly parse the command and populate all relevant fields (headers, parameters, authentication)
### Method 2: Import via home screen
1. Navigate to the Data Fetcher home screen
2. Click the ⬆️ button (**Upload requests or import cURL**) in the top menu
3. Select the **Import from cURL** tab
4. Enter your cURL command into the text field
5. Click **Import** to create a new pre-configured request
# Upload requests from JSON file
Source: https://help.datafetcher.com/organise-requests/import-requests-from-json-file
Data Fetcher allows you to easily upload requests from previously [downloaded JSON files](/organise-requests/export-requests-as-json-file), making it simple to share request configurations across different bases or workspaces.
Alternatively, if you want to share requests with your team, you can [add team members to your Data Fetcher workspace](/account/add-user-to-account).
Data Fetcher also supports [importing from cURL commands](/organise-requests/import-request-from-curl-command).
## How to import requests from a JSON file
Follow these steps to import previously exported Data Fetcher requests:
1. Click the ⬆️ button (**Upload requests or import cURL**) in the top menu
2. In the modal that appears, click **Upload file**.
3. Select a Data Fetcher requests JSON file that you previously downloaded.
4. Choose your desired **Destination table** and **Destination view** using the dropdown menus.
1. For requests that **Run for each record**, you will need to select an input table and input view for that too.
5. Click **Import** to complete the process.
# Run history
Source: https://help.datafetcher.com/organise-requests/run-history
Run history shows when your requests ran and helps you troubleshoot errors in scheduled, trigger, or webhook URL runs. Use it to identify issues and understand your usage patterns.
## How to view run history for a specific request
When you need to investigate the execution history of a particular request:
1. Navigate to the request in the Data Fetcher extension.
2. Click **Run History** in the top-right of the screen.
3. View the list of all runs for this request, with the most recent first.
1. Click on any individual run to reveal detailed information about that execution.
2. Use the date and status filters at the top to narrow down results and locate specific runs.
## How to view run history dashboard for all requests
For a comprehensive overview of all request and sequence executions across your workspace:
1. Sign in to your account at [https://datafetcher.com/sign-in](https://datafetcher.com/sign-in).
2. Make sure the correct workspace is selected in the top-left dropdown.
3. Click **Run History** in the top menu.
4. Review the complete run history for all requests and sequences.
1. Select any individual run to examine its detailed execution information.
2. Apply date and status filters to refine your view and find specific runs.
3. Use the request or sequence dropdown filters to analyze runs and usage for specific workflows
This dashboard also helps you understand your monthly usage patterns and allocation.
## Run storage
The length of time your run history data is stored depends on your subscription plan:
* Free, Starter: 31 days
* Pro, Business, Business +, Custom: 90 days
After these retention periods, run history data is automatically deleted.