Universal API Connector: Custom Data Ingestion

Availability

The Universal API Connector is available on Pro and Enterprise plans.

Step 1: Identify Your API Requirements

1. Gather the API documentation for the system you want to connect
2. Note the following:
   • Base URL of the API
   • Authentication method (API key, OAuth 2.0, Basic Auth, Bearer token)
   • Endpoints you want to pull data from
   • Pagination method (offset, cursor, page number)
   • Rate limits if applicable

Step 2: Contact BEEM to Configure

1. Send your API details to support@beemdata.com or your BEEM account manager
2. The BEEM Advanced Data Services team will:
   • Review the API documentation
   • Configure the connector with proper authentication
   • Set up endpoint mappings and data extraction rules
   • Handle pagination and rate limiting
3. A custom connector is built and deployed to your environment

Step 3: Verify the Connection

1. Go to Connections in the left sidebar
2. Find your new Universal API connection
3. Check that the status shows Active
4. Navigate to the Datalake to confirm source tables are appearing
5. Preview the data to verify it matches expectations

Step 4: Set Up Refresh Schedules

1. The connection syncs on a configurable schedule
2. Work with the BEEM team to set the appropriate refresh frequency
3. Monitor initial syncs in the Monitor section

Common Use Cases

• Internal APIs: Pull data from proprietary systems built in-house
• Niche SaaS tools: Connect to specialized industry software without a standard connector
• Government or open data APIs: Ingest public datasets
• Custom webhooks: Receive event-driven data

Key Details

• Each Universal API Connector is custom-configured by the BEEM team
• Supports REST APIs with JSON responses
• Authentication, pagination, and error handling are managed automatically
• Data is ingested into your Datalake like any other connection