# DataBridge

Source: https://docs.havilo.ai/reference/data-bridge

> **Enterprise Feature**
>
> DataBridge is available on Enterprise plans. Contact your administrator to enable the Webhook
> Integration feature for your workspace.

## Overview

DataBridge allows external systems to automatically sync data into your Havilo workspace. When configured, external systems send webhooks to Havilo, which creates or updates entities based on your field mappings.

\
A single DataBridge can sync to **multiple workspaces** simultaneously (fan-out model). Each
workspace configures its own entity type and field mappings independently, so the same webhook data
can create different entity structures in different workspaces.

**Use cases:** \
\- Sync contacts from your CRM \
\- Import profiles from partner systems
\
\- Migrate data from legacy platforms \
\- Keep entities up-to-date with external sources

***

## How It Works

1. **External system** sends a webhook when data changes
2. **Havilo** validates the webhook signature
3. **Field mappings** transform the data to match your entity fields
4. **Entity is created or updated** in your workspace

```
External System  →  Webhook  →  DataBridge  →  Entity
     (CRM)         (HTTPS)      (Mapping)     (Contact)
```

***

## Configuration

### Requirements

* **Feature**: `WEBHOOK_INTEGRATION` must be enabled for your workspace
* **Permission**: `MANAGE_WORKSPACE` permission required

### Setup Steps

1. Navigate to **Settings** > **Advanced** > **Data Bridges**
2. Select an available DataBridge from the list
3. Configure entity type mappings (which entity type receives each resource type)
4. Configure field mappings (map source fields to your entity fields)
5. Enable the DataBridge

> **Webhook Secrets**
>
> DataBridge webhook secrets are managed by system administrators. Contact your admin if you need
> the webhook secret for integration setup.

***

## Sending Webhooks

External systems send webhooks to sync data with your workspace.

### Endpoint

```
POST https://havilo.ai/api/webhooks/sync/{bridgeId}
```

### Required Headers

| Header                | Description                                                               |
| --------------------- | ------------------------------------------------------------------------- |
| `Content-Type`        | Must be `application/json`                                                |
| `X-Webhook-Signature` | HMAC-SHA256 signature (see [Signature Validation](#signature-validation)) |
| `X-Webhook-Timestamp` | ISO 8601 timestamp (for replay protection)                                |

### Payload Structure

**Push Mode**

In **Push mode**, include the full data in the webhook payload:

```json
{
  "resource": {
    "type": "users",
    "id": "ext-12345"
  },
  "operation": "create",
  "timestamp": "2025-01-15T10:30:00Z",
  "data": {
    "type": "users",
    "id": "ext-12345",
    "attributes": {
      "name": "Jane Smith",
      "email": "jane@example.com",
      "company": "Acme Inc"
    }
  }
}
```

**Pull Mode**

In **Pull mode**, include a URL where Havilo can fetch the full data:

```json
{
  "resource": {
    "type": "users",
    "id": "ext-12345"
  },
  "operation": "update",
  "timestamp": "2025-01-15T10:30:00Z",
  "fetch_url": "https://api.your-system.com/users/ext-12345"
}
```

Havilo will call the `fetch_url` with stored API credentials to retrieve the full data.

### Payload Fields

| Field           | Required    | Description                                          |
| --------------- | ----------- | ---------------------------------------------------- |
| `resource.type` | Yes         | Resource type (must be enabled in DataBridge config) |
| `resource.id`   | Yes         | Unique ID from your system                           |
| `operation`     | Yes         | One of: `create`, `update`, `delete`                 |
| `timestamp`     | Recommended | ISO 8601 timestamp of the change                     |
| `data`          | Push mode   | Full resource data with `attributes`                 |
| `fetch_url`     | Pull mode   | URL to fetch full resource data                      |
| `version`       | Optional    | Monotonic version number for ordering                |

***

## Operations

### Create & Update

Create and update operations work identically - Havilo uses the resource ID to determine whether to create a new entity or update an existing one.

```json
{
  "resource": { "type": "users", "id": "ext-12345" },
  "operation": "update",
  "data": {
    "type": "users",
    "id": "ext-12345",
    "attributes": {
      "name": "Jane Smith",
      "email": "jane.smith@newdomain.com"
    }
  }
}
```

### Delete

Delete operations remove the resource:

```json
{
  "resource": { "type": "users", "id": "ext-12345" },
  "operation": "delete",
  "timestamp": "2025-01-15T10:30:00Z"
}
```

***

## Relationship Resources

DataBridge supports syncing **relationship resources** that create connections between entities rather than creating new entities. This is useful for representing founder-startup relationships, employment, investments, or other associations.

### How Relationships Work

1. **Entity resources** (e.g., `users`, `companies`) create Entity records
2. **Relationship resources** (e.g., `relationship`) create EntityRelationship records linking two existing entities

```
users webhook        →  Creates Entity (Contact: John)
companies webhook    →  Creates Entity (Company: Acme)
relationship webhook →  Creates Relationship (John → Acme, role: "Founder")
```

### Configuration

Relationship resources require a different configuration than entity resources:

| Config Field         | Description                                  |
| -------------------- | -------------------------------------------- |
| `relationshipTypeId` | UUID of the EntityRelationshipType to use    |
| `fromField`          | Source field containing the "from" entity ID |
| `fromResourceType`   | Resource type of the "from" entity           |
| `toField`            | Source field containing the "to" entity ID   |
| `toResourceType`     | Resource type of the "to" entity             |

### Relationship Payload Example

```json
{
  "resource": {
    "type": "relationship",
    "id": "rel-123"
  },
  "operation": "create",
  "timestamp": "2025-01-15T10:30:00Z",
  "data": {
    "type": "relationship",
    "id": "rel-123",
    "attributes": {
      "user_id": "user-456",
      "company_id": "company-789",
      "role": "Founder"
    }
  }
}
```

### Relationship Modeling Approaches

There are two ways to model relationships, depending on your needs:

#### Approach A: Separate Relationship Types

Use when you need **distinct relationship types** with different labels (e.g., "Works At" vs "Worked At"):

1. Create multiple resource types for relationships (e.g., `works-at`, `worked-at`)
2. Each maps to a different relationship type in your workspace
3. Send webhooks with the appropriate resource type

```json
// "Works At" relationship
{
  "resource": { "type": "works-at", "id": "rel-1" },
  "data": { "attributes": { "user_id": "user-1", "company_id": "company-1" } }
}

// "Worked At" relationship
{
  "resource": { "type": "worked-at", "id": "rel-2" },
  "data": { "attributes": { "user_id": "user-1", "company_id": "company-2" } }
}
```

**Result:** Each appears under its own section in the UI.

#### Approach B: Single Type with Role Metadata

Use when you want **one relationship type** with role-based grouping:

1. Use a single relationship resource type (e.g., `relationship`)
2. Include a `role` field to differentiate relationships
3. The UI groups relationships by role automatically

```json
// All use same resource type, different roles
{ "resource": { "type": "relationship", "id": "rel-1" },
  "data": { "attributes": { "user_id": "user-1", "company_id": "company-1", "role": "CEO" } } }
{ "resource": { "type": "relationship", "id": "rel-2" },
  "data": { "attributes": { "user_id": "user-2", "company_id": "company-1", "role": "CTO" } } }
{ "resource": { "type": "relationship", "id": "rel-3" },
  "data": { "attributes": { "user_id": "user-3", "company_id": "company-1", "role": "Advisor" } } }
```

**Result:** All appear under one section, grouped by role.

### Relationship Metadata

Any field in `attributes` that is NOT a foreign key field (`fromField`/`toField`) becomes metadata on the relationship.

Common metadata fields:

* `role` - Role in the relationship (**special UI grouping**)
* `title` - Job title
* `start_date` - When the relationship started

Only the `role` field has special UI treatment—it's used to group relationships in the sidebar.

### Role Display in UI

When you sync relationships with a `role` field, Havilo groups them by role in the entity sidebar:

```
FOUNDED BY CONTACTS
  CEO
    - John Doe
  CTO
    - Jane Smith
  Advisor
    - Bob Wilson
    - Alice Brown
```

This provides a clear visual hierarchy when entities have multiple relationships with different roles.

### Important: Entity Order Matters

Relationship webhooks require both referenced entities to exist first. If the "from" or "to" entity doesn't exist, the relationship webhook is skipped (returns `processed_workspaces: 0`).

***

## Field Mappings

Field mappings define how source data maps to your entity fields.

### Example

If your external system sends:

```json
{
  "attributes": {
    "full_name": "Jane Smith",
    "email_address": "jane@example.com",
    "linkedin": "https://linkedin.com/in/janesmith"
  }
}
```

And you configure mappings:

| Source Field    | Entity Field (Havilo) |
| --------------- | --------------------- |
| `full_name`     | Name                  |
| `email_address` | Email                 |
| `linkedin`      | LinkedIn URL          |

The resulting entity will have:

* **Name**: Jane Smith
* **Email**: <jane@example.com>
* **LinkedIn URL**: <https://linkedin.com/in/janesmith>

### Unmapped Fields

Fields in your webhook that aren't mapped are ignored. This allows you to send complete data and only import what you need.

***

## Signature Validation

All webhooks must include a valid HMAC-SHA256 signature for security.

### Computing the Signature

**TypeScript**

```typescript
import crypto from 'crypto'

function signWebhook(secret: string, payload: string): string {
  // First, hash the secret
  const secretHash = crypto
    .createHash('sha256')
    .update(secret)
    .digest('hex')

  // Then compute HMAC using the hashed secret
  const signature = crypto
    .createHmac('sha256', secretHash)
    .update(payload)
    .digest('hex')

  return `sha256=${signature}`
}

// Usage
const webhookSecret = 'your-webhook-secret'
const payload = JSON.stringify({
  resource: { type: 'users', id: '123' },
  operation: 'create',
  data: { type: 'users', id: '123', attributes: { name: 'Jane' } }
})

const signature = signWebhook(webhookSecret, payload)

// Send webhook with signature header
await fetch('https://havilo.ai/api/webhooks/sync/bridge-id', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-Webhook-Signature': signature,
    'X-Webhook-Timestamp': new Date().toISOString(),
  },
  body: payload,
})
```

**Python**

```python
import hashlib
import hmac
import json
import requests
from datetime import datetime

def sign_webhook(secret: str, payload: str) -> str:
    # First, hash the secret
    secret_hash = hashlib.sha256(secret.encode()).hexdigest()

    # Then compute HMAC using the hashed secret
    signature = hmac.new(
        secret_hash.encode(),
        payload.encode(),
        hashlib.sha256
    ).hexdigest()

    return f"sha256={signature}"

# Usage
webhook_secret = "your-webhook-secret"
payload = json.dumps({
    "resource": {"type": "users", "id": "123"},
    "operation": "create",
    "data": {"type": "users", "id": "123", "attributes": {"name": "Jane"}}
})

signature = sign_webhook(webhook_secret, payload)

# Send webhook with signature header
response = requests.post(
    "https://havilo.ai/api/webhooks/sync/bridge-id",
    headers={
        "Content-Type": "application/json",
        "X-Webhook-Signature": signature,
        "X-Webhook-Timestamp": datetime.utcnow().isoformat() + "Z",
    },
    data=payload,
)
```

**Shell**

```bash
#!/bin/bash

WEBHOOK_SECRET="your-webhook-secret"
BRIDGE_ID="your-bridge-id"
PAYLOAD='{"resource":{"type":"users","id":"123"},"operation":"create","data":{"type":"users","id":"123","attributes":{"name":"Jane"}}}'

# Hash the secret first
SECRET_HASH=$(echo -n "$WEBHOOK_SECRET" | sha256sum | cut -d' ' -f1)

# Compute HMAC signature using the hashed secret
SIGNATURE=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET_HASH" | cut -d' ' -f2)

# Send webhook
curl -X POST "https://havilo.ai/api/webhooks/sync/$BRIDGE_ID" \
  -H "Content-Type: application/json" \
  -H "X-Webhook-Signature: sha256=$SIGNATURE" \
  -H "X-Webhook-Timestamp: $(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  -d "$PAYLOAD"
```

> **Important: Two-Step Signature**
>
> Havilo uses a **two-step signature process**:
>
> 1. Hash the webhook secret with SHA-256
> 2. Use the hashed secret as the HMAC key
>
> This differs from simple HMAC implementations. Make sure to hash the secret first.

### Timestamp Validation

Webhooks include timestamp validation to prevent replay attacks. Webhooks older than **5 minutes** are rejected.

Always include the `X-Webhook-Timestamp` header with the current time.

***

## Response Codes

### Success

```json
{
  "success": true,
  "processed_workspaces": 1
}
```

### Errors

| Status | Error                   | Description                                             |
| ------ | ----------------------- | ------------------------------------------------------- |
| 400    | `invalid_payload`       | Request body is invalid JSON or missing required fields |
| 401    | `invalid_signature`     | HMAC signature validation failed                        |
| 401    | `timestamp_expired`     | Webhook timestamp is older than 5 minutes               |
| 404    | `bridge_not_found`      | DataBridge ID doesn't exist or is inactive              |
| 422    | `unknown_resource_type` | Resource type not enabled for this DataBridge           |

**Example error response:**

```json
{
  "success": false,
  "error": "invalid_signature",
  "message": "Webhook signature validation failed"
}
```

***

## Best Practices

### 1. Include Timestamps

Always include the `timestamp` field in your payload and the `X-Webhook-Timestamp` header:

```json
{
  "resource": { "type": "users", "id": "123" },
  "operation": "update",
  "timestamp": "2025-01-15T10:30:00Z",
  ...
}
```

### 2. Use Version Numbers

For high-frequency updates, include a monotonic version number to handle out-of-order delivery:

```json
{
  "resource": { "type": "users", "id": "123" },
  "operation": "update",
  "version": 42,
  ...
}
```

Havilo tracks versions **per-entity**, so each resource has its own version history. Updates with versions lower than the last processed version for that specific entity are skipped. This means webhooks for different entities don't interfere with each other's version ordering.

### 3. Retry Failed Webhooks

Implement retry logic with exponential backoff for failed webhook deliveries:

```typescript
const delays = [1000, 5000, 30000, 120000] // 1s, 5s, 30s, 2m

for (const delay of delays) {
  const response = await sendWebhook(payload)
  if (response.ok) break
  await sleep(delay)
}
```

### 4. Log Webhook Responses

Keep logs of webhook responses for debugging:

```typescript
const response = await sendWebhook(payload)
console.log({
  bridgeId,
  resourceType: payload.resource.type,
  resourceId: payload.resource.id,
  status: response.status,
  success: response.ok,
})
```

### 5. Test in Development

Use a test DataBridge in development before connecting to production.

***

## Troubleshooting

### Webhook Returns 401

**Possible causes:**

* Incorrect webhook secret
* Not hashing the secret before computing HMAC (see [Signature Validation](#signature-validation))
* Timestamp too old (>5 minutes)

**Solution:** Verify your signature computation matches the two-step process and include a current timestamp.

### Webhook Returns 422

**Possible causes:**

* Resource type not enabled in DataBridge configuration
* Resource type not configured for your workspace

**Solution:** Contact your administrator to enable the resource type in DataBridge settings.

### Entity Not Created

**Possible causes:**

* No field mappings configured for the resource type
* Entity type not selected for the resource type
* DataBridge is disabled

**Solution:** Check your DataBridge configuration in Settings > Advanced > Data Bridges.

### Duplicate Entities

**Possible causes:**

* Different resource IDs for the same record
* Multiple DataBridges syncing the same data

**Solution:** Ensure your external system uses consistent, unique IDs for each resource.

***

## Need Help?

If you encounter issues:

1. Verify your webhook signature is computed correctly
2. Check that timestamps are current (within 5 minutes)
3. Confirm field mappings are configured in your workspace
4. Review DataBridge status in Settings > Advanced > Data Bridges
5. Contact your system administrator for webhook secrets or DataBridge access