This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Coordinator API

The main administrative API for the management of users, entity types, connectors and policies

1: Users
2: Entity Types
3: Data Connectors
4: Data Sharing Policy
5: User Data Access

The Coordinator API is the main administrative API for the BitBroker system. It is the API you will need to use to create and manipulate all the main system elements required to operate a BitBroker instance.

Before you use this API, you should become familiar with the general, system-wide API conventions - which are used across all three BitBroker API sets. This covers topics such as API architecture, authorization, error reporting and handling, etc.

1 - Users

APIs for creating and manipulating users

Users are a main component of the BitBroker system. You will find more details about users, within the key concepts section of this documentation.

Every fresh installation of BitBroker comes with one preinstalled user (uid: 1). This user is automatically created when the system is bought-up for the first time.

In our sample calls, we use the standard server name and port format outlined for local installations and demos. If you have an alternative API host base URL, then enter it in the box below to update all the sample calls on this page :

Your API Host Base URL

All API calls in BitBroker require authorization. The sample calls below contain a placeholder string where you should insert your coordinator API authorization token. If you already have a token, enter it in the box below to update all the sample calls on this page:

Your Coordinator API Authorization Token

Creating a New User

New users can be created by issuing an HTTP/POST to the /user end-point.

curl http://bbk-coordinator:8001/v1/user \
     --request POST \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Alice", \
         "email": "alice@domain.com" \
     }
EOF

This will result in a response as follows:

HTTP/1.1 201 Created
Location: http://bbk-coordinator:8001/v1/user/2

The ID for the new user will be returned within the Location attribute of the response header.

The following validation rules will be applied to the body of a new user request.

Attribute	Necessity	Validation Rules
`name`	required	String between 1 and 64 characters long
`email`	required	String between 1 and 256 characters long Must conform to email address format Must be unique across all users in the system

Email addresses need to be unique across an operating BitBroker instance. If you attempt to create a user with a duplicate email address, it will result in an HTTP/1.1 409 Conflict response.

New users will be present in the system right away. They can immediately take part as data consumers by being assigned policy-based, consumer authorization tokens. However, a further step is required to promote them to have coordinator status.

Updating a User

Existing users can have their profile updated by issuing an HTTP/PUT to the /user/:uid end-point.

In order to update a user, you must know their user ID (uid).

curl http://bbk-coordinator:8001/v1/user/2 \
     --request PUT \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Bob" \
     }
EOF

This will result in a response as follows:

HTTP/1.1 204 No Content

You cannot update the user’s email address, which was specified when they were created. Email addresses are system-wide unique identifiers. If you need to update an email address, your only option is to create a new user with that address.

The validation rules for updated user information, are the same as that for creating new users.

List of Existing Users

You can obtain a list of all the existing users by issuing an HTTP/GET to the /user end-point.

curl http://bbk-coordinator:8001/v1/user \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

[
    {
        "id": 1,
        "url": "http://bbk-coordinator:8001/v1/user/1",
        "name": "Admin",
        "email": "noreply@bit-broker.io",
        "coordinator": true,
        "accesses": []
    },
    {
        "id": 2,
        "url": "http://bbk-coordinator:8001/v1/user/2",
        "name": "Bob",
        "email": "alice@domain.com",
        "coordinator": false,
        "accesses": []
    }
    // ... other users here
]

Each user on the system will be returned within this array. Later sections of this document will explain what the coordinator and accesses attributes refer to. Note: There is currently no paging on this API.

Details of an Existing User

You can obtain the details of an existing user by issuing an HTTP/GET to the /user/:uid end-point.

In order to obtain details of a user, you must know their user ID (uid).

curl http://bbk-coordinator:8001/v1/user/2 \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON object as follows:

{
    "id": 2,
    "url": "http://bbk-coordinator:8001/v1/user/2",
    "name": "Bob",
    "email": "alice@domain.com",
    "coordinator": false,
    "accesses": [],
    "addendum": {}
}

Later sections of this document will explain what the coordinator, accesses and addendum attributes refer to.

Finding a User via Email Address

You can find an existing user via their email address by issuing an HTTP/GET to the /user/email/:email end-point.

In order to obtain details of a user, you must know their current email address.

curl http://bbk-coordinator:8001/v1/user/email/alice%40domain.com \
     --header "x-bbk-auth-token: your-token-goes-here"

For best practice, the email address within this RESTful URL should be url encoded.

This will return a JSON object as follows:

{
    "id": 2,
    "url": "http://bbk-coordinator:8001/v1/user/2",
    "name": "Bob",
    "email": "alice@domain.com",
    "coordinator": false,
    "accesses": [],
    "addendum": {}
}

Promoting a User to Coordinator

Existing users can be promoted to assign them coordinator status by issuing an HTTP/POST to the /user/:uid/coordinator end-point.

In order to promote a user, you must know their user ID (uid).

curl http://bbk-coordinator:8001/v1/user/2/coordinator \
     --request POST \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 200 OK

The body of this response will contain the coordinator authorization token, which the newly promoted user should utilize to authorize their own calls to the Coordinator API. For example:

4735d360-dc03-499c-91ce-68dfc1554691.5d5c9eab-1d9c-4c88-9478-9f4ab35568a7.423c9101-315a-4929-a64c-9b905837799c

It is expected that the promoting user will securely distribute this coordinator authorization token to the promoted user.

Promoted users will gain coordinator privileges right away. When getting details for such users, their coordinator status will be reflected in the coordinator attribute:

{
    "id": 2,
    "url": "http://bbk-coordinator:8001/v1/user/2",
    "name": "Bob",
    "email": "alice@domain.com",
    "coordinator": true,  // this is the new coordinator status
    "accesses": [],
    "addendum": {}
}

If you attempt to promote a user who is already a coordinator, it will result in an HTTP/1.1 409 Conflict response. This error is benign and will not impact the user’s status.

Demoting a User from Coordinator

Existing users can be demoted from coordinator status by issuing an HTTP/DELETE to the /user/:uid/coordinator end-point.

In order to demote a user, you must know their user ID (uid).

curl http://bbk-coordinator:8001/v1/user/2/coordinator \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

Demoted users will lose coordinator privileges right away. The coordinator authorization token they were holding will no longer be valid. When getting details for such users, their coordinator status will be reflected in the coordinator attribute:

{
    "id": 2,
    "url": "http://bbk-coordinator:8001/v1/user/2",
    "name": "Bob",
    "email": "alice@domain.com",
    "coordinator": false,  // this is the new coordinator status
    "accesses": [],
    "addendum": {}
}

If you attempt to demote a user who is not a coordinator, it will result in an HTTP/1.1 409 Conflict response. This error is benign and will not impact the user’s status.

Demoting a user has no impact on any policy authorization tokens which they may be holding for the Consumer API.

You cannot demote yourself from being a coordinator. If you attempt this, the system will return an HTTP/1.1 405 Method Not Allowed response. This is done in order to prevent a situation where there are no coordinators present within an operating instance. If you wish to demote yourself, you will need to ask another coordinator to perform this action on your behalf.

Deleting a User

Existing users can be deleted from the system by issuing an HTTP/DELETE to the /user/:uid end-point.

In order to delete a user, you must know their user ID (uid).

curl http://bbk-coordinator:8001/v1/user/2 \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

Any policy authorization tokens which were assigned to a deleted user will be rescinded as part of the deletion process. Consumer API requests made with such tokens will fail (after a short propagation delay).

You cannot delete yourself from the system. If you attempt this, the system will return an HTTP/1.1 405 Method Not Allowed response. This is done in order to prevent a situation where there are no users present within an operating instance. If you wish to delete yourself, you will need to ask another coordinator to perform this action on your behalf.

User Addendum Information

Observant readers will have noticed an addendum section at the bottom of the user details object:

{
    "id": 2,
    "url": "http://bbk-coordinator:8001/v1/user/2",
    "name": "Bob",
    "email": "alice@domain.com",
    "coordinator": false,  
    "accesses": [],
    "addendum": {}  // this is the addendum section
}

This section is reserved for use by the up-coming BitBroker Portal. We request that you do not use this section at this time.

2 - Entity Types

APIs for creating and manipulating entity types

Entity Types are a main component of the BitBroker system. You will find more details about entity types, within the key concepts section of this documentation.

Creating a New Entity Type

New entity types can be created by issuing an HTTP/POST to the /entity/:eid end-point.

In order to create an entity type, you must select a unique entity type ID (eid) for it.

curl http://bbk-coordinator:8001/v1/entity/country \
     --request POST \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Countries",  \
         "description": "All the countries in the world as defined by the UN", \
         "schema": {}, \
         "timeseries": { \
             "population": { \
                 "period": "P1Y", \
                 "value": "people", \
                 "unit": "x1000" \
             } \
         } \
     }
EOF

This will result in a response as follows:

HTTP/1.1 201 Created
Location: http://bbk-coordinator:8001/v1/entity/country

The following validation rules will be applied to the body of a new entity type request.

Attribute	Necessity	Validation Rules
`eid`	required	String between 3 and 32 characters long Consisting of lowercase letters, numbers and dashes only Starting with a lowercase letter Conforming to the regex expression `^[a-z][a-z0-9-]+$`
`name`	required	String between 1 and 64 characters long
`description`	required	String between 1 and 2048 characters long
`schema`	optional	A valid JSON Schema document
`timeseries`	optional	A list of timeseries which are present on this entity type
`timeseries.id`	required	String between 3 and 32 characters long Consisting of lowercase letters, numbers and dashes only Starting with a lowercase letter Conforming to the regex expression `^[a-z][a-z0-9-]+$`
`timeseries.period`	required	A string conforming to the ISO8601 duration format
`timeseries.value`	required	String between 1 and 256 characters long
`timeseries.unit`	required	String between 1 and 256 characters long

Details about the meaning of these attributes can be found within the key concepts section of this documentation. The schema attribute is a powerful concept, which is explained in more detail there.

Entity type IDs are required to be unique across an operating BitBroker instance.

Timeseries IDs are required to be unique within their housing entity type.

By convention, entity types are expressed in the singular, non-plural form. So, for example, product is preferred over products.

You should choose your entity type IDs (eid) with care, as these cannot be changed once created.

Updating an Entity Type

Existing entity types can have their profile updated by issuing an HTTP/PUT to the /entity/:eid end-point.

In order to update an entity type, you must know its entity type ID (eid).

curl http://bbk-coordinator:8001/v1/entity/country \
     --request PUT \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Countries",  \
         "description": "This is a new description for countries", \
         "schema": {}, \
         "timeseries": { \
             "population": { \
                 "period": "P1Y", \
                 "value": "people", \
                 "unit": "x1000" \
             } \
         } \
     }
EOF

This will result in a response as follows:

HTTP/1.1 204 No Content

Great care should be taken when updating the JSON schema associated with an entity type. Consider whether this breaks an implicit contract that you may have made with existing data connectors for this entity type. It may make more sense to create a new entity type, if the amended schema definition is incompatible with the existing one.

The validation rules for updated entity type information, are the same as that for creating new entity types.

List of Existing Entity Types

You can obtain a list of all the existing entity types by issuing an HTTP/GET to the /entity end-point.

curl http://bbk-coordinator:8001/v1/entity \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

[
    {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country",
        "name": "Countries",
        "description": "This is a new description for countries"
    }
    // ... other entity types here
]

Each entity type on the system will be returned within this array. Note: There is currently no paging on this API.

Details of an Existing Entity Type

You can obtain the details of an existing entity type by issuing an HTTP/GET to the /entity/:eid end-point.

In order to obtain details of an entity type, you must know its entity type ID (eid).

curl http://bbk-coordinator:8001/v1/entity/country \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON object as follows:

{
    "id": "country",
    "url": "http://bbk-coordinator:8001/v1/entity/country",
    "name": "Countries",
    "description": "This is a new description for countries",
    "schema": {},
    "timeseries": {
        "population": {
            "unit": "x1000",
            "value": "people",
            "period": "P1Y"
        }
    }
}

Deleting an Entity Type

Existing entity types can be deleted from the system by issuing an HTTP/DELETE to the /entity/:eid end-point.

In order to delete an entity type, you must know its entity type ID (eid).

curl http://bbk-coordinator:8001/v1/entity/country \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

Deleting an entity type will also delete all entity instances associated with that type from the Catalog.

Deleting an entity type will also delete all data connectors associated with that entity type. No further data contribution will be accepted for it.

Any policy authorization tokens which were issued where this entity type formed part of the data segment, will no longer return entity instances of this type. In some circumstances, this could render policy authorization tokens unfit for purpose.

3 - Data Connectors

APIs for creating and manipulating data connectors

Connectors are a main component of the BitBroker system. You will find more details about connectors, within the key concepts section of this documentation.

Connectors are always created within the context of housing entity types, which can be created and manipulated using other part of this API, described earlier in this documentation.

In order to use the sample calls in this section, first create the housing entity as outlined in the previous section.

Creating a New Connector

New connectors can be created by issuing an HTTP/POST to the /entity/:eid/connector/:cid end-point.

Connectors are always created within the context of a housing entity type and, hence, you must know its ID (eid). In order to create a connector, you must select a unique connector ID (cid) for it.

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --request POST \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Wikipedia", \
         "description": "Country information from Wikipedia", \
         "webhook": "http://my-domain.com/connectors/1", \
         "cache": 0 \
     }
EOF

This will result in a response as follows:

HTTP/1.1 201 Created
Location: http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia

The body of this response will contain the connector ID and connector authorization token, which the new data connector should utilize to make its data contributions. For example:

{
    "id":"9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "token":"ef6ba361-ef55-4a7a-ae48-b4ecef9dabb5.5ab6b7a2-6a50-4d0a-a6b4-f43dd6fe12d9.7777df3f-e26b-4f4e-8c80-628f915871b4"
}

It is expected that the coordinator user will securely distribute the connector ID and authorization token to the operator of the new data connector.

Connector authorization tokens are not stored within the system. If you lose this connector authorization token, you will be forced to delete and recreate the connector to obtain a new one. This can have major implications for an operating instance.

The following validation rules will be applied to the body of a new connector request.

Attribute	Necessity	Validation Rules
`cid`	required	String between 3 and 32 characters long Consisting of lowercase letters, numbers and dashes only Starting with a lowercase letter Conforming to the regex expression `^[a-z][a-z0-9-]+$`
`name`	required	String between 1 and 64 characters long
`description`	required	String between 1 and 2048 characters long
`webhook`	optional	String between 1 and 1024 characters long Must conform to URI format
`cache`	optional	Integer between 0 and 31536000

Details about the meaning of these attributes can be found within the key concepts section of this documentation.

Connector IDs are required to be unique across an operating BitBroker instance.

Connector who present no webhook are required to submit complete entity instance records to the catalog. Before making the decision as to whether to host a webhook or not, careful note should be taken on the distinction between data vs metadata within the catalog.

You should choose your connector ID (cid) with care, as these cannot be changed once created.

Updating a connector

Existing connectors can have their profile updated by issuing an HTTP/PUT to the /entity/:eid/connector/:cid end-point.

In order to update a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --request PUT \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "Wikipedia", \
         "description": "A new description for the Wikipedia connector", \
         "webhook": "http://my-domain.com/connectors/1", \
         "cache": 0 \
     }
EOF

This will result in a response as follows:

HTTP/1.1 204 No Content

The validation rules for updated connector information, are the same as that for creating new connectors.

List of Existing Connectors

You can obtain a list of all the existing connectors housed within a parent entity type by issuing an HTTP/GET to the /entity/:eid/connector end-point.

curl http://bbk-coordinator:8001/v1/entity/country/connector \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

[
    {
        "id": "wikipedia",
        "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
        "name": "Wikipedia",
        "description": "A new description for the Wikipedia connector"
    }
    // ... other connectors here
]

Each connector, housed within a parent entity type, will be returned within this array. Note: There is currently no paging on this API.

Details of an Existing Connector

You can obtain the details of an existing connector by issuing an HTTP/GET to the /entity/:eid/connector/:cid end-point.

In order to obtain details of a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON object as follows:

{
    "id": "wikipedia",
    "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
    "name": "Wikipedia",
    "description": "A new description for the Wikipedia connector",
    "entity": {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country"
    },
    "contribution_id": "9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "webhook": "http://my-domain.com/connectors/1",
    "cache": 0,
    "is_live": false,
    "in_session": false
}

Connector authorization tokens are not stored in an accessible way within the system and hence not present in this returned document.

Other sections of this document will explain what the is_live and in_session attributes refer to.

Promoting a Connector to Live

Connectors can be promoted to live status by issuing an HTTP/POST to the /entity/:eid/connector/:cid/live end-point.

In order to promote a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia/live \
     --request POST \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

When getting details for promoted connectors, their is_live status will be reflected in the corresponding attribute:

{
    "id": "wikipedia",
    "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
    "name": "Wikipedia",
    "description": "A new description for the Wikipedia connector",
    "entity": {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country"
    },
    "contribution_id": "9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "webhook": "http://my-domain.com/connectors/1",
    "cache": 0,
    "is_live": true,
    "in_session": false
}

If you attempt to promote a connector which is already live, it will still result in an HTTP/1.1 204 No Content response. Such requests are benign and will not impact the connector’s status.

Promoting connectors will cause all the staged records which they have contributed to the Catalog to become visible for calls to the Consumer API.

Demoting a Connector from Live

Existing connectors can be demoted from live status by issuing an HTTP/DELETE to the /entity/:eid/connector/:cid/live end-point.

In order to demote a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia/live \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

When getting details for such users, their is_live status will be reflected in the corresponding attribute:

{
    "id": "wikipedia",
    "url": "http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia",
    "name": "Wikipedia",
    "description": "A new description for the Wikipedia connector",
    "entity": {
        "id": "country",
        "url": "http://bbk-coordinator:8001/v1/entity/country"
    },
    "contribution_id": "9afcf3235500836c6fcd9e82110dbc05ffbb734b",
    "webhook": "http://my-domain.com/connectors/1",
    "cache": 0,
    "is_live": false,
    "in_session": false
}

If you attempt to demote a connector which is not live, this will still result in an HTTP/1.1 204 No Content response. Such requests are benign and will not impact the connector’s status.

Demoting connectors will remove all the records which they have contributed to the Catalog from calls to the Consumer API. Users holding on to full qualified URLs for such records, will instead receive HTTP/1.1 404 Not Found responses.

Deleting a connector

Existing connectors can be deleted from the system by issuing an HTTP/DELETE to the /entity/:eid/connector/:cid end-point.

In order to delete a connector, you must know the ID of its housing entity type (eid) and it’s connector ID (cid).

curl http://bbk-coordinator:8001/v1/entity/country/connector/wikipedia \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

Deleting a connector will also remove all the records which it has contributed to the Catalog.

Deleting a connector will also remove its ability to contribute data. No further contribution will be accepted and all subsequent calls to the Contributor API will fail.

Any policy authorization tokens which were issued where this connector’s records formed part of the data segment, will no longer return entity instances contributed by it. In some circumstances, this could render policy authorization tokens unfit for purpose.

4 - Data Sharing Policy

APIs for creating and manipulating policies

Data Sharing Policies are a main component of the BitBroker system. You will find more details about policies, within the key concepts section of this documentation.

Creating a New Policy

New policies can be created by issuing an HTTP/POST to the /policy/:pid end-point.

In order to create a policy, you must select a unique policy ID (pid) for it.

curl http://bbk-coordinator:8001/v1/policy/over-a-billion \
     --request POST \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "The Most Populated Countries", \
         "description": "Countries with a population of over a billion", \
         "policy": { \
             "access_control": { \
                 "enabled": true, \
                 "quota": { \
                     "max_number": 86400, \
                     "interval_type": "day" \
                 }, \
                 "rate": 250 \
             }, \
             "data_segment": { \
                 "segment_query": { \
                     "type": "country", \
                     "entity.population": { "$gt": 1000000000 } \
                 }, \
                 "field_masks": [] \
             }, \
             "legal_context": [ { \
                     "type": "attribution", \
                     "text": "Data is supplied by Wikipedia", \
                     "link": "https://en.wikipedia.org/" \
                 } \
             ] \
         } \
     }
EOF

This will result in a response as follows:

HTTP/1.1 201 Created
Location: http://bbk-coordinator:8001/v1/policy/country

The following validation rules will be applied to the body of a new policy request.

Attribute	Necessity	Validation Rules
`pid`	required	String between 3 and 32 characters long Consisting of lowercase letters, numbers and dashes only Starting with a lowercase letter Conforming to the regex expression `^[a-z][a-z0-9-]+$`
`name`	required	String between 1 and 64 characters long
`description`	required	String between 1 and 2048 characters long
`access_control`	required	An object describing how data can be accessed
`access_control.enabled`	required	A boolean string either `true` or `false`
`access_control.quota`	optional	An object describing allowable data quotas
`access_control.quota.max_number`	optional	An integer greater than 0
`access_control.quota.interval_type`	optional	One of an enumeration of either `day` or `month`
`access_control.rate`	optional	An integer greater than 0
`data_segment`	required	An object outlining the shared data subset
`data_segment.segment_query`	required	A valid catalog query, from the Consumer API
`data_segment.field_masks`	optional	An array of strings An empty array
`legal_context`	optional	An array of 0 to 100 of object outlining the legal basis of data sharing
`legal_context.type`	required	One of an enumeration `attribution`, `contact` `license`, `note`, `source` or `terms`
`legal_context.text`	required	String between 1 and 256 characters long
`legal_context.link`	required	String between 1 and 1024 characters long Must conform to URI format

Details about the meaning of these attributes can be found within the key concepts section of this documentation.

Policy IDs are required to be unique across an operating BitBroker instance.

You should choose your policy IDs (pid) with care, as these cannot be changed once created.

Data sharing policies are complex objects. You should refer to the detailed explanation of these of the key concepts area, if you are unclear about any aspects of their construction.

Updating a Policy

Existing policies can have their profile updated by issuing an HTTP/PUT to the /policy/:pid end-point.

In order to update a policy, you must know its policy ID (pid).

curl http://bbk-coordinator:8001/v1/policy/over-a-billion \
     --request PUT \
     --include \
     --header "Content-Type: application/json" \
     --header "x-bbk-auth-token: your-token-goes-here" \
     --data-binary @- << EOF
     { \
         "name": "This is a new name", \
         "description": "This is a new description", \
         "policy": { \
             "access_control": { \
                 "enabled": true, \
                 "quota": { \
                     "max_number": 86400, \
                     "interval_type": "day" \
                 }, \
                 "rate": 250 \
             }, \
             "data_segment": { \
                 "segment_query": { \
                     "type": "country", \
                     "entity.population": { "$gt": 1000000000 } \
                 }, \
                 "field_masks": [] \
             }, \
             "legal_context": [ { \
                     "type": "attribution", \
                     "text": "Data is supplied by Wikipedia", \
                     "link": "https://en.wikipedia.org/" \
                 } \
             ] \
         } \
     }
EOF

This will result in a response as follows:

HTTP/1.1 204 No Content

Great care should be taken when updating policies, since authorization tokens may have been issued in the previous context. The ability to change the definition of live policies is a powerful feature, but comes with responsibility.

The new policy definition will come into play right away (after a small propagation delay). Authorization tokens issued prior to the change, will automatically become subject to the new definition.

The validation rules for updated policy information, are the same as that for creating new policies.

List of Existing Policies

You can obtain a list of all the existing policies by issuing an HTTP/GET to the /policy end-point.

curl http://bbk-coordinator:8001/v1/policy \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

[
    {
        "id": "over-a-billion",
        "url": "http://bbk-coordinator:8001/v1/policy/over-a-billion",
        "name": "This is a new name",
        "description": "This is a new description"
    }
    // ... other policies here
]

Each policy on the system will be returned within this array. Note: There is currently no paging on this API.

Details of an Existing Policy

You can obtain the details of an existing policy by issuing an HTTP/GET to the /policy/:pid end-point.

In order to obtain details of a policy, you must know its policy ID (pid).

curl http://bbk-coordinator:8001/v1/policy/over-a-billion \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON object as follows:

{
    "id": "over-a-billion",
    "url": "http://bbk-coordinator:8001/v1/policy/over-a-billion",
    "name": "This is a new name",
    "description": "This is a new description",
    "policy": {
        "data_segment": {
            "field_masks": [],
            "segment_query": {
                "type": "country",
                "entity.population": {
                    "$gt": 1000000000
                }
            }
        },
        "legal_context": [
            {
                "link": "https://en.wikipedia.org/",
                "text": "Data is supplied by Wikipedia",
                "type": "attribution"
            }
        ],
        "access_control": {
            "rate": 250,
            "quota": {
                "max_number": 86400,
                "interval_type": "day"
            },
            "enabled": true
        }
    }
}

Deleting a Policy

Existing policies can be deleted from the system by issuing an HTTP/DELETE to the /policy/:pid end-point.

In order to delete a policy, you must know its policy ID (pid).

curl http://bbk-coordinator:8001/v1/policy/over-a-billion \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

Any authorization tokens which were issued for this policy will no longer return any results for any of the end-points which form the Consumer API.

5 - User Data Access

APIs for creating and managing user’s data access

Data access is a main component of the BitBroker system. You will find more details about data access, within the key concepts section of this documentation.

Accesses are always created within the context of a user and a policy. These can be created and manipulated using other parts of this API, described elsewhere in this documentation.

In order to use the sample calls in this section, you must create first the user and the policy, as outlined in the previous sections.

Creating a New Access

New accesses can be created by issuing an HTTP/POST to the /user/:uid/access/:pid end-point.

Accesses are always created within the context of a user and a policy. Hence, you must know the user ID (uid) and the policy ID (pid) in order to create one.

curl http://bbk-coordinator:8001/v1/user/2/access/over-a-billion \
     --request POST \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 201 Created
Location: http://bbk-coordinator:8001/v1/policy/country

The body of this response will contain the authorization token, which the user should utilize to authorize their calls to the Consumer API. For example:

7b4cb14a-7227-4be2-80a8-4550b841b4f4.2edbb472-f278-4a0d-91db-9d21ada03c77.037d5a90-a26b-4b9f-ab57-57d4309d487c

It is expected that coordinator user will securely distribute this consumer authorization token to the user.

Reissuing an Access

Existing accesses can be reissued by issuing an HTTP/PUT to the /user/:uid/access/:pid end-point.

Accesses are always created within the context of a user and a policy. Hence, you must know the user ID (uid) and the policy ID (pid) in order to reissue one.

curl http://bbk-coordinator:8001/v1/user/2/access/over-a-billion \
     --request PUT \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

The body of this response will contain the new authorization token, which the user should utilize to authorize their calls to the Consumer API. For example:

2f5ddf4b-e1cd-484f-a218-fdc4b27c5c02.ed46b1eb-01b8-46de-868a-f08a99416716.4f466f48-3e9b-4aa6-a94f-e0aab49d9b94

It is expected that coordinator user will securely distribute this new consumer authorization token to the user.

Reissuing a new authorization token will invalidate the previous (now old) authorization token.

List of Accesses

You can obtain a list of all the existing accesses a user has by issuing an HTTP/GET to the /user/:uid/access end-point.

Accesses are always created within the context of a user and, hence, you must know the user ID (uid) to list their accesses.

curl http://bbk-coordinator:8001/v1/user/2/access \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

[
    {
        "id": "over-a-billion",
        "url": "http://bbk-coordinator:8001/v1/user/2/access/over-a-billion",
        "policy": {
            "id": "over-a-billion",
            "url": "http://bbk-coordinator:8001/v1/policy/over-a-billion"
        },
        "created": "2022-06-01T14:18:30.635Z"
    }
    // ... other accesses here
]

Each access the user has will be returned within this array. Note: There is currently no paging on this API.

A shorter list of accesses by user can also be obtained when you ask for the details of an individual user. For example:

curl http://bbk-coordinator:8001/v1/user/2 \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON array as follows:

{
    "id": 2,
    "url": "http://bbk-coordinator:8001/v1/user/2",
    "name": "Alice",
    "email": "alice@domain.com",
    "coordinator": false,
    "accesses": [
        {
            "id": "over-a-billion",
            "url": "http://bbk-coordinator:8001/v1/user/2/access/over-a-billion"
        }
        // ... other accesses here
    ],
    "addendum": {}
}

Details of an Existing Access

You can obtain the details of an existing access by issuing an HTTP/GET to the /user/:uid/access/:pid end-point.

Accesses are always created within the context of a user and a policy. Hence, you must know the user ID (uid) and the policy ID (pid) to get its details.

curl http://bbk-coordinator:8001/v1/user/2/access/over-a-billion \
     --header "x-bbk-auth-token: your-token-goes-here"

This will return a JSON object as follows:

{
    "id": "over-a-billion",
    "url": "http://bbk-coordinator:8001/v1/user/2/access/over-a-billion",
    "policy": {
        "id": "over-a-billion",
        "url": "http://bbk-coordinator:8001/v1/policy/over-a-billion"
    },
    "created": "2022-06-01T14:18:30.635Z"
}

Consumer authorization tokens are not stored in an accessible way within the system and hence not present in this returned document.

Deleting an Access

Existing accesses can be deleted from the system by issuing an HTTP/DELETE to the /user/:uid/access/:pid end-point.

Accesses are always created within the context of a user and a policy. Hence, you must know the user ID (uid) and the policy ID (pid) to delete one.

curl http://bbk-coordinator:8001/v1/user/2/access/over-a-billion \
     --request DELETE \
     --include \
     --header "x-bbk-auth-token: your-token-goes-here"

This will result in a response as follows:

HTTP/1.1 204 No Content

Once the access is deleted, the authorization token issued with it will no longer return any results for any of the end-points which form the Consumer API.