LDAP
This plugin extracts the following:
- People
- Names, emails, titles, and manager information for each person
- List of groups
CLI based Ingestion
Install the Plugin
pip install 'acryl-datahub[ldap]'
Starter Recipe
Check out the following recipe to get started with ingestion! See below for full configuration options.
For general pointers on writing and running a recipe, see our main recipe guide.
source:
type: "ldap"
config:
# Coordinates
ldap_server: ldap://localhost
# Credentials
ldap_user: "cn=admin,dc=example,dc=org"
ldap_password: "admin"
# Options
base_dn: "dc=example,dc=org"
sink:
# sink configs
Config Details
- Options
- Schema
Note that a .
is used to denote nested fields in the YAML recipe.
Field | Description |
---|---|
base_dn ✅ string | LDAP DN. |
ldap_password ✅ string | LDAP password. |
ldap_server ✅ string | LDAP server URL. |
ldap_user ✅ string | LDAP user. |
attrs_list array(string) | |
custom_props_list array(string) | |
drop_missing_first_last_name boolean | If set to true, any users without first and last names will be dropped. Default: True |
filter string | LDAP extractor filter. Default: (objectClass=*) |
group_attrs_map object | Default: {} |
manager_filter_enabled boolean | Use LDAP extractor filter to search managers. Default: True |
manager_pagination_enabled boolean | Use pagination while search for managers (enabled by default). Default: True |
page_size integer | Size of each page to fetch when extracting metadata. Default: 20 |
platform_instance string | The instance of the platform that all assets produced by this recipe belong to |
user_attrs_map object | Default: {} |
env string | The environment that all assets produced by this connector belong to Default: PROD |
stateful_ingestion StatefulStaleMetadataRemovalConfig | Base specialized config for Stateful Ingestion with stale metadata removal capability. |
stateful_ingestion.enabled boolean | The type of the ingestion state provider registered with datahub. Default: False |
stateful_ingestion.ignore_new_state boolean | If set to True, ignores the current checkpoint state. Default: False |
stateful_ingestion.ignore_old_state boolean | If set to True, ignores the previous checkpoint state. Default: False |
stateful_ingestion.remove_stale_metadata boolean | Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled. Default: True |
The JSONSchema for this configuration is inlined below.
{
"title": "LDAPSourceConfig",
"description": "Config used by the LDAP Source.",
"type": "object",
"properties": {
"env": {
"title": "Env",
"description": "The environment that all assets produced by this connector belong to",
"default": "PROD",
"type": "string"
},
"platform_instance": {
"title": "Platform Instance",
"description": "The instance of the platform that all assets produced by this recipe belong to",
"type": "string"
},
"stateful_ingestion": {
"$ref": "#/definitions/StatefulStaleMetadataRemovalConfig"
},
"ldap_server": {
"title": "Ldap Server",
"description": "LDAP server URL.",
"type": "string"
},
"ldap_user": {
"title": "Ldap User",
"description": "LDAP user.",
"type": "string"
},
"ldap_password": {
"title": "Ldap Password",
"description": "LDAP password.",
"type": "string"
},
"base_dn": {
"title": "Base Dn",
"description": "LDAP DN.",
"type": "string"
},
"filter": {
"title": "Filter",
"description": "LDAP extractor filter.",
"default": "(objectClass=*)",
"type": "string"
},
"attrs_list": {
"title": "Attrs List",
"description": "Retrieved attributes list",
"type": "array",
"items": {
"type": "string"
}
},
"custom_props_list": {
"title": "Custom Props List",
"description": "A list of custom attributes to extract from the LDAP provider.",
"type": "array",
"items": {
"type": "string"
}
},
"drop_missing_first_last_name": {
"title": "Drop Missing First Last Name",
"description": "If set to true, any users without first and last names will be dropped.",
"default": true,
"type": "boolean"
},
"page_size": {
"title": "Page Size",
"description": "Size of each page to fetch when extracting metadata.",
"default": 20,
"type": "integer"
},
"manager_filter_enabled": {
"title": "Manager Filter Enabled",
"description": "Use LDAP extractor filter to search managers.",
"default": true,
"type": "boolean"
},
"manager_pagination_enabled": {
"title": "Manager Pagination Enabled",
"description": "Use pagination while search for managers (enabled by default).",
"default": true,
"type": "boolean"
},
"user_attrs_map": {
"title": "User Attrs Map",
"default": {},
"type": "object"
},
"group_attrs_map": {
"title": "Group Attrs Map",
"default": {},
"type": "object"
}
},
"required": [
"ldap_server",
"ldap_user",
"ldap_password",
"base_dn"
],
"additionalProperties": false,
"definitions": {
"DynamicTypedStateProviderConfig": {
"title": "DynamicTypedStateProviderConfig",
"type": "object",
"properties": {
"type": {
"title": "Type",
"description": "The type of the state provider to use. For DataHub use `datahub`",
"type": "string"
},
"config": {
"title": "Config",
"description": "The configuration required for initializing the state provider. Default: The datahub_api config if set at pipeline level. Otherwise, the default DatahubClientConfig. See the defaults (https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/graph/client.py#L19)."
}
},
"required": [
"type"
],
"additionalProperties": false
},
"StatefulStaleMetadataRemovalConfig": {
"title": "StatefulStaleMetadataRemovalConfig",
"description": "Base specialized config for Stateful Ingestion with stale metadata removal capability.",
"type": "object",
"properties": {
"enabled": {
"title": "Enabled",
"description": "The type of the ingestion state provider registered with datahub.",
"default": false,
"type": "boolean"
},
"ignore_old_state": {
"title": "Ignore Old State",
"description": "If set to True, ignores the previous checkpoint state.",
"default": false,
"type": "boolean"
},
"ignore_new_state": {
"title": "Ignore New State",
"description": "If set to True, ignores the current checkpoint state.",
"default": false,
"type": "boolean"
},
"remove_stale_metadata": {
"title": "Remove Stale Metadata",
"description": "Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.",
"default": true,
"type": "boolean"
}
},
"additionalProperties": false
}
}
}
Code Coordinates
- Class Name:
datahub.ingestion.source.ldap.LDAPSource
- Browse on GitHub
Questions
If you've got any questions on configuring ingestion for LDAP, feel free to ping us on our Slack.