Hacktoberfest 2022 — Connector Extravaganza!

Connectors are at the heart of OpenMetadata, bringing metadata from source systems into the centralized metadata platform.

With the Python Ingestion Framework, the OpenMetadata community has a powerful and easy way of connecting to popular systems; like Snowflake or Airbyte. Reading metadata, transforming it, and displaying it in full glory in the OpenMetadata UI. The Python API is a type-safe and gentle wrapper for the OpenMetadata backend, guiding the metadata home.

Join the team in creating new connectors in our “Connector Extravaganza” themed Hacktoberfest! Every individual who completes and submits a connector to the OpenMetadata by October 31st (midnight PST time!) will receive a $500 USD cash prize and some sweet, sweet OpenMetadata swag.

🙅 No competition between peers (cause how could we choose between some of our favourite Open Source projects and people?)
🙅 No pressure
🙅 No stress

Just some good old-fashioned open source hacking- with an added bonus of 500 bucks! 🤑

3 easy steps to start

1. Join the OpenMetadata slack community
2. Discuss your ideas for a connector with the team to stake your claim (pssst 🤫 if you want to participate but don’t have a connector in mind, come by, and we can help you choose!)
3. Start coding!

Contribution guidelines

• It should be a new connector.
• It should adequately translate the incoming metadata into the metadata standard.
• All connectors should extend their Service Source (example DatabaseServiceSource)
• The SQLAlchemy-based connectors should extend the Common Database Source class.
• The ingestion should work from the UI and CLI.
• The following elements should accompany the connector:
• JSON Schema defining the Connection class (here’s an example)
• Documentation for UI, CLI, and Airflow executions (and another)
• Testing
• Instructions on how to deploy the source system and feed sample data (e.g., a Docker container)

What Connectors can I contribute?

Any source system not currently present is welcome! Existing sources are:

• Database: Athena, AzureSQL, BigQuery, Clickhouse, Databricks, DataLake (S3, GCS), DB2, Druid, DynamoDB, Glue, Hive, MariaDB, MSSQL, MySQL, Oracle, PinotDB, Postgres, Presto, Redshift, Salesforce, SingleStore, Snowflake, SQLite, Trino, Vertica
• Messaging: Kafka, Redpanda
• Dashboard: Looker, Metabase, Mode, PowerBI, Redash, Superset, Tableau
• Pipeline: Airbyte, Airflow, Dagster, Fivetran, Glue, Nifi
• ML Model: MLflow

Reach out on Slack if you plan to contribute a connector so we can discuss if it’s a contribution that is available to develop (so that multiple people aren’t working on the same connector!!) We prepared a tutorial on how to create your own unique connector, you can watch the video on YouTube here or check out the example PR made in the video here.

Looking for an idea? Check out some options the team put together below:

• Teradata
• SiSense
• SAP
• Qlik
• SageMaker
• Microstrategy
• Pulsar
• Prefect
• Databricks Jobs
• Vertex AI
• AWS QuickSight
• Iceberg Native

Give us a ⭐ on GitHub to follow our project!