Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,018 questions
How can you import data into the Workflow Orchestration Manager Database?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 19%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
aeteq
0
Reputation points
We have been using Airflow on premise for a long time and would like to migrate to WOM without losing our execution history. Is there a way to import data into the Airflow Meta-DB?
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga 5,590 Reputation points Microsoft Vendor2024-07-03T13:09:41.53+00:00 Hi @aeteq
Welcome to Microsoft Q&A platform and thanks for posting your question here.To migrate your Airflow metadata database to a new environment like while keeping your execution history, start by making sure you have a full backup of your current Meta-DB. This is crucial to avoid losing any data during the migration.
Next, use a tool like SQLAlchemy to export the data from your existing Meta-DB by running SQL queries to extract the necessary tables and data.
Once you have the data exported, set up your new WOM environment to use a compatible database, such as PostgreSQL or MySQL, ensuring the schema matches your existing Airflow Meta-DB schema. Then, import the exported data into the new Meta-DB using SQL Alchemy or other database import tools, making sure all tables and relationships are correctly imported. After importing the data, check that all execution history and metadata are intact by looking for any discrepancies or missing data.
Finally, update your Airflow configuration to point to the new Meta-DB by changing the database connection settings in your
airflow.cfgfile. Run a few test DAGs to ensure the new setup is working correctly and that the execution history is accessible.Reference
https://www.astronomer.io/docs/learn/airflow-databaseI hope this information helps you. Let me know if you have any further questions or concerns.
-
Harishga 5,590 Reputation points Microsoft Vendor
2024-07-04T08:58:16.1833333+00:00 Hi @aeteq
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help. -
Harishga 5,590 Reputation points Microsoft Vendor
2024-07-05T06:49:22.5466667+00:00 Hi @aeteq
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help. -
aeteq 0 Reputation points
2024-07-05T11:24:13.59+00:00 set up your new WOM environment to use a compatible database, such as PostgreSQL or MySQL
How do I change the database used by WOM? Can you please not just copy answers out of Copilot? This makes no sense. There's no way to use SQLAlchemy in WOM either. And why are you referencing the docs of a competing product (Astronomer)?
Sign in to comment