Export historical data from Splunk
This article describes how to export your historical data from Splunk. After you complete the steps in this article, you can select a target platform to host the exported data, and then select an ingestion tool to migrate the data.
You can export data from Splunk in several ways. Your selection of an export method depends on the data volumes involved and your level of interactivity. For example, exporting a single, on-demand search via Splunk Web might be appropriate for a low-volume export. Alternatively, if you want to set up a higher-volume, scheduled export, the SDK and REST options work best.
For large exports, the most stable method for data retrieval is dump
or the Command Line Interface (CLI). You can export the logs to a local folder on the Splunk server or to another server accessible by Splunk.
To export your historical data from Splunk, use one of the Splunk export methods. The output format should be CSV.
This CLI example searches for events from the _internal
index that occur during the time window that the search string specifies. The example then specifies to output the events in a CSV format to the data.csv file.You can export a maximum of 100 events by default. To increase this number, set the -maxout
argument. For example, if you set -maxout
to 0
, you can export an unlimited number of events.
This CLI command exports data recorded between 23:59 and 01:00 on September 14, 2021 to a CSV file:
splunk search "index=_internal earliest=09/14/2021:23:59:00 latest=09/16/2021:01:00:00 " -output csv > c:/data.csv
This dump
command exports all events from the bigdata
index to the YYYYmmdd/HH/host
location under the $SPLUNK_HOME/var/run/splunk/dispatch/<sid>/dump/
directory on a local disk. The command uses MyExport
as the prefix for export filenames, and outputs the results to a CSV file. The command partitions the exported data using the eval
function before the dump
command.
index=bigdata | eval _dstpath=strftime(_time, "%Y%m%d/%H") + "/" + host | dump basefilename=MyExport format=csv