Integrating Splunk with MuleSoft

5 min readDec 21, 2021

Introduction:

Splunk is a software platform widely used for monitoring, searching, analyzing and visualizing the machine-generated data in real time. It performs capturing, indexing, and correlating the real time data in a searchable container and produces graphs, alerts, dashboards and visualizations.

Why we use Splunk:

The main reason for using a custom logging framework is that MuleSoft in Cloud hub stores data for 30 days or up to 100 MB. After 30days Cloud hub logs will deleted permanently.

Different type of Appenders:

Log4J2CloudhubLogAppender: Sends log data to Cloud Hub.

2. Rolling File: Sends log data to the filesystem of the VM.

3. Splunk HTTP: Appender sends log data to the Splunk system.

How to download the Splunk:

For installing the Splunk on-premise or on your system, you can use below link for Free download and create an account https://www.splunk.com/en_us/download/splunk-enterprise.html

Once you can create a Splunk account, it will give you the option to download Splunk Enterprise.

After Splunk is installed, you can browse to http://localhost:8000 or Home | Splunk 8.2.0 and it will open you to Splunk web page.

To Login into Splunk account, we have to provide username “admin” and password that you have created during installation.

Create HTTP Event Collector Token

For creating new tokens, Open Settings → Data inputs → HTTP Event Collector.

We can select the HTTP Event Collector, it will open new window from where we can create a New Token. Click on the New Token button.

We can click on the New Token button, it will open new form and start filling out the details like Name. Click on Next at the top of the web page for filling further details.

On the next screen, you can select Source type as log4j, as we will be using log4j as a source for sending data to Splunk from the MuleSoft application.

On the same screen, select all index main, summary, and history.

Once you click on the Submit button, it will create a token that we will be using for authentication purposes.

Here you can find new generated token, Make note of this token number.

Click on Settings → Data inputs → HTTP Event Collector to see the newly created token.

Also, make sure that token is enabled. In case it is not enabled, you can click on Global Settings and enable the token as shown in the below screenshot.

Creating a sample Mule Application:

Here, we are using HTTP Listener and Database Connector. the Transform message from the Mule Palette and drag and drop it after the database component.

Fetching details from the employee table and same logger we can able to see log’s in Splunk.

Here we added Appenders and Loggers.

Add token Id which we have created in → HTTP Event Collector

Log4j2.xml:

<?xml version=”1.0" encoding=”utf-8"?>

<! — These are some of the loggers you can enable.

There are several more you can find in the documentation.

Besides this log4j configuration, you can also use Java VM environment variables

to enable other logs like network (-Djavax.net.debug=ssl or all) and

Garbage Collector (-XX:+PrintGC). These will be append to the console, so you will

see them in the mule_ee.log file. →

<RollingFile name=”file” fileName=”${sys:mule.home}${sys:file.separator}logs${sys:file.separator}user-subscription-sapi.log”

filePattern=”${sys:mule.home}${sys:file.separator}logs${sys:file.separator}user-subscription-sapi-%i.log”>

</RollingFile>

</Http>

</Appenders>

<! — Http Logger shows wire traffic on DEBUG. →

<! — AsyncLogger name=”org.mule.service.http.impl.service.HttpMessageLogger” level=”DEBUG” / →

<! — Mule logger →

</AsyncRoot>

</Loggers>

</Configuration>

Testing:

You can send HTTP request using Postman to test the application.

Splunk Logger: Success response:

Error message In Splunk: DB CONNECTIVITY Error

Conclusion:

Splunk is a useful and powerful tool for logging, analyzing, reporting, searching, and visualizing events and data. MuleSoft provides the capability to easily integrate Splunk using Anypoint Studio.