Integrating Elastic Stack (ELK Stack) with WSO2 IS via HTTP

WSO2 Identity Server (IS) provides various ways to access their events through their event publishers. Apart from the existing WSO2 IS Analytics Solution, users can create their own analytics using those event publishers offered by WSO2 IS. I am going to guide you through the process of configuring WSO2 IS to publish their events to a locally installed Elastic Stack via HTTP.

Prerequisites

• Elastic Stack installation. (Logstash, ElasticSearch, Kibana): Guide Here
• Locally running WSO2 IS instance: Guide Here

Objectives

• Install “http-input-plugin” for Logstash.
• Configure Logstash to receive input via HTTP.
• Configure the WSO2 IS to publish events to Logstash via HTTP.
• Run and see it in action.

Install “http-input-plugin” for Logstash

The plugin can be installed easily from the Logstash itself. You just have to navigate to the “bin” folder of the Logstash installation directory and run the following command,

logstash-plugin install http-input-plugin

Visit here for more information on ‘logstash-plugin’ commands and usage.

Configure Logstash to receive input via HTTP

Logstash input method must be changed as follows after installing the http-input-plugin. This is the minimal config file of the Logstash with input and output only. The default 8080 port is changed to 8084 in order to avoid other services from conflicting with our Logstash service.

input {
    http {
        host => "127.0.0.1" # default: 0.0.0.0
        port => 8084 # default: 8080
    }
}
output {
    elasticsearch {
        hosts => ["localhost:9200"] # default 9200
    }
}

Configure the WSO2 IS to publish events to Logstash via HTTP

Next thing that needs to be done is the configuration of WSO2 IS to send the events to our Logstash service via HTTP.

IS configuration change for Authentication events can be be done as follows. Change the “IsAnalytics-Publisher-wso2event-AuthenticationData.xml” file as follows to enable HTTP event publishing.

<?xml version="1.0" encoding="UTF-8"?>
<eventPublisher
  name="IsAnalytics-Publisher-wso2event-AuthenticationData"
  statistics="disable" trace="disable" xmlns="http://wso2.org/carbon/eventpublisher">
  <from streamName="org.wso2.is.analytics.stream.OverallAuthentication" version="1.0.0"/>
  <mapping customMapping="disable" type="json"/>
  <to eventAdapterType="http">
 <property name="publishingMode">non-blocking</property>
 <property name="publishTimeout">0</property>
 <property name="http.url">http://localhost:8084</property>
 
  </to>
</eventPublisher>

Same steps can be followed with other event publishers as well. You can check out the event publishers here. Read this to gather knowledge on the level of security provided by the http-input-plugin.

Run and see it in action

Run the Elastic Stack and WSO2 IS and make sure all the services started without throwing errors. Now go to Kibana dashboard (http://localhost:5601/app/kibana) and create an index for ElasticSearch. After that, go to the user dashboard of WSO2 IS and log in. Then head back to Kibana dashboard and you will see the HTTP event of the login will be captured and displayed there.

If you have trouble understanding the basics of Elastic Stack, please follow this tutorial: https://logz.io/learn/complete-guide-elk-stack/#intro. It will guide you through the basic installation with the configuration of Elastic Stack and creating an index in Kibana Dashboard as well.