# ELK4QRadar **Project is available on** [**github** ](https://github.com/H1L021/ELK4QRADAR/) ![](https://1286158324-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MO79pt4NiZPFhlGCglR%2F-MQ6t_InUkOovca1oFHz%2F-MQ6uB2VHSPYum3uNbaY%2Fimage.png?alt=media\&token=2b81d743-fa27-4c89-bfe3-548a306073b9) ## Guide 1. `PUT _template/`. In this repository we provide an index template that you can in your Elastic Stack 2. Populate the YAML files in `/etc/logstash` with the appropriate data to your context. We Provide samples in this project: * [timezone.yml](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/timezone.yml): Contains dictionary of client name and their correspondant timezones. * [clientnames.yml](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/clientnames.yml) : Contains a dictionary of input configuration tags and their correspondant client names 3. Copy conf.d configuration in your Logstash conf.d folder and customize to your needs. * [input sample](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/conf.d/0001-input-sample.conf) * [filter sample](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/conf.d/0020-filter.conf) * [output sample](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/conf.d/0030-output.conf) 4. Create a `/home/USER/Offenses/` folder to save the extracted search data from QRadar in CSV. 5. Create the following scripted fields in Kibana | Name | Lang | Script | Format | | --------------------- | -------- | --------------------------------------- | ------ | | offense.day\_of\_week | painless | `doc['@timestamp'].value.dayOfWeekEnum` | String | | offense.hour\_of\_day | painless | `doc['@timestamp'].value.hourOfDay` | Number | ### Metrics samples * **Busiest Day** ![](https://1286158324-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MO79pt4NiZPFhlGCglR%2F-MQ6t_InUkOovca1oFHz%2F-MQ6ucNDESm7g_S0nGXP%2Fimage.png?alt=media\&token=4b538962-590b-4ec4-bc19-feb357bd8e54) * **Busiest Hour** ![](https://1286158324-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MO79pt4NiZPFhlGCglR%2F-MQ6t_InUkOovca1oFHz%2F-MQ6ug4PLnhVWXGlw6VC%2Fimage.png?alt=media\&token=14fcbc4c-3ae6-465d-924c-4ec6ea653039) * **Offenses average by day of week** ![](https://1286158324-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MO79pt4NiZPFhlGCglR%2F-MQ6t_InUkOovca1oFHz%2F-MQ6umTcrbvtbo7_IUqJ%2Fimage.png?alt=media\&token=ae86cea1-0506-45c5-95e8-3eebeebc6fa8) ## Index Template I created a custom template for this use case: ```javascript { "index_patterns": [ "soc-statistics-offenses-*" ], "template": { "settings": { "number_of_shards": 1, "number_of_replicas": 0 }, "mappings": { "dynamic_templates": [ { "strings_as_keyword": { "mapping": { "ignore_above": 1024, "type": "keyword" }, "match_mapping_type": "string" } } ], "properties": { "@timestamp": { "type": "date" }, "offense": { "properties": { "owner": { "type": "keyword" }, "note": { "properties": { "date": { "format": "MMM d, yyyy, h:mm:ss a", "type": "date" } } }, "storagetime": { "format": "yyyy-MM-dd hh:mm:ss a", "type": "date" }, "assigned": { "properties": { "date": { "format": "MMM d, yyyy, h:mm:ss a", "type": "date" } } }, "id": { "type": "keyword" }, "starttime": { "format": "yyyy-MM-dd hh:mm:ss a", "type": "date" }, "logsourcetime": { "format": "yyyy-MM-dd hh:mm:ss a", "type": "date" }, "close": { "properties": { "date": { "format": "MMM d, yyyy, h:mm:ss a", "type": "date" }, "reason": { "type": "text" }, "analyst": { "type": "keyword" } } }, "hour_of_day": { "type": "keyword" }, "status": { "type": "keyword" }, "day_of_week": { "type": "keyword" } } }, "domain": { "properties": { "name": { "type": "keyword" } } }, "rule": { "properties": { "severity": { "type": "keyword" }, "risk_score": { "type": "keyword" }, "name": { "type": "keyword" }, "threat": { "properties": { "technique": { "properties": { "name": { "type": "keyword" }, "id": { "type": "keyword" } } }, "tactic": { "properties": { "name": { "type": "keyword" }, "id": { "type": "keyword" } } } } }, "category": { "type": "keyword" }, "class": { "type": "keyword" } } }, "client": { "properties": { "name": { "type": "keyword" } } }, "analyst": { "type": "nested", "properties": { "notes": { "type": "text" }, "username": { "type": "keyword" } } }, "event": { "properties": { "timezone": { "type": "keyword" }, "name": { "type": "keyword" } } }, "tags": { "type": "keyword" } } } } } ``` ## Logstash Configuration Files This part of the project contains logstash configuration files that will process and parse files CSV files saved by the python script in `/home/elk/Offenses` notice here that I am storing my AQL search results in Offenses folder at `elk` user's home folder. > PS : Please see the index template definition to have basic understanding of the defined fields used in this project. Logstash pipelines ar organized in three parts : * **Input configurations** : Make an input configuration for each file you wanna ingest into elasticsearch. > Example : > > ``` > input { > file { > path => "/home///.csv" > start_position => beginning > tags => "" > type => "OFFENSES" > } > } > ``` * **Filter configuration** : For processing and enriching the incoming data and normalizing event fields. * **Output configuration** : Used for sending data to Elasticsearch. > Example : > > ``` > output { > if [type] == "OFFENSES" { > elasticsearch { > hosts => ["https://localhost:9200"] > index => "soc-statistics-offenses-%{[client][name]}-%{+yyyy.MM}" > #manage_template => false > cacert => "/etc/logstash/root-ca.pem" > user => "" > password => "" > ssl => true > ssl_certificate_verification => false > } > } > } > ```