# ELK4QRadar

**Project is available on** [**github** ](https://github.com/H1L021/ELK4QRADAR/)

![](/files/-MQ6uB2VHSPYum3uNbaY)

## Guide

1. `PUT _template/<YOUR_TEMPLATE_NAME>`. In this repository we provide an index template that you can in your Elastic Stack
2. Populate the YAML files in `/etc/logstash` with the appropriate data to your context. We Provide samples in this project:
   * [timezone.yml](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/timezone.yml): Contains dictionary of client name and their correspondant timezones.
   * [clientnames.yml](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/clientnames.yml) : Contains a dictionary of input configuration tags and their correspondant client names
3. Copy conf.d configuration in your Logstash conf.d folder and customize to your needs.
   * [input sample](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/conf.d/0001-input-sample.conf)
   * [filter sample](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/conf.d/0020-filter.conf)
   * [output sample](https://github.com/H1L021/ELK4QRADAR/blob/main/logstash/conf.d/0030-output.conf)
4. Create a `/home/USER/Offenses/` folder to save the extracted search data from QRadar in CSV.
5. Create the following scripted fields in Kibana

| Name                  | Lang     | Script                                  | Format |
| --------------------- | -------- | --------------------------------------- | ------ |
| offense.day\_of\_week | painless | `doc['@timestamp'].value.dayOfWeekEnum` | String |
| offense.hour\_of\_day | painless | `doc['@timestamp'].value.hourOfDay`     | Number |

### Metrics samples

* **Busiest Day**

![](/files/-MQ6ucNDESm7g_S0nGXP)

* **Busiest Hour**

![](/files/-MQ6ug4PLnhVWXGlw6VC)

* **Offenses average by day of week**

![](/files/-MQ6umTcrbvtbo7_IUqJ)

## Index Template

I created a custom template for this use case:

```javascript
{
    "index_patterns": [
        "soc-statistics-offenses-*"
    ],
    "template": {
        "settings": {
            "number_of_shards": 1,
            "number_of_replicas": 0
        },
        "mappings": {
            "dynamic_templates": [
                {
                    "strings_as_keyword": {
                        "mapping": {
                            "ignore_above": 1024,
                            "type": "keyword"
                        },
                        "match_mapping_type": "string"
                    }
                }
            ],
            "properties": {
                "@timestamp": {
                    "type": "date"
                },
                "offense": {
                    "properties": {
                        "owner": {
                            "type": "keyword"
                        },
                        "note": {
                            "properties": {
                                "date": {
                                    "format": "MMM d, yyyy, h:mm:ss a",
                                    "type": "date"
                                }
                            }
                        },
                        "storagetime": {
                            "format": "yyyy-MM-dd hh:mm:ss a",
                            "type": "date"
                        },
                        "assigned": {
                            "properties": {
                                "date": {
                                    "format": "MMM d, yyyy, h:mm:ss a",
                                    "type": "date"
                                }
                            }
                        },
                        "id": {
                            "type": "keyword"
                        },
                        "starttime": {
                            "format": "yyyy-MM-dd hh:mm:ss a",
                            "type": "date"
                        },
                        "logsourcetime": {
                            "format": "yyyy-MM-dd hh:mm:ss a",
                            "type": "date"
                        },
                        "close": {
                            "properties": {
                                "date": {
                                    "format": "MMM d, yyyy, h:mm:ss a",
                                    "type": "date"
                                },
                                "reason": {
                                    "type": "text"
                                },
                                "analyst": {
                                    "type": "keyword"
                                }
                            }
                        },
                        "hour_of_day": {
                            "type": "keyword"
                        },
                        "status": {
                            "type": "keyword"
                        },
                        "day_of_week": {
                            "type": "keyword"
                        }
                    }
                },
                "domain": {
                    "properties": {
                        "name": {
                            "type": "keyword"
                        }
                    }
                },
                "rule": {
                    "properties": {
                        "severity": {
                            "type": "keyword"
                        },
                        "risk_score": {
                            "type": "keyword"
                        },
                        "name": {
                            "type": "keyword"
                        },
                        "threat": {
                            "properties": {
                                "technique": {
                                    "properties": {
                                        "name": {
                                            "type": "keyword"
                                        },
                                        "id": {
                                            "type": "keyword"
                                        }
                                    }
                                },
                                "tactic": {
                                    "properties": {
                                        "name": {
                                            "type": "keyword"
                                        },
                                        "id": {
                                            "type": "keyword"
                                        }
                                    }
                                }
                            }
                        },
                        "category": {
                            "type": "keyword"
                        },
                        "class": {
                            "type": "keyword"
                        }
                    }
                },
                "client": {
                    "properties": {
                        "name": {
                            "type": "keyword"
                        }
                    }
                },
                "analyst": {
                    "type": "nested",
                    "properties": {
                        "notes": {
                            "type": "text"
                        },
                        "username": {
                            "type": "keyword"
                        }
                    }
                },
                "event": {
                    "properties": {
                        "timezone": {
                            "type": "keyword"
                        },
                        "name": {
                            "type": "keyword"
                        }
                    }
                },
                "tags": {
                    "type": "keyword"
                }
            }
        }
    }
}
```

## Logstash Configuration Files

This part of the project contains logstash configuration files that will process and parse files CSV files saved by the python script in `/home/elk/Offenses` notice here that I am storing my AQL search results in Offenses folder at `elk` user's home folder.

> PS : Please see the index template definition to have basic understanding of the defined fields used in this project.

Logstash pipelines ar organized in three parts :

* **Input configurations** : Make an input configuration for each file you wanna ingest into elasticsearch.

> Example :
>
> ```
> input {
>    file {
>            path => "/home/<USER>/<FOLDER NAME>/<FILENAME>.csv"
>            start_position => beginning
>            tags => "<MY_CLIENT>"
>            type => "OFFENSES"
>        }
> }
> ```

* **Filter configuration** : For processing and enriching the incoming data and normalizing event fields.
* **Output configuration** : Used for sending data to Elasticsearch.

> Example :
>
> ```
> output {
>    if [type] == "OFFENSES" {
>        elasticsearch {
>            hosts => ["https://localhost:9200"]
>            index => "soc-statistics-offenses-%{[client][name]}-%{+yyyy.MM}"
>            #manage_template => false
>            cacert => "/etc/logstash/root-ca.pem"
>            user => "<USERNAME>"
>            password => "<PASSWORD>"
>            ssl => true
>          ssl_certificate_verification => false
>        }
>    }
> }
> ```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://www.unh4ck.com/github-projects/elk4qradar.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.