How to Create your Own Elastic XDR Solution

Elastic Agent is back and better than ever! Let's dive straight into creating our own Elastic Stack and empowering our Enterprise with an XDR solution!!

Installing Elasticsearch

For my Elastic Endpoint Detection and Response setup, I’m going to use an Ubuntu Server 20.04 virtual machine running on Proxmox. This server will run Elasticsearch and Kibana.

SSH into your ELK server and install transport-https

sudo apt-get install curl apt-transport-https

Next add the Elastic repositories to your source list. (You will need to sudo su into root for this command)

sudo su

curl -s https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-8.x.list

exit

Then update.

sudo apt-get update

Now install Elasticsearch (Note: I had a better experience setting this up using sudo rather than running these commands as root)

sudo apt-get install elasticsearch

After elasticsearch is installed you will be presented with “Security Auto Configuration Info”. Ensure you save this info.

Next, we need to make a couple of changes to the configuration file. This file is in /etc/elasticsearch/elasticsearch.yml In order to access this file, you need root privileges.

First, we need to change the network.host and uncomment http.port value. Its default is set to the localhost and http.port is set to 9200. Change it to the IP address of the host you installed Elasticsearch onto.

network.host: <elasticsearch_ip>

http.port : 9200

After you have adjusted your IP and Port we can check the elasticsearch service and status to ensure we made the right changes.

sudo service elasticsearch start

sudo service elasticsearch status

Installing Kibana

Run the command below to install Kibana.

sudo apt-get install kibana

Once the installation is complete, edit your /etc/kibana/kibana.yml and specify the IP address and port hosting Kibana.

sudo nano /etc/kibana/kibana.yml

server.port: 5601

server.host: "Your_IP”

In the same file, also specify the IP address of your Elastic Instance.

elasticsearch.hosts: ["https://Your_ElasiticSearch_IP:9200"]

Do not start Kibana yet because we have a few more lines of code we need to amend to the kibana.yml file to utilize the full force of X-Pack security.

Configuring X-Pack Security

What is X-pack?

X-Pack is an Elastic Stack extension that provides security, alerting, monitoring, reporting, machine learning, and many other capabilities. By default, when you install Elasticsearch, X-Pack is installed. However, there are a few additional things we need to create and change to enable the full force of Elastics X-Pack Power!!!

One key component that is required is to configure SSL connections between Elastic and Kibana. If the stack were split, then we would refer to Elastic and Kibana as “nodes”. In a robust Enterprise environment, we may install Kibana on one instance and Elastic on another for more efficiency. Configuring SSL will allow the services to communicate securely over the wire. However, even if it’s an all-in-one build we still need to configure SSL to use X-Pack features.

On the host with Elasticsearch installed, we need to create a YAML file called instances.yml in, /usr/share/elasticsearch/instances.yml which is going to contain the different nodes/instances that we want to secure with SSL. In my case, I only have Elasticsearch, Kibana.

sudo nano /usr/share/elasticsearch/instances.yml

instances:

- name: "elasticsearch"

ip:

- "your ELK IP"

- name: "kibana"

ip:

- "your ELK IP"

Next, we are going to use Elastic’s certgen tool to generate certificates for our instances. This will also generate a Certificate Authority as well.

If you have an enterprise CA that your endpoints use, then specify that. In this case you can generate a self-signed cert.

sudo /usr/share/elasticsearch/bin/elasticsearch-certgen cert ca --in instances.yml --out certs.zip

This will create a .crt and .key file for each of our instances, and a ca.crt file as well.

You can unzip the different certificates with unzip.

sudo apt-get install unzip

sudo unzip /usr/share/elasticsearch/certs.zip -d /usr/share/elasticsearch/

Now we have our certificates, we can configure each instance.

Configuring Elasticsearch SSL

Create a folder to store our certificates in on our Elasticsearch host.

Note: Before Elastic 8.0.0 you would need to manually create certs for both Elastic and Kibana to use SSL or X-Pack. When installing Elastic 8.X certs are automatically created and stored in “/etc/elasticsearch/certs” folder as seen below. However, to secure Kibana web connections you will still need to create certs for the Kibana service.

Minimal configuration is needed for the elasticsearch.yml file located in “/etc/elasticsearch/” folder. Ensure your YAML file closely resembles the one below before starting the elasticsearch service.

You may reference elasticsearch YAML file below if need be.

sudo nano /etc/elasticsearch/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================

node.name: elk-edr

path.data: /var/lib/elasticsearch

path.logs: /var/log/elasticsearch

network.host: 192.168.1.233

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents

xpack.security.http.ssl:

enabled: true

keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes

xpack.security.transport.ssl:

enabled: true

verification_mode: certificate

keystore.path: certs/transport.p12

truststore.path: certs/transport.p12

# nodes

cluster.initial_master_nodes: ["elk-edr"]

# Allow HTTP API connections from localhost and local networks

# Connections are encrypted and require user authentication

http.host: [_local_, _site_]

Now restart Elasticsearch if you made any changes.

sudo service elasticsearch restart

sudo service elasticsearch status

Configuring Kibana SSL

Now we are going to repeat the process but this time it’ll be for Kibana. The configuration is slightly different for Kibana so please read carefully.

Move your certificates to the correct folder and set the correct permissions.

Also, for proper elastic REST authentication between Kibana and Elastic you will need to retrieve the “Kibana” password and amend your kibana.yml file with. You can retrieve the password by resting the Kibana account and issuing the commands below.

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana

Copy and save it for the kibana.yml file in the next steps.

# Before moving forward ensure you’re in /usr/share/elasticsearch

sudo mkdir /etc/kibana/certs/ca -p

sudo cp kibana/kibana.crt /etc/kibana/certs

sudo cp kibana/kibana.key /etc/kibana/certs

sudo chown -R kibana: /etc/kibana/certs

sudo chmod -R 770 /etc/kibana/certs

sudo cp /etc/elasticsearch/certs/http_ca.crt /etc/kibana/certs/

sudo chown -R kibana: /etc/kibana/certs/http_ca.crt

sudo chmod -R 770 /etc/kibana/certs/http_ca.crt

Next, in your file /etc/kibana/kibana.yml add the settings for SSL between Elasticsearch and Kibana as well as the password you retrieved for Kibana.

sudo nano /etc/kibana/kibana.yml

#

# =================== System: Kibana Server ===================

# Kibana is served by a back-end server. This setting specifies the port to use.

server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.

server.host: 192.168.1.233

# =================== System: Logging ===================

# Enables you to specify a file where Kibana stores log output.

logging:

appenders:

file:

type: file

fileName: /var/log/kibana/kibana.log

layout:

type: json

root:

appenders:

- default

- file

# layout:

# type: json

# Specifies the path where Kibana creates the process ID file.

pid.file: /run/kibana/kibana.pid

# The URLs of the Elasticsearch instances to use for all your queries.

elasticsearch.hosts: ["https://192.168.1.233:9200"]

elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/http_ca.crt"]

# These settings enable SSL for outgoing requests from the Kibana server to the browser.

server.ssl.enabled: true

server.ssl.certificate: "/etc/kibana/certs/kibana.crt"

server.ssl.key: "/etc/kibana/certs/kibana.key"

elasticsearch.username: "kibana"

elasticsearch.password: "Dt2BTokT*2+HgnN5rJLc"

# X-Pack Key

xpack.security.encryptionKey: "something_at_least_32_characters"

xpack.encryptedSavedObjects.encryptionKey: "something_at_least_32_characters"

The Kibana.yml file should look like one below.

Then, restart Kibana.

sudo service kibana restart

Before logging in let’s create a user and give the account superuser rights.

sudo /usr/share/elasticsearch/bin/elasticsearch-users useradd username -p password -r superuser

sudo /usr/share/elasticsearch/bin/elasticsearch-users list

Open your browser and go to https://X.X.X.X:5601

Use your newly created username and password to log in for the first time.

Fleet Configuration and Installation

You should now be able to view the built-in detection rules and create your own. But first lets setup the fleet server and begin the integration process. Navigate to the left-hand menu and select fleet. Here you will see options to begin setting up the fleet server so that you can manage your endpoints. First, let us make some IP changes in the settings portion.

Ensure you edit and add your elastic IP over HTTPS as seen below for both Fleet Server Hosts and Outputs for Elasticsearch.

Next, let us begin the fleet enrollment process. Click on Add Agent, select Enroll in Fleet, and Add Fleet Server.

You will need to give fleet server your elastic IP since everything is hosted on the same box as seen below. This should have been taken care of since we made changes in settings.

Click generate fleet server policy and you’ll be presented with instructions on how to download elastic agent and apply fleet server policy as seen below.

Use the syntax with the enrollment token to install the fleet server using the Linux based Elastic-Agent.

curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.4.3-linux-x86_64.tar.gz

tar xzvf elastic-agent-8.4.3-linux-x86_64.tar.gz

cd elastic-agent-8.4.3-linux-x86_64

sudo ./elastic-agent install \

--fleet-server-es=https://192.168.1.240:9200 \

--fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NjcwNDc3MDQ1NjQ6UFd6VXF1VEVTU3VWV1BZQU5Sejc0Zw \

--fleet-server-policy=fleet-server-policy \

--fleet-server-es-insecure

If all is well, you will have successfully started the fleet server on your Elastic instance as seen below.

Deploy EDR to Endpoints

First, select Endpoints in the left-hand menu and click “Add Endpoint Security”

Here you will create security integration package by clicking Endpoint Security.

Click “Add Endpoint Security”

Name your policy and save it.

Click “Add”

Copy the Windows install syntax

Open PowerShell session as Admin or workstation admin. Ensure you add -- insecure to end of the syntax so windows will ignore certificate error.

Success! Elastic installed and created a new folder under Program Files! You can see it checked in via Fleet as well. Perfect!!!

Although the endpoint is communicating with fleet, we still need to import the “http_ca.crt” into the local cert store for it to talk with Elastic over port 9200.

Copy http_ca.crt to home folder and chown it with proper low level user priv’s. Then SCP the cert to windows endpoint. Finally, import cert into machines cert store.

Finally, let’s configure our new elastic agent on windows endpoint to act as our primary Anti-Virus XDR solution.

Click Fleet
Click Windows Security

Click three dots under Actions
Click Edit integration

Scroll all the way down and click “Register as Antivirus” and click Save integration.

Finally, click Save and deploy changes

Once this is complete Elastic Agent will act as primary Antivirus.

Cyber Threat Emulation

Elastic EDR 8.X

VS.

PowerShell Empire 5.0

Elastic offers a free open source EDR that prevents a wide variety of malware from being executed on endpoints. Aside from protection/detection Elastic Agent captures endpoint network connections and allows for additional package integration.

ALL FOR FREE!!!

Below are screenshots that will demo the power of free ELK EDR. PowerShell Empire 4.3.3 will be the C2

Star Killer GUI for Empire (Malicious Payloads)