The Pulsar Beat output plugin is a lightweight data shipper, written in Go, that you install on your servers to capture all sorts of operational data.
This section describes how to install the Pulsar Beat output plugin.
Download the Pulsar Beat output plugin from here.
This section gives an example about how to use the Pulsar Beat output to capture information about the Filebeat.
Install the Docker. For details about how to install the Docker, see here.
Build the Pulsar Beat images.
docker build -t pulsar-beat.
Create a network.
docker network create pulsar-beat
Start the Pulsar service.
docker run -d -it --network pulsar-beat -p 6650:6650 -p 8080:8080 -v $PWD/data:/pulsar/data --name pulsar-beat-standalone apachepulsar/pulsar:2.4.0 bin/pulsar standalone
Add the following configuration to filebeat.yml
.
output.pulsar:
url: "pulsar://pulsar-beat-standalone:6650"
topic: my_topic
name: test123
Start the Filebeat.
docker pull golang:1.12.4
docker run -it --network pulsar-beat --name filebeat golang:1.12.4 /bin/bash
mkdir -p $GOPATH/src/github.com/streamnative/
cd $GOPATH/src/github.com/streamnative/
git clone https://github.com/streamnative/pulsar-beat-output
cd pulsar-beat-output
go build -o filebeat main.go
chown -R root:root filebeat.yml test_module/modules.d/system.yml test_module/module/system
cp test_module/module/system/auth/test/test.log /var/log/messages.log
cp filebeat filebeat.yml test_module
cd test_module
./filebeat modules enable system
./filebeat -c filebeat.yml -e
Open another terminal window, execute the following commands to collect information from the Filebeat.
docker cp pulsar-client.py pulsar-beat-standalone:/pulsar
docker exec -it pulsar-beat-standalone /bin/bash
python pulsar-client.py
This section lists how to troubleshoot some problems that you may encounter, when you use the Pulsar Beat output plugin.
To troubleshoot problems related to building Packetbeat, see here.
If you get an error systemd/sd-journal.h: No such file or directory
, use the following commands to troubleshoot the problem.
apt-get update
apt-get install libsystemd-dev
If you get an error vendor/github.com/elastic/beats/x-pack/auditbeat/module/system/package/rpm_linux.go:23:24: fatal error: rpm/rpmlib.h: No such file or directory
, use the following command to troubleshoot the problem.
aapt-get install librpm-dev
If you get an error Exiting: error loading config file: config file ("filebeat.yml") must be owned by the user identifier (uid=0) or root
, use the following command to troubleshoot the problem.
chown -R root:root filebeat.yml